2020-09-08T15:02:50.343 INFO:root:teuthology version: 1.0.0-55bd2dc 2020-09-08T15:02:50.346 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/kchai-2020-09-08_12:29:35-rados-wip-kefu-testing-2020-09-08-0934-distro-basic-smithi/5417828 --name kchai-2020-09-08_12:29:35-rados-wip-kefu-testing-2020-09-08-0934-distro-basic-smithi --owner scheduled_kchai@teuthology --description rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/radosbench} --block --verbose --lock -- /tmp/teuthology-worker.a2x5lpvv.tmp 2020-09-08T15:02:50.384 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-09-08T15:02:50.446 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/kchai-2020-09-08_12:29:35-rados-wip-kefu-testing-2020-09-08-0934-distro-basic-smithi/5417828 branch: wip-kefu-testing-2020-09-08-0934 description: rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/radosbench} email: tchaikov@gmail.com first_in_suite: false job_id: '5417828' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: kchai-2020-09-08_12:29:35-rados-wip-kefu-testing-2020-09-08-0934-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 30 - volumes: count: 4 size: 10 os_type: ubuntu os_version: '18.04' overrides: admin_socket: branch: wip-kefu-testing-2020-09-08-0934 ceph: conf: client.2: debug ms: 1 debug objecter: 20 debug rados: 20 global: mon client directed command retry: 5 mon warn on msgr2 not enabled: false ms inject socket failures: 2500 ms tcp read timeout: 5 ms type: async osd_pool_default_min_size: 2 osd_pool_default_size: 2 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 osd backoff on degraded: true osd backoff on peering: true osd debug reject backfill probability: 0.3 osd debug verify cached snaps: true osd debug verify missing on start: true osd max backfills: 3 osd max markdown count: 1000 osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(REQUEST_STUCK\) - \(MON_DOWN\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) mon_bind_msgr2: false sha1: dce4d291bd4243c9802c17d67b85a6da6bbc0f8f ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: dce4d291bd4243c9802c17d67b85a6da6bbc0f8f workunit: branch: wip-kefu-testing-2020-09-08-0934 sha1: dce4d291bd4243c9802c17d67b85a6da6bbc0f8f owner: scheduled_kchai@teuthology priority: 90 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - - mon.c - osd.8 - osd.9 - osd.10 - osd.11 - client.2 sha1: dce4d291bd4243c9802c17d67b85a6da6bbc0f8f sleep_before_teardown: 0 suite: rados suite_branch: wip-kefu-testing-2020-09-08-0934 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-08-0934/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: dce4d291bd4243c9802c17d67b85a6da6bbc0f8f tasks: - install: branch: mimic exclude_packages: - librados3 - ceph-mgr-dashboard - ceph-mgr-diskprediction-local - ceph-mgr-rook - ceph-mgr-cephadm - cephadm - ceph-immutable-object-cache - ceph-base - python3-rados - python3-rgw - python3-rbd - python3-cephfs - librados-devel extra_packages: - librados2 - python-rados - python-rgw - python-rbd - python-cephfs - cephadm: null - exec: mon.a: - while ! ceph balancer status ; do sleep 1 ; done - ceph balancer mode crush-compat - ceph balancer on - thrashosds: chance_pgnum_grow: 1 chance_pgpnum_fix: 1 timeout: 1200 - full_sequential: - radosbench: clients: - client.2 time: 90 - radosbench: clients: - client.2 time: 90 - radosbench: clients: - client.2 time: 90 - radosbench: clients: - client.2 time: 90 - radosbench: clients: - client.2 time: 90 - radosbench: clients: - client.2 time: 90 - radosbench: clients: - client.2 time: 90 - radosbench: clients: - client.2 time: 90 teuthology_branch: master timestamp: 2020-09-08_12:29:35 tube: smithi user: kchai verbose: false verify_ceph_hash: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.12115 2020-09-08T15:02:50.447 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-08-0934/qa; will attempt to use it 2020-09-08T15:02:50.448 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-08-0934/qa/tasks 2020-09-08T15:02:50.448 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-09-08T15:02:50.451 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-09-08T15:02:50.452 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-09-08T15:02:50.499 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:03:00.514 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-09-08T15:03:10.535 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:03:20.555 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:03:30.578 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:03:40.598 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:03:50.618 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:04:00.638 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:04:10.657 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:04:20.675 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:04:30.695 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:04:40.713 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:04:50.740 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-09-08T15:05:00.760 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:05:10.780 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:05:20.804 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:05:30.844 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:05:40.893 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:05:50.932 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:06:00.959 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:06:10.980 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:06:20.996 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:06:31.019 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:06:41.038 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:06:51.089 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:07:01.115 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:07:11.146 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:07:21.184 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:07:31.205 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:07:41.225 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:07:51.263 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:08:01.290 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:08:11.309 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:08:21.334 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-09-08T15:08:31.359 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-09-08T15:08:41.377 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:08:51.401 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:09:01.419 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:09:11.435 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:09:21.464 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-09-08T15:09:31.495 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-09-08T15:09:41.514 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-09-08T15:09:51.536 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-09-08T15:10:01.575 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-09-08T15:10:11.594 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-09-08T15:10:21.612 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-09-08T15:10:31.634 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-09-08T15:10:41.653 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-09-08T15:10:51.673 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-09-08T15:11:01.691 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-09-08T15:11:11.707 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-09-08T15:11:21.729 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-09-08T15:11:31.755 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-09-08T15:11:41.773 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-09-08T15:11:51.792 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_kchai@teuthology', 'count': 3, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/kchai-2020-09-08_12:29:35-rados-wip-kefu-testing-2020-09-08-0934-distro-basic-smithi/5417828'} 2020-09-08T15:11:51.845 DEBUG:teuthology.lock.ops:locked ubuntu@smithi104.front.sepia.ceph.com, ubuntu@smithi055.front.sepia.ceph.com, ubuntu@smithi166.front.sepia.ceph.com 2020-09-08T15:11:51.978 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4f1c98e6a8>, signals=[15]) 2020-09-08T15:11:52.200 INFO:teuthology.provision.fog.smithi104:Scheduling deploy of ubuntu 18.04 2020-09-08T15:11:52.205 INFO:teuthology.provision.fog.smithi166:Scheduling deploy of ubuntu 18.04 2020-09-08T15:11:52.210 INFO:teuthology.provision.fog.smithi055:Scheduling deploy of ubuntu 18.04 2020-09-08T15:11:52.611 INFO:teuthology.orchestra.console:Power off smithi166 2020-09-08T15:11:52.614 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi166.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-09-08T15:11:52.644 INFO:teuthology.orchestra.console:Power off smithi104 2020-09-08T15:11:52.645 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-09-08T15:11:52.667 INFO:teuthology.orchestra.console:Power off smithi055 2020-09-08T15:11:52.670 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-09-08T15:11:52.690 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-08T15:11:52.707 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi166.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-08T15:11:52.720 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-08T15:11:52.842 INFO:teuthology.orchestra.console:Power off for smithi166 completed 2020-09-08T15:11:52.945 INFO:teuthology.orchestra.console:Power on smithi166 2020-09-08T15:11:52.945 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi166.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-09-08T15:11:52.972 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi166.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-08T15:11:56.742 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-08T15:11:56.759 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-08T15:11:57.003 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi166.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-08T15:12:00.862 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-08T15:12:00.877 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-08T15:12:01.133 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi166.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-08T15:12:01.348 INFO:teuthology.orchestra.console:Power on for smithi166 completed 2020-09-08T15:12:01.449 INFO:teuthology.provision.fog.smithi166:Waiting for deploy to finish 2020-09-08T15:12:04.981 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-08T15:12:04.996 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-08T15:12:05.197 INFO:teuthology.orchestra.console:Power off for smithi104 completed 2020-09-08T15:12:05.211 INFO:teuthology.orchestra.console:Power off for smithi055 completed 2020-09-08T15:12:05.299 INFO:teuthology.orchestra.console:Power on smithi104 2020-09-08T15:12:05.299 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-09-08T15:12:05.317 INFO:teuthology.orchestra.console:Power on smithi055 2020-09-08T15:12:05.318 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-09-08T15:12:05.330 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-08T15:12:05.344 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-08T15:12:09.362 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-08T15:12:09.376 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-08T15:12:09.592 INFO:teuthology.orchestra.console:Power on for smithi055 completed 2020-09-08T15:12:09.694 INFO:teuthology.provision.fog.smithi055:Waiting for deploy to finish 2020-09-08T15:12:13.481 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-09-08T15:12:13.704 INFO:teuthology.orchestra.console:Power on for smithi104 completed 2020-09-08T15:12:13.806 INFO:teuthology.provision.fog.smithi104:Waiting for deploy to finish 2020-09-08T15:14:56.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:15:00.069 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:15:00.173 ERROR:teuthology.orchestra.connection:Error connecting to smithi104.front.sepia.ceph.com Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/connection.py", line 108, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 446, in connect passphrase, File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 764, in _auth raise saved_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 740, in _auth self._transport.auth_publickey(username, key) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 1580, in auth_publickey return self.auth_handler.wait_for_response(my_event) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/auth_handler.py", line 250, in wait_for_response raise e paramiko.ssh_exception.AuthenticationException: Authentication failed. 2020-09-08T15:15:02.784 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi166.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:15:05.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:15:11.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi166.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:15:14.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:15:20.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi166.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:15:23.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:15:29.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi166.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:15:32.305 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:15:39.055 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi166.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:15:41.360 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:15:48.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi166.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:15:57.168 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi166.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:16:07.180 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:16:18.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:16:18.908 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-09-08T15:16:24.913 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:16:24.925 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-09-08T15:16:30.927 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:16:30.938 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-09-08T15:16:36.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:16:37.030 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-09-08T15:16:37.033 INFO:teuthology.orchestra.run.smithi055:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-09-08T15:16:37.495 INFO:teuthology.provision.fog.smithi055:Node is ready 2020-09-08T15:16:37.496 INFO:teuthology.orchestra.run.smithi055:> hostname 2020-09-08T15:16:37.542 INFO:teuthology.orchestra.run.smithi055.stdout:smithi055.front.sepia.ceph.com 2020-09-08T15:16:37.543 INFO:teuthology.orchestra.run.smithi055:> grep smithi055.front.sepia.ceph.com /etc/hosts 2020-09-08T15:16:37.814 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55 smithi055.front.sepia.ceph.com smithi055 2020-09-08T15:16:37.815 INFO:teuthology.orchestra.run.smithi055:> sudo hostname smithi055 2020-09-08T15:16:37.834 INFO:teuthology.orchestra.run.smithi055:> sudo sed -i -e 's/smithi055.front.sepia.ceph.com/smithi055/g' /etc/hosts 2020-09-08T15:16:37.892 INFO:teuthology.orchestra.run.smithi055:> sudo sed -i -e 's/172.21.15.55/172.21.15.55/g' /etc/hosts 2020-09-08T15:16:38.077 INFO:teuthology.orchestra.run.smithi055:> sudo sed -i -e 's/smithi055.front.sepia.ceph.com/smithi055/g' /etc/hostname 2020-09-08T15:16:38.093 INFO:teuthology.orchestra.run.smithi055:> sudo hostname smithi055 2020-09-08T15:16:38.150 INFO:teuthology.provision.fog.smithi055:Deploy complete! 2020-09-08T15:16:38.151 DEBUG:teuthology.parallel:result is None 2020-09-08T15:17:03.172 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi166.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:17:03.248 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-09-08T15:17:03.248 INFO:teuthology.orchestra.run.smithi166:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-09-08T15:17:03.687 INFO:teuthology.provision.fog.smithi166:Node is ready 2020-09-08T15:17:03.687 INFO:teuthology.orchestra.run.smithi166:> hostname 2020-09-08T15:17:03.734 INFO:teuthology.orchestra.run.smithi166.stdout:smithi166.front.sepia.ceph.com 2020-09-08T15:17:03.735 INFO:teuthology.orchestra.run.smithi166:> grep smithi166.front.sepia.ceph.com /etc/hosts 2020-09-08T15:17:03.784 INFO:teuthology.orchestra.run.smithi166.stdout:172.21.15.166 smithi166.front.sepia.ceph.com smithi166 2020-09-08T15:17:03.785 INFO:teuthology.orchestra.run.smithi166:> sudo hostname smithi166 2020-09-08T15:17:03.855 INFO:teuthology.orchestra.run.smithi166:> sudo sed -i -e 's/smithi166.front.sepia.ceph.com/smithi166/g' /etc/hosts 2020-09-08T15:17:03.913 INFO:teuthology.orchestra.run.smithi166:> sudo sed -i -e 's/172.21.15.166/172.21.15.166/g' /etc/hosts 2020-09-08T15:17:03.969 INFO:teuthology.orchestra.run.smithi166:> sudo sed -i -e 's/smithi166.front.sepia.ceph.com/smithi166/g' /etc/hostname 2020-09-08T15:17:04.032 INFO:teuthology.orchestra.run.smithi166:> sudo hostname smithi166 2020-09-08T15:17:04.086 INFO:teuthology.provision.fog.smithi166:Deploy complete! 2020-09-08T15:17:04.087 DEBUG:teuthology.parallel:result is None 2020-09-08T15:17:13.188 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:17:13.271 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-09-08T15:17:13.272 INFO:teuthology.orchestra.run.smithi104:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-09-08T15:17:13.655 INFO:teuthology.provision.fog.smithi104:Node is ready 2020-09-08T15:17:13.656 INFO:teuthology.orchestra.run.smithi104:> hostname 2020-09-08T15:17:13.702 INFO:teuthology.orchestra.run.smithi104.stdout:smithi104.front.sepia.ceph.com 2020-09-08T15:17:13.703 INFO:teuthology.orchestra.run.smithi104:> grep smithi104.front.sepia.ceph.com /etc/hosts 2020-09-08T15:17:13.750 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.15.104 smithi104.front.sepia.ceph.com smithi104 2020-09-08T15:17:13.751 INFO:teuthology.orchestra.run.smithi104:> sudo hostname smithi104 2020-09-08T15:17:13.851 INFO:teuthology.orchestra.run.smithi104:> sudo sed -i -e 's/smithi104.front.sepia.ceph.com/smithi104/g' /etc/hosts 2020-09-08T15:17:13.867 INFO:teuthology.orchestra.run.smithi104:> sudo sed -i -e 's/172.21.15.104/172.21.15.104/g' /etc/hosts 2020-09-08T15:17:13.923 INFO:teuthology.orchestra.run.smithi104:> sudo sed -i -e 's/smithi104.front.sepia.ceph.com/smithi104/g' /etc/hostname 2020-09-08T15:17:13.980 INFO:teuthology.orchestra.run.smithi104:> sudo hostname smithi104 2020-09-08T15:17:14.036 INFO:teuthology.provision.fog.smithi104:Deploy complete! 2020-09-08T15:17:14.037 DEBUG:teuthology.parallel:result is None 2020-09-08T15:17:14.288 INFO:teuthology.lock.ops:Checking smithi104.front.sepia.ceph.com 2020-09-08T15:17:14.289 INFO:teuthology.lock.ops:New key found. Updating... 2020-09-08T15:17:14.315 INFO:teuthology.lock.ops:Checking smithi055.front.sepia.ceph.com 2020-09-08T15:17:14.316 INFO:teuthology.lock.ops:New key found. Updating... 2020-09-08T15:17:14.333 INFO:teuthology.lock.ops:Checking smithi166.front.sepia.ceph.com 2020-09-08T15:17:14.333 INFO:teuthology.lock.ops:New key found. Updating... 2020-09-08T15:17:14.350 INFO:teuthology.lock.ops:Updating [smithi104.front.sepia.ceph.com]: set os type and version on server 2020-09-08T15:17:14.351 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-09-08T15:17:14.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:17:14.452 INFO:teuthology.orchestra.run.smithi104:> true 2020-09-08T15:17:14.635 INFO:teuthology.orchestra.run.smithi104:> uname -m 2020-09-08T15:17:14.682 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2020-09-08T15:17:14.683 INFO:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2020-09-08T15:17:14.731 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="Ubuntu" 2020-09-08T15:17:14.731 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-09-08T15:17:14.732 INFO:teuthology.orchestra.run.smithi104.stdout:ID=ubuntu 2020-09-08T15:17:14.732 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE=debian 2020-09-08T15:17:14.732 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-09-08T15:17:14.733 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="18.04" 2020-09-08T15:17:14.733 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://www.ubuntu.com/" 2020-09-08T15:17:14.733 INFO:teuthology.orchestra.run.smithi104.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-09-08T15:17:14.735 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-09-08T15:17:14.735 INFO:teuthology.orchestra.run.smithi104.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-09-08T15:17:14.735 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_CODENAME=bionic 2020-09-08T15:17:14.735 INFO:teuthology.orchestra.run.smithi104.stdout:UBUNTU_CODENAME=bionic 2020-09-08T15:17:14.737 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2020-09-08T15:17:14.752 INFO:teuthology.lock.ops:Updating [smithi055.front.sepia.ceph.com]: set os type and version on server 2020-09-08T15:17:14.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-09-08T15:17:14.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:17:14.833 INFO:teuthology.orchestra.run.smithi055:> true 2020-09-08T15:17:15.071 INFO:teuthology.orchestra.run.smithi055:> uname -m 2020-09-08T15:17:15.119 INFO:teuthology.orchestra.run.smithi055.stdout:x86_64 2020-09-08T15:17:15.120 INFO:teuthology.orchestra.run.smithi055:> cat /etc/os-release 2020-09-08T15:17:15.167 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="Ubuntu" 2020-09-08T15:17:15.167 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-09-08T15:17:15.167 INFO:teuthology.orchestra.run.smithi055.stdout:ID=ubuntu 2020-09-08T15:17:15.168 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE=debian 2020-09-08T15:17:15.168 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-09-08T15:17:15.168 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="18.04" 2020-09-08T15:17:15.168 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://www.ubuntu.com/" 2020-09-08T15:17:15.168 INFO:teuthology.orchestra.run.smithi055.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-09-08T15:17:15.169 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-09-08T15:17:15.169 INFO:teuthology.orchestra.run.smithi055.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-09-08T15:17:15.169 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_CODENAME=bionic 2020-09-08T15:17:15.169 INFO:teuthology.orchestra.run.smithi055.stdout:UBUNTU_CODENAME=bionic 2020-09-08T15:17:15.170 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2020-09-08T15:17:15.189 INFO:teuthology.lock.ops:Updating [smithi166.front.sepia.ceph.com]: set os type and version on server 2020-09-08T15:17:15.190 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-09-08T15:17:15.190 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi166.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:17:15.310 INFO:teuthology.orchestra.run.smithi166:> true 2020-09-08T15:17:15.551 INFO:teuthology.orchestra.run.smithi166:> uname -m 2020-09-08T15:17:15.598 INFO:teuthology.orchestra.run.smithi166.stdout:x86_64 2020-09-08T15:17:15.599 INFO:teuthology.orchestra.run.smithi166:> cat /etc/os-release 2020-09-08T15:17:15.652 INFO:teuthology.orchestra.run.smithi166.stdout:NAME="Ubuntu" 2020-09-08T15:17:15.652 INFO:teuthology.orchestra.run.smithi166.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-09-08T15:17:15.652 INFO:teuthology.orchestra.run.smithi166.stdout:ID=ubuntu 2020-09-08T15:17:15.652 INFO:teuthology.orchestra.run.smithi166.stdout:ID_LIKE=debian 2020-09-08T15:17:15.653 INFO:teuthology.orchestra.run.smithi166.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-09-08T15:17:15.653 INFO:teuthology.orchestra.run.smithi166.stdout:VERSION_ID="18.04" 2020-09-08T15:17:15.653 INFO:teuthology.orchestra.run.smithi166.stdout:HOME_URL="https://www.ubuntu.com/" 2020-09-08T15:17:15.653 INFO:teuthology.orchestra.run.smithi166.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-09-08T15:17:15.653 INFO:teuthology.orchestra.run.smithi166.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-09-08T15:17:15.654 INFO:teuthology.orchestra.run.smithi166.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-09-08T15:17:15.654 INFO:teuthology.orchestra.run.smithi166.stdout:VERSION_CODENAME=bionic 2020-09-08T15:17:15.654 INFO:teuthology.orchestra.run.smithi166.stdout:UBUNTU_CODENAME=bionic 2020-09-08T15:17:15.654 INFO:teuthology.lock.ops:Updating smithi166.front.sepia.ceph.com on lock server 2020-09-08T15:17:15.672 INFO:teuthology.task.internal.lock_machines:3 smithi machines locked this try, 3/3 locked so far 2020-09-08T15:17:15.741 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi055.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOAfch+tXrtEIBxFC2+eSVoAHkvg1O/bXTUb1S7641gQpBiFhjn2Br3B31nHr18A8Oy6IXaUmc/pCdv71jF3kFpp3OkoNpUBGGemA7H9sbHcJZPAUaw23NSdHheb+tL1KmB8E3nPFNLn3yo9Jhm1pbCm8m/ZoQp54jxWv9TuyIAurCF7yhjBnLMPa2a123T+2k7qzh1R2wYU/LHcoOXrBrT5pPtao8yFNH7rcfgXoYxhfZ8m75BUEC1l9fzMpVYOHwn1r7ttm+tSJ1M8G4qp9Eys/BokH/deX1baYAS8CsYk4G+rJsqF2Yx5w56Bd5f4dFkbS1YsxEC7LmOsYKvDkN smithi104.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCmYeYsZGk0S7fQ/H4e08dNA6Ne1AodtMmhXsp+baTpbkiF1MJoWxYoQzvPT+q/8sgbymz3er5SzLqJskWTFKmTXyGjesL2m7D8C82vtO+rWaaoHLyHnYE+y4AOkZ5j6cBYt6OmubxXvC3FEmdCyMw7qEMX9eldQg7LhgXIUnZvF4YBbU+63lo5gr235VS50hSrxl4C8aJ/5a0RyR7u5k3dtV9OPesvBhld6RvV/RdBIMeh3zaeAPBDh1haUZR/NOwolRivZlnWD3vE2+QSkdsT6JD8hxOAc9DER3wqyIIBpbK2gVxVFd4R9LgpZ1IwmMtRSWa59/Qjz3EmSApt8bk/ smithi166.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9qoTdK2zfD3AuVFVC0bzO1VTFwz/eZSYQdfuTT3oqfjN+5q1t61C8xidklqIeDNT38RCVPIJJnl2HyMBTWtLJE+x0SYOGImDNrKWqWZ8kpl8FFN5qlNYDIPDKLFi4u1akZ5D9zdtYoY34NVGTiJG2XyC+jLAe8WaXn13Zx32NGmI+3h70CRqbjNu6YOzvUassNOqGMvLBbG2fKqiPBIc5YcxdHPQm9R/VlffvmuC+Dfu/iEsoxT4yhbsAPINWSDPcaN95vpk6iVQ87bU60xNFnp8vfeZpun/Mv8547b02j/pq3X2GmWjF8OCw+HnEotESJlUpRADpx9KTVGN7FNZf 2020-09-08T15:17:15.742 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-09-08T15:17:15.817 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-09-08T15:17:16.365 INFO:teuthology.task.internal:Saving configuration 2020-09-08T15:17:16.604 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-09-08T15:17:16.612 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-09-08T15:17:16.633 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-09-08 15:11:51.800017', 'description': '/home/teuthworker/archive/kchai-2020-09-08_12:29:35-rados-wip-kefu-testing-2020-09-08-0934-distro-basic-smithi/5417828', 'locked_by': 'scheduled_kchai@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFZFYYqcQeKOAEI4/OsLMX0vKTTkGGSmvbO4akFilXVk', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi104.front.sepia.ceph.com'} 2020-09-08T15:17:16.650 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-09-08 15:11:51.800036', 'description': '/home/teuthworker/archive/kchai-2020-09-08_12:29:35-rados-wip-kefu-testing-2020-09-08-0934-distro-basic-smithi/5417828', 'locked_by': 'scheduled_kchai@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINdI5G1veMSxQU9d9tnJsQYpiTo5LuV03HDZQKa8DQIL', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi055.front.sepia.ceph.com'} 2020-09-08T15:17:16.669 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-09-08 15:11:51.800051', 'description': '/home/teuthworker/archive/kchai-2020-09-08_12:29:35-rados-wip-kefu-testing-2020-09-08-0934-distro-basic-smithi/5417828', 'locked_by': 'scheduled_kchai@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICRje85XLnLm4p6Ns4d4FX6qaLV2Zb6Bm7eYlZygwasK', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi166.front.sepia.ceph.com'} 2020-09-08T15:17:16.669 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-09-08T15:17:16.676 INFO:teuthology.task.internal:roles: ubuntu@smithi104.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2020-09-08T15:17:16.677 INFO:teuthology.task.internal:roles: ubuntu@smithi055.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2020-09-08T15:17:16.677 INFO:teuthology.task.internal:roles: ubuntu@smithi166.front.sepia.ceph.com - ['mon.c', 'osd.8', 'osd.9', 'osd.10', 'osd.11', 'client.2'] 2020-09-08T15:17:16.677 INFO:teuthology.run_tasks:Running task console_log... 2020-09-08T15:17:16.850 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4f1c98eea0>, signals=[15]) 2020-09-08T15:17:16.852 INFO:teuthology.run_tasks:Running task internal.connect... 2020-09-08T15:17:16.861 INFO:teuthology.task.internal:Opening connections... 2020-09-08T15:17:16.864 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi104.front.sepia.ceph.com 2020-09-08T15:17:16.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:17:16.995 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi055.front.sepia.ceph.com 2020-09-08T15:17:16.996 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:17:17.086 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi166.front.sepia.ceph.com 2020-09-08T15:17:17.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi166.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:17:17.208 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-09-08T15:17:17.247 INFO:teuthology.orchestra.run.smithi104:> uname -m 2020-09-08T15:17:17.255 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2020-09-08T15:17:17.255 INFO:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2020-09-08T15:17:17.301 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="Ubuntu" 2020-09-08T15:17:17.301 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-09-08T15:17:17.301 INFO:teuthology.orchestra.run.smithi104.stdout:ID=ubuntu 2020-09-08T15:17:17.302 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE=debian 2020-09-08T15:17:17.302 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-09-08T15:17:17.302 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="18.04" 2020-09-08T15:17:17.302 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://www.ubuntu.com/" 2020-09-08T15:17:17.302 INFO:teuthology.orchestra.run.smithi104.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-09-08T15:17:17.303 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-09-08T15:17:17.303 INFO:teuthology.orchestra.run.smithi104.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-09-08T15:17:17.304 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_CODENAME=bionic 2020-09-08T15:17:17.304 INFO:teuthology.orchestra.run.smithi104.stdout:UBUNTU_CODENAME=bionic 2020-09-08T15:17:17.304 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2020-09-08T15:17:17.322 INFO:teuthology.orchestra.run.smithi055:> uname -m 2020-09-08T15:17:17.329 INFO:teuthology.orchestra.run.smithi055.stdout:x86_64 2020-09-08T15:17:17.330 INFO:teuthology.orchestra.run.smithi055:> cat /etc/os-release 2020-09-08T15:17:17.377 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="Ubuntu" 2020-09-08T15:17:17.378 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-09-08T15:17:17.379 INFO:teuthology.orchestra.run.smithi055.stdout:ID=ubuntu 2020-09-08T15:17:17.379 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE=debian 2020-09-08T15:17:17.379 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-09-08T15:17:17.379 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="18.04" 2020-09-08T15:17:17.379 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://www.ubuntu.com/" 2020-09-08T15:17:17.380 INFO:teuthology.orchestra.run.smithi055.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-09-08T15:17:17.380 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-09-08T15:17:17.380 INFO:teuthology.orchestra.run.smithi055.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-09-08T15:17:17.380 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_CODENAME=bionic 2020-09-08T15:17:17.380 INFO:teuthology.orchestra.run.smithi055.stdout:UBUNTU_CODENAME=bionic 2020-09-08T15:17:17.381 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2020-09-08T15:17:17.396 INFO:teuthology.orchestra.run.smithi166:> uname -m 2020-09-08T15:17:17.402 INFO:teuthology.orchestra.run.smithi166.stdout:x86_64 2020-09-08T15:17:17.403 INFO:teuthology.orchestra.run.smithi166:> cat /etc/os-release 2020-09-08T15:17:17.451 INFO:teuthology.orchestra.run.smithi166.stdout:NAME="Ubuntu" 2020-09-08T15:17:17.452 INFO:teuthology.orchestra.run.smithi166.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-09-08T15:17:17.452 INFO:teuthology.orchestra.run.smithi166.stdout:ID=ubuntu 2020-09-08T15:17:17.452 INFO:teuthology.orchestra.run.smithi166.stdout:ID_LIKE=debian 2020-09-08T15:17:17.452 INFO:teuthology.orchestra.run.smithi166.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-09-08T15:17:17.453 INFO:teuthology.orchestra.run.smithi166.stdout:VERSION_ID="18.04" 2020-09-08T15:17:17.453 INFO:teuthology.orchestra.run.smithi166.stdout:HOME_URL="https://www.ubuntu.com/" 2020-09-08T15:17:17.453 INFO:teuthology.orchestra.run.smithi166.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-09-08T15:17:17.454 INFO:teuthology.orchestra.run.smithi166.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-09-08T15:17:17.454 INFO:teuthology.orchestra.run.smithi166.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-09-08T15:17:17.454 INFO:teuthology.orchestra.run.smithi166.stdout:VERSION_CODENAME=bionic 2020-09-08T15:17:17.455 INFO:teuthology.orchestra.run.smithi166.stdout:UBUNTU_CODENAME=bionic 2020-09-08T15:17:17.455 INFO:teuthology.lock.ops:Updating smithi166.front.sepia.ceph.com on lock server 2020-09-08T15:17:17.470 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-09-08T15:17:17.487 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-09-08T15:17:17.498 INFO:teuthology.task.internal:Checking for old test directory... 2020-09-08T15:17:17.499 INFO:teuthology.orchestra.run.smithi055:> test '!' -e /home/ubuntu/cephtest 2020-09-08T15:17:17.504 INFO:teuthology.orchestra.run.smithi104:> test '!' -e /home/ubuntu/cephtest 2020-09-08T15:17:17.507 INFO:teuthology.orchestra.run.smithi166:> test '!' -e /home/ubuntu/cephtest 2020-09-08T15:17:17.512 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-09-08T15:17:17.523 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-09-08T15:17:17.524 INFO:teuthology.orchestra.run.smithi055:> test -z $(ls -A /var/lib/ceph) 2020-09-08T15:17:17.550 INFO:teuthology.orchestra.run.smithi104:> test -z $(ls -A /var/lib/ceph) 2020-09-08T15:17:17.554 INFO:teuthology.orchestra.run.smithi166:> test -z $(ls -A /var/lib/ceph) 2020-09-08T15:17:17.562 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-09-08T15:17:17.654 INFO:teuthology.run_tasks:Running task kernel... 2020-09-08T15:17:17.678 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'osd.3': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'osd.4': {'kdb': True, 'sha1': 'distro'}, 'osd.5': {'kdb': True, 'sha1': 'distro'}, 'osd.6': {'kdb': True, 'sha1': 'distro'}, 'osd.7': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}, 'osd.8': {'kdb': True, 'sha1': 'distro'}, 'osd.9': {'kdb': True, 'sha1': 'distro'}, 'osd.10': {'kdb': True, 'sha1': 'distro'}, 'osd.11': {'kdb': True, 'sha1': 'distro'}, 'client.2': {'kdb': True, 'sha1': 'distro'}} 2020-09-08T15:17:17.679 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2020-09-08T15:17:17.679 INFO:teuthology.orchestra.run.smithi104:> uname -r 2020-09-08T15:17:17.686 INFO:teuthology.orchestra.run.smithi104.stdout:4.15.0-115-generic 2020-09-08T15:17:17.687 INFO:teuthology.task.kernel:Running kernel on smithi104: 4.15.0-115-generic 2020-09-08T15:17:17.687 INFO:teuthology.orchestra.run.smithi104:> sudo apt-get clean 2020-09-08T15:17:17.810 INFO:teuthology.orchestra.run.smithi104:> sudo apt-get update 2020-09-08T15:17:18.086 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-09-08T15:17:18.087 INFO:teuthology.orchestra.run.smithi104.stdout:Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-09-08T15:17:18.159 INFO:teuthology.orchestra.run.smithi104.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-09-08T15:17:18.466 INFO:teuthology.orchestra.run.smithi104.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-09-08T15:17:18.567 INFO:teuthology.orchestra.run.smithi104.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [536 kB] 2020-09-08T15:17:18.684 INFO:teuthology.orchestra.run.smithi104.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,072 kB] 2020-09-08T15:17:18.913 INFO:teuthology.orchestra.run.smithi104.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [849 kB] 2020-09-08T15:17:19.026 INFO:teuthology.orchestra.run.smithi104.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [264 kB] 2020-09-08T15:17:19.045 INFO:teuthology.orchestra.run.smithi104.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [87.3 kB] 2020-09-08T15:17:19.046 INFO:teuthology.orchestra.run.smithi104.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [741 kB] 2020-09-08T15:17:19.052 INFO:teuthology.orchestra.run.smithi104.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [18.8 kB] 2020-09-08T15:17:19.072 INFO:teuthology.orchestra.run.smithi104.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [356 kB] 2020-09-08T15:17:19.085 INFO:teuthology.orchestra.run.smithi104.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [96.4 kB] 2020-09-08T15:17:19.087 INFO:teuthology.orchestra.run.smithi104.stdout:Get:14 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [636 kB] 2020-09-08T15:17:19.089 INFO:teuthology.orchestra.run.smithi104.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [20.9 kB] 2020-09-08T15:17:19.089 INFO:teuthology.orchestra.run.smithi104.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,028 kB] 2020-09-08T15:17:19.112 INFO:teuthology.orchestra.run.smithi104.stdout:Get:17 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [701 kB] 2020-09-08T15:17:19.144 INFO:teuthology.orchestra.run.smithi104.stdout:Get:18 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [232 kB] 2020-09-08T15:17:19.144 INFO:teuthology.orchestra.run.smithi104.stdout:Get:19 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [8,512 B] 2020-09-08T15:17:19.145 INFO:teuthology.orchestra.run.smithi104.stdout:Get:20 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [2,908 B] 2020-09-08T15:17:19.174 INFO:teuthology.orchestra.run.smithi104.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,107 kB] 2020-09-08T15:17:19.221 INFO:teuthology.orchestra.run.smithi104.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [344 kB] 2020-09-08T15:17:19.226 INFO:teuthology.orchestra.run.smithi104.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [19.6 kB] 2020-09-08T15:17:19.226 INFO:teuthology.orchestra.run.smithi104.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6,768 B] 2020-09-08T15:17:20.189 INFO:teuthology.orchestra.run.smithi104.stdout:Fetched 8,378 kB in 2s (3,835 kB/s) 2020-09-08T15:17:21.510 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2020-09-08T15:17:21.556 INFO:teuthology.orchestra.run.smithi104:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-09-08T15:17:21.660 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2020-09-08T15:17:21.861 INFO:teuthology.orchestra.run.smithi104.stdout:Building dependency tree... 2020-09-08T15:17:21.861 INFO:teuthology.orchestra.run.smithi104.stdout:Reading state information... 2020-09-08T15:17:21.936 INFO:teuthology.orchestra.run.smithi104.stderr:E: Unable to locate package linux-image-current-generic 2020-09-08T15:17:21.938 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-09-08T15:17:21.938 INFO:teuthology.orchestra.run.smithi104:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-09-08T15:17:22.017 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2020-09-08T15:17:22.241 INFO:teuthology.orchestra.run.smithi104.stdout:Building dependency tree... 2020-09-08T15:17:22.242 INFO:teuthology.orchestra.run.smithi104.stdout:Reading state information... 2020-09-08T15:17:22.407 INFO:teuthology.orchestra.run.smithi104.stdout:The following additional packages will be installed: 2020-09-08T15:17:22.408 INFO:teuthology.orchestra.run.smithi104.stdout: linux-generic linux-headers-4.15.0-117 linux-headers-4.15.0-117-generic 2020-09-08T15:17:22.408 INFO:teuthology.orchestra.run.smithi104.stdout: linux-headers-generic linux-image-4.15.0-117-generic 2020-09-08T15:17:22.408 INFO:teuthology.orchestra.run.smithi104.stdout: linux-modules-4.15.0-117-generic linux-modules-extra-4.15.0-117-generic 2020-09-08T15:17:22.409 INFO:teuthology.orchestra.run.smithi104.stdout:Suggested packages: 2020-09-08T15:17:22.409 INFO:teuthology.orchestra.run.smithi104.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2020-09-08T15:17:22.410 INFO:teuthology.orchestra.run.smithi104.stdout:Recommended packages: 2020-09-08T15:17:22.410 INFO:teuthology.orchestra.run.smithi104.stdout: thermald 2020-09-08T15:17:22.441 INFO:teuthology.orchestra.run.smithi104.stdout:The following NEW packages will be installed: 2020-09-08T15:17:22.442 INFO:teuthology.orchestra.run.smithi104.stdout: linux-headers-4.15.0-117 linux-headers-4.15.0-117-generic 2020-09-08T15:17:22.442 INFO:teuthology.orchestra.run.smithi104.stdout: linux-image-4.15.0-117-generic linux-modules-4.15.0-117-generic 2020-09-08T15:17:22.442 INFO:teuthology.orchestra.run.smithi104.stdout: linux-modules-extra-4.15.0-117-generic 2020-09-08T15:17:22.443 INFO:teuthology.orchestra.run.smithi104.stdout:The following packages will be upgraded: 2020-09-08T15:17:22.444 INFO:teuthology.orchestra.run.smithi104.stdout: linux-generic linux-headers-generic linux-image-generic 2020-09-08T15:17:22.640 INFO:teuthology.orchestra.run.smithi104.stdout:3 upgraded, 5 newly installed, 0 to remove and 11 not upgraded. 2020-09-08T15:17:22.641 INFO:teuthology.orchestra.run.smithi104.stdout:Need to get 65.9 MB of archives. 2020-09-08T15:17:22.641 INFO:teuthology.orchestra.run.smithi104.stdout:After this operation, 333 MB of additional disk space will be used. 2020-09-08T15:17:22.641 INFO:teuthology.orchestra.run.smithi104.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-117-generic amd64 4.15.0-117.118 [13.1 MB] 2020-09-08T15:17:23.733 INFO:teuthology.orchestra.run.smithi104.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-117-generic amd64 4.15.0-117.118 [8,012 kB] 2020-09-08T15:17:23.938 INFO:teuthology.orchestra.run.smithi104.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-117-generic amd64 4.15.0-117.118 [32.8 MB] 2020-09-08T15:17:25.015 INFO:teuthology.orchestra.run.smithi104.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.117.104 [1,872 B] 2020-09-08T15:17:25.016 INFO:teuthology.orchestra.run.smithi104.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.117.104 [2,676 B] 2020-09-08T15:17:25.016 INFO:teuthology.orchestra.run.smithi104.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-117 all 4.15.0-117.118 [10.9 MB] 2020-09-08T15:17:25.414 INFO:teuthology.orchestra.run.smithi104.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-117-generic amd64 4.15.0-117.118 [1,100 kB] 2020-09-08T15:17:25.455 INFO:teuthology.orchestra.run.smithi104.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.117.104 [2,572 B] 2020-09-08T15:17:25.908 INFO:teuthology.orchestra.run.smithi104.stdout:Fetched 65.9 MB in 3s (22.0 MB/s) 2020-09-08T15:17:26.082 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package linux-modules-4.15.0-117-generic. 2020-09-08T15:17:26.120 INFO:teuthology.orchestra.run.smithi104.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102058 files and directories currently installed.) 2020-09-08T15:17:26.123 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../0-linux-modules-4.15.0-117-generic_4.15.0-117.118_amd64.deb ... 2020-09-08T15:17:26.169 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking linux-modules-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:17:27.785 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package linux-image-4.15.0-117-generic. 2020-09-08T15:17:27.802 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../1-linux-image-4.15.0-117-generic_4.15.0-117.118_amd64.deb ... 2020-09-08T15:17:27.881 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking linux-image-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:17:28.380 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-117-generic. 2020-09-08T15:17:28.391 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-117-generic_4.15.0-117.118_amd64.deb ... 2020-09-08T15:17:28.429 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking linux-modules-extra-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:17:32.517 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../3-linux-generic_4.15.0.117.104_amd64.deb ... 2020-09-08T15:17:32.628 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking linux-generic (4.15.0.117.104) over (4.15.0.115.103) ... 2020-09-08T15:17:32.972 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.117.104_amd64.deb ... 2020-09-08T15:17:33.099 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking linux-image-generic (4.15.0.117.104) over (4.15.0.115.103) ... 2020-09-08T15:17:33.413 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package linux-headers-4.15.0-117. 2020-09-08T15:17:33.424 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../5-linux-headers-4.15.0-117_4.15.0-117.118_all.deb ... 2020-09-08T15:17:33.462 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking linux-headers-4.15.0-117 (4.15.0-117.118) ... 2020-09-08T15:17:37.920 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package linux-headers-4.15.0-117-generic. 2020-09-08T15:17:37.943 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../6-linux-headers-4.15.0-117-generic_4.15.0-117.118_amd64.deb ... 2020-09-08T15:17:37.987 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking linux-headers-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:17:40.362 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.117.104_amd64.deb ... 2020-09-08T15:17:40.485 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking linux-headers-generic (4.15.0.117.104) over (4.15.0.115.103) ... 2020-09-08T15:17:40.822 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up linux-modules-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:17:41.397 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up linux-headers-4.15.0-117 (4.15.0-117.118) ... 2020-09-08T15:17:41.549 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up linux-image-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:17:42.011 INFO:teuthology.orchestra.run.smithi104.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-117-generic 2020-09-08T15:17:42.011 INFO:teuthology.orchestra.run.smithi104.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-117-generic 2020-09-08T15:17:42.153 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up linux-headers-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:17:42.284 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up linux-modules-extra-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:17:42.769 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up linux-image-generic (4.15.0.117.104) ... 2020-09-08T15:17:42.898 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up linux-headers-generic (4.15.0.117.104) ... 2020-09-08T15:17:43.022 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up linux-generic (4.15.0.117.104) ... 2020-09-08T15:17:43.148 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for linux-image-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:17:43.329 INFO:teuthology.orchestra.run.smithi104.stdout:/etc/kernel/postinst.d/initramfs-tools: 2020-09-08T15:17:43.330 INFO:teuthology.orchestra.run.smithi104.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-117-generic 2020-09-08T15:17:58.880 INFO:teuthology.orchestra.run.smithi104.stdout:/etc/kernel/postinst.d/zz-update-grub: 2020-09-08T15:17:58.881 INFO:teuthology.orchestra.run.smithi104.stdout:Sourcing file `/etc/default/grub' 2020-09-08T15:17:58.921 INFO:teuthology.orchestra.run.smithi104.stdout:Generating grub configuration file ... 2020-09-08T15:17:59.254 INFO:teuthology.orchestra.run.smithi104.stdout:Found linux image: /boot/vmlinuz-4.15.0-117-generic 2020-09-08T15:17:59.282 INFO:teuthology.orchestra.run.smithi104.stdout:Found initrd image: /boot/initrd.img-4.15.0-117-generic 2020-09-08T15:17:59.609 INFO:teuthology.orchestra.run.smithi104.stdout:Found linux image: /boot/vmlinuz-4.15.0-115-generic 2020-09-08T15:17:59.639 INFO:teuthology.orchestra.run.smithi104.stdout:Found initrd image: /boot/initrd.img-4.15.0-115-generic 2020-09-08T15:18:01.272 INFO:teuthology.orchestra.run.smithi104.stdout:done 2020-09-08T15:18:01.695 INFO:teuthology.orchestra.run.smithi104:> dpkg -s linux-image-generic 2020-09-08T15:18:01.722 INFO:teuthology.orchestra.run.smithi104.stdout:Package: linux-image-generic 2020-09-08T15:18:01.722 INFO:teuthology.orchestra.run.smithi104.stdout:Status: install ok installed 2020-09-08T15:18:01.722 INFO:teuthology.orchestra.run.smithi104.stdout:Priority: optional 2020-09-08T15:18:01.723 INFO:teuthology.orchestra.run.smithi104.stdout:Section: kernel 2020-09-08T15:18:01.723 INFO:teuthology.orchestra.run.smithi104.stdout:Installed-Size: 16 2020-09-08T15:18:01.723 INFO:teuthology.orchestra.run.smithi104.stdout:Maintainer: Ubuntu Kernel Team 2020-09-08T15:18:01.723 INFO:teuthology.orchestra.run.smithi104.stdout:Architecture: amd64 2020-09-08T15:18:01.724 INFO:teuthology.orchestra.run.smithi104.stdout:Source: linux-meta 2020-09-08T15:18:01.724 INFO:teuthology.orchestra.run.smithi104.stdout:Version: 4.15.0.117.104 2020-09-08T15:18:01.724 INFO:teuthology.orchestra.run.smithi104.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.9) 2020-09-08T15:18:01.724 INFO:teuthology.orchestra.run.smithi104.stdout:Depends: linux-image-4.15.0-117-generic, linux-modules-extra-4.15.0-117-generic, linux-firmware, intel-microcode, amd64-microcode 2020-09-08T15:18:01.724 INFO:teuthology.orchestra.run.smithi104.stdout:Recommends: thermald 2020-09-08T15:18:01.725 INFO:teuthology.orchestra.run.smithi104.stdout:Description: Generic Linux kernel image 2020-09-08T15:18:01.725 INFO:teuthology.orchestra.run.smithi104.stdout: This package will always depend on the latest generic kernel image 2020-09-08T15:18:01.725 INFO:teuthology.orchestra.run.smithi104.stdout: available. 2020-09-08T15:18:01.726 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-115-generic Expected: 4.15.0-117-generic 2020-09-08T15:18:01.726 INFO:teuthology.orchestra.run.smithi055:> uname -r 2020-09-08T15:18:01.740 INFO:teuthology.orchestra.run.smithi055.stdout:4.15.0-115-generic 2020-09-08T15:18:01.741 INFO:teuthology.task.kernel:Running kernel on smithi055: 4.15.0-115-generic 2020-09-08T15:18:01.741 INFO:teuthology.orchestra.run.smithi055:> sudo apt-get clean 2020-09-08T15:18:01.851 INFO:teuthology.orchestra.run.smithi055:> sudo apt-get update 2020-09-08T15:18:02.126 INFO:teuthology.orchestra.run.smithi055.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-09-08T15:18:02.147 INFO:teuthology.orchestra.run.smithi055.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-09-08T15:18:02.218 INFO:teuthology.orchestra.run.smithi055.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-09-08T15:18:02.522 INFO:teuthology.orchestra.run.smithi055.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-09-08T15:18:02.590 INFO:teuthology.orchestra.run.smithi055.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [849 kB] 2020-09-08T15:18:02.729 INFO:teuthology.orchestra.run.smithi055.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [741 kB] 2020-09-08T15:18:02.991 INFO:teuthology.orchestra.run.smithi055.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [536 kB] 2020-09-08T15:18:03.028 INFO:teuthology.orchestra.run.smithi055.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [264 kB] 2020-09-08T15:18:03.043 INFO:teuthology.orchestra.run.smithi055.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [87.3 kB] 2020-09-08T15:18:03.050 INFO:teuthology.orchestra.run.smithi055.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [18.8 kB] 2020-09-08T15:18:03.050 INFO:teuthology.orchestra.run.smithi055.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [636 kB] 2020-09-08T15:18:03.056 INFO:teuthology.orchestra.run.smithi055.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,072 kB] 2020-09-08T15:18:03.122 INFO:teuthology.orchestra.run.smithi055.stdout:Get:13 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [701 kB] 2020-09-08T15:18:03.160 INFO:teuthology.orchestra.run.smithi055.stdout:Get:14 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [232 kB] 2020-09-08T15:18:03.161 INFO:teuthology.orchestra.run.smithi055.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [356 kB] 2020-09-08T15:18:03.168 INFO:teuthology.orchestra.run.smithi055.stdout:Get:16 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [8,512 B] 2020-09-08T15:18:03.169 INFO:teuthology.orchestra.run.smithi055.stdout:Get:17 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [2,908 B] 2020-09-08T15:18:03.186 INFO:teuthology.orchestra.run.smithi055.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [96.4 kB] 2020-09-08T15:18:03.197 INFO:teuthology.orchestra.run.smithi055.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [20.9 kB] 2020-09-08T15:18:03.198 INFO:teuthology.orchestra.run.smithi055.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,107 kB] 2020-09-08T15:18:03.254 INFO:teuthology.orchestra.run.smithi055.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,028 kB] 2020-09-08T15:18:03.286 INFO:teuthology.orchestra.run.smithi055.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [344 kB] 2020-09-08T15:18:03.296 INFO:teuthology.orchestra.run.smithi055.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [19.6 kB] 2020-09-08T15:18:03.297 INFO:teuthology.orchestra.run.smithi055.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6,768 B] 2020-09-08T15:18:04.257 INFO:teuthology.orchestra.run.smithi055.stdout:Fetched 8,378 kB in 2s (3,840 kB/s) 2020-09-08T15:18:05.658 INFO:teuthology.orchestra.run.smithi055.stdout:Reading package lists... 2020-09-08T15:18:05.704 INFO:teuthology.orchestra.run.smithi055:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-09-08T15:18:05.790 INFO:teuthology.orchestra.run.smithi055.stdout:Reading package lists... 2020-09-08T15:18:06.046 INFO:teuthology.orchestra.run.smithi055.stdout:Building dependency tree... 2020-09-08T15:18:06.047 INFO:teuthology.orchestra.run.smithi055.stdout:Reading state information... 2020-09-08T15:18:06.137 INFO:teuthology.orchestra.run.smithi055.stderr:E: Unable to locate package linux-image-current-generic 2020-09-08T15:18:06.139 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-09-08T15:18:06.139 INFO:teuthology.orchestra.run.smithi055:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-09-08T15:18:06.216 INFO:teuthology.orchestra.run.smithi055.stdout:Reading package lists... 2020-09-08T15:18:06.476 INFO:teuthology.orchestra.run.smithi055.stdout:Building dependency tree... 2020-09-08T15:18:06.477 INFO:teuthology.orchestra.run.smithi055.stdout:Reading state information... 2020-09-08T15:18:06.668 INFO:teuthology.orchestra.run.smithi055.stdout:The following additional packages will be installed: 2020-09-08T15:18:06.668 INFO:teuthology.orchestra.run.smithi055.stdout: linux-generic linux-headers-4.15.0-117 linux-headers-4.15.0-117-generic 2020-09-08T15:18:06.668 INFO:teuthology.orchestra.run.smithi055.stdout: linux-headers-generic linux-image-4.15.0-117-generic 2020-09-08T15:18:06.668 INFO:teuthology.orchestra.run.smithi055.stdout: linux-modules-4.15.0-117-generic linux-modules-extra-4.15.0-117-generic 2020-09-08T15:18:06.669 INFO:teuthology.orchestra.run.smithi055.stdout:Suggested packages: 2020-09-08T15:18:06.669 INFO:teuthology.orchestra.run.smithi055.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2020-09-08T15:18:06.669 INFO:teuthology.orchestra.run.smithi055.stdout:Recommended packages: 2020-09-08T15:18:06.670 INFO:teuthology.orchestra.run.smithi055.stdout: thermald 2020-09-08T15:18:06.702 INFO:teuthology.orchestra.run.smithi055.stdout:The following NEW packages will be installed: 2020-09-08T15:18:06.702 INFO:teuthology.orchestra.run.smithi055.stdout: linux-headers-4.15.0-117 linux-headers-4.15.0-117-generic 2020-09-08T15:18:06.703 INFO:teuthology.orchestra.run.smithi055.stdout: linux-image-4.15.0-117-generic linux-modules-4.15.0-117-generic 2020-09-08T15:18:06.703 INFO:teuthology.orchestra.run.smithi055.stdout: linux-modules-extra-4.15.0-117-generic 2020-09-08T15:18:06.704 INFO:teuthology.orchestra.run.smithi055.stdout:The following packages will be upgraded: 2020-09-08T15:18:06.704 INFO:teuthology.orchestra.run.smithi055.stdout: linux-generic linux-headers-generic linux-image-generic 2020-09-08T15:18:06.903 INFO:teuthology.orchestra.run.smithi055.stdout:3 upgraded, 5 newly installed, 0 to remove and 11 not upgraded. 2020-09-08T15:18:06.903 INFO:teuthology.orchestra.run.smithi055.stdout:Need to get 65.9 MB of archives. 2020-09-08T15:18:06.904 INFO:teuthology.orchestra.run.smithi055.stdout:After this operation, 333 MB of additional disk space will be used. 2020-09-08T15:18:06.904 INFO:teuthology.orchestra.run.smithi055.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-117-generic amd64 4.15.0-117.118 [13.1 MB] 2020-09-08T15:18:08.008 INFO:teuthology.orchestra.run.smithi055.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-117-generic amd64 4.15.0-117.118 [8,012 kB] 2020-09-08T15:18:08.216 INFO:teuthology.orchestra.run.smithi055.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-117-generic amd64 4.15.0-117.118 [32.8 MB] 2020-09-08T15:18:09.250 INFO:teuthology.orchestra.run.smithi055.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.117.104 [1,872 B] 2020-09-08T15:18:09.251 INFO:teuthology.orchestra.run.smithi055.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.117.104 [2,676 B] 2020-09-08T15:18:09.251 INFO:teuthology.orchestra.run.smithi055.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-117 all 4.15.0-117.118 [10.9 MB] 2020-09-08T15:18:09.564 INFO:teuthology.orchestra.run.smithi055.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-117-generic amd64 4.15.0-117.118 [1,100 kB] 2020-09-08T15:18:09.603 INFO:teuthology.orchestra.run.smithi055.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.117.104 [2,572 B] 2020-09-08T15:18:10.089 INFO:teuthology.orchestra.run.smithi055.stdout:Fetched 65.9 MB in 3s (22.9 MB/s) 2020-09-08T15:18:10.244 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package linux-modules-4.15.0-117-generic. 2020-09-08T15:18:10.283 INFO:teuthology.orchestra.run.smithi055.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102058 files and directories currently installed.) 2020-09-08T15:18:10.286 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../0-linux-modules-4.15.0-117-generic_4.15.0-117.118_amd64.deb ... 2020-09-08T15:18:10.330 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking linux-modules-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:18:12.039 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package linux-image-4.15.0-117-generic. 2020-09-08T15:18:12.050 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../1-linux-image-4.15.0-117-generic_4.15.0-117.118_amd64.deb ... 2020-09-08T15:18:12.139 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking linux-image-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:18:12.650 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-117-generic. 2020-09-08T15:18:12.662 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-117-generic_4.15.0-117.118_amd64.deb ... 2020-09-08T15:18:12.701 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking linux-modules-extra-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:18:16.666 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../3-linux-generic_4.15.0.117.104_amd64.deb ... 2020-09-08T15:18:16.786 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking linux-generic (4.15.0.117.104) over (4.15.0.115.103) ... 2020-09-08T15:18:17.128 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.117.104_amd64.deb ... 2020-09-08T15:18:17.239 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking linux-image-generic (4.15.0.117.104) over (4.15.0.115.103) ... 2020-09-08T15:18:17.534 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package linux-headers-4.15.0-117. 2020-09-08T15:18:17.548 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../5-linux-headers-4.15.0-117_4.15.0-117.118_all.deb ... 2020-09-08T15:18:17.593 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking linux-headers-4.15.0-117 (4.15.0-117.118) ... 2020-09-08T15:18:22.330 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package linux-headers-4.15.0-117-generic. 2020-09-08T15:18:22.349 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../6-linux-headers-4.15.0-117-generic_4.15.0-117.118_amd64.deb ... 2020-09-08T15:18:22.391 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking linux-headers-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:18:24.903 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.117.104_amd64.deb ... 2020-09-08T15:18:25.022 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking linux-headers-generic (4.15.0.117.104) over (4.15.0.115.103) ... 2020-09-08T15:18:25.366 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up linux-modules-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:18:25.970 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up linux-headers-4.15.0-117 (4.15.0-117.118) ... 2020-09-08T15:18:26.096 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up linux-image-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:18:26.564 INFO:teuthology.orchestra.run.smithi055.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-117-generic 2020-09-08T15:18:26.565 INFO:teuthology.orchestra.run.smithi055.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-117-generic 2020-09-08T15:18:26.698 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up linux-headers-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:18:26.824 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up linux-modules-extra-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:18:27.337 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up linux-image-generic (4.15.0.117.104) ... 2020-09-08T15:18:27.502 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up linux-headers-generic (4.15.0.117.104) ... 2020-09-08T15:18:27.628 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up linux-generic (4.15.0.117.104) ... 2020-09-08T15:18:27.754 INFO:teuthology.orchestra.run.smithi055.stdout:Processing triggers for linux-image-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:18:27.951 INFO:teuthology.orchestra.run.smithi055.stdout:/etc/kernel/postinst.d/initramfs-tools: 2020-09-08T15:18:27.952 INFO:teuthology.orchestra.run.smithi055.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-117-generic 2020-09-08T15:18:43.785 INFO:teuthology.orchestra.run.smithi055.stdout:/etc/kernel/postinst.d/zz-update-grub: 2020-09-08T15:18:43.785 INFO:teuthology.orchestra.run.smithi055.stdout:Sourcing file `/etc/default/grub' 2020-09-08T15:18:43.826 INFO:teuthology.orchestra.run.smithi055.stdout:Generating grub configuration file ... 2020-09-08T15:18:44.151 INFO:teuthology.orchestra.run.smithi055.stdout:Found linux image: /boot/vmlinuz-4.15.0-117-generic 2020-09-08T15:18:44.186 INFO:teuthology.orchestra.run.smithi055.stdout:Found initrd image: /boot/initrd.img-4.15.0-117-generic 2020-09-08T15:18:44.485 INFO:teuthology.orchestra.run.smithi055.stdout:Found linux image: /boot/vmlinuz-4.15.0-115-generic 2020-09-08T15:18:44.515 INFO:teuthology.orchestra.run.smithi055.stdout:Found initrd image: /boot/initrd.img-4.15.0-115-generic 2020-09-08T15:18:45.535 INFO:teuthology.orchestra.run.smithi055.stdout:done 2020-09-08T15:18:45.924 INFO:teuthology.orchestra.run.smithi055:> dpkg -s linux-image-generic 2020-09-08T15:18:45.951 INFO:teuthology.orchestra.run.smithi055.stdout:Package: linux-image-generic 2020-09-08T15:18:45.951 INFO:teuthology.orchestra.run.smithi055.stdout:Status: install ok installed 2020-09-08T15:18:45.951 INFO:teuthology.orchestra.run.smithi055.stdout:Priority: optional 2020-09-08T15:18:45.951 INFO:teuthology.orchestra.run.smithi055.stdout:Section: kernel 2020-09-08T15:18:45.952 INFO:teuthology.orchestra.run.smithi055.stdout:Installed-Size: 16 2020-09-08T15:18:45.952 INFO:teuthology.orchestra.run.smithi055.stdout:Maintainer: Ubuntu Kernel Team 2020-09-08T15:18:45.952 INFO:teuthology.orchestra.run.smithi055.stdout:Architecture: amd64 2020-09-08T15:18:45.952 INFO:teuthology.orchestra.run.smithi055.stdout:Source: linux-meta 2020-09-08T15:18:45.952 INFO:teuthology.orchestra.run.smithi055.stdout:Version: 4.15.0.117.104 2020-09-08T15:18:45.953 INFO:teuthology.orchestra.run.smithi055.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.9) 2020-09-08T15:18:45.953 INFO:teuthology.orchestra.run.smithi055.stdout:Depends: linux-image-4.15.0-117-generic, linux-modules-extra-4.15.0-117-generic, linux-firmware, intel-microcode, amd64-microcode 2020-09-08T15:18:45.953 INFO:teuthology.orchestra.run.smithi055.stdout:Recommends: thermald 2020-09-08T15:18:45.953 INFO:teuthology.orchestra.run.smithi055.stdout:Description: Generic Linux kernel image 2020-09-08T15:18:45.954 INFO:teuthology.orchestra.run.smithi055.stdout: This package will always depend on the latest generic kernel image 2020-09-08T15:18:45.954 INFO:teuthology.orchestra.run.smithi055.stdout: available. 2020-09-08T15:18:45.954 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-115-generic Expected: 4.15.0-117-generic 2020-09-08T15:18:45.955 INFO:teuthology.orchestra.run.smithi166:> uname -r 2020-09-08T15:18:45.974 INFO:teuthology.orchestra.run.smithi166.stdout:4.15.0-115-generic 2020-09-08T15:18:45.975 INFO:teuthology.task.kernel:Running kernel on smithi166: 4.15.0-115-generic 2020-09-08T15:18:45.975 INFO:teuthology.orchestra.run.smithi166:> sudo apt-get clean 2020-09-08T15:18:46.158 INFO:teuthology.orchestra.run.smithi166:> sudo apt-get update 2020-09-08T15:18:46.442 INFO:teuthology.orchestra.run.smithi166.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-09-08T15:18:46.459 INFO:teuthology.orchestra.run.smithi166.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-09-08T15:18:46.533 INFO:teuthology.orchestra.run.smithi166.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-09-08T15:18:46.823 INFO:teuthology.orchestra.run.smithi166.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-09-08T15:18:46.890 INFO:teuthology.orchestra.run.smithi166.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [849 kB] 2020-09-08T15:18:47.021 INFO:teuthology.orchestra.run.smithi166.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [741 kB] 2020-09-08T15:18:47.277 INFO:teuthology.orchestra.run.smithi166.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [536 kB] 2020-09-08T15:18:47.315 INFO:teuthology.orchestra.run.smithi166.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [264 kB] 2020-09-08T15:18:47.334 INFO:teuthology.orchestra.run.smithi166.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [87.3 kB] 2020-09-08T15:18:47.344 INFO:teuthology.orchestra.run.smithi166.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [18.8 kB] 2020-09-08T15:18:47.345 INFO:teuthology.orchestra.run.smithi166.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [701 kB] 2020-09-08T15:18:47.351 INFO:teuthology.orchestra.run.smithi166.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,072 kB] 2020-09-08T15:18:47.413 INFO:teuthology.orchestra.run.smithi166.stdout:Get:13 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [636 kB] 2020-09-08T15:18:47.447 INFO:teuthology.orchestra.run.smithi166.stdout:Get:14 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [232 kB] 2020-09-08T15:18:47.454 INFO:teuthology.orchestra.run.smithi166.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [356 kB] 2020-09-08T15:18:47.455 INFO:teuthology.orchestra.run.smithi166.stdout:Get:16 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [8,512 B] 2020-09-08T15:18:47.455 INFO:teuthology.orchestra.run.smithi166.stdout:Get:17 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [2,908 B] 2020-09-08T15:18:47.483 INFO:teuthology.orchestra.run.smithi166.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [96.4 kB] 2020-09-08T15:18:47.485 INFO:teuthology.orchestra.run.smithi166.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [20.9 kB] 2020-09-08T15:18:47.487 INFO:teuthology.orchestra.run.smithi166.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,028 kB] 2020-09-08T15:18:47.548 INFO:teuthology.orchestra.run.smithi166.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,107 kB] 2020-09-08T15:18:47.578 INFO:teuthology.orchestra.run.smithi166.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [344 kB] 2020-09-08T15:18:47.587 INFO:teuthology.orchestra.run.smithi166.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [19.6 kB] 2020-09-08T15:18:47.587 INFO:teuthology.orchestra.run.smithi166.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6,768 B] 2020-09-08T15:18:48.510 INFO:teuthology.orchestra.run.smithi166.stdout:Fetched 8,378 kB in 2s (3,908 kB/s) 2020-09-08T15:18:49.856 INFO:teuthology.orchestra.run.smithi166.stdout:Reading package lists... 2020-09-08T15:18:49.903 INFO:teuthology.orchestra.run.smithi166:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-09-08T15:18:49.983 INFO:teuthology.orchestra.run.smithi166.stdout:Reading package lists... 2020-09-08T15:18:50.216 INFO:teuthology.orchestra.run.smithi166.stdout:Building dependency tree... 2020-09-08T15:18:50.216 INFO:teuthology.orchestra.run.smithi166.stdout:Reading state information... 2020-09-08T15:18:50.293 INFO:teuthology.orchestra.run.smithi166.stderr:E: Unable to locate package linux-image-current-generic 2020-09-08T15:18:50.294 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-09-08T15:18:50.295 INFO:teuthology.orchestra.run.smithi166:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-09-08T15:18:50.374 INFO:teuthology.orchestra.run.smithi166.stdout:Reading package lists... 2020-09-08T15:18:50.630 INFO:teuthology.orchestra.run.smithi166.stdout:Building dependency tree... 2020-09-08T15:18:50.630 INFO:teuthology.orchestra.run.smithi166.stdout:Reading state information... 2020-09-08T15:18:50.807 INFO:teuthology.orchestra.run.smithi166.stdout:The following additional packages will be installed: 2020-09-08T15:18:50.807 INFO:teuthology.orchestra.run.smithi166.stdout: linux-generic linux-headers-4.15.0-117 linux-headers-4.15.0-117-generic 2020-09-08T15:18:50.807 INFO:teuthology.orchestra.run.smithi166.stdout: linux-headers-generic linux-image-4.15.0-117-generic 2020-09-08T15:18:50.807 INFO:teuthology.orchestra.run.smithi166.stdout: linux-modules-4.15.0-117-generic linux-modules-extra-4.15.0-117-generic 2020-09-08T15:18:50.809 INFO:teuthology.orchestra.run.smithi166.stdout:Suggested packages: 2020-09-08T15:18:50.809 INFO:teuthology.orchestra.run.smithi166.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2020-09-08T15:18:50.809 INFO:teuthology.orchestra.run.smithi166.stdout:Recommended packages: 2020-09-08T15:18:50.809 INFO:teuthology.orchestra.run.smithi166.stdout: thermald 2020-09-08T15:18:50.841 INFO:teuthology.orchestra.run.smithi166.stdout:The following NEW packages will be installed: 2020-09-08T15:18:50.841 INFO:teuthology.orchestra.run.smithi166.stdout: linux-headers-4.15.0-117 linux-headers-4.15.0-117-generic 2020-09-08T15:18:50.842 INFO:teuthology.orchestra.run.smithi166.stdout: linux-image-4.15.0-117-generic linux-modules-4.15.0-117-generic 2020-09-08T15:18:50.842 INFO:teuthology.orchestra.run.smithi166.stdout: linux-modules-extra-4.15.0-117-generic 2020-09-08T15:18:50.843 INFO:teuthology.orchestra.run.smithi166.stdout:The following packages will be upgraded: 2020-09-08T15:18:50.843 INFO:teuthology.orchestra.run.smithi166.stdout: linux-generic linux-headers-generic linux-image-generic 2020-09-08T15:18:51.038 INFO:teuthology.orchestra.run.smithi166.stdout:3 upgraded, 5 newly installed, 0 to remove and 11 not upgraded. 2020-09-08T15:18:51.038 INFO:teuthology.orchestra.run.smithi166.stdout:Need to get 65.9 MB of archives. 2020-09-08T15:18:51.038 INFO:teuthology.orchestra.run.smithi166.stdout:After this operation, 333 MB of additional disk space will be used. 2020-09-08T15:18:51.039 INFO:teuthology.orchestra.run.smithi166.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-117-generic amd64 4.15.0-117.118 [13.1 MB] 2020-09-08T15:18:52.128 INFO:teuthology.orchestra.run.smithi166.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-117-generic amd64 4.15.0-117.118 [8,012 kB] 2020-09-08T15:18:52.337 INFO:teuthology.orchestra.run.smithi166.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-117-generic amd64 4.15.0-117.118 [32.8 MB] 2020-09-08T15:18:53.480 INFO:teuthology.orchestra.run.smithi166.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.117.104 [1,872 B] 2020-09-08T15:18:53.481 INFO:teuthology.orchestra.run.smithi166.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.117.104 [2,676 B] 2020-09-08T15:18:53.481 INFO:teuthology.orchestra.run.smithi166.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-117 all 4.15.0-117.118 [10.9 MB] 2020-09-08T15:18:53.828 INFO:teuthology.orchestra.run.smithi166.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-117-generic amd64 4.15.0-117.118 [1,100 kB] 2020-09-08T15:18:53.866 INFO:teuthology.orchestra.run.smithi166.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.117.104 [2,572 B] 2020-09-08T15:18:54.430 INFO:teuthology.orchestra.run.smithi166.stdout:Fetched 65.9 MB in 3s (22.0 MB/s) 2020-09-08T15:18:54.598 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package linux-modules-4.15.0-117-generic. 2020-09-08T15:18:54.632 INFO:teuthology.orchestra.run.smithi166.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102058 files and directories currently installed.) 2020-09-08T15:18:54.638 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../0-linux-modules-4.15.0-117-generic_4.15.0-117.118_amd64.deb ... 2020-09-08T15:18:54.686 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking linux-modules-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:18:56.289 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package linux-image-4.15.0-117-generic. 2020-09-08T15:18:56.306 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../1-linux-image-4.15.0-117-generic_4.15.0-117.118_amd64.deb ... 2020-09-08T15:18:56.398 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking linux-image-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:18:56.878 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-117-generic. 2020-09-08T15:18:56.894 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-117-generic_4.15.0-117.118_amd64.deb ... 2020-09-08T15:18:56.918 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking linux-modules-extra-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:19:00.639 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../3-linux-generic_4.15.0.117.104_amd64.deb ... 2020-09-08T15:19:00.734 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking linux-generic (4.15.0.117.104) over (4.15.0.115.103) ... 2020-09-08T15:19:01.114 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.117.104_amd64.deb ... 2020-09-08T15:19:01.238 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking linux-image-generic (4.15.0.117.104) over (4.15.0.115.103) ... 2020-09-08T15:19:01.565 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package linux-headers-4.15.0-117. 2020-09-08T15:19:01.578 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../5-linux-headers-4.15.0-117_4.15.0-117.118_all.deb ... 2020-09-08T15:19:01.616 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking linux-headers-4.15.0-117 (4.15.0-117.118) ... 2020-09-08T15:19:06.602 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package linux-headers-4.15.0-117-generic. 2020-09-08T15:19:06.619 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../6-linux-headers-4.15.0-117-generic_4.15.0-117.118_amd64.deb ... 2020-09-08T15:19:06.657 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking linux-headers-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:19:09.098 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.117.104_amd64.deb ... 2020-09-08T15:19:09.228 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking linux-headers-generic (4.15.0.117.104) over (4.15.0.115.103) ... 2020-09-08T15:19:09.551 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up linux-modules-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:19:10.114 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up linux-headers-4.15.0-117 (4.15.0-117.118) ... 2020-09-08T15:19:10.223 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up linux-image-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:19:10.663 INFO:teuthology.orchestra.run.smithi166.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-117-generic 2020-09-08T15:19:10.663 INFO:teuthology.orchestra.run.smithi166.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-117-generic 2020-09-08T15:19:10.800 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up linux-headers-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:19:10.926 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up linux-modules-extra-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:19:11.412 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up linux-image-generic (4.15.0.117.104) ... 2020-09-08T15:19:11.538 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up linux-headers-generic (4.15.0.117.104) ... 2020-09-08T15:19:11.664 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up linux-generic (4.15.0.117.104) ... 2020-09-08T15:19:11.773 INFO:teuthology.orchestra.run.smithi166.stdout:Processing triggers for linux-image-4.15.0-117-generic (4.15.0-117.118) ... 2020-09-08T15:19:11.953 INFO:teuthology.orchestra.run.smithi166.stdout:/etc/kernel/postinst.d/initramfs-tools: 2020-09-08T15:19:11.954 INFO:teuthology.orchestra.run.smithi166.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-117-generic 2020-09-08T15:19:26.851 INFO:teuthology.orchestra.run.smithi166.stdout:/etc/kernel/postinst.d/zz-update-grub: 2020-09-08T15:19:26.851 INFO:teuthology.orchestra.run.smithi166.stdout:Sourcing file `/etc/default/grub' 2020-09-08T15:19:26.892 INFO:teuthology.orchestra.run.smithi166.stdout:Generating grub configuration file ... 2020-09-08T15:19:27.350 INFO:teuthology.orchestra.run.smithi166.stdout:Found linux image: /boot/vmlinuz-4.15.0-117-generic 2020-09-08T15:19:27.377 INFO:teuthology.orchestra.run.smithi166.stdout:Found initrd image: /boot/initrd.img-4.15.0-117-generic 2020-09-08T15:19:27.716 INFO:teuthology.orchestra.run.smithi166.stdout:Found linux image: /boot/vmlinuz-4.15.0-115-generic 2020-09-08T15:19:27.758 INFO:teuthology.orchestra.run.smithi166.stdout:Found initrd image: /boot/initrd.img-4.15.0-115-generic 2020-09-08T15:19:28.637 INFO:teuthology.orchestra.run.smithi166.stdout:done 2020-09-08T15:19:29.035 INFO:teuthology.orchestra.run.smithi166:> dpkg -s linux-image-generic 2020-09-08T15:19:29.065 INFO:teuthology.orchestra.run.smithi166.stdout:Package: linux-image-generic 2020-09-08T15:19:29.066 INFO:teuthology.orchestra.run.smithi166.stdout:Status: install ok installed 2020-09-08T15:19:29.066 INFO:teuthology.orchestra.run.smithi166.stdout:Priority: optional 2020-09-08T15:19:29.066 INFO:teuthology.orchestra.run.smithi166.stdout:Section: kernel 2020-09-08T15:19:29.067 INFO:teuthology.orchestra.run.smithi166.stdout:Installed-Size: 16 2020-09-08T15:19:29.067 INFO:teuthology.orchestra.run.smithi166.stdout:Maintainer: Ubuntu Kernel Team 2020-09-08T15:19:29.067 INFO:teuthology.orchestra.run.smithi166.stdout:Architecture: amd64 2020-09-08T15:19:29.067 INFO:teuthology.orchestra.run.smithi166.stdout:Source: linux-meta 2020-09-08T15:19:29.067 INFO:teuthology.orchestra.run.smithi166.stdout:Version: 4.15.0.117.104 2020-09-08T15:19:29.068 INFO:teuthology.orchestra.run.smithi166.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.9) 2020-09-08T15:19:29.068 INFO:teuthology.orchestra.run.smithi166.stdout:Depends: linux-image-4.15.0-117-generic, linux-modules-extra-4.15.0-117-generic, linux-firmware, intel-microcode, amd64-microcode 2020-09-08T15:19:29.068 INFO:teuthology.orchestra.run.smithi166.stdout:Recommends: thermald 2020-09-08T15:19:29.068 INFO:teuthology.orchestra.run.smithi166.stdout:Description: Generic Linux kernel image 2020-09-08T15:19:29.068 INFO:teuthology.orchestra.run.smithi166.stdout: This package will always depend on the latest generic kernel image 2020-09-08T15:19:29.069 INFO:teuthology.orchestra.run.smithi166.stdout: available. 2020-09-08T15:19:29.069 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-115-generic Expected: 4.15.0-117-generic 2020-09-08T15:19:29.070 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2020-09-08T15:19:29.070 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-09-08T15:19:29.070 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-09-08T15:19:29.070 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-09-08T15:19:29.071 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2020-09-08T15:19:29.071 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi104.front.sepia.ceph.com, path=None, version=distro) 2020-09-08T15:19:29.072 INFO:teuthology.orchestra.run.smithi104:> sudo apt-get clean 2020-09-08T15:19:29.363 INFO:teuthology.orchestra.run.smithi104:> sudo apt-get update 2020-09-08T15:19:29.627 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-09-08T15:19:29.628 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-09-08T15:19:29.706 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-09-08T15:19:29.795 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-09-08T15:19:31.456 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2020-09-08T15:19:31.492 INFO:teuthology.orchestra.run.smithi104:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-09-08T15:19:31.573 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2020-09-08T15:19:31.802 INFO:teuthology.orchestra.run.smithi104.stdout:Building dependency tree... 2020-09-08T15:19:31.803 INFO:teuthology.orchestra.run.smithi104.stdout:Reading state information... 2020-09-08T15:19:31.880 INFO:teuthology.orchestra.run.smithi104.stderr:E: Unable to locate package linux-image-current-generic 2020-09-08T15:19:31.882 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-09-08T15:19:31.883 INFO:teuthology.orchestra.run.smithi104:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-09-08T15:19:31.963 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2020-09-08T15:19:32.186 INFO:teuthology.orchestra.run.smithi104.stdout:Building dependency tree... 2020-09-08T15:19:32.186 INFO:teuthology.orchestra.run.smithi104.stdout:Reading state information... 2020-09-08T15:19:32.376 INFO:teuthology.orchestra.run.smithi104.stdout:linux-image-generic is already the newest version (4.15.0.117.104). 2020-09-08T15:19:32.376 INFO:teuthology.orchestra.run.smithi104.stdout:0 upgraded, 0 newly installed, 0 to remove and 11 not upgraded. 2020-09-08T15:19:32.378 INFO:teuthology.orchestra.run.smithi104:> dpkg -s linux-image-generic 2020-09-08T15:19:32.403 INFO:teuthology.orchestra.run.smithi104.stdout:Package: linux-image-generic 2020-09-08T15:19:32.403 INFO:teuthology.orchestra.run.smithi104.stdout:Status: install ok installed 2020-09-08T15:19:32.404 INFO:teuthology.orchestra.run.smithi104.stdout:Priority: optional 2020-09-08T15:19:32.404 INFO:teuthology.orchestra.run.smithi104.stdout:Section: kernel 2020-09-08T15:19:32.404 INFO:teuthology.orchestra.run.smithi104.stdout:Installed-Size: 16 2020-09-08T15:19:32.404 INFO:teuthology.orchestra.run.smithi104.stdout:Maintainer: Ubuntu Kernel Team 2020-09-08T15:19:32.404 INFO:teuthology.orchestra.run.smithi104.stdout:Architecture: amd64 2020-09-08T15:19:32.405 INFO:teuthology.orchestra.run.smithi104.stdout:Source: linux-meta 2020-09-08T15:19:32.405 INFO:teuthology.orchestra.run.smithi104.stdout:Version: 4.15.0.117.104 2020-09-08T15:19:32.405 INFO:teuthology.orchestra.run.smithi104.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.9) 2020-09-08T15:19:32.405 INFO:teuthology.orchestra.run.smithi104.stdout:Depends: linux-image-4.15.0-117-generic, linux-modules-extra-4.15.0-117-generic, linux-firmware, intel-microcode, amd64-microcode 2020-09-08T15:19:32.405 INFO:teuthology.orchestra.run.smithi104.stdout:Recommends: thermald 2020-09-08T15:19:32.409 INFO:teuthology.orchestra.run.smithi104.stdout:Description: Generic Linux kernel image 2020-09-08T15:19:32.409 INFO:teuthology.orchestra.run.smithi104.stdout: This package will always depend on the latest generic kernel image 2020-09-08T15:19:32.409 INFO:teuthology.orchestra.run.smithi104.stdout: available. 2020-09-08T15:19:32.410 INFO:teuthology.orchestra.run.smithi104:> mktemp 2020-09-08T15:19:32.451 INFO:teuthology.orchestra.run.smithi104.stdout:/tmp/tmp.bfklv66SBB 2020-09-08T15:19:32.452 INFO:teuthology.orchestra.run.smithi104:> sudo cp /boot/grub/grub.cfg /tmp/tmp.bfklv66SBB 2020-09-08T15:19:32.510 INFO:teuthology.orchestra.run.smithi104:> sudo chmod 0666 /tmp/tmp.bfklv66SBB 2020-09-08T15:19:32.625 DEBUG:teuthology.orchestra.remote:smithi104:/tmp/tmp.bfklv66SBB is 8KB 2020-09-08T15:19:32.679 INFO:teuthology.orchestra.run.smithi104:> rm -fr /tmp/tmp.bfklv66SBB 2020-09-08T15:19:32.685 INFO:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2020-09-08T15:19:32.740 INFO:teuthology.orchestra.run.smithi104:> set -ex 2020-09-08T15:19:32.740 INFO:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/grub.d/01_ceph_kernel 2020-09-08T15:19:32.741 INFO:teuthology.orchestra.run.smithi104:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-09-08T15:19:32.786 INFO:teuthology.orchestra.run.smithi104.stderr:+ sudo dd of=/etc/grub.d/01_ceph_kernel 2020-09-08T15:19:32.804 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records in 2020-09-08T15:19:32.805 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records out 2020-09-08T15:19:32.805 INFO:teuthology.orchestra.run.smithi104.stderr:93 bytes copied, 3.1141e-05 s, 3.0 MB/s 2020-09-08T15:19:32.805 INFO:teuthology.orchestra.run.smithi104.stderr:+ sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-09-08T15:19:32.814 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-117-generic 2020-09-08T15:19:32.815 INFO:teuthology.orchestra.run.smithi104:> sudo update-grub 2020-09-08T15:19:33.995 INFO:teuthology.orchestra.run.smithi104.stderr:Sourcing file `/etc/default/grub' 2020-09-08T15:19:34.016 INFO:teuthology.orchestra.run.smithi104.stderr:Generating grub configuration file ... 2020-09-08T15:19:34.268 INFO:teuthology.orchestra.run.smithi104.stderr:Found linux image: /boot/vmlinuz-4.15.0-117-generic 2020-09-08T15:19:34.276 INFO:teuthology.orchestra.run.smithi104.stderr:Found initrd image: /boot/initrd.img-4.15.0-117-generic 2020-09-08T15:19:34.587 INFO:teuthology.orchestra.run.smithi104.stderr:Found linux image: /boot/vmlinuz-4.15.0-115-generic 2020-09-08T15:19:34.602 INFO:teuthology.orchestra.run.smithi104.stderr:Found initrd image: /boot/initrd.img-4.15.0-115-generic 2020-09-08T15:19:35.890 INFO:teuthology.orchestra.run.smithi104.stderr:done 2020-09-08T15:19:35.891 INFO:teuthology.orchestra.run.smithi104:> sudo shutdown -r now 2020-09-08T15:19:35.899 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2020-09-08T15:19:35.900 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi055.front.sepia.ceph.com, path=None, version=distro) 2020-09-08T15:19:35.900 INFO:teuthology.orchestra.run.smithi055:> sudo apt-get clean 2020-09-08T15:19:35.944 INFO:teuthology.orchestra.run.smithi055:> sudo apt-get update 2020-09-08T15:19:36.249 INFO:teuthology.orchestra.run.smithi055.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-09-08T15:19:36.250 INFO:teuthology.orchestra.run.smithi055.stdout:Hit:2 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-09-08T15:19:36.325 INFO:teuthology.orchestra.run.smithi055.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-09-08T15:19:36.414 INFO:teuthology.orchestra.run.smithi055.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-09-08T15:19:38.154 INFO:teuthology.orchestra.run.smithi055.stdout:Reading package lists... 2020-09-08T15:19:38.203 INFO:teuthology.orchestra.run.smithi055:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-09-08T15:19:38.288 INFO:teuthology.orchestra.run.smithi055.stdout:Reading package lists... 2020-09-08T15:19:38.519 INFO:teuthology.orchestra.run.smithi055.stdout:Building dependency tree... 2020-09-08T15:19:38.520 INFO:teuthology.orchestra.run.smithi055.stdout:Reading state information... 2020-09-08T15:19:38.612 INFO:teuthology.orchestra.run.smithi055.stderr:E: Unable to locate package linux-image-current-generic 2020-09-08T15:19:38.613 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-09-08T15:19:38.614 INFO:teuthology.orchestra.run.smithi055:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-09-08T15:19:38.697 INFO:teuthology.orchestra.run.smithi055.stdout:Reading package lists... 2020-09-08T15:19:38.957 INFO:teuthology.orchestra.run.smithi055.stdout:Building dependency tree... 2020-09-08T15:19:38.958 INFO:teuthology.orchestra.run.smithi055.stdout:Reading state information... 2020-09-08T15:19:39.177 INFO:teuthology.orchestra.run.smithi055.stdout:linux-image-generic is already the newest version (4.15.0.117.104). 2020-09-08T15:19:39.177 INFO:teuthology.orchestra.run.smithi055.stdout:0 upgraded, 0 newly installed, 0 to remove and 11 not upgraded. 2020-09-08T15:19:39.178 INFO:teuthology.orchestra.run.smithi055:> dpkg -s linux-image-generic 2020-09-08T15:19:39.215 INFO:teuthology.orchestra.run.smithi055.stdout:Package: linux-image-generic 2020-09-08T15:19:39.215 INFO:teuthology.orchestra.run.smithi055.stdout:Status: install ok installed 2020-09-08T15:19:39.216 INFO:teuthology.orchestra.run.smithi055.stdout:Priority: optional 2020-09-08T15:19:39.216 INFO:teuthology.orchestra.run.smithi055.stdout:Section: kernel 2020-09-08T15:19:39.216 INFO:teuthology.orchestra.run.smithi055.stdout:Installed-Size: 16 2020-09-08T15:19:39.216 INFO:teuthology.orchestra.run.smithi055.stdout:Maintainer: Ubuntu Kernel Team 2020-09-08T15:19:39.217 INFO:teuthology.orchestra.run.smithi055.stdout:Architecture: amd64 2020-09-08T15:19:39.217 INFO:teuthology.orchestra.run.smithi055.stdout:Source: linux-meta 2020-09-08T15:19:39.217 INFO:teuthology.orchestra.run.smithi055.stdout:Version: 4.15.0.117.104 2020-09-08T15:19:39.217 INFO:teuthology.orchestra.run.smithi055.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.9) 2020-09-08T15:19:39.217 INFO:teuthology.orchestra.run.smithi055.stdout:Depends: linux-image-4.15.0-117-generic, linux-modules-extra-4.15.0-117-generic, linux-firmware, intel-microcode, amd64-microcode 2020-09-08T15:19:39.218 INFO:teuthology.orchestra.run.smithi055.stdout:Recommends: thermald 2020-09-08T15:19:39.218 INFO:teuthology.orchestra.run.smithi055.stdout:Description: Generic Linux kernel image 2020-09-08T15:19:39.218 INFO:teuthology.orchestra.run.smithi055.stdout: This package will always depend on the latest generic kernel image 2020-09-08T15:19:39.219 INFO:teuthology.orchestra.run.smithi055.stdout: available. 2020-09-08T15:19:39.219 INFO:teuthology.orchestra.run.smithi055:> mktemp 2020-09-08T15:19:39.263 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.PjFpOKDJEt 2020-09-08T15:19:39.265 INFO:teuthology.orchestra.run.smithi055:> sudo cp /boot/grub/grub.cfg /tmp/tmp.PjFpOKDJEt 2020-09-08T15:19:39.321 INFO:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.PjFpOKDJEt 2020-09-08T15:19:39.440 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.PjFpOKDJEt is 8KB 2020-09-08T15:19:39.495 INFO:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.PjFpOKDJEt 2020-09-08T15:19:39.503 INFO:teuthology.orchestra.run.smithi055:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2020-09-08T15:19:39.560 INFO:teuthology.orchestra.run.smithi055:> set -ex 2020-09-08T15:19:39.561 INFO:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/grub.d/01_ceph_kernel 2020-09-08T15:19:39.561 INFO:teuthology.orchestra.run.smithi055:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-09-08T15:19:39.618 INFO:teuthology.orchestra.run.smithi055.stderr:+ sudo dd of=/etc/grub.d/01_ceph_kernel 2020-09-08T15:19:39.626 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records in 2020-09-08T15:19:39.626 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records out 2020-09-08T15:19:39.627 INFO:teuthology.orchestra.run.smithi055.stderr:93 bytes copied, 3.5555e-05 s, 2.6 MB/s 2020-09-08T15:19:39.627 INFO:teuthology.orchestra.run.smithi055.stderr:+ sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-09-08T15:19:39.638 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-117-generic 2020-09-08T15:19:39.638 INFO:teuthology.orchestra.run.smithi055:> sudo update-grub 2020-09-08T15:19:40.900 INFO:teuthology.orchestra.run.smithi055.stderr:Sourcing file `/etc/default/grub' 2020-09-08T15:19:40.917 INFO:teuthology.orchestra.run.smithi055.stderr:Generating grub configuration file ... 2020-09-08T15:19:41.131 INFO:teuthology.orchestra.run.smithi055.stderr:Found linux image: /boot/vmlinuz-4.15.0-117-generic 2020-09-08T15:19:41.142 INFO:teuthology.orchestra.run.smithi055.stderr:Found initrd image: /boot/initrd.img-4.15.0-117-generic 2020-09-08T15:19:41.453 INFO:teuthology.orchestra.run.smithi055.stderr:Found linux image: /boot/vmlinuz-4.15.0-115-generic 2020-09-08T15:19:41.459 INFO:teuthology.orchestra.run.smithi055.stderr:Found initrd image: /boot/initrd.img-4.15.0-115-generic 2020-09-08T15:19:42.116 INFO:teuthology.orchestra.run.smithi055.stderr:done 2020-09-08T15:19:42.120 INFO:teuthology.orchestra.run.smithi055:> sudo shutdown -r now 2020-09-08T15:19:42.131 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2020-09-08T15:19:42.131 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi166.front.sepia.ceph.com, path=None, version=distro) 2020-09-08T15:19:42.132 INFO:teuthology.orchestra.run.smithi166:> sudo apt-get clean 2020-09-08T15:19:42.226 INFO:teuthology.orchestra.run.smithi166:> sudo apt-get update 2020-09-08T15:19:42.383 INFO:teuthology.orchestra.run.smithi166.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-09-08T15:19:42.483 INFO:teuthology.orchestra.run.smithi166.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-09-08T15:19:42.571 INFO:teuthology.orchestra.run.smithi166.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-09-08T15:19:42.667 INFO:teuthology.orchestra.run.smithi166.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-09-08T15:19:44.224 INFO:teuthology.orchestra.run.smithi166.stdout:Reading package lists... 2020-09-08T15:19:44.268 INFO:teuthology.orchestra.run.smithi166:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-09-08T15:19:44.370 INFO:teuthology.orchestra.run.smithi166.stdout:Reading package lists... 2020-09-08T15:19:44.619 INFO:teuthology.orchestra.run.smithi166.stdout:Building dependency tree... 2020-09-08T15:19:44.619 INFO:teuthology.orchestra.run.smithi166.stdout:Reading state information... 2020-09-08T15:19:44.710 INFO:teuthology.orchestra.run.smithi166.stderr:E: Unable to locate package linux-image-current-generic 2020-09-08T15:19:44.711 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-09-08T15:19:44.712 INFO:teuthology.orchestra.run.smithi166:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-09-08T15:19:44.833 INFO:teuthology.orchestra.run.smithi166.stdout:Reading package lists... 2020-09-08T15:19:45.079 INFO:teuthology.orchestra.run.smithi166.stdout:Building dependency tree... 2020-09-08T15:19:45.080 INFO:teuthology.orchestra.run.smithi166.stdout:Reading state information... 2020-09-08T15:19:45.285 INFO:teuthology.orchestra.run.smithi166.stdout:linux-image-generic is already the newest version (4.15.0.117.104). 2020-09-08T15:19:45.285 INFO:teuthology.orchestra.run.smithi166.stdout:0 upgraded, 0 newly installed, 0 to remove and 11 not upgraded. 2020-09-08T15:19:45.287 INFO:teuthology.orchestra.run.smithi166:> dpkg -s linux-image-generic 2020-09-08T15:19:45.304 INFO:teuthology.orchestra.run.smithi166.stdout:Package: linux-image-generic 2020-09-08T15:19:45.305 INFO:teuthology.orchestra.run.smithi166.stdout:Status: install ok installed 2020-09-08T15:19:45.305 INFO:teuthology.orchestra.run.smithi166.stdout:Priority: optional 2020-09-08T15:19:45.305 INFO:teuthology.orchestra.run.smithi166.stdout:Section: kernel 2020-09-08T15:19:45.306 INFO:teuthology.orchestra.run.smithi166.stdout:Installed-Size: 16 2020-09-08T15:19:45.306 INFO:teuthology.orchestra.run.smithi166.stdout:Maintainer: Ubuntu Kernel Team 2020-09-08T15:19:45.306 INFO:teuthology.orchestra.run.smithi166.stdout:Architecture: amd64 2020-09-08T15:19:45.306 INFO:teuthology.orchestra.run.smithi166.stdout:Source: linux-meta 2020-09-08T15:19:45.306 INFO:teuthology.orchestra.run.smithi166.stdout:Version: 4.15.0.117.104 2020-09-08T15:19:45.307 INFO:teuthology.orchestra.run.smithi166.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.9) 2020-09-08T15:19:45.307 INFO:teuthology.orchestra.run.smithi166.stdout:Depends: linux-image-4.15.0-117-generic, linux-modules-extra-4.15.0-117-generic, linux-firmware, intel-microcode, amd64-microcode 2020-09-08T15:19:45.307 INFO:teuthology.orchestra.run.smithi166.stdout:Recommends: thermald 2020-09-08T15:19:45.307 INFO:teuthology.orchestra.run.smithi166.stdout:Description: Generic Linux kernel image 2020-09-08T15:19:45.307 INFO:teuthology.orchestra.run.smithi166.stdout: This package will always depend on the latest generic kernel image 2020-09-08T15:19:45.308 INFO:teuthology.orchestra.run.smithi166.stdout: available. 2020-09-08T15:19:45.308 INFO:teuthology.orchestra.run.smithi166:> mktemp 2020-09-08T15:19:45.355 INFO:teuthology.orchestra.run.smithi166.stdout:/tmp/tmp.aLc64KY630 2020-09-08T15:19:45.355 INFO:teuthology.orchestra.run.smithi166:> sudo cp /boot/grub/grub.cfg /tmp/tmp.aLc64KY630 2020-09-08T15:19:45.716 INFO:teuthology.orchestra.run.smithi166:> sudo chmod 0666 /tmp/tmp.aLc64KY630 2020-09-08T15:19:45.795 DEBUG:teuthology.orchestra.remote:smithi166:/tmp/tmp.aLc64KY630 is 8KB 2020-09-08T15:19:45.812 INFO:teuthology.orchestra.run.smithi166:> rm -fr /tmp/tmp.aLc64KY630 2020-09-08T15:19:45.820 INFO:teuthology.orchestra.run.smithi166:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2020-09-08T15:19:45.880 INFO:teuthology.orchestra.run.smithi166:> set -ex 2020-09-08T15:19:45.881 INFO:teuthology.orchestra.run.smithi166:> sudo dd of=/etc/grub.d/01_ceph_kernel 2020-09-08T15:19:45.881 INFO:teuthology.orchestra.run.smithi166:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-09-08T15:19:45.927 INFO:teuthology.orchestra.run.smithi166.stderr:+ sudo dd of=/etc/grub.d/01_ceph_kernel 2020-09-08T15:19:45.946 INFO:teuthology.orchestra.run.smithi166.stderr:0+1 records in 2020-09-08T15:19:45.947 INFO:teuthology.orchestra.run.smithi166.stderr:0+1 records out 2020-09-08T15:19:45.947 INFO:teuthology.orchestra.run.smithi166.stderr:93 bytes copied, 3.053e-05 s, 3.0 MB/s 2020-09-08T15:19:45.948 INFO:teuthology.orchestra.run.smithi166.stderr:+ sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-09-08T15:19:45.957 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-117-generic 2020-09-08T15:19:45.957 INFO:teuthology.orchestra.run.smithi166:> sudo update-grub 2020-09-08T15:19:47.321 INFO:teuthology.orchestra.run.smithi166.stderr:Sourcing file `/etc/default/grub' 2020-09-08T15:19:47.340 INFO:teuthology.orchestra.run.smithi166.stderr:Generating grub configuration file ... 2020-09-08T15:19:47.598 INFO:teuthology.orchestra.run.smithi166.stderr:Found linux image: /boot/vmlinuz-4.15.0-117-generic 2020-09-08T15:19:47.605 INFO:teuthology.orchestra.run.smithi166.stderr:Found initrd image: /boot/initrd.img-4.15.0-117-generic 2020-09-08T15:19:47.908 INFO:teuthology.orchestra.run.smithi166.stderr:Found linux image: /boot/vmlinuz-4.15.0-115-generic 2020-09-08T15:19:47.920 INFO:teuthology.orchestra.run.smithi166.stderr:Found initrd image: /boot/initrd.img-4.15.0-115-generic 2020-09-08T15:19:48.472 INFO:teuthology.orchestra.run.smithi166.stderr:done 2020-09-08T15:19:48.474 INFO:teuthology.orchestra.run.smithi166:> sudo shutdown -r now 2020-09-08T15:20:18.478 INFO:teuthology.misc:Re-opening connections... 2020-09-08T15:20:18.480 INFO:teuthology.misc:trying to connect to ubuntu@smithi104.front.sepia.ceph.com 2020-09-08T15:20:18.480 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-09-08T15:20:18.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:20:36.830 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.104 2020-09-08T15:20:36.832 INFO:teuthology.misc:trying to connect to ubuntu@smithi055.front.sepia.ceph.com 2020-09-08T15:20:36.832 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-09-08T15:20:36.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:21:08.319 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.55 2020-09-08T15:21:08.321 INFO:teuthology.misc:trying to connect to ubuntu@smithi166.front.sepia.ceph.com 2020-09-08T15:21:08.321 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-09-08T15:21:08.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi166.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:21:26.751 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.166 2020-09-08T15:21:26.752 DEBUG:teuthology.misc:waited 68.27283048629761 2020-09-08T15:21:27.754 INFO:teuthology.misc:trying to connect to ubuntu@smithi104.front.sepia.ceph.com 2020-09-08T15:21:27.755 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-09-08T15:21:27.756 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:21:27.837 INFO:teuthology.orchestra.run.smithi104:> true 2020-09-08T15:21:28.215 INFO:teuthology.misc:trying to connect to ubuntu@smithi166.front.sepia.ceph.com 2020-09-08T15:21:28.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-09-08T15:21:28.217 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi166.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:21:30.973 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.166 2020-09-08T15:21:30.974 DEBUG:teuthology.misc:waited 72.49473929405212 2020-09-08T15:21:31.976 INFO:teuthology.misc:trying to connect to ubuntu@smithi055.front.sepia.ceph.com 2020-09-08T15:21:31.976 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-09-08T15:21:31.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:21:32.137 INFO:teuthology.orchestra.run.smithi055:> true 2020-09-08T15:21:32.567 DEBUG:teuthology.misc:waited 74.08743643760681 2020-09-08T15:21:33.568 INFO:teuthology.misc:trying to connect to ubuntu@smithi166.front.sepia.ceph.com 2020-09-08T15:21:33.568 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-09-08T15:21:33.569 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi166.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:21:33.674 INFO:teuthology.orchestra.run.smithi166:> true 2020-09-08T15:21:33.983 DEBUG:teuthology.misc:waited 75.50389409065247 2020-09-08T15:21:34.986 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2020-09-08T15:21:34.987 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.15.0-117-generic"... 2020-09-08T15:21:34.988 INFO:teuthology.orchestra.run.smithi104:> uname -r 2020-09-08T15:21:34.994 INFO:teuthology.orchestra.run.smithi104.stdout:4.15.0-117-generic 2020-09-08T15:21:34.995 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-117-generic vs 4.15.0-117-generic 2020-09-08T15:21:34.995 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-09-08T15:21:34.996 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2020-09-08T15:21:34.996 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2020-09-08T15:21:34.996 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.15.0-117-generic"... 2020-09-08T15:21:34.997 INFO:teuthology.orchestra.run.smithi055:> uname -r 2020-09-08T15:21:35.016 INFO:teuthology.orchestra.run.smithi055.stdout:4.15.0-117-generic 2020-09-08T15:21:35.017 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-117-generic vs 4.15.0-117-generic 2020-09-08T15:21:35.017 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-09-08T15:21:35.019 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2020-09-08T15:21:35.019 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2020-09-08T15:21:35.020 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "4.15.0-117-generic"... 2020-09-08T15:21:35.020 INFO:teuthology.orchestra.run.smithi166:> uname -r 2020-09-08T15:21:35.027 INFO:teuthology.orchestra.run.smithi166.stdout:4.15.0-117-generic 2020-09-08T15:21:35.027 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-117-generic vs 4.15.0-117-generic 2020-09-08T15:21:35.027 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-09-08T15:21:35.028 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2020-09-08T15:21:36.029 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2020-09-08T15:21:36.030 INFO:teuthology.orchestra.run.smithi104:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-09-08T15:21:36.063 INFO:teuthology.orchestra.run.smithi104.stdout:ttyS1 2020-09-08T15:21:36.087 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2020-09-08T15:21:36.087 INFO:teuthology.orchestra.run.smithi055:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-09-08T15:21:36.127 INFO:teuthology.orchestra.run.smithi055.stdout:ttyS1 2020-09-08T15:21:36.146 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2020-09-08T15:21:36.147 INFO:teuthology.orchestra.run.smithi166:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-09-08T15:21:36.195 INFO:teuthology.orchestra.run.smithi166.stdout:ttyS1 2020-09-08T15:21:36.227 INFO:teuthology.run_tasks:Running task internal.base... 2020-09-08T15:21:36.237 INFO:teuthology.task.internal:Creating test directory... 2020-09-08T15:21:36.237 INFO:teuthology.orchestra.run.smithi055:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-09-08T15:21:36.241 INFO:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-09-08T15:21:36.244 INFO:teuthology.orchestra.run.smithi166:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-09-08T15:21:36.251 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-09-08T15:21:36.393 INFO:teuthology.run_tasks:Running task internal.archive... 2020-09-08T15:21:36.455 INFO:teuthology.task.internal:Creating archive directory... 2020-09-08T15:21:36.455 INFO:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-09-08T15:21:36.474 INFO:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-09-08T15:21:36.477 INFO:teuthology.orchestra.run.smithi166:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-09-08T15:21:36.494 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-09-08T15:21:36.501 INFO:teuthology.task.internal:Enabling coredump saving... 2020-09-08T15:21:36.503 INFO:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-09-08T15:21:36.536 INFO:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-09-08T15:21:36.538 INFO:teuthology.orchestra.run.smithi166:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-09-08T15:21:36.550 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-08T15:21:36.557 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-08T15:21:36.561 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-08T15:21:36.564 INFO:teuthology.orchestra.run.smithi166.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-08T15:21:36.569 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-08T15:21:36.571 INFO:teuthology.orchestra.run.smithi166.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-09-08T15:21:36.572 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-09-08T15:21:36.666 INFO:teuthology.task.internal:Configuring sudo... 2020-09-08T15:21:36.666 INFO:teuthology.orchestra.run.smithi055:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-09-08T15:21:36.671 INFO:teuthology.orchestra.run.smithi104:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-09-08T15:21:36.677 INFO:teuthology.orchestra.run.smithi166:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-09-08T15:21:36.693 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-09-08T15:21:36.763 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-09-08T15:21:36.764 INFO:teuthology.orchestra.run.smithi055:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-09-08T15:21:36.767 INFO:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-09-08T15:21:36.770 INFO:teuthology.orchestra.run.smithi166:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-09-08T15:21:36.786 INFO:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-09-08T15:21:36.828 INFO:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-09-08T15:21:36.876 INFO:teuthology.orchestra.run.smithi104:> set -ex 2020-09-08T15:21:36.876 INFO:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-09-08T15:21:36.925 INFO:teuthology.orchestra.run.smithi104.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-09-08T15:21:36.943 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records in 2020-09-08T15:21:36.944 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records out 2020-09-08T15:21:36.944 INFO:teuthology.orchestra.run.smithi104.stderr:152 bytes copied, 3.3515e-05 s, 4.5 MB/s 2020-09-08T15:21:36.945 INFO:teuthology.orchestra.run.smithi055:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-09-08T15:21:36.952 INFO:teuthology.orchestra.run.smithi055:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-09-08T15:21:37.002 INFO:teuthology.orchestra.run.smithi055:> set -ex 2020-09-08T15:21:37.002 INFO:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-09-08T15:21:37.047 INFO:teuthology.orchestra.run.smithi055.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-09-08T15:21:37.061 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records in 2020-09-08T15:21:37.061 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records out 2020-09-08T15:21:37.070 INFO:teuthology.orchestra.run.smithi055.stderr:152 bytes copied, 3.184e-05 s, 4.8 MB/s 2020-09-08T15:21:37.071 INFO:teuthology.orchestra.run.smithi166:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-09-08T15:21:37.080 INFO:teuthology.orchestra.run.smithi166:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-09-08T15:21:37.130 INFO:teuthology.orchestra.run.smithi166:> set -ex 2020-09-08T15:21:37.131 INFO:teuthology.orchestra.run.smithi166:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-09-08T15:21:37.176 INFO:teuthology.orchestra.run.smithi166.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-09-08T15:21:37.198 INFO:teuthology.orchestra.run.smithi166.stderr:0+1 records in 2020-09-08T15:21:37.199 INFO:teuthology.orchestra.run.smithi166.stderr:0+1 records out 2020-09-08T15:21:37.199 INFO:teuthology.orchestra.run.smithi166.stderr:152 bytes copied, 3.1481e-05 s, 4.8 MB/s 2020-09-08T15:21:37.200 INFO:teuthology.orchestra.run.smithi055:> sudo service rsyslog restart 2020-09-08T15:21:37.202 INFO:teuthology.orchestra.run.smithi104:> sudo service rsyslog restart 2020-09-08T15:21:37.209 INFO:teuthology.orchestra.run.smithi166:> sudo service rsyslog restart 2020-09-08T15:21:37.291 INFO:teuthology.run_tasks:Running task internal.timer... 2020-09-08T15:21:37.299 INFO:teuthology.task.internal:Starting timer... 2020-09-08T15:21:37.300 INFO:teuthology.run_tasks:Running task pcp... 2020-09-08T15:21:37.390 INFO:teuthology.run_tasks:Running task selinux... 2020-09-08T15:21:37.490 DEBUG:teuthology.task.selinux:Excluding smithi104: OS 'ubuntu' does not support SELinux 2020-09-08T15:21:37.491 DEBUG:teuthology.task.selinux:Excluding smithi055: OS 'ubuntu' does not support SELinux 2020-09-08T15:21:37.491 DEBUG:teuthology.task.selinux:Excluding smithi166: OS 'ubuntu' does not support SELinux 2020-09-08T15:21:37.491 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-09-08T15:21:37.492 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2020-09-08T15:21:37.492 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-09-08T15:21:37.492 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-09-08T15:21:37.584 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2020-09-08T15:21:37.585 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-09-08T15:21:37.605 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-09-08T15:21:37.608 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi104.front.sepia.ceph.com,smithi055.front.sepia.ceph.com,smithi166.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-09-08T15:21:37.998 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-09-08T15:21:39.078 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-09-08T15:21:41.542 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-09-08T15:21:41.564 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:41 +0000 (0:00:00.084) 0:00:00.084 ***** 2020-09-08T15:21:41.583 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:41 +0000 (0:00:00.018) 0:00:00.102 ***** 2020-09-08T15:21:41.601 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:41 +0000 (0:00:00.018) 0:00:00.120 ***** 2020-09-08T15:21:43.427 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi104.front.sepia.ceph.com] 2020-09-08T15:21:43.436 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] 2020-09-08T15:21:43.447 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] 2020-09-08T15:21:43.589 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:43 +0000 (0:00:01.988) 0:00:02.109 ***** 2020-09-08T15:21:43.628 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:43 +0000 (0:00:00.038) 0:00:02.147 ***** 2020-09-08T15:21:43.667 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:43 +0000 (0:00:00.039) 0:00:02.186 ***** 2020-09-08T15:21:43.980 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2020-09-08T15:21:43.982 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-09-08T15:21:44.022 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:44 +0000 (0:00:00.353) 0:00:02.540 ***** 2020-09-08T15:21:44.034 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-09-08T15:21:44.036 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-09-08T15:21:44.074 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:44 +0000 (0:00:00.053) 0:00:02.594 ***** 2020-09-08T15:21:44.113 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:44 +0000 (0:00:00.038) 0:00:02.632 ***** 2020-09-08T15:21:44.630 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2020-09-08T15:21:44.631 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-09-08T15:21:44.692 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:44 +0000 (0:00:00.579) 0:00:03.212 ***** 2020-09-08T15:21:44.706 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-09-08T15:21:44.707 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-09-08T15:21:44.756 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:44 +0000 (0:00:00.063) 0:00:03.275 ***** 2020-09-08T15:21:44.815 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:44 +0000 (0:00:00.044) 0:00:03.319 ***** 2020-09-08T15:21:45.190 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** changed: [smithi166.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.063941", "end": "2020-09-08 15:21:44.886071", "rc": 0, "start": "2020-09-08 15:21:44.822130", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2020-09-08T15:21:45.193 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.074946", "end": "2020-09-08 15:21:45.591113", "rc": 0, "start": "2020-09-08 15:21:45.516167", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} changed: [smithi055.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.076145", "end": "2020-09-08 15:21:44.756417", "rc": 0, "start": "2020-09-08 15:21:44.680272", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2020-09-08T15:21:45.226 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:45 +0000 (0:00:00.425) 0:00:03.745 ***** 2020-09-08T15:21:45.266 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:45 +0000 (0:00:00.038) 0:00:03.783 ***** 2020-09-08T15:21:45.304 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:45 +0000 (0:00:00.038) 0:00:03.822 ***** 2020-09-08T15:21:45.546 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** ok: [smithi166.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} ok: [smithi055.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} ok: [smithi104.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-09-08T15:21:45.591 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:45 +0000 (0:00:00.289) 0:00:04.111 ***** 2020-09-08T15:21:45.635 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:45 +0000 (0:00:00.042) 0:00:04.154 ***** 2020-09-08T15:21:45.671 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:45 +0000 (0:00:00.036) 0:00:04.190 ***** 2020-09-08T15:21:46.375 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2020-09-08T15:21:46.378 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1599578505.6524012-23642282106307/source", "state": "file", "uid": 0} changed: [smithi166.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1599578505.6906831-117405961151642/source", "state": "file", "uid": 0} 2020-09-08T15:21:46.415 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:46 +0000 (0:00:00.743) 0:00:04.934 ***** 2020-09-08T15:21:46.452 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:46 +0000 (0:00:00.037) 0:00:04.972 ***** 2020-09-08T15:21:46.468 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1599578505.615239-213975793234550/source", "state": "file", "uid": 0} 2020-09-08T15:21:46.502 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:46 +0000 (0:00:00.050) 0:00:05.022 ***** 2020-09-08T15:21:46.765 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2020-09-08T15:21:46.769 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-09-08T15:21:46.772 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail 2020-09-08T15:21:46.778 INFO:teuthology.task.ansible.out:.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@ 2020-09-08T15:21:46.780 INFO:teuthology.task.ansible.out:aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-09-08T15:21:46.850 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-09-08T15:21:46.852 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-09-08T15:21:46.994 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2020-09-08T15:21:47.001 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2020-09-08T15:21:47.147 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:47 +0000 (0:00:00.644) 0:00:05.666 ***** 2020-09-08T15:21:47.196 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:47 +0000 (0:00:00.049) 0:00:05.715 ***** 2020-09-08T15:21:47.242 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:47 +0000 (0:00:00.041) 0:00:05.757 ***** 2020-09-08T15:21:47.957 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2020-09-08T15:21:47.959 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] 2020-09-08T15:21:47.968 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] 2020-09-08T15:21:48.082 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2020-09-08T15:21:48.084 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:48 +0000 (0:00:00.844) 0:00:06.602 ***** 2020-09-08T15:21:48.126 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:48 +0000 (0:00:00.043) 0:00:06.645 ***** 2020-09-08T15:21:48.714 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** changed: [smithi055.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.278904", "end": "2020-09-08 15:21:48.612978", "rc": 0, "start": "2020-09-08 15:21:48.334074", "stderr": "", "stderr_lines": [], "stdout": "d7c0190f3c79f0107823a9b3caef24ede18680e0", "stdout_lines": ["d7c0190f3c79f0107823a9b3caef24ede18680e0"]} 2020-09-08T15:21:48.749 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:48 +0000 (0:00:00.623) 0:00:07.268 ***** 2020-09-08T15:21:48.760 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.277552", "end": "2020-09-08 15:21:48.627799", "rc": 0, "start": "2020-09-08 15:21:48.350247", "stderr": "", "stderr_lines": [], "stdout": "d7c0190f3c79f0107823a9b3caef24ede18680e0", "stdout_lines": ["d7c0190f3c79f0107823a9b3caef24ede18680e0"]} 2020-09-08T15:21:48.802 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:48 +0000 (0:00:00.052) 0:00:07.321 ***** 2020-09-08T15:21:48.920 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-09-08T15:21:48.922 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.014211", "end": "2020-09-08 15:21:48.484380", "failed_when_result": false, "rc": 0, "start": "2020-09-08 15:21:48.470169", "stderr": "", "stderr_lines": [], "stdout": "d7c0190f3c79f0107823a9b3caef24ede18680e0", "stdout_lines": ["d7c0190f3c79f0107823a9b3caef24ede18680e0"]} 2020-09-08T15:21:48.981 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:48 +0000 (0:00:00.179) 0:00:07.500 ***** 2020-09-08T15:21:48.995 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.008186", "end": "2020-09-08 15:21:49.374624", "failed_when_result": false, "rc": 0, "start": "2020-09-08 15:21:49.366438", "stderr": "", "stderr_lines": [], "stdout": "d7c0190f3c79f0107823a9b3caef24ede18680e0", "stdout_lines": ["d7c0190f3c79f0107823a9b3caef24ede18680e0"]} 2020-09-08T15:21:49.042 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:49 +0000 (0:00:00.058) 0:00:07.559 ***** 2020-09-08T15:21:49.057 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi166.front.sepia.ceph.com] 2020-09-08T15:21:49.069 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** ok: [smithi055.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-09-08T15:21:49.097 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:49 +0000 (0:00:00.056) 0:00:07.616 ***** 2020-09-08T15:21:49.135 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-09-08T15:21:49.136 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-09-08T15:21:49.136 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.171 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:49 +0000 (0:00:00.073) 0:00:07.690 ***** 2020-09-08T15:21:49.208 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:49 +0000 (0:00:00.037) 0:00:07.727 ***** 2020-09-08T15:21:49.250 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:49 +0000 (0:00:00.042) 0:00:07.770 ***** 2020-09-08T15:21:49.260 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-09-08T15:21:49.262 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Merge extra_admin_users into managed_admin_users] **************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.303 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:49 +0000 (0:00:00.053) 0:00:07.823 ***** 2020-09-08T15:21:49.345 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:49 +0000 (0:00:00.041) 0:00:07.864 ***** 2020-09-08T15:21:49.354 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-09-08T15:21:49.356 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.407 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:49 +0000 (0:00:00.062) 0:00:07.926 ***** 2020-09-08T15:21:49.418 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-09-08T15:21:49.419 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.463 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:49 +0000 (0:00:00.055) 0:00:07.982 ***** 2020-09-08T15:21:49.479 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2020-09-08T15:21:49.481 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.520 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:49 +0000 (0:00:00.057) 0:00:08.039 ***** 2020-09-08T15:21:49.529 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-09-08T15:21:49.532 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.568 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:49 +0000 (0:00:00.047) 0:00:08.088 ***** 2020-09-08T15:21:49.583 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.624 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:49 +0000 (0:00:00.055) 0:00:08.143 ***** 2020-09-08T15:21:49.634 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2020-09-08T15:21:49.636 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.683 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:49 +0000 (0:00:00.058) 0:00:08.201 ***** 2020-09-08T15:21:49.702 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-09-08T15:21:49.704 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ 2020-09-08T15:21:49.707 INFO:teuthology.task.ansible.out:ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.715 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.717 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} TASK [users : Set UID_MIN to 1001] ********************************************* skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.755 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:49 +0000 (0:00:00.073) 0:00:08.274 ***** 2020-09-08T15:21:49.770 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rae 2020-09-08T15:21:49.778 INFO:teuthology.task.ansible.out:l 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7d 2020-09-08T15:21:49.780 INFO:teuthology.task.ansible.out:acd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.780 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.786 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "ite 2020-09-08T15:21:49.790 INFO:teuthology.task.ansible.out:m", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9e 2020-09-08T15:21:49.792 INFO:teuthology.task.ansible.out:defafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.802 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} TASK [users : Check keys_repo HEAD sha1] *************************************** changed: [smithi166.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.282116", "end": "2020-09-08 15:21:49.763833", "rc": 0, "start": "2020-09-08 15:21:49.481717", "stderr": "", "stderr_lines": [], "stdout": "d7c0190f3c79f0107823a9b3caef24ede18680e0", "stdout_lines": ["d7c0190f3c79f0107823a9b3caef24ede18680e0"]} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31e 2020-09-08T15:21:49.803 INFO:teuthology.task.ansible.out:dc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.849 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:49 +0000 (0:00:00.089) 0:00:08.364 ***** 2020-09-08T15:21:49.862 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.864 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.865 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.866 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": 2020-09-08T15:21:49.870 INFO:teuthology.task.ansible.out:false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 2020-09-08T15:21:49.874 INFO:teuthology.task.ansible.out: 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [sm 2020-09-08T15:21:49.877 INFO:teuthology.task.ansible.out:ithi104.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa11 2020-09-08T15:21:49.884 INFO:teuthology.task.ansible.out:0d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9 2020-09-08T15:21:49.886 INFO:teuthology.task.ansible.out:Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKP 2020-09-08T15:21:49.888 INFO:teuthology.task.ansible.out:JC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.898 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.900 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a3 2020-09-08T15:21:49.902 INFO:teuthology.task.ansible.out:3bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.914 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.916 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.929 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@la 2020-09-08T15:21:49.931 INFO:teuthology.task.ansible.out:ptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.938 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.941 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n' 2020-09-08T15:21:49.942 INFO:teuthology.task.ansible.out:}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.953 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.964 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.978 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.980 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.991 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:49.993 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.004 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.014 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, 2020-09-08T15:21:50.018 INFO:teuthology.task.ansible.out:"item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.019 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea 2020-09-08T15:21:50.021 INFO:teuthology.task.ansible.out: eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.025 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hch 2020-09-08T15:21:50.028 INFO:teuthology.task.ansible.out:en@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [smithi166.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.011721", "end": "2020-09-08 15:21:49.709501", "failed_when_result": false, "rc": 0, "start": "2020-09-08 15:21:49.697780", "stderr": "", "stderr_lines": [], "stdout 2020-09-08T15:21:50.030 INFO:teuthology.task.ansible.out:": "d7c0190f3c79f0107823a9b3caef24ede18680e0", "stdout_lines": ["d7c0190f3c79f0107823a9b3caef24ede18680e0"]} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.078 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:50 +0000 (0:00:00.232) 0:00:08.597 ***** 2020-09-08T15:21:50.107 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacro 2020-09-08T15:21:50.118 INFO:teuthology.task.ansible.out:ix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 2020-09-08T15:21:50.122 INFO:teuthology.task.ansible.out: 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye 2020-09-08T15:21:50.126 INFO:teuthology.task.ansible.out:/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.s 2020-09-08T15:21:50.129 INFO:teuthology.task.ansible.out:epia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf 2020-09-08T15:21:50.131 INFO:teuthology.task.ansible.out:84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.132 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.134 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.146 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.148 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.164 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result 2020-09-08T15:21:50.166 INFO:teuthology.task.ansible.out: was False"} TASK [users : Determine if we can skip users and pubkeys updates] ************** ok: [smithi166.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.206 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:50 +0000 (0:00:00.128) 0:00:08.726 ***** 2020-09-08T15:21:50.220 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.222 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.222 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'rdias', 'ov 2020-09-08T15:21:50.224 INFO:teuthology.task.ansible.out:pn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.224 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.225 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w 2020-09-08T15:21:50.227 INFO:teuthology.task.ansible.out:f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.227 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d71 2020-09-08T15:21:50.229 INFO:teuthology.task.ansible.out:2ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.231 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.233 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.234 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes 2020-09-08T15:21:50.235 INFO:teuthology.task.ansible.out:LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.236 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f 2020-09-08T15:21:50.237 INFO:teuthology.task.ansible.out:708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.238 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional 2020-09-08T15:21:50.239 INFO:teuthology.task.ansible.out:result was False"} TASK [users : Merge extra_admin_users into managed_admin_users] **************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.289 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:50 +0000 (0:00:00.077) 0:00:08.803 ***** 2020-09-08T15:21:50.294 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.296 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.297 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.297 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80a 2020-09-08T15:21:50.299 INFO:teuthology.task.ansible.out:fabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.309 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.311 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.316 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prps 2020-09-08T15:21:50.318 INFO:teuthology.task.ansible.out:aAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b 2020-09-08T15:21:50.319 INFO:teuthology.task.ansible.out:539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "skip_reason": "Conditio 2020-09-08T15:21:50.321 INFO:teuthology.task.ansible.out:nal result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} TASK [users : Remove managed_admin_users from managed_users] ******************* skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57d 2020-09-08T15:21:50.322 INFO:teuthology.task.ansible.out:bdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.366 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:50 +0000 (0:00:00.082) 0:00:08.886 ***** 2020-09-08T15:21:50.376 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.379 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.379 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.381 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.382 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.382 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.384 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.385 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.385 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.386 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aF 2020-09-08T15:21:50.388 INFO:teuthology.task.ansible.out:eh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb545 2020-09-08T15:21:50.390 INFO:teuthology.task.ansible.out:8c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.398 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.400 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.401 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn 2020-09-08T15:21:50.402 INFO:teuthology.task.ansible.out:": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.403 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.408 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.431 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.444 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-09-08T15:21:50.446 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.446 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 2020-09-08T15:21:50.448 INFO:teuthology.task.ansible.out: 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.488 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:50 +0000 (0:00:00.121) 0:00:09.007 ***** 2020-09-08T15:21:50.533 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:50 +0000 (0:00:00.045) 0:00:09.052 ***** 2020-09-08T15:21:50.543 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.547 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCY 2020-09-08T15:21:50.548 INFO:teuthology.task.ansible.out:M5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.549 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-09-08T15:21:50.550 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ********************* 2020-09-08T15:21:50.552 INFO:teuthology.task.ansible.out:*** skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.591 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:50 +0000 (0:00:00.056) 0:00:09.109 ***** 2020-09-08T15:21:50.601 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.603 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.603 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.604 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.604 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.607 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.607 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": 2020-09-08T15:21:50.609 INFO:teuthology.task.ansible.out: "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190d 2020-09-08T15:21:50.611 INFO:teuthology.task.ansible.out:e4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.648 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:50 +0000 (0:00:00.057) 0:00:09.166 ***** 2020-09-08T15:21:50.657 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.665 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.675 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.720 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:50 +0000 (0:00:00.071) 0:00:09.237 ***** 2020-09-08T15:21:50.759 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:50 +0000 (0:00:00.041) 0:00:09.279 ***** 2020-09-08T15:21:50.773 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2020-09-08T15:21:50.775 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Clone the keys repo] ********************************************* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.815 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:50 +0000 (0:00:00.055) 0:00:09.334 ***** 2020-09-08T15:21:50.855 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:50 +0000 (0:00:00.039) 0:00:09.374 ***** 2020-09-08T15:21:50.862 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-09-08T15:21:50.864 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.906 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:50 +0000 (0:00:00.051) 0:00:09.425 ***** 2020-09-08T15:21:50.953 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:50 +0000 (0:00:00.046) 0:00:09.471 ***** 2020-09-08T15:21:50.964 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.966 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:50.967 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166a 2020-09-08T15:21:50.969 INFO:teuthology.task.ansible.out:f59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-09-08T15:21:50.969 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway 2020-09-08T15:21:50.972 INFO:teuthology.task.ansible.out:", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.010 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:51 +0000 (0:00:00.057) 0:00:09.529 ***** 2020-09-08T15:21:51.025 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} TASK [users : Merge managed_users and managed_admin users] ********************* 2020-09-08T15:21:51.027 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "change 2020-09-08T15:21:51.028 INFO:teuthology.task.ansible.out:d": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++ 2020-09-08T15:21:51.030 INFO:teuthology.task.ansible.out:pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237bab 2020-09-08T15:21:51.038 INFO:teuthology.task.ansible.out:a4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.075 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:51 +0000 (0:00:00.065) 0:00:09.594 ***** 2020-09-08T15:21:51.085 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.087 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-09-08T15:21:51.088 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.090 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.091 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d8 2020-09-08T15:21:51.093 INFO:teuthology.task.ansible.out:9bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ov 2020-09-08T15:21:51.095 INFO:teuthology.task.ansible.out:pn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.141 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:51 +0000 (0:00:00.065) 0:00:09.660 ***** 2020-09-08T15:21:51.152 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.153 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.154 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.154 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2020-09-08T15:21:51.155 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.156 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.157 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.157 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.158 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item 2020-09-08T15:21:51.160 INFO:teuthology.task.ansible.out:": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.205 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:51 +0000 (0:00:00.063) 0:00:09.724 ***** 2020-09-08T15:21:51.220 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.222 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.223 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.223 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-09-08T15:21:51.223 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329 2020-09-08T15:21:51.225 INFO:teuthology.task.ansible.out:a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.226 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.226 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} TASK [users : Create all admin us 2020-09-08T15:21:51.228 INFO:teuthology.task.ansible.out:ers with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.228 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.229 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc52 2020-09-08T15:21:51.232 INFO:teuthology.task.ansible.out:2b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.242 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.244 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.244 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-09-08T15:21:51.245 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc9 2020-09-08T15:21:51.246 INFO:teuthology.task.ansible.out:3bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.286 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:51 +0000 (0:00:00.081) 0:00:09.805 ***** 2020-09-08T15:21:51.300 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.301 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-09-08T15:21:51.301 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.302 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-09-08T15:21:51.302 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.303 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.303 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.303 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.304 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-09-08T15:21:51.304 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'lese 2020-09-08T15:21:51.312 INFO:teuthology.task.ansible.out:b@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.316 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** 2020-09-08T15:21:51.316 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.318 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-09-08T15:21:51.318 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** 2020-09-08T15:21:51.319 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.320 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-09-08T15:21:51.320 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw 2020-09-08T15:21:51.322 INFO:teuthology.task.ansible.out:ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.332 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-09-08T15:21:51.334 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** 2020-09-08T15:21:51.334 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional resul 2020-09-08T15:21:51.339 INFO:teuthology.task.ansible.out:t was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.347 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-09-08T15:21:51.349 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-09-08T15:21:51.349 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.388 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:51 +0000 (0:00:00.101) 0:00:09.907 ***** 2020-09-08T15:21:51.398 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-09-08T15:21:51.401 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.402 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449 2020-09-08T15:21:51.405 INFO:teuthology.task.ansible.out:598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.414 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-09-08T15:21:51.416 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={ 2020-09-08T15:21:51.418 INFO:teuthology.task.ansible.out:'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-09-08T15:21:51.419 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi 2020-09-08T15:21:51.420 INFO:teuthology.task.ansible.out:055.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.422 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-09-08T15:21:51.424 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-09-08T15:21:51.425 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-09-08T15:21:51.426 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b3 2020-09-08T15:21:51.430 INFO:teuthology.task.ansible.out:47ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.464 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:51 +0000 (0:00:00.076) 0:00:09.983 ***** 2020-09-08T15:21:51.506 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:51 +0000 (0:00:00.041) 0:00:10.025 ***** 2020-09-08T15:21:51.527 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.528 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.529 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.529 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.531 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.531 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.532 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.532 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4ires 2020-09-08T15:21:51.534 INFO:teuthology.task.ansible.out:is", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.534 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGm 2020-09-08T15:21:51.536 INFO:teuthology.task.ansible.out:kY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.549 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@z 2020-09-08T15:21:51.556 INFO:teuthology.task.ansible.out:ulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.561 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi104.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.578 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi104.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.583 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-09-08T15:21:51.585 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.596 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.600 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi104.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.608 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.610 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi104.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.620 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-09-08T15:21:51.621 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi104.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi104.front.sepia.ceph.com] => (i 2020-09-08T15:21:51.624 INFO:teuthology.task.ansible.out:tem=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.633 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.634 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.644 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.646 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi104.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.661 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi104.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.675 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-09-08T15:21:51.681 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.682 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.725 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:51 +0000 (0:00:00.218) 0:00:10.243 ***** 2020-09-08T15:21:51.734 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.735 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.737 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.738 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.738 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.739 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.772 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-09-08T15:21:51.774 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi104.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.774 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.816 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:51 +0000 (0:00:00.092) 0:00:10.335 ***** 2020-09-08T15:21:51.829 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.833 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.833 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.834 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.834 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.835 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.875 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:51 +0000 (0:00:00.059) 0:00:10.395 ***** 2020-09-08T15:21:51.887 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-09-08T15:21:51.889 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:51.937 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:51 +0000 (0:00:00.061) 0:00:10.456 ***** 2020-09-08T15:21:51.979 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2020-09-08T15:21:51.981 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.016 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:52 +0000 (0:00:00.078) 0:00:10.535 ***** 2020-09-08T15:21:52.056 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-09-08T15:21:52.058 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.101 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:52 +0000 (0:00:00.085) 0:00:10.621 ***** 2020-09-08T15:21:52.158 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.196 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:52 +0000 (0:00:00.094) 0:00:10.715 ***** 2020-09-08T15:21:52.197 INFO:teuthology.task.ansible.out: 2020-09-08T15:21:52.207 INFO:teuthology.task.ansible.out: 2020-09-08T15:21:52.208 INFO:teuthology.task.ansible.out:TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "checksum": "e4960b14bdbab8632f61f7e37ba0ea7d7809d19f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} 2020-09-08T15:21:52.254 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:52 +0000 (0:00:00.058) 0:00:10.773 ***** 2020-09-08T15:21:52.264 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-09-08T15:21:52.265 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-09-08T15:21:52.266 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "checksum": "e4960b14bdbab8632f61f7e37ba0ea7d7809d19f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} 2020-09-08T15:21:52.304 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:52 +0000 (0:00:00.049) 0:00:10.823 ***** 2020-09-08T15:21:52.349 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:52 +0000 (0:00:00.045) 0:00:10.869 ***** 2020-09-08T15:21:52.360 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-09-08T15:21:52.364 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => { "msg": "Host smithi055.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } ok: [smithi104.front.sepia.ceph.com] => { "msg": "Host smithi104.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2020-09-08T15:21:52.403 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:52 +0000 (0:00:00.053) 0:00:10.922 ***** 2020-09-08T15:21:52.445 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:52 +0000 (0:00:00.042) 0:00:10.964 ***** 2020-09-08T15:21:52.454 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-09-08T15:21:52.456 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.457 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.457 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.459 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.460 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.460 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.460 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** ok: [smithi055.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-09-08T15:21:52.472 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi166.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.474 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.516 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:52 +0000 (0:00:00.071) 0:00:11.035 ***** 2020-09-08T15:21:52.527 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.537 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.542 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} TASK [common : Including major version specific variables.] ******************** ok: [smithi104.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-09-08T15:21:52.552 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-09-08T15:21:52.554 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.555 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.555 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.556 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.556 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.556 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.558 INFO:teuthology.task.ansible.out: 2020-09-08T15:21:52.603 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:52 +0000 (0:00:00.086) 0:00:11.122 ***** 2020-09-08T15:21:52.614 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.615 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-09-08T15:21:52.616 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi166.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.616 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.617 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.620 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.662 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:52 +0000 (0:00:00.058) 0:00:11.181 ***** 2020-09-08T15:21:52.704 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:52 +0000 (0:00:00.042) 0:00:11.223 ***** 2020-09-08T15:21:52.714 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-09-08T15:21:52.716 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.767 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:52 +0000 (0:00:00.063) 0:00:11.287 ***** 2020-09-08T15:21:52.807 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:52 +0000 (0:00:00.039) 0:00:11.327 ***** 2020-09-08T15:21:52.827 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-09-08T15:21:52.829 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.869 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:52 +0000 (0:00:00.061) 0:00:11.388 ***** 2020-09-08T15:21:52.880 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-09-08T15:21:52.882 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.920 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:52 +0000 (0:00:00.051) 0:00:11.439 ***** 2020-09-08T15:21:52.930 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-09-08T15:21:52.932 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:52.973 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:52 +0000 (0:00:00.053) 0:00:11.492 ***** 2020-09-08T15:21:52.984 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-09-08T15:21:52.986 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:53.026 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.052) 0:00:11.545 ***** 2020-09-08T15:21:53.037 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:53.077 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.051) 0:00:11.596 ***** 2020-09-08T15:21:53.088 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "checksum": "e4960b14bdbab8632f61f7e37ba0ea7d7809d19f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} 2020-09-08T15:21:53.090 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:53.128 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.050) 0:00:11.647 ***** 2020-09-08T15:21:53.164 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.036) 0:00:11.683 ***** 2020-09-08T15:21:53.173 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:53.174 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:53.217 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.051) 0:00:11.735 ***** 2020-09-08T15:21:53.258 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.041) 0:00:11.777 ***** 2020-09-08T15:21:53.267 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-09-08T15:21:53.269 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => { "msg": "Host smithi166.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:53.307 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.049) 0:00:11.826 ***** 2020-09-08T15:21:53.357 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.049) 0:00:11.876 ***** 2020-09-08T15:21:53.368 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-09-08T15:21:53.370 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set entitlements_path] ****************************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:53.408 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.051) 0:00:11.927 ***** 2020-09-08T15:21:53.444 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.036) 0:00:11.963 ***** 2020-09-08T15:21:53.453 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-09-08T15:21:53.454 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-09-08T15:21:53.455 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi055.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-08T15:21:53.456 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-08T15:21:53.493 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.048) 0:00:12.012 ***** 2020-09-08T15:21:53.532 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.038) 0:00:12.051 ***** 2020-09-08T15:21:53.541 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-09-08T15:21:53.543 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set have_entitlements] ****************************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:53.583 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.050) 0:00:12.101 ***** 2020-09-08T15:21:53.625 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.037) 0:00:12.139 ***** 2020-09-08T15:21:53.627 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-09-08T15:21:53.629 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:53.666 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.046) 0:00:12.185 ***** 2020-09-08T15:21:53.703 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.037) 0:00:12.223 ***** 2020-09-08T15:21:53.712 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-09-08T15:21:53.713 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi055.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-08T15:21:53.750 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.045) 0:00:12.268 ***** 2020-09-08T15:21:53.789 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.039) 0:00:12.308 ***** 2020-09-08T15:21:53.798 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-09-08T15:21:53.799 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:53.838 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.048) 0:00:12.357 ***** 2020-09-08T15:21:53.875 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.036) 0:00:12.394 ***** 2020-09-08T15:21:53.884 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-09-08T15:21:53.885 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi104.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Register with subscription-manager.] **************************** skipping: [smithi055.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-08T15:21:53.922 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.047) 0:00:12.442 ***** 2020-09-08T15:21:53.959 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:53 +0000 (0:00:00.036) 0:00:12.479 ***** 2020-09-08T15:21:53.978 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-09-08T15:21:53.980 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:54.014 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.055) 0:00:12.534 ***** 2020-09-08T15:21:54.053 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.038) 0:00:12.572 ***** 2020-09-08T15:21:54.062 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-09-08T15:21:54.065 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : List CDN releases available to system] ************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:54.102 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.048) 0:00:12.621 ***** 2020-09-08T15:21:54.141 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.039) 0:00:12.660 ***** 2020-09-08T15:21:54.151 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-09-08T15:21:54.154 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:54.196 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.055) 0:00:12.716 ***** 2020-09-08T15:21:54.236 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.039) 0:00:12.756 ***** 2020-09-08T15:21:54.246 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-09-08T15:21:54.248 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:54.287 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.050) 0:00:12.807 ***** 2020-09-08T15:21:54.326 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.038) 0:00:12.845 ***** 2020-09-08T15:21:54.341 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi055.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-08T15:21:54.376 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.050) 0:00:12.895 ***** 2020-09-08T15:21:54.420 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.044) 0:00:12.939 ***** 2020-09-08T15:21:54.421 INFO:teuthology.task.ansible.out: 2020-09-08T15:21:54.431 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-09-08T15:21:54.432 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Clean yum cache] ************************************************ skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:54.473 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.050) 0:00:12.990 ***** 2020-09-08T15:21:54.520 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.048) 0:00:13.039 ***** 2020-09-08T15:21:54.846 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.060) 0:00:13.099 ***** Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.036) 0:00:13.136 ***** TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Store list of enabled repos] ************************************ skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.047) 0:00:13.184 ***** Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.045) 0:00:13.230 ***** TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.047) 0:00:13.277 ***** Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.044) 0:00:13.321 ***** TASK [common : Register with subscription-manager.] **************************** skipping: [smithi104.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Set replace_repos true if rhsm_repos d 2020-09-08T15:21:54.848 INFO:teuthology.task.ansible.out:iffers from repo_list] **** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:54.852 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.049) 0:00:13.371 ***** 2020-09-08T15:21:54.897 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.044) 0:00:13.415 ***** 2020-09-08T15:21:54.906 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-09-08T15:21:54.908 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:54.945 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.049) 0:00:13.465 ***** 2020-09-08T15:21:54.985 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:54 +0000 (0:00:00.039) 0:00:13.505 ***** 2020-09-08T15:21:54.995 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-09-08T15:21:54.997 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Disable all rhsm repos] ***************************************** skipping: [smithi055.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-08T15:21:55.036 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.050) 0:00:13.555 ***** 2020-09-08T15:21:55.072 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.035) 0:00:13.591 ***** 2020-09-08T15:21:55.080 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-09-08T15:21:55.085 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi166.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:55.120 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.048) 0:00:13.639 ***** 2020-09-08T15:21:55.158 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.038) 0:00:13.677 ***** 2020-09-08T15:21:55.168 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-09-08T15:21:55.169 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:55.207 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.049) 0:00:13.726 ***** 2020-09-08T15:21:55.246 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.039) 0:00:13.766 ***** 2020-09-08T15:21:55.256 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-09-08T15:21:55.258 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Increase the yum timeout.] ************************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:55.295 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.048) 0:00:13.815 ***** 2020-09-08T15:21:55.333 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.037) 0:00:13.852 ***** 2020-09-08T15:21:55.355 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-09-08T15:21:55.356 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:55.396 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.062) 0:00:13.915 ***** 2020-09-08T15:21:55.435 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.039) 0:00:13.954 ***** 2020-09-08T15:21:55.454 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure local epel mirrorlists] ******************************* skipping: [smithi055.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:55.483 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.048) 0:00:14.002 ***** 2020-09-08T15:21:55.494 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-09-08T15:21:55.496 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:55.532 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.048) 0:00:14.050 ***** 2020-09-08T15:21:55.568 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.037) 0:00:14.088 ***** 2020-09-08T15:21:55.578 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-09-08T15:21:55.582 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Clean yum cache] ************************************************ skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:55.625 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.056) 0:00:14.144 ***** 2020-09-08T15:21:55.663 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.037) 0:00:14.182 ***** 2020-09-08T15:21:55.673 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-09-08T15:21:55.675 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-08T15:21:55.712 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.049) 0:00:14.231 ***** 2020-09-08T15:21:55.723 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-09-08T15:21:55.725 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:55.761 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.048) 0:00:14.280 ***** 2020-09-08T15:21:55.770 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-09-08T15:21:55.772 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:55.822 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.054) 0:00:14.335 ***** 2020-09-08T15:21:55.826 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.014853", "end": "2020-09-08 15:21:55.356404", "rc": 0, "start": "2020-09-08 15:21:55.341551", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-09-08T15:21:55.826 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** skipping: [smithi166.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-08T15:21:55.866 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.049) 0:00:14.384 ***** 2020-09-08T15:21:55.903 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.038) 0:00:14.422 ***** 2020-09-08T15:21:55.913 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-09-08T15:21:55.915 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:55.962 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.055) 0:00:14.477 ***** 2020-09-08T15:21:55.996 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:55 +0000 (0:00:00.038) 0:00:14.516 ***** 2020-09-08T15:21:56.010 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:56.045 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.049) 0:00:14.565 ***** 2020-09-08T15:21:56.089 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.043) 0:00:14.609 ***** 2020-09-08T15:21:56.100 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2020-09-08T15:21:56.102 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get the current timezone] *************************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:56.142 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.052) 0:00:14.661 ***** 2020-09-08T15:21:56.181 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.039) 0:00:14.701 ***** 2020-09-08T15:21:56.193 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-09-08T15:21:56.195 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the timezone] *********************************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:56.236 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.054) 0:00:14.755 ***** 2020-09-08T15:21:56.275 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.039) 0:00:14.794 ***** 2020-09-08T15:21:56.283 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-09-08T15:21:56.285 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:56.323 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.048) 0:00:14.843 ***** 2020-09-08T15:21:56.363 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.040) 0:00:14.883 ***** 2020-09-08T15:21:56.377 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-09-08T15:21:56.379 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:56.421 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.057) 0:00:14.940 ***** 2020-09-08T15:21:56.459 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.037) 0:00:14.978 ***** 2020-09-08T15:21:56.469 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-09-08T15:21:56.471 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Refresh repos] ************************************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:56.507 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.048) 0:00:15.026 ***** 2020-09-08T15:21:56.544 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.036) 0:00:15.063 ***** 2020-09-08T15:21:56.554 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-09-08T15:21:56.556 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:56.596 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.051) 0:00:15.115 ***** 2020-09-08T15:21:56.607 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-09-08T15:21:56.609 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:56.647 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.051) 0:00:15.167 ***** 2020-09-08T15:21:56.659 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-09-08T15:21:56.660 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-08T15:21:56.698 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.050) 0:00:15.218 ***** 2020-09-08T15:21:56.706 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-09-08T15:21:56.708 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-08T15:21:56.751 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.048) 0:00:15.266 ***** 2020-09-08T15:21:56.764 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-09-08T15:21:56.766 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:56.801 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.053) 0:00:15.320 ***** 2020-09-08T15:21:56.809 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-09-08T15:21:56.811 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:56.848 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.047) 0:00:15.368 ***** 2020-09-08T15:21:56.859 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-09-08T15:21:56.860 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:56.897 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.048) 0:00:15.416 ***** 2020-09-08T15:21:56.905 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-09-08T15:21:56.906 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:56.945 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.047) 0:00:15.464 ***** 2020-09-08T15:21:56.953 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-09-08T15:21:56.955 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.284037", "end": "2020-09-08 15:21:56.500369", "rc": 0, "start": "2020-09-08 15:21:56.216332", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [common : Store list of enabled repos] ************************************ skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:56.993 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:56 +0000 (0:00:00.048) 0:00:15.512 ***** 2020-09-08T15:21:57.029 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:57 +0000 (0:00:00.036) 0:00:15.549 ***** 2020-09-08T15:21:57.039 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-09-08T15:21:57.040 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install Kerberos Packages (RedHat)] ***************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:57.076 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:57 +0000 (0:00:00.046) 0:00:15.596 ***** 2020-09-08T15:21:57.113 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:57 +0000 (0:00:00.036) 0:00:15.632 ***** 2020-09-08T15:21:57.124 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-09-08T15:21:57.127 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:57.166 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:57 +0000 (0:00:00.052) 0:00:15.685 ***** 2020-09-08T15:21:57.176 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-09-08T15:21:57.177 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:57.178 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:57.216 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:57 +0000 (0:00:00.050) 0:00:15.736 ***** 2020-09-08T15:21:57.230 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-09-08T15:21:57.232 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:57.273 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:57 +0000 (0:00:00.056) 0:00:15.792 ***** 2020-09-08T15:21:57.281 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-09-08T15:21:57.283 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the apt module rather than running 'apt-get'. If you need to use command because apt is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi055.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.050687", "end": "2020-09-08 15:21:56.832382", "rc": 0, "start": "2020-09-08 15:21:56.781695", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [common : Configure local epel mirrorlists] ******************************* 2020-09-08T15:21:57.283 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:57.284 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:57.323 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:57 +0000 (0:00:00.050) 0:00:15.843 ***** 2020-09-08T15:21:57.364 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:57 +0000 (0:00:00.040) 0:00:15.883 ***** 2020-09-08T15:21:57.376 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-09-08T15:21:57.381 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:57.414 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:57 +0000 (0:00:00.049) 0:00:15.933 ***** 2020-09-08T15:21:57.425 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-09-08T15:21:57.432 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:57.483 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:57 +0000 (0:00:00.069) 0:00:16.002 ***** 2020-09-08T15:21:57.493 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-09-08T15:21:57.497 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-08T15:21:57.535 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:57 +0000 (0:00:00.051) 0:00:16.053 ***** 2020-09-08T15:21:57.589 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:57.625 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:57 +0000 (0:00:00.090) 0:00:16.144 ***** 2020-09-08T15:21:57.656 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-09-08T15:21:57.658 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.015984", "end": "2020-09-08 15:21:58.052436", "rc": 0, "start": "2020-09-08 15:21:58.036452", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:57.695 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:57 +0000 (0:00:00.069) 0:00:16.214 ***** 2020-09-08T15:21:57.733 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:57 +0000 (0:00:00.038) 0:00:16.252 ***** 2020-09-08T15:21:57.743 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-09-08T15:21:57.746 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:57.784 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:57 +0000 (0:00:00.050) 0:00:16.303 ***** 2020-09-08T15:21:57.794 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-09-08T15:21:57.796 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:57.841 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:57 +0000 (0:00:00.050) 0:00:16.354 ***** 2020-09-08T15:21:57.851 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-09-08T15:21:57.853 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:57.894 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:57 +0000 (0:00:00.059) 0:00:16.413 ***** 2020-09-08T15:21:57.905 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-09-08T15:21:57.906 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:57.947 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:57 +0000 (0:00:00.053) 0:00:16.467 ***** 2020-09-08T15:21:57.961 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:58.002 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:57 +0000 (0:00:00.054) 0:00:16.521 ***** 2020-09-08T15:21:58.013 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-09-08T15:21:58.015 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2020-09-08T15:21:58.052 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:58 +0000 (0:00:00.050) 0:00:16.571 ***** 2020-09-08T15:21:58.060 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-09-08T15:21:58.063 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:58.100 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:58 +0000 (0:00:00.047) 0:00:16.619 ***** 2020-09-08T15:21:58.110 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-09-08T15:21:58.114 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:58.149 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:58 +0000 (0:00:00.048) 0:00:16.668 ***** 2020-09-08T15:21:58.160 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:58.199 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:58 +0000 (0:00:00.050) 0:00:16.719 ***** 2020-09-08T15:21:58.268 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:58.298 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:58 +0000 (0:00:00.098) 0:00:16.817 ***** 2020-09-08T15:21:58.320 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-09-08T15:21:58.322 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.009921", "end": "2020-09-08 15:21:58.021122", "rc": 0, "start": "2020-09-08 15:21:58.011201", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-09-08T15:21:58.356 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:58 +0000 (0:00:00.058) 0:00:16.875 ***** 2020-09-08T15:21:58.365 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-09-08T15:21:58.367 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:58.404 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:58 +0000 (0:00:00.047) 0:00:16.923 ***** 2020-09-08T15:21:58.412 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-09-08T15:21:58.413 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:58.454 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:58 +0000 (0:00:00.049) 0:00:16.973 ***** 2020-09-08T15:21:58.494 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-09-08T15:21:58.498 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:58.531 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:58 +0000 (0:00:00.077) 0:00:17.050 ***** 2020-09-08T15:21:58.561 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-09-08T15:21:58.563 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:58.598 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:58 +0000 (0:00:00.066) 0:00:17.117 ***** 2020-09-08T15:21:58.628 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-09-08T15:21:58.630 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:58.669 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:58 +0000 (0:00:00.071) 0:00:17.188 ***** 2020-09-08T15:21:58.699 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-09-08T15:21:58.701 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:58.736 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:58 +0000 (0:00:00.067) 0:00:17.255 ***** 2020-09-08T15:21:58.777 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-09-08T15:21:58.780 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:58.814 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:58 +0000 (0:00:00.077) 0:00:17.333 ***** 2020-09-08T15:21:58.845 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:58.887 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:58 +0000 (0:00:00.072) 0:00:17.406 ***** 2020-09-08T15:21:59.108 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-09-08T15:21:59.110 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.561162", "end": "2020-09-08 15:21:59.500313", "rc": 0, "start": "2020-09-08 15:21:58.939151", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:21:59.147 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:59 +0000 (0:00:00.259) 0:00:17.666 ***** 2020-09-08T15:21:59.189 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-09-08T15:21:59.190 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:59.226 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:59 +0000 (0:00:00.079) 0:00:17.746 ***** 2020-09-08T15:21:59.419 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-09-08T15:21:59.421 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.054845", "end": "2020-09-08 15:21:59.824588", "rc": 0, "start": "2020-09-08 15:21:59.769743", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:21:59.455 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:59 +0000 (0:00:00.228) 0:00:17.974 ***** 2020-09-08T15:21:59.785 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-09-08T15:21:59.787 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.761940", "end": "2020-09-08 15:21:59.500450", "rc": 0, "start": "2020-09-08 15:21:58.738510", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:21:59.823 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:59 +0000 (0:00:00.367) 0:00:18.342 ***** 2020-09-08T15:21:59.854 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-09-08T15:21:59.856 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:21:59.898 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:21:59 +0000 (0:00:00.074) 0:00:18.417 ***** 2020-09-08T15:22:00.098 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-09-08T15:22:00.100 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.046816", "end": "2020-09-08 15:21:59.810575", "rc": 0, "start": "2020-09-08 15:21:59.763759", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:22:00.135 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:00 +0000 (0:00:00.237) 0:00:18.654 ***** 2020-09-08T15:22:01.102 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-09-08T15:22:01.104 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1599578284, "cache_updated": false, "changed": false} 2020-09-08T15:22:01.140 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:01 +0000 (0:00:01.004) 0:00:19.659 ***** 2020-09-08T15:22:02.047 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** ok: [smithi055.front.sepia.ceph.com] => {"cache_update_time": 1599578284, "cache_updated": false, "changed": false} 2020-09-08T15:22:02.111 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:02 +0000 (0:00:00.971) 0:00:20.631 ***** 2020-09-08T15:22:02.161 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-09-08T15:22:02.163 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:02.200 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:02 +0000 (0:00:00.089) 0:00:20.720 ***** 2020-09-08T15:22:02.578 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-09-08T15:22:02.619 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:02 +0000 (0:00:00.418) 0:00:21.138 ***** 2020-09-08T15:22:03.143 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** ok: [smithi104.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1599578240, "cache_updated": false, "changed": false} 2020-09-08T15:22:03.187 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:03 +0000 (0:00:00.567) 0:00:21.706 ***** 2020-09-08T15:22:03.541 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** ok: [smithi055.front.sepia.ceph.com] => {"cache_update_time": 1599578284, "cache_updated": false, "changed": false} 2020-09-08T15:22:03.613 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:03 +0000 (0:00:00.426) 0:00:22.132 ***** 2020-09-08T15:22:03.693 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** ok: [smithi166.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1599578328, "cache_updated": false, "changed": false} 2020-09-08T15:22:03.741 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:03 +0000 (0:00:00.126) 0:00:22.259 ***** 2020-09-08T15:22:04.013 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-09-08T15:22:04.015 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-09-08T15:22:04.130 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-09-08T15:22:04.132 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"cache_update_time": 1599578240, "cache_updated": false, "changed": false} 2020-09-08T15:22:04.168 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:04 +0000 (0:00:00.428) 0:00:22.687 ***** 2020-09-08T15:22:04.199 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-09-08T15:22:04.201 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:04.237 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:04 +0000 (0:00:00.068) 0:00:22.756 ***** 2020-09-08T15:22:04.362 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-09-08T15:22:04.364 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-09-08T15:22:04.600 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-09-08T15:22:04.602 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"cache_update_time": 1599578328, "cache_updated": false, "changed": false} 2020-09-08T15:22:04.790 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:04 +0000 (0:00:00.548) 0:00:23.305 ***** 2020-09-08T15:22:04.797 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-09-08T15:22:04.798 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-09-08T15:22:04.799 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi055.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-09-08T15:22:04.841 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:04 +0000 (0:00:00.055) 0:00:23.360 ***** 2020-09-08T15:22:04.882 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:04 +0000 (0:00:00.040) 0:00:23.401 ***** 2020-09-08T15:22:04.893 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-09-08T15:22:04.894 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:04.934 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:04 +0000 (0:00:00.052) 0:00:23.454 ***** 2020-09-08T15:22:04.979 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-09-08T15:22:05.022 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:05 +0000 (0:00:00.084) 0:00:23.538 ***** 2020-09-08T15:22:05.325 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-09-08T15:22:05.327 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-09-08T15:22:05.370 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:05 +0000 (0:00:00.351) 0:00:23.890 ***** 2020-09-08T15:22:05.396 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-09-08T15:22:05.398 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 12725, "state": "file", "uid": 0} 2020-09-08T15:22:05.696 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-09-08T15:22:05.740 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** ok: [smithi104.front.sepia.ceph.com] => {"cache_update_time": 1599578240, "cache_updated": false, "changed": false} 2020-09-08T15:22:05.784 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:05 +0000 (0:00:00.413) 0:00:24.304 ***** 2020-09-08T15:22:06.020 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-09-08T15:22:06.022 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-09-08T15:22:06.061 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:06 +0000 (0:00:00.277) 0:00:24.581 ***** 2020-09-08T15:22:06.104 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-09-08T15:22:06.106 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-09-08T15:22:06.143 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:06 +0000 (0:00:00.081) 0:00:24.663 ***** 2020-09-08T15:22:06.176 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-09-08T15:22:06.178 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:06.226 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:06 +0000 (0:00:00.082) 0:00:24.746 ***** 2020-09-08T15:22:06.239 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-09-08T15:22:06.240 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-09-08T15:22:06.261 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-09-08T15:22:06.263 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-08T15:22:06.299 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:06 +0000 (0:00:00.072) 0:00:24.818 ***** 2020-09-08T15:22:06.317 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-09-08T15:22:06.319 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"cache_update_time": 1599578328, "cache_updated": false, "changed": false} 2020-09-08T15:22:06.354 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:06 +0000 (0:00:00.054) 0:00:24.873 ***** 2020-09-08T15:22:06.378 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:06.379 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:06.419 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:06 +0000 (0:00:00.065) 0:00:24.938 ***** 2020-09-08T15:22:06.450 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-09-08T15:22:06.452 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:06.488 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:06 +0000 (0:00:00.068) 0:00:25.007 ***** 2020-09-08T15:22:06.531 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-09-08T15:22:06.533 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-09-08T15:22:06.725 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-09-08T15:22:06.861 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-09-08T15:22:06.908 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:06 +0000 (0:00:00.413) 0:00:25.421 ***** 2020-09-08T15:22:07.042 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-09-08T15:22:07.044 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-09-08T15:22:07.082 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:07 +0000 (0:00:00.179) 0:00:25.601 ***** 2020-09-08T15:22:07.095 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-09-08T15:22:07.097 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-09-08T15:22:07.406 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-09-08T15:22:07.408 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"cache_update_time": 1599578284, "cache_updated": false, "changed": false} TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi166.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-09-08T15:22:07.455 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:07 +0000 (0:00:00.373) 0:00:25.975 ***** 2020-09-08T15:22:07.501 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:07 +0000 (0:00:00.040) 0:00:26.015 ***** 2020-09-08T15:22:07.504 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-09-08T15:22:07.506 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 12725, "state": "file", "uid": 0} 2020-09-08T15:22:07.647 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-09-08T15:22:07.690 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:07 +0000 (0:00:00.194) 0:00:26.209 ***** 2020-09-08T15:22:07.807 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-09-08T15:22:07.810 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-09-08T15:22:07.861 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:07 +0000 (0:00:00.170) 0:00:26.380 ***** 2020-09-08T15:22:07.885 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-09-08T15:22:07.887 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-09-08T15:22:08.061 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 12725, "state": "file", "uid": 0} 2020-09-08T15:22:08.189 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-09-08T15:22:08.228 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:08 +0000 (0:00:00.367) 0:00:26.748 ***** 2020-09-08T15:22:08.252 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-09-08T15:22:08.253 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-09-08T15:22:08.297 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:08 +0000 (0:00:00.068) 0:00:26.817 ***** 2020-09-08T15:22:08.309 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-09-08T15:22:08.311 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-09-08T15:22:08.350 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:08 +0000 (0:00:00.052) 0:00:26.869 ***** 2020-09-08T15:22:08.374 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-09-08T15:22:08.376 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-09-08T15:22:08.386 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:08.427 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:08 +0000 (0:00:00.076) 0:00:26.946 ***** 2020-09-08T15:22:08.458 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-09-08T15:22:08.460 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} TASK [common : Check for epel] ************************************************* skipping: [smithi104.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-08T15:22:08.497 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:08 +0000 (0:00:00.070) 0:00:27.017 ***** 2020-09-08T15:22:08.543 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-09-08T15:22:08.544 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:08.573 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:08.575 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:08.621 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:08 +0000 (0:00:00.123) 0:00:27.140 ***** 2020-09-08T15:22:08.632 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-09-08T15:22:08.650 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-09-08T15:22:08.676 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:08 +0000 (0:00:00.054) 0:00:27.195 ***** 2020-09-08T15:22:08.687 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-09-08T15:22:08.688 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:08.725 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:08 +0000 (0:00:00.049) 0:00:27.244 ***** 2020-09-08T15:22:08.734 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-09-08T15:22:08.737 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi166.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-09-08T15:22:08.776 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:08 +0000 (0:00:00.050) 0:00:27.295 ***** 2020-09-08T15:22:08.816 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:08 +0000 (0:00:00.039) 0:00:27.335 ***** 2020-09-08T15:22:08.824 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-09-08T15:22:08.826 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-08T15:22:08.863 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:08 +0000 (0:00:00.047) 0:00:27.382 ***** 2020-09-08T15:22:08.873 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-09-08T15:22:08.875 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-09-08T15:22:08.912 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:08 +0000 (0:00:00.049) 0:00:27.432 ***** 2020-09-08T15:22:08.922 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-09-08T15:22:08.924 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:08.961 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:08 +0000 (0:00:00.048) 0:00:27.480 ***** 2020-09-08T15:22:08.979 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-09-08T15:22:08.981 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:09.015 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:09 +0000 (0:00:00.054) 0:00:27.534 ***** 2020-09-08T15:22:09.056 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-09-08T15:22:09.058 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-08T15:22:09.093 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:09 +0000 (0:00:00.078) 0:00:27.613 ***** 2020-09-08T15:22:09.154 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-09-08T15:22:09.158 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:09.177 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:09.179 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:09.226 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:09 +0000 (0:00:00.132) 0:00:27.745 ***** 2020-09-08T15:22:09.257 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-09-08T15:22:09.259 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:09.300 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:09 +0000 (0:00:00.072) 0:00:27.818 ***** 2020-09-08T15:22:09.346 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-09-08T15:22:09.381 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:09 +0000 (0:00:00.082) 0:00:27.900 ***** 2020-09-08T15:22:09.629 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-09-08T15:22:09.630 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"cache_update_time": 1599578240, "cache_updated": false, "changed": false} 2020-09-08T15:22:09.678 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:09 +0000 (0:00:00.296) 0:00:28.197 ***** 2020-09-08T15:22:10.013 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-09-08T15:22:10.051 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:10 +0000 (0:00:00.373) 0:00:28.570 ***** 2020-09-08T15:22:10.088 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-09-08T15:22:10.089 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Tue 2020-09-08 15:21:11 UTC", "ActiveEnterTimestampMonotonic": "19088910", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "nss-lookup.target var-run.mount network.target time-sync.target sysinit.target remote-fs.target local-fs.target system.slice -.mount systemd-tmpfiles-setup.service basic.target systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Tue 2020-09-08 15:21:11 UTC", "AssertTimestampMonotonic": "19087963", "Before": "multi-user.target getty@tty1.service xdm.service plymouth-quit.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Tue 2020-09-08 15:21:11 UTC", "ConditionTimestampMonotonic": "19087963", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-ser 2020-09-08T15:22:10.092 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1233", "ExecMainStartTimestamp": "Tue 2020-09-08 15:21:11 UTC", "ExecMainStartTimestampMonotonic": "19088881", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Tue 2020-09-08 15:21:11 UTC] ; stop_time=[n/a] ; pid=1233 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Tue 2020-09-08 15:21:11 UTC", "InactiveExitTimestampMonotonic": "19088910", "InvocationID": "9849e1737fa64819ab59677eed29bb96", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "i 2020-09-08T15:22:10.093 INFO:teuthology.task.ansible.out:nfinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128055", "LimitNPROCSoft": "128055", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128055", "LimitSIGPENDINGSoft": "128055", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1233", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2020-09-08T15:22:10.095 INFO:teuthology.task.ansible.out: "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Tue 2020-09-08 15:21:11 UTC", "StateChangeTimestampMonotonic": "19088910", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Tue 2020-09-08 15:21:11 UTC", "WatchdogTimestampMonotonic": "19088910", "WatchdogUSec": "0"}} 2020-09-08T15:22:10.135 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:10 +0000 (0:00:00.083) 0:00:28.654 ***** 2020-09-08T15:22:10.165 INFO:teuthology.task.ansible.out: 2020-09-08T15:22:10.167 INFO:teuthology.task.ansible.out:TASK [common : Get SELinux status] ********************************************* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:10.210 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:10 +0000 (0:00:00.075) 0:00:28.730 ***** 2020-09-08T15:22:10.212 INFO:teuthology.task.ansible.out: 2020-09-08T15:22:10.220 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-09-08T15:22:10.222 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"cache_update_time": 1599578328, "cache_updated": false, "changed": false} 2020-09-08T15:22:10.261 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:10 +0000 (0:00:00.050) 0:00:28.780 ***** 2020-09-08T15:22:10.271 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-09-08T15:22:10.273 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:10.325 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:10 +0000 (0:00:00.064) 0:00:28.844 ***** 2020-09-08T15:22:10.355 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-09-08T15:22:10.357 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:10.395 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:10 +0000 (0:00:00.069) 0:00:28.914 ***** 2020-09-08T15:22:10.404 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-09-08T15:22:10.406 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-09-08T15:22:10.446 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:10 +0000 (0:00:00.051) 0:00:28.965 ***** 2020-09-08T15:22:10.458 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-09-08T15:22:10.459 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:10.495 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:10 +0000 (0:00:00.049) 0:00:29.015 ***** 2020-09-08T15:22:10.537 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-09-08T15:22:10.539 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:10.576 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:10 +0000 (0:00:00.080) 0:00:29.095 ***** 2020-09-08T15:22:10.608 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-09-08T15:22:10.610 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} TASK [common : nrpe - Copy SELinux type enforcement file] ********************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:10.646 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:10 +0000 (0:00:00.070) 0:00:29.165 ***** 2020-09-08T15:22:10.658 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-09-08T15:22:10.659 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-09-08T15:22:10.700 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:10 +0000 (0:00:00.053) 0:00:29.219 ***** 2020-09-08T15:22:10.714 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:10.750 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:10 +0000 (0:00:00.050) 0:00:29.270 ***** 2020-09-08T15:22:10.761 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-09-08T15:22:10.762 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-09-08T15:22:10.802 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:10 +0000 (0:00:00.052) 0:00:29.322 ***** 2020-09-08T15:22:10.813 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:10.851 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:10 +0000 (0:00:00.048) 0:00:29.370 ***** 2020-09-08T15:22:10.860 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-09-08T15:22:10.862 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:10.905 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:10 +0000 (0:00:00.053) 0:00:29.424 ***** 2020-09-08T15:22:10.922 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-09-08T15:22:10.924 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:10.965 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:10 +0000 (0:00:00.059) 0:00:29.484 ***** 2020-09-08T15:22:10.974 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-09-08T15:22:10.976 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-08T15:22:11.013 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:11 +0000 (0:00:00.047) 0:00:29.532 ***** 2020-09-08T15:22:11.023 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-09-08T15:22:11.025 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:11.061 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:11 +0000 (0:00:00.048) 0:00:29.581 ***** 2020-09-08T15:22:11.072 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-09-08T15:22:11.075 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Upload nagios check_mem script] ********************************* ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-09-08T15:22:11.116 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:11 +0000 (0:00:00.054) 0:00:29.636 ***** 2020-09-08T15:22:11.155 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:11 +0000 (0:00:00.038) 0:00:29.674 ***** 2020-09-08T15:22:11.170 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** ok: [smithi055.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-09-08T15:22:11.222 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:11 +0000 (0:00:00.054) 0:00:29.728 ***** 2020-09-08T15:22:11.260 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-09-08T15:22:11.261 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-09-08T15:22:11.302 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:11 +0000 (0:00:00.092) 0:00:29.821 ***** 2020-09-08T15:22:11.327 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ ok: [smithi166.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-09-08T15:22:11.360 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** ok: [smithi055.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-09-08T15:22:11.402 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:11 +0000 (0:00:00.098) 0:00:29.920 ***** 2020-09-08T15:22:11.442 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-09-08T15:22:11.444 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-09-08T15:22:11.484 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:11 +0000 (0:00:00.084) 0:00:30.004 ***** 2020-09-08T15:22:11.495 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-09-08T15:22:11.497 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-09-08T15:22:11.536 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:11 +0000 (0:00:00.051) 0:00:30.055 ***** 2020-09-08T15:22:11.547 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-09-08T15:22:11.549 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:11.590 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:11 +0000 (0:00:00.054) 0:00:30.110 ***** 2020-09-08T15:22:11.638 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:11 +0000 (0:00:00.039) 0:00:30.149 ***** 2020-09-08T15:22:11.660 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** skipping: [smithi166.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-09-08T15:22:11.697 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:11 +0000 (0:00:00.067) 0:00:30.216 ***** 2020-09-08T15:22:11.708 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-09-08T15:22:11.709 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-09-08T15:22:11.755 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:11 +0000 (0:00:00.058) 0:00:30.275 ***** 2020-09-08T15:22:11.766 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-09-08T15:22:11.768 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:11.806 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:11 +0000 (0:00:00.050) 0:00:30.325 ***** 2020-09-08T15:22:11.826 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-09-08T15:22:11.827 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Tue 2020-09-08 15:21:19 UTC", "ActiveEnterTimestampMonotonic": "22390930", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sysinit.target network.target time-sync.target systemd-journald.socket remote-fs.target -.mount nss-lookup.target var-run.mount basic.target system.slice local-fs.target systemd-tmpfiles-setup.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Tue 2020-09-08 15:21:19 UTC", "AssertTimestampMonotonic": "22388099", "Before": "xdm.service multi-user.target getty@tty1.service plymouth-quit.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Tue 2020-09-08 15:21:19 UTC", "ConditionTimestampMonotonic": "22388098", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-ser 2020-09-08T15:22:11.829 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1136", "ExecMainStartTimestamp": "Tue 2020-09-08 15:21:19 UTC", "ExecMainStartTimestampMonotonic": "22390823", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Tue 2020-09-08 15:21:19 UTC] ; stop_time=[n/a] ; pid=1136 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Tue 2020-09-08 15:21:19 UTC", "InactiveExitTimestampMonotonic": "22390930", "InvocationID": "dbfca802a27e4222a883d682e29c299f", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "i 2020-09-08T15:22:11.831 INFO:teuthology.task.ansible.out:nfinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128031", "LimitNPROCSoft": "128031", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128031", "LimitSIGPENDINGSoft": "128031", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1136", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2020-09-08T15:22:11.833 INFO:teuthology.task.ansible.out: "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Tue 2020-09-08 15:21:19 UTC", "StateChangeTimestampMonotonic": "22390930", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Tue 2020-09-08 15:21:19 UTC", "WatchdogTimestampMonotonic": "22390927", "WatchdogUSec": "0"}} 2020-09-08T15:22:11.872 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:11 +0000 (0:00:00.066) 0:00:30.392 ***** 2020-09-08T15:22:11.906 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-09-08T15:22:11.907 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "gid": 1229, "name": "ubuntu", "state": "present", "system": false} 2020-09-08T15:22:11.955 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:11 +0000 (0:00:00.082) 0:00:30.474 ***** 2020-09-08T15:22:11.966 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-09-08T15:22:11.968 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:12.005 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:11 +0000 (0:00:00.049) 0:00:30.524 ***** 2020-09-08T15:22:12.015 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-09-08T15:22:12.017 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:12.053 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:12 +0000 (0:00:00.048) 0:00:30.572 ***** 2020-09-08T15:22:12.072 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:12.109 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:12 +0000 (0:00:00.056) 0:00:30.628 ***** 2020-09-08T15:22:12.118 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-09-08T15:22:12.120 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:12.156 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:12 +0000 (0:00:00.046) 0:00:30.675 ***** 2020-09-08T15:22:12.167 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-09-08T15:22:12.170 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:12.207 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:12 +0000 (0:00:00.050) 0:00:30.726 ***** 2020-09-08T15:22:12.216 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-09-08T15:22:12.218 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-09-08T15:22:12.255 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:12 +0000 (0:00:00.048) 0:00:30.775 ***** 2020-09-08T15:22:12.265 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-09-08T15:22:12.267 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:12.305 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:12 +0000 (0:00:00.049) 0:00:30.824 ***** 2020-09-08T15:22:12.325 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** ok: [smithi055.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-09-08T15:22:12.374 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:12 +0000 (0:00:00.069) 0:00:30.894 ***** 2020-09-08T15:22:12.390 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:12.424 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:12 +0000 (0:00:00.049) 0:00:30.943 ***** 2020-09-08T15:22:12.475 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:12.512 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:12 +0000 (0:00:00.088) 0:00:31.031 ***** 2020-09-08T15:22:12.525 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-09-08T15:22:12.527 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Tue 2020-09-08 15:21:31 UTC", "ActiveEnterTimestampMonotonic": "21655409", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sysinit.target system.slice remote-fs.target time-sync.target local-fs.target systemd-tmpfiles-setup.service systemd-journald.socket nss-lookup.target var-run.mount network.target -.mount basic.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Tue 2020-09-08 15:21:31 UTC", "AssertTimestampMonotonic": "21653681", "Before": "xdm.service plymouth-quit.service getty@tty1.service shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Tue 2020-09-08 15:21:31 UTC", "ConditionTimestampMonotonic": "21653680", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-ser 2020-09-08T15:22:12.529 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1341", "ExecMainStartTimestamp": "Tue 2020-09-08 15:21:31 UTC", "ExecMainStartTimestampMonotonic": "21655339", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Tue 2020-09-08 15:21:31 UTC] ; stop_time=[n/a] ; pid=1341 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Tue 2020-09-08 15:21:31 UTC", "InactiveExitTimestampMonotonic": "21655409", "InvocationID": "3c20ffb331bf4121a5a5076e9f767906", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "i 2020-09-08T15:22:12.530 INFO:teuthology.task.ansible.out:nfinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128030", "LimitNPROCSoft": "128030", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128030", "LimitSIGPENDINGSoft": "128030", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1341", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2020-09-08T15:22:12.532 INFO:teuthology.task.ansible.out: "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Tue 2020-09-08 15:21:31 UTC", "StateChangeTimestampMonotonic": "21655409", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Tue 2020-09-08 15:21:31 UTC", "WatchdogTimestampMonotonic": "21655408", "WatchdogUSec": "0"}} 2020-09-08T15:22:12.573 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:12 +0000 (0:00:00.061) 0:00:31.093 ***** 2020-09-08T15:22:12.584 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-09-08T15:22:12.586 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:12.624 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:12 +0000 (0:00:00.050) 0:00:31.143 ***** 2020-09-08T15:22:12.635 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-09-08T15:22:12.637 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:12.683 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:12 +0000 (0:00:00.058) 0:00:31.202 ***** 2020-09-08T15:22:12.695 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-09-08T15:22:12.697 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:12.734 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:12 +0000 (0:00:00.051) 0:00:31.254 ***** 2020-09-08T15:22:12.745 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-09-08T15:22:12.747 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:12.798 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:12 +0000 (0:00:00.059) 0:00:31.313 ***** 2020-09-08T15:22:12.804 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-09-08T15:22:12.806 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:12.844 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:12 +0000 (0:00:00.050) 0:00:31.364 ***** 2020-09-08T15:22:12.856 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-09-08T15:22:12.857 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:12.896 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:12 +0000 (0:00:00.051) 0:00:31.415 ***** 2020-09-08T15:22:12.935 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:12 +0000 (0:00:00.039) 0:00:31.454 ***** 2020-09-08T15:22:12.946 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-09-08T15:22:12.948 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set hostname var] ********************************************* ok: [smithi055.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi055"}, "changed": false} 2020-09-08T15:22:12.989 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:12 +0000 (0:00:00.053) 0:00:31.508 ***** 2020-09-08T15:22:13.027 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:13 +0000 (0:00:00.038) 0:00:31.546 ***** 2020-09-08T15:22:13.038 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-09-08T15:22:13.040 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:13.079 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:13 +0000 (0:00:00.051) 0:00:31.598 ***** 2020-09-08T15:22:13.090 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-09-08T15:22:13.092 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-09-08T15:22:13.129 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:13 +0000 (0:00:00.050) 0:00:31.648 ***** 2020-09-08T15:22:13.140 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-09-08T15:22:13.141 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:13.184 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:13 +0000 (0:00:00.054) 0:00:31.703 ***** 2020-09-08T15:22:13.197 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-09-08T15:22:13.199 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-09-08T15:22:13.237 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:13 +0000 (0:00:00.053) 0:00:31.757 ***** 2020-09-08T15:22:13.246 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-09-08T15:22:13.248 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:13.285 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:13 +0000 (0:00:00.048) 0:00:31.805 ***** 2020-09-08T15:22:13.296 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-09-08T15:22:13.298 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-09-08T15:22:13.336 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:13 +0000 (0:00:00.050) 0:00:31.855 ***** 2020-09-08T15:22:13.348 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-09-08T15:22:13.350 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:13.392 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:13 +0000 (0:00:00.055) 0:00:31.911 ***** 2020-09-08T15:22:13.401 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-09-08T15:22:13.404 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-09-08T15:22:13.452 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:13 +0000 (0:00:00.058) 0:00:31.970 ***** 2020-09-08T15:22:13.461 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-09-08T15:22:13.462 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:13.514 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:13 +0000 (0:00:00.063) 0:00:32.033 ***** 2020-09-08T15:22:13.556 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-09-08T15:22:13.558 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:13.594 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:13 +0000 (0:00:00.079) 0:00:32.113 ***** 2020-09-08T15:22:13.618 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-09-08T15:22:13.662 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:13 +0000 (0:00:00.064) 0:00:32.178 ***** 2020-09-08T15:22:13.674 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:13.709 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:13 +0000 (0:00:00.050) 0:00:32.228 ***** 2020-09-08T15:22:13.764 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-09-08T15:22:13.769 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-09-08T15:22:13.804 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:13 +0000 (0:00:00.095) 0:00:32.323 ***** 2020-09-08T15:22:13.828 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-09-08T15:22:13.831 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "gid": 1229, "name": "ubuntu", "state": "present", "system": false} 2020-09-08T15:22:13.881 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:13 +0000 (0:00:00.077) 0:00:32.400 ***** 2020-09-08T15:22:13.898 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* ok: [smithi166.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-09-08T15:22:13.958 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:13 +0000 (0:00:00.073) 0:00:32.474 ***** 2020-09-08T15:22:13.967 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-09-08T15:22:13.987 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:14.017 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:14 +0000 (0:00:00.056) 0:00:32.531 ***** 2020-09-08T15:22:14.031 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ changed: [smithi055.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi055.front.sepia.ceph.com", "ansible_hostname": "smithi055", "ansible_nodename": "smithi055"}, "changed": true, "name": "smithi055"} TASK [testnode : Including major version specific variables.] ****************** ok: [smithi166.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-09-08T15:22:14.068 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:14 +0000 (0:00:00.056) 0:00:32.587 ***** 2020-09-08T15:22:14.129 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:14 +0000 (0:00:00.059) 0:00:32.647 ***** 2020-09-08T15:22:14.138 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-09-08T15:22:14.140 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:14.198 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:14 +0000 (0:00:00.068) 0:00:32.715 ***** 2020-09-08T15:22:14.210 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ ok: [smithi166.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-09-08T15:22:14.252 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:14 +0000 (0:00:00.055) 0:00:32.771 ***** 2020-09-08T15:22:14.391 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-09-08T15:22:14.393 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-09-08T15:22:14.431 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:14 +0000 (0:00:00.179) 0:00:32.951 ***** 2020-09-08T15:22:14.453 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-09-08T15:22:14.454 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-09-08T15:22:14.509 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:14 +0000 (0:00:00.077) 0:00:33.028 ***** 2020-09-08T15:22:14.533 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-09-08T15:22:14.535 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2020-09-08T15:22:14.576 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:14 +0000 (0:00:00.066) 0:00:33.095 ***** 2020-09-08T15:22:14.672 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-09-08T15:22:14.674 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "gid": 1229, "name": "ubuntu", "state": "present", "system": false} 2020-09-08T15:22:14.714 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:14 +0000 (0:00:00.136) 0:00:33.232 ***** 2020-09-08T15:22:14.744 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-09-08T15:22:14.746 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:14.784 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:14 +0000 (0:00:00.071) 0:00:33.304 ***** 2020-09-08T15:22:14.828 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:14.867 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:14 +0000 (0:00:00.082) 0:00:33.386 ***** 2020-09-08T15:22:14.890 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-09-08T15:22:14.891 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-09-08T15:22:14.950 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:14 +0000 (0:00:00.082) 0:00:33.469 ***** 2020-09-08T15:22:14.970 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-09-08T15:22:14.972 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2020-09-08T15:22:15.016 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:15 +0000 (0:00:00.065) 0:00:33.535 ***** 2020-09-08T15:22:15.028 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-09-08T15:22:15.030 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi104"}, "changed": false} 2020-09-08T15:22:15.068 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:15 +0000 (0:00:00.052) 0:00:33.587 ***** 2020-09-08T15:22:15.080 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-09-08T15:22:15.082 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-09-08T15:22:15.120 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:15 +0000 (0:00:00.051) 0:00:33.639 ***** 2020-09-08T15:22:15.533 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-09-08T15:22:15.535 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi104.front.sepia.ceph.com", "ansible_hostname": "smithi104", "ansible_nodename": "smithi104"}, "changed": true, "name": "smithi104"} 2020-09-08T15:22:15.573 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:15 +0000 (0:00:00.453) 0:00:34.092 ***** 2020-09-08T15:22:15.584 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-09-08T15:22:15.586 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-09-08T15:22:15.645 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:15 +0000 (0:00:00.071) 0:00:34.164 ***** 2020-09-08T15:22:15.697 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-09-08T15:22:15.699 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi166"}, "changed": false} 2020-09-08T15:22:15.735 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:15 +0000 (0:00:00.089) 0:00:34.254 ***** 2020-09-08T15:22:15.747 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-09-08T15:22:15.749 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-09-08T15:22:15.786 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:15 +0000 (0:00:00.051) 0:00:34.306 ***** 2020-09-08T15:22:15.844 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-09-08T15:22:15.846 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:15.892 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:15 +0000 (0:00:00.105) 0:00:34.411 ***** 2020-09-08T15:22:15.936 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-09-08T15:22:15.938 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:15.978 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:15 +0000 (0:00:00.084) 0:00:34.495 ***** 2020-09-08T15:22:15.990 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2020-09-08T15:22:16.027 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:16 +0000 (0:00:00.051) 0:00:34.546 ***** 2020-09-08T15:22:16.039 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-09-08T15:22:16.041 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:16.078 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:16 +0000 (0:00:00.050) 0:00:34.597 ***** 2020-09-08T15:22:16.109 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-09-08T15:22:16.118 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:16.146 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:16 +0000 (0:00:00.068) 0:00:34.665 ***** 2020-09-08T15:22:16.177 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:16.216 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:16 +0000 (0:00:00.070) 0:00:34.735 ***** 2020-09-08T15:22:16.234 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-09-08T15:22:16.236 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi166.front.sepia.ceph.com", "ansible_hostname": "smithi166", "ansible_nodename": "smithi166"}, "changed": true, "name": "smithi166"} 2020-09-08T15:22:16.277 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:16 +0000 (0:00:00.059) 0:00:34.795 ***** 2020-09-08T15:22:16.286 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-09-08T15:22:16.289 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:16.327 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:16 +0000 (0:00:00.050) 0:00:34.846 ***** 2020-09-08T15:22:16.357 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-09-08T15:22:16.359 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:16.395 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:16 +0000 (0:00:00.068) 0:00:34.914 ***** 2020-09-08T15:22:16.410 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2020-09-08T15:22:16.445 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:16 +0000 (0:00:00.049) 0:00:34.964 ***** 2020-09-08T15:22:16.458 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:16.495 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:16 +0000 (0:00:00.049) 0:00:35.014 ***** 2020-09-08T15:22:16.537 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-09-08T15:22:16.574 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:16 +0000 (0:00:00.079) 0:00:35.093 ***** 2020-09-08T15:22:16.605 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-09-08T15:22:16.648 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:16 +0000 (0:00:00.074) 0:00:35.168 ***** 2020-09-08T15:22:16.670 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2020-09-08T15:22:16.699 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:16 +0000 (0:00:00.050) 0:00:35.218 ***** 2020-09-08T15:22:16.709 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-09-08T15:22:16.749 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:16 +0000 (0:00:00.050) 0:00:35.268 ***** 2020-09-08T15:22:16.794 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:16.826 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:16 +0000 (0:00:00.076) 0:00:35.345 ***** 2020-09-08T15:22:16.886 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* skipping: [smithi055.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:16.924 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:16 +0000 (0:00:00.098) 0:00:35.443 ***** 2020-09-08T15:22:16.956 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-09-08T15:22:16.957 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:16.995 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:16 +0000 (0:00:00.071) 0:00:35.514 ***** 2020-09-08T15:22:17.030 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:17.070 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:17 +0000 (0:00:00.075) 0:00:35.590 ***** 2020-09-08T15:22:17.080 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-09-08T15:22:17.082 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2020-09-08T15:22:17.121 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:17 +0000 (0:00:00.049) 0:00:35.640 ***** 2020-09-08T15:22:17.134 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add ssh pubkeys] ********************************************** changed: [smithi104.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-09-08T15:22:17.169 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:17 +0000 (0:00:00.048) 0:00:35.689 ***** 2020-09-08T15:22:17.210 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:17 +0000 (0:00:00.040) 0:00:35.729 ***** 2020-09-08T15:22:17.219 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-09-08T15:22:17.221 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:17.287 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:17 +0000 (0:00:00.052) 0:00:35.781 ***** TASK [testnode : Create remote.conf] ******************************************* skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:17.313 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:17 +0000 (0:00:00.050) 0:00:35.832 ***** 2020-09-08T15:22:17.324 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-09-08T15:22:17.326 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:17.366 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:17 +0000 (0:00:00.050) 0:00:35.883 ***** 2020-09-08T15:22:17.375 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-09-08T15:22:17.376 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:17.415 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:17 +0000 (0:00:00.050) 0:00:35.934 ***** 2020-09-08T15:22:17.425 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-09-08T15:22:17.426 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:17.472 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:17 +0000 (0:00:00.055) 0:00:35.990 ***** 2020-09-08T15:22:17.480 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-09-08T15:22:17.482 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:17.519 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:17 +0000 (0:00:00.048) 0:00:36.038 ***** 2020-09-08T15:22:17.528 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-09-08T15:22:17.530 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:17.571 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:17 +0000 (0:00:00.048) 0:00:36.087 ***** 2020-09-08T15:22:17.588 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-09-08T15:22:17.590 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:17.629 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:17 +0000 (0:00:00.061) 0:00:36.149 ***** 2020-09-08T15:22:17.640 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-09-08T15:22:17.642 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:17.687 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:17 +0000 (0:00:00.057) 0:00:36.206 ***** 2020-09-08T15:22:17.697 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-09-08T15:22:17.699 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:17.742 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:17 +0000 (0:00:00.050) 0:00:36.257 ***** 2020-09-08T15:22:17.749 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:17.790 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:17 +0000 (0:00:00.051) 0:00:36.309 ***** 2020-09-08T15:22:17.802 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:17.806 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** changed: [smithi166.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-09-08T15:22:17.843 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:17 +0000 (0:00:00.053) 0:00:36.362 ***** 2020-09-08T15:22:17.890 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:17 +0000 (0:00:00.046) 0:00:36.409 ***** 2020-09-08T15:22:17.900 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-09-08T15:22:17.902 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:17.941 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:17 +0000 (0:00:00.050) 0:00:36.459 ***** 2020-09-08T15:22:17.979 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:17 +0000 (0:00:00.039) 0:00:36.498 ***** 2020-09-08T15:22:17.990 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-09-08T15:22:18.002 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove packages] ********************************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:18.029 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.050) 0:00:36.549 ***** 2020-09-08T15:22:18.075 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.045) 0:00:36.594 ***** 2020-09-08T15:22:18.090 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:18.126 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.051) 0:00:36.646 ***** 2020-09-08T15:22:18.171 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.044) 0:00:36.690 ***** 2020-09-08T15:22:18.182 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:18.187 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:18.225 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.054) 0:00:36.745 ***** 2020-09-08T15:22:18.270 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.044) 0:00:36.789 ***** 2020-09-08T15:22:18.280 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-09-08T15:22:18.282 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:18.326 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.055) 0:00:36.845 ***** 2020-09-08T15:22:18.366 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.040) 0:00:36.885 ***** 2020-09-08T15:22:18.376 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-09-08T15:22:18.381 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable SELinux] *********************************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:18.417 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.050) 0:00:36.936 ***** 2020-09-08T15:22:18.462 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.041) 0:00:36.977 ***** 2020-09-08T15:22:18.470 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** TASK [testnode : Install abrt] ************************************************* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:18.510 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.049) 0:00:37.027 ***** 2020-09-08T15:22:18.548 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.040) 0:00:37.067 ***** 2020-09-08T15:22:18.557 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-09-08T15:22:18.558 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable abrt-auto-reporting] *********************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:18.598 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.048) 0:00:37.115 ***** 2020-09-08T15:22:18.634 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.038) 0:00:37.153 ***** 2020-09-08T15:22:18.648 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:18.686 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.051) 0:00:37.205 ***** 2020-09-08T15:22:18.725 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.038) 0:00:37.244 ***** 2020-09-08T15:22:18.735 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-09-08T15:22:18.737 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:18.774 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.049) 0:00:37.294 ***** 2020-09-08T15:22:18.815 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.039) 0:00:37.334 ***** 2020-09-08T15:22:18.826 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-09-08T15:22:18.828 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Restart abrtd] ************************************************ skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:18.865 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.051) 0:00:37.385 ***** 2020-09-08T15:22:18.914 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.044) 0:00:37.429 ***** 2020-09-08T15:22:18.921 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-09-08T15:22:18.924 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:18.960 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:18 +0000 (0:00:00.048) 0:00:37.478 ***** 2020-09-08T15:22:18.985 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* skipping: [smithi104.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:19.025 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:19 +0000 (0:00:00.062) 0:00:37.541 ***** 2020-09-08T15:22:19.034 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-09-08T15:22:19.037 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for custom repos] *************************************** ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005220", "end": "2020-09-08 15:22:18.607578", "rc": 0, "start": "2020-09-08 15:22:18.602358", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:22:19.075 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:19 +0000 (0:00:00.053) 0:00:37.594 ***** 2020-09-08T15:22:19.121 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:19 +0000 (0:00:00.046) 0:00:37.640 ***** 2020-09-08T15:22:19.130 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-09-08T15:22:19.132 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove custom repos] ****************************************** 2020-09-08T15:22:19.171 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:19 +0000 (0:00:00.049) 0:00:37.690 ***** 2020-09-08T15:22:19.211 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:19 +0000 (0:00:00.039) 0:00:37.729 ***** 2020-09-08T15:22:19.219 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-09-08T15:22:19.221 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:19.262 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:19 +0000 (0:00:00.052) 0:00:37.782 ***** 2020-09-08T15:22:19.278 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:19.315 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:19 +0000 (0:00:00.052) 0:00:37.834 ***** 2020-09-08T15:22:19.327 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-09-08T15:22:19.369 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:19 +0000 (0:00:00.054) 0:00:37.889 ***** 2020-09-08T15:22:19.379 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-09-08T15:22:19.381 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:19.428 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:19 +0000 (0:00:00.058) 0:00:37.947 ***** 2020-09-08T15:22:19.442 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-09-08T15:22:19.479 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:19 +0000 (0:00:00.051) 0:00:37.998 ***** 2020-09-08T15:22:19.489 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-09-08T15:22:19.491 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:19.530 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:19 +0000 (0:00:00.051) 0:00:38.049 ***** 2020-09-08T15:22:19.543 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** TASK [testnode : Set apt preferences] ****************************************** ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-09-08T15:22:19.581 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:19 +0000 (0:00:00.051) 0:00:38.101 ***** 2020-09-08T15:22:19.620 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:19 +0000 (0:00:00.038) 0:00:38.140 ***** 2020-09-08T15:22:19.632 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-09-08T15:22:19.634 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add sources list] ********************************************* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:19.672 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:19 +0000 (0:00:00.051) 0:00:38.191 ***** 2020-09-08T15:22:19.721 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:19 +0000 (0:00:00.049) 0:00:38.240 ***** 2020-09-08T15:22:19.731 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-09-08T15:22:19.733 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:19.772 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:19 +0000 (0:00:00.051) 0:00:38.291 ***** 2020-09-08T15:22:19.782 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-09-08T15:22:19.784 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:19.823 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:19 +0000 (0:00:00.050) 0:00:38.342 ***** 2020-09-08T15:22:19.833 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-09-08T15:22:19.835 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:19.845 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:19.882 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:19 +0000 (0:00:00.060) 0:00:38.402 ***** 2020-09-08T15:22:19.892 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-09-08T15:22:19.894 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:19.941 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:19 +0000 (0:00:00.058) 0:00:38.460 ***** 2020-09-08T15:22:19.955 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:19.996 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:19 +0000 (0:00:00.055) 0:00:38.515 ***** 2020-09-08T15:22:20.005 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:20.053 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:20 +0000 (0:00:00.057) 0:00:38.573 ***** 2020-09-08T15:22:20.063 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-08T15:22:20.065 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:20.104 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:20 +0000 (0:00:00.050) 0:00:38.623 ***** 2020-09-08T15:22:20.119 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-09-08T15:22:20.121 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:20.165 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:20 +0000 (0:00:00.060) 0:00:38.683 ***** 2020-09-08T15:22:20.175 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-09-08T15:22:20.176 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:20.218 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:20 +0000 (0:00:00.053) 0:00:38.737 ***** 2020-09-08T15:22:20.229 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-09-08T15:22:20.231 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:20.282 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:20 +0000 (0:00:00.054) 0:00:38.791 ***** 2020-09-08T15:22:20.285 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:20.327 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:20 +0000 (0:00:00.055) 0:00:38.846 ***** 2020-09-08T15:22:20.337 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-09-08T15:22:20.339 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:20.376 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:20 +0000 (0:00:00.048) 0:00:38.895 ***** 2020-09-08T15:22:20.386 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-09-08T15:22:20.387 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:20.431 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:20 +0000 (0:00:00.055) 0:00:38.951 ***** 2020-09-08T15:22:20.447 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:20.488 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:20 +0000 (0:00:00.056) 0:00:39.007 ***** 2020-09-08T15:22:20.500 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-09-08T15:22:20.502 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:20.549 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:20 +0000 (0:00:00.061) 0:00:39.068 ***** 2020-09-08T15:22:20.562 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:20.605 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:20 +0000 (0:00:00.051) 0:00:39.119 ***** 2020-09-08T15:22:20.611 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-09-08T15:22:20.612 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:20.650 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:20 +0000 (0:00:00.049) 0:00:39.169 ***** 2020-09-08T15:22:20.659 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-09-08T15:22:20.663 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:20.702 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:20 +0000 (0:00:00.049) 0:00:39.218 ***** 2020-09-08T15:22:20.711 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:20.756 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:20 +0000 (0:00:00.056) 0:00:39.275 ***** 2020-09-08T15:22:20.771 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:20.809 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:20 +0000 (0:00:00.053) 0:00:39.328 ***** 2020-09-08T15:22:20.822 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-09-08T15:22:20.824 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:20.862 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:20 +0000 (0:00:00.053) 0:00:39.382 ***** 2020-09-08T15:22:20.873 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-09-08T15:22:20.875 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:20.915 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:20 +0000 (0:00:00.052) 0:00:39.434 ***** 2020-09-08T15:22:20.926 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-09-08T15:22:20.926 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:20.973 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:20 +0000 (0:00:00.057) 0:00:39.492 ***** 2020-09-08T15:22:20.983 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2020-09-08T15:22:20.986 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:21.032 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:21 +0000 (0:00:00.054) 0:00:39.547 ***** 2020-09-08T15:22:21.042 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:21.089 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:21 +0000 (0:00:00.061) 0:00:39.608 ***** 2020-09-08T15:22:21.101 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-09-08T15:22:21.103 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-09-08T15:22:21.115 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:21.161 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:21 +0000 (0:00:00.072) 0:00:39.680 ***** 2020-09-08T15:22:21.172 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-09-08T15:22:21.174 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:21.213 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:21 +0000 (0:00:00.051) 0:00:39.732 ***** 2020-09-08T15:22:21.224 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-09-08T15:22:21.225 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:21.262 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:21 +0000 (0:00:00.049) 0:00:39.782 ***** 2020-09-08T15:22:21.272 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-09-08T15:22:21.274 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:21.317 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:21 +0000 (0:00:00.054) 0:00:39.836 ***** 2020-09-08T15:22:21.336 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:21.368 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:21 +0000 (0:00:00.049) 0:00:39.886 ***** 2020-09-08T15:22:21.378 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-09-08T15:22:21.382 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:21.422 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:21 +0000 (0:00:00.049) 0:00:39.936 ***** 2020-09-08T15:22:21.451 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:21.496 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:21 +0000 (0:00:00.078) 0:00:40.014 ***** 2020-09-08T15:22:21.522 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005026", "end": "2020-09-08 15:22:21.900341", "rc": 0, "start": "2020-09-08 15:22:21.895315", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:22:21.562 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:21 +0000 (0:00:00.066) 0:00:40.081 ***** 2020-09-08T15:22:21.574 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-09-08T15:22:21.576 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:21.623 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:21 +0000 (0:00:00.057) 0:00:40.139 ***** 2020-09-08T15:22:21.630 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-09-08T15:22:21.674 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:21 +0000 (0:00:00.052) 0:00:40.191 ***** 2020-09-08T15:22:21.682 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:21.724 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:21 +0000 (0:00:00.051) 0:00:40.243 ***** 2020-09-08T15:22:21.763 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:21.801 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:21 +0000 (0:00:00.077) 0:00:40.320 ***** 2020-09-08T15:22:21.843 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:21.889 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:21 +0000 (0:00:00.087) 0:00:40.408 ***** 2020-09-08T15:22:21.932 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:21.979 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:21 +0000 (0:00:00.090) 0:00:40.498 ***** 2020-09-08T15:22:22.011 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-09-08T15:22:22.013 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-09-08T15:22:22.054 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:22 +0000 (0:00:00.075) 0:00:40.573 ***** 2020-09-08T15:22:22.077 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-09-08T15:22:22.078 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-09-08T15:22:22.116 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:22 +0000 (0:00:00.062) 0:00:40.636 ***** 2020-09-08T15:22:22.138 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-09-08T15:22:22.148 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.004786", "end": "2020-09-08 15:22:21.863958", "rc": 0, "start": "2020-09-08 15:22:21.859172", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:22:22.187 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:22 +0000 (0:00:00.070) 0:00:40.706 ***** 2020-09-08T15:22:22.199 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:22.239 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:22 +0000 (0:00:00.051) 0:00:40.758 ***** 2020-09-08T15:22:22.252 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-09-08T15:22:22.287 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:22 +0000 (0:00:00.048) 0:00:40.806 ***** 2020-09-08T15:22:22.662 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-09-08T15:22:22.703 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:22 +0000 (0:00:00.415) 0:00:41.221 ***** 2020-09-08T15:22:22.746 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:22.789 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:22 +0000 (0:00:00.086) 0:00:41.308 ***** 2020-09-08T15:22:22.991 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* ok: [smithi055.front.sepia.ceph.com] => {"cache_update_time": 1599578284, "cache_updated": false, "changed": false} 2020-09-08T15:22:23.027 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:23 +0000 (0:00:00.238) 0:00:41.546 ***** 2020-09-08T15:22:23.075 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-09-08T15:22:23.119 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:23 +0000 (0:00:00.092) 0:00:41.638 ***** 2020-09-08T15:22:23.282 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-09-08T15:22:23.284 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-09-08T15:22:23.316 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-09-08T15:22:23.318 INFO:teuthology.task.ansible.out:[WARNING]: Consider using 'become', 'become_method', and 'become_user' rather than running sudo changed: [smithi055.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.035879", "end": "2020-09-08 15:22:22.895720", "rc": 0, "start": "2020-09-08 15:22:22.859841", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:22:23.359 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:23 +0000 (0:00:00.240) 0:00:41.878 ***** 2020-09-08T15:22:23.831 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* ok: [smithi166.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-09-08T15:22:24.218 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-09-08T15:22:24.260 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:24 +0000 (0:00:00.899) 0:00:42.778 ***** 2020-09-08T15:22:24.770 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-09-08T15:22:24.821 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:24 +0000 (0:00:00.561) 0:00:43.340 ***** 2020-09-08T15:22:25.188 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-09-08T15:22:25.190 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"cache_update_time": 1599578240, "cache_updated": false, "changed": false} 2020-09-08T15:22:25.233 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:25 +0000 (0:00:00.412) 0:00:43.752 ***** 2020-09-08T15:22:25.267 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-09-08T15:22:25.307 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:25 +0000 (0:00:00.073) 0:00:43.826 ***** 2020-09-08T15:22:25.494 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** changed: [smithi104.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.029730", "end": "2020-09-08 15:22:25.876162", "rc": 0, "start": "2020-09-08 15:22:25.846432", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:22:25.539 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:25 +0000 (0:00:00.229) 0:00:44.055 ***** 2020-09-08T15:22:25.734 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-09-08T15:22:25.736 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"cache_update_time": 1599578328, "cache_updated": false, "changed": false} 2020-09-08T15:22:25.780 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:25 +0000 (0:00:00.244) 0:00:44.300 ***** 2020-09-08T15:22:25.812 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-09-08T15:22:25.857 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:25 +0000 (0:00:00.076) 0:00:44.376 ***** 2020-09-08T15:22:26.060 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** changed: [smithi166.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.030959", "end": "2020-09-08 15:22:25.786535", "rc": 0, "start": "2020-09-08 15:22:25.755576", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:22:26.096 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:26 +0000 (0:00:00.238) 0:00:44.615 ***** 2020-09-08T15:22:26.421 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** ok: [smithi055.front.sepia.ceph.com] => {"cache_update_time": 1599578284, "cache_updated": false, "changed": false} 2020-09-08T15:22:26.474 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:26 +0000 (0:00:00.372) 0:00:44.987 ***** 2020-09-08T15:22:28.357 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"cache_update_time": 1599578240, "cache_updated": false, "changed": false} 2020-09-08T15:22:28.407 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:28 +0000 (0:00:01.938) 0:00:46.926 ***** 2020-09-08T15:22:29.052 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"cache_update_time": 1599578328, "cache_updated": false, "changed": false} 2020-09-08T15:22:29.098 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:29 +0000 (0:00:00.684) 0:00:47.611 ***** 2020-09-08T15:22:35.832 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** changed: [smithi055.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 9 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 2020-09-08T15:22:35.838 INFO:teuthology.task.ansible.out: 95%\r(Reading database ... 100%\r(Reading database ... 137585 files and directories currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.31) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.31) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.31) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.31) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.2) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.2) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 9 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading d 2020-09-08T15:22:35.840 INFO:teuthology.task.ansible.out:atabase ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137585 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.31) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.31) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.31) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.31) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.2) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.2) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.2) ..."]} 2020-09-08T15:22:35.889 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:35 +0000 (0:00:06.797) 0:00:54.408 ***** 2020-09-08T15:22:36.819 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-09-08T15:22:36.822 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"cache_update_time": 1599578284, "cache_updated": false, "changed": false} 2020-09-08T15:22:36.859 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:36 +0000 (0:00:00.970) 0:00:55.379 ***** 2020-09-08T15:22:36.902 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-09-08T15:22:36.905 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:36.941 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:36 +0000 (0:00:00.081) 0:00:55.461 ***** 2020-09-08T15:22:37.336 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** changed: [smithi104.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 9 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 2020-09-08T15:22:37.339 INFO:teuthology.task.ansible.out: 95%\r(Reading database ... 100%\r(Reading database ... 137585 files and directories currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.31) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.31) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.31) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.31) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.2) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.2) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 9 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading d 2020-09-08T15:22:37.340 INFO:teuthology.task.ansible.out:atabase ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137585 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.31) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.31) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.31) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.31) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.2) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.2) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.2) ..."]} 2020-09-08T15:22:37.389 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:37 +0000 (0:00:00.447) 0:00:55.909 ***** 2020-09-08T15:22:38.300 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-09-08T15:22:38.302 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"cache_update_time": 1599578240, "cache_updated": false, "changed": false} 2020-09-08T15:22:38.342 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:38 +0000 (0:00:00.950) 0:00:56.859 ***** 2020-09-08T15:22:38.382 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:38.429 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:38 +0000 (0:00:00.086) 0:00:56.946 ***** 2020-09-08T15:22:38.540 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-09-08T15:22:38.546 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 9 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137585 files and directories 2020-09-08T15:22:38.548 INFO:teuthology.task.ansible.out: currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.31) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.31) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.31) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.31) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.2) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.2) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 9 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Readin 2020-09-08T15:22:38.549 INFO:teuthology.task.ansible.out:g database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137585 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.31) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.31) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.31) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.31) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.2) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.2) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.2) ..."]} 2020-09-08T15:22:38.585 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:38 +0000 (0:00:00.156) 0:00:57.102 ***** 2020-09-08T15:22:39.490 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-09-08T15:22:39.492 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"cache_update_time": 1599578328, "cache_updated": false, "changed": false} 2020-09-08T15:22:39.537 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:39 +0000 (0:00:00.954) 0:00:58.056 ***** 2020-09-08T15:22:39.569 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:39.614 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:39 +0000 (0:00:00.077) 0:00:58.134 ***** 2020-09-08T15:22:45.907 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-09-08T15:22:45.909 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => {"cache_update_time": 1599578284, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 9 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.3 [2723 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.3 [922 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.31 [40.4 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.31 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.31 [5190 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.31 [872 kB]\nFetched 10.4 MB in 1s (8537 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 2020-09-08T15:22:45.911 INFO:teuthology.task.ansible.out: 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137443 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.3_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.3) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.3_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.3) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.31) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.31) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.31) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.31) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.3) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.3) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.31) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.31) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.31) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.31) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu 2020-09-08T15:22:45.912 INFO:teuthology.task.ansible.out:1.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 9 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.3 [2723 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.3 [922 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.31 [40.4 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.31 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.31 [5190 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.31 [872 kB]", "Fetched 10.4 MB in 1s (8537 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Rea 2020-09-08T15:22:45.914 INFO:teuthology.task.ansible.out:ding database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137443 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.3_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.3) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.3_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.3) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.31) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.31) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.31) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.31) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.3) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.3) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.31) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.31) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.31) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.31) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.2) ..."]} 2020-09-08T15:22:45.948 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:45 +0000 (0:00:06.333) 0:01:04.467 ***** 2020-09-08T15:22:46.858 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-09-08T15:22:46.860 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"cache_update_time": 1599578284, "cache_updated": false, "changed": false} 2020-09-08T15:22:46.898 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:46 +0000 (0:00:00.949) 0:01:05.417 ***** 2020-09-08T15:22:47.045 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-09-08T15:22:47.047 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-09-08T15:22:47.084 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:47 +0000 (0:00:00.185) 0:01:05.603 ***** 2020-09-08T15:22:47.116 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-09-08T15:22:47.118 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:47.157 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:47 +0000 (0:00:00.073) 0:01:05.677 ***** 2020-09-08T15:22:47.306 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-09-08T15:22:47.344 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:47 +0000 (0:00:00.186) 0:01:05.863 ***** 2020-09-08T15:22:47.346 INFO:teuthology.task.ansible.out: 2020-09-08T15:22:47.513 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-09-08T15:22:47.515 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => {"cache_update_time": 1599578240, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 9 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.3 [2723 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.3 [922 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.31 [40.4 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.31 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.31 [5190 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.31 [872 kB]\nFetched 10.4 MB in 1s (8447 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 2020-09-08T15:22:47.516 INFO:teuthology.task.ansible.out: 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137443 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.3_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.3) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.3_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.3) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.31) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.31) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.31) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.31) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.3) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.3) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.31) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.31) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.31) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.31) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu 2020-09-08T15:22:47.518 INFO:teuthology.task.ansible.out:1.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 9 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.3 [2723 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.3 [922 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.31 [40.4 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.31 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.31 [5190 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.31 [872 kB]", "Fetched 10.4 MB in 1s (8447 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Rea 2020-09-08T15:22:47.521 INFO:teuthology.task.ansible.out:ding database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137443 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.3_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.3) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.3_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.3) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.31) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.31) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.31) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.31) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.3) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.3) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.31) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.31) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.31) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.31) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.2) ..."]} 2020-09-08T15:22:47.576 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:47 +0000 (0:00:00.225) 0:01:06.089 ***** 2020-09-08T15:22:47.750 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-09-08T15:22:47.790 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:47 +0000 (0:00:00.220) 0:01:06.309 ***** 2020-09-08T15:22:47.970 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ ok: [smithi055.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1229, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-09-08T15:22:48.013 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:48 +0000 (0:00:00.222) 0:01:06.532 ***** 2020-09-08T15:22:48.165 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** ok: [smithi055.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-09-08T15:22:48.199 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:48 +0000 (0:00:00.186) 0:01:06.719 ***** 2020-09-08T15:22:48.469 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* changed: [smithi166.front.sepia.ceph.com] => {"cache_update_time": 1599578328, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 9 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.3 [2723 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.3 [922 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.31 [40.4 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.31 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.31 [5190 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.31 [872 kB]\nFetched 10.4 MB in 1s (8753 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database 2020-09-08T15:22:48.471 INFO:teuthology.task.ansible.out:... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137443 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.3_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.3) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.3_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.3) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.31) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.31) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.31) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.31) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.3) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.3) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.31) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.31) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.31) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.31) ...\r\nProcessing triggers 2020-09-08T15:22:48.473 INFO:teuthology.task.ansible.out: for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 9 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.3 [2723 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.3 [922 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.31 [40.4 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.31 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.31 [5190 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.31 [872 kB]", "Fetched 10.4 MB in 1s (8753 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Readin 2020-09-08T15:22:48.475 INFO:teuthology.task.ansible.out:g database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137443 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.3_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.3) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.3_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.3) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.31) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.31) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.31) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.31_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.31) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.3) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.3) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.31) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.31) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.31) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.31) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.2) ..."]} 2020-09-08T15:22:48.515 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:48 +0000 (0:00:00.314) 0:01:07.034 ***** 2020-09-08T15:22:48.533 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-09-08T15:22:48.535 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"cache_update_time": 1599578240, "cache_updated": false, "changed": false} TASK [testnode : Stop apache2] ************************************************* 2020-09-08T15:22:48.537 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Tue 2020-09-08 15:21:12 UTC", "ActiveEnterTimestampMonotonic": "19731240", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sysinit.target systemd-tmpfiles-setup.service nss-lookup.target remote-fs.target system.slice network.target systemd-journald.socket basic.target -.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Tue 2020-09-08 15:21:11 UTC", "AssertTimestampMonotonic": "19089084", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Tue 2020-09-08 15:21:11 UTC", "ConditionTimestampMonotonic": "19089084", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2020-09-08T15:22:48.539 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1362", "ExecMainStartTimestamp": "Tue 2020-09-08 15:21:12 UTC", "ExecMainStartTimestampMonotonic": "19731229", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Tue 2020-09-08 15:21:11 UTC", "InactiveExitTimestampMonotonic": "19090114", "InvocationID": "bffdd1d604d4497085a1a0d0e889f26f", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2020-09-08T15:22:48.541 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128055", "LimitNPROCSoft": "128055", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128055", "LimitSIGPENDINGSoft": "128055", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1362", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2020-09-08T15:22:48.543 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Tue 2020-09-08 15:21:12 UTC", "StateChangeTimestampMonotonic": "19731240", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Tue 2020-09-08 15:21:12 UTC", "WatchdogTimestampMonotonic": "19731239", "WatchdogUSec": "0"}} 2020-09-08T15:22:48.575 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:48 +0000 (0:00:00.060) 0:01:07.095 ***** 2020-09-08T15:22:48.577 INFO:teuthology.task.ansible.out: 2020-09-08T15:22:48.615 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:48 +0000 (0:00:00.039) 0:01:07.135 ***** 2020-09-08T15:22:48.625 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-09-08T15:22:48.627 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:48.664 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:48 +0000 (0:00:00.049) 0:01:07.184 ***** 2020-09-08T15:22:48.706 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-09-08T15:22:48.708 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:48.743 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:48 +0000 (0:00:00.078) 0:01:07.263 ***** 2020-09-08T15:22:48.755 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-09-08T15:22:48.757 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-09-08T15:22:48.794 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:48 +0000 (0:00:00.050) 0:01:07.313 ***** 2020-09-08T15:22:48.806 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-09-08T15:22:48.808 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:48.848 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:48 +0000 (0:00:00.054) 0:01:07.368 ***** 2020-09-08T15:22:48.859 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-09-08T15:22:48.862 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:48.903 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:48 +0000 (0:00:00.054) 0:01:07.422 ***** 2020-09-08T15:22:48.916 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:48.952 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:48 +0000 (0:00:00.049) 0:01:07.471 ***** 2020-09-08T15:22:48.983 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-09-08T15:22:48.985 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:49.021 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:49 +0000 (0:00:00.067) 0:01:07.539 ***** 2020-09-08T15:22:49.051 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-09-08T15:22:49.054 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:49.098 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:49 +0000 (0:00:00.071) 0:01:07.610 ***** 2020-09-08T15:22:49.134 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:49 +0000 (0:00:00.042) 0:01:07.653 ***** 2020-09-08T15:22:49.144 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-09-08T15:22:49.146 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:49.191 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:49 +0000 (0:00:00.057) 0:01:07.710 ***** 2020-09-08T15:22:49.234 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:49.272 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:49 +0000 (0:00:00.081) 0:01:07.791 ***** 2020-09-08T15:22:49.307 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-08T15:22:49.310 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:49.348 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:49 +0000 (0:00:00.076) 0:01:07.868 ***** 2020-09-08T15:22:49.381 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-09-08T15:22:49.383 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:49.421 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:49 +0000 (0:00:00.072) 0:01:07.941 ***** 2020-09-08T15:22:49.432 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-09-08T15:22:49.434 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"cache_update_time": 1599578328, "cache_updated": false, "changed": false} 2020-09-08T15:22:49.477 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:49 +0000 (0:00:00.051) 0:01:07.992 ***** 2020-09-08T15:22:49.486 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:49.530 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:49 +0000 (0:00:00.057) 0:01:08.049 ***** 2020-09-08T15:22:49.540 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-09-08T15:22:49.544 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-09-08T15:22:49.591 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:49 +0000 (0:00:00.060) 0:01:08.109 ***** 2020-09-08T15:22:49.603 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:49.642 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:49 +0000 (0:00:00.050) 0:01:08.159 ***** 2020-09-08T15:22:49.652 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-09-08T15:22:49.654 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-09-08T15:22:49.691 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:49 +0000 (0:00:00.051) 0:01:08.211 ***** 2020-09-08T15:22:49.711 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:49.749 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:49 +0000 (0:00:00.057) 0:01:08.268 ***** 2020-09-08T15:22:49.766 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ ok: [smithi104.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1229, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-09-08T15:22:49.804 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:49 +0000 (0:00:00.054) 0:01:08.323 ***** 2020-09-08T15:22:49.847 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:49 +0000 (0:00:00.042) 0:01:08.366 ***** 2020-09-08T15:22:49.859 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:49.909 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:49 +0000 (0:00:00.062) 0:01:08.428 ***** 2020-09-08T15:22:49.953 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-09-08T15:22:49.955 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:50.000 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:49 +0000 (0:00:00.090) 0:01:08.519 ***** 2020-09-08T15:22:50.018 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** ok: [smithi104.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-09-08T15:22:50.052 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:50 +0000 (0:00:00.052) 0:01:08.571 ***** 2020-09-08T15:22:50.234 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-09-08T15:22:50.237 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:50.272 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:50 +0000 (0:00:00.219) 0:01:08.791 ***** 2020-09-08T15:22:50.318 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:50 +0000 (0:00:00.046) 0:01:08.837 ***** 2020-09-08T15:22:50.328 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-09-08T15:22:50.330 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:50.369 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:50 +0000 (0:00:00.051) 0:01:08.888 ***** 2020-09-08T15:22:50.403 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-09-08T15:22:50.410 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:50.440 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:50 +0000 (0:00:00.071) 0:01:08.959 ***** 2020-09-08T15:22:50.471 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-09-08T15:22:50.473 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:50.509 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:50 +0000 (0:00:00.068) 0:01:09.028 ***** 2020-09-08T15:22:50.530 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-09-08T15:22:50.531 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Tue 2020-09-08 15:21:19 UTC", "ActiveEnterTimestampMonotonic": "23110524", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target basic.target nss-lookup.target systemd-tmpfiles-setup.service -.mount system.slice sysinit.target systemd-journald.socket remote-fs.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Tue 2020-09-08 15:21:19 UTC", "AssertTimestampMonotonic": "22386232", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Tue 2020-09-08 15:21:19 UTC", "ConditionTimestampMonotonic": "22386232", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2020-09-08T15:22:50.532 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1203", "ExecMainStartTimestamp": "Tue 2020-09-08 15:21:19 UTC", "ExecMainStartTimestampMonotonic": "23110479", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Tue 2020-09-08 15:21:19 UTC", "InactiveExitTimestampMonotonic": "22387976", "InvocationID": "3ef7a61e12f544a78a2ca85ab86d460f", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2020-09-08T15:22:50.534 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128031", "LimitNPROCSoft": "128031", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128031", "LimitSIGPENDINGSoft": "128031", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1203", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2020-09-08T15:22:50.536 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Tue 2020-09-08 15:21:19 UTC", "StateChangeTimestampMonotonic": "23110524", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Tue 2020-09-08 15:21:19 UTC", "WatchdogTimestampMonotonic": "23110521", "WatchdogUSec": "0"}} 2020-09-08T15:22:50.569 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:50 +0000 (0:00:00.060) 0:01:09.088 ***** 2020-09-08T15:22:50.578 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-09-08T15:22:50.581 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:50.618 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:50 +0000 (0:00:00.047) 0:01:09.136 ***** 2020-09-08T15:22:50.629 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:50.666 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:50 +0000 (0:00:00.048) 0:01:09.185 ***** 2020-09-08T15:22:50.677 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:50.714 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:50 +0000 (0:00:00.048) 0:01:09.233 ***** 2020-09-08T15:22:50.728 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-09-08T15:22:50.730 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload /etc/fuse.conf.] *************************************** ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-09-08T15:22:50.767 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:50 +0000 (0:00:00.052) 0:01:09.286 ***** 2020-09-08T15:22:50.803 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:50 +0000 (0:00:00.036) 0:01:09.322 ***** 2020-09-08T15:22:50.812 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-09-08T15:22:50.813 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:50.850 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:50 +0000 (0:00:00.046) 0:01:09.369 ***** 2020-09-08T15:22:50.893 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:50 +0000 (0:00:00.042) 0:01:09.412 ***** 2020-09-08T15:22:50.903 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-09-08T15:22:50.905 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:50.944 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:50 +0000 (0:00:00.051) 0:01:09.463 ***** 2020-09-08T15:22:50.959 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:50.995 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:50 +0000 (0:00:00.050) 0:01:09.514 ***** 2020-09-08T15:22:51.005 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-09-08T15:22:51.011 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1229, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:51.045 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:51 +0000 (0:00:00.050) 0:01:09.564 ***** 2020-09-08T15:22:51.084 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:51 +0000 (0:00:00.038) 0:01:09.603 ***** 2020-09-08T15:22:51.095 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-09-08T15:22:51.097 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:51.140 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:51 +0000 (0:00:00.056) 0:01:09.660 ***** 2020-09-08T15:22:51.178 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:51 +0000 (0:00:00.037) 0:01:09.698 ***** 2020-09-08T15:22:51.189 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-09-08T15:22:51.191 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:51.227 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:51 +0000 (0:00:00.049) 0:01:09.747 ***** 2020-09-08T15:22:51.238 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-09-08T15:22:51.240 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:51.278 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:51 +0000 (0:00:00.048) 0:01:09.796 ***** 2020-09-08T15:22:51.287 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-09-08T15:22:51.289 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-09-08T15:22:51.326 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:51 +0000 (0:00:00.050) 0:01:09.846 ***** 2020-09-08T15:22:51.343 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-09-08T15:22:51.344 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:51.393 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:51 +0000 (0:00:00.066) 0:01:09.913 ***** 2020-09-08T15:22:51.405 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-09-08T15:22:51.407 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-09-08T15:22:51.446 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:51 +0000 (0:00:00.049) 0:01:09.963 ***** 2020-09-08T15:22:51.455 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-09-08T15:22:51.457 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:51.494 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:51 +0000 (0:00:00.050) 0:01:10.013 ***** 2020-09-08T15:22:51.543 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:51.579 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:51 +0000 (0:00:00.085) 0:01:10.099 ***** 2020-09-08T15:22:51.617 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-09-08T15:22:51.618 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:51.654 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:51 +0000 (0:00:00.074) 0:01:10.173 ***** 2020-09-08T15:22:51.665 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-09-08T15:22:51.667 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Tue 2020-09-08 15:21:32 UTC", "ActiveEnterTimestampMonotonic": "22338827", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "system.slice -.mount systemd-tmpfiles-setup.service network.target basic.target sysinit.target systemd-journald.socket remote-fs.target nss-lookup.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Tue 2020-09-08 15:21:31 UTC", "AssertTimestampMonotonic": "21655666", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Tue 2020-09-08 15:21:31 UTC", "ConditionTimestampMonotonic": "21655666", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2020-09-08T15:22:51.669 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1476", "ExecMainStartTimestamp": "Tue 2020-09-08 15:21:32 UTC", "ExecMainStartTimestampMonotonic": "22338799", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Tue 2020-09-08 15:21:31 UTC", "InactiveExitTimestampMonotonic": "21657390", "InvocationID": "74020e63d7294e049eb19193d1001aef", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2020-09-08T15:22:51.670 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128030", "LimitNPROCSoft": "128030", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128030", "LimitSIGPENDINGSoft": "128030", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1476", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2020-09-08T15:22:51.674 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Tue 2020-09-08 15:21:32 UTC", "StateChangeTimestampMonotonic": "22338827", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Tue 2020-09-08 15:21:32 UTC", "WatchdogTimestampMonotonic": "22338825", "WatchdogUSec": "0"}} 2020-09-08T15:22:51.706 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:51 +0000 (0:00:00.052) 0:01:10.226 ***** 2020-09-08T15:22:51.714 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-09-08T15:22:51.716 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:51.761 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:51 +0000 (0:00:00.054) 0:01:10.280 ***** 2020-09-08T15:22:51.771 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-09-08T15:22:51.772 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:51.809 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:51 +0000 (0:00:00.048) 0:01:10.328 ***** 2020-09-08T15:22:51.819 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-09-08T15:22:51.821 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:51.858 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:51 +0000 (0:00:00.049) 0:01:10.378 ***** 2020-09-08T15:22:51.868 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-09-08T15:22:51.870 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-09-08T15:22:51.908 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:51 +0000 (0:00:00.049) 0:01:10.427 ***** 2020-09-08T15:22:51.947 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:51 +0000 (0:00:00.038) 0:01:10.465 ***** 2020-09-08T15:22:51.962 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:51.994 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:51 +0000 (0:00:00.047) 0:01:10.513 ***** 2020-09-08T15:22:52.032 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:52 +0000 (0:00:00.037) 0:01:10.551 ***** 2020-09-08T15:22:52.040 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-09-08T15:22:52.042 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:52.079 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:52 +0000 (0:00:00.047) 0:01:10.599 ***** 2020-09-08T15:22:52.089 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-09-08T15:22:52.091 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:52.129 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:52 +0000 (0:00:00.049) 0:01:10.648 ***** 2020-09-08T15:22:52.139 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-09-08T15:22:52.141 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:52.181 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:52 +0000 (0:00:00.051) 0:01:10.700 ***** 2020-09-08T15:22:52.191 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-09-08T15:22:52.192 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:52.234 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:52 +0000 (0:00:00.052) 0:01:10.752 ***** 2020-09-08T15:22:52.244 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-09-08T15:22:52.247 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:52.286 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:52 +0000 (0:00:00.053) 0:01:10.805 ***** 2020-09-08T15:22:52.297 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-09-08T15:22:52.298 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:52.346 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:52 +0000 (0:00:00.060) 0:01:10.865 ***** 2020-09-08T15:22:52.370 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload weekly kernel-clean crontab.] ************************** ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-09-08T15:22:52.414 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:52 +0000 (0:00:00.068) 0:01:10.934 ***** 2020-09-08T15:22:52.455 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:52 +0000 (0:00:00.039) 0:01:10.973 ***** 2020-09-08T15:22:52.466 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:52.510 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:52 +0000 (0:00:00.055) 0:01:11.029 ***** 2020-09-08T15:22:52.521 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-09-08T15:22:52.523 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:52.560 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:52 +0000 (0:00:00.050) 0:01:11.079 ***** 2020-09-08T15:22:52.570 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-09-08T15:22:52.572 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:52.610 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:52 +0000 (0:00:00.049) 0:01:11.129 ***** 2020-09-08T15:22:52.623 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:52.659 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:52 +0000 (0:00:00.049) 0:01:11.179 ***** 2020-09-08T15:22:52.668 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-09-08T15:22:52.669 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:52.708 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:52 +0000 (0:00:00.048) 0:01:11.227 ***** 2020-09-08T15:22:52.719 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-08T15:22:52.721 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:52.765 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:52 +0000 (0:00:00.049) 0:01:11.277 ***** 2020-09-08T15:22:52.771 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-09-08T15:22:52.773 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:52.810 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:52 +0000 (0:00:00.052) 0:01:11.330 ***** 2020-09-08T15:22:52.822 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-09-08T15:22:52.824 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-09-08T15:22:52.867 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:52 +0000 (0:00:00.055) 0:01:11.386 ***** 2020-09-08T15:22:52.905 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:52 +0000 (0:00:00.038) 0:01:11.424 ***** 2020-09-08T15:22:52.913 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-09-08T15:22:52.924 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:52.953 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:52 +0000 (0:00:00.048) 0:01:11.472 ***** 2020-09-08T15:22:52.971 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-09-08T15:22:52.972 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:53.012 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:53 +0000 (0:00:00.058) 0:01:11.531 ***** 2020-09-08T15:22:53.022 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:53.068 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:53 +0000 (0:00:00.055) 0:01:11.587 ***** 2020-09-08T15:22:53.090 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:53.123 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:53 +0000 (0:00:00.055) 0:01:11.642 ***** 2020-09-08T15:22:53.133 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-09-08T15:22:53.142 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:53.180 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:53 +0000 (0:00:00.055) 0:01:11.698 ***** 2020-09-08T15:22:53.188 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-09-08T15:22:53.190 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:53.228 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:53 +0000 (0:00:00.049) 0:01:11.748 ***** 2020-09-08T15:22:53.241 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-09-08T15:22:53.244 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable kernel modules to load at boot time.] ****************** ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-09-08T15:22:53.280 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:53 +0000 (0:00:00.051) 0:01:11.799 ***** 2020-09-08T15:22:53.319 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:53 +0000 (0:00:00.039) 0:01:11.838 ***** 2020-09-08T15:22:53.332 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:53.377 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:53 +0000 (0:00:00.051) 0:01:11.890 ***** 2020-09-08T15:22:53.386 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-09-08T15:22:53.388 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:53.434 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:53 +0000 (0:00:00.059) 0:01:11.949 ***** 2020-09-08T15:22:53.442 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-09-08T15:22:53.443 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:53.482 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:53 +0000 (0:00:00.052) 0:01:12.002 ***** 2020-09-08T15:22:53.503 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** ok: [smithi055.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} TASK [testnode : Set grub config.] ********************************************* skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:53.536 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:53 +0000 (0:00:00.053) 0:01:12.055 ***** 2020-09-08T15:22:53.582 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:53 +0000 (0:00:00.044) 0:01:12.100 ***** 2020-09-08T15:22:53.592 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-09-08T15:22:53.595 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:53.634 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:53 +0000 (0:00:00.053) 0:01:12.153 ***** 2020-09-08T15:22:53.675 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:53 +0000 (0:00:00.041) 0:01:12.194 ***** 2020-09-08T15:22:53.685 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-09-08T15:22:53.690 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:53.728 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:53 +0000 (0:00:00.051) 0:01:12.245 ***** 2020-09-08T15:22:53.736 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove /etc/ceph] ********************************************* 2020-09-08T15:22:53.742 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-09-08T15:22:53.777 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:53 +0000 (0:00:00.050) 0:01:12.296 ***** 2020-09-08T15:22:53.832 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:53 +0000 (0:00:00.051) 0:01:12.347 ***** 2020-09-08T15:22:53.838 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-09-08T15:22:53.840 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:53.895 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:53 +0000 (0:00:00.066) 0:01:12.414 ***** 2020-09-08T15:22:53.910 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-09-08T15:22:53.912 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:53.975 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:53 +0000 (0:00:00.080) 0:01:12.494 ***** 2020-09-08T15:22:53.990 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-09-08T15:22:53.992 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:54.039 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:54 +0000 (0:00:00.064) 0:01:12.558 ***** 2020-09-08T15:22:54.050 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-09-08T15:22:54.052 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:54.097 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:54 +0000 (0:00:00.058) 0:01:12.617 ***** 2020-09-08T15:22:54.108 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-09-08T15:22:54.109 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:54.148 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:54 +0000 (0:00:00.050) 0:01:12.667 ***** 2020-09-08T15:22:54.160 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-09-08T15:22:54.162 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:54.207 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:54 +0000 (0:00:00.058) 0:01:12.726 ***** 2020-09-08T15:22:54.218 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-09-08T15:22:54.219 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} TASK [testnode : set_fact] ***************************************************** ok: [smithi055.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-09-08T15:22:54.265 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:54 +0000 (0:00:00.053) 0:01:12.780 ***** 2020-09-08T15:22:54.301 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:54 +0000 (0:00:00.040) 0:01:12.821 ***** 2020-09-08T15:22:54.316 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:54.356 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:54 +0000 (0:00:00.054) 0:01:12.875 ***** 2020-09-08T15:22:54.367 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-09-08T15:22:54.369 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:54.369 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-09-08T15:22:54.370 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:54.372 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:54.421 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:54 +0000 (0:00:00.064) 0:01:12.940 ***** 2020-09-08T15:22:54.434 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-09-08T15:22:54.436 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:54.437 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:54.437 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi055.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:54.437 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:54.438 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:54.439 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:54.482 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:54 +0000 (0:00:00.061) 0:01:13.002 ***** 2020-09-08T15:22:54.528 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:54 +0000 (0:00:00.045) 0:01:13.047 ***** 2020-09-08T15:22:54.559 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-09-08T15:22:54.561 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:54.615 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:54 +0000 (0:00:00.087) 0:01:13.134 ***** 2020-09-08T15:22:54.648 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-09-08T15:22:54.649 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:54.710 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:54 +0000 (0:00:00.091) 0:01:13.226 ***** 2020-09-08T15:22:54.742 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-09-08T15:22:54.744 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:54.781 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:54 +0000 (0:00:00.074) 0:01:13.301 ***** 2020-09-08T15:22:54.791 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-09-08T15:22:54.794 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.007769", "end": "2020-09-08 15:22:54.711336", "rc": 0, "start": "2020-09-08 15:22:54.703567", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-09-08T15:22:54.850 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:54 +0000 (0:00:00.064) 0:01:13.365 ***** 2020-09-08T15:22:54.857 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-09-08T15:22:54.859 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} TASK [testnode : Set grub config.] ********************************************* skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:54.903 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:54 +0000 (0:00:00.056) 0:01:13.422 ***** 2020-09-08T15:22:54.955 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:54 +0000 (0:00:00.052) 0:01:13.474 ***** 2020-09-08T15:22:54.967 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-09-08T15:22:54.969 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:55.032 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:55 +0000 (0:00:00.077) 0:01:13.552 ***** 2020-09-08T15:22:55.071 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:55.116 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:55 +0000 (0:00:00.083) 0:01:13.635 ***** 2020-09-08T15:22:55.127 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-09-08T15:22:55.130 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-09-08T15:22:55.168 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:55 +0000 (0:00:00.052) 0:01:13.688 ***** 2020-09-08T15:22:55.179 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-09-08T15:22:55.181 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:55.219 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:55 +0000 (0:00:00.050) 0:01:13.738 ***** 2020-09-08T15:22:55.282 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-09-08T15:22:55.284 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-09-08T15:22:55.325 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:55 +0000 (0:00:00.105) 0:01:13.844 ***** 2020-09-08T15:22:55.528 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-09-08T15:22:55.530 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-09-08T15:22:55.571 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:55 +0000 (0:00:00.241) 0:01:14.086 ***** 2020-09-08T15:22:55.613 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:55.657 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:55 +0000 (0:00:00.090) 0:01:14.176 ***** 2020-09-08T15:22:55.698 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-09-08T15:22:55.700 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-09-08T15:22:55.745 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:55 +0000 (0:00:00.087) 0:01:14.264 ***** 2020-09-08T15:22:55.906 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-09-08T15:22:55.909 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-09-08T15:22:55.947 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:55 +0000 (0:00:00.202) 0:01:14.466 ***** 2020-09-08T15:22:55.988 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-09-08T15:22:55.990 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:56.028 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:56 +0000 (0:00:00.081) 0:01:14.547 ***** 2020-09-08T15:22:56.052 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-09-08T15:22:56.053 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-09-08T15:22:56.093 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:56 +0000 (0:00:00.064) 0:01:14.612 ***** 2020-09-08T15:22:56.118 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:56.149 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:56 +0000 (0:00:00.056) 0:01:14.668 ***** 2020-09-08T15:22:56.159 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-09-08T15:22:56.161 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"cache_update_time": 1599578284, "cache_updated": false, "changed": false} 2020-09-08T15:22:56.207 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:56 +0000 (0:00:00.057) 0:01:14.726 ***** 2020-09-08T15:22:56.220 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-09-08T15:22:56.222 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:56.263 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:56 +0000 (0:00:00.056) 0:01:14.782 ***** 2020-09-08T15:22:56.275 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-09-08T15:22:56.277 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:56.314 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:56 +0000 (0:00:00.049) 0:01:14.832 ***** 2020-09-08T15:22:56.325 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-09-08T15:22:56.326 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:56.365 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:56 +0000 (0:00:00.050) 0:01:14.883 ***** 2020-09-08T15:22:56.682 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ ok: [smithi055.front.sepia.ceph.com] => (item={'block_used': 13746806, 'uuid': 'b40bdd1d-29ce-4dab-993d-cbf4157b4f12', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226348302, 'size_available': 927122644992, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149445, 'block_size': 4096, 'inode_available': 60905531}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226348302, "block_size": 4096, "block_total": 240095108, "block_used": 13746806, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905531, "inode_total": 61054976, "inode_used": 149445, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927122644992, "size_total": 983429562368, "uuid": "b40bdd1d-29ce-4dab-993d-cbf4157b4f12"}} Tuesday 08 September 2020 15:22:56 +0000 (0:00:00.071) 0:01:14.955 ***** TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Tuesday 08 September 2020 15:22:56 +0000 (0:00:00.053) 0:01:15.008 ***** TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} Tuesday 08 September 2020 15:22:56 +0000 (0:00:00.064) 0:01:15.073 ***** TASK [testnode : set_fact] ***************************************************** ok: [smithi104.front.sepia.ceph.com] => {"ansible_facts": {"nvme 2020-09-08T15:22:56.684 INFO:teuthology.task.ansible.out:_card_present": false}, "changed": false} Tuesday 08 September 2020 15:22:56 +0000 (0:00:00.057) 0:01:15.130 ***** TASK [testnode : Compile list of non-root partitions] ************************** changed: [smithi055.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.010067", "end": "2020-09-08 15:22:56.227234", "rc": 0, "start": "2020-09-08 15:22:56.217167", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} Tuesday 08 September 2020 15:22:56 +0000 (0:00:00.067) 0:01:15.197 ***** 2020-09-08T15:22:56.694 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi104.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:56.696 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:56.706 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:56.708 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:56.729 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-09-08T15:22:56.732 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** skipping: [smithi055.front.sepia.ceph.com] => (item={'block_used': 13746806, 'uuid': 'b40bdd1d-29ce-4dab-993d-cbf4157b4f12', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226348302, 'size_available': 927122644992, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149445, 'block_size': 4096, 'inode_available': 60905531}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226348302, "block_size": 4096, "block_total": 240095108, "block_used": 13746806, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905531, "inode_total": 61054976, "inode_used": 149445, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927122644992, "size_total": 983429562368, "uuid": "b40bdd1d-29ce-4dab-993d-cbf4157b4f12"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:56.741 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-09-08T15:22:56.743 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:56.745 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:56.782 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:56 +0000 (0:00:00.104) 0:01:15.302 ***** 2020-09-08T15:22:56.794 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:56.797 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:56.799 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:56.810 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:56.862 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:56 +0000 (0:00:00.079) 0:01:15.381 ***** 2020-09-08T15:22:56.872 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-09-08T15:22:56.915 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:56 +0000 (0:00:00.053) 0:01:15.434 ***** 2020-09-08T15:22:56.937 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-09-08T15:22:56.938 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-09-08T15:22:56.987 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:56 +0000 (0:00:00.071) 0:01:15.505 ***** 2020-09-08T15:22:56.999 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-09-08T15:22:57.000 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:57.122 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-09-08T15:22:57.125 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.005941", "end": "2020-09-08 15:22:57.095033", "rc": 0, "start": "2020-09-08 15:22:57.089092", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-09-08T15:22:57.188 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:57 +0000 (0:00:00.201) 0:01:15.707 ***** 2020-09-08T15:22:57.203 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-09-08T15:22:57.206 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-09-08T15:22:57.247 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:57 +0000 (0:00:00.058) 0:01:15.766 ***** 2020-09-08T15:22:57.266 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-09-08T15:22:57.269 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:57.314 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:57 +0000 (0:00:00.063) 0:01:15.830 ***** 2020-09-08T15:22:57.321 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-09-08T15:22:57.323 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:57.364 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:57 +0000 (0:00:00.053) 0:01:15.884 ***** 2020-09-08T15:22:57.375 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-09-08T15:22:57.377 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:57.416 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:57 +0000 (0:00:00.051) 0:01:15.935 ***** 2020-09-08T15:22:57.427 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-09-08T15:22:57.431 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:57.473 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:57 +0000 (0:00:00.057) 0:01:15.992 ***** 2020-09-08T15:22:57.486 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:57.520 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:57 +0000 (0:00:00.047) 0:01:16.040 ***** 2020-09-08T15:22:57.530 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-09-08T15:22:57.533 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:57.573 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:57 +0000 (0:00:00.052) 0:01:16.092 ***** 2020-09-08T15:22:57.605 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:57.642 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:57 +0000 (0:00:00.068) 0:01:16.161 ***** 2020-09-08T15:22:57.684 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:57.723 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:57 +0000 (0:00:00.080) 0:01:16.242 ***** 2020-09-08T15:22:57.758 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-08T15:22:57.760 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-09-08T15:22:57.803 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:57 +0000 (0:00:00.080) 0:01:16.322 ***** 2020-09-08T15:22:57.855 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-09-08T15:22:57.858 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:57.873 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:57.878 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:57.889 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:57.900 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:57.910 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-09-08T15:22:57.922 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} skipping: [smithi166.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:57.933 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:57.943 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:57.954 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:57.965 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:57.967 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:57.976 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-09-08T15:22:58.026 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:58 +0000 (0:00:00.223) 0:01:16.546 ***** 2020-09-08T15:22:58.267 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi166.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.005197", "end": "2020-09-08 15:22:58.239491", "rc": 0, "start": "2020-09-08 15:22:58.234294", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-09-08T15:22:58.322 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:58 +0000 (0:00:00.295) 0:01:16.841 ***** 2020-09-08T15:22:58.363 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-09-08T15:22:58.365 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:58.410 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:58 +0000 (0:00:00.083) 0:01:16.924 ***** 2020-09-08T15:22:58.418 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* ok: [smithi104.front.sepia.ceph.com] => {"cache_update_time": 1599578240, "cache_updated": false, "changed": false} 2020-09-08T15:22:58.461 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:58 +0000 (0:00:00.056) 0:01:16.981 ***** 2020-09-08T15:22:58.474 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:58.509 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:58 +0000 (0:00:00.047) 0:01:17.028 ***** 2020-09-08T15:22:58.519 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-09-08T15:22:58.521 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:58.559 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:58 +0000 (0:00:00.049) 0:01:17.078 ***** 2020-09-08T15:22:58.568 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-09-08T15:22:58.570 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:58.611 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:58 +0000 (0:00:00.051) 0:01:17.129 ***** 2020-09-08T15:22:58.619 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-09-08T15:22:58.620 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => (item={'block_used': 13746804, 'uuid': 'b40bdd1d-29ce-4dab-993d-cbf4157b4f12', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226348304, 'size_available': 927122653184, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149446, 'block_size': 4096, 'inode_available': 60905530}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226348304, "block_size": 4096, "block_total": 240095108, "block_used": 13746804, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905530, "inode_total": 61054976, "inode_used": 149446, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927122653184, "size_total": 983429562368, "uuid": "b40bdd1d-29ce-4dab-993d-cbf4157b4f12"}} 2020-09-08T15:22:58.659 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:58 +0000 (0:00:00.049) 0:01:17.178 ***** 2020-09-08T15:22:58.815 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-09-08T15:22:58.817 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.006694", "end": "2020-09-08 15:22:59.184970", "rc": 0, "start": "2020-09-08 15:22:59.178276", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:22:58.852 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:58 +0000 (0:00:00.193) 0:01:17.372 ***** 2020-09-08T15:22:58.908 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-09-08T15:22:58.909 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'block_used': 13746804, 'uuid': 'b40bdd1d-29ce-4dab-993d-cbf4157b4f12', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226348304, 'size_available': 927122653184, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149446, 'block_size': 4096, 'inode_available': 60905530}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226348304, "block_size": 4096, "block_total": 240095108, "block_used": 13746804, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905530, "inode_total": 61054976, "inode_used": 149446, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927122653184, "size_total": 983429562368, "uuid": "b40bdd1d-29ce-4dab-993d-cbf4157b4f12"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:58.946 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:58 +0000 (0:00:00.093) 0:01:17.465 ***** 2020-09-08T15:22:58.977 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-09-08T15:22:59.014 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:59 +0000 (0:00:00.068) 0:01:17.533 ***** 2020-09-08T15:22:59.066 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-09-08T15:22:59.068 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:59.122 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-ul4ak8gsmd4yUeUc3unDw0QWAkgVoVDekZNTM3YjVapYgXEIn6P5i0rw5Ly1XO3a'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:02.015674", "end": "2020-09-08 15:22:58.712199", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-ul4ak8gsmd4yUeUc3unDw0QWAkgVoVDekZNTM3YjVapYgXEIn6P5i0rw5Ly1XO3a"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:22:56.696525", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-08T15:22:59.134 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:59.135 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:59.511 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-09-08T15:22:59.513 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"cache_update_time": 1599578328, "cache_updated": false, "changed": false} 2020-09-08T15:22:59.554 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:59 +0000 (0:00:00.539) 0:01:18.073 ***** 2020-09-08T15:22:59.586 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-09-08T15:22:59.588 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:22:59.637 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:59 +0000 (0:00:00.083) 0:01:18.157 ***** 2020-09-08T15:22:59.689 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-09-08T15:22:59.691 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => (item={'block_used': 13746799, 'uuid': 'b40bdd1d-29ce-4dab-993d-cbf4157b4f12', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226348309, 'size_available': 927122673664, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149446, 'block_size': 4096, 'inode_available': 60905530}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226348309, "block_size": 4096, "block_total": 240095108, "block_used": 13746799, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905530, "inode_total": 61054976, "inode_used": 149446, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927122673664, "size_total": 983429562368, "uuid": "b40bdd1d-29ce-4dab-993d-cbf4157b4f12"}} 2020-09-08T15:22:59.728 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:59 +0000 (0:00:00.090) 0:01:18.247 ***** 2020-09-08T15:22:59.874 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-09-08T15:22:59.876 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.010002", "end": "2020-09-08 15:22:59.626395", "rc": 0, "start": "2020-09-08 15:22:59.616393", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:22:59.919 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:22:59 +0000 (0:00:00.190) 0:01:18.438 ***** 2020-09-08T15:22:59.968 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-09-08T15:22:59.970 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'block_used': 13746799, 'uuid': 'b40bdd1d-29ce-4dab-993d-cbf4157b4f12', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226348309, 'size_available': 927122673664, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149446, 'block_size': 4096, 'inode_available': 60905530}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226348309, "block_size": 4096, "block_total": 240095108, "block_used": 13746799, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905530, "inode_total": 61054976, "inode_used": 149446, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927122673664, "size_total": 983429562368, "uuid": "b40bdd1d-29ce-4dab-993d-cbf4157b4f12"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:00.009 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:00 +0000 (0:00:00.089) 0:01:18.528 ***** 2020-09-08T15:23:00.040 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-09-08T15:23:00.080 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:00 +0000 (0:00:00.071) 0:01:18.600 ***** 2020-09-08T15:23:00.145 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-09-08T15:23:00.147 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:00.402 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-ul4ak8gsmd4yUeUc3unDw0QWAkgVoVDe25WbbJldWPcnCscWJjZrIa0C7iEUyDoA'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.124390", "end": "2020-09-08 15:22:59.993286", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-ul4ak8gsmd4yUeUc3unDw0QWAkgVoVDe25WbbJldWPcnCscWJjZrIa0C7iEUyDoA"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:22:58.868896", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-08T15:23:00.403 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4H48L', 'wwn-0x5000c500871b28cf'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'b40bdd1d-29ce-4dab-993d-cbf4157b4f12', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4H48L-part1', 'wwn-0x5000c500871b28cf-part1'], 'uuids': ['b40bdd1d-29ce-4dab-993d-cbf4157b4f12']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500871b28cf', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4H48L', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4H48L", "wwn-0x5000c500871b28cf"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4H48L-part1", "wwn-0x5000c500871b28cf-part1"], "labels": [], "masters": [], "uuids": ["b40bdd1d-29ce-4dab-993d-cbf4157b4f12"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "b40bdd1d-29ce-4dab-993d-cbf4157b4f12"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4H48L", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871b28cf"}}, "skip_reason": 2020-09-08T15:23:00.418 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-09-08T15:23:01.156 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-OrhzFD6qClMQKlWOMAhCCesFSJyzSZhIm3WNT4imdZhSG7UrDLHeqnFTWfer2AbJ'], 'uuids': ['6d4b424b-98e4-497d-a009-49605d6d9b68']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.939105", "end": "2020-09-08 15:23:01.509120", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-OrhzFD6qClMQKlWOMAhCCesFSJyzSZhIm3WNT4imdZhSG7UrDLHeqnFTWfer2AbJ"], "labels": [], "masters": [], "uuids": ["6d4b424b-98e4-497d-a009-49605d6d9b68"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:22:59.570015", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-08T15:23:01.167 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:01.170 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:01.696 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '7814037168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-yqn4Uq-Fgd1-7uIp-q9B6-pcFT-xstw-9Mmki4', 'nvme-INTEL_SSDPEDKE040T7_BTLE736102VS4P0KGN', 'nvme-eui.0100000001000000e4d25c000034df14'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Express Flash NVMe P4500/P4600', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDKE040T7', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '3.64 TB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.176374", "end": "2020-09-08 15:23:01.296771", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Express Flash NVMe P4500/P4600", "links": {"ids": ["lvm-pv-uuid-yqn4Uq-Fgd1-7uIp-q9B6-pcFT-xstw-9Mmki4", "nvme-INTEL_SSDPEDKE040T7_BTLE736102VS4P0KGN", "nvme-eui.0100000001000000e4d25c000034df14"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDKE040T7", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "7814037168", "sectorsize": "512", "size": "3.64 TB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:00.120397", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT 2020-09-08T15:23:01.699 INFO:teuthology.task.ansible.out: data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-08T15:23:01.708 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:01.752 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:02.175 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-4NjFX2q6zuU85JDAzIriwXqp6PdvCKL4MH8i33aReF3e2RUTDTw0SdwGLRQH2ozO'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.884783", "end": "2020-09-08 15:23:01.905888", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-4NjFX2q6zuU85JDAzIriwXqp6PdvCKL4MH8i33aReF3e2RUTDTw0SdwGLRQH2ozO"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:00.021105", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-08T15:23:02.187 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:02.199 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:02.465 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-OrhzFD6qClMQKlWOMAhCCesFSJyzSZhILZLrdPaFFirn4of78A7s7dq4pTPSseUp'], 'uuids': ['c7254c65-40e2-43c5-8d7e-b94af710a54d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.168345", "end": "2020-09-08 15:23:02.817439", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-OrhzFD6qClMQKlWOMAhCCesFSJyzSZhILZLrdPaFFirn4of78A7s7dq4pTPSseUp"], "labels": [], "masters": [], "uuids": ["c7254c65-40e2-43c5-8d7e-b94af710a54d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:01.649094", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-08T15:23:02.493 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QGJ7', 'wwn-0x5000c50091dfe2b2'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'b40bdd1d-29ce-4dab-993d-cbf4157b4f12', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QGJ7-part1', 'wwn-0x5000c50091dfe2b2-part1'], 'uuids': ['b40bdd1d-29ce-4dab-993d-cbf4157b4f12']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50091dfe2b2', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5QGJ7', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QGJ7", "wwn-0x5000c50091dfe2b2"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QGJ7-part1", "wwn-0x5000c50091dfe2b2-part1"], "labels": [], "masters": [], "uuids": ["b40bdd1d-29ce-4dab-993d-cbf4157b4f12"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "b40bdd1d-29ce-4dab-993d-cbf4157b4f12"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QGJ7", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091dfe2b2"}}, "skip_reason": 2020-09-08T15:23:02.495 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-09-08T15:23:03.016 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-ul4ak8gsmd4yUeUc3unDw0QWAkgVoVDekObk2bXCfzTvpNz8BpJ3V4QdEsgwbwGl'], 'uuids': ['4a892248-dbd2-47e5-b698-04248f047af0']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.139267", "end": "2020-09-08 15:23:02.603719", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-ul4ak8gsmd4yUeUc3unDw0QWAkgVoVDekObk2bXCfzTvpNz8BpJ3V4QdEsgwbwGl"], "labels": [], "masters": [], "uuids": ["4a892248-dbd2-47e5-b698-04248f047af0"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:01.464452", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-08T15:23:03.028 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:03.039 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:03.461 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-4NjFX2q6zuU85JDAzIriwXqp6PdvCKL4e5x3QmUYIEOiSacPyNIZr2lvRCX3Hacv'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.131275", "end": "2020-09-08 15:23:03.203543", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-4NjFX2q6zuU85JDAzIriwXqp6PdvCKL4e5x3QmUYIEOiSacPyNIZr2lvRCX3Hacv"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:02.072268", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-08T15:23:03.473 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XMNS', 'wwn-0x5000c5009293883c'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'b40bdd1d-29ce-4dab-993d-cbf4157b4f12', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XMNS-part1', 'wwn-0x5000c5009293883c-part1'], 'uuids': ['b40bdd1d-29ce-4dab-993d-cbf4157b4f12']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c5009293883c', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XMNS', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XMNS", "wwn-0x5000c5009293883c"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XMNS-part1", "wwn-0x5000c5009293883c-part1"], "labels": [], "masters": [], "uuids": ["b40bdd1d-29ce-4dab-993d-cbf4157b4f12"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "b40bdd1d-29ce-4dab-993d-cbf4157b4f12"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XMNS", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009293883c"}}, "skip_reason": 2020-09-08T15:23:03.475 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-09-08T15:23:03.869 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-5GVwVB-Ha6X-mYt1-AHvG-Sgm2-js5u-22cF6i', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400NW400BGN', 'nvme-nvme.8086-504846543632303430304e5734303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.220604", "end": "2020-09-08 15:23:04.212559", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-5GVwVB-Ha6X-mYt1-AHvG-Sgm2-js5u-22cF6i", "nvme-INTEL_SSDPEDMD400G4_PHFT620400NW400BGN", "nvme-nvme.8086-504846543632303430304e5734303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:02.991955", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-09-08T15:23:03.874 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:03.884 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:04.319 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-ul4ak8gsmd4yUeUc3unDw0QWAkgVoVDetSNxK90WUdmUg2mb1yrmkF8d0Uxk27mi'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.142432", "end": "2020-09-08 15:23:03.921756", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-ul4ak8gsmd4yUeUc3unDw0QWAkgVoVDetSNxK90WUdmUg2mb1yrmkF8d0Uxk27mi"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:02.779324", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-08T15:23:04.331 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:04.835 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-FUWalD-5bQm-WXqJ-Gu1y-K3O7-4ct8-GENs2u', 'nvme-INTEL_SSDPEDMD400G4_PHFT620401AC400BGN', 'nvme-nvme.8086-50484654363230343031414334303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.228252", "end": "2020-09-08 15:23:04.572157", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-FUWalD-5bQm-WXqJ-Gu1y-K3O7-4ct8-GENs2u", "nvme-INTEL_SSDPEDMD400G4_PHFT620401AC400BGN", "nvme-nvme.8086-50484654363230343031414334303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:03.343905", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-09-08T15:23:04.837 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-08T15:23:04.847 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:04.848 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:05.159 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-OrhzFD6qClMQKlWOMAhCCesFSJyzSZhId2SlrUEREoW1JfjJaw8irkDErvUAxG86'], 'uuids': ['0a303d69-2b2c-49d9-909d-60ec3b6a75ac']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.133923", "end": "2020-09-08 15:23:05.514084", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-OrhzFD6qClMQKlWOMAhCCesFSJyzSZhId2SlrUEREoW1JfjJaw8irkDErvUAxG86"], "labels": [], "masters": [], "uuids": ["0a303d69-2b2c-49d9-909d-60ec3b6a75ac"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:04.380161", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-08T15:23:05.176 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:05.198 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:05.592 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-ul4ak8gsmd4yUeUc3unDw0QWAkgVoVDekB10nSjtV8eWxx1VUHw61MWsJh9Hk00f'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.112425", "end": "2020-09-08 15:23:05.189478", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-ul4ak8gsmd4yUeUc3unDw0QWAkgVoVDekB10nSjtV8eWxx1VUHw61MWsJh9Hk00f"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:04.077053", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-08T15:23:05.639 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:05 +0000 (0:00:05.558) 0:01:24.159 ***** 2020-09-08T15:23:05.692 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-09-08T15:23:05.694 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:06.000 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-ul4ak8gsmd4yUeUc3unDw0QWAkgVoVDekZNTM3YjVapYgXEIn6P5i0rw5Ly1XO3a'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.166466", "end": "2020-09-08 15:23:05.593856", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-ul4ak8gsmd4yUeUc3unDw0QWAkgVoVDekZNTM3YjVapYgXEIn6P5i0rw5Ly1XO3a"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:05.427390", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.161799 s, 713 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.161799 s, 713 MB/s"], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:06.012 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:06.014 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:06.143 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-09-08T15:23:06.145 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-4NjFX2q6zuU85JDAzIriwXqp6PdvCKL4Nh2XcP4g3OJLxKhZmIgAjqUyEFnXpnQP'], 'uuids': ['ddbc648e-5091-4f25-8e41-5e2a8e600bca']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.154432", "end": "2020-09-08 15:23:05.881037", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-4NjFX2q6zuU85JDAzIriwXqp6PdvCKL4Nh2XcP4g3OJLxKhZmIgAjqUyEFnXpnQP"], "labels": [], "masters": [], "uuids": ["ddbc648e-5091-4f25-8e41-5e2a8e600bca"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:04.726605", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-08T15:23:06.145 INFO:teuthology.task.ansible.out: 2020-09-08T15:23:06.157 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:06.170 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:06.325 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-09-08T15:23:06.327 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-ul4ak8gsmd4yUeUc3unDw0QWAkgVoVDe25WbbJldWPcnCscWJjZrIa0C7iEUyDoA'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.169679", "end": "2020-09-08 15:23:05.927027", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-ul4ak8gsmd4yUeUc3unDw0QWAkgVoVDe25WbbJldWPcnCscWJjZrIa0C7iEUyDoA"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:05.757348", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.164937 s, 699 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.164937 s, 699 MB/s"], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:06.338 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4H48L', 'wwn-0x5000c500871b28cf'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'b40bdd1d-29ce-4dab-993d-cbf4157b4f12', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4H48L-part1', 'wwn-0x5000c500871b28cf-part1'], 'uuids': ['b40bdd1d-29ce-4dab-993d-cbf4157b4f12']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500871b28cf', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4H48L', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4H48L", "wwn-0x5000c500871b28cf"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4H48L-part1", "wwn-0x5000c500871b28cf-part1"], "labels": [], "masters": [], "uuids": ["b40bdd1d-29ce-4dab-993d-cbf4157b4f12"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "b40bdd1d-29ce-4dab-993d-cbf4157b4f12"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4H48L", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871b28cf"}}, "skip_reason": 2020-09-08T15:23:06.339 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-09-08T15:23:06.459 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-09-08T15:23:06.465 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-OrhzFD6qClMQKlWOMAhCCesFSJyzSZhIkBTUifBtlg5wE7bkJW6dVAxFBB4p7Gdy'], 'uuids': ['edb8a0e1-d3a4-4aa5-9bef-4cf35465fbda']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.144510", "end": "2020-09-08 15:23:06.813550", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-OrhzFD6qClMQKlWOMAhCCesFSJyzSZhIkBTUifBtlg5wE7bkJW6dVAxFBB4p7Gdy"], "labels": [], "masters": [], "uuids": ["edb8a0e1-d3a4-4aa5-9bef-4cf35465fbda"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:05.669040", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-08T15:23:06.471 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:06.526 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-09-08T15:23:06.528 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '7814037168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-yqn4Uq-Fgd1-7uIp-q9B6-pcFT-xstw-9Mmki4', 'nvme-INTEL_SSDPEDKE040T7_BTLE736102VS4P0KGN', 'nvme-eui.0100000001000000e4d25c000034df14'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Express Flash NVMe P4500/P4600', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDKE040T7', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '3.64 TB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.055472", "end": "2020-09-08 15:23:06.128259", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Express Flash NVMe P4500/P4600", "links": {"ids": ["lvm-pv-uuid-yqn4Uq-Fgd1-7uIp-q9B6-pcFT-xstw-9Mmki4", "nvme-INTEL_SSDPEDKE040T7_BTLE736102VS4P0KGN", "nvme-eui.0100000001000000e4d25c000034df14"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDKE040T7", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "7814037168", "sectorsize": "512", "size": "3.64 TB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:06.072787", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0507399 s, 2.3 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0507399 s, 2.3 GB 2020-09-08T15:23:06.529 INFO:teuthology.task.ansible.out:/s"], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:06.538 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:06.549 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:06.841 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-ul4ak8gsmd4yUeUc3unDw0QWAkgVoVDekObk2bXCfzTvpNz8BpJ3V4QdEsgwbwGl'], 'uuids': ['4a892248-dbd2-47e5-b698-04248f047af0']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.163439", "end": "2020-09-08 15:23:06.446964", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-ul4ak8gsmd4yUeUc3unDw0QWAkgVoVDekObk2bXCfzTvpNz8BpJ3V4QdEsgwbwGl"], "labels": [], "masters": [], "uuids": ["4a892248-dbd2-47e5-b698-04248f047af0"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:06.283525", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.160517 s, 719 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.160517 s, 719 MB/s"], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:06.875 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:06.886 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:07.185 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-ul4ak8gsmd4yUeUc3unDw0QWAkgVoVDetSNxK90WUdmUg2mb1yrmkF8d0Uxk27mi'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.163432", "end": "2020-09-08 15:23:06.782499", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-ul4ak8gsmd4yUeUc3unDw0QWAkgVoVDetSNxK90WUdmUg2mb1yrmkF8d0Uxk27mi"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:06.619067", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.158655 s, 727 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.158655 s, 727 MB/s"], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:07.196 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:07.442 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-09-08T15:23:07.444 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-4NjFX2q6zuU85JDAzIriwXqp6PdvCKL4RAzYLrH2Mlu6127YOpCp0bvnMTRaIxYt'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.136349", "end": "2020-09-08 15:23:07.181307", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-4NjFX2q6zuU85JDAzIriwXqp6PdvCKL4RAzYLrH2Mlu6127YOpCp0bvnMTRaIxYt"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:06.044958", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-08T15:23:07.454 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:07.497 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-09-08T15:23:07.498 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-ul4ak8gsmd4yUeUc3unDw0QWAkgVoVDekB10nSjtV8eWxx1VUHw61MWsJh9Hk00f'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.169436", "end": "2020-09-08 15:23:07.102281", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-ul4ak8gsmd4yUeUc3unDw0QWAkgVoVDekB10nSjtV8eWxx1VUHw61MWsJh9Hk00f"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:06.932845", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.164552 s, 701 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.164552 s, 701 MB/s"], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:07.554 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:07 +0000 (0:00:01.915) 0:01:26.074 ***** 2020-09-08T15:23:07.749 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-OrhzFD6qClMQKlWOMAhCCesFSJyzSZhIiBQcSm2kPiL4tM6sYBVWjZWGrXaRfwjU'], 'uuids': ['99701571-abba-42ff-ae88-dc60bfcf7e83']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.136003", "end": "2020-09-08 15:23:08.104274", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-OrhzFD6qClMQKlWOMAhCCesFSJyzSZhIiBQcSm2kPiL4tM6sYBVWjZWGrXaRfwjU"], "labels": [], "masters": [], "uuids": ["99701571-abba-42ff-ae88-dc60bfcf7e83"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:06.968271", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk u 2020-09-08T15:23:07.751 INFO:teuthology.task.ansible.out:sing fdisk or", "other utilities."]} 2020-09-08T15:23:07.798 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:07 +0000 (0:00:00.243) 0:01:26.317 ***** 2020-09-08T15:23:07.853 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:07.876 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-09-08T15:23:07.878 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.157264", "end": "2020-09-08 15:23:07.468850", "rc": 0, "start": "2020-09-08 15:23:07.311586", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:07.917 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:07 +0000 (0:00:00.117) 0:01:26.435 ***** 2020-09-08T15:23:08.108 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-09-08T15:23:08.110 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.049743", "end": "2020-09-08 15:23:07.719027", "rc": 0, "start": "2020-09-08 15:23:07.669284", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:08.148 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:08 +0000 (0:00:00.232) 0:01:26.667 ***** 2020-09-08T15:23:08.161 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-09-08T15:23:08.162 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-OrhzFD6qClMQKlWOMAhCCesFSJyzSZhIm3WNT4imdZhSG7UrDLHeqnFTWfer2AbJ'], 'uuids': ['6d4b424b-98e4-497d-a009-49605d6d9b68']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.154265", "end": "2020-09-08 15:23:08.499717", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-OrhzFD6qClMQKlWOMAhCCesFSJyzSZhIm3WNT4imdZhSG7UrDLHeqnFTWfer2AbJ"], "labels": [], "masters": [], "uuids": ["6d4b424b-98e4-497d-a009-49605d6d9b68"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:08.345452", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.149436 s, 772 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.149436 s, 772 MB/s"], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:08.167 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:08.177 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Remove physical volumes] ************************************** 2020-09-08T15:23:08.217 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:08 +0000 (0:00:00.068) 0:01:26.736 ***** 2020-09-08T15:23:08.261 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:08.310 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:08 +0000 (0:00:00.092) 0:01:26.829 ***** 2020-09-08T15:23:08.342 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-09-08T15:23:08.344 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:08.379 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:08 +0000 (0:00:00.068) 0:01:26.898 ***** 2020-09-08T15:23:08.410 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-09-08T15:23:08.412 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:08.455 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:08 +0000 (0:00:00.076) 0:01:26.975 ***** 2020-09-08T15:23:08.474 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-OrhzFD6qClMQKlWOMAhCCesFSJyzSZhILZLrdPaFFirn4of78A7s7dq4pTPSseUp'], 'uuids': ['c7254c65-40e2-43c5-8d7e-b94af710a54d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.166514", "end": "2020-09-08 15:23:08.815033", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-OrhzFD6qClMQKlWOMAhCCesFSJyzSZhILZLrdPaFFirn4of78A7s7dq4pTPSseUp"], "labels": [], "masters": [], "uuids": ["c7254c65-40e2-43c5-8d7e-b94af710a54d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:08.648519", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.162074 s, 712 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.162074 s, 712 MB/s"], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:08.481 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QGJ7', 'wwn-0x5000c50091dfe2b2'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'b40bdd1d-29ce-4dab-993d-cbf4157b4f12', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QGJ7-part1', 'wwn-0x5000c50091dfe2b2-part1'], 'uuids': ['b40bdd1d-29ce-4dab-993d-cbf4157b4f12']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50091dfe2b2', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5QGJ7', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QGJ7", "wwn-0x5000c50091dfe2b2"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QGJ7-part1", "wwn-0x5000c50091dfe2b2-part1"], "labels": [], "masters": [], "uuids": ["b40bdd1d-29ce-4dab-993d-cbf4157b4f12"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "b40bdd1d-29ce-4dab-993d-cbf4157b4f12"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QGJ7", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091dfe2b2"}}, "skip_reason": 2020-09-08T15:23:08.485 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-09-08T15:23:08.494 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:08.535 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:08 +0000 (0:00:00.079) 0:01:27.054 ***** 2020-09-08T15:23:08.625 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-09-08T15:23:08.627 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-5GVwVB-Ha6X-mYt1-AHvG-Sgm2-js5u-22cF6i', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400NW400BGN', 'nvme-nvme.8086-504846543632303430304e5734303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.044673", "end": "2020-09-08 15:23:08.982251", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-5GVwVB-Ha6X-mYt1-AHvG-Sgm2-js5u-22cF6i", "nvme-INTEL_SSDPEDMD400G4_PHFT620400NW400BGN", "nvme-nvme.8086-504846543632303430304e5734303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:08.937578", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0406644 s, 2.8 GB/s", "std 2020-09-08T15:23:08.629 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0406644 s, 2.8 GB/s"], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:08.638 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:08.639 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:08.682 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-09-08T15:23:08.684 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-09-08T15:23:08.740 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:08 +0000 (0:00:00.204) 0:01:27.259 ***** 2020-09-08T15:23:08.761 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-09-08T15:23:08.763 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-4NjFX2q6zuU85JDAzIriwXqp6PdvCKL4hUz8f3dD1p54NG8fhQqj5MExpPgLX0yK'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.140858", "end": "2020-09-08 15:23:08.465406", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-4NjFX2q6zuU85JDAzIriwXqp6PdvCKL4hUz8f3dD1p54NG8fhQqj5MExpPgLX0yK"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:07.324548", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-09-08T15:23:08.809 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:08 +0000 (0:00:00.068) 0:01:27.328 ***** 2020-09-08T15:23:08.819 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-09-08T15:23:08.821 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'block_used': 13746806, 'uuid': 'b40bdd1d-29ce-4dab-993d-cbf4157b4f12', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226348302, 'size_available': 927122644992, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149445, 'block_size': 4096, 'inode_available': 60905531}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226348302, "block_size": 4096, "block_total": 240095108, "block_used": 13746806, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905531, "inode_total": 61054976, "inode_used": 149445, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927122644992, "size_total": 983429562368, "uuid": "b40bdd1d-29ce-4dab-993d-cbf4157b4f12"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:08.860 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:08 +0000 (0:00:00.050) 0:01:27.379 ***** 2020-09-08T15:23:08.886 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:08.911 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:08.958 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:08 +0000 (0:00:00.090) 0:01:27.470 ***** 2020-09-08T15:23:08.966 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-OrhzFD6qClMQKlWOMAhCCesFSJyzSZhId2SlrUEREoW1JfjJaw8irkDErvUAxG86'], 'uuids': ['0a303d69-2b2c-49d9-909d-60ec3b6a75ac']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.148351", "end": "2020-09-08 15:23:09.275518", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-OrhzFD6qClMQKlWOMAhCCesFSJyzSZhId2SlrUEREoW1JfjJaw8irkDErvUAxG86"], "labels": [], "masters": [], "uuids": ["0a303d69-2b2c-49d9-909d-60ec3b6a75ac"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:09.127167", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.145709 s, 792 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.145709 s, 792 MB/s"], "stdout": "", "stdout_lines": []} skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0 2020-09-08T15:23:08.968 INFO:teuthology.task.ansible.out:', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:09.003 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-08T15:23:09.005 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:09.044 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:09 +0000 (0:00:00.089) 0:01:27.559 ***** 2020-09-08T15:23:09.083 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-08T15:23:09.085 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:09.122 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:09 +0000 (0:00:00.081) 0:01:27.641 ***** 2020-09-08T15:23:09.152 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-09-08T15:23:09.153 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-4NjFX2q6zuU85JDAzIriwXqp6PdvCKL4MH8i33aReF3e2RUTDTw0SdwGLRQH2ozO'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.148685", "end": "2020-09-08 15:23:08.891587", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-4NjFX2q6zuU85JDAzIriwXqp6PdvCKL4MH8i33aReF3e2RUTDTw0SdwGLRQH2ozO"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:08.742902", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.144581 s, 798 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.144581 s, 798 MB/s"], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:09.155 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:09.175 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:09.176 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:09.221 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:09 +0000 (0:00:00.098) 0:01:27.740 ***** 2020-09-08T15:23:09.263 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-OrhzFD6qClMQKlWOMAhCCesFSJyzSZhIkBTUifBtlg5wE7bkJW6dVAxFBB4p7Gdy'], 'uuids': ['edb8a0e1-d3a4-4aa5-9bef-4cf35465fbda']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.156885", "end": "2020-09-08 15:23:09.607193", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-OrhzFD6qClMQKlWOMAhCCesFSJyzSZhIkBTUifBtlg5wE7bkJW6dVAxFBB4p7Gdy"], "labels": [], "masters": [], "uuids": ["edb8a0e1-d3a4-4aa5-9bef-4cf35465fbda"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:09.450308", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.153909 s, 749 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.153909 s, 749 MB/s"], "stdout": "", "stdout_lines": []} TASK [testnode : Determine desired logical volume percentage size] ************* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor 2020-09-08T15:23:09.267 INFO:teuthology.task.ansible.out:': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:09.313 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:09 +0000 (0:00:00.092) 0:01:27.832 ***** 2020-09-08T15:23:09.345 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-09-08T15:23:09.347 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:09.383 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:09 +0000 (0:00:00.069) 0:01:27.902 ***** 2020-09-08T15:23:09.481 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-09-08T15:23:09.483 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-4NjFX2q6zuU85JDAzIriwXqp6PdvCKL4e5x3QmUYIEOiSacPyNIZr2lvRCX3Hacv'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.156446", "end": "2020-09-08 15:23:09.217203", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-4NjFX2q6zuU85JDAzIriwXqp6PdvCKL4e5x3QmUYIEOiSacPyNIZr2lvRCX3Hacv"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:09.060757", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.151752 s, 760 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.151752 s, 760 MB/s"], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:09.493 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XMNS', 'wwn-0x5000c5009293883c'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'b40bdd1d-29ce-4dab-993d-cbf4157b4f12', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XMNS-part1', 'wwn-0x5000c5009293883c-part1'], 'uuids': ['b40bdd1d-29ce-4dab-993d-cbf4157b4f12']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c5009293883c', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XMNS', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XMNS", "wwn-0x5000c5009293883c"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XMNS-part1", "wwn-0x5000c5009293883c-part1"], "labels": [], "masters": [], "uuids": ["b40bdd1d-29ce-4dab-993d-cbf4157b4f12"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "b40bdd1d-29ce-4dab-993d-cbf4157b4f12"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XMNS", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009293883c"}}, "skip_reason": 2020-09-08T15:23:09.495 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-09-08T15:23:09.516 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-OrhzFD6qClMQKlWOMAhCCesFSJyzSZhIiBQcSm2kPiL4tM6sYBVWjZWGrXaRfwjU'], 'uuids': ['99701571-abba-42ff-ae88-dc60bfcf7e83']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.152651", "end": "2020-09-08 15:23:09.878761", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-OrhzFD6qClMQKlWOMAhCCesFSJyzSZhIiBQcSm2kPiL4tM6sYBVWjZWGrXaRfwjU"], "labels": [], "masters": [], "uuids": ["99701571-abba-42ff-ae88-dc60bfcf7e83"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:09.726110", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.150121 s, 768 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.150121 s, 768 MB/s"], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:09.566 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:09 +0000 (0:00:00.183) 0:01:28.086 ***** 2020-09-08T15:23:09.706 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-FUWalD-5bQm-WXqJ-Gu1y-K3O7-4ct8-GENs2u', 'nvme-INTEL_SSDPEDMD400G4_PHFT620401AC400BGN', 'nvme-nvme.8086-50484654363230343031414334303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.052802", "end": "2020-09-08 15:23:09.449110", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-FUWalD-5bQm-WXqJ-Gu1y-K3O7-4ct8-GENs2u", "nvme-INTEL_SSDPEDMD400G4_PHFT620401AC400BGN", "nvme-nvme.8086-50484654363230343031414334303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:09.396308", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0481332 s, 2.4 GB/s", "std 2020-09-08T15:23:09.708 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0481332 s, 2.4 GB/s"], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:09.729 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:09.741 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:09.846 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-09-08T15:23:09.848 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.120053", "end": "2020-09-08 15:23:10.200151", "rc": 0, "start": "2020-09-08 15:23:10.080098", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:09.886 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:09 +0000 (0:00:00.319) 0:01:28.405 ***** 2020-09-08T15:23:10.020 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-4NjFX2q6zuU85JDAzIriwXqp6PdvCKL4Nh2XcP4g3OJLxKhZmIgAjqUyEFnXpnQP'], 'uuids': ['ddbc648e-5091-4f25-8e41-5e2a8e600bca']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.153229", "end": "2020-09-08 15:23:09.764184", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-4NjFX2q6zuU85JDAzIriwXqp6PdvCKL4Nh2XcP4g3OJLxKhZmIgAjqUyEFnXpnQP"], "labels": [], "masters": [], "uuids": ["ddbc648e-5091-4f25-8e41-5e2a8e600bca"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:09.610955", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.149371 s, 772 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.149371 s, 772 MB/s"], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:10.032 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:10.045 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:10.055 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-09-08T15:23:10.057 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-09-08T15:23:10.100 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:10 +0000 (0:00:00.214) 0:01:28.620 ***** 2020-09-08T15:23:10.124 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-09-08T15:23:10.125 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.062235", "end": "2020-09-08 15:23:10.477487", "rc": 0, "start": "2020-09-08 15:23:10.415252", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:10.165 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:10 +0000 (0:00:00.064) 0:01:28.684 ***** 2020-09-08T15:23:10.203 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-09-08T15:23:10.255 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:10 +0000 (0:00:00.088) 0:01:28.773 ***** 2020-09-08T15:23:10.321 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-09-08T15:23:10.323 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:10.368 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:10 +0000 (0:00:00.114) 0:01:28.887 ***** 2020-09-08T15:23:10.383 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-09-08T15:23:10.384 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-4NjFX2q6zuU85JDAzIriwXqp6PdvCKL4RAzYLrH2Mlu6127YOpCp0bvnMTRaIxYt'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.162592", "end": "2020-09-08 15:23:10.078695", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-4NjFX2q6zuU85JDAzIriwXqp6PdvCKL4RAzYLrH2Mlu6127YOpCp0bvnMTRaIxYt"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:09.916103", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.157859 s, 731 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.157859 s, 731 MB/s"], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:10.386 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:10.408 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-09-08T15:23:10.410 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:10.449 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:10 +0000 (0:00:00.081) 0:01:28.969 ***** 2020-09-08T15:23:10.480 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-09-08T15:23:10.482 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:10.524 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:10 +0000 (0:00:00.074) 0:01:29.043 ***** 2020-09-08T15:23:10.564 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-09-08T15:23:10.566 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:10.603 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:10 +0000 (0:00:00.079) 0:01:29.123 ***** 2020-09-08T15:23:10.605 INFO:teuthology.task.ansible.out: 2020-09-08T15:23:10.637 INFO:teuthology.task.ansible.out: 2020-09-08T15:23:10.640 INFO:teuthology.task.ansible.out:TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-4NjFX2q6zuU85JDAzIriwXqp6PdvCKL4hUz8f3dD1p54NG8fhQqj5MExpPgLX0yK'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.153703", "end": "2020-09-08 15:23:10.385107", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-4NjFX2q6zuU85JDAzIriwXqp6PdvCKL4hUz8f3dD1p54NG8fhQqj5MExpPgLX0yK"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-09-08 15:23:10.231404", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.150772 s, 765 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.150772 s, 765 MB/s"], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:10.699 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:10 +0000 (0:00:00.095) 0:01:29.218 ***** 2020-09-08T15:23:10.721 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-09-08T15:23:10.723 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-09-08T15:23:10.763 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-09-08T15:23:10.765 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-09-08T15:23:10.809 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:10 +0000 (0:00:00.109) 0:01:29.327 ***** 2020-09-08T15:23:10.861 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ skipping: [smithi104.front.sepia.ceph.com] => (item={'block_used': 13746804, 'uuid': 'b40bdd1d-29ce-4dab-993d-cbf4157b4f12', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226348304, 'size_available': 927122653184, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149446, 'block_size': 4096, 'inode_available': 60905530}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226348304, "block_size": 4096, "block_total": 240095108, "block_used": 13746804, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905530, "inode_total": 61054976, "inode_used": 149446, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927122653184, "size_total": 983429562368, "uuid": "b40bdd1d-29ce-4dab-993d-cbf4157b4f12"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:10.910 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:10 +0000 (0:00:00.101) 0:01:29.429 ***** 2020-09-08T15:23:10.964 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-09-08T15:23:10.966 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:11.019 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:11 +0000 (0:00:00.104) 0:01:29.534 ***** 2020-09-08T15:23:11.028 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-09-08T15:23:11.030 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-09-08T15:23:11.052 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-09-08T15:23:11.054 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.168701", "end": "2020-09-08 15:23:10.788075", "rc": 0, "start": "2020-09-08 15:23:10.619374", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : set_fact] ***************************************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:11.093 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:11 +0000 (0:00:00.078) 0:01:29.612 ***** 2020-09-08T15:23:11.136 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:11 +0000 (0:00:00.042) 0:01:29.655 ***** 2020-09-08T15:23:11.147 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-08T15:23:11.148 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:11.189 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:11 +0000 (0:00:00.052) 0:01:29.708 ***** 2020-09-08T15:23:11.232 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-09-08T15:23:11.234 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:11.271 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:11 +0000 (0:00:00.082) 0:01:29.790 ***** 2020-09-08T15:23:11.282 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-09-08T15:23:11.284 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-09-08T15:23:11.305 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-09-08T15:23:11.307 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:11.344 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:11 +0000 (0:00:00.072) 0:01:29.863 ***** 2020-09-08T15:23:11.355 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-09-08T15:23:11.357 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.065111", "end": "2020-09-08 15:23:11.097090", "rc": 0, "start": "2020-09-08 15:23:11.031979", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:11.396 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:11 +0000 (0:00:00.052) 0:01:29.915 ***** 2020-09-08T15:23:11.406 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:11.446 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:11 +0000 (0:00:00.049) 0:01:29.964 ***** 2020-09-08T15:23:11.456 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-09-08T15:23:11.501 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:11 +0000 (0:00:00.055) 0:01:30.020 ***** 2020-09-08T15:23:11.520 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-09-08T15:23:11.522 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-09-08T15:23:11.543 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-09-08T15:23:11.545 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:11.586 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:11 +0000 (0:00:00.082) 0:01:30.102 ***** 2020-09-08T15:23:11.614 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-09-08T15:23:11.616 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:11.653 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:11 +0000 (0:00:00.069) 0:01:30.172 ***** 2020-09-08T15:23:11.684 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-09-08T15:23:11.686 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:11.725 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:11 +0000 (0:00:00.072) 0:01:30.244 ***** 2020-09-08T15:23:11.767 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-09-08T15:23:11.769 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:11.811 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:11 +0000 (0:00:00.085) 0:01:30.330 ***** 2020-09-08T15:23:11.822 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-09-08T15:23:11.824 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-09-08T15:23:11.825 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-09-08T15:23:11.867 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:11 +0000 (0:00:00.056) 0:01:30.387 ***** 2020-09-08T15:23:11.910 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:11 +0000 (0:00:00.042) 0:01:30.430 ***** 2020-09-08T15:23:11.953 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-09-08T15:23:11.955 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-09-08T15:23:11.993 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:11 +0000 (0:00:00.082) 0:01:30.513 ***** 2020-09-08T15:23:12.026 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-09-08T15:23:12.028 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-09-08T15:23:12.068 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:12 +0000 (0:00:00.072) 0:01:30.585 ***** 2020-09-08T15:23:12.078 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-09-08T15:23:12.080 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'block_used': 13746799, 'uuid': 'b40bdd1d-29ce-4dab-993d-cbf4157b4f12', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226348309, 'size_available': 927122673664, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149446, 'block_size': 4096, 'inode_available': 60905530}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226348309, "block_size": 4096, "block_total": 240095108, "block_used": 13746799, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905530, "inode_total": 61054976, "inode_used": 149446, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927122673664, "size_total": 983429562368, "uuid": "b40bdd1d-29ce-4dab-993d-cbf4157b4f12"}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:12.119 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:12 +0000 (0:00:00.053) 0:01:30.639 ***** 2020-09-08T15:23:12.166 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:12.203 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:12 +0000 (0:00:00.083) 0:01:30.722 ***** 2020-09-08T15:23:12.229 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-09-08T15:23:12.247 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-08T15:23:12.250 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-09-08T15:23:12.285 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:12 +0000 (0:00:00.082) 0:01:30.805 ***** 2020-09-08T15:23:12.326 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-08T15:23:12.328 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:12.364 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:12 +0000 (0:00:00.078) 0:01:30.883 ***** 2020-09-08T15:23:12.409 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-09-08T15:23:12.411 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-09-08T15:23:12.446 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:12 +0000 (0:00:00.082) 0:01:30.966 ***** 2020-09-08T15:23:12.468 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-09-08T15:23:12.470 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-09-08T15:23:12.480 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-09-08T15:23:12.481 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:12.520 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:12 +0000 (0:00:00.071) 0:01:31.038 ***** 2020-09-08T15:23:12.530 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-09-08T15:23:12.534 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-09-08T15:23:12.560 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:12.599 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:12 +0000 (0:00:00.079) 0:01:31.117 ***** 2020-09-08T15:23:12.631 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-09-08T15:23:12.633 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-09-08T15:23:12.643 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:12.683 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:12 +0000 (0:00:00.085) 0:01:31.202 ***** 2020-09-08T15:23:12.716 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-09-08T15:23:12.718 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-09-08T15:23:12.832 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-09-08T15:23:12.834 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-09-08T15:23:12.874 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:12 +0000 (0:00:00.191) 0:01:31.394 ***** 2020-09-08T15:23:12.938 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-09-08T15:23:12.940 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-09-08T15:23:12.979 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:12 +0000 (0:00:00.104) 0:01:31.498 ***** 2020-09-08T15:23:12.993 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} TASK [testnode : Create volume group(s)] *************************************** changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-09-08T15:23:13.034 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:13 +0000 (0:00:00.053) 0:01:31.552 ***** 2020-09-08T15:23:13.180 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-09-08T15:23:13.188 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-09-08T15:23:13.227 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:13 +0000 (0:00:00.193) 0:01:31.746 ***** 2020-09-08T15:23:13.339 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-09-08T15:23:13.371 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-09-08T15:23:13.373 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-09-08T15:23:13.429 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:13 +0000 (0:00:00.201) 0:01:31.948 ***** 2020-09-08T15:23:13.579 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-09-08T15:23:13.589 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-09-08T15:23:13.596 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-09-08T15:23:13.706 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-09-08T15:23:13.708 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => {"changed": true} 2020-09-08T15:23:13.747 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:13 +0000 (0:00:00.318) 0:01:32.266 ***** 2020-09-08T15:23:13.761 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-09-08T15:23:13.763 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-09-08T15:23:13.853 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-09-08T15:23:13.866 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-09-08T15:23:13.998 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-09-08T15:23:14.023 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:14.113 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:14 +0000 (0:00:00.362) 0:01:32.629 ***** 2020-09-08T15:23:14.124 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-09-08T15:23:14.126 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-09-08T15:23:14.268 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-09-08T15:23:14.269 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-09-08T15:23:14.306 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:14 +0000 (0:00:00.194) 0:01:32.825 ***** 2020-09-08T15:23:14.326 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-09-08T15:23:14.327 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-09-08T15:23:14.363 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:14 +0000 (0:00:00.057) 0:01:32.882 ***** 2020-09-08T15:23:14.376 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-09-08T15:23:14.378 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-09-08T15:23:14.423 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:14 +0000 (0:00:00.060) 0:01:32.942 ***** 2020-09-08T15:23:14.464 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:14 +0000 (0:00:00.041) 0:01:32.984 ***** 2020-09-08T15:23:14.532 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-09-08T15:23:14.534 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "gid": 1229, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-09-08T15:23:14.576 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:14 +0000 (0:00:00.111) 0:01:33.095 ***** 2020-09-08T15:23:14.619 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-09-08T15:23:14.621 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-09-08T15:23:14.673 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:14 +0000 (0:00:00.097) 0:01:33.192 ***** 2020-09-08T15:23:14.683 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-09-08T15:23:14.685 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => {"changed": true} 2020-09-08T15:23:14.730 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:14 +0000 (0:00:00.056) 0:01:33.249 ***** 2020-09-08T15:23:14.743 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-09-08T15:23:14.745 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-09-08T15:23:14.792 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:14 +0000 (0:00:00.061) 0:01:33.311 ***** 2020-09-08T15:23:14.838 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-09-08T15:23:14.840 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-09-08T15:23:14.935 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [smithi104.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-09-08T15:23:14.980 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:14 +0000 (0:00:00.188) 0:01:33.500 ***** 2020-09-08T15:23:14.994 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-09-08T15:23:14.996 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-09-08T15:23:15.085 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-09-08T15:23:15.167 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-09-08T15:23:15.169 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "gid": 1229, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-09-08T15:23:15.206 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:15 +0000 (0:00:00.225) 0:01:33.725 ***** 2020-09-08T15:23:15.219 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-09-08T15:23:15.221 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1229, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} TASK [testnode : Write /scratch_devs] ****************************************** 2020-09-08T15:23:15.221 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-09-08T15:23:15.261 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:15 +0000 (0:00:00.055) 0:01:33.780 ***** 2020-09-08T15:23:15.278 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:15.320 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:15 +0000 (0:00:00.059) 0:01:33.840 ***** 2020-09-08T15:23:15.352 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-09-08T15:23:15.354 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-09-08T15:23:15.393 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:15 +0000 (0:00:00.072) 0:01:33.912 ***** 2020-09-08T15:23:15.478 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-09-08T15:23:15.479 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-09-08T15:23:15.519 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:15 +0000 (0:00:00.126) 0:01:34.039 ***** 2020-09-08T15:23:15.562 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-09-08T15:23:15.564 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-09-08T15:23:15.603 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:15 +0000 (0:00:00.083) 0:01:34.122 ***** 2020-09-08T15:23:15.640 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-09-08T15:23:15.642 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-09-08T15:23:15.683 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:15 +0000 (0:00:00.080) 0:01:34.202 ***** 2020-09-08T15:23:15.814 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1229, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-09-08T15:23:15.882 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:15 +0000 (0:00:00.196) 0:01:34.399 ***** 2020-09-08T15:23:16.257 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-09-08T15:23:16.259 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-09-08T15:23:16.297 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:16 +0000 (0:00:00.416) 0:01:34.816 ***** 2020-09-08T15:23:16.624 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** ok: [smithi055.front.sepia.ceph.com] => {"cache_update_time": 1599578284, "cache_updated": false, "changed": false} 2020-09-08T15:23:16.665 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:16 +0000 (0:00:00.368) 0:01:35.184 ***** 2020-09-08T15:23:16.866 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [smithi166.front.sepia.ceph.com] => {"changed": true} 2020-09-08T15:23:16.903 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:16 +0000 (0:00:00.237) 0:01:35.422 ***** 2020-09-08T15:23:16.915 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-09-08T15:23:16.917 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.048488", "end": "2020-09-08 15:23:16.476930", "rc": 0, "start": "2020-09-08 15:23:16.428442", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-09-08T15:23:16.957 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:16 +0000 (0:00:00.054) 0:01:35.477 ***** 2020-09-08T15:23:17.002 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-09-08T15:23:17.005 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:17.046 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:17 +0000 (0:00:00.088) 0:01:35.565 ***** 2020-09-08T15:23:17.078 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-09-08T15:23:17.080 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:17.117 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:17 +0000 (0:00:00.071) 0:01:35.637 ***** 2020-09-08T15:23:17.163 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-09-08T15:23:17.165 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-09-08T15:23:17.204 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:17 +0000 (0:00:00.086) 0:01:35.723 ***** 2020-09-08T15:23:17.237 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-09-08T15:23:17.239 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"cache_update_time": 1599578240, "cache_updated": false, "changed": false} 2020-09-08T15:23:17.281 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:17 +0000 (0:00:00.077) 0:01:35.800 ***** 2020-09-08T15:23:17.366 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-09-08T15:23:17.368 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "gid": 1229, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-09-08T15:23:17.405 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:17 +0000 (0:00:00.124) 0:01:35.924 ***** 2020-09-08T15:23:17.478 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-09-08T15:23:17.480 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.045874", "end": "2020-09-08 15:23:17.836844", "rc": 0, "start": "2020-09-08 15:23:17.790970", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-09-08T15:23:17.516 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:17 +0000 (0:00:00.111) 0:01:36.036 ***** 2020-09-08T15:23:17.558 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-09-08T15:23:17.560 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-09-08T15:23:17.601 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:17 +0000 (0:00:00.084) 0:01:36.120 ***** 2020-09-08T15:23:17.611 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-09-08T15:23:17.613 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:17.651 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:17 +0000 (0:00:00.050) 0:01:36.170 ***** 2020-09-08T15:23:17.698 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-09-08T15:23:17.700 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:17.737 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:17 +0000 (0:00:00.085) 0:01:36.256 ***** 2020-09-08T15:23:17.989 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-09-08T15:23:17.990 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"cache_update_time": 1599578284, "cache_updated": false, "changed": false} 2020-09-08T15:23:18.027 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:18 +0000 (0:00:00.289) 0:01:36.546 ***** 2020-09-08T15:23:18.039 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-09-08T15:23:18.040 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1229, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-09-08T15:23:18.078 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:18 +0000 (0:00:00.051) 0:01:36.598 ***** 2020-09-08T15:23:18.090 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:18.129 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:18 +0000 (0:00:00.050) 0:01:36.649 ***** 2020-09-08T15:23:18.466 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-09-08T15:23:18.508 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:18 +0000 (0:00:00.378) 0:01:37.027 ***** 2020-09-08T15:23:18.517 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-09-08T15:23:18.518 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-09-08T15:23:18.557 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:18 +0000 (0:00:00.049) 0:01:37.076 ***** 2020-09-08T15:23:18.589 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-09-08T15:23:18.591 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:18.629 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:18 +0000 (0:00:00.072) 0:01:37.149 ***** 2020-09-08T15:23:18.661 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** ok: [smithi104.front.sepia.ceph.com] => {"cache_update_time": 1599578240, "cache_updated": false, "changed": false} 2020-09-08T15:23:18.704 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:18 +0000 (0:00:00.071) 0:01:37.220 ***** 2020-09-08T15:23:18.746 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-09-08T15:23:18.747 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:18.784 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:18 +0000 (0:00:00.083) 0:01:37.304 ***** 2020-09-08T15:23:18.850 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-09-08T15:23:18.853 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Tue 2020-09-08 15:21:17 UTC", "ActiveEnterTimestampMonotonic": "25273517", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sysinit.target -.mount basic.target systemd-journald.socket system.slice network.target systemd-tmpfiles-setup.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Tue 2020-09-08 15:21:11 UTC", "AssertTimestampMonotonic": "18736353", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Tue 2020-09-08 15:21:11 UTC", "ConditionTimestampMonotonic": "18736352", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2020-09-08T15:23:18.854 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1726", "ExecMainStartTimestamp": "Tue 2020-09-08 15:21:17 UTC", "ExecMainStartTimestampMonotonic": "25273503", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Tue 2020-09-08 15:21:11 UTC", "InactiveExitTimestampMonotonic": "18737235", "InvocationID": "2196f5fd41f54bcda3a4ed3cff69672a", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128055", "LimitNPROCSoft": "128055", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2020-09-08T15:23:18.858 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128055", "LimitSIGPENDINGSoft": "128055", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1726", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Tue 2020-09-08 15:21:17 UTC", "S 2020-09-08T15:23:18.859 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "25273517", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Tue 2020-09-08 15:21:17 UTC", "WatchdogTimestampMonotonic": "25273515", "WatchdogUSec": "0"}} 2020-09-08T15:23:18.910 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:18 +0000 (0:00:00.125) 0:01:37.429 ***** 2020-09-08T15:23:18.945 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-08T15:23:18.947 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-09-08T15:23:18.995 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:18 +0000 (0:00:00.084) 0:01:37.514 ***** 2020-09-08T15:23:19.046 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:19.086 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:19 +0000 (0:00:00.091) 0:01:37.605 ***** 2020-09-08T15:23:19.118 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-09-08T15:23:19.120 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:19.156 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:19 +0000 (0:00:00.070) 0:01:37.676 ***** 2020-09-08T15:23:19.181 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-09-08T15:23:19.183 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-09-08T15:23:19.220 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:19 +0000 (0:00:00.063) 0:01:37.740 ***** 2020-09-08T15:23:19.253 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-09-08T15:23:19.255 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:19.291 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:19 +0000 (0:00:00.070) 0:01:37.810 ***** 2020-09-08T15:23:19.437 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-09-08T15:23:19.439 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"cache_update_time": 1599578328, "cache_updated": false, "changed": false} 2020-09-08T15:23:19.474 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:19 +0000 (0:00:00.183) 0:01:37.994 ***** 2020-09-08T15:23:19.517 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-09-08T15:23:19.519 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Tue 2020-09-08 15:21:25 UTC", "ActiveEnterTimestampMonotonic": "28624803", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "-.mount network.target systemd-journald.socket system.slice systemd-tmpfiles-setup.service basic.target sysinit.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Tue 2020-09-08 15:21:18 UTC", "AssertTimestampMonotonic": "22208130", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Tue 2020-09-08 15:21:18 UTC", "ConditionTimestampMonotonic": "22208129", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2020-09-08T15:23:19.520 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1615", "ExecMainStartTimestamp": "Tue 2020-09-08 15:21:25 UTC", "ExecMainStartTimestampMonotonic": "28624790", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Tue 2020-09-08 15:21:18 UTC", "InactiveExitTimestampMonotonic": "22209924", "InvocationID": "6c53a5a0b5414ba98801e0fd651d3699", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128031", "LimitNPROCSoft": "128031", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2020-09-08T15:23:19.522 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128031", "LimitSIGPENDINGSoft": "128031", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1615", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Tue 2020-09-08 15:21:25 UTC", "S 2020-09-08T15:23:19.524 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "28624803", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Tue 2020-09-08 15:21:25 UTC", "WatchdogTimestampMonotonic": "28624802", "WatchdogUSec": "0"}} 2020-09-08T15:23:19.557 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:19 +0000 (0:00:00.082) 0:01:38.077 ***** 2020-09-08T15:23:19.588 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-08T15:23:19.590 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-09-08T15:23:19.626 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:19 +0000 (0:00:00.068) 0:01:38.145 ***** 2020-09-08T15:23:19.667 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-08T15:23:19.669 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:19.713 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:19 +0000 (0:00:00.078) 0:01:38.224 ***** 2020-09-08T15:23:19.722 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.046718", "end": "2020-09-08 15:23:19.431780", "rc": 0, "start": "2020-09-08 15:23:19.385062", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-09-08T15:23:19.760 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:19 +0000 (0:00:00.055) 0:01:38.280 ***** 2020-09-08T15:23:19.770 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-09-08T15:23:19.771 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:19.814 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:19 +0000 (0:00:00.054) 0:01:38.334 ***** 2020-09-08T15:23:19.826 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-09-08T15:23:19.828 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:19.864 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:19 +0000 (0:00:00.049) 0:01:38.384 ***** 2020-09-08T15:23:19.910 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:19.953 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:19 +0000 (0:00:00.086) 0:01:38.470 ***** 2020-09-08T15:23:20.091 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* ok: [smithi055.front.sepia.ceph.com] => {"cache_update_time": 1599578284, "cache_updated": false, "changed": false} 2020-09-08T15:23:20.136 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:20 +0000 (0:00:00.184) 0:01:38.655 ***** 2020-09-08T15:23:20.169 INFO:teuthology.task.ansible.out: 2020-09-08T15:23:20.171 INFO:teuthology.task.ansible.out:TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:20.214 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:20 +0000 (0:00:00.077) 0:01:38.733 ***** 2020-09-08T15:23:20.354 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-09-08T15:23:20.356 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "gid": 1229, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-09-08T15:23:20.397 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:20 +0000 (0:00:00.183) 0:01:38.916 ***** 2020-09-08T15:23:20.686 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-09-08T15:23:20.688 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"cache_update_time": 1599578240, "cache_updated": false, "changed": false} 2020-09-08T15:23:20.725 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:20 +0000 (0:00:00.328) 0:01:39.244 ***** 2020-09-08T15:23:20.769 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-09-08T15:23:20.771 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1229, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:20.809 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:20 +0000 (0:00:00.083) 0:01:39.328 ***** 2020-09-08T15:23:20.852 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:20 +0000 (0:00:00.042) 0:01:39.371 ***** 2020-09-08T15:23:20.866 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-09-08T15:23:20.868 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"cache_update_time": 1599578328, "cache_updated": false, "changed": false} 2020-09-08T15:23:20.908 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:20 +0000 (0:00:00.055) 0:01:39.427 ***** 2020-09-08T15:23:20.953 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-09-08T15:23:20.956 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:20.993 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:20 +0000 (0:00:00.085) 0:01:39.513 ***** 2020-09-08T15:23:21.017 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-09-08T15:23:21.019 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "gid": 1229, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-09-08T15:23:21.094 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:21 +0000 (0:00:00.089) 0:01:39.602 ***** 2020-09-08T15:23:21.361 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-09-08T15:23:21.363 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-09-08T15:23:21.403 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:21 +0000 (0:00:00.320) 0:01:39.923 ***** 2020-09-08T15:23:21.435 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-09-08T15:23:21.437 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:21.475 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:21 +0000 (0:00:00.071) 0:01:39.994 ***** 2020-09-08T15:23:21.486 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-09-08T15:23:21.487 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1229, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-09-08T15:23:21.531 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:21 +0000 (0:00:00.055) 0:01:40.050 ***** 2020-09-08T15:23:21.695 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-09-08T15:23:21.696 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Tue 2020-09-08 15:21:37 UTC", "ActiveEnterTimestampMonotonic": "27732337", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target system.slice -.mount systemd-tmpfiles-setup.service basic.target sysinit.target systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Tue 2020-09-08 15:21:30 UTC", "AssertTimestampMonotonic": "21308127", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Tue 2020-09-08 15:21:30 UTC", "ConditionTimestampMonotonic": "21308127", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2020-09-08T15:23:21.698 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1873", "ExecMainStartTimestamp": "Tue 2020-09-08 15:21:37 UTC", "ExecMainStartTimestampMonotonic": "27732277", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Tue 2020-09-08 15:21:30 UTC", "InactiveExitTimestampMonotonic": "21310381", "InvocationID": "c908c42e2c1a4613932a7fae9c7391d8", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128030", "LimitNPROCSoft": "128030", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2020-09-08T15:23:21.700 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128030", "LimitSIGPENDINGSoft": "128030", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1873", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Tue 2020-09-08 15:21:37 UTC", "S 2020-09-08T15:23:21.701 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "27732337", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Tue 2020-09-08 15:21:37 UTC", "WatchdogTimestampMonotonic": "27732333", "WatchdogUSec": "0"}} 2020-09-08T15:23:21.735 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:21 +0000 (0:00:00.203) 0:01:40.254 ***** 2020-09-08T15:23:21.769 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-08T15:23:21.771 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-09-08T15:23:21.819 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:21 +0000 (0:00:00.084) 0:01:40.339 ***** 2020-09-08T15:23:21.858 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-08T15:23:21.860 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:21.897 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:21 +0000 (0:00:00.077) 0:01:40.416 ***** 2020-09-08T15:23:21.934 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-09-08T15:23:21.936 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:21.976 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:21 +0000 (0:00:00.078) 0:01:40.495 ***** 2020-09-08T15:23:21.988 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-09-08T15:23:21.990 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-09-08T15:23:22.033 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:22 +0000 (0:00:00.057) 0:01:40.552 ***** 2020-09-08T15:23:22.356 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-09-08T15:23:22.398 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:22 +0000 (0:00:00.364) 0:01:40.917 ***** 2020-09-08T15:23:22.591 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-09-08T15:23:22.594 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false} 2020-09-08T15:23:22.656 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:22 +0000 (0:00:00.249) 0:01:41.167 ***** 2020-09-08T15:23:22.691 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-09-08T15:23:22.693 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} 2020-09-08T15:23:22.742 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:22 +0000 (0:00:00.094) 0:01:41.261 ***** 2020-09-08T15:23:22.855 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-09-08T15:23:22.857 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"cache_update_time": 1599578328, "cache_updated": false, "changed": false} 2020-09-08T15:23:22.894 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:22 +0000 (0:00:00.151) 0:01:41.413 ***** 2020-09-08T15:23:22.918 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-09-08T15:23:22.920 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1599578189.520735, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1599578189.5167332, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 98058, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1599578189.5167332, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 67, "uid": 0, "version": "3927509321", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-09-08T15:23:22.957 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:22 +0000 (0:00:00.062) 0:01:41.476 ***** 2020-09-08T15:23:22.969 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-09-08T15:23:22.971 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false} TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:23.023 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:23 +0000 (0:00:00.065) 0:01:41.542 ***** 2020-09-08T15:23:23.070 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:23 +0000 (0:00:00.046) 0:01:41.589 ***** 2020-09-08T15:23:23.083 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ ok: [smithi104.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-09-08T15:23:23.272 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:23 +0000 (0:00:00.201) 0:01:41.790 ***** 2020-09-08T15:23:23.396 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "gid": 1229, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-09-08T15:23:23.438 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:23 +0000 (0:00:00.166) 0:01:41.957 ***** 2020-09-08T15:23:23.450 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-09-08T15:23:23.452 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => {"changed": false, "msg": ""} TASK [testnode : Check for presence of /etc/network/interfaces] **************** ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1599578217.7401884, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1599578217.7361884, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 98058, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1599578217.7361884, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "3927509321", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-09-08T15:23:23.492 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:23 +0000 (0:00:00.053) 0:01:42.011 ***** 2020-09-08T15:23:23.534 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:23 +0000 (0:00:00.040) 0:01:42.052 ***** 2020-09-08T15:23:23.546 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** ok: [smithi055.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-09-08T15:23:23.592 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:23 +0000 (0:00:00.051) 0:01:42.103 ***** 2020-09-08T15:23:23.618 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-09-08T15:23:23.620 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:23.655 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:23 +0000 (0:00:00.071) 0:01:42.175 ***** 2020-09-08T15:23:23.698 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-09-08T15:23:23.700 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-09-08T15:23:23.762 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:23 +0000 (0:00:00.106) 0:01:42.281 ***** 2020-09-08T15:23:23.809 INFO:teuthology.task.ansible.out: 2020-09-08T15:23:23.811 INFO:teuthology.task.ansible.out:TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** ok: [smithi104.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-09-08T15:23:23.813 INFO:teuthology.task.ansible.out:TASK [testnode : Create pip.conf and configure it to use our mirror] *********** ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1229, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-09-08T15:23:23.815 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** changed: [smithi055.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-09-08T15:23:23.853 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:23 +0000 (0:00:00.090) 0:01:42.372 ***** 2020-09-08T15:23:23.855 INFO:teuthology.task.ansible.out: 2020-09-08T15:23:23.899 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:23 +0000 (0:00:00.046) 0:01:42.418 ***** 2020-09-08T15:23:23.901 INFO:teuthology.task.ansible.out: 2020-09-08T15:23:23.943 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:23 +0000 (0:00:00.043) 0:01:42.462 ***** 2020-09-08T15:23:23.945 INFO:teuthology.task.ansible.out: 2020-09-08T15:23:23.956 INFO:teuthology.task.ansible.out: 2020-09-08T15:23:23.957 INFO:teuthology.task.ansible.out:TASK [testnode : ifdown and ifup] ********************************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:24.004 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:23 +0000 (0:00:00.061) 0:01:42.524 ***** 2020-09-08T15:23:24.031 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* changed: [smithi055.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-09-08T15:23:24.068 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:24 +0000 (0:00:00.063) 0:01:42.587 ***** 2020-09-08T15:23:24.100 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-09-08T15:23:24.102 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:24.162 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:24 +0000 (0:00:00.093) 0:01:42.681 ***** 2020-09-08T15:23:24.174 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-09-08T15:23:24.176 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-09-08T15:23:24.222 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:24 +0000 (0:00:00.059) 0:01:42.741 ***** 2020-09-08T15:23:24.234 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [smithi055.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} 2020-09-08T15:23:24.236 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} 2020-09-08T15:23:24.277 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:24 +0000 (0:00:00.055) 0:01:42.796 ***** 2020-09-08T15:23:24.334 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-09-08T15:23:24.336 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:24.346 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi055.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:24.357 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:24.363 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:24.400 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:24 +0000 (0:00:00.123) 0:01:42.919 ***** 2020-09-08T15:23:24.411 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-09-08T15:23:24.412 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-09-08T15:23:24.465 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:24 +0000 (0:00:00.065) 0:01:42.985 ***** 2020-09-08T15:23:24.477 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-08T15:23:24.479 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:24.522 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:24 +0000 (0:00:00.053) 0:01:43.038 ***** 2020-09-08T15:23:24.530 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-09-08T15:23:24.531 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:24.571 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:24 +0000 (0:00:00.051) 0:01:43.090 ***** 2020-09-08T15:23:24.582 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-09-08T15:23:24.584 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:24.632 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:24 +0000 (0:00:00.060) 0:01:43.151 ***** 2020-09-08T15:23:24.649 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-09-08T15:23:24.652 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-09-08T15:23:24.692 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:24 +0000 (0:00:00.060) 0:01:43.212 ***** 2020-09-08T15:23:24.725 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:24.750 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:24 +0000 (0:00:00.054) 0:01:43.266 ***** 2020-09-08T15:23:24.762 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-09-08T15:23:24.801 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:24 +0000 (0:00:00.053) 0:01:43.320 ***** 2020-09-08T15:23:24.834 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** skipping: [smithi104.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Define net_to_configure var] ********************************** skipping: [smithi104.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_r 2020-09-08T15:23:24.842 INFO:teuthology.task.ansible.out:eason": "Conditional result was False"} skipping: [smithi104.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:24.865 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:24 +0000 (0:00:00.064) 0:01:43.384 ***** 2020-09-08T15:23:24.906 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:24 +0000 (0:00:00.041) 0:01:43.425 ***** 2020-09-08T15:23:24.915 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-09-08T15:23:24.917 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:24.955 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:24 +0000 (0:00:00.049) 0:01:43.474 ***** 2020-09-08T15:23:24.966 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:25.010 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:25 +0000 (0:00:00.054) 0:01:43.529 ***** 2020-09-08T15:23:25.044 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-09-08T15:23:25.046 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:25.087 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:25 +0000 (0:00:00.076) 0:01:43.606 ***** 2020-09-08T15:23:25.096 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:25.135 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:25 +0000 (0:00:00.047) 0:01:43.654 ***** 2020-09-08T15:23:25.154 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:25.186 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:25 +0000 (0:00:00.051) 0:01:43.706 ***** 2020-09-08T15:23:25.197 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-09-08T15:23:25.199 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:25.241 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:25 +0000 (0:00:00.054) 0:01:43.760 ***** 2020-09-08T15:23:25.253 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-09-08T15:23:25.255 INFO:teuthology.task.ansible.out:skipping: [smithi055.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:25.292 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:25 +0000 (0:00:00.051) 0:01:43.811 ***** 2020-09-08T15:23:25.303 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-09-08T15:23:25.310 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:25.344 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:25 +0000 (0:00:00.052) 0:01:43.864 ***** 2020-09-08T15:23:25.362 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* ok: [smithi166.front.sepia.ceph.com] => {"changed": false} 2020-09-08T15:23:25.401 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:25 +0000 (0:00:00.056) 0:01:43.920 ***** 2020-09-08T15:23:25.412 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-09-08T15:23:25.414 INFO:teuthology.task.ansible.out:skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:25.454 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:25 +0000 (0:00:00.052) 0:01:43.973 ***** 2020-09-08T15:23:25.466 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ ok: [smithi166.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-09-08T15:23:25.507 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:25 +0000 (0:00:00.053) 0:01:44.026 ***** 2020-09-08T15:23:25.526 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi055.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.010717", "end": "2020-09-08 15:23:25.071493", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-09-08 15:23:25.060776", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:25.567 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:25 +0000 (0:00:00.060) 0:01:44.086 ***** 2020-09-08T15:23:25.604 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi055.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006124", "end": "2020-09-08 15:23:25.204928", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-09-08 15:23:25.198804", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:25.643 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:25 +0000 (0:00:00.076) 0:01:44.163 ***** 2020-09-08T15:23:25.678 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-09-08T15:23:25.680 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1599578208.5676723, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1599578208.5636723, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 98058, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1599578208.5636723, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "3927509321", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi104.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:25.722 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:25 +0000 (0:00:00.074) 0:01:44.238 ***** 2020-09-08T15:23:25.759 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:25 +0000 (0:00:00.039) 0:01:44.277 ***** 2020-09-08T15:23:25.782 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-09-08T15:23:25.784 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005666", "end": "2020-09-08 15:23:25.335081", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-09-08 15:23:25.329415", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:25.864 INFO:teuthology.task.ansible.out:ok: [smithi055.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.018778", "end": "2020-09-08 15:23:25.475619", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-09-08 15:23:25.456841", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:25.905 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:25 +0000 (0:00:00.146) 0:01:44.423 ***** 2020-09-08T15:23:25.918 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.011022", "end": "2020-09-08 15:23:26.226152", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-09-08 15:23:26.215130", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:25.928 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-09-08T15:23:25.930 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-09-08T15:23:25.971 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:25 +0000 (0:00:00.067) 0:01:44.490 ***** 2020-09-08T15:23:25.983 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-09-08T15:23:26.021 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:26 +0000 (0:00:00.049) 0:01:44.540 ***** 2020-09-08T15:23:26.032 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-09-08T15:23:26.034 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-09-08T15:23:26.076 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:26 +0000 (0:00:00.054) 0:01:44.595 ***** 2020-09-08T15:23:26.087 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi104.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006008", "end": "2020-09-08 15:23:26.384491", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-09-08 15:23:26.378483", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:26.122 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:26.179 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:26 +0000 (0:00:00.102) 0:01:44.698 ***** 2020-09-08T15:23:26.189 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-09-08T15:23:26.191 INFO:teuthology.task.ansible.out:changed: [smithi055.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi104.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005478", "end": "2020-09-08 15:23:26.509325", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-09-08 15:23:26.503847", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:26.296 INFO:teuthology.task.ansible.out:ok: [smithi104.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.021108", "end": "2020-09-08 15:23:26.643251", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-09-08 15:23:26.622143", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:26.333 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:26 +0000 (0:00:00.153) 0:01:44.852 ***** 2020-09-08T15:23:26.343 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-09-08T15:23:26.345 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-09-08T15:23:26.382 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:26 +0000 (0:00:00.049) 0:01:44.901 ***** 2020-09-08T15:23:26.392 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-09-08T15:23:26.431 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:26 +0000 (0:00:00.048) 0:01:44.950 ***** 2020-09-08T15:23:26.528 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-09-08T15:23:26.530 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-09-08T15:23:26.567 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:26 +0000 (0:00:00.136) 0:01:45.087 ***** 2020-09-08T15:23:26.588 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-09-08T15:23:26.590 INFO:teuthology.task.ansible.out:changed: [smithi104.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-09-08T15:23:26.626 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-09-08T15:23:26.628 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:26.686 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:26 +0000 (0:00:00.118) 0:01:45.205 ***** 2020-09-08T15:23:26.727 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-09-08T15:23:26.729 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2020-09-08T15:23:26.738 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-09-08T15:23:26.740 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-09-08T15:23:26.750 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-09-08T15:23:26.751 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-09-08T15:23:26.788 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:26 +0000 (0:00:00.102) 0:01:45.307 ***** 2020-09-08T15:23:26.829 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-09-08T15:23:26.831 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:26.840 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:26.842 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:26.852 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:26.864 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:26.902 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:26 +0000 (0:00:00.114) 0:01:45.421 ***** 2020-09-08T15:23:26.932 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-09-08T15:23:26.934 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:26.974 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:26 +0000 (0:00:00.072) 0:01:45.494 ***** 2020-09-08T15:23:27.005 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-09-08T15:23:27.008 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:27.045 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:27 +0000 (0:00:00.070) 0:01:45.564 ***** 2020-09-08T15:23:27.077 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-09-08T15:23:27.079 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:27.117 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:27 +0000 (0:00:00.072) 0:01:45.637 ***** 2020-09-08T15:23:27.151 INFO:teuthology.task.ansible.out: 2020-09-08T15:23:27.153 INFO:teuthology.task.ansible.out:TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:27.192 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:27 +0000 (0:00:00.074) 0:01:45.711 ***** 2020-09-08T15:23:27.234 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-09-08T15:23:27.236 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:27.282 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:27 +0000 (0:00:00.090) 0:01:45.801 ***** 2020-09-08T15:23:27.314 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-09-08T15:23:27.316 INFO:teuthology.task.ansible.out:skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-09-08T15:23:27.376 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:27 +0000 (0:00:00.086) 0:01:45.888 ***** 2020-09-08T15:23:27.869 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Tuesday 08 September 2020 15:23:27 +0000 (0:00:00.075) 0:01:45.964 ***** TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi166.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Tuesday 08 September 2020 15:23:27 +0000 (0:00:00.070) 0:01:46.035 ***** TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi166.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.011209", "end": "2020-09-08 15:23:27.481955", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-09-08 15:23:27.470746", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:27.878 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006138", "end": "2020-09-08 15:23:27.632041", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-09-08 15:23:27.625903", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:28.017 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005571", "end": "2020-09-08 15:23:27.772531", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-09-08 15:23:27.766960", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:28.183 INFO:teuthology.task.ansible.out:ok: [smithi166.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.042377", "end": "2020-09-08 15:23:27.933571", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-09-08 15:23:27.891194", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-09-08T15:23:28.229 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:28 +0000 (0:00:00.713) 0:01:46.748 ***** 2020-09-08T15:23:28.287 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-09-08T15:23:28.320 INFO:teuthology.task.ansible.out:Tuesday 08 September 2020 15:23:28 +0000 (0:00:00.091) 0:01:46.840 ***** 2020-09-08T15:23:28.479 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-09-08T15:23:28.481 INFO:teuthology.task.ansible.out:changed: [smithi166.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-09-08T15:23:28.582 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2020-09-08T15:23:28.586 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** 2020-09-08T15:23:28.588 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2020-09-08T15:23:28.592 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched PLAY RECAP ********************************************************************* smithi055.front.sepia.ceph.com : ok=108 changed=28 unreachable=0 failed=0 skipped=181 rescued=0 ignored=0 smithi104.front.sepia.ceph.com : ok=108 changed=28 unreachable=0 failed=0 skipped=181 rescued=0 ignored=0 smithi166.front.sepia.ceph.com : ok=108 changed=28 unreachable=0 failed=0 skipped=181 rescued=0 ignored=0 Tuesday 08 September 2020 15:23:28 +0000 (0:00:00.270) 0:01:47.110 ***** =============================================================================== testnode : Ensure packages are not present. ----------------------------- 6.80s testnode : Install packages --------------------------------------------- 6.33s testnode : Zap all non-root disks --------------------------------------- 5.56s Gathering Facts --------------------------------------------------------- 1.99s common : Update apt cache ----------------------------------------------- 1.01s testnode : Upgrade packages --------------------------------------------- 0.95s Gathering Facts --------------------------------------------------------- 0.84s ansible-managed : Create the cephlab_sudo sudoers.d file. --------------- 0.74s testnode : List any leftover Ceph artifacts from previous jobs ---------- 0.71s ansible-managed : Add authorized keys for the ansible user. ------------- 0.64s ansible-managed : Create the ansible user. ------------------------------ 0.58s testnode : Set a high max open files limit for the teuthology user. ----- 0.45s common : Install Kerberos Packages (Debian) ----------------------------- 0.43s ansible-managed : Delete the ansible users password. -------------------- 0.43s testnode : Set apt preferences ------------------------------------------ 0.42s common : Make sure smartmontools is installed -------- 2020-09-08T15:23:28.594 INFO:teuthology.task.ansible.out:------------------- 0.41s testnode : Install python-apt ------------------------------------------- 0.41s testnode : Update apt cache. -------------------------------------------- 0.37s ansible-managed : Create the sudo group. -------------------------------- 0.35s testnode : Check for teuthology-lock command ---------------------------- 0.30s 2020-09-08T15:23:28.957 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi104.front.sepia.ceph.com'), Remote(name='ubuntu@smithi055.front.sepia.ceph.com'), Remote(name='ubuntu@smithi166.front.sepia.ceph.com')] 2020-09-08T15:23:28.959 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-09-08T15:23:28.962 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:23:29.041 INFO:teuthology.orchestra.run.smithi104:> true 2020-09-08T15:23:29.223 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-09-08T15:23:29.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:23:29.300 INFO:teuthology.orchestra.run.smithi055:> true 2020-09-08T15:23:29.507 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-09-08T15:23:29.508 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi166.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-09-08T15:23:29.581 INFO:teuthology.orchestra.run.smithi166:> true 2020-09-08T15:23:29.775 INFO:teuthology.run_tasks:Running task clock... 2020-09-08T15:23:29.786 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2020-09-08T15:23:29.787 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-09-08T15:23:29.788 INFO:teuthology.orchestra.run.smithi104:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-09-08T15:23:29.863 INFO:teuthology.orchestra.run.smithi104.stdout: 8 Sep 15:23:30 ntpd[5559]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-09-08T15:23:29.863 INFO:teuthology.orchestra.run.smithi104.stdout: 8 Sep 15:23:30 ntpd[5559]: Command line: ntpd -gq 2020-09-08T15:23:29.865 INFO:teuthology.orchestra.run.smithi104.stdout: 8 Sep 15:23:30 ntpd[5559]: proto: precision = 0.124 usec (-23) 2020-09-08T15:23:29.865 INFO:teuthology.orchestra.run.smithi104.stdout: 8 Sep 15:23:30 ntpd[5559]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-09-08T15:23:29.865 INFO:teuthology.orchestra.run.smithi104.stdout: 8 Sep 15:23:30 ntpd[5559]: restrict ::: KOD does nothing without LIMITED. 2020-09-08T15:23:29.866 INFO:teuthology.orchestra.run.smithi104.stdout: 8 Sep 15:23:30 ntpd[5559]: Listen and drop on 0 v6wildcard [::]:123 2020-09-08T15:23:29.866 INFO:teuthology.orchestra.run.smithi104.stdout: 8 Sep 15:23:30 ntpd[5559]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-09-08T15:23:29.866 INFO:teuthology.orchestra.run.smithi104.stdout: 8 Sep 15:23:30 ntpd[5559]: Listen normally on 2 lo 127.0.0.1:123 2020-09-08T15:23:29.867 INFO:teuthology.orchestra.run.smithi104.stdout: 8 Sep 15:23:30 ntpd[5559]: Listen normally on 3 enp3s0f1 172.21.15.104:123 2020-09-08T15:23:29.867 INFO:teuthology.orchestra.run.smithi104.stdout: 8 Sep 15:23:30 ntpd[5559]: Listen normally on 4 lo [::1]:123 2020-09-08T15:23:29.868 INFO:teuthology.orchestra.run.smithi104.stdout: 8 Sep 15:23:30 ntpd[5559]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:ce89%5]:123 2020-09-08T15:23:29.868 INFO:teuthology.orchestra.run.smithi104.stdout: 8 Sep 15:23:30 ntpd[5559]: Listening on routing socket on fd #22 for interface updates 2020-09-08T15:23:29.868 INFO:teuthology.orchestra.run.smithi104.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-09-08T15:23:29.869 INFO:teuthology.orchestra.run.smithi104.stderr:restrict ::: KOD does nothing without LIMITED. 2020-09-08T15:23:30.864 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:31 ntpd[5559]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:30.865 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:31 ntpd[5559]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:30.865 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:31 ntpd[5559]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:30.865 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:31 ntpd[5559]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:31.867 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:32 ntpd[5559]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:31.867 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:32 ntpd[5559]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:31.867 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:32 ntpd[5559]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:31.867 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:32 ntpd[5559]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:31.868 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:32 ntpd[5559]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:31.868 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:32 ntpd[5559]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:31.868 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:32 ntpd[5559]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:31.868 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:32 ntpd[5559]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:32.865 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:33 ntpd[5559]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:32.865 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:33 ntpd[5559]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:32.865 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:33 ntpd[5559]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:32.866 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:33 ntpd[5559]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:33.866 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:34 ntpd[5559]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:33.866 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:34 ntpd[5559]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:33.866 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:34 ntpd[5559]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:33.866 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:34 ntpd[5559]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:33.867 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:34 ntpd[5559]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:33.867 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:34 ntpd[5559]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:33.867 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:34 ntpd[5559]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:33.867 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:34 ntpd[5559]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:34.866 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:35 ntpd[5559]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:34.867 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:35 ntpd[5559]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:34.867 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:35 ntpd[5559]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:34.867 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:35 ntpd[5559]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:34.867 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:35 ntpd[5559]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:34.867 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:35 ntpd[5559]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:34.868 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:35 ntpd[5559]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:34.868 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:35 ntpd[5559]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:35.875 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:36 ntpd[5559]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:35.875 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:36 ntpd[5559]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:35.875 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:36 ntpd[5559]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:35.876 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:36 ntpd[5559]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:35.876 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:36 ntpd[5559]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:35.876 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:36 ntpd[5559]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:35.876 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:36 ntpd[5559]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:35.876 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:36 ntpd[5559]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:36.872 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:37 ntpd[5559]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:36.872 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:37 ntpd[5559]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:36.872 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:37 ntpd[5559]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:36.873 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:37 ntpd[5559]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:36.873 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:36 ntpd[5559]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-09-08T15:23:36.873 INFO:teuthology.orchestra.run.smithi104.stderr: 8 Sep 15:23:36 ntpd[5559]: can't open /var/log/ntpstats/loopstats.20200908: Permission denied 2020-09-08T15:23:36.873 INFO:teuthology.orchestra.run.smithi104.stdout: 8 Sep 15:23:36 ntpd[5559]: ntpd: time set -0.367005 s 2020-09-08T15:23:36.874 INFO:teuthology.orchestra.run.smithi104.stdout:ntpd: time set -0.367005s 2020-09-08T15:23:36.935 INFO:teuthology.orchestra.run.smithi104.stdout: remote refid st t when poll reach delay offset jitter 2020-09-08T15:23:36.935 INFO:teuthology.orchestra.run.smithi104.stdout:============================================================================== 2020-09-08T15:23:36.936 INFO:teuthology.orchestra.run.smithi104.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-09-08T15:23:36.936 INFO:teuthology.orchestra.run.smithi104.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-09-08T15:23:36.936 INFO:teuthology.orchestra.run.smithi104.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-09-08T15:23:36.936 INFO:teuthology.orchestra.run.smithi104.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-09-08T15:23:36.937 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-09-08T15:23:36.938 INFO:teuthology.orchestra.run.smithi055:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-09-08T15:23:36.992 INFO:teuthology.orchestra.run.smithi055.stdout: 8 Sep 15:23:36 ntpd[5670]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-09-08T15:23:36.992 INFO:teuthology.orchestra.run.smithi055.stdout: 8 Sep 15:23:36 ntpd[5670]: Command line: ntpd -gq 2020-09-08T15:23:36.993 INFO:teuthology.orchestra.run.smithi055.stdout: 8 Sep 15:23:36 ntpd[5670]: proto: precision = 0.127 usec (-23) 2020-09-08T15:23:36.994 INFO:teuthology.orchestra.run.smithi055.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-09-08T15:23:36.995 INFO:teuthology.orchestra.run.smithi055.stderr:restrict ::: KOD does nothing without LIMITED. 2020-09-08T15:23:36.995 INFO:teuthology.orchestra.run.smithi055.stdout: 8 Sep 15:23:36 ntpd[5670]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-09-08T15:23:36.995 INFO:teuthology.orchestra.run.smithi055.stdout: 8 Sep 15:23:36 ntpd[5670]: restrict ::: KOD does nothing without LIMITED. 2020-09-08T15:23:36.995 INFO:teuthology.orchestra.run.smithi055.stdout: 8 Sep 15:23:36 ntpd[5670]: Listen and drop on 0 v6wildcard [::]:123 2020-09-08T15:23:36.995 INFO:teuthology.orchestra.run.smithi055.stdout: 8 Sep 15:23:36 ntpd[5670]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-09-08T15:23:36.996 INFO:teuthology.orchestra.run.smithi055.stdout: 8 Sep 15:23:36 ntpd[5670]: Listen normally on 2 lo 127.0.0.1:123 2020-09-08T15:23:36.996 INFO:teuthology.orchestra.run.smithi055.stdout: 8 Sep 15:23:36 ntpd[5670]: Listen normally on 3 ens1f0 172.21.15.55:123 2020-09-08T15:23:36.996 INFO:teuthology.orchestra.run.smithi055.stdout: 8 Sep 15:23:36 ntpd[5670]: Listen normally on 4 lo [::1]:123 2020-09-08T15:23:36.996 INFO:teuthology.orchestra.run.smithi055.stdout: 8 Sep 15:23:36 ntpd[5670]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:2606%3]:123 2020-09-08T15:23:36.996 INFO:teuthology.orchestra.run.smithi055.stdout: 8 Sep 15:23:36 ntpd[5670]: Listening on routing socket on fd #22 for interface updates 2020-09-08T15:23:37.995 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:37 ntpd[5670]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:37.996 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:37 ntpd[5670]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:37.996 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:37 ntpd[5670]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:37.997 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:37 ntpd[5670]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:37.997 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:37 ntpd[5670]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:37.997 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:37 ntpd[5670]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:37.998 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:37 ntpd[5670]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:37.998 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:37 ntpd[5670]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:37.998 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:37 ntpd[5670]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:37.998 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:37 ntpd[5670]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:37.999 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:37 ntpd[5670]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:37.999 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:37 ntpd[5670]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:39.992 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:39 ntpd[5670]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:39.992 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:39 ntpd[5670]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:39.992 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:39 ntpd[5670]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:39.992 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:39 ntpd[5670]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:39.993 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:39 ntpd[5670]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:39.993 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:39 ntpd[5670]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:39.993 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:39 ntpd[5670]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:39.993 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:39 ntpd[5670]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:39.995 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:39 ntpd[5670]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:39.996 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:39 ntpd[5670]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:39.996 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:39 ntpd[5670]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:39.996 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:39 ntpd[5670]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:40.992 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:40 ntpd[5670]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:40.992 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:40 ntpd[5670]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:40.993 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:40 ntpd[5670]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:40.993 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:40 ntpd[5670]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:41.992 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:41 ntpd[5670]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:41.993 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:41 ntpd[5670]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:41.993 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:41 ntpd[5670]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:41.994 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:41 ntpd[5670]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:41.994 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:41 ntpd[5670]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:41.994 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:41 ntpd[5670]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:41.995 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:41 ntpd[5670]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:41.995 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:41 ntpd[5670]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:41.995 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:41 ntpd[5670]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:41.996 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:41 ntpd[5670]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:41.996 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:41 ntpd[5670]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:41.996 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:41 ntpd[5670]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:42.997 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:42 ntpd[5670]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:42.997 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:42 ntpd[5670]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:42.997 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:42 ntpd[5670]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:42.998 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:42 ntpd[5670]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:43.990 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:43 ntpd[5670]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:43.990 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:43 ntpd[5670]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:43.990 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:43 ntpd[5670]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:43.991 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:43 ntpd[5670]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:43.991 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:43 ntpd[5670]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-09-08T15:23:43.991 INFO:teuthology.orchestra.run.smithi055.stderr: 8 Sep 15:23:43 ntpd[5670]: can't open /var/log/ntpstats/loopstats.20200908: Permission denied 2020-09-08T15:23:43.991 INFO:teuthology.orchestra.run.smithi055.stdout: 8 Sep 15:23:43 ntpd[5670]: ntpd: time set +0.358452 s 2020-09-08T15:23:43.992 INFO:teuthology.orchestra.run.smithi055.stdout:ntpd: time set +0.358452s 2020-09-08T15:23:44.044 INFO:teuthology.orchestra.run.smithi055.stdout: remote refid st t when poll reach delay offset jitter 2020-09-08T15:23:44.044 INFO:teuthology.orchestra.run.smithi055.stdout:============================================================================== 2020-09-08T15:23:44.044 INFO:teuthology.orchestra.run.smithi055.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-09-08T15:23:44.044 INFO:teuthology.orchestra.run.smithi055.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-09-08T15:23:44.045 INFO:teuthology.orchestra.run.smithi055.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-09-08T15:23:44.045 INFO:teuthology.orchestra.run.smithi055.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-09-08T15:23:44.046 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-09-08T15:23:44.046 INFO:teuthology.orchestra.run.smithi166:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-09-08T15:23:44.110 INFO:teuthology.orchestra.run.smithi166.stdout: 8 Sep 15:23:43 ntpd[5743]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-09-08T15:23:44.110 INFO:teuthology.orchestra.run.smithi166.stdout: 8 Sep 15:23:43 ntpd[5743]: Command line: ntpd -gq 2020-09-08T15:23:44.111 INFO:teuthology.orchestra.run.smithi166.stdout: 8 Sep 15:23:43 ntpd[5743]: proto: precision = 0.127 usec (-23) 2020-09-08T15:23:44.111 INFO:teuthology.orchestra.run.smithi166.stdout: 8 Sep 15:23:43 ntpd[5743]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-09-08T15:23:44.111 INFO:teuthology.orchestra.run.smithi166.stdout: 8 Sep 15:23:43 ntpd[5743]: restrict ::: KOD does nothing without LIMITED. 2020-09-08T15:23:44.112 INFO:teuthology.orchestra.run.smithi166.stdout: 8 Sep 15:23:43 ntpd[5743]: Listen and drop on 0 v6wildcard [::]:123 2020-09-08T15:23:44.112 INFO:teuthology.orchestra.run.smithi166.stdout: 8 Sep 15:23:43 ntpd[5743]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-09-08T15:23:44.112 INFO:teuthology.orchestra.run.smithi166.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-09-08T15:23:44.112 INFO:teuthology.orchestra.run.smithi166.stderr:restrict ::: KOD does nothing without LIMITED. 2020-09-08T15:23:44.113 INFO:teuthology.orchestra.run.smithi166.stdout: 8 Sep 15:23:43 ntpd[5743]: Listen normally on 2 lo 127.0.0.1:123 2020-09-08T15:23:44.113 INFO:teuthology.orchestra.run.smithi166.stdout: 8 Sep 15:23:43 ntpd[5743]: Listen normally on 3 enp3s0f1 172.21.15.166:123 2020-09-08T15:23:44.114 INFO:teuthology.orchestra.run.smithi166.stdout: 8 Sep 15:23:43 ntpd[5743]: Listen normally on 4 lo [::1]:123 2020-09-08T15:23:44.114 INFO:teuthology.orchestra.run.smithi166.stdout: 8 Sep 15:23:43 ntpd[5743]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:77ed%5]:123 2020-09-08T15:23:44.114 INFO:teuthology.orchestra.run.smithi166.stdout: 8 Sep 15:23:43 ntpd[5743]: Listening on routing socket on fd #22 for interface updates 2020-09-08T15:23:45.110 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:44 ntpd[5743]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:45.111 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:44 ntpd[5743]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:45.111 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:44 ntpd[5743]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:45.111 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:44 ntpd[5743]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:46.111 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:45 ntpd[5743]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:46.112 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:45 ntpd[5743]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:46.112 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:45 ntpd[5743]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:46.112 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:45 ntpd[5743]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:47.109 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:46 ntpd[5743]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:47.110 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:46 ntpd[5743]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:47.110 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:46 ntpd[5743]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:47.110 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:46 ntpd[5743]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:47.110 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:46 ntpd[5743]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:47.111 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:46 ntpd[5743]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:47.111 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:46 ntpd[5743]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:47.111 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:46 ntpd[5743]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:48.114 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:47 ntpd[5743]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:48.115 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:47 ntpd[5743]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:48.115 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:47 ntpd[5743]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:48.115 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:47 ntpd[5743]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:48.115 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:47 ntpd[5743]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:48.116 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:47 ntpd[5743]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:48.116 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:47 ntpd[5743]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:48.116 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:47 ntpd[5743]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:49.109 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:48 ntpd[5743]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:49.109 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:48 ntpd[5743]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:49.109 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:48 ntpd[5743]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:49.110 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:48 ntpd[5743]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:49.110 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:48 ntpd[5743]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:49.110 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:48 ntpd[5743]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:49.110 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:48 ntpd[5743]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:49.110 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:48 ntpd[5743]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:50.108 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:49 ntpd[5743]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:50.108 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:49 ntpd[5743]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:50.109 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:49 ntpd[5743]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:50.109 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:49 ntpd[5743]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:50.109 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:49 ntpd[5743]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:50.109 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:49 ntpd[5743]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:50.110 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:49 ntpd[5743]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:50.110 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:49 ntpd[5743]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:51.114 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:50 ntpd[5743]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-09-08T15:23:51.115 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:50 ntpd[5743]: can't open /var/log/ntpstats/rawstats.20200908: Permission denied 2020-09-08T15:23:51.116 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:50 ntpd[5743]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-09-08T15:23:51.116 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:50 ntpd[5743]: can't open /var/log/ntpstats/peerstats.20200908: Permission denied 2020-09-08T15:23:51.116 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:51 ntpd[5743]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-09-08T15:23:51.117 INFO:teuthology.orchestra.run.smithi166.stderr: 8 Sep 15:23:51 ntpd[5743]: can't open /var/log/ntpstats/loopstats.20200908: Permission denied 2020-09-08T15:23:51.117 INFO:teuthology.orchestra.run.smithi166.stdout: 8 Sep 15:23:51 ntpd[5743]: ntpd: time set +0.213140 s 2020-09-08T15:23:51.118 INFO:teuthology.orchestra.run.smithi166.stdout:ntpd: time set +0.213140s 2020-09-08T15:23:51.162 INFO:teuthology.orchestra.run.smithi166.stdout: remote refid st t when poll reach delay offset jitter 2020-09-08T15:23:51.163 INFO:teuthology.orchestra.run.smithi166.stdout:============================================================================== 2020-09-08T15:23:51.163 INFO:teuthology.orchestra.run.smithi166.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-09-08T15:23:51.163 INFO:teuthology.orchestra.run.smithi166.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-09-08T15:23:51.164 INFO:teuthology.orchestra.run.smithi166.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-09-08T15:23:51.164 INFO:teuthology.orchestra.run.smithi166.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-09-08T15:23:51.165 INFO:teuthology.run_tasks:Running task install... 2020-09-08T15:23:51.184 DEBUG:teuthology.task.install:project ceph 2020-09-08T15:23:51.185 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': 'dce4d291bd4243c9802c17d67b85a6da6bbc0f8f'}} 2020-09-08T15:23:51.185 DEBUG:teuthology.task.install:config {'branch': 'mimic', 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-immutable-object-cache', 'ceph-base', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'librados-devel'], 'extra_packages': ['librados2', 'python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'sha1': 'dce4d291bd4243c9802c17d67b85a6da6bbc0f8f'} 2020-09-08T15:23:51.186 INFO:teuthology.task.install:Using flavor: basic 2020-09-08T15:23:51.197 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-mgr', 'ceph-fuse', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2020-09-08T15:23:51.198 INFO:teuthology.task.install:extra packages: ['librados2', 'python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'] 2020-09-08T15:23:51.199 INFO:teuthology.orchestra.run.smithi104:> sudo lsb_release -is 2020-09-08T15:23:51.251 INFO:teuthology.orchestra.run.smithi104.stdout:Ubuntu 2020-09-08T15:23:51.257 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-09-08T15:23:51.258 INFO:teuthology.orchestra.run.smithi055:> sudo lsb_release -is 2020-09-08T15:23:51.259 INFO:teuthology.orchestra.run.smithi104:> sudo apt-key list | grep Ceph 2020-09-08T15:23:51.313 INFO:teuthology.orchestra.run.smithi055.stdout:Ubuntu 2020-09-08T15:23:51.318 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-09-08T15:23:51.318 INFO:teuthology.orchestra.run.smithi166:> sudo lsb_release -is 2020-09-08T15:23:51.319 INFO:teuthology.orchestra.run.smithi055:> sudo apt-key list | grep Ceph 2020-09-08T15:23:51.351 INFO:teuthology.orchestra.run.smithi104.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-09-08T15:23:51.373 INFO:teuthology.orchestra.run.smithi166.stdout:Ubuntu 2020-09-08T15:23:51.378 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-09-08T15:23:51.379 INFO:teuthology.orchestra.run.smithi166:> sudo apt-key list | grep Ceph 2020-09-08T15:23:51.409 INFO:teuthology.orchestra.run.smithi104.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-09-08T15:23:51.410 INFO:teuthology.orchestra.run.smithi104.stdout:uid [ unknown] Ceph.com (release key) 2020-09-08T15:23:51.411 INFO:teuthology.orchestra.run.smithi104:> sudo lsb_release -is 2020-09-08T15:23:51.414 INFO:teuthology.orchestra.run.smithi055.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-09-08T15:23:51.465 INFO:teuthology.orchestra.run.smithi104.stdout:Ubuntu 2020-09-08T15:23:51.470 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-09-08T15:23:51.471 INFO:teuthology.task.install.deb:Installing packages: ceph, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, librados2, python-rados, python-rgw, python-rbd, python-cephfs on remote deb x86_64 2020-09-08T15:23:51.471 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-09-08T15:23:51.472 INFO:teuthology.packaging:ref: None 2020-09-08T15:23:51.472 INFO:teuthology.packaging:tag: None 2020-09-08T15:23:51.472 INFO:teuthology.packaging:branch: mimic 2020-09-08T15:23:51.472 INFO:teuthology.packaging:sha1: dce4d291bd4243c9802c17d67b85a6da6bbc0f8f 2020-09-08T15:23:51.473 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=mimic 2020-09-08T15:23:51.482 INFO:teuthology.orchestra.run.smithi166.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-09-08T15:23:51.482 INFO:teuthology.orchestra.run.smithi055.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-09-08T15:23:51.483 INFO:teuthology.orchestra.run.smithi055.stdout:uid [ unknown] Ceph.com (release key) 2020-09-08T15:23:51.483 INFO:teuthology.orchestra.run.smithi055:> sudo lsb_release -is 2020-09-08T15:23:51.537 INFO:teuthology.orchestra.run.smithi055.stdout:Ubuntu 2020-09-08T15:23:51.539 INFO:teuthology.orchestra.run.smithi166.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-09-08T15:23:51.539 INFO:teuthology.orchestra.run.smithi166.stdout:uid [ unknown] Ceph.com (release key) 2020-09-08T15:23:51.540 INFO:teuthology.orchestra.run.smithi166:> sudo lsb_release -is 2020-09-08T15:23:51.542 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-09-08T15:23:51.542 INFO:teuthology.task.install.deb:Installing packages: ceph, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, librados2, python-rados, python-rgw, python-rbd, python-cephfs on remote deb x86_64 2020-09-08T15:23:51.543 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-09-08T15:23:51.543 INFO:teuthology.packaging:ref: None 2020-09-08T15:23:51.543 INFO:teuthology.packaging:tag: None 2020-09-08T15:23:51.543 INFO:teuthology.packaging:branch: mimic 2020-09-08T15:23:51.546 INFO:teuthology.packaging:sha1: dce4d291bd4243c9802c17d67b85a6da6bbc0f8f 2020-09-08T15:23:51.546 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=mimic 2020-09-08T15:23:51.600 INFO:teuthology.orchestra.run.smithi166.stdout:Ubuntu 2020-09-08T15:23:51.605 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-09-08T15:23:51.606 INFO:teuthology.task.install.deb:Installing packages: ceph, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, librados2, python-rados, python-rgw, python-rbd, python-cephfs on remote deb x86_64 2020-09-08T15:23:51.607 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-09-08T15:23:51.607 INFO:teuthology.packaging:ref: None 2020-09-08T15:23:51.607 INFO:teuthology.packaging:tag: None 2020-09-08T15:23:51.607 INFO:teuthology.packaging:branch: mimic 2020-09-08T15:23:51.608 INFO:teuthology.packaging:sha1: dce4d291bd4243c9802c17d67b85a6da6bbc0f8f 2020-09-08T15:23:51.608 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=mimic 2020-09-08T15:23:51.773 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default/ 2020-09-08T15:23:51.774 INFO:teuthology.task.install.deb:Package version is 13.2.10-45-g94aa832295-1bionic 2020-09-08T15:23:51.838 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default/ 2020-09-08T15:23:51.839 INFO:teuthology.task.install.deb:Package version is 13.2.10-45-g94aa832295-1bionic 2020-09-08T15:23:51.846 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default/ 2020-09-08T15:23:51.847 INFO:teuthology.task.install.deb:Package version is 13.2.10-45-g94aa832295-1bionic 2020-09-08T15:23:51.938 INFO:teuthology.orchestra.run.smithi104:> set -ex 2020-09-08T15:23:51.940 INFO:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-09-08T15:23:51.947 INFO:teuthology.orchestra.run.smithi104.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-09-08T15:23:51.955 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records in 2020-09-08T15:23:51.956 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records out 2020-09-08T15:23:51.956 INFO:teuthology.orchestra.run.smithi104.stderr:141 bytes copied, 5.6675e-05 s, 2.5 MB/s 2020-09-08T15:23:51.957 INFO:teuthology.orchestra.run.smithi104:> sudo apt-get update 2020-09-08T15:23:51.977 INFO:teuthology.orchestra.run.smithi055:> set -ex 2020-09-08T15:23:51.978 INFO:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-09-08T15:23:51.983 INFO:teuthology.orchestra.run.smithi055.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-09-08T15:23:51.993 INFO:teuthology.orchestra.run.smithi166:> set -ex 2020-09-08T15:23:51.994 INFO:teuthology.orchestra.run.smithi166:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-09-08T15:23:51.995 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records in 2020-09-08T15:23:51.995 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records out 2020-09-08T15:23:51.996 INFO:teuthology.orchestra.run.smithi055.stderr:141 bytes copied, 7.5957e-05 s, 1.9 MB/s 2020-09-08T15:23:51.996 INFO:teuthology.orchestra.run.smithi055:> sudo apt-get update 2020-09-08T15:23:52.001 INFO:teuthology.orchestra.run.smithi166.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-09-08T15:23:52.010 INFO:teuthology.orchestra.run.smithi166.stderr:0+1 records in 2020-09-08T15:23:52.010 INFO:teuthology.orchestra.run.smithi166.stderr:0+1 records out 2020-09-08T15:23:52.010 INFO:teuthology.orchestra.run.smithi166.stderr:141 bytes copied, 7.3587e-05 s, 1.9 MB/s 2020-09-08T15:23:52.011 INFO:teuthology.orchestra.run.smithi166:> sudo apt-get update 2020-09-08T15:23:52.214 INFO:teuthology.orchestra.run.smithi055.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-09-08T15:23:52.229 INFO:teuthology.orchestra.run.smithi166.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-09-08T15:23:52.266 INFO:teuthology.orchestra.run.smithi104.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic InRelease 2020-09-08T15:23:52.267 INFO:teuthology.orchestra.run.smithi055.stdout:Ign:2 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic InRelease 2020-09-08T15:23:52.288 INFO:teuthology.orchestra.run.smithi104.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic Release [7,591 B] 2020-09-08T15:23:52.289 INFO:teuthology.orchestra.run.smithi055.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic Release [7,591 B] 2020-09-08T15:23:52.294 INFO:teuthology.orchestra.run.smithi166.stdout:Ign:2 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic InRelease 2020-09-08T15:23:52.315 INFO:teuthology.orchestra.run.smithi055.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic Release.gpg 2020-09-08T15:23:52.317 INFO:teuthology.orchestra.run.smithi166.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic Release [7,591 B] 2020-09-08T15:23:52.318 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-09-08T15:23:52.319 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:4 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-09-08T15:23:52.320 INFO:teuthology.orchestra.run.smithi104.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic Release.gpg 2020-09-08T15:23:52.330 INFO:teuthology.orchestra.run.smithi166.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-09-08T15:23:52.338 INFO:teuthology.orchestra.run.smithi055.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main i386 Packages [432 B] 2020-09-08T15:23:52.342 INFO:teuthology.orchestra.run.smithi166.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic Release.gpg 2020-09-08T15:23:52.344 INFO:teuthology.orchestra.run.smithi104.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main i386 Packages [432 B] 2020-09-08T15:23:52.366 INFO:teuthology.orchestra.run.smithi166.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 Packages [13.8 kB] 2020-09-08T15:23:52.379 INFO:teuthology.orchestra.run.smithi055.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 Packages [13.8 kB] 2020-09-08T15:23:52.383 INFO:teuthology.orchestra.run.smithi104.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 Packages [13.8 kB] 2020-09-08T15:23:52.395 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-09-08T15:23:52.408 INFO:teuthology.orchestra.run.smithi166.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main i386 Packages [432 B] 2020-09-08T15:23:52.416 INFO:teuthology.orchestra.run.smithi166.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-09-08T15:23:52.485 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-09-08T15:23:52.503 INFO:teuthology.orchestra.run.smithi055.stdout:Hit:7 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-09-08T15:23:52.504 INFO:teuthology.orchestra.run.smithi166.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-09-08T15:23:52.591 INFO:teuthology.orchestra.run.smithi055.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-09-08T15:23:52.679 INFO:teuthology.orchestra.run.smithi055.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-09-08T15:23:52.815 INFO:teuthology.orchestra.run.smithi166.stdout:Fetched 21.8 kB in 1s (36.4 kB/s) 2020-09-08T15:23:52.910 INFO:teuthology.orchestra.run.smithi104.stdout:Fetched 21.8 kB in 1s (29.1 kB/s) 2020-09-08T15:23:52.994 INFO:teuthology.orchestra.run.smithi055.stdout:Fetched 21.8 kB in 1s (27.7 kB/s) 2020-09-08T15:23:54.026 INFO:teuthology.orchestra.run.smithi166.stdout:Reading package lists... 2020-09-08T15:23:54.074 INFO:teuthology.orchestra.run.smithi166:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=13.2.10-45-g94aa832295-1bionic ceph-mds=13.2.10-45-g94aa832295-1bionic ceph-mgr=13.2.10-45-g94aa832295-1bionic ceph-common=13.2.10-45-g94aa832295-1bionic ceph-fuse=13.2.10-45-g94aa832295-1bionic ceph-test=13.2.10-45-g94aa832295-1bionic radosgw=13.2.10-45-g94aa832295-1bionic libcephfs2=13.2.10-45-g94aa832295-1bionic libcephfs-dev=13.2.10-45-g94aa832295-1bionic librados2=13.2.10-45-g94aa832295-1bionic librbd1=13.2.10-45-g94aa832295-1bionic rbd-fuse=13.2.10-45-g94aa832295-1bionic librados2=13.2.10-45-g94aa832295-1bionic python-rados=13.2.10-45-g94aa832295-1bionic python-rgw=13.2.10-45-g94aa832295-1bionic python-rbd=13.2.10-45-g94aa832295-1bionic python-cephfs=13.2.10-45-g94aa832295-1bionic 2020-09-08T15:23:54.109 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2020-09-08T15:23:54.148 INFO:teuthology.orchestra.run.smithi166.stdout:Reading package lists... 2020-09-08T15:23:54.156 INFO:teuthology.orchestra.run.smithi104:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=13.2.10-45-g94aa832295-1bionic ceph-mds=13.2.10-45-g94aa832295-1bionic ceph-mgr=13.2.10-45-g94aa832295-1bionic ceph-common=13.2.10-45-g94aa832295-1bionic ceph-fuse=13.2.10-45-g94aa832295-1bionic ceph-test=13.2.10-45-g94aa832295-1bionic radosgw=13.2.10-45-g94aa832295-1bionic libcephfs2=13.2.10-45-g94aa832295-1bionic libcephfs-dev=13.2.10-45-g94aa832295-1bionic librados2=13.2.10-45-g94aa832295-1bionic librbd1=13.2.10-45-g94aa832295-1bionic rbd-fuse=13.2.10-45-g94aa832295-1bionic librados2=13.2.10-45-g94aa832295-1bionic python-rados=13.2.10-45-g94aa832295-1bionic python-rgw=13.2.10-45-g94aa832295-1bionic python-rbd=13.2.10-45-g94aa832295-1bionic python-cephfs=13.2.10-45-g94aa832295-1bionic 2020-09-08T15:23:54.236 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2020-09-08T15:23:54.246 INFO:teuthology.orchestra.run.smithi055.stdout:Reading package lists... 2020-09-08T15:23:54.296 INFO:teuthology.orchestra.run.smithi055:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=13.2.10-45-g94aa832295-1bionic ceph-mds=13.2.10-45-g94aa832295-1bionic ceph-mgr=13.2.10-45-g94aa832295-1bionic ceph-common=13.2.10-45-g94aa832295-1bionic ceph-fuse=13.2.10-45-g94aa832295-1bionic ceph-test=13.2.10-45-g94aa832295-1bionic radosgw=13.2.10-45-g94aa832295-1bionic libcephfs2=13.2.10-45-g94aa832295-1bionic libcephfs-dev=13.2.10-45-g94aa832295-1bionic librados2=13.2.10-45-g94aa832295-1bionic librbd1=13.2.10-45-g94aa832295-1bionic rbd-fuse=13.2.10-45-g94aa832295-1bionic librados2=13.2.10-45-g94aa832295-1bionic python-rados=13.2.10-45-g94aa832295-1bionic python-rgw=13.2.10-45-g94aa832295-1bionic python-rbd=13.2.10-45-g94aa832295-1bionic python-cephfs=13.2.10-45-g94aa832295-1bionic 2020-09-08T15:23:54.377 INFO:teuthology.orchestra.run.smithi166.stdout:Building dependency tree... 2020-09-08T15:23:54.377 INFO:teuthology.orchestra.run.smithi055.stdout:Reading package lists... 2020-09-08T15:23:54.378 INFO:teuthology.orchestra.run.smithi166.stdout:Reading state information... 2020-09-08T15:23:54.487 INFO:teuthology.orchestra.run.smithi104.stdout:Building dependency tree... 2020-09-08T15:23:54.487 INFO:teuthology.orchestra.run.smithi104.stdout:Reading state information... 2020-09-08T15:23:54.542 INFO:teuthology.orchestra.run.smithi166.stdout:The following additional packages will be installed: 2020-09-08T15:23:54.542 INFO:teuthology.orchestra.run.smithi166.stdout: ceph-base ceph-mon ceph-osd formencode-i18n jq libjq1 libjs-sphinxdoc 2020-09-08T15:23:54.542 INFO:teuthology.orchestra.run.smithi166.stdout: libjs-underscore libleveldb1v5 liblttng-ust0 liboath0 libonig4 2020-09-08T15:23:54.543 INFO:teuthology.orchestra.run.smithi166.stdout: libradosstriper1 librgw2 libsnappy1v5 libxslt1.1 python-bcrypt python-bs4 2020-09-08T15:23:54.543 INFO:teuthology.orchestra.run.smithi166.stdout: python-ceph-argparse python-certifi python-chardet python-cherrypy3 2020-09-08T15:23:54.544 INFO:teuthology.orchestra.run.smithi166.stdout: python-dnspython python-formencode python-html5lib python-jinja2 2020-09-08T15:23:54.544 INFO:teuthology.orchestra.run.smithi166.stdout: python-logutils python-lxml python-mako python-markupsafe python-openid 2020-09-08T15:23:54.544 INFO:teuthology.orchestra.run.smithi166.stdout: python-openssl python-paste python-pastedeploy python-pastedeploy-tpl 2020-09-08T15:23:54.544 INFO:teuthology.orchestra.run.smithi166.stdout: python-pastescript python-pecan python-prettytable python-pyinotify 2020-09-08T15:23:54.545 INFO:teuthology.orchestra.run.smithi166.stdout: python-repoze.lru python-requests python-routes python-scgi 2020-09-08T15:23:54.545 INFO:teuthology.orchestra.run.smithi166.stdout: python-simplegeneric python-simplejson python-singledispatch python-tempita 2020-09-08T15:23:54.545 INFO:teuthology.orchestra.run.smithi166.stdout: python-urllib3 python-waitress python-webencodings python-webob 2020-09-08T15:23:54.545 INFO:teuthology.orchestra.run.smithi166.stdout: python-webtest python-werkzeug socat xmlstarlet 2020-09-08T15:23:54.546 INFO:teuthology.orchestra.run.smithi166.stdout:Suggested packages: 2020-09-08T15:23:54.546 INFO:teuthology.orchestra.run.smithi166.stdout: python-influxdb python-egenix-mxdatetime python-genshi python-jinja2-doc 2020-09-08T15:23:54.546 INFO:teuthology.orchestra.run.smithi166.stdout: python-lxml-dbg python-lxml-doc python-beaker python-mako-doc 2020-09-08T15:23:54.547 INFO:teuthology.orchestra.run.smithi166.stdout: python-openssl-doc python-openssl-dbg httpd-wsgi libapache2-mod-python 2020-09-08T15:23:54.547 INFO:teuthology.orchestra.run.smithi166.stdout: libapache2-mod-scgi libjs-mochikit python-pastewebkit python-cheetah 2020-09-08T15:23:54.547 INFO:teuthology.orchestra.run.smithi166.stdout: python-cherrypy python-flup python-pecan-doc python-pyinotify-doc 2020-09-08T15:23:54.547 INFO:teuthology.orchestra.run.smithi166.stdout: python-socks python-ntlm python-waitress-doc python-webob-doc 2020-09-08T15:23:54.548 INFO:teuthology.orchestra.run.smithi166.stdout: python-webtest-doc python-pyquery ipython python-redis python-pylibmc 2020-09-08T15:23:54.548 INFO:teuthology.orchestra.run.smithi166.stdout: | python-memcache python-termcolor python-watchdog python-werkzeug-doc 2020-09-08T15:23:54.627 INFO:teuthology.orchestra.run.smithi166.stdout:The following NEW packages will be installed: 2020-09-08T15:23:54.627 INFO:teuthology.orchestra.run.smithi166.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mon ceph-osd 2020-09-08T15:23:54.628 INFO:teuthology.orchestra.run.smithi166.stdout: ceph-test formencode-i18n jq libcephfs-dev libcephfs2 libjq1 libjs-sphinxdoc 2020-09-08T15:23:54.628 INFO:teuthology.orchestra.run.smithi166.stdout: libjs-underscore libleveldb1v5 liblttng-ust0 liboath0 libonig4 2020-09-08T15:23:54.628 INFO:teuthology.orchestra.run.smithi166.stdout: libradosstriper1 librgw2 libsnappy1v5 libxslt1.1 python-bcrypt python-bs4 2020-09-08T15:23:54.629 INFO:teuthology.orchestra.run.smithi166.stdout: python-ceph-argparse python-cephfs python-certifi python-chardet 2020-09-08T15:23:54.632 INFO:teuthology.orchestra.run.smithi166.stdout: python-cherrypy3 python-dnspython python-formencode python-html5lib 2020-09-08T15:23:54.632 INFO:teuthology.orchestra.run.smithi166.stdout: python-jinja2 python-logutils python-lxml python-mako python-markupsafe 2020-09-08T15:23:54.633 INFO:teuthology.orchestra.run.smithi166.stdout: python-openid python-openssl python-paste python-pastedeploy 2020-09-08T15:23:54.633 INFO:teuthology.orchestra.run.smithi166.stdout: python-pastedeploy-tpl python-pastescript python-pecan python-prettytable 2020-09-08T15:23:54.633 INFO:teuthology.orchestra.run.smithi166.stdout: python-pyinotify python-rados python-rbd python-repoze.lru python-requests 2020-09-08T15:23:54.633 INFO:teuthology.orchestra.run.smithi166.stdout: python-rgw python-routes python-scgi python-simplegeneric python-simplejson 2020-09-08T15:23:54.634 INFO:teuthology.orchestra.run.smithi166.stdout: python-singledispatch python-tempita python-urllib3 python-waitress 2020-09-08T15:23:54.634 INFO:teuthology.orchestra.run.smithi166.stdout: python-webencodings python-webob python-webtest python-werkzeug radosgw 2020-09-08T15:23:54.634 INFO:teuthology.orchestra.run.smithi166.stdout: rbd-fuse socat xmlstarlet 2020-09-08T15:23:54.634 INFO:teuthology.orchestra.run.smithi166.stdout:The following packages will be upgraded: 2020-09-08T15:23:54.635 INFO:teuthology.orchestra.run.smithi166.stdout: librados2 librbd1 2020-09-08T15:23:54.635 INFO:teuthology.orchestra.run.smithi055.stdout:Building dependency tree... 2020-09-08T15:23:54.636 INFO:teuthology.orchestra.run.smithi055.stdout:Reading state information... 2020-09-08T15:23:54.648 INFO:teuthology.orchestra.run.smithi104.stdout:The following additional packages will be installed: 2020-09-08T15:23:54.652 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base ceph-mon ceph-osd formencode-i18n jq libjq1 libjs-sphinxdoc 2020-09-08T15:23:54.652 INFO:teuthology.orchestra.run.smithi104.stdout: libjs-underscore libleveldb1v5 liblttng-ust0 liboath0 libonig4 2020-09-08T15:23:54.652 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1 librgw2 libsnappy1v5 libxslt1.1 python-bcrypt python-bs4 2020-09-08T15:23:54.653 INFO:teuthology.orchestra.run.smithi104.stdout: python-ceph-argparse python-certifi python-chardet python-cherrypy3 2020-09-08T15:23:54.653 INFO:teuthology.orchestra.run.smithi104.stdout: python-dnspython python-formencode python-html5lib python-jinja2 2020-09-08T15:23:54.653 INFO:teuthology.orchestra.run.smithi104.stdout: python-logutils python-lxml python-mako python-markupsafe python-openid 2020-09-08T15:23:54.653 INFO:teuthology.orchestra.run.smithi104.stdout: python-openssl python-paste python-pastedeploy python-pastedeploy-tpl 2020-09-08T15:23:54.654 INFO:teuthology.orchestra.run.smithi104.stdout: python-pastescript python-pecan python-prettytable python-pyinotify 2020-09-08T15:23:54.654 INFO:teuthology.orchestra.run.smithi104.stdout: python-repoze.lru python-requests python-routes python-scgi 2020-09-08T15:23:54.654 INFO:teuthology.orchestra.run.smithi104.stdout: python-simplegeneric python-simplejson python-singledispatch python-tempita 2020-09-08T15:23:54.654 INFO:teuthology.orchestra.run.smithi104.stdout: python-urllib3 python-waitress python-webencodings python-webob 2020-09-08T15:23:54.654 INFO:teuthology.orchestra.run.smithi104.stdout: python-webtest python-werkzeug socat xmlstarlet 2020-09-08T15:23:54.655 INFO:teuthology.orchestra.run.smithi104.stdout:Suggested packages: 2020-09-08T15:23:54.655 INFO:teuthology.orchestra.run.smithi104.stdout: python-influxdb python-egenix-mxdatetime python-genshi python-jinja2-doc 2020-09-08T15:23:54.655 INFO:teuthology.orchestra.run.smithi104.stdout: python-lxml-dbg python-lxml-doc python-beaker python-mako-doc 2020-09-08T15:23:54.655 INFO:teuthology.orchestra.run.smithi104.stdout: python-openssl-doc python-openssl-dbg httpd-wsgi libapache2-mod-python 2020-09-08T15:23:54.655 INFO:teuthology.orchestra.run.smithi104.stdout: libapache2-mod-scgi libjs-mochikit python-pastewebkit python-cheetah 2020-09-08T15:23:54.656 INFO:teuthology.orchestra.run.smithi104.stdout: python-cherrypy python-flup python-pecan-doc python-pyinotify-doc 2020-09-08T15:23:54.656 INFO:teuthology.orchestra.run.smithi104.stdout: python-socks python-ntlm python-waitress-doc python-webob-doc 2020-09-08T15:23:54.656 INFO:teuthology.orchestra.run.smithi104.stdout: python-webtest-doc python-pyquery ipython python-redis python-pylibmc 2020-09-08T15:23:54.656 INFO:teuthology.orchestra.run.smithi104.stdout: | python-memcache python-termcolor python-watchdog python-werkzeug-doc 2020-09-08T15:23:54.723 INFO:teuthology.orchestra.run.smithi104.stdout:The following NEW packages will be installed: 2020-09-08T15:23:54.724 INFO:teuthology.orchestra.run.smithi104.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mon ceph-osd 2020-09-08T15:23:54.725 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test formencode-i18n jq libcephfs-dev libcephfs2 libjq1 libjs-sphinxdoc 2020-09-08T15:23:54.725 INFO:teuthology.orchestra.run.smithi104.stdout: libjs-underscore libleveldb1v5 liblttng-ust0 liboath0 libonig4 2020-09-08T15:23:54.725 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1 librgw2 libsnappy1v5 libxslt1.1 python-bcrypt python-bs4 2020-09-08T15:23:54.726 INFO:teuthology.orchestra.run.smithi104.stdout: python-ceph-argparse python-cephfs python-certifi python-chardet 2020-09-08T15:23:54.726 INFO:teuthology.orchestra.run.smithi104.stdout: python-cherrypy3 python-dnspython python-formencode python-html5lib 2020-09-08T15:23:54.726 INFO:teuthology.orchestra.run.smithi104.stdout: python-jinja2 python-logutils python-lxml python-mako python-markupsafe 2020-09-08T15:23:54.726 INFO:teuthology.orchestra.run.smithi104.stdout: python-openid python-openssl python-paste python-pastedeploy 2020-09-08T15:23:54.726 INFO:teuthology.orchestra.run.smithi104.stdout: python-pastedeploy-tpl python-pastescript python-pecan python-prettytable 2020-09-08T15:23:54.727 INFO:teuthology.orchestra.run.smithi104.stdout: python-pyinotify python-rados python-rbd python-repoze.lru python-requests 2020-09-08T15:23:54.727 INFO:teuthology.orchestra.run.smithi104.stdout: python-rgw python-routes python-scgi python-simplegeneric python-simplejson 2020-09-08T15:23:54.727 INFO:teuthology.orchestra.run.smithi104.stdout: python-singledispatch python-tempita python-urllib3 python-waitress 2020-09-08T15:23:54.727 INFO:teuthology.orchestra.run.smithi104.stdout: python-webencodings python-webob python-webtest python-werkzeug radosgw 2020-09-08T15:23:54.727 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse socat xmlstarlet 2020-09-08T15:23:54.728 INFO:teuthology.orchestra.run.smithi104.stdout:The following packages will be upgraded: 2020-09-08T15:23:54.729 INFO:teuthology.orchestra.run.smithi104.stdout: librados2 librbd1 2020-09-08T15:23:54.783 INFO:teuthology.orchestra.run.smithi166.stdout:2 upgraded, 69 newly installed, 0 to remove and 9 not upgraded. 2020-09-08T15:23:54.784 INFO:teuthology.orchestra.run.smithi166.stdout:Need to get 74.2 MB of archives. 2020-09-08T15:23:54.784 INFO:teuthology.orchestra.run.smithi166.stdout:After this operation, 299 MB of additional disk space will be used. 2020-09-08T15:23:54.785 INFO:teuthology.orchestra.run.smithi166.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 13.2.10-45-g94aa832295-1bionic [1,086 kB] 2020-09-08T15:23:54.824 INFO:teuthology.orchestra.run.smithi055.stdout:The following additional packages will be installed: 2020-09-08T15:23:54.825 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-base ceph-mon ceph-osd formencode-i18n jq libjq1 libjs-sphinxdoc 2020-09-08T15:23:54.825 INFO:teuthology.orchestra.run.smithi055.stdout: libjs-underscore libleveldb1v5 liblttng-ust0 liboath0 libonig4 2020-09-08T15:23:54.825 INFO:teuthology.orchestra.run.smithi055.stdout: libradosstriper1 librgw2 libsnappy1v5 libxslt1.1 python-bcrypt python-bs4 2020-09-08T15:23:54.825 INFO:teuthology.orchestra.run.smithi055.stdout: python-ceph-argparse python-certifi python-chardet python-cherrypy3 2020-09-08T15:23:54.826 INFO:teuthology.orchestra.run.smithi055.stdout: python-dnspython python-formencode python-html5lib python-jinja2 2020-09-08T15:23:54.826 INFO:teuthology.orchestra.run.smithi055.stdout: python-logutils python-lxml python-mako python-markupsafe python-openid 2020-09-08T15:23:54.826 INFO:teuthology.orchestra.run.smithi055.stdout: python-openssl python-paste python-pastedeploy python-pastedeploy-tpl 2020-09-08T15:23:54.826 INFO:teuthology.orchestra.run.smithi055.stdout: python-pastescript python-pecan python-prettytable python-pyinotify 2020-09-08T15:23:54.826 INFO:teuthology.orchestra.run.smithi055.stdout: python-repoze.lru python-requests python-routes python-scgi 2020-09-08T15:23:54.827 INFO:teuthology.orchestra.run.smithi055.stdout: python-simplegeneric python-simplejson python-singledispatch python-tempita 2020-09-08T15:23:54.827 INFO:teuthology.orchestra.run.smithi055.stdout: python-urllib3 python-waitress python-webencodings python-webob 2020-09-08T15:23:54.827 INFO:teuthology.orchestra.run.smithi055.stdout: python-webtest python-werkzeug socat xmlstarlet 2020-09-08T15:23:54.827 INFO:teuthology.orchestra.run.smithi055.stdout:Suggested packages: 2020-09-08T15:23:54.828 INFO:teuthology.orchestra.run.smithi055.stdout: python-influxdb python-egenix-mxdatetime python-genshi python-jinja2-doc 2020-09-08T15:23:54.828 INFO:teuthology.orchestra.run.smithi055.stdout: python-lxml-dbg python-lxml-doc python-beaker python-mako-doc 2020-09-08T15:23:54.828 INFO:teuthology.orchestra.run.smithi055.stdout: python-openssl-doc python-openssl-dbg httpd-wsgi libapache2-mod-python 2020-09-08T15:23:54.828 INFO:teuthology.orchestra.run.smithi055.stdout: libapache2-mod-scgi libjs-mochikit python-pastewebkit python-cheetah 2020-09-08T15:23:54.828 INFO:teuthology.orchestra.run.smithi055.stdout: python-cherrypy python-flup python-pecan-doc python-pyinotify-doc 2020-09-08T15:23:54.829 INFO:teuthology.orchestra.run.smithi055.stdout: python-socks python-ntlm python-waitress-doc python-webob-doc 2020-09-08T15:23:54.829 INFO:teuthology.orchestra.run.smithi055.stdout: python-webtest-doc python-pyquery ipython python-redis python-pylibmc 2020-09-08T15:23:54.829 INFO:teuthology.orchestra.run.smithi055.stdout: | python-memcache python-termcolor python-watchdog python-werkzeug-doc 2020-09-08T15:23:54.830 INFO:teuthology.orchestra.run.smithi166.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-09-08T15:23:54.888 INFO:teuthology.orchestra.run.smithi104.stdout:2 upgraded, 69 newly installed, 0 to remove and 9 not upgraded. 2020-09-08T15:23:54.889 INFO:teuthology.orchestra.run.smithi104.stdout:Need to get 74.2 MB of archives. 2020-09-08T15:23:54.889 INFO:teuthology.orchestra.run.smithi104.stdout:After this operation, 299 MB of additional disk space will be used. 2020-09-08T15:23:54.889 INFO:teuthology.orchestra.run.smithi104.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 13.2.10-45-g94aa832295-1bionic [1,086 kB] 2020-09-08T15:23:54.895 INFO:teuthology.orchestra.run.smithi055.stdout:The following NEW packages will be installed: 2020-09-08T15:23:54.895 INFO:teuthology.orchestra.run.smithi055.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mon ceph-osd 2020-09-08T15:23:54.897 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-test formencode-i18n jq libcephfs-dev libcephfs2 libjq1 libjs-sphinxdoc 2020-09-08T15:23:54.897 INFO:teuthology.orchestra.run.smithi055.stdout: libjs-underscore libleveldb1v5 liblttng-ust0 liboath0 libonig4 2020-09-08T15:23:54.897 INFO:teuthology.orchestra.run.smithi055.stdout: libradosstriper1 librgw2 libsnappy1v5 libxslt1.1 python-bcrypt python-bs4 2020-09-08T15:23:54.898 INFO:teuthology.orchestra.run.smithi055.stdout: python-ceph-argparse python-cephfs python-certifi python-chardet 2020-09-08T15:23:54.898 INFO:teuthology.orchestra.run.smithi055.stdout: python-cherrypy3 python-dnspython python-formencode python-html5lib 2020-09-08T15:23:54.898 INFO:teuthology.orchestra.run.smithi055.stdout: python-jinja2 python-logutils python-lxml python-mako python-markupsafe 2020-09-08T15:23:54.898 INFO:teuthology.orchestra.run.smithi055.stdout: python-openid python-openssl python-paste python-pastedeploy 2020-09-08T15:23:54.898 INFO:teuthology.orchestra.run.smithi055.stdout: python-pastedeploy-tpl python-pastescript python-pecan python-prettytable 2020-09-08T15:23:54.899 INFO:teuthology.orchestra.run.smithi055.stdout: python-pyinotify python-rados python-rbd python-repoze.lru python-requests 2020-09-08T15:23:54.899 INFO:teuthology.orchestra.run.smithi055.stdout: python-rgw python-routes python-scgi python-simplegeneric python-simplejson 2020-09-08T15:23:54.899 INFO:teuthology.orchestra.run.smithi055.stdout: python-singledispatch python-tempita python-urllib3 python-waitress 2020-09-08T15:23:54.899 INFO:teuthology.orchestra.run.smithi055.stdout: python-webencodings python-webob python-webtest python-werkzeug radosgw 2020-09-08T15:23:54.900 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-fuse socat xmlstarlet 2020-09-08T15:23:54.900 INFO:teuthology.orchestra.run.smithi055.stdout:The following packages will be upgraded: 2020-09-08T15:23:54.900 INFO:teuthology.orchestra.run.smithi055.stdout: librados2 librbd1 2020-09-08T15:23:54.929 INFO:teuthology.orchestra.run.smithi104.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-09-08T15:23:54.989 INFO:teuthology.orchestra.run.smithi166.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 13.2.10-45-g94aa832295-1bionic [2,601 kB] 2020-09-08T15:23:55.059 INFO:teuthology.orchestra.run.smithi055.stdout:2 upgraded, 69 newly installed, 0 to remove and 9 not upgraded. 2020-09-08T15:23:55.059 INFO:teuthology.orchestra.run.smithi055.stdout:Need to get 74.2 MB of archives. 2020-09-08T15:23:55.059 INFO:teuthology.orchestra.run.smithi055.stdout:After this operation, 299 MB of additional disk space will be used. 2020-09-08T15:23:55.060 INFO:teuthology.orchestra.run.smithi055.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 13.2.10-45-g94aa832295-1bionic [1,086 kB] 2020-09-08T15:23:55.076 INFO:teuthology.orchestra.run.smithi104.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 13.2.10-45-g94aa832295-1bionic [2,601 kB] 2020-09-08T15:23:55.105 INFO:teuthology.orchestra.run.smithi055.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-09-08T15:23:55.207 INFO:teuthology.orchestra.run.smithi166.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 13.2.10-45-g94aa832295-1bionic [386 kB] 2020-09-08T15:23:55.225 INFO:teuthology.orchestra.run.smithi166.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-prettytable all 0.7.2-3 [19.6 kB] 2020-09-08T15:23:55.226 INFO:teuthology.orchestra.run.smithi166.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-certifi all 2018.1.18-2 [144 kB] 2020-09-08T15:23:55.236 INFO:teuthology.orchestra.run.smithi166.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-chardet all 3.0.4-1 [80.3 kB] 2020-09-08T15:23:55.237 INFO:teuthology.orchestra.run.smithi166.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 python-rados amd64 13.2.10-45-g94aa832295-1bionic [288 kB] 2020-09-08T15:23:55.249 INFO:teuthology.orchestra.run.smithi055.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 13.2.10-45-g94aa832295-1bionic [2,601 kB] 2020-09-08T15:23:55.261 INFO:teuthology.orchestra.run.smithi166.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 python-ceph-argparse amd64 13.2.10-45-g94aa832295-1bionic [19.0 kB] 2020-09-08T15:23:55.261 INFO:teuthology.orchestra.run.smithi166.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 python-cephfs amd64 13.2.10-45-g94aa832295-1bionic [93.9 kB] 2020-09-08T15:23:55.264 INFO:teuthology.orchestra.run.smithi166.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 python-rbd amd64 13.2.10-45-g94aa832295-1bionic [179 kB] 2020-09-08T15:23:55.288 INFO:teuthology.orchestra.run.smithi166.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 13.2.10-45-g94aa832295-1bionic [1,883 kB] 2020-09-08T15:23:55.304 INFO:teuthology.orchestra.run.smithi104.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 13.2.10-45-g94aa832295-1bionic [386 kB] 2020-09-08T15:23:55.330 INFO:teuthology.orchestra.run.smithi166.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python-urllib3 all 1.22-1ubuntu0.18.04.1 [85.9 kB] 2020-09-08T15:23:55.334 INFO:teuthology.orchestra.run.smithi104.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-prettytable all 0.7.2-3 [19.6 kB] 2020-09-08T15:23:55.336 INFO:teuthology.orchestra.run.smithi104.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 python-rados amd64 13.2.10-45-g94aa832295-1bionic [288 kB] 2020-09-08T15:23:55.344 INFO:teuthology.orchestra.run.smithi104.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-certifi all 2018.1.18-2 [144 kB] 2020-09-08T15:23:55.347 INFO:teuthology.orchestra.run.smithi166.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python-requests all 2.18.4-2ubuntu0.1 [58.5 kB] 2020-09-08T15:23:55.358 INFO:teuthology.orchestra.run.smithi166.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-09-08T15:23:55.368 INFO:teuthology.orchestra.run.smithi104.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 python-ceph-argparse amd64 13.2.10-45-g94aa832295-1bionic [19.0 kB] 2020-09-08T15:23:55.369 INFO:teuthology.orchestra.run.smithi104.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 python-cephfs amd64 13.2.10-45-g94aa832295-1bionic [93.9 kB] 2020-09-08T15:23:55.394 INFO:teuthology.orchestra.run.smithi104.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 python-rbd amd64 13.2.10-45-g94aa832295-1bionic [179 kB] 2020-09-08T15:23:55.401 INFO:teuthology.orchestra.run.smithi104.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 13.2.10-45-g94aa832295-1bionic [1,883 kB] 2020-09-08T15:23:55.407 INFO:teuthology.orchestra.run.smithi166.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-09-08T15:23:55.409 INFO:teuthology.orchestra.run.smithi166.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-09-08T15:23:55.423 INFO:teuthology.orchestra.run.smithi104.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-chardet all 3.0.4-1 [80.3 kB] 2020-09-08T15:23:55.430 INFO:teuthology.orchestra.run.smithi166.stdout:Get:18 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 python-rgw amd64 13.2.10-45-g94aa832295-1bionic [98.9 kB] 2020-09-08T15:23:55.441 INFO:teuthology.orchestra.run.smithi166.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-cherrypy3 all 8.9.1-2 [160 kB] 2020-09-08T15:23:55.446 INFO:teuthology.orchestra.run.smithi166.stdout:Get:20 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 13.2.10-45-g94aa832295-1bionic [306 kB] 2020-09-08T15:23:55.459 INFO:teuthology.orchestra.run.smithi104.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python-urllib3 all 1.22-1ubuntu0.18.04.1 [85.9 kB] 2020-09-08T15:23:55.463 INFO:teuthology.orchestra.run.smithi166.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-markupsafe amd64 1.0-1build1 [13.0 kB] 2020-09-08T15:23:55.466 INFO:teuthology.orchestra.run.smithi166.stdout:Get:22 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 13.2.10-45-g94aa832295-1bionic [12.0 MB] 2020-09-08T15:23:55.466 INFO:teuthology.orchestra.run.smithi166.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python-jinja2 all 2.10-1ubuntu0.18.04.1 [94.8 kB] 2020-09-08T15:23:55.478 INFO:teuthology.orchestra.run.smithi104.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python-requests all 2.18.4-2ubuntu0.1 [58.5 kB] 2020-09-08T15:23:55.483 INFO:teuthology.orchestra.run.smithi166.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-openssl all 17.5.0-1ubuntu1 [41.3 kB] 2020-09-08T15:23:55.490 INFO:teuthology.orchestra.run.smithi166.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-logutils all 0.3.3-5 [16.7 kB] 2020-09-08T15:23:55.492 INFO:teuthology.orchestra.run.smithi166.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-mako all 1.0.7+ds1-1 [60.1 kB] 2020-09-08T15:23:55.494 INFO:teuthology.orchestra.run.smithi104.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-09-08T15:23:55.498 INFO:teuthology.orchestra.run.smithi055.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-prettytable all 0.7.2-3 [19.6 kB] 2020-09-08T15:23:55.500 INFO:teuthology.orchestra.run.smithi166.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-simplegeneric all 0.8.1-1 [11.5 kB] 2020-09-08T15:23:55.506 INFO:teuthology.orchestra.run.smithi055.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-certifi all 2018.1.18-2 [144 kB] 2020-09-08T15:23:55.513 INFO:teuthology.orchestra.run.smithi055.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-chardet all 3.0.4-1 [80.3 kB] 2020-09-08T15:23:55.514 INFO:teuthology.orchestra.run.smithi104.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-09-08T15:23:55.515 INFO:teuthology.orchestra.run.smithi104.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-09-08T15:23:55.550 INFO:teuthology.orchestra.run.smithi104.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-cherrypy3 all 8.9.1-2 [160 kB] 2020-09-08T15:23:55.566 INFO:teuthology.orchestra.run.smithi104.stdout:Get:19 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 python-rgw amd64 13.2.10-45-g94aa832295-1bionic [98.9 kB] 2020-09-08T15:23:55.576 INFO:teuthology.orchestra.run.smithi104.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-markupsafe amd64 1.0-1build1 [13.0 kB] 2020-09-08T15:23:55.578 INFO:teuthology.orchestra.run.smithi166.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-singledispatch all 3.4.0.3-2 [9,848 B] 2020-09-08T15:23:55.579 INFO:teuthology.orchestra.run.smithi166.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-simplejson amd64 3.13.2-1 [61.2 kB] 2020-09-08T15:23:55.579 INFO:teuthology.orchestra.run.smithi104.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python-jinja2 all 2.10-1ubuntu0.18.04.1 [94.8 kB] 2020-09-08T15:23:55.580 INFO:teuthology.orchestra.run.smithi055.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 13.2.10-45-g94aa832295-1bionic [386 kB] 2020-09-08T15:23:55.582 INFO:teuthology.orchestra.run.smithi166.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-09-08T15:23:55.586 INFO:teuthology.orchestra.run.smithi166.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-bs4 all 4.6.0-1 [67.9 kB] 2020-09-08T15:23:55.590 INFO:teuthology.orchestra.run.smithi166.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-waitress all 1.0.1-1 [53.3 kB] 2020-09-08T15:23:55.593 INFO:teuthology.orchestra.run.smithi166.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-dnspython all 1.15.0-1 [84.9 kB] 2020-09-08T15:23:55.598 INFO:teuthology.orchestra.run.smithi104.stdout:Get:22 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 13.2.10-45-g94aa832295-1bionic [306 kB] 2020-09-08T15:23:55.601 INFO:teuthology.orchestra.run.smithi055.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python-urllib3 all 1.22-1ubuntu0.18.04.1 [85.9 kB] 2020-09-08T15:23:55.602 INFO:teuthology.orchestra.run.smithi104.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-openssl all 17.5.0-1ubuntu1 [41.3 kB] 2020-09-08T15:23:55.603 INFO:teuthology.orchestra.run.smithi055.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python-requests all 2.18.4-2ubuntu0.1 [58.5 kB] 2020-09-08T15:23:55.606 INFO:teuthology.orchestra.run.smithi104.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-logutils all 0.3.3-5 [16.7 kB] 2020-09-08T15:23:55.608 INFO:teuthology.orchestra.run.smithi104.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-mako all 1.0.7+ds1-1 [60.1 kB] 2020-09-08T15:23:55.609 INFO:teuthology.orchestra.run.smithi104.stdout:Get:26 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 13.2.10-45-g94aa832295-1bionic [12.0 MB] 2020-09-08T15:23:55.615 INFO:teuthology.orchestra.run.smithi104.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-simplegeneric all 0.8.1-1 [11.5 kB] 2020-09-08T15:23:55.615 INFO:teuthology.orchestra.run.smithi055.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-09-08T15:23:55.624 INFO:teuthology.orchestra.run.smithi055.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-09-08T15:23:55.627 INFO:teuthology.orchestra.run.smithi055.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 python-rados amd64 13.2.10-45-g94aa832295-1bionic [288 kB] 2020-09-08T15:23:55.628 INFO:teuthology.orchestra.run.smithi055.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-09-08T15:23:55.666 INFO:teuthology.orchestra.run.smithi166.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 formencode-i18n all 1.3.0-0ubuntu5 [3,382 B] 2020-09-08T15:23:55.667 INFO:teuthology.orchestra.run.smithi166.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-formencode all 1.3.0-0ubuntu5 [69.1 kB] 2020-09-08T15:23:55.670 INFO:teuthology.orchestra.run.smithi166.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-tempita all 0.5.2-2 [13.9 kB] 2020-09-08T15:23:55.671 INFO:teuthology.orchestra.run.smithi166.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-09-08T15:23:55.672 INFO:teuthology.orchestra.run.smithi055.stdout:Get:14 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 python-ceph-argparse amd64 13.2.10-45-g94aa832295-1bionic [19.0 kB] 2020-09-08T15:23:55.673 INFO:teuthology.orchestra.run.smithi055.stdout:Get:15 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 python-cephfs amd64 13.2.10-45-g94aa832295-1bionic [93.9 kB] 2020-09-08T15:23:55.690 INFO:teuthology.orchestra.run.smithi104.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-singledispatch all 3.4.0.3-2 [9,848 B] 2020-09-08T15:23:55.691 INFO:teuthology.orchestra.run.smithi104.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-simplejson amd64 3.13.2-1 [61.2 kB] 2020-09-08T15:23:55.694 INFO:teuthology.orchestra.run.smithi104.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-09-08T15:23:55.695 INFO:teuthology.orchestra.run.smithi055.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-cherrypy3 all 8.9.1-2 [160 kB] 2020-09-08T15:23:55.699 INFO:teuthology.orchestra.run.smithi104.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-bs4 all 4.6.0-1 [67.9 kB] 2020-09-08T15:23:55.703 INFO:teuthology.orchestra.run.smithi104.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-waitress all 1.0.1-1 [53.3 kB] 2020-09-08T15:23:55.706 INFO:teuthology.orchestra.run.smithi104.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-dnspython all 1.15.0-1 [84.9 kB] 2020-09-08T15:23:55.713 INFO:teuthology.orchestra.run.smithi055.stdout:Get:17 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 python-rbd amd64 13.2.10-45-g94aa832295-1bionic [179 kB] 2020-09-08T15:23:55.720 INFO:teuthology.orchestra.run.smithi055.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-markupsafe amd64 1.0-1build1 [13.0 kB] 2020-09-08T15:23:55.720 INFO:teuthology.orchestra.run.smithi055.stdout:Get:19 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 13.2.10-45-g94aa832295-1bionic [1,883 kB] 2020-09-08T15:23:55.721 INFO:teuthology.orchestra.run.smithi055.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python-jinja2 all 2.10-1ubuntu0.18.04.1 [94.8 kB] 2020-09-08T15:23:55.733 INFO:teuthology.orchestra.run.smithi055.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-openssl all 17.5.0-1ubuntu1 [41.3 kB] 2020-09-08T15:23:55.754 INFO:teuthology.orchestra.run.smithi166.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-09-08T15:23:55.755 INFO:teuthology.orchestra.run.smithi166.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy all 1.5.2-4 [25.6 kB] 2020-09-08T15:23:55.756 INFO:teuthology.orchestra.run.smithi166.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-webtest all 2.0.28-1ubuntu1 [27.8 kB] 2020-09-08T15:23:55.757 INFO:teuthology.orchestra.run.smithi166.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pecan all 1.2.1-2 [86.1 kB] 2020-09-08T15:23:55.762 INFO:teuthology.orchestra.run.smithi166.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-bcrypt amd64 3.1.4-2 [29.8 kB] 2020-09-08T15:23:55.763 INFO:teuthology.orchestra.run.smithi055.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-logutils all 0.3.3-5 [16.7 kB] 2020-09-08T15:23:55.764 INFO:teuthology.orchestra.run.smithi166.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-werkzeug all 0.14.1+dfsg1-1 [174 kB] 2020-09-08T15:23:55.765 INFO:teuthology.orchestra.run.smithi055.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-mako all 1.0.7+ds1-1 [60.1 kB] 2020-09-08T15:23:55.770 INFO:teuthology.orchestra.run.smithi055.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-simplegeneric all 0.8.1-1 [11.5 kB] 2020-09-08T15:23:55.771 INFO:teuthology.orchestra.run.smithi055.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-singledispatch all 3.4.0.3-2 [9,848 B] 2020-09-08T15:23:55.772 INFO:teuthology.orchestra.run.smithi055.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-simplejson amd64 3.13.2-1 [61.2 kB] 2020-09-08T15:23:55.778 INFO:teuthology.orchestra.run.smithi104.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 formencode-i18n all 1.3.0-0ubuntu5 [3,382 B] 2020-09-08T15:23:55.779 INFO:teuthology.orchestra.run.smithi104.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-formencode all 1.3.0-0ubuntu5 [69.1 kB] 2020-09-08T15:23:55.783 INFO:teuthology.orchestra.run.smithi104.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-tempita all 0.5.2-2 [13.9 kB] 2020-09-08T15:23:55.784 INFO:teuthology.orchestra.run.smithi104.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-09-08T15:23:55.850 INFO:teuthology.orchestra.run.smithi166.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-09-08T15:23:55.850 INFO:teuthology.orchestra.run.smithi166.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-09-08T15:23:55.851 INFO:teuthology.orchestra.run.smithi055.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-09-08T15:23:55.854 INFO:teuthology.orchestra.run.smithi166.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-09-08T15:23:55.856 INFO:teuthology.orchestra.run.smithi166.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-09-08T15:23:55.856 INFO:teuthology.orchestra.run.smithi055.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-bs4 all 4.6.0-1 [67.9 kB] 2020-09-08T15:23:55.860 INFO:teuthology.orchestra.run.smithi055.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-waitress all 1.0.1-1 [53.3 kB] 2020-09-08T15:23:55.863 INFO:teuthology.orchestra.run.smithi055.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-dnspython all 1.15.0-1 [84.9 kB] 2020-09-08T15:23:55.867 INFO:teuthology.orchestra.run.smithi104.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-09-08T15:23:55.867 INFO:teuthology.orchestra.run.smithi055.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 formencode-i18n all 1.3.0-0ubuntu5 [3,382 B] 2020-09-08T15:23:55.868 INFO:teuthology.orchestra.run.smithi104.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy all 1.5.2-4 [25.6 kB] 2020-09-08T15:23:55.868 INFO:teuthology.orchestra.run.smithi104.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-webtest all 2.0.28-1ubuntu1 [27.8 kB] 2020-09-08T15:23:55.870 INFO:teuthology.orchestra.run.smithi104.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pecan all 1.2.1-2 [86.1 kB] 2020-09-08T15:23:55.880 INFO:teuthology.orchestra.run.smithi104.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-bcrypt amd64 3.1.4-2 [29.8 kB] 2020-09-08T15:23:55.881 INFO:teuthology.orchestra.run.smithi104.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-werkzeug all 0.14.1+dfsg1-1 [174 kB] 2020-09-08T15:23:55.930 INFO:teuthology.orchestra.run.smithi166.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-09-08T15:23:55.938 INFO:teuthology.orchestra.run.smithi166.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-09-08T15:23:55.940 INFO:teuthology.orchestra.run.smithi055.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-formencode all 1.3.0-0ubuntu5 [69.1 kB] 2020-09-08T15:23:55.943 INFO:teuthology.orchestra.run.smithi055.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-tempita all 0.5.2-2 [13.9 kB] 2020-09-08T15:23:55.944 INFO:teuthology.orchestra.run.smithi055.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-09-08T15:23:55.955 INFO:teuthology.orchestra.run.smithi104.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-09-08T15:23:55.955 INFO:teuthology.orchestra.run.smithi055.stdout:Get:35 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 python-rgw amd64 13.2.10-45-g94aa832295-1bionic [98.9 kB] 2020-09-08T15:23:55.958 INFO:teuthology.orchestra.run.smithi166.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 libjs-underscore all 1.8.3~dfsg-1 [59.9 kB] 2020-09-08T15:23:55.961 INFO:teuthology.orchestra.run.smithi104.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-09-08T15:23:55.961 INFO:teuthology.orchestra.run.smithi166.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/main amd64 libjs-sphinxdoc all 1.6.7-1ubuntu1 [85.6 kB] 2020-09-08T15:23:55.965 INFO:teuthology.orchestra.run.smithi166.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-webencodings all 0.5-2 [10.3 kB] 2020-09-08T15:23:55.966 INFO:teuthology.orchestra.run.smithi166.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-html5lib all 0.999999999-1 [83.6 kB] 2020-09-08T15:23:55.967 INFO:teuthology.orchestra.run.smithi104.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-09-08T15:23:55.973 INFO:teuthology.orchestra.run.smithi055.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-09-08T15:23:55.973 INFO:teuthology.orchestra.run.smithi055.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy all 1.5.2-4 [25.6 kB] 2020-09-08T15:23:55.974 INFO:teuthology.orchestra.run.smithi104.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-09-08T15:23:56.006 INFO:teuthology.orchestra.run.smithi055.stdout:Get:38 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 13.2.10-45-g94aa832295-1bionic [306 kB] 2020-09-08T15:23:56.016 INFO:teuthology.orchestra.run.smithi055.stdout:Get:39 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 13.2.10-45-g94aa832295-1bionic [12.0 MB] 2020-09-08T15:23:56.018 INFO:teuthology.orchestra.run.smithi166.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python-lxml amd64 4.2.1-1ubuntu0.1 [1,075 kB] 2020-09-08T15:23:56.029 INFO:teuthology.orchestra.run.smithi055.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-webtest all 2.0.28-1ubuntu1 [27.8 kB] 2020-09-08T15:23:56.031 INFO:teuthology.orchestra.run.smithi055.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pecan all 1.2.1-2 [86.1 kB] 2020-09-08T15:23:56.034 INFO:teuthology.orchestra.run.smithi055.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-bcrypt amd64 3.1.4-2 [29.8 kB] 2020-09-08T15:23:56.036 INFO:teuthology.orchestra.run.smithi055.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-werkzeug all 0.14.1+dfsg1-1 [174 kB] 2020-09-08T15:23:56.044 INFO:teuthology.orchestra.run.smithi104.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-09-08T15:23:56.057 INFO:teuthology.orchestra.run.smithi104.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-09-08T15:23:56.058 INFO:teuthology.orchestra.run.smithi055.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-09-08T15:23:56.063 INFO:teuthology.orchestra.run.smithi104.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 libjs-underscore all 1.8.3~dfsg-1 [59.9 kB] 2020-09-08T15:23:56.065 INFO:teuthology.orchestra.run.smithi104.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/main amd64 libjs-sphinxdoc all 1.6.7-1ubuntu1 [85.6 kB] 2020-09-08T15:23:56.066 INFO:teuthology.orchestra.run.smithi166.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-openid all 2.2.5-6 [97.8 kB] 2020-09-08T15:23:56.068 INFO:teuthology.orchestra.run.smithi104.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-webencodings all 0.5-2 [10.3 kB] 2020-09-08T15:23:56.068 INFO:teuthology.orchestra.run.smithi104.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-html5lib all 0.999999999-1 [83.6 kB] 2020-09-08T15:23:56.070 INFO:teuthology.orchestra.run.smithi166.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastescript all 2.0.2-2 [54.9 kB] 2020-09-08T15:23:56.073 INFO:teuthology.orchestra.run.smithi166.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pyinotify all 0.9.6-1 [24.6 kB] 2020-09-08T15:23:56.106 INFO:teuthology.orchestra.run.smithi166.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-repoze.lru all 0.7-1 [11.9 kB] 2020-09-08T15:23:56.107 INFO:teuthology.orchestra.run.smithi166.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-routes all 2.4.1-1 [88.3 kB] 2020-09-08T15:23:56.112 INFO:teuthology.orchestra.run.smithi166.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-scgi amd64 1.13-1.1build1 [18.4 kB] 2020-09-08T15:23:56.116 INFO:teuthology.orchestra.run.smithi055.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-09-08T15:23:56.122 INFO:teuthology.orchestra.run.smithi055.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-09-08T15:23:56.124 INFO:teuthology.orchestra.run.smithi055.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-09-08T15:23:56.131 INFO:teuthology.orchestra.run.smithi104.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python-lxml amd64 4.2.1-1ubuntu0.1 [1,075 kB] 2020-09-08T15:23:56.134 INFO:teuthology.orchestra.run.smithi055.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-09-08T15:23:56.140 INFO:teuthology.orchestra.run.smithi055.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-09-08T15:23:56.167 INFO:teuthology.orchestra.run.smithi104.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-openid all 2.2.5-6 [97.8 kB] 2020-09-08T15:23:56.169 INFO:teuthology.orchestra.run.smithi104.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastescript all 2.0.2-2 [54.9 kB] 2020-09-08T15:23:56.171 INFO:teuthology.orchestra.run.smithi104.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pyinotify all 0.9.6-1 [24.6 kB] 2020-09-08T15:23:56.204 INFO:teuthology.orchestra.run.smithi055.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 libjs-underscore all 1.8.3~dfsg-1 [59.9 kB] 2020-09-08T15:23:56.207 INFO:teuthology.orchestra.run.smithi055.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/main amd64 libjs-sphinxdoc all 1.6.7-1ubuntu1 [85.6 kB] 2020-09-08T15:23:56.210 INFO:teuthology.orchestra.run.smithi055.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-webencodings all 0.5-2 [10.3 kB] 2020-09-08T15:23:56.211 INFO:teuthology.orchestra.run.smithi055.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-html5lib all 0.999999999-1 [83.6 kB] 2020-09-08T15:23:56.214 INFO:teuthology.orchestra.run.smithi055.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python-lxml amd64 4.2.1-1ubuntu0.1 [1,075 kB] 2020-09-08T15:23:56.220 INFO:teuthology.orchestra.run.smithi104.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-repoze.lru all 0.7-1 [11.9 kB] 2020-09-08T15:23:56.220 INFO:teuthology.orchestra.run.smithi104.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-routes all 2.4.1-1 [88.3 kB] 2020-09-08T15:23:56.223 INFO:teuthology.orchestra.run.smithi104.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-scgi amd64 1.13-1.1build1 [18.4 kB] 2020-09-08T15:23:56.292 INFO:teuthology.orchestra.run.smithi055.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-openid all 2.2.5-6 [97.8 kB] 2020-09-08T15:23:56.295 INFO:teuthology.orchestra.run.smithi055.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastescript all 2.0.2-2 [54.9 kB] 2020-09-08T15:23:56.297 INFO:teuthology.orchestra.run.smithi055.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pyinotify all 0.9.6-1 [24.6 kB] 2020-09-08T15:23:56.298 INFO:teuthology.orchestra.run.smithi055.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-repoze.lru all 0.7-1 [11.9 kB] 2020-09-08T15:23:56.299 INFO:teuthology.orchestra.run.smithi055.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-routes all 2.4.1-1 [88.3 kB] 2020-09-08T15:23:56.381 INFO:teuthology.orchestra.run.smithi055.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-scgi amd64 1.13-1.1build1 [18.4 kB] 2020-09-08T15:23:56.512 INFO:teuthology.orchestra.run.smithi166.stdout:Get:61 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 13.2.10-45-g94aa832295-1bionic [3,645 kB] 2020-09-08T15:23:56.734 INFO:teuthology.orchestra.run.smithi166.stdout:Get:62 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 13.2.10-45-g94aa832295-1bionic [2,738 kB] 2020-09-08T15:23:56.745 INFO:teuthology.orchestra.run.smithi104.stdout:Get:61 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 13.2.10-45-g94aa832295-1bionic [3,645 kB] 2020-09-08T15:23:56.957 INFO:teuthology.orchestra.run.smithi166.stdout:Get:63 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 13.2.10-45-g94aa832295-1bionic [2,907 kB] 2020-09-08T15:23:57.071 INFO:teuthology.orchestra.run.smithi104.stdout:Get:62 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 13.2.10-45-g94aa832295-1bionic [2,738 kB] 2020-09-08T15:23:57.185 INFO:teuthology.orchestra.run.smithi166.stdout:Get:64 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 13.2.10-45-g94aa832295-1bionic [13.2 MB] 2020-09-08T15:23:57.331 INFO:teuthology.orchestra.run.smithi104.stdout:Get:63 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 13.2.10-45-g94aa832295-1bionic [2,907 kB] 2020-09-08T15:23:57.602 INFO:teuthology.orchestra.run.smithi104.stdout:Get:64 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 13.2.10-45-g94aa832295-1bionic [13.2 MB] 2020-09-08T15:23:57.642 INFO:teuthology.orchestra.run.smithi055.stdout:Get:61 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 13.2.10-45-g94aa832295-1bionic [3,645 kB] 2020-09-08T15:23:58.134 INFO:teuthology.orchestra.run.smithi055.stdout:Get:62 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 13.2.10-45-g94aa832295-1bionic [2,738 kB] 2020-09-08T15:23:58.267 INFO:teuthology.orchestra.run.smithi166.stdout:Get:65 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 13.2.10-45-g94aa832295-1bionic [3,896 B] 2020-09-08T15:23:58.267 INFO:teuthology.orchestra.run.smithi166.stdout:Get:66 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 13.2.10-45-g94aa832295-1bionic [460 kB] 2020-09-08T15:23:58.299 INFO:teuthology.orchestra.run.smithi166.stdout:Get:67 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 13.2.10-45-g94aa832295-1bionic [1,645 kB] 2020-09-08T15:23:58.442 INFO:teuthology.orchestra.run.smithi166.stdout:Get:68 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 13.2.10-45-g94aa832295-1bionic [22.3 MB] 2020-09-08T15:23:58.501 INFO:teuthology.orchestra.run.smithi055.stdout:Get:63 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 13.2.10-45-g94aa832295-1bionic [2,907 kB] 2020-09-08T15:23:58.844 INFO:teuthology.orchestra.run.smithi104.stdout:Get:65 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 13.2.10-45-g94aa832295-1bionic [3,896 B] 2020-09-08T15:23:59.336 INFO:teuthology.orchestra.run.smithi104.stdout:Get:66 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 13.2.10-45-g94aa832295-1bionic [460 kB] 2020-09-08T15:23:59.338 INFO:teuthology.orchestra.run.smithi055.stdout:Get:64 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 13.2.10-45-g94aa832295-1bionic [13.2 MB] 2020-09-08T15:23:59.338 INFO:teuthology.orchestra.run.smithi104.stdout:Get:67 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 13.2.10-45-g94aa832295-1bionic [1,645 kB] 2020-09-08T15:23:59.339 INFO:teuthology.orchestra.run.smithi104.stdout:Get:68 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 13.2.10-45-g94aa832295-1bionic [22.3 MB] 2020-09-08T15:24:00.259 INFO:teuthology.orchestra.run.smithi166.stdout:Get:69 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 13.2.10-45-g94aa832295-1bionic [16.4 kB] 2020-09-08T15:24:00.259 INFO:teuthology.orchestra.run.smithi166.stdout:Get:70 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 13.2.10-45-g94aa832295-1bionic [3,219 kB] 2020-09-08T15:24:00.502 INFO:teuthology.orchestra.run.smithi166.stdout:Get:71 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 13.2.10-45-g94aa832295-1bionic [13.7 kB] 2020-09-08T15:24:00.623 INFO:teuthology.orchestra.run.smithi055.stdout:Get:65 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 13.2.10-45-g94aa832295-1bionic [3,896 B] 2020-09-08T15:24:00.623 INFO:teuthology.orchestra.run.smithi055.stdout:Get:66 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 13.2.10-45-g94aa832295-1bionic [460 kB] 2020-09-08T15:24:00.671 INFO:teuthology.orchestra.run.smithi055.stdout:Get:67 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 13.2.10-45-g94aa832295-1bionic [1,645 kB] 2020-09-08T15:24:00.815 INFO:teuthology.orchestra.run.smithi104.stdout:Get:69 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 13.2.10-45-g94aa832295-1bionic [16.4 kB] 2020-09-08T15:24:00.815 INFO:teuthology.orchestra.run.smithi104.stdout:Get:70 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 13.2.10-45-g94aa832295-1bionic [3,219 kB] 2020-09-08T15:24:00.836 INFO:teuthology.orchestra.run.smithi055.stdout:Get:68 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 13.2.10-45-g94aa832295-1bionic [22.3 MB] 2020-09-08T15:24:00.979 INFO:teuthology.orchestra.run.smithi104.stdout:Get:71 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 13.2.10-45-g94aa832295-1bionic [13.7 kB] 2020-09-08T15:24:01.076 INFO:teuthology.orchestra.run.smithi166.stdout:Fetched 74.2 MB in 6s (12.7 MB/s) 2020-09-08T15:24:01.255 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-09-08T15:24:01.304 INFO:teuthology.orchestra.run.smithi166.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137586 files and directories currently installed.) 2020-09-08T15:24:01.308 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../00-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-09-08T15:24:01.412 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-09-08T15:24:01.566 INFO:teuthology.orchestra.run.smithi104.stdout:Fetched 74.2 MB in 6s (11.9 MB/s) 2020-09-08T15:24:01.703 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-09-08T15:24:01.738 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../01-librbd1_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:01.749 INFO:teuthology.orchestra.run.smithi104.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137586 files and directories currently installed.) 2020-09-08T15:24:01.753 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../00-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-09-08T15:24:01.841 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking librbd1 (13.2.10-45-g94aa832295-1bionic) over (12.2.13-0ubuntu0.18.04.3) ... 2020-09-08T15:24:01.842 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-09-08T15:24:01.988 INFO:teuthology.orchestra.run.smithi055.stdout:Get:69 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 13.2.10-45-g94aa832295-1bionic [16.4 kB] 2020-09-08T15:24:01.989 INFO:teuthology.orchestra.run.smithi055.stdout:Get:70 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 13.2.10-45-g94aa832295-1bionic [3,219 kB] 2020-09-08T15:24:02.138 INFO:teuthology.orchestra.run.smithi055.stdout:Get:71 https://1.chacra.ceph.com/r/ceph/mimic/94aa832295faacd5e1039dde5544bbf1ad3458d8/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 13.2.10-45-g94aa832295-1bionic [13.7 kB] 2020-09-08T15:24:02.185 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../01-librbd1_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:02.333 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../02-librados2_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:02.343 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking librbd1 (13.2.10-45-g94aa832295-1bionic) over (12.2.13-0ubuntu0.18.04.3) ... 2020-09-08T15:24:02.461 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking librados2 (13.2.10-45-g94aa832295-1bionic) over (12.2.13-0ubuntu0.18.04.3) ... 2020-09-08T15:24:02.733 INFO:teuthology.orchestra.run.smithi055.stdout:Fetched 74.2 MB in 7s (10.3 MB/s) 2020-09-08T15:24:02.846 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../02-librados2_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:02.904 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-09-08T15:24:02.955 INFO:teuthology.orchestra.run.smithi055.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137586 files and directories currently installed.) 2020-09-08T15:24:02.956 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking librados2 (13.2.10-45-g94aa832295-1bionic) over (12.2.13-0ubuntu0.18.04.3) ... 2020-09-08T15:24:02.962 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../00-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-09-08T15:24:03.042 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package libcephfs2. 2020-09-08T15:24:03.046 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-09-08T15:24:03.055 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../03-libcephfs2_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:03.085 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking libcephfs2 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:03.352 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../01-librbd1_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:03.377 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-rados. 2020-09-08T15:24:03.390 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../04-python-rados_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:03.419 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-rados (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:03.449 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking librbd1 (13.2.10-45-g94aa832295-1bionic) over (12.2.13-0ubuntu0.18.04.3) ... 2020-09-08T15:24:03.554 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libcephfs2. 2020-09-08T15:24:03.570 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../03-libcephfs2_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:03.613 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libcephfs2 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:03.730 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-ceph-argparse. 2020-09-08T15:24:03.744 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../05-python-ceph-argparse_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:03.789 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-ceph-argparse (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:03.906 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-rados. 2020-09-08T15:24:03.920 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../04-python-rados_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:03.924 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../02-librados2_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:03.998 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-rados (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:04.044 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking librados2 (13.2.10-45-g94aa832295-1bionic) over (12.2.13-0ubuntu0.18.04.3) ... 2020-09-08T15:24:04.065 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-cephfs. 2020-09-08T15:24:04.080 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../06-python-cephfs_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:04.119 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-cephfs (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:04.296 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-ceph-argparse. 2020-09-08T15:24:04.306 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../05-python-ceph-argparse_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:04.342 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-ceph-argparse (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:04.443 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-prettytable. 2020-09-08T15:24:04.457 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../07-python-prettytable_0.7.2-3_all.deb ... 2020-09-08T15:24:04.485 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-prettytable (0.7.2-3) ... 2020-09-08T15:24:04.637 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-cephfs. 2020-09-08T15:24:04.639 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libcephfs2. 2020-09-08T15:24:04.649 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../06-python-cephfs_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:04.655 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../03-libcephfs2_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:04.686 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-cephfs (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:04.689 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libcephfs2 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:04.737 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-rbd. 2020-09-08T15:24:04.752 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../08-python-rbd_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:04.796 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-rbd (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:04.959 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-rados. 2020-09-08T15:24:04.975 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../04-python-rados_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:05.016 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-rados (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:05.071 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-prettytable. 2020-09-08T15:24:05.084 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../07-python-prettytable_0.7.2-3_all.deb ... 2020-09-08T15:24:05.121 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-prettytable (0.7.2-3) ... 2020-09-08T15:24:05.131 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-certifi. 2020-09-08T15:24:05.145 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../09-python-certifi_2018.1.18-2_all.deb ... 2020-09-08T15:24:05.182 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-certifi (2018.1.18-2) ... 2020-09-08T15:24:05.335 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-ceph-argparse. 2020-09-08T15:24:05.350 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../05-python-ceph-argparse_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:05.382 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-rbd. 2020-09-08T15:24:05.394 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-ceph-argparse (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:05.395 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../08-python-rbd_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:05.434 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-rbd (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:05.453 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-chardet. 2020-09-08T15:24:05.470 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../10-python-chardet_3.0.4-1_all.deb ... 2020-09-08T15:24:05.526 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-chardet (3.0.4-1) ... 2020-09-08T15:24:05.702 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-cephfs. 2020-09-08T15:24:05.712 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../06-python-cephfs_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:05.755 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-cephfs (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:05.777 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-certifi. 2020-09-08T15:24:05.791 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../09-python-certifi_2018.1.18-2_all.deb ... 2020-09-08T15:24:05.826 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-certifi (2018.1.18-2) ... 2020-09-08T15:24:05.913 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-urllib3. 2020-09-08T15:24:05.927 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../11-python-urllib3_1.22-1ubuntu0.18.04.1_all.deb ... 2020-09-08T15:24:05.971 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-urllib3 (1.22-1ubuntu0.18.04.1) ... 2020-09-08T15:24:06.041 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-prettytable. 2020-09-08T15:24:06.056 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../07-python-prettytable_0.7.2-3_all.deb ... 2020-09-08T15:24:06.121 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-chardet. 2020-09-08T15:24:06.137 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../10-python-chardet_3.0.4-1_all.deb ... 2020-09-08T15:24:06.171 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-chardet (3.0.4-1) ... 2020-09-08T15:24:06.266 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-prettytable (0.7.2-3) ... 2020-09-08T15:24:06.291 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-requests. 2020-09-08T15:24:06.304 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../12-python-requests_2.18.4-2ubuntu0.1_all.deb ... 2020-09-08T15:24:06.341 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-requests (2.18.4-2ubuntu0.1) ... 2020-09-08T15:24:06.493 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-rbd. 2020-09-08T15:24:06.510 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../08-python-rbd_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:06.552 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-rbd (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:06.583 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-urllib3. 2020-09-08T15:24:06.596 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../11-python-urllib3_1.22-1ubuntu0.18.04.1_all.deb ... 2020-09-08T15:24:06.641 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-urllib3 (1.22-1ubuntu0.18.04.1) ... 2020-09-08T15:24:06.719 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package librgw2. 2020-09-08T15:24:06.732 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../13-librgw2_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:06.777 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking librgw2 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:06.904 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-certifi. 2020-09-08T15:24:06.920 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../09-python-certifi_2018.1.18-2_all.deb ... 2020-09-08T15:24:06.963 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-certifi (2018.1.18-2) ... 2020-09-08T15:24:07.011 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-requests. 2020-09-08T15:24:07.024 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../12-python-requests_2.18.4-2ubuntu0.1_all.deb ... 2020-09-08T15:24:07.069 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-requests (2.18.4-2ubuntu0.1) ... 2020-09-08T15:24:07.221 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-rgw. 2020-09-08T15:24:07.234 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../14-python-rgw_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:07.297 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-chardet. 2020-09-08T15:24:07.298 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-rgw (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:07.310 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../10-python-chardet_3.0.4-1_all.deb ... 2020-09-08T15:24:07.349 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-chardet (3.0.4-1) ... 2020-09-08T15:24:07.422 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package librgw2. 2020-09-08T15:24:07.436 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../13-librgw2_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:07.474 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking librgw2 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:07.657 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package liboath0. 2020-09-08T15:24:07.671 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../15-liboath0_2.6.1-1_amd64.deb ... 2020-09-08T15:24:07.699 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-09-08T15:24:07.724 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-urllib3. 2020-09-08T15:24:07.740 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../11-python-urllib3_1.22-1ubuntu0.18.04.1_all.deb ... 2020-09-08T15:24:07.778 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-urllib3 (1.22-1ubuntu0.18.04.1) ... 2020-09-08T15:24:07.932 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-rgw. 2020-09-08T15:24:07.947 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../14-python-rgw_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:07.990 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-rgw (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:08.062 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package libradosstriper1. 2020-09-08T15:24:08.076 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../16-libradosstriper1_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:08.091 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-requests. 2020-09-08T15:24:08.094 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../12-python-requests_2.18.4-2ubuntu0.1_all.deb ... 2020-09-08T15:24:08.118 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking libradosstriper1 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:08.122 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-requests (2.18.4-2ubuntu0.1) ... 2020-09-08T15:24:08.478 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package librgw2. 2020-09-08T15:24:08.483 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../13-librgw2_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:08.527 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking librgw2 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:08.580 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package ceph-common. 2020-09-08T15:24:08.599 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../17-ceph-common_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:08.642 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking ceph-common (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:08.727 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package liboath0. 2020-09-08T15:24:08.741 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../15-liboath0_2.6.1-1_amd64.deb ... 2020-09-08T15:24:08.786 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-09-08T15:24:08.965 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-rgw. 2020-09-08T15:24:08.976 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../14-python-rgw_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:09.007 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-rgw (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:09.129 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libradosstriper1. 2020-09-08T15:24:09.142 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../16-libradosstriper1_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:09.179 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libradosstriper1 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:09.350 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package liboath0. 2020-09-08T15:24:09.365 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../15-liboath0_2.6.1-1_amd64.deb ... 2020-09-08T15:24:09.408 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-09-08T15:24:09.599 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-common. 2020-09-08T15:24:09.618 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../17-ceph-common_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:09.657 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-common (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:09.698 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libradosstriper1. 2020-09-08T15:24:09.711 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../16-libradosstriper1_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:09.753 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libradosstriper1 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:10.086 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-09-08T15:24:10.110 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../18-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-09-08T15:24:10.144 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-09-08T15:24:10.156 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-common. 2020-09-08T15:24:10.170 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../17-ceph-common_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:10.215 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-common (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:10.471 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-09-08T15:24:10.492 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../19-libleveldb1v5_1.20-2_amd64.deb ... 2020-09-08T15:24:10.521 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-09-08T15:24:10.888 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-09-08T15:24:10.901 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package ceph-base. 2020-09-08T15:24:10.901 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../18-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-09-08T15:24:10.914 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../20-ceph-base_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:10.946 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-09-08T15:24:10.984 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking ceph-base (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:11.240 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-09-08T15:24:11.257 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../19-libleveldb1v5_1.20-2_amd64.deb ... 2020-09-08T15:24:11.290 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-09-08T15:24:11.571 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-09-08T15:24:11.586 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../18-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-09-08T15:24:11.630 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-09-08T15:24:11.739 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-cherrypy3. 2020-09-08T15:24:11.752 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../21-python-cherrypy3_8.9.1-2_all.deb ... 2020-09-08T15:24:11.793 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-base. 2020-09-08T15:24:11.796 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-cherrypy3 (8.9.1-2) ... 2020-09-08T15:24:11.806 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../20-ceph-base_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:11.852 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-base (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:11.941 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-09-08T15:24:11.955 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../19-libleveldb1v5_1.20-2_amd64.deb ... 2020-09-08T15:24:11.991 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-09-08T15:24:12.125 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-markupsafe. 2020-09-08T15:24:12.139 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../22-python-markupsafe_1.0-1build1_amd64.deb ... 2020-09-08T15:24:12.192 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-markupsafe (1.0-1build1) ... 2020-09-08T15:24:12.437 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-base. 2020-09-08T15:24:12.453 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../20-ceph-base_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:12.502 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-jinja2. 2020-09-08T15:24:12.504 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-base (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:12.518 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../23-python-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-09-08T15:24:12.561 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-09-08T15:24:12.607 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-cherrypy3. 2020-09-08T15:24:12.622 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../21-python-cherrypy3_8.9.1-2_all.deb ... 2020-09-08T15:24:12.648 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-cherrypy3 (8.9.1-2) ... 2020-09-08T15:24:12.897 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-openssl. 2020-09-08T15:24:12.911 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../24-python-openssl_17.5.0-1ubuntu1_all.deb ... 2020-09-08T15:24:12.947 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-openssl (17.5.0-1ubuntu1) ... 2020-09-08T15:24:13.010 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-markupsafe. 2020-09-08T15:24:13.023 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../22-python-markupsafe_1.0-1build1_amd64.deb ... 2020-09-08T15:24:13.060 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-markupsafe (1.0-1build1) ... 2020-09-08T15:24:13.276 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-cherrypy3. 2020-09-08T15:24:13.290 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../21-python-cherrypy3_8.9.1-2_all.deb ... 2020-09-08T15:24:13.300 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-logutils. 2020-09-08T15:24:13.315 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../25-python-logutils_0.3.3-5_all.deb ... 2020-09-08T15:24:13.360 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-cherrypy3 (8.9.1-2) ... 2020-09-08T15:24:13.366 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-logutils (0.3.3-5) ... 2020-09-08T15:24:13.371 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-jinja2. 2020-09-08T15:24:13.385 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../23-python-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-09-08T15:24:13.446 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-09-08T15:24:13.681 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-mako. 2020-09-08T15:24:13.691 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../26-python-mako_1.0.7+ds1-1_all.deb ... 2020-09-08T15:24:13.714 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-markupsafe. 2020-09-08T15:24:13.727 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../22-python-markupsafe_1.0-1build1_amd64.deb ... 2020-09-08T15:24:13.727 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-mako (1.0.7+ds1-1) ... 2020-09-08T15:24:13.764 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-markupsafe (1.0-1build1) ... 2020-09-08T15:24:13.832 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-openssl. 2020-09-08T15:24:13.847 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../24-python-openssl_17.5.0-1ubuntu1_all.deb ... 2020-09-08T15:24:13.892 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-openssl (17.5.0-1ubuntu1) ... 2020-09-08T15:24:14.067 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-jinja2. 2020-09-08T15:24:14.081 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../23-python-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-09-08T15:24:14.118 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-simplegeneric. 2020-09-08T15:24:14.121 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../27-python-simplegeneric_0.8.1-1_all.deb ... 2020-09-08T15:24:14.127 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-09-08T15:24:14.163 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-simplegeneric (0.8.1-1) ... 2020-09-08T15:24:14.236 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-logutils. 2020-09-08T15:24:14.252 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../25-python-logutils_0.3.3-5_all.deb ... 2020-09-08T15:24:14.295 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-logutils (0.3.3-5) ... 2020-09-08T15:24:14.487 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-openssl. 2020-09-08T15:24:14.502 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../24-python-openssl_17.5.0-1ubuntu1_all.deb ... 2020-09-08T15:24:14.546 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-openssl (17.5.0-1ubuntu1) ... 2020-09-08T15:24:14.622 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-mako. 2020-09-08T15:24:14.636 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../26-python-mako_1.0.7+ds1-1_all.deb ... 2020-09-08T15:24:14.681 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-mako (1.0.7+ds1-1) ... 2020-09-08T15:24:14.733 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-singledispatch. 2020-09-08T15:24:14.746 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../28-python-singledispatch_3.4.0.3-2_all.deb ... 2020-09-08T15:24:14.783 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-singledispatch (3.4.0.3-2) ... 2020-09-08T15:24:14.856 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-logutils. 2020-09-08T15:24:14.871 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../25-python-logutils_0.3.3-5_all.deb ... 2020-09-08T15:24:14.915 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-logutils (0.3.3-5) ... 2020-09-08T15:24:15.034 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-simplegeneric. 2020-09-08T15:24:15.048 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../27-python-simplegeneric_0.8.1-1_all.deb ... 2020-09-08T15:24:15.094 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-simplegeneric (0.8.1-1) ... 2020-09-08T15:24:15.122 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-simplejson. 2020-09-08T15:24:15.133 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../29-python-simplejson_3.13.2-1_amd64.deb ... 2020-09-08T15:24:15.186 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-simplejson (3.13.2-1) ... 2020-09-08T15:24:15.237 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-mako. 2020-09-08T15:24:15.243 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../26-python-mako_1.0.7+ds1-1_all.deb ... 2020-09-08T15:24:15.284 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-mako (1.0.7+ds1-1) ... 2020-09-08T15:24:15.422 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-singledispatch. 2020-09-08T15:24:15.434 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../28-python-singledispatch_3.4.0.3-2_all.deb ... 2020-09-08T15:24:15.470 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-singledispatch (3.4.0.3-2) ... 2020-09-08T15:24:15.550 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-webob. 2020-09-08T15:24:15.562 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../30-python-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-09-08T15:24:15.605 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-webob (1:1.7.3-2fakesync1) ... 2020-09-08T15:24:15.641 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-simplegeneric. 2020-09-08T15:24:15.658 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../27-python-simplegeneric_0.8.1-1_all.deb ... 2020-09-08T15:24:15.696 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-simplegeneric (0.8.1-1) ... 2020-09-08T15:24:15.774 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-simplejson. 2020-09-08T15:24:15.785 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../29-python-simplejson_3.13.2-1_amd64.deb ... 2020-09-08T15:24:15.849 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-simplejson (3.13.2-1) ... 2020-09-08T15:24:15.924 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-bs4. 2020-09-08T15:24:15.941 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../31-python-bs4_4.6.0-1_all.deb ... 2020-09-08T15:24:15.990 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-bs4 (4.6.0-1) ... 2020-09-08T15:24:16.016 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-singledispatch. 2020-09-08T15:24:16.031 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../28-python-singledispatch_3.4.0.3-2_all.deb ... 2020-09-08T15:24:16.074 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-singledispatch (3.4.0.3-2) ... 2020-09-08T15:24:16.234 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-webob. 2020-09-08T15:24:16.251 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../30-python-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-09-08T15:24:16.299 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-webob (1:1.7.3-2fakesync1) ... 2020-09-08T15:24:16.319 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-waitress. 2020-09-08T15:24:16.333 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../32-python-waitress_1.0.1-1_all.deb ... 2020-09-08T15:24:16.369 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-waitress (1.0.1-1) ... 2020-09-08T15:24:16.530 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-simplejson. 2020-09-08T15:24:16.542 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../29-python-simplejson_3.13.2-1_amd64.deb ... 2020-09-08T15:24:16.577 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-simplejson (3.13.2-1) ... 2020-09-08T15:24:16.670 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-bs4. 2020-09-08T15:24:16.685 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../31-python-bs4_4.6.0-1_all.deb ... 2020-09-08T15:24:16.700 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-dnspython. 2020-09-08T15:24:16.712 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../33-python-dnspython_1.15.0-1_all.deb ... 2020-09-08T15:24:16.727 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-bs4 (4.6.0-1) ... 2020-09-08T15:24:16.764 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-dnspython (1.15.0-1) ... 2020-09-08T15:24:16.922 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-webob. 2020-09-08T15:24:16.937 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../30-python-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-09-08T15:24:16.981 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-webob (1:1.7.3-2fakesync1) ... 2020-09-08T15:24:17.039 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-waitress. 2020-09-08T15:24:17.052 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../32-python-waitress_1.0.1-1_all.deb ... 2020-09-08T15:24:17.066 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package formencode-i18n. 2020-09-08T15:24:17.080 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../34-formencode-i18n_1.3.0-0ubuntu5_all.deb ... 2020-09-08T15:24:17.113 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-waitress (1.0.1-1) ... 2020-09-08T15:24:17.125 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking formencode-i18n (1.3.0-0ubuntu5) ... 2020-09-08T15:24:17.306 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-bs4. 2020-09-08T15:24:17.314 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../31-python-bs4_4.6.0-1_all.deb ... 2020-09-08T15:24:17.359 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-bs4 (4.6.0-1) ... 2020-09-08T15:24:17.402 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-formencode. 2020-09-08T15:24:17.418 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../35-python-formencode_1.3.0-0ubuntu5_all.deb ... 2020-09-08T15:24:17.441 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-dnspython. 2020-09-08T15:24:17.456 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../33-python-dnspython_1.15.0-1_all.deb ... 2020-09-08T15:24:17.462 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-formencode (1.3.0-0ubuntu5) ... 2020-09-08T15:24:17.529 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-dnspython (1.15.0-1) ... 2020-09-08T15:24:17.713 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-waitress. 2020-09-08T15:24:17.726 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../32-python-waitress_1.0.1-1_all.deb ... 2020-09-08T15:24:17.782 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-waitress (1.0.1-1) ... 2020-09-08T15:24:17.802 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package formencode-i18n. 2020-09-08T15:24:17.818 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../34-formencode-i18n_1.3.0-0ubuntu5_all.deb ... 2020-09-08T15:24:17.822 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-tempita. 2020-09-08T15:24:17.836 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../36-python-tempita_0.5.2-2_all.deb ... 2020-09-08T15:24:17.862 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking formencode-i18n (1.3.0-0ubuntu5) ... 2020-09-08T15:24:17.882 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-tempita (0.5.2-2) ... 2020-09-08T15:24:18.142 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-dnspython. 2020-09-08T15:24:18.156 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../33-python-dnspython_1.15.0-1_all.deb ... 2020-09-08T15:24:18.180 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-formencode. 2020-09-08T15:24:18.191 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-paste. 2020-09-08T15:24:18.195 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../35-python-formencode_1.3.0-0ubuntu5_all.deb ... 2020-09-08T15:24:18.199 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-dnspython (1.15.0-1) ... 2020-09-08T15:24:18.205 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../37-python-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-09-08T15:24:18.238 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-formencode (1.3.0-0ubuntu5) ... 2020-09-08T15:24:18.258 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-paste (2.0.3+dfsg-4ubuntu1) ... 2020-09-08T15:24:18.499 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package formencode-i18n. 2020-09-08T15:24:18.507 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../34-formencode-i18n_1.3.0-0ubuntu5_all.deb ... 2020-09-08T15:24:18.544 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking formencode-i18n (1.3.0-0ubuntu5) ... 2020-09-08T15:24:18.591 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-tempita. 2020-09-08T15:24:18.603 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-09-08T15:24:18.606 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../36-python-tempita_0.5.2-2_all.deb ... 2020-09-08T15:24:18.618 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../38-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-09-08T15:24:18.649 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-tempita (0.5.2-2) ... 2020-09-08T15:24:18.661 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-09-08T15:24:18.847 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-formencode. 2020-09-08T15:24:18.862 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../35-python-formencode_1.3.0-0ubuntu5_all.deb ... 2020-09-08T15:24:18.914 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-formencode (1.3.0-0ubuntu5) ... 2020-09-08T15:24:18.969 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-paste. 2020-09-08T15:24:18.983 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../37-python-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-09-08T15:24:18.988 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-pastedeploy. 2020-09-08T15:24:19.003 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../39-python-pastedeploy_1.5.2-4_all.deb ... 2020-09-08T15:24:19.027 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-paste (2.0.3+dfsg-4ubuntu1) ... 2020-09-08T15:24:19.047 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-pastedeploy (1.5.2-4) ... 2020-09-08T15:24:19.238 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-tempita. 2020-09-08T15:24:19.247 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../36-python-tempita_0.5.2-2_all.deb ... 2020-09-08T15:24:19.285 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-tempita (0.5.2-2) ... 2020-09-08T15:24:19.355 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-09-08T15:24:19.366 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-webtest. 2020-09-08T15:24:19.368 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../38-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-09-08T15:24:19.382 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../40-python-webtest_2.0.28-1ubuntu1_all.deb ... 2020-09-08T15:24:19.413 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-09-08T15:24:19.431 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-webtest (2.0.28-1ubuntu1) ... 2020-09-08T15:24:19.587 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-paste. 2020-09-08T15:24:19.602 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../37-python-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-09-08T15:24:19.645 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-paste (2.0.3+dfsg-4ubuntu1) ... 2020-09-08T15:24:19.757 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-pastedeploy. 2020-09-08T15:24:19.772 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../39-python-pastedeploy_1.5.2-4_all.deb ... 2020-09-08T15:24:19.803 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-pecan. 2020-09-08T15:24:19.815 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-pastedeploy (1.5.2-4) ... 2020-09-08T15:24:19.816 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../41-python-pecan_1.2.1-2_all.deb ... 2020-09-08T15:24:19.881 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-pecan (1.2.1-2) ... 2020-09-08T15:24:19.990 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-09-08T15:24:20.010 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../38-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-09-08T15:24:20.050 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-09-08T15:24:20.177 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-webtest. 2020-09-08T15:24:20.192 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../40-python-webtest_2.0.28-1ubuntu1_all.deb ... 2020-09-08T15:24:20.232 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-bcrypt. 2020-09-08T15:24:20.235 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-webtest (2.0.28-1ubuntu1) ... 2020-09-08T15:24:20.247 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../42-python-bcrypt_3.1.4-2_amd64.deb ... 2020-09-08T15:24:20.290 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-bcrypt (3.1.4-2) ... 2020-09-08T15:24:20.338 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-pastedeploy. 2020-09-08T15:24:20.354 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../39-python-pastedeploy_1.5.2-4_all.deb ... 2020-09-08T15:24:20.393 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-pastedeploy (1.5.2-4) ... 2020-09-08T15:24:20.611 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-werkzeug. 2020-09-08T15:24:20.614 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-pecan. 2020-09-08T15:24:20.624 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../43-python-werkzeug_0.14.1+dfsg1-1_all.deb ... 2020-09-08T15:24:20.630 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../41-python-pecan_1.2.1-2_all.deb ... 2020-09-08T15:24:20.666 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-pecan (1.2.1-2) ... 2020-09-08T15:24:20.674 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-werkzeug (0.14.1+dfsg1-1) ... 2020-09-08T15:24:20.730 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-webtest. 2020-09-08T15:24:20.745 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../40-python-webtest_2.0.28-1ubuntu1_all.deb ... 2020-09-08T15:24:20.788 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-webtest (2.0.28-1ubuntu1) ... 2020-09-08T15:24:21.026 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-bcrypt. 2020-09-08T15:24:21.040 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../42-python-bcrypt_3.1.4-2_amd64.deb ... 2020-09-08T15:24:21.063 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package ceph-mgr. 2020-09-08T15:24:21.077 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../44-ceph-mgr_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:21.084 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-bcrypt (3.1.4-2) ... 2020-09-08T15:24:21.121 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking ceph-mgr (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:21.166 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-pecan. 2020-09-08T15:24:21.181 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../41-python-pecan_1.2.1-2_all.deb ... 2020-09-08T15:24:21.228 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-pecan (1.2.1-2) ... 2020-09-08T15:24:21.396 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-werkzeug. 2020-09-08T15:24:21.410 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../43-python-werkzeug_0.14.1+dfsg1-1_all.deb ... 2020-09-08T15:24:21.457 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-werkzeug (0.14.1+dfsg1-1) ... 2020-09-08T15:24:21.597 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-bcrypt. 2020-09-08T15:24:21.611 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../42-python-bcrypt_3.1.4-2_amd64.deb ... 2020-09-08T15:24:21.655 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-bcrypt (3.1.4-2) ... 2020-09-08T15:24:21.864 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package ceph-mon. 2020-09-08T15:24:21.873 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mgr. 2020-09-08T15:24:21.879 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../45-ceph-mon_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:21.889 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../44-ceph-mgr_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:21.922 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking ceph-mon (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:21.931 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mgr (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:21.958 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-werkzeug. 2020-09-08T15:24:21.973 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../43-python-werkzeug_0.14.1+dfsg1-1_all.deb ... 2020-09-08T15:24:22.017 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-werkzeug (0.14.1+dfsg1-1) ... 2020-09-08T15:24:22.404 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-mgr. 2020-09-08T15:24:22.418 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../44-ceph-mgr_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:22.462 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-mgr (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:22.581 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package ceph-osd. 2020-09-08T15:24:22.589 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../46-ceph-osd_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:22.634 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking ceph-osd (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:22.682 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mon. 2020-09-08T15:24:22.697 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../45-ceph-mon_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:22.740 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mon (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:23.193 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-mon. 2020-09-08T15:24:23.199 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../45-ceph-mon_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:23.240 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-mon (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:23.485 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-osd. 2020-09-08T15:24:23.498 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../46-ceph-osd_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:23.535 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-osd (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:23.855 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package ceph. 2020-09-08T15:24:23.870 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../47-ceph_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:23.914 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking ceph (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:23.927 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-osd. 2020-09-08T15:24:23.941 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../46-ceph-osd_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:23.985 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-osd (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:24.255 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package ceph-fuse. 2020-09-08T15:24:24.270 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../48-ceph-fuse_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:24.316 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking ceph-fuse (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:24.765 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph. 2020-09-08T15:24:24.778 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../47-ceph_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:24.815 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:25.150 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-fuse. 2020-09-08T15:24:25.165 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../48-ceph-fuse_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:25.217 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-fuse (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:25.236 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph. 2020-09-08T15:24:25.246 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../47-ceph_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:25.282 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:25.470 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package ceph-mds. 2020-09-08T15:24:25.489 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../49-ceph-mds_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:25.529 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking ceph-mds (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:25.603 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-fuse. 2020-09-08T15:24:25.621 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../48-ceph-fuse_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:25.656 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mds. 2020-09-08T15:24:25.660 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-fuse (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:25.668 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../49-ceph-mds_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:25.712 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mds (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:26.056 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package libonig4:amd64. 2020-09-08T15:24:26.071 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../50-libonig4_6.7.0-1_amd64.deb ... 2020-09-08T15:24:26.081 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-mds. 2020-09-08T15:24:26.094 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../49-ceph-mds_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:26.114 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-09-08T15:24:26.139 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-mds (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:26.223 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libonig4:amd64. 2020-09-08T15:24:26.236 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../50-libonig4_6.7.0-1_amd64.deb ... 2020-09-08T15:24:26.274 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-09-08T15:24:26.450 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package libjq1:amd64. 2020-09-08T15:24:26.466 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../51-libjq1_1.5+dfsg-2_amd64.deb ... 2020-09-08T15:24:26.529 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-09-08T15:24:26.634 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libjq1:amd64. 2020-09-08T15:24:26.651 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../51-libjq1_1.5+dfsg-2_amd64.deb ... 2020-09-08T15:24:26.701 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-09-08T15:24:26.803 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package jq. 2020-09-08T15:24:26.816 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../52-jq_1.5+dfsg-2_amd64.deb ... 2020-09-08T15:24:26.869 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-09-08T15:24:26.995 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package jq. 2020-09-08T15:24:27.009 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../52-jq_1.5+dfsg-2_amd64.deb ... 2020-09-08T15:24:27.053 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-09-08T15:24:27.130 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package socat. 2020-09-08T15:24:27.144 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../53-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-09-08T15:24:27.188 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-09-08T15:24:27.327 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libonig4:amd64. 2020-09-08T15:24:27.330 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package socat. 2020-09-08T15:24:27.343 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../50-libonig4_6.7.0-1_amd64.deb ... 2020-09-08T15:24:27.344 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../53-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-09-08T15:24:27.385 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-09-08T15:24:27.389 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-09-08T15:24:27.592 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-09-08T15:24:27.609 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../54-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-09-08T15:24:27.650 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-09-08T15:24:27.775 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-09-08T15:24:27.789 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../54-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-09-08T15:24:27.796 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libjq1:amd64. 2020-09-08T15:24:27.811 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../51-libjq1_1.5+dfsg-2_amd64.deb ... 2020-09-08T15:24:27.833 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-09-08T15:24:27.872 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-09-08T15:24:27.903 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package xmlstarlet. 2020-09-08T15:24:27.916 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../55-xmlstarlet_1.6.1-2_amd64.deb ... 2020-09-08T15:24:27.982 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-09-08T15:24:28.134 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package xmlstarlet. 2020-09-08T15:24:28.148 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../55-xmlstarlet_1.6.1-2_amd64.deb ... 2020-09-08T15:24:28.157 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package jq. 2020-09-08T15:24:28.173 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../52-jq_1.5+dfsg-2_amd64.deb ... 2020-09-08T15:24:28.192 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-09-08T15:24:28.216 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-09-08T15:24:28.322 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package ceph-test. 2020-09-08T15:24:28.337 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../56-ceph-test_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:28.382 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking ceph-test (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:28.469 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package socat. 2020-09-08T15:24:28.483 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../53-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-09-08T15:24:28.519 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-09-08T15:24:28.550 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-test. 2020-09-08T15:24:28.562 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../56-ceph-test_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:28.606 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-test (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:28.906 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-09-08T15:24:28.921 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../54-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-09-08T15:24:28.964 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-09-08T15:24:29.234 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package xmlstarlet. 2020-09-08T15:24:29.250 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../55-xmlstarlet_1.6.1-2_amd64.deb ... 2020-09-08T15:24:29.284 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-09-08T15:24:29.600 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-test. 2020-09-08T15:24:29.615 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../56-ceph-test_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:29.649 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-test (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:30.235 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package libcephfs-dev. 2020-09-08T15:24:30.249 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../57-libcephfs-dev_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:30.293 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking libcephfs-dev (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:30.550 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package libjs-underscore. 2020-09-08T15:24:30.562 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../58-libjs-underscore_1.8.3~dfsg-1_all.deb ... 2020-09-08T15:24:30.604 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking libjs-underscore (1.8.3~dfsg-1) ... 2020-09-08T15:24:30.608 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libcephfs-dev. 2020-09-08T15:24:30.623 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../57-libcephfs-dev_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:30.667 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libcephfs-dev (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:30.890 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package libjs-sphinxdoc. 2020-09-08T15:24:30.903 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../59-libjs-sphinxdoc_1.6.7-1ubuntu1_all.deb ... 2020-09-08T15:24:30.962 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libjs-underscore. 2020-09-08T15:24:30.973 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking libjs-sphinxdoc (1.6.7-1ubuntu1) ... 2020-09-08T15:24:30.974 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../58-libjs-underscore_1.8.3~dfsg-1_all.deb ... 2020-09-08T15:24:31.012 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libjs-underscore (1.8.3~dfsg-1) ... 2020-09-08T15:24:31.292 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-webencodings. 2020-09-08T15:24:31.307 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../60-python-webencodings_0.5-2_all.deb ... 2020-09-08T15:24:31.315 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libjs-sphinxdoc. 2020-09-08T15:24:31.328 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../59-libjs-sphinxdoc_1.6.7-1ubuntu1_all.deb ... 2020-09-08T15:24:31.351 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-webencodings (0.5-2) ... 2020-09-08T15:24:31.373 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libjs-sphinxdoc (1.6.7-1ubuntu1) ... 2020-09-08T15:24:31.537 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libcephfs-dev. 2020-09-08T15:24:31.558 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../57-libcephfs-dev_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:31.595 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libcephfs-dev (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:31.686 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-html5lib. 2020-09-08T15:24:31.698 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../61-python-html5lib_0.999999999-1_all.deb ... 2020-09-08T15:24:31.708 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-webencodings. 2020-09-08T15:24:31.721 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../60-python-webencodings_0.5-2_all.deb ... 2020-09-08T15:24:31.737 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-html5lib (0.999999999-1) ... 2020-09-08T15:24:31.765 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-webencodings (0.5-2) ... 2020-09-08T15:24:31.840 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libjs-underscore. 2020-09-08T15:24:31.856 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../58-libjs-underscore_1.8.3~dfsg-1_all.deb ... 2020-09-08T15:24:31.890 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libjs-underscore (1.8.3~dfsg-1) ... 2020-09-08T15:24:32.081 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-lxml:amd64. 2020-09-08T15:24:32.093 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-html5lib. 2020-09-08T15:24:32.097 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../62-python-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-09-08T15:24:32.108 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../61-python-html5lib_0.999999999-1_all.deb ... 2020-09-08T15:24:32.139 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-09-08T15:24:32.151 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libjs-sphinxdoc. 2020-09-08T15:24:32.152 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-html5lib (0.999999999-1) ... 2020-09-08T15:24:32.165 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../59-libjs-sphinxdoc_1.6.7-1ubuntu1_all.deb ... 2020-09-08T15:24:32.209 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libjs-sphinxdoc (1.6.7-1ubuntu1) ... 2020-09-08T15:24:32.537 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-webencodings. 2020-09-08T15:24:32.553 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../60-python-webencodings_0.5-2_all.deb ... 2020-09-08T15:24:32.554 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-lxml:amd64. 2020-09-08T15:24:32.568 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../62-python-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-09-08T15:24:32.592 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-openid. 2020-09-08T15:24:32.595 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-webencodings (0.5-2) ... 2020-09-08T15:24:32.602 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../63-python-openid_2.2.5-6_all.deb ... 2020-09-08T15:24:32.622 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-09-08T15:24:32.652 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-openid (2.2.5-6) ... 2020-09-08T15:24:32.867 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-html5lib. 2020-09-08T15:24:32.883 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../61-python-html5lib_0.999999999-1_all.deb ... 2020-09-08T15:24:32.917 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-html5lib (0.999999999-1) ... 2020-09-08T15:24:33.031 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-pastescript. 2020-09-08T15:24:33.045 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../64-python-pastescript_2.0.2-2_all.deb ... 2020-09-08T15:24:33.089 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-pastescript (2.0.2-2) ... 2020-09-08T15:24:33.099 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-openid. 2020-09-08T15:24:33.113 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../63-python-openid_2.2.5-6_all.deb ... 2020-09-08T15:24:33.166 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-openid (2.2.5-6) ... 2020-09-08T15:24:33.220 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-lxml:amd64. 2020-09-08T15:24:33.237 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../62-python-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-09-08T15:24:33.279 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-09-08T15:24:33.476 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-pyinotify. 2020-09-08T15:24:33.489 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../65-python-pyinotify_0.9.6-1_all.deb ... 2020-09-08T15:24:33.526 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-pyinotify (0.9.6-1) ... 2020-09-08T15:24:33.560 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-pastescript. 2020-09-08T15:24:33.577 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../64-python-pastescript_2.0.2-2_all.deb ... 2020-09-08T15:24:33.618 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-pastescript (2.0.2-2) ... 2020-09-08T15:24:33.708 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-openid. 2020-09-08T15:24:33.722 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../63-python-openid_2.2.5-6_all.deb ... 2020-09-08T15:24:33.766 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-openid (2.2.5-6) ... 2020-09-08T15:24:33.837 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-repoze.lru. 2020-09-08T15:24:33.854 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../66-python-repoze.lru_0.7-1_all.deb ... 2020-09-08T15:24:33.914 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-repoze.lru (0.7-1) ... 2020-09-08T15:24:34.039 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-pyinotify. 2020-09-08T15:24:34.054 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../65-python-pyinotify_0.9.6-1_all.deb ... 2020-09-08T15:24:34.097 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-pyinotify (0.9.6-1) ... 2020-09-08T15:24:34.102 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-pastescript. 2020-09-08T15:24:34.115 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../64-python-pastescript_2.0.2-2_all.deb ... 2020-09-08T15:24:34.152 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-pastescript (2.0.2-2) ... 2020-09-08T15:24:34.215 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-routes. 2020-09-08T15:24:34.229 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../67-python-routes_2.4.1-1_all.deb ... 2020-09-08T15:24:34.273 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-routes (2.4.1-1) ... 2020-09-08T15:24:34.426 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-repoze.lru. 2020-09-08T15:24:34.438 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../66-python-repoze.lru_0.7-1_all.deb ... 2020-09-08T15:24:34.474 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-repoze.lru (0.7-1) ... 2020-09-08T15:24:34.514 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-pyinotify. 2020-09-08T15:24:34.531 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../65-python-pyinotify_0.9.6-1_all.deb ... 2020-09-08T15:24:34.594 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-pyinotify (0.9.6-1) ... 2020-09-08T15:24:34.620 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package python-scgi. 2020-09-08T15:24:34.637 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../68-python-scgi_1.13-1.1build1_amd64.deb ... 2020-09-08T15:24:34.726 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking python-scgi (1.13-1.1build1) ... 2020-09-08T15:24:34.804 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-routes. 2020-09-08T15:24:34.821 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../67-python-routes_2.4.1-1_all.deb ... 2020-09-08T15:24:34.869 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-routes (2.4.1-1) ... 2020-09-08T15:24:34.900 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-repoze.lru. 2020-09-08T15:24:34.914 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../66-python-repoze.lru_0.7-1_all.deb ... 2020-09-08T15:24:34.968 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-repoze.lru (0.7-1) ... 2020-09-08T15:24:35.090 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package radosgw. 2020-09-08T15:24:35.104 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../69-radosgw_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:35.149 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking radosgw (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:35.227 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-scgi. 2020-09-08T15:24:35.242 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../68-python-scgi_1.13-1.1build1_amd64.deb ... 2020-09-08T15:24:35.286 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-routes. 2020-09-08T15:24:35.295 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-scgi (1.13-1.1build1) ... 2020-09-08T15:24:35.302 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../67-python-routes_2.4.1-1_all.deb ... 2020-09-08T15:24:35.344 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-routes (2.4.1-1) ... 2020-09-08T15:24:35.673 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-scgi. 2020-09-08T15:24:35.698 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package radosgw. 2020-09-08T15:24:35.699 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../68-python-scgi_1.13-1.1build1_amd64.deb ... 2020-09-08T15:24:35.706 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../69-radosgw_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:35.723 INFO:teuthology.orchestra.run.smithi166.stdout:Selecting previously unselected package rbd-fuse. 2020-09-08T15:24:35.731 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-scgi (1.13-1.1build1) ... 2020-09-08T15:24:35.736 INFO:teuthology.orchestra.run.smithi166.stdout:Preparing to unpack .../70-rbd-fuse_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:35.749 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking radosgw (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:35.781 INFO:teuthology.orchestra.run.smithi166.stdout:Unpacking rbd-fuse (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:36.067 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-dnspython (1.15.0-1) ... 2020-09-08T15:24:36.101 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package radosgw. 2020-09-08T15:24:36.116 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../69-radosgw_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:36.167 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking radosgw (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:36.294 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package rbd-fuse. 2020-09-08T15:24:36.308 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../70-rbd-fuse_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:36.345 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-simplejson (3.13.2-1) ... 2020-09-08T15:24:36.352 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking rbd-fuse (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:36.579 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-repoze.lru (0.7-1) ... 2020-09-08T15:24:36.680 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-dnspython (1.15.0-1) ... 2020-09-08T15:24:36.720 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package rbd-fuse. 2020-09-08T15:24:36.734 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../70-rbd-fuse_13.2.10-45-g94aa832295-1bionic_amd64.deb ... 2020-09-08T15:24:36.773 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-openid (2.2.5-6) ... 2020-09-08T15:24:36.778 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking rbd-fuse (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:36.925 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-simplejson (3.13.2-1) ... 2020-09-08T15:24:37.024 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-prettytable (0.7.2-3) ... 2020-09-08T15:24:37.106 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-dnspython (1.15.0-1) ... 2020-09-08T15:24:37.151 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-repoze.lru (0.7-1) ... 2020-09-08T15:24:37.226 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-urllib3 (1.22-1ubuntu0.18.04.1) ... 2020-09-08T15:24:37.344 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-openid (2.2.5-6) ... 2020-09-08T15:24:37.460 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-simplegeneric (0.8.1-1) ... 2020-09-08T15:24:37.590 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-prettytable (0.7.2-3) ... 2020-09-08T15:24:37.635 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-simplejson (3.13.2-1) ... 2020-09-08T15:24:37.679 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-chardet (3.0.4-1) ... 2020-09-08T15:24:37.797 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-urllib3 (1.22-1ubuntu0.18.04.1) ... 2020-09-08T15:24:37.886 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-repoze.lru (0.7-1) ... 2020-09-08T15:24:37.962 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up libjs-underscore (1.8.3~dfsg-1) ... 2020-09-08T15:24:38.073 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-simplegeneric (0.8.1-1) ... 2020-09-08T15:24:38.080 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-openid (2.2.5-6) ... 2020-09-08T15:24:38.096 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-openssl (17.5.0-1ubuntu1) ... 2020-09-08T15:24:38.266 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-chardet (3.0.4-1) ... 2020-09-08T15:24:38.306 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-prettytable (0.7.2-3) ... 2020-09-08T15:24:38.331 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-09-08T15:24:38.457 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-09-08T15:24:38.507 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-urllib3 (1.22-1ubuntu0.18.04.1) ... 2020-09-08T15:24:38.571 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libjs-underscore (1.8.3~dfsg-1) ... 2020-09-08T15:24:38.583 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-certifi (2018.1.18-2) ... 2020-09-08T15:24:38.716 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-openssl (17.5.0-1ubuntu1) ... 2020-09-08T15:24:38.725 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-simplegeneric (0.8.1-1) ... 2020-09-08T15:24:38.784 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up formencode-i18n (1.3.0-0ubuntu5) ... 2020-09-08T15:24:38.910 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up libjs-sphinxdoc (1.6.7-1ubuntu1) ... 2020-09-08T15:24:38.926 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-chardet (3.0.4-1) ... 2020-09-08T15:24:38.927 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-09-08T15:24:39.061 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-webob (1:1.7.3-2fakesync1) ... 2020-09-08T15:24:39.220 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libjs-underscore (1.8.3~dfsg-1) ... 2020-09-08T15:24:39.288 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-ceph-argparse (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:39.345 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-openssl (17.5.0-1ubuntu1) ... 2020-09-08T15:24:39.405 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-waitress (1.0.1-1) ... 2020-09-08T15:24:39.412 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-09-08T15:24:39.546 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-certifi (2018.1.18-2) ... 2020-09-08T15:24:39.555 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-09-08T15:24:39.681 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-09-08T15:24:39.753 INFO:teuthology.orchestra.run.smithi166.stdout:update-alternatives: using /usr/bin/waitress-serve-python2 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-09-08T15:24:39.755 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up formencode-i18n (1.3.0-0ubuntu5) ... 2020-09-08T15:24:39.807 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-certifi (2018.1.18-2) ... 2020-09-08T15:24:39.881 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libjs-sphinxdoc (1.6.7-1ubuntu1) ... 2020-09-08T15:24:39.966 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-markupsafe (1.0-1build1) ... 2020-09-08T15:24:39.999 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up formencode-i18n (1.3.0-0ubuntu5) ... 2020-09-08T15:24:40.032 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-webob (1:1.7.3-2fakesync1) ... 2020-09-08T15:24:40.126 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libjs-sphinxdoc (1.6.7-1ubuntu1) ... 2020-09-08T15:24:40.167 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-werkzeug (0.14.1+dfsg1-1) ... 2020-09-08T15:24:40.261 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-ceph-argparse (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:40.277 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-webob (1:1.7.3-2fakesync1) ... 2020-09-08T15:24:40.410 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-waitress (1.0.1-1) ... 2020-09-08T15:24:40.420 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-pyinotify (0.9.6-1) ... 2020-09-08T15:24:40.503 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-ceph-argparse (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:40.629 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-waitress (1.0.1-1) ... 2020-09-08T15:24:40.649 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-scgi (1.13-1.1build1) ... 2020-09-08T15:24:40.768 INFO:teuthology.orchestra.run.smithi104.stdout:update-alternatives: using /usr/bin/waitress-serve-python2 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-09-08T15:24:40.838 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-09-08T15:24:40.963 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-markupsafe (1.0-1build1) ... 2020-09-08T15:24:40.963 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-09-08T15:24:40.964 INFO:teuthology.orchestra.run.smithi055.stdout:update-alternatives: using /usr/bin/waitress-serve-python2 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-09-08T15:24:41.072 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-bs4 (4.6.0-1) ... 2020-09-08T15:24:41.149 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-markupsafe (1.0-1build1) ... 2020-09-08T15:24:41.172 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-werkzeug (0.14.1+dfsg1-1) ... 2020-09-08T15:24:41.290 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-mako (1.0.7+ds1-1) ... 2020-09-08T15:24:41.341 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-werkzeug (0.14.1+dfsg1-1) ... 2020-09-08T15:24:41.440 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-pyinotify (0.9.6-1) ... 2020-09-08T15:24:41.483 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-09-08T15:24:41.610 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-09-08T15:24:41.618 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-pyinotify (0.9.6-1) ... 2020-09-08T15:24:41.666 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-scgi (1.13-1.1build1) ... 2020-09-08T15:24:41.821 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-requests (2.18.4-2ubuntu0.1) ... 2020-09-08T15:24:41.827 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-scgi (1.13-1.1build1) ... 2020-09-08T15:24:41.858 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-09-08T15:24:41.986 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-09-08T15:24:42.020 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-09-08T15:24:42.028 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up liboath0 (2.6.1-1) ... 2020-09-08T15:24:42.135 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-bs4 (4.6.0-1) ... 2020-09-08T15:24:42.146 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-09-08T15:24:42.196 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-singledispatch (3.4.0.3-2) ... 2020-09-08T15:24:42.281 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-bs4 (4.6.0-1) ... 2020-09-08T15:24:42.374 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-mako (1.0.7+ds1-1) ... 2020-09-08T15:24:42.380 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-09-08T15:24:42.489 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-tempita (0.5.2-2) ... 2020-09-08T15:24:42.507 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-mako (1.0.7+ds1-1) ... 2020-09-08T15:24:42.594 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-09-08T15:24:42.682 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-09-08T15:24:42.730 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-09-08T15:24:42.753 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-09-08T15:24:42.800 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-webencodings (0.5-2) ... 2020-09-08T15:24:42.884 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-09-08T15:24:42.948 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-requests (2.18.4-2ubuntu0.1) ... 2020-09-08T15:24:42.992 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-cherrypy3 (8.9.1-2) ... 2020-09-08T15:24:43.111 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-requests (2.18.4-2ubuntu0.1) ... 2020-09-08T15:24:43.158 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up liboath0 (2.6.1-1) ... 2020-09-08T15:24:43.243 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-bcrypt (3.1.4-2) ... 2020-09-08T15:24:43.337 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up liboath0 (2.6.1-1) ... 2020-09-08T15:24:43.359 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-singledispatch (3.4.0.3-2) ... 2020-09-08T15:24:43.462 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-logutils (0.3.3-5) ... 2020-09-08T15:24:43.513 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-singledispatch (3.4.0.3-2) ... 2020-09-08T15:24:43.560 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-09-08T15:24:43.662 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-routes (2.4.1-1) ... 2020-09-08T15:24:43.698 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-09-08T15:24:43.699 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-tempita (0.5.2-2) ... 2020-09-08T15:24:43.825 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-tempita (0.5.2-2) ... 2020-09-08T15:24:43.874 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-formencode (1.3.0-0ubuntu5) ... 2020-09-08T15:24:43.914 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-09-08T15:24:44.026 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-09-08T15:24:44.041 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-webencodings (0.5-2) ... 2020-09-08T15:24:44.090 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up jq (1.5+dfsg-2) ... 2020-09-08T15:24:44.159 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-webencodings (0.5-2) ... 2020-09-08T15:24:44.215 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-09-08T15:24:44.233 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-cherrypy3 (8.9.1-2) ... 2020-09-08T15:24:44.354 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-cherrypy3 (8.9.1-2) ... 2020-09-08T15:24:44.434 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-paste (2.0.3+dfsg-4ubuntu1) ... 2020-09-08T15:24:44.468 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-bcrypt (3.1.4-2) ... 2020-09-08T15:24:44.655 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-bcrypt (3.1.4-2) ... 2020-09-08T15:24:44.661 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-logutils (0.3.3-5) ... 2020-09-08T15:24:44.726 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-09-08T15:24:44.853 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-09-08T15:24:44.854 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-logutils (0.3.3-5) ... 2020-09-08T15:24:44.886 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-routes (2.4.1-1) ... 2020-09-08T15:24:45.057 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-routes (2.4.1-1) ... 2020-09-08T15:24:45.087 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-formencode (1.3.0-0ubuntu5) ... 2020-09-08T15:24:45.252 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up librados2 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:45.257 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-formencode (1.3.0-0ubuntu5) ... 2020-09-08T15:24:45.314 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up jq (1.5+dfsg-2) ... 2020-09-08T15:24:45.371 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up libcephfs2 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:45.440 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-09-08T15:24:45.496 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up libcephfs-dev (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:45.500 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up jq (1.5+dfsg-2) ... 2020-09-08T15:24:45.623 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-html5lib (0.999999999-1) ... 2020-09-08T15:24:45.626 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-09-08T15:24:45.649 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-paste (2.0.3+dfsg-4ubuntu1) ... 2020-09-08T15:24:45.867 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-rados (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:45.869 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-paste (2.0.3+dfsg-4ubuntu1) ... 2020-09-08T15:24:45.901 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-09-08T15:24:45.967 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-cephfs (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:46.035 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-09-08T15:24:46.161 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up librados2 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:46.168 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up libradosstriper1 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:46.170 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-09-08T15:24:46.294 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up librgw2 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:46.296 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-09-08T15:24:46.312 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libcephfs2 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:46.422 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up librados2 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:46.436 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-rgw (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:46.447 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libcephfs-dev (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:46.548 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libcephfs2 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:46.562 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-pastedeploy (1.5.2-4) ... 2020-09-08T15:24:46.573 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-html5lib (0.999999999-1) ... 2020-09-08T15:24:46.674 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libcephfs-dev (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:46.766 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up ceph-fuse (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:46.801 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-html5lib (0.999999999-1) ... 2020-09-08T15:24:46.817 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-rados (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:46.917 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-cephfs (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:46.994 INFO:teuthology.orchestra.run.smithi166.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-09-08T15:24:46.994 INFO:teuthology.orchestra.run.smithi166.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-09-08T15:24:47.053 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-rados (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:47.110 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libradosstriper1 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:47.230 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-cephfs (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:47.230 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up librgw2 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:47.299 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up librbd1 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:47.354 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-rgw (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:47.400 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-pastescript (2.0.2-2) ... 2020-09-08T15:24:47.463 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libradosstriper1 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:47.480 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-pastedeploy (1.5.2-4) ... 2020-09-08T15:24:47.590 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up librgw2 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:47.618 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-rbd (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:47.697 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-fuse (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:47.741 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-rgw (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:47.746 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up ceph-common (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:47.867 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-pastedeploy (1.5.2-4) ... 2020-09-08T15:24:47.934 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-09-08T15:24:47.936 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-09-08T15:24:48.069 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-fuse (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:48.224 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up librbd1 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:48.301 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-09-08T15:24:48.301 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-09-08T15:24:48.326 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-pastescript (2.0.2-2) ... 2020-09-08T15:24:48.345 INFO:teuthology.orchestra.run.smithi166.stdout:Adding group ceph....done 2020-09-08T15:24:48.561 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up librbd1 (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:48.576 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-rbd (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:48.694 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-common (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:48.712 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-pastescript (2.0.2-2) ... 2020-09-08T15:24:48.781 INFO:teuthology.orchestra.run.smithi166.stdout:Adding system user ceph....done 2020-09-08T15:24:48.922 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-rbd (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:48.973 INFO:teuthology.orchestra.run.smithi166.stdout:Setting system user ceph properties....done 2020-09-08T15:24:49.050 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-common (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:49.139 INFO:teuthology.orchestra.run.smithi166.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-09-08T15:24:49.336 INFO:teuthology.orchestra.run.smithi104.stdout:Adding group ceph....done 2020-09-08T15:24:49.371 INFO:teuthology.orchestra.run.smithi166.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-09-08T15:24:49.672 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up radosgw (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:49.691 INFO:teuthology.orchestra.run.smithi055.stdout:Adding group ceph....done 2020-09-08T15:24:49.773 INFO:teuthology.orchestra.run.smithi104.stdout:Adding system user ceph....done 2020-09-08T15:24:49.974 INFO:teuthology.orchestra.run.smithi104.stdout:Setting system user ceph properties....done 2020-09-08T15:24:50.062 INFO:teuthology.orchestra.run.smithi166.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-09-08T15:24:50.063 INFO:teuthology.orchestra.run.smithi166.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-09-08T15:24:50.152 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-09-08T15:24:50.158 INFO:teuthology.orchestra.run.smithi055.stdout:Adding system user ceph....done 2020-09-08T15:24:50.354 INFO:teuthology.orchestra.run.smithi055.stdout:Setting system user ceph properties....done 2020-09-08T15:24:50.362 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-09-08T15:24:50.457 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up rbd-fuse (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:50.511 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-09-08T15:24:50.558 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-webtest (2.0.28-1ubuntu1) ... 2020-09-08T15:24:50.638 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up radosgw (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:50.731 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-09-08T15:24:50.734 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up ceph-test (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:50.868 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up ceph-base (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:51.001 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up radosgw (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:51.044 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-09-08T15:24:51.045 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-09-08T15:24:51.300 INFO:teuthology.orchestra.run.smithi166.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-09-08T15:24:51.398 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-09-08T15:24:51.399 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-09-08T15:24:51.466 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up rbd-fuse (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:51.570 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-webtest (2.0.28-1ubuntu1) ... 2020-09-08T15:24:51.589 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up python-pecan (1.2.1-2) ... 2020-09-08T15:24:51.685 INFO:teuthology.orchestra.run.smithi166.stdout:update-alternatives: using /usr/bin/python2-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-09-08T15:24:51.730 INFO:teuthology.orchestra.run.smithi166.stdout:update-alternatives: using /usr/bin/python2-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-09-08T15:24:51.782 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-test (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:51.788 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up rbd-fuse (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:51.893 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-base (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:51.914 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-webtest (2.0.28-1ubuntu1) ... 2020-09-08T15:24:51.931 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up ceph-osd (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:52.124 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-test (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:52.249 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-base (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:52.336 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-09-08T15:24:52.396 INFO:teuthology.orchestra.run.smithi166.stdout:chown: cannot access '/var/lib/ceph/osd/*/block*': No such file or directory 2020-09-08T15:24:52.626 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-pecan (1.2.1-2) ... 2020-09-08T15:24:52.660 INFO:teuthology.orchestra.run.smithi166.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-09-08T15:24:52.661 INFO:teuthology.orchestra.run.smithi166.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-09-08T15:24:52.699 INFO:teuthology.orchestra.run.smithi104.stdout:update-alternatives: using /usr/bin/python2-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-09-08T15:24:52.709 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-09-08T15:24:52.734 INFO:teuthology.orchestra.run.smithi104.stdout:update-alternatives: using /usr/bin/python2-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-09-08T15:24:52.952 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up ceph-mds (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:52.963 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-osd (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:53.038 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-pecan (1.2.1-2) ... 2020-09-08T15:24:53.142 INFO:teuthology.orchestra.run.smithi055.stdout:update-alternatives: using /usr/bin/python2-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-09-08T15:24:53.182 INFO:teuthology.orchestra.run.smithi055.stdout:update-alternatives: using /usr/bin/python2-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-09-08T15:24:53.334 INFO:teuthology.orchestra.run.smithi166.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-09-08T15:24:53.334 INFO:teuthology.orchestra.run.smithi166.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-09-08T15:24:53.378 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-osd (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:53.422 INFO:teuthology.orchestra.run.smithi104.stdout:chown: cannot access '/var/lib/ceph/osd/*/block*': No such file or directory 2020-09-08T15:24:53.642 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up ceph-mon (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:53.682 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-09-08T15:24:53.683 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-09-08T15:24:53.831 INFO:teuthology.orchestra.run.smithi055.stdout:chown: cannot access '/var/lib/ceph/osd/*/block*': No such file or directory 2020-09-08T15:24:53.942 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mds (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:54.022 INFO:teuthology.orchestra.run.smithi166.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-09-08T15:24:54.023 INFO:teuthology.orchestra.run.smithi166.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-09-08T15:24:54.093 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-09-08T15:24:54.093 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-09-08T15:24:54.286 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-09-08T15:24:54.286 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-09-08T15:24:54.332 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up ceph-mgr (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:54.360 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-mds (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:54.565 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mon (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:54.734 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-09-08T15:24:54.734 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-09-08T15:24:54.843 INFO:teuthology.orchestra.run.smithi166.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-09-08T15:24:54.843 INFO:teuthology.orchestra.run.smithi166.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-09-08T15:24:54.927 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-09-08T15:24:54.927 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-09-08T15:24:55.019 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-mon (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:55.152 INFO:teuthology.orchestra.run.smithi166.stdout:Setting up ceph (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:55.214 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mgr (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:55.295 INFO:teuthology.orchestra.run.smithi166.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-09-08T15:24:55.365 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-09-08T15:24:55.365 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-09-08T15:24:55.479 INFO:teuthology.orchestra.run.smithi166.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-09-08T15:24:55.647 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-mgr (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:55.708 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-09-08T15:24:55.709 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-09-08T15:24:55.992 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:56.093 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-09-08T15:24:56.155 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-09-08T15:24:56.155 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-09-08T15:24:56.294 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-09-08T15:24:56.408 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph (13.2.10-45-g94aa832295-1bionic) ... 2020-09-08T15:24:56.498 INFO:teuthology.orchestra.run.smithi166.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-09-08T15:24:56.535 INFO:teuthology.orchestra.run.smithi055.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-09-08T15:24:56.550 INFO:teuthology.orchestra.run.smithi166.stdout:ureadahead will be reprofiled on next reboot 2020-09-08T15:24:56.598 INFO:teuthology.orchestra.run.smithi166.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.2) ... 2020-09-08T15:24:56.736 INFO:teuthology.orchestra.run.smithi055.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-09-08T15:24:57.068 INFO:teuthology.orchestra.run.smithi166.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-09-08T15:24:57.072 DEBUG:teuthology.parallel:result is None 2020-09-08T15:24:57.341 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-09-08T15:24:57.372 INFO:teuthology.orchestra.run.smithi104.stdout:ureadahead will be reprofiled on next reboot 2020-09-08T15:24:57.431 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.2) ... 2020-09-08T15:24:57.822 INFO:teuthology.orchestra.run.smithi055.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-09-08T15:24:57.873 INFO:teuthology.orchestra.run.smithi055.stdout:ureadahead will be reprofiled on next reboot 2020-09-08T15:24:57.910 INFO:teuthology.orchestra.run.smithi104.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-09-08T15:24:57.912 DEBUG:teuthology.parallel:result is None 2020-09-08T15:24:57.931 INFO:teuthology.orchestra.run.smithi055.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.2) ... 2020-09-08T15:24:58.390 INFO:teuthology.orchestra.run.smithi055.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-09-08T15:24:58.394 DEBUG:teuthology.parallel:result is None 2020-09-08T15:24:58.396 INFO:teuthology.orchestra.run.smithi104:> sudo lsb_release -is 2020-09-08T15:24:58.452 INFO:teuthology.orchestra.run.smithi104.stdout:Ubuntu 2020-09-08T15:24:58.457 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-09-08T15:24:58.458 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-09-08T15:24:58.458 INFO:teuthology.packaging:ref: None 2020-09-08T15:24:58.459 INFO:teuthology.packaging:tag: None 2020-09-08T15:24:58.459 INFO:teuthology.packaging:branch: mimic 2020-09-08T15:24:58.459 INFO:teuthology.packaging:sha1: dce4d291bd4243c9802c17d67b85a6da6bbc0f8f 2020-09-08T15:24:58.460 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=mimic 2020-09-08T15:24:58.714 INFO:teuthology.orchestra.run.smithi104:> dpkg-query -W -f '${Version}' ceph 2020-09-08T15:24:58.750 INFO:teuthology.orchestra.run.smithi104.stdout:13.2.10-45-g94aa832295-1bionic 2020-09-08T15:24:58.752 INFO:teuthology.packaging:The installed version of ceph is 13.2.10-45-g94aa832295-1bionic 2020-09-08T15:24:58.752 INFO:teuthology.task.install:The correct ceph version 13.2.10-45-g94aa832295-1bionic is installed. 2020-09-08T15:24:58.753 INFO:teuthology.orchestra.run.smithi055:> sudo lsb_release -is 2020-09-08T15:24:58.807 INFO:teuthology.orchestra.run.smithi055.stdout:Ubuntu 2020-09-08T15:24:58.814 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-09-08T15:24:58.814 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-09-08T15:24:58.815 INFO:teuthology.packaging:ref: None 2020-09-08T15:24:58.816 INFO:teuthology.packaging:tag: None 2020-09-08T15:24:58.816 INFO:teuthology.packaging:branch: mimic 2020-09-08T15:24:58.817 INFO:teuthology.packaging:sha1: dce4d291bd4243c9802c17d67b85a6da6bbc0f8f 2020-09-08T15:24:58.818 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=mimic 2020-09-08T15:24:59.085 INFO:teuthology.orchestra.run.smithi055:> dpkg-query -W -f '${Version}' ceph 2020-09-08T15:24:59.121 INFO:teuthology.orchestra.run.smithi055.stdout:13.2.10-45-g94aa832295-1bionic 2020-09-08T15:24:59.121 INFO:teuthology.packaging:The installed version of ceph is 13.2.10-45-g94aa832295-1bionic 2020-09-08T15:24:59.121 INFO:teuthology.task.install:The correct ceph version 13.2.10-45-g94aa832295-1bionic is installed. 2020-09-08T15:24:59.122 INFO:teuthology.orchestra.run.smithi166:> sudo lsb_release -is 2020-09-08T15:24:59.177 INFO:teuthology.orchestra.run.smithi166.stdout:Ubuntu 2020-09-08T15:24:59.181 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-09-08T15:24:59.183 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-09-08T15:24:59.183 INFO:teuthology.packaging:ref: None 2020-09-08T15:24:59.184 INFO:teuthology.packaging:tag: None 2020-09-08T15:24:59.184 INFO:teuthology.packaging:branch: mimic 2020-09-08T15:24:59.184 INFO:teuthology.packaging:sha1: dce4d291bd4243c9802c17d67b85a6da6bbc0f8f 2020-09-08T15:24:59.185 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=mimic 2020-09-08T15:24:59.451 INFO:teuthology.orchestra.run.smithi166:> dpkg-query -W -f '${Version}' ceph 2020-09-08T15:24:59.477 INFO:teuthology.orchestra.run.smithi166.stdout:13.2.10-45-g94aa832295-1bionic 2020-09-08T15:24:59.478 INFO:teuthology.packaging:The installed version of ceph is 13.2.10-45-g94aa832295-1bionic 2020-09-08T15:24:59.478 INFO:teuthology.task.install:The correct ceph version 13.2.10-45-g94aa832295-1bionic is installed. 2020-09-08T15:24:59.479 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2020-09-08T15:24:59.479 INFO:teuthology.orchestra.run.smithi104:> set -ex 2020-09-08T15:24:59.480 INFO:teuthology.orchestra.run.smithi104:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-09-08T15:24:59.488 INFO:teuthology.orchestra.run.smithi104.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-09-08T15:24:59.496 INFO:teuthology.orchestra.run.smithi104.stderr:24+1 records in 2020-09-08T15:24:59.496 INFO:teuthology.orchestra.run.smithi104.stderr:24+1 records out 2020-09-08T15:24:59.497 INFO:teuthology.orchestra.run.smithi104.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000218617 s, 57.9 MB/s 2020-09-08T15:24:59.498 INFO:teuthology.orchestra.run.smithi055:> set -ex 2020-09-08T15:24:59.498 INFO:teuthology.orchestra.run.smithi055:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-09-08T15:24:59.505 INFO:teuthology.orchestra.run.smithi055.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-09-08T15:24:59.514 INFO:teuthology.orchestra.run.smithi055.stderr:24+1 records in 2020-09-08T15:24:59.514 INFO:teuthology.orchestra.run.smithi055.stderr:24+1 records out 2020-09-08T15:24:59.515 INFO:teuthology.orchestra.run.smithi055.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000220367 s, 57.5 MB/s 2020-09-08T15:24:59.516 INFO:teuthology.orchestra.run.smithi166:> set -ex 2020-09-08T15:24:59.516 INFO:teuthology.orchestra.run.smithi166:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-09-08T15:24:59.522 INFO:teuthology.orchestra.run.smithi166.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-09-08T15:24:59.530 INFO:teuthology.orchestra.run.smithi166.stderr:24+1 records in 2020-09-08T15:24:59.533 INFO:teuthology.orchestra.run.smithi166.stderr:24+1 records out 2020-09-08T15:24:59.534 INFO:teuthology.orchestra.run.smithi166.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000213244 s, 59.4 MB/s 2020-09-08T15:24:59.542 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2020-09-08T15:24:59.542 INFO:teuthology.orchestra.run.smithi104:> set -ex 2020-09-08T15:24:59.543 INFO:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/daemon-helper 2020-09-08T15:24:59.552 INFO:teuthology.orchestra.run.smithi104.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-09-08T15:24:59.557 INFO:teuthology.orchestra.run.smithi104.stderr:5+1 records in 2020-09-08T15:24:59.558 INFO:teuthology.orchestra.run.smithi104.stderr:5+1 records out 2020-09-08T15:24:59.558 INFO:teuthology.orchestra.run.smithi104.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 9.6996e-05 s, 31.5 MB/s 2020-09-08T15:24:59.558 INFO:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-09-08T15:24:59.621 INFO:teuthology.orchestra.run.smithi055:> set -ex 2020-09-08T15:24:59.622 INFO:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/daemon-helper 2020-09-08T15:24:59.634 INFO:teuthology.orchestra.run.smithi055.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-09-08T15:24:59.645 INFO:teuthology.orchestra.run.smithi055.stderr:5+1 records in 2020-09-08T15:24:59.645 INFO:teuthology.orchestra.run.smithi055.stderr:5+1 records out 2020-09-08T15:24:59.646 INFO:teuthology.orchestra.run.smithi055.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 8.1848e-05 s, 37.4 MB/s 2020-09-08T15:24:59.646 INFO:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-09-08T15:24:59.706 INFO:teuthology.orchestra.run.smithi166:> set -ex 2020-09-08T15:24:59.707 INFO:teuthology.orchestra.run.smithi166:> sudo dd of=/usr/bin/daemon-helper 2020-09-08T15:24:59.718 INFO:teuthology.orchestra.run.smithi166.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-09-08T15:24:59.726 INFO:teuthology.orchestra.run.smithi166.stderr:5+1 records in 2020-09-08T15:24:59.726 INFO:teuthology.orchestra.run.smithi166.stderr:5+1 records out 2020-09-08T15:24:59.726 INFO:teuthology.orchestra.run.smithi166.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 0.00010094 s, 30.3 MB/s 2020-09-08T15:24:59.727 INFO:teuthology.orchestra.run.smithi166:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-09-08T15:24:59.784 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2020-09-08T15:24:59.785 INFO:teuthology.orchestra.run.smithi104:> set -ex 2020-09-08T15:24:59.785 INFO:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/adjust-ulimits 2020-09-08T15:24:59.790 INFO:teuthology.orchestra.run.smithi104.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-09-08T15:24:59.799 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records in 2020-09-08T15:24:59.799 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records out 2020-09-08T15:24:59.799 INFO:teuthology.orchestra.run.smithi104.stderr:299 bytes copied, 7.1433e-05 s, 4.2 MB/s 2020-09-08T15:24:59.800 INFO:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-09-08T15:24:59.857 INFO:teuthology.orchestra.run.smithi055:> set -ex 2020-09-08T15:24:59.858 INFO:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/adjust-ulimits 2020-09-08T15:24:59.864 INFO:teuthology.orchestra.run.smithi055.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-09-08T15:24:59.873 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records in 2020-09-08T15:24:59.874 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records out 2020-09-08T15:24:59.874 INFO:teuthology.orchestra.run.smithi055.stderr:299 bytes copied, 7.19e-05 s, 4.2 MB/s 2020-09-08T15:24:59.876 INFO:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-09-08T15:24:59.930 INFO:teuthology.orchestra.run.smithi166:> set -ex 2020-09-08T15:24:59.931 INFO:teuthology.orchestra.run.smithi166:> sudo dd of=/usr/bin/adjust-ulimits 2020-09-08T15:24:59.937 INFO:teuthology.orchestra.run.smithi166.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-09-08T15:24:59.950 INFO:teuthology.orchestra.run.smithi166.stderr:0+1 records in 2020-09-08T15:24:59.950 INFO:teuthology.orchestra.run.smithi166.stderr:0+1 records out 2020-09-08T15:24:59.951 INFO:teuthology.orchestra.run.smithi166.stderr:299 bytes copied, 7.0477e-05 s, 4.2 MB/s 2020-09-08T15:24:59.951 INFO:teuthology.orchestra.run.smithi166:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-09-08T15:25:00.012 INFO:teuthology.run_tasks:Running task cephadm... 2020-09-08T15:25:00.104 INFO:tasks.cephadm:Config: {'conf': {'client.2': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'mon client directed command retry': 5, 'mon warn on msgr2 not enabled': False, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'ms type': 'async', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', '\\(MON_DOWN\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'mon_bind_msgr2': False, 'sha1': 'dce4d291bd4243c9802c17d67b85a6da6bbc0f8f'} 2020-09-08T15:25:00.105 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f 2020-09-08T15:25:00.106 INFO:tasks.cephadm:Cluster fsid is 759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:25:00.106 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2020-09-08T15:25:00.107 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '[v1:172.21.15.104:6789]', 'mon.b': '[v1:172.21.15.55:6789]', 'mon.c': '[v1:172.21.15.166:6789]'} 2020-09-08T15:25:00.107 INFO:tasks.cephadm:First mon is mon.a on smithi104 2020-09-08T15:25:00.108 INFO:tasks.cephadm:First mgr is y 2020-09-08T15:25:00.108 INFO:tasks.cephadm:Normalizing hostnames... 2020-09-08T15:25:00.108 INFO:teuthology.orchestra.run.smithi055:> sudo hostname $(hostname -s) 2020-09-08T15:25:00.125 INFO:teuthology.orchestra.run.smithi104:> sudo hostname $(hostname -s) 2020-09-08T15:25:00.140 INFO:teuthology.orchestra.run.smithi166:> sudo hostname $(hostname -s) 2020-09-08T15:25:00.156 INFO:tasks.cephadm:Downloading cephadm (repo git://git.ceph.com/ceph-ci.git ref dce4d291bd4243c9802c17d67b85a6da6bbc0f8f)... 2020-09-08T15:25:00.157 INFO:teuthology.orchestra.run.smithi055:> git archive --remote=git://git.ceph.com/ceph-ci.git dce4d291bd4243c9802c17d67b85a6da6bbc0f8f src/cephadm/cephadm | tar -xO src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm 2020-09-08T15:25:00.285 INFO:teuthology.orchestra.run.smithi104:> git archive --remote=git://git.ceph.com/ceph-ci.git dce4d291bd4243c9802c17d67b85a6da6bbc0f8f src/cephadm/cephadm | tar -xO src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm 2020-09-08T15:25:00.392 INFO:teuthology.orchestra.run.smithi166:> git archive --remote=git://git.ceph.com/ceph-ci.git dce4d291bd4243c9802c17d67b85a6da6bbc0f8f src/cephadm/cephadm | tar -xO src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm 2020-09-08T15:25:00.495 INFO:teuthology.orchestra.run.smithi055:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2020-09-08T15:25:00.505 INFO:teuthology.orchestra.run.smithi104:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2020-09-08T15:25:00.514 INFO:teuthology.orchestra.run.smithi166:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2020-09-08T15:25:00.572 INFO:teuthology.orchestra.run.smithi055:> sudo mkdir -p /etc/ceph 2020-09-08T15:25:00.587 INFO:teuthology.orchestra.run.smithi104:> sudo mkdir -p /etc/ceph 2020-09-08T15:25:00.603 INFO:teuthology.orchestra.run.smithi166:> sudo mkdir -p /etc/ceph 2020-09-08T15:25:00.617 INFO:teuthology.orchestra.run.smithi055:> sudo chmod 777 /etc/ceph 2020-09-08T15:25:00.649 INFO:teuthology.orchestra.run.smithi104:> sudo chmod 777 /etc/ceph 2020-09-08T15:25:00.664 INFO:teuthology.orchestra.run.smithi166:> sudo chmod 777 /etc/ceph 2020-09-08T15:25:00.682 INFO:tasks.cephadm:Adding local image mirror vossi04.front.sepia.ceph.com:5000 2020-09-08T15:25:00.682 INFO:teuthology.orchestra.run.smithi104:> set -ex 2020-09-08T15:25:00.682 INFO:teuthology.orchestra.run.smithi104:> dd if=/etc/containers/registries.conf of=/dev/stdout 2020-09-08T15:25:00.710 INFO:teuthology.orchestra.run.smithi104.stderr:+ dd if=/etc/containers/registries.conf of=/dev/stdout 2020-09-08T15:25:00.712 INFO:teuthology.orchestra.run.smithi104.stderr:dd: failed to open '/etc/containers/registries.conf': No such file or directory 2020-09-08T15:25:00.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-09-08T15:25:00.712 INFO:tasks.cephadm:Failed to add mirror: [Errno 2] Cannot find file on the remote 'ubuntu@smithi104.front.sepia.ceph.com': '/etc/containers/registries.conf' 2020-09-08T15:25:00.713 INFO:teuthology.orchestra.run.smithi055:> set -ex 2020-09-08T15:25:00.713 INFO:teuthology.orchestra.run.smithi055:> dd if=/etc/containers/registries.conf of=/dev/stdout 2020-09-08T15:25:00.718 INFO:teuthology.orchestra.run.smithi055.stderr:+ dd if=/etc/containers/registries.conf of=/dev/stdout 2020-09-08T15:25:00.720 INFO:teuthology.orchestra.run.smithi055.stderr:dd: failed to open '/etc/containers/registries.conf': No such file or directory 2020-09-08T15:25:00.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-09-08T15:25:00.720 INFO:tasks.cephadm:Failed to add mirror: [Errno 2] Cannot find file on the remote 'ubuntu@smithi055.front.sepia.ceph.com': '/etc/containers/registries.conf' 2020-09-08T15:25:00.721 INFO:teuthology.orchestra.run.smithi166:> set -ex 2020-09-08T15:25:00.721 INFO:teuthology.orchestra.run.smithi166:> dd if=/etc/containers/registries.conf of=/dev/stdout 2020-09-08T15:25:00.726 INFO:teuthology.orchestra.run.smithi166.stderr:+ dd if=/etc/containers/registries.conf of=/dev/stdout 2020-09-08T15:25:00.727 INFO:teuthology.orchestra.run.smithi166.stderr:dd: failed to open '/etc/containers/registries.conf': No such file or directory 2020-09-08T15:25:00.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-09-08T15:25:00.728 INFO:tasks.cephadm:Failed to add mirror: [Errno 2] Cannot find file on the remote 'ubuntu@smithi166.front.sepia.ceph.com': '/etc/containers/registries.conf' 2020-09-08T15:25:00.728 INFO:tasks.cephadm:Writing seed config... 2020-09-08T15:25:00.730 INFO:tasks.cephadm: override: [client.2] debug ms = 1 2020-09-08T15:25:00.730 INFO:tasks.cephadm: override: [client.2] debug objecter = 20 2020-09-08T15:25:00.730 INFO:tasks.cephadm: override: [client.2] debug rados = 20 2020-09-08T15:25:00.731 INFO:tasks.cephadm: override: [global] mon client directed command retry = 5 2020-09-08T15:25:00.731 INFO:tasks.cephadm: override: [global] mon warn on msgr2 not enabled = False 2020-09-08T15:25:00.731 INFO:tasks.cephadm: override: [global] ms inject socket failures = 2500 2020-09-08T15:25:00.731 INFO:tasks.cephadm: override: [global] ms tcp read timeout = 5 2020-09-08T15:25:00.732 INFO:tasks.cephadm: override: [global] ms type = async 2020-09-08T15:25:00.732 INFO:tasks.cephadm: override: [global] osd_pool_default_min_size = 2 2020-09-08T15:25:00.732 INFO:tasks.cephadm: override: [global] osd_pool_default_size = 2 2020-09-08T15:25:00.733 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2020-09-08T15:25:00.733 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2020-09-08T15:25:00.733 INFO:tasks.cephadm: override: [mon] debug mon = 20 2020-09-08T15:25:00.734 INFO:tasks.cephadm: override: [mon] debug ms = 1 2020-09-08T15:25:00.734 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2020-09-08T15:25:00.734 INFO:tasks.cephadm: override: [mon] mon min osdmap epochs = 50 2020-09-08T15:25:00.734 INFO:tasks.cephadm: override: [mon] mon osdmap full prune interval = 2 2020-09-08T15:25:00.735 INFO:tasks.cephadm: override: [mon] mon osdmap full prune min = 15 2020-09-08T15:25:00.735 INFO:tasks.cephadm: override: [mon] mon osdmap full prune txsize = 2 2020-09-08T15:25:00.735 INFO:tasks.cephadm: override: [mon] paxos service trim min = 10 2020-09-08T15:25:00.736 INFO:tasks.cephadm: override: [osd] debug filestore = 20 2020-09-08T15:25:00.736 INFO:tasks.cephadm: override: [osd] debug journal = 20 2020-09-08T15:25:00.736 INFO:tasks.cephadm: override: [osd] debug ms = 20 2020-09-08T15:25:00.737 INFO:tasks.cephadm: override: [osd] debug osd = 25 2020-09-08T15:25:00.737 INFO:tasks.cephadm: override: [osd] osd backoff on degraded = True 2020-09-08T15:25:00.737 INFO:tasks.cephadm: override: [osd] osd backoff on peering = True 2020-09-08T15:25:00.737 INFO:tasks.cephadm: override: [osd] osd debug reject backfill probability = 0.3 2020-09-08T15:25:00.738 INFO:tasks.cephadm: override: [osd] osd debug verify cached snaps = True 2020-09-08T15:25:00.738 INFO:tasks.cephadm: override: [osd] osd debug verify missing on start = True 2020-09-08T15:25:00.738 INFO:tasks.cephadm: override: [osd] osd max backfills = 3 2020-09-08T15:25:00.739 INFO:tasks.cephadm: override: [osd] osd max markdown count = 1000 2020-09-08T15:25:00.739 INFO:tasks.cephadm: override: [osd] osd op queue = debug_random 2020-09-08T15:25:00.739 INFO:tasks.cephadm: override: [osd] osd op queue cut off = debug_random 2020-09-08T15:25:00.739 INFO:tasks.cephadm: override: [osd] osd scrub max interval = 120 2020-09-08T15:25:00.740 INFO:tasks.cephadm: override: [osd] osd scrub min interval = 60 2020-09-08T15:25:00.740 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2020-09-08T15:25:00.740 INFO:tasks.cephadm: override: [osd] osd snap trim sleep = 2 2020-09-08T15:25:00.741 INFO:teuthology.orchestra.run.smithi104:> set -ex 2020-09-08T15:25:00.742 INFO:teuthology.orchestra.run.smithi104:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2020-09-08T15:25:00.766 INFO:teuthology.orchestra.run.smithi104.stderr:+ dd of=/home/ubuntu/cephtest/seed.ceph.conf 2020-09-08T15:25:00.766 INFO:teuthology.orchestra.run.smithi104.stderr:5+1 records in 2020-09-08T15:25:00.767 INFO:teuthology.orchestra.run.smithi104.stderr:5+1 records out 2020-09-08T15:25:00.767 INFO:teuthology.orchestra.run.smithi104.stderr:2867 bytes (2.9 kB, 2.8 KiB) copied, 0.000107027 s, 26.8 MB/s 2020-09-08T15:25:00.767 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 759b27b6-f1e7-11ea-a080-001a4aab830c mon client directed command retry = 5 mon warn on msgr2 not enabled = False ms inject socket failures = 2500 ms tcp read timeout = 5 ms type = async osd_pool_default_min_size = 2 osd_pool_default_size = 2 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 120 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = True osd debug verify missing on start = True osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug filestore = 20 debug journal = 20 debug ms = 20 debug osd = 25 osd backoff on degraded = True osd backoff on peering = True osd debug reject backfill probability = 0.3 osd max backfills = 3 osd max markdown count = 1000 osd scrub min interval = 60 osd snap trim sleep = 2 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 debug mon = 20 debug ms = 1 debug paxos = 20 mon min osdmap epochs = 50 mon osdmap full prune interval = 2 mon osdmap full prune min = 15 mon osdmap full prune txsize = 2 paxos service trim min = 10 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client.2] debug ms = 1 debug objecter = 20 debug rados = 20 2020-09-08T15:25:00.768 INFO:teuthology.orchestra.run.smithi104:mon.a> sudo journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mon.a.service 2020-09-08T15:25:00.808 INFO:teuthology.orchestra.run.smithi104:mgr.y> sudo journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mgr.y.service 2020-09-08T15:25:00.853 INFO:tasks.cephadm:Bootstrapping... 2020-09-08T15:25:00.854 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -v bootstrap --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-addrv '[v1:172.21.15.104:6789]' && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2020-09-08T15:25:00.863 INFO:journalctl@ceph.mon.a.smithi104.stdout:-- Logs begin at Tue 2020-09-01 16:49:30 UTC. -- 2020-09-08T15:25:00.894 INFO:journalctl@ceph.mgr.y.smithi104.stdout:-- Logs begin at Tue 2020-09-01 16:49:30 UTC. -- 2020-09-08T15:25:01.160 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Could not locate podman: podman not found 2020-09-08T15:25:01.161 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Verifying podman|docker is present... 2020-09-08T15:25:01.161 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Verifying lvm2 is present... 2020-09-08T15:25:01.161 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Verifying time synchronization is in place... 2020-09-08T15:25:01.161 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl is-enabled chrony.service 2020-09-08T15:25:01.166 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stderr Failed to get unit file state for chrony.service: No such file or directory 2020-09-08T15:25:01.172 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.004778623580932617, stop=False, exit=None, reads=[5] 2020-09-08T15:25:01.173 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.004914283752441406, stop=False, exit=None, reads=[3, 5] 2020-09-08T15:25:01.175 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.004987239837646484, stop=True, exit=1, reads=[3, 5] 2020-09-08T15:25:01.175 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl is-active chrony.service 2020-09-08T15:25:01.176 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stdout inactive 2020-09-08T15:25:01.177 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005753278732299805, stop=False, exit=3, reads=[3] 2020-09-08T15:25:01.177 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005866527557373047, stop=True, exit=3, reads=[3] 2020-09-08T15:25:01.177 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl is-enabled chronyd.service 2020-09-08T15:25:01.182 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stderr Failed to get unit file state for chronyd.service: No such file or directory 2020-09-08T15:25:01.182 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.006277799606323242, stop=False, exit=1, reads=[5] 2020-09-08T15:25:01.183 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.0064923763275146484, stop=True, exit=1, reads=[5] 2020-09-08T15:25:01.183 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl is-active chronyd.service 2020-09-08T15:25:01.189 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stdout inactive 2020-09-08T15:25:01.327 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005562305450439453, stop=False, exit=3, reads=[3] 2020-09-08T15:25:01.327 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.0057752132415771484, stop=True, exit=3, reads=[3] 2020-09-08T15:25:01.327 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl is-enabled systemd-timesyncd.service 2020-09-08T15:25:01.348 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stdout enabled 2020-09-08T15:25:01.349 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.004834175109863281, stop=False, exit=None, reads=[3] 2020-09-08T15:25:01.349 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.0049784183502197266, stop=True, exit=0, reads=[3] 2020-09-08T15:25:01.350 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl is-active systemd-timesyncd.service 2020-09-08T15:25:01.350 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stdout inactive 2020-09-08T15:25:01.350 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.0051343441009521484, stop=False, exit=3, reads=[3] 2020-09-08T15:25:01.350 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005323648452758789, stop=True, exit=3, reads=[3] 2020-09-08T15:25:01.350 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl is-enabled ntpd.service 2020-09-08T15:25:01.351 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stderr Failed to get unit file state for ntpd.service: No such file or directory 2020-09-08T15:25:01.351 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005375862121582031, stop=False, exit=1, reads=[5] 2020-09-08T15:25:01.351 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.00554656982421875, stop=True, exit=1, reads=[5] 2020-09-08T15:25:01.351 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl is-active ntpd.service 2020-09-08T15:25:01.351 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stdout inactive 2020-09-08T15:25:01.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005162954330444336, stop=False, exit=3, reads=[3] 2020-09-08T15:25:01.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005354404449462891, stop=True, exit=3, reads=[3] 2020-09-08T15:25:01.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl is-enabled ntp.service 2020-09-08T15:25:01.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stdout enabled 2020-09-08T15:25:01.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.006770133972167969, stop=False, exit=0, reads=[3] 2020-09-08T15:25:01.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.006957530975341797, stop=True, exit=0, reads=[3] 2020-09-08T15:25:01.353 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl is-active ntp.service 2020-09-08T15:25:01.353 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stdout active 2020-09-08T15:25:01.353 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.004808664321899414, stop=False, exit=0, reads=[3] 2020-09-08T15:25:01.353 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.004987955093383789, stop=True, exit=0, reads=[3] 2020-09-08T15:25:01.353 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Unit ntp.service is enabled and running 2020-09-08T15:25:01.354 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Repeating the final host check... 2020-09-08T15:25:01.354 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Could not locate podman: podman not found 2020-09-08T15:25:01.354 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:podman|docker (/usr/bin/docker) is present 2020-09-08T15:25:01.354 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:systemctl is present 2020-09-08T15:25:01.354 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:lvcreate is present 2020-09-08T15:25:01.355 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl is-enabled chrony.service 2020-09-08T15:25:01.355 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stderr Failed to get unit file state for chrony.service: No such file or directory 2020-09-08T15:25:01.355 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.0052258968353271484, stop=False, exit=None, reads=[5] 2020-09-08T15:25:01.355 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.00540614128112793, stop=True, exit=1, reads=[5] 2020-09-08T15:25:01.355 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl is-active chrony.service 2020-09-08T15:25:01.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stdout inactive 2020-09-08T15:25:01.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005388498306274414, stop=False, exit=3, reads=[3] 2020-09-08T15:25:01.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005578756332397461, stop=True, exit=3, reads=[3] 2020-09-08T15:25:01.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl is-enabled chronyd.service 2020-09-08T15:25:01.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stderr Failed to get unit file state for chronyd.service: No such file or directory 2020-09-08T15:25:01.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005214691162109375, stop=False, exit=1, reads=[5] 2020-09-08T15:25:01.357 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005406618118286133, stop=True, exit=1, reads=[5] 2020-09-08T15:25:01.357 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl is-active chronyd.service 2020-09-08T15:25:01.357 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stdout inactive 2020-09-08T15:25:01.357 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005294084548950195, stop=False, exit=3, reads=[3] 2020-09-08T15:25:01.357 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005463838577270508, stop=True, exit=3, reads=[3] 2020-09-08T15:25:01.358 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl is-enabled systemd-timesyncd.service 2020-09-08T15:25:01.358 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stdout enabled 2020-09-08T15:25:01.358 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.006848335266113281, stop=False, exit=0, reads=[3] 2020-09-08T15:25:01.358 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.007081747055053711, stop=True, exit=0, reads=[3] 2020-09-08T15:25:01.358 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl is-active systemd-timesyncd.service 2020-09-08T15:25:01.359 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stdout inactive 2020-09-08T15:25:01.359 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005769968032836914, stop=False, exit=3, reads=[3] 2020-09-08T15:25:01.359 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005930662155151367, stop=True, exit=3, reads=[3] 2020-09-08T15:25:01.359 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl is-enabled ntpd.service 2020-09-08T15:25:01.359 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stderr Failed to get unit file state for ntpd.service: No such file or directory 2020-09-08T15:25:01.359 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.004887819290161133, stop=False, exit=1, reads=[5] 2020-09-08T15:25:01.360 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005038738250732422, stop=True, exit=1, reads=[5] 2020-09-08T15:25:01.360 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl is-active ntpd.service 2020-09-08T15:25:01.360 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stdout inactive 2020-09-08T15:25:01.360 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005404949188232422, stop=False, exit=3, reads=[3] 2020-09-08T15:25:01.360 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.0055828094482421875, stop=True, exit=3, reads=[3] 2020-09-08T15:25:01.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl is-enabled ntp.service 2020-09-08T15:25:01.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stdout enabled 2020-09-08T15:25:01.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005121707916259766, stop=False, exit=0, reads=[3] 2020-09-08T15:25:01.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005297660827636719, stop=True, exit=0, reads=[3] 2020-09-08T15:25:01.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl is-active ntp.service 2020-09-08T15:25:01.362 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stdout active 2020-09-08T15:25:01.362 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.004511594772338867, stop=False, exit=0, reads=[3] 2020-09-08T15:25:01.362 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.004689693450927734, stop=True, exit=0, reads=[3] 2020-09-08T15:25:01.362 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Unit ntp.service is enabled and running 2020-09-08T15:25:01.362 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Host looks OK 2020-09-08T15:25:01.363 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:root:Cluster fsid: 759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:25:01.363 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Acquiring lock 140001992038104 on /run/cephadm/759b27b6-f1e7-11ea-a080-001a4aab830c.lock 2020-09-08T15:25:01.363 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Lock 140001992038104 acquired on /run/cephadm/759b27b6-f1e7-11ea-a080-001a4aab830c.lock 2020-09-08T15:25:01.363 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Verifying IP 172.21.15.104 port 6789 ... 2020-09-08T15:25:01.363 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Base mon IP is 172.21.15.104, final addrv is [v1:172.21.15.104:6789] 2020-09-08T15:25:01.363 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /sbin/ip route ls 2020-09-08T15:25:01.364 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/sbin/ip:stdout default via 172.21.15.254 dev enp3s0f1 2020-09-08T15:25:01.364 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/sbin/ip:stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.104 2020-09-08T15:25:01.364 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/sbin/ip:profile rt=0.0011975765228271484, stop=False, exit=0, reads=[4] 2020-09-08T15:25:01.364 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/sbin/ip:profile rt=0.0013356208801269531, stop=True, exit=0, reads=[4] 2020-09-08T15:25:01.364 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /sbin/ip -6 route ls 2020-09-08T15:25:01.365 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/sbin/ip:stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2020-09-08T15:25:01.365 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/sbin/ip:stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1799sec hoplimit 64 pref medium 2020-09-08T15:25:01.365 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/sbin/ip:profile rt=0.0011456012725830078, stop=False, exit=0, reads=[4] 2020-09-08T15:25:01.365 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/sbin/ip:profile rt=0.001287221908569336, stop=True, exit=0, reads=[4] 2020-09-08T15:25:01.365 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /sbin/ip -6 addr ls 2020-09-08T15:25:01.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/sbin/ip:stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2020-09-08T15:25:01.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/sbin/ip:stdout inet6 ::1/128 scope host 2020-09-08T15:25:01.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/sbin/ip:stdout valid_lft forever preferred_lft forever 2020-09-08T15:25:01.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/sbin/ip:stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2020-09-08T15:25:01.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/sbin/ip:stdout inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2020-09-08T15:25:01.367 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/sbin/ip:stdout valid_lft forever preferred_lft forever 2020-09-08T15:25:01.367 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/sbin/ip:profile rt=0.0013225078582763672, stop=False, exit=0, reads=[4] 2020-09-08T15:25:01.367 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/sbin/ip:profile rt=0.0014526844024658203, stop=True, exit=0, reads=[4] 2020-09-08T15:25:01.367 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Mon IP 172.21.15.104 is in CIDR network 172.21.0.0/20 2020-09-08T15:25:01.367 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Pulling container image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f... 2020-09-08T15:25:01.367 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f 2020-09-08T15:25:06.251 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:stdout dce4d291bd4243c9802c17d67b85a6da6bbc0f8f: Pulling from ceph-ci/ceph 2020-09-08T15:25:06.251 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:profile rt=4.918435096740723, stop=False, exit=None, reads=[4] 2020-09-08T15:25:06.252 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:stdout d71c5eab96b2: Pulling fs layer 2020-09-08T15:25:06.252 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:stdout fb32b57d33bc: Pulling fs layer 2020-09-08T15:25:06.252 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:profile rt=4.918846607208252, stop=False, exit=None, reads=[4] 2020-09-08T15:25:06.921 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:stdout d71c5eab96b2: Verifying Checksum 2020-09-08T15:25:06.921 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:stdout d71c5eab96b2: Download complete 2020-09-08T15:25:06.922 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:profile rt=5.584994554519653, stop=False, exit=None, reads=[4] 2020-09-08T15:25:09.363 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:stdout fb32b57d33bc: Verifying Checksum 2020-09-08T15:25:09.364 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:stdout fb32b57d33bc: Download complete 2020-09-08T15:25:09.364 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:profile rt=8.032106161117554, stop=False, exit=None, reads=[4] 2020-09-08T15:25:09.541 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:stdout d71c5eab96b2: Pull complete 2020-09-08T15:25:09.541 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:profile rt=8.20934534072876, stop=False, exit=None, reads=[4] 2020-09-08T15:25:21.021 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:stdout fb32b57d33bc: Pull complete 2020-09-08T15:25:21.022 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:profile rt=19.688846349716187, stop=False, exit=None, reads=[4] 2020-09-08T15:25:21.097 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:stdout Digest: sha256:ba21ddc7c086e32f7ab2e6009923bb329cbd985d9f87c14ef1db1f9fd9565a12 2020-09-08T15:25:21.097 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:profile rt=19.764520168304443, stop=False, exit=None, reads=[4] 2020-09-08T15:25:21.171 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:stdout Status: Downloaded newer image for quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f 2020-09-08T15:25:21.172 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:profile rt=19.839786529541016, stop=False, exit=None, reads=[4] 2020-09-08T15:25:21.172 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:stdout quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f 2020-09-08T15:25:21.172 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:profile rt=19.840418577194214, stop=False, exit=None, reads=[4] 2020-09-08T15:25:21.175 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:profile rt=19.844019889831543, stop=False, exit=0, reads=[4, 6] 2020-09-08T15:25:21.175 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/docker:profile rt=19.844228267669678, stop=True, exit=0, reads=[4, 6] 2020-09-08T15:25:21.176 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Extracting ceph user uid/gid from container image... 2020-09-08T15:25:21.176 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '--entrypoint', 'stat', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-c', '%u %g', '/var/lib/ceph'] 2020-09-08T15:25:21.176 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 --entrypoint stat quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -c %u %g /var/lib/ceph 2020-09-08T15:25:31.199 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:stat:stdout 167 167 2020-09-08T15:25:31.199 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:stat:profile rt=10.019196033477783, stop=False, exit=None, reads=[4] 2020-09-08T15:25:31.789 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:stat:profile rt=10.611318349838257, stop=False, exit=0, reads=[4, 6] 2020-09-08T15:25:31.790 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:stat:profile rt=10.611592531204224, stop=True, exit=0, reads=[4, 6] 2020-09-08T15:25:31.790 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Creating initial keys... 2020-09-08T15:25:31.790 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '--entrypoint', '/usr/bin/ceph-authtool', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '--gen-print-key'] 2020-09-08T15:25:31.791 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 --entrypoint /usr/bin/ceph-authtool quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f --gen-print-key 2020-09-08T15:25:32.819 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-authtool:stdout AQBsoldfwGjlMBAAXUPss7n5JOSeSfSGyEPOOw== 2020-09-08T15:25:32.819 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-authtool:profile rt=1.0233573913574219, stop=False, exit=None, reads=[4] 2020-09-08T15:25:33.349 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-authtool:profile rt=1.5569708347320557, stop=False, exit=0, reads=[4, 6] 2020-09-08T15:25:33.350 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-authtool:profile rt=1.5572564601898193, stop=True, exit=0, reads=[4, 6] 2020-09-08T15:25:33.350 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '--entrypoint', '/usr/bin/ceph-authtool', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '--gen-print-key'] 2020-09-08T15:25:33.350 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 --entrypoint /usr/bin/ceph-authtool quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f --gen-print-key 2020-09-08T15:25:34.433 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-authtool:stdout AQBuoldfJAEbGhAASgQe9cXWjBWmNsNcTBGEGQ== 2020-09-08T15:25:34.433 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-authtool:profile rt=1.0815784931182861, stop=False, exit=None, reads=[4] 2020-09-08T15:25:34.999 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-authtool:profile rt=1.6468617916107178, stop=False, exit=0, reads=[4, 6] 2020-09-08T15:25:35.000 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-authtool:profile rt=1.6471335887908936, stop=True, exit=0, reads=[4, 6] 2020-09-08T15:25:35.000 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '--entrypoint', '/usr/bin/ceph-authtool', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '--gen-print-key'] 2020-09-08T15:25:35.000 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 --entrypoint /usr/bin/ceph-authtool quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f --gen-print-key 2020-09-08T15:25:35.988 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-authtool:stdout AQBvoldfTP42OxAAE6PCmZHLHIwHbPSFvWTk6g== 2020-09-08T15:25:35.989 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-authtool:profile rt=0.9874720573425293, stop=False, exit=None, reads=[4] 2020-09-08T15:25:36.524 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-authtool:profile rt=1.522850513458252, stop=False, exit=0, reads=[4, 6] 2020-09-08T15:25:36.524 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-authtool:profile rt=1.523120641708374, stop=True, exit=0, reads=[4, 6] 2020-09-08T15:25:36.525 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Creating initial monmap... 2020-09-08T15:25:36.526 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/tmp/ceph-tmp3n0rmw_o:/tmp/monmap:z', '--entrypoint', '/usr/bin/monmaptool', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '--create', '--clobber', '--fsid', '759b27b6-f1e7-11ea-a080-001a4aab830c', '--addv', 'a', '[v1:172.21.15.104:6789]', '/tmp/monmap'] 2020-09-08T15:25:36.526 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /tmp/ceph-tmp3n0rmw_o:/tmp/monmap:z --entrypoint /usr/bin/monmaptool quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f --create --clobber --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c --addv a [v1:172.21.15.104:6789] /tmp/monmap 2020-09-08T15:25:37.534 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/monmaptool:stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2020-09-08T15:25:37.535 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/monmaptool:stdout /usr/bin/monmaptool: set fsid to 759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:25:37.535 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/monmaptool:stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2020-09-08T15:25:37.536 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/monmaptool:profile rt=0.9938318729400635, stop=False, exit=None, reads=[6] 2020-09-08T15:25:38.070 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/monmaptool:profile rt=1.5414323806762695, stop=False, exit=0, reads=[6, 8] 2020-09-08T15:25:38.071 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/monmaptool:profile rt=1.5417089462280273, stop=True, exit=0, reads=[6, 8] 2020-09-08T15:25:38.071 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Creating mon... 2020-09-08T15:25:38.072 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a:/var/lib/ceph/mon/ceph-a:z', '-v', '/tmp/ceph-tmpl06oyd4c:/tmp/keyring:z', '-v', '/tmp/ceph-tmp3n0rmw_o:/tmp/monmap:z', '--entrypoint', '/usr/bin/ceph-mon', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '--mkfs', '-i', 'a', '--fsid', '759b27b6-f1e7-11ea-a080-001a4aab830c', '-c', '/dev/null', '--monmap', '/tmp/monmap', '--keyring', '/tmp/keyring', '--setuser', 'ceph', '--setgroup', 'ceph', '--default-log-to-file=false', '--default-log-to-stderr=true', '--default-log-stderr-prefix="debug "', '--default-mon-cluster-log-to-file=false', '--default-mon-cluster-log-to-stderr=true'] 2020-09-08T15:25:38.072 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a:/var/lib/ceph/mon/ceph-a:z -v /tmp/ceph-tmpl06oyd4c:/tmp/keyring:z -v /tmp/ceph-tmp3n0rmw_o:/tmp/monmap:z --entrypoint /usr/bin/ceph-mon quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f --mkfs -i a --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -c /dev/null --monmap /tmp/monmap --keyring /tmp/keyring --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix="debug " --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true 2020-09-08T15:25:38.989 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 0 set uid:gid to 167:167 (ceph:ceph) 2020-09-08T15:25:38.990 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 1 imported monmap: 2020-09-08T15:25:38.990 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr epoch 0 2020-09-08T15:25:38.991 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr fsid 759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:25:38.991 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr last_changed 2020-09-08T15:25:37.526987+0000 2020-09-08T15:25:38.991 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr created 2020-09-08T15:25:37.526987+0000 2020-09-08T15:25:38.991 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr min_mon_release 0 (unknown) 2020-09-08T15:25:38.992 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 0: v1:172.21.15.104:6789/0 mon.a 2020-09-08T15:25:38.992 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2020-09-08T15:25:38.992 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 0 /usr/bin/ceph-mon: set fsid to 759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:25:38.993 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9165501594543457, stop=False, exit=None, reads=[8] 2020-09-08T15:25:38.993 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: RocksDB version: 6.8.1 2020-09-08T15:25:38.993 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2020-09-08T15:25:39.021 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2020-09-08T15:25:39.022 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Compile date Sep 8 2020 2020-09-08T15:25:39.022 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: DB SUMMARY 2020-09-08T15:25:39.022 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2020-09-08T15:25:39.023 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 0, files: 2020-09-08T15:25:39.023 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2020-09-08T15:25:39.023 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 2020-09-08T15:25:39.023 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2020-09-08T15:25:39.024 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.error_if_exists: 0 2020-09-08T15:25:39.024 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.create_if_missing: 1 2020-09-08T15:25:39.024 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.paranoid_checks: 1 2020-09-08T15:25:39.024 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9176862239837646, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.024 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.env: 0x560b5a86ec00 2020-09-08T15:25:39.025 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.fs: Posix File System 2020-09-08T15:25:39.025 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.info_log: 0x560b5cd1e880 2020-09-08T15:25:39.025 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.max_file_opening_threads: 16 2020-09-08T15:25:39.025 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.statistics: (nil) 2020-09-08T15:25:39.025 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.use_fsync: 0 2020-09-08T15:25:39.026 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.max_log_file_size: 0 2020-09-08T15:25:39.026 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2020-09-08T15:25:39.026 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.log_file_time_to_roll: 0 2020-09-08T15:25:39.026 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9181263446807861, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.026 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.keep_log_file_num: 1000 2020-09-08T15:25:39.027 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.recycle_log_file_num: 0 2020-09-08T15:25:39.027 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.allow_fallocate: 1 2020-09-08T15:25:39.027 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.allow_mmap_reads: 0 2020-09-08T15:25:39.027 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.allow_mmap_writes: 0 2020-09-08T15:25:39.028 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.use_direct_reads: 0 2020-09-08T15:25:39.028 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2020-09-08T15:25:39.028 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.create_missing_column_families: 0 2020-09-08T15:25:39.028 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.db_log_dir: 2020-09-08T15:25:39.029 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9185497760772705, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.029 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2020-09-08T15:25:39.029 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.table_cache_numshardbits: 6 2020-09-08T15:25:39.029 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.max_subcompactions: 1 2020-09-08T15:25:39.030 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.max_background_flushes: -1 2020-09-08T15:25:39.030 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.WAL_ttl_seconds: 0 2020-09-08T15:25:39.030 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.WAL_size_limit_MB: 0 2020-09-08T15:25:39.032 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2020-09-08T15:25:39.032 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2020-09-08T15:25:39.032 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9191229343414307, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.033 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.is_fd_close_on_exec: 1 2020-09-08T15:25:39.033 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.advise_random_on_open: 1 2020-09-08T15:25:39.033 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.db_write_buffer_size: 0 2020-09-08T15:25:39.033 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.write_buffer_manager: 0x560b5ccdecc0 2020-09-08T15:25:39.033 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2020-09-08T15:25:39.034 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2020-09-08T15:25:39.034 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2020-09-08T15:25:39.034 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.use_adaptive_mutex: 0 2020-09-08T15:25:39.034 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.rate_limiter: (nil) 2020-09-08T15:25:39.035 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9196991920471191, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.035 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2020-09-08T15:25:39.035 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.wal_recovery_mode: 2 2020-09-08T15:25:39.035 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.enable_thread_tracking: 0 2020-09-08T15:25:39.035 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.enable_pipelined_write: 0 2020-09-08T15:25:39.036 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.unordered_write: 0 2020-09-08T15:25:39.036 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2020-09-08T15:25:39.036 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2020-09-08T15:25:39.036 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2020-09-08T15:25:39.037 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2020-09-08T15:25:39.037 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9201464653015137, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.037 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.row_cache: None 2020-09-08T15:25:39.038 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.wal_filter: None 2020-09-08T15:25:39.038 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2020-09-08T15:25:39.038 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.allow_ingest_behind: 0 2020-09-08T15:25:39.038 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.preserve_deletes: 0 2020-09-08T15:25:39.039 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.two_write_queues: 0 2020-09-08T15:25:39.039 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.manual_wal_flush: 0 2020-09-08T15:25:39.039 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.atomic_flush: 0 2020-09-08T15:25:39.039 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2020-09-08T15:25:39.039 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.persist_stats_to_disk: 0 2020-09-08T15:25:39.040 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9206147193908691, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.040 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.write_dbid_to_manifest: 0 2020-09-08T15:25:39.040 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.log_readahead_size: 0 2020-09-08T15:25:39.040 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2020-09-08T15:25:39.040 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.max_background_jobs: 2 2020-09-08T15:25:39.041 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.max_background_compactions: -1 2020-09-08T15:25:39.041 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2020-09-08T15:25:39.041 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2020-09-08T15:25:39.041 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.delayed_write_rate : 16777216 2020-09-08T15:25:39.041 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.max_total_wal_size: 0 2020-09-08T15:25:39.042 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9210476875305176, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.042 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2020-09-08T15:25:39.042 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.stats_dump_period_sec: 600 2020-09-08T15:25:39.042 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.stats_persist_period_sec: 600 2020-09-08T15:25:39.042 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2020-09-08T15:25:39.043 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.max_open_files: -1 2020-09-08T15:25:39.043 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.bytes_per_sync: 0 2020-09-08T15:25:39.043 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.wal_bytes_per_sync: 0 2020-09-08T15:25:39.043 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.strict_bytes_per_sync: 0 2020-09-08T15:25:39.043 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Options.compaction_readahead_size: 0 2020-09-08T15:25:39.044 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9215188026428223, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.044 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Compression algorithms supported: 2020-09-08T15:25:39.044 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2020-09-08T15:25:39.044 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: kZSTD supported: 0 2020-09-08T15:25:39.045 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: kXpressCompression supported: 0 2020-09-08T15:25:39.045 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: kLZ4HCCompression supported: 1 2020-09-08T15:25:39.045 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: kLZ4Compression supported: 1 2020-09-08T15:25:39.045 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: kBZip2Compression supported: 0 2020-09-08T15:25:39.046 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: kZlibCompression supported: 1 2020-09-08T15:25:39.046 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: kSnappyCompression supported: 1 2020-09-08T15:25:39.047 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: Fast CRC32 supported: Supported on x86 2020-09-08T15:25:39.047 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9221796989440918, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.047 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.991+0000 7fd6a5440700 4 rocksdb: [db_impl/db_impl_open.cc:273] Creating manifest 1 2020-09-08T15:25:39.048 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2020-09-08T15:25:39.048 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 2020-09-08T15:25:39.048 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2020-09-08T15:25:39.048 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2020-09-08T15:25:39.049 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2020-09-08T15:25:39.049 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2020-09-08T15:25:39.049 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.merge_operator: 2020-09-08T15:25:39.049 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.compaction_filter: None 2020-09-08T15:25:39.049 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.compaction_filter_factory: None 2020-09-08T15:25:39.050 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.memtable_factory: SkipListFactory 2020-09-08T15:25:39.050 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.table_factory: BlockBasedTable 2020-09-08T15:25:39.050 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9227702617645264, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.050 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560b5cc3ba50) 2020-09-08T15:25:39.050 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr cache_index_and_filter_blocks: 1 2020-09-08T15:25:39.051 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr cache_index_and_filter_blocks_with_high_priority: 1 2020-09-08T15:25:39.051 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr pin_l0_filter_and_index_blocks_in_cache: 0 2020-09-08T15:25:39.051 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr pin_top_level_index_and_filter: 1 2020-09-08T15:25:39.051 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr index_type: 0 2020-09-08T15:25:39.051 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr data_block_index_type: 0 2020-09-08T15:25:39.052 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr index_shortening: 1 2020-09-08T15:25:39.052 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr data_block_hash_table_util_ratio: 0.750000 2020-09-08T15:25:39.052 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr hash_index_allow_collision: 1 2020-09-08T15:25:39.052 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr checksum: 1 2020-09-08T15:25:39.052 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr no_block_cache: 0 2020-09-08T15:25:39.053 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr block_cache: 0x560b5cc72510 2020-09-08T15:25:39.053 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr block_cache_name: BinnedLRUCache 2020-09-08T15:25:39.053 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr block_cache_options: 2020-09-08T15:25:39.053 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr capacity : 536870912 2020-09-08T15:25:39.053 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr num_shard_bits : 4 2020-09-08T15:25:39.054 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr strict_capacity_limit : 0 2020-09-08T15:25:39.054 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr high_pri_pool_ratio: 0.000 2020-09-08T15:25:39.054 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr block_cache_compressed: (nil) 2020-09-08T15:25:39.054 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr persistent_cache: (nil) 2020-09-08T15:25:39.054 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr block_size: 4096 2020-09-08T15:25:39.055 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr block_size_deviation: 10 2020-09-08T15:25:39.055 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr block_restart_interval: 16 2020-09-08T15:25:39.055 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr index_block_restart_interval: 1 2020-09-08T15:25:39.055 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr metadata_block_size: 4096 2020-09-08T15:25:39.056 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr partition_filters: 0 2020-09-08T15:25:39.056 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr use_delta_encoding: 1 2020-09-08T15:25:39.056 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr filter_policy: rocksdb.BuiltinBloomFilter 2020-09-08T15:25:39.056 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr whole_key_filtering: 1 2020-09-08T15:25:39.056 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr verify_compression: 0 2020-09-08T15:25:39.057 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9241881370544434, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.057 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr read_amp_bytes_per_bit: 0 2020-09-08T15:25:39.057 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr format_version: 2 2020-09-08T15:25:39.058 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr enable_index_compression: 1 2020-09-08T15:25:39.058 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr block_align: 0 2020-09-08T15:25:39.059 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2020-09-08T15:25:39.059 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.write_buffer_size: 33554432 2020-09-08T15:25:39.059 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.max_write_buffer_number: 2 2020-09-08T15:25:39.060 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.compression: NoCompression 2020-09-08T15:25:39.060 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.bottommost_compression: Disabled 2020-09-08T15:25:39.060 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.prefix_extractor: nullptr 2020-09-08T15:25:39.061 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2020-09-08T15:25:39.061 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.num_levels: 7 2020-09-08T15:25:39.061 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2020-09-08T15:25:39.061 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9248833656311035, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.062 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2020-09-08T15:25:39.062 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2020-09-08T15:25:39.062 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2020-09-08T15:25:39.062 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2020-09-08T15:25:39.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2020-09-08T15:25:39.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2020-09-08T15:25:39.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2020-09-08T15:25:39.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2020-09-08T15:25:39.064 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.compression_opts.window_bits: -14 2020-09-08T15:25:39.064 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9257364273071289, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.064 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.compression_opts.level: 32767 2020-09-08T15:25:39.064 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.compression_opts.strategy: 0 2020-09-08T15:25:39.064 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2020-09-08T15:25:39.065 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2020-09-08T15:25:39.065 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.compression_opts.enabled: false 2020-09-08T15:25:39.065 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2020-09-08T15:25:39.065 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2020-09-08T15:25:39.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2020-09-08T15:25:39.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9262337684631348, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.target_file_size_base: 67108864 2020-09-08T15:25:39.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.target_file_size_multiplier: 1 2020-09-08T15:25:39.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2020-09-08T15:25:39.067 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2020-09-08T15:25:39.067 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2020-09-08T15:25:39.067 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2020-09-08T15:25:39.067 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2020-09-08T15:25:39.067 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2020-09-08T15:25:39.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2020-09-08T15:25:39.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9267444610595703, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2020-09-08T15:25:39.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2020-09-08T15:25:39.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2020-09-08T15:25:39.069 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2020-09-08T15:25:39.069 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2020-09-08T15:25:39.069 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.arena_block_size: 4194304 2020-09-08T15:25:39.069 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2020-09-08T15:25:39.069 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2020-09-08T15:25:39.070 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2020-09-08T15:25:39.070 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9272708892822266, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.070 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.disable_auto_compactions: 0 2020-09-08T15:25:39.070 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2020-09-08T15:25:39.070 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2020-09-08T15:25:39.071 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2020-09-08T15:25:39.071 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2020-09-08T15:25:39.071 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2020-09-08T15:25:39.071 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2020-09-08T15:25:39.071 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2020-09-08T15:25:39.072 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9277582168579102, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.072 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2020-09-08T15:25:39.072 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2020-09-08T15:25:39.072 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2020-09-08T15:25:39.072 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.table_properties_collectors: 2020-09-08T15:25:39.073 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.inplace_update_support: 0 2020-09-08T15:25:39.073 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.inplace_update_num_locks: 10000 2020-09-08T15:25:39.073 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2020-09-08T15:25:39.073 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2020-09-08T15:25:39.073 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.memtable_huge_page_size: 0 2020-09-08T15:25:39.073 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9284765720367432, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.074 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.bloom_locality: 0 2020-09-08T15:25:39.074 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.max_successive_merges: 0 2020-09-08T15:25:39.074 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.optimize_filters_for_hits: 0 2020-09-08T15:25:39.074 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.paranoid_file_checks: 0 2020-09-08T15:25:39.074 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.force_consistency_checks: 0 2020-09-08T15:25:39.075 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.report_bg_io_stats: 0 2020-09-08T15:25:39.075 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.ttl: 2592000 2020-09-08T15:25:39.075 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.995+0000 7fd6a5440700 4 rocksdb: Options.periodic_compaction_seconds: 0 2020-09-08T15:25:39.076 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9289736747741699, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.076 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.999+0000 7fd6a5440700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2020-09-08T15:25:39.077 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2020-09-08T15:25:39.077 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.999+0000 7fd6a5440700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2020-09-08T15:25:39.077 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2020-09-08T15:25:39.078 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.999+0000 7fd6a5440700 4 rocksdb: DB pointer 0x560b5ccbf800 2020-09-08T15:25:39.078 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.999+0000 7fd68e2fd700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2020-09-08T15:25:39.078 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:38.999+0000 7fd68e2fd700 4 rocksdb: [db_impl/db_impl.cc:851] 2020-09-08T15:25:39.079 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ** DB Stats ** 2020-09-08T15:25:39.079 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Uptime(secs): 0.0 total, 0.0 interval 2020-09-08T15:25:39.079 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2020-09-08T15:25:39.079 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2020-09-08T15:25:39.079 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2020-09-08T15:25:39.080 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2020-09-08T15:25:39.080 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9297044277191162, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.080 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2020-09-08T15:25:39.080 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2020-09-08T15:25:39.081 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2020-09-08T15:25:39.081 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ** Compaction Stats [default] ** 2020-09-08T15:25:39.081 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2020-09-08T15:25:39.081 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2020-09-08T15:25:39.081 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2020-09-08T15:25:39.082 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2020-09-08T15:25:39.082 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2020-09-08T15:25:39.082 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ** Compaction Stats [default] ** 2020-09-08T15:25:39.082 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2020-09-08T15:25:39.082 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9300687313079834, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.083 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2020-09-08T15:25:39.083 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Uptime(secs): 0.0 total, 0.0 interval 2020-09-08T15:25:39.083 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Flush(GB): cumulative 0.000, interval 0.000 2020-09-08T15:25:39.083 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr AddFile(GB): cumulative 0.000, interval 0.000 2020-09-08T15:25:39.085 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr AddFile(Total Files): cumulative 0, interval 0 2020-09-08T15:25:39.085 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr AddFile(L0 Files): cumulative 0, interval 0 2020-09-08T15:25:39.086 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr AddFile(Keys): cumulative 0, interval 0 2020-09-08T15:25:39.086 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2020-09-08T15:25:39.086 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2020-09-08T15:25:39.087 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2020-09-08T15:25:39.087 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2020-09-08T15:25:39.087 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ** File Read Latency Histogram By Level [default] ** 2020-09-08T15:25:39.088 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2020-09-08T15:25:39.088 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ** Compaction Stats [default] ** 2020-09-08T15:25:39.088 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9305543899536133, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.089 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2020-09-08T15:25:39.089 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2020-09-08T15:25:39.089 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2020-09-08T15:25:39.090 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2020-09-08T15:25:39.090 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2020-09-08T15:25:39.090 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ** Compaction Stats [default] ** 2020-09-08T15:25:39.090 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2020-09-08T15:25:39.091 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9308102130889893, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.091 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2020-09-08T15:25:39.091 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Uptime(secs): 0.0 total, 0.0 interval 2020-09-08T15:25:39.091 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Flush(GB): cumulative 0.000, interval 0.000 2020-09-08T15:25:39.092 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr AddFile(GB): cumulative 0.000, interval 0.000 2020-09-08T15:25:39.092 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr AddFile(Total Files): cumulative 0, interval 0 2020-09-08T15:25:39.092 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr AddFile(L0 Files): cumulative 0, interval 0 2020-09-08T15:25:39.092 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr AddFile(Keys): cumulative 0, interval 0 2020-09-08T15:25:39.093 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2020-09-08T15:25:39.093 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2020-09-08T15:25:39.093 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2020-09-08T15:25:39.093 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2020-09-08T15:25:39.094 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr ** File Read Latency Histogram By Level [default] ** 2020-09-08T15:25:39.094 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr 2020-09-08T15:25:39.094 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:39.003+0000 7fd6a5440700 4 rocksdb: [db_impl/db_impl.cc:397] Shutdown: canceling all background work 2020-09-08T15:25:39.094 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:39.003+0000 7fd6a5440700 4 rocksdb: [db_impl/db_impl.cc:573] Shutdown complete 2020-09-08T15:25:39.095 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9312212467193604, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.095 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:stderr "debug "2020-09-08T15:25:39.003+0000 7fd6a5440700 0 /usr/bin/ceph-mon: created monfs at /var/lib/ceph/mon/ceph-a for mon.a 2020-09-08T15:25:39.095 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=0.9313151836395264, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.571 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=1.4945013523101807, stop=False, exit=0, reads=[6, 8] 2020-09-08T15:25:39.572 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph-mon:profile rt=1.49472975730896, stop=True, exit=0, reads=[6, 8] 2020-09-08T15:25:39.572 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: install -d -m0770 -o 167 -g 167 /var/run/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:25:39.573 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:install:profile rt=0.002330303192138672, stop=False, exit=None, reads=[6, 8] 2020-09-08T15:25:39.573 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:install:profile rt=0.00267791748046875, stop=True, exit=0, reads=[6, 8] 2020-09-08T15:25:39.574 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl disable ceph.target 2020-09-08T15:25:39.586 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2020-09-08T15:25:39.587 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.007098674774169922, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.677 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.10140657424926758, stop=False, exit=0, reads=[6, 8] 2020-09-08T15:25:39.677 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.1016535758972168, stop=True, exit=0, reads=[6, 8] 2020-09-08T15:25:39.677 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl enable ceph.target 2020-09-08T15:25:39.684 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /etc/systemd/system/ceph.target. 2020-09-08T15:25:39.684 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005015850067138672, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.787 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.10764622688293457, stop=False, exit=0, reads=[6, 8] 2020-09-08T15:25:39.787 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.10780858993530273, stop=True, exit=0, reads=[6, 8] 2020-09-08T15:25:39.788 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl start ceph.target 2020-09-08T15:25:39.794 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.004562854766845703, stop=False, exit=0, reads=[6, 8] 2020-09-08T15:25:39.794 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.004756927490234375, stop=True, exit=0, reads=[6, 8] 2020-09-08T15:25:39.795 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl enable ceph-759b27b6-f1e7-11ea-a080-001a4aab830c.target 2020-09-08T15:25:39.797 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-759b27b6-f1e7-11ea-a080-001a4aab830c.target -> /etc/systemd/system/ceph-759b27b6-f1e7-11ea-a080-001a4aab830c.target. 2020-09-08T15:25:39.798 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-759b27b6-f1e7-11ea-a080-001a4aab830c.target -> /etc/systemd/system/ceph-759b27b6-f1e7-11ea-a080-001a4aab830c.target. 2020-09-08T15:25:39.798 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.0032389163970947266, stop=False, exit=None, reads=[8] 2020-09-08T15:25:39.905 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.10793185234069824, stop=False, exit=0, reads=[6, 8] 2020-09-08T15:25:39.906 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.10819315910339355, stop=True, exit=0, reads=[6, 8] 2020-09-08T15:25:39.906 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl start ceph-759b27b6-f1e7-11ea-a080-001a4aab830c.target 2020-09-08T15:25:39.914 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005078554153442383, stop=False, exit=0, reads=[6, 8] 2020-09-08T15:25:39.915 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.005290031433105469, stop=True, exit=0, reads=[6, 8] 2020-09-08T15:25:39.915 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl daemon-reload 2020-09-08T15:25:40.036 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.12386441230773926, stop=False, exit=0, reads=[6, 8] 2020-09-08T15:25:40.036 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.1241452693939209, stop=True, exit=0, reads=[6, 8] 2020-09-08T15:25:40.036 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl stop ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mon.a 2020-09-08T15:25:40.059 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.007076740264892578, stop=False, exit=0, reads=[6, 8] 2020-09-08T15:25:40.059 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.007284641265869141, stop=True, exit=0, reads=[6, 8] 2020-09-08T15:25:40.059 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl reset-failed ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mon.a 2020-09-08T15:25:40.059 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stderr Failed to reset failed state of unit ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mon.a.service: Unit ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mon.a.service is not loaded. 2020-09-08T15:25:40.060 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.003515958786010742, stop=False, exit=1, reads=[8] 2020-09-08T15:25:40.060 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.0036432743072509766, stop=True, exit=1, reads=[8] 2020-09-08T15:25:40.060 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl enable ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mon.a 2020-09-08T15:25:40.060 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stderr Created symlink /etc/systemd/system/ceph-759b27b6-f1e7-11ea-a080-001a4aab830c.target.wants/ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mon.a.service -> /etc/systemd/system/ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@.service. 2020-09-08T15:25:40.060 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.00559544563293457, stop=False, exit=None, reads=[8] 2020-09-08T15:25:40.152 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.10158681869506836, stop=False, exit=0, reads=[6, 8] 2020-09-08T15:25:40.152 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.10179996490478516, stop=True, exit=0, reads=[6, 8] 2020-09-08T15:25:40.153 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl start ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mon.a 2020-09-08T15:25:40.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:40 smithi104 systemd[1]: Starting Ceph mon.a for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T15:25:40.210 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.05313563346862793, stop=False, exit=0, reads=[6, 8] 2020-09-08T15:25:40.211 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.05352640151977539, stop=True, exit=0, reads=[6, 8] 2020-09-08T15:25:40.211 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:firewalld does not appear to be present 2020-09-08T15:25:40.211 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Not possible to enable service . firewalld.service is not available 2020-09-08T15:25:40.211 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Not possible to open ports <[]>. firewalld.service is not available 2020-09-08T15:25:40.212 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Waiting for mon to start... 2020-09-08T15:25:40.212 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Waiting for mon... 2020-09-08T15:25:40.212 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a:/var/lib/ceph/mon/ceph-a:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f status 2020-09-08T15:25:40.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:40 smithi104 docker[8702]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-mon.a 2020-09-08T15:25:40.684 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:40 smithi104 systemd[1]: Started Ceph mon.a for 759b27b6-f1e7-11ea-a080-001a4aab830c. 2020-09-08T15:25:40.684 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:40 smithi104 bash[8725]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-mon.a 2020-09-08T15:25:41.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:41 smithi104 bash[8725]: cluster 2020-09-08T15:25:41.788597+0000 mon.a (mon.0) 0 : [INF] mkfs 759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:25:41.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:41 smithi104 bash[8725]: cluster 2020-09-08T15:25:41.773002+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2020-09-08T15:25:42.027 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout cluster: 2020-09-08T15:25:42.028 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout id: 759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:25:42.028 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout health: HEALTH_OK 2020-09-08T15:25:42.028 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 2020-09-08T15:25:42.029 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout services: 2020-09-08T15:25:42.029 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout mon: 1 daemons, quorum a (age 0.245527s) 2020-09-08T15:25:42.029 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout mgr: no daemons active 2020-09-08T15:25:42.029 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout osd: 0 osds: 0 up, 0 in 2020-09-08T15:25:42.029 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 2020-09-08T15:25:42.030 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout data: 2020-09-08T15:25:42.030 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout pools: 0 pools, 0 pgs 2020-09-08T15:25:42.030 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout objects: 0 objects, 0 B 2020-09-08T15:25:42.030 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout usage: 0 B used, 0 B / 0 B avail 2020-09-08T15:25:42.030 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout pgs: 2020-09-08T15:25:42.030 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 2020-09-08T15:25:42.031 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.8175265789031982, stop=False, exit=None, reads=[8] 2020-09-08T15:25:42.631 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.417316436767578, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:25:42.631 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.417552947998047, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:25:42.631 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:mon is available 2020-09-08T15:25:42.631 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Assimilating anything we can from ceph.conf... 2020-09-08T15:25:42.632 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '-v', '/var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a:/var/lib/ceph/mon/ceph-a:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'config', 'assimilate-conf', '-i', '/var/lib/ceph/mon/ceph-a/config'] 2020-09-08T15:25:42.632 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z -v /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a:/var/lib/ceph/mon/ceph-a:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f config assimilate-conf -i /var/lib/ceph/mon/ceph-a/config 2020-09-08T15:25:43.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:42 smithi104 bash[8725]: cluster 2020-09-08T15:25:41.786040+0000 mon.a (mon.0) 2 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2020-09-08T15:25:43.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:42 smithi104 bash[8725]: cluster 2020-09-08T15:25:41.787926+0000 mon.a (mon.0) 3 : cluster [DBG] monmap e1: 1 mons at {a=v1:172.21.15.104:6789/0} 2020-09-08T15:25:43.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:42 smithi104 bash[8725]: cluster 2020-09-08T15:25:41.794822+0000 mon.a (mon.0) 4 : cluster [DBG] fsmap 2020-09-08T15:25:43.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:42 smithi104 bash[8725]: cluster 2020-09-08T15:25:41.803889+0000 mon.a (mon.0) 5 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2020-09-08T15:25:43.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:42 smithi104 bash[8725]: cluster 2020-09-08T15:25:41.804758+0000 mon.a (mon.0) 6 : cluster [DBG] mgrmap e1: no daemons active 2020-09-08T15:25:43.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:42 smithi104 bash[8725]: audit 2020-09-08T15:25:42.031527+0000 mon.a (mon.0) 7 : audit [DBG] from='client.? v1:172.21.15.104:0/2111114824' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2020-09-08T15:25:43.914 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 2020-09-08T15:25:43.914 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout [global] 2020-09-08T15:25:43.915 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout fsid = 759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:25:43.915 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout mon_host = [v1:172.21.15.104:6789] 2020-09-08T15:25:43.915 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout mon_osd_allow_pg_remap = true 2020-09-08T15:25:43.916 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout mon_osd_allow_primary_affinity = true 2020-09-08T15:25:43.917 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout mon_warn_on_no_sortbitwise = false 2020-09-08T15:25:43.917 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ms_tcp_read_timeout = 5 2020-09-08T15:25:43.917 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout osd_crush_chooseleaf_type = 0 2020-09-08T15:25:43.917 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 2020-09-08T15:25:43.918 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout [mgr] 2020-09-08T15:25:43.918 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout mgr/telemetry/nag = false 2020-09-08T15:25:43.918 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 2020-09-08T15:25:43.918 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout [osd] 2020-09-08T15:25:43.919 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout osd_map_max_advance = 10 2020-09-08T15:25:43.919 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout osd_sloppy_crc = true 2020-09-08T15:25:43.919 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.2848970890045166, stop=False, exit=None, reads=[8] 2020-09-08T15:25:44.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:43 smithi104 bash[8725]: audit 2020-09-08T15:25:43.911751+0000 mon.a (mon.0) 8 : audit [INF] from='client.? v1:172.21.15.104:0/1710043030' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2020-09-08T15:25:44.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:43 smithi104 bash[8725]: audit 2020-09-08T15:25:43.917653+0000 mon.a (mon.0) 9 : audit [INF] from='client.? v1:172.21.15.104:0/1710043030' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2020-09-08T15:25:44.493 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.86299729347229, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:25:44.493 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.863276481628418, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:25:44.493 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Generating new minimal ceph.conf... 2020-09-08T15:25:44.494 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '-v', '/var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a:/var/lib/ceph/mon/ceph-a:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'config', 'generate-minimal-conf', '-o', '/var/lib/ceph/mon/ceph-a/config'] 2020-09-08T15:25:44.494 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z -v /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a:/var/lib/ceph/mon/ceph-a:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f config generate-minimal-conf -o /var/lib/ceph/mon/ceph-a/config 2020-09-08T15:25:46.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:45 smithi104 bash[8725]: audit 2020-09-08T15:25:45.687071+0000 mon.a (mon.0) 10 : audit [DBG] from='client.? v1:172.21.15.104:0/2639554761' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:25:46.292 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.7956194877624512, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:25:46.293 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.7962088584899902, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:25:46.293 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Restarting the monitor... 2020-09-08T15:25:46.293 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl restart ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mon.a 2020-09-08T15:25:46.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:46 smithi104 systemd[1]: Stopping Ceph mon.a for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T15:25:46.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:46 smithi104 bash[8725]: debug 2020-09-08T15:25:46.399+0000 7f78ce6cb700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:25:46.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:46 smithi104 bash[8725]: debug 2020-09-08T15:25:46.399+0000 7f78ce6cb700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2020-09-08T15:25:47.222 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.9273624420166016, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:25:47.223 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.9279458522796631, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:25:47.223 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Setting mon public_network... 2020-09-08T15:25:47.223 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'config', 'set', 'mon', 'public_network', '172.21.0.0/20'] 2020-09-08T15:25:47.224 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f config set mon public_network 172.21.0.0/20 2020-09-08T15:25:47.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:47 smithi104 docker[9146]: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-mon.a 2020-09-08T15:25:47.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:47 smithi104 systemd[1]: Stopped Ceph mon.a for 759b27b6-f1e7-11ea-a080-001a4aab830c. 2020-09-08T15:25:47.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:47 smithi104 systemd[1]: Starting Ceph mon.a for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T15:25:47.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:47 smithi104 docker[9191]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-mon.a 2020-09-08T15:25:47.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:47 smithi104 systemd[1]: Started Ceph mon.a for 759b27b6-f1e7-11ea-a080-001a4aab830c. 2020-09-08T15:25:47.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:47 smithi104 bash[9203]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-mon.a 2020-09-08T15:25:48.644 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.575+0000 7efd1b42e700 0 set uid:gid to 167:167 (ceph:ceph) 2020-09-08T15:25:48.644 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.575+0000 7efd1b42e700 0 ceph version 16.0.0-5272-gdce4d291bd4 (dce4d291bd4243c9802c17d67b85a6da6bbc0f8f) pacific (dev), process ceph-mon, pid 1 2020-09-08T15:25:48.644 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.575+0000 7efd1b42e700 0 pidfile_write: ignore empty --pid-file 2020-09-08T15:25:48.645 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 0 load: jerasure load: lrc load: isa 2020-09-08T15:25:48.645 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: RocksDB version: 6.8.1 2020-09-08T15:25:48.645 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2020-09-08T15:25:48.645 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Compile date Sep 8 2020 2020-09-08T15:25:48.645 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: DB SUMMARY 2020-09-08T15:25:48.646 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: CURRENT file: CURRENT 2020-09-08T15:25:48.646 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: IDENTITY file: IDENTITY 2020-09-08T15:25:48.646 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 131 Bytes 2020-09-08T15:25:48.646 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000004.sst 2020-09-08T15:25:48.646 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000006.log size: 73178 ; 2020-09-08T15:25:48.647 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.error_if_exists: 0 2020-09-08T15:25:48.647 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.create_if_missing: 0 2020-09-08T15:25:48.647 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.paranoid_checks: 1 2020-09-08T15:25:48.647 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.env: 0x561b77f4fc00 2020-09-08T15:25:48.647 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.fs: Posix File System 2020-09-08T15:25:48.648 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.info_log: 0x561b7a208c20 2020-09-08T15:25:48.648 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_file_opening_threads: 16 2020-09-08T15:25:48.648 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.statistics: (nil) 2020-09-08T15:25:48.652 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.use_fsync: 0 2020-09-08T15:25:48.652 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_log_file_size: 0 2020-09-08T15:25:48.653 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2020-09-08T15:25:48.653 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.log_file_time_to_roll: 0 2020-09-08T15:25:48.654 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.keep_log_file_num: 1000 2020-09-08T15:25:48.654 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.recycle_log_file_num: 0 2020-09-08T15:25:48.654 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.allow_fallocate: 1 2020-09-08T15:25:48.654 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.allow_mmap_reads: 0 2020-09-08T15:25:48.655 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.allow_mmap_writes: 0 2020-09-08T15:25:48.655 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.use_direct_reads: 0 2020-09-08T15:25:48.655 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2020-09-08T15:25:48.655 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.create_missing_column_families: 0 2020-09-08T15:25:48.655 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.db_log_dir: 2020-09-08T15:25:48.656 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2020-09-08T15:25:48.656 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.table_cache_numshardbits: 6 2020-09-08T15:25:48.656 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_subcompactions: 1 2020-09-08T15:25:48.656 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_background_flushes: -1 2020-09-08T15:25:48.657 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.WAL_ttl_seconds: 0 2020-09-08T15:25:48.657 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.WAL_size_limit_MB: 0 2020-09-08T15:25:48.657 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2020-09-08T15:25:48.657 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2020-09-08T15:25:48.657 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.is_fd_close_on_exec: 1 2020-09-08T15:25:48.658 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.advise_random_on_open: 1 2020-09-08T15:25:48.658 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.db_write_buffer_size: 0 2020-09-08T15:25:48.658 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.write_buffer_manager: 0x561b7a2516b0 2020-09-08T15:25:48.659 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2020-09-08T15:25:48.659 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2020-09-08T15:25:48.659 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2020-09-08T15:25:48.660 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.use_adaptive_mutex: 0 2020-09-08T15:25:48.660 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.rate_limiter: (nil) 2020-09-08T15:25:48.660 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2020-09-08T15:25:48.660 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.wal_recovery_mode: 2 2020-09-08T15:25:48.660 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.enable_thread_tracking: 0 2020-09-08T15:25:48.662 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.enable_pipelined_write: 0 2020-09-08T15:25:48.663 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.unordered_write: 0 2020-09-08T15:25:48.663 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2020-09-08T15:25:48.663 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2020-09-08T15:25:48.663 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2020-09-08T15:25:48.664 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2020-09-08T15:25:48.664 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.row_cache: None 2020-09-08T15:25:48.664 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.wal_filter: None 2020-09-08T15:25:48.664 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2020-09-08T15:25:48.664 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.allow_ingest_behind: 0 2020-09-08T15:25:48.665 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.preserve_deletes: 0 2020-09-08T15:25:48.665 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.two_write_queues: 0 2020-09-08T15:25:48.665 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.manual_wal_flush: 0 2020-09-08T15:25:48.665 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.atomic_flush: 0 2020-09-08T15:25:48.666 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2020-09-08T15:25:48.666 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.persist_stats_to_disk: 0 2020-09-08T15:25:48.666 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.write_dbid_to_manifest: 0 2020-09-08T15:25:48.666 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.log_readahead_size: 0 2020-09-08T15:25:48.666 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2020-09-08T15:25:48.667 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_background_jobs: 2 2020-09-08T15:25:48.667 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_background_compactions: -1 2020-09-08T15:25:48.667 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2020-09-08T15:25:48.667 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2020-09-08T15:25:48.667 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.delayed_write_rate : 16777216 2020-09-08T15:25:48.668 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_total_wal_size: 0 2020-09-08T15:25:48.668 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2020-09-08T15:25:48.668 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.stats_dump_period_sec: 600 2020-09-08T15:25:48.668 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.stats_persist_period_sec: 600 2020-09-08T15:25:48.668 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2020-09-08T15:25:48.669 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_open_files: -1 2020-09-08T15:25:48.669 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.bytes_per_sync: 0 2020-09-08T15:25:48.669 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.wal_bytes_per_sync: 0 2020-09-08T15:25:48.669 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.strict_bytes_per_sync: 0 2020-09-08T15:25:48.669 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.compaction_readahead_size: 0 2020-09-08T15:25:48.670 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Compression algorithms supported: 2020-09-08T15:25:48.670 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2020-09-08T15:25:48.670 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: kZSTD supported: 0 2020-09-08T15:25:48.670 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: kXpressCompression supported: 0 2020-09-08T15:25:48.670 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: kLZ4HCCompression supported: 1 2020-09-08T15:25:48.671 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: kLZ4Compression supported: 1 2020-09-08T15:25:48.671 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: kBZip2Compression supported: 0 2020-09-08T15:25:48.671 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: kZlibCompression supported: 1 2020-09-08T15:25:48.671 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: kSnappyCompression supported: 1 2020-09-08T15:25:48.671 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Fast CRC32 supported: Supported on x86 2020-09-08T15:25:48.672 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000005 2020-09-08T15:25:48.672 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2020-09-08T15:25:48.672 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2020-09-08T15:25:48.672 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.merge_operator: 2020-09-08T15:25:48.672 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.compaction_filter: None 2020-09-08T15:25:48.673 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.compaction_filter_factory: None 2020-09-08T15:25:48.673 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.memtable_factory: SkipListFactory 2020-09-08T15:25:48.673 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.table_factory: BlockBasedTable 2020-09-08T15:25:48.673 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561b7a1ada48) 2020-09-08T15:25:48.673 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: cache_index_and_filter_blocks: 1 2020-09-08T15:25:48.674 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: cache_index_and_filter_blocks_with_high_priority: 1 2020-09-08T15:25:48.674 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: pin_l0_filter_and_index_blocks_in_cache: 0 2020-09-08T15:25:48.674 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: pin_top_level_index_and_filter: 1 2020-09-08T15:25:48.674 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: index_type: 0 2020-09-08T15:25:48.674 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: data_block_index_type: 0 2020-09-08T15:25:48.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: index_shortening: 1 2020-09-08T15:25:48.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: data_block_hash_table_util_ratio: 0.750000 2020-09-08T15:25:48.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: hash_index_allow_collision: 1 2020-09-08T15:25:48.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: checksum: 1 2020-09-08T15:25:48.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: no_block_cache: 0 2020-09-08T15:25:48.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: block_cache: 0x561b7a1e4610 2020-09-08T15:25:48.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: block_cache_name: BinnedLRUCache 2020-09-08T15:25:48.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: block_cache_options: 2020-09-08T15:25:48.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: capacity : 536870912 2020-09-08T15:25:48.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: num_shard_bits : 4 2020-09-08T15:25:48.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: strict_capacity_limit : 0 2020-09-08T15:25:48.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: high_pri_pool_ratio: 0.000 2020-09-08T15:25:48.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: block_cache_compressed: (nil) 2020-09-08T15:25:48.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: persistent_cache: (nil) 2020-09-08T15:25:48.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: block_size: 4096 2020-09-08T15:25:48.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: block_size_deviation: 10 2020-09-08T15:25:48.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: block_restart_interval: 16 2020-09-08T15:25:48.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: index_block_restart_interval: 1 2020-09-08T15:25:48.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: metadata_block_size: 4096 2020-09-08T15:25:48.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: partition_filters: 0 2020-09-08T15:25:48.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: use_delta_encoding: 1 2020-09-08T15:25:48.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: filter_policy: rocksdb.BuiltinBloomFilter 2020-09-08T15:25:48.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: whole_key_filtering: 1 2020-09-08T15:25:48.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: verify_compression: 0 2020-09-08T15:25:48.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: read_amp_bytes_per_bit: 0 2020-09-08T15:25:48.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: format_version: 2 2020-09-08T15:25:48.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: enable_index_compression: 1 2020-09-08T15:25:48.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: block_align: 0 2020-09-08T15:25:48.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.write_buffer_size: 33554432 2020-09-08T15:25:48.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_write_buffer_number: 2 2020-09-08T15:25:48.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.compression: NoCompression 2020-09-08T15:25:48.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.bottommost_compression: Disabled 2020-09-08T15:25:48.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.prefix_extractor: nullptr 2020-09-08T15:25:48.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2020-09-08T15:25:48.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.num_levels: 7 2020-09-08T15:25:48.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2020-09-08T15:25:48.682 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2020-09-08T15:25:48.682 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2020-09-08T15:25:48.682 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2020-09-08T15:25:48.682 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2020-09-08T15:25:48.682 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2020-09-08T15:25:48.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2020-09-08T15:25:48.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2020-09-08T15:25:48.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2020-09-08T15:25:48.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.compression_opts.window_bits: -14 2020-09-08T15:25:48.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.compression_opts.level: 32767 2020-09-08T15:25:48.684 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.compression_opts.strategy: 0 2020-09-08T15:25:48.684 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2020-09-08T15:25:48.684 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2020-09-08T15:25:48.684 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.compression_opts.enabled: false 2020-09-08T15:25:48.684 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2020-09-08T15:25:48.685 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2020-09-08T15:25:48.685 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2020-09-08T15:25:48.685 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.target_file_size_base: 67108864 2020-09-08T15:25:48.685 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.target_file_size_multiplier: 1 2020-09-08T15:25:48.686 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2020-09-08T15:25:48.686 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2020-09-08T15:25:48.687 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2020-09-08T15:25:48.687 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2020-09-08T15:25:48.687 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2020-09-08T15:25:48.687 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2020-09-08T15:25:48.687 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2020-09-08T15:25:48.688 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2020-09-08T15:25:48.688 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2020-09-08T15:25:48.688 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2020-09-08T15:25:48.688 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2020-09-08T15:25:48.688 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2020-09-08T15:25:48.689 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.arena_block_size: 4194304 2020-09-08T15:25:48.689 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2020-09-08T15:25:48.689 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2020-09-08T15:25:48.689 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2020-09-08T15:25:48.690 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.disable_auto_compactions: 0 2020-09-08T15:25:48.690 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2020-09-08T15:25:48.690 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2020-09-08T15:25:48.690 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2020-09-08T15:25:48.691 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2020-09-08T15:25:48.691 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2020-09-08T15:25:48.691 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2020-09-08T15:25:48.691 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2020-09-08T15:25:48.691 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2020-09-08T15:25:48.691 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2020-09-08T15:25:48.692 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2020-09-08T15:25:48.692 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.table_properties_collectors: 2020-09-08T15:25:48.692 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.inplace_update_support: 0 2020-09-08T15:25:48.692 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.inplace_update_num_locks: 10000 2020-09-08T15:25:48.692 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2020-09-08T15:25:48.693 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2020-09-08T15:25:48.693 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.memtable_huge_page_size: 0 2020-09-08T15:25:48.693 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.bloom_locality: 0 2020-09-08T15:25:48.693 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.max_successive_merges: 0 2020-09-08T15:25:48.694 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.optimize_filters_for_hits: 0 2020-09-08T15:25:48.694 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.paranoid_file_checks: 0 2020-09-08T15:25:48.694 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.force_consistency_checks: 0 2020-09-08T15:25:48.694 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.report_bg_io_stats: 0 2020-09-08T15:25:48.694 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.ttl: 2592000 2020-09-08T15:25:48.694 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: Options.periodic_compaction_seconds: 0 2020-09-08T15:25:48.695 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 5, log_number is 4,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2020-09-08T15:25:48.695 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.579+0000 7efd1b42e700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 4 2020-09-08T15:25:48.695 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.583+0000 7efd1b42e700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578748584602, "job": 1, "event": "recovery_started", "log_files": [6]} 2020-09-08T15:25:48.695 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.583+0000 7efd1b42e700 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #6 mode 2 2020-09-08T15:25:48.695 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.583+0000 7efd1b42e700 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2020-09-08T15:25:48.696 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.583+0000 7efd1b42e700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578748585837, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 7, "file_size": 69069, "table_properties": {"data_size": 67348, "index_size": 326, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 581, "raw_key_size": 11249, "raw_average_key_size": 49, "raw_value_size": 61185, "raw_average_value_size": 269, "num_data_blocks": 10, "num_entries": 227, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1599578748, "oldest_key_time": 3, "file_creation_time": 0}} 2020-09-08T15:25:48.696 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.583+0000 7efd1b42e700 4 rocksdb: [version_set.cc:3826] Creating manifest 8 2020-09-08T15:25:48.696 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.583+0000 7efd1b42e700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578748586713, "job": 1, "event": "recovery_finished"} 2020-09-08T15:25:48.696 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.583+0000 7efd1b42e700 4 rocksdb: DB pointer 0x561b7a231800 2020-09-08T15:25:48.697 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.583+0000 7efd0226d700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2020-09-08T15:25:48.697 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.583+0000 7efd0226d700 4 rocksdb: [db_impl/db_impl.cc:851] 2020-09-08T15:25:48.697 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: ** DB Stats ** 2020-09-08T15:25:48.697 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Uptime(secs): 0.0 total, 0.0 interval 2020-09-08T15:25:48.697 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2020-09-08T15:25:48.698 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2020-09-08T15:25:48.698 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2020-09-08T15:25:48.698 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2020-09-08T15:25:48.700 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2020-09-08T15:25:48.700 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2020-09-08T15:25:48.700 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: ** Compaction Stats [default] ** 2020-09-08T15:25:48.700 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2020-09-08T15:25:48.701 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2020-09-08T15:25:48.701 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: L0 2/0 69.05 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 82.4 0.00 0.00 1 0.001 0 0 2020-09-08T15:25:48.701 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Sum 2/0 69.05 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 82.4 0.00 0.00 1 0.001 0 0 2020-09-08T15:25:48.701 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 82.4 0.00 0.00 1 0.001 0 0 2020-09-08T15:25:48.701 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: ** Compaction Stats [default] ** 2020-09-08T15:25:48.702 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2020-09-08T15:25:48.702 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2020-09-08T15:25:48.702 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 82.4 0.00 0.00 1 0.001 0 0 2020-09-08T15:25:48.702 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Uptime(secs): 0.0 total, 0.0 interval 2020-09-08T15:25:48.702 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Flush(GB): cumulative 0.000, interval 0.000 2020-09-08T15:25:48.703 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: AddFile(GB): cumulative 0.000, interval 0.000 2020-09-08T15:25:48.703 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: AddFile(Total Files): cumulative 0, interval 0 2020-09-08T15:25:48.703 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: AddFile(L0 Files): cumulative 0, interval 0 2020-09-08T15:25:48.703 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: AddFile(Keys): cumulative 0, interval 0 2020-09-08T15:25:48.703 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Cumulative compaction: 0.00 GB write, 14.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2020-09-08T15:25:48.704 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Interval compaction: 0.00 GB write, 14.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2020-09-08T15:25:48.704 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2020-09-08T15:25:48.704 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: ** File Read Latency Histogram By Level [default] ** 2020-09-08T15:25:48.704 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: ** Compaction Stats [default] ** 2020-09-08T15:25:48.704 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2020-09-08T15:25:48.705 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2020-09-08T15:25:48.705 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: L0 2/0 69.05 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 82.4 0.00 0.00 1 0.001 0 0 2020-09-08T15:25:48.705 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Sum 2/0 69.05 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 82.4 0.00 0.00 1 0.001 0 0 2020-09-08T15:25:48.705 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2020-09-08T15:25:48.705 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: ** Compaction Stats [default] ** 2020-09-08T15:25:48.706 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2020-09-08T15:25:48.706 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2020-09-08T15:25:48.706 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 82.4 0.00 0.00 1 0.001 0 0 2020-09-08T15:25:48.706 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Uptime(secs): 0.0 total, 0.0 interval 2020-09-08T15:25:48.706 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Flush(GB): cumulative 0.000, interval 0.000 2020-09-08T15:25:48.707 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: AddFile(GB): cumulative 0.000, interval 0.000 2020-09-08T15:25:48.707 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: AddFile(Total Files): cumulative 0, interval 0 2020-09-08T15:25:48.707 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: AddFile(L0 Files): cumulative 0, interval 0 2020-09-08T15:25:48.707 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: AddFile(Keys): cumulative 0, interval 0 2020-09-08T15:25:48.707 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Cumulative compaction: 0.00 GB write, 14.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2020-09-08T15:25:48.708 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2020-09-08T15:25:48.708 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2020-09-08T15:25:48.708 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: ** File Read Latency Histogram By Level [default] ** 2020-09-08T15:25:48.708 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.587+0000 7efd1b42e700 0 starting mon.a rank 0 at public addrs v1:172.21.15.104:6789/0 at bind addrs v1:172.21.15.104:6789/0 mon_data /var/lib/ceph/mon/ceph-a fsid 759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:25:48.708 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.587+0000 7efd1b42e700 1 mon.a@-1(???) e1 preinit fsid 759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:25:48.708 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.587+0000 7efd1b42e700 0 mon.a@-1(???).mds e1 new map 2020-09-08T15:25:48.709 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.587+0000 7efd1b42e700 0 mon.a@-1(???).mds e1 print_map 2020-09-08T15:25:48.709 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: e1 2020-09-08T15:25:48.709 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: enable_multiple, ever_enabled_multiple: 0,0 2020-09-08T15:25:48.709 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2020-09-08T15:25:48.709 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: legacy client fscid: -1 2020-09-08T15:25:48.710 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: 2020-09-08T15:25:48.710 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: No filesystems configured 2020-09-08T15:25:48.710 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.587+0000 7efd1b42e700 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2020-09-08T15:25:48.710 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.587+0000 7efd1b42e700 0 mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2020-09-08T15:25:48.710 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.587+0000 7efd1b42e700 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2020-09-08T15:25:48.711 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: audit 2020-09-08T15:25:45.687071+0000 mon.a (mon.0) 10 : audit [DBG] from='client.? v1:172.21.15.104:0/2639554761' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:25:48.711 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.587+0000 7efd1b42e700 1 mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2020-09-08T15:25:48.711 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.587+0000 7efd1b42e700 20 expand_channel_meta expand map: {default=false} 2020-09-08T15:25:48.711 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.587+0000 7efd1b42e700 20 expand_channel_meta from 'false' to 'false' 2020-09-08T15:25:48.711 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.587+0000 7efd1b42e700 20 expand_channel_meta expanded map: {default=false} 2020-09-08T15:25:48.712 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.587+0000 7efd1b42e700 20 expand_channel_meta expand map: {default=info} 2020-09-08T15:25:48.712 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.587+0000 7efd1b42e700 20 expand_channel_meta from 'info' to 'info' 2020-09-08T15:25:48.712 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.587+0000 7efd1b42e700 20 expand_channel_meta expanded map: {default=info} 2020-09-08T15:25:48.712 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.587+0000 7efd1b42e700 20 expand_channel_meta expand map: {default=daemon} 2020-09-08T15:25:48.712 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.587+0000 7efd1b42e700 20 expand_channel_meta from 'daemon' to 'daemon' 2020-09-08T15:25:48.713 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.587+0000 7efd1b42e700 20 expand_channel_meta expanded map: {default=daemon} 2020-09-08T15:25:48.713 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.587+0000 7efd1b42e700 20 expand_channel_meta expand map: {default=debug} 2020-09-08T15:25:48.713 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.587+0000 7efd1b42e700 20 expand_channel_meta from 'debug' to 'debug' 2020-09-08T15:25:48.713 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: debug 2020-09-08T15:25:48.587+0000 7efd1b42e700 20 expand_channel_meta expanded map: {default=debug} 2020-09-08T15:25:48.713 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: cluster 2020-09-08T15:25:48.596002+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2020-09-08T15:25:48.714 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: cluster 2020-09-08T15:25:48.596071+0000 mon.a (mon.0) 2 : cluster [DBG] monmap e1: 1 mons at {a=v1:172.21.15.104:6789/0} 2020-09-08T15:25:48.714 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: cluster 2020-09-08T15:25:48.596126+0000 mon.a (mon.0) 3 : cluster [DBG] fsmap 2020-09-08T15:25:48.714 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: cluster 2020-09-08T15:25:48.596160+0000 mon.a (mon.0) 4 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2020-09-08T15:25:48.714 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:48 smithi104 bash[9203]: cluster 2020-09-08T15:25:48.596596+0000 mon.a (mon.0) 5 : cluster [DBG] mgrmap e1: no daemons active 2020-09-08T15:25:49.492 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.2607457637786865, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:25:49.492 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.261195182800293, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:25:49.493 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Creating mgr... 2020-09-08T15:25:49.493 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Verifying port 9283 ... 2020-09-08T15:25:49.493 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl daemon-reload 2020-09-08T15:25:49.590 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.10047435760498047, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:25:49.591 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.10099673271179199, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:25:49.591 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl stop ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mgr.y 2020-09-08T15:25:49.599 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.0064029693603515625, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:25:49.599 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.00682377815246582, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:25:49.599 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl reset-failed ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mgr.y 2020-09-08T15:25:49.606 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stderr Failed to reset failed state of unit ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mgr.y.service: Unit ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mgr.y.service is not loaded. 2020-09-08T15:25:49.606 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.004885435104370117, stop=False, exit=None, reads=[10] 2020-09-08T15:25:49.607 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.0050907135009765625, stop=True, exit=1, reads=[10] 2020-09-08T15:25:49.607 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl enable ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mgr.y 2020-09-08T15:25:49.614 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:stderr Created symlink /etc/systemd/system/ceph-759b27b6-f1e7-11ea-a080-001a4aab830c.target.wants/ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mgr.y.service -> /etc/systemd/system/ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@.service. 2020-09-08T15:25:49.614 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.006418943405151367, stop=False, exit=None, reads=[10] 2020-09-08T15:25:49.706 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.09806513786315918, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:25:49.707 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.09827542304992676, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:25:49.707 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: systemctl start ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mgr.y 2020-09-08T15:25:49.752 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:49 smithi104 bash[9203]: audit 2020-09-08T15:25:48.828254+0000 mon.a (mon.0) 6 : audit [INF] from='client.? v1:172.21.15.104:0/1156950887' entity='client.admin' cmd=[{"prefix": "config set", "who": "mon", "name": "public_network", "value": "172.21.0.0/20"}]: dispatch 2020-09-08T15:25:49.770 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:49 smithi104 bash[9203]: audit 2020-09-08T15:25:48.831518+0000 mon.a (mon.0) 7 : audit [INF] from='client.? v1:172.21.15.104:0/1156950887' entity='client.admin' cmd='[{"prefix": "config set", "who": "mon", "name": "public_network", "value": "172.21.0.0/20"}]': finished 2020-09-08T15:25:49.770 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:25:49 smithi104 systemd[1]: Starting Ceph mgr.y for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T15:25:49.771 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.048598527908325195, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:25:49.772 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:systemctl:profile rt=0.04921531677246094, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:25:49.772 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:firewalld does not appear to be present 2020-09-08T15:25:49.772 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Not possible to enable service . firewalld.service is not available 2020-09-08T15:25:49.772 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Not possible to open ports <[]>. firewalld.service is not available 2020-09-08T15:25:49.772 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:firewalld does not appear to be present 2020-09-08T15:25:49.773 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Not possible to open ports <[9283]>. firewalld.service is not available 2020-09-08T15:25:49.773 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2020-09-08T15:25:49.773 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Wrote config to /etc/ceph/ceph.conf 2020-09-08T15:25:49.773 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Waiting for mgr to start... 2020-09-08T15:25:49.774 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Waiting for mgr... 2020-09-08T15:25:49.774 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'status', '-f', 'json-pretty'] 2020-09-08T15:25:49.774 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f status -f json-pretty 2020-09-08T15:25:50.183 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:25:49 smithi104 docker[9462]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-mgr.y 2020-09-08T15:25:50.184 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:25:49 smithi104 systemd[1]: Started Ceph mgr.y for 759b27b6-f1e7-11ea-a080-001a4aab830c. 2020-09-08T15:25:50.184 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:25:49 smithi104 bash[9475]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-mgr.y 2020-09-08T15:25:51.477 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 2020-09-08T15:25:51.477 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:25:51.479 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "fsid": "759b27b6-f1e7-11ea-a080-001a4aab830c", 2020-09-08T15:25:51.479 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "health": { 2020-09-08T15:25:51.479 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "status": "HEALTH_OK", 2020-09-08T15:25:51.479 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "checks": {}, 2020-09-08T15:25:51.479 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mutes": [] 2020-09-08T15:25:51.480 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:25:51.480 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "election_epoch": 5, 2020-09-08T15:25:51.480 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "quorum": [ 2020-09-08T15:25:51.480 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 0 2020-09-08T15:25:51.480 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:25:51.481 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "quorum_names": [ 2020-09-08T15:25:51.481 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "a" 2020-09-08T15:25:51.481 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:25:51.481 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "quorum_age": 2, 2020-09-08T15:25:51.481 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "monmap": { 2020-09-08T15:25:51.482 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2020-09-08T15:25:51.482 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min_mon_release_name": "pacific", 2020-09-08T15:25:51.482 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_mons": 1 2020-09-08T15:25:51.482 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:25:51.482 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "osdmap": { 2020-09-08T15:25:51.483 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2020-09-08T15:25:51.483 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_osds": 0, 2020-09-08T15:25:51.483 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_up_osds": 0, 2020-09-08T15:25:51.483 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "osd_up_since": 0, 2020-09-08T15:25:51.483 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_in_osds": 0, 2020-09-08T15:25:51.483 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "osd_in_since": 0, 2020-09-08T15:25:51.484 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_remapped_pgs": 0 2020-09-08T15:25:51.484 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:25:51.484 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pgmap": { 2020-09-08T15:25:51.484 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pgs_by_state": [], 2020-09-08T15:25:51.484 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_pgs": 0, 2020-09-08T15:25:51.485 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_pools": 0, 2020-09-08T15:25:51.485 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_objects": 0, 2020-09-08T15:25:51.485 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "data_bytes": 0, 2020-09-08T15:25:51.485 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "bytes_used": 0, 2020-09-08T15:25:51.485 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "bytes_avail": 0, 2020-09-08T15:25:51.486 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "bytes_total": 0 2020-09-08T15:25:51.486 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:25:51.486 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "fsmap": { 2020-09-08T15:25:51.486 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2020-09-08T15:25:51.486 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "by_rank": [], 2020-09-08T15:25:51.486 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "up:standby": 0 2020-09-08T15:25:51.487 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:25:51.487 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mgrmap": { 2020-09-08T15:25:51.487 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "available": false, 2020-09-08T15:25:51.487 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_standbys": 0, 2020-09-08T15:25:51.487 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "modules": [ 2020-09-08T15:25:51.488 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "iostat", 2020-09-08T15:25:51.488 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.714238166809082, stop=False, exit=None, reads=[8] 2020-09-08T15:25:51.488 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "restful" 2020-09-08T15:25:51.488 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:25:51.488 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "services": {} 2020-09-08T15:25:51.489 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:25:51.489 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "servicemap": { 2020-09-08T15:25:51.489 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2020-09-08T15:25:51.489 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "modified": "2020-09-08T15:25:41.789147+0000", 2020-09-08T15:25:51.489 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "services": {} 2020-09-08T15:25:51.489 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:25:51.490 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "progress_events": {} 2020-09-08T15:25:51.490 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:25:51.490 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.7148315906524658, stop=False, exit=None, reads=[8] 2020-09-08T15:25:51.933 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:51 smithi104 bash[9203]: audit 2020-09-08T15:25:51.477194+0000 mon.a (mon.0) 8 : audit [DBG] from='client.? v1:172.21.15.104:0/3463734218' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2020-09-08T15:25:52.145 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.3834006786346436, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:25:52.145 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.3836495876312256, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:25:52.145 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:mgr not available, waiting (1/10)... 2020-09-08T15:25:53.147 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'status', '-f', 'json-pretty'] 2020-09-08T15:25:53.147 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f status -f json-pretty 2020-09-08T15:25:54.350 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 2020-09-08T15:25:54.350 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:25:54.350 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "fsid": "759b27b6-f1e7-11ea-a080-001a4aab830c", 2020-09-08T15:25:54.351 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "health": { 2020-09-08T15:25:54.351 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "status": "HEALTH_OK", 2020-09-08T15:25:54.351 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "checks": {}, 2020-09-08T15:25:54.351 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mutes": [] 2020-09-08T15:25:54.351 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:25:54.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "election_epoch": 5, 2020-09-08T15:25:54.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "quorum": [ 2020-09-08T15:25:54.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 0 2020-09-08T15:25:54.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:25:54.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "quorum_names": [ 2020-09-08T15:25:54.353 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "a" 2020-09-08T15:25:54.354 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:25:54.354 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "quorum_age": 5, 2020-09-08T15:25:54.355 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "monmap": { 2020-09-08T15:25:54.355 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2020-09-08T15:25:54.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min_mon_release_name": "pacific", 2020-09-08T15:25:54.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_mons": 1 2020-09-08T15:25:54.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:25:54.357 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "osdmap": { 2020-09-08T15:25:54.357 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2020-09-08T15:25:54.357 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_osds": 0, 2020-09-08T15:25:54.357 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_up_osds": 0, 2020-09-08T15:25:54.357 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "osd_up_since": 0, 2020-09-08T15:25:54.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_in_osds": 0, 2020-09-08T15:25:54.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "osd_in_since": 0, 2020-09-08T15:25:54.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_remapped_pgs": 0 2020-09-08T15:25:54.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:25:54.365 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pgmap": { 2020-09-08T15:25:54.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pgs_by_state": [], 2020-09-08T15:25:54.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_pgs": 0, 2020-09-08T15:25:54.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_pools": 0, 2020-09-08T15:25:54.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_objects": 0, 2020-09-08T15:25:54.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "data_bytes": 0, 2020-09-08T15:25:54.367 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "bytes_used": 0, 2020-09-08T15:25:54.367 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "bytes_avail": 0, 2020-09-08T15:25:54.367 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "bytes_total": 0 2020-09-08T15:25:54.367 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:25:54.367 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "fsmap": { 2020-09-08T15:25:54.368 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2020-09-08T15:25:54.368 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "by_rank": [], 2020-09-08T15:25:54.368 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "up:standby": 0 2020-09-08T15:25:54.368 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:25:54.368 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mgrmap": { 2020-09-08T15:25:54.369 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "available": false, 2020-09-08T15:25:54.369 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_standbys": 0, 2020-09-08T15:25:54.369 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "modules": [ 2020-09-08T15:25:54.369 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "iostat", 2020-09-08T15:25:54.369 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.1955111026763916, stop=False, exit=None, reads=[8] 2020-09-08T15:25:54.369 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "restful" 2020-09-08T15:25:54.370 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:25:54.370 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "services": {} 2020-09-08T15:25:54.370 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:25:54.370 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "servicemap": { 2020-09-08T15:25:54.370 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2020-09-08T15:25:54.371 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "modified": "2020-09-08T15:25:41.789147+0000", 2020-09-08T15:25:54.371 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "services": {} 2020-09-08T15:25:54.371 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:25:54.371 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "progress_events": {} 2020-09-08T15:25:54.371 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:25:54.372 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.196077585220337, stop=False, exit=None, reads=[8] 2020-09-08T15:25:54.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:54 smithi104 bash[9203]: audit 2020-09-08T15:25:54.346692+0000 mon.a (mon.0) 9 : audit [DBG] from='client.? v1:172.21.15.104:0/1278325735' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2020-09-08T15:25:54.951 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.7988054752349854, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:25:54.952 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.7991313934326172, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:25:54.952 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:mgr not available, waiting (2/10)... 2020-09-08T15:25:55.950 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'status', '-f', 'json-pretty'] 2020-09-08T15:25:55.950 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f status -f json-pretty 2020-09-08T15:25:56.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:56 smithi104 bash[9203]: cluster 2020-09-08T15:25:55.870353+0000 mon.a (mon.0) 10 : cluster [INF] Activating manager daemon y 2020-09-08T15:25:56.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:56 smithi104 bash[9203]: cluster 2020-09-08T15:25:55.873989+0000 mon.a (mon.0) 11 : cluster [DBG] mgrmap e2: y(active, starting, since 0.00372468s) 2020-09-08T15:25:56.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:56 smithi104 bash[9203]: audit 2020-09-08T15:25:55.877234+0000 mon.a (mon.0) 12 : audit [DBG] from='mgr.14100 v1:172.21.15.104:0/3434220998' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2020-09-08T15:25:56.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:56 smithi104 bash[9203]: audit 2020-09-08T15:25:55.877706+0000 mon.a (mon.0) 13 : audit [DBG] from='mgr.14100 v1:172.21.15.104:0/3434220998' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2020-09-08T15:25:56.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:56 smithi104 bash[9203]: audit 2020-09-08T15:25:55.878227+0000 mon.a (mon.0) 14 : audit [DBG] from='mgr.14100 v1:172.21.15.104:0/3434220998' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2020-09-08T15:25:56.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:56 smithi104 bash[9203]: audit 2020-09-08T15:25:55.878642+0000 mon.a (mon.0) 15 : audit [DBG] from='mgr.14100 v1:172.21.15.104:0/3434220998' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2020-09-08T15:25:56.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:56 smithi104 bash[9203]: audit 2020-09-08T15:25:55.878993+0000 mon.a (mon.0) 16 : audit [DBG] from='mgr.14100 v1:172.21.15.104:0/3434220998' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2020-09-08T15:25:56.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:56 smithi104 bash[9203]: audit 2020-09-08T15:25:55.879317+0000 mon.a (mon.0) 17 : audit [DBG] from='mgr.14100 v1:172.21.15.104:0/3434220998' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2020-09-08T15:25:56.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:56 smithi104 bash[9203]: audit 2020-09-08T15:25:55.879698+0000 mon.a (mon.0) 18 : audit [DBG] from='mgr.14100 v1:172.21.15.104:0/3434220998' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2020-09-08T15:25:56.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:56 smithi104 bash[9203]: audit 2020-09-08T15:25:55.880062+0000 mon.a (mon.0) 19 : audit [DBG] from='mgr.14100 v1:172.21.15.104:0/3434220998' entity='mgr.y' cmd=[{"prefix": "config-key ls"}]: dispatch 2020-09-08T15:25:56.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:56 smithi104 bash[9203]: cluster 2020-09-08T15:25:55.883904+0000 mon.a (mon.0) 20 : cluster [INF] Manager daemon y is now available 2020-09-08T15:25:57.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:57 smithi104 bash[9203]: audit 2020-09-08T15:25:55.903047+0000 mon.a (mon.0) 21 : audit [INF] from='mgr.14100 v1:172.21.15.104:0/3434220998' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]: dispatch 2020-09-08T15:25:57.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:57 smithi104 bash[9203]: audit 2020-09-08T15:25:56.323588+0000 mon.a (mon.0) 22 : audit [INF] from='mgr.14100 v1:172.21.15.104:0/3434220998' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]': finished 2020-09-08T15:25:57.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:57 smithi104 bash[9203]: cluster 2020-09-08T15:25:56.323647+0000 mon.a (mon.0) 23 : cluster [DBG] osdmap e2: 0 total, 0 up, 0 in 2020-09-08T15:25:57.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:57 smithi104 bash[9203]: audit 2020-09-08T15:25:56.328780+0000 mon.a (mon.0) 24 : audit [INF] from='mgr.14100 v1:172.21.15.104:0/3434220998' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2020-09-08T15:25:57.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:57 smithi104 bash[9203]: audit 2020-09-08T15:25:56.329616+0000 mon.a (mon.0) 25 : audit [INF] from='mgr.14100 v1:172.21.15.104:0/3434220998' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/telemetry/report_id","val":"b4e211b0-87fd-406f-8d95-62adfb931295"}]: dispatch 2020-09-08T15:25:57.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:57 smithi104 bash[9203]: audit 2020-09-08T15:25:56.333926+0000 mon.a (mon.0) 26 : audit [INF] from='mgr.14100 v1:172.21.15.104:0/3434220998' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/telemetry/report_id","val":"b4e211b0-87fd-406f-8d95-62adfb931295"}]': finished 2020-09-08T15:25:57.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:57 smithi104 bash[9203]: audit 2020-09-08T15:25:56.334294+0000 mon.a (mon.0) 27 : audit [INF] from='mgr.14100 v1:172.21.15.104:0/3434220998' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/telemetry/salt","val":"d6514ea0-a5ed-4f46-a11a-9164d3bdaf84"}]: dispatch 2020-09-08T15:25:57.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:57 smithi104 bash[9203]: audit 2020-09-08T15:25:56.337423+0000 mon.a (mon.0) 28 : audit [INF] from='mgr.14100 v1:172.21.15.104:0/3434220998' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/telemetry/salt","val":"d6514ea0-a5ed-4f46-a11a-9164d3bdaf84"}]': finished 2020-09-08T15:25:57.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:57 smithi104 bash[9203]: cluster 2020-09-08T15:25:56.882461+0000 mon.a (mon.0) 29 : cluster [DBG] mgrmap e3: y(active, since 1.01219s) 2020-09-08T15:25:58.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:58 smithi104 bash[9203]: audit 2020-09-08T15:25:57.331609+0000 mon.a (mon.0) 30 : audit [INF] from='mgr.14100 v1:172.21.15.104:0/3434220998' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2020-09-08T15:25:58.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:58 smithi104 bash[9203]: cluster 2020-09-08T15:25:57.331723+0000 mon.a (mon.0) 31 : cluster [DBG] osdmap e3: 0 total, 0 up, 0 in 2020-09-08T15:25:58.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:58 smithi104 bash[9203]: audit 2020-09-08T15:25:57.355410+0000 mon.a (mon.0) 32 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2020-09-08T15:25:58.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:58 smithi104 bash[9203]: audit 2020-09-08T15:25:58.232669+0000 mon.a (mon.0) 33 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2020-09-08T15:25:59.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:25:59 smithi104 bash[9203]: cluster 2020-09-08T15:25:58.349254+0000 mon.a (mon.0) 34 : cluster [DBG] mgrmap e4: y(active, since 2s) 2020-09-08T15:26:00.771 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 2020-09-08T15:26:00.772 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:00.772 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "fsid": "759b27b6-f1e7-11ea-a080-001a4aab830c", 2020-09-08T15:26:00.772 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "health": { 2020-09-08T15:26:00.772 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "status": "HEALTH_OK", 2020-09-08T15:26:00.773 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "checks": {}, 2020-09-08T15:26:00.773 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mutes": [] 2020-09-08T15:26:00.773 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:00.773 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "election_epoch": 5, 2020-09-08T15:26:00.774 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "quorum": [ 2020-09-08T15:26:00.774 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 0 2020-09-08T15:26:00.774 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:00.774 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "quorum_names": [ 2020-09-08T15:26:00.774 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "a" 2020-09-08T15:26:00.775 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:00.775 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "quorum_age": 12, 2020-09-08T15:26:00.775 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "monmap": { 2020-09-08T15:26:00.775 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2020-09-08T15:26:00.775 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min_mon_release_name": "pacific", 2020-09-08T15:26:00.776 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_mons": 1 2020-09-08T15:26:00.776 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:00.777 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "osdmap": { 2020-09-08T15:26:00.777 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 3, 2020-09-08T15:26:00.777 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_osds": 0, 2020-09-08T15:26:00.777 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_up_osds": 0, 2020-09-08T15:26:00.777 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "osd_up_since": 0, 2020-09-08T15:26:00.781 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_in_osds": 0, 2020-09-08T15:26:00.781 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "osd_in_since": 0, 2020-09-08T15:26:00.781 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_remapped_pgs": 0 2020-09-08T15:26:00.781 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:00.782 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pgmap": { 2020-09-08T15:26:00.782 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pgs_by_state": [], 2020-09-08T15:26:00.782 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_pgs": 0, 2020-09-08T15:26:00.782 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_pools": 0, 2020-09-08T15:26:00.782 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_objects": 0, 2020-09-08T15:26:00.783 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "data_bytes": 0, 2020-09-08T15:26:00.783 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "bytes_used": 0, 2020-09-08T15:26:00.783 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "bytes_avail": 0, 2020-09-08T15:26:00.783 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "bytes_total": 0 2020-09-08T15:26:00.783 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:00.784 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "fsmap": { 2020-09-08T15:26:00.784 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2020-09-08T15:26:00.784 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "by_rank": [], 2020-09-08T15:26:00.784 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "up:standby": 0 2020-09-08T15:26:00.784 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:00.785 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mgrmap": { 2020-09-08T15:26:00.785 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "available": true, 2020-09-08T15:26:00.785 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_standbys": 0, 2020-09-08T15:26:00.785 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "modules": [ 2020-09-08T15:26:00.785 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "iostat", 2020-09-08T15:26:00.785 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=4.817283391952515, stop=False, exit=None, reads=[8] 2020-09-08T15:26:00.786 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "restful" 2020-09-08T15:26:00.786 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:00.787 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "services": {} 2020-09-08T15:26:00.787 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:00.787 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "servicemap": { 2020-09-08T15:26:00.787 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 1, 2020-09-08T15:26:00.787 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "modified": "2020-09-08T15:25:41.789147+0000", 2020-09-08T15:26:00.788 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "services": {} 2020-09-08T15:26:00.788 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:00.788 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "progress_events": {} 2020-09-08T15:26:00.788 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:00.788 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=4.817949056625366, stop=False, exit=None, reads=[8] 2020-09-08T15:26:01.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:00 smithi104 bash[9203]: audit 2020-09-08T15:26:00.770590+0000 mon.a (mon.0) 35 : audit [DBG] from='client.? v1:172.21.15.104:0/34173908' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2020-09-08T15:26:01.387 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=5.429385423660278, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:26:01.387 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=5.429569721221924, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:26:01.388 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:mgr is available 2020-09-08T15:26:01.388 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'config-key', 'set', 'mgr/cephadm/ssh_user', 'root'] 2020-09-08T15:26:01.388 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f config-key set mgr/cephadm/ssh_user root 2020-09-08T15:26:02.990 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stderr set mgr/cephadm/ssh_user 2020-09-08T15:26:02.990 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6061489582061768, stop=False, exit=None, reads=[10] 2020-09-08T15:26:03.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:03 smithi104 bash[9203]: audit 2020-09-08T15:26:02.986667+0000 mon.a (mon.0) 36 : audit [INF] from='client.? v1:172.21.15.104:0/3574138446' entity='client.admin' cmd=[{"prefix": "config-key set", "key": "mgr/cephadm/ssh_user", "val": "root"}]: dispatch 2020-09-08T15:26:03.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:03 smithi104 bash[9203]: audit 2020-09-08T15:26:02.990585+0000 mon.a (mon.0) 37 : audit [INF] from='client.? v1:172.21.15.104:0/3574138446' entity='client.admin' cmd='[{"prefix": "config-key set", "key": "mgr/cephadm/ssh_user", "val": "root"}]': finished 2020-09-08T15:26:03.598 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.2092320919036865, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:26:03.599 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.20949649810791, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:26:03.600 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Enabling cephadm module... 2020-09-08T15:26:03.600 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'mgr', 'module', 'enable', 'cephadm'] 2020-09-08T15:26:03.600 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f mgr module enable cephadm 2020-09-08T15:26:05.429 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:05 smithi104 bash[9475]: ignoring --setuser ceph since I am not root 2020-09-08T15:26:05.430 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:05 smithi104 bash[9475]: ignoring --setgroup ceph since I am not root 2020-09-08T15:26:05.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:05 smithi104 bash[9203]: audit 2020-09-08T15:26:05.114123+0000 mon.a (mon.0) 38 : audit [INF] from='client.? v1:172.21.15.104:0/2829118376' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2020-09-08T15:26:05.812 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.2163381576538086, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:26:05.812 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.216871738433838, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:26:05.812 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'mgr', 'dump'] 2020-09-08T15:26:05.813 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f mgr dump 2020-09-08T15:26:06.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:06 smithi104 bash[9203]: audit 2020-09-08T15:26:05.168665+0000 mon.a (mon.0) 39 : audit [INF] from='client.? v1:172.21.15.104:0/2829118376' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2020-09-08T15:26:06.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:06 smithi104 bash[9203]: cluster 2020-09-08T15:26:05.168739+0000 mon.a (mon.0) 40 : cluster [DBG] mgrmap e5: y(active, since 9s) 2020-09-08T15:26:07.360 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:07.360 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 5, 2020-09-08T15:26:07.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_gid": 14100, 2020-09-08T15:26:07.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_name": "y", 2020-09-08T15:26:07.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_addrs": { 2020-09-08T15:26:07.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addrvec": [ 2020-09-08T15:26:07.363 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:07.363 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "v2", 2020-09-08T15:26:07.364 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addr": "172.21.15.104:6800", 2020-09-08T15:26:07.364 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nonce": 1138921844 2020-09-08T15:26:07.364 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.364 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:07.364 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "v1", 2020-09-08T15:26:07.365 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addr": "172.21.15.104:6801", 2020-09-08T15:26:07.365 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nonce": 1138921844 2020-09-08T15:26:07.365 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:07.365 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ] 2020-09-08T15:26:07.365 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_addr": "172.21.15.104:6801/1138921844", 2020-09-08T15:26:07.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_change": "2020-09-08T15:25:55.870253+0000", 2020-09-08T15:26:07.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_mgr_features": 4540138292837744639, 2020-09-08T15:26:07.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "available": true, 2020-09-08T15:26:07.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "standbys": [], 2020-09-08T15:26:07.367 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "modules": [ 2020-09-08T15:26:07.367 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "cephadm", 2020-09-08T15:26:07.367 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "iostat", 2020-09-08T15:26:07.801 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "restful" 2020-09-08T15:26:07.801 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:07.802 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "available_modules": [ 2020-09-08T15:26:07.802 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:07.802 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "alerts", 2020-09-08T15:26:07.802 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:07.803 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:07.803 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:07.803 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "interval": { 2020-09-08T15:26:07.803 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "interval", 2020-09-08T15:26:07.803 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:07.804 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.804 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5449719429016113, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.804 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.804 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "60", 2020-09-08T15:26:07.804 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.804 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.805 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.805 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "How frequently to reexamine health status", 2020-09-08T15:26:07.805 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.805 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.805 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.806 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.806 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:07.806 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:07.806 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.806 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.807 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.807 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:07.807 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.807 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.807 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:07.808 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:07.808 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:07.808 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:07.808 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:07.808 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:07.809 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:07.809 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:07.809 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:07.809 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.809 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.809 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.810 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5461218357086182, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.810 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.810 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:07.810 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:07.812 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:07.812 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.813 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.813 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:07.814 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.814 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.814 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.814 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:07.815 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.815 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.815 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.815 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.815 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:07.816 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:07.816 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.816 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.816 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.816 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:07.816 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.817 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.817 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:07.817 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:07.817 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:07.817 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:07.818 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:07.818 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:07.818 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5467755794525146, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.818 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:07.818 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:07.819 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:07.819 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.819 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.819 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.819 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.821 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.821 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:07.821 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.821 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.821 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.821 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:07.822 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.822 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.822 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.823 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.823 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_destination": { 2020-09-08T15:26:07.823 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_destination", 2020-09-08T15:26:07.823 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.824 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.824 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.824 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:07.824 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.824 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.825 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5478034019470215, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.825 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.825 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Email address to send alerts to", 2020-09-08T15:26:07.825 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.825 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.826 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.826 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.826 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_from_name": { 2020-09-08T15:26:07.826 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_from_name", 2020-09-08T15:26:07.826 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.827 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.827 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.827 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "Ceph", 2020-09-08T15:26:07.827 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.828 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.828 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.828 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Email From: name", 2020-09-08T15:26:07.828 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.828 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_host": { 2020-09-08T15:26:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_host", 2020-09-08T15:26:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.830 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.830 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.830 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:07.830 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.830 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.831 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.831 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5490803718566895, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.831 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "SMTP server", 2020-09-08T15:26:07.831 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.831 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.832 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.832 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.832 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_password": { 2020-09-08T15:26:07.832 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_password", 2020-09-08T15:26:07.832 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.833 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.833 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.833 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:07.833 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.833 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.834 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.835 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Password to authenticate with", 2020-09-08T15:26:07.835 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.835 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.835 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.835 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.836 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_port": { 2020-09-08T15:26:07.836 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_port", 2020-09-08T15:26:07.836 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:07.836 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.836 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.837 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "465", 2020-09-08T15:26:07.837 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.837 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.837 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.837 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5503063201904297, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.838 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "SMTP port", 2020-09-08T15:26:07.838 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.838 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.838 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.839 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.839 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_sender": { 2020-09-08T15:26:07.839 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_sender", 2020-09-08T15:26:07.839 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.839 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.840 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.840 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:07.840 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.840 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.840 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.841 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "SMTP envelope sender", 2020-09-08T15:26:07.841 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.841 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.841 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.841 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.842 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_ssl": { 2020-09-08T15:26:07.842 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_ssl", 2020-09-08T15:26:07.842 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:07.842 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.843 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.843 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:07.843 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.843 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.843 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.844 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.551647424697876, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.844 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Use SSL to connect to SMTP server", 2020-09-08T15:26:07.844 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.845 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.846 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.846 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.846 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_user": { 2020-09-08T15:26:07.847 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_user", 2020-09-08T15:26:07.847 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.847 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.847 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.847 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:07.848 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.848 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.848 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.849 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "User to authenticate as", 2020-09-08T15:26:07.849 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.849 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.849 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.850 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:07.850 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:07.850 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.850 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:07.850 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "balancer", 2020-09-08T15:26:07.851 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:07.851 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:07.851 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:07.851 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active": { 2020-09-08T15:26:07.851 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "active", 2020-09-08T15:26:07.852 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:07.852 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.852 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.852 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:07.852 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5531671047210693, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.853 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.853 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.853 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.853 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "automatically balance PGs across cluster", 2020-09-08T15:26:07.853 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.854 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.854 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.854 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.854 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "begin_time": { 2020-09-08T15:26:07.854 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "begin_time", 2020-09-08T15:26:07.855 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.855 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.855 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.855 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "0000", 2020-09-08T15:26:07.855 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.856 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.856 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.856 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "beginning time of day to automatically balance", 2020-09-08T15:26:07.856 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "This is a time of day in the format HHMM.", 2020-09-08T15:26:07.856 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.856 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.857 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.857 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "begin_weekday": { 2020-09-08T15:26:07.857 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "begin_weekday", 2020-09-08T15:26:07.857 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "uint", 2020-09-08T15:26:07.857 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.858 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.554321527481079, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.858 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.858 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "0", 2020-09-08T15:26:07.858 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "0", 2020-09-08T15:26:07.858 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "7", 2020-09-08T15:26:07.859 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.859 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Restrict automatic balancing to this day of the week or later", 2020-09-08T15:26:07.859 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2020-09-08T15:26:07.860 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.860 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.860 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.860 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crush_compat_max_iterations": { 2020-09-08T15:26:07.860 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "crush_compat_max_iterations", 2020-09-08T15:26:07.861 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "uint", 2020-09-08T15:26:07.861 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.861 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.861 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "25", 2020-09-08T15:26:07.861 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "1", 2020-09-08T15:26:07.862 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "250", 2020-09-08T15:26:07.862 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.862 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "maximum number of iterations to attempt optimization", 2020-09-08T15:26:07.862 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.862 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.863 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.863 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.863 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.555372953414917, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.863 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crush_compat_metrics": { 2020-09-08T15:26:07.863 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "crush_compat_metrics", 2020-09-08T15:26:07.864 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.864 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.864 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.864 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "pgs,objects,bytes", 2020-09-08T15:26:07.864 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.864 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.865 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.865 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "metrics with which to calculate OSD utilization", 2020-09-08T15:26:07.865 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2020-09-08T15:26:07.865 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.865 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.866 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.866 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crush_compat_step": { 2020-09-08T15:26:07.866 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "crush_compat_step", 2020-09-08T15:26:07.866 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "float", 2020-09-08T15:26:07.866 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.867 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.867 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "0.5", 2020-09-08T15:26:07.867 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "0.001", 2020-09-08T15:26:07.867 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "0.999", 2020-09-08T15:26:07.867 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.867 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5564286708831787, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.868 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "aggressiveness of optimization", 2020-09-08T15:26:07.868 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": ".99 is very aggressive, .01 is less aggressive", 2020-09-08T15:26:07.868 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.868 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.868 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.869 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "end_time": { 2020-09-08T15:26:07.869 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "end_time", 2020-09-08T15:26:07.869 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.870 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.870 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.870 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "2400", 2020-09-08T15:26:07.870 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.870 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.871 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.871 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "ending time of day to automatically balance", 2020-09-08T15:26:07.871 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "This is a time of day in the format HHMM.", 2020-09-08T15:26:07.871 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.871 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.872 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.872 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "end_weekday": { 2020-09-08T15:26:07.872 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "end_weekday", 2020-09-08T15:26:07.872 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "uint", 2020-09-08T15:26:07.872 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.873 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.873 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "7", 2020-09-08T15:26:07.873 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5575942993164062, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.873 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "0", 2020-09-08T15:26:07.873 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "7", 2020-09-08T15:26:07.874 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.874 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Restrict automatic balancing to days of the week earlier than this", 2020-09-08T15:26:07.874 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2020-09-08T15:26:07.874 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.874 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.874 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.875 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:07.875 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:07.875 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.875 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.876 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.876 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:07.876 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.876 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.876 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:07.876 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:07.877 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:07.877 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:07.877 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:07.877 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:07.877 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:07.878 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:07.878 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:07.878 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.878 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.878 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.879 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5587809085845947, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.879 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.879 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:07.879 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:07.879 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:07.880 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.880 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.880 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:07.881 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.881 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.881 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.881 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:07.881 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.882 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.882 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.882 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.882 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:07.882 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:07.883 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.883 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.883 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.883 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:07.883 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.883 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.884 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:07.884 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:07.884 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:07.884 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:07.884 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:07.885 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:07.885 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.559943437576294, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.885 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:07.885 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:07.885 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:07.886 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.886 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.886 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.886 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.886 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:07.887 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:07.887 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:07.887 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.887 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.887 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:07.887 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.888 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.888 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.888 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:07.888 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.888 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.889 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.889 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.889 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min_score": { 2020-09-08T15:26:07.889 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "min_score", 2020-09-08T15:26:07.889 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "float", 2020-09-08T15:26:07.890 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.890 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.890 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "0", 2020-09-08T15:26:07.891 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.891 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.891 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.891 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5609731674194336, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.891 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "minimum score, below which no optimization is attempted", 2020-09-08T15:26:07.892 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.892 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.892 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.892 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.892 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mode": { 2020-09-08T15:26:07.893 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "mode", 2020-09-08T15:26:07.893 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.893 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.893 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.893 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "upmap", 2020-09-08T15:26:07.894 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.894 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.894 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:07.894 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crush-compat", 2020-09-08T15:26:07.894 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "none", 2020-09-08T15:26:07.894 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "upmap" 2020-09-08T15:26:07.895 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:07.895 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Balancer mode", 2020-09-08T15:26:07.895 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.895 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.895 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.896 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.896 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pool_ids": { 2020-09-08T15:26:07.896 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "pool_ids", 2020-09-08T15:26:07.896 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.896 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.897 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.897 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5614781379699707, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.897 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:07.897 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.897 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.897 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.898 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "pools which the automatic balancing will be limited to", 2020-09-08T15:26:07.898 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.898 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.898 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.899 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.899 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "sleep_interval": { 2020-09-08T15:26:07.899 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "sleep_interval", 2020-09-08T15:26:07.899 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:07.899 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.899 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.900 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "60", 2020-09-08T15:26:07.900 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.900 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.900 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.900 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how frequently to wake up and attempt optimization", 2020-09-08T15:26:07.901 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.901 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.902 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.902 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.902 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "upmap_max_deviation": { 2020-09-08T15:26:07.902 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "upmap_max_deviation", 2020-09-08T15:26:07.902 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:07.903 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5619418621063232, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.903 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.903 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.903 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "5", 2020-09-08T15:26:07.903 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "1", 2020-09-08T15:26:07.903 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.904 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.904 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "deviation below which no optimization is attempted", 2020-09-08T15:26:07.904 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2020-09-08T15:26:07.904 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.904 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.905 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.905 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "upmap_max_optimizations": { 2020-09-08T15:26:07.905 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "upmap_max_optimizations", 2020-09-08T15:26:07.905 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "uint", 2020-09-08T15:26:07.905 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.906 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.906 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "10", 2020-09-08T15:26:07.906 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.906 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.907 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.907 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "maximum upmap optimizations to make per attempt", 2020-09-08T15:26:07.907 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.908 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.908 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.908 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5623650550842285, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.908 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:07.908 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:07.909 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.909 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:07.909 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "cephadm", 2020-09-08T15:26:07.909 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:07.909 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:07.910 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:07.910 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "allow_ptrace": { 2020-09-08T15:26:07.910 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "allow_ptrace", 2020-09-08T15:26:07.910 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:07.910 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.911 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.911 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:07.911 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.912 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.912 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.912 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "allow SYS_PTRACE capability on ceph containers", 2020-09-08T15:26:07.912 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2020-09-08T15:26:07.912 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.913 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.913 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.913 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "config_dashboard": { 2020-09-08T15:26:07.913 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "config_dashboard", 2020-09-08T15:26:07.913 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:07.914 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.914 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5628273487091064, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.914 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.914 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:07.914 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.914 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.915 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.915 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "manage configs like API endpoints in Dashboard.", 2020-09-08T15:26:07.915 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.915 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.915 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.916 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.916 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "container_image_alertmanager": { 2020-09-08T15:26:07.916 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "container_image_alertmanager", 2020-09-08T15:26:07.916 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.916 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.917 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.917 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "docker.io/prom/alertmanager:v0.20.0", 2020-09-08T15:26:07.917 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.917 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.917 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.917 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Prometheus container image", 2020-09-08T15:26:07.918 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.918 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.918 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.918 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.918 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "container_image_base": { 2020-09-08T15:26:07.919 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5632696151733398, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.919 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "container_image_base", 2020-09-08T15:26:07.919 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.919 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.919 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.920 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "docker.io/ceph/ceph", 2020-09-08T15:26:07.920 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.920 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.920 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.921 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Container image name, without the tag", 2020-09-08T15:26:07.921 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.921 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.921 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.922 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.922 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "container_image_grafana": { 2020-09-08T15:26:07.922 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "container_image_grafana", 2020-09-08T15:26:07.923 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.923 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.923 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.923 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "docker.io/ceph/ceph-grafana:6.6.2", 2020-09-08T15:26:07.923 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.924 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.924 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.924 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Prometheus container image", 2020-09-08T15:26:07.924 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.924 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.924 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.563709020614624, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.925 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.925 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.925 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "container_image_node_exporter": { 2020-09-08T15:26:07.925 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "container_image_node_exporter", 2020-09-08T15:26:07.925 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.926 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.926 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.926 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "docker.io/prom/node-exporter:v0.18.1", 2020-09-08T15:26:07.926 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.926 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.927 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.927 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Prometheus container image", 2020-09-08T15:26:07.927 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.927 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.927 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.928 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.928 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "container_image_prometheus": { 2020-09-08T15:26:07.928 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "container_image_prometheus", 2020-09-08T15:26:07.928 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.928 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.928 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.929 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "docker.io/prom/prometheus:v2.18.1", 2020-09-08T15:26:07.929 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.929 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.929 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.929 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5641567707061768, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.930 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Prometheus container image", 2020-09-08T15:26:07.930 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.930 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.930 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.930 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.931 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "daemon_cache_timeout": { 2020-09-08T15:26:07.931 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "daemon_cache_timeout", 2020-09-08T15:26:07.931 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:07.931 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.931 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.931 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "600", 2020-09-08T15:26:07.932 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.932 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.932 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.933 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "seconds to cache service (daemon) inventory", 2020-09-08T15:26:07.933 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.933 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.933 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.933 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.934 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "device_cache_timeout": { 2020-09-08T15:26:07.934 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "device_cache_timeout", 2020-09-08T15:26:07.934 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:07.934 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.934 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.935 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "1800", 2020-09-08T15:26:07.935 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.935 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.935 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5646405220031738, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.935 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.936 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "seconds to cache device inventory", 2020-09-08T15:26:07.939 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.939 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.939 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.940 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.940 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "host_check_interval": { 2020-09-08T15:26:07.940 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "host_check_interval", 2020-09-08T15:26:07.940 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:07.940 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.941 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.941 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "600", 2020-09-08T15:26:07.941 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.941 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.941 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.942 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how frequently to perform a host check", 2020-09-08T15:26:07.942 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.942 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.942 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.943 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.943 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:07.943 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:07.943 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.943 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.944 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.944 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:07.944 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.944 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.565117597579956, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.944 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.945 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:07.945 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:07.945 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:07.945 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:07.945 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:07.946 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:07.946 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:07.946 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:07.946 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:07.946 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.946 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.947 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.947 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.947 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:07.947 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:07.948 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:07.948 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.948 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.948 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:07.948 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.948 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.949 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.949 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "log to the \"cephadm\" cluster log channel\"", 2020-09-08T15:26:07.949 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.949 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.949 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.950 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.950 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:07.950 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.565624475479126, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.950 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:07.950 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.951 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.951 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.951 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:07.951 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.951 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.951 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:07.952 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:07.952 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:07.952 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:07.952 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:07.952 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:07.953 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:07.953 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:07.953 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:07.954 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.954 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.954 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.954 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.954 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:07.955 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:07.955 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:07.955 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.955 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:07.955 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:07.956 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.956 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.956 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.956 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5661277770996094, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.956 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:07.956 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.957 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.957 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.957 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.957 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "manage_etc_ceph_ceph_conf": { 2020-09-08T15:26:07.957 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "manage_etc_ceph_ceph_conf", 2020-09-08T15:26:07.958 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:07.958 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.958 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.958 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:07.958 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.959 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.959 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.959 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2020-09-08T15:26:07.959 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.959 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.959 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.960 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.960 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "migration_current": { 2020-09-08T15:26:07.960 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "migration_current", 2020-09-08T15:26:07.960 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:07.960 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.961 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.961 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:07.961 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.961 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.961 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5665974617004395, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.962 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.962 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "internal - do not modify", 2020-09-08T15:26:07.962 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.962 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.962 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.962 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.963 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mode": { 2020-09-08T15:26:07.963 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "mode", 2020-09-08T15:26:07.963 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.964 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.964 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.964 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "root", 2020-09-08T15:26:07.964 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.964 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.965 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:07.965 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "cephadm-package", 2020-09-08T15:26:07.965 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "root" 2020-09-08T15:26:07.965 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:07.965 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "mode for remote execution of cephadm", 2020-09-08T15:26:07.966 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.966 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.966 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.966 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.966 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "prometheus_alerts_path": { 2020-09-08T15:26:07.967 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "prometheus_alerts_path", 2020-09-08T15:26:07.967 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.967 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.967 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.967 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5670859813690186, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.967 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2020-09-08T15:26:07.968 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.968 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.968 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.968 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "location of alerts to include in prometheus deployments", 2020-09-08T15:26:07.968 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.969 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.969 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.969 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.969 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "registry_password": { 2020-09-08T15:26:07.969 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "registry_password", 2020-09-08T15:26:07.970 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.970 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.970 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.970 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:07.970 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.971 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.971 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.971 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Custom repository password", 2020-09-08T15:26:07.971 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.971 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.972 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.972 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.972 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "registry_url": { 2020-09-08T15:26:07.972 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "registry_url", 2020-09-08T15:26:07.972 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.972 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.567539930343628, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.973 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.973 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.973 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:07.973 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.974 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.974 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.974 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Custom repository url", 2020-09-08T15:26:07.974 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.975 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.975 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.975 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.975 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "registry_username": { 2020-09-08T15:26:07.975 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "registry_username", 2020-09-08T15:26:07.976 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.976 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.976 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.976 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:07.976 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.976 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.977 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.977 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Custom repository username", 2020-09-08T15:26:07.977 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.977 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.977 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.978 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.978 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ssh_config_file": { 2020-09-08T15:26:07.978 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ssh_config_file", 2020-09-08T15:26:07.978 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:07.978 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5680222511291504, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.979 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.979 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.979 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:07.979 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.979 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.979 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.980 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "customized SSH config file to connect to managed hosts", 2020-09-08T15:26:07.980 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.980 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.980 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.980 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.981 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "use_repo_digest": { 2020-09-08T15:26:07.981 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "use_repo_digest", 2020-09-08T15:26:07.981 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:07.981 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.981 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.982 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:07.982 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.982 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.982 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.982 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2020-09-08T15:26:07.983 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.983 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.983 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.983 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.983 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.568471908569336, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.983 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warn_on_failed_host_check": { 2020-09-08T15:26:07.984 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "warn_on_failed_host_check", 2020-09-08T15:26:07.984 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:07.984 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.985 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.995 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:07.995 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.995 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.995 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.996 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "raise a health warning if the host check fails", 2020-09-08T15:26:07.996 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.996 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.996 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.996 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.997 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warn_on_stray_daemons": { 2020-09-08T15:26:07.997 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "warn_on_stray_daemons", 2020-09-08T15:26:07.997 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:07.997 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:07.997 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:07.998 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:07.998 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:07.998 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:07.998 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:07.998 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2020-09-08T15:26:07.999 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5689024925231934, stop=False, exit=None, reads=[8] 2020-09-08T15:26:07.999 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:07.999 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:07.999 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:07.999 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:07.999 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warn_on_stray_hosts": { 2020-09-08T15:26:08.000 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "warn_on_stray_hosts", 2020-09-08T15:26:08.000 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.000 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.000 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.000 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.001 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.001 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.001 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.001 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2020-09-08T15:26:08.001 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.002 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.002 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.002 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.002 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.002 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.003 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.003 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "crash", 2020-09-08T15:26:08.003 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.003 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.003 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.004 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.004 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.004 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.004 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.004 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5694129467010498, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.005 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.005 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.005 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.006 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.006 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.006 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.006 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.006 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.007 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.007 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.007 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.008 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.008 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.008 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.008 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.009 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.009 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.009 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.009 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.009 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.010 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.010 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.010 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.010 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.010 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.011 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.011 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.011 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.011 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.011 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.012 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.012 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5699419975280762, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.012 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.012 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.012 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.012 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.013 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.013 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.013 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.013 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.013 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.014 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.014 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.014 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.014 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.014 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.015 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.015 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.015 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.016 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.016 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.016 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.016 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.016 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.017 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.017 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.017 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.017 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.017 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.018 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.018 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.018 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5704419612884521, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.018 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.018 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.019 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.019 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.019 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.019 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.019 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "retain_interval": { 2020-09-08T15:26:08.019 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "retain_interval", 2020-09-08T15:26:08.020 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:08.020 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.020 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.020 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "31536000", 2020-09-08T15:26:08.021 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.021 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.021 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.021 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how long to retain crashes before pruning them", 2020-09-08T15:26:08.021 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.022 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.022 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.022 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.022 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warn_recent_interval": { 2020-09-08T15:26:08.022 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "warn_recent_interval", 2020-09-08T15:26:08.023 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:08.023 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.023 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.023 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "1209600", 2020-09-08T15:26:08.023 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.024 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5709164142608643, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.024 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.024 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.024 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "time interval in which to warn about recent crashes", 2020-09-08T15:26:08.024 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.025 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.025 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.025 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.025 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.026 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.026 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.026 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "dashboard", 2020-09-08T15:26:08.026 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.027 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.027 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.027 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ALERTMANAGER_API_HOST": { 2020-09-08T15:26:08.027 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ALERTMANAGER_API_HOST", 2020-09-08T15:26:08.027 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.028 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.028 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.028 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.028 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.028 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.029 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.029 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.029 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.029 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.029 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.030 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.030 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "AUDIT_API_ENABLED": { 2020-09-08T15:26:08.030 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "AUDIT_API_ENABLED", 2020-09-08T15:26:08.030 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5714349746704102, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.030 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.030 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.031 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.031 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.031 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.031 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.031 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.032 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.032 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.032 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.032 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.032 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.033 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "AUDIT_API_LOG_PAYLOAD": { 2020-09-08T15:26:08.033 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "AUDIT_API_LOG_PAYLOAD", 2020-09-08T15:26:08.033 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.033 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.033 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.033 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.034 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.034 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.034 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.034 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.034 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.035 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.035 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.035 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.035 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ENABLE_BROWSABLE_API": { 2020-09-08T15:26:08.035 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ENABLE_BROWSABLE_API", 2020-09-08T15:26:08.036 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5719165802001953, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.036 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.036 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.037 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.037 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.037 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.037 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.037 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.038 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.038 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.038 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.038 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.038 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.039 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_cephfs": { 2020-09-08T15:26:08.039 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_cephfs", 2020-09-08T15:26:08.039 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.039 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.039 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.039 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.040 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.040 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.040 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.040 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.040 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.041 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.041 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.041 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.041 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_iscsi": { 2020-09-08T15:26:08.041 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_iscsi", 2020-09-08T15:26:08.042 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5723919868469238, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.042 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.042 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.042 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.042 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.042 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.043 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.043 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.043 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.043 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.043 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.044 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.044 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.044 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_mirroring": { 2020-09-08T15:26:08.044 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_mirroring", 2020-09-08T15:26:08.044 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.045 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.045 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.045 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.045 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.045 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.045 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.046 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.046 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.046 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.047 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.047 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.047 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_nfs": { 2020-09-08T15:26:08.047 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_nfs", 2020-09-08T15:26:08.047 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.048 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5728983879089355, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.048 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.048 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.048 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.049 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.049 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.049 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.049 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.049 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.050 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.050 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.050 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.050 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_rbd": { 2020-09-08T15:26:08.050 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_rbd", 2020-09-08T15:26:08.050 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.051 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.051 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.051 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.051 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.051 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.052 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.052 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.052 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.052 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.052 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.053 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.053 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_rgw": { 2020-09-08T15:26:08.053 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_rgw", 2020-09-08T15:26:08.053 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.053 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.573394775390625, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.053 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.054 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.054 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.054 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.054 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.054 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.055 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.055 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.055 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.055 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.055 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.056 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2020-09-08T15:26:08.056 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2020-09-08T15:26:08.056 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.056 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.056 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.056 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.057 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.057 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.058 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.058 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.058 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.058 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.058 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.059 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.059 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GRAFANA_API_PASSWORD": { 2020-09-08T15:26:08.059 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GRAFANA_API_PASSWORD", 2020-09-08T15:26:08.059 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.059 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5738911628723145, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.059 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.060 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.060 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "admin", 2020-09-08T15:26:08.060 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.060 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.060 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.061 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.061 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.061 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.061 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.061 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.062 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GRAFANA_API_SSL_VERIFY": { 2020-09-08T15:26:08.062 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GRAFANA_API_SSL_VERIFY", 2020-09-08T15:26:08.062 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.062 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.062 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.064 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.064 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.064 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.064 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GRAFANA_API_URL": { 2020-09-08T15:26:08.064 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GRAFANA_API_URL", 2020-09-08T15:26:08.065 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.065 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5743777751922607, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.065 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.065 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.065 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.067 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.067 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.067 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GRAFANA_API_USERNAME": { 2020-09-08T15:26:08.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GRAFANA_API_USERNAME", 2020-09-08T15:26:08.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "admin", 2020-09-08T15:26:08.069 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.069 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.069 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.069 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.069 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.070 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.070 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.070 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.070 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GRAFANA_UPDATE_DASHBOARDS": { 2020-09-08T15:26:08.070 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GRAFANA_UPDATE_DASHBOARDS", 2020-09-08T15:26:08.071 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.071 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.071 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5748894214630127, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.071 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.071 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.072 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.072 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.072 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.072 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.072 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.073 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.073 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.073 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.073 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ISCSI_API_SSL_VERIFICATION": { 2020-09-08T15:26:08.073 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ISCSI_API_SSL_VERIFICATION", 2020-09-08T15:26:08.074 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.074 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.074 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.074 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.074 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.074 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.075 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.075 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.075 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.075 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.075 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.076 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.076 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PROMETHEUS_API_HOST": { 2020-09-08T15:26:08.076 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PROMETHEUS_API_HOST", 2020-09-08T15:26:08.076 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.076 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.077 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5753779411315918, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.077 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.077 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.077 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.077 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.077 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.078 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.078 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.079 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.079 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.079 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.079 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2020-09-08T15:26:08.079 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2020-09-08T15:26:08.079 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.080 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.080 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.080 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.080 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.080 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.081 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.081 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.081 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.081 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.081 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.082 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.082 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2020-09-08T15:26:08.082 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2020-09-08T15:26:08.082 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.575836181640625, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.082 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.083 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.083 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.083 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.083 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.083 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.084 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.084 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.084 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.084 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.084 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.084 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.085 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2020-09-08T15:26:08.085 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2020-09-08T15:26:08.085 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.085 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.085 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.086 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.086 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.086 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.086 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.086 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.087 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.087 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.087 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.087 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.087 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2020-09-08T15:26:08.087 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2020-09-08T15:26:08.088 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5763294696807861, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.088 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.088 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.089 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.089 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.089 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.089 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.089 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.090 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.090 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.090 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.090 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.090 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.091 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2020-09-08T15:26:08.091 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2020-09-08T15:26:08.091 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.091 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.091 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.092 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.092 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.092 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.092 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.092 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.093 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.093 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.093 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.093 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.093 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2020-09-08T15:26:08.094 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5767903327941895, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.094 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2020-09-08T15:26:08.094 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.094 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.094 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.095 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.095 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.095 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.095 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.095 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.095 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.096 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.096 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.096 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.096 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2020-09-08T15:26:08.096 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2020-09-08T15:26:08.097 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.097 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.097 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.097 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.097 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.098 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.098 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.098 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.098 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.098 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.099 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.099 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.099 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.577265977859497, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.100 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_ENABLED": { 2020-09-08T15:26:08.100 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_ENABLED", 2020-09-08T15:26:08.100 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.100 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.100 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.101 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.101 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.101 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.101 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.101 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.102 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.102 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.102 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.102 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.102 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_EXCLUSION_LIST": { 2020-09-08T15:26:08.102 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_EXCLUSION_LIST", 2020-09-08T15:26:08.103 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.103 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.103 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.103 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2020-09-08T15:26:08.103 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.104 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.104 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.104 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.108 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.108 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.108 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5777244567871094, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.109 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.109 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.109 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_MIN_COMPLEXITY": { 2020-09-08T15:26:08.109 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_MIN_COMPLEXITY", 2020-09-08T15:26:08.110 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.110 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.110 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.110 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "10", 2020-09-08T15:26:08.110 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.111 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.111 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.111 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.111 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.111 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.111 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.112 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.112 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_MIN_LENGTH": { 2020-09-08T15:26:08.112 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_MIN_LENGTH", 2020-09-08T15:26:08.112 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.112 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.113 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.113 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "8", 2020-09-08T15:26:08.113 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.113 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.113 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.114 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.114 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.114 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.114 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.578212022781372, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.114 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.115 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.115 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "REST_REQUESTS_TIMEOUT": { 2020-09-08T15:26:08.115 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "REST_REQUESTS_TIMEOUT", 2020-09-08T15:26:08.115 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.115 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.115 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.116 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "45", 2020-09-08T15:26:08.116 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.116 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.116 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.116 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.117 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.117 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.117 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.117 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.117 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_ACCESS_KEY": { 2020-09-08T15:26:08.118 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_ACCESS_KEY", 2020-09-08T15:26:08.118 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.118 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.118 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.118 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.119 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.119 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.119 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.119 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.119 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.120 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.120 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.121 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.578723430633545, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.121 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.121 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_ADMIN_RESOURCE": { 2020-09-08T15:26:08.122 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_ADMIN_RESOURCE", 2020-09-08T15:26:08.122 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.122 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.122 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.122 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "admin", 2020-09-08T15:26:08.123 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.123 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.123 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.123 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.123 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.124 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.124 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.124 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.124 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_HOST": { 2020-09-08T15:26:08.124 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_HOST", 2020-09-08T15:26:08.124 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.125 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.125 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.125 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.125 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.125 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.126 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.126 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.126 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.126 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.126 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.127 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.127 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5792405605316162, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.127 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_PORT": { 2020-09-08T15:26:08.127 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_PORT", 2020-09-08T15:26:08.127 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.128 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.128 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.128 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "80", 2020-09-08T15:26:08.128 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.128 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.128 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.129 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.129 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.129 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.129 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.129 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.130 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_SCHEME": { 2020-09-08T15:26:08.130 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_SCHEME", 2020-09-08T15:26:08.130 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.130 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.130 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.131 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "http", 2020-09-08T15:26:08.131 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.131 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.131 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.131 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.132 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.132 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.132 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.132 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.132 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_SECRET_KEY": { 2020-09-08T15:26:08.132 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.579735517501831, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.133 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_SECRET_KEY", 2020-09-08T15:26:08.133 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.133 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.133 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.133 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.134 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.134 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.135 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.135 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.136 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.136 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.136 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.136 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.136 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_SSL_VERIFY": { 2020-09-08T15:26:08.137 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_SSL_VERIFY", 2020-09-08T15:26:08.137 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.137 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.137 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.137 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.137 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.138 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.138 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.138 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.138 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.138 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.139 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.139 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.139 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_USER_ID": { 2020-09-08T15:26:08.139 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_USER_ID", 2020-09-08T15:26:08.139 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5802264213562012, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.140 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.140 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.140 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.140 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.140 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.140 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.141 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.141 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.141 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.141 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.141 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.142 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.142 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "USER_PWD_EXPIRATION_SPAN": { 2020-09-08T15:26:08.142 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "USER_PWD_EXPIRATION_SPAN", 2020-09-08T15:26:08.142 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.143 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.143 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.143 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "0", 2020-09-08T15:26:08.143 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.143 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.144 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.145 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.145 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.145 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.145 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.146 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.146 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "USER_PWD_EXPIRATION_WARNING_1": { 2020-09-08T15:26:08.146 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "USER_PWD_EXPIRATION_WARNING_1", 2020-09-08T15:26:08.146 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.580723762512207, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.146 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.147 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.147 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.147 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "10", 2020-09-08T15:26:08.147 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.147 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.148 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.148 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.148 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.148 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.148 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.149 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.149 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "USER_PWD_EXPIRATION_WARNING_2": { 2020-09-08T15:26:08.149 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "USER_PWD_EXPIRATION_WARNING_2", 2020-09-08T15:26:08.149 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.149 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.149 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.150 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "5", 2020-09-08T15:26:08.150 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.150 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.150 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.150 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.151 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.151 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.151 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.151 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.151 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crt_file": { 2020-09-08T15:26:08.152 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "crt_file", 2020-09-08T15:26:08.152 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5812277793884277, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.152 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.152 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.152 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.152 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.153 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.153 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.153 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.153 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.153 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.154 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.154 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.154 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.154 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug": { 2020-09-08T15:26:08.154 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "debug", 2020-09-08T15:26:08.155 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.155 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.156 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.156 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.156 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.156 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.156 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.157 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Enable/disable debug options", 2020-09-08T15:26:08.157 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.157 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.157 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.157 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.158 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "jwt_token_ttl": { 2020-09-08T15:26:08.158 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "jwt_token_ttl", 2020-09-08T15:26:08.158 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.158 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5817408561706543, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.158 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.158 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.159 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "28800", 2020-09-08T15:26:08.159 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.159 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.159 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.159 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.160 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.160 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.160 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.160 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.161 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "key_file": { 2020-09-08T15:26:08.161 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "key_file", 2020-09-08T15:26:08.161 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.161 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.162 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.162 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.167 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.167 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.168 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.168 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.168 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.169 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.169 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.169 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.169 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.169 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.170 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.170 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.170 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.170 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5822644233703613, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.170 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.171 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.171 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.171 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.171 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.171 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.172 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.172 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.172 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.174 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.174 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.174 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.175 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.175 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.175 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.176 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.176 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.176 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.177 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.177 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.178 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.178 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.178 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.178 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.178 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.179 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.179 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.179 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.179 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.179 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.180 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5827860832214355, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.180 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.180 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.180 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.180 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.181 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.181 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.181 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.181 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.181 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.182 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.182 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.182 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.182 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.182 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.183 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.183 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.183 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.183 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.183 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.183 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.184 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.184 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.184 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.184 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.184 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.185 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.185 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.185 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.186 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.186 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.186 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5833089351654053, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.186 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.186 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.187 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.187 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.187 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.187 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "password": { 2020-09-08T15:26:08.187 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "password", 2020-09-08T15:26:08.188 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.188 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.189 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.189 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.189 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.189 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.189 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.190 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.190 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.190 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.190 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.190 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.191 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_addr": { 2020-09-08T15:26:08.191 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_addr", 2020-09-08T15:26:08.191 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.191 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.191 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.191 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "::", 2020-09-08T15:26:08.192 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.192 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.192 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.192 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.192 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5838115215301514, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.193 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.193 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.193 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.193 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.194 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_port": { 2020-09-08T15:26:08.194 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_port", 2020-09-08T15:26:08.194 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.194 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.194 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.194 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "8080", 2020-09-08T15:26:08.195 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.195 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.195 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.195 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.195 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.196 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.196 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.196 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.196 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ssl": { 2020-09-08T15:26:08.196 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ssl", 2020-09-08T15:26:08.196 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.197 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.197 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.197 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.197 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.197 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.198 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.198 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.199 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.199 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.199 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5843396186828613, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.199 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.199 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.200 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ssl_server_port": { 2020-09-08T15:26:08.200 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ssl_server_port", 2020-09-08T15:26:08.200 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.200 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.200 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.200 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "8443", 2020-09-08T15:26:08.201 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.201 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.201 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.201 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.202 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.202 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.202 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.202 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.202 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "standby_behaviour": { 2020-09-08T15:26:08.202 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "standby_behaviour", 2020-09-08T15:26:08.203 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.203 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.203 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.203 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "redirect", 2020-09-08T15:26:08.203 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.204 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.204 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.204 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.204 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "redirect" 2020-09-08T15:26:08.204 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.205 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.205 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.205 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5848686695098877, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.205 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.205 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.206 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.206 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "standby_error_status_code": { 2020-09-08T15:26:08.206 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "standby_error_status_code", 2020-09-08T15:26:08.206 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.206 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.207 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.207 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "500", 2020-09-08T15:26:08.207 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "400", 2020-09-08T15:26:08.207 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "599", 2020-09-08T15:26:08.207 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.208 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.208 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.208 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.208 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.208 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.208 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "url_prefix": { 2020-09-08T15:26:08.209 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "url_prefix", 2020-09-08T15:26:08.209 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.210 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.210 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.210 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.210 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.210 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.211 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.211 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.211 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.211 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5853681564331055, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.211 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.212 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.212 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.212 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "username": { 2020-09-08T15:26:08.212 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "username", 2020-09-08T15:26:08.212 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.213 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.213 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.213 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.213 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.213 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.213 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.214 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.214 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.214 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.214 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.214 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.215 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.215 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.215 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.215 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "devicehealth", 2020-09-08T15:26:08.216 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.216 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.216 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.216 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enable_monitoring": { 2020-09-08T15:26:08.216 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "enable_monitoring", 2020-09-08T15:26:08.217 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.217 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.217 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.217 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.217 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.218 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.218 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.585928201675415, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.218 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.218 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "monitor device health metrics", 2020-09-08T15:26:08.218 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.219 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.219 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.219 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.220 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.220 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.220 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.220 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.221 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.221 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.221 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.221 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.221 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.222 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.222 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.222 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.222 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.223 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.223 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.223 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.223 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.223 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.224 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.224 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.224 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.224 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.224 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.224 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.225 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5864577293395996, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.225 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.225 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.225 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.225 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.226 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.226 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.226 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.226 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.226 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.227 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.227 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.227 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.227 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.227 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.228 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.228 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.228 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.228 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.228 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.228 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.229 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.229 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.229 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.229 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.229 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.230 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.230 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.231 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.231 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.231 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5869629383087158, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.231 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.232 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.232 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.232 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.232 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.232 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.233 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.233 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.233 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.233 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.233 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.234 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.234 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.234 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.234 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.234 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.235 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.235 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mark_out_threshold": { 2020-09-08T15:26:08.235 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "mark_out_threshold", 2020-09-08T15:26:08.235 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:08.235 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.235 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.236 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "2419200", 2020-09-08T15:26:08.236 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.236 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.236 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.236 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "automatically mark OSD if it may fail before this long", 2020-09-08T15:26:08.237 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.237 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.587456226348877, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.237 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.237 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.237 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.238 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pool_name": { 2020-09-08T15:26:08.238 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "pool_name", 2020-09-08T15:26:08.238 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.238 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.238 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.239 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "device_health_metrics", 2020-09-08T15:26:08.239 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.239 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.239 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.239 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "name of pool in which to store device health metrics", 2020-09-08T15:26:08.239 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.240 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.240 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.240 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.241 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "retention_period": { 2020-09-08T15:26:08.242 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "retention_period", 2020-09-08T15:26:08.242 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:08.242 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.243 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.243 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "15552000", 2020-09-08T15:26:08.243 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.243 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.244 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.244 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5879168510437012, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.244 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how long to retain device health metrics", 2020-09-08T15:26:08.244 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.244 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.245 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.245 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.245 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "scrape_frequency": { 2020-09-08T15:26:08.245 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "scrape_frequency", 2020-09-08T15:26:08.245 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:08.246 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.246 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.246 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "86400", 2020-09-08T15:26:08.246 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.246 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.247 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.247 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how frequently to scrape device health metrics", 2020-09-08T15:26:08.247 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.247 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.247 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.247 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.248 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "self_heal": { 2020-09-08T15:26:08.248 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "self_heal", 2020-09-08T15:26:08.248 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.248 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.248 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.249 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.249 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.249 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.249 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.249 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5884058475494385, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.250 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "preemptively heal cluster around devices that may fail", 2020-09-08T15:26:08.250 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.250 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.250 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.250 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.251 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "sleep_interval": { 2020-09-08T15:26:08.251 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "sleep_interval", 2020-09-08T15:26:08.251 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:08.251 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.252 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.252 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "600", 2020-09-08T15:26:08.252 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.253 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.253 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.253 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how frequently to wake up and check device health", 2020-09-08T15:26:08.253 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.254 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.254 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.254 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.254 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warn_threshold": { 2020-09-08T15:26:08.255 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "warn_threshold", 2020-09-08T15:26:08.255 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:08.255 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.255 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.255 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "7257600", 2020-09-08T15:26:08.256 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.256 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.58905029296875, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.256 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.256 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.256 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "raise health warning if OSD may fail before this long", 2020-09-08T15:26:08.257 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.257 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.257 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.257 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.257 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.258 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.258 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.258 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "diskprediction_local", 2020-09-08T15:26:08.258 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.258 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.259 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.259 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.259 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.259 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.259 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.259 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.260 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.260 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.260 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.260 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.261 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.261 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.261 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.261 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.261 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.262 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.262 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.263 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.263 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5895709991455078, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.263 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.263 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.263 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.264 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.264 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.264 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.264 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.264 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.265 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.265 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.265 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.265 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.265 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.265 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.266 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.266 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.266 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.266 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.266 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.267 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.267 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.267 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.267 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.268 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.268 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.268 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.268 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.270 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.270 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.270 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5900442600250244, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.271 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.271 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.271 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.271 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.271 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.272 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.272 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.272 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.272 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.273 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.273 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.273 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.273 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.274 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.274 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.274 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.274 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.274 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.275 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.275 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.275 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.275 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.275 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.276 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.276 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "predict_interval": { 2020-09-08T15:26:08.276 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "predict_interval", 2020-09-08T15:26:08.276 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.277 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.277 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.277 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5905189514160156, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.277 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "86400", 2020-09-08T15:26:08.277 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.278 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.278 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.278 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.278 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.278 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.279 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.279 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.279 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "predictor_model": { 2020-09-08T15:26:08.279 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "predictor_model", 2020-09-08T15:26:08.279 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.280 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.280 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.280 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "prophetstor", 2020-09-08T15:26:08.280 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.280 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.281 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.281 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.290 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.290 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.290 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.290 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.290 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "sleep_interval": { 2020-09-08T15:26:08.291 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "sleep_interval", 2020-09-08T15:26:08.291 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.291 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.291 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.291 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "600", 2020-09-08T15:26:08.292 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5910274982452393, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.292 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.292 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.292 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.292 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.292 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.293 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.293 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.293 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.293 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.293 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.294 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.294 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "influx", 2020-09-08T15:26:08.294 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": false, 2020-09-08T15:26:08.294 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "influxdb python module not found", 2020-09-08T15:26:08.294 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.295 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "batch_size": { 2020-09-08T15:26:08.295 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "batch_size", 2020-09-08T15:26:08.295 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.295 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.295 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.296 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "5000", 2020-09-08T15:26:08.296 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.296 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.296 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.296 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.296 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.297 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.297 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.297 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.297 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "database": { 2020-09-08T15:26:08.297 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "database", 2020-09-08T15:26:08.298 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.298 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5915708541870117, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.298 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.298 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.298 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "ceph", 2020-09-08T15:26:08.299 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.299 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.299 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.300 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.300 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.300 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.300 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.300 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.301 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "hostname": { 2020-09-08T15:26:08.301 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "hostname", 2020-09-08T15:26:08.301 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.301 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.301 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.302 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:08.302 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.302 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.302 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.302 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.303 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.303 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.303 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.303 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.303 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "interval": { 2020-09-08T15:26:08.303 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "interval", 2020-09-08T15:26:08.304 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.304 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.304 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.304 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5920772552490234, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.304 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "30", 2020-09-08T15:26:08.305 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.305 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.305 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.305 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.305 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.306 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.306 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.306 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.306 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.306 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.306 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.307 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.307 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.307 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.307 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.307 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.308 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.308 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.308 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.308 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.308 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.313 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.313 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.314 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.314 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.314 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.314 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.314 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.315 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.315 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5926847457885742, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.315 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.315 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.315 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.316 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.316 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.316 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.317 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.317 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.317 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.317 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.317 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.318 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.318 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.318 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.319 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.319 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.320 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.320 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.320 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.320 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.320 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.321 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.321 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.321 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.321 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.321 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.322 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.322 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.322 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.322 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5934293270111084, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.322 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.323 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.323 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.324 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.324 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.324 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.324 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.324 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.324 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.325 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.325 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.325 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.325 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.325 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.326 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.326 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.326 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.326 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.326 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.327 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.327 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "password": { 2020-09-08T15:26:08.327 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "password", 2020-09-08T15:26:08.327 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.327 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.328 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.328 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:08.328 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.328 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.328 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.329 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.329 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5941553115844727, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.329 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.330 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.330 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.330 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.330 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "port": { 2020-09-08T15:26:08.331 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "port", 2020-09-08T15:26:08.331 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.331 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.331 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.331 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "8086", 2020-09-08T15:26:08.332 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.332 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.332 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.332 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.334 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.334 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.334 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.335 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.335 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ssl": { 2020-09-08T15:26:08.335 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ssl", 2020-09-08T15:26:08.335 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.335 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.336 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.336 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "false", 2020-09-08T15:26:08.336 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.336 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.336 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.337 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.337 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.337 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.337 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5946509838104248, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.337 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.338 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.338 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "threads": { 2020-09-08T15:26:08.338 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "threads", 2020-09-08T15:26:08.338 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.338 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.338 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.339 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "5", 2020-09-08T15:26:08.339 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.339 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.339 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.339 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.340 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.340 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.340 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.340 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.340 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "username": { 2020-09-08T15:26:08.341 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "username", 2020-09-08T15:26:08.341 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.341 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.341 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.341 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:08.342 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.342 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.342 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.342 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.343 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.343 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.343 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.343 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.344 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "verify_ssl": { 2020-09-08T15:26:08.344 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.595151662826538, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.344 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "verify_ssl", 2020-09-08T15:26:08.344 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.345 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.345 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.346 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "true", 2020-09-08T15:26:08.346 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.346 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.346 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.347 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.347 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.347 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.347 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.347 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.348 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.348 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.348 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.349 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "insights", 2020-09-08T15:26:08.350 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.350 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.350 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.351 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.351 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.351 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.351 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.353 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.353 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.353 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.353 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.595667839050293, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.353 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.353 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.354 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.354 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.354 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.354 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.355 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.355 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.355 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.357 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.357 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.357 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.357 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.357 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.358 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.358 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.358 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.358 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.358 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.359 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.359 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.359 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.359 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.359 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5961782932281494, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.360 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.360 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.360 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.360 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.360 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.362 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.362 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.362 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.362 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.362 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.363 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.363 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.364 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.364 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.365 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.365 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.365 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.365 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.370 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.371 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.371 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.371 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.371 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.371 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.372 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.372 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.372 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "iostat", 2020-09-08T15:26:08.372 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5967373847961426, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.372 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.373 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.373 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.373 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.373 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.373 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.374 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.374 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.374 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.374 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.374 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.375 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.375 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.375 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.375 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.376 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.376 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.376 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.376 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.376 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.376 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.377 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.377 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.377 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.377 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.377 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.378 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.378 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.378 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.378 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.378 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5972323417663574, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.379 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.379 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.379 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.379 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.379 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.380 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.380 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.380 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.381 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.381 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.381 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.381 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.381 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.382 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.382 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.382 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.382 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.382 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.383 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.383 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.383 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.383 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.383 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.384 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.384 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.384 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.384 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.384 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.385 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.385 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.385 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.597719669342041, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.385 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.385 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.385 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.386 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.386 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.386 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.386 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.386 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.387 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.387 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.387 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.387 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.387 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.388 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.388 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.388 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.388 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "k8sevents", 2020-09-08T15:26:08.388 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.389 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.389 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.389 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ceph_event_retention_days": { 2020-09-08T15:26:08.389 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ceph_event_retention_days", 2020-09-08T15:26:08.389 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.390 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.390 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.390 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "7", 2020-09-08T15:26:08.390 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.390 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.391 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.391 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5982286930084229, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.392 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Days to hold ceph event information within local cache", 2020-09-08T15:26:08.392 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.392 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.392 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.392 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.393 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "config_check_secs": { 2020-09-08T15:26:08.393 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "config_check_secs", 2020-09-08T15:26:08.393 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.393 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.393 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.394 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "10", 2020-09-08T15:26:08.394 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "10", 2020-09-08T15:26:08.394 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.394 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.394 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "interval (secs) to check for cluster configuration changes", 2020-09-08T15:26:08.395 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.395 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.395 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.395 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.395 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.395 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.396 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.396 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.396 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.396 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.396 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.397 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.397 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.397 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.598693609237671, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.397 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.397 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.398 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.398 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.398 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.398 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.398 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.399 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.399 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.399 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.399 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.399 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.400 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.400 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.400 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.400 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.400 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.401 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.401 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.401 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.402 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.402 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.402 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.402 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.402 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.403 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.403 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.403 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.403 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.403 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.404 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5991771221160889, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.404 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.404 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.404 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.404 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.404 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.405 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.405 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.405 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.405 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.405 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.406 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.406 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.406 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.406 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.406 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.407 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.407 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.407 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.407 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.407 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.407 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.408 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.408 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.408 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.408 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.408 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.409 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.409 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.409 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.409 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.409 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5996663570404053, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.410 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.410 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.410 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.410 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.410 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "localpool", 2020-09-08T15:26:08.411 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.411 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.411 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.411 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "failure_domain": { 2020-09-08T15:26:08.411 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "failure_domain", 2020-09-08T15:26:08.412 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.412 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.412 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.413 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "host", 2020-09-08T15:26:08.413 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.413 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.413 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.413 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "failure domain for any created local pool", 2020-09-08T15:26:08.414 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.414 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.414 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.414 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.414 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.415 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.415 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.415 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.415 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.415 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.416 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.416 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.416 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6001689434051514, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.416 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.416 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.417 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.417 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.417 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.417 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.417 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.417 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.418 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.418 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.418 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.418 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.418 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.419 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.419 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.419 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.419 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.419 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.419 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.420 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.420 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.420 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.420 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.421 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.421 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.421 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.421 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.422 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.422 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.422 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.423 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.600679874420166, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.423 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.423 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.423 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.424 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.424 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.424 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.424 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.424 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.425 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.425 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.425 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.425 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.425 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.426 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.426 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.426 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.426 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.426 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.427 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.427 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.427 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.427 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.427 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.427 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.428 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.428 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.428 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.428 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.428 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.429 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.429 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6011762619018555, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.429 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.429 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.429 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min_size": { 2020-09-08T15:26:08.430 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "min_size", 2020-09-08T15:26:08.430 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.430 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.430 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.430 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.430 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.431 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.431 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.431 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "default min_size for any created local pool", 2020-09-08T15:26:08.431 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.431 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.432 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.432 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.432 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_rep": { 2020-09-08T15:26:08.432 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "num_rep", 2020-09-08T15:26:08.432 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.436 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.436 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.436 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "3", 2020-09-08T15:26:08.437 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.437 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.437 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.437 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "default replica count for any created local pool", 2020-09-08T15:26:08.438 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.438 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.438 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.601639986038208, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.438 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.438 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.439 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pg_num": { 2020-09-08T15:26:08.439 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "pg_num", 2020-09-08T15:26:08.439 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.439 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.440 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.440 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "128", 2020-09-08T15:26:08.440 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.440 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.440 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.440 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "default pg_num for any created local pool", 2020-09-08T15:26:08.441 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.441 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.441 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.441 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.441 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "prefix": { 2020-09-08T15:26:08.442 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "prefix", 2020-09-08T15:26:08.442 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.442 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.442 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.442 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.443 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.443 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.443 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.443 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "name prefix for any created local pool", 2020-09-08T15:26:08.443 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.444 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.444 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.60213303565979, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.444 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.444 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.444 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "subtree": { 2020-09-08T15:26:08.444 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "subtree", 2020-09-08T15:26:08.445 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.445 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.445 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.445 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "rack", 2020-09-08T15:26:08.445 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.446 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.447 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.447 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "CRUSH level for which to create a local pool", 2020-09-08T15:26:08.447 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.447 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.448 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.448 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.448 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.448 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.448 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.448 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "mds_autoscaler", 2020-09-08T15:26:08.449 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.449 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.449 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.449 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.450 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.450 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.450 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.450 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.450 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.451 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.451 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.451 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6026997566223145, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.451 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.451 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.452 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.452 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.452 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.452 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.452 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.452 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.453 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.453 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.453 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.453 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.453 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.454 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.454 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.454 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.454 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.454 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.455 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.455 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.455 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.455 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.455 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.456 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.456 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.456 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.459 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.459 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.459 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.459 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.603236436843872, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.459 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.460 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.460 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.460 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.460 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.460 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.460 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.461 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.461 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.461 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.461 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.461 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.462 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.462 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.462 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.462 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.462 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.463 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.463 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.463 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.463 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.463 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.464 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.464 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.464 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.464 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.469 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.469 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.469 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.469 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.470 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6037681102752686, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.470 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.470 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.470 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.470 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.471 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.471 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.471 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "orchestrator", 2020-09-08T15:26:08.471 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.471 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.472 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.472 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.472 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.472 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.472 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.473 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.473 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.473 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.473 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.473 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.474 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.474 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.474 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.474 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.474 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.474 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.475 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.475 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.475 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.475 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.475 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.476 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.476 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.476 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.476 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6043550968170166, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.476 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.477 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.477 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.477 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.477 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.477 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.478 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.478 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.479 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.479 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.479 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.479 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.479 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.480 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.480 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.480 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.480 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.480 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.481 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.481 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.481 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.481 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.481 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.482 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.482 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.482 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.482 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.482 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.482 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.483 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6048767566680908, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.483 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.483 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.483 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.483 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.484 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.484 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.484 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.484 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.484 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.485 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.485 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.485 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.485 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.485 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.485 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.486 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.486 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.486 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.486 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "orchestrator": { 2020-09-08T15:26:08.486 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "orchestrator", 2020-09-08T15:26:08.487 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.487 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.487 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.487 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:08.487 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.488 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.488 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.488 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "cephadm", 2020-09-08T15:26:08.488 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rook", 2020-09-08T15:26:08.488 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "test_orchestrator" 2020-09-08T15:26:08.489 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.605400800704956, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.489 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.489 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Orchestrator backend", 2020-09-08T15:26:08.489 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.489 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.490 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.490 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.494 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.494 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.494 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.495 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "osd_support", 2020-09-08T15:26:08.495 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.495 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.495 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.495 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.496 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.496 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.496 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.496 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.496 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.497 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.497 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.498 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.498 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.498 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.498 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.498 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.499 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.499 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.499 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.499 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.499 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.500 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.500 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.605942726135254, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.500 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.500 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.500 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.500 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.501 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.501 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.501 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.501 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.501 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.502 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.502 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.502 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.502 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.502 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.503 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.503 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.503 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.506 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.506 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.506 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.506 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.507 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.507 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.507 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.507 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.507 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.508 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.508 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.508 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.508 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6064281463623047, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.508 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.508 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.509 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.509 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.510 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.510 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.510 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.510 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.510 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.511 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.511 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.511 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.511 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.511 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.512 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.512 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.512 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.512 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.512 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.512 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.513 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.513 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.513 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.513 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.513 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.514 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "pg_autoscaler", 2020-09-08T15:26:08.514 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.514 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.514 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.514 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.515 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.515 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.515 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.515 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6069765090942383, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.515 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.516 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.516 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.516 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.516 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.516 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.516 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.517 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.517 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.517 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.517 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.517 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.518 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.518 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.518 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.518 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.518 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.519 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.519 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.519 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.521 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.521 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.521 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.521 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.521 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.522 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.522 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.522 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.522 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.522 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.523 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6074819564819336, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.523 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.523 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.523 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.524 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.524 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.524 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.524 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.524 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.525 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.525 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.525 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.525 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.525 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.526 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.526 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.526 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.526 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.526 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.526 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.527 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.527 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.527 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.527 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.527 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.528 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.528 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.528 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.528 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.528 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.529 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6080098152160645, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.529 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.529 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.529 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.529 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.530 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.530 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.530 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.531 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "sleep_interval": { 2020-09-08T15:26:08.531 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "sleep_interval", 2020-09-08T15:26:08.531 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.531 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.531 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.531 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "60", 2020-09-08T15:26:08.532 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.532 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.532 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.532 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.532 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.533 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.533 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.533 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.533 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.533 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.534 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.534 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "progress", 2020-09-08T15:26:08.534 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.534 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.534 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.534 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.535 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.535 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.535 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.535 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6085805892944336, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.536 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.536 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.536 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.536 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.536 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.536 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.537 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.537 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.537 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.537 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.537 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.538 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.538 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.538 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.538 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.538 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.539 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.539 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.539 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.539 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.540 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.540 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.540 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.541 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.541 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.541 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.541 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.541 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.542 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.542 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.542 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.542 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6091148853302002, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.542 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.543 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.543 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.543 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.543 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.543 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.544 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.544 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.544 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.544 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.545 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.545 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.545 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.545 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.545 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.546 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.546 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.546 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.546 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.546 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.546 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.547 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.547 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.547 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.547 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.547 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.548 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.548 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.548 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.548 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6096000671386719, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.548 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.549 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.549 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.549 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.549 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.549 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.549 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max_completed_events": { 2020-09-08T15:26:08.550 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "max_completed_events", 2020-09-08T15:26:08.550 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.550 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.551 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.551 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "50", 2020-09-08T15:26:08.551 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.551 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.551 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.552 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "number of past completed events to remember", 2020-09-08T15:26:08.552 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.552 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.552 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.553 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.553 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "persist_interval": { 2020-09-08T15:26:08.553 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "persist_interval", 2020-09-08T15:26:08.553 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:08.553 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.554 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.554 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "5", 2020-09-08T15:26:08.554 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.554 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.554 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6100664138793945, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.555 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.555 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how frequently to persist completed events", 2020-09-08T15:26:08.555 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.555 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.555 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.556 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.556 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.556 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.556 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.556 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "prometheus", 2020-09-08T15:26:08.556 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.557 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.557 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.557 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.557 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.557 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.558 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.558 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.558 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.558 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.558 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.559 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.559 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.559 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.559 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.559 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.559 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.560 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.560 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.560 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.560 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.560 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6105737686157227, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.561 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.561 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.561 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.562 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.562 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.562 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.562 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.562 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.563 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.563 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.563 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.563 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.563 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.564 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.564 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.564 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.564 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.564 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.564 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.565 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.565 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.565 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.565 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.565 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.566 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.566 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.566 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.566 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.566 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.567 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6110994815826416, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.567 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.567 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.567 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.567 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.567 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.568 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.568 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.568 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.568 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.568 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.569 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.569 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.569 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.569 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.569 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.570 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.570 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.570 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.570 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.570 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.570 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.571 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.571 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.571 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rbd_stats_pools": { 2020-09-08T15:26:08.572 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rbd_stats_pools", 2020-09-08T15:26:08.572 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.574 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.574 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.575 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.575 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.575 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6118786334991455, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.575 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.575 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.575 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.576 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.576 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.576 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.576 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.576 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rbd_stats_pools_refresh_interval": { 2020-09-08T15:26:08.577 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rbd_stats_pools_refresh_interval", 2020-09-08T15:26:08.577 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.577 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.577 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.577 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "300", 2020-09-08T15:26:08.578 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.578 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.578 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.578 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.578 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.578 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.579 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.579 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.579 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "scrape_interval": { 2020-09-08T15:26:08.579 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "scrape_interval", 2020-09-08T15:26:08.579 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.580 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.580 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.580 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.580 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.580 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.612550973892212, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.581 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.581 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.581 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.581 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.581 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.582 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.582 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.582 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_addr": { 2020-09-08T15:26:08.582 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_addr", 2020-09-08T15:26:08.583 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.583 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.583 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.583 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.583 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.584 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.584 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.584 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.584 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.584 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.585 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.585 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.585 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_port": { 2020-09-08T15:26:08.585 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_port", 2020-09-08T15:26:08.585 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.586 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.586 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.586 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.586 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.586 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.587 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.587 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6131484508514404, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.587 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.587 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.587 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.587 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.588 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.588 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "stale_cache_strategy": { 2020-09-08T15:26:08.588 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "stale_cache_strategy", 2020-09-08T15:26:08.588 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.588 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.589 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.589 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.589 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.589 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.589 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.590 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.590 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.590 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.590 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.590 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.590 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.591 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.591 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.591 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rbd_support", 2020-09-08T15:26:08.591 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.591 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.592 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.592 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.592 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.592 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.593 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.593 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.593 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.593 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6137664318084717, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.593 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.594 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.594 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.594 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.594 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.594 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.595 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.595 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.595 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.595 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.595 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.596 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.596 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.596 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.596 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.596 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.596 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.597 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.597 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.597 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.597 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.597 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.598 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.598 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.598 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.598 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.598 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.599 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.599 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.599 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.599 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6143088340759277, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.599 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.599 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.600 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.600 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.600 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.600 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.600 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.601 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.601 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.601 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.601 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.601 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.602 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.602 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.602 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.603 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.603 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.603 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.603 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.604 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.604 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.604 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.604 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.604 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.604 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.605 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.605 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.605 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.605 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.605 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.614943504333496, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.606 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.606 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.606 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.606 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.606 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.607 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mirror_snapshot_schedule": { 2020-09-08T15:26:08.607 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "mirror_snapshot_schedule", 2020-09-08T15:26:08.607 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.607 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.607 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.608 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.608 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.608 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.608 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.608 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.608 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.609 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.609 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.609 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.609 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "trash_purge_schedule": { 2020-09-08T15:26:08.609 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "trash_purge_schedule", 2020-09-08T15:26:08.610 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.610 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.610 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.610 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.610 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.611 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.611 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.611 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.611 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.615471363067627, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.611 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.611 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.612 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.612 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.612 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.612 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.612 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.613 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "restful", 2020-09-08T15:26:08.613 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.613 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.614 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.614 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enable_auth": { 2020-09-08T15:26:08.614 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "enable_auth", 2020-09-08T15:26:08.614 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.614 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.615 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.615 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.615 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.615 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.615 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.615 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.616 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.616 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.616 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.616 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.616 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "key_file": { 2020-09-08T15:26:08.617 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "key_file", 2020-09-08T15:26:08.617 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.617 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.617 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.617 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.618 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.618 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6160242557525635, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.618 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.618 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.618 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.618 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.619 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.619 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.619 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.619 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.619 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.620 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.620 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.620 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.620 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.620 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.621 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.621 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.621 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.621 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.621 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.621 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.622 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.622 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.622 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.622 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.622 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.623 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.623 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.623 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.624 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.624 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.624 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.624 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6166212558746338, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.624 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.625 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.625 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.625 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.625 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.625 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.626 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.626 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.626 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.626 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.626 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.626 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.627 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.627 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.627 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.627 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.627 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.628 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.628 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.628 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.628 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.628 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.628 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.629 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.629 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.629 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.629 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.629 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.630 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.630 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6172385215759277, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.630 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.630 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.631 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.631 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.631 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.631 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.631 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.632 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.632 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.632 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.632 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.632 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.632 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.633 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.633 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.633 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.634 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.634 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_addr": { 2020-09-08T15:26:08.634 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_addr", 2020-09-08T15:26:08.634 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.634 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.635 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.635 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.635 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.635 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.635 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.636 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.636 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.636 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.636 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.636 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6179008483886719, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.637 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.637 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_port": { 2020-09-08T15:26:08.637 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_port", 2020-09-08T15:26:08.637 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.637 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.638 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.638 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.638 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.638 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.638 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.639 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.639 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.639 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.639 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.639 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.640 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.640 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.640 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.640 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rook", 2020-09-08T15:26:08.641 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.641 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.641 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.641 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.641 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.642 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.642 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.642 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.642 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.642 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.643 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.643 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.643 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.643 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6184461116790771, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.643 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.644 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.644 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.644 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.645 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.645 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.645 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.645 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.646 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.646 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.646 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.646 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.646 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.647 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.647 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.647 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.647 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.647 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.648 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.648 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.648 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.648 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.648 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.648 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.649 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.649 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.649 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.649 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.649 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.650 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.650 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6189405918121338, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.650 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.650 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.650 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.651 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.651 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.651 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.651 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.651 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.651 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.652 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.652 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.652 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.652 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.653 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.653 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.653 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.653 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.653 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.654 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.654 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.654 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.655 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.655 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.655 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.655 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.655 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.656 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.656 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.656 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.656 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.656 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.657 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6194496154785156, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.657 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.657 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.657 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "selftest", 2020-09-08T15:26:08.657 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.657 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.658 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.658 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.658 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.658 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.658 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.659 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.659 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.659 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.659 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.659 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.659 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.660 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.660 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.660 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.660 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.660 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.661 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.661 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.661 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.661 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.661 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.662 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.662 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.662 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.662 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.662 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.662 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6199603080749512, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.663 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.663 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.663 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.663 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.663 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.664 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.664 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.664 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.664 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.665 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.665 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.665 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.665 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.666 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.666 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.666 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.666 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.666 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.666 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.667 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.667 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.667 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.667 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.667 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.668 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.668 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.668 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.668 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.668 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.669 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.669 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.620457410812378, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.669 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.669 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.669 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.669 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.670 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.670 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.670 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.670 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.670 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.671 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.671 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.671 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.671 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.671 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.672 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.672 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "roption1": { 2020-09-08T15:26:08.672 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "roption1", 2020-09-08T15:26:08.672 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.673 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.673 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.674 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.674 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.674 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.674 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.674 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.674 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.675 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.675 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.675 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.676 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "roption2": { 2020-09-08T15:26:08.676 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6209983825683594, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.676 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "roption2", 2020-09-08T15:26:08.676 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.676 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.677 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.677 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "xyz", 2020-09-08T15:26:08.677 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.677 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.677 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.678 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.678 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.678 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.678 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.678 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.678 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption1": { 2020-09-08T15:26:08.679 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption1", 2020-09-08T15:26:08.679 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.679 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.679 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.679 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.680 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.680 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.680 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.680 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.680 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.681 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.681 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.681 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.681 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption2": { 2020-09-08T15:26:08.681 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption2", 2020-09-08T15:26:08.682 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.682 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6215250492095947, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.682 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.682 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.683 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.683 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.683 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.683 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.683 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.683 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.684 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.684 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.684 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.684 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption3": { 2020-09-08T15:26:08.684 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption3", 2020-09-08T15:26:08.685 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "float", 2020-09-08T15:26:08.685 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.685 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.685 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.686 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.686 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.686 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.686 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.687 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.687 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.687 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.687 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.687 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption4": { 2020-09-08T15:26:08.687 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption4", 2020-09-08T15:26:08.688 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.688 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.688 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.688 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6220638751983643, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.688 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.689 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.689 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.689 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.689 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.689 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.690 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.690 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.690 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.690 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption5": { 2020-09-08T15:26:08.690 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption5", 2020-09-08T15:26:08.690 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.691 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.691 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.691 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.691 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.691 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.692 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.692 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.692 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.693 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.693 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.693 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.693 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption6": { 2020-09-08T15:26:08.693 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption6", 2020-09-08T15:26:08.694 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.694 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.694 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.694 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.694 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.695 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.622600793838501, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.695 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.695 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.695 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.695 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.695 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.696 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.696 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.696 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "testkey": { 2020-09-08T15:26:08.697 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "testkey", 2020-09-08T15:26:08.697 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.697 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.697 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.697 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.698 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.698 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.698 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.698 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.698 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.698 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.699 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.699 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.699 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "testlkey": { 2020-09-08T15:26:08.699 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "testlkey", 2020-09-08T15:26:08.699 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.700 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.700 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.700 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.700 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.700 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.701 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.701 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6231319904327393, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.701 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.701 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.701 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.701 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.702 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.702 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "testnewline": { 2020-09-08T15:26:08.702 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "testnewline", 2020-09-08T15:26:08.702 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.702 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.703 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.703 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.704 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.704 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.704 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.705 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.705 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.705 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.705 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.705 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.706 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.706 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.706 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.706 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "snap_schedule", 2020-09-08T15:26:08.707 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.707 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.707 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.707 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.707 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.708 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.708 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.708 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.708 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.708 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6236929893493652, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.709 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.709 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.709 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.709 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.709 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.710 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.710 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.710 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.710 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.710 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.710 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.711 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.711 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.711 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.711 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.711 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.712 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.712 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.712 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.712 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.712 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.713 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.713 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.713 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.713 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.713 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.713 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.714 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.714 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.714 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.714 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.624225378036499, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.714 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.715 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.715 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.715 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.715 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.715 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.716 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.716 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.716 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.716 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.716 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.717 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.717 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.717 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.717 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.718 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.718 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.718 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.718 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.718 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.719 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.719 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.719 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.719 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.719 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.719 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.720 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.720 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.720 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.720 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.720 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6247689723968506, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.721 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.721 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.721 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.721 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.721 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.722 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.722 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.722 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "status", 2020-09-08T15:26:08.722 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.722 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.722 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.723 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.723 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.723 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.723 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.723 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.724 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.724 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.724 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.724 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.724 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.725 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.725 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.725 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.725 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.725 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.725 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.726 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.726 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.726 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.726 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.726 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.727 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.727 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.625361442565918, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.727 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.728 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.728 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.728 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.728 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.728 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.729 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.729 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.729 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.729 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.729 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.730 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.730 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.730 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.730 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.730 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.730 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.731 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.731 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.731 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.731 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.731 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.732 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.732 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.732 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.732 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.732 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.733 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.733 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.733 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6258761882781982, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.733 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.733 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.733 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.734 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.734 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.734 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.734 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.736 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.736 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.736 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.736 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.737 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.737 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.737 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.737 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.737 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.738 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.738 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.738 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.738 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.738 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.739 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.739 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "telegraf", 2020-09-08T15:26:08.739 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.739 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.739 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.740 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "address": { 2020-09-08T15:26:08.740 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "address", 2020-09-08T15:26:08.740 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.740 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.740 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.741 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.626427412033081, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.741 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "unixgram:///tmp/telegraf.sock", 2020-09-08T15:26:08.741 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.741 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.741 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.742 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.742 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.742 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.742 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.742 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.743 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "interval": { 2020-09-08T15:26:08.743 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "interval", 2020-09-08T15:26:08.743 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:08.743 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.743 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.744 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "15", 2020-09-08T15:26:08.744 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.744 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.744 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.744 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.744 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.745 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.745 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.745 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.745 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.746 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.747 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.747 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.748 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.748 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.748 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.748 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.626953125, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.748 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.749 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.749 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.750 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.750 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.750 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.750 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.750 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.751 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.751 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.751 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.751 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.751 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.752 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.752 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.752 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.752 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.752 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.753 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.753 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.753 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.753 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.753 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.754 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.754 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.754 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.754 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.754 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.755 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.755 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.755 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6275537014007568, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.755 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.755 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.756 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.756 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.756 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.756 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.756 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.756 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.757 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.757 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.757 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.757 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.757 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.758 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.758 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.758 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.758 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.758 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.759 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.759 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.759 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.759 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.759 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.760 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.760 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.760 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.760 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.760 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.760 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.761 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.761 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.628159523010254, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.761 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.761 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.761 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.762 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.762 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.762 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.762 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "telemetry", 2020-09-08T15:26:08.762 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.763 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.763 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.763 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "channel_basic": { 2020-09-08T15:26:08.763 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "channel_basic", 2020-09-08T15:26:08.763 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.763 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.764 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.764 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.764 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.764 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.764 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.765 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Share basic cluster information (size, version)", 2020-09-08T15:26:08.765 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.765 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.765 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.765 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.766 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "channel_crash": { 2020-09-08T15:26:08.766 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "channel_crash", 2020-09-08T15:26:08.766 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.766 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.766 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.766 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.767 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.628749132156372, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.767 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.767 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.767 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.767 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.768 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.768 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.768 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.768 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.768 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "channel_device": { 2020-09-08T15:26:08.768 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "channel_device", 2020-09-08T15:26:08.769 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.769 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.769 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.769 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:08.769 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.770 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.770 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.770 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.770 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.770 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.771 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.771 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.771 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "channel_ident": { 2020-09-08T15:26:08.771 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "channel_ident", 2020-09-08T15:26:08.771 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.771 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.772 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.772 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.772 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.772 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6293039321899414, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.772 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.773 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.773 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.773 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.773 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.773 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.774 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.775 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "contact": { 2020-09-08T15:26:08.775 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "contact", 2020-09-08T15:26:08.775 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.776 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.776 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.776 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:08.776 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.776 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.777 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.777 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.777 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.777 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.777 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.778 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.778 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "description": { 2020-09-08T15:26:08.778 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "description", 2020-09-08T15:26:08.778 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.778 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.779 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.779 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:08.779 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.779 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.779 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.780 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6298775672912598, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.780 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.780 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.780 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.780 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.780 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.783 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "device_url": { 2020-09-08T15:26:08.783 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "device_url", 2020-09-08T15:26:08.783 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.783 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.784 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.784 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "https://telemetry.ceph.com/device", 2020-09-08T15:26:08.784 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.784 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.786 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.787 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.787 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.787 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.787 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.787 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.788 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enabled": { 2020-09-08T15:26:08.788 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "enabled", 2020-09-08T15:26:08.788 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.788 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.788 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.789 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.789 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.789 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.789 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.789 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.790 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6304283142089844, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.790 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.790 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.790 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.790 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.791 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "interval": { 2020-09-08T15:26:08.791 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "interval", 2020-09-08T15:26:08.791 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.791 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.791 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.791 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "24", 2020-09-08T15:26:08.792 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "8", 2020-09-08T15:26:08.792 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.792 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.792 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.792 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.793 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.793 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.793 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.793 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "last_opt_revision": { 2020-09-08T15:26:08.793 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "last_opt_revision", 2020-09-08T15:26:08.794 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.794 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.794 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.794 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "1", 2020-09-08T15:26:08.794 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.794 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.795 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.795 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.795 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.795 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.630981683731079, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.795 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.796 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.796 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.796 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "leaderboard": { 2020-09-08T15:26:08.796 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "leaderboard", 2020-09-08T15:26:08.796 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.797 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.797 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.797 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.797 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.797 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.798 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.798 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.798 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.798 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.798 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.798 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.799 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.799 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.799 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.799 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.799 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.800 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.800 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.800 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.800 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.801 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.801 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.801 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.801 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.801 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.631528615951538, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.802 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.802 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.802 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.802 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.802 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.803 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.803 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.803 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.803 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.803 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.804 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.804 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.804 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.804 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.804 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.804 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.805 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.805 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.805 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.805 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.805 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.806 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.806 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.806 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.806 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.806 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.807 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.807 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.807 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.807 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.807 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.63206148147583, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.808 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.808 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.809 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.809 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.809 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.810 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.810 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.810 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.810 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.810 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.811 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.811 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.811 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.811 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.812 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.813 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.813 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.813 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.813 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.814 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.814 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.814 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.814 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.814 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.815 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.815 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.815 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.815 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "organization": { 2020-09-08T15:26:08.815 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "organization", 2020-09-08T15:26:08.816 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.816 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.63260817527771, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.816 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.816 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.816 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:08.816 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.817 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.817 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.817 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.817 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.817 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.818 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.818 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.818 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "proxy": { 2020-09-08T15:26:08.818 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "proxy", 2020-09-08T15:26:08.818 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.819 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.819 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.819 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:08.819 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.819 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.819 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.820 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.820 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.820 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.820 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.820 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.821 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "url": { 2020-09-08T15:26:08.821 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "url", 2020-09-08T15:26:08.821 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.823 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.823 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.823 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6331443786621094, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.825 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "https://telemetry.ceph.com/report", 2020-09-08T15:26:08.825 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.825 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.825 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.825 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.826 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.826 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.826 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.826 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.826 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.827 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.827 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.827 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "test_orchestrator", 2020-09-08T15:26:08.827 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.827 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.827 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.828 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.828 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.828 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.828 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.828 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.829 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.829 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.829 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.829 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.829 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.830 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.830 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.830 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.830 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.830 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.830 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.831 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6337003707885742, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.831 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.831 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.831 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.831 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.832 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.832 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.832 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.832 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.833 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.833 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.833 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.833 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.833 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.834 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.834 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.834 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.834 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.834 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.846 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.847 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.847 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.847 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.848 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.848 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.848 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.848 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.848 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.849 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.849 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.849 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6342153549194336, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.849 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.849 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.850 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.850 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.850 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.850 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.851 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.851 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.851 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.851 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.851 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.851 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.852 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.852 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.852 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.852 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.852 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.853 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.853 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.853 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.853 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.853 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.854 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.854 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.854 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.854 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.854 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.855 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.855 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "volumes", 2020-09-08T15:26:08.855 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.855 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.855 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.858 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.858 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.634793996810913, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.859 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.859 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.859 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.859 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.859 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.860 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.860 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.860 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.860 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.860 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.861 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.861 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.861 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.861 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.861 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.862 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.862 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.862 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.862 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.862 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.863 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.863 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.863 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.863 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.863 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.864 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.864 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.864 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.864 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.864 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6353049278259277, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.865 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.865 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.865 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.865 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.865 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.865 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.866 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.866 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.866 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.866 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.866 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.867 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.867 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.867 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.867 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.867 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.868 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.868 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.868 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.880 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.881 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.881 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.881 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.881 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.882 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.882 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.882 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.882 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.882 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.883 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.883 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.635840892791748, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.883 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.883 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.883 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.884 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.884 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.884 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.884 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.884 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.884 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.885 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.885 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.885 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.885 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.885 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "zabbix", 2020-09-08T15:26:08.886 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:08.886 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:08.886 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:08.886 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "discovery_interval": { 2020-09-08T15:26:08.886 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "discovery_interval", 2020-09-08T15:26:08.887 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.887 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.887 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.887 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "100", 2020-09-08T15:26:08.887 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.888 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.888 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.888 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.888 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.888 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.888 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.889 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.889 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "identifier": { 2020-09-08T15:26:08.889 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6363990306854248, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.889 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "identifier", 2020-09-08T15:26:08.889 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.890 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.890 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.890 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.890 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.890 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.891 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.891 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.891 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.891 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.891 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.891 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.892 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "interval": { 2020-09-08T15:26:08.892 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "interval", 2020-09-08T15:26:08.892 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:08.892 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.892 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.893 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "60", 2020-09-08T15:26:08.893 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.893 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.893 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.893 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.894 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.894 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.894 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.895 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.895 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:08.895 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:08.898 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.899 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6369388103485107, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.899 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.899 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.899 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:08.900 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.900 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.900 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.900 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.900 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.901 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.901 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.901 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.901 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.901 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.902 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.903 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.903 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.903 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.903 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.903 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:08.904 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:08.904 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.904 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.904 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.904 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.905 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.905 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.905 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.905 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.905 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.910 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.910 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6374671459197998, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.910 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.911 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.911 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:08.911 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:08.911 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.911 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.912 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.912 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:08.912 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.912 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.912 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:08.913 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:08.913 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:08.913 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:08.913 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:08.913 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:08.913 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:08.914 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.914 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.914 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.914 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.915 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.915 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.915 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:08.915 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:08.915 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:08.916 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.916 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:08.916 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:08.916 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6379830837249756, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.916 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.917 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.917 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.917 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.917 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.918 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.918 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.918 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.918 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "zabbix_host": { 2020-09-08T15:26:08.918 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "zabbix_host", 2020-09-08T15:26:08.919 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.919 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.919 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.919 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:08.930 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.930 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.930 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.931 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.931 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.931 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.931 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.931 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.932 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "zabbix_port": { 2020-09-08T15:26:08.932 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "zabbix_port", 2020-09-08T15:26:08.932 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:08.932 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.932 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.933 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "10051", 2020-09-08T15:26:08.933 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.933 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.933 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6385087966918945, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.936 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.937 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.937 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.937 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.937 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.938 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.938 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "zabbix_sender": { 2020-09-08T15:26:08.938 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "zabbix_sender", 2020-09-08T15:26:08.938 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:08.938 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:08.938 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:08.939 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "/usr/bin/zabbix_sender", 2020-09-08T15:26:08.939 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:08.939 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:08.939 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:08.939 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:08.940 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:08.940 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:08.940 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:08.940 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.940 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.941 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.941 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.941 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "services": {}, 2020-09-08T15:26:08.941 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "always_on_modules": { 2020-09-08T15:26:08.941 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nautilus": [ 2020-09-08T15:26:08.942 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "balancer", 2020-09-08T15:26:08.942 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crash", 2020-09-08T15:26:08.943 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "devicehealth", 2020-09-08T15:26:08.943 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "orchestrator_cli", 2020-09-08T15:26:08.943 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "progress", 2020-09-08T15:26:08.943 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rbd_support", 2020-09-08T15:26:08.943 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "status", 2020-09-08T15:26:08.944 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "volumes" 2020-09-08T15:26:08.944 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.944 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "octopus": [ 2020-09-08T15:26:08.944 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6391336917877197, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.945 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "balancer", 2020-09-08T15:26:08.945 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crash", 2020-09-08T15:26:08.946 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "devicehealth", 2020-09-08T15:26:08.946 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "orchestrator", 2020-09-08T15:26:08.946 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pg_autoscaler", 2020-09-08T15:26:08.946 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "progress", 2020-09-08T15:26:08.947 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rbd_support", 2020-09-08T15:26:08.947 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "status", 2020-09-08T15:26:08.947 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "telemetry", 2020-09-08T15:26:08.947 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "volumes" 2020-09-08T15:26:08.947 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.948 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pacific": [ 2020-09-08T15:26:08.948 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "balancer", 2020-09-08T15:26:08.948 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crash", 2020-09-08T15:26:08.948 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "devicehealth", 2020-09-08T15:26:08.949 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "orchestrator", 2020-09-08T15:26:08.949 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pg_autoscaler", 2020-09-08T15:26:08.950 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "progress", 2020-09-08T15:26:08.950 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rbd_support", 2020-09-08T15:26:08.951 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "status", 2020-09-08T15:26:08.951 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "telemetry", 2020-09-08T15:26:08.951 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "volumes" 2020-09-08T15:26:08.951 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:08.952 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "last_failure_osd_epoch": 0, 2020-09-08T15:26:08.952 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_clients": [ 2020-09-08T15:26:08.952 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.952 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addrvec": [ 2020-09-08T15:26:08.952 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.952 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "v2", 2020-09-08T15:26:08.953 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addr": "172.21.15.104:0", 2020-09-08T15:26:08.953 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nonce": 1662035381 2020-09-08T15:26:08.953 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.953 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ] 2020-09-08T15:26:08.954 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.954 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.954 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addrvec": [ 2020-09-08T15:26:08.954 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.954 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "v2", 2020-09-08T15:26:08.955 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addr": "172.21.15.104:0", 2020-09-08T15:26:08.955 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6398074626922607, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.955 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nonce": 2169968102 2020-09-08T15:26:08.955 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.955 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ] 2020-09-08T15:26:08.956 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:08.956 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.956 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addrvec": [ 2020-09-08T15:26:08.957 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:08.957 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "v2", 2020-09-08T15:26:08.958 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addr": "172.21.15.104:0", 2020-09-08T15:26:08.958 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nonce": 3827328474 2020-09-08T15:26:08.958 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.958 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ] 2020-09-08T15:26:08.959 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.959 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ] 2020-09-08T15:26:08.959 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.959 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:08.959 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.6401052474975586, stop=False, exit=None, reads=[8] 2020-09-08T15:26:08.962 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:07 smithi104 bash[9203]: audit 2020-09-08T15:26:07.358915+0000 mon.a (mon.0) 41 : audit [DBG] from='client.? v1:172.21.15.104:0/3794204063' entity='client.admin' cmd=[{"prefix": "mgr dump"}]: dispatch 2020-09-08T15:26:08.963 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.159647226333618, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:26:08.963 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.1599040031433105, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:26:08.964 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Waiting for the mgr to restart... 2020-09-08T15:26:08.964 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Waiting for mgr epoch 5... 2020-09-08T15:26:08.964 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'tell', 'mgr', 'mgr_status'] 2020-09-08T15:26:08.964 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f tell mgr mgr_status 2020-09-08T15:26:10.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:09 smithi104 bash[9203]: cluster 2020-09-08T15:26:09.867192+0000 mon.a (mon.0) 42 : cluster [INF] Active manager daemon y restarted 2020-09-08T15:26:10.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:09 smithi104 bash[9203]: cluster 2020-09-08T15:26:09.867905+0000 mon.a (mon.0) 43 : cluster [INF] Activating manager daemon y 2020-09-08T15:26:10.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:09 smithi104 bash[9203]: cluster 2020-09-08T15:26:09.872131+0000 mon.a (mon.0) 44 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2020-09-08T15:26:11.002 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:11.002 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mgrmap_epoch": 7, 2020-09-08T15:26:11.002 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "initialized": true 2020-09-08T15:26:11.002 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:11.003 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=3.0211033821105957, stop=False, exit=None, reads=[8] 2020-09-08T15:26:11.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: cluster 2020-09-08T15:26:09.923730+0000 mon.a (mon.0) 45 : cluster [DBG] mgrmap e6: y(active, starting, since 0.0558948s) 2020-09-08T15:26:11.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: audit 2020-09-08T15:26:09.931244+0000 mon.a (mon.0) 46 : audit [DBG] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2020-09-08T15:26:11.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: audit 2020-09-08T15:26:09.932569+0000 mon.a (mon.0) 47 : audit [DBG] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2020-09-08T15:26:11.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: audit 2020-09-08T15:26:09.932751+0000 mon.a (mon.0) 48 : audit [DBG] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2020-09-08T15:26:11.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: audit 2020-09-08T15:26:09.932915+0000 mon.a (mon.0) 49 : audit [DBG] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2020-09-08T15:26:11.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: audit 2020-09-08T15:26:09.933347+0000 mon.a (mon.0) 50 : audit [DBG] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix": "config-key ls"}]: dispatch 2020-09-08T15:26:11.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: audit 2020-09-08T15:26:09.934364+0000 mon.a (mon.0) 51 : audit [DBG] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_user"}]: dispatch 2020-09-08T15:26:11.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: audit 2020-09-08T15:26:09.934668+0000 mon.a (mon.0) 52 : audit [DBG] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/report_id"}]: dispatch 2020-09-08T15:26:11.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: audit 2020-09-08T15:26:09.934946+0000 mon.a (mon.0) 53 : audit [DBG] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/salt"}]: dispatch 2020-09-08T15:26:11.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: cluster 2020-09-08T15:26:09.937982+0000 mon.a (mon.0) 54 : cluster [INF] Manager daemon y is now available 2020-09-08T15:26:11.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: audit 2020-09-08T15:26:09.955791+0000 mon.a (mon.0) 55 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix":"config set","value":"0","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2020-09-08T15:26:11.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: audit 2020-09-08T15:26:09.960945+0000 mon.a (mon.0) 56 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd='[{"prefix":"config set","value":"0","who":"mgr","name":"mgr/cephadm/migration_current"}]': finished 2020-09-08T15:26:11.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: audit 2020-09-08T15:26:09.962005+0000 mon.a (mon.0) 57 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix":"config set","value":"1","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2020-09-08T15:26:11.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: audit 2020-09-08T15:26:09.967627+0000 mon.a (mon.0) 58 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd='[{"prefix":"config set","value":"1","who":"mgr","name":"mgr/cephadm/migration_current"}]': finished 2020-09-08T15:26:11.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: audit 2020-09-08T15:26:09.968102+0000 mon.a (mon.0) 59 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix":"config set","value":"2","who":"mgr","name":"mgr/cephadm/migration_current"}]: dispatch 2020-09-08T15:26:11.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: audit 2020-09-08T15:26:09.972425+0000 mon.a (mon.0) 60 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd='[{"prefix":"config set","value":"2","who":"mgr","name":"mgr/cephadm/migration_current"}]': finished 2020-09-08T15:26:11.431 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: audit 2020-09-08T15:26:10.111004+0000 mon.a (mon.0) 61 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:26:11.431 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: audit 2020-09-08T15:26:10.114091+0000 mon.a (mon.0) 62 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:26:11.431 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: audit 2020-09-08T15:26:10.116335+0000 mon.a (mon.0) 63 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:26:11.431 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: audit 2020-09-08T15:26:10.120193+0000 mon.a (mon.0) 64 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:26:11.433 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:10 smithi104 bash[9203]: audit 2020-09-08T15:26:10.131157+0000 mon.a (mon.0) 65 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2020-09-08T15:26:11.669 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=3.6895172595977783, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:26:11.669 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=3.689788818359375, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:26:11.669 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:mgr epoch 5 is available 2020-09-08T15:26:11.670 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Setting orchestrator backend to cephadm... 2020-09-08T15:26:11.670 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'orch', 'set', 'backend', 'cephadm'] 2020-09-08T15:26:11.670 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f orch set backend cephadm 2020-09-08T15:26:12.419 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:11 smithi104 bash[9203]: audit 2020-09-08T15:26:10.981018+0000 mon.a (mon.0) 66 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2020-09-08T15:26:12.419 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:11 smithi104 bash[9203]: audit 2020-09-08T15:26:10.988083+0000 mgr.y (mgr.14119) 1 : audit [DBG] from='client.14123 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2020-09-08T15:26:12.420 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:11 smithi104 bash[9203]: cluster 2020-09-08T15:26:10.990736+0000 mon.a (mon.0) 67 : cluster [DBG] mgrmap e7: y(active, since 1.1229s) 2020-09-08T15:26:12.420 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:11 smithi104 bash[9203]: audit 2020-09-08T15:26:11.005425+0000 mgr.y (mgr.14119) 2 : audit [DBG] from='client.14123 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2020-09-08T15:26:13.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:12 smithi104 bash[9203]: cluster 2020-09-08T15:26:12.001842+0000 mon.a (mon.0) 68 : cluster [DBG] mgrmap e8: y(active, since 2s) 2020-09-08T15:26:13.663 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.9889326095581055, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:26:13.663 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.989248275756836, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:26:13.663 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Generating ssh key... 2020-09-08T15:26:13.663 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'cephadm', 'generate-key'] 2020-09-08T15:26:13.664 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f cephadm generate-key 2020-09-08T15:26:14.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:14 smithi104 bash[9203]: audit 2020-09-08T15:26:13.069226+0000 mgr.y (mgr.14119) 3 : audit [DBG] from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:26:14.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:14 smithi104 bash[9203]: audit 2020-09-08T15:26:13.072711+0000 mon.a (mon.0) 69 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix":"config set","value":"cephadm","who":"mgr","name":"mgr/orchestrator/orchestrator"}]: dispatch 2020-09-08T15:26:14.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:14 smithi104 bash[9203]: audit 2020-09-08T15:26:13.079106+0000 mon.a (mon.0) 70 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd='[{"prefix":"config set","value":"cephadm","who":"mgr","name":"mgr/orchestrator/orchestrator"}]': finished 2020-09-08T15:26:14.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:14 smithi104 bash[9203]: audit 2020-09-08T15:26:13.085180+0000 mon.a (mon.0) 71 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:26:14.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:14 smithi104 bash[9203]: audit 2020-09-08T15:26:13.088816+0000 mon.a (mon.0) 72 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:26:15.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:15 smithi104 bash[9203]: audit 2020-09-08T15:26:15.107906+0000 mgr.y (mgr.14119) 4 : audit [DBG] from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:26:15.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:15 smithi104 bash[9203]: cephadm 2020-09-08T15:26:15.108179+0000 mgr.y (mgr.14119) 5 : cephadm [INF] Generating ssh key... 2020-09-08T15:26:15.677 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:15 smithi104 bash[9475]: Generating public/private rsa key pair. 2020-09-08T15:26:15.678 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:15 smithi104 bash[9475]: Your identification has been saved in /tmp/tmpnd1qidyi/key. 2020-09-08T15:26:15.678 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:15 smithi104 bash[9475]: Your public key has been saved in /tmp/tmpnd1qidyi/key.pub. 2020-09-08T15:26:15.678 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:15 smithi104 bash[9475]: The key fingerprint is: 2020-09-08T15:26:15.678 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:15 smithi104 bash[9475]: SHA256:Pmetxew9EZRMkRpFwhUK5NG8Jfhs2sJupZ6QspdeQvQ ceph-759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:26:15.678 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:15 smithi104 bash[9475]: The key's randomart image is: 2020-09-08T15:26:15.679 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:15 smithi104 bash[9475]: +---[RSA 3072]----+ 2020-09-08T15:26:15.679 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:15 smithi104 bash[9475]: | .+=.*B= | 2020-09-08T15:26:15.679 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:15 smithi104 bash[9475]: | ..o*o* | 2020-09-08T15:26:15.679 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:15 smithi104 bash[9475]: | . .o.B | 2020-09-08T15:26:15.679 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:15 smithi104 bash[9475]: | . . * . | 2020-09-08T15:26:15.680 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:15 smithi104 bash[9475]: | S.E+ . | 2020-09-08T15:26:15.680 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:15 smithi104 bash[9475]: | o .++o . | 2020-09-08T15:26:15.680 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:15 smithi104 bash[9475]: | . *o++= . | 2020-09-08T15:26:15.680 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:15 smithi104 bash[9475]: | ooB+= .. | 2020-09-08T15:26:15.680 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:15 smithi104 bash[9475]: | .o.o+ . .. | 2020-09-08T15:26:15.681 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:15 smithi104 bash[9475]: +----[SHA256]-----+ 2020-09-08T15:26:15.971 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.307642698287964, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:26:15.972 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.307954788208008, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:26:15.972 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'cephadm', 'get-pub-key'] 2020-09-08T15:26:15.972 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f cephadm get-pub-key 2020-09-08T15:26:16.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:16 smithi104 bash[9203]: audit 2020-09-08T15:26:15.322209+0000 mon.a (mon.0) 73 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/ssh_identity_key","val":"-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABlwAAAAdzc2gtcn\nNhAAAAAwEAAQAAAYEAuJ2NE4UL1e4Dm+sX5k07z2Qs7/YVLRqiJbRtdNpLPyH4iSzUsheN\ntZfrN28eoy2Q6bG1EAxOjkMFUXd95Drd4QpwON1KgfemlQQdRtIjmOoYCqUIH/Gd9fLbNQ\nkvzz9K46a6cs+jPEzZyxjK6RIBs1h95UaTfYg2uzCnhueHoiqpvPGzFBzrfXRD7xd5tkfP\nty72ckGaIH2qwiBgQ1ubf7Ke0lV6TlJn1dfGocfKFsZgPBEYVQdwd0aoS6gOmC/XQgY9/w\nXyl3MV9Om+8ir/CG+66Vu8eEMDgAk8llI5yPP0RJbiQAYX/gsMuzZAbARjk8Rcr+rGjRI7\n/G33s/a6DolhWWQK6/HpD37NV5U7I74CYAZFkCxeSf01Lu95pK1mjGYoiebWjhF5j+Xr9f\nASLWrQMs4OJYynOVRNjnsGB1AnqrYNu7Zsb3IS2EB3MlzwhwuhN0RpQVVqjDz8ahRRevit\ndIGLmqBWdxlRNjq4XeM53NP/d/cpp0RIuUVZAyMFAAAFoKljdIGpY3SBAAAAB3NzaC1yc2\nEAAAGBALidjROFC9XuA5vrF+ZNO89kLO/2FS0aoiW0bXTaSz8h+Iks1LIXjbWX6zdvHqMt\nkOmxtRAMTo5DBVF3feQ63eEKcDjdSoH3ppUEHUbSI5jqGAqlCB/xnfXy2zUJL88/SuOmun\nLPozxM2csYyukSAbNYfeVGk32INrswp4bnh6IqqbzxsxQc6310Q+8XebZHz7cu9nJBmiB9\nqsIgYENbm3+yntJVek5SZ9XXxqHHyhbGYDwRGFUHcHdGqEuoDpgv10IGPf8F8pdzFfTpvv\nIq/whvuulbvHhDA4AJPJZSOcjz9ESW4kAGF/4LDLs2QGwEY5PEXK/qxo0SO/xt97P2ug6J\nYVlkCuvx6Q9+zVeVOyO+AmAGRZAsXkn9NS7veaStZoxmKInm1o4ReY/l6/XwEi1q0DLODi\nWMpzlUTY57BgdQJ6q2Dbu2bG9yEthAdzJc8IcLoTdEaUFVaow8/GoUUXr4rXSBi5qgVncZ\nUTY6uF3jOdzT/3f3KadESLlFWQMjBQAAAAMBAAEAAAGAF7RrK+l2dpakMf4CyJh8KIoFVx\n5y46K61TqQn2N0K/wRrJNsEB4EHSAPnMOQ3IT6SXd+9zuNTqj+YtKEAK0qVj3/4N1RNtT0\nt8eadsB+v6C3S46bN3bPwIfZa7rtP6Ax3n9HC3zryc9YW8KoMzsTqtKe/7SLNaAq3uhFao\nYKHWF6//7M7rUz65zlKCsMfsJDwpKuJEeWIbvhFCISGjQGCGSIDhvZo3rjSEr3f0hnA3uf\nWbccN7bH0xjofSZsEVB7DGhhKEjs6S7nN36vS3PffRirG87SXy0dVKPIzVx9Ab8VvWoj7S\nJW9eR3zoT6JVF7QqBIbcHEXAEvLAkaW/1rvNv7KiUK7lnZgBS+0Cel1tkYIokAknowHVYd\ncu6a0OCH9fizYK1zAvNZBv2pVFYgMSMhkWre4SFfq4YwXPjNbADmpr22ny8V5RRGWjgTkr\nn+s45LA/5ceaTEz5edbj3mPHTe/32IRIt7HonVWH9mBb4fIjxxzZPmWkw9wK2++iUVAAAA\nwQCSJARrLQ+s7UnjSwTw0VosKkfswbI36NkMvqMG2xO4s0Wr+9+Nec1JaXhctCjgUBmRUT\nrPyJhmHy4KtEXxX2bANqo79hWcdB/I3l2hzFO+AHJ5CyesP62IK6JYWbPtI+brwUR2rJZp\n/3kXdWdjn06ENS1hS997z5kC+Ssu5Mu49zOCZHIb0m8f9pSJjei33r4Xs5MaNO/sjbBVEH\nHAUIc/mCkqz0eI5a96wMfQtxJRmTA8Kz5yzL9wH7GxPk7U790AAADBAOJfUkezHXsmm5Ge\n9jR4dnRaqBLs2WlPvlxI9D75lQvqGY0LDjOBMzRJqG+pyaO14jeEZRBLPUzKi3kR/kJ4U8\n/ixtkNFhm1RvRbyVRkS82bB+kDJDzzgNldtpoAcj7wW7To8udz/M15LPMbPtYCW7V7NIDa\nq58kbHcG99eZv2dWbkOiPZ/6HCXllbRCSWR9jZ/6TvfPiPm0YcWJa8Sp0Bsum9N52OKDxY\n27DltZkT/sBEcCcdxbEscbUzKOBnrIBwAAAMEA0Mcml7Z1R0KULGoA3XuvRqdlgoBX+D2i\n2LJlkno/m+gCrRWwmV1ASj6Odc+tE5TqEbMkji99Jm9yDONevs1py6opgryX7moG9OOW3K\nHzyxxY0HqQKvNsnqTyThwm9e1yzvNrEwPKMdBto+t94S9REH3mdJOTzvSh7DKBk+uTsHwX\nqRTtcCpJSQbFh5S+OQgT6SLy1hBRTACbH/I7vjrxAvgNip+aEzuTfzKEnfPJG/QUYlPB7s\nrJ/lnQqEljY8GTAAAAKWNlcGgtNzU5YjI3YjYtZjFlNy0xMWVhLWEwODAtMDAxYTRhYWI4\nMzBjAQ==\n-----END OPENSSH PRIVATE KEY-----\n"}]: dispatch 2020-09-08T15:26:16.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:16 smithi104 bash[9203]: audit 2020-09-08T15:26:15.326473+0000 mon.a (mon.0) 74 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/ssh_identity_key","val":"-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABlwAAAAdzc2gtcn\nNhAAAAAwEAAQAAAYEAuJ2NE4UL1e4Dm+sX5k07z2Qs7/YVLRqiJbRtdNpLPyH4iSzUsheN\ntZfrN28eoy2Q6bG1EAxOjkMFUXd95Drd4QpwON1KgfemlQQdRtIjmOoYCqUIH/Gd9fLbNQ\nkvzz9K46a6cs+jPEzZyxjK6RIBs1h95UaTfYg2uzCnhueHoiqpvPGzFBzrfXRD7xd5tkfP\nty72ckGaIH2qwiBgQ1ubf7Ke0lV6TlJn1dfGocfKFsZgPBEYVQdwd0aoS6gOmC/XQgY9/w\nXyl3MV9Om+8ir/CG+66Vu8eEMDgAk8llI5yPP0RJbiQAYX/gsMuzZAbARjk8Rcr+rGjRI7\n/G33s/a6DolhWWQK6/HpD37NV5U7I74CYAZFkCxeSf01Lu95pK1mjGYoiebWjhF5j+Xr9f\nASLWrQMs4OJYynOVRNjnsGB1AnqrYNu7Zsb3IS2EB3MlzwhwuhN0RpQVVqjDz8ahRRevit\ndIGLmqBWdxlRNjq4XeM53NP/d/cpp0RIuUVZAyMFAAAFoKljdIGpY3SBAAAAB3NzaC1yc2\nEAAAGBALidjROFC9XuA5vrF+ZNO89kLO/2FS0aoiW0bXTaSz8h+Iks1LIXjbWX6zdvHqMt\nkOmxtRAMTo5DBVF3feQ63eEKcDjdSoH3ppUEHUbSI5jqGAqlCB/xnfXy2zUJL88/SuOmun\nLPozxM2csYyukSAbNYfeVGk32INrswp4bnh6IqqbzxsxQc6310Q+8XebZHz7cu9nJBmiB9\nqsIgYENbm3+yntJVek5SZ9XXxqHHyhbGYDwRGFUHcHdGqEuoDpgv10IGPf8F8pdzFfTpvv\nIq/whvuulbvHhDA4AJPJZSOcjz9ESW4kAGF/4LDLs2QGwEY5PEXK/qxo0SO/xt97P2ug6J\nYVlkCuvx6Q9+zVeVOyO+AmAGRZAsXkn9NS7veaStZoxmKInm1o4ReY/l6/XwEi1q0DLODi\nWMpzlUTY57BgdQJ6q2Dbu2bG9yEthAdzJc8IcLoTdEaUFVaow8/GoUUXr4rXSBi5qgVncZ\nUTY6uF3jOdzT/3f3KadESLlFWQMjBQAAAAMBAAEAAAGAF7RrK+l2dpakMf4CyJh8KIoFVx\n5y46K61TqQn2N0K/wRrJNsEB4EHSAPnMOQ3IT6SXd+9zuNTqj+YtKEAK0qVj3/4N1RNtT0\nt8eadsB+v6C3S46bN3bPwIfZa7rtP6Ax3n9HC3zryc9YW8KoMzsTqtKe/7SLNaAq3uhFao\nYKHWF6//7M7rUz65zlKCsMfsJDwpKuJEeWIbvhFCISGjQGCGSIDhvZo3rjSEr3f0hnA3uf\nWbccN7bH0xjofSZsEVB7DGhhKEjs6S7nN36vS3PffRirG87SXy0dVKPIzVx9Ab8VvWoj7S\nJW9eR3zoT6JVF7QqBIbcHEXAEvLAkaW/1rvNv7KiUK7lnZgBS+0Cel1tkYIokAknowHVYd\ncu6a0OCH9fizYK1zAvNZBv2pVFYgMSMhkWre4SFfq4YwXPjNbADmpr22ny8V5RRGWjgTkr\nn+s45LA/5ceaTEz5edbj3mPHTe/32IRIt7HonVWH9mBb4fIjxxzZPmWkw9wK2++iUVAAAA\nwQCSJARrLQ+s7UnjSwTw0VosKkfswbI36NkMvqMG2xO4s0Wr+9+Nec1JaXhctCjgUBmRUT\nrPyJhmHy4KtEXxX2bANqo79hWcdB/I3l2hzFO+AHJ5CyesP62IK6JYWbPtI+brwUR2rJZp\n/3kXdWdjn06ENS1hS997z5kC+Ssu5Mu49zOCZHIb0m8f9pSJjei33r4Xs5MaNO/sjbBVEH\nHAUIc/mCkqz0eI5a96wMfQtxJRmTA8Kz5yzL9wH7GxPk7U790AAADBAOJfUkezHXsmm5Ge\n9jR4dnRaqBLs2WlPvlxI9D75lQvqGY0LDjOBMzRJqG+pyaO14jeEZRBLPUzKi3kR/kJ4U8\n/ixtkNFhm1RvRbyVRkS82bB+kDJDzzgNldtpoAcj7wW7To8udz/M15LPMbPtYCW7V7NIDa\nq58kbHcG99eZv2dWbkOiPZ/6HCXllbRCSWR9jZ/6TvfPiPm0YcWJa8Sp0Bsum9N52OKDxY\n27DltZkT/sBEcCcdxbEscbUzKOBnrIBwAAAMEA0Mcml7Z1R0KULGoA3XuvRqdlgoBX+D2i\n2LJlkno/m+gCrRWwmV1ASj6Odc+tE5TqEbMkji99Jm9yDONevs1py6opgryX7moG9OOW3K\nHzyxxY0HqQKvNsnqTyThwm9e1yzvNrEwPKMdBto+t94S9REH3mdJOTzvSh7DKBk+uTsHwX\nqRTtcCpJSQbFh5S+OQgT6SLy1hBRTACbH/I7vjrxAvgNip+aEzuTfzKEnfPJG/QUYlPB7s\nrJ/lnQqEljY8GTAAAAKWNlcGgtNzU5YjI3YjYtZjFlNy0xMWVhLWEwODAtMDAxYTRhYWI4\nMzBjAQ==\n-----END OPENSSH PRIVATE KEY-----\n"}]': finished 2020-09-08T15:26:16.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:16 smithi104 bash[9203]: audit 2020-09-08T15:26:15.327209+0000 mon.a (mon.0) 75 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/ssh_identity_pub","val":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4nY0ThQvV7gOb6xfmTTvPZCzv9hUtGqIltG102ks/IfiJLNSyF421l+s3bx6jLZDpsbUQDE6OQwVRd33kOt3hCnA43UqB96aVBB1G0iOY6hgKpQgf8Z318ts1CS/PP0rjprpyz6M8TNnLGMrpEgGzWH3lRpN9iDa7MKeG54eiKqm88bMUHOt9dEPvF3m2R8+3LvZyQZogfarCIGBDW5t/sp7SVXpOUmfV18ahx8oWxmA8ERhVB3B3RqhLqA6YL9dCBj3/BfKXcxX06b7yKv8Ib7rpW7x4QwOACTyWUjnI8/REluJABhf+Cwy7NkBsBGOTxFyv6saNEjv8bfez9roOiWFZZArr8ekPfs1XlTsjvgJgBkWQLF5J/TUu73mkrWaMZiiJ5taOEXmP5ev18BItatAyzg4ljKc5VE2OewYHUCeqtg27tmxvchLYQHcyXPCHC6E3RGlBVWqMPPxqFFF6+K10gYuaoFZ3GVE2Orhd4znc0/939ymnREi5RVkDIwU= ceph-759b27b6-f1e7-11ea-a080-001a4aab830c\n"}]: dispatch 2020-09-08T15:26:16.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:16 smithi104 bash[9203]: audit 2020-09-08T15:26:15.330709+0000 mon.a (mon.0) 76 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/ssh_identity_pub","val":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4nY0ThQvV7gOb6xfmTTvPZCzv9hUtGqIltG102ks/IfiJLNSyF421l+s3bx6jLZDpsbUQDE6OQwVRd33kOt3hCnA43UqB96aVBB1G0iOY6hgKpQgf8Z318ts1CS/PP0rjprpyz6M8TNnLGMrpEgGzWH3lRpN9iDa7MKeG54eiKqm88bMUHOt9dEPvF3m2R8+3LvZyQZogfarCIGBDW5t/sp7SVXpOUmfV18ahx8oWxmA8ERhVB3B3RqhLqA6YL9dCBj3/BfKXcxX06b7yKv8Ib7rpW7x4QwOACTyWUjnI8/REluJABhf+Cwy7NkBsBGOTxFyv6saNEjv8bfez9roOiWFZZArr8ekPfs1XlTsjvgJgBkWQLF5J/TUu73mkrWaMZiiJ5taOEXmP5ev18BItatAyzg4ljKc5VE2OewYHUCeqtg27tmxvchLYQHcyXPCHC6E3RGlBVWqMPPxqFFF6+K10gYuaoFZ3GVE2Orhd4znc0/939ymnREi5RVkDIwU= ceph-759b27b6-f1e7-11ea-a080-001a4aab830c\n"}]': finished 2020-09-08T15:26:17.525 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4nY0ThQvV7gOb6xfmTTvPZCzv9hUtGqIltG102ks/IfiJLNSyF421l+s3bx6jLZDpsbUQDE6OQwVRd33kOt3hCnA43UqB96aVBB1G0iOY6hgKpQgf8Z318ts1CS/PP0rjprpyz6M8TNnLGMrpEgGzWH3lRpN9iDa7MKeG54eiKqm88bMUHOt9dEPvF3m2R8+3LvZyQZogfarCIGBDW5t/sp7SVXpOUmfV18ahx8oWxmA8ERhVB3B3RqhLqA6YL9dCBj3/BfKXcxX06b7yKv8Ib7rpW7x4QwOACTyWUjnI8/REluJABhf+Cwy7NkBsBGOTxFyv6saNEjv8bfez9roOiWFZZArr8ekPfs1XlTsjvgJgBkWQLF5J/TUu73mkrWaMZiiJ5taOEXmP5ev18BItatAyzg4ljKc5VE2OewYHUCeqtg27tmxvchLYQHcyXPCHC6E3RGlBVWqMPPxqFFF6+K10gYuaoFZ3GVE2Orhd4znc0/939ymnREi5RVkDIwU= ceph-759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:26:17.525 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5513694286346436, stop=False, exit=None, reads=[8] 2020-09-08T15:26:18.115 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.139697551727295, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:26:18.117 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.139986038208008, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:26:18.117 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Wrote public SSH key to to /home/ubuntu/cephtest/ceph.pub 2020-09-08T15:26:18.117 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Adding key to root@localhost's authorized_keys... 2020-09-08T15:26:18.118 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Adding host smithi104... 2020-09-08T15:26:18.118 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'orch', 'host', 'add', 'smithi104'] 2020-09-08T15:26:18.118 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f orch host add smithi104 2020-09-08T15:26:18.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:18 smithi104 bash[9203]: audit 2020-09-08T15:26:17.529574+0000 mgr.y (mgr.14119) 6 : audit [DBG] from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:26:23.177 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:22 smithi104 bash[9475]: Warning: Permanently added 'smithi104,172.21.15.104' (ECDSA) to the list of known hosts. 2020-09-08T15:26:23.506 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout Added host 'smithi104' 2020-09-08T15:26:23.507 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=5.389338731765747, stop=False, exit=None, reads=[8] 2020-09-08T15:26:23.615 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:23 smithi104 bash[9203]: audit 2020-09-08T15:26:22.753506+0000 mgr.y (mgr.14119) 7 : audit [DBG] from='client.14139 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:26:24.298 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=6.177996873855591, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:26:24.299 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=6.178142309188843, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:26:24.299 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'config', 'set', 'mgr', 'mgr/dashboard/ssl_server_port', '8443'] 2020-09-08T15:26:24.299 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f config set mgr mgr/dashboard/ssl_server_port 8443 2020-09-08T15:26:24.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:24 smithi104 bash[9203]: audit 2020-09-08T15:26:23.506516+0000 mon.a (mon.0) 77 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/inventory","val":"{\"smithi104\": {\"hostname\": \"smithi104\", \"addr\": \"smithi104\", \"labels\": [], \"status\": \"\"}}"}]: dispatch 2020-09-08T15:26:24.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:24 smithi104 bash[9203]: audit 2020-09-08T15:26:23.509869+0000 mon.a (mon.0) 78 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/inventory","val":"{\"smithi104\": {\"hostname\": \"smithi104\", \"addr\": \"smithi104\", \"labels\": [], \"status\": \"\"}}"}]': finished 2020-09-08T15:26:24.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:24 smithi104 bash[9203]: cephadm 2020-09-08T15:26:23.510459+0000 mgr.y (mgr.14119) 8 : cephadm [INF] Added host smithi104 2020-09-08T15:26:24.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:24 smithi104 bash[9203]: audit 2020-09-08T15:26:23.836763+0000 mon.a (mon.0) 79 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {}, \"devices\": [], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"networks\": {}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:26:24.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:24 smithi104 bash[9203]: audit 2020-09-08T15:26:23.841166+0000 mon.a (mon.0) 80 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {}, \"devices\": [], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"networks\": {}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:26:24.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:24 smithi104 bash[9203]: audit 2020-09-08T15:26:23.841856+0000 mon.a (mon.0) 81 : audit [DBG] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:26:25.529 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:25 smithi104 bash[9203]: audit 2020-09-08T15:26:24.428153+0000 mon.a (mon.0) 82 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:26:24.426031\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:26:24.427485\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2020-09-08T15:26:24.427579\", \"networks\": {}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:26:25.530 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:25 smithi104 bash[9203]: audit 2020-09-08T15:26:24.432315+0000 mon.a (mon.0) 83 : audit [INF] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:26:24.426031\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:26:24.427485\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2020-09-08T15:26:24.427579\", \"networks\": {}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:26:25.530 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:25 smithi104 bash[9203]: audit 2020-09-08T15:26:24.433068+0000 mon.a (mon.0) 84 : audit [DBG] from='mgr.14119 v1:172.21.15.104:0/4023281740' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:26:26.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:26 smithi104 bash[9203]: audit 2020-09-08T15:26:26.141843+0000 mon.a (mon.0) 85 : audit [INF] from='client.? v1:172.21.15.104:0/1754589862' entity='client.admin' cmd=[{"prefix": "config set", "who": "mgr", "name": "mgr/dashboard/ssl_server_port", "value": "8443"}]: dispatch 2020-09-08T15:26:26.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:26 smithi104 bash[9203]: audit 2020-09-08T15:26:26.148780+0000 mon.a (mon.0) 86 : audit [INF] from='client.? v1:172.21.15.104:0/1754589862' entity='client.admin' cmd='[{"prefix": "config set", "who": "mgr", "name": "mgr/dashboard/ssl_server_port", "value": "8443"}]': finished 2020-09-08T15:26:26.993 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.696279287338257, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:26:26.994 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.6968584060668945, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:26:26.994 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Enabling the dashboard module... 2020-09-08T15:26:26.995 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'mgr', 'module', 'enable', 'dashboard'] 2020-09-08T15:26:26.995 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f mgr module enable dashboard 2020-09-08T15:26:28.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:28 smithi104 bash[9203]: audit 2020-09-08T15:26:28.497953+0000 mon.a (mon.0) 87 : audit [INF] from='client.? v1:172.21.15.104:0/3541772006' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2020-09-08T15:26:28.927 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:28 smithi104 bash[9475]: ignoring --setuser ceph since I am not root 2020-09-08T15:26:28.927 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:28 smithi104 bash[9475]: ignoring --setgroup ceph since I am not root 2020-09-08T15:26:28.927 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:28 smithi104 bash[9475]: debug 2020-09-08T15:26:28.587+0000 7f6f14d2a700 1 -- 172.21.15.104:0/2604681314 <== mon.0 v1:172.21.15.104:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 1515889645 0 0) 0x5601eb7284e0 con 0x5601eb784800 2020-09-08T15:26:29.176 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.1800973415374756, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:26:29.177 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.1804213523864746, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:26:29.177 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'mgr', 'dump'] 2020-09-08T15:26:29.177 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f mgr dump 2020-09-08T15:26:29.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:29 smithi104 bash[9203]: audit 2020-09-08T15:26:28.552968+0000 mon.a (mon.0) 88 : audit [INF] from='client.? v1:172.21.15.104:0/3541772006' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2020-09-08T15:26:29.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:29 smithi104 bash[9203]: cluster 2020-09-08T15:26:28.553052+0000 mon.a (mon.0) 89 : cluster [DBG] mgrmap e9: y(active, since 18s) 2020-09-08T15:26:30.670 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:30.670 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "epoch": 9, 2020-09-08T15:26:30.675 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_gid": 14119, 2020-09-08T15:26:30.675 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_name": "y", 2020-09-08T15:26:30.676 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_addrs": { 2020-09-08T15:26:30.676 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addrvec": [ 2020-09-08T15:26:30.676 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:30.676 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "v2", 2020-09-08T15:26:30.676 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addr": "172.21.15.104:6800", 2020-09-08T15:26:30.677 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nonce": 3129754347 2020-09-08T15:26:30.677 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:30.677 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:30.677 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "v1", 2020-09-08T15:26:30.677 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addr": "172.21.15.104:6801", 2020-09-08T15:26:30.678 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nonce": 3129754347 2020-09-08T15:26:30.678 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:30.678 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ] 2020-09-08T15:26:30.678 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:30.678 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_addr": "172.21.15.104:6801/3129754347", 2020-09-08T15:26:30.679 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_change": "2020-09-08T15:26:09.867814+0000", 2020-09-08T15:26:30.679 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_mgr_features": 4540138292837744639, 2020-09-08T15:26:30.679 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "available": true, 2020-09-08T15:26:30.679 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "standbys": [], 2020-09-08T15:26:30.679 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "modules": [ 2020-09-08T15:26:30.680 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "cephadm", 2020-09-08T15:26:30.680 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "dashboard", 2020-09-08T15:26:30.680 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "iostat", 2020-09-08T15:26:30.680 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "restful" 2020-09-08T15:26:30.680 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:30.680 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "available_modules": [ 2020-09-08T15:26:30.681 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:30.681 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "alerts", 2020-09-08T15:26:30.681 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:30.681 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:30.681 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:30.682 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "interval": { 2020-09-08T15:26:30.682 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "interval", 2020-09-08T15:26:30.682 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:30.682 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.061 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.4923830032348633, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.062 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.062 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "60", 2020-09-08T15:26:31.062 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "How frequently to reexamine health status", 2020-09-08T15:26:31.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.064 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.064 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.064 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.064 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.064 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.065 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.065 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.065 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.065 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.065 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.067 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.067 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.067 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.067 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.067 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.4948816299438477, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.069 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.069 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.069 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.069 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.069 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.070 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.070 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.070 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.077 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.077 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.077 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.077 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.078 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.078 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.078 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.078 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.078 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.079 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.079 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.079 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.079 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.079 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.080 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.080 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.080 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.080 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.498215675354004, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.080 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.081 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.081 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.081 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.081 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.081 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.082 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.082 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.082 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.082 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.082 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.082 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.083 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.083 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.083 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.083 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.083 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.084 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.084 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.084 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.084 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.084 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.085 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_destination": { 2020-09-08T15:26:31.085 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_destination", 2020-09-08T15:26:31.085 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.085 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.085 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.086 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.086 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.086 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.086 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5016353130340576, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.086 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.086 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Email address to send alerts to", 2020-09-08T15:26:31.087 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.087 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.087 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.087 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.087 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_from_name": { 2020-09-08T15:26:31.088 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_from_name", 2020-09-08T15:26:31.088 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.088 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.088 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.088 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "Ceph", 2020-09-08T15:26:31.089 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.089 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.089 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.089 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Email From: name", 2020-09-08T15:26:31.089 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.089 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.090 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.090 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.090 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_host": { 2020-09-08T15:26:31.090 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_host", 2020-09-08T15:26:31.090 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.091 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.091 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.091 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.091 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.092 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.092 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5043435096740723, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.092 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.092 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "SMTP server", 2020-09-08T15:26:31.092 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.092 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.093 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.093 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.093 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_password": { 2020-09-08T15:26:31.093 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_password", 2020-09-08T15:26:31.093 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.094 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.094 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.094 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.094 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.094 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.095 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.095 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Password to authenticate with", 2020-09-08T15:26:31.095 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.095 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.095 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.096 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.096 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_port": { 2020-09-08T15:26:31.096 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_port", 2020-09-08T15:26:31.096 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.096 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.097 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.097 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "465", 2020-09-08T15:26:31.097 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.097 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.097 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5056777000427246, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.097 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.098 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "SMTP port", 2020-09-08T15:26:31.098 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.098 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.098 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.098 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.099 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_sender": { 2020-09-08T15:26:31.099 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_sender", 2020-09-08T15:26:31.099 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.099 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.099 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.100 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.100 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.100 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.100 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.100 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "SMTP envelope sender", 2020-09-08T15:26:31.100 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.101 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.101 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.101 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.101 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_ssl": { 2020-09-08T15:26:31.101 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_ssl", 2020-09-08T15:26:31.102 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.102 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.102 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.102 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.102 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.103 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.103 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.103 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5069525241851807, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.103 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Use SSL to connect to SMTP server", 2020-09-08T15:26:31.103 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.104 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.104 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.104 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.104 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "smtp_user": { 2020-09-08T15:26:31.104 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "smtp_user", 2020-09-08T15:26:31.104 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.105 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.105 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.105 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.105 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.105 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.106 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.106 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "User to authenticate as", 2020-09-08T15:26:31.106 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.106 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.106 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.107 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.107 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.107 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.107 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.107 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "balancer", 2020-09-08T15:26:31.108 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.108 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.108 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.108 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active": { 2020-09-08T15:26:31.108 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "active", 2020-09-08T15:26:31.109 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.109 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.109 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.109 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.508392572402954, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.109 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.110 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.110 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.110 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.110 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "automatically balance PGs across cluster", 2020-09-08T15:26:31.110 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.110 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.111 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.111 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.111 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "begin_time": { 2020-09-08T15:26:31.111 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "begin_time", 2020-09-08T15:26:31.111 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.112 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.112 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.113 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "0000", 2020-09-08T15:26:31.113 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.113 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.113 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.113 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "beginning time of day to automatically balance", 2020-09-08T15:26:31.114 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "This is a time of day in the format HHMM.", 2020-09-08T15:26:31.114 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.114 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.114 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.114 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "begin_weekday": { 2020-09-08T15:26:31.115 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "begin_weekday", 2020-09-08T15:26:31.115 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "uint", 2020-09-08T15:26:31.115 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5095913410186768, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.115 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.115 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.116 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "0", 2020-09-08T15:26:31.116 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "0", 2020-09-08T15:26:31.116 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "7", 2020-09-08T15:26:31.116 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.116 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Restrict automatic balancing to this day of the week or later", 2020-09-08T15:26:31.117 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2020-09-08T15:26:31.117 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.117 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.117 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.117 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crush_compat_max_iterations": { 2020-09-08T15:26:31.118 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "crush_compat_max_iterations", 2020-09-08T15:26:31.118 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "uint", 2020-09-08T15:26:31.118 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.118 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.118 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "25", 2020-09-08T15:26:31.119 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "1", 2020-09-08T15:26:31.119 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "250", 2020-09-08T15:26:31.119 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.119 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "maximum number of iterations to attempt optimization", 2020-09-08T15:26:31.119 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.119 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.120 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.120 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.120 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5106632709503174, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.120 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crush_compat_metrics": { 2020-09-08T15:26:31.120 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "crush_compat_metrics", 2020-09-08T15:26:31.121 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.121 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.121 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.121 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "pgs,objects,bytes", 2020-09-08T15:26:31.121 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.122 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.122 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.122 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "metrics with which to calculate OSD utilization", 2020-09-08T15:26:31.122 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2020-09-08T15:26:31.122 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.122 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.123 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.123 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crush_compat_step": { 2020-09-08T15:26:31.123 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "crush_compat_step", 2020-09-08T15:26:31.123 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "float", 2020-09-08T15:26:31.123 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.124 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.124 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "0.5", 2020-09-08T15:26:31.124 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "0.001", 2020-09-08T15:26:31.124 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "0.999", 2020-09-08T15:26:31.124 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.511592149734497, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.125 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.125 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "aggressiveness of optimization", 2020-09-08T15:26:31.125 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": ".99 is very aggressive, .01 is less aggressive", 2020-09-08T15:26:31.125 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.125 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.126 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.126 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "end_time": { 2020-09-08T15:26:31.126 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "end_time", 2020-09-08T15:26:31.126 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.126 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.126 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.127 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "2400", 2020-09-08T15:26:31.127 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.127 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.127 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.127 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "ending time of day to automatically balance", 2020-09-08T15:26:31.128 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "This is a time of day in the format HHMM.", 2020-09-08T15:26:31.128 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.128 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.128 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.128 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "end_weekday": { 2020-09-08T15:26:31.128 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "end_weekday", 2020-09-08T15:26:31.129 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "uint", 2020-09-08T15:26:31.129 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.129 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.129 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5126020908355713, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.129 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "7", 2020-09-08T15:26:31.130 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "0", 2020-09-08T15:26:31.130 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "7", 2020-09-08T15:26:31.130 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.130 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Restrict automatic balancing to days of the week earlier than this", 2020-09-08T15:26:31.130 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2020-09-08T15:26:31.131 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.131 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.131 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.131 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.131 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.132 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.132 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.132 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.132 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.132 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.132 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.133 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.133 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.133 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.133 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.133 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.134 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.134 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.134 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.134 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.134 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.135 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.135 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5130939483642578, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.135 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.135 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.135 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.136 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.136 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.136 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.136 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.136 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.137 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.137 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.137 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.137 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.137 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.138 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.138 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.138 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.138 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.138 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.138 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.139 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.139 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.139 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.139 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.139 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.140 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.140 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.140 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.140 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.140 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.141 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5135877132415771, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.141 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.141 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.141 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.141 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.142 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.142 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.142 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.142 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.142 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.142 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.143 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.143 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.143 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.143 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.143 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.144 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.144 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.144 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.144 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.144 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.144 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.145 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.145 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min_score": { 2020-09-08T15:26:31.145 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "min_score", 2020-09-08T15:26:31.145 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "float", 2020-09-08T15:26:31.145 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.146 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.146 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "0", 2020-09-08T15:26:31.146 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.146 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.146 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5140950679779053, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.147 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.147 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "minimum score, below which no optimization is attempted", 2020-09-08T15:26:31.147 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.147 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.147 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.148 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.148 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mode": { 2020-09-08T15:26:31.148 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "mode", 2020-09-08T15:26:31.148 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.148 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.148 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.149 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "upmap", 2020-09-08T15:26:31.149 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.149 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.149 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.149 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crush-compat", 2020-09-08T15:26:31.150 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "none", 2020-09-08T15:26:31.150 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "upmap" 2020-09-08T15:26:31.150 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.150 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Balancer mode", 2020-09-08T15:26:31.150 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.151 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.151 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.151 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.151 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pool_ids": { 2020-09-08T15:26:31.151 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "pool_ids", 2020-09-08T15:26:31.151 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.152 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.152 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.152 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5145998001098633, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.152 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.152 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.153 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.153 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.153 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "pools which the automatic balancing will be limited to", 2020-09-08T15:26:31.153 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.153 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.154 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.154 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.154 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "sleep_interval": { 2020-09-08T15:26:31.154 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "sleep_interval", 2020-09-08T15:26:31.154 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:31.154 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.155 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.155 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "60", 2020-09-08T15:26:31.155 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.155 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.155 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.156 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how frequently to wake up and attempt optimization", 2020-09-08T15:26:31.156 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.156 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.156 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.156 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.157 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "upmap_max_deviation": { 2020-09-08T15:26:31.157 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "upmap_max_deviation", 2020-09-08T15:26:31.157 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.157 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5150423049926758, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.157 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.158 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.158 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "5", 2020-09-08T15:26:31.158 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "1", 2020-09-08T15:26:31.158 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.158 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.159 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "deviation below which no optimization is attempted", 2020-09-08T15:26:31.159 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2020-09-08T15:26:31.159 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.159 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.159 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.159 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "upmap_max_optimizations": { 2020-09-08T15:26:31.160 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "upmap_max_optimizations", 2020-09-08T15:26:31.160 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "uint", 2020-09-08T15:26:31.160 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.160 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.160 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "10", 2020-09-08T15:26:31.161 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.161 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.161 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.161 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "maximum upmap optimizations to make per attempt", 2020-09-08T15:26:31.161 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.162 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.162 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5154521465301514, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.162 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.162 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.162 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.163 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.163 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.163 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "cephadm", 2020-09-08T15:26:31.163 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.163 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.164 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.164 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "allow_ptrace": { 2020-09-08T15:26:31.164 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "allow_ptrace", 2020-09-08T15:26:31.164 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.164 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.164 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.165 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.165 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.165 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.165 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.165 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "allow SYS_PTRACE capability on ceph containers", 2020-09-08T15:26:31.166 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2020-09-08T15:26:31.166 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.166 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.166 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.166 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "config_dashboard": { 2020-09-08T15:26:31.167 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "config_dashboard", 2020-09-08T15:26:31.167 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.167 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5159187316894531, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.167 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.167 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.168 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.168 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.168 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.168 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.168 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "manage configs like API endpoints in Dashboard.", 2020-09-08T15:26:31.168 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.169 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.169 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.169 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.169 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "container_image_alertmanager": { 2020-09-08T15:26:31.169 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "container_image_alertmanager", 2020-09-08T15:26:31.170 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.170 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.170 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.170 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "docker.io/prom/alertmanager:v0.20.0", 2020-09-08T15:26:31.170 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.171 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.171 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.171 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Prometheus container image", 2020-09-08T15:26:31.171 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.171 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.171 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.172 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.172 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "container_image_base": { 2020-09-08T15:26:31.172 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.516364336013794, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.172 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "container_image_base", 2020-09-08T15:26:31.172 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.173 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.173 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.173 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "docker.io/ceph/ceph", 2020-09-08T15:26:31.173 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.173 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.174 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.174 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Container image name, without the tag", 2020-09-08T15:26:31.174 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.174 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.174 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.174 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.175 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "container_image_grafana": { 2020-09-08T15:26:31.175 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "container_image_grafana", 2020-09-08T15:26:31.175 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.175 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.175 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.176 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "docker.io/ceph/ceph-grafana:6.6.2", 2020-09-08T15:26:31.176 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.176 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.176 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.176 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Prometheus container image", 2020-09-08T15:26:31.177 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.177 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.177 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.516794204711914, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.177 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.178 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.178 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "container_image_node_exporter": { 2020-09-08T15:26:31.178 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "container_image_node_exporter", 2020-09-08T15:26:31.178 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.178 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.179 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.179 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "docker.io/prom/node-exporter:v0.18.1", 2020-09-08T15:26:31.179 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.179 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.179 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.180 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Prometheus container image", 2020-09-08T15:26:31.180 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.180 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.180 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.180 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.181 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "container_image_prometheus": { 2020-09-08T15:26:31.181 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "container_image_prometheus", 2020-09-08T15:26:31.181 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.181 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.181 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.182 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "docker.io/prom/prometheus:v2.18.1", 2020-09-08T15:26:31.182 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.182 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.182 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.182 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5172195434570312, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.183 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Prometheus container image", 2020-09-08T15:26:31.183 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.183 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.183 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.183 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.183 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "daemon_cache_timeout": { 2020-09-08T15:26:31.184 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "daemon_cache_timeout", 2020-09-08T15:26:31.184 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:31.184 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.184 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.184 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "600", 2020-09-08T15:26:31.185 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.185 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.185 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.185 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "seconds to cache service (daemon) inventory", 2020-09-08T15:26:31.185 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.186 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.186 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.186 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.186 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "device_cache_timeout": { 2020-09-08T15:26:31.186 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "device_cache_timeout", 2020-09-08T15:26:31.187 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:31.187 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.187 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.187 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "1800", 2020-09-08T15:26:31.187 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.187 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5176589488983154, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.188 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.188 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.188 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "seconds to cache device inventory", 2020-09-08T15:26:31.188 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.189 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.189 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.189 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.189 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "host_check_interval": { 2020-09-08T15:26:31.189 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "host_check_interval", 2020-09-08T15:26:31.190 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:31.190 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.190 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.190 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "600", 2020-09-08T15:26:31.190 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.191 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.191 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.191 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how frequently to perform a host check", 2020-09-08T15:26:31.191 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.191 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.192 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.192 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.192 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.193 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.193 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.193 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.193 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.193 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.194 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5181124210357666, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.194 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.194 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.194 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.194 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.194 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.195 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.195 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.195 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.195 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.195 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.196 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.196 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.196 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.196 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.196 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.197 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.197 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.197 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.197 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.197 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.197 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.198 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.198 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.198 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.198 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "log to the \"cephadm\" cluster log channel\"", 2020-09-08T15:26:31.199 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.199 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.199 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.199 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.199 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5186007022857666, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.199 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.200 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.200 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.200 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.200 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.200 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.201 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.201 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.201 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.201 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.201 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.202 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.202 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.202 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.202 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.202 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.203 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.203 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.203 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.203 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.203 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.204 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.204 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.204 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.204 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.204 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.204 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.205 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.205 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.205 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.205 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5191047191619873, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.205 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.206 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.206 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.206 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.206 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.206 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "manage_etc_ceph_ceph_conf": { 2020-09-08T15:26:31.207 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "manage_etc_ceph_ceph_conf", 2020-09-08T15:26:31.207 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.207 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.207 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.207 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.207 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.208 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.208 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.208 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2020-09-08T15:26:31.208 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.208 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.209 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.209 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.209 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "migration_current": { 2020-09-08T15:26:31.209 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "migration_current", 2020-09-08T15:26:31.209 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.210 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.210 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.210 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:31.210 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.210 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.211 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.519557237625122, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.211 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.211 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "internal - do not modify", 2020-09-08T15:26:31.211 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.211 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.212 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.212 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.212 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mode": { 2020-09-08T15:26:31.212 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "mode", 2020-09-08T15:26:31.212 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.213 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.213 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.213 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "root", 2020-09-08T15:26:31.213 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.214 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.214 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.214 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "cephadm-package", 2020-09-08T15:26:31.214 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "root" 2020-09-08T15:26:31.214 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.215 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "mode for remote execution of cephadm", 2020-09-08T15:26:31.215 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.215 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.215 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.215 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.215 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "prometheus_alerts_path": { 2020-09-08T15:26:31.216 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "prometheus_alerts_path", 2020-09-08T15:26:31.216 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.216 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.216 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.216 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.520043134689331, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.217 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2020-09-08T15:26:31.217 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.217 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.217 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.217 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "location of alerts to include in prometheus deployments", 2020-09-08T15:26:31.218 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.218 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.218 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.218 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.218 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "registry_password": { 2020-09-08T15:26:31.218 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "registry_password", 2020-09-08T15:26:31.219 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.219 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.219 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.219 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:31.219 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.220 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.220 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.220 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Custom repository password", 2020-09-08T15:26:31.220 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.220 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.221 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.221 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.221 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "registry_url": { 2020-09-08T15:26:31.221 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "registry_url", 2020-09-08T15:26:31.221 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.520470380783081, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.222 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.222 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.222 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.222 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:31.222 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.223 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.223 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.223 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Custom repository url", 2020-09-08T15:26:31.223 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.223 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.224 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.224 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.224 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "registry_username": { 2020-09-08T15:26:31.224 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "registry_username", 2020-09-08T15:26:31.224 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.224 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.225 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.225 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:31.225 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.225 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.225 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.226 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Custom repository username", 2020-09-08T15:26:31.226 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.226 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.226 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.226 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.227 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ssh_config_file": { 2020-09-08T15:26:31.227 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ssh_config_file", 2020-09-08T15:26:31.227 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5209410190582275, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.227 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.227 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.227 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.228 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:31.228 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.228 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.228 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.228 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "customized SSH config file to connect to managed hosts", 2020-09-08T15:26:31.229 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.229 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.229 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.229 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.229 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "use_repo_digest": { 2020-09-08T15:26:31.230 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "use_repo_digest", 2020-09-08T15:26:31.230 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.230 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.230 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.230 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.230 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.231 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.231 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.231 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2020-09-08T15:26:31.231 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.231 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.232 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.232 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5213623046875, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.232 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.232 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warn_on_failed_host_check": { 2020-09-08T15:26:31.232 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "warn_on_failed_host_check", 2020-09-08T15:26:31.233 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.233 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.233 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.233 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.233 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.234 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.234 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.234 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "raise a health warning if the host check fails", 2020-09-08T15:26:31.234 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.234 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.234 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.235 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.235 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warn_on_stray_daemons": { 2020-09-08T15:26:31.235 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "warn_on_stray_daemons", 2020-09-08T15:26:31.235 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.235 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.236 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.236 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.236 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.236 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.236 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.237 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.521772861480713, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.237 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2020-09-08T15:26:31.237 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.237 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.237 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.238 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.238 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warn_on_stray_hosts": { 2020-09-08T15:26:31.238 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "warn_on_stray_hosts", 2020-09-08T15:26:31.238 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.238 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.239 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.239 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.239 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.239 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.239 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.239 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2020-09-08T15:26:31.240 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.240 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.240 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.240 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.240 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.241 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.241 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.241 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "crash", 2020-09-08T15:26:31.241 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.241 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.242 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.242 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.242 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.242 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.242 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5222599506378174, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.243 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.243 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.243 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.243 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.243 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.244 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.244 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.244 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.244 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.244 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.245 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.245 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.245 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.245 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.245 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.246 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.246 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.246 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.246 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.246 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.247 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.247 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.247 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.247 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.247 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.247 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.248 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.248 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.248 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.248 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.248 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5227727890014648, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.249 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.249 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.249 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.249 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.249 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.250 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.250 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.250 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.250 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.250 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.251 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.251 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.251 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.251 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.251 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.251 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.252 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.252 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.252 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.252 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.252 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.253 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.253 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.253 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.253 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.253 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.254 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.254 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.254 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.254 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.254 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5232737064361572, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.254 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.255 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.255 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.255 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.255 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.255 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.256 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.256 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "retain_interval": { 2020-09-08T15:26:31.256 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "retain_interval", 2020-09-08T15:26:31.256 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:31.256 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.257 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.257 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "31536000", 2020-09-08T15:26:31.257 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.257 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.257 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.258 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how long to retain crashes before pruning them", 2020-09-08T15:26:31.258 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.258 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.258 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.258 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.259 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warn_recent_interval": { 2020-09-08T15:26:31.259 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "warn_recent_interval", 2020-09-08T15:26:31.259 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:31.259 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.259 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.260 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "1209600", 2020-09-08T15:26:31.260 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5237300395965576, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.260 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.260 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.260 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.260 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "time interval in which to warn about recent crashes", 2020-09-08T15:26:31.261 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.261 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.261 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.261 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.261 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.262 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.262 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.262 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "dashboard", 2020-09-08T15:26:31.262 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.262 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.263 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.263 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ALERTMANAGER_API_HOST": { 2020-09-08T15:26:31.263 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ALERTMANAGER_API_HOST", 2020-09-08T15:26:31.263 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.263 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.264 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.264 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.264 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.264 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.264 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.265 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.265 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.265 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.265 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.265 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.266 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "AUDIT_API_ENABLED": { 2020-09-08T15:26:31.266 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5242366790771484, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.266 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "AUDIT_API_ENABLED", 2020-09-08T15:26:31.266 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.266 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.266 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.267 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.267 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.267 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.267 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.267 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.268 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.268 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.268 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.269 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.269 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "AUDIT_API_LOG_PAYLOAD": { 2020-09-08T15:26:31.269 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "AUDIT_API_LOG_PAYLOAD", 2020-09-08T15:26:31.269 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.270 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.270 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.270 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.270 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.270 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.271 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.271 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.271 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.271 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.271 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.272 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.272 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ENABLE_BROWSABLE_API": { 2020-09-08T15:26:31.272 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5247023105621338, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.272 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ENABLE_BROWSABLE_API", 2020-09-08T15:26:31.272 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.273 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.273 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.273 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.273 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.273 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.274 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.280 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.281 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.281 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.281 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.281 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.281 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_cephfs": { 2020-09-08T15:26:31.282 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_cephfs", 2020-09-08T15:26:31.282 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.282 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.282 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.282 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.283 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.283 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.283 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.283 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.283 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.283 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.284 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.284 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.284 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_iscsi": { 2020-09-08T15:26:31.284 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_iscsi", 2020-09-08T15:26:31.284 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5251834392547607, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.285 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.285 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.285 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.285 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.285 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.286 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.286 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.286 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.286 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.286 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.287 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.287 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.287 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_mirroring": { 2020-09-08T15:26:31.287 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_mirroring", 2020-09-08T15:26:31.287 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.287 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.288 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.288 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.288 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.288 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.288 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.289 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.289 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.289 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.289 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.289 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.290 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_nfs": { 2020-09-08T15:26:31.290 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_nfs", 2020-09-08T15:26:31.290 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5256552696228027, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.290 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.290 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.290 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.291 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.291 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.291 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.291 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.291 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.292 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.292 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.292 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.292 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.292 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_rbd": { 2020-09-08T15:26:31.293 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_rbd", 2020-09-08T15:26:31.293 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.293 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.293 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.293 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.294 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.294 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.294 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.294 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.294 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.295 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.295 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.295 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.295 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "FEATURE_TOGGLE_rgw": { 2020-09-08T15:26:31.295 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "FEATURE_TOGGLE_rgw", 2020-09-08T15:26:31.295 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.296 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5261473655700684, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.296 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.296 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.296 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.296 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.297 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.297 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.297 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.297 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.297 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.298 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.298 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.298 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2020-09-08T15:26:31.298 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2020-09-08T15:26:31.298 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.299 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.299 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.299 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.299 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.299 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.299 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.300 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.300 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.300 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.300 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.300 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.301 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GRAFANA_API_PASSWORD": { 2020-09-08T15:26:31.301 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GRAFANA_API_PASSWORD", 2020-09-08T15:26:31.301 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5266165733337402, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.301 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.301 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.302 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.302 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "admin", 2020-09-08T15:26:31.302 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.302 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.302 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.303 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.303 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.303 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.303 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.303 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.303 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GRAFANA_API_SSL_VERIFY": { 2020-09-08T15:26:31.304 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GRAFANA_API_SSL_VERIFY", 2020-09-08T15:26:31.304 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.304 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.304 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.304 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.305 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.305 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.305 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.305 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.305 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.306 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.306 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.306 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.306 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GRAFANA_API_URL": { 2020-09-08T15:26:31.306 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GRAFANA_API_URL", 2020-09-08T15:26:31.307 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.307 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.527113437652588, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.307 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.307 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.307 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.307 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.308 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.308 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.308 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.308 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.308 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.309 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.309 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.309 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GRAFANA_API_USERNAME": { 2020-09-08T15:26:31.309 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GRAFANA_API_USERNAME", 2020-09-08T15:26:31.309 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.310 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.310 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.310 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "admin", 2020-09-08T15:26:31.310 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.310 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.310 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.311 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.311 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.311 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.311 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.311 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.312 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "GRAFANA_UPDATE_DASHBOARDS": { 2020-09-08T15:26:31.312 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "GRAFANA_UPDATE_DASHBOARDS", 2020-09-08T15:26:31.312 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.312 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.527606725692749, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.312 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.313 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.313 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.313 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.313 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.314 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.314 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.314 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.314 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.314 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.315 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.315 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ISCSI_API_SSL_VERIFICATION": { 2020-09-08T15:26:31.315 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ISCSI_API_SSL_VERIFICATION", 2020-09-08T15:26:31.315 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.315 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.316 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.316 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.316 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.316 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.316 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.317 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.317 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.317 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.317 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.317 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.318 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PROMETHEUS_API_HOST": { 2020-09-08T15:26:31.318 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PROMETHEUS_API_HOST", 2020-09-08T15:26:31.318 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.318 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5281028747558594, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.318 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.318 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.319 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.319 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.319 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.319 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.319 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.320 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.320 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.320 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.320 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.320 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2020-09-08T15:26:31.321 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2020-09-08T15:26:31.321 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.321 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.321 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.321 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.321 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.322 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.322 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.322 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.322 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.322 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.323 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.323 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.323 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2020-09-08T15:26:31.323 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2020-09-08T15:26:31.323 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5285732746124268, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.324 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.324 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.324 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.324 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.324 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.324 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.325 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.325 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.325 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.325 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.325 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.326 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.326 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2020-09-08T15:26:31.326 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2020-09-08T15:26:31.326 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.326 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.327 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.327 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.327 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.327 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.327 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.328 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.328 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.328 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.328 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.328 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.328 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2020-09-08T15:26:31.329 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.529031753540039, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.329 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2020-09-08T15:26:31.329 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.329 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.330 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.330 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.330 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.330 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.330 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.330 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.331 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.331 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.331 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.331 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.331 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2020-09-08T15:26:31.332 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2020-09-08T15:26:31.332 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.332 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.332 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.332 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.333 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.333 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.333 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.333 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.333 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.334 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.334 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.334 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.334 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5294840335845947, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.334 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2020-09-08T15:26:31.334 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2020-09-08T15:26:31.335 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.335 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.335 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.335 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.335 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.336 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.336 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.336 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.336 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.337 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.337 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.337 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.337 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2020-09-08T15:26:31.337 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2020-09-08T15:26:31.338 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.338 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.338 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.338 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.338 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.338 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.339 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.339 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.339 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.339 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.339 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.340 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.529937505722046, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.340 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.340 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_ENABLED": { 2020-09-08T15:26:31.340 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_ENABLED", 2020-09-08T15:26:31.340 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.341 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.341 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.341 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.341 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.341 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.341 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.342 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.342 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.342 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.342 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.342 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.343 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_EXCLUSION_LIST": { 2020-09-08T15:26:31.343 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_EXCLUSION_LIST", 2020-09-08T15:26:31.343 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.343 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.343 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.344 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2020-09-08T15:26:31.344 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.344 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.344 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.344 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.344 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.345 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5303809642791748, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.345 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.345 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.345 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.345 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_MIN_COMPLEXITY": { 2020-09-08T15:26:31.346 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_MIN_COMPLEXITY", 2020-09-08T15:26:31.346 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.346 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.346 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.346 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "10", 2020-09-08T15:26:31.347 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.347 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.347 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.347 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.347 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.347 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.348 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.348 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.348 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "PWD_POLICY_MIN_LENGTH": { 2020-09-08T15:26:31.348 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "PWD_POLICY_MIN_LENGTH", 2020-09-08T15:26:31.348 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.349 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.349 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.349 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "8", 2020-09-08T15:26:31.349 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.349 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.350 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.350 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.350 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.350 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5308656692504883, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.350 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.350 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.351 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.351 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "REST_REQUESTS_TIMEOUT": { 2020-09-08T15:26:31.351 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "REST_REQUESTS_TIMEOUT", 2020-09-08T15:26:31.351 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.351 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "45", 2020-09-08T15:26:31.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.352 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.353 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.353 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.353 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.353 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.353 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.353 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_ACCESS_KEY": { 2020-09-08T15:26:31.354 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_ACCESS_KEY", 2020-09-08T15:26:31.354 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.354 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.354 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.354 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.355 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.355 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.355 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.355 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.355 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5313646793365479, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_ADMIN_RESOURCE": { 2020-09-08T15:26:31.356 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_ADMIN_RESOURCE", 2020-09-08T15:26:31.357 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.357 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.357 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.357 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "admin", 2020-09-08T15:26:31.357 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.358 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.358 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.358 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.358 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.358 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.359 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.359 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.359 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_HOST": { 2020-09-08T15:26:31.359 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_HOST", 2020-09-08T15:26:31.359 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.360 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.360 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.360 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.360 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.360 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.360 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.361 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5318810939788818, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.362 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.362 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_PORT": { 2020-09-08T15:26:31.362 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_PORT", 2020-09-08T15:26:31.362 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.362 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.363 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.363 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "80", 2020-09-08T15:26:31.363 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.363 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.363 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.363 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.364 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.364 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.364 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.364 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.364 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_SCHEME": { 2020-09-08T15:26:31.365 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_SCHEME", 2020-09-08T15:26:31.365 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.365 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.365 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.365 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "http", 2020-09-08T15:26:31.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.366 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.367 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.367 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.367 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_SECRET_KEY": { 2020-09-08T15:26:31.367 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.53239107131958, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.367 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_SECRET_KEY", 2020-09-08T15:26:31.368 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.368 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.368 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.368 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.368 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.368 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.369 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.369 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.369 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.369 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.369 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.370 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.370 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_SSL_VERIFY": { 2020-09-08T15:26:31.370 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_SSL_VERIFY", 2020-09-08T15:26:31.370 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.370 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.371 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.371 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.371 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.371 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.371 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.372 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.372 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.372 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.372 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.372 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.373 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "RGW_API_USER_ID": { 2020-09-08T15:26:31.373 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.532872200012207, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.373 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "RGW_API_USER_ID", 2020-09-08T15:26:31.373 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.373 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.374 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.374 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.374 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.374 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.374 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.374 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.375 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.375 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.375 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.375 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.375 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "USER_PWD_EXPIRATION_SPAN": { 2020-09-08T15:26:31.376 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "USER_PWD_EXPIRATION_SPAN", 2020-09-08T15:26:31.376 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.376 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.376 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.376 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "0", 2020-09-08T15:26:31.376 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.377 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.377 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.377 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.377 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.377 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.378 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.378 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.378 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "USER_PWD_EXPIRATION_WARNING_1": { 2020-09-08T15:26:31.378 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5333576202392578, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.378 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "USER_PWD_EXPIRATION_WARNING_1", 2020-09-08T15:26:31.379 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.379 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.379 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.379 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "10", 2020-09-08T15:26:31.379 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.380 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.380 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.380 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.380 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.380 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.381 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.381 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.381 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "USER_PWD_EXPIRATION_WARNING_2": { 2020-09-08T15:26:31.381 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "USER_PWD_EXPIRATION_WARNING_2", 2020-09-08T15:26:31.382 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.382 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.382 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.382 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "5", 2020-09-08T15:26:31.382 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.383 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.383 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.383 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.383 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.383 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.384 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.384 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.384 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crt_file": { 2020-09-08T15:26:31.384 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "crt_file", 2020-09-08T15:26:31.384 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5338621139526367, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.385 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.385 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.385 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.385 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.385 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.385 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.386 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.386 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.386 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.386 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.386 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.387 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.387 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug": { 2020-09-08T15:26:31.387 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "debug", 2020-09-08T15:26:31.387 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.387 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.388 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.388 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.388 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.388 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.388 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.388 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Enable/disable debug options", 2020-09-08T15:26:31.389 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.389 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.389 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.389 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.389 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "jwt_token_ttl": { 2020-09-08T15:26:31.390 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "jwt_token_ttl", 2020-09-08T15:26:31.390 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.390 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5343666076660156, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.390 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.390 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.391 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "28800", 2020-09-08T15:26:31.391 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.391 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.391 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.391 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.391 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.392 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.392 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.392 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.392 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "key_file": { 2020-09-08T15:26:31.392 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "key_file", 2020-09-08T15:26:31.393 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.393 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.393 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.393 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.393 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.394 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.394 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.394 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.394 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.394 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.394 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.395 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.395 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.395 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.395 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.395 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.396 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5348668098449707, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.396 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.396 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.396 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.396 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.397 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.397 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.397 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.397 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.397 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.397 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.398 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.398 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.398 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.398 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.398 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.399 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.399 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.399 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.399 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.399 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.400 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.400 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.400 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.400 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.400 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.400 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.401 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.401 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.401 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.401 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.401 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.402 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.535407304763794, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.402 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.402 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.402 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.402 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.403 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.403 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.403 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.403 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.403 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.404 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.404 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.404 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.404 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.404 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.404 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.405 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.405 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.405 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.405 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.405 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.406 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.406 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.406 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.406 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.406 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.407 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.407 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.407 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.407 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.407 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5359153747558594, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.407 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.408 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.408 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.408 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.408 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.408 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.409 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "password": { 2020-09-08T15:26:31.409 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "password", 2020-09-08T15:26:31.409 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.409 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.409 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.410 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.410 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.410 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.410 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.410 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.410 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.411 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.411 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.411 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.411 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_addr": { 2020-09-08T15:26:31.411 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_addr", 2020-09-08T15:26:31.412 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.412 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.412 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.412 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "::", 2020-09-08T15:26:31.412 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.413 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.413 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.413 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.413 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5364067554473877, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.413 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.414 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.414 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.414 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.414 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_port": { 2020-09-08T15:26:31.415 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_port", 2020-09-08T15:26:31.415 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.415 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.415 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.415 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "8080", 2020-09-08T15:26:31.416 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.416 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.416 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.416 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.416 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.417 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.417 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.417 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.417 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ssl": { 2020-09-08T15:26:31.417 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ssl", 2020-09-08T15:26:31.418 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.418 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.418 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.418 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.418 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.419 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.419 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.419 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.419 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.419 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.420 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5370755195617676, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.420 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.420 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.420 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ssl_server_port": { 2020-09-08T15:26:31.421 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ssl_server_port", 2020-09-08T15:26:31.421 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.421 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.421 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.421 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "8443", 2020-09-08T15:26:31.422 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.422 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.422 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.422 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.422 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.423 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.423 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.423 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.423 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "standby_behaviour": { 2020-09-08T15:26:31.423 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "standby_behaviour", 2020-09-08T15:26:31.424 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.424 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.424 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.424 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "redirect", 2020-09-08T15:26:31.424 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.425 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.425 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.425 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.425 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "redirect" 2020-09-08T15:26:31.425 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.426 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.426 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.53767991065979, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.426 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.426 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.426 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.427 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.427 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "standby_error_status_code": { 2020-09-08T15:26:31.427 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "standby_error_status_code", 2020-09-08T15:26:31.427 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.427 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.427 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.428 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "500", 2020-09-08T15:26:31.428 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "400", 2020-09-08T15:26:31.428 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "599", 2020-09-08T15:26:31.429 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.429 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.429 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.429 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.429 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.430 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.430 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "url_prefix": { 2020-09-08T15:26:31.430 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "url_prefix", 2020-09-08T15:26:31.430 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.430 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.431 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.431 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.431 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.431 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.431 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.432 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.432 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.432 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5383837223052979, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.432 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.432 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.433 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.433 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "username": { 2020-09-08T15:26:31.433 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "username", 2020-09-08T15:26:31.433 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.433 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.433 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.434 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.434 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.434 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.434 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.434 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.435 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.435 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.435 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.435 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.435 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.436 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.436 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.436 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "devicehealth", 2020-09-08T15:26:31.436 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.437 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.437 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.437 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enable_monitoring": { 2020-09-08T15:26:31.437 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "enable_monitoring", 2020-09-08T15:26:31.437 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.438 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.438 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.438 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.438 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.438 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.439 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5389258861541748, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.439 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.439 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "monitor device health metrics", 2020-09-08T15:26:31.439 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.439 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.440 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.440 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.440 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.440 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.440 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.440 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.441 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.441 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.441 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.441 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.441 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.442 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.442 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.442 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.442 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.442 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.443 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.443 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.443 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.443 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.443 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.443 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.444 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.444 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.444 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.444 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5394110679626465, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.445 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.445 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.445 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.445 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.446 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.446 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.446 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.446 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.446 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.447 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.447 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.447 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.447 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.447 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.448 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.448 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.448 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.448 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.448 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.449 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.449 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.449 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.449 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.449 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.450 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.450 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.450 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.450 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.450 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.451 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.451 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5399270057678223, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.451 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.451 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.451 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.451 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.452 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.452 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.452 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.452 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.453 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.453 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.453 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.453 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.454 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.454 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.454 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.454 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.454 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.455 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mark_out_threshold": { 2020-09-08T15:26:31.455 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "mark_out_threshold", 2020-09-08T15:26:31.455 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:31.455 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.455 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.456 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "2419200", 2020-09-08T15:26:31.456 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.456 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.456 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.456 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "automatically mark OSD if it may fail before this long", 2020-09-08T15:26:31.456 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.540384292602539, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.457 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.457 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.457 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.457 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.457 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pool_name": { 2020-09-08T15:26:31.458 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "pool_name", 2020-09-08T15:26:31.458 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.458 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.458 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.458 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "device_health_metrics", 2020-09-08T15:26:31.459 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.459 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.459 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.459 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "name of pool in which to store device health metrics", 2020-09-08T15:26:31.459 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.459 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.460 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.460 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.460 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "retention_period": { 2020-09-08T15:26:31.460 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "retention_period", 2020-09-08T15:26:31.460 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:31.461 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.461 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.461 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "15552000", 2020-09-08T15:26:31.461 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.461 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.462 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.462 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5408844947814941, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.462 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how long to retain device health metrics", 2020-09-08T15:26:31.462 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.463 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.463 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.463 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.463 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "scrape_frequency": { 2020-09-08T15:26:31.463 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "scrape_frequency", 2020-09-08T15:26:31.464 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:31.464 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.464 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.464 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "86400", 2020-09-08T15:26:31.464 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.465 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.465 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.465 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how frequently to scrape device health metrics", 2020-09-08T15:26:31.465 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.465 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.466 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.466 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.466 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "self_heal": { 2020-09-08T15:26:31.466 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "self_heal", 2020-09-08T15:26:31.466 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.466 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.467 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.467 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.467 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.467 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.467 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5413434505462646, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.468 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.468 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "preemptively heal cluster around devices that may fail", 2020-09-08T15:26:31.468 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.468 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.469 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.469 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.469 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "sleep_interval": { 2020-09-08T15:26:31.469 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "sleep_interval", 2020-09-08T15:26:31.469 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:31.470 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.470 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.470 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "600", 2020-09-08T15:26:31.470 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.470 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.471 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.471 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how frequently to wake up and check device health", 2020-09-08T15:26:31.471 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.472 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.472 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.472 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.472 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warn_threshold": { 2020-09-08T15:26:31.472 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "warn_threshold", 2020-09-08T15:26:31.473 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:31.473 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.473 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.473 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "7257600", 2020-09-08T15:26:31.473 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.541785478591919, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.473 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.474 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.474 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.474 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "raise health warning if OSD may fail before this long", 2020-09-08T15:26:31.474 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.474 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.475 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.475 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.475 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.475 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.475 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.476 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "diskprediction_local", 2020-09-08T15:26:31.476 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.476 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.476 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.476 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.476 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.477 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.477 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.477 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.477 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.477 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.478 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.478 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.478 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.478 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.484 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.485 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.485 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.485 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.485 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.485 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5422980785369873, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.486 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.486 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.486 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.486 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.486 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.487 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.487 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.487 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.487 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.488 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.488 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.488 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.488 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.489 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.489 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.489 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.489 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.489 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.490 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.490 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.490 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.490 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.490 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.491 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.491 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.491 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.491 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.491 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.491 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.492 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5428006649017334, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.492 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.492 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.492 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.492 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.493 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.493 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.493 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.493 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.493 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.494 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.494 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.494 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.494 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.494 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.494 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.495 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.495 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.495 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.495 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.495 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.496 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.496 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.496 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.497 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.497 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.497 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "predict_interval": { 2020-09-08T15:26:31.497 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "predict_interval", 2020-09-08T15:26:31.497 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.498 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.498 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.498 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5433006286621094, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.498 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "86400", 2020-09-08T15:26:31.498 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.499 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.499 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.499 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.499 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.499 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.500 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.500 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.500 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "predictor_model": { 2020-09-08T15:26:31.500 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "predictor_model", 2020-09-08T15:26:31.500 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.501 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.501 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.501 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "prophetstor", 2020-09-08T15:26:31.501 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.501 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.502 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.502 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.502 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.502 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.502 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.503 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.503 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "sleep_interval": { 2020-09-08T15:26:31.503 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "sleep_interval", 2020-09-08T15:26:31.503 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.503 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.504 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.504 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "600", 2020-09-08T15:26:31.504 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5437936782836914, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.504 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.504 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.505 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.505 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.505 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.505 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.506 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.506 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.506 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.506 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.506 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.507 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "influx", 2020-09-08T15:26:31.507 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": false, 2020-09-08T15:26:31.507 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "influxdb python module not found", 2020-09-08T15:26:31.507 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.508 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "batch_size": { 2020-09-08T15:26:31.508 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "batch_size", 2020-09-08T15:26:31.508 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.508 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.508 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.508 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "5000", 2020-09-08T15:26:31.509 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.509 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.509 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.509 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.509 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.510 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.510 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.510 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.510 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "database": { 2020-09-08T15:26:31.510 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "database", 2020-09-08T15:26:31.511 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5443153381347656, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.511 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.511 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.511 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.511 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "ceph", 2020-09-08T15:26:31.512 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.512 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.512 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.512 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.512 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.512 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.513 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.513 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.513 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "hostname": { 2020-09-08T15:26:31.514 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "hostname", 2020-09-08T15:26:31.514 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.514 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.514 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.515 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:31.515 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.515 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.515 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.516 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.516 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.516 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.516 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.516 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.517 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "interval": { 2020-09-08T15:26:31.517 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "interval", 2020-09-08T15:26:31.517 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.517 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.517 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5448439121246338, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.518 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.518 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "30", 2020-09-08T15:26:31.518 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.518 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.518 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.519 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.519 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.519 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.519 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.519 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.520 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.520 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.520 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.520 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.520 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.521 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.521 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.521 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.521 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.521 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.522 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.522 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.522 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.522 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.523 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.523 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.523 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.523 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.523 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.524 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.524 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.524 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5453519821166992, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.524 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.524 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.525 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.525 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.525 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.525 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.525 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.526 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.526 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.526 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.526 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.526 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.527 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.527 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.527 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.527 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.527 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.528 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.528 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.528 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.528 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.528 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.529 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.529 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.529 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.529 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.529 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.530 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.530 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5458214282989502, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.530 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.530 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.531 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.531 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.531 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.531 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.531 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.532 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.532 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.532 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.532 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.532 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.533 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.533 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.533 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.533 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.533 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.533 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.534 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.534 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.534 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.534 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "password": { 2020-09-08T15:26:31.534 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "password", 2020-09-08T15:26:31.535 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.535 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.535 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.535 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:31.535 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.536 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.536 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.536 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5463190078735352, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.536 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.536 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.536 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.537 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.537 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.537 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "port": { 2020-09-08T15:26:31.537 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "port", 2020-09-08T15:26:31.537 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.538 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.538 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.538 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "8086", 2020-09-08T15:26:31.539 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.539 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.539 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.539 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.539 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.540 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.540 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.540 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.540 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ssl": { 2020-09-08T15:26:31.540 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ssl", 2020-09-08T15:26:31.541 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.541 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.541 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.541 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "false", 2020-09-08T15:26:31.541 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.541 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.542 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.542 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.542 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.542 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.542 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5468332767486572, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.543 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.543 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.543 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "threads": { 2020-09-08T15:26:31.543 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "threads", 2020-09-08T15:26:31.543 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.544 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.544 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.544 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "5", 2020-09-08T15:26:31.544 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.544 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.544 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.545 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.545 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.545 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.545 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.545 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.546 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "username": { 2020-09-08T15:26:31.546 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "username", 2020-09-08T15:26:31.546 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.546 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.546 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.547 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:31.547 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.547 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.547 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.548 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.548 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.549 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.549 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.549 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.549 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5473432540893555, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.550 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "verify_ssl": { 2020-09-08T15:26:31.550 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "verify_ssl", 2020-09-08T15:26:31.550 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.550 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.550 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.551 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "true", 2020-09-08T15:26:31.551 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.551 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.551 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.551 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.551 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.552 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.552 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.552 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.552 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.552 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.553 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.553 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "insights", 2020-09-08T15:26:31.553 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.553 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.554 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.554 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.554 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.554 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.554 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.555 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.555 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.555 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.555 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.556 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.556 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.556 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.556 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5479199886322021, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.556 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.557 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.557 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.557 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.557 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.557 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.558 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.558 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.558 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.558 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.559 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.559 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.559 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.559 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.559 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.560 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.560 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.560 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.560 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.560 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.561 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.561 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.561 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.561 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.561 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.562 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.562 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.563 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.563 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.563 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5484399795532227, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.563 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.563 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.564 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.564 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.564 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.565 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.565 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.565 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.565 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.565 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.566 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.566 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.566 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.566 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.566 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.566 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.567 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.567 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.567 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.567 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.567 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.568 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.568 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.568 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.568 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.568 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.569 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.569 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.569 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.569 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.569 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.570 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.570 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.570 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5490143299102783, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.570 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "iostat", 2020-09-08T15:26:31.570 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.571 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.571 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.571 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.571 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.571 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.571 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.572 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.572 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.572 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.572 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.573 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.573 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.573 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.573 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.574 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.574 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.574 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.574 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.574 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.575 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.575 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.575 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.575 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.575 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.576 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.576 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.576 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.576 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.576 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.549501657485962, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.577 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.577 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.577 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.577 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.577 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.578 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.578 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.578 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.578 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.578 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.579 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.579 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.579 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.579 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.579 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.580 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.580 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.580 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.580 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.581 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.581 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.581 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.581 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.582 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.582 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.582 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.582 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.582 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.582 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.583 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.583 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.583 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5500071048736572, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.583 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.583 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.584 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.584 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.584 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.584 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.584 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.585 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.585 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.585 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.585 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.586 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.586 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.586 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.586 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.586 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.587 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "k8sevents", 2020-09-08T15:26:31.587 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.587 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.587 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.587 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "ceph_event_retention_days": { 2020-09-08T15:26:31.588 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "ceph_event_retention_days", 2020-09-08T15:26:31.588 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.588 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.588 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.588 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "7", 2020-09-08T15:26:31.589 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.589 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.589 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.589 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5504837036132812, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.590 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Days to hold ceph event information within local cache", 2020-09-08T15:26:31.590 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.590 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.590 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.590 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.591 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "config_check_secs": { 2020-09-08T15:26:31.591 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "config_check_secs", 2020-09-08T15:26:31.591 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.591 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.591 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.592 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "10", 2020-09-08T15:26:31.592 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "10", 2020-09-08T15:26:31.592 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.592 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.592 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "interval (secs) to check for cluster configuration changes", 2020-09-08T15:26:31.592 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.593 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.593 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.593 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.593 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.593 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.594 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.594 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.594 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.594 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.594 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.595 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.595 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5509395599365234, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.595 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.595 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.595 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.596 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.596 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.596 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.596 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.596 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.596 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.597 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.597 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.597 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.597 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.598 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.598 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.598 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.598 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.599 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.599 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.599 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.599 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.599 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.600 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.600 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.600 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.600 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.600 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.600 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.601 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.601 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.601 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.551436185836792, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.601 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.601 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.602 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.602 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.602 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.602 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.602 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.603 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.603 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.603 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.603 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.603 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.604 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.604 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.604 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.604 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.604 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.604 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.605 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.605 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.605 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.605 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.605 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.606 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.606 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.606 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.607 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.607 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.607 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.607 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.607 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.551978588104248, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.607 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.608 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.608 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.608 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.608 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.608 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "localpool", 2020-09-08T15:26:31.609 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.609 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.609 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.609 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "failure_domain": { 2020-09-08T15:26:31.609 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "failure_domain", 2020-09-08T15:26:31.610 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.610 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.610 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.610 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "host", 2020-09-08T15:26:31.610 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.611 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.611 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.611 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "failure domain for any created local pool", 2020-09-08T15:26:31.611 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.611 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.612 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.612 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.612 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.612 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.612 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.613 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.613 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.613 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.613 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.613 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.614 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5525283813476562, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.614 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.614 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.614 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.615 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.615 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.615 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.615 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.616 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.616 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.616 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.616 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.617 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.617 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.617 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.617 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.617 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.618 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.618 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.618 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.618 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.618 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.618 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.619 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.619 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.619 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.619 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.619 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.620 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.620 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.620 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.620 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5530648231506348, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.620 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.621 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.621 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.621 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.621 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.621 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.622 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.622 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.622 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.623 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.623 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.623 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.623 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.623 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.624 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.624 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.624 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.624 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.624 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.625 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.625 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.625 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.625 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.625 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.625 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.626 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.626 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.626 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.626 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.626 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.553579330444336, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.627 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.627 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.627 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.627 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min_size": { 2020-09-08T15:26:31.627 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "min_size", 2020-09-08T15:26:31.628 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.628 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.628 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.628 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.628 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.628 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.629 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.629 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "default min_size for any created local pool", 2020-09-08T15:26:31.629 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.629 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.629 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.630 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.630 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "num_rep": { 2020-09-08T15:26:31.630 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "num_rep", 2020-09-08T15:26:31.630 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.630 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.631 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.631 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "3", 2020-09-08T15:26:31.631 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.631 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.632 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.632 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "default replica count for any created local pool", 2020-09-08T15:26:31.632 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.632 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5540666580200195, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.633 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.633 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.633 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.633 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pg_num": { 2020-09-08T15:26:31.633 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "pg_num", 2020-09-08T15:26:31.634 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.634 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.634 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.634 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "128", 2020-09-08T15:26:31.634 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.634 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.635 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.635 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "default pg_num for any created local pool", 2020-09-08T15:26:31.635 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.635 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.635 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.636 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.636 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "prefix": { 2020-09-08T15:26:31.636 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "prefix", 2020-09-08T15:26:31.636 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.636 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.637 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.637 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.637 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.637 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.637 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.638 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "name prefix for any created local pool", 2020-09-08T15:26:31.638 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.638 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5545575618743896, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.638 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.638 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.638 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.639 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "subtree": { 2020-09-08T15:26:31.639 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "subtree", 2020-09-08T15:26:31.639 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.639 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.640 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.640 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "rack", 2020-09-08T15:26:31.640 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.640 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.641 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.641 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "CRUSH level for which to create a local pool", 2020-09-08T15:26:31.641 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.641 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.641 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.642 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.642 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.642 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.642 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.642 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "mds_autoscaler", 2020-09-08T15:26:31.642 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.643 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.643 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.643 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.643 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.643 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.644 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.644 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.644 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.644 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.644 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5550892353057861, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.645 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.645 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.645 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.645 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.645 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.646 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.646 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.646 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.646 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.646 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.646 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.647 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.647 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.647 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.647 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.647 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.648 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.648 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.648 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.649 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.649 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.649 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.649 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.649 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.650 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.650 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.650 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.650 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.650 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.650 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.651 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5556058883666992, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.651 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.651 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.651 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.651 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.652 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.652 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.652 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.652 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.652 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.653 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.653 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.653 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.653 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.653 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.653 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.654 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.654 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.654 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.654 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.654 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.655 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.655 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.655 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.655 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.655 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.656 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.656 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.656 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.656 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.656 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5563569068908691, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.657 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.657 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.657 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.658 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.658 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.658 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.658 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.658 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "orchestrator", 2020-09-08T15:26:31.659 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.659 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.659 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.659 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.659 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.660 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.660 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.660 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.660 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.660 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.660 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.661 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.661 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.661 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.661 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.661 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.662 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.662 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.662 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.662 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.662 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.663 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.663 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.663 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.663 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.663 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.556983232498169, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.663 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.664 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.664 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.664 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.664 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.664 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.665 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.665 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.665 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.665 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.666 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.666 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.666 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.666 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.666 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.667 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.667 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.667 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.667 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.667 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.668 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.668 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.668 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.668 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.668 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.669 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.669 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.669 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.669 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.669 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.670 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5575847625732422, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.670 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.670 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.670 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.670 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.671 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.671 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.671 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.671 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.671 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.671 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.672 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.672 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.672 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.672 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.672 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.673 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.673 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.673 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.674 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "orchestrator": { 2020-09-08T15:26:31.674 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "orchestrator", 2020-09-08T15:26:31.674 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.674 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.674 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.675 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:31.675 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.675 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.675 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.675 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "cephadm", 2020-09-08T15:26:31.676 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rook", 2020-09-08T15:26:31.676 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5581412315368652, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.676 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "test_orchestrator" 2020-09-08T15:26:31.676 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.676 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Orchestrator backend", 2020-09-08T15:26:31.676 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.677 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.677 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.677 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.677 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.677 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.678 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.678 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "osd_support", 2020-09-08T15:26:31.678 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.678 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.678 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.679 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.679 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.679 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.679 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.679 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.680 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.680 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.680 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.680 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.680 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.681 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.681 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.681 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.681 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.681 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.682 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.682 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.689 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.689 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.689 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5587797164916992, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.689 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.689 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.690 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.690 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.690 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.690 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.691 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.691 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.691 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.692 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.692 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.692 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.692 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.692 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.693 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.693 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.693 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.693 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.693 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.694 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.694 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.694 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.694 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.695 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.695 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.695 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.695 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.695 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.696 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.696 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5593249797821045, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.696 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.696 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.696 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.697 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.697 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.697 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.697 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.697 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.698 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.698 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.698 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.698 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.698 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.699 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.699 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.699 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.699 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.700 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.700 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.700 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.700 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.701 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.701 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.701 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.701 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.701 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "pg_autoscaler", 2020-09-08T15:26:31.702 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.702 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.702 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.702 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.702 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.703 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.703 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5598809719085693, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.703 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.703 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.703 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.704 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.704 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.704 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.704 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.704 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.704 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.705 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.705 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.705 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.705 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.705 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.706 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.706 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.706 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.706 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.706 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.707 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.707 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.707 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.707 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.707 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.708 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.708 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.708 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.709 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.709 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.709 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.709 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5603926181793213, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.709 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.710 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.710 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.710 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.710 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.710 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.710 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.711 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.711 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.711 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.711 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.711 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.712 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.712 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.712 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.712 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.712 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.713 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.713 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.713 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.713 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.713 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.714 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.714 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.714 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.714 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.714 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.714 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.715 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.715 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.715 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.560905933380127, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.715 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.715 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.716 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.716 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.716 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.717 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.717 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.717 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "sleep_interval": { 2020-09-08T15:26:31.717 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "sleep_interval", 2020-09-08T15:26:31.718 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.718 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.718 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.718 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "60", 2020-09-08T15:26:31.718 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.719 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.719 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.719 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.719 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.719 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.720 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.720 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.720 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.720 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.720 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.720 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "progress", 2020-09-08T15:26:31.721 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.721 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.721 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.721 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.721 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.722 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.722 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.722 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.561450719833374, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.722 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.722 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.722 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.723 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.723 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.723 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.723 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.724 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.724 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.724 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.724 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.725 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.725 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.725 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.725 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.726 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.726 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.726 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.726 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.726 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.727 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.727 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.727 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.727 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.727 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.728 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.728 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.728 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.728 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.728 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.729 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5619566440582275, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.729 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.729 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.729 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.729 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.730 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.730 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.730 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.730 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.730 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.731 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.731 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.731 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.731 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.731 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.731 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.732 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.732 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.732 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.732 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.733 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.733 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.733 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.733 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.734 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.734 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.734 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.734 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.734 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.735 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.735 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.735 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5624644756317139, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.735 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.736 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.736 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.736 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.736 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.736 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.736 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max_completed_events": { 2020-09-08T15:26:31.737 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "max_completed_events", 2020-09-08T15:26:31.737 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.737 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.737 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.737 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "50", 2020-09-08T15:26:31.738 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.738 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.738 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.738 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "number of past completed events to remember", 2020-09-08T15:26:31.738 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.739 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.739 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.739 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.739 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "persist_interval": { 2020-09-08T15:26:31.739 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "persist_interval", 2020-09-08T15:26:31.740 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:31.740 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.740 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.740 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "5", 2020-09-08T15:26:31.740 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.740 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5629851818084717, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.741 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.741 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.741 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "how frequently to persist completed events", 2020-09-08T15:26:31.742 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.742 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.742 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.742 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.742 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.743 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.743 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.743 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "prometheus", 2020-09-08T15:26:31.743 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.743 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.744 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.744 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.744 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.744 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.744 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.744 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.745 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.745 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.745 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.745 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.745 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.746 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.746 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.746 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.746 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.746 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.747 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.747 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.747 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.747 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5635969638824463, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.747 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.748 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.748 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.748 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.748 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.748 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.748 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.749 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.749 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.749 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.749 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.749 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.750 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.750 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.750 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.751 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.751 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.751 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.751 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.751 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.752 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.752 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.752 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.752 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.752 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.753 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.753 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.753 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.753 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.753 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5641686916351318, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.754 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.754 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.754 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.754 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.754 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.755 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.755 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.755 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.755 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.755 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.755 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.756 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.756 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.756 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.756 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.756 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.757 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.757 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.757 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.757 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.758 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.758 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.758 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.758 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rbd_stats_pools": { 2020-09-08T15:26:31.759 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rbd_stats_pools", 2020-09-08T15:26:31.759 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.759 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.759 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.759 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.760 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.564692735671997, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.760 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.760 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.760 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.760 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.761 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.761 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.761 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.761 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.761 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rbd_stats_pools_refresh_interval": { 2020-09-08T15:26:31.762 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rbd_stats_pools_refresh_interval", 2020-09-08T15:26:31.762 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.762 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.762 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.762 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "300", 2020-09-08T15:26:31.763 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.763 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.763 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.763 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.763 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.764 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.764 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.764 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.764 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "scrape_interval": { 2020-09-08T15:26:31.764 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "scrape_interval", 2020-09-08T15:26:31.765 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.765 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.765 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.765 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.765 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.765 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.565185546875, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.766 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.766 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.766 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.766 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.766 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.767 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.767 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.767 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_addr": { 2020-09-08T15:26:31.767 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_addr", 2020-09-08T15:26:31.767 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.768 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.768 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.768 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.768 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.768 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.769 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.769 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.769 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.769 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.769 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.770 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.770 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_port": { 2020-09-08T15:26:31.770 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_port", 2020-09-08T15:26:31.770 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.770 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.770 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.771 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.771 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.771 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.771 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5657167434692383, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.772 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.772 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.772 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.772 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.772 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.772 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.773 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "stale_cache_strategy": { 2020-09-08T15:26:31.773 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "stale_cache_strategy", 2020-09-08T15:26:31.773 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.773 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.773 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.774 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.774 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.774 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.774 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.774 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.775 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.775 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.775 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.775 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.775 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.776 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.776 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.776 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rbd_support", 2020-09-08T15:26:31.776 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.776 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.776 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.777 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.777 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.777 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.777 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.777 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.778 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5663259029388428, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.778 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.778 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.778 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.778 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.779 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.779 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.779 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.779 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.779 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.780 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.780 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.780 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.780 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.780 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.780 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.781 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.781 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.781 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.781 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.781 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.782 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.782 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.782 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.782 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.782 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.783 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.783 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.783 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.783 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.783 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.783 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5668823719024658, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.784 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.784 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.784 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.784 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.784 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.785 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.785 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.785 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.785 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.785 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.786 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.786 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.786 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.786 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.786 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.786 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.787 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.787 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.787 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.787 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.787 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.788 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.788 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.788 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.788 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.788 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.789 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.789 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.789 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.789 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.790 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5674498081207275, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.790 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.790 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.790 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.790 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.791 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.791 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mirror_snapshot_schedule": { 2020-09-08T15:26:31.791 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "mirror_snapshot_schedule", 2020-09-08T15:26:31.791 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.791 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.792 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.792 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.792 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.792 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.792 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.793 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.793 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.793 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.793 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.793 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.794 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "trash_purge_schedule": { 2020-09-08T15:26:31.794 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "trash_purge_schedule", 2020-09-08T15:26:31.794 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.794 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.794 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.795 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.795 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.795 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.795 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.795 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5679893493652344, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.796 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.796 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.796 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.796 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.796 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.797 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.797 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.797 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.797 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "restful", 2020-09-08T15:26:31.797 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.797 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.798 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.798 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enable_auth": { 2020-09-08T15:26:31.798 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "enable_auth", 2020-09-08T15:26:31.798 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.798 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.799 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.799 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.799 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.799 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.799 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.800 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.800 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.800 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.800 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.800 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.801 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "key_file": { 2020-09-08T15:26:31.801 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "key_file", 2020-09-08T15:26:31.801 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.801 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.801 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.801 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.802 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5686900615692139, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.802 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.802 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.802 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.802 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.803 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.803 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.803 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.803 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.803 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.804 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.804 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.804 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.804 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.804 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.805 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.805 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.805 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.805 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.806 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.806 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.806 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.806 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.806 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.806 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.807 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.807 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.807 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.807 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.807 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.808 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.808 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.808 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.569347620010376, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.808 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.808 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.809 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.809 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.809 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.809 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.809 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.809 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.810 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.810 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.810 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.810 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.810 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.811 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.811 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.811 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.811 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.811 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.812 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.812 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.812 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.812 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.812 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.812 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.813 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.813 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.813 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.813 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.813 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.814 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5699646472930908, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.814 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.814 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.814 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.814 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.815 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.815 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.815 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.815 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.815 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.815 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.816 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.816 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.816 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.816 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.817 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.817 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.817 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.817 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.818 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_addr": { 2020-09-08T15:26:31.818 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_addr", 2020-09-08T15:26:31.818 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.818 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.818 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.819 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.819 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.819 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.819 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.819 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.820 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.820 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.820 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5705080032348633, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.820 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.820 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.821 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "server_port": { 2020-09-08T15:26:31.821 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "server_port", 2020-09-08T15:26:31.821 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.821 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.821 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.822 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.822 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.822 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.822 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.822 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.823 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.823 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.823 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.823 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.823 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.824 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.824 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.824 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rook", 2020-09-08T15:26:31.824 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.824 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.825 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.825 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.825 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.825 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.825 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.826 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.826 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.826 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.826 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.826 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.827 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.827 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5710957050323486, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.827 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.827 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.827 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.828 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.828 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.828 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.828 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.828 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.829 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.829 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.829 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.829 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.829 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.830 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.830 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.830 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.830 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.830 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.830 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.831 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.831 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.831 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.831 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.831 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.832 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.832 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.832 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.832 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.832 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.833 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5716979503631592, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.833 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.833 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.833 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.833 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.833 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.834 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.834 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.834 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.834 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.834 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.835 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.835 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.835 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.835 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.835 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.836 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.836 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.836 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.836 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.836 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.836 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.837 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.837 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.837 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.837 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.837 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.838 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.838 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.838 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.838 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.838 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.839 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5723564624786377, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.839 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.839 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.839 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.839 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "selftest", 2020-09-08T15:26:31.839 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.840 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.840 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.840 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.840 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.840 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.841 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.841 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.841 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.841 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.841 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.842 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.842 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.842 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.842 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.842 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.843 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.843 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.843 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.843 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.844 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.844 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.844 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.844 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.844 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.844 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.845 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.845 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.845 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5730266571044922, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.845 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.845 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.846 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.846 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.846 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.846 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.847 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.847 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.847 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.847 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.847 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.848 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.848 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.848 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.848 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.848 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.849 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.849 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.850 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.850 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.850 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.850 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.850 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.851 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.851 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.851 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.851 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.851 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.852 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.852 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.573641300201416, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.852 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.852 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.852 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.853 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.853 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.853 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.853 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.853 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.853 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.854 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.854 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.854 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.854 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.854 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.855 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.855 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.855 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "roption1": { 2020-09-08T15:26:31.855 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "roption1", 2020-09-08T15:26:31.855 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.856 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.856 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.856 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.856 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.856 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.857 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.857 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.857 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.857 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.857 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.857 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.858 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "roption2": { 2020-09-08T15:26:31.858 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5742757320404053, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.858 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "roption2", 2020-09-08T15:26:31.858 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.858 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.859 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.859 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "xyz", 2020-09-08T15:26:31.859 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.859 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.859 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.860 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.860 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.860 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.860 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.860 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.860 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption1": { 2020-09-08T15:26:31.861 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption1", 2020-09-08T15:26:31.861 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.861 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.861 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.861 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.862 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.862 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.862 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.862 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.862 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.863 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.863 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.863 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.863 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption2": { 2020-09-08T15:26:31.863 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption2", 2020-09-08T15:26:31.864 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5748836994171143, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.864 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.864 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.864 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.864 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.864 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.865 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.865 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.865 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.865 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.865 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.866 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.866 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.866 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption3": { 2020-09-08T15:26:31.866 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption3", 2020-09-08T15:26:31.866 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "float", 2020-09-08T15:26:31.867 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.867 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.867 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.867 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.867 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.868 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.868 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.868 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.868 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.868 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.868 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.869 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption4": { 2020-09-08T15:26:31.869 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption4", 2020-09-08T15:26:31.869 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.869 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.869 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5754361152648926, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.870 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.870 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.870 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.870 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.870 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.871 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.871 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.871 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.871 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.871 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.871 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption5": { 2020-09-08T15:26:31.872 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption5", 2020-09-08T15:26:31.872 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.872 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.872 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.872 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.873 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.873 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.873 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.873 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.873 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.874 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.874 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.874 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.874 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rwoption6": { 2020-09-08T15:26:31.874 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "rwoption6", 2020-09-08T15:26:31.875 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.875 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.875 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.875 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.875 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5759830474853516, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.876 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.876 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.876 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.876 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.876 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.877 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.877 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.877 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.877 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "testkey": { 2020-09-08T15:26:31.877 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "testkey", 2020-09-08T15:26:31.877 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.878 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.878 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.878 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.878 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.878 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.879 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.879 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.879 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.879 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.879 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.880 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.880 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "testlkey": { 2020-09-08T15:26:31.880 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "testlkey", 2020-09-08T15:26:31.880 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.880 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.881 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.881 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.881 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.881 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.881 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5765979290008545, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.882 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.882 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.882 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.882 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.882 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.882 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.883 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "testnewline": { 2020-09-08T15:26:31.883 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "testnewline", 2020-09-08T15:26:31.883 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.883 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.883 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.884 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.884 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.884 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.884 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.884 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.885 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.885 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.885 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.885 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.885 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.886 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.886 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.886 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "snap_schedule", 2020-09-08T15:26:31.886 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.892 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.892 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.893 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.893 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.893 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.893 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.893 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.894 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5775113105773926, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.894 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.894 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.894 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.894 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.895 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.895 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.895 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.895 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.895 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.896 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.896 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.896 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.896 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.896 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.897 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.897 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.897 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.897 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.897 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.898 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.898 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.898 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.898 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.898 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.899 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.899 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.899 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.899 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.899 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.899 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.900 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.900 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5784478187561035, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.900 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.900 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.900 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.901 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.901 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.901 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.901 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.901 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.902 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.902 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.902 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.902 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.902 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.902 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.903 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.903 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.903 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.903 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.903 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.904 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.904 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.904 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.904 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.904 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.905 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.905 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.905 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.905 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.905 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.905 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5793719291687012, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.906 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.906 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.906 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.906 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.906 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.907 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.907 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.907 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.907 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "status", 2020-09-08T15:26:31.907 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.908 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.908 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.908 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.908 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.908 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.909 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.909 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.909 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.909 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.909 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.910 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.910 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.910 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.910 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.910 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.910 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.911 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.911 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.911 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.911 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.911 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.912 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.912 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.912 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5800142288208008, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.912 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.912 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.912 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.913 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.913 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.913 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.913 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.913 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.914 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.914 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.914 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.914 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.914 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.915 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.915 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.915 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.915 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.915 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.916 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.916 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.916 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.916 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.917 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.917 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.917 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.917 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.917 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.918 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.918 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.918 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5805001258850098, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.918 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.919 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.919 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.919 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.919 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.919 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.920 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.920 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.920 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.920 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.920 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.921 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.921 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.921 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.921 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.921 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.922 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.922 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.922 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.922 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.922 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.922 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.923 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.923 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "telegraf", 2020-09-08T15:26:31.923 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.923 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.923 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.924 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "address": { 2020-09-08T15:26:31.924 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "address", 2020-09-08T15:26:31.924 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.924 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.924 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.925 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.581033706665039, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.925 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "unixgram:///tmp/telegraf.sock", 2020-09-08T15:26:31.925 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.925 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.925 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.925 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.926 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.926 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.926 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.926 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.926 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "interval": { 2020-09-08T15:26:31.927 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "interval", 2020-09-08T15:26:31.927 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:31.927 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.927 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.927 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "15", 2020-09-08T15:26:31.928 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.928 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.928 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.928 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.928 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.928 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.929 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.929 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.929 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.929 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.929 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.930 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.930 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.930 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.930 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.930 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.581533432006836, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.931 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.931 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.931 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.931 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.931 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.932 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.932 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.932 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.932 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.932 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.932 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.933 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.933 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.933 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.933 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.933 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.934 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.934 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.934 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.934 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.934 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.935 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.935 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.935 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.935 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.935 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.935 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.936 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.936 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.936 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5820353031158447, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.936 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.936 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.937 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.937 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.937 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.937 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.937 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.938 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.938 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.938 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.938 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.938 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.939 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.939 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.939 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.939 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.939 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.939 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.940 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.940 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.940 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.940 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.940 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.941 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.941 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.941 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.941 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.941 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.942 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.942 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.942 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.582554578781128, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.942 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.942 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.942 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.943 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.943 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.943 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.943 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.943 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "telemetry", 2020-09-08T15:26:31.944 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.944 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.944 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.944 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "channel_basic": { 2020-09-08T15:26:31.944 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "channel_basic", 2020-09-08T15:26:31.945 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.945 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.945 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.945 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.945 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.945 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.946 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.946 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "Share basic cluster information (size, version)", 2020-09-08T15:26:31.946 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.946 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.946 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.947 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.947 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "channel_crash": { 2020-09-08T15:26:31.947 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "channel_crash", 2020-09-08T15:26:31.947 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.947 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.948 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.948 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5830469131469727, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.948 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.948 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.948 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.948 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.949 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.949 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.949 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.949 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.949 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.950 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "channel_device": { 2020-09-08T15:26:31.950 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "channel_device", 2020-09-08T15:26:31.950 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.950 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.950 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.951 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "True", 2020-09-08T15:26:31.951 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.951 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.951 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.951 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.951 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.952 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.952 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.952 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.952 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "channel_ident": { 2020-09-08T15:26:31.952 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "channel_ident", 2020-09-08T15:26:31.953 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.953 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.953 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.953 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.953 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.954 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5835363864898682, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.954 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.954 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.954 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.954 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.954 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.955 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.955 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.955 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "contact": { 2020-09-08T15:26:31.955 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "contact", 2020-09-08T15:26:31.955 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.956 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.956 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.956 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:31.956 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.956 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.957 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.957 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.957 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.957 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.957 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.957 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.958 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "description": { 2020-09-08T15:26:31.958 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "description", 2020-09-08T15:26:31.958 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.958 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.958 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.959 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:31.959 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.959 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.959 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.584054946899414, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.959 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.960 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.960 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.960 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.960 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.960 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.961 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "device_url": { 2020-09-08T15:26:31.961 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "device_url", 2020-09-08T15:26:31.961 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.961 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.961 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.961 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "https://telemetry.ceph.com/device", 2020-09-08T15:26:31.962 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.962 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.962 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.962 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.962 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.963 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.963 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.963 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.963 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enabled": { 2020-09-08T15:26:31.963 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "enabled", 2020-09-08T15:26:31.964 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.964 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.964 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.964 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.964 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.964 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.965 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.965 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5845415592193604, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.965 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.965 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.965 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.966 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.966 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.966 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "interval": { 2020-09-08T15:26:31.966 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "interval", 2020-09-08T15:26:31.966 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.967 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.967 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.967 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "24", 2020-09-08T15:26:31.967 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "8", 2020-09-08T15:26:31.967 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.967 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.968 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.968 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.968 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.968 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.968 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.969 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "last_opt_revision": { 2020-09-08T15:26:31.969 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "last_opt_revision", 2020-09-08T15:26:31.969 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:31.969 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.969 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.970 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "1", 2020-09-08T15:26:31.970 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.970 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.970 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.970 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.971 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.971 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5850424766540527, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.971 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.971 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.971 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.972 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "leaderboard": { 2020-09-08T15:26:31.972 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "leaderboard", 2020-09-08T15:26:31.972 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.972 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.972 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.973 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.973 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.973 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.973 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.973 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.973 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.974 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.974 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.974 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.974 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.974 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.975 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.975 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.975 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.975 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.975 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.976 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.976 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.976 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.976 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.976 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.976 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.977 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5855967998504639, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.977 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.977 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.977 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.977 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.978 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.978 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.978 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.978 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.978 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:31.979 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:31.979 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.979 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.979 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.979 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.980 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.980 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.980 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.980 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.980 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.981 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.981 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.981 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.981 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:31.981 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:31.982 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.982 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.982 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.982 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:31.982 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.983 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.586151361465454, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.983 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.983 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:31.983 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:31.983 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:31.984 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:31.984 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:31.984 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:31.984 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:31.984 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:31.984 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.985 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.985 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.985 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.985 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.985 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:31.986 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:31.986 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:31.986 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.986 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.986 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:31.987 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.987 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.987 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.987 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.987 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.988 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.988 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.988 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.988 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "organization": { 2020-09-08T15:26:31.988 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "organization", 2020-09-08T15:26:31.988 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5867314338684082, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.989 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.989 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.989 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.989 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:31.989 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.990 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.990 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.990 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.990 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.990 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.991 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.991 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.991 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "proxy": { 2020-09-08T15:26:31.991 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "proxy", 2020-09-08T15:26:31.991 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.991 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.992 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.992 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:31.992 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.992 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.993 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.993 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.993 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.993 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.994 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.994 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.994 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "url": { 2020-09-08T15:26:31.994 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "url", 2020-09-08T15:26:31.994 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.995 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.995 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:31.995 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5872793197631836, stop=False, exit=None, reads=[8] 2020-09-08T15:26:31.995 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "https://telemetry.ceph.com/report", 2020-09-08T15:26:31.995 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:31.995 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:31.996 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:31.996 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:31.996 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:31.996 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:31.996 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:31.997 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.997 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:31.997 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:31.997 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:31.997 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "test_orchestrator", 2020-09-08T15:26:31.998 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:31.998 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:31.998 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:31.998 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:31.998 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:31.999 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:31.999 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:31.999 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:31.999 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:31.999 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:32.000 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:32.000 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:32.000 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:32.000 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:32.000 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:32.000 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:32.001 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:32.001 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:32.001 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5878376960754395, stop=False, exit=None, reads=[8] 2020-09-08T15:26:32.001 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:32.001 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:32.002 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:32.002 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:32.002 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:32.002 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:32.002 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:32.003 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:32.003 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:32.003 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:32.003 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:32.003 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:32.004 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:32.004 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:32.004 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:32.004 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:32.004 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:32.005 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:32.005 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:32.005 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:32.005 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:32.005 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:32.006 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:32.006 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:32.006 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:32.006 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:32.006 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:32.006 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:32.007 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:32.007 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.588348150253296, stop=False, exit=None, reads=[8] 2020-09-08T15:26:32.007 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:32.007 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:32.007 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:32.008 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:32.008 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:32.008 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:32.008 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:32.008 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:32.009 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:32.009 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:32.009 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:32.009 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:32.009 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:32.010 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:32.010 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:32.010 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:32.010 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:32.010 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:32.010 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:32.011 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:32.011 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:32.011 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:32.011 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:32.011 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:32.012 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:32.012 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:32.012 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:32.012 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:32.012 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:32.012 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "volumes", 2020-09-08T15:26:32.013 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:32.013 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:32.013 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:32.013 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5889294147491455, stop=False, exit=None, reads=[8] 2020-09-08T15:26:32.013 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:32.014 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:32.014 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:32.014 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:32.014 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:32.014 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:32.015 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:32.015 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:32.015 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:32.015 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:32.015 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:32.016 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:32.016 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:32.016 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:32.016 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:32.016 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:32.016 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:32.017 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:32.017 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:32.017 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:32.017 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:32.017 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:32.018 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:32.018 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:32.018 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:32.018 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:32.019 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:32.019 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:32.019 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:32.019 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:32.020 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.589461088180542, stop=False, exit=None, reads=[8] 2020-09-08T15:26:32.020 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:32.020 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:32.020 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:32.020 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:32.021 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:32.021 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:32.021 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:32.021 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:32.021 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:32.022 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:32.022 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:32.022 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:32.022 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:32.022 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:32.023 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:32.023 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:32.023 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:32.023 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:32.023 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:32.024 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:32.024 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:32.024 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:32.024 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:32.024 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:32.024 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:32.025 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:32.025 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:32.025 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:32.025 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:32.025 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:32.026 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5899853706359863, stop=False, exit=None, reads=[8] 2020-09-08T15:26:32.026 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:32.026 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:32.026 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:32.026 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:32.027 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:32.027 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:32.027 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:32.027 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:32.027 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:32.028 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:32.028 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:32.028 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:32.028 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:32.028 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "zabbix", 2020-09-08T15:26:32.028 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "can_run": true, 2020-09-08T15:26:32.029 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error_string": "", 2020-09-08T15:26:32.029 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "module_options": { 2020-09-08T15:26:32.029 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "discovery_interval": { 2020-09-08T15:26:32.029 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "discovery_interval", 2020-09-08T15:26:32.029 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:32.030 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:32.030 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:32.030 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "100", 2020-09-08T15:26:32.030 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:32.030 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:32.031 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:32.031 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:32.031 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:32.031 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:32.031 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:32.031 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:32.032 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "identifier": { 2020-09-08T15:26:32.032 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5905437469482422, stop=False, exit=None, reads=[8] 2020-09-08T15:26:32.032 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "identifier", 2020-09-08T15:26:32.032 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:32.032 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:32.033 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:32.033 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:32.033 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:32.033 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:32.033 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:32.034 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:32.034 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:32.034 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:32.034 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:32.034 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:32.035 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "interval": { 2020-09-08T15:26:32.035 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "interval", 2020-09-08T15:26:32.035 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "secs", 2020-09-08T15:26:32.035 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:32.035 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:32.035 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "60", 2020-09-08T15:26:32.036 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:32.036 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:32.036 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:32.036 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:32.036 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:32.037 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:32.037 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:32.037 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:32.037 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_level": { 2020-09-08T15:26:32.037 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_level", 2020-09-08T15:26:32.038 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5910475254058838, stop=False, exit=None, reads=[8] 2020-09-08T15:26:32.038 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:32.038 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:32.038 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:32.038 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "", 2020-09-08T15:26:32.039 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:32.039 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:32.039 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:32.039 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:32.039 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:32.039 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:32.040 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:32.040 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:32.040 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:32.040 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:32.040 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:32.041 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:32.041 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:32.041 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:32.041 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:32.041 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster": { 2020-09-08T15:26:32.042 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster", 2020-09-08T15:26:32.042 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:32.042 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:32.042 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:32.042 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:32.043 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:32.043 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:32.043 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:32.043 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:32.043 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:32.043 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5915782451629639, stop=False, exit=None, reads=[8] 2020-09-08T15:26:32.044 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:32.044 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:32.044 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:32.044 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_cluster_level": { 2020-09-08T15:26:32.044 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_cluster_level", 2020-09-08T15:26:32.045 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:32.045 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:32.045 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:32.045 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "info", 2020-09-08T15:26:32.045 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:32.046 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:32.046 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [ 2020-09-08T15:26:32.046 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "", 2020-09-08T15:26:32.046 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "critical", 2020-09-08T15:26:32.046 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "debug", 2020-09-08T15:26:32.046 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "error", 2020-09-08T15:26:32.047 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "info", 2020-09-08T15:26:32.047 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "warning" 2020-09-08T15:26:32.047 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:32.047 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:32.047 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:32.048 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:32.048 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:32.048 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:32.048 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "log_to_file": { 2020-09-08T15:26:32.048 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "log_to_file", 2020-09-08T15:26:32.049 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "bool", 2020-09-08T15:26:32.049 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:32.049 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 1, 2020-09-08T15:26:32.049 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "False", 2020-09-08T15:26:32.049 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5921130180358887, stop=False, exit=None, reads=[8] 2020-09-08T15:26:32.049 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:32.050 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:32.050 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:32.050 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:32.050 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:32.050 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:32.051 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:32.051 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:32.051 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "zabbix_host": { 2020-09-08T15:26:32.051 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "zabbix_host", 2020-09-08T15:26:32.051 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:32.052 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:32.052 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:32.052 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "None", 2020-09-08T15:26:32.052 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:32.052 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:32.053 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:32.053 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:32.053 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:32.053 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:32.053 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:32.053 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:32.054 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "zabbix_port": { 2020-09-08T15:26:32.054 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "zabbix_port", 2020-09-08T15:26:32.054 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "int", 2020-09-08T15:26:32.054 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:32.054 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:32.055 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "10051", 2020-09-08T15:26:32.055 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:32.055 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.592628002166748, stop=False, exit=None, reads=[8] 2020-09-08T15:26:32.055 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:32.055 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:32.056 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:32.056 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:32.056 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:32.056 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:32.056 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:32.056 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "zabbix_sender": { 2020-09-08T15:26:32.057 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "name": "zabbix_sender", 2020-09-08T15:26:32.057 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "str", 2020-09-08T15:26:32.057 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "level": "advanced", 2020-09-08T15:26:32.057 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "flags": 0, 2020-09-08T15:26:32.057 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "default_value": "/usr/bin/zabbix_sender", 2020-09-08T15:26:32.058 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "min": "", 2020-09-08T15:26:32.058 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "max": "", 2020-09-08T15:26:32.058 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "enum_allowed": [], 2020-09-08T15:26:32.058 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "desc": "", 2020-09-08T15:26:32.058 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "long_desc": "", 2020-09-08T15:26:32.059 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "tags": [], 2020-09-08T15:26:32.059 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "see_also": [] 2020-09-08T15:26:32.059 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:32.059 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:32.059 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:32.059 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:32.060 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "services": {}, 2020-09-08T15:26:32.060 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "always_on_modules": { 2020-09-08T15:26:32.060 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nautilus": [ 2020-09-08T15:26:32.060 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "balancer", 2020-09-08T15:26:32.060 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crash", 2020-09-08T15:26:32.061 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "devicehealth", 2020-09-08T15:26:32.061 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "orchestrator_cli", 2020-09-08T15:26:32.061 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "progress", 2020-09-08T15:26:32.061 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rbd_support", 2020-09-08T15:26:32.061 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "status", 2020-09-08T15:26:32.062 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "volumes" 2020-09-08T15:26:32.062 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:32.062 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.593247652053833, stop=False, exit=None, reads=[8] 2020-09-08T15:26:32.062 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "octopus": [ 2020-09-08T15:26:32.062 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "balancer", 2020-09-08T15:26:32.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crash", 2020-09-08T15:26:32.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "devicehealth", 2020-09-08T15:26:32.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "orchestrator", 2020-09-08T15:26:32.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pg_autoscaler", 2020-09-08T15:26:32.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "progress", 2020-09-08T15:26:32.063 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rbd_support", 2020-09-08T15:26:32.064 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "status", 2020-09-08T15:26:32.064 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "telemetry", 2020-09-08T15:26:32.064 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "volumes" 2020-09-08T15:26:32.064 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:32.064 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pacific": [ 2020-09-08T15:26:32.065 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "balancer", 2020-09-08T15:26:32.065 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "crash", 2020-09-08T15:26:32.065 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "devicehealth", 2020-09-08T15:26:32.065 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "orchestrator", 2020-09-08T15:26:32.065 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "pg_autoscaler", 2020-09-08T15:26:32.065 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "progress", 2020-09-08T15:26:32.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "rbd_support", 2020-09-08T15:26:32.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "status", 2020-09-08T15:26:32.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "telemetry", 2020-09-08T15:26:32.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "volumes" 2020-09-08T15:26:32.066 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ], 2020-09-08T15:26:32.067 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "last_failure_osd_epoch": 4, 2020-09-08T15:26:32.067 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "active_clients": [ 2020-09-08T15:26:32.067 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:32.067 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addrvec": [ 2020-09-08T15:26:32.067 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:32.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "v2", 2020-09-08T15:26:32.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addr": "172.21.15.104:0", 2020-09-08T15:26:32.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nonce": 2568432190 2020-09-08T15:26:32.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:32.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ] 2020-09-08T15:26:32.068 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:32.069 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:32.069 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addrvec": [ 2020-09-08T15:26:32.069 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:32.069 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "v2", 2020-09-08T15:26:32.069 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5939185619354248, stop=False, exit=None, reads=[8] 2020-09-08T15:26:32.070 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addr": "172.21.15.104:0", 2020-09-08T15:26:32.070 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nonce": 2021933044 2020-09-08T15:26:32.070 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:32.070 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ] 2020-09-08T15:26:32.070 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout }, 2020-09-08T15:26:32.071 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:32.071 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addrvec": [ 2020-09-08T15:26:32.071 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:32.071 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "type": "v2", 2020-09-08T15:26:32.071 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "addr": "172.21.15.104:0", 2020-09-08T15:26:32.071 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "nonce": 1270031705 2020-09-08T15:26:32.072 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:32.072 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ] 2020-09-08T15:26:32.072 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:32.072 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout ] 2020-09-08T15:26:32.072 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:32.073 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:32.073 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.5942392349243164, stop=False, exit=None, reads=[8] 2020-09-08T15:26:32.074 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:30 smithi104 bash[9203]: audit 2020-09-08T15:26:30.668598+0000 mon.a (mon.0) 90 : audit [DBG] from='client.? v1:172.21.15.104:0/3632961811' entity='client.admin' cmd=[{"prefix": "mgr dump"}]: dispatch 2020-09-08T15:26:32.075 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.0991384983062744, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:26:32.075 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.0992801189422607, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:26:32.075 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Waiting for the mgr to restart... 2020-09-08T15:26:32.075 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Waiting for mgr epoch 9... 2020-09-08T15:26:32.075 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'tell', 'mgr', 'mgr_status'] 2020-09-08T15:26:32.076 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f tell mgr mgr_status 2020-09-08T15:26:33.578 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:26:33 smithi104 bash[9475]: Warning: Permanently added 'smithi104,172.21.15.104' (ECDSA) to the list of known hosts. 2020-09-08T15:26:33.578 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:33 smithi104 bash[9203]: cluster 2020-09-08T15:26:33.248134+0000 mon.a (mon.0) 91 : cluster [INF] Active manager daemon y restarted 2020-09-08T15:26:33.579 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:33 smithi104 bash[9203]: cluster 2020-09-08T15:26:33.249134+0000 mon.a (mon.0) 92 : cluster [INF] Activating manager daemon y 2020-09-08T15:26:33.579 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:33 smithi104 bash[9203]: cluster 2020-09-08T15:26:33.253073+0000 mon.a (mon.0) 93 : cluster [DBG] osdmap e5: 0 total, 0 up, 0 in 2020-09-08T15:26:34.421 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout { 2020-09-08T15:26:34.421 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "mgrmap_epoch": 11, 2020-09-08T15:26:34.422 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout "initialized": true 2020-09-08T15:26:34.422 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout } 2020-09-08T15:26:34.422 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=3.1355762481689453, stop=False, exit=None, reads=[8] 2020-09-08T15:26:34.682 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:34 smithi104 bash[9203]: cluster 2020-09-08T15:26:33.303250+0000 mon.a (mon.0) 94 : cluster [DBG] mgrmap e10: y(active, starting, since 0.0541942s) 2020-09-08T15:26:34.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:34 smithi104 bash[9203]: audit 2020-09-08T15:26:33.310954+0000 mon.a (mon.0) 95 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2020-09-08T15:26:34.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:34 smithi104 bash[9203]: audit 2020-09-08T15:26:33.312352+0000 mon.a (mon.0) 96 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2020-09-08T15:26:34.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:34 smithi104 bash[9203]: audit 2020-09-08T15:26:33.312683+0000 mon.a (mon.0) 97 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2020-09-08T15:26:34.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:34 smithi104 bash[9203]: audit 2020-09-08T15:26:33.312928+0000 mon.a (mon.0) 98 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2020-09-08T15:26:34.684 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:34 smithi104 bash[9203]: audit 2020-09-08T15:26:33.313409+0000 mon.a (mon.0) 99 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config-key ls"}]: dispatch 2020-09-08T15:26:34.684 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:34 smithi104 bash[9203]: audit 2020-09-08T15:26:33.314719+0000 mon.a (mon.0) 100 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.smithi104"}]: dispatch 2020-09-08T15:26:34.684 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:34 smithi104 bash[9203]: audit 2020-09-08T15:26:33.315076+0000 mon.a (mon.0) 101 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/inventory"}]: dispatch 2020-09-08T15:26:34.684 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:34 smithi104 bash[9203]: audit 2020-09-08T15:26:33.315412+0000 mon.a (mon.0) 102 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/osd_remove_queue"}]: dispatch 2020-09-08T15:26:34.684 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:34 smithi104 bash[9203]: audit 2020-09-08T15:26:33.315710+0000 mon.a (mon.0) 103 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_key"}]: dispatch 2020-09-08T15:26:34.685 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:34 smithi104 bash[9203]: audit 2020-09-08T15:26:33.316000+0000 mon.a (mon.0) 104 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_pub"}]: dispatch 2020-09-08T15:26:34.685 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:34 smithi104 bash[9203]: audit 2020-09-08T15:26:33.316287+0000 mon.a (mon.0) 105 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_user"}]: dispatch 2020-09-08T15:26:34.685 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:34 smithi104 bash[9203]: audit 2020-09-08T15:26:33.316585+0000 mon.a (mon.0) 106 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/report_id"}]: dispatch 2020-09-08T15:26:34.687 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:34 smithi104 bash[9203]: audit 2020-09-08T15:26:33.316871+0000 mon.a (mon.0) 107 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/salt"}]: dispatch 2020-09-08T15:26:34.687 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:34 smithi104 bash[9203]: cluster 2020-09-08T15:26:33.319154+0000 mon.a (mon.0) 108 : cluster [INF] Manager daemon y is now available 2020-09-08T15:26:34.687 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:34 smithi104 bash[9203]: audit 2020-09-08T15:26:33.510804+0000 mon.a (mon.0) 109 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2020-09-08T15:26:35.117 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=3.83720064163208, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:26:35.118 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=3.837376356124878, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:26:35.118 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:mgr epoch 9 is available 2020-09-08T15:26:35.119 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Generating a dashboard self-signed certificate... 2020-09-08T15:26:35.119 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'dashboard', 'create-self-signed-cert'] 2020-09-08T15:26:35.119 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f dashboard create-self-signed-cert 2020-09-08T15:26:35.691 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:35 smithi104 bash[9203]: audit 2020-09-08T15:26:34.390710+0000 mon.a (mon.0) 110 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2020-09-08T15:26:35.692 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:35 smithi104 bash[9203]: audit 2020-09-08T15:26:34.397707+0000 mon.a (mon.0) 111 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:26:35.692 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:35 smithi104 bash[9203]: cluster 2020-09-08T15:26:34.402158+0000 mon.a (mon.0) 112 : cluster [DBG] mgrmap e11: y(active, since 1.15311s) 2020-09-08T15:26:35.693 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:35 smithi104 bash[9203]: audit 2020-09-08T15:26:34.404678+0000 mgr.y (mgr.14145) 1 : audit [DBG] from='client.14149 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2020-09-08T15:26:35.693 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:35 smithi104 bash[9203]: audit 2020-09-08T15:26:34.421414+0000 mgr.y (mgr.14145) 2 : audit [DBG] from='client.14149 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2020-09-08T15:26:35.693 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:35 smithi104 bash[9203]: audit 2020-09-08T15:26:35.100887+0000 mon.a (mon.0) 113 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:26:35.100055\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:26:35.100147\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2020-09-08T15:26:35.100257\", \"networks\": {}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:26:35.694 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:35 smithi104 bash[9203]: audit 2020-09-08T15:26:35.102987+0000 mon.a (mon.0) 114 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:26:35.100055\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:26:35.100147\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2020-09-08T15:26:35.100257\", \"networks\": {}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:26:35.694 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:35 smithi104 bash[9203]: audit 2020-09-08T15:26:35.103861+0000 mon.a (mon.0) 115 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:26:37.090 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout Self-signed certificate created 2020-09-08T15:26:37.090 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.9680440425872803, stop=False, exit=None, reads=[8] 2020-09-08T15:26:37.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:37 smithi104 bash[9203]: cluster 2020-09-08T15:26:36.109282+0000 mon.a (mon.0) 116 : cluster [DBG] mgrmap e12: y(active, since 2s) 2020-09-08T15:26:37.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:37 smithi104 bash[9203]: audit 2020-09-08T15:26:37.087394+0000 mon.a (mon.0) 117 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/dashboard/crt","val":"-----BEGIN CERTIFICATE-----\nMIICzzCCAbcCEEc15TsYf0qyqzedQ3CEni4wDQYJKoZIhvcNAQENBQAwJjELMAkG\nA1UECgwCSVQxFzAVBgNVBAMMDmNlcGgtZGFzaGJvYXJkMB4XDTIwMDkwODE1MjYz\nN1oXDTMwMDkwNjE1MjYzN1owJjELMAkGA1UECgwCSVQxFzAVBgNVBAMMDmNlcGgt\nZGFzaGJvYXJkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA9S6k+dZa\nT/p5NvEA1nixJ7P9c4wYfWgLRjsWpV45wZkAFHSRoE5WSqIB64cDClYxfF8UcQZG\nYTyilMWcdLB6n4mkkXg+K08cXxN8inZUMK1pK+i3I9YR5r18o8Xk/MJU8bXxVjTV\nJQBaXBbtBPMRTyV32Ol5zX25um8Bs9SoBIhvdbFbKE11243xsUBrBE6zAwwtFt/V\n6qSE1Nc/YXJW8HXIVvZgbaN1m/qk0eWH/E4htCq1VHM3qBBWDadJBy33ITnVnwIH\nuWCloeeZufmG2LmtKZqXx8eD5AqcxZlBSblIbEuutUl89hrofVKv0kWB4h3sFa3A\nbGf+klLCfjldeQIDAQABMA0GCSqGSIb3DQEBDQUAA4IBAQC4qCA+f3mlZKOlmiFv\nXhz8f9m3TsoLDu6tZQ5WYsVUVm1GO+y2PP9yMXdt4nh/x2Oh1Ed2UsH1JdsyWlBa\nVYWxQ+eu4KbGBtsHn1t6RSPhjtUZCNypctZQiClCpqtvJkx3SwRE+TPq7itg2+7Z\nHXHZyal60Qsw58bQsOhKRh07FGNTrX0BS2JKbNXB/f00ooWZvAFDACH0ihPaFsmm\nmGQwtXvC6RfntRX8RU9L4AEmUQAPtdNjfP6jelN5KPsj3vwUHrK5LwPhHm32ruoA\naeppRuOVvSx1VJiN54kb/xCzAl75sukGyn0wjQGDvBoQ6IwuSw9+J+NjvyhrUNqt\nhsIA\n-----END CERTIFICATE-----\n"}]: dispatch 2020-09-08T15:26:37.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:37 smithi104 bash[9203]: audit 2020-09-08T15:26:37.089634+0000 mon.a (mon.0) 118 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/dashboard/crt","val":"-----BEGIN CERTIFICATE-----\nMIICzzCCAbcCEEc15TsYf0qyqzedQ3CEni4wDQYJKoZIhvcNAQENBQAwJjELMAkG\nA1UECgwCSVQxFzAVBgNVBAMMDmNlcGgtZGFzaGJvYXJkMB4XDTIwMDkwODE1MjYz\nN1oXDTMwMDkwNjE1MjYzN1owJjELMAkGA1UECgwCSVQxFzAVBgNVBAMMDmNlcGgt\nZGFzaGJvYXJkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA9S6k+dZa\nT/p5NvEA1nixJ7P9c4wYfWgLRjsWpV45wZkAFHSRoE5WSqIB64cDClYxfF8UcQZG\nYTyilMWcdLB6n4mkkXg+K08cXxN8inZUMK1pK+i3I9YR5r18o8Xk/MJU8bXxVjTV\nJQBaXBbtBPMRTyV32Ol5zX25um8Bs9SoBIhvdbFbKE11243xsUBrBE6zAwwtFt/V\n6qSE1Nc/YXJW8HXIVvZgbaN1m/qk0eWH/E4htCq1VHM3qBBWDadJBy33ITnVnwIH\nuWCloeeZufmG2LmtKZqXx8eD5AqcxZlBSblIbEuutUl89hrofVKv0kWB4h3sFa3A\nbGf+klLCfjldeQIDAQABMA0GCSqGSIb3DQEBDQUAA4IBAQC4qCA+f3mlZKOlmiFv\nXhz8f9m3TsoLDu6tZQ5WYsVUVm1GO+y2PP9yMXdt4nh/x2Oh1Ed2UsH1JdsyWlBa\nVYWxQ+eu4KbGBtsHn1t6RSPhjtUZCNypctZQiClCpqtvJkx3SwRE+TPq7itg2+7Z\nHXHZyal60Qsw58bQsOhKRh07FGNTrX0BS2JKbNXB/f00ooWZvAFDACH0ihPaFsmm\nmGQwtXvC6RfntRX8RU9L4AEmUQAPtdNjfP6jelN5KPsj3vwUHrK5LwPhHm32ruoA\naeppRuOVvSx1VJiN54kb/xCzAl75sukGyn0wjQGDvBoQ6IwuSw9+J+NjvyhrUNqt\nhsIA\n-----END CERTIFICATE-----\n"}]': finished 2020-09-08T15:26:37.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:37 smithi104 bash[9203]: audit 2020-09-08T15:26:37.090148+0000 mon.a (mon.0) 119 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/dashboard/key","val":"-----BEGIN PRIVATE KEY-----\nMIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQD1LqT51lpP+nk2\n8QDWeLEns/1zjBh9aAtGOxalXjnBmQAUdJGgTlZKogHrhwMKVjF8XxRxBkZhPKKU\nxZx0sHqfiaSReD4rTxxfE3yKdlQwrWkr6Lcj1hHmvXyjxeT8wlTxtfFWNNUlAFpc\nFu0E8xFPJXfY6XnNfbm6bwGz1KgEiG91sVsoTXXbjfGxQGsETrMDDC0W39XqpITU\n1z9hclbwdchW9mBto3Wb+qTR5Yf8TiG0KrVUczeoEFYNp0kHLfchOdWfAge5YKWh\n55m5+YbYua0pmpfHx4PkCpzFmUFJuUhsS661SXz2Guh9Uq/SRYHiHewVrcBsZ/6S\nUsJ+OV15AgMBAAECggEBANe3k2VRiCJIEPAa+Q66H7v766oG/+EOcqT3BgS3M3+3\nGs/kg+AULG37vk64eh9VGLMBgJ201KVm0PCuMOql/y1wz/9BWQmH0EGChpfYZlhx\nqk/8r21zQkrOOuRpEACnBk+oMc8UKtgkSFwvu5gYOAl3wMtzGDyJOqjiVBAhsTZS\nD0OJUwtZYFaySbRLkrAjDh9itqRYfc+Tqg5YYQol1RS3Jk1+n9R3Xc0JsxTAFoSV\nRy9XjrIk9xtI9hxHnRlkiD2Burk45u2oYilDdIldvyfXJvXDSZqFFcas2mqATLMB\nRvqyNhqez8ZqoPlrXg3lgBJ0HipP/xSP81z1s29iAoECgYEA/ZY3JsqsKG5K0D+f\njFKs25t8LqIJ+oX/JPnNaF3pBbVCeWK4PcjZ7kxEPFCWJOvMjdp88OwrD7gf5Wzd\nPCI895424/WYe/D02iP80Gbvm7V98SZLQ2w3TTnyaFT320r9gRbIS2V4hCRRcjmk\nzLxwnF3RDNNniLkXcygBpzVeGKkCgYEA94P0Mn069XmlddT3ZorbS4PiYveraG4e\nSJiEDjBH21KUPuSFm/tBOQu5euEUPC7Dj+DgcluYNls/W+uIltCb6Fl1yAas2PBO\nbB+R4sqhflJFYybgJLc0l/wSVKYSqgvLmz0W4Ifxo0hKcG9NedcK1cqbcaToKhFl\npsX9BB9BEFECgYEA0FibxYZc/fboBsNPPcYreV79H39IfYmcQG/xAN+vdV4gWITw\nZOrhivUkVv3K7ljBvoaVgog3NPoNUr0n46EpwDRAWvjMqIAnXBaXP05jbOYnrHrO\n+X0uzodej7PH1Gd/cpEaTT/4TXFdbWDI8uGcEAzQlh4XAXE5QAbNnOiu/8kCgYAn\nitcqccNHkEWUy48yvKBIMiuqr/KcXmtMIcGHGsOAEHNKkOuffcZvvKXHMcovyzDG\n/aAd5PAuTumSX9CUd4NQ3WnpqNdYWRvOPkTqPIEMU0xuiLlqZDJszYKIL3fv3q6K\ngHn1KQK7mUL7L2zrOCWs/g9TWosUnW7l8992FyAn4QKBgQCTAOpYvVlV6JgKOjwW\nep8SN6Tjb4TN9Gss82HUOZvEE+f3cSTG+kG8PXA3GmXhcZdJRXZ4hmeu5hz4f1WT\n45TXl/Q6L2Wb6Ct16kMgqtl0/QpcByhpnjovlRFsmQQKQd2mLDn2lAI7XrvFwny2\nZHL1VU1WaiuxrvHwfBheM/biVg==\n-----END PRIVATE KEY-----\n"}]: dispatch 2020-09-08T15:26:37.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:37 smithi104 bash[9203]: audit 2020-09-08T15:26:37.093234+0000 mon.a (mon.0) 120 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/dashboard/key","val":"-----BEGIN PRIVATE KEY-----\nMIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQD1LqT51lpP+nk2\n8QDWeLEns/1zjBh9aAtGOxalXjnBmQAUdJGgTlZKogHrhwMKVjF8XxRxBkZhPKKU\nxZx0sHqfiaSReD4rTxxfE3yKdlQwrWkr6Lcj1hHmvXyjxeT8wlTxtfFWNNUlAFpc\nFu0E8xFPJXfY6XnNfbm6bwGz1KgEiG91sVsoTXXbjfGxQGsETrMDDC0W39XqpITU\n1z9hclbwdchW9mBto3Wb+qTR5Yf8TiG0KrVUczeoEFYNp0kHLfchOdWfAge5YKWh\n55m5+YbYua0pmpfHx4PkCpzFmUFJuUhsS661SXz2Guh9Uq/SRYHiHewVrcBsZ/6S\nUsJ+OV15AgMBAAECggEBANe3k2VRiCJIEPAa+Q66H7v766oG/+EOcqT3BgS3M3+3\nGs/kg+AULG37vk64eh9VGLMBgJ201KVm0PCuMOql/y1wz/9BWQmH0EGChpfYZlhx\nqk/8r21zQkrOOuRpEACnBk+oMc8UKtgkSFwvu5gYOAl3wMtzGDyJOqjiVBAhsTZS\nD0OJUwtZYFaySbRLkrAjDh9itqRYfc+Tqg5YYQol1RS3Jk1+n9R3Xc0JsxTAFoSV\nRy9XjrIk9xtI9hxHnRlkiD2Burk45u2oYilDdIldvyfXJvXDSZqFFcas2mqATLMB\nRvqyNhqez8ZqoPlrXg3lgBJ0HipP/xSP81z1s29iAoECgYEA/ZY3JsqsKG5K0D+f\njFKs25t8LqIJ+oX/JPnNaF3pBbVCeWK4PcjZ7kxEPFCWJOvMjdp88OwrD7gf5Wzd\nPCI895424/WYe/D02iP80Gbvm7V98SZLQ2w3TTnyaFT320r9gRbIS2V4hCRRcjmk\nzLxwnF3RDNNniLkXcygBpzVeGKkCgYEA94P0Mn069XmlddT3ZorbS4PiYveraG4e\nSJiEDjBH21KUPuSFm/tBOQu5euEUPC7Dj+DgcluYNls/W+uIltCb6Fl1yAas2PBO\nbB+R4sqhflJFYybgJLc0l/wSVKYSqgvLmz0W4Ifxo0hKcG9NedcK1cqbcaToKhFl\npsX9BB9BEFECgYEA0FibxYZc/fboBsNPPcYreV79H39IfYmcQG/xAN+vdV4gWITw\nZOrhivUkVv3K7ljBvoaVgog3NPoNUr0n46EpwDRAWvjMqIAnXBaXP05jbOYnrHrO\n+X0uzodej7PH1Gd/cpEaTT/4TXFdbWDI8uGcEAzQlh4XAXE5QAbNnOiu/8kCgYAn\nitcqccNHkEWUy48yvKBIMiuqr/KcXmtMIcGHGsOAEHNKkOuffcZvvKXHMcovyzDG\n/aAd5PAuTumSX9CUd4NQ3WnpqNdYWRvOPkTqPIEMU0xuiLlqZDJszYKIL3fv3q6K\ngHn1KQK7mUL7L2zrOCWs/g9TWosUnW7l8992FyAn4QKBgQCTAOpYvVlV6JgKOjwW\nep8SN6Tjb4TN9Gss82HUOZvEE+f3cSTG+kG8PXA3GmXhcZdJRXZ4hmeu5hz4f1WT\n45TXl/Q6L2Wb6Ct16kMgqtl0/QpcByhpnjovlRFsmQQKQd2mLDn2lAI7XrvFwny2\nZHL1VU1WaiuxrvHwfBheM/biVg==\n-----END PRIVATE KEY-----\n"}]': finished 2020-09-08T15:26:37.823 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.702346086502075, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:26:37.823 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.702620029449463, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:26:37.823 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Creating initial admin user... 2020-09-08T15:26:37.823 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'dashboard', 'ac-user-create', 'admin', 'g2w0m8j729', 'administrator', '--force-password', '--pwd-update-required'] 2020-09-08T15:26:37.824 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f dashboard ac-user-create admin g2w0m8j729 administrator --force-password --pwd-update-required 2020-09-08T15:26:38.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:38 smithi104 bash[9203]: audit 2020-09-08T15:26:37.021210+0000 mgr.y (mgr.14145) 3 : audit [DBG] from='client.14159 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:26:39.584 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout {"username": "admin", "password": "$2b$12$/NU7gUuC.am.fmFiIt.cIuiQGaVpqL5hKHHEXIaPC4WpEbEhE2f4i", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1599578799, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2020-09-08T15:26:39.585 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.7565937042236328, stop=False, exit=None, reads=[8] 2020-09-08T15:26:40.201 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.3746328353881836, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:26:40.202 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.3748152256011963, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:26:40.202 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Fetching dashboard port number... 2020-09-08T15:26:40.202 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:['/usr/bin/docker', 'run', '--rm', '--net=host', '--ipc=host', '-e', 'CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', '-e', 'NODE_NAME=smithi104', '-v', '/var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z', '-v', '/tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z', '-v', '/tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z', '--entrypoint', '/usr/bin/ceph', 'quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f', 'config', 'get', 'mgr', 'mgr/dashboard/ssl_server_port'] 2020-09-08T15:26:40.202 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Running command: /usr/bin/docker run --rm --net=host --ipc=host -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f -e NODE_NAME=smithi104 -v /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmpwnaeq3le:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp4q14swr4:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f config get mgr mgr/dashboard/ssl_server_port 2020-09-08T15:26:40.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:40 smithi104 bash[9203]: cluster 2020-09-08T15:26:39.307503+0000 mon.a (mon.0) 121 : cluster [DBG] mgrmap e13: y(active, since 6s) 2020-09-08T15:26:40.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:40 smithi104 bash[9203]: audit 2020-09-08T15:26:39.347556+0000 mgr.y (mgr.14145) 4 : audit [DBG] from='client.14161 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "password": "g2w0m8j729", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:26:40.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:40 smithi104 bash[9203]: audit 2020-09-08T15:26:39.583477+0000 mon.a (mon.0) 122 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/dashboard/accessdb_v2","val":"{\"users\": {\"admin\": {\"username\": \"admin\", \"password\": \"$2b$12$/NU7gUuC.am.fmFiIt.cIuiQGaVpqL5hKHHEXIaPC4WpEbEhE2f4i\", \"roles\": [\"administrator\"], \"name\": null, \"email\": null, \"lastUpdate\": 1599578799, \"enabled\": true, \"pwdExpirationDate\": null, \"pwdUpdateRequired\": true}}, \"roles\": {}, \"version\": 2}"}]: dispatch 2020-09-08T15:26:40.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:40 smithi104 bash[9203]: audit 2020-09-08T15:26:39.586874+0000 mon.a (mon.0) 123 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/dashboard/accessdb_v2","val":"{\"users\": {\"admin\": {\"username\": \"admin\", \"password\": \"$2b$12$/NU7gUuC.am.fmFiIt.cIuiQGaVpqL5hKHHEXIaPC4WpEbEhE2f4i\", \"roles\": [\"administrator\"], \"name\": null, \"email\": null, \"lastUpdate\": 1599578799, \"enabled\": true, \"pwdExpirationDate\": null, \"pwdUpdateRequired\": true}}, \"roles\": {}, \"version\": 2}"}]': finished 2020-09-08T15:26:41.617 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:stdout 8443 2020-09-08T15:26:41.617 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=1.4150950908660889, stop=False, exit=None, reads=[8] 2020-09-08T15:26:41.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:41 smithi104 bash[9203]: audit 2020-09-08T15:26:41.622186+0000 mon.a (mon.0) 124 : audit [DBG] from='client.? v1:172.21.15.104:0/1571444898' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2020-09-08T15:26:42.261 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.0587165355682373, stop=False, exit=0, reads=[8, 10] 2020-09-08T15:26:42.261 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:/usr/bin/ceph:profile rt=2.058897018432617, stop=True, exit=0, reads=[8, 10] 2020-09-08T15:26:42.262 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:firewalld does not appear to be present 2020-09-08T15:26:42.262 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Not possible to open ports <[8443]>. firewalld.service is not available 2020-09-08T15:26:42.276 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Ceph Dashboard is now available at: 2020-09-08T15:26:42.276 INFO:teuthology.orchestra.run.smithi104.stderr: 2020-09-08T15:26:42.276 INFO:teuthology.orchestra.run.smithi104.stderr: URL: https://smithi104.front.sepia.ceph.com:8443/ 2020-09-08T15:26:42.276 INFO:teuthology.orchestra.run.smithi104.stderr: User: admin 2020-09-08T15:26:42.277 INFO:teuthology.orchestra.run.smithi104.stderr: Password: g2w0m8j729 2020-09-08T15:26:42.277 INFO:teuthology.orchestra.run.smithi104.stderr: 2020-09-08T15:26:42.277 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:You can access the Ceph CLI with: 2020-09-08T15:26:42.278 INFO:teuthology.orchestra.run.smithi104.stderr: 2020-09-08T15:26:42.278 INFO:teuthology.orchestra.run.smithi104.stderr: sudo /home/ubuntu/cephtest/cephadm shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2020-09-08T15:26:42.278 INFO:teuthology.orchestra.run.smithi104.stderr: 2020-09-08T15:26:42.278 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Please consider enabling telemetry to help improve Ceph: 2020-09-08T15:26:42.279 INFO:teuthology.orchestra.run.smithi104.stderr: 2020-09-08T15:26:42.279 INFO:teuthology.orchestra.run.smithi104.stderr: ceph telemetry on 2020-09-08T15:26:42.279 INFO:teuthology.orchestra.run.smithi104.stderr: 2020-09-08T15:26:42.279 INFO:teuthology.orchestra.run.smithi104.stderr:For more information see: 2020-09-08T15:26:42.279 INFO:teuthology.orchestra.run.smithi104.stderr: 2020-09-08T15:26:42.280 INFO:teuthology.orchestra.run.smithi104.stderr: https://docs.ceph.com/docs/master/mgr/telemetry/ 2020-09-08T15:26:42.280 INFO:teuthology.orchestra.run.smithi104.stderr: 2020-09-08T15:26:42.280 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Bootstrap complete. 2020-09-08T15:26:42.281 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Releasing lock 140001992038104 on /run/cephadm/759b27b6-f1e7-11ea-a080-001a4aab830c.lock 2020-09-08T15:26:42.281 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG:cephadm:Lock 140001992038104 released on /run/cephadm/759b27b6-f1e7-11ea-a080-001a4aab830c.lock 2020-09-08T15:26:42.300 INFO:tasks.cephadm:Fetching config... 2020-09-08T15:26:42.303 INFO:teuthology.orchestra.run.smithi104:> set -ex 2020-09-08T15:26:42.303 INFO:teuthology.orchestra.run.smithi104:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2020-09-08T15:26:42.310 INFO:teuthology.orchestra.run.smithi104.stderr:+ dd if=/etc/ceph/ceph.conf of=/dev/stdout 2020-09-08T15:26:42.312 INFO:teuthology.orchestra.run.smithi104.stdout:# minimal ceph.conf for 759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:26:42.312 INFO:teuthology.orchestra.run.smithi104.stdout:[global] 2020-09-08T15:26:42.312 INFO:teuthology.orchestra.run.smithi104.stdout: fsid = 759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:26:42.312 INFO:teuthology.orchestra.run.smithi104.stdout: mon_host = 172.21.15.104:6789/0 2020-09-08T15:26:42.313 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records in 2020-09-08T15:26:42.313 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records out 2020-09-08T15:26:42.313 INFO:teuthology.orchestra.run.smithi104.stderr:148 bytes copied, 0.000155572 s, 951 kB/s 2020-09-08T15:26:42.315 INFO:tasks.cephadm:Fetching client.admin keyring... 2020-09-08T15:26:42.316 INFO:teuthology.orchestra.run.smithi104:> set -ex 2020-09-08T15:26:42.317 INFO:teuthology.orchestra.run.smithi104:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2020-09-08T15:26:42.365 INFO:teuthology.orchestra.run.smithi104.stderr:+ dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2020-09-08T15:26:42.365 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records in 2020-09-08T15:26:42.366 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records out 2020-09-08T15:26:42.366 INFO:teuthology.orchestra.run.smithi104.stderr:63 bytes copied, 0.000113483 s, 555 kB/s 2020-09-08T15:26:42.366 INFO:teuthology.orchestra.run.smithi104.stdout:[client.admin] 2020-09-08T15:26:42.366 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQBuoldfJAEbGhAASgQe9cXWjBWmNsNcTBGEGQ== 2020-09-08T15:26:42.367 INFO:tasks.cephadm:Fetching mon keyring... 2020-09-08T15:26:42.367 INFO:teuthology.orchestra.run.smithi104:> set -ex 2020-09-08T15:26:42.367 INFO:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/keyring of=/dev/stdout 2020-09-08T15:26:42.421 INFO:teuthology.orchestra.run.smithi104.stderr:+ sudo dd if=/var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/keyring of=/dev/stdout 2020-09-08T15:26:42.429 INFO:teuthology.orchestra.run.smithi104.stdout:[mon.] 2020-09-08T15:26:42.430 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQBsoldfwGjlMBAAXUPss7n5JOSeSfSGyEPOOw== 2020-09-08T15:26:42.430 INFO:teuthology.orchestra.run.smithi104.stdout: caps mon = "allow *" 2020-09-08T15:26:42.430 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records in 2020-09-08T15:26:42.430 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records out 2020-09-08T15:26:42.431 INFO:teuthology.orchestra.run.smithi104.stderr:77 bytes copied, 0.000164804 s, 467 kB/s 2020-09-08T15:26:42.431 INFO:tasks.cephadm:Fetching pub ssh key... 2020-09-08T15:26:42.432 INFO:teuthology.orchestra.run.smithi104:> set -ex 2020-09-08T15:26:42.432 INFO:teuthology.orchestra.run.smithi104:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2020-09-08T15:26:42.478 INFO:teuthology.orchestra.run.smithi104.stderr:+ dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2020-09-08T15:26:42.479 INFO:teuthology.orchestra.run.smithi104.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4nY0ThQvV7gOb6xfmTTvPZCzv9hUtGqIltG102ks/IfiJLNSyF421l+s3bx6jLZDpsbUQDE6OQwVRd33kOt3hCnA43UqB96aVBB1G0iOY6hgKpQgf8Z318ts1CS/PP0rjprpyz6M8TNnLGMrpEgGzWH3lRpN9iDa7MKeG54eiKqm88bMUHOt9dEPvF3m2R8+3LvZyQZogfarCIGBDW5t/sp7SVXpOUmfV18ahx8oWxmA8ERhVB3B3RqhLqA6YL9dCBj3/BfKXcxX06b7yKv8Ib7rpW7x4QwOACTyWUjnI8/REluJABhf+Cwy7NkBsBGOTxFyv6saNEjv8bfez9roOiWFZZArr8ekPfs1XlTsjvgJgBkWQLF5J/TUu73mkrWaMZiiJ5taOEXmP5ev18BItatAyzg4ljKc5VE2OewYHUCeqtg27tmxvchLYQHcyXPCHC6E3RGlBVWqMPPxqFFF6+K10gYuaoFZ3GVE2Orhd4znc0/939ymnREi5RVkDIwU= ceph-759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:26:42.480 INFO:teuthology.orchestra.run.smithi104.stderr:1+1 records in 2020-09-08T15:26:42.480 INFO:teuthology.orchestra.run.smithi104.stderr:1+1 records out 2020-09-08T15:26:42.480 INFO:teuthology.orchestra.run.smithi104.stderr:595 bytes copied, 0.000229103 s, 2.6 MB/s 2020-09-08T15:26:42.480 INFO:tasks.cephadm:Installing pub ssh key for root users... 2020-09-08T15:26:42.481 INFO:teuthology.orchestra.run.smithi055:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4nY0ThQvV7gOb6xfmTTvPZCzv9hUtGqIltG102ks/IfiJLNSyF421l+s3bx6jLZDpsbUQDE6OQwVRd33kOt3hCnA43UqB96aVBB1G0iOY6hgKpQgf8Z318ts1CS/PP0rjprpyz6M8TNnLGMrpEgGzWH3lRpN9iDa7MKeG54eiKqm88bMUHOt9dEPvF3m2R8+3LvZyQZogfarCIGBDW5t/sp7SVXpOUmfV18ahx8oWxmA8ERhVB3B3RqhLqA6YL9dCBj3/BfKXcxX06b7yKv8Ib7rpW7x4QwOACTyWUjnI8/REluJABhf+Cwy7NkBsBGOTxFyv6saNEjv8bfez9roOiWFZZArr8ekPfs1XlTsjvgJgBkWQLF5J/TUu73mkrWaMZiiJ5taOEXmP5ev18BItatAyzg4ljKc5VE2OewYHUCeqtg27tmxvchLYQHcyXPCHC6E3RGlBVWqMPPxqFFF6+K10gYuaoFZ3GVE2Orhd4znc0/939ymnREi5RVkDIwU= ceph-759b27b6-f1e7-11ea-a080-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2020-09-08T15:26:42.502 INFO:teuthology.orchestra.run.smithi055.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4nY0ThQvV7gOb6xfmTTvPZCzv9hUtGqIltG102ks/IfiJLNSyF421l+s3bx6jLZDpsbUQDE6OQwVRd33kOt3hCnA43UqB96aVBB1G0iOY6hgKpQgf8Z318ts1CS/PP0rjprpyz6M8TNnLGMrpEgGzWH3lRpN9iDa7MKeG54eiKqm88bMUHOt9dEPvF3m2R8+3LvZyQZogfarCIGBDW5t/sp7SVXpOUmfV18ahx8oWxmA8ERhVB3B3RqhLqA6YL9dCBj3/BfKXcxX06b7yKv8Ib7rpW7x4QwOACTyWUjnI8/REluJABhf+Cwy7NkBsBGOTxFyv6saNEjv8bfez9roOiWFZZArr8ekPfs1XlTsjvgJgBkWQLF5J/TUu73mkrWaMZiiJ5taOEXmP5ev18BItatAyzg4ljKc5VE2OewYHUCeqtg27tmxvchLYQHcyXPCHC6E3RGlBVWqMPPxqFFF6+K10gYuaoFZ3GVE2Orhd4znc0/939ymnREi5RVkDIwU= ceph-759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:26:42.511 INFO:teuthology.orchestra.run.smithi104:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4nY0ThQvV7gOb6xfmTTvPZCzv9hUtGqIltG102ks/IfiJLNSyF421l+s3bx6jLZDpsbUQDE6OQwVRd33kOt3hCnA43UqB96aVBB1G0iOY6hgKpQgf8Z318ts1CS/PP0rjprpyz6M8TNnLGMrpEgGzWH3lRpN9iDa7MKeG54eiKqm88bMUHOt9dEPvF3m2R8+3LvZyQZogfarCIGBDW5t/sp7SVXpOUmfV18ahx8oWxmA8ERhVB3B3RqhLqA6YL9dCBj3/BfKXcxX06b7yKv8Ib7rpW7x4QwOACTyWUjnI8/REluJABhf+Cwy7NkBsBGOTxFyv6saNEjv8bfez9roOiWFZZArr8ekPfs1XlTsjvgJgBkWQLF5J/TUu73mkrWaMZiiJ5taOEXmP5ev18BItatAyzg4ljKc5VE2OewYHUCeqtg27tmxvchLYQHcyXPCHC6E3RGlBVWqMPPxqFFF6+K10gYuaoFZ3GVE2Orhd4znc0/939ymnREi5RVkDIwU= ceph-759b27b6-f1e7-11ea-a080-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2020-09-08T15:26:42.546 INFO:teuthology.orchestra.run.smithi104.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4nY0ThQvV7gOb6xfmTTvPZCzv9hUtGqIltG102ks/IfiJLNSyF421l+s3bx6jLZDpsbUQDE6OQwVRd33kOt3hCnA43UqB96aVBB1G0iOY6hgKpQgf8Z318ts1CS/PP0rjprpyz6M8TNnLGMrpEgGzWH3lRpN9iDa7MKeG54eiKqm88bMUHOt9dEPvF3m2R8+3LvZyQZogfarCIGBDW5t/sp7SVXpOUmfV18ahx8oWxmA8ERhVB3B3RqhLqA6YL9dCBj3/BfKXcxX06b7yKv8Ib7rpW7x4QwOACTyWUjnI8/REluJABhf+Cwy7NkBsBGOTxFyv6saNEjv8bfez9roOiWFZZArr8ekPfs1XlTsjvgJgBkWQLF5J/TUu73mkrWaMZiiJ5taOEXmP5ev18BItatAyzg4ljKc5VE2OewYHUCeqtg27tmxvchLYQHcyXPCHC6E3RGlBVWqMPPxqFFF6+K10gYuaoFZ3GVE2Orhd4znc0/939ymnREi5RVkDIwU= ceph-759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:26:42.556 INFO:teuthology.orchestra.run.smithi166:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4nY0ThQvV7gOb6xfmTTvPZCzv9hUtGqIltG102ks/IfiJLNSyF421l+s3bx6jLZDpsbUQDE6OQwVRd33kOt3hCnA43UqB96aVBB1G0iOY6hgKpQgf8Z318ts1CS/PP0rjprpyz6M8TNnLGMrpEgGzWH3lRpN9iDa7MKeG54eiKqm88bMUHOt9dEPvF3m2R8+3LvZyQZogfarCIGBDW5t/sp7SVXpOUmfV18ahx8oWxmA8ERhVB3B3RqhLqA6YL9dCBj3/BfKXcxX06b7yKv8Ib7rpW7x4QwOACTyWUjnI8/REluJABhf+Cwy7NkBsBGOTxFyv6saNEjv8bfez9roOiWFZZArr8ekPfs1XlTsjvgJgBkWQLF5J/TUu73mkrWaMZiiJ5taOEXmP5ev18BItatAyzg4ljKc5VE2OewYHUCeqtg27tmxvchLYQHcyXPCHC6E3RGlBVWqMPPxqFFF6+K10gYuaoFZ3GVE2Orhd4znc0/939ymnREi5RVkDIwU= ceph-759b27b6-f1e7-11ea-a080-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2020-09-08T15:26:42.582 INFO:teuthology.orchestra.run.smithi166.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4nY0ThQvV7gOb6xfmTTvPZCzv9hUtGqIltG102ks/IfiJLNSyF421l+s3bx6jLZDpsbUQDE6OQwVRd33kOt3hCnA43UqB96aVBB1G0iOY6hgKpQgf8Z318ts1CS/PP0rjprpyz6M8TNnLGMrpEgGzWH3lRpN9iDa7MKeG54eiKqm88bMUHOt9dEPvF3m2R8+3LvZyQZogfarCIGBDW5t/sp7SVXpOUmfV18ahx8oWxmA8ERhVB3B3RqhLqA6YL9dCBj3/BfKXcxX06b7yKv8Ib7rpW7x4QwOACTyWUjnI8/REluJABhf+Cwy7NkBsBGOTxFyv6saNEjv8bfez9roOiWFZZArr8ekPfs1XlTsjvgJgBkWQLF5J/TUu73mkrWaMZiiJ5taOEXmP5ev18BItatAyzg4ljKc5VE2OewYHUCeqtg27tmxvchLYQHcyXPCHC6E3RGlBVWqMPPxqFFF6+K10gYuaoFZ3GVE2Orhd4znc0/939ymnREi5RVkDIwU= ceph-759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:26:42.593 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2020-09-08T15:26:46.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:46 smithi104 bash[9203]: audit 2020-09-08T15:26:46.063755+0000 mon.a (mon.0) 125 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:26:47.317 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi055 2020-09-08T15:26:47.318 INFO:teuthology.orchestra.run.smithi055:> set -ex 2020-09-08T15:26:47.318 INFO:teuthology.orchestra.run.smithi055:> dd of=/etc/ceph/ceph.conf 2020-09-08T15:26:47.324 INFO:teuthology.orchestra.run.smithi055.stderr:+ dd of=/etc/ceph/ceph.conf 2020-09-08T15:26:47.325 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records in 2020-09-08T15:26:47.325 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records out 2020-09-08T15:26:47.326 INFO:teuthology.orchestra.run.smithi055.stderr:148 bytes copied, 7.5586e-05 s, 2.0 MB/s 2020-09-08T15:26:47.326 INFO:teuthology.orchestra.run.smithi055:> set -ex 2020-09-08T15:26:47.326 INFO:teuthology.orchestra.run.smithi055:> dd of=/etc/ceph/ceph.client.admin.keyring 2020-09-08T15:26:47.363 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:47 smithi104 bash[9203]: audit 2020-09-08T15:26:46.357484+0000 mon.a (mon.0) 126 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:26:35.100055\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:26:35.100147\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2020-09-08T15:26:35.100257\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:26:47.364 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:47 smithi104 bash[9203]: audit 2020-09-08T15:26:46.360490+0000 mon.a (mon.0) 127 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:26:35.100055\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:26:35.100147\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2020-09-08T15:26:35.100257\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:26:47.364 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:47 smithi104 bash[9203]: audit 2020-09-08T15:26:46.361603+0000 mon.a (mon.0) 128 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:26:47.364 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:47 smithi104 bash[9203]: audit 2020-09-08T15:26:46.363208+0000 mon.a (mon.0) 129 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:26:47.364 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:47 smithi104 bash[9203]: cephadm 2020-09-08T15:26:46.363611+0000 mgr.y (mgr.14145) 5 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2020-09-08T15:26:47.365 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:47 smithi104 bash[9203]: audit 2020-09-08T15:26:46.364022+0000 mon.a (mon.0) 130 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2020-09-08T15:26:47.365 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:47 smithi104 bash[9203]: audit 2020-09-08T15:26:46.365250+0000 mon.a (mon.0) 131 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:26:47.365 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:47 smithi104 bash[9203]: cephadm 2020-09-08T15:26:46.366026+0000 mgr.y (mgr.14145) 6 : cephadm [INF] Reconfiguring daemon mon.a on smithi104 2020-09-08T15:26:47.365 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:47 smithi104 bash[9203]: audit 2020-09-08T15:26:46.366479+0000 mon.a (mon.0) 132 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon.a", "key": "container_image"}]: dispatch 2020-09-08T15:26:47.365 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:47 smithi104 bash[9203]: audit 2020-09-08T15:26:46.504740+0000 mon.a (mon.0) 133 : audit [INF] from='client.? v1:172.21.15.104:0/1063710935' entity='client.admin' cmd=[{"prefix": "config set", "who": "mgr", "name": "mgr/cephadm/allow_ptrace", "value": "true"}]: dispatch 2020-09-08T15:26:47.366 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:47 smithi104 bash[9203]: audit 2020-09-08T15:26:46.507866+0000 mon.a (mon.0) 134 : audit [INF] from='client.? v1:172.21.15.104:0/1063710935' entity='client.admin' cmd='[{"prefix": "config set", "who": "mgr", "name": "mgr/cephadm/allow_ptrace", "value": "true"}]': finished 2020-09-08T15:26:47.370 INFO:teuthology.orchestra.run.smithi055.stderr:+ dd of=/etc/ceph/ceph.client.admin.keyring 2020-09-08T15:26:47.372 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records in 2020-09-08T15:26:47.372 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records out 2020-09-08T15:26:47.372 INFO:teuthology.orchestra.run.smithi055.stderr:63 bytes copied, 7.8477e-05 s, 803 kB/s 2020-09-08T15:26:47.373 INFO:tasks.cephadm:Adding host smithi055 to orchestrator... 2020-09-08T15:26:47.373 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph orch host add smithi055 2020-09-08T15:26:51.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:50 smithi104 bash[9203]: audit 2020-09-08T15:26:50.945319+0000 mon.a (mon.0) 135 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:26:35.100147\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:26:46.363695\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:26:51.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:50 smithi104 bash[9203]: audit 2020-09-08T15:26:50.949820+0000 mon.a (mon.0) 136 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:26:35.100147\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:26:46.363695\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:26:51.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:50 smithi104 bash[9203]: audit 2020-09-08T15:26:50.950741+0000 mon.a (mon.0) 137 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mgr.y"}]: dispatch 2020-09-08T15:26:51.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:50 smithi104 bash[9203]: audit 2020-09-08T15:26:50.951933+0000 mon.a (mon.0) 138 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:26:51.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:50 smithi104 bash[9203]: audit 2020-09-08T15:26:50.954150+0000 mon.a (mon.0) 139 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.y", "key": "container_image"}]: dispatch 2020-09-08T15:26:52.319 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:51 smithi104 bash[9203]: cephadm 2020-09-08T15:26:50.950373+0000 mgr.y (mgr.14145) 7 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2020-09-08T15:26:52.320 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:51 smithi104 bash[9203]: cephadm 2020-09-08T15:26:50.953155+0000 mgr.y (mgr.14145) 8 : cephadm [INF] Reconfiguring daemon mgr.y on smithi104 2020-09-08T15:26:53.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:53 smithi104 bash[9203]: audit 2020-09-08T15:26:52.828211+0000 mon.a (mon.0) 140 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:26:46.363695\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:26:50.950427\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:26:53.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:53 smithi104 bash[9203]: audit 2020-09-08T15:26:52.832664+0000 mon.a (mon.0) 141 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:26:46.363695\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:26:50.950427\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:26:53.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:53 smithi104 bash[9203]: audit 2020-09-08T15:26:52.835821+0000 mon.a (mon.0) 142 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:26:54.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:54 smithi104 bash[9203]: cluster 2020-09-08T15:26:53.313940+0000 mgr.y (mgr.14145) 9 : cluster [DBG] pgmap v3: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:26:54.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:54 smithi104 bash[9203]: audit 2020-09-08T15:26:53.465286+0000 mon.a (mon.0) 143 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:26:53.463527\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:26:53.463730\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:26:46.363695\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:26:50.950427\"}}, \"last_daemon_update\": \"2020-09-08T15:26:53.463968\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:26:54.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:54 smithi104 bash[9203]: audit 2020-09-08T15:26:53.471023+0000 mon.a (mon.0) 144 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:26:53.463527\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:26:53.463730\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:26:46.363695\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:26:50.950427\"}}, \"last_daemon_update\": \"2020-09-08T15:26:53.463968\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:26:54.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:54 smithi104 bash[9203]: audit 2020-09-08T15:26:53.472470+0000 mon.a (mon.0) 145 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:26:54.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:54 smithi104 bash[9203]: audit 2020-09-08T15:26:53.477342+0000 mon.a (mon.0) 146 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:26:56.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:56 smithi104 bash[9203]: cluster 2020-09-08T15:26:55.314455+0000 mgr.y (mgr.14145) 10 : cluster [DBG] pgmap v4: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:26:58.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:26:58 smithi104 bash[9203]: cluster 2020-09-08T15:26:57.314966+0000 mgr.y (mgr.14145) 11 : cluster [DBG] pgmap v5: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:00.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:00 smithi104 bash[9203]: cluster 2020-09-08T15:26:59.315496+0000 mgr.y (mgr.14145) 12 : cluster [DBG] pgmap v6: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:02.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:02 smithi104 bash[9203]: cluster 2020-09-08T15:27:01.316056+0000 mgr.y (mgr.14145) 13 : cluster [DBG] pgmap v7: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:04.689 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:04 smithi104 bash[9203]: cluster 2020-09-08T15:27:03.316587+0000 mgr.y (mgr.14145) 14 : cluster [DBG] pgmap v8: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:06.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:06 smithi104 bash[9203]: cluster 2020-09-08T15:27:05.317036+0000 mgr.y (mgr.14145) 15 : cluster [DBG] pgmap v9: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:08.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:08 smithi104 bash[9203]: cluster 2020-09-08T15:27:07.317519+0000 mgr.y (mgr.14145) 16 : cluster [DBG] pgmap v10: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:10.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:10 smithi104 bash[9203]: cluster 2020-09-08T15:27:09.318017+0000 mgr.y (mgr.14145) 17 : cluster [DBG] pgmap v11: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:12.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:12 smithi104 bash[9203]: cluster 2020-09-08T15:27:11.318460+0000 mgr.y (mgr.14145) 18 : cluster [DBG] pgmap v12: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:14.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:14 smithi104 bash[9203]: cluster 2020-09-08T15:27:13.318907+0000 mgr.y (mgr.14145) 19 : cluster [DBG] pgmap v13: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:16.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:16 smithi104 bash[9203]: cluster 2020-09-08T15:27:15.319334+0000 mgr.y (mgr.14145) 20 : cluster [DBG] pgmap v14: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:18.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:18 smithi104 bash[9203]: cluster 2020-09-08T15:27:17.319752+0000 mgr.y (mgr.14145) 21 : cluster [DBG] pgmap v15: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:20.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:20 smithi104 bash[9203]: cluster 2020-09-08T15:27:19.320183+0000 mgr.y (mgr.14145) 22 : cluster [DBG] pgmap v16: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:20.681 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:27:20 smithi104 bash[9475]: Warning: Permanently added 'smithi055,172.21.15.55' (ECDSA) to the list of known hosts. 2020-09-08T15:27:21.132 INFO:teuthology.orchestra.run.smithi055.stdout:Added host 'smithi055' 2020-09-08T15:27:21.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:21 smithi104 bash[9203]: audit 2020-09-08T15:27:20.344877+0000 mgr.y (mgr.14145) 23 : audit [DBG] from='client.14167 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi055", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:27:21.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:21 smithi104 bash[9203]: audit 2020-09-08T15:27:21.128946+0000 mon.a (mon.0) 147 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/inventory","val":"{\"smithi104\": {\"hostname\": \"smithi104\", \"addr\": \"smithi104\", \"labels\": [], \"status\": \"\"}, \"smithi055\": {\"hostname\": \"smithi055\", \"addr\": \"smithi055\", \"labels\": [], \"status\": \"\"}}"}]: dispatch 2020-09-08T15:27:21.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:21 smithi104 bash[9203]: audit 2020-09-08T15:27:21.132913+0000 mon.a (mon.0) 148 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/inventory","val":"{\"smithi104\": {\"hostname\": \"smithi104\", \"addr\": \"smithi104\", \"labels\": [], \"status\": \"\"}, \"smithi055\": {\"hostname\": \"smithi055\", \"addr\": \"smithi055\", \"labels\": [], \"status\": \"\"}}"}]': finished 2020-09-08T15:27:21.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:21 smithi104 bash[9203]: audit 2020-09-08T15:27:21.137468+0000 mon.a (mon.0) 149 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:27:21.901 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph orch host ls --format=json 2020-09-08T15:27:22.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:22 smithi104 bash[9203]: cephadm 2020-09-08T15:27:21.133629+0000 mgr.y (mgr.14145) 24 : cephadm [INF] Added host smithi055 2020-09-08T15:27:22.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:22 smithi104 bash[9203]: cluster 2020-09-08T15:27:21.320658+0000 mgr.y (mgr.14145) 25 : cluster [DBG] pgmap v17: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:22.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:22 smithi104 bash[9203]: audit 2020-09-08T15:27:21.494557+0000 mon.a (mon.0) 150 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {}, \"devices\": [], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"networks\": {}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:27:22.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:22 smithi104 bash[9203]: audit 2020-09-08T15:27:21.498770+0000 mon.a (mon.0) 151 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {}, \"devices\": [], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"networks\": {}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:27:22.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:22 smithi104 bash[9203]: audit 2020-09-08T15:27:21.500136+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:27:22.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:22 smithi104 bash[9203]: audit 2020-09-08T15:27:21.769289+0000 mon.a (mon.0) 153 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:27:21.767382\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:27:21.767610\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:26:46.363695\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:26:50.950427\"}}, \"last_daemon_update\": \"2020-09-08T15:27:21.767805\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:27:22.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:22 smithi104 bash[9203]: audit 2020-09-08T15:27:21.770171+0000 mon.a (mon.0) 154 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {}, \"devices\": [], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2020-09-08T15:27:21.768986\", \"networks\": {}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:27:22.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:22 smithi104 bash[9203]: audit 2020-09-08T15:27:21.773447+0000 mon.a (mon.0) 155 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:27:21.767382\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:27:21.767610\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:26:46.363695\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:26:50.950427\"}}, \"last_daemon_update\": \"2020-09-08T15:27:21.767805\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:27:22.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:22 smithi104 bash[9203]: audit 2020-09-08T15:27:21.775430+0000 mon.a (mon.0) 156 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {}, \"devices\": [], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2020-09-08T15:27:21.768986\", \"networks\": {}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:27:22.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:22 smithi104 bash[9203]: audit 2020-09-08T15:27:21.776273+0000 mon.a (mon.0) 157 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:27:24.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:24 smithi104 bash[9203]: cluster 2020-09-08T15:27:23.321130+0000 mgr.y (mgr.14145) 26 : cluster [DBG] pgmap v18: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:26.052 INFO:teuthology.orchestra.run.smithi055.stdout: 2020-09-08T15:27:26.053 INFO:teuthology.orchestra.run.smithi055.stdout:[{"addr": "smithi104", "hostname": "smithi104", "labels": [], "status": ""}, {"addr": "smithi055", "hostname": "smithi055", "labels": [], "status": ""}] 2020-09-08T15:27:26.819 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi166 2020-09-08T15:27:26.820 INFO:teuthology.orchestra.run.smithi166:> set -ex 2020-09-08T15:27:26.820 INFO:teuthology.orchestra.run.smithi166:> dd of=/etc/ceph/ceph.conf 2020-09-08T15:27:26.839 INFO:teuthology.orchestra.run.smithi166.stderr:+ dd of=/etc/ceph/ceph.conf 2020-09-08T15:27:26.841 INFO:teuthology.orchestra.run.smithi166.stderr:0+1 records in 2020-09-08T15:27:26.841 INFO:teuthology.orchestra.run.smithi166.stderr:0+1 records out 2020-09-08T15:27:26.842 INFO:teuthology.orchestra.run.smithi166.stderr:148 bytes copied, 7.6716e-05 s, 1.9 MB/s 2020-09-08T15:27:26.842 INFO:teuthology.orchestra.run.smithi166:> set -ex 2020-09-08T15:27:26.843 INFO:teuthology.orchestra.run.smithi166:> dd of=/etc/ceph/ceph.client.admin.keyring 2020-09-08T15:27:26.902 INFO:teuthology.orchestra.run.smithi166.stderr:+ dd of=/etc/ceph/ceph.client.admin.keyring 2020-09-08T15:27:26.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:26 smithi104 bash[9203]: cluster 2020-09-08T15:27:25.321638+0000 mgr.y (mgr.14145) 27 : cluster [DBG] pgmap v19: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:26.946 INFO:teuthology.orchestra.run.smithi166.stderr:0+1 records in 2020-09-08T15:27:26.946 INFO:teuthology.orchestra.run.smithi166.stderr:0+1 records out 2020-09-08T15:27:26.947 INFO:teuthology.orchestra.run.smithi166.stderr:63 bytes copied, 0.0531443 s, 1.2 kB/s 2020-09-08T15:27:26.947 INFO:tasks.cephadm:Adding host smithi166 to orchestrator... 2020-09-08T15:27:26.948 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph orch host add smithi166 2020-09-08T15:27:27.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:27 smithi104 bash[9203]: audit 2020-09-08T15:27:26.054808+0000 mgr.y (mgr.14145) 28 : audit [DBG] from='client.14169 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:27:28.430 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:27:28 smithi104 bash[9475]: debug 2020-09-08T15:27:28.145+0000 7f6ee9d53700 -1 ceph_set_health_checks check CEPHADM_REFRESH_FAILED detail item 0 not a [unicode] string 2020-09-08T15:27:28.431 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:27:28 smithi104 bash[9475]: debug 2020-09-08T15:27:28.145+0000 7f6ee9d53700 -1 ceph_set_health_checks check CEPHADM_REFRESH_FAILED detail item 0 not a [unicode] string 2020-09-08T15:27:28.943 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:28 smithi104 bash[9203]: cluster 2020-09-08T15:27:27.322081+0000 mgr.y (mgr.14145) 29 : cluster [DBG] pgmap v20: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:28.943 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:28 smithi104 bash[9203]: audit 2020-09-08T15:27:27.848204+0000 mon.a (mon.0) 158 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:27:28.943 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:28 smithi104 bash[9203]: audit 2020-09-08T15:27:28.139336+0000 mon.a (mon.0) 159 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2020-09-08T15:27:21.768986\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:27:28.944 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:28 smithi104 bash[9203]: audit 2020-09-08T15:27:28.142629+0000 mon.a (mon.0) 160 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2020-09-08T15:27:21.768986\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:27:28.944 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:28 smithi104 bash[9203]: audit 2020-09-08T15:27:28.144038+0000 mon.a (mon.0) 161 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2020-09-08T15:27:21.768986\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:27:28.944 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:28 smithi104 bash[9203]: audit 2020-09-08T15:27:28.148145+0000 mon.a (mon.0) 162 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2020-09-08T15:27:21.768986\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:27:28.944 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:28 smithi104 bash[9203]: audit 2020-09-08T15:27:28.150368+0000 mon.a (mon.0) 163 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:27:28.945 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:28 smithi104 bash[9203]: audit 2020-09-08T15:27:28.153749+0000 mon.a (mon.0) 164 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:27:29.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:29 smithi104 bash[9203]: cluster 2020-09-08T15:27:28.149372+0000 mgr.y (mgr.14145) 30 : cluster [DBG] pgmap v21: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:29.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:29 smithi104 bash[9203]: cluster 2020-09-08T15:27:29.145221+0000 mon.a (mon.0) 165 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2020-09-08T15:27:31.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:31 smithi104 bash[9203]: cluster 2020-09-08T15:27:30.149886+0000 mgr.y (mgr.14145) 31 : cluster [DBG] pgmap v22: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:33.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:33 smithi104 bash[9203]: cluster 2020-09-08T15:27:32.150367+0000 mgr.y (mgr.14145) 32 : cluster [DBG] pgmap v23: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:35.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:35 smithi104 bash[9203]: cluster 2020-09-08T15:27:34.150853+0000 mgr.y (mgr.14145) 33 : cluster [DBG] pgmap v24: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:35.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:35 smithi104 bash[9203]: cluster 2020-09-08T15:27:34.452208+0000 mon.a (mon.0) 166 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2020-09-08T15:27:37.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:37 smithi104 bash[9203]: cluster 2020-09-08T15:27:36.151381+0000 mgr.y (mgr.14145) 34 : cluster [DBG] pgmap v25: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:39.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:39 smithi104 bash[9203]: cluster 2020-09-08T15:27:38.151853+0000 mgr.y (mgr.14145) 35 : cluster [DBG] pgmap v26: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:41.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:41 smithi104 bash[9203]: cluster 2020-09-08T15:27:40.152328+0000 mgr.y (mgr.14145) 36 : cluster [DBG] pgmap v27: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:43.686 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:43 smithi104 bash[9203]: cluster 2020-09-08T15:27:42.152783+0000 mgr.y (mgr.14145) 37 : cluster [DBG] pgmap v28: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:45.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:45 smithi104 bash[9203]: cluster 2020-09-08T15:27:44.153290+0000 mgr.y (mgr.14145) 38 : cluster [DBG] pgmap v29: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:47.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:47 smithi104 bash[9203]: cluster 2020-09-08T15:27:46.153772+0000 mgr.y (mgr.14145) 39 : cluster [DBG] pgmap v30: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:49.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:49 smithi104 bash[9203]: cluster 2020-09-08T15:27:48.154247+0000 mgr.y (mgr.14145) 40 : cluster [DBG] pgmap v31: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:51.682 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:51 smithi104 bash[9203]: cluster 2020-09-08T15:27:50.154738+0000 mgr.y (mgr.14145) 41 : cluster [DBG] pgmap v32: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:53.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:53 smithi104 bash[9203]: cluster 2020-09-08T15:27:52.155241+0000 mgr.y (mgr.14145) 42 : cluster [DBG] pgmap v33: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:55.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:55 smithi104 bash[9203]: cluster 2020-09-08T15:27:54.155737+0000 mgr.y (mgr.14145) 43 : cluster [DBG] pgmap v34: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:57.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:57 smithi104 bash[9203]: cluster 2020-09-08T15:27:56.156209+0000 mgr.y (mgr.14145) 44 : cluster [DBG] pgmap v35: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:27:59.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:27:59 smithi104 bash[9203]: cluster 2020-09-08T15:27:58.156734+0000 mgr.y (mgr.14145) 45 : cluster [DBG] pgmap v36: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:01.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:01 smithi104 bash[9203]: cluster 2020-09-08T15:28:00.157217+0000 mgr.y (mgr.14145) 46 : cluster [DBG] pgmap v37: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:03.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:03 smithi104 bash[9203]: cluster 2020-09-08T15:28:02.157719+0000 mgr.y (mgr.14145) 47 : cluster [DBG] pgmap v38: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:05.934 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:05 smithi104 bash[9203]: cluster 2020-09-08T15:28:04.158178+0000 mgr.y (mgr.14145) 48 : cluster [DBG] pgmap v39: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:07.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:07 smithi104 bash[9203]: cluster 2020-09-08T15:28:06.158718+0000 mgr.y (mgr.14145) 49 : cluster [DBG] pgmap v40: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:09.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:09 smithi104 bash[9203]: cluster 2020-09-08T15:28:08.159227+0000 mgr.y (mgr.14145) 50 : cluster [DBG] pgmap v41: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:11.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:11 smithi104 bash[9203]: audit 2020-09-08T15:28:10.622138+0000 mon.a (mon.0) 167 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:28:11.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:11 smithi104 bash[9203]: audit 2020-09-08T15:28:10.627885+0000 mon.a (mon.0) 168 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:28:12.934 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:12 smithi104 bash[9203]: cluster 2020-09-08T15:28:10.621037+0000 mgr.y (mgr.14145) 52 : cluster [DBG] pgmap v43: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:12.935 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:12 smithi104 bash[9203]: cluster 2020-09-08T15:28:11.448122+0000 mon.a (mon.0) 169 : cluster [INF] Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2020-09-08T15:28:14.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:14 smithi104 bash[9203]: cluster 2020-09-08T15:28:12.621489+0000 mgr.y (mgr.14145) 53 : cluster [DBG] pgmap v44: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:16.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:16 smithi104 bash[9203]: cluster 2020-09-08T15:28:14.621967+0000 mgr.y (mgr.14145) 54 : cluster [DBG] pgmap v45: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:18.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:18 smithi104 bash[9203]: cluster 2020-09-08T15:28:16.622392+0000 mgr.y (mgr.14145) 55 : cluster [DBG] pgmap v46: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:18.679 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:28:18 smithi104 bash[9475]: Warning: Permanently added 'smithi166,172.21.15.166' (ECDSA) to the list of known hosts. 2020-09-08T15:28:19.039 INFO:teuthology.orchestra.run.smithi166.stdout:Added host 'smithi166' 2020-09-08T15:28:19.709 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph orch host ls --format=json 2020-09-08T15:28:19.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:19 smithi104 bash[9203]: audit 2020-09-08T15:28:18.269776+0000 mgr.y (mgr.14145) 56 : audit [DBG] from='client.14171 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi166", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:28:19.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:19 smithi104 bash[9203]: audit 2020-09-08T15:28:19.029149+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/inventory","val":"{\"smithi104\": {\"hostname\": \"smithi104\", \"addr\": \"smithi104\", \"labels\": [], \"status\": \"\"}, \"smithi055\": {\"hostname\": \"smithi055\", \"addr\": \"smithi055\", \"labels\": [], \"status\": \"\"}, \"smithi166\": {\"hostname\": \"smithi166\", \"addr\": \"smithi166\", \"labels\": [], \"status\": \"\"}}"}]: dispatch 2020-09-08T15:28:19.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:19 smithi104 bash[9203]: audit 2020-09-08T15:28:19.033601+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/inventory","val":"{\"smithi104\": {\"hostname\": \"smithi104\", \"addr\": \"smithi104\", \"labels\": [], \"status\": \"\"}, \"smithi055\": {\"hostname\": \"smithi055\", \"addr\": \"smithi055\", \"labels\": [], \"status\": \"\"}, \"smithi166\": {\"hostname\": \"smithi166\", \"addr\": \"smithi166\", \"labels\": [], \"status\": \"\"}}"}]': finished 2020-09-08T15:28:19.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:19 smithi104 bash[9203]: audit 2020-09-08T15:28:19.387467+0000 mon.a (mon.0) 172 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {}, \"devices\": [], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"networks\": {}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:28:19.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:19 smithi104 bash[9203]: audit 2020-09-08T15:28:19.391935+0000 mon.a (mon.0) 173 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {}, \"devices\": [], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"networks\": {}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:28:19.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:19 smithi104 bash[9203]: audit 2020-09-08T15:28:19.393852+0000 mon.a (mon.0) 174 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:28:20.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:20 smithi104 bash[9203]: cluster 2020-09-08T15:28:18.623003+0000 mgr.y (mgr.14145) 57 : cluster [DBG] pgmap v47: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:20.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:20 smithi104 bash[9203]: cephadm 2020-09-08T15:28:19.034161+0000 mgr.y (mgr.14145) 58 : cephadm [INF] Added host smithi166 2020-09-08T15:28:20.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:20 smithi104 bash[9203]: audit 2020-09-08T15:28:19.676207+0000 mon.a (mon.0) 175 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {}, \"devices\": [], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2020-09-08T15:28:19.675698\", \"networks\": {}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:28:20.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:20 smithi104 bash[9203]: audit 2020-09-08T15:28:19.678722+0000 mon.a (mon.0) 176 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {}, \"devices\": [], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2020-09-08T15:28:19.675698\", \"networks\": {}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:28:20.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:20 smithi104 bash[9203]: audit 2020-09-08T15:28:19.680120+0000 mon.a (mon.0) 177 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:28:22.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:22 smithi104 bash[9203]: cluster 2020-09-08T15:28:20.623445+0000 mgr.y (mgr.14145) 59 : cluster [DBG] pgmap v48: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:23.834 INFO:teuthology.orchestra.run.smithi166.stdout: 2020-09-08T15:28:23.835 INFO:teuthology.orchestra.run.smithi166.stdout:[{"addr": "smithi104", "hostname": "smithi104", "labels": [], "status": ""}, {"addr": "smithi055", "hostname": "smithi055", "labels": [], "status": ""}, {"addr": "smithi166", "hostname": "smithi166", "labels": [], "status": ""}] 2020-09-08T15:28:24.570 INFO:tasks.cephadm:Setting crush tunables to default 2020-09-08T15:28:24.572 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd crush tunables default 2020-09-08T15:28:24.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:24 smithi104 bash[9203]: cluster 2020-09-08T15:28:22.623907+0000 mgr.y (mgr.14145) 60 : cluster [DBG] pgmap v49: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:25.851 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:25 smithi104 bash[9203]: audit 2020-09-08T15:28:23.837723+0000 mgr.y (mgr.14145) 61 : audit [DBG] from='client.14173 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:28:26.178 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:28:26 smithi104 bash[9475]: debug 2020-09-08T15:28:26.021+0000 7f6ee9d53700 -1 ceph_set_health_checks check CEPHADM_REFRESH_FAILED detail item 0 not a [unicode] string 2020-09-08T15:28:26.178 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:28:26 smithi104 bash[9475]: debug 2020-09-08T15:28:26.021+0000 7f6ee9d53700 -1 ceph_set_health_checks check CEPHADM_REFRESH_FAILED detail item 0 not a [unicode] string 2020-09-08T15:28:26.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:26 smithi104 bash[9203]: cluster 2020-09-08T15:28:24.624420+0000 mgr.y (mgr.14145) 62 : cluster [DBG] pgmap v50: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:26.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:26 smithi104 bash[9203]: audit 2020-09-08T15:28:25.721070+0000 mon.a (mon.0) 178 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:28:26.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:26 smithi104 bash[9203]: audit 2020-09-08T15:28:26.012600+0000 mon.a (mon.0) 179 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2020-09-08T15:28:19.675698\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:28:26.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:26 smithi104 bash[9203]: audit 2020-09-08T15:28:26.017219+0000 mon.a (mon.0) 180 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2020-09-08T15:28:19.675698\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:28:26.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:26 smithi104 bash[9203]: audit 2020-09-08T15:28:26.019131+0000 mon.a (mon.0) 181 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2020-09-08T15:28:19.675698\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:28:26.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:26 smithi104 bash[9203]: audit 2020-09-08T15:28:26.023840+0000 mon.a (mon.0) 182 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {}, \"last_daemon_update\": \"2020-09-08T15:28:19.675698\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:28:26.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:26 smithi104 bash[9203]: audit 2020-09-08T15:28:26.026304+0000 mon.a (mon.0) 183 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:28:26.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:26 smithi104 bash[9203]: audit 2020-09-08T15:28:26.030159+0000 mon.a (mon.0) 184 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:28:27.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:27 smithi104 bash[9203]: cluster 2020-09-08T15:28:26.025556+0000 mgr.y (mgr.14145) 63 : cluster [DBG] pgmap v51: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:27.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:27 smithi104 bash[9203]: cluster 2020-09-08T15:28:27.020790+0000 mon.a (mon.0) 185 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2020-09-08T15:28:28.487 INFO:teuthology.orchestra.run.smithi104.stderr:adjusted tunables profile to default 2020-09-08T15:28:28.933 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:28 smithi104 bash[9203]: audit 2020-09-08T15:28:27.841173+0000 mon.a (mon.0) 186 : audit [INF] from='client.? v1:172.21.15.104:0/1491953069' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2020-09-08T15:28:29.062 INFO:tasks.cephadm:Adding mon.b on smithi055 2020-09-08T15:28:29.063 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph orch daemon add mon 'smithi055:[v1:172.21.15.55:6789]=b' 2020-09-08T15:28:29.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:29 smithi104 bash[9203]: cluster 2020-09-08T15:28:28.026052+0000 mgr.y (mgr.14145) 64 : cluster [DBG] pgmap v52: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:29.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:29 smithi104 bash[9203]: audit 2020-09-08T15:28:28.487974+0000 mon.a (mon.0) 187 : audit [INF] from='client.? v1:172.21.15.104:0/1491953069' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2020-09-08T15:28:29.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:29 smithi104 bash[9203]: cluster 2020-09-08T15:28:28.488107+0000 mon.a (mon.0) 188 : cluster [DBG] osdmap e6: 0 total, 0 up, 0 in 2020-09-08T15:28:31.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:31 smithi104 bash[9203]: cluster 2020-09-08T15:28:30.026548+0000 mgr.y (mgr.14145) 65 : cluster [DBG] pgmap v54: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:32.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:32 smithi104 bash[9203]: audit 2020-09-08T15:28:32.226192+0000 mon.a (mon.0) 189 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2020-09-08T15:28:32.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:32 smithi104 bash[9203]: audit 2020-09-08T15:28:32.227831+0000 mon.a (mon.0) 190 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:28:32.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:32 smithi104 bash[9203]: audit 2020-09-08T15:28:32.229702+0000 mon.a (mon.0) 191 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon.b", "key": "container_image"}]: dispatch 2020-09-08T15:28:33.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:33 smithi104 bash[9203]: cluster 2020-09-08T15:28:32.026986+0000 mgr.y (mgr.14145) 66 : cluster [DBG] pgmap v55: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:33.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:33 smithi104 bash[9203]: audit 2020-09-08T15:28:32.222038+0000 mgr.y (mgr.14145) 67 : audit [DBG] from='client.14177 -' entity='client.admin' cmd=[{"prefix": "orch daemon add", "daemon_type": "mon", "placement": "smithi055:[v1:172.21.15.55:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:28:33.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:33 smithi104 bash[9203]: cephadm 2020-09-08T15:28:32.229121+0000 mgr.y (mgr.14145) 68 : cephadm [INF] Deploying daemon mon.b on smithi055 2020-09-08T15:28:35.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:35 smithi104 bash[9203]: cluster 2020-09-08T15:28:34.027475+0000 mgr.y (mgr.14145) 69 : cluster [DBG] pgmap v56: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:36.486 INFO:teuthology.orchestra.run.smithi055.stdout:Deployed mon.b on host 'smithi055' 2020-09-08T15:28:36.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:36 smithi104 bash[9203]: audit 2020-09-08T15:28:36.468596+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:32.227420\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:28:36.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:36 smithi104 bash[9203]: audit 2020-09-08T15:28:36.469471+0000 mon.a (mon.0) 193 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:28:36.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:36 smithi104 bash[9203]: audit 2020-09-08T15:28:36.473168+0000 mon.a (mon.0) 194 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:32.227420\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:28:37.607 INFO:teuthology.orchestra.run.smithi055:mon.b> sudo journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mon.b.service 2020-09-08T15:28:37.615 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2020-09-08T15:28:37.616 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph mon dump -f json 2020-09-08T15:28:37.703 INFO:journalctl@ceph.mon.b.smithi055.stdout:-- Logs begin at Tue 2020-09-01 16:49:30 UTC. -- 2020-09-08T15:28:37.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:37 smithi104 bash[9203]: cluster 2020-09-08T15:28:36.027967+0000 mgr.y (mgr.14145) 70 : cluster [DBG] pgmap v57: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:37.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:37 smithi104 bash[9203]: audit 2020-09-08T15:28:37.059376+0000 mon.a (mon.0) 195 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:37.057914\", \"created\": \"2020-09-08T15:28:36.438141\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:32.227420\"}}, \"last_daemon_update\": \"2020-09-08T15:28:37.058241\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:28:37.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:37 smithi104 bash[9203]: audit 2020-09-08T15:28:37.063658+0000 mon.a (mon.0) 196 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:37.057914\", \"created\": \"2020-09-08T15:28:36.438141\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:32.227420\"}}, \"last_daemon_update\": \"2020-09-08T15:28:37.058241\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:28:37.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:37 smithi104 bash[9203]: audit 2020-09-08T15:28:37.065940+0000 mon.a (mon.0) 197 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:28:37.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:37 smithi104 bash[9203]: audit 2020-09-08T15:28:37.071762+0000 mon.a (mon.0) 198 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:28:38.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.914+0000 7ff84e697700 0 set uid:gid to 167:167 (ceph:ceph) 2020-09-08T15:28:38.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.914+0000 7ff84e697700 0 ceph version 16.0.0-5272-gdce4d291bd4 (dce4d291bd4243c9802c17d67b85a6da6bbc0f8f) pacific (dev), process ceph-mon, pid 1 2020-09-08T15:28:38.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.914+0000 7ff84e697700 0 pidfile_write: ignore empty --pid-file 2020-09-08T15:28:38.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 0 load: jerasure load: lrc load: isa 2020-09-08T15:28:38.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: RocksDB version: 6.8.1 2020-09-08T15:28:38.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2020-09-08T15:28:38.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Compile date Sep 8 2020 2020-09-08T15:28:38.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: DB SUMMARY 2020-09-08T15:28:38.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: CURRENT file: CURRENT 2020-09-08T15:28:38.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: IDENTITY file: IDENTITY 2020-09-08T15:28:38.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: MANIFEST file: MANIFEST-000001 size: 13 Bytes 2020-09-08T15:28:38.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2020-09-08T15:28:38.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000003.log size: 454 ; 2020-09-08T15:28:38.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.error_if_exists: 0 2020-09-08T15:28:38.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.create_if_missing: 0 2020-09-08T15:28:38.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.paranoid_checks: 1 2020-09-08T15:28:38.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.env: 0x55c189f85c00 2020-09-08T15:28:38.413 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.fs: Posix File System 2020-09-08T15:28:38.413 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.info_log: 0x55c18b3c5f60 2020-09-08T15:28:38.413 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_file_opening_threads: 16 2020-09-08T15:28:38.414 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.statistics: (nil) 2020-09-08T15:28:38.414 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.use_fsync: 0 2020-09-08T15:28:38.414 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_log_file_size: 0 2020-09-08T15:28:38.414 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2020-09-08T15:28:38.415 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.log_file_time_to_roll: 0 2020-09-08T15:28:38.415 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.keep_log_file_num: 1000 2020-09-08T15:28:38.415 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.recycle_log_file_num: 0 2020-09-08T15:28:38.415 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.allow_fallocate: 1 2020-09-08T15:28:38.416 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.allow_mmap_reads: 0 2020-09-08T15:28:38.416 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.allow_mmap_writes: 0 2020-09-08T15:28:38.416 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.use_direct_reads: 0 2020-09-08T15:28:38.416 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2020-09-08T15:28:38.417 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.create_missing_column_families: 0 2020-09-08T15:28:38.417 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.db_log_dir: 2020-09-08T15:28:38.417 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2020-09-08T15:28:38.417 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.table_cache_numshardbits: 6 2020-09-08T15:28:38.417 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_subcompactions: 1 2020-09-08T15:28:38.418 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_background_flushes: -1 2020-09-08T15:28:38.418 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.WAL_ttl_seconds: 0 2020-09-08T15:28:38.418 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.WAL_size_limit_MB: 0 2020-09-08T15:28:38.418 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2020-09-08T15:28:38.418 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2020-09-08T15:28:38.419 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.is_fd_close_on_exec: 1 2020-09-08T15:28:38.419 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.advise_random_on_open: 1 2020-09-08T15:28:38.419 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.db_write_buffer_size: 0 2020-09-08T15:28:38.420 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.write_buffer_manager: 0x55c18b40d710 2020-09-08T15:28:38.420 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2020-09-08T15:28:38.421 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2020-09-08T15:28:38.421 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2020-09-08T15:28:38.422 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.use_adaptive_mutex: 0 2020-09-08T15:28:38.422 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.rate_limiter: (nil) 2020-09-08T15:28:38.423 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2020-09-08T15:28:38.423 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.wal_recovery_mode: 2 2020-09-08T15:28:38.423 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.enable_thread_tracking: 0 2020-09-08T15:28:38.426 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.enable_pipelined_write: 0 2020-09-08T15:28:38.427 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.unordered_write: 0 2020-09-08T15:28:38.427 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2020-09-08T15:28:38.427 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2020-09-08T15:28:38.427 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2020-09-08T15:28:38.427 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2020-09-08T15:28:38.428 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.row_cache: None 2020-09-08T15:28:38.428 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.wal_filter: None 2020-09-08T15:28:38.428 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2020-09-08T15:28:38.428 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.allow_ingest_behind: 0 2020-09-08T15:28:38.429 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.preserve_deletes: 0 2020-09-08T15:28:38.429 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.two_write_queues: 0 2020-09-08T15:28:38.429 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.manual_wal_flush: 0 2020-09-08T15:28:38.430 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.atomic_flush: 0 2020-09-08T15:28:38.430 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2020-09-08T15:28:38.430 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.persist_stats_to_disk: 0 2020-09-08T15:28:38.430 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.write_dbid_to_manifest: 0 2020-09-08T15:28:38.431 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.log_readahead_size: 0 2020-09-08T15:28:38.431 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2020-09-08T15:28:38.431 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_background_jobs: 2 2020-09-08T15:28:38.431 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_background_compactions: -1 2020-09-08T15:28:38.432 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2020-09-08T15:28:38.432 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2020-09-08T15:28:38.432 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.delayed_write_rate : 16777216 2020-09-08T15:28:38.432 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_total_wal_size: 0 2020-09-08T15:28:38.433 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2020-09-08T15:28:38.433 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.stats_dump_period_sec: 600 2020-09-08T15:28:38.433 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.stats_persist_period_sec: 600 2020-09-08T15:28:38.433 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2020-09-08T15:28:38.433 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_open_files: -1 2020-09-08T15:28:38.434 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.bytes_per_sync: 0 2020-09-08T15:28:38.434 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.wal_bytes_per_sync: 0 2020-09-08T15:28:38.434 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.strict_bytes_per_sync: 0 2020-09-08T15:28:38.435 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.compaction_readahead_size: 0 2020-09-08T15:28:38.435 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Compression algorithms supported: 2020-09-08T15:28:38.435 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2020-09-08T15:28:38.435 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: kZSTD supported: 0 2020-09-08T15:28:38.435 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: kXpressCompression supported: 0 2020-09-08T15:28:38.436 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: kLZ4HCCompression supported: 1 2020-09-08T15:28:38.436 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: kLZ4Compression supported: 1 2020-09-08T15:28:38.436 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: kBZip2Compression supported: 0 2020-09-08T15:28:38.436 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: kZlibCompression supported: 1 2020-09-08T15:28:38.436 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: kSnappyCompression supported: 1 2020-09-08T15:28:38.437 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Fast CRC32 supported: Supported on x86 2020-09-08T15:28:38.437 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000001 2020-09-08T15:28:38.437 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2020-09-08T15:28:38.437 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2020-09-08T15:28:38.437 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.merge_operator: 2020-09-08T15:28:38.438 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.compaction_filter: None 2020-09-08T15:28:38.438 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.compaction_filter_factory: None 2020-09-08T15:28:38.438 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.memtable_factory: SkipListFactory 2020-09-08T15:28:38.438 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.table_factory: BlockBasedTable 2020-09-08T15:28:38.438 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c18b369a48) 2020-09-08T15:28:38.439 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: cache_index_and_filter_blocks: 1 2020-09-08T15:28:38.439 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: cache_index_and_filter_blocks_with_high_priority: 1 2020-09-08T15:28:38.439 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: pin_l0_filter_and_index_blocks_in_cache: 0 2020-09-08T15:28:38.439 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: pin_top_level_index_and_filter: 1 2020-09-08T15:28:38.439 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: index_type: 0 2020-09-08T15:28:38.440 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: data_block_index_type: 0 2020-09-08T15:28:38.440 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: index_shortening: 1 2020-09-08T15:28:38.440 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: data_block_hash_table_util_ratio: 0.750000 2020-09-08T15:28:38.440 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: hash_index_allow_collision: 1 2020-09-08T15:28:38.440 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: checksum: 1 2020-09-08T15:28:38.441 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: no_block_cache: 0 2020-09-08T15:28:38.441 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: block_cache: 0x55c18b3a0610 2020-09-08T15:28:38.441 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: block_cache_name: BinnedLRUCache 2020-09-08T15:28:38.441 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: block_cache_options: 2020-09-08T15:28:38.441 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: capacity : 536870912 2020-09-08T15:28:38.442 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: num_shard_bits : 4 2020-09-08T15:28:38.442 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: strict_capacity_limit : 0 2020-09-08T15:28:38.442 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: high_pri_pool_ratio: 0.000 2020-09-08T15:28:38.442 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: block_cache_compressed: (nil) 2020-09-08T15:28:38.442 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: persistent_cache: (nil) 2020-09-08T15:28:38.442 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: block_size: 4096 2020-09-08T15:28:38.443 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: block_size_deviation: 10 2020-09-08T15:28:38.443 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: block_restart_interval: 16 2020-09-08T15:28:38.443 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: index_block_restart_interval: 1 2020-09-08T15:28:38.443 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: metadata_block_size: 4096 2020-09-08T15:28:38.443 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: partition_filters: 0 2020-09-08T15:28:38.444 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: use_delta_encoding: 1 2020-09-08T15:28:38.444 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: filter_policy: rocksdb.BuiltinBloomFilter 2020-09-08T15:28:38.444 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: whole_key_filtering: 1 2020-09-08T15:28:38.444 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: verify_compression: 0 2020-09-08T15:28:38.444 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: read_amp_bytes_per_bit: 0 2020-09-08T15:28:38.445 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: format_version: 2 2020-09-08T15:28:38.445 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: enable_index_compression: 1 2020-09-08T15:28:38.445 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: block_align: 0 2020-09-08T15:28:38.445 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.write_buffer_size: 33554432 2020-09-08T15:28:38.445 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_write_buffer_number: 2 2020-09-08T15:28:38.445 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.compression: NoCompression 2020-09-08T15:28:38.446 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.bottommost_compression: Disabled 2020-09-08T15:28:38.446 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.prefix_extractor: nullptr 2020-09-08T15:28:38.446 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2020-09-08T15:28:38.446 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.num_levels: 7 2020-09-08T15:28:38.446 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2020-09-08T15:28:38.447 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2020-09-08T15:28:38.447 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2020-09-08T15:28:38.447 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2020-09-08T15:28:38.447 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2020-09-08T15:28:38.447 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2020-09-08T15:28:38.448 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2020-09-08T15:28:38.448 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2020-09-08T15:28:38.448 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2020-09-08T15:28:38.448 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.compression_opts.window_bits: -14 2020-09-08T15:28:38.448 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.compression_opts.level: 32767 2020-09-08T15:28:38.449 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.compression_opts.strategy: 0 2020-09-08T15:28:38.449 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2020-09-08T15:28:38.449 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2020-09-08T15:28:38.449 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.compression_opts.enabled: false 2020-09-08T15:28:38.449 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2020-09-08T15:28:38.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2020-09-08T15:28:38.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2020-09-08T15:28:38.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.target_file_size_base: 67108864 2020-09-08T15:28:38.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.target_file_size_multiplier: 1 2020-09-08T15:28:38.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2020-09-08T15:28:38.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2020-09-08T15:28:38.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2020-09-08T15:28:38.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2020-09-08T15:28:38.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2020-09-08T15:28:38.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2020-09-08T15:28:38.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2020-09-08T15:28:38.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2020-09-08T15:28:38.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2020-09-08T15:28:38.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2020-09-08T15:28:38.453 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2020-09-08T15:28:38.453 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2020-09-08T15:28:38.453 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.arena_block_size: 4194304 2020-09-08T15:28:38.453 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2020-09-08T15:28:38.453 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2020-09-08T15:28:38.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2020-09-08T15:28:38.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.disable_auto_compactions: 0 2020-09-08T15:28:38.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2020-09-08T15:28:38.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2020-09-08T15:28:38.454 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2020-09-08T15:28:38.455 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2020-09-08T15:28:38.455 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2020-09-08T15:28:38.455 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2020-09-08T15:28:38.455 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2020-09-08T15:28:38.456 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2020-09-08T15:28:38.456 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2020-09-08T15:28:38.456 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2020-09-08T15:28:38.456 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.table_properties_collectors: 2020-09-08T15:28:38.457 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.inplace_update_support: 0 2020-09-08T15:28:38.458 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.inplace_update_num_locks: 10000 2020-09-08T15:28:38.458 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2020-09-08T15:28:38.459 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2020-09-08T15:28:38.459 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.memtable_huge_page_size: 0 2020-09-08T15:28:38.459 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.bloom_locality: 0 2020-09-08T15:28:38.459 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.max_successive_merges: 0 2020-09-08T15:28:38.459 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.optimize_filters_for_hits: 0 2020-09-08T15:28:38.460 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.paranoid_file_checks: 0 2020-09-08T15:28:38.460 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.force_consistency_checks: 0 2020-09-08T15:28:38.460 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.report_bg_io_stats: 0 2020-09-08T15:28:38.460 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.ttl: 2592000 2020-09-08T15:28:38.460 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: Options.periodic_compaction_seconds: 0 2020-09-08T15:28:38.460 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2020-09-08T15:28:38.461 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2020-09-08T15:28:38.461 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578917921898, "job": 1, "event": "recovery_started", "log_files": [3]} 2020-09-08T15:28:38.461 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #3 mode 2 2020-09-08T15:28:38.461 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.918+0000 7ff84e697700 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2020-09-08T15:28:38.462 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.922+0000 7ff84e697700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578917922686, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 4, "file_size": 1380, "table_properties": {"data_size": 466, "index_size": 40, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 344, "raw_average_value_size": 68, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1599578917, "oldest_key_time": 3, "file_creation_time": 0}} 2020-09-08T15:28:38.462 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.922+0000 7ff84e697700 4 rocksdb: [version_set.cc:3826] Creating manifest 5 2020-09-08T15:28:38.462 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.922+0000 7ff84e697700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578917923930, "job": 1, "event": "recovery_finished"} 2020-09-08T15:28:38.462 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.922+0000 7ff84e697700 4 rocksdb: DB pointer 0x55c18b3ed800 2020-09-08T15:28:38.462 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.922+0000 7ff8354d6700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2020-09-08T15:28:38.463 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: debug 2020-09-08T15:28:37.922+0000 7ff8354d6700 4 rocksdb: [db_impl/db_impl.cc:851] 2020-09-08T15:28:38.463 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: ** DB Stats ** 2020-09-08T15:28:38.463 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: Uptime(secs): 0.0 total, 0.0 interval 2020-09-08T15:28:38.463 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2020-09-08T15:28:38.463 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2020-09-08T15:28:38.463 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2020-09-08T15:28:38.464 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2020-09-08T15:28:38.464 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2020-09-08T15:28:38.464 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2020-09-08T15:28:38.464 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: ** Compaction Stats [default] ** 2020-09-08T15:28:38.464 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2020-09-08T15:28:38.465 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2020-09-08T15:28:38.465 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: L0 1/0 1.35 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 2020-09-08T15:28:38.465 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: Sum 1/0 1.35 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 2020-09-08T15:28:38.465 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 2020-09-08T15:28:38.466 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: ** Compaction Stats [default] ** 2020-09-08T15:28:38.466 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2020-09-08T15:28:38.466 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2020-09-08T15:28:38.466 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.7 0.00 0.00 1 0.001 0 0 2020-09-08T15:28:38.466 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: Uptime(secs): 0.0 total, 0.0 interval 2020-09-08T15:28:38.467 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: Flush(GB): cumulative 0.000, interval 0.000 2020-09-08T15:28:38.467 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: AddFile(GB): cumulative 0.000, interval 0.000 2020-09-08T15:28:38.467 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: AddFile(Total Files): cumulative 0, interval 0 2020-09-08T15:28:38.467 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: AddFile(L0 Files): cumulative 0, interval 0 2020-09-08T15:28:38.468 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: AddFile(Keys): cumulative 0, interval 0 2020-09-08T15:28:38.468 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:37 smithi055 bash[9296]: Cumulative compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2020-09-08T15:28:38.468 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: Interval compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2020-09-08T15:28:38.468 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2020-09-08T15:28:38.468 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: ** File Read Latency Histogram By Level [default] ** 2020-09-08T15:28:38.469 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: ** Compaction Stats [default] ** 2020-09-08T15:28:38.469 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2020-09-08T15:28:38.469 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2020-09-08T15:28:38.469 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: L0 1/0 1.35 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 2020-09-08T15:28:38.469 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: Sum 1/0 1.35 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 2020-09-08T15:28:38.470 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2020-09-08T15:28:38.470 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: ** Compaction Stats [default] ** 2020-09-08T15:28:38.470 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2020-09-08T15:28:38.470 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2020-09-08T15:28:38.470 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.7 0.00 0.00 1 0.001 0 0 2020-09-08T15:28:38.471 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: Uptime(secs): 0.0 total, 0.0 interval 2020-09-08T15:28:38.471 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: Flush(GB): cumulative 0.000, interval 0.000 2020-09-08T15:28:38.471 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: AddFile(GB): cumulative 0.000, interval 0.000 2020-09-08T15:28:38.471 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: AddFile(Total Files): cumulative 0, interval 0 2020-09-08T15:28:38.471 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: AddFile(L0 Files): cumulative 0, interval 0 2020-09-08T15:28:38.472 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: AddFile(Keys): cumulative 0, interval 0 2020-09-08T15:28:38.472 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: Cumulative compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2020-09-08T15:28:38.472 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2020-09-08T15:28:38.472 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2020-09-08T15:28:38.472 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: ** File Read Latency Histogram By Level [default] ** 2020-09-08T15:28:38.473 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.922+0000 7ff84e697700 0 mon.b does not exist in monmap, will attempt to join an existing cluster 2020-09-08T15:28:38.473 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.922+0000 7ff84e697700 0 using public_addrv v1:172.21.15.55:6789/0 2020-09-08T15:28:38.473 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.922+0000 7ff84e697700 0 starting mon.b rank -1 at public addrs v1:172.21.15.55:6789/0 at bind addrs v1:172.21.15.55:6789/0 mon_data /var/lib/ceph/mon/ceph-b fsid 759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:28:38.473 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.926+0000 7ff84e697700 1 mon.b@-1(???) e0 preinit fsid 759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:28:38.473 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.970+0000 7ff8334d2700 4 rocksdb: [db_impl/db_impl_write.cc:1668] [default] New memtable created with log file: #9. Immutable memtables: 0. 2020-09-08T15:28:38.474 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.970+0000 7ff83d4e6700 4 rocksdb: (Original Log Time 2020/09/08-15:28:37.971703) [db_impl/db_impl_compaction_flush.cc:2198] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 2020-09-08T15:28:38.474 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.970+0000 7ff83d4e6700 4 rocksdb: [flush_job.cc:321] [default] [JOB 3] Flushing memtable with next log file: 9 2020-09-08T15:28:38.474 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.970+0000 7ff83d4e6700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578917971822, "job": 3, "event": "flush_started", "num_memtables": 1, "num_entries": 973, "num_deletes": 11, "total_data_size": 6428002, "memory_usage": 6475072, "flush_reason": "Manual Compaction"} 2020-09-08T15:28:38.474 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.970+0000 7ff83d4e6700 4 rocksdb: [flush_job.cc:350] [default] [JOB 3] Level-0 flush table #10: started 2020-09-08T15:28:38.474 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.982+0000 7ff83d4e6700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578917983398, "cf_name": "default", "job": 3, "event": "table_file_creation", "file_number": 10, "file_size": 6235535, "table_properties": {"data_size": 6227431, "index_size": 5780, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 1477, "raw_key_size": 18468, "raw_average_key_size": 31, "raw_value_size": 6212230, "raw_average_value_size": 10601, "num_data_blocks": 186, "num_entries": 586, "num_deletions": 11, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1599578917, "oldest_key_time": 1599578917, "file_creation_time": 1599578917}} 2020-09-08T15:28:38.475 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.982+0000 7ff83d4e6700 4 rocksdb: [flush_job.cc:401] [default] [JOB 3] Level-0 flush table #10: 6235535 bytes OK 2020-09-08T15:28:38.475 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.982+0000 7ff83d4e6700 4 rocksdb: (Original Log Time 2020/09/08-15:28:37.983449) [memtable_list.cc:447] [default] Level-0 commit table #10 started 2020-09-08T15:28:38.475 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.982+0000 7ff83d4e6700 4 rocksdb: (Original Log Time 2020/09/08-15:28:37.983607) [memtable_list.cc:503] [default] Level-0 commit table #10: memtable #1 done 2020-09-08T15:28:38.475 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.982+0000 7ff83d4e6700 4 rocksdb: (Original Log Time 2020/09/08-15:28:37.983621) EVENT_LOG_v1 {"time_micros": 1599578917983616, "job": 3, "event": "flush_finished", "output_compression": "NoCompression", "lsm_state": [2, 0, 0, 0, 0, 0, 0], "immutable_memtables": 0} 2020-09-08T15:28:38.476 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.982+0000 7ff83d4e6700 4 rocksdb: (Original Log Time 2020/09/08-15:28:37.983635) [db_impl/db_impl_compaction_flush.cc:205] [default] Level summary: files[2 0 0 0 0 0 0] max score 0.50 2020-09-08T15:28:38.476 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.982+0000 7ff83d4e6700 4 rocksdb: [db_impl/db_impl_files.cc:353] [JOB 3] Try to delete WAL files size 6421252, prev total WAL file size 7510983, number of live WAL files 2. 2020-09-08T15:28:38.476 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.982+0000 7ff8334d2700 4 rocksdb: [db_impl/db_impl_compaction_flush.cc:1443] [default] Manual compaction starting 2020-09-08T15:28:38.476 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.982+0000 7ff83dce7700 4 rocksdb: (Original Log Time 2020/09/08-15:28:37.984831) [db_impl/db_impl_compaction_flush.cc:2516] [default] Manual compaction from level-0 to level-6 from 'mgrstat .. 'mgrstat; will stop at (end) 2020-09-08T15:28:38.476 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.982+0000 7ff83dce7700 4 rocksdb: [compaction/compaction_job.cc:1676] [default] [JOB 4] Compacting 2@0 files to L6, score -1.00 2020-09-08T15:28:38.477 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.982+0000 7ff83dce7700 4 rocksdb: [compaction/compaction_job.cc:1680] [default] Compaction start summary: Base version 3 Base level 0, inputs: [10(6089KB) 4(1380B)] 2020-09-08T15:28:38.477 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.982+0000 7ff83dce7700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578917984910, "job": 4, "event": "compaction_started", "compaction_reason": "ManualCompaction", "files_L0": [10, 4], "score": -1, "input_data_size": 6236915} 2020-09-08T15:28:38.477 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.994+0000 7ff83dce7700 4 rocksdb: [compaction/compaction_job.cc:1349] [default] [JOB 4] Generated table #11: 580 keys, 6235803 bytes 2020-09-08T15:28:38.477 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.994+0000 7ff83dce7700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578917998210, "cf_name": "default", "job": 4, "event": "table_file_creation", "file_number": 11, "file_size": 6235803, "table_properties": {"data_size": 6227705, "index_size": 5778, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 1477, "raw_key_size": 18394, "raw_average_key_size": 31, "raw_value_size": 6212574, "raw_average_value_size": 10711, "num_data_blocks": 186, "num_entries": 580, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1599578917, "oldest_key_time": 0, "file_creation_time": 1599578917}} 2020-09-08T15:28:38.477 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.998+0000 7ff83dce7700 4 rocksdb: [compaction/compaction_job.cc:1415] [default] [JOB 4] Compacted 2@0 files to L6 => 6235803 bytes 2020-09-08T15:28:38.478 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.998+0000 7ff83dce7700 4 rocksdb: [version_set.cc:3291] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2020-09-08T15:28:38.478 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.998+0000 7ff83dce7700 4 rocksdb: (Original Log Time 2020/09/08-15:28:37.998642) [compaction/compaction_job.cc:760] [default] compacted to: base level 6 level multiplier 10.00 max bytes base 268435456 files[0 0 0 0 0 0 1] max score 0.00, MB/sec: 465.6 rd, 465.5 wr, level 6, files in(2, 0) out(1) MB in(5.9, 0.0) out(5.9), read-write-amplify(2.0) write-amplify(1.0) OK, records in: 591, records dropped: 11 output_compression: NoCompression 2020-09-08T15:28:38.478 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.998+0000 7ff83dce7700 4 rocksdb: (Original Log Time 2020/09/08-15:28:37.998666) EVENT_LOG_v1 {"time_micros": 1599578917998657, "job": 4, "event": "compaction_finished", "compaction_time_micros": 13395, "compaction_time_cpu_micros": 7409, "output_level": 6, "num_output_files": 1, "total_output_size": 6235803, "num_input_records": 591, "num_output_records": 580, "num_subcompactions": 1, "output_compression": "NoCompression", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "lsm_state": [0, 0, 0, 0, 0, 0, 1]} 2020-09-08T15:28:38.478 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.998+0000 7ff83dce7700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578917999828, "job": 4, "event": "table_file_deletion", "file_number": 10} 2020-09-08T15:28:38.479 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:37.998+0000 7ff83dce7700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578917999928, "job": 4, "event": "table_file_deletion", "file_number": 4} 2020-09-08T15:28:38.479 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.002+0000 7ff8334d2700 4 rocksdb: [db_impl/db_impl_write.cc:1668] [default] New memtable created with log file: #12. Immutable memtables: 0. 2020-09-08T15:28:38.479 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.002+0000 7ff83d4e6700 4 rocksdb: (Original Log Time 2020/09/08-15:28:38.003015) [db_impl/db_impl_compaction_flush.cc:2198] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 2020-09-08T15:28:38.479 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.002+0000 7ff83d4e6700 4 rocksdb: [flush_job.cc:321] [default] [JOB 5] Flushing memtable with next log file: 12 2020-09-08T15:28:38.479 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.002+0000 7ff83d4e6700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578918003046, "job": 5, "event": "flush_started", "num_memtables": 1, "num_entries": 337, "num_deletes": 31, "total_data_size": 4283612, "memory_usage": 4338576, "flush_reason": "Manual Compaction"} 2020-09-08T15:28:38.480 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.002+0000 7ff83d4e6700 4 rocksdb: [flush_job.cc:350] [default] [JOB 5] Level-0 flush table #13: started 2020-09-08T15:28:38.480 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.006+0000 7ff83d4e6700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578918009319, "cf_name": "default", "job": 5, "event": "table_file_creation", "file_number": 13, "file_size": 4234588, "table_properties": {"data_size": 4231446, "index_size": 1844, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 453, "raw_key_size": 3150, "raw_average_key_size": 18, "raw_value_size": 4227693, "raw_average_value_size": 24723, "num_data_blocks": 58, "num_entries": 171, "num_deletions": 31, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1599578917, "oldest_key_time": 1599578917, "file_creation_time": 1599578918}} 2020-09-08T15:28:38.480 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.006+0000 7ff83d4e6700 4 rocksdb: [flush_job.cc:401] [default] [JOB 5] Level-0 flush table #13: 4234588 bytes OK 2020-09-08T15:28:38.480 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.006+0000 7ff83d4e6700 4 rocksdb: [version_set.cc:3291] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2020-09-08T15:28:38.481 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff83d4e6700 4 rocksdb: (Original Log Time 2020/09/08-15:28:38.009393) [memtable_list.cc:447] [default] Level-0 commit table #13 started 2020-09-08T15:28:38.481 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff83d4e6700 4 rocksdb: (Original Log Time 2020/09/08-15:28:38.010615) [memtable_list.cc:503] [default] Level-0 commit table #13: memtable #1 done 2020-09-08T15:28:38.481 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff83d4e6700 4 rocksdb: (Original Log Time 2020/09/08-15:28:38.010633) EVENT_LOG_v1 {"time_micros": 1599578918010627, "job": 5, "event": "flush_finished", "output_compression": "NoCompression", "lsm_state": [1, 0, 0, 0, 0, 0, 1], "immutable_memtables": 0} 2020-09-08T15:28:38.481 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff83d4e6700 4 rocksdb: (Original Log Time 2020/09/08-15:28:38.010649) [db_impl/db_impl_compaction_flush.cc:205] [default] Level summary: base level 6 level multiplier 10.00 max bytes base 268435456 files[1 0 0 0 0 0 1] max score 0.25 2020-09-08T15:28:38.482 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff83d4e6700 4 rocksdb: [db_impl/db_impl_files.cc:353] [JOB 5] Try to delete WAL files size 4281270, prev total WAL file size 5982596, number of live WAL files 2. 2020-09-08T15:28:38.482 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff8334d2700 4 rocksdb: [db_impl/db_impl_compaction_flush.cc:1443] [default] Manual compaction starting 2020-09-08T15:28:38.482 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff83dce7700 4 rocksdb: (Original Log Time 2020/09/08-15:28:38.011543) [db_impl/db_impl_compaction_flush.cc:2516] [default] Manual compaction from level-0 to level-6 from 'mgrstat .. 'mgrstat; will stop at (end) 2020-09-08T15:28:38.482 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff83dce7700 4 rocksdb: [compaction/compaction_job.cc:1676] [default] [JOB 6] Compacting 1@0 + 1@6 files to L6, score -1.00 2020-09-08T15:28:38.482 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff83dce7700 4 rocksdb: [compaction/compaction_job.cc:1680] [default] Compaction start summary: Base version 5 Base level 0, inputs: [13(4135KB)], [11(6089KB)] 2020-09-08T15:28:38.483 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff83dce7700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578918011598, "job": 6, "event": "compaction_started", "compaction_reason": "ManualCompaction", "files_L0": [13], "files_L6": [11], "score": -1, "input_data_size": 10470391} 2020-09-08T15:28:38.483 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff837cdb700 0 mon.b@-1(synchronizing).mds e1 new map 2020-09-08T15:28:38.483 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff837cdb700 0 mon.b@-1(synchronizing).mds e1 print_map 2020-09-08T15:28:38.483 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: e1 2020-09-08T15:28:38.483 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: enable_multiple, ever_enabled_multiple: 0,0 2020-09-08T15:28:38.484 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2020-09-08T15:28:38.484 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: legacy client fscid: -1 2020-09-08T15:28:38.484 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: 2020-09-08T15:28:38.484 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: No filesystems configured 2020-09-08T15:28:38.484 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff837cdb700 1 mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2020-09-08T15:28:38.485 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff837cdb700 1 mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2020-09-08T15:28:38.485 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff837cdb700 1 mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2020-09-08T15:28:38.485 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff837cdb700 1 mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2020-09-08T15:28:38.485 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff837cdb700 1 mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2020-09-08T15:28:38.486 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff837cdb700 1 mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2020-09-08T15:28:38.486 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff837cdb700 1 mon.b@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2020-09-08T15:28:38.486 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff837cdb700 1 mon.b@-1(synchronizing).osd e6 e6: 0 total, 0 up, 0 in 2020-09-08T15:28:38.486 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff837cdb700 0 mon.b@-1(synchronizing).osd e6 crush map has features 288514051259236352, adjusting msgr requires 2020-09-08T15:28:38.486 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff837cdb700 0 mon.b@-1(synchronizing).osd e6 crush map has features 3314933000852226048, adjusting msgr requires 2020-09-08T15:28:38.487 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff837cdb700 0 mon.b@-1(synchronizing).osd e6 crush map has features 288514051259236352, adjusting msgr requires 2020-09-08T15:28:38.487 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: cluster 2020-09-08T15:28:36.027967+0000 mgr.y (mgr.14145) 70 : cluster [DBG] pgmap v57: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:38.487 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: audit 2020-09-08T15:28:37.059376+0000 mon.a (mon.0) 195 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:37.057914\", \"created\": \"2020-09-08T15:28:36.438141\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:32.227420\"}}, \"last_daemon_update\": \"2020-09-08T15:28:37.058241\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:28:38.487 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: audit 2020-09-08T15:28:37.063658+0000 mon.a (mon.0) 196 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:37.057914\", \"created\": \"2020-09-08T15:28:36.438141\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:32.227420\"}}, \"last_daemon_update\": \"2020-09-08T15:28:37.058241\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:28:38.488 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: audit 2020-09-08T15:28:37.065940+0000 mon.a (mon.0) 197 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:28:38.488 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: audit 2020-09-08T15:28:37.071762+0000 mon.a (mon.0) 198 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:28:38.488 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.010+0000 7ff837cdb700 1 mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2020-09-08T15:28:38.489 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.014+0000 7ff837cdb700 10 mon.b@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete,mon_osdmap_full_prune_interval,mon_osdmap_full_prune_min,mon_osdmap_full_prune_txsize 2020-09-08T15:28:38.489 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.014+0000 7ff837cdb700 20 expand_channel_meta expand map: {default=false} 2020-09-08T15:28:38.489 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.014+0000 7ff837cdb700 20 expand_channel_meta from 'false' to 'false' 2020-09-08T15:28:38.489 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.014+0000 7ff837cdb700 20 expand_channel_meta expanded map: {default=false} 2020-09-08T15:28:38.490 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.014+0000 7ff837cdb700 20 expand_channel_meta expand map: {default=info} 2020-09-08T15:28:38.490 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.014+0000 7ff837cdb700 20 expand_channel_meta from 'info' to 'info' 2020-09-08T15:28:38.490 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.014+0000 7ff837cdb700 20 expand_channel_meta expanded map: {default=info} 2020-09-08T15:28:38.490 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.014+0000 7ff837cdb700 20 expand_channel_meta expand map: {default=daemon} 2020-09-08T15:28:38.490 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.014+0000 7ff837cdb700 20 expand_channel_meta from 'daemon' to 'daemon' 2020-09-08T15:28:38.491 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.014+0000 7ff837cdb700 20 expand_channel_meta expanded map: {default=daemon} 2020-09-08T15:28:38.491 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.014+0000 7ff837cdb700 20 expand_channel_meta expand map: {default=debug} 2020-09-08T15:28:38.491 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.014+0000 7ff837cdb700 20 expand_channel_meta from 'debug' to 'debug' 2020-09-08T15:28:38.491 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:38 smithi055 bash[9296]: debug 2020-09-08T15:28:38.014+0000 7ff837cdb700 20 expand_channel_meta expanded map: {default=debug} 2020-09-08T15:28:43.348 INFO:teuthology.orchestra.run.smithi055.stdout: 2020-09-08T15:28:43.349 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":2,"fsid":"759b27b6-f1e7-11ea-a080-001a4aab830c","modified":"2020-09-08T15:28:38.018118Z","created":"2020-09-08T15:25:37.526987Z","min_mon_release":16,"min_mon_release_name":"pacific","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6789","nonce":0}]},"addr":"172.21.15.104:6789/0","public_addr":"172.21.15.104:6789/0","priority":0,"weight":0},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0}],"quorum":[0,1]} 2020-09-08T15:28:43.350 INFO:teuthology.orchestra.run.smithi055.stderr:dumped monmap epoch 2 2020-09-08T15:28:43.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:43 smithi055 bash[9296]: cluster 2020-09-08T15:28:38.024125+0000 mon.a (mon.0) 200 : cluster [INF] mon.a calling monitor election 2020-09-08T15:28:43.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:43 smithi055 bash[9296]: audit 2020-09-08T15:28:39.018099+0000 mon.a (mon.0) 201 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2020-09-08T15:28:43.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:43 smithi055 bash[9296]: cluster 2020-09-08T15:28:40.020756+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2020-09-08T15:28:43.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:43 smithi055 bash[9296]: cluster 2020-09-08T15:28:41.066061+0000 mgr.y (mgr.14145) 73 : cluster [DBG] pgmap v60: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:43.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:43 smithi055 bash[9296]: cluster 2020-09-08T15:28:43.066490+0000 mgr.y (mgr.14145) 74 : cluster [DBG] pgmap v61: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:43.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:43 smithi055 bash[9296]: cluster 2020-09-08T15:28:43.067396+0000 mon.a (mon.0) 202 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2020-09-08T15:28:43.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:43 smithi055 bash[9296]: cluster 2020-09-08T15:28:43.078590+0000 mon.a (mon.0) 203 : cluster [DBG] monmap e2: 2 mons at {a=v1:172.21.15.104:6789/0,b=v1:172.21.15.55:6789/0} 2020-09-08T15:28:43.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:43 smithi055 bash[9296]: cluster 2020-09-08T15:28:43.078692+0000 mon.a (mon.0) 204 : cluster [DBG] fsmap 2020-09-08T15:28:43.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:43 smithi055 bash[9296]: cluster 2020-09-08T15:28:43.078768+0000 mon.a (mon.0) 205 : cluster [DBG] osdmap e6: 0 total, 0 up, 0 in 2020-09-08T15:28:43.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:43 smithi055 bash[9296]: cluster 2020-09-08T15:28:43.079151+0000 mon.a (mon.0) 206 : cluster [DBG] mgrmap e13: y(active, since 2m) 2020-09-08T15:28:43.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:43 smithi055 bash[9296]: audit 2020-09-08T15:28:43.081905+0000 mon.a (mon.0) 207 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2020-09-08T15:28:43.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:43 smithi055 bash[9296]: audit 2020-09-08T15:28:43.085032+0000 mon.a (mon.0) 208 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2020-09-08T15:28:43.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:43 smithi055 bash[9296]: cluster 2020-09-08T15:28:43.090202+0000 mon.a (mon.0) 209 : cluster [WRN] overall HEALTH_WARN failed to probe daemons or devices; Reduced data availability: 1 pg inactive 2020-09-08T15:28:43.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:43 smithi055 bash[9296]: audit 2020-09-08T15:28:43.090813+0000 mon.a (mon.0) 210 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:28:43.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:43 smithi055 bash[9296]: audit 2020-09-08T15:28:43.095946+0000 mon.a (mon.0) 211 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:28:43.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:43 smithi055 bash[9296]: audit 2020-09-08T15:28:43.101780+0000 mon.a (mon.0) 212 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2020-09-08T15:28:43.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:43 smithi055 bash[9296]: audit 2020-09-08T15:28:43.104138+0000 mon.a (mon.0) 213 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:28:43.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:43 smithi055 bash[9296]: audit 2020-09-08T15:28:43.105607+0000 mon.a (mon.0) 214 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon.a", "key": "container_image"}]: dispatch 2020-09-08T15:28:43.434 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:43 smithi104 bash[9203]: cluster 2020-09-08T15:28:38.024125+0000 mon.a (mon.0) 200 : cluster [INF] mon.a calling monitor election 2020-09-08T15:28:43.436 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:43 smithi104 bash[9203]: audit 2020-09-08T15:28:39.018099+0000 mon.a (mon.0) 201 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2020-09-08T15:28:43.440 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:43 smithi104 bash[9203]: cluster 2020-09-08T15:28:40.020756+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2020-09-08T15:28:43.440 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:43 smithi104 bash[9203]: cluster 2020-09-08T15:28:41.066061+0000 mgr.y (mgr.14145) 73 : cluster [DBG] pgmap v60: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:43.440 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:43 smithi104 bash[9203]: cluster 2020-09-08T15:28:43.066490+0000 mgr.y (mgr.14145) 74 : cluster [DBG] pgmap v61: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:43.441 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:43 smithi104 bash[9203]: cluster 2020-09-08T15:28:43.067396+0000 mon.a (mon.0) 202 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2020-09-08T15:28:43.441 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:43 smithi104 bash[9203]: cluster 2020-09-08T15:28:43.078590+0000 mon.a (mon.0) 203 : cluster [DBG] monmap e2: 2 mons at {a=v1:172.21.15.104:6789/0,b=v1:172.21.15.55:6789/0} 2020-09-08T15:28:43.441 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:43 smithi104 bash[9203]: cluster 2020-09-08T15:28:43.078692+0000 mon.a (mon.0) 204 : cluster [DBG] fsmap 2020-09-08T15:28:43.441 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:43 smithi104 bash[9203]: cluster 2020-09-08T15:28:43.078768+0000 mon.a (mon.0) 205 : cluster [DBG] osdmap e6: 0 total, 0 up, 0 in 2020-09-08T15:28:43.442 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:43 smithi104 bash[9203]: cluster 2020-09-08T15:28:43.079151+0000 mon.a (mon.0) 206 : cluster [DBG] mgrmap e13: y(active, since 2m) 2020-09-08T15:28:43.442 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:43 smithi104 bash[9203]: audit 2020-09-08T15:28:43.081905+0000 mon.a (mon.0) 207 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2020-09-08T15:28:43.442 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:43 smithi104 bash[9203]: audit 2020-09-08T15:28:43.085032+0000 mon.a (mon.0) 208 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2020-09-08T15:28:43.442 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:43 smithi104 bash[9203]: cluster 2020-09-08T15:28:43.090202+0000 mon.a (mon.0) 209 : cluster [WRN] overall HEALTH_WARN failed to probe daemons or devices; Reduced data availability: 1 pg inactive 2020-09-08T15:28:43.443 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:43 smithi104 bash[9203]: audit 2020-09-08T15:28:43.090813+0000 mon.a (mon.0) 210 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:28:43.443 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:43 smithi104 bash[9203]: audit 2020-09-08T15:28:43.095946+0000 mon.a (mon.0) 211 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:28:43.443 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:43 smithi104 bash[9203]: audit 2020-09-08T15:28:43.101780+0000 mon.a (mon.0) 212 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2020-09-08T15:28:43.443 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:43 smithi104 bash[9203]: audit 2020-09-08T15:28:43.104138+0000 mon.a (mon.0) 213 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:28:43.443 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:43 smithi104 bash[9203]: audit 2020-09-08T15:28:43.105607+0000 mon.a (mon.0) 214 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon.a", "key": "container_image"}]: dispatch 2020-09-08T15:28:43.968 INFO:tasks.cephadm:Adding mon.c on smithi166 2020-09-08T15:28:43.969 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph orch daemon add mon 'smithi166:[v1:172.21.15.166:6789]=c' 2020-09-08T15:28:44.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:44 smithi055 bash[9296]: cluster 2020-09-08T15:28:43.133032+0000 mon.a (mon.0) 215 : cluster [INF] Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2020-09-08T15:28:44.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:44 smithi055 bash[9296]: audit 2020-09-08T15:28:43.352757+0000 mon.a (mon.0) 216 : audit [DBG] from='client.? v1:172.21.15.55:0/737208561' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2020-09-08T15:28:44.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:44 smithi055 bash[9296]: audit 2020-09-08T15:28:44.018608+0000 mon.a (mon.0) 217 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2020-09-08T15:28:44.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:44 smithi104 bash[9203]: cluster 2020-09-08T15:28:43.133032+0000 mon.a (mon.0) 215 : cluster [INF] Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2020-09-08T15:28:44.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:44 smithi104 bash[9203]: audit 2020-09-08T15:28:43.352757+0000 mon.a (mon.0) 216 : audit [DBG] from='client.? v1:172.21.15.55:0/737208561' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2020-09-08T15:28:44.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:44 smithi104 bash[9203]: audit 2020-09-08T15:28:44.018608+0000 mon.a (mon.0) 217 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2020-09-08T15:28:45.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:45 smithi104 bash[9203]: audit 2020-09-08T15:28:45.054380+0000 mon.a (mon.0) 218 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:27:21.767610\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:26:50.950427\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:28:45.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:45 smithi104 bash[9203]: audit 2020-09-08T15:28:45.060025+0000 mon.a (mon.0) 219 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:27:21.767610\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:26:50.950427\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:28:45.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:45 smithi104 bash[9203]: audit 2020-09-08T15:28:45.061050+0000 mon.a (mon.0) 220 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mgr.y"}]: dispatch 2020-09-08T15:28:45.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:45 smithi104 bash[9203]: audit 2020-09-08T15:28:45.062277+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:28:45.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:45 smithi104 bash[9203]: audit 2020-09-08T15:28:45.063837+0000 mon.a (mon.0) 222 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.y", "key": "container_image"}]: dispatch 2020-09-08T15:28:45.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:45 smithi055 bash[9296]: audit 2020-09-08T15:28:45.054380+0000 mon.a (mon.0) 218 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:27:21.767610\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:26:50.950427\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:28:45.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:45 smithi055 bash[9296]: audit 2020-09-08T15:28:45.060025+0000 mon.a (mon.0) 219 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:27:21.767610\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:26:50.950427\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:28:45.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:45 smithi055 bash[9296]: audit 2020-09-08T15:28:45.061050+0000 mon.a (mon.0) 220 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mgr.y"}]: dispatch 2020-09-08T15:28:45.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:45 smithi055 bash[9296]: audit 2020-09-08T15:28:45.062277+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:28:45.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:45 smithi055 bash[9296]: audit 2020-09-08T15:28:45.063837+0000 mon.a (mon.0) 222 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.y", "key": "container_image"}]: dispatch 2020-09-08T15:28:47.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:47 smithi055 bash[9296]: cephadm 2020-09-08T15:28:43.096462+0000 mgr.y (mgr.14145) 75 : cephadm [INF] Reconfiguring mon.a (monmap changed)... 2020-09-08T15:28:47.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:47 smithi055 bash[9296]: cephadm 2020-09-08T15:28:43.104925+0000 mgr.y (mgr.14145) 76 : cephadm [INF] Reconfiguring daemon mon.a on smithi104 2020-09-08T15:28:47.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:47 smithi055 bash[9296]: cephadm 2020-09-08T15:28:45.060619+0000 mgr.y (mgr.14145) 77 : cephadm [INF] Reconfiguring mgr.y (monmap changed)... 2020-09-08T15:28:47.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:47 smithi055 bash[9296]: cephadm 2020-09-08T15:28:45.063100+0000 mgr.y (mgr.14145) 78 : cephadm [INF] Reconfiguring daemon mgr.y on smithi104 2020-09-08T15:28:47.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:47 smithi055 bash[9296]: cluster 2020-09-08T15:28:45.066893+0000 mgr.y (mgr.14145) 79 : cluster [DBG] pgmap v62: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:47.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:47 smithi055 bash[9296]: audit 2020-09-08T15:28:46.950617+0000 mon.a (mon.0) 223 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:28:47.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:47 smithi055 bash[9296]: audit 2020-09-08T15:28:46.956422+0000 mon.a (mon.0) 224 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:28:47.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:47 smithi055 bash[9296]: audit 2020-09-08T15:28:46.957699+0000 mon.a (mon.0) 225 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2020-09-08T15:28:47.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:47 smithi055 bash[9296]: audit 2020-09-08T15:28:46.958478+0000 mon.a (mon.0) 226 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2020-09-08T15:28:47.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:47 smithi055 bash[9296]: audit 2020-09-08T15:28:46.958858+0000 mon.a (mon.0) 227 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:28:47.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:47 smithi055 bash[9296]: audit 2020-09-08T15:28:46.959793+0000 mon.a (mon.0) 228 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:28:47.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:47 smithi055 bash[9296]: audit 2020-09-08T15:28:46.960171+0000 mon.a (mon.0) 229 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon.b", "key": "container_image"}]: dispatch 2020-09-08T15:28:47.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:47 smithi055 bash[9296]: audit 2020-09-08T15:28:46.961877+0000 mon.a (mon.0) 230 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon.c", "key": "container_image"}]: dispatch 2020-09-08T15:28:47.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:47 smithi104 bash[9203]: cephadm 2020-09-08T15:28:43.096462+0000 mgr.y (mgr.14145) 75 : cephadm [INF] Reconfiguring mon.a (monmap changed)... 2020-09-08T15:28:47.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:47 smithi104 bash[9203]: cephadm 2020-09-08T15:28:43.104925+0000 mgr.y (mgr.14145) 76 : cephadm [INF] Reconfiguring daemon mon.a on smithi104 2020-09-08T15:28:47.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:47 smithi104 bash[9203]: cephadm 2020-09-08T15:28:45.060619+0000 mgr.y (mgr.14145) 77 : cephadm [INF] Reconfiguring mgr.y (monmap changed)... 2020-09-08T15:28:47.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:47 smithi104 bash[9203]: cephadm 2020-09-08T15:28:45.063100+0000 mgr.y (mgr.14145) 78 : cephadm [INF] Reconfiguring daemon mgr.y on smithi104 2020-09-08T15:28:47.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:47 smithi104 bash[9203]: cluster 2020-09-08T15:28:45.066893+0000 mgr.y (mgr.14145) 79 : cluster [DBG] pgmap v62: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:47.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:47 smithi104 bash[9203]: audit 2020-09-08T15:28:46.950617+0000 mon.a (mon.0) 223 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:28:47.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:47 smithi104 bash[9203]: audit 2020-09-08T15:28:46.956422+0000 mon.a (mon.0) 224 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:28:47.434 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:47 smithi104 bash[9203]: audit 2020-09-08T15:28:46.957699+0000 mon.a (mon.0) 225 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2020-09-08T15:28:47.434 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:47 smithi104 bash[9203]: audit 2020-09-08T15:28:46.958478+0000 mon.a (mon.0) 226 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2020-09-08T15:28:47.435 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:47 smithi104 bash[9203]: audit 2020-09-08T15:28:46.958858+0000 mon.a (mon.0) 227 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:28:47.435 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:47 smithi104 bash[9203]: audit 2020-09-08T15:28:46.959793+0000 mon.a (mon.0) 228 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:28:47.435 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:47 smithi104 bash[9203]: audit 2020-09-08T15:28:46.960171+0000 mon.a (mon.0) 229 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon.b", "key": "container_image"}]: dispatch 2020-09-08T15:28:47.435 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:47 smithi104 bash[9203]: audit 2020-09-08T15:28:46.961877+0000 mon.a (mon.0) 230 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon.c", "key": "container_image"}]: dispatch 2020-09-08T15:28:48.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:48 smithi055 bash[9296]: audit 2020-09-08T15:28:46.956851+0000 mgr.y (mgr.14145) 80 : audit [DBG] from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch daemon add", "daemon_type": "mon", "placement": "smithi166:[v1:172.21.15.166:6789]=c", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:28:48.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:48 smithi055 bash[9296]: cephadm 2020-09-08T15:28:46.957071+0000 mgr.y (mgr.14145) 81 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2020-09-08T15:28:48.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:48 smithi055 bash[9296]: cephadm 2020-09-08T15:28:46.959804+0000 mgr.y (mgr.14145) 82 : cephadm [INF] Reconfiguring daemon mon.b on smithi055 2020-09-08T15:28:48.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:48 smithi055 bash[9296]: cephadm 2020-09-08T15:28:46.960990+0000 mgr.y (mgr.14145) 83 : cephadm [INF] Deploying daemon mon.c on smithi166 2020-09-08T15:28:48.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:48 smithi055 bash[9296]: cluster 2020-09-08T15:28:47.067285+0000 mgr.y (mgr.14145) 84 : cluster [DBG] pgmap v63: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:48.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:48 smithi104 bash[9203]: audit 2020-09-08T15:28:46.956851+0000 mgr.y (mgr.14145) 80 : audit [DBG] from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch daemon add", "daemon_type": "mon", "placement": "smithi166:[v1:172.21.15.166:6789]=c", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:28:48.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:48 smithi104 bash[9203]: cephadm 2020-09-08T15:28:46.957071+0000 mgr.y (mgr.14145) 81 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2020-09-08T15:28:48.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:48 smithi104 bash[9203]: cephadm 2020-09-08T15:28:46.959804+0000 mgr.y (mgr.14145) 82 : cephadm [INF] Reconfiguring daemon mon.b on smithi055 2020-09-08T15:28:48.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:48 smithi104 bash[9203]: cephadm 2020-09-08T15:28:46.960990+0000 mgr.y (mgr.14145) 83 : cephadm [INF] Deploying daemon mon.c on smithi166 2020-09-08T15:28:48.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:48 smithi104 bash[9203]: cluster 2020-09-08T15:28:47.067285+0000 mgr.y (mgr.14145) 84 : cluster [DBG] pgmap v63: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:49.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:49 smithi104 bash[9203]: audit 2020-09-08T15:28:49.024355+0000 mon.a (mon.0) 231 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.957288\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:28:49.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:49 smithi104 bash[9203]: audit 2020-09-08T15:28:49.030407+0000 mon.a (mon.0) 232 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.957288\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:28:49.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:49 smithi104 bash[9203]: audit 2020-09-08T15:28:49.032693+0000 mon.a (mon.0) 233 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:28:49.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:49 smithi104 bash[9203]: audit 2020-09-08T15:28:49.033291+0000 mon.a (mon.0) 234 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:28:49.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:49 smithi055 bash[9296]: audit 2020-09-08T15:28:49.024355+0000 mon.a (mon.0) 231 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.957288\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:28:49.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:49 smithi055 bash[9296]: audit 2020-09-08T15:28:49.030407+0000 mon.a (mon.0) 232 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.957288\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:28:49.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:49 smithi055 bash[9296]: audit 2020-09-08T15:28:49.032693+0000 mon.a (mon.0) 233 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:28:49.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:49 smithi055 bash[9296]: audit 2020-09-08T15:28:49.033291+0000 mon.a (mon.0) 234 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:28:50.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:50 smithi055 bash[9296]: cluster 2020-09-08T15:28:49.067592+0000 mgr.y (mgr.14145) 85 : cluster [DBG] pgmap v64: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:50.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:50 smithi055 bash[9296]: audit 2020-09-08T15:28:49.612883+0000 mon.a (mon.0) 235 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:49.611779\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.957288\"}}, \"last_daemon_update\": \"2020-09-08T15:28:49.611986\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:28:50.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:50 smithi055 bash[9296]: audit 2020-09-08T15:28:49.617246+0000 mon.a (mon.0) 236 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:49.611779\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.957288\"}}, \"last_daemon_update\": \"2020-09-08T15:28:49.611986\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:28:50.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:50 smithi055 bash[9296]: audit 2020-09-08T15:28:49.665542+0000 mon.a (mon.0) 237 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:49.664236\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:49.664383\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:28:49.664577\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:28:50.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:50 smithi055 bash[9296]: audit 2020-09-08T15:28:49.672026+0000 mon.a (mon.0) 238 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:49.664236\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:49.664383\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:28:49.664577\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:28:50.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:50 smithi055 bash[9296]: audit 2020-09-08T15:28:49.673231+0000 mon.a (mon.0) 239 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:28:50.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:50 smithi055 bash[9296]: audit 2020-09-08T15:28:49.677242+0000 mon.a (mon.0) 240 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:28:50.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:50 smithi104 bash[9203]: cluster 2020-09-08T15:28:49.067592+0000 mgr.y (mgr.14145) 85 : cluster [DBG] pgmap v64: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:50.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:50 smithi104 bash[9203]: audit 2020-09-08T15:28:49.612883+0000 mon.a (mon.0) 235 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:49.611779\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.957288\"}}, \"last_daemon_update\": \"2020-09-08T15:28:49.611986\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:28:50.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:50 smithi104 bash[9203]: audit 2020-09-08T15:28:49.617246+0000 mon.a (mon.0) 236 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:49.611779\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.957288\"}}, \"last_daemon_update\": \"2020-09-08T15:28:49.611986\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:28:50.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:50 smithi104 bash[9203]: audit 2020-09-08T15:28:49.665542+0000 mon.a (mon.0) 237 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:49.664236\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:49.664383\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:28:49.664577\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:28:50.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:50 smithi104 bash[9203]: audit 2020-09-08T15:28:49.672026+0000 mon.a (mon.0) 238 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:49.664236\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:49.664383\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:28:49.664577\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:28:50.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:50 smithi104 bash[9203]: audit 2020-09-08T15:28:49.673231+0000 mon.a (mon.0) 239 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:28:50.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:50 smithi104 bash[9203]: audit 2020-09-08T15:28:49.677242+0000 mon.a (mon.0) 240 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:28:51.117 INFO:teuthology.orchestra.run.smithi166.stdout:Deployed mon.c on host 'smithi166' 2020-09-08T15:28:52.291 INFO:teuthology.orchestra.run.smithi166:mon.c> sudo journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mon.c.service 2020-09-08T15:28:52.299 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2020-09-08T15:28:52.300 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph mon dump -f json 2020-09-08T15:28:52.342 INFO:journalctl@ceph.mon.c.smithi166.stdout:-- Logs begin at Tue 2020-09-01 16:49:30 UTC. -- 2020-09-08T15:28:52.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:52 smithi055 bash[9296]: cluster 2020-09-08T15:28:51.068125+0000 mgr.y (mgr.14145) 86 : cluster [DBG] pgmap v65: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:52.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:52 smithi055 bash[9296]: audit 2020-09-08T15:28:51.115149+0000 mon.a (mon.0) 241 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.959478\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:28:52.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:52 smithi055 bash[9296]: audit 2020-09-08T15:28:51.116078+0000 mon.a (mon.0) 242 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:28:52.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:52 smithi055 bash[9296]: audit 2020-09-08T15:28:51.116650+0000 mon.a (mon.0) 243 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:28:52.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:52 smithi055 bash[9296]: audit 2020-09-08T15:28:51.120151+0000 mon.a (mon.0) 244 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.959478\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:28:52.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:52 smithi055 bash[9296]: audit 2020-09-08T15:28:51.748366+0000 mon.a (mon.0) 245 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:51.747163\", \"created\": \"2020-09-08T15:28:51.082052\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.959478\"}}, \"last_daemon_update\": \"2020-09-08T15:28:51.747350\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:28:52.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:52 smithi055 bash[9296]: audit 2020-09-08T15:28:51.753726+0000 mon.a (mon.0) 246 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:51.747163\", \"created\": \"2020-09-08T15:28:51.082052\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.959478\"}}, \"last_daemon_update\": \"2020-09-08T15:28:51.747350\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:28:52.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:52 smithi055 bash[9296]: audit 2020-09-08T15:28:51.792483+0000 mon.a (mon.0) 247 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:51.790729\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:51.790921\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:28:51.791153\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:28:52.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:52 smithi055 bash[9296]: audit 2020-09-08T15:28:51.798159+0000 mon.a (mon.0) 248 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:51.790729\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:51.790921\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:28:51.791153\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:28:52.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:52 smithi055 bash[9296]: audit 2020-09-08T15:28:51.799354+0000 mon.a (mon.0) 249 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:28:52.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:52 smithi055 bash[9296]: audit 2020-09-08T15:28:51.802646+0000 mon.a (mon.0) 250 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:28:52.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:52 smithi104 bash[9203]: cluster 2020-09-08T15:28:51.068125+0000 mgr.y (mgr.14145) 86 : cluster [DBG] pgmap v65: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:52.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:52 smithi104 bash[9203]: audit 2020-09-08T15:28:51.115149+0000 mon.a (mon.0) 241 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.959478\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:28:52.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:52 smithi104 bash[9203]: audit 2020-09-08T15:28:51.116078+0000 mon.a (mon.0) 242 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:28:52.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:52 smithi104 bash[9203]: audit 2020-09-08T15:28:51.116650+0000 mon.a (mon.0) 243 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:28:52.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:52 smithi104 bash[9203]: audit 2020-09-08T15:28:51.120151+0000 mon.a (mon.0) 244 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.959478\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:28:52.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:52 smithi104 bash[9203]: audit 2020-09-08T15:28:51.748366+0000 mon.a (mon.0) 245 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:51.747163\", \"created\": \"2020-09-08T15:28:51.082052\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.959478\"}}, \"last_daemon_update\": \"2020-09-08T15:28:51.747350\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:28:52.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:52 smithi104 bash[9203]: audit 2020-09-08T15:28:51.753726+0000 mon.a (mon.0) 246 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:51.747163\", \"created\": \"2020-09-08T15:28:51.082052\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.959478\"}}, \"last_daemon_update\": \"2020-09-08T15:28:51.747350\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:28:52.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:52 smithi104 bash[9203]: audit 2020-09-08T15:28:51.792483+0000 mon.a (mon.0) 247 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:51.790729\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:51.790921\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:28:51.791153\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:28:52.431 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:52 smithi104 bash[9203]: audit 2020-09-08T15:28:51.798159+0000 mon.a (mon.0) 248 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:51.790729\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:51.790921\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:28:51.791153\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:28:52.431 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:52 smithi104 bash[9203]: audit 2020-09-08T15:28:51.799354+0000 mon.a (mon.0) 249 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:28:52.431 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:52 smithi104 bash[9203]: audit 2020-09-08T15:28:51.802646+0000 mon.a (mon.0) 250 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:28:52.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.614+0000 7f01bdee5700 0 set uid:gid to 167:167 (ceph:ceph) 2020-09-08T15:28:52.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.614+0000 7f01bdee5700 0 ceph version 16.0.0-5272-gdce4d291bd4 (dce4d291bd4243c9802c17d67b85a6da6bbc0f8f) pacific (dev), process ceph-mon, pid 1 2020-09-08T15:28:52.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.614+0000 7f01bdee5700 0 pidfile_write: ignore empty --pid-file 2020-09-08T15:28:52.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.618+0000 7f01bdee5700 0 load: jerasure load: lrc load: isa 2020-09-08T15:28:52.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: RocksDB version: 6.8.1 2020-09-08T15:28:52.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2020-09-08T15:28:52.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Compile date Sep 8 2020 2020-09-08T15:28:52.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: DB SUMMARY 2020-09-08T15:28:52.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: CURRENT file: CURRENT 2020-09-08T15:28:52.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: IDENTITY file: IDENTITY 2020-09-08T15:28:52.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: MANIFEST file: MANIFEST-000001 size: 13 Bytes 2020-09-08T15:28:52.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2020-09-08T15:28:52.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000003.log size: 540 ; 2020-09-08T15:28:52.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.error_if_exists: 0 2020-09-08T15:28:52.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.create_if_missing: 0 2020-09-08T15:28:52.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.paranoid_checks: 1 2020-09-08T15:28:52.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.env: 0x55c10136ec00 2020-09-08T15:28:52.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.fs: Posix File System 2020-09-08T15:28:52.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.info_log: 0x55c102242ca0 2020-09-08T15:28:52.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_file_opening_threads: 16 2020-09-08T15:28:52.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.statistics: (nil) 2020-09-08T15:28:52.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.use_fsync: 0 2020-09-08T15:28:52.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_log_file_size: 0 2020-09-08T15:28:52.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2020-09-08T15:28:52.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.log_file_time_to_roll: 0 2020-09-08T15:28:52.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.keep_log_file_num: 1000 2020-09-08T15:28:52.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.recycle_log_file_num: 0 2020-09-08T15:28:52.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.allow_fallocate: 1 2020-09-08T15:28:52.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.allow_mmap_reads: 0 2020-09-08T15:28:52.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.allow_mmap_writes: 0 2020-09-08T15:28:52.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.use_direct_reads: 0 2020-09-08T15:28:52.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2020-09-08T15:28:52.872 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.create_missing_column_families: 0 2020-09-08T15:28:52.872 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.db_log_dir: 2020-09-08T15:28:52.872 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2020-09-08T15:28:52.872 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.table_cache_numshardbits: 6 2020-09-08T15:28:52.872 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_subcompactions: 1 2020-09-08T15:28:52.873 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_background_flushes: -1 2020-09-08T15:28:52.873 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.WAL_ttl_seconds: 0 2020-09-08T15:28:52.873 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.WAL_size_limit_MB: 0 2020-09-08T15:28:52.873 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2020-09-08T15:28:52.873 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2020-09-08T15:28:52.874 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.is_fd_close_on_exec: 1 2020-09-08T15:28:52.874 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.advise_random_on_open: 1 2020-09-08T15:28:52.874 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.db_write_buffer_size: 0 2020-09-08T15:28:52.874 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.write_buffer_manager: 0x55c10228b770 2020-09-08T15:28:52.874 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2020-09-08T15:28:52.875 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2020-09-08T15:28:52.875 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2020-09-08T15:28:52.875 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.use_adaptive_mutex: 0 2020-09-08T15:28:52.875 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.rate_limiter: (nil) 2020-09-08T15:28:52.876 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2020-09-08T15:28:52.876 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.wal_recovery_mode: 2 2020-09-08T15:28:52.876 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.enable_thread_tracking: 0 2020-09-08T15:28:52.879 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.enable_pipelined_write: 0 2020-09-08T15:28:52.880 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.unordered_write: 0 2020-09-08T15:28:52.880 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2020-09-08T15:28:52.880 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2020-09-08T15:28:52.881 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2020-09-08T15:28:52.881 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2020-09-08T15:28:52.881 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.row_cache: None 2020-09-08T15:28:52.881 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.wal_filter: None 2020-09-08T15:28:52.881 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2020-09-08T15:28:52.882 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.allow_ingest_behind: 0 2020-09-08T15:28:52.882 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.preserve_deletes: 0 2020-09-08T15:28:52.882 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.two_write_queues: 0 2020-09-08T15:28:52.882 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.manual_wal_flush: 0 2020-09-08T15:28:52.883 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.atomic_flush: 0 2020-09-08T15:28:52.883 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2020-09-08T15:28:52.883 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.persist_stats_to_disk: 0 2020-09-08T15:28:52.885 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.write_dbid_to_manifest: 0 2020-09-08T15:28:52.885 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.log_readahead_size: 0 2020-09-08T15:28:52.886 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2020-09-08T15:28:52.886 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_background_jobs: 2 2020-09-08T15:28:52.886 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_background_compactions: -1 2020-09-08T15:28:52.886 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2020-09-08T15:28:52.886 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2020-09-08T15:28:52.887 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.delayed_write_rate : 16777216 2020-09-08T15:28:52.887 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_total_wal_size: 0 2020-09-08T15:28:52.887 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2020-09-08T15:28:52.888 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.stats_dump_period_sec: 600 2020-09-08T15:28:52.888 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.stats_persist_period_sec: 600 2020-09-08T15:28:52.888 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2020-09-08T15:28:52.888 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_open_files: -1 2020-09-08T15:28:52.889 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.bytes_per_sync: 0 2020-09-08T15:28:52.889 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.wal_bytes_per_sync: 0 2020-09-08T15:28:52.889 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.strict_bytes_per_sync: 0 2020-09-08T15:28:52.889 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.compaction_readahead_size: 0 2020-09-08T15:28:52.890 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Compression algorithms supported: 2020-09-08T15:28:52.890 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2020-09-08T15:28:52.890 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: kZSTD supported: 0 2020-09-08T15:28:52.890 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: kXpressCompression supported: 0 2020-09-08T15:28:52.890 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: kLZ4HCCompression supported: 1 2020-09-08T15:28:52.891 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: kLZ4Compression supported: 1 2020-09-08T15:28:52.891 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: kBZip2Compression supported: 0 2020-09-08T15:28:52.891 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: kZlibCompression supported: 1 2020-09-08T15:28:52.891 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: kSnappyCompression supported: 1 2020-09-08T15:28:52.891 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Fast CRC32 supported: Supported on x86 2020-09-08T15:28:52.892 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000001 2020-09-08T15:28:52.892 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2020-09-08T15:28:52.892 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2020-09-08T15:28:52.892 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.merge_operator: 2020-09-08T15:28:52.892 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.compaction_filter: None 2020-09-08T15:28:52.893 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.compaction_filter_factory: None 2020-09-08T15:28:52.893 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.memtable_factory: SkipListFactory 2020-09-08T15:28:52.893 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.table_factory: BlockBasedTable 2020-09-08T15:28:52.893 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c1021e7a48) 2020-09-08T15:28:52.893 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: cache_index_and_filter_blocks: 1 2020-09-08T15:28:52.894 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: cache_index_and_filter_blocks_with_high_priority: 1 2020-09-08T15:28:52.894 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: pin_l0_filter_and_index_blocks_in_cache: 0 2020-09-08T15:28:52.894 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: pin_top_level_index_and_filter: 1 2020-09-08T15:28:52.894 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: index_type: 0 2020-09-08T15:28:52.894 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: data_block_index_type: 0 2020-09-08T15:28:52.895 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: index_shortening: 1 2020-09-08T15:28:52.895 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: data_block_hash_table_util_ratio: 0.750000 2020-09-08T15:28:52.895 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: hash_index_allow_collision: 1 2020-09-08T15:28:52.895 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: checksum: 1 2020-09-08T15:28:52.896 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: no_block_cache: 0 2020-09-08T15:28:52.896 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: block_cache: 0x55c10221e610 2020-09-08T15:28:52.896 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: block_cache_name: BinnedLRUCache 2020-09-08T15:28:52.896 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: block_cache_options: 2020-09-08T15:28:52.896 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: capacity : 536870912 2020-09-08T15:28:52.896 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: num_shard_bits : 4 2020-09-08T15:28:52.897 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: strict_capacity_limit : 0 2020-09-08T15:28:52.897 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: high_pri_pool_ratio: 0.000 2020-09-08T15:28:52.897 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: block_cache_compressed: (nil) 2020-09-08T15:28:52.897 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: persistent_cache: (nil) 2020-09-08T15:28:52.897 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: block_size: 4096 2020-09-08T15:28:52.898 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: block_size_deviation: 10 2020-09-08T15:28:52.898 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: block_restart_interval: 16 2020-09-08T15:28:52.898 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: index_block_restart_interval: 1 2020-09-08T15:28:52.898 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: metadata_block_size: 4096 2020-09-08T15:28:52.898 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: partition_filters: 0 2020-09-08T15:28:52.899 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: use_delta_encoding: 1 2020-09-08T15:28:52.899 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: filter_policy: rocksdb.BuiltinBloomFilter 2020-09-08T15:28:52.899 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: whole_key_filtering: 1 2020-09-08T15:28:52.899 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: verify_compression: 0 2020-09-08T15:28:52.899 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: read_amp_bytes_per_bit: 0 2020-09-08T15:28:52.900 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: format_version: 2 2020-09-08T15:28:52.900 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: enable_index_compression: 1 2020-09-08T15:28:52.900 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: block_align: 0 2020-09-08T15:28:52.900 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.write_buffer_size: 33554432 2020-09-08T15:28:52.900 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_write_buffer_number: 2 2020-09-08T15:28:52.901 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.compression: NoCompression 2020-09-08T15:28:52.901 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.bottommost_compression: Disabled 2020-09-08T15:28:52.901 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.prefix_extractor: nullptr 2020-09-08T15:28:52.901 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2020-09-08T15:28:52.902 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.num_levels: 7 2020-09-08T15:28:52.902 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2020-09-08T15:28:52.902 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2020-09-08T15:28:52.902 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2020-09-08T15:28:52.903 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2020-09-08T15:28:52.903 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2020-09-08T15:28:52.903 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2020-09-08T15:28:52.903 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2020-09-08T15:28:52.903 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2020-09-08T15:28:52.903 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2020-09-08T15:28:52.904 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.compression_opts.window_bits: -14 2020-09-08T15:28:52.904 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.compression_opts.level: 32767 2020-09-08T15:28:52.904 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.compression_opts.strategy: 0 2020-09-08T15:28:52.904 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2020-09-08T15:28:52.904 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2020-09-08T15:28:52.905 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.compression_opts.enabled: false 2020-09-08T15:28:52.905 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2020-09-08T15:28:52.905 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2020-09-08T15:28:52.905 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2020-09-08T15:28:52.906 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.target_file_size_base: 67108864 2020-09-08T15:28:52.906 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.target_file_size_multiplier: 1 2020-09-08T15:28:52.906 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2020-09-08T15:28:52.906 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2020-09-08T15:28:52.906 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2020-09-08T15:28:52.907 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2020-09-08T15:28:52.907 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2020-09-08T15:28:52.907 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2020-09-08T15:28:52.909 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2020-09-08T15:28:52.909 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2020-09-08T15:28:52.909 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2020-09-08T15:28:52.910 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2020-09-08T15:28:52.911 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2020-09-08T15:28:52.911 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2020-09-08T15:28:52.911 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.arena_block_size: 4194304 2020-09-08T15:28:52.911 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2020-09-08T15:28:52.911 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2020-09-08T15:28:52.912 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2020-09-08T15:28:52.912 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.disable_auto_compactions: 0 2020-09-08T15:28:52.912 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2020-09-08T15:28:52.912 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2020-09-08T15:28:52.913 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2020-09-08T15:28:52.913 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2020-09-08T15:28:52.913 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2020-09-08T15:28:52.913 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2020-09-08T15:28:52.915 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2020-09-08T15:28:52.916 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2020-09-08T15:28:52.916 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2020-09-08T15:28:52.916 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2020-09-08T15:28:52.916 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.table_properties_collectors: 2020-09-08T15:28:52.917 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.inplace_update_support: 0 2020-09-08T15:28:52.917 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.inplace_update_num_locks: 10000 2020-09-08T15:28:52.917 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2020-09-08T15:28:52.917 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2020-09-08T15:28:52.917 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.memtable_huge_page_size: 0 2020-09-08T15:28:52.918 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.bloom_locality: 0 2020-09-08T15:28:52.918 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.max_successive_merges: 0 2020-09-08T15:28:52.918 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.optimize_filters_for_hits: 0 2020-09-08T15:28:52.918 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.paranoid_file_checks: 0 2020-09-08T15:28:52.918 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.force_consistency_checks: 0 2020-09-08T15:28:52.919 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.report_bg_io_stats: 0 2020-09-08T15:28:52.919 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.ttl: 2592000 2020-09-08T15:28:52.919 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: Options.periodic_compaction_seconds: 0 2020-09-08T15:28:52.919 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2020-09-08T15:28:52.920 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2020-09-08T15:28:52.920 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578932627016, "job": 1, "event": "recovery_started", "log_files": [3]} 2020-09-08T15:28:52.920 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #3 mode 2 2020-09-08T15:28:52.920 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2020-09-08T15:28:52.921 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578932627782, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 4, "file_size": 1466, "table_properties": {"data_size": 552, "index_size": 40, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 430, "raw_average_value_size": 86, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1599578932, "oldest_key_time": 3, "file_creation_time": 0}} 2020-09-08T15:28:52.921 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.622+0000 7f01bdee5700 4 rocksdb: [version_set.cc:3826] Creating manifest 5 2020-09-08T15:28:52.921 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.626+0000 7f01bdee5700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578932628578, "job": 1, "event": "recovery_finished"} 2020-09-08T15:28:52.921 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.626+0000 7f01bdee5700 4 rocksdb: DB pointer 0x55c10226b800 2020-09-08T15:28:52.922 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.626+0000 7f01a4d24700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2020-09-08T15:28:52.922 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.626+0000 7f01a4d24700 4 rocksdb: [db_impl/db_impl.cc:851] 2020-09-08T15:28:52.922 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: ** DB Stats ** 2020-09-08T15:28:52.922 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Uptime(secs): 0.0 total, 0.0 interval 2020-09-08T15:28:52.922 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2020-09-08T15:28:52.923 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2020-09-08T15:28:52.923 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2020-09-08T15:28:52.923 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2020-09-08T15:28:52.923 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2020-09-08T15:28:52.923 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2020-09-08T15:28:52.924 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: ** Compaction Stats [default] ** 2020-09-08T15:28:52.924 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2020-09-08T15:28:52.924 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2020-09-08T15:28:52.924 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: L0 1/0 1.43 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 2020-09-08T15:28:52.924 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Sum 1/0 1.43 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 2020-09-08T15:28:52.925 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 2020-09-08T15:28:52.925 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: ** Compaction Stats [default] ** 2020-09-08T15:28:52.925 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2020-09-08T15:28:52.925 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2020-09-08T15:28:52.926 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.9 0.00 0.00 1 0.001 0 0 2020-09-08T15:28:52.934 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Uptime(secs): 0.0 total, 0.0 interval 2020-09-08T15:28:52.934 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Flush(GB): cumulative 0.000, interval 0.000 2020-09-08T15:28:52.934 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: AddFile(GB): cumulative 0.000, interval 0.000 2020-09-08T15:28:52.935 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: AddFile(Total Files): cumulative 0, interval 0 2020-09-08T15:28:52.935 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: AddFile(L0 Files): cumulative 0, interval 0 2020-09-08T15:28:52.935 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: AddFile(Keys): cumulative 0, interval 0 2020-09-08T15:28:52.935 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Cumulative compaction: 0.00 GB write, 0.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2020-09-08T15:28:52.935 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Interval compaction: 0.00 GB write, 0.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2020-09-08T15:28:52.936 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2020-09-08T15:28:52.936 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: ** File Read Latency Histogram By Level [default] ** 2020-09-08T15:28:52.936 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: ** Compaction Stats [default] ** 2020-09-08T15:28:52.936 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2020-09-08T15:28:52.936 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2020-09-08T15:28:52.937 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: L0 1/0 1.43 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 2020-09-08T15:28:52.937 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Sum 1/0 1.43 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 2020-09-08T15:28:52.937 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2020-09-08T15:28:52.937 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: ** Compaction Stats [default] ** 2020-09-08T15:28:52.937 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2020-09-08T15:28:52.938 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2020-09-08T15:28:52.938 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.9 0.00 0.00 1 0.001 0 0 2020-09-08T15:28:52.938 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Uptime(secs): 0.0 total, 0.0 interval 2020-09-08T15:28:52.938 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Flush(GB): cumulative 0.000, interval 0.000 2020-09-08T15:28:52.938 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: AddFile(GB): cumulative 0.000, interval 0.000 2020-09-08T15:28:52.939 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: AddFile(Total Files): cumulative 0, interval 0 2020-09-08T15:28:52.939 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: AddFile(L0 Files): cumulative 0, interval 0 2020-09-08T15:28:52.939 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: AddFile(Keys): cumulative 0, interval 0 2020-09-08T15:28:52.939 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Cumulative compaction: 0.00 GB write, 0.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2020-09-08T15:28:52.939 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2020-09-08T15:28:52.940 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2020-09-08T15:28:52.940 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: ** File Read Latency Histogram By Level [default] ** 2020-09-08T15:28:52.940 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.626+0000 7f01bdee5700 0 mon.c does not exist in monmap, will attempt to join an existing cluster 2020-09-08T15:28:52.940 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.626+0000 7f01bdee5700 0 using public_addrv v1:172.21.15.166:6789/0 2020-09-08T15:28:52.940 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.630+0000 7f01bdee5700 0 starting mon.c rank -1 at public addrs v1:172.21.15.166:6789/0 at bind addrs v1:172.21.15.166:6789/0 mon_data /var/lib/ceph/mon/ceph-c fsid 759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:28:52.941 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.630+0000 7f01bdee5700 1 mon.c@-1(???) e0 preinit fsid 759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:28:52.941 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.674+0000 7f01a2d20700 4 rocksdb: [db_impl/db_impl_write.cc:1668] [default] New memtable created with log file: #9. Immutable memtables: 0. 2020-09-08T15:28:52.941 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.674+0000 7f01acd34700 4 rocksdb: (Original Log Time 2020/09/08-15:28:52.677871) [db_impl/db_impl_compaction_flush.cc:2198] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 2020-09-08T15:28:52.941 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.674+0000 7f01acd34700 4 rocksdb: [flush_job.cc:321] [default] [JOB 3] Flushing memtable with next log file: 9 2020-09-08T15:28:52.941 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.674+0000 7f01acd34700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578932677966, "job": 3, "event": "flush_started", "num_memtables": 1, "num_entries": 973, "num_deletes": 11, "total_data_size": 6428002, "memory_usage": 6475200, "flush_reason": "Manual Compaction"} 2020-09-08T15:28:52.942 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.674+0000 7f01acd34700 4 rocksdb: [flush_job.cc:350] [default] [JOB 3] Level-0 flush table #10: started 2020-09-08T15:28:52.942 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.686+0000 7f01acd34700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578932691799, "cf_name": "default", "job": 3, "event": "table_file_creation", "file_number": 10, "file_size": 6235535, "table_properties": {"data_size": 6227431, "index_size": 5780, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 1477, "raw_key_size": 18468, "raw_average_key_size": 31, "raw_value_size": 6212230, "raw_average_value_size": 10601, "num_data_blocks": 186, "num_entries": 586, "num_deletions": 11, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1599578932, "oldest_key_time": 1599578932, "file_creation_time": 1599578932}} 2020-09-08T15:28:52.942 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.686+0000 7f01acd34700 4 rocksdb: [flush_job.cc:401] [default] [JOB 3] Level-0 flush table #10: 6235535 bytes OK 2020-09-08T15:28:52.942 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.690+0000 7f01acd34700 4 rocksdb: (Original Log Time 2020/09/08-15:28:52.691872) [memtable_list.cc:447] [default] Level-0 commit table #10 started 2020-09-08T15:28:52.943 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.690+0000 7f01acd34700 4 rocksdb: (Original Log Time 2020/09/08-15:28:52.692048) [memtable_list.cc:503] [default] Level-0 commit table #10: memtable #1 done 2020-09-08T15:28:52.943 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.690+0000 7f01acd34700 4 rocksdb: (Original Log Time 2020/09/08-15:28:52.692070) EVENT_LOG_v1 {"time_micros": 1599578932692063, "job": 3, "event": "flush_finished", "output_compression": "NoCompression", "lsm_state": [2, 0, 0, 0, 0, 0, 0], "immutable_memtables": 0} 2020-09-08T15:28:52.943 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.690+0000 7f01acd34700 4 rocksdb: (Original Log Time 2020/09/08-15:28:52.692092) [db_impl/db_impl_compaction_flush.cc:205] [default] Level summary: files[2 0 0 0 0 0 0] max score 0.50 2020-09-08T15:28:52.943 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.690+0000 7f01acd34700 4 rocksdb: [db_impl/db_impl_files.cc:353] [JOB 3] Try to delete WAL files size 6421252, prev total WAL file size 7510983, number of live WAL files 2. 2020-09-08T15:28:52.943 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.690+0000 7f01a2d20700 4 rocksdb: [db_impl/db_impl_compaction_flush.cc:1443] [default] Manual compaction starting 2020-09-08T15:28:52.944 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.690+0000 7f01ad535700 4 rocksdb: (Original Log Time 2020/09/08-15:28:52.694199) [db_impl/db_impl_compaction_flush.cc:2516] [default] Manual compaction from level-0 to level-6 from 'mgrstat .. 'mgrstat; will stop at (end) 2020-09-08T15:28:52.944 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.690+0000 7f01ad535700 4 rocksdb: [compaction/compaction_job.cc:1676] [default] [JOB 4] Compacting 2@0 files to L6, score -1.00 2020-09-08T15:28:52.944 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.690+0000 7f01ad535700 4 rocksdb: [compaction/compaction_job.cc:1680] [default] Compaction start summary: Base version 3 Base level 0, inputs: [10(6089KB) 4(1466B)] 2020-09-08T15:28:52.944 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.690+0000 7f01ad535700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578932694293, "job": 4, "event": "compaction_started", "compaction_reason": "ManualCompaction", "files_L0": [10, 4], "score": -1, "input_data_size": 6237001} 2020-09-08T15:28:52.944 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.706+0000 7f01ad535700 4 rocksdb: [compaction/compaction_job.cc:1349] [default] [JOB 4] Generated table #11: 580 keys, 6235868 bytes 2020-09-08T15:28:52.945 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.706+0000 7f01ad535700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578932711991, "cf_name": "default", "job": 4, "event": "table_file_creation", "file_number": 11, "file_size": 6235868, "table_properties": {"data_size": 6227770, "index_size": 5778, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 1477, "raw_key_size": 18394, "raw_average_key_size": 31, "raw_value_size": 6212660, "raw_average_value_size": 10711, "num_data_blocks": 186, "num_entries": 580, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1599578932, "oldest_key_time": 0, "file_creation_time": 1599578932}} 2020-09-08T15:28:52.945 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.710+0000 7f01ad535700 4 rocksdb: [compaction/compaction_job.cc:1415] [default] [JOB 4] Compacted 2@0 files to L6 => 6235868 bytes 2020-09-08T15:28:52.945 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.710+0000 7f01ad535700 4 rocksdb: [version_set.cc:3291] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2020-09-08T15:28:52.945 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.710+0000 7f01ad535700 4 rocksdb: (Original Log Time 2020/09/08-15:28:52.712590) [compaction/compaction_job.cc:760] [default] compacted to: base level 6 level multiplier 10.00 max bytes base 268435456 files[0 0 0 0 0 0 1] max score 0.00, MB/sec: 350.9 rd, 350.8 wr, level 6, files in(2, 0) out(1) MB in(5.9, 0.0) out(5.9), read-write-amplify(2.0) write-amplify(1.0) OK, records in: 591, records dropped: 11 output_compression: NoCompression 2020-09-08T15:28:52.945 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.710+0000 7f01ad535700 4 rocksdb: (Original Log Time 2020/09/08-15:28:52.712620) EVENT_LOG_v1 {"time_micros": 1599578932712609, "job": 4, "event": "compaction_finished", "compaction_time_micros": 17776, "compaction_time_cpu_micros": 12167, "output_level": 6, "num_output_files": 1, "total_output_size": 6235868, "num_input_records": 591, "num_output_records": 580, "num_subcompactions": 1, "output_compression": "NoCompression", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "lsm_state": [0, 0, 0, 0, 0, 0, 1]} 2020-09-08T15:28:52.946 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.710+0000 7f01ad535700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578932714078, "job": 4, "event": "table_file_deletion", "file_number": 10} 2020-09-08T15:28:52.946 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.710+0000 7f01ad535700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578932714230, "job": 4, "event": "table_file_deletion", "file_number": 4} 2020-09-08T15:28:52.946 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.710+0000 7f01a2d20700 4 rocksdb: [db_impl/db_impl_write.cc:1668] [default] New memtable created with log file: #12. Immutable memtables: 0. 2020-09-08T15:28:52.946 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.710+0000 7f01acd34700 4 rocksdb: (Original Log Time 2020/09/08-15:28:52.714561) [db_impl/db_impl_compaction_flush.cc:2198] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 2020-09-08T15:28:52.947 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.710+0000 7f01acd34700 4 rocksdb: [flush_job.cc:321] [default] [JOB 5] Flushing memtable with next log file: 12 2020-09-08T15:28:52.947 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.710+0000 7f01acd34700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578932714591, "job": 5, "event": "flush_started", "num_memtables": 1, "num_entries": 337, "num_deletes": 31, "total_data_size": 4283612, "memory_usage": 4338672, "flush_reason": "Manual Compaction"} 2020-09-08T15:28:52.947 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.710+0000 7f01acd34700 4 rocksdb: [flush_job.cc:350] [default] [JOB 5] Level-0 flush table #13: started 2020-09-08T15:28:52.947 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.718+0000 7f01acd34700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578932721977, "cf_name": "default", "job": 5, "event": "table_file_creation", "file_number": 13, "file_size": 4234588, "table_properties": {"data_size": 4231446, "index_size": 1844, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 453, "raw_key_size": 3150, "raw_average_key_size": 18, "raw_value_size": 4227693, "raw_average_value_size": 24723, "num_data_blocks": 58, "num_entries": 171, "num_deletions": 31, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1599578932, "oldest_key_time": 1599578932, "file_creation_time": 1599578932}} 2020-09-08T15:28:52.947 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.718+0000 7f01acd34700 4 rocksdb: [flush_job.cc:401] [default] [JOB 5] Level-0 flush table #13: 4234588 bytes OK 2020-09-08T15:28:52.948 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.718+0000 7f01acd34700 4 rocksdb: [version_set.cc:3291] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2020-09-08T15:28:52.948 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.718+0000 7f01acd34700 4 rocksdb: (Original Log Time 2020/09/08-15:28:52.722050) [memtable_list.cc:447] [default] Level-0 commit table #13 started 2020-09-08T15:28:52.948 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.718+0000 7f01acd34700 4 rocksdb: (Original Log Time 2020/09/08-15:28:52.722258) [memtable_list.cc:503] [default] Level-0 commit table #13: memtable #1 done 2020-09-08T15:28:52.948 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.718+0000 7f01acd34700 4 rocksdb: (Original Log Time 2020/09/08-15:28:52.722272) EVENT_LOG_v1 {"time_micros": 1599578932722266, "job": 5, "event": "flush_finished", "output_compression": "NoCompression", "lsm_state": [1, 0, 0, 0, 0, 0, 1], "immutable_memtables": 0} 2020-09-08T15:28:52.948 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.718+0000 7f01acd34700 4 rocksdb: (Original Log Time 2020/09/08-15:28:52.722289) [db_impl/db_impl_compaction_flush.cc:205] [default] Level summary: base level 6 level multiplier 10.00 max bytes base 268435456 files[1 0 0 0 0 0 1] max score 0.25 2020-09-08T15:28:52.949 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.718+0000 7f01acd34700 4 rocksdb: [db_impl/db_impl_files.cc:353] [JOB 5] Try to delete WAL files size 4281270, prev total WAL file size 6416626, number of live WAL files 2. 2020-09-08T15:28:52.949 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.718+0000 7f01a2d20700 4 rocksdb: [db_impl/db_impl_compaction_flush.cc:1443] [default] Manual compaction starting 2020-09-08T15:28:52.949 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.718+0000 7f01ad535700 4 rocksdb: (Original Log Time 2020/09/08-15:28:52.723001) [db_impl/db_impl_compaction_flush.cc:2516] [default] Manual compaction from level-0 to level-6 from 'mgrstat .. 'mgrstat; will stop at (end) 2020-09-08T15:28:52.949 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.718+0000 7f01ad535700 4 rocksdb: [compaction/compaction_job.cc:1676] [default] [JOB 6] Compacting 1@0 + 1@6 files to L6, score -1.00 2020-09-08T15:28:52.949 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.718+0000 7f01ad535700 4 rocksdb: [compaction/compaction_job.cc:1680] [default] Compaction start summary: Base version 5 Base level 0, inputs: [13(4135KB)], [11(6089KB)] 2020-09-08T15:28:52.950 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.718+0000 7f01ad535700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1599578932723047, "job": 6, "event": "compaction_started", "compaction_reason": "ManualCompaction", "files_L0": [13], "files_L6": [11], "score": -1, "input_data_size": 10470456} 2020-09-08T15:28:52.950 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.722+0000 7f01a7529700 0 mon.c@-1(synchronizing).mds e1 new map 2020-09-08T15:28:52.950 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.722+0000 7f01a7529700 0 mon.c@-1(synchronizing).mds e1 print_map 2020-09-08T15:28:52.950 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: e1 2020-09-08T15:28:52.950 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: enable_multiple, ever_enabled_multiple: 0,0 2020-09-08T15:28:52.951 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2020-09-08T15:28:52.951 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: legacy client fscid: -1 2020-09-08T15:28:52.951 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: 2020-09-08T15:28:52.951 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: No filesystems configured 2020-09-08T15:28:52.951 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.726+0000 7f01a7529700 1 mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2020-09-08T15:28:52.952 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.726+0000 7f01a7529700 1 mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2020-09-08T15:28:52.952 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.726+0000 7f01a7529700 1 mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2020-09-08T15:28:52.952 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.726+0000 7f01a7529700 1 mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2020-09-08T15:28:52.952 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.726+0000 7f01a7529700 1 mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2020-09-08T15:28:52.952 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.726+0000 7f01a7529700 1 mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2020-09-08T15:28:52.953 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.726+0000 7f01a7529700 1 mon.c@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2020-09-08T15:28:52.953 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.726+0000 7f01a7529700 1 mon.c@-1(synchronizing).osd e6 e6: 0 total, 0 up, 0 in 2020-09-08T15:28:52.953 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.726+0000 7f01a7529700 0 mon.c@-1(synchronizing).osd e6 crush map has features 288514051259236352, adjusting msgr requires 2020-09-08T15:28:52.953 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.726+0000 7f01a7529700 0 mon.c@-1(synchronizing).osd e6 crush map has features 3314933000852226048, adjusting msgr requires 2020-09-08T15:28:52.953 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.726+0000 7f01a7529700 0 mon.c@-1(synchronizing).osd e6 crush map has features 288514051259236352, adjusting msgr requires 2020-09-08T15:28:52.954 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: cluster 2020-09-08T15:28:51.068125+0000 mgr.y (mgr.14145) 86 : cluster [DBG] pgmap v65: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:52.954 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: audit 2020-09-08T15:28:51.115149+0000 mon.a (mon.0) 241 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.959478\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:28:52.954 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: audit 2020-09-08T15:28:51.116078+0000 mon.a (mon.0) 242 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:28:52.954 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: audit 2020-09-08T15:28:51.116650+0000 mon.a (mon.0) 243 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:28:52.955 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: audit 2020-09-08T15:28:51.120151+0000 mon.a (mon.0) 244 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.959478\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:28:52.955 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: audit 2020-09-08T15:28:51.748366+0000 mon.a (mon.0) 245 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:51.747163\", \"created\": \"2020-09-08T15:28:51.082052\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.959478\"}}, \"last_daemon_update\": \"2020-09-08T15:28:51.747350\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:28:52.955 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: audit 2020-09-08T15:28:51.753726+0000 mon.a (mon.0) 246 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:51.747163\", \"created\": \"2020-09-08T15:28:51.082052\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.959478\"}}, \"last_daemon_update\": \"2020-09-08T15:28:51.747350\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:28:52.955 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: audit 2020-09-08T15:28:51.792483+0000 mon.a (mon.0) 247 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:51.790729\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:51.790921\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:28:51.791153\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:28:52.956 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: audit 2020-09-08T15:28:51.798159+0000 mon.a (mon.0) 248 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:51.790729\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"mgr.y\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"y\", \"hostname\": \"smithi104\", \"container_id\": \"fde649239f10\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:51.790921\", \"created\": \"2020-09-08T15:25:49.759648\", \"started\": \"2020-09-08T15:25:49.853124\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:28:51.791153\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:28:52.956 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: audit 2020-09-08T15:28:51.799354+0000 mon.a (mon.0) 249 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:28:52.956 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: audit 2020-09-08T15:28:51.802646+0000 mon.a (mon.0) 250 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:28:52.956 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.726+0000 7f01a7529700 1 mon.c@-1(synchronizing).paxosservice(auth 1..5) refresh upgraded, format 0 -> 3 2020-09-08T15:28:52.956 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.730+0000 7f01a7529700 10 mon.c@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete,mon_osdmap_full_prune_interval,mon_osdmap_full_prune_min,mon_osdmap_full_prune_txsize 2020-09-08T15:28:52.957 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.730+0000 7f01a7529700 20 expand_channel_meta expand map: {default=false} 2020-09-08T15:28:52.957 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.730+0000 7f01a7529700 20 expand_channel_meta from 'false' to 'false' 2020-09-08T15:28:52.957 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.730+0000 7f01a7529700 20 expand_channel_meta expanded map: {default=false} 2020-09-08T15:28:52.957 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.730+0000 7f01a7529700 20 expand_channel_meta expand map: {default=info} 2020-09-08T15:28:52.958 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.730+0000 7f01a7529700 20 expand_channel_meta from 'info' to 'info' 2020-09-08T15:28:52.958 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.730+0000 7f01a7529700 20 expand_channel_meta expanded map: {default=info} 2020-09-08T15:28:52.959 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.730+0000 7f01a7529700 20 expand_channel_meta expand map: {default=daemon} 2020-09-08T15:28:52.959 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.730+0000 7f01a7529700 20 expand_channel_meta from 'daemon' to 'daemon' 2020-09-08T15:28:52.959 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.730+0000 7f01a7529700 20 expand_channel_meta expanded map: {default=daemon} 2020-09-08T15:28:52.959 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.730+0000 7f01a7529700 20 expand_channel_meta expand map: {default=debug} 2020-09-08T15:28:52.960 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.730+0000 7f01a7529700 20 expand_channel_meta from 'debug' to 'debug' 2020-09-08T15:28:52.960 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:52 smithi166 bash[9446]: debug 2020-09-08T15:28:52.730+0000 7f01a7529700 20 expand_channel_meta expanded map: {default=debug} 2020-09-08T15:28:58.036 INFO:teuthology.orchestra.run.smithi166.stdout: 2020-09-08T15:28:58.037 INFO:teuthology.orchestra.run.smithi166.stdout:{"epoch":3,"fsid":"759b27b6-f1e7-11ea-a080-001a4aab830c","modified":"2020-09-08T15:28:52.734017Z","created":"2020-09-08T15:25:37.526987Z","min_mon_release":16,"min_mon_release_name":"pacific","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6789","nonce":0}]},"addr":"172.21.15.104:6789/0","public_addr":"172.21.15.104:6789/0","priority":0,"weight":0},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6789","nonce":0}]},"addr":"172.21.15.166:6789/0","public_addr":"172.21.15.166:6789/0","priority":0,"weight":0}],"quorum":[0,1,2]} 2020-09-08T15:28:58.038 INFO:teuthology.orchestra.run.smithi166.stderr:dumped monmap epoch 3 2020-09-08T15:28:58.098 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:57 smithi166 bash[9446]: cluster 2020-09-08T15:28:52.740706+0000 mon.a (mon.0) 252 : cluster [INF] mon.a calling monitor election 2020-09-08T15:28:58.099 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:57 smithi166 bash[9446]: cluster 2020-09-08T15:28:52.746677+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2020-09-08T15:28:58.099 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:57 smithi166 bash[9446]: cluster 2020-09-08T15:28:53.068671+0000 mgr.y (mgr.14145) 87 : cluster [DBG] pgmap v66: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:58.099 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:57 smithi166 bash[9446]: audit 2020-09-08T15:28:53.733891+0000 mon.a (mon.0) 253 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2020-09-08T15:28:58.100 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:57 smithi166 bash[9446]: audit 2020-09-08T15:28:54.733728+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2020-09-08T15:28:58.100 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:57 smithi166 bash[9446]: cluster 2020-09-08T15:28:54.738778+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2020-09-08T15:28:58.100 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:57 smithi166 bash[9446]: cluster 2020-09-08T15:28:55.069145+0000 mgr.y (mgr.14145) 88 : cluster [DBG] pgmap v67: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:58.101 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:57 smithi166 bash[9446]: audit 2020-09-08T15:28:55.733827+0000 mon.a (mon.0) 255 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2020-09-08T15:28:58.101 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:57 smithi166 bash[9446]: audit 2020-09-08T15:28:56.733879+0000 mon.a (mon.0) 256 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2020-09-08T15:28:58.101 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:57 smithi166 bash[9446]: cluster 2020-09-08T15:28:57.069617+0000 mgr.y (mgr.14145) 89 : cluster [DBG] pgmap v68: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:58.101 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:57 smithi166 bash[9446]: audit 2020-09-08T15:28:57.733959+0000 mon.a (mon.0) 257 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2020-09-08T15:28:58.101 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:57 smithi166 bash[9446]: cluster 2020-09-08T15:28:57.763324+0000 mon.a (mon.0) 258 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2020-09-08T15:28:58.102 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:57 smithi166 bash[9446]: cluster 2020-09-08T15:28:57.769506+0000 mon.a (mon.0) 259 : cluster [DBG] monmap e3: 3 mons at {a=v1:172.21.15.104:6789/0,b=v1:172.21.15.55:6789/0,c=v1:172.21.15.166:6789/0} 2020-09-08T15:28:58.102 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:57 smithi166 bash[9446]: cluster 2020-09-08T15:28:57.769600+0000 mon.a (mon.0) 260 : cluster [DBG] fsmap 2020-09-08T15:28:58.103 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:57 smithi166 bash[9446]: cluster 2020-09-08T15:28:57.769655+0000 mon.a (mon.0) 261 : cluster [DBG] osdmap e6: 0 total, 0 up, 0 in 2020-09-08T15:28:58.103 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:57 smithi166 bash[9446]: cluster 2020-09-08T15:28:57.770038+0000 mon.a (mon.0) 262 : cluster [DBG] mgrmap e13: y(active, since 2m) 2020-09-08T15:28:58.103 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:57 smithi166 bash[9446]: cluster 2020-09-08T15:28:57.778074+0000 mon.a (mon.0) 263 : cluster [WRN] overall HEALTH_WARN Reduced data availability: 1 pg inactive 2020-09-08T15:28:58.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:57 smithi055 bash[9296]: cluster 2020-09-08T15:28:52.740706+0000 mon.a (mon.0) 252 : cluster [INF] mon.a calling monitor election 2020-09-08T15:28:58.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:57 smithi055 bash[9296]: cluster 2020-09-08T15:28:52.746677+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2020-09-08T15:28:58.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:57 smithi055 bash[9296]: cluster 2020-09-08T15:28:53.068671+0000 mgr.y (mgr.14145) 87 : cluster [DBG] pgmap v66: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:58.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:57 smithi055 bash[9296]: audit 2020-09-08T15:28:53.733891+0000 mon.a (mon.0) 253 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2020-09-08T15:28:58.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:57 smithi055 bash[9296]: audit 2020-09-08T15:28:54.733728+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2020-09-08T15:28:58.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:57 smithi055 bash[9296]: cluster 2020-09-08T15:28:54.738778+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2020-09-08T15:28:58.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:57 smithi055 bash[9296]: cluster 2020-09-08T15:28:55.069145+0000 mgr.y (mgr.14145) 88 : cluster [DBG] pgmap v67: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:58.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:57 smithi055 bash[9296]: audit 2020-09-08T15:28:55.733827+0000 mon.a (mon.0) 255 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2020-09-08T15:28:58.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:57 smithi055 bash[9296]: audit 2020-09-08T15:28:56.733879+0000 mon.a (mon.0) 256 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2020-09-08T15:28:58.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:57 smithi055 bash[9296]: cluster 2020-09-08T15:28:57.069617+0000 mgr.y (mgr.14145) 89 : cluster [DBG] pgmap v68: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:58.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:57 smithi055 bash[9296]: audit 2020-09-08T15:28:57.733959+0000 mon.a (mon.0) 257 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2020-09-08T15:28:58.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:57 smithi055 bash[9296]: cluster 2020-09-08T15:28:57.763324+0000 mon.a (mon.0) 258 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2020-09-08T15:28:58.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:57 smithi055 bash[9296]: cluster 2020-09-08T15:28:57.769506+0000 mon.a (mon.0) 259 : cluster [DBG] monmap e3: 3 mons at {a=v1:172.21.15.104:6789/0,b=v1:172.21.15.55:6789/0,c=v1:172.21.15.166:6789/0} 2020-09-08T15:28:58.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:57 smithi055 bash[9296]: cluster 2020-09-08T15:28:57.769600+0000 mon.a (mon.0) 260 : cluster [DBG] fsmap 2020-09-08T15:28:58.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:57 smithi055 bash[9296]: cluster 2020-09-08T15:28:57.769655+0000 mon.a (mon.0) 261 : cluster [DBG] osdmap e6: 0 total, 0 up, 0 in 2020-09-08T15:28:58.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:57 smithi055 bash[9296]: cluster 2020-09-08T15:28:57.770038+0000 mon.a (mon.0) 262 : cluster [DBG] mgrmap e13: y(active, since 2m) 2020-09-08T15:28:58.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:57 smithi055 bash[9296]: cluster 2020-09-08T15:28:57.778074+0000 mon.a (mon.0) 263 : cluster [WRN] overall HEALTH_WARN Reduced data availability: 1 pg inactive 2020-09-08T15:28:58.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:57 smithi104 bash[9203]: cluster 2020-09-08T15:28:52.740706+0000 mon.a (mon.0) 252 : cluster [INF] mon.a calling monitor election 2020-09-08T15:28:58.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:57 smithi104 bash[9203]: cluster 2020-09-08T15:28:52.746677+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2020-09-08T15:28:58.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:57 smithi104 bash[9203]: cluster 2020-09-08T15:28:53.068671+0000 mgr.y (mgr.14145) 87 : cluster [DBG] pgmap v66: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:58.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:57 smithi104 bash[9203]: audit 2020-09-08T15:28:53.733891+0000 mon.a (mon.0) 253 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2020-09-08T15:28:58.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:57 smithi104 bash[9203]: audit 2020-09-08T15:28:54.733728+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2020-09-08T15:28:58.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:57 smithi104 bash[9203]: cluster 2020-09-08T15:28:54.738778+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2020-09-08T15:28:58.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:57 smithi104 bash[9203]: cluster 2020-09-08T15:28:55.069145+0000 mgr.y (mgr.14145) 88 : cluster [DBG] pgmap v67: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:58.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:57 smithi104 bash[9203]: audit 2020-09-08T15:28:55.733827+0000 mon.a (mon.0) 255 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2020-09-08T15:28:58.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:57 smithi104 bash[9203]: audit 2020-09-08T15:28:56.733879+0000 mon.a (mon.0) 256 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2020-09-08T15:28:58.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:57 smithi104 bash[9203]: cluster 2020-09-08T15:28:57.069617+0000 mgr.y (mgr.14145) 89 : cluster [DBG] pgmap v68: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:28:58.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:57 smithi104 bash[9203]: audit 2020-09-08T15:28:57.733959+0000 mon.a (mon.0) 257 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2020-09-08T15:28:58.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:57 smithi104 bash[9203]: cluster 2020-09-08T15:28:57.763324+0000 mon.a (mon.0) 258 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2020-09-08T15:28:58.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:57 smithi104 bash[9203]: cluster 2020-09-08T15:28:57.769506+0000 mon.a (mon.0) 259 : cluster [DBG] monmap e3: 3 mons at {a=v1:172.21.15.104:6789/0,b=v1:172.21.15.55:6789/0,c=v1:172.21.15.166:6789/0} 2020-09-08T15:28:58.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:57 smithi104 bash[9203]: cluster 2020-09-08T15:28:57.769600+0000 mon.a (mon.0) 260 : cluster [DBG] fsmap 2020-09-08T15:28:58.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:57 smithi104 bash[9203]: cluster 2020-09-08T15:28:57.769655+0000 mon.a (mon.0) 261 : cluster [DBG] osdmap e6: 0 total, 0 up, 0 in 2020-09-08T15:28:58.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:57 smithi104 bash[9203]: cluster 2020-09-08T15:28:57.770038+0000 mon.a (mon.0) 262 : cluster [DBG] mgrmap e13: y(active, since 2m) 2020-09-08T15:28:58.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:57 smithi104 bash[9203]: cluster 2020-09-08T15:28:57.778074+0000 mon.a (mon.0) 263 : cluster [WRN] overall HEALTH_WARN Reduced data availability: 1 pg inactive 2020-09-08T15:28:58.664 INFO:tasks.cephadm:Generating final ceph.conf file... 2020-09-08T15:28:58.665 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph config generate-minimal-conf 2020-09-08T15:28:59.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:58 smithi166 bash[9446]: audit 2020-09-08T15:28:58.039689+0000 mon.a (mon.0) 264 : audit [DBG] from='client.? v1:172.21.15.166:0/1139368653' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2020-09-08T15:28:59.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:58 smithi166 bash[9446]: audit 2020-09-08T15:28:58.734263+0000 mon.a (mon.0) 265 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2020-09-08T15:28:59.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:58 smithi055 bash[9296]: audit 2020-09-08T15:28:58.039689+0000 mon.a (mon.0) 264 : audit [DBG] from='client.? v1:172.21.15.166:0/1139368653' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2020-09-08T15:28:59.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:58 smithi055 bash[9296]: audit 2020-09-08T15:28:58.734263+0000 mon.a (mon.0) 265 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2020-09-08T15:28:59.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:58 smithi104 bash[9203]: audit 2020-09-08T15:28:58.039689+0000 mon.a (mon.0) 264 : audit [DBG] from='client.? v1:172.21.15.166:0/1139368653' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2020-09-08T15:28:59.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:58 smithi104 bash[9203]: audit 2020-09-08T15:28:58.734263+0000 mon.a (mon.0) 265 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2020-09-08T15:29:00.026 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:28:59 smithi166 bash[9446]: cluster 2020-09-08T15:28:59.070096+0000 mgr.y (mgr.14145) 90 : cluster [DBG] pgmap v69: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:29:00.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:28:59 smithi055 bash[9296]: cluster 2020-09-08T15:28:59.070096+0000 mgr.y (mgr.14145) 90 : cluster [DBG] pgmap v69: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:29:00.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:28:59 smithi104 bash[9203]: cluster 2020-09-08T15:28:59.070096+0000 mgr.y (mgr.14145) 90 : cluster [DBG] pgmap v69: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:29:01.737 INFO:teuthology.orchestra.run.smithi166.stdout:# minimal ceph.conf for 759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:29:01.738 INFO:teuthology.orchestra.run.smithi166.stdout:[global] 2020-09-08T15:29:01.738 INFO:teuthology.orchestra.run.smithi166.stdout: fsid = 759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:29:01.738 INFO:teuthology.orchestra.run.smithi166.stdout: mon_host = 172.21.15.104:6789/0 172.21.15.55:6789/0 172.21.15.166:6789/0 2020-09-08T15:29:02.303 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2020-09-08T15:29:02.304 INFO:teuthology.orchestra.run.smithi104:> set -ex 2020-09-08T15:29:02.304 INFO:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.conf 2020-09-08T15:29:02.309 INFO:teuthology.orchestra.run.smithi104.stderr:+ sudo dd of=/etc/ceph/ceph.conf 2020-09-08T15:29:02.316 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records in 2020-09-08T15:29:02.316 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records out 2020-09-08T15:29:02.316 INFO:teuthology.orchestra.run.smithi104.stderr:189 bytes copied, 0.000140113 s, 1.3 MB/s 2020-09-08T15:29:02.317 INFO:teuthology.orchestra.run.smithi104:> set -ex 2020-09-08T15:29:02.318 INFO:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2020-09-08T15:29:02.362 INFO:teuthology.orchestra.run.smithi104.stderr:+ sudo dd of=/etc/ceph/ceph.client.admin.keyring 2020-09-08T15:29:02.371 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records in 2020-09-08T15:29:02.371 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records out 2020-09-08T15:29:02.372 INFO:teuthology.orchestra.run.smithi104.stderr:63 bytes copied, 5.338e-05 s, 1.2 MB/s 2020-09-08T15:29:02.373 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:02 smithi104 bash[9203]: cluster 2020-09-08T15:29:01.070613+0000 mgr.y (mgr.14145) 91 : cluster [DBG] pgmap v70: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:29:02.373 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:02 smithi104 bash[9203]: audit 2020-09-08T15:29:01.745060+0000 mon.c (mon.2) 2 : audit [DBG] from='client.? v1:172.21.15.166:0/1898023724' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:02.373 INFO:teuthology.orchestra.run.smithi055:> set -ex 2020-09-08T15:29:02.374 INFO:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/ceph/ceph.conf 2020-09-08T15:29:02.379 INFO:teuthology.orchestra.run.smithi055.stderr:+ sudo dd of=/etc/ceph/ceph.conf 2020-09-08T15:29:02.386 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:02 smithi055 bash[9296]: cluster 2020-09-08T15:29:01.070613+0000 mgr.y (mgr.14145) 91 : cluster [DBG] pgmap v70: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:29:02.387 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:02 smithi055 bash[9296]: audit 2020-09-08T15:29:01.745060+0000 mon.c (mon.2) 2 : audit [DBG] from='client.? v1:172.21.15.166:0/1898023724' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:02.389 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records in 2020-09-08T15:29:02.389 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records out 2020-09-08T15:29:02.389 INFO:teuthology.orchestra.run.smithi055.stderr:189 bytes copied, 0.000134891 s, 1.4 MB/s 2020-09-08T15:29:02.390 INFO:teuthology.orchestra.run.smithi055:> set -ex 2020-09-08T15:29:02.391 INFO:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2020-09-08T15:29:02.435 INFO:teuthology.orchestra.run.smithi055.stderr:+ sudo dd of=/etc/ceph/ceph.client.admin.keyring 2020-09-08T15:29:02.445 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records in 2020-09-08T15:29:02.445 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records out 2020-09-08T15:29:02.445 INFO:teuthology.orchestra.run.smithi055.stderr:63 bytes copied, 0.000116617 s, 540 kB/s 2020-09-08T15:29:02.446 INFO:teuthology.orchestra.run.smithi166:> set -ex 2020-09-08T15:29:02.448 INFO:teuthology.orchestra.run.smithi166:> sudo dd of=/etc/ceph/ceph.conf 2020-09-08T15:29:02.453 INFO:teuthology.orchestra.run.smithi166.stderr:+ sudo dd of=/etc/ceph/ceph.conf 2020-09-08T15:29:02.460 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:02 smithi166 bash[9446]: cluster 2020-09-08T15:29:01.070613+0000 mgr.y (mgr.14145) 91 : cluster [DBG] pgmap v70: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:29:02.460 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:02 smithi166 bash[9446]: audit 2020-09-08T15:29:01.745060+0000 mon.c (mon.2) 2 : audit [DBG] from='client.? v1:172.21.15.166:0/1898023724' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:02.462 INFO:teuthology.orchestra.run.smithi166.stderr:0+1 records in 2020-09-08T15:29:02.463 INFO:teuthology.orchestra.run.smithi166.stderr:0+1 records out 2020-09-08T15:29:02.463 INFO:teuthology.orchestra.run.smithi166.stderr:189 bytes copied, 0.000144336 s, 1.3 MB/s 2020-09-08T15:29:02.464 INFO:teuthology.orchestra.run.smithi166:> set -ex 2020-09-08T15:29:02.464 INFO:teuthology.orchestra.run.smithi166:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2020-09-08T15:29:02.511 INFO:teuthology.orchestra.run.smithi166.stderr:+ sudo dd of=/etc/ceph/ceph.client.admin.keyring 2020-09-08T15:29:02.520 INFO:teuthology.orchestra.run.smithi166.stderr:0+1 records in 2020-09-08T15:29:02.520 INFO:teuthology.orchestra.run.smithi166.stderr:0+1 records out 2020-09-08T15:29:02.521 INFO:teuthology.orchestra.run.smithi166.stderr:63 bytes copied, 0.000123343 s, 511 kB/s 2020-09-08T15:29:02.522 INFO:tasks.cephadm:Adding mgr.x on smithi055 2020-09-08T15:29:02.523 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph orch apply mgr '2;smithi055=x' 2020-09-08T15:29:04.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:04 smithi055 bash[9296]: cluster 2020-09-08T15:29:03.071138+0000 mgr.y (mgr.14145) 92 : cluster [DBG] pgmap v71: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:29:04.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:04 smithi104 bash[9203]: cluster 2020-09-08T15:29:03.071138+0000 mgr.y (mgr.14145) 92 : cluster [DBG] pgmap v71: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:29:04.614 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:04 smithi166 bash[9446]: cluster 2020-09-08T15:29:03.071138+0000 mgr.y (mgr.14145) 92 : cluster [DBG] pgmap v71: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:29:05.612 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:05 smithi166 bash[9446]: cluster 2020-09-08T15:29:05.071619+0000 mgr.y (mgr.14145) 93 : cluster [DBG] pgmap v72: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:29:05.725 INFO:teuthology.orchestra.run.smithi166.stdout:Scheduled mgr update... 2020-09-08T15:29:06.046 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:05 smithi104 bash[9203]: cluster 2020-09-08T15:29:05.071619+0000 mgr.y (mgr.14145) 93 : cluster [DBG] pgmap v72: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:29:06.047 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:05 smithi055 bash[9296]: cluster 2020-09-08T15:29:05.071619+0000 mgr.y (mgr.14145) 93 : cluster [DBG] pgmap v72: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:29:06.347 INFO:teuthology.orchestra.run.smithi055:mgr.x> sudo journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mgr.x.service 2020-09-08T15:29:06.353 INFO:tasks.cephadm:Deploying OSDs... 2020-09-08T15:29:06.354 INFO:teuthology.orchestra.run.smithi104:> set -ex 2020-09-08T15:29:06.354 INFO:teuthology.orchestra.run.smithi104:> dd if=/scratch_devs of=/dev/stdout 2020-09-08T15:29:06.359 INFO:teuthology.orchestra.run.smithi104.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-09-08T15:29:06.361 INFO:teuthology.orchestra.run.smithi104.stdout:/dev/vg_nvme/lv_1 2020-09-08T15:29:06.361 INFO:teuthology.orchestra.run.smithi104.stdout:/dev/vg_nvme/lv_2 2020-09-08T15:29:06.361 INFO:teuthology.orchestra.run.smithi104.stdout:/dev/vg_nvme/lv_3 2020-09-08T15:29:06.362 INFO:teuthology.orchestra.run.smithi104.stdout:/dev/vg_nvme/lv_4 2020-09-08T15:29:06.362 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records in 2020-09-08T15:29:06.362 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records out 2020-09-08T15:29:06.363 INFO:teuthology.orchestra.run.smithi104.stderr:72 bytes copied, 0.000170447 s, 422 kB/s 2020-09-08T15:29:06.363 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-09-08T15:29:06.363 INFO:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-09-08T15:29:06.408 INFO:journalctl@ceph.mgr.x.smithi055.stdout:-- Logs begin at Tue 2020-09-01 16:49:30 UTC. -- 2020-09-08T15:29:06.409 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-09-08T15:29:06.409 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-09-08T15:29:06.409 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 6h/6d Inode: 341 Links: 1 2020-09-08T15:29:06.410 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-09-08T15:29:06.410 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2020-09-08 15:23:12.721869432 +0000 2020-09-08T15:29:06.410 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2020-09-08 15:23:12.605927434 +0000 2020-09-08T15:29:06.410 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2020-09-08 15:23:12.605927434 +0000 2020-09-08T15:29:06.411 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: - 2020-09-08T15:29:06.418 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2020-09-08T15:29:06.418 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2020-09-08T15:29:06.418 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000309078 s, 1.7 MB/s 2020-09-08T15:29:06.421 INFO:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-09-08T15:29:06.469 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-09-08T15:29:06.469 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-09-08T15:29:06.470 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 6h/6d Inode: 348 Links: 1 2020-09-08T15:29:06.470 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-09-08T15:29:06.470 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2020-09-08 15:23:12.957751426 +0000 2020-09-08T15:29:06.470 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2020-09-08 15:23:12.849805429 +0000 2020-09-08T15:29:06.470 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2020-09-08 15:23:12.849805429 +0000 2020-09-08T15:29:06.471 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: - 2020-09-08T15:29:06.479 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2020-09-08T15:29:06.479 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2020-09-08T15:29:06.480 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000369128 s, 1.4 MB/s 2020-09-08T15:29:06.485 INFO:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-09-08T15:29:06.533 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-09-08T15:29:06.533 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-09-08T15:29:06.534 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 6h/6d Inode: 354 Links: 1 2020-09-08T15:29:06.534 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-09-08T15:29:06.534 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2020-09-08 15:23:13.089685424 +0000 2020-09-08T15:29:06.535 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2020-09-08 15:23:13.085687423 +0000 2020-09-08T15:29:06.535 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2020-09-08 15:23:13.085687423 +0000 2020-09-08T15:29:06.535 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: - 2020-09-08T15:29:06.542 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2020-09-08T15:29:06.543 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2020-09-08T15:29:06.543 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000411087 s, 1.2 MB/s 2020-09-08T15:29:06.547 INFO:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-09-08T15:29:06.596 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-09-08T15:29:06.597 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-09-08T15:29:06.597 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 6h/6d Inode: 360 Links: 1 2020-09-08T15:29:06.598 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-09-08T15:29:06.598 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2020-09-08 15:23:13.445507416 +0000 2020-09-08T15:29:06.599 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2020-09-08 15:23:13.345557418 +0000 2020-09-08T15:29:06.599 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2020-09-08 15:23:13.345557418 +0000 2020-09-08T15:29:06.599 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: - 2020-09-08T15:29:06.606 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2020-09-08T15:29:06.607 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2020-09-08T15:29:06.607 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000376459 s, 1.4 MB/s 2020-09-08T15:29:06.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:06 smithi166 bash[9446]: audit 2020-09-08T15:29:05.721338+0000 mgr.y (mgr.14145) 94 : audit [DBG] from='client.14202 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi055=x", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:29:06.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:06 smithi166 bash[9446]: cephadm 2020-09-08T15:29:05.722960+0000 mgr.y (mgr.14145) 95 : cephadm [INF] Saving service mgr spec with placement smithi055=x;count:2 2020-09-08T15:29:06.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:06 smithi166 bash[9446]: audit 2020-09-08T15:29:05.723639+0000 mon.a (mon.0) 266 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2020-09-08T15:29:05.722985\", \"spec\": {\"placement\": {\"count\": 2, \"hosts\": [\"smithi055=x\"]}, \"service_name\": \"mgr\", \"service_type\": \"mgr\"}}"}]: dispatch 2020-09-08T15:29:06.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:06 smithi166 bash[9446]: audit 2020-09-08T15:29:05.729486+0000 mon.a (mon.0) 267 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2020-09-08T15:29:05.722985\", \"spec\": {\"placement\": {\"count\": 2, \"hosts\": [\"smithi055=x\"]}, \"service_name\": \"mgr\", \"service_type\": \"mgr\"}}"}]': finished 2020-09-08T15:29:06.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:06 smithi166 bash[9446]: audit 2020-09-08T15:29:05.732802+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:29:06.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:06 smithi166 bash[9446]: audit 2020-09-08T15:29:05.737853+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:29:06.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:06 smithi166 bash[9446]: audit 2020-09-08T15:29:05.739778+0000 mon.a (mon.0) 270 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2020-09-08T15:29:06.620 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:06 smithi166 bash[9446]: audit 2020-09-08T15:29:05.747014+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2020-09-08T15:29:06.620 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:06 smithi166 bash[9446]: audit 2020-09-08T15:29:05.748397+0000 mon.a (mon.0) 272 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2020-09-08T15:29:06.620 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:06 smithi166 bash[9446]: audit 2020-09-08T15:29:05.750185+0000 mon.a (mon.0) 273 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:06.621 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:06 smithi166 bash[9446]: cephadm 2020-09-08T15:29:05.751119+0000 mgr.y (mgr.14145) 96 : cephadm [INF] Deploying daemon mgr.x on smithi055 2020-09-08T15:29:06.621 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:06 smithi166 bash[9446]: audit 2020-09-08T15:29:05.751989+0000 mon.a (mon.0) 274 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.x", "key": "container_image"}]: dispatch 2020-09-08T15:29:06.622 INFO:teuthology.orchestra.run.smithi055:> set -ex 2020-09-08T15:29:06.623 INFO:teuthology.orchestra.run.smithi055:> dd if=/scratch_devs of=/dev/stdout 2020-09-08T15:29:06.627 INFO:teuthology.orchestra.run.smithi055.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-09-08T15:29:06.628 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/vg_nvme/lv_1 2020-09-08T15:29:06.629 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/vg_nvme/lv_2 2020-09-08T15:29:06.629 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/vg_nvme/lv_3 2020-09-08T15:29:06.630 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/vg_nvme/lv_4 2020-09-08T15:29:06.630 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records in 2020-09-08T15:29:06.630 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records out 2020-09-08T15:29:06.631 INFO:teuthology.orchestra.run.smithi055.stderr:72 bytes copied, 6.5798e-05 s, 1.1 MB/s 2020-09-08T15:29:06.631 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-09-08T15:29:06.632 INFO:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-09-08T15:29:06.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:06 smithi055 bash[9296]: audit 2020-09-08T15:29:05.721338+0000 mgr.y (mgr.14145) 94 : audit [DBG] from='client.14202 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi055=x", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:29:06.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:06 smithi055 bash[9296]: cephadm 2020-09-08T15:29:05.722960+0000 mgr.y (mgr.14145) 95 : cephadm [INF] Saving service mgr spec with placement smithi055=x;count:2 2020-09-08T15:29:06.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:06 smithi055 bash[9296]: audit 2020-09-08T15:29:05.723639+0000 mon.a (mon.0) 266 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2020-09-08T15:29:05.722985\", \"spec\": {\"placement\": {\"count\": 2, \"hosts\": [\"smithi055=x\"]}, \"service_name\": \"mgr\", \"service_type\": \"mgr\"}}"}]: dispatch 2020-09-08T15:29:06.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:06 smithi055 bash[9296]: audit 2020-09-08T15:29:05.729486+0000 mon.a (mon.0) 267 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2020-09-08T15:29:05.722985\", \"spec\": {\"placement\": {\"count\": 2, \"hosts\": [\"smithi055=x\"]}, \"service_name\": \"mgr\", \"service_type\": \"mgr\"}}"}]': finished 2020-09-08T15:29:06.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:06 smithi055 bash[9296]: audit 2020-09-08T15:29:05.732802+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:29:06.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:06 smithi055 bash[9296]: audit 2020-09-08T15:29:05.737853+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:29:06.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:06 smithi055 bash[9296]: audit 2020-09-08T15:29:05.739778+0000 mon.a (mon.0) 270 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2020-09-08T15:29:06.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:06 smithi055 bash[9296]: audit 2020-09-08T15:29:05.747014+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2020-09-08T15:29:06.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:06 smithi055 bash[9296]: audit 2020-09-08T15:29:05.748397+0000 mon.a (mon.0) 272 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2020-09-08T15:29:06.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:06 smithi055 bash[9296]: audit 2020-09-08T15:29:05.750185+0000 mon.a (mon.0) 273 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:06.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:06 smithi055 bash[9296]: cephadm 2020-09-08T15:29:05.751119+0000 mgr.y (mgr.14145) 96 : cephadm [INF] Deploying daemon mgr.x on smithi055 2020-09-08T15:29:06.660 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:06 smithi055 bash[9296]: audit 2020-09-08T15:29:05.751989+0000 mon.a (mon.0) 274 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.x", "key": "container_image"}]: dispatch 2020-09-08T15:29:06.665 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-09-08T15:29:06.665 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-09-08T15:29:06.665 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 6h/6d Inode: 351 Links: 1 2020-09-08T15:29:06.666 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-09-08T15:29:06.666 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2020-09-08 15:23:10.480123849 +0000 2020-09-08T15:29:06.666 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2020-09-08 15:23:10.344055847 +0000 2020-09-08T15:29:06.666 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2020-09-08 15:23:10.344055847 +0000 2020-09-08T15:29:06.667 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: - 2020-09-08T15:29:06.674 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2020-09-08T15:29:06.675 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2020-09-08T15:29:06.675 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000542551 s, 944 kB/s 2020-09-08T15:29:06.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:06 smithi104 bash[9203]: audit 2020-09-08T15:29:05.721338+0000 mgr.y (mgr.14145) 94 : audit [DBG] from='client.14202 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi055=x", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:29:06.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:06 smithi104 bash[9203]: cephadm 2020-09-08T15:29:05.722960+0000 mgr.y (mgr.14145) 95 : cephadm [INF] Saving service mgr spec with placement smithi055=x;count:2 2020-09-08T15:29:06.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:06 smithi104 bash[9203]: audit 2020-09-08T15:29:05.723639+0000 mon.a (mon.0) 266 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2020-09-08T15:29:05.722985\", \"spec\": {\"placement\": {\"count\": 2, \"hosts\": [\"smithi055=x\"]}, \"service_name\": \"mgr\", \"service_type\": \"mgr\"}}"}]: dispatch 2020-09-08T15:29:06.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:06 smithi104 bash[9203]: audit 2020-09-08T15:29:05.729486+0000 mon.a (mon.0) 267 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/spec.mgr","val":"{\"created\": \"2020-09-08T15:29:05.722985\", \"spec\": {\"placement\": {\"count\": 2, \"hosts\": [\"smithi055=x\"]}, \"service_name\": \"mgr\", \"service_type\": \"mgr\"}}"}]': finished 2020-09-08T15:29:06.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:06 smithi104 bash[9203]: audit 2020-09-08T15:29:05.732802+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:29:06.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:06 smithi104 bash[9203]: audit 2020-09-08T15:29:05.737853+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:29:06.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:06 smithi104 bash[9203]: audit 2020-09-08T15:29:05.739778+0000 mon.a (mon.0) 270 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2020-09-08T15:29:06.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:06 smithi104 bash[9203]: audit 2020-09-08T15:29:05.747014+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2020-09-08T15:29:06.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:06 smithi104 bash[9203]: audit 2020-09-08T15:29:05.748397+0000 mon.a (mon.0) 272 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2020-09-08T15:29:06.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:06 smithi104 bash[9203]: audit 2020-09-08T15:29:05.750185+0000 mon.a (mon.0) 273 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:06.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:06 smithi104 bash[9203]: cephadm 2020-09-08T15:29:05.751119+0000 mgr.y (mgr.14145) 96 : cephadm [INF] Deploying daemon mgr.x on smithi055 2020-09-08T15:29:06.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:06 smithi104 bash[9203]: audit 2020-09-08T15:29:05.751989+0000 mon.a (mon.0) 274 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.x", "key": "container_image"}]: dispatch 2020-09-08T15:29:06.680 INFO:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-09-08T15:29:06.728 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-09-08T15:29:06.729 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-09-08T15:29:06.729 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 6h/6d Inode: 357 Links: 1 2020-09-08T15:29:06.729 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-09-08T15:29:06.730 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2020-09-08 15:23:10.768267857 +0000 2020-09-08T15:29:06.730 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2020-09-08 15:23:10.624195853 +0000 2020-09-08T15:29:06.730 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2020-09-08 15:23:10.624195853 +0000 2020-09-08T15:29:06.731 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: - 2020-09-08T15:29:06.737 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2020-09-08T15:29:06.737 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2020-09-08T15:29:06.737 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000421205 s, 1.2 MB/s 2020-09-08T15:29:06.739 INFO:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-09-08T15:29:06.789 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-09-08T15:29:06.790 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-09-08T15:29:06.790 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 6h/6d Inode: 363 Links: 1 2020-09-08T15:29:06.790 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-09-08T15:29:06.791 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2020-09-08 15:23:11.012389864 +0000 2020-09-08T15:29:06.791 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2020-09-08 15:23:10.904335860 +0000 2020-09-08T15:29:06.791 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2020-09-08 15:23:10.904335860 +0000 2020-09-08T15:29:06.791 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: - 2020-09-08T15:29:06.799 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2020-09-08T15:29:06.799 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2020-09-08T15:29:06.799 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000432051 s, 1.2 MB/s 2020-09-08T15:29:06.802 INFO:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-09-08T15:29:06.850 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-09-08T15:29:06.850 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-09-08T15:29:06.851 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 6h/6d Inode: 369 Links: 1 2020-09-08T15:29:06.851 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-09-08T15:29:06.851 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2020-09-08 15:23:11.288527870 +0000 2020-09-08T15:29:06.851 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2020-09-08 15:23:11.156461868 +0000 2020-09-08T15:29:06.852 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2020-09-08 15:23:11.156461868 +0000 2020-09-08T15:29:06.852 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: - 2020-09-08T15:29:06.859 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2020-09-08T15:29:06.860 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2020-09-08T15:29:06.860 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000449905 s, 1.1 MB/s 2020-09-08T15:29:06.863 INFO:teuthology.orchestra.run.smithi166:> set -ex 2020-09-08T15:29:06.863 INFO:teuthology.orchestra.run.smithi166:> dd if=/scratch_devs of=/dev/stdout 2020-09-08T15:29:06.869 INFO:teuthology.orchestra.run.smithi166.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-09-08T15:29:06.870 INFO:teuthology.orchestra.run.smithi166.stdout:/dev/vg_nvme/lv_1 2020-09-08T15:29:06.871 INFO:teuthology.orchestra.run.smithi166.stdout:/dev/vg_nvme/lv_2 2020-09-08T15:29:06.871 INFO:teuthology.orchestra.run.smithi166.stdout:/dev/vg_nvme/lv_3 2020-09-08T15:29:06.871 INFO:teuthology.orchestra.run.smithi166.stdout:/dev/vg_nvme/lv_4 2020-09-08T15:29:06.871 INFO:teuthology.orchestra.run.smithi166.stderr:0+1 records in 2020-09-08T15:29:06.871 INFO:teuthology.orchestra.run.smithi166.stderr:0+1 records out 2020-09-08T15:29:06.872 INFO:teuthology.orchestra.run.smithi166.stderr:72 bytes copied, 0.000172653 s, 417 kB/s 2020-09-08T15:29:06.872 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-09-08T15:29:06.872 INFO:teuthology.orchestra.run.smithi166:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-09-08T15:29:06.918 INFO:teuthology.orchestra.run.smithi166.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-09-08T15:29:06.918 INFO:teuthology.orchestra.run.smithi166.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-09-08T15:29:06.918 INFO:teuthology.orchestra.run.smithi166.stdout:Device: 6h/6d Inode: 341 Links: 1 2020-09-08T15:29:06.919 INFO:teuthology.orchestra.run.smithi166.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-09-08T15:29:06.919 INFO:teuthology.orchestra.run.smithi166.stdout:Access: 2020-09-08 15:23:13.226820744 +0000 2020-09-08T15:29:06.919 INFO:teuthology.orchestra.run.smithi166.stdout:Modify: 2020-09-08 15:23:13.102758746 +0000 2020-09-08T15:29:06.919 INFO:teuthology.orchestra.run.smithi166.stdout:Change: 2020-09-08 15:23:13.102758746 +0000 2020-09-08T15:29:06.920 INFO:teuthology.orchestra.run.smithi166.stdout: Birth: - 2020-09-08T15:29:06.928 INFO:teuthology.orchestra.run.smithi166.stderr:1+0 records in 2020-09-08T15:29:06.928 INFO:teuthology.orchestra.run.smithi166.stderr:1+0 records out 2020-09-08T15:29:06.928 INFO:teuthology.orchestra.run.smithi166.stderr:512 bytes copied, 0.000394356 s, 1.3 MB/s 2020-09-08T15:29:06.932 INFO:teuthology.orchestra.run.smithi166:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-09-08T15:29:06.981 INFO:teuthology.orchestra.run.smithi166.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-09-08T15:29:06.982 INFO:teuthology.orchestra.run.smithi166.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-09-08T15:29:06.982 INFO:teuthology.orchestra.run.smithi166.stdout:Device: 6h/6d Inode: 347 Links: 1 2020-09-08T15:29:06.982 INFO:teuthology.orchestra.run.smithi166.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-09-08T15:29:06.983 INFO:teuthology.orchestra.run.smithi166.stdout:Access: 2020-09-08 15:23:13.474944738 +0000 2020-09-08T15:29:06.983 INFO:teuthology.orchestra.run.smithi166.stdout:Modify: 2020-09-08 15:23:13.342878741 +0000 2020-09-08T15:29:06.983 INFO:teuthology.orchestra.run.smithi166.stdout:Change: 2020-09-08 15:23:13.342878741 +0000 2020-09-08T15:29:06.983 INFO:teuthology.orchestra.run.smithi166.stdout: Birth: - 2020-09-08T15:29:06.991 INFO:teuthology.orchestra.run.smithi166.stderr:1+0 records in 2020-09-08T15:29:06.992 INFO:teuthology.orchestra.run.smithi166.stderr:1+0 records out 2020-09-08T15:29:06.992 INFO:teuthology.orchestra.run.smithi166.stderr:512 bytes copied, 0.000352583 s, 1.5 MB/s 2020-09-08T15:29:06.996 INFO:teuthology.orchestra.run.smithi166:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-09-08T15:29:07.046 INFO:teuthology.orchestra.run.smithi166.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-09-08T15:29:07.047 INFO:teuthology.orchestra.run.smithi166.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-09-08T15:29:07.047 INFO:teuthology.orchestra.run.smithi166.stdout:Device: 6h/6d Inode: 353 Links: 1 2020-09-08T15:29:07.047 INFO:teuthology.orchestra.run.smithi166.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-09-08T15:29:07.048 INFO:teuthology.orchestra.run.smithi166.stdout:Access: 2020-09-08 15:23:13.747080732 +0000 2020-09-08T15:29:07.048 INFO:teuthology.orchestra.run.smithi166.stdout:Modify: 2020-09-08 15:23:13.619016735 +0000 2020-09-08T15:29:07.048 INFO:teuthology.orchestra.run.smithi166.stdout:Change: 2020-09-08 15:23:13.619016735 +0000 2020-09-08T15:29:07.049 INFO:teuthology.orchestra.run.smithi166.stdout: Birth: - 2020-09-08T15:29:07.057 INFO:teuthology.orchestra.run.smithi166.stderr:1+0 records in 2020-09-08T15:29:07.057 INFO:teuthology.orchestra.run.smithi166.stderr:1+0 records out 2020-09-08T15:29:07.058 INFO:teuthology.orchestra.run.smithi166.stderr:512 bytes copied, 0.00040606 s, 1.3 MB/s 2020-09-08T15:29:07.059 INFO:teuthology.orchestra.run.smithi166:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-09-08T15:29:07.113 INFO:teuthology.orchestra.run.smithi166.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-09-08T15:29:07.114 INFO:teuthology.orchestra.run.smithi166.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-09-08T15:29:07.114 INFO:teuthology.orchestra.run.smithi166.stdout:Device: 6h/6d Inode: 360 Links: 1 2020-09-08T15:29:07.114 INFO:teuthology.orchestra.run.smithi166.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-09-08T15:29:07.115 INFO:teuthology.orchestra.run.smithi166.stdout:Access: 2020-09-08 15:23:14.003208727 +0000 2020-09-08T15:29:07.115 INFO:teuthology.orchestra.run.smithi166.stdout:Modify: 2020-09-08 15:23:13.895154730 +0000 2020-09-08T15:29:07.115 INFO:teuthology.orchestra.run.smithi166.stdout:Change: 2020-09-08 15:23:13.895154730 +0000 2020-09-08T15:29:07.115 INFO:teuthology.orchestra.run.smithi166.stdout: Birth: - 2020-09-08T15:29:07.122 INFO:teuthology.orchestra.run.smithi166.stderr:1+0 records in 2020-09-08T15:29:07.123 INFO:teuthology.orchestra.run.smithi166.stderr:1+0 records out 2020-09-08T15:29:07.123 INFO:teuthology.orchestra.run.smithi166.stderr:512 bytes copied, 0.000305661 s, 1.7 MB/s 2020-09-08T15:29:07.127 INFO:tasks.cephadm:Deploying osd.0 on smithi104 with /dev/vg_nvme/lv_4... 2020-09-08T15:29:07.127 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2020-09-08T15:29:07.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:07 smithi055 bash[9296]: cluster 2020-09-08T15:29:07.072168+0000 mgr.y (mgr.14145) 97 : cluster [DBG] pgmap v73: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:29:07.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:07 smithi166 bash[9446]: cluster 2020-09-08T15:29:07.072168+0000 mgr.y (mgr.14145) 97 : cluster [DBG] pgmap v73: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:29:07.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:07 smithi104 bash[9203]: cluster 2020-09-08T15:29:07.072168+0000 mgr.y (mgr.14145) 97 : cluster [DBG] pgmap v73: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:29:08.190 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Sep 08 15:29:08 smithi055 systemd[1]: Starting Ceph mgr.x for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T15:29:08.444 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Sep 08 15:29:08 smithi055 docker[9959]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-mgr.x 2020-09-08T15:29:08.445 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Sep 08 15:29:08 smithi055 systemd[1]: Started Ceph mgr.x for 759b27b6-f1e7-11ea-a080-001a4aab830c. 2020-09-08T15:29:08.445 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Sep 08 15:29:08 smithi055 bash[9970]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-mgr.x 2020-09-08T15:29:08.531 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:29:08 smithi104 systemd[1]: Stopping Ceph mgr.y for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T15:29:08.531 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:08 smithi104 bash[9203]: audit 2020-09-08T15:29:08.221795+0000 mon.a (mon.0) 275 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:49.611779\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.957288\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:29:08.531 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:08 smithi104 bash[9203]: audit 2020-09-08T15:29:08.228039+0000 mon.a (mon.0) 276 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:49.611779\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.957288\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:29:08.532 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:08 smithi104 bash[9203]: cephadm 2020-09-08T15:29:08.228969+0000 mgr.y (mgr.14145) 98 : cephadm [INF] It is presumed safe to stop ['mgr.y'] 2020-09-08T15:29:08.532 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:08 smithi104 bash[9203]: cephadm 2020-09-08T15:29:08.229222+0000 mgr.y (mgr.14145) 99 : cephadm [INF] Removing daemon mgr.y from smithi104 2020-09-08T15:29:08.532 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:08 smithi104 bash[9203]: audit 2020-09-08T15:29:08.229709+0000 mon.a (mon.0) 277 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.y", "key": "container_image"}]: dispatch 2020-09-08T15:29:08.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:08 smithi166 bash[9446]: audit 2020-09-08T15:29:08.221795+0000 mon.a (mon.0) 275 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:49.611779\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.957288\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:29:08.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:08 smithi166 bash[9446]: audit 2020-09-08T15:29:08.228039+0000 mon.a (mon.0) 276 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:49.611779\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.957288\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:29:08.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:08 smithi166 bash[9446]: cephadm 2020-09-08T15:29:08.228969+0000 mgr.y (mgr.14145) 98 : cephadm [INF] It is presumed safe to stop ['mgr.y'] 2020-09-08T15:29:08.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:08 smithi166 bash[9446]: cephadm 2020-09-08T15:29:08.229222+0000 mgr.y (mgr.14145) 99 : cephadm [INF] Removing daemon mgr.y from smithi104 2020-09-08T15:29:08.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:08 smithi166 bash[9446]: audit 2020-09-08T15:29:08.229709+0000 mon.a (mon.0) 277 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.y", "key": "container_image"}]: dispatch 2020-09-08T15:29:08.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:08 smithi055 bash[9296]: audit 2020-09-08T15:29:08.221795+0000 mon.a (mon.0) 275 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:49.611779\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.957288\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:29:08.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:08 smithi055 bash[9296]: audit 2020-09-08T15:29:08.228039+0000 mon.a (mon.0) 276 : audit [INF] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:28:49.611779\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.957288\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:29:08.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:08 smithi055 bash[9296]: cephadm 2020-09-08T15:29:08.228969+0000 mgr.y (mgr.14145) 98 : cephadm [INF] It is presumed safe to stop ['mgr.y'] 2020-09-08T15:29:08.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:08 smithi055 bash[9296]: cephadm 2020-09-08T15:29:08.229222+0000 mgr.y (mgr.14145) 99 : cephadm [INF] Removing daemon mgr.y from smithi104 2020-09-08T15:29:08.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:08 smithi055 bash[9296]: audit 2020-09-08T15:29:08.229709+0000 mon.a (mon.0) 277 : audit [DBG] from='mgr.14145 v1:172.21.15.104:0/860232929' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mgr.y", "key": "container_image"}]: dispatch 2020-09-08T15:29:08.926 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:29:08 smithi104 bash[9475]: debug 2020-09-08T15:29:08.625+0000 7f6f13527700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:29:08.927 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:29:08 smithi104 bash[9475]: debug 2020-09-08T15:29:08.625+0000 7f6f13527700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2020-09-08T15:29:09.659 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Sep 08 15:29:09 smithi055 bash[9970]: debug 2020-09-08T15:29:09.518+0000 7fa0b999c700 1 -- 172.21.15.55:0/1495437136 <== mon.1 v1:172.21.15.55:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 4185153011 0 0) 0x560ce366c4e0 con 0x560ce36c6800 2020-09-08T15:29:09.927 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:29:09 smithi104 docker[13550]: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-mgr.y 2020-09-08T15:29:09.927 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 08 15:29:09 smithi104 systemd[1]: Stopped Ceph mgr.y for 759b27b6-f1e7-11ea-a080-001a4aab830c. 2020-09-08T15:29:10.948 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_4 2020-09-08T15:29:10.949 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2020-09-08T15:29:10.949 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2020-09-08T15:29:10.949 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2020-09-08T15:29:10.950 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0151742 s, 691 MB/s 2020-09-08T15:29:10.950 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2020-09-08T15:29:11.564 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph orch daemon add osd smithi104:vg_nvme/lv_4 2020-09-08T15:29:14.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:14 smithi055 bash[9296]: cluster 2020-09-08T15:29:14.332011+0000 mon.a (mon.0) 278 : cluster [DBG] Standby manager daemon x started 2020-09-08T15:29:14.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:14 smithi055 bash[9296]: audit 2020-09-08T15:29:14.336160+0000 mon.b (mon.1) 3 : audit [DBG] from='mgr.? v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2020-09-08T15:29:14.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:14 smithi055 bash[9296]: audit 2020-09-08T15:29:14.338176+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2020-09-08T15:29:14.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:14 smithi104 bash[9203]: cluster 2020-09-08T15:29:14.332011+0000 mon.a (mon.0) 278 : cluster [DBG] Standby manager daemon x started 2020-09-08T15:29:14.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:14 smithi104 bash[9203]: audit 2020-09-08T15:29:14.336160+0000 mon.b (mon.1) 3 : audit [DBG] from='mgr.? v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2020-09-08T15:29:14.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:14 smithi104 bash[9203]: audit 2020-09-08T15:29:14.338176+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2020-09-08T15:29:14.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:14 smithi166 bash[9446]: cluster 2020-09-08T15:29:14.332011+0000 mon.a (mon.0) 278 : cluster [DBG] Standby manager daemon x started 2020-09-08T15:29:14.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:14 smithi166 bash[9446]: audit 2020-09-08T15:29:14.336160+0000 mon.b (mon.1) 3 : audit [DBG] from='mgr.? v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2020-09-08T15:29:14.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:14 smithi166 bash[9446]: audit 2020-09-08T15:29:14.338176+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2020-09-08T15:29:15.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:15 smithi055 bash[9296]: cluster 2020-09-08T15:29:14.389893+0000 mon.a (mon.0) 279 : cluster [DBG] mgrmap e14: y(active, since 2m), standbys: x 2020-09-08T15:29:15.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:15 smithi104 bash[9203]: cluster 2020-09-08T15:29:14.389893+0000 mon.a (mon.0) 279 : cluster [DBG] mgrmap e14: y(active, since 2m), standbys: x 2020-09-08T15:29:15.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:15 smithi166 bash[9446]: cluster 2020-09-08T15:29:14.389893+0000 mon.a (mon.0) 279 : cluster [DBG] mgrmap e14: y(active, since 2m), standbys: x 2020-09-08T15:29:39.760 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9970]: Warning: Permanently added 'smithi055,172.21.15.55' (ECDSA) to the list of known hosts. 2020-09-08T15:29:39.761 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9970]: Warning: Permanently added 'smithi166,172.21.15.166' (ECDSA) to the list of known hosts. 2020-09-08T15:29:39.761 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9970]: Warning: Permanently added 'smithi104,172.21.15.104' (ECDSA) to the list of known hosts. 2020-09-08T15:29:39.762 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: cluster 2020-09-08T15:29:39.431160+0000 mon.a (mon.0) 280 : cluster [INF] Manager daemon y is unresponsive, replacing it with standby daemon x 2020-09-08T15:29:39.762 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: cluster 2020-09-08T15:29:39.436996+0000 mon.a (mon.0) 281 : cluster [DBG] osdmap e7: 0 total, 0 up, 0 in 2020-09-08T15:29:39.762 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: cluster 2020-09-08T15:29:39.445142+0000 mon.a (mon.0) 282 : cluster [DBG] mgrmap e15: x(active, starting, since 0.0140576s) 2020-09-08T15:29:39.762 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: audit 2020-09-08T15:29:39.449861+0000 mon.a (mon.0) 283 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2020-09-08T15:29:39.763 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: audit 2020-09-08T15:29:39.451396+0000 mon.a (mon.0) 284 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2020-09-08T15:29:39.763 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: audit 2020-09-08T15:29:39.451608+0000 mon.a (mon.0) 285 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2020-09-08T15:29:39.763 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: audit 2020-09-08T15:29:39.451839+0000 mon.a (mon.0) 286 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2020-09-08T15:29:39.763 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: audit 2020-09-08T15:29:39.452844+0000 mon.a (mon.0) 287 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key ls"}]: dispatch 2020-09-08T15:29:39.764 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: audit 2020-09-08T15:29:39.454893+0000 mon.a (mon.0) 288 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.smithi055"}]: dispatch 2020-09-08T15:29:39.764 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: audit 2020-09-08T15:29:39.455669+0000 mon.a (mon.0) 289 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.smithi104"}]: dispatch 2020-09-08T15:29:39.764 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: audit 2020-09-08T15:29:39.456439+0000 mon.a (mon.0) 290 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.smithi166"}]: dispatch 2020-09-08T15:29:39.764 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: audit 2020-09-08T15:29:39.457258+0000 mon.a (mon.0) 291 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/inventory"}]: dispatch 2020-09-08T15:29:39.765 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: audit 2020-09-08T15:29:39.458035+0000 mon.a (mon.0) 292 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/osd_remove_queue"}]: dispatch 2020-09-08T15:29:39.765 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: audit 2020-09-08T15:29:39.458772+0000 mon.a (mon.0) 293 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.mgr"}]: dispatch 2020-09-08T15:29:39.765 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: audit 2020-09-08T15:29:39.459500+0000 mon.a (mon.0) 294 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_key"}]: dispatch 2020-09-08T15:29:39.766 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: audit 2020-09-08T15:29:39.460280+0000 mon.a (mon.0) 295 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_pub"}]: dispatch 2020-09-08T15:29:39.766 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: audit 2020-09-08T15:29:39.461045+0000 mon.a (mon.0) 296 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_user"}]: dispatch 2020-09-08T15:29:39.766 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: audit 2020-09-08T15:29:39.461718+0000 mon.a (mon.0) 297 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/accessdb_v2"}]: dispatch 2020-09-08T15:29:39.766 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: audit 2020-09-08T15:29:39.462345+0000 mon.a (mon.0) 298 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2020-09-08T15:29:39.767 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: audit 2020-09-08T15:29:39.462981+0000 mon.a (mon.0) 299 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2020-09-08T15:29:39.767 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: audit 2020-09-08T15:29:39.463689+0000 mon.a (mon.0) 300 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/report_id"}]: dispatch 2020-09-08T15:29:39.767 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: audit 2020-09-08T15:29:39.464361+0000 mon.a (mon.0) 301 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/salt"}]: dispatch 2020-09-08T15:29:39.767 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:39 smithi055 bash[9296]: cluster 2020-09-08T15:29:39.471360+0000 mon.a (mon.0) 302 : cluster [INF] Manager daemon x is now available 2020-09-08T15:29:39.768 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: cluster 2020-09-08T15:29:39.431160+0000 mon.a (mon.0) 280 : cluster [INF] Manager daemon y is unresponsive, replacing it with standby daemon x 2020-09-08T15:29:39.768 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: cluster 2020-09-08T15:29:39.436996+0000 mon.a (mon.0) 281 : cluster [DBG] osdmap e7: 0 total, 0 up, 0 in 2020-09-08T15:29:39.769 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: cluster 2020-09-08T15:29:39.445142+0000 mon.a (mon.0) 282 : cluster [DBG] mgrmap e15: x(active, starting, since 0.0140576s) 2020-09-08T15:29:39.774 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: audit 2020-09-08T15:29:39.449861+0000 mon.a (mon.0) 283 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2020-09-08T15:29:39.774 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: audit 2020-09-08T15:29:39.451396+0000 mon.a (mon.0) 284 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2020-09-08T15:29:39.774 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: audit 2020-09-08T15:29:39.451608+0000 mon.a (mon.0) 285 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2020-09-08T15:29:39.775 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: audit 2020-09-08T15:29:39.451839+0000 mon.a (mon.0) 286 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2020-09-08T15:29:39.775 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: audit 2020-09-08T15:29:39.452844+0000 mon.a (mon.0) 287 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key ls"}]: dispatch 2020-09-08T15:29:39.775 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: audit 2020-09-08T15:29:39.454893+0000 mon.a (mon.0) 288 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.smithi055"}]: dispatch 2020-09-08T15:29:39.775 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: audit 2020-09-08T15:29:39.455669+0000 mon.a (mon.0) 289 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.smithi104"}]: dispatch 2020-09-08T15:29:39.776 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: audit 2020-09-08T15:29:39.456439+0000 mon.a (mon.0) 290 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.smithi166"}]: dispatch 2020-09-08T15:29:39.777 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: audit 2020-09-08T15:29:39.457258+0000 mon.a (mon.0) 291 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/inventory"}]: dispatch 2020-09-08T15:29:39.777 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: audit 2020-09-08T15:29:39.458035+0000 mon.a (mon.0) 292 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/osd_remove_queue"}]: dispatch 2020-09-08T15:29:39.778 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: audit 2020-09-08T15:29:39.458772+0000 mon.a (mon.0) 293 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.mgr"}]: dispatch 2020-09-08T15:29:39.778 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: audit 2020-09-08T15:29:39.459500+0000 mon.a (mon.0) 294 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_key"}]: dispatch 2020-09-08T15:29:39.778 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: audit 2020-09-08T15:29:39.460280+0000 mon.a (mon.0) 295 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_pub"}]: dispatch 2020-09-08T15:29:39.779 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: audit 2020-09-08T15:29:39.461045+0000 mon.a (mon.0) 296 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_user"}]: dispatch 2020-09-08T15:29:39.779 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: audit 2020-09-08T15:29:39.461718+0000 mon.a (mon.0) 297 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/accessdb_v2"}]: dispatch 2020-09-08T15:29:39.779 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: audit 2020-09-08T15:29:39.462345+0000 mon.a (mon.0) 298 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2020-09-08T15:29:39.779 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: audit 2020-09-08T15:29:39.462981+0000 mon.a (mon.0) 299 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2020-09-08T15:29:39.779 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: audit 2020-09-08T15:29:39.463689+0000 mon.a (mon.0) 300 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/report_id"}]: dispatch 2020-09-08T15:29:39.780 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: audit 2020-09-08T15:29:39.464361+0000 mon.a (mon.0) 301 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/salt"}]: dispatch 2020-09-08T15:29:39.780 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:39 smithi166 bash[9446]: cluster 2020-09-08T15:29:39.471360+0000 mon.a (mon.0) 302 : cluster [INF] Manager daemon x is now available 2020-09-08T15:29:39.781 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: cluster 2020-09-08T15:29:39.431160+0000 mon.a (mon.0) 280 : cluster [INF] Manager daemon y is unresponsive, replacing it with standby daemon x 2020-09-08T15:29:39.781 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: cluster 2020-09-08T15:29:39.436996+0000 mon.a (mon.0) 281 : cluster [DBG] osdmap e7: 0 total, 0 up, 0 in 2020-09-08T15:29:39.782 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: cluster 2020-09-08T15:29:39.445142+0000 mon.a (mon.0) 282 : cluster [DBG] mgrmap e15: x(active, starting, since 0.0140576s) 2020-09-08T15:29:39.782 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: audit 2020-09-08T15:29:39.449861+0000 mon.a (mon.0) 283 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2020-09-08T15:29:39.782 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: audit 2020-09-08T15:29:39.451396+0000 mon.a (mon.0) 284 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2020-09-08T15:29:39.782 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: audit 2020-09-08T15:29:39.451608+0000 mon.a (mon.0) 285 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2020-09-08T15:29:39.783 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: audit 2020-09-08T15:29:39.451839+0000 mon.a (mon.0) 286 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2020-09-08T15:29:39.783 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: audit 2020-09-08T15:29:39.452844+0000 mon.a (mon.0) 287 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key ls"}]: dispatch 2020-09-08T15:29:39.783 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: audit 2020-09-08T15:29:39.454893+0000 mon.a (mon.0) 288 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.smithi055"}]: dispatch 2020-09-08T15:29:39.783 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: audit 2020-09-08T15:29:39.455669+0000 mon.a (mon.0) 289 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.smithi104"}]: dispatch 2020-09-08T15:29:39.783 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: audit 2020-09-08T15:29:39.456439+0000 mon.a (mon.0) 290 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/host.smithi166"}]: dispatch 2020-09-08T15:29:39.784 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: audit 2020-09-08T15:29:39.457258+0000 mon.a (mon.0) 291 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/inventory"}]: dispatch 2020-09-08T15:29:39.784 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: audit 2020-09-08T15:29:39.458035+0000 mon.a (mon.0) 292 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/osd_remove_queue"}]: dispatch 2020-09-08T15:29:39.784 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: audit 2020-09-08T15:29:39.458772+0000 mon.a (mon.0) 293 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/spec.mgr"}]: dispatch 2020-09-08T15:29:39.784 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: audit 2020-09-08T15:29:39.459500+0000 mon.a (mon.0) 294 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_key"}]: dispatch 2020-09-08T15:29:39.785 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: audit 2020-09-08T15:29:39.460280+0000 mon.a (mon.0) 295 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_identity_pub"}]: dispatch 2020-09-08T15:29:39.785 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: audit 2020-09-08T15:29:39.461045+0000 mon.a (mon.0) 296 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/cephadm/ssh_user"}]: dispatch 2020-09-08T15:29:39.785 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: audit 2020-09-08T15:29:39.461718+0000 mon.a (mon.0) 297 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/accessdb_v2"}]: dispatch 2020-09-08T15:29:39.785 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: audit 2020-09-08T15:29:39.462345+0000 mon.a (mon.0) 298 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2020-09-08T15:29:39.786 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: audit 2020-09-08T15:29:39.462981+0000 mon.a (mon.0) 299 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2020-09-08T15:29:39.786 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: audit 2020-09-08T15:29:39.463689+0000 mon.a (mon.0) 300 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/report_id"}]: dispatch 2020-09-08T15:29:39.786 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: audit 2020-09-08T15:29:39.464361+0000 mon.a (mon.0) 301 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/telemetry/salt"}]: dispatch 2020-09-08T15:29:39.786 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:39 smithi104 bash[9203]: cluster 2020-09-08T15:29:39.471360+0000 mon.a (mon.0) 302 : cluster [INF] Manager daemon x is now available 2020-09-08T15:29:41.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:40 smithi166 bash[9446]: audit 2020-09-08T15:29:39.872123+0000 mon.a (mon.0) 303 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2020-09-08T15:29:41.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:40 smithi166 bash[9446]: audit 2020-09-08T15:29:40.748878+0000 mon.a (mon.0) 304 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2020-09-08T15:29:41.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:40 smithi166 bash[9446]: audit 2020-09-08T15:29:40.750987+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:41.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:40 smithi166 bash[9446]: audit 2020-09-08T15:29:40.751176+0000 mon.a (mon.0) 306 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:41.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:40 smithi166 bash[9446]: audit 2020-09-08T15:29:40.751313+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:41.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:40 smithi166 bash[9446]: cluster 2020-09-08T15:29:40.755657+0000 mon.a (mon.0) 308 : cluster [DBG] mgrmap e16: x(active, since 1.32458s) 2020-09-08T15:29:41.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:40 smithi166 bash[9446]: audit 2020-09-08T15:29:40.763190+0000 mon.a (mon.0) 309 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:29:41.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:40 smithi166 bash[9446]: audit 2020-09-08T15:29:40.767612+0000 mon.a (mon.0) 310 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:29:41.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:40 smithi166 bash[9446]: audit 2020-09-08T15:29:40.793290+0000 mon.a (mon.0) 311 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:41.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:40 smithi166 bash[9446]: audit 2020-09-08T15:29:40.798744+0000 mon.a (mon.0) 312 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:29:41.155 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:40 smithi055 bash[9296]: audit 2020-09-08T15:29:39.872123+0000 mon.a (mon.0) 303 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2020-09-08T15:29:41.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:40 smithi055 bash[9296]: audit 2020-09-08T15:29:40.748878+0000 mon.a (mon.0) 304 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2020-09-08T15:29:41.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:40 smithi055 bash[9296]: audit 2020-09-08T15:29:40.750987+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:41.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:40 smithi055 bash[9296]: audit 2020-09-08T15:29:40.751176+0000 mon.a (mon.0) 306 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:41.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:40 smithi055 bash[9296]: audit 2020-09-08T15:29:40.751313+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:41.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:40 smithi055 bash[9296]: cluster 2020-09-08T15:29:40.755657+0000 mon.a (mon.0) 308 : cluster [DBG] mgrmap e16: x(active, since 1.32458s) 2020-09-08T15:29:41.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:40 smithi055 bash[9296]: audit 2020-09-08T15:29:40.763190+0000 mon.a (mon.0) 309 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:29:41.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:40 smithi055 bash[9296]: audit 2020-09-08T15:29:40.767612+0000 mon.a (mon.0) 310 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:29:41.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:40 smithi055 bash[9296]: audit 2020-09-08T15:29:40.793290+0000 mon.a (mon.0) 311 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:41.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:40 smithi055 bash[9296]: audit 2020-09-08T15:29:40.798744+0000 mon.a (mon.0) 312 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:29:41.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:40 smithi104 bash[9203]: audit 2020-09-08T15:29:39.872123+0000 mon.a (mon.0) 303 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2020-09-08T15:29:41.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:40 smithi104 bash[9203]: audit 2020-09-08T15:29:40.748878+0000 mon.a (mon.0) 304 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2020-09-08T15:29:41.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:40 smithi104 bash[9203]: audit 2020-09-08T15:29:40.750987+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:41.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:40 smithi104 bash[9203]: audit 2020-09-08T15:29:40.751176+0000 mon.a (mon.0) 306 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:41.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:40 smithi104 bash[9203]: audit 2020-09-08T15:29:40.751313+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:41.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:40 smithi104 bash[9203]: cluster 2020-09-08T15:29:40.755657+0000 mon.a (mon.0) 308 : cluster [DBG] mgrmap e16: x(active, since 1.32458s) 2020-09-08T15:29:41.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:40 smithi104 bash[9203]: audit 2020-09-08T15:29:40.763190+0000 mon.a (mon.0) 309 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:29:41.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:40 smithi104 bash[9203]: audit 2020-09-08T15:29:40.767612+0000 mon.a (mon.0) 310 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:29:41.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:40 smithi104 bash[9203]: audit 2020-09-08T15:29:40.793290+0000 mon.a (mon.0) 311 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:41.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:40 smithi104 bash[9203]: audit 2020-09-08T15:29:40.798744+0000 mon.a (mon.0) 312 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:29:42.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:41 smithi166 bash[9446]: audit 2020-09-08T15:29:40.759904+0000 mgr.x (mgr.24118) 1 : audit [DBG] from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:29:42.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:41 smithi166 bash[9446]: cephadm 2020-09-08T15:29:40.766133+0000 mgr.x (mgr.24118) 2 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:29:42.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:41 smithi166 bash[9446]: cephadm 2020-09-08T15:29:40.766435+0000 mgr.x (mgr.24118) 3 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:29:42.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:41 smithi166 bash[9446]: cephadm 2020-09-08T15:29:40.768253+0000 mgr.x (mgr.24118) 4 : cephadm [INF] Applying None on host smithi104... 2020-09-08T15:29:42.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:41 smithi166 bash[9446]: audit 2020-09-08T15:29:41.376427+0000 mon.a (mon.0) 313 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.376015\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.959478\"}}, \"last_daemon_update\": \"2020-09-08T15:29:41.376354\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:29:42.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:41 smithi166 bash[9446]: audit 2020-09-08T15:29:41.379749+0000 mon.a (mon.0) 314 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.376015\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.959478\"}}, \"last_daemon_update\": \"2020-09-08T15:29:41.376354\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:29:42.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:41 smithi166 bash[9446]: audit 2020-09-08T15:29:41.406665+0000 mon.a (mon.0) 315 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.406841\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.406940\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.957288\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_daemon_update\": \"2020-09-08T15:29:41.407054\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:29:42.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:41 smithi166 bash[9446]: audit 2020-09-08T15:29:41.413445+0000 mon.a (mon.0) 316 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.406841\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.406940\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.957288\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_daemon_update\": \"2020-09-08T15:29:41.407054\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:29:42.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:41 smithi166 bash[9446]: audit 2020-09-08T15:29:41.433592+0000 mon.a (mon.0) 317 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.433523\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:29:41.433815\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:29:42.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:41 smithi166 bash[9446]: audit 2020-09-08T15:29:41.437607+0000 mon.a (mon.0) 318 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.433523\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:29:41.433815\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:29:42.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:41 smithi166 bash[9446]: audit 2020-09-08T15:29:41.439289+0000 mon.a (mon.0) 319 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:29:42.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:41 smithi166 bash[9446]: audit 2020-09-08T15:29:41.443220+0000 mon.a (mon.0) 320 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:29:42.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:41 smithi166 bash[9446]: audit 2020-09-08T15:29:41.445633+0000 mon.a (mon.0) 321 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2020-09-08T15:29:42.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:41 smithi166 bash[9446]: cephadm 2020-09-08T15:29:41.446358+0000 mgr.x (mgr.24118) 5 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2020-09-08T15:29:42.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:41 smithi166 bash[9446]: audit 2020-09-08T15:29:41.447115+0000 mon.a (mon.0) 322 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:42.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:41 smithi166 bash[9446]: audit 2020-09-08T15:29:41.448844+0000 mon.a (mon.0) 323 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon.b", "key": "container_image"}]: dispatch 2020-09-08T15:29:42.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:41 smithi166 bash[9446]: cephadm 2020-09-08T15:29:41.449640+0000 mgr.x (mgr.24118) 6 : cephadm [INF] Reconfiguring daemon mon.b on smithi055 2020-09-08T15:29:42.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:41 smithi166 bash[9446]: cluster 2020-09-08T15:29:41.762396+0000 mon.a (mon.0) 324 : cluster [DBG] mgrmap e17: x(active, since 2s) 2020-09-08T15:29:42.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:41 smithi055 bash[9296]: audit 2020-09-08T15:29:40.759904+0000 mgr.x (mgr.24118) 1 : audit [DBG] from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:29:42.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:41 smithi055 bash[9296]: cephadm 2020-09-08T15:29:40.766133+0000 mgr.x (mgr.24118) 2 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:29:42.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:41 smithi055 bash[9296]: cephadm 2020-09-08T15:29:40.766435+0000 mgr.x (mgr.24118) 3 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:29:42.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:41 smithi055 bash[9296]: cephadm 2020-09-08T15:29:40.768253+0000 mgr.x (mgr.24118) 4 : cephadm [INF] Applying None on host smithi104... 2020-09-08T15:29:42.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:41 smithi055 bash[9296]: audit 2020-09-08T15:29:41.376427+0000 mon.a (mon.0) 313 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.376015\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.959478\"}}, \"last_daemon_update\": \"2020-09-08T15:29:41.376354\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:29:42.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:41 smithi055 bash[9296]: audit 2020-09-08T15:29:41.379749+0000 mon.a (mon.0) 314 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.376015\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.959478\"}}, \"last_daemon_update\": \"2020-09-08T15:29:41.376354\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:29:42.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:41 smithi055 bash[9296]: audit 2020-09-08T15:29:41.406665+0000 mon.a (mon.0) 315 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.406841\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.406940\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.957288\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_daemon_update\": \"2020-09-08T15:29:41.407054\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:29:42.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:41 smithi055 bash[9296]: audit 2020-09-08T15:29:41.413445+0000 mon.a (mon.0) 316 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.406841\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.406940\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.957288\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_daemon_update\": \"2020-09-08T15:29:41.407054\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:29:42.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:41 smithi055 bash[9296]: audit 2020-09-08T15:29:41.433592+0000 mon.a (mon.0) 317 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.433523\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:29:41.433815\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:29:42.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:41 smithi055 bash[9296]: audit 2020-09-08T15:29:41.437607+0000 mon.a (mon.0) 318 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.433523\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:29:41.433815\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:29:42.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:41 smithi055 bash[9296]: audit 2020-09-08T15:29:41.439289+0000 mon.a (mon.0) 319 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:29:42.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:41 smithi055 bash[9296]: audit 2020-09-08T15:29:41.443220+0000 mon.a (mon.0) 320 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:29:42.162 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:41 smithi055 bash[9296]: audit 2020-09-08T15:29:41.445633+0000 mon.a (mon.0) 321 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2020-09-08T15:29:42.163 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:41 smithi055 bash[9296]: cephadm 2020-09-08T15:29:41.446358+0000 mgr.x (mgr.24118) 5 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2020-09-08T15:29:42.163 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:41 smithi055 bash[9296]: audit 2020-09-08T15:29:41.447115+0000 mon.a (mon.0) 322 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:42.163 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:41 smithi055 bash[9296]: audit 2020-09-08T15:29:41.448844+0000 mon.a (mon.0) 323 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon.b", "key": "container_image"}]: dispatch 2020-09-08T15:29:42.164 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:41 smithi055 bash[9296]: cephadm 2020-09-08T15:29:41.449640+0000 mgr.x (mgr.24118) 6 : cephadm [INF] Reconfiguring daemon mon.b on smithi055 2020-09-08T15:29:42.165 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:41 smithi055 bash[9296]: cluster 2020-09-08T15:29:41.762396+0000 mon.a (mon.0) 324 : cluster [DBG] mgrmap e17: x(active, since 2s) 2020-09-08T15:29:42.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:41 smithi104 bash[9203]: audit 2020-09-08T15:29:40.759904+0000 mgr.x (mgr.24118) 1 : audit [DBG] from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:29:42.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:41 smithi104 bash[9203]: cephadm 2020-09-08T15:29:40.766133+0000 mgr.x (mgr.24118) 2 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:29:42.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:41 smithi104 bash[9203]: cephadm 2020-09-08T15:29:40.766435+0000 mgr.x (mgr.24118) 3 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:29:42.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:41 smithi104 bash[9203]: cephadm 2020-09-08T15:29:40.768253+0000 mgr.x (mgr.24118) 4 : cephadm [INF] Applying None on host smithi104... 2020-09-08T15:29:42.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:41 smithi104 bash[9203]: audit 2020-09-08T15:29:41.376427+0000 mon.a (mon.0) 313 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.376015\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.959478\"}}, \"last_daemon_update\": \"2020-09-08T15:29:41.376354\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:29:42.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:41 smithi104 bash[9203]: audit 2020-09-08T15:29:41.379749+0000 mon.a (mon.0) 314 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.376015\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.959478\"}}, \"last_daemon_update\": \"2020-09-08T15:29:41.376354\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:29:42.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:41 smithi104 bash[9203]: audit 2020-09-08T15:29:41.406665+0000 mon.a (mon.0) 315 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.406841\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.406940\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.957288\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_daemon_update\": \"2020-09-08T15:29:41.407054\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:29:42.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:41 smithi104 bash[9203]: audit 2020-09-08T15:29:41.413445+0000 mon.a (mon.0) 316 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.406841\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.406940\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:46.957288\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_daemon_update\": \"2020-09-08T15:29:41.407054\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:29:42.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:41 smithi104 bash[9203]: audit 2020-09-08T15:29:41.433592+0000 mon.a (mon.0) 317 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.433523\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:29:41.433815\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:29:42.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:41 smithi104 bash[9203]: audit 2020-09-08T15:29:41.437607+0000 mon.a (mon.0) 318 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.433523\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:43.096563\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:29:41.433815\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:29:42.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:41 smithi104 bash[9203]: audit 2020-09-08T15:29:41.439289+0000 mon.a (mon.0) 319 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:29:42.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:41 smithi104 bash[9203]: audit 2020-09-08T15:29:41.443220+0000 mon.a (mon.0) 320 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:29:42.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:41 smithi104 bash[9203]: audit 2020-09-08T15:29:41.445633+0000 mon.a (mon.0) 321 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2020-09-08T15:29:42.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:41 smithi104 bash[9203]: cephadm 2020-09-08T15:29:41.446358+0000 mgr.x (mgr.24118) 5 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2020-09-08T15:29:42.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:41 smithi104 bash[9203]: audit 2020-09-08T15:29:41.447115+0000 mon.a (mon.0) 322 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:42.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:41 smithi104 bash[9203]: audit 2020-09-08T15:29:41.448844+0000 mon.a (mon.0) 323 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon.b", "key": "container_image"}]: dispatch 2020-09-08T15:29:42.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:41 smithi104 bash[9203]: cephadm 2020-09-08T15:29:41.449640+0000 mgr.x (mgr.24118) 6 : cephadm [INF] Reconfiguring daemon mon.b on smithi055 2020-09-08T15:29:42.183 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:41 smithi104 bash[9203]: cluster 2020-09-08T15:29:41.762396+0000 mon.a (mon.0) 324 : cluster [DBG] mgrmap e17: x(active, since 2s) 2020-09-08T15:29:43.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:43 smithi055 bash[9296]: audit 2020-09-08T15:29:43.597184+0000 mon.a (mon.0) 325 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.406940\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:29:43.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:43 smithi055 bash[9296]: audit 2020-09-08T15:29:43.602514+0000 mon.a (mon.0) 326 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.406940\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:29:43.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:43 smithi055 bash[9296]: audit 2020-09-08T15:29:43.603847+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2020-09-08T15:29:43.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:43 smithi055 bash[9296]: audit 2020-09-08T15:29:43.605219+0000 mon.a (mon.0) 328 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:43.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:43 smithi055 bash[9296]: audit 2020-09-08T15:29:43.607086+0000 mon.a (mon.0) 329 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon.a", "key": "container_image"}]: dispatch 2020-09-08T15:29:43.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:43 smithi104 bash[9203]: audit 2020-09-08T15:29:43.597184+0000 mon.a (mon.0) 325 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.406940\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:29:43.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:43 smithi104 bash[9203]: audit 2020-09-08T15:29:43.602514+0000 mon.a (mon.0) 326 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.406940\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:29:43.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:43 smithi104 bash[9203]: audit 2020-09-08T15:29:43.603847+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2020-09-08T15:29:43.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:43 smithi104 bash[9203]: audit 2020-09-08T15:29:43.605219+0000 mon.a (mon.0) 328 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:43.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:43 smithi104 bash[9203]: audit 2020-09-08T15:29:43.607086+0000 mon.a (mon.0) 329 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon.a", "key": "container_image"}]: dispatch 2020-09-08T15:29:44.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:43 smithi166 bash[9446]: audit 2020-09-08T15:29:43.597184+0000 mon.a (mon.0) 325 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.406940\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:29:44.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:43 smithi166 bash[9446]: audit 2020-09-08T15:29:43.602514+0000 mon.a (mon.0) 326 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:41.406940\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:29:44.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:43 smithi166 bash[9446]: audit 2020-09-08T15:29:43.603847+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2020-09-08T15:29:44.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:43 smithi166 bash[9446]: audit 2020-09-08T15:29:43.605219+0000 mon.a (mon.0) 328 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:44.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:43 smithi166 bash[9446]: audit 2020-09-08T15:29:43.607086+0000 mon.a (mon.0) 329 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon.a", "key": "container_image"}]: dispatch 2020-09-08T15:29:44.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:44 smithi055 bash[9296]: cephadm 2020-09-08T15:29:43.604702+0000 mgr.x (mgr.24118) 7 : cephadm [INF] Reconfiguring mon.a (monmap changed)... 2020-09-08T15:29:44.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:44 smithi055 bash[9296]: cephadm 2020-09-08T15:29:43.607842+0000 mgr.x (mgr.24118) 8 : cephadm [INF] Reconfiguring daemon mon.a on smithi104 2020-09-08T15:29:44.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:44 smithi055 bash[9296]: audit 2020-09-08T15:29:44.426381+0000 mon.a (mon.0) 330 : audit [INF] from='client.? v1:172.21.15.104:0/690402734' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e49a7341-7c75-4883-b061-3e8fe9467e6e"}]: dispatch 2020-09-08T15:29:44.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:44 smithi055 bash[9296]: audit 2020-09-08T15:29:44.433211+0000 mon.a (mon.0) 331 : audit [INF] from='client.? v1:172.21.15.104:0/690402734' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e49a7341-7c75-4883-b061-3e8fe9467e6e"}]': finished 2020-09-08T15:29:44.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:44 smithi055 bash[9296]: cluster 2020-09-08T15:29:44.433304+0000 mon.a (mon.0) 332 : cluster [DBG] osdmap e8: 1 total, 0 up, 0 in 2020-09-08T15:29:44.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:44 smithi055 bash[9296]: audit 2020-09-08T15:29:44.433463+0000 mon.a (mon.0) 333 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2020-09-08T15:29:44.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:44 smithi104 bash[9203]: cephadm 2020-09-08T15:29:43.604702+0000 mgr.x (mgr.24118) 7 : cephadm [INF] Reconfiguring mon.a (monmap changed)... 2020-09-08T15:29:44.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:44 smithi104 bash[9203]: cephadm 2020-09-08T15:29:43.607842+0000 mgr.x (mgr.24118) 8 : cephadm [INF] Reconfiguring daemon mon.a on smithi104 2020-09-08T15:29:44.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:44 smithi104 bash[9203]: audit 2020-09-08T15:29:44.426381+0000 mon.a (mon.0) 330 : audit [INF] from='client.? v1:172.21.15.104:0/690402734' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e49a7341-7c75-4883-b061-3e8fe9467e6e"}]: dispatch 2020-09-08T15:29:44.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:44 smithi104 bash[9203]: audit 2020-09-08T15:29:44.433211+0000 mon.a (mon.0) 331 : audit [INF] from='client.? v1:172.21.15.104:0/690402734' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e49a7341-7c75-4883-b061-3e8fe9467e6e"}]': finished 2020-09-08T15:29:44.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:44 smithi104 bash[9203]: cluster 2020-09-08T15:29:44.433304+0000 mon.a (mon.0) 332 : cluster [DBG] osdmap e8: 1 total, 0 up, 0 in 2020-09-08T15:29:44.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:44 smithi104 bash[9203]: audit 2020-09-08T15:29:44.433463+0000 mon.a (mon.0) 333 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2020-09-08T15:29:45.111 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:44 smithi166 bash[9446]: cephadm 2020-09-08T15:29:43.604702+0000 mgr.x (mgr.24118) 7 : cephadm [INF] Reconfiguring mon.a (monmap changed)... 2020-09-08T15:29:45.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:44 smithi166 bash[9446]: cephadm 2020-09-08T15:29:43.607842+0000 mgr.x (mgr.24118) 8 : cephadm [INF] Reconfiguring daemon mon.a on smithi104 2020-09-08T15:29:45.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:44 smithi166 bash[9446]: audit 2020-09-08T15:29:44.426381+0000 mon.a (mon.0) 330 : audit [INF] from='client.? v1:172.21.15.104:0/690402734' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e49a7341-7c75-4883-b061-3e8fe9467e6e"}]: dispatch 2020-09-08T15:29:45.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:44 smithi166 bash[9446]: audit 2020-09-08T15:29:44.433211+0000 mon.a (mon.0) 331 : audit [INF] from='client.? v1:172.21.15.104:0/690402734' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e49a7341-7c75-4883-b061-3e8fe9467e6e"}]': finished 2020-09-08T15:29:45.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:44 smithi166 bash[9446]: cluster 2020-09-08T15:29:44.433304+0000 mon.a (mon.0) 332 : cluster [DBG] osdmap e8: 1 total, 0 up, 0 in 2020-09-08T15:29:45.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:44 smithi166 bash[9446]: audit 2020-09-08T15:29:44.433463+0000 mon.a (mon.0) 333 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2020-09-08T15:29:45.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:45 smithi055 bash[9296]: audit 2020-09-08T15:29:45.075399+0000 mon.a (mon.0) 334 : audit [DBG] from='client.? v1:172.21.15.104:0/185764859' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:29:45.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:45 smithi104 bash[9203]: audit 2020-09-08T15:29:45.075399+0000 mon.a (mon.0) 334 : audit [DBG] from='client.? v1:172.21.15.104:0/185764859' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:29:46.111 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:45 smithi166 bash[9446]: audit 2020-09-08T15:29:45.075399+0000 mon.a (mon.0) 334 : audit [DBG] from='client.? v1:172.21.15.104:0/185764859' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:29:48.362 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:48 smithi166 bash[9446]: audit 2020-09-08T15:29:48.021215+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:29:48.405 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:48 smithi055 bash[9296]: audit 2020-09-08T15:29:48.021215+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:29:48.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:48 smithi104 bash[9203]: audit 2020-09-08T15:29:48.021215+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:29:51.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:50 smithi166 bash[9446]: audit 2020-09-08T15:29:50.658628+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:29:51.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:50 smithi166 bash[9446]: audit 2020-09-08T15:29:50.664791+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:29:51.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:50 smithi166 bash[9446]: audit 2020-09-08T15:29:50.666192+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2020-09-08T15:29:51.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:50 smithi166 bash[9446]: audit 2020-09-08T15:29:50.667983+0000 mon.a (mon.0) 339 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:51.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:50 smithi166 bash[9446]: audit 2020-09-08T15:29:50.670028+0000 mon.a (mon.0) 340 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon.c", "key": "container_image"}]: dispatch 2020-09-08T15:29:51.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:50 smithi055 bash[9296]: audit 2020-09-08T15:29:50.658628+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:29:51.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:50 smithi055 bash[9296]: audit 2020-09-08T15:29:50.664791+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:29:51.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:50 smithi055 bash[9296]: audit 2020-09-08T15:29:50.666192+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2020-09-08T15:29:51.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:50 smithi055 bash[9296]: audit 2020-09-08T15:29:50.667983+0000 mon.a (mon.0) 339 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:51.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:50 smithi055 bash[9296]: audit 2020-09-08T15:29:50.670028+0000 mon.a (mon.0) 340 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon.c", "key": "container_image"}]: dispatch 2020-09-08T15:29:51.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:50 smithi104 bash[9203]: audit 2020-09-08T15:29:50.658628+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:29:51.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:50 smithi104 bash[9203]: audit 2020-09-08T15:29:50.664791+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:29:51.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:50 smithi104 bash[9203]: audit 2020-09-08T15:29:50.666192+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2020-09-08T15:29:51.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:50 smithi104 bash[9203]: audit 2020-09-08T15:29:50.667983+0000 mon.a (mon.0) 339 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:51.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:50 smithi104 bash[9203]: audit 2020-09-08T15:29:50.670028+0000 mon.a (mon.0) 340 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon.c", "key": "container_image"}]: dispatch 2020-09-08T15:29:51.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:51 smithi104 bash[9203]: cephadm 2020-09-08T15:29:50.667071+0000 mgr.x (mgr.24118) 9 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2020-09-08T15:29:51.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:51 smithi104 bash[9203]: cephadm 2020-09-08T15:29:50.670782+0000 mgr.x (mgr.24118) 10 : cephadm [INF] Reconfiguring daemon mon.c on smithi166 2020-09-08T15:29:52.054 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:51 smithi166 bash[9446]: cephadm 2020-09-08T15:29:50.667071+0000 mgr.x (mgr.24118) 9 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2020-09-08T15:29:52.055 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:51 smithi166 bash[9446]: cephadm 2020-09-08T15:29:50.670782+0000 mgr.x (mgr.24118) 10 : cephadm [INF] Reconfiguring daemon mon.c on smithi166 2020-09-08T15:29:52.155 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:51 smithi055 bash[9296]: cephadm 2020-09-08T15:29:50.667071+0000 mgr.x (mgr.24118) 9 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2020-09-08T15:29:52.155 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:51 smithi055 bash[9296]: cephadm 2020-09-08T15:29:50.670782+0000 mgr.x (mgr.24118) 10 : cephadm [INF] Reconfiguring daemon mon.c on smithi166 2020-09-08T15:29:53.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:52 smithi166 bash[9446]: audit 2020-09-08T15:29:52.595990+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:29:53.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:52 smithi166 bash[9446]: audit 2020-09-08T15:29:52.602029+0000 mon.a (mon.0) 342 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:29:53.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:52 smithi166 bash[9446]: audit 2020-09-08T15:29:52.605522+0000 mon.a (mon.0) 343 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:53.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:52 smithi166 bash[9446]: audit 2020-09-08T15:29:52.605962+0000 mon.a (mon.0) 344 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:53.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:52 smithi166 bash[9446]: audit 2020-09-08T15:29:52.606519+0000 mon.a (mon.0) 345 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:53.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:52 smithi166 bash[9446]: audit 2020-09-08T15:29:52.689116+0000 mon.a (mon.0) 346 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2020-09-08T15:29:53.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:52 smithi166 bash[9446]: audit 2020-09-08T15:29:52.690297+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:53.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:52 smithi166 bash[9446]: audit 2020-09-08T15:29:52.691499+0000 mon.a (mon.0) 348 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.0", "key": "container_image"}]: dispatch 2020-09-08T15:29:53.155 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:52 smithi055 bash[9296]: audit 2020-09-08T15:29:52.595990+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:29:53.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:52 smithi055 bash[9296]: audit 2020-09-08T15:29:52.602029+0000 mon.a (mon.0) 342 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:29:53.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:52 smithi055 bash[9296]: audit 2020-09-08T15:29:52.605522+0000 mon.a (mon.0) 343 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:53.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:52 smithi055 bash[9296]: audit 2020-09-08T15:29:52.605962+0000 mon.a (mon.0) 344 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:53.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:52 smithi055 bash[9296]: audit 2020-09-08T15:29:52.606519+0000 mon.a (mon.0) 345 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:53.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:52 smithi055 bash[9296]: audit 2020-09-08T15:29:52.689116+0000 mon.a (mon.0) 346 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2020-09-08T15:29:53.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:52 smithi055 bash[9296]: audit 2020-09-08T15:29:52.690297+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:53.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:52 smithi055 bash[9296]: audit 2020-09-08T15:29:52.691499+0000 mon.a (mon.0) 348 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.0", "key": "container_image"}]: dispatch 2020-09-08T15:29:53.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:52 smithi104 bash[9203]: audit 2020-09-08T15:29:52.595990+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:29:53.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:52 smithi104 bash[9203]: audit 2020-09-08T15:29:52.602029+0000 mon.a (mon.0) 342 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:29:53.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:52 smithi104 bash[9203]: audit 2020-09-08T15:29:52.605522+0000 mon.a (mon.0) 343 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:53.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:52 smithi104 bash[9203]: audit 2020-09-08T15:29:52.605962+0000 mon.a (mon.0) 344 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:53.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:52 smithi104 bash[9203]: audit 2020-09-08T15:29:52.606519+0000 mon.a (mon.0) 345 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:53.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:52 smithi104 bash[9203]: audit 2020-09-08T15:29:52.689116+0000 mon.a (mon.0) 346 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2020-09-08T15:29:53.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:52 smithi104 bash[9203]: audit 2020-09-08T15:29:52.690297+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:29:53.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:52 smithi104 bash[9203]: audit 2020-09-08T15:29:52.691499+0000 mon.a (mon.0) 348 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.0", "key": "container_image"}]: dispatch 2020-09-08T15:29:53.991 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:53 smithi104 bash[9203]: cephadm 2020-09-08T15:29:52.692570+0000 mgr.x (mgr.24118) 11 : cephadm [INF] Deploying daemon osd.0 on smithi104 2020-09-08T15:29:53.991 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:53 smithi104 bash[9203]: audit 2020-09-08T15:29:53.170340+0000 mon.a (mon.0) 349 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.170728\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}}, \"last_daemon_update\": \"2020-09-08T15:29:53.170931\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:29:53.992 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:53 smithi104 bash[9203]: audit 2020-09-08T15:29:53.175255+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.170728\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}}, \"last_daemon_update\": \"2020-09-08T15:29:53.170931\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:29:53.992 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:53 smithi104 bash[9203]: audit 2020-09-08T15:29:53.192603+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.193014\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:29:53.193192\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:29:53.992 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:53 smithi104 bash[9203]: audit 2020-09-08T15:29:53.197400+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.193014\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:29:53.193192\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:29:53.993 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:53 smithi104 bash[9203]: audit 2020-09-08T15:29:53.256286+0000 mon.a (mon.0) 353 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.256049\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.256219\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_daemon_update\": \"2020-09-08T15:29:53.256435\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:29:53.993 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:53 smithi104 bash[9203]: audit 2020-09-08T15:29:53.261888+0000 mon.a (mon.0) 354 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.256049\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.256219\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_daemon_update\": \"2020-09-08T15:29:53.256435\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:29:53.994 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:53 smithi104 bash[9203]: audit 2020-09-08T15:29:53.263412+0000 mon.a (mon.0) 355 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:29:53.994 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:53 smithi104 bash[9203]: audit 2020-09-08T15:29:53.268421+0000 mon.a (mon.0) 356 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:29:54.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:53 smithi166 bash[9446]: cephadm 2020-09-08T15:29:52.692570+0000 mgr.x (mgr.24118) 11 : cephadm [INF] Deploying daemon osd.0 on smithi104 2020-09-08T15:29:54.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:53 smithi166 bash[9446]: audit 2020-09-08T15:29:53.170340+0000 mon.a (mon.0) 349 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.170728\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}}, \"last_daemon_update\": \"2020-09-08T15:29:53.170931\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:29:54.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:53 smithi166 bash[9446]: audit 2020-09-08T15:29:53.175255+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.170728\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}}, \"last_daemon_update\": \"2020-09-08T15:29:53.170931\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:29:54.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:53 smithi166 bash[9446]: audit 2020-09-08T15:29:53.192603+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.193014\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:29:53.193192\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:29:54.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:53 smithi166 bash[9446]: audit 2020-09-08T15:29:53.197400+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.193014\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:29:53.193192\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:29:54.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:53 smithi166 bash[9446]: audit 2020-09-08T15:29:53.256286+0000 mon.a (mon.0) 353 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.256049\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.256219\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_daemon_update\": \"2020-09-08T15:29:53.256435\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:29:54.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:53 smithi166 bash[9446]: audit 2020-09-08T15:29:53.261888+0000 mon.a (mon.0) 354 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.256049\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.256219\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_daemon_update\": \"2020-09-08T15:29:53.256435\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:29:54.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:53 smithi166 bash[9446]: audit 2020-09-08T15:29:53.263412+0000 mon.a (mon.0) 355 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:29:54.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:53 smithi166 bash[9446]: audit 2020-09-08T15:29:53.268421+0000 mon.a (mon.0) 356 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:29:54.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:53 smithi055 bash[9296]: cephadm 2020-09-08T15:29:52.692570+0000 mgr.x (mgr.24118) 11 : cephadm [INF] Deploying daemon osd.0 on smithi104 2020-09-08T15:29:54.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:53 smithi055 bash[9296]: audit 2020-09-08T15:29:53.170340+0000 mon.a (mon.0) 349 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.170728\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}}, \"last_daemon_update\": \"2020-09-08T15:29:53.170931\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:29:54.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:53 smithi055 bash[9296]: audit 2020-09-08T15:29:53.175255+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.170728\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}}, \"last_daemon_update\": \"2020-09-08T15:29:53.170931\", \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:29:54.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:53 smithi055 bash[9296]: audit 2020-09-08T15:29:53.192603+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.193014\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:29:53.193192\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:29:54.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:53 smithi055 bash[9296]: audit 2020-09-08T15:29:53.197400+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.193014\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}}, \"last_daemon_update\": \"2020-09-08T15:29:53.193192\", \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:29:54.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:53 smithi055 bash[9296]: audit 2020-09-08T15:29:53.256286+0000 mon.a (mon.0) 353 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.256049\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.256219\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_daemon_update\": \"2020-09-08T15:29:53.256435\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:29:54.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:53 smithi055 bash[9296]: audit 2020-09-08T15:29:53.261888+0000 mon.a (mon.0) 354 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.256049\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.256219\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}}, \"last_daemon_update\": \"2020-09-08T15:29:53.256435\", \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:29:54.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:53 smithi055 bash[9296]: audit 2020-09-08T15:29:53.263412+0000 mon.a (mon.0) 355 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:29:54.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:53 smithi055 bash[9296]: audit 2020-09-08T15:29:53.268421+0000 mon.a (mon.0) 356 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:29:55.032 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 0 on host 'smithi104' 2020-09-08T15:29:55.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:55 smithi166 bash[9446]: audit 2020-09-08T15:29:55.026074+0000 mon.a (mon.0) 357 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.193014\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:29:55.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:55 smithi166 bash[9446]: audit 2020-09-08T15:29:55.026821+0000 mon.a (mon.0) 358 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:55.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:55 smithi166 bash[9446]: audit 2020-09-08T15:29:55.031607+0000 mon.a (mon.0) 359 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.193014\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:29:55.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:55 smithi055 bash[9296]: audit 2020-09-08T15:29:55.026074+0000 mon.a (mon.0) 357 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.193014\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:29:55.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:55 smithi055 bash[9296]: audit 2020-09-08T15:29:55.026821+0000 mon.a (mon.0) 358 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:55.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:55 smithi055 bash[9296]: audit 2020-09-08T15:29:55.031607+0000 mon.a (mon.0) 359 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.193014\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:29:55.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:55 smithi104 bash[9203]: audit 2020-09-08T15:29:55.026074+0000 mon.a (mon.0) 357 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.193014\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:29:55.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:55 smithi104 bash[9203]: audit 2020-09-08T15:29:55.026821+0000 mon.a (mon.0) 358 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:29:55.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:55 smithi104 bash[9203]: audit 2020-09-08T15:29:55.031607+0000 mon.a (mon.0) 359 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.193014\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}}, \"last_device_update\": \"2020-09-08T15:26:46.356296\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:29:56.378 INFO:teuthology.orchestra.run.smithi104:osd.0> sudo journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.0.service 2020-09-08T15:29:56.384 INFO:tasks.cephadm:Deploying osd.1 on smithi104 with /dev/vg_nvme/lv_3... 2020-09-08T15:29:56.385 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2020-09-08T15:29:56.458 INFO:journalctl@ceph.osd.0.smithi104.stdout:-- Logs begin at Tue 2020-09-01 16:49:30 UTC. -- 2020-09-08T15:29:56.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:56 smithi104 bash[9203]: audit 2020-09-08T15:29:55.696557+0000 mon.a (mon.0) 360 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696253\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696447\", \"created\": \"2020-09-08T15:29:55.001341\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}}, \"last_daemon_update\": \"2020-09-08T15:29:55.696661\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:29:56.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:56 smithi104 bash[9203]: audit 2020-09-08T15:29:55.702202+0000 mon.a (mon.0) 361 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696253\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696447\", \"created\": \"2020-09-08T15:29:55.001341\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}}, \"last_daemon_update\": \"2020-09-08T15:29:55.696661\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:29:56.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:29:56 smithi104 bash[9203]: audit 2020-09-08T15:29:55.703520+0000 mon.a (mon.0) 362 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:29:56.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:56 smithi166 bash[9446]: audit 2020-09-08T15:29:55.696557+0000 mon.a (mon.0) 360 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696253\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696447\", \"created\": \"2020-09-08T15:29:55.001341\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}}, \"last_daemon_update\": \"2020-09-08T15:29:55.696661\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:29:56.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:56 smithi166 bash[9446]: audit 2020-09-08T15:29:55.702202+0000 mon.a (mon.0) 361 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696253\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696447\", \"created\": \"2020-09-08T15:29:55.001341\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}}, \"last_daemon_update\": \"2020-09-08T15:29:55.696661\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:29:56.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:29:56 smithi166 bash[9446]: audit 2020-09-08T15:29:55.703520+0000 mon.a (mon.0) 362 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:29:56.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:56 smithi055 bash[9296]: audit 2020-09-08T15:29:55.696557+0000 mon.a (mon.0) 360 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696253\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696447\", \"created\": \"2020-09-08T15:29:55.001341\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}}, \"last_daemon_update\": \"2020-09-08T15:29:55.696661\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:29:56.919 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:56 smithi055 bash[9296]: audit 2020-09-08T15:29:55.702202+0000 mon.a (mon.0) 361 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696253\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696447\", \"created\": \"2020-09-08T15:29:55.001341\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}}, \"last_daemon_update\": \"2020-09-08T15:29:55.696661\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:29:56.919 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:29:56 smithi055 bash[9296]: audit 2020-09-08T15:29:55.703520+0000 mon.a (mon.0) 362 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:29:57.802 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:29:57 smithi104 bash[15273]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2020-09-08T15:29:57.803 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:29:57 smithi104 bash[15273]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-0 --no-mon-config 2020-09-08T15:29:57.803 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:29:57 smithi104 bash[15273]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0/block 2020-09-08T15:29:57.804 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:29:57 smithi104 bash[15273]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-0/block 2020-09-08T15:29:57.804 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:29:57 smithi104 bash[15273]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2020-09-08T15:29:57.804 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:29:57 smithi104 bash[15273]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2020-09-08T15:29:57.804 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:29:57 smithi104 bash[15273]: --> ceph-volume lvm activate successful for osd ID: 0 2020-09-08T15:29:58.806 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:29:58 smithi104 bash[15273]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:30:00.733 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[9203]: cluster 2020-09-08T15:29:59.454561+0000 mgr.x (mgr.24118) 12 : cluster [DBG] pgmap v4: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:30:00.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:00 smithi166 bash[9446]: cluster 2020-09-08T15:29:59.454561+0000 mgr.x (mgr.24118) 12 : cluster [DBG] pgmap v4: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:30:00.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:00 smithi055 bash[9296]: cluster 2020-09-08T15:29:59.454561+0000 mgr.x (mgr.24118) 12 : cluster [DBG] pgmap v4: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:30:01.251 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 --1- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 0x55e987b40000 :-1 s=OPENED pgs=142 cs=1 l=1).handle_message begin MSG 2020-09-08T15:30:01.251 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a80e5a700 10 -- 172.21.15.104:0/763349714 dispatch_throttle_release 278 to dispatch throttler 278/104857600 2020-09-08T15:30:01.251 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a80e5a700 20 -- 172.21.15.104:0/763349714 done calling dispatch on 0x55e987ad6c60 2020-09-08T15:30:01.251 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 -- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 legacy=0x55e987b40000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read start len=53 2020-09-08T15:30:01.252 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 --1- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 0x55e987b40000 :-1 s=OPENED pgs=142 cs=1 l=1).handle_message_header r=0 2020-09-08T15:30:01.252 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 --1- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 0x55e987b40000 :-1 s=OPENED pgs=142 cs=1 l=1).handle_message_header got MSG header 2020-09-08T15:30:01.252 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 --1- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 0x55e987b40000 :-1 s=OPENED pgs=142 cs=1 l=1).handle_message_header got envelope type=18 src mon.0 front=194 data=0 off 0 2020-09-08T15:30:01.252 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 --1- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 0x55e987b40000 :-1 s=THROTTLE_MESSAGE pgs=142 cs=1 l=1).throttle_message 2020-09-08T15:30:01.253 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 --1- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 0x55e987b40000 :-1 s=THROTTLE_BYTES pgs=142 cs=1 l=1).throttle_bytes 2020-09-08T15:30:01.253 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 --1- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 0x55e987b40000 :-1 s=THROTTLE_DISPATCH_QUEUE pgs=142 cs=1 l=1).throttle_dispatch_queue 2020-09-08T15:30:01.253 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 --1- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 0x55e987b40000 :-1 s=READ_MESSAGE_FRONT pgs=142 cs=1 l=1).read_message_front 2020-09-08T15:30:01.253 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 -- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 legacy=0x55e987b40000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read start len=194 2020-09-08T15:30:01.253 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 --1- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 0x55e987b40000 :-1 s=READ_MESSAGE_FRONT pgs=142 cs=1 l=1).handle_message_front r=0 2020-09-08T15:30:01.254 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 --1- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 0x55e987b40000 :-1 s=READ_MESSAGE_FRONT pgs=142 cs=1 l=1).handle_message_front got front 194 2020-09-08T15:30:01.254 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 --1- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 0x55e987b40000 :-1 s=READ_MESSAGE_FRONT pgs=142 cs=1 l=1).read_message_middle 2020-09-08T15:30:01.254 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 --1- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 0x55e987b40000 :-1 s=READ_MESSAGE_FRONT pgs=142 cs=1 l=1).read_message_data_prepare 2020-09-08T15:30:01.254 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 --1- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 0x55e987b40000 :-1 s=READ_MESSAGE_FRONT pgs=142 cs=1 l=1).read_message_data msg_left=0 2020-09-08T15:30:01.254 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 --1- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 0x55e987b40000 :-1 s=READ_MESSAGE_FRONT pgs=142 cs=1 l=1).read_message_footer 2020-09-08T15:30:01.255 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 -- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 legacy=0x55e987b40000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read start len=21 2020-09-08T15:30:01.255 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 --1- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 0x55e987b40000 :-1 s=READ_FOOTER_AND_DISPATCH pgs=142 cs=1 l=1).handle_message_footer r=0 2020-09-08T15:30:01.255 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 10 --1- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 0x55e987b40000 :-1 s=READ_FOOTER_AND_DISPATCH pgs=142 cs=1 l=1).handle_message_footer aborted = 0 2020-09-08T15:30:01.255 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 --1- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 0x55e987b40000 :-1 s=READ_FOOTER_AND_DISPATCH pgs=142 cs=1 l=1).handle_message_footer got 194 + 0 + 0 byte message 2020-09-08T15:30:01.255 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 10 --1- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 0x55e987b40000 :-1 s=READ_FOOTER_AND_DISPATCH pgs=142 cs=1 l=1).handle_message_footer no session security set 2020-09-08T15:30:01.256 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 5 --1- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 0x55e987b40000 :-1 s=READ_FOOTER_AND_DISPATCH pgs=142 cs=1 l=1). rx mon.0 seq 6 0x55e987ae04e0 auth_reply(proto 2 0 (0) Success) v1 2020-09-08T15:30:01.256 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 -- 172.21.15.104:0/763349714 queue 0x55e987ae04e0 prio 196 2020-09-08T15:30:01.256 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 --1- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 0x55e987b40000 :-1 s=OPENED pgs=142 cs=1 l=1).wait_message 2020-09-08T15:30:01.256 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a8165b700 20 -- 172.21.15.104:0/763349714 >> v1:172.21.15.104:6789/0 conn(0x55e987b57000 legacy=0x55e987b40000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read start len=1 2020-09-08T15:30:01.256 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a80e5a700 1 -- 172.21.15.104:0/763349714 <== mon.0 v1:172.21.15.104:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 830458898 0 0) 0x55e987ae04e0 con 0x55e987b57000 2020-09-08T15:30:01.257 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a80e5a700 10 -- 172.21.15.104:0/763349714 dispatch_throttle_release 194 to dispatch throttler 194/104857600 2020-09-08T15:30:01.257 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:00 smithi104 bash[15273]: debug 2020-09-08T15:30:00.733+0000 7f5a80e5a700 20 -- 172.21.15.104:0/763349714 done calling dispatch on 0x55e987ae04e0 2020-09-08T15:30:01.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:01 smithi104 bash[9203]: cluster 2020-09-08T15:30:00.438802+0000 mon.a (mon.0) 363 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2020-09-08T15:30:01.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:01 smithi104 bash[9203]: cluster 2020-09-08T15:30:00.438852+0000 mon.a (mon.0) 364 : cluster [INF] Cluster is now healthy 2020-09-08T15:30:01.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:01 smithi104 bash[15273]: debug 2020-09-08T15:30:01.285+0000 7f5a888daf40 -1 Falling back to public interface 2020-09-08T15:30:01.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:01 smithi166 bash[9446]: cluster 2020-09-08T15:30:00.438802+0000 mon.a (mon.0) 363 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2020-09-08T15:30:01.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:01 smithi166 bash[9446]: cluster 2020-09-08T15:30:00.438852+0000 mon.a (mon.0) 364 : cluster [INF] Cluster is now healthy 2020-09-08T15:30:01.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:01 smithi055 bash[9296]: cluster 2020-09-08T15:30:00.438802+0000 mon.a (mon.0) 363 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2020-09-08T15:30:01.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:01 smithi055 bash[9296]: cluster 2020-09-08T15:30:00.438852+0000 mon.a (mon.0) 364 : cluster [INF] Cluster is now healthy 2020-09-08T15:30:02.177 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:01 smithi104 bash[15273]: debug 2020-09-08T15:30:01.913+0000 7f5a888daf40 -1 osd.0 0 log_to_monitors {default=true} 2020-09-08T15:30:02.527 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_3 2020-09-08T15:30:02.527 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2020-09-08T15:30:02.527 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2020-09-08T15:30:02.528 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2020-09-08T15:30:02.528 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0182852 s, 573 MB/s 2020-09-08T15:30:02.528 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2020-09-08T15:30:02.715 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:02 smithi104 bash[9203]: cluster 2020-09-08T15:30:01.455148+0000 mgr.x (mgr.24118) 13 : cluster [DBG] pgmap v5: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:30:02.715 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:02 smithi104 bash[9203]: audit 2020-09-08T15:30:01.923881+0000 mon.a (mon.0) 365 : audit [INF] from='osd.0 v1:172.21.15.104:6801/82470352' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2020-09-08T15:30:02.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:02 smithi166 bash[9446]: cluster 2020-09-08T15:30:01.455148+0000 mgr.x (mgr.24118) 13 : cluster [DBG] pgmap v5: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:30:02.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:02 smithi166 bash[9446]: audit 2020-09-08T15:30:01.923881+0000 mon.a (mon.0) 365 : audit [INF] from='osd.0 v1:172.21.15.104:6801/82470352' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2020-09-08T15:30:02.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:02 smithi055 bash[9296]: cluster 2020-09-08T15:30:01.455148+0000 mgr.x (mgr.24118) 13 : cluster [DBG] pgmap v5: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:30:02.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:02 smithi055 bash[9296]: audit 2020-09-08T15:30:01.923881+0000 mon.a (mon.0) 365 : audit [INF] from='osd.0 v1:172.21.15.104:6801/82470352' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2020-09-08T15:30:03.526 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph orch daemon add osd smithi104:vg_nvme/lv_3 2020-09-08T15:30:03.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:03 smithi166 bash[9446]: audit 2020-09-08T15:30:02.466099+0000 mon.a (mon.0) 366 : audit [INF] from='osd.0 v1:172.21.15.104:6801/82470352' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2020-09-08T15:30:03.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:03 smithi166 bash[9446]: cluster 2020-09-08T15:30:02.466165+0000 mon.a (mon.0) 367 : cluster [DBG] osdmap e9: 1 total, 0 up, 0 in 2020-09-08T15:30:03.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:03 smithi166 bash[9446]: audit 2020-09-08T15:30:02.466654+0000 mon.a (mon.0) 368 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2020-09-08T15:30:03.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:03 smithi166 bash[9446]: audit 2020-09-08T15:30:02.467739+0000 mon.a (mon.0) 369 : audit [INF] from='osd.0 v1:172.21.15.104:6801/82470352' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2020-09-08T15:30:03.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:03 smithi055 bash[9296]: audit 2020-09-08T15:30:02.466099+0000 mon.a (mon.0) 366 : audit [INF] from='osd.0 v1:172.21.15.104:6801/82470352' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2020-09-08T15:30:03.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:03 smithi055 bash[9296]: cluster 2020-09-08T15:30:02.466165+0000 mon.a (mon.0) 367 : cluster [DBG] osdmap e9: 1 total, 0 up, 0 in 2020-09-08T15:30:03.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:03 smithi055 bash[9296]: audit 2020-09-08T15:30:02.466654+0000 mon.a (mon.0) 368 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2020-09-08T15:30:03.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:03 smithi055 bash[9296]: audit 2020-09-08T15:30:02.467739+0000 mon.a (mon.0) 369 : audit [INF] from='osd.0 v1:172.21.15.104:6801/82470352' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2020-09-08T15:30:03.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:30:03 smithi104 bash[15273]: debug 2020-09-08T15:30:03.473+0000 7f5a80659700 -1 osd.0 0 waiting for initial osdmap 2020-09-08T15:30:03.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:03 smithi104 bash[9203]: audit 2020-09-08T15:30:02.466099+0000 mon.a (mon.0) 366 : audit [INF] from='osd.0 v1:172.21.15.104:6801/82470352' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2020-09-08T15:30:03.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:03 smithi104 bash[9203]: cluster 2020-09-08T15:30:02.466165+0000 mon.a (mon.0) 367 : cluster [DBG] osdmap e9: 1 total, 0 up, 0 in 2020-09-08T15:30:03.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:03 smithi104 bash[9203]: audit 2020-09-08T15:30:02.466654+0000 mon.a (mon.0) 368 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2020-09-08T15:30:03.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:03 smithi104 bash[9203]: audit 2020-09-08T15:30:02.467739+0000 mon.a (mon.0) 369 : audit [INF] from='osd.0 v1:172.21.15.104:6801/82470352' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2020-09-08T15:30:04.758 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:04 smithi104 bash[9203]: cluster 2020-09-08T15:30:03.455657+0000 mgr.x (mgr.24118) 14 : cluster [DBG] pgmap v7: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:30:05.125 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:04 smithi104 bash[9203]: audit 2020-09-08T15:30:03.475474+0000 mon.a (mon.0) 370 : audit [INF] from='osd.0 v1:172.21.15.104:6801/82470352' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2020-09-08T15:30:05.126 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:04 smithi104 bash[9203]: cluster 2020-09-08T15:30:03.475576+0000 mon.a (mon.0) 371 : cluster [DBG] osdmap e10: 1 total, 0 up, 0 in 2020-09-08T15:30:05.126 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:04 smithi104 bash[9203]: audit 2020-09-08T15:30:03.475997+0000 mon.a (mon.0) 372 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2020-09-08T15:30:05.127 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:04 smithi104 bash[9203]: audit 2020-09-08T15:30:03.481469+0000 mon.a (mon.0) 373 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2020-09-08T15:30:05.127 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:04 smithi104 bash[9203]: audit 2020-09-08T15:30:03.666118+0000 mon.a (mon.0) 374 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:30:05.127 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:04 smithi104 bash[9203]: audit 2020-09-08T15:30:03.942477+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696253\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696447\", \"created\": \"2020-09-08T15:29:55.001341\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}}, \"last_daemon_update\": \"2020-09-08T15:29:55.696661\", \"last_device_update\": \"2020-09-08T15:30:03.942759\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:05.128 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:04 smithi104 bash[9203]: audit 2020-09-08T15:30:03.947785+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696253\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696447\", \"created\": \"2020-09-08T15:29:55.001341\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}}, \"last_daemon_update\": \"2020-09-08T15:29:55.696661\", \"last_device_update\": \"2020-09-08T15:30:03.942759\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:05.128 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:04 smithi104 bash[9203]: audit 2020-09-08T15:30:03.949344+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:30:05.129 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:04 smithi104 bash[9203]: audit 2020-09-08T15:30:03.955402+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:30:05.129 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:04 smithi104 bash[9203]: audit 2020-09-08T15:30:03.958371+0000 mon.a (mon.0) 379 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:30:05.131 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:04 smithi104 bash[9203]: audit 2020-09-08T15:30:03.964754+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:30:05.133 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:04 smithi055 bash[9296]: cluster 2020-09-08T15:30:03.455657+0000 mgr.x (mgr.24118) 14 : cluster [DBG] pgmap v7: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:30:05.134 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:04 smithi055 bash[9296]: audit 2020-09-08T15:30:03.475474+0000 mon.a (mon.0) 370 : audit [INF] from='osd.0 v1:172.21.15.104:6801/82470352' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2020-09-08T15:30:05.134 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:04 smithi055 bash[9296]: cluster 2020-09-08T15:30:03.475576+0000 mon.a (mon.0) 371 : cluster [DBG] osdmap e10: 1 total, 0 up, 0 in 2020-09-08T15:30:05.135 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:04 smithi055 bash[9296]: audit 2020-09-08T15:30:03.475997+0000 mon.a (mon.0) 372 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2020-09-08T15:30:05.135 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:04 smithi055 bash[9296]: audit 2020-09-08T15:30:03.481469+0000 mon.a (mon.0) 373 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2020-09-08T15:30:05.135 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:04 smithi055 bash[9296]: audit 2020-09-08T15:30:03.666118+0000 mon.a (mon.0) 374 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:30:05.136 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:04 smithi055 bash[9296]: audit 2020-09-08T15:30:03.942477+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696253\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696447\", \"created\": \"2020-09-08T15:29:55.001341\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}}, \"last_daemon_update\": \"2020-09-08T15:29:55.696661\", \"last_device_update\": \"2020-09-08T15:30:03.942759\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:05.136 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:04 smithi055 bash[9296]: audit 2020-09-08T15:30:03.947785+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696253\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696447\", \"created\": \"2020-09-08T15:29:55.001341\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}}, \"last_daemon_update\": \"2020-09-08T15:29:55.696661\", \"last_device_update\": \"2020-09-08T15:30:03.942759\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:05.136 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:04 smithi055 bash[9296]: audit 2020-09-08T15:30:03.949344+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:30:05.137 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:04 smithi055 bash[9296]: audit 2020-09-08T15:30:03.955402+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:30:05.137 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:04 smithi055 bash[9296]: audit 2020-09-08T15:30:03.958371+0000 mon.a (mon.0) 379 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:30:05.138 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:04 smithi055 bash[9296]: audit 2020-09-08T15:30:03.964754+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:30:05.138 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:04 smithi166 bash[9446]: cluster 2020-09-08T15:30:03.455657+0000 mgr.x (mgr.24118) 14 : cluster [DBG] pgmap v7: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:30:05.139 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:04 smithi166 bash[9446]: audit 2020-09-08T15:30:03.475474+0000 mon.a (mon.0) 370 : audit [INF] from='osd.0 v1:172.21.15.104:6801/82470352' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2020-09-08T15:30:05.139 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:04 smithi166 bash[9446]: cluster 2020-09-08T15:30:03.475576+0000 mon.a (mon.0) 371 : cluster [DBG] osdmap e10: 1 total, 0 up, 0 in 2020-09-08T15:30:05.139 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:04 smithi166 bash[9446]: audit 2020-09-08T15:30:03.475997+0000 mon.a (mon.0) 372 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2020-09-08T15:30:05.140 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:04 smithi166 bash[9446]: audit 2020-09-08T15:30:03.481469+0000 mon.a (mon.0) 373 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2020-09-08T15:30:05.140 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:04 smithi166 bash[9446]: audit 2020-09-08T15:30:03.666118+0000 mon.a (mon.0) 374 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:30:05.141 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:04 smithi166 bash[9446]: audit 2020-09-08T15:30:03.942477+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696253\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696447\", \"created\": \"2020-09-08T15:29:55.001341\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}}, \"last_daemon_update\": \"2020-09-08T15:29:55.696661\", \"last_device_update\": \"2020-09-08T15:30:03.942759\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:05.142 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:04 smithi166 bash[9446]: audit 2020-09-08T15:30:03.947785+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696253\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696447\", \"created\": \"2020-09-08T15:29:55.001341\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}}, \"last_daemon_update\": \"2020-09-08T15:29:55.696661\", \"last_device_update\": \"2020-09-08T15:30:03.942759\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:05.142 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:04 smithi166 bash[9446]: audit 2020-09-08T15:30:03.949344+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:30:05.143 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:04 smithi166 bash[9446]: audit 2020-09-08T15:30:03.955402+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:30:05.144 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:04 smithi166 bash[9446]: audit 2020-09-08T15:30:03.958371+0000 mon.a (mon.0) 379 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:30:05.144 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:04 smithi166 bash[9446]: audit 2020-09-08T15:30:03.964754+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:30:05.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:05 smithi166 bash[9446]: cluster 2020-09-08T15:30:02.891132+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:30:05.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:05 smithi166 bash[9446]: cluster 2020-09-08T15:30:02.891321+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:30:05.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:05 smithi166 bash[9446]: audit 2020-09-08T15:30:04.481158+0000 mon.a (mon.0) 381 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2020-09-08T15:30:05.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:05 smithi166 bash[9446]: cluster 2020-09-08T15:30:04.504582+0000 mon.a (mon.0) 382 : cluster [INF] osd.0 v1:172.21.15.104:6801/82470352 boot 2020-09-08T15:30:05.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:05 smithi166 bash[9446]: cluster 2020-09-08T15:30:04.504615+0000 mon.a (mon.0) 383 : cluster [DBG] osdmap e11: 1 total, 1 up, 1 in 2020-09-08T15:30:05.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:05 smithi166 bash[9446]: audit 2020-09-08T15:30:04.504783+0000 mon.a (mon.0) 384 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2020-09-08T15:30:05.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:05 smithi055 bash[9296]: cluster 2020-09-08T15:30:02.891132+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:30:05.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:05 smithi055 bash[9296]: cluster 2020-09-08T15:30:02.891321+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:30:05.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:05 smithi055 bash[9296]: audit 2020-09-08T15:30:04.481158+0000 mon.a (mon.0) 381 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2020-09-08T15:30:05.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:05 smithi055 bash[9296]: cluster 2020-09-08T15:30:04.504582+0000 mon.a (mon.0) 382 : cluster [INF] osd.0 v1:172.21.15.104:6801/82470352 boot 2020-09-08T15:30:05.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:05 smithi055 bash[9296]: cluster 2020-09-08T15:30:04.504615+0000 mon.a (mon.0) 383 : cluster [DBG] osdmap e11: 1 total, 1 up, 1 in 2020-09-08T15:30:05.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:05 smithi055 bash[9296]: audit 2020-09-08T15:30:04.504783+0000 mon.a (mon.0) 384 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2020-09-08T15:30:05.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:05 smithi104 bash[9203]: cluster 2020-09-08T15:30:02.891132+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:30:05.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:05 smithi104 bash[9203]: cluster 2020-09-08T15:30:02.891321+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:30:05.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:05 smithi104 bash[9203]: audit 2020-09-08T15:30:04.481158+0000 mon.a (mon.0) 381 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2020-09-08T15:30:05.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:05 smithi104 bash[9203]: cluster 2020-09-08T15:30:04.504582+0000 mon.a (mon.0) 382 : cluster [INF] osd.0 v1:172.21.15.104:6801/82470352 boot 2020-09-08T15:30:05.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:05 smithi104 bash[9203]: cluster 2020-09-08T15:30:04.504615+0000 mon.a (mon.0) 383 : cluster [DBG] osdmap e11: 1 total, 1 up, 1 in 2020-09-08T15:30:05.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:05 smithi104 bash[9203]: audit 2020-09-08T15:30:04.504783+0000 mon.a (mon.0) 384 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2020-09-08T15:30:06.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:06 smithi166 bash[9446]: cluster 2020-09-08T15:30:05.456106+0000 mgr.x (mgr.24118) 15 : cluster [DBG] pgmap v10: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:30:06.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:06 smithi166 bash[9446]: cluster 2020-09-08T15:30:05.511371+0000 mon.a (mon.0) 385 : cluster [DBG] osdmap e12: 1 total, 1 up, 1 in 2020-09-08T15:30:06.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:06 smithi055 bash[9296]: cluster 2020-09-08T15:30:05.456106+0000 mgr.x (mgr.24118) 15 : cluster [DBG] pgmap v10: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:30:06.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:06 smithi055 bash[9296]: cluster 2020-09-08T15:30:05.511371+0000 mon.a (mon.0) 385 : cluster [DBG] osdmap e12: 1 total, 1 up, 1 in 2020-09-08T15:30:06.935 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:06 smithi104 bash[9203]: cluster 2020-09-08T15:30:05.456106+0000 mgr.x (mgr.24118) 15 : cluster [DBG] pgmap v10: 1 pgs: 1 unknown; 0 B data, 0 B used, 0 B / 0 B avail 2020-09-08T15:30:06.935 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:06 smithi104 bash[9203]: cluster 2020-09-08T15:30:05.511371+0000 mon.a (mon.0) 385 : cluster [DBG] osdmap e12: 1 total, 1 up, 1 in 2020-09-08T15:30:07.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:07 smithi166 bash[9446]: cluster 2020-09-08T15:30:06.515811+0000 mon.a (mon.0) 386 : cluster [DBG] osdmap e13: 1 total, 1 up, 1 in 2020-09-08T15:30:07.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:07 smithi166 bash[9446]: audit 2020-09-08T15:30:07.048141+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:30:07.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:07 smithi166 bash[9446]: audit 2020-09-08T15:30:07.051773+0000 mon.a (mon.0) 388 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:30:07.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:07 smithi166 bash[9446]: audit 2020-09-08T15:30:07.053599+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:30:07.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:07 smithi166 bash[9446]: audit 2020-09-08T15:30:07.055651+0000 mon.a (mon.0) 390 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:30:07.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:07 smithi055 bash[9296]: cluster 2020-09-08T15:30:06.515811+0000 mon.a (mon.0) 386 : cluster [DBG] osdmap e13: 1 total, 1 up, 1 in 2020-09-08T15:30:07.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:07 smithi055 bash[9296]: audit 2020-09-08T15:30:07.048141+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:30:07.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:07 smithi055 bash[9296]: audit 2020-09-08T15:30:07.051773+0000 mon.a (mon.0) 388 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:30:07.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:07 smithi055 bash[9296]: audit 2020-09-08T15:30:07.053599+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:30:07.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:07 smithi055 bash[9296]: audit 2020-09-08T15:30:07.055651+0000 mon.a (mon.0) 390 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:30:07.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:07 smithi104 bash[9203]: cluster 2020-09-08T15:30:06.515811+0000 mon.a (mon.0) 386 : cluster [DBG] osdmap e13: 1 total, 1 up, 1 in 2020-09-08T15:30:07.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:07 smithi104 bash[9203]: audit 2020-09-08T15:30:07.048141+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:30:07.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:07 smithi104 bash[9203]: audit 2020-09-08T15:30:07.051773+0000 mon.a (mon.0) 388 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:30:07.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:07 smithi104 bash[9203]: audit 2020-09-08T15:30:07.053599+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:30:07.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:07 smithi104 bash[9203]: audit 2020-09-08T15:30:07.055651+0000 mon.a (mon.0) 390 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:30:08.612 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:08 smithi104 bash[9203]: audit 2020-09-08T15:30:07.047725+0000 mgr.x (mgr.24118) 16 : audit [DBG] from='client.14259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:30:08.613 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:08 smithi104 bash[9203]: cephadm 2020-09-08T15:30:07.051379+0000 mgr.x (mgr.24118) 17 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:30:08.613 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:08 smithi104 bash[9203]: cephadm 2020-09-08T15:30:07.051560+0000 mgr.x (mgr.24118) 18 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:30:08.613 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:08 smithi104 bash[9203]: cephadm 2020-09-08T15:30:07.052507+0000 mgr.x (mgr.24118) 19 : cephadm [INF] Applying None on host smithi104... 2020-09-08T15:30:08.613 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:08 smithi104 bash[9203]: cluster 2020-09-08T15:30:07.456679+0000 mgr.x (mgr.24118) 20 : cluster [DBG] pgmap v13: 1 pgs: 1 creating+peering; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:08.614 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:08 smithi104 bash[9203]: cluster 2020-09-08T15:30:07.519302+0000 mon.a (mon.0) 391 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2020-09-08T15:30:08.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:08 smithi166 bash[9446]: audit 2020-09-08T15:30:07.047725+0000 mgr.x (mgr.24118) 16 : audit [DBG] from='client.14259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:30:08.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:08 smithi166 bash[9446]: cephadm 2020-09-08T15:30:07.051379+0000 mgr.x (mgr.24118) 17 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:30:08.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:08 smithi166 bash[9446]: cephadm 2020-09-08T15:30:07.051560+0000 mgr.x (mgr.24118) 18 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:30:08.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:08 smithi166 bash[9446]: cephadm 2020-09-08T15:30:07.052507+0000 mgr.x (mgr.24118) 19 : cephadm [INF] Applying None on host smithi104... 2020-09-08T15:30:08.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:08 smithi166 bash[9446]: cluster 2020-09-08T15:30:07.456679+0000 mgr.x (mgr.24118) 20 : cluster [DBG] pgmap v13: 1 pgs: 1 creating+peering; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:08.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:08 smithi166 bash[9446]: cluster 2020-09-08T15:30:07.519302+0000 mon.a (mon.0) 391 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2020-09-08T15:30:08.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:08 smithi055 bash[9296]: audit 2020-09-08T15:30:07.047725+0000 mgr.x (mgr.24118) 16 : audit [DBG] from='client.14259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:30:08.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:08 smithi055 bash[9296]: cephadm 2020-09-08T15:30:07.051379+0000 mgr.x (mgr.24118) 17 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:30:08.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:08 smithi055 bash[9296]: cephadm 2020-09-08T15:30:07.051560+0000 mgr.x (mgr.24118) 18 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:30:08.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:08 smithi055 bash[9296]: cephadm 2020-09-08T15:30:07.052507+0000 mgr.x (mgr.24118) 19 : cephadm [INF] Applying None on host smithi104... 2020-09-08T15:30:08.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:08 smithi055 bash[9296]: cluster 2020-09-08T15:30:07.456679+0000 mgr.x (mgr.24118) 20 : cluster [DBG] pgmap v13: 1 pgs: 1 creating+peering; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:08.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:08 smithi055 bash[9296]: cluster 2020-09-08T15:30:07.519302+0000 mon.a (mon.0) 391 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2020-09-08T15:30:10.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:10 smithi166 bash[9446]: cluster 2020-09-08T15:30:09.457229+0000 mgr.x (mgr.24118) 21 : cluster [DBG] pgmap v14: 1 pgs: 1 creating+peering; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:10.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:10 smithi055 bash[9296]: cluster 2020-09-08T15:30:09.457229+0000 mgr.x (mgr.24118) 21 : cluster [DBG] pgmap v14: 1 pgs: 1 creating+peering; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:10.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:10 smithi104 bash[9203]: cluster 2020-09-08T15:30:09.457229+0000 mgr.x (mgr.24118) 21 : cluster [DBG] pgmap v14: 1 pgs: 1 creating+peering; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:11.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:11 smithi166 bash[9446]: audit 2020-09-08T15:30:10.848324+0000 mon.a (mon.0) 392 : audit [INF] from='client.? v1:172.21.15.104:0/2013105910' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a83df412-110d-440a-b411-ad2a7c36f81a"}]: dispatch 2020-09-08T15:30:11.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:11 smithi166 bash[9446]: audit 2020-09-08T15:30:10.854892+0000 mon.a (mon.0) 393 : audit [INF] from='client.? v1:172.21.15.104:0/2013105910' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a83df412-110d-440a-b411-ad2a7c36f81a"}]': finished 2020-09-08T15:30:11.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:11 smithi166 bash[9446]: cluster 2020-09-08T15:30:10.854963+0000 mon.a (mon.0) 394 : cluster [DBG] osdmap e14: 2 total, 1 up, 1 in 2020-09-08T15:30:11.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:11 smithi166 bash[9446]: audit 2020-09-08T15:30:10.855196+0000 mon.a (mon.0) 395 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2020-09-08T15:30:11.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:11 smithi055 bash[9296]: audit 2020-09-08T15:30:10.848324+0000 mon.a (mon.0) 392 : audit [INF] from='client.? v1:172.21.15.104:0/2013105910' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a83df412-110d-440a-b411-ad2a7c36f81a"}]: dispatch 2020-09-08T15:30:11.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:11 smithi055 bash[9296]: audit 2020-09-08T15:30:10.854892+0000 mon.a (mon.0) 393 : audit [INF] from='client.? v1:172.21.15.104:0/2013105910' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a83df412-110d-440a-b411-ad2a7c36f81a"}]': finished 2020-09-08T15:30:11.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:11 smithi055 bash[9296]: cluster 2020-09-08T15:30:10.854963+0000 mon.a (mon.0) 394 : cluster [DBG] osdmap e14: 2 total, 1 up, 1 in 2020-09-08T15:30:11.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:11 smithi055 bash[9296]: audit 2020-09-08T15:30:10.855196+0000 mon.a (mon.0) 395 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2020-09-08T15:30:11.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:11 smithi104 bash[9203]: audit 2020-09-08T15:30:10.848324+0000 mon.a (mon.0) 392 : audit [INF] from='client.? v1:172.21.15.104:0/2013105910' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a83df412-110d-440a-b411-ad2a7c36f81a"}]: dispatch 2020-09-08T15:30:11.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:11 smithi104 bash[9203]: audit 2020-09-08T15:30:10.854892+0000 mon.a (mon.0) 393 : audit [INF] from='client.? v1:172.21.15.104:0/2013105910' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a83df412-110d-440a-b411-ad2a7c36f81a"}]': finished 2020-09-08T15:30:11.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:11 smithi104 bash[9203]: cluster 2020-09-08T15:30:10.854963+0000 mon.a (mon.0) 394 : cluster [DBG] osdmap e14: 2 total, 1 up, 1 in 2020-09-08T15:30:11.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:11 smithi104 bash[9203]: audit 2020-09-08T15:30:10.855196+0000 mon.a (mon.0) 395 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2020-09-08T15:30:12.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:12 smithi166 bash[9446]: cluster 2020-09-08T15:30:11.457763+0000 mgr.x (mgr.24118) 22 : cluster [DBG] pgmap v16: 1 pgs: 1 undersized+peered; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:12.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:12 smithi166 bash[9446]: audit 2020-09-08T15:30:11.484251+0000 mon.a (mon.0) 396 : audit [DBG] from='client.? v1:172.21.15.104:0/3650743947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:30:12.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:12 smithi055 bash[9296]: cluster 2020-09-08T15:30:11.457763+0000 mgr.x (mgr.24118) 22 : cluster [DBG] pgmap v16: 1 pgs: 1 undersized+peered; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:12.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:12 smithi055 bash[9296]: audit 2020-09-08T15:30:11.484251+0000 mon.a (mon.0) 396 : audit [DBG] from='client.? v1:172.21.15.104:0/3650743947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:30:12.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:12 smithi104 bash[9203]: cluster 2020-09-08T15:30:11.457763+0000 mgr.x (mgr.24118) 22 : cluster [DBG] pgmap v16: 1 pgs: 1 undersized+peered; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:12.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:12 smithi104 bash[9203]: audit 2020-09-08T15:30:11.484251+0000 mon.a (mon.0) 396 : audit [DBG] from='client.? v1:172.21.15.104:0/3650743947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:30:14.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:14 smithi166 bash[9446]: cluster 2020-09-08T15:30:13.458322+0000 mgr.x (mgr.24118) 23 : cluster [DBG] pgmap v17: 1 pgs: 1 undersized+peered; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:14.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:14 smithi166 bash[9446]: cluster 2020-09-08T15:30:14.440327+0000 mon.a (mon.0) 397 : cluster [WRN] Health check update: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2020-09-08T15:30:14.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:14 smithi055 bash[9296]: cluster 2020-09-08T15:30:13.458322+0000 mgr.x (mgr.24118) 23 : cluster [DBG] pgmap v17: 1 pgs: 1 undersized+peered; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:14.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:14 smithi055 bash[9296]: cluster 2020-09-08T15:30:14.440327+0000 mon.a (mon.0) 397 : cluster [WRN] Health check update: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2020-09-08T15:30:14.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:14 smithi104 bash[9203]: cluster 2020-09-08T15:30:13.458322+0000 mgr.x (mgr.24118) 23 : cluster [DBG] pgmap v17: 1 pgs: 1 undersized+peered; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:14.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:14 smithi104 bash[9203]: cluster 2020-09-08T15:30:14.440327+0000 mon.a (mon.0) 397 : cluster [WRN] Health check update: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2020-09-08T15:30:15.730 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:15 smithi104 bash[9203]: audit 2020-09-08T15:30:14.469204+0000 mon.a (mon.0) 398 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:30:15.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:15 smithi166 bash[9446]: audit 2020-09-08T15:30:14.469204+0000 mon.a (mon.0) 398 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:30:15.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:15 smithi055 bash[9296]: audit 2020-09-08T15:30:14.469204+0000 mon.a (mon.0) 398 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:30:16.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:16 smithi166 bash[9446]: cluster 2020-09-08T15:30:15.458870+0000 mgr.x (mgr.24118) 24 : cluster [DBG] pgmap v18: 1 pgs: 1 undersized+peered; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:16.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:16 smithi055 bash[9296]: cluster 2020-09-08T15:30:15.458870+0000 mgr.x (mgr.24118) 24 : cluster [DBG] pgmap v18: 1 pgs: 1 undersized+peered; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:16.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:16 smithi104 bash[9203]: cluster 2020-09-08T15:30:15.458870+0000 mgr.x (mgr.24118) 24 : cluster [DBG] pgmap v18: 1 pgs: 1 undersized+peered; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:18.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:18 smithi166 bash[9446]: cluster 2020-09-08T15:30:17.459468+0000 mgr.x (mgr.24118) 25 : cluster [DBG] pgmap v19: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:18.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:18 smithi055 bash[9296]: cluster 2020-09-08T15:30:17.459468+0000 mgr.x (mgr.24118) 25 : cluster [DBG] pgmap v19: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:18.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:18 smithi104 bash[9203]: cluster 2020-09-08T15:30:17.459468+0000 mgr.x (mgr.24118) 25 : cluster [DBG] pgmap v19: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:19.841 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:19 smithi104 bash[9203]: audit 2020-09-08T15:30:18.549936+0000 mon.a (mon.0) 399 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2020-09-08T15:30:19.841 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:19 smithi104 bash[9203]: audit 2020-09-08T15:30:18.552020+0000 mon.a (mon.0) 400 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:30:19.842 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:19 smithi104 bash[9203]: audit 2020-09-08T15:30:18.554438+0000 mon.a (mon.0) 401 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.1", "key": "container_image"}]: dispatch 2020-09-08T15:30:19.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:19 smithi166 bash[9446]: audit 2020-09-08T15:30:18.549936+0000 mon.a (mon.0) 399 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2020-09-08T15:30:19.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:19 smithi166 bash[9446]: audit 2020-09-08T15:30:18.552020+0000 mon.a (mon.0) 400 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:30:19.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:19 smithi166 bash[9446]: audit 2020-09-08T15:30:18.554438+0000 mon.a (mon.0) 401 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.1", "key": "container_image"}]: dispatch 2020-09-08T15:30:19.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:19 smithi055 bash[9296]: audit 2020-09-08T15:30:18.549936+0000 mon.a (mon.0) 399 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2020-09-08T15:30:19.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:19 smithi055 bash[9296]: audit 2020-09-08T15:30:18.552020+0000 mon.a (mon.0) 400 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:30:19.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:19 smithi055 bash[9296]: audit 2020-09-08T15:30:18.554438+0000 mon.a (mon.0) 401 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.1", "key": "container_image"}]: dispatch 2020-09-08T15:30:20.811 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:20 smithi104 bash[9203]: cephadm 2020-09-08T15:30:18.555354+0000 mgr.x (mgr.24118) 26 : cephadm [INF] Deploying daemon osd.1 on smithi104 2020-09-08T15:30:20.813 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:20 smithi104 bash[9203]: cluster 2020-09-08T15:30:19.459931+0000 mgr.x (mgr.24118) 27 : cluster [DBG] pgmap v20: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:20.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:20 smithi166 bash[9446]: cephadm 2020-09-08T15:30:18.555354+0000 mgr.x (mgr.24118) 26 : cephadm [INF] Deploying daemon osd.1 on smithi104 2020-09-08T15:30:20.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:20 smithi166 bash[9446]: cluster 2020-09-08T15:30:19.459931+0000 mgr.x (mgr.24118) 27 : cluster [DBG] pgmap v20: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:20.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:20 smithi055 bash[9296]: cephadm 2020-09-08T15:30:18.555354+0000 mgr.x (mgr.24118) 26 : cephadm [INF] Deploying daemon osd.1 on smithi104 2020-09-08T15:30:20.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:20 smithi055 bash[9296]: cluster 2020-09-08T15:30:19.459931+0000 mgr.x (mgr.24118) 27 : cluster [DBG] pgmap v20: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:21.009 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 1 on host 'smithi104' 2020-09-08T15:30:21.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:21 smithi166 bash[9446]: audit 2020-09-08T15:30:21.004937+0000 mon.a (mon.0) 402 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696253\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696447\", \"created\": \"2020-09-08T15:29:55.001341\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}}, \"last_device_update\": \"2020-09-08T15:30:03.942759\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:21.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:21 smithi166 bash[9446]: audit 2020-09-08T15:30:21.005706+0000 mon.a (mon.0) 403 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:30:21.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:21 smithi166 bash[9446]: audit 2020-09-08T15:30:21.009827+0000 mon.a (mon.0) 404 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696253\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696447\", \"created\": \"2020-09-08T15:29:55.001341\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}}, \"last_device_update\": \"2020-09-08T15:30:03.942759\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:21.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:21 smithi055 bash[9296]: audit 2020-09-08T15:30:21.004937+0000 mon.a (mon.0) 402 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696253\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696447\", \"created\": \"2020-09-08T15:29:55.001341\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}}, \"last_device_update\": \"2020-09-08T15:30:03.942759\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:21.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:21 smithi055 bash[9296]: audit 2020-09-08T15:30:21.005706+0000 mon.a (mon.0) 403 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:30:21.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:21 smithi055 bash[9296]: audit 2020-09-08T15:30:21.009827+0000 mon.a (mon.0) 404 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696253\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696447\", \"created\": \"2020-09-08T15:29:55.001341\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}}, \"last_device_update\": \"2020-09-08T15:30:03.942759\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:21.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:21 smithi104 bash[9203]: audit 2020-09-08T15:30:21.004937+0000 mon.a (mon.0) 402 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696253\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696447\", \"created\": \"2020-09-08T15:29:55.001341\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}}, \"last_device_update\": \"2020-09-08T15:30:03.942759\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:21.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:21 smithi104 bash[9203]: audit 2020-09-08T15:30:21.005706+0000 mon.a (mon.0) 403 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:30:21.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:21 smithi104 bash[9203]: audit 2020-09-08T15:30:21.009827+0000 mon.a (mon.0) 404 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696253\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:55.696447\", \"created\": \"2020-09-08T15:29:55.001341\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}}, \"last_device_update\": \"2020-09-08T15:30:03.942759\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:22.347 INFO:teuthology.orchestra.run.smithi104:osd.1> sudo journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.1.service 2020-09-08T15:30:22.350 INFO:tasks.cephadm:Deploying osd.2 on smithi104 with /dev/vg_nvme/lv_2... 2020-09-08T15:30:22.351 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2020-09-08T15:30:22.373 INFO:journalctl@ceph.osd.1.smithi104.stdout:-- Logs begin at Tue 2020-09-01 16:49:30 UTC. -- 2020-09-08T15:30:22.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:22 smithi104 bash[9203]: cluster 2020-09-08T15:30:21.460485+0000 mgr.x (mgr.24118) 28 : cluster [DBG] pgmap v21: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:22.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:22 smithi104 bash[9203]: audit 2020-09-08T15:30:21.667973+0000 mon.a (mon.0) 405 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667305\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667699\", \"created\": \"2020-09-08T15:30:20.977340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}}, \"last_daemon_update\": \"2020-09-08T15:30:21.667959\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:22.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:22 smithi104 bash[9203]: audit 2020-09-08T15:30:21.673841+0000 mon.a (mon.0) 406 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667305\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667699\", \"created\": \"2020-09-08T15:30:20.977340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}}, \"last_daemon_update\": \"2020-09-08T15:30:21.667959\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:22.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:22 smithi104 bash[9203]: audit 2020-09-08T15:30:21.675195+0000 mon.a (mon.0) 407 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:30:22.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:22 smithi166 bash[9446]: cluster 2020-09-08T15:30:21.460485+0000 mgr.x (mgr.24118) 28 : cluster [DBG] pgmap v21: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:22.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:22 smithi166 bash[9446]: audit 2020-09-08T15:30:21.667973+0000 mon.a (mon.0) 405 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667305\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667699\", \"created\": \"2020-09-08T15:30:20.977340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}}, \"last_daemon_update\": \"2020-09-08T15:30:21.667959\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:22.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:22 smithi166 bash[9446]: audit 2020-09-08T15:30:21.673841+0000 mon.a (mon.0) 406 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667305\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667699\", \"created\": \"2020-09-08T15:30:20.977340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}}, \"last_daemon_update\": \"2020-09-08T15:30:21.667959\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:22.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:22 smithi166 bash[9446]: audit 2020-09-08T15:30:21.675195+0000 mon.a (mon.0) 407 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:30:22.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:22 smithi055 bash[9296]: cluster 2020-09-08T15:30:21.460485+0000 mgr.x (mgr.24118) 28 : cluster [DBG] pgmap v21: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:22.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:22 smithi055 bash[9296]: audit 2020-09-08T15:30:21.667973+0000 mon.a (mon.0) 405 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667305\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667699\", \"created\": \"2020-09-08T15:30:20.977340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}}, \"last_daemon_update\": \"2020-09-08T15:30:21.667959\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:22.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:22 smithi055 bash[9296]: audit 2020-09-08T15:30:21.673841+0000 mon.a (mon.0) 406 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667305\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667699\", \"created\": \"2020-09-08T15:30:20.977340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}}, \"last_daemon_update\": \"2020-09-08T15:30:21.667959\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:22.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:22 smithi055 bash[9296]: audit 2020-09-08T15:30:21.675195+0000 mon.a (mon.0) 407 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:30:24.018 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:30:23 smithi104 bash[17588]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2020-09-08T15:30:24.018 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:30:23 smithi104 bash[17588]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-1 --no-mon-config 2020-09-08T15:30:24.018 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:30:23 smithi104 bash[17588]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-1/block 2020-09-08T15:30:24.018 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:30:23 smithi104 bash[17588]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-1/block 2020-09-08T15:30:24.019 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:30:23 smithi104 bash[17588]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2020-09-08T15:30:24.019 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:30:23 smithi104 bash[17588]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2020-09-08T15:30:24.019 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:30:23 smithi104 bash[17588]: --> ceph-volume lvm activate successful for osd ID: 1 2020-09-08T15:30:24.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:24 smithi166 bash[9446]: cluster 2020-09-08T15:30:23.461055+0000 mgr.x (mgr.24118) 29 : cluster [DBG] pgmap v22: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:24.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:24 smithi055 bash[9296]: cluster 2020-09-08T15:30:23.461055+0000 mgr.x (mgr.24118) 29 : cluster [DBG] pgmap v22: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:24.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:24 smithi104 bash[9203]: cluster 2020-09-08T15:30:23.461055+0000 mgr.x (mgr.24118) 29 : cluster [DBG] pgmap v22: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:25.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:30:24 smithi104 bash[17588]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:30:26.855 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:26 smithi104 bash[9203]: cluster 2020-09-08T15:30:25.461712+0000 mgr.x (mgr.24118) 30 : cluster [DBG] pgmap v23: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:26.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:26 smithi166 bash[9446]: cluster 2020-09-08T15:30:25.461712+0000 mgr.x (mgr.24118) 30 : cluster [DBG] pgmap v23: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:26.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:26 smithi055 bash[9296]: cluster 2020-09-08T15:30:25.461712+0000 mgr.x (mgr.24118) 30 : cluster [DBG] pgmap v23: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:27.139 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:30:26 smithi104 bash[17588]: debug 2020-09-08T15:30:26.993+0000 7fc3a4798700 10 --1- 172.21.15.104:0/3487839291 >> v1:172.21.15.104:6789/0 conn(0x56328f562c00 0x56328f54b800 :-1 s=READ_FOOTER_AND_DISPATCH pgs=154 cs=1 l=1).handle_message_footer no session security set 2020-09-08T15:30:27.140 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:30:26 smithi104 bash[17588]: debug 2020-09-08T15:30:26.993+0000 7fc3a4798700 5 --1- 172.21.15.104:0/3487839291 >> v1:172.21.15.104:6789/0 conn(0x56328f562c00 0x56328f54b800 :-1 s=READ_FOOTER_AND_DISPATCH pgs=154 cs=1 l=1). rx mon.0 seq 6 0x56328f4ec4e0 auth_reply(proto 2 0 (0) Success) v1 2020-09-08T15:30:27.140 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:30:26 smithi104 bash[17588]: debug 2020-09-08T15:30:26.993+0000 7fc3a4798700 20 -- 172.21.15.104:0/3487839291 queue 0x56328f4ec4e0 prio 196 2020-09-08T15:30:27.140 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:30:26 smithi104 bash[17588]: debug 2020-09-08T15:30:26.993+0000 7fc3a4798700 20 --1- 172.21.15.104:0/3487839291 >> v1:172.21.15.104:6789/0 conn(0x56328f562c00 0x56328f54b800 :-1 s=OPENED pgs=154 cs=1 l=1).wait_message 2020-09-08T15:30:27.140 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:30:26 smithi104 bash[17588]: debug 2020-09-08T15:30:26.993+0000 7fc3a3796700 1 -- 172.21.15.104:0/3487839291 <== mon.0 v1:172.21.15.104:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 2961516306 0 0) 0x56328f4ec4e0 con 0x56328f562c00 2020-09-08T15:30:27.140 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:30:26 smithi104 bash[17588]: debug 2020-09-08T15:30:26.993+0000 7fc3a4798700 20 -- 172.21.15.104:0/3487839291 >> v1:172.21.15.104:6789/0 conn(0x56328f562c00 legacy=0x56328f54b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read start len=1 2020-09-08T15:30:27.141 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:30:26 smithi104 bash[17588]: debug 2020-09-08T15:30:26.993+0000 7fc3a3796700 10 -- 172.21.15.104:0/3487839291 dispatch_throttle_release 194 to dispatch throttler 194/104857600 2020-09-08T15:30:27.141 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:30:26 smithi104 bash[17588]: debug 2020-09-08T15:30:26.993+0000 7fc3a3796700 20 -- 172.21.15.104:0/3487839291 done calling dispatch on 0x56328f4ec4e0 2020-09-08T15:30:27.795 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_2 2020-09-08T15:30:27.796 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2020-09-08T15:30:27.796 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2020-09-08T15:30:27.796 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2020-09-08T15:30:27.797 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0139876 s, 750 MB/s 2020-09-08T15:30:27.797 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2020-09-08T15:30:27.886 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:30:27 smithi104 bash[17588]: debug 2020-09-08T15:30:27.541+0000 7fc3ab216f40 -1 Falling back to public interface 2020-09-08T15:30:28.519 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:30:28 smithi104 bash[17588]: debug 2020-09-08T15:30:28.205+0000 7fc3ab216f40 -1 osd.1 0 log_to_monitors {default=true} 2020-09-08T15:30:29.005 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:28 smithi055 bash[9296]: cluster 2020-09-08T15:30:27.462443+0000 mgr.x (mgr.24118) 31 : cluster [DBG] pgmap v24: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:29.006 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:28 smithi055 bash[9296]: audit 2020-09-08T15:30:28.213572+0000 mon.a (mon.0) 408 : audit [INF] from='osd.1 v1:172.21.15.104:6809/1421798992' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2020-09-08T15:30:29.006 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:28 smithi166 bash[9446]: cluster 2020-09-08T15:30:27.462443+0000 mgr.x (mgr.24118) 31 : cluster [DBG] pgmap v24: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:29.006 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:28 smithi166 bash[9446]: audit 2020-09-08T15:30:28.213572+0000 mon.a (mon.0) 408 : audit [INF] from='osd.1 v1:172.21.15.104:6809/1421798992' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2020-09-08T15:30:29.007 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph orch daemon add osd smithi104:vg_nvme/lv_2 2020-09-08T15:30:29.008 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:28 smithi104 bash[9203]: cluster 2020-09-08T15:30:27.462443+0000 mgr.x (mgr.24118) 31 : cluster [DBG] pgmap v24: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:29.008 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:28 smithi104 bash[9203]: audit 2020-09-08T15:30:28.213572+0000 mon.a (mon.0) 408 : audit [INF] from='osd.1 v1:172.21.15.104:6809/1421798992' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2020-09-08T15:30:29.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:29 smithi166 bash[9446]: audit 2020-09-08T15:30:28.521568+0000 mon.a (mon.0) 409 : audit [INF] from='osd.1 v1:172.21.15.104:6809/1421798992' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2020-09-08T15:30:29.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:29 smithi166 bash[9446]: cluster 2020-09-08T15:30:28.521765+0000 mon.a (mon.0) 410 : cluster [DBG] osdmap e15: 2 total, 1 up, 1 in 2020-09-08T15:30:29.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:29 smithi166 bash[9446]: audit 2020-09-08T15:30:28.522867+0000 mon.a (mon.0) 411 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2020-09-08T15:30:29.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:29 smithi166 bash[9446]: audit 2020-09-08T15:30:28.524032+0000 mon.a (mon.0) 412 : audit [INF] from='osd.1 v1:172.21.15.104:6809/1421798992' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2020-09-08T15:30:29.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:29 smithi166 bash[9446]: audit 2020-09-08T15:30:29.393428+0000 mon.a (mon.0) 413 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:30:29.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:29 smithi166 bash[9446]: audit 2020-09-08T15:30:29.521683+0000 mon.a (mon.0) 414 : audit [INF] from='osd.1 v1:172.21.15.104:6809/1421798992' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2020-09-08T15:30:29.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:29 smithi166 bash[9446]: cluster 2020-09-08T15:30:29.521769+0000 mon.a (mon.0) 415 : cluster [DBG] osdmap e16: 2 total, 1 up, 1 in 2020-09-08T15:30:29.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:29 smithi166 bash[9446]: audit 2020-09-08T15:30:29.521986+0000 mon.a (mon.0) 416 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2020-09-08T15:30:29.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:29 smithi055 bash[9296]: audit 2020-09-08T15:30:28.521568+0000 mon.a (mon.0) 409 : audit [INF] from='osd.1 v1:172.21.15.104:6809/1421798992' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2020-09-08T15:30:29.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:29 smithi055 bash[9296]: cluster 2020-09-08T15:30:28.521765+0000 mon.a (mon.0) 410 : cluster [DBG] osdmap e15: 2 total, 1 up, 1 in 2020-09-08T15:30:29.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:29 smithi055 bash[9296]: audit 2020-09-08T15:30:28.522867+0000 mon.a (mon.0) 411 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2020-09-08T15:30:29.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:29 smithi055 bash[9296]: audit 2020-09-08T15:30:28.524032+0000 mon.a (mon.0) 412 : audit [INF] from='osd.1 v1:172.21.15.104:6809/1421798992' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2020-09-08T15:30:29.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:29 smithi055 bash[9296]: audit 2020-09-08T15:30:29.393428+0000 mon.a (mon.0) 413 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:30:29.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:29 smithi055 bash[9296]: audit 2020-09-08T15:30:29.521683+0000 mon.a (mon.0) 414 : audit [INF] from='osd.1 v1:172.21.15.104:6809/1421798992' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2020-09-08T15:30:29.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:29 smithi055 bash[9296]: cluster 2020-09-08T15:30:29.521769+0000 mon.a (mon.0) 415 : cluster [DBG] osdmap e16: 2 total, 1 up, 1 in 2020-09-08T15:30:29.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:29 smithi055 bash[9296]: audit 2020-09-08T15:30:29.521986+0000 mon.a (mon.0) 416 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2020-09-08T15:30:29.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:30:29 smithi104 bash[17588]: debug 2020-09-08T15:30:29.521+0000 7fc3a1792700 -1 osd.1 0 waiting for initial osdmap 2020-09-08T15:30:29.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:29 smithi104 bash[9203]: audit 2020-09-08T15:30:28.521568+0000 mon.a (mon.0) 409 : audit [INF] from='osd.1 v1:172.21.15.104:6809/1421798992' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2020-09-08T15:30:29.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:29 smithi104 bash[9203]: cluster 2020-09-08T15:30:28.521765+0000 mon.a (mon.0) 410 : cluster [DBG] osdmap e15: 2 total, 1 up, 1 in 2020-09-08T15:30:29.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:29 smithi104 bash[9203]: audit 2020-09-08T15:30:28.522867+0000 mon.a (mon.0) 411 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2020-09-08T15:30:29.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:29 smithi104 bash[9203]: audit 2020-09-08T15:30:28.524032+0000 mon.a (mon.0) 412 : audit [INF] from='osd.1 v1:172.21.15.104:6809/1421798992' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2020-09-08T15:30:29.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:29 smithi104 bash[9203]: audit 2020-09-08T15:30:29.393428+0000 mon.a (mon.0) 413 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:30:29.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:29 smithi104 bash[9203]: audit 2020-09-08T15:30:29.521683+0000 mon.a (mon.0) 414 : audit [INF] from='osd.1 v1:172.21.15.104:6809/1421798992' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2020-09-08T15:30:29.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:29 smithi104 bash[9203]: cluster 2020-09-08T15:30:29.521769+0000 mon.a (mon.0) 415 : cluster [DBG] osdmap e16: 2 total, 1 up, 1 in 2020-09-08T15:30:29.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:29 smithi104 bash[9203]: audit 2020-09-08T15:30:29.521986+0000 mon.a (mon.0) 416 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2020-09-08T15:30:30.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:30 smithi104 bash[9203]: cluster 2020-09-08T15:30:29.462833+0000 mgr.x (mgr.24118) 32 : cluster [DBG] pgmap v26: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:30.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:30 smithi104 bash[9203]: audit 2020-09-08T15:30:29.527926+0000 mon.a (mon.0) 417 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2020-09-08T15:30:30.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:30 smithi104 bash[9203]: audit 2020-09-08T15:30:29.687116+0000 mon.a (mon.0) 418 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667305\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667699\", \"created\": \"2020-09-08T15:30:20.977340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}}, \"last_daemon_update\": \"2020-09-08T15:30:21.667959\", \"last_device_update\": \"2020-09-08T15:30:29.687041\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:30.682 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:30 smithi104 bash[9203]: audit 2020-09-08T15:30:29.691474+0000 mon.a (mon.0) 419 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667305\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667699\", \"created\": \"2020-09-08T15:30:20.977340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}}, \"last_daemon_update\": \"2020-09-08T15:30:21.667959\", \"last_device_update\": \"2020-09-08T15:30:29.687041\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:30.682 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:30 smithi104 bash[9203]: audit 2020-09-08T15:30:29.692980+0000 mon.a (mon.0) 420 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:30:30.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:30 smithi104 bash[9203]: audit 2020-09-08T15:30:29.698526+0000 mon.a (mon.0) 421 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:30:30.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:30 smithi104 bash[9203]: audit 2020-09-08T15:30:29.702404+0000 mon.a (mon.0) 422 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:30:30.684 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:30 smithi104 bash[9203]: audit 2020-09-08T15:30:29.706966+0000 mon.a (mon.0) 423 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:30:30.684 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:30 smithi104 bash[9203]: audit 2020-09-08T15:30:30.527282+0000 mon.a (mon.0) 424 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2020-09-08T15:30:30.879 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:30 smithi166 bash[9446]: cluster 2020-09-08T15:30:29.462833+0000 mgr.x (mgr.24118) 32 : cluster [DBG] pgmap v26: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:30.879 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:30 smithi166 bash[9446]: audit 2020-09-08T15:30:29.527926+0000 mon.a (mon.0) 417 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2020-09-08T15:30:30.879 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:30 smithi166 bash[9446]: audit 2020-09-08T15:30:29.687116+0000 mon.a (mon.0) 418 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667305\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667699\", \"created\": \"2020-09-08T15:30:20.977340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}}, \"last_daemon_update\": \"2020-09-08T15:30:21.667959\", \"last_device_update\": \"2020-09-08T15:30:29.687041\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:30.880 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:30 smithi166 bash[9446]: audit 2020-09-08T15:30:29.691474+0000 mon.a (mon.0) 419 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667305\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667699\", \"created\": \"2020-09-08T15:30:20.977340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}}, \"last_daemon_update\": \"2020-09-08T15:30:21.667959\", \"last_device_update\": \"2020-09-08T15:30:29.687041\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:30.880 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:30 smithi166 bash[9446]: audit 2020-09-08T15:30:29.692980+0000 mon.a (mon.0) 420 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:30:30.880 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:30 smithi166 bash[9446]: audit 2020-09-08T15:30:29.698526+0000 mon.a (mon.0) 421 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:30:30.881 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:30 smithi166 bash[9446]: audit 2020-09-08T15:30:29.702404+0000 mon.a (mon.0) 422 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:30:30.881 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:30 smithi166 bash[9446]: audit 2020-09-08T15:30:29.706966+0000 mon.a (mon.0) 423 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:30:30.881 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:30 smithi166 bash[9446]: audit 2020-09-08T15:30:30.527282+0000 mon.a (mon.0) 424 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2020-09-08T15:30:30.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:30 smithi055 bash[9296]: cluster 2020-09-08T15:30:29.462833+0000 mgr.x (mgr.24118) 32 : cluster [DBG] pgmap v26: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:30.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:30 smithi055 bash[9296]: audit 2020-09-08T15:30:29.527926+0000 mon.a (mon.0) 417 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2020-09-08T15:30:30.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:30 smithi055 bash[9296]: audit 2020-09-08T15:30:29.687116+0000 mon.a (mon.0) 418 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667305\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667699\", \"created\": \"2020-09-08T15:30:20.977340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}}, \"last_daemon_update\": \"2020-09-08T15:30:21.667959\", \"last_device_update\": \"2020-09-08T15:30:29.687041\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:30.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:30 smithi055 bash[9296]: audit 2020-09-08T15:30:29.691474+0000 mon.a (mon.0) 419 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667305\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667699\", \"created\": \"2020-09-08T15:30:20.977340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}}, \"last_daemon_update\": \"2020-09-08T15:30:21.667959\", \"last_device_update\": \"2020-09-08T15:30:29.687041\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:30.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:30 smithi055 bash[9296]: audit 2020-09-08T15:30:29.692980+0000 mon.a (mon.0) 420 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:30:30.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:30 smithi055 bash[9296]: audit 2020-09-08T15:30:29.698526+0000 mon.a (mon.0) 421 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:30:30.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:30 smithi055 bash[9296]: audit 2020-09-08T15:30:29.702404+0000 mon.a (mon.0) 422 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:30:30.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:30 smithi055 bash[9296]: audit 2020-09-08T15:30:29.706966+0000 mon.a (mon.0) 423 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:30:30.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:30 smithi055 bash[9296]: audit 2020-09-08T15:30:30.527282+0000 mon.a (mon.0) 424 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2020-09-08T15:30:31.856 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:31 smithi104 bash[9203]: cluster 2020-09-08T15:30:29.252836+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:30:31.857 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:31 smithi104 bash[9203]: cluster 2020-09-08T15:30:29.253000+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:30:31.857 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:31 smithi104 bash[9203]: cluster 2020-09-08T15:30:30.537600+0000 mon.a (mon.0) 425 : cluster [INF] osd.1 v1:172.21.15.104:6809/1421798992 boot 2020-09-08T15:30:31.857 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:31 smithi104 bash[9203]: cluster 2020-09-08T15:30:30.537671+0000 mon.a (mon.0) 426 : cluster [DBG] osdmap e17: 2 total, 2 up, 2 in 2020-09-08T15:30:31.857 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:31 smithi104 bash[9203]: audit 2020-09-08T15:30:30.538443+0000 mon.a (mon.0) 427 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2020-09-08T15:30:31.858 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:31 smithi104 bash[9203]: cluster 2020-09-08T15:30:31.463355+0000 mgr.x (mgr.24118) 33 : cluster [DBG] pgmap v29: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:31.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:31 smithi166 bash[9446]: cluster 2020-09-08T15:30:29.252836+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:30:31.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:31 smithi166 bash[9446]: cluster 2020-09-08T15:30:29.253000+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:30:31.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:31 smithi166 bash[9446]: cluster 2020-09-08T15:30:30.537600+0000 mon.a (mon.0) 425 : cluster [INF] osd.1 v1:172.21.15.104:6809/1421798992 boot 2020-09-08T15:30:31.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:31 smithi166 bash[9446]: cluster 2020-09-08T15:30:30.537671+0000 mon.a (mon.0) 426 : cluster [DBG] osdmap e17: 2 total, 2 up, 2 in 2020-09-08T15:30:31.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:31 smithi166 bash[9446]: audit 2020-09-08T15:30:30.538443+0000 mon.a (mon.0) 427 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2020-09-08T15:30:31.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:31 smithi166 bash[9446]: cluster 2020-09-08T15:30:31.463355+0000 mgr.x (mgr.24118) 33 : cluster [DBG] pgmap v29: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:31.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:31 smithi055 bash[9296]: cluster 2020-09-08T15:30:29.252836+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:30:31.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:31 smithi055 bash[9296]: cluster 2020-09-08T15:30:29.253000+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:30:31.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:31 smithi055 bash[9296]: cluster 2020-09-08T15:30:30.537600+0000 mon.a (mon.0) 425 : cluster [INF] osd.1 v1:172.21.15.104:6809/1421798992 boot 2020-09-08T15:30:31.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:31 smithi055 bash[9296]: cluster 2020-09-08T15:30:30.537671+0000 mon.a (mon.0) 426 : cluster [DBG] osdmap e17: 2 total, 2 up, 2 in 2020-09-08T15:30:31.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:31 smithi055 bash[9296]: audit 2020-09-08T15:30:30.538443+0000 mon.a (mon.0) 427 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2020-09-08T15:30:31.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:31 smithi055 bash[9296]: cluster 2020-09-08T15:30:31.463355+0000 mgr.x (mgr.24118) 33 : cluster [DBG] pgmap v29: 1 pgs: 1 undersized+peered; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2020-09-08T15:30:32.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:32 smithi166 bash[9446]: cluster 2020-09-08T15:30:31.555432+0000 mon.a (mon.0) 428 : cluster [DBG] osdmap e18: 2 total, 2 up, 2 in 2020-09-08T15:30:32.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:32 smithi166 bash[9446]: audit 2020-09-08T15:30:32.320063+0000 mgr.x (mgr.24118) 34 : audit [DBG] from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:30:32.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:32 smithi166 bash[9446]: audit 2020-09-08T15:30:32.320460+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:30:32.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:32 smithi166 bash[9446]: cephadm 2020-09-08T15:30:32.324058+0000 mgr.x (mgr.24118) 35 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:30:32.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:32 smithi166 bash[9446]: cephadm 2020-09-08T15:30:32.324290+0000 mgr.x (mgr.24118) 36 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:30:32.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:32 smithi166 bash[9446]: audit 2020-09-08T15:30:32.324407+0000 mon.a (mon.0) 430 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:30:32.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:32 smithi166 bash[9446]: cephadm 2020-09-08T15:30:32.325437+0000 mgr.x (mgr.24118) 37 : cephadm [INF] Applying None on host smithi104... 2020-09-08T15:30:32.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:32 smithi166 bash[9446]: audit 2020-09-08T15:30:32.326066+0000 mon.a (mon.0) 431 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:30:32.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:32 smithi166 bash[9446]: audit 2020-09-08T15:30:32.328046+0000 mon.a (mon.0) 432 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:30:32.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:32 smithi055 bash[9296]: cluster 2020-09-08T15:30:31.555432+0000 mon.a (mon.0) 428 : cluster [DBG] osdmap e18: 2 total, 2 up, 2 in 2020-09-08T15:30:32.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:32 smithi055 bash[9296]: audit 2020-09-08T15:30:32.320063+0000 mgr.x (mgr.24118) 34 : audit [DBG] from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:30:32.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:32 smithi055 bash[9296]: audit 2020-09-08T15:30:32.320460+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:30:32.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:32 smithi055 bash[9296]: cephadm 2020-09-08T15:30:32.324058+0000 mgr.x (mgr.24118) 35 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:30:32.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:32 smithi055 bash[9296]: cephadm 2020-09-08T15:30:32.324290+0000 mgr.x (mgr.24118) 36 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:30:32.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:32 smithi055 bash[9296]: audit 2020-09-08T15:30:32.324407+0000 mon.a (mon.0) 430 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:30:32.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:32 smithi055 bash[9296]: cephadm 2020-09-08T15:30:32.325437+0000 mgr.x (mgr.24118) 37 : cephadm [INF] Applying None on host smithi104... 2020-09-08T15:30:32.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:32 smithi055 bash[9296]: audit 2020-09-08T15:30:32.326066+0000 mon.a (mon.0) 431 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:30:32.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:32 smithi055 bash[9296]: audit 2020-09-08T15:30:32.328046+0000 mon.a (mon.0) 432 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:30:32.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:32 smithi104 bash[9203]: cluster 2020-09-08T15:30:31.555432+0000 mon.a (mon.0) 428 : cluster [DBG] osdmap e18: 2 total, 2 up, 2 in 2020-09-08T15:30:32.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:32 smithi104 bash[9203]: audit 2020-09-08T15:30:32.320063+0000 mgr.x (mgr.24118) 34 : audit [DBG] from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:30:32.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:32 smithi104 bash[9203]: audit 2020-09-08T15:30:32.320460+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:30:32.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:32 smithi104 bash[9203]: cephadm 2020-09-08T15:30:32.324058+0000 mgr.x (mgr.24118) 35 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:30:32.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:32 smithi104 bash[9203]: cephadm 2020-09-08T15:30:32.324290+0000 mgr.x (mgr.24118) 36 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:30:32.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:32 smithi104 bash[9203]: audit 2020-09-08T15:30:32.324407+0000 mon.a (mon.0) 430 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:30:32.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:32 smithi104 bash[9203]: cephadm 2020-09-08T15:30:32.325437+0000 mgr.x (mgr.24118) 37 : cephadm [INF] Applying None on host smithi104... 2020-09-08T15:30:32.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:32 smithi104 bash[9203]: audit 2020-09-08T15:30:32.326066+0000 mon.a (mon.0) 431 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:30:32.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:32 smithi104 bash[9203]: audit 2020-09-08T15:30:32.328046+0000 mon.a (mon.0) 432 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:30:33.867 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:33 smithi104 bash[9203]: cluster 2020-09-08T15:30:32.565232+0000 mon.a (mon.0) 433 : cluster [DBG] osdmap e19: 2 total, 2 up, 2 in 2020-09-08T15:30:33.868 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:33 smithi104 bash[9203]: cluster 2020-09-08T15:30:33.463821+0000 mgr.x (mgr.24118) 38 : cluster [DBG] pgmap v32: 1 pgs: 1 undersized+peered; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:33.914 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:33 smithi055 bash[9296]: cluster 2020-09-08T15:30:32.565232+0000 mon.a (mon.0) 433 : cluster [DBG] osdmap e19: 2 total, 2 up, 2 in 2020-09-08T15:30:33.914 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:33 smithi055 bash[9296]: cluster 2020-09-08T15:30:33.463821+0000 mgr.x (mgr.24118) 38 : cluster [DBG] pgmap v32: 1 pgs: 1 undersized+peered; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:34.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:33 smithi166 bash[9446]: cluster 2020-09-08T15:30:32.565232+0000 mon.a (mon.0) 433 : cluster [DBG] osdmap e19: 2 total, 2 up, 2 in 2020-09-08T15:30:34.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:33 smithi166 bash[9446]: cluster 2020-09-08T15:30:33.463821+0000 mgr.x (mgr.24118) 38 : cluster [DBG] pgmap v32: 1 pgs: 1 undersized+peered; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:34.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:34 smithi055 bash[9296]: audit 2020-09-08T15:30:32.723724+0000 mon.b (mon.1) 5 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2020-09-08T15:30:34.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:34 smithi055 bash[9296]: cluster 2020-09-08T15:30:33.221877+0000 osd.1 (osd.1) 3 : cluster [DBG] 1.0 deep-scrub starts 2020-09-08T15:30:34.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:34 smithi055 bash[9296]: cluster 2020-09-08T15:30:33.225800+0000 osd.1 (osd.1) 4 : cluster [DBG] 1.0 deep-scrub ok 2020-09-08T15:30:34.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:34 smithi055 bash[9296]: audit 2020-09-08T15:30:33.622690+0000 mon.b (mon.1) 6 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2020-09-08T15:30:34.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:34 smithi055 bash[9296]: audit 2020-09-08T15:30:33.641421+0000 mon.c (mon.2) 3 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2020-09-08T15:30:34.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:34 smithi055 bash[9296]: audit 2020-09-08T15:30:34.508792+0000 mon.c (mon.2) 4 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2020-09-08T15:30:34.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:34 smithi055 bash[9296]: audit 2020-09-08T15:30:34.513374+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/devicehealth/last_scrape","val":"20200908-152939"}]: dispatch 2020-09-08T15:30:34.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:34 smithi055 bash[9296]: audit 2020-09-08T15:30:34.519023+0000 mon.a (mon.0) 435 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/devicehealth/last_scrape","val":"20200908-152939"}]': finished 2020-09-08T15:30:34.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:34 smithi104 bash[9203]: audit 2020-09-08T15:30:32.723724+0000 mon.b (mon.1) 5 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2020-09-08T15:30:34.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:34 smithi104 bash[9203]: cluster 2020-09-08T15:30:33.221877+0000 osd.1 (osd.1) 3 : cluster [DBG] 1.0 deep-scrub starts 2020-09-08T15:30:34.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:34 smithi104 bash[9203]: cluster 2020-09-08T15:30:33.225800+0000 osd.1 (osd.1) 4 : cluster [DBG] 1.0 deep-scrub ok 2020-09-08T15:30:34.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:34 smithi104 bash[9203]: audit 2020-09-08T15:30:33.622690+0000 mon.b (mon.1) 6 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2020-09-08T15:30:34.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:34 smithi104 bash[9203]: audit 2020-09-08T15:30:33.641421+0000 mon.c (mon.2) 3 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2020-09-08T15:30:34.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:34 smithi104 bash[9203]: audit 2020-09-08T15:30:34.508792+0000 mon.c (mon.2) 4 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2020-09-08T15:30:34.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:34 smithi104 bash[9203]: audit 2020-09-08T15:30:34.513374+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/devicehealth/last_scrape","val":"20200908-152939"}]: dispatch 2020-09-08T15:30:34.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:34 smithi104 bash[9203]: audit 2020-09-08T15:30:34.519023+0000 mon.a (mon.0) 435 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/devicehealth/last_scrape","val":"20200908-152939"}]': finished 2020-09-08T15:30:35.110 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:34 smithi166 bash[9446]: audit 2020-09-08T15:30:32.723724+0000 mon.b (mon.1) 5 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2020-09-08T15:30:35.111 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:34 smithi166 bash[9446]: cluster 2020-09-08T15:30:33.221877+0000 osd.1 (osd.1) 3 : cluster [DBG] 1.0 deep-scrub starts 2020-09-08T15:30:35.111 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:34 smithi166 bash[9446]: cluster 2020-09-08T15:30:33.225800+0000 osd.1 (osd.1) 4 : cluster [DBG] 1.0 deep-scrub ok 2020-09-08T15:30:35.111 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:34 smithi166 bash[9446]: audit 2020-09-08T15:30:33.622690+0000 mon.b (mon.1) 6 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2020-09-08T15:30:35.111 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:34 smithi166 bash[9446]: audit 2020-09-08T15:30:33.641421+0000 mon.c (mon.2) 3 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2020-09-08T15:30:35.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:34 smithi166 bash[9446]: audit 2020-09-08T15:30:34.508792+0000 mon.c (mon.2) 4 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2020-09-08T15:30:35.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:34 smithi166 bash[9446]: audit 2020-09-08T15:30:34.513374+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/devicehealth/last_scrape","val":"20200908-152939"}]: dispatch 2020-09-08T15:30:35.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:34 smithi166 bash[9446]: audit 2020-09-08T15:30:34.519023+0000 mon.a (mon.0) 435 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/devicehealth/last_scrape","val":"20200908-152939"}]': finished 2020-09-08T15:30:35.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:35 smithi055 bash[9296]: cluster 2020-09-08T15:30:35.464361+0000 mgr.x (mgr.24118) 39 : cluster [DBG] pgmap v33: 1 pgs: 1 undersized+peered; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:35.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:35 smithi104 bash[9203]: cluster 2020-09-08T15:30:35.464361+0000 mgr.x (mgr.24118) 39 : cluster [DBG] pgmap v33: 1 pgs: 1 undersized+peered; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:36.110 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:35 smithi166 bash[9446]: cluster 2020-09-08T15:30:35.464361+0000 mgr.x (mgr.24118) 39 : cluster [DBG] pgmap v33: 1 pgs: 1 undersized+peered; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:36.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:36 smithi055 bash[9296]: audit 2020-09-08T15:30:36.281458+0000 mon.a (mon.0) 436 : audit [INF] from='client.? v1:172.21.15.104:0/3325675494' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca6da1cc-1d8d-494b-93e6-e065ff70d434"}]: dispatch 2020-09-08T15:30:36.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:36 smithi055 bash[9296]: audit 2020-09-08T15:30:36.288191+0000 mon.a (mon.0) 437 : audit [INF] from='client.? v1:172.21.15.104:0/3325675494' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca6da1cc-1d8d-494b-93e6-e065ff70d434"}]': finished 2020-09-08T15:30:36.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:36 smithi055 bash[9296]: cluster 2020-09-08T15:30:36.288261+0000 mon.a (mon.0) 438 : cluster [DBG] osdmap e20: 3 total, 2 up, 2 in 2020-09-08T15:30:36.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:36 smithi055 bash[9296]: audit 2020-09-08T15:30:36.288379+0000 mon.a (mon.0) 439 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2020-09-08T15:30:36.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:36 smithi104 bash[9203]: audit 2020-09-08T15:30:36.281458+0000 mon.a (mon.0) 436 : audit [INF] from='client.? v1:172.21.15.104:0/3325675494' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca6da1cc-1d8d-494b-93e6-e065ff70d434"}]: dispatch 2020-09-08T15:30:36.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:36 smithi104 bash[9203]: audit 2020-09-08T15:30:36.288191+0000 mon.a (mon.0) 437 : audit [INF] from='client.? v1:172.21.15.104:0/3325675494' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca6da1cc-1d8d-494b-93e6-e065ff70d434"}]': finished 2020-09-08T15:30:36.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:36 smithi104 bash[9203]: cluster 2020-09-08T15:30:36.288261+0000 mon.a (mon.0) 438 : cluster [DBG] osdmap e20: 3 total, 2 up, 2 in 2020-09-08T15:30:36.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:36 smithi104 bash[9203]: audit 2020-09-08T15:30:36.288379+0000 mon.a (mon.0) 439 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2020-09-08T15:30:37.110 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:36 smithi166 bash[9446]: audit 2020-09-08T15:30:36.281458+0000 mon.a (mon.0) 436 : audit [INF] from='client.? v1:172.21.15.104:0/3325675494' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca6da1cc-1d8d-494b-93e6-e065ff70d434"}]: dispatch 2020-09-08T15:30:37.111 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:36 smithi166 bash[9446]: audit 2020-09-08T15:30:36.288191+0000 mon.a (mon.0) 437 : audit [INF] from='client.? v1:172.21.15.104:0/3325675494' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca6da1cc-1d8d-494b-93e6-e065ff70d434"}]': finished 2020-09-08T15:30:37.111 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:36 smithi166 bash[9446]: cluster 2020-09-08T15:30:36.288261+0000 mon.a (mon.0) 438 : cluster [DBG] osdmap e20: 3 total, 2 up, 2 in 2020-09-08T15:30:37.111 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:36 smithi166 bash[9446]: audit 2020-09-08T15:30:36.288379+0000 mon.a (mon.0) 439 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2020-09-08T15:30:37.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:37 smithi055 bash[9296]: audit 2020-09-08T15:30:36.952262+0000 mon.a (mon.0) 440 : audit [DBG] from='client.? v1:172.21.15.104:0/1827710619' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:30:37.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:37 smithi055 bash[9296]: cluster 2020-09-08T15:30:37.464881+0000 mgr.x (mgr.24118) 40 : cluster [DBG] pgmap v35: 1 pgs: 1 active+clean; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:37.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:37 smithi104 bash[9203]: audit 2020-09-08T15:30:36.952262+0000 mon.a (mon.0) 440 : audit [DBG] from='client.? v1:172.21.15.104:0/1827710619' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:30:37.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:37 smithi104 bash[9203]: cluster 2020-09-08T15:30:37.464881+0000 mgr.x (mgr.24118) 40 : cluster [DBG] pgmap v35: 1 pgs: 1 active+clean; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:38.110 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:37 smithi166 bash[9446]: audit 2020-09-08T15:30:36.952262+0000 mon.a (mon.0) 440 : audit [DBG] from='client.? v1:172.21.15.104:0/1827710619' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:30:38.111 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:37 smithi166 bash[9446]: cluster 2020-09-08T15:30:37.464881+0000 mgr.x (mgr.24118) 40 : cluster [DBG] pgmap v35: 1 pgs: 1 active+clean; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:38.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:38 smithi055 bash[9296]: cluster 2020-09-08T15:30:37.646258+0000 mon.a (mon.0) 441 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2020-09-08T15:30:38.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:38 smithi055 bash[9296]: cluster 2020-09-08T15:30:37.646291+0000 mon.a (mon.0) 442 : cluster [INF] Cluster is now healthy 2020-09-08T15:30:38.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:38 smithi104 bash[9203]: cluster 2020-09-08T15:30:37.646258+0000 mon.a (mon.0) 441 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2020-09-08T15:30:38.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:38 smithi104 bash[9203]: cluster 2020-09-08T15:30:37.646291+0000 mon.a (mon.0) 442 : cluster [INF] Cluster is now healthy 2020-09-08T15:30:39.110 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:38 smithi166 bash[9446]: cluster 2020-09-08T15:30:37.646258+0000 mon.a (mon.0) 441 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2020-09-08T15:30:39.111 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:38 smithi166 bash[9446]: cluster 2020-09-08T15:30:37.646291+0000 mon.a (mon.0) 442 : cluster [INF] Cluster is now healthy 2020-09-08T15:30:40.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:40 smithi166 bash[9446]: cluster 2020-09-08T15:30:39.465275+0000 mgr.x (mgr.24118) 41 : cluster [DBG] pgmap v36: 1 pgs: 1 active+clean; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:40.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:40 smithi166 bash[9446]: audit 2020-09-08T15:30:39.924061+0000 mon.a (mon.0) 443 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:30:40.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:40 smithi055 bash[9296]: cluster 2020-09-08T15:30:39.465275+0000 mgr.x (mgr.24118) 41 : cluster [DBG] pgmap v36: 1 pgs: 1 active+clean; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:40.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:40 smithi055 bash[9296]: audit 2020-09-08T15:30:39.924061+0000 mon.a (mon.0) 443 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:30:40.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:40 smithi104 bash[9203]: cluster 2020-09-08T15:30:39.465275+0000 mgr.x (mgr.24118) 41 : cluster [DBG] pgmap v36: 1 pgs: 1 active+clean; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:40.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:40 smithi104 bash[9203]: audit 2020-09-08T15:30:39.924061+0000 mon.a (mon.0) 443 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:30:42.794 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:42 smithi104 bash[9203]: cluster 2020-09-08T15:30:41.465808+0000 mgr.x (mgr.24118) 42 : cluster [DBG] pgmap v37: 1 pgs: 1 active+clean; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:42.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:42 smithi166 bash[9446]: cluster 2020-09-08T15:30:41.465808+0000 mgr.x (mgr.24118) 42 : cluster [DBG] pgmap v37: 1 pgs: 1 active+clean; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:42.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:42 smithi055 bash[9296]: cluster 2020-09-08T15:30:41.465808+0000 mgr.x (mgr.24118) 42 : cluster [DBG] pgmap v37: 1 pgs: 1 active+clean; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:44.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:44 smithi166 bash[9446]: cluster 2020-09-08T15:30:43.466353+0000 mgr.x (mgr.24118) 43 : cluster [DBG] pgmap v38: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:44.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:44 smithi166 bash[9446]: audit 2020-09-08T15:30:43.956010+0000 mon.a (mon.0) 444 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2020-09-08T15:30:44.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:44 smithi166 bash[9446]: audit 2020-09-08T15:30:43.957820+0000 mon.a (mon.0) 445 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:30:44.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:44 smithi166 bash[9446]: audit 2020-09-08T15:30:43.959949+0000 mon.a (mon.0) 446 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.2", "key": "container_image"}]: dispatch 2020-09-08T15:30:44.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:44 smithi055 bash[9296]: cluster 2020-09-08T15:30:43.466353+0000 mgr.x (mgr.24118) 43 : cluster [DBG] pgmap v38: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:44.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:44 smithi055 bash[9296]: audit 2020-09-08T15:30:43.956010+0000 mon.a (mon.0) 444 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2020-09-08T15:30:44.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:44 smithi055 bash[9296]: audit 2020-09-08T15:30:43.957820+0000 mon.a (mon.0) 445 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:30:44.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:44 smithi055 bash[9296]: audit 2020-09-08T15:30:43.959949+0000 mon.a (mon.0) 446 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.2", "key": "container_image"}]: dispatch 2020-09-08T15:30:44.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:44 smithi104 bash[9203]: cluster 2020-09-08T15:30:43.466353+0000 mgr.x (mgr.24118) 43 : cluster [DBG] pgmap v38: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:44.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:44 smithi104 bash[9203]: audit 2020-09-08T15:30:43.956010+0000 mon.a (mon.0) 444 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2020-09-08T15:30:44.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:44 smithi104 bash[9203]: audit 2020-09-08T15:30:43.957820+0000 mon.a (mon.0) 445 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:30:44.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:44 smithi104 bash[9203]: audit 2020-09-08T15:30:43.959949+0000 mon.a (mon.0) 446 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.2", "key": "container_image"}]: dispatch 2020-09-08T15:30:45.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:45 smithi104 bash[9203]: cephadm 2020-09-08T15:30:43.961094+0000 mgr.x (mgr.24118) 44 : cephadm [INF] Deploying daemon osd.2 on smithi104 2020-09-08T15:30:45.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:45 smithi166 bash[9446]: cephadm 2020-09-08T15:30:43.961094+0000 mgr.x (mgr.24118) 44 : cephadm [INF] Deploying daemon osd.2 on smithi104 2020-09-08T15:30:45.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:45 smithi055 bash[9296]: cephadm 2020-09-08T15:30:43.961094+0000 mgr.x (mgr.24118) 44 : cephadm [INF] Deploying daemon osd.2 on smithi104 2020-09-08T15:30:46.270 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 2 on host 'smithi104' 2020-09-08T15:30:46.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:46 smithi166 bash[9446]: cluster 2020-09-08T15:30:45.466886+0000 mgr.x (mgr.24118) 45 : cluster [DBG] pgmap v39: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:46.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:46 smithi166 bash[9446]: audit 2020-09-08T15:30:46.265065+0000 mon.a (mon.0) 447 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667305\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667699\", \"created\": \"2020-09-08T15:30:20.977340\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}}, \"last_device_update\": \"2020-09-08T15:30:29.687041\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:46.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:46 smithi166 bash[9446]: audit 2020-09-08T15:30:46.265962+0000 mon.a (mon.0) 448 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:30:46.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:46 smithi166 bash[9446]: audit 2020-09-08T15:30:46.270889+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667305\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667699\", \"created\": \"2020-09-08T15:30:20.977340\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}}, \"last_device_update\": \"2020-09-08T15:30:29.687041\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:46.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:46 smithi055 bash[9296]: cluster 2020-09-08T15:30:45.466886+0000 mgr.x (mgr.24118) 45 : cluster [DBG] pgmap v39: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:46.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:46 smithi055 bash[9296]: audit 2020-09-08T15:30:46.265065+0000 mon.a (mon.0) 447 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667305\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667699\", \"created\": \"2020-09-08T15:30:20.977340\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}}, \"last_device_update\": \"2020-09-08T15:30:29.687041\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:46.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:46 smithi055 bash[9296]: audit 2020-09-08T15:30:46.265962+0000 mon.a (mon.0) 448 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:30:46.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:46 smithi055 bash[9296]: audit 2020-09-08T15:30:46.270889+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667305\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667699\", \"created\": \"2020-09-08T15:30:20.977340\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}}, \"last_device_update\": \"2020-09-08T15:30:29.687041\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:46.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:46 smithi104 bash[9203]: cluster 2020-09-08T15:30:45.466886+0000 mgr.x (mgr.24118) 45 : cluster [DBG] pgmap v39: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:46.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:46 smithi104 bash[9203]: audit 2020-09-08T15:30:46.265065+0000 mon.a (mon.0) 447 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667305\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667699\", \"created\": \"2020-09-08T15:30:20.977340\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}}, \"last_device_update\": \"2020-09-08T15:30:29.687041\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:46.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:46 smithi104 bash[9203]: audit 2020-09-08T15:30:46.265962+0000 mon.a (mon.0) 448 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:30:46.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:46 smithi104 bash[9203]: audit 2020-09-08T15:30:46.270889+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667305\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:21.667699\", \"created\": \"2020-09-08T15:30:20.977340\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}}, \"last_device_update\": \"2020-09-08T15:30:29.687041\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:47.397 INFO:teuthology.orchestra.run.smithi104:osd.2> sudo journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.2.service 2020-09-08T15:30:47.406 INFO:tasks.cephadm:Deploying osd.3 on smithi104 with /dev/vg_nvme/lv_1... 2020-09-08T15:30:47.407 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2020-09-08T15:30:47.481 INFO:journalctl@ceph.osd.2.smithi104.stdout:-- Logs begin at Tue 2020-09-01 16:49:30 UTC. -- 2020-09-08T15:30:47.682 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:47 smithi104 bash[9203]: audit 2020-09-08T15:30:47.028675+0000 mon.a (mon.0) 450 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.027917\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028113\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028365\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028587\", \"created\": \"2020-09-08T15:30:46.237340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}}, \"last_daemon_update\": \"2020-09-08T15:30:47.028883\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:47.682 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:47 smithi104 bash[9203]: audit 2020-09-08T15:30:47.034332+0000 mon.a (mon.0) 451 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.027917\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028113\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028365\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028587\", \"created\": \"2020-09-08T15:30:46.237340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}}, \"last_daemon_update\": \"2020-09-08T15:30:47.028883\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:47.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:47 smithi104 bash[9203]: audit 2020-09-08T15:30:47.035871+0000 mon.a (mon.0) 452 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:30:47.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:47 smithi104 bash[9203]: cluster 2020-09-08T15:30:47.467429+0000 mgr.x (mgr.24118) 46 : cluster [DBG] pgmap v40: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:47.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:47 smithi166 bash[9446]: audit 2020-09-08T15:30:47.028675+0000 mon.a (mon.0) 450 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.027917\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028113\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028365\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028587\", \"created\": \"2020-09-08T15:30:46.237340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}}, \"last_daemon_update\": \"2020-09-08T15:30:47.028883\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:47.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:47 smithi166 bash[9446]: audit 2020-09-08T15:30:47.034332+0000 mon.a (mon.0) 451 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.027917\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028113\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028365\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028587\", \"created\": \"2020-09-08T15:30:46.237340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}}, \"last_daemon_update\": \"2020-09-08T15:30:47.028883\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:47.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:47 smithi166 bash[9446]: audit 2020-09-08T15:30:47.035871+0000 mon.a (mon.0) 452 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:30:47.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:47 smithi166 bash[9446]: cluster 2020-09-08T15:30:47.467429+0000 mgr.x (mgr.24118) 46 : cluster [DBG] pgmap v40: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:47.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:47 smithi055 bash[9296]: audit 2020-09-08T15:30:47.028675+0000 mon.a (mon.0) 450 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.027917\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028113\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028365\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028587\", \"created\": \"2020-09-08T15:30:46.237340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}}, \"last_daemon_update\": \"2020-09-08T15:30:47.028883\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:47.914 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:47 smithi055 bash[9296]: audit 2020-09-08T15:30:47.034332+0000 mon.a (mon.0) 451 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.027917\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028113\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028365\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028587\", \"created\": \"2020-09-08T15:30:46.237340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}}, \"last_daemon_update\": \"2020-09-08T15:30:47.028883\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:47.914 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:47 smithi055 bash[9296]: audit 2020-09-08T15:30:47.035871+0000 mon.a (mon.0) 452 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:30:47.922 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:47 smithi055 bash[9296]: cluster 2020-09-08T15:30:47.467429+0000 mgr.x (mgr.24118) 46 : cluster [DBG] pgmap v40: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:49.022 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:30:48 smithi104 bash[19922]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2020-09-08T15:30:49.023 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:30:48 smithi104 bash[19922]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-2 --no-mon-config 2020-09-08T15:30:49.023 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:30:48 smithi104 bash[19922]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2/block 2020-09-08T15:30:49.023 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:30:48 smithi104 bash[19922]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-2/block 2020-09-08T15:30:49.023 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:30:48 smithi104 bash[19922]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2020-09-08T15:30:49.023 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:30:48 smithi104 bash[19922]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2020-09-08T15:30:49.024 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:30:48 smithi104 bash[19922]: --> ceph-volume lvm activate successful for osd ID: 2 2020-09-08T15:30:50.175 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:30:49 smithi104 bash[19922]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:30:50.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:50 smithi104 bash[9203]: cluster 2020-09-08T15:30:49.467979+0000 mgr.x (mgr.24118) 47 : cluster [DBG] pgmap v41: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:50.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:50 smithi166 bash[9446]: cluster 2020-09-08T15:30:49.467979+0000 mgr.x (mgr.24118) 47 : cluster [DBG] pgmap v41: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:50.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:50 smithi055 bash[9296]: cluster 2020-09-08T15:30:49.467979+0000 mgr.x (mgr.24118) 47 : cluster [DBG] pgmap v41: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:52.776 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:30:52 smithi104 bash[19922]: debug 2020-09-08T15:30:52.501+0000 7fe3908abf40 -1 Falling back to public interface 2020-09-08T15:30:52.776 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:52 smithi104 bash[9203]: cluster 2020-09-08T15:30:51.468476+0000 mgr.x (mgr.24118) 48 : cluster [DBG] pgmap v42: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:52.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:52 smithi166 bash[9446]: cluster 2020-09-08T15:30:51.468476+0000 mgr.x (mgr.24118) 48 : cluster [DBG] pgmap v42: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:52.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:52 smithi055 bash[9296]: cluster 2020-09-08T15:30:51.468476+0000 mgr.x (mgr.24118) 48 : cluster [DBG] pgmap v42: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:53.175 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:30:52 smithi104 bash[19922]: debug 2020-09-08T15:30:52.897+0000 7fe3908abf40 -1 osd.2 0 log_to_monitors {default=true} 2020-09-08T15:30:53.449 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_1 2020-09-08T15:30:53.449 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2020-09-08T15:30:53.449 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2020-09-08T15:30:53.450 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2020-09-08T15:30:53.450 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0148532 s, 706 MB/s 2020-09-08T15:30:53.450 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2020-09-08T15:30:53.852 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:53 smithi104 bash[9203]: audit 2020-09-08T15:30:52.907516+0000 mon.a (mon.0) 453 : audit [INF] from='osd.2 v1:172.21.15.104:6817/2713413369' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2020-09-08T15:30:53.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:53 smithi166 bash[9446]: audit 2020-09-08T15:30:52.907516+0000 mon.a (mon.0) 453 : audit [INF] from='osd.2 v1:172.21.15.104:6817/2713413369' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2020-09-08T15:30:53.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:53 smithi055 bash[9296]: audit 2020-09-08T15:30:52.907516+0000 mon.a (mon.0) 453 : audit [INF] from='osd.2 v1:172.21.15.104:6817/2713413369' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2020-09-08T15:30:54.354 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph orch daemon add osd smithi104:vg_nvme/lv_1 2020-09-08T15:30:54.678 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:30:54 smithi104 bash[19922]: debug 2020-09-08T15:30:54.537+0000 7fe386e27700 -1 osd.2 0 waiting for initial osdmap 2020-09-08T15:30:54.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:54 smithi104 bash[9203]: cluster 2020-09-08T15:30:53.469018+0000 mgr.x (mgr.24118) 49 : cluster [DBG] pgmap v43: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:54.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:54 smithi104 bash[9203]: audit 2020-09-08T15:30:53.536161+0000 mon.a (mon.0) 454 : audit [INF] from='osd.2 v1:172.21.15.104:6817/2713413369' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2020-09-08T15:30:54.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:54 smithi104 bash[9203]: cluster 2020-09-08T15:30:53.536239+0000 mon.a (mon.0) 455 : cluster [DBG] osdmap e21: 3 total, 2 up, 2 in 2020-09-08T15:30:54.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:54 smithi104 bash[9203]: audit 2020-09-08T15:30:53.536463+0000 mon.a (mon.0) 456 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2020-09-08T15:30:54.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:54 smithi104 bash[9203]: audit 2020-09-08T15:30:53.537250+0000 mon.a (mon.0) 457 : audit [INF] from='osd.2 v1:172.21.15.104:6817/2713413369' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2020-09-08T15:30:54.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:54 smithi166 bash[9446]: cluster 2020-09-08T15:30:53.469018+0000 mgr.x (mgr.24118) 49 : cluster [DBG] pgmap v43: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:54.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:54 smithi166 bash[9446]: audit 2020-09-08T15:30:53.536161+0000 mon.a (mon.0) 454 : audit [INF] from='osd.2 v1:172.21.15.104:6817/2713413369' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2020-09-08T15:30:54.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:54 smithi166 bash[9446]: cluster 2020-09-08T15:30:53.536239+0000 mon.a (mon.0) 455 : cluster [DBG] osdmap e21: 3 total, 2 up, 2 in 2020-09-08T15:30:54.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:54 smithi166 bash[9446]: audit 2020-09-08T15:30:53.536463+0000 mon.a (mon.0) 456 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2020-09-08T15:30:54.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:54 smithi166 bash[9446]: audit 2020-09-08T15:30:53.537250+0000 mon.a (mon.0) 457 : audit [INF] from='osd.2 v1:172.21.15.104:6817/2713413369' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2020-09-08T15:30:54.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:54 smithi055 bash[9296]: cluster 2020-09-08T15:30:53.469018+0000 mgr.x (mgr.24118) 49 : cluster [DBG] pgmap v43: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:54.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:54 smithi055 bash[9296]: audit 2020-09-08T15:30:53.536161+0000 mon.a (mon.0) 454 : audit [INF] from='osd.2 v1:172.21.15.104:6817/2713413369' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2020-09-08T15:30:54.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:54 smithi055 bash[9296]: cluster 2020-09-08T15:30:53.536239+0000 mon.a (mon.0) 455 : cluster [DBG] osdmap e21: 3 total, 2 up, 2 in 2020-09-08T15:30:54.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:54 smithi055 bash[9296]: audit 2020-09-08T15:30:53.536463+0000 mon.a (mon.0) 456 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2020-09-08T15:30:54.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:54 smithi055 bash[9296]: audit 2020-09-08T15:30:53.537250+0000 mon.a (mon.0) 457 : audit [INF] from='osd.2 v1:172.21.15.104:6817/2713413369' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2020-09-08T15:30:55.806 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:55 smithi104 bash[9203]: audit 2020-09-08T15:30:54.538412+0000 mon.a (mon.0) 458 : audit [INF] from='osd.2 v1:172.21.15.104:6817/2713413369' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2020-09-08T15:30:55.807 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:55 smithi104 bash[9203]: cluster 2020-09-08T15:30:54.538508+0000 mon.a (mon.0) 459 : cluster [DBG] osdmap e22: 3 total, 2 up, 2 in 2020-09-08T15:30:55.807 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:55 smithi104 bash[9203]: audit 2020-09-08T15:30:54.539717+0000 mon.a (mon.0) 460 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2020-09-08T15:30:55.807 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:55 smithi104 bash[9203]: audit 2020-09-08T15:30:54.546436+0000 mon.a (mon.0) 461 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2020-09-08T15:30:55.807 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:55 smithi104 bash[9203]: audit 2020-09-08T15:30:54.559352+0000 mon.a (mon.0) 462 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:30:55.808 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:55 smithi104 bash[9203]: audit 2020-09-08T15:30:54.840092+0000 mon.a (mon.0) 463 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.027917\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028113\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028365\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028587\", \"created\": \"2020-09-08T15:30:46.237340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}}, \"last_daemon_update\": \"2020-09-08T15:30:47.028883\", \"last_device_update\": \"2020-09-08T15:30:54.840162\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:55.808 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:55 smithi104 bash[9203]: audit 2020-09-08T15:30:54.844120+0000 mon.a (mon.0) 464 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.027917\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028113\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028365\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028587\", \"created\": \"2020-09-08T15:30:46.237340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}}, \"last_daemon_update\": \"2020-09-08T15:30:47.028883\", \"last_device_update\": \"2020-09-08T15:30:54.840162\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:55.808 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:55 smithi104 bash[9203]: audit 2020-09-08T15:30:54.846046+0000 mon.a (mon.0) 465 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:30:55.809 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:55 smithi104 bash[9203]: audit 2020-09-08T15:30:54.853220+0000 mon.a (mon.0) 466 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:30:55.809 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:55 smithi104 bash[9203]: audit 2020-09-08T15:30:54.857602+0000 mon.a (mon.0) 467 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:30:55.809 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:55 smithi104 bash[9203]: audit 2020-09-08T15:30:54.862638+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:30:55.809 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:55 smithi104 bash[9203]: cluster 2020-09-08T15:30:55.469543+0000 mgr.x (mgr.24118) 50 : cluster [DBG] pgmap v46: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:55.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:55 smithi166 bash[9446]: audit 2020-09-08T15:30:54.538412+0000 mon.a (mon.0) 458 : audit [INF] from='osd.2 v1:172.21.15.104:6817/2713413369' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2020-09-08T15:30:55.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:55 smithi166 bash[9446]: cluster 2020-09-08T15:30:54.538508+0000 mon.a (mon.0) 459 : cluster [DBG] osdmap e22: 3 total, 2 up, 2 in 2020-09-08T15:30:55.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:55 smithi166 bash[9446]: audit 2020-09-08T15:30:54.539717+0000 mon.a (mon.0) 460 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2020-09-08T15:30:55.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:55 smithi166 bash[9446]: audit 2020-09-08T15:30:54.546436+0000 mon.a (mon.0) 461 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2020-09-08T15:30:55.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:55 smithi166 bash[9446]: audit 2020-09-08T15:30:54.559352+0000 mon.a (mon.0) 462 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:30:55.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:55 smithi166 bash[9446]: audit 2020-09-08T15:30:54.840092+0000 mon.a (mon.0) 463 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.027917\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028113\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028365\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028587\", \"created\": \"2020-09-08T15:30:46.237340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}}, \"last_daemon_update\": \"2020-09-08T15:30:47.028883\", \"last_device_update\": \"2020-09-08T15:30:54.840162\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:55.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:55 smithi166 bash[9446]: audit 2020-09-08T15:30:54.844120+0000 mon.a (mon.0) 464 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.027917\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028113\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028365\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028587\", \"created\": \"2020-09-08T15:30:46.237340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}}, \"last_daemon_update\": \"2020-09-08T15:30:47.028883\", \"last_device_update\": \"2020-09-08T15:30:54.840162\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:55.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:55 smithi166 bash[9446]: audit 2020-09-08T15:30:54.846046+0000 mon.a (mon.0) 465 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:30:55.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:55 smithi166 bash[9446]: audit 2020-09-08T15:30:54.853220+0000 mon.a (mon.0) 466 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:30:55.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:55 smithi166 bash[9446]: audit 2020-09-08T15:30:54.857602+0000 mon.a (mon.0) 467 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:30:55.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:55 smithi166 bash[9446]: audit 2020-09-08T15:30:54.862638+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:30:55.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:55 smithi166 bash[9446]: cluster 2020-09-08T15:30:55.469543+0000 mgr.x (mgr.24118) 50 : cluster [DBG] pgmap v46: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:55.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:55 smithi055 bash[9296]: audit 2020-09-08T15:30:54.538412+0000 mon.a (mon.0) 458 : audit [INF] from='osd.2 v1:172.21.15.104:6817/2713413369' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2020-09-08T15:30:55.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:55 smithi055 bash[9296]: cluster 2020-09-08T15:30:54.538508+0000 mon.a (mon.0) 459 : cluster [DBG] osdmap e22: 3 total, 2 up, 2 in 2020-09-08T15:30:55.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:55 smithi055 bash[9296]: audit 2020-09-08T15:30:54.539717+0000 mon.a (mon.0) 460 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2020-09-08T15:30:55.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:55 smithi055 bash[9296]: audit 2020-09-08T15:30:54.546436+0000 mon.a (mon.0) 461 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2020-09-08T15:30:55.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:55 smithi055 bash[9296]: audit 2020-09-08T15:30:54.559352+0000 mon.a (mon.0) 462 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:30:55.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:55 smithi055 bash[9296]: audit 2020-09-08T15:30:54.840092+0000 mon.a (mon.0) 463 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.027917\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028113\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028365\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028587\", \"created\": \"2020-09-08T15:30:46.237340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}}, \"last_daemon_update\": \"2020-09-08T15:30:47.028883\", \"last_device_update\": \"2020-09-08T15:30:54.840162\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:30:55.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:55 smithi055 bash[9296]: audit 2020-09-08T15:30:54.844120+0000 mon.a (mon.0) 464 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.027917\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028113\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028365\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028587\", \"created\": \"2020-09-08T15:30:46.237340\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}}, \"last_daemon_update\": \"2020-09-08T15:30:47.028883\", \"last_device_update\": \"2020-09-08T15:30:54.840162\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:30:55.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:55 smithi055 bash[9296]: audit 2020-09-08T15:30:54.846046+0000 mon.a (mon.0) 465 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:30:55.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:55 smithi055 bash[9296]: audit 2020-09-08T15:30:54.853220+0000 mon.a (mon.0) 466 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:30:55.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:55 smithi055 bash[9296]: audit 2020-09-08T15:30:54.857602+0000 mon.a (mon.0) 467 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:30:55.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:55 smithi055 bash[9296]: audit 2020-09-08T15:30:54.862638+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:30:55.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:55 smithi055 bash[9296]: cluster 2020-09-08T15:30:55.469543+0000 mgr.x (mgr.24118) 50 : cluster [DBG] pgmap v46: 1 pgs: 1 active+clean; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2020-09-08T15:30:56.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:56 smithi166 bash[9446]: cluster 2020-09-08T15:30:53.909655+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:30:56.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:56 smithi166 bash[9446]: cluster 2020-09-08T15:30:53.909834+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:30:56.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:56 smithi166 bash[9446]: cluster 2020-09-08T15:30:55.547232+0000 mon.a (mon.0) 469 : cluster [INF] osd.2 v1:172.21.15.104:6817/2713413369 boot 2020-09-08T15:30:56.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:56 smithi166 bash[9446]: cluster 2020-09-08T15:30:55.547281+0000 mon.a (mon.0) 470 : cluster [DBG] osdmap e23: 3 total, 3 up, 3 in 2020-09-08T15:30:56.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:56 smithi166 bash[9446]: audit 2020-09-08T15:30:55.547900+0000 mon.a (mon.0) 471 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2020-09-08T15:30:56.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:56 smithi055 bash[9296]: cluster 2020-09-08T15:30:53.909655+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:30:56.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:56 smithi055 bash[9296]: cluster 2020-09-08T15:30:53.909834+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:30:56.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:56 smithi055 bash[9296]: cluster 2020-09-08T15:30:55.547232+0000 mon.a (mon.0) 469 : cluster [INF] osd.2 v1:172.21.15.104:6817/2713413369 boot 2020-09-08T15:30:56.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:56 smithi055 bash[9296]: cluster 2020-09-08T15:30:55.547281+0000 mon.a (mon.0) 470 : cluster [DBG] osdmap e23: 3 total, 3 up, 3 in 2020-09-08T15:30:56.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:56 smithi055 bash[9296]: audit 2020-09-08T15:30:55.547900+0000 mon.a (mon.0) 471 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2020-09-08T15:30:56.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:56 smithi104 bash[9203]: cluster 2020-09-08T15:30:53.909655+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:30:56.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:56 smithi104 bash[9203]: cluster 2020-09-08T15:30:53.909834+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:30:56.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:56 smithi104 bash[9203]: cluster 2020-09-08T15:30:55.547232+0000 mon.a (mon.0) 469 : cluster [INF] osd.2 v1:172.21.15.104:6817/2713413369 boot 2020-09-08T15:30:56.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:56 smithi104 bash[9203]: cluster 2020-09-08T15:30:55.547281+0000 mon.a (mon.0) 470 : cluster [DBG] osdmap e23: 3 total, 3 up, 3 in 2020-09-08T15:30:56.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:56 smithi104 bash[9203]: audit 2020-09-08T15:30:55.547900+0000 mon.a (mon.0) 471 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2020-09-08T15:30:57.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:57 smithi166 bash[9446]: cluster 2020-09-08T15:30:56.572618+0000 mon.a (mon.0) 472 : cluster [DBG] osdmap e24: 3 total, 3 up, 3 in 2020-09-08T15:30:57.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:57 smithi166 bash[9446]: cluster 2020-09-08T15:30:57.470103+0000 mgr.x (mgr.24118) 51 : cluster [DBG] pgmap v49: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:30:57.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:57 smithi055 bash[9296]: cluster 2020-09-08T15:30:56.572618+0000 mon.a (mon.0) 472 : cluster [DBG] osdmap e24: 3 total, 3 up, 3 in 2020-09-08T15:30:57.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:57 smithi055 bash[9296]: cluster 2020-09-08T15:30:57.470103+0000 mgr.x (mgr.24118) 51 : cluster [DBG] pgmap v49: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:30:57.935 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:57 smithi104 bash[9203]: cluster 2020-09-08T15:30:56.572618+0000 mon.a (mon.0) 472 : cluster [DBG] osdmap e24: 3 total, 3 up, 3 in 2020-09-08T15:30:57.936 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:57 smithi104 bash[9203]: cluster 2020-09-08T15:30:57.470103+0000 mgr.x (mgr.24118) 51 : cluster [DBG] pgmap v49: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:30:58.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:58 smithi166 bash[9446]: audit 2020-09-08T15:30:57.760195+0000 mon.a (mon.0) 473 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:30:58.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:58 smithi166 bash[9446]: audit 2020-09-08T15:30:57.760469+0000 mgr.x (mgr.24118) 52 : audit [DBG] from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:30:58.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:58 smithi166 bash[9446]: audit 2020-09-08T15:30:57.762924+0000 mon.a (mon.0) 474 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:30:58.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:58 smithi166 bash[9446]: cephadm 2020-09-08T15:30:57.763201+0000 mgr.x (mgr.24118) 53 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:30:58.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:58 smithi166 bash[9446]: cephadm 2020-09-08T15:30:57.763322+0000 mgr.x (mgr.24118) 54 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:30:58.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:58 smithi166 bash[9446]: cephadm 2020-09-08T15:30:57.764245+0000 mgr.x (mgr.24118) 55 : cephadm [INF] Applying None on host smithi104... 2020-09-08T15:30:58.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:58 smithi166 bash[9446]: audit 2020-09-08T15:30:57.764386+0000 mon.a (mon.0) 475 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:30:58.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:30:58 smithi166 bash[9446]: audit 2020-09-08T15:30:57.766314+0000 mon.a (mon.0) 476 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:30:58.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:58 smithi055 bash[9296]: audit 2020-09-08T15:30:57.760195+0000 mon.a (mon.0) 473 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:30:58.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:58 smithi055 bash[9296]: audit 2020-09-08T15:30:57.760469+0000 mgr.x (mgr.24118) 52 : audit [DBG] from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:30:58.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:58 smithi055 bash[9296]: audit 2020-09-08T15:30:57.762924+0000 mon.a (mon.0) 474 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:30:58.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:58 smithi055 bash[9296]: cephadm 2020-09-08T15:30:57.763201+0000 mgr.x (mgr.24118) 53 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:30:58.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:58 smithi055 bash[9296]: cephadm 2020-09-08T15:30:57.763322+0000 mgr.x (mgr.24118) 54 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:30:58.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:58 smithi055 bash[9296]: cephadm 2020-09-08T15:30:57.764245+0000 mgr.x (mgr.24118) 55 : cephadm [INF] Applying None on host smithi104... 2020-09-08T15:30:58.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:58 smithi055 bash[9296]: audit 2020-09-08T15:30:57.764386+0000 mon.a (mon.0) 475 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:30:58.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:30:58 smithi055 bash[9296]: audit 2020-09-08T15:30:57.766314+0000 mon.a (mon.0) 476 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:30:58.907 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:58 smithi104 bash[9203]: audit 2020-09-08T15:30:57.760195+0000 mon.a (mon.0) 473 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:30:58.908 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:58 smithi104 bash[9203]: audit 2020-09-08T15:30:57.760469+0000 mgr.x (mgr.24118) 52 : audit [DBG] from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:30:58.908 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:58 smithi104 bash[9203]: audit 2020-09-08T15:30:57.762924+0000 mon.a (mon.0) 474 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:30:58.908 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:58 smithi104 bash[9203]: cephadm 2020-09-08T15:30:57.763201+0000 mgr.x (mgr.24118) 53 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:30:58.909 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:58 smithi104 bash[9203]: cephadm 2020-09-08T15:30:57.763322+0000 mgr.x (mgr.24118) 54 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:30:58.909 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:58 smithi104 bash[9203]: cephadm 2020-09-08T15:30:57.764245+0000 mgr.x (mgr.24118) 55 : cephadm [INF] Applying None on host smithi104... 2020-09-08T15:30:58.909 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:58 smithi104 bash[9203]: audit 2020-09-08T15:30:57.764386+0000 mon.a (mon.0) 475 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:30:58.911 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:30:58 smithi104 bash[9203]: audit 2020-09-08T15:30:57.766314+0000 mon.a (mon.0) 476 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:00.790 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:00 smithi104 bash[9203]: cluster 2020-09-08T15:30:59.470640+0000 mgr.x (mgr.24118) 56 : cluster [DBG] pgmap v50: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:00.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:00 smithi166 bash[9446]: cluster 2020-09-08T15:30:59.470640+0000 mgr.x (mgr.24118) 56 : cluster [DBG] pgmap v50: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:00.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:00 smithi055 bash[9296]: cluster 2020-09-08T15:30:59.470640+0000 mgr.x (mgr.24118) 56 : cluster [DBG] pgmap v50: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:02.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:02 smithi166 bash[9446]: cluster 2020-09-08T15:31:01.471188+0000 mgr.x (mgr.24118) 57 : cluster [DBG] pgmap v51: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:02.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:02 smithi166 bash[9446]: audit 2020-09-08T15:31:01.606517+0000 mon.a (mon.0) 477 : audit [INF] from='client.? v1:172.21.15.104:0/535670654' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e"}]: dispatch 2020-09-08T15:31:02.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:02 smithi166 bash[9446]: audit 2020-09-08T15:31:01.613899+0000 mon.a (mon.0) 478 : audit [INF] from='client.? v1:172.21.15.104:0/535670654' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e"}]': finished 2020-09-08T15:31:02.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:02 smithi166 bash[9446]: cluster 2020-09-08T15:31:01.613980+0000 mon.a (mon.0) 479 : cluster [DBG] osdmap e25: 4 total, 3 up, 3 in 2020-09-08T15:31:02.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:02 smithi166 bash[9446]: audit 2020-09-08T15:31:01.614100+0000 mon.a (mon.0) 480 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:02.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:02 smithi166 bash[9446]: audit 2020-09-08T15:31:02.285870+0000 mon.a (mon.0) 481 : audit [DBG] from='client.? v1:172.21.15.104:0/2314688937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:31:02.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:02 smithi055 bash[9296]: cluster 2020-09-08T15:31:01.471188+0000 mgr.x (mgr.24118) 57 : cluster [DBG] pgmap v51: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:02.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:02 smithi055 bash[9296]: audit 2020-09-08T15:31:01.606517+0000 mon.a (mon.0) 477 : audit [INF] from='client.? v1:172.21.15.104:0/535670654' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e"}]: dispatch 2020-09-08T15:31:02.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:02 smithi055 bash[9296]: audit 2020-09-08T15:31:01.613899+0000 mon.a (mon.0) 478 : audit [INF] from='client.? v1:172.21.15.104:0/535670654' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e"}]': finished 2020-09-08T15:31:02.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:02 smithi055 bash[9296]: cluster 2020-09-08T15:31:01.613980+0000 mon.a (mon.0) 479 : cluster [DBG] osdmap e25: 4 total, 3 up, 3 in 2020-09-08T15:31:02.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:02 smithi055 bash[9296]: audit 2020-09-08T15:31:01.614100+0000 mon.a (mon.0) 480 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:02.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:02 smithi055 bash[9296]: audit 2020-09-08T15:31:02.285870+0000 mon.a (mon.0) 481 : audit [DBG] from='client.? v1:172.21.15.104:0/2314688937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:31:02.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:02 smithi104 bash[9203]: cluster 2020-09-08T15:31:01.471188+0000 mgr.x (mgr.24118) 57 : cluster [DBG] pgmap v51: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:02.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:02 smithi104 bash[9203]: audit 2020-09-08T15:31:01.606517+0000 mon.a (mon.0) 477 : audit [INF] from='client.? v1:172.21.15.104:0/535670654' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e"}]: dispatch 2020-09-08T15:31:02.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:02 smithi104 bash[9203]: audit 2020-09-08T15:31:01.613899+0000 mon.a (mon.0) 478 : audit [INF] from='client.? v1:172.21.15.104:0/535670654' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e"}]': finished 2020-09-08T15:31:02.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:02 smithi104 bash[9203]: cluster 2020-09-08T15:31:01.613980+0000 mon.a (mon.0) 479 : cluster [DBG] osdmap e25: 4 total, 3 up, 3 in 2020-09-08T15:31:02.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:02 smithi104 bash[9203]: audit 2020-09-08T15:31:01.614100+0000 mon.a (mon.0) 480 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:02.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:02 smithi104 bash[9203]: audit 2020-09-08T15:31:02.285870+0000 mon.a (mon.0) 481 : audit [DBG] from='client.? v1:172.21.15.104:0/2314688937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:31:04.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:04 smithi166 bash[9446]: cluster 2020-09-08T15:31:02.937954+0000 osd.0 (osd.0) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:31:04.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:04 smithi166 bash[9446]: cluster 2020-09-08T15:31:02.938110+0000 osd.0 (osd.0) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:31:04.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:04 smithi166 bash[9446]: cluster 2020-09-08T15:31:03.471716+0000 mgr.x (mgr.24118) 58 : cluster [DBG] pgmap v53: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:04.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:04 smithi166 bash[9446]: cluster 2020-09-08T15:31:03.536998+0000 mon.a (mon.0) 482 : cluster [DBG] osdmap e26: 4 total, 3 up, 3 in 2020-09-08T15:31:04.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:04 smithi166 bash[9446]: audit 2020-09-08T15:31:03.537259+0000 mon.a (mon.0) 483 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:04.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:04 smithi055 bash[9296]: cluster 2020-09-08T15:31:02.937954+0000 osd.0 (osd.0) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:31:04.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:04 smithi055 bash[9296]: cluster 2020-09-08T15:31:02.938110+0000 osd.0 (osd.0) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:31:04.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:04 smithi055 bash[9296]: cluster 2020-09-08T15:31:03.471716+0000 mgr.x (mgr.24118) 58 : cluster [DBG] pgmap v53: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:04.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:04 smithi055 bash[9296]: cluster 2020-09-08T15:31:03.536998+0000 mon.a (mon.0) 482 : cluster [DBG] osdmap e26: 4 total, 3 up, 3 in 2020-09-08T15:31:04.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:04 smithi055 bash[9296]: audit 2020-09-08T15:31:03.537259+0000 mon.a (mon.0) 483 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:04.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:04 smithi104 bash[9203]: cluster 2020-09-08T15:31:02.937954+0000 osd.0 (osd.0) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:31:04.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:04 smithi104 bash[9203]: cluster 2020-09-08T15:31:02.938110+0000 osd.0 (osd.0) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:31:04.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:04 smithi104 bash[9203]: cluster 2020-09-08T15:31:03.471716+0000 mgr.x (mgr.24118) 58 : cluster [DBG] pgmap v53: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:04.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:04 smithi104 bash[9203]: cluster 2020-09-08T15:31:03.536998+0000 mon.a (mon.0) 482 : cluster [DBG] osdmap e26: 4 total, 3 up, 3 in 2020-09-08T15:31:04.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:04 smithi104 bash[9203]: audit 2020-09-08T15:31:03.537259+0000 mon.a (mon.0) 483 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:05.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:05 smithi166 bash[9446]: audit 2020-09-08T15:31:05.268211+0000 mon.a (mon.0) 484 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:05.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:05 smithi055 bash[9296]: audit 2020-09-08T15:31:05.268211+0000 mon.a (mon.0) 484 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:05.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:05 smithi104 bash[9203]: audit 2020-09-08T15:31:05.268211+0000 mon.a (mon.0) 484 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:06.752 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:06 smithi104 bash[9203]: cluster 2020-09-08T15:31:05.472268+0000 mgr.x (mgr.24118) 59 : cluster [DBG] pgmap v55: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:06.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:06 smithi166 bash[9446]: cluster 2020-09-08T15:31:05.472268+0000 mgr.x (mgr.24118) 59 : cluster [DBG] pgmap v55: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:06.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:06 smithi055 bash[9296]: cluster 2020-09-08T15:31:05.472268+0000 mgr.x (mgr.24118) 59 : cluster [DBG] pgmap v55: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:07.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:07 smithi166 bash[9446]: cluster 2020-09-08T15:31:07.472838+0000 mgr.x (mgr.24118) 60 : cluster [DBG] pgmap v56: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:07.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:07 smithi055 bash[9296]: cluster 2020-09-08T15:31:07.472838+0000 mgr.x (mgr.24118) 60 : cluster [DBG] pgmap v56: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:07.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:07 smithi104 bash[9203]: cluster 2020-09-08T15:31:07.472838+0000 mgr.x (mgr.24118) 60 : cluster [DBG] pgmap v56: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:10.734 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:10 smithi104 bash[9203]: cluster 2020-09-08T15:31:09.473353+0000 mgr.x (mgr.24118) 61 : cluster [DBG] pgmap v57: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:10.734 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:10 smithi104 bash[9203]: audit 2020-09-08T15:31:09.590885+0000 mon.a (mon.0) 485 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2020-09-08T15:31:10.734 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:10 smithi104 bash[9203]: audit 2020-09-08T15:31:09.592761+0000 mon.a (mon.0) 486 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:31:10.735 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:10 smithi104 bash[9203]: audit 2020-09-08T15:31:09.594876+0000 mon.a (mon.0) 487 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.3", "key": "container_image"}]: dispatch 2020-09-08T15:31:10.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:10 smithi166 bash[9446]: cluster 2020-09-08T15:31:09.473353+0000 mgr.x (mgr.24118) 61 : cluster [DBG] pgmap v57: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:10.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:10 smithi166 bash[9446]: audit 2020-09-08T15:31:09.590885+0000 mon.a (mon.0) 485 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2020-09-08T15:31:10.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:10 smithi166 bash[9446]: audit 2020-09-08T15:31:09.592761+0000 mon.a (mon.0) 486 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:31:10.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:10 smithi166 bash[9446]: audit 2020-09-08T15:31:09.594876+0000 mon.a (mon.0) 487 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.3", "key": "container_image"}]: dispatch 2020-09-08T15:31:10.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:10 smithi055 bash[9296]: cluster 2020-09-08T15:31:09.473353+0000 mgr.x (mgr.24118) 61 : cluster [DBG] pgmap v57: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:10.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:10 smithi055 bash[9296]: audit 2020-09-08T15:31:09.590885+0000 mon.a (mon.0) 485 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2020-09-08T15:31:10.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:10 smithi055 bash[9296]: audit 2020-09-08T15:31:09.592761+0000 mon.a (mon.0) 486 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:31:10.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:10 smithi055 bash[9296]: audit 2020-09-08T15:31:09.594876+0000 mon.a (mon.0) 487 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.3", "key": "container_image"}]: dispatch 2020-09-08T15:31:11.668 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:11 smithi104 bash[9203]: cephadm 2020-09-08T15:31:09.596377+0000 mgr.x (mgr.24118) 62 : cephadm [INF] Deploying daemon osd.3 on smithi104 2020-09-08T15:31:11.751 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 3 on host 'smithi104' 2020-09-08T15:31:11.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:11 smithi166 bash[9446]: cephadm 2020-09-08T15:31:09.596377+0000 mgr.x (mgr.24118) 62 : cephadm [INF] Deploying daemon osd.3 on smithi104 2020-09-08T15:31:11.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:11 smithi055 bash[9296]: cephadm 2020-09-08T15:31:09.596377+0000 mgr.x (mgr.24118) 62 : cephadm [INF] Deploying daemon osd.3 on smithi104 2020-09-08T15:31:12.860 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:12 smithi104 bash[9203]: cluster 2020-09-08T15:31:11.473875+0000 mgr.x (mgr.24118) 63 : cluster [DBG] pgmap v58: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:12.860 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:12 smithi104 bash[9203]: audit 2020-09-08T15:31:11.752807+0000 mon.a (mon.0) 488 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.027917\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028113\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028365\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028587\", \"created\": \"2020-09-08T15:30:46.237340\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_device_update\": \"2020-09-08T15:30:54.840162\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:31:12.860 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:12 smithi104 bash[9203]: audit 2020-09-08T15:31:11.753940+0000 mon.a (mon.0) 489 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:31:12.861 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:12 smithi104 bash[9203]: audit 2020-09-08T15:31:11.756624+0000 mon.a (mon.0) 490 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.027917\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028113\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028365\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028587\", \"created\": \"2020-09-08T15:30:46.237340\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_device_update\": \"2020-09-08T15:30:54.840162\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:31:12.862 INFO:teuthology.orchestra.run.smithi104:osd.3> sudo journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.3.service 2020-09-08T15:31:12.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:12 smithi166 bash[9446]: cluster 2020-09-08T15:31:11.473875+0000 mgr.x (mgr.24118) 63 : cluster [DBG] pgmap v58: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:12.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:12 smithi166 bash[9446]: audit 2020-09-08T15:31:11.752807+0000 mon.a (mon.0) 488 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.027917\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028113\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028365\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028587\", \"created\": \"2020-09-08T15:30:46.237340\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_device_update\": \"2020-09-08T15:30:54.840162\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:31:12.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:12 smithi166 bash[9446]: audit 2020-09-08T15:31:11.753940+0000 mon.a (mon.0) 489 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:31:12.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:12 smithi166 bash[9446]: audit 2020-09-08T15:31:11.756624+0000 mon.a (mon.0) 490 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.027917\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028113\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028365\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028587\", \"created\": \"2020-09-08T15:30:46.237340\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_device_update\": \"2020-09-08T15:30:54.840162\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:31:12.883 INFO:tasks.cephadm:Deploying osd.4 on smithi055 with /dev/vg_nvme/lv_4... 2020-09-08T15:31:12.884 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2020-09-08T15:31:12.897 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:12 smithi055 bash[9296]: cluster 2020-09-08T15:31:11.473875+0000 mgr.x (mgr.24118) 63 : cluster [DBG] pgmap v58: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:12.898 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:12 smithi055 bash[9296]: audit 2020-09-08T15:31:11.752807+0000 mon.a (mon.0) 488 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.027917\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028113\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028365\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028587\", \"created\": \"2020-09-08T15:30:46.237340\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_device_update\": \"2020-09-08T15:30:54.840162\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:31:12.899 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:12 smithi055 bash[9296]: audit 2020-09-08T15:31:11.753940+0000 mon.a (mon.0) 489 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:31:12.899 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:12 smithi055 bash[9296]: audit 2020-09-08T15:31:11.756624+0000 mon.a (mon.0) 490 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.027917\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028113\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028365\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:30:47.028587\", \"created\": \"2020-09-08T15:30:46.237340\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_device_update\": \"2020-09-08T15:30:54.840162\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:31:12.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:-- Logs begin at Tue 2020-09-01 16:49:30 UTC. -- 2020-09-08T15:31:13.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:13 smithi104 bash[9203]: audit 2020-09-08T15:31:12.580964+0000 mon.a (mon.0) 491 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580086\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580289\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580578\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580831\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.581077\", \"created\": \"2020-09-08T15:31:11.717339\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:31:12.581399\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:31:13.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:13 smithi104 bash[9203]: audit 2020-09-08T15:31:12.584718+0000 mon.a (mon.0) 492 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580086\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580289\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580578\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580831\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.581077\", \"created\": \"2020-09-08T15:31:11.717339\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:31:12.581399\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:31:13.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:13 smithi104 bash[9203]: audit 2020-09-08T15:31:12.586399+0000 mon.a (mon.0) 493 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:13.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:13 smithi166 bash[9446]: audit 2020-09-08T15:31:12.580964+0000 mon.a (mon.0) 491 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580086\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580289\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580578\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580831\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.581077\", \"created\": \"2020-09-08T15:31:11.717339\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:31:12.581399\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:31:13.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:13 smithi166 bash[9446]: audit 2020-09-08T15:31:12.584718+0000 mon.a (mon.0) 492 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580086\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580289\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580578\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580831\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.581077\", \"created\": \"2020-09-08T15:31:11.717339\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:31:12.581399\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:31:13.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:13 smithi166 bash[9446]: audit 2020-09-08T15:31:12.586399+0000 mon.a (mon.0) 493 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:13.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:13 smithi055 bash[9296]: audit 2020-09-08T15:31:12.580964+0000 mon.a (mon.0) 491 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580086\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580289\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580578\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580831\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.581077\", \"created\": \"2020-09-08T15:31:11.717339\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:31:12.581399\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:31:13.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:13 smithi055 bash[9296]: audit 2020-09-08T15:31:12.584718+0000 mon.a (mon.0) 492 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580086\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580289\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580578\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580831\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.581077\", \"created\": \"2020-09-08T15:31:11.717339\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:31:12.581399\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:31:13.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:13 smithi055 bash[9296]: audit 2020-09-08T15:31:12.586399+0000 mon.a (mon.0) 493 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:14.175 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:31:13 smithi104 bash[22305]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2020-09-08T15:31:14.175 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:31:13 smithi104 bash[22305]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_1 --path /var/lib/ceph/osd/ceph-3 --no-mon-config 2020-09-08T15:31:14.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:31:13 smithi104 bash[22305]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3/block 2020-09-08T15:31:14.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:31:13 smithi104 bash[22305]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-3/block 2020-09-08T15:31:14.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:31:13 smithi104 bash[22305]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2020-09-08T15:31:14.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:31:13 smithi104 bash[22305]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2020-09-08T15:31:14.177 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:31:13 smithi104 bash[22305]: --> ceph-volume lvm activate successful for osd ID: 3 2020-09-08T15:31:14.653 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:14 smithi055 bash[9296]: cluster 2020-09-08T15:31:13.474264+0000 mgr.x (mgr.24118) 64 : cluster [DBG] pgmap v59: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:14.758 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:14 smithi104 bash[9203]: cluster 2020-09-08T15:31:13.474264+0000 mgr.x (mgr.24118) 64 : cluster [DBG] pgmap v59: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:14.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:14 smithi166 bash[9446]: cluster 2020-09-08T15:31:13.474264+0000 mgr.x (mgr.24118) 64 : cluster [DBG] pgmap v59: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:15.175 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:31:14 smithi104 bash[22305]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:31:16.349 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping: /dev/vg_nvme/lv_4 2020-09-08T15:31:16.350 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2020-09-08T15:31:16.350 INFO:teuthology.orchestra.run.smithi055.stderr: stderr: 10+0 records in 2020-09-08T15:31:16.350 INFO:teuthology.orchestra.run.smithi055.stderr:10+0 records out 2020-09-08T15:31:16.350 INFO:teuthology.orchestra.run.smithi055.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0220837 s, 475 MB/s 2020-09-08T15:31:16.351 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping successful for: 2020-09-08T15:31:16.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:16 smithi166 bash[9446]: cluster 2020-09-08T15:31:15.474760+0000 mgr.x (mgr.24118) 65 : cluster [DBG] pgmap v60: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:16.903 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:16 smithi104 bash[9203]: cluster 2020-09-08T15:31:15.474760+0000 mgr.x (mgr.24118) 65 : cluster [DBG] pgmap v60: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:16.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:16 smithi055 bash[9296]: cluster 2020-09-08T15:31:15.474760+0000 mgr.x (mgr.24118) 65 : cluster [DBG] pgmap v60: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:16.983 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph orch daemon add osd smithi055:vg_nvme/lv_4 2020-09-08T15:31:17.175 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:31:16 smithi104 bash[22305]: debug 2020-09-08T15:31:16.905+0000 7fb4aa8baf40 -1 Falling back to public interface 2020-09-08T15:31:17.822 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:31:17 smithi104 bash[22305]: debug 2020-09-08T15:31:17.561+0000 7fb4aa8baf40 -1 osd.3 0 log_to_monitors {default=true} 2020-09-08T15:31:18.654 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:18 smithi055 bash[9296]: cluster 2020-09-08T15:31:17.475337+0000 mgr.x (mgr.24118) 66 : cluster [DBG] pgmap v61: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:18.654 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:18 smithi055 bash[9296]: audit 2020-09-08T15:31:17.571383+0000 mon.a (mon.0) 494 : audit [INF] from='osd.3 v1:172.21.15.104:6825/1495946248' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2020-09-08T15:31:18.654 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:18 smithi055 bash[9296]: audit 2020-09-08T15:31:18.463584+0000 mon.a (mon.0) 495 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:31:18.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:18 smithi166 bash[9446]: cluster 2020-09-08T15:31:17.475337+0000 mgr.x (mgr.24118) 66 : cluster [DBG] pgmap v61: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:18.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:18 smithi166 bash[9446]: audit 2020-09-08T15:31:17.571383+0000 mon.a (mon.0) 494 : audit [INF] from='osd.3 v1:172.21.15.104:6825/1495946248' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2020-09-08T15:31:18.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:18 smithi166 bash[9446]: audit 2020-09-08T15:31:18.463584+0000 mon.a (mon.0) 495 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:31:18.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:18 smithi104 bash[9203]: cluster 2020-09-08T15:31:17.475337+0000 mgr.x (mgr.24118) 66 : cluster [DBG] pgmap v61: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:18.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:18 smithi104 bash[9203]: audit 2020-09-08T15:31:17.571383+0000 mon.a (mon.0) 494 : audit [INF] from='osd.3 v1:172.21.15.104:6825/1495946248' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2020-09-08T15:31:18.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:18 smithi104 bash[9203]: audit 2020-09-08T15:31:18.463584+0000 mon.a (mon.0) 495 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:31:19.777 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:19 smithi055 bash[9296]: audit 2020-09-08T15:31:18.498163+0000 mon.a (mon.0) 496 : audit [INF] from='osd.3 v1:172.21.15.104:6825/1495946248' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2020-09-08T15:31:19.778 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:19 smithi055 bash[9296]: cluster 2020-09-08T15:31:18.498240+0000 mon.a (mon.0) 497 : cluster [DBG] osdmap e27: 4 total, 3 up, 3 in 2020-09-08T15:31:19.778 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:19 smithi055 bash[9296]: audit 2020-09-08T15:31:18.499155+0000 mon.a (mon.0) 498 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:19.778 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:19 smithi055 bash[9296]: audit 2020-09-08T15:31:18.499874+0000 mon.a (mon.0) 499 : audit [INF] from='osd.3 v1:172.21.15.104:6825/1495946248' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2020-09-08T15:31:19.779 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:19 smithi055 bash[9296]: audit 2020-09-08T15:31:18.755961+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580086\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580289\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580578\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580831\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.581077\", \"created\": \"2020-09-08T15:31:11.717339\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:31:12.581399\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:31:19.779 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:19 smithi055 bash[9296]: audit 2020-09-08T15:31:18.759929+0000 mon.a (mon.0) 501 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580086\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580289\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580578\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580831\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.581077\", \"created\": \"2020-09-08T15:31:11.717339\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:31:12.581399\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:31:19.780 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:19 smithi055 bash[9296]: audit 2020-09-08T15:31:18.762375+0000 mon.a (mon.0) 502 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:31:19.780 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:19 smithi055 bash[9296]: audit 2020-09-08T15:31:18.765015+0000 mon.a (mon.0) 503 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:31:19.780 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:19 smithi055 bash[9296]: audit 2020-09-08T15:31:18.769401+0000 mon.a (mon.0) 504 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:31:19.780 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:19 smithi055 bash[9296]: audit 2020-09-08T15:31:18.773949+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:31:19.781 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:19 smithi055 bash[9296]: audit 2020-09-08T15:31:19.497591+0000 mon.a (mon.0) 506 : audit [INF] from='osd.3 v1:172.21.15.104:6825/1495946248' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2020-09-08T15:31:19.781 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:19 smithi055 bash[9296]: cluster 2020-09-08T15:31:19.497642+0000 mon.a (mon.0) 507 : cluster [DBG] osdmap e28: 4 total, 3 up, 3 in 2020-09-08T15:31:19.781 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:19 smithi055 bash[9296]: audit 2020-09-08T15:31:19.497885+0000 mon.a (mon.0) 508 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:19.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:19 smithi166 bash[9446]: audit 2020-09-08T15:31:18.498163+0000 mon.a (mon.0) 496 : audit [INF] from='osd.3 v1:172.21.15.104:6825/1495946248' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2020-09-08T15:31:19.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:19 smithi166 bash[9446]: cluster 2020-09-08T15:31:18.498240+0000 mon.a (mon.0) 497 : cluster [DBG] osdmap e27: 4 total, 3 up, 3 in 2020-09-08T15:31:19.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:19 smithi166 bash[9446]: audit 2020-09-08T15:31:18.499155+0000 mon.a (mon.0) 498 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:19.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:19 smithi166 bash[9446]: audit 2020-09-08T15:31:18.499874+0000 mon.a (mon.0) 499 : audit [INF] from='osd.3 v1:172.21.15.104:6825/1495946248' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2020-09-08T15:31:19.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:19 smithi166 bash[9446]: audit 2020-09-08T15:31:18.755961+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580086\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580289\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580578\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580831\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.581077\", \"created\": \"2020-09-08T15:31:11.717339\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:31:12.581399\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:31:19.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:19 smithi166 bash[9446]: audit 2020-09-08T15:31:18.759929+0000 mon.a (mon.0) 501 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580086\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580289\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580578\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580831\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.581077\", \"created\": \"2020-09-08T15:31:11.717339\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:31:12.581399\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:31:19.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:19 smithi166 bash[9446]: audit 2020-09-08T15:31:18.762375+0000 mon.a (mon.0) 502 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:31:19.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:19 smithi166 bash[9446]: audit 2020-09-08T15:31:18.765015+0000 mon.a (mon.0) 503 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:31:19.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:19 smithi166 bash[9446]: audit 2020-09-08T15:31:18.769401+0000 mon.a (mon.0) 504 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:31:19.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:19 smithi166 bash[9446]: audit 2020-09-08T15:31:18.773949+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:31:19.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:19 smithi166 bash[9446]: audit 2020-09-08T15:31:19.497591+0000 mon.a (mon.0) 506 : audit [INF] from='osd.3 v1:172.21.15.104:6825/1495946248' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2020-09-08T15:31:19.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:19 smithi166 bash[9446]: cluster 2020-09-08T15:31:19.497642+0000 mon.a (mon.0) 507 : cluster [DBG] osdmap e28: 4 total, 3 up, 3 in 2020-09-08T15:31:19.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:19 smithi166 bash[9446]: audit 2020-09-08T15:31:19.497885+0000 mon.a (mon.0) 508 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:19.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:31:19 smithi104 bash[22305]: debug 2020-09-08T15:31:19.497+0000 7fb4a0e36700 -1 osd.3 0 waiting for initial osdmap 2020-09-08T15:31:19.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:19 smithi104 bash[9203]: audit 2020-09-08T15:31:18.498163+0000 mon.a (mon.0) 496 : audit [INF] from='osd.3 v1:172.21.15.104:6825/1495946248' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2020-09-08T15:31:19.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:19 smithi104 bash[9203]: cluster 2020-09-08T15:31:18.498240+0000 mon.a (mon.0) 497 : cluster [DBG] osdmap e27: 4 total, 3 up, 3 in 2020-09-08T15:31:19.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:19 smithi104 bash[9203]: audit 2020-09-08T15:31:18.499155+0000 mon.a (mon.0) 498 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:19.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:19 smithi104 bash[9203]: audit 2020-09-08T15:31:18.499874+0000 mon.a (mon.0) 499 : audit [INF] from='osd.3 v1:172.21.15.104:6825/1495946248' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2020-09-08T15:31:19.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:19 smithi104 bash[9203]: audit 2020-09-08T15:31:18.755961+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580086\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580289\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580578\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580831\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.581077\", \"created\": \"2020-09-08T15:31:11.717339\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:31:12.581399\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]: dispatch 2020-09-08T15:31:19.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:19 smithi104 bash[9203]: audit 2020-09-08T15:31:18.759929+0000 mon.a (mon.0) 501 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580086\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580289\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580578\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580831\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.581077\", \"created\": \"2020-09-08T15:31:11.717339\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:31:12.581399\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:26:23.836251\"}"}]': finished 2020-09-08T15:31:19.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:19 smithi104 bash[9203]: audit 2020-09-08T15:31:18.762375+0000 mon.a (mon.0) 502 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:31:19.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:19 smithi104 bash[9203]: audit 2020-09-08T15:31:18.765015+0000 mon.a (mon.0) 503 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:31:19.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:19 smithi104 bash[9203]: audit 2020-09-08T15:31:18.769401+0000 mon.a (mon.0) 504 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:31:19.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:19 smithi104 bash[9203]: audit 2020-09-08T15:31:18.773949+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:31:19.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:19 smithi104 bash[9203]: audit 2020-09-08T15:31:19.497591+0000 mon.a (mon.0) 506 : audit [INF] from='osd.3 v1:172.21.15.104:6825/1495946248' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2020-09-08T15:31:19.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:19 smithi104 bash[9203]: cluster 2020-09-08T15:31:19.497642+0000 mon.a (mon.0) 507 : cluster [DBG] osdmap e28: 4 total, 3 up, 3 in 2020-09-08T15:31:19.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:19 smithi104 bash[9203]: audit 2020-09-08T15:31:19.497885+0000 mon.a (mon.0) 508 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:20.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:20 smithi166 bash[9446]: cluster 2020-09-08T15:31:19.475742+0000 mgr.x (mgr.24118) 67 : cluster [DBG] pgmap v63: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:20.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:20 smithi166 bash[9446]: audit 2020-09-08T15:31:19.504187+0000 mon.a (mon.0) 509 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:20.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:20 smithi166 bash[9446]: audit 2020-09-08T15:31:20.147157+0000 mon.a (mon.0) 510 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:31:20.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:20 smithi166 bash[9446]: audit 2020-09-08T15:31:20.151026+0000 mon.a (mon.0) 511 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:31:20.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:20 smithi166 bash[9446]: audit 2020-09-08T15:31:20.152966+0000 mon.a (mon.0) 512 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:31:20.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:20 smithi166 bash[9446]: audit 2020-09-08T15:31:20.155026+0000 mon.a (mon.0) 513 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:20.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:20 smithi166 bash[9446]: audit 2020-09-08T15:31:20.503748+0000 mon.a (mon.0) 514 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:20.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:20 smithi055 bash[9296]: cluster 2020-09-08T15:31:19.475742+0000 mgr.x (mgr.24118) 67 : cluster [DBG] pgmap v63: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:20.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:20 smithi055 bash[9296]: audit 2020-09-08T15:31:19.504187+0000 mon.a (mon.0) 509 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:20.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:20 smithi055 bash[9296]: audit 2020-09-08T15:31:20.147157+0000 mon.a (mon.0) 510 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:31:20.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:20 smithi055 bash[9296]: audit 2020-09-08T15:31:20.151026+0000 mon.a (mon.0) 511 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:31:20.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:20 smithi055 bash[9296]: audit 2020-09-08T15:31:20.152966+0000 mon.a (mon.0) 512 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:31:20.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:20 smithi055 bash[9296]: audit 2020-09-08T15:31:20.155026+0000 mon.a (mon.0) 513 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:20.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:20 smithi055 bash[9296]: audit 2020-09-08T15:31:20.503748+0000 mon.a (mon.0) 514 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:20.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:20 smithi104 bash[9203]: cluster 2020-09-08T15:31:19.475742+0000 mgr.x (mgr.24118) 67 : cluster [DBG] pgmap v63: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:20.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:20 smithi104 bash[9203]: audit 2020-09-08T15:31:19.504187+0000 mon.a (mon.0) 509 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:20.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:20 smithi104 bash[9203]: audit 2020-09-08T15:31:20.147157+0000 mon.a (mon.0) 510 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:31:20.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:20 smithi104 bash[9203]: audit 2020-09-08T15:31:20.151026+0000 mon.a (mon.0) 511 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:31:20.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:20 smithi104 bash[9203]: audit 2020-09-08T15:31:20.152966+0000 mon.a (mon.0) 512 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:31:20.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:20 smithi104 bash[9203]: audit 2020-09-08T15:31:20.155026+0000 mon.a (mon.0) 513 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:20.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:20 smithi104 bash[9203]: audit 2020-09-08T15:31:20.503748+0000 mon.a (mon.0) 514 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:21.801 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:21 smithi055 bash[9296]: cluster 2020-09-08T15:31:18.528549+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:31:21.802 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:21 smithi055 bash[9296]: cluster 2020-09-08T15:31:18.528711+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:31:21.802 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:21 smithi055 bash[9296]: audit 2020-09-08T15:31:20.147654+0000 mgr.x (mgr.24118) 68 : audit [DBG] from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:31:21.802 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:21 smithi055 bash[9296]: cephadm 2020-09-08T15:31:20.151037+0000 mgr.x (mgr.24118) 69 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:31:21.802 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:21 smithi055 bash[9296]: cephadm 2020-09-08T15:31:20.151193+0000 mgr.x (mgr.24118) 70 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:31:21.802 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:21 smithi055 bash[9296]: cephadm 2020-09-08T15:31:20.152428+0000 mgr.x (mgr.24118) 71 : cephadm [INF] Applying None on host smithi055... 2020-09-08T15:31:21.803 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:21 smithi055 bash[9296]: cluster 2020-09-08T15:31:20.515367+0000 mon.a (mon.0) 515 : cluster [INF] osd.3 v1:172.21.15.104:6825/1495946248 boot 2020-09-08T15:31:21.803 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:21 smithi055 bash[9296]: cluster 2020-09-08T15:31:20.515448+0000 mon.a (mon.0) 516 : cluster [DBG] osdmap e29: 4 total, 4 up, 4 in 2020-09-08T15:31:21.806 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:21 smithi055 bash[9296]: audit 2020-09-08T15:31:20.516149+0000 mon.a (mon.0) 517 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:21.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:21 smithi166 bash[9446]: cluster 2020-09-08T15:31:18.528549+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:31:21.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:21 smithi166 bash[9446]: cluster 2020-09-08T15:31:18.528711+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:31:21.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:21 smithi166 bash[9446]: audit 2020-09-08T15:31:20.147654+0000 mgr.x (mgr.24118) 68 : audit [DBG] from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:31:21.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:21 smithi166 bash[9446]: cephadm 2020-09-08T15:31:20.151037+0000 mgr.x (mgr.24118) 69 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:31:21.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:21 smithi166 bash[9446]: cephadm 2020-09-08T15:31:20.151193+0000 mgr.x (mgr.24118) 70 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:31:21.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:21 smithi166 bash[9446]: cephadm 2020-09-08T15:31:20.152428+0000 mgr.x (mgr.24118) 71 : cephadm [INF] Applying None on host smithi055... 2020-09-08T15:31:21.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:21 smithi166 bash[9446]: cluster 2020-09-08T15:31:20.515367+0000 mon.a (mon.0) 515 : cluster [INF] osd.3 v1:172.21.15.104:6825/1495946248 boot 2020-09-08T15:31:21.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:21 smithi166 bash[9446]: cluster 2020-09-08T15:31:20.515448+0000 mon.a (mon.0) 516 : cluster [DBG] osdmap e29: 4 total, 4 up, 4 in 2020-09-08T15:31:21.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:21 smithi166 bash[9446]: audit 2020-09-08T15:31:20.516149+0000 mon.a (mon.0) 517 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:21.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:21 smithi104 bash[9203]: cluster 2020-09-08T15:31:18.528549+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:31:21.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:21 smithi104 bash[9203]: cluster 2020-09-08T15:31:18.528711+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:31:21.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:21 smithi104 bash[9203]: audit 2020-09-08T15:31:20.147654+0000 mgr.x (mgr.24118) 68 : audit [DBG] from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:31:21.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:21 smithi104 bash[9203]: cephadm 2020-09-08T15:31:20.151037+0000 mgr.x (mgr.24118) 69 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:31:21.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:21 smithi104 bash[9203]: cephadm 2020-09-08T15:31:20.151193+0000 mgr.x (mgr.24118) 70 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:31:21.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:21 smithi104 bash[9203]: cephadm 2020-09-08T15:31:20.152428+0000 mgr.x (mgr.24118) 71 : cephadm [INF] Applying None on host smithi055... 2020-09-08T15:31:21.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:21 smithi104 bash[9203]: cluster 2020-09-08T15:31:20.515367+0000 mon.a (mon.0) 515 : cluster [INF] osd.3 v1:172.21.15.104:6825/1495946248 boot 2020-09-08T15:31:21.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:21 smithi104 bash[9203]: cluster 2020-09-08T15:31:20.515448+0000 mon.a (mon.0) 516 : cluster [DBG] osdmap e29: 4 total, 4 up, 4 in 2020-09-08T15:31:21.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:21 smithi104 bash[9203]: audit 2020-09-08T15:31:20.516149+0000 mon.a (mon.0) 517 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2020-09-08T15:31:22.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:22 smithi166 bash[9446]: cluster 2020-09-08T15:31:21.476306+0000 mgr.x (mgr.24118) 72 : cluster [DBG] pgmap v66: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:22.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:22 smithi166 bash[9446]: cluster 2020-09-08T15:31:21.533190+0000 mon.a (mon.0) 518 : cluster [DBG] osdmap e30: 4 total, 4 up, 4 in 2020-09-08T15:31:22.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:22 smithi055 bash[9296]: cluster 2020-09-08T15:31:21.476306+0000 mgr.x (mgr.24118) 72 : cluster [DBG] pgmap v66: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:22.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:22 smithi055 bash[9296]: cluster 2020-09-08T15:31:21.533190+0000 mon.a (mon.0) 518 : cluster [DBG] osdmap e30: 4 total, 4 up, 4 in 2020-09-08T15:31:22.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:22 smithi104 bash[9203]: cluster 2020-09-08T15:31:21.476306+0000 mgr.x (mgr.24118) 72 : cluster [DBG] pgmap v66: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2020-09-08T15:31:22.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:22 smithi104 bash[9203]: cluster 2020-09-08T15:31:21.533190+0000 mon.a (mon.0) 518 : cluster [DBG] osdmap e30: 4 total, 4 up, 4 in 2020-09-08T15:31:24.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:24 smithi166 bash[9446]: cluster 2020-09-08T15:31:23.476821+0000 mgr.x (mgr.24118) 73 : cluster [DBG] pgmap v68: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:24.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:24 smithi166 bash[9446]: audit 2020-09-08T15:31:24.191512+0000 mon.a (mon.0) 519 : audit [INF] from='client.24253 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8334c70-f776-4652-944d-aed35a8e43f7"}]: dispatch 2020-09-08T15:31:24.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:24 smithi166 bash[9446]: audit 2020-09-08T15:31:24.193049+0000 mon.b (mon.1) 7 : audit [INF] from='client.? v1:172.21.15.55:0/718733343' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8334c70-f776-4652-944d-aed35a8e43f7"}]: dispatch 2020-09-08T15:31:24.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:24 smithi166 bash[9446]: audit 2020-09-08T15:31:24.199273+0000 mon.a (mon.0) 520 : audit [INF] from='client.24253 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8334c70-f776-4652-944d-aed35a8e43f7"}]': finished 2020-09-08T15:31:24.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:24 smithi166 bash[9446]: cluster 2020-09-08T15:31:24.199356+0000 mon.a (mon.0) 521 : cluster [DBG] osdmap e31: 5 total, 4 up, 4 in 2020-09-08T15:31:24.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:24 smithi166 bash[9446]: audit 2020-09-08T15:31:24.199567+0000 mon.a (mon.0) 522 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:24.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:24 smithi055 bash[9296]: cluster 2020-09-08T15:31:23.476821+0000 mgr.x (mgr.24118) 73 : cluster [DBG] pgmap v68: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:24.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:24 smithi055 bash[9296]: audit 2020-09-08T15:31:24.191512+0000 mon.a (mon.0) 519 : audit [INF] from='client.24253 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8334c70-f776-4652-944d-aed35a8e43f7"}]: dispatch 2020-09-08T15:31:24.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:24 smithi055 bash[9296]: audit 2020-09-08T15:31:24.193049+0000 mon.b (mon.1) 7 : audit [INF] from='client.? v1:172.21.15.55:0/718733343' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8334c70-f776-4652-944d-aed35a8e43f7"}]: dispatch 2020-09-08T15:31:24.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:24 smithi055 bash[9296]: audit 2020-09-08T15:31:24.199273+0000 mon.a (mon.0) 520 : audit [INF] from='client.24253 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8334c70-f776-4652-944d-aed35a8e43f7"}]': finished 2020-09-08T15:31:24.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:24 smithi055 bash[9296]: cluster 2020-09-08T15:31:24.199356+0000 mon.a (mon.0) 521 : cluster [DBG] osdmap e31: 5 total, 4 up, 4 in 2020-09-08T15:31:24.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:24 smithi055 bash[9296]: audit 2020-09-08T15:31:24.199567+0000 mon.a (mon.0) 522 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:24.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:24 smithi104 bash[9203]: cluster 2020-09-08T15:31:23.476821+0000 mgr.x (mgr.24118) 73 : cluster [DBG] pgmap v68: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:24.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:24 smithi104 bash[9203]: audit 2020-09-08T15:31:24.191512+0000 mon.a (mon.0) 519 : audit [INF] from='client.24253 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8334c70-f776-4652-944d-aed35a8e43f7"}]: dispatch 2020-09-08T15:31:24.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:24 smithi104 bash[9203]: audit 2020-09-08T15:31:24.193049+0000 mon.b (mon.1) 7 : audit [INF] from='client.? v1:172.21.15.55:0/718733343' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8334c70-f776-4652-944d-aed35a8e43f7"}]: dispatch 2020-09-08T15:31:24.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:24 smithi104 bash[9203]: audit 2020-09-08T15:31:24.199273+0000 mon.a (mon.0) 520 : audit [INF] from='client.24253 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8334c70-f776-4652-944d-aed35a8e43f7"}]': finished 2020-09-08T15:31:24.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:24 smithi104 bash[9203]: cluster 2020-09-08T15:31:24.199356+0000 mon.a (mon.0) 521 : cluster [DBG] osdmap e31: 5 total, 4 up, 4 in 2020-09-08T15:31:24.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:24 smithi104 bash[9203]: audit 2020-09-08T15:31:24.199567+0000 mon.a (mon.0) 522 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:25.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:25 smithi166 bash[9446]: audit 2020-09-08T15:31:24.823248+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? v1:172.21.15.55:0/3531219068' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:31:25.914 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:25 smithi055 bash[9296]: audit 2020-09-08T15:31:24.823248+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? v1:172.21.15.55:0/3531219068' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:31:25.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:25 smithi104 bash[9203]: audit 2020-09-08T15:31:24.823248+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? v1:172.21.15.55:0/3531219068' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:31:26.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:26 smithi166 bash[9446]: cluster 2020-09-08T15:31:25.477382+0000 mgr.x (mgr.24118) 74 : cluster [DBG] pgmap v70: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:26.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:26 smithi055 bash[9296]: cluster 2020-09-08T15:31:25.477382+0000 mgr.x (mgr.24118) 74 : cluster [DBG] pgmap v70: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:26.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:26 smithi104 bash[9203]: cluster 2020-09-08T15:31:25.477382+0000 mgr.x (mgr.24118) 74 : cluster [DBG] pgmap v70: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:27.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:27 smithi166 bash[9446]: cluster 2020-09-08T15:31:27.478060+0000 mgr.x (mgr.24118) 75 : cluster [DBG] pgmap v71: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:27.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:27 smithi055 bash[9296]: cluster 2020-09-08T15:31:27.478060+0000 mgr.x (mgr.24118) 75 : cluster [DBG] pgmap v71: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:27.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:27 smithi104 bash[9203]: cluster 2020-09-08T15:31:27.478060+0000 mgr.x (mgr.24118) 75 : cluster [DBG] pgmap v71: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:28.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:28 smithi166 bash[9446]: audit 2020-09-08T15:31:27.820839+0000 mon.a (mon.0) 523 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:28.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:28 smithi055 bash[9296]: audit 2020-09-08T15:31:27.820839+0000 mon.a (mon.0) 523 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:28.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:28 smithi104 bash[9203]: audit 2020-09-08T15:31:27.820839+0000 mon.a (mon.0) 523 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:30.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:30 smithi166 bash[9446]: cluster 2020-09-08T15:31:29.321758+0000 osd.1 (osd.1) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:31:30.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:30 smithi166 bash[9446]: cluster 2020-09-08T15:31:29.321903+0000 osd.1 (osd.1) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:31:30.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:30 smithi166 bash[9446]: cluster 2020-09-08T15:31:29.478617+0000 mgr.x (mgr.24118) 76 : cluster [DBG] pgmap v72: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:30.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:30 smithi166 bash[9446]: cluster 2020-09-08T15:31:29.570678+0000 mon.a (mon.0) 524 : cluster [DBG] osdmap e32: 5 total, 4 up, 4 in 2020-09-08T15:31:30.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:30 smithi166 bash[9446]: audit 2020-09-08T15:31:29.576014+0000 mon.a (mon.0) 525 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:30.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:30 smithi055 bash[9296]: cluster 2020-09-08T15:31:29.321758+0000 osd.1 (osd.1) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:31:30.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:30 smithi055 bash[9296]: cluster 2020-09-08T15:31:29.321903+0000 osd.1 (osd.1) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:31:30.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:30 smithi055 bash[9296]: cluster 2020-09-08T15:31:29.478617+0000 mgr.x (mgr.24118) 76 : cluster [DBG] pgmap v72: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:30.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:30 smithi055 bash[9296]: cluster 2020-09-08T15:31:29.570678+0000 mon.a (mon.0) 524 : cluster [DBG] osdmap e32: 5 total, 4 up, 4 in 2020-09-08T15:31:30.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:30 smithi055 bash[9296]: audit 2020-09-08T15:31:29.576014+0000 mon.a (mon.0) 525 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:30.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:30 smithi104 bash[9203]: cluster 2020-09-08T15:31:29.321758+0000 osd.1 (osd.1) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:31:30.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:30 smithi104 bash[9203]: cluster 2020-09-08T15:31:29.321903+0000 osd.1 (osd.1) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:31:30.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:30 smithi104 bash[9203]: cluster 2020-09-08T15:31:29.478617+0000 mgr.x (mgr.24118) 76 : cluster [DBG] pgmap v72: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:30.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:30 smithi104 bash[9203]: cluster 2020-09-08T15:31:29.570678+0000 mon.a (mon.0) 524 : cluster [DBG] osdmap e32: 5 total, 4 up, 4 in 2020-09-08T15:31:30.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:30 smithi104 bash[9203]: audit 2020-09-08T15:31:29.576014+0000 mon.a (mon.0) 525 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:32.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:32 smithi166 bash[9446]: cluster 2020-09-08T15:31:31.479213+0000 mgr.x (mgr.24118) 77 : cluster [DBG] pgmap v74: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:32.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:32 smithi166 bash[9446]: audit 2020-09-08T15:31:31.713655+0000 mon.a (mon.0) 526 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2020-09-08T15:31:32.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:32 smithi166 bash[9446]: audit 2020-09-08T15:31:31.715683+0000 mon.a (mon.0) 527 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:31:32.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:32 smithi166 bash[9446]: audit 2020-09-08T15:31:31.718010+0000 mon.a (mon.0) 528 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.4", "key": "container_image"}]: dispatch 2020-09-08T15:31:32.876 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:32 smithi055 bash[9296]: cluster 2020-09-08T15:31:31.479213+0000 mgr.x (mgr.24118) 77 : cluster [DBG] pgmap v74: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:32.876 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:32 smithi055 bash[9296]: audit 2020-09-08T15:31:31.713655+0000 mon.a (mon.0) 526 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2020-09-08T15:31:32.876 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:32 smithi055 bash[9296]: audit 2020-09-08T15:31:31.715683+0000 mon.a (mon.0) 527 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:31:32.876 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:32 smithi055 bash[9296]: audit 2020-09-08T15:31:31.718010+0000 mon.a (mon.0) 528 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.4", "key": "container_image"}]: dispatch 2020-09-08T15:31:32.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:32 smithi104 bash[9203]: cluster 2020-09-08T15:31:31.479213+0000 mgr.x (mgr.24118) 77 : cluster [DBG] pgmap v74: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:32.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:32 smithi104 bash[9203]: audit 2020-09-08T15:31:31.713655+0000 mon.a (mon.0) 526 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2020-09-08T15:31:32.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:32 smithi104 bash[9203]: audit 2020-09-08T15:31:31.715683+0000 mon.a (mon.0) 527 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:31:32.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:32 smithi104 bash[9203]: audit 2020-09-08T15:31:31.718010+0000 mon.a (mon.0) 528 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.4", "key": "container_image"}]: dispatch 2020-09-08T15:31:33.839 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:33 smithi055 bash[9296]: cephadm 2020-09-08T15:31:31.719540+0000 mgr.x (mgr.24118) 78 : cephadm [INF] Deploying daemon osd.4 on smithi055 2020-09-08T15:31:33.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:33 smithi166 bash[9446]: cephadm 2020-09-08T15:31:31.719540+0000 mgr.x (mgr.24118) 78 : cephadm [INF] Deploying daemon osd.4 on smithi055 2020-09-08T15:31:33.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:33 smithi104 bash[9203]: cephadm 2020-09-08T15:31:31.719540+0000 mgr.x (mgr.24118) 78 : cephadm [INF] Deploying daemon osd.4 on smithi055 2020-09-08T15:31:34.045 INFO:teuthology.orchestra.run.smithi055.stdout:Created osd(s) 4 on host 'smithi055' 2020-09-08T15:31:34.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:34 smithi166 bash[9446]: cluster 2020-09-08T15:31:33.479879+0000 mgr.x (mgr.24118) 79 : cluster [DBG] pgmap v75: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:34.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:34 smithi166 bash[9446]: audit 2020-09-08T15:31:34.042339+0000 mon.a (mon.0) 529 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.256049\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:29:53.256219\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:31:34.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:34 smithi166 bash[9446]: audit 2020-09-08T15:31:34.043254+0000 mon.a (mon.0) 530 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:31:34.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:34 smithi166 bash[9446]: audit 2020-09-08T15:31:34.047956+0000 mon.a (mon.0) 531 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.256049\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:29:53.256219\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:31:34.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:34 smithi104 bash[9203]: cluster 2020-09-08T15:31:33.479879+0000 mgr.x (mgr.24118) 79 : cluster [DBG] pgmap v75: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:34.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:34 smithi104 bash[9203]: audit 2020-09-08T15:31:34.042339+0000 mon.a (mon.0) 529 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.256049\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:29:53.256219\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:31:34.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:34 smithi104 bash[9203]: audit 2020-09-08T15:31:34.043254+0000 mon.a (mon.0) 530 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:31:34.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:34 smithi104 bash[9203]: audit 2020-09-08T15:31:34.047956+0000 mon.a (mon.0) 531 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.256049\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:29:53.256219\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:31:34.981 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:34 smithi055 bash[9296]: cluster 2020-09-08T15:31:33.479879+0000 mgr.x (mgr.24118) 79 : cluster [DBG] pgmap v75: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:34.981 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:34 smithi055 bash[9296]: audit 2020-09-08T15:31:34.042339+0000 mon.a (mon.0) 529 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.256049\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:29:53.256219\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:31:34.982 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:34 smithi055 bash[9296]: audit 2020-09-08T15:31:34.043254+0000 mon.a (mon.0) 530 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:31:34.982 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:34 smithi055 bash[9296]: audit 2020-09-08T15:31:34.047956+0000 mon.a (mon.0) 531 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.256049\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:29:53.256219\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}}, \"last_device_update\": \"2020-09-08T15:27:28.138620\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:31:35.576 INFO:teuthology.orchestra.run.smithi055:osd.4> sudo journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.4.service 2020-09-08T15:31:35.581 INFO:tasks.cephadm:Deploying osd.5 on smithi055 with /dev/vg_nvme/lv_3... 2020-09-08T15:31:35.582 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2020-09-08T15:31:35.760 INFO:journalctl@ceph.osd.4.smithi055.stdout:-- Logs begin at Tue 2020-09-01 16:49:30 UTC. -- 2020-09-08T15:31:35.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:35 smithi166 bash[9446]: audit 2020-09-08T15:31:34.801418+0000 mon.a (mon.0) 532 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802396\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802501\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802552\", \"created\": \"2020-09-08T15:31:34.022146\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}}, \"last_daemon_update\": \"2020-09-08T15:31:34.802761\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:31:35.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:35 smithi166 bash[9446]: audit 2020-09-08T15:31:34.806232+0000 mon.a (mon.0) 533 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802396\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802501\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802552\", \"created\": \"2020-09-08T15:31:34.022146\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}}, \"last_daemon_update\": \"2020-09-08T15:31:34.802761\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:31:35.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:35 smithi166 bash[9446]: audit 2020-09-08T15:31:34.807372+0000 mon.a (mon.0) 534 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:35.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:35 smithi055 bash[9296]: audit 2020-09-08T15:31:34.801418+0000 mon.a (mon.0) 532 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802396\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802501\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802552\", \"created\": \"2020-09-08T15:31:34.022146\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}}, \"last_daemon_update\": \"2020-09-08T15:31:34.802761\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:31:35.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:35 smithi055 bash[9296]: audit 2020-09-08T15:31:34.806232+0000 mon.a (mon.0) 533 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802396\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802501\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802552\", \"created\": \"2020-09-08T15:31:34.022146\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}}, \"last_daemon_update\": \"2020-09-08T15:31:34.802761\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:31:35.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:35 smithi055 bash[9296]: audit 2020-09-08T15:31:34.807372+0000 mon.a (mon.0) 534 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:35.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:35 smithi104 bash[9203]: audit 2020-09-08T15:31:34.801418+0000 mon.a (mon.0) 532 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802396\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802501\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802552\", \"created\": \"2020-09-08T15:31:34.022146\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}}, \"last_daemon_update\": \"2020-09-08T15:31:34.802761\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:31:35.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:35 smithi104 bash[9203]: audit 2020-09-08T15:31:34.806232+0000 mon.a (mon.0) 533 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802396\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802501\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802552\", \"created\": \"2020-09-08T15:31:34.022146\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}}, \"last_daemon_update\": \"2020-09-08T15:31:34.802761\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:31:35.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:35 smithi104 bash[9203]: audit 2020-09-08T15:31:34.807372+0000 mon.a (mon.0) 534 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:36.653 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:36 smithi055 bash[9296]: cluster 2020-09-08T15:31:35.480386+0000 mgr.x (mgr.24118) 80 : cluster [DBG] pgmap v76: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:36.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:36 smithi166 bash[9446]: cluster 2020-09-08T15:31:35.480386+0000 mgr.x (mgr.24118) 80 : cluster [DBG] pgmap v76: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:36.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:36 smithi104 bash[9203]: cluster 2020-09-08T15:31:35.480386+0000 mgr.x (mgr.24118) 80 : cluster [DBG] pgmap v76: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:37.125 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:31:36 smithi055 bash[11956]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2020-09-08T15:31:37.126 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:31:36 smithi055 bash[11956]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-4 --no-mon-config 2020-09-08T15:31:37.126 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:31:36 smithi055 bash[11956]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-4/block 2020-09-08T15:31:37.126 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:31:36 smithi055 bash[11956]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-4/block 2020-09-08T15:31:37.126 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:31:36 smithi055 bash[11956]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2020-09-08T15:31:37.127 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:31:36 smithi055 bash[11956]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2020-09-08T15:31:37.127 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:31:36 smithi055 bash[11956]: --> ceph-volume lvm activate successful for osd ID: 4 2020-09-08T15:31:37.830 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:37 smithi055 bash[9296]: cluster 2020-09-08T15:31:37.480891+0000 mgr.x (mgr.24118) 81 : cluster [DBG] pgmap v77: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:37.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:37 smithi166 bash[9446]: cluster 2020-09-08T15:31:37.480891+0000 mgr.x (mgr.24118) 81 : cluster [DBG] pgmap v77: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:37.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:37 smithi104 bash[9203]: cluster 2020-09-08T15:31:37.480891+0000 mgr.x (mgr.24118) 81 : cluster [DBG] pgmap v77: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:38.153 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:31:37 smithi055 bash[11956]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:31:40.162 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:31:40 smithi055 bash[11956]: debug 2020-09-08T15:31:40.002+0000 7f4e1c2fc700 10 -- 172.21.15.55:0/22142281 dispatch_throttle_release 194 to dispatch throttler 194/104857600 2020-09-08T15:31:40.163 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:31:40 smithi055 bash[11956]: debug 2020-09-08T15:31:40.002+0000 7f4e1c2fc700 20 -- 172.21.15.55:0/22142281 done calling dispatch on 0x55a7629c64e0 2020-09-08T15:31:40.664 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:40 smithi055 bash[9296]: cluster 2020-09-08T15:31:39.481289+0000 mgr.x (mgr.24118) 82 : cluster [DBG] pgmap v78: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:40.664 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:31:40 smithi055 bash[11956]: debug 2020-09-08T15:31:40.546+0000 7f4e23d7cf40 -1 Falling back to public interface 2020-09-08T15:31:40.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:40 smithi166 bash[9446]: cluster 2020-09-08T15:31:39.481289+0000 mgr.x (mgr.24118) 82 : cluster [DBG] pgmap v78: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:40.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:40 smithi104 bash[9203]: cluster 2020-09-08T15:31:39.481289+0000 mgr.x (mgr.24118) 82 : cluster [DBG] pgmap v78: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:41.222 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:31:40 smithi055 bash[11956]: debug 2020-09-08T15:31:40.954+0000 7f4e23d7cf40 -1 osd.4 0 log_to_monitors {default=true} 2020-09-08T15:31:41.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:41 smithi166 bash[9446]: audit 2020-09-08T15:31:40.959226+0000 mon.a (mon.0) 535 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2020-09-08T15:31:41.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:41 smithi166 bash[9446]: audit 2020-09-08T15:31:40.960963+0000 mon.b (mon.1) 9 : audit [INF] from='osd.4 v1:172.21.15.55:6803/1173103260' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2020-09-08T15:31:41.872 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping: /dev/vg_nvme/lv_3 2020-09-08T15:31:41.872 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2020-09-08T15:31:41.872 INFO:teuthology.orchestra.run.smithi055.stderr: stderr: 10+0 records in 2020-09-08T15:31:41.872 INFO:teuthology.orchestra.run.smithi055.stderr:10+0 records out 2020-09-08T15:31:41.873 INFO:teuthology.orchestra.run.smithi055.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0170605 s, 615 MB/s 2020-09-08T15:31:41.873 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping successful for: 2020-09-08T15:31:41.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:41 smithi055 bash[9296]: audit 2020-09-08T15:31:40.959226+0000 mon.a (mon.0) 535 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2020-09-08T15:31:41.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:41 smithi055 bash[9296]: audit 2020-09-08T15:31:40.960963+0000 mon.b (mon.1) 9 : audit [INF] from='osd.4 v1:172.21.15.55:6803/1173103260' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2020-09-08T15:31:41.939 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:41 smithi104 bash[9203]: audit 2020-09-08T15:31:40.959226+0000 mon.a (mon.0) 535 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2020-09-08T15:31:41.939 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:41 smithi104 bash[9203]: audit 2020-09-08T15:31:40.960963+0000 mon.b (mon.1) 9 : audit [INF] from='osd.4 v1:172.21.15.55:6803/1173103260' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2020-09-08T15:31:42.836 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:31:42 smithi055 bash[11956]: debug 2020-09-08T15:31:42.502+0000 7f4e1a2f8700 -1 osd.4 0 waiting for initial osdmap 2020-09-08T15:31:42.837 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:42 smithi055 bash[9296]: cluster 2020-09-08T15:31:41.481746+0000 mgr.x (mgr.24118) 83 : cluster [DBG] pgmap v79: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:42.837 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:42 smithi055 bash[9296]: audit 2020-09-08T15:31:41.489419+0000 mon.a (mon.0) 536 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2020-09-08T15:31:42.837 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:42 smithi055 bash[9296]: cluster 2020-09-08T15:31:41.489482+0000 mon.a (mon.0) 537 : cluster [DBG] osdmap e33: 5 total, 4 up, 4 in 2020-09-08T15:31:42.838 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:42 smithi055 bash[9296]: audit 2020-09-08T15:31:41.489707+0000 mon.a (mon.0) 538 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:42.838 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:42 smithi055 bash[9296]: audit 2020-09-08T15:31:41.492256+0000 mon.a (mon.0) 539 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:31:42.838 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:42 smithi055 bash[9296]: audit 2020-09-08T15:31:41.494044+0000 mon.b (mon.1) 10 : audit [INF] from='osd.4 v1:172.21.15.55:6803/1173103260' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:31:42.839 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph orch daemon add osd smithi055:vg_nvme/lv_3 2020-09-08T15:31:42.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:42 smithi166 bash[9446]: cluster 2020-09-08T15:31:41.481746+0000 mgr.x (mgr.24118) 83 : cluster [DBG] pgmap v79: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:42.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:42 smithi166 bash[9446]: audit 2020-09-08T15:31:41.489419+0000 mon.a (mon.0) 536 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2020-09-08T15:31:42.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:42 smithi166 bash[9446]: cluster 2020-09-08T15:31:41.489482+0000 mon.a (mon.0) 537 : cluster [DBG] osdmap e33: 5 total, 4 up, 4 in 2020-09-08T15:31:42.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:42 smithi166 bash[9446]: audit 2020-09-08T15:31:41.489707+0000 mon.a (mon.0) 538 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:42.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:42 smithi166 bash[9446]: audit 2020-09-08T15:31:41.492256+0000 mon.a (mon.0) 539 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:31:42.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:42 smithi166 bash[9446]: audit 2020-09-08T15:31:41.494044+0000 mon.b (mon.1) 10 : audit [INF] from='osd.4 v1:172.21.15.55:6803/1173103260' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:31:42.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:42 smithi104 bash[9203]: cluster 2020-09-08T15:31:41.481746+0000 mgr.x (mgr.24118) 83 : cluster [DBG] pgmap v79: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:42.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:42 smithi104 bash[9203]: audit 2020-09-08T15:31:41.489419+0000 mon.a (mon.0) 536 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2020-09-08T15:31:42.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:42 smithi104 bash[9203]: cluster 2020-09-08T15:31:41.489482+0000 mon.a (mon.0) 537 : cluster [DBG] osdmap e33: 5 total, 4 up, 4 in 2020-09-08T15:31:42.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:42 smithi104 bash[9203]: audit 2020-09-08T15:31:41.489707+0000 mon.a (mon.0) 538 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:42.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:42 smithi104 bash[9203]: audit 2020-09-08T15:31:41.492256+0000 mon.a (mon.0) 539 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:31:42.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:42 smithi104 bash[9203]: audit 2020-09-08T15:31:41.494044+0000 mon.b (mon.1) 10 : audit [INF] from='osd.4 v1:172.21.15.55:6803/1173103260' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:31:43.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:43 smithi166 bash[9446]: audit 2020-09-08T15:31:42.491582+0000 mon.a (mon.0) 540 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2020-09-08T15:31:43.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:43 smithi166 bash[9446]: cluster 2020-09-08T15:31:42.491700+0000 mon.a (mon.0) 541 : cluster [DBG] osdmap e34: 5 total, 4 up, 4 in 2020-09-08T15:31:43.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:43 smithi166 bash[9446]: audit 2020-09-08T15:31:42.492682+0000 mon.a (mon.0) 542 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:43.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:43 smithi166 bash[9446]: audit 2020-09-08T15:31:42.501023+0000 mon.a (mon.0) 543 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:43.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:43 smithi166 bash[9446]: audit 2020-09-08T15:31:42.968289+0000 mon.a (mon.0) 544 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:31:43.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:43 smithi166 bash[9446]: audit 2020-09-08T15:31:43.253302+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802396\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802501\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802552\", \"created\": \"2020-09-08T15:31:34.022146\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}}, \"last_daemon_update\": \"2020-09-08T15:31:34.802761\", \"last_device_update\": \"2020-09-08T15:31:43.253776\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:31:43.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:43 smithi166 bash[9446]: audit 2020-09-08T15:31:43.259196+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802396\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802501\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802552\", \"created\": \"2020-09-08T15:31:34.022146\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}}, \"last_daemon_update\": \"2020-09-08T15:31:34.802761\", \"last_device_update\": \"2020-09-08T15:31:43.253776\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:31:43.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:43 smithi166 bash[9446]: audit 2020-09-08T15:31:43.261482+0000 mon.a (mon.0) 547 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:31:43.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:43 smithi166 bash[9446]: audit 2020-09-08T15:31:43.266421+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:31:43.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:43 smithi166 bash[9446]: audit 2020-09-08T15:31:43.270797+0000 mon.a (mon.0) 549 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:31:43.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:43 smithi166 bash[9446]: audit 2020-09-08T15:31:43.275633+0000 mon.a (mon.0) 550 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:31:43.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:43 smithi055 bash[9296]: audit 2020-09-08T15:31:42.491582+0000 mon.a (mon.0) 540 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2020-09-08T15:31:43.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:43 smithi055 bash[9296]: cluster 2020-09-08T15:31:42.491700+0000 mon.a (mon.0) 541 : cluster [DBG] osdmap e34: 5 total, 4 up, 4 in 2020-09-08T15:31:43.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:43 smithi055 bash[9296]: audit 2020-09-08T15:31:42.492682+0000 mon.a (mon.0) 542 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:43.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:43 smithi055 bash[9296]: audit 2020-09-08T15:31:42.501023+0000 mon.a (mon.0) 543 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:43.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:43 smithi055 bash[9296]: audit 2020-09-08T15:31:42.968289+0000 mon.a (mon.0) 544 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:31:43.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:43 smithi055 bash[9296]: audit 2020-09-08T15:31:43.253302+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802396\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802501\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802552\", \"created\": \"2020-09-08T15:31:34.022146\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}}, \"last_daemon_update\": \"2020-09-08T15:31:34.802761\", \"last_device_update\": \"2020-09-08T15:31:43.253776\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:31:43.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:43 smithi055 bash[9296]: audit 2020-09-08T15:31:43.259196+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802396\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802501\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802552\", \"created\": \"2020-09-08T15:31:34.022146\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}}, \"last_daemon_update\": \"2020-09-08T15:31:34.802761\", \"last_device_update\": \"2020-09-08T15:31:43.253776\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:31:43.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:43 smithi055 bash[9296]: audit 2020-09-08T15:31:43.261482+0000 mon.a (mon.0) 547 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:31:43.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:43 smithi055 bash[9296]: audit 2020-09-08T15:31:43.266421+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:31:43.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:43 smithi055 bash[9296]: audit 2020-09-08T15:31:43.270797+0000 mon.a (mon.0) 549 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:31:43.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:43 smithi055 bash[9296]: audit 2020-09-08T15:31:43.275633+0000 mon.a (mon.0) 550 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:31:43.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:43 smithi104 bash[9203]: audit 2020-09-08T15:31:42.491582+0000 mon.a (mon.0) 540 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2020-09-08T15:31:43.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:43 smithi104 bash[9203]: cluster 2020-09-08T15:31:42.491700+0000 mon.a (mon.0) 541 : cluster [DBG] osdmap e34: 5 total, 4 up, 4 in 2020-09-08T15:31:43.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:43 smithi104 bash[9203]: audit 2020-09-08T15:31:42.492682+0000 mon.a (mon.0) 542 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:43.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:43 smithi104 bash[9203]: audit 2020-09-08T15:31:42.501023+0000 mon.a (mon.0) 543 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:43.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:43 smithi104 bash[9203]: audit 2020-09-08T15:31:42.968289+0000 mon.a (mon.0) 544 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:31:43.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:43 smithi104 bash[9203]: audit 2020-09-08T15:31:43.253302+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802396\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802501\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802552\", \"created\": \"2020-09-08T15:31:34.022146\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}}, \"last_daemon_update\": \"2020-09-08T15:31:34.802761\", \"last_device_update\": \"2020-09-08T15:31:43.253776\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:31:43.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:43 smithi104 bash[9203]: audit 2020-09-08T15:31:43.259196+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802396\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802501\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802552\", \"created\": \"2020-09-08T15:31:34.022146\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}}, \"last_daemon_update\": \"2020-09-08T15:31:34.802761\", \"last_device_update\": \"2020-09-08T15:31:43.253776\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:31:43.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:43 smithi104 bash[9203]: audit 2020-09-08T15:31:43.261482+0000 mon.a (mon.0) 547 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:31:43.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:43 smithi104 bash[9203]: audit 2020-09-08T15:31:43.266421+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:31:43.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:43 smithi104 bash[9203]: audit 2020-09-08T15:31:43.270797+0000 mon.a (mon.0) 549 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:31:43.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:43 smithi104 bash[9203]: audit 2020-09-08T15:31:43.275633+0000 mon.a (mon.0) 550 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:31:44.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:44 smithi166 bash[9446]: cluster 2020-09-08T15:31:41.997692+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:31:44.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:44 smithi166 bash[9446]: cluster 2020-09-08T15:31:41.997868+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:31:44.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:44 smithi166 bash[9446]: cluster 2020-09-08T15:31:43.482290+0000 mgr.x (mgr.24118) 84 : cluster [DBG] pgmap v82: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:44.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:44 smithi166 bash[9446]: audit 2020-09-08T15:31:43.498778+0000 mon.a (mon.0) 551 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:44.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:44 smithi166 bash[9446]: cluster 2020-09-08T15:31:43.514790+0000 mon.a (mon.0) 552 : cluster [INF] osd.4 v1:172.21.15.55:6803/1173103260 boot 2020-09-08T15:31:44.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:44 smithi166 bash[9446]: cluster 2020-09-08T15:31:43.514865+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e35: 5 total, 5 up, 5 in 2020-09-08T15:31:44.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:44 smithi166 bash[9446]: audit 2020-09-08T15:31:43.515017+0000 mon.a (mon.0) 554 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:44.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:44 smithi055 bash[9296]: cluster 2020-09-08T15:31:41.997692+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:31:44.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:44 smithi055 bash[9296]: cluster 2020-09-08T15:31:41.997868+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:31:44.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:44 smithi055 bash[9296]: cluster 2020-09-08T15:31:43.482290+0000 mgr.x (mgr.24118) 84 : cluster [DBG] pgmap v82: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:44.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:44 smithi055 bash[9296]: audit 2020-09-08T15:31:43.498778+0000 mon.a (mon.0) 551 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:44.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:44 smithi055 bash[9296]: cluster 2020-09-08T15:31:43.514790+0000 mon.a (mon.0) 552 : cluster [INF] osd.4 v1:172.21.15.55:6803/1173103260 boot 2020-09-08T15:31:44.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:44 smithi055 bash[9296]: cluster 2020-09-08T15:31:43.514865+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e35: 5 total, 5 up, 5 in 2020-09-08T15:31:44.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:44 smithi055 bash[9296]: audit 2020-09-08T15:31:43.515017+0000 mon.a (mon.0) 554 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:44.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:44 smithi104 bash[9203]: cluster 2020-09-08T15:31:41.997692+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:31:44.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:44 smithi104 bash[9203]: cluster 2020-09-08T15:31:41.997868+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:31:44.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:44 smithi104 bash[9203]: cluster 2020-09-08T15:31:43.482290+0000 mgr.x (mgr.24118) 84 : cluster [DBG] pgmap v82: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2020-09-08T15:31:44.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:44 smithi104 bash[9203]: audit 2020-09-08T15:31:43.498778+0000 mon.a (mon.0) 551 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:44.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:44 smithi104 bash[9203]: cluster 2020-09-08T15:31:43.514790+0000 mon.a (mon.0) 552 : cluster [INF] osd.4 v1:172.21.15.55:6803/1173103260 boot 2020-09-08T15:31:44.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:44 smithi104 bash[9203]: cluster 2020-09-08T15:31:43.514865+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e35: 5 total, 5 up, 5 in 2020-09-08T15:31:44.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:44 smithi104 bash[9203]: audit 2020-09-08T15:31:43.515017+0000 mon.a (mon.0) 554 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2020-09-08T15:31:45.824 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:45 smithi055 bash[9296]: cluster 2020-09-08T15:31:44.516471+0000 mon.a (mon.0) 555 : cluster [DBG] osdmap e36: 5 total, 5 up, 5 in 2020-09-08T15:31:45.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:45 smithi166 bash[9446]: cluster 2020-09-08T15:31:44.516471+0000 mon.a (mon.0) 555 : cluster [DBG] osdmap e36: 5 total, 5 up, 5 in 2020-09-08T15:31:45.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:45 smithi104 bash[9203]: cluster 2020-09-08T15:31:44.516471+0000 mon.a (mon.0) 555 : cluster [DBG] osdmap e36: 5 total, 5 up, 5 in 2020-09-08T15:31:46.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:46 smithi166 bash[9446]: cluster 2020-09-08T15:31:45.482742+0000 mgr.x (mgr.24118) 85 : cluster [DBG] pgmap v85: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:46.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:46 smithi166 bash[9446]: audit 2020-09-08T15:31:46.197396+0000 mon.a (mon.0) 556 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:31:46.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:46 smithi166 bash[9446]: audit 2020-09-08T15:31:46.200952+0000 mon.a (mon.0) 557 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:31:46.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:46 smithi166 bash[9446]: audit 2020-09-08T15:31:46.202565+0000 mon.a (mon.0) 558 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:31:46.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:46 smithi166 bash[9446]: audit 2020-09-08T15:31:46.204442+0000 mon.a (mon.0) 559 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:46.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:46 smithi055 bash[9296]: cluster 2020-09-08T15:31:45.482742+0000 mgr.x (mgr.24118) 85 : cluster [DBG] pgmap v85: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:46.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:46 smithi055 bash[9296]: audit 2020-09-08T15:31:46.197396+0000 mon.a (mon.0) 556 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:31:46.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:46 smithi055 bash[9296]: audit 2020-09-08T15:31:46.200952+0000 mon.a (mon.0) 557 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:31:46.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:46 smithi055 bash[9296]: audit 2020-09-08T15:31:46.202565+0000 mon.a (mon.0) 558 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:31:46.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:46 smithi055 bash[9296]: audit 2020-09-08T15:31:46.204442+0000 mon.a (mon.0) 559 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:46.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:46 smithi104 bash[9203]: cluster 2020-09-08T15:31:45.482742+0000 mgr.x (mgr.24118) 85 : cluster [DBG] pgmap v85: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:46.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:46 smithi104 bash[9203]: audit 2020-09-08T15:31:46.197396+0000 mon.a (mon.0) 556 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:31:46.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:46 smithi104 bash[9203]: audit 2020-09-08T15:31:46.200952+0000 mon.a (mon.0) 557 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:31:46.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:46 smithi104 bash[9203]: audit 2020-09-08T15:31:46.202565+0000 mon.a (mon.0) 558 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:31:46.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:46 smithi104 bash[9203]: audit 2020-09-08T15:31:46.204442+0000 mon.a (mon.0) 559 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:47.815 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:47 smithi055 bash[9296]: audit 2020-09-08T15:31:46.198134+0000 mgr.x (mgr.24118) 86 : audit [DBG] from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:31:47.816 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:47 smithi055 bash[9296]: cephadm 2020-09-08T15:31:46.201461+0000 mgr.x (mgr.24118) 87 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:31:47.816 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:47 smithi055 bash[9296]: cephadm 2020-09-08T15:31:46.201636+0000 mgr.x (mgr.24118) 88 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:31:47.816 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:47 smithi055 bash[9296]: cephadm 2020-09-08T15:31:46.202584+0000 mgr.x (mgr.24118) 89 : cephadm [INF] Applying None on host smithi055... 2020-09-08T15:31:47.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:47 smithi166 bash[9446]: audit 2020-09-08T15:31:46.198134+0000 mgr.x (mgr.24118) 86 : audit [DBG] from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:31:47.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:47 smithi166 bash[9446]: cephadm 2020-09-08T15:31:46.201461+0000 mgr.x (mgr.24118) 87 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:31:47.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:47 smithi166 bash[9446]: cephadm 2020-09-08T15:31:46.201636+0000 mgr.x (mgr.24118) 88 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:31:47.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:47 smithi166 bash[9446]: cephadm 2020-09-08T15:31:46.202584+0000 mgr.x (mgr.24118) 89 : cephadm [INF] Applying None on host smithi055... 2020-09-08T15:31:47.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:47 smithi104 bash[9203]: audit 2020-09-08T15:31:46.198134+0000 mgr.x (mgr.24118) 86 : audit [DBG] from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:31:47.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:47 smithi104 bash[9203]: cephadm 2020-09-08T15:31:46.201461+0000 mgr.x (mgr.24118) 87 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:31:47.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:47 smithi104 bash[9203]: cephadm 2020-09-08T15:31:46.201636+0000 mgr.x (mgr.24118) 88 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:31:47.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:47 smithi104 bash[9203]: cephadm 2020-09-08T15:31:46.202584+0000 mgr.x (mgr.24118) 89 : cephadm [INF] Applying None on host smithi055... 2020-09-08T15:31:48.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:48 smithi166 bash[9446]: cluster 2020-09-08T15:31:47.483306+0000 mgr.x (mgr.24118) 90 : cluster [DBG] pgmap v86: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:48.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:48 smithi055 bash[9296]: cluster 2020-09-08T15:31:47.483306+0000 mgr.x (mgr.24118) 90 : cluster [DBG] pgmap v86: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:48.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:48 smithi104 bash[9203]: cluster 2020-09-08T15:31:47.483306+0000 mgr.x (mgr.24118) 90 : cluster [DBG] pgmap v86: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:49.653 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:49 smithi055 bash[9296]: cluster 2020-09-08T15:31:48.386047+0000 osd.1 (osd.1) 7 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:31:49.653 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:49 smithi055 bash[9296]: cluster 2020-09-08T15:31:48.387850+0000 osd.1 (osd.1) 8 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:31:49.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:49 smithi166 bash[9446]: cluster 2020-09-08T15:31:48.386047+0000 osd.1 (osd.1) 7 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:31:49.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:49 smithi166 bash[9446]: cluster 2020-09-08T15:31:48.387850+0000 osd.1 (osd.1) 8 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:31:49.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:49 smithi104 bash[9203]: cluster 2020-09-08T15:31:48.386047+0000 osd.1 (osd.1) 7 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:31:49.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:49 smithi104 bash[9203]: cluster 2020-09-08T15:31:48.387850+0000 osd.1 (osd.1) 8 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:31:50.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:50 smithi166 bash[9446]: cluster 2020-09-08T15:31:49.483802+0000 mgr.x (mgr.24118) 91 : cluster [DBG] pgmap v87: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:50.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:50 smithi166 bash[9446]: audit 2020-09-08T15:31:50.232545+0000 mon.a (mon.0) 560 : audit [INF] from='client.24280 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6789abd2-9e9b-4c04-81fb-0ce2d2849e39"}]: dispatch 2020-09-08T15:31:50.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:50 smithi166 bash[9446]: audit 2020-09-08T15:31:50.234343+0000 mon.b (mon.1) 11 : audit [INF] from='client.? v1:172.21.15.55:0/2393852262' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6789abd2-9e9b-4c04-81fb-0ce2d2849e39"}]: dispatch 2020-09-08T15:31:50.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:50 smithi166 bash[9446]: audit 2020-09-08T15:31:50.240586+0000 mon.a (mon.0) 561 : audit [INF] from='client.24280 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6789abd2-9e9b-4c04-81fb-0ce2d2849e39"}]': finished 2020-09-08T15:31:50.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:50 smithi166 bash[9446]: cluster 2020-09-08T15:31:50.240654+0000 mon.a (mon.0) 562 : cluster [DBG] osdmap e37: 6 total, 5 up, 5 in 2020-09-08T15:31:50.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:50 smithi166 bash[9446]: audit 2020-09-08T15:31:50.240863+0000 mon.a (mon.0) 563 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:31:50.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:50 smithi055 bash[9296]: cluster 2020-09-08T15:31:49.483802+0000 mgr.x (mgr.24118) 91 : cluster [DBG] pgmap v87: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:50.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:50 smithi055 bash[9296]: audit 2020-09-08T15:31:50.232545+0000 mon.a (mon.0) 560 : audit [INF] from='client.24280 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6789abd2-9e9b-4c04-81fb-0ce2d2849e39"}]: dispatch 2020-09-08T15:31:50.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:50 smithi055 bash[9296]: audit 2020-09-08T15:31:50.234343+0000 mon.b (mon.1) 11 : audit [INF] from='client.? v1:172.21.15.55:0/2393852262' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6789abd2-9e9b-4c04-81fb-0ce2d2849e39"}]: dispatch 2020-09-08T15:31:50.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:50 smithi055 bash[9296]: audit 2020-09-08T15:31:50.240586+0000 mon.a (mon.0) 561 : audit [INF] from='client.24280 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6789abd2-9e9b-4c04-81fb-0ce2d2849e39"}]': finished 2020-09-08T15:31:50.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:50 smithi055 bash[9296]: cluster 2020-09-08T15:31:50.240654+0000 mon.a (mon.0) 562 : cluster [DBG] osdmap e37: 6 total, 5 up, 5 in 2020-09-08T15:31:50.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:50 smithi055 bash[9296]: audit 2020-09-08T15:31:50.240863+0000 mon.a (mon.0) 563 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:31:50.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:50 smithi104 bash[9203]: cluster 2020-09-08T15:31:49.483802+0000 mgr.x (mgr.24118) 91 : cluster [DBG] pgmap v87: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:50.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:50 smithi104 bash[9203]: audit 2020-09-08T15:31:50.232545+0000 mon.a (mon.0) 560 : audit [INF] from='client.24280 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6789abd2-9e9b-4c04-81fb-0ce2d2849e39"}]: dispatch 2020-09-08T15:31:50.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:50 smithi104 bash[9203]: audit 2020-09-08T15:31:50.234343+0000 mon.b (mon.1) 11 : audit [INF] from='client.? v1:172.21.15.55:0/2393852262' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6789abd2-9e9b-4c04-81fb-0ce2d2849e39"}]: dispatch 2020-09-08T15:31:50.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:50 smithi104 bash[9203]: audit 2020-09-08T15:31:50.240586+0000 mon.a (mon.0) 561 : audit [INF] from='client.24280 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6789abd2-9e9b-4c04-81fb-0ce2d2849e39"}]': finished 2020-09-08T15:31:50.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:50 smithi104 bash[9203]: cluster 2020-09-08T15:31:50.240654+0000 mon.a (mon.0) 562 : cluster [DBG] osdmap e37: 6 total, 5 up, 5 in 2020-09-08T15:31:50.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:50 smithi104 bash[9203]: audit 2020-09-08T15:31:50.240863+0000 mon.a (mon.0) 563 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:31:51.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:51 smithi166 bash[9446]: audit 2020-09-08T15:31:50.961237+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? v1:172.21.15.55:0/186768491' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:31:51.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:51 smithi055 bash[9296]: audit 2020-09-08T15:31:50.961237+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? v1:172.21.15.55:0/186768491' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:31:51.936 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:51 smithi104 bash[9203]: audit 2020-09-08T15:31:50.961237+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? v1:172.21.15.55:0/186768491' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:31:52.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:52 smithi166 bash[9446]: cluster 2020-09-08T15:31:51.484486+0000 mgr.x (mgr.24118) 92 : cluster [DBG] pgmap v89: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:52.902 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:52 smithi055 bash[9296]: cluster 2020-09-08T15:31:51.484486+0000 mgr.x (mgr.24118) 92 : cluster [DBG] pgmap v89: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:52.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:52 smithi104 bash[9203]: cluster 2020-09-08T15:31:51.484486+0000 mgr.x (mgr.24118) 92 : cluster [DBG] pgmap v89: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:53.654 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:53 smithi055 bash[9296]: cluster 2020-09-08T15:31:53.485172+0000 mgr.x (mgr.24118) 93 : cluster [DBG] pgmap v90: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:53.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:53 smithi166 bash[9446]: cluster 2020-09-08T15:31:53.485172+0000 mgr.x (mgr.24118) 93 : cluster [DBG] pgmap v90: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:53.924 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:53 smithi104 bash[9203]: cluster 2020-09-08T15:31:53.485172+0000 mgr.x (mgr.24118) 93 : cluster [DBG] pgmap v90: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:54.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:54 smithi166 bash[9446]: audit 2020-09-08T15:31:53.932222+0000 mon.a (mon.0) 564 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:54.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:54 smithi055 bash[9296]: audit 2020-09-08T15:31:53.932222+0000 mon.a (mon.0) 564 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:54.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:54 smithi104 bash[9203]: audit 2020-09-08T15:31:53.932222+0000 mon.a (mon.0) 564 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:31:55.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:55 smithi166 bash[9446]: cluster 2020-09-08T15:31:54.091173+0000 osd.2 (osd.2) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:31:55.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:55 smithi166 bash[9446]: cluster 2020-09-08T15:31:54.091298+0000 osd.2 (osd.2) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:31:55.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:55 smithi166 bash[9446]: cluster 2020-09-08T15:31:54.574241+0000 mon.a (mon.0) 565 : cluster [DBG] osdmap e38: 6 total, 5 up, 5 in 2020-09-08T15:31:55.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:55 smithi166 bash[9446]: audit 2020-09-08T15:31:54.574994+0000 mon.a (mon.0) 566 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:31:55.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:55 smithi166 bash[9446]: cluster 2020-09-08T15:31:55.485895+0000 mgr.x (mgr.24118) 94 : cluster [DBG] pgmap v92: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:55.902 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:55 smithi055 bash[9296]: cluster 2020-09-08T15:31:54.091173+0000 osd.2 (osd.2) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:31:55.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:55 smithi055 bash[9296]: cluster 2020-09-08T15:31:54.091298+0000 osd.2 (osd.2) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:31:55.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:55 smithi055 bash[9296]: cluster 2020-09-08T15:31:54.574241+0000 mon.a (mon.0) 565 : cluster [DBG] osdmap e38: 6 total, 5 up, 5 in 2020-09-08T15:31:55.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:55 smithi055 bash[9296]: audit 2020-09-08T15:31:54.574994+0000 mon.a (mon.0) 566 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:31:55.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:55 smithi055 bash[9296]: cluster 2020-09-08T15:31:55.485895+0000 mgr.x (mgr.24118) 94 : cluster [DBG] pgmap v92: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:55.924 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:55 smithi104 bash[9203]: cluster 2020-09-08T15:31:54.091173+0000 osd.2 (osd.2) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:31:55.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:55 smithi104 bash[9203]: cluster 2020-09-08T15:31:54.091298+0000 osd.2 (osd.2) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:31:55.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:55 smithi104 bash[9203]: cluster 2020-09-08T15:31:54.574241+0000 mon.a (mon.0) 565 : cluster [DBG] osdmap e38: 6 total, 5 up, 5 in 2020-09-08T15:31:55.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:55 smithi104 bash[9203]: audit 2020-09-08T15:31:54.574994+0000 mon.a (mon.0) 566 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:31:55.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:55 smithi104 bash[9203]: cluster 2020-09-08T15:31:55.485895+0000 mgr.x (mgr.24118) 94 : cluster [DBG] pgmap v92: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:58.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:58 smithi166 bash[9446]: cluster 2020-09-08T15:31:57.486618+0000 mgr.x (mgr.24118) 95 : cluster [DBG] pgmap v93: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:58.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:58 smithi166 bash[9446]: audit 2020-09-08T15:31:58.173613+0000 mon.a (mon.0) 567 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2020-09-08T15:31:58.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:58 smithi166 bash[9446]: audit 2020-09-08T15:31:58.175567+0000 mon.a (mon.0) 568 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:31:58.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:58 smithi166 bash[9446]: audit 2020-09-08T15:31:58.177460+0000 mon.a (mon.0) 569 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.5", "key": "container_image"}]: dispatch 2020-09-08T15:31:58.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:58 smithi055 bash[9296]: cluster 2020-09-08T15:31:57.486618+0000 mgr.x (mgr.24118) 95 : cluster [DBG] pgmap v93: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:58.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:58 smithi055 bash[9296]: audit 2020-09-08T15:31:58.173613+0000 mon.a (mon.0) 567 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2020-09-08T15:31:58.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:58 smithi055 bash[9296]: audit 2020-09-08T15:31:58.175567+0000 mon.a (mon.0) 568 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:31:58.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:58 smithi055 bash[9296]: audit 2020-09-08T15:31:58.177460+0000 mon.a (mon.0) 569 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.5", "key": "container_image"}]: dispatch 2020-09-08T15:31:58.924 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:58 smithi104 bash[9203]: cluster 2020-09-08T15:31:57.486618+0000 mgr.x (mgr.24118) 95 : cluster [DBG] pgmap v93: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:31:58.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:58 smithi104 bash[9203]: audit 2020-09-08T15:31:58.173613+0000 mon.a (mon.0) 567 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2020-09-08T15:31:58.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:58 smithi104 bash[9203]: audit 2020-09-08T15:31:58.175567+0000 mon.a (mon.0) 568 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:31:58.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:58 smithi104 bash[9203]: audit 2020-09-08T15:31:58.177460+0000 mon.a (mon.0) 569 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.5", "key": "container_image"}]: dispatch 2020-09-08T15:31:59.667 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:31:59 smithi055 bash[9296]: cephadm 2020-09-08T15:31:58.179331+0000 mgr.x (mgr.24118) 96 : cephadm [INF] Deploying daemon osd.5 on smithi055 2020-09-08T15:31:59.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:31:59 smithi166 bash[9446]: cephadm 2020-09-08T15:31:58.179331+0000 mgr.x (mgr.24118) 96 : cephadm [INF] Deploying daemon osd.5 on smithi055 2020-09-08T15:31:59.924 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:31:59 smithi104 bash[9203]: cephadm 2020-09-08T15:31:58.179331+0000 mgr.x (mgr.24118) 96 : cephadm [INF] Deploying daemon osd.5 on smithi055 2020-09-08T15:32:00.537 INFO:teuthology.orchestra.run.smithi055.stdout:Created osd(s) 5 on host 'smithi055' 2020-09-08T15:32:00.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:00 smithi166 bash[9446]: cluster 2020-09-08T15:31:59.487031+0000 mgr.x (mgr.24118) 97 : cluster [DBG] pgmap v94: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:00.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:00 smithi166 bash[9446]: audit 2020-09-08T15:32:00.534710+0000 mon.a (mon.0) 570 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802396\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:31:34.802501\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802552\", \"created\": \"2020-09-08T15:31:34.022146\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}}, \"last_device_update\": \"2020-09-08T15:31:43.253776\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:00.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:00 smithi166 bash[9446]: audit 2020-09-08T15:32:00.535568+0000 mon.a (mon.0) 571 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:32:00.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:00 smithi166 bash[9446]: audit 2020-09-08T15:32:00.540624+0000 mon.a (mon.0) 572 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802396\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:31:34.802501\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802552\", \"created\": \"2020-09-08T15:31:34.022146\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}}, \"last_device_update\": \"2020-09-08T15:31:43.253776\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:00.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:00 smithi055 bash[9296]: cluster 2020-09-08T15:31:59.487031+0000 mgr.x (mgr.24118) 97 : cluster [DBG] pgmap v94: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:00.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:00 smithi055 bash[9296]: audit 2020-09-08T15:32:00.534710+0000 mon.a (mon.0) 570 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802396\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:31:34.802501\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802552\", \"created\": \"2020-09-08T15:31:34.022146\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}}, \"last_device_update\": \"2020-09-08T15:31:43.253776\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:00.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:00 smithi055 bash[9296]: audit 2020-09-08T15:32:00.535568+0000 mon.a (mon.0) 571 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:32:00.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:00 smithi055 bash[9296]: audit 2020-09-08T15:32:00.540624+0000 mon.a (mon.0) 572 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802396\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:31:34.802501\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802552\", \"created\": \"2020-09-08T15:31:34.022146\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}}, \"last_device_update\": \"2020-09-08T15:31:43.253776\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:00.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:00 smithi104 bash[9203]: cluster 2020-09-08T15:31:59.487031+0000 mgr.x (mgr.24118) 97 : cluster [DBG] pgmap v94: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:00.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:00 smithi104 bash[9203]: audit 2020-09-08T15:32:00.534710+0000 mon.a (mon.0) 570 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802396\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:31:34.802501\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802552\", \"created\": \"2020-09-08T15:31:34.022146\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}}, \"last_device_update\": \"2020-09-08T15:31:43.253776\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:00.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:00 smithi104 bash[9203]: audit 2020-09-08T15:32:00.535568+0000 mon.a (mon.0) 571 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:32:00.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:00 smithi104 bash[9203]: audit 2020-09-08T15:32:00.540624+0000 mon.a (mon.0) 572 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802396\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:31:34.802501\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:34.802552\", \"created\": \"2020-09-08T15:31:34.022146\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}}, \"last_device_update\": \"2020-09-08T15:31:43.253776\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:01.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:01 smithi166 bash[9446]: audit 2020-09-08T15:32:01.282543+0000 mon.a (mon.0) 573 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}}, \"last_daemon_update\": \"2020-09-08T15:32:01.283458\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:01.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:01 smithi166 bash[9446]: audit 2020-09-08T15:32:01.288129+0000 mon.a (mon.0) 574 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}}, \"last_daemon_update\": \"2020-09-08T15:32:01.283458\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:01.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:01 smithi166 bash[9446]: audit 2020-09-08T15:32:01.289879+0000 mon.a (mon.0) 575 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:01.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:01 smithi055 bash[9296]: audit 2020-09-08T15:32:01.282543+0000 mon.a (mon.0) 573 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}}, \"last_daemon_update\": \"2020-09-08T15:32:01.283458\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:01.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:01 smithi055 bash[9296]: audit 2020-09-08T15:32:01.288129+0000 mon.a (mon.0) 574 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}}, \"last_daemon_update\": \"2020-09-08T15:32:01.283458\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:01.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:01 smithi055 bash[9296]: audit 2020-09-08T15:32:01.289879+0000 mon.a (mon.0) 575 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:01.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:01 smithi104 bash[9203]: audit 2020-09-08T15:32:01.282543+0000 mon.a (mon.0) 573 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}}, \"last_daemon_update\": \"2020-09-08T15:32:01.283458\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:01.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:01 smithi104 bash[9203]: audit 2020-09-08T15:32:01.288129+0000 mon.a (mon.0) 574 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}}, \"last_daemon_update\": \"2020-09-08T15:32:01.283458\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:01.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:01 smithi104 bash[9203]: audit 2020-09-08T15:32:01.289879+0000 mon.a (mon.0) 575 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:02.049 INFO:teuthology.orchestra.run.smithi055:osd.5> sudo journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.5.service 2020-09-08T15:32:02.052 INFO:tasks.cephadm:Deploying osd.6 on smithi055 with /dev/vg_nvme/lv_2... 2020-09-08T15:32:02.053 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2020-09-08T15:32:02.120 INFO:journalctl@ceph.osd.5.smithi055.stdout:-- Logs begin at Tue 2020-09-01 16:49:30 UTC. -- 2020-09-08T15:32:02.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:02 smithi166 bash[9446]: cluster 2020-09-08T15:32:01.487574+0000 mgr.x (mgr.24118) 98 : cluster [DBG] pgmap v95: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:02.924 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:02 smithi104 bash[9203]: cluster 2020-09-08T15:32:01.487574+0000 mgr.x (mgr.24118) 98 : cluster [DBG] pgmap v95: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:03.106 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:02 smithi055 bash[9296]: cluster 2020-09-08T15:32:01.487574+0000 mgr.x (mgr.24118) 98 : cluster [DBG] pgmap v95: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:03.903 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:03 smithi055 bash[14301]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2020-09-08T15:32:03.903 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:03 smithi055 bash[14301]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-5 --no-mon-config 2020-09-08T15:32:03.903 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:03 smithi055 bash[14301]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5/block 2020-09-08T15:32:03.904 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:03 smithi055 bash[14301]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-5/block 2020-09-08T15:32:03.904 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:03 smithi055 bash[14301]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2020-09-08T15:32:03.904 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:03 smithi055 bash[14301]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2020-09-08T15:32:03.904 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:03 smithi055 bash[14301]: --> ceph-volume lvm activate successful for osd ID: 5 2020-09-08T15:32:04.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:04 smithi166 bash[9446]: cluster 2020-09-08T15:32:03.488024+0000 mgr.x (mgr.24118) 99 : cluster [DBG] pgmap v96: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:04.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:04 smithi166 bash[9446]: cluster 2020-09-08T15:32:03.664655+0000 osd.0 (osd.0) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:04.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:04 smithi166 bash[9446]: cluster 2020-09-08T15:32:03.664816+0000 osd.0 (osd.0) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:04.877 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:04 smithi055 bash[9296]: cluster 2020-09-08T15:32:03.488024+0000 mgr.x (mgr.24118) 99 : cluster [DBG] pgmap v96: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:04.877 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:04 smithi055 bash[9296]: cluster 2020-09-08T15:32:03.664655+0000 osd.0 (osd.0) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:04.877 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:04 smithi055 bash[9296]: cluster 2020-09-08T15:32:03.664816+0000 osd.0 (osd.0) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:04.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:04 smithi104 bash[9203]: cluster 2020-09-08T15:32:03.488024+0000 mgr.x (mgr.24118) 99 : cluster [DBG] pgmap v96: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:04.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:04 smithi104 bash[9203]: cluster 2020-09-08T15:32:03.664655+0000 osd.0 (osd.0) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:04.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:04 smithi104 bash[9203]: cluster 2020-09-08T15:32:03.664816+0000 osd.0 (osd.0) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:05.154 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:04 smithi055 bash[14301]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:32:05.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:05 smithi166 bash[9446]: cluster 2020-09-08T15:32:04.559942+0000 mon.a (mon.0) 576 : cluster [DBG] osdmap e39: 6 total, 5 up, 5 in 2020-09-08T15:32:05.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:05 smithi166 bash[9446]: audit 2020-09-08T15:32:04.560729+0000 mon.a (mon.0) 577 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:32:05.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:05 smithi166 bash[9446]: cluster 2020-09-08T15:32:05.488593+0000 mgr.x (mgr.24118) 100 : cluster [DBG] pgmap v98: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:05.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:05 smithi055 bash[9296]: cluster 2020-09-08T15:32:04.559942+0000 mon.a (mon.0) 576 : cluster [DBG] osdmap e39: 6 total, 5 up, 5 in 2020-09-08T15:32:05.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:05 smithi055 bash[9296]: audit 2020-09-08T15:32:04.560729+0000 mon.a (mon.0) 577 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:32:05.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:05 smithi055 bash[9296]: cluster 2020-09-08T15:32:05.488593+0000 mgr.x (mgr.24118) 100 : cluster [DBG] pgmap v98: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:05.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:05 smithi104 bash[9203]: cluster 2020-09-08T15:32:04.559942+0000 mon.a (mon.0) 576 : cluster [DBG] osdmap e39: 6 total, 5 up, 5 in 2020-09-08T15:32:05.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:05 smithi104 bash[9203]: audit 2020-09-08T15:32:04.560729+0000 mon.a (mon.0) 577 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:32:05.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:05 smithi104 bash[9203]: cluster 2020-09-08T15:32:05.488593+0000 mgr.x (mgr.24118) 100 : cluster [DBG] pgmap v98: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:07.154 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d1751a700 10 -- 172.21.15.55:0/1815387893 dispatch_throttle_release 278 to dispatch throttler 278/104857600 2020-09-08T15:32:07.154 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d1751a700 20 -- 172.21.15.55:0/1815387893 done calling dispatch on 0x561b1b808c60 2020-09-08T15:32:07.155 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 -- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 legacy=0x561b1b872800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).process 2020-09-08T15:32:07.155 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 -- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 legacy=0x561b1b872800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read continue len=1 2020-09-08T15:32:07.155 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=OPENED pgs=99 cs=1 l=1).handle_message r=0 2020-09-08T15:32:07.156 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=OPENED pgs=99 cs=1 l=1).handle_message process tag 7 2020-09-08T15:32:07.156 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=OPENED pgs=99 cs=1 l=1).handle_message begin MSG 2020-09-08T15:32:07.156 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 -- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 legacy=0x561b1b872800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read start len=53 2020-09-08T15:32:07.156 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=OPENED pgs=99 cs=1 l=1).handle_message_header r=0 2020-09-08T15:32:07.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=OPENED pgs=99 cs=1 l=1).handle_message_header got MSG header 2020-09-08T15:32:07.160 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=OPENED pgs=99 cs=1 l=1).handle_message_header got envelope type=18 src mon.1 front=194 data=0 off 0 2020-09-08T15:32:07.160 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=THROTTLE_MESSAGE pgs=99 cs=1 l=1).throttle_message 2020-09-08T15:32:07.160 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=THROTTLE_BYTES pgs=99 cs=1 l=1).throttle_bytes 2020-09-08T15:32:07.163 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=THROTTLE_DISPATCH_QUEUE pgs=99 cs=1 l=1).throttle_dispatch_queue 2020-09-08T15:32:07.163 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=READ_MESSAGE_FRONT pgs=99 cs=1 l=1).read_message_front 2020-09-08T15:32:07.163 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 -- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 legacy=0x561b1b872800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read start len=194 2020-09-08T15:32:07.163 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=READ_MESSAGE_FRONT pgs=99 cs=1 l=1).handle_message_front r=0 2020-09-08T15:32:07.168 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=READ_MESSAGE_FRONT pgs=99 cs=1 l=1).handle_message_front got front 194 2020-09-08T15:32:07.168 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=READ_MESSAGE_FRONT pgs=99 cs=1 l=1).read_message_middle 2020-09-08T15:32:07.168 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=READ_MESSAGE_FRONT pgs=99 cs=1 l=1).read_message_data_prepare 2020-09-08T15:32:07.168 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=READ_MESSAGE_FRONT pgs=99 cs=1 l=1).read_message_data msg_left=0 2020-09-08T15:32:07.169 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=READ_MESSAGE_FRONT pgs=99 cs=1 l=1).read_message_footer 2020-09-08T15:32:07.169 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 -- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 legacy=0x561b1b872800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read start len=21 2020-09-08T15:32:07.169 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=READ_FOOTER_AND_DISPATCH pgs=99 cs=1 l=1).handle_message_footer r=0 2020-09-08T15:32:07.169 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 10 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=READ_FOOTER_AND_DISPATCH pgs=99 cs=1 l=1).handle_message_footer aborted = 0 2020-09-08T15:32:07.169 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=READ_FOOTER_AND_DISPATCH pgs=99 cs=1 l=1).handle_message_footer got 194 + 0 + 0 byte message 2020-09-08T15:32:07.170 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 10 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=READ_FOOTER_AND_DISPATCH pgs=99 cs=1 l=1).handle_message_footer no session security set 2020-09-08T15:32:07.170 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 5 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=READ_FOOTER_AND_DISPATCH pgs=99 cs=1 l=1). rx mon.1 seq 6 0x561b1b8124e0 auth_reply(proto 2 0 (0) Success) v1 2020-09-08T15:32:07.170 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 -- 172.21.15.55:0/1815387893 queue 0x561b1b8124e0 prio 196 2020-09-08T15:32:07.170 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 --1- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 0x561b1b872800 :-1 s=OPENED pgs=99 cs=1 l=1).wait_message 2020-09-08T15:32:07.171 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d18d1d700 20 -- 172.21.15.55:0/1815387893 >> v1:172.21.15.55:6789/0 conn(0x561b1b889400 legacy=0x561b1b872800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read start len=1 2020-09-08T15:32:07.171 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d1751a700 1 -- 172.21.15.55:0/1815387893 <== mon.1 v1:172.21.15.55:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 1713080698 0 0) 0x561b1b8124e0 con 0x561b1b889400 2020-09-08T15:32:07.171 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d1751a700 10 -- 172.21.15.55:0/1815387893 dispatch_throttle_release 194 to dispatch throttler 194/104857600 2020-09-08T15:32:07.171 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.006+0000 7f9d1751a700 20 -- 172.21.15.55:0/1815387893 done calling dispatch on 0x561b1b8124e0 2020-09-08T15:32:07.902 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:07 smithi055 bash[14301]: debug 2020-09-08T15:32:07.566+0000 7f9d1ef9af40 -1 Falling back to public interface 2020-09-08T15:32:07.948 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping: /dev/vg_nvme/lv_2 2020-09-08T15:32:07.949 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2020-09-08T15:32:07.949 INFO:teuthology.orchestra.run.smithi055.stderr: stderr: 10+0 records in 2020-09-08T15:32:07.949 INFO:teuthology.orchestra.run.smithi055.stderr:10+0 records out 2020-09-08T15:32:07.950 INFO:teuthology.orchestra.run.smithi055.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0104745 s, 1.0 GB/s 2020-09-08T15:32:07.950 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping successful for: 2020-09-08T15:32:08.403 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:08 smithi055 bash[14301]: debug 2020-09-08T15:32:08.206+0000 7f9d1ef9af40 -1 osd.5 0 log_to_monitors {default=true} 2020-09-08T15:32:08.785 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph orch daemon add osd smithi055:vg_nvme/lv_2 2020-09-08T15:32:08.840 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:08 smithi055 bash[9296]: cluster 2020-09-08T15:32:07.488996+0000 mgr.x (mgr.24118) 101 : cluster [DBG] pgmap v99: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:08.840 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:08 smithi055 bash[9296]: audit 2020-09-08T15:32:08.211083+0000 mon.a (mon.0) 578 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2020-09-08T15:32:08.841 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:08 smithi055 bash[9296]: audit 2020-09-08T15:32:08.214561+0000 mon.c (mon.2) 5 : audit [INF] from='osd.5 v1:172.21.15.55:6811/1047046376' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2020-09-08T15:32:08.858 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:08 smithi166 bash[9446]: cluster 2020-09-08T15:32:07.488996+0000 mgr.x (mgr.24118) 101 : cluster [DBG] pgmap v99: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:08.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:08 smithi166 bash[9446]: audit 2020-09-08T15:32:08.211083+0000 mon.a (mon.0) 578 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2020-09-08T15:32:08.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:08 smithi166 bash[9446]: audit 2020-09-08T15:32:08.214561+0000 mon.c (mon.2) 5 : audit [INF] from='osd.5 v1:172.21.15.55:6811/1047046376' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2020-09-08T15:32:08.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:08 smithi104 bash[9203]: cluster 2020-09-08T15:32:07.488996+0000 mgr.x (mgr.24118) 101 : cluster [DBG] pgmap v99: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:08.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:08 smithi104 bash[9203]: audit 2020-09-08T15:32:08.211083+0000 mon.a (mon.0) 578 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2020-09-08T15:32:08.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:08 smithi104 bash[9203]: audit 2020-09-08T15:32:08.214561+0000 mon.c (mon.2) 5 : audit [INF] from='osd.5 v1:172.21.15.55:6811/1047046376' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2020-09-08T15:32:09.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:09 smithi166 bash[9446]: audit 2020-09-08T15:32:08.546461+0000 mon.a (mon.0) 579 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2020-09-08T15:32:09.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:09 smithi166 bash[9446]: cluster 2020-09-08T15:32:08.546571+0000 mon.a (mon.0) 580 : cluster [DBG] osdmap e40: 6 total, 5 up, 5 in 2020-09-08T15:32:09.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:09 smithi166 bash[9446]: audit 2020-09-08T15:32:08.547446+0000 mon.a (mon.0) 581 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:32:09.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:09 smithi166 bash[9446]: audit 2020-09-08T15:32:08.552818+0000 mon.c (mon.2) 6 : audit [INF] from='osd.5 v1:172.21.15.55:6811/1047046376' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:32:09.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:09 smithi166 bash[9446]: audit 2020-09-08T15:32:08.554907+0000 mon.a (mon.0) 582 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:32:09.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:09 smithi166 bash[9446]: audit 2020-09-08T15:32:09.531322+0000 mon.a (mon.0) 583 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:32:09.902 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:32:09 smithi055 bash[14301]: debug 2020-09-08T15:32:09.566+0000 7f9d15516700 -1 osd.5 0 waiting for initial osdmap 2020-09-08T15:32:09.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:09 smithi055 bash[9296]: audit 2020-09-08T15:32:08.546461+0000 mon.a (mon.0) 579 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2020-09-08T15:32:09.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:09 smithi055 bash[9296]: cluster 2020-09-08T15:32:08.546571+0000 mon.a (mon.0) 580 : cluster [DBG] osdmap e40: 6 total, 5 up, 5 in 2020-09-08T15:32:09.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:09 smithi055 bash[9296]: audit 2020-09-08T15:32:08.547446+0000 mon.a (mon.0) 581 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:32:09.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:09 smithi055 bash[9296]: audit 2020-09-08T15:32:08.552818+0000 mon.c (mon.2) 6 : audit [INF] from='osd.5 v1:172.21.15.55:6811/1047046376' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:32:09.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:09 smithi055 bash[9296]: audit 2020-09-08T15:32:08.554907+0000 mon.a (mon.0) 582 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:32:09.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:09 smithi055 bash[9296]: audit 2020-09-08T15:32:09.531322+0000 mon.a (mon.0) 583 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:32:09.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:09 smithi104 bash[9203]: audit 2020-09-08T15:32:08.546461+0000 mon.a (mon.0) 579 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2020-09-08T15:32:09.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:09 smithi104 bash[9203]: cluster 2020-09-08T15:32:08.546571+0000 mon.a (mon.0) 580 : cluster [DBG] osdmap e40: 6 total, 5 up, 5 in 2020-09-08T15:32:09.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:09 smithi104 bash[9203]: audit 2020-09-08T15:32:08.547446+0000 mon.a (mon.0) 581 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:32:09.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:09 smithi104 bash[9203]: audit 2020-09-08T15:32:08.552818+0000 mon.c (mon.2) 6 : audit [INF] from='osd.5 v1:172.21.15.55:6811/1047046376' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:32:09.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:09 smithi104 bash[9203]: audit 2020-09-08T15:32:08.554907+0000 mon.a (mon.0) 582 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:32:09.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:09 smithi104 bash[9203]: audit 2020-09-08T15:32:09.531322+0000 mon.a (mon.0) 583 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:32:10.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:10 smithi166 bash[9446]: cluster 2020-09-08T15:32:09.489488+0000 mgr.x (mgr.24118) 102 : cluster [DBG] pgmap v101: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:10.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:10 smithi166 bash[9446]: audit 2020-09-08T15:32:09.559320+0000 mon.a (mon.0) 584 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2020-09-08T15:32:10.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:10 smithi166 bash[9446]: cluster 2020-09-08T15:32:09.559407+0000 mon.a (mon.0) 585 : cluster [DBG] osdmap e41: 6 total, 5 up, 5 in 2020-09-08T15:32:10.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:10 smithi166 bash[9446]: audit 2020-09-08T15:32:09.560622+0000 mon.a (mon.0) 586 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:32:10.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:10 smithi166 bash[9446]: audit 2020-09-08T15:32:09.567543+0000 mon.a (mon.0) 587 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:32:10.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:10 smithi166 bash[9446]: audit 2020-09-08T15:32:09.820307+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}}, \"last_daemon_update\": \"2020-09-08T15:32:01.283458\", \"last_device_update\": \"2020-09-08T15:32:09.821478\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:10.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:10 smithi166 bash[9446]: audit 2020-09-08T15:32:09.826006+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}}, \"last_daemon_update\": \"2020-09-08T15:32:01.283458\", \"last_device_update\": \"2020-09-08T15:32:09.821478\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:10.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:10 smithi166 bash[9446]: audit 2020-09-08T15:32:09.828109+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:32:10.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:10 smithi166 bash[9446]: audit 2020-09-08T15:32:09.833252+0000 mon.a (mon.0) 591 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:32:10.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:10 smithi166 bash[9446]: audit 2020-09-08T15:32:09.837105+0000 mon.a (mon.0) 592 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:32:10.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:10 smithi166 bash[9446]: audit 2020-09-08T15:32:09.842569+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:32:10.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:10 smithi055 bash[9296]: cluster 2020-09-08T15:32:09.489488+0000 mgr.x (mgr.24118) 102 : cluster [DBG] pgmap v101: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:10.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:10 smithi055 bash[9296]: audit 2020-09-08T15:32:09.559320+0000 mon.a (mon.0) 584 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2020-09-08T15:32:10.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:10 smithi055 bash[9296]: cluster 2020-09-08T15:32:09.559407+0000 mon.a (mon.0) 585 : cluster [DBG] osdmap e41: 6 total, 5 up, 5 in 2020-09-08T15:32:10.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:10 smithi055 bash[9296]: audit 2020-09-08T15:32:09.560622+0000 mon.a (mon.0) 586 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:32:10.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:10 smithi055 bash[9296]: audit 2020-09-08T15:32:09.567543+0000 mon.a (mon.0) 587 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:32:10.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:10 smithi055 bash[9296]: audit 2020-09-08T15:32:09.820307+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}}, \"last_daemon_update\": \"2020-09-08T15:32:01.283458\", \"last_device_update\": \"2020-09-08T15:32:09.821478\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:10.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:10 smithi055 bash[9296]: audit 2020-09-08T15:32:09.826006+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}}, \"last_daemon_update\": \"2020-09-08T15:32:01.283458\", \"last_device_update\": \"2020-09-08T15:32:09.821478\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:10.912 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:10 smithi055 bash[9296]: audit 2020-09-08T15:32:09.828109+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:32:10.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:10 smithi055 bash[9296]: audit 2020-09-08T15:32:09.833252+0000 mon.a (mon.0) 591 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:32:10.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:10 smithi055 bash[9296]: audit 2020-09-08T15:32:09.837105+0000 mon.a (mon.0) 592 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:32:10.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:10 smithi055 bash[9296]: audit 2020-09-08T15:32:09.842569+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:32:10.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:10 smithi104 bash[9203]: cluster 2020-09-08T15:32:09.489488+0000 mgr.x (mgr.24118) 102 : cluster [DBG] pgmap v101: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:10.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:10 smithi104 bash[9203]: audit 2020-09-08T15:32:09.559320+0000 mon.a (mon.0) 584 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2020-09-08T15:32:10.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:10 smithi104 bash[9203]: cluster 2020-09-08T15:32:09.559407+0000 mon.a (mon.0) 585 : cluster [DBG] osdmap e41: 6 total, 5 up, 5 in 2020-09-08T15:32:10.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:10 smithi104 bash[9203]: audit 2020-09-08T15:32:09.560622+0000 mon.a (mon.0) 586 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:32:10.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:10 smithi104 bash[9203]: audit 2020-09-08T15:32:09.567543+0000 mon.a (mon.0) 587 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:32:10.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:10 smithi104 bash[9203]: audit 2020-09-08T15:32:09.820307+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}}, \"last_daemon_update\": \"2020-09-08T15:32:01.283458\", \"last_device_update\": \"2020-09-08T15:32:09.821478\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:10.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:10 smithi104 bash[9203]: audit 2020-09-08T15:32:09.826006+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}}, \"last_daemon_update\": \"2020-09-08T15:32:01.283458\", \"last_device_update\": \"2020-09-08T15:32:09.821478\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:10.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:10 smithi104 bash[9203]: audit 2020-09-08T15:32:09.828109+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:32:10.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:10 smithi104 bash[9203]: audit 2020-09-08T15:32:09.833252+0000 mon.a (mon.0) 591 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:32:10.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:10 smithi104 bash[9203]: audit 2020-09-08T15:32:09.837105+0000 mon.a (mon.0) 592 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:32:10.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:10 smithi104 bash[9203]: audit 2020-09-08T15:32:09.842569+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:32:11.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:11 smithi166 bash[9446]: cluster 2020-09-08T15:32:09.212462+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:11.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:11 smithi166 bash[9446]: cluster 2020-09-08T15:32:09.212669+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:11.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:11 smithi166 bash[9446]: cluster 2020-09-08T15:32:10.570685+0000 mon.a (mon.0) 594 : cluster [INF] osd.5 v1:172.21.15.55:6811/1047046376 boot 2020-09-08T15:32:11.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:11 smithi166 bash[9446]: cluster 2020-09-08T15:32:10.570821+0000 mon.a (mon.0) 595 : cluster [DBG] osdmap e42: 6 total, 6 up, 6 in 2020-09-08T15:32:11.872 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:11 smithi166 bash[9446]: audit 2020-09-08T15:32:10.571565+0000 mon.a (mon.0) 596 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:32:11.872 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:11 smithi166 bash[9446]: cluster 2020-09-08T15:32:11.490087+0000 mgr.x (mgr.24118) 103 : cluster [DBG] pgmap v104: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:11.892 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:11 smithi055 bash[9296]: cluster 2020-09-08T15:32:09.212462+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:11.893 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:11 smithi055 bash[9296]: cluster 2020-09-08T15:32:09.212669+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:11.893 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:11 smithi055 bash[9296]: cluster 2020-09-08T15:32:10.570685+0000 mon.a (mon.0) 594 : cluster [INF] osd.5 v1:172.21.15.55:6811/1047046376 boot 2020-09-08T15:32:11.893 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:11 smithi055 bash[9296]: cluster 2020-09-08T15:32:10.570821+0000 mon.a (mon.0) 595 : cluster [DBG] osdmap e42: 6 total, 6 up, 6 in 2020-09-08T15:32:11.893 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:11 smithi055 bash[9296]: audit 2020-09-08T15:32:10.571565+0000 mon.a (mon.0) 596 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:32:11.894 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:11 smithi055 bash[9296]: cluster 2020-09-08T15:32:11.490087+0000 mgr.x (mgr.24118) 103 : cluster [DBG] pgmap v104: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:11.924 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:11 smithi104 bash[9203]: cluster 2020-09-08T15:32:09.212462+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:11.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:11 smithi104 bash[9203]: cluster 2020-09-08T15:32:09.212669+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:11.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:11 smithi104 bash[9203]: cluster 2020-09-08T15:32:10.570685+0000 mon.a (mon.0) 594 : cluster [INF] osd.5 v1:172.21.15.55:6811/1047046376 boot 2020-09-08T15:32:11.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:11 smithi104 bash[9203]: cluster 2020-09-08T15:32:10.570821+0000 mon.a (mon.0) 595 : cluster [DBG] osdmap e42: 6 total, 6 up, 6 in 2020-09-08T15:32:11.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:11 smithi104 bash[9203]: audit 2020-09-08T15:32:10.571565+0000 mon.a (mon.0) 596 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2020-09-08T15:32:11.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:11 smithi104 bash[9203]: cluster 2020-09-08T15:32:11.490087+0000 mgr.x (mgr.24118) 103 : cluster [DBG] pgmap v104: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2020-09-08T15:32:12.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:12 smithi166 bash[9446]: cluster 2020-09-08T15:32:11.595793+0000 mon.a (mon.0) 597 : cluster [DBG] osdmap e43: 6 total, 6 up, 6 in 2020-09-08T15:32:12.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:12 smithi166 bash[9446]: audit 2020-09-08T15:32:12.262021+0000 mon.a (mon.0) 598 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:32:12.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:12 smithi166 bash[9446]: audit 2020-09-08T15:32:12.263350+0000 mgr.x (mgr.24118) 104 : audit [DBG] from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:32:12.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:12 smithi166 bash[9446]: audit 2020-09-08T15:32:12.264847+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:32:12.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:12 smithi166 bash[9446]: cephadm 2020-09-08T15:32:12.266013+0000 mgr.x (mgr.24118) 105 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:32:12.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:12 smithi166 bash[9446]: cephadm 2020-09-08T15:32:12.266085+0000 mgr.x (mgr.24118) 106 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:32:12.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:12 smithi166 bash[9446]: audit 2020-09-08T15:32:12.266554+0000 mon.a (mon.0) 600 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:32:12.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:12 smithi166 bash[9446]: cephadm 2020-09-08T15:32:12.266681+0000 mgr.x (mgr.24118) 107 : cephadm [INF] Applying None on host smithi055... 2020-09-08T15:32:12.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:12 smithi166 bash[9446]: audit 2020-09-08T15:32:12.268269+0000 mon.a (mon.0) 601 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:12.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:12 smithi055 bash[9296]: cluster 2020-09-08T15:32:11.595793+0000 mon.a (mon.0) 597 : cluster [DBG] osdmap e43: 6 total, 6 up, 6 in 2020-09-08T15:32:12.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:12 smithi055 bash[9296]: audit 2020-09-08T15:32:12.262021+0000 mon.a (mon.0) 598 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:32:12.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:12 smithi055 bash[9296]: audit 2020-09-08T15:32:12.263350+0000 mgr.x (mgr.24118) 104 : audit [DBG] from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:32:12.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:12 smithi055 bash[9296]: audit 2020-09-08T15:32:12.264847+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:32:12.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:12 smithi055 bash[9296]: cephadm 2020-09-08T15:32:12.266013+0000 mgr.x (mgr.24118) 105 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:32:12.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:12 smithi055 bash[9296]: cephadm 2020-09-08T15:32:12.266085+0000 mgr.x (mgr.24118) 106 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:32:12.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:12 smithi055 bash[9296]: audit 2020-09-08T15:32:12.266554+0000 mon.a (mon.0) 600 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:32:12.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:12 smithi055 bash[9296]: cephadm 2020-09-08T15:32:12.266681+0000 mgr.x (mgr.24118) 107 : cephadm [INF] Applying None on host smithi055... 2020-09-08T15:32:12.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:12 smithi055 bash[9296]: audit 2020-09-08T15:32:12.268269+0000 mon.a (mon.0) 601 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:12.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:12 smithi104 bash[9203]: cluster 2020-09-08T15:32:11.595793+0000 mon.a (mon.0) 597 : cluster [DBG] osdmap e43: 6 total, 6 up, 6 in 2020-09-08T15:32:12.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:12 smithi104 bash[9203]: audit 2020-09-08T15:32:12.262021+0000 mon.a (mon.0) 598 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:32:12.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:12 smithi104 bash[9203]: audit 2020-09-08T15:32:12.263350+0000 mgr.x (mgr.24118) 104 : audit [DBG] from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:32:12.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:12 smithi104 bash[9203]: audit 2020-09-08T15:32:12.264847+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:32:12.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:12 smithi104 bash[9203]: cephadm 2020-09-08T15:32:12.266013+0000 mgr.x (mgr.24118) 105 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:32:12.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:12 smithi104 bash[9203]: cephadm 2020-09-08T15:32:12.266085+0000 mgr.x (mgr.24118) 106 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:32:12.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:12 smithi104 bash[9203]: audit 2020-09-08T15:32:12.266554+0000 mon.a (mon.0) 600 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:32:12.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:12 smithi104 bash[9203]: cephadm 2020-09-08T15:32:12.266681+0000 mgr.x (mgr.24118) 107 : cephadm [INF] Applying None on host smithi055... 2020-09-08T15:32:12.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:12 smithi104 bash[9203]: audit 2020-09-08T15:32:12.268269+0000 mon.a (mon.0) 601 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:13.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:13 smithi166 bash[9446]: cluster 2020-09-08T15:32:13.490757+0000 mgr.x (mgr.24118) 108 : cluster [DBG] pgmap v106: 1 pgs: 1 peering; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:13.902 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:13 smithi055 bash[9296]: cluster 2020-09-08T15:32:13.490757+0000 mgr.x (mgr.24118) 108 : cluster [DBG] pgmap v106: 1 pgs: 1 peering; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:13.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:13 smithi104 bash[9203]: cluster 2020-09-08T15:32:13.490757+0000 mgr.x (mgr.24118) 108 : cluster [DBG] pgmap v106: 1 pgs: 1 peering; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:16.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:16 smithi166 bash[9446]: cluster 2020-09-08T15:32:15.491351+0000 mgr.x (mgr.24118) 109 : cluster [DBG] pgmap v107: 1 pgs: 1 peering; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:16.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:16 smithi166 bash[9446]: audit 2020-09-08T15:32:16.367393+0000 mon.a (mon.0) 602 : audit [INF] from='client.24307 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad9a93fe-0ca0-4052-bfa9-e2abb268829e"}]: dispatch 2020-09-08T15:32:16.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:16 smithi166 bash[9446]: audit 2020-09-08T15:32:16.369423+0000 mon.b (mon.1) 13 : audit [INF] from='client.? v1:172.21.15.55:0/835838804' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad9a93fe-0ca0-4052-bfa9-e2abb268829e"}]: dispatch 2020-09-08T15:32:16.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:16 smithi166 bash[9446]: audit 2020-09-08T15:32:16.375672+0000 mon.a (mon.0) 603 : audit [INF] from='client.24307 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad9a93fe-0ca0-4052-bfa9-e2abb268829e"}]': finished 2020-09-08T15:32:16.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:16 smithi166 bash[9446]: cluster 2020-09-08T15:32:16.375760+0000 mon.a (mon.0) 604 : cluster [DBG] osdmap e44: 7 total, 6 up, 6 in 2020-09-08T15:32:16.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:16 smithi166 bash[9446]: audit 2020-09-08T15:32:16.375958+0000 mon.a (mon.0) 605 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:16.902 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:16 smithi055 bash[9296]: cluster 2020-09-08T15:32:15.491351+0000 mgr.x (mgr.24118) 109 : cluster [DBG] pgmap v107: 1 pgs: 1 peering; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:16.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:16 smithi055 bash[9296]: audit 2020-09-08T15:32:16.367393+0000 mon.a (mon.0) 602 : audit [INF] from='client.24307 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad9a93fe-0ca0-4052-bfa9-e2abb268829e"}]: dispatch 2020-09-08T15:32:16.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:16 smithi055 bash[9296]: audit 2020-09-08T15:32:16.369423+0000 mon.b (mon.1) 13 : audit [INF] from='client.? v1:172.21.15.55:0/835838804' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad9a93fe-0ca0-4052-bfa9-e2abb268829e"}]: dispatch 2020-09-08T15:32:16.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:16 smithi055 bash[9296]: audit 2020-09-08T15:32:16.375672+0000 mon.a (mon.0) 603 : audit [INF] from='client.24307 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad9a93fe-0ca0-4052-bfa9-e2abb268829e"}]': finished 2020-09-08T15:32:16.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:16 smithi055 bash[9296]: cluster 2020-09-08T15:32:16.375760+0000 mon.a (mon.0) 604 : cluster [DBG] osdmap e44: 7 total, 6 up, 6 in 2020-09-08T15:32:16.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:16 smithi055 bash[9296]: audit 2020-09-08T15:32:16.375958+0000 mon.a (mon.0) 605 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:16.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:16 smithi104 bash[9203]: cluster 2020-09-08T15:32:15.491351+0000 mgr.x (mgr.24118) 109 : cluster [DBG] pgmap v107: 1 pgs: 1 peering; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:16.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:16 smithi104 bash[9203]: audit 2020-09-08T15:32:16.367393+0000 mon.a (mon.0) 602 : audit [INF] from='client.24307 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad9a93fe-0ca0-4052-bfa9-e2abb268829e"}]: dispatch 2020-09-08T15:32:16.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:16 smithi104 bash[9203]: audit 2020-09-08T15:32:16.369423+0000 mon.b (mon.1) 13 : audit [INF] from='client.? v1:172.21.15.55:0/835838804' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad9a93fe-0ca0-4052-bfa9-e2abb268829e"}]: dispatch 2020-09-08T15:32:16.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:16 smithi104 bash[9203]: audit 2020-09-08T15:32:16.375672+0000 mon.a (mon.0) 603 : audit [INF] from='client.24307 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad9a93fe-0ca0-4052-bfa9-e2abb268829e"}]': finished 2020-09-08T15:32:16.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:16 smithi104 bash[9203]: cluster 2020-09-08T15:32:16.375760+0000 mon.a (mon.0) 604 : cluster [DBG] osdmap e44: 7 total, 6 up, 6 in 2020-09-08T15:32:16.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:16 smithi104 bash[9203]: audit 2020-09-08T15:32:16.375958+0000 mon.a (mon.0) 605 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:17.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:17 smithi166 bash[9446]: audit 2020-09-08T15:32:17.027626+0000 mon.b (mon.1) 14 : audit [DBG] from='client.? v1:172.21.15.55:0/3552192428' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:32:17.902 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:17 smithi055 bash[9296]: audit 2020-09-08T15:32:17.027626+0000 mon.b (mon.1) 14 : audit [DBG] from='client.? v1:172.21.15.55:0/3552192428' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:32:17.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:17 smithi104 bash[9203]: audit 2020-09-08T15:32:17.027626+0000 mon.b (mon.1) 14 : audit [DBG] from='client.? v1:172.21.15.55:0/3552192428' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:32:18.858 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:18 smithi166 bash[9446]: cluster 2020-09-08T15:32:17.492030+0000 mgr.x (mgr.24118) 110 : cluster [DBG] pgmap v109: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:18.902 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:18 smithi055 bash[9296]: cluster 2020-09-08T15:32:17.492030+0000 mgr.x (mgr.24118) 110 : cluster [DBG] pgmap v109: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:18.924 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:18 smithi104 bash[9203]: cluster 2020-09-08T15:32:17.492030+0000 mgr.x (mgr.24118) 110 : cluster [DBG] pgmap v109: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:20.858 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:20 smithi166 bash[9446]: cluster 2020-09-08T15:32:18.677862+0000 osd.3 (osd.3) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:20.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:20 smithi166 bash[9446]: cluster 2020-09-08T15:32:18.678004+0000 osd.3 (osd.3) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:20.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:20 smithi166 bash[9446]: cluster 2020-09-08T15:32:19.492664+0000 mgr.x (mgr.24118) 111 : cluster [DBG] pgmap v110: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:20.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:20 smithi166 bash[9446]: cluster 2020-09-08T15:32:19.563414+0000 mon.a (mon.0) 606 : cluster [DBG] osdmap e45: 7 total, 6 up, 6 in 2020-09-08T15:32:20.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:20 smithi166 bash[9446]: audit 2020-09-08T15:32:19.568095+0000 mon.b (mon.1) 15 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:20.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:20 smithi166 bash[9446]: audit 2020-09-08T15:32:20.046702+0000 mon.b (mon.1) 16 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:20.902 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:20 smithi055 bash[9296]: cluster 2020-09-08T15:32:18.677862+0000 osd.3 (osd.3) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:20.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:20 smithi055 bash[9296]: cluster 2020-09-08T15:32:18.678004+0000 osd.3 (osd.3) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:20.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:20 smithi055 bash[9296]: cluster 2020-09-08T15:32:19.492664+0000 mgr.x (mgr.24118) 111 : cluster [DBG] pgmap v110: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:20.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:20 smithi055 bash[9296]: cluster 2020-09-08T15:32:19.563414+0000 mon.a (mon.0) 606 : cluster [DBG] osdmap e45: 7 total, 6 up, 6 in 2020-09-08T15:32:20.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:20 smithi055 bash[9296]: audit 2020-09-08T15:32:19.568095+0000 mon.b (mon.1) 15 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:20.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:20 smithi055 bash[9296]: audit 2020-09-08T15:32:20.046702+0000 mon.b (mon.1) 16 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:20.924 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:20 smithi104 bash[9203]: cluster 2020-09-08T15:32:18.677862+0000 osd.3 (osd.3) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:20.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:20 smithi104 bash[9203]: cluster 2020-09-08T15:32:18.678004+0000 osd.3 (osd.3) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:20.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:20 smithi104 bash[9203]: cluster 2020-09-08T15:32:19.492664+0000 mgr.x (mgr.24118) 111 : cluster [DBG] pgmap v110: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:20.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:20 smithi104 bash[9203]: cluster 2020-09-08T15:32:19.563414+0000 mon.a (mon.0) 606 : cluster [DBG] osdmap e45: 7 total, 6 up, 6 in 2020-09-08T15:32:20.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:20 smithi104 bash[9203]: audit 2020-09-08T15:32:19.568095+0000 mon.b (mon.1) 15 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:20.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:20 smithi104 bash[9203]: audit 2020-09-08T15:32:20.046702+0000 mon.b (mon.1) 16 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:22.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:22 smithi166 bash[9446]: cluster 2020-09-08T15:32:21.493276+0000 mgr.x (mgr.24118) 112 : cluster [DBG] pgmap v112: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:22.902 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:22 smithi055 bash[9296]: cluster 2020-09-08T15:32:21.493276+0000 mgr.x (mgr.24118) 112 : cluster [DBG] pgmap v112: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:22.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:22 smithi104 bash[9203]: cluster 2020-09-08T15:32:21.493276+0000 mgr.x (mgr.24118) 112 : cluster [DBG] pgmap v112: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:24.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:24 smithi166 bash[9446]: cluster 2020-09-08T15:32:23.493944+0000 mgr.x (mgr.24118) 113 : cluster [DBG] pgmap v113: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:24.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:24 smithi166 bash[9446]: audit 2020-09-08T15:32:24.324761+0000 mon.b (mon.1) 17 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2020-09-08T15:32:24.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:24 smithi166 bash[9446]: audit 2020-09-08T15:32:24.326451+0000 mon.b (mon.1) 18 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:32:24.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:24 smithi166 bash[9446]: audit 2020-09-08T15:32:24.328497+0000 mon.b (mon.1) 19 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.6", "key": "container_image"}]: dispatch 2020-09-08T15:32:24.902 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:24 smithi055 bash[9296]: cluster 2020-09-08T15:32:23.493944+0000 mgr.x (mgr.24118) 113 : cluster [DBG] pgmap v113: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:24.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:24 smithi055 bash[9296]: audit 2020-09-08T15:32:24.324761+0000 mon.b (mon.1) 17 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2020-09-08T15:32:24.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:24 smithi055 bash[9296]: audit 2020-09-08T15:32:24.326451+0000 mon.b (mon.1) 18 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:32:24.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:24 smithi055 bash[9296]: audit 2020-09-08T15:32:24.328497+0000 mon.b (mon.1) 19 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.6", "key": "container_image"}]: dispatch 2020-09-08T15:32:24.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:24 smithi104 bash[9203]: cluster 2020-09-08T15:32:23.493944+0000 mgr.x (mgr.24118) 113 : cluster [DBG] pgmap v113: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:24.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:24 smithi104 bash[9203]: audit 2020-09-08T15:32:24.324761+0000 mon.b (mon.1) 17 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2020-09-08T15:32:24.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:24 smithi104 bash[9203]: audit 2020-09-08T15:32:24.326451+0000 mon.b (mon.1) 18 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:32:24.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:24 smithi104 bash[9203]: audit 2020-09-08T15:32:24.328497+0000 mon.b (mon.1) 19 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.6", "key": "container_image"}]: dispatch 2020-09-08T15:32:25.858 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:25 smithi166 bash[9446]: cephadm 2020-09-08T15:32:24.327729+0000 mgr.x (mgr.24118) 114 : cephadm [INF] Deploying daemon osd.6 on smithi055 2020-09-08T15:32:25.889 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:25 smithi055 bash[9296]: cephadm 2020-09-08T15:32:24.327729+0000 mgr.x (mgr.24118) 114 : cephadm [INF] Deploying daemon osd.6 on smithi055 2020-09-08T15:32:25.924 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:25 smithi104 bash[9203]: cephadm 2020-09-08T15:32:24.327729+0000 mgr.x (mgr.24118) 114 : cephadm [INF] Deploying daemon osd.6 on smithi055 2020-09-08T15:32:26.717 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:26 smithi055 bash[9296]: cluster 2020-09-08T15:32:25.494615+0000 mgr.x (mgr.24118) 115 : cluster [DBG] pgmap v114: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:26.754 INFO:teuthology.orchestra.run.smithi055.stdout:Created osd(s) 6 on host 'smithi055' 2020-09-08T15:32:26.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:26 smithi166 bash[9446]: cluster 2020-09-08T15:32:25.494615+0000 mgr.x (mgr.24118) 115 : cluster [DBG] pgmap v114: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:26.924 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:26 smithi104 bash[9203]: cluster 2020-09-08T15:32:25.494615+0000 mgr.x (mgr.24118) 115 : cluster [DBG] pgmap v114: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:27.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:27 smithi166 bash[9446]: audit 2020-09-08T15:32:26.754078+0000 mon.a (mon.0) 607 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_device_update\": \"2020-09-08T15:32:09.821478\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:27.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:27 smithi166 bash[9446]: audit 2020-09-08T15:32:26.756309+0000 mon.b (mon.1) 20 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_device_update\": \"2020-09-08T15:32:09.821478\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:27.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:27 smithi166 bash[9446]: audit 2020-09-08T15:32:26.756699+0000 mon.b (mon.1) 21 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:32:27.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:27 smithi166 bash[9446]: audit 2020-09-08T15:32:26.760945+0000 mon.a (mon.0) 608 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_device_update\": \"2020-09-08T15:32:09.821478\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:27.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:27 smithi166 bash[9446]: cluster 2020-09-08T15:32:27.495160+0000 mgr.x (mgr.24118) 116 : cluster [DBG] pgmap v115: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:27.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:27 smithi166 bash[9446]: audit 2020-09-08T15:32:27.562794+0000 mon.a (mon.0) 609 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_daemon_update\": \"2020-09-08T15:32:27.563932\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:27.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:27 smithi166 bash[9446]: audit 2020-09-08T15:32:27.565004+0000 mon.b (mon.1) 22 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_daemon_update\": \"2020-09-08T15:32:27.563932\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:27.902 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:27 smithi055 bash[9296]: audit 2020-09-08T15:32:26.754078+0000 mon.a (mon.0) 607 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_device_update\": \"2020-09-08T15:32:09.821478\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:27.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:27 smithi055 bash[9296]: audit 2020-09-08T15:32:26.756309+0000 mon.b (mon.1) 20 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_device_update\": \"2020-09-08T15:32:09.821478\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:27.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:27 smithi055 bash[9296]: audit 2020-09-08T15:32:26.756699+0000 mon.b (mon.1) 21 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:32:27.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:27 smithi055 bash[9296]: audit 2020-09-08T15:32:26.760945+0000 mon.a (mon.0) 608 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_device_update\": \"2020-09-08T15:32:09.821478\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:27.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:27 smithi055 bash[9296]: cluster 2020-09-08T15:32:27.495160+0000 mgr.x (mgr.24118) 116 : cluster [DBG] pgmap v115: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:27.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:27 smithi055 bash[9296]: audit 2020-09-08T15:32:27.562794+0000 mon.a (mon.0) 609 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_daemon_update\": \"2020-09-08T15:32:27.563932\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:27.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:27 smithi055 bash[9296]: audit 2020-09-08T15:32:27.565004+0000 mon.b (mon.1) 22 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_daemon_update\": \"2020-09-08T15:32:27.563932\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:27.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:27 smithi104 bash[9203]: audit 2020-09-08T15:32:26.754078+0000 mon.a (mon.0) 607 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_device_update\": \"2020-09-08T15:32:09.821478\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:27.933 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:27 smithi104 bash[9203]: audit 2020-09-08T15:32:26.756309+0000 mon.b (mon.1) 20 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_device_update\": \"2020-09-08T15:32:09.821478\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:27.933 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:27 smithi104 bash[9203]: audit 2020-09-08T15:32:26.756699+0000 mon.b (mon.1) 21 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:32:27.933 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:27 smithi104 bash[9203]: audit 2020-09-08T15:32:26.760945+0000 mon.a (mon.0) 608 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282263\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:01.282560\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.282691\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:01.283062\", \"created\": \"2020-09-08T15:32:00.506147\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_device_update\": \"2020-09-08T15:32:09.821478\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:27.934 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:27 smithi104 bash[9203]: cluster 2020-09-08T15:32:27.495160+0000 mgr.x (mgr.24118) 116 : cluster [DBG] pgmap v115: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:27.934 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:27 smithi104 bash[9203]: audit 2020-09-08T15:32:27.562794+0000 mon.a (mon.0) 609 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_daemon_update\": \"2020-09-08T15:32:27.563932\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:27.934 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:27 smithi104 bash[9203]: audit 2020-09-08T15:32:27.565004+0000 mon.b (mon.1) 22 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_daemon_update\": \"2020-09-08T15:32:27.563932\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:28.230 INFO:teuthology.orchestra.run.smithi055:osd.6> sudo journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.6.service 2020-09-08T15:32:28.273 INFO:tasks.cephadm:Deploying osd.7 on smithi055 with /dev/vg_nvme/lv_1... 2020-09-08T15:32:28.274 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2020-09-08T15:32:28.286 INFO:journalctl@ceph.osd.6.smithi055.stdout:-- Logs begin at Tue 2020-09-01 16:49:30 UTC. -- 2020-09-08T15:32:28.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:28 smithi166 bash[9446]: audit 2020-09-08T15:32:27.567828+0000 mon.a (mon.0) 610 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_daemon_update\": \"2020-09-08T15:32:27.563932\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:28.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:28 smithi166 bash[9446]: audit 2020-09-08T15:32:27.573965+0000 mon.b (mon.1) 23 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:28.902 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:28 smithi055 bash[9296]: audit 2020-09-08T15:32:27.567828+0000 mon.a (mon.0) 610 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_daemon_update\": \"2020-09-08T15:32:27.563932\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:28.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:28 smithi055 bash[9296]: audit 2020-09-08T15:32:27.573965+0000 mon.b (mon.1) 23 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:28.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:28 smithi104 bash[9203]: audit 2020-09-08T15:32:27.567828+0000 mon.a (mon.0) 610 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_daemon_update\": \"2020-09-08T15:32:27.563932\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:28.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:28 smithi104 bash[9203]: audit 2020-09-08T15:32:27.573965+0000 mon.b (mon.1) 23 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:29.652 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:32:29 smithi055 bash[16646]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2020-09-08T15:32:29.652 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:32:29 smithi055 bash[16646]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-6 --no-mon-config 2020-09-08T15:32:29.653 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:32:29 smithi055 bash[16646]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6/block 2020-09-08T15:32:29.653 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:32:29 smithi055 bash[16646]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-6/block 2020-09-08T15:32:29.653 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:32:29 smithi055 bash[16646]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2020-09-08T15:32:29.653 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:32:29 smithi055 bash[16646]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2020-09-08T15:32:29.654 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:32:29 smithi055 bash[16646]: --> ceph-volume lvm activate successful for osd ID: 6 2020-09-08T15:32:30.652 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:32:30 smithi055 bash[16646]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:32:30.653 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:30 smithi055 bash[9296]: cluster 2020-09-08T15:32:29.495680+0000 mgr.x (mgr.24118) 117 : cluster [DBG] pgmap v116: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:30.653 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:30 smithi055 bash[9296]: audit 2020-09-08T15:32:29.665930+0000 mon.a (mon.0) 611 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:32:30.653 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:30 smithi055 bash[9296]: audit 2020-09-08T15:32:29.668562+0000 mon.b (mon.1) 24 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:32:30.654 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:30 smithi055 bash[9296]: audit 2020-09-08T15:32:29.672594+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:32:30.858 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:30 smithi166 bash[9446]: cluster 2020-09-08T15:32:29.495680+0000 mgr.x (mgr.24118) 117 : cluster [DBG] pgmap v116: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:30.982 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:30 smithi166 bash[9446]: audit 2020-09-08T15:32:29.665930+0000 mon.a (mon.0) 611 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:32:30.983 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:30 smithi166 bash[9446]: audit 2020-09-08T15:32:29.668562+0000 mon.b (mon.1) 24 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:32:30.983 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:30 smithi166 bash[9446]: audit 2020-09-08T15:32:29.672594+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:32:30.984 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:30 smithi104 bash[9203]: cluster 2020-09-08T15:32:29.495680+0000 mgr.x (mgr.24118) 117 : cluster [DBG] pgmap v116: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:30.985 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:30 smithi104 bash[9203]: audit 2020-09-08T15:32:29.665930+0000 mon.a (mon.0) 611 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:32:30.985 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:30 smithi104 bash[9203]: audit 2020-09-08T15:32:29.668562+0000 mon.b (mon.1) 24 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:32:30.985 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:30 smithi104 bash[9203]: audit 2020-09-08T15:32:29.672594+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:32:31.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:31 smithi104 bash[9203]: cluster 2020-09-08T15:32:30.322283+0000 osd.1 (osd.1) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:31.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:31 smithi104 bash[9203]: cluster 2020-09-08T15:32:30.322457+0000 osd.1 (osd.1) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:31.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:31 smithi104 bash[9203]: cluster 2020-09-08T15:32:30.676000+0000 mon.a (mon.0) 613 : cluster [DBG] osdmap e46: 7 total, 6 up, 6 in 2020-09-08T15:32:31.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:31 smithi104 bash[9203]: audit 2020-09-08T15:32:30.682373+0000 mon.b (mon.1) 25 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:31.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:31 smithi104 bash[9203]: cluster 2020-09-08T15:32:31.496083+0000 mgr.x (mgr.24118) 118 : cluster [DBG] pgmap v118: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:32.108 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:31 smithi166 bash[9446]: cluster 2020-09-08T15:32:30.322283+0000 osd.1 (osd.1) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:32.109 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:31 smithi166 bash[9446]: cluster 2020-09-08T15:32:30.322457+0000 osd.1 (osd.1) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:32.109 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:31 smithi166 bash[9446]: cluster 2020-09-08T15:32:30.676000+0000 mon.a (mon.0) 613 : cluster [DBG] osdmap e46: 7 total, 6 up, 6 in 2020-09-08T15:32:32.110 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:31 smithi166 bash[9446]: audit 2020-09-08T15:32:30.682373+0000 mon.b (mon.1) 25 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:32.110 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:31 smithi166 bash[9446]: cluster 2020-09-08T15:32:31.496083+0000 mgr.x (mgr.24118) 118 : cluster [DBG] pgmap v118: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:32.283 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:31 smithi055 bash[9296]: cluster 2020-09-08T15:32:30.322283+0000 osd.1 (osd.1) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:32.284 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:31 smithi055 bash[9296]: cluster 2020-09-08T15:32:30.322457+0000 osd.1 (osd.1) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:32.284 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:31 smithi055 bash[9296]: cluster 2020-09-08T15:32:30.676000+0000 mon.a (mon.0) 613 : cluster [DBG] osdmap e46: 7 total, 6 up, 6 in 2020-09-08T15:32:32.284 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:31 smithi055 bash[9296]: audit 2020-09-08T15:32:30.682373+0000 mon.b (mon.1) 25 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:32.284 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:31 smithi055 bash[9296]: cluster 2020-09-08T15:32:31.496083+0000 mgr.x (mgr.24118) 118 : cluster [DBG] pgmap v118: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:33.152 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:32:32 smithi055 bash[16646]: debug 2020-09-08T15:32:32.997+0000 7f5a96940700 10 -- 172.21.15.55:0/185825700 dispatch_throttle_release 194 to dispatch throttler 194/104857600 2020-09-08T15:32:33.152 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:32:32 smithi055 bash[16646]: debug 2020-09-08T15:32:32.997+0000 7f5a96940700 20 -- 172.21.15.55:0/185825700 done calling dispatch on 0x55d7b68904e0 2020-09-08T15:32:33.864 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:32:33 smithi055 bash[16646]: debug 2020-09-08T15:32:33.553+0000 7f5a9e3c0f40 -1 Falling back to public interface 2020-09-08T15:32:34.514 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping: /dev/vg_nvme/lv_1 2020-09-08T15:32:34.515 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2020-09-08T15:32:34.515 INFO:teuthology.orchestra.run.smithi055.stderr: stderr: 10+0 records in 2020-09-08T15:32:34.515 INFO:teuthology.orchestra.run.smithi055.stderr:10+0 records out 2020-09-08T15:32:34.516 INFO:teuthology.orchestra.run.smithi055.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0167201 s, 627 MB/s 2020-09-08T15:32:34.516 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping successful for: 2020-09-08T15:32:34.553 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:32:34 smithi055 bash[16646]: debug 2020-09-08T15:32:34.189+0000 7f5a9e3c0f40 -1 osd.6 0 log_to_monitors {default=true} 2020-09-08T15:32:34.553 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:34 smithi055 bash[9296]: cluster 2020-09-08T15:32:33.496486+0000 mgr.x (mgr.24118) 119 : cluster [DBG] pgmap v119: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:34.554 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:34 smithi055 bash[9296]: audit 2020-09-08T15:32:34.194627+0000 mon.a (mon.0) 614 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2020-09-08T15:32:34.554 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:34 smithi055 bash[9296]: audit 2020-09-08T15:32:34.197161+0000 mon.b (mon.1) 26 : audit [INF] from='osd.6 v1:172.21.15.55:6819/2501844793' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2020-09-08T15:32:34.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:34 smithi166 bash[9446]: cluster 2020-09-08T15:32:33.496486+0000 mgr.x (mgr.24118) 119 : cluster [DBG] pgmap v119: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:34.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:34 smithi166 bash[9446]: audit 2020-09-08T15:32:34.194627+0000 mon.a (mon.0) 614 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2020-09-08T15:32:34.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:34 smithi166 bash[9446]: audit 2020-09-08T15:32:34.197161+0000 mon.b (mon.1) 26 : audit [INF] from='osd.6 v1:172.21.15.55:6819/2501844793' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2020-09-08T15:32:34.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:34 smithi104 bash[9203]: cluster 2020-09-08T15:32:33.496486+0000 mgr.x (mgr.24118) 119 : cluster [DBG] pgmap v119: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:34.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:34 smithi104 bash[9203]: audit 2020-09-08T15:32:34.194627+0000 mon.a (mon.0) 614 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2020-09-08T15:32:34.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:34 smithi104 bash[9203]: audit 2020-09-08T15:32:34.197161+0000 mon.b (mon.1) 26 : audit [INF] from='osd.6 v1:172.21.15.55:6819/2501844793' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2020-09-08T15:32:35.537 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph orch daemon add osd smithi055:vg_nvme/lv_1 2020-09-08T15:32:35.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:35 smithi166 bash[9446]: audit 2020-09-08T15:32:34.553272+0000 mon.a (mon.0) 615 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2020-09-08T15:32:35.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:35 smithi166 bash[9446]: cluster 2020-09-08T15:32:34.553350+0000 mon.a (mon.0) 616 : cluster [DBG] osdmap e47: 7 total, 6 up, 6 in 2020-09-08T15:32:35.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:35 smithi166 bash[9446]: audit 2020-09-08T15:32:34.556819+0000 mon.b (mon.1) 27 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:35.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:35 smithi166 bash[9446]: audit 2020-09-08T15:32:34.558338+0000 mon.b (mon.1) 28 : audit [INF] from='osd.6 v1:172.21.15.55:6819/2501844793' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:32:35.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:35 smithi166 bash[9446]: audit 2020-09-08T15:32:34.564915+0000 mon.a (mon.0) 617 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:32:35.902 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:32:35 smithi055 bash[16646]: debug 2020-09-08T15:32:35.581+0000 7f5a9493c700 -1 osd.6 0 waiting for initial osdmap 2020-09-08T15:32:35.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:35 smithi055 bash[9296]: audit 2020-09-08T15:32:34.553272+0000 mon.a (mon.0) 615 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2020-09-08T15:32:35.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:35 smithi055 bash[9296]: cluster 2020-09-08T15:32:34.553350+0000 mon.a (mon.0) 616 : cluster [DBG] osdmap e47: 7 total, 6 up, 6 in 2020-09-08T15:32:35.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:35 smithi055 bash[9296]: audit 2020-09-08T15:32:34.556819+0000 mon.b (mon.1) 27 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:35.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:35 smithi055 bash[9296]: audit 2020-09-08T15:32:34.558338+0000 mon.b (mon.1) 28 : audit [INF] from='osd.6 v1:172.21.15.55:6819/2501844793' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:32:35.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:35 smithi055 bash[9296]: audit 2020-09-08T15:32:34.564915+0000 mon.a (mon.0) 617 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:32:35.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:35 smithi104 bash[9203]: audit 2020-09-08T15:32:34.553272+0000 mon.a (mon.0) 615 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2020-09-08T15:32:35.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:35 smithi104 bash[9203]: cluster 2020-09-08T15:32:34.553350+0000 mon.a (mon.0) 616 : cluster [DBG] osdmap e47: 7 total, 6 up, 6 in 2020-09-08T15:32:35.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:35 smithi104 bash[9203]: audit 2020-09-08T15:32:34.556819+0000 mon.b (mon.1) 27 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:35.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:35 smithi104 bash[9203]: audit 2020-09-08T15:32:34.558338+0000 mon.b (mon.1) 28 : audit [INF] from='osd.6 v1:172.21.15.55:6819/2501844793' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:32:35.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:35 smithi104 bash[9203]: audit 2020-09-08T15:32:34.564915+0000 mon.a (mon.0) 617 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:32:36.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:36 smithi166 bash[9446]: cluster 2020-09-08T15:32:35.496864+0000 mgr.x (mgr.24118) 120 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:36.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:36 smithi166 bash[9446]: audit 2020-09-08T15:32:35.573021+0000 mon.a (mon.0) 618 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2020-09-08T15:32:36.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:36 smithi166 bash[9446]: cluster 2020-09-08T15:32:35.573120+0000 mon.a (mon.0) 619 : cluster [DBG] osdmap e48: 7 total, 6 up, 6 in 2020-09-08T15:32:36.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:36 smithi166 bash[9446]: audit 2020-09-08T15:32:35.580853+0000 mon.b (mon.1) 29 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:36.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:36 smithi166 bash[9446]: audit 2020-09-08T15:32:35.585217+0000 mon.b (mon.1) 30 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:36.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:36 smithi166 bash[9446]: audit 2020-09-08T15:32:35.665890+0000 mon.b (mon.1) 31 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:32:36.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:36 smithi166 bash[9446]: audit 2020-09-08T15:32:35.947505+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_daemon_update\": \"2020-09-08T15:32:27.563932\", \"last_device_update\": \"2020-09-08T15:32:35.947589\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:36.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:36 smithi166 bash[9446]: audit 2020-09-08T15:32:35.949603+0000 mon.b (mon.1) 32 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_daemon_update\": \"2020-09-08T15:32:27.563932\", \"last_device_update\": \"2020-09-08T15:32:35.947589\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:36.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:36 smithi166 bash[9446]: audit 2020-09-08T15:32:35.953506+0000 mon.a (mon.0) 621 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_daemon_update\": \"2020-09-08T15:32:27.563932\", \"last_device_update\": \"2020-09-08T15:32:35.947589\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:36.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:36 smithi166 bash[9446]: audit 2020-09-08T15:32:35.956620+0000 mon.a (mon.0) 622 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:32:36.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:36 smithi166 bash[9446]: audit 2020-09-08T15:32:35.959149+0000 mon.b (mon.1) 33 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:32:36.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:36 smithi166 bash[9446]: audit 2020-09-08T15:32:35.961626+0000 mon.a (mon.0) 623 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:32:36.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:36 smithi166 bash[9446]: audit 2020-09-08T15:32:35.966500+0000 mon.a (mon.0) 624 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:32:36.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:36 smithi166 bash[9446]: audit 2020-09-08T15:32:35.969082+0000 mon.b (mon.1) 34 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:32:36.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:36 smithi166 bash[9446]: audit 2020-09-08T15:32:35.971682+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:32:36.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:36 smithi055 bash[9296]: cluster 2020-09-08T15:32:35.496864+0000 mgr.x (mgr.24118) 120 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:36.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:36 smithi055 bash[9296]: audit 2020-09-08T15:32:35.573021+0000 mon.a (mon.0) 618 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2020-09-08T15:32:36.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:36 smithi055 bash[9296]: cluster 2020-09-08T15:32:35.573120+0000 mon.a (mon.0) 619 : cluster [DBG] osdmap e48: 7 total, 6 up, 6 in 2020-09-08T15:32:36.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:36 smithi055 bash[9296]: audit 2020-09-08T15:32:35.580853+0000 mon.b (mon.1) 29 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:36.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:36 smithi055 bash[9296]: audit 2020-09-08T15:32:35.585217+0000 mon.b (mon.1) 30 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:36.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:36 smithi055 bash[9296]: audit 2020-09-08T15:32:35.665890+0000 mon.b (mon.1) 31 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:32:36.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:36 smithi055 bash[9296]: audit 2020-09-08T15:32:35.947505+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_daemon_update\": \"2020-09-08T15:32:27.563932\", \"last_device_update\": \"2020-09-08T15:32:35.947589\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:36.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:36 smithi055 bash[9296]: audit 2020-09-08T15:32:35.949603+0000 mon.b (mon.1) 32 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_daemon_update\": \"2020-09-08T15:32:27.563932\", \"last_device_update\": \"2020-09-08T15:32:35.947589\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:36.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:36 smithi055 bash[9296]: audit 2020-09-08T15:32:35.953506+0000 mon.a (mon.0) 621 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_daemon_update\": \"2020-09-08T15:32:27.563932\", \"last_device_update\": \"2020-09-08T15:32:35.947589\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:36.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:36 smithi055 bash[9296]: audit 2020-09-08T15:32:35.956620+0000 mon.a (mon.0) 622 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:32:36.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:36 smithi055 bash[9296]: audit 2020-09-08T15:32:35.959149+0000 mon.b (mon.1) 33 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:32:36.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:36 smithi055 bash[9296]: audit 2020-09-08T15:32:35.961626+0000 mon.a (mon.0) 623 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:32:36.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:36 smithi055 bash[9296]: audit 2020-09-08T15:32:35.966500+0000 mon.a (mon.0) 624 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:32:36.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:36 smithi055 bash[9296]: audit 2020-09-08T15:32:35.969082+0000 mon.b (mon.1) 34 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:32:36.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:36 smithi055 bash[9296]: audit 2020-09-08T15:32:35.971682+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:32:36.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:36 smithi104 bash[9203]: cluster 2020-09-08T15:32:35.496864+0000 mgr.x (mgr.24118) 120 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2020-09-08T15:32:36.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:36 smithi104 bash[9203]: audit 2020-09-08T15:32:35.573021+0000 mon.a (mon.0) 618 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2020-09-08T15:32:36.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:36 smithi104 bash[9203]: cluster 2020-09-08T15:32:35.573120+0000 mon.a (mon.0) 619 : cluster [DBG] osdmap e48: 7 total, 6 up, 6 in 2020-09-08T15:32:36.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:36 smithi104 bash[9203]: audit 2020-09-08T15:32:35.580853+0000 mon.b (mon.1) 29 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:36.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:36 smithi104 bash[9203]: audit 2020-09-08T15:32:35.585217+0000 mon.b (mon.1) 30 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:36.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:36 smithi104 bash[9203]: audit 2020-09-08T15:32:35.665890+0000 mon.b (mon.1) 31 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:32:36.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:36 smithi104 bash[9203]: audit 2020-09-08T15:32:35.947505+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_daemon_update\": \"2020-09-08T15:32:27.563932\", \"last_device_update\": \"2020-09-08T15:32:35.947589\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:36.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:36 smithi104 bash[9203]: audit 2020-09-08T15:32:35.949603+0000 mon.b (mon.1) 32 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_daemon_update\": \"2020-09-08T15:32:27.563932\", \"last_device_update\": \"2020-09-08T15:32:35.947589\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:36.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:36 smithi104 bash[9203]: audit 2020-09-08T15:32:35.953506+0000 mon.a (mon.0) 621 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}}, \"last_daemon_update\": \"2020-09-08T15:32:27.563932\", \"last_device_update\": \"2020-09-08T15:32:35.947589\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:36.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:36 smithi104 bash[9203]: audit 2020-09-08T15:32:35.956620+0000 mon.a (mon.0) 622 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:32:36.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:36 smithi104 bash[9203]: audit 2020-09-08T15:32:35.959149+0000 mon.b (mon.1) 33 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:32:36.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:36 smithi104 bash[9203]: audit 2020-09-08T15:32:35.961626+0000 mon.a (mon.0) 623 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:32:36.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:36 smithi104 bash[9203]: audit 2020-09-08T15:32:35.966500+0000 mon.a (mon.0) 624 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:32:36.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:36 smithi104 bash[9203]: audit 2020-09-08T15:32:35.969082+0000 mon.b (mon.1) 34 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:32:36.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:36 smithi104 bash[9203]: audit 2020-09-08T15:32:35.971682+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:32:37.859 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:37 smithi166 bash[9446]: cluster 2020-09-08T15:32:35.207943+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:37.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:37 smithi166 bash[9446]: cluster 2020-09-08T15:32:35.208125+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:37.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:37 smithi166 bash[9446]: cluster 2020-09-08T15:32:36.575023+0000 mon.a (mon.0) 626 : cluster [INF] osd.6 v1:172.21.15.55:6819/2501844793 boot 2020-09-08T15:32:37.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:37 smithi166 bash[9446]: cluster 2020-09-08T15:32:36.575197+0000 mon.a (mon.0) 627 : cluster [DBG] osdmap e49: 7 total, 7 up, 7 in 2020-09-08T15:32:37.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:37 smithi166 bash[9446]: audit 2020-09-08T15:32:36.582278+0000 mon.b (mon.1) 35 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:37.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:37 smithi166 bash[9446]: cluster 2020-09-08T15:32:37.497267+0000 mgr.x (mgr.24118) 121 : cluster [DBG] pgmap v124: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:37.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:37 smithi055 bash[9296]: cluster 2020-09-08T15:32:35.207943+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:37.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:37 smithi055 bash[9296]: cluster 2020-09-08T15:32:35.208125+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:37.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:37 smithi055 bash[9296]: cluster 2020-09-08T15:32:36.575023+0000 mon.a (mon.0) 626 : cluster [INF] osd.6 v1:172.21.15.55:6819/2501844793 boot 2020-09-08T15:32:37.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:37 smithi055 bash[9296]: cluster 2020-09-08T15:32:36.575197+0000 mon.a (mon.0) 627 : cluster [DBG] osdmap e49: 7 total, 7 up, 7 in 2020-09-08T15:32:37.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:37 smithi055 bash[9296]: audit 2020-09-08T15:32:36.582278+0000 mon.b (mon.1) 35 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:37.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:37 smithi055 bash[9296]: cluster 2020-09-08T15:32:37.497267+0000 mgr.x (mgr.24118) 121 : cluster [DBG] pgmap v124: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:37.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:37 smithi104 bash[9203]: cluster 2020-09-08T15:32:35.207943+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:37.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:37 smithi104 bash[9203]: cluster 2020-09-08T15:32:35.208125+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:37.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:37 smithi104 bash[9203]: cluster 2020-09-08T15:32:36.575023+0000 mon.a (mon.0) 626 : cluster [INF] osd.6 v1:172.21.15.55:6819/2501844793 boot 2020-09-08T15:32:37.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:37 smithi104 bash[9203]: cluster 2020-09-08T15:32:36.575197+0000 mon.a (mon.0) 627 : cluster [DBG] osdmap e49: 7 total, 7 up, 7 in 2020-09-08T15:32:37.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:37 smithi104 bash[9203]: audit 2020-09-08T15:32:36.582278+0000 mon.b (mon.1) 35 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2020-09-08T15:32:37.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:37 smithi104 bash[9203]: cluster 2020-09-08T15:32:37.497267+0000 mgr.x (mgr.24118) 121 : cluster [DBG] pgmap v124: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:38.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:38 smithi166 bash[9446]: cluster 2020-09-08T15:32:37.594266+0000 mon.a (mon.0) 628 : cluster [DBG] osdmap e50: 7 total, 7 up, 7 in 2020-09-08T15:32:38.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:38 smithi055 bash[9296]: cluster 2020-09-08T15:32:37.594266+0000 mon.a (mon.0) 628 : cluster [DBG] osdmap e50: 7 total, 7 up, 7 in 2020-09-08T15:32:38.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:38 smithi104 bash[9203]: cluster 2020-09-08T15:32:37.594266+0000 mon.a (mon.0) 628 : cluster [DBG] osdmap e50: 7 total, 7 up, 7 in 2020-09-08T15:32:39.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:39 smithi166 bash[9446]: audit 2020-09-08T15:32:39.062048+0000 mgr.x (mgr.24118) 122 : audit [DBG] from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:32:39.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:39 smithi166 bash[9446]: audit 2020-09-08T15:32:39.063178+0000 mon.b (mon.1) 36 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:32:39.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:39 smithi166 bash[9446]: cephadm 2020-09-08T15:32:39.063991+0000 mgr.x (mgr.24118) 123 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:32:39.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:39 smithi166 bash[9446]: cephadm 2020-09-08T15:32:39.064061+0000 mgr.x (mgr.24118) 124 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:32:39.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:39 smithi166 bash[9446]: cephadm 2020-09-08T15:32:39.064515+0000 mgr.x (mgr.24118) 125 : cephadm [INF] Applying None on host smithi055... 2020-09-08T15:32:39.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:39 smithi166 bash[9446]: audit 2020-09-08T15:32:39.064953+0000 mon.b (mon.1) 37 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:32:39.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:39 smithi166 bash[9446]: audit 2020-09-08T15:32:39.065999+0000 mon.b (mon.1) 38 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:32:39.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:39 smithi166 bash[9446]: audit 2020-09-08T15:32:39.067029+0000 mon.b (mon.1) 39 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:39.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:39 smithi166 bash[9446]: cluster 2020-09-08T15:32:39.497767+0000 mgr.x (mgr.24118) 126 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:39.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:39 smithi055 bash[9296]: audit 2020-09-08T15:32:39.062048+0000 mgr.x (mgr.24118) 122 : audit [DBG] from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:32:39.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:39 smithi055 bash[9296]: audit 2020-09-08T15:32:39.063178+0000 mon.b (mon.1) 36 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:32:39.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:39 smithi055 bash[9296]: cephadm 2020-09-08T15:32:39.063991+0000 mgr.x (mgr.24118) 123 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:32:39.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:39 smithi055 bash[9296]: cephadm 2020-09-08T15:32:39.064061+0000 mgr.x (mgr.24118) 124 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:32:39.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:39 smithi055 bash[9296]: cephadm 2020-09-08T15:32:39.064515+0000 mgr.x (mgr.24118) 125 : cephadm [INF] Applying None on host smithi055... 2020-09-08T15:32:39.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:39 smithi055 bash[9296]: audit 2020-09-08T15:32:39.064953+0000 mon.b (mon.1) 37 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:32:39.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:39 smithi055 bash[9296]: audit 2020-09-08T15:32:39.065999+0000 mon.b (mon.1) 38 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:32:39.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:39 smithi055 bash[9296]: audit 2020-09-08T15:32:39.067029+0000 mon.b (mon.1) 39 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:39.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:39 smithi055 bash[9296]: cluster 2020-09-08T15:32:39.497767+0000 mgr.x (mgr.24118) 126 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:39.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:39 smithi104 bash[9203]: audit 2020-09-08T15:32:39.062048+0000 mgr.x (mgr.24118) 122 : audit [DBG] from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:32:39.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:39 smithi104 bash[9203]: audit 2020-09-08T15:32:39.063178+0000 mon.b (mon.1) 36 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:32:39.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:39 smithi104 bash[9203]: cephadm 2020-09-08T15:32:39.063991+0000 mgr.x (mgr.24118) 123 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:32:39.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:39 smithi104 bash[9203]: cephadm 2020-09-08T15:32:39.064061+0000 mgr.x (mgr.24118) 124 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:32:39.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:39 smithi104 bash[9203]: cephadm 2020-09-08T15:32:39.064515+0000 mgr.x (mgr.24118) 125 : cephadm [INF] Applying None on host smithi055... 2020-09-08T15:32:39.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:39 smithi104 bash[9203]: audit 2020-09-08T15:32:39.064953+0000 mon.b (mon.1) 37 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:32:39.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:39 smithi104 bash[9203]: audit 2020-09-08T15:32:39.065999+0000 mon.b (mon.1) 38 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:32:39.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:39 smithi104 bash[9203]: audit 2020-09-08T15:32:39.067029+0000 mon.b (mon.1) 39 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:39.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:39 smithi104 bash[9203]: cluster 2020-09-08T15:32:39.497767+0000 mgr.x (mgr.24118) 126 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:42.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:42 smithi166 bash[9446]: cluster 2020-09-08T15:32:41.498406+0000 mgr.x (mgr.24118) 127 : cluster [DBG] pgmap v127: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:42.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:42 smithi055 bash[9296]: cluster 2020-09-08T15:32:41.498406+0000 mgr.x (mgr.24118) 127 : cluster [DBG] pgmap v127: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:42.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:42 smithi104 bash[9203]: cluster 2020-09-08T15:32:41.498406+0000 mgr.x (mgr.24118) 127 : cluster [DBG] pgmap v127: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:43.860 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:43 smithi166 bash[9446]: cluster 2020-09-08T15:32:42.516193+0000 osd.4 (osd.4) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:43.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:43 smithi166 bash[9446]: cluster 2020-09-08T15:32:42.516341+0000 osd.4 (osd.4) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:43.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:43 smithi166 bash[9446]: cluster 2020-09-08T15:32:42.558551+0000 mon.a (mon.0) 629 : cluster [DBG] osdmap e51: 7 total, 7 up, 7 in 2020-09-08T15:32:43.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:43 smithi166 bash[9446]: audit 2020-09-08T15:32:42.949925+0000 mon.a (mon.0) 630 : audit [INF] from='client.24334 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3795c86-4540-442c-a151-b2769fa577e4"}]: dispatch 2020-09-08T15:32:43.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:43 smithi166 bash[9446]: audit 2020-09-08T15:32:42.951663+0000 mon.b (mon.1) 40 : audit [INF] from='client.? v1:172.21.15.55:0/2880305680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3795c86-4540-442c-a151-b2769fa577e4"}]: dispatch 2020-09-08T15:32:43.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:43 smithi166 bash[9446]: audit 2020-09-08T15:32:42.958324+0000 mon.a (mon.0) 631 : audit [INF] from='client.24334 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3795c86-4540-442c-a151-b2769fa577e4"}]': finished 2020-09-08T15:32:43.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:43 smithi166 bash[9446]: cluster 2020-09-08T15:32:42.958424+0000 mon.a (mon.0) 632 : cluster [DBG] osdmap e52: 8 total, 7 up, 7 in 2020-09-08T15:32:43.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:43 smithi166 bash[9446]: audit 2020-09-08T15:32:42.961976+0000 mon.b (mon.1) 41 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2020-09-08T15:32:43.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:43 smithi166 bash[9446]: cluster 2020-09-08T15:32:43.498950+0000 mgr.x (mgr.24118) 128 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:43.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:43 smithi055 bash[9296]: cluster 2020-09-08T15:32:42.516193+0000 osd.4 (osd.4) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:43.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:43 smithi055 bash[9296]: cluster 2020-09-08T15:32:42.516341+0000 osd.4 (osd.4) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:43.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:43 smithi055 bash[9296]: cluster 2020-09-08T15:32:42.558551+0000 mon.a (mon.0) 629 : cluster [DBG] osdmap e51: 7 total, 7 up, 7 in 2020-09-08T15:32:43.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:43 smithi055 bash[9296]: audit 2020-09-08T15:32:42.949925+0000 mon.a (mon.0) 630 : audit [INF] from='client.24334 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3795c86-4540-442c-a151-b2769fa577e4"}]: dispatch 2020-09-08T15:32:43.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:43 smithi055 bash[9296]: audit 2020-09-08T15:32:42.951663+0000 mon.b (mon.1) 40 : audit [INF] from='client.? v1:172.21.15.55:0/2880305680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3795c86-4540-442c-a151-b2769fa577e4"}]: dispatch 2020-09-08T15:32:43.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:43 smithi055 bash[9296]: audit 2020-09-08T15:32:42.958324+0000 mon.a (mon.0) 631 : audit [INF] from='client.24334 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3795c86-4540-442c-a151-b2769fa577e4"}]': finished 2020-09-08T15:32:43.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:43 smithi055 bash[9296]: cluster 2020-09-08T15:32:42.958424+0000 mon.a (mon.0) 632 : cluster [DBG] osdmap e52: 8 total, 7 up, 7 in 2020-09-08T15:32:43.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:43 smithi055 bash[9296]: audit 2020-09-08T15:32:42.961976+0000 mon.b (mon.1) 41 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2020-09-08T15:32:43.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:43 smithi055 bash[9296]: cluster 2020-09-08T15:32:43.498950+0000 mgr.x (mgr.24118) 128 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:43.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:43 smithi104 bash[9203]: cluster 2020-09-08T15:32:42.516193+0000 osd.4 (osd.4) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:43.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:43 smithi104 bash[9203]: cluster 2020-09-08T15:32:42.516341+0000 osd.4 (osd.4) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:43.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:43 smithi104 bash[9203]: cluster 2020-09-08T15:32:42.558551+0000 mon.a (mon.0) 629 : cluster [DBG] osdmap e51: 7 total, 7 up, 7 in 2020-09-08T15:32:43.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:43 smithi104 bash[9203]: audit 2020-09-08T15:32:42.949925+0000 mon.a (mon.0) 630 : audit [INF] from='client.24334 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3795c86-4540-442c-a151-b2769fa577e4"}]: dispatch 2020-09-08T15:32:43.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:43 smithi104 bash[9203]: audit 2020-09-08T15:32:42.951663+0000 mon.b (mon.1) 40 : audit [INF] from='client.? v1:172.21.15.55:0/2880305680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3795c86-4540-442c-a151-b2769fa577e4"}]: dispatch 2020-09-08T15:32:43.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:43 smithi104 bash[9203]: audit 2020-09-08T15:32:42.958324+0000 mon.a (mon.0) 631 : audit [INF] from='client.24334 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3795c86-4540-442c-a151-b2769fa577e4"}]': finished 2020-09-08T15:32:43.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:43 smithi104 bash[9203]: cluster 2020-09-08T15:32:42.958424+0000 mon.a (mon.0) 632 : cluster [DBG] osdmap e52: 8 total, 7 up, 7 in 2020-09-08T15:32:43.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:43 smithi104 bash[9203]: audit 2020-09-08T15:32:42.961976+0000 mon.b (mon.1) 41 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2020-09-08T15:32:43.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:43 smithi104 bash[9203]: cluster 2020-09-08T15:32:43.498950+0000 mgr.x (mgr.24118) 128 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:44.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:44 smithi166 bash[9446]: audit 2020-09-08T15:32:43.604926+0000 mon.b (mon.1) 42 : audit [DBG] from='client.? v1:172.21.15.55:0/3973796461' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:32:44.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:44 smithi055 bash[9296]: audit 2020-09-08T15:32:43.604926+0000 mon.b (mon.1) 42 : audit [DBG] from='client.? v1:172.21.15.55:0/3973796461' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:32:44.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:44 smithi104 bash[9203]: audit 2020-09-08T15:32:43.604926+0000 mon.b (mon.1) 42 : audit [DBG] from='client.? v1:172.21.15.55:0/3973796461' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:32:45.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:45 smithi166 bash[9446]: cluster 2020-09-08T15:32:45.499589+0000 mgr.x (mgr.24118) 129 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:45.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:45 smithi055 bash[9296]: cluster 2020-09-08T15:32:45.499589+0000 mgr.x (mgr.24118) 129 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:45.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:45 smithi104 bash[9203]: cluster 2020-09-08T15:32:45.499589+0000 mgr.x (mgr.24118) 129 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:46.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:46 smithi055 bash[9296]: audit 2020-09-08T15:32:46.578805+0000 mon.b (mon.1) 43 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:46.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:46 smithi104 bash[9203]: audit 2020-09-08T15:32:46.578805+0000 mon.b (mon.1) 43 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:47.111 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:46 smithi166 bash[9446]: audit 2020-09-08T15:32:46.578805+0000 mon.b (mon.1) 43 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:47.897 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:47 smithi055 bash[9296]: cluster 2020-09-08T15:32:47.500013+0000 mgr.x (mgr.24118) 130 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:47.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:47 smithi104 bash[9203]: cluster 2020-09-08T15:32:47.500013+0000 mgr.x (mgr.24118) 130 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:48.111 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:47 smithi166 bash[9446]: cluster 2020-09-08T15:32:47.500013+0000 mgr.x (mgr.24118) 130 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:50.654 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:50 smithi055 bash[9296]: cluster 2020-09-08T15:32:49.500385+0000 mgr.x (mgr.24118) 131 : cluster [DBG] pgmap v133: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:50.861 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:50 smithi166 bash[9446]: cluster 2020-09-08T15:32:49.500385+0000 mgr.x (mgr.24118) 131 : cluster [DBG] pgmap v133: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:50.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:50 smithi104 bash[9203]: cluster 2020-09-08T15:32:49.500385+0000 mgr.x (mgr.24118) 131 : cluster [DBG] pgmap v133: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:51.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:51 smithi166 bash[9446]: audit 2020-09-08T15:32:51.002344+0000 mon.b (mon.1) 44 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2020-09-08T15:32:51.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:51 smithi166 bash[9446]: audit 2020-09-08T15:32:51.003432+0000 mon.b (mon.1) 45 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:32:51.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:51 smithi166 bash[9446]: audit 2020-09-08T15:32:51.004626+0000 mon.b (mon.1) 46 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.7", "key": "container_image"}]: dispatch 2020-09-08T15:32:51.904 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:51 smithi055 bash[9296]: audit 2020-09-08T15:32:51.002344+0000 mon.b (mon.1) 44 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2020-09-08T15:32:51.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:51 smithi055 bash[9296]: audit 2020-09-08T15:32:51.003432+0000 mon.b (mon.1) 45 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:32:51.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:51 smithi055 bash[9296]: audit 2020-09-08T15:32:51.004626+0000 mon.b (mon.1) 46 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.7", "key": "container_image"}]: dispatch 2020-09-08T15:32:51.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:51 smithi104 bash[9203]: audit 2020-09-08T15:32:51.002344+0000 mon.b (mon.1) 44 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2020-09-08T15:32:51.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:51 smithi104 bash[9203]: audit 2020-09-08T15:32:51.003432+0000 mon.b (mon.1) 45 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:32:51.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:51 smithi104 bash[9203]: audit 2020-09-08T15:32:51.004626+0000 mon.b (mon.1) 46 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.7", "key": "container_image"}]: dispatch 2020-09-08T15:32:52.545 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:52 smithi055 bash[9296]: cephadm 2020-09-08T15:32:51.004274+0000 mgr.x (mgr.24118) 132 : cephadm [INF] Deploying daemon osd.7 on smithi055 2020-09-08T15:32:52.545 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:52 smithi055 bash[9296]: cluster 2020-09-08T15:32:51.500869+0000 mgr.x (mgr.24118) 133 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:52.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:52 smithi166 bash[9446]: cephadm 2020-09-08T15:32:51.004274+0000 mgr.x (mgr.24118) 132 : cephadm [INF] Deploying daemon osd.7 on smithi055 2020-09-08T15:32:52.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:52 smithi166 bash[9446]: cluster 2020-09-08T15:32:51.500869+0000 mgr.x (mgr.24118) 133 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:52.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:52 smithi104 bash[9203]: cephadm 2020-09-08T15:32:51.004274+0000 mgr.x (mgr.24118) 132 : cephadm [INF] Deploying daemon osd.7 on smithi055 2020-09-08T15:32:52.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:52 smithi104 bash[9203]: cluster 2020-09-08T15:32:51.500869+0000 mgr.x (mgr.24118) 133 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:53.389 INFO:teuthology.orchestra.run.smithi055.stdout:Created osd(s) 7 on host 'smithi055' 2020-09-08T15:32:53.655 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:53 smithi055 bash[9296]: audit 2020-09-08T15:32:53.386107+0000 mon.a (mon.0) 633 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_device_update\": \"2020-09-08T15:32:35.947589\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:53.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:53 smithi055 bash[9296]: audit 2020-09-08T15:32:53.387130+0000 mon.b (mon.1) 47 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_device_update\": \"2020-09-08T15:32:35.947589\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:53.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:53 smithi055 bash[9296]: audit 2020-09-08T15:32:53.387738+0000 mon.b (mon.1) 48 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:32:53.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:53 smithi055 bash[9296]: audit 2020-09-08T15:32:53.391422+0000 mon.a (mon.0) 634 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_device_update\": \"2020-09-08T15:32:35.947589\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:53.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:53 smithi166 bash[9446]: audit 2020-09-08T15:32:53.386107+0000 mon.a (mon.0) 633 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_device_update\": \"2020-09-08T15:32:35.947589\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:53.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:53 smithi166 bash[9446]: audit 2020-09-08T15:32:53.387130+0000 mon.b (mon.1) 47 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_device_update\": \"2020-09-08T15:32:35.947589\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:53.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:53 smithi166 bash[9446]: audit 2020-09-08T15:32:53.387738+0000 mon.b (mon.1) 48 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:32:53.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:53 smithi166 bash[9446]: audit 2020-09-08T15:32:53.391422+0000 mon.a (mon.0) 634 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_device_update\": \"2020-09-08T15:32:35.947589\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:53.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:53 smithi104 bash[9203]: audit 2020-09-08T15:32:53.386107+0000 mon.a (mon.0) 633 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_device_update\": \"2020-09-08T15:32:35.947589\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:53.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:53 smithi104 bash[9203]: audit 2020-09-08T15:32:53.387130+0000 mon.b (mon.1) 47 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_device_update\": \"2020-09-08T15:32:35.947589\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:53.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:53 smithi104 bash[9203]: audit 2020-09-08T15:32:53.387738+0000 mon.b (mon.1) 48 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:32:53.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:53 smithi104 bash[9203]: audit 2020-09-08T15:32:53.391422+0000 mon.a (mon.0) 634 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563269\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:27.563370\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563420\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563618\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:27.563756\", \"created\": \"2020-09-08T15:32:26.730148\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_device_update\": \"2020-09-08T15:32:35.947589\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:54.827 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:54 smithi055 bash[9296]: cluster 2020-09-08T15:32:53.501214+0000 mgr.x (mgr.24118) 134 : cluster [DBG] pgmap v135: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:54.827 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:54 smithi055 bash[9296]: audit 2020-09-08T15:32:54.269721+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:54.828 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:54 smithi055 bash[9296]: audit 2020-09-08T15:32:54.270611+0000 mon.b (mon.1) 49 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:54.828 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:54 smithi055 bash[9296]: audit 2020-09-08T15:32:54.275619+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:54.829 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:54 smithi055 bash[9296]: audit 2020-09-08T15:32:54.279844+0000 mon.b (mon.1) 50 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:54.832 INFO:teuthology.orchestra.run.smithi055:osd.7> sudo journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.7.service 2020-09-08T15:32:54.836 INFO:tasks.cephadm:Deploying osd.8 on smithi166 with /dev/vg_nvme/lv_4... 2020-09-08T15:32:54.837 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2020-09-08T15:32:54.847 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:54 smithi166 bash[9446]: cluster 2020-09-08T15:32:53.501214+0000 mgr.x (mgr.24118) 134 : cluster [DBG] pgmap v135: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:54.848 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:54 smithi166 bash[9446]: audit 2020-09-08T15:32:54.269721+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:54.848 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:54 smithi166 bash[9446]: audit 2020-09-08T15:32:54.270611+0000 mon.b (mon.1) 49 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:54.849 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:54 smithi166 bash[9446]: audit 2020-09-08T15:32:54.275619+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:54.849 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:54 smithi166 bash[9446]: audit 2020-09-08T15:32:54.279844+0000 mon.b (mon.1) 50 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:54.865 INFO:journalctl@ceph.osd.7.smithi055.stdout:-- Logs begin at Tue 2020-09-01 16:49:30 UTC. -- 2020-09-08T15:32:54.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:54 smithi104 bash[9203]: cluster 2020-09-08T15:32:53.501214+0000 mgr.x (mgr.24118) 134 : cluster [DBG] pgmap v135: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:54.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:54 smithi104 bash[9203]: audit 2020-09-08T15:32:54.269721+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:54.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:54 smithi104 bash[9203]: audit 2020-09-08T15:32:54.270611+0000 mon.b (mon.1) 49 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:32:54.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:54 smithi104 bash[9203]: audit 2020-09-08T15:32:54.275619+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:32:54.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:54 smithi104 bash[9203]: audit 2020-09-08T15:32:54.279844+0000 mon.b (mon.1) 50 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:32:55.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:55 smithi055 bash[9296]: audit 2020-09-08T15:32:54.681923+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:32:55.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:55 smithi055 bash[9296]: audit 2020-09-08T15:32:54.683082+0000 mon.b (mon.1) 51 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:32:55.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:55 smithi055 bash[9296]: audit 2020-09-08T15:32:54.688329+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:32:55.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:55 smithi055 bash[9296]: cluster 2020-09-08T15:32:54.903119+0000 osd.2 (osd.2) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:55.660 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:55 smithi055 bash[9296]: cluster 2020-09-08T15:32:54.903281+0000 osd.2 (osd.2) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:55.833 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:55 smithi166 bash[9446]: audit 2020-09-08T15:32:54.681923+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:32:55.834 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:55 smithi166 bash[9446]: audit 2020-09-08T15:32:54.683082+0000 mon.b (mon.1) 51 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:32:55.834 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:55 smithi166 bash[9446]: audit 2020-09-08T15:32:54.688329+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:32:55.834 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:55 smithi166 bash[9446]: cluster 2020-09-08T15:32:54.903119+0000 osd.2 (osd.2) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:55.835 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:55 smithi166 bash[9446]: cluster 2020-09-08T15:32:54.903281+0000 osd.2 (osd.2) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:55.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:55 smithi104 bash[9203]: audit 2020-09-08T15:32:54.681923+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:32:55.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:55 smithi104 bash[9203]: audit 2020-09-08T15:32:54.683082+0000 mon.b (mon.1) 51 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:32:55.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:55 smithi104 bash[9203]: audit 2020-09-08T15:32:54.688329+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:32:55.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:55 smithi104 bash[9203]: cluster 2020-09-08T15:32:54.903119+0000 osd.2 (osd.2) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:32:55.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:55 smithi104 bash[9203]: cluster 2020-09-08T15:32:54.903281+0000 osd.2 (osd.2) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:32:56.155 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:32:55 smithi055 bash[19003]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2020-09-08T15:32:56.155 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:32:55 smithi055 bash[19003]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_1 --path /var/lib/ceph/osd/ceph-7 --no-mon-config 2020-09-08T15:32:56.155 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:32:55 smithi055 bash[19003]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-7/block 2020-09-08T15:32:56.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:32:55 smithi055 bash[19003]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-7/block 2020-09-08T15:32:56.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:32:55 smithi055 bash[19003]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2020-09-08T15:32:56.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:32:55 smithi055 bash[19003]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2020-09-08T15:32:56.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:32:55 smithi055 bash[19003]: --> ceph-volume lvm activate successful for osd ID: 7 2020-09-08T15:32:56.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:56 smithi166 bash[9446]: cluster 2020-09-08T15:32:55.501583+0000 mgr.x (mgr.24118) 135 : cluster [DBG] pgmap v136: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:56.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:56 smithi166 bash[9446]: cluster 2020-09-08T15:32:55.698762+0000 mon.a (mon.0) 639 : cluster [DBG] osdmap e53: 8 total, 7 up, 7 in 2020-09-08T15:32:56.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:56 smithi166 bash[9446]: audit 2020-09-08T15:32:55.706153+0000 mon.b (mon.1) 52 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2020-09-08T15:32:56.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:56 smithi055 bash[9296]: cluster 2020-09-08T15:32:55.501583+0000 mgr.x (mgr.24118) 135 : cluster [DBG] pgmap v136: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:56.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:56 smithi055 bash[9296]: cluster 2020-09-08T15:32:55.698762+0000 mon.a (mon.0) 639 : cluster [DBG] osdmap e53: 8 total, 7 up, 7 in 2020-09-08T15:32:56.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:56 smithi055 bash[9296]: audit 2020-09-08T15:32:55.706153+0000 mon.b (mon.1) 52 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2020-09-08T15:32:56.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:56 smithi104 bash[9203]: cluster 2020-09-08T15:32:55.501583+0000 mgr.x (mgr.24118) 135 : cluster [DBG] pgmap v136: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:56.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:56 smithi104 bash[9203]: cluster 2020-09-08T15:32:55.698762+0000 mon.a (mon.0) 639 : cluster [DBG] osdmap e53: 8 total, 7 up, 7 in 2020-09-08T15:32:56.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:56 smithi104 bash[9203]: audit 2020-09-08T15:32:55.706153+0000 mon.b (mon.1) 52 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2020-09-08T15:32:57.409 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:32:57 smithi055 bash[19003]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:32:58.209 INFO:teuthology.orchestra.run.smithi166.stderr:--> Zapping: /dev/vg_nvme/lv_4 2020-09-08T15:32:58.210 INFO:teuthology.orchestra.run.smithi166.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2020-09-08T15:32:58.210 INFO:teuthology.orchestra.run.smithi166.stderr: stderr: 10+0 records in 2020-09-08T15:32:58.210 INFO:teuthology.orchestra.run.smithi166.stderr:10+0 records out 2020-09-08T15:32:58.210 INFO:teuthology.orchestra.run.smithi166.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0177205 s, 592 MB/s 2020-09-08T15:32:58.211 INFO:teuthology.orchestra.run.smithi166.stderr:--> Zapping successful for: 2020-09-08T15:32:58.612 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:32:58 smithi166 bash[9446]: cluster 2020-09-08T15:32:57.502076+0000 mgr.x (mgr.24118) 136 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:58.655 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:32:58 smithi055 bash[9296]: cluster 2020-09-08T15:32:57.502076+0000 mgr.x (mgr.24118) 136 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:58.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:32:58 smithi055 bash[19003]: debug 2020-09-08T15:32:58.466+0000 7f9bb0956700 10 -- 172.21.15.55:0/2040994509 dispatch_throttle_release 278 to dispatch throttler 472/104857600 2020-09-08T15:32:58.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:32:58 smithi055 bash[19003]: debug 2020-09-08T15:32:58.466+0000 7f9bb0956700 20 -- 172.21.15.55:0/2040994509 done calling dispatch on 0x55f2f720ac60 2020-09-08T15:32:58.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:32:58 smithi055 bash[19003]: debug 2020-09-08T15:32:58.466+0000 7f9bb0956700 1 -- 172.21.15.55:0/2040994509 <== mon.1 v1:172.21.15.55:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 1349361734 0 0) 0x55f2f72144e0 con 0x55f2f728b400 2020-09-08T15:32:58.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:32:58 smithi055 bash[19003]: debug 2020-09-08T15:32:58.466+0000 7f9bb0956700 10 -- 172.21.15.55:0/2040994509 dispatch_throttle_release 194 to dispatch throttler 194/104857600 2020-09-08T15:32:58.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:32:58 smithi055 bash[19003]: debug 2020-09-08T15:32:58.466+0000 7f9bb0956700 20 -- 172.21.15.55:0/2040994509 done calling dispatch on 0x55f2f72144e0 2020-09-08T15:32:58.819 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph orch daemon add osd smithi166:vg_nvme/lv_4 2020-09-08T15:32:58.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:32:58 smithi104 bash[9203]: cluster 2020-09-08T15:32:57.502076+0000 mgr.x (mgr.24118) 136 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:32:59.405 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:32:59 smithi055 bash[19003]: debug 2020-09-08T15:32:59.010+0000 7f9bb83d6f40 -1 Falling back to public interface 2020-09-08T15:33:00.155 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:32:59 smithi055 bash[19003]: debug 2020-09-08T15:32:59.674+0000 7f9bb83d6f40 -1 osd.7 0 log_to_monitors {default=true} 2020-09-08T15:33:00.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:00 smithi166 bash[9446]: cluster 2020-09-08T15:32:59.502658+0000 mgr.x (mgr.24118) 137 : cluster [DBG] pgmap v139: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:33:00.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:00 smithi166 bash[9446]: audit 2020-09-08T15:32:59.684248+0000 mon.a (mon.0) 640 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2020-09-08T15:33:00.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:00 smithi166 bash[9446]: audit 2020-09-08T15:32:59.685462+0000 mon.c (mon.2) 7 : audit [INF] from='osd.7 v1:172.21.15.55:6827/128909153' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2020-09-08T15:33:00.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:00 smithi055 bash[9296]: cluster 2020-09-08T15:32:59.502658+0000 mgr.x (mgr.24118) 137 : cluster [DBG] pgmap v139: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:33:00.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:00 smithi055 bash[9296]: audit 2020-09-08T15:32:59.684248+0000 mon.a (mon.0) 640 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2020-09-08T15:33:00.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:00 smithi055 bash[9296]: audit 2020-09-08T15:32:59.685462+0000 mon.c (mon.2) 7 : audit [INF] from='osd.7 v1:172.21.15.55:6827/128909153' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2020-09-08T15:33:00.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:00 smithi104 bash[9203]: cluster 2020-09-08T15:32:59.502658+0000 mgr.x (mgr.24118) 137 : cluster [DBG] pgmap v139: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:33:00.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:00 smithi104 bash[9203]: audit 2020-09-08T15:32:59.684248+0000 mon.a (mon.0) 640 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2020-09-08T15:33:00.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:00 smithi104 bash[9203]: audit 2020-09-08T15:32:59.685462+0000 mon.c (mon.2) 7 : audit [INF] from='osd.7 v1:172.21.15.55:6827/128909153' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2020-09-08T15:33:01.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:01 smithi166 bash[9446]: audit 2020-09-08T15:33:00.496987+0000 mon.a (mon.0) 641 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2020-09-08T15:33:01.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:01 smithi166 bash[9446]: cluster 2020-09-08T15:33:00.497106+0000 mon.a (mon.0) 642 : cluster [DBG] osdmap e54: 8 total, 7 up, 7 in 2020-09-08T15:33:01.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:01 smithi166 bash[9446]: audit 2020-09-08T15:33:00.499375+0000 mon.a (mon.0) 643 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:33:01.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:01 smithi166 bash[9446]: audit 2020-09-08T15:33:00.500807+0000 mon.c (mon.2) 8 : audit [INF] from='osd.7 v1:172.21.15.55:6827/128909153' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:33:01.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:01 smithi166 bash[9446]: audit 2020-09-08T15:33:00.504967+0000 mon.b (mon.1) 53 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2020-09-08T15:33:01.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:01 smithi166 bash[9446]: audit 2020-09-08T15:33:00.573618+0000 mon.b (mon.1) 54 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:33:01.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:01 smithi166 bash[9446]: audit 2020-09-08T15:33:00.861880+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:33:01.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:01 smithi166 bash[9446]: audit 2020-09-08T15:33:00.862360+0000 mon.b (mon.1) 55 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:33:01.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:01 smithi166 bash[9446]: audit 2020-09-08T15:33:00.867952+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:33:01.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:01 smithi166 bash[9446]: audit 2020-09-08T15:33:00.871429+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:01.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:01 smithi166 bash[9446]: audit 2020-09-08T15:33:00.872466+0000 mon.b (mon.1) 56 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:01.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:01 smithi166 bash[9446]: audit 2020-09-08T15:33:00.876326+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:33:01.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:01 smithi166 bash[9446]: audit 2020-09-08T15:33:00.883944+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:01.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:01 smithi166 bash[9446]: audit 2020-09-08T15:33:00.884964+0000 mon.b (mon.1) 57 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:01.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:01 smithi166 bash[9446]: audit 2020-09-08T15:33:00.888792+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:33:01.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:33:01 smithi055 bash[19003]: debug 2020-09-08T15:33:01.514+0000 7f9bae952700 -1 osd.7 0 waiting for initial osdmap 2020-09-08T15:33:01.917 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:01 smithi055 bash[9296]: audit 2020-09-08T15:33:00.496987+0000 mon.a (mon.0) 641 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2020-09-08T15:33:01.917 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:01 smithi055 bash[9296]: cluster 2020-09-08T15:33:00.497106+0000 mon.a (mon.0) 642 : cluster [DBG] osdmap e54: 8 total, 7 up, 7 in 2020-09-08T15:33:01.923 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:01 smithi055 bash[9296]: audit 2020-09-08T15:33:00.499375+0000 mon.a (mon.0) 643 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:33:01.924 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:01 smithi055 bash[9296]: audit 2020-09-08T15:33:00.500807+0000 mon.c (mon.2) 8 : audit [INF] from='osd.7 v1:172.21.15.55:6827/128909153' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:33:01.924 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:01 smithi055 bash[9296]: audit 2020-09-08T15:33:00.504967+0000 mon.b (mon.1) 53 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2020-09-08T15:33:01.924 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:01 smithi055 bash[9296]: audit 2020-09-08T15:33:00.573618+0000 mon.b (mon.1) 54 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:33:01.925 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:01 smithi055 bash[9296]: audit 2020-09-08T15:33:00.861880+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:33:01.926 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:01 smithi055 bash[9296]: audit 2020-09-08T15:33:00.862360+0000 mon.b (mon.1) 55 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:33:01.926 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:01 smithi055 bash[9296]: audit 2020-09-08T15:33:00.867952+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:33:01.934 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:01 smithi055 bash[9296]: audit 2020-09-08T15:33:00.871429+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:01.935 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:01 smithi055 bash[9296]: audit 2020-09-08T15:33:00.872466+0000 mon.b (mon.1) 56 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:01.935 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:01 smithi055 bash[9296]: audit 2020-09-08T15:33:00.876326+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:33:01.935 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:01 smithi055 bash[9296]: audit 2020-09-08T15:33:00.883944+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:01.935 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:01 smithi055 bash[9296]: audit 2020-09-08T15:33:00.884964+0000 mon.b (mon.1) 57 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:01.936 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:01 smithi055 bash[9296]: audit 2020-09-08T15:33:00.888792+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:33:01.937 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:01 smithi104 bash[9203]: audit 2020-09-08T15:33:00.496987+0000 mon.a (mon.0) 641 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2020-09-08T15:33:01.937 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:01 smithi104 bash[9203]: cluster 2020-09-08T15:33:00.497106+0000 mon.a (mon.0) 642 : cluster [DBG] osdmap e54: 8 total, 7 up, 7 in 2020-09-08T15:33:01.938 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:01 smithi104 bash[9203]: audit 2020-09-08T15:33:00.499375+0000 mon.a (mon.0) 643 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:33:01.938 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:01 smithi104 bash[9203]: audit 2020-09-08T15:33:00.500807+0000 mon.c (mon.2) 8 : audit [INF] from='osd.7 v1:172.21.15.55:6827/128909153' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2020-09-08T15:33:01.938 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:01 smithi104 bash[9203]: audit 2020-09-08T15:33:00.504967+0000 mon.b (mon.1) 53 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2020-09-08T15:33:01.938 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:01 smithi104 bash[9203]: audit 2020-09-08T15:33:00.573618+0000 mon.b (mon.1) 54 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:33:01.939 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:01 smithi104 bash[9203]: audit 2020-09-08T15:33:00.861880+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:33:01.939 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:01 smithi104 bash[9203]: audit 2020-09-08T15:33:00.862360+0000 mon.b (mon.1) 55 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]: dispatch 2020-09-08T15:33:01.939 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:01 smithi104 bash[9203]: audit 2020-09-08T15:33:00.867952+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:27:21.493911\"}"}]': finished 2020-09-08T15:33:01.940 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:01 smithi104 bash[9203]: audit 2020-09-08T15:33:00.871429+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:01.940 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:01 smithi104 bash[9203]: audit 2020-09-08T15:33:00.872466+0000 mon.b (mon.1) 56 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:01.940 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:01 smithi104 bash[9203]: audit 2020-09-08T15:33:00.876326+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:33:01.940 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:01 smithi104 bash[9203]: audit 2020-09-08T15:33:00.883944+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:01.941 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:01 smithi104 bash[9203]: audit 2020-09-08T15:33:00.884964+0000 mon.b (mon.1) 57 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:01.941 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:01 smithi104 bash[9203]: audit 2020-09-08T15:33:00.888792+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:33:02.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:02 smithi166 bash[9446]: audit 2020-09-08T15:33:01.499013+0000 mon.a (mon.0) 650 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2020-09-08T15:33:02.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:02 smithi166 bash[9446]: cluster 2020-09-08T15:33:01.499110+0000 mon.a (mon.0) 651 : cluster [DBG] osdmap e55: 8 total, 7 up, 7 in 2020-09-08T15:33:02.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:02 smithi166 bash[9446]: cluster 2020-09-08T15:33:01.503220+0000 mgr.x (mgr.24118) 138 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:33:02.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:02 smithi166 bash[9446]: audit 2020-09-08T15:33:01.511097+0000 mon.b (mon.1) 58 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2020-09-08T15:33:02.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:02 smithi166 bash[9446]: audit 2020-09-08T15:33:01.952213+0000 mon.b (mon.1) 59 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:33:02.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:02 smithi166 bash[9446]: audit 2020-09-08T15:33:01.955979+0000 mon.b (mon.1) 60 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:33:02.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:02 smithi166 bash[9446]: audit 2020-09-08T15:33:01.957472+0000 mon.b (mon.1) 61 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:33:02.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:02 smithi166 bash[9446]: audit 2020-09-08T15:33:01.959160+0000 mon.b (mon.1) 62 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:02.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:02 smithi166 bash[9446]: audit 2020-09-08T15:33:02.506744+0000 mon.b (mon.1) 63 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2020-09-08T15:33:02.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:02 smithi055 bash[9296]: audit 2020-09-08T15:33:01.499013+0000 mon.a (mon.0) 650 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2020-09-08T15:33:02.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:02 smithi055 bash[9296]: cluster 2020-09-08T15:33:01.499110+0000 mon.a (mon.0) 651 : cluster [DBG] osdmap e55: 8 total, 7 up, 7 in 2020-09-08T15:33:02.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:02 smithi055 bash[9296]: cluster 2020-09-08T15:33:01.503220+0000 mgr.x (mgr.24118) 138 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:33:02.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:02 smithi055 bash[9296]: audit 2020-09-08T15:33:01.511097+0000 mon.b (mon.1) 58 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2020-09-08T15:33:02.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:02 smithi055 bash[9296]: audit 2020-09-08T15:33:01.952213+0000 mon.b (mon.1) 59 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:33:02.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:02 smithi055 bash[9296]: audit 2020-09-08T15:33:01.955979+0000 mon.b (mon.1) 60 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:33:02.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:02 smithi055 bash[9296]: audit 2020-09-08T15:33:01.957472+0000 mon.b (mon.1) 61 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:33:02.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:02 smithi055 bash[9296]: audit 2020-09-08T15:33:01.959160+0000 mon.b (mon.1) 62 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:02.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:02 smithi055 bash[9296]: audit 2020-09-08T15:33:02.506744+0000 mon.b (mon.1) 63 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2020-09-08T15:33:02.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:02 smithi104 bash[9203]: audit 2020-09-08T15:33:01.499013+0000 mon.a (mon.0) 650 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2020-09-08T15:33:02.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:02 smithi104 bash[9203]: cluster 2020-09-08T15:33:01.499110+0000 mon.a (mon.0) 651 : cluster [DBG] osdmap e55: 8 total, 7 up, 7 in 2020-09-08T15:33:02.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:02 smithi104 bash[9203]: cluster 2020-09-08T15:33:01.503220+0000 mgr.x (mgr.24118) 138 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:33:02.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:02 smithi104 bash[9203]: audit 2020-09-08T15:33:01.511097+0000 mon.b (mon.1) 58 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2020-09-08T15:33:02.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:02 smithi104 bash[9203]: audit 2020-09-08T15:33:01.952213+0000 mon.b (mon.1) 59 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:33:02.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:02 smithi104 bash[9203]: audit 2020-09-08T15:33:01.955979+0000 mon.b (mon.1) 60 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:33:02.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:02 smithi104 bash[9203]: audit 2020-09-08T15:33:01.957472+0000 mon.b (mon.1) 61 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:33:02.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:02 smithi104 bash[9203]: audit 2020-09-08T15:33:01.959160+0000 mon.b (mon.1) 62 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:02.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:02 smithi104 bash[9203]: audit 2020-09-08T15:33:02.506744+0000 mon.b (mon.1) 63 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2020-09-08T15:33:03.581 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:03 smithi166 bash[9446]: cluster 2020-09-08T15:33:00.706762+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:03.581 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:03 smithi166 bash[9446]: cluster 2020-09-08T15:33:00.706922+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:03.582 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:03 smithi166 bash[9446]: audit 2020-09-08T15:33:01.950195+0000 mgr.x (mgr.24118) 139 : audit [DBG] from='client.34349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi166:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:33:03.582 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:03 smithi166 bash[9446]: cephadm 2020-09-08T15:33:01.953571+0000 mgr.x (mgr.24118) 140 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:33:03.582 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:03 smithi166 bash[9446]: cephadm 2020-09-08T15:33:01.953752+0000 mgr.x (mgr.24118) 141 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:33:03.582 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:03 smithi166 bash[9446]: cephadm 2020-09-08T15:33:01.955168+0000 mgr.x (mgr.24118) 142 : cephadm [INF] Applying None on host smithi166... 2020-09-08T15:33:03.583 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:03 smithi166 bash[9446]: cluster 2020-09-08T15:33:02.527195+0000 mon.a (mon.0) 652 : cluster [INF] osd.7 v1:172.21.15.55:6827/128909153 boot 2020-09-08T15:33:03.583 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:03 smithi166 bash[9446]: cluster 2020-09-08T15:33:02.527253+0000 mon.a (mon.0) 653 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2020-09-08T15:33:03.583 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:03 smithi166 bash[9446]: audit 2020-09-08T15:33:02.537168+0000 mon.b (mon.1) 64 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2020-09-08T15:33:03.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:03 smithi055 bash[9296]: cluster 2020-09-08T15:33:00.706762+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:03.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:03 smithi055 bash[9296]: cluster 2020-09-08T15:33:00.706922+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:03.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:03 smithi055 bash[9296]: audit 2020-09-08T15:33:01.950195+0000 mgr.x (mgr.24118) 139 : audit [DBG] from='client.34349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi166:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:33:03.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:03 smithi055 bash[9296]: cephadm 2020-09-08T15:33:01.953571+0000 mgr.x (mgr.24118) 140 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:33:03.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:03 smithi055 bash[9296]: cephadm 2020-09-08T15:33:01.953752+0000 mgr.x (mgr.24118) 141 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:33:03.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:03 smithi055 bash[9296]: cephadm 2020-09-08T15:33:01.955168+0000 mgr.x (mgr.24118) 142 : cephadm [INF] Applying None on host smithi166... 2020-09-08T15:33:03.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:03 smithi055 bash[9296]: cluster 2020-09-08T15:33:02.527195+0000 mon.a (mon.0) 652 : cluster [INF] osd.7 v1:172.21.15.55:6827/128909153 boot 2020-09-08T15:33:03.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:03 smithi055 bash[9296]: cluster 2020-09-08T15:33:02.527253+0000 mon.a (mon.0) 653 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2020-09-08T15:33:03.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:03 smithi055 bash[9296]: audit 2020-09-08T15:33:02.537168+0000 mon.b (mon.1) 64 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2020-09-08T15:33:03.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:03 smithi104 bash[9203]: cluster 2020-09-08T15:33:00.706762+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:03.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:03 smithi104 bash[9203]: cluster 2020-09-08T15:33:00.706922+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:03.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:03 smithi104 bash[9203]: audit 2020-09-08T15:33:01.950195+0000 mgr.x (mgr.24118) 139 : audit [DBG] from='client.34349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi166:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:33:03.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:03 smithi104 bash[9203]: cephadm 2020-09-08T15:33:01.953571+0000 mgr.x (mgr.24118) 140 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:33:03.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:03 smithi104 bash[9203]: cephadm 2020-09-08T15:33:01.953752+0000 mgr.x (mgr.24118) 141 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:33:03.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:03 smithi104 bash[9203]: cephadm 2020-09-08T15:33:01.955168+0000 mgr.x (mgr.24118) 142 : cephadm [INF] Applying None on host smithi166... 2020-09-08T15:33:03.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:03 smithi104 bash[9203]: cluster 2020-09-08T15:33:02.527195+0000 mon.a (mon.0) 652 : cluster [INF] osd.7 v1:172.21.15.55:6827/128909153 boot 2020-09-08T15:33:03.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:03 smithi104 bash[9203]: cluster 2020-09-08T15:33:02.527253+0000 mon.a (mon.0) 653 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2020-09-08T15:33:03.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:03 smithi104 bash[9203]: audit 2020-09-08T15:33:02.537168+0000 mon.b (mon.1) 64 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2020-09-08T15:33:04.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:04 smithi166 bash[9446]: cluster 2020-09-08T15:33:03.503914+0000 mgr.x (mgr.24118) 143 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:33:04.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:04 smithi166 bash[9446]: cluster 2020-09-08T15:33:03.534875+0000 mon.a (mon.0) 654 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2020-09-08T15:33:04.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:04 smithi166 bash[9446]: cluster 2020-09-08T15:33:03.645410+0000 osd.7 (osd.7) 3 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:33:04.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:04 smithi166 bash[9446]: cluster 2020-09-08T15:33:03.648373+0000 osd.7 (osd.7) 4 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:33:04.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:04 smithi166 bash[9446]: cluster 2020-09-08T15:33:03.693709+0000 osd.0 (osd.0) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:04.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:04 smithi166 bash[9446]: cluster 2020-09-08T15:33:03.693861+0000 osd.0 (osd.0) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:04.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:04 smithi055 bash[9296]: cluster 2020-09-08T15:33:03.503914+0000 mgr.x (mgr.24118) 143 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:33:04.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:04 smithi055 bash[9296]: cluster 2020-09-08T15:33:03.534875+0000 mon.a (mon.0) 654 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2020-09-08T15:33:04.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:04 smithi055 bash[9296]: cluster 2020-09-08T15:33:03.645410+0000 osd.7 (osd.7) 3 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:33:04.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:04 smithi055 bash[9296]: cluster 2020-09-08T15:33:03.648373+0000 osd.7 (osd.7) 4 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:33:04.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:04 smithi055 bash[9296]: cluster 2020-09-08T15:33:03.693709+0000 osd.0 (osd.0) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:04.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:04 smithi055 bash[9296]: cluster 2020-09-08T15:33:03.693861+0000 osd.0 (osd.0) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:04.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:04 smithi104 bash[9203]: cluster 2020-09-08T15:33:03.503914+0000 mgr.x (mgr.24118) 143 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 37 MiB used, 626 GiB / 626 GiB avail 2020-09-08T15:33:04.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:04 smithi104 bash[9203]: cluster 2020-09-08T15:33:03.534875+0000 mon.a (mon.0) 654 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2020-09-08T15:33:04.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:04 smithi104 bash[9203]: cluster 2020-09-08T15:33:03.645410+0000 osd.7 (osd.7) 3 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:33:04.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:04 smithi104 bash[9203]: cluster 2020-09-08T15:33:03.648373+0000 osd.7 (osd.7) 4 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:33:04.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:04 smithi104 bash[9203]: cluster 2020-09-08T15:33:03.693709+0000 osd.0 (osd.0) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:04.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:04 smithi104 bash[9203]: cluster 2020-09-08T15:33:03.693861+0000 osd.0 (osd.0) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:05.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:05 smithi166 bash[9446]: cluster 2020-09-08T15:33:04.576997+0000 mon.a (mon.0) 655 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2020-09-08T15:33:05.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:05 smithi166 bash[9446]: cluster 2020-09-08T15:33:05.504552+0000 mgr.x (mgr.24118) 144 : cluster [DBG] pgmap v147: 1 pgs: 1 peering; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:05.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:05 smithi055 bash[9296]: cluster 2020-09-08T15:33:04.576997+0000 mon.a (mon.0) 655 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2020-09-08T15:33:05.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:05 smithi055 bash[9296]: cluster 2020-09-08T15:33:05.504552+0000 mgr.x (mgr.24118) 144 : cluster [DBG] pgmap v147: 1 pgs: 1 peering; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:05.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:05 smithi104 bash[9203]: cluster 2020-09-08T15:33:04.576997+0000 mon.a (mon.0) 655 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2020-09-08T15:33:05.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:05 smithi104 bash[9203]: cluster 2020-09-08T15:33:05.504552+0000 mgr.x (mgr.24118) 144 : cluster [DBG] pgmap v147: 1 pgs: 1 peering; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:06.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:06 smithi166 bash[9446]: audit 2020-09-08T15:33:05.798945+0000 mon.a (mon.0) 656 : audit [INF] from='client.? v1:172.21.15.166:0/109866555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b0ac380-878e-4c1c-9278-80ce67ca2df4"}]: dispatch 2020-09-08T15:33:06.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:06 smithi166 bash[9446]: audit 2020-09-08T15:33:05.806928+0000 mon.a (mon.0) 657 : audit [INF] from='client.? v1:172.21.15.166:0/109866555' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b0ac380-878e-4c1c-9278-80ce67ca2df4"}]': finished 2020-09-08T15:33:06.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:06 smithi166 bash[9446]: cluster 2020-09-08T15:33:05.807032+0000 mon.a (mon.0) 658 : cluster [DBG] osdmap e59: 9 total, 8 up, 8 in 2020-09-08T15:33:06.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:06 smithi166 bash[9446]: audit 2020-09-08T15:33:05.810321+0000 mon.b (mon.1) 65 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:06.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:06 smithi166 bash[9446]: audit 2020-09-08T15:33:06.427857+0000 mon.c (mon.2) 9 : audit [DBG] from='client.? v1:172.21.15.166:0/1635525017' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:33:06.905 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:06 smithi055 bash[9296]: audit 2020-09-08T15:33:05.798945+0000 mon.a (mon.0) 656 : audit [INF] from='client.? v1:172.21.15.166:0/109866555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b0ac380-878e-4c1c-9278-80ce67ca2df4"}]: dispatch 2020-09-08T15:33:06.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:06 smithi055 bash[9296]: audit 2020-09-08T15:33:05.806928+0000 mon.a (mon.0) 657 : audit [INF] from='client.? v1:172.21.15.166:0/109866555' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b0ac380-878e-4c1c-9278-80ce67ca2df4"}]': finished 2020-09-08T15:33:06.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:06 smithi055 bash[9296]: cluster 2020-09-08T15:33:05.807032+0000 mon.a (mon.0) 658 : cluster [DBG] osdmap e59: 9 total, 8 up, 8 in 2020-09-08T15:33:06.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:06 smithi055 bash[9296]: audit 2020-09-08T15:33:05.810321+0000 mon.b (mon.1) 65 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:06.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:06 smithi055 bash[9296]: audit 2020-09-08T15:33:06.427857+0000 mon.c (mon.2) 9 : audit [DBG] from='client.? v1:172.21.15.166:0/1635525017' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:33:06.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:06 smithi104 bash[9203]: audit 2020-09-08T15:33:05.798945+0000 mon.a (mon.0) 656 : audit [INF] from='client.? v1:172.21.15.166:0/109866555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b0ac380-878e-4c1c-9278-80ce67ca2df4"}]: dispatch 2020-09-08T15:33:06.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:06 smithi104 bash[9203]: audit 2020-09-08T15:33:05.806928+0000 mon.a (mon.0) 657 : audit [INF] from='client.? v1:172.21.15.166:0/109866555' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b0ac380-878e-4c1c-9278-80ce67ca2df4"}]': finished 2020-09-08T15:33:06.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:06 smithi104 bash[9203]: cluster 2020-09-08T15:33:05.807032+0000 mon.a (mon.0) 658 : cluster [DBG] osdmap e59: 9 total, 8 up, 8 in 2020-09-08T15:33:06.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:06 smithi104 bash[9203]: audit 2020-09-08T15:33:05.810321+0000 mon.b (mon.1) 65 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:06.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:06 smithi104 bash[9203]: audit 2020-09-08T15:33:06.427857+0000 mon.c (mon.2) 9 : audit [DBG] from='client.? v1:172.21.15.166:0/1635525017' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:33:09.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:09 smithi166 bash[9446]: audit 2020-09-08T15:33:09.411295+0000 mon.c (mon.2) 10 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:09.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:09 smithi055 bash[9296]: audit 2020-09-08T15:33:09.411295+0000 mon.c (mon.2) 10 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:09.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:09 smithi104 bash[9203]: audit 2020-09-08T15:33:09.411295+0000 mon.c (mon.2) 10 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:10.761 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:10 smithi166 bash[9446]: cluster 2020-09-08T15:33:09.505847+0000 mgr.x (mgr.24118) 146 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:10.761 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:10 smithi166 bash[9446]: cluster 2020-09-08T15:33:09.996153+0000 osd.5 (osd.5) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:10.761 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:10 smithi166 bash[9446]: cluster 2020-09-08T15:33:09.996312+0000 osd.5 (osd.5) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:10.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:10 smithi055 bash[9296]: cluster 2020-09-08T15:33:09.505847+0000 mgr.x (mgr.24118) 146 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:10.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:10 smithi055 bash[9296]: cluster 2020-09-08T15:33:09.996153+0000 osd.5 (osd.5) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:10.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:10 smithi055 bash[9296]: cluster 2020-09-08T15:33:09.996312+0000 osd.5 (osd.5) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:10.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:10 smithi104 bash[9203]: cluster 2020-09-08T15:33:09.505847+0000 mgr.x (mgr.24118) 146 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:10.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:10 smithi104 bash[9203]: cluster 2020-09-08T15:33:09.996153+0000 osd.5 (osd.5) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:10.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:10 smithi104 bash[9203]: cluster 2020-09-08T15:33:09.996312+0000 osd.5 (osd.5) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:11.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:11 smithi166 bash[9446]: cluster 2020-09-08T15:33:10.498712+0000 mon.a (mon.0) 659 : cluster [DBG] osdmap e60: 9 total, 8 up, 8 in 2020-09-08T15:33:11.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:11 smithi166 bash[9446]: audit 2020-09-08T15:33:10.503765+0000 mon.c (mon.2) 11 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:11.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:11 smithi055 bash[9296]: cluster 2020-09-08T15:33:10.498712+0000 mon.a (mon.0) 659 : cluster [DBG] osdmap e60: 9 total, 8 up, 8 in 2020-09-08T15:33:11.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:11 smithi055 bash[9296]: audit 2020-09-08T15:33:10.503765+0000 mon.c (mon.2) 11 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:11.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:11 smithi104 bash[9203]: cluster 2020-09-08T15:33:10.498712+0000 mon.a (mon.0) 659 : cluster [DBG] osdmap e60: 9 total, 8 up, 8 in 2020-09-08T15:33:11.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:11 smithi104 bash[9203]: audit 2020-09-08T15:33:10.503765+0000 mon.c (mon.2) 11 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:12.863 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:12 smithi166 bash[9446]: cluster 2020-09-08T15:33:11.506416+0000 mgr.x (mgr.24118) 147 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:12.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:12 smithi055 bash[9296]: cluster 2020-09-08T15:33:11.506416+0000 mgr.x (mgr.24118) 147 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:12.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:12 smithi104 bash[9203]: cluster 2020-09-08T15:33:11.506416+0000 mgr.x (mgr.24118) 147 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:14.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:14 smithi166 bash[9446]: cluster 2020-09-08T15:33:13.507037+0000 mgr.x (mgr.24118) 148 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:14.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:14 smithi166 bash[9446]: audit 2020-09-08T15:33:13.573751+0000 mon.c (mon.2) 12 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2020-09-08T15:33:14.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:14 smithi166 bash[9446]: audit 2020-09-08T15:33:13.575309+0000 mon.c (mon.2) 13 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:33:14.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:14 smithi166 bash[9446]: audit 2020-09-08T15:33:13.577261+0000 mon.c (mon.2) 14 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.8", "key": "container_image"}]: dispatch 2020-09-08T15:33:14.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:14 smithi055 bash[9296]: cluster 2020-09-08T15:33:13.507037+0000 mgr.x (mgr.24118) 148 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:14.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:14 smithi055 bash[9296]: audit 2020-09-08T15:33:13.573751+0000 mon.c (mon.2) 12 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2020-09-08T15:33:14.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:14 smithi055 bash[9296]: audit 2020-09-08T15:33:13.575309+0000 mon.c (mon.2) 13 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:33:14.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:14 smithi055 bash[9296]: audit 2020-09-08T15:33:13.577261+0000 mon.c (mon.2) 14 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.8", "key": "container_image"}]: dispatch 2020-09-08T15:33:14.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:14 smithi104 bash[9203]: cluster 2020-09-08T15:33:13.507037+0000 mgr.x (mgr.24118) 148 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:14.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:14 smithi104 bash[9203]: audit 2020-09-08T15:33:13.573751+0000 mon.c (mon.2) 12 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2020-09-08T15:33:14.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:14 smithi104 bash[9203]: audit 2020-09-08T15:33:13.575309+0000 mon.c (mon.2) 13 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:33:14.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:14 smithi104 bash[9203]: audit 2020-09-08T15:33:13.577261+0000 mon.c (mon.2) 14 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.8", "key": "container_image"}]: dispatch 2020-09-08T15:33:15.789 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:15 smithi166 bash[9446]: cephadm 2020-09-08T15:33:13.576439+0000 mgr.x (mgr.24118) 149 : cephadm [INF] Deploying daemon osd.8 on smithi166 2020-09-08T15:33:15.829 INFO:teuthology.orchestra.run.smithi166.stdout:Created osd(s) 8 on host 'smithi166' 2020-09-08T15:33:15.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:15 smithi055 bash[9296]: cephadm 2020-09-08T15:33:13.576439+0000 mgr.x (mgr.24118) 149 : cephadm [INF] Deploying daemon osd.8 on smithi166 2020-09-08T15:33:15.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:15 smithi104 bash[9203]: cephadm 2020-09-08T15:33:13.576439+0000 mgr.x (mgr.24118) 149 : cephadm [INF] Deploying daemon osd.8 on smithi166 2020-09-08T15:33:16.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:16 smithi166 bash[9446]: cluster 2020-09-08T15:33:15.507631+0000 mgr.x (mgr.24118) 150 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:16.872 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:16 smithi166 bash[9446]: audit 2020-09-08T15:33:15.825106+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.170728\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:16.872 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:16 smithi166 bash[9446]: audit 2020-09-08T15:33:15.825476+0000 mon.c (mon.2) 15 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.170728\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:16.873 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:16 smithi166 bash[9446]: audit 2020-09-08T15:33:15.826247+0000 mon.c (mon.2) 16 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:33:16.873 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:16 smithi166 bash[9446]: audit 2020-09-08T15:33:15.830649+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.170728\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:33:16.873 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:16 smithi166 bash[9446]: audit 2020-09-08T15:33:16.481974+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_daemon_update\": \"2020-09-08T15:33:16.480729\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:16.873 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:16 smithi166 bash[9446]: audit 2020-09-08T15:33:16.482205+0000 mon.c (mon.2) 17 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_daemon_update\": \"2020-09-08T15:33:16.480729\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:16.874 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:16 smithi166 bash[9446]: audit 2020-09-08T15:33:16.487733+0000 mon.a (mon.0) 663 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_daemon_update\": \"2020-09-08T15:33:16.480729\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:33:16.874 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:16 smithi166 bash[9446]: audit 2020-09-08T15:33:16.490557+0000 mon.c (mon.2) 18 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:16.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:16 smithi055 bash[9296]: cluster 2020-09-08T15:33:15.507631+0000 mgr.x (mgr.24118) 150 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:16.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:16 smithi055 bash[9296]: audit 2020-09-08T15:33:15.825106+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.170728\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:16.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:16 smithi055 bash[9296]: audit 2020-09-08T15:33:15.825476+0000 mon.c (mon.2) 15 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.170728\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:16.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:16 smithi055 bash[9296]: audit 2020-09-08T15:33:15.826247+0000 mon.c (mon.2) 16 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:33:16.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:16 smithi055 bash[9296]: audit 2020-09-08T15:33:15.830649+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.170728\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:33:16.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:16 smithi055 bash[9296]: audit 2020-09-08T15:33:16.481974+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_daemon_update\": \"2020-09-08T15:33:16.480729\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:16.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:16 smithi055 bash[9296]: audit 2020-09-08T15:33:16.482205+0000 mon.c (mon.2) 17 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_daemon_update\": \"2020-09-08T15:33:16.480729\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:16.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:16 smithi055 bash[9296]: audit 2020-09-08T15:33:16.487733+0000 mon.a (mon.0) 663 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_daemon_update\": \"2020-09-08T15:33:16.480729\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:33:16.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:16 smithi055 bash[9296]: audit 2020-09-08T15:33:16.490557+0000 mon.c (mon.2) 18 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:16.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:16 smithi104 bash[9203]: cluster 2020-09-08T15:33:15.507631+0000 mgr.x (mgr.24118) 150 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:16.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:16 smithi104 bash[9203]: audit 2020-09-08T15:33:15.825106+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.170728\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:16.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:16 smithi104 bash[9203]: audit 2020-09-08T15:33:15.825476+0000 mon.c (mon.2) 15 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.170728\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:16.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:16 smithi104 bash[9203]: audit 2020-09-08T15:33:15.826247+0000 mon.c (mon.2) 16 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:33:16.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:16 smithi104 bash[9203]: audit 2020-09-08T15:33:15.830649+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:29:53.170728\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_device_update\": \"2020-09-08T15:28:26.010628\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:33:16.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:16 smithi104 bash[9203]: audit 2020-09-08T15:33:16.481974+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_daemon_update\": \"2020-09-08T15:33:16.480729\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:16.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:16 smithi104 bash[9203]: audit 2020-09-08T15:33:16.482205+0000 mon.c (mon.2) 17 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_daemon_update\": \"2020-09-08T15:33:16.480729\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:16.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:16 smithi104 bash[9203]: audit 2020-09-08T15:33:16.487733+0000 mon.a (mon.0) 663 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_daemon_update\": \"2020-09-08T15:33:16.480729\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:33:16.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:16 smithi104 bash[9203]: audit 2020-09-08T15:33:16.490557+0000 mon.c (mon.2) 18 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:17.140 INFO:teuthology.orchestra.run.smithi166:osd.8> sudo journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.8.service 2020-09-08T15:33:17.145 INFO:tasks.cephadm:Deploying osd.9 on smithi166 with /dev/vg_nvme/lv_3... 2020-09-08T15:33:17.145 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2020-09-08T15:33:17.270 INFO:journalctl@ceph.osd.8.smithi166.stdout:-- Logs begin at Tue 2020-09-01 16:49:30 UTC. -- 2020-09-08T15:33:17.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:17 smithi166 bash[9446]: cluster 2020-09-08T15:33:17.508227+0000 mgr.x (mgr.24118) 151 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:17.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:17 smithi055 bash[9296]: cluster 2020-09-08T15:33:17.508227+0000 mgr.x (mgr.24118) 151 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:17.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:17 smithi104 bash[9203]: cluster 2020-09-08T15:33:17.508227+0000 mgr.x (mgr.24118) 151 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:18.933 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:33:18 smithi166 bash[11928]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2020-09-08T15:33:18.933 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:33:18 smithi166 bash[11928]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-8 --no-mon-config 2020-09-08T15:33:18.933 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:33:18 smithi166 bash[11928]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-8/block 2020-09-08T15:33:18.934 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:33:18 smithi166 bash[11928]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-8/block 2020-09-08T15:33:18.934 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:33:18 smithi166 bash[11928]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2020-09-08T15:33:18.934 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:33:18 smithi166 bash[11928]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2020-09-08T15:33:18.934 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:33:18 smithi166 bash[11928]: --> ceph-volume lvm activate successful for osd ID: 8 2020-09-08T15:33:20.364 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:33:20 smithi166 bash[11928]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:33:20.364 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:20 smithi166 bash[9446]: cluster 2020-09-08T15:33:18.975557+0000 osd.3 (osd.3) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:20.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:20 smithi166 bash[9446]: cluster 2020-09-08T15:33:18.975711+0000 osd.3 (osd.3) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:20.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:20 smithi166 bash[9446]: cluster 2020-09-08T15:33:19.033342+0000 mon.a (mon.0) 664 : cluster [DBG] osdmap e61: 9 total, 8 up, 8 in 2020-09-08T15:33:20.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:20 smithi166 bash[9446]: audit 2020-09-08T15:33:19.034020+0000 mon.c (mon.2) 19 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:20.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:20 smithi166 bash[9446]: cluster 2020-09-08T15:33:19.508729+0000 mgr.x (mgr.24118) 152 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:20.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:20 smithi055 bash[9296]: cluster 2020-09-08T15:33:18.975557+0000 osd.3 (osd.3) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:20.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:20 smithi055 bash[9296]: cluster 2020-09-08T15:33:18.975711+0000 osd.3 (osd.3) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:20.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:20 smithi055 bash[9296]: cluster 2020-09-08T15:33:19.033342+0000 mon.a (mon.0) 664 : cluster [DBG] osdmap e61: 9 total, 8 up, 8 in 2020-09-08T15:33:20.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:20 smithi055 bash[9296]: audit 2020-09-08T15:33:19.034020+0000 mon.c (mon.2) 19 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:20.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:20 smithi055 bash[9296]: cluster 2020-09-08T15:33:19.508729+0000 mgr.x (mgr.24118) 152 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:20.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:20 smithi104 bash[9203]: cluster 2020-09-08T15:33:18.975557+0000 osd.3 (osd.3) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:20.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:20 smithi104 bash[9203]: cluster 2020-09-08T15:33:18.975711+0000 osd.3 (osd.3) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:20.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:20 smithi104 bash[9203]: cluster 2020-09-08T15:33:19.033342+0000 mon.a (mon.0) 664 : cluster [DBG] osdmap e61: 9 total, 8 up, 8 in 2020-09-08T15:33:20.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:20 smithi104 bash[9203]: audit 2020-09-08T15:33:19.034020+0000 mon.c (mon.2) 19 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:20.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:20 smithi104 bash[9203]: cluster 2020-09-08T15:33:19.508729+0000 mgr.x (mgr.24118) 152 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:22.064 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:33:21 smithi166 bash[11928]: debug 2020-09-08T15:33:21.827+0000 7ff3eda51700 10 -- 172.21.15.166:0/872140328 dispatch_throttle_release 278 to dispatch throttler 472/104857600 2020-09-08T15:33:22.065 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:33:21 smithi166 bash[11928]: debug 2020-09-08T15:33:21.827+0000 7ff3eda51700 20 -- 172.21.15.166:0/872140328 done calling dispatch on 0x55d7c9f3cc60 2020-09-08T15:33:22.065 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:33:21 smithi166 bash[11928]: debug 2020-09-08T15:33:21.827+0000 7ff3eda51700 1 -- 172.21.15.166:0/872140328 <== mon.2 v1:172.21.15.166:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 2194627810 0 0) 0x55d7c9f464e0 con 0x55d7c9fbd000 2020-09-08T15:33:22.065 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:33:21 smithi166 bash[11928]: debug 2020-09-08T15:33:21.827+0000 7ff3eda51700 10 -- 172.21.15.166:0/872140328 dispatch_throttle_release 194 to dispatch throttler 194/104857600 2020-09-08T15:33:22.065 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:33:21 smithi166 bash[11928]: debug 2020-09-08T15:33:21.827+0000 7ff3eda51700 20 -- 172.21.15.166:0/872140328 done calling dispatch on 0x55d7c9f464e0 2020-09-08T15:33:22.691 INFO:teuthology.orchestra.run.smithi166.stderr:--> Zapping: /dev/vg_nvme/lv_3 2020-09-08T15:33:22.692 INFO:teuthology.orchestra.run.smithi166.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2020-09-08T15:33:22.692 INFO:teuthology.orchestra.run.smithi166.stderr: stderr: 10+0 records in 2020-09-08T15:33:22.692 INFO:teuthology.orchestra.run.smithi166.stderr:10+0 records out 2020-09-08T15:33:22.692 INFO:teuthology.orchestra.run.smithi166.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0158363 s, 662 MB/s 2020-09-08T15:33:22.692 INFO:teuthology.orchestra.run.smithi166.stderr:--> Zapping successful for: 2020-09-08T15:33:22.790 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:22 smithi166 bash[9446]: cluster 2020-09-08T15:33:21.509199+0000 mgr.x (mgr.24118) 153 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:22.791 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:33:22 smithi166 bash[11928]: debug 2020-09-08T15:33:22.371+0000 7ff3f54d1f40 -1 Falling back to public interface 2020-09-08T15:33:22.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:22 smithi055 bash[9296]: cluster 2020-09-08T15:33:21.509199+0000 mgr.x (mgr.24118) 153 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:22.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:22 smithi104 bash[9203]: cluster 2020-09-08T15:33:21.509199+0000 mgr.x (mgr.24118) 153 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 43 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:23.114 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:33:23 smithi166 bash[11928]: debug 2020-09-08T15:33:23.023+0000 7ff3f54d1f40 -1 osd.8 0 log_to_monitors {default=true} 2020-09-08T15:33:23.465 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph orch daemon add osd smithi166:vg_nvme/lv_3 2020-09-08T15:33:23.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:23 smithi166 bash[9446]: audit 2020-09-08T15:33:23.032227+0000 mon.a (mon.0) 665 : audit [INF] from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2020-09-08T15:33:23.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:23 smithi166 bash[9446]: audit 2020-09-08T15:33:23.032513+0000 mon.b (mon.1) 66 : audit [INF] from='osd.8 v1:172.21.15.166:6801/2108117574' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2020-09-08T15:33:23.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:23 smithi166 bash[9446]: cluster 2020-09-08T15:33:23.509755+0000 mgr.x (mgr.24118) 154 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 44 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:23.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:23 smithi055 bash[9296]: audit 2020-09-08T15:33:23.032227+0000 mon.a (mon.0) 665 : audit [INF] from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2020-09-08T15:33:23.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:23 smithi055 bash[9296]: audit 2020-09-08T15:33:23.032513+0000 mon.b (mon.1) 66 : audit [INF] from='osd.8 v1:172.21.15.166:6801/2108117574' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2020-09-08T15:33:23.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:23 smithi055 bash[9296]: cluster 2020-09-08T15:33:23.509755+0000 mgr.x (mgr.24118) 154 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 44 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:23.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:23 smithi104 bash[9203]: audit 2020-09-08T15:33:23.032227+0000 mon.a (mon.0) 665 : audit [INF] from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2020-09-08T15:33:23.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:23 smithi104 bash[9203]: audit 2020-09-08T15:33:23.032513+0000 mon.b (mon.1) 66 : audit [INF] from='osd.8 v1:172.21.15.166:6801/2108117574' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2020-09-08T15:33:23.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:23 smithi104 bash[9203]: cluster 2020-09-08T15:33:23.509755+0000 mgr.x (mgr.24118) 154 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 44 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:24.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:33:24 smithi166 bash[11928]: debug 2020-09-08T15:33:24.599+0000 7ff3eba4d700 -1 osd.8 0 waiting for initial osdmap 2020-09-08T15:33:24.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:33:24 smithi166 bash[11928]: debug 2020-09-08T15:33:24.599+0000 7ff3ddb5c700 -1 osd.8 0 failed to load OSD map for epoch 10, got 0 bytes 2020-09-08T15:33:24.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:24 smithi166 bash[9446]: audit 2020-09-08T15:33:23.589019+0000 mon.a (mon.0) 666 : audit [INF] from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2020-09-08T15:33:24.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:24 smithi166 bash[9446]: cluster 2020-09-08T15:33:23.589105+0000 mon.a (mon.0) 667 : cluster [DBG] osdmap e62: 9 total, 8 up, 8 in 2020-09-08T15:33:24.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:24 smithi166 bash[9446]: audit 2020-09-08T15:33:23.592284+0000 mon.a (mon.0) 668 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:24.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:24 smithi166 bash[9446]: audit 2020-09-08T15:33:23.592935+0000 mon.a (mon.0) 669 : audit [INF] from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:33:24.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:24 smithi166 bash[9446]: audit 2020-09-08T15:33:23.593262+0000 mon.b (mon.1) 67 : audit [INF] from='osd.8 v1:172.21.15.166:6801/2108117574' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:33:24.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:24 smithi055 bash[9296]: audit 2020-09-08T15:33:23.589019+0000 mon.a (mon.0) 666 : audit [INF] from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2020-09-08T15:33:24.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:24 smithi055 bash[9296]: cluster 2020-09-08T15:33:23.589105+0000 mon.a (mon.0) 667 : cluster [DBG] osdmap e62: 9 total, 8 up, 8 in 2020-09-08T15:33:24.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:24 smithi055 bash[9296]: audit 2020-09-08T15:33:23.592284+0000 mon.a (mon.0) 668 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:24.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:24 smithi055 bash[9296]: audit 2020-09-08T15:33:23.592935+0000 mon.a (mon.0) 669 : audit [INF] from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:33:24.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:24 smithi055 bash[9296]: audit 2020-09-08T15:33:23.593262+0000 mon.b (mon.1) 67 : audit [INF] from='osd.8 v1:172.21.15.166:6801/2108117574' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:33:24.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:24 smithi104 bash[9203]: audit 2020-09-08T15:33:23.589019+0000 mon.a (mon.0) 666 : audit [INF] from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2020-09-08T15:33:24.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:24 smithi104 bash[9203]: cluster 2020-09-08T15:33:23.589105+0000 mon.a (mon.0) 667 : cluster [DBG] osdmap e62: 9 total, 8 up, 8 in 2020-09-08T15:33:24.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:24 smithi104 bash[9203]: audit 2020-09-08T15:33:23.592284+0000 mon.a (mon.0) 668 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:24.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:24 smithi104 bash[9203]: audit 2020-09-08T15:33:23.592935+0000 mon.a (mon.0) 669 : audit [INF] from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:33:24.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:24 smithi104 bash[9203]: audit 2020-09-08T15:33:23.593262+0000 mon.b (mon.1) 67 : audit [INF] from='osd.8 v1:172.21.15.166:6801/2108117574' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:33:25.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:25 smithi166 bash[9446]: audit 2020-09-08T15:33:24.591451+0000 mon.a (mon.0) 670 : audit [INF] from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]': finished 2020-09-08T15:33:25.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:25 smithi166 bash[9446]: cluster 2020-09-08T15:33:24.591533+0000 mon.a (mon.0) 671 : cluster [DBG] osdmap e63: 9 total, 8 up, 8 in 2020-09-08T15:33:25.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:25 smithi166 bash[9446]: audit 2020-09-08T15:33:24.592456+0000 mon.a (mon.0) 672 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:25.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:25 smithi166 bash[9446]: audit 2020-09-08T15:33:24.599753+0000 mon.a (mon.0) 673 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:25.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:25 smithi166 bash[9446]: audit 2020-09-08T15:33:24.687458+0000 mon.a (mon.0) 674 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:33:25.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:25 smithi166 bash[9446]: audit 2020-09-08T15:33:24.698134+0000 mon.a (mon.0) 675 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:33:25.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:25 smithi166 bash[9446]: audit 2020-09-08T15:33:24.704884+0000 mon.a (mon.0) 676 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:33:25.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:25 smithi166 bash[9446]: audit 2020-09-08T15:33:24.983794+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_daemon_update\": \"2020-09-08T15:33:16.480729\", \"last_device_update\": \"2020-09-08T15:33:24.982955\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:25.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:25 smithi166 bash[9446]: audit 2020-09-08T15:33:24.990541+0000 mon.a (mon.0) 678 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_daemon_update\": \"2020-09-08T15:33:16.480729\", \"last_device_update\": \"2020-09-08T15:33:24.982955\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:33:25.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:25 smithi166 bash[9446]: audit 2020-09-08T15:33:24.992863+0000 mon.a (mon.0) 679 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:25.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:25 smithi166 bash[9446]: audit 2020-09-08T15:33:24.998033+0000 mon.a (mon.0) 680 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:33:25.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:25 smithi166 bash[9446]: audit 2020-09-08T15:33:25.003968+0000 mon.a (mon.0) 681 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:25.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:25 smithi166 bash[9446]: audit 2020-09-08T15:33:25.009166+0000 mon.a (mon.0) 682 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:33:25.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:25 smithi166 bash[9446]: cluster 2020-09-08T15:33:25.510355+0000 mgr.x (mgr.24118) 155 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 44 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:25.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:25 smithi055 bash[9296]: audit 2020-09-08T15:33:24.591451+0000 mon.a (mon.0) 670 : audit [INF] from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]': finished 2020-09-08T15:33:25.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:25 smithi055 bash[9296]: cluster 2020-09-08T15:33:24.591533+0000 mon.a (mon.0) 671 : cluster [DBG] osdmap e63: 9 total, 8 up, 8 in 2020-09-08T15:33:25.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:25 smithi055 bash[9296]: audit 2020-09-08T15:33:24.592456+0000 mon.a (mon.0) 672 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:25.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:25 smithi055 bash[9296]: audit 2020-09-08T15:33:24.599753+0000 mon.a (mon.0) 673 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:25.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:25 smithi055 bash[9296]: audit 2020-09-08T15:33:24.687458+0000 mon.a (mon.0) 674 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:33:25.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:25 smithi055 bash[9296]: audit 2020-09-08T15:33:24.698134+0000 mon.a (mon.0) 675 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:33:25.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:25 smithi055 bash[9296]: audit 2020-09-08T15:33:24.704884+0000 mon.a (mon.0) 676 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:33:25.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:25 smithi055 bash[9296]: audit 2020-09-08T15:33:24.983794+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_daemon_update\": \"2020-09-08T15:33:16.480729\", \"last_device_update\": \"2020-09-08T15:33:24.982955\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:25.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:25 smithi055 bash[9296]: audit 2020-09-08T15:33:24.990541+0000 mon.a (mon.0) 678 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_daemon_update\": \"2020-09-08T15:33:16.480729\", \"last_device_update\": \"2020-09-08T15:33:24.982955\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:33:25.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:25 smithi055 bash[9296]: audit 2020-09-08T15:33:24.992863+0000 mon.a (mon.0) 679 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:25.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:25 smithi055 bash[9296]: audit 2020-09-08T15:33:24.998033+0000 mon.a (mon.0) 680 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:33:25.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:25 smithi055 bash[9296]: audit 2020-09-08T15:33:25.003968+0000 mon.a (mon.0) 681 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:25.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:25 smithi055 bash[9296]: audit 2020-09-08T15:33:25.009166+0000 mon.a (mon.0) 682 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:33:25.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:25 smithi055 bash[9296]: cluster 2020-09-08T15:33:25.510355+0000 mgr.x (mgr.24118) 155 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 44 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:25.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:25 smithi104 bash[9203]: audit 2020-09-08T15:33:24.591451+0000 mon.a (mon.0) 670 : audit [INF] from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]': finished 2020-09-08T15:33:25.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:25 smithi104 bash[9203]: cluster 2020-09-08T15:33:24.591533+0000 mon.a (mon.0) 671 : cluster [DBG] osdmap e63: 9 total, 8 up, 8 in 2020-09-08T15:33:25.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:25 smithi104 bash[9203]: audit 2020-09-08T15:33:24.592456+0000 mon.a (mon.0) 672 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:25.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:25 smithi104 bash[9203]: audit 2020-09-08T15:33:24.599753+0000 mon.a (mon.0) 673 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:25.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:25 smithi104 bash[9203]: audit 2020-09-08T15:33:24.687458+0000 mon.a (mon.0) 674 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:33:25.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:25 smithi104 bash[9203]: audit 2020-09-08T15:33:24.698134+0000 mon.a (mon.0) 675 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:33:25.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:25 smithi104 bash[9203]: audit 2020-09-08T15:33:24.704884+0000 mon.a (mon.0) 676 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:33:25.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:25 smithi104 bash[9203]: audit 2020-09-08T15:33:24.983794+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_daemon_update\": \"2020-09-08T15:33:16.480729\", \"last_device_update\": \"2020-09-08T15:33:24.982955\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:25.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:25 smithi104 bash[9203]: audit 2020-09-08T15:33:24.990541+0000 mon.a (mon.0) 678 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}}, \"last_daemon_update\": \"2020-09-08T15:33:16.480729\", \"last_device_update\": \"2020-09-08T15:33:24.982955\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:33:25.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:25 smithi104 bash[9203]: audit 2020-09-08T15:33:24.992863+0000 mon.a (mon.0) 679 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:25.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:25 smithi104 bash[9203]: audit 2020-09-08T15:33:24.998033+0000 mon.a (mon.0) 680 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:33:25.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:25 smithi104 bash[9203]: audit 2020-09-08T15:33:25.003968+0000 mon.a (mon.0) 681 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:25.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:25 smithi104 bash[9203]: audit 2020-09-08T15:33:25.009166+0000 mon.a (mon.0) 682 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:33:25.946 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:25 smithi104 bash[9203]: cluster 2020-09-08T15:33:25.510355+0000 mgr.x (mgr.24118) 155 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 44 MiB used, 715 GiB / 715 GiB avail 2020-09-08T15:33:26.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:26 smithi166 bash[9446]: cluster 2020-09-08T15:33:24.024415+0000 osd.8 (osd.8) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:26.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:26 smithi166 bash[9446]: cluster 2020-09-08T15:33:24.024572+0000 osd.8 (osd.8) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:26.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:26 smithi166 bash[9446]: cluster 2020-09-08T15:33:25.602663+0000 mon.a (mon.0) 683 : cluster [INF] osd.8 v1:172.21.15.166:6801/2108117574 boot 2020-09-08T15:33:26.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:26 smithi166 bash[9446]: cluster 2020-09-08T15:33:25.602755+0000 mon.a (mon.0) 684 : cluster [DBG] osdmap e64: 9 total, 9 up, 9 in 2020-09-08T15:33:26.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:26 smithi166 bash[9446]: audit 2020-09-08T15:33:25.603689+0000 mon.a (mon.0) 685 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:26.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:26 smithi055 bash[9296]: cluster 2020-09-08T15:33:24.024415+0000 osd.8 (osd.8) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:26.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:26 smithi055 bash[9296]: cluster 2020-09-08T15:33:24.024572+0000 osd.8 (osd.8) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:26.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:26 smithi055 bash[9296]: cluster 2020-09-08T15:33:25.602663+0000 mon.a (mon.0) 683 : cluster [INF] osd.8 v1:172.21.15.166:6801/2108117574 boot 2020-09-08T15:33:26.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:26 smithi055 bash[9296]: cluster 2020-09-08T15:33:25.602755+0000 mon.a (mon.0) 684 : cluster [DBG] osdmap e64: 9 total, 9 up, 9 in 2020-09-08T15:33:26.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:26 smithi055 bash[9296]: audit 2020-09-08T15:33:25.603689+0000 mon.a (mon.0) 685 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:26.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:26 smithi104 bash[9203]: cluster 2020-09-08T15:33:24.024415+0000 osd.8 (osd.8) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:26.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:26 smithi104 bash[9203]: cluster 2020-09-08T15:33:24.024572+0000 osd.8 (osd.8) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:26.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:26 smithi104 bash[9203]: cluster 2020-09-08T15:33:25.602663+0000 mon.a (mon.0) 683 : cluster [INF] osd.8 v1:172.21.15.166:6801/2108117574 boot 2020-09-08T15:33:26.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:26 smithi104 bash[9203]: cluster 2020-09-08T15:33:25.602755+0000 mon.a (mon.0) 684 : cluster [DBG] osdmap e64: 9 total, 9 up, 9 in 2020-09-08T15:33:26.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:26 smithi104 bash[9203]: audit 2020-09-08T15:33:25.603689+0000 mon.a (mon.0) 685 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2020-09-08T15:33:27.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:27 smithi055 bash[9296]: cluster 2020-09-08T15:33:26.615114+0000 mon.a (mon.0) 686 : cluster [DBG] osdmap e65: 9 total, 9 up, 9 in 2020-09-08T15:33:27.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:27 smithi055 bash[9296]: audit 2020-09-08T15:33:27.044580+0000 mgr.x (mgr.24118) 156 : audit [DBG] from='client.34373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi166:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:33:27.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:27 smithi055 bash[9296]: audit 2020-09-08T15:33:27.045839+0000 mon.a (mon.0) 687 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:33:27.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:27 smithi055 bash[9296]: cephadm 2020-09-08T15:33:27.047906+0000 mgr.x (mgr.24118) 157 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:33:27.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:27 smithi055 bash[9296]: cephadm 2020-09-08T15:33:27.048098+0000 mgr.x (mgr.24118) 158 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:33:27.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:27 smithi055 bash[9296]: cephadm 2020-09-08T15:33:27.049009+0000 mgr.x (mgr.24118) 159 : cephadm [INF] Applying None on host smithi166... 2020-09-08T15:33:27.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:27 smithi055 bash[9296]: audit 2020-09-08T15:33:27.049041+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:33:27.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:27 smithi055 bash[9296]: audit 2020-09-08T15:33:27.050627+0000 mon.a (mon.0) 689 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:33:27.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:27 smithi055 bash[9296]: audit 2020-09-08T15:33:27.052181+0000 mon.a (mon.0) 690 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:27.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:27 smithi055 bash[9296]: cluster 2020-09-08T15:33:27.510912+0000 mgr.x (mgr.24118) 160 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:27.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:27 smithi104 bash[9203]: cluster 2020-09-08T15:33:26.615114+0000 mon.a (mon.0) 686 : cluster [DBG] osdmap e65: 9 total, 9 up, 9 in 2020-09-08T15:33:27.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:27 smithi104 bash[9203]: audit 2020-09-08T15:33:27.044580+0000 mgr.x (mgr.24118) 156 : audit [DBG] from='client.34373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi166:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:33:27.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:27 smithi104 bash[9203]: audit 2020-09-08T15:33:27.045839+0000 mon.a (mon.0) 687 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:33:27.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:27 smithi104 bash[9203]: cephadm 2020-09-08T15:33:27.047906+0000 mgr.x (mgr.24118) 157 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:33:27.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:27 smithi104 bash[9203]: cephadm 2020-09-08T15:33:27.048098+0000 mgr.x (mgr.24118) 158 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:33:27.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:27 smithi104 bash[9203]: cephadm 2020-09-08T15:33:27.049009+0000 mgr.x (mgr.24118) 159 : cephadm [INF] Applying None on host smithi166... 2020-09-08T15:33:27.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:27 smithi104 bash[9203]: audit 2020-09-08T15:33:27.049041+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:33:27.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:27 smithi104 bash[9203]: audit 2020-09-08T15:33:27.050627+0000 mon.a (mon.0) 689 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:33:27.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:27 smithi104 bash[9203]: audit 2020-09-08T15:33:27.052181+0000 mon.a (mon.0) 690 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:27.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:27 smithi104 bash[9203]: cluster 2020-09-08T15:33:27.510912+0000 mgr.x (mgr.24118) 160 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:28.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:27 smithi166 bash[9446]: cluster 2020-09-08T15:33:26.615114+0000 mon.a (mon.0) 686 : cluster [DBG] osdmap e65: 9 total, 9 up, 9 in 2020-09-08T15:33:28.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:27 smithi166 bash[9446]: audit 2020-09-08T15:33:27.044580+0000 mgr.x (mgr.24118) 156 : audit [DBG] from='client.34373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi166:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:33:28.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:27 smithi166 bash[9446]: audit 2020-09-08T15:33:27.045839+0000 mon.a (mon.0) 687 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:33:28.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:27 smithi166 bash[9446]: cephadm 2020-09-08T15:33:27.047906+0000 mgr.x (mgr.24118) 157 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:33:28.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:27 smithi166 bash[9446]: cephadm 2020-09-08T15:33:27.048098+0000 mgr.x (mgr.24118) 158 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:33:28.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:27 smithi166 bash[9446]: cephadm 2020-09-08T15:33:27.049009+0000 mgr.x (mgr.24118) 159 : cephadm [INF] Applying None on host smithi166... 2020-09-08T15:33:28.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:27 smithi166 bash[9446]: audit 2020-09-08T15:33:27.049041+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:33:28.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:27 smithi166 bash[9446]: audit 2020-09-08T15:33:27.050627+0000 mon.a (mon.0) 689 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:33:28.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:27 smithi166 bash[9446]: audit 2020-09-08T15:33:27.052181+0000 mon.a (mon.0) 690 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:28.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:27 smithi166 bash[9446]: cluster 2020-09-08T15:33:27.510912+0000 mgr.x (mgr.24118) 160 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:30.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:30 smithi166 bash[9446]: cluster 2020-09-08T15:33:29.511496+0000 mgr.x (mgr.24118) 161 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:33:30.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:30 smithi055 bash[9296]: cluster 2020-09-08T15:33:29.511496+0000 mgr.x (mgr.24118) 161 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:33:30.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:30 smithi104 bash[9203]: cluster 2020-09-08T15:33:29.511496+0000 mgr.x (mgr.24118) 161 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:33:31.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:31 smithi166 bash[9446]: audit 2020-09-08T15:33:30.834236+0000 mon.c (mon.2) 20 : audit [INF] from='client.? v1:172.21.15.166:0/1305441454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08"}]: dispatch 2020-09-08T15:33:31.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:31 smithi166 bash[9446]: audit 2020-09-08T15:33:30.834335+0000 mon.a (mon.0) 691 : audit [INF] from='client.34379 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08"}]: dispatch 2020-09-08T15:33:31.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:31 smithi166 bash[9446]: audit 2020-09-08T15:33:30.842366+0000 mon.a (mon.0) 692 : audit [INF] from='client.34379 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08"}]': finished 2020-09-08T15:33:31.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:31 smithi166 bash[9446]: cluster 2020-09-08T15:33:30.842463+0000 mon.a (mon.0) 693 : cluster [DBG] osdmap e66: 10 total, 9 up, 9 in 2020-09-08T15:33:31.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:31 smithi166 bash[9446]: audit 2020-09-08T15:33:30.842681+0000 mon.a (mon.0) 694 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:31.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:31 smithi166 bash[9446]: cluster 2020-09-08T15:33:30.965622+0000 osd.1 (osd.1) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:31.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:31 smithi166 bash[9446]: cluster 2020-09-08T15:33:30.965765+0000 osd.1 (osd.1) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:31.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:31 smithi055 bash[9296]: audit 2020-09-08T15:33:30.834236+0000 mon.c (mon.2) 20 : audit [INF] from='client.? v1:172.21.15.166:0/1305441454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08"}]: dispatch 2020-09-08T15:33:31.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:31 smithi055 bash[9296]: audit 2020-09-08T15:33:30.834335+0000 mon.a (mon.0) 691 : audit [INF] from='client.34379 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08"}]: dispatch 2020-09-08T15:33:31.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:31 smithi055 bash[9296]: audit 2020-09-08T15:33:30.842366+0000 mon.a (mon.0) 692 : audit [INF] from='client.34379 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08"}]': finished 2020-09-08T15:33:31.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:31 smithi055 bash[9296]: cluster 2020-09-08T15:33:30.842463+0000 mon.a (mon.0) 693 : cluster [DBG] osdmap e66: 10 total, 9 up, 9 in 2020-09-08T15:33:31.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:31 smithi055 bash[9296]: audit 2020-09-08T15:33:30.842681+0000 mon.a (mon.0) 694 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:31.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:31 smithi055 bash[9296]: cluster 2020-09-08T15:33:30.965622+0000 osd.1 (osd.1) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:31.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:31 smithi055 bash[9296]: cluster 2020-09-08T15:33:30.965765+0000 osd.1 (osd.1) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:31.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:31 smithi104 bash[9203]: audit 2020-09-08T15:33:30.834236+0000 mon.c (mon.2) 20 : audit [INF] from='client.? v1:172.21.15.166:0/1305441454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08"}]: dispatch 2020-09-08T15:33:31.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:31 smithi104 bash[9203]: audit 2020-09-08T15:33:30.834335+0000 mon.a (mon.0) 691 : audit [INF] from='client.34379 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08"}]: dispatch 2020-09-08T15:33:31.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:31 smithi104 bash[9203]: audit 2020-09-08T15:33:30.842366+0000 mon.a (mon.0) 692 : audit [INF] from='client.34379 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08"}]': finished 2020-09-08T15:33:31.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:31 smithi104 bash[9203]: cluster 2020-09-08T15:33:30.842463+0000 mon.a (mon.0) 693 : cluster [DBG] osdmap e66: 10 total, 9 up, 9 in 2020-09-08T15:33:31.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:31 smithi104 bash[9203]: audit 2020-09-08T15:33:30.842681+0000 mon.a (mon.0) 694 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:31.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:31 smithi104 bash[9203]: cluster 2020-09-08T15:33:30.965622+0000 osd.1 (osd.1) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:31.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:31 smithi104 bash[9203]: cluster 2020-09-08T15:33:30.965765+0000 osd.1 (osd.1) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:32.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:32 smithi166 bash[9446]: audit 2020-09-08T15:33:31.501310+0000 mon.c (mon.2) 21 : audit [DBG] from='client.? v1:172.21.15.166:0/2916207152' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:33:32.889 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:32 smithi166 bash[9446]: cluster 2020-09-08T15:33:31.512050+0000 mgr.x (mgr.24118) 162 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:33:32.889 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:32 smithi166 bash[9446]: cluster 2020-09-08T15:33:31.844802+0000 mon.a (mon.0) 695 : cluster [DBG] osdmap e67: 10 total, 9 up, 9 in 2020-09-08T15:33:32.889 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:32 smithi166 bash[9446]: audit 2020-09-08T15:33:31.845082+0000 mon.a (mon.0) 696 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:32.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:32 smithi055 bash[9296]: audit 2020-09-08T15:33:31.501310+0000 mon.c (mon.2) 21 : audit [DBG] from='client.? v1:172.21.15.166:0/2916207152' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:33:32.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:32 smithi055 bash[9296]: cluster 2020-09-08T15:33:31.512050+0000 mgr.x (mgr.24118) 162 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:33:32.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:32 smithi055 bash[9296]: cluster 2020-09-08T15:33:31.844802+0000 mon.a (mon.0) 695 : cluster [DBG] osdmap e67: 10 total, 9 up, 9 in 2020-09-08T15:33:32.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:32 smithi055 bash[9296]: audit 2020-09-08T15:33:31.845082+0000 mon.a (mon.0) 696 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:32.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:32 smithi104 bash[9203]: audit 2020-09-08T15:33:31.501310+0000 mon.c (mon.2) 21 : audit [DBG] from='client.? v1:172.21.15.166:0/2916207152' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:33:32.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:32 smithi104 bash[9203]: cluster 2020-09-08T15:33:31.512050+0000 mgr.x (mgr.24118) 162 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:33:32.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:32 smithi104 bash[9203]: cluster 2020-09-08T15:33:31.844802+0000 mon.a (mon.0) 695 : cluster [DBG] osdmap e67: 10 total, 9 up, 9 in 2020-09-08T15:33:32.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:32 smithi104 bash[9203]: audit 2020-09-08T15:33:31.845082+0000 mon.a (mon.0) 696 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:34.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:34 smithi166 bash[9446]: cluster 2020-09-08T15:33:33.512662+0000 mgr.x (mgr.24118) 163 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:33:34.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:34 smithi166 bash[9446]: audit 2020-09-08T15:33:34.558415+0000 mon.a (mon.0) 697 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:34.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:34 smithi055 bash[9296]: cluster 2020-09-08T15:33:33.512662+0000 mgr.x (mgr.24118) 163 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:33:34.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:34 smithi055 bash[9296]: audit 2020-09-08T15:33:34.558415+0000 mon.a (mon.0) 697 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:34.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:34 smithi104 bash[9203]: cluster 2020-09-08T15:33:33.512662+0000 mgr.x (mgr.24118) 163 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:33:34.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:34 smithi104 bash[9203]: audit 2020-09-08T15:33:34.558415+0000 mon.a (mon.0) 697 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:36.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:36 smithi166 bash[9446]: cluster 2020-09-08T15:33:35.329822+0000 osd.6 (osd.6) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:36.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:36 smithi166 bash[9446]: cluster 2020-09-08T15:33:35.329999+0000 osd.6 (osd.6) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:36.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:36 smithi166 bash[9446]: cluster 2020-09-08T15:33:35.513243+0000 mgr.x (mgr.24118) 164 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:33:36.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:36 smithi166 bash[9446]: cluster 2020-09-08T15:33:35.580733+0000 mon.a (mon.0) 698 : cluster [DBG] osdmap e68: 10 total, 9 up, 9 in 2020-09-08T15:33:36.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:36 smithi166 bash[9446]: audit 2020-09-08T15:33:35.580939+0000 mon.a (mon.0) 699 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:36.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:36 smithi055 bash[9296]: cluster 2020-09-08T15:33:35.329822+0000 osd.6 (osd.6) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:36.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:36 smithi055 bash[9296]: cluster 2020-09-08T15:33:35.329999+0000 osd.6 (osd.6) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:36.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:36 smithi055 bash[9296]: cluster 2020-09-08T15:33:35.513243+0000 mgr.x (mgr.24118) 164 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:33:36.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:36 smithi055 bash[9296]: cluster 2020-09-08T15:33:35.580733+0000 mon.a (mon.0) 698 : cluster [DBG] osdmap e68: 10 total, 9 up, 9 in 2020-09-08T15:33:36.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:36 smithi055 bash[9296]: audit 2020-09-08T15:33:35.580939+0000 mon.a (mon.0) 699 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:36.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:36 smithi104 bash[9203]: cluster 2020-09-08T15:33:35.329822+0000 osd.6 (osd.6) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:36.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:36 smithi104 bash[9203]: cluster 2020-09-08T15:33:35.329999+0000 osd.6 (osd.6) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:36.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:36 smithi104 bash[9203]: cluster 2020-09-08T15:33:35.513243+0000 mgr.x (mgr.24118) 164 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:33:36.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:36 smithi104 bash[9203]: cluster 2020-09-08T15:33:35.580733+0000 mon.a (mon.0) 698 : cluster [DBG] osdmap e68: 10 total, 9 up, 9 in 2020-09-08T15:33:36.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:36 smithi104 bash[9203]: audit 2020-09-08T15:33:35.580939+0000 mon.a (mon.0) 699 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:37.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:37 smithi166 bash[9446]: cluster 2020-09-08T15:33:37.513887+0000 mgr.x (mgr.24118) 165 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:37.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:37 smithi055 bash[9296]: cluster 2020-09-08T15:33:37.513887+0000 mgr.x (mgr.24118) 165 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:37.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:37 smithi104 bash[9203]: cluster 2020-09-08T15:33:37.513887+0000 mgr.x (mgr.24118) 165 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:39.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:38 smithi166 bash[9446]: audit 2020-09-08T15:33:38.715443+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2020-09-08T15:33:39.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:38 smithi166 bash[9446]: audit 2020-09-08T15:33:38.717596+0000 mon.a (mon.0) 701 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:33:39.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:38 smithi166 bash[9446]: audit 2020-09-08T15:33:38.720032+0000 mon.a (mon.0) 702 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.9", "key": "container_image"}]: dispatch 2020-09-08T15:33:39.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:38 smithi055 bash[9296]: audit 2020-09-08T15:33:38.715443+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2020-09-08T15:33:39.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:38 smithi055 bash[9296]: audit 2020-09-08T15:33:38.717596+0000 mon.a (mon.0) 701 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:33:39.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:38 smithi055 bash[9296]: audit 2020-09-08T15:33:38.720032+0000 mon.a (mon.0) 702 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.9", "key": "container_image"}]: dispatch 2020-09-08T15:33:39.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:38 smithi104 bash[9203]: audit 2020-09-08T15:33:38.715443+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2020-09-08T15:33:39.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:38 smithi104 bash[9203]: audit 2020-09-08T15:33:38.717596+0000 mon.a (mon.0) 701 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:33:39.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:38 smithi104 bash[9203]: audit 2020-09-08T15:33:38.720032+0000 mon.a (mon.0) 702 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.9", "key": "container_image"}]: dispatch 2020-09-08T15:33:40.615 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:40 smithi166 bash[9446]: cephadm 2020-09-08T15:33:38.720173+0000 mgr.x (mgr.24118) 166 : cephadm [INF] Deploying daemon osd.9 on smithi166 2020-09-08T15:33:40.615 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:40 smithi166 bash[9446]: cluster 2020-09-08T15:33:39.514458+0000 mgr.x (mgr.24118) 167 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:40.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:40 smithi055 bash[9296]: cephadm 2020-09-08T15:33:38.720173+0000 mgr.x (mgr.24118) 166 : cephadm [INF] Deploying daemon osd.9 on smithi166 2020-09-08T15:33:40.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:40 smithi055 bash[9296]: cluster 2020-09-08T15:33:39.514458+0000 mgr.x (mgr.24118) 167 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:40.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:40 smithi104 bash[9203]: cephadm 2020-09-08T15:33:38.720173+0000 mgr.x (mgr.24118) 166 : cephadm [INF] Deploying daemon osd.9 on smithi166 2020-09-08T15:33:40.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:40 smithi104 bash[9203]: cluster 2020-09-08T15:33:39.514458+0000 mgr.x (mgr.24118) 167 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:41.140 INFO:teuthology.orchestra.run.smithi166.stdout:Created osd(s) 9 on host 'smithi166' 2020-09-08T15:33:41.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:41 smithi166 bash[9446]: audit 2020-09-08T15:33:41.139973+0000 mon.a (mon.0) 703 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}}, \"last_device_update\": \"2020-09-08T15:33:24.982955\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:41.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:41 smithi166 bash[9446]: audit 2020-09-08T15:33:41.141015+0000 mon.a (mon.0) 704 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:33:41.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:41 smithi166 bash[9446]: audit 2020-09-08T15:33:41.143636+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}}, \"last_device_update\": \"2020-09-08T15:33:24.982955\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:33:41.912 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:41 smithi055 bash[9296]: audit 2020-09-08T15:33:41.139973+0000 mon.a (mon.0) 703 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}}, \"last_device_update\": \"2020-09-08T15:33:24.982955\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:41.912 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:41 smithi055 bash[9296]: audit 2020-09-08T15:33:41.141015+0000 mon.a (mon.0) 704 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:33:41.912 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:41 smithi055 bash[9296]: audit 2020-09-08T15:33:41.143636+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}}, \"last_device_update\": \"2020-09-08T15:33:24.982955\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:33:41.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:41 smithi104 bash[9203]: audit 2020-09-08T15:33:41.139973+0000 mon.a (mon.0) 703 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}}, \"last_device_update\": \"2020-09-08T15:33:24.982955\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:41.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:41 smithi104 bash[9203]: audit 2020-09-08T15:33:41.141015+0000 mon.a (mon.0) 704 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:33:41.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:41 smithi104 bash[9203]: audit 2020-09-08T15:33:41.143636+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.479960\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:16.480145\", \"created\": \"2020-09-08T15:33:15.799848\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}}, \"last_device_update\": \"2020-09-08T15:33:24.982955\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:33:42.479 INFO:teuthology.orchestra.run.smithi166:osd.9> sudo journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.9.service 2020-09-08T15:33:42.482 INFO:tasks.cephadm:Deploying osd.10 on smithi166 with /dev/vg_nvme/lv_2... 2020-09-08T15:33:42.483 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2020-09-08T15:33:42.568 INFO:journalctl@ceph.osd.9.smithi166.stdout:-- Logs begin at Tue 2020-09-01 16:49:30 UTC. -- 2020-09-08T15:33:42.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:42 smithi166 bash[9446]: cluster 2020-09-08T15:33:41.514886+0000 mgr.x (mgr.24118) 168 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:42.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:42 smithi166 bash[9446]: audit 2020-09-08T15:33:41.958450+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956337\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956546\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.957138\", \"created\": \"2020-09-08T15:33:41.114066\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}}, \"last_daemon_update\": \"2020-09-08T15:33:41.957678\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:42.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:42 smithi166 bash[9446]: audit 2020-09-08T15:33:41.965550+0000 mon.a (mon.0) 707 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956337\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956546\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.957138\", \"created\": \"2020-09-08T15:33:41.114066\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}}, \"last_daemon_update\": \"2020-09-08T15:33:41.957678\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:33:42.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:42 smithi166 bash[9446]: audit 2020-09-08T15:33:41.967577+0000 mon.a (mon.0) 708 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:42.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:42 smithi055 bash[9296]: cluster 2020-09-08T15:33:41.514886+0000 mgr.x (mgr.24118) 168 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:42.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:42 smithi055 bash[9296]: audit 2020-09-08T15:33:41.958450+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956337\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956546\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.957138\", \"created\": \"2020-09-08T15:33:41.114066\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}}, \"last_daemon_update\": \"2020-09-08T15:33:41.957678\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:42.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:42 smithi055 bash[9296]: audit 2020-09-08T15:33:41.965550+0000 mon.a (mon.0) 707 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956337\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956546\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.957138\", \"created\": \"2020-09-08T15:33:41.114066\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}}, \"last_daemon_update\": \"2020-09-08T15:33:41.957678\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:33:42.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:42 smithi055 bash[9296]: audit 2020-09-08T15:33:41.967577+0000 mon.a (mon.0) 708 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:42.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:42 smithi104 bash[9203]: cluster 2020-09-08T15:33:41.514886+0000 mgr.x (mgr.24118) 168 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:42.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:42 smithi104 bash[9203]: audit 2020-09-08T15:33:41.958450+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956337\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956546\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.957138\", \"created\": \"2020-09-08T15:33:41.114066\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}}, \"last_daemon_update\": \"2020-09-08T15:33:41.957678\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:42.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:42 smithi104 bash[9203]: audit 2020-09-08T15:33:41.965550+0000 mon.a (mon.0) 707 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956337\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956546\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.957138\", \"created\": \"2020-09-08T15:33:41.114066\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}}, \"last_daemon_update\": \"2020-09-08T15:33:41.957678\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:33:42.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:42 smithi104 bash[9203]: audit 2020-09-08T15:33:41.967577+0000 mon.a (mon.0) 708 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:44.304 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:44 smithi166 bash[14254]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2020-09-08T15:33:44.305 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:44 smithi166 bash[14254]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-9 --no-mon-config 2020-09-08T15:33:44.319 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:44 smithi166 bash[14254]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-9/block 2020-09-08T15:33:44.319 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:44 smithi166 bash[14254]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-9/block 2020-09-08T15:33:44.323 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:44 smithi166 bash[14254]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2020-09-08T15:33:44.323 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:44 smithi166 bash[14254]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2020-09-08T15:33:44.324 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:44 smithi166 bash[14254]: --> ceph-volume lvm activate successful for osd ID: 9 2020-09-08T15:33:44.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:44 smithi166 bash[9446]: cluster 2020-09-08T15:33:43.309737+0000 osd.4 (osd.4) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:44.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:44 smithi166 bash[9446]: cluster 2020-09-08T15:33:43.309895+0000 osd.4 (osd.4) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:44.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:44 smithi166 bash[9446]: cluster 2020-09-08T15:33:43.515572+0000 mgr.x (mgr.24118) 169 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:44.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:44 smithi166 bash[9446]: cluster 2020-09-08T15:33:43.533011+0000 mon.a (mon.0) 709 : cluster [DBG] osdmap e69: 10 total, 9 up, 9 in 2020-09-08T15:33:44.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:44 smithi166 bash[9446]: audit 2020-09-08T15:33:43.533271+0000 mon.a (mon.0) 710 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:44.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:44 smithi055 bash[9296]: cluster 2020-09-08T15:33:43.309737+0000 osd.4 (osd.4) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:44.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:44 smithi055 bash[9296]: cluster 2020-09-08T15:33:43.309895+0000 osd.4 (osd.4) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:44.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:44 smithi055 bash[9296]: cluster 2020-09-08T15:33:43.515572+0000 mgr.x (mgr.24118) 169 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:44.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:44 smithi055 bash[9296]: cluster 2020-09-08T15:33:43.533011+0000 mon.a (mon.0) 709 : cluster [DBG] osdmap e69: 10 total, 9 up, 9 in 2020-09-08T15:33:44.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:44 smithi055 bash[9296]: audit 2020-09-08T15:33:43.533271+0000 mon.a (mon.0) 710 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:44.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:44 smithi104 bash[9203]: cluster 2020-09-08T15:33:43.309737+0000 osd.4 (osd.4) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:44.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:44 smithi104 bash[9203]: cluster 2020-09-08T15:33:43.309895+0000 osd.4 (osd.4) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:44.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:44 smithi104 bash[9203]: cluster 2020-09-08T15:33:43.515572+0000 mgr.x (mgr.24118) 169 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:44.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:44 smithi104 bash[9203]: cluster 2020-09-08T15:33:43.533011+0000 mon.a (mon.0) 709 : cluster [DBG] osdmap e69: 10 total, 9 up, 9 in 2020-09-08T15:33:44.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:44 smithi104 bash[9203]: audit 2020-09-08T15:33:43.533271+0000 mon.a (mon.0) 710 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:45.865 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:45 smithi166 bash[14254]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:33:45.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:45 smithi166 bash[9446]: audit 2020-09-08T15:33:44.707361+0000 mon.a (mon.0) 711 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:33:45.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:45 smithi166 bash[9446]: audit 2020-09-08T15:33:44.713276+0000 mon.a (mon.0) 712 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:33:45.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:45 smithi055 bash[9296]: audit 2020-09-08T15:33:44.707361+0000 mon.a (mon.0) 711 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:33:45.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:45 smithi055 bash[9296]: audit 2020-09-08T15:33:44.713276+0000 mon.a (mon.0) 712 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:33:45.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:45 smithi104 bash[9203]: audit 2020-09-08T15:33:44.707361+0000 mon.a (mon.0) 711 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:33:45.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:45 smithi104 bash[9203]: audit 2020-09-08T15:33:44.713276+0000 mon.a (mon.0) 712 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:33:46.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:46 smithi166 bash[9446]: cluster 2020-09-08T15:33:45.516232+0000 mgr.x (mgr.24118) 170 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:46.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:46 smithi055 bash[9296]: cluster 2020-09-08T15:33:45.516232+0000 mgr.x (mgr.24118) 170 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:46.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:46 smithi104 bash[9203]: cluster 2020-09-08T15:33:45.516232+0000 mgr.x (mgr.24118) 170 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 49 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:47.844 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc426972700 10 -- 172.21.15.166:0/2653425694 dispatch_throttle_release 278 to dispatch throttler 278/104857600 2020-09-08T15:33:47.844 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc426972700 20 -- 172.21.15.166:0/2653425694 done calling dispatch on 0x55df00b98c60 2020-09-08T15:33:47.844 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 -- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 legacy=0x55df00c02800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).process 2020-09-08T15:33:47.845 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 -- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 legacy=0x55df00c02800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read continue len=1 2020-09-08T15:33:47.845 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=OPENED pgs=133 cs=1 l=1).handle_message r=0 2020-09-08T15:33:47.845 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=OPENED pgs=133 cs=1 l=1).handle_message process tag 7 2020-09-08T15:33:47.846 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=OPENED pgs=133 cs=1 l=1).handle_message begin MSG 2020-09-08T15:33:47.846 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 -- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 legacy=0x55df00c02800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read start len=53 2020-09-08T15:33:47.846 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=OPENED pgs=133 cs=1 l=1).handle_message_header r=0 2020-09-08T15:33:47.846 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=OPENED pgs=133 cs=1 l=1).handle_message_header got MSG header 2020-09-08T15:33:47.847 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=OPENED pgs=133 cs=1 l=1).handle_message_header got envelope type=18 src mon.2 front=194 data=0 off 0 2020-09-08T15:33:47.847 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=THROTTLE_MESSAGE pgs=133 cs=1 l=1).throttle_message 2020-09-08T15:33:47.847 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=THROTTLE_BYTES pgs=133 cs=1 l=1).throttle_bytes 2020-09-08T15:33:47.847 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=THROTTLE_DISPATCH_QUEUE pgs=133 cs=1 l=1).throttle_dispatch_queue 2020-09-08T15:33:47.848 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=READ_MESSAGE_FRONT pgs=133 cs=1 l=1).read_message_front 2020-09-08T15:33:47.848 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 -- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 legacy=0x55df00c02800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read start len=194 2020-09-08T15:33:47.848 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=READ_MESSAGE_FRONT pgs=133 cs=1 l=1).handle_message_front r=0 2020-09-08T15:33:47.849 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=READ_MESSAGE_FRONT pgs=133 cs=1 l=1).handle_message_front got front 194 2020-09-08T15:33:47.849 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=READ_MESSAGE_FRONT pgs=133 cs=1 l=1).read_message_middle 2020-09-08T15:33:47.849 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=READ_MESSAGE_FRONT pgs=133 cs=1 l=1).read_message_data_prepare 2020-09-08T15:33:47.849 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=READ_MESSAGE_FRONT pgs=133 cs=1 l=1).read_message_data msg_left=0 2020-09-08T15:33:47.850 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=READ_MESSAGE_FRONT pgs=133 cs=1 l=1).read_message_footer 2020-09-08T15:33:47.850 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 -- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 legacy=0x55df00c02800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read start len=21 2020-09-08T15:33:47.850 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=READ_FOOTER_AND_DISPATCH pgs=133 cs=1 l=1).handle_message_footer r=0 2020-09-08T15:33:47.850 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 10 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=READ_FOOTER_AND_DISPATCH pgs=133 cs=1 l=1).handle_message_footer aborted = 0 2020-09-08T15:33:47.850 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=READ_FOOTER_AND_DISPATCH pgs=133 cs=1 l=1).handle_message_footer got 194 + 0 + 0 byte message 2020-09-08T15:33:47.851 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 10 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=READ_FOOTER_AND_DISPATCH pgs=133 cs=1 l=1).handle_message_footer no session security set 2020-09-08T15:33:47.851 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 5 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=READ_FOOTER_AND_DISPATCH pgs=133 cs=1 l=1). rx mon.2 seq 6 0x55df00ba24e0 auth_reply(proto 2 0 (0) Success) v1 2020-09-08T15:33:47.851 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 -- 172.21.15.166:0/2653425694 queue 0x55df00ba24e0 prio 196 2020-09-08T15:33:47.851 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 --1- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 0x55df00c02800 :-1 s=OPENED pgs=133 cs=1 l=1).wait_message 2020-09-08T15:33:47.852 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc428175700 20 -- 172.21.15.166:0/2653425694 >> v1:172.21.15.166:6789/0 conn(0x55df00c19400 legacy=0x55df00c02800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read start len=1 2020-09-08T15:33:47.852 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc426972700 1 -- 172.21.15.166:0/2653425694 <== mon.2 v1:172.21.15.166:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 851617617 0 0) 0x55df00ba24e0 con 0x55df00c19400 2020-09-08T15:33:47.852 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc426972700 10 -- 172.21.15.166:0/2653425694 dispatch_throttle_release 194 to dispatch throttler 194/104857600 2020-09-08T15:33:47.852 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:47 smithi166 bash[14254]: debug 2020-09-08T15:33:47.609+0000 7fc426972700 20 -- 172.21.15.166:0/2653425694 done calling dispatch on 0x55df00ba24e0 2020-09-08T15:33:48.537 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:48 smithi166 bash[14254]: debug 2020-09-08T15:33:48.161+0000 7fc42e3f2f40 -1 Falling back to public interface 2020-09-08T15:33:48.675 INFO:teuthology.orchestra.run.smithi166.stderr:--> Zapping: /dev/vg_nvme/lv_2 2020-09-08T15:33:48.675 INFO:teuthology.orchestra.run.smithi166.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2020-09-08T15:33:48.676 INFO:teuthology.orchestra.run.smithi166.stderr: stderr: 10+0 records in 2020-09-08T15:33:48.676 INFO:teuthology.orchestra.run.smithi166.stderr:10+0 records out 2020-09-08T15:33:48.676 INFO:teuthology.orchestra.run.smithi166.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123227 s, 851 MB/s 2020-09-08T15:33:48.676 INFO:teuthology.orchestra.run.smithi166.stderr:--> Zapping successful for: 2020-09-08T15:33:48.790 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:48 smithi166 bash[9446]: cluster 2020-09-08T15:33:47.516873+0000 mgr.x (mgr.24118) 171 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 0 B data, 50 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:48.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:48 smithi055 bash[9296]: cluster 2020-09-08T15:33:47.516873+0000 mgr.x (mgr.24118) 171 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 0 B data, 50 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:48.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:48 smithi104 bash[9203]: cluster 2020-09-08T15:33:47.516873+0000 mgr.x (mgr.24118) 171 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 0 B data, 50 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:49.118 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:48 smithi166 bash[14254]: debug 2020-09-08T15:33:48.789+0000 7fc42e3f2f40 -1 osd.9 0 log_to_monitors {default=true} 2020-09-08T15:33:49.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:49 smithi166 bash[9446]: audit 2020-09-08T15:33:48.797167+0000 mon.c (mon.2) 22 : audit [INF] from='osd.9 v1:172.21.15.166:6809/1458870080' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2020-09-08T15:33:49.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:49 smithi166 bash[9446]: audit 2020-09-08T15:33:48.798037+0000 mon.a (mon.0) 713 : audit [INF] from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2020-09-08T15:33:49.640 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph orch daemon add osd smithi166:vg_nvme/lv_2 2020-09-08T15:33:49.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:49 smithi055 bash[9296]: audit 2020-09-08T15:33:48.797167+0000 mon.c (mon.2) 22 : audit [INF] from='osd.9 v1:172.21.15.166:6809/1458870080' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2020-09-08T15:33:49.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:49 smithi055 bash[9296]: audit 2020-09-08T15:33:48.798037+0000 mon.a (mon.0) 713 : audit [INF] from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2020-09-08T15:33:49.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:49 smithi104 bash[9203]: audit 2020-09-08T15:33:48.797167+0000 mon.c (mon.2) 22 : audit [INF] from='osd.9 v1:172.21.15.166:6809/1458870080' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2020-09-08T15:33:49.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:49 smithi104 bash[9203]: audit 2020-09-08T15:33:48.798037+0000 mon.a (mon.0) 713 : audit [INF] from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2020-09-08T15:33:50.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:50 smithi166 bash[14254]: debug 2020-09-08T15:33:50.560+0000 7fc42496e700 -1 osd.9 0 waiting for initial osdmap 2020-09-08T15:33:50.867 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:33:50 smithi166 bash[14254]: debug 2020-09-08T15:33:50.560+0000 7fc412a75700 -1 osd.9 0 failed to load OSD map for epoch 10, got 0 bytes 2020-09-08T15:33:50.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:50 smithi166 bash[9446]: cluster 2020-09-08T15:33:49.517476+0000 mgr.x (mgr.24118) 172 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 50 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:50.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:50 smithi166 bash[9446]: audit 2020-09-08T15:33:49.547649+0000 mon.a (mon.0) 714 : audit [INF] from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2020-09-08T15:33:50.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:50 smithi166 bash[9446]: cluster 2020-09-08T15:33:49.547733+0000 mon.a (mon.0) 715 : cluster [DBG] osdmap e70: 10 total, 9 up, 9 in 2020-09-08T15:33:50.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:50 smithi166 bash[9446]: audit 2020-09-08T15:33:49.548557+0000 mon.a (mon.0) 716 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:50.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:50 smithi166 bash[9446]: audit 2020-09-08T15:33:49.549506+0000 mon.c (mon.2) 23 : audit [INF] from='osd.9 v1:172.21.15.166:6809/1458870080' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:33:50.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:50 smithi166 bash[9446]: audit 2020-09-08T15:33:49.556629+0000 mon.a (mon.0) 717 : audit [INF] from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:33:50.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:50 smithi166 bash[9446]: audit 2020-09-08T15:33:50.329676+0000 mon.a (mon.0) 718 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:33:50.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:50 smithi055 bash[9296]: cluster 2020-09-08T15:33:49.517476+0000 mgr.x (mgr.24118) 172 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 50 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:50.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:50 smithi055 bash[9296]: audit 2020-09-08T15:33:49.547649+0000 mon.a (mon.0) 714 : audit [INF] from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2020-09-08T15:33:50.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:50 smithi055 bash[9296]: cluster 2020-09-08T15:33:49.547733+0000 mon.a (mon.0) 715 : cluster [DBG] osdmap e70: 10 total, 9 up, 9 in 2020-09-08T15:33:50.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:50 smithi055 bash[9296]: audit 2020-09-08T15:33:49.548557+0000 mon.a (mon.0) 716 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:50.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:50 smithi055 bash[9296]: audit 2020-09-08T15:33:49.549506+0000 mon.c (mon.2) 23 : audit [INF] from='osd.9 v1:172.21.15.166:6809/1458870080' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:33:50.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:50 smithi055 bash[9296]: audit 2020-09-08T15:33:49.556629+0000 mon.a (mon.0) 717 : audit [INF] from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:33:50.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:50 smithi055 bash[9296]: audit 2020-09-08T15:33:50.329676+0000 mon.a (mon.0) 718 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:33:50.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:50 smithi104 bash[9203]: cluster 2020-09-08T15:33:49.517476+0000 mgr.x (mgr.24118) 172 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 50 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:50.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:50 smithi104 bash[9203]: audit 2020-09-08T15:33:49.547649+0000 mon.a (mon.0) 714 : audit [INF] from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2020-09-08T15:33:50.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:50 smithi104 bash[9203]: cluster 2020-09-08T15:33:49.547733+0000 mon.a (mon.0) 715 : cluster [DBG] osdmap e70: 10 total, 9 up, 9 in 2020-09-08T15:33:50.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:50 smithi104 bash[9203]: audit 2020-09-08T15:33:49.548557+0000 mon.a (mon.0) 716 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:50.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:50 smithi104 bash[9203]: audit 2020-09-08T15:33:49.549506+0000 mon.c (mon.2) 23 : audit [INF] from='osd.9 v1:172.21.15.166:6809/1458870080' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:33:50.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:50 smithi104 bash[9203]: audit 2020-09-08T15:33:49.556629+0000 mon.a (mon.0) 717 : audit [INF] from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:33:50.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:50 smithi104 bash[9203]: audit 2020-09-08T15:33:50.329676+0000 mon.a (mon.0) 718 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:33:51.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:51 smithi166 bash[9446]: audit 2020-09-08T15:33:50.558006+0000 mon.a (mon.0) 719 : audit [INF] from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]': finished 2020-09-08T15:33:51.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:51 smithi166 bash[9446]: cluster 2020-09-08T15:33:50.558110+0000 mon.a (mon.0) 720 : cluster [DBG] osdmap e71: 10 total, 9 up, 9 in 2020-09-08T15:33:51.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:51 smithi166 bash[9446]: audit 2020-09-08T15:33:50.558845+0000 mon.a (mon.0) 721 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:51.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:51 smithi166 bash[9446]: audit 2020-09-08T15:33:50.566438+0000 mon.a (mon.0) 722 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:51.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:51 smithi166 bash[9446]: audit 2020-09-08T15:33:50.622439+0000 mon.a (mon.0) 723 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956337\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956546\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.957138\", \"created\": \"2020-09-08T15:33:41.114066\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}}, \"last_daemon_update\": \"2020-09-08T15:33:41.957678\", \"last_device_update\": \"2020-09-08T15:33:50.621089\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:51.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:51 smithi166 bash[9446]: audit 2020-09-08T15:33:50.628264+0000 mon.a (mon.0) 724 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956337\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956546\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.957138\", \"created\": \"2020-09-08T15:33:41.114066\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}}, \"last_daemon_update\": \"2020-09-08T15:33:41.957678\", \"last_device_update\": \"2020-09-08T15:33:50.621089\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:33:51.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:51 smithi166 bash[9446]: audit 2020-09-08T15:33:50.630522+0000 mon.a (mon.0) 725 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:51.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:51 smithi166 bash[9446]: audit 2020-09-08T15:33:50.635599+0000 mon.a (mon.0) 726 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:33:51.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:51 smithi166 bash[9446]: audit 2020-09-08T15:33:50.641280+0000 mon.a (mon.0) 727 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:51.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:51 smithi166 bash[9446]: audit 2020-09-08T15:33:50.646600+0000 mon.a (mon.0) 728 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:33:51.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:51 smithi055 bash[9296]: audit 2020-09-08T15:33:50.558006+0000 mon.a (mon.0) 719 : audit [INF] from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]': finished 2020-09-08T15:33:51.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:51 smithi055 bash[9296]: cluster 2020-09-08T15:33:50.558110+0000 mon.a (mon.0) 720 : cluster [DBG] osdmap e71: 10 total, 9 up, 9 in 2020-09-08T15:33:51.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:51 smithi055 bash[9296]: audit 2020-09-08T15:33:50.558845+0000 mon.a (mon.0) 721 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:51.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:51 smithi055 bash[9296]: audit 2020-09-08T15:33:50.566438+0000 mon.a (mon.0) 722 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:51.912 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:51 smithi055 bash[9296]: audit 2020-09-08T15:33:50.622439+0000 mon.a (mon.0) 723 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956337\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956546\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.957138\", \"created\": \"2020-09-08T15:33:41.114066\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}}, \"last_daemon_update\": \"2020-09-08T15:33:41.957678\", \"last_device_update\": \"2020-09-08T15:33:50.621089\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:51.912 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:51 smithi055 bash[9296]: audit 2020-09-08T15:33:50.628264+0000 mon.a (mon.0) 724 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956337\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956546\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.957138\", \"created\": \"2020-09-08T15:33:41.114066\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}}, \"last_daemon_update\": \"2020-09-08T15:33:41.957678\", \"last_device_update\": \"2020-09-08T15:33:50.621089\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:33:51.912 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:51 smithi055 bash[9296]: audit 2020-09-08T15:33:50.630522+0000 mon.a (mon.0) 725 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:51.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:51 smithi055 bash[9296]: audit 2020-09-08T15:33:50.635599+0000 mon.a (mon.0) 726 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:33:51.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:51 smithi055 bash[9296]: audit 2020-09-08T15:33:50.641280+0000 mon.a (mon.0) 727 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:51.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:51 smithi055 bash[9296]: audit 2020-09-08T15:33:50.646600+0000 mon.a (mon.0) 728 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:33:51.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:51 smithi104 bash[9203]: audit 2020-09-08T15:33:50.558006+0000 mon.a (mon.0) 719 : audit [INF] from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]': finished 2020-09-08T15:33:51.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:51 smithi104 bash[9203]: cluster 2020-09-08T15:33:50.558110+0000 mon.a (mon.0) 720 : cluster [DBG] osdmap e71: 10 total, 9 up, 9 in 2020-09-08T15:33:51.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:51 smithi104 bash[9203]: audit 2020-09-08T15:33:50.558845+0000 mon.a (mon.0) 721 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:51.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:51 smithi104 bash[9203]: audit 2020-09-08T15:33:50.566438+0000 mon.a (mon.0) 722 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:51.934 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:51 smithi104 bash[9203]: audit 2020-09-08T15:33:50.622439+0000 mon.a (mon.0) 723 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956337\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956546\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.957138\", \"created\": \"2020-09-08T15:33:41.114066\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}}, \"last_daemon_update\": \"2020-09-08T15:33:41.957678\", \"last_device_update\": \"2020-09-08T15:33:50.621089\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:33:51.934 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:51 smithi104 bash[9203]: audit 2020-09-08T15:33:50.628264+0000 mon.a (mon.0) 724 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956337\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956546\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.957138\", \"created\": \"2020-09-08T15:33:41.114066\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}}, \"last_daemon_update\": \"2020-09-08T15:33:41.957678\", \"last_device_update\": \"2020-09-08T15:33:50.621089\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:33:51.934 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:51 smithi104 bash[9203]: audit 2020-09-08T15:33:50.630522+0000 mon.a (mon.0) 725 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:51.934 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:51 smithi104 bash[9203]: audit 2020-09-08T15:33:50.635599+0000 mon.a (mon.0) 726 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:33:51.935 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:51 smithi104 bash[9203]: audit 2020-09-08T15:33:50.641280+0000 mon.a (mon.0) 727 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:33:51.935 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:51 smithi104 bash[9203]: audit 2020-09-08T15:33:50.646600+0000 mon.a (mon.0) 728 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:33:52.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:52 smithi166 bash[9446]: cluster 2020-09-08T15:33:49.783385+0000 osd.9 (osd.9) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:52.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:52 smithi166 bash[9446]: cluster 2020-09-08T15:33:49.783570+0000 osd.9 (osd.9) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:52.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:52 smithi166 bash[9446]: cluster 2020-09-08T15:33:51.518072+0000 mgr.x (mgr.24118) 173 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 50 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:52.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:52 smithi166 bash[9446]: cluster 2020-09-08T15:33:51.568653+0000 mon.a (mon.0) 729 : cluster [INF] osd.9 v1:172.21.15.166:6809/1458870080 boot 2020-09-08T15:33:52.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:52 smithi166 bash[9446]: cluster 2020-09-08T15:33:51.568736+0000 mon.a (mon.0) 730 : cluster [DBG] osdmap e72: 10 total, 10 up, 10 in 2020-09-08T15:33:52.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:52 smithi166 bash[9446]: audit 2020-09-08T15:33:51.569506+0000 mon.a (mon.0) 731 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:52.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:52 smithi055 bash[9296]: cluster 2020-09-08T15:33:49.783385+0000 osd.9 (osd.9) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:52.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:52 smithi055 bash[9296]: cluster 2020-09-08T15:33:49.783570+0000 osd.9 (osd.9) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:52.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:52 smithi055 bash[9296]: cluster 2020-09-08T15:33:51.518072+0000 mgr.x (mgr.24118) 173 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 50 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:52.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:52 smithi055 bash[9296]: cluster 2020-09-08T15:33:51.568653+0000 mon.a (mon.0) 729 : cluster [INF] osd.9 v1:172.21.15.166:6809/1458870080 boot 2020-09-08T15:33:52.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:52 smithi055 bash[9296]: cluster 2020-09-08T15:33:51.568736+0000 mon.a (mon.0) 730 : cluster [DBG] osdmap e72: 10 total, 10 up, 10 in 2020-09-08T15:33:52.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:52 smithi055 bash[9296]: audit 2020-09-08T15:33:51.569506+0000 mon.a (mon.0) 731 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:52.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:52 smithi104 bash[9203]: cluster 2020-09-08T15:33:49.783385+0000 osd.9 (osd.9) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:52.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:52 smithi104 bash[9203]: cluster 2020-09-08T15:33:49.783570+0000 osd.9 (osd.9) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:52.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:52 smithi104 bash[9203]: cluster 2020-09-08T15:33:51.518072+0000 mgr.x (mgr.24118) 173 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 50 MiB used, 805 GiB / 805 GiB avail 2020-09-08T15:33:52.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:52 smithi104 bash[9203]: cluster 2020-09-08T15:33:51.568653+0000 mon.a (mon.0) 729 : cluster [INF] osd.9 v1:172.21.15.166:6809/1458870080 boot 2020-09-08T15:33:52.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:52 smithi104 bash[9203]: cluster 2020-09-08T15:33:51.568736+0000 mon.a (mon.0) 730 : cluster [DBG] osdmap e72: 10 total, 10 up, 10 in 2020-09-08T15:33:52.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:52 smithi104 bash[9203]: audit 2020-09-08T15:33:51.569506+0000 mon.a (mon.0) 731 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2020-09-08T15:33:53.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:53 smithi055 bash[9296]: cluster 2020-09-08T15:33:52.651752+0000 mon.a (mon.0) 732 : cluster [DBG] osdmap e73: 10 total, 10 up, 10 in 2020-09-08T15:33:53.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:53 smithi055 bash[9296]: audit 2020-09-08T15:33:53.077386+0000 mgr.x (mgr.24118) 174 : audit [DBG] from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi166:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:33:53.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:53 smithi055 bash[9296]: audit 2020-09-08T15:33:53.078947+0000 mon.a (mon.0) 733 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:33:53.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:53 smithi055 bash[9296]: cephadm 2020-09-08T15:33:53.081129+0000 mgr.x (mgr.24118) 175 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:33:53.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:53 smithi055 bash[9296]: cephadm 2020-09-08T15:33:53.081312+0000 mgr.x (mgr.24118) 176 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:33:53.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:53 smithi055 bash[9296]: cephadm 2020-09-08T15:33:53.082391+0000 mgr.x (mgr.24118) 177 : cephadm [INF] Applying None on host smithi166... 2020-09-08T15:33:53.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:53 smithi055 bash[9296]: audit 2020-09-08T15:33:53.082848+0000 mon.a (mon.0) 734 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:33:53.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:53 smithi055 bash[9296]: audit 2020-09-08T15:33:53.084999+0000 mon.a (mon.0) 735 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:33:53.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:53 smithi055 bash[9296]: audit 2020-09-08T15:33:53.087322+0000 mon.a (mon.0) 736 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:53.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:53 smithi055 bash[9296]: cluster 2020-09-08T15:33:53.518687+0000 mgr.x (mgr.24118) 178 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 55 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:33:53.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:53 smithi104 bash[9203]: cluster 2020-09-08T15:33:52.651752+0000 mon.a (mon.0) 732 : cluster [DBG] osdmap e73: 10 total, 10 up, 10 in 2020-09-08T15:33:53.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:53 smithi104 bash[9203]: audit 2020-09-08T15:33:53.077386+0000 mgr.x (mgr.24118) 174 : audit [DBG] from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi166:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:33:53.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:53 smithi104 bash[9203]: audit 2020-09-08T15:33:53.078947+0000 mon.a (mon.0) 733 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:33:53.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:53 smithi104 bash[9203]: cephadm 2020-09-08T15:33:53.081129+0000 mgr.x (mgr.24118) 175 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:33:53.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:53 smithi104 bash[9203]: cephadm 2020-09-08T15:33:53.081312+0000 mgr.x (mgr.24118) 176 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:33:53.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:53 smithi104 bash[9203]: cephadm 2020-09-08T15:33:53.082391+0000 mgr.x (mgr.24118) 177 : cephadm [INF] Applying None on host smithi166... 2020-09-08T15:33:53.933 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:53 smithi104 bash[9203]: audit 2020-09-08T15:33:53.082848+0000 mon.a (mon.0) 734 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:33:53.933 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:53 smithi104 bash[9203]: audit 2020-09-08T15:33:53.084999+0000 mon.a (mon.0) 735 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:33:53.933 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:53 smithi104 bash[9203]: audit 2020-09-08T15:33:53.087322+0000 mon.a (mon.0) 736 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:53.937 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:53 smithi104 bash[9203]: cluster 2020-09-08T15:33:53.518687+0000 mgr.x (mgr.24118) 178 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 55 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:33:54.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:53 smithi166 bash[9446]: cluster 2020-09-08T15:33:52.651752+0000 mon.a (mon.0) 732 : cluster [DBG] osdmap e73: 10 total, 10 up, 10 in 2020-09-08T15:33:54.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:53 smithi166 bash[9446]: audit 2020-09-08T15:33:53.077386+0000 mgr.x (mgr.24118) 174 : audit [DBG] from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi166:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:33:54.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:53 smithi166 bash[9446]: audit 2020-09-08T15:33:53.078947+0000 mon.a (mon.0) 733 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:33:54.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:53 smithi166 bash[9446]: cephadm 2020-09-08T15:33:53.081129+0000 mgr.x (mgr.24118) 175 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:33:54.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:53 smithi166 bash[9446]: cephadm 2020-09-08T15:33:53.081312+0000 mgr.x (mgr.24118) 176 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:33:54.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:53 smithi166 bash[9446]: cephadm 2020-09-08T15:33:53.082391+0000 mgr.x (mgr.24118) 177 : cephadm [INF] Applying None on host smithi166... 2020-09-08T15:33:54.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:53 smithi166 bash[9446]: audit 2020-09-08T15:33:53.082848+0000 mon.a (mon.0) 734 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:33:54.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:53 smithi166 bash[9446]: audit 2020-09-08T15:33:53.084999+0000 mon.a (mon.0) 735 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:33:54.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:53 smithi166 bash[9446]: audit 2020-09-08T15:33:53.087322+0000 mon.a (mon.0) 736 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:33:54.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:53 smithi166 bash[9446]: cluster 2020-09-08T15:33:53.518687+0000 mgr.x (mgr.24118) 178 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 55 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:33:55.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:55 smithi166 bash[9446]: cluster 2020-09-08T15:33:54.913386+0000 osd.2 (osd.2) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:55.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:55 smithi166 bash[9446]: cluster 2020-09-08T15:33:54.913511+0000 osd.2 (osd.2) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:55.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:55 smithi055 bash[9296]: cluster 2020-09-08T15:33:54.913386+0000 osd.2 (osd.2) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:55.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:55 smithi055 bash[9296]: cluster 2020-09-08T15:33:54.913511+0000 osd.2 (osd.2) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:55.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:55 smithi104 bash[9203]: cluster 2020-09-08T15:33:54.913386+0000 osd.2 (osd.2) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:33:55.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:55 smithi104 bash[9203]: cluster 2020-09-08T15:33:54.913511+0000 osd.2 (osd.2) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:33:56.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:56 smithi166 bash[9446]: cluster 2020-09-08T15:33:55.510353+0000 mon.a (mon.0) 737 : cluster [DBG] osdmap e74: 10 total, 10 up, 10 in 2020-09-08T15:33:56.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:56 smithi166 bash[9446]: cluster 2020-09-08T15:33:55.519348+0000 mgr.x (mgr.24118) 179 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 0 B data, 55 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:33:56.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:56 smithi055 bash[9296]: cluster 2020-09-08T15:33:55.510353+0000 mon.a (mon.0) 737 : cluster [DBG] osdmap e74: 10 total, 10 up, 10 in 2020-09-08T15:33:56.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:56 smithi055 bash[9296]: cluster 2020-09-08T15:33:55.519348+0000 mgr.x (mgr.24118) 179 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 0 B data, 55 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:33:56.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:56 smithi104 bash[9203]: cluster 2020-09-08T15:33:55.510353+0000 mon.a (mon.0) 737 : cluster [DBG] osdmap e74: 10 total, 10 up, 10 in 2020-09-08T15:33:56.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:56 smithi104 bash[9203]: cluster 2020-09-08T15:33:55.519348+0000 mgr.x (mgr.24118) 179 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 0 B data, 55 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:33:57.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:57 smithi166 bash[9446]: audit 2020-09-08T15:33:57.061931+0000 mon.c (mon.2) 24 : audit [INF] from='client.? v1:172.21.15.166:0/4044606008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c194210e-bd89-4a1f-98fb-6bf44f0b324c"}]: dispatch 2020-09-08T15:33:57.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:57 smithi166 bash[9446]: audit 2020-09-08T15:33:57.063610+0000 mon.a (mon.0) 738 : audit [INF] from='client.34406 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c194210e-bd89-4a1f-98fb-6bf44f0b324c"}]: dispatch 2020-09-08T15:33:57.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:57 smithi166 bash[9446]: audit 2020-09-08T15:33:57.071946+0000 mon.a (mon.0) 739 : audit [INF] from='client.34406 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c194210e-bd89-4a1f-98fb-6bf44f0b324c"}]': finished 2020-09-08T15:33:57.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:57 smithi166 bash[9446]: cluster 2020-09-08T15:33:57.072033+0000 mon.a (mon.0) 740 : cluster [DBG] osdmap e75: 11 total, 10 up, 10 in 2020-09-08T15:33:57.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:57 smithi166 bash[9446]: audit 2020-09-08T15:33:57.072282+0000 mon.a (mon.0) 741 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:33:57.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:57 smithi055 bash[9296]: audit 2020-09-08T15:33:57.061931+0000 mon.c (mon.2) 24 : audit [INF] from='client.? v1:172.21.15.166:0/4044606008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c194210e-bd89-4a1f-98fb-6bf44f0b324c"}]: dispatch 2020-09-08T15:33:57.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:57 smithi055 bash[9296]: audit 2020-09-08T15:33:57.063610+0000 mon.a (mon.0) 738 : audit [INF] from='client.34406 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c194210e-bd89-4a1f-98fb-6bf44f0b324c"}]: dispatch 2020-09-08T15:33:57.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:57 smithi055 bash[9296]: audit 2020-09-08T15:33:57.071946+0000 mon.a (mon.0) 739 : audit [INF] from='client.34406 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c194210e-bd89-4a1f-98fb-6bf44f0b324c"}]': finished 2020-09-08T15:33:57.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:57 smithi055 bash[9296]: cluster 2020-09-08T15:33:57.072033+0000 mon.a (mon.0) 740 : cluster [DBG] osdmap e75: 11 total, 10 up, 10 in 2020-09-08T15:33:57.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:57 smithi055 bash[9296]: audit 2020-09-08T15:33:57.072282+0000 mon.a (mon.0) 741 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:33:57.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:57 smithi104 bash[9203]: audit 2020-09-08T15:33:57.061931+0000 mon.c (mon.2) 24 : audit [INF] from='client.? v1:172.21.15.166:0/4044606008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c194210e-bd89-4a1f-98fb-6bf44f0b324c"}]: dispatch 2020-09-08T15:33:57.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:57 smithi104 bash[9203]: audit 2020-09-08T15:33:57.063610+0000 mon.a (mon.0) 738 : audit [INF] from='client.34406 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c194210e-bd89-4a1f-98fb-6bf44f0b324c"}]: dispatch 2020-09-08T15:33:57.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:57 smithi104 bash[9203]: audit 2020-09-08T15:33:57.071946+0000 mon.a (mon.0) 739 : audit [INF] from='client.34406 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c194210e-bd89-4a1f-98fb-6bf44f0b324c"}]': finished 2020-09-08T15:33:57.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:57 smithi104 bash[9203]: cluster 2020-09-08T15:33:57.072033+0000 mon.a (mon.0) 740 : cluster [DBG] osdmap e75: 11 total, 10 up, 10 in 2020-09-08T15:33:57.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:57 smithi104 bash[9203]: audit 2020-09-08T15:33:57.072282+0000 mon.a (mon.0) 741 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:33:58.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:58 smithi166 bash[9446]: cluster 2020-09-08T15:33:57.520020+0000 mgr.x (mgr.24118) 180 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:33:58.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:33:58 smithi166 bash[9446]: audit 2020-09-08T15:33:57.742076+0000 mon.c (mon.2) 25 : audit [DBG] from='client.? v1:172.21.15.166:0/1310938220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:33:58.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:58 smithi055 bash[9296]: cluster 2020-09-08T15:33:57.520020+0000 mgr.x (mgr.24118) 180 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:33:58.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:33:58 smithi055 bash[9296]: audit 2020-09-08T15:33:57.742076+0000 mon.c (mon.2) 25 : audit [DBG] from='client.? v1:172.21.15.166:0/1310938220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:33:58.934 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:58 smithi104 bash[9203]: cluster 2020-09-08T15:33:57.520020+0000 mgr.x (mgr.24118) 180 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:33:58.934 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:33:58 smithi104 bash[9203]: audit 2020-09-08T15:33:57.742076+0000 mon.c (mon.2) 25 : audit [DBG] from='client.? v1:172.21.15.166:0/1310938220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:34:00.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:00 smithi166 bash[9446]: cluster 2020-09-08T15:33:59.520643+0000 mgr.x (mgr.24118) 181 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:00.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:00 smithi055 bash[9296]: cluster 2020-09-08T15:33:59.520643+0000 mgr.x (mgr.24118) 181 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:00.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:00 smithi104 bash[9203]: cluster 2020-09-08T15:33:59.520643+0000 mgr.x (mgr.24118) 181 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:01.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:01 smithi166 bash[9446]: audit 2020-09-08T15:34:00.783678+0000 mon.a (mon.0) 742 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:34:01.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:01 smithi055 bash[9296]: audit 2020-09-08T15:34:00.783678+0000 mon.a (mon.0) 742 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:34:01.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:01 smithi104 bash[9203]: audit 2020-09-08T15:34:00.783678+0000 mon.a (mon.0) 742 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:34:02.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:02 smithi166 bash[9446]: cluster 2020-09-08T15:34:01.462733+0000 osd.7 (osd.7) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:02.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:02 smithi166 bash[9446]: cluster 2020-09-08T15:34:01.462910+0000 osd.7 (osd.7) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:02.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:02 smithi166 bash[9446]: cluster 2020-09-08T15:34:01.513392+0000 mon.a (mon.0) 743 : cluster [DBG] osdmap e76: 11 total, 10 up, 10 in 2020-09-08T15:34:02.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:02 smithi166 bash[9446]: audit 2020-09-08T15:34:01.514210+0000 mon.a (mon.0) 744 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:02.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:02 smithi166 bash[9446]: cluster 2020-09-08T15:34:01.521260+0000 mgr.x (mgr.24118) 182 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:02.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:02 smithi055 bash[9296]: cluster 2020-09-08T15:34:01.462733+0000 osd.7 (osd.7) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:02.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:02 smithi055 bash[9296]: cluster 2020-09-08T15:34:01.462910+0000 osd.7 (osd.7) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:02.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:02 smithi055 bash[9296]: cluster 2020-09-08T15:34:01.513392+0000 mon.a (mon.0) 743 : cluster [DBG] osdmap e76: 11 total, 10 up, 10 in 2020-09-08T15:34:02.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:02 smithi055 bash[9296]: audit 2020-09-08T15:34:01.514210+0000 mon.a (mon.0) 744 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:02.914 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:02 smithi055 bash[9296]: cluster 2020-09-08T15:34:01.521260+0000 mgr.x (mgr.24118) 182 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:02.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:02 smithi104 bash[9203]: cluster 2020-09-08T15:34:01.462733+0000 osd.7 (osd.7) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:02.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:02 smithi104 bash[9203]: cluster 2020-09-08T15:34:01.462910+0000 osd.7 (osd.7) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:02.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:02 smithi104 bash[9203]: cluster 2020-09-08T15:34:01.513392+0000 mon.a (mon.0) 743 : cluster [DBG] osdmap e76: 11 total, 10 up, 10 in 2020-09-08T15:34:02.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:02 smithi104 bash[9203]: audit 2020-09-08T15:34:01.514210+0000 mon.a (mon.0) 744 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:02.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:02 smithi104 bash[9203]: cluster 2020-09-08T15:34:01.521260+0000 mgr.x (mgr.24118) 182 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:04.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:04 smithi166 bash[9446]: cluster 2020-09-08T15:34:03.521883+0000 mgr.x (mgr.24118) 183 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:04.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:04 smithi055 bash[9296]: cluster 2020-09-08T15:34:03.521883+0000 mgr.x (mgr.24118) 183 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:04.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:04 smithi104 bash[9203]: cluster 2020-09-08T15:34:03.521883+0000 mgr.x (mgr.24118) 183 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:05.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:05 smithi166 bash[9446]: cluster 2020-09-08T15:34:03.959394+0000 osd.0 (osd.0) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:05.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:05 smithi166 bash[9446]: cluster 2020-09-08T15:34:03.959524+0000 osd.0 (osd.0) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:05.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:05 smithi166 bash[9446]: cluster 2020-09-08T15:34:04.539703+0000 mon.a (mon.0) 745 : cluster [DBG] osdmap e77: 11 total, 10 up, 10 in 2020-09-08T15:34:05.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:05 smithi166 bash[9446]: audit 2020-09-08T15:34:04.540647+0000 mon.a (mon.0) 746 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:05.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:05 smithi166 bash[9446]: audit 2020-09-08T15:34:04.901205+0000 mon.a (mon.0) 747 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2020-09-08T15:34:05.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:05 smithi166 bash[9446]: audit 2020-09-08T15:34:04.903415+0000 mon.a (mon.0) 748 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:34:05.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:05 smithi166 bash[9446]: audit 2020-09-08T15:34:04.905899+0000 mon.a (mon.0) 749 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.10", "key": "container_image"}]: dispatch 2020-09-08T15:34:05.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:05 smithi055 bash[9296]: cluster 2020-09-08T15:34:03.959394+0000 osd.0 (osd.0) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:05.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:05 smithi055 bash[9296]: cluster 2020-09-08T15:34:03.959524+0000 osd.0 (osd.0) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:05.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:05 smithi055 bash[9296]: cluster 2020-09-08T15:34:04.539703+0000 mon.a (mon.0) 745 : cluster [DBG] osdmap e77: 11 total, 10 up, 10 in 2020-09-08T15:34:05.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:05 smithi055 bash[9296]: audit 2020-09-08T15:34:04.540647+0000 mon.a (mon.0) 746 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:05.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:05 smithi055 bash[9296]: audit 2020-09-08T15:34:04.901205+0000 mon.a (mon.0) 747 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2020-09-08T15:34:05.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:05 smithi055 bash[9296]: audit 2020-09-08T15:34:04.903415+0000 mon.a (mon.0) 748 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:34:05.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:05 smithi055 bash[9296]: audit 2020-09-08T15:34:04.905899+0000 mon.a (mon.0) 749 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.10", "key": "container_image"}]: dispatch 2020-09-08T15:34:05.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:05 smithi104 bash[9203]: cluster 2020-09-08T15:34:03.959394+0000 osd.0 (osd.0) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:05.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:05 smithi104 bash[9203]: cluster 2020-09-08T15:34:03.959524+0000 osd.0 (osd.0) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:05.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:05 smithi104 bash[9203]: cluster 2020-09-08T15:34:04.539703+0000 mon.a (mon.0) 745 : cluster [DBG] osdmap e77: 11 total, 10 up, 10 in 2020-09-08T15:34:05.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:05 smithi104 bash[9203]: audit 2020-09-08T15:34:04.540647+0000 mon.a (mon.0) 746 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:05.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:05 smithi104 bash[9203]: audit 2020-09-08T15:34:04.901205+0000 mon.a (mon.0) 747 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2020-09-08T15:34:05.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:05 smithi104 bash[9203]: audit 2020-09-08T15:34:04.903415+0000 mon.a (mon.0) 748 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:34:05.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:05 smithi104 bash[9203]: audit 2020-09-08T15:34:04.905899+0000 mon.a (mon.0) 749 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.10", "key": "container_image"}]: dispatch 2020-09-08T15:34:06.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:06 smithi166 bash[9446]: cephadm 2020-09-08T15:34:04.905439+0000 mgr.x (mgr.24118) 184 : cephadm [INF] Deploying daemon osd.10 on smithi166 2020-09-08T15:34:06.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:06 smithi166 bash[9446]: cluster 2020-09-08T15:34:05.522539+0000 mgr.x (mgr.24118) 185 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:06.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:06 smithi055 bash[9296]: cephadm 2020-09-08T15:34:04.905439+0000 mgr.x (mgr.24118) 184 : cephadm [INF] Deploying daemon osd.10 on smithi166 2020-09-08T15:34:06.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:06 smithi055 bash[9296]: cluster 2020-09-08T15:34:05.522539+0000 mgr.x (mgr.24118) 185 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:06.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:06 smithi104 bash[9203]: cephadm 2020-09-08T15:34:04.905439+0000 mgr.x (mgr.24118) 184 : cephadm [INF] Deploying daemon osd.10 on smithi166 2020-09-08T15:34:06.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:06 smithi104 bash[9203]: cluster 2020-09-08T15:34:05.522539+0000 mgr.x (mgr.24118) 185 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:07.289 INFO:teuthology.orchestra.run.smithi166.stdout:Created osd(s) 10 on host 'smithi166' 2020-09-08T15:34:07.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:07 smithi166 bash[9446]: audit 2020-09-08T15:34:07.282176+0000 mon.a (mon.0) 750 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956337\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956546\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.957138\", \"created\": \"2020-09-08T15:33:41.114066\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}}, \"last_device_update\": \"2020-09-08T15:33:50.621089\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:34:07.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:07 smithi166 bash[9446]: audit 2020-09-08T15:34:07.283200+0000 mon.a (mon.0) 751 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:34:07.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:07 smithi166 bash[9446]: audit 2020-09-08T15:34:07.288088+0000 mon.a (mon.0) 752 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956337\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956546\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.957138\", \"created\": \"2020-09-08T15:33:41.114066\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}}, \"last_device_update\": \"2020-09-08T15:33:50.621089\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:34:07.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:07 smithi055 bash[9296]: audit 2020-09-08T15:34:07.282176+0000 mon.a (mon.0) 750 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956337\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956546\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.957138\", \"created\": \"2020-09-08T15:33:41.114066\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}}, \"last_device_update\": \"2020-09-08T15:33:50.621089\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:34:07.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:07 smithi055 bash[9296]: audit 2020-09-08T15:34:07.283200+0000 mon.a (mon.0) 751 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:34:07.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:07 smithi055 bash[9296]: audit 2020-09-08T15:34:07.288088+0000 mon.a (mon.0) 752 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956337\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956546\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.957138\", \"created\": \"2020-09-08T15:33:41.114066\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}}, \"last_device_update\": \"2020-09-08T15:33:50.621089\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:34:07.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:07 smithi104 bash[9203]: audit 2020-09-08T15:34:07.282176+0000 mon.a (mon.0) 750 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956337\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956546\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.957138\", \"created\": \"2020-09-08T15:33:41.114066\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}}, \"last_device_update\": \"2020-09-08T15:33:50.621089\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:34:07.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:07 smithi104 bash[9203]: audit 2020-09-08T15:34:07.283200+0000 mon.a (mon.0) 751 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:34:07.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:07 smithi104 bash[9203]: audit 2020-09-08T15:34:07.288088+0000 mon.a (mon.0) 752 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956337\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.956546\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:33:41.957138\", \"created\": \"2020-09-08T15:33:41.114066\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}}, \"last_device_update\": \"2020-09-08T15:33:50.621089\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:34:08.540 INFO:teuthology.orchestra.run.smithi166:osd.10> sudo journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.10.service 2020-09-08T15:34:08.544 INFO:tasks.cephadm:Deploying osd.11 on smithi166 with /dev/vg_nvme/lv_1... 2020-09-08T15:34:08.544 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2020-09-08T15:34:08.586 INFO:journalctl@ceph.osd.10.smithi166.stdout:-- Logs begin at Tue 2020-09-01 16:49:30 UTC. -- 2020-09-08T15:34:08.874 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:08 smithi166 bash[9446]: cluster 2020-09-08T15:34:07.523103+0000 mgr.x (mgr.24118) 186 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 55 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:08.875 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:08 smithi166 bash[9446]: audit 2020-09-08T15:34:08.084760+0000 mon.a (mon.0) 753 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081275\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081482\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082131\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082658\", \"created\": \"2020-09-08T15:34:07.251523\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}}, \"last_daemon_update\": \"2020-09-08T15:34:08.083227\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:34:08.876 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:08 smithi166 bash[9446]: audit 2020-09-08T15:34:08.090534+0000 mon.a (mon.0) 754 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081275\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081482\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082131\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082658\", \"created\": \"2020-09-08T15:34:07.251523\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}}, \"last_daemon_update\": \"2020-09-08T15:34:08.083227\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:34:08.876 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:08 smithi166 bash[9446]: audit 2020-09-08T15:34:08.092421+0000 mon.a (mon.0) 755 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:34:08.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:08 smithi055 bash[9296]: cluster 2020-09-08T15:34:07.523103+0000 mgr.x (mgr.24118) 186 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 55 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:08.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:08 smithi055 bash[9296]: audit 2020-09-08T15:34:08.084760+0000 mon.a (mon.0) 753 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081275\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081482\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082131\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082658\", \"created\": \"2020-09-08T15:34:07.251523\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}}, \"last_daemon_update\": \"2020-09-08T15:34:08.083227\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:34:08.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:08 smithi055 bash[9296]: audit 2020-09-08T15:34:08.090534+0000 mon.a (mon.0) 754 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081275\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081482\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082131\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082658\", \"created\": \"2020-09-08T15:34:07.251523\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}}, \"last_daemon_update\": \"2020-09-08T15:34:08.083227\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:34:08.912 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:08 smithi055 bash[9296]: audit 2020-09-08T15:34:08.092421+0000 mon.a (mon.0) 755 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:34:08.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:08 smithi104 bash[9203]: cluster 2020-09-08T15:34:07.523103+0000 mgr.x (mgr.24118) 186 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 55 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:08.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:08 smithi104 bash[9203]: audit 2020-09-08T15:34:08.084760+0000 mon.a (mon.0) 753 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081275\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081482\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082131\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082658\", \"created\": \"2020-09-08T15:34:07.251523\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}}, \"last_daemon_update\": \"2020-09-08T15:34:08.083227\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:34:08.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:08 smithi104 bash[9203]: audit 2020-09-08T15:34:08.090534+0000 mon.a (mon.0) 754 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081275\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081482\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082131\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082658\", \"created\": \"2020-09-08T15:34:07.251523\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}}, \"last_daemon_update\": \"2020-09-08T15:34:08.083227\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:34:08.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:08 smithi104 bash[9203]: audit 2020-09-08T15:34:08.092421+0000 mon.a (mon.0) 755 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:34:10.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:34:09 smithi166 bash[16593]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-10 2020-09-08T15:34:10.118 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:34:09 smithi166 bash[16593]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-10 --no-mon-config 2020-09-08T15:34:10.118 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:34:09 smithi166 bash[16593]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-10/block 2020-09-08T15:34:10.118 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:34:09 smithi166 bash[16593]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-10/block 2020-09-08T15:34:10.118 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:34:09 smithi166 bash[16593]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2020-09-08T15:34:10.118 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:34:09 smithi166 bash[16593]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-10 2020-09-08T15:34:10.119 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:34:09 smithi166 bash[16593]: --> ceph-volume lvm activate successful for osd ID: 10 2020-09-08T15:34:10.753 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:10 smithi166 bash[9446]: cluster 2020-09-08T15:34:09.523649+0000 mgr.x (mgr.24118) 187 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 55 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:10.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:10 smithi055 bash[9296]: cluster 2020-09-08T15:34:09.523649+0000 mgr.x (mgr.24118) 187 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 55 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:10.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:10 smithi104 bash[9203]: cluster 2020-09-08T15:34:09.523649+0000 mgr.x (mgr.24118) 187 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 55 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:11.225 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:34:10 smithi166 bash[16593]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:34:11.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:11 smithi166 bash[9446]: cluster 2020-09-08T15:34:10.301018+0000 osd.7 (osd.7) 7 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:34:11.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:11 smithi166 bash[9446]: cluster 2020-09-08T15:34:10.304073+0000 osd.7 (osd.7) 8 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:34:11.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:11 smithi166 bash[9446]: cluster 2020-09-08T15:34:10.954972+0000 osd.5 (osd.5) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:11.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:11 smithi166 bash[9446]: cluster 2020-09-08T15:34:10.955409+0000 osd.5 (osd.5) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:11.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:11 smithi055 bash[9296]: cluster 2020-09-08T15:34:10.301018+0000 osd.7 (osd.7) 7 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:34:11.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:11 smithi055 bash[9296]: cluster 2020-09-08T15:34:10.304073+0000 osd.7 (osd.7) 8 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:34:11.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:11 smithi055 bash[9296]: cluster 2020-09-08T15:34:10.954972+0000 osd.5 (osd.5) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:11.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:11 smithi055 bash[9296]: cluster 2020-09-08T15:34:10.955409+0000 osd.5 (osd.5) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:11.936 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:11 smithi104 bash[9203]: cluster 2020-09-08T15:34:10.301018+0000 osd.7 (osd.7) 7 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:34:11.936 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:11 smithi104 bash[9203]: cluster 2020-09-08T15:34:10.304073+0000 osd.7 (osd.7) 8 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:34:11.936 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:11 smithi104 bash[9203]: cluster 2020-09-08T15:34:10.954972+0000 osd.5 (osd.5) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:11.936 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:11 smithi104 bash[9203]: cluster 2020-09-08T15:34:10.955409+0000 osd.5 (osd.5) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:12.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:12 smithi166 bash[9446]: cluster 2020-09-08T15:34:11.524224+0000 mgr.x (mgr.24118) 188 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 0 B data, 55 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:12.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:12 smithi166 bash[9446]: cluster 2020-09-08T15:34:11.531773+0000 mon.a (mon.0) 756 : cluster [DBG] osdmap e78: 11 total, 10 up, 10 in 2020-09-08T15:34:12.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:12 smithi166 bash[9446]: audit 2020-09-08T15:34:11.531998+0000 mon.a (mon.0) 757 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:12.912 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:12 smithi055 bash[9296]: cluster 2020-09-08T15:34:11.524224+0000 mgr.x (mgr.24118) 188 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 0 B data, 55 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:12.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:12 smithi055 bash[9296]: cluster 2020-09-08T15:34:11.531773+0000 mon.a (mon.0) 756 : cluster [DBG] osdmap e78: 11 total, 10 up, 10 in 2020-09-08T15:34:12.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:12 smithi055 bash[9296]: audit 2020-09-08T15:34:11.531998+0000 mon.a (mon.0) 757 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:12.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:12 smithi104 bash[9203]: cluster 2020-09-08T15:34:11.524224+0000 mgr.x (mgr.24118) 188 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 0 B data, 55 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:12.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:12 smithi104 bash[9203]: cluster 2020-09-08T15:34:11.531773+0000 mon.a (mon.0) 756 : cluster [DBG] osdmap e78: 11 total, 10 up, 10 in 2020-09-08T15:34:12.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:12 smithi104 bash[9203]: audit 2020-09-08T15:34:11.531998+0000 mon.a (mon.0) 757 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:13.373 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:34:13 smithi166 bash[16593]: debug 2020-09-08T15:34:13.131+0000 7f897727b700 20 -- 172.21.15.166:0/3379680054 done calling dispatch on 0x55567bc4cc60 2020-09-08T15:34:13.374 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:34:13 smithi166 bash[16593]: debug 2020-09-08T15:34:13.131+0000 7f897727b700 1 -- 172.21.15.166:0/3379680054 <== mon.2 v1:172.21.15.166:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 2522065246 0 0) 0x55567bc564e0 con 0x55567bcccc00 2020-09-08T15:34:13.374 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:34:13 smithi166 bash[16593]: debug 2020-09-08T15:34:13.131+0000 7f897727b700 10 -- 172.21.15.166:0/3379680054 dispatch_throttle_release 194 to dispatch throttler 194/104857600 2020-09-08T15:34:13.374 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:34:13 smithi166 bash[16593]: debug 2020-09-08T15:34:13.131+0000 7f897727b700 20 -- 172.21.15.166:0/3379680054 done calling dispatch on 0x55567bc564e0 2020-09-08T15:34:13.996 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:34:13 smithi166 bash[16593]: debug 2020-09-08T15:34:13.675+0000 7f897ecfbf40 -1 Falling back to public interface 2020-09-08T15:34:14.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:14 smithi166 bash[9446]: cluster 2020-09-08T15:34:13.524847+0000 mgr.x (mgr.24118) 189 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:14.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:14 smithi166 bash[9446]: audit 2020-09-08T15:34:14.327037+0000 mon.c (mon.2) 26 : audit [INF] from='osd.10 v1:172.21.15.166:6817/62517848' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2020-09-08T15:34:14.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:14 smithi166 bash[9446]: audit 2020-09-08T15:34:14.329157+0000 mon.a (mon.0) 758 : audit [INF] from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2020-09-08T15:34:14.618 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:34:14 smithi166 bash[16593]: debug 2020-09-08T15:34:14.319+0000 7f897ecfbf40 -1 osd.10 0 log_to_monitors {default=true} 2020-09-08T15:34:14.639 INFO:teuthology.orchestra.run.smithi166.stderr:--> Zapping: /dev/vg_nvme/lv_1 2020-09-08T15:34:14.639 INFO:teuthology.orchestra.run.smithi166.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2020-09-08T15:34:14.639 INFO:teuthology.orchestra.run.smithi166.stderr: stderr: 10+0 records in 2020-09-08T15:34:14.640 INFO:teuthology.orchestra.run.smithi166.stderr:10+0 records out 2020-09-08T15:34:14.640 INFO:teuthology.orchestra.run.smithi166.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0158816 s, 660 MB/s 2020-09-08T15:34:14.640 INFO:teuthology.orchestra.run.smithi166.stderr:--> Zapping successful for: 2020-09-08T15:34:14.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:14 smithi055 bash[9296]: cluster 2020-09-08T15:34:13.524847+0000 mgr.x (mgr.24118) 189 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:14.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:14 smithi055 bash[9296]: audit 2020-09-08T15:34:14.327037+0000 mon.c (mon.2) 26 : audit [INF] from='osd.10 v1:172.21.15.166:6817/62517848' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2020-09-08T15:34:14.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:14 smithi055 bash[9296]: audit 2020-09-08T15:34:14.329157+0000 mon.a (mon.0) 758 : audit [INF] from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2020-09-08T15:34:14.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:14 smithi104 bash[9203]: cluster 2020-09-08T15:34:13.524847+0000 mgr.x (mgr.24118) 189 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:14.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:14 smithi104 bash[9203]: audit 2020-09-08T15:34:14.327037+0000 mon.c (mon.2) 26 : audit [INF] from='osd.10 v1:172.21.15.166:6817/62517848' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2020-09-08T15:34:14.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:14 smithi104 bash[9203]: audit 2020-09-08T15:34:14.329157+0000 mon.a (mon.0) 758 : audit [INF] from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2020-09-08T15:34:15.507 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph orch daemon add osd smithi166:vg_nvme/lv_1 2020-09-08T15:34:15.867 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:34:15 smithi166 bash[16593]: debug 2020-09-08T15:34:15.554+0000 7f8975277700 -1 osd.10 0 waiting for initial osdmap 2020-09-08T15:34:15.868 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:34:15 smithi166 bash[16593]: debug 2020-09-08T15:34:15.554+0000 7f8967386700 -1 osd.10 0 failed to load OSD map for epoch 22, got 0 bytes 2020-09-08T15:34:15.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:15 smithi166 bash[9446]: audit 2020-09-08T15:34:14.544472+0000 mon.c (mon.2) 27 : audit [INF] from='osd.10 v1:172.21.15.166:6817/62517848' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:34:15.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:15 smithi166 bash[9446]: audit 2020-09-08T15:34:14.544887+0000 mon.a (mon.0) 759 : audit [INF] from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2020-09-08T15:34:15.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:15 smithi166 bash[9446]: cluster 2020-09-08T15:34:14.544993+0000 mon.a (mon.0) 760 : cluster [DBG] osdmap e79: 11 total, 10 up, 10 in 2020-09-08T15:34:15.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:15 smithi166 bash[9446]: audit 2020-09-08T15:34:14.545296+0000 mon.a (mon.0) 761 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:15.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:15 smithi166 bash[9446]: audit 2020-09-08T15:34:14.546547+0000 mon.a (mon.0) 762 : audit [INF] from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:34:15.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:15 smithi055 bash[9296]: audit 2020-09-08T15:34:14.544472+0000 mon.c (mon.2) 27 : audit [INF] from='osd.10 v1:172.21.15.166:6817/62517848' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:34:15.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:15 smithi055 bash[9296]: audit 2020-09-08T15:34:14.544887+0000 mon.a (mon.0) 759 : audit [INF] from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2020-09-08T15:34:15.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:15 smithi055 bash[9296]: cluster 2020-09-08T15:34:14.544993+0000 mon.a (mon.0) 760 : cluster [DBG] osdmap e79: 11 total, 10 up, 10 in 2020-09-08T15:34:15.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:15 smithi055 bash[9296]: audit 2020-09-08T15:34:14.545296+0000 mon.a (mon.0) 761 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:15.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:15 smithi055 bash[9296]: audit 2020-09-08T15:34:14.546547+0000 mon.a (mon.0) 762 : audit [INF] from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:34:15.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:15 smithi104 bash[9203]: audit 2020-09-08T15:34:14.544472+0000 mon.c (mon.2) 27 : audit [INF] from='osd.10 v1:172.21.15.166:6817/62517848' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:34:15.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:15 smithi104 bash[9203]: audit 2020-09-08T15:34:14.544887+0000 mon.a (mon.0) 759 : audit [INF] from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2020-09-08T15:34:15.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:15 smithi104 bash[9203]: cluster 2020-09-08T15:34:14.544993+0000 mon.a (mon.0) 760 : cluster [DBG] osdmap e79: 11 total, 10 up, 10 in 2020-09-08T15:34:15.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:15 smithi104 bash[9203]: audit 2020-09-08T15:34:14.545296+0000 mon.a (mon.0) 761 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:15.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:15 smithi104 bash[9203]: audit 2020-09-08T15:34:14.546547+0000 mon.a (mon.0) 762 : audit [INF] from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:34:16.825 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:16 smithi166 bash[9446]: cluster 2020-09-08T15:34:15.525463+0000 mgr.x (mgr.24118) 190 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:16.826 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:16 smithi166 bash[9446]: audit 2020-09-08T15:34:15.547981+0000 mon.a (mon.0) 763 : audit [INF] from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]': finished 2020-09-08T15:34:16.826 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:16 smithi166 bash[9446]: cluster 2020-09-08T15:34:15.548070+0000 mon.a (mon.0) 764 : cluster [DBG] osdmap e80: 11 total, 10 up, 10 in 2020-09-08T15:34:16.826 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:16 smithi166 bash[9446]: audit 2020-09-08T15:34:15.549082+0000 mon.a (mon.0) 765 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:16.826 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:16 smithi166 bash[9446]: audit 2020-09-08T15:34:15.554051+0000 mon.a (mon.0) 766 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:16.827 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:16 smithi166 bash[9446]: audit 2020-09-08T15:34:16.261170+0000 mon.a (mon.0) 767 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:34:16.827 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:16 smithi166 bash[9446]: audit 2020-09-08T15:34:16.553133+0000 mon.a (mon.0) 768 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081275\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081482\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082131\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082658\", \"created\": \"2020-09-08T15:34:07.251523\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}}, \"last_daemon_update\": \"2020-09-08T15:34:08.083227\", \"last_device_update\": \"2020-09-08T15:34:16.551291\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:34:16.827 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:16 smithi166 bash[9446]: audit 2020-09-08T15:34:16.554178+0000 mon.a (mon.0) 769 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:16.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:16 smithi055 bash[9296]: cluster 2020-09-08T15:34:15.525463+0000 mgr.x (mgr.24118) 190 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:16.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:16 smithi055 bash[9296]: audit 2020-09-08T15:34:15.547981+0000 mon.a (mon.0) 763 : audit [INF] from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]': finished 2020-09-08T15:34:16.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:16 smithi055 bash[9296]: cluster 2020-09-08T15:34:15.548070+0000 mon.a (mon.0) 764 : cluster [DBG] osdmap e80: 11 total, 10 up, 10 in 2020-09-08T15:34:16.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:16 smithi055 bash[9296]: audit 2020-09-08T15:34:15.549082+0000 mon.a (mon.0) 765 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:16.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:16 smithi055 bash[9296]: audit 2020-09-08T15:34:15.554051+0000 mon.a (mon.0) 766 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:16.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:16 smithi055 bash[9296]: audit 2020-09-08T15:34:16.261170+0000 mon.a (mon.0) 767 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:34:16.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:16 smithi055 bash[9296]: audit 2020-09-08T15:34:16.553133+0000 mon.a (mon.0) 768 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081275\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081482\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082131\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082658\", \"created\": \"2020-09-08T15:34:07.251523\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}}, \"last_daemon_update\": \"2020-09-08T15:34:08.083227\", \"last_device_update\": \"2020-09-08T15:34:16.551291\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:34:16.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:16 smithi055 bash[9296]: audit 2020-09-08T15:34:16.554178+0000 mon.a (mon.0) 769 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:16.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:16 smithi104 bash[9203]: cluster 2020-09-08T15:34:15.525463+0000 mgr.x (mgr.24118) 190 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:16.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:16 smithi104 bash[9203]: audit 2020-09-08T15:34:15.547981+0000 mon.a (mon.0) 763 : audit [INF] from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]': finished 2020-09-08T15:34:16.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:16 smithi104 bash[9203]: cluster 2020-09-08T15:34:15.548070+0000 mon.a (mon.0) 764 : cluster [DBG] osdmap e80: 11 total, 10 up, 10 in 2020-09-08T15:34:16.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:16 smithi104 bash[9203]: audit 2020-09-08T15:34:15.549082+0000 mon.a (mon.0) 765 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:16.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:16 smithi104 bash[9203]: audit 2020-09-08T15:34:15.554051+0000 mon.a (mon.0) 766 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:16.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:16 smithi104 bash[9203]: audit 2020-09-08T15:34:16.261170+0000 mon.a (mon.0) 767 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:34:16.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:16 smithi104 bash[9203]: audit 2020-09-08T15:34:16.553133+0000 mon.a (mon.0) 768 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081275\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081482\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082131\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082658\", \"created\": \"2020-09-08T15:34:07.251523\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}}, \"last_daemon_update\": \"2020-09-08T15:34:08.083227\", \"last_device_update\": \"2020-09-08T15:34:16.551291\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:34:16.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:16 smithi104 bash[9203]: audit 2020-09-08T15:34:16.554178+0000 mon.a (mon.0) 769 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:17.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:17 smithi166 bash[9446]: cluster 2020-09-08T15:34:15.373077+0000 osd.10 (osd.10) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:17.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:17 smithi166 bash[9446]: cluster 2020-09-08T15:34:15.373254+0000 osd.10 (osd.10) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:17.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:17 smithi166 bash[9446]: audit 2020-09-08T15:34:16.559673+0000 mon.a (mon.0) 770 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081275\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081482\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082131\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082658\", \"created\": \"2020-09-08T15:34:07.251523\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}}, \"last_daemon_update\": \"2020-09-08T15:34:08.083227\", \"last_device_update\": \"2020-09-08T15:34:16.551291\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:34:17.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:17 smithi166 bash[9446]: audit 2020-09-08T15:34:16.561970+0000 mon.a (mon.0) 771 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:34:17.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:17 smithi166 bash[9446]: cluster 2020-09-08T15:34:16.571550+0000 mon.a (mon.0) 772 : cluster [INF] osd.10 v1:172.21.15.166:6817/62517848 boot 2020-09-08T15:34:17.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:17 smithi166 bash[9446]: cluster 2020-09-08T15:34:16.571684+0000 mon.a (mon.0) 773 : cluster [DBG] osdmap e81: 11 total, 11 up, 11 in 2020-09-08T15:34:17.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:17 smithi166 bash[9446]: audit 2020-09-08T15:34:16.572736+0000 mon.a (mon.0) 774 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:17.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:17 smithi166 bash[9446]: audit 2020-09-08T15:34:16.577793+0000 mon.a (mon.0) 775 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:34:17.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:17 smithi166 bash[9446]: audit 2020-09-08T15:34:16.583737+0000 mon.a (mon.0) 776 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:34:17.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:17 smithi166 bash[9446]: audit 2020-09-08T15:34:16.588234+0000 mon.a (mon.0) 777 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:34:17.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:17 smithi055 bash[9296]: cluster 2020-09-08T15:34:15.373077+0000 osd.10 (osd.10) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:17.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:17 smithi055 bash[9296]: cluster 2020-09-08T15:34:15.373254+0000 osd.10 (osd.10) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:17.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:17 smithi055 bash[9296]: audit 2020-09-08T15:34:16.559673+0000 mon.a (mon.0) 770 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081275\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081482\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082131\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082658\", \"created\": \"2020-09-08T15:34:07.251523\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}}, \"last_daemon_update\": \"2020-09-08T15:34:08.083227\", \"last_device_update\": \"2020-09-08T15:34:16.551291\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:34:17.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:17 smithi055 bash[9296]: audit 2020-09-08T15:34:16.561970+0000 mon.a (mon.0) 771 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:34:17.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:17 smithi055 bash[9296]: cluster 2020-09-08T15:34:16.571550+0000 mon.a (mon.0) 772 : cluster [INF] osd.10 v1:172.21.15.166:6817/62517848 boot 2020-09-08T15:34:17.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:17 smithi055 bash[9296]: cluster 2020-09-08T15:34:16.571684+0000 mon.a (mon.0) 773 : cluster [DBG] osdmap e81: 11 total, 11 up, 11 in 2020-09-08T15:34:17.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:17 smithi055 bash[9296]: audit 2020-09-08T15:34:16.572736+0000 mon.a (mon.0) 774 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:17.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:17 smithi055 bash[9296]: audit 2020-09-08T15:34:16.577793+0000 mon.a (mon.0) 775 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:34:17.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:17 smithi055 bash[9296]: audit 2020-09-08T15:34:16.583737+0000 mon.a (mon.0) 776 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:34:17.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:17 smithi055 bash[9296]: audit 2020-09-08T15:34:16.588234+0000 mon.a (mon.0) 777 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:34:17.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:17 smithi104 bash[9203]: cluster 2020-09-08T15:34:15.373077+0000 osd.10 (osd.10) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:17.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:17 smithi104 bash[9203]: cluster 2020-09-08T15:34:15.373254+0000 osd.10 (osd.10) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:17.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:17 smithi104 bash[9203]: audit 2020-09-08T15:34:16.559673+0000 mon.a (mon.0) 770 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081275\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081482\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082131\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082658\", \"created\": \"2020-09-08T15:34:07.251523\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}}, \"last_daemon_update\": \"2020-09-08T15:34:08.083227\", \"last_device_update\": \"2020-09-08T15:34:16.551291\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:34:17.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:17 smithi104 bash[9203]: audit 2020-09-08T15:34:16.561970+0000 mon.a (mon.0) 771 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:34:17.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:17 smithi104 bash[9203]: cluster 2020-09-08T15:34:16.571550+0000 mon.a (mon.0) 772 : cluster [INF] osd.10 v1:172.21.15.166:6817/62517848 boot 2020-09-08T15:34:17.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:17 smithi104 bash[9203]: cluster 2020-09-08T15:34:16.571684+0000 mon.a (mon.0) 773 : cluster [DBG] osdmap e81: 11 total, 11 up, 11 in 2020-09-08T15:34:17.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:17 smithi104 bash[9203]: audit 2020-09-08T15:34:16.572736+0000 mon.a (mon.0) 774 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2020-09-08T15:34:17.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:17 smithi104 bash[9203]: audit 2020-09-08T15:34:16.577793+0000 mon.a (mon.0) 775 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:34:17.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:17 smithi104 bash[9203]: audit 2020-09-08T15:34:16.583737+0000 mon.a (mon.0) 776 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:34:17.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:17 smithi104 bash[9203]: audit 2020-09-08T15:34:16.588234+0000 mon.a (mon.0) 777 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:34:18.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:18 smithi166 bash[9446]: cluster 2020-09-08T15:34:17.526097+0000 mgr.x (mgr.24118) 191 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:18.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:18 smithi166 bash[9446]: cluster 2020-09-08T15:34:17.596378+0000 mon.a (mon.0) 778 : cluster [DBG] osdmap e82: 11 total, 11 up, 11 in 2020-09-08T15:34:18.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:18 smithi055 bash[9296]: cluster 2020-09-08T15:34:17.526097+0000 mgr.x (mgr.24118) 191 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:18.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:18 smithi055 bash[9296]: cluster 2020-09-08T15:34:17.596378+0000 mon.a (mon.0) 778 : cluster [DBG] osdmap e82: 11 total, 11 up, 11 in 2020-09-08T15:34:18.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:18 smithi104 bash[9203]: cluster 2020-09-08T15:34:17.526097+0000 mgr.x (mgr.24118) 191 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 0 B data, 56 MiB used, 894 GiB / 894 GiB avail 2020-09-08T15:34:18.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:18 smithi104 bash[9203]: cluster 2020-09-08T15:34:17.596378+0000 mon.a (mon.0) 778 : cluster [DBG] osdmap e82: 11 total, 11 up, 11 in 2020-09-08T15:34:19.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:19 smithi166 bash[9446]: audit 2020-09-08T15:34:18.979361+0000 mon.a (mon.0) 779 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:34:19.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:19 smithi166 bash[9446]: audit 2020-09-08T15:34:18.983258+0000 mon.a (mon.0) 780 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:34:19.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:19 smithi166 bash[9446]: audit 2020-09-08T15:34:18.985209+0000 mon.a (mon.0) 781 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:34:19.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:19 smithi166 bash[9446]: audit 2020-09-08T15:34:18.987254+0000 mon.a (mon.0) 782 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:34:19.916 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:19 smithi055 bash[9296]: audit 2020-09-08T15:34:18.979361+0000 mon.a (mon.0) 779 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:34:19.916 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:19 smithi055 bash[9296]: audit 2020-09-08T15:34:18.983258+0000 mon.a (mon.0) 780 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:34:19.916 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:19 smithi055 bash[9296]: audit 2020-09-08T15:34:18.985209+0000 mon.a (mon.0) 781 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:34:19.917 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:19 smithi055 bash[9296]: audit 2020-09-08T15:34:18.987254+0000 mon.a (mon.0) 782 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:34:19.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:19 smithi104 bash[9203]: audit 2020-09-08T15:34:18.979361+0000 mon.a (mon.0) 779 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2020-09-08T15:34:19.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:19 smithi104 bash[9203]: audit 2020-09-08T15:34:18.983258+0000 mon.a (mon.0) 780 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2020-09-08T15:34:19.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:19 smithi104 bash[9203]: audit 2020-09-08T15:34:18.985209+0000 mon.a (mon.0) 781 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:34:19.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:19 smithi104 bash[9203]: audit 2020-09-08T15:34:18.987254+0000 mon.a (mon.0) 782 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:34:20.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:20 smithi166 bash[9446]: audit 2020-09-08T15:34:18.977478+0000 mgr.x (mgr.24118) 192 : audit [DBG] from='client.34424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi166:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:34:20.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:20 smithi166 bash[9446]: cephadm 2020-09-08T15:34:18.981160+0000 mgr.x (mgr.24118) 193 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:34:20.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:20 smithi166 bash[9446]: cephadm 2020-09-08T15:34:18.981344+0000 mgr.x (mgr.24118) 194 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:34:20.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:20 smithi166 bash[9446]: cephadm 2020-09-08T15:34:18.982432+0000 mgr.x (mgr.24118) 195 : cephadm [INF] Applying None on host smithi166... 2020-09-08T15:34:20.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:20 smithi166 bash[9446]: cluster 2020-09-08T15:34:19.062692+0000 osd.3 (osd.3) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:20.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:20 smithi166 bash[9446]: cluster 2020-09-08T15:34:19.062837+0000 osd.3 (osd.3) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:20.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:20 smithi166 bash[9446]: cluster 2020-09-08T15:34:19.526732+0000 mgr.x (mgr.24118) 196 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:20.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:20 smithi166 bash[9446]: cluster 2020-09-08T15:34:19.577066+0000 mon.a (mon.0) 783 : cluster [DBG] osdmap e83: 11 total, 11 up, 11 in 2020-09-08T15:34:20.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:20 smithi055 bash[9296]: audit 2020-09-08T15:34:18.977478+0000 mgr.x (mgr.24118) 192 : audit [DBG] from='client.34424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi166:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:34:20.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:20 smithi055 bash[9296]: cephadm 2020-09-08T15:34:18.981160+0000 mgr.x (mgr.24118) 193 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:34:20.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:20 smithi055 bash[9296]: cephadm 2020-09-08T15:34:18.981344+0000 mgr.x (mgr.24118) 194 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:34:20.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:20 smithi055 bash[9296]: cephadm 2020-09-08T15:34:18.982432+0000 mgr.x (mgr.24118) 195 : cephadm [INF] Applying None on host smithi166... 2020-09-08T15:34:20.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:20 smithi055 bash[9296]: cluster 2020-09-08T15:34:19.062692+0000 osd.3 (osd.3) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:20.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:20 smithi055 bash[9296]: cluster 2020-09-08T15:34:19.062837+0000 osd.3 (osd.3) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:20.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:20 smithi055 bash[9296]: cluster 2020-09-08T15:34:19.526732+0000 mgr.x (mgr.24118) 196 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:20.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:20 smithi055 bash[9296]: cluster 2020-09-08T15:34:19.577066+0000 mon.a (mon.0) 783 : cluster [DBG] osdmap e83: 11 total, 11 up, 11 in 2020-09-08T15:34:20.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:20 smithi104 bash[9203]: audit 2020-09-08T15:34:18.977478+0000 mgr.x (mgr.24118) 192 : audit [DBG] from='client.34424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi166:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2020-09-08T15:34:20.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:20 smithi104 bash[9203]: cephadm 2020-09-08T15:34:18.981160+0000 mgr.x (mgr.24118) 193 : cephadm [INF] Found osd claims -> {} 2020-09-08T15:34:20.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:20 smithi104 bash[9203]: cephadm 2020-09-08T15:34:18.981344+0000 mgr.x (mgr.24118) 194 : cephadm [INF] Found osd claims for drivegroup None -> {} 2020-09-08T15:34:20.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:20 smithi104 bash[9203]: cephadm 2020-09-08T15:34:18.982432+0000 mgr.x (mgr.24118) 195 : cephadm [INF] Applying None on host smithi166... 2020-09-08T15:34:20.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:20 smithi104 bash[9203]: cluster 2020-09-08T15:34:19.062692+0000 osd.3 (osd.3) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:20.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:20 smithi104 bash[9203]: cluster 2020-09-08T15:34:19.062837+0000 osd.3 (osd.3) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:20.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:20 smithi104 bash[9203]: cluster 2020-09-08T15:34:19.526732+0000 mgr.x (mgr.24118) 196 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:20.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:20 smithi104 bash[9203]: cluster 2020-09-08T15:34:19.577066+0000 mon.a (mon.0) 783 : cluster [DBG] osdmap e83: 11 total, 11 up, 11 in 2020-09-08T15:34:21.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:21 smithi166 bash[9446]: cluster 2020-09-08T15:34:21.527453+0000 mgr.x (mgr.24118) 197 : cluster [DBG] pgmap v210: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:21.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:21 smithi055 bash[9296]: cluster 2020-09-08T15:34:21.527453+0000 mgr.x (mgr.24118) 197 : cluster [DBG] pgmap v210: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:21.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:21 smithi104 bash[9203]: cluster 2020-09-08T15:34:21.527453+0000 mgr.x (mgr.24118) 197 : cluster [DBG] pgmap v210: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:23.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:22 smithi104 bash[9203]: audit 2020-09-08T15:34:22.857644+0000 mon.c (mon.2) 28 : audit [INF] from='client.? v1:172.21.15.166:0/4237982898' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "121301fe-a41c-40c0-ad09-b00de37a55e1"}]: dispatch 2020-09-08T15:34:23.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:22 smithi104 bash[9203]: audit 2020-09-08T15:34:22.860123+0000 mon.a (mon.0) 784 : audit [INF] from='client.34430 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "121301fe-a41c-40c0-ad09-b00de37a55e1"}]: dispatch 2020-09-08T15:34:23.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:22 smithi104 bash[9203]: audit 2020-09-08T15:34:22.869713+0000 mon.a (mon.0) 785 : audit [INF] from='client.34430 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "121301fe-a41c-40c0-ad09-b00de37a55e1"}]': finished 2020-09-08T15:34:23.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:22 smithi104 bash[9203]: cluster 2020-09-08T15:34:22.869822+0000 mon.a (mon.0) 786 : cluster [DBG] osdmap e84: 12 total, 11 up, 11 in 2020-09-08T15:34:23.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:22 smithi104 bash[9203]: audit 2020-09-08T15:34:22.870127+0000 mon.a (mon.0) 787 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:23.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:22 smithi166 bash[9446]: audit 2020-09-08T15:34:22.857644+0000 mon.c (mon.2) 28 : audit [INF] from='client.? v1:172.21.15.166:0/4237982898' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "121301fe-a41c-40c0-ad09-b00de37a55e1"}]: dispatch 2020-09-08T15:34:23.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:22 smithi166 bash[9446]: audit 2020-09-08T15:34:22.860123+0000 mon.a (mon.0) 784 : audit [INF] from='client.34430 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "121301fe-a41c-40c0-ad09-b00de37a55e1"}]: dispatch 2020-09-08T15:34:23.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:22 smithi166 bash[9446]: audit 2020-09-08T15:34:22.869713+0000 mon.a (mon.0) 785 : audit [INF] from='client.34430 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "121301fe-a41c-40c0-ad09-b00de37a55e1"}]': finished 2020-09-08T15:34:23.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:22 smithi166 bash[9446]: cluster 2020-09-08T15:34:22.869822+0000 mon.a (mon.0) 786 : cluster [DBG] osdmap e84: 12 total, 11 up, 11 in 2020-09-08T15:34:23.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:22 smithi166 bash[9446]: audit 2020-09-08T15:34:22.870127+0000 mon.a (mon.0) 787 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:23.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:22 smithi055 bash[9296]: audit 2020-09-08T15:34:22.857644+0000 mon.c (mon.2) 28 : audit [INF] from='client.? v1:172.21.15.166:0/4237982898' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "121301fe-a41c-40c0-ad09-b00de37a55e1"}]: dispatch 2020-09-08T15:34:23.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:22 smithi055 bash[9296]: audit 2020-09-08T15:34:22.860123+0000 mon.a (mon.0) 784 : audit [INF] from='client.34430 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "121301fe-a41c-40c0-ad09-b00de37a55e1"}]: dispatch 2020-09-08T15:34:23.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:22 smithi055 bash[9296]: audit 2020-09-08T15:34:22.869713+0000 mon.a (mon.0) 785 : audit [INF] from='client.34430 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "121301fe-a41c-40c0-ad09-b00de37a55e1"}]': finished 2020-09-08T15:34:23.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:22 smithi055 bash[9296]: cluster 2020-09-08T15:34:22.869822+0000 mon.a (mon.0) 786 : cluster [DBG] osdmap e84: 12 total, 11 up, 11 in 2020-09-08T15:34:23.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:22 smithi055 bash[9296]: audit 2020-09-08T15:34:22.870127+0000 mon.a (mon.0) 787 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:24.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:23 smithi104 bash[9203]: audit 2020-09-08T15:34:23.490511+0000 mon.c (mon.2) 29 : audit [DBG] from='client.? v1:172.21.15.166:0/67384485' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:34:24.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:23 smithi104 bash[9203]: cluster 2020-09-08T15:34:23.528061+0000 mgr.x (mgr.24118) 198 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:24.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:23 smithi166 bash[9446]: audit 2020-09-08T15:34:23.490511+0000 mon.c (mon.2) 29 : audit [DBG] from='client.? v1:172.21.15.166:0/67384485' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:34:24.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:23 smithi166 bash[9446]: cluster 2020-09-08T15:34:23.528061+0000 mgr.x (mgr.24118) 198 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:24.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:23 smithi055 bash[9296]: audit 2020-09-08T15:34:23.490511+0000 mon.c (mon.2) 29 : audit [DBG] from='client.? v1:172.21.15.166:0/67384485' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2020-09-08T15:34:24.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:23 smithi055 bash[9296]: cluster 2020-09-08T15:34:23.528061+0000 mgr.x (mgr.24118) 198 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:26.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:25 smithi166 bash[9446]: cluster 2020-09-08T15:34:24.293021+0000 osd.8 (osd.8) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:26.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:25 smithi166 bash[9446]: cluster 2020-09-08T15:34:24.293186+0000 osd.8 (osd.8) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:26.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:25 smithi166 bash[9446]: cluster 2020-09-08T15:34:24.933213+0000 mon.a (mon.0) 788 : cluster [DBG] osdmap e85: 12 total, 11 up, 11 in 2020-09-08T15:34:26.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:25 smithi166 bash[9446]: audit 2020-09-08T15:34:24.933466+0000 mon.a (mon.0) 789 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:26.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:25 smithi166 bash[9446]: cluster 2020-09-08T15:34:25.528657+0000 mgr.x (mgr.24118) 199 : cluster [DBG] pgmap v214: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:26.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:25 smithi055 bash[9296]: cluster 2020-09-08T15:34:24.293021+0000 osd.8 (osd.8) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:26.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:25 smithi055 bash[9296]: cluster 2020-09-08T15:34:24.293186+0000 osd.8 (osd.8) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:26.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:25 smithi055 bash[9296]: cluster 2020-09-08T15:34:24.933213+0000 mon.a (mon.0) 788 : cluster [DBG] osdmap e85: 12 total, 11 up, 11 in 2020-09-08T15:34:26.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:25 smithi055 bash[9296]: audit 2020-09-08T15:34:24.933466+0000 mon.a (mon.0) 789 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:26.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:25 smithi055 bash[9296]: cluster 2020-09-08T15:34:25.528657+0000 mgr.x (mgr.24118) 199 : cluster [DBG] pgmap v214: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:26.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:25 smithi104 bash[9203]: cluster 2020-09-08T15:34:24.293021+0000 osd.8 (osd.8) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:26.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:25 smithi104 bash[9203]: cluster 2020-09-08T15:34:24.293186+0000 osd.8 (osd.8) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:26.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:25 smithi104 bash[9203]: cluster 2020-09-08T15:34:24.933213+0000 mon.a (mon.0) 788 : cluster [DBG] osdmap e85: 12 total, 11 up, 11 in 2020-09-08T15:34:26.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:25 smithi104 bash[9203]: audit 2020-09-08T15:34:24.933466+0000 mon.a (mon.0) 789 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:26.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:25 smithi104 bash[9203]: cluster 2020-09-08T15:34:25.528657+0000 mgr.x (mgr.24118) 199 : cluster [DBG] pgmap v214: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:27.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:26 smithi166 bash[9446]: audit 2020-09-08T15:34:26.507823+0000 mon.a (mon.0) 790 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:34:27.414 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:26 smithi055 bash[9296]: audit 2020-09-08T15:34:26.507823+0000 mon.a (mon.0) 790 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:34:27.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:26 smithi104 bash[9203]: audit 2020-09-08T15:34:26.507823+0000 mon.a (mon.0) 790 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:34:28.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:27 smithi166 bash[9446]: cluster 2020-09-08T15:34:27.529329+0000 mgr.x (mgr.24118) 200 : cluster [DBG] pgmap v215: 1 pgs: 1 active+clean; 0 B data, 63 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:28.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:27 smithi055 bash[9296]: cluster 2020-09-08T15:34:27.529329+0000 mgr.x (mgr.24118) 200 : cluster [DBG] pgmap v215: 1 pgs: 1 active+clean; 0 B data, 63 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:28.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:27 smithi104 bash[9203]: cluster 2020-09-08T15:34:27.529329+0000 mgr.x (mgr.24118) 200 : cluster [DBG] pgmap v215: 1 pgs: 1 active+clean; 0 B data, 63 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:30.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:30 smithi166 bash[9446]: cluster 2020-09-08T15:34:29.529954+0000 mgr.x (mgr.24118) 201 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 0 B data, 63 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:30.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:30 smithi055 bash[9296]: cluster 2020-09-08T15:34:29.529954+0000 mgr.x (mgr.24118) 201 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 0 B data, 63 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:30.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:30 smithi104 bash[9203]: cluster 2020-09-08T15:34:29.529954+0000 mgr.x (mgr.24118) 201 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 0 B data, 63 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:31.782 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:31 smithi166 bash[9446]: audit 2020-09-08T15:34:30.550247+0000 mon.a (mon.0) 791 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2020-09-08T15:34:31.782 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:31 smithi166 bash[9446]: audit 2020-09-08T15:34:30.552219+0000 mon.a (mon.0) 792 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:34:31.783 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:31 smithi166 bash[9446]: cephadm 2020-09-08T15:34:30.553848+0000 mgr.x (mgr.24118) 202 : cephadm [INF] Deploying daemon osd.11 on smithi166 2020-09-08T15:34:31.783 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:31 smithi166 bash[9446]: audit 2020-09-08T15:34:30.554590+0000 mon.a (mon.0) 793 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.11", "key": "container_image"}]: dispatch 2020-09-08T15:34:31.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:31 smithi055 bash[9296]: audit 2020-09-08T15:34:30.550247+0000 mon.a (mon.0) 791 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2020-09-08T15:34:31.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:31 smithi055 bash[9296]: audit 2020-09-08T15:34:30.552219+0000 mon.a (mon.0) 792 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:34:31.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:31 smithi055 bash[9296]: cephadm 2020-09-08T15:34:30.553848+0000 mgr.x (mgr.24118) 202 : cephadm [INF] Deploying daemon osd.11 on smithi166 2020-09-08T15:34:31.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:31 smithi055 bash[9296]: audit 2020-09-08T15:34:30.554590+0000 mon.a (mon.0) 793 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.11", "key": "container_image"}]: dispatch 2020-09-08T15:34:31.934 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:31 smithi104 bash[9203]: audit 2020-09-08T15:34:30.550247+0000 mon.a (mon.0) 791 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2020-09-08T15:34:31.935 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:31 smithi104 bash[9203]: audit 2020-09-08T15:34:30.552219+0000 mon.a (mon.0) 792 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2020-09-08T15:34:31.935 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:31 smithi104 bash[9203]: cephadm 2020-09-08T15:34:30.553848+0000 mgr.x (mgr.24118) 202 : cephadm [INF] Deploying daemon osd.11 on smithi166 2020-09-08T15:34:31.935 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:31 smithi104 bash[9203]: audit 2020-09-08T15:34:30.554590+0000 mon.a (mon.0) 793 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd.11", "key": "container_image"}]: dispatch 2020-09-08T15:34:32.811 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:32 smithi166 bash[9446]: cluster 2020-09-08T15:34:31.530609+0000 mgr.x (mgr.24118) 203 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 0 B data, 63 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:32.812 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:32 smithi166 bash[9446]: cluster 2020-09-08T15:34:31.705173+0000 osd.1 (osd.1) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:32.812 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:32 smithi166 bash[9446]: cluster 2020-09-08T15:34:31.705303+0000 osd.1 (osd.1) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:32.894 INFO:teuthology.orchestra.run.smithi166.stdout:Created osd(s) 11 on host 'smithi166' 2020-09-08T15:34:32.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:32 smithi055 bash[9296]: cluster 2020-09-08T15:34:31.530609+0000 mgr.x (mgr.24118) 203 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 0 B data, 63 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:32.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:32 smithi055 bash[9296]: cluster 2020-09-08T15:34:31.705173+0000 osd.1 (osd.1) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:32.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:32 smithi055 bash[9296]: cluster 2020-09-08T15:34:31.705303+0000 osd.1 (osd.1) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:32.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:32 smithi104 bash[9203]: cluster 2020-09-08T15:34:31.530609+0000 mgr.x (mgr.24118) 203 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 0 B data, 63 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:32.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:32 smithi104 bash[9203]: cluster 2020-09-08T15:34:31.705173+0000 osd.1 (osd.1) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:32.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:32 smithi104 bash[9203]: cluster 2020-09-08T15:34:31.705303+0000 osd.1 (osd.1) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:33.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:33 smithi166 bash[9446]: cluster 2020-09-08T15:34:32.537814+0000 mon.a (mon.0) 794 : cluster [DBG] osdmap e86: 12 total, 11 up, 11 in 2020-09-08T15:34:33.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:33 smithi166 bash[9446]: audit 2020-09-08T15:34:32.538222+0000 mon.a (mon.0) 795 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:33.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:33 smithi166 bash[9446]: audit 2020-09-08T15:34:32.887214+0000 mon.a (mon.0) 796 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081275\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081482\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082131\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082658\", \"created\": \"2020-09-08T15:34:07.251523\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_device_update\": \"2020-09-08T15:34:16.551291\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:34:33.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:33 smithi166 bash[9446]: audit 2020-09-08T15:34:32.888275+0000 mon.a (mon.0) 797 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:34:33.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:33 smithi166 bash[9446]: audit 2020-09-08T15:34:32.893081+0000 mon.a (mon.0) 798 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081275\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081482\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082131\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082658\", \"created\": \"2020-09-08T15:34:07.251523\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_device_update\": \"2020-09-08T15:34:16.551291\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:34:33.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:33 smithi055 bash[9296]: cluster 2020-09-08T15:34:32.537814+0000 mon.a (mon.0) 794 : cluster [DBG] osdmap e86: 12 total, 11 up, 11 in 2020-09-08T15:34:33.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:33 smithi055 bash[9296]: audit 2020-09-08T15:34:32.538222+0000 mon.a (mon.0) 795 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:33.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:33 smithi055 bash[9296]: audit 2020-09-08T15:34:32.887214+0000 mon.a (mon.0) 796 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081275\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081482\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082131\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082658\", \"created\": \"2020-09-08T15:34:07.251523\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_device_update\": \"2020-09-08T15:34:16.551291\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:34:33.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:33 smithi055 bash[9296]: audit 2020-09-08T15:34:32.888275+0000 mon.a (mon.0) 797 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:34:33.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:33 smithi055 bash[9296]: audit 2020-09-08T15:34:32.893081+0000 mon.a (mon.0) 798 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081275\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081482\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082131\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082658\", \"created\": \"2020-09-08T15:34:07.251523\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_device_update\": \"2020-09-08T15:34:16.551291\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:34:33.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:33 smithi104 bash[9203]: cluster 2020-09-08T15:34:32.537814+0000 mon.a (mon.0) 794 : cluster [DBG] osdmap e86: 12 total, 11 up, 11 in 2020-09-08T15:34:33.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:33 smithi104 bash[9203]: audit 2020-09-08T15:34:32.538222+0000 mon.a (mon.0) 795 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:33.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:33 smithi104 bash[9203]: audit 2020-09-08T15:34:32.887214+0000 mon.a (mon.0) 796 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081275\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081482\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082131\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082658\", \"created\": \"2020-09-08T15:34:07.251523\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_device_update\": \"2020-09-08T15:34:16.551291\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:34:33.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:33 smithi104 bash[9203]: audit 2020-09-08T15:34:32.888275+0000 mon.a (mon.0) 797 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:34:33.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:33 smithi104 bash[9203]: audit 2020-09-08T15:34:32.893081+0000 mon.a (mon.0) 798 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081275\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.081482\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082131\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:08.082658\", \"created\": \"2020-09-08T15:34:07.251523\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"status\": 1, \"status_desc\": \"starting\", \"is_active\": false}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_device_update\": \"2020-09-08T15:34:16.551291\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:34:34.312 INFO:teuthology.orchestra.run.smithi166:osd.11> sudo journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.11.service 2020-09-08T15:34:34.316 INFO:tasks.cephadm:Setting up client nodes... 2020-09-08T15:34:34.317 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2020-09-08T15:34:34.357 INFO:journalctl@ceph.osd.11.smithi166.stdout:-- Logs begin at Tue 2020-09-01 16:49:30 UTC. -- 2020-09-08T15:34:34.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:34 smithi166 bash[9446]: cluster 2020-09-08T15:34:33.531186+0000 mgr.x (mgr.24118) 204 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 0 B data, 63 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:34.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:34 smithi166 bash[9446]: audit 2020-09-08T15:34:33.888363+0000 mon.a (mon.0) 799 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:34:34.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:34 smithi166 bash[9446]: audit 2020-09-08T15:34:33.894066+0000 mon.a (mon.0) 800 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:34:34.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:34 smithi166 bash[9446]: audit 2020-09-08T15:34:33.895987+0000 mon.a (mon.0) 801 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:34:34.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:34 smithi104 bash[9203]: cluster 2020-09-08T15:34:33.531186+0000 mgr.x (mgr.24118) 204 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 0 B data, 63 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:34.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:34 smithi104 bash[9203]: audit 2020-09-08T15:34:33.888363+0000 mon.a (mon.0) 799 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:34:34.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:34 smithi104 bash[9203]: audit 2020-09-08T15:34:33.894066+0000 mon.a (mon.0) 800 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:34:34.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:34 smithi104 bash[9203]: audit 2020-09-08T15:34:33.895987+0000 mon.a (mon.0) 801 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:34:34.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:34 smithi055 bash[9296]: cluster 2020-09-08T15:34:33.531186+0000 mgr.x (mgr.24118) 204 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 0 B data, 63 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:34.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:34 smithi055 bash[9296]: audit 2020-09-08T15:34:33.888363+0000 mon.a (mon.0) 799 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:34:34.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:34 smithi055 bash[9296]: audit 2020-09-08T15:34:33.894066+0000 mon.a (mon.0) 800 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"Insufficient space (<5GB) on vgs\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"comment\": \"not used by ceph\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:34:34.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:34 smithi055 bash[9296]: audit 2020-09-08T15:34:33.895987+0000 mon.a (mon.0) 801 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T15:34:35.650 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:34:35 smithi166 bash[18960]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-11 2020-09-08T15:34:35.650 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:34:35 smithi166 bash[18960]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_1 --path /var/lib/ceph/osd/ceph-11 --no-mon-config 2020-09-08T15:34:35.651 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:34:35 smithi166 bash[18960]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-11/block 2020-09-08T15:34:35.651 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:34:35 smithi166 bash[18960]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-11/block 2020-09-08T15:34:35.651 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:34:35 smithi166 bash[18960]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2020-09-08T15:34:35.651 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:34:35 smithi166 bash[18960]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-11 2020-09-08T15:34:35.651 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:34:35 smithi166 bash[18960]: --> ceph-volume lvm activate successful for osd ID: 11 2020-09-08T15:34:36.868 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:34:36 smithi166 bash[18960]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:34:36.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:36 smithi166 bash[9446]: cluster 2020-09-08T15:34:35.531788+0000 mgr.x (mgr.24118) 205 : cluster [DBG] pgmap v220: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:36.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:36 smithi166 bash[9446]: cluster 2020-09-08T15:34:36.092735+0000 osd.6 (osd.6) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:36.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:36 smithi166 bash[9446]: cluster 2020-09-08T15:34:36.092904+0000 osd.6 (osd.6) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:36.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:36 smithi055 bash[9296]: cluster 2020-09-08T15:34:35.531788+0000 mgr.x (mgr.24118) 205 : cluster [DBG] pgmap v220: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:36.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:36 smithi055 bash[9296]: cluster 2020-09-08T15:34:36.092735+0000 osd.6 (osd.6) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:36.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:36 smithi055 bash[9296]: cluster 2020-09-08T15:34:36.092904+0000 osd.6 (osd.6) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:36.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:36 smithi104 bash[9203]: cluster 2020-09-08T15:34:35.531788+0000 mgr.x (mgr.24118) 205 : cluster [DBG] pgmap v220: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:36.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:36 smithi104 bash[9203]: cluster 2020-09-08T15:34:36.092735+0000 osd.6 (osd.6) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:36.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:36 smithi104 bash[9203]: cluster 2020-09-08T15:34:36.092904+0000 osd.6 (osd.6) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:37.827 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:37 smithi166 bash[9446]: cluster 2020-09-08T15:34:36.554415+0000 mon.a (mon.0) 802 : cluster [DBG] osdmap e87: 12 total, 11 up, 11 in 2020-09-08T15:34:37.827 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:37 smithi166 bash[9446]: audit 2020-09-08T15:34:36.554589+0000 mon.a (mon.0) 803 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:37.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:37 smithi055 bash[9296]: cluster 2020-09-08T15:34:36.554415+0000 mon.a (mon.0) 802 : cluster [DBG] osdmap e87: 12 total, 11 up, 11 in 2020-09-08T15:34:37.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:37 smithi055 bash[9296]: audit 2020-09-08T15:34:36.554589+0000 mon.a (mon.0) 803 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:37.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:37 smithi104 bash[9203]: cluster 2020-09-08T15:34:36.554415+0000 mon.a (mon.0) 802 : cluster [DBG] osdmap e87: 12 total, 11 up, 11 in 2020-09-08T15:34:37.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:37 smithi104 bash[9203]: audit 2020-09-08T15:34:36.554589+0000 mon.a (mon.0) 803 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:37.946 INFO:teuthology.orchestra.run.smithi104.stdout:[client.0] 2020-09-08T15:34:37.946 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQCNpFdfes9IOBAAaDt4C3hzZL+i+HItt3Qu1w== 2020-09-08T15:34:38.118 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:34:37 smithi166 bash[18960]: debug 2020-09-08T15:34:37.885+0000 7f9c1940a700 10 -- 172.21.15.166:0/3566491129 dispatch_throttle_release 194 to dispatch throttler 194/104857600 2020-09-08T15:34:38.118 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:34:37 smithi166 bash[18960]: debug 2020-09-08T15:34:37.885+0000 7f9c1940a700 20 -- 172.21.15.166:0/3566491129 done calling dispatch on 0x56302e17a4e0 2020-09-08T15:34:38.632 INFO:teuthology.orchestra.run.smithi104:> set -ex 2020-09-08T15:34:38.634 INFO:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2020-09-08T15:34:38.638 INFO:teuthology.orchestra.run.smithi104:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2020-09-08T15:34:38.647 INFO:teuthology.orchestra.run.smithi104.stderr:+ sudo dd of=/etc/ceph/ceph.client.0.keyring 2020-09-08T15:34:38.657 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records in 2020-09-08T15:34:38.658 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records out 2020-09-08T15:34:38.658 INFO:teuthology.orchestra.run.smithi104.stderr:59 bytes copied, 7.5175e-05 s, 785 kB/s 2020-09-08T15:34:38.660 INFO:teuthology.orchestra.run.smithi104.stderr:+ sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2020-09-08T15:34:38.668 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2020-09-08T15:34:38.868 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:34:38 smithi166 bash[18960]: debug 2020-09-08T15:34:38.425+0000 7f9c20e8af40 -1 Falling back to public interface 2020-09-08T15:34:38.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:38 smithi166 bash[9446]: cluster 2020-09-08T15:34:37.532408+0000 mgr.x (mgr.24118) 206 : cluster [DBG] pgmap v222: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:38.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:38 smithi166 bash[9446]: audit 2020-09-08T15:34:37.944028+0000 mon.a (mon.0) 804 : audit [INF] from='client.? v1:172.21.15.104:0/4166044432' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2020-09-08T15:34:38.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:38 smithi166 bash[9446]: audit 2020-09-08T15:34:37.950806+0000 mon.a (mon.0) 805 : audit [INF] from='client.? v1:172.21.15.104:0/4166044432' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2020-09-08T15:34:38.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:38 smithi055 bash[9296]: cluster 2020-09-08T15:34:37.532408+0000 mgr.x (mgr.24118) 206 : cluster [DBG] pgmap v222: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:38.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:38 smithi055 bash[9296]: audit 2020-09-08T15:34:37.944028+0000 mon.a (mon.0) 804 : audit [INF] from='client.? v1:172.21.15.104:0/4166044432' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2020-09-08T15:34:38.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:38 smithi055 bash[9296]: audit 2020-09-08T15:34:37.950806+0000 mon.a (mon.0) 805 : audit [INF] from='client.? v1:172.21.15.104:0/4166044432' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2020-09-08T15:34:38.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:38 smithi104 bash[9203]: cluster 2020-09-08T15:34:37.532408+0000 mgr.x (mgr.24118) 206 : cluster [DBG] pgmap v222: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:38.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:38 smithi104 bash[9203]: audit 2020-09-08T15:34:37.944028+0000 mon.a (mon.0) 804 : audit [INF] from='client.? v1:172.21.15.104:0/4166044432' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2020-09-08T15:34:38.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:38 smithi104 bash[9203]: audit 2020-09-08T15:34:37.950806+0000 mon.a (mon.0) 805 : audit [INF] from='client.? v1:172.21.15.104:0/4166044432' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2020-09-08T15:34:39.374 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:34:39 smithi166 bash[18960]: debug 2020-09-08T15:34:39.077+0000 7f9c20e8af40 -1 osd.11 0 log_to_monitors {default=true} 2020-09-08T15:34:39.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:39 smithi166 bash[9446]: audit 2020-09-08T15:34:39.089190+0000 mon.c (mon.2) 30 : audit [INF] from='osd.11 v1:172.21.15.166:6825/691735125' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2020-09-08T15:34:39.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:39 smithi166 bash[9446]: audit 2020-09-08T15:34:39.091992+0000 mon.a (mon.0) 806 : audit [INF] from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2020-09-08T15:34:39.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:39 smithi055 bash[9296]: audit 2020-09-08T15:34:39.089190+0000 mon.c (mon.2) 30 : audit [INF] from='osd.11 v1:172.21.15.166:6825/691735125' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2020-09-08T15:34:39.914 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:39 smithi055 bash[9296]: audit 2020-09-08T15:34:39.091992+0000 mon.a (mon.0) 806 : audit [INF] from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2020-09-08T15:34:39.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:39 smithi104 bash[9203]: audit 2020-09-08T15:34:39.089190+0000 mon.c (mon.2) 30 : audit [INF] from='osd.11 v1:172.21.15.166:6825/691735125' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2020-09-08T15:34:39.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:39 smithi104 bash[9203]: audit 2020-09-08T15:34:39.091992+0000 mon.a (mon.0) 806 : audit [INF] from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2020-09-08T15:34:40.868 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:34:40 smithi166 bash[18960]: debug 2020-09-08T15:34:40.589+0000 7f9c17406700 -1 osd.11 0 waiting for initial osdmap 2020-09-08T15:34:40.869 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:34:40 smithi166 bash[18960]: debug 2020-09-08T15:34:40.589+0000 7f9c09515700 -1 osd.11 0 failed to load OSD map for epoch 33, got 0 bytes 2020-09-08T15:34:40.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:40 smithi166 bash[9446]: cluster 2020-09-08T15:34:39.532877+0000 mgr.x (mgr.24118) 207 : cluster [DBG] pgmap v223: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:40.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:40 smithi166 bash[9446]: audit 2020-09-08T15:34:39.581380+0000 mon.c (mon.2) 31 : audit [INF] from='osd.11 v1:172.21.15.166:6825/691735125' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:34:40.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:40 smithi166 bash[9446]: audit 2020-09-08T15:34:39.581996+0000 mon.a (mon.0) 807 : audit [INF] from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2020-09-08T15:34:40.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:40 smithi166 bash[9446]: cluster 2020-09-08T15:34:39.582070+0000 mon.a (mon.0) 808 : cluster [DBG] osdmap e88: 12 total, 11 up, 11 in 2020-09-08T15:34:40.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:40 smithi166 bash[9446]: audit 2020-09-08T15:34:39.582283+0000 mon.a (mon.0) 809 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:40.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:40 smithi166 bash[9446]: audit 2020-09-08T15:34:39.584226+0000 mon.a (mon.0) 810 : audit [INF] from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:34:40.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:40 smithi166 bash[9446]: audit 2020-09-08T15:34:39.853244+0000 mon.a (mon.0) 811 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:34:40.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:40 smithi166 bash[9446]: audit 2020-09-08T15:34:40.137751+0000 mon.a (mon.0) 812 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:34:40.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:40 smithi166 bash[9446]: audit 2020-09-08T15:34:40.144015+0000 mon.a (mon.0) 813 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:34:40.872 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:40 smithi166 bash[9446]: audit 2020-09-08T15:34:40.146147+0000 mon.a (mon.0) 814 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:34:40.872 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:40 smithi166 bash[9446]: audit 2020-09-08T15:34:40.150988+0000 mon.a (mon.0) 815 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:34:40.872 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:40 smithi166 bash[9446]: audit 2020-09-08T15:34:40.155694+0000 mon.a (mon.0) 816 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:34:40.872 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:40 smithi166 bash[9446]: audit 2020-09-08T15:34:40.160747+0000 mon.a (mon.0) 817 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:34:40.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:40 smithi055 bash[9296]: cluster 2020-09-08T15:34:39.532877+0000 mgr.x (mgr.24118) 207 : cluster [DBG] pgmap v223: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:40.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:40 smithi055 bash[9296]: audit 2020-09-08T15:34:39.581380+0000 mon.c (mon.2) 31 : audit [INF] from='osd.11 v1:172.21.15.166:6825/691735125' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:34:40.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:40 smithi055 bash[9296]: audit 2020-09-08T15:34:39.581996+0000 mon.a (mon.0) 807 : audit [INF] from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2020-09-08T15:34:40.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:40 smithi055 bash[9296]: cluster 2020-09-08T15:34:39.582070+0000 mon.a (mon.0) 808 : cluster [DBG] osdmap e88: 12 total, 11 up, 11 in 2020-09-08T15:34:40.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:40 smithi055 bash[9296]: audit 2020-09-08T15:34:39.582283+0000 mon.a (mon.0) 809 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:40.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:40 smithi055 bash[9296]: audit 2020-09-08T15:34:39.584226+0000 mon.a (mon.0) 810 : audit [INF] from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:34:40.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:40 smithi055 bash[9296]: audit 2020-09-08T15:34:39.853244+0000 mon.a (mon.0) 811 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:34:40.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:40 smithi055 bash[9296]: audit 2020-09-08T15:34:40.137751+0000 mon.a (mon.0) 812 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:34:40.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:40 smithi055 bash[9296]: audit 2020-09-08T15:34:40.144015+0000 mon.a (mon.0) 813 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:34:40.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:40 smithi055 bash[9296]: audit 2020-09-08T15:34:40.146147+0000 mon.a (mon.0) 814 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:34:40.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:40 smithi055 bash[9296]: audit 2020-09-08T15:34:40.150988+0000 mon.a (mon.0) 815 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:34:40.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:40 smithi055 bash[9296]: audit 2020-09-08T15:34:40.155694+0000 mon.a (mon.0) 816 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:34:40.912 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:40 smithi055 bash[9296]: audit 2020-09-08T15:34:40.160747+0000 mon.a (mon.0) 817 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:34:40.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:40 smithi104 bash[9203]: cluster 2020-09-08T15:34:39.532877+0000 mgr.x (mgr.24118) 207 : cluster [DBG] pgmap v223: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:40.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:40 smithi104 bash[9203]: audit 2020-09-08T15:34:39.581380+0000 mon.c (mon.2) 31 : audit [INF] from='osd.11 v1:172.21.15.166:6825/691735125' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:34:40.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:40 smithi104 bash[9203]: audit 2020-09-08T15:34:39.581996+0000 mon.a (mon.0) 807 : audit [INF] from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2020-09-08T15:34:40.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:40 smithi104 bash[9203]: cluster 2020-09-08T15:34:39.582070+0000 mon.a (mon.0) 808 : cluster [DBG] osdmap e88: 12 total, 11 up, 11 in 2020-09-08T15:34:40.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:40 smithi104 bash[9203]: audit 2020-09-08T15:34:39.582283+0000 mon.a (mon.0) 809 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:40.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:40 smithi104 bash[9203]: audit 2020-09-08T15:34:39.584226+0000 mon.a (mon.0) 810 : audit [INF] from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]: dispatch 2020-09-08T15:34:40.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:40 smithi104 bash[9203]: audit 2020-09-08T15:34:39.853244+0000 mon.a (mon.0) 811 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:34:40.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:40 smithi104 bash[9203]: audit 2020-09-08T15:34:40.137751+0000 mon.a (mon.0) 812 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]: dispatch 2020-09-08T15:34:40.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:40 smithi104 bash[9203]: audit 2020-09-08T15:34:40.144015+0000 mon.a (mon.0) 813 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:28:19.386845\"}"}]': finished 2020-09-08T15:34:40.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:40 smithi104 bash[9203]: audit 2020-09-08T15:34:40.146147+0000 mon.a (mon.0) 814 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:34:40.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:40 smithi104 bash[9203]: audit 2020-09-08T15:34:40.150988+0000 mon.a (mon.0) 815 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:34:40.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:40 smithi104 bash[9203]: audit 2020-09-08T15:34:40.155694+0000 mon.a (mon.0) 816 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:34:40.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:40 smithi104 bash[9203]: audit 2020-09-08T15:34:40.160747+0000 mon.a (mon.0) 817 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:34:41.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:41 smithi166 bash[9446]: audit 2020-09-08T15:34:40.584419+0000 mon.a (mon.0) 818 : audit [INF] from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]': finished 2020-09-08T15:34:41.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:41 smithi166 bash[9446]: cluster 2020-09-08T15:34:40.584587+0000 mon.a (mon.0) 819 : cluster [DBG] osdmap e89: 12 total, 11 up, 11 in 2020-09-08T15:34:41.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:41 smithi166 bash[9446]: audit 2020-09-08T15:34:40.585435+0000 mon.a (mon.0) 820 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:41.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:41 smithi166 bash[9446]: audit 2020-09-08T15:34:40.595999+0000 mon.a (mon.0) 821 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:41.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:41 smithi166 bash[9446]: cluster 2020-09-08T15:34:41.533285+0000 mgr.x (mgr.24118) 208 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:41.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:41 smithi055 bash[9296]: audit 2020-09-08T15:34:40.584419+0000 mon.a (mon.0) 818 : audit [INF] from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]': finished 2020-09-08T15:34:41.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:41 smithi055 bash[9296]: cluster 2020-09-08T15:34:40.584587+0000 mon.a (mon.0) 819 : cluster [DBG] osdmap e89: 12 total, 11 up, 11 in 2020-09-08T15:34:41.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:41 smithi055 bash[9296]: audit 2020-09-08T15:34:40.585435+0000 mon.a (mon.0) 820 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:41.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:41 smithi055 bash[9296]: audit 2020-09-08T15:34:40.595999+0000 mon.a (mon.0) 821 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:41.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:41 smithi055 bash[9296]: cluster 2020-09-08T15:34:41.533285+0000 mgr.x (mgr.24118) 208 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:41.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:41 smithi104 bash[9203]: audit 2020-09-08T15:34:40.584419+0000 mon.a (mon.0) 818 : audit [INF] from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi166", "root=default"]}]': finished 2020-09-08T15:34:41.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:41 smithi104 bash[9203]: cluster 2020-09-08T15:34:40.584587+0000 mon.a (mon.0) 819 : cluster [DBG] osdmap e89: 12 total, 11 up, 11 in 2020-09-08T15:34:41.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:41 smithi104 bash[9203]: audit 2020-09-08T15:34:40.585435+0000 mon.a (mon.0) 820 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:41.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:41 smithi104 bash[9203]: audit 2020-09-08T15:34:40.595999+0000 mon.a (mon.0) 821 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:41.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:41 smithi104 bash[9203]: cluster 2020-09-08T15:34:41.533285+0000 mgr.x (mgr.24118) 208 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 0 B data, 62 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:34:42.336 INFO:teuthology.orchestra.run.smithi055.stdout:[client.1] 2020-09-08T15:34:42.336 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQCSpFdfCOfnExAAupTAI5/qgUkmKMcQ8cZRQw== 2020-09-08T15:34:42.881 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:42 smithi166 bash[9446]: cluster 2020-09-08T15:34:40.059696+0000 osd.11 (osd.11) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:42.883 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:42 smithi166 bash[9446]: cluster 2020-09-08T15:34:40.059866+0000 osd.11 (osd.11) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:42.884 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:42 smithi166 bash[9446]: audit 2020-09-08T15:34:41.591998+0000 mon.a (mon.0) 822 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:42.884 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:42 smithi166 bash[9446]: cluster 2020-09-08T15:34:41.606033+0000 mon.a (mon.0) 823 : cluster [INF] osd.11 v1:172.21.15.166:6825/691735125 boot 2020-09-08T15:34:42.884 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:42 smithi166 bash[9446]: cluster 2020-09-08T15:34:41.606110+0000 mon.a (mon.0) 824 : cluster [DBG] osdmap e90: 12 total, 12 up, 12 in 2020-09-08T15:34:42.884 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:42 smithi166 bash[9446]: audit 2020-09-08T15:34:41.606371+0000 mon.a (mon.0) 825 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:42.884 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:42 smithi166 bash[9446]: audit 2020-09-08T15:34:42.333273+0000 mon.b (mon.1) 68 : audit [INF] from='client.? v1:172.21.15.55:0/3026990978' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2020-09-08T15:34:42.885 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:42 smithi166 bash[9446]: audit 2020-09-08T15:34:42.333750+0000 mon.a (mon.0) 826 : audit [INF] from='client.24460 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2020-09-08T15:34:42.885 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:42 smithi166 bash[9446]: audit 2020-09-08T15:34:42.340373+0000 mon.a (mon.0) 827 : audit [INF] from='client.24460 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2020-09-08T15:34:42.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:42 smithi055 bash[9296]: cluster 2020-09-08T15:34:40.059696+0000 osd.11 (osd.11) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:42.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:42 smithi055 bash[9296]: cluster 2020-09-08T15:34:40.059866+0000 osd.11 (osd.11) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:42.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:42 smithi055 bash[9296]: audit 2020-09-08T15:34:41.591998+0000 mon.a (mon.0) 822 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:42.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:42 smithi055 bash[9296]: cluster 2020-09-08T15:34:41.606033+0000 mon.a (mon.0) 823 : cluster [INF] osd.11 v1:172.21.15.166:6825/691735125 boot 2020-09-08T15:34:42.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:42 smithi055 bash[9296]: cluster 2020-09-08T15:34:41.606110+0000 mon.a (mon.0) 824 : cluster [DBG] osdmap e90: 12 total, 12 up, 12 in 2020-09-08T15:34:42.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:42 smithi055 bash[9296]: audit 2020-09-08T15:34:41.606371+0000 mon.a (mon.0) 825 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:42.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:42 smithi055 bash[9296]: audit 2020-09-08T15:34:42.333273+0000 mon.b (mon.1) 68 : audit [INF] from='client.? v1:172.21.15.55:0/3026990978' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2020-09-08T15:34:42.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:42 smithi055 bash[9296]: audit 2020-09-08T15:34:42.333750+0000 mon.a (mon.0) 826 : audit [INF] from='client.24460 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2020-09-08T15:34:42.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:42 smithi055 bash[9296]: audit 2020-09-08T15:34:42.340373+0000 mon.a (mon.0) 827 : audit [INF] from='client.24460 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2020-09-08T15:34:42.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:42 smithi104 bash[9203]: cluster 2020-09-08T15:34:40.059696+0000 osd.11 (osd.11) 1 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:42.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:42 smithi104 bash[9203]: cluster 2020-09-08T15:34:40.059866+0000 osd.11 (osd.11) 2 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:42.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:42 smithi104 bash[9203]: audit 2020-09-08T15:34:41.591998+0000 mon.a (mon.0) 822 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:42.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:42 smithi104 bash[9203]: cluster 2020-09-08T15:34:41.606033+0000 mon.a (mon.0) 823 : cluster [INF] osd.11 v1:172.21.15.166:6825/691735125 boot 2020-09-08T15:34:42.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:42 smithi104 bash[9203]: cluster 2020-09-08T15:34:41.606110+0000 mon.a (mon.0) 824 : cluster [DBG] osdmap e90: 12 total, 12 up, 12 in 2020-09-08T15:34:42.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:42 smithi104 bash[9203]: audit 2020-09-08T15:34:41.606371+0000 mon.a (mon.0) 825 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2020-09-08T15:34:42.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:42 smithi104 bash[9203]: audit 2020-09-08T15:34:42.333273+0000 mon.b (mon.1) 68 : audit [INF] from='client.? v1:172.21.15.55:0/3026990978' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2020-09-08T15:34:42.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:42 smithi104 bash[9203]: audit 2020-09-08T15:34:42.333750+0000 mon.a (mon.0) 826 : audit [INF] from='client.24460 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2020-09-08T15:34:42.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:42 smithi104 bash[9203]: audit 2020-09-08T15:34:42.340373+0000 mon.a (mon.0) 827 : audit [INF] from='client.24460 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2020-09-08T15:34:43.061 INFO:teuthology.orchestra.run.smithi055:> set -ex 2020-09-08T15:34:43.062 INFO:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2020-09-08T15:34:43.062 INFO:teuthology.orchestra.run.smithi055:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2020-09-08T15:34:43.067 INFO:teuthology.orchestra.run.smithi055.stderr:+ sudo dd of=/etc/ceph/ceph.client.1.keyring 2020-09-08T15:34:43.075 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records in 2020-09-08T15:34:43.076 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records out 2020-09-08T15:34:43.076 INFO:teuthology.orchestra.run.smithi055.stderr:59 bytes copied, 4.8667e-05 s, 1.2 MB/s 2020-09-08T15:34:43.076 INFO:teuthology.orchestra.run.smithi055.stderr:+ sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2020-09-08T15:34:43.100 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph auth get-or-create client.2 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2020-09-08T15:34:43.877 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:43 smithi166 bash[9446]: cluster 2020-09-08T15:34:42.615357+0000 mon.a (mon.0) 828 : cluster [DBG] osdmap e91: 12 total, 12 up, 12 in 2020-09-08T15:34:43.878 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:43 smithi166 bash[9446]: cluster 2020-09-08T15:34:43.533901+0000 mgr.x (mgr.24118) 209 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 0 B data, 68 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:34:43.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:43 smithi055 bash[9296]: cluster 2020-09-08T15:34:42.615357+0000 mon.a (mon.0) 828 : cluster [DBG] osdmap e91: 12 total, 12 up, 12 in 2020-09-08T15:34:43.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:43 smithi055 bash[9296]: cluster 2020-09-08T15:34:43.533901+0000 mgr.x (mgr.24118) 209 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 0 B data, 68 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:34:43.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:43 smithi104 bash[9203]: cluster 2020-09-08T15:34:42.615357+0000 mon.a (mon.0) 828 : cluster [DBG] osdmap e91: 12 total, 12 up, 12 in 2020-09-08T15:34:43.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:43 smithi104 bash[9203]: cluster 2020-09-08T15:34:43.533901+0000 mgr.x (mgr.24118) 209 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 0 B data, 68 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:34:44.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:44 smithi166 bash[9446]: cluster 2020-09-08T15:34:43.811266+0000 osd.4 (osd.4) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:44.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:44 smithi166 bash[9446]: cluster 2020-09-08T15:34:43.811421+0000 osd.4 (osd.4) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:44.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:44 smithi055 bash[9296]: cluster 2020-09-08T15:34:43.811266+0000 osd.4 (osd.4) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:44.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:44 smithi055 bash[9296]: cluster 2020-09-08T15:34:43.811421+0000 osd.4 (osd.4) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:44.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:44 smithi104 bash[9203]: cluster 2020-09-08T15:34:43.811266+0000 osd.4 (osd.4) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:44.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:44 smithi104 bash[9203]: cluster 2020-09-08T15:34:43.811421+0000 osd.4 (osd.4) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:45.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:45 smithi055 bash[9296]: cluster 2020-09-08T15:34:44.628597+0000 mon.a (mon.0) 829 : cluster [DBG] osdmap e92: 12 total, 12 up, 12 in 2020-09-08T15:34:45.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:45 smithi055 bash[9296]: audit 2020-09-08T15:34:44.717747+0000 mon.a (mon.0) 830 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}, {\"id\": \"448d5c57-f6de-443a-ab88-169d81b78450\", \"message\": \"Rebalancing after osd.11 marked in\", \"refs\": [[\"osd\", 11]], \"started_at\": 1599579281.6064675, \"finished_at\": 1599579283.5990696}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:34:45.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:45 smithi055 bash[9296]: audit 2020-09-08T15:34:44.724844+0000 mon.a (mon.0) 831 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}, {\"id\": \"448d5c57-f6de-443a-ab88-169d81b78450\", \"message\": \"Rebalancing after osd.11 marked in\", \"refs\": [[\"osd\", 11]], \"started_at\": 1599579281.6064675, \"finished_at\": 1599579283.5990696}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:34:45.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:45 smithi055 bash[9296]: cluster 2020-09-08T15:34:45.534575+0000 mgr.x (mgr.24118) 210 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 0 B data, 68 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:34:45.933 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:45 smithi104 bash[9203]: cluster 2020-09-08T15:34:44.628597+0000 mon.a (mon.0) 829 : cluster [DBG] osdmap e92: 12 total, 12 up, 12 in 2020-09-08T15:34:45.933 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:45 smithi104 bash[9203]: audit 2020-09-08T15:34:44.717747+0000 mon.a (mon.0) 830 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}, {\"id\": \"448d5c57-f6de-443a-ab88-169d81b78450\", \"message\": \"Rebalancing after osd.11 marked in\", \"refs\": [[\"osd\", 11]], \"started_at\": 1599579281.6064675, \"finished_at\": 1599579283.5990696}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:34:45.933 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:45 smithi104 bash[9203]: audit 2020-09-08T15:34:44.724844+0000 mon.a (mon.0) 831 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}, {\"id\": \"448d5c57-f6de-443a-ab88-169d81b78450\", \"message\": \"Rebalancing after osd.11 marked in\", \"refs\": [[\"osd\", 11]], \"started_at\": 1599579281.6064675, \"finished_at\": 1599579283.5990696}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:34:45.934 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:45 smithi104 bash[9203]: cluster 2020-09-08T15:34:45.534575+0000 mgr.x (mgr.24118) 210 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 0 B data, 68 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:34:45.969 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:45 smithi166 bash[9446]: cluster 2020-09-08T15:34:44.628597+0000 mon.a (mon.0) 829 : cluster [DBG] osdmap e92: 12 total, 12 up, 12 in 2020-09-08T15:34:45.969 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:45 smithi166 bash[9446]: audit 2020-09-08T15:34:44.717747+0000 mon.a (mon.0) 830 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}, {\"id\": \"448d5c57-f6de-443a-ab88-169d81b78450\", \"message\": \"Rebalancing after osd.11 marked in\", \"refs\": [[\"osd\", 11]], \"started_at\": 1599579281.6064675, \"finished_at\": 1599579283.5990696}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:34:45.969 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:45 smithi166 bash[9446]: audit 2020-09-08T15:34:44.724844+0000 mon.a (mon.0) 831 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}, {\"id\": \"448d5c57-f6de-443a-ab88-169d81b78450\", \"message\": \"Rebalancing after osd.11 marked in\", \"refs\": [[\"osd\", 11]], \"started_at\": 1599579281.6064675, \"finished_at\": 1599579283.5990696}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:34:45.970 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:45 smithi166 bash[9446]: cluster 2020-09-08T15:34:45.534575+0000 mgr.x (mgr.24118) 210 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 0 B data, 68 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:34:46.477 INFO:teuthology.orchestra.run.smithi166.stdout:[client.2] 2020-09-08T15:34:46.477 INFO:teuthology.orchestra.run.smithi166.stdout: key = AQCWpFdflB/fGxAAH3UdDyl+PGXcBRfWCcCWgg== 2020-09-08T15:34:46.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:46 smithi166 bash[9446]: audit 2020-09-08T15:34:46.464672+0000 mon.c (mon.2) 32 : audit [INF] from='client.? v1:172.21.15.166:0/2510457111' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2020-09-08T15:34:46.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:46 smithi166 bash[9446]: audit 2020-09-08T15:34:46.467329+0000 mon.a (mon.0) 832 : audit [INF] from='client.34463 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2020-09-08T15:34:46.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:46 smithi166 bash[9446]: audit 2020-09-08T15:34:46.475252+0000 mon.a (mon.0) 833 : audit [INF] from='client.34463 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2020-09-08T15:34:47.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:46 smithi055 bash[9296]: audit 2020-09-08T15:34:46.464672+0000 mon.c (mon.2) 32 : audit [INF] from='client.? v1:172.21.15.166:0/2510457111' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2020-09-08T15:34:47.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:46 smithi055 bash[9296]: audit 2020-09-08T15:34:46.467329+0000 mon.a (mon.0) 832 : audit [INF] from='client.34463 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2020-09-08T15:34:47.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:46 smithi055 bash[9296]: audit 2020-09-08T15:34:46.475252+0000 mon.a (mon.0) 833 : audit [INF] from='client.34463 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2020-09-08T15:34:47.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:46 smithi104 bash[9203]: audit 2020-09-08T15:34:46.464672+0000 mon.c (mon.2) 32 : audit [INF] from='client.? v1:172.21.15.166:0/2510457111' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2020-09-08T15:34:47.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:46 smithi104 bash[9203]: audit 2020-09-08T15:34:46.467329+0000 mon.a (mon.0) 832 : audit [INF] from='client.34463 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2020-09-08T15:34:47.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:46 smithi104 bash[9203]: audit 2020-09-08T15:34:46.475252+0000 mon.a (mon.0) 833 : audit [INF] from='client.34463 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2020-09-08T15:34:47.223 INFO:teuthology.orchestra.run.smithi166:> set -ex 2020-09-08T15:34:47.224 INFO:teuthology.orchestra.run.smithi166:> sudo dd of=/etc/ceph/ceph.client.2.keyring 2020-09-08T15:34:47.224 INFO:teuthology.orchestra.run.smithi166:> sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2020-09-08T15:34:47.228 INFO:teuthology.orchestra.run.smithi166.stderr:+ sudo dd of=/etc/ceph/ceph.client.2.keyring 2020-09-08T15:34:47.233 INFO:teuthology.orchestra.run.smithi166.stderr:0+1 records in 2020-09-08T15:34:47.234 INFO:teuthology.orchestra.run.smithi166.stderr:0+1 records out 2020-09-08T15:34:47.234 INFO:teuthology.orchestra.run.smithi166.stderr:59 bytes copied, 6.5925e-05 s, 895 kB/s 2020-09-08T15:34:47.234 INFO:teuthology.orchestra.run.smithi166.stderr:+ sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2020-09-08T15:34:47.242 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd dump --format=json 2020-09-08T15:34:47.379 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:34:48.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:47 smithi166 bash[9446]: cluster 2020-09-08T15:34:47.535222+0000 mgr.x (mgr.24118) 211 : cluster [DBG] pgmap v232: 1 pgs: 1 active+clean; 0 B data, 68 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:34:48.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:47 smithi055 bash[9296]: cluster 2020-09-08T15:34:47.535222+0000 mgr.x (mgr.24118) 211 : cluster [DBG] pgmap v232: 1 pgs: 1 active+clean; 0 B data, 68 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:34:48.186 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:47 smithi104 bash[9203]: cluster 2020-09-08T15:34:47.535222+0000 mgr.x (mgr.24118) 211 : cluster [DBG] pgmap v232: 1 pgs: 1 active+clean; 0 B data, 68 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:34:50.587 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T15:34:50.588 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":92,"fsid":"759b27b6-f1e7-11ea-a080-001a4aab830c","created":"2020-09-08T15:25:41.788365+0000","modified":"2020-09-08T15:34:44.619541+0000","last_up_change":"2020-09-08T15:34:41.591462+0000","last_in_change":"2020-09-08T15:34:41.591462+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-09-08T15:25:56.314385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"e49a7341-7c75-4883-b061-3e8fe9467e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6801","nonce":82470352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":82470352},{"type":"v1","addr":"172.21.15.104:6803","nonce":82470352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":82470352},{"type":"v1","addr":"172.21.15.104:6807","nonce":82470352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":82470352},{"type":"v1","addr":"172.21.15.104:6805","nonce":82470352}]},"public_addr":"172.21.15.104:6801/82470352","cluster_addr":"172.21.15.104:6803/82470352","heartbeat_back_addr":"172.21.15.104:6807/82470352","heartbeat_front_addr":"172.21.15.104:6805/82470352","state":["exists","up"]},{"osd":1,"uuid":"a83df412-110d-440a-b411-ad2a7c36f81a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6809","nonce":1421798992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1421798992},{"type":"v1","addr":"172.21.15.104:6811","nonce":1421798992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1421798992},{"type":"v1","addr":"172.21.15.104:6815","nonce":1421798992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1421798992},{"type":"v1","addr":"172.21.15.104:6813","nonce":1421798992}]},"public_addr":"172.21.15.104:6809/1421798992","cluster_addr":"172.21.15.104:6811/1421798992","heartbeat_back_addr":"172.21.15.104:6815/1421798992","heartbeat_front_addr":"172.21.15.104:6813/1421798992","state":["exists","up"]},{"osd":2,"uuid":"ca6da1cc-1d8d-494b-93e6-e065ff70d434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6817","nonce":2713413369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2713413369},{"type":"v1","addr":"172.21.15.104:6819","nonce":2713413369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2713413369},{"type":"v1","addr":"172.21.15.104:6823","nonce":2713413369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2713413369},{"type":"v1","addr":"172.21.15.104:6821","nonce":2713413369}]},"public_addr":"172.21.15.104:6817/2713413369","cluster_addr":"172.21.15.104:6819/2713413369","heartbeat_back_addr":"172.21.15.104:6823/2713413369","heartbeat_front_addr":"172.21.15.104:6821/2713413369","state":["exists","up"]},{"osd":3,"uuid":"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6825","nonce":1495946248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1495946248},{"type":"v1","addr":"172.21.15.104:6827","nonce":1495946248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1495946248},{"type":"v1","addr":"172.21.15.104:6831","nonce":1495946248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1495946248},{"type":"v1","addr":"172.21.15.104:6829","nonce":1495946248}]},"public_addr":"172.21.15.104:6825/1495946248","cluster_addr":"172.21.15.104:6827/1495946248","heartbeat_back_addr":"172.21.15.104:6831/1495946248","heartbeat_front_addr":"172.21.15.104:6829/1495946248","state":["exists","up"]},{"osd":4,"uuid":"d8334c70-f776-4652-944d-aed35a8e43f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6803","nonce":1173103260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":1173103260},{"type":"v1","addr":"172.21.15.55:6805","nonce":1173103260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":1173103260},{"type":"v1","addr":"172.21.15.55:6809","nonce":1173103260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":1173103260},{"type":"v1","addr":"172.21.15.55:6807","nonce":1173103260}]},"public_addr":"172.21.15.55:6803/1173103260","cluster_addr":"172.21.15.55:6805/1173103260","heartbeat_back_addr":"172.21.15.55:6809/1173103260","heartbeat_front_addr":"172.21.15.55:6807/1173103260","state":["exists","up"]},{"osd":5,"uuid":"6789abd2-9e9b-4c04-81fb-0ce2d2849e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6811","nonce":1047046376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":1047046376},{"type":"v1","addr":"172.21.15.55:6813","nonce":1047046376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":1047046376},{"type":"v1","addr":"172.21.15.55:6817","nonce":1047046376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":1047046376},{"type":"v1","addr":"172.21.15.55:6815","nonce":1047046376}]},"public_addr":"172.21.15.55:6811/1047046376","cluster_addr":"172.21.15.55:6813/1047046376","heartbeat_back_addr":"172.21.15.55:6817/1047046376","heartbeat_front_addr":"172.21.15.55:6815/1047046376","state":["exists","up"]},{"osd":6,"uuid":"ad9a93fe-0ca0-4052-bfa9-e2abb268829e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6819","nonce":2501844793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":2501844793},{"type":"v1","addr":"172.21.15.55:6821","nonce":2501844793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2501844793},{"type":"v1","addr":"172.21.15.55:6825","nonce":2501844793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":2501844793},{"type":"v1","addr":"172.21.15.55:6823","nonce":2501844793}]},"public_addr":"172.21.15.55:6819/2501844793","cluster_addr":"172.21.15.55:6821/2501844793","heartbeat_back_addr":"172.21.15.55:6825/2501844793","heartbeat_front_addr":"172.21.15.55:6823/2501844793","state":["exists","up"]},{"osd":7,"uuid":"f3795c86-4540-442c-a151-b2769fa577e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6827","nonce":128909153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":128909153},{"type":"v1","addr":"172.21.15.55:6829","nonce":128909153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":128909153},{"type":"v1","addr":"172.21.15.55:6833","nonce":128909153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":128909153},{"type":"v1","addr":"172.21.15.55:6831","nonce":128909153}]},"public_addr":"172.21.15.55:6827/128909153","cluster_addr":"172.21.15.55:6829/128909153","heartbeat_back_addr":"172.21.15.55:6833/128909153","heartbeat_front_addr":"172.21.15.55:6831/128909153","state":["exists","up"]},{"osd":8,"uuid":"3b0ac380-878e-4c1c-9278-80ce67ca2df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6801","nonce":2108117574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6802","nonce":2108117574},{"type":"v1","addr":"172.21.15.166:6803","nonce":2108117574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6806","nonce":2108117574},{"type":"v1","addr":"172.21.15.166:6807","nonce":2108117574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6804","nonce":2108117574},{"type":"v1","addr":"172.21.15.166:6805","nonce":2108117574}]},"public_addr":"172.21.15.166:6801/2108117574","cluster_addr":"172.21.15.166:6803/2108117574","heartbeat_back_addr":"172.21.15.166:6807/2108117574","heartbeat_front_addr":"172.21.15.166:6805/2108117574","state":["exists","up"]},{"osd":9,"uuid":"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6809","nonce":1458870080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6810","nonce":1458870080},{"type":"v1","addr":"172.21.15.166:6811","nonce":1458870080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6814","nonce":1458870080},{"type":"v1","addr":"172.21.15.166:6815","nonce":1458870080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6812","nonce":1458870080},{"type":"v1","addr":"172.21.15.166:6813","nonce":1458870080}]},"public_addr":"172.21.15.166:6809/1458870080","cluster_addr":"172.21.15.166:6811/1458870080","heartbeat_back_addr":"172.21.15.166:6815/1458870080","heartbeat_front_addr":"172.21.15.166:6813/1458870080","state":["exists","up"]},{"osd":10,"uuid":"c194210e-bd89-4a1f-98fb-6bf44f0b324c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6817","nonce":62517848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6818","nonce":62517848},{"type":"v1","addr":"172.21.15.166:6819","nonce":62517848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6822","nonce":62517848},{"type":"v1","addr":"172.21.15.166:6823","nonce":62517848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6820","nonce":62517848},{"type":"v1","addr":"172.21.15.166:6821","nonce":62517848}]},"public_addr":"172.21.15.166:6817/62517848","cluster_addr":"172.21.15.166:6819/62517848","heartbeat_back_addr":"172.21.15.166:6823/62517848","heartbeat_front_addr":"172.21.15.166:6821/62517848","state":["exists","up"]},{"osd":11,"uuid":"121301fe-a41c-40c0-ad09-b00de37a55e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":90,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6825","nonce":691735125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6826","nonce":691735125},{"type":"v1","addr":"172.21.15.166:6827","nonce":691735125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6830","nonce":691735125},{"type":"v1","addr":"172.21.15.166:6831","nonce":691735125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6828","nonce":691735125},{"type":"v1","addr":"172.21.15.166:6829","nonce":691735125}]},"public_addr":"172.21.15.166:6825/691735125","cluster_addr":"172.21.15.166:6827/691735125","heartbeat_back_addr":"172.21.15.166:6831/691735125","heartbeat_front_addr":"172.21.15.166:6829/691735125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:03.959529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:31.705308+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:33:54.913516+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:19.062843+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:43.811427+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:10.955416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:36.092911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:01.462918+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:24.293191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:33:49.783579+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:15.373259+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:40.059871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.104:0/1257365091":"2020-09-09T15:29:39.430456+0000","172.21.15.104:0/1399123455":"2020-09-09T15:29:39.430456+0000","172.21.15.104:0/2810441605":"2020-09-09T15:29:39.430456+0000","172.21.15.104:6800/182241824":"2020-09-09T15:29:39.430456+0000","172.21.15.104:6801/182241824":"2020-09-09T15:29:39.430456+0000","172.21.15.104:0/2021933044":"2020-09-09T15:26:33.248197+0000","172.21.15.104:6800/3129754347":"2020-09-09T15:26:33.248197+0000","172.21.15.104:0/1270031705":"2020-09-09T15:26:33.248197+0000","172.21.15.104:0/2568432190":"2020-09-09T15:26:33.248197+0000","172.21.15.104:6800/1138921844":"2020-09-09T15:26:09.867236+0000","172.21.15.104:0/3827328474":"2020-09-09T15:26:09.867236+0000","172.21.15.104:6801/1138921844":"2020-09-09T15:26:09.867236+0000","172.21.15.104:0/1662035381":"2020-09-09T15:26:09.867236+0000","172.21.15.104:6801/3129754347":"2020-09-09T15:26:33.248197+0000","172.21.15.104:0/2169968102":"2020-09-09T15:26:09.867236+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-09-08T15:34:50.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:50 smithi166 bash[9446]: cluster 2020-09-08T15:34:49.535863+0000 mgr.x (mgr.24118) 212 : cluster [DBG] pgmap v233: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:34:50.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:50 smithi055 bash[9296]: cluster 2020-09-08T15:34:49.535863+0000 mgr.x (mgr.24118) 212 : cluster [DBG] pgmap v233: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:34:50.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:50 smithi104 bash[9203]: cluster 2020-09-08T15:34:49.535863+0000 mgr.x (mgr.24118) 212 : cluster [DBG] pgmap v233: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 keys/s, 0 objects/s recovering 2020-09-08T15:34:51.276 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-09-08T15:25:56.314385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}] 2020-09-08T15:34:51.277 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool get device_health_metrics pg_num 2020-09-08T15:34:51.413 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:34:51.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:51 smithi166 bash[9446]: cluster 2020-09-08T15:34:50.565832+0000 osd.9 (osd.9) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:51.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:51 smithi166 bash[9446]: cluster 2020-09-08T15:34:50.565999+0000 osd.9 (osd.9) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:51.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:51 smithi166 bash[9446]: audit 2020-09-08T15:34:50.589676+0000 mon.a (mon.0) 834 : audit [DBG] from='client.? v1:172.21.15.104:0/3260792427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2020-09-08T15:34:51.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:51 smithi055 bash[9296]: cluster 2020-09-08T15:34:50.565832+0000 osd.9 (osd.9) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:51.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:51 smithi055 bash[9296]: cluster 2020-09-08T15:34:50.565999+0000 osd.9 (osd.9) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:51.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:51 smithi055 bash[9296]: audit 2020-09-08T15:34:50.589676+0000 mon.a (mon.0) 834 : audit [DBG] from='client.? v1:172.21.15.104:0/3260792427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2020-09-08T15:34:51.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:51 smithi104 bash[9203]: cluster 2020-09-08T15:34:50.565832+0000 osd.9 (osd.9) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:51.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:51 smithi104 bash[9203]: cluster 2020-09-08T15:34:50.565999+0000 osd.9 (osd.9) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:51.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:51 smithi104 bash[9203]: audit 2020-09-08T15:34:50.589676+0000 mon.a (mon.0) 834 : audit [DBG] from='client.? v1:172.21.15.104:0/3260792427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2020-09-08T15:34:52.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:52 smithi104 bash[9203]: cluster 2020-09-08T15:34:51.526821+0000 mon.a (mon.0) 835 : cluster [DBG] osdmap e93: 12 total, 12 up, 12 in 2020-09-08T15:34:52.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:52 smithi104 bash[9203]: cluster 2020-09-08T15:34:51.536549+0000 mgr.x (mgr.24118) 213 : cluster [DBG] pgmap v235: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:34:52.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:52 smithi166 bash[9446]: cluster 2020-09-08T15:34:51.526821+0000 mon.a (mon.0) 835 : cluster [DBG] osdmap e93: 12 total, 12 up, 12 in 2020-09-08T15:34:52.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:52 smithi166 bash[9446]: cluster 2020-09-08T15:34:51.536549+0000 mgr.x (mgr.24118) 213 : cluster [DBG] pgmap v235: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:34:52.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:52 smithi055 bash[9296]: cluster 2020-09-08T15:34:51.526821+0000 mon.a (mon.0) 835 : cluster [DBG] osdmap e93: 12 total, 12 up, 12 in 2020-09-08T15:34:52.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:52 smithi055 bash[9296]: cluster 2020-09-08T15:34:51.536549+0000 mgr.x (mgr.24118) 213 : cluster [DBG] pgmap v235: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:34:54.628 INFO:teuthology.orchestra.run.smithi104.stdout:pg_num: 1 2020-09-08T15:34:54.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:54 smithi166 bash[9446]: cluster 2020-09-08T15:34:53.537335+0000 mgr.x (mgr.24118) 214 : cluster [DBG] pgmap v236: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:34:54.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:54 smithi055 bash[9296]: cluster 2020-09-08T15:34:53.537335+0000 mgr.x (mgr.24118) 214 : cluster [DBG] pgmap v236: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:34:54.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:54 smithi104 bash[9203]: cluster 2020-09-08T15:34:53.537335+0000 mgr.x (mgr.24118) 214 : cluster [DBG] pgmap v236: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:34:55.341 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2020-09-08T15:34:55.343 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2020-09-08T15:34:55.343 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph mgr dump --format=json 2020-09-08T15:34:55.476 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:34:55.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:55 smithi104 bash[9203]: audit 2020-09-08T15:34:54.630244+0000 mon.a (mon.0) 836 : audit [DBG] from='client.? v1:172.21.15.104:0/3165716447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2020-09-08T15:34:55.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:55 smithi166 bash[9446]: audit 2020-09-08T15:34:54.630244+0000 mon.a (mon.0) 836 : audit [DBG] from='client.? v1:172.21.15.104:0/3165716447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2020-09-08T15:34:55.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:55 smithi055 bash[9296]: audit 2020-09-08T15:34:54.630244+0000 mon.a (mon.0) 836 : audit [DBG] from='client.? v1:172.21.15.104:0/3165716447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2020-09-08T15:34:56.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:56 smithi166 bash[9446]: cluster 2020-09-08T15:34:55.029254+0000 osd.2 (osd.2) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:56.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:56 smithi166 bash[9446]: cluster 2020-09-08T15:34:55.029383+0000 osd.2 (osd.2) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:56.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:56 smithi166 bash[9446]: cluster 2020-09-08T15:34:55.538068+0000 mgr.x (mgr.24118) 215 : cluster [DBG] pgmap v237: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:34:56.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:56 smithi166 bash[9446]: cluster 2020-09-08T15:34:55.556041+0000 mon.a (mon.0) 837 : cluster [DBG] osdmap e94: 12 total, 12 up, 12 in 2020-09-08T15:34:56.874 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:56 smithi104 bash[9203]: cluster 2020-09-08T15:34:55.029254+0000 osd.2 (osd.2) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:56.875 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:56 smithi104 bash[9203]: cluster 2020-09-08T15:34:55.029383+0000 osd.2 (osd.2) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:56.875 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:56 smithi104 bash[9203]: cluster 2020-09-08T15:34:55.538068+0000 mgr.x (mgr.24118) 215 : cluster [DBG] pgmap v237: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:34:56.875 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:56 smithi104 bash[9203]: cluster 2020-09-08T15:34:55.556041+0000 mon.a (mon.0) 837 : cluster [DBG] osdmap e94: 12 total, 12 up, 12 in 2020-09-08T15:34:56.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:56 smithi055 bash[9296]: cluster 2020-09-08T15:34:55.029254+0000 osd.2 (osd.2) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:34:56.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:56 smithi055 bash[9296]: cluster 2020-09-08T15:34:55.029383+0000 osd.2 (osd.2) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:34:56.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:56 smithi055 bash[9296]: cluster 2020-09-08T15:34:55.538068+0000 mgr.x (mgr.24118) 215 : cluster [DBG] pgmap v237: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:34:56.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:56 smithi055 bash[9296]: cluster 2020-09-08T15:34:55.556041+0000 mon.a (mon.0) 837 : cluster [DBG] osdmap e94: 12 total, 12 up, 12 in 2020-09-08T15:34:58.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:58 smithi104 bash[9203]: cluster 2020-09-08T15:34:57.538749+0000 mgr.x (mgr.24118) 216 : cluster [DBG] pgmap v239: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:34:58.813 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T15:34:58.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:58 smithi166 bash[9446]: cluster 2020-09-08T15:34:57.538749+0000 mgr.x (mgr.24118) 216 : cluster [DBG] pgmap v239: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:34:58.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:58 smithi055 bash[9296]: cluster 2020-09-08T15:34:57.538749+0000 mgr.x (mgr.24118) 216 : cluster [DBG] pgmap v239: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:34:59.496 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":17,"active_gid":24118,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":1575158417},{"type":"v1","addr":"172.21.15.55:6801","nonce":1575158417}]},"active_addr":"172.21.15.55:6801/1575158417","active_change":"2020-09-08T15:29:39.431068+0000","active_mgr_features":4540138292837744639,"available":true,"standbys":[],"modules":["cephadm","dashboard","iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://smithi055.front.sepia.ceph.com:8443/"},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":7,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.55:0","nonce":1149479755}]},{"addrvec":[{"type":"v2","addr":"172.21.15.55:0","nonce":2738789931}]},{"addrvec":[{"type":"v2","addr":"172.21.15.55:0","nonce":3021027729}]}]}} 2020-09-08T15:34:59.499 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2020-09-08T15:34:59.499 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2020-09-08T15:34:59.500 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd dump --format=json 2020-09-08T15:34:59.631 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:34:59.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:34:59 smithi166 bash[9446]: audit 2020-09-08T15:34:58.812597+0000 mon.a (mon.0) 838 : audit [DBG] from='client.? v1:172.21.15.104:0/690550864' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2020-09-08T15:34:59.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:34:59 smithi055 bash[9296]: audit 2020-09-08T15:34:58.812597+0000 mon.a (mon.0) 838 : audit [DBG] from='client.? v1:172.21.15.104:0/690550864' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2020-09-08T15:34:59.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:34:59 smithi104 bash[9203]: audit 2020-09-08T15:34:58.812597+0000 mon.a (mon.0) 838 : audit [DBG] from='client.? v1:172.21.15.104:0/690550864' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2020-09-08T15:35:00.839 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:00 smithi104 bash[9203]: cluster 2020-09-08T15:34:59.539471+0000 mgr.x (mgr.24118) 217 : cluster [DBG] pgmap v240: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:00.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:00 smithi166 bash[9446]: cluster 2020-09-08T15:34:59.539471+0000 mgr.x (mgr.24118) 217 : cluster [DBG] pgmap v240: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:00.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:00 smithi055 bash[9296]: cluster 2020-09-08T15:34:59.539471+0000 mgr.x (mgr.24118) 217 : cluster [DBG] pgmap v240: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:02.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:02 smithi166 bash[9446]: cluster 2020-09-08T15:35:01.540168+0000 mgr.x (mgr.24118) 218 : cluster [DBG] pgmap v241: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:02.903 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T15:35:02.903 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":95,"fsid":"759b27b6-f1e7-11ea-a080-001a4aab830c","created":"2020-09-08T15:25:41.788365+0000","modified":"2020-09-08T15:35:02.569399+0000","last_up_change":"2020-09-08T15:34:41.591462+0000","last_in_change":"2020-09-08T15:34:41.591462+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-09-08T15:25:56.314385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"e49a7341-7c75-4883-b061-3e8fe9467e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6801","nonce":82470352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":82470352},{"type":"v1","addr":"172.21.15.104:6803","nonce":82470352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":82470352},{"type":"v1","addr":"172.21.15.104:6807","nonce":82470352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":82470352},{"type":"v1","addr":"172.21.15.104:6805","nonce":82470352}]},"public_addr":"172.21.15.104:6801/82470352","cluster_addr":"172.21.15.104:6803/82470352","heartbeat_back_addr":"172.21.15.104:6807/82470352","heartbeat_front_addr":"172.21.15.104:6805/82470352","state":["exists","up"]},{"osd":1,"uuid":"a83df412-110d-440a-b411-ad2a7c36f81a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6809","nonce":1421798992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1421798992},{"type":"v1","addr":"172.21.15.104:6811","nonce":1421798992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1421798992},{"type":"v1","addr":"172.21.15.104:6815","nonce":1421798992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1421798992},{"type":"v1","addr":"172.21.15.104:6813","nonce":1421798992}]},"public_addr":"172.21.15.104:6809/1421798992","cluster_addr":"172.21.15.104:6811/1421798992","heartbeat_back_addr":"172.21.15.104:6815/1421798992","heartbeat_front_addr":"172.21.15.104:6813/1421798992","state":["exists","up"]},{"osd":2,"uuid":"ca6da1cc-1d8d-494b-93e6-e065ff70d434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6817","nonce":2713413369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2713413369},{"type":"v1","addr":"172.21.15.104:6819","nonce":2713413369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2713413369},{"type":"v1","addr":"172.21.15.104:6823","nonce":2713413369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2713413369},{"type":"v1","addr":"172.21.15.104:6821","nonce":2713413369}]},"public_addr":"172.21.15.104:6817/2713413369","cluster_addr":"172.21.15.104:6819/2713413369","heartbeat_back_addr":"172.21.15.104:6823/2713413369","heartbeat_front_addr":"172.21.15.104:6821/2713413369","state":["exists","up"]},{"osd":3,"uuid":"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6825","nonce":1495946248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1495946248},{"type":"v1","addr":"172.21.15.104:6827","nonce":1495946248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1495946248},{"type":"v1","addr":"172.21.15.104:6831","nonce":1495946248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1495946248},{"type":"v1","addr":"172.21.15.104:6829","nonce":1495946248}]},"public_addr":"172.21.15.104:6825/1495946248","cluster_addr":"172.21.15.104:6827/1495946248","heartbeat_back_addr":"172.21.15.104:6831/1495946248","heartbeat_front_addr":"172.21.15.104:6829/1495946248","state":["exists","up"]},{"osd":4,"uuid":"d8334c70-f776-4652-944d-aed35a8e43f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6803","nonce":1173103260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":1173103260},{"type":"v1","addr":"172.21.15.55:6805","nonce":1173103260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":1173103260},{"type":"v1","addr":"172.21.15.55:6809","nonce":1173103260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":1173103260},{"type":"v1","addr":"172.21.15.55:6807","nonce":1173103260}]},"public_addr":"172.21.15.55:6803/1173103260","cluster_addr":"172.21.15.55:6805/1173103260","heartbeat_back_addr":"172.21.15.55:6809/1173103260","heartbeat_front_addr":"172.21.15.55:6807/1173103260","state":["exists","up"]},{"osd":5,"uuid":"6789abd2-9e9b-4c04-81fb-0ce2d2849e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6811","nonce":1047046376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":1047046376},{"type":"v1","addr":"172.21.15.55:6813","nonce":1047046376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":1047046376},{"type":"v1","addr":"172.21.15.55:6817","nonce":1047046376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":1047046376},{"type":"v1","addr":"172.21.15.55:6815","nonce":1047046376}]},"public_addr":"172.21.15.55:6811/1047046376","cluster_addr":"172.21.15.55:6813/1047046376","heartbeat_back_addr":"172.21.15.55:6817/1047046376","heartbeat_front_addr":"172.21.15.55:6815/1047046376","state":["exists","up"]},{"osd":6,"uuid":"ad9a93fe-0ca0-4052-bfa9-e2abb268829e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6819","nonce":2501844793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":2501844793},{"type":"v1","addr":"172.21.15.55:6821","nonce":2501844793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2501844793},{"type":"v1","addr":"172.21.15.55:6825","nonce":2501844793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":2501844793},{"type":"v1","addr":"172.21.15.55:6823","nonce":2501844793}]},"public_addr":"172.21.15.55:6819/2501844793","cluster_addr":"172.21.15.55:6821/2501844793","heartbeat_back_addr":"172.21.15.55:6825/2501844793","heartbeat_front_addr":"172.21.15.55:6823/2501844793","state":["exists","up"]},{"osd":7,"uuid":"f3795c86-4540-442c-a151-b2769fa577e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6827","nonce":128909153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":128909153},{"type":"v1","addr":"172.21.15.55:6829","nonce":128909153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":128909153},{"type":"v1","addr":"172.21.15.55:6833","nonce":128909153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":128909153},{"type":"v1","addr":"172.21.15.55:6831","nonce":128909153}]},"public_addr":"172.21.15.55:6827/128909153","cluster_addr":"172.21.15.55:6829/128909153","heartbeat_back_addr":"172.21.15.55:6833/128909153","heartbeat_front_addr":"172.21.15.55:6831/128909153","state":["exists","up"]},{"osd":8,"uuid":"3b0ac380-878e-4c1c-9278-80ce67ca2df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6801","nonce":2108117574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6802","nonce":2108117574},{"type":"v1","addr":"172.21.15.166:6803","nonce":2108117574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6806","nonce":2108117574},{"type":"v1","addr":"172.21.15.166:6807","nonce":2108117574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6804","nonce":2108117574},{"type":"v1","addr":"172.21.15.166:6805","nonce":2108117574}]},"public_addr":"172.21.15.166:6801/2108117574","cluster_addr":"172.21.15.166:6803/2108117574","heartbeat_back_addr":"172.21.15.166:6807/2108117574","heartbeat_front_addr":"172.21.15.166:6805/2108117574","state":["exists","up"]},{"osd":9,"uuid":"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6809","nonce":1458870080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6810","nonce":1458870080},{"type":"v1","addr":"172.21.15.166:6811","nonce":1458870080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6814","nonce":1458870080},{"type":"v1","addr":"172.21.15.166:6815","nonce":1458870080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6812","nonce":1458870080},{"type":"v1","addr":"172.21.15.166:6813","nonce":1458870080}]},"public_addr":"172.21.15.166:6809/1458870080","cluster_addr":"172.21.15.166:6811/1458870080","heartbeat_back_addr":"172.21.15.166:6815/1458870080","heartbeat_front_addr":"172.21.15.166:6813/1458870080","state":["exists","up"]},{"osd":10,"uuid":"c194210e-bd89-4a1f-98fb-6bf44f0b324c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6817","nonce":62517848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6818","nonce":62517848},{"type":"v1","addr":"172.21.15.166:6819","nonce":62517848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6822","nonce":62517848},{"type":"v1","addr":"172.21.15.166:6823","nonce":62517848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6820","nonce":62517848},{"type":"v1","addr":"172.21.15.166:6821","nonce":62517848}]},"public_addr":"172.21.15.166:6817/62517848","cluster_addr":"172.21.15.166:6819/62517848","heartbeat_back_addr":"172.21.15.166:6823/62517848","heartbeat_front_addr":"172.21.15.166:6821/62517848","state":["exists","up"]},{"osd":11,"uuid":"121301fe-a41c-40c0-ad09-b00de37a55e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":90,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6825","nonce":691735125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6826","nonce":691735125},{"type":"v1","addr":"172.21.15.166:6827","nonce":691735125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6830","nonce":691735125},{"type":"v1","addr":"172.21.15.166:6831","nonce":691735125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6828","nonce":691735125},{"type":"v1","addr":"172.21.15.166:6829","nonce":691735125}]},"public_addr":"172.21.15.166:6825/691735125","cluster_addr":"172.21.15.166:6827/691735125","heartbeat_back_addr":"172.21.15.166:6831/691735125","heartbeat_front_addr":"172.21.15.166:6829/691735125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:03.959529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:31.705308+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:55.029388+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:19.062843+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:43.811427+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:10.955416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:36.092911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:35:02.444912+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:24.293191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:50.566005+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:15.373259+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:40.059871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.104:0/1257365091":"2020-09-09T15:29:39.430456+0000","172.21.15.104:0/1399123455":"2020-09-09T15:29:39.430456+0000","172.21.15.104:0/2810441605":"2020-09-09T15:29:39.430456+0000","172.21.15.104:6800/182241824":"2020-09-09T15:29:39.430456+0000","172.21.15.104:6801/182241824":"2020-09-09T15:29:39.430456+0000","172.21.15.104:0/2021933044":"2020-09-09T15:26:33.248197+0000","172.21.15.104:6800/3129754347":"2020-09-09T15:26:33.248197+0000","172.21.15.104:0/1270031705":"2020-09-09T15:26:33.248197+0000","172.21.15.104:0/2568432190":"2020-09-09T15:26:33.248197+0000","172.21.15.104:6800/1138921844":"2020-09-09T15:26:09.867236+0000","172.21.15.104:0/3827328474":"2020-09-09T15:26:09.867236+0000","172.21.15.104:6801/1138921844":"2020-09-09T15:26:09.867236+0000","172.21.15.104:0/1662035381":"2020-09-09T15:26:09.867236+0000","172.21.15.104:6801/3129754347":"2020-09-09T15:26:33.248197+0000","172.21.15.104:0/2169968102":"2020-09-09T15:26:09.867236+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-09-08T15:35:02.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:02 smithi055 bash[9296]: cluster 2020-09-08T15:35:01.540168+0000 mgr.x (mgr.24118) 218 : cluster [DBG] pgmap v241: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:02.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:02 smithi104 bash[9203]: cluster 2020-09-08T15:35:01.540168+0000 mgr.x (mgr.24118) 218 : cluster [DBG] pgmap v241: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:03.560 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2020-09-08T15:35:03.561 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd dump --format=json 2020-09-08T15:35:03.696 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:35:03.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:03 smithi166 bash[9446]: cluster 2020-09-08T15:35:02.444716+0000 osd.7 (osd.7) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:03.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:03 smithi166 bash[9446]: cluster 2020-09-08T15:35:02.444905+0000 osd.7 (osd.7) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:03.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:03 smithi166 bash[9446]: cluster 2020-09-08T15:35:02.582306+0000 mon.a (mon.0) 839 : cluster [DBG] osdmap e95: 12 total, 12 up, 12 in 2020-09-08T15:35:03.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:03 smithi166 bash[9446]: audit 2020-09-08T15:35:02.885682+0000 mon.a (mon.0) 840 : audit [DBG] from='client.? v1:172.21.15.104:0/2022899695' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2020-09-08T15:35:03.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:03 smithi055 bash[9296]: cluster 2020-09-08T15:35:02.444716+0000 osd.7 (osd.7) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:03.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:03 smithi055 bash[9296]: cluster 2020-09-08T15:35:02.444905+0000 osd.7 (osd.7) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:03.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:03 smithi055 bash[9296]: cluster 2020-09-08T15:35:02.582306+0000 mon.a (mon.0) 839 : cluster [DBG] osdmap e95: 12 total, 12 up, 12 in 2020-09-08T15:35:03.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:03 smithi055 bash[9296]: audit 2020-09-08T15:35:02.885682+0000 mon.a (mon.0) 840 : audit [DBG] from='client.? v1:172.21.15.104:0/2022899695' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2020-09-08T15:35:03.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:03 smithi104 bash[9203]: cluster 2020-09-08T15:35:02.444716+0000 osd.7 (osd.7) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:03.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:03 smithi104 bash[9203]: cluster 2020-09-08T15:35:02.444905+0000 osd.7 (osd.7) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:03.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:03 smithi104 bash[9203]: cluster 2020-09-08T15:35:02.582306+0000 mon.a (mon.0) 839 : cluster [DBG] osdmap e95: 12 total, 12 up, 12 in 2020-09-08T15:35:03.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:03 smithi104 bash[9203]: audit 2020-09-08T15:35:02.885682+0000 mon.a (mon.0) 840 : audit [DBG] from='client.? v1:172.21.15.104:0/2022899695' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2020-09-08T15:35:04.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:04 smithi166 bash[9446]: cluster 2020-09-08T15:35:03.540814+0000 mgr.x (mgr.24118) 219 : cluster [DBG] pgmap v243: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:04.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:04 smithi055 bash[9296]: cluster 2020-09-08T15:35:03.540814+0000 mgr.x (mgr.24118) 219 : cluster [DBG] pgmap v243: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:04.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:04 smithi104 bash[9203]: cluster 2020-09-08T15:35:03.540814+0000 mgr.x (mgr.24118) 219 : cluster [DBG] pgmap v243: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:05.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:05 smithi055 bash[9296]: cluster 2020-09-08T15:35:04.018551+0000 osd.0 (osd.0) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:05.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:05 smithi055 bash[9296]: cluster 2020-09-08T15:35:04.018688+0000 osd.0 (osd.0) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:05.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:05 smithi055 bash[9296]: cluster 2020-09-08T15:35:04.628716+0000 mon.a (mon.0) 841 : cluster [DBG] osdmap e96: 12 total, 12 up, 12 in 2020-09-08T15:35:05.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:05 smithi055 bash[9296]: cluster 2020-09-08T15:35:05.541423+0000 mgr.x (mgr.24118) 220 : cluster [DBG] pgmap v245: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:05.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:05 smithi104 bash[9203]: cluster 2020-09-08T15:35:04.018551+0000 osd.0 (osd.0) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:05.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:05 smithi104 bash[9203]: cluster 2020-09-08T15:35:04.018688+0000 osd.0 (osd.0) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:05.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:05 smithi104 bash[9203]: cluster 2020-09-08T15:35:04.628716+0000 mon.a (mon.0) 841 : cluster [DBG] osdmap e96: 12 total, 12 up, 12 in 2020-09-08T15:35:05.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:05 smithi104 bash[9203]: cluster 2020-09-08T15:35:05.541423+0000 mgr.x (mgr.24118) 220 : cluster [DBG] pgmap v245: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:06.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:05 smithi166 bash[9446]: cluster 2020-09-08T15:35:04.018551+0000 osd.0 (osd.0) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:06.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:05 smithi166 bash[9446]: cluster 2020-09-08T15:35:04.018688+0000 osd.0 (osd.0) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:06.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:05 smithi166 bash[9446]: cluster 2020-09-08T15:35:04.628716+0000 mon.a (mon.0) 841 : cluster [DBG] osdmap e96: 12 total, 12 up, 12 in 2020-09-08T15:35:06.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:05 smithi166 bash[9446]: cluster 2020-09-08T15:35:05.541423+0000 mgr.x (mgr.24118) 220 : cluster [DBG] pgmap v245: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:06.879 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T15:35:06.879 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":96,"fsid":"759b27b6-f1e7-11ea-a080-001a4aab830c","created":"2020-09-08T15:25:41.788365+0000","modified":"2020-09-08T15:35:04.608131+0000","last_up_change":"2020-09-08T15:34:41.591462+0000","last_in_change":"2020-09-08T15:34:41.591462+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-09-08T15:25:56.314385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"e49a7341-7c75-4883-b061-3e8fe9467e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6801","nonce":82470352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":82470352},{"type":"v1","addr":"172.21.15.104:6803","nonce":82470352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":82470352},{"type":"v1","addr":"172.21.15.104:6807","nonce":82470352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":82470352},{"type":"v1","addr":"172.21.15.104:6805","nonce":82470352}]},"public_addr":"172.21.15.104:6801/82470352","cluster_addr":"172.21.15.104:6803/82470352","heartbeat_back_addr":"172.21.15.104:6807/82470352","heartbeat_front_addr":"172.21.15.104:6805/82470352","state":["exists","up"]},{"osd":1,"uuid":"a83df412-110d-440a-b411-ad2a7c36f81a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6809","nonce":1421798992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1421798992},{"type":"v1","addr":"172.21.15.104:6811","nonce":1421798992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1421798992},{"type":"v1","addr":"172.21.15.104:6815","nonce":1421798992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1421798992},{"type":"v1","addr":"172.21.15.104:6813","nonce":1421798992}]},"public_addr":"172.21.15.104:6809/1421798992","cluster_addr":"172.21.15.104:6811/1421798992","heartbeat_back_addr":"172.21.15.104:6815/1421798992","heartbeat_front_addr":"172.21.15.104:6813/1421798992","state":["exists","up"]},{"osd":2,"uuid":"ca6da1cc-1d8d-494b-93e6-e065ff70d434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6817","nonce":2713413369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2713413369},{"type":"v1","addr":"172.21.15.104:6819","nonce":2713413369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2713413369},{"type":"v1","addr":"172.21.15.104:6823","nonce":2713413369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2713413369},{"type":"v1","addr":"172.21.15.104:6821","nonce":2713413369}]},"public_addr":"172.21.15.104:6817/2713413369","cluster_addr":"172.21.15.104:6819/2713413369","heartbeat_back_addr":"172.21.15.104:6823/2713413369","heartbeat_front_addr":"172.21.15.104:6821/2713413369","state":["exists","up"]},{"osd":3,"uuid":"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6825","nonce":1495946248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1495946248},{"type":"v1","addr":"172.21.15.104:6827","nonce":1495946248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1495946248},{"type":"v1","addr":"172.21.15.104:6831","nonce":1495946248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1495946248},{"type":"v1","addr":"172.21.15.104:6829","nonce":1495946248}]},"public_addr":"172.21.15.104:6825/1495946248","cluster_addr":"172.21.15.104:6827/1495946248","heartbeat_back_addr":"172.21.15.104:6831/1495946248","heartbeat_front_addr":"172.21.15.104:6829/1495946248","state":["exists","up"]},{"osd":4,"uuid":"d8334c70-f776-4652-944d-aed35a8e43f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6803","nonce":1173103260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":1173103260},{"type":"v1","addr":"172.21.15.55:6805","nonce":1173103260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":1173103260},{"type":"v1","addr":"172.21.15.55:6809","nonce":1173103260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":1173103260},{"type":"v1","addr":"172.21.15.55:6807","nonce":1173103260}]},"public_addr":"172.21.15.55:6803/1173103260","cluster_addr":"172.21.15.55:6805/1173103260","heartbeat_back_addr":"172.21.15.55:6809/1173103260","heartbeat_front_addr":"172.21.15.55:6807/1173103260","state":["exists","up"]},{"osd":5,"uuid":"6789abd2-9e9b-4c04-81fb-0ce2d2849e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6811","nonce":1047046376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":1047046376},{"type":"v1","addr":"172.21.15.55:6813","nonce":1047046376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":1047046376},{"type":"v1","addr":"172.21.15.55:6817","nonce":1047046376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":1047046376},{"type":"v1","addr":"172.21.15.55:6815","nonce":1047046376}]},"public_addr":"172.21.15.55:6811/1047046376","cluster_addr":"172.21.15.55:6813/1047046376","heartbeat_back_addr":"172.21.15.55:6817/1047046376","heartbeat_front_addr":"172.21.15.55:6815/1047046376","state":["exists","up"]},{"osd":6,"uuid":"ad9a93fe-0ca0-4052-bfa9-e2abb268829e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6819","nonce":2501844793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":2501844793},{"type":"v1","addr":"172.21.15.55:6821","nonce":2501844793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2501844793},{"type":"v1","addr":"172.21.15.55:6825","nonce":2501844793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":2501844793},{"type":"v1","addr":"172.21.15.55:6823","nonce":2501844793}]},"public_addr":"172.21.15.55:6819/2501844793","cluster_addr":"172.21.15.55:6821/2501844793","heartbeat_back_addr":"172.21.15.55:6825/2501844793","heartbeat_front_addr":"172.21.15.55:6823/2501844793","state":["exists","up"]},{"osd":7,"uuid":"f3795c86-4540-442c-a151-b2769fa577e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6827","nonce":128909153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":128909153},{"type":"v1","addr":"172.21.15.55:6829","nonce":128909153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":128909153},{"type":"v1","addr":"172.21.15.55:6833","nonce":128909153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":128909153},{"type":"v1","addr":"172.21.15.55:6831","nonce":128909153}]},"public_addr":"172.21.15.55:6827/128909153","cluster_addr":"172.21.15.55:6829/128909153","heartbeat_back_addr":"172.21.15.55:6833/128909153","heartbeat_front_addr":"172.21.15.55:6831/128909153","state":["exists","up"]},{"osd":8,"uuid":"3b0ac380-878e-4c1c-9278-80ce67ca2df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6801","nonce":2108117574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6802","nonce":2108117574},{"type":"v1","addr":"172.21.15.166:6803","nonce":2108117574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6806","nonce":2108117574},{"type":"v1","addr":"172.21.15.166:6807","nonce":2108117574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6804","nonce":2108117574},{"type":"v1","addr":"172.21.15.166:6805","nonce":2108117574}]},"public_addr":"172.21.15.166:6801/2108117574","cluster_addr":"172.21.15.166:6803/2108117574","heartbeat_back_addr":"172.21.15.166:6807/2108117574","heartbeat_front_addr":"172.21.15.166:6805/2108117574","state":["exists","up"]},{"osd":9,"uuid":"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6809","nonce":1458870080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6810","nonce":1458870080},{"type":"v1","addr":"172.21.15.166:6811","nonce":1458870080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6814","nonce":1458870080},{"type":"v1","addr":"172.21.15.166:6815","nonce":1458870080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6812","nonce":1458870080},{"type":"v1","addr":"172.21.15.166:6813","nonce":1458870080}]},"public_addr":"172.21.15.166:6809/1458870080","cluster_addr":"172.21.15.166:6811/1458870080","heartbeat_back_addr":"172.21.15.166:6815/1458870080","heartbeat_front_addr":"172.21.15.166:6813/1458870080","state":["exists","up"]},{"osd":10,"uuid":"c194210e-bd89-4a1f-98fb-6bf44f0b324c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6817","nonce":62517848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6818","nonce":62517848},{"type":"v1","addr":"172.21.15.166:6819","nonce":62517848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6822","nonce":62517848},{"type":"v1","addr":"172.21.15.166:6823","nonce":62517848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6820","nonce":62517848},{"type":"v1","addr":"172.21.15.166:6821","nonce":62517848}]},"public_addr":"172.21.15.166:6817/62517848","cluster_addr":"172.21.15.166:6819/62517848","heartbeat_back_addr":"172.21.15.166:6823/62517848","heartbeat_front_addr":"172.21.15.166:6821/62517848","state":["exists","up"]},{"osd":11,"uuid":"121301fe-a41c-40c0-ad09-b00de37a55e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":90,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6825","nonce":691735125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6826","nonce":691735125},{"type":"v1","addr":"172.21.15.166:6827","nonce":691735125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6830","nonce":691735125},{"type":"v1","addr":"172.21.15.166:6831","nonce":691735125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6828","nonce":691735125},{"type":"v1","addr":"172.21.15.166:6829","nonce":691735125}]},"public_addr":"172.21.15.166:6825/691735125","cluster_addr":"172.21.15.166:6827/691735125","heartbeat_back_addr":"172.21.15.166:6831/691735125","heartbeat_front_addr":"172.21.15.166:6829/691735125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:35:04.018693+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:31.705308+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:55.029388+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:19.062843+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:43.811427+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:10.955416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:36.092911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:35:02.444912+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:24.293191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:50.566005+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:15.373259+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T15:34:40.059871+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.104:0/1257365091":"2020-09-09T15:29:39.430456+0000","172.21.15.104:0/1399123455":"2020-09-09T15:29:39.430456+0000","172.21.15.104:0/2810441605":"2020-09-09T15:29:39.430456+0000","172.21.15.104:6800/182241824":"2020-09-09T15:29:39.430456+0000","172.21.15.104:6801/182241824":"2020-09-09T15:29:39.430456+0000","172.21.15.104:0/2021933044":"2020-09-09T15:26:33.248197+0000","172.21.15.104:6800/3129754347":"2020-09-09T15:26:33.248197+0000","172.21.15.104:0/1270031705":"2020-09-09T15:26:33.248197+0000","172.21.15.104:0/2568432190":"2020-09-09T15:26:33.248197+0000","172.21.15.104:6800/1138921844":"2020-09-09T15:26:09.867236+0000","172.21.15.104:0/3827328474":"2020-09-09T15:26:09.867236+0000","172.21.15.104:6801/1138921844":"2020-09-09T15:26:09.867236+0000","172.21.15.104:0/1662035381":"2020-09-09T15:26:09.867236+0000","172.21.15.104:6801/3129754347":"2020-09-09T15:26:33.248197+0000","172.21.15.104:0/2169968102":"2020-09-09T15:26:09.867236+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-09-08T15:35:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:06 smithi166 bash[9446]: audit 2020-09-08T15:35:06.882507+0000 mon.a (mon.0) 842 : audit [DBG] from='client.? v1:172.21.15.104:0/2420068876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2020-09-08T15:35:07.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:06 smithi055 bash[9296]: audit 2020-09-08T15:35:06.882507+0000 mon.a (mon.0) 842 : audit [DBG] from='client.? v1:172.21.15.104:0/2420068876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2020-09-08T15:35:07.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:06 smithi104 bash[9203]: audit 2020-09-08T15:35:06.882507+0000 mon.a (mon.0) 842 : audit [DBG] from='client.? v1:172.21.15.104:0/2420068876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2020-09-08T15:35:07.561 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2020-09-08T15:35:07.689 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:35:08.373 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:07 smithi166 bash[9446]: cluster 2020-09-08T15:35:07.542071+0000 mgr.x (mgr.24118) 221 : cluster [DBG] pgmap v246: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:08.415 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:07 smithi055 bash[9296]: cluster 2020-09-08T15:35:07.542071+0000 mgr.x (mgr.24118) 221 : cluster [DBG] pgmap v246: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:08.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:07 smithi104 bash[9203]: cluster 2020-09-08T15:35:07.542071+0000 mgr.x (mgr.24118) 221 : cluster [DBG] pgmap v246: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:10.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:10 smithi104 bash[9203]: cluster 2020-09-08T15:35:09.542788+0000 mgr.x (mgr.24118) 222 : cluster [DBG] pgmap v247: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:10.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:10 smithi166 bash[9446]: cluster 2020-09-08T15:35:09.542788+0000 mgr.x (mgr.24118) 222 : cluster [DBG] pgmap v247: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:10.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:10 smithi055 bash[9296]: cluster 2020-09-08T15:35:09.542788+0000 mgr.x (mgr.24118) 222 : cluster [DBG] pgmap v247: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:11.340 INFO:teuthology.orchestra.run.smithi104.stdout:63 2020-09-08T15:35:11.341 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2020-09-08T15:35:11.470 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:35:11.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:11 smithi055 bash[9296]: cluster 2020-09-08T15:35:11.543447+0000 mgr.x (mgr.24118) 223 : cluster [DBG] pgmap v248: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:11.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:11 smithi104 bash[9203]: cluster 2020-09-08T15:35:11.543447+0000 mgr.x (mgr.24118) 223 : cluster [DBG] pgmap v248: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:12.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:11 smithi166 bash[9446]: cluster 2020-09-08T15:35:11.543447+0000 mgr.x (mgr.24118) 223 : cluster [DBG] pgmap v248: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:12.802 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:12 smithi104 bash[9203]: cluster 2020-09-08T15:35:11.782253+0000 osd.5 (osd.5) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:12.802 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:12 smithi104 bash[9203]: cluster 2020-09-08T15:35:11.782443+0000 osd.5 (osd.5) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:12.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:12 smithi055 bash[9296]: cluster 2020-09-08T15:35:11.782253+0000 osd.5 (osd.5) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:12.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:12 smithi055 bash[9296]: cluster 2020-09-08T15:35:11.782443+0000 osd.5 (osd.5) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:13.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:12 smithi166 bash[9446]: cluster 2020-09-08T15:35:11.782253+0000 osd.5 (osd.5) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:13.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:12 smithi166 bash[9446]: cluster 2020-09-08T15:35:11.782443+0000 osd.5 (osd.5) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:13.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:13 smithi104 bash[9203]: cluster 2020-09-08T15:35:12.649641+0000 mon.a (mon.0) 843 : cluster [DBG] osdmap e97: 12 total, 12 up, 12 in 2020-09-08T15:35:13.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:13 smithi104 bash[9203]: cluster 2020-09-08T15:35:13.544053+0000 mgr.x (mgr.24118) 224 : cluster [DBG] pgmap v250: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:14.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:13 smithi166 bash[9446]: cluster 2020-09-08T15:35:12.649641+0000 mon.a (mon.0) 843 : cluster [DBG] osdmap e97: 12 total, 12 up, 12 in 2020-09-08T15:35:14.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:13 smithi166 bash[9446]: cluster 2020-09-08T15:35:13.544053+0000 mgr.x (mgr.24118) 224 : cluster [DBG] pgmap v250: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:14.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:13 smithi055 bash[9296]: cluster 2020-09-08T15:35:12.649641+0000 mon.a (mon.0) 843 : cluster [DBG] osdmap e97: 12 total, 12 up, 12 in 2020-09-08T15:35:14.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:13 smithi055 bash[9296]: cluster 2020-09-08T15:35:13.544053+0000 mgr.x (mgr.24118) 224 : cluster [DBG] pgmap v250: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:15.103 INFO:teuthology.orchestra.run.smithi104.stdout:59 2020-09-08T15:35:15.106 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2020-09-08T15:35:15.237 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:35:15.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:15 smithi055 bash[9296]: cluster 2020-09-08T15:35:15.544674+0000 mgr.x (mgr.24118) 225 : cluster [DBG] pgmap v251: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:15.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:15 smithi104 bash[9203]: cluster 2020-09-08T15:35:15.544674+0000 mgr.x (mgr.24118) 225 : cluster [DBG] pgmap v251: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:16.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:15 smithi166 bash[9446]: cluster 2020-09-08T15:35:15.544674+0000 mgr.x (mgr.24118) 225 : cluster [DBG] pgmap v251: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:17.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:17 smithi055 bash[9296]: cluster 2020-09-08T15:35:16.365095+0000 osd.10 (osd.10) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:17.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:17 smithi055 bash[9296]: cluster 2020-09-08T15:35:16.365227+0000 osd.10 (osd.10) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:17.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:17 smithi055 bash[9296]: cluster 2020-09-08T15:35:16.648730+0000 mon.a (mon.0) 844 : cluster [DBG] osdmap e98: 12 total, 12 up, 12 in 2020-09-08T15:35:17.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:17 smithi055 bash[9296]: cluster 2020-09-08T15:35:17.545270+0000 mgr.x (mgr.24118) 226 : cluster [DBG] pgmap v253: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:17.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:17 smithi104 bash[9203]: cluster 2020-09-08T15:35:16.365095+0000 osd.10 (osd.10) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:17.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:17 smithi104 bash[9203]: cluster 2020-09-08T15:35:16.365227+0000 osd.10 (osd.10) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:17.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:17 smithi104 bash[9203]: cluster 2020-09-08T15:35:16.648730+0000 mon.a (mon.0) 844 : cluster [DBG] osdmap e98: 12 total, 12 up, 12 in 2020-09-08T15:35:17.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:17 smithi104 bash[9203]: cluster 2020-09-08T15:35:17.545270+0000 mgr.x (mgr.24118) 226 : cluster [DBG] pgmap v253: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:18.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:17 smithi166 bash[9446]: cluster 2020-09-08T15:35:16.365095+0000 osd.10 (osd.10) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:18.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:17 smithi166 bash[9446]: cluster 2020-09-08T15:35:16.365227+0000 osd.10 (osd.10) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:18.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:17 smithi166 bash[9446]: cluster 2020-09-08T15:35:16.648730+0000 mon.a (mon.0) 844 : cluster [DBG] osdmap e98: 12 total, 12 up, 12 in 2020-09-08T15:35:18.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:17 smithi166 bash[9446]: cluster 2020-09-08T15:35:17.545270+0000 mgr.x (mgr.24118) 226 : cluster [DBG] pgmap v253: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:18.928 INFO:teuthology.orchestra.run.smithi104.stdout:55 2020-09-08T15:35:18.929 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2020-09-08T15:35:19.067 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:35:20.290 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:20 smithi104 bash[9203]: cluster 2020-09-08T15:35:19.165995+0000 osd.3 (osd.3) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:20.291 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:20 smithi104 bash[9203]: cluster 2020-09-08T15:35:19.166124+0000 osd.3 (osd.3) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:20.291 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:20 smithi104 bash[9203]: cluster 2020-09-08T15:35:19.225823+0000 mon.a (mon.0) 845 : cluster [DBG] osdmap e99: 12 total, 12 up, 12 in 2020-09-08T15:35:20.291 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:20 smithi104 bash[9203]: cluster 2020-09-08T15:35:19.545883+0000 mgr.x (mgr.24118) 227 : cluster [DBG] pgmap v255: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:20.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:20 smithi166 bash[9446]: cluster 2020-09-08T15:35:19.165995+0000 osd.3 (osd.3) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:20.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:20 smithi166 bash[9446]: cluster 2020-09-08T15:35:19.166124+0000 osd.3 (osd.3) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:20.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:20 smithi166 bash[9446]: cluster 2020-09-08T15:35:19.225823+0000 mon.a (mon.0) 845 : cluster [DBG] osdmap e99: 12 total, 12 up, 12 in 2020-09-08T15:35:20.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:20 smithi166 bash[9446]: cluster 2020-09-08T15:35:19.545883+0000 mgr.x (mgr.24118) 227 : cluster [DBG] pgmap v255: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:20.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:20 smithi055 bash[9296]: cluster 2020-09-08T15:35:19.165995+0000 osd.3 (osd.3) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:20.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:20 smithi055 bash[9296]: cluster 2020-09-08T15:35:19.166124+0000 osd.3 (osd.3) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:20.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:20 smithi055 bash[9296]: cluster 2020-09-08T15:35:19.225823+0000 mon.a (mon.0) 845 : cluster [DBG] osdmap e99: 12 total, 12 up, 12 in 2020-09-08T15:35:20.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:20 smithi055 bash[9296]: cluster 2020-09-08T15:35:19.545883+0000 mgr.x (mgr.24118) 227 : cluster [DBG] pgmap v255: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:21.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:21 smithi055 bash[9296]: cluster 2020-09-08T15:35:21.546522+0000 mgr.x (mgr.24118) 228 : cluster [DBG] pgmap v256: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:21.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:21 smithi104 bash[9203]: cluster 2020-09-08T15:35:21.546522+0000 mgr.x (mgr.24118) 228 : cluster [DBG] pgmap v256: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:22.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:21 smithi166 bash[9446]: cluster 2020-09-08T15:35:21.546522+0000 mgr.x (mgr.24118) 228 : cluster [DBG] pgmap v256: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:22.687 INFO:teuthology.orchestra.run.smithi104.stdout:51 2020-09-08T15:35:22.688 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2020-09-08T15:35:22.816 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:35:23.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:23 smithi055 bash[9296]: cluster 2020-09-08T15:35:23.547168+0000 mgr.x (mgr.24118) 229 : cluster [DBG] pgmap v257: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:23.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:23 smithi104 bash[9203]: cluster 2020-09-08T15:35:23.547168+0000 mgr.x (mgr.24118) 229 : cluster [DBG] pgmap v257: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:24.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:23 smithi166 bash[9446]: cluster 2020-09-08T15:35:23.547168+0000 mgr.x (mgr.24118) 229 : cluster [DBG] pgmap v257: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:24.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:24 smithi055 bash[9296]: cluster 2020-09-08T15:35:23.125758+0000 osd.7 (osd.7) 11 : cluster [DBG] 1.0 deep-scrub starts 2020-09-08T15:35:24.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:24 smithi055 bash[9296]: cluster 2020-09-08T15:35:23.132462+0000 osd.7 (osd.7) 12 : cluster [DBG] 1.0 deep-scrub ok 2020-09-08T15:35:24.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:24 smithi104 bash[9203]: cluster 2020-09-08T15:35:23.125758+0000 osd.7 (osd.7) 11 : cluster [DBG] 1.0 deep-scrub starts 2020-09-08T15:35:24.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:24 smithi104 bash[9203]: cluster 2020-09-08T15:35:23.132462+0000 osd.7 (osd.7) 12 : cluster [DBG] 1.0 deep-scrub ok 2020-09-08T15:35:25.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:24 smithi166 bash[9446]: cluster 2020-09-08T15:35:23.125758+0000 osd.7 (osd.7) 11 : cluster [DBG] 1.0 deep-scrub starts 2020-09-08T15:35:25.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:24 smithi166 bash[9446]: cluster 2020-09-08T15:35:23.132462+0000 osd.7 (osd.7) 12 : cluster [DBG] 1.0 deep-scrub ok 2020-09-08T15:35:25.885 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:25 smithi104 bash[9203]: cluster 2020-09-08T15:35:24.441033+0000 osd.8 (osd.8) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:25.885 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:25 smithi104 bash[9203]: cluster 2020-09-08T15:35:24.441213+0000 osd.8 (osd.8) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:25.886 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:25 smithi104 bash[9203]: cluster 2020-09-08T15:35:24.657979+0000 mon.a (mon.0) 846 : cluster [DBG] osdmap e100: 12 total, 12 up, 12 in 2020-09-08T15:35:25.886 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:25 smithi104 bash[9203]: cluster 2020-09-08T15:35:25.547740+0000 mgr.x (mgr.24118) 230 : cluster [DBG] pgmap v259: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:26.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:25 smithi166 bash[9446]: cluster 2020-09-08T15:35:24.441033+0000 osd.8 (osd.8) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:26.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:25 smithi166 bash[9446]: cluster 2020-09-08T15:35:24.441213+0000 osd.8 (osd.8) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:26.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:25 smithi166 bash[9446]: cluster 2020-09-08T15:35:24.657979+0000 mon.a (mon.0) 846 : cluster [DBG] osdmap e100: 12 total, 12 up, 12 in 2020-09-08T15:35:26.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:25 smithi166 bash[9446]: cluster 2020-09-08T15:35:25.547740+0000 mgr.x (mgr.24118) 230 : cluster [DBG] pgmap v259: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:26.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:25 smithi055 bash[9296]: cluster 2020-09-08T15:35:24.441033+0000 osd.8 (osd.8) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:26.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:25 smithi055 bash[9296]: cluster 2020-09-08T15:35:24.441213+0000 osd.8 (osd.8) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:26.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:25 smithi055 bash[9296]: cluster 2020-09-08T15:35:24.657979+0000 mon.a (mon.0) 846 : cluster [DBG] osdmap e100: 12 total, 12 up, 12 in 2020-09-08T15:35:26.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:25 smithi055 bash[9296]: cluster 2020-09-08T15:35:25.547740+0000 mgr.x (mgr.24118) 230 : cluster [DBG] pgmap v259: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:26.476 INFO:teuthology.orchestra.run.smithi104.stdout:47 2020-09-08T15:35:26.477 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2020-09-08T15:35:26.608 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:35:27.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:27 smithi055 bash[9296]: cluster 2020-09-08T15:35:27.548371+0000 mgr.x (mgr.24118) 231 : cluster [DBG] pgmap v260: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:27.909 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:27 smithi104 bash[9203]: cluster 2020-09-08T15:35:27.548371+0000 mgr.x (mgr.24118) 231 : cluster [DBG] pgmap v260: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:28.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:27 smithi166 bash[9446]: cluster 2020-09-08T15:35:27.548371+0000 mgr.x (mgr.24118) 231 : cluster [DBG] pgmap v260: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:30.195 INFO:teuthology.orchestra.run.smithi104.stdout:42 2020-09-08T15:35:30.196 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2020-09-08T15:35:30.323 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:35:30.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:30 smithi166 bash[9446]: cluster 2020-09-08T15:35:29.548994+0000 mgr.x (mgr.24118) 232 : cluster [DBG] pgmap v261: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:30.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:30 smithi055 bash[9296]: cluster 2020-09-08T15:35:29.548994+0000 mgr.x (mgr.24118) 232 : cluster [DBG] pgmap v261: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:30.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:30 smithi104 bash[9203]: cluster 2020-09-08T15:35:29.548994+0000 mgr.x (mgr.24118) 232 : cluster [DBG] pgmap v261: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:31.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:31 smithi055 bash[9296]: cluster 2020-09-08T15:35:31.549619+0000 mgr.x (mgr.24118) 233 : cluster [DBG] pgmap v262: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:31.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:31 smithi104 bash[9203]: cluster 2020-09-08T15:35:31.549619+0000 mgr.x (mgr.24118) 233 : cluster [DBG] pgmap v262: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:32.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:31 smithi166 bash[9446]: cluster 2020-09-08T15:35:31.549619+0000 mgr.x (mgr.24118) 233 : cluster [DBG] pgmap v262: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:32.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:32 smithi104 bash[9203]: cluster 2020-09-08T15:35:31.870095+0000 osd.1 (osd.1) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:32.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:32 smithi104 bash[9203]: cluster 2020-09-08T15:35:31.870248+0000 osd.1 (osd.1) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:33.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:32 smithi166 bash[9446]: cluster 2020-09-08T15:35:31.870095+0000 osd.1 (osd.1) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:33.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:32 smithi166 bash[9446]: cluster 2020-09-08T15:35:31.870248+0000 osd.1 (osd.1) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:33.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:32 smithi055 bash[9296]: cluster 2020-09-08T15:35:31.870095+0000 osd.1 (osd.1) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:33.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:32 smithi055 bash[9296]: cluster 2020-09-08T15:35:31.870248+0000 osd.1 (osd.1) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:33.911 INFO:teuthology.orchestra.run.smithi104.stdout:38 2020-09-08T15:35:33.912 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2020-09-08T15:35:33.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:33 smithi104 bash[9203]: cluster 2020-09-08T15:35:32.655682+0000 mon.a (mon.0) 847 : cluster [DBG] osdmap e101: 12 total, 12 up, 12 in 2020-09-08T15:35:33.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:33 smithi104 bash[9203]: cluster 2020-09-08T15:35:33.550278+0000 mgr.x (mgr.24118) 234 : cluster [DBG] pgmap v264: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:34.050 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:35:34.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:33 smithi166 bash[9446]: cluster 2020-09-08T15:35:32.655682+0000 mon.a (mon.0) 847 : cluster [DBG] osdmap e101: 12 total, 12 up, 12 in 2020-09-08T15:35:34.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:33 smithi166 bash[9446]: cluster 2020-09-08T15:35:33.550278+0000 mgr.x (mgr.24118) 234 : cluster [DBG] pgmap v264: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:34.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:33 smithi055 bash[9296]: cluster 2020-09-08T15:35:32.655682+0000 mon.a (mon.0) 847 : cluster [DBG] osdmap e101: 12 total, 12 up, 12 in 2020-09-08T15:35:34.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:33 smithi055 bash[9296]: cluster 2020-09-08T15:35:33.550278+0000 mgr.x (mgr.24118) 234 : cluster [DBG] pgmap v264: 1 pgs: 1 active+clean; 0 B data, 69 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:35.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:35 smithi055 bash[9296]: cluster 2020-09-08T15:35:35.550921+0000 mgr.x (mgr.24118) 235 : cluster [DBG] pgmap v265: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:35.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:35 smithi104 bash[9203]: cluster 2020-09-08T15:35:35.550921+0000 mgr.x (mgr.24118) 235 : cluster [DBG] pgmap v265: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:36.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:35 smithi166 bash[9446]: cluster 2020-09-08T15:35:35.550921+0000 mgr.x (mgr.24118) 235 : cluster [DBG] pgmap v265: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:37.692 INFO:teuthology.orchestra.run.smithi104.stdout:35 2020-09-08T15:35:37.692 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2020-09-08T15:35:37.872 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:35:38.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:38 smithi166 bash[9446]: cluster 2020-09-08T15:35:36.968391+0000 osd.6 (osd.6) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:38.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:38 smithi166 bash[9446]: cluster 2020-09-08T15:35:36.968561+0000 osd.6 (osd.6) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:38.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:38 smithi166 bash[9446]: cluster 2020-09-08T15:35:37.027184+0000 mon.a (mon.0) 848 : cluster [DBG] osdmap e102: 12 total, 12 up, 12 in 2020-09-08T15:35:38.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:38 smithi166 bash[9446]: cluster 2020-09-08T15:35:37.551615+0000 mgr.x (mgr.24118) 236 : cluster [DBG] pgmap v267: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:38.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:38 smithi055 bash[9296]: cluster 2020-09-08T15:35:36.968391+0000 osd.6 (osd.6) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:38.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:38 smithi055 bash[9296]: cluster 2020-09-08T15:35:36.968561+0000 osd.6 (osd.6) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:38.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:38 smithi055 bash[9296]: cluster 2020-09-08T15:35:37.027184+0000 mon.a (mon.0) 848 : cluster [DBG] osdmap e102: 12 total, 12 up, 12 in 2020-09-08T15:35:38.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:38 smithi055 bash[9296]: cluster 2020-09-08T15:35:37.551615+0000 mgr.x (mgr.24118) 236 : cluster [DBG] pgmap v267: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:38.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:38 smithi104 bash[9203]: cluster 2020-09-08T15:35:36.968391+0000 osd.6 (osd.6) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:38.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:38 smithi104 bash[9203]: cluster 2020-09-08T15:35:36.968561+0000 osd.6 (osd.6) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:38.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:38 smithi104 bash[9203]: cluster 2020-09-08T15:35:37.027184+0000 mon.a (mon.0) 848 : cluster [DBG] osdmap e102: 12 total, 12 up, 12 in 2020-09-08T15:35:38.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:38 smithi104 bash[9203]: cluster 2020-09-08T15:35:37.551615+0000 mgr.x (mgr.24118) 236 : cluster [DBG] pgmap v267: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:40.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:40 smithi166 bash[9446]: cluster 2020-09-08T15:35:39.552280+0000 mgr.x (mgr.24118) 237 : cluster [DBG] pgmap v268: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:40.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:40 smithi166 bash[9446]: cluster 2020-09-08T15:35:40.067611+0000 osd.11 (osd.11) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:40.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:40 smithi166 bash[9446]: cluster 2020-09-08T15:35:40.067755+0000 osd.11 (osd.11) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:40.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:40 smithi055 bash[9296]: cluster 2020-09-08T15:35:39.552280+0000 mgr.x (mgr.24118) 237 : cluster [DBG] pgmap v268: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:40.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:40 smithi055 bash[9296]: cluster 2020-09-08T15:35:40.067611+0000 osd.11 (osd.11) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:40.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:40 smithi055 bash[9296]: cluster 2020-09-08T15:35:40.067755+0000 osd.11 (osd.11) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:40.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:40 smithi104 bash[9203]: cluster 2020-09-08T15:35:39.552280+0000 mgr.x (mgr.24118) 237 : cluster [DBG] pgmap v268: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:40.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:40 smithi104 bash[9203]: cluster 2020-09-08T15:35:40.067611+0000 osd.11 (osd.11) 3 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:40.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:40 smithi104 bash[9203]: cluster 2020-09-08T15:35:40.067755+0000 osd.11 (osd.11) 4 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:41.574 INFO:teuthology.orchestra.run.smithi104.stdout:30 2020-09-08T15:35:41.575 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2020-09-08T15:35:41.701 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:35:41.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:41 smithi166 bash[9446]: cluster 2020-09-08T15:35:40.537454+0000 mon.a (mon.0) 849 : cluster [DBG] osdmap e103: 12 total, 12 up, 12 in 2020-09-08T15:35:41.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:41 smithi055 bash[9296]: cluster 2020-09-08T15:35:40.537454+0000 mon.a (mon.0) 849 : cluster [DBG] osdmap e103: 12 total, 12 up, 12 in 2020-09-08T15:35:41.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:41 smithi104 bash[9203]: cluster 2020-09-08T15:35:40.537454+0000 mon.a (mon.0) 849 : cluster [DBG] osdmap e103: 12 total, 12 up, 12 in 2020-09-08T15:35:42.812 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:42 smithi104 bash[9203]: cluster 2020-09-08T15:35:41.552927+0000 mgr.x (mgr.24118) 238 : cluster [DBG] pgmap v270: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:42.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:42 smithi166 bash[9446]: cluster 2020-09-08T15:35:41.552927+0000 mgr.x (mgr.24118) 238 : cluster [DBG] pgmap v270: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:42.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:42 smithi055 bash[9296]: cluster 2020-09-08T15:35:41.552927+0000 mgr.x (mgr.24118) 238 : cluster [DBG] pgmap v270: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:43.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:43 smithi055 bash[9296]: cluster 2020-09-08T15:35:43.553630+0000 mgr.x (mgr.24118) 239 : cluster [DBG] pgmap v271: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:43.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:43 smithi104 bash[9203]: cluster 2020-09-08T15:35:43.553630+0000 mgr.x (mgr.24118) 239 : cluster [DBG] pgmap v271: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:44.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:43 smithi166 bash[9446]: cluster 2020-09-08T15:35:43.553630+0000 mgr.x (mgr.24118) 239 : cluster [DBG] pgmap v271: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:45.398 INFO:teuthology.orchestra.run.smithi104.stdout:25 2020-09-08T15:35:45.399 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2020-09-08T15:35:45.532 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:35:45.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:45 smithi104 bash[9203]: cluster 2020-09-08T15:35:44.774027+0000 osd.4 (osd.4) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:45.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:45 smithi104 bash[9203]: cluster 2020-09-08T15:35:44.774166+0000 osd.4 (osd.4) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:45.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:45 smithi166 bash[9446]: cluster 2020-09-08T15:35:44.774027+0000 osd.4 (osd.4) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:45.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:45 smithi166 bash[9446]: cluster 2020-09-08T15:35:44.774166+0000 osd.4 (osd.4) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:45.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:45 smithi055 bash[9296]: cluster 2020-09-08T15:35:44.774027+0000 osd.4 (osd.4) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:45.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:45 smithi055 bash[9296]: cluster 2020-09-08T15:35:44.774166+0000 osd.4 (osd.4) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:46.761 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:46 smithi104 bash[9203]: cluster 2020-09-08T15:35:45.540511+0000 mon.a (mon.0) 850 : cluster [DBG] osdmap e104: 12 total, 12 up, 12 in 2020-09-08T15:35:46.762 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:46 smithi104 bash[9203]: cluster 2020-09-08T15:35:45.554142+0000 mgr.x (mgr.24118) 240 : cluster [DBG] pgmap v273: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:46.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:46 smithi166 bash[9446]: cluster 2020-09-08T15:35:45.540511+0000 mon.a (mon.0) 850 : cluster [DBG] osdmap e104: 12 total, 12 up, 12 in 2020-09-08T15:35:46.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:46 smithi166 bash[9446]: cluster 2020-09-08T15:35:45.554142+0000 mgr.x (mgr.24118) 240 : cluster [DBG] pgmap v273: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:46.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:46 smithi055 bash[9296]: cluster 2020-09-08T15:35:45.540511+0000 mon.a (mon.0) 850 : cluster [DBG] osdmap e104: 12 total, 12 up, 12 in 2020-09-08T15:35:46.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:46 smithi055 bash[9296]: cluster 2020-09-08T15:35:45.554142+0000 mgr.x (mgr.24118) 240 : cluster [DBG] pgmap v273: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:47.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:47 smithi055 bash[9296]: cluster 2020-09-08T15:35:47.554753+0000 mgr.x (mgr.24118) 241 : cluster [DBG] pgmap v274: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:47.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:47 smithi104 bash[9203]: cluster 2020-09-08T15:35:47.554753+0000 mgr.x (mgr.24118) 241 : cluster [DBG] pgmap v274: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:48.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:47 smithi166 bash[9446]: cluster 2020-09-08T15:35:47.554753+0000 mgr.x (mgr.24118) 241 : cluster [DBG] pgmap v274: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:49.211 INFO:teuthology.orchestra.run.smithi104.stdout:22 2020-09-08T15:35:49.212 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2020-09-08T15:35:49.339 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:35:50.568 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:50 smithi104 bash[9203]: cluster 2020-09-08T15:35:49.555314+0000 mgr.x (mgr.24118) 242 : cluster [DBG] pgmap v275: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:50.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:50 smithi166 bash[9446]: cluster 2020-09-08T15:35:49.555314+0000 mgr.x (mgr.24118) 242 : cluster [DBG] pgmap v275: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:50.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:50 smithi055 bash[9296]: cluster 2020-09-08T15:35:49.555314+0000 mgr.x (mgr.24118) 242 : cluster [DBG] pgmap v275: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:52.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:52 smithi104 bash[9203]: cluster 2020-09-08T15:35:51.513793+0000 osd.9 (osd.9) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:52.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:52 smithi104 bash[9203]: cluster 2020-09-08T15:35:51.513958+0000 osd.9 (osd.9) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:52.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:52 smithi104 bash[9203]: cluster 2020-09-08T15:35:51.549206+0000 mon.a (mon.0) 851 : cluster [DBG] osdmap e105: 12 total, 12 up, 12 in 2020-09-08T15:35:52.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:52 smithi104 bash[9203]: cluster 2020-09-08T15:35:51.555871+0000 mgr.x (mgr.24118) 243 : cluster [DBG] pgmap v277: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:52.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:52 smithi166 bash[9446]: cluster 2020-09-08T15:35:51.513793+0000 osd.9 (osd.9) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:52.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:52 smithi166 bash[9446]: cluster 2020-09-08T15:35:51.513958+0000 osd.9 (osd.9) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:52.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:52 smithi166 bash[9446]: cluster 2020-09-08T15:35:51.549206+0000 mon.a (mon.0) 851 : cluster [DBG] osdmap e105: 12 total, 12 up, 12 in 2020-09-08T15:35:52.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:52 smithi166 bash[9446]: cluster 2020-09-08T15:35:51.555871+0000 mgr.x (mgr.24118) 243 : cluster [DBG] pgmap v277: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:52.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:52 smithi055 bash[9296]: cluster 2020-09-08T15:35:51.513793+0000 osd.9 (osd.9) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:52.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:52 smithi055 bash[9296]: cluster 2020-09-08T15:35:51.513958+0000 osd.9 (osd.9) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:52.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:52 smithi055 bash[9296]: cluster 2020-09-08T15:35:51.549206+0000 mon.a (mon.0) 851 : cluster [DBG] osdmap e105: 12 total, 12 up, 12 in 2020-09-08T15:35:52.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:52 smithi055 bash[9296]: cluster 2020-09-08T15:35:51.555871+0000 mgr.x (mgr.24118) 243 : cluster [DBG] pgmap v277: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:52.963 INFO:teuthology.orchestra.run.smithi104.stdout:17 2020-09-08T15:35:52.964 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.0 2020-09-08T15:35:53.091 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:35:53.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:53 smithi055 bash[9296]: cluster 2020-09-08T15:35:53.556485+0000 mgr.x (mgr.24118) 244 : cluster [DBG] pgmap v278: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:53.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:53 smithi104 bash[9203]: cluster 2020-09-08T15:35:53.556485+0000 mgr.x (mgr.24118) 244 : cluster [DBG] pgmap v278: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:54.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:53 smithi166 bash[9446]: cluster 2020-09-08T15:35:53.556485+0000 mgr.x (mgr.24118) 244 : cluster [DBG] pgmap v278: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:56.171 INFO:teuthology.orchestra.run.smithi104.stdout:71 2020-09-08T15:35:56.836 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:56 smithi104 bash[9203]: cluster 2020-09-08T15:35:55.202062+0000 osd.2 (osd.2) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:56.836 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:56 smithi104 bash[9203]: cluster 2020-09-08T15:35:55.202204+0000 osd.2 (osd.2) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:56.836 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:56 smithi104 bash[9203]: cluster 2020-09-08T15:35:55.531321+0000 mon.a (mon.0) 852 : cluster [DBG] osdmap e106: 12 total, 12 up, 12 in 2020-09-08T15:35:56.837 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:56 smithi104 bash[9203]: cluster 2020-09-08T15:35:55.557036+0000 mgr.x (mgr.24118) 245 : cluster [DBG] pgmap v280: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:56.837 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:56 smithi104 bash[9203]: audit 2020-09-08T15:35:56.175671+0000 mon.a (mon.0) 853 : audit [DBG] from='client.? v1:172.21.15.104:0/2398923222' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2020-09-08T15:35:56.838 INFO:tasks.cephadm.ceph_manager.ceph:need seq 63 got 71 for osd.0 2020-09-08T15:35:56.839 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.1 2020-09-08T15:35:56.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:56 smithi166 bash[9446]: cluster 2020-09-08T15:35:55.202062+0000 osd.2 (osd.2) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:56.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:56 smithi166 bash[9446]: cluster 2020-09-08T15:35:55.202204+0000 osd.2 (osd.2) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:56.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:56 smithi166 bash[9446]: cluster 2020-09-08T15:35:55.531321+0000 mon.a (mon.0) 852 : cluster [DBG] osdmap e106: 12 total, 12 up, 12 in 2020-09-08T15:35:56.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:56 smithi166 bash[9446]: cluster 2020-09-08T15:35:55.557036+0000 mgr.x (mgr.24118) 245 : cluster [DBG] pgmap v280: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:56.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:56 smithi166 bash[9446]: audit 2020-09-08T15:35:56.175671+0000 mon.a (mon.0) 853 : audit [DBG] from='client.? v1:172.21.15.104:0/2398923222' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2020-09-08T15:35:56.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:56 smithi055 bash[9296]: cluster 2020-09-08T15:35:55.202062+0000 osd.2 (osd.2) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:35:56.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:56 smithi055 bash[9296]: cluster 2020-09-08T15:35:55.202204+0000 osd.2 (osd.2) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:35:56.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:56 smithi055 bash[9296]: cluster 2020-09-08T15:35:55.531321+0000 mon.a (mon.0) 852 : cluster [DBG] osdmap e106: 12 total, 12 up, 12 in 2020-09-08T15:35:56.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:56 smithi055 bash[9296]: cluster 2020-09-08T15:35:55.557036+0000 mgr.x (mgr.24118) 245 : cluster [DBG] pgmap v280: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:56.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:56 smithi055 bash[9296]: audit 2020-09-08T15:35:56.175671+0000 mon.a (mon.0) 853 : audit [DBG] from='client.? v1:172.21.15.104:0/2398923222' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2020-09-08T15:35:56.973 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:35:57.918 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:35:57 smithi055 bash[9296]: cluster 2020-09-08T15:35:57.557687+0000 mgr.x (mgr.24118) 246 : cluster [DBG] pgmap v281: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:57.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:35:57 smithi104 bash[9203]: cluster 2020-09-08T15:35:57.557687+0000 mgr.x (mgr.24118) 246 : cluster [DBG] pgmap v281: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:58.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:35:57 smithi166 bash[9446]: cluster 2020-09-08T15:35:57.557687+0000 mgr.x (mgr.24118) 246 : cluster [DBG] pgmap v281: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:35:59.997 INFO:teuthology.orchestra.run.smithi104.stdout:68 2020-09-08T15:36:00.683 INFO:tasks.cephadm.ceph_manager.ceph:need seq 59 got 68 for osd.1 2020-09-08T15:36:00.684 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.2 2020-09-08T15:36:00.820 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:36:00.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:00 smithi166 bash[9446]: cluster 2020-09-08T15:35:59.558337+0000 mgr.x (mgr.24118) 247 : cluster [DBG] pgmap v282: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:00.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:00 smithi166 bash[9446]: audit 2020-09-08T15:36:00.001923+0000 mon.a (mon.0) 854 : audit [DBG] from='client.? v1:172.21.15.104:0/3979249521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2020-09-08T15:36:00.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:00 smithi055 bash[9296]: cluster 2020-09-08T15:35:59.558337+0000 mgr.x (mgr.24118) 247 : cluster [DBG] pgmap v282: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:00.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:00 smithi055 bash[9296]: audit 2020-09-08T15:36:00.001923+0000 mon.a (mon.0) 854 : audit [DBG] from='client.? v1:172.21.15.104:0/3979249521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2020-09-08T15:36:00.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:00 smithi104 bash[9203]: cluster 2020-09-08T15:35:59.558337+0000 mgr.x (mgr.24118) 247 : cluster [DBG] pgmap v282: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:00.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:00 smithi104 bash[9203]: audit 2020-09-08T15:36:00.001923+0000 mon.a (mon.0) 854 : audit [DBG] from='client.? v1:172.21.15.104:0/3979249521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2020-09-08T15:36:01.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:01 smithi055 bash[9296]: cluster 2020-09-08T15:36:01.558899+0000 mgr.x (mgr.24118) 248 : cluster [DBG] pgmap v283: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:01.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:01 smithi104 bash[9203]: cluster 2020-09-08T15:36:01.558899+0000 mgr.x (mgr.24118) 248 : cluster [DBG] pgmap v283: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:02.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:01 smithi166 bash[9446]: cluster 2020-09-08T15:36:01.558899+0000 mgr.x (mgr.24118) 248 : cluster [DBG] pgmap v283: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:03.877 INFO:teuthology.orchestra.run.smithi104.stdout:64 2020-09-08T15:36:04.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:03 smithi166 bash[9446]: cluster 2020-09-08T15:36:02.771553+0000 osd.7 (osd.7) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:04.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:03 smithi166 bash[9446]: cluster 2020-09-08T15:36:02.771725+0000 osd.7 (osd.7) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:04.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:03 smithi166 bash[9446]: cluster 2020-09-08T15:36:02.829963+0000 mon.a (mon.0) 855 : cluster [DBG] osdmap e107: 12 total, 12 up, 12 in 2020-09-08T15:36:04.123 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:03 smithi166 bash[9446]: cluster 2020-09-08T15:36:03.559513+0000 mgr.x (mgr.24118) 249 : cluster [DBG] pgmap v285: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:04.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:03 smithi055 bash[9296]: cluster 2020-09-08T15:36:02.771553+0000 osd.7 (osd.7) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:04.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:03 smithi055 bash[9296]: cluster 2020-09-08T15:36:02.771725+0000 osd.7 (osd.7) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:04.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:03 smithi055 bash[9296]: cluster 2020-09-08T15:36:02.829963+0000 mon.a (mon.0) 855 : cluster [DBG] osdmap e107: 12 total, 12 up, 12 in 2020-09-08T15:36:04.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:03 smithi055 bash[9296]: cluster 2020-09-08T15:36:03.559513+0000 mgr.x (mgr.24118) 249 : cluster [DBG] pgmap v285: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:04.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:03 smithi104 bash[9203]: cluster 2020-09-08T15:36:02.771553+0000 osd.7 (osd.7) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:04.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:03 smithi104 bash[9203]: cluster 2020-09-08T15:36:02.771725+0000 osd.7 (osd.7) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:04.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:03 smithi104 bash[9203]: cluster 2020-09-08T15:36:02.829963+0000 mon.a (mon.0) 855 : cluster [DBG] osdmap e107: 12 total, 12 up, 12 in 2020-09-08T15:36:04.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:03 smithi104 bash[9203]: cluster 2020-09-08T15:36:03.559513+0000 mgr.x (mgr.24118) 249 : cluster [DBG] pgmap v285: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:04.529 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55 got 64 for osd.2 2020-09-08T15:36:04.647 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.3 2020-09-08T15:36:04.784 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:36:05.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:04 smithi166 bash[9446]: audit 2020-09-08T15:36:03.881351+0000 mon.a (mon.0) 856 : audit [DBG] from='client.? v1:172.21.15.104:0/1630694550' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2020-09-08T15:36:05.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:04 smithi055 bash[9296]: audit 2020-09-08T15:36:03.881351+0000 mon.a (mon.0) 856 : audit [DBG] from='client.? v1:172.21.15.104:0/1630694550' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2020-09-08T15:36:05.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:04 smithi104 bash[9203]: audit 2020-09-08T15:36:03.881351+0000 mon.a (mon.0) 856 : audit [DBG] from='client.? v1:172.21.15.104:0/1630694550' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2020-09-08T15:36:06.037 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:05 smithi104 bash[9203]: cluster 2020-09-08T15:36:04.176819+0000 osd.0 (osd.0) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:06.037 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:05 smithi104 bash[9203]: cluster 2020-09-08T15:36:04.176954+0000 osd.0 (osd.0) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:06.037 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:05 smithi104 bash[9203]: cluster 2020-09-08T15:36:04.850125+0000 mon.a (mon.0) 857 : cluster [DBG] osdmap e108: 12 total, 12 up, 12 in 2020-09-08T15:36:06.038 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:05 smithi104 bash[9203]: cluster 2020-09-08T15:36:05.560232+0000 mgr.x (mgr.24118) 250 : cluster [DBG] pgmap v287: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:06.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:05 smithi166 bash[9446]: cluster 2020-09-08T15:36:04.176819+0000 osd.0 (osd.0) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:06.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:05 smithi166 bash[9446]: cluster 2020-09-08T15:36:04.176954+0000 osd.0 (osd.0) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:06.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:05 smithi166 bash[9446]: cluster 2020-09-08T15:36:04.850125+0000 mon.a (mon.0) 857 : cluster [DBG] osdmap e108: 12 total, 12 up, 12 in 2020-09-08T15:36:06.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:05 smithi166 bash[9446]: cluster 2020-09-08T15:36:05.560232+0000 mgr.x (mgr.24118) 250 : cluster [DBG] pgmap v287: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:06.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:05 smithi055 bash[9296]: cluster 2020-09-08T15:36:04.176819+0000 osd.0 (osd.0) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:06.164 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:05 smithi055 bash[9296]: cluster 2020-09-08T15:36:04.176954+0000 osd.0 (osd.0) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:06.164 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:05 smithi055 bash[9296]: cluster 2020-09-08T15:36:04.850125+0000 mon.a (mon.0) 857 : cluster [DBG] osdmap e108: 12 total, 12 up, 12 in 2020-09-08T15:36:06.165 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:05 smithi055 bash[9296]: cluster 2020-09-08T15:36:05.560232+0000 mgr.x (mgr.24118) 250 : cluster [DBG] pgmap v287: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:07.844 INFO:teuthology.orchestra.run.smithi104.stdout:60 2020-09-08T15:36:07.910 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:07 smithi104 bash[9203]: cluster 2020-09-08T15:36:07.561019+0000 mgr.x (mgr.24118) 251 : cluster [DBG] pgmap v288: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:07.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:07 smithi055 bash[9296]: cluster 2020-09-08T15:36:07.561019+0000 mgr.x (mgr.24118) 251 : cluster [DBG] pgmap v288: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:08.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:07 smithi166 bash[9446]: cluster 2020-09-08T15:36:07.561019+0000 mgr.x (mgr.24118) 251 : cluster [DBG] pgmap v288: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:08.510 INFO:tasks.cephadm.ceph_manager.ceph:need seq 51 got 60 for osd.3 2020-09-08T15:36:08.511 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.4 2020-09-08T15:36:08.647 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:36:08.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:08 smithi055 bash[9296]: audit 2020-09-08T15:36:07.848134+0000 mon.a (mon.0) 858 : audit [DBG] from='client.? v1:172.21.15.104:0/2962787082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2020-09-08T15:36:08.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:08 smithi104 bash[9203]: audit 2020-09-08T15:36:07.848134+0000 mon.a (mon.0) 858 : audit [DBG] from='client.? v1:172.21.15.104:0/2962787082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2020-09-08T15:36:09.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:08 smithi166 bash[9446]: audit 2020-09-08T15:36:07.848134+0000 mon.a (mon.0) 858 : audit [DBG] from='client.? v1:172.21.15.104:0/2962787082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2020-09-08T15:36:10.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:10 smithi166 bash[9446]: cluster 2020-09-08T15:36:09.561709+0000 mgr.x (mgr.24118) 252 : cluster [DBG] pgmap v289: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:10.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:10 smithi055 bash[9296]: cluster 2020-09-08T15:36:09.561709+0000 mgr.x (mgr.24118) 252 : cluster [DBG] pgmap v289: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:10.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:10 smithi104 bash[9203]: cluster 2020-09-08T15:36:09.561709+0000 mgr.x (mgr.24118) 252 : cluster [DBG] pgmap v289: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:11.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:11 smithi055 bash[9296]: cluster 2020-09-08T15:36:11.562335+0000 mgr.x (mgr.24118) 253 : cluster [DBG] pgmap v290: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:11.917 INFO:teuthology.orchestra.run.smithi104.stdout:56 2020-09-08T15:36:11.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:11 smithi104 bash[9203]: cluster 2020-09-08T15:36:11.562335+0000 mgr.x (mgr.24118) 253 : cluster [DBG] pgmap v290: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:12.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:11 smithi166 bash[9446]: cluster 2020-09-08T15:36:11.562335+0000 mgr.x (mgr.24118) 253 : cluster [DBG] pgmap v290: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:12.622 INFO:tasks.cephadm.ceph_manager.ceph:need seq 47 got 56 for osd.4 2020-09-08T15:36:12.623 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.5 2020-09-08T15:36:12.763 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:36:12.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:12 smithi055 bash[9296]: cluster 2020-09-08T15:36:11.836508+0000 osd.5 (osd.5) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:12.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:12 smithi055 bash[9296]: cluster 2020-09-08T15:36:11.836672+0000 osd.5 (osd.5) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:12.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:12 smithi055 bash[9296]: audit 2020-09-08T15:36:11.921225+0000 mon.a (mon.0) 859 : audit [DBG] from='client.? v1:172.21.15.104:0/2368912776' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2020-09-08T15:36:12.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:12 smithi104 bash[9203]: cluster 2020-09-08T15:36:11.836508+0000 osd.5 (osd.5) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:12.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:12 smithi104 bash[9203]: cluster 2020-09-08T15:36:11.836672+0000 osd.5 (osd.5) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:12.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:12 smithi104 bash[9203]: audit 2020-09-08T15:36:11.921225+0000 mon.a (mon.0) 859 : audit [DBG] from='client.? v1:172.21.15.104:0/2368912776' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2020-09-08T15:36:13.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:12 smithi166 bash[9446]: cluster 2020-09-08T15:36:11.836508+0000 osd.5 (osd.5) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:13.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:12 smithi166 bash[9446]: cluster 2020-09-08T15:36:11.836672+0000 osd.5 (osd.5) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:13.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:12 smithi166 bash[9446]: audit 2020-09-08T15:36:11.921225+0000 mon.a (mon.0) 859 : audit [DBG] from='client.? v1:172.21.15.104:0/2368912776' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2020-09-08T15:36:13.937 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:13 smithi104 bash[9203]: cluster 2020-09-08T15:36:12.668093+0000 mon.a (mon.0) 860 : cluster [DBG] osdmap e109: 12 total, 12 up, 12 in 2020-09-08T15:36:13.937 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:13 smithi104 bash[9203]: cluster 2020-09-08T15:36:13.562948+0000 mgr.x (mgr.24118) 254 : cluster [DBG] pgmap v292: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:14.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:13 smithi166 bash[9446]: cluster 2020-09-08T15:36:12.668093+0000 mon.a (mon.0) 860 : cluster [DBG] osdmap e109: 12 total, 12 up, 12 in 2020-09-08T15:36:14.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:13 smithi166 bash[9446]: cluster 2020-09-08T15:36:13.562948+0000 mgr.x (mgr.24118) 254 : cluster [DBG] pgmap v292: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:14.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:13 smithi055 bash[9296]: cluster 2020-09-08T15:36:12.668093+0000 mon.a (mon.0) 860 : cluster [DBG] osdmap e109: 12 total, 12 up, 12 in 2020-09-08T15:36:14.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:13 smithi055 bash[9296]: cluster 2020-09-08T15:36:13.562948+0000 mgr.x (mgr.24118) 254 : cluster [DBG] pgmap v292: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:15.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:15 smithi055 bash[9296]: cluster 2020-09-08T15:36:15.563585+0000 mgr.x (mgr.24118) 255 : cluster [DBG] pgmap v293: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:15.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:15 smithi104 bash[9203]: cluster 2020-09-08T15:36:15.563585+0000 mgr.x (mgr.24118) 255 : cluster [DBG] pgmap v293: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:15.972 INFO:teuthology.orchestra.run.smithi104.stdout:51 2020-09-08T15:36:16.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:15 smithi166 bash[9446]: cluster 2020-09-08T15:36:15.563585+0000 mgr.x (mgr.24118) 255 : cluster [DBG] pgmap v293: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:16.586 INFO:tasks.cephadm.ceph_manager.ceph:need seq 42 got 51 for osd.5 2020-09-08T15:36:16.586 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.6 2020-09-08T15:36:16.723 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:36:16.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:16 smithi104 bash[9203]: audit 2020-09-08T15:36:15.976630+0000 mon.a (mon.0) 861 : audit [DBG] from='client.? v1:172.21.15.104:0/3179446020' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2020-09-08T15:36:17.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:16 smithi166 bash[9446]: audit 2020-09-08T15:36:15.976630+0000 mon.a (mon.0) 861 : audit [DBG] from='client.? v1:172.21.15.104:0/3179446020' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2020-09-08T15:36:17.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:16 smithi055 bash[9296]: audit 2020-09-08T15:36:15.976630+0000 mon.a (mon.0) 861 : audit [DBG] from='client.? v1:172.21.15.104:0/3179446020' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2020-09-08T15:36:17.912 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:17 smithi104 bash[9203]: cluster 2020-09-08T15:36:16.401493+0000 osd.10 (osd.10) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:17.913 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:17 smithi104 bash[9203]: cluster 2020-09-08T15:36:16.401649+0000 osd.10 (osd.10) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:17.913 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:17 smithi104 bash[9203]: cluster 2020-09-08T15:36:16.660658+0000 mon.a (mon.0) 862 : cluster [DBG] osdmap e110: 12 total, 12 up, 12 in 2020-09-08T15:36:17.913 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:17 smithi104 bash[9203]: cluster 2020-09-08T15:36:17.564242+0000 mgr.x (mgr.24118) 256 : cluster [DBG] pgmap v295: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:18.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:17 smithi166 bash[9446]: cluster 2020-09-08T15:36:16.401493+0000 osd.10 (osd.10) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:18.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:17 smithi166 bash[9446]: cluster 2020-09-08T15:36:16.401649+0000 osd.10 (osd.10) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:18.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:17 smithi166 bash[9446]: cluster 2020-09-08T15:36:16.660658+0000 mon.a (mon.0) 862 : cluster [DBG] osdmap e110: 12 total, 12 up, 12 in 2020-09-08T15:36:18.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:17 smithi166 bash[9446]: cluster 2020-09-08T15:36:17.564242+0000 mgr.x (mgr.24118) 256 : cluster [DBG] pgmap v295: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:18.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:17 smithi055 bash[9296]: cluster 2020-09-08T15:36:16.401493+0000 osd.10 (osd.10) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:18.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:17 smithi055 bash[9296]: cluster 2020-09-08T15:36:16.401649+0000 osd.10 (osd.10) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:18.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:17 smithi055 bash[9296]: cluster 2020-09-08T15:36:16.660658+0000 mon.a (mon.0) 862 : cluster [DBG] osdmap e110: 12 total, 12 up, 12 in 2020-09-08T15:36:18.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:17 smithi055 bash[9296]: cluster 2020-09-08T15:36:17.564242+0000 mgr.x (mgr.24118) 256 : cluster [DBG] pgmap v295: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:19.751 INFO:teuthology.orchestra.run.smithi104.stdout:47 2020-09-08T15:36:20.503 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38 got 47 for osd.6 2020-09-08T15:36:20.503 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.7 2020-09-08T15:36:20.620 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:20 smithi166 bash[9446]: cluster 2020-09-08T15:36:19.286850+0000 osd.3 (osd.3) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:20.620 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:20 smithi166 bash[9446]: cluster 2020-09-08T15:36:19.286981+0000 osd.3 (osd.3) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:20.620 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:20 smithi166 bash[9446]: cluster 2020-09-08T15:36:19.346125+0000 mon.a (mon.0) 863 : cluster [DBG] osdmap e111: 12 total, 12 up, 12 in 2020-09-08T15:36:20.621 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:20 smithi166 bash[9446]: cluster 2020-09-08T15:36:19.564833+0000 mgr.x (mgr.24118) 257 : cluster [DBG] pgmap v297: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:20.621 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:20 smithi166 bash[9446]: audit 2020-09-08T15:36:19.755980+0000 mon.a (mon.0) 864 : audit [DBG] from='client.? v1:172.21.15.104:0/170972994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2020-09-08T15:36:20.641 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:36:20.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:20 smithi055 bash[9296]: cluster 2020-09-08T15:36:19.286850+0000 osd.3 (osd.3) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:20.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:20 smithi055 bash[9296]: cluster 2020-09-08T15:36:19.286981+0000 osd.3 (osd.3) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:20.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:20 smithi055 bash[9296]: cluster 2020-09-08T15:36:19.346125+0000 mon.a (mon.0) 863 : cluster [DBG] osdmap e111: 12 total, 12 up, 12 in 2020-09-08T15:36:20.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:20 smithi055 bash[9296]: cluster 2020-09-08T15:36:19.564833+0000 mgr.x (mgr.24118) 257 : cluster [DBG] pgmap v297: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:20.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:20 smithi055 bash[9296]: audit 2020-09-08T15:36:19.755980+0000 mon.a (mon.0) 864 : audit [DBG] from='client.? v1:172.21.15.104:0/170972994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2020-09-08T15:36:20.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:20 smithi104 bash[9203]: cluster 2020-09-08T15:36:19.286850+0000 osd.3 (osd.3) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:20.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:20 smithi104 bash[9203]: cluster 2020-09-08T15:36:19.286981+0000 osd.3 (osd.3) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:20.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:20 smithi104 bash[9203]: cluster 2020-09-08T15:36:19.346125+0000 mon.a (mon.0) 863 : cluster [DBG] osdmap e111: 12 total, 12 up, 12 in 2020-09-08T15:36:20.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:20 smithi104 bash[9203]: cluster 2020-09-08T15:36:19.564833+0000 mgr.x (mgr.24118) 257 : cluster [DBG] pgmap v297: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:20.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:20 smithi104 bash[9203]: audit 2020-09-08T15:36:19.755980+0000 mon.a (mon.0) 864 : audit [DBG] from='client.? v1:172.21.15.104:0/170972994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2020-09-08T15:36:21.859 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:21 smithi104 bash[9203]: cluster 2020-09-08T15:36:21.565448+0000 mgr.x (mgr.24118) 258 : cluster [DBG] pgmap v298: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:21.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:21 smithi055 bash[9296]: cluster 2020-09-08T15:36:21.565448+0000 mgr.x (mgr.24118) 258 : cluster [DBG] pgmap v298: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:22.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:21 smithi166 bash[9446]: cluster 2020-09-08T15:36:21.565448+0000 mgr.x (mgr.24118) 258 : cluster [DBG] pgmap v298: 1 pgs: 1 active+clean; 0 B data, 70 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:23.777 INFO:teuthology.orchestra.run.smithi104.stdout:45 2020-09-08T15:36:23.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:23 smithi055 bash[9296]: cluster 2020-09-08T15:36:23.566116+0000 mgr.x (mgr.24118) 259 : cluster [DBG] pgmap v299: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:23.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:23 smithi104 bash[9203]: cluster 2020-09-08T15:36:23.566116+0000 mgr.x (mgr.24118) 259 : cluster [DBG] pgmap v299: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:24.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:23 smithi166 bash[9446]: cluster 2020-09-08T15:36:23.566116+0000 mgr.x (mgr.24118) 259 : cluster [DBG] pgmap v299: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:24.480 INFO:tasks.cephadm.ceph_manager.ceph:need seq 35 got 45 for osd.7 2020-09-08T15:36:24.481 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.8 2020-09-08T15:36:24.617 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:36:24.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:24 smithi055 bash[9296]: audit 2020-09-08T15:36:23.782435+0000 mon.a (mon.0) 865 : audit [DBG] from='client.? v1:172.21.15.104:0/3484907270' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2020-09-08T15:36:24.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:24 smithi104 bash[9203]: audit 2020-09-08T15:36:23.782435+0000 mon.a (mon.0) 865 : audit [DBG] from='client.? v1:172.21.15.104:0/3484907270' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2020-09-08T15:36:25.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:24 smithi166 bash[9446]: audit 2020-09-08T15:36:23.782435+0000 mon.a (mon.0) 865 : audit [DBG] from='client.? v1:172.21.15.104:0/3484907270' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2020-09-08T15:36:25.899 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:25 smithi104 bash[9203]: cluster 2020-09-08T15:36:24.181854+0000 osd.7 (osd.7) 15 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:36:25.899 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:25 smithi104 bash[9203]: cluster 2020-09-08T15:36:24.184847+0000 osd.7 (osd.7) 16 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:36:25.900 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:25 smithi104 bash[9203]: cluster 2020-09-08T15:36:25.566744+0000 mgr.x (mgr.24118) 260 : cluster [DBG] pgmap v300: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:26.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:25 smithi166 bash[9446]: cluster 2020-09-08T15:36:24.181854+0000 osd.7 (osd.7) 15 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:36:26.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:25 smithi166 bash[9446]: cluster 2020-09-08T15:36:24.184847+0000 osd.7 (osd.7) 16 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:36:26.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:25 smithi166 bash[9446]: cluster 2020-09-08T15:36:25.566744+0000 mgr.x (mgr.24118) 260 : cluster [DBG] pgmap v300: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:26.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:25 smithi055 bash[9296]: cluster 2020-09-08T15:36:24.181854+0000 osd.7 (osd.7) 15 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:36:26.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:25 smithi055 bash[9296]: cluster 2020-09-08T15:36:24.184847+0000 osd.7 (osd.7) 16 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:36:26.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:25 smithi055 bash[9296]: cluster 2020-09-08T15:36:25.566744+0000 mgr.x (mgr.24118) 260 : cluster [DBG] pgmap v300: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:26.958 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:26 smithi104 bash[9203]: cluster 2020-09-08T15:36:25.286091+0000 osd.8 (osd.8) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:26.958 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:26 smithi104 bash[9203]: cluster 2020-09-08T15:36:25.286265+0000 osd.8 (osd.8) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:26.959 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:26 smithi104 bash[9203]: cluster 2020-09-08T15:36:25.674073+0000 mon.a (mon.0) 866 : cluster [DBG] osdmap e112: 12 total, 12 up, 12 in 2020-09-08T15:36:27.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:26 smithi166 bash[9446]: cluster 2020-09-08T15:36:25.286091+0000 osd.8 (osd.8) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:27.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:26 smithi166 bash[9446]: cluster 2020-09-08T15:36:25.286265+0000 osd.8 (osd.8) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:27.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:26 smithi166 bash[9446]: cluster 2020-09-08T15:36:25.674073+0000 mon.a (mon.0) 866 : cluster [DBG] osdmap e112: 12 total, 12 up, 12 in 2020-09-08T15:36:27.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:26 smithi055 bash[9296]: cluster 2020-09-08T15:36:25.286091+0000 osd.8 (osd.8) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:27.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:26 smithi055 bash[9296]: cluster 2020-09-08T15:36:25.286265+0000 osd.8 (osd.8) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:27.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:26 smithi055 bash[9296]: cluster 2020-09-08T15:36:25.674073+0000 mon.a (mon.0) 866 : cluster [DBG] osdmap e112: 12 total, 12 up, 12 in 2020-09-08T15:36:27.863 INFO:teuthology.orchestra.run.smithi104.stdout:40 2020-09-08T15:36:27.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:27 smithi104 bash[9203]: cluster 2020-09-08T15:36:27.567406+0000 mgr.x (mgr.24118) 261 : cluster [DBG] pgmap v302: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:28.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:27 smithi166 bash[9446]: cluster 2020-09-08T15:36:27.567406+0000 mgr.x (mgr.24118) 261 : cluster [DBG] pgmap v302: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:28.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:27 smithi055 bash[9296]: cluster 2020-09-08T15:36:27.567406+0000 mgr.x (mgr.24118) 261 : cluster [DBG] pgmap v302: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:28.516 INFO:tasks.cephadm.ceph_manager.ceph:need seq 30 got 40 for osd.8 2020-09-08T15:36:28.518 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.9 2020-09-08T15:36:28.649 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:36:28.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:28 smithi104 bash[9203]: audit 2020-09-08T15:36:27.867622+0000 mon.a (mon.0) 867 : audit [DBG] from='client.? v1:172.21.15.104:0/965370526' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2020-09-08T15:36:29.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:28 smithi166 bash[9446]: audit 2020-09-08T15:36:27.867622+0000 mon.a (mon.0) 867 : audit [DBG] from='client.? v1:172.21.15.104:0/965370526' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2020-09-08T15:36:29.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:28 smithi055 bash[9296]: audit 2020-09-08T15:36:27.867622+0000 mon.a (mon.0) 867 : audit [DBG] from='client.? v1:172.21.15.104:0/965370526' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2020-09-08T15:36:29.909 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:29 smithi104 bash[9203]: audit 2020-09-08T15:36:29.050454+0000 mon.a (mon.0) 868 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580086\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580289\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580578\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580831\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.581077\", \"created\": \"2020-09-08T15:31:11.717339\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:31:12.581399\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:36:29.049062\"}"}]: dispatch 2020-09-08T15:36:29.910 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:29 smithi104 bash[9203]: audit 2020-09-08T15:36:29.054879+0000 mon.a (mon.0) 869 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580086\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580289\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580578\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580831\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.581077\", \"created\": \"2020-09-08T15:31:11.717339\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:31:12.581399\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:36:29.049062\"}"}]': finished 2020-09-08T15:36:29.910 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:29 smithi104 bash[9203]: audit 2020-09-08T15:36:29.057206+0000 mon.a (mon.0) 870 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:29.910 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:29 smithi104 bash[9203]: audit 2020-09-08T15:36:29.061707+0000 mon.a (mon.0) 871 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:36:29.911 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:29 smithi104 bash[9203]: cluster 2020-09-08T15:36:29.568089+0000 mgr.x (mgr.24118) 262 : cluster [DBG] pgmap v303: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:30.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:29 smithi166 bash[9446]: audit 2020-09-08T15:36:29.050454+0000 mon.a (mon.0) 868 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580086\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580289\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580578\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580831\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.581077\", \"created\": \"2020-09-08T15:31:11.717339\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:31:12.581399\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:36:29.049062\"}"}]: dispatch 2020-09-08T15:36:30.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:29 smithi166 bash[9446]: audit 2020-09-08T15:36:29.054879+0000 mon.a (mon.0) 869 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580086\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580289\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580578\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580831\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.581077\", \"created\": \"2020-09-08T15:31:11.717339\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:31:12.581399\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:36:29.049062\"}"}]': finished 2020-09-08T15:36:30.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:29 smithi166 bash[9446]: audit 2020-09-08T15:36:29.057206+0000 mon.a (mon.0) 870 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:30.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:29 smithi166 bash[9446]: audit 2020-09-08T15:36:29.061707+0000 mon.a (mon.0) 871 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:36:30.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:29 smithi166 bash[9446]: cluster 2020-09-08T15:36:29.568089+0000 mgr.x (mgr.24118) 262 : cluster [DBG] pgmap v303: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:30.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:29 smithi055 bash[9296]: audit 2020-09-08T15:36:29.050454+0000 mon.a (mon.0) 868 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580086\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580289\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580578\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580831\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.581077\", \"created\": \"2020-09-08T15:31:11.717339\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:31:12.581399\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:36:29.049062\"}"}]: dispatch 2020-09-08T15:36:30.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:29 smithi055 bash[9296]: audit 2020-09-08T15:36:29.054879+0000 mon.a (mon.0) 869 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580086\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580289\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580578\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.580831\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:31:12.581077\", \"created\": \"2020-09-08T15:31:11.717339\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:31:12.581399\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:36:29.049062\"}"}]': finished 2020-09-08T15:36:30.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:29 smithi055 bash[9296]: audit 2020-09-08T15:36:29.057206+0000 mon.a (mon.0) 870 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:30.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:29 smithi055 bash[9296]: audit 2020-09-08T15:36:29.061707+0000 mon.a (mon.0) 871 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:36:30.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:29 smithi055 bash[9296]: cluster 2020-09-08T15:36:29.568089+0000 mgr.x (mgr.24118) 262 : cluster [DBG] pgmap v303: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:31.799 INFO:teuthology.orchestra.run.smithi104.stdout:34 2020-09-08T15:36:31.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:31 smithi055 bash[9296]: cluster 2020-09-08T15:36:31.568766+0000 mgr.x (mgr.24118) 263 : cluster [DBG] pgmap v304: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:31.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:31 smithi104 bash[9203]: cluster 2020-09-08T15:36:31.568766+0000 mgr.x (mgr.24118) 263 : cluster [DBG] pgmap v304: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:32.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:31 smithi166 bash[9446]: cluster 2020-09-08T15:36:31.568766+0000 mgr.x (mgr.24118) 263 : cluster [DBG] pgmap v304: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:32.520 INFO:tasks.cephadm.ceph_manager.ceph:need seq 25 got 34 for osd.9 2020-09-08T15:36:32.521 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.10 2020-09-08T15:36:32.654 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:36:32.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:32 smithi055 bash[9296]: audit 2020-09-08T15:36:31.803805+0000 mon.a (mon.0) 872 : audit [DBG] from='client.? v1:172.21.15.104:0/1417448510' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2020-09-08T15:36:32.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:32 smithi104 bash[9203]: audit 2020-09-08T15:36:31.803805+0000 mon.a (mon.0) 872 : audit [DBG] from='client.? v1:172.21.15.104:0/1417448510' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2020-09-08T15:36:33.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:32 smithi166 bash[9446]: audit 2020-09-08T15:36:31.803805+0000 mon.a (mon.0) 872 : audit [DBG] from='client.? v1:172.21.15.104:0/1417448510' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2020-09-08T15:36:33.910 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:33 smithi104 bash[9203]: cluster 2020-09-08T15:36:32.693215+0000 osd.1 (osd.1) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:33.911 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:33 smithi104 bash[9203]: cluster 2020-09-08T15:36:32.693318+0000 osd.1 (osd.1) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:33.911 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:33 smithi104 bash[9203]: cluster 2020-09-08T15:36:33.569421+0000 mgr.x (mgr.24118) 264 : cluster [DBG] pgmap v305: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:34.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:33 smithi166 bash[9446]: cluster 2020-09-08T15:36:32.693215+0000 osd.1 (osd.1) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:34.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:33 smithi166 bash[9446]: cluster 2020-09-08T15:36:32.693318+0000 osd.1 (osd.1) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:34.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:33 smithi166 bash[9446]: cluster 2020-09-08T15:36:33.569421+0000 mgr.x (mgr.24118) 264 : cluster [DBG] pgmap v305: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:34.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:33 smithi055 bash[9296]: cluster 2020-09-08T15:36:32.693215+0000 osd.1 (osd.1) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:34.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:33 smithi055 bash[9296]: cluster 2020-09-08T15:36:32.693318+0000 osd.1 (osd.1) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:34.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:33 smithi055 bash[9296]: cluster 2020-09-08T15:36:33.569421+0000 mgr.x (mgr.24118) 264 : cluster [DBG] pgmap v305: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:34.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:34 smithi104 bash[9203]: cluster 2020-09-08T15:36:33.673444+0000 mon.a (mon.0) 873 : cluster [DBG] osdmap e113: 12 total, 12 up, 12 in 2020-09-08T15:36:35.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:34 smithi166 bash[9446]: cluster 2020-09-08T15:36:33.673444+0000 mon.a (mon.0) 873 : cluster [DBG] osdmap e113: 12 total, 12 up, 12 in 2020-09-08T15:36:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:34 smithi055 bash[9296]: cluster 2020-09-08T15:36:33.673444+0000 mon.a (mon.0) 873 : cluster [DBG] osdmap e113: 12 total, 12 up, 12 in 2020-09-08T15:36:35.873 INFO:teuthology.orchestra.run.smithi104.stdout:32 2020-09-08T15:36:35.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:35 smithi104 bash[9203]: cluster 2020-09-08T15:36:35.570104+0000 mgr.x (mgr.24118) 265 : cluster [DBG] pgmap v307: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:36.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:35 smithi166 bash[9446]: cluster 2020-09-08T15:36:35.570104+0000 mgr.x (mgr.24118) 265 : cluster [DBG] pgmap v307: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:36.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:35 smithi055 bash[9296]: cluster 2020-09-08T15:36:35.570104+0000 mgr.x (mgr.24118) 265 : cluster [DBG] pgmap v307: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:36.609 INFO:tasks.cephadm.ceph_manager.ceph:need seq 22 got 32 for osd.10 2020-09-08T15:36:36.610 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.11 2020-09-08T15:36:36.745 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:36:36.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:36 smithi104 bash[9203]: audit 2020-09-08T15:36:35.878137+0000 mon.a (mon.0) 874 : audit [DBG] from='client.? v1:172.21.15.104:0/3994342158' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2020-09-08T15:36:37.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:36 smithi166 bash[9446]: audit 2020-09-08T15:36:35.878137+0000 mon.a (mon.0) 874 : audit [DBG] from='client.? v1:172.21.15.104:0/3994342158' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2020-09-08T15:36:37.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:36 smithi055 bash[9296]: audit 2020-09-08T15:36:35.878137+0000 mon.a (mon.0) 874 : audit [DBG] from='client.? v1:172.21.15.104:0/3994342158' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2020-09-08T15:36:38.064 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:37 smithi104 bash[9203]: cluster 2020-09-08T15:36:37.570734+0000 mgr.x (mgr.24118) 266 : cluster [DBG] pgmap v308: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:38.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:37 smithi166 bash[9446]: cluster 2020-09-08T15:36:37.570734+0000 mgr.x (mgr.24118) 266 : cluster [DBG] pgmap v308: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:38.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:37 smithi055 bash[9296]: cluster 2020-09-08T15:36:37.570734+0000 mgr.x (mgr.24118) 266 : cluster [DBG] pgmap v308: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:39.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:38 smithi166 bash[9446]: cluster 2020-09-08T15:36:37.725462+0000 osd.6 (osd.6) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:39.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:38 smithi166 bash[9446]: cluster 2020-09-08T15:36:37.725612+0000 osd.6 (osd.6) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:39.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:38 smithi055 bash[9296]: cluster 2020-09-08T15:36:37.725462+0000 osd.6 (osd.6) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:39.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:38 smithi055 bash[9296]: cluster 2020-09-08T15:36:37.725612+0000 osd.6 (osd.6) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:39.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:38 smithi104 bash[9203]: cluster 2020-09-08T15:36:37.725462+0000 osd.6 (osd.6) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:39.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:38 smithi104 bash[9203]: cluster 2020-09-08T15:36:37.725612+0000 osd.6 (osd.6) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:39.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:39 smithi104 bash[9203]: cluster 2020-09-08T15:36:38.701079+0000 mon.a (mon.0) 875 : cluster [DBG] osdmap e114: 12 total, 12 up, 12 in 2020-09-08T15:36:39.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:39 smithi104 bash[9203]: cluster 2020-09-08T15:36:39.571366+0000 mgr.x (mgr.24118) 267 : cluster [DBG] pgmap v310: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:40.045 INFO:teuthology.orchestra.run.smithi104.stdout:26 2020-09-08T15:36:40.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:39 smithi166 bash[9446]: cluster 2020-09-08T15:36:38.701079+0000 mon.a (mon.0) 875 : cluster [DBG] osdmap e114: 12 total, 12 up, 12 in 2020-09-08T15:36:40.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:39 smithi166 bash[9446]: cluster 2020-09-08T15:36:39.571366+0000 mgr.x (mgr.24118) 267 : cluster [DBG] pgmap v310: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:40.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:39 smithi055 bash[9296]: cluster 2020-09-08T15:36:38.701079+0000 mon.a (mon.0) 875 : cluster [DBG] osdmap e114: 12 total, 12 up, 12 in 2020-09-08T15:36:40.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:39 smithi055 bash[9296]: cluster 2020-09-08T15:36:39.571366+0000 mgr.x (mgr.24118) 267 : cluster [DBG] pgmap v310: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:40.711 INFO:tasks.cephadm.ceph_manager.ceph:need seq 17 got 26 for osd.11 2020-09-08T15:36:40.712 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2020-09-08T15:36:40.712 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph pg dump --format=json 2020-09-08T15:36:40.839 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:36:41.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:40 smithi166 bash[9446]: audit 2020-09-08T15:36:40.049627+0000 mon.a (mon.0) 876 : audit [DBG] from='client.? v1:172.21.15.104:0/1409565815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2020-09-08T15:36:41.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:40 smithi055 bash[9296]: audit 2020-09-08T15:36:40.049627+0000 mon.a (mon.0) 876 : audit [DBG] from='client.? v1:172.21.15.104:0/1409565815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2020-09-08T15:36:41.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:40 smithi104 bash[9203]: audit 2020-09-08T15:36:40.049627+0000 mon.a (mon.0) 876 : audit [DBG] from='client.? v1:172.21.15.104:0/1409565815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2020-09-08T15:36:42.075 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:41 smithi104 bash[9203]: cluster 2020-09-08T15:36:40.186102+0000 osd.11 (osd.11) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:42.076 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:41 smithi104 bash[9203]: cluster 2020-09-08T15:36:40.186280+0000 osd.11 (osd.11) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:42.076 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:41 smithi104 bash[9203]: cluster 2020-09-08T15:36:40.724543+0000 mon.a (mon.0) 877 : cluster [DBG] osdmap e115: 12 total, 12 up, 12 in 2020-09-08T15:36:42.077 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:41 smithi104 bash[9203]: cluster 2020-09-08T15:36:41.571996+0000 mgr.x (mgr.24118) 268 : cluster [DBG] pgmap v312: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:42.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:41 smithi166 bash[9446]: cluster 2020-09-08T15:36:40.186102+0000 osd.11 (osd.11) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:42.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:41 smithi166 bash[9446]: cluster 2020-09-08T15:36:40.186280+0000 osd.11 (osd.11) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:42.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:41 smithi166 bash[9446]: cluster 2020-09-08T15:36:40.724543+0000 mon.a (mon.0) 877 : cluster [DBG] osdmap e115: 12 total, 12 up, 12 in 2020-09-08T15:36:42.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:41 smithi166 bash[9446]: cluster 2020-09-08T15:36:41.571996+0000 mgr.x (mgr.24118) 268 : cluster [DBG] pgmap v312: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:42.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:41 smithi055 bash[9296]: cluster 2020-09-08T15:36:40.186102+0000 osd.11 (osd.11) 5 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:42.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:41 smithi055 bash[9296]: cluster 2020-09-08T15:36:40.186280+0000 osd.11 (osd.11) 6 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:42.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:41 smithi055 bash[9296]: cluster 2020-09-08T15:36:40.724543+0000 mon.a (mon.0) 877 : cluster [DBG] osdmap e115: 12 total, 12 up, 12 in 2020-09-08T15:36:42.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:41 smithi055 bash[9296]: cluster 2020-09-08T15:36:41.571996+0000 mgr.x (mgr.24118) 268 : cluster [DBG] pgmap v312: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:43.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:43 smithi055 bash[9296]: cluster 2020-09-08T15:36:43.572671+0000 mgr.x (mgr.24118) 269 : cluster [DBG] pgmap v313: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:43.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:43 smithi104 bash[9203]: cluster 2020-09-08T15:36:43.572671+0000 mgr.x (mgr.24118) 269 : cluster [DBG] pgmap v313: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:43.946 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T15:36:43.947 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2020-09-08T15:36:44.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:43 smithi166 bash[9446]: cluster 2020-09-08T15:36:43.572671+0000 mgr.x (mgr.24118) 269 : cluster [DBG] pgmap v313: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:44.634 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2020-09-08T15:36:43.572315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":12,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":1124941824,"kb_used":73368,"kb_used_data":8232,"kb_used_omap":0,"kb_used_meta":64320,"kb_avail":1124868456,"statfs":{"total":1151940427776,"available":1151865298944,"internally_reserved":0,"allocated":8429568,"data_stored":4694584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65863680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002524"},"pg_stats":[{"pgid":"1.0","version":"99'6","reported_seq":"122","reported_epoch":"114","state":"active+clean","last_fresh":"2020-09-08T15:36:38.746975+0000","last_change":"2020-09-08T15:36:24.184987+0000","last_active":"2020-09-08T15:36:38.746975+0000","last_peered":"2020-09-08T15:36:38.746975+0000","last_clean":"2020-09-08T15:36:38.746975+0000","last_became_active":"2020-09-08T15:34:43.031697+0000","last_became_peered":"2020-09-08T15:34:43.031697+0000","last_unstale":"2020-09-08T15:36:38.746975+0000","last_undegraded":"2020-09-08T15:36:38.746975+0000","last_fullsized":"2020-09-08T15:36:38.746975+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":2,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'6","last_scrub_stamp":"2020-09-08T15:36:24.184849+0000","last_deep_scrub":"99'6","last_deep_scrub_stamp":"2020-09-08T15:35:23.132465+0000","last_clean_scrub_stamp":"2020-09-08T15:36:24.184849+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":2,"acting":2,"num_store_stats":7}],"osd_stats":[{"osd":11,"up_from":0,"seq":28,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5740,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":93739412,"statfs":{"total":95995035648,"available":95989157888,"internally_reserved":0,"allocated":757760,"data_stored":417277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"min":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"max":{"1min":2.4849999999999999,"5min":2.4849999999999999,"15min":2.4849999999999999},"last":1.3700000000000001},{"interface":"front","average":{"1min":3.3119999999999998,"5min":3.3119999999999998,"15min":3.3119999999999998},"min":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"max":{"1min":4.5970000000000004,"5min":4.5970000000000004,"15min":4.5970000000000004},"last":2.573}]},{"osd":1,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"min":{"1min":1.395,"5min":1.395,"15min":1.395},"max":{"1min":3.2250000000000001,"5min":3.2250000000000001,"15min":3.2250000000000001},"last":2.4780000000000002},{"interface":"front","average":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"min":{"1min":1.591,"5min":1.591,"15min":1.591},"max":{"1min":3.8580000000000001,"5min":3.8580000000000001,"15min":3.8580000000000001},"last":3.0339999999999998}]},{"osd":2,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"min":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"max":{"1min":3.7949999999999999,"5min":3.7949999999999999,"15min":3.7949999999999999},"last":3.593},{"interface":"front","average":{"1min":2.8679999999999999,"5min":2.8679999999999999,"15min":2.8679999999999999},"min":{"1min":1.893,"5min":1.893,"15min":1.893},"max":{"1min":4.2939999999999996,"5min":4.2939999999999996,"15min":4.2939999999999996},"last":1.702}]},{"osd":3,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"min":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"max":{"1min":3.5049999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":2.8740000000000001},{"interface":"front","average":{"1min":3.137,"5min":3.137,"15min":3.137},"min":{"1min":1.498,"5min":1.498,"15min":1.498},"max":{"1min":4.8949999999999996,"5min":4.8949999999999996,"15min":4.8949999999999996},"last":2.3490000000000002}]},{"osd":4,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":3.6099999999999999,"5min":3.6099999999999999,"15min":3.6099999999999999},"min":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"max":{"1min":5.5599999999999996,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":1.9610000000000001},{"interface":"front","average":{"1min":2.806,"5min":2.806,"15min":2.806},"min":{"1min":1.673,"5min":1.673,"15min":1.673},"max":{"1min":4.0839999999999996,"5min":4.0839999999999996,"15min":4.0839999999999996},"last":3.1600000000000001}]},{"osd":5,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":3.4359999999999999,"5min":3.4359999999999999,"15min":3.4359999999999999},"min":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"max":{"1min":6.1319999999999997,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":3.3220000000000001},{"interface":"front","average":{"1min":4,"5min":4,"15min":4},"min":{"1min":2.5059999999999998,"5min":2.5059999999999998,"15min":2.5059999999999998},"max":{"1min":6.3630000000000004,"5min":6.3630000000000004,"15min":6.3630000000000004},"last":2.8210000000000002}]},{"osd":6,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":3.4359999999999999,"5min":3.4359999999999999,"15min":3.4359999999999999},"min":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.3730000000000002},"max":{"1min":5.5439999999999996,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":3.3820000000000001},{"interface":"front","average":{"1min":3.621,"5min":3.621,"15min":3.621},"min":{"1min":2.786,"5min":2.786,"15min":2.786},"max":{"1min":4.6879999999999997,"5min":4.6879999999999997,"15min":4.6879999999999997},"last":3.129}]},{"osd":7,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":3.7599999999999998,"5min":3.7599999999999998,"15min":3.7599999999999998},"min":{"1min":2.8929999999999998,"5min":2.8929999999999998,"15min":2.8929999999999998},"max":{"1min":6.3220000000000001,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":2.673},{"interface":"front","average":{"1min":3.891,"5min":3.891,"15min":3.891},"min":{"1min":3.0089999999999999,"5min":3.0089999999999999,"15min":3.0089999999999999},"max":{"1min":5.4539999999999997,"5min":5.4539999999999997,"15min":5.4539999999999997},"last":3.5369999999999999}]},{"osd":8,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":4.1379999999999999,"5min":4.1379999999999999,"15min":4.1379999999999999},"min":{"1min":2.923,"5min":2.923,"15min":2.923},"max":{"1min":5.944,"5min":5.944,"15min":5.944},"last":3.2360000000000002},{"interface":"front","average":{"1min":3.907,"5min":3.907,"15min":3.907},"min":{"1min":2.1469999999999998,"5min":2.1469999999999998,"15min":2.1469999999999998},"max":{"1min":6.2400000000000002,"5min":6.2400000000000002,"15min":6.2400000000000002},"last":2.403}]},{"osd":10,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":4.4980000000000002,"5min":4.4980000000000002,"15min":4.4980000000000002},"min":{"1min":3.0459999999999998,"5min":3.0459999999999998,"15min":3.0459999999999998},"max":{"1min":6.673,"5min":6.673,"15min":6.673},"last":3.472},{"interface":"front","average":{"1min":4.2910000000000004,"5min":4.2910000000000004,"15min":4.2910000000000004},"min":{"1min":2.7040000000000002,"5min":2.7040000000000002,"15min":2.7040000000000002},"max":{"1min":6.6859999999999999,"5min":6.6859999999999999,"15min":6.6859999999999999},"last":2.9430000000000001}]}]},{"osd":10,"up_from":0,"seq":34,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5816,"kb_used_data":632,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739336,"statfs":{"total":95995035648,"available":95989080064,"internally_reserved":0,"allocated":647168,"data_stored":365127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7330000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":1.921,"5min":2.3119999999999998,"15min":2.3769999999999998},"min":{"1min":1.5109999999999999,"5min":1.419,"15min":1.419},"max":{"1min":2.4969999999999999,"5min":4.3380000000000001,"15min":4.3380000000000001},"last":1.734},{"interface":"front","average":{"1min":2.6440000000000001,"5min":2.5750000000000002,"15min":2.5630000000000002},"min":{"1min":1.6739999999999999,"5min":1.4750000000000001,"15min":1.4750000000000001},"max":{"1min":3.7759999999999998,"5min":5.1029999999999998,"15min":5.1029999999999998},"last":1.498}]},{"osd":2,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":2.3109999999999999,"5min":2.2599999999999998,"15min":2.2519999999999998},"min":{"1min":1.6519999999999999,"5min":1.508,"15min":1.508},"max":{"1min":3.0939999999999999,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":2.2410000000000001},{"interface":"front","average":{"1min":2.1970000000000001,"5min":2.6960000000000002,"15min":2.7789999999999999},"min":{"1min":1.492,"5min":1.4359999999999999,"15min":1.4359999999999999},"max":{"1min":3.4260000000000002,"5min":4.3769999999999998,"15min":4.3769999999999998},"last":2.1139999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":1.9159999999999999,"5min":2.2090000000000001,"15min":2.258},"min":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"max":{"1min":2.6469999999999998,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":1.5880000000000001},{"interface":"front","average":{"1min":2.7200000000000002,"5min":3.016,"15min":3.0649999999999999},"min":{"1min":1.609,"5min":1.609,"15min":1.609},"max":{"1min":3.605,"5min":4.2140000000000004,"15min":4.2140000000000004},"last":2.6800000000000002}]},{"osd":4,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.1469999999999998,"5min":3.4039999999999999,"15min":3.4470000000000001},"min":{"1min":1.8,"5min":1.8,"15min":1.8},"max":{"1min":4.0720000000000001,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":2.8500000000000001},{"interface":"front","average":{"1min":2.9780000000000002,"5min":3.0649999999999999,"15min":3.0790000000000002},"min":{"1min":1.958,"5min":1.958,"15min":1.958},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":2.3849999999999998}]},{"osd":5,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.218,"5min":3.5510000000000002,"15min":3.6070000000000002},"min":{"1min":2.653,"5min":2.427,"15min":2.427},"max":{"1min":4.1059999999999999,"5min":4.7160000000000002,"15min":4.7160000000000002},"last":3.032},{"interface":"front","average":{"1min":3.6600000000000001,"5min":3.8199999999999998,"15min":3.847},"min":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"max":{"1min":4.7089999999999996,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":2.5070000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.3300000000000001,"5min":3.5339999999999998,"15min":3.5680000000000001},"min":{"1min":2.3359999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"max":{"1min":4.7949999999999999,"5min":4.7949999999999999,"15min":4.7949999999999999},"last":3.1280000000000001},{"interface":"front","average":{"1min":3.6800000000000002,"5min":3.8620000000000001,"15min":3.8919999999999999},"min":{"1min":3.0270000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"max":{"1min":4.4939999999999998,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":3.423}]},{"osd":7,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.7400000000000002,"5min":3.907,"15min":3.9350000000000001},"min":{"1min":2.3410000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"max":{"1min":4.7220000000000004,"5min":7.0490000000000004,"15min":7.0490000000000004},"last":2.7549999999999999},{"interface":"front","average":{"1min":3.9009999999999998,"5min":4.077,"15min":4.1059999999999999},"min":{"1min":3.1789999999999998,"5min":2.806,"15min":2.806},"max":{"1min":4.8929999999999998,"5min":5.0899999999999999,"15min":5.0899999999999999},"last":3.7719999999999998}]},{"osd":8,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":4.0149999999999997,"5min":4.1459999999999999,"15min":4.1680000000000001},"min":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"max":{"1min":5.3390000000000004,"5min":5.7949999999999999,"15min":5.7949999999999999},"last":3.2069999999999999},{"interface":"front","average":{"1min":4.0670000000000002,"5min":4.0309999999999997,"15min":4.0250000000000004},"min":{"1min":2.6920000000000002,"5min":1.901,"15min":1.901},"max":{"1min":5.1200000000000001,"5min":5.6509999999999998,"15min":5.6509999999999998},"last":3.5}]},{"osd":9,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":4.2160000000000002,"5min":4.3780000000000001,"15min":4.4050000000000002},"min":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"max":{"1min":5.4020000000000001,"5min":5.8369999999999997,"15min":5.8369999999999997},"last":3.5899999999999999},{"interface":"front","average":{"1min":4.2629999999999999,"5min":4.2149999999999999,"15min":4.2069999999999999},"min":{"1min":2.5590000000000002,"5min":2.4969999999999999,"15min":2.4969999999999999},"max":{"1min":5.3129999999999997,"5min":5.3129999999999997,"15min":5.3129999999999997},"last":4.3719999999999999}]},{"osd":11,"last update":"Tue Sep 8 15:35:43 2020","interfaces":[{"interface":"back","average":{"1min":4.3890000000000002,"5min":4.3890000000000002,"15min":4.3890000000000002},"min":{"1min":1.831,"5min":1.831,"15min":1.831},"max":{"1min":5.6980000000000004,"5min":5.6980000000000004,"15min":5.6980000000000004},"last":4.0670000000000002},{"interface":"front","average":{"1min":4.423,"5min":4.423,"15min":4.423},"min":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"max":{"1min":5.4740000000000002,"5min":5.4740000000000002,"15min":5.4740000000000002},"last":3.8860000000000001}]}]},{"osd":9,"up_from":0,"seq":37,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5944,"kb_used_data":632,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739208,"statfs":{"total":95995035648,"available":95988948992,"internally_reserved":0,"allocated":647168,"data_stored":365127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":2.129,"5min":1.9119999999999999,"15min":1.8759999999999999},"min":{"1min":1.4390000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"max":{"1min":3.1259999999999999,"5min":3.149,"15min":3.149},"last":2.6600000000000001},{"interface":"front","average":{"1min":1.9490000000000001,"5min":1.9119999999999999,"15min":1.9059999999999999},"min":{"1min":1.3580000000000001,"5min":1.2549999999999999,"15min":1.2549999999999999},"max":{"1min":4.149,"5min":4.149,"15min":4.149},"last":2.5579999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":2.173,"5min":1.9930000000000001,"15min":1.9630000000000001},"min":{"1min":1.3120000000000001,"5min":1.3009999999999999,"15min":1.3009999999999999},"max":{"1min":3.6269999999999998,"5min":3.6269999999999998,"15min":3.6269999999999998},"last":2.4510000000000001},{"interface":"front","average":{"1min":2.4889999999999999,"5min":2.3250000000000002,"15min":2.298},"min":{"1min":1.474,"5min":1.266,"15min":1.266},"max":{"1min":3.9950000000000001,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":3.3599999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":2.7210000000000001,"5min":2.5219999999999998,"15min":2.4889999999999999},"min":{"1min":1.4359999999999999,"5min":1.327,"15min":1.327},"max":{"1min":4.1020000000000003,"5min":4.508,"15min":4.508},"last":4.1710000000000003},{"interface":"front","average":{"1min":2.5489999999999999,"5min":2.3330000000000002,"15min":2.2970000000000002},"min":{"1min":1.4259999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"max":{"1min":3.6800000000000002,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":3.5379999999999998}]},{"osd":3,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":2.395,"5min":2.351,"15min":2.343},"min":{"1min":1.3420000000000001,"5min":1.198,"15min":1.198},"max":{"1min":4.3760000000000003,"5min":4.3760000000000003,"15min":4.3760000000000003},"last":3.8359999999999999},{"interface":"front","average":{"1min":2.629,"5min":2.641,"15min":2.6429999999999998},"min":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"max":{"1min":4.4550000000000001,"5min":4.4550000000000001,"15min":4.4550000000000001},"last":4.4980000000000002}]},{"osd":4,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":3.0470000000000002,"5min":2.7570000000000001,"15min":2.7090000000000001},"min":{"1min":1.905,"5min":1.1819999999999999,"15min":1.1819999999999999},"max":{"1min":5.0279999999999996,"5min":5.0279999999999996,"15min":5.0279999999999996},"last":4.6109999999999998},{"interface":"front","average":{"1min":3.5390000000000001,"5min":3.1040000000000001,"15min":3.032},"min":{"1min":2.5750000000000002,"5min":1.883,"15min":1.883},"max":{"1min":4.6280000000000001,"5min":4.6280000000000001,"15min":4.6280000000000001},"last":4.306}]},{"osd":5,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":3.452,"5min":3.2149999999999999,"15min":3.1749999999999998},"min":{"1min":2.6259999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"max":{"1min":5.0519999999999996,"5min":6.3840000000000003,"15min":6.3840000000000003},"last":4.9349999999999996},{"interface":"front","average":{"1min":3.456,"5min":3.2240000000000002,"15min":3.1859999999999999},"min":{"1min":1.907,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":4.875,"5min":4.875,"15min":4.875},"last":4.6600000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":3.7589999999999999,"5min":3.3109999999999999,"15min":3.2370000000000001},"min":{"1min":2.3820000000000001,"5min":1.4179999999999999,"15min":1.4179999999999999},"max":{"1min":5.3659999999999997,"5min":5.3659999999999997,"15min":5.3659999999999997},"last":4.7309999999999999},{"interface":"front","average":{"1min":3.621,"5min":3.3090000000000002,"15min":3.2570000000000001},"min":{"1min":2.1720000000000002,"5min":1.8959999999999999,"15min":1.8959999999999999},"max":{"1min":5.7130000000000001,"5min":5.7130000000000001,"15min":5.7130000000000001},"last":4.0289999999999999}]},{"osd":7,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":3.9399999999999999,"5min":3.5,"15min":3.427},"min":{"1min":1.968,"5min":1.7949999999999999,"15min":1.7949999999999999},"max":{"1min":5.1459999999999999,"5min":5.1459999999999999,"15min":5.1459999999999999},"last":5.367},{"interface":"front","average":{"1min":4.0599999999999996,"5min":3.7000000000000002,"15min":3.6400000000000001},"min":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"max":{"1min":5.7629999999999999,"5min":5.7629999999999999,"15min":5.7629999999999999},"last":4.9029999999999996}]},{"osd":8,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":3.7400000000000002,"5min":3.278,"15min":3.2010000000000001},"min":{"1min":2.2879999999999998,"5min":1.3180000000000001,"15min":1.3180000000000001},"max":{"1min":4.9119999999999999,"5min":5.3049999999999997,"15min":5.3049999999999997},"last":4.8319999999999999},{"interface":"front","average":{"1min":4.0499999999999998,"5min":3.3969999999999998,"15min":3.2879999999999998},"min":{"1min":2.335,"5min":1.444,"15min":1.444},"max":{"1min":5.4199999999999999,"5min":6.0140000000000002,"15min":6.0140000000000002},"last":5.0919999999999996}]},{"osd":10,"last update":"Tue Sep 8 15:36:18 2020","interfaces":[{"interface":"back","average":{"1min":3.9500000000000002,"5min":3.9350000000000001,"15min":3.9329999999999998},"min":{"1min":3.3839999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"max":{"1min":4.8780000000000001,"5min":5.5629999999999997,"15min":5.5629999999999997},"last":5.1630000000000003},{"interface":"front","average":{"1min":3.9649999999999999,"5min":3.8239999999999998,"15min":3.7999999999999998},"min":{"1min":3.3130000000000002,"5min":2.056,"15min":2.056},"max":{"1min":4.6379999999999999,"5min":5.492,"15min":5.492},"last":4.3810000000000002}]}]},{"osd":8,"up_from":0,"seq":43,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5996,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":93739156,"statfs":{"total":95995035648,"available":95988895744,"internally_reserved":0,"allocated":757760,"data_stored":417277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.2749999999999999,"5min":2.202,"15min":2.1920000000000002},"min":{"1min":1.3799999999999999,"5min":1.266,"15min":1.266},"max":{"1min":3.2200000000000002,"5min":3.5830000000000002,"15min":3.5830000000000002},"last":1.482},{"interface":"front","average":{"1min":2.5459999999999998,"5min":2.5070000000000001,"15min":2.4300000000000002},"min":{"1min":1.7669999999999999,"5min":1.7250000000000001,"15min":1.7250000000000001},"max":{"1min":3.444,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":1.9990000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.492,"5min":2.0659999999999998,"15min":1.8839999999999999},"min":{"1min":1.4690000000000001,"5min":1.137,"15min":1.137},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":2.1240000000000001},{"interface":"front","average":{"1min":2.7200000000000002,"5min":2.427,"15min":2.347},"min":{"1min":1.351,"5min":1.046,"15min":1.046},"max":{"1min":4.444,"5min":4.444,"15min":4.444},"last":1.429}]},{"osd":2,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.2330000000000001,"5min":1.9410000000000001,"15min":1.845},"min":{"1min":1.4379999999999999,"5min":1.27,"15min":1.27},"max":{"1min":3.9129999999999998,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":3.3969999999999998},{"interface":"front","average":{"1min":2.4889999999999999,"5min":2.504,"15min":2.46},"min":{"1min":1.208,"5min":1.208,"15min":1.208},"max":{"1min":3.7509999999999999,"5min":3.895,"15min":3.895},"last":2.7109999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.9750000000000001,"5min":2.7869999999999999,"15min":2.7370000000000001},"min":{"1min":1.587,"5min":1.2789999999999999,"15min":1.2789999999999999},"max":{"1min":4.3929999999999998,"5min":4.3929999999999998,"15min":4.3929999999999998},"last":2.6259999999999999},{"interface":"front","average":{"1min":2.4649999999999999,"5min":2.4260000000000002,"15min":2.3820000000000001},"min":{"1min":1.131,"5min":1.131,"15min":1.131},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.875}]},{"osd":4,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":3.149,"5min":2.9870000000000001,"15min":2.875},"min":{"1min":2.3029999999999999,"5min":1.4730000000000001,"15min":1.4730000000000001},"max":{"1min":4.4669999999999996,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":3.54},{"interface":"front","average":{"1min":2.8820000000000001,"5min":2.6869999999999998,"15min":2.5579999999999998},"min":{"1min":1.734,"5min":1.2549999999999999,"15min":1.2549999999999999},"max":{"1min":4.2649999999999997,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":2.4510000000000001}]},{"osd":5,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":3.5529999999999999,"5min":3.294,"15min":3.1619999999999999},"min":{"1min":1.944,"5min":1.74,"15min":1.74},"max":{"1min":5.6109999999999998,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":3.0169999999999999},{"interface":"front","average":{"1min":3.7360000000000002,"5min":3.419,"15min":3.3719999999999999},"min":{"1min":2.2080000000000002,"5min":1.762,"15min":1.762},"max":{"1min":6.0549999999999997,"5min":6.0549999999999997,"15min":6.0549999999999997},"last":3.089}]},{"osd":6,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":3.8090000000000002,"5min":3.4199999999999999,"15min":3.2639999999999998},"min":{"1min":2.4980000000000002,"5min":1.9299999999999999,"15min":1.9299999999999999},"max":{"1min":5.1929999999999996,"5min":5.6180000000000003,"15min":5.6180000000000003},"last":3.8919999999999999},{"interface":"front","average":{"1min":3.2839999999999998,"5min":3.0499999999999998,"15min":2.9220000000000002},"min":{"1min":2.0019999999999998,"5min":1.669,"15min":1.669},"max":{"1min":4.7279999999999998,"5min":4.7279999999999998,"15min":4.7279999999999998},"last":2.839}]},{"osd":7,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":4.0419999999999998,"5min":3.7269999999999999,"15min":3.6240000000000001},"min":{"1min":2.3849999999999998,"5min":2.0630000000000002,"15min":2.0630000000000002},"max":{"1min":5.3680000000000003,"5min":5.3680000000000003,"15min":5.3680000000000003},"last":4.258},{"interface":"front","average":{"1min":3.6429999999999998,"5min":3.3679999999999999,"15min":3.2440000000000002},"min":{"1min":2.7269999999999999,"5min":2.0830000000000002,"15min":2.0830000000000002},"max":{"1min":5.1029999999999998,"5min":5.141,"15min":5.141},"last":3.2149999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:35:56 2020","interfaces":[{"interface":"back","average":{"1min":3.9590000000000001,"5min":3.4990000000000001,"15min":3.4220000000000002},"min":{"1min":2.4180000000000001,"5min":1.157,"15min":1.157},"max":{"1min":5.1840000000000002,"5min":5.1840000000000002,"15min":5.1840000000000002},"last":3.7789999999999999},{"interface":"front","average":{"1min":4.1639999999999997,"5min":3.4660000000000002,"15min":3.3500000000000001},"min":{"1min":2.0219999999999998,"5min":1.1919999999999999,"15min":1.1919999999999999},"max":{"1min":5.3959999999999999,"5min":5.3959999999999999,"15min":5.3959999999999999},"last":4.4939999999999998}]},{"osd":10,"last update":"Tue Sep 8 15:36:27 2020","interfaces":[{"interface":"back","average":{"1min":3.7200000000000002,"5min":3.7320000000000002,"15min":3.734},"min":{"1min":2.0510000000000002,"5min":1.3740000000000001,"15min":1.3740000000000001},"max":{"1min":5.5190000000000001,"5min":5.5190000000000001,"15min":5.5190000000000001},"last":3.4980000000000002},{"interface":"front","average":{"1min":4.1180000000000003,"5min":4.0830000000000002,"15min":4.077},"min":{"1min":2.7669999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"max":{"1min":5.3049999999999997,"5min":5.3049999999999997,"15min":5.3049999999999997},"last":4.1269999999999998}]}]},{"osd":7,"up_from":0,"seq":49,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6072,"kb_used_data":632,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93739080,"statfs":{"total":95995035648,"available":95988817920,"internally_reserved":0,"allocated":647168,"data_stored":365147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:02 2020","interfaces":[{"interface":"back","average":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"min":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"max":{"1min":4.0190000000000001,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":2.2189999999999999},{"interface":"front","average":{"1min":2.6259999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"min":{"1min":1.542,"5min":1.542,"15min":1.542},"max":{"1min":4.5229999999999997,"5min":4.5229999999999997,"15min":4.5229999999999997},"last":2.3079999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.661,"5min":2.468,"15min":2.355},"min":{"1min":2.1309999999999998,"5min":1.498,"15min":1.498},"max":{"1min":3.3940000000000001,"5min":3.8780000000000001,"15min":3.8780000000000001},"last":2.0110000000000001},{"interface":"front","average":{"1min":2.1699999999999999,"5min":2.359,"15min":2.3599999999999999},"min":{"1min":1.7749999999999999,"5min":1.631,"15min":1.631},"max":{"1min":2.6800000000000002,"5min":3.633,"15min":3.633},"last":2.0910000000000002}]},{"osd":2,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.4049999999999998,"5min":2.528,"15min":2.4399999999999999},"min":{"1min":1.899,"5min":1.504,"15min":1.504},"max":{"1min":3.2240000000000002,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":2.1840000000000002},{"interface":"front","average":{"1min":2.468,"5min":2.879,"15min":2.8370000000000002},"min":{"1min":1.982,"5min":1.982,"15min":1.982},"max":{"1min":3.2570000000000001,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":2.395}]},{"osd":3,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.7869999999999999,"5min":2.9849999999999999,"15min":2.891},"min":{"1min":1.861,"5min":1.861,"15min":1.861},"max":{"1min":3.8460000000000001,"5min":4.875,"15min":4.875},"last":1.9179999999999999},{"interface":"front","average":{"1min":2.8159999999999998,"5min":3.254,"15min":3.234},"min":{"1min":1.978,"5min":1.5660000000000001,"15min":1.5660000000000001},"max":{"1min":3.8330000000000002,"5min":5.3929999999999998,"15min":5.3929999999999998},"last":3.4319999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":3.0150000000000001,"5min":3.0289999999999999,"15min":2.9329999999999998},"min":{"1min":2.181,"5min":1.9279999999999999,"15min":1.9279999999999999},"max":{"1min":3.98,"5min":5.9729999999999999,"15min":5.9729999999999999},"last":2.6379999999999999},{"interface":"front","average":{"1min":3.3780000000000001,"5min":3.456,"15min":3.3100000000000001},"min":{"1min":2.5150000000000001,"5min":2.464,"15min":2.464},"max":{"1min":4.4260000000000002,"5min":5.6470000000000002,"15min":5.6470000000000002},"last":3.2669999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":3.4790000000000001,"5min":3.7400000000000002,"15min":3.6859999999999999},"min":{"1min":2.794,"5min":2.794,"15min":2.794},"max":{"1min":4.593,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":2.9750000000000001},{"interface":"front","average":{"1min":3.3769999999999998,"5min":3.5590000000000002,"15min":3.5129999999999999},"min":{"1min":2.7080000000000002,"5min":2.0859999999999999,"15min":2.0859999999999999},"max":{"1min":4.4779999999999998,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":2.8450000000000002}]},{"osd":6,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":3.5529999999999999,"5min":3.9319999999999999,"15min":3.9369999999999998},"min":{"1min":2.7309999999999999,"5min":2.6320000000000001,"15min":2.6320000000000001},"max":{"1min":4.7599999999999998,"5min":7.3070000000000004,"15min":7.3070000000000004},"last":3.1190000000000002},{"interface":"front","average":{"1min":3.9039999999999999,"5min":4.0140000000000002,"15min":3.9180000000000001},"min":{"1min":3.1349999999999998,"5min":2.6549999999999998,"15min":2.6549999999999998},"max":{"1min":4.6470000000000002,"5min":7.2480000000000002,"15min":7.2480000000000002},"last":3.5070000000000001}]},{"osd":8,"last update":"Tue Sep 8 15:36:33 2020","interfaces":[{"interface":"back","average":{"1min":4.1619999999999999,"5min":4.2220000000000004,"15min":4.1609999999999996},"min":{"1min":2.7250000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"max":{"1min":5.5300000000000002,"5min":6.9530000000000003,"15min":6.9530000000000003},"last":3.9500000000000002},{"interface":"front","average":{"1min":4.444,"5min":4.4059999999999997,"15min":4.2830000000000004},"min":{"1min":3.5579999999999998,"5min":2.8479999999999999,"15min":2.8479999999999999},"max":{"1min":5.6929999999999996,"5min":7.6509999999999998,"15min":7.6509999999999998},"last":4.3959999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:35:55 2020","interfaces":[{"interface":"back","average":{"1min":4.8460000000000001,"5min":4.7880000000000003,"15min":4.7779999999999996},"min":{"1min":3.7810000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"max":{"1min":6.524,"5min":6.524,"15min":6.524},"last":3.9969999999999999},{"interface":"front","average":{"1min":4.2880000000000003,"5min":4.4219999999999997,"15min":4.444},"min":{"1min":3.1989999999999998,"5min":3.1989999999999998,"15min":3.1989999999999998},"max":{"1min":6.274,"5min":6.274,"15min":6.274},"last":3.7000000000000002}]},{"osd":10,"last update":"Tue Sep 8 15:36:22 2020","interfaces":[{"interface":"back","average":{"1min":4.2460000000000004,"5min":4.649,"15min":4.7160000000000002},"min":{"1min":3.371,"5min":3.218,"15min":3.218},"max":{"1min":5.3390000000000004,"5min":7.7800000000000002,"15min":7.7800000000000002},"last":3.3540000000000001},{"interface":"front","average":{"1min":4.8840000000000003,"5min":5.1219999999999999,"15min":5.1619999999999999},"min":{"1min":3.7410000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"max":{"1min":5.8760000000000003,"5min":8.3650000000000002,"15min":8.3650000000000002},"last":3.75}]},{"osd":11,"last update":"Tue Sep 8 15:35:44 2020","interfaces":[{"interface":"back","average":{"1min":4.8380000000000001,"5min":4.8380000000000001,"15min":4.8380000000000001},"min":{"1min":3.5430000000000001,"5min":3.5430000000000001,"15min":3.5430000000000001},"max":{"1min":6.1589999999999998,"5min":6.1589999999999998,"15min":6.1589999999999998},"last":3.6419999999999999},{"interface":"front","average":{"1min":5.3170000000000002,"5min":5.3170000000000002,"15min":5.3170000000000002},"min":{"1min":3.9660000000000002,"5min":3.9660000000000002,"15min":3.9660000000000002},"max":{"1min":6.742,"5min":6.742,"15min":6.742},"last":4.2089999999999996}]}]},{"osd":6,"up_from":0,"seq":52,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6136,"kb_used_data":632,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93739016,"statfs":{"total":95995035648,"available":95988752384,"internally_reserved":0,"allocated":647168,"data_stored":365147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:35:47 2020","interfaces":[{"interface":"back","average":{"1min":2.5550000000000002,"5min":2.044,"15min":1.8999999999999999},"min":{"1min":1.841,"5min":1.141,"15min":1.141},"max":{"1min":3.714,"5min":3.714,"15min":3.714},"last":2.899},{"interface":"front","average":{"1min":2.2360000000000002,"5min":2.0529999999999999,"15min":1.9670000000000001},"min":{"1min":1.6839999999999999,"5min":1.256,"15min":1.256},"max":{"1min":2.6899999999999999,"5min":3.3279999999999998,"15min":3.3279999999999998},"last":1.7929999999999999}]},{"osd":1,"last update":"Tue Sep 8 15:35:47 2020","interfaces":[{"interface":"back","average":{"1min":2.242,"5min":2.181,"15min":2.177},"min":{"1min":1.6319999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"max":{"1min":2.7759999999999998,"5min":3.141,"15min":3.141},"last":2.0329999999999999},{"interface":"front","average":{"1min":2.9750000000000001,"5min":2.5779999999999998,"15min":2.4359999999999999},"min":{"1min":1.8,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":4.0940000000000003,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":1.9319999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:35:47 2020","interfaces":[{"interface":"back","average":{"1min":3.085,"5min":2.5840000000000001,"15min":2.4220000000000002},"min":{"1min":1.794,"5min":1.462,"15min":1.462},"max":{"1min":4.2750000000000004,"5min":4.2750000000000004,"15min":4.2750000000000004},"last":2.5219999999999998},{"interface":"front","average":{"1min":3.0910000000000002,"5min":2.5640000000000001,"15min":2.3860000000000001},"min":{"1min":2.0710000000000002,"5min":1.4790000000000001,"15min":1.4790000000000001},"max":{"1min":3.9020000000000001,"5min":4.0970000000000004,"15min":4.0970000000000004},"last":2.1819999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:35:47 2020","interfaces":[{"interface":"back","average":{"1min":3.3029999999999999,"5min":2.6589999999999998,"15min":2.4399999999999999},"min":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"max":{"1min":4.8109999999999999,"5min":4.8109999999999999,"15min":4.8109999999999999},"last":3.0430000000000001},{"interface":"front","average":{"1min":3.6949999999999998,"5min":3.0899999999999999,"15min":2.863},"min":{"1min":1.9019999999999999,"5min":1.7030000000000001,"15min":1.7030000000000001},"max":{"1min":4.8120000000000003,"5min":4.8120000000000003,"15min":4.8120000000000003},"last":3.2349999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:35:47 2020","interfaces":[{"interface":"back","average":{"1min":3.621,"5min":2.919,"15min":2.6629999999999998},"min":{"1min":2.8079999999999998,"5min":1.607,"15min":1.607},"max":{"1min":4.6550000000000002,"5min":4.702,"15min":4.702},"last":2.7589999999999999},{"interface":"front","average":{"1min":3.6179999999999999,"5min":3.085,"15min":2.8969999999999998},"min":{"1min":2.6110000000000002,"5min":1.5049999999999999,"15min":1.5049999999999999},"max":{"1min":4.8769999999999998,"5min":4.8769999999999998,"15min":4.8769999999999998},"last":3.895}]},{"osd":5,"last update":"Tue Sep 8 15:35:47 2020","interfaces":[{"interface":"back","average":{"1min":3.9649999999999999,"5min":3.3439999999999999,"15min":3.1389999999999998},"min":{"1min":3.0579999999999998,"5min":1.7509999999999999,"15min":1.7509999999999999},"max":{"1min":5.3070000000000004,"5min":6.1269999999999998,"15min":6.1269999999999998},"last":3.4910000000000001},{"interface":"front","average":{"1min":4.0549999999999997,"5min":3.3849999999999998,"15min":3.1549999999999998},"min":{"1min":2.919,"5min":2.1659999999999999,"15min":2.1659999999999999},"max":{"1min":5.8200000000000003,"5min":6.1820000000000004,"15min":6.1820000000000004},"last":2.9670000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":4.0549999999999997,"5min":3.8460000000000001,"15min":3.714},"min":{"1min":3.2879999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"max":{"1min":4.8970000000000002,"5min":5.4050000000000002,"15min":5.4050000000000002},"last":3.6309999999999998},{"interface":"front","average":{"1min":4.5359999999999996,"5min":4.0469999999999997,"15min":3.8090000000000002},"min":{"1min":3.387,"5min":2.1640000000000001,"15min":2.1640000000000001},"max":{"1min":5.5609999999999999,"5min":6.2679999999999998,"15min":6.2679999999999998},"last":3.8010000000000002}]},{"osd":8,"last update":"Tue Sep 8 15:36:30 2020","interfaces":[{"interface":"back","average":{"1min":4.3600000000000003,"5min":4.0759999999999996,"15min":3.9500000000000002},"min":{"1min":3.258,"5min":3.0579999999999998,"15min":3.0579999999999998},"max":{"1min":5.6639999999999997,"5min":5.6639999999999997,"15min":5.6639999999999997},"last":4.3399999999999999},{"interface":"front","average":{"1min":4.2759999999999998,"5min":4.2309999999999999,"15min":4.1929999999999996},"min":{"1min":3.5259999999999998,"5min":3.1269999999999998,"15min":3.1269999999999998},"max":{"1min":5.3819999999999997,"5min":5.5499999999999998,"15min":5.5499999999999998},"last":4.0549999999999997}]},{"osd":9,"last update":"Tue Sep 8 15:35:54 2020","interfaces":[{"interface":"back","average":{"1min":4.7919999999999998,"5min":4.4960000000000004,"15min":4.4470000000000001},"min":{"1min":3.4750000000000001,"5min":3.1110000000000002,"15min":3.1110000000000002},"max":{"1min":6.3170000000000002,"5min":6.3170000000000002,"15min":6.3170000000000002},"last":5.069},{"interface":"front","average":{"1min":4.5060000000000002,"5min":4.5179999999999998,"15min":4.5199999999999996},"min":{"1min":3.6659999999999999,"5min":3.5649999999999999,"15min":3.5649999999999999},"max":{"1min":5.3540000000000001,"5min":6.0979999999999999,"15min":6.0979999999999999},"last":3.367}]},{"osd":10,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":4.6909999999999998,"5min":4.694,"15min":4.694},"min":{"1min":3.7730000000000001,"5min":3.4590000000000001,"15min":3.4590000000000001},"max":{"1min":6.0720000000000001,"5min":6.0720000000000001,"15min":6.0720000000000001},"last":4.7069999999999999},{"interface":"front","average":{"1min":4.6159999999999997,"5min":4.6260000000000003,"15min":4.6280000000000001},"min":{"1min":3.4660000000000002,"5min":3.4159999999999999,"15min":3.4159999999999999},"max":{"1min":5.6980000000000004,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":4.2060000000000004}]}]},{"osd":1,"up_from":0,"seq":77,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6328,"kb_used_data":632,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738824,"statfs":{"total":95995035648,"available":95988555776,"internally_reserved":0,"allocated":647168,"data_stored":365147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:35:39 2020","interfaces":[{"interface":"back","average":{"1min":2.194,"5min":1.758,"15min":1.385},"min":{"1min":1.073,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":3.1040000000000001,"5min":3.363,"15min":3.363},"last":3.3319999999999999},{"interface":"front","average":{"1min":2.4300000000000002,"5min":2.0089999999999999,"15min":1.536},"min":{"1min":1.1399999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":3.403,"5min":3.698,"15min":3.698},"last":2.6899999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:36:04 2020","interfaces":[{"interface":"back","average":{"1min":2.5539999999999998,"5min":2.153,"15min":1.712},"min":{"1min":1.3460000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":4.4080000000000004,"5min":4.4080000000000004,"15min":4.4080000000000004},"last":2.101},{"interface":"front","average":{"1min":2.7290000000000001,"5min":2.2770000000000001,"15min":1.8680000000000001},"min":{"1min":1.431,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":4.6559999999999997,"5min":4.6559999999999997,"15min":4.6559999999999997},"last":2.5819999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:36:34 2020","interfaces":[{"interface":"back","average":{"1min":2.7509999999999999,"5min":2.5230000000000001,"15min":2.2770000000000001},"min":{"1min":0.99299999999999999,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":4.0469999999999997,"5min":4.5119999999999996,"15min":4.5119999999999996},"last":2.5070000000000001},{"interface":"front","average":{"1min":2.593,"5min":2.548,"15min":2.4079999999999999},"min":{"1min":1.746,"5min":1.2410000000000001,"15min":1.2410000000000001},"max":{"1min":4.2069999999999999,"5min":4.2069999999999999,"15min":4.2069999999999999},"last":2.996}]},{"osd":4,"last update":"Tue Sep 8 15:35:51 2020","interfaces":[{"interface":"back","average":{"1min":2.1000000000000001,"5min":2.2829999999999999,"15min":2.1160000000000001},"min":{"1min":1.4670000000000001,"5min":1.387,"15min":1.387},"max":{"1min":3.008,"5min":5.1829999999999998,"15min":5.1829999999999998},"last":2.052},{"interface":"front","average":{"1min":3.29,"5min":2.903,"15min":2.7549999999999999},"min":{"1min":2.028,"5min":1.706,"15min":1.706},"max":{"1min":4.8010000000000002,"5min":4.8010000000000002,"15min":4.8010000000000002},"last":4.1319999999999997}]},{"osd":5,"last update":"Tue Sep 8 15:36:14 2020","interfaces":[{"interface":"back","average":{"1min":3.0859999999999999,"5min":3.089,"15min":2.9449999999999998},"min":{"1min":2.105,"5min":1.8879999999999999,"15min":1.8879999999999999},"max":{"1min":4.6440000000000001,"5min":4.6440000000000001,"15min":4.6440000000000001},"last":5.2510000000000003},{"interface":"front","average":{"1min":2.6019999999999999,"5min":2.5470000000000002,"15min":2.4409999999999998},"min":{"1min":1.9410000000000001,"5min":1.5189999999999999,"15min":1.5189999999999999},"max":{"1min":3.532,"5min":4.6319999999999997,"15min":4.6319999999999997},"last":2.3900000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:36:40 2020","interfaces":[{"interface":"back","average":{"1min":3.9529999999999998,"5min":3.7440000000000002,"15min":3.621},"min":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"max":{"1min":5.367,"5min":6.1399999999999997,"15min":6.1399999999999997},"last":4.8970000000000002},{"interface":"front","average":{"1min":3.8079999999999998,"5min":3.6299999999999999,"15min":3.5609999999999999},"min":{"1min":2.5640000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"max":{"1min":5.7069999999999999,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":5.7069999999999999}]},{"osd":7,"last update":"Tue Sep 8 15:36:09 2020","interfaces":[{"interface":"back","average":{"1min":3.9950000000000001,"5min":3.4540000000000002,"15min":3.2250000000000001},"min":{"1min":2.8860000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"max":{"1min":5.3239999999999998,"5min":5.3239999999999998,"15min":5.3239999999999998},"last":6.0780000000000003},{"interface":"front","average":{"1min":3.9510000000000001,"5min":3.9100000000000001,"15min":3.883},"min":{"1min":1.907,"5min":1.907,"15min":1.907},"max":{"1min":5.375,"5min":5.5650000000000004,"15min":5.5650000000000004},"last":5.8479999999999999}]},{"osd":8,"last update":"Tue Sep 8 15:36:34 2020","interfaces":[{"interface":"back","average":{"1min":4.1890000000000001,"5min":4.2190000000000003,"15min":4.2930000000000001},"min":{"1min":2.6419999999999999,"5min":1.7290000000000001,"15min":1.7290000000000001},"max":{"1min":5.6340000000000003,"5min":5.6340000000000003,"15min":5.6340000000000003},"last":6.9720000000000004},{"interface":"front","average":{"1min":4.2210000000000001,"5min":4.0700000000000003,"15min":4.0800000000000001},"min":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"max":{"1min":6.0960000000000001,"5min":6.0960000000000001,"15min":6.0960000000000001},"last":4.5250000000000004}]},{"osd":9,"last update":"Tue Sep 8 15:35:58 2020","interfaces":[{"interface":"back","average":{"1min":3.5009999999999999,"5min":3.2919999999999998,"15min":3.2570000000000001},"min":{"1min":2.423,"5min":2.089,"15min":2.089},"max":{"1min":4.3250000000000002,"5min":4.8789999999999996,"15min":4.8789999999999996},"last":3.335},{"interface":"front","average":{"1min":4.3659999999999997,"5min":4.2560000000000002,"15min":4.2380000000000004},"min":{"1min":3.3410000000000002,"5min":2.649,"15min":2.649},"max":{"1min":6.7380000000000004,"5min":6.7380000000000004,"15min":6.7380000000000004},"last":5.3810000000000002}]},{"osd":10,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":4.3220000000000001,"5min":4.2320000000000002,"15min":4.2169999999999996},"min":{"1min":2.7949999999999999,"5min":2.3210000000000002,"15min":2.3210000000000002},"max":{"1min":5.968,"5min":5.968,"15min":5.968},"last":7.46},{"interface":"front","average":{"1min":3.4620000000000002,"5min":3.4889999999999999,"15min":3.4929999999999999},"min":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"max":{"1min":4.9640000000000004,"5min":4.9790000000000001,"15min":4.9790000000000001},"last":3.1560000000000001}]}]},{"osd":0,"up_from":0,"seq":81,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6444,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738708,"statfs":{"total":95995035648,"available":95988436992,"internally_reserved":0,"allocated":757760,"data_stored":417297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 8 15:35:42 2020","interfaces":[{"interface":"back","average":{"1min":2.1120000000000001,"5min":1.6990000000000001,"15min":1.351},"min":{"1min":1.585,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.9580000000000002,"5min":2.9580000000000002,"15min":2.9580000000000002},"last":2.2610000000000001},{"interface":"front","average":{"1min":1.9710000000000001,"5min":1.752,"15min":1.3999999999999999},"min":{"1min":1.4790000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.4750000000000001,"5min":3.8300000000000001,"15min":3.8300000000000001},"last":2.206}]},{"osd":2,"last update":"Tue Sep 8 15:34:05 2020","interfaces":[{"interface":"back","average":{"1min":1.9950000000000001,"5min":1.621,"15min":1.5189999999999999},"min":{"1min":1.3999999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.8519999999999999,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":2.6589999999999998},{"interface":"front","average":{"1min":2.629,"5min":1.956,"15min":1.72},"min":{"1min":1.6419999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":2.4049999999999998}]},{"osd":3,"last update":"Tue Sep 8 15:36:32 2020","interfaces":[{"interface":"back","average":{"1min":2.6760000000000002,"5min":2.617,"15min":2.992},"min":{"1min":1.8460000000000001,"5min":1.1879999999999999,"15min":1.1879999999999999},"max":{"1min":4.3140000000000001,"5min":27.298999999999999,"15min":27.298999999999999},"last":3.0590000000000002},{"interface":"front","average":{"1min":3.4289999999999998,"5min":3.3639999999999999,"15min":3.6219999999999999},"min":{"1min":2.2069999999999999,"5min":2.0179999999999998,"15min":2.0179999999999998},"max":{"1min":5.2590000000000003,"5min":28.314,"15min":28.314},"last":3.4399999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:35:53 2020","interfaces":[{"interface":"back","average":{"1min":3.0139999999999998,"5min":2.8250000000000002,"15min":2.4809999999999999},"min":{"1min":2.1240000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"max":{"1min":3.8199999999999998,"5min":4.4390000000000001,"15min":4.4390000000000001},"last":3.2530000000000001},{"interface":"front","average":{"1min":3.3740000000000001,"5min":3.0680000000000001,"15min":2.7549999999999999},"min":{"1min":2.5139999999999998,"5min":1.8080000000000001,"15min":1.8080000000000001},"max":{"1min":4.8129999999999997,"5min":5.1050000000000004,"15min":5.1050000000000004},"last":3.641}]},{"osd":5,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":3.8079999999999998,"5min":3.3399999999999999,"15min":3.0089999999999999},"min":{"1min":2.6400000000000001,"5min":2.0990000000000002,"15min":2.0990000000000002},"max":{"1min":4.9809999999999999,"5min":4.9809999999999999,"15min":4.9809999999999999},"last":3.8679999999999999},{"interface":"front","average":{"1min":2.3839999999999999,"5min":3.0259999999999998,"15min":3.1840000000000002},"min":{"1min":1.845,"5min":1.7110000000000001,"15min":1.7110000000000001},"max":{"1min":3.0459999999999998,"5min":5.7439999999999998,"15min":5.7439999999999998},"last":2.3439999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:35:42 2020","interfaces":[{"interface":"back","average":{"1min":2.6379999999999999,"5min":3.4329999999999998,"15min":3.573},"min":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"max":{"1min":4.7640000000000002,"5min":5.3869999999999996,"15min":5.3869999999999996},"last":3.1949999999999998},{"interface":"front","average":{"1min":2.9609999999999999,"5min":3.847,"15min":3.98},"min":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"max":{"1min":4.3760000000000003,"5min":6.0019999999999998,"15min":6.0019999999999998},"last":2.6699999999999999}]},{"osd":7,"last update":"Tue Sep 8 15:36:08 2020","interfaces":[{"interface":"back","average":{"1min":3.2669999999999999,"5min":4.069,"15min":4.3120000000000003},"min":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"max":{"1min":4.758,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":2.843},{"interface":"front","average":{"1min":3.9319999999999999,"5min":4.0860000000000003,"15min":4.1040000000000001},"min":{"1min":2.585,"5min":2.585,"15min":2.585},"max":{"1min":5.6429999999999998,"5min":5.8659999999999997,"15min":5.8659999999999997},"last":4.0650000000000004}]},{"osd":8,"last update":"Tue Sep 8 15:36:32 2020","interfaces":[{"interface":"back","average":{"1min":4.5739999999999998,"5min":4.1109999999999998,"15min":3.9489999999999998},"min":{"1min":3.0329999999999999,"5min":2.6629999999999998,"15min":2.6629999999999998},"max":{"1min":6.5049999999999999,"5min":6.6379999999999999,"15min":6.6379999999999999},"last":4.718},{"interface":"front","average":{"1min":3.395,"5min":3.9039999999999999,"15min":4.133},"min":{"1min":2.5049999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"max":{"1min":4.5540000000000003,"5min":6.2030000000000003,"15min":6.2030000000000003},"last":3.0009999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:35:53 2020","interfaces":[{"interface":"back","average":{"1min":4.2889999999999997,"5min":4.2850000000000001,"15min":4.2850000000000001},"min":{"1min":2.734,"5min":2.714,"15min":2.714},"max":{"1min":5.7770000000000001,"5min":6.1769999999999996,"15min":6.1769999999999996},"last":4.923},{"interface":"front","average":{"1min":3.472,"5min":4,"15min":4.0880000000000001},"min":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"max":{"1min":5.4790000000000001,"5min":5.4790000000000001,"15min":5.4790000000000001},"last":3.3410000000000002}]},{"osd":10,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":3.3039999999999998,"5min":3.157,"15min":3.133},"min":{"1min":2.649,"5min":2.3210000000000002,"15min":2.3210000000000002},"max":{"1min":4.1529999999999996,"5min":4.1680000000000001,"15min":4.1680000000000001},"last":3.1150000000000002},{"interface":"front","average":{"1min":4.944,"5min":4.4969999999999999,"15min":4.423},"min":{"1min":3.452,"5min":2.8340000000000001,"15min":2.8340000000000001},"max":{"1min":6.758,"5min":6.758,"15min":6.758},"last":4.516}]},{"osd":11,"last update":"Tue Sep 8 15:35:42 2020","interfaces":[{"interface":"back","average":{"1min":3.8889999999999998,"5min":3.8889999999999998,"15min":3.8889999999999998},"min":{"1min":2.9460000000000002,"5min":2.9460000000000002,"15min":2.9460000000000002},"max":{"1min":5.2599999999999998,"5min":5.2599999999999998,"15min":5.2599999999999998},"last":3.484},{"interface":"front","average":{"1min":4.8609999999999998,"5min":4.8609999999999998,"15min":4.8609999999999998},"min":{"1min":2.911,"5min":2.911,"15min":2.911},"max":{"1min":7.0220000000000002,"5min":7.0220000000000002,"15min":7.0220000000000002},"last":4.2949999999999999}]}]},{"osd":2,"up_from":0,"seq":73,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6252,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738900,"statfs":{"total":95995035648,"available":95988633600,"internally_reserved":0,"allocated":757760,"data_stored":417297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.1970000000000001,"5min":1.9179999999999999,"15min":1.623},"min":{"1min":1.4259999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":3.883,"5min":3.883,"15min":3.883},"last":3.105},{"interface":"front","average":{"1min":2.0030000000000001,"5min":1.9419999999999999,"15min":1.702},"min":{"1min":1.599,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.7160000000000002,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":2.8119999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.0390000000000001,"5min":1.915,"15min":1.6870000000000001},"min":{"1min":1.306,"5min":0.999,"15min":0.999},"max":{"1min":3.363,"5min":3.363,"15min":3.363},"last":2.1899999999999999},{"interface":"front","average":{"1min":2.6629999999999998,"5min":2.4340000000000002,"15min":2.0899999999999999},"min":{"1min":1.2809999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"max":{"1min":3.456,"5min":3.9399999999999999,"15min":3.9399999999999999},"last":2.2709999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:36:32 2020","interfaces":[{"interface":"back","average":{"1min":2.944,"5min":2.706,"15min":2.3799999999999999},"min":{"1min":1.7130000000000001,"5min":1.587,"15min":1.587},"max":{"1min":4.1429999999999998,"5min":4.3209999999999997,"15min":4.3209999999999997},"last":3.4340000000000002},{"interface":"front","average":{"1min":2.7490000000000001,"5min":2.5600000000000001,"15min":2.3679999999999999},"min":{"1min":1.9059999999999999,"5min":0.755,"15min":0.755},"max":{"1min":3.641,"5min":3.6709999999999998,"15min":3.6709999999999998},"last":3.0419999999999998}]},{"osd":4,"last update":"Tue Sep 8 15:35:52 2020","interfaces":[{"interface":"back","average":{"1min":2.9329999999999998,"5min":2.6680000000000001,"15min":2.4399999999999999},"min":{"1min":1.9350000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"max":{"1min":4.4340000000000002,"5min":4.4340000000000002,"15min":4.4340000000000002},"last":2.3559999999999999},{"interface":"front","average":{"1min":2.9849999999999999,"5min":2.6600000000000001,"15min":2.3860000000000001},"min":{"1min":1.7210000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"max":{"1min":4.7080000000000002,"5min":4.827,"15min":4.827},"last":3.3370000000000002}]},{"osd":5,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":3.5569999999999999,"5min":3.222,"15min":2.9380000000000002},"min":{"1min":2.4060000000000001,"5min":1.353,"15min":1.353},"max":{"1min":4.6050000000000004,"5min":4.8449999999999998,"15min":4.8449999999999998},"last":3.6909999999999998},{"interface":"front","average":{"1min":3.7599999999999998,"5min":3.2879999999999998,"15min":2.98},"min":{"1min":2.4630000000000001,"5min":1.232,"15min":1.232},"max":{"1min":4.4530000000000003,"5min":5.2119999999999997,"15min":5.2119999999999997},"last":3.843}]},{"osd":6,"last update":"Tue Sep 8 15:35:43 2020","interfaces":[{"interface":"back","average":{"1min":3.4969999999999999,"5min":3.2650000000000001,"15min":3.1360000000000001},"min":{"1min":2.1779999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"max":{"1min":5.1440000000000001,"5min":5.3929999999999998,"15min":5.3929999999999998},"last":4.375},{"interface":"front","average":{"1min":3.9660000000000002,"5min":3.6800000000000002,"15min":3.536},"min":{"1min":2.2679999999999998,"5min":2.0169999999999999,"15min":2.0169999999999999},"max":{"1min":5.6639999999999997,"5min":5.6639999999999997,"15min":5.6639999999999997},"last":4.2939999999999996}]},{"osd":7,"last update":"Tue Sep 8 15:36:09 2020","interfaces":[{"interface":"back","average":{"1min":4.4720000000000004,"5min":4.1779999999999999,"15min":4.0419999999999998},"min":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"max":{"1min":5.4340000000000002,"5min":6.5780000000000003,"15min":6.5780000000000003},"last":4.7709999999999999},{"interface":"front","average":{"1min":4.0419999999999998,"5min":3.7490000000000001,"15min":3.722},"min":{"1min":2.847,"5min":2.1299999999999999,"15min":2.1299999999999999},"max":{"1min":7.8659999999999997,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":3.7320000000000002}]},{"osd":8,"last update":"Tue Sep 8 15:36:26 2020","interfaces":[{"interface":"back","average":{"1min":3.9900000000000002,"5min":3.7370000000000001,"15min":3.6850000000000001},"min":{"1min":2.5369999999999999,"5min":1.907,"15min":1.907},"max":{"1min":5.5359999999999996,"5min":5.5359999999999996,"15min":5.5359999999999996},"last":4.1710000000000003},{"interface":"front","average":{"1min":4.2439999999999998,"5min":4.0410000000000004,"15min":3.9889999999999999},"min":{"1min":3.0699999999999998,"5min":2.7719999999999998,"15min":2.7719999999999998},"max":{"1min":5.4939999999999998,"5min":6.3150000000000004,"15min":6.3150000000000004},"last":4.5529999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:35:59 2020","interfaces":[{"interface":"back","average":{"1min":4.0670000000000002,"5min":4.0499999999999998,"15min":4.0469999999999997},"min":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"max":{"1min":6.3559999999999999,"5min":6.3559999999999999,"15min":6.3559999999999999},"last":3.9660000000000002},{"interface":"front","average":{"1min":4.2220000000000004,"5min":4.2850000000000001,"15min":4.2949999999999999},"min":{"1min":2.71,"5min":2.71,"15min":2.71},"max":{"1min":6.1070000000000002,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":4.0800000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:36:22 2020","interfaces":[{"interface":"back","average":{"1min":4.4729999999999999,"5min":4.1130000000000004,"15min":4.0529999999999999},"min":{"1min":3.145,"5min":2.7480000000000002,"15min":2.7480000000000002},"max":{"1min":6.915,"5min":6.915,"15min":6.915},"last":4.6289999999999996},{"interface":"front","average":{"1min":3.895,"5min":3.8140000000000001,"15min":3.7999999999999998},"min":{"1min":2.6890000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"max":{"1min":7.6580000000000004,"5min":7.6580000000000004,"15min":7.6580000000000004},"last":4.0380000000000003}]}]},{"osd":3,"up_from":0,"seq":67,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6252,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738900,"statfs":{"total":95995035648,"available":95988633600,"internally_reserved":0,"allocated":757760,"data_stored":417297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:31 2020","interfaces":[{"interface":"back","average":{"1min":2.1379999999999999,"5min":1.7210000000000001,"15min":1.629},"min":{"1min":1.79,"5min":1.1419999999999999,"15min":1.1419999999999999},"max":{"1min":2.7930000000000001,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":3.169},{"interface":"front","average":{"1min":2.6800000000000002,"5min":2.2309999999999999,"15min":2.0990000000000002},"min":{"1min":2.0270000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"max":{"1min":3.5409999999999999,"5min":3.5409999999999999,"15min":3.5409999999999999},"last":2.5579999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:36:35 2020","interfaces":[{"interface":"back","average":{"1min":2.133,"5min":2.0939999999999999,"15min":1.8740000000000001},"min":{"1min":1.2809999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"max":{"1min":3.3399999999999999,"5min":3.6829999999999998,"15min":3.6829999999999998},"last":1.6539999999999999},{"interface":"front","average":{"1min":2.1709999999999998,"5min":2.1400000000000001,"15min":1.8460000000000001},"min":{"1min":1.4059999999999999,"5min":1.341,"15min":1.341},"max":{"1min":3.1549999999999998,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":1.577}]},{"osd":2,"last update":"Tue Sep 8 15:36:35 2020","interfaces":[{"interface":"back","average":{"1min":2.5619999999999998,"5min":2.387,"15min":2.145},"min":{"1min":1.7669999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":3.5009999999999999,"5min":3.7810000000000001,"15min":3.7810000000000001},"last":1.885},{"interface":"front","average":{"1min":2.1869999999999998,"5min":2.464,"15min":2.2389999999999999},"min":{"1min":1.585,"5min":1.3580000000000001,"15min":1.3580000000000001},"max":{"1min":3.1699999999999999,"5min":3.8010000000000002,"15min":3.8010000000000002},"last":1.716}]},{"osd":4,"last update":"Tue Sep 8 15:35:54 2020","interfaces":[{"interface":"back","average":{"1min":2.8540000000000001,"5min":2.7320000000000002,"15min":2.5419999999999998},"min":{"1min":1.8839999999999999,"5min":1.4850000000000001,"15min":1.4850000000000001},"max":{"1min":3.778,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":2.8620000000000001},{"interface":"front","average":{"1min":2.8010000000000002,"5min":2.7189999999999999,"15min":2.4830000000000001},"min":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"max":{"1min":3.4700000000000002,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":2.3450000000000002}]},{"osd":5,"last update":"Tue Sep 8 15:36:15 2020","interfaces":[{"interface":"back","average":{"1min":2.9300000000000002,"5min":2.9830000000000001,"15min":2.8530000000000002},"min":{"1min":1.323,"5min":1.323,"15min":1.323},"max":{"1min":3.9489999999999998,"5min":5.0289999999999999,"15min":5.0289999999999999},"last":3.3559999999999999},{"interface":"front","average":{"1min":3.4569999999999999,"5min":3.3929999999999998,"15min":3.2330000000000001},"min":{"1min":1.988,"5min":1.988,"15min":1.988},"max":{"1min":4.2910000000000004,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":3.2810000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:35:40 2020","interfaces":[{"interface":"back","average":{"1min":3.2370000000000001,"5min":3.419,"15min":3.339},"min":{"1min":1.714,"5min":1.714,"15min":1.714},"max":{"1min":4.569,"5min":5.4160000000000004,"15min":5.4160000000000004},"last":3.649},{"interface":"front","average":{"1min":3.7280000000000002,"5min":3.661,"15min":3.5409999999999999},"min":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"max":{"1min":5.2130000000000001,"5min":5.3499999999999996,"15min":5.3499999999999996},"last":3.113}]},{"osd":7,"last update":"Tue Sep 8 15:36:11 2020","interfaces":[{"interface":"back","average":{"1min":4.234,"5min":4.2649999999999997,"15min":4.2530000000000001},"min":{"1min":3.081,"5min":2.8069999999999999,"15min":2.8069999999999999},"max":{"1min":5.1440000000000001,"5min":5.5640000000000001,"15min":5.5640000000000001},"last":4.048},{"interface":"front","average":{"1min":3.577,"5min":3.629,"15min":3.617},"min":{"1min":2.601,"5min":2.601,"15min":2.601},"max":{"1min":4.258,"5min":5.258,"15min":5.258},"last":2.7320000000000002}]},{"osd":8,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":3.8889999999999998,"5min":3.758,"15min":3.7829999999999999},"min":{"1min":2.9119999999999999,"5min":2.2440000000000002,"15min":2.2440000000000002},"max":{"1min":5.3390000000000004,"5min":5.4610000000000003,"15min":5.4610000000000003},"last":3.5449999999999999},{"interface":"front","average":{"1min":4.5220000000000002,"5min":4.3319999999999999,"15min":4.3079999999999998},"min":{"1min":3.206,"5min":2.351,"15min":2.351},"max":{"1min":5.5309999999999997,"5min":6.1239999999999997,"15min":6.1239999999999997},"last":4.5579999999999998}]},{"osd":9,"last update":"Tue Sep 8 15:35:54 2020","interfaces":[{"interface":"back","average":{"1min":3.7930000000000001,"5min":3.96,"15min":3.988},"min":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"max":{"1min":4.468,"5min":4.6120000000000001,"15min":4.6120000000000001},"last":3.5099999999999998},{"interface":"front","average":{"1min":3.5840000000000001,"5min":4.2480000000000002,"15min":4.3579999999999997},"min":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"max":{"1min":4.7889999999999997,"5min":5.2530000000000001,"15min":5.2530000000000001},"last":4.1260000000000003}]},{"osd":10,"last update":"Tue Sep 8 15:36:21 2020","interfaces":[{"interface":"back","average":{"1min":4.6360000000000001,"5min":4.4450000000000003,"15min":4.4130000000000003},"min":{"1min":2.875,"5min":2.875,"15min":2.875},"max":{"1min":5.7880000000000003,"5min":5.7880000000000003,"15min":5.7880000000000003},"last":4.2919999999999998},{"interface":"front","average":{"1min":4.0990000000000002,"5min":3.8290000000000002,"15min":3.7839999999999998},"min":{"1min":3.194,"5min":2.8759999999999999,"15min":2.8759999999999999},"max":{"1min":4.7599999999999998,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":3.044}]},{"osd":11,"last update":"Tue Sep 8 15:35:50 2020","interfaces":[{"interface":"back","average":{"1min":4.8159999999999998,"5min":4.8159999999999998,"15min":4.8159999999999998},"min":{"1min":3.7509999999999999,"5min":3.7509999999999999,"15min":3.7509999999999999},"max":{"1min":5.9829999999999997,"5min":5.9829999999999997,"15min":5.9829999999999997},"last":4.7839999999999998},{"interface":"front","average":{"1min":4.1459999999999999,"5min":4.1459999999999999,"15min":4.1459999999999999},"min":{"1min":3.04,"5min":3.04,"15min":3.04},"max":{"1min":5.1589999999999998,"5min":5.1589999999999998,"15min":5.1589999999999998},"last":3.7090000000000001}]}]},{"osd":4,"up_from":0,"seq":63,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6188,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738964,"statfs":{"total":95995035648,"available":95988699136,"internally_reserved":0,"allocated":757760,"data_stored":417297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:35:52 2020","interfaces":[{"interface":"back","average":{"1min":2.2679999999999998,"5min":1.9890000000000001,"15min":1.845},"min":{"1min":1.5740000000000001,"5min":1.01,"15min":1.01},"max":{"1min":3.0880000000000001,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":2.21},{"interface":"front","average":{"1min":2.5600000000000001,"5min":2.0699999999999998,"15min":1.792},"min":{"1min":1.8759999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.4169999999999998,"5min":3.7679999999999998,"15min":3.7679999999999998},"last":2.5310000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:35:52 2020","interfaces":[{"interface":"back","average":{"1min":2.4049999999999998,"5min":1.998,"15min":1.839},"min":{"1min":1.51,"5min":1.1759999999999999,"15min":1.1759999999999999},"max":{"1min":3.5270000000000001,"5min":5.3659999999999997,"15min":5.3659999999999997},"last":1.8460000000000001},{"interface":"front","average":{"1min":2.8279999999999998,"5min":2.391,"15min":2.0590000000000002},"min":{"1min":1.7090000000000001,"5min":1.1699999999999999,"15min":1.1699999999999999},"max":{"1min":4.0339999999999998,"5min":4.5700000000000003,"15min":4.5700000000000003},"last":2.028}]},{"osd":2,"last update":"Tue Sep 8 15:35:52 2020","interfaces":[{"interface":"back","average":{"1min":3.0510000000000002,"5min":2.4540000000000002,"15min":2.0600000000000001},"min":{"1min":1.7270000000000001,"5min":1.236,"15min":1.236},"max":{"1min":3.7429999999999999,"5min":4.1550000000000002,"15min":4.1550000000000002},"last":2.7890000000000001},{"interface":"front","average":{"1min":3.2000000000000002,"5min":2.6080000000000001,"15min":2.2400000000000002},"min":{"1min":1.8879999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"max":{"1min":4.5609999999999999,"5min":4.5609999999999999,"15min":4.5609999999999999},"last":2.6230000000000002}]},{"osd":3,"last update":"Tue Sep 8 15:35:52 2020","interfaces":[{"interface":"back","average":{"1min":3.3660000000000001,"5min":2.8109999999999999,"15min":2.4369999999999998},"min":{"1min":1.9670000000000001,"5min":1.389,"15min":1.389},"max":{"1min":4.968,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":3.0649999999999999},{"interface":"front","average":{"1min":3.032,"5min":2.6240000000000001,"15min":2.302},"min":{"1min":2.1379999999999999,"5min":1.3440000000000001,"15min":1.3440000000000001},"max":{"1min":4.4580000000000002,"5min":4.6180000000000003,"15min":4.6180000000000003},"last":3.1699999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":3.9569999999999999,"5min":3.294,"15min":2.7519999999999998},"min":{"1min":2.7149999999999999,"5min":1.014,"15min":1.014},"max":{"1min":6.9800000000000004,"5min":6.9800000000000004,"15min":6.9800000000000004},"last":4.2839999999999998},{"interface":"front","average":{"1min":3.9449999999999998,"5min":3.274,"15min":2.8199999999999998},"min":{"1min":2.5800000000000001,"5min":1.129,"15min":1.129},"max":{"1min":6.2279999999999998,"5min":6.2279999999999998,"15min":6.2279999999999998},"last":3.6480000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":4.157,"5min":3.4980000000000002,"15min":3.29},"min":{"1min":2.8759999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"max":{"1min":6.6870000000000003,"5min":6.6870000000000003,"15min":6.6870000000000003},"last":4.0540000000000003},{"interface":"front","average":{"1min":3.7909999999999999,"5min":3.5030000000000001,"15min":3.3290000000000002},"min":{"1min":2.8050000000000002,"5min":2.2429999999999999,"15min":2.2429999999999999},"max":{"1min":4.8150000000000004,"5min":5.1260000000000003,"15min":5.1260000000000003},"last":2.9980000000000002}]},{"osd":7,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":4.1150000000000002,"5min":3.9860000000000002,"15min":3.8700000000000001},"min":{"1min":2.5739999999999998,"5min":1.75,"15min":1.75},"max":{"1min":7.4699999999999998,"5min":7.4699999999999998,"15min":7.4699999999999998},"last":3.855},{"interface":"front","average":{"1min":4.0430000000000001,"5min":4.0570000000000004,"15min":3.98},"min":{"1min":2.718,"5min":2.6619999999999999,"15min":2.6619999999999999},"max":{"1min":6.6399999999999997,"5min":6.6399999999999997,"15min":6.6399999999999997},"last":4.5090000000000003}]},{"osd":8,"last update":"Tue Sep 8 15:36:30 2020","interfaces":[{"interface":"back","average":{"1min":4.1280000000000001,"5min":4,"15min":3.9359999999999999},"min":{"1min":3.327,"5min":3.0739999999999998,"15min":3.0739999999999998},"max":{"1min":5.1920000000000002,"5min":5.3929999999999998,"15min":5.3929999999999998},"last":4.6100000000000003},{"interface":"front","average":{"1min":4.681,"5min":4.4619999999999997,"15min":4.3799999999999999},"min":{"1min":3.746,"5min":3.0840000000000001,"15min":3.0840000000000001},"max":{"1min":6.5540000000000003,"5min":6.5540000000000003,"15min":6.5540000000000003},"last":4.6920000000000002}]},{"osd":9,"last update":"Tue Sep 8 15:35:55 2020","interfaces":[{"interface":"back","average":{"1min":4.4349999999999996,"5min":4.3860000000000001,"15min":4.3780000000000001},"min":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"max":{"1min":5.5590000000000002,"5min":5.5590000000000002,"15min":5.5590000000000002},"last":5.1310000000000002},{"interface":"front","average":{"1min":4.2939999999999996,"5min":4.4020000000000001,"15min":4.4199999999999999},"min":{"1min":3.5289999999999999,"5min":3.5289999999999999,"15min":3.5289999999999999},"max":{"1min":5.2320000000000002,"5min":5.7779999999999996,"15min":5.7779999999999996},"last":3.98}]},{"osd":10,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":4.8129999999999997,"5min":4.9130000000000003,"15min":4.9299999999999997},"min":{"1min":3.9769999999999999,"5min":3.3439999999999999,"15min":3.3439999999999999},"max":{"1min":7.9279999999999999,"5min":7.9279999999999999,"15min":7.9279999999999999},"last":5.2069999999999999},{"interface":"front","average":{"1min":4.8440000000000003,"5min":4.9109999999999996,"15min":4.9219999999999997},"min":{"1min":3.7890000000000001,"5min":3.7890000000000001,"15min":3.7890000000000001},"max":{"1min":7.7590000000000003,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":5.5250000000000004}]}]},{"osd":5,"up_from":0,"seq":57,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6200,"kb_used_data":632,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738952,"statfs":{"total":95995035648,"available":95988686848,"internally_reserved":0,"allocated":647168,"data_stored":365147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":2.2589999999999999,"5min":2.1240000000000001,"15min":1.8740000000000001},"min":{"1min":1.7749999999999999,"5min":1.119,"15min":1.119},"max":{"1min":3.1320000000000001,"5min":3.96,"15min":3.96},"last":1.7949999999999999},{"interface":"front","average":{"1min":2.5790000000000002,"5min":2.2290000000000001,"15min":1.9670000000000001},"min":{"1min":1.929,"5min":1.4510000000000001,"15min":1.4510000000000001},"max":{"1min":3.7570000000000001,"5min":3.7570000000000001,"15min":3.7570000000000001},"last":2.004}]},{"osd":1,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.0049999999999999,"5min":2.5259999999999998,"15min":2.3370000000000002},"min":{"1min":1.643,"5min":1.339,"15min":1.339},"max":{"1min":4.2240000000000002,"5min":4.2240000000000002,"15min":4.2240000000000002},"last":2.1299999999999999},{"interface":"front","average":{"1min":2.1779999999999999,"5min":2.0760000000000001,"15min":2.0059999999999998},"min":{"1min":1.7090000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"max":{"1min":2.9830000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":2.7690000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":2.456,"5min":2.2839999999999998,"15min":2.1339999999999999},"min":{"1min":1.8109999999999999,"5min":1.474,"15min":1.474},"max":{"1min":3.5539999999999998,"5min":3.7570000000000001,"15min":3.7570000000000001},"last":2.835},{"interface":"front","average":{"1min":3.4980000000000002,"5min":2.9969999999999999,"15min":2.7410000000000001},"min":{"1min":2.1240000000000001,"5min":1.796,"15min":1.796},"max":{"1min":4.9000000000000004,"5min":4.9000000000000004,"15min":4.9000000000000004},"last":3.052}]},{"osd":3,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.7639999999999998,"5min":3.234,"15min":2.839},"min":{"1min":2.4249999999999998,"5min":1.6719999999999999,"15min":1.6719999999999999},"max":{"1min":4.718,"5min":5.3899999999999997,"15min":5.3899999999999997},"last":3.4820000000000002},{"interface":"front","average":{"1min":3.2280000000000002,"5min":3.1139999999999999,"15min":2.9129999999999998},"min":{"1min":2.3679999999999999,"5min":1.9990000000000001,"15min":1.9990000000000001},"max":{"1min":4.2050000000000001,"5min":5.4889999999999999,"15min":5.4889999999999999},"last":3.3500000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":2.9780000000000002,"5min":2.5710000000000002,"15min":2.3010000000000002},"min":{"1min":2.0910000000000002,"5min":1.6220000000000001,"15min":1.6220000000000001},"max":{"1min":3.5499999999999998,"5min":3.9710000000000001,"15min":3.9710000000000001},"last":2.7200000000000002},{"interface":"front","average":{"1min":3.7210000000000001,"5min":3.0779999999999998,"15min":2.6829999999999998},"min":{"1min":2.819,"5min":1.3979999999999999,"15min":1.3979999999999999},"max":{"1min":6.5190000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":3.1000000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:35:43 2020","interfaces":[{"interface":"back","average":{"1min":4.1769999999999996,"5min":3.6960000000000002,"15min":3.5680000000000001},"min":{"1min":3.1549999999999998,"5min":2.3959999999999999,"15min":2.3959999999999999},"max":{"1min":5.9870000000000001,"5min":5.9870000000000001,"15min":5.9870000000000001},"last":3.2829999999999999},{"interface":"front","average":{"1min":4.3019999999999996,"5min":3.2650000000000001,"15min":3.0150000000000001},"min":{"1min":3.532,"5min":1.948,"15min":1.948},"max":{"1min":5.6600000000000001,"5min":5.6600000000000001,"15min":5.6600000000000001},"last":3.766}]},{"osd":7,"last update":"Tue Sep 8 15:36:03 2020","interfaces":[{"interface":"back","average":{"1min":4.8559999999999999,"5min":3.9359999999999999,"15min":3.617},"min":{"1min":3.1909999999999998,"5min":2.5859999999999999,"15min":2.5859999999999999},"max":{"1min":6.782,"5min":6.782,"15min":6.782},"last":2.948},{"interface":"front","average":{"1min":3.7669999999999999,"5min":3.9849999999999999,"15min":4.0099999999999998},"min":{"1min":2.665,"5min":2.665,"15min":2.665},"max":{"1min":4.8520000000000003,"5min":5.9059999999999997,"15min":5.9059999999999997},"last":4.0369999999999999}]},{"osd":8,"last update":"Tue Sep 8 15:36:34 2020","interfaces":[{"interface":"back","average":{"1min":3.621,"5min":3.677,"15min":3.5990000000000002},"min":{"1min":3.101,"5min":2.8940000000000001,"15min":2.8940000000000001},"max":{"1min":4.2649999999999997,"5min":5.2610000000000001,"15min":5.2610000000000001},"last":3.2109999999999999},{"interface":"front","average":{"1min":4.0570000000000004,"5min":4.0629999999999997,"15min":4.0529999999999999},"min":{"1min":3.4279999999999999,"5min":3.2400000000000002,"15min":3.2400000000000002},"max":{"1min":4.6230000000000002,"5min":5.3319999999999999,"15min":5.3319999999999999},"last":3.8679999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:35:58 2020","interfaces":[{"interface":"back","average":{"1min":5.2080000000000002,"5min":4.6870000000000003,"15min":4.5999999999999996},"min":{"1min":4.1040000000000001,"5min":3.4550000000000001,"15min":3.4550000000000001},"max":{"1min":7.1719999999999997,"5min":7.1719999999999997,"15min":7.1719999999999997},"last":3.714},{"interface":"front","average":{"1min":4.218,"5min":4.218,"15min":4.218},"min":{"1min":3.629,"5min":2.6930000000000001,"15min":2.6930000000000001},"max":{"1min":4.8799999999999999,"5min":5.6929999999999996,"15min":5.6929999999999996},"last":4.2469999999999999}]},{"osd":10,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":4.2569999999999997,"5min":4.2409999999999997,"15min":4.2380000000000004},"min":{"1min":3.3849999999999998,"5min":3.0190000000000001,"15min":3.0190000000000001},"max":{"1min":5.3609999999999998,"5min":5.3609999999999998,"15min":5.3609999999999998},"last":3.5859999999999999},{"interface":"front","average":{"1min":5.4649999999999999,"5min":4.9219999999999997,"15min":4.8319999999999999},"min":{"1min":3.9039999999999999,"5min":3.9039999999999999,"15min":3.9039999999999999},"max":{"1min":6.3390000000000004,"5min":6.3390000000000004,"15min":6.3390000000000004},"last":4.4379999999999997}]}]}]}} 2020-09-08T15:36:44.641 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph pg dump --format=json 2020-09-08T15:36:44.768 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:36:45.828 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:45 smithi104 bash[9203]: audit 2020-09-08T15:36:43.935553+0000 mgr.x (mgr.24118) 270 : audit [DBG] from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:36:45.828 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:45 smithi104 bash[9203]: cluster 2020-09-08T15:36:44.922727+0000 osd.4 (osd.4) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:45.828 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:45 smithi104 bash[9203]: cluster 2020-09-08T15:36:44.922855+0000 osd.4 (osd.4) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:45.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:45 smithi166 bash[9446]: audit 2020-09-08T15:36:43.935553+0000 mgr.x (mgr.24118) 270 : audit [DBG] from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:36:45.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:45 smithi166 bash[9446]: cluster 2020-09-08T15:36:44.922727+0000 osd.4 (osd.4) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:45.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:45 smithi166 bash[9446]: cluster 2020-09-08T15:36:44.922855+0000 osd.4 (osd.4) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:45.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:45 smithi055 bash[9296]: audit 2020-09-08T15:36:43.935553+0000 mgr.x (mgr.24118) 270 : audit [DBG] from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:36:45.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:45 smithi055 bash[9296]: cluster 2020-09-08T15:36:44.922727+0000 osd.4 (osd.4) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:45.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:45 smithi055 bash[9296]: cluster 2020-09-08T15:36:44.922855+0000 osd.4 (osd.4) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:46.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:46 smithi166 bash[9446]: cluster 2020-09-08T15:36:45.563302+0000 mon.a (mon.0) 878 : cluster [DBG] osdmap e116: 12 total, 12 up, 12 in 2020-09-08T15:36:46.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:46 smithi166 bash[9446]: audit 2020-09-08T15:36:45.585822+0000 mon.a (mon.0) 879 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:46.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:46 smithi166 bash[9446]: audit 2020-09-08T15:36:45.585893+0000 mon.b (mon.1) 69 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:46.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:46 smithi166 bash[9446]: audit 2020-09-08T15:36:45.590989+0000 mon.a (mon.0) 880 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:36:46.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:46 smithi055 bash[9296]: cluster 2020-09-08T15:36:45.563302+0000 mon.a (mon.0) 878 : cluster [DBG] osdmap e116: 12 total, 12 up, 12 in 2020-09-08T15:36:46.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:46 smithi055 bash[9296]: audit 2020-09-08T15:36:45.585822+0000 mon.a (mon.0) 879 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:46.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:46 smithi055 bash[9296]: audit 2020-09-08T15:36:45.585893+0000 mon.b (mon.1) 69 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:46.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:46 smithi055 bash[9296]: audit 2020-09-08T15:36:45.590989+0000 mon.a (mon.0) 880 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:36:46.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:46 smithi104 bash[9203]: cluster 2020-09-08T15:36:45.563302+0000 mon.a (mon.0) 878 : cluster [DBG] osdmap e116: 12 total, 12 up, 12 in 2020-09-08T15:36:46.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:46 smithi104 bash[9203]: audit 2020-09-08T15:36:45.585822+0000 mon.a (mon.0) 879 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:46.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:46 smithi104 bash[9203]: audit 2020-09-08T15:36:45.585893+0000 mon.b (mon.1) 69 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:46.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:46 smithi104 bash[9203]: audit 2020-09-08T15:36:45.590989+0000 mon.a (mon.0) 880 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:36:47.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:47 smithi104 bash[9203]: cluster 2020-09-08T15:36:47.573887+0000 mgr.x (mgr.24118) 272 : cluster [DBG] pgmap v316: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:48.005 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T15:36:48.006 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2020-09-08T15:36:48.136 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:47 smithi166 bash[9446]: cluster 2020-09-08T15:36:47.573887+0000 mgr.x (mgr.24118) 272 : cluster [DBG] pgmap v316: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:48.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:47 smithi055 bash[9296]: cluster 2020-09-08T15:36:47.573887+0000 mgr.x (mgr.24118) 272 : cluster [DBG] pgmap v316: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:48.673 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2020-09-08T15:36:47.573485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":12,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":1124941824,"kb_used":73060,"kb_used_data":7740,"kb_used_omap":0,"kb_used_meta":64384,"kb_avail":1124868764,"statfs":{"total":1151940427776,"available":1151865614336,"internally_reserved":0,"allocated":7925760,"data_stored":4462662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65929216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002450"},"pg_stats":[{"pgid":"1.0","version":"99'6","reported_seq":"122","reported_epoch":"114","state":"active+clean","last_fresh":"2020-09-08T15:36:38.746975+0000","last_change":"2020-09-08T15:36:24.184987+0000","last_active":"2020-09-08T15:36:38.746975+0000","last_peered":"2020-09-08T15:36:38.746975+0000","last_clean":"2020-09-08T15:36:38.746975+0000","last_became_active":"2020-09-08T15:34:43.031697+0000","last_became_peered":"2020-09-08T15:34:43.031697+0000","last_unstale":"2020-09-08T15:36:38.746975+0000","last_undegraded":"2020-09-08T15:36:38.746975+0000","last_fullsized":"2020-09-08T15:36:38.746975+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":2,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'6","last_scrub_stamp":"2020-09-08T15:36:24.184849+0000","last_deep_scrub":"99'6","last_deep_scrub_stamp":"2020-09-08T15:35:23.132465+0000","last_clean_scrub_stamp":"2020-09-08T15:36:24.184849+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":2,"acting":2,"num_store_stats":7}],"osd_stats":[{"osd":11,"up_from":0,"seq":28,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5740,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":93739412,"statfs":{"total":95995035648,"available":95989157888,"internally_reserved":0,"allocated":757760,"data_stored":417277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"min":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"max":{"1min":2.4849999999999999,"5min":2.4849999999999999,"15min":2.4849999999999999},"last":1.3700000000000001},{"interface":"front","average":{"1min":3.3119999999999998,"5min":3.3119999999999998,"15min":3.3119999999999998},"min":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"max":{"1min":4.5970000000000004,"5min":4.5970000000000004,"15min":4.5970000000000004},"last":2.573}]},{"osd":1,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"min":{"1min":1.395,"5min":1.395,"15min":1.395},"max":{"1min":3.2250000000000001,"5min":3.2250000000000001,"15min":3.2250000000000001},"last":2.4780000000000002},{"interface":"front","average":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"min":{"1min":1.591,"5min":1.591,"15min":1.591},"max":{"1min":3.8580000000000001,"5min":3.8580000000000001,"15min":3.8580000000000001},"last":3.0339999999999998}]},{"osd":2,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"min":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"max":{"1min":3.7949999999999999,"5min":3.7949999999999999,"15min":3.7949999999999999},"last":3.593},{"interface":"front","average":{"1min":2.8679999999999999,"5min":2.8679999999999999,"15min":2.8679999999999999},"min":{"1min":1.893,"5min":1.893,"15min":1.893},"max":{"1min":4.2939999999999996,"5min":4.2939999999999996,"15min":4.2939999999999996},"last":1.702}]},{"osd":3,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"min":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"max":{"1min":3.5049999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":2.8740000000000001},{"interface":"front","average":{"1min":3.137,"5min":3.137,"15min":3.137},"min":{"1min":1.498,"5min":1.498,"15min":1.498},"max":{"1min":4.8949999999999996,"5min":4.8949999999999996,"15min":4.8949999999999996},"last":2.3490000000000002}]},{"osd":4,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":3.6099999999999999,"5min":3.6099999999999999,"15min":3.6099999999999999},"min":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"max":{"1min":5.5599999999999996,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":1.9610000000000001},{"interface":"front","average":{"1min":2.806,"5min":2.806,"15min":2.806},"min":{"1min":1.673,"5min":1.673,"15min":1.673},"max":{"1min":4.0839999999999996,"5min":4.0839999999999996,"15min":4.0839999999999996},"last":3.1600000000000001}]},{"osd":5,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":3.4359999999999999,"5min":3.4359999999999999,"15min":3.4359999999999999},"min":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"max":{"1min":6.1319999999999997,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":3.3220000000000001},{"interface":"front","average":{"1min":4,"5min":4,"15min":4},"min":{"1min":2.5059999999999998,"5min":2.5059999999999998,"15min":2.5059999999999998},"max":{"1min":6.3630000000000004,"5min":6.3630000000000004,"15min":6.3630000000000004},"last":2.8210000000000002}]},{"osd":6,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":3.4359999999999999,"5min":3.4359999999999999,"15min":3.4359999999999999},"min":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.3730000000000002},"max":{"1min":5.5439999999999996,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":3.3820000000000001},{"interface":"front","average":{"1min":3.621,"5min":3.621,"15min":3.621},"min":{"1min":2.786,"5min":2.786,"15min":2.786},"max":{"1min":4.6879999999999997,"5min":4.6879999999999997,"15min":4.6879999999999997},"last":3.129}]},{"osd":7,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":3.7599999999999998,"5min":3.7599999999999998,"15min":3.7599999999999998},"min":{"1min":2.8929999999999998,"5min":2.8929999999999998,"15min":2.8929999999999998},"max":{"1min":6.3220000000000001,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":2.673},{"interface":"front","average":{"1min":3.891,"5min":3.891,"15min":3.891},"min":{"1min":3.0089999999999999,"5min":3.0089999999999999,"15min":3.0089999999999999},"max":{"1min":5.4539999999999997,"5min":5.4539999999999997,"15min":5.4539999999999997},"last":3.5369999999999999}]},{"osd":8,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":4.1379999999999999,"5min":4.1379999999999999,"15min":4.1379999999999999},"min":{"1min":2.923,"5min":2.923,"15min":2.923},"max":{"1min":5.944,"5min":5.944,"15min":5.944},"last":3.2360000000000002},{"interface":"front","average":{"1min":3.907,"5min":3.907,"15min":3.907},"min":{"1min":2.1469999999999998,"5min":2.1469999999999998,"15min":2.1469999999999998},"max":{"1min":6.2400000000000002,"5min":6.2400000000000002,"15min":6.2400000000000002},"last":2.403}]},{"osd":10,"last update":"Tue Sep 8 15:35:41 2020","interfaces":[{"interface":"back","average":{"1min":4.4980000000000002,"5min":4.4980000000000002,"15min":4.4980000000000002},"min":{"1min":3.0459999999999998,"5min":3.0459999999999998,"15min":3.0459999999999998},"max":{"1min":6.673,"5min":6.673,"15min":6.673},"last":3.472},{"interface":"front","average":{"1min":4.2910000000000004,"5min":4.2910000000000004,"15min":4.2910000000000004},"min":{"1min":2.7040000000000002,"5min":2.7040000000000002,"15min":2.7040000000000002},"max":{"1min":6.6859999999999999,"5min":6.6859999999999999,"15min":6.6859999999999999},"last":2.9430000000000001}]}]},{"osd":10,"up_from":0,"seq":34,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5816,"kb_used_data":632,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739336,"statfs":{"total":95995035648,"available":95989080064,"internally_reserved":0,"allocated":647168,"data_stored":365127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7330000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":1.921,"5min":2.3119999999999998,"15min":2.3769999999999998},"min":{"1min":1.5109999999999999,"5min":1.419,"15min":1.419},"max":{"1min":2.4969999999999999,"5min":4.3380000000000001,"15min":4.3380000000000001},"last":1.734},{"interface":"front","average":{"1min":2.6440000000000001,"5min":2.5750000000000002,"15min":2.5630000000000002},"min":{"1min":1.6739999999999999,"5min":1.4750000000000001,"15min":1.4750000000000001},"max":{"1min":3.7759999999999998,"5min":5.1029999999999998,"15min":5.1029999999999998},"last":1.498}]},{"osd":2,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":2.3109999999999999,"5min":2.2599999999999998,"15min":2.2519999999999998},"min":{"1min":1.6519999999999999,"5min":1.508,"15min":1.508},"max":{"1min":3.0939999999999999,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":2.2410000000000001},{"interface":"front","average":{"1min":2.1970000000000001,"5min":2.6960000000000002,"15min":2.7789999999999999},"min":{"1min":1.492,"5min":1.4359999999999999,"15min":1.4359999999999999},"max":{"1min":3.4260000000000002,"5min":4.3769999999999998,"15min":4.3769999999999998},"last":2.1139999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":1.9159999999999999,"5min":2.2090000000000001,"15min":2.258},"min":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"max":{"1min":2.6469999999999998,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":1.5880000000000001},{"interface":"front","average":{"1min":2.7200000000000002,"5min":3.016,"15min":3.0649999999999999},"min":{"1min":1.609,"5min":1.609,"15min":1.609},"max":{"1min":3.605,"5min":4.2140000000000004,"15min":4.2140000000000004},"last":2.6800000000000002}]},{"osd":4,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.1469999999999998,"5min":3.4039999999999999,"15min":3.4470000000000001},"min":{"1min":1.8,"5min":1.8,"15min":1.8},"max":{"1min":4.0720000000000001,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":2.8500000000000001},{"interface":"front","average":{"1min":2.9780000000000002,"5min":3.0649999999999999,"15min":3.0790000000000002},"min":{"1min":1.958,"5min":1.958,"15min":1.958},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":2.3849999999999998}]},{"osd":5,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.218,"5min":3.5510000000000002,"15min":3.6070000000000002},"min":{"1min":2.653,"5min":2.427,"15min":2.427},"max":{"1min":4.1059999999999999,"5min":4.7160000000000002,"15min":4.7160000000000002},"last":3.032},{"interface":"front","average":{"1min":3.6600000000000001,"5min":3.8199999999999998,"15min":3.847},"min":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"max":{"1min":4.7089999999999996,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":2.5070000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.3300000000000001,"5min":3.5339999999999998,"15min":3.5680000000000001},"min":{"1min":2.3359999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"max":{"1min":4.7949999999999999,"5min":4.7949999999999999,"15min":4.7949999999999999},"last":3.1280000000000001},{"interface":"front","average":{"1min":3.6800000000000002,"5min":3.8620000000000001,"15min":3.8919999999999999},"min":{"1min":3.0270000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"max":{"1min":4.4939999999999998,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":3.423}]},{"osd":7,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.7400000000000002,"5min":3.907,"15min":3.9350000000000001},"min":{"1min":2.3410000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"max":{"1min":4.7220000000000004,"5min":7.0490000000000004,"15min":7.0490000000000004},"last":2.7549999999999999},{"interface":"front","average":{"1min":3.9009999999999998,"5min":4.077,"15min":4.1059999999999999},"min":{"1min":3.1789999999999998,"5min":2.806,"15min":2.806},"max":{"1min":4.8929999999999998,"5min":5.0899999999999999,"15min":5.0899999999999999},"last":3.7719999999999998}]},{"osd":8,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":4.0149999999999997,"5min":4.1459999999999999,"15min":4.1680000000000001},"min":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"max":{"1min":5.3390000000000004,"5min":5.7949999999999999,"15min":5.7949999999999999},"last":3.2069999999999999},{"interface":"front","average":{"1min":4.0670000000000002,"5min":4.0309999999999997,"15min":4.0250000000000004},"min":{"1min":2.6920000000000002,"5min":1.901,"15min":1.901},"max":{"1min":5.1200000000000001,"5min":5.6509999999999998,"15min":5.6509999999999998},"last":3.5}]},{"osd":9,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":4.2160000000000002,"5min":4.3780000000000001,"15min":4.4050000000000002},"min":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"max":{"1min":5.4020000000000001,"5min":5.8369999999999997,"15min":5.8369999999999997},"last":3.5899999999999999},{"interface":"front","average":{"1min":4.2629999999999999,"5min":4.2149999999999999,"15min":4.2069999999999999},"min":{"1min":2.5590000000000002,"5min":2.4969999999999999,"15min":2.4969999999999999},"max":{"1min":5.3129999999999997,"5min":5.3129999999999997,"15min":5.3129999999999997},"last":4.3719999999999999}]},{"osd":11,"last update":"Tue Sep 8 15:35:43 2020","interfaces":[{"interface":"back","average":{"1min":4.3890000000000002,"5min":4.3890000000000002,"15min":4.3890000000000002},"min":{"1min":1.831,"5min":1.831,"15min":1.831},"max":{"1min":5.6980000000000004,"5min":5.6980000000000004,"15min":5.6980000000000004},"last":4.0670000000000002},{"interface":"front","average":{"1min":4.423,"5min":4.423,"15min":4.423},"min":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"max":{"1min":5.4740000000000002,"5min":5.4740000000000002,"15min":5.4740000000000002},"last":3.8860000000000001}]}]},{"osd":9,"up_from":0,"seq":37,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5944,"kb_used_data":632,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739208,"statfs":{"total":95995035648,"available":95988948992,"internally_reserved":0,"allocated":647168,"data_stored":365127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":2.129,"5min":1.9119999999999999,"15min":1.8759999999999999},"min":{"1min":1.4390000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"max":{"1min":3.1259999999999999,"5min":3.149,"15min":3.149},"last":2.6600000000000001},{"interface":"front","average":{"1min":1.9490000000000001,"5min":1.9119999999999999,"15min":1.9059999999999999},"min":{"1min":1.3580000000000001,"5min":1.2549999999999999,"15min":1.2549999999999999},"max":{"1min":4.149,"5min":4.149,"15min":4.149},"last":2.5579999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":2.173,"5min":1.9930000000000001,"15min":1.9630000000000001},"min":{"1min":1.3120000000000001,"5min":1.3009999999999999,"15min":1.3009999999999999},"max":{"1min":3.6269999999999998,"5min":3.6269999999999998,"15min":3.6269999999999998},"last":2.4510000000000001},{"interface":"front","average":{"1min":2.4889999999999999,"5min":2.3250000000000002,"15min":2.298},"min":{"1min":1.474,"5min":1.266,"15min":1.266},"max":{"1min":3.9950000000000001,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":3.3599999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":2.7210000000000001,"5min":2.5219999999999998,"15min":2.4889999999999999},"min":{"1min":1.4359999999999999,"5min":1.327,"15min":1.327},"max":{"1min":4.1020000000000003,"5min":4.508,"15min":4.508},"last":4.1710000000000003},{"interface":"front","average":{"1min":2.5489999999999999,"5min":2.3330000000000002,"15min":2.2970000000000002},"min":{"1min":1.4259999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"max":{"1min":3.6800000000000002,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":3.5379999999999998}]},{"osd":3,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":2.395,"5min":2.351,"15min":2.343},"min":{"1min":1.3420000000000001,"5min":1.198,"15min":1.198},"max":{"1min":4.3760000000000003,"5min":4.3760000000000003,"15min":4.3760000000000003},"last":3.8359999999999999},{"interface":"front","average":{"1min":2.629,"5min":2.641,"15min":2.6429999999999998},"min":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"max":{"1min":4.4550000000000001,"5min":4.4550000000000001,"15min":4.4550000000000001},"last":4.4980000000000002}]},{"osd":4,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":3.0470000000000002,"5min":2.7570000000000001,"15min":2.7090000000000001},"min":{"1min":1.905,"5min":1.1819999999999999,"15min":1.1819999999999999},"max":{"1min":5.0279999999999996,"5min":5.0279999999999996,"15min":5.0279999999999996},"last":4.6109999999999998},{"interface":"front","average":{"1min":3.5390000000000001,"5min":3.1040000000000001,"15min":3.032},"min":{"1min":2.5750000000000002,"5min":1.883,"15min":1.883},"max":{"1min":4.6280000000000001,"5min":4.6280000000000001,"15min":4.6280000000000001},"last":4.306}]},{"osd":5,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":3.452,"5min":3.2149999999999999,"15min":3.1749999999999998},"min":{"1min":2.6259999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"max":{"1min":5.0519999999999996,"5min":6.3840000000000003,"15min":6.3840000000000003},"last":4.9349999999999996},{"interface":"front","average":{"1min":3.456,"5min":3.2240000000000002,"15min":3.1859999999999999},"min":{"1min":1.907,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":4.875,"5min":4.875,"15min":4.875},"last":4.6600000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":3.7589999999999999,"5min":3.3109999999999999,"15min":3.2370000000000001},"min":{"1min":2.3820000000000001,"5min":1.4179999999999999,"15min":1.4179999999999999},"max":{"1min":5.3659999999999997,"5min":5.3659999999999997,"15min":5.3659999999999997},"last":4.7309999999999999},{"interface":"front","average":{"1min":3.621,"5min":3.3090000000000002,"15min":3.2570000000000001},"min":{"1min":2.1720000000000002,"5min":1.8959999999999999,"15min":1.8959999999999999},"max":{"1min":5.7130000000000001,"5min":5.7130000000000001,"15min":5.7130000000000001},"last":4.0289999999999999}]},{"osd":7,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":3.9399999999999999,"5min":3.5,"15min":3.427},"min":{"1min":1.968,"5min":1.7949999999999999,"15min":1.7949999999999999},"max":{"1min":5.1459999999999999,"5min":5.1459999999999999,"15min":5.1459999999999999},"last":5.367},{"interface":"front","average":{"1min":4.0599999999999996,"5min":3.7000000000000002,"15min":3.6400000000000001},"min":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"max":{"1min":5.7629999999999999,"5min":5.7629999999999999,"15min":5.7629999999999999},"last":4.9029999999999996}]},{"osd":8,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":3.7400000000000002,"5min":3.278,"15min":3.2010000000000001},"min":{"1min":2.2879999999999998,"5min":1.3180000000000001,"15min":1.3180000000000001},"max":{"1min":4.9119999999999999,"5min":5.3049999999999997,"15min":5.3049999999999997},"last":4.8319999999999999},{"interface":"front","average":{"1min":4.0499999999999998,"5min":3.3969999999999998,"15min":3.2879999999999998},"min":{"1min":2.335,"5min":1.444,"15min":1.444},"max":{"1min":5.4199999999999999,"5min":6.0140000000000002,"15min":6.0140000000000002},"last":5.0919999999999996}]},{"osd":10,"last update":"Tue Sep 8 15:36:18 2020","interfaces":[{"interface":"back","average":{"1min":3.9500000000000002,"5min":3.9350000000000001,"15min":3.9329999999999998},"min":{"1min":3.3839999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"max":{"1min":4.8780000000000001,"5min":5.5629999999999997,"15min":5.5629999999999997},"last":5.1630000000000003},{"interface":"front","average":{"1min":3.9649999999999999,"5min":3.8239999999999998,"15min":3.7999999999999998},"min":{"1min":3.3130000000000002,"5min":2.056,"15min":2.056},"max":{"1min":4.6379999999999999,"5min":5.492,"15min":5.492},"last":4.3810000000000002}]}]},{"osd":8,"up_from":0,"seq":44,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6008,"kb_used_data":632,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":93739144,"statfs":{"total":95995035648,"available":95988883456,"internally_reserved":0,"allocated":647168,"data_stored":365127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.2749999999999999,"5min":2.202,"15min":2.1920000000000002},"min":{"1min":1.3799999999999999,"5min":1.266,"15min":1.266},"max":{"1min":3.2200000000000002,"5min":3.5830000000000002,"15min":3.5830000000000002},"last":3.0009999999999999},{"interface":"front","average":{"1min":2.5459999999999998,"5min":2.5070000000000001,"15min":2.4300000000000002},"min":{"1min":1.7669999999999999,"5min":1.7250000000000001,"15min":1.7250000000000001},"max":{"1min":3.444,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":2.8919999999999999}]},{"osd":1,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.492,"5min":2.0659999999999998,"15min":1.8839999999999999},"min":{"1min":1.4690000000000001,"5min":1.137,"15min":1.137},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":2.2930000000000001},{"interface":"front","average":{"1min":2.7200000000000002,"5min":2.427,"15min":2.347},"min":{"1min":1.351,"5min":1.046,"15min":1.046},"max":{"1min":4.444,"5min":4.444,"15min":4.444},"last":1.675}]},{"osd":2,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.2330000000000001,"5min":1.9410000000000001,"15min":1.845},"min":{"1min":1.4379999999999999,"5min":1.27,"15min":1.27},"max":{"1min":3.9129999999999998,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":3.1099999999999999},{"interface":"front","average":{"1min":2.4889999999999999,"5min":2.504,"15min":2.46},"min":{"1min":1.208,"5min":1.208,"15min":1.208},"max":{"1min":3.7509999999999999,"5min":3.895,"15min":3.895},"last":2.2040000000000002}]},{"osd":3,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.9750000000000001,"5min":2.7869999999999999,"15min":2.7370000000000001},"min":{"1min":1.587,"5min":1.2789999999999999,"15min":1.2789999999999999},"max":{"1min":4.3929999999999998,"5min":4.3929999999999998,"15min":4.3929999999999998},"last":2.7519999999999998},{"interface":"front","average":{"1min":2.4649999999999999,"5min":2.4260000000000002,"15min":2.3820000000000001},"min":{"1min":1.131,"5min":1.131,"15min":1.131},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.569}]},{"osd":4,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":3.149,"5min":2.9870000000000001,"15min":2.875},"min":{"1min":2.3029999999999999,"5min":1.4730000000000001,"15min":1.4730000000000001},"max":{"1min":4.4669999999999996,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":2.5920000000000001},{"interface":"front","average":{"1min":2.8820000000000001,"5min":2.6869999999999998,"15min":2.5579999999999998},"min":{"1min":1.734,"5min":1.2549999999999999,"15min":1.2549999999999999},"max":{"1min":4.2649999999999997,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":1.9390000000000001}]},{"osd":5,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":3.5529999999999999,"5min":3.294,"15min":3.1619999999999999},"min":{"1min":1.944,"5min":1.74,"15min":1.74},"max":{"1min":5.6109999999999998,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":3.3519999999999999},{"interface":"front","average":{"1min":3.7360000000000002,"5min":3.419,"15min":3.3719999999999999},"min":{"1min":2.2080000000000002,"5min":1.762,"15min":1.762},"max":{"1min":6.0549999999999997,"5min":6.0549999999999997,"15min":6.0549999999999997},"last":3.653}]},{"osd":6,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":3.8090000000000002,"5min":3.4199999999999999,"15min":3.2639999999999998},"min":{"1min":2.4980000000000002,"5min":1.9299999999999999,"15min":1.9299999999999999},"max":{"1min":5.1929999999999996,"5min":5.6180000000000003,"15min":5.6180000000000003},"last":3.2160000000000002},{"interface":"front","average":{"1min":3.2839999999999998,"5min":3.0499999999999998,"15min":2.9220000000000002},"min":{"1min":2.0019999999999998,"5min":1.669,"15min":1.669},"max":{"1min":4.7279999999999998,"5min":4.7279999999999998,"15min":4.7279999999999998},"last":2.395}]},{"osd":7,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":4.0419999999999998,"5min":3.7269999999999999,"15min":3.6240000000000001},"min":{"1min":2.3849999999999998,"5min":2.0630000000000002,"15min":2.0630000000000002},"max":{"1min":5.3680000000000003,"5min":5.3680000000000003,"15min":5.3680000000000003},"last":3.4380000000000002},{"interface":"front","average":{"1min":3.6429999999999998,"5min":3.3679999999999999,"15min":3.2440000000000002},"min":{"1min":2.7269999999999999,"5min":2.0830000000000002,"15min":2.0830000000000002},"max":{"1min":5.1029999999999998,"5min":5.141,"15min":5.141},"last":3.391}]},{"osd":9,"last update":"Tue Sep 8 15:35:56 2020","interfaces":[{"interface":"back","average":{"1min":3.9590000000000001,"5min":3.4990000000000001,"15min":3.4220000000000002},"min":{"1min":2.4180000000000001,"5min":1.157,"15min":1.157},"max":{"1min":5.1840000000000002,"5min":5.1840000000000002,"15min":5.1840000000000002},"last":3.5419999999999998},{"interface":"front","average":{"1min":4.1639999999999997,"5min":3.4660000000000002,"15min":3.3500000000000001},"min":{"1min":2.0219999999999998,"5min":1.1919999999999999,"15min":1.1919999999999999},"max":{"1min":5.3959999999999999,"5min":5.3959999999999999,"15min":5.3959999999999999},"last":3.843}]},{"osd":10,"last update":"Tue Sep 8 15:36:27 2020","interfaces":[{"interface":"back","average":{"1min":3.7200000000000002,"5min":3.7320000000000002,"15min":3.734},"min":{"1min":2.0510000000000002,"5min":1.3740000000000001,"15min":1.3740000000000001},"max":{"1min":5.5190000000000001,"5min":5.5190000000000001,"15min":5.5190000000000001},"last":3.1680000000000001},{"interface":"front","average":{"1min":4.1180000000000003,"5min":4.0830000000000002,"15min":4.077},"min":{"1min":2.7669999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"max":{"1min":5.3049999999999997,"5min":5.3049999999999997,"15min":5.3049999999999997},"last":3.73}]}]},{"osd":7,"up_from":0,"seq":49,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6072,"kb_used_data":632,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93739080,"statfs":{"total":95995035648,"available":95988817920,"internally_reserved":0,"allocated":647168,"data_stored":365147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:02 2020","interfaces":[{"interface":"back","average":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"min":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"max":{"1min":4.0190000000000001,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":2.2189999999999999},{"interface":"front","average":{"1min":2.6259999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"min":{"1min":1.542,"5min":1.542,"15min":1.542},"max":{"1min":4.5229999999999997,"5min":4.5229999999999997,"15min":4.5229999999999997},"last":2.3079999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.661,"5min":2.468,"15min":2.355},"min":{"1min":2.1309999999999998,"5min":1.498,"15min":1.498},"max":{"1min":3.3940000000000001,"5min":3.8780000000000001,"15min":3.8780000000000001},"last":2.0110000000000001},{"interface":"front","average":{"1min":2.1699999999999999,"5min":2.359,"15min":2.3599999999999999},"min":{"1min":1.7749999999999999,"5min":1.631,"15min":1.631},"max":{"1min":2.6800000000000002,"5min":3.633,"15min":3.633},"last":2.0910000000000002}]},{"osd":2,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.4049999999999998,"5min":2.528,"15min":2.4399999999999999},"min":{"1min":1.899,"5min":1.504,"15min":1.504},"max":{"1min":3.2240000000000002,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":2.1840000000000002},{"interface":"front","average":{"1min":2.468,"5min":2.879,"15min":2.8370000000000002},"min":{"1min":1.982,"5min":1.982,"15min":1.982},"max":{"1min":3.2570000000000001,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":2.395}]},{"osd":3,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.7869999999999999,"5min":2.9849999999999999,"15min":2.891},"min":{"1min":1.861,"5min":1.861,"15min":1.861},"max":{"1min":3.8460000000000001,"5min":4.875,"15min":4.875},"last":1.9179999999999999},{"interface":"front","average":{"1min":2.8159999999999998,"5min":3.254,"15min":3.234},"min":{"1min":1.978,"5min":1.5660000000000001,"15min":1.5660000000000001},"max":{"1min":3.8330000000000002,"5min":5.3929999999999998,"15min":5.3929999999999998},"last":3.4319999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":3.0150000000000001,"5min":3.0289999999999999,"15min":2.9329999999999998},"min":{"1min":2.181,"5min":1.9279999999999999,"15min":1.9279999999999999},"max":{"1min":3.98,"5min":5.9729999999999999,"15min":5.9729999999999999},"last":2.6379999999999999},{"interface":"front","average":{"1min":3.3780000000000001,"5min":3.456,"15min":3.3100000000000001},"min":{"1min":2.5150000000000001,"5min":2.464,"15min":2.464},"max":{"1min":4.4260000000000002,"5min":5.6470000000000002,"15min":5.6470000000000002},"last":3.2669999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":3.4790000000000001,"5min":3.7400000000000002,"15min":3.6859999999999999},"min":{"1min":2.794,"5min":2.794,"15min":2.794},"max":{"1min":4.593,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":2.9750000000000001},{"interface":"front","average":{"1min":3.3769999999999998,"5min":3.5590000000000002,"15min":3.5129999999999999},"min":{"1min":2.7080000000000002,"5min":2.0859999999999999,"15min":2.0859999999999999},"max":{"1min":4.4779999999999998,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":2.8450000000000002}]},{"osd":6,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":3.5529999999999999,"5min":3.9319999999999999,"15min":3.9369999999999998},"min":{"1min":2.7309999999999999,"5min":2.6320000000000001,"15min":2.6320000000000001},"max":{"1min":4.7599999999999998,"5min":7.3070000000000004,"15min":7.3070000000000004},"last":3.1190000000000002},{"interface":"front","average":{"1min":3.9039999999999999,"5min":4.0140000000000002,"15min":3.9180000000000001},"min":{"1min":3.1349999999999998,"5min":2.6549999999999998,"15min":2.6549999999999998},"max":{"1min":4.6470000000000002,"5min":7.2480000000000002,"15min":7.2480000000000002},"last":3.5070000000000001}]},{"osd":8,"last update":"Tue Sep 8 15:36:33 2020","interfaces":[{"interface":"back","average":{"1min":4.1619999999999999,"5min":4.2220000000000004,"15min":4.1609999999999996},"min":{"1min":2.7250000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"max":{"1min":5.5300000000000002,"5min":6.9530000000000003,"15min":6.9530000000000003},"last":3.9500000000000002},{"interface":"front","average":{"1min":4.444,"5min":4.4059999999999997,"15min":4.2830000000000004},"min":{"1min":3.5579999999999998,"5min":2.8479999999999999,"15min":2.8479999999999999},"max":{"1min":5.6929999999999996,"5min":7.6509999999999998,"15min":7.6509999999999998},"last":4.3959999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:35:55 2020","interfaces":[{"interface":"back","average":{"1min":4.8460000000000001,"5min":4.7880000000000003,"15min":4.7779999999999996},"min":{"1min":3.7810000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"max":{"1min":6.524,"5min":6.524,"15min":6.524},"last":3.9969999999999999},{"interface":"front","average":{"1min":4.2880000000000003,"5min":4.4219999999999997,"15min":4.444},"min":{"1min":3.1989999999999998,"5min":3.1989999999999998,"15min":3.1989999999999998},"max":{"1min":6.274,"5min":6.274,"15min":6.274},"last":3.7000000000000002}]},{"osd":10,"last update":"Tue Sep 8 15:36:22 2020","interfaces":[{"interface":"back","average":{"1min":4.2460000000000004,"5min":4.649,"15min":4.7160000000000002},"min":{"1min":3.371,"5min":3.218,"15min":3.218},"max":{"1min":5.3390000000000004,"5min":7.7800000000000002,"15min":7.7800000000000002},"last":3.3540000000000001},{"interface":"front","average":{"1min":4.8840000000000003,"5min":5.1219999999999999,"15min":5.1619999999999999},"min":{"1min":3.7410000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"max":{"1min":5.8760000000000003,"5min":8.3650000000000002,"15min":8.3650000000000002},"last":3.75}]},{"osd":11,"last update":"Tue Sep 8 15:35:44 2020","interfaces":[{"interface":"back","average":{"1min":4.8380000000000001,"5min":4.8380000000000001,"15min":4.8380000000000001},"min":{"1min":3.5430000000000001,"5min":3.5430000000000001,"15min":3.5430000000000001},"max":{"1min":6.1589999999999998,"5min":6.1589999999999998,"15min":6.1589999999999998},"last":3.6419999999999999},{"interface":"front","average":{"1min":5.3170000000000002,"5min":5.3170000000000002,"15min":5.3170000000000002},"min":{"1min":3.9660000000000002,"5min":3.9660000000000002,"15min":3.9660000000000002},"max":{"1min":6.742,"5min":6.742,"15min":6.742},"last":4.2089999999999996}]}]},{"osd":6,"up_from":0,"seq":53,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6028,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93739124,"statfs":{"total":95995035648,"available":95988862976,"internally_reserved":0,"allocated":659456,"data_stored":372354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:35:47 2020","interfaces":[{"interface":"back","average":{"1min":2.5550000000000002,"5min":2.044,"15min":1.8999999999999999},"min":{"1min":1.841,"5min":1.141,"15min":1.141},"max":{"1min":3.714,"5min":3.714,"15min":3.714},"last":3.1709999999999998},{"interface":"front","average":{"1min":2.2360000000000002,"5min":2.0529999999999999,"15min":1.9670000000000001},"min":{"1min":1.6839999999999999,"5min":1.256,"15min":1.256},"max":{"1min":2.6899999999999999,"5min":3.3279999999999998,"15min":3.3279999999999998},"last":1.9279999999999999}]},{"osd":1,"last update":"Tue Sep 8 15:35:47 2020","interfaces":[{"interface":"back","average":{"1min":2.242,"5min":2.181,"15min":2.177},"min":{"1min":1.6319999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"max":{"1min":2.7759999999999998,"5min":3.141,"15min":3.141},"last":1.7350000000000001},{"interface":"front","average":{"1min":2.9750000000000001,"5min":2.5779999999999998,"15min":2.4359999999999999},"min":{"1min":1.8,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":4.0940000000000003,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":2.569}]},{"osd":2,"last update":"Tue Sep 8 15:35:47 2020","interfaces":[{"interface":"back","average":{"1min":3.085,"5min":2.5840000000000001,"15min":2.4220000000000002},"min":{"1min":1.794,"5min":1.462,"15min":1.462},"max":{"1min":4.2750000000000004,"5min":4.2750000000000004,"15min":4.2750000000000004},"last":1.819},{"interface":"front","average":{"1min":3.0910000000000002,"5min":2.5640000000000001,"15min":2.3860000000000001},"min":{"1min":2.0710000000000002,"5min":1.4790000000000001,"15min":1.4790000000000001},"max":{"1min":3.9020000000000001,"5min":4.0970000000000004,"15min":4.0970000000000004},"last":2.911}]},{"osd":3,"last update":"Tue Sep 8 15:35:47 2020","interfaces":[{"interface":"back","average":{"1min":3.3029999999999999,"5min":2.6589999999999998,"15min":2.4399999999999999},"min":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"max":{"1min":4.8109999999999999,"5min":4.8109999999999999,"15min":4.8109999999999999},"last":3.028},{"interface":"front","average":{"1min":3.6949999999999998,"5min":3.0899999999999999,"15min":2.863},"min":{"1min":1.9019999999999999,"5min":1.7030000000000001,"15min":1.7030000000000001},"max":{"1min":4.8120000000000003,"5min":4.8120000000000003,"15min":4.8120000000000003},"last":3.4399999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:35:47 2020","interfaces":[{"interface":"back","average":{"1min":3.621,"5min":2.919,"15min":2.6629999999999998},"min":{"1min":2.8079999999999998,"5min":1.607,"15min":1.607},"max":{"1min":4.6550000000000002,"5min":4.702,"15min":4.702},"last":3.2429999999999999},{"interface":"front","average":{"1min":3.6179999999999999,"5min":3.085,"15min":2.8969999999999998},"min":{"1min":2.6110000000000002,"5min":1.5049999999999999,"15min":1.5049999999999999},"max":{"1min":4.8769999999999998,"5min":4.8769999999999998,"15min":4.8769999999999998},"last":3.3799999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:35:47 2020","interfaces":[{"interface":"back","average":{"1min":3.9649999999999999,"5min":3.3439999999999999,"15min":3.1389999999999998},"min":{"1min":3.0579999999999998,"5min":1.7509999999999999,"15min":1.7509999999999999},"max":{"1min":5.3070000000000004,"5min":6.1269999999999998,"15min":6.1269999999999998},"last":3.6099999999999999},{"interface":"front","average":{"1min":4.0549999999999997,"5min":3.3849999999999998,"15min":3.1549999999999998},"min":{"1min":2.919,"5min":2.1659999999999999,"15min":2.1659999999999999},"max":{"1min":5.8200000000000003,"5min":6.1820000000000004,"15min":6.1820000000000004},"last":3.5030000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":4.0549999999999997,"5min":3.8460000000000001,"15min":3.714},"min":{"1min":3.2879999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"max":{"1min":4.8970000000000002,"5min":5.4050000000000002,"15min":5.4050000000000002},"last":3.677},{"interface":"front","average":{"1min":4.5359999999999996,"5min":4.0469999999999997,"15min":3.8090000000000002},"min":{"1min":3.387,"5min":2.1640000000000001,"15min":2.1640000000000001},"max":{"1min":5.5609999999999999,"5min":6.2679999999999998,"15min":6.2679999999999998},"last":3.7160000000000002}]},{"osd":8,"last update":"Tue Sep 8 15:36:30 2020","interfaces":[{"interface":"back","average":{"1min":4.3600000000000003,"5min":4.0759999999999996,"15min":3.9500000000000002},"min":{"1min":3.258,"5min":3.0579999999999998,"15min":3.0579999999999998},"max":{"1min":5.6639999999999997,"5min":5.6639999999999997,"15min":5.6639999999999997},"last":3.8260000000000001},{"interface":"front","average":{"1min":4.2759999999999998,"5min":4.2309999999999999,"15min":4.1929999999999996},"min":{"1min":3.5259999999999998,"5min":3.1269999999999998,"15min":3.1269999999999998},"max":{"1min":5.3819999999999997,"5min":5.5499999999999998,"15min":5.5499999999999998},"last":3.8740000000000001}]},{"osd":9,"last update":"Tue Sep 8 15:35:54 2020","interfaces":[{"interface":"back","average":{"1min":4.7919999999999998,"5min":4.4960000000000004,"15min":4.4470000000000001},"min":{"1min":3.4750000000000001,"5min":3.1110000000000002,"15min":3.1110000000000002},"max":{"1min":6.3170000000000002,"5min":6.3170000000000002,"15min":6.3170000000000002},"last":4.218},{"interface":"front","average":{"1min":4.5060000000000002,"5min":4.5179999999999998,"15min":4.5199999999999996},"min":{"1min":3.6659999999999999,"5min":3.5649999999999999,"15min":3.5649999999999999},"max":{"1min":5.3540000000000001,"5min":6.0979999999999999,"15min":6.0979999999999999},"last":3.9470000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":4.6909999999999998,"5min":4.694,"15min":4.694},"min":{"1min":3.7730000000000001,"5min":3.4590000000000001,"15min":3.4590000000000001},"max":{"1min":6.0720000000000001,"5min":6.0720000000000001,"15min":6.0720000000000001},"last":4.0199999999999996},{"interface":"front","average":{"1min":4.6159999999999997,"5min":4.6260000000000003,"15min":4.6280000000000001},"min":{"1min":3.4660000000000002,"5min":3.4159999999999999,"15min":3.4159999999999999},"max":{"1min":5.6980000000000004,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":4.1479999999999997}]}]},{"osd":1,"up_from":0,"seq":78,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6328,"kb_used_data":632,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738824,"statfs":{"total":95995035648,"available":95988555776,"internally_reserved":0,"allocated":647168,"data_stored":365147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:35:39 2020","interfaces":[{"interface":"back","average":{"1min":2.194,"5min":1.758,"15min":1.385},"min":{"1min":1.073,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":3.1040000000000001,"5min":3.363,"15min":3.363},"last":3.3319999999999999},{"interface":"front","average":{"1min":2.4300000000000002,"5min":2.0089999999999999,"15min":1.536},"min":{"1min":1.1399999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":3.403,"5min":3.698,"15min":3.698},"last":2.6899999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:36:04 2020","interfaces":[{"interface":"back","average":{"1min":2.5539999999999998,"5min":2.153,"15min":1.712},"min":{"1min":1.3460000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":4.4080000000000004,"5min":4.4080000000000004,"15min":4.4080000000000004},"last":2.101},{"interface":"front","average":{"1min":2.7290000000000001,"5min":2.2770000000000001,"15min":1.8680000000000001},"min":{"1min":1.431,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":4.6559999999999997,"5min":4.6559999999999997,"15min":4.6559999999999997},"last":2.5819999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:36:34 2020","interfaces":[{"interface":"back","average":{"1min":2.7509999999999999,"5min":2.5230000000000001,"15min":2.2770000000000001},"min":{"1min":0.99299999999999999,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":4.0469999999999997,"5min":4.5119999999999996,"15min":4.5119999999999996},"last":2.5070000000000001},{"interface":"front","average":{"1min":2.593,"5min":2.548,"15min":2.4079999999999999},"min":{"1min":1.746,"5min":1.2410000000000001,"15min":1.2410000000000001},"max":{"1min":4.2069999999999999,"5min":4.2069999999999999,"15min":4.2069999999999999},"last":2.996}]},{"osd":4,"last update":"Tue Sep 8 15:35:51 2020","interfaces":[{"interface":"back","average":{"1min":2.1000000000000001,"5min":2.2829999999999999,"15min":2.1160000000000001},"min":{"1min":1.4670000000000001,"5min":1.387,"15min":1.387},"max":{"1min":3.008,"5min":5.1829999999999998,"15min":5.1829999999999998},"last":2.052},{"interface":"front","average":{"1min":3.29,"5min":2.903,"15min":2.7549999999999999},"min":{"1min":2.028,"5min":1.706,"15min":1.706},"max":{"1min":4.8010000000000002,"5min":4.8010000000000002,"15min":4.8010000000000002},"last":4.1319999999999997}]},{"osd":5,"last update":"Tue Sep 8 15:36:14 2020","interfaces":[{"interface":"back","average":{"1min":3.0859999999999999,"5min":3.089,"15min":2.9449999999999998},"min":{"1min":2.105,"5min":1.8879999999999999,"15min":1.8879999999999999},"max":{"1min":4.6440000000000001,"5min":4.6440000000000001,"15min":4.6440000000000001},"last":5.2510000000000003},{"interface":"front","average":{"1min":2.6019999999999999,"5min":2.5470000000000002,"15min":2.4409999999999998},"min":{"1min":1.9410000000000001,"5min":1.5189999999999999,"15min":1.5189999999999999},"max":{"1min":3.532,"5min":4.6319999999999997,"15min":4.6319999999999997},"last":2.3900000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:36:40 2020","interfaces":[{"interface":"back","average":{"1min":3.9529999999999998,"5min":3.7440000000000002,"15min":3.621},"min":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"max":{"1min":5.367,"5min":6.1399999999999997,"15min":6.1399999999999997},"last":4.8970000000000002},{"interface":"front","average":{"1min":3.8079999999999998,"5min":3.6299999999999999,"15min":3.5609999999999999},"min":{"1min":2.5640000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"max":{"1min":5.7069999999999999,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":5.7069999999999999}]},{"osd":7,"last update":"Tue Sep 8 15:36:09 2020","interfaces":[{"interface":"back","average":{"1min":3.9950000000000001,"5min":3.4540000000000002,"15min":3.2250000000000001},"min":{"1min":2.8860000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"max":{"1min":5.3239999999999998,"5min":5.3239999999999998,"15min":5.3239999999999998},"last":6.0780000000000003},{"interface":"front","average":{"1min":3.9510000000000001,"5min":3.9100000000000001,"15min":3.883},"min":{"1min":1.907,"5min":1.907,"15min":1.907},"max":{"1min":5.375,"5min":5.5650000000000004,"15min":5.5650000000000004},"last":5.8479999999999999}]},{"osd":8,"last update":"Tue Sep 8 15:36:34 2020","interfaces":[{"interface":"back","average":{"1min":4.1890000000000001,"5min":4.2190000000000003,"15min":4.2930000000000001},"min":{"1min":2.6419999999999999,"5min":1.7290000000000001,"15min":1.7290000000000001},"max":{"1min":5.6340000000000003,"5min":5.6340000000000003,"15min":5.6340000000000003},"last":6.9720000000000004},{"interface":"front","average":{"1min":4.2210000000000001,"5min":4.0700000000000003,"15min":4.0800000000000001},"min":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"max":{"1min":6.0960000000000001,"5min":6.0960000000000001,"15min":6.0960000000000001},"last":4.5250000000000004}]},{"osd":9,"last update":"Tue Sep 8 15:35:58 2020","interfaces":[{"interface":"back","average":{"1min":3.5009999999999999,"5min":3.2919999999999998,"15min":3.2570000000000001},"min":{"1min":2.423,"5min":2.089,"15min":2.089},"max":{"1min":4.3250000000000002,"5min":4.8789999999999996,"15min":4.8789999999999996},"last":3.335},{"interface":"front","average":{"1min":4.3659999999999997,"5min":4.2560000000000002,"15min":4.2380000000000004},"min":{"1min":3.3410000000000002,"5min":2.649,"15min":2.649},"max":{"1min":6.7380000000000004,"5min":6.7380000000000004,"15min":6.7380000000000004},"last":5.3810000000000002}]},{"osd":10,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":4.3220000000000001,"5min":4.2320000000000002,"15min":4.2169999999999996},"min":{"1min":2.7949999999999999,"5min":2.3210000000000002,"15min":2.3210000000000002},"max":{"1min":5.968,"5min":5.968,"15min":5.968},"last":7.46},{"interface":"front","average":{"1min":3.4620000000000002,"5min":3.4889999999999999,"15min":3.4929999999999999},"min":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"max":{"1min":4.9640000000000004,"5min":4.9790000000000001,"15min":4.9790000000000001},"last":3.1560000000000001}]}]},{"osd":0,"up_from":0,"seq":82,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6456,"kb_used_data":632,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738696,"statfs":{"total":95995035648,"available":95988424704,"internally_reserved":0,"allocated":647168,"data_stored":365147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 8 15:36:42 2020","interfaces":[{"interface":"back","average":{"1min":2.407,"5min":1.9450000000000001,"15min":1.4330000000000001},"min":{"1min":1.6279999999999999,"5min":1.02,"15min":0.76100000000000001},"max":{"1min":4.3339999999999996,"5min":4.3339999999999996,"15min":4.3339999999999996},"last":3.5539999999999998},{"interface":"front","average":{"1min":2.121,"5min":1.9319999999999999,"15min":1.46},"min":{"1min":1.6619999999999999,"5min":1.2050000000000001,"15min":0.71999999999999997},"max":{"1min":2.71,"5min":3.8300000000000001,"15min":3.8300000000000001},"last":2.383}]},{"osd":2,"last update":"Tue Sep 8 15:34:05 2020","interfaces":[{"interface":"back","average":{"1min":1.9950000000000001,"5min":1.621,"15min":1.5189999999999999},"min":{"1min":1.3999999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.8519999999999999,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":2.6589999999999998},{"interface":"front","average":{"1min":2.629,"5min":1.956,"15min":1.72},"min":{"1min":1.6419999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":2.4049999999999998}]},{"osd":3,"last update":"Tue Sep 8 15:36:32 2020","interfaces":[{"interface":"back","average":{"1min":2.6760000000000002,"5min":2.617,"15min":2.992},"min":{"1min":1.8460000000000001,"5min":1.1879999999999999,"15min":1.1879999999999999},"max":{"1min":4.3140000000000001,"5min":27.298999999999999,"15min":27.298999999999999},"last":2.3239999999999998},{"interface":"front","average":{"1min":3.4289999999999998,"5min":3.3639999999999999,"15min":3.6219999999999999},"min":{"1min":2.2069999999999999,"5min":2.0179999999999998,"15min":2.0179999999999998},"max":{"1min":5.2590000000000003,"5min":28.314,"15min":28.314},"last":5.1420000000000003}]},{"osd":4,"last update":"Tue Sep 8 15:35:53 2020","interfaces":[{"interface":"back","average":{"1min":3.0139999999999998,"5min":2.8250000000000002,"15min":2.4809999999999999},"min":{"1min":2.1240000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"max":{"1min":3.8199999999999998,"5min":4.4390000000000001,"15min":4.4390000000000001},"last":4.0129999999999999},{"interface":"front","average":{"1min":3.3740000000000001,"5min":3.0680000000000001,"15min":2.7549999999999999},"min":{"1min":2.5139999999999998,"5min":1.8080000000000001,"15min":1.8080000000000001},"max":{"1min":4.8129999999999997,"5min":5.1050000000000004,"15min":5.1050000000000004},"last":4.7800000000000002}]},{"osd":5,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":3.8079999999999998,"5min":3.3399999999999999,"15min":3.0089999999999999},"min":{"1min":2.6400000000000001,"5min":2.0990000000000002,"15min":2.0990000000000002},"max":{"1min":4.9809999999999999,"5min":4.9809999999999999,"15min":4.9809999999999999},"last":4.4089999999999998},{"interface":"front","average":{"1min":2.3839999999999999,"5min":3.0259999999999998,"15min":3.1840000000000002},"min":{"1min":1.845,"5min":1.7110000000000001,"15min":1.7110000000000001},"max":{"1min":3.0459999999999998,"5min":5.7439999999999998,"15min":5.7439999999999998},"last":3.5110000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:36:42 2020","interfaces":[{"interface":"back","average":{"1min":2.8679999999999999,"5min":3.2770000000000001,"15min":3.5209999999999999},"min":{"1min":1.746,"5min":1.6439999999999999,"15min":1.6439999999999999},"max":{"1min":5.8730000000000002,"5min":5.8730000000000002,"15min":5.8730000000000002},"last":2.4359999999999999},{"interface":"front","average":{"1min":3.1899999999999999,"5min":3.6749999999999998,"15min":3.923},"min":{"1min":2.5350000000000001,"5min":1.9099999999999999,"15min":1.9099999999999999},"max":{"1min":6.2560000000000002,"5min":6.2560000000000002,"15min":6.2560000000000002},"last":3.109}]},{"osd":7,"last update":"Tue Sep 8 15:36:08 2020","interfaces":[{"interface":"back","average":{"1min":3.2669999999999999,"5min":4.069,"15min":4.3120000000000003},"min":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"max":{"1min":4.758,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":4.234},{"interface":"front","average":{"1min":3.9319999999999999,"5min":4.0860000000000003,"15min":4.1040000000000001},"min":{"1min":2.585,"5min":2.585,"15min":2.585},"max":{"1min":5.6429999999999998,"5min":5.8659999999999997,"15min":5.8659999999999997},"last":5.343}]},{"osd":8,"last update":"Tue Sep 8 15:36:32 2020","interfaces":[{"interface":"back","average":{"1min":4.5739999999999998,"5min":4.1109999999999998,"15min":3.9489999999999998},"min":{"1min":3.0329999999999999,"5min":2.6629999999999998,"15min":2.6629999999999998},"max":{"1min":6.5049999999999999,"5min":6.6379999999999999,"15min":6.6379999999999999},"last":5.5439999999999996},{"interface":"front","average":{"1min":3.395,"5min":3.9039999999999999,"15min":4.133},"min":{"1min":2.5049999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"max":{"1min":4.5540000000000003,"5min":6.2030000000000003,"15min":6.2030000000000003},"last":3.8610000000000002}]},{"osd":9,"last update":"Tue Sep 8 15:35:53 2020","interfaces":[{"interface":"back","average":{"1min":4.2889999999999997,"5min":4.2850000000000001,"15min":4.2850000000000001},"min":{"1min":2.734,"5min":2.714,"15min":2.714},"max":{"1min":5.7770000000000001,"5min":6.1769999999999996,"15min":6.1769999999999996},"last":5.7569999999999997},{"interface":"front","average":{"1min":3.472,"5min":4,"15min":4.0880000000000001},"min":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"max":{"1min":5.4790000000000001,"5min":5.4790000000000001,"15min":5.4790000000000001},"last":4.6299999999999999}]},{"osd":10,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":3.3039999999999998,"5min":3.157,"15min":3.133},"min":{"1min":2.649,"5min":2.3210000000000002,"15min":2.3210000000000002},"max":{"1min":4.1529999999999996,"5min":4.1680000000000001,"15min":4.1680000000000001},"last":3.7450000000000001},{"interface":"front","average":{"1min":4.944,"5min":4.4969999999999999,"15min":4.423},"min":{"1min":3.452,"5min":2.8340000000000001,"15min":2.8340000000000001},"max":{"1min":6.758,"5min":6.758,"15min":6.758},"last":6.1769999999999996}]},{"osd":11,"last update":"Tue Sep 8 15:36:42 2020","interfaces":[{"interface":"back","average":{"1min":4.1379999999999999,"5min":3.9380000000000002,"15min":3.9049999999999998},"min":{"1min":3.1640000000000001,"5min":2.9460000000000002,"15min":2.9460000000000002},"max":{"1min":6.1159999999999997,"5min":6.1159999999999997,"15min":6.1159999999999997},"last":5.0119999999999996},{"interface":"front","average":{"1min":4.8639999999999999,"5min":4.8609999999999998,"15min":4.8609999999999998},"min":{"1min":3.5800000000000001,"5min":2.911,"15min":2.911},"max":{"1min":6.407,"5min":7.0220000000000002,"15min":7.0220000000000002},"last":5.9450000000000003}]}]},{"osd":2,"up_from":0,"seq":74,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6220,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738932,"statfs":{"total":95995035648,"available":95988666368,"internally_reserved":0,"allocated":659456,"data_stored":372354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.1970000000000001,"5min":1.9179999999999999,"15min":1.623},"min":{"1min":1.4259999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":3.883,"5min":3.883,"15min":3.883},"last":2.9140000000000001},{"interface":"front","average":{"1min":2.0030000000000001,"5min":1.9419999999999999,"15min":1.702},"min":{"1min":1.599,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.7160000000000002,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":1.766}]},{"osd":1,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.0390000000000001,"5min":1.915,"15min":1.6870000000000001},"min":{"1min":1.306,"5min":0.999,"15min":0.999},"max":{"1min":3.363,"5min":3.363,"15min":3.363},"last":1.917},{"interface":"front","average":{"1min":2.6629999999999998,"5min":2.4340000000000002,"15min":2.0899999999999999},"min":{"1min":1.2809999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"max":{"1min":3.456,"5min":3.9399999999999999,"15min":3.9399999999999999},"last":1.8340000000000001}]},{"osd":3,"last update":"Tue Sep 8 15:36:32 2020","interfaces":[{"interface":"back","average":{"1min":2.944,"5min":2.706,"15min":2.3799999999999999},"min":{"1min":1.7130000000000001,"5min":1.587,"15min":1.587},"max":{"1min":4.1429999999999998,"5min":4.3209999999999997,"15min":4.3209999999999997},"last":3.3700000000000001},{"interface":"front","average":{"1min":2.7490000000000001,"5min":2.5600000000000001,"15min":2.3679999999999999},"min":{"1min":1.9059999999999999,"5min":0.755,"15min":0.755},"max":{"1min":3.641,"5min":3.6709999999999998,"15min":3.6709999999999998},"last":2.4700000000000002}]},{"osd":4,"last update":"Tue Sep 8 15:35:52 2020","interfaces":[{"interface":"back","average":{"1min":2.9329999999999998,"5min":2.6680000000000001,"15min":2.4399999999999999},"min":{"1min":1.9350000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"max":{"1min":4.4340000000000002,"5min":4.4340000000000002,"15min":4.4340000000000002},"last":2.2799999999999998},{"interface":"front","average":{"1min":2.9849999999999999,"5min":2.6600000000000001,"15min":2.3860000000000001},"min":{"1min":1.7210000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"max":{"1min":4.7080000000000002,"5min":4.827,"15min":4.827},"last":3.1200000000000001}]},{"osd":5,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":3.5569999999999999,"5min":3.222,"15min":2.9380000000000002},"min":{"1min":2.4060000000000001,"5min":1.353,"15min":1.353},"max":{"1min":4.6050000000000004,"5min":4.8449999999999998,"15min":4.8449999999999998},"last":3.6699999999999999},{"interface":"front","average":{"1min":3.7599999999999998,"5min":3.2879999999999998,"15min":2.98},"min":{"1min":2.4630000000000001,"5min":1.232,"15min":1.232},"max":{"1min":4.4530000000000003,"5min":5.2119999999999997,"15min":5.2119999999999997},"last":3.7599999999999998}]},{"osd":6,"last update":"Tue Sep 8 15:36:44 2020","interfaces":[{"interface":"back","average":{"1min":3.8450000000000002,"5min":3.4199999999999999,"15min":3.1880000000000002},"min":{"1min":2.9780000000000002,"5min":1.5069999999999999,"15min":1.5069999999999999},"max":{"1min":5.4980000000000002,"5min":5.4980000000000002,"15min":5.4980000000000002},"last":4.2599999999999998},{"interface":"front","average":{"1min":3.931,"5min":3.774,"15min":3.5670000000000002},"min":{"1min":3.339,"5min":2.0169999999999999,"15min":2.0169999999999999},"max":{"1min":5.8300000000000001,"5min":5.8300000000000001,"15min":5.8300000000000001},"last":4.5099999999999998}]},{"osd":7,"last update":"Tue Sep 8 15:36:09 2020","interfaces":[{"interface":"back","average":{"1min":4.4720000000000004,"5min":4.1779999999999999,"15min":4.0419999999999998},"min":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"max":{"1min":5.4340000000000002,"5min":6.5780000000000003,"15min":6.5780000000000003},"last":5.2480000000000002},{"interface":"front","average":{"1min":4.0419999999999998,"5min":3.7490000000000001,"15min":3.722},"min":{"1min":2.847,"5min":2.1299999999999999,"15min":2.1299999999999999},"max":{"1min":7.8659999999999997,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":3.1920000000000002}]},{"osd":8,"last update":"Tue Sep 8 15:36:26 2020","interfaces":[{"interface":"back","average":{"1min":3.9900000000000002,"5min":3.7370000000000001,"15min":3.6850000000000001},"min":{"1min":2.5369999999999999,"5min":1.907,"15min":1.907},"max":{"1min":5.5359999999999996,"5min":5.5359999999999996,"15min":5.5359999999999996},"last":3.4489999999999998},{"interface":"front","average":{"1min":4.2439999999999998,"5min":4.0410000000000004,"15min":3.9889999999999999},"min":{"1min":3.0699999999999998,"5min":2.7719999999999998,"15min":2.7719999999999998},"max":{"1min":5.4939999999999998,"5min":6.3150000000000004,"15min":6.3150000000000004},"last":4.8630000000000004}]},{"osd":9,"last update":"Tue Sep 8 15:35:59 2020","interfaces":[{"interface":"back","average":{"1min":4.0670000000000002,"5min":4.0499999999999998,"15min":4.0469999999999997},"min":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"max":{"1min":6.3559999999999999,"5min":6.3559999999999999,"15min":6.3559999999999999},"last":3.8519999999999999},{"interface":"front","average":{"1min":4.2220000000000004,"5min":4.2850000000000001,"15min":4.2949999999999999},"min":{"1min":2.71,"5min":2.71,"15min":2.71},"max":{"1min":6.1070000000000002,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":4.1399999999999997}]},{"osd":10,"last update":"Tue Sep 8 15:36:22 2020","interfaces":[{"interface":"back","average":{"1min":4.4729999999999999,"5min":4.1130000000000004,"15min":4.0529999999999999},"min":{"1min":3.145,"5min":2.7480000000000002,"15min":2.7480000000000002},"max":{"1min":6.915,"5min":6.915,"15min":6.915},"last":4.0259999999999998},{"interface":"front","average":{"1min":3.895,"5min":3.8140000000000001,"15min":3.7999999999999998},"min":{"1min":2.6890000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"max":{"1min":7.6580000000000004,"5min":7.6580000000000004,"15min":7.6580000000000004},"last":2.8530000000000002}]}]},{"osd":3,"up_from":0,"seq":68,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6156,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738996,"statfs":{"total":95995035648,"available":95988731904,"internally_reserved":0,"allocated":659456,"data_stored":372354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:31 2020","interfaces":[{"interface":"back","average":{"1min":2.1379999999999999,"5min":1.7210000000000001,"15min":1.629},"min":{"1min":1.79,"5min":1.1419999999999999,"15min":1.1419999999999999},"max":{"1min":2.7930000000000001,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":3.169},{"interface":"front","average":{"1min":2.6800000000000002,"5min":2.2309999999999999,"15min":2.0990000000000002},"min":{"1min":2.0270000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"max":{"1min":3.5409999999999999,"5min":3.5409999999999999,"15min":3.5409999999999999},"last":2.5579999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:36:35 2020","interfaces":[{"interface":"back","average":{"1min":2.133,"5min":2.0939999999999999,"15min":1.8740000000000001},"min":{"1min":1.2809999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"max":{"1min":3.3399999999999999,"5min":3.6829999999999998,"15min":3.6829999999999998},"last":1.498},{"interface":"front","average":{"1min":2.1709999999999998,"5min":2.1400000000000001,"15min":1.8460000000000001},"min":{"1min":1.4059999999999999,"5min":1.341,"15min":1.341},"max":{"1min":3.1549999999999998,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":1.444}]},{"osd":2,"last update":"Tue Sep 8 15:36:35 2020","interfaces":[{"interface":"back","average":{"1min":2.5619999999999998,"5min":2.387,"15min":2.145},"min":{"1min":1.7669999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":3.5009999999999999,"5min":3.7810000000000001,"15min":3.7810000000000001},"last":1.948},{"interface":"front","average":{"1min":2.1869999999999998,"5min":2.464,"15min":2.2389999999999999},"min":{"1min":1.585,"5min":1.3580000000000001,"15min":1.3580000000000001},"max":{"1min":3.1699999999999999,"5min":3.8010000000000002,"15min":3.8010000000000002},"last":1.411}]},{"osd":4,"last update":"Tue Sep 8 15:35:54 2020","interfaces":[{"interface":"back","average":{"1min":2.8540000000000001,"5min":2.7320000000000002,"15min":2.5419999999999998},"min":{"1min":1.8839999999999999,"5min":1.4850000000000001,"15min":1.4850000000000001},"max":{"1min":3.778,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":1.649},{"interface":"front","average":{"1min":2.8010000000000002,"5min":2.7189999999999999,"15min":2.4830000000000001},"min":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"max":{"1min":3.4700000000000002,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":2.0089999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:36:15 2020","interfaces":[{"interface":"back","average":{"1min":2.9300000000000002,"5min":2.9830000000000001,"15min":2.8530000000000002},"min":{"1min":1.323,"5min":1.323,"15min":1.323},"max":{"1min":3.9489999999999998,"5min":5.0289999999999999,"15min":5.0289999999999999},"last":1.8100000000000001},{"interface":"front","average":{"1min":3.4569999999999999,"5min":3.3929999999999998,"15min":3.2330000000000001},"min":{"1min":1.988,"5min":1.988,"15min":1.988},"max":{"1min":4.2910000000000004,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":1.7549999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:36:44 2020","interfaces":[{"interface":"back","average":{"1min":3.4249999999999998,"5min":3.444,"15min":3.3479999999999999},"min":{"1min":1.8680000000000001,"5min":1.714,"15min":1.714},"max":{"1min":4.6159999999999997,"5min":5.4160000000000004,"15min":5.4160000000000004},"last":1.8680000000000001},{"interface":"front","average":{"1min":3.544,"5min":3.6739999999999999,"15min":3.5449999999999999},"min":{"1min":2.093,"5min":2.093,"15min":2.093},"max":{"1min":4.694,"5min":5.3499999999999996,"15min":5.3499999999999996},"last":2.093}]},{"osd":7,"last update":"Tue Sep 8 15:36:11 2020","interfaces":[{"interface":"back","average":{"1min":4.234,"5min":4.2649999999999997,"15min":4.2530000000000001},"min":{"1min":3.081,"5min":2.8069999999999999,"15min":2.8069999999999999},"max":{"1min":5.1440000000000001,"5min":5.5640000000000001,"15min":5.5640000000000001},"last":2.262},{"interface":"front","average":{"1min":3.577,"5min":3.629,"15min":3.617},"min":{"1min":2.601,"5min":2.601,"15min":2.601},"max":{"1min":4.258,"5min":5.258,"15min":5.258},"last":2.6989999999999998}]},{"osd":8,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":3.8889999999999998,"5min":3.758,"15min":3.7829999999999999},"min":{"1min":2.9119999999999999,"5min":2.2440000000000002,"15min":2.2440000000000002},"max":{"1min":5.3390000000000004,"5min":5.4610000000000003,"15min":5.4610000000000003},"last":2.6240000000000001},{"interface":"front","average":{"1min":4.5220000000000002,"5min":4.3319999999999999,"15min":4.3079999999999998},"min":{"1min":3.206,"5min":2.351,"15min":2.351},"max":{"1min":5.5309999999999997,"5min":6.1239999999999997,"15min":6.1239999999999997},"last":2.9239999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:35:54 2020","interfaces":[{"interface":"back","average":{"1min":3.7930000000000001,"5min":3.96,"15min":3.988},"min":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"max":{"1min":4.468,"5min":4.6120000000000001,"15min":4.6120000000000001},"last":3.4670000000000001},{"interface":"front","average":{"1min":3.5840000000000001,"5min":4.2480000000000002,"15min":4.3579999999999997},"min":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"max":{"1min":4.7889999999999997,"5min":5.2530000000000001,"15min":5.2530000000000001},"last":2.8540000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:36:21 2020","interfaces":[{"interface":"back","average":{"1min":4.6360000000000001,"5min":4.4450000000000003,"15min":4.4130000000000003},"min":{"1min":2.875,"5min":2.875,"15min":2.875},"max":{"1min":5.7880000000000003,"5min":5.7880000000000003,"15min":5.7880000000000003},"last":2.4369999999999998},{"interface":"front","average":{"1min":4.0990000000000002,"5min":3.8290000000000002,"15min":3.7839999999999998},"min":{"1min":3.194,"5min":2.8759999999999999,"15min":2.8759999999999999},"max":{"1min":4.7599999999999998,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":3.1499999999999999}]},{"osd":11,"last update":"Tue Sep 8 15:35:50 2020","interfaces":[{"interface":"back","average":{"1min":4.8159999999999998,"5min":4.8159999999999998,"15min":4.8159999999999998},"min":{"1min":3.7509999999999999,"5min":3.7509999999999999,"15min":3.7509999999999999},"max":{"1min":5.9829999999999997,"5min":5.9829999999999997,"15min":5.9829999999999997},"last":2.5329999999999999},{"interface":"front","average":{"1min":4.1459999999999999,"5min":4.1459999999999999,"15min":4.1459999999999999},"min":{"1min":3.04,"5min":3.04,"15min":3.04},"max":{"1min":5.1589999999999998,"5min":5.1589999999999998,"15min":5.1589999999999998},"last":3.3090000000000002}]}]},{"osd":4,"up_from":0,"seq":64,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6200,"kb_used_data":632,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738952,"statfs":{"total":95995035648,"available":95988686848,"internally_reserved":0,"allocated":647168,"data_stored":365147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:35:52 2020","interfaces":[{"interface":"back","average":{"1min":2.2679999999999998,"5min":1.9890000000000001,"15min":1.845},"min":{"1min":1.5740000000000001,"5min":1.01,"15min":1.01},"max":{"1min":3.0880000000000001,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":2.294},{"interface":"front","average":{"1min":2.5600000000000001,"5min":2.0699999999999998,"15min":1.792},"min":{"1min":1.8759999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.4169999999999998,"5min":3.7679999999999998,"15min":3.7679999999999998},"last":2.3660000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:35:52 2020","interfaces":[{"interface":"back","average":{"1min":2.4049999999999998,"5min":1.998,"15min":1.839},"min":{"1min":1.51,"5min":1.1759999999999999,"15min":1.1759999999999999},"max":{"1min":3.5270000000000001,"5min":5.3659999999999997,"15min":5.3659999999999997},"last":2.4790000000000001},{"interface":"front","average":{"1min":2.8279999999999998,"5min":2.391,"15min":2.0590000000000002},"min":{"1min":1.7090000000000001,"5min":1.1699999999999999,"15min":1.1699999999999999},"max":{"1min":4.0339999999999998,"5min":4.5700000000000003,"15min":4.5700000000000003},"last":3.6930000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:35:52 2020","interfaces":[{"interface":"back","average":{"1min":3.0510000000000002,"5min":2.4540000000000002,"15min":2.0600000000000001},"min":{"1min":1.7270000000000001,"5min":1.236,"15min":1.236},"max":{"1min":3.7429999999999999,"5min":4.1550000000000002,"15min":4.1550000000000002},"last":3.351},{"interface":"front","average":{"1min":3.2000000000000002,"5min":2.6080000000000001,"15min":2.2400000000000002},"min":{"1min":1.8879999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"max":{"1min":4.5609999999999999,"5min":4.5609999999999999,"15min":4.5609999999999999},"last":3.52}]},{"osd":3,"last update":"Tue Sep 8 15:35:52 2020","interfaces":[{"interface":"back","average":{"1min":3.3660000000000001,"5min":2.8109999999999999,"15min":2.4369999999999998},"min":{"1min":1.9670000000000001,"5min":1.389,"15min":1.389},"max":{"1min":4.968,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":3.0670000000000002},{"interface":"front","average":{"1min":3.032,"5min":2.6240000000000001,"15min":2.302},"min":{"1min":2.1379999999999999,"5min":1.3440000000000001,"15min":1.3440000000000001},"max":{"1min":4.4580000000000002,"5min":4.6180000000000003,"15min":4.6180000000000003},"last":3.4129999999999998}]},{"osd":5,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":3.9569999999999999,"5min":3.294,"15min":2.7519999999999998},"min":{"1min":2.7149999999999999,"5min":1.014,"15min":1.014},"max":{"1min":6.9800000000000004,"5min":6.9800000000000004,"15min":6.9800000000000004},"last":3.786},{"interface":"front","average":{"1min":3.9449999999999998,"5min":3.274,"15min":2.8199999999999998},"min":{"1min":2.5800000000000001,"5min":1.129,"15min":1.129},"max":{"1min":6.2279999999999998,"5min":6.2279999999999998,"15min":6.2279999999999998},"last":4.2930000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:36:42 2020","interfaces":[{"interface":"back","average":{"1min":4.0700000000000003,"5min":3.6749999999999998,"15min":3.3490000000000002},"min":{"1min":2.9860000000000002,"5min":2.0449999999999999,"15min":2.0449999999999999},"max":{"1min":5.8179999999999996,"5min":6.6870000000000003,"15min":6.6870000000000003},"last":3.875},{"interface":"front","average":{"1min":3.8079999999999998,"5min":3.6160000000000001,"15min":3.3660000000000001},"min":{"1min":2.9980000000000002,"5min":2.2429999999999999,"15min":2.2429999999999999},"max":{"1min":5.2030000000000003,"5min":5.2030000000000003,"15min":5.2030000000000003},"last":3.968}]},{"osd":7,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":4.1150000000000002,"5min":3.9860000000000002,"15min":3.8700000000000001},"min":{"1min":2.5739999999999998,"5min":1.75,"15min":1.75},"max":{"1min":7.4699999999999998,"5min":7.4699999999999998,"15min":7.4699999999999998},"last":4.0439999999999996},{"interface":"front","average":{"1min":4.0430000000000001,"5min":4.0570000000000004,"15min":3.98},"min":{"1min":2.718,"5min":2.6619999999999999,"15min":2.6619999999999999},"max":{"1min":6.6399999999999997,"5min":6.6399999999999997,"15min":6.6399999999999997},"last":4.7080000000000002}]},{"osd":8,"last update":"Tue Sep 8 15:36:30 2020","interfaces":[{"interface":"back","average":{"1min":4.1280000000000001,"5min":4,"15min":3.9359999999999999},"min":{"1min":3.327,"5min":3.0739999999999998,"15min":3.0739999999999998},"max":{"1min":5.1920000000000002,"5min":5.3929999999999998,"15min":5.3929999999999998},"last":4.375},{"interface":"front","average":{"1min":4.681,"5min":4.4619999999999997,"15min":4.3799999999999999},"min":{"1min":3.746,"5min":3.0840000000000001,"15min":3.0840000000000001},"max":{"1min":6.5540000000000003,"5min":6.5540000000000003,"15min":6.5540000000000003},"last":4.2309999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:35:55 2020","interfaces":[{"interface":"back","average":{"1min":4.4349999999999996,"5min":4.3860000000000001,"15min":4.3780000000000001},"min":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"max":{"1min":5.5590000000000002,"5min":5.5590000000000002,"15min":5.5590000000000002},"last":5.0899999999999999},{"interface":"front","average":{"1min":4.2939999999999996,"5min":4.4020000000000001,"15min":4.4199999999999999},"min":{"1min":3.5289999999999999,"5min":3.5289999999999999,"15min":3.5289999999999999},"max":{"1min":5.2320000000000002,"5min":5.7779999999999996,"15min":5.7779999999999996},"last":4.4720000000000004}]},{"osd":10,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":4.8129999999999997,"5min":4.9130000000000003,"15min":4.9299999999999997},"min":{"1min":3.9769999999999999,"5min":3.3439999999999999,"15min":3.3439999999999999},"max":{"1min":7.9279999999999999,"5min":7.9279999999999999,"15min":7.9279999999999999},"last":4.5890000000000004},{"interface":"front","average":{"1min":4.8440000000000003,"5min":4.9109999999999996,"15min":4.9219999999999997},"min":{"1min":3.7890000000000001,"5min":3.7890000000000001,"15min":3.7890000000000001},"max":{"1min":7.7590000000000003,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":5.5010000000000003}]}]},{"osd":5,"up_from":0,"seq":58,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6092,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93739060,"statfs":{"total":95995035648,"available":95988797440,"internally_reserved":0,"allocated":659456,"data_stored":372354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":2.2589999999999999,"5min":2.1240000000000001,"15min":1.8740000000000001},"min":{"1min":1.7749999999999999,"5min":1.119,"15min":1.119},"max":{"1min":3.1320000000000001,"5min":3.96,"15min":3.96},"last":1.8160000000000001},{"interface":"front","average":{"1min":2.5790000000000002,"5min":2.2290000000000001,"15min":1.9670000000000001},"min":{"1min":1.929,"5min":1.4510000000000001,"15min":1.4510000000000001},"max":{"1min":3.7570000000000001,"5min":3.7570000000000001,"15min":3.7570000000000001},"last":1.8779999999999999}]},{"osd":1,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.0049999999999999,"5min":2.5259999999999998,"15min":2.3370000000000002},"min":{"1min":1.643,"5min":1.339,"15min":1.339},"max":{"1min":4.2240000000000002,"5min":4.2240000000000002,"15min":4.2240000000000002},"last":1.954},{"interface":"front","average":{"1min":2.1779999999999999,"5min":2.0760000000000001,"15min":2.0059999999999998},"min":{"1min":1.7090000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"max":{"1min":2.9830000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":4.2380000000000004}]},{"osd":2,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":2.456,"5min":2.2839999999999998,"15min":2.1339999999999999},"min":{"1min":1.8109999999999999,"5min":1.474,"15min":1.474},"max":{"1min":3.5539999999999998,"5min":3.7570000000000001,"15min":3.7570000000000001},"last":3.2810000000000001},{"interface":"front","average":{"1min":3.4980000000000002,"5min":2.9969999999999999,"15min":2.7410000000000001},"min":{"1min":2.1240000000000001,"5min":1.796,"15min":1.796},"max":{"1min":4.9000000000000004,"5min":4.9000000000000004,"15min":4.9000000000000004},"last":3.1190000000000002}]},{"osd":3,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.7639999999999998,"5min":3.234,"15min":2.839},"min":{"1min":2.4249999999999998,"5min":1.6719999999999999,"15min":1.6719999999999999},"max":{"1min":4.718,"5min":5.3899999999999997,"15min":5.3899999999999997},"last":3.6920000000000002},{"interface":"front","average":{"1min":3.2280000000000002,"5min":3.1139999999999999,"15min":2.9129999999999998},"min":{"1min":2.3679999999999999,"5min":1.9990000000000001,"15min":1.9990000000000001},"max":{"1min":4.2050000000000001,"5min":5.4889999999999999,"15min":5.4889999999999999},"last":2.512}]},{"osd":4,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":2.9780000000000002,"5min":2.5710000000000002,"15min":2.3010000000000002},"min":{"1min":2.0910000000000002,"5min":1.6220000000000001,"15min":1.6220000000000001},"max":{"1min":3.5499999999999998,"5min":3.9710000000000001,"15min":3.9710000000000001},"last":3.0169999999999999},{"interface":"front","average":{"1min":3.7210000000000001,"5min":3.0779999999999998,"15min":2.6829999999999998},"min":{"1min":2.819,"5min":1.3979999999999999,"15min":1.3979999999999999},"max":{"1min":6.5190000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":3.6120000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":4.1189999999999998,"5min":3.819,"15min":3.609},"min":{"1min":3.2829999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"max":{"1min":4.718,"5min":5.9870000000000001,"15min":5.9870000000000001},"last":4.7110000000000003},{"interface":"front","average":{"1min":4.7560000000000002,"5min":3.6379999999999999,"15min":3.1400000000000001},"min":{"1min":3.766,"5min":1.948,"15min":1.948},"max":{"1min":5.5590000000000002,"5min":5.6600000000000001,"15min":5.6600000000000001},"last":5.4039999999999999}]},{"osd":7,"last update":"Tue Sep 8 15:36:03 2020","interfaces":[{"interface":"back","average":{"1min":4.8559999999999999,"5min":3.9359999999999999,"15min":3.617},"min":{"1min":3.1909999999999998,"5min":2.5859999999999999,"15min":2.5859999999999999},"max":{"1min":6.782,"5min":6.782,"15min":6.782},"last":4.4219999999999997},{"interface":"front","average":{"1min":3.7669999999999999,"5min":3.9849999999999999,"15min":4.0099999999999998},"min":{"1min":2.665,"5min":2.665,"15min":2.665},"max":{"1min":4.8520000000000003,"5min":5.9059999999999997,"15min":5.9059999999999997},"last":5.6180000000000003}]},{"osd":8,"last update":"Tue Sep 8 15:36:34 2020","interfaces":[{"interface":"back","average":{"1min":3.621,"5min":3.677,"15min":3.5990000000000002},"min":{"1min":3.101,"5min":2.8940000000000001,"15min":2.8940000000000001},"max":{"1min":4.2649999999999997,"5min":5.2610000000000001,"15min":5.2610000000000001},"last":3.4820000000000002},{"interface":"front","average":{"1min":4.0570000000000004,"5min":4.0629999999999997,"15min":4.0529999999999999},"min":{"1min":3.4279999999999999,"5min":3.2400000000000002,"15min":3.2400000000000002},"max":{"1min":4.6230000000000002,"5min":5.3319999999999999,"15min":5.3319999999999999},"last":3.823}]},{"osd":9,"last update":"Tue Sep 8 15:35:58 2020","interfaces":[{"interface":"back","average":{"1min":5.2080000000000002,"5min":4.6870000000000003,"15min":4.5999999999999996},"min":{"1min":4.1040000000000001,"5min":3.4550000000000001,"15min":3.4550000000000001},"max":{"1min":7.1719999999999997,"5min":7.1719999999999997,"15min":7.1719999999999997},"last":4.0940000000000003},{"interface":"front","average":{"1min":4.218,"5min":4.218,"15min":4.218},"min":{"1min":3.629,"5min":2.6930000000000001,"15min":2.6930000000000001},"max":{"1min":4.8799999999999999,"5min":5.6929999999999996,"15min":5.6929999999999996},"last":5.2089999999999996}]},{"osd":10,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":4.2569999999999997,"5min":4.2409999999999997,"15min":4.2380000000000004},"min":{"1min":3.3849999999999998,"5min":3.0190000000000001,"15min":3.0190000000000001},"max":{"1min":5.3609999999999998,"5min":5.3609999999999998,"15min":5.3609999999999998},"last":4.0259999999999998},{"interface":"front","average":{"1min":5.4649999999999999,"5min":4.9219999999999997,"15min":4.8319999999999999},"min":{"1min":3.9039999999999999,"5min":3.9039999999999999,"15min":3.9039999999999999},"max":{"1min":6.3390000000000004,"5min":6.3390000000000004,"15min":6.3390000000000004},"last":4.9870000000000001}]}]}]}} 2020-09-08T15:36:48.677 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2020-09-08T15:36:48.677 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2020-09-08T15:36:48.678 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2020-09-08T15:36:48.678 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph health --format=json 2020-09-08T15:36:48.810 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:36:48.938 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:48 smithi104 bash[9203]: audit 2020-09-08T15:36:47.995874+0000 mgr.x (mgr.24118) 273 : audit [DBG] from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:36:49.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:48 smithi166 bash[9446]: audit 2020-09-08T15:36:47.995874+0000 mgr.x (mgr.24118) 273 : audit [DBG] from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:36:49.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:48 smithi055 bash[9296]: audit 2020-09-08T15:36:47.995874+0000 mgr.x (mgr.24118) 273 : audit [DBG] from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:36:50.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:50 smithi166 bash[9446]: cluster 2020-09-08T15:36:49.574581+0000 mgr.x (mgr.24118) 274 : cluster [DBG] pgmap v317: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:50.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:50 smithi055 bash[9296]: cluster 2020-09-08T15:36:49.574581+0000 mgr.x (mgr.24118) 274 : cluster [DBG] pgmap v317: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:50.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:50 smithi104 bash[9203]: cluster 2020-09-08T15:36:49.574581+0000 mgr.x (mgr.24118) 274 : cluster [DBG] pgmap v317: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:51.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:51 smithi104 bash[9203]: cluster 2020-09-08T15:36:51.575214+0000 mgr.x (mgr.24118) 275 : cluster [DBG] pgmap v318: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:52.042 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T15:36:52.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:51 smithi166 bash[9446]: cluster 2020-09-08T15:36:51.575214+0000 mgr.x (mgr.24118) 275 : cluster [DBG] pgmap v318: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:52.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:51 smithi055 bash[9296]: cluster 2020-09-08T15:36:51.575214+0000 mgr.x (mgr.24118) 275 : cluster [DBG] pgmap v318: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:52.725 INFO:teuthology.orchestra.run.smithi104.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2020-09-08T15:36:52.726 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2020-09-08T15:36:52.727 INFO:tasks.cephadm:Setup complete, yielding 2020-09-08T15:36:52.728 INFO:teuthology.run_tasks:Running task exec... 2020-09-08T15:36:52.741 INFO:teuthology.task.exec:Executing custom commands... 2020-09-08T15:36:52.742 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi104.front.sepia.ceph.com 2020-09-08T15:36:52.743 INFO:teuthology.orchestra.run.smithi104:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'while ! ceph balancer status ; do sleep 1 ; done' 2020-09-08T15:36:52.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:52 smithi104 bash[9203]: audit 2020-09-08T15:36:52.046932+0000 mon.a (mon.0) 881 : audit [DBG] from='client.? v1:172.21.15.104:0/3108637294' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2020-09-08T15:36:53.113 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:36:53.114 INFO:teuthology.orchestra.run.smithi104.stdout: "active": true, 2020-09-08T15:36:53.114 INFO:teuthology.orchestra.run.smithi104.stdout: "last_optimize_duration": "0:00:00.000953", 2020-09-08T15:36:53.114 INFO:teuthology.orchestra.run.smithi104.stdout: "last_optimize_started": "Tue Sep 8 15:36:39 2020", 2020-09-08T15:36:53.114 INFO:teuthology.orchestra.run.smithi104.stdout: "mode": "upmap", 2020-09-08T15:36:53.115 INFO:teuthology.orchestra.run.smithi104.stdout: "optimize_result": "Unable to find further optimization, or pool(s) pg_num is decreasing, or distribution is already perfect", 2020-09-08T15:36:53.115 INFO:teuthology.orchestra.run.smithi104.stdout: "plans": [] 2020-09-08T15:36:53.115 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:36:53.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:52 smithi166 bash[9446]: audit 2020-09-08T15:36:52.046932+0000 mon.a (mon.0) 881 : audit [DBG] from='client.? v1:172.21.15.104:0/3108637294' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2020-09-08T15:36:53.144 INFO:teuthology.orchestra.run.smithi104:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph balancer mode crush-compat' 2020-09-08T15:36:53.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:52 smithi055 bash[9296]: audit 2020-09-08T15:36:52.046932+0000 mon.a (mon.0) 881 : audit [DBG] from='client.? v1:172.21.15.104:0/3108637294' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2020-09-08T15:36:53.733 INFO:teuthology.orchestra.run.smithi104:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph balancer on' 2020-09-08T15:36:53.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:53 smithi104 bash[9203]: cluster 2020-09-08T15:36:52.477004+0000 osd.9 (osd.9) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:53.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:53 smithi104 bash[9203]: cluster 2020-09-08T15:36:52.477186+0000 osd.9 (osd.9) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:53.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:53 smithi104 bash[9203]: cluster 2020-09-08T15:36:52.674115+0000 mon.a (mon.0) 882 : cluster [DBG] osdmap e117: 12 total, 12 up, 12 in 2020-09-08T15:36:53.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:53 smithi104 bash[9203]: audit 2020-09-08T15:36:53.116931+0000 mgr.x (mgr.24118) 276 : audit [DBG] from='client.14766 v1:172.21.15.104:0/1943448146' entity='client.admin' cmd=[{"prefix": "balancer status", "target": ["mgr", ""]}]: dispatch 2020-09-08T15:36:53.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:53 smithi104 bash[9203]: audit 2020-09-08T15:36:53.457481+0000 mgr.x (mgr.24118) 277 : audit [DBG] from='client.14772 v1:172.21.15.104:0/4120859771' entity='client.admin' cmd=[{"prefix": "balancer mode", "mode": "crush-compat", "target": ["mgr", ""]}]: dispatch 2020-09-08T15:36:53.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:53 smithi104 bash[9203]: audit 2020-09-08T15:36:53.462252+0000 mon.b (mon.1) 70 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd crush weight-set create-compat", "format": "json"}]: dispatch 2020-09-08T15:36:53.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:53 smithi104 bash[9203]: audit 2020-09-08T15:36:53.462303+0000 mon.a (mon.0) 883 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix": "osd crush weight-set create-compat", "format": "json"}]: dispatch 2020-09-08T15:36:53.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:53 smithi104 bash[9203]: cluster 2020-09-08T15:36:53.575854+0000 mgr.x (mgr.24118) 278 : cluster [DBG] pgmap v320: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:54.060 INFO:teuthology.run_tasks:Running task thrashosds... 2020-09-08T15:36:54.077 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2020-09-08T15:36:54.078 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-kefu-testing-2020-09-08-0934'}, 'ceph': {'conf': {'client.2': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'mon client directed command retry': 5, 'mon warn on msgr2 not enabled': False, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'ms type': 'async', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', '\\(MON_DOWN\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'mon_bind_msgr2': False, 'sha1': 'dce4d291bd4243c9802c17d67b85a6da6bbc0f8f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': 'dce4d291bd4243c9802c17d67b85a6da6bbc0f8f'}}, 'workunit': {'branch': 'wip-kefu-testing-2020-09-08-0934', 'sha1': 'dce4d291bd4243c9802c17d67b85a6da6bbc0f8f'}} 2020-09-08T15:36:54.079 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2020-09-08T15:36:54.079 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2020-09-08T15:36:54.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:53 smithi166 bash[9446]: cluster 2020-09-08T15:36:52.477004+0000 osd.9 (osd.9) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:54.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:53 smithi166 bash[9446]: cluster 2020-09-08T15:36:52.477186+0000 osd.9 (osd.9) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:54.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:53 smithi166 bash[9446]: cluster 2020-09-08T15:36:52.674115+0000 mon.a (mon.0) 882 : cluster [DBG] osdmap e117: 12 total, 12 up, 12 in 2020-09-08T15:36:54.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:53 smithi166 bash[9446]: audit 2020-09-08T15:36:53.116931+0000 mgr.x (mgr.24118) 276 : audit [DBG] from='client.14766 v1:172.21.15.104:0/1943448146' entity='client.admin' cmd=[{"prefix": "balancer status", "target": ["mgr", ""]}]: dispatch 2020-09-08T15:36:54.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:53 smithi166 bash[9446]: audit 2020-09-08T15:36:53.457481+0000 mgr.x (mgr.24118) 277 : audit [DBG] from='client.14772 v1:172.21.15.104:0/4120859771' entity='client.admin' cmd=[{"prefix": "balancer mode", "mode": "crush-compat", "target": ["mgr", ""]}]: dispatch 2020-09-08T15:36:54.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:53 smithi166 bash[9446]: audit 2020-09-08T15:36:53.462252+0000 mon.b (mon.1) 70 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd crush weight-set create-compat", "format": "json"}]: dispatch 2020-09-08T15:36:54.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:53 smithi166 bash[9446]: audit 2020-09-08T15:36:53.462303+0000 mon.a (mon.0) 883 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix": "osd crush weight-set create-compat", "format": "json"}]: dispatch 2020-09-08T15:36:54.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:53 smithi166 bash[9446]: cluster 2020-09-08T15:36:53.575854+0000 mgr.x (mgr.24118) 278 : cluster [DBG] pgmap v320: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:54.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:53 smithi055 bash[9296]: cluster 2020-09-08T15:36:52.477004+0000 osd.9 (osd.9) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:54.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:53 smithi055 bash[9296]: cluster 2020-09-08T15:36:52.477186+0000 osd.9 (osd.9) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:54.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:53 smithi055 bash[9296]: cluster 2020-09-08T15:36:52.674115+0000 mon.a (mon.0) 882 : cluster [DBG] osdmap e117: 12 total, 12 up, 12 in 2020-09-08T15:36:54.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:53 smithi055 bash[9296]: audit 2020-09-08T15:36:53.116931+0000 mgr.x (mgr.24118) 276 : audit [DBG] from='client.14766 v1:172.21.15.104:0/1943448146' entity='client.admin' cmd=[{"prefix": "balancer status", "target": ["mgr", ""]}]: dispatch 2020-09-08T15:36:54.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:53 smithi055 bash[9296]: audit 2020-09-08T15:36:53.457481+0000 mgr.x (mgr.24118) 277 : audit [DBG] from='client.14772 v1:172.21.15.104:0/4120859771' entity='client.admin' cmd=[{"prefix": "balancer mode", "mode": "crush-compat", "target": ["mgr", ""]}]: dispatch 2020-09-08T15:36:54.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:53 smithi055 bash[9296]: audit 2020-09-08T15:36:53.462252+0000 mon.b (mon.1) 70 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd crush weight-set create-compat", "format": "json"}]: dispatch 2020-09-08T15:36:54.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:53 smithi055 bash[9296]: audit 2020-09-08T15:36:53.462303+0000 mon.a (mon.0) 883 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix": "osd crush weight-set create-compat", "format": "json"}]: dispatch 2020-09-08T15:36:54.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:53 smithi055 bash[9296]: cluster 2020-09-08T15:36:53.575854+0000 mgr.x (mgr.24118) 278 : cluster [DBG] pgmap v320: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:54.213 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:36:55.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:54 smithi166 bash[9446]: audit 2020-09-08T15:36:53.686496+0000 mon.a (mon.0) 884 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix": "osd crush weight-set create-compat", "format": "json"}]': finished 2020-09-08T15:36:55.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:54 smithi166 bash[9446]: cluster 2020-09-08T15:36:53.686632+0000 mon.a (mon.0) 885 : cluster [DBG] osdmap e118: 12 total, 12 up, 12 in 2020-09-08T15:36:55.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:54 smithi166 bash[9446]: audit 2020-09-08T15:36:53.691396+0000 mon.b (mon.1) 71 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd crush dump", "format": "json"}]: dispatch 2020-09-08T15:36:55.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:54 smithi166 bash[9446]: audit 2020-09-08T15:36:53.694532+0000 mon.a (mon.0) 886 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config set","value":"crush-compat","who":"mgr","name":"mgr/balancer/mode"}]: dispatch 2020-09-08T15:36:55.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:54 smithi166 bash[9446]: audit 2020-09-08T15:36:53.694645+0000 mon.b (mon.1) 72 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config set","value":"crush-compat","who":"mgr","name":"mgr/balancer/mode"}]: dispatch 2020-09-08T15:36:55.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:54 smithi166 bash[9446]: audit 2020-09-08T15:36:53.701581+0000 mon.a (mon.0) 887 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config set","value":"crush-compat","who":"mgr","name":"mgr/balancer/mode"}]': finished 2020-09-08T15:36:55.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:54 smithi166 bash[9446]: audit 2020-09-08T15:36:53.714086+0000 mon.a (mon.0) 888 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:55.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:54 smithi166 bash[9446]: audit 2020-09-08T15:36:53.714188+0000 mon.b (mon.1) 73 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:55.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:54 smithi166 bash[9446]: audit 2020-09-08T15:36:53.718823+0000 mon.a (mon.0) 889 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:36:55.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:54 smithi166 bash[9446]: audit 2020-09-08T15:36:54.049310+0000 mgr.x (mgr.24118) 279 : audit [DBG] from='client.24661 v1:172.21.15.104:0/1847354615' entity='client.admin' cmd=[{"prefix": "balancer on", "target": ["mgr", ""]}]: dispatch 2020-09-08T15:36:55.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:54 smithi055 bash[9296]: audit 2020-09-08T15:36:53.686496+0000 mon.a (mon.0) 884 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix": "osd crush weight-set create-compat", "format": "json"}]': finished 2020-09-08T15:36:55.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:54 smithi055 bash[9296]: cluster 2020-09-08T15:36:53.686632+0000 mon.a (mon.0) 885 : cluster [DBG] osdmap e118: 12 total, 12 up, 12 in 2020-09-08T15:36:55.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:54 smithi055 bash[9296]: audit 2020-09-08T15:36:53.691396+0000 mon.b (mon.1) 71 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd crush dump", "format": "json"}]: dispatch 2020-09-08T15:36:55.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:54 smithi055 bash[9296]: audit 2020-09-08T15:36:53.694532+0000 mon.a (mon.0) 886 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config set","value":"crush-compat","who":"mgr","name":"mgr/balancer/mode"}]: dispatch 2020-09-08T15:36:55.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:54 smithi055 bash[9296]: audit 2020-09-08T15:36:53.694645+0000 mon.b (mon.1) 72 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config set","value":"crush-compat","who":"mgr","name":"mgr/balancer/mode"}]: dispatch 2020-09-08T15:36:55.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:54 smithi055 bash[9296]: audit 2020-09-08T15:36:53.701581+0000 mon.a (mon.0) 887 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config set","value":"crush-compat","who":"mgr","name":"mgr/balancer/mode"}]': finished 2020-09-08T15:36:55.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:54 smithi055 bash[9296]: audit 2020-09-08T15:36:53.714086+0000 mon.a (mon.0) 888 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:55.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:54 smithi055 bash[9296]: audit 2020-09-08T15:36:53.714188+0000 mon.b (mon.1) 73 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:55.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:54 smithi055 bash[9296]: audit 2020-09-08T15:36:53.718823+0000 mon.a (mon.0) 889 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:36:55.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:54 smithi055 bash[9296]: audit 2020-09-08T15:36:54.049310+0000 mgr.x (mgr.24118) 279 : audit [DBG] from='client.24661 v1:172.21.15.104:0/1847354615' entity='client.admin' cmd=[{"prefix": "balancer on", "target": ["mgr", ""]}]: dispatch 2020-09-08T15:36:55.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:54 smithi104 bash[9203]: audit 2020-09-08T15:36:53.686496+0000 mon.a (mon.0) 884 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix": "osd crush weight-set create-compat", "format": "json"}]': finished 2020-09-08T15:36:55.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:54 smithi104 bash[9203]: cluster 2020-09-08T15:36:53.686632+0000 mon.a (mon.0) 885 : cluster [DBG] osdmap e118: 12 total, 12 up, 12 in 2020-09-08T15:36:55.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:54 smithi104 bash[9203]: audit 2020-09-08T15:36:53.691396+0000 mon.b (mon.1) 71 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "osd crush dump", "format": "json"}]: dispatch 2020-09-08T15:36:55.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:54 smithi104 bash[9203]: audit 2020-09-08T15:36:53.694532+0000 mon.a (mon.0) 886 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config set","value":"crush-compat","who":"mgr","name":"mgr/balancer/mode"}]: dispatch 2020-09-08T15:36:55.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:54 smithi104 bash[9203]: audit 2020-09-08T15:36:53.694645+0000 mon.b (mon.1) 72 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config set","value":"crush-compat","who":"mgr","name":"mgr/balancer/mode"}]: dispatch 2020-09-08T15:36:55.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:54 smithi104 bash[9203]: audit 2020-09-08T15:36:53.701581+0000 mon.a (mon.0) 887 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config set","value":"crush-compat","who":"mgr","name":"mgr/balancer/mode"}]': finished 2020-09-08T15:36:55.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:54 smithi104 bash[9203]: audit 2020-09-08T15:36:53.714086+0000 mon.a (mon.0) 888 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:55.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:54 smithi104 bash[9203]: audit 2020-09-08T15:36:53.714188+0000 mon.b (mon.1) 73 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:55.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:54 smithi104 bash[9203]: audit 2020-09-08T15:36:53.718823+0000 mon.a (mon.0) 889 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:36:55.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:54 smithi104 bash[9203]: audit 2020-09-08T15:36:54.049310+0000 mgr.x (mgr.24118) 279 : audit [DBG] from='client.24661 v1:172.21.15.104:0/1847354615' entity='client.admin' cmd=[{"prefix": "balancer on", "target": ["mgr", ""]}]: dispatch 2020-09-08T15:36:55.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:55 smithi104 bash[9203]: cluster 2020-09-08T15:36:55.576443+0000 mgr.x (mgr.24118) 280 : cluster [DBG] pgmap v322: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:56.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:55 smithi166 bash[9446]: cluster 2020-09-08T15:36:55.576443+0000 mgr.x (mgr.24118) 280 : cluster [DBG] pgmap v322: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:56.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:55 smithi055 bash[9296]: cluster 2020-09-08T15:36:55.576443+0000 mgr.x (mgr.24118) 280 : cluster [DBG] pgmap v322: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:57.088 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:56 smithi104 bash[9203]: cluster 2020-09-08T15:36:55.322381+0000 osd.2 (osd.2) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:57.089 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:56 smithi104 bash[9203]: cluster 2020-09-08T15:36:55.322485+0000 osd.2 (osd.2) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:57.089 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:56 smithi104 bash[9203]: cluster 2020-09-08T15:36:55.696066+0000 mon.a (mon.0) 890 : cluster [DBG] osdmap e119: 12 total, 12 up, 12 in 2020-09-08T15:36:57.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:56 smithi166 bash[9446]: cluster 2020-09-08T15:36:55.322381+0000 osd.2 (osd.2) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:57.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:56 smithi166 bash[9446]: cluster 2020-09-08T15:36:55.322485+0000 osd.2 (osd.2) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:57.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:56 smithi166 bash[9446]: cluster 2020-09-08T15:36:55.696066+0000 mon.a (mon.0) 890 : cluster [DBG] osdmap e119: 12 total, 12 up, 12 in 2020-09-08T15:36:57.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:56 smithi055 bash[9296]: cluster 2020-09-08T15:36:55.322381+0000 osd.2 (osd.2) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:36:57.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:56 smithi055 bash[9296]: cluster 2020-09-08T15:36:55.322485+0000 osd.2 (osd.2) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:36:57.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:56 smithi055 bash[9296]: cluster 2020-09-08T15:36:55.696066+0000 mon.a (mon.0) 890 : cluster [DBG] osdmap e119: 12 total, 12 up, 12 in 2020-09-08T15:36:57.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:57 smithi104 bash[9203]: audit 2020-09-08T15:36:57.508805+0000 mon.a (mon.0) 891 : audit [INF] from='client.? v1:172.21.15.104:0/3487816932' entity='client.admin' cmd=[{"prefix": "config set", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes", "value": "true"}]: dispatch 2020-09-08T15:36:57.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:57 smithi104 bash[9203]: audit 2020-09-08T15:36:57.516799+0000 mon.a (mon.0) 892 : audit [INF] from='client.? v1:172.21.15.104:0/3487816932' entity='client.admin' cmd='[{"prefix": "config set", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes", "value": "true"}]': finished 2020-09-08T15:36:57.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:57 smithi104 bash[9203]: audit 2020-09-08T15:36:57.529948+0000 mon.a (mon.0) 893 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:57.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:57 smithi104 bash[9203]: audit 2020-09-08T15:36:57.529972+0000 mon.b (mon.1) 74 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:57.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:57 smithi104 bash[9203]: audit 2020-09-08T15:36:57.533305+0000 mon.a (mon.0) 894 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:36:57.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:36:57 smithi104 bash[9203]: cluster 2020-09-08T15:36:57.577071+0000 mgr.x (mgr.24118) 281 : cluster [DBG] pgmap v324: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:58.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:57 smithi166 bash[9446]: audit 2020-09-08T15:36:57.508805+0000 mon.a (mon.0) 891 : audit [INF] from='client.? v1:172.21.15.104:0/3487816932' entity='client.admin' cmd=[{"prefix": "config set", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes", "value": "true"}]: dispatch 2020-09-08T15:36:58.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:57 smithi166 bash[9446]: audit 2020-09-08T15:36:57.516799+0000 mon.a (mon.0) 892 : audit [INF] from='client.? v1:172.21.15.104:0/3487816932' entity='client.admin' cmd='[{"prefix": "config set", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes", "value": "true"}]': finished 2020-09-08T15:36:58.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:57 smithi166 bash[9446]: audit 2020-09-08T15:36:57.529948+0000 mon.a (mon.0) 893 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:58.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:57 smithi166 bash[9446]: audit 2020-09-08T15:36:57.529972+0000 mon.b (mon.1) 74 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:58.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:57 smithi166 bash[9446]: audit 2020-09-08T15:36:57.533305+0000 mon.a (mon.0) 894 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:36:58.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:36:57 smithi166 bash[9446]: cluster 2020-09-08T15:36:57.577071+0000 mgr.x (mgr.24118) 281 : cluster [DBG] pgmap v324: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:58.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:57 smithi055 bash[9296]: audit 2020-09-08T15:36:57.508805+0000 mon.a (mon.0) 891 : audit [INF] from='client.? v1:172.21.15.104:0/3487816932' entity='client.admin' cmd=[{"prefix": "config set", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes", "value": "true"}]: dispatch 2020-09-08T15:36:58.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:57 smithi055 bash[9296]: audit 2020-09-08T15:36:57.516799+0000 mon.a (mon.0) 892 : audit [INF] from='client.? v1:172.21.15.104:0/3487816932' entity='client.admin' cmd='[{"prefix": "config set", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes", "value": "true"}]': finished 2020-09-08T15:36:58.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:57 smithi055 bash[9296]: audit 2020-09-08T15:36:57.529948+0000 mon.a (mon.0) 893 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:58.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:57 smithi055 bash[9296]: audit 2020-09-08T15:36:57.529972+0000 mon.b (mon.1) 74 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:36:58.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:57 smithi055 bash[9296]: audit 2020-09-08T15:36:57.533305+0000 mon.a (mon.0) 894 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:36:58.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:36:57 smithi055 bash[9296]: cluster 2020-09-08T15:36:57.577071+0000 mgr.x (mgr.24118) 281 : cluster [DBG] pgmap v324: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:36:58.225 INFO:tasks.thrashosds:Beginning thrashosds... 2020-09-08T15:36:58.226 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2020-09-08T15:36:58.227 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph pg dump --format=json 2020-09-08T15:36:58.365 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:37:00.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:00 smithi166 bash[9446]: cluster 2020-09-08T15:36:59.577651+0000 mgr.x (mgr.24118) 282 : cluster [DBG] pgmap v325: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:00.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:00 smithi055 bash[9296]: cluster 2020-09-08T15:36:59.577651+0000 mgr.x (mgr.24118) 282 : cluster [DBG] pgmap v325: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:00.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:00 smithi104 bash[9203]: cluster 2020-09-08T15:36:59.577651+0000 mgr.x (mgr.24118) 282 : cluster [DBG] pgmap v325: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:01.548 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T15:37:01.551 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2020-09-08T15:37:01.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:01 smithi104 bash[9203]: audit 2020-09-08T15:37:01.540221+0000 mgr.x (mgr.24118) 283 : audit [DBG] from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:37:01.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:01 smithi104 bash[9203]: cluster 2020-09-08T15:37:01.578150+0000 mgr.x (mgr.24118) 284 : cluster [DBG] pgmap v326: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:02.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:01 smithi166 bash[9446]: audit 2020-09-08T15:37:01.540221+0000 mgr.x (mgr.24118) 283 : audit [DBG] from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:37:02.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:01 smithi166 bash[9446]: cluster 2020-09-08T15:37:01.578150+0000 mgr.x (mgr.24118) 284 : cluster [DBG] pgmap v326: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:02.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:01 smithi055 bash[9296]: audit 2020-09-08T15:37:01.540221+0000 mgr.x (mgr.24118) 283 : audit [DBG] from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:37:02.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:01 smithi055 bash[9296]: cluster 2020-09-08T15:37:01.578150+0000 mgr.x (mgr.24118) 284 : cluster [DBG] pgmap v326: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:02.302 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2020-09-08T15:36:59.577323+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":12,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":1124941824,"kb_used":73040,"kb_used_data":8112,"kb_used_omap":0,"kb_used_meta":64832,"kb_avail":1124868784,"statfs":{"total":1151940427776,"available":1151865634816,"internally_reserved":0,"allocated":8306688,"data_stored":4720888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66387968},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.903539"},"pg_stats":[{"pgid":"1.0","version":"99'6","reported_seq":"126","reported_epoch":"118","state":"active+clean","last_fresh":"2020-09-08T15:36:53.856661+0000","last_change":"2020-09-08T15:36:24.184987+0000","last_active":"2020-09-08T15:36:53.856661+0000","last_peered":"2020-09-08T15:36:53.856661+0000","last_clean":"2020-09-08T15:36:53.856661+0000","last_became_active":"2020-09-08T15:34:43.031697+0000","last_became_peered":"2020-09-08T15:34:43.031697+0000","last_unstale":"2020-09-08T15:36:53.856661+0000","last_undegraded":"2020-09-08T15:36:53.856661+0000","last_fullsized":"2020-09-08T15:36:53.856661+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":2,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'6","last_scrub_stamp":"2020-09-08T15:36:24.184849+0000","last_deep_scrub":"99'6","last_deep_scrub_stamp":"2020-09-08T15:35:23.132465+0000","last_clean_scrub_stamp":"2020-09-08T15:36:24.184849+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":2,"acting":2,"num_store_stats":7}],"osd_stats":[{"osd":11,"up_from":0,"seq":31,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5744,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739408,"statfs":{"total":95995035648,"available":95989153792,"internally_reserved":0,"allocated":696320,"data_stored":395883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":2.2389999999999999,"5min":1.927,"15min":1.875},"min":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"max":{"1min":3.2730000000000001,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.7649999999999999},{"interface":"front","average":{"1min":3.4089999999999998,"5min":3.331,"15min":3.3180000000000001},"min":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"max":{"1min":4.4699999999999998,"5min":4.5970000000000004,"15min":4.5970000000000004},"last":3.2770000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":2.0129999999999999,"5min":2.0499999999999998,"15min":2.056},"min":{"1min":1.4330000000000001,"5min":1.395,"15min":1.395},"max":{"1min":2.629,"5min":3.2250000000000001,"15min":3.2250000000000001},"last":1.8260000000000001},{"interface":"front","average":{"1min":2.3279999999999998,"5min":2.5790000000000002,"15min":2.621},"min":{"1min":1.363,"5min":1.363,"15min":1.363},"max":{"1min":3.2629999999999999,"5min":3.8580000000000001,"15min":3.8580000000000001},"last":2.5179999999999998}]},{"osd":2,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":2.3769999999999998,"5min":2.141,"15min":2.101},"min":{"1min":1.5760000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"max":{"1min":4.3940000000000001,"5min":4.3940000000000001,"15min":4.3940000000000001},"last":2.0049999999999999},{"interface":"front","average":{"1min":3.0459999999999998,"5min":2.903,"15min":2.879},"min":{"1min":1.702,"5min":1.702,"15min":1.702},"max":{"1min":4.2720000000000002,"5min":4.2939999999999996,"15min":4.2939999999999996},"last":3.1469999999999998}]},{"osd":3,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":2.7810000000000001,"5min":2.419,"15min":2.359},"min":{"1min":1.665,"5min":1.4670000000000001,"15min":1.4670000000000001},"max":{"1min":3.8820000000000001,"5min":3.8820000000000001,"15min":3.8820000000000001},"last":2.7090000000000001},{"interface":"front","average":{"1min":3.1880000000000002,"5min":3.1469999999999998,"15min":3.1400000000000001},"min":{"1min":1.454,"5min":1.454,"15min":1.454},"max":{"1min":4.4409999999999998,"5min":4.8949999999999996,"15min":4.8949999999999996},"last":3.5099999999999998}]},{"osd":4,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":3.4159999999999999,"5min":3.5710000000000002,"15min":3.597},"min":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"max":{"1min":5.0300000000000002,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":2.927},{"interface":"front","average":{"1min":2.9649999999999999,"5min":2.8370000000000002,"15min":2.8159999999999998},"min":{"1min":1.7290000000000001,"5min":1.673,"15min":1.673},"max":{"1min":4.7320000000000002,"5min":4.7320000000000002,"15min":4.7320000000000002},"last":2.601}]},{"osd":5,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":3.427,"5min":3.4340000000000002,"15min":3.4350000000000001},"min":{"1min":2.262,"5min":2.262,"15min":2.262},"max":{"1min":4.4119999999999999,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":3.6920000000000002},{"interface":"front","average":{"1min":3.7869999999999999,"5min":3.9569999999999999,"15min":3.9849999999999999},"min":{"1min":2.5510000000000002,"5min":2.5059999999999998,"15min":2.5059999999999998},"max":{"1min":5.1829999999999998,"5min":6.3630000000000004,"15min":6.3630000000000004},"last":4.2009999999999996}]},{"osd":6,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":3.5590000000000002,"5min":3.46,"15min":3.444},"min":{"1min":2.4910000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"max":{"1min":5.101,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":3.5710000000000002},{"interface":"front","average":{"1min":3.827,"5min":3.6619999999999999,"15min":3.6339999999999999},"min":{"1min":1.877,"5min":1.877,"15min":1.877},"max":{"1min":4.9199999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":4.0720000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":3.8029999999999999,"5min":3.7679999999999998,"15min":3.762},"min":{"1min":2.673,"5min":2.673,"15min":2.673},"max":{"1min":5.1139999999999999,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":3.3370000000000002},{"interface":"front","average":{"1min":3.9630000000000001,"5min":3.9049999999999998,"15min":3.895},"min":{"1min":2.4390000000000001,"5min":2.4390000000000001,"15min":2.4390000000000001},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":4.4560000000000004}]},{"osd":8,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":4.2069999999999999,"5min":4.1509999999999998,"15min":4.1420000000000003},"min":{"1min":3.2360000000000002,"5min":2.923,"15min":2.923},"max":{"1min":5.2030000000000003,"5min":5.944,"15min":5.944},"last":4.407},{"interface":"front","average":{"1min":3.8620000000000001,"5min":3.8980000000000001,"15min":3.9039999999999999},"min":{"1min":2.403,"5min":2.1469999999999998,"15min":2.1469999999999998},"max":{"1min":5.117,"5min":6.2400000000000002,"15min":6.2400000000000002},"last":3.0110000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":4.3639999999999999,"5min":4.4710000000000001,"15min":4.4889999999999999},"min":{"1min":3.472,"5min":3.0459999999999998,"15min":3.0459999999999998},"max":{"1min":5.383,"5min":6.673,"15min":6.673},"last":3.8479999999999999},{"interface":"front","average":{"1min":4.3280000000000003,"5min":4.298,"15min":4.2930000000000001},"min":{"1min":2.9430000000000001,"5min":2.7040000000000002,"15min":2.7040000000000002},"max":{"1min":5.6609999999999996,"5min":6.6859999999999999,"15min":6.6859999999999999},"last":3.7999999999999998}]}]},{"osd":10,"up_from":0,"seq":37,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5808,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739344,"statfs":{"total":95995035648,"available":95989088256,"internally_reserved":0,"allocated":696320,"data_stored":395883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7330000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":1.921,"5min":2.3119999999999998,"15min":2.3769999999999998},"min":{"1min":1.5109999999999999,"5min":1.419,"15min":1.419},"max":{"1min":2.4969999999999999,"5min":4.3380000000000001,"15min":4.3380000000000001},"last":1.6759999999999999},{"interface":"front","average":{"1min":2.6440000000000001,"5min":2.5750000000000002,"15min":2.5630000000000002},"min":{"1min":1.6739999999999999,"5min":1.4750000000000001,"15min":1.4750000000000001},"max":{"1min":3.7759999999999998,"5min":5.1029999999999998,"15min":5.1029999999999998},"last":1.952}]},{"osd":2,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":2.3109999999999999,"5min":2.2599999999999998,"15min":2.2519999999999998},"min":{"1min":1.6519999999999999,"5min":1.508,"15min":1.508},"max":{"1min":3.0939999999999999,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":2.1619999999999999},{"interface":"front","average":{"1min":2.1970000000000001,"5min":2.6960000000000002,"15min":2.7789999999999999},"min":{"1min":1.492,"5min":1.4359999999999999,"15min":1.4359999999999999},"max":{"1min":3.4260000000000002,"5min":4.3769999999999998,"15min":4.3769999999999998},"last":3.1469999999999998}]},{"osd":3,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":1.9159999999999999,"5min":2.2090000000000001,"15min":2.258},"min":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"max":{"1min":2.6469999999999998,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":1.8100000000000001},{"interface":"front","average":{"1min":2.7200000000000002,"5min":3.016,"15min":3.0649999999999999},"min":{"1min":1.609,"5min":1.609,"15min":1.609},"max":{"1min":3.605,"5min":4.2140000000000004,"15min":4.2140000000000004},"last":2.5640000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.1469999999999998,"5min":3.4039999999999999,"15min":3.4470000000000001},"min":{"1min":1.8,"5min":1.8,"15min":1.8},"max":{"1min":4.0720000000000001,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":2.6899999999999999},{"interface":"front","average":{"1min":2.9780000000000002,"5min":3.0649999999999999,"15min":3.0790000000000002},"min":{"1min":1.958,"5min":1.958,"15min":1.958},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":2.8530000000000002}]},{"osd":5,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.218,"5min":3.5510000000000002,"15min":3.6070000000000002},"min":{"1min":2.653,"5min":2.427,"15min":2.427},"max":{"1min":4.1059999999999999,"5min":4.7160000000000002,"15min":4.7160000000000002},"last":2.9500000000000002},{"interface":"front","average":{"1min":3.6600000000000001,"5min":3.8199999999999998,"15min":3.847},"min":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"max":{"1min":4.7089999999999996,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":3.4089999999999998}]},{"osd":6,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.3300000000000001,"5min":3.5339999999999998,"15min":3.5680000000000001},"min":{"1min":2.3359999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"max":{"1min":4.7949999999999999,"5min":4.7949999999999999,"15min":4.7949999999999999},"last":3.6459999999999999},{"interface":"front","average":{"1min":3.6800000000000002,"5min":3.8620000000000001,"15min":3.8919999999999999},"min":{"1min":3.0270000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"max":{"1min":4.4939999999999998,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":3.3279999999999998}]},{"osd":7,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.7400000000000002,"5min":3.907,"15min":3.9350000000000001},"min":{"1min":2.3410000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"max":{"1min":4.7220000000000004,"5min":7.0490000000000004,"15min":7.0490000000000004},"last":3.2440000000000002},{"interface":"front","average":{"1min":3.9009999999999998,"5min":4.077,"15min":4.1059999999999999},"min":{"1min":3.1789999999999998,"5min":2.806,"15min":2.806},"max":{"1min":4.8929999999999998,"5min":5.0899999999999999,"15min":5.0899999999999999},"last":3.7770000000000001}]},{"osd":8,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":4.0149999999999997,"5min":4.1459999999999999,"15min":4.1680000000000001},"min":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"max":{"1min":5.3390000000000004,"5min":5.7949999999999999,"15min":5.7949999999999999},"last":4.1890000000000001},{"interface":"front","average":{"1min":4.0670000000000002,"5min":4.0309999999999997,"15min":4.0250000000000004},"min":{"1min":2.6920000000000002,"5min":1.901,"15min":1.901},"max":{"1min":5.1200000000000001,"5min":5.6509999999999998,"15min":5.6509999999999998},"last":4.1109999999999998}]},{"osd":9,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":4.2160000000000002,"5min":4.3780000000000001,"15min":4.4050000000000002},"min":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"max":{"1min":5.4020000000000001,"5min":5.8369999999999997,"15min":5.8369999999999997},"last":4.5119999999999996},{"interface":"front","average":{"1min":4.2629999999999999,"5min":4.2149999999999999,"15min":4.2069999999999999},"min":{"1min":2.5590000000000002,"5min":2.4969999999999999,"15min":2.4969999999999999},"max":{"1min":5.3129999999999997,"5min":5.3129999999999997,"15min":5.3129999999999997},"last":4.3360000000000003}]},{"osd":11,"last update":"Tue Sep 8 15:36:47 2020","interfaces":[{"interface":"back","average":{"1min":4.3899999999999997,"5min":4.3890000000000002,"15min":4.3890000000000002},"min":{"1min":3.7909999999999999,"5min":1.831,"15min":1.831},"max":{"1min":5.3940000000000001,"5min":5.6980000000000004,"15min":5.6980000000000004},"last":3.8980000000000001},{"interface":"front","average":{"1min":4.5609999999999999,"5min":4.4500000000000002,"15min":4.4320000000000004},"min":{"1min":3.8860000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"max":{"1min":5.5069999999999997,"5min":5.5069999999999997,"15min":5.5069999999999997},"last":4.5599999999999996}]}]},{"osd":9,"up_from":0,"seq":40,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5872,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739280,"statfs":{"total":95995035648,"available":95989022720,"internally_reserved":0,"allocated":696320,"data_stored":395883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":2.129,"5min":1.9119999999999999,"15min":1.8759999999999999},"min":{"1min":1.4390000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"max":{"1min":3.1259999999999999,"5min":3.149,"15min":3.149},"last":2.2280000000000002},{"interface":"front","average":{"1min":1.9490000000000001,"5min":1.9119999999999999,"15min":1.9059999999999999},"min":{"1min":1.3580000000000001,"5min":1.2549999999999999,"15min":1.2549999999999999},"max":{"1min":4.149,"5min":4.149,"15min":4.149},"last":2.1539999999999999}]},{"osd":1,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":2.173,"5min":1.9930000000000001,"15min":1.9630000000000001},"min":{"1min":1.3120000000000001,"5min":1.3009999999999999,"15min":1.3009999999999999},"max":{"1min":3.6269999999999998,"5min":3.6269999999999998,"15min":3.6269999999999998},"last":1.607},{"interface":"front","average":{"1min":2.4889999999999999,"5min":2.3250000000000002,"15min":2.298},"min":{"1min":1.474,"5min":1.266,"15min":1.266},"max":{"1min":3.9950000000000001,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":2.4009999999999998}]},{"osd":2,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":2.7210000000000001,"5min":2.5219999999999998,"15min":2.4889999999999999},"min":{"1min":1.4359999999999999,"5min":1.327,"15min":1.327},"max":{"1min":4.1020000000000003,"5min":4.508,"15min":4.508},"last":1.5289999999999999},{"interface":"front","average":{"1min":2.5489999999999999,"5min":2.3330000000000002,"15min":2.2970000000000002},"min":{"1min":1.4259999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"max":{"1min":3.6800000000000002,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":1.6830000000000001}]},{"osd":3,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":2.395,"5min":2.351,"15min":2.343},"min":{"1min":1.3420000000000001,"5min":1.198,"15min":1.198},"max":{"1min":4.3760000000000003,"5min":4.3760000000000003,"15min":4.3760000000000003},"last":2.8279999999999998},{"interface":"front","average":{"1min":2.629,"5min":2.641,"15min":2.6429999999999998},"min":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"max":{"1min":4.4550000000000001,"5min":4.4550000000000001,"15min":4.4550000000000001},"last":2.6440000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":3.0470000000000002,"5min":2.7570000000000001,"15min":2.7090000000000001},"min":{"1min":1.905,"5min":1.1819999999999999,"15min":1.1819999999999999},"max":{"1min":5.0279999999999996,"5min":5.0279999999999996,"15min":5.0279999999999996},"last":2.351},{"interface":"front","average":{"1min":3.5390000000000001,"5min":3.1040000000000001,"15min":3.032},"min":{"1min":2.5750000000000002,"5min":1.883,"15min":1.883},"max":{"1min":4.6280000000000001,"5min":4.6280000000000001,"15min":4.6280000000000001},"last":3.637}]},{"osd":5,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":3.452,"5min":3.2149999999999999,"15min":3.1749999999999998},"min":{"1min":2.6259999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"max":{"1min":5.0519999999999996,"5min":6.3840000000000003,"15min":6.3840000000000003},"last":2.9350000000000001},{"interface":"front","average":{"1min":3.456,"5min":3.2240000000000002,"15min":3.1859999999999999},"min":{"1min":1.907,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":4.875,"5min":4.875,"15min":4.875},"last":3.2360000000000002}]},{"osd":6,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":3.7589999999999999,"5min":3.3109999999999999,"15min":3.2370000000000001},"min":{"1min":2.3820000000000001,"5min":1.4179999999999999,"15min":1.4179999999999999},"max":{"1min":5.3659999999999997,"5min":5.3659999999999997,"15min":5.3659999999999997},"last":3.9089999999999998},{"interface":"front","average":{"1min":3.621,"5min":3.3090000000000002,"15min":3.2570000000000001},"min":{"1min":2.1720000000000002,"5min":1.8959999999999999,"15min":1.8959999999999999},"max":{"1min":5.7130000000000001,"5min":5.7130000000000001,"15min":5.7130000000000001},"last":3.1280000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":3.9399999999999999,"5min":3.5,"15min":3.427},"min":{"1min":1.968,"5min":1.7949999999999999,"15min":1.7949999999999999},"max":{"1min":5.1459999999999999,"5min":5.1459999999999999,"15min":5.1459999999999999},"last":3.7269999999999999},{"interface":"front","average":{"1min":4.0599999999999996,"5min":3.7000000000000002,"15min":3.6400000000000001},"min":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"max":{"1min":5.7629999999999999,"5min":5.7629999999999999,"15min":5.7629999999999999},"last":4.0279999999999996}]},{"osd":8,"last update":"Tue Sep 8 15:35:57 2020","interfaces":[{"interface":"back","average":{"1min":3.7400000000000002,"5min":3.278,"15min":3.2010000000000001},"min":{"1min":2.2879999999999998,"5min":1.3180000000000001,"15min":1.3180000000000001},"max":{"1min":4.9119999999999999,"5min":5.3049999999999997,"15min":5.3049999999999997},"last":2.544},{"interface":"front","average":{"1min":4.0499999999999998,"5min":3.3969999999999998,"15min":3.2879999999999998},"min":{"1min":2.335,"5min":1.444,"15min":1.444},"max":{"1min":5.4199999999999999,"5min":6.0140000000000002,"15min":6.0140000000000002},"last":4.3780000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:36:18 2020","interfaces":[{"interface":"back","average":{"1min":3.9500000000000002,"5min":3.9350000000000001,"15min":3.9329999999999998},"min":{"1min":3.3839999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"max":{"1min":4.8780000000000001,"5min":5.5629999999999997,"15min":5.5629999999999997},"last":3.3410000000000002},{"interface":"front","average":{"1min":3.9649999999999999,"5min":3.8239999999999998,"15min":3.7999999999999998},"min":{"1min":3.3130000000000002,"5min":2.056,"15min":2.056},"max":{"1min":4.6379999999999999,"5min":5.492,"15min":5.492},"last":2.7309999999999999}]}]},{"osd":8,"up_from":0,"seq":46,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5924,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":93739228,"statfs":{"total":95995035648,"available":95988969472,"internally_reserved":0,"allocated":684032,"data_stored":388416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.2749999999999999,"5min":2.202,"15min":2.1920000000000002},"min":{"1min":1.3799999999999999,"5min":1.266,"15min":1.266},"max":{"1min":3.2200000000000002,"5min":3.5830000000000002,"15min":3.5830000000000002},"last":3.262},{"interface":"front","average":{"1min":2.5459999999999998,"5min":2.5070000000000001,"15min":2.4300000000000002},"min":{"1min":1.7669999999999999,"5min":1.7250000000000001,"15min":1.7250000000000001},"max":{"1min":3.444,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":2.0760000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.492,"5min":2.0659999999999998,"15min":1.8839999999999999},"min":{"1min":1.4690000000000001,"5min":1.137,"15min":1.137},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":3.4540000000000002},{"interface":"front","average":{"1min":2.7200000000000002,"5min":2.427,"15min":2.347},"min":{"1min":1.351,"5min":1.046,"15min":1.046},"max":{"1min":4.444,"5min":4.444,"15min":4.444},"last":2.7770000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.2330000000000001,"5min":1.9410000000000001,"15min":1.845},"min":{"1min":1.4379999999999999,"5min":1.27,"15min":1.27},"max":{"1min":3.9129999999999998,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":3.125},{"interface":"front","average":{"1min":2.4889999999999999,"5min":2.504,"15min":2.46},"min":{"1min":1.208,"5min":1.208,"15min":1.208},"max":{"1min":3.7509999999999999,"5min":3.895,"15min":3.895},"last":1.8480000000000001}]},{"osd":3,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.9750000000000001,"5min":2.7869999999999999,"15min":2.7370000000000001},"min":{"1min":1.587,"5min":1.2789999999999999,"15min":1.2789999999999999},"max":{"1min":4.3929999999999998,"5min":4.3929999999999998,"15min":4.3929999999999998},"last":1.9590000000000001},{"interface":"front","average":{"1min":2.4649999999999999,"5min":2.4260000000000002,"15min":2.3820000000000001},"min":{"1min":1.131,"5min":1.131,"15min":1.131},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":2.867}]},{"osd":4,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":3.149,"5min":2.9870000000000001,"15min":2.875},"min":{"1min":2.3029999999999999,"5min":1.4730000000000001,"15min":1.4730000000000001},"max":{"1min":4.4669999999999996,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":3.609},{"interface":"front","average":{"1min":2.8820000000000001,"5min":2.6869999999999998,"15min":2.5579999999999998},"min":{"1min":1.734,"5min":1.2549999999999999,"15min":1.2549999999999999},"max":{"1min":4.2649999999999997,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":2.278}]},{"osd":5,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":3.5529999999999999,"5min":3.294,"15min":3.1619999999999999},"min":{"1min":1.944,"5min":1.74,"15min":1.74},"max":{"1min":5.6109999999999998,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":3.6960000000000002},{"interface":"front","average":{"1min":3.7360000000000002,"5min":3.419,"15min":3.3719999999999999},"min":{"1min":2.2080000000000002,"5min":1.762,"15min":1.762},"max":{"1min":6.0549999999999997,"5min":6.0549999999999997,"15min":6.0549999999999997},"last":3.8069999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":3.8090000000000002,"5min":3.4199999999999999,"15min":3.2639999999999998},"min":{"1min":2.4980000000000002,"5min":1.9299999999999999,"15min":1.9299999999999999},"max":{"1min":5.1929999999999996,"5min":5.6180000000000003,"15min":5.6180000000000003},"last":3.9980000000000002},{"interface":"front","average":{"1min":3.2839999999999998,"5min":3.0499999999999998,"15min":2.9220000000000002},"min":{"1min":2.0019999999999998,"5min":1.669,"15min":1.669},"max":{"1min":4.7279999999999998,"5min":4.7279999999999998,"15min":4.7279999999999998},"last":2.6190000000000002}]},{"osd":7,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":4.0419999999999998,"5min":3.7269999999999999,"15min":3.6240000000000001},"min":{"1min":2.3849999999999998,"5min":2.0630000000000002,"15min":2.0630000000000002},"max":{"1min":5.3680000000000003,"5min":5.3680000000000003,"15min":5.3680000000000003},"last":4.1870000000000003},{"interface":"front","average":{"1min":3.6429999999999998,"5min":3.3679999999999999,"15min":3.2440000000000002},"min":{"1min":2.7269999999999999,"5min":2.0830000000000002,"15min":2.0830000000000002},"max":{"1min":5.1029999999999998,"5min":5.141,"15min":5.141},"last":2.9630000000000001}]},{"osd":9,"last update":"Tue Sep 8 15:35:56 2020","interfaces":[{"interface":"back","average":{"1min":3.9590000000000001,"5min":3.4990000000000001,"15min":3.4220000000000002},"min":{"1min":2.4180000000000001,"5min":1.157,"15min":1.157},"max":{"1min":5.1840000000000002,"5min":5.1840000000000002,"15min":5.1840000000000002},"last":3.8879999999999999},{"interface":"front","average":{"1min":4.1639999999999997,"5min":3.4660000000000002,"15min":3.3500000000000001},"min":{"1min":2.0219999999999998,"5min":1.1919999999999999,"15min":1.1919999999999999},"max":{"1min":5.3959999999999999,"5min":5.3959999999999999,"15min":5.3959999999999999},"last":4.375}]},{"osd":10,"last update":"Tue Sep 8 15:36:27 2020","interfaces":[{"interface":"back","average":{"1min":3.7200000000000002,"5min":3.7320000000000002,"15min":3.734},"min":{"1min":2.0510000000000002,"5min":1.3740000000000001,"15min":1.3740000000000001},"max":{"1min":5.5190000000000001,"5min":5.5190000000000001,"15min":5.5190000000000001},"last":3.379},{"interface":"front","average":{"1min":4.1180000000000003,"5min":4.0830000000000002,"15min":4.077},"min":{"1min":2.7669999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"max":{"1min":5.3049999999999997,"5min":5.3049999999999997,"15min":5.3049999999999997},"last":4.0679999999999996}]}]},{"osd":7,"up_from":0,"seq":52,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6064,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93739088,"statfs":{"total":95995035648,"available":95988826112,"internally_reserved":0,"allocated":696320,"data_stored":395903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:02 2020","interfaces":[{"interface":"back","average":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"min":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"max":{"1min":4.0190000000000001,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":2.2189999999999999},{"interface":"front","average":{"1min":2.6259999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"min":{"1min":1.542,"5min":1.542,"15min":1.542},"max":{"1min":4.5229999999999997,"5min":4.5229999999999997,"15min":4.5229999999999997},"last":2.3079999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.661,"5min":2.468,"15min":2.355},"min":{"1min":2.1309999999999998,"5min":1.498,"15min":1.498},"max":{"1min":3.3940000000000001,"5min":3.8780000000000001,"15min":3.8780000000000001},"last":2.2690000000000001},{"interface":"front","average":{"1min":2.1699999999999999,"5min":2.359,"15min":2.3599999999999999},"min":{"1min":1.7749999999999999,"5min":1.631,"15min":1.631},"max":{"1min":2.6800000000000002,"5min":3.633,"15min":3.633},"last":2.3519999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.4049999999999998,"5min":2.528,"15min":2.4399999999999999},"min":{"1min":1.899,"5min":1.504,"15min":1.504},"max":{"1min":3.2240000000000002,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":2.423},{"interface":"front","average":{"1min":2.468,"5min":2.879,"15min":2.8370000000000002},"min":{"1min":1.982,"5min":1.982,"15min":1.982},"max":{"1min":3.2570000000000001,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":3.2069999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.7869999999999999,"5min":2.9849999999999999,"15min":2.891},"min":{"1min":1.861,"5min":1.861,"15min":1.861},"max":{"1min":3.8460000000000001,"5min":4.875,"15min":4.875},"last":2.8210000000000002},{"interface":"front","average":{"1min":2.8159999999999998,"5min":3.254,"15min":3.234},"min":{"1min":1.978,"5min":1.5660000000000001,"15min":1.5660000000000001},"max":{"1min":3.8330000000000002,"5min":5.3929999999999998,"15min":5.3929999999999998},"last":2.6840000000000002}]},{"osd":4,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":3.0150000000000001,"5min":3.0289999999999999,"15min":2.9329999999999998},"min":{"1min":2.181,"5min":1.9279999999999999,"15min":1.9279999999999999},"max":{"1min":3.98,"5min":5.9729999999999999,"15min":5.9729999999999999},"last":3.3029999999999999},{"interface":"front","average":{"1min":3.3780000000000001,"5min":3.456,"15min":3.3100000000000001},"min":{"1min":2.5150000000000001,"5min":2.464,"15min":2.464},"max":{"1min":4.4260000000000002,"5min":5.6470000000000002,"15min":5.6470000000000002},"last":4.2160000000000002}]},{"osd":5,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":3.4790000000000001,"5min":3.7400000000000002,"15min":3.6859999999999999},"min":{"1min":2.794,"5min":2.794,"15min":2.794},"max":{"1min":4.593,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":3.6779999999999999},{"interface":"front","average":{"1min":3.3769999999999998,"5min":3.5590000000000002,"15min":3.5129999999999999},"min":{"1min":2.7080000000000002,"5min":2.0859999999999999,"15min":2.0859999999999999},"max":{"1min":4.4779999999999998,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":3.544}]},{"osd":6,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":3.5529999999999999,"5min":3.9319999999999999,"15min":3.9369999999999998},"min":{"1min":2.7309999999999999,"5min":2.6320000000000001,"15min":2.6320000000000001},"max":{"1min":4.7599999999999998,"5min":7.3070000000000004,"15min":7.3070000000000004},"last":4.1310000000000002},{"interface":"front","average":{"1min":3.9039999999999999,"5min":4.0140000000000002,"15min":3.9180000000000001},"min":{"1min":3.1349999999999998,"5min":2.6549999999999998,"15min":2.6549999999999998},"max":{"1min":4.6470000000000002,"5min":7.2480000000000002,"15min":7.2480000000000002},"last":4.6280000000000001}]},{"osd":8,"last update":"Tue Sep 8 15:36:33 2020","interfaces":[{"interface":"back","average":{"1min":4.1619999999999999,"5min":4.2220000000000004,"15min":4.1609999999999996},"min":{"1min":2.7250000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"max":{"1min":5.5300000000000002,"5min":6.9530000000000003,"15min":6.9530000000000003},"last":4.548},{"interface":"front","average":{"1min":4.444,"5min":4.4059999999999997,"15min":4.2830000000000004},"min":{"1min":3.5579999999999998,"5min":2.8479999999999999,"15min":2.8479999999999999},"max":{"1min":5.6929999999999996,"5min":7.6509999999999998,"15min":7.6509999999999998},"last":5.0960000000000001}]},{"osd":9,"last update":"Tue Sep 8 15:36:55 2020","interfaces":[{"interface":"back","average":{"1min":4.7350000000000003,"5min":4.7800000000000002,"15min":4.7759999999999998},"min":{"1min":3.282,"5min":3.0030000000000001,"15min":3.0030000000000001},"max":{"1min":5.9299999999999997,"5min":6.524,"15min":6.524},"last":5.5259999999999998},{"interface":"front","average":{"1min":4.008,"5min":4.3319999999999999,"15min":4.4139999999999997},"min":{"1min":3.0489999999999999,"5min":3.0489999999999999,"15min":3.0489999999999999},"max":{"1min":5.3200000000000003,"5min":6.274,"15min":6.274},"last":4.4199999999999999}]},{"osd":10,"last update":"Tue Sep 8 15:36:22 2020","interfaces":[{"interface":"back","average":{"1min":4.2460000000000004,"5min":4.649,"15min":4.7160000000000002},"min":{"1min":3.371,"5min":3.218,"15min":3.218},"max":{"1min":5.3390000000000004,"5min":7.7800000000000002,"15min":7.7800000000000002},"last":5.0309999999999997},{"interface":"front","average":{"1min":4.8840000000000003,"5min":5.1219999999999999,"15min":5.1619999999999999},"min":{"1min":3.7410000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"max":{"1min":5.8760000000000003,"5min":8.3650000000000002,"15min":8.3650000000000002},"last":5.7770000000000001}]},{"osd":11,"last update":"Tue Sep 8 15:36:46 2020","interfaces":[{"interface":"back","average":{"1min":4.4409999999999998,"5min":4.758,"15min":4.8109999999999999},"min":{"1min":3.0190000000000001,"5min":3.0190000000000001,"15min":3.0190000000000001},"max":{"1min":5.7329999999999997,"5min":6.1589999999999998,"15min":6.1589999999999998},"last":5.4340000000000002},{"interface":"front","average":{"1min":5.0099999999999998,"5min":5.2549999999999999,"15min":5.2960000000000003},"min":{"1min":3.4260000000000002,"5min":3.4260000000000002,"15min":3.4260000000000002},"max":{"1min":6.069,"5min":6.742,"15min":6.742},"last":5.9820000000000002}]}]},{"osd":6,"up_from":0,"seq":55,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6128,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93739024,"statfs":{"total":95995035648,"available":95988760576,"internally_reserved":0,"allocated":696320,"data_stored":395903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:47 2020","interfaces":[{"interface":"back","average":{"1min":2.2770000000000001,"5min":2.1339999999999999,"15min":1.9299999999999999},"min":{"1min":1.6699999999999999,"5min":1.141,"15min":1.141},"max":{"1min":3.1709999999999998,"5min":3.714,"15min":3.714},"last":2.532},{"interface":"front","average":{"1min":2.3849999999999998,"5min":2.145,"15min":1.998},"min":{"1min":1.7170000000000001,"5min":1.256,"15min":1.256},"max":{"1min":3.6970000000000001,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":2.6480000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:36:47 2020","interfaces":[{"interface":"back","average":{"1min":2.2410000000000001,"5min":2.194,"15min":2.1819999999999999},"min":{"1min":1.7350000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"max":{"1min":3.645,"5min":3.645,"15min":3.645},"last":2.7610000000000001},{"interface":"front","average":{"1min":2.9260000000000002,"5min":2.6899999999999999,"15min":2.4729999999999999},"min":{"1min":1.9319999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":3.8500000000000001,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":3.9980000000000002}]},{"osd":2,"last update":"Tue Sep 8 15:36:47 2020","interfaces":[{"interface":"back","average":{"1min":2.7730000000000001,"5min":2.6709999999999998,"15min":2.4510000000000001},"min":{"1min":1.819,"5min":1.462,"15min":1.462},"max":{"1min":4.4619999999999997,"5min":4.4619999999999997,"15min":4.4619999999999997},"last":3.4489999999999998},{"interface":"front","average":{"1min":3.0350000000000001,"5min":2.7109999999999999,"15min":2.4350000000000001},"min":{"1min":1.9339999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"max":{"1min":4.4009999999999998,"5min":4.4009999999999998,"15min":4.4009999999999998},"last":4.4509999999999996}]},{"osd":3,"last update":"Tue Sep 8 15:36:47 2020","interfaces":[{"interface":"back","average":{"1min":3.0499999999999998,"5min":2.8029999999999999,"15min":2.488},"min":{"1min":2.0590000000000002,"5min":1.7809999999999999,"15min":1.7809999999999999},"max":{"1min":3.7989999999999999,"5min":4.8109999999999999,"15min":4.8109999999999999},"last":4.3390000000000004},{"interface":"front","average":{"1min":3.5129999999999999,"5min":3.2429999999999999,"15min":2.9140000000000001},"min":{"1min":2.5310000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"max":{"1min":4.7430000000000003,"5min":4.8120000000000003,"15min":4.8120000000000003},"last":5.234}]},{"osd":4,"last update":"Tue Sep 8 15:36:47 2020","interfaces":[{"interface":"back","average":{"1min":3.4039999999999999,"5min":3.093,"15min":2.7210000000000001},"min":{"1min":2.141,"5min":1.607,"15min":1.607},"max":{"1min":4.7309999999999999,"5min":4.7309999999999999,"15min":4.7309999999999999},"last":4.9980000000000002},{"interface":"front","average":{"1min":3.5299999999999998,"5min":3.23,"15min":2.9449999999999998},"min":{"1min":2.5470000000000002,"5min":1.5049999999999999,"15min":1.5049999999999999},"max":{"1min":4.3079999999999998,"5min":4.8769999999999998,"15min":4.8769999999999998},"last":6.1950000000000003}]},{"osd":5,"last update":"Tue Sep 8 15:36:47 2020","interfaces":[{"interface":"back","average":{"1min":4.032,"5min":3.5430000000000001,"15min":3.2050000000000001},"min":{"1min":2.8849999999999998,"5min":1.7509999999999999,"15min":1.7509999999999999},"max":{"1min":5.8460000000000001,"5min":6.1269999999999998,"15min":6.1269999999999998},"last":5.3099999999999996},{"interface":"front","average":{"1min":3.855,"5min":3.548,"15min":3.21},"min":{"1min":2.9670000000000001,"5min":2.1659999999999999,"15min":2.1659999999999999},"max":{"1min":4.9580000000000002,"5min":6.1820000000000004,"15min":6.1820000000000004},"last":5.5259999999999998}]},{"osd":7,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":4.0549999999999997,"5min":3.8460000000000001,"15min":3.714},"min":{"1min":3.2879999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"max":{"1min":4.8970000000000002,"5min":5.4050000000000002,"15min":5.4050000000000002},"last":5.6310000000000002},{"interface":"front","average":{"1min":4.5359999999999996,"5min":4.0469999999999997,"15min":3.8090000000000002},"min":{"1min":3.387,"5min":2.1640000000000001,"15min":2.1640000000000001},"max":{"1min":5.5609999999999999,"5min":6.2679999999999998,"15min":6.2679999999999998},"last":6.5300000000000002}]},{"osd":8,"last update":"Tue Sep 8 15:36:30 2020","interfaces":[{"interface":"back","average":{"1min":4.3600000000000003,"5min":4.0759999999999996,"15min":3.9500000000000002},"min":{"1min":3.258,"5min":3.0579999999999998,"15min":3.0579999999999998},"max":{"1min":5.6639999999999997,"5min":5.6639999999999997,"15min":5.6639999999999997},"last":4.8399999999999999},{"interface":"front","average":{"1min":4.2759999999999998,"5min":4.2309999999999999,"15min":4.1929999999999996},"min":{"1min":3.5259999999999998,"5min":3.1269999999999998,"15min":3.1269999999999998},"max":{"1min":5.3819999999999997,"5min":5.5499999999999998,"15min":5.5499999999999998},"last":6.0199999999999996}]},{"osd":9,"last update":"Tue Sep 8 15:36:55 2020","interfaces":[{"interface":"back","average":{"1min":4.7510000000000003,"5min":4.5620000000000003,"15min":4.4690000000000003},"min":{"1min":3.8860000000000001,"5min":3.1110000000000002,"15min":3.1110000000000002},"max":{"1min":6.4059999999999997,"5min":6.4059999999999997,"15min":6.4059999999999997},"last":5.774},{"interface":"front","average":{"1min":4.641,"5min":4.5419999999999998,"15min":4.5279999999999996},"min":{"1min":3.367,"5min":3.367,"15min":3.367},"max":{"1min":6.1139999999999999,"5min":6.1139999999999999,"15min":6.1139999999999999},"last":6.1139999999999999}]},{"osd":10,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":4.6909999999999998,"5min":4.694,"15min":4.694},"min":{"1min":3.7730000000000001,"5min":3.4590000000000001,"15min":3.4590000000000001},"max":{"1min":6.0720000000000001,"5min":6.0720000000000001,"15min":6.0720000000000001},"last":6.6630000000000003},{"interface":"front","average":{"1min":4.6159999999999997,"5min":4.6260000000000003,"15min":4.6280000000000001},"min":{"1min":3.4660000000000002,"5min":3.4159999999999999,"15min":3.4159999999999999},"max":{"1min":5.6980000000000004,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":6.9790000000000001}]}]},{"osd":1,"up_from":0,"seq":80,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6308,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738844,"statfs":{"total":95995035648,"available":95988576256,"internally_reserved":0,"allocated":684032,"data_stored":388436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:46 2020","interfaces":[{"interface":"back","average":{"1min":2.2669999999999999,"5min":1.9710000000000001,"15min":1.456},"min":{"1min":1.0129999999999999,"5min":0.99299999999999999,"15min":0.75600000000000001},"max":{"1min":3.3319999999999999,"5min":3.363,"15min":3.363},"last":2.2869999999999999},{"interface":"front","average":{"1min":2.6240000000000001,"5min":2.274,"15min":1.6240000000000001},"min":{"1min":1.6739999999999999,"5min":1.1399999999999999,"15min":0.75800000000000001},"max":{"1min":3.601,"5min":3.698,"15min":3.698},"last":4.1779999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:36:04 2020","interfaces":[{"interface":"back","average":{"1min":2.5539999999999998,"5min":2.153,"15min":1.712},"min":{"1min":1.3460000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":4.4080000000000004,"5min":4.4080000000000004,"15min":4.4080000000000004},"last":3.3050000000000002},{"interface":"front","average":{"1min":2.7290000000000001,"5min":2.2770000000000001,"15min":1.8680000000000001},"min":{"1min":1.431,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":4.6559999999999997,"5min":4.6559999999999997,"15min":4.6559999999999997},"last":2.8570000000000002}]},{"osd":3,"last update":"Tue Sep 8 15:36:34 2020","interfaces":[{"interface":"back","average":{"1min":2.7509999999999999,"5min":2.5230000000000001,"15min":2.2770000000000001},"min":{"1min":0.99299999999999999,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":4.0469999999999997,"5min":4.5119999999999996,"15min":4.5119999999999996},"last":2.21},{"interface":"front","average":{"1min":2.593,"5min":2.548,"15min":2.4079999999999999},"min":{"1min":1.746,"5min":1.2410000000000001,"15min":1.2410000000000001},"max":{"1min":4.2069999999999999,"5min":4.2069999999999999,"15min":4.2069999999999999},"last":2.6949999999999998}]},{"osd":4,"last update":"Tue Sep 8 15:36:51 2020","interfaces":[{"interface":"back","average":{"1min":2.2589999999999999,"5min":2.3279999999999998,"15min":2.1309999999999998},"min":{"1min":1.7250000000000001,"5min":1.387,"15min":1.387},"max":{"1min":3.681,"5min":5.1829999999999998,"15min":5.1829999999999998},"last":2.3370000000000002},{"interface":"front","average":{"1min":3.3500000000000001,"5min":3.0369999999999999,"15min":2.7989999999999999},"min":{"1min":1.321,"5min":1.321,"15min":1.321},"max":{"1min":4.8700000000000001,"5min":4.8700000000000001,"15min":4.8700000000000001},"last":3.7879999999999998}]},{"osd":5,"last update":"Tue Sep 8 15:36:14 2020","interfaces":[{"interface":"back","average":{"1min":3.0859999999999999,"5min":3.089,"15min":2.9449999999999998},"min":{"1min":2.105,"5min":1.8879999999999999,"15min":1.8879999999999999},"max":{"1min":4.6440000000000001,"5min":4.6440000000000001,"15min":4.6440000000000001},"last":3.0070000000000001},{"interface":"front","average":{"1min":2.6019999999999999,"5min":2.5470000000000002,"15min":2.4409999999999998},"min":{"1min":1.9410000000000001,"5min":1.5189999999999999,"15min":1.5189999999999999},"max":{"1min":3.532,"5min":4.6319999999999997,"15min":4.6319999999999997},"last":2.8100000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:36:40 2020","interfaces":[{"interface":"back","average":{"1min":3.9529999999999998,"5min":3.7440000000000002,"15min":3.621},"min":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"max":{"1min":5.367,"5min":6.1399999999999997,"15min":6.1399999999999997},"last":4.6029999999999998},{"interface":"front","average":{"1min":3.8079999999999998,"5min":3.6299999999999999,"15min":3.5609999999999999},"min":{"1min":2.5640000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"max":{"1min":5.7069999999999999,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":3.1680000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:36:09 2020","interfaces":[{"interface":"back","average":{"1min":3.9950000000000001,"5min":3.4540000000000002,"15min":3.2250000000000001},"min":{"1min":2.8860000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"max":{"1min":5.3239999999999998,"5min":5.3239999999999998,"15min":5.3239999999999998},"last":3.5550000000000002},{"interface":"front","average":{"1min":3.9510000000000001,"5min":3.9100000000000001,"15min":3.883},"min":{"1min":1.907,"5min":1.907,"15min":1.907},"max":{"1min":5.375,"5min":5.5650000000000004,"15min":5.5650000000000004},"last":5.6929999999999996}]},{"osd":8,"last update":"Tue Sep 8 15:36:34 2020","interfaces":[{"interface":"back","average":{"1min":4.1890000000000001,"5min":4.2190000000000003,"15min":4.2930000000000001},"min":{"1min":2.6419999999999999,"5min":1.7290000000000001,"15min":1.7290000000000001},"max":{"1min":5.6340000000000003,"5min":5.6340000000000003,"15min":5.6340000000000003},"last":4.093},{"interface":"front","average":{"1min":4.2210000000000001,"5min":4.0700000000000003,"15min":4.0800000000000001},"min":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"max":{"1min":6.0960000000000001,"5min":6.0960000000000001,"15min":6.0960000000000001},"last":4.9740000000000002}]},{"osd":9,"last update":"Tue Sep 8 15:35:58 2020","interfaces":[{"interface":"back","average":{"1min":3.5009999999999999,"5min":3.2919999999999998,"15min":3.2570000000000001},"min":{"1min":2.423,"5min":2.089,"15min":2.089},"max":{"1min":4.3250000000000002,"5min":4.8789999999999996,"15min":4.8789999999999996},"last":3.5099999999999998},{"interface":"front","average":{"1min":4.3659999999999997,"5min":4.2560000000000002,"15min":4.2380000000000004},"min":{"1min":3.3410000000000002,"5min":2.649,"15min":2.649},"max":{"1min":6.7380000000000004,"5min":6.7380000000000004,"15min":6.7380000000000004},"last":5.3289999999999997}]},{"osd":10,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":4.3220000000000001,"5min":4.2320000000000002,"15min":4.2169999999999996},"min":{"1min":2.7949999999999999,"5min":2.3210000000000002,"15min":2.3210000000000002},"max":{"1min":5.968,"5min":5.968,"15min":5.968},"last":3.879},{"interface":"front","average":{"1min":3.4620000000000002,"5min":3.4889999999999999,"15min":3.4929999999999999},"min":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"max":{"1min":4.9640000000000004,"5min":4.9790000000000001,"15min":4.9790000000000001},"last":3.9289999999999998}]}]},{"osd":0,"up_from":0,"seq":84,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6372,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738780,"statfs":{"total":95995035648,"available":95988510720,"internally_reserved":0,"allocated":684032,"data_stored":388436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 8 15:36:42 2020","interfaces":[{"interface":"back","average":{"1min":2.407,"5min":1.9450000000000001,"15min":1.4330000000000001},"min":{"1min":1.6279999999999999,"5min":1.02,"15min":0.76100000000000001},"max":{"1min":4.3339999999999996,"5min":4.3339999999999996,"15min":4.3339999999999996},"last":1.7410000000000001},{"interface":"front","average":{"1min":2.121,"5min":1.9319999999999999,"15min":1.46},"min":{"1min":1.6619999999999999,"5min":1.2050000000000001,"15min":0.71999999999999997},"max":{"1min":2.71,"5min":3.8300000000000001,"15min":3.8300000000000001},"last":1.7769999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:34:05 2020","interfaces":[{"interface":"back","average":{"1min":1.9950000000000001,"5min":1.621,"15min":1.5189999999999999},"min":{"1min":1.3999999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.8519999999999999,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":2.6589999999999998},{"interface":"front","average":{"1min":2.629,"5min":1.956,"15min":1.72},"min":{"1min":1.6419999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":2.4049999999999998}]},{"osd":3,"last update":"Tue Sep 8 15:36:32 2020","interfaces":[{"interface":"back","average":{"1min":2.6760000000000002,"5min":2.617,"15min":2.992},"min":{"1min":1.8460000000000001,"5min":1.1879999999999999,"15min":1.1879999999999999},"max":{"1min":4.3140000000000001,"5min":27.298999999999999,"15min":27.298999999999999},"last":2.048},{"interface":"front","average":{"1min":3.4289999999999998,"5min":3.3639999999999999,"15min":3.6219999999999999},"min":{"1min":2.2069999999999999,"5min":2.0179999999999998,"15min":2.0179999999999998},"max":{"1min":5.2590000000000003,"5min":28.314,"15min":28.314},"last":2.403}]},{"osd":4,"last update":"Tue Sep 8 15:35:53 2020","interfaces":[{"interface":"back","average":{"1min":3.0139999999999998,"5min":2.8250000000000002,"15min":2.4809999999999999},"min":{"1min":2.1240000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"max":{"1min":3.8199999999999998,"5min":4.4390000000000001,"15min":4.4390000000000001},"last":2.3279999999999998},{"interface":"front","average":{"1min":3.3740000000000001,"5min":3.0680000000000001,"15min":2.7549999999999999},"min":{"1min":2.5139999999999998,"5min":1.8080000000000001,"15min":1.8080000000000001},"max":{"1min":4.8129999999999997,"5min":5.1050000000000004,"15min":5.1050000000000004},"last":2.1429999999999998}]},{"osd":5,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":3.8079999999999998,"5min":3.3399999999999999,"15min":3.0089999999999999},"min":{"1min":2.6400000000000001,"5min":2.0990000000000002,"15min":2.0990000000000002},"max":{"1min":4.9809999999999999,"5min":4.9809999999999999,"15min":4.9809999999999999},"last":2.2269999999999999},{"interface":"front","average":{"1min":2.3839999999999999,"5min":3.0259999999999998,"15min":3.1840000000000002},"min":{"1min":1.845,"5min":1.7110000000000001,"15min":1.7110000000000001},"max":{"1min":3.0459999999999998,"5min":5.7439999999999998,"15min":5.7439999999999998},"last":1.891}]},{"osd":6,"last update":"Tue Sep 8 15:36:42 2020","interfaces":[{"interface":"back","average":{"1min":2.8679999999999999,"5min":3.2770000000000001,"15min":3.5209999999999999},"min":{"1min":1.746,"5min":1.6439999999999999,"15min":1.6439999999999999},"max":{"1min":5.8730000000000002,"5min":5.8730000000000002,"15min":5.8730000000000002},"last":2.8580000000000001},{"interface":"front","average":{"1min":3.1899999999999999,"5min":3.6749999999999998,"15min":3.923},"min":{"1min":2.5350000000000001,"5min":1.9099999999999999,"15min":1.9099999999999999},"max":{"1min":6.2560000000000002,"5min":6.2560000000000002,"15min":6.2560000000000002},"last":2.4769999999999999}]},{"osd":7,"last update":"Tue Sep 8 15:36:08 2020","interfaces":[{"interface":"back","average":{"1min":3.2669999999999999,"5min":4.069,"15min":4.3120000000000003},"min":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"max":{"1min":4.758,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":2.27},{"interface":"front","average":{"1min":3.9319999999999999,"5min":4.0860000000000003,"15min":4.1040000000000001},"min":{"1min":2.585,"5min":2.585,"15min":2.585},"max":{"1min":5.6429999999999998,"5min":5.8659999999999997,"15min":5.8659999999999997},"last":2.5150000000000001}]},{"osd":8,"last update":"Tue Sep 8 15:36:32 2020","interfaces":[{"interface":"back","average":{"1min":4.5739999999999998,"5min":4.1109999999999998,"15min":3.9489999999999998},"min":{"1min":3.0329999999999999,"5min":2.6629999999999998,"15min":2.6629999999999998},"max":{"1min":6.5049999999999999,"5min":6.6379999999999999,"15min":6.6379999999999999},"last":2.7839999999999998},{"interface":"front","average":{"1min":3.395,"5min":3.9039999999999999,"15min":4.133},"min":{"1min":2.5049999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"max":{"1min":4.5540000000000003,"5min":6.2030000000000003,"15min":6.2030000000000003},"last":2.6640000000000001}]},{"osd":9,"last update":"Tue Sep 8 15:35:53 2020","interfaces":[{"interface":"back","average":{"1min":4.2889999999999997,"5min":4.2850000000000001,"15min":4.2850000000000001},"min":{"1min":2.734,"5min":2.714,"15min":2.714},"max":{"1min":5.7770000000000001,"5min":6.1769999999999996,"15min":6.1769999999999996},"last":2.7069999999999999},{"interface":"front","average":{"1min":3.472,"5min":4,"15min":4.0880000000000001},"min":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"max":{"1min":5.4790000000000001,"5min":5.4790000000000001,"15min":5.4790000000000001},"last":3.089}]},{"osd":10,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":3.3039999999999998,"5min":3.157,"15min":3.133},"min":{"1min":2.649,"5min":2.3210000000000002,"15min":2.3210000000000002},"max":{"1min":4.1529999999999996,"5min":4.1680000000000001,"15min":4.1680000000000001},"last":3.2360000000000002},{"interface":"front","average":{"1min":4.944,"5min":4.4969999999999999,"15min":4.423},"min":{"1min":3.452,"5min":2.8340000000000001,"15min":2.8340000000000001},"max":{"1min":6.758,"5min":6.758,"15min":6.758},"last":2.9359999999999999}]},{"osd":11,"last update":"Tue Sep 8 15:36:42 2020","interfaces":[{"interface":"back","average":{"1min":4.1379999999999999,"5min":3.9380000000000002,"15min":3.9049999999999998},"min":{"1min":3.1640000000000001,"5min":2.9460000000000002,"15min":2.9460000000000002},"max":{"1min":6.1159999999999997,"5min":6.1159999999999997,"15min":6.1159999999999997},"last":3.3479999999999999},{"interface":"front","average":{"1min":4.8639999999999999,"5min":4.8609999999999998,"15min":4.8609999999999998},"min":{"1min":3.5800000000000001,"5min":2.911,"15min":2.911},"max":{"1min":6.407,"5min":7.0220000000000002,"15min":7.0220000000000002},"last":3.04}]}]},{"osd":2,"up_from":0,"seq":76,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6256,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738896,"statfs":{"total":95995035648,"available":95988629504,"internally_reserved":0,"allocated":696320,"data_stored":395903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.1970000000000001,"5min":1.9179999999999999,"15min":1.623},"min":{"1min":1.4259999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":3.883,"5min":3.883,"15min":3.883},"last":2.2789999999999999},{"interface":"front","average":{"1min":2.0030000000000001,"5min":1.9419999999999999,"15min":1.702},"min":{"1min":1.599,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.7160000000000002,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":2.3250000000000002}]},{"osd":1,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.0390000000000001,"5min":1.915,"15min":1.6870000000000001},"min":{"1min":1.306,"5min":0.999,"15min":0.999},"max":{"1min":3.363,"5min":3.363,"15min":3.363},"last":2.9660000000000002},{"interface":"front","average":{"1min":2.6629999999999998,"5min":2.4340000000000002,"15min":2.0899999999999999},"min":{"1min":1.2809999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"max":{"1min":3.456,"5min":3.9399999999999999,"15min":3.9399999999999999},"last":2.7120000000000002}]},{"osd":3,"last update":"Tue Sep 8 15:36:32 2020","interfaces":[{"interface":"back","average":{"1min":2.944,"5min":2.706,"15min":2.3799999999999999},"min":{"1min":1.7130000000000001,"5min":1.587,"15min":1.587},"max":{"1min":4.1429999999999998,"5min":4.3209999999999997,"15min":4.3209999999999997},"last":2.8780000000000001},{"interface":"front","average":{"1min":2.7490000000000001,"5min":2.5600000000000001,"15min":2.3679999999999999},"min":{"1min":1.9059999999999999,"5min":0.755,"15min":0.755},"max":{"1min":3.641,"5min":3.6709999999999998,"15min":3.6709999999999998},"last":2.8279999999999998}]},{"osd":4,"last update":"Tue Sep 8 15:36:54 2020","interfaces":[{"interface":"back","average":{"1min":3.1080000000000001,"5min":2.8239999999999998,"15min":2.492},"min":{"1min":1.6479999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"max":{"1min":5.0890000000000004,"5min":5.0890000000000004,"15min":5.0890000000000004},"last":3.3809999999999998},{"interface":"front","average":{"1min":3.1360000000000001,"5min":2.8380000000000001,"15min":2.4460000000000002},"min":{"1min":1.6240000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"max":{"1min":4.4459999999999997,"5min":4.827,"15min":4.827},"last":2.4359999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":3.5569999999999999,"5min":3.222,"15min":2.9380000000000002},"min":{"1min":2.4060000000000001,"5min":1.353,"15min":1.353},"max":{"1min":4.6050000000000004,"5min":4.8449999999999998,"15min":4.8449999999999998},"last":3.2650000000000001},{"interface":"front","average":{"1min":3.7599999999999998,"5min":3.2879999999999998,"15min":2.98},"min":{"1min":2.4630000000000001,"5min":1.232,"15min":1.232},"max":{"1min":4.4530000000000003,"5min":5.2119999999999997,"15min":5.2119999999999997},"last":4.3099999999999996}]},{"osd":6,"last update":"Tue Sep 8 15:36:44 2020","interfaces":[{"interface":"back","average":{"1min":3.8450000000000002,"5min":3.4199999999999999,"15min":3.1880000000000002},"min":{"1min":2.9780000000000002,"5min":1.5069999999999999,"15min":1.5069999999999999},"max":{"1min":5.4980000000000002,"5min":5.4980000000000002,"15min":5.4980000000000002},"last":4.0170000000000003},{"interface":"front","average":{"1min":3.931,"5min":3.774,"15min":3.5670000000000002},"min":{"1min":3.339,"5min":2.0169999999999999,"15min":2.0169999999999999},"max":{"1min":5.8300000000000001,"5min":5.8300000000000001,"15min":5.8300000000000001},"last":3.9329999999999998}]},{"osd":7,"last update":"Tue Sep 8 15:36:09 2020","interfaces":[{"interface":"back","average":{"1min":4.4720000000000004,"5min":4.1779999999999999,"15min":4.0419999999999998},"min":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"max":{"1min":5.4340000000000002,"5min":6.5780000000000003,"15min":6.5780000000000003},"last":3.5009999999999999},{"interface":"front","average":{"1min":4.0419999999999998,"5min":3.7490000000000001,"15min":3.722},"min":{"1min":2.847,"5min":2.1299999999999999,"15min":2.1299999999999999},"max":{"1min":7.8659999999999997,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":4.4130000000000003}]},{"osd":8,"last update":"Tue Sep 8 15:36:26 2020","interfaces":[{"interface":"back","average":{"1min":3.9900000000000002,"5min":3.7370000000000001,"15min":3.6850000000000001},"min":{"1min":2.5369999999999999,"5min":1.907,"15min":1.907},"max":{"1min":5.5359999999999996,"5min":5.5359999999999996,"15min":5.5359999999999996},"last":4.4939999999999998},{"interface":"front","average":{"1min":4.2439999999999998,"5min":4.0410000000000004,"15min":3.9889999999999999},"min":{"1min":3.0699999999999998,"5min":2.7719999999999998,"15min":2.7719999999999998},"max":{"1min":5.4939999999999998,"5min":6.3150000000000004,"15min":6.3150000000000004},"last":4.9370000000000003}]},{"osd":9,"last update":"Tue Sep 8 15:35:59 2020","interfaces":[{"interface":"back","average":{"1min":4.0670000000000002,"5min":4.0499999999999998,"15min":4.0469999999999997},"min":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"max":{"1min":6.3559999999999999,"5min":6.3559999999999999,"15min":6.3559999999999999},"last":4.8070000000000004},{"interface":"front","average":{"1min":4.2220000000000004,"5min":4.2850000000000001,"15min":4.2949999999999999},"min":{"1min":2.71,"5min":2.71,"15min":2.71},"max":{"1min":6.1070000000000002,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":5.2409999999999997}]},{"osd":10,"last update":"Tue Sep 8 15:36:22 2020","interfaces":[{"interface":"back","average":{"1min":4.4729999999999999,"5min":4.1130000000000004,"15min":4.0529999999999999},"min":{"1min":3.145,"5min":2.7480000000000002,"15min":2.7480000000000002},"max":{"1min":6.915,"5min":6.915,"15min":6.915},"last":5.1040000000000001},{"interface":"front","average":{"1min":3.895,"5min":3.8140000000000001,"15min":3.7999999999999998},"min":{"1min":2.6890000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"max":{"1min":7.6580000000000004,"5min":7.6580000000000004,"15min":7.6580000000000004},"last":4.5670000000000002}]}]},{"osd":3,"up_from":0,"seq":70,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6180,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738972,"statfs":{"total":95995035648,"available":95988707328,"internally_reserved":0,"allocated":684032,"data_stored":388436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:31 2020","interfaces":[{"interface":"back","average":{"1min":2.1379999999999999,"5min":1.7210000000000001,"15min":1.629},"min":{"1min":1.79,"5min":1.1419999999999999,"15min":1.1419999999999999},"max":{"1min":2.7930000000000001,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":3.169},{"interface":"front","average":{"1min":2.6800000000000002,"5min":2.2309999999999999,"15min":2.0990000000000002},"min":{"1min":2.0270000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"max":{"1min":3.5409999999999999,"5min":3.5409999999999999,"15min":3.5409999999999999},"last":2.5579999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:36:35 2020","interfaces":[{"interface":"back","average":{"1min":2.133,"5min":2.0939999999999999,"15min":1.8740000000000001},"min":{"1min":1.2809999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"max":{"1min":3.3399999999999999,"5min":3.6829999999999998,"15min":3.6829999999999998},"last":1.9159999999999999},{"interface":"front","average":{"1min":2.1709999999999998,"5min":2.1400000000000001,"15min":1.8460000000000001},"min":{"1min":1.4059999999999999,"5min":1.341,"15min":1.341},"max":{"1min":3.1549999999999998,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":2.0390000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:36:35 2020","interfaces":[{"interface":"back","average":{"1min":2.5619999999999998,"5min":2.387,"15min":2.145},"min":{"1min":1.7669999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":3.5009999999999999,"5min":3.7810000000000001,"15min":3.7810000000000001},"last":2.3530000000000002},{"interface":"front","average":{"1min":2.1869999999999998,"5min":2.464,"15min":2.2389999999999999},"min":{"1min":1.585,"5min":1.3580000000000001,"15min":1.3580000000000001},"max":{"1min":3.1699999999999999,"5min":3.8010000000000002,"15min":3.8010000000000002},"last":1.9790000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:36:55 2020","interfaces":[{"interface":"back","average":{"1min":2.7280000000000002,"5min":2.7879999999999998,"15min":2.5609999999999999},"min":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"max":{"1min":3.8620000000000001,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":2.8359999999999999},{"interface":"front","average":{"1min":2.7879999999999998,"5min":2.8029999999999999,"15min":2.5110000000000001},"min":{"1min":2.0089999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"max":{"1min":3.6949999999999998,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":2.7639999999999998}]},{"osd":5,"last update":"Tue Sep 8 15:36:15 2020","interfaces":[{"interface":"back","average":{"1min":2.9300000000000002,"5min":2.9830000000000001,"15min":2.8530000000000002},"min":{"1min":1.323,"5min":1.323,"15min":1.323},"max":{"1min":3.9489999999999998,"5min":5.0289999999999999,"15min":5.0289999999999999},"last":3.0089999999999999},{"interface":"front","average":{"1min":3.4569999999999999,"5min":3.3929999999999998,"15min":3.2330000000000001},"min":{"1min":1.988,"5min":1.988,"15min":1.988},"max":{"1min":4.2910000000000004,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":3.2170000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:36:44 2020","interfaces":[{"interface":"back","average":{"1min":3.4249999999999998,"5min":3.444,"15min":3.3479999999999999},"min":{"1min":1.8680000000000001,"5min":1.714,"15min":1.714},"max":{"1min":4.6159999999999997,"5min":5.4160000000000004,"15min":5.4160000000000004},"last":3.6230000000000002},{"interface":"front","average":{"1min":3.544,"5min":3.6739999999999999,"15min":3.5449999999999999},"min":{"1min":2.093,"5min":2.093,"15min":2.093},"max":{"1min":4.694,"5min":5.3499999999999996,"15min":5.3499999999999996},"last":3.359}]},{"osd":7,"last update":"Tue Sep 8 15:36:11 2020","interfaces":[{"interface":"back","average":{"1min":4.234,"5min":4.2649999999999997,"15min":4.2530000000000001},"min":{"1min":3.081,"5min":2.8069999999999999,"15min":2.8069999999999999},"max":{"1min":5.1440000000000001,"5min":5.5640000000000001,"15min":5.5640000000000001},"last":4.6470000000000002},{"interface":"front","average":{"1min":3.577,"5min":3.629,"15min":3.617},"min":{"1min":2.601,"5min":2.601,"15min":2.601},"max":{"1min":4.258,"5min":5.258,"15min":5.258},"last":3.298}]},{"osd":8,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":3.8889999999999998,"5min":3.758,"15min":3.7829999999999999},"min":{"1min":2.9119999999999999,"5min":2.2440000000000002,"15min":2.2440000000000002},"max":{"1min":5.3390000000000004,"5min":5.4610000000000003,"15min":5.4610000000000003},"last":3.8149999999999999},{"interface":"front","average":{"1min":4.5220000000000002,"5min":4.3319999999999999,"15min":4.3079999999999998},"min":{"1min":3.206,"5min":2.351,"15min":2.351},"max":{"1min":5.5309999999999997,"5min":6.1239999999999997,"15min":6.1239999999999997},"last":4.0430000000000001}]},{"osd":9,"last update":"Tue Sep 8 15:36:55 2020","interfaces":[{"interface":"back","average":{"1min":3.6899999999999999,"5min":3.8969999999999998,"15min":3.9670000000000001},"min":{"1min":3.125,"5min":2.6709999999999998,"15min":2.6709999999999998},"max":{"1min":4.4359999999999999,"5min":4.6120000000000001,"15min":4.6120000000000001},"last":3.706},{"interface":"front","average":{"1min":4.093,"5min":4.1829999999999998,"15min":4.3369999999999997},"min":{"1min":2.8540000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"max":{"1min":5.2089999999999996,"5min":5.2530000000000001,"15min":5.2530000000000001},"last":4.2720000000000002}]},{"osd":10,"last update":"Tue Sep 8 15:36:21 2020","interfaces":[{"interface":"back","average":{"1min":4.6360000000000001,"5min":4.4450000000000003,"15min":4.4130000000000003},"min":{"1min":2.875,"5min":2.875,"15min":2.875},"max":{"1min":5.7880000000000003,"5min":5.7880000000000003,"15min":5.7880000000000003},"last":4.3920000000000003},{"interface":"front","average":{"1min":4.0990000000000002,"5min":3.8290000000000002,"15min":3.7839999999999998},"min":{"1min":3.194,"5min":2.8759999999999999,"15min":2.8759999999999999},"max":{"1min":4.7599999999999998,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":3.9020000000000001}]},{"osd":11,"last update":"Tue Sep 8 15:36:51 2020","interfaces":[{"interface":"back","average":{"1min":4.6799999999999997,"5min":4.7880000000000003,"15min":4.806},"min":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"max":{"1min":6.0590000000000002,"5min":6.0590000000000002,"15min":6.0590000000000002},"last":4.984},{"interface":"front","average":{"1min":4.2190000000000003,"5min":4.1600000000000001,"15min":4.1500000000000004},"min":{"1min":3.3090000000000002,"5min":3.04,"15min":3.04},"max":{"1min":5.2389999999999999,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":3.5089999999999999}]}]},{"osd":4,"up_from":0,"seq":67,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6192,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738960,"statfs":{"total":95995035648,"available":95988695040,"internally_reserved":0,"allocated":696320,"data_stored":395903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:54 2020","interfaces":[{"interface":"back","average":{"1min":2.4329999999999998,"5min":2.121,"15min":1.889},"min":{"1min":1.5880000000000001,"5min":1.01,"15min":1.01},"max":{"1min":5.7130000000000001,"5min":5.7130000000000001,"15min":5.7130000000000001},"last":1.9079999999999999},{"interface":"front","average":{"1min":2.5369999999999999,"5min":2.2469999999999999,"15min":1.851},"min":{"1min":1.6930000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":4.3650000000000002,"5min":4.3650000000000002,"15min":4.3650000000000002},"last":3.3999999999999999}]},{"osd":1,"last update":"Tue Sep 8 15:36:54 2020","interfaces":[{"interface":"back","average":{"1min":2.3460000000000001,"5min":2.1150000000000002,"15min":1.8779999999999999},"min":{"1min":1.6859999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"max":{"1min":3.4500000000000002,"5min":5.3659999999999997,"15min":5.3659999999999997},"last":1.964},{"interface":"front","average":{"1min":2.903,"5min":2.593,"15min":2.1269999999999998},"min":{"1min":1.8899999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"max":{"1min":3.9870000000000001,"5min":4.5700000000000003,"15min":4.5700000000000003},"last":1.8100000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:36:54 2020","interfaces":[{"interface":"back","average":{"1min":3.052,"5min":2.6909999999999998,"15min":2.1389999999999998},"min":{"1min":1.946,"5min":1.236,"15min":1.236},"max":{"1min":4.6150000000000002,"5min":4.6150000000000002,"15min":4.6150000000000002},"last":2.7160000000000002},{"interface":"front","average":{"1min":3.3380000000000001,"5min":2.8639999999999999,"15min":2.3250000000000002},"min":{"1min":1.8089999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"max":{"1min":5.2779999999999996,"5min":5.2779999999999996,"15min":5.2779999999999996},"last":3.202}]},{"osd":3,"last update":"Tue Sep 8 15:36:54 2020","interfaces":[{"interface":"back","average":{"1min":3.5179999999999998,"5min":3.0640000000000001,"15min":2.5219999999999998},"min":{"1min":2.5790000000000002,"5min":1.389,"15min":1.389},"max":{"1min":6.1429999999999998,"5min":6.1429999999999998,"15min":6.1429999999999998},"last":3.7919999999999998},{"interface":"front","average":{"1min":3.1669999999999998,"5min":2.8290000000000002,"15min":2.3700000000000001},"min":{"1min":2.0390000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"max":{"1min":5.3550000000000004,"5min":5.3550000000000004,"15min":5.3550000000000004},"last":3.6230000000000002}]},{"osd":5,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":3.9569999999999999,"5min":3.294,"15min":2.7519999999999998},"min":{"1min":2.7149999999999999,"5min":1.014,"15min":1.014},"max":{"1min":6.9800000000000004,"5min":6.9800000000000004,"15min":6.9800000000000004},"last":3.5270000000000001},{"interface":"front","average":{"1min":3.9449999999999998,"5min":3.274,"15min":2.8199999999999998},"min":{"1min":2.5800000000000001,"5min":1.129,"15min":1.129},"max":{"1min":6.2279999999999998,"5min":6.2279999999999998,"15min":6.2279999999999998},"last":3.8719999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:36:42 2020","interfaces":[{"interface":"back","average":{"1min":4.0700000000000003,"5min":3.6749999999999998,"15min":3.3490000000000002},"min":{"1min":2.9860000000000002,"5min":2.0449999999999999,"15min":2.0449999999999999},"max":{"1min":5.8179999999999996,"5min":6.6870000000000003,"15min":6.6870000000000003},"last":3.669},{"interface":"front","average":{"1min":3.8079999999999998,"5min":3.6160000000000001,"15min":3.3660000000000001},"min":{"1min":2.9980000000000002,"5min":2.2429999999999999,"15min":2.2429999999999999},"max":{"1min":5.2030000000000003,"5min":5.2030000000000003,"15min":5.2030000000000003},"last":3.3199999999999998}]},{"osd":7,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":4.1150000000000002,"5min":3.9860000000000002,"15min":3.8700000000000001},"min":{"1min":2.5739999999999998,"5min":1.75,"15min":1.75},"max":{"1min":7.4699999999999998,"5min":7.4699999999999998,"15min":7.4699999999999998},"last":4.0679999999999996},{"interface":"front","average":{"1min":4.0430000000000001,"5min":4.0570000000000004,"15min":3.98},"min":{"1min":2.718,"5min":2.6619999999999999,"15min":2.6619999999999999},"max":{"1min":6.6399999999999997,"5min":6.6399999999999997,"15min":6.6399999999999997},"last":4.1239999999999997}]},{"osd":8,"last update":"Tue Sep 8 15:36:30 2020","interfaces":[{"interface":"back","average":{"1min":4.1280000000000001,"5min":4,"15min":3.9359999999999999},"min":{"1min":3.327,"5min":3.0739999999999998,"15min":3.0739999999999998},"max":{"1min":5.1920000000000002,"5min":5.3929999999999998,"15min":5.3929999999999998},"last":4.2859999999999996},{"interface":"front","average":{"1min":4.681,"5min":4.4619999999999997,"15min":4.3799999999999999},"min":{"1min":3.746,"5min":3.0840000000000001,"15min":3.0840000000000001},"max":{"1min":6.5540000000000003,"5min":6.5540000000000003,"15min":6.5540000000000003},"last":4.1779999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:36:58 2020","interfaces":[{"interface":"back","average":{"1min":4.7400000000000002,"5min":4.4589999999999996,"15min":4.4020000000000001},"min":{"1min":3.403,"5min":2.6299999999999999,"15min":2.6299999999999999},"max":{"1min":7.319,"5min":7.319,"15min":7.319},"last":4.3630000000000004},{"interface":"front","average":{"1min":4.4610000000000003,"5min":4.4080000000000004,"15min":4.4219999999999997},"min":{"1min":3.5640000000000001,"5min":3.5289999999999999,"15min":3.5289999999999999},"max":{"1min":5.6040000000000001,"5min":5.7779999999999996,"15min":5.7779999999999996},"last":4.4569999999999999}]},{"osd":10,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":4.8129999999999997,"5min":4.9130000000000003,"15min":4.9299999999999997},"min":{"1min":3.9769999999999999,"5min":3.3439999999999999,"15min":3.3439999999999999},"max":{"1min":7.9279999999999999,"5min":7.9279999999999999,"15min":7.9279999999999999},"last":4.6040000000000001},{"interface":"front","average":{"1min":4.8440000000000003,"5min":4.9109999999999996,"15min":4.9219999999999997},"min":{"1min":3.7890000000000001,"5min":3.7890000000000001,"15min":3.7890000000000001},"max":{"1min":7.7590000000000003,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":4.5620000000000003}]}]},{"osd":5,"up_from":0,"seq":60,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6192,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738960,"statfs":{"total":95995035648,"available":95988695040,"internally_reserved":0,"allocated":696320,"data_stored":395903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":2.2589999999999999,"5min":2.1240000000000001,"15min":1.8740000000000001},"min":{"1min":1.7749999999999999,"5min":1.119,"15min":1.119},"max":{"1min":3.1320000000000001,"5min":3.96,"15min":3.96},"last":2.3740000000000001},{"interface":"front","average":{"1min":2.5790000000000002,"5min":2.2290000000000001,"15min":1.9670000000000001},"min":{"1min":1.929,"5min":1.4510000000000001,"15min":1.4510000000000001},"max":{"1min":3.7570000000000001,"5min":3.7570000000000001,"15min":3.7570000000000001},"last":2.2719999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.0049999999999999,"5min":2.5259999999999998,"15min":2.3370000000000002},"min":{"1min":1.643,"5min":1.339,"15min":1.339},"max":{"1min":4.2240000000000002,"5min":4.2240000000000002,"15min":4.2240000000000002},"last":2.4660000000000002},{"interface":"front","average":{"1min":2.1779999999999999,"5min":2.0760000000000001,"15min":2.0059999999999998},"min":{"1min":1.7090000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"max":{"1min":2.9830000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":3.7999999999999998}]},{"osd":2,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":2.456,"5min":2.2839999999999998,"15min":2.1339999999999999},"min":{"1min":1.8109999999999999,"5min":1.474,"15min":1.474},"max":{"1min":3.5539999999999998,"5min":3.7570000000000001,"15min":3.7570000000000001},"last":3.266},{"interface":"front","average":{"1min":3.4980000000000002,"5min":2.9969999999999999,"15min":2.7410000000000001},"min":{"1min":2.1240000000000001,"5min":1.796,"15min":1.796},"max":{"1min":4.9000000000000004,"5min":4.9000000000000004,"15min":4.9000000000000004},"last":3.3290000000000002}]},{"osd":3,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.7639999999999998,"5min":3.234,"15min":2.839},"min":{"1min":2.4249999999999998,"5min":1.6719999999999999,"15min":1.6719999999999999},"max":{"1min":4.718,"5min":5.3899999999999997,"15min":5.3899999999999997},"last":4.2080000000000002},{"interface":"front","average":{"1min":3.2280000000000002,"5min":3.1139999999999999,"15min":2.9129999999999998},"min":{"1min":2.3679999999999999,"5min":1.9990000000000001,"15min":1.9990000000000001},"max":{"1min":4.2050000000000001,"5min":5.4889999999999999,"15min":5.4889999999999999},"last":3.6459999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":2.9780000000000002,"5min":2.5710000000000002,"15min":2.3010000000000002},"min":{"1min":2.0910000000000002,"5min":1.6220000000000001,"15min":1.6220000000000001},"max":{"1min":3.5499999999999998,"5min":3.9710000000000001,"15min":3.9710000000000001},"last":3.4849999999999999},{"interface":"front","average":{"1min":3.7210000000000001,"5min":3.0779999999999998,"15min":2.6829999999999998},"min":{"1min":2.819,"5min":1.3979999999999999,"15min":1.3979999999999999},"max":{"1min":6.5190000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":4.048}]},{"osd":6,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":4.1189999999999998,"5min":3.819,"15min":3.609},"min":{"1min":3.2829999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"max":{"1min":4.718,"5min":5.9870000000000001,"15min":5.9870000000000001},"last":4.4809999999999999},{"interface":"front","average":{"1min":4.7560000000000002,"5min":3.6379999999999999,"15min":3.1400000000000001},"min":{"1min":3.766,"5min":1.948,"15min":1.948},"max":{"1min":5.5590000000000002,"5min":5.6600000000000001,"15min":5.6600000000000001},"last":5.1150000000000002}]},{"osd":7,"last update":"Tue Sep 8 15:36:03 2020","interfaces":[{"interface":"back","average":{"1min":4.8559999999999999,"5min":3.9359999999999999,"15min":3.617},"min":{"1min":3.1909999999999998,"5min":2.5859999999999999,"15min":2.5859999999999999},"max":{"1min":6.782,"5min":6.782,"15min":6.782},"last":4.609},{"interface":"front","average":{"1min":3.7669999999999999,"5min":3.9849999999999999,"15min":4.0099999999999998},"min":{"1min":2.665,"5min":2.665,"15min":2.665},"max":{"1min":4.8520000000000003,"5min":5.9059999999999997,"15min":5.9059999999999997},"last":5.3339999999999996}]},{"osd":8,"last update":"Tue Sep 8 15:36:34 2020","interfaces":[{"interface":"back","average":{"1min":3.621,"5min":3.677,"15min":3.5990000000000002},"min":{"1min":3.101,"5min":2.8940000000000001,"15min":2.8940000000000001},"max":{"1min":4.2649999999999997,"5min":5.2610000000000001,"15min":5.2610000000000001},"last":4.1239999999999997},{"interface":"front","average":{"1min":4.0570000000000004,"5min":4.0629999999999997,"15min":4.0529999999999999},"min":{"1min":3.4279999999999999,"5min":3.2400000000000002,"15min":3.2400000000000002},"max":{"1min":4.6230000000000002,"5min":5.3319999999999999,"15min":5.3319999999999999},"last":4.5309999999999997}]},{"osd":9,"last update":"Tue Sep 8 15:35:58 2020","interfaces":[{"interface":"back","average":{"1min":5.2080000000000002,"5min":4.6870000000000003,"15min":4.5999999999999996},"min":{"1min":4.1040000000000001,"5min":3.4550000000000001,"15min":3.4550000000000001},"max":{"1min":7.1719999999999997,"5min":7.1719999999999997,"15min":7.1719999999999997},"last":5.0030000000000001},{"interface":"front","average":{"1min":4.218,"5min":4.218,"15min":4.218},"min":{"1min":3.629,"5min":2.6930000000000001,"15min":2.6930000000000001},"max":{"1min":4.8799999999999999,"5min":5.6929999999999996,"15min":5.6929999999999996},"last":4.6799999999999997}]},{"osd":10,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":4.2569999999999997,"5min":4.2409999999999997,"15min":4.2380000000000004},"min":{"1min":3.3849999999999998,"5min":3.0190000000000001,"15min":3.0190000000000001},"max":{"1min":5.3609999999999998,"5min":5.3609999999999998,"15min":5.3609999999999998},"last":4.3150000000000004},{"interface":"front","average":{"1min":5.4649999999999999,"5min":4.9219999999999997,"15min":4.8319999999999999},"min":{"1min":3.9039999999999999,"5min":3.9039999999999999,"15min":3.9039999999999999},"max":{"1min":6.3390000000000004,"5min":6.3390000000000004,"15min":6.3390000000000004},"last":4.8979999999999997}]}]}]}} 2020-09-08T15:37:02.314 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph pg dump --format=json 2020-09-08T15:37:02.446 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:37:04.615 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:04 smithi166 bash[9446]: cluster 2020-09-08T15:37:03.267348+0000 osd.7 (osd.7) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:04.615 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:04 smithi166 bash[9446]: cluster 2020-09-08T15:37:03.267502+0000 osd.7 (osd.7) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:04.615 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:04 smithi166 bash[9446]: cluster 2020-09-08T15:37:03.323257+0000 mon.a (mon.0) 895 : cluster [DBG] osdmap e120: 12 total, 12 up, 12 in 2020-09-08T15:37:04.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:04 smithi166 bash[9446]: cluster 2020-09-08T15:37:03.578786+0000 mgr.x (mgr.24118) 285 : cluster [DBG] pgmap v328: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:04.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:04 smithi055 bash[9296]: cluster 2020-09-08T15:37:03.267348+0000 osd.7 (osd.7) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:04.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:04 smithi055 bash[9296]: cluster 2020-09-08T15:37:03.267502+0000 osd.7 (osd.7) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:04.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:04 smithi055 bash[9296]: cluster 2020-09-08T15:37:03.323257+0000 mon.a (mon.0) 895 : cluster [DBG] osdmap e120: 12 total, 12 up, 12 in 2020-09-08T15:37:04.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:04 smithi055 bash[9296]: cluster 2020-09-08T15:37:03.578786+0000 mgr.x (mgr.24118) 285 : cluster [DBG] pgmap v328: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:04.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:04 smithi104 bash[9203]: cluster 2020-09-08T15:37:03.267348+0000 osd.7 (osd.7) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:04.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:04 smithi104 bash[9203]: cluster 2020-09-08T15:37:03.267502+0000 osd.7 (osd.7) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:04.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:04 smithi104 bash[9203]: cluster 2020-09-08T15:37:03.323257+0000 mon.a (mon.0) 895 : cluster [DBG] osdmap e120: 12 total, 12 up, 12 in 2020-09-08T15:37:04.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:04 smithi104 bash[9203]: cluster 2020-09-08T15:37:03.578786+0000 mgr.x (mgr.24118) 285 : cluster [DBG] pgmap v328: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:05.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:05 smithi055 bash[9296]: cluster 2020-09-08T15:37:04.321212+0000 osd.0 (osd.0) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:05.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:05 smithi055 bash[9296]: cluster 2020-09-08T15:37:04.321308+0000 osd.0 (osd.0) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:05.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:05 smithi055 bash[9296]: cluster 2020-09-08T15:37:04.381759+0000 mon.a (mon.0) 896 : cluster [DBG] osdmap e121: 12 total, 12 up, 12 in 2020-09-08T15:37:05.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:05 smithi104 bash[9203]: cluster 2020-09-08T15:37:04.321212+0000 osd.0 (osd.0) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:05.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:05 smithi104 bash[9203]: cluster 2020-09-08T15:37:04.321308+0000 osd.0 (osd.0) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:05.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:05 smithi104 bash[9203]: cluster 2020-09-08T15:37:04.381759+0000 mon.a (mon.0) 896 : cluster [DBG] osdmap e121: 12 total, 12 up, 12 in 2020-09-08T15:37:05.686 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T15:37:05.689 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2020-09-08T15:37:05.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:05 smithi166 bash[9446]: cluster 2020-09-08T15:37:04.321212+0000 osd.0 (osd.0) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:05.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:05 smithi166 bash[9446]: cluster 2020-09-08T15:37:04.321308+0000 osd.0 (osd.0) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:05.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:05 smithi166 bash[9446]: cluster 2020-09-08T15:37:04.381759+0000 mon.a (mon.0) 896 : cluster [DBG] osdmap e121: 12 total, 12 up, 12 in 2020-09-08T15:37:06.378 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2020-09-08T15:37:05.579044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":12,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":1124941824,"kb_used":73176,"kb_used_data":8184,"kb_used_omap":0,"kb_used_meta":64896,"kb_avail":1124868648,"statfs":{"total":1151940427776,"available":1151865495552,"internally_reserved":0,"allocated":8380416,"data_stored":4765690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66453504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002281"},"pg_stats":[{"pgid":"1.0","version":"99'6","reported_seq":"126","reported_epoch":"118","state":"active+clean","last_fresh":"2020-09-08T15:36:53.856661+0000","last_change":"2020-09-08T15:36:24.184987+0000","last_active":"2020-09-08T15:36:53.856661+0000","last_peered":"2020-09-08T15:36:53.856661+0000","last_clean":"2020-09-08T15:36:53.856661+0000","last_became_active":"2020-09-08T15:34:43.031697+0000","last_became_peered":"2020-09-08T15:34:43.031697+0000","last_unstale":"2020-09-08T15:36:53.856661+0000","last_undegraded":"2020-09-08T15:36:53.856661+0000","last_fullsized":"2020-09-08T15:36:53.856661+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":2,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'6","last_scrub_stamp":"2020-09-08T15:36:24.184849+0000","last_deep_scrub":"99'6","last_deep_scrub_stamp":"2020-09-08T15:35:23.132465+0000","last_clean_scrub_stamp":"2020-09-08T15:36:24.184849+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":2,"acting":2,"num_store_stats":7}],"osd_stats":[{"osd":11,"up_from":0,"seq":32,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5744,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739408,"statfs":{"total":95995035648,"available":95989153792,"internally_reserved":0,"allocated":696320,"data_stored":395883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":2.2389999999999999,"5min":1.927,"15min":1.875},"min":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"max":{"1min":3.2730000000000001,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.5209999999999999},{"interface":"front","average":{"1min":3.4089999999999998,"5min":3.331,"15min":3.3180000000000001},"min":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"max":{"1min":4.4699999999999998,"5min":4.5970000000000004,"15min":4.5970000000000004},"last":3.2770000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":2.0129999999999999,"5min":2.0499999999999998,"15min":2.056},"min":{"1min":1.4330000000000001,"5min":1.395,"15min":1.395},"max":{"1min":2.629,"5min":3.2250000000000001,"15min":3.2250000000000001},"last":1.6619999999999999},{"interface":"front","average":{"1min":2.3279999999999998,"5min":2.5790000000000002,"15min":2.621},"min":{"1min":1.363,"5min":1.363,"15min":1.363},"max":{"1min":3.2629999999999999,"5min":3.8580000000000001,"15min":3.8580000000000001},"last":2.8700000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":2.3769999999999998,"5min":2.141,"15min":2.101},"min":{"1min":1.5760000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"max":{"1min":4.3940000000000001,"5min":4.3940000000000001,"15min":4.3940000000000001},"last":2.7639999999999998},{"interface":"front","average":{"1min":3.0459999999999998,"5min":2.903,"15min":2.879},"min":{"1min":1.702,"5min":1.702,"15min":1.702},"max":{"1min":4.2720000000000002,"5min":4.2939999999999996,"15min":4.2939999999999996},"last":2.5609999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":2.7810000000000001,"5min":2.419,"15min":2.359},"min":{"1min":1.665,"5min":1.4670000000000001,"15min":1.4670000000000001},"max":{"1min":3.8820000000000001,"5min":3.8820000000000001,"15min":3.8820000000000001},"last":2.0710000000000002},{"interface":"front","average":{"1min":3.1880000000000002,"5min":3.1469999999999998,"15min":3.1400000000000001},"min":{"1min":1.454,"5min":1.454,"15min":1.454},"max":{"1min":4.4409999999999998,"5min":4.8949999999999996,"15min":4.8949999999999996},"last":2.4849999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":3.4159999999999999,"5min":3.5710000000000002,"15min":3.597},"min":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"max":{"1min":5.0300000000000002,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":3.9969999999999999},{"interface":"front","average":{"1min":2.9649999999999999,"5min":2.8370000000000002,"15min":2.8159999999999998},"min":{"1min":1.7290000000000001,"5min":1.673,"15min":1.673},"max":{"1min":4.7320000000000002,"5min":4.7320000000000002,"15min":4.7320000000000002},"last":1.9630000000000001}]},{"osd":5,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":3.427,"5min":3.4340000000000002,"15min":3.4350000000000001},"min":{"1min":2.262,"5min":2.262,"15min":2.262},"max":{"1min":4.4119999999999999,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":3.7629999999999999},{"interface":"front","average":{"1min":3.7869999999999999,"5min":3.9569999999999999,"15min":3.9849999999999999},"min":{"1min":2.5510000000000002,"5min":2.5059999999999998,"15min":2.5059999999999998},"max":{"1min":5.1829999999999998,"5min":6.3630000000000004,"15min":6.3630000000000004},"last":3.6680000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":3.5590000000000002,"5min":3.46,"15min":3.444},"min":{"1min":2.4910000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"max":{"1min":5.101,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":2.3969999999999998},{"interface":"front","average":{"1min":3.827,"5min":3.6619999999999999,"15min":3.6339999999999999},"min":{"1min":1.877,"5min":1.877,"15min":1.877},"max":{"1min":4.9199999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":2.9399999999999999}]},{"osd":7,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":3.8029999999999999,"5min":3.7679999999999998,"15min":3.762},"min":{"1min":2.673,"5min":2.673,"15min":2.673},"max":{"1min":5.1139999999999999,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":3.2349999999999999},{"interface":"front","average":{"1min":3.9630000000000001,"5min":3.9049999999999998,"15min":3.895},"min":{"1min":2.4390000000000001,"5min":2.4390000000000001,"15min":2.4390000000000001},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":3.343}]},{"osd":8,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":4.2069999999999999,"5min":4.1509999999999998,"15min":4.1420000000000003},"min":{"1min":3.2360000000000002,"5min":2.923,"15min":2.923},"max":{"1min":5.2030000000000003,"5min":5.944,"15min":5.944},"last":4.2060000000000004},{"interface":"front","average":{"1min":3.8620000000000001,"5min":3.8980000000000001,"15min":3.9039999999999999},"min":{"1min":2.403,"5min":2.1469999999999998,"15min":2.1469999999999998},"max":{"1min":5.117,"5min":6.2400000000000002,"15min":6.2400000000000002},"last":3.0419999999999998}]},{"osd":10,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":4.3639999999999999,"5min":4.4710000000000001,"15min":4.4889999999999999},"min":{"1min":3.472,"5min":3.0459999999999998,"15min":3.0459999999999998},"max":{"1min":5.383,"5min":6.673,"15min":6.673},"last":4.4009999999999998},{"interface":"front","average":{"1min":4.3280000000000003,"5min":4.298,"15min":4.2930000000000001},"min":{"1min":2.9430000000000001,"5min":2.7040000000000002,"15min":2.7040000000000002},"max":{"1min":5.6609999999999996,"5min":6.6859999999999999,"15min":6.6859999999999999},"last":3.4399999999999999}]}]},{"osd":10,"up_from":0,"seq":38,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5808,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739344,"statfs":{"total":95995035648,"available":95989088256,"internally_reserved":0,"allocated":696320,"data_stored":395883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7330000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":1.921,"5min":2.3119999999999998,"15min":2.3769999999999998},"min":{"1min":1.5109999999999999,"5min":1.419,"15min":1.419},"max":{"1min":2.4969999999999999,"5min":4.3380000000000001,"15min":4.3380000000000001},"last":1.722},{"interface":"front","average":{"1min":2.6440000000000001,"5min":2.5750000000000002,"15min":2.5630000000000002},"min":{"1min":1.6739999999999999,"5min":1.4750000000000001,"15min":1.4750000000000001},"max":{"1min":3.7759999999999998,"5min":5.1029999999999998,"15min":5.1029999999999998},"last":3.1509999999999998}]},{"osd":2,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":2.3109999999999999,"5min":2.2599999999999998,"15min":2.2519999999999998},"min":{"1min":1.6519999999999999,"5min":1.508,"15min":1.508},"max":{"1min":3.0939999999999999,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":2.0950000000000002},{"interface":"front","average":{"1min":2.1970000000000001,"5min":2.6960000000000002,"15min":2.7789999999999999},"min":{"1min":1.492,"5min":1.4359999999999999,"15min":1.4359999999999999},"max":{"1min":3.4260000000000002,"5min":4.3769999999999998,"15min":4.3769999999999998},"last":1.841}]},{"osd":3,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":1.9159999999999999,"5min":2.2090000000000001,"15min":2.258},"min":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"max":{"1min":2.6469999999999998,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":1.7709999999999999},{"interface":"front","average":{"1min":2.7200000000000002,"5min":3.016,"15min":3.0649999999999999},"min":{"1min":1.609,"5min":1.609,"15min":1.609},"max":{"1min":3.605,"5min":4.2140000000000004,"15min":4.2140000000000004},"last":2.5209999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.1469999999999998,"5min":3.4039999999999999,"15min":3.4470000000000001},"min":{"1min":1.8,"5min":1.8,"15min":1.8},"max":{"1min":4.0720000000000001,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":3.415},{"interface":"front","average":{"1min":2.9780000000000002,"5min":3.0649999999999999,"15min":3.0790000000000002},"min":{"1min":1.958,"5min":1.958,"15min":1.958},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":4.0279999999999996}]},{"osd":5,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.218,"5min":3.5510000000000002,"15min":3.6070000000000002},"min":{"1min":2.653,"5min":2.427,"15min":2.427},"max":{"1min":4.1059999999999999,"5min":4.7160000000000002,"15min":4.7160000000000002},"last":3.4710000000000001},{"interface":"front","average":{"1min":3.6600000000000001,"5min":3.8199999999999998,"15min":3.847},"min":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"max":{"1min":4.7089999999999996,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":3.6070000000000002}]},{"osd":6,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.3300000000000001,"5min":3.5339999999999998,"15min":3.5680000000000001},"min":{"1min":2.3359999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"max":{"1min":4.7949999999999999,"5min":4.7949999999999999,"15min":4.7949999999999999},"last":3.29},{"interface":"front","average":{"1min":3.6800000000000002,"5min":3.8620000000000001,"15min":3.8919999999999999},"min":{"1min":3.0270000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"max":{"1min":4.4939999999999998,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":3.8980000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.7400000000000002,"5min":3.907,"15min":3.9350000000000001},"min":{"1min":2.3410000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"max":{"1min":4.7220000000000004,"5min":7.0490000000000004,"15min":7.0490000000000004},"last":3.6699999999999999},{"interface":"front","average":{"1min":3.9009999999999998,"5min":4.077,"15min":4.1059999999999999},"min":{"1min":3.1789999999999998,"5min":2.806,"15min":2.806},"max":{"1min":4.8929999999999998,"5min":5.0899999999999999,"15min":5.0899999999999999},"last":4.3479999999999999}]},{"osd":8,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":4.0149999999999997,"5min":4.1459999999999999,"15min":4.1680000000000001},"min":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"max":{"1min":5.3390000000000004,"5min":5.7949999999999999,"15min":5.7949999999999999},"last":4.0979999999999999},{"interface":"front","average":{"1min":4.0670000000000002,"5min":4.0309999999999997,"15min":4.0250000000000004},"min":{"1min":2.6920000000000002,"5min":1.901,"15min":1.901},"max":{"1min":5.1200000000000001,"5min":5.6509999999999998,"15min":5.6509999999999998},"last":4.4210000000000003}]},{"osd":9,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":4.2160000000000002,"5min":4.3780000000000001,"15min":4.4050000000000002},"min":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"max":{"1min":5.4020000000000001,"5min":5.8369999999999997,"15min":5.8369999999999997},"last":3.8220000000000001},{"interface":"front","average":{"1min":4.2629999999999999,"5min":4.2149999999999999,"15min":4.2069999999999999},"min":{"1min":2.5590000000000002,"5min":2.4969999999999999,"15min":2.4969999999999999},"max":{"1min":5.3129999999999997,"5min":5.3129999999999997,"15min":5.3129999999999997},"last":4.8710000000000004}]},{"osd":11,"last update":"Tue Sep 8 15:36:47 2020","interfaces":[{"interface":"back","average":{"1min":4.3899999999999997,"5min":4.3890000000000002,"15min":4.3890000000000002},"min":{"1min":3.7909999999999999,"5min":1.831,"15min":1.831},"max":{"1min":5.3940000000000001,"5min":5.6980000000000004,"15min":5.6980000000000004},"last":4.2999999999999998},{"interface":"front","average":{"1min":4.5609999999999999,"5min":4.4500000000000002,"15min":4.4320000000000004},"min":{"1min":3.8860000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"max":{"1min":5.5069999999999997,"5min":5.5069999999999997,"15min":5.5069999999999997},"last":4.6609999999999996}]}]},{"osd":9,"up_from":0,"seq":41,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5872,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739280,"statfs":{"total":95995035648,"available":95989022720,"internally_reserved":0,"allocated":696320,"data_stored":395883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:57 2020","interfaces":[{"interface":"back","average":{"1min":2.1440000000000001,"5min":1.9690000000000001,"15min":1.895},"min":{"1min":1.3999999999999999,"5min":1.3140000000000001,"15min":1.3140000000000001},"max":{"1min":3.3380000000000001,"5min":3.3380000000000001,"15min":3.3380000000000001},"last":2.1739999999999999},{"interface":"front","average":{"1min":2.0539999999999998,"5min":1.9419999999999999,"15min":1.9159999999999999},"min":{"1min":1.349,"5min":1.2549999999999999,"15min":1.2549999999999999},"max":{"1min":2.851,"5min":4.149,"15min":4.149},"last":1.8420000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:36:57 2020","interfaces":[{"interface":"back","average":{"1min":2.1139999999999999,"5min":2.0259999999999998,"15min":1.974},"min":{"1min":1.5660000000000001,"5min":1.3009999999999999,"15min":1.3009999999999999},"max":{"1min":3.113,"5min":3.6269999999999998,"15min":3.6269999999999998},"last":1.5680000000000001},{"interface":"front","average":{"1min":2.4780000000000002,"5min":2.3639999999999999,"15min":2.3109999999999999},"min":{"1min":1.417,"5min":1.266,"15min":1.266},"max":{"1min":3.8980000000000001,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":2.319}]},{"osd":2,"last update":"Tue Sep 8 15:36:57 2020","interfaces":[{"interface":"back","average":{"1min":2.3050000000000002,"5min":2.4889999999999999,"15min":2.4780000000000002},"min":{"1min":1.357,"5min":1.327,"15min":1.327},"max":{"1min":4.1710000000000003,"5min":4.508,"15min":4.508},"last":1.444},{"interface":"front","average":{"1min":2.5230000000000001,"5min":2.3809999999999998,"15min":2.3130000000000002},"min":{"1min":1.5209999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"max":{"1min":3.7879999999999998,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":1.5209999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:36:57 2020","interfaces":[{"interface":"back","average":{"1min":2.6040000000000001,"5min":2.403,"15min":2.3610000000000002},"min":{"1min":1.397,"5min":1.198,"15min":1.198},"max":{"1min":3.8359999999999999,"5min":4.3760000000000003,"15min":4.3760000000000003},"last":2.8119999999999998},{"interface":"front","average":{"1min":2.944,"5min":2.7010000000000001,"15min":2.6629999999999998},"min":{"1min":1.681,"5min":1.4610000000000001,"15min":1.4610000000000001},"max":{"1min":4.4980000000000002,"5min":4.4980000000000002,"15min":4.4980000000000002},"last":2.6299999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:36:57 2020","interfaces":[{"interface":"back","average":{"1min":2.9489999999999998,"5min":2.8100000000000001,"15min":2.726},"min":{"1min":1.6359999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"max":{"1min":4.6109999999999998,"5min":5.0279999999999996,"15min":5.0279999999999996},"last":2.1080000000000001},{"interface":"front","average":{"1min":3.5499999999999998,"5min":3.2149999999999999,"15min":3.069},"min":{"1min":2.867,"5min":1.883,"15min":1.883},"max":{"1min":4.3470000000000004,"5min":4.6280000000000001,"15min":4.6280000000000001},"last":3.5249999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:36:57 2020","interfaces":[{"interface":"back","average":{"1min":3.2850000000000001,"5min":3.2410000000000001,"15min":3.1840000000000002},"min":{"1min":1.9119999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"max":{"1min":4.9349999999999996,"5min":6.3840000000000003,"15min":6.3840000000000003},"last":3.0579999999999998},{"interface":"front","average":{"1min":3.5030000000000001,"5min":3.2919999999999998,"15min":3.2080000000000002},"min":{"1min":2.7200000000000002,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":4.7149999999999999,"5min":4.875,"15min":4.875},"last":3.1619999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:36:57 2020","interfaces":[{"interface":"back","average":{"1min":3.7250000000000001,"5min":3.4159999999999999,"15min":3.2719999999999998},"min":{"1min":2.2050000000000001,"5min":1.4179999999999999,"15min":1.4179999999999999},"max":{"1min":4.7889999999999997,"5min":5.3659999999999997,"15min":5.3659999999999997},"last":3.4319999999999999},{"interface":"front","average":{"1min":3.4860000000000002,"5min":3.3599999999999999,"15min":3.274},"min":{"1min":2.4380000000000002,"5min":1.8959999999999999,"15min":1.8959999999999999},"max":{"1min":5.3840000000000003,"5min":5.7130000000000001,"15min":5.7130000000000001},"last":2.7639999999999998}]},{"osd":7,"last update":"Tue Sep 8 15:36:57 2020","interfaces":[{"interface":"back","average":{"1min":4.0019999999999998,"5min":3.6230000000000002,"15min":3.468},"min":{"1min":2.3740000000000001,"5min":1.7949999999999999,"15min":1.7949999999999999},"max":{"1min":5.367,"5min":5.367,"15min":5.367},"last":3.8210000000000002},{"interface":"front","average":{"1min":4.1470000000000002,"5min":3.8069999999999999,"15min":3.6749999999999998},"min":{"1min":2.5830000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"max":{"1min":5.4800000000000004,"5min":5.7629999999999999,"15min":5.7629999999999999},"last":4.3010000000000002}]},{"osd":8,"last update":"Tue Sep 8 15:36:57 2020","interfaces":[{"interface":"back","average":{"1min":3.597,"5min":3.3650000000000002,"15min":3.23},"min":{"1min":2.2669999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"max":{"1min":4.8319999999999999,"5min":5.3049999999999997,"15min":5.3049999999999997},"last":2.2669999999999999},{"interface":"front","average":{"1min":4.2320000000000002,"5min":3.5960000000000001,"15min":3.3540000000000001},"min":{"1min":3.0459999999999998,"5min":1.444,"15min":1.444},"max":{"1min":5.1319999999999997,"5min":6.0140000000000002,"15min":6.0140000000000002},"last":3.9199999999999999}]},{"osd":10,"last update":"Tue Sep 8 15:36:18 2020","interfaces":[{"interface":"back","average":{"1min":3.9500000000000002,"5min":3.9350000000000001,"15min":3.9329999999999998},"min":{"1min":3.3839999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"max":{"1min":4.8780000000000001,"5min":5.5629999999999997,"15min":5.5629999999999997},"last":4.2270000000000003},{"interface":"front","average":{"1min":3.9649999999999999,"5min":3.8239999999999998,"15min":3.7999999999999998},"min":{"1min":3.3130000000000002,"5min":2.056,"15min":2.056},"max":{"1min":4.6379999999999999,"5min":5.492,"15min":5.492},"last":3.008}]}]},{"osd":8,"up_from":0,"seq":47,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6000,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93739152,"statfs":{"total":95995035648,"available":95988891648,"internally_reserved":0,"allocated":696320,"data_stored":395883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.2749999999999999,"5min":2.202,"15min":2.1920000000000002},"min":{"1min":1.3799999999999999,"5min":1.266,"15min":1.266},"max":{"1min":3.2200000000000002,"5min":3.5830000000000002,"15min":3.5830000000000002},"last":2.093},{"interface":"front","average":{"1min":2.5459999999999998,"5min":2.5070000000000001,"15min":2.4300000000000002},"min":{"1min":1.7669999999999999,"5min":1.7250000000000001,"15min":1.7250000000000001},"max":{"1min":3.444,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":2.016}]},{"osd":1,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.492,"5min":2.0659999999999998,"15min":1.8839999999999999},"min":{"1min":1.4690000000000001,"5min":1.137,"15min":1.137},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":2.8610000000000002},{"interface":"front","average":{"1min":2.7200000000000002,"5min":2.427,"15min":2.347},"min":{"1min":1.351,"5min":1.046,"15min":1.046},"max":{"1min":4.444,"5min":4.444,"15min":4.444},"last":2.1789999999999998}]},{"osd":2,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.2330000000000001,"5min":1.9410000000000001,"15min":1.845},"min":{"1min":1.4379999999999999,"5min":1.27,"15min":1.27},"max":{"1min":3.9129999999999998,"5min":3.9129999999999998,"15min":3.9129999999999998},"last":3.2959999999999998},{"interface":"front","average":{"1min":2.4889999999999999,"5min":2.504,"15min":2.46},"min":{"1min":1.208,"5min":1.208,"15min":1.208},"max":{"1min":3.7509999999999999,"5min":3.895,"15min":3.895},"last":3.4780000000000002}]},{"osd":3,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.9750000000000001,"5min":2.7869999999999999,"15min":2.7370000000000001},"min":{"1min":1.587,"5min":1.2789999999999999,"15min":1.2789999999999999},"max":{"1min":4.3929999999999998,"5min":4.3929999999999998,"15min":4.3929999999999998},"last":3.532},{"interface":"front","average":{"1min":2.4649999999999999,"5min":2.4260000000000002,"15min":2.3820000000000001},"min":{"1min":1.131,"5min":1.131,"15min":1.131},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":3.6859999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":3.149,"5min":2.9870000000000001,"15min":2.875},"min":{"1min":2.3029999999999999,"5min":1.4730000000000001,"15min":1.4730000000000001},"max":{"1min":4.4669999999999996,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":3.173},{"interface":"front","average":{"1min":2.8820000000000001,"5min":2.6869999999999998,"15min":2.5579999999999998},"min":{"1min":1.734,"5min":1.2549999999999999,"15min":1.2549999999999999},"max":{"1min":4.2649999999999997,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":2.6579999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":3.5529999999999999,"5min":3.294,"15min":3.1619999999999999},"min":{"1min":1.944,"5min":1.74,"15min":1.74},"max":{"1min":5.6109999999999998,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":4.0199999999999996},{"interface":"front","average":{"1min":3.7360000000000002,"5min":3.419,"15min":3.3719999999999999},"min":{"1min":2.2080000000000002,"5min":1.762,"15min":1.762},"max":{"1min":6.0549999999999997,"5min":6.0549999999999997,"15min":6.0549999999999997},"last":3.891}]},{"osd":6,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":3.8090000000000002,"5min":3.4199999999999999,"15min":3.2639999999999998},"min":{"1min":2.4980000000000002,"5min":1.9299999999999999,"15min":1.9299999999999999},"max":{"1min":5.1929999999999996,"5min":5.6180000000000003,"15min":5.6180000000000003},"last":4.1029999999999998},{"interface":"front","average":{"1min":3.2839999999999998,"5min":3.0499999999999998,"15min":2.9220000000000002},"min":{"1min":2.0019999999999998,"5min":1.669,"15min":1.669},"max":{"1min":4.7279999999999998,"5min":4.7279999999999998,"15min":4.7279999999999998},"last":3.0830000000000002}]},{"osd":7,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":4.0419999999999998,"5min":3.7269999999999999,"15min":3.6240000000000001},"min":{"1min":2.3849999999999998,"5min":2.0630000000000002,"15min":2.0630000000000002},"max":{"1min":5.3680000000000003,"5min":5.3680000000000003,"15min":5.3680000000000003},"last":4.4829999999999997},{"interface":"front","average":{"1min":3.6429999999999998,"5min":3.3679999999999999,"15min":3.2440000000000002},"min":{"1min":2.7269999999999999,"5min":2.0830000000000002,"15min":2.0830000000000002},"max":{"1min":5.1029999999999998,"5min":5.141,"15min":5.141},"last":3.395}]},{"osd":9,"last update":"Tue Sep 8 15:36:57 2020","interfaces":[{"interface":"back","average":{"1min":3.8769999999999998,"5min":3.597,"15min":3.4550000000000001},"min":{"1min":3.1379999999999999,"5min":1.157,"15min":1.157},"max":{"1min":4.9749999999999996,"5min":5.1840000000000002,"15min":5.1840000000000002},"last":3.802},{"interface":"front","average":{"1min":4.2789999999999999,"5min":3.6629999999999998,"15min":3.415},"min":{"1min":3.3860000000000001,"5min":1.1919999999999999,"15min":1.1919999999999999},"max":{"1min":5.7030000000000003,"5min":5.7030000000000003,"15min":5.7030000000000003},"last":4.2939999999999996}]},{"osd":10,"last update":"Tue Sep 8 15:36:27 2020","interfaces":[{"interface":"back","average":{"1min":3.7200000000000002,"5min":3.7320000000000002,"15min":3.734},"min":{"1min":2.0510000000000002,"5min":1.3740000000000001,"15min":1.3740000000000001},"max":{"1min":5.5190000000000001,"5min":5.5190000000000001,"15min":5.5190000000000001},"last":3.73},{"interface":"front","average":{"1min":4.1180000000000003,"5min":4.0830000000000002,"15min":4.077},"min":{"1min":2.7669999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"max":{"1min":5.3049999999999997,"5min":5.3049999999999997,"15min":5.3049999999999997},"last":4.2249999999999996}]}]},{"osd":7,"up_from":0,"seq":53,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6064,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93739088,"statfs":{"total":95995035648,"available":95988826112,"internally_reserved":0,"allocated":696320,"data_stored":395903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:02 2020","interfaces":[{"interface":"back","average":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"min":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"max":{"1min":4.0190000000000001,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":2.2189999999999999},{"interface":"front","average":{"1min":2.6259999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"min":{"1min":1.542,"5min":1.542,"15min":1.542},"max":{"1min":4.5229999999999997,"5min":4.5229999999999997,"15min":4.5229999999999997},"last":2.3079999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.661,"5min":2.468,"15min":2.355},"min":{"1min":2.1309999999999998,"5min":1.498,"15min":1.498},"max":{"1min":3.3940000000000001,"5min":3.8780000000000001,"15min":3.8780000000000001},"last":2.238},{"interface":"front","average":{"1min":2.1699999999999999,"5min":2.359,"15min":2.3599999999999999},"min":{"1min":1.7749999999999999,"5min":1.631,"15min":1.631},"max":{"1min":2.6800000000000002,"5min":3.633,"15min":3.633},"last":2.2949999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.4049999999999998,"5min":2.528,"15min":2.4399999999999999},"min":{"1min":1.899,"5min":1.504,"15min":1.504},"max":{"1min":3.2240000000000002,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":2.657},{"interface":"front","average":{"1min":2.468,"5min":2.879,"15min":2.8370000000000002},"min":{"1min":1.982,"5min":1.982,"15min":1.982},"max":{"1min":3.2570000000000001,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":2.9249999999999998}]},{"osd":3,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.7869999999999999,"5min":2.9849999999999999,"15min":2.891},"min":{"1min":1.861,"5min":1.861,"15min":1.861},"max":{"1min":3.8460000000000001,"5min":4.875,"15min":4.875},"last":3.4700000000000002},{"interface":"front","average":{"1min":2.8159999999999998,"5min":3.254,"15min":3.234},"min":{"1min":1.978,"5min":1.5660000000000001,"15min":1.5660000000000001},"max":{"1min":3.8330000000000002,"5min":5.3929999999999998,"15min":5.3929999999999998},"last":2.4100000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":3.0150000000000001,"5min":3.0289999999999999,"15min":2.9329999999999998},"min":{"1min":2.181,"5min":1.9279999999999999,"15min":1.9279999999999999},"max":{"1min":3.98,"5min":5.9729999999999999,"15min":5.9729999999999999},"last":3.0800000000000001},{"interface":"front","average":{"1min":3.3780000000000001,"5min":3.456,"15min":3.3100000000000001},"min":{"1min":2.5150000000000001,"5min":2.464,"15min":2.464},"max":{"1min":4.4260000000000002,"5min":5.6470000000000002,"15min":5.6470000000000002},"last":3.1989999999999998}]},{"osd":5,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":3.4790000000000001,"5min":3.7400000000000002,"15min":3.6859999999999999},"min":{"1min":2.794,"5min":2.794,"15min":2.794},"max":{"1min":4.593,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":3.5539999999999998},{"interface":"front","average":{"1min":3.3769999999999998,"5min":3.5590000000000002,"15min":3.5129999999999999},"min":{"1min":2.7080000000000002,"5min":2.0859999999999999,"15min":2.0859999999999999},"max":{"1min":4.4779999999999998,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":3.9489999999999998}]},{"osd":6,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":3.5529999999999999,"5min":3.9319999999999999,"15min":3.9369999999999998},"min":{"1min":2.7309999999999999,"5min":2.6320000000000001,"15min":2.6320000000000001},"max":{"1min":4.7599999999999998,"5min":7.3070000000000004,"15min":7.3070000000000004},"last":4.0670000000000002},{"interface":"front","average":{"1min":3.9039999999999999,"5min":4.0140000000000002,"15min":3.9180000000000001},"min":{"1min":3.1349999999999998,"5min":2.6549999999999998,"15min":2.6549999999999998},"max":{"1min":4.6470000000000002,"5min":7.2480000000000002,"15min":7.2480000000000002},"last":3.6669999999999998}]},{"osd":8,"last update":"Tue Sep 8 15:36:33 2020","interfaces":[{"interface":"back","average":{"1min":4.1619999999999999,"5min":4.2220000000000004,"15min":4.1609999999999996},"min":{"1min":2.7250000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"max":{"1min":5.5300000000000002,"5min":6.9530000000000003,"15min":6.9530000000000003},"last":4.4950000000000001},{"interface":"front","average":{"1min":4.444,"5min":4.4059999999999997,"15min":4.2830000000000004},"min":{"1min":3.5579999999999998,"5min":2.8479999999999999,"15min":2.8479999999999999},"max":{"1min":5.6929999999999996,"5min":7.6509999999999998,"15min":7.6509999999999998},"last":3.4319999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:36:55 2020","interfaces":[{"interface":"back","average":{"1min":4.7350000000000003,"5min":4.7800000000000002,"15min":4.7759999999999998},"min":{"1min":3.282,"5min":3.0030000000000001,"15min":3.0030000000000001},"max":{"1min":5.9299999999999997,"5min":6.524,"15min":6.524},"last":3.8809999999999998},{"interface":"front","average":{"1min":4.008,"5min":4.3319999999999999,"15min":4.4139999999999997},"min":{"1min":3.0489999999999999,"5min":3.0489999999999999,"15min":3.0489999999999999},"max":{"1min":5.3200000000000003,"5min":6.274,"15min":6.274},"last":4.4039999999999999}]},{"osd":10,"last update":"Tue Sep 8 15:36:22 2020","interfaces":[{"interface":"back","average":{"1min":4.2460000000000004,"5min":4.649,"15min":4.7160000000000002},"min":{"1min":3.371,"5min":3.218,"15min":3.218},"max":{"1min":5.3390000000000004,"5min":7.7800000000000002,"15min":7.7800000000000002},"last":4.7290000000000001},{"interface":"front","average":{"1min":4.8840000000000003,"5min":5.1219999999999999,"15min":5.1619999999999999},"min":{"1min":3.7410000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"max":{"1min":5.8760000000000003,"5min":8.3650000000000002,"15min":8.3650000000000002},"last":4.3600000000000003}]},{"osd":11,"last update":"Tue Sep 8 15:36:46 2020","interfaces":[{"interface":"back","average":{"1min":4.4409999999999998,"5min":4.758,"15min":4.8109999999999999},"min":{"1min":3.0190000000000001,"5min":3.0190000000000001,"15min":3.0190000000000001},"max":{"1min":5.7329999999999997,"5min":6.1589999999999998,"15min":6.1589999999999998},"last":4.9279999999999999},{"interface":"front","average":{"1min":5.0099999999999998,"5min":5.2549999999999999,"15min":5.2960000000000003},"min":{"1min":3.4260000000000002,"5min":3.4260000000000002,"15min":3.4260000000000002},"max":{"1min":6.069,"5min":6.742,"15min":6.742},"last":4.1619999999999999}]}]},{"osd":6,"up_from":0,"seq":56,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6128,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93739024,"statfs":{"total":95995035648,"available":95988760576,"internally_reserved":0,"allocated":696320,"data_stored":395903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:47 2020","interfaces":[{"interface":"back","average":{"1min":2.2770000000000001,"5min":2.1339999999999999,"15min":1.9299999999999999},"min":{"1min":1.6699999999999999,"5min":1.141,"15min":1.141},"max":{"1min":3.1709999999999998,"5min":3.714,"15min":3.714},"last":2.532},{"interface":"front","average":{"1min":2.3849999999999998,"5min":2.145,"15min":1.998},"min":{"1min":1.7170000000000001,"5min":1.256,"15min":1.256},"max":{"1min":3.6970000000000001,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":2.6480000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:36:47 2020","interfaces":[{"interface":"back","average":{"1min":2.2410000000000001,"5min":2.194,"15min":2.1819999999999999},"min":{"1min":1.7350000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"max":{"1min":3.645,"5min":3.645,"15min":3.645},"last":2.7610000000000001},{"interface":"front","average":{"1min":2.9260000000000002,"5min":2.6899999999999999,"15min":2.4729999999999999},"min":{"1min":1.9319999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":3.8500000000000001,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":3.9980000000000002}]},{"osd":2,"last update":"Tue Sep 8 15:36:47 2020","interfaces":[{"interface":"back","average":{"1min":2.7730000000000001,"5min":2.6709999999999998,"15min":2.4510000000000001},"min":{"1min":1.819,"5min":1.462,"15min":1.462},"max":{"1min":4.4619999999999997,"5min":4.4619999999999997,"15min":4.4619999999999997},"last":3.4489999999999998},{"interface":"front","average":{"1min":3.0350000000000001,"5min":2.7109999999999999,"15min":2.4350000000000001},"min":{"1min":1.9339999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"max":{"1min":4.4009999999999998,"5min":4.4009999999999998,"15min":4.4009999999999998},"last":4.4509999999999996}]},{"osd":3,"last update":"Tue Sep 8 15:36:47 2020","interfaces":[{"interface":"back","average":{"1min":3.0499999999999998,"5min":2.8029999999999999,"15min":2.488},"min":{"1min":2.0590000000000002,"5min":1.7809999999999999,"15min":1.7809999999999999},"max":{"1min":3.7989999999999999,"5min":4.8109999999999999,"15min":4.8109999999999999},"last":4.3390000000000004},{"interface":"front","average":{"1min":3.5129999999999999,"5min":3.2429999999999999,"15min":2.9140000000000001},"min":{"1min":2.5310000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"max":{"1min":4.7430000000000003,"5min":4.8120000000000003,"15min":4.8120000000000003},"last":5.234}]},{"osd":4,"last update":"Tue Sep 8 15:36:47 2020","interfaces":[{"interface":"back","average":{"1min":3.4039999999999999,"5min":3.093,"15min":2.7210000000000001},"min":{"1min":2.141,"5min":1.607,"15min":1.607},"max":{"1min":4.7309999999999999,"5min":4.7309999999999999,"15min":4.7309999999999999},"last":4.9980000000000002},{"interface":"front","average":{"1min":3.5299999999999998,"5min":3.23,"15min":2.9449999999999998},"min":{"1min":2.5470000000000002,"5min":1.5049999999999999,"15min":1.5049999999999999},"max":{"1min":4.3079999999999998,"5min":4.8769999999999998,"15min":4.8769999999999998},"last":6.1950000000000003}]},{"osd":5,"last update":"Tue Sep 8 15:36:47 2020","interfaces":[{"interface":"back","average":{"1min":4.032,"5min":3.5430000000000001,"15min":3.2050000000000001},"min":{"1min":2.8849999999999998,"5min":1.7509999999999999,"15min":1.7509999999999999},"max":{"1min":5.8460000000000001,"5min":6.1269999999999998,"15min":6.1269999999999998},"last":5.3099999999999996},{"interface":"front","average":{"1min":3.855,"5min":3.548,"15min":3.21},"min":{"1min":2.9670000000000001,"5min":2.1659999999999999,"15min":2.1659999999999999},"max":{"1min":4.9580000000000002,"5min":6.1820000000000004,"15min":6.1820000000000004},"last":5.5259999999999998}]},{"osd":7,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":4.0549999999999997,"5min":3.8460000000000001,"15min":3.714},"min":{"1min":3.2879999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"max":{"1min":4.8970000000000002,"5min":5.4050000000000002,"15min":5.4050000000000002},"last":5.6310000000000002},{"interface":"front","average":{"1min":4.5359999999999996,"5min":4.0469999999999997,"15min":3.8090000000000002},"min":{"1min":3.387,"5min":2.1640000000000001,"15min":2.1640000000000001},"max":{"1min":5.5609999999999999,"5min":6.2679999999999998,"15min":6.2679999999999998},"last":6.5300000000000002}]},{"osd":8,"last update":"Tue Sep 8 15:36:30 2020","interfaces":[{"interface":"back","average":{"1min":4.3600000000000003,"5min":4.0759999999999996,"15min":3.9500000000000002},"min":{"1min":3.258,"5min":3.0579999999999998,"15min":3.0579999999999998},"max":{"1min":5.6639999999999997,"5min":5.6639999999999997,"15min":5.6639999999999997},"last":4.8399999999999999},{"interface":"front","average":{"1min":4.2759999999999998,"5min":4.2309999999999999,"15min":4.1929999999999996},"min":{"1min":3.5259999999999998,"5min":3.1269999999999998,"15min":3.1269999999999998},"max":{"1min":5.3819999999999997,"5min":5.5499999999999998,"15min":5.5499999999999998},"last":6.0199999999999996}]},{"osd":9,"last update":"Tue Sep 8 15:36:55 2020","interfaces":[{"interface":"back","average":{"1min":4.7510000000000003,"5min":4.5620000000000003,"15min":4.4690000000000003},"min":{"1min":3.8860000000000001,"5min":3.1110000000000002,"15min":3.1110000000000002},"max":{"1min":6.4059999999999997,"5min":6.4059999999999997,"15min":6.4059999999999997},"last":5.774},{"interface":"front","average":{"1min":4.641,"5min":4.5419999999999998,"15min":4.5279999999999996},"min":{"1min":3.367,"5min":3.367,"15min":3.367},"max":{"1min":6.1139999999999999,"5min":6.1139999999999999,"15min":6.1139999999999999},"last":6.1139999999999999}]},{"osd":10,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":4.6909999999999998,"5min":4.694,"15min":4.694},"min":{"1min":3.7730000000000001,"5min":3.4590000000000001,"15min":3.4590000000000001},"max":{"1min":6.0720000000000001,"5min":6.0720000000000001,"15min":6.0720000000000001},"last":6.6630000000000003},{"interface":"front","average":{"1min":4.6159999999999997,"5min":4.6260000000000003,"15min":4.6280000000000001},"min":{"1min":3.4660000000000002,"5min":3.4159999999999999,"15min":3.4159999999999999},"max":{"1min":5.6980000000000004,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":6.9790000000000001}]}]},{"osd":1,"up_from":0,"seq":81,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6320,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738832,"statfs":{"total":95995035648,"available":95988563968,"internally_reserved":0,"allocated":696320,"data_stored":395903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:46 2020","interfaces":[{"interface":"back","average":{"1min":2.2669999999999999,"5min":1.9710000000000001,"15min":1.456},"min":{"1min":1.0129999999999999,"5min":0.99299999999999999,"15min":0.75600000000000001},"max":{"1min":3.3319999999999999,"5min":3.363,"15min":3.363},"last":2.3610000000000002},{"interface":"front","average":{"1min":2.6240000000000001,"5min":2.274,"15min":1.6240000000000001},"min":{"1min":1.6739999999999999,"5min":1.1399999999999999,"15min":0.75800000000000001},"max":{"1min":3.601,"5min":3.698,"15min":3.698},"last":1.9199999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:36:04 2020","interfaces":[{"interface":"back","average":{"1min":2.5539999999999998,"5min":2.153,"15min":1.712},"min":{"1min":1.3460000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":4.4080000000000004,"5min":4.4080000000000004,"15min":4.4080000000000004},"last":1.7609999999999999},{"interface":"front","average":{"1min":2.7290000000000001,"5min":2.2770000000000001,"15min":1.8680000000000001},"min":{"1min":1.431,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":4.6559999999999997,"5min":4.6559999999999997,"15min":4.6559999999999997},"last":2.4809999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:36:34 2020","interfaces":[{"interface":"back","average":{"1min":2.7509999999999999,"5min":2.5230000000000001,"15min":2.2770000000000001},"min":{"1min":0.99299999999999999,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":4.0469999999999997,"5min":4.5119999999999996,"15min":4.5119999999999996},"last":1.5860000000000001},{"interface":"front","average":{"1min":2.593,"5min":2.548,"15min":2.4079999999999999},"min":{"1min":1.746,"5min":1.2410000000000001,"15min":1.2410000000000001},"max":{"1min":4.2069999999999999,"5min":4.2069999999999999,"15min":4.2069999999999999},"last":2.2090000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:36:51 2020","interfaces":[{"interface":"back","average":{"1min":2.2589999999999999,"5min":2.3279999999999998,"15min":2.1309999999999998},"min":{"1min":1.7250000000000001,"5min":1.387,"15min":1.387},"max":{"1min":3.681,"5min":5.1829999999999998,"15min":5.1829999999999998},"last":3.004},{"interface":"front","average":{"1min":3.3500000000000001,"5min":3.0369999999999999,"15min":2.7989999999999999},"min":{"1min":1.321,"5min":1.321,"15min":1.321},"max":{"1min":4.8700000000000001,"5min":4.8700000000000001,"15min":4.8700000000000001},"last":2.3109999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:36:14 2020","interfaces":[{"interface":"back","average":{"1min":3.0859999999999999,"5min":3.089,"15min":2.9449999999999998},"min":{"1min":2.105,"5min":1.8879999999999999,"15min":1.8879999999999999},"max":{"1min":4.6440000000000001,"5min":4.6440000000000001,"15min":4.6440000000000001},"last":3.556},{"interface":"front","average":{"1min":2.6019999999999999,"5min":2.5470000000000002,"15min":2.4409999999999998},"min":{"1min":1.9410000000000001,"5min":1.5189999999999999,"15min":1.5189999999999999},"max":{"1min":3.532,"5min":4.6319999999999997,"15min":4.6319999999999997},"last":3.3740000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:36:40 2020","interfaces":[{"interface":"back","average":{"1min":3.9529999999999998,"5min":3.7440000000000002,"15min":3.621},"min":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"max":{"1min":5.367,"5min":6.1399999999999997,"15min":6.1399999999999997},"last":3.4510000000000001},{"interface":"front","average":{"1min":3.8079999999999998,"5min":3.6299999999999999,"15min":3.5609999999999999},"min":{"1min":2.5640000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"max":{"1min":5.7069999999999999,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":3.4159999999999999}]},{"osd":7,"last update":"Tue Sep 8 15:36:09 2020","interfaces":[{"interface":"back","average":{"1min":3.9950000000000001,"5min":3.4540000000000002,"15min":3.2250000000000001},"min":{"1min":2.8860000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"max":{"1min":5.3239999999999998,"5min":5.3239999999999998,"15min":5.3239999999999998},"last":4.3860000000000001},{"interface":"front","average":{"1min":3.9510000000000001,"5min":3.9100000000000001,"15min":3.883},"min":{"1min":1.907,"5min":1.907,"15min":1.907},"max":{"1min":5.375,"5min":5.5650000000000004,"15min":5.5650000000000004},"last":3.1000000000000001}]},{"osd":8,"last update":"Tue Sep 8 15:36:34 2020","interfaces":[{"interface":"back","average":{"1min":4.1890000000000001,"5min":4.2190000000000003,"15min":4.2930000000000001},"min":{"1min":2.6419999999999999,"5min":1.7290000000000001,"15min":1.7290000000000001},"max":{"1min":5.6340000000000003,"5min":5.6340000000000003,"15min":5.6340000000000003},"last":3.6789999999999998},{"interface":"front","average":{"1min":4.2210000000000001,"5min":4.0700000000000003,"15min":4.0800000000000001},"min":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"max":{"1min":6.0960000000000001,"5min":6.0960000000000001,"15min":6.0960000000000001},"last":3.294}]},{"osd":9,"last update":"Tue Sep 8 15:37:00 2020","interfaces":[{"interface":"back","average":{"1min":3.2679999999999998,"5min":3.2970000000000002,"15min":3.2589999999999999},"min":{"1min":2.1970000000000001,"5min":2.089,"15min":2.089},"max":{"1min":4.6230000000000002,"5min":4.8789999999999996,"15min":4.8789999999999996},"last":3.9209999999999998},{"interface":"front","average":{"1min":4.3070000000000004,"5min":4.2720000000000002,"15min":4.2430000000000003},"min":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"max":{"1min":5.8120000000000003,"5min":6.7380000000000004,"15min":6.7380000000000004},"last":3.6349999999999998}]},{"osd":10,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":4.3220000000000001,"5min":4.2320000000000002,"15min":4.2169999999999996},"min":{"1min":2.7949999999999999,"5min":2.3210000000000002,"15min":2.3210000000000002},"max":{"1min":5.968,"5min":5.968,"15min":5.968},"last":4.0570000000000004},{"interface":"front","average":{"1min":3.4620000000000002,"5min":3.4889999999999999,"15min":3.4929999999999999},"min":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"max":{"1min":4.9640000000000004,"5min":4.9790000000000001,"15min":4.9790000000000001},"last":3.5870000000000002}]}]},{"osd":0,"up_from":0,"seq":86,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6396,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738756,"statfs":{"total":95995035648,"available":95988486144,"internally_reserved":0,"allocated":708608,"data_stored":403370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 8 15:36:42 2020","interfaces":[{"interface":"back","average":{"1min":2.407,"5min":1.9450000000000001,"15min":1.4330000000000001},"min":{"1min":1.6279999999999999,"5min":1.02,"15min":0.76100000000000001},"max":{"1min":4.3339999999999996,"5min":4.3339999999999996,"15min":4.3339999999999996},"last":2.7320000000000002},{"interface":"front","average":{"1min":2.121,"5min":1.9319999999999999,"15min":1.46},"min":{"1min":1.6619999999999999,"5min":1.2050000000000001,"15min":0.71999999999999997},"max":{"1min":2.71,"5min":3.8300000000000001,"15min":3.8300000000000001},"last":2.6150000000000002}]},{"osd":2,"last update":"Tue Sep 8 15:34:05 2020","interfaces":[{"interface":"back","average":{"1min":1.9950000000000001,"5min":1.621,"15min":1.5189999999999999},"min":{"1min":1.3999999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.8519999999999999,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":2.6589999999999998},{"interface":"front","average":{"1min":2.629,"5min":1.956,"15min":1.72},"min":{"1min":1.6419999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":2.4049999999999998}]},{"osd":3,"last update":"Tue Sep 8 15:36:32 2020","interfaces":[{"interface":"back","average":{"1min":2.6760000000000002,"5min":2.617,"15min":2.992},"min":{"1min":1.8460000000000001,"5min":1.1879999999999999,"15min":1.1879999999999999},"max":{"1min":4.3140000000000001,"5min":27.298999999999999,"15min":27.298999999999999},"last":2.5710000000000002},{"interface":"front","average":{"1min":3.4289999999999998,"5min":3.3639999999999999,"15min":3.6219999999999999},"min":{"1min":2.2069999999999999,"5min":2.0179999999999998,"15min":2.0179999999999998},"max":{"1min":5.2590000000000003,"5min":28.314,"15min":28.314},"last":3.1920000000000002}]},{"osd":4,"last update":"Tue Sep 8 15:36:56 2020","interfaces":[{"interface":"back","average":{"1min":3.2690000000000001,"5min":3.0169999999999999,"15min":2.5449999999999999},"min":{"1min":2.3279999999999998,"5min":1.7070000000000001,"15min":1.7070000000000001},"max":{"1min":4.1630000000000003,"5min":4.4390000000000001,"15min":4.4390000000000001},"last":2.9950000000000001},{"interface":"front","average":{"1min":3.6850000000000001,"5min":3.2850000000000001,"15min":2.827},"min":{"1min":2.1429999999999998,"5min":1.8080000000000001,"15min":1.8080000000000001},"max":{"1min":4.7800000000000002,"5min":5.1050000000000004,"15min":5.1050000000000004},"last":3.0960000000000001}]},{"osd":5,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":3.8079999999999998,"5min":3.3399999999999999,"15min":3.0089999999999999},"min":{"1min":2.6400000000000001,"5min":2.0990000000000002,"15min":2.0990000000000002},"max":{"1min":4.9809999999999999,"5min":4.9809999999999999,"15min":4.9809999999999999},"last":3.3050000000000002},{"interface":"front","average":{"1min":2.3839999999999999,"5min":3.0259999999999998,"15min":3.1840000000000002},"min":{"1min":1.845,"5min":1.7110000000000001,"15min":1.7110000000000001},"max":{"1min":3.0459999999999998,"5min":5.7439999999999998,"15min":5.7439999999999998},"last":2.6869999999999998}]},{"osd":6,"last update":"Tue Sep 8 15:36:42 2020","interfaces":[{"interface":"back","average":{"1min":2.8679999999999999,"5min":3.2770000000000001,"15min":3.5209999999999999},"min":{"1min":1.746,"5min":1.6439999999999999,"15min":1.6439999999999999},"max":{"1min":5.8730000000000002,"5min":5.8730000000000002,"15min":5.8730000000000002},"last":3.2250000000000001},{"interface":"front","average":{"1min":3.1899999999999999,"5min":3.6749999999999998,"15min":3.923},"min":{"1min":2.5350000000000001,"5min":1.9099999999999999,"15min":1.9099999999999999},"max":{"1min":6.2560000000000002,"5min":6.2560000000000002,"15min":6.2560000000000002},"last":3.444}]},{"osd":7,"last update":"Tue Sep 8 15:36:08 2020","interfaces":[{"interface":"back","average":{"1min":3.2669999999999999,"5min":4.069,"15min":4.3120000000000003},"min":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"max":{"1min":4.758,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":3.6640000000000001},{"interface":"front","average":{"1min":3.9319999999999999,"5min":4.0860000000000003,"15min":4.1040000000000001},"min":{"1min":2.585,"5min":2.585,"15min":2.585},"max":{"1min":5.6429999999999998,"5min":5.8659999999999997,"15min":5.8659999999999997},"last":3.5470000000000002}]},{"osd":8,"last update":"Tue Sep 8 15:36:32 2020","interfaces":[{"interface":"back","average":{"1min":4.5739999999999998,"5min":4.1109999999999998,"15min":3.9489999999999998},"min":{"1min":3.0329999999999999,"5min":2.6629999999999998,"15min":2.6629999999999998},"max":{"1min":6.5049999999999999,"5min":6.6379999999999999,"15min":6.6379999999999999},"last":3.9889999999999999},{"interface":"front","average":{"1min":3.395,"5min":3.9039999999999999,"15min":4.133},"min":{"1min":2.5049999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"max":{"1min":4.5540000000000003,"5min":6.2030000000000003,"15min":6.2030000000000003},"last":4.04}]},{"osd":9,"last update":"Tue Sep 8 15:36:56 2020","interfaces":[{"interface":"back","average":{"1min":4.8120000000000003,"5min":4.391,"15min":4.3200000000000003},"min":{"1min":2.7069999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"max":{"1min":6.1639999999999997,"5min":6.1769999999999996,"15min":6.1769999999999996},"last":3.782},{"interface":"front","average":{"1min":3.8130000000000002,"5min":3.9359999999999999,"15min":4.0670000000000002},"min":{"1min":2.956,"5min":2.2930000000000001,"15min":2.2930000000000001},"max":{"1min":5.3540000000000001,"5min":5.4790000000000001,"15min":5.4790000000000001},"last":4.3120000000000003}]},{"osd":10,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":3.3039999999999998,"5min":3.157,"15min":3.133},"min":{"1min":2.649,"5min":2.3210000000000002,"15min":2.3210000000000002},"max":{"1min":4.1529999999999996,"5min":4.1680000000000001,"15min":4.1680000000000001},"last":4.5890000000000004},{"interface":"front","average":{"1min":4.944,"5min":4.4969999999999999,"15min":4.423},"min":{"1min":3.452,"5min":2.8340000000000001,"15min":2.8340000000000001},"max":{"1min":6.758,"5min":6.758,"15min":6.758},"last":4.4080000000000004}]},{"osd":11,"last update":"Tue Sep 8 15:36:42 2020","interfaces":[{"interface":"back","average":{"1min":4.1379999999999999,"5min":3.9380000000000002,"15min":3.9049999999999998},"min":{"1min":3.1640000000000001,"5min":2.9460000000000002,"15min":2.9460000000000002},"max":{"1min":6.1159999999999997,"5min":6.1159999999999997,"15min":6.1159999999999997},"last":4.54},{"interface":"front","average":{"1min":4.8639999999999999,"5min":4.8609999999999998,"15min":4.8609999999999998},"min":{"1min":3.5800000000000001,"5min":2.911,"15min":2.911},"max":{"1min":6.407,"5min":7.0220000000000002,"15min":7.0220000000000002},"last":4.1749999999999998}]}]},{"osd":2,"up_from":0,"seq":77,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6256,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738896,"statfs":{"total":95995035648,"available":95988629504,"internally_reserved":0,"allocated":696320,"data_stored":395903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.1970000000000001,"5min":1.9179999999999999,"15min":1.623},"min":{"1min":1.4259999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":3.883,"5min":3.883,"15min":3.883},"last":2.2669999999999999},{"interface":"front","average":{"1min":2.0030000000000001,"5min":1.9419999999999999,"15min":1.702},"min":{"1min":1.599,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.7160000000000002,"5min":2.9929999999999999,"15min":2.9929999999999999},"last":2.0579999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.0390000000000001,"5min":1.915,"15min":1.6870000000000001},"min":{"1min":1.306,"5min":0.999,"15min":0.999},"max":{"1min":3.363,"5min":3.363,"15min":3.363},"last":1.903},{"interface":"front","average":{"1min":2.6629999999999998,"5min":2.4340000000000002,"15min":2.0899999999999999},"min":{"1min":1.2809999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"max":{"1min":3.456,"5min":3.9399999999999999,"15min":3.9399999999999999},"last":2.5339999999999998}]},{"osd":3,"last update":"Tue Sep 8 15:36:32 2020","interfaces":[{"interface":"back","average":{"1min":2.944,"5min":2.706,"15min":2.3799999999999999},"min":{"1min":1.7130000000000001,"5min":1.587,"15min":1.587},"max":{"1min":4.1429999999999998,"5min":4.3209999999999997,"15min":4.3209999999999997},"last":1.972},{"interface":"front","average":{"1min":2.7490000000000001,"5min":2.5600000000000001,"15min":2.3679999999999999},"min":{"1min":1.9059999999999999,"5min":0.755,"15min":0.755},"max":{"1min":3.641,"5min":3.6709999999999998,"15min":3.6709999999999998},"last":2.8519999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:36:54 2020","interfaces":[{"interface":"back","average":{"1min":3.1080000000000001,"5min":2.8239999999999998,"15min":2.492},"min":{"1min":1.6479999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"max":{"1min":5.0890000000000004,"5min":5.0890000000000004,"15min":5.0890000000000004},"last":2.6059999999999999},{"interface":"front","average":{"1min":3.1360000000000001,"5min":2.8380000000000001,"15min":2.4460000000000002},"min":{"1min":1.6240000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"max":{"1min":4.4459999999999997,"5min":4.827,"15min":4.827},"last":3.0870000000000002}]},{"osd":5,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":3.5569999999999999,"5min":3.222,"15min":2.9380000000000002},"min":{"1min":2.4060000000000001,"5min":1.353,"15min":1.353},"max":{"1min":4.6050000000000004,"5min":4.8449999999999998,"15min":4.8449999999999998},"last":3.4860000000000002},{"interface":"front","average":{"1min":3.7599999999999998,"5min":3.2879999999999998,"15min":2.98},"min":{"1min":2.4630000000000001,"5min":1.232,"15min":1.232},"max":{"1min":4.4530000000000003,"5min":5.2119999999999997,"15min":5.2119999999999997},"last":3.2040000000000002}]},{"osd":6,"last update":"Tue Sep 8 15:36:44 2020","interfaces":[{"interface":"back","average":{"1min":3.8450000000000002,"5min":3.4199999999999999,"15min":3.1880000000000002},"min":{"1min":2.9780000000000002,"5min":1.5069999999999999,"15min":1.5069999999999999},"max":{"1min":5.4980000000000002,"5min":5.4980000000000002,"15min":5.4980000000000002},"last":3.8399999999999999},{"interface":"front","average":{"1min":3.931,"5min":3.774,"15min":3.5670000000000002},"min":{"1min":3.339,"5min":2.0169999999999999,"15min":2.0169999999999999},"max":{"1min":5.8300000000000001,"5min":5.8300000000000001,"15min":5.8300000000000001},"last":4.0599999999999996}]},{"osd":7,"last update":"Tue Sep 8 15:36:09 2020","interfaces":[{"interface":"back","average":{"1min":4.4720000000000004,"5min":4.1779999999999999,"15min":4.0419999999999998},"min":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"max":{"1min":5.4340000000000002,"5min":6.5780000000000003,"15min":6.5780000000000003},"last":3.6120000000000001},{"interface":"front","average":{"1min":4.0419999999999998,"5min":3.7490000000000001,"15min":3.722},"min":{"1min":2.847,"5min":2.1299999999999999,"15min":2.1299999999999999},"max":{"1min":7.8659999999999997,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":3.2679999999999998}]},{"osd":8,"last update":"Tue Sep 8 15:36:26 2020","interfaces":[{"interface":"back","average":{"1min":3.9900000000000002,"5min":3.7370000000000001,"15min":3.6850000000000001},"min":{"1min":2.5369999999999999,"5min":1.907,"15min":1.907},"max":{"1min":5.5359999999999996,"5min":5.5359999999999996,"15min":5.5359999999999996},"last":4.21},{"interface":"front","average":{"1min":4.2439999999999998,"5min":4.0410000000000004,"15min":3.9889999999999999},"min":{"1min":3.0699999999999998,"5min":2.7719999999999998,"15min":2.7719999999999998},"max":{"1min":5.4939999999999998,"5min":6.3150000000000004,"15min":6.3150000000000004},"last":3.8730000000000002}]},{"osd":9,"last update":"Tue Sep 8 15:35:59 2020","interfaces":[{"interface":"back","average":{"1min":4.0670000000000002,"5min":4.0499999999999998,"15min":4.0469999999999997},"min":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"max":{"1min":6.3559999999999999,"5min":6.3559999999999999,"15min":6.3559999999999999},"last":4.024},{"interface":"front","average":{"1min":4.2220000000000004,"5min":4.2850000000000001,"15min":4.2949999999999999},"min":{"1min":2.71,"5min":2.71,"15min":2.71},"max":{"1min":6.1070000000000002,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":4.2640000000000002}]},{"osd":10,"last update":"Tue Sep 8 15:36:22 2020","interfaces":[{"interface":"back","average":{"1min":4.4729999999999999,"5min":4.1130000000000004,"15min":4.0529999999999999},"min":{"1min":3.145,"5min":2.7480000000000002,"15min":2.7480000000000002},"max":{"1min":6.915,"5min":6.915,"15min":6.915},"last":4.4100000000000001},{"interface":"front","average":{"1min":3.895,"5min":3.8140000000000001,"15min":3.7999999999999998},"min":{"1min":2.6890000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"max":{"1min":7.6580000000000004,"5min":7.6580000000000004,"15min":7.6580000000000004},"last":3.681}]}]},{"osd":3,"up_from":0,"seq":71,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6192,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738960,"statfs":{"total":95995035648,"available":95988695040,"internally_reserved":0,"allocated":696320,"data_stored":395903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:31 2020","interfaces":[{"interface":"back","average":{"1min":2.1379999999999999,"5min":1.7210000000000001,"15min":1.629},"min":{"1min":1.79,"5min":1.1419999999999999,"15min":1.1419999999999999},"max":{"1min":2.7930000000000001,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":3.169},{"interface":"front","average":{"1min":2.6800000000000002,"5min":2.2309999999999999,"15min":2.0990000000000002},"min":{"1min":2.0270000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"max":{"1min":3.5409999999999999,"5min":3.5409999999999999,"15min":3.5409999999999999},"last":2.5579999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:36:35 2020","interfaces":[{"interface":"back","average":{"1min":2.133,"5min":2.0939999999999999,"15min":1.8740000000000001},"min":{"1min":1.2809999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"max":{"1min":3.3399999999999999,"5min":3.6829999999999998,"15min":3.6829999999999998},"last":2.1030000000000002},{"interface":"front","average":{"1min":2.1709999999999998,"5min":2.1400000000000001,"15min":1.8460000000000001},"min":{"1min":1.4059999999999999,"5min":1.341,"15min":1.341},"max":{"1min":3.1549999999999998,"5min":3.3149999999999999,"15min":3.3149999999999999},"last":2.0289999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:36:35 2020","interfaces":[{"interface":"back","average":{"1min":2.5619999999999998,"5min":2.387,"15min":2.145},"min":{"1min":1.7669999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":3.5009999999999999,"5min":3.7810000000000001,"15min":3.7810000000000001},"last":2.4660000000000002},{"interface":"front","average":{"1min":2.1869999999999998,"5min":2.464,"15min":2.2389999999999999},"min":{"1min":1.585,"5min":1.3580000000000001,"15min":1.3580000000000001},"max":{"1min":3.1699999999999999,"5min":3.8010000000000002,"15min":3.8010000000000002},"last":2.1659999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:36:55 2020","interfaces":[{"interface":"back","average":{"1min":2.7280000000000002,"5min":2.7879999999999998,"15min":2.5609999999999999},"min":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"max":{"1min":3.8620000000000001,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":2.7970000000000002},{"interface":"front","average":{"1min":2.7879999999999998,"5min":2.8029999999999999,"15min":2.5110000000000001},"min":{"1min":2.0089999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"max":{"1min":3.6949999999999998,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":3.3860000000000001}]},{"osd":5,"last update":"Tue Sep 8 15:36:15 2020","interfaces":[{"interface":"back","average":{"1min":2.9300000000000002,"5min":2.9830000000000001,"15min":2.8530000000000002},"min":{"1min":1.323,"5min":1.323,"15min":1.323},"max":{"1min":3.9489999999999998,"5min":5.0289999999999999,"15min":5.0289999999999999},"last":3.8300000000000001},{"interface":"front","average":{"1min":3.4569999999999999,"5min":3.3929999999999998,"15min":3.2330000000000001},"min":{"1min":1.988,"5min":1.988,"15min":1.988},"max":{"1min":4.2910000000000004,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":3.347}]},{"osd":6,"last update":"Tue Sep 8 15:36:44 2020","interfaces":[{"interface":"back","average":{"1min":3.4249999999999998,"5min":3.444,"15min":3.3479999999999999},"min":{"1min":1.8680000000000001,"5min":1.714,"15min":1.714},"max":{"1min":4.6159999999999997,"5min":5.4160000000000004,"15min":5.4160000000000004},"last":3.0470000000000002},{"interface":"front","average":{"1min":3.544,"5min":3.6739999999999999,"15min":3.5449999999999999},"min":{"1min":2.093,"5min":2.093,"15min":2.093},"max":{"1min":4.694,"5min":5.3499999999999996,"15min":5.3499999999999996},"last":4.2750000000000004}]},{"osd":7,"last update":"Tue Sep 8 15:36:11 2020","interfaces":[{"interface":"back","average":{"1min":4.234,"5min":4.2649999999999997,"15min":4.2530000000000001},"min":{"1min":3.081,"5min":2.8069999999999999,"15min":2.8069999999999999},"max":{"1min":5.1440000000000001,"5min":5.5640000000000001,"15min":5.5640000000000001},"last":3.5659999999999998},{"interface":"front","average":{"1min":3.577,"5min":3.629,"15min":3.617},"min":{"1min":2.601,"5min":2.601,"15min":2.601},"max":{"1min":4.258,"5min":5.258,"15min":5.258},"last":4.4329999999999998}]},{"osd":8,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":3.8889999999999998,"5min":3.758,"15min":3.7829999999999999},"min":{"1min":2.9119999999999999,"5min":2.2440000000000002,"15min":2.2440000000000002},"max":{"1min":5.3390000000000004,"5min":5.4610000000000003,"15min":5.4610000000000003},"last":4.6820000000000004},{"interface":"front","average":{"1min":4.5220000000000002,"5min":4.3319999999999999,"15min":4.3079999999999998},"min":{"1min":3.206,"5min":2.351,"15min":2.351},"max":{"1min":5.5309999999999997,"5min":6.1239999999999997,"15min":6.1239999999999997},"last":3.7709999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:36:55 2020","interfaces":[{"interface":"back","average":{"1min":3.6899999999999999,"5min":3.8969999999999998,"15min":3.9670000000000001},"min":{"1min":3.125,"5min":2.6709999999999998,"15min":2.6709999999999998},"max":{"1min":4.4359999999999999,"5min":4.6120000000000001,"15min":4.6120000000000001},"last":3.669},{"interface":"front","average":{"1min":4.093,"5min":4.1829999999999998,"15min":4.3369999999999997},"min":{"1min":2.8540000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"max":{"1min":5.2089999999999996,"5min":5.2530000000000001,"15min":5.2530000000000001},"last":5.0709999999999997}]},{"osd":10,"last update":"Tue Sep 8 15:36:21 2020","interfaces":[{"interface":"back","average":{"1min":4.6360000000000001,"5min":4.4450000000000003,"15min":4.4130000000000003},"min":{"1min":2.875,"5min":2.875,"15min":2.875},"max":{"1min":5.7880000000000003,"5min":5.7880000000000003,"15min":5.7880000000000003},"last":4.0490000000000004},{"interface":"front","average":{"1min":4.0990000000000002,"5min":3.8290000000000002,"15min":3.7839999999999998},"min":{"1min":3.194,"5min":2.8759999999999999,"15min":2.8759999999999999},"max":{"1min":4.7599999999999998,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":4.1920000000000002}]},{"osd":11,"last update":"Tue Sep 8 15:36:51 2020","interfaces":[{"interface":"back","average":{"1min":4.6799999999999997,"5min":4.7880000000000003,"15min":4.806},"min":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"max":{"1min":6.0590000000000002,"5min":6.0590000000000002,"15min":6.0590000000000002},"last":4.3620000000000001},{"interface":"front","average":{"1min":4.2190000000000003,"5min":4.1600000000000001,"15min":4.1500000000000004},"min":{"1min":3.3090000000000002,"5min":3.04,"15min":3.04},"max":{"1min":5.2389999999999999,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":3.9340000000000002}]}]},{"osd":4,"up_from":0,"seq":68,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6204,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738948,"statfs":{"total":95995035648,"available":95988682752,"internally_reserved":0,"allocated":708608,"data_stored":403370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:54 2020","interfaces":[{"interface":"back","average":{"1min":2.4329999999999998,"5min":2.121,"15min":1.889},"min":{"1min":1.5880000000000001,"5min":1.01,"15min":1.01},"max":{"1min":5.7130000000000001,"5min":5.7130000000000001,"15min":5.7130000000000001},"last":2.069},{"interface":"front","average":{"1min":2.5369999999999999,"5min":2.2469999999999999,"15min":1.851},"min":{"1min":1.6930000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":4.3650000000000002,"5min":4.3650000000000002,"15min":4.3650000000000002},"last":3.4790000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:36:54 2020","interfaces":[{"interface":"back","average":{"1min":2.3460000000000001,"5min":2.1150000000000002,"15min":1.8779999999999999},"min":{"1min":1.6859999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"max":{"1min":3.4500000000000002,"5min":5.3659999999999997,"15min":5.3659999999999997},"last":2.1600000000000001},{"interface":"front","average":{"1min":2.903,"5min":2.593,"15min":2.1269999999999998},"min":{"1min":1.8899999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"max":{"1min":3.9870000000000001,"5min":4.5700000000000003,"15min":4.5700000000000003},"last":1.9610000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:36:54 2020","interfaces":[{"interface":"back","average":{"1min":3.052,"5min":2.6909999999999998,"15min":2.1389999999999998},"min":{"1min":1.946,"5min":1.236,"15min":1.236},"max":{"1min":4.6150000000000002,"5min":4.6150000000000002,"15min":4.6150000000000002},"last":2.359},{"interface":"front","average":{"1min":3.3380000000000001,"5min":2.8639999999999999,"15min":2.3250000000000002},"min":{"1min":1.8089999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"max":{"1min":5.2779999999999996,"5min":5.2779999999999996,"15min":5.2779999999999996},"last":2.851}]},{"osd":3,"last update":"Tue Sep 8 15:36:54 2020","interfaces":[{"interface":"back","average":{"1min":3.5179999999999998,"5min":3.0640000000000001,"15min":2.5219999999999998},"min":{"1min":2.5790000000000002,"5min":1.389,"15min":1.389},"max":{"1min":6.1429999999999998,"5min":6.1429999999999998,"15min":6.1429999999999998},"last":3.2589999999999999},{"interface":"front","average":{"1min":3.1669999999999998,"5min":2.8290000000000002,"15min":2.3700000000000001},"min":{"1min":2.0390000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"max":{"1min":5.3550000000000004,"5min":5.3550000000000004,"15min":5.3550000000000004},"last":2.762}]},{"osd":5,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":3.9569999999999999,"5min":3.294,"15min":2.7519999999999998},"min":{"1min":2.7149999999999999,"5min":1.014,"15min":1.014},"max":{"1min":6.9800000000000004,"5min":6.9800000000000004,"15min":6.9800000000000004},"last":4.1470000000000002},{"interface":"front","average":{"1min":3.9449999999999998,"5min":3.274,"15min":2.8199999999999998},"min":{"1min":2.5800000000000001,"5min":1.129,"15min":1.129},"max":{"1min":6.2279999999999998,"5min":6.2279999999999998,"15min":6.2279999999999998},"last":3.419}]},{"osd":6,"last update":"Tue Sep 8 15:36:42 2020","interfaces":[{"interface":"back","average":{"1min":4.0700000000000003,"5min":3.6749999999999998,"15min":3.3490000000000002},"min":{"1min":2.9860000000000002,"5min":2.0449999999999999,"15min":2.0449999999999999},"max":{"1min":5.8179999999999996,"5min":6.6870000000000003,"15min":6.6870000000000003},"last":3.0910000000000002},{"interface":"front","average":{"1min":3.8079999999999998,"5min":3.6160000000000001,"15min":3.3660000000000001},"min":{"1min":2.9980000000000002,"5min":2.2429999999999999,"15min":2.2429999999999999},"max":{"1min":5.2030000000000003,"5min":5.2030000000000003,"15min":5.2030000000000003},"last":3.1549999999999998}]},{"osd":7,"last update":"Tue Sep 8 15:36:05 2020","interfaces":[{"interface":"back","average":{"1min":4.1150000000000002,"5min":3.9860000000000002,"15min":3.8700000000000001},"min":{"1min":2.5739999999999998,"5min":1.75,"15min":1.75},"max":{"1min":7.4699999999999998,"5min":7.4699999999999998,"15min":7.4699999999999998},"last":3.7250000000000001},{"interface":"front","average":{"1min":4.0430000000000001,"5min":4.0570000000000004,"15min":3.98},"min":{"1min":2.718,"5min":2.6619999999999999,"15min":2.6619999999999999},"max":{"1min":6.6399999999999997,"5min":6.6399999999999997,"15min":6.6399999999999997},"last":3.6579999999999999}]},{"osd":8,"last update":"Tue Sep 8 15:36:30 2020","interfaces":[{"interface":"back","average":{"1min":4.1280000000000001,"5min":4,"15min":3.9359999999999999},"min":{"1min":3.327,"5min":3.0739999999999998,"15min":3.0739999999999998},"max":{"1min":5.1920000000000002,"5min":5.3929999999999998,"15min":5.3929999999999998},"last":4.7649999999999997},{"interface":"front","average":{"1min":4.681,"5min":4.4619999999999997,"15min":4.3799999999999999},"min":{"1min":3.746,"5min":3.0840000000000001,"15min":3.0840000000000001},"max":{"1min":6.5540000000000003,"5min":6.5540000000000003,"15min":6.5540000000000003},"last":3.9049999999999998}]},{"osd":9,"last update":"Tue Sep 8 15:36:58 2020","interfaces":[{"interface":"back","average":{"1min":4.7400000000000002,"5min":4.4589999999999996,"15min":4.4020000000000001},"min":{"1min":3.403,"5min":2.6299999999999999,"15min":2.6299999999999999},"max":{"1min":7.319,"5min":7.319,"15min":7.319},"last":3.8519999999999999},{"interface":"front","average":{"1min":4.4610000000000003,"5min":4.4080000000000004,"15min":4.4219999999999997},"min":{"1min":3.5640000000000001,"5min":3.5289999999999999,"15min":3.5289999999999999},"max":{"1min":5.6040000000000001,"5min":5.7779999999999996,"15min":5.7779999999999996},"last":4.2880000000000003}]},{"osd":10,"last update":"Tue Sep 8 15:36:20 2020","interfaces":[{"interface":"back","average":{"1min":4.8129999999999997,"5min":4.9130000000000003,"15min":4.9299999999999997},"min":{"1min":3.9769999999999999,"5min":3.3439999999999999,"15min":3.3439999999999999},"max":{"1min":7.9279999999999999,"5min":7.9279999999999999,"15min":7.9279999999999999},"last":4.5140000000000002},{"interface":"front","average":{"1min":4.8440000000000003,"5min":4.9109999999999996,"15min":4.9219999999999997},"min":{"1min":3.7890000000000001,"5min":3.7890000000000001,"15min":3.7890000000000001},"max":{"1min":7.7590000000000003,"5min":7.7590000000000003,"15min":7.7590000000000003},"last":4.0529999999999999}]}]},{"osd":5,"up_from":0,"seq":61,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6192,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738960,"statfs":{"total":95995035648,"available":95988695040,"internally_reserved":0,"allocated":696320,"data_stored":395903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":2.2589999999999999,"5min":2.1240000000000001,"15min":1.8740000000000001},"min":{"1min":1.7749999999999999,"5min":1.119,"15min":1.119},"max":{"1min":3.1320000000000001,"5min":3.96,"15min":3.96},"last":2.1829999999999998},{"interface":"front","average":{"1min":2.5790000000000002,"5min":2.2290000000000001,"15min":1.9670000000000001},"min":{"1min":1.929,"5min":1.4510000000000001,"15min":1.4510000000000001},"max":{"1min":3.7570000000000001,"5min":3.7570000000000001,"15min":3.7570000000000001},"last":2.4209999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.0049999999999999,"5min":2.5259999999999998,"15min":2.3370000000000002},"min":{"1min":1.643,"5min":1.339,"15min":1.339},"max":{"1min":4.2240000000000002,"5min":4.2240000000000002,"15min":4.2240000000000002},"last":2.0659999999999998},{"interface":"front","average":{"1min":2.1779999999999999,"5min":2.0760000000000001,"15min":2.0059999999999998},"min":{"1min":1.7090000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"max":{"1min":2.9830000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":1.8089999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":2.456,"5min":2.2839999999999998,"15min":2.1339999999999999},"min":{"1min":1.8109999999999999,"5min":1.474,"15min":1.474},"max":{"1min":3.5539999999999998,"5min":3.7570000000000001,"15min":3.7570000000000001},"last":2.637},{"interface":"front","average":{"1min":3.4980000000000002,"5min":2.9969999999999999,"15min":2.7410000000000001},"min":{"1min":2.1240000000000001,"5min":1.796,"15min":1.796},"max":{"1min":4.9000000000000004,"5min":4.9000000000000004,"15min":4.9000000000000004},"last":2.7839999999999998}]},{"osd":3,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":3.7639999999999998,"5min":3.234,"15min":2.839},"min":{"1min":2.4249999999999998,"5min":1.6719999999999999,"15min":1.6719999999999999},"max":{"1min":4.718,"5min":5.3899999999999997,"15min":5.3899999999999997},"last":3.3149999999999999},{"interface":"front","average":{"1min":3.2280000000000002,"5min":3.1139999999999999,"15min":2.9129999999999998},"min":{"1min":2.3679999999999999,"5min":1.9990000000000001,"15min":1.9990000000000001},"max":{"1min":4.2050000000000001,"5min":5.4889999999999999,"15min":5.4889999999999999},"last":2.911}]},{"osd":4,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":2.9780000000000002,"5min":2.5710000000000002,"15min":2.3010000000000002},"min":{"1min":2.0910000000000002,"5min":1.6220000000000001,"15min":1.6220000000000001},"max":{"1min":3.5499999999999998,"5min":3.9710000000000001,"15min":3.9710000000000001},"last":2.7149999999999999},{"interface":"front","average":{"1min":3.7210000000000001,"5min":3.0779999999999998,"15min":2.6829999999999998},"min":{"1min":2.819,"5min":1.3979999999999999,"15min":1.3979999999999999},"max":{"1min":6.5190000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":3.153}]},{"osd":6,"last update":"Tue Sep 8 15:36:45 2020","interfaces":[{"interface":"back","average":{"1min":4.1189999999999998,"5min":3.819,"15min":3.609},"min":{"1min":3.2829999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"max":{"1min":4.718,"5min":5.9870000000000001,"15min":5.9870000000000001},"last":4.3860000000000001},{"interface":"front","average":{"1min":4.7560000000000002,"5min":3.6379999999999999,"15min":3.1400000000000001},"min":{"1min":3.766,"5min":1.948,"15min":1.948},"max":{"1min":5.5590000000000002,"5min":5.6600000000000001,"15min":5.6600000000000001},"last":4.7850000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:36:03 2020","interfaces":[{"interface":"back","average":{"1min":4.8559999999999999,"5min":3.9359999999999999,"15min":3.617},"min":{"1min":3.1909999999999998,"5min":2.5859999999999999,"15min":2.5859999999999999},"max":{"1min":6.782,"5min":6.782,"15min":6.782},"last":3.6480000000000001},{"interface":"front","average":{"1min":3.7669999999999999,"5min":3.9849999999999999,"15min":4.0099999999999998},"min":{"1min":2.665,"5min":2.665,"15min":2.665},"max":{"1min":4.8520000000000003,"5min":5.9059999999999997,"15min":5.9059999999999997},"last":3.8039999999999998}]},{"osd":8,"last update":"Tue Sep 8 15:36:34 2020","interfaces":[{"interface":"back","average":{"1min":3.621,"5min":3.677,"15min":3.5990000000000002},"min":{"1min":3.101,"5min":2.8940000000000001,"15min":2.8940000000000001},"max":{"1min":4.2649999999999997,"5min":5.2610000000000001,"15min":5.2610000000000001},"last":3.2589999999999999},{"interface":"front","average":{"1min":4.0570000000000004,"5min":4.0629999999999997,"15min":4.0529999999999999},"min":{"1min":3.4279999999999999,"5min":3.2400000000000002,"15min":3.2400000000000002},"max":{"1min":4.6230000000000002,"5min":5.3319999999999999,"15min":5.3319999999999999},"last":3.718}]},{"osd":9,"last update":"Tue Sep 8 15:37:00 2020","interfaces":[{"interface":"back","average":{"1min":4.5279999999999996,"5min":4.681,"15min":4.5979999999999999},"min":{"1min":3.6349999999999998,"5min":3.4550000000000001,"15min":3.4550000000000001},"max":{"1min":5.7309999999999999,"5min":7.1719999999999997,"15min":7.1719999999999997},"last":4.0179999999999998},{"interface":"front","average":{"1min":4.7400000000000002,"5min":4.3230000000000004,"15min":4.2530000000000001},"min":{"1min":3.6619999999999999,"5min":2.6930000000000001,"15min":2.6930000000000001},"max":{"1min":6.9900000000000002,"5min":6.9900000000000002,"15min":6.9900000000000002},"last":4.1820000000000004}]},{"osd":10,"last update":"Tue Sep 8 15:36:19 2020","interfaces":[{"interface":"back","average":{"1min":4.2569999999999997,"5min":4.2409999999999997,"15min":4.2380000000000004},"min":{"1min":3.3849999999999998,"5min":3.0190000000000001,"15min":3.0190000000000001},"max":{"1min":5.3609999999999998,"5min":5.3609999999999998,"15min":5.3609999999999998},"last":3.444},{"interface":"front","average":{"1min":5.4649999999999999,"5min":4.9219999999999997,"15min":4.8319999999999999},"min":{"1min":3.9039999999999999,"5min":3.9039999999999999,"15min":3.9039999999999999},"max":{"1min":6.3390000000000004,"5min":6.3390000000000004,"15min":6.3390000000000004},"last":4.5730000000000004}]}]}]}} 2020-09-08T15:37:06.597 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2020-09-08T15:37:06.598 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd dump 2020-09-08T15:37:06.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:06 smithi055 bash[9296]: cluster 2020-09-08T15:37:05.579345+0000 mgr.x (mgr.24118) 286 : cluster [DBG] pgmap v330: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:06.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:06 smithi104 bash[9203]: cluster 2020-09-08T15:37:05.579345+0000 mgr.x (mgr.24118) 286 : cluster [DBG] pgmap v330: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:06.731 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:37:06.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:06 smithi166 bash[9446]: cluster 2020-09-08T15:37:05.579345+0000 mgr.x (mgr.24118) 286 : cluster [DBG] pgmap v330: 1 pgs: 1 active+clean; 0 B data, 71 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:07.670 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:07 smithi055 bash[9296]: audit 2020-09-08T15:37:05.677682+0000 mgr.x (mgr.24118) 287 : audit [DBG] from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:37:07.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:07 smithi104 bash[9203]: audit 2020-09-08T15:37:05.677682+0000 mgr.x (mgr.24118) 287 : audit [DBG] from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:37:07.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:07 smithi166 bash[9446]: audit 2020-09-08T15:37:05.677682+0000 mgr.x (mgr.24118) 287 : audit [DBG] from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:37:08.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:08 smithi055 bash[9296]: cluster 2020-09-08T15:37:07.579997+0000 mgr.x (mgr.24118) 288 : cluster [DBG] pgmap v331: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:08.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:08 smithi104 bash[9203]: cluster 2020-09-08T15:37:07.579997+0000 mgr.x (mgr.24118) 288 : cluster [DBG] pgmap v331: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:08.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:08 smithi166 bash[9446]: cluster 2020-09-08T15:37:07.579997+0000 mgr.x (mgr.24118) 288 : cluster [DBG] pgmap v331: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:10.001 INFO:teuthology.orchestra.run.smithi104.stdout:epoch 121 2020-09-08T15:37:10.002 INFO:teuthology.orchestra.run.smithi104.stdout:fsid 759b27b6-f1e7-11ea-a080-001a4aab830c 2020-09-08T15:37:10.002 INFO:teuthology.orchestra.run.smithi104.stdout:created 2020-09-08T15:25:41.788365+0000 2020-09-08T15:37:10.002 INFO:teuthology.orchestra.run.smithi104.stdout:modified 2020-09-08T15:37:04.371880+0000 2020-09-08T15:37:10.002 INFO:teuthology.orchestra.run.smithi104.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2020-09-08T15:37:10.003 INFO:teuthology.orchestra.run.smithi104.stdout:crush_version 27 2020-09-08T15:37:10.003 INFO:teuthology.orchestra.run.smithi104.stdout:full_ratio 0.95 2020-09-08T15:37:10.003 INFO:teuthology.orchestra.run.smithi104.stdout:backfillfull_ratio 0.9 2020-09-08T15:37:10.004 INFO:teuthology.orchestra.run.smithi104.stdout:nearfull_ratio 0.85 2020-09-08T15:37:10.004 INFO:teuthology.orchestra.run.smithi104.stdout:require_min_compat_client luminous 2020-09-08T15:37:10.004 INFO:teuthology.orchestra.run.smithi104.stdout:min_compat_client jewel 2020-09-08T15:37:10.004 INFO:teuthology.orchestra.run.smithi104.stdout:require_osd_release pacific 2020-09-08T15:37:10.004 INFO:teuthology.orchestra.run.smithi104.stdout:pool 1 'device_health_metrics' replicated size 2 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 13 flags hashpspool stripe_width 0 pg_num_min 1 application mgr_devicehealth 2020-09-08T15:37:10.005 INFO:teuthology.orchestra.run.smithi104.stdout:max_osd 12 2020-09-08T15:37:10.005 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0 up in weight 1 up_from 11 up_thru 55 down_at 0 last_clean_interval [0,0) v1:172.21.15.104:6801/82470352 [v2:172.21.15.104:6802/82470352,v1:172.21.15.104:6803/82470352] exists,up e49a7341-7c75-4883-b061-3e8fe9467e6e 2020-09-08T15:37:10.005 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1 up in weight 1 up_from 17 up_thru 18 down_at 0 last_clean_interval [0,0) v1:172.21.15.104:6809/1421798992 [v2:172.21.15.104:6810/1421798992,v1:172.21.15.104:6811/1421798992] exists,up a83df412-110d-440a-b411-ad2a7c36f81a 2020-09-08T15:37:10.005 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.104:6817/2713413369 [v2:172.21.15.104:6818/2713413369,v1:172.21.15.104:6819/2713413369] exists,up ca6da1cc-1d8d-494b-93e6-e065ff70d434 2020-09-08T15:37:10.005 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3 up in weight 1 up_from 29 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.104:6825/1495946248 [v2:172.21.15.104:6826/1495946248,v1:172.21.15.104:6827/1495946248] exists,up e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e 2020-09-08T15:37:10.006 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4 up in weight 1 up_from 35 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.55:6803/1173103260 [v2:172.21.15.55:6804/1173103260,v1:172.21.15.55:6805/1173103260] exists,up d8334c70-f776-4652-944d-aed35a8e43f7 2020-09-08T15:37:10.006 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5 up in weight 1 up_from 42 up_thru 42 down_at 0 last_clean_interval [0,0) v1:172.21.15.55:6811/1047046376 [v2:172.21.15.55:6812/1047046376,v1:172.21.15.55:6813/1047046376] exists,up 6789abd2-9e9b-4c04-81fb-0ce2d2849e39 2020-09-08T15:37:10.006 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6 up in weight 1 up_from 49 up_thru 49 down_at 0 last_clean_interval [0,0) v1:172.21.15.55:6819/2501844793 [v2:172.21.15.55:6820/2501844793,v1:172.21.15.55:6821/2501844793] exists,up ad9a93fe-0ca0-4052-bfa9-e2abb268829e 2020-09-08T15:37:10.006 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7 up in weight 1 up_from 56 up_thru 90 down_at 0 last_clean_interval [0,0) v1:172.21.15.55:6827/128909153 [v2:172.21.15.55:6828/128909153,v1:172.21.15.55:6829/128909153] exists,up f3795c86-4540-442c-a151-b2769fa577e4 2020-09-08T15:37:10.006 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8 up in weight 1 up_from 64 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.166:6801/2108117574 [v2:172.21.15.166:6802/2108117574,v1:172.21.15.166:6803/2108117574] exists,up 3b0ac380-878e-4c1c-9278-80ce67ca2df4 2020-09-08T15:37:10.007 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9 up in weight 1 up_from 72 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.166:6809/1458870080 [v2:172.21.15.166:6810/1458870080,v1:172.21.15.166:6811/1458870080] exists,up ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08 2020-09-08T15:37:10.007 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10 up in weight 1 up_from 81 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.166:6817/62517848 [v2:172.21.15.166:6818/62517848,v1:172.21.15.166:6819/62517848] exists,up c194210e-bd89-4a1f-98fb-6bf44f0b324c 2020-09-08T15:37:10.007 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11 up in weight 1 up_from 90 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.166:6825/691735125 [v2:172.21.15.166:6826/691735125,v1:172.21.15.166:6827/691735125] exists,up 121301fe-a41c-40c0-ad09-b00de37a55e1 2020-09-08T15:37:10.007 INFO:teuthology.orchestra.run.smithi104.stdout:blocklist 172.21.15.104:0/1257365091 expires 2020-09-09T15:29:39.430456+0000 2020-09-08T15:37:10.007 INFO:teuthology.orchestra.run.smithi104.stdout:blocklist 172.21.15.104:0/1399123455 expires 2020-09-09T15:29:39.430456+0000 2020-09-08T15:37:10.008 INFO:teuthology.orchestra.run.smithi104.stdout:blocklist 172.21.15.104:0/2810441605 expires 2020-09-09T15:29:39.430456+0000 2020-09-08T15:37:10.008 INFO:teuthology.orchestra.run.smithi104.stdout:blocklist 172.21.15.104:6800/182241824 expires 2020-09-09T15:29:39.430456+0000 2020-09-08T15:37:10.008 INFO:teuthology.orchestra.run.smithi104.stdout:blocklist 172.21.15.104:6801/182241824 expires 2020-09-09T15:29:39.430456+0000 2020-09-08T15:37:10.008 INFO:teuthology.orchestra.run.smithi104.stdout:blocklist 172.21.15.104:0/2021933044 expires 2020-09-09T15:26:33.248197+0000 2020-09-08T15:37:10.008 INFO:teuthology.orchestra.run.smithi104.stdout:blocklist 172.21.15.104:6800/3129754347 expires 2020-09-09T15:26:33.248197+0000 2020-09-08T15:37:10.009 INFO:teuthology.orchestra.run.smithi104.stdout:blocklist 172.21.15.104:0/1270031705 expires 2020-09-09T15:26:33.248197+0000 2020-09-08T15:37:10.009 INFO:teuthology.orchestra.run.smithi104.stdout:blocklist 172.21.15.104:0/2568432190 expires 2020-09-09T15:26:33.248197+0000 2020-09-08T15:37:10.009 INFO:teuthology.orchestra.run.smithi104.stdout:blocklist 172.21.15.104:6800/1138921844 expires 2020-09-09T15:26:09.867236+0000 2020-09-08T15:37:10.009 INFO:teuthology.orchestra.run.smithi104.stdout:blocklist 172.21.15.104:0/3827328474 expires 2020-09-09T15:26:09.867236+0000 2020-09-08T15:37:10.009 INFO:teuthology.orchestra.run.smithi104.stdout:blocklist 172.21.15.104:6801/1138921844 expires 2020-09-09T15:26:09.867236+0000 2020-09-08T15:37:10.009 INFO:teuthology.orchestra.run.smithi104.stdout:blocklist 172.21.15.104:0/1662035381 expires 2020-09-09T15:26:09.867236+0000 2020-09-08T15:37:10.010 INFO:teuthology.orchestra.run.smithi104.stdout:blocklist 172.21.15.104:6801/3129754347 expires 2020-09-09T15:26:33.248197+0000 2020-09-08T15:37:10.010 INFO:teuthology.orchestra.run.smithi104.stdout:blocklist 172.21.15.104:0/2169968102 expires 2020-09-09T15:26:09.867236+0000 2020-09-08T15:37:10.738 INFO:tasks.cephadm.ceph_manager.ceph:['osd.0 up in weight 1 up_from 11 up_thru 55 down_at 0 last_clean_interval [0,0) v1:172.21.15.104:6801/82470352 [v2:172.21.15.104:6802/82470352,v1:172.21.15.104:6803/82470352] exists,up e49a7341-7c75-4883-b061-3e8fe9467e6e', 'osd.1 up in weight 1 up_from 17 up_thru 18 down_at 0 last_clean_interval [0,0) v1:172.21.15.104:6809/1421798992 [v2:172.21.15.104:6810/1421798992,v1:172.21.15.104:6811/1421798992] exists,up a83df412-110d-440a-b411-ad2a7c36f81a', 'osd.2 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.104:6817/2713413369 [v2:172.21.15.104:6818/2713413369,v1:172.21.15.104:6819/2713413369] exists,up ca6da1cc-1d8d-494b-93e6-e065ff70d434', 'osd.3 up in weight 1 up_from 29 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.104:6825/1495946248 [v2:172.21.15.104:6826/1495946248,v1:172.21.15.104:6827/1495946248] exists,up e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e', 'osd.4 up in weight 1 up_from 35 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.55:6803/1173103260 [v2:172.21.15.55:6804/1173103260,v1:172.21.15.55:6805/1173103260] exists,up d8334c70-f776-4652-944d-aed35a8e43f7', 'osd.5 up in weight 1 up_from 42 up_thru 42 down_at 0 last_clean_interval [0,0) v1:172.21.15.55:6811/1047046376 [v2:172.21.15.55:6812/1047046376,v1:172.21.15.55:6813/1047046376] exists,up 6789abd2-9e9b-4c04-81fb-0ce2d2849e39', 'osd.6 up in weight 1 up_from 49 up_thru 49 down_at 0 last_clean_interval [0,0) v1:172.21.15.55:6819/2501844793 [v2:172.21.15.55:6820/2501844793,v1:172.21.15.55:6821/2501844793] exists,up ad9a93fe-0ca0-4052-bfa9-e2abb268829e', 'osd.7 up in weight 1 up_from 56 up_thru 90 down_at 0 last_clean_interval [0,0) v1:172.21.15.55:6827/128909153 [v2:172.21.15.55:6828/128909153,v1:172.21.15.55:6829/128909153] exists,up f3795c86-4540-442c-a151-b2769fa577e4', 'osd.8 up in weight 1 up_from 64 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.166:6801/2108117574 [v2:172.21.15.166:6802/2108117574,v1:172.21.15.166:6803/2108117574] exists,up 3b0ac380-878e-4c1c-9278-80ce67ca2df4', 'osd.9 up in weight 1 up_from 72 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.166:6809/1458870080 [v2:172.21.15.166:6810/1458870080,v1:172.21.15.166:6811/1458870080] exists,up ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08', 'osd.10 up in weight 1 up_from 81 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.166:6817/62517848 [v2:172.21.15.166:6818/62517848,v1:172.21.15.166:6819/62517848] exists,up c194210e-bd89-4a1f-98fb-6bf44f0b324c', 'osd.11 up in weight 1 up_from 90 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.166:6825/691735125 [v2:172.21.15.166:6826/691735125,v1:172.21.15.166:6827/691735125] exists,up 121301fe-a41c-40c0-ad09-b00de37a55e1'] 2020-09-08T15:37:10.739 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon mon.a config show 2020-09-08T15:37:10.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:10 smithi166 bash[9446]: cluster 2020-09-08T15:37:09.580663+0000 mgr.x (mgr.24118) 289 : cluster [DBG] pgmap v332: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:10.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:10 smithi166 bash[9446]: audit 2020-09-08T15:37:10.005952+0000 mon.a (mon.0) 897 : audit [DBG] from='client.? v1:172.21.15.104:0/1543984045' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2020-09-08T15:37:10.887 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:37:10.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:10 smithi055 bash[9296]: cluster 2020-09-08T15:37:09.580663+0000 mgr.x (mgr.24118) 289 : cluster [DBG] pgmap v332: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:10.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:10 smithi055 bash[9296]: audit 2020-09-08T15:37:10.005952+0000 mon.a (mon.0) 897 : audit [DBG] from='client.? v1:172.21.15.104:0/1543984045' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2020-09-08T15:37:10.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:10 smithi104 bash[9203]: cluster 2020-09-08T15:37:09.580663+0000 mgr.x (mgr.24118) 289 : cluster [DBG] pgmap v332: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:10.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:10 smithi104 bash[9203]: audit 2020-09-08T15:37:10.005952+0000 mon.a (mon.0) 897 : audit [DBG] from='client.? v1:172.21.15.104:0/1543984045' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2020-09-08T15:37:11.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:11 smithi104 bash[9203]: cluster 2020-09-08T15:37:11.581312+0000 mgr.x (mgr.24118) 290 : cluster [DBG] pgmap v333: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:12.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:11 smithi166 bash[9446]: cluster 2020-09-08T15:37:11.581312+0000 mgr.x (mgr.24118) 290 : cluster [DBG] pgmap v333: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:12.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:11 smithi055 bash[9296]: cluster 2020-09-08T15:37:11.581312+0000 mgr.x (mgr.24118) 290 : cluster [DBG] pgmap v333: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:12.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:12 smithi104 bash[9203]: cluster 2020-09-08T15:37:11.906445+0000 osd.5 (osd.5) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:12.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:12 smithi104 bash[9203]: cluster 2020-09-08T15:37:11.906588+0000 osd.5 (osd.5) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:13.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:12 smithi166 bash[9446]: cluster 2020-09-08T15:37:11.906445+0000 osd.5 (osd.5) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:13.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:12 smithi166 bash[9446]: cluster 2020-09-08T15:37:11.906588+0000 osd.5 (osd.5) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:13.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:12 smithi055 bash[9296]: cluster 2020-09-08T15:37:11.906445+0000 osd.5 (osd.5) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:13.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:12 smithi055 bash[9296]: cluster 2020-09-08T15:37:11.906588+0000 osd.5 (osd.5) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:14.040 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:37:14.041 INFO:teuthology.orchestra.run.smithi104.stdout: "name": "mon.a", 2020-09-08T15:37:14.041 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster": "ceph", 2020-09-08T15:37:14.041 INFO:teuthology.orchestra.run.smithi104.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2020-09-08T15:37:14.042 INFO:teuthology.orchestra.run.smithi104.stdout: "admin_socket_mode": "", 2020-09-08T15:37:14.042 INFO:teuthology.orchestra.run.smithi104.stdout: "auth_client_required": "cephx, none", 2020-09-08T15:37:14.042 INFO:teuthology.orchestra.run.smithi104.stdout: "auth_cluster_required": "cephx", 2020-09-08T15:37:14.042 INFO:teuthology.orchestra.run.smithi104.stdout: "auth_debug": "true", 2020-09-08T15:37:14.042 INFO:teuthology.orchestra.run.smithi104.stdout: "auth_mon_ticket_ttl": "43200.000000", 2020-09-08T15:37:14.043 INFO:teuthology.orchestra.run.smithi104.stdout: "auth_service_required": "cephx", 2020-09-08T15:37:14.043 INFO:teuthology.orchestra.run.smithi104.stdout: "auth_service_ticket_ttl": "3600.000000", 2020-09-08T15:37:14.043 INFO:teuthology.orchestra.run.smithi104.stdout: "auth_supported": "", 2020-09-08T15:37:14.043 INFO:teuthology.orchestra.run.smithi104.stdout: "bdev_aio": "true", 2020-09-08T15:37:14.043 INFO:teuthology.orchestra.run.smithi104.stdout: "bdev_aio_max_queue_depth": "1024", 2020-09-08T15:37:14.044 INFO:teuthology.orchestra.run.smithi104.stdout: "bdev_aio_poll_ms": "250", 2020-09-08T15:37:14.044 INFO:teuthology.orchestra.run.smithi104.stdout: "bdev_aio_reap_max": "16", 2020-09-08T15:37:14.044 INFO:teuthology.orchestra.run.smithi104.stdout: "bdev_async_discard": "false", 2020-09-08T15:37:14.044 INFO:teuthology.orchestra.run.smithi104.stdout: "bdev_block_size": "4096", 2020-09-08T15:37:14.044 INFO:teuthology.orchestra.run.smithi104.stdout: "bdev_debug_aio": "false", 2020-09-08T15:37:14.045 INFO:teuthology.orchestra.run.smithi104.stdout: "bdev_debug_aio_log_age": "5.000000", 2020-09-08T15:37:14.045 INFO:teuthology.orchestra.run.smithi104.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2020-09-08T15:37:14.046 INFO:teuthology.orchestra.run.smithi104.stdout: "bdev_debug_inflight_ios": "false", 2020-09-08T15:37:14.046 INFO:teuthology.orchestra.run.smithi104.stdout: "bdev_enable_discard": "false", 2020-09-08T15:37:14.046 INFO:teuthology.orchestra.run.smithi104.stdout: "bdev_flock_retry": "3", 2020-09-08T15:37:14.047 INFO:teuthology.orchestra.run.smithi104.stdout: "bdev_flock_retry_interval": "0.100000", 2020-09-08T15:37:14.047 INFO:teuthology.orchestra.run.smithi104.stdout: "bdev_inject_crash": "0", 2020-09-08T15:37:14.047 INFO:teuthology.orchestra.run.smithi104.stdout: "bdev_inject_crash_flush_delay": "2", 2020-09-08T15:37:14.047 INFO:teuthology.orchestra.run.smithi104.stdout: "bdev_ioring": "false", 2020-09-08T15:37:14.047 INFO:teuthology.orchestra.run.smithi104.stdout: "bdev_nvme_unbind_from_kernel": "false", 2020-09-08T15:37:14.048 INFO:teuthology.orchestra.run.smithi104.stdout: "bdev_type": "", 2020-09-08T15:37:14.048 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_alloc_size": "1048576", 2020-09-08T15:37:14.048 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_allocator": "hybrid", 2020-09-08T15:37:14.048 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_buffered_io": "false", 2020-09-08T15:37:14.048 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_compact_log_sync": "false", 2020-09-08T15:37:14.049 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2020-09-08T15:37:14.049 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_log_compact_min_size": "16777216", 2020-09-08T15:37:14.049 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_log_replay_check_allocations": "true", 2020-09-08T15:37:14.049 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_max_log_runway": "4194304", 2020-09-08T15:37:14.050 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_max_prefetch": "1048576", 2020-09-08T15:37:14.050 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_min_flush_size": "524288", 2020-09-08T15:37:14.050 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_min_log_runway": "1048576", 2020-09-08T15:37:14.052 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_replay_recovery": "false", 2020-09-08T15:37:14.052 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_replay_recovery_disable_compact": "false", 2020-09-08T15:37:14.052 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_shared_alloc_size": "65536", 2020-09-08T15:37:14.052 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_sync_write": "false", 2020-09-08T15:37:14.053 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2020-09-08T15:37:14.053 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2020-09-08T15:37:14.053 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2020-09-08T15:37:14.053 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_allocator": "hybrid", 2020-09-08T15:37:14.053 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2020-09-08T15:37:14.054 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2020-09-08T15:37:14.054 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2020-09-08T15:37:14.054 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_bitmapallocator_span_size": "1024", 2020-09-08T15:37:14.054 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blobid_prealloc": "10240", 2020-09-08T15:37:14.054 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_block_create": "true", 2020-09-08T15:37:14.055 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_block_db_create": "false", 2020-09-08T15:37:14.055 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_block_db_path": "", 2020-09-08T15:37:14.055 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_block_db_size": "0", 2020-09-08T15:37:14.055 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_block_path": "", 2020-09-08T15:37:14.055 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_block_preallocate_file": "false", 2020-09-08T15:37:14.056 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_block_size": "107374182400", 2020-09-08T15:37:14.056 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_block_wal_create": "false", 2020-09-08T15:37:14.056 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_block_wal_path": "", 2020-09-08T15:37:14.056 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_block_wal_size": "100663296", 2020-09-08T15:37:14.056 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_bluefs": "true", 2020-09-08T15:37:14.057 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2020-09-08T15:37:14.057 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_bluefs_env_mirror": "false", 2020-09-08T15:37:14.057 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_autotune": "true", 2020-09-08T15:37:14.057 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_autotune_interval": "5.000000", 2020-09-08T15:37:14.057 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_kv_ratio": "0.400000", 2020-09-08T15:37:14.058 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_ratio": "0.400000", 2020-09-08T15:37:14.058 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_size": "0", 2020-09-08T15:37:14.058 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_size_hdd": "1073741824", 2020-09-08T15:37:14.058 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_size_ssd": "3221225472", 2020-09-08T15:37:14.058 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_trim_interval": "0.050000", 2020-09-08T15:37:14.059 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_trim_max_skip_pinned": "64", 2020-09-08T15:37:14.059 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_type": "2q", 2020-09-08T15:37:14.059 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_clone_cow": "true", 2020-09-08T15:37:14.059 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_compression_algorithm": "snappy", 2020-09-08T15:37:14.059 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_compression_max_blob_size": "0", 2020-09-08T15:37:14.059 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2020-09-08T15:37:14.060 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2020-09-08T15:37:14.060 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_compression_min_blob_size": "0", 2020-09-08T15:37:14.061 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2020-09-08T15:37:14.061 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_compression_min_blob_size_ssd": "8192", 2020-09-08T15:37:14.062 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_compression_mode": "none", 2020-09-08T15:37:14.062 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_compression_required_ratio": "0.875000", 2020-09-08T15:37:14.062 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_csum_type": "crc32c", 2020-09-08T15:37:14.062 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_debug_enforce_settings": "default", 2020-09-08T15:37:14.062 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_debug_freelist": "false", 2020-09-08T15:37:14.062 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_debug_fsck_abort": "false", 2020-09-08T15:37:14.063 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_debug_inject_bug21040": "false", 2020-09-08T15:37:14.063 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2020-09-08T15:37:14.063 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_debug_inject_read_err": "false", 2020-09-08T15:37:14.063 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_debug_misc": "false", 2020-09-08T15:37:14.063 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_debug_no_reuse_blocks": "false", 2020-09-08T15:37:14.064 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_debug_omit_block_device_write": "false", 2020-09-08T15:37:14.064 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_debug_omit_kv_commit": "false", 2020-09-08T15:37:14.064 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2020-09-08T15:37:14.064 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_debug_prefill": "0.000000", 2020-09-08T15:37:14.064 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_debug_prefragment_max": "1048576", 2020-09-08T15:37:14.065 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_debug_random_read_err": "0.000000", 2020-09-08T15:37:14.065 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2020-09-08T15:37:14.065 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_debug_small_allocations": "0", 2020-09-08T15:37:14.065 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2020-09-08T15:37:14.065 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_default_buffered_read": "true", 2020-09-08T15:37:14.066 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_default_buffered_write": "false", 2020-09-08T15:37:14.066 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_deferred_batch_ops": "0", 2020-09-08T15:37:14.066 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2020-09-08T15:37:14.066 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2020-09-08T15:37:14.066 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2020-09-08T15:37:14.067 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_map_shard_max_size": "1200", 2020-09-08T15:37:14.067 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_map_shard_min_size": "150", 2020-09-08T15:37:14.067 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_map_shard_target_size": "500", 2020-09-08T15:37:14.067 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2020-09-08T15:37:14.068 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_freelist_blocks_per_key": "128", 2020-09-08T15:37:14.068 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2020-09-08T15:37:14.068 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2020-09-08T15:37:14.068 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_on_mkfs": "true", 2020-09-08T15:37:14.068 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2020-09-08T15:37:14.069 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_on_mount": "false", 2020-09-08T15:37:14.069 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_on_mount_deep": "false", 2020-09-08T15:37:14.069 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_on_umount": "false", 2020-09-08T15:37:14.069 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_on_umount_deep": "false", 2020-09-08T15:37:14.069 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_quick_fix_on_mount": "true", 2020-09-08T15:37:14.070 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_quick_fix_threads": "2", 2020-09-08T15:37:14.070 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2020-09-08T15:37:14.070 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_gc_enable_blob_threshold": "0", 2020-09-08T15:37:14.070 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_gc_enable_total_threshold": "0", 2020-09-08T15:37:14.070 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2020-09-08T15:37:14.071 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_ignore_data_csum": "false", 2020-09-08T15:37:14.071 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_kvbackend": "rocksdb", 2020-09-08T15:37:14.071 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_log_collection_list_age": "60.000000", 2020-09-08T15:37:14.071 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2020-09-08T15:37:14.071 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_log_op_age": "5.000000", 2020-09-08T15:37:14.071 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_max_alloc_size": "0", 2020-09-08T15:37:14.072 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_max_blob_size": "0", 2020-09-08T15:37:14.072 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_max_blob_size_hdd": "65536", 2020-09-08T15:37:14.072 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_max_blob_size_ssd": "65536", 2020-09-08T15:37:14.072 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_max_defer_interval": "3.000000", 2020-09-08T15:37:14.072 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_max_deferred_txc": "32", 2020-09-08T15:37:14.073 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_min_alloc_size": "0", 2020-09-08T15:37:14.073 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_min_alloc_size_hdd": "4096", 2020-09-08T15:37:14.073 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_min_alloc_size_ssd": "4096", 2020-09-08T15:37:14.073 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_nid_prealloc": "1024", 2020-09-08T15:37:14.073 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_prefer_deferred_size": "0", 2020-09-08T15:37:14.074 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2020-09-08T15:37:14.074 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2020-09-08T15:37:14.074 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_retry_disk_reads": "3", 2020-09-08T15:37:14.074 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_rocksdb_cf": "true", 2020-09-08T15:37:14.074 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_rocksdb_cfs": "m(3) O(3,0-13) L", 2020-09-08T15:37:14.074 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2020-09-08T15:37:14.075 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_spdk_coremask": "0x1", 2020-09-08T15:37:14.075 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_spdk_io_sleep": "5", 2020-09-08T15:37:14.075 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_spdk_max_io_completion": "0", 2020-09-08T15:37:14.075 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_spdk_mem": "512", 2020-09-08T15:37:14.075 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_sync_submit_transaction": "false", 2020-09-08T15:37:14.076 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_throttle_bytes": "67108864", 2020-09-08T15:37:14.076 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_throttle_cost_per_io": "0", 2020-09-08T15:37:14.076 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2020-09-08T15:37:14.076 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2020-09-08T15:37:14.076 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2020-09-08T15:37:14.077 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_throttle_trace_rate": "0.000000", 2020-09-08T15:37:14.077 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_tracing": "false", 2020-09-08T15:37:14.077 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2020-09-08T15:37:14.077 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_volume_selection_reserved": "0", 2020-09-08T15:37:14.077 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2020-09-08T15:37:14.078 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2020-09-08T15:37:14.078 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_warn_on_legacy_statfs": "true", 2020-09-08T15:37:14.078 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2020-09-08T15:37:14.078 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2020-09-08T15:37:14.078 INFO:teuthology.orchestra.run.smithi104.stdout: "cephadm_path": "/usr/sbin/cephadm", 2020-09-08T15:37:14.079 INFO:teuthology.orchestra.run.smithi104.stdout: "cephfs_mirror_directory_choose_policy": "random", 2020-09-08T15:37:14.079 INFO:teuthology.orchestra.run.smithi104.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2020-09-08T15:37:14.079 INFO:teuthology.orchestra.run.smithi104.stdout: "cephx_cluster_require_signatures": "false", 2020-09-08T15:37:14.079 INFO:teuthology.orchestra.run.smithi104.stdout: "cephx_cluster_require_version": "1", 2020-09-08T15:37:14.079 INFO:teuthology.orchestra.run.smithi104.stdout: "cephx_require_signatures": "false", 2020-09-08T15:37:14.080 INFO:teuthology.orchestra.run.smithi104.stdout: "cephx_require_version": "1", 2020-09-08T15:37:14.080 INFO:teuthology.orchestra.run.smithi104.stdout: "cephx_service_require_signatures": "false", 2020-09-08T15:37:14.080 INFO:teuthology.orchestra.run.smithi104.stdout: "cephx_service_require_version": "1", 2020-09-08T15:37:14.080 INFO:teuthology.orchestra.run.smithi104.stdout: "cephx_sign_messages": "true", 2020-09-08T15:37:14.080 INFO:teuthology.orchestra.run.smithi104.stdout: "chdir": "", 2020-09-08T15:37:14.080 INFO:teuthology.orchestra.run.smithi104.stdout: "client_acl_type": "", 2020-09-08T15:37:14.081 INFO:teuthology.orchestra.run.smithi104.stdout: "client_asio_thread_count": "2", 2020-09-08T15:37:14.081 INFO:teuthology.orchestra.run.smithi104.stdout: "client_cache_mid": "0.750000", 2020-09-08T15:37:14.082 INFO:teuthology.orchestra.run.smithi104.stdout: "client_cache_size": "16384", 2020-09-08T15:37:14.082 INFO:teuthology.orchestra.run.smithi104.stdout: "client_caps_release_delay": "5", 2020-09-08T15:37:14.083 INFO:teuthology.orchestra.run.smithi104.stdout: "client_check_pool_perm": "true", 2020-09-08T15:37:14.083 INFO:teuthology.orchestra.run.smithi104.stdout: "client_debug_force_sync_read": "false", 2020-09-08T15:37:14.083 INFO:teuthology.orchestra.run.smithi104.stdout: "client_debug_getattr_caps": "false", 2020-09-08T15:37:14.084 INFO:teuthology.orchestra.run.smithi104.stdout: "client_debug_inject_tick_delay": "0", 2020-09-08T15:37:14.084 INFO:teuthology.orchestra.run.smithi104.stdout: "client_die_on_failed_dentry_invalidate": "true", 2020-09-08T15:37:14.084 INFO:teuthology.orchestra.run.smithi104.stdout: "client_die_on_failed_remount": "false", 2020-09-08T15:37:14.084 INFO:teuthology.orchestra.run.smithi104.stdout: "client_dirsize_rbytes": "true", 2020-09-08T15:37:14.084 INFO:teuthology.orchestra.run.smithi104.stdout: "client_force_lazyio": "false", 2020-09-08T15:37:14.084 INFO:teuthology.orchestra.run.smithi104.stdout: "client_fs": "", 2020-09-08T15:37:14.085 INFO:teuthology.orchestra.run.smithi104.stdout: "client_inject_fixed_oldest_tid": "false", 2020-09-08T15:37:14.085 INFO:teuthology.orchestra.run.smithi104.stdout: "client_inject_release_failure": "false", 2020-09-08T15:37:14.085 INFO:teuthology.orchestra.run.smithi104.stdout: "client_max_inline_size": "4096", 2020-09-08T15:37:14.085 INFO:teuthology.orchestra.run.smithi104.stdout: "client_mds_namespace": "", 2020-09-08T15:37:14.085 INFO:teuthology.orchestra.run.smithi104.stdout: "client_metadata": "", 2020-09-08T15:37:14.086 INFO:teuthology.orchestra.run.smithi104.stdout: "client_mount_gid": "-1", 2020-09-08T15:37:14.086 INFO:teuthology.orchestra.run.smithi104.stdout: "client_mount_timeout": "300.000000", 2020-09-08T15:37:14.086 INFO:teuthology.orchestra.run.smithi104.stdout: "client_mount_uid": "-1", 2020-09-08T15:37:14.086 INFO:teuthology.orchestra.run.smithi104.stdout: "client_mountpoint": "/", 2020-09-08T15:37:14.087 INFO:teuthology.orchestra.run.smithi104.stdout: "client_notify_timeout": "10", 2020-09-08T15:37:14.087 INFO:teuthology.orchestra.run.smithi104.stdout: "client_oc": "true", 2020-09-08T15:37:14.087 INFO:teuthology.orchestra.run.smithi104.stdout: "client_oc_max_dirty": "104857600", 2020-09-08T15:37:14.087 INFO:teuthology.orchestra.run.smithi104.stdout: "client_oc_max_dirty_age": "5.000000", 2020-09-08T15:37:14.087 INFO:teuthology.orchestra.run.smithi104.stdout: "client_oc_max_objects": "1000", 2020-09-08T15:37:14.087 INFO:teuthology.orchestra.run.smithi104.stdout: "client_oc_size": "209715200", 2020-09-08T15:37:14.088 INFO:teuthology.orchestra.run.smithi104.stdout: "client_oc_target_dirty": "8388608", 2020-09-08T15:37:14.088 INFO:teuthology.orchestra.run.smithi104.stdout: "client_permissions": "true", 2020-09-08T15:37:14.088 INFO:teuthology.orchestra.run.smithi104.stdout: "client_quota_df": "true", 2020-09-08T15:37:14.088 INFO:teuthology.orchestra.run.smithi104.stdout: "client_readahead_max_bytes": "0", 2020-09-08T15:37:14.088 INFO:teuthology.orchestra.run.smithi104.stdout: "client_readahead_max_periods": "4", 2020-09-08T15:37:14.089 INFO:teuthology.orchestra.run.smithi104.stdout: "client_readahead_min": "131072", 2020-09-08T15:37:14.089 INFO:teuthology.orchestra.run.smithi104.stdout: "client_reconnect_stale": "false", 2020-09-08T15:37:14.089 INFO:teuthology.orchestra.run.smithi104.stdout: "client_shutdown_timeout": "30", 2020-09-08T15:37:14.089 INFO:teuthology.orchestra.run.smithi104.stdout: "client_snapdir": ".snap", 2020-09-08T15:37:14.090 INFO:teuthology.orchestra.run.smithi104.stdout: "client_tick_interval": "1.000000", 2020-09-08T15:37:14.090 INFO:teuthology.orchestra.run.smithi104.stdout: "client_trace": "", 2020-09-08T15:37:14.090 INFO:teuthology.orchestra.run.smithi104.stdout: "client_try_dentry_invalidate": "false", 2020-09-08T15:37:14.090 INFO:teuthology.orchestra.run.smithi104.stdout: "client_use_faked_inos": "false", 2020-09-08T15:37:14.090 INFO:teuthology.orchestra.run.smithi104.stdout: "client_use_random_mds": "false", 2020-09-08T15:37:14.090 INFO:teuthology.orchestra.run.smithi104.stdout: "clog_to_graylog": "false", 2020-09-08T15:37:14.091 INFO:teuthology.orchestra.run.smithi104.stdout: "clog_to_graylog_host": "127.0.0.1", 2020-09-08T15:37:14.091 INFO:teuthology.orchestra.run.smithi104.stdout: "clog_to_graylog_port": "12201", 2020-09-08T15:37:14.091 INFO:teuthology.orchestra.run.smithi104.stdout: "clog_to_monitors": "default=true", 2020-09-08T15:37:14.091 INFO:teuthology.orchestra.run.smithi104.stdout: "clog_to_syslog": "false", 2020-09-08T15:37:14.091 INFO:teuthology.orchestra.run.smithi104.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2020-09-08T15:37:14.092 INFO:teuthology.orchestra.run.smithi104.stdout: "clog_to_syslog_level": "info", 2020-09-08T15:37:14.092 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "-", 2020-09-08T15:37:14.092 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_network": "", 2020-09-08T15:37:14.092 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_network_interface": "", 2020-09-08T15:37:14.092 INFO:teuthology.orchestra.run.smithi104.stdout: "compressor_zlib_isal": "false", 2020-09-08T15:37:14.093 INFO:teuthology.orchestra.run.smithi104.stdout: "compressor_zlib_level": "5", 2020-09-08T15:37:14.093 INFO:teuthology.orchestra.run.smithi104.stdout: "compressor_zlib_winsize": "-15", 2020-09-08T15:37:14.093 INFO:teuthology.orchestra.run.smithi104.stdout: "compressor_zstd_level": "1", 2020-09-08T15:37:14.093 INFO:teuthology.orchestra.run.smithi104.stdout: "container_image": "quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f", 2020-09-08T15:37:14.093 INFO:teuthology.orchestra.run.smithi104.stdout: "crash_dir": "/var/lib/ceph/crash", 2020-09-08T15:37:14.094 INFO:teuthology.orchestra.run.smithi104.stdout: "crimson_osd_obc_lru_size": "10", 2020-09-08T15:37:14.094 INFO:teuthology.orchestra.run.smithi104.stdout: "crimson_osd_scheduler_concurrency": "0", 2020-09-08T15:37:14.094 INFO:teuthology.orchestra.run.smithi104.stdout: "crush_location": "", 2020-09-08T15:37:14.094 INFO:teuthology.orchestra.run.smithi104.stdout: "crush_location_hook": "", 2020-09-08T15:37:14.094 INFO:teuthology.orchestra.run.smithi104.stdout: "crush_location_hook_timeout": "10", 2020-09-08T15:37:14.095 INFO:teuthology.orchestra.run.smithi104.stdout: "daemonize": "false", 2020-09-08T15:37:14.095 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_allow_any_pool_priority": "false", 2020-09-08T15:37:14.095 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_asok": "1/5", 2020-09-08T15:37:14.095 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_asok_assert_abort": "false", 2020-09-08T15:37:14.096 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_asserts_on_shutdown": "true", 2020-09-08T15:37:14.096 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_auth": "1/5", 2020-09-08T15:37:14.097 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_bdev": "1/3", 2020-09-08T15:37:14.097 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_bluefs": "1/5", 2020-09-08T15:37:14.097 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_bluestore": "1/5", 2020-09-08T15:37:14.097 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_buffer": "0/1", 2020-09-08T15:37:14.098 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_cephfs_mirror": "20/20", 2020-09-08T15:37:14.098 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_civetweb": "1/10", 2020-09-08T15:37:14.098 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_client": "0/5", 2020-09-08T15:37:14.098 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_compressor": "1/5", 2020-09-08T15:37:14.098 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_context": "0/1", 2020-09-08T15:37:14.099 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_crush": "1/1", 2020-09-08T15:37:14.099 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_crypto": "1/5", 2020-09-08T15:37:14.099 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_deliberately_leak_memory": "false", 2020-09-08T15:37:14.099 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_disable_randomized_ping": "false", 2020-09-08T15:37:14.099 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_dpdk": "1/5", 2020-09-08T15:37:14.100 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_eventtrace": "1/5", 2020-09-08T15:37:14.100 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_filer": "0/1", 2020-09-08T15:37:14.100 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_filestore": "1/3", 2020-09-08T15:37:14.100 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_finisher": "1/1", 2020-09-08T15:37:14.100 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_fuse": "1/5", 2020-09-08T15:37:14.100 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_heartbeat_testing_span": "0", 2020-09-08T15:37:14.101 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_heartbeatmap": "1/5", 2020-09-08T15:37:14.101 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_immutable_obj_cache": "0/5", 2020-09-08T15:37:14.101 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_javaclient": "1/5", 2020-09-08T15:37:14.101 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_journal": "1/3", 2020-09-08T15:37:14.102 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_journaler": "0/5", 2020-09-08T15:37:14.102 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_kstore": "1/5", 2020-09-08T15:37:14.102 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_leveldb": "4/5", 2020-09-08T15:37:14.102 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_lockdep": "0/1", 2020-09-08T15:37:14.102 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_mds": "1/5", 2020-09-08T15:37:14.102 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_mds_balancer": "1/5", 2020-09-08T15:37:14.103 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_mds_locker": "1/5", 2020-09-08T15:37:14.103 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_mds_log": "1/5", 2020-09-08T15:37:14.103 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_mds_log_expire": "1/5", 2020-09-08T15:37:14.103 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_mds_migrator": "1/5", 2020-09-08T15:37:14.103 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_memdb": "4/5", 2020-09-08T15:37:14.104 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_mgr": "1/5", 2020-09-08T15:37:14.104 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_mgrc": "1/5", 2020-09-08T15:37:14.104 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_mon": "20/20", 2020-09-08T15:37:14.104 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_monc": "0/10", 2020-09-08T15:37:14.104 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_ms": "1/1", 2020-09-08T15:37:14.105 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_none": "0/5", 2020-09-08T15:37:14.105 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_objclass": "0/5", 2020-09-08T15:37:14.105 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_objectcacher": "0/5", 2020-09-08T15:37:14.105 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_objecter": "0/1", 2020-09-08T15:37:14.105 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_optracker": "0/5", 2020-09-08T15:37:14.105 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_osd": "1/5", 2020-09-08T15:37:14.106 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_paxos": "20/20", 2020-09-08T15:37:14.106 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_perfcounter": "1/5", 2020-09-08T15:37:14.106 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_prioritycache": "1/5", 2020-09-08T15:37:14.106 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_rados": "0/5", 2020-09-08T15:37:14.106 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_rbd": "0/5", 2020-09-08T15:37:14.107 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_rbd_mirror": "0/5", 2020-09-08T15:37:14.107 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_rbd_replay": "0/5", 2020-09-08T15:37:14.107 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_rbd_rwl": "0/5", 2020-09-08T15:37:14.107 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_refs": "0/0", 2020-09-08T15:37:14.107 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_reserver": "1/1", 2020-09-08T15:37:14.108 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_rgw": "1/5", 2020-09-08T15:37:14.108 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_rgw_sync": "1/5", 2020-09-08T15:37:14.108 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_rocksdb": "4/5", 2020-09-08T15:37:14.108 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_striper": "0/1", 2020-09-08T15:37:14.108 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_test": "0/5", 2020-09-08T15:37:14.108 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_throttle": "1/1", 2020-09-08T15:37:14.109 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_timer": "0/1", 2020-09-08T15:37:14.109 INFO:teuthology.orchestra.run.smithi104.stdout: "debug_tp": "0/5", 2020-09-08T15:37:14.109 INFO:teuthology.orchestra.run.smithi104.stdout: "device_failure_prediction_mode": "none", 2020-09-08T15:37:14.109 INFO:teuthology.orchestra.run.smithi104.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2020-09-08T15:37:14.109 INFO:teuthology.orchestra.run.smithi104.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2020-09-08T15:37:14.110 INFO:teuthology.orchestra.run.smithi104.stdout: "err_to_graylog": "false", 2020-09-08T15:37:14.111 INFO:teuthology.orchestra.run.smithi104.stdout: "err_to_stderr": "true", 2020-09-08T15:37:14.111 INFO:teuthology.orchestra.run.smithi104.stdout: "err_to_syslog": "false", 2020-09-08T15:37:14.112 INFO:teuthology.orchestra.run.smithi104.stdout: "event_tracing": "false", 2020-09-08T15:37:14.112 INFO:teuthology.orchestra.run.smithi104.stdout: "fake_statfs_for_testing": "0", 2020-09-08T15:37:14.112 INFO:teuthology.orchestra.run.smithi104.stdout: "fatal_signal_handlers": "true", 2020-09-08T15:37:14.112 INFO:teuthology.orchestra.run.smithi104.stdout: "filer_max_purge_ops": "10", 2020-09-08T15:37:14.112 INFO:teuthology.orchestra.run.smithi104.stdout: "filer_max_truncate_ops": "128", 2020-09-08T15:37:14.112 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_apply_finisher_threads": "1", 2020-09-08T15:37:14.113 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_blackhole": "false", 2020-09-08T15:37:14.113 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_btrfs_clone_range": "true", 2020-09-08T15:37:14.113 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_btrfs_snap": "true", 2020-09-08T15:37:14.113 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_caller_concurrency": "10", 2020-09-08T15:37:14.114 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_collect_device_partition_information": "true", 2020-09-08T15:37:14.114 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_commit_timeout": "600.000000", 2020-09-08T15:37:14.114 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_debug_inject_read_err": "false", 2020-09-08T15:37:14.114 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_debug_omap_check": "false", 2020-09-08T15:37:14.114 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_debug_random_read_err": "0.000000", 2020-09-08T15:37:14.115 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_debug_verify_split": "false", 2020-09-08T15:37:14.115 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_dump_file": "", 2020-09-08T15:37:14.115 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2020-09-08T15:37:14.115 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_expected_throughput_ops": "200.000000", 2020-09-08T15:37:14.115 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_fadvise": "true", 2020-09-08T15:37:14.115 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_fail_eio": "true", 2020-09-08T15:37:14.116 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_fd_cache_shards": "16", 2020-09-08T15:37:14.116 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_fd_cache_size": "128", 2020-09-08T15:37:14.116 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_fiemap": "false", 2020-09-08T15:37:14.116 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_fiemap_threshold": "4096", 2020-09-08T15:37:14.116 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_fsync_flushes_journal_data": "false", 2020-09-08T15:37:14.117 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_index_retry_probability": "0.000000", 2020-09-08T15:37:14.117 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_inject_stall": "0", 2020-09-08T15:37:14.117 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_journal_parallel": "false", 2020-09-08T15:37:14.117 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_journal_trailing": "false", 2020-09-08T15:37:14.117 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_journal_writeahead": "false", 2020-09-08T15:37:14.118 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_kill_at": "0", 2020-09-08T15:37:14.118 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_max_alloc_hint_size": "1048576", 2020-09-08T15:37:14.119 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_max_inline_xattr_size": "0", 2020-09-08T15:37:14.119 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2020-09-08T15:37:14.119 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_max_inline_xattr_size_other": "512", 2020-09-08T15:37:14.119 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2020-09-08T15:37:14.120 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_max_inline_xattrs": "0", 2020-09-08T15:37:14.120 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2020-09-08T15:37:14.120 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_max_inline_xattrs_other": "2", 2020-09-08T15:37:14.120 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_max_inline_xattrs_xfs": "10", 2020-09-08T15:37:14.120 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_max_sync_interval": "5.000000", 2020-09-08T15:37:14.121 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_max_xattr_value_size": "0", 2020-09-08T15:37:14.121 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2020-09-08T15:37:14.121 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_max_xattr_value_size_other": "1024", 2020-09-08T15:37:14.121 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2020-09-08T15:37:14.121 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_merge_threshold": "-10", 2020-09-08T15:37:14.122 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_min_sync_interval": "0.010000", 2020-09-08T15:37:14.122 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_odsync_write": "false", 2020-09-08T15:37:14.122 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_omap_backend": "rocksdb", 2020-09-08T15:37:14.122 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_omap_backend_path": "", 2020-09-08T15:37:14.122 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_omap_header_cache_size": "1024", 2020-09-08T15:37:14.123 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_ondisk_finisher_threads": "1", 2020-09-08T15:37:14.123 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_op_thread_suicide_timeout": "180", 2020-09-08T15:37:14.123 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_op_thread_timeout": "60", 2020-09-08T15:37:14.123 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_op_threads": "2", 2020-09-08T15:37:14.123 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_punch_hole": "false", 2020-09-08T15:37:14.124 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2020-09-08T15:37:14.124 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2020-09-08T15:37:14.124 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2020-09-08T15:37:14.124 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_queue_high_threshhold": "0.900000", 2020-09-08T15:37:14.124 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_queue_low_threshhold": "0.300000", 2020-09-08T15:37:14.125 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_queue_max_bytes": "104857600", 2020-09-08T15:37:14.125 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2020-09-08T15:37:14.125 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2020-09-08T15:37:14.125 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2020-09-08T15:37:14.125 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_queue_max_ops": "50", 2020-09-08T15:37:14.126 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2020-09-08T15:37:14.126 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_seek_data_hole": "false", 2020-09-08T15:37:14.126 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_sloppy_crc": "false", 2020-09-08T15:37:14.126 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_sloppy_crc_block_size": "65536", 2020-09-08T15:37:14.126 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_splice": "false", 2020-09-08T15:37:14.126 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_split_multiple": "2", 2020-09-08T15:37:14.127 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_split_rand_factor": "20", 2020-09-08T15:37:14.127 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_update_to": "1000", 2020-09-08T15:37:14.127 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2020-09-08T15:37:14.127 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2020-09-08T15:37:14.127 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2020-09-08T15:37:14.128 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2020-09-08T15:37:14.128 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2020-09-08T15:37:14.128 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2020-09-08T15:37:14.128 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_wbthrottle_enable": "true", 2020-09-08T15:37:14.128 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2020-09-08T15:37:14.129 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2020-09-08T15:37:14.130 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2020-09-08T15:37:14.130 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2020-09-08T15:37:14.130 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2020-09-08T15:37:14.130 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2020-09-08T15:37:14.130 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_xfs_extsize": "false", 2020-09-08T15:37:14.131 INFO:teuthology.orchestra.run.smithi104.stdout: "filestore_zfs_snap": "false", 2020-09-08T15:37:14.131 INFO:teuthology.orchestra.run.smithi104.stdout: "fio_dir": "/tmp/fio", 2020-09-08T15:37:14.131 INFO:teuthology.orchestra.run.smithi104.stdout: "fsid": "759b27b6-f1e7-11ea-a080-001a4aab830c", 2020-09-08T15:37:14.131 INFO:teuthology.orchestra.run.smithi104.stdout: "fuse_allow_other": "true", 2020-09-08T15:37:14.131 INFO:teuthology.orchestra.run.smithi104.stdout: "fuse_atomic_o_trunc": "true", 2020-09-08T15:37:14.132 INFO:teuthology.orchestra.run.smithi104.stdout: "fuse_big_writes": "true", 2020-09-08T15:37:14.132 INFO:teuthology.orchestra.run.smithi104.stdout: "fuse_debug": "false", 2020-09-08T15:37:14.132 INFO:teuthology.orchestra.run.smithi104.stdout: "fuse_default_permissions": "false", 2020-09-08T15:37:14.132 INFO:teuthology.orchestra.run.smithi104.stdout: "fuse_disable_pagecache": "false", 2020-09-08T15:37:14.133 INFO:teuthology.orchestra.run.smithi104.stdout: "fuse_max_write": "0", 2020-09-08T15:37:14.133 INFO:teuthology.orchestra.run.smithi104.stdout: "fuse_multithreaded": "true", 2020-09-08T15:37:14.133 INFO:teuthology.orchestra.run.smithi104.stdout: "fuse_require_active_mds": "true", 2020-09-08T15:37:14.133 INFO:teuthology.orchestra.run.smithi104.stdout: "fuse_set_user_groups": "true", 2020-09-08T15:37:14.133 INFO:teuthology.orchestra.run.smithi104.stdout: "fuse_splice_move": "true", 2020-09-08T15:37:14.134 INFO:teuthology.orchestra.run.smithi104.stdout: "fuse_splice_read": "true", 2020-09-08T15:37:14.134 INFO:teuthology.orchestra.run.smithi104.stdout: "fuse_splice_write": "true", 2020-09-08T15:37:14.134 INFO:teuthology.orchestra.run.smithi104.stdout: "fuse_syncfs_on_mksnap": "true", 2020-09-08T15:37:14.134 INFO:teuthology.orchestra.run.smithi104.stdout: "fuse_use_invalidate_cb": "true", 2020-09-08T15:37:14.134 INFO:teuthology.orchestra.run.smithi104.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2020-09-08T15:37:14.135 INFO:teuthology.orchestra.run.smithi104.stdout: "gss_target_name": "ceph", 2020-09-08T15:37:14.135 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_file": "", 2020-09-08T15:37:14.135 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_inject_failure": "0", 2020-09-08T15:37:14.135 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_interval": "5", 2020-09-08T15:37:14.135 INFO:teuthology.orchestra.run.smithi104.stdout: "host": "", 2020-09-08T15:37:14.136 INFO:teuthology.orchestra.run.smithi104.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2020-09-08T15:37:14.136 INFO:teuthology.orchestra.run.smithi104.stdout: "immutable_object_cache_max_inflight_ops": "128", 2020-09-08T15:37:14.137 INFO:teuthology.orchestra.run.smithi104.stdout: "immutable_object_cache_max_size": "1073741824", 2020-09-08T15:37:14.137 INFO:teuthology.orchestra.run.smithi104.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2020-09-08T15:37:14.137 INFO:teuthology.orchestra.run.smithi104.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2020-09-08T15:37:14.137 INFO:teuthology.orchestra.run.smithi104.stdout: "immutable_object_cache_watermark": "0.100000", 2020-09-08T15:37:14.137 INFO:teuthology.orchestra.run.smithi104.stdout: "inject_early_sigterm": "false", 2020-09-08T15:37:14.138 INFO:teuthology.orchestra.run.smithi104.stdout: "journal_aio": "true", 2020-09-08T15:37:14.138 INFO:teuthology.orchestra.run.smithi104.stdout: "journal_align_min_size": "65536", 2020-09-08T15:37:14.138 INFO:teuthology.orchestra.run.smithi104.stdout: "journal_block_align": "true", 2020-09-08T15:37:14.138 INFO:teuthology.orchestra.run.smithi104.stdout: "journal_block_size": "4096", 2020-09-08T15:37:14.138 INFO:teuthology.orchestra.run.smithi104.stdout: "journal_dio": "true", 2020-09-08T15:37:14.139 INFO:teuthology.orchestra.run.smithi104.stdout: "journal_discard": "false", 2020-09-08T15:37:14.139 INFO:teuthology.orchestra.run.smithi104.stdout: "journal_force_aio": "false", 2020-09-08T15:37:14.139 INFO:teuthology.orchestra.run.smithi104.stdout: "journal_ignore_corruption": "false", 2020-09-08T15:37:14.139 INFO:teuthology.orchestra.run.smithi104.stdout: "journal_max_write_bytes": "10485760", 2020-09-08T15:37:14.139 INFO:teuthology.orchestra.run.smithi104.stdout: "journal_max_write_entries": "100", 2020-09-08T15:37:14.140 INFO:teuthology.orchestra.run.smithi104.stdout: "journal_replay_from": "0", 2020-09-08T15:37:14.140 INFO:teuthology.orchestra.run.smithi104.stdout: "journal_throttle_high_multiple": "0.000000", 2020-09-08T15:37:14.140 INFO:teuthology.orchestra.run.smithi104.stdout: "journal_throttle_high_threshhold": "0.900000", 2020-09-08T15:37:14.140 INFO:teuthology.orchestra.run.smithi104.stdout: "journal_throttle_low_threshhold": "0.600000", 2020-09-08T15:37:14.140 INFO:teuthology.orchestra.run.smithi104.stdout: "journal_throttle_max_multiple": "0.000000", 2020-09-08T15:37:14.140 INFO:teuthology.orchestra.run.smithi104.stdout: "journal_write_header_frequency": "0", 2020-09-08T15:37:14.141 INFO:teuthology.orchestra.run.smithi104.stdout: "journal_zero_on_create": "false", 2020-09-08T15:37:14.141 INFO:teuthology.orchestra.run.smithi104.stdout: "journaler_prefetch_periods": "10", 2020-09-08T15:37:14.141 INFO:teuthology.orchestra.run.smithi104.stdout: "journaler_prezero_periods": "5", 2020-09-08T15:37:14.141 INFO:teuthology.orchestra.run.smithi104.stdout: "journaler_write_head_interval": "15", 2020-09-08T15:37:14.141 INFO:teuthology.orchestra.run.smithi104.stdout: "key": "", 2020-09-08T15:37:14.142 INFO:teuthology.orchestra.run.smithi104.stdout: "keyfile": "", 2020-09-08T15:37:14.142 INFO:teuthology.orchestra.run.smithi104.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2020-09-08T15:37:14.142 INFO:teuthology.orchestra.run.smithi104.stdout: "kstore_backend": "rocksdb", 2020-09-08T15:37:14.143 INFO:teuthology.orchestra.run.smithi104.stdout: "kstore_default_stripe_size": "65536", 2020-09-08T15:37:14.143 INFO:teuthology.orchestra.run.smithi104.stdout: "kstore_fsck_on_mount": "false", 2020-09-08T15:37:14.143 INFO:teuthology.orchestra.run.smithi104.stdout: "kstore_fsck_on_mount_deep": "true", 2020-09-08T15:37:14.144 INFO:teuthology.orchestra.run.smithi104.stdout: "kstore_max_bytes": "67108864", 2020-09-08T15:37:14.144 INFO:teuthology.orchestra.run.smithi104.stdout: "kstore_max_ops": "512", 2020-09-08T15:37:14.144 INFO:teuthology.orchestra.run.smithi104.stdout: "kstore_nid_prealloc": "1024", 2020-09-08T15:37:14.144 INFO:teuthology.orchestra.run.smithi104.stdout: "kstore_onode_map_size": "1024", 2020-09-08T15:37:14.144 INFO:teuthology.orchestra.run.smithi104.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2020-09-08T15:37:14.145 INFO:teuthology.orchestra.run.smithi104.stdout: "kstore_sync_submit_transaction": "false", 2020-09-08T15:37:14.145 INFO:teuthology.orchestra.run.smithi104.stdout: "kstore_sync_transaction": "false", 2020-09-08T15:37:14.145 INFO:teuthology.orchestra.run.smithi104.stdout: "leveldb_block_size": "65536", 2020-09-08T15:37:14.145 INFO:teuthology.orchestra.run.smithi104.stdout: "leveldb_bloom_size": "0", 2020-09-08T15:37:14.145 INFO:teuthology.orchestra.run.smithi104.stdout: "leveldb_cache_size": "536870912", 2020-09-08T15:37:14.146 INFO:teuthology.orchestra.run.smithi104.stdout: "leveldb_compact_on_mount": "false", 2020-09-08T15:37:14.146 INFO:teuthology.orchestra.run.smithi104.stdout: "leveldb_compression": "false", 2020-09-08T15:37:14.146 INFO:teuthology.orchestra.run.smithi104.stdout: "leveldb_log": "", 2020-09-08T15:37:14.146 INFO:teuthology.orchestra.run.smithi104.stdout: "leveldb_log_to_ceph_log": "true", 2020-09-08T15:37:14.146 INFO:teuthology.orchestra.run.smithi104.stdout: "leveldb_max_open_files": "0", 2020-09-08T15:37:14.147 INFO:teuthology.orchestra.run.smithi104.stdout: "leveldb_paranoid": "false", 2020-09-08T15:37:14.147 INFO:teuthology.orchestra.run.smithi104.stdout: "leveldb_write_buffer_size": "33554432", 2020-09-08T15:37:14.147 INFO:teuthology.orchestra.run.smithi104.stdout: "librados_thread_count": "2", 2020-09-08T15:37:14.147 INFO:teuthology.orchestra.run.smithi104.stdout: "lockdep": "false", 2020-09-08T15:37:14.147 INFO:teuthology.orchestra.run.smithi104.stdout: "lockdep_force_backtrace": "false", 2020-09-08T15:37:14.148 INFO:teuthology.orchestra.run.smithi104.stdout: "log_coarse_timestamps": "true", 2020-09-08T15:37:14.148 INFO:teuthology.orchestra.run.smithi104.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2020-09-08T15:37:14.148 INFO:teuthology.orchestra.run.smithi104.stdout: "log_flush_on_exit": "false", 2020-09-08T15:37:14.148 INFO:teuthology.orchestra.run.smithi104.stdout: "log_graylog_host": "127.0.0.1", 2020-09-08T15:37:14.148 INFO:teuthology.orchestra.run.smithi104.stdout: "log_graylog_port": "12201", 2020-09-08T15:37:14.148 INFO:teuthology.orchestra.run.smithi104.stdout: "log_max_new": "1000", 2020-09-08T15:37:14.149 INFO:teuthology.orchestra.run.smithi104.stdout: "log_max_recent": "10000", 2020-09-08T15:37:14.149 INFO:teuthology.orchestra.run.smithi104.stdout: "log_stderr_prefix": "debug ", 2020-09-08T15:37:14.149 INFO:teuthology.orchestra.run.smithi104.stdout: "log_stop_at_utilization": "0.970000", 2020-09-08T15:37:14.149 INFO:teuthology.orchestra.run.smithi104.stdout: "log_to_file": "true", 2020-09-08T15:37:14.150 INFO:teuthology.orchestra.run.smithi104.stdout: "log_to_graylog": "false", 2020-09-08T15:37:14.150 INFO:teuthology.orchestra.run.smithi104.stdout: "log_to_stderr": "false", 2020-09-08T15:37:14.150 INFO:teuthology.orchestra.run.smithi104.stdout: "log_to_syslog": "false", 2020-09-08T15:37:14.151 INFO:teuthology.orchestra.run.smithi104.stdout: "max_rotating_auth_attempts": "10", 2020-09-08T15:37:14.151 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_action_on_write_error": "1", 2020-09-08T15:37:14.151 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_asio_thread_count": "2", 2020-09-08T15:37:14.151 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_export_pin": "true", 2020-09-08T15:37:14.152 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_fragment_dirs": "true", 2020-09-08T15:37:14.152 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2020-09-08T15:37:14.152 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_fragment_interval": "5", 2020-09-08T15:37:14.152 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_fragment_size_max": "100000", 2020-09-08T15:37:14.152 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_idle_threshold": "0.000000", 2020-09-08T15:37:14.153 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_interval": "10", 2020-09-08T15:37:14.153 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_max": "-1", 2020-09-08T15:37:14.153 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_max_until": "-1", 2020-09-08T15:37:14.153 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_merge_size": "50", 2020-09-08T15:37:14.153 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_midchunk": "0.300000", 2020-09-08T15:37:14.154 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_min_rebalance": "0.100000", 2020-09-08T15:37:14.154 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_min_start": "0.200000", 2020-09-08T15:37:14.154 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_minchunk": "0.001000", 2020-09-08T15:37:14.154 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_mode": "0", 2020-09-08T15:37:14.155 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_need_max": "1.200000", 2020-09-08T15:37:14.155 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_need_min": "0.800000", 2020-09-08T15:37:14.156 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_replicate_threshold": "8000.000000", 2020-09-08T15:37:14.156 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_sample_interval": "3.000000", 2020-09-08T15:37:14.156 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_split_bits": "3", 2020-09-08T15:37:14.157 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_split_rd": "25000.000000", 2020-09-08T15:37:14.157 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_split_size": "10000", 2020-09-08T15:37:14.157 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_split_wr": "10000.000000", 2020-09-08T15:37:14.157 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_target_decay": "10.000000", 2020-09-08T15:37:14.157 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2020-09-08T15:37:14.158 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_beacon_grace": "15.000000", 2020-09-08T15:37:14.158 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_beacon_interval": "4.000000", 2020-09-08T15:37:14.158 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache_memory_limit": "4294967296", 2020-09-08T15:37:14.158 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache_mid": "0.700000", 2020-09-08T15:37:14.158 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache_release_free_interval": "10", 2020-09-08T15:37:14.159 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache_reservation": "0.050000", 2020-09-08T15:37:14.159 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache_trim_decay_rate": "1.000000", 2020-09-08T15:37:14.159 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache_trim_interval": "1", 2020-09-08T15:37:14.159 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache_trim_threshold": "65536", 2020-09-08T15:37:14.159 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2020-09-08T15:37:14.160 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_client_delegate_inos_pct": "50", 2020-09-08T15:37:14.160 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_client_prealloc_inos": "1000", 2020-09-08T15:37:14.160 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2020-09-08T15:37:14.160 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_damage_table_max_entries": "10000", 2020-09-08T15:37:14.160 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2020-09-08T15:37:14.160 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_debug_auth_pins": "false", 2020-09-08T15:37:14.161 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_debug_frag": "false", 2020-09-08T15:37:14.161 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_debug_scatterstat": "false", 2020-09-08T15:37:14.161 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_debug_subtrees": "false", 2020-09-08T15:37:14.161 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_decay_halflife": "5.000000", 2020-09-08T15:37:14.161 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_default_dir_hash": "2", 2020-09-08T15:37:14.162 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_defer_session_stale": "true", 2020-09-08T15:37:14.162 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_deny_all_reconnect": "false", 2020-09-08T15:37:14.162 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_dir_keys_per_op": "16384", 2020-09-08T15:37:14.162 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_dir_max_commit_size": "10", 2020-09-08T15:37:14.162 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_dirstat_min_interval": "1.000000", 2020-09-08T15:37:14.163 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_dump_cache_after_rejoin": "false", 2020-09-08T15:37:14.163 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_dump_cache_on_map": "false", 2020-09-08T15:37:14.163 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_dump_cache_threshold_file": "0", 2020-09-08T15:37:14.163 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2020-09-08T15:37:14.163 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_early_reply": "true", 2020-09-08T15:37:14.163 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_enable_op_tracker": "true", 2020-09-08T15:37:14.164 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_enforce_unique_name": "true", 2020-09-08T15:37:14.164 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_export_ephemeral_distributed": "true", 2020-09-08T15:37:14.164 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_export_ephemeral_random": "true", 2020-09-08T15:37:14.164 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_export_ephemeral_random_max": "0.010000", 2020-09-08T15:37:14.164 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_forward_all_requests_to_auth": "false", 2020-09-08T15:37:14.165 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_freeze_tree_timeout": "30.000000", 2020-09-08T15:37:14.165 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2020-09-08T15:37:14.165 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_health_cache_threshold": "1.500000", 2020-09-08T15:37:14.165 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_health_summarize_threshold": "10", 2020-09-08T15:37:14.165 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_heartbeat_grace": "15.000000", 2020-09-08T15:37:14.166 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_inject_migrator_session_race": "false", 2020-09-08T15:37:14.166 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2020-09-08T15:37:14.166 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_join_fs": "", 2020-09-08T15:37:14.166 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_journal_format": "1", 2020-09-08T15:37:14.166 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_kill_create_at": "0", 2020-09-08T15:37:14.166 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_kill_export_at": "0", 2020-09-08T15:37:14.167 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_kill_import_at": "0", 2020-09-08T15:37:14.167 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_kill_journal_at": "0", 2020-09-08T15:37:14.167 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_kill_journal_expire_at": "0", 2020-09-08T15:37:14.167 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_kill_journal_replay_at": "0", 2020-09-08T15:37:14.167 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_kill_link_at": "0", 2020-09-08T15:37:14.168 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_kill_mdstable_at": "0", 2020-09-08T15:37:14.168 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_kill_openc_at": "0", 2020-09-08T15:37:14.168 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_kill_rename_at": "0", 2020-09-08T15:37:14.168 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log_events_per_segment": "1024", 2020-09-08T15:37:14.168 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log_max_events": "-1", 2020-09-08T15:37:14.168 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log_max_segments": "128", 2020-09-08T15:37:14.169 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log_pause": "false", 2020-09-08T15:37:14.169 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log_segment_size": "0", 2020-09-08T15:37:14.169 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log_skip_corrupt_events": "false", 2020-09-08T15:37:14.169 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log_warn_factor": "2.000000", 2020-09-08T15:37:14.169 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_max_caps_per_client": "1048576", 2020-09-08T15:37:14.170 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_max_completed_flushes": "100000", 2020-09-08T15:37:14.170 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_max_completed_requests": "100000", 2020-09-08T15:37:14.170 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_max_export_size": "20971520", 2020-09-08T15:37:14.170 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_max_file_recover": "32", 2020-09-08T15:37:14.170 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_max_purge_files": "64", 2020-09-08T15:37:14.171 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_max_purge_ops": "8192", 2020-09-08T15:37:14.171 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2020-09-08T15:37:14.171 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_max_retries_on_remount_failure": "5", 2020-09-08T15:37:14.171 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_max_scrub_ops_in_progress": "5", 2020-09-08T15:37:14.171 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_max_snaps_per_dir": "100", 2020-09-08T15:37:14.172 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_max_xattr_pairs_size": "65536", 2020-09-08T15:37:14.172 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_metrics_update_interval": "2", 2020-09-08T15:37:14.172 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_min_caps_per_client": "100", 2020-09-08T15:37:14.172 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_min_caps_working_set": "10000", 2020-09-08T15:37:14.172 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mon_shutdown_timeout": "5.000000", 2020-09-08T15:37:14.172 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_numa_node": "-1", 2020-09-08T15:37:14.173 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_oft_prefetch_dirfrags": "true", 2020-09-08T15:37:14.173 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_op_complaint_time": "30.000000", 2020-09-08T15:37:14.173 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_op_history_duration": "600", 2020-09-08T15:37:14.173 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_op_history_size": "20", 2020-09-08T15:37:14.173 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_op_log_threshold": "5", 2020-09-08T15:37:14.174 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_ping_grace": "15", 2020-09-08T15:37:14.174 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_ping_interval": "5", 2020-09-08T15:37:14.174 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2020-09-08T15:37:14.174 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_recall_global_max_decay_threshold": "65536", 2020-09-08T15:37:14.174 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_recall_max_caps": "5000", 2020-09-08T15:37:14.175 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_recall_max_decay_rate": "2.500000", 2020-09-08T15:37:14.175 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_recall_max_decay_threshold": "16384", 2020-09-08T15:37:14.175 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_recall_warning_decay_rate": "60.000000", 2020-09-08T15:37:14.175 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_recall_warning_threshold": "32768", 2020-09-08T15:37:14.175 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_reconnect_timeout": "45.000000", 2020-09-08T15:37:14.175 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_replay_interval": "1.000000", 2020-09-08T15:37:14.176 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_replay_unsafe_with_closed_session": "false", 2020-09-08T15:37:14.176 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_request_load_average_decay_rate": "60.000000", 2020-09-08T15:37:14.176 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_root_ino_gid": "0", 2020-09-08T15:37:14.176 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_root_ino_uid": "0", 2020-09-08T15:37:14.176 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_scatter_nudge_interval": "5.000000", 2020-09-08T15:37:14.177 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_session_blocklist_on_evict": "true", 2020-09-08T15:37:14.177 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_session_blocklist_on_timeout": "true", 2020-09-08T15:37:14.177 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2020-09-08T15:37:14.177 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_session_cache_liveness_magnitude": "10", 2020-09-08T15:37:14.177 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessionmap_keys_per_op": "1024", 2020-09-08T15:37:14.178 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_shutdown_check": "0", 2020-09-08T15:37:14.178 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_skip_ino": "0", 2020-09-08T15:37:14.178 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_snap_max_uid": "4294967294", 2020-09-08T15:37:14.178 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_snap_min_uid": "0", 2020-09-08T15:37:14.178 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_snap_rstat": "false", 2020-09-08T15:37:14.178 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_task_status_update_interval": "2.000000", 2020-09-08T15:37:14.179 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_thrash_exports": "0", 2020-09-08T15:37:14.179 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_thrash_fragments": "0", 2020-09-08T15:37:14.179 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_tick_interval": "5.000000", 2020-09-08T15:37:14.179 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_verify_backtrace": "1", 2020-09-08T15:37:14.179 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_verify_scatter": "false", 2020-09-08T15:37:14.180 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_wipe_ino_prealloc": "false", 2020-09-08T15:37:14.180 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_wipe_sessions": "false", 2020-09-08T15:37:14.180 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool_debug": "false", 2020-09-08T15:37:14.180 INFO:teuthology.orchestra.run.smithi104.stdout: "memstore_debug_omit_block_device_write": "false", 2020-09-08T15:37:14.180 INFO:teuthology.orchestra.run.smithi104.stdout: "memstore_device_bytes": "1073741824", 2020-09-08T15:37:14.181 INFO:teuthology.orchestra.run.smithi104.stdout: "memstore_page_set": "false", 2020-09-08T15:37:14.181 INFO:teuthology.orchestra.run.smithi104.stdout: "memstore_page_size": "65536", 2020-09-08T15:37:14.181 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_client_bytes": "134217728", 2020-09-08T15:37:14.181 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_client_messages": "512", 2020-09-08T15:37:14.181 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2020-09-08T15:37:14.181 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_connect_retry_interval": "1.000000", 2020-09-08T15:37:14.182 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2020-09-08T15:37:14.182 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2020-09-08T15:37:14.182 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_disabled_modules": "", 2020-09-08T15:37:14.182 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_initial_modules": "restful iostat", 2020-09-08T15:37:14.182 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_mds_bytes": "134217728", 2020-09-08T15:37:14.183 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_mds_messages": "128", 2020-09-08T15:37:14.183 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2020-09-08T15:37:14.183 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_mon_bytes": "134217728", 2020-09-08T15:37:14.183 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_mon_messages": "128", 2020-09-08T15:37:14.183 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_osd_bytes": "536870912", 2020-09-08T15:37:14.184 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_osd_messages": "8192", 2020-09-08T15:37:14.184 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_service_beacon_grace": "60.000000", 2020-09-08T15:37:14.184 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_stats_period": "5", 2020-09-08T15:37:14.184 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_stats_threshold": "5", 2020-09-08T15:37:14.184 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_tick_period": "2", 2020-09-08T15:37:14.185 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_accept_timeout_factor": "2.000000", 2020-09-08T15:37:14.185 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_allow_pool_delete": "true", 2020-09-08T15:37:14.185 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_allow_pool_size_one": "false", 2020-09-08T15:37:14.185 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2020-09-08T15:37:14.185 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2020-09-08T15:37:14.185 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_client_bytes": "104857600", 2020-09-08T15:37:14.186 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_client_directed_command_retry": "5", 2020-09-08T15:37:14.186 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_client_hunt_interval": "3.000000", 2020-09-08T15:37:14.186 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2020-09-08T15:37:14.186 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2020-09-08T15:37:14.186 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2020-09-08T15:37:14.187 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_client_hunt_parallel": "3", 2020-09-08T15:37:14.187 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_client_log_interval": "1.000000", 2020-09-08T15:37:14.187 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_client_max_log_entries_per_message": "1000", 2020-09-08T15:37:14.187 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_client_ping_interval": "10.000000", 2020-09-08T15:37:14.187 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_client_ping_timeout": "30.000000", 2020-09-08T15:37:14.188 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_clock_drift_allowed": "1.000000", 2020-09-08T15:37:14.188 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2020-09-08T15:37:14.188 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2020-09-08T15:37:14.188 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_cluster_log_file_level": "debug", 2020-09-08T15:37:14.188 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_cluster_log_to_file": "false", 2020-09-08T15:37:14.188 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_cluster_log_to_graylog": "false", 2020-09-08T15:37:14.189 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2020-09-08T15:37:14.189 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_cluster_log_to_graylog_port": "12201", 2020-09-08T15:37:14.189 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_cluster_log_to_stderr": "true", 2020-09-08T15:37:14.189 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_cluster_log_to_syslog": "default=false", 2020-09-08T15:37:14.189 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2020-09-08T15:37:14.190 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_cluster_log_to_syslog_level": "info", 2020-09-08T15:37:14.190 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_compact_on_bootstrap": "false", 2020-09-08T15:37:14.190 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_compact_on_start": "false", 2020-09-08T15:37:14.190 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_compact_on_trim": "true", 2020-09-08T15:37:14.190 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_config_key_max_entry_size": "65536", 2020-09-08T15:37:14.191 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_cpu_threads": "4", 2020-09-08T15:37:14.191 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_crush_min_required_version": "hammer", 2020-09-08T15:37:14.191 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_daemon_bytes": "419430400", 2020-09-08T15:37:14.191 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2020-09-08T15:37:14.191 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_data_avail_crit": "5", 2020-09-08T15:37:14.191 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_data_avail_warn": "5", 2020-09-08T15:37:14.192 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_data_size_warn": "16106127360", 2020-09-08T15:37:14.192 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_debug_block_osdmap_trim": "false", 2020-09-08T15:37:14.192 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_debug_deprecated_as_obsolete": "false", 2020-09-08T15:37:14.192 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_debug_dump_json": "false", 2020-09-08T15:37:14.192 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2020-09-08T15:37:14.193 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_debug_dump_transactions": "false", 2020-09-08T15:37:14.193 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_debug_extra_checks": "false", 2020-09-08T15:37:14.193 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_debug_no_initial_persistent_features": "false", 2020-09-08T15:37:14.193 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2020-09-08T15:37:14.193 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_debug_no_require_octopus": "false", 2020-09-08T15:37:14.194 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_debug_no_require_pacific": "false", 2020-09-08T15:37:14.194 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2020-09-08T15:37:14.194 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_delta_reset_interval": "10.000000", 2020-09-08T15:37:14.194 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_dns_srv_name": "ceph-mon", 2020-09-08T15:37:14.194 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_election_timeout": "5.000000", 2020-09-08T15:37:14.194 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_enable_op_tracker": "true", 2020-09-08T15:37:14.195 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_fake_pool_delete": "false", 2020-09-08T15:37:14.195 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_force_quorum_join": "false", 2020-09-08T15:37:14.195 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_globalid_prealloc": "10000", 2020-09-08T15:37:14.195 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_health_log_update_period": "5", 2020-09-08T15:37:14.195 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_health_max_detail": "50", 2020-09-08T15:37:14.196 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_health_to_clog": "true", 2020-09-08T15:37:14.196 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_health_to_clog_interval": "3600", 2020-09-08T15:37:14.196 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2020-09-08T15:37:14.196 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_host": "172.21.15.104:6789/0", 2020-09-08T15:37:14.196 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_initial_members": "", 2020-09-08T15:37:14.197 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2020-09-08T15:37:14.197 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2020-09-08T15:37:14.197 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_inject_transaction_delay_max": "10.000000", 2020-09-08T15:37:14.197 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2020-09-08T15:37:14.197 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_keyvaluedb": "rocksdb", 2020-09-08T15:37:14.197 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_lease": "5.000000", 2020-09-08T15:37:14.198 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2020-09-08T15:37:14.198 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_lease_renew_interval_factor": "0.600000", 2020-09-08T15:37:14.198 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_log_max_summary": "50", 2020-09-08T15:37:14.198 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_max_log_entries_per_event": "4096", 2020-09-08T15:37:14.198 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_max_log_epochs": "500", 2020-09-08T15:37:14.199 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_max_mdsmap_epochs": "500", 2020-09-08T15:37:14.199 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_max_mgrmap_epochs": "500", 2020-09-08T15:37:14.199 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_max_osd": "10000", 2020-09-08T15:37:14.199 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_max_pg_per_osd": "10000", 2020-09-08T15:37:14.199 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_max_pool_pg_num": "65536", 2020-09-08T15:37:14.200 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_max_snap_prune_per_epoch": "100", 2020-09-08T15:37:14.200 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_mds_blocklist_interval": "86400.000000", 2020-09-08T15:37:14.200 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_mds_force_trim_to": "0", 2020-09-08T15:37:14.200 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_mds_skip_sanity": "false", 2020-09-08T15:37:14.200 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_memory_autotune": "true", 2020-09-08T15:37:14.200 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_memory_target": "2147483648", 2020-09-08T15:37:14.201 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_mgr_beacon_grace": "30", 2020-09-08T15:37:14.201 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2020-09-08T15:37:14.201 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_mgr_digest_period": "5", 2020-09-08T15:37:14.201 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_mgr_inactive_grace": "60", 2020-09-08T15:37:14.201 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_mgr_mkfs_grace": "240", 2020-09-08T15:37:14.202 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2020-09-08T15:37:14.202 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_min_osdmap_epochs": "50", 2020-09-08T15:37:14.202 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_op_complaint_time": "30", 2020-09-08T15:37:14.202 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_op_history_duration": "600", 2020-09-08T15:37:14.202 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_op_history_size": "20", 2020-09-08T15:37:14.203 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_op_history_slow_op_size": "20", 2020-09-08T15:37:14.203 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_op_history_slow_op_threshold": "10", 2020-09-08T15:37:14.203 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_op_log_threshold": "5", 2020-09-08T15:37:14.203 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_adjust_down_out_interval": "true", 2020-09-08T15:37:14.203 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2020-09-08T15:37:14.203 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2020-09-08T15:37:14.204 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_auto_mark_in": "false", 2020-09-08T15:37:14.204 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_auto_mark_new_in": "true", 2020-09-08T15:37:14.204 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2020-09-08T15:37:14.204 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2020-09-08T15:37:14.204 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_cache_size": "500", 2020-09-08T15:37:14.205 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_cache_size_min": "134217728", 2020-09-08T15:37:14.205 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_crush_smoke_test": "true", 2020-09-08T15:37:14.205 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_destroyed_out_interval": "600", 2020-09-08T15:37:14.205 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_down_out_interval": "600", 2020-09-08T15:37:14.205 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_down_out_subtree_limit": "rack", 2020-09-08T15:37:14.206 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2020-09-08T15:37:14.206 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_force_trim_to": "0", 2020-09-08T15:37:14.206 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_full_ratio": "0.950000", 2020-09-08T15:37:14.206 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2020-09-08T15:37:14.206 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_laggy_halflife": "3600", 2020-09-08T15:37:14.206 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_laggy_max_interval": "300", 2020-09-08T15:37:14.207 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_laggy_weight": "0.300000", 2020-09-08T15:37:14.207 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2020-09-08T15:37:14.207 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_max_creating_pgs": "1024", 2020-09-08T15:37:14.207 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_max_initial_pgs": "1024", 2020-09-08T15:37:14.207 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_min_down_reporters": "2", 2020-09-08T15:37:14.208 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_min_in_ratio": "0.750000", 2020-09-08T15:37:14.208 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_min_up_ratio": "0.300000", 2020-09-08T15:37:14.208 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_nearfull_ratio": "0.850000", 2020-09-08T15:37:14.208 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_prime_pg_temp": "true", 2020-09-08T15:37:14.208 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2020-09-08T15:37:14.209 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2020-09-08T15:37:14.209 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_report_timeout": "900", 2020-09-08T15:37:14.209 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_reporter_subtree_level": "osd", 2020-09-08T15:37:14.209 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2020-09-08T15:37:14.209 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_warn_num_repaired": "10", 2020-09-08T15:37:14.210 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osd_warn_op_age": "32.000000", 2020-09-08T15:37:14.210 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osdmap_full_prune_enabled": "true", 2020-09-08T15:37:14.210 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osdmap_full_prune_interval": "2", 2020-09-08T15:37:14.210 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osdmap_full_prune_min": "15", 2020-09-08T15:37:14.210 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_osdmap_full_prune_txsize": "2", 2020-09-08T15:37:14.210 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2020-09-08T15:37:14.211 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_pg_stuck_threshold": "60", 2020-09-08T15:37:14.211 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2020-09-08T15:37:14.211 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_pg_warn_min_objects": "10000", 2020-09-08T15:37:14.211 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_pg_warn_min_per_osd": "0", 2020-09-08T15:37:14.211 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_pg_warn_min_pool_objects": "1000", 2020-09-08T15:37:14.212 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_pool_quota_crit_threshold": "0", 2020-09-08T15:37:14.212 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_pool_quota_warn_threshold": "0", 2020-09-08T15:37:14.212 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_probe_timeout": "2.000000", 2020-09-08T15:37:14.212 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_reweight_max_change": "0.050000", 2020-09-08T15:37:14.212 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_reweight_max_osds": "4", 2020-09-08T15:37:14.212 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_reweight_min_bytes_per_osd": "10", 2020-09-08T15:37:14.213 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_reweight_min_pgs_per_osd": "4", 2020-09-08T15:37:14.213 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2020-09-08T15:37:14.213 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2020-09-08T15:37:14.213 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2020-09-08T15:37:14.213 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_scrub_interval": "86400", 2020-09-08T15:37:14.214 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_scrub_max_keys": "100", 2020-09-08T15:37:14.214 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_scrub_timeout": "300", 2020-09-08T15:37:14.214 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_session_timeout": "300", 2020-09-08T15:37:14.214 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_smart_report_timeout": "5", 2020-09-08T15:37:14.214 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_stat_smooth_intervals": "6", 2020-09-08T15:37:14.215 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_subscribe_interval": "86400.000000", 2020-09-08T15:37:14.215 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_sync_debug": "false", 2020-09-08T15:37:14.215 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_sync_max_payload_keys": "2000", 2020-09-08T15:37:14.215 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_sync_max_payload_size": "1048576", 2020-09-08T15:37:14.215 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_sync_provider_kill_at": "0", 2020-09-08T15:37:14.215 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_sync_requester_kill_at": "0", 2020-09-08T15:37:14.216 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_sync_timeout": "60.000000", 2020-09-08T15:37:14.216 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_target_pg_per_osd": "100", 2020-09-08T15:37:14.216 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_tick_interval": "5", 2020-09-08T15:37:14.216 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_timecheck_interval": "300.000000", 2020-09-08T15:37:14.216 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_timecheck_skew_interval": "30.000000", 2020-09-08T15:37:14.217 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2020-09-08T15:37:14.217 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2020-09-08T15:37:14.217 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2020-09-08T15:37:14.217 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_warn_on_misplaced": "false", 2020-09-08T15:37:14.217 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_warn_on_msgr2_not_enabled": "false", 2020-09-08T15:37:14.218 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2020-09-08T15:37:14.218 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_warn_on_pool_no_app": "true", 2020-09-08T15:37:14.218 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_warn_on_pool_no_redundancy": "true", 2020-09-08T15:37:14.218 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2020-09-08T15:37:14.218 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2020-09-08T15:37:14.218 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2020-09-08T15:37:14.219 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_warn_on_too_few_osds": "false", 2020-09-08T15:37:14.219 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2020-09-08T15:37:14.219 INFO:teuthology.orchestra.run.smithi104.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2020-09-08T15:37:14.219 INFO:teuthology.orchestra.run.smithi104.stdout: "monmap": "", 2020-09-08T15:37:14.219 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_async_max_op_threads": "5", 2020-09-08T15:37:14.220 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_async_op_threads": "3", 2020-09-08T15:37:14.220 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_async_rdma_buffer_size": "131072", 2020-09-08T15:37:14.220 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_async_rdma_cm": "false", 2020-09-08T15:37:14.220 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_async_rdma_device_name": "", 2020-09-08T15:37:14.220 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_async_rdma_dscp": "96", 2020-09-08T15:37:14.221 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_async_rdma_enable_hugepage": "false", 2020-09-08T15:37:14.221 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_async_rdma_gid_idx": "0", 2020-09-08T15:37:14.221 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_async_rdma_local_gid": "", 2020-09-08T15:37:14.221 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_async_rdma_polling_us": "1000", 2020-09-08T15:37:14.221 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_async_rdma_port_num": "1", 2020-09-08T15:37:14.222 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_async_rdma_receive_buffers": "32768", 2020-09-08T15:37:14.222 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_async_rdma_receive_queue_len": "4096", 2020-09-08T15:37:14.222 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_async_rdma_roce_ver": "1", 2020-09-08T15:37:14.222 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_async_rdma_send_buffers": "1024", 2020-09-08T15:37:14.222 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_async_rdma_sl": "3", 2020-09-08T15:37:14.222 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_async_rdma_support_srq": "true", 2020-09-08T15:37:14.223 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_async_rdma_type": "ib", 2020-09-08T15:37:14.223 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_bind_before_connect": "false", 2020-09-08T15:37:14.223 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_bind_ipv4": "true", 2020-09-08T15:37:14.223 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_bind_ipv6": "false", 2020-09-08T15:37:14.223 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_bind_msgr1": "true", 2020-09-08T15:37:14.224 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_bind_msgr2": "true", 2020-09-08T15:37:14.224 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_bind_port_max": "7300", 2020-09-08T15:37:14.224 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_bind_port_min": "6800", 2020-09-08T15:37:14.224 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_bind_prefer_ipv4": "false", 2020-09-08T15:37:14.224 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_bind_retry_count": "3", 2020-09-08T15:37:14.225 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_bind_retry_delay": "5", 2020-09-08T15:37:14.225 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_blackhole_client": "false", 2020-09-08T15:37:14.225 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_blackhole_mds": "false", 2020-09-08T15:37:14.225 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_blackhole_mgr": "false", 2020-09-08T15:37:14.225 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_blackhole_mon": "false", 2020-09-08T15:37:14.225 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_blackhole_osd": "false", 2020-09-08T15:37:14.226 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_client_mode": "crc secure", 2020-09-08T15:37:14.226 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_cluster_mode": "crc secure", 2020-09-08T15:37:14.226 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_cluster_type": "", 2020-09-08T15:37:14.226 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_connection_idle_timeout": "900", 2020-09-08T15:37:14.226 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_connection_ready_timeout": "10", 2020-09-08T15:37:14.227 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_crc_data": "true", 2020-09-08T15:37:14.227 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_crc_header": "true", 2020-09-08T15:37:14.227 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_die_on_bad_msg": "false", 2020-09-08T15:37:14.227 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_die_on_bug": "true", 2020-09-08T15:37:14.227 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_die_on_old_message": "true", 2020-09-08T15:37:14.228 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_die_on_skipped_message": "false", 2020-09-08T15:37:14.228 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_die_on_unhandled_msg": "false", 2020-09-08T15:37:14.228 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_dispatch_throttle_bytes": "104857600", 2020-09-08T15:37:14.228 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_dpdk_coremask": "0xF", 2020-09-08T15:37:14.228 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_dpdk_debug_allow_loopback": "false", 2020-09-08T15:37:14.228 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2020-09-08T15:37:14.229 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_dpdk_host_ipv4_addr": "", 2020-09-08T15:37:14.229 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_dpdk_hugepages": "", 2020-09-08T15:37:14.229 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_dpdk_hw_flow_control": "true", 2020-09-08T15:37:14.229 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2020-09-08T15:37:14.229 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_dpdk_lro": "true", 2020-09-08T15:37:14.230 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_dpdk_memory_channel": "4", 2020-09-08T15:37:14.230 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2020-09-08T15:37:14.230 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_dpdk_pmd": "", 2020-09-08T15:37:14.230 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_dpdk_port_id": "0", 2020-09-08T15:37:14.230 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2020-09-08T15:37:14.231 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_dump_corrupt_message_level": "1", 2020-09-08T15:37:14.231 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_dump_on_send": "false", 2020-09-08T15:37:14.231 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_initial_backoff": "0.200000", 2020-09-08T15:37:14.231 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_inject_delay_max": "1.000000", 2020-09-08T15:37:14.231 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_inject_delay_msg_type": "", 2020-09-08T15:37:14.231 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_inject_delay_probability": "0.000000", 2020-09-08T15:37:14.232 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_inject_delay_type": "", 2020-09-08T15:37:14.232 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_inject_internal_delays": "0.000000", 2020-09-08T15:37:14.232 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_inject_socket_failures": "2500", 2020-09-08T15:37:14.232 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_learn_addr_from_peer": "true", 2020-09-08T15:37:14.232 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_max_accept_failures": "4", 2020-09-08T15:37:14.233 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_max_backoff": "15.000000", 2020-09-08T15:37:14.233 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_mon_client_mode": "secure crc", 2020-09-08T15:37:14.233 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_mon_cluster_mode": "secure crc", 2020-09-08T15:37:14.233 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_mon_service_mode": "secure crc", 2020-09-08T15:37:14.233 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2020-09-08T15:37:14.234 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_pq_min_cost": "65536", 2020-09-08T15:37:14.234 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_public_type": "", 2020-09-08T15:37:14.234 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_service_mode": "crc secure", 2020-09-08T15:37:14.234 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_tcp_listen_backlog": "512", 2020-09-08T15:37:14.234 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_tcp_nodelay": "true", 2020-09-08T15:37:14.234 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_tcp_prefetch_max_size": "4096", 2020-09-08T15:37:14.235 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_tcp_rcvbuf": "0", 2020-09-08T15:37:14.235 INFO:teuthology.orchestra.run.smithi104.stdout: "ms_type": "async+posix", 2020-09-08T15:37:14.235 INFO:teuthology.orchestra.run.smithi104.stdout: "no_config_file": "false", 2020-09-08T15:37:14.235 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter_completion_locks_per_session": "32", 2020-09-08T15:37:14.235 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter_debug_inject_relock_delay": "false", 2020-09-08T15:37:14.236 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter_inflight_op_bytes": "104857600", 2020-09-08T15:37:14.236 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter_inflight_ops": "1024", 2020-09-08T15:37:14.236 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter_inject_no_watch_ping": "false", 2020-09-08T15:37:14.236 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter_retry_writes_after_first_reply": "false", 2020-09-08T15:37:14.236 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter_tick_interval": "5.000000", 2020-09-08T15:37:14.237 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter_timeout": "10.000000", 2020-09-08T15:37:14.237 INFO:teuthology.orchestra.run.smithi104.stdout: "objectstore_blackhole": "false", 2020-09-08T15:37:14.237 INFO:teuthology.orchestra.run.smithi104.stdout: "openssl_engine_opts": "", 2020-09-08T15:37:14.237 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_agent_delay_time": "5.000000", 2020-09-08T15:37:14.237 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_agent_hist_halflife": "1000", 2020-09-08T15:37:14.237 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_agent_max_low_ops": "2", 2020-09-08T15:37:14.238 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_agent_max_ops": "4", 2020-09-08T15:37:14.238 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_agent_min_evict_effort": "0.100000", 2020-09-08T15:37:14.238 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_agent_quantize_effort": "0.100000", 2020-09-08T15:37:14.238 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_agent_slop": "0.020000", 2020-09-08T15:37:14.238 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_allow_recovery_below_min_size": "true", 2020-09-08T15:37:14.239 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_asio_thread_count": "2", 2020-09-08T15:37:14.239 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_async_recovery_min_cost": "100", 2020-09-08T15:37:14.239 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_auto_mark_unfound_lost": "false", 2020-09-08T15:37:14.239 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_backfill_retry_interval": "30.000000", 2020-09-08T15:37:14.239 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_backfill_scan_max": "512", 2020-09-08T15:37:14.239 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_backfill_scan_min": "64", 2020-09-08T15:37:14.240 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_backoff_on_degraded": "false", 2020-09-08T15:37:14.240 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_backoff_on_peering": "false", 2020-09-08T15:37:14.240 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_backoff_on_unfound": "true", 2020-09-08T15:37:14.240 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_beacon_report_interval": "300", 2020-09-08T15:37:14.240 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bench_duration": "30", 2020-09-08T15:37:14.241 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bench_large_size_max_throughput": "104857600", 2020-09-08T15:37:14.241 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bench_max_block_size": "67108864", 2020-09-08T15:37:14.241 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bench_small_size_max_iops": "100", 2020-09-08T15:37:14.241 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_blkin_trace_all": "false", 2020-09-08T15:37:14.241 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2020-09-08T15:37:14.242 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2020-09-08T15:37:14.242 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_check_for_log_corruption": "false", 2020-09-08T15:37:14.242 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_check_max_object_name_len_on_startup": "true", 2020-09-08T15:37:14.242 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue", 2020-09-08T15:37:14.242 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2020-09-08T15:37:14.242 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue", 2020-09-08T15:37:14.243 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_class_update_on_start": "true", 2020-09-08T15:37:14.243 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_client_message_cap": "0", 2020-09-08T15:37:14.243 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_client_message_size_cap": "524288000", 2020-09-08T15:37:14.243 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_client_op_priority": "63", 2020-09-08T15:37:14.243 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_client_watch_timeout": "30", 2020-09-08T15:37:14.244 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_command_max_records": "256", 2020-09-08T15:37:14.244 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_command_thread_suicide_timeout": "900", 2020-09-08T15:37:14.244 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_command_thread_timeout": "600", 2020-09-08T15:37:14.244 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_compact_on_start": "false", 2020-09-08T15:37:14.244 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_copyfrom_max_chunk": "8388608", 2020-09-08T15:37:14.245 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_crush_chooseleaf_type": "1", 2020-09-08T15:37:14.245 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_crush_initial_weight": "-1.000000", 2020-09-08T15:37:14.245 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_crush_update_on_start": "true", 2020-09-08T15:37:14.245 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_crush_update_weight_set": "true", 2020-09-08T15:37:14.245 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2020-09-08T15:37:14.245 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2020-09-08T15:37:14.246 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2020-09-08T15:37:14.246 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_drop_ping_duration": "0", 2020-09-08T15:37:14.246 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_drop_ping_probability": "0.000000", 2020-09-08T15:37:14.246 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_feed_pullee": "-1", 2020-09-08T15:37:14.246 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_inject_copyfrom_error": "false", 2020-09-08T15:37:14.247 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2020-09-08T15:37:14.247 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2020-09-08T15:37:14.247 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_misdirected_ops": "false", 2020-09-08T15:37:14.247 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_no_acting_change": "false", 2020-09-08T15:37:14.247 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_no_purge_strays": "false", 2020-09-08T15:37:14.248 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_op_order": "false", 2020-09-08T15:37:14.248 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_pg_log_writeout": "false", 2020-09-08T15:37:14.248 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_pretend_recovery_active": "false", 2020-09-08T15:37:14.248 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_random_push_read_error": "0.000000", 2020-09-08T15:37:14.248 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2020-09-08T15:37:14.248 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_shutdown": "false", 2020-09-08T15:37:14.249 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2020-09-08T15:37:14.249 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2020-09-08T15:37:14.249 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_verify_cached_snaps": "false", 2020-09-08T15:37:14.249 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_verify_missing_on_start": "false", 2020-09-08T15:37:14.249 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_verify_snaps": "false", 2020-09-08T15:37:14.250 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_debug_verify_stray_on_activate": "false", 2020-09-08T15:37:14.250 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_deep_scrub_interval": "604800.000000", 2020-09-08T15:37:14.250 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_deep_scrub_keys": "1024", 2020-09-08T15:37:14.250 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2020-09-08T15:37:14.250 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2020-09-08T15:37:14.251 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2020-09-08T15:37:14.251 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_deep_scrub_stride": "524288", 2020-09-08T15:37:14.251 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2020-09-08T15:37:14.251 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_default_data_pool_replay_window": "45", 2020-09-08T15:37:14.251 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_default_notify_timeout": "30", 2020-09-08T15:37:14.251 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_delete_sleep": "0.000000", 2020-09-08T15:37:14.252 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_delete_sleep_hdd": "5.000000", 2020-09-08T15:37:14.252 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_delete_sleep_hybrid": "2.000000", 2020-09-08T15:37:14.252 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_delete_sleep_ssd": "0.000000", 2020-09-08T15:37:14.252 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_discard_disconnected_ops": "true", 2020-09-08T15:37:14.252 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_enable_op_tracker": "true", 2020-09-08T15:37:14.253 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_erasure_code_plugins": "jerasure lrc isa", 2020-09-08T15:37:14.253 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_failsafe_full_ratio": "0.970000", 2020-09-08T15:37:14.253 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_fast_fail_on_connection_refused": "true", 2020-09-08T15:37:14.253 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_fast_info": "true", 2020-09-08T15:37:14.253 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_fast_shutdown": "true", 2020-09-08T15:37:14.253 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_find_best_info_ignore_history_les": "false", 2020-09-08T15:37:14.254 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_force_auth_primary_missing_objects": "100", 2020-09-08T15:37:14.254 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2020-09-08T15:37:14.254 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_function_tracing": "false", 2020-09-08T15:37:14.254 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_heartbeat_grace": "20", 2020-09-08T15:37:14.254 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_heartbeat_interval": "6", 2020-09-08T15:37:14.260 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2020-09-08T15:37:14.260 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_heartbeat_min_peers": "10", 2020-09-08T15:37:14.260 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_heartbeat_min_size": "2000", 2020-09-08T15:37:14.260 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_heartbeat_stale": "600", 2020-09-08T15:37:14.260 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2020-09-08T15:37:14.261 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_hit_set_max_size": "100000", 2020-09-08T15:37:14.261 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_hit_set_min_size": "1000", 2020-09-08T15:37:14.261 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_hit_set_namespace": ".ceph-internal", 2020-09-08T15:37:14.261 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_ignore_stale_divergent_priors": "false", 2020-09-08T15:37:14.261 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2020-09-08T15:37:14.262 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_inject_failure_on_pg_removal": "false", 2020-09-08T15:37:14.262 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2020-09-08T15:37:14.262 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_journal_flush_on_shutdown": "true", 2020-09-08T15:37:14.262 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_journal_size": "5120", 2020-09-08T15:37:14.262 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_kill_backfill_at": "0", 2020-09-08T15:37:14.262 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_loop_before_reset_tphandle": "64", 2020-09-08T15:37:14.263 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_map_cache_size": "50", 2020-09-08T15:37:14.263 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_map_dedup": "true", 2020-09-08T15:37:14.263 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_map_message_max": "40", 2020-09-08T15:37:14.263 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_map_message_max_bytes": "10485760", 2020-09-08T15:37:14.263 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_map_share_max_epochs": "40", 2020-09-08T15:37:14.264 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_attr_name_len": "100", 2020-09-08T15:37:14.264 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_attr_size": "0", 2020-09-08T15:37:14.264 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_backfills": "1", 2020-09-08T15:37:14.264 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_markdown_count": "5", 2020-09-08T15:37:14.264 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_markdown_period": "600", 2020-09-08T15:37:14.265 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_object_name_len": "2048", 2020-09-08T15:37:14.265 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_object_namespace_len": "256", 2020-09-08T15:37:14.265 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_object_size": "134217728", 2020-09-08T15:37:14.265 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2020-09-08T15:37:14.265 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_omap_entries_per_request": "1024", 2020-09-08T15:37:14.265 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_pg_blocked_by": "16", 2020-09-08T15:37:14.266 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_pg_log_entries": "10000", 2020-09-08T15:37:14.266 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2020-09-08T15:37:14.266 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_pgls": "1024", 2020-09-08T15:37:14.266 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_push_cost": "8388608", 2020-09-08T15:37:14.266 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_push_objects": "10", 2020-09-08T15:37:14.267 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_scrubs": "1", 2020-09-08T15:37:14.267 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2020-09-08T15:37:14.267 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_trimming_pgs": "2", 2020-09-08T15:37:14.267 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_write_op_reply_len": "32", 2020-09-08T15:37:14.267 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_max_write_size": "90", 2020-09-08T15:37:14.268 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2020-09-08T15:37:14.268 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mclock_scheduler_background_best_effort_lim": "999999", 2020-09-08T15:37:14.268 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mclock_scheduler_background_best_effort_res": "1", 2020-09-08T15:37:14.268 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2020-09-08T15:37:14.268 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mclock_scheduler_background_recovery_lim": "999999", 2020-09-08T15:37:14.268 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mclock_scheduler_background_recovery_res": "1", 2020-09-08T15:37:14.269 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2020-09-08T15:37:14.269 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mclock_scheduler_client_lim": "999999", 2020-09-08T15:37:14.269 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mclock_scheduler_client_res": "1", 2020-09-08T15:37:14.269 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mclock_scheduler_client_wgt": "1", 2020-09-08T15:37:14.269 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_memory_base": "805306368", 2020-09-08T15:37:14.270 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_memory_cache_min": "134217728", 2020-09-08T15:37:14.270 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_memory_cache_resize_interval": "1.000000", 2020-09-08T15:37:14.270 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_memory_expected_fragmentation": "0.150000", 2020-09-08T15:37:14.270 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_memory_target": "4294967296", 2020-09-08T15:37:14.270 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2020-09-08T15:37:14.271 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_min_pg_log_entries": "250", 2020-09-08T15:37:14.271 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_min_recovery_priority": "0", 2020-09-08T15:37:14.271 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mon_ack_timeout": "30.000000", 2020-09-08T15:37:14.271 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mon_heartbeat_interval": "30", 2020-09-08T15:37:14.271 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2020-09-08T15:37:14.272 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mon_report_interval": "5", 2020-09-08T15:37:14.272 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mon_report_max_in_flight": "2", 2020-09-08T15:37:14.272 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mon_shutdown_timeout": "5.000000", 2020-09-08T15:37:14.272 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_num_cache_shards": "32", 2020-09-08T15:37:14.272 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_num_op_tracker_shard": "32", 2020-09-08T15:37:14.272 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_numa_auto_affinity": "true", 2020-09-08T15:37:14.273 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_numa_node": "-1", 2020-09-08T15:37:14.273 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_numa_prefer_iface": "true", 2020-09-08T15:37:14.273 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_object_clean_region_max_num_intervals": "10", 2020-09-08T15:37:14.273 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_objecter_finishers": "1", 2020-09-08T15:37:14.273 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_objectstore": "bluestore", 2020-09-08T15:37:14.274 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_objectstore_fuse": "false", 2020-09-08T15:37:14.274 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_objectstore_tracing": "false", 2020-09-08T15:37:14.274 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_op_complaint_time": "30.000000", 2020-09-08T15:37:14.274 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_op_history_duration": "600", 2020-09-08T15:37:14.274 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_op_history_size": "20", 2020-09-08T15:37:14.275 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_op_history_slow_op_size": "20", 2020-09-08T15:37:14.275 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2020-09-08T15:37:14.275 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_op_log_threshold": "5", 2020-09-08T15:37:14.275 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_op_num_shards": "0", 2020-09-08T15:37:14.275 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_op_num_shards_hdd": "5", 2020-09-08T15:37:14.275 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_op_num_shards_ssd": "8", 2020-09-08T15:37:14.276 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_op_num_threads_per_shard": "0", 2020-09-08T15:37:14.276 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2020-09-08T15:37:14.276 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2020-09-08T15:37:14.276 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2020-09-08T15:37:14.276 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_op_pq_min_cost": "65536", 2020-09-08T15:37:14.277 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_op_queue": "wpq", 2020-09-08T15:37:14.277 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_op_queue_cut_off": "high", 2020-09-08T15:37:14.277 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_op_thread_suicide_timeout": "150", 2020-09-08T15:37:14.277 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_op_thread_timeout": "15", 2020-09-08T15:37:14.277 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_open_classes_on_start": "true", 2020-09-08T15:37:14.278 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_os_flags": "0", 2020-09-08T15:37:14.278 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_peering_op_priority": "255", 2020-09-08T15:37:14.278 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pg_delete_cost": "1048576", 2020-09-08T15:37:14.278 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pg_delete_priority": "5", 2020-09-08T15:37:14.278 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2020-09-08T15:37:14.278 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2020-09-08T15:37:14.279 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pg_log_dups_tracked": "3000", 2020-09-08T15:37:14.279 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pg_log_trim_max": "10000", 2020-09-08T15:37:14.279 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pg_log_trim_min": "100", 2020-09-08T15:37:14.279 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2020-09-08T15:37:14.279 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pg_object_context_cache_count": "64", 2020-09-08T15:37:14.280 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pg_stat_report_interval_max": "500", 2020-09-08T15:37:14.280 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2020-09-08T15:37:14.280 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_cache_min_evict_age": "0", 2020-09-08T15:37:14.280 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_cache_min_flush_age": "0", 2020-09-08T15:37:14.280 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2020-09-08T15:37:14.280 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2020-09-08T15:37:14.281 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2020-09-08T15:37:14.281 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_crush_rule": "-1", 2020-09-08T15:37:14.281 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_ec_fast_read": "false", 2020-09-08T15:37:14.281 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd", 2020-09-08T15:37:14.281 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_flag_hashpspool": "true", 2020-09-08T15:37:14.282 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_flag_nodelete": "false", 2020-09-08T15:37:14.282 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_flag_nopgchange": "false", 2020-09-08T15:37:14.282 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_flag_nosizechange": "false", 2020-09-08T15:37:14.282 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_flags": "0", 2020-09-08T15:37:14.282 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2020-09-08T15:37:14.283 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_min_size": "2", 2020-09-08T15:37:14.283 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2020-09-08T15:37:14.283 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_pg_num": "32", 2020-09-08T15:37:14.283 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_pgp_num": "0", 2020-09-08T15:37:14.283 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2020-09-08T15:37:14.283 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_size": "2", 2020-09-08T15:37:14.284 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_default_type": "replicated", 2020-09-08T15:37:14.284 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2020-09-08T15:37:14.284 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pool_use_gmt_hitset": "true", 2020-09-08T15:37:14.284 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_push_per_object_cost": "1000", 2020-09-08T15:37:14.284 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_read_ec_check_for_errors": "false", 2020-09-08T15:37:14.285 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_recover_clone_overlap": "true", 2020-09-08T15:37:14.285 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_recover_clone_overlap_limit": "10", 2020-09-08T15:37:14.285 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_recovery_cost": "20971520", 2020-09-08T15:37:14.285 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_recovery_delay_start": "0.000000", 2020-09-08T15:37:14.285 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_recovery_max_active": "0", 2020-09-08T15:37:14.285 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_recovery_max_active_hdd": "3", 2020-09-08T15:37:14.286 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_recovery_max_active_ssd": "10", 2020-09-08T15:37:14.286 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_recovery_max_chunk": "8388608", 2020-09-08T15:37:14.286 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2020-09-08T15:37:14.286 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_recovery_max_single_start": "1", 2020-09-08T15:37:14.286 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_recovery_op_priority": "3", 2020-09-08T15:37:14.287 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_recovery_op_warn_multiple": "16", 2020-09-08T15:37:14.287 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_recovery_priority": "5", 2020-09-08T15:37:14.287 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_recovery_retry_interval": "30.000000", 2020-09-08T15:37:14.287 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_recovery_sleep": "0.000000", 2020-09-08T15:37:14.287 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_recovery_sleep_hdd": "0.100000", 2020-09-08T15:37:14.288 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2020-09-08T15:37:14.288 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_recovery_sleep_ssd": "0.000000", 2020-09-08T15:37:14.288 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_repair_during_recovery": "false", 2020-09-08T15:37:14.288 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_requested_scrub_priority": "120", 2020-09-08T15:37:14.288 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_rollback_to_cluster_snap": "", 2020-09-08T15:37:14.288 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_scrub_auto_repair": "false", 2020-09-08T15:37:14.289 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_scrub_auto_repair_num_errors": "5", 2020-09-08T15:37:14.289 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_scrub_backoff_ratio": "0.660000", 2020-09-08T15:37:14.289 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_scrub_begin_hour": "0", 2020-09-08T15:37:14.289 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_scrub_begin_week_day": "0", 2020-09-08T15:37:14.289 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_scrub_chunk_max": "25", 2020-09-08T15:37:14.290 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_scrub_chunk_min": "5", 2020-09-08T15:37:14.290 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_scrub_cost": "52428800", 2020-09-08T15:37:14.290 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_scrub_during_recovery": "false", 2020-09-08T15:37:14.290 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_scrub_end_hour": "24", 2020-09-08T15:37:14.290 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_scrub_end_week_day": "7", 2020-09-08T15:37:14.291 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_scrub_extended_sleep": "0.000000", 2020-09-08T15:37:14.291 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2020-09-08T15:37:14.291 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_scrub_invalid_stats": "true", 2020-09-08T15:37:14.291 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_scrub_load_threshold": "0.500000", 2020-09-08T15:37:14.291 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_scrub_max_interval": "604800.000000", 2020-09-08T15:37:14.291 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_scrub_max_preemptions": "5", 2020-09-08T15:37:14.292 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_scrub_min_interval": "86400.000000", 2020-09-08T15:37:14.292 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_scrub_priority": "5", 2020-09-08T15:37:14.292 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_scrub_sleep": "0.000000", 2020-09-08T15:37:14.292 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_shutdown_pgref_assert": "false", 2020-09-08T15:37:14.292 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_skip_data_digest": "false", 2020-09-08T15:37:14.293 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_smart_report_timeout": "5", 2020-09-08T15:37:14.293 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_snap_trim_cost": "1048576", 2020-09-08T15:37:14.293 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_snap_trim_priority": "5", 2020-09-08T15:37:14.293 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_snap_trim_sleep": "0.000000", 2020-09-08T15:37:14.293 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2020-09-08T15:37:14.294 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2020-09-08T15:37:14.294 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2020-09-08T15:37:14.294 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_stats_ack_timeout_decay": "0.900000", 2020-09-08T15:37:14.294 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_stats_ack_timeout_factor": "2.000000", 2020-09-08T15:37:14.294 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2020-09-08T15:37:14.294 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_target_transaction_size": "30", 2020-09-08T15:37:14.295 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_tier_default_cache_hit_set_count": "4", 2020-09-08T15:37:14.295 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2020-09-08T15:37:14.295 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2020-09-08T15:37:14.295 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2020-09-08T15:37:14.295 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2020-09-08T15:37:14.296 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2020-09-08T15:37:14.296 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2020-09-08T15:37:14.296 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_tier_default_cache_mode": "writeback", 2020-09-08T15:37:14.296 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2020-09-08T15:37:14.296 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_tier_promote_max_objects_sec": "25", 2020-09-08T15:37:14.296 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_tracing": "false", 2020-09-08T15:37:14.297 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_use_stale_snap": "false", 2020-09-08T15:37:14.297 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2020-09-08T15:37:14.297 INFO:teuthology.orchestra.run.smithi104.stdout: "osdc_blkin_trace_all": "false", 2020-09-08T15:37:14.297 INFO:teuthology.orchestra.run.smithi104.stdout: "paxos_kill_at": "0", 2020-09-08T15:37:14.297 INFO:teuthology.orchestra.run.smithi104.stdout: "paxos_max_join_drift": "10", 2020-09-08T15:37:14.298 INFO:teuthology.orchestra.run.smithi104.stdout: "paxos_min": "500", 2020-09-08T15:37:14.298 INFO:teuthology.orchestra.run.smithi104.stdout: "paxos_min_wait": "0.050000", 2020-09-08T15:37:14.298 INFO:teuthology.orchestra.run.smithi104.stdout: "paxos_propose_interval": "1.000000", 2020-09-08T15:37:14.298 INFO:teuthology.orchestra.run.smithi104.stdout: "paxos_service_trim_max": "500", 2020-09-08T15:37:14.298 INFO:teuthology.orchestra.run.smithi104.stdout: "paxos_service_trim_min": "10", 2020-09-08T15:37:14.299 INFO:teuthology.orchestra.run.smithi104.stdout: "paxos_stash_full_interval": "25", 2020-09-08T15:37:14.299 INFO:teuthology.orchestra.run.smithi104.stdout: "paxos_trim_max": "500", 2020-09-08T15:37:14.299 INFO:teuthology.orchestra.run.smithi104.stdout: "paxos_trim_min": "250", 2020-09-08T15:37:14.300 INFO:teuthology.orchestra.run.smithi104.stdout: "perf": "true", 2020-09-08T15:37:14.300 INFO:teuthology.orchestra.run.smithi104.stdout: "pid_file": "", 2020-09-08T15:37:14.300 INFO:teuthology.orchestra.run.smithi104.stdout: "plugin_crypto_accelerator": "crypto_isal", 2020-09-08T15:37:14.300 INFO:teuthology.orchestra.run.smithi104.stdout: "plugin_dir": "/usr/lib64/ceph", 2020-09-08T15:37:14.300 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "-", 2020-09-08T15:37:14.300 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrv": "", 2020-09-08T15:37:14.301 INFO:teuthology.orchestra.run.smithi104.stdout: "public_bind_addr": "-", 2020-09-08T15:37:14.301 INFO:teuthology.orchestra.run.smithi104.stdout: "public_network": "", 2020-09-08T15:37:14.301 INFO:teuthology.orchestra.run.smithi104.stdout: "public_network_interface": "", 2020-09-08T15:37:14.301 INFO:teuthology.orchestra.run.smithi104.stdout: "qat_compressor_enabled": "false", 2020-09-08T15:37:14.301 INFO:teuthology.orchestra.run.smithi104.stdout: "rados_mon_op_timeout": "0.000000", 2020-09-08T15:37:14.302 INFO:teuthology.orchestra.run.smithi104.stdout: "rados_osd_op_timeout": "0.000000", 2020-09-08T15:37:14.302 INFO:teuthology.orchestra.run.smithi104.stdout: "rados_tracing": "false", 2020-09-08T15:37:14.302 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_atime_update_interval": "60", 2020-09-08T15:37:14.302 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2020-09-08T15:37:14.302 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_balance_parent_reads": "false", 2020-09-08T15:37:14.303 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_balance_snap_reads": "false", 2020-09-08T15:37:14.303 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_blkin_trace_all": "false", 2020-09-08T15:37:14.303 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_blocklist_expire_seconds": "0", 2020-09-08T15:37:14.303 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_blocklist_on_break_lock": "true", 2020-09-08T15:37:14.303 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_cache": "true", 2020-09-08T15:37:14.303 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_cache_block_writes_upfront": "false", 2020-09-08T15:37:14.304 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_cache_max_dirty": "25165824", 2020-09-08T15:37:14.304 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_cache_max_dirty_age": "1.000000", 2020-09-08T15:37:14.304 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_cache_max_dirty_object": "0", 2020-09-08T15:37:14.304 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_cache_policy": "writearound", 2020-09-08T15:37:14.304 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_cache_size": "33554432", 2020-09-08T15:37:14.305 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_cache_target_dirty": "16777216", 2020-09-08T15:37:14.305 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_cache_writethrough_until_flush": "true", 2020-09-08T15:37:14.305 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_clone_copy_on_read": "false", 2020-09-08T15:37:14.305 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_compression_hint": "none", 2020-09-08T15:37:14.305 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_concurrent_management_ops": "10", 2020-09-08T15:37:14.306 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_config_pool_override_update_timestamp": "0", 2020-09-08T15:37:14.306 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_default_clone_format": "auto", 2020-09-08T15:37:14.306 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_default_data_pool": "", 2020-09-08T15:37:14.306 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_default_features": "61", 2020-09-08T15:37:14.306 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_default_format": "2", 2020-09-08T15:37:14.306 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_default_map_options": "", 2020-09-08T15:37:14.307 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_default_order": "22", 2020-09-08T15:37:14.307 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_default_pool": "rbd", 2020-09-08T15:37:14.307 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2020-09-08T15:37:14.307 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_default_stripe_count": "0", 2020-09-08T15:37:14.307 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_default_stripe_unit": "0", 2020-09-08T15:37:14.308 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_disable_zero_copy_writes": "true", 2020-09-08T15:37:14.308 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_discard_granularity_bytes": "65536", 2020-09-08T15:37:14.308 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_discard_on_zeroed_write_same": "true", 2020-09-08T15:37:14.308 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_enable_alloc_hint": "true", 2020-09-08T15:37:14.308 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2020-09-08T15:37:14.308 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_io_scheduler": "simple", 2020-09-08T15:37:14.309 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2020-09-08T15:37:14.309 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_journal_commit_age": "5.000000", 2020-09-08T15:37:14.309 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2020-09-08T15:37:14.309 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_journal_max_payload_bytes": "16384", 2020-09-08T15:37:14.309 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_journal_object_flush_age": "0.000000", 2020-09-08T15:37:14.310 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_journal_object_flush_bytes": "1048576", 2020-09-08T15:37:14.310 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_journal_object_flush_interval": "0", 2020-09-08T15:37:14.310 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2020-09-08T15:37:14.310 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2020-09-08T15:37:14.310 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_journal_order": "24", 2020-09-08T15:37:14.311 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_journal_pool": "", 2020-09-08T15:37:14.311 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_journal_splay_width": "4", 2020-09-08T15:37:14.311 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_localize_parent_reads": "false", 2020-09-08T15:37:14.311 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_localize_snap_reads": "false", 2020-09-08T15:37:14.311 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2020-09-08T15:37:14.311 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2020-09-08T15:37:14.312 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2020-09-08T15:37:14.312 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2020-09-08T15:37:14.312 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2020-09-08T15:37:14.312 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2020-09-08T15:37:14.312 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_image_policy_type": "simple", 2020-09-08T15:37:14.313 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2020-09-08T15:37:14.313 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_image_state_check_interval": "30", 2020-09-08T15:37:14.313 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2020-09-08T15:37:14.313 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2020-09-08T15:37:14.313 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2020-09-08T15:37:14.313 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2020-09-08T15:37:14.314 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2020-09-08T15:37:14.314 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_memory_autotune": "true", 2020-09-08T15:37:14.314 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_memory_base": "805306368", 2020-09-08T15:37:14.314 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2020-09-08T15:37:14.314 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_memory_cache_min": "134217728", 2020-09-08T15:37:14.315 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2020-09-08T15:37:14.315 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2020-09-08T15:37:14.315 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_memory_target": "4294967296", 2020-09-08T15:37:14.315 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_perf_stats_prio": "5", 2020-09-08T15:37:14.315 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2020-09-08T15:37:14.316 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2020-09-08T15:37:14.316 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirroring_delete_delay": "0", 2020-09-08T15:37:14.316 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirroring_max_mirroring_snapshots": "3", 2020-09-08T15:37:14.316 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirroring_replay_delay": "0", 2020-09-08T15:37:14.316 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2020-09-08T15:37:14.316 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2020-09-08T15:37:14.317 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_move_to_trash_on_remove": "false", 2020-09-08T15:37:14.317 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2020-09-08T15:37:14.317 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_mtime_update_interval": "60", 2020-09-08T15:37:14.317 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_non_blocking_aio": "true", 2020-09-08T15:37:14.317 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_op_thread_timeout": "60", 2020-09-08T15:37:14.318 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_op_threads": "1", 2020-09-08T15:37:14.318 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_parent_cache_enabled": "false", 2020-09-08T15:37:14.318 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_plugins": "", 2020-09-08T15:37:14.318 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_qos_bps_burst": "0", 2020-09-08T15:37:14.318 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_qos_bps_burst_seconds": "1", 2020-09-08T15:37:14.318 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_qos_bps_limit": "0", 2020-09-08T15:37:14.319 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_qos_iops_burst": "0", 2020-09-08T15:37:14.319 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_qos_iops_burst_seconds": "1", 2020-09-08T15:37:14.319 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_qos_iops_limit": "0", 2020-09-08T15:37:14.319 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_qos_read_bps_burst": "0", 2020-09-08T15:37:14.319 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2020-09-08T15:37:14.320 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_qos_read_bps_limit": "0", 2020-09-08T15:37:14.320 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_qos_read_iops_burst": "0", 2020-09-08T15:37:14.320 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2020-09-08T15:37:14.320 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_qos_read_iops_limit": "0", 2020-09-08T15:37:14.320 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_qos_schedule_tick_min": "50", 2020-09-08T15:37:14.321 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_qos_write_bps_burst": "0", 2020-09-08T15:37:14.321 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2020-09-08T15:37:14.321 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_qos_write_bps_limit": "0", 2020-09-08T15:37:14.321 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_qos_write_iops_burst": "0", 2020-09-08T15:37:14.321 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2020-09-08T15:37:14.321 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_qos_write_iops_limit": "0", 2020-09-08T15:37:14.322 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_quiesce_notification_attempts": "10", 2020-09-08T15:37:14.322 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_read_from_replica_policy": "default", 2020-09-08T15:37:14.322 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2020-09-08T15:37:14.322 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_readahead_max_bytes": "524288", 2020-09-08T15:37:14.322 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_readahead_trigger_requests": "10", 2020-09-08T15:37:14.323 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_request_timed_out_seconds": "30", 2020-09-08T15:37:14.323 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_rwl_enabled": "false", 2020-09-08T15:37:14.323 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_rwl_log_periodic_stats": "false", 2020-09-08T15:37:14.323 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_rwl_path": "/tmp", 2020-09-08T15:37:14.323 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_rwl_size": "1073741824", 2020-09-08T15:37:14.324 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_skip_partial_discard": "true", 2020-09-08T15:37:14.324 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2020-09-08T15:37:14.324 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_tracing": "false", 2020-09-08T15:37:14.324 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_validate_names": "true", 2020-09-08T15:37:14.324 INFO:teuthology.orchestra.run.smithi104.stdout: "rbd_validate_pool": "true", 2020-09-08T15:37:14.324 INFO:teuthology.orchestra.run.smithi104.stdout: "restapi_base_url": "", 2020-09-08T15:37:14.325 INFO:teuthology.orchestra.run.smithi104.stdout: "restapi_log_level": "", 2020-09-08T15:37:14.325 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_acl_grants_max_num": "100", 2020-09-08T15:37:14.325 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_admin_entry": "admin", 2020-09-08T15:37:14.325 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_barbican_url": "", 2020-09-08T15:37:14.325 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2020-09-08T15:37:14.326 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_bucket_default_quota_max_size": "-1", 2020-09-08T15:37:14.326 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_bucket_index_max_aio": "128", 2020-09-08T15:37:14.326 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_bucket_quota_cache_size": "10000", 2020-09-08T15:37:14.326 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_bucket_quota_soft_threshold": "0.950000", 2020-09-08T15:37:14.326 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_bucket_quota_ttl": "600", 2020-09-08T15:37:14.327 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_cache_enabled": "true", 2020-09-08T15:37:14.327 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_cache_expiry_interval": "900", 2020-09-08T15:37:14.327 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_cache_lru_size": "10000", 2020-09-08T15:37:14.327 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_content_length_compat": "false", 2020-09-08T15:37:14.327 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_copy_obj_progress": "true", 2020-09-08T15:37:14.327 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2020-09-08T15:37:14.328 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_cors_rules_max_num": "100", 2020-09-08T15:37:14.328 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_cross_domain_policy": "", 2020-09-08T15:37:14.328 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_crypt_default_encryption_key": "", 2020-09-08T15:37:14.328 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_crypt_require_ssl": "true", 2020-09-08T15:37:14.328 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2020-09-08T15:37:14.329 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2020-09-08T15:37:14.329 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_crypt_suppress_logs": "true", 2020-09-08T15:37:14.329 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_crypt_vault_addr": "", 2020-09-08T15:37:14.329 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_crypt_vault_auth": "token", 2020-09-08T15:37:14.329 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_crypt_vault_namespace": "", 2020-09-08T15:37:14.330 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_crypt_vault_prefix": "", 2020-09-08T15:37:14.330 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_crypt_vault_secret_engine": "transit", 2020-09-08T15:37:14.330 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_crypt_vault_token_file": "", 2020-09-08T15:37:14.330 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_curl_low_speed_limit": "1024", 2020-09-08T15:37:14.330 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_curl_low_speed_time": "300", 2020-09-08T15:37:14.330 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_curl_wait_timeout_ms": "1000", 2020-09-08T15:37:14.331 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2020-09-08T15:37:14.331 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_data_log_changes_size": "1000", 2020-09-08T15:37:14.331 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_data_log_num_shards": "128", 2020-09-08T15:37:14.331 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_data_log_obj_prefix": "data_log", 2020-09-08T15:37:14.331 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_data_log_window": "30", 2020-09-08T15:37:14.332 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_data_notify_interval_msec": "200", 2020-09-08T15:37:14.332 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_default_realm_info_oid": "default.realm", 2020-09-08T15:37:14.332 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_default_region_info_oid": "default.region", 2020-09-08T15:37:14.332 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_default_zone_info_oid": "default.zone", 2020-09-08T15:37:14.332 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2020-09-08T15:37:14.332 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_defer_to_bucket_acls": "", 2020-09-08T15:37:14.333 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_delete_multi_obj_max_num": "1000", 2020-09-08T15:37:14.333 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_dmclock_admin_lim": "0.000000", 2020-09-08T15:37:14.333 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_dmclock_admin_res": "100.000000", 2020-09-08T15:37:14.333 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2020-09-08T15:37:14.333 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_dmclock_auth_lim": "0.000000", 2020-09-08T15:37:14.334 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_dmclock_auth_res": "200.000000", 2020-09-08T15:37:14.334 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2020-09-08T15:37:14.334 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_dmclock_data_lim": "0.000000", 2020-09-08T15:37:14.334 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_dmclock_data_res": "500.000000", 2020-09-08T15:37:14.334 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_dmclock_data_wgt": "500.000000", 2020-09-08T15:37:14.335 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2020-09-08T15:37:14.335 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_dmclock_metadata_res": "500.000000", 2020-09-08T15:37:14.335 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2020-09-08T15:37:14.335 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_dns_name": "", 2020-09-08T15:37:14.335 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_dns_s3website_name": "", 2020-09-08T15:37:14.335 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_dynamic_resharding": "true", 2020-09-08T15:37:14.336 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, pubsub", 2020-09-08T15:37:14.336 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_enable_gc_threads": "true", 2020-09-08T15:37:14.336 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_enable_lc_threads": "true", 2020-09-08T15:37:14.336 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_enable_ops_log": "false", 2020-09-08T15:37:14.336 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_enable_quota_threads": "true", 2020-09-08T15:37:14.337 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_enable_static_website": "false", 2020-09-08T15:37:14.337 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_enable_usage_log": "false", 2020-09-08T15:37:14.337 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_enforce_swift_acls": "true", 2020-09-08T15:37:14.337 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_exit_timeout_secs": "120", 2020-09-08T15:37:14.337 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_expose_bucket": "false", 2020-09-08T15:37:14.337 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_extended_http_attrs": "", 2020-09-08T15:37:14.338 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_fcgi_socket_backlog": "1024", 2020-09-08T15:37:14.338 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2020-09-08T15:37:14.338 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_frontends": "beast port=7480", 2020-09-08T15:37:14.338 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_gc_max_concurrent_io": "10", 2020-09-08T15:37:14.338 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_gc_max_deferred": "50", 2020-09-08T15:37:14.339 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2020-09-08T15:37:14.339 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_gc_max_objs": "32", 2020-09-08T15:37:14.339 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_gc_max_queue_size": "134213632", 2020-09-08T15:37:14.339 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_gc_max_trim_chunk": "16", 2020-09-08T15:37:14.339 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_gc_obj_min_wait": "7200", 2020-09-08T15:37:14.340 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_gc_processor_max_time": "3600", 2020-09-08T15:37:14.340 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_gc_processor_period": "3600", 2020-09-08T15:37:14.340 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_get_obj_max_req_size": "4194304", 2020-09-08T15:37:14.340 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_get_obj_window_size": "16777216", 2020-09-08T15:37:14.340 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_healthcheck_disabling_path": "", 2020-09-08T15:37:14.340 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_host": "", 2020-09-08T15:37:14.341 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_ignore_get_invalid_range": "false", 2020-09-08T15:37:14.341 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_init_timeout": "300", 2020-09-08T15:37:14.341 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2020-09-08T15:37:14.341 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_keystone_accepted_admin_roles": "", 2020-09-08T15:37:14.341 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2020-09-08T15:37:14.342 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_keystone_admin_domain": "", 2020-09-08T15:37:14.342 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_keystone_admin_password": "", 2020-09-08T15:37:14.342 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_keystone_admin_password_path": "", 2020-09-08T15:37:14.342 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_keystone_admin_project": "", 2020-09-08T15:37:14.342 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_keystone_admin_tenant": "", 2020-09-08T15:37:14.343 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_keystone_admin_token": "", 2020-09-08T15:37:14.343 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_keystone_admin_token_path": "", 2020-09-08T15:37:14.343 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_keystone_admin_user": "", 2020-09-08T15:37:14.343 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_keystone_api_version": "2", 2020-09-08T15:37:14.343 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_keystone_barbican_domain": "", 2020-09-08T15:37:14.343 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_keystone_barbican_password": "", 2020-09-08T15:37:14.344 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_keystone_barbican_project": "", 2020-09-08T15:37:14.344 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_keystone_barbican_tenant": "", 2020-09-08T15:37:14.344 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_keystone_barbican_user": "", 2020-09-08T15:37:14.344 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_keystone_implicit_tenants": "false", 2020-09-08T15:37:14.344 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_keystone_token_cache_size": "10000", 2020-09-08T15:37:14.345 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_keystone_url": "", 2020-09-08T15:37:14.345 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_keystone_verify_ssl": "true", 2020-09-08T15:37:14.345 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_lc_debug_interval": "-1", 2020-09-08T15:37:14.345 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_lc_lock_max_time": "90", 2020-09-08T15:37:14.345 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_lc_max_objs": "32", 2020-09-08T15:37:14.345 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_lc_max_rules": "1000", 2020-09-08T15:37:14.346 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_lc_max_worker": "3", 2020-09-08T15:37:14.346 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_lc_max_wp_worker": "3", 2020-09-08T15:37:14.346 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_lc_thread_delay": "0", 2020-09-08T15:37:14.346 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2020-09-08T15:37:14.346 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_ldap_dnattr": "uid", 2020-09-08T15:37:14.347 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2020-09-08T15:37:14.347 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_ldap_searchfilter": "", 2020-09-08T15:37:14.347 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2020-09-08T15:37:14.347 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_ldap_uri": "ldaps://", 2020-09-08T15:37:14.347 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2020-09-08T15:37:14.348 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_list_bucket_min_readahead": "1000", 2020-09-08T15:37:14.348 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_list_buckets_max_chunk": "1000", 2020-09-08T15:37:14.348 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_log_http_headers": "", 2020-09-08T15:37:14.348 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_log_nonexistent_bucket": "false", 2020-09-08T15:37:14.348 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2020-09-08T15:37:14.348 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_log_object_name_utc": "false", 2020-09-08T15:37:14.349 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_max_attr_name_len": "0", 2020-09-08T15:37:14.349 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_max_attr_size": "0", 2020-09-08T15:37:14.349 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_max_attrs_num_in_req": "0", 2020-09-08T15:37:14.349 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_max_chunk_size": "4194304", 2020-09-08T15:37:14.349 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_max_concurrent_requests": "1024", 2020-09-08T15:37:14.350 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_max_dynamic_shards": "1999", 2020-09-08T15:37:14.350 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_max_listing_results": "1000", 2020-09-08T15:37:14.350 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_max_notify_retries": "3", 2020-09-08T15:37:14.350 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_max_objs_per_shard": "100000", 2020-09-08T15:37:14.350 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_max_put_param_size": "1048576", 2020-09-08T15:37:14.350 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_max_put_size": "5368709120", 2020-09-08T15:37:14.351 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_max_slo_entries": "1000", 2020-09-08T15:37:14.351 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_md_log_max_shards": "64", 2020-09-08T15:37:14.351 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_md_notify_interval_msec": "200", 2020-09-08T15:37:14.351 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_mime_types_file": "/etc/mime.types", 2020-09-08T15:37:14.351 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_mp_lock_max_time": "600", 2020-09-08T15:37:14.352 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_multipart_min_part_size": "5242880", 2020-09-08T15:37:14.352 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_multipart_part_upload_limit": "10000", 2020-09-08T15:37:14.352 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_nfs_fhcache_partitions": "3", 2020-09-08T15:37:14.352 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_nfs_fhcache_size": "2017", 2020-09-08T15:37:14.352 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2020-09-08T15:37:14.352 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_nfs_lru_lanes": "5", 2020-09-08T15:37:14.353 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_nfs_max_gc": "300", 2020-09-08T15:37:14.353 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_nfs_namespace_expire_secs": "300", 2020-09-08T15:37:14.353 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_nfs_run_gc_threads": "false", 2020-09-08T15:37:14.353 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_nfs_run_lc_threads": "false", 2020-09-08T15:37:14.353 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_nfs_run_quota_threads": "false", 2020-09-08T15:37:14.354 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_nfs_run_sync_thread": "false", 2020-09-08T15:37:14.354 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_nfs_s3_fast_attrs": "false", 2020-09-08T15:37:14.354 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_nfs_write_completion_interval_s": "10", 2020-09-08T15:37:14.354 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_num_async_rados_threads": "32", 2020-09-08T15:37:14.354 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_num_control_oids": "8", 2020-09-08T15:37:14.355 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_num_rados_handles": "1", 2020-09-08T15:37:14.355 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_numa_node": "-1", 2020-09-08T15:37:14.355 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_obj_stripe_size": "4194304", 2020-09-08T15:37:14.355 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_obj_tombstone_cache_size": "1000", 2020-09-08T15:37:14.355 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_objexp_chunk_size": "100", 2020-09-08T15:37:14.355 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_objexp_gc_interval": "600", 2020-09-08T15:37:14.356 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_objexp_hints_num_shards": "127", 2020-09-08T15:37:14.356 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_olh_pending_timeout_sec": "3600", 2020-09-08T15:37:14.356 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_op_thread_suicide_timeout": "0", 2020-09-08T15:37:14.356 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_op_thread_timeout": "600", 2020-09-08T15:37:14.356 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_op_tracing": "false", 2020-09-08T15:37:14.357 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_opa_token": "", 2020-09-08T15:37:14.357 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_opa_url": "", 2020-09-08T15:37:14.357 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_opa_verify_ssl": "true", 2020-09-08T15:37:14.357 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_ops_log_data_backlog": "5242880", 2020-09-08T15:37:14.357 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_ops_log_rados": "true", 2020-09-08T15:37:14.357 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_ops_log_socket_path": "", 2020-09-08T15:37:14.358 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_override_bucket_index_max_shards": "0", 2020-09-08T15:37:14.358 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2020-09-08T15:37:14.358 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_period_push_interval": "2.000000", 2020-09-08T15:37:14.358 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_period_push_interval_max": "30.000000", 2020-09-08T15:37:14.358 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_period_root_pool": ".rgw.root", 2020-09-08T15:37:14.359 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_port": "", 2020-09-08T15:37:14.359 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_print_continue": "true", 2020-09-08T15:37:14.359 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_print_prohibited_content_length": "false", 2020-09-08T15:37:14.359 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_put_obj_max_window_size": "67108864", 2020-09-08T15:37:14.359 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_put_obj_min_window_size": "16777216", 2020-09-08T15:37:14.359 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2020-09-08T15:37:14.360 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_rados_pool_pg_num_min": "8", 2020-09-08T15:37:14.360 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_rados_pool_recovery_priority": "5", 2020-09-08T15:37:14.360 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_rados_tracing": "false", 2020-09-08T15:37:14.360 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_realm": "", 2020-09-08T15:37:14.360 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_realm_root_pool": ".rgw.root", 2020-09-08T15:37:14.361 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_region": "", 2020-09-08T15:37:14.361 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_region_root_pool": ".rgw.root", 2020-09-08T15:37:14.361 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_relaxed_region_enforcement": "false", 2020-09-08T15:37:14.361 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_relaxed_s3_bucket_names": "false", 2020-09-08T15:37:14.361 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2020-09-08T15:37:14.362 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_request_uri": "", 2020-09-08T15:37:14.362 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_reshard_batch_size": "64", 2020-09-08T15:37:14.362 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_reshard_bucket_lock_duration": "360", 2020-09-08T15:37:14.362 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_reshard_max_aio": "128", 2020-09-08T15:37:14.362 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_reshard_num_logs": "16", 2020-09-08T15:37:14.362 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_reshard_thread_interval": "600", 2020-09-08T15:37:14.363 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_resolve_cname": "false", 2020-09-08T15:37:14.363 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_rest_getusage_op_compat": "false", 2020-09-08T15:37:14.363 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_run_sync_thread": "true", 2020-09-08T15:37:14.363 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_s3_auth_order": "sts, external, local", 2020-09-08T15:37:14.363 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_s3_auth_use_keystone": "false", 2020-09-08T15:37:14.364 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_s3_auth_use_ldap": "false", 2020-09-08T15:37:14.364 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_s3_auth_use_rados": "true", 2020-09-08T15:37:14.364 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_s3_auth_use_sts": "false", 2020-09-08T15:37:14.364 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_s3_success_create_obj_status": "0", 2020-09-08T15:37:14.364 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_safe_max_objects_per_shard": "102400", 2020-09-08T15:37:14.365 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_scheduler_type": "throttler", 2020-09-08T15:37:14.365 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_script_uri": "", 2020-09-08T15:37:14.365 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_service_provider_name": "", 2020-09-08T15:37:14.365 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_shard_warning_threshold": "90.000000", 2020-09-08T15:37:14.365 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_socket_path": "", 2020-09-08T15:37:14.365 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_sts_client_id": "", 2020-09-08T15:37:14.366 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_sts_client_secret": "", 2020-09-08T15:37:14.366 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_sts_entry": "sts", 2020-09-08T15:37:14.366 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_sts_key": "sts", 2020-09-08T15:37:14.366 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_sts_max_session_duration": "43200", 2020-09-08T15:37:14.366 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_sts_token_introspection_url": "", 2020-09-08T15:37:14.367 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_swift_account_in_url": "false", 2020-09-08T15:37:14.367 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_swift_auth_entry": "auth", 2020-09-08T15:37:14.367 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_swift_auth_url": "", 2020-09-08T15:37:14.367 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_swift_custom_header": "", 2020-09-08T15:37:14.367 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_swift_enforce_content_length": "false", 2020-09-08T15:37:14.368 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_swift_need_stats": "true", 2020-09-08T15:37:14.368 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_swift_tenant_name": "", 2020-09-08T15:37:14.368 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_swift_token_expiration": "86400", 2020-09-08T15:37:14.368 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_swift_url": "", 2020-09-08T15:37:14.368 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_swift_url_prefix": "swift", 2020-09-08T15:37:14.368 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_swift_versioning_enabled": "false", 2020-09-08T15:37:14.369 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2020-09-08T15:37:14.369 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_sync_lease_period": "120", 2020-09-08T15:37:14.369 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2020-09-08T15:37:14.369 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_sync_log_trim_interval": "1200", 2020-09-08T15:37:14.369 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_sync_log_trim_max_buckets": "16", 2020-09-08T15:37:14.370 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2020-09-08T15:37:14.370 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2020-09-08T15:37:14.370 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_sync_obj_etag_verify": "false", 2020-09-08T15:37:14.370 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_sync_trace_history_size": "4096", 2020-09-08T15:37:14.370 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_sync_trace_per_node_log_size": "32", 2020-09-08T15:37:14.371 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2020-09-08T15:37:14.371 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_thread_pool_size": "512", 2020-09-08T15:37:14.371 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_torrent_comment": "", 2020-09-08T15:37:14.371 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_torrent_createby": "", 2020-09-08T15:37:14.371 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_torrent_encoding": "", 2020-09-08T15:37:14.371 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_torrent_flag": "false", 2020-09-08T15:37:14.372 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_torrent_origin": "", 2020-09-08T15:37:14.372 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_torrent_sha_unit": "524288", 2020-09-08T15:37:14.372 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_torrent_tracker": "", 2020-09-08T15:37:14.372 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_trust_forwarded_https": "false", 2020-09-08T15:37:14.372 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_usage_log_flush_threshold": "1024", 2020-09-08T15:37:14.373 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_usage_log_tick_interval": "30", 2020-09-08T15:37:14.373 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_usage_max_shards": "32", 2020-09-08T15:37:14.373 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_usage_max_user_shards": "1", 2020-09-08T15:37:14.373 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_use_opa_authz": "false", 2020-09-08T15:37:14.373 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_user_default_quota_max_objects": "-1", 2020-09-08T15:37:14.373 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_user_default_quota_max_size": "-1", 2020-09-08T15:37:14.374 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_user_max_buckets": "1000", 2020-09-08T15:37:14.374 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2020-09-08T15:37:14.374 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_user_quota_sync_idle_users": "false", 2020-09-08T15:37:14.374 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_user_quota_sync_interval": "86400", 2020-09-08T15:37:14.374 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_user_quota_sync_wait_time": "86400", 2020-09-08T15:37:14.375 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_user_unique_email": "true", 2020-09-08T15:37:14.375 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_verify_ssl": "true", 2020-09-08T15:37:14.375 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_website_routing_rules_max_num": "50", 2020-09-08T15:37:14.375 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_zone": "", 2020-09-08T15:37:14.375 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_zone_root_pool": ".rgw.root", 2020-09-08T15:37:14.375 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_zonegroup": "", 2020-09-08T15:37:14.376 INFO:teuthology.orchestra.run.smithi104.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2020-09-08T15:37:14.376 INFO:teuthology.orchestra.run.smithi104.stdout: "rocksdb_block_size": "4096", 2020-09-08T15:37:14.376 INFO:teuthology.orchestra.run.smithi104.stdout: "rocksdb_bloom_bits_per_key": "20", 2020-09-08T15:37:14.376 INFO:teuthology.orchestra.run.smithi104.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2020-09-08T15:37:14.376 INFO:teuthology.orchestra.run.smithi104.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "true", 2020-09-08T15:37:14.377 INFO:teuthology.orchestra.run.smithi104.stdout: "rocksdb_cache_row_ratio": "0.000000", 2020-09-08T15:37:14.377 INFO:teuthology.orchestra.run.smithi104.stdout: "rocksdb_cache_shard_bits": "4", 2020-09-08T15:37:14.377 INFO:teuthology.orchestra.run.smithi104.stdout: "rocksdb_cache_size": "536870912", 2020-09-08T15:37:14.377 INFO:teuthology.orchestra.run.smithi104.stdout: "rocksdb_cache_type": "binned_lru", 2020-09-08T15:37:14.377 INFO:teuthology.orchestra.run.smithi104.stdout: "rocksdb_collect_compaction_stats": "false", 2020-09-08T15:37:14.378 INFO:teuthology.orchestra.run.smithi104.stdout: "rocksdb_collect_extended_stats": "false", 2020-09-08T15:37:14.378 INFO:teuthology.orchestra.run.smithi104.stdout: "rocksdb_collect_memory_stats": "false", 2020-09-08T15:37:14.378 INFO:teuthology.orchestra.run.smithi104.stdout: "rocksdb_delete_range_threshold": "1048576", 2020-09-08T15:37:14.378 INFO:teuthology.orchestra.run.smithi104.stdout: "rocksdb_index_type": "binary_search", 2020-09-08T15:37:14.378 INFO:teuthology.orchestra.run.smithi104.stdout: "rocksdb_log_to_ceph_log": "true", 2020-09-08T15:37:14.378 INFO:teuthology.orchestra.run.smithi104.stdout: "rocksdb_metadata_block_size": "4096", 2020-09-08T15:37:14.379 INFO:teuthology.orchestra.run.smithi104.stdout: "rocksdb_partition_filters": "false", 2020-09-08T15:37:14.379 INFO:teuthology.orchestra.run.smithi104.stdout: "rocksdb_perf": "false", 2020-09-08T15:37:14.379 INFO:teuthology.orchestra.run.smithi104.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2020-09-08T15:37:14.379 INFO:teuthology.orchestra.run.smithi104.stdout: "rotating_keys_bootstrap_timeout": "30", 2020-09-08T15:37:14.379 INFO:teuthology.orchestra.run.smithi104.stdout: "rotating_keys_renewal_timeout": "10", 2020-09-08T15:37:14.380 INFO:teuthology.orchestra.run.smithi104.stdout: "run_dir": "/var/run/ceph", 2020-09-08T15:37:14.380 INFO:teuthology.orchestra.run.smithi104.stdout: "setgroup": "ceph", 2020-09-08T15:37:14.380 INFO:teuthology.orchestra.run.smithi104.stdout: "setuser": "ceph", 2020-09-08T15:37:14.380 INFO:teuthology.orchestra.run.smithi104.stdout: "setuser_match_path": "", 2020-09-08T15:37:14.380 INFO:teuthology.orchestra.run.smithi104.stdout: "target_max_misplaced_ratio": "0.050000", 2020-09-08T15:37:14.380 INFO:teuthology.orchestra.run.smithi104.stdout: "thp": "false", 2020-09-08T15:37:14.381 INFO:teuthology.orchestra.run.smithi104.stdout: "threadpool_default_timeout": "60", 2020-09-08T15:37:14.381 INFO:teuthology.orchestra.run.smithi104.stdout: "threadpool_empty_queue_max_wait": "2", 2020-09-08T15:37:14.381 INFO:teuthology.orchestra.run.smithi104.stdout: "throttler_perf_counter": "true" 2020-09-08T15:37:14.381 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:37:14.382 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:13 smithi055 bash[9296]: cluster 2020-09-08T15:37:12.681117+0000 mon.a (mon.0) 898 : cluster [DBG] osdmap e122: 12 total, 12 up, 12 in 2020-09-08T15:37:14.383 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:13 smithi055 bash[9296]: cluster 2020-09-08T15:37:13.582055+0000 mgr.x (mgr.24118) 291 : cluster [DBG] pgmap v335: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:14.383 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:13 smithi166 bash[9446]: cluster 2020-09-08T15:37:12.681117+0000 mon.a (mon.0) 898 : cluster [DBG] osdmap e122: 12 total, 12 up, 12 in 2020-09-08T15:37:14.383 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:13 smithi166 bash[9446]: cluster 2020-09-08T15:37:13.582055+0000 mgr.x (mgr.24118) 291 : cluster [DBG] pgmap v335: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:14.383 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:13 smithi104 bash[9203]: cluster 2020-09-08T15:37:12.681117+0000 mon.a (mon.0) 898 : cluster [DBG] osdmap e122: 12 total, 12 up, 12 in 2020-09-08T15:37:14.384 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:13 smithi104 bash[9203]: cluster 2020-09-08T15:37:13.582055+0000 mgr.x (mgr.24118) 291 : cluster [DBG] pgmap v335: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:14.739 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2020-09-08T15:37:14.884 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:37:15.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:15 smithi104 bash[9203]: cluster 2020-09-08T15:37:15.582724+0000 mgr.x (mgr.24118) 292 : cluster [DBG] pgmap v336: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:16.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:15 smithi166 bash[9446]: cluster 2020-09-08T15:37:15.582724+0000 mgr.x (mgr.24118) 292 : cluster [DBG] pgmap v336: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:16.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:15 smithi055 bash[9296]: cluster 2020-09-08T15:37:15.582724+0000 mgr.x (mgr.24118) 292 : cluster [DBG] pgmap v336: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:17.809 INFO:teuthology.orchestra.run.smithi104.stdout:mon.a: {} 2020-09-08T15:37:17.826 INFO:teuthology.orchestra.run.smithi104.stdout:mon.b: {} 2020-09-08T15:37:17.843 INFO:teuthology.orchestra.run.smithi104.stdout:mon.c: {} 2020-09-08T15:37:17.845 INFO:teuthology.orchestra.run.smithi104.stderr:mon.a: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2020-09-08T15:37:17.846 INFO:teuthology.orchestra.run.smithi104.stderr:mon.b: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2020-09-08T15:37:17.846 INFO:teuthology.orchestra.run.smithi104.stderr:mon.c: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2020-09-08T15:37:17.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:17 smithi104 bash[9203]: cluster 2020-09-08T15:37:16.590119+0000 osd.10 (osd.10) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:17.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:17 smithi104 bash[9203]: cluster 2020-09-08T15:37:16.590282+0000 osd.10 (osd.10) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:17.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:17 smithi104 bash[9203]: cluster 2020-09-08T15:37:16.675563+0000 mon.a (mon.0) 899 : cluster [DBG] osdmap e123: 12 total, 12 up, 12 in 2020-09-08T15:37:17.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:17 smithi104 bash[9203]: cluster 2020-09-08T15:37:17.583331+0000 mgr.x (mgr.24118) 293 : cluster [DBG] pgmap v338: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:18.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:17 smithi166 bash[9446]: cluster 2020-09-08T15:37:16.590119+0000 osd.10 (osd.10) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:18.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:17 smithi166 bash[9446]: cluster 2020-09-08T15:37:16.590282+0000 osd.10 (osd.10) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:18.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:17 smithi166 bash[9446]: cluster 2020-09-08T15:37:16.675563+0000 mon.a (mon.0) 899 : cluster [DBG] osdmap e123: 12 total, 12 up, 12 in 2020-09-08T15:37:18.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:17 smithi166 bash[9446]: cluster 2020-09-08T15:37:17.583331+0000 mgr.x (mgr.24118) 293 : cluster [DBG] pgmap v338: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:18.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:17 smithi055 bash[9296]: cluster 2020-09-08T15:37:16.590119+0000 osd.10 (osd.10) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:18.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:17 smithi055 bash[9296]: cluster 2020-09-08T15:37:16.590282+0000 osd.10 (osd.10) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:18.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:17 smithi055 bash[9296]: cluster 2020-09-08T15:37:16.675563+0000 mon.a (mon.0) 899 : cluster [DBG] osdmap e123: 12 total, 12 up, 12 in 2020-09-08T15:37:18.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:17 smithi055 bash[9296]: cluster 2020-09-08T15:37:17.583331+0000 mgr.x (mgr.24118) 293 : cluster [DBG] pgmap v338: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:18.537 INFO:teuthology.run_tasks:Running task full_sequential... 2020-09-08T15:37:18.551 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2020-09-08T15:37:18.556 INFO:tasks.radosbench:Beginning radosbench... 2020-09-08T15:37:18.557 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_0 2020-09-08T15:37:18.557 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool create unique_pool_0 16 2020-09-08T15:37:18.558 INFO:tasks.thrashosds.thrasher:starting do_thrash 2020-09-08T15:37:18.558 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2020-09-08T15:37:18.559 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-09-08T15:37:18.559 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd dump -f json-pretty 2020-09-08T15:37:18.560 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2020-09-08T15:37:18.560 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:18.561 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2020-09-08T15:37:18.561 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:37:18.562 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2020-09-08T15:37:18.562 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2020-09-08T15:37:18.563 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2020-09-08T15:37:18.563 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:37:18.685 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:18.699 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:37:18.700 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:37:18.700 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:37:18.750 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:37:18.764 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:37:18.787 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:37:18.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:18 smithi055 bash[9296]: audit 2020-09-08T15:37:17.793618+0000 mon.a (mon.0) 900 : audit [DBG] from='client.? v1:172.21.15.104:0/1072574097' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2020-09-08T15:37:18.908 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:37:18 smithi055 bash[19003]: debug 2020-09-08T15:37:18.683+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:18.923 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:37:18.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:18 smithi104 bash[9203]: audit 2020-09-08T15:37:17.793618+0000 mon.a (mon.0) 900 : audit [DBG] from='client.? v1:172.21.15.104:0/1072574097' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2020-09-08T15:37:19.030 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:37:19.040 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:18 smithi166 bash[9446]: audit 2020-09-08T15:37:17.793618+0000 mon.a (mon.0) 900 : audit [DBG] from='client.? v1:172.21.15.104:0/1072574097' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2020-09-08T15:37:19.197 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:37:19.302 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:37:19.315 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:37:19 smithi166 bash[11928]: debug 2020-09-08T15:37:19.197+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:19.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:37:18 smithi055 bash[11956]: debug 2020-09-08T15:37:18.923+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:19.421 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:37:19.526 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:19.615 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:37:19 smithi166 bash[14254]: debug 2020-09-08T15:37:19.421+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:19.655 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:19.763 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:19.971 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:20.080 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:20.093 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:37:19 smithi166 bash[18960]: debug 2020-09-08T15:37:19.657+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:20.182 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:37:19 smithi104 bash[19922]: debug 2020-09-08T15:37:19.970+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:20.214 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:20.319 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:20.365 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:37:20 smithi166 bash[18960]: debug 2020-09-08T15:37:20.213+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:20.504 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:20.610 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:20.679 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:37:20 smithi104 bash[19922]: debug 2020-09-08T15:37:20.506+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:20.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:20 smithi104 bash[9203]: cluster 2020-09-08T15:37:19.565118+0000 osd.3 (osd.3) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:20.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:20 smithi104 bash[9203]: cluster 2020-09-08T15:37:19.565238+0000 osd.3 (osd.3) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:20.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:20 smithi104 bash[9203]: cluster 2020-09-08T15:37:19.584050+0000 mgr.x (mgr.24118) 294 : cluster [DBG] pgmap v339: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:20.732 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:20.842 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:37:20.856 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:20 smithi055 bash[9296]: cluster 2020-09-08T15:37:19.565118+0000 osd.3 (osd.3) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:20.857 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:20 smithi055 bash[9296]: cluster 2020-09-08T15:37:19.565238+0000 osd.3 (osd.3) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:20.857 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:20 smithi055 bash[9296]: cluster 2020-09-08T15:37:19.584050+0000 mgr.x (mgr.24118) 294 : cluster [DBG] pgmap v339: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:20.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:20 smithi166 bash[9446]: cluster 2020-09-08T15:37:19.565118+0000 osd.3 (osd.3) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:20.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:20 smithi166 bash[9446]: cluster 2020-09-08T15:37:19.565238+0000 osd.3 (osd.3) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:20.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:20 smithi166 bash[9446]: cluster 2020-09-08T15:37:19.584050+0000 mgr.x (mgr.24118) 294 : cluster [DBG] pgmap v339: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:20.865 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:37:20 smithi166 bash[18960]: debug 2020-09-08T15:37:20.733+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:20.970 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:37:21.072 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:37:21.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:37:20 smithi055 bash[11956]: debug 2020-09-08T15:37:20.967+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:21.184 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:37:21.290 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:37:21.443 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:37:21.548 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:37:21.555 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:37:21 smithi055 bash[14301]: debug 2020-09-08T15:37:21.183+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:21.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:21 smithi166 bash[9446]: cluster 2020-09-08T15:37:20.562472+0000 mon.a (mon.0) 901 : cluster [DBG] osdmap e124: 12 total, 12 up, 12 in 2020-09-08T15:37:21.622 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:37:21 smithi166 bash[11928]: debug 2020-09-08T15:37:21.445+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:21.657 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:21 smithi104 bash[9203]: cluster 2020-09-08T15:37:20.562472+0000 mon.a (mon.0) 901 : cluster [DBG] osdmap e124: 12 total, 12 up, 12 in 2020-09-08T15:37:21.675 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:37:21.780 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:37:21.916 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:37:21.917 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:21 smithi055 bash[9296]: cluster 2020-09-08T15:37:20.562472+0000 mon.a (mon.0) 901 : cluster [DBG] osdmap e124: 12 total, 12 up, 12 in 2020-09-08T15:37:21.930 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:37:21 smithi055 bash[11956]: debug 2020-09-08T15:37:21.675+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:22.032 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:37:22.115 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:37:21 smithi166 bash[11928]: debug 2020-09-08T15:37:21.917+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:22.165 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:37:22.271 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:37:22.402 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:37:22 smithi055 bash[16646]: debug 2020-09-08T15:37:22.163+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:22.403 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:37:22.507 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:37:22.634 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:37:22.660 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:22 smithi055 bash[9296]: cluster 2020-09-08T15:37:21.584756+0000 mgr.x (mgr.24118) 295 : cluster [DBG] pgmap v341: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:22.661 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:37:22 smithi055 bash[11956]: debug 2020-09-08T15:37:22.403+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:22.661 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:37:22 smithi055 bash[11956]: debug 2020-09-08T15:37:22.635+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:22.738 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:37:22.752 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:22 smithi104 bash[9203]: cluster 2020-09-08T15:37:21.584756+0000 mgr.x (mgr.24118) 295 : cluster [DBG] pgmap v341: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:22.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:22 smithi166 bash[9446]: cluster 2020-09-08T15:37:21.584756+0000 mgr.x (mgr.24118) 295 : cluster [DBG] pgmap v341: 1 pgs: 1 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:23.009 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:37:23.114 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:37:23.237 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:37:23.341 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:23.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:37:23 smithi055 bash[16646]: debug 2020-09-08T15:37:23.234+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:23.427 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:37:23 smithi104 bash[17588]: debug 2020-09-08T15:37:23.010+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:23.460 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:23.569 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:23.708 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:23.712 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:23 smithi055 bash[9296]: cluster 2020-09-08T15:37:23.585595+0000 mgr.x (mgr.24118) 296 : cluster [DBG] pgmap v342: 1 pgs: 1 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:23.712 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:37:23 smithi055 bash[19003]: debug 2020-09-08T15:37:23.458+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:23.712 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:37:23 smithi055 bash[19003]: debug 2020-09-08T15:37:23.706+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:23.820 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:37:23.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:23 smithi104 bash[9203]: cluster 2020-09-08T15:37:23.585595+0000 mgr.x (mgr.24118) 296 : cluster [DBG] pgmap v342: 1 pgs: 1 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:24.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:23 smithi166 bash[9446]: cluster 2020-09-08T15:37:23.585595+0000 mgr.x (mgr.24118) 296 : cluster [DBG] pgmap v342: 1 pgs: 1 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:24.177 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:37:24.281 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:24.424 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:24.427 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:37:24 smithi104 bash[17588]: debug 2020-09-08T15:37:24.178+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:24.528 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:24.637 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:37:24 smithi166 bash[18960]: debug 2020-09-08T15:37:24.425+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:24.842 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:24.943 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:37:25.177 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:37:24 smithi104 bash[19922]: debug 2020-09-08T15:37:24.842+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:25.325 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:37:25.426 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:37:25.575 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:37:25.663 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:37:25 smithi104 bash[17588]: debug 2020-09-08T15:37:25.322+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:25.680 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:25.822 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:25.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:25 smithi166 bash[9446]: cluster 2020-09-08T15:37:25.586406+0000 mgr.x (mgr.24118) 297 : cluster [DBG] pgmap v343: 1 pgs: 1 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:25.865 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:37:25 smithi166 bash[16593]: debug 2020-09-08T15:37:25.569+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:25.927 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:37:25.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:25 smithi104 bash[9203]: cluster 2020-09-08T15:37:25.586406+0000 mgr.x (mgr.24118) 297 : cluster [DBG] pgmap v343: 1 pgs: 1 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:26.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:25 smithi055 bash[9296]: cluster 2020-09-08T15:37:25.586406+0000 mgr.x (mgr.24118) 297 : cluster [DBG] pgmap v343: 1 pgs: 1 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:26.158 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:37:25 smithi055 bash[19003]: debug 2020-09-08T15:37:25.822+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:26.311 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:37:26.413 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:26.614 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:37:26 smithi104 bash[15273]: debug 2020-09-08T15:37:26.298+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:26.615 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:26.719 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:37:26.858 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:37:26.927 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:37:26 smithi104 bash[19922]: debug 2020-09-08T15:37:26.614+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:26.961 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:37:27.088 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:37:27.114 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:37:26 smithi166 bash[14254]: debug 2020-09-08T15:37:26.857+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:27.193 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:37:27.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:37:27 smithi055 bash[16646]: debug 2020-09-08T15:37:27.086+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:27.408 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:37:27.500 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:37:27 smithi104 bash[15273]: debug 2020-09-08T15:37:27.410+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:27.513 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:37:27.646 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:37:27.715 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:37:27.752 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:27.756 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:37:27.762 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:27 smithi104 bash[9203]: cluster 2020-09-08T15:37:26.118658+0000 osd.8 (osd.8) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:27.762 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:27 smithi104 bash[9203]: cluster 2020-09-08T15:37:26.118794+0000 osd.8 (osd.8) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:27.763 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:27 smithi104 bash[9203]: cluster 2020-09-08T15:37:26.675959+0000 mon.a (mon.0) 902 : cluster [DBG] osdmap e125: 12 total, 12 up, 12 in 2020-09-08T15:37:27.763 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:27 smithi104 bash[9203]: cluster 2020-09-08T15:37:27.587101+0000 mgr.x (mgr.24118) 298 : cluster [DBG] pgmap v345: 1 pgs: 1 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:27.790 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:37:27.791 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:37:27.798 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:37:27.836 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:37:27.865 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:37:27 smithi166 bash[11928]: debug 2020-09-08T15:37:27.645+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:27.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:27 smithi166 bash[9446]: cluster 2020-09-08T15:37:26.118658+0000 osd.8 (osd.8) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:27.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:27 smithi166 bash[9446]: cluster 2020-09-08T15:37:26.118794+0000 osd.8 (osd.8) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:27.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:27 smithi166 bash[9446]: cluster 2020-09-08T15:37:26.675959+0000 mon.a (mon.0) 902 : cluster [DBG] osdmap e125: 12 total, 12 up, 12 in 2020-09-08T15:37:27.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:27 smithi166 bash[9446]: cluster 2020-09-08T15:37:27.587101+0000 mgr.x (mgr.24118) 298 : cluster [DBG] pgmap v345: 1 pgs: 1 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:27.869 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:37:27.900 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:37:27.928 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:27.932 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:37:27.959 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:37:27.995 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:37:28.019 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:37:27 smithi104 bash[19922]: debug 2020-09-08T15:37:27.926+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:28.024 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:37:28.034 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:37:28.046 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:27 smithi055 bash[9296]: cluster 2020-09-08T15:37:26.118658+0000 osd.8 (osd.8) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:28.046 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:27 smithi055 bash[9296]: cluster 2020-09-08T15:37:26.118794+0000 osd.8 (osd.8) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:28.046 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:27 smithi055 bash[9296]: cluster 2020-09-08T15:37:26.675959+0000 mon.a (mon.0) 902 : cluster [DBG] osdmap e125: 12 total, 12 up, 12 in 2020-09-08T15:37:28.047 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:27 smithi055 bash[9296]: cluster 2020-09-08T15:37:27.587101+0000 mgr.x (mgr.24118) 298 : cluster [DBG] pgmap v345: 1 pgs: 1 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:28.055 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:37:28.083 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:37:28.085 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:37:28.086 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:37:28.086 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:37:28.087 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:37:28.087 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:37:28.087 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:37:28.088 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:37:28.088 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:37:28.088 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:37:28.089 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:37:28.089 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:37:28.089 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:37:28.176 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:37:28.281 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:37:28.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:37:28 smithi055 bash[14301]: debug 2020-09-08T15:37:28.174+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:28.694 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:37:28.797 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:37:28.858 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T15:37:28.859 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:37:28.859 INFO:teuthology.orchestra.run.smithi104.stdout: "epoch": 125, 2020-09-08T15:37:28.860 INFO:teuthology.orchestra.run.smithi104.stdout: "fsid": "759b27b6-f1e7-11ea-a080-001a4aab830c", 2020-09-08T15:37:28.860 INFO:teuthology.orchestra.run.smithi104.stdout: "created": "2020-09-08T15:25:41.788365+0000", 2020-09-08T15:37:28.860 INFO:teuthology.orchestra.run.smithi104.stdout: "modified": "2020-09-08T15:37:26.668179+0000", 2020-09-08T15:37:28.860 INFO:teuthology.orchestra.run.smithi104.stdout: "last_up_change": "2020-09-08T15:34:41.591462+0000", 2020-09-08T15:37:28.861 INFO:teuthology.orchestra.run.smithi104.stdout: "last_in_change": "2020-09-08T15:34:41.591462+0000", 2020-09-08T15:37:28.861 INFO:teuthology.orchestra.run.smithi104.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-09-08T15:37:28.861 INFO:teuthology.orchestra.run.smithi104.stdout: "flags_num": 5799936, 2020-09-08T15:37:28.861 INFO:teuthology.orchestra.run.smithi104.stdout: "flags_set": [ 2020-09-08T15:37:28.861 INFO:teuthology.orchestra.run.smithi104.stdout: "pglog_hardlimit", 2020-09-08T15:37:28.862 INFO:teuthology.orchestra.run.smithi104.stdout: "purged_snapdirs", 2020-09-08T15:37:28.862 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_deletes", 2020-09-08T15:37:28.862 INFO:teuthology.orchestra.run.smithi104.stdout: "sortbitwise" 2020-09-08T15:37:28.862 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T15:37:28.862 INFO:teuthology.orchestra.run.smithi104.stdout: "crush_version": 27, 2020-09-08T15:37:28.863 INFO:teuthology.orchestra.run.smithi104.stdout: "full_ratio": 0.94999998807907104, 2020-09-08T15:37:28.863 INFO:teuthology.orchestra.run.smithi104.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-09-08T15:37:28.863 INFO:teuthology.orchestra.run.smithi104.stdout: "nearfull_ratio": 0.85000002384185791, 2020-09-08T15:37:28.863 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_snapshot": "", 2020-09-08T15:37:28.863 INFO:teuthology.orchestra.run.smithi104.stdout: "pool_max": 1, 2020-09-08T15:37:28.863 INFO:teuthology.orchestra.run.smithi104.stdout: "max_osd": 12, 2020-09-08T15:37:28.864 INFO:teuthology.orchestra.run.smithi104.stdout: "require_min_compat_client": "luminous", 2020-09-08T15:37:28.864 INFO:teuthology.orchestra.run.smithi104.stdout: "min_compat_client": "jewel", 2020-09-08T15:37:28.864 INFO:teuthology.orchestra.run.smithi104.stdout: "require_osd_release": "pacific", 2020-09-08T15:37:28.864 INFO:teuthology.orchestra.run.smithi104.stdout: "pools": [ 2020-09-08T15:37:28.864 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.865 INFO:teuthology.orchestra.run.smithi104.stdout: "pool": 1, 2020-09-08T15:37:28.865 INFO:teuthology.orchestra.run.smithi104.stdout: "pool_name": "device_health_metrics", 2020-09-08T15:37:28.865 INFO:teuthology.orchestra.run.smithi104.stdout: "create_time": "2020-09-08T15:25:56.314385+0000", 2020-09-08T15:37:28.865 INFO:teuthology.orchestra.run.smithi104.stdout: "flags": 1, 2020-09-08T15:37:28.865 INFO:teuthology.orchestra.run.smithi104.stdout: "flags_names": "hashpspool", 2020-09-08T15:37:28.866 INFO:teuthology.orchestra.run.smithi104.stdout: "type": 1, 2020-09-08T15:37:28.866 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 2, 2020-09-08T15:37:28.866 INFO:teuthology.orchestra.run.smithi104.stdout: "min_size": 2, 2020-09-08T15:37:28.866 INFO:teuthology.orchestra.run.smithi104.stdout: "crush_rule": 0, 2020-09-08T15:37:28.866 INFO:teuthology.orchestra.run.smithi104.stdout: "object_hash": 2, 2020-09-08T15:37:28.867 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_autoscale_mode": "off", 2020-09-08T15:37:28.867 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num": 1, 2020-09-08T15:37:28.867 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_placement_num": 1, 2020-09-08T15:37:28.867 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_placement_num_target": 1, 2020-09-08T15:37:28.867 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num_target": 1, 2020-09-08T15:37:28.868 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num_pending": 1, 2020-09-08T15:37:28.868 INFO:teuthology.orchestra.run.smithi104.stdout: "last_pg_merge_meta": { 2020-09-08T15:37:28.868 INFO:teuthology.orchestra.run.smithi104.stdout: "source_pgid": "0.0", 2020-09-08T15:37:28.868 INFO:teuthology.orchestra.run.smithi104.stdout: "ready_epoch": 0, 2020-09-08T15:37:28.868 INFO:teuthology.orchestra.run.smithi104.stdout: "last_epoch_started": 0, 2020-09-08T15:37:28.869 INFO:teuthology.orchestra.run.smithi104.stdout: "last_epoch_clean": 0, 2020-09-08T15:37:28.869 INFO:teuthology.orchestra.run.smithi104.stdout: "source_version": "0'0", 2020-09-08T15:37:28.869 INFO:teuthology.orchestra.run.smithi104.stdout: "target_version": "0'0" 2020-09-08T15:37:28.871 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.871 INFO:teuthology.orchestra.run.smithi104.stdout: "last_change": "13", 2020-09-08T15:37:28.871 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend": "0", 2020-09-08T15:37:28.871 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend_prenautilus": "0", 2020-09-08T15:37:28.872 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend_preluminous": "0", 2020-09-08T15:37:28.872 INFO:teuthology.orchestra.run.smithi104.stdout: "auid": 0, 2020-09-08T15:37:28.872 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_mode": "selfmanaged", 2020-09-08T15:37:28.872 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_seq": 0, 2020-09-08T15:37:28.872 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_epoch": 0, 2020-09-08T15:37:28.873 INFO:teuthology.orchestra.run.smithi104.stdout: "pool_snaps": [], 2020-09-08T15:37:28.873 INFO:teuthology.orchestra.run.smithi104.stdout: "removed_snaps": "[]", 2020-09-08T15:37:28.873 INFO:teuthology.orchestra.run.smithi104.stdout: "quota_max_bytes": 0, 2020-09-08T15:37:28.873 INFO:teuthology.orchestra.run.smithi104.stdout: "quota_max_objects": 0, 2020-09-08T15:37:28.873 INFO:teuthology.orchestra.run.smithi104.stdout: "tiers": [], 2020-09-08T15:37:28.874 INFO:teuthology.orchestra.run.smithi104.stdout: "tier_of": -1, 2020-09-08T15:37:28.874 INFO:teuthology.orchestra.run.smithi104.stdout: "read_tier": -1, 2020-09-08T15:37:28.874 INFO:teuthology.orchestra.run.smithi104.stdout: "write_tier": -1, 2020-09-08T15:37:28.874 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_mode": "none", 2020-09-08T15:37:28.874 INFO:teuthology.orchestra.run.smithi104.stdout: "target_max_bytes": 0, 2020-09-08T15:37:28.875 INFO:teuthology.orchestra.run.smithi104.stdout: "target_max_objects": 0, 2020-09-08T15:37:28.875 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-09-08T15:37:28.875 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-09-08T15:37:28.876 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_full_ratio_micro": 800000, 2020-09-08T15:37:28.876 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_min_flush_age": 0, 2020-09-08T15:37:28.876 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_min_evict_age": 0, 2020-09-08T15:37:28.876 INFO:teuthology.orchestra.run.smithi104.stdout: "erasure_code_profile": "", 2020-09-08T15:37:28.876 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_params": { 2020-09-08T15:37:28.876 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "none" 2020-09-08T15:37:28.877 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.877 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_period": 0, 2020-09-08T15:37:28.877 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_count": 0, 2020-09-08T15:37:28.877 INFO:teuthology.orchestra.run.smithi104.stdout: "use_gmt_hitset": true, 2020-09-08T15:37:28.877 INFO:teuthology.orchestra.run.smithi104.stdout: "min_read_recency_for_promote": 0, 2020-09-08T15:37:28.878 INFO:teuthology.orchestra.run.smithi104.stdout: "min_write_recency_for_promote": 0, 2020-09-08T15:37:28.878 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_grade_decay_rate": 0, 2020-09-08T15:37:28.878 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_search_last_n": 0, 2020-09-08T15:37:28.878 INFO:teuthology.orchestra.run.smithi104.stdout: "grade_table": [], 2020-09-08T15:37:28.878 INFO:teuthology.orchestra.run.smithi104.stdout: "stripe_width": 0, 2020-09-08T15:37:28.879 INFO:teuthology.orchestra.run.smithi104.stdout: "expected_num_objects": 0, 2020-09-08T15:37:28.879 INFO:teuthology.orchestra.run.smithi104.stdout: "fast_read": false, 2020-09-08T15:37:28.879 INFO:teuthology.orchestra.run.smithi104.stdout: "options": { 2020-09-08T15:37:28.879 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num_min": 1 2020-09-08T15:37:28.879 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.880 INFO:teuthology.orchestra.run.smithi104.stdout: "application_metadata": { 2020-09-08T15:37:28.881 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_devicehealth": {} 2020-09-08T15:37:28.881 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.881 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.881 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T15:37:28.881 INFO:teuthology.orchestra.run.smithi104.stdout: "osds": [ 2020-09-08T15:37:28.882 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.882 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 0, 2020-09-08T15:37:28.882 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "e49a7341-7c75-4883-b061-3e8fe9467e6e", 2020-09-08T15:37:28.882 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:37:28.882 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:37:28.882 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:37:28.883 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:37:28.883 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:37:28.883 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:37:28.883 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 11, 2020-09-08T15:37:28.883 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 55, 2020-09-08T15:37:28.884 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:37:28.884 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:37:28.884 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:37:28.884 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.884 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.885 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.885 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6801", 2020-09-08T15:37:28.885 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:37:28.885 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.885 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.886 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.886 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:37:28.886 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.886 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.886 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.886 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6802", 2020-09-08T15:37:28.887 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:37:28.887 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.887 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.887 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.887 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6803", 2020-09-08T15:37:28.888 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:37:28.888 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.888 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.888 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.888 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:37:28.889 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.889 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.889 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.889 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6806", 2020-09-08T15:37:28.889 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:37:28.890 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.890 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.890 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.891 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6807", 2020-09-08T15:37:28.891 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:37:28.891 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.891 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.891 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.892 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:37:28.892 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.892 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.892 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.892 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6804", 2020-09-08T15:37:28.893 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:37:28.893 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.893 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.893 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.893 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6805", 2020-09-08T15:37:28.894 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:37:28.894 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.894 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.894 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.895 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.104:6801/82470352", 2020-09-08T15:37:28.895 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.104:6803/82470352", 2020-09-08T15:37:28.895 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.104:6807/82470352", 2020-09-08T15:37:28.895 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.104:6805/82470352", 2020-09-08T15:37:28.895 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:37:28.895 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:37:28.896 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:37:28.896 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.896 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.896 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.896 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 1, 2020-09-08T15:37:28.897 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "a83df412-110d-440a-b411-ad2a7c36f81a", 2020-09-08T15:37:28.897 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:37:28.897 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:37:28.897 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:37:28.897 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:37:28.898 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:37:28.898 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:37:28.898 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 17, 2020-09-08T15:37:28.898 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 18, 2020-09-08T15:37:28.898 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:37:28.898 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:37:28.899 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:37:28.899 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.899 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.899 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.899 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6809", 2020-09-08T15:37:28.900 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:37:28.900 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.900 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.900 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.900 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:37:28.902 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.902 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.902 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.902 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6810", 2020-09-08T15:37:28.903 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:37:28.903 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.903 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.903 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.903 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6811", 2020-09-08T15:37:28.904 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:37:28.904 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.904 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.904 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.904 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:37:28.905 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.905 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.905 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.905 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6814", 2020-09-08T15:37:28.905 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:37:28.906 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.906 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.906 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.906 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6815", 2020-09-08T15:37:28.907 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:37:28.907 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.907 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.907 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.908 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:37:28.908 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.908 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.909 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.909 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6812", 2020-09-08T15:37:28.909 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:37:28.909 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.909 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.910 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.910 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6813", 2020-09-08T15:37:28.910 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:37:28.910 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.910 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.911 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.911 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.104:6809/1421798992", 2020-09-08T15:37:28.912 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.104:6811/1421798992", 2020-09-08T15:37:28.912 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.104:6815/1421798992", 2020-09-08T15:37:28.912 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.104:6813/1421798992", 2020-09-08T15:37:28.912 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:37:28.912 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:37:28.913 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:37:28.913 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.913 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.913 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.913 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 2, 2020-09-08T15:37:28.914 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "ca6da1cc-1d8d-494b-93e6-e065ff70d434", 2020-09-08T15:37:28.914 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:37:28.914 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:37:28.914 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:37:28.914 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:37:28.915 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:37:28.915 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:37:28.915 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 23, 2020-09-08T15:37:28.915 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 0, 2020-09-08T15:37:28.915 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:37:28.916 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:37:28.916 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:37:28.916 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.916 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.916 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.916 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6817", 2020-09-08T15:37:28.917 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:37:28.917 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.917 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.917 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.917 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:37:28.919 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.919 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.919 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.919 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6818", 2020-09-08T15:37:28.919 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:37:28.920 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.920 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.920 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.920 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6819", 2020-09-08T15:37:28.920 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:37:28.921 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.921 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.921 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.922 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:37:28.922 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.925 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.925 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.925 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6822", 2020-09-08T15:37:28.925 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:37:28.925 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.926 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.926 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.926 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6823", 2020-09-08T15:37:28.926 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:37:28.926 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.927 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.927 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.927 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:37:28.928 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.928 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.928 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.928 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6820", 2020-09-08T15:37:28.928 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:37:28.929 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.929 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.929 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.929 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6821", 2020-09-08T15:37:28.929 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:37:28.930 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.930 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.930 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.930 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.104:6817/2713413369", 2020-09-08T15:37:28.930 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.104:6819/2713413369", 2020-09-08T15:37:28.931 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.104:6823/2713413369", 2020-09-08T15:37:28.931 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.104:6821/2713413369", 2020-09-08T15:37:28.931 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:37:28.931 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:37:28.932 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:37:28.932 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.932 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.933 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.933 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 3, 2020-09-08T15:37:28.933 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e", 2020-09-08T15:37:28.933 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:37:28.933 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:37:28.934 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:37:28.934 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:37:28.934 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:37:28.934 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:37:28.934 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 29, 2020-09-08T15:37:28.935 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 0, 2020-09-08T15:37:28.935 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:37:28.935 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:37:28.935 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:37:28.935 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.936 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.936 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.936 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6825", 2020-09-08T15:37:28.936 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:37:28.936 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.937 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.937 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.937 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:37:28.937 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.937 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.937 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.938 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6826", 2020-09-08T15:37:28.938 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:37:28.938 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.938 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.938 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.939 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6827", 2020-09-08T15:37:28.939 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:37:28.939 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.939 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.939 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.940 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:37:28.940 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.940 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.940 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.940 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6830", 2020-09-08T15:37:28.940 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:37:28.941 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.941 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.941 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.941 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6831", 2020-09-08T15:37:28.941 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:37:28.942 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.942 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.942 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.943 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:37:28.943 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.943 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.943 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.943 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6828", 2020-09-08T15:37:28.944 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:37:28.944 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.944 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.944 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.944 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6829", 2020-09-08T15:37:28.945 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:37:28.945 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.945 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.945 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.945 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.104:6825/1495946248", 2020-09-08T15:37:28.946 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.104:6827/1495946248", 2020-09-08T15:37:28.946 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.104:6831/1495946248", 2020-09-08T15:37:28.946 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.104:6829/1495946248", 2020-09-08T15:37:28.946 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:37:28.946 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:37:28.947 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:37:28.947 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.947 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.947 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.947 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 4, 2020-09-08T15:37:28.947 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "d8334c70-f776-4652-944d-aed35a8e43f7", 2020-09-08T15:37:28.948 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:37:28.948 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:37:28.948 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:37:28.948 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:37:28.948 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:37:28.949 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:37:28.949 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 35, 2020-09-08T15:37:28.949 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 0, 2020-09-08T15:37:28.949 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:37:28.949 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:37:28.950 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:37:28.950 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.950 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.950 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.950 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6803", 2020-09-08T15:37:28.950 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:37:28.951 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.951 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.951 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.951 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:37:28.951 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.952 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.952 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.952 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6804", 2020-09-08T15:37:28.952 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:37:28.953 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.953 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.953 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.953 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6805", 2020-09-08T15:37:28.954 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:37:28.954 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.954 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.954 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.954 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:37:28.954 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.955 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.955 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.955 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6808", 2020-09-08T15:37:28.955 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:37:28.955 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.956 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.956 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.956 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6809", 2020-09-08T15:37:28.956 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:37:28.956 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.957 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.957 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.957 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:37:28.957 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.957 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.957 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.958 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6806", 2020-09-08T15:37:28.958 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:37:28.958 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.958 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.958 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.959 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6807", 2020-09-08T15:37:28.959 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:37:28.959 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.959 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.959 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.960 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.55:6803/1173103260", 2020-09-08T15:37:28.960 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.55:6805/1173103260", 2020-09-08T15:37:28.960 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.55:6809/1173103260", 2020-09-08T15:37:28.960 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.55:6807/1173103260", 2020-09-08T15:37:28.961 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:37:28.961 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:37:28.961 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:37:28.961 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.961 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.962 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.962 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 5, 2020-09-08T15:37:28.962 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "6789abd2-9e9b-4c04-81fb-0ce2d2849e39", 2020-09-08T15:37:28.962 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:37:28.962 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:37:28.963 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:37:28.963 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:37:28.963 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:37:28.964 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:37:28.964 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 42, 2020-09-08T15:37:28.964 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 42, 2020-09-08T15:37:28.964 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:37:28.964 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:37:28.964 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:37:28.965 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.965 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.965 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.965 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6811", 2020-09-08T15:37:28.965 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:37:28.966 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.966 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.966 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.966 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:37:28.966 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.967 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.967 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.967 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6812", 2020-09-08T15:37:28.967 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:37:28.967 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.968 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.968 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.968 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6813", 2020-09-08T15:37:28.968 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:37:28.968 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.968 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.969 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.969 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:37:28.969 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.969 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.969 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.970 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6816", 2020-09-08T15:37:28.970 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:37:28.970 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.970 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.970 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.971 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6817", 2020-09-08T15:37:28.971 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:37:28.971 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.971 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.971 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.971 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:37:28.972 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.972 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.972 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.972 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6814", 2020-09-08T15:37:28.972 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:37:28.973 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.973 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.973 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.974 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6815", 2020-09-08T15:37:28.974 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:37:28.974 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.974 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.974 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.975 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.55:6811/1047046376", 2020-09-08T15:37:28.975 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.55:6813/1047046376", 2020-09-08T15:37:28.975 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.55:6817/1047046376", 2020-09-08T15:37:28.975 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.55:6815/1047046376", 2020-09-08T15:37:28.975 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:37:28.976 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:37:28.976 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:37:28.976 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.976 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.977 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.977 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 6, 2020-09-08T15:37:28.977 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "ad9a93fe-0ca0-4052-bfa9-e2abb268829e", 2020-09-08T15:37:28.977 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:37:28.977 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:37:28.978 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:37:28.978 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:37:28.978 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:37:28.978 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:37:28.978 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 49, 2020-09-08T15:37:28.978 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 49, 2020-09-08T15:37:28.979 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:37:28.979 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:37:28.979 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:37:28.979 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.979 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.980 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.980 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6819", 2020-09-08T15:37:28.980 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:37:28.980 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.980 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.981 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.981 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:37:28.981 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.981 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.981 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.981 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6820", 2020-09-08T15:37:28.982 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:37:28.982 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.982 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.982 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.982 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6821", 2020-09-08T15:37:28.983 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:37:28.983 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.983 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.983 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.983 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:37:28.984 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.984 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.984 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.985 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6824", 2020-09-08T15:37:28.985 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:37:28.985 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.985 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.985 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.986 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6825", 2020-09-08T15:37:28.986 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:37:28.986 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.986 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.986 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.986 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:37:28.987 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.987 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.987 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.987 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6822", 2020-09-08T15:37:28.987 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:37:28.988 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.988 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.988 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.988 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6823", 2020-09-08T15:37:28.988 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:37:28.989 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.989 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.989 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.989 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.55:6819/2501844793", 2020-09-08T15:37:28.989 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.55:6821/2501844793", 2020-09-08T15:37:28.990 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.55:6825/2501844793", 2020-09-08T15:37:28.990 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.55:6823/2501844793", 2020-09-08T15:37:28.990 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:37:28.990 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:37:28.990 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:37:28.991 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.991 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.991 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.991 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 7, 2020-09-08T15:37:28.991 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "f3795c86-4540-442c-a151-b2769fa577e4", 2020-09-08T15:37:28.991 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:37:28.992 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:37:28.992 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:37:28.992 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:37:28.992 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:37:28.992 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:37:28.993 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 56, 2020-09-08T15:37:28.993 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 90, 2020-09-08T15:37:28.993 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:37:28.993 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:37:28.993 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:37:28.994 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.994 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.994 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.994 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6827", 2020-09-08T15:37:28.995 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:37:28.995 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.995 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.995 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.995 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:37:28.996 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.996 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.996 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.996 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6828", 2020-09-08T15:37:28.996 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:37:28.997 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.997 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.997 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:28.997 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6829", 2020-09-08T15:37:28.997 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:37:28.998 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:28.998 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:28.998 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.998 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:37:28.998 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:28.999 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:28.999 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:28.999 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6832", 2020-09-08T15:37:28.999 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:37:28.999 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:28.999 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.000 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:29.000 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6833", 2020-09-08T15:37:29.000 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:37:29.000 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.000 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.001 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.001 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:37:29.001 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:29.001 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.001 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:29.002 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6830", 2020-09-08T15:37:29.002 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:37:29.002 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.003 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.003 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:29.003 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6831", 2020-09-08T15:37:29.003 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:37:29.004 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.004 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.004 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.005 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.55:6827/128909153", 2020-09-08T15:37:29.005 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.55:6829/128909153", 2020-09-08T15:37:29.005 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.55:6833/128909153", 2020-09-08T15:37:29.005 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.55:6831/128909153", 2020-09-08T15:37:29.005 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:37:29.006 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:37:29.013 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:37:29.013 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.013 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.013 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.014 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 8, 2020-09-08T15:37:29.014 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "3b0ac380-878e-4c1c-9278-80ce67ca2df4", 2020-09-08T15:37:29.014 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:37:29.014 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:37:29.014 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:37:29.015 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:37:29.015 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:37:29.015 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:37:29.015 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 64, 2020-09-08T15:37:29.015 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 0, 2020-09-08T15:37:29.016 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:37:29.016 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:37:29.016 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:37:29.016 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:29.016 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.017 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:29.017 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6801", 2020-09-08T15:37:29.017 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:37:29.017 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.017 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.018 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.018 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:37:29.018 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:29.018 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.018 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:29.019 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6802", 2020-09-08T15:37:29.019 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:37:29.019 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.019 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.019 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:29.020 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6803", 2020-09-08T15:37:29.020 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:37:29.020 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.020 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.020 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.021 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:37:29.021 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:29.021 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.021 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:29.022 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6806", 2020-09-08T15:37:29.022 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:37:29.022 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.022 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.022 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:29.023 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6807", 2020-09-08T15:37:29.024 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:37:29.024 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.024 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.024 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.024 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:37:29.025 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:29.025 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.025 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:29.025 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6804", 2020-09-08T15:37:29.025 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:37:29.026 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.026 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.026 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:29.026 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6805", 2020-09-08T15:37:29.027 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:37:29.027 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.027 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.027 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.027 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.166:6801/2108117574", 2020-09-08T15:37:29.028 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.166:6803/2108117574", 2020-09-08T15:37:29.028 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.166:6807/2108117574", 2020-09-08T15:37:29.029 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.166:6805/2108117574", 2020-09-08T15:37:29.029 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:37:29.029 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:37:29.029 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:37:29.029 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.030 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.030 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.030 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 9, 2020-09-08T15:37:29.030 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08", 2020-09-08T15:37:29.031 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:37:29.031 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:37:29.031 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:37:29.031 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:37:29.031 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:37:29.032 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:37:29.032 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 72, 2020-09-08T15:37:29.032 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 0, 2020-09-08T15:37:29.032 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:37:29.033 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:37:29.033 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:37:29.033 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:29.033 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.033 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:29.034 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6809", 2020-09-08T15:37:29.034 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:37:29.034 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.034 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.034 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.034 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:37:29.035 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:29.035 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.035 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:29.035 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6810", 2020-09-08T15:37:29.035 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:37:29.036 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.036 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.036 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:29.036 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6811", 2020-09-08T15:37:29.036 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:37:29.037 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.037 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.037 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.037 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:37:29.037 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:29.038 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.038 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:29.038 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6814", 2020-09-08T15:37:29.038 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:37:29.039 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.039 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.039 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:29.039 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6815", 2020-09-08T15:37:29.040 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:37:29.040 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.040 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.040 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.040 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:37:29.040 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:29.041 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.041 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:29.041 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6812", 2020-09-08T15:37:29.041 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:37:29.041 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.042 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.042 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:29.042 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6813", 2020-09-08T15:37:29.042 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:37:29.042 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.043 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.043 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.043 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.166:6809/1458870080", 2020-09-08T15:37:29.043 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.166:6811/1458870080", 2020-09-08T15:37:29.043 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.166:6815/1458870080", 2020-09-08T15:37:29.044 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.166:6813/1458870080", 2020-09-08T15:37:29.044 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:37:29.044 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:37:29.044 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:37:29.044 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.044 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.045 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.045 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 10, 2020-09-08T15:37:29.045 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "c194210e-bd89-4a1f-98fb-6bf44f0b324c", 2020-09-08T15:37:29.045 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:37:29.045 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:37:29.046 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:37:29.046 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:37:29.046 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:37:29.046 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:37:29.046 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 81, 2020-09-08T15:37:29.047 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 0, 2020-09-08T15:37:29.047 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:37:29.047 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:37:29.047 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:37:29.047 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:29.047 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.048 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:29.048 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6817", 2020-09-08T15:37:29.048 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:37:29.048 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.048 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.049 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.049 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:37:29.049 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:29.050 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.050 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:29.050 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6818", 2020-09-08T15:37:29.050 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:37:29.050 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.051 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.051 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:29.051 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6819", 2020-09-08T15:37:29.051 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:37:29.051 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.052 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.052 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.052 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:37:29.052 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:29.052 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.052 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:29.053 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6822", 2020-09-08T15:37:29.053 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:37:29.053 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.053 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.053 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:29.054 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6823", 2020-09-08T15:37:29.054 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:37:29.054 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.054 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.054 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.055 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:37:29.055 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:29.055 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.055 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:29.055 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6820", 2020-09-08T15:37:29.055 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:37:29.056 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.056 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.056 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:29.056 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6821", 2020-09-08T15:37:29.056 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:37:29.057 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.057 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.057 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.057 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.166:6817/62517848", 2020-09-08T15:37:29.057 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.166:6819/62517848", 2020-09-08T15:37:29.058 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.166:6823/62517848", 2020-09-08T15:37:29.058 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.166:6821/62517848", 2020-09-08T15:37:29.058 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:37:29.058 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:37:29.058 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:37:29.058 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.059 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.059 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.059 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 11, 2020-09-08T15:37:29.060 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "121301fe-a41c-40c0-ad09-b00de37a55e1", 2020-09-08T15:37:29.060 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:37:29.060 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:37:29.060 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:37:29.060 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:37:29.061 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:37:29.061 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:37:29.061 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 90, 2020-09-08T15:37:29.062 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 0, 2020-09-08T15:37:29.062 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:37:29.062 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:37:29.062 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:37:29.062 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:29.062 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.063 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:29.063 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6825", 2020-09-08T15:37:29.063 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:37:29.063 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.063 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.064 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.064 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:37:29.064 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:29.064 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.064 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:29.065 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6826", 2020-09-08T15:37:29.065 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:37:29.065 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.065 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.065 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:29.065 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6827", 2020-09-08T15:37:29.066 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:37:29.066 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.066 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.066 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.066 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:37:29.067 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:29.067 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.067 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:29.067 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6830", 2020-09-08T15:37:29.067 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:37:29.068 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.068 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.068 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:29.068 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6831", 2020-09-08T15:37:29.068 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:37:29.068 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.069 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.069 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.069 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:37:29.069 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:37:29.069 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.070 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:37:29.070 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6828", 2020-09-08T15:37:29.070 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:37:29.071 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.071 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.071 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:37:29.071 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6829", 2020-09-08T15:37:29.071 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:37:29.072 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.072 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.072 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.072 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.166:6825/691735125", 2020-09-08T15:37:29.072 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.166:6827/691735125", 2020-09-08T15:37:29.072 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.166:6831/691735125", 2020-09-08T15:37:29.073 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.166:6829/691735125", 2020-09-08T15:37:29.073 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:37:29.073 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:37:29.073 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:37:29.073 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:29.074 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.074 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T15:37:29.074 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_xinfo": [ 2020-09-08T15:37:29.074 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.074 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 0, 2020-09-08T15:37:29.075 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:37:29.075 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:37:29.075 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:37:29.075 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:37:29.075 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:37:29.076 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:37:04.321311+0000", 2020-09-08T15:37:29.076 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:37:29.076 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.076 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.077 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 1, 2020-09-08T15:37:29.077 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:37:29.077 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:37:29.077 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:37:29.077 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:37:29.078 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:37:29.078 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:36:32.693321+0000", 2020-09-08T15:37:29.078 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:37:29.078 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.078 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.078 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 2, 2020-09-08T15:37:29.079 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:37:29.079 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:37:29.079 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:37:29.079 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:37:29.079 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:37:29.080 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:36:55.322488+0000", 2020-09-08T15:37:29.080 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:37:29.080 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.081 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.081 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 3, 2020-09-08T15:37:29.081 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:37:29.081 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:37:29.081 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:37:29.082 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:37:29.082 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:37:29.082 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:37:19.565242+0000", 2020-09-08T15:37:29.082 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:37:29.082 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.082 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.083 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 4, 2020-09-08T15:37:29.083 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:37:29.083 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:37:29.083 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:37:29.083 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:37:29.084 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:37:29.084 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:36:44.922860+0000", 2020-09-08T15:37:29.084 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:37:29.084 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.084 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.085 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 5, 2020-09-08T15:37:29.085 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:37:29.085 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:37:29.085 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:37:29.085 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:37:29.086 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:37:29.086 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:37:11.906593+0000", 2020-09-08T15:37:29.086 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:37:29.086 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.086 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.087 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 6, 2020-09-08T15:37:29.087 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:37:29.087 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:37:29.087 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:37:29.087 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:37:29.087 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:37:29.088 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:36:37.725618+0000", 2020-09-08T15:37:29.088 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:37:29.088 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.088 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.088 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 7, 2020-09-08T15:37:29.089 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:37:29.089 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:37:29.089 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:37:29.089 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:37:29.089 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:37:29.090 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:37:03.267508+0000", 2020-09-08T15:37:29.090 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:37:29.090 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.090 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.091 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 8, 2020-09-08T15:37:29.091 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:37:29.091 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:37:29.091 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:37:29.091 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:37:29.092 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:37:29.092 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:37:26.118800+0000", 2020-09-08T15:37:29.092 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:37:29.092 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.092 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.093 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 9, 2020-09-08T15:37:29.093 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:37:29.093 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:37:29.093 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:37:29.093 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:37:29.094 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:37:29.094 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:36:52.477193+0000", 2020-09-08T15:37:29.094 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:37:29.094 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.094 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.095 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 10, 2020-09-08T15:37:29.095 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:37:29.095 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:37:29.095 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:37:29.095 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:37:29.095 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:37:29.096 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:37:16.590288+0000", 2020-09-08T15:37:29.096 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:37:29.096 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.096 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:29.096 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 11, 2020-09-08T15:37:29.097 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:37:29.097 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:37:29.097 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:37:29.097 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:37:29.097 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:37:29.098 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:36:40.186285+0000", 2020-09-08T15:37:29.098 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:37:29.098 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.098 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T15:37:29.098 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_upmap": [], 2020-09-08T15:37:29.098 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_upmap_items": [], 2020-09-08T15:37:29.099 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_temp": [], 2020-09-08T15:37:29.099 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_temp": [], 2020-09-08T15:37:29.099 INFO:teuthology.orchestra.run.smithi104.stdout: "blocklist": { 2020-09-08T15:37:29.099 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/1257365091": "2020-09-09T15:29:39.430456+0000", 2020-09-08T15:37:29.099 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/1399123455": "2020-09-09T15:29:39.430456+0000", 2020-09-08T15:37:29.100 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/2810441605": "2020-09-09T15:29:39.430456+0000", 2020-09-08T15:37:29.100 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6800/182241824": "2020-09-09T15:29:39.430456+0000", 2020-09-08T15:37:29.100 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6801/182241824": "2020-09-09T15:29:39.430456+0000", 2020-09-08T15:37:29.100 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/2021933044": "2020-09-09T15:26:33.248197+0000", 2020-09-08T15:37:29.100 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6800/3129754347": "2020-09-09T15:26:33.248197+0000", 2020-09-08T15:37:29.101 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/1270031705": "2020-09-09T15:26:33.248197+0000", 2020-09-08T15:37:29.101 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/2568432190": "2020-09-09T15:26:33.248197+0000", 2020-09-08T15:37:29.101 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6800/1138921844": "2020-09-09T15:26:09.867236+0000", 2020-09-08T15:37:29.102 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/3827328474": "2020-09-09T15:26:09.867236+0000", 2020-09-08T15:37:29.102 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6801/1138921844": "2020-09-09T15:26:09.867236+0000", 2020-09-08T15:37:29.102 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/1662035381": "2020-09-09T15:26:09.867236+0000", 2020-09-08T15:37:29.103 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6801/3129754347": "2020-09-09T15:26:33.248197+0000", 2020-09-08T15:37:29.103 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/2169968102": "2020-09-09T15:26:09.867236+0000" 2020-09-08T15:37:29.103 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.103 INFO:teuthology.orchestra.run.smithi104.stdout: "erasure_code_profiles": { 2020-09-08T15:37:29.103 INFO:teuthology.orchestra.run.smithi104.stdout: "default": { 2020-09-08T15:37:29.104 INFO:teuthology.orchestra.run.smithi104.stdout: "crush-failure-domain": "osd", 2020-09-08T15:37:29.104 INFO:teuthology.orchestra.run.smithi104.stdout: "k": "2", 2020-09-08T15:37:29.104 INFO:teuthology.orchestra.run.smithi104.stdout: "m": "1", 2020-09-08T15:37:29.104 INFO:teuthology.orchestra.run.smithi104.stdout: "plugin": "jerasure", 2020-09-08T15:37:29.105 INFO:teuthology.orchestra.run.smithi104.stdout: "ruleset-failure-domain": "osd", 2020-09-08T15:37:29.105 INFO:teuthology.orchestra.run.smithi104.stdout: "technique": "reed_sol_van" 2020-09-08T15:37:29.105 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:29.105 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:29.105 INFO:teuthology.orchestra.run.smithi104.stdout: "removed_snaps_queue": [], 2020-09-08T15:37:29.106 INFO:teuthology.orchestra.run.smithi104.stdout: "new_removed_snaps": [], 2020-09-08T15:37:29.106 INFO:teuthology.orchestra.run.smithi104.stdout: "new_purged_snaps": [], 2020-09-08T15:37:29.106 INFO:teuthology.orchestra.run.smithi104.stdout: "crush_node_flags": {}, 2020-09-08T15:37:29.106 INFO:teuthology.orchestra.run.smithi104.stdout: "device_class_flags": {} 2020-09-08T15:37:29.106 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:37:29.108 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:37:29.108 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:37:29.109 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'unique_pool_0' created 2020-09-08T15:37:29.109 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:37:28 smithi104 bash[17588]: debug 2020-09-08T15:37:28.694+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:29.109 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:28 smithi104 bash[9203]: audit 2020-09-08T15:37:27.692117+0000 mon.a (mon.0) 903 : audit [DBG] from='client.? v1:172.21.15.104:0/4090501441' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:37:29.110 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:28 smithi104 bash[9203]: audit 2020-09-08T15:37:28.068468+0000 mon.a (mon.0) 904 : audit [INF] from='client.? v1:172.21.15.104:0/3414353366' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:37:29.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:28 smithi055 bash[9296]: audit 2020-09-08T15:37:27.692117+0000 mon.a (mon.0) 903 : audit [DBG] from='client.? v1:172.21.15.104:0/4090501441' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:37:29.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:28 smithi055 bash[9296]: audit 2020-09-08T15:37:28.068468+0000 mon.a (mon.0) 904 : audit [INF] from='client.? v1:172.21.15.104:0/3414353366' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:37:29.158 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:37:28 smithi055 bash[14301]: debug 2020-09-08T15:37:28.918+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:29.211 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:37:29.253 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2020-09-08T15:37:29.364 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:28 smithi166 bash[9446]: audit 2020-09-08T15:37:27.692117+0000 mon.a (mon.0) 903 : audit [DBG] from='client.? v1:172.21.15.104:0/4090501441' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:37:29.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:28 smithi166 bash[9446]: audit 2020-09-08T15:37:28.068468+0000 mon.a (mon.0) 904 : audit [INF] from='client.? v1:172.21.15.104:0/3414353366' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:37:29.426 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:37:29.715 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:37:29.828 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:30.101 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:37:29 smithi104 bash[15273]: debug 2020-09-08T15:37:29.714+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:30.101 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:29 smithi104 bash[9203]: audit 2020-09-08T15:37:28.405052+0000 mon.a (mon.0) 905 : audit [INF] from='client.? v1:172.21.15.104:0/453658996' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2020-09-08T15:37:30.102 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:29 smithi104 bash[9203]: cluster 2020-09-08T15:37:28.859199+0000 mon.a (mon.0) 906 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:37:30.102 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:29 smithi104 bash[9203]: audit 2020-09-08T15:37:28.860380+0000 mon.a (mon.0) 907 : audit [DBG] from='client.? v1:172.21.15.104:0/4184543982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2020-09-08T15:37:30.102 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:29 smithi104 bash[9203]: audit 2020-09-08T15:37:28.868460+0000 mon.a (mon.0) 908 : audit [INF] from='client.? v1:172.21.15.104:0/3414353366' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:37:30.102 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:29 smithi104 bash[9203]: audit 2020-09-08T15:37:28.868594+0000 mon.a (mon.0) 909 : audit [INF] from='client.? v1:172.21.15.104:0/453658996' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2020-09-08T15:37:30.102 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:29 smithi104 bash[9203]: cluster 2020-09-08T15:37:28.868730+0000 mon.a (mon.0) 910 : cluster [DBG] osdmap e126: 12 total, 12 up, 12 in 2020-09-08T15:37:30.103 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:29 smithi104 bash[9203]: cluster 2020-09-08T15:37:29.587705+0000 mgr.x (mgr.24118) 299 : cluster [DBG] pgmap v347: 17 pgs: 16 unknown, 1 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:30.103 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:29 smithi104 bash[9203]: cluster 2020-09-08T15:37:29.870154+0000 mon.a (mon.0) 911 : cluster [DBG] osdmap e127: 12 total, 12 up, 12 in 2020-09-08T15:37:30.103 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:30.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:29 smithi055 bash[9296]: audit 2020-09-08T15:37:28.405052+0000 mon.a (mon.0) 905 : audit [INF] from='client.? v1:172.21.15.104:0/453658996' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2020-09-08T15:37:30.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:29 smithi055 bash[9296]: cluster 2020-09-08T15:37:28.859199+0000 mon.a (mon.0) 906 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:37:30.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:29 smithi055 bash[9296]: audit 2020-09-08T15:37:28.860380+0000 mon.a (mon.0) 907 : audit [DBG] from='client.? v1:172.21.15.104:0/4184543982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2020-09-08T15:37:30.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:29 smithi055 bash[9296]: audit 2020-09-08T15:37:28.868460+0000 mon.a (mon.0) 908 : audit [INF] from='client.? v1:172.21.15.104:0/3414353366' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:37:30.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:29 smithi055 bash[9296]: audit 2020-09-08T15:37:28.868594+0000 mon.a (mon.0) 909 : audit [INF] from='client.? v1:172.21.15.104:0/453658996' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2020-09-08T15:37:30.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:29 smithi055 bash[9296]: cluster 2020-09-08T15:37:28.868730+0000 mon.a (mon.0) 910 : cluster [DBG] osdmap e126: 12 total, 12 up, 12 in 2020-09-08T15:37:30.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:29 smithi055 bash[9296]: cluster 2020-09-08T15:37:29.587705+0000 mgr.x (mgr.24118) 299 : cluster [DBG] pgmap v347: 17 pgs: 16 unknown, 1 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:30.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:29 smithi055 bash[9296]: cluster 2020-09-08T15:37:29.870154+0000 mon.a (mon.0) 911 : cluster [DBG] osdmap e127: 12 total, 12 up, 12 in 2020-09-08T15:37:30.205 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:37:30.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:29 smithi166 bash[9446]: audit 2020-09-08T15:37:28.405052+0000 mon.a (mon.0) 905 : audit [INF] from='client.? v1:172.21.15.104:0/453658996' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2020-09-08T15:37:30.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:29 smithi166 bash[9446]: cluster 2020-09-08T15:37:28.859199+0000 mon.a (mon.0) 906 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:37:30.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:29 smithi166 bash[9446]: audit 2020-09-08T15:37:28.860380+0000 mon.a (mon.0) 907 : audit [DBG] from='client.? v1:172.21.15.104:0/4184543982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2020-09-08T15:37:30.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:29 smithi166 bash[9446]: audit 2020-09-08T15:37:28.868460+0000 mon.a (mon.0) 908 : audit [INF] from='client.? v1:172.21.15.104:0/3414353366' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:37:30.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:29 smithi166 bash[9446]: audit 2020-09-08T15:37:28.868594+0000 mon.a (mon.0) 909 : audit [INF] from='client.? v1:172.21.15.104:0/453658996' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2020-09-08T15:37:30.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:29 smithi166 bash[9446]: cluster 2020-09-08T15:37:28.868730+0000 mon.a (mon.0) 910 : cluster [DBG] osdmap e126: 12 total, 12 up, 12 in 2020-09-08T15:37:30.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:29 smithi166 bash[9446]: cluster 2020-09-08T15:37:29.587705+0000 mgr.x (mgr.24118) 299 : cluster [DBG] pgmap v347: 17 pgs: 16 unknown, 1 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:30.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:29 smithi166 bash[9446]: cluster 2020-09-08T15:37:29.870154+0000 mon.a (mon.0) 911 : cluster [DBG] osdmap e127: 12 total, 12 up, 12 in 2020-09-08T15:37:30.427 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:37:30 smithi104 bash[22305]: debug 2020-09-08T15:37:30.102+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:30.679 INFO:tasks.thrashosds.thrasher:j is {'epoch': 125, 'fsid': '759b27b6-f1e7-11ea-a080-001a4aab830c', 'created': '2020-09-08T15:25:41.788365+0000', 'modified': '2020-09-08T15:37:26.668179+0000', 'last_up_change': '2020-09-08T15:34:41.591462+0000', 'last_in_change': '2020-09-08T15:34:41.591462+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 27, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 1, 'max_osd': 12, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-09-08T15:25:56.314385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}], 'osds': [{'osd': 0, 'uuid': 'e49a7341-7c75-4883-b061-3e8fe9467e6e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6801', 'nonce': 82470352}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6802', 'nonce': 82470352}, {'type': 'v1', 'addr': '172.21.15.104:6803', 'nonce': 82470352}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6806', 'nonce': 82470352}, {'type': 'v1', 'addr': '172.21.15.104:6807', 'nonce': 82470352}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6804', 'nonce': 82470352}, {'type': 'v1', 'addr': '172.21.15.104:6805', 'nonce': 82470352}]}, 'public_addr': '172.21.15.104:6801/82470352', 'cluster_addr': '172.21.15.104:6803/82470352', 'heartbeat_back_addr': '172.21.15.104:6807/82470352', 'heartbeat_front_addr': '172.21.15.104:6805/82470352', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a83df412-110d-440a-b411-ad2a7c36f81a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6809', 'nonce': 1421798992}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6810', 'nonce': 1421798992}, {'type': 'v1', 'addr': '172.21.15.104:6811', 'nonce': 1421798992}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6814', 'nonce': 1421798992}, {'type': 'v1', 'addr': '172.21.15.104:6815', 'nonce': 1421798992}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6812', 'nonce': 1421798992}, {'type': 'v1', 'addr': '172.21.15.104:6813', 'nonce': 1421798992}]}, 'public_addr': '172.21.15.104:6809/1421798992', 'cluster_addr': '172.21.15.104:6811/1421798992', 'heartbeat_back_addr': '172.21.15.104:6815/1421798992', 'heartbeat_front_addr': '172.21.15.104:6813/1421798992', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'ca6da1cc-1d8d-494b-93e6-e065ff70d434', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6817', 'nonce': 2713413369}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6818', 'nonce': 2713413369}, {'type': 'v1', 'addr': '172.21.15.104:6819', 'nonce': 2713413369}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6822', 'nonce': 2713413369}, {'type': 'v1', 'addr': '172.21.15.104:6823', 'nonce': 2713413369}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6820', 'nonce': 2713413369}, {'type': 'v1', 'addr': '172.21.15.104:6821', 'nonce': 2713413369}]}, 'public_addr': '172.21.15.104:6817/2713413369', 'cluster_addr': '172.21.15.104:6819/2713413369', 'heartbeat_back_addr': '172.21.15.104:6823/2713413369', 'heartbeat_front_addr': '172.21.15.104:6821/2713413369', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 29, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6825', 'nonce': 1495946248}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6826', 'nonce': 1495946248}, {'type': 'v1', 'addr': '172.21.15.104:6827', 'nonce': 1495946248}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6830', 'nonce': 1495946248}, {'type': 'v1', 'addr': '172.21.15.104:6831', 'nonce': 1495946248}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6828', 'nonce': 1495946248}, {'type': 'v1', 'addr': '172.21.15.104:6829', 'nonce': 1495946248}]}, 'public_addr': '172.21.15.104:6825/1495946248', 'cluster_addr': '172.21.15.104:6827/1495946248', 'heartbeat_back_addr': '172.21.15.104:6831/1495946248', 'heartbeat_front_addr': '172.21.15.104:6829/1495946248', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd8334c70-f776-4652-944d-aed35a8e43f7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 35, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.55:6803', 'nonce': 1173103260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6804', 'nonce': 1173103260}, {'type': 'v1', 'addr': '172.21.15.55:6805', 'nonce': 1173103260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6808', 'nonce': 1173103260}, {'type': 'v1', 'addr': '172.21.15.55:6809', 'nonce': 1173103260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6806', 'nonce': 1173103260}, {'type': 'v1', 'addr': '172.21.15.55:6807', 'nonce': 1173103260}]}, 'public_addr': '172.21.15.55:6803/1173103260', 'cluster_addr': '172.21.15.55:6805/1173103260', 'heartbeat_back_addr': '172.21.15.55:6809/1173103260', 'heartbeat_front_addr': '172.21.15.55:6807/1173103260', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '6789abd2-9e9b-4c04-81fb-0ce2d2849e39', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 42, 'up_thru': 42, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.55:6811', 'nonce': 1047046376}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6812', 'nonce': 1047046376}, {'type': 'v1', 'addr': '172.21.15.55:6813', 'nonce': 1047046376}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6816', 'nonce': 1047046376}, {'type': 'v1', 'addr': '172.21.15.55:6817', 'nonce': 1047046376}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6814', 'nonce': 1047046376}, {'type': 'v1', 'addr': '172.21.15.55:6815', 'nonce': 1047046376}]}, 'public_addr': '172.21.15.55:6811/1047046376', 'cluster_addr': '172.21.15.55:6813/1047046376', 'heartbeat_back_addr': '172.21.15.55:6817/1047046376', 'heartbeat_front_addr': '172.21.15.55:6815/1047046376', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ad9a93fe-0ca0-4052-bfa9-e2abb268829e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 49, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.55:6819', 'nonce': 2501844793}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6820', 'nonce': 2501844793}, {'type': 'v1', 'addr': '172.21.15.55:6821', 'nonce': 2501844793}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6824', 'nonce': 2501844793}, {'type': 'v1', 'addr': '172.21.15.55:6825', 'nonce': 2501844793}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6822', 'nonce': 2501844793}, {'type': 'v1', 'addr': '172.21.15.55:6823', 'nonce': 2501844793}]}, 'public_addr': '172.21.15.55:6819/2501844793', 'cluster_addr': '172.21.15.55:6821/2501844793', 'heartbeat_back_addr': '172.21.15.55:6825/2501844793', 'heartbeat_front_addr': '172.21.15.55:6823/2501844793', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'f3795c86-4540-442c-a151-b2769fa577e4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 56, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.55:6827', 'nonce': 128909153}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6828', 'nonce': 128909153}, {'type': 'v1', 'addr': '172.21.15.55:6829', 'nonce': 128909153}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6832', 'nonce': 128909153}, {'type': 'v1', 'addr': '172.21.15.55:6833', 'nonce': 128909153}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6830', 'nonce': 128909153}, {'type': 'v1', 'addr': '172.21.15.55:6831', 'nonce': 128909153}]}, 'public_addr': '172.21.15.55:6827/128909153', 'cluster_addr': '172.21.15.55:6829/128909153', 'heartbeat_back_addr': '172.21.15.55:6833/128909153', 'heartbeat_front_addr': '172.21.15.55:6831/128909153', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '3b0ac380-878e-4c1c-9278-80ce67ca2df4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 64, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.166:6801', 'nonce': 2108117574}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6802', 'nonce': 2108117574}, {'type': 'v1', 'addr': '172.21.15.166:6803', 'nonce': 2108117574}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6806', 'nonce': 2108117574}, {'type': 'v1', 'addr': '172.21.15.166:6807', 'nonce': 2108117574}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6804', 'nonce': 2108117574}, {'type': 'v1', 'addr': '172.21.15.166:6805', 'nonce': 2108117574}]}, 'public_addr': '172.21.15.166:6801/2108117574', 'cluster_addr': '172.21.15.166:6803/2108117574', 'heartbeat_back_addr': '172.21.15.166:6807/2108117574', 'heartbeat_front_addr': '172.21.15.166:6805/2108117574', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': 'ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 72, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.166:6809', 'nonce': 1458870080}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6810', 'nonce': 1458870080}, {'type': 'v1', 'addr': '172.21.15.166:6811', 'nonce': 1458870080}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6814', 'nonce': 1458870080}, {'type': 'v1', 'addr': '172.21.15.166:6815', 'nonce': 1458870080}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6812', 'nonce': 1458870080}, {'type': 'v1', 'addr': '172.21.15.166:6813', 'nonce': 1458870080}]}, 'public_addr': '172.21.15.166:6809/1458870080', 'cluster_addr': '172.21.15.166:6811/1458870080', 'heartbeat_back_addr': '172.21.15.166:6815/1458870080', 'heartbeat_front_addr': '172.21.15.166:6813/1458870080', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': 'c194210e-bd89-4a1f-98fb-6bf44f0b324c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 81, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.166:6817', 'nonce': 62517848}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6818', 'nonce': 62517848}, {'type': 'v1', 'addr': '172.21.15.166:6819', 'nonce': 62517848}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6822', 'nonce': 62517848}, {'type': 'v1', 'addr': '172.21.15.166:6823', 'nonce': 62517848}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6820', 'nonce': 62517848}, {'type': 'v1', 'addr': '172.21.15.166:6821', 'nonce': 62517848}]}, 'public_addr': '172.21.15.166:6817/62517848', 'cluster_addr': '172.21.15.166:6819/62517848', 'heartbeat_back_addr': '172.21.15.166:6823/62517848', 'heartbeat_front_addr': '172.21.15.166:6821/62517848', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '121301fe-a41c-40c0-ad09-b00de37a55e1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 90, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.166:6825', 'nonce': 691735125}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6826', 'nonce': 691735125}, {'type': 'v1', 'addr': '172.21.15.166:6827', 'nonce': 691735125}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6830', 'nonce': 691735125}, {'type': 'v1', 'addr': '172.21.15.166:6831', 'nonce': 691735125}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6828', 'nonce': 691735125}, {'type': 'v1', 'addr': '172.21.15.166:6829', 'nonce': 691735125}]}, 'public_addr': '172.21.15.166:6825/691735125', 'cluster_addr': '172.21.15.166:6827/691735125', 'heartbeat_back_addr': '172.21.15.166:6831/691735125', 'heartbeat_front_addr': '172.21.15.166:6829/691735125', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:37:04.321311+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:36:32.693321+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:36:55.322488+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:37:19.565242+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:36:44.922860+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:37:11.906593+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:36:37.725618+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:37:03.267508+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:37:26.118800+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:36:52.477193+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:37:16.590288+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:36:40.186285+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.104:0/1257365091': '2020-09-09T15:29:39.430456+0000', '172.21.15.104:0/1399123455': '2020-09-09T15:29:39.430456+0000', '172.21.15.104:0/2810441605': '2020-09-09T15:29:39.430456+0000', '172.21.15.104:6800/182241824': '2020-09-09T15:29:39.430456+0000', '172.21.15.104:6801/182241824': '2020-09-09T15:29:39.430456+0000', '172.21.15.104:0/2021933044': '2020-09-09T15:26:33.248197+0000', '172.21.15.104:6800/3129754347': '2020-09-09T15:26:33.248197+0000', '172.21.15.104:0/1270031705': '2020-09-09T15:26:33.248197+0000', '172.21.15.104:0/2568432190': '2020-09-09T15:26:33.248197+0000', '172.21.15.104:6800/1138921844': '2020-09-09T15:26:09.867236+0000', '172.21.15.104:0/3827328474': '2020-09-09T15:26:09.867236+0000', '172.21.15.104:6801/1138921844': '2020-09-09T15:26:09.867236+0000', '172.21.15.104:0/1662035381': '2020-09-09T15:26:09.867236+0000', '172.21.15.104:6801/3129754347': '2020-09-09T15:26:33.248197+0000', '172.21.15.104:0/2169968102': '2020-09-09T15:26:09.867236+0000'}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}} 2020-09-08T15:37:30.680 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2020-09-08T15:37:30.700 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:37:30.805 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:37:30.934 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:37:31.039 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:37:31.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:37:30 smithi055 bash[14301]: debug 2020-09-08T15:37:30.934+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:31.177 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:37:30 smithi104 bash[17588]: debug 2020-09-08T15:37:30.702+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:31.192 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:37:31.293 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:37:31.364 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:37:31 smithi166 bash[11928]: debug 2020-09-08T15:37:31.185+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:31.455 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2020-09-08T15:37:31.465 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:37:31.567 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:37:31 smithi104 bash[15273]: debug 2020-09-08T15:37:31.462+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:31.570 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:37:31.594 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:37:31.693 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:37:31.795 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:31.848 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:37:31.876 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:37:31 smithi055 bash[11956]: debug 2020-09-08T15:37:31.686+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:31.971 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:37:32.117 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:31 smithi104 bash[9203]: cluster 2020-09-08T15:37:30.880349+0000 mon.a (mon.0) 912 : cluster [DBG] osdmap e128: 12 total, 12 up, 12 in 2020-09-08T15:37:32.118 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:31 smithi104 bash[9203]: cluster 2020-09-08T15:37:31.588603+0000 mgr.x (mgr.24118) 300 : cluster [DBG] pgmap v350: 17 pgs: 13 unknown, 4 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:32.118 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:32.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:31 smithi055 bash[9296]: cluster 2020-09-08T15:37:30.880349+0000 mon.a (mon.0) 912 : cluster [DBG] osdmap e128: 12 total, 12 up, 12 in 2020-09-08T15:37:32.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:31 smithi055 bash[9296]: cluster 2020-09-08T15:37:31.588603+0000 mgr.x (mgr.24118) 300 : cluster [DBG] pgmap v350: 17 pgs: 13 unknown, 4 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:32.222 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:32.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:31 smithi166 bash[9446]: cluster 2020-09-08T15:37:30.880349+0000 mon.a (mon.0) 912 : cluster [DBG] osdmap e128: 12 total, 12 up, 12 in 2020-09-08T15:37:32.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:31 smithi166 bash[9446]: cluster 2020-09-08T15:37:31.588603+0000 mgr.x (mgr.24118) 300 : cluster [DBG] pgmap v350: 17 pgs: 13 unknown, 4 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:32.427 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:37:32 smithi104 bash[19922]: debug 2020-09-08T15:37:32.118+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:32.750 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:32.855 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:37:33.135 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:37:32 smithi104 bash[22305]: debug 2020-09-08T15:37:32.750+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:33.135 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:37:33.240 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:33.247 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:37:33.412 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:37:33.427 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:37:33 smithi104 bash[15273]: debug 2020-09-08T15:37:33.134+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:33.556 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:33.662 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:33.829 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:33.896 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:37:33 smithi104 bash[22305]: debug 2020-09-08T15:37:33.558+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:33.934 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:34.115 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:37:33 smithi166 bash[18960]: debug 2020-09-08T15:37:33.829+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:34.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:33 smithi166 bash[9446]: cluster 2020-09-08T15:37:32.839733+0000 osd.1 (osd.1) 19 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:34.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:33 smithi166 bash[9446]: cluster 2020-09-08T15:37:32.839872+0000 osd.1 (osd.1) 20 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:34.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:33 smithi166 bash[9446]: cluster 2020-09-08T15:37:32.891738+0000 mon.a (mon.0) 913 : cluster [DBG] osdmap e129: 12 total, 12 up, 12 in 2020-09-08T15:37:34.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:33 smithi166 bash[9446]: cluster 2020-09-08T15:37:33.589381+0000 mgr.x (mgr.24118) 301 : cluster [DBG] pgmap v352: 17 pgs: 17 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:34.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:33 smithi055 bash[9296]: cluster 2020-09-08T15:37:32.839733+0000 osd.1 (osd.1) 19 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:34.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:33 smithi055 bash[9296]: cluster 2020-09-08T15:37:32.839872+0000 osd.1 (osd.1) 20 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:34.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:33 smithi055 bash[9296]: cluster 2020-09-08T15:37:32.891738+0000 mon.a (mon.0) 913 : cluster [DBG] osdmap e129: 12 total, 12 up, 12 in 2020-09-08T15:37:34.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:33 smithi055 bash[9296]: cluster 2020-09-08T15:37:33.589381+0000 mgr.x (mgr.24118) 301 : cluster [DBG] pgmap v352: 17 pgs: 17 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:34.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:33 smithi104 bash[9203]: cluster 2020-09-08T15:37:32.839733+0000 osd.1 (osd.1) 19 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:34.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:33 smithi104 bash[9203]: cluster 2020-09-08T15:37:32.839872+0000 osd.1 (osd.1) 20 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:34.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:33 smithi104 bash[9203]: cluster 2020-09-08T15:37:32.891738+0000 mon.a (mon.0) 913 : cluster [DBG] osdmap e129: 12 total, 12 up, 12 in 2020-09-08T15:37:34.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:33 smithi104 bash[9203]: cluster 2020-09-08T15:37:33.589381+0000 mgr.x (mgr.24118) 301 : cluster [DBG] pgmap v352: 17 pgs: 17 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:34.195 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:34.300 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:37:34.677 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:37:34 smithi104 bash[19922]: debug 2020-09-08T15:37:34.194+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:34.737 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:37:34.841 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:37:34.990 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:37:34 smithi104 bash[15273]: debug 2020-09-08T15:37:34.734+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:35.053 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:37:35.157 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:37:35.306 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:37:35.361 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:37:35 smithi104 bash[15273]: debug 2020-09-08T15:37:35.054+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:35.411 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:37:35.557 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:37:35.614 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:37:35 smithi166 bash[16593]: debug 2020-09-08T15:37:35.305+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:35.658 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:35.681 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2020-09-08T15:37:35.681 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-09-08T15:37:35.682 INFO:tasks.thrashosds.thrasher:inject_pause on 10 2020-09-08T15:37:35.682 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-09-08T15:37:35.682 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-09-08T15:37:35.683 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 config set filestore_inject_stall 3 2020-09-08T15:37:35.818 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:37:35.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:35 smithi055 bash[9296]: cluster 2020-09-08T15:37:35.590051+0000 mgr.x (mgr.24118) 302 : cluster [DBG] pgmap v353: 17 pgs: 17 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:35.908 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:37:35 smithi055 bash[16646]: debug 2020-09-08T15:37:35.546+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:36.079 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:35 smithi104 bash[9203]: cluster 2020-09-08T15:37:35.590051+0000 mgr.x (mgr.24118) 302 : cluster [DBG] pgmap v353: 17 pgs: 17 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:36.080 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:36.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:35 smithi166 bash[9446]: cluster 2020-09-08T15:37:35.590051+0000 mgr.x (mgr.24118) 302 : cluster [DBG] pgmap v353: 17 pgs: 17 active+clean; 0 B data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:36.185 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:36.421 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:37:36.422 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:37:36.423 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:37:36 smithi104 bash[19922]: debug 2020-09-08T15:37:36.078+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:36.453 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:36.560 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:37:36.686 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:37:36.792 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:37:36.805 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:37:36 smithi166 bash[18960]: debug 2020-09-08T15:37:36.453+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:36.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:37:36 smithi055 bash[16646]: debug 2020-09-08T15:37:36.686+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:36.993 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:37:37.098 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:37.114 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:37:36 smithi166 bash[14254]: debug 2020-09-08T15:37:36.993+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:37.217 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:37.321 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:37.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:37:37 smithi055 bash[19003]: debug 2020-09-08T15:37:37.218+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:37.434 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:37.539 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:37.621 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2020-09-08T15:37:37.870 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:37.882 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:37 smithi104 bash[9203]: cluster 2020-09-08T15:37:37.590634+0000 mgr.x (mgr.24118) 303 : cluster [DBG] pgmap v354: 17 pgs: 17 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:37.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:37 smithi055 bash[9296]: cluster 2020-09-08T15:37:37.590634+0000 mgr.x (mgr.24118) 303 : cluster [DBG] pgmap v354: 17 pgs: 17 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:37.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:37:37 smithi055 bash[19003]: debug 2020-09-08T15:37:37.434+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:37.908 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:37:37 smithi055 bash[19003]: debug 2020-09-08T15:37:37.646+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:37.972 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:37:38.011 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:37:38.098 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:37:38.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:37 smithi166 bash[9446]: cluster 2020-09-08T15:37:37.590634+0000 mgr.x (mgr.24118) 303 : cluster [DBG] pgmap v354: 17 pgs: 17 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:38.205 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:38.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:37:38 smithi055 bash[14301]: debug 2020-09-08T15:37:38.098+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:38.555 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:38.661 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:37:38.927 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:37:38 smithi104 bash[22305]: debug 2020-09-08T15:37:38.554+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:38.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:38 smithi104 bash[9203]: cluster 2020-09-08T15:37:37.904921+0000 osd.6 (osd.6) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:38.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:38 smithi104 bash[9203]: cluster 2020-09-08T15:37:37.905061+0000 osd.6 (osd.6) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:38.991 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:38 smithi166 bash[9446]: cluster 2020-09-08T15:37:37.904921+0000 osd.6 (osd.6) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:38.991 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:38 smithi166 bash[9446]: cluster 2020-09-08T15:37:37.905061+0000 osd.6 (osd.6) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:39.048 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:37:39.103 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:37:39.129 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:37:39.148 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:37:39.149 INFO:teuthology.orchestra.run.smithi166.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2020-09-08T15:37:39.149 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:37:39.151 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:37:39.152 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:39.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:38 smithi055 bash[9296]: cluster 2020-09-08T15:37:37.904921+0000 osd.6 (osd.6) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:39.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:38 smithi055 bash[9296]: cluster 2020-09-08T15:37:37.905061+0000 osd.6 (osd.6) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:39.179 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:37:39.226 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:37:39.258 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:37:39.283 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:37:39.312 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:37:39.339 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:37:39.369 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:37:39.385 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:37:39 smithi104 bash[15273]: debug 2020-09-08T15:37:39.050+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:39.385 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:39.412 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:37:39.450 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:37:39.452 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:37:39.452 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:37:39.452 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:37:39.453 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:37:39.453 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:37:39.453 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:37:39.453 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:37:39.453 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:37:39.454 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:37:39.454 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:37:39.454 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:37:39.454 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:37:39.488 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:39.677 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:37:39 smithi104 bash[19922]: debug 2020-09-08T15:37:39.386+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:39.700 INFO:teuthology.orchestra.run.smithi104.stderr:enabled application 'rados' on pool 'unique_pool_0' 2020-09-08T15:37:39.709 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:39.814 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:39.947 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:39 smithi166 bash[9446]: cluster 2020-09-08T15:37:38.679139+0000 mon.a (mon.0) 914 : cluster [DBG] osdmap e130: 12 total, 12 up, 12 in 2020-09-08T15:37:39.948 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:39 smithi166 bash[9446]: audit 2020-09-08T15:37:39.079130+0000 mon.a (mon.0) 915 : audit [DBG] from='client.? v1:172.21.15.104:0/1375495931' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:37:39.948 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:39 smithi166 bash[9446]: audit 2020-09-08T15:37:39.528278+0000 mon.a (mon.0) 916 : audit [INF] from='client.? v1:172.21.15.104:0/2528058151' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T15:37:39.948 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:39 smithi166 bash[9446]: cluster 2020-09-08T15:37:39.591325+0000 mgr.x (mgr.24118) 304 : cluster [DBG] pgmap v356: 17 pgs: 17 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:39.949 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:37:39 smithi166 bash[18960]: debug 2020-09-08T15:37:39.949+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:39.949 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:40.052 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:37:40.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:39 smithi055 bash[9296]: cluster 2020-09-08T15:37:38.679139+0000 mon.a (mon.0) 914 : cluster [DBG] osdmap e130: 12 total, 12 up, 12 in 2020-09-08T15:37:40.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:39 smithi055 bash[9296]: audit 2020-09-08T15:37:39.079130+0000 mon.a (mon.0) 915 : audit [DBG] from='client.? v1:172.21.15.104:0/1375495931' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:37:40.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:39 smithi055 bash[9296]: audit 2020-09-08T15:37:39.528278+0000 mon.a (mon.0) 916 : audit [INF] from='client.? v1:172.21.15.104:0/2528058151' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T15:37:40.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:39 smithi055 bash[9296]: cluster 2020-09-08T15:37:39.591325+0000 mgr.x (mgr.24118) 304 : cluster [DBG] pgmap v356: 17 pgs: 17 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:40.187 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:37:40.205 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:37:39 smithi104 bash[22305]: debug 2020-09-08T15:37:39.710+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:40.205 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:39 smithi104 bash[9203]: cluster 2020-09-08T15:37:38.679139+0000 mon.a (mon.0) 914 : cluster [DBG] osdmap e130: 12 total, 12 up, 12 in 2020-09-08T15:37:40.206 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:39 smithi104 bash[9203]: audit 2020-09-08T15:37:39.079130+0000 mon.a (mon.0) 915 : audit [DBG] from='client.? v1:172.21.15.104:0/1375495931' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:37:40.206 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:39 smithi104 bash[9203]: audit 2020-09-08T15:37:39.528278+0000 mon.a (mon.0) 916 : audit [INF] from='client.? v1:172.21.15.104:0/2528058151' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T15:37:40.206 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:39 smithi104 bash[9203]: cluster 2020-09-08T15:37:39.591325+0000 mgr.x (mgr.24118) 304 : cluster [DBG] pgmap v356: 17 pgs: 17 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:40.294 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:37:40.364 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:37:40 smithi166 bash[16593]: debug 2020-09-08T15:37:40.189+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:40.594 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:37:40.698 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:40.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:40 smithi104 bash[9203]: audit 2020-09-08T15:37:39.700354+0000 mon.a (mon.0) 917 : audit [INF] from='client.? v1:172.21.15.104:0/2528058151' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T15:37:40.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:40 smithi104 bash[9203]: cluster 2020-09-08T15:37:39.700429+0000 mon.a (mon.0) 918 : cluster [DBG] osdmap e131: 12 total, 12 up, 12 in 2020-09-08T15:37:40.928 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:37:40 smithi104 bash[17588]: debug 2020-09-08T15:37:40.594+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:41.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:40 smithi166 bash[9446]: audit 2020-09-08T15:37:39.700354+0000 mon.a (mon.0) 917 : audit [INF] from='client.? v1:172.21.15.104:0/2528058151' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T15:37:41.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:40 smithi166 bash[9446]: cluster 2020-09-08T15:37:39.700429+0000 mon.a (mon.0) 918 : cluster [DBG] osdmap e131: 12 total, 12 up, 12 in 2020-09-08T15:37:41.129 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:41.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:40 smithi055 bash[9296]: audit 2020-09-08T15:37:39.700354+0000 mon.a (mon.0) 917 : audit [INF] from='client.? v1:172.21.15.104:0/2528058151' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T15:37:41.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:40 smithi055 bash[9296]: cluster 2020-09-08T15:37:39.700429+0000 mon.a (mon.0) 918 : cluster [DBG] osdmap e131: 12 total, 12 up, 12 in 2020-09-08T15:37:41.234 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:37:41.427 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:37:41 smithi104 bash[22305]: debug 2020-09-08T15:37:41.130+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:41.500 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:37:41.605 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:37:41.723 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:37:41.750 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:37:41.816 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:41 smithi104 bash[9203]: cluster 2020-09-08T15:37:40.440195+0000 osd.11 (osd.11) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:41.823 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:41 smithi104 bash[9203]: cluster 2020-09-08T15:37:40.440392+0000 osd.11 (osd.11) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:41.823 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:41 smithi104 bash[9203]: cluster 2020-09-08T15:37:40.702601+0000 mon.a (mon.0) 919 : cluster [DBG] osdmap e132: 12 total, 12 up, 12 in 2020-09-08T15:37:41.824 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:41 smithi104 bash[9203]: audit 2020-09-08T15:37:41.460097+0000 mon.a (mon.0) 920 : audit [INF] from='client.? v1:172.21.15.104:0/1924966600' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:37:41.824 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:41 smithi104 bash[9203]: cluster 2020-09-08T15:37:41.592039+0000 mgr.x (mgr.24118) 305 : cluster [DBG] pgmap v359: 17 pgs: 17 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:41.856 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:41.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:37:41 smithi055 bash[14301]: debug 2020-09-08T15:37:41.498+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:41.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:41 smithi055 bash[9296]: cluster 2020-09-08T15:37:40.440195+0000 osd.11 (osd.11) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:41.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:41 smithi055 bash[9296]: cluster 2020-09-08T15:37:40.440392+0000 osd.11 (osd.11) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:41.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:41 smithi055 bash[9296]: cluster 2020-09-08T15:37:40.702601+0000 mon.a (mon.0) 919 : cluster [DBG] osdmap e132: 12 total, 12 up, 12 in 2020-09-08T15:37:41.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:41 smithi055 bash[9296]: audit 2020-09-08T15:37:41.460097+0000 mon.a (mon.0) 920 : audit [INF] from='client.? v1:172.21.15.104:0/1924966600' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:37:41.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:41 smithi055 bash[9296]: cluster 2020-09-08T15:37:41.592039+0000 mgr.x (mgr.24118) 305 : cluster [DBG] pgmap v359: 17 pgs: 17 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:42.115 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:37:41 smithi166 bash[14254]: debug 2020-09-08T15:37:41.749+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:42.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:41 smithi166 bash[9446]: cluster 2020-09-08T15:37:40.440195+0000 osd.11 (osd.11) 7 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:42.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:41 smithi166 bash[9446]: cluster 2020-09-08T15:37:40.440392+0000 osd.11 (osd.11) 8 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:42.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:41 smithi166 bash[9446]: cluster 2020-09-08T15:37:40.702601+0000 mon.a (mon.0) 919 : cluster [DBG] osdmap e132: 12 total, 12 up, 12 in 2020-09-08T15:37:42.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:41 smithi166 bash[9446]: audit 2020-09-08T15:37:41.460097+0000 mon.a (mon.0) 920 : audit [INF] from='client.? v1:172.21.15.104:0/1924966600' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:37:42.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:41 smithi166 bash[9446]: cluster 2020-09-08T15:37:41.592039+0000 mgr.x (mgr.24118) 305 : cluster [DBG] pgmap v359: 17 pgs: 17 active+clean; 0 B data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:37:42.336 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:42.442 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:42.677 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:37:42 smithi104 bash[19922]: debug 2020-09-08T15:37:42.334+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:42.695 INFO:teuthology.orchestra.run.smithi166:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.2 -b 65536 --object-size 65536 -p unique_pool_0 bench 90 write' 2020-09-08T15:37:42.699 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2020-09-08T15:37:42.699 INFO:teuthology.orchestra.run:waiting for 3000 2020-09-08T15:37:42.752 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:42.766 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:hints = 1 2020-09-08T15:37:42.766 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2020-09-08T15:37:42.766 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Object prefix: benchmark_data_smithi166_20551 2020-09-08T15:37:42.767 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:37:42.767 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 0 0 0 0 0 0 - 0 2020-09-08T15:37:42.858 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:37:43.014 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:42 smithi166 bash[9446]: cluster 2020-09-08T15:37:41.709096+0000 mon.a (mon.0) 921 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:37:43.015 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:42 smithi166 bash[9446]: audit 2020-09-08T15:37:41.723244+0000 mon.a (mon.0) 922 : audit [INF] from='client.? v1:172.21.15.104:0/1924966600' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:37:43.015 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:42 smithi166 bash[9446]: cluster 2020-09-08T15:37:41.723307+0000 mon.a (mon.0) 923 : cluster [DBG] osdmap e133: 12 total, 12 up, 12 in 2020-09-08T15:37:43.016 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:37:43.118 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:43.122 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:37:42 smithi104 bash[22305]: debug 2020-09-08T15:37:42.754+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:43.123 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:42 smithi104 bash[9203]: cluster 2020-09-08T15:37:41.709096+0000 mon.a (mon.0) 921 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:37:43.123 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:42 smithi104 bash[9203]: audit 2020-09-08T15:37:41.723244+0000 mon.a (mon.0) 922 : audit [INF] from='client.? v1:172.21.15.104:0/1924966600' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:37:43.123 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:42 smithi104 bash[9203]: cluster 2020-09-08T15:37:41.723307+0000 mon.a (mon.0) 923 : cluster [DBG] osdmap e133: 12 total, 12 up, 12 in 2020-09-08T15:37:43.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:42 smithi055 bash[9296]: cluster 2020-09-08T15:37:41.709096+0000 mon.a (mon.0) 921 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:37:43.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:42 smithi055 bash[9296]: audit 2020-09-08T15:37:41.723244+0000 mon.a (mon.0) 922 : audit [INF] from='client.? v1:172.21.15.104:0/1924966600' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:37:43.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:42 smithi055 bash[9296]: cluster 2020-09-08T15:37:41.723307+0000 mon.a (mon.0) 923 : cluster [DBG] osdmap e133: 12 total, 12 up, 12 in 2020-09-08T15:37:43.236 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:43.340 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:43.348 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:37:43 smithi166 bash[11928]: debug 2020-09-08T15:37:43.013+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:43.349 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:37:43 smithi166 bash[18960]: debug 2020-09-08T15:37:43.237+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:43.472 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:43.527 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:37:43.577 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:43.614 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:37:43 smithi166 bash[18960]: debug 2020-09-08T15:37:43.469+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:43.713 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:43.714 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:37:43.722 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:43 smithi104 bash[9203]: cluster 2020-09-08T15:37:43.592675+0000 mgr.x (mgr.24118) 306 : cluster [DBG] pgmap v361: 17 pgs: 17 active+clean; 22 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s wr, 72 op/s 2020-09-08T15:37:43.735 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:37:43.735 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2020-09-08T15:37:43.736 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2020-09-08T15:37:43.736 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T15:37:43.736 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.736 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:1254 2.4 e133/126 2:2753aafe:::benchmark_data_smithi166_20551_object1253:head v 133'76, mlcod=133'74)", 2020-09-08T15:37:43.736 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:43.088212+0000", 2020-09-08T15:37:43.737 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.64959021500000003, 2020-09-08T15:37:43.737 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0045696959999999998, 2020-09-08T15:37:43.737 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:37:43.737 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:37:43.737 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:37:43.738 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.738 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:37:43.738 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088212+0000", 2020-09-08T15:37:43.738 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:37:43.739 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.739 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.739 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:37:43.739 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088212+0000", 2020-09-08T15:37:43.739 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0425000000000002e-05 2020-09-08T15:37:43.739 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.740 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.740 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:37:43.740 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088303+0000", 2020-09-08T15:37:43.740 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3629999999999994e-06 2020-09-08T15:37:43.740 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.741 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.741 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:37:43.741 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088311+0000", 2020-09-08T15:37:43.742 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3872e-05 2020-09-08T15:37:43.742 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.742 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.742 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:37:43.742 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088325+0000", 2020-09-08T15:37:43.743 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6799999999999998e-06 2020-09-08T15:37:43.743 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.743 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.743 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:37:43.744 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088331+0000", 2020-09-08T15:37:43.744 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2091e-05 2020-09-08T15:37:43.745 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.745 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.745 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:37:43.746 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088383+0000", 2020-09-08T15:37:43.746 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2308000000000001e-05 2020-09-08T15:37:43.746 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.746 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.746 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:37:43.747 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088405+0000", 2020-09-08T15:37:43.747 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0043043220000000002 2020-09-08T15:37:43.747 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.747 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.747 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:37:43.747 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.092709+0000", 2020-09-08T15:37:43.748 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.2634999999999998e-05 2020-09-08T15:37:43.748 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.748 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.748 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:37:43.749 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.092782+0000", 2020-09-08T15:37:43.749 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0045696959999999998 2020-09-08T15:37:43.749 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.749 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.749 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.750 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.750 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.750 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:1260 2.4 e133/126 2:247637d6:::benchmark_data_smithi166_20551_object1259:head v 133'77, mlcod=133'74)", 2020-09-08T15:37:43.750 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:43.088386+0000", 2020-09-08T15:37:43.751 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.64941676199999998, 2020-09-08T15:37:43.751 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0046167650000000001, 2020-09-08T15:37:43.751 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:37:43.751 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:37:43.751 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:37:43.752 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.752 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:37:43.752 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088386+0000", 2020-09-08T15:37:43.752 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:37:43.752 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.753 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.753 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:37:43.753 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088386+0000", 2020-09-08T15:37:43.753 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6749000000000006e-05 2020-09-08T15:37:43.753 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.754 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.754 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:37:43.758 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088472+0000", 2020-09-08T15:37:43.758 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1089999999999998e-06 2020-09-08T15:37:43.758 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.758 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.759 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:37:43.759 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088481+0000", 2020-09-08T15:37:43.759 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3856999999999999e-05 2020-09-08T15:37:43.759 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.759 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.760 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:37:43.760 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088504+0000", 2020-09-08T15:37:43.761 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9490000000000001e-06 2020-09-08T15:37:43.761 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.762 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.762 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:37:43.762 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088510+0000", 2020-09-08T15:37:43.762 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00024818000000000003 2020-09-08T15:37:43.763 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.763 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.763 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:37:43.763 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088759+0000", 2020-09-08T15:37:43.763 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3123000000000001e-05 2020-09-08T15:37:43.764 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.764 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.764 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:37:43.765 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088792+0000", 2020-09-08T15:37:43.765 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0039945329999999998 2020-09-08T15:37:43.765 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.766 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.766 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:37:43.766 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.092786+0000", 2020-09-08T15:37:43.766 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00021626499999999999 2020-09-08T15:37:43.767 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.767 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.767 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:37:43.767 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.093002+0000", 2020-09-08T15:37:43.768 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0046167650000000001 2020-09-08T15:37:43.768 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.768 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.768 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.768 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.769 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.769 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:1264 2.c e133/126 2:36550589:::benchmark_data_smithi166_20551_object1263:head v 133'83, mlcod=133'81)", 2020-09-08T15:37:43.769 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:43.088628+0000", 2020-09-08T15:37:43.769 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.64917462699999995, 2020-09-08T15:37:43.769 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0044334939999999996, 2020-09-08T15:37:43.770 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:37:43.770 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:37:43.770 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:37:43.770 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.771 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:37:43.771 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088628+0000", 2020-09-08T15:37:43.771 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:37:43.771 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.771 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.772 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:37:43.772 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088628+0000", 2020-09-08T15:37:43.772 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8488999999999998e-05 2020-09-08T15:37:43.772 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.772 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.773 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:37:43.773 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088726+0000", 2020-09-08T15:37:43.773 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4989999999999994e-06 2020-09-08T15:37:43.774 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.774 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.774 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:37:43.774 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088735+0000", 2020-09-08T15:37:43.775 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3288000000000001e-05 2020-09-08T15:37:43.775 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.776 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.776 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:37:43.776 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088748+0000", 2020-09-08T15:37:43.776 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.536e-06 2020-09-08T15:37:43.777 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.777 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.777 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:37:43.777 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088754+0000", 2020-09-08T15:37:43.778 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000101436 2020-09-08T15:37:43.778 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.778 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.778 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:37:43.778 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088855+0000", 2020-09-08T15:37:43.779 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2892999999999999e-05 2020-09-08T15:37:43.779 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.779 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.779 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:37:43.779 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.088878+0000", 2020-09-08T15:37:43.779 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0041273209999999998 2020-09-08T15:37:43.780 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.780 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.780 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:37:43.780 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.093005+0000", 2020-09-08T15:37:43.780 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6032000000000002e-05 2020-09-08T15:37:43.781 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.781 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.781 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:37:43.781 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.093061+0000", 2020-09-08T15:37:43.781 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0044334939999999996 2020-09-08T15:37:43.782 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.782 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.782 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.782 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.782 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.783 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:1263 2.4 e133/126 2:20059f85:::benchmark_data_smithi166_20551_object1262:head v 133'78, mlcod=133'74)", 2020-09-08T15:37:43.783 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:43.089111+0000", 2020-09-08T15:37:43.783 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.64869160000000003, 2020-09-08T15:37:43.783 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0041862679999999999, 2020-09-08T15:37:43.783 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:37:43.784 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:37:43.784 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:37:43.784 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.784 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:37:43.784 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.089111+0000", 2020-09-08T15:37:43.785 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:37:43.785 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.785 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.786 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:37:43.786 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.089111+0000", 2020-09-08T15:37:43.789 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010986500000000001 2020-09-08T15:37:43.789 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.790 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.790 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:37:43.791 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.089221+0000", 2020-09-08T15:37:43.791 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2399999999999998e-06 2020-09-08T15:37:43.791 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.791 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.791 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:37:43.792 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.089226+0000", 2020-09-08T15:37:43.792 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4449999999999996e-06 2020-09-08T15:37:43.792 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.792 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.793 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:37:43.793 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.089234+0000", 2020-09-08T15:37:43.793 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6330000000000004e-06 2020-09-08T15:37:43.793 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.794 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.794 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:37:43.794 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.089239+0000", 2020-09-08T15:37:43.795 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5655000000000002e-05 2020-09-08T15:37:43.795 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.795 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.795 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:37:43.796 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.089285+0000", 2020-09-08T15:37:43.796 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2388e-05 2020-09-08T15:37:43.796 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.797 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.797 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:37:43.797 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.089307+0000", 2020-09-08T15:37:43.797 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003756564 2020-09-08T15:37:43.797 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.798 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.798 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:37:43.798 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.093064+0000", 2020-09-08T15:37:43.798 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00023347799999999999 2020-09-08T15:37:43.799 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.799 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.799 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:37:43.800 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.093297+0000", 2020-09-08T15:37:43.800 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0041862679999999999 2020-09-08T15:37:43.800 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.800 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.800 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.801 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.801 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.801 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:1267 2.c e133/126 2:3863db83:::benchmark_data_smithi166_20551_object1266:head v 133'84, mlcod=133'81)", 2020-09-08T15:37:43.801 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:43.089600+0000", 2020-09-08T15:37:43.801 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.64820244900000001, 2020-09-08T15:37:43.802 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0037362960000000001, 2020-09-08T15:37:43.802 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:37:43.802 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:37:43.802 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:37:43.802 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.803 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:37:43.803 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.089600+0000", 2020-09-08T15:37:43.803 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:37:43.803 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.803 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.804 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:37:43.804 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.089600+0000", 2020-09-08T15:37:43.804 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000121556 2020-09-08T15:37:43.804 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.804 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.805 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:37:43.805 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.089722+0000", 2020-09-08T15:37:43.805 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7749999999999993e-06 2020-09-08T15:37:43.805 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.805 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.806 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:37:43.806 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.089729+0000", 2020-09-08T15:37:43.807 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2377e-05 2020-09-08T15:37:43.807 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.807 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.808 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:37:43.808 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.089742+0000", 2020-09-08T15:37:43.808 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6670000000000001e-06 2020-09-08T15:37:43.808 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.808 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.809 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:37:43.809 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.089747+0000", 2020-09-08T15:37:43.809 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.629e-05 2020-09-08T15:37:43.809 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.809 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.810 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:37:43.810 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.089784+0000", 2020-09-08T15:37:43.810 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1931000000000001e-05 2020-09-08T15:37:43.810 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.810 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.811 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:37:43.811 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.089806+0000", 2020-09-08T15:37:43.811 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0034937739999999998 2020-09-08T15:37:43.812 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.812 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.812 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:37:43.812 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.093299+0000", 2020-09-08T15:37:43.812 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6925999999999998e-05 2020-09-08T15:37:43.812 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.813 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.813 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:37:43.813 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.093336+0000", 2020-09-08T15:37:43.813 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0037362960000000001 2020-09-08T15:37:43.813 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.814 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.814 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.814 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.814 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.814 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:1288 2.c e133/126 2:3bee0cf7:::benchmark_data_smithi166_20551_object1287:head v 133'87, mlcod=133'85)", 2020-09-08T15:37:43.815 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:43.097473+0000", 2020-09-08T15:37:43.815 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.64032977899999999, 2020-09-08T15:37:43.815 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0033411389999999999, 2020-09-08T15:37:43.815 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:37:43.815 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:37:43.816 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:37:43.816 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.816 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:37:43.817 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.097473+0000", 2020-09-08T15:37:43.817 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:37:43.818 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.818 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.818 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:37:43.819 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.097473+0000", 2020-09-08T15:37:43.819 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010743000000000001 2020-09-08T15:37:43.819 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.819 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.819 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:37:43.819 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.097580+0000", 2020-09-08T15:37:43.820 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4649999999999994e-06 2020-09-08T15:37:43.820 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.820 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.820 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:37:43.820 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.097589+0000", 2020-09-08T15:37:43.821 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6305000000000001e-05 2020-09-08T15:37:43.821 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.821 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.821 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:37:43.821 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.097605+0000", 2020-09-08T15:37:43.822 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9510000000000003e-06 2020-09-08T15:37:43.822 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.822 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.822 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:37:43.822 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.097611+0000", 2020-09-08T15:37:43.823 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000115294 2020-09-08T15:37:43.823 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.823 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.823 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:37:43.823 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.097726+0000", 2020-09-08T15:37:43.824 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2124e-05 2020-09-08T15:37:43.824 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.824 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.824 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:37:43.824 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.097748+0000", 2020-09-08T15:37:43.825 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0029856119999999999 2020-09-08T15:37:43.825 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.825 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.825 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:37:43.825 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.100734+0000", 2020-09-08T15:37:43.826 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9957999999999996e-05 2020-09-08T15:37:43.826 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.826 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.826 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:37:43.827 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.100814+0000", 2020-09-08T15:37:43.827 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0033411389999999999 2020-09-08T15:37:43.827 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.828 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.828 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.828 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.829 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.829 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:2186 2.5 e133/126 2:acd6db27:::benchmark_data_smithi166_20551_object2185:head v 133'143, mlcod=133'141)", 2020-09-08T15:37:43.829 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:43.328214+0000", 2020-09-08T15:37:43.829 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.40958894899999998, 2020-09-08T15:37:43.829 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0040396190000000004, 2020-09-08T15:37:43.830 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:37:43.830 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:37:43.830 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:37:43.830 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.830 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:37:43.831 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.328214+0000", 2020-09-08T15:37:43.831 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:37:43.831 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.831 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.831 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:37:43.832 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.328214+0000", 2020-09-08T15:37:43.832 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3307999999999995e-05 2020-09-08T15:37:43.832 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.832 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.832 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:37:43.833 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.328307+0000", 2020-09-08T15:37:43.833 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8140000000000007e-06 2020-09-08T15:37:43.833 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.833 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.833 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:37:43.834 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.328316+0000", 2020-09-08T15:37:43.834 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4464000000000001e-05 2020-09-08T15:37:43.834 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.834 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.834 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:37:43.834 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.328330+0000", 2020-09-08T15:37:43.835 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0340000000000002e-06 2020-09-08T15:37:43.835 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.835 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.835 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:37:43.835 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.328336+0000", 2020-09-08T15:37:43.836 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0698000000000006e-05 2020-09-08T15:37:43.836 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.836 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.836 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:37:43.836 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.328407+0000", 2020-09-08T15:37:43.837 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8710000000000001e-05 2020-09-08T15:37:43.837 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.838 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.838 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:37:43.838 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.328436+0000", 2020-09-08T15:37:43.838 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0037319139999999998 2020-09-08T15:37:43.838 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.840 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.840 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:37:43.840 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.332167+0000", 2020-09-08T15:37:43.841 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5677e-05 2020-09-08T15:37:43.841 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.841 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.841 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:37:43.841 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.332253+0000", 2020-09-08T15:37:43.842 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0040396190000000004 2020-09-08T15:37:43.842 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.842 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.842 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.842 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.843 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.843 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:2192 2.c e133/126 2:33a77dfa:::benchmark_data_smithi166_20551_object2191:head v 133'138, mlcod=133'136)", 2020-09-08T15:37:43.843 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:43.328395+0000", 2020-09-08T15:37:43.843 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.40940755499999998, 2020-09-08T15:37:43.843 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0043673690000000003, 2020-09-08T15:37:43.844 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:37:43.844 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:37:43.844 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:37:43.844 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.844 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:37:43.845 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.328395+0000", 2020-09-08T15:37:43.845 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:37:43.845 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.845 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.845 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:37:43.846 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.328395+0000", 2020-09-08T15:37:43.846 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000114166 2020-09-08T15:37:43.846 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.846 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.846 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:37:43.847 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.328509+0000", 2020-09-08T15:37:43.847 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7350000000000002e-06 2020-09-08T15:37:43.847 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.847 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.847 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:37:43.847 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.328518+0000", 2020-09-08T15:37:43.848 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4619e-05 2020-09-08T15:37:43.848 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.848 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.848 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:37:43.848 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.328532+0000", 2020-09-08T15:37:43.849 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6079999999999998e-06 2020-09-08T15:37:43.849 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.849 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.849 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:37:43.849 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.328540+0000", 2020-09-08T15:37:43.850 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000109922 2020-09-08T15:37:43.850 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.850 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.850 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:37:43.850 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.328650+0000", 2020-09-08T15:37:43.850 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6100000000000001e-05 2020-09-08T15:37:43.851 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.851 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.851 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:37:43.851 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.328676+0000", 2020-09-08T15:37:43.851 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0040157020000000003 2020-09-08T15:37:43.852 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.852 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.852 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:37:43.852 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.332692+0000", 2020-09-08T15:37:43.852 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0517000000000003e-05 2020-09-08T15:37:43.853 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.853 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.853 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:37:43.853 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.332762+0000", 2020-09-08T15:37:43.853 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0043673690000000003 2020-09-08T15:37:43.854 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.854 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.854 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.854 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.854 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.855 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:2197 2.4 e133/126 2:2f81e16b:::benchmark_data_smithi166_20551_object2196:head v 133'136, mlcod=133'134)", 2020-09-08T15:37:43.855 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:43.328980+0000", 2020-09-08T15:37:43.855 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.408822091, 2020-09-08T15:37:43.855 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0038458709999999998, 2020-09-08T15:37:43.855 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:37:43.855 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:37:43.856 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:37:43.856 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.856 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:37:43.856 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.328980+0000", 2020-09-08T15:37:43.856 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:37:43.857 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.857 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.857 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:37:43.857 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.328980+0000", 2020-09-08T15:37:43.857 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2720000000000007e-05 2020-09-08T15:37:43.858 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.858 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.858 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:37:43.858 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.329073+0000", 2020-09-08T15:37:43.858 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7149999999999998e-06 2020-09-08T15:37:43.859 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.859 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.859 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:37:43.859 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.329081+0000", 2020-09-08T15:37:43.859 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8904000000000001e-05 2020-09-08T15:37:43.859 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.860 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.860 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:37:43.860 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.329100+0000", 2020-09-08T15:37:43.860 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.331e-06 2020-09-08T15:37:43.860 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.861 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.861 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:37:43.861 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.329106+0000", 2020-09-08T15:37:43.861 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6396000000000001e-05 2020-09-08T15:37:43.861 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.862 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.862 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:37:43.862 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.329152+0000", 2020-09-08T15:37:43.862 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.7563999999999999e-05 2020-09-08T15:37:43.862 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.862 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.863 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:37:43.863 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.329180+0000", 2020-09-08T15:37:43.863 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003585754 2020-09-08T15:37:43.863 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.863 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.864 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:37:43.864 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.332766+0000", 2020-09-08T15:37:43.864 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0486999999999999e-05 2020-09-08T15:37:43.864 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.864 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.865 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:37:43.865 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.332826+0000", 2020-09-08T15:37:43.865 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0038458709999999998 2020-09-08T15:37:43.865 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.865 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.865 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.866 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.866 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.866 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:2679 2.5 e133/126 2:a9a96f17:::benchmark_data_smithi166_20551_object2678:head v 133'172, mlcod=133'169)", 2020-09-08T15:37:43.866 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:43.457808+0000", 2020-09-08T15:37:43.866 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.27999483400000003, 2020-09-08T15:37:43.867 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0052559080000000001, 2020-09-08T15:37:43.867 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:37:43.867 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:37:43.867 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:37:43.867 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.868 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:37:43.868 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.457808+0000", 2020-09-08T15:37:43.868 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:37:43.868 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.868 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.868 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:37:43.869 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.457808+0000", 2020-09-08T15:37:43.869 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000117828 2020-09-08T15:37:43.869 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.869 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.869 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:37:43.870 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.457925+0000", 2020-09-08T15:37:43.870 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8530000000000004e-06 2020-09-08T15:37:43.870 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.870 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.870 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:37:43.871 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.457935+0000", 2020-09-08T15:37:43.871 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6201e-05 2020-09-08T15:37:43.871 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.871 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.871 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:37:43.871 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.457952+0000", 2020-09-08T15:37:43.872 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2179999999999996e-06 2020-09-08T15:37:43.872 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.872 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.872 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:37:43.872 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.457958+0000", 2020-09-08T15:37:43.873 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3144000000000004e-05 2020-09-08T15:37:43.873 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.873 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.873 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:37:43.873 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.458021+0000", 2020-09-08T15:37:43.874 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4496999999999999e-05 2020-09-08T15:37:43.874 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.874 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.874 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:37:43.874 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.458045+0000", 2020-09-08T15:37:43.874 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0034081089999999999 2020-09-08T15:37:43.875 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.875 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.875 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:37:43.875 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.461454+0000", 2020-09-08T15:37:43.875 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0016100579999999999 2020-09-08T15:37:43.876 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.876 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.876 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:37:43.876 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.463064+0000", 2020-09-08T15:37:43.876 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0052559080000000001 2020-09-08T15:37:43.877 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.877 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.877 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.877 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.877 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.877 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:2754 2.c e133/126 2:3470a6af:::benchmark_data_smithi166_20551_object2753:head v 133'172, mlcod=133'170)", 2020-09-08T15:37:43.878 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:43.478683+0000", 2020-09-08T15:37:43.878 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.25911979299999999, 2020-09-08T15:37:43.878 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0054067710000000003, 2020-09-08T15:37:43.878 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:37:43.878 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:37:43.879 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:37:43.879 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.879 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:37:43.879 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.478683+0000", 2020-09-08T15:37:43.879 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:37:43.880 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.880 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.880 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:37:43.880 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.478683+0000", 2020-09-08T15:37:43.880 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9303999999999997e-05 2020-09-08T15:37:43.880 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.881 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.881 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:37:43.881 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.478772+0000", 2020-09-08T15:37:43.881 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2709999999999998e-06 2020-09-08T15:37:43.881 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.882 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.882 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:37:43.882 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.478778+0000", 2020-09-08T15:37:43.882 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4900000000000006e-06 2020-09-08T15:37:43.882 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.883 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.883 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:37:43.883 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.478788+0000", 2020-09-08T15:37:43.883 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6420000000000004e-06 2020-09-08T15:37:43.883 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.884 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.884 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:37:43.884 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.478793+0000", 2020-09-08T15:37:43.884 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000107054 2020-09-08T15:37:43.884 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.885 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.885 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:37:43.885 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.478900+0000", 2020-09-08T15:37:43.885 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6182000000000001e-05 2020-09-08T15:37:43.885 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.885 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.886 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:37:43.886 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.478927+0000", 2020-09-08T15:37:43.886 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0049931990000000002 2020-09-08T15:37:43.886 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.886 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.887 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:37:43.887 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.483920+0000", 2020-09-08T15:37:43.887 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000169629 2020-09-08T15:37:43.887 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.887 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.888 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:37:43.888 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.484089+0000", 2020-09-08T15:37:43.888 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0054067710000000003 2020-09-08T15:37:43.888 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.888 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.889 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.889 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.889 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.889 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:2772 2.c e133/126 2:3e179b35:::benchmark_data_smithi166_20551_object2771:head v 133'174, mlcod=133'170)", 2020-09-08T15:37:43.889 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:43.484839+0000", 2020-09-08T15:37:43.889 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.252963724, 2020-09-08T15:37:43.890 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0040950689999999998, 2020-09-08T15:37:43.890 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:37:43.890 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:37:43.890 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:37:43.890 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.891 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:37:43.891 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.484839+0000", 2020-09-08T15:37:43.891 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:37:43.891 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.891 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.892 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:37:43.892 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.484839+0000", 2020-09-08T15:37:43.892 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000104131 2020-09-08T15:37:43.892 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.893 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.893 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:37:43.893 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.484943+0000", 2020-09-08T15:37:43.893 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2139999999999996e-06 2020-09-08T15:37:43.893 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.894 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.894 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:37:43.894 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.484951+0000", 2020-09-08T15:37:43.894 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6605000000000001e-05 2020-09-08T15:37:43.894 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.895 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.895 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:37:43.895 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.484968+0000", 2020-09-08T15:37:43.895 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.322e-06 2020-09-08T15:37:43.895 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.896 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.896 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:37:43.896 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.484973+0000", 2020-09-08T15:37:43.896 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014600299999999999 2020-09-08T15:37:43.896 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.896 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.897 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:37:43.897 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.485119+0000", 2020-09-08T15:37:43.897 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1183e-05 2020-09-08T15:37:43.897 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.897 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.898 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:37:43.898 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.485140+0000", 2020-09-08T15:37:43.898 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003717873 2020-09-08T15:37:43.898 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.898 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.899 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:37:43.899 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.488858+0000", 2020-09-08T15:37:43.899 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5737999999999999e-05 2020-09-08T15:37:43.899 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.899 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.899 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:37:43.900 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.488934+0000", 2020-09-08T15:37:43.900 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0040950689999999998 2020-09-08T15:37:43.900 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.900 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.900 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.901 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.901 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.901 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:3416 2.4 e133/126 2:2d4370b7:::benchmark_data_smithi166_20551_object3415:head v 133'209, mlcod=133'206)", 2020-09-08T15:37:43.901 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:43.660675+0000", 2020-09-08T15:37:43.901 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.077127351999999996, 2020-09-08T15:37:43.902 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0040942299999999999, 2020-09-08T15:37:43.902 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:37:43.902 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:37:43.902 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:37:43.902 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.903 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:37:43.903 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.660675+0000", 2020-09-08T15:37:43.903 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:37:43.903 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.903 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.903 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:37:43.904 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.660675+0000", 2020-09-08T15:37:43.904 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003336955 2020-09-08T15:37:43.904 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.904 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.904 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:37:43.905 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.664012+0000", 2020-09-08T15:37:43.905 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6819999999999992e-06 2020-09-08T15:37:43.905 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.905 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.905 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:37:43.906 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.664021+0000", 2020-09-08T15:37:43.906 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3073e-05 2020-09-08T15:37:43.906 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.906 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.906 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:37:43.906 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.664034+0000", 2020-09-08T15:37:43.907 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9351000000000002e-05 2020-09-08T15:37:43.907 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.907 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.907 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:37:43.907 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.664053+0000", 2020-09-08T15:37:43.908 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018093699999999999 2020-09-08T15:37:43.908 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.908 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.908 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:37:43.908 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.664234+0000", 2020-09-08T15:37:43.909 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4442000000000001e-05 2020-09-08T15:37:43.909 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.909 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.909 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:37:43.909 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.664259+0000", 2020-09-08T15:37:43.909 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000435038 2020-09-08T15:37:43.910 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.910 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.910 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:37:43.910 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.664694+0000", 2020-09-08T15:37:43.910 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5752000000000002e-05 2020-09-08T15:37:43.911 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.911 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.911 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:37:43.911 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.664769+0000", 2020-09-08T15:37:43.911 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0040942299999999999 2020-09-08T15:37:43.912 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.912 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.912 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.912 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.912 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.912 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:3460 2.4 e133/126 2:2e1784df:::benchmark_data_smithi166_20551_object3459:head v 133'212, mlcod=133'208)", 2020-09-08T15:37:43.913 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:43.672020+0000", 2020-09-08T15:37:43.913 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.065782594999999999, 2020-09-08T15:37:43.913 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0088729910000000002, 2020-09-08T15:37:43.913 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:37:43.913 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:37:43.914 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:37:43.914 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.914 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:37:43.914 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.672020+0000", 2020-09-08T15:37:43.914 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:37:43.915 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.915 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.915 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:37:43.915 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.672020+0000", 2020-09-08T15:37:43.915 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000106562 2020-09-08T15:37:43.915 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.916 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.916 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:37:43.916 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.672126+0000", 2020-09-08T15:37:43.916 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6850000000000007e-06 2020-09-08T15:37:43.916 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.917 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.917 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:37:43.917 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.672135+0000", 2020-09-08T15:37:43.917 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3509000000000001e-05 2020-09-08T15:37:43.917 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.918 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.918 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:37:43.918 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.672149+0000", 2020-09-08T15:37:43.918 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7629999999999997e-06 2020-09-08T15:37:43.918 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.918 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.919 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:37:43.919 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.672154+0000", 2020-09-08T15:37:43.919 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016609400000000001 2020-09-08T15:37:43.919 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.919 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.920 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:37:43.920 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.672321+0000", 2020-09-08T15:37:43.920 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9176999999999999e-05 2020-09-08T15:37:43.920 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.920 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.921 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:37:43.921 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.672350+0000", 2020-09-08T15:37:43.921 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0084803510000000006 2020-09-08T15:37:43.921 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.921 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.921 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:37:43.922 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.680830+0000", 2020-09-08T15:37:43.922 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2849999999999996e-05 2020-09-08T15:37:43.922 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.922 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.922 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:37:43.923 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.680893+0000", 2020-09-08T15:37:43.923 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0088729910000000002 2020-09-08T15:37:43.923 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.923 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.923 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.924 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.924 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.924 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:3462 2.4 e133/126 2:22d8f9f9:::benchmark_data_smithi166_20551_object3461:head v 133'213, mlcod=133'208)", 2020-09-08T15:37:43.924 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:43.673869+0000", 2020-09-08T15:37:43.924 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.063933100000000007, 2020-09-08T15:37:43.924 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0070781280000000004, 2020-09-08T15:37:43.925 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:37:43.925 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:37:43.925 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:37:43.925 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.925 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:37:43.926 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.673869+0000", 2020-09-08T15:37:43.926 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:37:43.926 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.926 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.926 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:37:43.927 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.673869+0000", 2020-09-08T15:37:43.927 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6115999999999999e-05 2020-09-08T15:37:43.927 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.927 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.927 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:37:43.928 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.673966+0000", 2020-09-08T15:37:43.928 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2949999999999999e-06 2020-09-08T15:37:43.928 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.928 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.928 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:37:43.928 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.673975+0000", 2020-09-08T15:37:43.929 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3833e-05 2020-09-08T15:37:43.929 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.929 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.929 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:37:43.929 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.673989+0000", 2020-09-08T15:37:43.930 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.844e-06 2020-09-08T15:37:43.930 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.930 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.930 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:37:43.930 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.673995+0000", 2020-09-08T15:37:43.931 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00029272899999999998 2020-09-08T15:37:43.931 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.931 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.931 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:37:43.931 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.674288+0000", 2020-09-08T15:37:43.932 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4958000000000001e-05 2020-09-08T15:37:43.932 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.932 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.932 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:37:43.932 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.674313+0000", 2020-09-08T15:37:43.932 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0065822779999999996 2020-09-08T15:37:43.933 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.933 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.933 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:37:43.933 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.680895+0000", 2020-09-08T15:37:43.933 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2074999999999999e-05 2020-09-08T15:37:43.934 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.934 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.934 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:37:43.934 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.680948+0000", 2020-09-08T15:37:43.934 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0070781280000000004 2020-09-08T15:37:43.935 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.935 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.935 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.935 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.935 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.935 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:3471 2.5 e133/126 2:a689eb90:::benchmark_data_smithi166_20551_object3470:head v 133'226, mlcod=133'224)", 2020-09-08T15:37:43.936 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:43.674114+0000", 2020-09-08T15:37:43.936 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.063688821000000007, 2020-09-08T15:37:43.936 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0067684620000000003, 2020-09-08T15:37:43.936 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:37:43.936 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:37:43.937 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:37:43.937 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.937 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:37:43.937 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.674114+0000", 2020-09-08T15:37:43.937 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:37:43.938 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.938 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.938 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:37:43.938 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.674114+0000", 2020-09-08T15:37:43.938 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6890000000000002e-05 2020-09-08T15:37:43.939 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.939 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.939 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:37:43.939 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.674211+0000", 2020-09-08T15:37:43.939 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4489999999999999e-06 2020-09-08T15:37:43.940 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.940 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.940 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:37:43.941 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.674219+0000", 2020-09-08T15:37:43.941 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.398e-05 2020-09-08T15:37:43.941 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.941 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.941 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:37:43.942 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.674233+0000", 2020-09-08T15:37:43.942 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2909999999999998e-06 2020-09-08T15:37:43.942 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.942 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.942 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:37:43.943 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.674238+0000", 2020-09-08T15:37:43.943 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00041336299999999998 2020-09-08T15:37:43.943 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.943 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.943 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:37:43.944 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.674652+0000", 2020-09-08T15:37:43.944 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2317e-05 2020-09-08T15:37:43.944 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.944 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.944 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:37:43.944 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.674674+0000", 2020-09-08T15:37:43.945 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0061348440000000004 2020-09-08T15:37:43.945 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.945 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.945 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:37:43.945 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.680809+0000", 2020-09-08T15:37:43.946 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3328e-05 2020-09-08T15:37:43.946 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.946 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.946 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:37:43.946 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.680882+0000", 2020-09-08T15:37:43.947 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0067684620000000003 2020-09-08T15:37:43.947 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.947 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.947 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.947 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.948 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.948 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:3473 2.c e133/126 2:33b63706:::benchmark_data_smithi166_20551_object3472:head v 133'208, mlcod=133'206)", 2020-09-08T15:37:43.948 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:43.674966+0000", 2020-09-08T15:37:43.948 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.062836562999999998, 2020-09-08T15:37:43.948 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0060556250000000002, 2020-09-08T15:37:43.948 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:37:43.949 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:37:43.949 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:37:43.949 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.949 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:37:43.949 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.674966+0000", 2020-09-08T15:37:43.950 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:37:43.950 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.950 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.950 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:37:43.950 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.674966+0000", 2020-09-08T15:37:43.951 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000114448 2020-09-08T15:37:43.951 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.951 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.951 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:37:43.951 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.675080+0000", 2020-09-08T15:37:43.951 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0641e-05 2020-09-08T15:37:43.952 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.952 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.952 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:37:43.952 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.675091+0000", 2020-09-08T15:37:43.952 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6047999999999999e-05 2020-09-08T15:37:43.953 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.953 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.953 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:37:43.953 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.675107+0000", 2020-09-08T15:37:43.953 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7470000000000002e-06 2020-09-08T15:37:43.954 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.954 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.954 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:37:43.954 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.675113+0000", 2020-09-08T15:37:43.954 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8999000000000002e-05 2020-09-08T15:37:43.954 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.955 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.955 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:37:43.955 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.675172+0000", 2020-09-08T15:37:43.955 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.5473000000000002e-05 2020-09-08T15:37:43.955 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.956 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.956 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:37:43.956 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.675197+0000", 2020-09-08T15:37:43.956 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0057526330000000001 2020-09-08T15:37:43.956 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.957 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.957 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:37:43.957 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.680950+0000", 2020-09-08T15:37:43.957 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1636000000000003e-05 2020-09-08T15:37:43.957 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.957 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.958 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:37:43.958 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.681022+0000", 2020-09-08T15:37:43.958 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0060556250000000002 2020-09-08T15:37:43.958 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.958 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.959 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.959 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.959 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.959 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:3477 2.c e133/126 2:3a038eab:::benchmark_data_smithi166_20551_object3476:head v 133'209, mlcod=133'206)", 2020-09-08T15:37:43.959 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:43.675912+0000", 2020-09-08T15:37:43.960 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.061890292999999999, 2020-09-08T15:37:43.960 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0051648670000000001, 2020-09-08T15:37:43.960 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:37:43.960 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:37:43.960 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:37:43.960 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.961 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:37:43.961 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.675912+0000", 2020-09-08T15:37:43.961 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:37:43.961 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.961 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.962 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:37:43.962 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.675912+0000", 2020-09-08T15:37:43.962 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000101116 2020-09-08T15:37:43.962 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.962 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.963 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:37:43.963 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676013+0000", 2020-09-08T15:37:43.963 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0144e-05 2020-09-08T15:37:43.963 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.963 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.964 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:37:43.964 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676023+0000", 2020-09-08T15:37:43.964 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3827e-05 2020-09-08T15:37:43.964 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.964 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.964 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:37:43.965 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676037+0000", 2020-09-08T15:37:43.965 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7389999999999996e-06 2020-09-08T15:37:43.965 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.965 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.965 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:37:43.966 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676043+0000", 2020-09-08T15:37:43.966 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00041216000000000001 2020-09-08T15:37:43.966 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.966 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.966 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:37:43.967 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676455+0000", 2020-09-08T15:37:43.967 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2489000000000001e-05 2020-09-08T15:37:43.967 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.967 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.967 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:37:43.967 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676478+0000", 2020-09-08T15:37:43.968 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0045462719999999996 2020-09-08T15:37:43.968 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.968 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.968 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:37:43.968 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.681024+0000", 2020-09-08T15:37:43.969 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.312e-05 2020-09-08T15:37:43.969 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.969 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.969 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:37:43.969 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.681077+0000", 2020-09-08T15:37:43.970 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0051648670000000001 2020-09-08T15:37:43.970 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.970 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.970 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.970 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.970 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.971 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:3464 2.4 e133/126 2:2c144718:::benchmark_data_smithi166_20551_object3463:head v 133'214, mlcod=133'210)", 2020-09-08T15:37:43.971 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:43.676016+0000", 2020-09-08T15:37:43.971 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.061786951999999999, 2020-09-08T15:37:43.971 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0051109119999999996, 2020-09-08T15:37:43.971 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:37:43.972 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:37:43.972 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:37:43.972 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.972 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:37:43.972 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676016+0000", 2020-09-08T15:37:43.973 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:37:43.973 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.973 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.973 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:37:43.973 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676016+0000", 2020-09-08T15:37:43.973 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8850000000000005e-05 2020-09-08T15:37:43.974 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.974 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.974 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:37:43.974 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676104+0000", 2020-09-08T15:37:43.974 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.225e-06 2020-09-08T15:37:43.975 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.975 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.975 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:37:43.975 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676113+0000", 2020-09-08T15:37:43.975 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2629000000000001e-05 2020-09-08T15:37:43.976 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.976 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.976 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:37:43.976 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676125+0000", 2020-09-08T15:37:43.976 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9259999999999997e-06 2020-09-08T15:37:43.976 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.977 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.977 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:37:43.977 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676131+0000", 2020-09-08T15:37:43.977 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00066743499999999997 2020-09-08T15:37:43.977 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.978 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.978 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:37:43.978 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676799+0000", 2020-09-08T15:37:43.978 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2966999999999999e-05 2020-09-08T15:37:43.978 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.978 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.979 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:37:43.979 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676822+0000", 2020-09-08T15:37:43.979 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0042573539999999997 2020-09-08T15:37:43.979 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.979 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.980 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:37:43.980 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.681079+0000", 2020-09-08T15:37:43.980 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7525999999999999e-05 2020-09-08T15:37:43.980 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.980 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.981 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:37:43.981 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.681126+0000", 2020-09-08T15:37:43.981 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0051109119999999996 2020-09-08T15:37:43.981 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.981 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.981 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.982 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.982 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.982 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:3479 2.5 e133/126 2:a9d30284:::benchmark_data_smithi166_20551_object3478:head v 133'227, mlcod=133'224)", 2020-09-08T15:37:43.982 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:43.676239+0000", 2020-09-08T15:37:43.982 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.061563110999999997, 2020-09-08T15:37:43.983 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0047039990000000004, 2020-09-08T15:37:43.983 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:37:43.983 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:37:43.983 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:37:43.983 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.984 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:37:43.984 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676239+0000", 2020-09-08T15:37:43.984 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:37:43.984 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.984 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.984 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:37:43.985 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676239+0000", 2020-09-08T15:37:43.985 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8936000000000004e-05 2020-09-08T15:37:43.985 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.985 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.985 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:37:43.986 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676328+0000", 2020-09-08T15:37:43.986 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4050000000000003e-06 2020-09-08T15:37:43.986 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.986 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.986 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:37:43.987 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676336+0000", 2020-09-08T15:37:43.987 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2539e-05 2020-09-08T15:37:43.987 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.987 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.987 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:37:43.987 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676348+0000", 2020-09-08T15:37:43.988 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3430000000000003e-06 2020-09-08T15:37:43.988 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.988 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.988 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:37:43.988 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676354+0000", 2020-09-08T15:37:43.989 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4143e-05 2020-09-08T15:37:43.989 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.989 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.989 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:37:43.989 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676408+0000", 2020-09-08T15:37:43.989 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8520000000000001e-05 2020-09-08T15:37:43.990 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.990 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.990 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:37:43.990 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.676436+0000", 2020-09-08T15:37:43.990 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0044509040000000003 2020-09-08T15:37:43.991 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.991 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.991 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:37:43.991 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.680887+0000", 2020-09-08T15:37:43.991 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6208999999999997e-05 2020-09-08T15:37:43.992 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:37:43.992 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:37:43.992 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:37:43.992 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:43.680943+0000", 2020-09-08T15:37:43.993 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0047039990000000004 2020-09-08T15:37:43.993 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.993 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.993 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.993 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:37:43.994 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:37:43.994 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:37:43.994 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:37:43.995 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 1 16 3805 3789 236.8 236.812 0.00562168 0.00420567 2020-09-08T15:37:44.004 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:43 smithi055 bash[9296]: cluster 2020-09-08T15:37:43.592675+0000 mgr.x (mgr.24118) 306 : cluster [DBG] pgmap v361: 17 pgs: 17 active+clean; 22 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s wr, 72 op/s 2020-09-08T15:37:44.114 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:37:43 smithi166 bash[18960]: debug 2020-09-08T15:37:43.713+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:44.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:43 smithi166 bash[9446]: cluster 2020-09-08T15:37:43.592675+0000 mgr.x (mgr.24118) 306 : cluster [DBG] pgmap v361: 17 pgs: 17 active+clean; 22 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s wr, 72 op/s 2020-09-08T15:37:44.143 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:37:44.246 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:37:44.371 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:37:44.417 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:37:44 smithi055 bash[16646]: debug 2020-09-08T15:37:44.138+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:44.475 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:44.586 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:44.614 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:37:44 smithi166 bash[14254]: debug 2020-09-08T15:37:44.373+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:44.694 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:37:44.769 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 2 15 7244 7229 225.88 215 0.00472533 0.00442132 2020-09-08T15:37:44.794 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2020-09-08T15:37:44.795 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-09-08T15:37:44.795 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.000000 2020-09-08T15:37:44.795 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd primary-affinity 4 0 2020-09-08T15:37:44.812 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:37:44 smithi055 bash[19003]: debug 2020-09-08T15:37:44.586+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:44.813 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:37:44.917 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:44.983 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:37:45.126 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:37:45.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:37:44 smithi055 bash[14301]: debug 2020-09-08T15:37:44.810+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:45.308 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:37:45.769 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 3 16 11083 11067 230.538 239.875 0.00200728 0.00432795 2020-09-08T15:37:46.767 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 4 16 14710 14694 229.571 226.688 0.00541792 0.0043519 2020-09-08T15:37:46.789 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:46.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:46 smithi166 bash[9446]: cluster 2020-09-08T15:37:45.031238+0000 osd.4 (osd.4) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:46.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:46 smithi166 bash[9446]: cluster 2020-09-08T15:37:45.031314+0000 osd.4 (osd.4) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:46.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:46 smithi166 bash[9446]: cluster 2020-09-08T15:37:45.558802+0000 mon.a (mon.0) 924 : cluster [DBG] osdmap e134: 12 total, 12 up, 12 in 2020-09-08T15:37:46.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:46 smithi166 bash[9446]: cluster 2020-09-08T15:37:45.593124+0000 mgr.x (mgr.24118) 307 : cluster [DBG] pgmap v363: 17 pgs: 17 active+clean; 22 MiB data, 183 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s wr, 60 op/s 2020-09-08T15:37:46.897 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:37:46.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:46 smithi055 bash[9296]: cluster 2020-09-08T15:37:45.031238+0000 osd.4 (osd.4) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:46.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:46 smithi055 bash[9296]: cluster 2020-09-08T15:37:45.031314+0000 osd.4 (osd.4) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:46.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:46 smithi055 bash[9296]: cluster 2020-09-08T15:37:45.558802+0000 mon.a (mon.0) 924 : cluster [DBG] osdmap e134: 12 total, 12 up, 12 in 2020-09-08T15:37:46.912 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:46 smithi055 bash[9296]: cluster 2020-09-08T15:37:45.593124+0000 mgr.x (mgr.24118) 307 : cluster [DBG] pgmap v363: 17 pgs: 17 active+clean; 22 MiB data, 183 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s wr, 60 op/s 2020-09-08T15:37:46.927 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:37:46 smithi104 bash[22305]: debug 2020-09-08T15:37:46.790+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:46.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:46 smithi104 bash[9203]: cluster 2020-09-08T15:37:45.031238+0000 osd.4 (osd.4) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:46.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:46 smithi104 bash[9203]: cluster 2020-09-08T15:37:45.031314+0000 osd.4 (osd.4) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:46.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:46 smithi104 bash[9203]: cluster 2020-09-08T15:37:45.558802+0000 mon.a (mon.0) 924 : cluster [DBG] osdmap e134: 12 total, 12 up, 12 in 2020-09-08T15:37:46.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:46 smithi104 bash[9203]: cluster 2020-09-08T15:37:45.593124+0000 mgr.x (mgr.24118) 307 : cluster [DBG] pgmap v363: 17 pgs: 17 active+clean; 22 MiB data, 183 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s wr, 60 op/s 2020-09-08T15:37:46.996 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:37:47.102 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:47.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:37:47 smithi055 bash[14301]: debug 2020-09-08T15:37:46.994+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:47.770 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 5 16 18295 18279 228.465 224.062 0.00545071 0.00437352 2020-09-08T15:37:47.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:47 smithi104 bash[9203]: cluster 2020-09-08T15:37:47.593734+0000 mgr.x (mgr.24118) 308 : cluster [DBG] pgmap v364: 17 pgs: 17 active+clean; 631 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.46k op/s 2020-09-08T15:37:48.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:47 smithi166 bash[9446]: cluster 2020-09-08T15:37:47.593734+0000 mgr.x (mgr.24118) 308 : cluster [DBG] pgmap v364: 17 pgs: 17 active+clean; 631 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.46k op/s 2020-09-08T15:37:48.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:47 smithi055 bash[9296]: cluster 2020-09-08T15:37:47.593734+0000 mgr.x (mgr.24118) 308 : cluster [DBG] pgmap v364: 17 pgs: 17 active+clean; 631 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.46k op/s 2020-09-08T15:37:48.421 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2020-09-08T15:37:48.608 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:37:48.643 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:48.677 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:37:48 smithi104 bash[19922]: debug 2020-09-08T15:37:48.634+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:48.744 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:48.767 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 6 15 21682 21667 225.677 211.75 0.00362528 0.00441953 2020-09-08T15:37:49.683 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:49.767 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 7 16 23797 23781 212.311 132.125 0.00609203 0.00470709 2020-09-08T15:37:49.788 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:49.920 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:50.023 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:37:50.132 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:37:50.132 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:37:49 smithi055 bash[19003]: debug 2020-09-08T15:37:49.918+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:50.259 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:50.356 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:50.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:37:50 smithi055 bash[19003]: debug 2020-09-08T15:37:50.354+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:50.408 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:37:50 smithi055 bash[14301]: debug 2020-09-08T15:37:50.130+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:50.459 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:37:50.465 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:37:49 smithi104 bash[22305]: debug 2020-09-08T15:37:49.682+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:50.587 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:37:50.692 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:50.767 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 8 16 26765 26749 208.957 185.5 0.00292025 0.00478293 2020-09-08T15:37:50.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:50 smithi166 bash[9446]: cluster 2020-09-08T15:37:49.594214+0000 mgr.x (mgr.24118) 309 : cluster [DBG] pgmap v365: 17 pgs: 17 active+clean; 979 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 122 MiB/s wr, 1.96k op/s 2020-09-08T15:37:50.865 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:37:50 smithi166 bash[11928]: debug 2020-09-08T15:37:50.589+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:50.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:50 smithi055 bash[9296]: cluster 2020-09-08T15:37:49.594214+0000 mgr.x (mgr.24118) 309 : cluster [DBG] pgmap v365: 17 pgs: 17 active+clean; 979 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 122 MiB/s wr, 1.96k op/s 2020-09-08T15:37:50.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:50 smithi104 bash[9203]: cluster 2020-09-08T15:37:49.594214+0000 mgr.x (mgr.24118) 309 : cluster [DBG] pgmap v365: 17 pgs: 17 active+clean; 979 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 122 MiB/s wr, 1.96k op/s 2020-09-08T15:37:51.049 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:51.152 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:37:51.272 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:37:51.333 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:37:51 smithi104 bash[22305]: debug 2020-09-08T15:37:51.050+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:51.375 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:37:51.659 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:37:51 smithi055 bash[11956]: debug 2020-09-08T15:37:51.270+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:51.672 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:37:51.769 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 9 16 29709 29693 206.182 184 0.00583566 0.00484738 2020-09-08T15:37:51.781 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:37:51.902 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:37:51.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:51 smithi104 bash[9203]: cluster 2020-09-08T15:37:51.594723+0000 mgr.x (mgr.24118) 310 : cluster [DBG] pgmap v366: 17 pgs: 17 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s wr, 2.06k op/s 2020-09-08T15:37:51.927 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:37:51 smithi104 bash[15273]: debug 2020-09-08T15:37:51.674+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:52.008 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:37:52.019 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:51 smithi055 bash[9296]: cluster 2020-09-08T15:37:51.594723+0000 mgr.x (mgr.24118) 310 : cluster [DBG] pgmap v366: 17 pgs: 17 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s wr, 2.06k op/s 2020-09-08T15:37:52.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:51 smithi166 bash[9446]: cluster 2020-09-08T15:37:51.594723+0000 mgr.x (mgr.24118) 310 : cluster [DBG] pgmap v366: 17 pgs: 17 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s wr, 2.06k op/s 2020-09-08T15:37:52.115 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:37:51 smithi166 bash[14254]: debug 2020-09-08T15:37:51.901+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:52.158 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:37:52.261 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:37:52.378 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:37:52.408 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:37:52 smithi055 bash[11956]: debug 2020-09-08T15:37:52.158+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:52.481 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:52.591 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:52.591 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:37:52 smithi166 bash[14254]: debug 2020-09-08T15:37:52.373+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:52.591 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:37:52 smithi166 bash[18960]: debug 2020-09-08T15:37:52.589+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:52.697 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:52.768 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 10 16 32478 32462 202.868 173.062 0.00262262 0.00491152 2020-09-08T15:37:53.051 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:53.154 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:53.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:37:53 smithi055 bash[19003]: debug 2020-09-08T15:37:53.042+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:53.768 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 11 16 35004 34988 198.777 157.875 0.00868769 0.00502824 2020-09-08T15:37:53.899 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:53 smithi104 bash[9203]: cluster 2020-09-08T15:37:52.516455+0000 osd.9 (osd.9) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:53.900 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:53 smithi104 bash[9203]: cluster 2020-09-08T15:37:52.516533+0000 osd.9 (osd.9) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:53.900 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:53 smithi104 bash[9203]: cluster 2020-09-08T15:37:52.677045+0000 mon.a (mon.0) 925 : cluster [DBG] osdmap e135: 12 total, 12 up, 12 in 2020-09-08T15:37:53.900 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:53 smithi104 bash[9203]: cluster 2020-09-08T15:37:53.595220+0000 mgr.x (mgr.24118) 311 : cluster [DBG] pgmap v368: 17 pgs: 17 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 234 MiB/s wr, 3.74k op/s 2020-09-08T15:37:53.932 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:54.033 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:37:54.043 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:53 smithi055 bash[9296]: cluster 2020-09-08T15:37:52.516455+0000 osd.9 (osd.9) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:54.044 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:53 smithi055 bash[9296]: cluster 2020-09-08T15:37:52.516533+0000 osd.9 (osd.9) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:54.044 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:53 smithi055 bash[9296]: cluster 2020-09-08T15:37:52.677045+0000 mon.a (mon.0) 925 : cluster [DBG] osdmap e135: 12 total, 12 up, 12 in 2020-09-08T15:37:54.044 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:53 smithi055 bash[9296]: cluster 2020-09-08T15:37:53.595220+0000 mgr.x (mgr.24118) 311 : cluster [DBG] pgmap v368: 17 pgs: 17 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 234 MiB/s wr, 3.74k op/s 2020-09-08T15:37:54.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:53 smithi166 bash[9446]: cluster 2020-09-08T15:37:52.516455+0000 osd.9 (osd.9) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:54.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:53 smithi166 bash[9446]: cluster 2020-09-08T15:37:52.516533+0000 osd.9 (osd.9) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:54.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:53 smithi166 bash[9446]: cluster 2020-09-08T15:37:52.677045+0000 mon.a (mon.0) 925 : cluster [DBG] osdmap e135: 12 total, 12 up, 12 in 2020-09-08T15:37:54.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:53 smithi166 bash[9446]: cluster 2020-09-08T15:37:53.595220+0000 mgr.x (mgr.24118) 311 : cluster [DBG] pgmap v368: 17 pgs: 17 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 234 MiB/s wr, 3.74k op/s 2020-09-08T15:37:54.162 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:37:54.177 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:37:53 smithi104 bash[22305]: debug 2020-09-08T15:37:53.926+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:54.266 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:37:54.385 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:37:54 smithi055 bash[16646]: debug 2020-09-08T15:37:54.162+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:54.387 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:37:54.491 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:37:54.613 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:37:54.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:37:54 smithi055 bash[16646]: debug 2020-09-08T15:37:54.382+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:54.717 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:37:54.768 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 12 16 36946 36930 192.326 121.375 0.00470775 0.00519605 2020-09-08T15:37:54.865 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:37:54 smithi166 bash[14254]: debug 2020-09-08T15:37:54.613+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:55.768 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 13 16 39524 39508 189.925 161.125 0.00383154 0.00526252 2020-09-08T15:37:56.078 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:37:56.180 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:56.283 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:37:56.369 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:37:56 smithi166 bash[11928]: debug 2020-09-08T15:37:56.077+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:56.369 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:37:56 smithi166 bash[18960]: debug 2020-09-08T15:37:56.285+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:56.391 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:37:56.545 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:37:56.648 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:37:56.658 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:37:56 smithi055 bash[11956]: debug 2020-09-08T15:37:56.542+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:56.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:56 smithi055 bash[9296]: cluster 2020-09-08T15:37:55.456625+0000 osd.2 (osd.2) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:56.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:56 smithi055 bash[9296]: cluster 2020-09-08T15:37:55.456703+0000 osd.2 (osd.2) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:56.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:56 smithi055 bash[9296]: cluster 2020-09-08T15:37:55.566534+0000 mon.a (mon.0) 926 : cluster [DBG] osdmap e136: 12 total, 12 up, 12 in 2020-09-08T15:37:56.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:56 smithi055 bash[9296]: cluster 2020-09-08T15:37:55.595676+0000 mgr.x (mgr.24118) 312 : cluster [DBG] pgmap v370: 17 pgs: 17 active+clean; 1.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 159 MiB/s wr, 2.54k op/s 2020-09-08T15:37:56.768 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 14 16 42046 42030 187.617 157.625 0.0065477 0.00532742 2020-09-08T15:37:56.770 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:37:56.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:56 smithi166 bash[9446]: cluster 2020-09-08T15:37:55.456625+0000 osd.2 (osd.2) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:56.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:56 smithi166 bash[9446]: cluster 2020-09-08T15:37:55.456703+0000 osd.2 (osd.2) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:56.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:56 smithi166 bash[9446]: cluster 2020-09-08T15:37:55.566534+0000 mon.a (mon.0) 926 : cluster [DBG] osdmap e136: 12 total, 12 up, 12 in 2020-09-08T15:37:56.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:56 smithi166 bash[9446]: cluster 2020-09-08T15:37:55.595676+0000 mgr.x (mgr.24118) 312 : cluster [DBG] pgmap v370: 17 pgs: 17 active+clean; 1.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 159 MiB/s wr, 2.54k op/s 2020-09-08T15:37:56.876 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:56.886 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:56 smithi104 bash[9203]: cluster 2020-09-08T15:37:55.456625+0000 osd.2 (osd.2) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:37:56.886 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:56 smithi104 bash[9203]: cluster 2020-09-08T15:37:55.456703+0000 osd.2 (osd.2) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:37:56.887 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:56 smithi104 bash[9203]: cluster 2020-09-08T15:37:55.566534+0000 mon.a (mon.0) 926 : cluster [DBG] osdmap e136: 12 total, 12 up, 12 in 2020-09-08T15:37:56.887 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:56 smithi104 bash[9203]: cluster 2020-09-08T15:37:55.595676+0000 mgr.x (mgr.24118) 312 : cluster [DBG] pgmap v370: 17 pgs: 17 active+clean; 1.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 159 MiB/s wr, 2.54k op/s 2020-09-08T15:37:56.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:37:56 smithi055 bash[16646]: debug 2020-09-08T15:37:56.770+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:57.738 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:37:57.768 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 15 16 44505 44489 185.354 153.688 0.00685889 0.00539219 2020-09-08T15:37:57.847 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:37:57.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:37:57 smithi104 bash[9203]: cluster 2020-09-08T15:37:57.596198+0000 mgr.x (mgr.24118) 313 : cluster [DBG] pgmap v371: 17 pgs: 17 active+clean; 2.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 182 MiB/s wr, 2.92k op/s 2020-09-08T15:37:57.927 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:37:57 smithi104 bash[19922]: debug 2020-09-08T15:37:57.738+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:57.981 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:37:57 smithi055 bash[9296]: cluster 2020-09-08T15:37:57.596198+0000 mgr.x (mgr.24118) 313 : cluster [DBG] pgmap v371: 17 pgs: 17 active+clean; 2.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 182 MiB/s wr, 2.92k op/s 2020-09-08T15:37:57.981 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:37:58.088 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:58.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:37:57 smithi166 bash[9446]: cluster 2020-09-08T15:37:57.596198+0000 mgr.x (mgr.24118) 313 : cluster [DBG] pgmap v371: 17 pgs: 17 active+clean; 2.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 182 MiB/s wr, 2.92k op/s 2020-09-08T15:37:58.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:37:57 smithi055 bash[16646]: debug 2020-09-08T15:37:57.978+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:58.768 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 16 16 46163 46147 180.245 103.625 0.00617243 0.00554596 2020-09-08T15:37:58.851 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:37:58.954 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:37:59.353 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:37:58 smithi104 bash[22305]: debug 2020-09-08T15:37:58.850+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:37:59.768 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 17 16 48364 48348 177.734 137.562 0.00774525 0.00562413 2020-09-08T15:38:00.768 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 18 16 50128 50112 173.984 110.25 0.00878493 0.00574539 2020-09-08T15:38:00.815 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:00 smithi104 bash[9203]: cluster 2020-09-08T15:37:59.596739+0000 mgr.x (mgr.24118) 314 : cluster [DBG] pgmap v372: 17 pgs: 17 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 MiB/s wr, 2.79k op/s 2020-09-08T15:38:00.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:00 smithi166 bash[9446]: cluster 2020-09-08T15:37:59.596739+0000 mgr.x (mgr.24118) 314 : cluster [DBG] pgmap v372: 17 pgs: 17 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 MiB/s wr, 2.79k op/s 2020-09-08T15:38:00.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:00 smithi055 bash[9296]: cluster 2020-09-08T15:37:59.596739+0000 mgr.x (mgr.24118) 314 : cluster [DBG] pgmap v372: 17 pgs: 17 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 MiB/s wr, 2.79k op/s 2020-09-08T15:38:01.008 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:38:01.062 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:38:01.106 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:38:01.153 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:38:01.154 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:38:01.154 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:38:01.192 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:38:01.237 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:38:01.274 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:38:01.299 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:38:01.456 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:38:01.518 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:38:01.554 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:38:01.585 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:38:01.585 INFO:teuthology.orchestra.run.smithi104.stderr:set osd.4 primary-affinity to 0 (802) 2020-09-08T15:38:01.597 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:38:01.602 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:38:01.603 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:38:01.603 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:38:01.603 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:38:01.603 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:38:01.604 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:38:01.604 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:38:01.604 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:38:01.604 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:38:01.604 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:38:01.605 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:38:01.605 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:38:01.768 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 19 16 51690 51674 169.965 97.625 0.00982358 0.00588134 2020-09-08T15:38:01.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:01 smithi166 bash[9446]: audit 2020-09-08T15:38:00.574516+0000 mon.a (mon.0) 927 : audit [INF] from='client.? v1:172.21.15.104:0/137487915' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:38:01.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:01 smithi166 bash[9446]: audit 2020-09-08T15:38:00.967936+0000 mon.a (mon.0) 928 : audit [DBG] from='client.? v1:172.21.15.104:0/1756253724' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:38:01.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:01 smithi166 bash[9446]: audit 2020-09-08T15:38:01.235126+0000 mon.a (mon.0) 929 : audit [INF] from='client.34754 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 0.0}]: dispatch 2020-09-08T15:38:01.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:01 smithi166 bash[9446]: audit 2020-09-08T15:38:01.235531+0000 mon.c (mon.2) 33 : audit [INF] from='client.? v1:172.21.15.104:0/4118863488' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 0.0}]: dispatch 2020-09-08T15:38:01.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:01 smithi055 bash[9296]: audit 2020-09-08T15:38:00.574516+0000 mon.a (mon.0) 927 : audit [INF] from='client.? v1:172.21.15.104:0/137487915' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:38:01.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:01 smithi055 bash[9296]: audit 2020-09-08T15:38:00.967936+0000 mon.a (mon.0) 928 : audit [DBG] from='client.? v1:172.21.15.104:0/1756253724' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:38:01.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:01 smithi055 bash[9296]: audit 2020-09-08T15:38:01.235126+0000 mon.a (mon.0) 929 : audit [INF] from='client.34754 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 0.0}]: dispatch 2020-09-08T15:38:01.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:01 smithi055 bash[9296]: audit 2020-09-08T15:38:01.235531+0000 mon.c (mon.2) 33 : audit [INF] from='client.? v1:172.21.15.104:0/4118863488' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 0.0}]: dispatch 2020-09-08T15:38:01.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:01 smithi104 bash[9203]: audit 2020-09-08T15:38:00.574516+0000 mon.a (mon.0) 927 : audit [INF] from='client.? v1:172.21.15.104:0/137487915' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:38:01.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:01 smithi104 bash[9203]: audit 2020-09-08T15:38:00.967936+0000 mon.a (mon.0) 928 : audit [DBG] from='client.? v1:172.21.15.104:0/1756253724' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:38:01.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:01 smithi104 bash[9203]: audit 2020-09-08T15:38:01.235126+0000 mon.a (mon.0) 929 : audit [INF] from='client.34754 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 0.0}]: dispatch 2020-09-08T15:38:01.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:01 smithi104 bash[9203]: audit 2020-09-08T15:38:01.235531+0000 mon.c (mon.2) 33 : audit [INF] from='client.? v1:172.21.15.104:0/4118863488' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 0.0}]: dispatch 2020-09-08T15:38:02.769 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 15:38:02.775179 min lat: 0.00138961 max lat: 0.356103 avg lat: 0.00593324 2020-09-08T15:38:02.769 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:38:02.769 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 20 16 53927 53911 168.457 139.812 0.0203327 0.00593324 2020-09-08T15:38:02.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:02 smithi166 bash[9446]: cluster 2020-09-08T15:38:01.567542+0000 mon.a (mon.0) 930 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:38:02.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:02 smithi166 bash[9446]: audit 2020-09-08T15:38:01.578510+0000 mon.a (mon.0) 931 : audit [INF] from='client.? v1:172.21.15.104:0/137487915' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:38:02.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:02 smithi166 bash[9446]: audit 2020-09-08T15:38:01.578707+0000 mon.a (mon.0) 932 : audit [INF] from='client.34754 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 0.0}]': finished 2020-09-08T15:38:02.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:02 smithi166 bash[9446]: cluster 2020-09-08T15:38:01.578745+0000 mon.a (mon.0) 933 : cluster [DBG] osdmap e137: 12 total, 12 up, 12 in 2020-09-08T15:38:02.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:02 smithi166 bash[9446]: cluster 2020-09-08T15:38:01.597142+0000 mgr.x (mgr.24118) 315 : cluster [DBG] pgmap v374: 17 pgs: 17 active+clean; 2.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 120 MiB/s wr, 1.92k op/s 2020-09-08T15:38:02.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:02 smithi055 bash[9296]: cluster 2020-09-08T15:38:01.567542+0000 mon.a (mon.0) 930 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:38:02.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:02 smithi055 bash[9296]: audit 2020-09-08T15:38:01.578510+0000 mon.a (mon.0) 931 : audit [INF] from='client.? v1:172.21.15.104:0/137487915' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:38:02.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:02 smithi055 bash[9296]: audit 2020-09-08T15:38:01.578707+0000 mon.a (mon.0) 932 : audit [INF] from='client.34754 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 0.0}]': finished 2020-09-08T15:38:02.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:02 smithi055 bash[9296]: cluster 2020-09-08T15:38:01.578745+0000 mon.a (mon.0) 933 : cluster [DBG] osdmap e137: 12 total, 12 up, 12 in 2020-09-08T15:38:02.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:02 smithi055 bash[9296]: cluster 2020-09-08T15:38:01.597142+0000 mgr.x (mgr.24118) 315 : cluster [DBG] pgmap v374: 17 pgs: 17 active+clean; 2.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 120 MiB/s wr, 1.92k op/s 2020-09-08T15:38:02.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:02 smithi104 bash[9203]: cluster 2020-09-08T15:38:01.567542+0000 mon.a (mon.0) 930 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:38:02.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:02 smithi104 bash[9203]: audit 2020-09-08T15:38:01.578510+0000 mon.a (mon.0) 931 : audit [INF] from='client.? v1:172.21.15.104:0/137487915' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:38:02.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:02 smithi104 bash[9203]: audit 2020-09-08T15:38:01.578707+0000 mon.a (mon.0) 932 : audit [INF] from='client.34754 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 0.0}]': finished 2020-09-08T15:38:02.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:02 smithi104 bash[9203]: cluster 2020-09-08T15:38:01.578745+0000 mon.a (mon.0) 933 : cluster [DBG] osdmap e137: 12 total, 12 up, 12 in 2020-09-08T15:38:02.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:02 smithi104 bash[9203]: cluster 2020-09-08T15:38:01.597142+0000 mgr.x (mgr.24118) 315 : cluster [DBG] pgmap v374: 17 pgs: 17 active+clean; 2.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 120 MiB/s wr, 1.92k op/s 2020-09-08T15:38:03.769 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 21 15 56034 56019 166.708 131.75 0.00950808 0.00599684 2020-09-08T15:38:03.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:03 smithi104 bash[9203]: cluster 2020-09-08T15:38:03.597719+0000 mgr.x (mgr.24118) 316 : cluster [DBG] pgmap v375: 17 pgs: 17 active+clean; 3.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 170 MiB/s wr, 2.72k op/s 2020-09-08T15:38:03.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:03 smithi104 bash[9203]: cluster 2020-09-08T15:38:03.651436+0000 mon.a (mon.0) 934 : cluster [DBG] osdmap e138: 12 total, 12 up, 12 in 2020-09-08T15:38:04.100 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:03 smithi055 bash[9296]: cluster 2020-09-08T15:38:03.597719+0000 mgr.x (mgr.24118) 316 : cluster [DBG] pgmap v375: 17 pgs: 17 active+clean; 3.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 170 MiB/s wr, 2.72k op/s 2020-09-08T15:38:04.101 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:03 smithi055 bash[9296]: cluster 2020-09-08T15:38:03.651436+0000 mon.a (mon.0) 934 : cluster [DBG] osdmap e138: 12 total, 12 up, 12 in 2020-09-08T15:38:04.101 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:04.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:03 smithi166 bash[9446]: cluster 2020-09-08T15:38:03.597719+0000 mgr.x (mgr.24118) 316 : cluster [DBG] pgmap v375: 17 pgs: 17 active+clean; 3.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 170 MiB/s wr, 2.72k op/s 2020-09-08T15:38:04.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:03 smithi166 bash[9446]: cluster 2020-09-08T15:38:03.651436+0000 mon.a (mon.0) 934 : cluster [DBG] osdmap e138: 12 total, 12 up, 12 in 2020-09-08T15:38:04.205 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:38:04.351 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:38:04 smithi055 bash[19003]: debug 2020-09-08T15:38:04.102+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:04.352 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:38:04.456 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:38:04.657 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:38:04 smithi055 bash[11956]: debug 2020-09-08T15:38:04.350+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:04.768 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:04 smithi104 bash[9203]: cluster 2020-09-08T15:38:03.594589+0000 osd.7 (osd.7) 19 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:04.768 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:04 smithi104 bash[9203]: cluster 2020-09-08T15:38:03.594670+0000 osd.7 (osd.7) 20 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:04.769 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2020-09-08T15:38:04.770 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 22 16 57743 57727 163.982 106.75 0.0047963 0.00608173 2020-09-08T15:38:04.784 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:38:04.886 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:04.896 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:38:05.042 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:38:04 smithi104 bash[17588]: debug 2020-09-08T15:38:04.774+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:05.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:04 smithi166 bash[9446]: cluster 2020-09-08T15:38:03.594589+0000 osd.7 (osd.7) 19 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:05.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:04 smithi166 bash[9446]: cluster 2020-09-08T15:38:03.594670+0000 osd.7 (osd.7) 20 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:05.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:04 smithi055 bash[9296]: cluster 2020-09-08T15:38:03.594589+0000 osd.7 (osd.7) 19 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:05.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:04 smithi055 bash[9296]: cluster 2020-09-08T15:38:03.594670+0000 osd.7 (osd.7) 20 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:05.523 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:05.627 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:38:05.750 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:38:05.769 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 23 16 59457 59441 161.51 107.125 0.00799546 0.00618983 2020-09-08T15:38:05.854 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:05.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:38:05 smithi055 bash[11956]: debug 2020-09-08T15:38:05.750+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:05.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:05 smithi055 bash[9296]: cluster 2020-09-08T15:38:05.598261+0000 mgr.x (mgr.24118) 317 : cluster [DBG] pgmap v377: 17 pgs: 17 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s wr, 1.66k op/s 2020-09-08T15:38:05.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:05 smithi104 bash[9203]: cluster 2020-09-08T15:38:05.598261+0000 mgr.x (mgr.24118) 317 : cluster [DBG] pgmap v377: 17 pgs: 17 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s wr, 1.66k op/s 2020-09-08T15:38:05.927 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:38:05 smithi104 bash[22305]: debug 2020-09-08T15:38:05.522+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:05.974 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:06.078 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:06.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:05 smithi166 bash[9446]: cluster 2020-09-08T15:38:05.598261+0000 mgr.x (mgr.24118) 317 : cluster [DBG] pgmap v377: 17 pgs: 17 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s wr, 1.66k op/s 2020-09-08T15:38:06.202 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:06.304 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:38:06.414 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:38:05 smithi055 bash[14301]: debug 2020-09-08T15:38:05.970+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:06.415 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:38:06 smithi055 bash[19003]: debug 2020-09-08T15:38:06.198+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:06.769 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 24 16 61072 61056 158.985 100.938 0.00833405 0.00628786 2020-09-08T15:38:06.993 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:38:07.010 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:06 smithi104 bash[9203]: cluster 2020-09-08T15:38:05.169284+0000 osd.0 (osd.0) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:07.011 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:06 smithi104 bash[9203]: cluster 2020-09-08T15:38:05.169354+0000 osd.0 (osd.0) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:07.012 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:06 smithi104 bash[9203]: cluster 2020-09-08T15:38:05.687789+0000 mon.a (mon.0) 935 : cluster [DBG] osdmap e139: 12 total, 12 up, 12 in 2020-09-08T15:38:07.045 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:38:07.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:06 smithi166 bash[9446]: cluster 2020-09-08T15:38:05.169284+0000 osd.0 (osd.0) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:07.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:06 smithi166 bash[9446]: cluster 2020-09-08T15:38:05.169354+0000 osd.0 (osd.0) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:07.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:06 smithi166 bash[9446]: cluster 2020-09-08T15:38:05.687789+0000 mon.a (mon.0) 935 : cluster [DBG] osdmap e139: 12 total, 12 up, 12 in 2020-09-08T15:38:07.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:06 smithi055 bash[9296]: cluster 2020-09-08T15:38:05.169284+0000 osd.0 (osd.0) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:07.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:06 smithi055 bash[9296]: cluster 2020-09-08T15:38:05.169354+0000 osd.0 (osd.0) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:07.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:06 smithi055 bash[9296]: cluster 2020-09-08T15:38:05.687789+0000 mon.a (mon.0) 935 : cluster [DBG] osdmap e139: 12 total, 12 up, 12 in 2020-09-08T15:38:07.171 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:38:07.220 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:38:07.769 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 25 16 63049 63033 157.568 123.562 0.00433706 0.00634459 2020-09-08T15:38:08.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:07 smithi166 bash[9446]: cluster 2020-09-08T15:38:07.598864+0000 mgr.x (mgr.24118) 318 : cluster [DBG] pgmap v379: 17 pgs: 17 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 167 MiB/s wr, 2.68k op/s 2020-09-08T15:38:08.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:07 smithi055 bash[9296]: cluster 2020-09-08T15:38:07.598864+0000 mgr.x (mgr.24118) 318 : cluster [DBG] pgmap v379: 17 pgs: 17 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 167 MiB/s wr, 2.68k op/s 2020-09-08T15:38:08.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:07 smithi104 bash[9203]: cluster 2020-09-08T15:38:07.598864+0000 mgr.x (mgr.24118) 318 : cluster [DBG] pgmap v379: 17 pgs: 17 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 167 MiB/s wr, 2.68k op/s 2020-09-08T15:38:08.770 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 26 16 64773 64757 155.652 107.75 0.00342655 0.00642278 2020-09-08T15:38:09.485 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:38:09.589 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:38:09.754 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:38:09.769 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 27 16 66265 66249 153.34 93.25 0.0113694 0.00651905 2020-09-08T15:38:09.858 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:38:09.864 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:38:09 smithi166 bash[11928]: debug 2020-09-08T15:38:09.480+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:09.927 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:38:09 smithi104 bash[15273]: debug 2020-09-08T15:38:09.754+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:09.975 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:38:09.983 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2020-09-08T15:38:09.984 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd reweight 0 1 2020-09-08T15:38:10.080 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:38:10.127 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:38:10.186 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:38:09 smithi166 bash[14254]: debug 2020-09-08T15:38:09.976+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:10.187 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:38:10 smithi166 bash[18960]: debug 2020-09-08T15:38:10.188+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:10.188 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:38:10.292 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:10.426 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:10.531 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:10.558 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:10 smithi055 bash[9296]: cluster 2020-09-08T15:38:09.111230+0000 osd.7 (osd.7) 21 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:38:10.559 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:10 smithi055 bash[9296]: cluster 2020-09-08T15:38:09.114661+0000 osd.7 (osd.7) 22 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:38:10.559 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:10 smithi055 bash[9296]: cluster 2020-09-08T15:38:09.560181+0000 mon.a (mon.0) 936 : cluster [DBG] osdmap e140: 12 total, 12 up, 12 in 2020-09-08T15:38:10.559 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:38:10 smithi055 bash[16646]: debug 2020-09-08T15:38:10.430+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:10.631 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:10.735 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:38:10.769 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 28 16 67888 67872 151.486 101.438 0.00634343 0.00659913 2020-09-08T15:38:10.878 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:10 smithi166 bash[9446]: cluster 2020-09-08T15:38:09.111230+0000 osd.7 (osd.7) 21 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:38:10.878 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:10 smithi166 bash[9446]: cluster 2020-09-08T15:38:09.114661+0000 osd.7 (osd.7) 22 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:38:10.878 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:10 smithi166 bash[9446]: cluster 2020-09-08T15:38:09.560181+0000 mon.a (mon.0) 936 : cluster [DBG] osdmap e140: 12 total, 12 up, 12 in 2020-09-08T15:38:10.879 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:10 smithi166 bash[9446]: cluster 2020-09-08T15:38:09.599326+0000 mgr.x (mgr.24118) 319 : cluster [DBG] pgmap v381: 17 pgs: 17 active+clean; 3.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 MiB/s wr, 1.55k op/s 2020-09-08T15:38:10.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:10 smithi055 bash[9296]: cluster 2020-09-08T15:38:09.599326+0000 mgr.x (mgr.24118) 319 : cluster [DBG] pgmap v381: 17 pgs: 17 active+clean; 3.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 MiB/s wr, 1.55k op/s 2020-09-08T15:38:10.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:38:10 smithi055 bash[16646]: debug 2020-09-08T15:38:10.634+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:10.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:10 smithi104 bash[9203]: cluster 2020-09-08T15:38:09.111230+0000 osd.7 (osd.7) 21 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:38:10.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:10 smithi104 bash[9203]: cluster 2020-09-08T15:38:09.114661+0000 osd.7 (osd.7) 22 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:38:10.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:10 smithi104 bash[9203]: cluster 2020-09-08T15:38:09.560181+0000 mon.a (mon.0) 936 : cluster [DBG] osdmap e140: 12 total, 12 up, 12 in 2020-09-08T15:38:10.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:10 smithi104 bash[9203]: cluster 2020-09-08T15:38:09.599326+0000 mgr.x (mgr.24118) 319 : cluster [DBG] pgmap v381: 17 pgs: 17 active+clean; 3.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 MiB/s wr, 1.55k op/s 2020-09-08T15:38:10.982 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:38:11.085 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:38:11.427 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:38:10 smithi104 bash[19922]: debug 2020-09-08T15:38:10.982+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:11.769 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 29 16 69755 69739 150.286 116.688 0.0117064 0.00665179 2020-09-08T15:38:11.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:11 smithi104 bash[9203]: cluster 2020-09-08T15:38:11.599749+0000 mgr.x (mgr.24118) 320 : cluster [DBG] pgmap v382: 17 pgs: 17 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 94 MiB/s wr, 1.51k op/s 2020-09-08T15:38:12.109 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:38:12.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:11 smithi166 bash[9446]: cluster 2020-09-08T15:38:11.599749+0000 mgr.x (mgr.24118) 320 : cluster [DBG] pgmap v382: 17 pgs: 17 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 94 MiB/s wr, 1.51k op/s 2020-09-08T15:38:12.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:11 smithi055 bash[9296]: cluster 2020-09-08T15:38:11.599749+0000 mgr.x (mgr.24118) 320 : cluster [DBG] pgmap v382: 17 pgs: 17 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 94 MiB/s wr, 1.51k op/s 2020-09-08T15:38:12.211 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:38:12.345 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:38:12.367 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:38:12 smithi104 bash[17588]: debug 2020-09-08T15:38:12.110+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:12.453 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:12.548 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:38:12.549 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:38:12.657 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:38:12 smithi055 bash[11956]: debug 2020-09-08T15:38:12.350+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:12.657 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:12.763 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:12.770 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 30 16 71236 71220 148.362 92.5625 0.00710775 0.00673818 2020-09-08T15:38:12.882 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:12.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:12 smithi104 bash[9203]: cluster 2020-09-08T15:38:11.972143+0000 osd.5 (osd.5) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:12.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:12 smithi104 bash[9203]: cluster 2020-09-08T15:38:11.972244+0000 osd.5 (osd.5) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:12.928 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:38:12 smithi104 bash[22305]: debug 2020-09-08T15:38:12.658+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:13.001 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:13.009 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:12 smithi166 bash[9446]: cluster 2020-09-08T15:38:11.972143+0000 osd.5 (osd.5) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:13.009 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:12 smithi166 bash[9446]: cluster 2020-09-08T15:38:11.972244+0000 osd.5 (osd.5) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:13.111 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:13.158 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:38:12 smithi055 bash[16646]: debug 2020-09-08T15:38:12.886+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:13.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:12 smithi055 bash[9296]: cluster 2020-09-08T15:38:11.972143+0000 osd.5 (osd.5) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:13.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:12 smithi055 bash[9296]: cluster 2020-09-08T15:38:11.972244+0000 osd.5 (osd.5) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:13.218 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:13.320 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:38:13 smithi166 bash[16593]: debug 2020-09-08T15:38:13.112+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:13.321 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:13.424 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:13.538 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:13.614 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:38:13 smithi166 bash[16593]: debug 2020-09-08T15:38:13.320+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:13.615 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:38:13 smithi166 bash[16593]: debug 2020-09-08T15:38:13.536+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:13.640 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:38:13.772 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 31 15 73051 73036 147.237 113.5 0.00926904 0.00678975 2020-09-08T15:38:13.773 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:38:13.876 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:14.029 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:38:14 smithi055 bash[16646]: debug 2020-09-08T15:38:14.030+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:14.030 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:13 smithi055 bash[9296]: cluster 2020-09-08T15:38:12.682045+0000 mon.a (mon.0) 937 : cluster [DBG] osdmap e141: 12 total, 12 up, 12 in 2020-09-08T15:38:14.030 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:13 smithi055 bash[9296]: cluster 2020-09-08T15:38:13.600298+0000 mgr.x (mgr.24118) 321 : cluster [DBG] pgmap v384: 17 pgs: 17 active+clean; 4.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 141 MiB/s wr, 2.26k op/s 2020-09-08T15:38:14.030 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:14.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:13 smithi166 bash[9446]: cluster 2020-09-08T15:38:12.682045+0000 mon.a (mon.0) 937 : cluster [DBG] osdmap e141: 12 total, 12 up, 12 in 2020-09-08T15:38:14.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:13 smithi166 bash[9446]: cluster 2020-09-08T15:38:13.600298+0000 mgr.x (mgr.24118) 321 : cluster [DBG] pgmap v384: 17 pgs: 17 active+clean; 4.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 141 MiB/s wr, 2.26k op/s 2020-09-08T15:38:14.115 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:38:13 smithi166 bash[18960]: debug 2020-09-08T15:38:13.768+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:14.136 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:14.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:13 smithi104 bash[9203]: cluster 2020-09-08T15:38:12.682045+0000 mon.a (mon.0) 937 : cluster [DBG] osdmap e141: 12 total, 12 up, 12 in 2020-09-08T15:38:14.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:13 smithi104 bash[9203]: cluster 2020-09-08T15:38:13.600298+0000 mgr.x (mgr.24118) 321 : cluster [DBG] pgmap v384: 17 pgs: 17 active+clean; 4.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 141 MiB/s wr, 2.26k op/s 2020-09-08T15:38:14.252 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:14.357 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:14.367 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:38:14 smithi055 bash[14301]: debug 2020-09-08T15:38:14.254+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:14.471 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:14.583 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:38:14.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:38:14 smithi055 bash[14301]: debug 2020-09-08T15:38:14.470+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:14.698 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:38:14.786 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 32 16 74643 74627 145.743 99.4375 0.00714298 0.00685944 2020-09-08T15:38:14.804 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:14.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:38:14 smithi166 bash[18960]: debug 2020-09-08T15:38:14.696+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:14.910 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:15.014 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:38:15.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:38:14 smithi055 bash[19003]: debug 2020-09-08T15:38:14.914+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:15.680 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:38:15.740 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2020-09-08T15:38:15.769 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 33 16 76358 76342 144.574 107.188 0.0112664 0.00691448 2020-09-08T15:38:15.789 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:38:15.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:15 smithi166 bash[9446]: cluster 2020-09-08T15:38:14.561390+0000 mon.a (mon.0) 938 : cluster [DBG] osdmap e142: 12 total, 12 up, 12 in 2020-09-08T15:38:15.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:15 smithi055 bash[9296]: cluster 2020-09-08T15:38:14.561390+0000 mon.a (mon.0) 938 : cluster [DBG] osdmap e142: 12 total, 12 up, 12 in 2020-09-08T15:38:15.912 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:38:15.923 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:38:15.927 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:38:15 smithi104 bash[17588]: debug 2020-09-08T15:38:15.678+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:15.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:15 smithi104 bash[9203]: cluster 2020-09-08T15:38:14.561390+0000 mon.a (mon.0) 938 : cluster [DBG] osdmap e142: 12 total, 12 up, 12 in 2020-09-08T15:38:16.023 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:16.152 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:16.256 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:16.354 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:38:16 smithi055 bash[19003]: debug 2020-09-08T15:38:16.154+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:16.355 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:16.364 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:38:15 smithi166 bash[11928]: debug 2020-09-08T15:38:15.916+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:16.458 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:38:16.577 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:38:16.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:38:16 smithi055 bash[16646]: debug 2020-09-08T15:38:16.358+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:16.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:16 smithi055 bash[9296]: cluster 2020-09-08T15:38:15.600772+0000 mgr.x (mgr.24118) 322 : cluster [DBG] pgmap v386: 17 pgs: 17 active+clean; 4.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 89 MiB/s wr, 1.42k op/s 2020-09-08T15:38:16.682 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:38:16.770 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 34 16 78117 78101 143.555 109.938 0.0039636 0.00696399 2020-09-08T15:38:16.864 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:38:16 smithi166 bash[18960]: debug 2020-09-08T15:38:16.576+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:16.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:16 smithi166 bash[9446]: cluster 2020-09-08T15:38:15.600772+0000 mgr.x (mgr.24118) 322 : cluster [DBG] pgmap v386: 17 pgs: 17 active+clean; 4.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 89 MiB/s wr, 1.42k op/s 2020-09-08T15:38:16.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:16 smithi104 bash[9203]: cluster 2020-09-08T15:38:15.600772+0000 mgr.x (mgr.24118) 322 : cluster [DBG] pgmap v386: 17 pgs: 17 active+clean; 4.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 89 MiB/s wr, 1.42k op/s 2020-09-08T15:38:17.066 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:38:17.184 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:38:17.433 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:38:17 smithi104 bash[17588]: debug 2020-09-08T15:38:17.066+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:17.770 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 35 16 79873 79857 142.589 109.75 0.0149469 0.00701136 2020-09-08T15:38:17.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:17 smithi166 bash[9446]: cluster 2020-09-08T15:38:16.698250+0000 osd.10 (osd.10) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:17.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:17 smithi166 bash[9446]: cluster 2020-09-08T15:38:16.698335+0000 osd.10 (osd.10) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:17.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:17 smithi055 bash[9296]: cluster 2020-09-08T15:38:16.698250+0000 osd.10 (osd.10) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:17.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:17 smithi055 bash[9296]: cluster 2020-09-08T15:38:16.698335+0000 osd.10 (osd.10) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:17.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:17 smithi104 bash[9203]: cluster 2020-09-08T15:38:16.698250+0000 osd.10 (osd.10) 9 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:17.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:17 smithi104 bash[9203]: cluster 2020-09-08T15:38:16.698335+0000 osd.10 (osd.10) 10 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:18.249 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:38:18.353 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:38:18.483 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:38:18.580 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:38:18 smithi104 bash[15273]: debug 2020-09-08T15:38:18.250+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:18.581 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:18 smithi104 bash[9203]: cluster 2020-09-08T15:38:17.581427+0000 mon.a (mon.0) 939 : cluster [DBG] osdmap e143: 12 total, 12 up, 12 in 2020-09-08T15:38:18.581 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:18 smithi104 bash[9203]: cluster 2020-09-08T15:38:17.601253+0000 mgr.x (mgr.24118) 323 : cluster [DBG] pgmap v388: 17 pgs: 17 active+clean; 4.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 MiB/s wr, 1.93k op/s 2020-09-08T15:38:18.586 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:18.615 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:18 smithi166 bash[9446]: cluster 2020-09-08T15:38:17.581427+0000 mon.a (mon.0) 939 : cluster [DBG] osdmap e143: 12 total, 12 up, 12 in 2020-09-08T15:38:18.615 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:18 smithi166 bash[9446]: cluster 2020-09-08T15:38:17.601253+0000 mgr.x (mgr.24118) 323 : cluster [DBG] pgmap v388: 17 pgs: 17 active+clean; 4.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 MiB/s wr, 1.93k op/s 2020-09-08T15:38:18.615 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:38:18 smithi166 bash[11928]: debug 2020-09-08T15:38:18.484+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:18.702 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:18.771 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 36 15 81256 81241 141.031 86.5 0.00733175 0.00708841 2020-09-08T15:38:18.806 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:38:18.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:18 smithi055 bash[9296]: cluster 2020-09-08T15:38:17.581427+0000 mon.a (mon.0) 939 : cluster [DBG] osdmap e143: 12 total, 12 up, 12 in 2020-09-08T15:38:18.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:18 smithi055 bash[9296]: cluster 2020-09-08T15:38:17.601253+0000 mgr.x (mgr.24118) 323 : cluster [DBG] pgmap v388: 17 pgs: 17 active+clean; 4.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 MiB/s wr, 1.93k op/s 2020-09-08T15:38:18.935 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:38:18.973 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:38:19.023 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:38:19.041 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:38:19.057 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:38:19.081 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:38:19.114 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:38:18 smithi166 bash[16593]: debug 2020-09-08T15:38:18.696+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:19.121 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:38:19.145 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:19.162 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:38:19.201 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:38:19.240 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:38:19.277 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:38:19.314 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:38:19.339 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:38:19.341 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:38:19.341 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:38:19.341 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:38:19.342 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:38:19.342 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:38:19.342 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:38:19.342 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:38:19.343 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:38:19.343 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:38:19.343 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:38:19.343 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:38:19.343 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:38:19.345 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:38:19 smithi104 bash[19922]: debug 2020-09-08T15:38:19.038+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:19.404 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:19.507 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:19.616 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:19.629 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:19 smithi104 bash[9203]: audit 2020-09-08T15:38:18.898636+0000 mon.b (mon.1) 75 : audit [DBG] from='client.? v1:172.21.15.104:0/3182347875' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:38:19.629 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:19 smithi104 bash[9203]: cluster 2020-09-08T15:38:19.565556+0000 mon.a (mon.0) 940 : cluster [DBG] osdmap e144: 12 total, 12 up, 12 in 2020-09-08T15:38:19.630 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:38:19 smithi104 bash[22305]: debug 2020-09-08T15:38:19.402+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:19.740 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:38:19.770 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 37 16 82755 82739 139.749 93.625 0.0105209 0.00715331 2020-09-08T15:38:19.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:19 smithi166 bash[9446]: audit 2020-09-08T15:38:18.898636+0000 mon.b (mon.1) 75 : audit [DBG] from='client.? v1:172.21.15.104:0/3182347875' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:38:19.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:19 smithi166 bash[9446]: cluster 2020-09-08T15:38:19.565556+0000 mon.a (mon.0) 940 : cluster [DBG] osdmap e144: 12 total, 12 up, 12 in 2020-09-08T15:38:19.878 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:38:19.911 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:38:19 smithi055 bash[14301]: debug 2020-09-08T15:38:19.614+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:19.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:19 smithi055 bash[9296]: audit 2020-09-08T15:38:18.898636+0000 mon.b (mon.1) 75 : audit [DBG] from='client.? v1:172.21.15.104:0/3182347875' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:38:19.912 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:19 smithi055 bash[9296]: cluster 2020-09-08T15:38:19.565556+0000 mon.a (mon.0) 940 : cluster [DBG] osdmap e144: 12 total, 12 up, 12 in 2020-09-08T15:38:19.981 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:38:20.094 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:38:20.195 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:38:20.365 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:38:19 smithi166 bash[11928]: debug 2020-09-08T15:38:19.876+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:20.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:38:20 smithi055 bash[11956]: debug 2020-09-08T15:38:20.094+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:20.600 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:38:20.607 INFO:teuthology.orchestra.run.smithi104.stderr:reweighted osd.0 to 1 (10000) 2020-09-08T15:38:20.770 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 38 16 84336 84320 138.672 98.8125 0.00716234 0.00720949 2020-09-08T15:38:20.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:20 smithi166 bash[9446]: cluster 2020-09-08T15:38:19.601678+0000 mgr.x (mgr.24118) 324 : cluster [DBG] pgmap v390: 17 pgs: 17 active+clean; 4.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 88 MiB/s wr, 1.40k op/s 2020-09-08T15:38:20.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:20 smithi166 bash[9446]: audit 2020-09-08T15:38:19.947716+0000 mon.a (mon.0) 941 : audit [INF] from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:38:20.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:20 smithi166 bash[9446]: audit 2020-09-08T15:38:19.947933+0000 mon.b (mon.1) 76 : audit [INF] from='client.? v1:172.21.15.104:0/1508198084' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:38:20.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:20 smithi166 bash[9446]: audit 2020-09-08T15:38:20.344589+0000 mon.a (mon.0) 942 : audit [INF] from='client.? v1:172.21.15.104:0/1880432442' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2020-09-08T15:38:20.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:20 smithi055 bash[9296]: cluster 2020-09-08T15:38:19.601678+0000 mgr.x (mgr.24118) 324 : cluster [DBG] pgmap v390: 17 pgs: 17 active+clean; 4.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 88 MiB/s wr, 1.40k op/s 2020-09-08T15:38:20.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:20 smithi055 bash[9296]: audit 2020-09-08T15:38:19.947716+0000 mon.a (mon.0) 941 : audit [INF] from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:38:20.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:20 smithi055 bash[9296]: audit 2020-09-08T15:38:19.947933+0000 mon.b (mon.1) 76 : audit [INF] from='client.? v1:172.21.15.104:0/1508198084' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:38:20.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:20 smithi055 bash[9296]: audit 2020-09-08T15:38:20.344589+0000 mon.a (mon.0) 942 : audit [INF] from='client.? v1:172.21.15.104:0/1880432442' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2020-09-08T15:38:20.998 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:20 smithi104 bash[9203]: cluster 2020-09-08T15:38:19.601678+0000 mgr.x (mgr.24118) 324 : cluster [DBG] pgmap v390: 17 pgs: 17 active+clean; 4.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 88 MiB/s wr, 1.40k op/s 2020-09-08T15:38:20.999 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:20 smithi104 bash[9203]: audit 2020-09-08T15:38:19.947716+0000 mon.a (mon.0) 941 : audit [INF] from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:38:20.999 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:20 smithi104 bash[9203]: audit 2020-09-08T15:38:19.947933+0000 mon.b (mon.1) 76 : audit [INF] from='client.? v1:172.21.15.104:0/1508198084' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:38:20.999 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:20 smithi104 bash[9203]: audit 2020-09-08T15:38:20.344589+0000 mon.a (mon.0) 942 : audit [INF] from='client.? v1:172.21.15.104:0/1880432442' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2020-09-08T15:38:21.591 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:38:21.693 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:21.770 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 39 16 85788 85772 137.443 90.75 0.00797377 0.00727386 2020-09-08T15:38:21.830 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:21.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:21 smithi166 bash[9446]: cluster 2020-09-08T15:38:20.091224+0000 osd.3 (osd.3) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:21.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:21 smithi166 bash[9446]: cluster 2020-09-08T15:38:20.091314+0000 osd.3 (osd.3) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:21.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:21 smithi166 bash[9446]: cluster 2020-09-08T15:38:20.588135+0000 mon.a (mon.0) 943 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:38:21.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:21 smithi166 bash[9446]: cluster 2020-09-08T15:38:20.588164+0000 mon.a (mon.0) 944 : cluster [INF] Cluster is now healthy 2020-09-08T15:38:21.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:21 smithi166 bash[9446]: audit 2020-09-08T15:38:20.599875+0000 mon.a (mon.0) 945 : audit [INF] from='client.24775 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:38:21.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:21 smithi166 bash[9446]: audit 2020-09-08T15:38:20.599927+0000 mon.a (mon.0) 946 : audit [INF] from='client.? v1:172.21.15.104:0/1880432442' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2020-09-08T15:38:21.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:21 smithi166 bash[9446]: cluster 2020-09-08T15:38:20.599962+0000 mon.a (mon.0) 947 : cluster [DBG] osdmap e145: 12 total, 12 up, 12 in 2020-09-08T15:38:21.910 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:38:21 smithi055 bash[16646]: debug 2020-09-08T15:38:21.834+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:21.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:21 smithi055 bash[9296]: cluster 2020-09-08T15:38:20.091224+0000 osd.3 (osd.3) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:21.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:21 smithi055 bash[9296]: cluster 2020-09-08T15:38:20.091314+0000 osd.3 (osd.3) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:21.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:21 smithi055 bash[9296]: cluster 2020-09-08T15:38:20.588135+0000 mon.a (mon.0) 943 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:38:21.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:21 smithi055 bash[9296]: cluster 2020-09-08T15:38:20.588164+0000 mon.a (mon.0) 944 : cluster [INF] Cluster is now healthy 2020-09-08T15:38:21.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:21 smithi055 bash[9296]: audit 2020-09-08T15:38:20.599875+0000 mon.a (mon.0) 945 : audit [INF] from='client.24775 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:38:21.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:21 smithi055 bash[9296]: audit 2020-09-08T15:38:20.599927+0000 mon.a (mon.0) 946 : audit [INF] from='client.? v1:172.21.15.104:0/1880432442' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2020-09-08T15:38:21.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:21 smithi055 bash[9296]: cluster 2020-09-08T15:38:20.599962+0000 mon.a (mon.0) 947 : cluster [DBG] osdmap e145: 12 total, 12 up, 12 in 2020-09-08T15:38:21.934 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:38:21 smithi104 bash[19922]: debug 2020-09-08T15:38:21.582+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:21.935 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:21 smithi104 bash[9203]: cluster 2020-09-08T15:38:20.091224+0000 osd.3 (osd.3) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:21.935 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:21 smithi104 bash[9203]: cluster 2020-09-08T15:38:20.091314+0000 osd.3 (osd.3) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:21.935 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:21 smithi104 bash[9203]: cluster 2020-09-08T15:38:20.588135+0000 mon.a (mon.0) 943 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:38:21.935 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:21 smithi104 bash[9203]: cluster 2020-09-08T15:38:20.588164+0000 mon.a (mon.0) 944 : cluster [INF] Cluster is now healthy 2020-09-08T15:38:21.936 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:21 smithi104 bash[9203]: audit 2020-09-08T15:38:20.599875+0000 mon.a (mon.0) 945 : audit [INF] from='client.24775 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:38:21.936 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:21 smithi104 bash[9203]: audit 2020-09-08T15:38:20.599927+0000 mon.a (mon.0) 946 : audit [INF] from='client.? v1:172.21.15.104:0/1880432442' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2020-09-08T15:38:21.936 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:21 smithi104 bash[9203]: cluster 2020-09-08T15:38:20.599962+0000 mon.a (mon.0) 947 : cluster [DBG] osdmap e145: 12 total, 12 up, 12 in 2020-09-08T15:38:21.940 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:38:22.080 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:38:22.183 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:38:22.301 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:38:22.364 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:38:22 smithi166 bash[14254]: debug 2020-09-08T15:38:22.080+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:22.410 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:38:22.600 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:38:22 smithi055 bash[11956]: debug 2020-09-08T15:38:22.302+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:22.770 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 15:38:22.777014 min lat: 0.00138961 max lat: 0.356103 avg lat: 0.00731635 2020-09-08T15:38:22.771 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:38:22.771 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 40 16 87316 87300 136.394 95.5 0.00361074 0.00731635 2020-09-08T15:38:22.778 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:22 smithi104 bash[9203]: cluster 2020-09-08T15:38:21.602213+0000 mgr.x (mgr.24118) 325 : cluster [DBG] pgmap v392: 17 pgs: 17 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 113 MiB/s wr, 1.80k op/s 2020-09-08T15:38:22.840 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:38:22.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:22 smithi166 bash[9446]: cluster 2020-09-08T15:38:21.602213+0000 mgr.x (mgr.24118) 325 : cluster [DBG] pgmap v392: 17 pgs: 17 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 113 MiB/s wr, 1.80k op/s 2020-09-08T15:38:22.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:22 smithi055 bash[9296]: cluster 2020-09-08T15:38:21.602213+0000 mgr.x (mgr.24118) 325 : cluster [DBG] pgmap v392: 17 pgs: 17 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 113 MiB/s wr, 1.80k op/s 2020-09-08T15:38:22.943 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:38:23.090 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:38:23.177 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:38:22 smithi104 bash[15273]: debug 2020-09-08T15:38:22.842+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:23.193 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:23.202 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:38:23 smithi166 bash[18960]: debug 2020-09-08T15:38:23.088+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:23.296 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:23.408 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:38:23.530 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:38:23 smithi166 bash[16593]: debug 2020-09-08T15:38:23.292+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:23.532 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd reweight 1 1 2020-09-08T15:38:23.534 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:38:23.641 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:23.733 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:38:23.759 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:23.770 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 41 16 88810 88794 135.344 93.375 0.0289061 0.00738502 2020-09-08T15:38:23.865 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:38:23 smithi166 bash[14254]: debug 2020-09-08T15:38:23.528+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:23.865 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:38:23 smithi166 bash[16593]: debug 2020-09-08T15:38:23.760+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:23.896 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:38:24.017 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:38:24.122 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:24.365 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:38:24 smithi166 bash[14254]: debug 2020-09-08T15:38:24.012+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:24.434 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:24.538 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:24.675 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:24.771 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 42 16 90152 90136 134.119 83.875 0.00456142 0.00745348 2020-09-08T15:38:24.778 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:38:24.811 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:38:24.819 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:24 smithi104 bash[9203]: cluster 2020-09-08T15:38:23.602722+0000 mgr.x (mgr.24118) 326 : cluster [DBG] pgmap v393: 17 pgs: 17 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 108 MiB/s wr, 1.72k op/s 2020-09-08T15:38:24.820 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:38:24 smithi104 bash[22305]: debug 2020-09-08T15:38:24.434+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:24.910 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:24 smithi166 bash[9446]: cluster 2020-09-08T15:38:23.602722+0000 mgr.x (mgr.24118) 326 : cluster [DBG] pgmap v393: 17 pgs: 17 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 108 MiB/s wr, 1.72k op/s 2020-09-08T15:38:24.911 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:38:24.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:24 smithi055 bash[9296]: cluster 2020-09-08T15:38:23.602722+0000 mgr.x (mgr.24118) 326 : cluster [DBG] pgmap v393: 17 pgs: 17 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 108 MiB/s wr, 1.72k op/s 2020-09-08T15:38:24.911 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:38:24 smithi055 bash[19003]: debug 2020-09-08T15:38:24.658+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:24.977 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:38:25.013 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:25.129 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:25.238 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:25.249 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:38:24 smithi166 bash[11928]: debug 2020-09-08T15:38:24.908+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:25.392 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:25.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:38:25 smithi055 bash[19003]: debug 2020-09-08T15:38:25.130+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:25.498 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:25.527 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:38:25.615 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:38:25 smithi166 bash[16593]: debug 2020-09-08T15:38:25.392+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:25.716 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:38:25.730 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:25.774 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 43 16 91655 91639 133.184 93.9375 0.00527923 0.00750637 2020-09-08T15:38:25.835 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:25.853 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:25 smithi104 bash[9203]: cluster 2020-09-08T15:38:25.603266+0000 mgr.x (mgr.24118) 327 : cluster [DBG] pgmap v394: 17 pgs: 17 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s wr, 1.29k op/s 2020-09-08T15:38:25.854 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:38:25 smithi104 bash[22305]: debug 2020-09-08T15:38:25.730+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:25.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:25 smithi055 bash[9296]: cluster 2020-09-08T15:38:25.603266+0000 mgr.x (mgr.24118) 327 : cluster [DBG] pgmap v394: 17 pgs: 17 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s wr, 1.29k op/s 2020-09-08T15:38:26.077 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:38:26.077 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2020-09-08T15:38:26.077 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2020-09-08T15:38:26.078 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T15:38:26.078 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.079 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24136.0:1 1.0 1.ae07d6f8 (undecoded) ondisk+retry+read+known_if_redirected e18)", 2020-09-08T15:38:26.079 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:30:31.594846+0000", 2020-09-08T15:38:26.079 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 474.48471078400001, 2020-09-08T15:38:26.079 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.60718419199999996, 2020-09-08T15:38:26.079 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:38:26.080 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2020-09-08T15:38:26.080 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:38:26.080 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24136", 2020-09-08T15:38:26.080 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.55:0/1149479755", 2020-09-08T15:38:26.080 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 1 2020-09-08T15:38:26.081 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.081 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:38:26.081 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.081 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:38:26.081 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:30:31.594846+0000", 2020-09-08T15:38:26.082 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:38:26.082 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.082 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.082 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:38:26.083 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:30:31.594846+0000", 2020-09-08T15:38:26.083 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999232 2020-09-08T15:38:26.083 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.083 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.083 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:38:26.084 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:30:31.594770+0000", 2020-09-08T15:38:26.084 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020688 2020-09-08T15:38:26.084 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.084 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.084 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:38:26.085 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:30:31.594977+0000", 2020-09-08T15:38:26.085 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6467e-05 2020-09-08T15:38:26.085 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.085 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.085 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:38:26.086 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:30:31.595033+0000", 2020-09-08T15:38:26.086 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.622e-05 2020-09-08T15:38:26.086 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.086 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.086 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:38:26.087 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:30:31.595059+0000", 2020-09-08T15:38:26.087 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.60682835300000004 2020-09-08T15:38:26.087 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.088 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.088 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.088 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:30:32.201888+0000", 2020-09-08T15:38:26.088 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000142829 2020-09-08T15:38:26.089 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.089 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.089 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:38:26.090 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:30:32.202031+0000", 2020-09-08T15:38:26.090 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.60718419199999996 2020-09-08T15:38:26.090 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.090 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.090 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.091 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.091 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.091 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24136.0:1 1.0 1:1f6be075:::INTEL_SSDPEDMD400G4_PHFT620400NW400BGN:head [omap-get-keys] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected e18)", 2020-09-08T15:38:26.091 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:30:32.564605+0000", 2020-09-08T15:38:26.091 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 473.51495205399999, 2020-09-08T15:38:26.092 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15059989400000001, 2020-09-08T15:38:26.092 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:38:26.092 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2020-09-08T15:38:26.092 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:38:26.092 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24136", 2020-09-08T15:38:26.093 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.55:0/1149479755", 2020-09-08T15:38:26.095 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 1 2020-09-08T15:38:26.095 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.095 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:38:26.096 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.096 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:38:26.097 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:30:32.564605+0000", 2020-09-08T15:38:26.097 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:38:26.097 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.097 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.098 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:38:26.098 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:30:32.564605+0000", 2020-09-08T15:38:26.098 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999137 2020-09-08T15:38:26.098 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.098 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.099 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:38:26.099 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:30:32.564519+0000", 2020-09-08T15:38:26.100 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00023854099999999999 2020-09-08T15:38:26.100 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.100 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.100 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:38:26.101 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:30:32.564757+0000", 2020-09-08T15:38:26.101 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8553000000000002e-05 2020-09-08T15:38:26.101 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.101 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.102 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:38:26.102 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:30:32.564796+0000", 2020-09-08T15:38:26.102 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8451999999999999e-05 2020-09-08T15:38:26.103 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.103 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.103 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:38:26.103 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:30:32.564814+0000", 2020-09-08T15:38:26.104 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014350300000000001 2020-09-08T15:38:26.105 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.105 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.105 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.105 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:30:32.564958+0000", 2020-09-08T15:38:26.107 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6866000000000001e-05 2020-09-08T15:38:26.107 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.107 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.107 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:38:26.107 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:30:32.564975+0000", 2020-09-08T15:38:26.108 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15012492599999999 2020-09-08T15:38:26.108 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.108 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.108 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.108 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:30:32.715099+0000", 2020-09-08T15:38:26.109 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000105445 2020-09-08T15:38:26.109 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.109 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.109 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:38:26.110 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:30:32.715205+0000", 2020-09-08T15:38:26.110 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15059989400000001 2020-09-08T15:38:26.110 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.111 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.111 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.111 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.111 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.112 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:23421 2.4 2:26ec0a9d:::benchmark_data_smithi166_20551_object23420:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e134)", 2020-09-08T15:38:26.112 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:49.478392+0000", 2020-09-08T15:38:26.112 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 36.601164675, 2020-09-08T15:38:26.112 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.095972216999999999, 2020-09-08T15:38:26.112 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:38:26.113 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:38:26.113 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:38:26.113 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:38:26.113 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:38:26.114 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 23421 2020-09-08T15:38:26.114 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.114 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:38:26.114 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.114 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:38:26.115 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.478392+0000", 2020-09-08T15:38:26.115 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:38:26.115 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.115 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.115 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:38:26.116 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.478392+0000", 2020-09-08T15:38:26.116 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999738 2020-09-08T15:38:26.116 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.116 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.117 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:38:26.117 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.478366+0000", 2020-09-08T15:38:26.117 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5017000000000002e-05 2020-09-08T15:38:26.117 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.117 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.118 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:38:26.118 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.478461+0000", 2020-09-08T15:38:26.118 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0559e-05 2020-09-08T15:38:26.118 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.118 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.119 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:38:26.119 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.478472+0000", 2020-09-08T15:38:26.119 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1739999999999999e-06 2020-09-08T15:38:26.119 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.120 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.120 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:38:26.120 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.478477+0000", 2020-09-08T15:38:26.121 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00091988200000000001 2020-09-08T15:38:26.121 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.121 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.121 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.122 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.479397+0000", 2020-09-08T15:38:26.122 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018040299999999999 2020-09-08T15:38:26.122 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.122 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.122 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:38:26.123 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.479577+0000", 2020-09-08T15:38:26.123 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00025561099999999999 2020-09-08T15:38:26.123 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.123 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.123 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0", 2020-09-08T15:38:26.124 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.479833+0000", 2020-09-08T15:38:26.124 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0016948690000000001 2020-09-08T15:38:26.124 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.124 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.125 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:38:26.125 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.481528+0000", 2020-09-08T15:38:26.125 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.092640950999999999 2020-09-08T15:38:26.125 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.126 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.126 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:38:26.126 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.574169+0000", 2020-09-08T15:38:26.126 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000110009 2020-09-08T15:38:26.127 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.127 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.127 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:38:26.127 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.574279+0000", 2020-09-08T15:38:26.127 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5865999999999994e-05 2020-09-08T15:38:26.128 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.128 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.128 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:38:26.128 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.574365+0000", 2020-09-08T15:38:26.128 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.095972216999999999 2020-09-08T15:38:26.129 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.129 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.129 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.129 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.129 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.130 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:23429 2.4 2:2bf08ecc:::benchmark_data_smithi166_20551_object23428:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e134)", 2020-09-08T15:38:26.130 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:49.479095+0000", 2020-09-08T15:38:26.130 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 36.600462362999998, 2020-09-08T15:38:26.130 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.095493065000000002, 2020-09-08T15:38:26.131 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:38:26.131 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:38:26.131 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:38:26.132 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:38:26.132 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:38:26.132 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 23429 2020-09-08T15:38:26.132 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.132 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:38:26.133 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.133 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:38:26.133 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.479095+0000", 2020-09-08T15:38:26.133 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:38:26.133 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.134 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.134 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:38:26.134 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.479095+0000", 2020-09-08T15:38:26.134 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999958 2020-09-08T15:38:26.134 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.135 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.135 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:38:26.135 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.479053+0000", 2020-09-08T15:38:26.135 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000134633 2020-09-08T15:38:26.135 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.136 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.136 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:38:26.136 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.479187+0000", 2020-09-08T15:38:26.136 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7584e-05 2020-09-08T15:38:26.136 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.137 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.137 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:38:26.137 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.479205+0000", 2020-09-08T15:38:26.137 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8980000000000004e-06 2020-09-08T15:38:26.137 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.138 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.138 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:38:26.138 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.479210+0000", 2020-09-08T15:38:26.138 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0013859199999999999 2020-09-08T15:38:26.138 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.139 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.139 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.139 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.480596+0000", 2020-09-08T15:38:26.139 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018630599999999999 2020-09-08T15:38:26.139 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.140 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.140 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:38:26.140 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.480782+0000", 2020-09-08T15:38:26.140 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000178245 2020-09-08T15:38:26.140 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.141 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.141 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0", 2020-09-08T15:38:26.141 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.480960+0000", 2020-09-08T15:38:26.142 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0011940779999999999 2020-09-08T15:38:26.142 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.142 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.143 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:38:26.143 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.482154+0000", 2020-09-08T15:38:26.143 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.092281074000000005 2020-09-08T15:38:26.143 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.143 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.144 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:38:26.144 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.574436+0000", 2020-09-08T15:38:26.144 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6258e-05 2020-09-08T15:38:26.144 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.145 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.146 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:38:26.146 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.574522+0000", 2020-09-08T15:38:26.146 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5974000000000002e-05 2020-09-08T15:38:26.146 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.146 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.147 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:38:26.147 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.574588+0000", 2020-09-08T15:38:26.147 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.095493065000000002 2020-09-08T15:38:26.148 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.148 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.148 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.148 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.148 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.149 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:23438 2.4 2:2b33612d:::benchmark_data_smithi166_20551_object23437:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e134)", 2020-09-08T15:38:26.150 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:49.483549+0000", 2020-09-08T15:38:26.150 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 36.596008539000003, 2020-09-08T15:38:26.150 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.094969364000000001, 2020-09-08T15:38:26.150 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:38:26.151 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:38:26.151 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:38:26.151 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:38:26.152 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:38:26.152 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 23438 2020-09-08T15:38:26.152 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.152 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:38:26.153 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.153 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:38:26.153 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.483549+0000", 2020-09-08T15:38:26.153 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:38:26.153 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.154 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.154 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:38:26.154 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.483549+0000", 2020-09-08T15:38:26.154 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999514 2020-09-08T15:38:26.154 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.155 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.155 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:38:26.155 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.483500+0000", 2020-09-08T15:38:26.155 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000156193 2020-09-08T15:38:26.156 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.156 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.156 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:38:26.156 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.483656+0000", 2020-09-08T15:38:26.156 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7986e-05 2020-09-08T15:38:26.157 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.157 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.157 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:38:26.157 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.483674+0000", 2020-09-08T15:38:26.157 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5839999999999997e-06 2020-09-08T15:38:26.158 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.158 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.158 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:38:26.158 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.483682+0000", 2020-09-08T15:38:26.159 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0607999999999999e-05 2020-09-08T15:38:26.159 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.159 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.159 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.159 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.483752+0000", 2020-09-08T15:38:26.160 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017509399999999999 2020-09-08T15:38:26.160 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.160 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.160 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:38:26.160 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.483927+0000", 2020-09-08T15:38:26.161 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016583900000000001 2020-09-08T15:38:26.161 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.161 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.161 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0", 2020-09-08T15:38:26.162 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.484093+0000", 2020-09-08T15:38:26.162 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00099057599999999996 2020-09-08T15:38:26.162 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.163 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.163 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:38:26.163 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.485084+0000", 2020-09-08T15:38:26.163 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.093327826000000003 2020-09-08T15:38:26.163 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.164 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.164 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:38:26.164 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.578412+0000", 2020-09-08T15:38:26.164 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1641000000000002e-05 2020-09-08T15:38:26.164 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.165 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.165 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:38:26.165 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.578463+0000", 2020-09-08T15:38:26.165 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4732999999999999e-05 2020-09-08T15:38:26.165 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.166 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.166 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:38:26.166 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:49.578518+0000", 2020-09-08T15:38:26.166 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.094969364000000001 2020-09-08T15:38:26.166 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.167 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.167 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.167 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.167 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.167 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:32411 2.4 2:2d81c0dc:::benchmark_data_smithi166_20551_object32410:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e135)", 2020-09-08T15:38:26.168 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:52.725040+0000", 2020-09-08T15:38:26.168 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 33.354517491999999, 2020-09-08T15:38:26.168 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.163565814, 2020-09-08T15:38:26.168 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:38:26.168 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:38:26.169 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:38:26.169 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:38:26.169 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:38:26.169 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 32411 2020-09-08T15:38:26.169 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.170 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:38:26.170 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.170 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:38:26.170 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.725040+0000", 2020-09-08T15:38:26.170 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:38:26.171 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.171 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.171 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:38:26.171 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.725040+0000", 2020-09-08T15:38:26.171 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999704 2020-09-08T15:38:26.172 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.172 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.172 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:38:26.173 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.725010+0000", 2020-09-08T15:38:26.173 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00024870900000000001 2020-09-08T15:38:26.173 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.174 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.174 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:38:26.174 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.725259+0000", 2020-09-08T15:38:26.174 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6727000000000001e-05 2020-09-08T15:38:26.174 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.175 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.175 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:38:26.175 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.725275+0000", 2020-09-08T15:38:26.175 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9310000000000002e-06 2020-09-08T15:38:26.175 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.176 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.176 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:38:26.176 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.725282+0000", 2020-09-08T15:38:26.176 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2176000000000003e-05 2020-09-08T15:38:26.176 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.177 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.177 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.177 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.725334+0000", 2020-09-08T15:38:26.177 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014729899999999999 2020-09-08T15:38:26.177 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.178 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.178 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:38:26.178 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.725482+0000", 2020-09-08T15:38:26.178 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019220399999999999 2020-09-08T15:38:26.178 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.179 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.179 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0", 2020-09-08T15:38:26.179 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.725674+0000", 2020-09-08T15:38:26.179 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0012154469999999999 2020-09-08T15:38:26.179 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.179 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.180 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:38:26.180 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.726889+0000", 2020-09-08T15:38:26.180 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16155966899999999 2020-09-08T15:38:26.180 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.180 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.181 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:38:26.181 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.888449+0000", 2020-09-08T15:38:26.181 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7992999999999995e-05 2020-09-08T15:38:26.181 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.181 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.182 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:38:26.182 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.888547+0000", 2020-09-08T15:38:26.182 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8455999999999997e-05 2020-09-08T15:38:26.182 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.182 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.183 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:38:26.183 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.888605+0000", 2020-09-08T15:38:26.184 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.163565814 2020-09-08T15:38:26.184 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.184 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.184 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.184 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.185 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.185 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:32415 2.4 2:23d81b7a:::benchmark_data_smithi166_20551_object32414:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e135)", 2020-09-08T15:38:26.185 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:52.728169+0000", 2020-09-08T15:38:26.185 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 33.351387871, 2020-09-08T15:38:26.186 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16079976800000001, 2020-09-08T15:38:26.186 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:38:26.186 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:38:26.186 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:38:26.186 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:38:26.187 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:38:26.187 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 32415 2020-09-08T15:38:26.187 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.187 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:38:26.187 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.188 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:38:26.188 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.728169+0000", 2020-09-08T15:38:26.188 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:38:26.188 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.188 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.189 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:38:26.189 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.728169+0000", 2020-09-08T15:38:26.189 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999213 2020-09-08T15:38:26.189 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.189 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.190 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:38:26.190 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.728090+0000", 2020-09-08T15:38:26.190 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00033498499999999999 2020-09-08T15:38:26.190 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.190 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.191 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:38:26.191 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.728425+0000", 2020-09-08T15:38:26.191 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1270000000000001e-05 2020-09-08T15:38:26.191 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.191 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.191 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:38:26.192 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.728447+0000", 2020-09-08T15:38:26.192 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.859e-06 2020-09-08T15:38:26.192 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.192 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.193 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:38:26.193 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.728457+0000", 2020-09-08T15:38:26.193 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6796000000000002e-05 2020-09-08T15:38:26.194 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.194 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.194 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.194 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.728523+0000", 2020-09-08T15:38:26.195 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000194398 2020-09-08T15:38:26.195 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.195 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.195 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:38:26.195 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.728718+0000", 2020-09-08T15:38:26.196 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000193943 2020-09-08T15:38:26.196 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.196 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.196 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0", 2020-09-08T15:38:26.196 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.728912+0000", 2020-09-08T15:38:26.196 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0029669140000000002 2020-09-08T15:38:26.197 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.197 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.197 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:38:26.197 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.731879+0000", 2020-09-08T15:38:26.197 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15698409399999999 2020-09-08T15:38:26.198 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.198 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.198 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:38:26.199 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.888863+0000", 2020-09-08T15:38:26.199 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7695000000000002e-05 2020-09-08T15:38:26.199 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.199 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.199 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:38:26.199 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.888920+0000", 2020-09-08T15:38:26.200 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8594000000000003e-05 2020-09-08T15:38:26.200 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.200 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.200 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:38:26.201 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.888969+0000", 2020-09-08T15:38:26.201 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16079976800000001 2020-09-08T15:38:26.201 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.201 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.202 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.202 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.202 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.202 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:32430 2.4 2:29145d6c:::benchmark_data_smithi166_20551_object32429:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e135)", 2020-09-08T15:38:26.202 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:52.733941+0000", 2020-09-08T15:38:26.203 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 33.345615643000002, 2020-09-08T15:38:26.203 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15599474599999999, 2020-09-08T15:38:26.203 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:38:26.203 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:38:26.203 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:38:26.204 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:38:26.204 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:38:26.205 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 32430 2020-09-08T15:38:26.205 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.205 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:38:26.205 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.205 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:38:26.206 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.733941+0000", 2020-09-08T15:38:26.206 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:38:26.206 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.206 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.206 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:38:26.207 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.733941+0000", 2020-09-08T15:38:26.207 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999552 2020-09-08T15:38:26.207 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.207 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.207 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:38:26.208 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.733897+0000", 2020-09-08T15:38:26.208 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000163552 2020-09-08T15:38:26.208 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.208 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.208 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:38:26.209 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.734060+0000", 2020-09-08T15:38:26.209 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7058000000000002e-05 2020-09-08T15:38:26.209 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.209 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.209 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:38:26.210 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.734077+0000", 2020-09-08T15:38:26.210 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.289e-06 2020-09-08T15:38:26.210 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.210 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.210 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:38:26.211 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.734085+0000", 2020-09-08T15:38:26.211 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5671000000000002e-05 2020-09-08T15:38:26.211 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.211 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.212 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.212 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.734150+0000", 2020-09-08T15:38:26.212 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012394199999999999 2020-09-08T15:38:26.212 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.212 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.213 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:38:26.213 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.734274+0000", 2020-09-08T15:38:26.213 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013091000000000001 2020-09-08T15:38:26.213 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.213 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.214 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0", 2020-09-08T15:38:26.214 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.734405+0000", 2020-09-08T15:38:26.214 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0011661429999999999 2020-09-08T15:38:26.215 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.215 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.215 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:38:26.215 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.735571+0000", 2020-09-08T15:38:26.216 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15417006799999999 2020-09-08T15:38:26.216 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.216 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.216 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:38:26.216 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.889741+0000", 2020-09-08T15:38:26.217 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010968600000000001 2020-09-08T15:38:26.217 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.217 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.217 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:38:26.217 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.889851+0000", 2020-09-08T15:38:26.218 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5228999999999993e-05 2020-09-08T15:38:26.218 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.218 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.218 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:38:26.218 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.889936+0000", 2020-09-08T15:38:26.219 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15599474599999999 2020-09-08T15:38:26.219 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.219 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.219 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.219 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.220 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.220 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:32448 2.4 2:22aecab5:::benchmark_data_smithi166_20551_object32447:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e135)", 2020-09-08T15:38:26.220 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:52.741814+0000", 2020-09-08T15:38:26.220 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 33.337743236999998, 2020-09-08T15:38:26.220 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14846495600000001, 2020-09-08T15:38:26.220 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:38:26.221 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:38:26.221 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:38:26.221 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:38:26.221 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:38:26.221 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 32448 2020-09-08T15:38:26.222 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.222 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:38:26.222 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.222 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:38:26.223 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.741814+0000", 2020-09-08T15:38:26.223 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:38:26.223 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.223 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.223 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:38:26.224 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.741814+0000", 2020-09-08T15:38:26.224 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999619 2020-09-08T15:38:26.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.224 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.225 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:38:26.225 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.741776+0000", 2020-09-08T15:38:26.226 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012643600000000001 2020-09-08T15:38:26.226 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.226 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.226 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:38:26.226 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.741902+0000", 2020-09-08T15:38:26.227 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4018e-05 2020-09-08T15:38:26.227 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.227 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.227 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:38:26.227 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.741916+0000", 2020-09-08T15:38:26.228 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3480000000000003e-06 2020-09-08T15:38:26.228 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.228 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.228 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:38:26.228 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.741921+0000", 2020-09-08T15:38:26.228 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5723999999999998e-05 2020-09-08T15:38:26.229 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.229 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.229 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.229 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.741967+0000", 2020-09-08T15:38:26.229 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000114413 2020-09-08T15:38:26.230 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.230 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.230 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:38:26.230 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.742082+0000", 2020-09-08T15:38:26.230 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000115788 2020-09-08T15:38:26.231 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.231 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.231 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0", 2020-09-08T15:38:26.231 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.742197+0000", 2020-09-08T15:38:26.231 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0014209470000000001 2020-09-08T15:38:26.232 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.232 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.232 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:38:26.232 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.743618+0000", 2020-09-08T15:38:26.232 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14647170900000001 2020-09-08T15:38:26.233 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.233 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.233 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:38:26.233 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.890090+0000", 2020-09-08T15:38:26.233 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000103793 2020-09-08T15:38:26.234 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.234 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.234 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:38:26.234 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.890194+0000", 2020-09-08T15:38:26.234 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5024000000000005e-05 2020-09-08T15:38:26.235 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.235 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.235 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:38:26.235 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.890279+0000", 2020-09-08T15:38:26.236 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14846495600000001 2020-09-08T15:38:26.236 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.237 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.237 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.237 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.237 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.237 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:32457 2.4 2:2c2525cc:::benchmark_data_smithi166_20551_object32456:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e135)", 2020-09-08T15:38:26.238 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:52.745639+0000", 2020-09-08T15:38:26.238 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 33.333918224000001, 2020-09-08T15:38:26.238 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14681170199999999, 2020-09-08T15:38:26.238 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:38:26.238 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:38:26.239 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:38:26.239 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:38:26.239 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:38:26.239 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 32457 2020-09-08T15:38:26.239 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.240 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:38:26.240 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.240 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:38:26.240 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.745639+0000", 2020-09-08T15:38:26.240 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:38:26.241 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.241 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.241 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:38:26.241 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.745639+0000", 2020-09-08T15:38:26.241 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999199 2020-09-08T15:38:26.242 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.242 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.242 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:38:26.242 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.745559+0000", 2020-09-08T15:38:26.242 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019272700000000001 2020-09-08T15:38:26.243 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.243 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.243 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:38:26.243 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.745752+0000", 2020-09-08T15:38:26.243 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8185999999999998e-05 2020-09-08T15:38:26.244 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.244 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.244 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:38:26.244 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.745770+0000", 2020-09-08T15:38:26.244 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3130000000000004e-06 2020-09-08T15:38:26.244 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.245 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.245 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:38:26.245 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.745779+0000", 2020-09-08T15:38:26.245 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4739000000000005e-05 2020-09-08T15:38:26.246 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.246 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.247 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.247 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.745844+0000", 2020-09-08T15:38:26.247 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00022162000000000001 2020-09-08T15:38:26.247 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.247 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.248 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:38:26.248 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.746066+0000", 2020-09-08T15:38:26.248 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000150517 2020-09-08T15:38:26.248 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.248 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.249 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0", 2020-09-08T15:38:26.249 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.746216+0000", 2020-09-08T15:38:26.249 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00113142 2020-09-08T15:38:26.250 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.250 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.250 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:38:26.250 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.747347+0000", 2020-09-08T15:38:26.251 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14498998199999999 2020-09-08T15:38:26.251 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.251 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.251 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:38:26.252 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.892337+0000", 2020-09-08T15:38:26.252 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2108999999999998e-05 2020-09-08T15:38:26.252 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.252 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.252 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:38:26.252 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.892400+0000", 2020-09-08T15:38:26.253 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1025e-05 2020-09-08T15:38:26.253 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.253 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.253 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:38:26.253 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.892451+0000", 2020-09-08T15:38:26.254 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14681170199999999 2020-09-08T15:38:26.254 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.254 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.254 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.254 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.255 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.255 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:45914 2.9 2:9add8887:::benchmark_data_smithi166_20551_object45913:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e136)", 2020-09-08T15:38:26.255 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:58.338386+0000", 2020-09-08T15:38:26.255 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 27.741171499, 2020-09-08T15:38:26.255 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.33603839000000002, 2020-09-08T15:38:26.256 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:38:26.256 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:38:26.256 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:38:26.256 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:38:26.257 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:38:26.257 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 45914 2020-09-08T15:38:26.258 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.258 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:38:26.258 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.258 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:38:26.258 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.338386+0000", 2020-09-08T15:38:26.259 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:38:26.259 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.259 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.259 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:38:26.259 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.338386+0000", 2020-09-08T15:38:26.260 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999657 2020-09-08T15:38:26.260 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.260 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.260 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:38:26.260 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.338351+0000", 2020-09-08T15:38:26.261 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012162700000000001 2020-09-08T15:38:26.261 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.261 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.261 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:38:26.262 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.338473+0000", 2020-09-08T15:38:26.262 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3688e-05 2020-09-08T15:38:26.262 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.262 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.262 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:38:26.263 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.338487+0000", 2020-09-08T15:38:26.263 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4189999999999999e-06 2020-09-08T15:38:26.263 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.263 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.263 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:38:26.264 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.338493+0000", 2020-09-08T15:38:26.264 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6821999999999999e-05 2020-09-08T15:38:26.264 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.264 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.264 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.265 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.338550+0000", 2020-09-08T15:38:26.265 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014732300000000001 2020-09-08T15:38:26.265 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.265 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.265 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:38:26.266 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.338697+0000", 2020-09-08T15:38:26.266 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017421200000000001 2020-09-08T15:38:26.266 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.266 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.266 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 7", 2020-09-08T15:38:26.267 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.338871+0000", 2020-09-08T15:38:26.267 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0019189579999999999 2020-09-08T15:38:26.267 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.268 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.268 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:38:26.268 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.340790+0000", 2020-09-08T15:38:26.268 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00096838999999999996 2020-09-08T15:38:26.268 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.269 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.269 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:38:26.269 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.341759+0000", 2020-09-08T15:38:26.269 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.33254169900000002 2020-09-08T15:38:26.269 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.270 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.270 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:38:26.270 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.674300+0000", 2020-09-08T15:38:26.270 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012358700000000001 2020-09-08T15:38:26.270 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.271 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.271 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:38:26.271 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.674424+0000", 2020-09-08T15:38:26.271 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.33603839000000002 2020-09-08T15:38:26.271 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.272 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.272 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.272 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.272 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.272 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop_reply(client.34742.0:45914 2.9 e136/126 ondisk, result = 0)", 2020-09-08T15:38:26.273 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:58.341646+0000", 2020-09-08T15:38:26.273 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 27.737911003000001, 2020-09-08T15:38:26.273 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.33279404800000001, 2020-09-08T15:38:26.273 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:38:26.273 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2020-09-08T15:38:26.274 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:38:26.274 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.274 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:38:26.274 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.341646+0000", 2020-09-08T15:38:26.275 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:38:26.275 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.275 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.275 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:38:26.275 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.341646+0000", 2020-09-08T15:38:26.275 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0679999999999998e-05 2020-09-08T15:38:26.276 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.276 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.276 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:38:26.276 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.341677+0000", 2020-09-08T15:38:26.276 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4219999999999999e-06 2020-09-08T15:38:26.277 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.277 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.277 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:38:26.277 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.341682+0000", 2020-09-08T15:38:26.278 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3870000000000001e-06 2020-09-08T15:38:26.278 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.278 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.279 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:38:26.279 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.341692+0000", 2020-09-08T15:38:26.279 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6439999999999999e-06 2020-09-08T15:38:26.279 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.279 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.280 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:38:26.280 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.341696+0000", 2020-09-08T15:38:26.280 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3989000000000002e-05 2020-09-08T15:38:26.280 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.280 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.281 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.281 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.341740+0000", 2020-09-08T15:38:26.281 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7999999999999993e-06 2020-09-08T15:38:26.281 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.281 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.282 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:38:26.282 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.341750+0000", 2020-09-08T15:38:26.282 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.33269012599999997 2020-09-08T15:38:26.282 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.282 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.282 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:38:26.283 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:58.674440+0000", 2020-09-08T15:38:26.283 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.33279404800000001 2020-09-08T15:38:26.283 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.283 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.283 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.284 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.284 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.284 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:84465 2.9 2:9ce63670:::benchmark_data_smithi166_20551_object84464:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:38:26.284 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:38:20.855834+0000", 2020-09-08T15:38:26.284 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 5.2237234839999998, 2020-09-08T15:38:26.285 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14177292499999999, 2020-09-08T15:38:26.285 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:38:26.285 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:38:26.285 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:38:26.285 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:38:26.286 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:38:26.286 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 84465 2020-09-08T15:38:26.286 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.286 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:38:26.287 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.287 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:38:26.287 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.855834+0000", 2020-09-08T15:38:26.287 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:38:26.288 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.288 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.288 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:38:26.289 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.855834+0000", 2020-09-08T15:38:26.289 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999728 2020-09-08T15:38:26.289 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.289 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.290 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:38:26.290 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.855806+0000", 2020-09-08T15:38:26.290 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000125697 2020-09-08T15:38:26.290 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.290 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.291 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:38:26.291 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.855932+0000", 2020-09-08T15:38:26.291 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3559e-05 2020-09-08T15:38:26.291 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.291 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.291 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:38:26.292 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.855946+0000", 2020-09-08T15:38:26.292 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4680000000000003e-06 2020-09-08T15:38:26.292 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.292 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.293 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:38:26.293 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.855953+0000", 2020-09-08T15:38:26.293 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1610999999999998e-05 2020-09-08T15:38:26.293 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.293 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.294 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.294 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.856005+0000", 2020-09-08T15:38:26.294 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013141499999999999 2020-09-08T15:38:26.294 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.294 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.294 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:38:26.295 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.856136+0000", 2020-09-08T15:38:26.295 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015049200000000001 2020-09-08T15:38:26.295 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.295 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.295 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 7", 2020-09-08T15:38:26.296 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.856287+0000", 2020-09-08T15:38:26.296 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0060231119999999997 2020-09-08T15:38:26.296 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.296 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.297 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:38:26.297 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.862310+0000", 2020-09-08T15:38:26.297 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13497637800000001 2020-09-08T15:38:26.297 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.297 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.298 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:38:26.298 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.997286+0000", 2020-09-08T15:38:26.298 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018540900000000001 2020-09-08T15:38:26.298 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.300 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.300 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:38:26.301 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.997471+0000", 2020-09-08T15:38:26.301 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013508899999999999 2020-09-08T15:38:26.301 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.301 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.302 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:38:26.302 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.997607+0000", 2020-09-08T15:38:26.302 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14177292499999999 2020-09-08T15:38:26.302 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.303 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.303 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.303 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.303 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.303 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:84468 2.9 2:9519f672:::benchmark_data_smithi166_20551_object84467:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:38:26.304 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:38:20.857689+0000", 2020-09-08T15:38:26.304 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 5.221868089, 2020-09-08T15:38:26.304 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.140301015, 2020-09-08T15:38:26.304 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:38:26.304 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:38:26.305 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:38:26.305 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:38:26.305 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:38:26.305 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 84468 2020-09-08T15:38:26.305 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.306 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:38:26.306 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.306 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:38:26.306 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.857689+0000", 2020-09-08T15:38:26.306 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:38:26.307 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.307 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.307 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:38:26.307 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.857689+0000", 2020-09-08T15:38:26.307 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999647 2020-09-08T15:38:26.308 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.308 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.308 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:38:26.308 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.857654+0000", 2020-09-08T15:38:26.308 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000126308 2020-09-08T15:38:26.309 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.309 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.310 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:38:26.310 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.857780+0000", 2020-09-08T15:38:26.310 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3217000000000001e-05 2020-09-08T15:38:26.310 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.310 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.311 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:38:26.311 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.857793+0000", 2020-09-08T15:38:26.311 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8880000000000002e-06 2020-09-08T15:38:26.311 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.312 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.312 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:38:26.312 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.857799+0000", 2020-09-08T15:38:26.312 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001316837 2020-09-08T15:38:26.312 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.313 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.313 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.313 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.859116+0000", 2020-09-08T15:38:26.313 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016222200000000001 2020-09-08T15:38:26.313 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.314 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.314 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:38:26.314 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.859278+0000", 2020-09-08T15:38:26.314 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000187386 2020-09-08T15:38:26.314 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.315 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.315 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 7", 2020-09-08T15:38:26.315 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.859465+0000", 2020-09-08T15:38:26.315 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002938146 2020-09-08T15:38:26.315 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.316 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.316 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:38:26.316 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.862404+0000", 2020-09-08T15:38:26.316 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13531863899999999 2020-09-08T15:38:26.316 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.316 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.317 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:38:26.317 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.997722+0000", 2020-09-08T15:38:26.317 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014556200000000001 2020-09-08T15:38:26.317 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.317 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.318 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:38:26.318 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.997868+0000", 2020-09-08T15:38:26.318 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012218299999999999 2020-09-08T15:38:26.318 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.318 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.319 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:38:26.319 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.997990+0000", 2020-09-08T15:38:26.319 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.140301015 2020-09-08T15:38:26.319 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.320 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.320 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.320 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.321 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.321 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:84469 2.4 2:21b78da7:::benchmark_data_smithi166_20551_object84468:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:38:26.321 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:38:20.857860+0000", 2020-09-08T15:38:26.321 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 5.2216970729999996, 2020-09-08T15:38:26.321 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.138338251, 2020-09-08T15:38:26.322 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:38:26.322 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:38:26.322 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:38:26.322 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:38:26.322 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:38:26.323 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 84469 2020-09-08T15:38:26.323 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.323 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:38:26.323 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.324 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:38:26.324 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.857860+0000", 2020-09-08T15:38:26.324 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:38:26.324 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.325 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.325 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:38:26.325 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.857860+0000", 2020-09-08T15:38:26.325 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999661 2020-09-08T15:38:26.325 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.325 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.326 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:38:26.326 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.857826+0000", 2020-09-08T15:38:26.326 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012272100000000001 2020-09-08T15:38:26.326 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.326 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.327 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:38:26.327 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.857949+0000", 2020-09-08T15:38:26.327 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1185e-05 2020-09-08T15:38:26.327 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.327 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.328 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:38:26.328 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.857960+0000", 2020-09-08T15:38:26.328 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5140000000000003e-06 2020-09-08T15:38:26.328 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.328 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.329 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:38:26.329 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.857965+0000", 2020-09-08T15:38:26.329 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0012059600000000001 2020-09-08T15:38:26.329 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.329 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.330 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.330 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.859171+0000", 2020-09-08T15:38:26.331 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016485299999999999 2020-09-08T15:38:26.331 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.331 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.331 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:38:26.331 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.859336+0000", 2020-09-08T15:38:26.332 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015021699999999999 2020-09-08T15:38:26.332 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.332 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.332 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0", 2020-09-08T15:38:26.332 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.859486+0000", 2020-09-08T15:38:26.332 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002701466 2020-09-08T15:38:26.333 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.333 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.333 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:38:26.333 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.862187+0000", 2020-09-08T15:38:26.334 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13388570999999999 2020-09-08T15:38:26.334 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.334 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.335 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:38:26.335 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.996073+0000", 2020-09-08T15:38:26.335 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1039999999999997e-05 2020-09-08T15:38:26.335 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.335 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.336 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:38:26.336 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.996144+0000", 2020-09-08T15:38:26.336 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4292000000000001e-05 2020-09-08T15:38:26.336 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.336 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.337 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:38:26.337 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.996198+0000", 2020-09-08T15:38:26.337 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.138338251 2020-09-08T15:38:26.337 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.337 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.338 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.338 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.338 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.338 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:84477 2.6 2:61ec41c6:::benchmark_data_smithi166_20551_object84476:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:38:26.339 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:38:20.862612+0000", 2020-09-08T15:38:26.339 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 5.2169448850000002, 2020-09-08T15:38:26.339 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.157649711, 2020-09-08T15:38:26.339 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:38:26.339 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:38:26.340 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:38:26.340 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:38:26.340 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:38:26.341 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 84477 2020-09-08T15:38:26.341 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.341 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:38:26.342 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.342 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:38:26.342 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.862612+0000", 2020-09-08T15:38:26.342 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:38:26.343 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.343 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.343 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:38:26.343 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.862612+0000", 2020-09-08T15:38:26.343 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999714 2020-09-08T15:38:26.344 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.344 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.344 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:38:26.344 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.862584+0000", 2020-09-08T15:38:26.344 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2379000000000004e-05 2020-09-08T15:38:26.345 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.345 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.345 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:38:26.345 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.862676+0000", 2020-09-08T15:38:26.345 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7070000000000004e-06 2020-09-08T15:38:26.346 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.346 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.346 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:38:26.346 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.862686+0000", 2020-09-08T15:38:26.346 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1050000000000003e-06 2020-09-08T15:38:26.347 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.347 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.347 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:38:26.347 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.862691+0000", 2020-09-08T15:38:26.347 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7036000000000002e-05 2020-09-08T15:38:26.348 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.348 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.348 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.348 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.862738+0000", 2020-09-08T15:38:26.348 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13393343999999999 2020-09-08T15:38:26.349 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.349 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.349 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:38:26.349 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.996671+0000", 2020-09-08T15:38:26.350 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00052789999999999998 2020-09-08T15:38:26.350 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.350 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.350 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8", 2020-09-08T15:38:26.351 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.997199+0000", 2020-09-08T15:38:26.351 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013889800000000001 2020-09-08T15:38:26.352 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.352 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.352 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:38:26.352 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:21.011089+0000", 2020-09-08T15:38:26.352 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0090960699999999995 2020-09-08T15:38:26.353 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.353 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.353 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:38:26.353 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:21.020185+0000", 2020-09-08T15:38:26.353 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3130000000000002e-05 2020-09-08T15:38:26.354 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.354 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.354 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:38:26.354 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:21.020228+0000", 2020-09-08T15:38:26.354 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3825000000000002e-05 2020-09-08T15:38:26.355 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.355 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.355 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:38:26.355 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:21.020262+0000", 2020-09-08T15:38:26.355 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.157649711 2020-09-08T15:38:26.356 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.356 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.356 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.356 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.356 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.357 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop_reply(client.34742.0:84469 2.4 e145/126 ondisk, result = 0)", 2020-09-08T15:38:26.357 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:38:20.862692+0000", 2020-09-08T15:38:26.357 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 5.2168646719999998, 2020-09-08T15:38:26.357 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13351410599999999, 2020-09-08T15:38:26.357 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:38:26.358 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2020-09-08T15:38:26.358 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:38:26.358 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.358 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:38:26.358 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.862692+0000", 2020-09-08T15:38:26.359 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:38:26.359 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.359 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.359 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:38:26.359 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.862692+0000", 2020-09-08T15:38:26.360 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7290000000000003e-05 2020-09-08T15:38:26.360 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.360 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.360 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:38:26.360 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.862740+0000", 2020-09-08T15:38:26.361 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9689999999999994e-06 2020-09-08T15:38:26.361 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.361 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.361 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:38:26.362 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.862748+0000", 2020-09-08T15:38:26.362 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3324e-05 2020-09-08T15:38:26.363 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.363 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.363 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:38:26.363 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.862761+0000", 2020-09-08T15:38:26.363 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8100000000000003e-06 2020-09-08T15:38:26.364 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.364 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.364 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:38:26.364 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.862767+0000", 2020-09-08T15:38:26.364 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.133282966 2020-09-08T15:38:26.365 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.365 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.365 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.365 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.996050+0000", 2020-09-08T15:38:26.365 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1627999999999999e-05 2020-09-08T15:38:26.366 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.366 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.366 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:38:26.366 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.996061+0000", 2020-09-08T15:38:26.366 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000145119 2020-09-08T15:38:26.367 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.367 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.367 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:38:26.367 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.996207+0000", 2020-09-08T15:38:26.367 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13351410599999999 2020-09-08T15:38:26.368 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.368 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.368 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.368 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.368 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.369 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:87286 2.4 2:238845f3:::benchmark_data_smithi166_20551_object87285:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:38:26.369 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:38:22.688237+0000", 2020-09-08T15:38:26.369 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 3.391320114, 2020-09-08T15:38:26.369 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.095563842999999996, 2020-09-08T15:38:26.369 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:38:26.370 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:38:26.370 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:38:26.370 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:38:26.370 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:38:26.370 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87286 2020-09-08T15:38:26.371 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.371 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:38:26.371 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.371 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:38:26.371 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.688237+0000", 2020-09-08T15:38:26.371 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:38:26.372 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.372 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.373 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:38:26.373 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.688237+0000", 2020-09-08T15:38:26.373 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999447 2020-09-08T15:38:26.373 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.373 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.374 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:38:26.374 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.688181+0000", 2020-09-08T15:38:26.374 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017531400000000001 2020-09-08T15:38:26.374 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.375 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.375 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:38:26.375 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.688357+0000", 2020-09-08T15:38:26.375 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6180000000000001e-05 2020-09-08T15:38:26.376 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.376 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.376 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:38:26.376 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.688373+0000", 2020-09-08T15:38:26.376 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5800000000000003e-06 2020-09-08T15:38:26.377 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.377 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.377 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:38:26.377 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.688381+0000", 2020-09-08T15:38:26.377 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7167999999999999e-05 2020-09-08T15:38:26.378 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.378 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.378 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.378 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.688438+0000", 2020-09-08T15:38:26.378 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016197299999999999 2020-09-08T15:38:26.379 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.379 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.379 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:38:26.379 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.688600+0000", 2020-09-08T15:38:26.379 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016998100000000001 2020-09-08T15:38:26.380 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.380 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.380 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0", 2020-09-08T15:38:26.380 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.688770+0000", 2020-09-08T15:38:26.380 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0052253350000000002 2020-09-08T15:38:26.381 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.381 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.381 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:38:26.381 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.693995+0000", 2020-09-08T15:38:26.381 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.089662012999999999 2020-09-08T15:38:26.382 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.382 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.382 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:38:26.382 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.783657+0000", 2020-09-08T15:38:26.382 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.8233999999999997e-05 2020-09-08T15:38:26.383 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.384 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.384 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:38:26.384 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.783735+0000", 2020-09-08T15:38:26.384 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5576999999999999e-05 2020-09-08T15:38:26.385 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.385 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.385 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:38:26.385 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.783801+0000", 2020-09-08T15:38:26.385 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.095563842999999996 2020-09-08T15:38:26.386 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.386 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.386 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.386 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.386 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.387 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:87291 2.4 2:23fa5ae9:::benchmark_data_smithi166_20551_object87290:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:38:26.387 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:38:22.691977+0000", 2020-09-08T15:38:26.387 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 3.387579611, 2020-09-08T15:38:26.387 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.103755561, 2020-09-08T15:38:26.388 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:38:26.388 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:38:26.388 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:38:26.388 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:38:26.389 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:38:26.389 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87291 2020-09-08T15:38:26.389 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.389 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:38:26.389 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.390 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:38:26.390 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.691977+0000", 2020-09-08T15:38:26.390 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:38:26.390 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.390 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.391 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:38:26.391 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.691977+0000", 2020-09-08T15:38:26.391 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999757 2020-09-08T15:38:26.391 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.391 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.392 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:38:26.392 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.691953+0000", 2020-09-08T15:38:26.392 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010499 2020-09-08T15:38:26.392 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.392 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.393 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:38:26.393 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.692058+0000", 2020-09-08T15:38:26.393 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5429999999999994e-06 2020-09-08T15:38:26.394 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.394 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.395 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:38:26.395 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.692067+0000", 2020-09-08T15:38:26.395 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7060000000000003e-06 2020-09-08T15:38:26.395 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.395 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.396 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:38:26.396 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.692071+0000", 2020-09-08T15:38:26.396 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0019755430000000002 2020-09-08T15:38:26.396 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.396 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.397 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.397 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.694047+0000", 2020-09-08T15:38:26.397 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000150395 2020-09-08T15:38:26.397 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.397 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.398 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:38:26.398 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.694197+0000", 2020-09-08T15:38:26.398 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00027820600000000003 2020-09-08T15:38:26.398 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.398 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.399 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0", 2020-09-08T15:38:26.399 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.694476+0000", 2020-09-08T15:38:26.399 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.089348897999999996 2020-09-08T15:38:26.399 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.399 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.400 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:38:26.400 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.783825+0000", 2020-09-08T15:38:26.400 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0066017949999999997 2020-09-08T15:38:26.400 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.400 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.400 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:38:26.401 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.790426+0000", 2020-09-08T15:38:26.401 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7404000000000006e-05 2020-09-08T15:38:26.401 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.401 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.402 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:38:26.402 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.790504+0000", 2020-09-08T15:38:26.402 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0052293139999999997 2020-09-08T15:38:26.402 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.403 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.403 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:38:26.403 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.795733+0000", 2020-09-08T15:38:26.403 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.103755561 2020-09-08T15:38:26.404 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.404 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.405 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.405 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.405 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.405 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:87293 2.2 e145/126 2:406fb1a3:::benchmark_data_smithi166_20551_object87292:head v 145'5428, mlcod=145'5426)", 2020-09-08T15:38:26.405 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:38:22.694996+0000", 2020-09-08T15:38:26.406 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 3.3845607690000001, 2020-09-08T15:38:26.406 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.101989514, 2020-09-08T15:38:26.406 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:38:26.406 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:38:26.406 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:38:26.407 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.407 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:38:26.407 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.694996+0000", 2020-09-08T15:38:26.407 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:38:26.407 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.408 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.408 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:38:26.408 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.694996+0000", 2020-09-08T15:38:26.408 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1357e-05 2020-09-08T15:38:26.408 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.409 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.409 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:38:26.409 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.695088+0000", 2020-09-08T15:38:26.409 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7889999999999995e-06 2020-09-08T15:38:26.409 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.410 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.410 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:38:26.410 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.695095+0000", 2020-09-08T15:38:26.410 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2513e-05 2020-09-08T15:38:26.410 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.411 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.411 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:38:26.411 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.695108+0000", 2020-09-08T15:38:26.411 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8940000000000001e-06 2020-09-08T15:38:26.411 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.411 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.412 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:38:26.412 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.695113+0000", 2020-09-08T15:38:26.412 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.088508892000000006 2020-09-08T15:38:26.412 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.412 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.413 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:38:26.413 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.783622+0000", 2020-09-08T15:38:26.413 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.8226000000000006e-05 2020-09-08T15:38:26.413 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.413 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.414 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:38:26.414 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.783700+0000", 2020-09-08T15:38:26.415 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013101843 2020-09-08T15:38:26.415 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.415 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.416 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:38:26.416 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.796802+0000", 2020-09-08T15:38:26.416 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000184 2020-09-08T15:38:26.416 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:38:26.416 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:38:26.417 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:38:26.417 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.796986+0000", 2020-09-08T15:38:26.417 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.101989514 2020-09-08T15:38:26.417 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.417 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.418 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.418 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:38:26.418 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:38:26.418 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:38:26.419 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:25 smithi166 bash[9446]: cluster 2020-09-08T15:38:25.603266+0000 mgr.x (mgr.24118) 327 : cluster [DBG] pgmap v394: 17 pgs: 17 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s wr, 1.29k op/s 2020-09-08T15:38:26.647 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:26.751 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:26.771 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 44 16 93121 93105 132.239 91.625 0.00371136 0.00755199 2020-09-08T15:38:26.910 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:26.927 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:38:26 smithi104 bash[22305]: debug 2020-09-08T15:38:26.646+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:27.014 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:38:27.115 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:38:26 smithi166 bash[16593]: debug 2020-09-08T15:38:26.908+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:27.742 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:38:27.778 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 45 16 94317 94301 130.962 74.75 0.0100045 0.00763386 2020-09-08T15:38:27.846 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:27.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:27 smithi055 bash[9296]: cluster 2020-09-08T15:38:26.255088+0000 osd.8 (osd.8) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:27.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:27 smithi055 bash[9296]: cluster 2020-09-08T15:38:26.255184+0000 osd.8 (osd.8) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:27.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:27 smithi055 bash[9296]: cluster 2020-09-08T15:38:26.654176+0000 mon.a (mon.0) 948 : cluster [DBG] osdmap e146: 12 total, 12 up, 12 in 2020-09-08T15:38:27.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:27 smithi055 bash[9296]: cluster 2020-09-08T15:38:27.603794+0000 mgr.x (mgr.24118) 328 : cluster [DBG] pgmap v396: 17 pgs: 17 active+clean; 5.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s wr, 1.59k op/s 2020-09-08T15:38:27.927 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:38:27 smithi104 bash[17588]: debug 2020-09-08T15:38:27.742+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:27.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:27 smithi104 bash[9203]: cluster 2020-09-08T15:38:26.255088+0000 osd.8 (osd.8) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:27.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:27 smithi104 bash[9203]: cluster 2020-09-08T15:38:26.255184+0000 osd.8 (osd.8) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:27.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:27 smithi104 bash[9203]: cluster 2020-09-08T15:38:26.654176+0000 mon.a (mon.0) 948 : cluster [DBG] osdmap e146: 12 total, 12 up, 12 in 2020-09-08T15:38:27.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:27 smithi104 bash[9203]: cluster 2020-09-08T15:38:27.603794+0000 mgr.x (mgr.24118) 328 : cluster [DBG] pgmap v396: 17 pgs: 17 active+clean; 5.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s wr, 1.59k op/s 2020-09-08T15:38:27.973 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:28.078 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:28.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:27 smithi166 bash[9446]: cluster 2020-09-08T15:38:26.255088+0000 osd.8 (osd.8) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:28.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:27 smithi166 bash[9446]: cluster 2020-09-08T15:38:26.255184+0000 osd.8 (osd.8) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:28.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:27 smithi166 bash[9446]: cluster 2020-09-08T15:38:26.654176+0000 mon.a (mon.0) 948 : cluster [DBG] osdmap e146: 12 total, 12 up, 12 in 2020-09-08T15:38:28.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:27 smithi166 bash[9446]: cluster 2020-09-08T15:38:27.603794+0000 mgr.x (mgr.24118) 328 : cluster [DBG] pgmap v396: 17 pgs: 17 active+clean; 5.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s wr, 1.59k op/s 2020-09-08T15:38:28.191 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:28.294 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:38:28.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:38:28 smithi055 bash[14301]: debug 2020-09-08T15:38:28.193+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:28.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:38:27 smithi055 bash[19003]: debug 2020-09-08T15:38:27.977+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:28.771 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 46 16 95829 95813 130.169 94.5 0.0114092 0.00768043 2020-09-08T15:38:29.193 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:38:29.207 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2020-09-08T15:38:29.296 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:29.403 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:38:29.677 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:38:29 smithi104 bash[17588]: debug 2020-09-08T15:38:29.190+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:29.771 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 47 16 97158 97142 129.166 83.0625 0.00834324 0.00774017 2020-09-08T15:38:30.134 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:30.245 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:38:30.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:38:30 smithi055 bash[19003]: debug 2020-09-08T15:38:30.137+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:30.562 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:30 smithi104 bash[9203]: cluster 2020-09-08T15:38:29.570052+0000 mon.a (mon.0) 949 : cluster [DBG] osdmap e147: 12 total, 12 up, 12 in 2020-09-08T15:38:30.563 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:30 smithi104 bash[9203]: cluster 2020-09-08T15:38:29.604345+0000 mgr.x (mgr.24118) 329 : cluster [DBG] pgmap v398: 17 pgs: 17 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 98 MiB/s wr, 1.56k op/s 2020-09-08T15:38:30.771 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 48 15 98670 98655 128.445 94.5625 0.0130884 0.00778337 2020-09-08T15:38:30.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:30 smithi166 bash[9446]: cluster 2020-09-08T15:38:29.570052+0000 mon.a (mon.0) 949 : cluster [DBG] osdmap e147: 12 total, 12 up, 12 in 2020-09-08T15:38:30.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:30 smithi166 bash[9446]: cluster 2020-09-08T15:38:29.604345+0000 mgr.x (mgr.24118) 329 : cluster [DBG] pgmap v398: 17 pgs: 17 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 98 MiB/s wr, 1.56k op/s 2020-09-08T15:38:30.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:30 smithi055 bash[9296]: cluster 2020-09-08T15:38:29.570052+0000 mon.a (mon.0) 949 : cluster [DBG] osdmap e147: 12 total, 12 up, 12 in 2020-09-08T15:38:30.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:30 smithi055 bash[9296]: cluster 2020-09-08T15:38:29.604345+0000 mgr.x (mgr.24118) 329 : cluster [DBG] pgmap v398: 17 pgs: 17 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 98 MiB/s wr, 1.56k op/s 2020-09-08T15:38:31.092 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:38:31.202 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:38:31.336 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:38:31.427 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:38:31 smithi104 bash[19922]: debug 2020-09-08T15:38:31.090+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:31.440 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:38:31.614 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:38:31 smithi166 bash[11928]: debug 2020-09-08T15:38:31.332+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:31.771 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 49 16 100124 100108 127.677 90.8125 0.00850075 0.00782817 2020-09-08T15:38:31.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:31 smithi104 bash[9203]: cluster 2020-09-08T15:38:31.604780+0000 mgr.x (mgr.24118) 330 : cluster [DBG] pgmap v399: 17 pgs: 17 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 MiB/s wr, 1.18k op/s 2020-09-08T15:38:32.074 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:38:32.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:31 smithi166 bash[9446]: cluster 2020-09-08T15:38:31.604780+0000 mgr.x (mgr.24118) 330 : cluster [DBG] pgmap v399: 17 pgs: 17 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 MiB/s wr, 1.18k op/s 2020-09-08T15:38:32.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:31 smithi055 bash[9296]: cluster 2020-09-08T15:38:31.604780+0000 mgr.x (mgr.24118) 330 : cluster [DBG] pgmap v399: 17 pgs: 17 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 MiB/s wr, 1.18k op/s 2020-09-08T15:38:32.176 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:32.313 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:32.417 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:38:32.427 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:38:32 smithi104 bash[19922]: debug 2020-09-08T15:38:32.058+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:32.564 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:38:32.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:38:32 smithi055 bash[16646]: debug 2020-09-08T15:38:32.313+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:32.671 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:32.771 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 50 16 101511 101495 126.857 86.6875 0.0181389 0.00788084 2020-09-08T15:38:32.778 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:32.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:38:32 smithi166 bash[18960]: debug 2020-09-08T15:38:32.564+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:32.882 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:38:33.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:38:32 smithi055 bash[19003]: debug 2020-09-08T15:38:32.777+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:33.270 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:38:33.374 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:33.499 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:33.603 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:33.677 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:38:33 smithi104 bash[15273]: debug 2020-09-08T15:38:33.270+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:33.719 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:33.771 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 51 16 102819 102803 125.973 81.75 0.0183009 0.00793628 2020-09-08T15:38:33.822 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:38:33.870 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:38:33 smithi166 bash[16593]: debug 2020-09-08T15:38:33.496+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:33.908 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:38:33 smithi055 bash[19003]: debug 2020-09-08T15:38:33.721+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:34.160 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:33 smithi104 bash[9203]: cluster 2020-09-08T15:38:32.892288+0000 osd.1 (osd.1) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:34.161 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:33 smithi104 bash[9203]: cluster 2020-09-08T15:38:32.892373+0000 osd.1 (osd.1) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:34.161 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:33 smithi104 bash[9203]: cluster 2020-09-08T15:38:32.951362+0000 mon.a (mon.0) 950 : cluster [DBG] osdmap e148: 12 total, 12 up, 12 in 2020-09-08T15:38:34.161 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:33 smithi104 bash[9203]: cluster 2020-09-08T15:38:33.038187+0000 osd.11 (osd.11) 9 : cluster [DBG] 2.c scrub starts 2020-09-08T15:38:34.161 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:33 smithi104 bash[9203]: cluster 2020-09-08T15:38:33.146167+0000 osd.7 (osd.7) 23 : cluster [DBG] 2.0 scrub starts 2020-09-08T15:38:34.162 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:33 smithi104 bash[9203]: cluster 2020-09-08T15:38:33.605233+0000 mgr.x (mgr.24118) 331 : cluster [DBG] pgmap v401: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 95 MiB/s wr, 1.52k op/s 2020-09-08T15:38:34.162 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:38:34.266 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:38:34.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:33 smithi166 bash[9446]: cluster 2020-09-08T15:38:32.892288+0000 osd.1 (osd.1) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:34.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:33 smithi166 bash[9446]: cluster 2020-09-08T15:38:32.892373+0000 osd.1 (osd.1) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:34.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:33 smithi166 bash[9446]: cluster 2020-09-08T15:38:32.951362+0000 mon.a (mon.0) 950 : cluster [DBG] osdmap e148: 12 total, 12 up, 12 in 2020-09-08T15:38:34.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:33 smithi166 bash[9446]: cluster 2020-09-08T15:38:33.038187+0000 osd.11 (osd.11) 9 : cluster [DBG] 2.c scrub starts 2020-09-08T15:38:34.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:33 smithi166 bash[9446]: cluster 2020-09-08T15:38:33.146167+0000 osd.7 (osd.7) 23 : cluster [DBG] 2.0 scrub starts 2020-09-08T15:38:34.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:33 smithi166 bash[9446]: cluster 2020-09-08T15:38:33.605233+0000 mgr.x (mgr.24118) 331 : cluster [DBG] pgmap v401: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 95 MiB/s wr, 1.52k op/s 2020-09-08T15:38:34.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:33 smithi055 bash[9296]: cluster 2020-09-08T15:38:32.892288+0000 osd.1 (osd.1) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:34.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:33 smithi055 bash[9296]: cluster 2020-09-08T15:38:32.892373+0000 osd.1 (osd.1) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:34.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:33 smithi055 bash[9296]: cluster 2020-09-08T15:38:32.951362+0000 mon.a (mon.0) 950 : cluster [DBG] osdmap e148: 12 total, 12 up, 12 in 2020-09-08T15:38:34.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:33 smithi055 bash[9296]: cluster 2020-09-08T15:38:33.038187+0000 osd.11 (osd.11) 9 : cluster [DBG] 2.c scrub starts 2020-09-08T15:38:34.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:33 smithi055 bash[9296]: cluster 2020-09-08T15:38:33.146167+0000 osd.7 (osd.7) 23 : cluster [DBG] 2.0 scrub starts 2020-09-08T15:38:34.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:33 smithi055 bash[9296]: cluster 2020-09-08T15:38:33.605233+0000 mgr.x (mgr.24118) 331 : cluster [DBG] pgmap v401: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 95 MiB/s wr, 1.52k op/s 2020-09-08T15:38:34.427 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:38:34 smithi104 bash[17588]: debug 2020-09-08T15:38:34.162+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:34.771 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 52 15 104146 104131 125.146 83 0.00635053 0.00798883 2020-09-08T15:38:35.042 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:38:35.150 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:35.367 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:35.388 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:38:35 smithi104 bash[19922]: debug 2020-09-08T15:38:35.042+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:35.389 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:38:35 smithi104 bash[22305]: debug 2020-09-08T15:38:35.358+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:35.470 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:35.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:35 smithi104 bash[9203]: cluster 2020-09-08T15:38:34.568886+0000 mon.a (mon.0) 951 : cluster [DBG] osdmap e149: 12 total, 12 up, 12 in 2020-09-08T15:38:35.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:35 smithi104 bash[9203]: cluster 2020-09-08T15:38:35.029678+0000 osd.11 (osd.11) 10 : cluster [DBG] 2.c scrub ok 2020-09-08T15:38:35.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:35 smithi104 bash[9203]: cluster 2020-09-08T15:38:35.132196+0000 osd.1 (osd.1) 23 : cluster [DBG] 2.6 scrub starts 2020-09-08T15:38:35.774 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 53 15 105329 105314 124.18 73.9375 0.0145637 0.00805099 2020-09-08T15:38:35.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:35 smithi166 bash[9446]: cluster 2020-09-08T15:38:34.568886+0000 mon.a (mon.0) 951 : cluster [DBG] osdmap e149: 12 total, 12 up, 12 in 2020-09-08T15:38:35.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:35 smithi166 bash[9446]: cluster 2020-09-08T15:38:35.029678+0000 osd.11 (osd.11) 10 : cluster [DBG] 2.c scrub ok 2020-09-08T15:38:35.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:35 smithi166 bash[9446]: cluster 2020-09-08T15:38:35.132196+0000 osd.1 (osd.1) 23 : cluster [DBG] 2.6 scrub starts 2020-09-08T15:38:35.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:35 smithi055 bash[9296]: cluster 2020-09-08T15:38:34.568886+0000 mon.a (mon.0) 951 : cluster [DBG] osdmap e149: 12 total, 12 up, 12 in 2020-09-08T15:38:35.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:35 smithi055 bash[9296]: cluster 2020-09-08T15:38:35.029678+0000 osd.11 (osd.11) 10 : cluster [DBG] 2.c scrub ok 2020-09-08T15:38:35.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:35 smithi055 bash[9296]: cluster 2020-09-08T15:38:35.132196+0000 osd.1 (osd.1) 23 : cluster [DBG] 2.6 scrub starts 2020-09-08T15:38:36.578 INFO:teuthology.orchestra.run.smithi104.stderr:reweighted osd.1 to 1 (10000) 2020-09-08T15:38:36.679 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:38:36.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:36 smithi104 bash[9203]: cluster 2020-09-08T15:38:34.801579+0000 osd.7 (osd.7) 24 : cluster [DBG] 2.0 scrub ok 2020-09-08T15:38:36.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:36 smithi104 bash[9203]: cluster 2020-09-08T15:38:35.098369+0000 osd.2 (osd.2) 17 : cluster [DBG] 2.e scrub starts 2020-09-08T15:38:36.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:36 smithi104 bash[9203]: cluster 2020-09-08T15:38:35.605629+0000 mgr.x (mgr.24118) 332 : cluster [DBG] pgmap v403: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s wr, 1.16k op/s 2020-09-08T15:38:36.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:36 smithi104 bash[9203]: audit 2020-09-08T15:38:35.653429+0000 mon.c (mon.2) 34 : audit [INF] from='client.? v1:172.21.15.104:0/1057273075' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2020-09-08T15:38:36.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:36 smithi104 bash[9203]: audit 2020-09-08T15:38:35.656150+0000 mon.a (mon.0) 952 : audit [INF] from='client.34778 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2020-09-08T15:38:36.725 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:38:36.773 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 54 16 106610 106594 123.361 80 0.0247917 0.00810389 2020-09-08T15:38:36.773 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:38:36.820 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:38:36.861 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:38:36.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:36 smithi166 bash[9446]: cluster 2020-09-08T15:38:34.801579+0000 osd.7 (osd.7) 24 : cluster [DBG] 2.0 scrub ok 2020-09-08T15:38:36.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:36 smithi166 bash[9446]: cluster 2020-09-08T15:38:35.098369+0000 osd.2 (osd.2) 17 : cluster [DBG] 2.e scrub starts 2020-09-08T15:38:36.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:36 smithi166 bash[9446]: cluster 2020-09-08T15:38:35.605629+0000 mgr.x (mgr.24118) 332 : cluster [DBG] pgmap v403: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s wr, 1.16k op/s 2020-09-08T15:38:36.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:36 smithi166 bash[9446]: audit 2020-09-08T15:38:35.653429+0000 mon.c (mon.2) 34 : audit [INF] from='client.? v1:172.21.15.104:0/1057273075' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2020-09-08T15:38:36.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:36 smithi166 bash[9446]: audit 2020-09-08T15:38:35.656150+0000 mon.a (mon.0) 952 : audit [INF] from='client.34778 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2020-09-08T15:38:36.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:36 smithi055 bash[9296]: cluster 2020-09-08T15:38:34.801579+0000 osd.7 (osd.7) 24 : cluster [DBG] 2.0 scrub ok 2020-09-08T15:38:36.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:36 smithi055 bash[9296]: cluster 2020-09-08T15:38:35.098369+0000 osd.2 (osd.2) 17 : cluster [DBG] 2.e scrub starts 2020-09-08T15:38:36.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:36 smithi055 bash[9296]: cluster 2020-09-08T15:38:35.605629+0000 mgr.x (mgr.24118) 332 : cluster [DBG] pgmap v403: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s wr, 1.16k op/s 2020-09-08T15:38:36.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:36 smithi055 bash[9296]: audit 2020-09-08T15:38:35.653429+0000 mon.c (mon.2) 34 : audit [INF] from='client.? v1:172.21.15.104:0/1057273075' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2020-09-08T15:38:36.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:36 smithi055 bash[9296]: audit 2020-09-08T15:38:35.656150+0000 mon.a (mon.0) 952 : audit [INF] from='client.34778 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2020-09-08T15:38:36.911 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:38:36.957 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:38:37.005 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:38:37.036 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:38:37.091 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:38:37.110 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:38:37.150 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:38:37.166 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:38:37.166 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:38:37.166 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:38:37.167 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:38:37.167 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:38:37.167 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:38:37.167 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:38:37.167 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:38:37.168 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:38:37.168 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:38:37.168 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:38:37.168 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:38:37.579 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:37 smithi104 bash[9203]: audit 2020-09-08T15:38:36.572231+0000 mon.a (mon.0) 953 : audit [INF] from='client.34778 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2020-09-08T15:38:37.580 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:37 smithi104 bash[9203]: cluster 2020-09-08T15:38:36.572298+0000 mon.a (mon.0) 954 : cluster [DBG] osdmap e150: 12 total, 12 up, 12 in 2020-09-08T15:38:37.580 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:37 smithi104 bash[9203]: audit 2020-09-08T15:38:36.632519+0000 mon.a (mon.0) 955 : audit [DBG] from='client.? v1:172.21.15.104:0/1407646918' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:38:37.580 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:37 smithi104 bash[9203]: audit 2020-09-08T15:38:37.224562+0000 mon.a (mon.0) 956 : audit [INF] from='client.34790 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:38:37.580 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:37 smithi104 bash[9203]: audit 2020-09-08T15:38:37.224771+0000 mon.c (mon.2) 35 : audit [INF] from='client.? v1:172.21.15.104:0/1895282992' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:38:37.581 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:38:37.772 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 55 16 107902 107886 122.587 80.75 0.0195079 0.00815565 2020-09-08T15:38:37.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:37 smithi166 bash[9446]: audit 2020-09-08T15:38:36.572231+0000 mon.a (mon.0) 953 : audit [INF] from='client.34778 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2020-09-08T15:38:37.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:37 smithi166 bash[9446]: cluster 2020-09-08T15:38:36.572298+0000 mon.a (mon.0) 954 : cluster [DBG] osdmap e150: 12 total, 12 up, 12 in 2020-09-08T15:38:37.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:37 smithi166 bash[9446]: audit 2020-09-08T15:38:36.632519+0000 mon.a (mon.0) 955 : audit [DBG] from='client.? v1:172.21.15.104:0/1407646918' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:38:37.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:37 smithi166 bash[9446]: audit 2020-09-08T15:38:37.224562+0000 mon.a (mon.0) 956 : audit [INF] from='client.34790 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:38:37.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:37 smithi166 bash[9446]: audit 2020-09-08T15:38:37.224771+0000 mon.c (mon.2) 35 : audit [INF] from='client.? v1:172.21.15.104:0/1895282992' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:38:37.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:37 smithi055 bash[9296]: audit 2020-09-08T15:38:36.572231+0000 mon.a (mon.0) 953 : audit [INF] from='client.34778 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2020-09-08T15:38:37.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:37 smithi055 bash[9296]: cluster 2020-09-08T15:38:36.572298+0000 mon.a (mon.0) 954 : cluster [DBG] osdmap e150: 12 total, 12 up, 12 in 2020-09-08T15:38:37.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:37 smithi055 bash[9296]: audit 2020-09-08T15:38:36.632519+0000 mon.a (mon.0) 955 : audit [DBG] from='client.? v1:172.21.15.104:0/1407646918' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:38:37.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:37 smithi055 bash[9296]: audit 2020-09-08T15:38:37.224562+0000 mon.a (mon.0) 956 : audit [INF] from='client.34790 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:38:37.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:37 smithi055 bash[9296]: audit 2020-09-08T15:38:37.224771+0000 mon.c (mon.2) 35 : audit [INF] from='client.? v1:172.21.15.104:0/1895282992' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:38:38.774 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 56 16 109141 109125 121.78 77.4375 0.00659508 0.00820956 2020-09-08T15:38:38.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:38 smithi166 bash[9446]: cluster 2020-09-08T15:38:37.574788+0000 mon.a (mon.0) 957 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:38:38.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:38 smithi166 bash[9446]: audit 2020-09-08T15:38:37.578164+0000 mon.a (mon.0) 958 : audit [INF] from='client.34790 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:38:38.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:38 smithi166 bash[9446]: cluster 2020-09-08T15:38:37.578211+0000 mon.a (mon.0) 959 : cluster [DBG] osdmap e151: 12 total, 12 up, 12 in 2020-09-08T15:38:38.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:38 smithi166 bash[9446]: cluster 2020-09-08T15:38:37.606079+0000 mgr.x (mgr.24118) 333 : cluster [DBG] pgmap v406: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s wr, 1.14k op/s 2020-09-08T15:38:38.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:38 smithi055 bash[9296]: cluster 2020-09-08T15:38:37.574788+0000 mon.a (mon.0) 957 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:38:38.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:38 smithi055 bash[9296]: audit 2020-09-08T15:38:37.578164+0000 mon.a (mon.0) 958 : audit [INF] from='client.34790 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:38:38.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:38 smithi055 bash[9296]: cluster 2020-09-08T15:38:37.578211+0000 mon.a (mon.0) 959 : cluster [DBG] osdmap e151: 12 total, 12 up, 12 in 2020-09-08T15:38:38.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:38 smithi055 bash[9296]: cluster 2020-09-08T15:38:37.606079+0000 mgr.x (mgr.24118) 333 : cluster [DBG] pgmap v406: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s wr, 1.14k op/s 2020-09-08T15:38:38.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:38 smithi104 bash[9203]: cluster 2020-09-08T15:38:37.574788+0000 mon.a (mon.0) 957 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:38:38.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:38 smithi104 bash[9203]: audit 2020-09-08T15:38:37.578164+0000 mon.a (mon.0) 958 : audit [INF] from='client.34790 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:38:38.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:38 smithi104 bash[9203]: cluster 2020-09-08T15:38:37.578211+0000 mon.a (mon.0) 959 : cluster [DBG] osdmap e151: 12 total, 12 up, 12 in 2020-09-08T15:38:38.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:38 smithi104 bash[9203]: cluster 2020-09-08T15:38:37.606079+0000 mgr.x (mgr.24118) 333 : cluster [DBG] pgmap v406: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s wr, 1.14k op/s 2020-09-08T15:38:39.772 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 57 16 110445 110429 121.073 81.5 0.010635 0.00825765 2020-09-08T15:38:39.830 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd reweight 2 1 2020-09-08T15:38:39.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:39 smithi166 bash[9446]: cluster 2020-09-08T15:38:38.302574+0000 osd.6 (osd.6) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:39.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:39 smithi166 bash[9446]: cluster 2020-09-08T15:38:38.302705+0000 osd.6 (osd.6) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:39.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:39 smithi166 bash[9446]: cluster 2020-09-08T15:38:38.594718+0000 mon.a (mon.0) 960 : cluster [DBG] osdmap e152: 12 total, 12 up, 12 in 2020-09-08T15:38:39.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:39 smithi166 bash[9446]: cluster 2020-09-08T15:38:39.569879+0000 mon.a (mon.0) 961 : cluster [DBG] osdmap e153: 12 total, 12 up, 12 in 2020-09-08T15:38:39.883 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:39 smithi104 bash[9203]: cluster 2020-09-08T15:38:38.302574+0000 osd.6 (osd.6) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:39.883 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:39 smithi104 bash[9203]: cluster 2020-09-08T15:38:38.302705+0000 osd.6 (osd.6) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:39.884 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:39 smithi104 bash[9203]: cluster 2020-09-08T15:38:38.594718+0000 mon.a (mon.0) 960 : cluster [DBG] osdmap e152: 12 total, 12 up, 12 in 2020-09-08T15:38:39.884 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:39 smithi104 bash[9203]: cluster 2020-09-08T15:38:39.569879+0000 mon.a (mon.0) 961 : cluster [DBG] osdmap e153: 12 total, 12 up, 12 in 2020-09-08T15:38:39.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:39 smithi055 bash[9296]: cluster 2020-09-08T15:38:38.302574+0000 osd.6 (osd.6) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:39.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:39 smithi055 bash[9296]: cluster 2020-09-08T15:38:38.302705+0000 osd.6 (osd.6) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:39.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:39 smithi055 bash[9296]: cluster 2020-09-08T15:38:38.594718+0000 mon.a (mon.0) 960 : cluster [DBG] osdmap e152: 12 total, 12 up, 12 in 2020-09-08T15:38:39.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:39 smithi055 bash[9296]: cluster 2020-09-08T15:38:39.569879+0000 mon.a (mon.0) 961 : cluster [DBG] osdmap e153: 12 total, 12 up, 12 in 2020-09-08T15:38:40.048 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:38:40.157 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:40.260 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:38:40.410 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:38:40 smithi055 bash[16646]: debug 2020-09-08T15:38:40.161+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:40.599 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:40 smithi104 bash[9203]: cluster 2020-09-08T15:38:39.606559+0000 mgr.x (mgr.24118) 334 : cluster [DBG] pgmap v409: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 102 MiB/s wr, 1.62k op/s 2020-09-08T15:38:40.774 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:38:40.775 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 58 16 111773 111757 120.417 83 0.0119483 0.00830218 2020-09-08T15:38:40.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:40 smithi166 bash[9446]: cluster 2020-09-08T15:38:39.606559+0000 mgr.x (mgr.24118) 334 : cluster [DBG] pgmap v409: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 102 MiB/s wr, 1.62k op/s 2020-09-08T15:38:40.876 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:38:40.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:40 smithi055 bash[9296]: cluster 2020-09-08T15:38:39.606559+0000 mgr.x (mgr.24118) 334 : cluster [DBG] pgmap v409: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 102 MiB/s wr, 1.62k op/s 2020-09-08T15:38:40.933 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:38:40 smithi104 bash[15273]: debug 2020-09-08T15:38:40.770+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:41.578 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:38:41.578 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:38:41.730 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:38:41.772 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 59 16 113100 113084 119.782 82.9375 0.00881347 0.00834659 2020-09-08T15:38:41.836 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:38:41.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:41 smithi166 bash[9446]: cluster 2020-09-08T15:38:40.791366+0000 osd.11 (osd.11) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:41.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:41 smithi166 bash[9446]: cluster 2020-09-08T15:38:40.791451+0000 osd.11 (osd.11) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:41.865 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:38:41 smithi166 bash[18960]: debug 2020-09-08T15:38:41.732+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:41.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:41 smithi055 bash[9296]: cluster 2020-09-08T15:38:40.791366+0000 osd.11 (osd.11) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:41.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:41 smithi055 bash[9296]: cluster 2020-09-08T15:38:40.791451+0000 osd.11 (osd.11) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:41.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:41 smithi104 bash[9203]: cluster 2020-09-08T15:38:40.791366+0000 osd.11 (osd.11) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:41.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:41 smithi104 bash[9203]: cluster 2020-09-08T15:38:40.791451+0000 osd.11 (osd.11) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:41.984 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:38:42.016 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:38:42.125 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:42.157 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:38:42.213 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:38:42.394 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:38:42.436 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:38:42 smithi104 bash[17588]: debug 2020-09-08T15:38:42.018+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:42.772 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 15:38:42.778801 min lat: 0.00138961 max lat: 0.356103 avg lat: 0.00839111 2020-09-08T15:38:42.773 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:38:42.773 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 60 16 114404 114388 119.143 81.5 0.0121655 0.00839111 2020-09-08T15:38:42.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:42 smithi166 bash[9446]: cluster 2020-09-08T15:38:41.607044+0000 mgr.x (mgr.24118) 335 : cluster [DBG] pgmap v410: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 95 MiB/s wr, 1.52k op/s 2020-09-08T15:38:42.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:42 smithi166 bash[9446]: cluster 2020-09-08T15:38:41.609143+0000 mon.a (mon.0) 962 : cluster [DBG] osdmap e154: 12 total, 12 up, 12 in 2020-09-08T15:38:42.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:42 smithi055 bash[9296]: cluster 2020-09-08T15:38:41.607044+0000 mgr.x (mgr.24118) 335 : cluster [DBG] pgmap v410: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 95 MiB/s wr, 1.52k op/s 2020-09-08T15:38:42.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:42 smithi055 bash[9296]: cluster 2020-09-08T15:38:41.609143+0000 mon.a (mon.0) 962 : cluster [DBG] osdmap e154: 12 total, 12 up, 12 in 2020-09-08T15:38:42.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:42 smithi104 bash[9203]: cluster 2020-09-08T15:38:41.607044+0000 mgr.x (mgr.24118) 335 : cluster [DBG] pgmap v410: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 95 MiB/s wr, 1.52k op/s 2020-09-08T15:38:42.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:42 smithi104 bash[9203]: cluster 2020-09-08T15:38:41.609143+0000 mon.a (mon.0) 962 : cluster [DBG] osdmap e154: 12 total, 12 up, 12 in 2020-09-08T15:38:43.021 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:43.127 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:38:43.358 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:38:43 smithi104 bash[22305]: debug 2020-09-08T15:38:43.018+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:43.360 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:38:43.360 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2020-09-08T15:38:43.463 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:43.589 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:38:43.685 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:38:43 smithi104 bash[15273]: debug 2020-09-08T15:38:43.358+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:43.772 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 61 15 115646 115631 118.464 77.6875 0.00900369 0.00843958 2020-09-08T15:38:44.221 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:44.324 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:44.620 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:38:44 smithi104 bash[22305]: debug 2020-09-08T15:38:44.222+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:44.773 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 62 16 116747 116731 117.662 68.75 0.00615562 0.00846869 2020-09-08T15:38:44.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:44 smithi055 bash[9296]: cluster 2020-09-08T15:38:43.607652+0000 mgr.x (mgr.24118) 336 : cluster [DBG] pgmap v412: 17 pgs: 17 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 88 MiB/s wr, 1.41k op/s 2020-09-08T15:38:44.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:44 smithi055 bash[9296]: cluster 2020-09-08T15:38:44.568737+0000 mon.a (mon.0) 963 : cluster [DBG] osdmap e155: 12 total, 12 up, 12 in 2020-09-08T15:38:44.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:44 smithi104 bash[9203]: cluster 2020-09-08T15:38:43.607652+0000 mgr.x (mgr.24118) 336 : cluster [DBG] pgmap v412: 17 pgs: 17 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 88 MiB/s wr, 1.41k op/s 2020-09-08T15:38:44.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:44 smithi104 bash[9203]: cluster 2020-09-08T15:38:44.568737+0000 mon.a (mon.0) 963 : cluster [DBG] osdmap e155: 12 total, 12 up, 12 in 2020-09-08T15:38:44.967 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:45.079 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:45.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:44 smithi166 bash[9446]: cluster 2020-09-08T15:38:43.607652+0000 mgr.x (mgr.24118) 336 : cluster [DBG] pgmap v412: 17 pgs: 17 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 88 MiB/s wr, 1.41k op/s 2020-09-08T15:38:45.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:44 smithi166 bash[9446]: cluster 2020-09-08T15:38:44.568737+0000 mon.a (mon.0) 963 : cluster [DBG] osdmap e155: 12 total, 12 up, 12 in 2020-09-08T15:38:45.427 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:38:44 smithi104 bash[22305]: debug 2020-09-08T15:38:44.966+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:45.727 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:45.773 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 63 16 117908 117892 116.946 72.5625 0.00834091 0.00854919 2020-09-08T15:38:45.831 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:38:45.963 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:38:46.047 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:38:45 smithi104 bash[22305]: debug 2020-09-08T15:38:45.730+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:46.068 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:46.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:38:45 smithi055 bash[11956]: debug 2020-09-08T15:38:45.965+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:46.188 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:46.292 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:46.631 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:38:46 smithi055 bash[16646]: debug 2020-09-08T15:38:46.189+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:46.631 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:46 smithi055 bash[9296]: cluster 2020-09-08T15:38:45.244867+0000 osd.4 (osd.4) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:46.767 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:46.772 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 64 16 119139 119123 116.32 76.9375 0.0189174 0.00859483 2020-09-08T15:38:46.872 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:46.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:46 smithi055 bash[9296]: cluster 2020-09-08T15:38:45.244984+0000 osd.4 (osd.4) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:46.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:46 smithi055 bash[9296]: cluster 2020-09-08T15:38:45.608218+0000 mgr.x (mgr.24118) 337 : cluster [DBG] pgmap v414: 17 pgs: 17 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 67 MiB/s wr, 1.07k op/s 2020-09-08T15:38:46.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:46 smithi055 bash[9296]: cluster 2020-09-08T15:38:45.637141+0000 mon.a (mon.0) 964 : cluster [DBG] osdmap e156: 12 total, 12 up, 12 in 2020-09-08T15:38:46.917 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:46 smithi104 bash[9203]: cluster 2020-09-08T15:38:45.244867+0000 osd.4 (osd.4) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:46.917 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:46 smithi104 bash[9203]: cluster 2020-09-08T15:38:45.244984+0000 osd.4 (osd.4) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:46.918 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:46 smithi104 bash[9203]: cluster 2020-09-08T15:38:45.608218+0000 mgr.x (mgr.24118) 337 : cluster [DBG] pgmap v414: 17 pgs: 17 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 67 MiB/s wr, 1.07k op/s 2020-09-08T15:38:46.918 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:46 smithi104 bash[9203]: cluster 2020-09-08T15:38:45.637141+0000 mon.a (mon.0) 964 : cluster [DBG] osdmap e156: 12 total, 12 up, 12 in 2020-09-08T15:38:46.918 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:38:46 smithi104 bash[22305]: debug 2020-09-08T15:38:46.766+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:46.986 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:47.098 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:47.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:46 smithi166 bash[9446]: cluster 2020-09-08T15:38:45.244867+0000 osd.4 (osd.4) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:47.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:46 smithi166 bash[9446]: cluster 2020-09-08T15:38:45.244984+0000 osd.4 (osd.4) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:47.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:46 smithi166 bash[9446]: cluster 2020-09-08T15:38:45.608218+0000 mgr.x (mgr.24118) 337 : cluster [DBG] pgmap v414: 17 pgs: 17 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 67 MiB/s wr, 1.07k op/s 2020-09-08T15:38:47.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:46 smithi166 bash[9446]: cluster 2020-09-08T15:38:45.637141+0000 mon.a (mon.0) 964 : cluster [DBG] osdmap e156: 12 total, 12 up, 12 in 2020-09-08T15:38:47.192 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:47.297 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:47.305 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:38:47 smithi055 bash[19003]: debug 2020-09-08T15:38:47.189+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:47.305 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:38:46 smithi055 bash[16646]: debug 2020-09-08T15:38:46.989+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:47.414 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:47.517 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:38:47.638 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:38:47 smithi055 bash[14301]: debug 2020-09-08T15:38:47.417+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:47.663 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:38:47.769 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:47.772 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 65 16 120419 120403 115.762 80 0.0133779 0.00863667 2020-09-08T15:38:47.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:47 smithi166 bash[9446]: cluster 2020-09-08T15:38:47.608718+0000 mgr.x (mgr.24118) 338 : cluster [DBG] pgmap v416: 17 pgs: 17 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s wr, 1.58k op/s 2020-09-08T15:38:47.865 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:38:47 smithi166 bash[14254]: debug 2020-09-08T15:38:47.667+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:47.886 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:47.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:47 smithi055 bash[9296]: cluster 2020-09-08T15:38:47.608718+0000 mgr.x (mgr.24118) 338 : cluster [DBG] pgmap v416: 17 pgs: 17 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s wr, 1.58k op/s 2020-09-08T15:38:47.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:47 smithi104 bash[9203]: cluster 2020-09-08T15:38:47.608718+0000 mgr.x (mgr.24118) 338 : cluster [DBG] pgmap v416: 17 pgs: 17 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s wr, 1.58k op/s 2020-09-08T15:38:47.994 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:38:48.106 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:38:48.209 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:38:48.218 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:38:48 smithi166 bash[11928]: debug 2020-09-08T15:38:48.107+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:48.218 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:38:47 smithi166 bash[16593]: debug 2020-09-08T15:38:47.887+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:48.327 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:38:48.434 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:48.615 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:38:48 smithi166 bash[14254]: debug 2020-09-08T15:38:48.323+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:48.782 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 66 16 121414 121398 114.95 62.1875 0.0103131 0.00869719 2020-09-08T15:38:49.194 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:49.296 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:38:49.422 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:38:49.523 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:49.615 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:38:49 smithi166 bash[14254]: debug 2020-09-08T15:38:49.419+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:49.639 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:49.677 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:38:49 smithi104 bash[22305]: debug 2020-09-08T15:38:49.178+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:49.743 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:38:49.778 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 67 16 122532 122516 114.277 69.875 0.00582067 0.0087487 2020-09-08T15:38:49.856 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:38:49.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:38:49 smithi055 bash[16646]: debug 2020-09-08T15:38:49.641+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:49.960 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:50.114 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:38:49 smithi166 bash[14254]: debug 2020-09-08T15:38:49.859+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:50.568 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:38:50.586 INFO:teuthology.orchestra.run.smithi104.stderr:reweighted osd.2 to 1 (10000) 2020-09-08T15:38:50.672 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:38:50.773 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 68 16 123759 123743 113.724 76.6875 0.0113608 0.00879119 2020-09-08T15:38:50.786 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:38:50.865 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:38:50 smithi166 bash[11928]: debug 2020-09-08T15:38:50.783+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:50.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:50 smithi166 bash[9446]: cluster 2020-09-08T15:38:49.572020+0000 mon.a (mon.0) 965 : cluster [DBG] osdmap e157: 12 total, 12 up, 12 in 2020-09-08T15:38:50.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:50 smithi166 bash[9446]: cluster 2020-09-08T15:38:49.609214+0000 mgr.x (mgr.24118) 339 : cluster [DBG] pgmap v418: 17 pgs: 17 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 MiB/s wr, 1.02k op/s 2020-09-08T15:38:50.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:50 smithi166 bash[9446]: audit 2020-09-08T15:38:49.710226+0000 mon.a (mon.0) 966 : audit [INF] from='client.34796 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2020-09-08T15:38:50.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:50 smithi166 bash[9446]: audit 2020-09-08T15:38:49.710295+0000 mon.c (mon.2) 36 : audit [INF] from='client.? v1:172.21.15.104:0/1002167671' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2020-09-08T15:38:50.899 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:38:50.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:50 smithi055 bash[9296]: cluster 2020-09-08T15:38:49.572020+0000 mon.a (mon.0) 965 : cluster [DBG] osdmap e157: 12 total, 12 up, 12 in 2020-09-08T15:38:50.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:50 smithi055 bash[9296]: cluster 2020-09-08T15:38:49.609214+0000 mgr.x (mgr.24118) 339 : cluster [DBG] pgmap v418: 17 pgs: 17 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 MiB/s wr, 1.02k op/s 2020-09-08T15:38:50.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:50 smithi055 bash[9296]: audit 2020-09-08T15:38:49.710226+0000 mon.a (mon.0) 966 : audit [INF] from='client.34796 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2020-09-08T15:38:50.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:50 smithi055 bash[9296]: audit 2020-09-08T15:38:49.710295+0000 mon.c (mon.2) 36 : audit [INF] from='client.? v1:172.21.15.104:0/1002167671' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2020-09-08T15:38:51.037 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:38:51.133 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:50 smithi104 bash[9203]: cluster 2020-09-08T15:38:49.572020+0000 mon.a (mon.0) 965 : cluster [DBG] osdmap e157: 12 total, 12 up, 12 in 2020-09-08T15:38:51.133 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:50 smithi104 bash[9203]: cluster 2020-09-08T15:38:49.609214+0000 mgr.x (mgr.24118) 339 : cluster [DBG] pgmap v418: 17 pgs: 17 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 MiB/s wr, 1.02k op/s 2020-09-08T15:38:51.134 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:50 smithi104 bash[9203]: audit 2020-09-08T15:38:49.710226+0000 mon.a (mon.0) 966 : audit [INF] from='client.34796 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2020-09-08T15:38:51.134 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:50 smithi104 bash[9203]: audit 2020-09-08T15:38:49.710295+0000 mon.c (mon.2) 36 : audit [INF] from='client.? v1:172.21.15.104:0/1002167671' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2020-09-08T15:38:51.134 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:38:50 smithi104 bash[22305]: debug 2020-09-08T15:38:50.570+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:51.138 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:51.256 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:51.361 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:38:51.365 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:38:51 smithi166 bash[14254]: debug 2020-09-08T15:38:51.035+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:51.410 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:38:51 smithi055 bash[16646]: debug 2020-09-08T15:38:51.257+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:51.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:51 smithi104 bash[9203]: audit 2020-09-08T15:38:50.580449+0000 mon.a (mon.0) 967 : audit [INF] from='client.34796 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2020-09-08T15:38:51.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:51 smithi104 bash[9203]: cluster 2020-09-08T15:38:50.580519+0000 mon.a (mon.0) 968 : cluster [DBG] osdmap e158: 12 total, 12 up, 12 in 2020-09-08T15:38:51.774 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 69 16 124845 124829 113.06 67.875 0.0051637 0.00884299 2020-09-08T15:38:51.827 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:38:51.864 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:51 smithi166 bash[9446]: audit 2020-09-08T15:38:50.580449+0000 mon.a (mon.0) 967 : audit [INF] from='client.34796 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2020-09-08T15:38:51.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:51 smithi166 bash[9446]: cluster 2020-09-08T15:38:50.580519+0000 mon.a (mon.0) 968 : cluster [DBG] osdmap e158: 12 total, 12 up, 12 in 2020-09-08T15:38:51.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:51 smithi055 bash[9296]: audit 2020-09-08T15:38:50.580449+0000 mon.a (mon.0) 967 : audit [INF] from='client.34796 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2020-09-08T15:38:51.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:51 smithi055 bash[9296]: cluster 2020-09-08T15:38:50.580519+0000 mon.a (mon.0) 968 : cluster [DBG] osdmap e158: 12 total, 12 up, 12 in 2020-09-08T15:38:51.931 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:52.028 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:52.115 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:38:51 smithi104 bash[19922]: debug 2020-09-08T15:38:51.826+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:52.134 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:52.242 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:38:52 smithi055 bash[16646]: debug 2020-09-08T15:38:52.029+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:52.243 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:38:52 smithi055 bash[14301]: debug 2020-09-08T15:38:52.241+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:52.244 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:52.299 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd reweight 3 1 2020-09-08T15:38:52.350 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:38:52.459 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:38:52.481 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:38:52.586 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:52.626 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:52 smithi166 bash[9446]: cluster 2020-09-08T15:38:51.609816+0000 mgr.x (mgr.24118) 340 : cluster [DBG] pgmap v420: 17 pgs: 17 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s wr, 1.19k op/s 2020-09-08T15:38:52.627 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:38:52 smithi166 bash[14254]: debug 2020-09-08T15:38:52.483+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:52.689 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:52.773 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 70 15 125950 125935 112.432 69.125 0.00749731 0.00889239 2020-09-08T15:38:52.793 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:38:52.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:52 smithi055 bash[9296]: cluster 2020-09-08T15:38:51.609816+0000 mgr.x (mgr.24118) 340 : cluster [DBG] pgmap v420: 17 pgs: 17 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s wr, 1.19k op/s 2020-09-08T15:38:52.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:38:52 smithi055 bash[14301]: debug 2020-09-08T15:38:52.693+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:52.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:52 smithi104 bash[9203]: cluster 2020-09-08T15:38:51.609816+0000 mgr.x (mgr.24118) 340 : cluster [DBG] pgmap v420: 17 pgs: 17 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s wr, 1.19k op/s 2020-09-08T15:38:53.556 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:38:53.591 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:38:53.662 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:53.773 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 71 16 127170 127154 111.921 76.1875 0.0101232 0.00893302 2020-09-08T15:38:53.795 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:38:53.886 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:53 smithi166 bash[9446]: audit 2020-09-08T15:38:52.662717+0000 mon.c (mon.2) 37 : audit [INF] from='client.? v1:172.21.15.104:0/2400229128' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:38:53.887 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:53 smithi166 bash[9446]: audit 2020-09-08T15:38:52.662858+0000 mon.a (mon.0) 969 : audit [INF] from='client.34802 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:38:53.887 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:38:53 smithi166 bash[16593]: debug 2020-09-08T15:38:53.799+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:53.898 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:53.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:53 smithi055 bash[9296]: audit 2020-09-08T15:38:52.662717+0000 mon.c (mon.2) 37 : audit [INF] from='client.? v1:172.21.15.104:0/2400229128' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:38:53.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:53 smithi055 bash[9296]: audit 2020-09-08T15:38:52.662858+0000 mon.a (mon.0) 969 : audit [INF] from='client.34802 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:38:53.927 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:38:53 smithi104 bash[17588]: debug 2020-09-08T15:38:53.550+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:53.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:53 smithi104 bash[9203]: audit 2020-09-08T15:38:52.662717+0000 mon.c (mon.2) 37 : audit [INF] from='client.? v1:172.21.15.104:0/2400229128' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:38:53.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:53 smithi104 bash[9203]: audit 2020-09-08T15:38:52.662858+0000 mon.a (mon.0) 969 : audit [INF] from='client.34802 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:38:54.042 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:54.123 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:38:54.152 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:54.156 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:38:54 smithi055 bash[14301]: debug 2020-09-08T15:38:54.041+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:54.163 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:38:54.198 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:38:54.225 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:38:54.257 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:38:54.266 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:38:54.292 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:38:54.330 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:38:54.362 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:38:54.371 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:38:54.405 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:38:54.448 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:38:54.482 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:38:54.490 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:38:54 smithi055 bash[16646]: debug 2020-09-08T15:38:54.269+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:54.491 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:38:54 smithi055 bash[11956]: debug 2020-09-08T15:38:54.485+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:54.491 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:38:54.514 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:38:54.516 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:38:54.516 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:38:54.516 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:38:54.516 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:38:54.517 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:38:54.517 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:38:54.517 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:38:54.517 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:38:54.517 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:38:54.518 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:38:54.518 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:38:54.518 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:38:54.594 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:38:54.695 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:38:54.777 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 72 16 128307 128291 111.354 71.0625 0.0118904 0.00897819 2020-09-08T15:38:54.804 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:54.819 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:38:54.820 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T15:38:54.820 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2020-09-08T15:38:54.821 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2020-09-08T15:38:54.821 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:38:54.821 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:54 smithi055 bash[9296]: cluster 2020-09-08T15:38:53.486841+0000 osd.9 (osd.9) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:54.821 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:54 smithi055 bash[9296]: cluster 2020-09-08T15:38:53.486959+0000 osd.9 (osd.9) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:54.822 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:54 smithi055 bash[9296]: cluster 2020-09-08T15:38:53.588254+0000 mon.a (mon.0) 970 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:38:54.822 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:54 smithi055 bash[9296]: audit 2020-09-08T15:38:53.592525+0000 mon.a (mon.0) 971 : audit [INF] from='client.34802 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:38:54.822 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:54 smithi055 bash[9296]: cluster 2020-09-08T15:38:53.592815+0000 mon.a (mon.0) 972 : cluster [DBG] osdmap e159: 12 total, 12 up, 12 in 2020-09-08T15:38:54.823 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:54 smithi055 bash[9296]: cluster 2020-09-08T15:38:53.610457+0000 mgr.x (mgr.24118) 341 : cluster [DBG] pgmap v422: 17 pgs: 17 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T15:38:54.823 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:54 smithi055 bash[9296]: audit 2020-09-08T15:38:54.090190+0000 mon.c (mon.2) 38 : audit [DBG] from='client.? v1:172.21.15.104:0/2254023303' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:38:54.865 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:38:54 smithi166 bash[11928]: debug 2020-09-08T15:38:54.699+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:54.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:54 smithi166 bash[9446]: cluster 2020-09-08T15:38:53.486841+0000 osd.9 (osd.9) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:54.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:54 smithi166 bash[9446]: cluster 2020-09-08T15:38:53.486959+0000 osd.9 (osd.9) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:54.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:54 smithi166 bash[9446]: cluster 2020-09-08T15:38:53.588254+0000 mon.a (mon.0) 970 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:38:54.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:54 smithi166 bash[9446]: audit 2020-09-08T15:38:53.592525+0000 mon.a (mon.0) 971 : audit [INF] from='client.34802 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:38:54.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:54 smithi166 bash[9446]: cluster 2020-09-08T15:38:53.592815+0000 mon.a (mon.0) 972 : cluster [DBG] osdmap e159: 12 total, 12 up, 12 in 2020-09-08T15:38:54.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:54 smithi166 bash[9446]: cluster 2020-09-08T15:38:53.610457+0000 mgr.x (mgr.24118) 341 : cluster [DBG] pgmap v422: 17 pgs: 17 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T15:38:54.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:54 smithi166 bash[9446]: audit 2020-09-08T15:38:54.090190+0000 mon.c (mon.2) 38 : audit [DBG] from='client.? v1:172.21.15.104:0/2254023303' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:38:54.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:54 smithi104 bash[9203]: cluster 2020-09-08T15:38:53.486841+0000 osd.9 (osd.9) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:54.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:54 smithi104 bash[9203]: cluster 2020-09-08T15:38:53.486959+0000 osd.9 (osd.9) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:54.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:54 smithi104 bash[9203]: cluster 2020-09-08T15:38:53.588254+0000 mon.a (mon.0) 970 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:38:54.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:54 smithi104 bash[9203]: audit 2020-09-08T15:38:53.592525+0000 mon.a (mon.0) 971 : audit [INF] from='client.34802 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:38:54.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:54 smithi104 bash[9203]: cluster 2020-09-08T15:38:53.592815+0000 mon.a (mon.0) 972 : cluster [DBG] osdmap e159: 12 total, 12 up, 12 in 2020-09-08T15:38:54.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:54 smithi104 bash[9203]: cluster 2020-09-08T15:38:53.610457+0000 mgr.x (mgr.24118) 341 : cluster [DBG] pgmap v422: 17 pgs: 17 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T15:38:54.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:54 smithi104 bash[9203]: audit 2020-09-08T15:38:54.090190+0000 mon.c (mon.2) 38 : audit [DBG] from='client.? v1:172.21.15.104:0/2254023303' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:38:54.929 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:55.033 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:55.156 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:38:54 smithi055 bash[14301]: debug 2020-09-08T15:38:54.929+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:55.157 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:55.258 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:38:55.410 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:38:55 smithi055 bash[14301]: debug 2020-09-08T15:38:55.153+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:55.774 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 73 16 129532 129516 110.877 76.5625 0.00500668 0.00901684 2020-09-08T15:38:55.911 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:38:55.911 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:38:55 smithi104 bash[17588]: debug 2020-09-08T15:38:55.910+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:56.024 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:38:56.654 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:38:56.765 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:38:56.778 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 74 16 130777 130761 110.43 77.8125 0.0216105 0.00905388 2020-09-08T15:38:56.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:56 smithi166 bash[9446]: cluster 2020-09-08T15:38:55.610897+0000 mgr.x (mgr.24118) 342 : cluster [DBG] pgmap v423: 17 pgs: 17 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s wr, 931 op/s 2020-09-08T15:38:56.871 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:56 smithi104 bash[9203]: cluster 2020-09-08T15:38:55.610897+0000 mgr.x (mgr.24118) 342 : cluster [DBG] pgmap v423: 17 pgs: 17 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s wr, 931 op/s 2020-09-08T15:38:56.871 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:38:56 smithi104 bash[15273]: debug 2020-09-08T15:38:56.654+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:56.885 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2020-09-08T15:38:56.899 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:38:56.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:56 smithi055 bash[9296]: cluster 2020-09-08T15:38:55.610897+0000 mgr.x (mgr.24118) 342 : cluster [DBG] pgmap v423: 17 pgs: 17 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s wr, 931 op/s 2020-09-08T15:38:57.004 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:57.058 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:38:57.127 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:38:57.232 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:38:57.365 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:38:56 smithi166 bash[11928]: debug 2020-09-08T15:38:56.903+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:57.410 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:38:57 smithi055 bash[19003]: debug 2020-09-08T15:38:57.129+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:57.774 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 75 16 131940 131924 109.927 72.6875 0.0104368 0.0090952 2020-09-08T15:38:57.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:57 smithi055 bash[9296]: cluster 2020-09-08T15:38:56.074002+0000 osd.2 (osd.2) 18 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:57.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:57 smithi055 bash[9296]: cluster 2020-09-08T15:38:56.074088+0000 osd.2 (osd.2) 19 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:57.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:57 smithi055 bash[9296]: cluster 2020-09-08T15:38:56.622899+0000 mon.a (mon.0) 973 : cluster [DBG] osdmap e160: 12 total, 12 up, 12 in 2020-09-08T15:38:57.925 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:38:57.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:57 smithi104 bash[9203]: cluster 2020-09-08T15:38:56.074002+0000 osd.2 (osd.2) 18 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:57.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:57 smithi104 bash[9203]: cluster 2020-09-08T15:38:56.074088+0000 osd.2 (osd.2) 19 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:57.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:57 smithi104 bash[9203]: cluster 2020-09-08T15:38:56.622899+0000 mon.a (mon.0) 973 : cluster [DBG] osdmap e160: 12 total, 12 up, 12 in 2020-09-08T15:38:58.082 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:38:58.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:57 smithi166 bash[9446]: cluster 2020-09-08T15:38:56.074002+0000 osd.2 (osd.2) 18 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:38:58.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:57 smithi166 bash[9446]: cluster 2020-09-08T15:38:56.074088+0000 osd.2 (osd.2) 19 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:38:58.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:57 smithi166 bash[9446]: cluster 2020-09-08T15:38:56.622899+0000 mon.a (mon.0) 973 : cluster [DBG] osdmap e160: 12 total, 12 up, 12 in 2020-09-08T15:38:58.245 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:38:58.248 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:38:58 smithi104 bash[15273]: debug 2020-09-08T15:38:58.246+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:58.350 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:58.464 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:38:58.577 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:38:58.625 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:38:58 smithi055 bash[14301]: debug 2020-09-08T15:38:58.465+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:58.707 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:38:58.774 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 76 16 133168 133152 109.49 76.75 0.0119636 0.00913141 2020-09-08T15:38:58.863 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:38:58.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:38:58 smithi055 bash[9296]: cluster 2020-09-08T15:38:57.611491+0000 mgr.x (mgr.24118) 343 : cluster [DBG] pgmap v425: 17 pgs: 17 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s wr, 1.37k op/s 2020-09-08T15:38:58.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:38:58 smithi104 bash[9203]: cluster 2020-09-08T15:38:57.611491+0000 mgr.x (mgr.24118) 343 : cluster [DBG] pgmap v425: 17 pgs: 17 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s wr, 1.37k op/s 2020-09-08T15:38:59.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:38:58 smithi166 bash[9446]: cluster 2020-09-08T15:38:57.611491+0000 mgr.x (mgr.24118) 343 : cluster [DBG] pgmap v425: 17 pgs: 17 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s wr, 1.37k op/s 2020-09-08T15:38:59.128 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:38:59.232 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:38:59.363 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:38:59.430 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:38:59 smithi104 bash[15273]: debug 2020-09-08T15:38:59.130+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:59.469 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:38:59.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:38:59 smithi166 bash[18960]: debug 2020-09-08T15:38:59.367+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:38:59.774 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 77 16 134327 134311 109.009 72.4375 0.0169578 0.00917192 2020-09-08T15:39:00.303 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:39:00.406 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:39:00.573 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:00 smithi104 bash[9203]: cluster 2020-09-08T15:38:59.612063+0000 mgr.x (mgr.24118) 344 : cluster [DBG] pgmap v426: 17 pgs: 17 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s wr, 1.31k op/s 2020-09-08T15:39:00.573 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:39:00 smithi104 bash[19922]: debug 2020-09-08T15:39:00.302+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:00.774 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 78 16 135523 135507 108.57 74.75 0.00549831 0.00920846 2020-09-08T15:39:00.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:00 smithi166 bash[9446]: cluster 2020-09-08T15:38:59.612063+0000 mgr.x (mgr.24118) 344 : cluster [DBG] pgmap v426: 17 pgs: 17 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s wr, 1.31k op/s 2020-09-08T15:39:00.895 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:39:00.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:00 smithi055 bash[9296]: cluster 2020-09-08T15:38:59.612063+0000 mgr.x (mgr.24118) 344 : cluster [DBG] pgmap v426: 17 pgs: 17 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s wr, 1.31k op/s 2020-09-08T15:39:00.998 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:01.122 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:01.177 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:39:00 smithi104 bash[17588]: debug 2020-09-08T15:39:00.894+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:01.225 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:01.333 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:39:01 smithi055 bash[14301]: debug 2020-09-08T15:39:01.125+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:01.335 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:01.440 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:01.542 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:01.645 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:39:01.656 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:39:01 smithi055 bash[14301]: debug 2020-09-08T15:39:01.545+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:01.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:39:01 smithi055 bash[19003]: debug 2020-09-08T15:39:01.337+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:01.774 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 79 16 136487 136471 107.958 60.25 0.0184693 0.0092605 2020-09-08T15:39:01.776 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:39:01.883 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:01.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:01 smithi104 bash[9203]: cluster 2020-09-08T15:39:01.612576+0000 mgr.x (mgr.24118) 345 : cluster [DBG] pgmap v427: 17 pgs: 17 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s wr, 743 op/s 2020-09-08T15:39:01.994 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:01 smithi055 bash[9296]: cluster 2020-09-08T15:39:01.612576+0000 mgr.x (mgr.24118) 345 : cluster [DBG] pgmap v427: 17 pgs: 17 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s wr, 743 op/s 2020-09-08T15:39:01.995 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:39:02 smithi055 bash[14301]: debug 2020-09-08T15:39:01.997+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:01.995 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:02.099 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:02.115 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:39:01 smithi166 bash[14254]: debug 2020-09-08T15:39:01.779+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:02.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:01 smithi166 bash[9446]: cluster 2020-09-08T15:39:01.612576+0000 mgr.x (mgr.24118) 345 : cluster [DBG] pgmap v427: 17 pgs: 17 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s wr, 743 op/s 2020-09-08T15:39:02.280 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:02.388 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:02.496 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:02.600 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:02.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:39:02 smithi055 bash[11956]: debug 2020-09-08T15:39:02.497+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:02.677 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:39:02 smithi104 bash[22305]: debug 2020-09-08T15:39:02.278+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:02.704 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:02.774 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 15:39:02.780603 min lat: 0.00138961 max lat: 0.356103 avg lat: 0.00930325 2020-09-08T15:39:02.775 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:39:02.775 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 80 16 137587 137571 107.468 68.75 0.015126 0.00930325 2020-09-08T15:39:02.811 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:02.933 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:03.039 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:03.047 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:39:02 smithi055 bash[11956]: debug 2020-09-08T15:39:02.705+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:03.048 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:39:02 smithi055 bash[16646]: debug 2020-09-08T15:39:02.933+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:03.150 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:03.255 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:39:03.398 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:39:03.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:39:03 smithi055 bash[11956]: debug 2020-09-08T15:39:03.145+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:03.501 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:03.610 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:03.615 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:39:03 smithi166 bash[14254]: debug 2020-09-08T15:39:03.395+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:03.717 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:03.774 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 81 16 138743 138727 107.033 72.25 0.0084439 0.00934131 2020-09-08T15:39:03.825 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:03 smithi055 bash[9296]: cluster 2020-09-08T15:39:03.614626+0000 mgr.x (mgr.24118) 346 : cluster [DBG] pgmap v428: 17 pgs: 17 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s wr, 1.17k op/s 2020-09-08T15:39:03.825 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:03 smithi055 bash[9296]: cluster 2020-09-08T15:39:03.676822+0000 mon.a (mon.0) 974 : cluster [DBG] osdmap e161: 12 total, 12 up, 12 in 2020-09-08T15:39:03.825 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:39:03 smithi055 bash[14301]: debug 2020-09-08T15:39:03.609+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:03.826 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:03.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:03 smithi104 bash[9203]: cluster 2020-09-08T15:39:03.614626+0000 mgr.x (mgr.24118) 346 : cluster [DBG] pgmap v428: 17 pgs: 17 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s wr, 1.17k op/s 2020-09-08T15:39:03.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:03 smithi104 bash[9203]: cluster 2020-09-08T15:39:03.676822+0000 mon.a (mon.0) 974 : cluster [DBG] osdmap e161: 12 total, 12 up, 12 in 2020-09-08T15:39:03.938 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:04.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:03 smithi166 bash[9446]: cluster 2020-09-08T15:39:03.614626+0000 mgr.x (mgr.24118) 346 : cluster [DBG] pgmap v428: 17 pgs: 17 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s wr, 1.17k op/s 2020-09-08T15:39:04.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:03 smithi166 bash[9446]: cluster 2020-09-08T15:39:03.676822+0000 mon.a (mon.0) 974 : cluster [DBG] osdmap e161: 12 total, 12 up, 12 in 2020-09-08T15:39:04.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:39:03 smithi055 bash[19003]: debug 2020-09-08T15:39:03.829+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:04.195 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:04.197 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:39:04 smithi104 bash[22305]: debug 2020-09-08T15:39:04.190+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:04.298 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:39:04.698 INFO:teuthology.orchestra.run.smithi104.stderr:reweighted osd.3 to 1 (10000) 2020-09-08T15:39:04.774 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 82 16 139844 139828 106.567 68.8125 0.0193501 0.00938134 2020-09-08T15:39:05.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:04 smithi166 bash[9446]: cluster 2020-09-08T15:39:03.618000+0000 osd.7 (osd.7) 25 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:05.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:04 smithi166 bash[9446]: cluster 2020-09-08T15:39:03.618072+0000 osd.7 (osd.7) 26 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:05.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:04 smithi166 bash[9446]: audit 2020-09-08T15:39:04.148166+0000 mon.a (mon.0) 975 : audit [INF] from='client.? v1:172.21.15.104:0/431668465' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2020-09-08T15:39:05.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:04 smithi055 bash[9296]: cluster 2020-09-08T15:39:03.618000+0000 osd.7 (osd.7) 25 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:05.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:04 smithi055 bash[9296]: cluster 2020-09-08T15:39:03.618072+0000 osd.7 (osd.7) 26 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:05.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:04 smithi055 bash[9296]: audit 2020-09-08T15:39:04.148166+0000 mon.a (mon.0) 975 : audit [INF] from='client.? v1:172.21.15.104:0/431668465' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2020-09-08T15:39:05.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:04 smithi104 bash[9203]: cluster 2020-09-08T15:39:03.618000+0000 osd.7 (osd.7) 25 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:05.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:04 smithi104 bash[9203]: cluster 2020-09-08T15:39:03.618072+0000 osd.7 (osd.7) 26 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:05.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:04 smithi104 bash[9203]: audit 2020-09-08T15:39:04.148166+0000 mon.a (mon.0) 975 : audit [INF] from='client.? v1:172.21.15.104:0/431668465' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2020-09-08T15:39:05.217 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:39:05.322 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:05.448 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:05.552 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:39:05.562 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:39:05 smithi104 bash[15273]: debug 2020-09-08T15:39:05.218+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:05.615 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:39:05 smithi166 bash[11928]: debug 2020-09-08T15:39:05.451+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:05.778 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 83 16 140870 140854 106.055 64.125 0.00858963 0.0094269 2020-09-08T15:39:05.828 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:05 smithi104 bash[9203]: audit 2020-09-08T15:39:04.691966+0000 mon.a (mon.0) 976 : audit [INF] from='client.? v1:172.21.15.104:0/431668465' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2020-09-08T15:39:05.829 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:05 smithi104 bash[9203]: cluster 2020-09-08T15:39:04.692019+0000 mon.a (mon.0) 977 : cluster [DBG] osdmap e162: 12 total, 12 up, 12 in 2020-09-08T15:39:05.829 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:05 smithi104 bash[9203]: cluster 2020-09-08T15:39:05.615100+0000 mgr.x (mgr.24118) 347 : cluster [DBG] pgmap v431: 17 pgs: 17 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s wr, 964 op/s 2020-09-08T15:39:05.829 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:39:05.941 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:06.057 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:05 smithi055 bash[9296]: audit 2020-09-08T15:39:04.691966+0000 mon.a (mon.0) 976 : audit [INF] from='client.? v1:172.21.15.104:0/431668465' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2020-09-08T15:39:06.058 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:05 smithi055 bash[9296]: cluster 2020-09-08T15:39:04.692019+0000 mon.a (mon.0) 977 : cluster [DBG] osdmap e162: 12 total, 12 up, 12 in 2020-09-08T15:39:06.058 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:05 smithi055 bash[9296]: cluster 2020-09-08T15:39:05.615100+0000 mgr.x (mgr.24118) 347 : cluster [DBG] pgmap v431: 17 pgs: 17 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s wr, 964 op/s 2020-09-08T15:39:06.058 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:06.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:05 smithi166 bash[9446]: audit 2020-09-08T15:39:04.691966+0000 mon.a (mon.0) 976 : audit [INF] from='client.? v1:172.21.15.104:0/431668465' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2020-09-08T15:39:06.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:05 smithi166 bash[9446]: cluster 2020-09-08T15:39:04.692019+0000 mon.a (mon.0) 977 : cluster [DBG] osdmap e162: 12 total, 12 up, 12 in 2020-09-08T15:39:06.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:05 smithi166 bash[9446]: cluster 2020-09-08T15:39:05.615100+0000 mgr.x (mgr.24118) 347 : cluster [DBG] pgmap v431: 17 pgs: 17 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s wr, 964 op/s 2020-09-08T15:39:06.162 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:39:06.172 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:39:05 smithi104 bash[19922]: debug 2020-09-08T15:39:05.830+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:06.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:39:06 smithi055 bash[16646]: debug 2020-09-08T15:39:06.061+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:06.719 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:39:06.775 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 84 16 142018 142002 105.647 71.75 0.00914448 0.0094634 2020-09-08T15:39:06.822 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:06.932 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:07.037 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:07.048 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:39:06 smithi055 bash[11956]: debug 2020-09-08T15:39:06.937+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:07.048 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:06 smithi055 bash[9296]: cluster 2020-09-08T15:39:05.197506+0000 osd.0 (osd.0) 19 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:07.049 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:06 smithi055 bash[9296]: cluster 2020-09-08T15:39:05.197608+0000 osd.0 (osd.0) 20 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:07.049 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:06 smithi055 bash[9296]: cluster 2020-09-08T15:39:05.708380+0000 mon.a (mon.0) 978 : cluster [DBG] osdmap e163: 12 total, 12 up, 12 in 2020-09-08T15:39:07.080 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:06 smithi104 bash[9203]: cluster 2020-09-08T15:39:05.197506+0000 osd.0 (osd.0) 19 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:07.081 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:06 smithi104 bash[9203]: cluster 2020-09-08T15:39:05.197608+0000 osd.0 (osd.0) 20 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:07.082 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:06 smithi104 bash[9203]: cluster 2020-09-08T15:39:05.708380+0000 mon.a (mon.0) 978 : cluster [DBG] osdmap e163: 12 total, 12 up, 12 in 2020-09-08T15:39:07.082 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:39:06 smithi104 bash[15273]: debug 2020-09-08T15:39:06.718+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:07.083 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd reweight 4 1 2020-09-08T15:39:07.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:06 smithi166 bash[9446]: cluster 2020-09-08T15:39:05.197506+0000 osd.0 (osd.0) 19 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:07.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:06 smithi166 bash[9446]: cluster 2020-09-08T15:39:05.197608+0000 osd.0 (osd.0) 20 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:07.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:06 smithi166 bash[9446]: cluster 2020-09-08T15:39:05.708380+0000 mon.a (mon.0) 978 : cluster [DBG] osdmap e163: 12 total, 12 up, 12 in 2020-09-08T15:39:07.155 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:07.246 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:39:07.258 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:39:07.413 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:39:07 smithi055 bash[19003]: debug 2020-09-08T15:39:07.153+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:07.774 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 85 16 143145 143129 105.232 70.4375 0.0071542 0.00950087 2020-09-08T15:39:08.020 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:07 smithi104 bash[9203]: cluster 2020-09-08T15:39:07.615633+0000 mgr.x (mgr.24118) 348 : cluster [DBG] pgmap v433: 17 pgs: 17 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s wr, 1.58k op/s 2020-09-08T15:39:08.023 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:39:08.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:07 smithi166 bash[9446]: cluster 2020-09-08T15:39:07.615633+0000 mgr.x (mgr.24118) 348 : cluster [DBG] pgmap v433: 17 pgs: 17 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s wr, 1.58k op/s 2020-09-08T15:39:08.130 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:08.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:07 smithi055 bash[9296]: cluster 2020-09-08T15:39:07.615633+0000 mgr.x (mgr.24118) 348 : cluster [DBG] pgmap v433: 17 pgs: 17 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s wr, 1.58k op/s 2020-09-08T15:39:08.420 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:39:08 smithi104 bash[19922]: debug 2020-09-08T15:39:08.022+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:08.420 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:39:08 smithi104 bash[22305]: debug 2020-09-08T15:39:08.422+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:08.421 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:08.525 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:08.662 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:08.771 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:08.775 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 86 16 143936 143920 104.584 49.4375 0.00876886 0.00953779 2020-09-08T15:39:08.865 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:39:08 smithi166 bash[11928]: debug 2020-09-08T15:39:08.663+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:09.311 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:09.382 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:39:09.383 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2020-09-08T15:39:09.383 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2020-09-08T15:39:09.384 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T15:39:09.384 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.384 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:5002 2.e 2:79109904:::benchmark_data_smithi166_20551_object5001:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e133)", 2020-09-08T15:39:09.385 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:44.074239+0000", 2020-09-08T15:39:09.385 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 85.309047528999997, 2020-09-08T15:39:09.385 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.080221955999999997, 2020-09-08T15:39:09.386 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:39:09.386 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:39:09.387 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:39:09.387 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:39:09.388 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:39:09.388 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 5002 2020-09-08T15:39:09.389 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.389 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:39:09.389 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.390 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:39:09.390 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:44.074239+0000", 2020-09-08T15:39:09.390 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:39:09.391 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.391 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.391 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:39:09.392 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:44.074239+0000", 2020-09-08T15:39:09.392 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999949 2020-09-08T15:39:09.392 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.393 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.393 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:39:09.393 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:44.074188+0000", 2020-09-08T15:39:09.398 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.074203878000000001 2020-09-08T15:39:09.400 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.400 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.401 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:39:09.401 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:44.148392+0000", 2020-09-08T15:39:09.401 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.5208e-05 2020-09-08T15:39:09.401 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.402 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.402 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:39:09.402 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:44.148417+0000", 2020-09-08T15:39:09.402 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2147000000000001e-05 2020-09-08T15:39:09.402 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.403 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.403 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:39:09.403 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:44.148429+0000", 2020-09-08T15:39:09.403 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000100898 2020-09-08T15:39:09.403 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.403 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.404 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:39:09.404 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:44.148530+0000", 2020-09-08T15:39:09.404 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00086350000000000001 2020-09-08T15:39:09.404 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.404 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.405 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:39:09.405 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:44.149393+0000", 2020-09-08T15:39:09.405 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00037822499999999999 2020-09-08T15:39:09.405 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.405 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.406 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3", 2020-09-08T15:39:09.406 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:44.149772+0000", 2020-09-08T15:39:09.406 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001898512 2020-09-08T15:39:09.406 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.406 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.407 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:39:09.407 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:44.151670+0000", 2020-09-08T15:39:09.407 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0026811890000000001 2020-09-08T15:39:09.407 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.407 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.408 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:39:09.408 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:44.154351+0000", 2020-09-08T15:39:09.408 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5186999999999999e-05 2020-09-08T15:39:09.410 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.410 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.411 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:39:09.411 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:44.154406+0000", 2020-09-08T15:39:09.411 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4277000000000002e-05 2020-09-08T15:39:09.411 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.411 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.412 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:39:09.412 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:44.154461+0000", 2020-09-08T15:39:09.412 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.080221955999999997 2020-09-08T15:39:09.412 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.412 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.412 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.413 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.413 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.413 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:84484 2.e 2:7ba21935:::benchmark_data_smithi166_20551_object84483:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:39:09.413 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:38:20.867301+0000", 2020-09-08T15:39:09.413 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 48.515985827000002, 2020-09-08T15:39:09.414 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14473607699999999, 2020-09-08T15:39:09.414 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:39:09.414 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:39:09.414 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:39:09.414 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:39:09.415 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:39:09.415 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 84484 2020-09-08T15:39:09.415 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.415 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:39:09.415 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.416 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:39:09.416 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.867301+0000", 2020-09-08T15:39:09.416 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:39:09.416 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.416 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.417 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:39:09.417 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.867301+0000", 2020-09-08T15:39:09.417 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999738 2020-09-08T15:39:09.417 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.417 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.417 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:39:09.418 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.867274+0000", 2020-09-08T15:39:09.418 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7307999999999997e-05 2020-09-08T15:39:09.418 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.418 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.418 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:39:09.419 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.867372+0000", 2020-09-08T15:39:09.419 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0889e-05 2020-09-08T15:39:09.419 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.419 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.419 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:39:09.420 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.867383+0000", 2020-09-08T15:39:09.420 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7609999999999998e-06 2020-09-08T15:39:09.420 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.420 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.420 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:39:09.420 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.867387+0000", 2020-09-08T15:39:09.421 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1984999999999999e-05 2020-09-08T15:39:09.421 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.421 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.421 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:39:09.421 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.867429+0000", 2020-09-08T15:39:09.422 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000105554 2020-09-08T15:39:09.422 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.422 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.422 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:39:09.422 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.867535+0000", 2020-09-08T15:39:09.423 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.12858234499999999 2020-09-08T15:39:09.423 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.423 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.423 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3", 2020-09-08T15:39:09.423 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.996117+0000", 2020-09-08T15:39:09.424 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0040692940000000002 2020-09-08T15:39:09.424 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.424 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.424 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:39:09.424 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:21.000187+0000", 2020-09-08T15:39:09.424 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0117365 2020-09-08T15:39:09.425 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.425 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.425 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:39:09.425 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:21.011923+0000", 2020-09-08T15:39:09.425 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7166e-05 2020-09-08T15:39:09.426 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.426 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.427 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:39:09.427 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:21.011990+0000", 2020-09-08T15:39:09.428 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6417e-05 2020-09-08T15:39:09.428 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.428 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.429 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:39:09.429 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:21.012037+0000", 2020-09-08T15:39:09.429 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14473607699999999 2020-09-08T15:39:09.429 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.429 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.430 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.430 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.430 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.430 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:84485 2.1 2:8ae1391b:::benchmark_data_smithi166_20551_object84484:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:39:09.430 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:38:20.867438+0000", 2020-09-08T15:39:09.431 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 48.515848175000002, 2020-09-08T15:39:09.431 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.138946245, 2020-09-08T15:39:09.431 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:39:09.431 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:39:09.431 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:39:09.431 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:39:09.432 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:39:09.432 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 84485 2020-09-08T15:39:09.432 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.432 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:39:09.432 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.433 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:39:09.433 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.867438+0000", 2020-09-08T15:39:09.433 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:39:09.434 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.434 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.434 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:39:09.434 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.867438+0000", 2020-09-08T15:39:09.435 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999752 2020-09-08T15:39:09.435 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.435 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.435 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:39:09.436 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.867414+0000", 2020-09-08T15:39:09.436 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4974000000000002e-05 2020-09-08T15:39:09.436 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.436 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.436 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:39:09.437 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.867509+0000", 2020-09-08T15:39:09.437 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9479999999999997e-06 2020-09-08T15:39:09.437 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.437 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.437 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:39:09.438 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.867517+0000", 2020-09-08T15:39:09.438 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7280000000000001e-06 2020-09-08T15:39:09.438 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.438 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.439 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:39:09.439 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.867521+0000", 2020-09-08T15:39:09.439 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7283999999999997e-05 2020-09-08T15:39:09.439 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.440 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.440 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:39:09.440 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.867558+0000", 2020-09-08T15:39:09.440 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.12865001700000001 2020-09-08T15:39:09.440 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.441 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.441 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:39:09.441 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.996209+0000", 2020-09-08T15:39:09.441 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00049853899999999999 2020-09-08T15:39:09.441 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.442 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.442 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3", 2020-09-08T15:39:09.442 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:20.996707+0000", 2020-09-08T15:39:09.442 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0094116310000000002 2020-09-08T15:39:09.442 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.443 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.443 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:39:09.443 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:21.006119+0000", 2020-09-08T15:39:09.443 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000119456 2020-09-08T15:39:09.443 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.443 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.444 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:39:09.444 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:21.006238+0000", 2020-09-08T15:39:09.445 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0068999999999995e-05 2020-09-08T15:39:09.445 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.445 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.445 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:39:09.445 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:21.006318+0000", 2020-09-08T15:39:09.446 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6199999999999996e-05 2020-09-08T15:39:09.446 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.446 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.446 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:39:09.446 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:21.006384+0000", 2020-09-08T15:39:09.447 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.138946245 2020-09-08T15:39:09.447 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.447 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.447 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.447 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.447 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.448 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:87297 2.e 2:72e044cc:::benchmark_data_smithi166_20551_object87296:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:39:09.448 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:38:22.694052+0000", 2020-09-08T15:39:09.448 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 46.689234540999998, 2020-09-08T15:39:09.448 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.090432888000000003, 2020-09-08T15:39:09.448 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:39:09.449 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:39:09.449 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:39:09.449 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:39:09.450 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:39:09.450 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87297 2020-09-08T15:39:09.451 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.451 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:39:09.451 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.451 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:39:09.452 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.694052+0000", 2020-09-08T15:39:09.452 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:39:09.452 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.452 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.453 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:39:09.453 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.694052+0000", 2020-09-08T15:39:09.453 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999619 2020-09-08T15:39:09.453 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.454 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.454 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:39:09.454 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.694014+0000", 2020-09-08T15:39:09.454 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000139233 2020-09-08T15:39:09.454 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.455 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.455 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:39:09.455 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.694153+0000", 2020-09-08T15:39:09.455 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8079999999999996e-05 2020-09-08T15:39:09.456 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.456 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.456 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:39:09.456 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.694251+0000", 2020-09-08T15:39:09.457 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5289999999999997e-06 2020-09-08T15:39:09.457 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.457 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.457 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:39:09.458 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.694258+0000", 2020-09-08T15:39:09.458 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4761e-05 2020-09-08T15:39:09.458 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.458 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.458 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:39:09.459 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.694312+0000", 2020-09-08T15:39:09.459 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000140383 2020-09-08T15:39:09.459 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.459 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.459 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:39:09.460 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.694453+0000", 2020-09-08T15:39:09.460 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017803400000000001 2020-09-08T15:39:09.460 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.460 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.461 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3", 2020-09-08T15:39:09.461 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.694631+0000", 2020-09-08T15:39:09.461 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0058946880000000004 2020-09-08T15:39:09.461 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.461 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.462 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:39:09.462 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.700526+0000", 2020-09-08T15:39:09.462 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4395999999999995e-05 2020-09-08T15:39:09.462 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.462 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.463 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:39:09.463 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.700620+0000", 2020-09-08T15:39:09.463 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.083742880000000006 2020-09-08T15:39:09.463 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.463 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.463 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:39:09.464 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.784363+0000", 2020-09-08T15:39:09.464 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000121858 2020-09-08T15:39:09.464 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.464 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.464 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:39:09.465 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.784485+0000", 2020-09-08T15:39:09.465 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.090432888000000003 2020-09-08T15:39:09.465 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.465 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.466 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.467 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.467 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.467 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:87301 2.e 2:79c41419:::benchmark_data_smithi166_20551_object87300:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:39:09.468 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:38:22.696463+0000", 2020-09-08T15:39:09.468 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 46.686823556, 2020-09-08T15:39:09.468 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.091497826000000004, 2020-09-08T15:39:09.468 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:39:09.468 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:39:09.469 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:39:09.469 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:39:09.469 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:39:09.469 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87301 2020-09-08T15:39:09.469 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.470 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:39:09.470 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.470 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:39:09.470 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.696463+0000", 2020-09-08T15:39:09.470 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:39:09.470 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.471 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.471 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:39:09.471 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.696463+0000", 2020-09-08T15:39:09.471 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999619 2020-09-08T15:39:09.471 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.472 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.472 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:39:09.472 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.696424+0000", 2020-09-08T15:39:09.472 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000152702 2020-09-08T15:39:09.472 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.473 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.473 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:39:09.473 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.696577+0000", 2020-09-08T15:39:09.473 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4630999999999999e-05 2020-09-08T15:39:09.473 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.474 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.474 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:39:09.474 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.696592+0000", 2020-09-08T15:39:09.474 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5760000000000002e-06 2020-09-08T15:39:09.474 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.474 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.475 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:39:09.475 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.696598+0000", 2020-09-08T15:39:09.475 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00064487400000000003 2020-09-08T15:39:09.475 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.475 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.476 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:39:09.476 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.697243+0000", 2020-09-08T15:39:09.476 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000177753 2020-09-08T15:39:09.476 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.476 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.477 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:39:09.477 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.697421+0000", 2020-09-08T15:39:09.477 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018763699999999999 2020-09-08T15:39:09.477 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.477 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.478 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3", 2020-09-08T15:39:09.478 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.697609+0000", 2020-09-08T15:39:09.478 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.089077594999999996 2020-09-08T15:39:09.478 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.478 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.479 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:39:09.479 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.786686+0000", 2020-09-08T15:39:09.479 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0010133329999999999 2020-09-08T15:39:09.479 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.479 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.480 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:39:09.480 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.787700+0000", 2020-09-08T15:39:09.480 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000150285 2020-09-08T15:39:09.480 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.480 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.481 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:39:09.481 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.787850+0000", 2020-09-08T15:39:09.481 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011076199999999999 2020-09-08T15:39:09.481 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.481 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.482 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:39:09.482 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.787961+0000", 2020-09-08T15:39:09.482 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.091497826000000004 2020-09-08T15:39:09.482 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.482 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.483 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.483 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.483 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.483 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:87302 2.1 2:83e64749:::benchmark_data_smithi166_20551_object87301:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:39:09.483 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:38:22.697389+0000", 2020-09-08T15:39:09.484 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 46.685897019000002, 2020-09-08T15:39:09.484 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.089558774999999993, 2020-09-08T15:39:09.484 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:39:09.484 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:39:09.484 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:39:09.484 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:39:09.485 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:39:09.485 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87302 2020-09-08T15:39:09.485 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.485 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:39:09.485 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.486 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:39:09.486 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.697389+0000", 2020-09-08T15:39:09.486 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:39:09.486 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.486 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.487 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:39:09.487 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.697389+0000", 2020-09-08T15:39:09.487 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999738 2020-09-08T15:39:09.487 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.487 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.488 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:39:09.488 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.697363+0000", 2020-09-08T15:39:09.488 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2892000000000004e-05 2020-09-08T15:39:09.488 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.488 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.488 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:39:09.489 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.697456+0000", 2020-09-08T15:39:09.489 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7780000000000002e-06 2020-09-08T15:39:09.489 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.489 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.489 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:39:09.490 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.697466+0000", 2020-09-08T15:39:09.490 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4190000000000004e-06 2020-09-08T15:39:09.490 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.490 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.490 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:39:09.491 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.697470+0000", 2020-09-08T15:39:09.491 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2388999999999997e-05 2020-09-08T15:39:09.491 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.491 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.491 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:39:09.492 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.697513+0000", 2020-09-08T15:39:09.492 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000100454 2020-09-08T15:39:09.492 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.492 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.492 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:39:09.492 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.697613+0000", 2020-09-08T15:39:09.493 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010014300000000001 2020-09-08T15:39:09.494 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.494 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.494 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3", 2020-09-08T15:39:09.494 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.697713+0000", 2020-09-08T15:39:09.494 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0024066930000000001 2020-09-08T15:39:09.495 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.495 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.495 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:39:09.495 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.700120+0000", 2020-09-08T15:39:09.495 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.086713442000000002 2020-09-08T15:39:09.496 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.496 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.496 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:39:09.496 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.786833+0000", 2020-09-08T15:39:09.496 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6458000000000006e-05 2020-09-08T15:39:09.497 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.497 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.497 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:39:09.497 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.786900+0000", 2020-09-08T15:39:09.497 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8179000000000001e-05 2020-09-08T15:39:09.498 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.498 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.498 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:39:09.498 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.786948+0000", 2020-09-08T15:39:09.498 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.089558774999999993 2020-09-08T15:39:09.498 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.499 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.499 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.499 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.499 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.499 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop_reply(client.34742.0:87297 2.e e145/126 ondisk, result = 0)", 2020-09-08T15:39:09.500 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:38:22.700512+0000", 2020-09-08T15:39:09.500 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 46.682774015, 2020-09-08T15:39:09.500 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.086133974000000002, 2020-09-08T15:39:09.500 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:39:09.500 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2020-09-08T15:39:09.501 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:39:09.501 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.501 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:39:09.501 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.700512+0000", 2020-09-08T15:39:09.501 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:39:09.502 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.502 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.502 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:39:09.502 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.700512+0000", 2020-09-08T15:39:09.502 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8853e-05 2020-09-08T15:39:09.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.503 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.503 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:39:09.503 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.700541+0000", 2020-09-08T15:39:09.503 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2599999999999996e-06 2020-09-08T15:39:09.504 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.504 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.504 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:39:09.504 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.700546+0000", 2020-09-08T15:39:09.504 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5820000000000002e-06 2020-09-08T15:39:09.504 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.505 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.505 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:39:09.505 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.700555+0000", 2020-09-08T15:39:09.505 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6319999999999996e-06 2020-09-08T15:39:09.505 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.506 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.506 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:39:09.506 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.700561+0000", 2020-09-08T15:39:09.506 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1866000000000002e-05 2020-09-08T15:39:09.506 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.507 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.507 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:39:09.507 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.700603+0000", 2020-09-08T15:39:09.507 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0429999999999996e-06 2020-09-08T15:39:09.507 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.508 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.508 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:39:09.508 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.700612+0000", 2020-09-08T15:39:09.508 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.086034737999999999 2020-09-08T15:39:09.508 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.508 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.509 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:39:09.509 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:22.786646+0000", 2020-09-08T15:39:09.509 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.086133974000000002 2020-09-08T15:39:09.509 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.509 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.510 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.510 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.510 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.510 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:93094 2.1 2:82bb6141:::benchmark_data_smithi166_20551_object93093:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e146)", 2020-09-08T15:39:09.510 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:38:26.714047+0000", 2020-09-08T15:39:09.511 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 42.669238866999997, 2020-09-08T15:39:09.511 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.114561169, 2020-09-08T15:39:09.511 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:39:09.511 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:39:09.511 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:39:09.512 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:39:09.512 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:39:09.512 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 93094 2020-09-08T15:39:09.512 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.512 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:39:09.512 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.513 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:39:09.513 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.714047+0000", 2020-09-08T15:39:09.513 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:39:09.513 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.513 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.514 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:39:09.514 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.714047+0000", 2020-09-08T15:39:09.514 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999576 2020-09-08T15:39:09.514 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.514 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.515 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:39:09.515 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.714005+0000", 2020-09-08T15:39:09.515 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013163000000000001 2020-09-08T15:39:09.515 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.515 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.516 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:39:09.516 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.714136+0000", 2020-09-08T15:39:09.516 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4961e-05 2020-09-08T15:39:09.516 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.516 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.517 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:39:09.517 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.714151+0000", 2020-09-08T15:39:09.517 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2820000000000002e-06 2020-09-08T15:39:09.517 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.517 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.518 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:39:09.518 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.714158+0000", 2020-09-08T15:39:09.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0377999999999997e-05 2020-09-08T15:39:09.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.518 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:39:09.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.714218+0000", 2020-09-08T15:39:09.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000153239 2020-09-08T15:39:09.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:39:09.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.714371+0000", 2020-09-08T15:39:09.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018021399999999999 2020-09-08T15:39:09.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3", 2020-09-08T15:39:09.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.714552+0000", 2020-09-08T15:39:09.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.113634847 2020-09-08T15:39:09.521 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:39:09.522 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.828186+0000", 2020-09-08T15:39:09.522 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2169000000000001e-05 2020-09-08T15:39:09.522 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.522 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.522 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:39:09.522 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.828239+0000", 2020-09-08T15:39:09.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000188238 2020-09-08T15:39:09.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:39:09.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.828427+0000", 2020-09-08T15:39:09.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018185200000000001 2020-09-08T15:39:09.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.524 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:39:09.524 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.828609+0000", 2020-09-08T15:39:09.525 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.114561169 2020-09-08T15:39:09.525 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.525 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.525 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.526 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:93087 2.3 e146/126 2:cf568eaf:::benchmark_data_smithi166_20551_object93086:head v 146'5902, mlcod=146'5899)", 2020-09-08T15:39:09.526 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:38:26.715174+0000", 2020-09-08T15:39:09.526 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 42.668112594999997, 2020-09-08T15:39:09.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.11313242599999999, 2020-09-08T15:39:09.527 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:39:09.527 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:39:09.527 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:39:09.527 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.527 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:39:09.527 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.715174+0000", 2020-09-08T15:39:09.528 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:39:09.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.528 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:39:09.528 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.715174+0000", 2020-09-08T15:39:09.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3232999999999995e-05 2020-09-08T15:39:09.529 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.529 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.529 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:39:09.529 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.715247+0000", 2020-09-08T15:39:09.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7080000000000002e-06 2020-09-08T15:39:09.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:39:09.530 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.715253+0000", 2020-09-08T15:39:09.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8429999999999999e-06 2020-09-08T15:39:09.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:39:09.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.715262+0000", 2020-09-08T15:39:09.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.425e-06 2020-09-08T15:39:09.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.532 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:39:09.532 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.715266+0000", 2020-09-08T15:39:09.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6523999999999998e-05 2020-09-08T15:39:09.533 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.533 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.533 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:39:09.533 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.715302+0000", 2020-09-08T15:39:09.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3334000000000001e-05 2020-09-08T15:39:09.533 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.534 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:39:09.534 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.715326+0000", 2020-09-08T15:39:09.534 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.11279009 2020-09-08T15:39:09.534 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:39:09.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.828116+0000", 2020-09-08T15:39:09.535 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019026899999999999 2020-09-08T15:39:09.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.536 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.536 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:39:09.536 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.828306+0000", 2020-09-08T15:39:09.536 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.11313242599999999 2020-09-08T15:39:09.536 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.536 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.537 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.537 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:93097 2.e 2:78709089:::benchmark_data_smithi166_20551_object93096:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e146)", 2020-09-08T15:39:09.537 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:38:26.715546+0000", 2020-09-08T15:39:09.538 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 42.667740682000002, 2020-09-08T15:39:09.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.11335471900000001, 2020-09-08T15:39:09.538 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:39:09.538 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:39:09.538 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:39:09.539 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:39:09.539 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:39:09.539 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 93097 2020-09-08T15:39:09.539 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.539 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:39:09.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:39:09.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.715546+0000", 2020-09-08T15:39:09.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:39:09.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:39:09.541 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.715546+0000", 2020-09-08T15:39:09.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999642 2020-09-08T15:39:09.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:39:09.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.715510+0000", 2020-09-08T15:39:09.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012815400000000001 2020-09-08T15:39:09.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.543 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:39:09.543 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.715638+0000", 2020-09-08T15:39:09.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3224000000000001e-05 2020-09-08T15:39:09.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.543 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:39:09.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.715651+0000", 2020-09-08T15:39:09.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1879999999999999e-06 2020-09-08T15:39:09.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:39:09.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.715657+0000", 2020-09-08T15:39:09.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00099329699999999997 2020-09-08T15:39:09.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:39:09.546 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.716651+0000", 2020-09-08T15:39:09.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000167398 2020-09-08T15:39:09.546 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.546 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:39:09.546 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.716818+0000", 2020-09-08T15:39:09.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000179316 2020-09-08T15:39:09.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.547 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3", 2020-09-08T15:39:09.547 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.716997+0000", 2020-09-08T15:39:09.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.111455318 2020-09-08T15:39:09.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:39:09.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.828453+0000", 2020-09-08T15:39:09.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000141342 2020-09-08T15:39:09.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:39:09.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.828594+0000", 2020-09-08T15:39:09.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016714599999999999 2020-09-08T15:39:09.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.550 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.550 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:39:09.550 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.828761+0000", 2020-09-08T15:39:09.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000139309 2020-09-08T15:39:09.551 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.551 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.551 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:39:09.551 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.828900+0000", 2020-09-08T15:39:09.551 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.11335471900000001 2020-09-08T15:39:09.552 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.552 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.552 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.553 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:93098 2.1 2:826ae3ce:::benchmark_data_smithi166_20551_object93097:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e146)", 2020-09-08T15:39:09.553 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:38:26.716687+0000", 2020-09-08T15:39:09.553 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 42.666599808999997, 2020-09-08T15:39:09.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.112369811, 2020-09-08T15:39:09.553 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:39:09.554 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:39:09.554 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:39:09.554 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:39:09.554 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:39:09.554 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 93098 2020-09-08T15:39:09.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.555 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:39:09.555 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.555 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:39:09.555 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.716687+0000", 2020-09-08T15:39:09.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:39:09.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.556 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.556 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:39:09.556 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.716687+0000", 2020-09-08T15:39:09.556 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999628 2020-09-08T15:39:09.557 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.557 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.557 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:39:09.557 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.716649+0000", 2020-09-08T15:39:09.558 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000141655 2020-09-08T15:39:09.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.558 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:39:09.558 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.716791+0000", 2020-09-08T15:39:09.559 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3695e-05 2020-09-08T15:39:09.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.559 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:39:09.559 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.716805+0000", 2020-09-08T15:39:09.559 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8479999999999997e-06 2020-09-08T15:39:09.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.560 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:39:09.560 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.716810+0000", 2020-09-08T15:39:09.561 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00050035100000000001 2020-09-08T15:39:09.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.561 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.561 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:39:09.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.717311+0000", 2020-09-08T15:39:09.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017896399999999999 2020-09-08T15:39:09.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.562 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.562 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:39:09.562 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.717490+0000", 2020-09-08T15:39:09.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000173602 2020-09-08T15:39:09.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3", 2020-09-08T15:39:09.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.717663+0000", 2020-09-08T15:39:09.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.110976405 2020-09-08T15:39:09.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.564 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:39:09.565 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.828640+0000", 2020-09-08T15:39:09.565 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000111483 2020-09-08T15:39:09.565 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.566 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.566 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:39:09.566 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.828751+0000", 2020-09-08T15:39:09.566 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017817199999999999 2020-09-08T15:39:09.567 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.567 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.567 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:39:09.567 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.828929+0000", 2020-09-08T15:39:09.568 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012693200000000001 2020-09-08T15:39:09.570 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.570 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.570 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:39:09.571 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.829056+0000", 2020-09-08T15:39:09.571 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.112369811 2020-09-08T15:39:09.571 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.571 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.571 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.572 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.572 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.572 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop_reply(client.34742.0:93094 2.1 e146/126 ondisk, result = 0)", 2020-09-08T15:39:09.572 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:38:26.717843+0000", 2020-09-08T15:39:09.572 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 42.665443207999999, 2020-09-08T15:39:09.573 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.110349956, 2020-09-08T15:39:09.573 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:39:09.573 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2020-09-08T15:39:09.573 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:39:09.573 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:39:09.574 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.717843+0000", 2020-09-08T15:39:09.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:39:09.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.574 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:39:09.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.717843+0000", 2020-09-08T15:39:09.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3687999999999999e-05 2020-09-08T15:39:09.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:39:09.576 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.717887+0000", 2020-09-08T15:39:09.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3320000000000003e-06 2020-09-08T15:39:09.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.576 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:39:09.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.717894+0000", 2020-09-08T15:39:09.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.272e-05 2020-09-08T15:39:09.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:39:09.578 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.717907+0000", 2020-09-08T15:39:09.578 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6829999999999996e-06 2020-09-08T15:39:09.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.578 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.578 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:39:09.579 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.717913+0000", 2020-09-08T15:39:09.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.110249316 2020-09-08T15:39:09.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.579 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:39:09.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.828162+0000", 2020-09-08T15:39:09.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3064000000000001e-05 2020-09-08T15:39:09.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:39:09.581 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.828175+0000", 2020-09-08T15:39:09.581 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8153000000000001e-05 2020-09-08T15:39:09.581 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.581 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.581 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:39:09.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:38:26.828193+0000", 2020-09-08T15:39:09.582 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.110349956 2020-09-08T15:39:09.582 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.582 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.582 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.583 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:136438 2.1 2:8a032ac8:::benchmark_data_smithi166_20551_object136437:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e160)", 2020-09-08T15:39:09.583 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:01.625243+0000", 2020-09-08T15:39:09.583 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 7.7580435459999997, 2020-09-08T15:39:09.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.103461394, 2020-09-08T15:39:09.584 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:39:09.584 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:39:09.584 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:39:09.584 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:39:09.584 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:39:09.585 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 136438 2020-09-08T15:39:09.585 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.585 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:39:09.586 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:39:09.586 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.625243+0000", 2020-09-08T15:39:09.586 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:39:09.586 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.587 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.587 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:39:09.587 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.625243+0000", 2020-09-08T15:39:09.587 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999633 2020-09-08T15:39:09.587 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.588 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.588 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:39:09.588 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.625206+0000", 2020-09-08T15:39:09.588 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000136468 2020-09-08T15:39:09.588 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.589 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.589 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:39:09.589 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.625342+0000", 2020-09-08T15:39:09.589 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6388999999999998e-05 2020-09-08T15:39:09.589 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.590 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.590 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:39:09.590 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.625359+0000", 2020-09-08T15:39:09.590 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3640000000000004e-06 2020-09-08T15:39:09.590 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.590 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.591 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:39:09.591 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.625365+0000", 2020-09-08T15:39:09.591 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3130999999999997e-05 2020-09-08T15:39:09.591 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.591 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.592 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:39:09.592 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.625418+0000", 2020-09-08T15:39:09.592 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000166897 2020-09-08T15:39:09.592 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.592 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:39:09.593 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.625585+0000", 2020-09-08T15:39:09.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000160171 2020-09-08T15:39:09.593 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.593 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.594 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3", 2020-09-08T15:39:09.594 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.625745+0000", 2020-09-08T15:39:09.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.101993339 2020-09-08T15:39:09.594 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.595 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:39:09.595 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.727739+0000", 2020-09-08T15:39:09.595 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00084019699999999999 2020-09-08T15:39:09.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.595 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:39:09.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.728579+0000", 2020-09-08T15:39:09.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8072999999999997e-05 2020-09-08T15:39:09.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:39:09.597 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.728647+0000", 2020-09-08T15:39:09.597 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7289e-05 2020-09-08T15:39:09.597 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.597 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:39:09.598 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.728704+0000", 2020-09-08T15:39:09.598 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.103461394 2020-09-08T15:39:09.598 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.598 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.598 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.599 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.599 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:136435 2.3 e160/126 2:c0ebca55:::benchmark_data_smithi166_20551_object136434:head v 160'8529, mlcod=160'8526)", 2020-09-08T15:39:09.599 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:01.628552+0000", 2020-09-08T15:39:09.599 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 7.7547346900000003, 2020-09-08T15:39:09.599 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.099344068999999993, 2020-09-08T15:39:09.600 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:39:09.600 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:39:09.600 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:39:09.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:39:09.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.628552+0000", 2020-09-08T15:39:09.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:39:09.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:39:09.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.628552+0000", 2020-09-08T15:39:09.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7092000000000001e-05 2020-09-08T15:39:09.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:39:09.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.628619+0000", 2020-09-08T15:39:09.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0159999999999997e-06 2020-09-08T15:39:09.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:39:09.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.628625+0000", 2020-09-08T15:39:09.604 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4639999999999995e-06 2020-09-08T15:39:09.604 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:39:09.604 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.628634+0000", 2020-09-08T15:39:09.605 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8930000000000004e-06 2020-09-08T15:39:09.605 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.605 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.605 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:39:09.605 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.628639+0000", 2020-09-08T15:39:09.606 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4993000000000001e-05 2020-09-08T15:39:09.606 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.606 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.606 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:39:09.606 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.628684+0000", 2020-09-08T15:39:09.606 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.5928999999999999e-05 2020-09-08T15:39:09.607 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.607 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:39:09.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.628710+0000", 2020-09-08T15:39:09.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.099072300000000002 2020-09-08T15:39:09.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:39:09.608 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.727782+0000", 2020-09-08T15:39:09.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011338200000000001 2020-09-08T15:39:09.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.609 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.609 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:39:09.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:01.727896+0000", 2020-09-08T15:39:09.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.099344068999999993 2020-09-08T15:39:09.610 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.610 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.610 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.610 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.614 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:140225 2.1 2:865c1839:::benchmark_data_smithi166_20551_object140224:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e162)", 2020-09-08T15:39:09.615 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:05.110109+0000", 2020-09-08T15:39:09.615 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 4.2731776869999996, 2020-09-08T15:39:09.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.109162196, 2020-09-08T15:39:09.615 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:39:09.616 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:39:09.616 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:39:09.616 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:39:09.616 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:39:09.617 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 140225 2020-09-08T15:39:09.617 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.617 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:39:09.617 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.618 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:39:09.618 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.110109+0000", 2020-09-08T15:39:09.618 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:39:09.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.618 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:39:09.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.110109+0000", 2020-09-08T15:39:09.619 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999948 2020-09-08T15:39:09.619 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:39:09.620 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.110057+0000", 2020-09-08T15:39:09.620 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014424300000000001 2020-09-08T15:39:09.620 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.620 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.620 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:39:09.621 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.110201+0000", 2020-09-08T15:39:09.621 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5071e-05 2020-09-08T15:39:09.621 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.621 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.621 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:39:09.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.110216+0000", 2020-09-08T15:39:09.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7800000000000003e-06 2020-09-08T15:39:09.622 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:39:09.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.110223+0000", 2020-09-08T15:39:09.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9665e-05 2020-09-08T15:39:09.623 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.623 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.623 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:39:09.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.110273+0000", 2020-09-08T15:39:09.624 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8474999999999995e-05 2020-09-08T15:39:09.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.624 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:39:09.624 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.110371+0000", 2020-09-08T15:39:09.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010622 2020-09-08T15:39:09.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.625 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3", 2020-09-08T15:39:09.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.110477+0000", 2020-09-08T15:39:09.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.097852507000000005 2020-09-08T15:39:09.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:39:09.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.208330+0000", 2020-09-08T15:39:09.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010819446999999999 2020-09-08T15:39:09.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.627 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:39:09.627 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.219149+0000", 2020-09-08T15:39:09.627 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7868000000000002e-05 2020-09-08T15:39:09.628 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.628 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.628 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:39:09.628 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.219207+0000", 2020-09-08T15:39:09.628 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3770999999999996e-05 2020-09-08T15:39:09.629 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.629 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:39:09.629 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.219271+0000", 2020-09-08T15:39:09.630 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.109162196 2020-09-08T15:39:09.630 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.630 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.630 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.631 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.631 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:140228 2.1 2:82d4a4c6:::benchmark_data_smithi166_20551_object140227:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e162)", 2020-09-08T15:39:09.631 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:05.112739+0000", 2020-09-08T15:39:09.631 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 4.2705469760000003, 2020-09-08T15:39:09.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.111005441, 2020-09-08T15:39:09.631 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:39:09.632 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:39:09.632 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:39:09.632 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:39:09.632 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:39:09.632 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 140228 2020-09-08T15:39:09.633 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.633 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:39:09.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.633 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:39:09.633 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.112739+0000", 2020-09-08T15:39:09.634 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:39:09.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.634 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.634 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:39:09.634 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.112739+0000", 2020-09-08T15:39:09.635 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999743 2020-09-08T15:39:09.635 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.635 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.635 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:39:09.635 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.112714+0000", 2020-09-08T15:39:09.636 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4423e-05 2020-09-08T15:39:09.636 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.636 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.636 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:39:09.636 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.112808+0000", 2020-09-08T15:39:09.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8679999999999994e-06 2020-09-08T15:39:09.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.637 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:39:09.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.112818+0000", 2020-09-08T15:39:09.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3720000000000004e-06 2020-09-08T15:39:09.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.638 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:39:09.639 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.112823+0000", 2020-09-08T15:39:09.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.095635390000000001 2020-09-08T15:39:09.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:39:09.640 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.208459+0000", 2020-09-08T15:39:09.640 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000319547 2020-09-08T15:39:09.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.640 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:39:09.641 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.208778+0000", 2020-09-08T15:39:09.641 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000369763 2020-09-08T15:39:09.641 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.641 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.641 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3", 2020-09-08T15:39:09.641 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.209148+0000", 2020-09-08T15:39:09.642 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.011708127 2020-09-08T15:39:09.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:39:09.642 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.220856+0000", 2020-09-08T15:39:09.643 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0027665630000000001 2020-09-08T15:39:09.643 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.643 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.643 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:39:09.643 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.223623+0000", 2020-09-08T15:39:09.644 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3059000000000006e-05 2020-09-08T15:39:09.644 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.644 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.644 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:39:09.644 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.223696+0000", 2020-09-08T15:39:09.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8866000000000003e-05 2020-09-08T15:39:09.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.645 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:39:09.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:05.223745+0000", 2020-09-08T15:39:09.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.111005441 2020-09-08T15:39:09.646 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.646 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.646 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.647 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:143227 2.3 e163/126 2:c04480ca:::benchmark_data_smithi166_20551_object143226:head v 163'8937, mlcod=163'8935)", 2020-09-08T15:39:09.647 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:07.854592+0000", 2020-09-08T15:39:09.647 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 1.5286940769999999, 2020-09-08T15:39:09.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.100932886, 2020-09-08T15:39:09.647 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:39:09.648 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:39:09.648 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:39:09.648 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.648 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:39:09.648 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.854592+0000", 2020-09-08T15:39:09.648 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:39:09.649 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.649 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.649 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:39:09.649 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.854592+0000", 2020-09-08T15:39:09.649 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013425000000000001 2020-09-08T15:39:09.650 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.650 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.650 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:39:09.650 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.854727+0000", 2020-09-08T15:39:09.650 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1109e-05 2020-09-08T15:39:09.651 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.651 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.651 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:39:09.651 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.854738+0000", 2020-09-08T15:39:09.651 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6918e-05 2020-09-08T15:39:09.651 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.652 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.652 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:39:09.652 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.854755+0000", 2020-09-08T15:39:09.652 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9889999999999992e-06 2020-09-08T15:39:09.652 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.653 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.653 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:39:09.653 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.854763+0000", 2020-09-08T15:39:09.653 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6790000000000003e-05 2020-09-08T15:39:09.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:39:09.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.854829+0000", 2020-09-08T15:39:09.654 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1031000000000002e-05 2020-09-08T15:39:09.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.655 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.655 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:39:09.655 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.854860+0000", 2020-09-08T15:39:09.655 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.100525584 2020-09-08T15:39:09.655 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.655 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.656 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:39:09.656 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.955386+0000", 2020-09-08T15:39:09.656 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013921499999999999 2020-09-08T15:39:09.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.656 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.657 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:39:09.657 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.955525+0000", 2020-09-08T15:39:09.657 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.100932886 2020-09-08T15:39:09.657 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.657 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.658 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.658 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.658 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.658 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:143229 2.e 2:7bcbd938:::benchmark_data_smithi166_20551_object143228:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e163)", 2020-09-08T15:39:09.658 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:07.856143+0000", 2020-09-08T15:39:09.658 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 1.5271429219999999, 2020-09-08T15:39:09.659 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.11611410699999999, 2020-09-08T15:39:09.659 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:39:09.659 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:39:09.659 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:39:09.659 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:39:09.660 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:39:09.660 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 143229 2020-09-08T15:39:09.660 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.660 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:39:09.660 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.661 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:39:09.661 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.856143+0000", 2020-09-08T15:39:09.661 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:39:09.661 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.661 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.661 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:39:09.662 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.856143+0000", 2020-09-08T15:39:09.662 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999461 2020-09-08T15:39:09.662 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.662 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.662 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:39:09.663 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.856089+0000", 2020-09-08T15:39:09.663 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.099425474 2020-09-08T15:39:09.663 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.663 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.664 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:39:09.664 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.955515+0000", 2020-09-08T15:39:09.664 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4870000000000002e-05 2020-09-08T15:39:09.664 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.664 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.665 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:39:09.665 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.955560+0000", 2020-09-08T15:39:09.665 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7793999999999999e-05 2020-09-08T15:39:09.665 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.665 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.665 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:39:09.666 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.955578+0000", 2020-09-08T15:39:09.666 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4038999999999999e-05 2020-09-08T15:39:09.666 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.666 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.666 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:39:09.667 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.955672+0000", 2020-09-08T15:39:09.667 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012723500000000001 2020-09-08T15:39:09.667 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.667 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.667 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:39:09.668 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.955799+0000", 2020-09-08T15:39:09.668 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012628199999999999 2020-09-08T15:39:09.668 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.668 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.668 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3", 2020-09-08T15:39:09.668 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.955925+0000", 2020-09-08T15:39:09.669 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.011598209 2020-09-08T15:39:09.669 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.669 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.669 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:39:09.669 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.967523+0000", 2020-09-08T15:39:09.670 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0045827480000000002 2020-09-08T15:39:09.670 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.670 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.670 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:39:09.670 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.972106+0000", 2020-09-08T15:39:09.671 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4778999999999995e-05 2020-09-08T15:39:09.671 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.671 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.671 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:39:09.671 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.972201+0000", 2020-09-08T15:39:09.672 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6697999999999998e-05 2020-09-08T15:39:09.672 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.672 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.672 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:39:09.672 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:07.972258+0000", 2020-09-08T15:39:09.673 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.11611410699999999 2020-09-08T15:39:09.673 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.673 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.673 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.673 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.673 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.674 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:143831 2.3 e163/126 2:c699b655:::benchmark_data_smithi166_20551_object143830:head v 163'8980, mlcod=163'8978)", 2020-09-08T15:39:09.674 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:08.509834+0000", 2020-09-08T15:39:09.674 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.87345273999999995, 2020-09-08T15:39:09.674 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.39543011700000003, 2020-09-08T15:39:09.674 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:39:09.675 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:39:09.675 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:39:09.675 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.675 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:39:09.675 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.509834+0000", 2020-09-08T15:39:09.676 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:39:09.676 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.676 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.676 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:39:09.676 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.509834+0000", 2020-09-08T15:39:09.676 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000115139 2020-09-08T15:39:09.677 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.677 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.677 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:39:09.677 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.509949+0000", 2020-09-08T15:39:09.677 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6490000000000003e-06 2020-09-08T15:39:09.678 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.678 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.678 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:39:09.678 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.509958+0000", 2020-09-08T15:39:09.678 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3651e-05 2020-09-08T15:39:09.679 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.679 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.679 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:39:09.679 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.509972+0000", 2020-09-08T15:39:09.679 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9179999999999999e-06 2020-09-08T15:39:09.680 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.680 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.680 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:39:09.680 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.509978+0000", 2020-09-08T15:39:09.681 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4193999999999997e-05 2020-09-08T15:39:09.681 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.681 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.681 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:39:09.681 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.510022+0000", 2020-09-08T15:39:09.681 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6381e-05 2020-09-08T15:39:09.682 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.682 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.682 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:39:09.682 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.510049+0000", 2020-09-08T15:39:09.682 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.39507293900000001 2020-09-08T15:39:09.683 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.683 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.683 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:39:09.683 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.905121+0000", 2020-09-08T15:39:09.683 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014224600000000001 2020-09-08T15:39:09.684 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.684 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.684 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:39:09.684 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.905264+0000", 2020-09-08T15:39:09.684 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.39543011700000003 2020-09-08T15:39:09.685 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.685 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.685 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.685 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.685 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.686 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.34742.0:143835 2.e 2:73b1cc29:::benchmark_data_smithi166_20551_object143834:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e163)", 2020-09-08T15:39:09.686 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:08.513148+0000", 2020-09-08T15:39:09.686 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.87013865700000004, 2020-09-08T15:39:09.686 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.41270290999999998, 2020-09-08T15:39:09.687 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:39:09.687 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:39:09.687 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:39:09.687 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.34742", 2020-09-08T15:39:09.687 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:39:09.688 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 143835 2020-09-08T15:39:09.688 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.688 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:39:09.688 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.688 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:39:09.689 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.513148+0000", 2020-09-08T15:39:09.689 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:39:09.689 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.689 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.689 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:39:09.690 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.513148+0000", 2020-09-08T15:39:09.690 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999452 2020-09-08T15:39:09.690 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.690 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.690 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:39:09.691 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.513093+0000", 2020-09-08T15:39:09.691 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000156114 2020-09-08T15:39:09.691 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.691 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.691 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:39:09.692 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.513249+0000", 2020-09-08T15:39:09.692 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4653e-05 2020-09-08T15:39:09.692 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.692 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.692 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:39:09.692 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.513263+0000", 2020-09-08T15:39:09.693 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6980000000000001e-06 2020-09-08T15:39:09.693 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.693 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.693 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:39:09.693 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.513270+0000", 2020-09-08T15:39:09.694 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3446000000000003e-05 2020-09-08T15:39:09.694 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.694 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.694 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:39:09.694 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.513324+0000", 2020-09-08T15:39:09.695 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000118749 2020-09-08T15:39:09.695 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.695 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.695 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:39:09.695 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.513442+0000", 2020-09-08T15:39:09.695 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.39208391599999998 2020-09-08T15:39:09.696 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.696 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.696 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3", 2020-09-08T15:39:09.696 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.905526+0000", 2020-09-08T15:39:09.696 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.014897713999999999 2020-09-08T15:39:09.697 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.697 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.697 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:39:09.697 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.920424+0000", 2020-09-08T15:39:09.697 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0052909339999999997 2020-09-08T15:39:09.698 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.698 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.698 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:39:09.698 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.925715+0000", 2020-09-08T15:39:09.698 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6810000000000004e-05 2020-09-08T15:39:09.698 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.699 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.699 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:39:09.699 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.925802+0000", 2020-09-08T15:39:09.699 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8828000000000001e-05 2020-09-08T15:39:09.699 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:39:09.700 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:39:09.700 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:39:09.700 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:08.925851+0000", 2020-09-08T15:39:09.700 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.41270290999999998 2020-09-08T15:39:09.700 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.700 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.701 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.701 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:39:09.701 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:39:09.701 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:39:09.702 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:09.703 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:39:09 smithi104 bash[22305]: debug 2020-09-08T15:39:09.310+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:09.766 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:39:09.775 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 87 16 144715 144699 103.941 48.6875 0.0457596 0.00961909 2020-09-08T15:39:09.805 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:39:09.807 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:09.844 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:39:09.879 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:39:09.911 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:09.916 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:39:09.945 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:39:09.984 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:39:10.037 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:39:10.079 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:39:10.105 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:39:09 smithi055 bash[16646]: debug 2020-09-08T15:39:09.809+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:10.106 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:39:10 smithi055 bash[16646]: debug 2020-09-08T15:39:10.105+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:10.106 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:10.121 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:39:10.158 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:39:10.189 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:39:10.191 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:39:10.191 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:39:10.192 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:39:10.192 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:39:10.192 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:39:10.192 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:39:10.192 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:39:10.193 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:39:10.193 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:39:10.193 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:39:10.193 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:39:10.193 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:39:10.213 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:10.581 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:10 smithi104 bash[9203]: cluster 2020-09-08T15:39:09.616099+0000 mgr.x (mgr.24118) 349 : cluster [DBG] pgmap v434: 17 pgs: 17 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s wr, 906 op/s 2020-09-08T15:39:11.083 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:10 smithi104 bash[9203]: audit 2020-09-08T15:39:09.713151+0000 mon.a (mon.0) 979 : audit [INF] from='client.? v1:172.21.15.104:0/1575249095' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:39:11.083 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:10 smithi104 bash[9203]: audit 2020-09-08T15:39:09.748670+0000 mon.a (mon.0) 980 : audit [DBG] from='client.? v1:172.21.15.104:0/2799032532' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:39:11.085 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:10 smithi055 bash[9296]: cluster 2020-09-08T15:39:09.616099+0000 mgr.x (mgr.24118) 349 : cluster [DBG] pgmap v434: 17 pgs: 17 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s wr, 906 op/s 2020-09-08T15:39:11.085 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:10 smithi055 bash[9296]: audit 2020-09-08T15:39:09.713151+0000 mon.a (mon.0) 979 : audit [INF] from='client.? v1:172.21.15.104:0/1575249095' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:39:11.085 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:10 smithi055 bash[9296]: audit 2020-09-08T15:39:09.748670+0000 mon.a (mon.0) 980 : audit [DBG] from='client.? v1:172.21.15.104:0/2799032532' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:39:11.086 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 88 16 145789 145773 103.523 67.125 0.0192408 0.00965788 2020-09-08T15:39:11.086 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:10 smithi166 bash[9446]: cluster 2020-09-08T15:39:09.616099+0000 mgr.x (mgr.24118) 349 : cluster [DBG] pgmap v434: 17 pgs: 17 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s wr, 906 op/s 2020-09-08T15:39:11.086 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:10 smithi166 bash[9446]: audit 2020-09-08T15:39:09.713151+0000 mon.a (mon.0) 979 : audit [INF] from='client.? v1:172.21.15.104:0/1575249095' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:39:11.086 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:10 smithi166 bash[9446]: audit 2020-09-08T15:39:09.748670+0000 mon.a (mon.0) 980 : audit [DBG] from='client.? v1:172.21.15.104:0/2799032532' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:39:11.087 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:39:11.087 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2020-09-08T15:39:11.232 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:39:11.775 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 89 16 146684 146668 102.988 55.9375 0.0107723 0.00970799 2020-09-08T15:39:11.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:11 smithi166 bash[9446]: cluster 2020-09-08T15:39:10.580986+0000 mon.a (mon.0) 981 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:39:11.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:11 smithi166 bash[9446]: audit 2020-09-08T15:39:10.589130+0000 mon.a (mon.0) 982 : audit [INF] from='client.? v1:172.21.15.104:0/1575249095' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:39:11.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:11 smithi166 bash[9446]: cluster 2020-09-08T15:39:10.589189+0000 mon.a (mon.0) 983 : cluster [DBG] osdmap e164: 12 total, 12 up, 12 in 2020-09-08T15:39:11.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:11 smithi166 bash[9446]: cluster 2020-09-08T15:39:10.681560+0000 osd.5 (osd.5) 15 : cluster [DBG] 2.3 scrub starts 2020-09-08T15:39:11.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:11 smithi166 bash[9446]: cluster 2020-09-08T15:39:10.846860+0000 osd.11 (osd.11) 13 : cluster [DBG] 2.a scrub starts 2020-09-08T15:39:11.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:11 smithi055 bash[9296]: cluster 2020-09-08T15:39:10.580986+0000 mon.a (mon.0) 981 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:39:11.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:11 smithi055 bash[9296]: audit 2020-09-08T15:39:10.589130+0000 mon.a (mon.0) 982 : audit [INF] from='client.? v1:172.21.15.104:0/1575249095' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:39:11.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:11 smithi055 bash[9296]: cluster 2020-09-08T15:39:10.589189+0000 mon.a (mon.0) 983 : cluster [DBG] osdmap e164: 12 total, 12 up, 12 in 2020-09-08T15:39:11.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:11 smithi055 bash[9296]: cluster 2020-09-08T15:39:10.681560+0000 osd.5 (osd.5) 15 : cluster [DBG] 2.3 scrub starts 2020-09-08T15:39:11.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:11 smithi055 bash[9296]: cluster 2020-09-08T15:39:10.846860+0000 osd.11 (osd.11) 13 : cluster [DBG] 2.a scrub starts 2020-09-08T15:39:11.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:11 smithi104 bash[9203]: cluster 2020-09-08T15:39:10.580986+0000 mon.a (mon.0) 981 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:39:11.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:11 smithi104 bash[9203]: audit 2020-09-08T15:39:10.589130+0000 mon.a (mon.0) 982 : audit [INF] from='client.? v1:172.21.15.104:0/1575249095' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:39:11.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:11 smithi104 bash[9203]: cluster 2020-09-08T15:39:10.589189+0000 mon.a (mon.0) 983 : cluster [DBG] osdmap e164: 12 total, 12 up, 12 in 2020-09-08T15:39:11.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:11 smithi104 bash[9203]: cluster 2020-09-08T15:39:10.681560+0000 osd.5 (osd.5) 15 : cluster [DBG] 2.3 scrub starts 2020-09-08T15:39:11.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:11 smithi104 bash[9203]: cluster 2020-09-08T15:39:10.846860+0000 osd.11 (osd.11) 13 : cluster [DBG] 2.a scrub starts 2020-09-08T15:39:12.775 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 90 16 147617 147601 102.491 58.3125 0.0288402 0.00975354 2020-09-08T15:39:12.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:12 smithi166 bash[9446]: cluster 2020-09-08T15:39:11.272072+0000 osd.9 (osd.9) 13 : cluster [DBG] 2.d scrub starts 2020-09-08T15:39:12.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:12 smithi166 bash[9446]: cluster 2020-09-08T15:39:11.440614+0000 osd.1 (osd.1) 24 : cluster [DBG] 2.6 scrub starts 2020-09-08T15:39:12.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:12 smithi166 bash[9446]: cluster 2020-09-08T15:39:11.502249+0000 osd.6 (osd.6) 15 : cluster [DBG] 2.7 scrub starts 2020-09-08T15:39:12.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:12 smithi166 bash[9446]: cluster 2020-09-08T15:39:11.616598+0000 mgr.x (mgr.24118) 350 : cluster [DBG] pgmap v436: 17 pgs: 17 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s wr, 785 op/s 2020-09-08T15:39:12.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:12 smithi166 bash[9446]: cluster 2020-09-08T15:39:12.093799+0000 osd.5 (osd.5) 16 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:12.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:12 smithi166 bash[9446]: cluster 2020-09-08T15:39:12.093859+0000 osd.5 (osd.5) 17 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:12.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:12 smithi055 bash[9296]: cluster 2020-09-08T15:39:11.272072+0000 osd.9 (osd.9) 13 : cluster [DBG] 2.d scrub starts 2020-09-08T15:39:12.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:12 smithi055 bash[9296]: cluster 2020-09-08T15:39:11.440614+0000 osd.1 (osd.1) 24 : cluster [DBG] 2.6 scrub starts 2020-09-08T15:39:12.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:12 smithi055 bash[9296]: cluster 2020-09-08T15:39:11.502249+0000 osd.6 (osd.6) 15 : cluster [DBG] 2.7 scrub starts 2020-09-08T15:39:12.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:12 smithi055 bash[9296]: cluster 2020-09-08T15:39:11.616598+0000 mgr.x (mgr.24118) 350 : cluster [DBG] pgmap v436: 17 pgs: 17 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s wr, 785 op/s 2020-09-08T15:39:12.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:12 smithi055 bash[9296]: cluster 2020-09-08T15:39:12.093799+0000 osd.5 (osd.5) 16 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:12.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:12 smithi055 bash[9296]: cluster 2020-09-08T15:39:12.093859+0000 osd.5 (osd.5) 17 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:12.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:12 smithi104 bash[9203]: cluster 2020-09-08T15:39:11.272072+0000 osd.9 (osd.9) 13 : cluster [DBG] 2.d scrub starts 2020-09-08T15:39:12.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:12 smithi104 bash[9203]: cluster 2020-09-08T15:39:11.440614+0000 osd.1 (osd.1) 24 : cluster [DBG] 2.6 scrub starts 2020-09-08T15:39:12.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:12 smithi104 bash[9203]: cluster 2020-09-08T15:39:11.502249+0000 osd.6 (osd.6) 15 : cluster [DBG] 2.7 scrub starts 2020-09-08T15:39:12.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:12 smithi104 bash[9203]: cluster 2020-09-08T15:39:11.616598+0000 mgr.x (mgr.24118) 350 : cluster [DBG] pgmap v436: 17 pgs: 17 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s wr, 785 op/s 2020-09-08T15:39:12.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:12 smithi104 bash[9203]: cluster 2020-09-08T15:39:12.093799+0000 osd.5 (osd.5) 16 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:12.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:12 smithi104 bash[9203]: cluster 2020-09-08T15:39:12.093859+0000 osd.5 (osd.5) 17 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:13.595 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:13 smithi104 bash[9203]: cluster 2020-09-08T15:39:12.592067+0000 mon.a (mon.0) 984 : cluster [DBG] osdmap e165: 12 total, 12 up, 12 in 2020-09-08T15:39:13.775 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Total time run: 90.0132 2020-09-08T15:39:13.776 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Total writes made: 147617 2020-09-08T15:39:13.776 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Write size: 65536 2020-09-08T15:39:13.776 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Object size: 65536 2020-09-08T15:39:13.776 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Bandwidth (MB/sec): 102.497 2020-09-08T15:39:13.777 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev Bandwidth: 43.859 2020-09-08T15:39:13.777 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max bandwidth (MB/sec): 239.875 2020-09-08T15:39:13.777 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min bandwidth (MB/sec): 48.6875 2020-09-08T15:39:13.777 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Average IOPS: 1639 2020-09-08T15:39:13.777 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev IOPS: 701 2020-09-08T15:39:13.778 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max IOPS: 3838 2020-09-08T15:39:13.778 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min IOPS: 779 2020-09-08T15:39:13.778 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Average Latency(s): 0.00975533 2020-09-08T15:39:13.778 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev Latency(s): 0.0123087 2020-09-08T15:39:13.779 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max latency(s): 0.426061 2020-09-08T15:39:13.779 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min latency(s): 0.00138961 2020-09-08T15:39:13.785 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Cleaning up (deleting benchmark objects) 2020-09-08T15:39:13.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:13 smithi166 bash[9446]: cluster 2020-09-08T15:39:12.592067+0000 mon.a (mon.0) 984 : cluster [DBG] osdmap e165: 12 total, 12 up, 12 in 2020-09-08T15:39:13.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:13 smithi166 bash[9446]: cluster 2020-09-08T15:39:12.946276+0000 osd.5 (osd.5) 18 : cluster [DBG] 2.3 scrub ok 2020-09-08T15:39:13.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:13 smithi055 bash[9296]: cluster 2020-09-08T15:39:12.592067+0000 mon.a (mon.0) 984 : cluster [DBG] osdmap e165: 12 total, 12 up, 12 in 2020-09-08T15:39:13.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:13 smithi055 bash[9296]: cluster 2020-09-08T15:39:12.946276+0000 osd.5 (osd.5) 18 : cluster [DBG] 2.3 scrub ok 2020-09-08T15:39:14.171 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:13 smithi104 bash[9203]: cluster 2020-09-08T15:39:12.946276+0000 osd.5 (osd.5) 18 : cluster [DBG] 2.3 scrub ok 2020-09-08T15:39:14.492 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:39:14.661 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:39:14.845 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:14.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:14 smithi166 bash[9446]: cluster 2020-09-08T15:39:13.355197+0000 osd.11 (osd.11) 14 : cluster [DBG] 2.a scrub ok 2020-09-08T15:39:14.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:14 smithi166 bash[9446]: cluster 2020-09-08T15:39:13.390654+0000 osd.9 (osd.9) 14 : cluster [DBG] 2.d scrub ok 2020-09-08T15:39:14.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:14 smithi166 bash[9446]: cluster 2020-09-08T15:39:13.448180+0000 osd.7 (osd.7) 27 : cluster [DBG] 2.8 scrub starts 2020-09-08T15:39:14.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:14 smithi166 bash[9446]: cluster 2020-09-08T15:39:13.461161+0000 osd.6 (osd.6) 16 : cluster [DBG] 2.7 scrub ok 2020-09-08T15:39:14.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:14 smithi166 bash[9446]: cluster 2020-09-08T15:39:13.617191+0000 mgr.x (mgr.24118) 351 : cluster [DBG] pgmap v438: 17 pgs: 6 active+clean+scrubbing, 11 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 79 MiB/s wr, 1.26k op/s 2020-09-08T15:39:14.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:14 smithi166 bash[9446]: cluster 2020-09-08T15:39:14.142348+0000 osd.1 (osd.1) 25 : cluster [DBG] 2.6 scrub ok 2020-09-08T15:39:14.910 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:39:14 smithi055 bash[16646]: debug 2020-09-08T15:39:14.849+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:14.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:14 smithi055 bash[9296]: cluster 2020-09-08T15:39:13.355197+0000 osd.11 (osd.11) 14 : cluster [DBG] 2.a scrub ok 2020-09-08T15:39:14.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:14 smithi055 bash[9296]: cluster 2020-09-08T15:39:13.390654+0000 osd.9 (osd.9) 14 : cluster [DBG] 2.d scrub ok 2020-09-08T15:39:14.912 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:14 smithi055 bash[9296]: cluster 2020-09-08T15:39:13.448180+0000 osd.7 (osd.7) 27 : cluster [DBG] 2.8 scrub starts 2020-09-08T15:39:14.912 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:14 smithi055 bash[9296]: cluster 2020-09-08T15:39:13.461161+0000 osd.6 (osd.6) 16 : cluster [DBG] 2.7 scrub ok 2020-09-08T15:39:14.912 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:14 smithi055 bash[9296]: cluster 2020-09-08T15:39:13.617191+0000 mgr.x (mgr.24118) 351 : cluster [DBG] pgmap v438: 17 pgs: 6 active+clean+scrubbing, 11 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 79 MiB/s wr, 1.26k op/s 2020-09-08T15:39:14.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:14 smithi055 bash[9296]: cluster 2020-09-08T15:39:14.142348+0000 osd.1 (osd.1) 25 : cluster [DBG] 2.6 scrub ok 2020-09-08T15:39:14.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:14 smithi104 bash[9203]: cluster 2020-09-08T15:39:13.355197+0000 osd.11 (osd.11) 14 : cluster [DBG] 2.a scrub ok 2020-09-08T15:39:14.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:14 smithi104 bash[9203]: cluster 2020-09-08T15:39:13.390654+0000 osd.9 (osd.9) 14 : cluster [DBG] 2.d scrub ok 2020-09-08T15:39:14.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:14 smithi104 bash[9203]: cluster 2020-09-08T15:39:13.448180+0000 osd.7 (osd.7) 27 : cluster [DBG] 2.8 scrub starts 2020-09-08T15:39:14.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:14 smithi104 bash[9203]: cluster 2020-09-08T15:39:13.461161+0000 osd.6 (osd.6) 16 : cluster [DBG] 2.7 scrub ok 2020-09-08T15:39:14.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:14 smithi104 bash[9203]: cluster 2020-09-08T15:39:13.617191+0000 mgr.x (mgr.24118) 351 : cluster [DBG] pgmap v438: 17 pgs: 6 active+clean+scrubbing, 11 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 79 MiB/s wr, 1.26k op/s 2020-09-08T15:39:14.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:14 smithi104 bash[9203]: cluster 2020-09-08T15:39:14.142348+0000 osd.1 (osd.1) 25 : cluster [DBG] 2.6 scrub ok 2020-09-08T15:39:14.948 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:15.050 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:15.153 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:15.210 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:39:15.386 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:39:15.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:39:15 smithi055 bash[11956]: debug 2020-09-08T15:39:15.053+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:15.555 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:15.657 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:39:15.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:15 smithi104 bash[9203]: cluster 2020-09-08T15:39:15.617760+0000 mgr.x (mgr.24118) 352 : cluster [DBG] pgmap v439: 17 pgs: 6 active+clean+scrubbing, 11 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s wr, 781 op/s 2020-09-08T15:39:15.927 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:39:15 smithi104 bash[22305]: debug 2020-09-08T15:39:15.546+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:16.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:15 smithi166 bash[9446]: cluster 2020-09-08T15:39:15.617760+0000 mgr.x (mgr.24118) 352 : cluster [DBG] pgmap v439: 17 pgs: 6 active+clean+scrubbing, 11 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s wr, 781 op/s 2020-09-08T15:39:16.162 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:15 smithi055 bash[9296]: cluster 2020-09-08T15:39:15.617760+0000 mgr.x (mgr.24118) 352 : cluster [DBG] pgmap v439: 17 pgs: 6 active+clean+scrubbing, 11 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s wr, 781 op/s 2020-09-08T15:39:16.699 INFO:teuthology.orchestra.run.smithi104.stderr:reweighted osd.4 to 1 (10000) 2020-09-08T15:39:17.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:16 smithi166 bash[9446]: cluster 2020-09-08T15:39:15.184885+0000 osd.2 (osd.2) 20 : cluster [DBG] 2.e scrub starts 2020-09-08T15:39:17.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:16 smithi166 bash[9446]: audit 2020-09-08T15:39:15.937687+0000 mon.a (mon.0) 985 : audit [INF] from='client.? v1:172.21.15.104:0/523978110' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2020-09-08T15:39:17.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:16 smithi055 bash[9296]: cluster 2020-09-08T15:39:15.184885+0000 osd.2 (osd.2) 20 : cluster [DBG] 2.e scrub starts 2020-09-08T15:39:17.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:16 smithi055 bash[9296]: audit 2020-09-08T15:39:15.937687+0000 mon.a (mon.0) 985 : audit [INF] from='client.? v1:172.21.15.104:0/523978110' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2020-09-08T15:39:17.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:16 smithi104 bash[9203]: cluster 2020-09-08T15:39:15.184885+0000 osd.2 (osd.2) 20 : cluster [DBG] 2.e scrub starts 2020-09-08T15:39:17.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:16 smithi104 bash[9203]: audit 2020-09-08T15:39:15.937687+0000 mon.a (mon.0) 985 : audit [INF] from='client.? v1:172.21.15.104:0/523978110' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2020-09-08T15:39:17.985 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:17 smithi166 bash[9446]: cluster 2020-09-08T15:39:15.907667+0000 osd.7 (osd.7) 28 : cluster [DBG] 2.8 scrub ok 2020-09-08T15:39:17.986 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:17 smithi166 bash[9446]: cluster 2020-09-08T15:39:16.486145+0000 osd.1 (osd.1) 26 : cluster [DBG] 2.9 scrub starts 2020-09-08T15:39:17.986 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:17 smithi166 bash[9446]: audit 2020-09-08T15:39:16.697112+0000 mon.a (mon.0) 986 : audit [INF] from='client.? v1:172.21.15.104:0/523978110' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2020-09-08T15:39:17.987 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:17 smithi166 bash[9446]: cluster 2020-09-08T15:39:16.697167+0000 mon.a (mon.0) 987 : cluster [DBG] osdmap e166: 12 total, 12 up, 12 in 2020-09-08T15:39:17.987 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:17 smithi166 bash[9446]: cluster 2020-09-08T15:39:17.084401+0000 osd.8 (osd.8) 13 : cluster [DBG] 2.b scrub starts 2020-09-08T15:39:17.987 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:17 smithi166 bash[9446]: cluster 2020-09-08T15:39:17.618277+0000 mgr.x (mgr.24118) 353 : cluster [DBG] pgmap v441: 17 pgs: 5 active+clean+scrubbing, 12 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s wr, 1.04k op/s 2020-09-08T15:39:17.987 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:39:18.090 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:39:18.098 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:17 smithi104 bash[9203]: cluster 2020-09-08T15:39:15.907667+0000 osd.7 (osd.7) 28 : cluster [DBG] 2.8 scrub ok 2020-09-08T15:39:18.098 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:17 smithi104 bash[9203]: cluster 2020-09-08T15:39:16.486145+0000 osd.1 (osd.1) 26 : cluster [DBG] 2.9 scrub starts 2020-09-08T15:39:18.098 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:17 smithi104 bash[9203]: audit 2020-09-08T15:39:16.697112+0000 mon.a (mon.0) 986 : audit [INF] from='client.? v1:172.21.15.104:0/523978110' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2020-09-08T15:39:18.098 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:17 smithi104 bash[9203]: cluster 2020-09-08T15:39:16.697167+0000 mon.a (mon.0) 987 : cluster [DBG] osdmap e166: 12 total, 12 up, 12 in 2020-09-08T15:39:18.099 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:17 smithi104 bash[9203]: cluster 2020-09-08T15:39:17.084401+0000 osd.8 (osd.8) 13 : cluster [DBG] 2.b scrub starts 2020-09-08T15:39:18.099 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:17 smithi104 bash[9203]: cluster 2020-09-08T15:39:17.618277+0000 mgr.x (mgr.24118) 353 : cluster [DBG] pgmap v441: 17 pgs: 5 active+clean+scrubbing, 12 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s wr, 1.04k op/s 2020-09-08T15:39:18.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:17 smithi055 bash[9296]: cluster 2020-09-08T15:39:15.907667+0000 osd.7 (osd.7) 28 : cluster [DBG] 2.8 scrub ok 2020-09-08T15:39:18.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:17 smithi055 bash[9296]: cluster 2020-09-08T15:39:16.486145+0000 osd.1 (osd.1) 26 : cluster [DBG] 2.9 scrub starts 2020-09-08T15:39:18.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:17 smithi055 bash[9296]: audit 2020-09-08T15:39:16.697112+0000 mon.a (mon.0) 986 : audit [INF] from='client.? v1:172.21.15.104:0/523978110' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2020-09-08T15:39:18.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:17 smithi055 bash[9296]: cluster 2020-09-08T15:39:16.697167+0000 mon.a (mon.0) 987 : cluster [DBG] osdmap e166: 12 total, 12 up, 12 in 2020-09-08T15:39:18.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:17 smithi055 bash[9296]: cluster 2020-09-08T15:39:17.084401+0000 osd.8 (osd.8) 13 : cluster [DBG] 2.b scrub starts 2020-09-08T15:39:18.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:17 smithi055 bash[9296]: cluster 2020-09-08T15:39:17.618277+0000 mgr.x (mgr.24118) 353 : cluster [DBG] pgmap v441: 17 pgs: 5 active+clean+scrubbing, 12 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s wr, 1.04k op/s 2020-09-08T15:39:18.365 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:39:17 smithi166 bash[14254]: debug 2020-09-08T15:39:17.987+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:18.412 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:39:18.514 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:18.645 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:18.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:39:18 smithi104 bash[17588]: debug 2020-09-08T15:39:18.410+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:18.750 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:18.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:39:18 smithi055 bash[19003]: debug 2020-09-08T15:39:18.649+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:18.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:18 smithi055 bash[9296]: cluster 2020-09-08T15:39:17.675891+0000 osd.10 (osd.10) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:18.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:18 smithi055 bash[9296]: cluster 2020-09-08T15:39:17.675968+0000 osd.10 (osd.10) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:18.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:18 smithi055 bash[9296]: cluster 2020-09-08T15:39:17.707496+0000 mon.a (mon.0) 988 : cluster [DBG] osdmap e167: 12 total, 12 up, 12 in 2020-09-08T15:39:19.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:18 smithi166 bash[9446]: cluster 2020-09-08T15:39:17.675891+0000 osd.10 (osd.10) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:19.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:18 smithi166 bash[9446]: cluster 2020-09-08T15:39:17.675968+0000 osd.10 (osd.10) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:19.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:18 smithi166 bash[9446]: cluster 2020-09-08T15:39:17.707496+0000 mon.a (mon.0) 988 : cluster [DBG] osdmap e167: 12 total, 12 up, 12 in 2020-09-08T15:39:19.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:18 smithi104 bash[9203]: cluster 2020-09-08T15:39:17.675891+0000 osd.10 (osd.10) 11 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:19.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:18 smithi104 bash[9203]: cluster 2020-09-08T15:39:17.675968+0000 osd.10 (osd.10) 12 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:19.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:18 smithi104 bash[9203]: cluster 2020-09-08T15:39:17.707496+0000 mon.a (mon.0) 988 : cluster [DBG] osdmap e167: 12 total, 12 up, 12 in 2020-09-08T15:39:19.214 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd reweight 5 1 2020-09-08T15:39:19.387 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:39:19.407 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:19.482 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:39:19 smithi104 bash[22305]: debug 2020-09-08T15:39:19.406+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:19.519 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:19.632 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:19.736 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:19.840 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:39:19 smithi055 bash[19003]: debug 2020-09-08T15:39:19.841+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:19.841 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:39:19 smithi055 bash[16646]: debug 2020-09-08T15:39:19.629+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:19.841 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:19 smithi055 bash[9296]: cluster 2020-09-08T15:39:18.552714+0000 osd.2 (osd.2) 21 : cluster [DBG] 2.e scrub ok 2020-09-08T15:39:19.842 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:19 smithi055 bash[9296]: cluster 2020-09-08T15:39:19.618781+0000 mgr.x (mgr.24118) 354 : cluster [DBG] pgmap v443: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 145 B/s rd, 18 MiB/s wr, 768 op/s 2020-09-08T15:39:19.842 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:19.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:19 smithi104 bash[9203]: cluster 2020-09-08T15:39:18.552714+0000 osd.2 (osd.2) 21 : cluster [DBG] 2.e scrub ok 2020-09-08T15:39:19.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:19 smithi104 bash[9203]: cluster 2020-09-08T15:39:19.618781+0000 mgr.x (mgr.24118) 354 : cluster [DBG] pgmap v443: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 145 B/s rd, 18 MiB/s wr, 768 op/s 2020-09-08T15:39:19.944 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:20.061 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:20.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:19 smithi166 bash[9446]: cluster 2020-09-08T15:39:18.552714+0000 osd.2 (osd.2) 21 : cluster [DBG] 2.e scrub ok 2020-09-08T15:39:20.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:19 smithi166 bash[9446]: cluster 2020-09-08T15:39:19.618781+0000 mgr.x (mgr.24118) 354 : cluster [DBG] pgmap v443: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 145 B/s rd, 18 MiB/s wr, 768 op/s 2020-09-08T15:39:20.156 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:39:20 smithi055 bash[14301]: debug 2020-09-08T15:39:20.065+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:20.165 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:39:20.298 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:39:20.402 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:20.615 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:39:20 smithi166 bash[16593]: debug 2020-09-08T15:39:20.299+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:20.645 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:20.750 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:20.856 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:20.927 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:39:20 smithi104 bash[22305]: debug 2020-09-08T15:39:20.642+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:20.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:20 smithi104 bash[9203]: cluster 2020-09-08T15:39:19.264669+0000 osd.8 (osd.8) 14 : cluster [DBG] 2.b scrub ok 2020-09-08T15:39:20.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:20 smithi104 bash[9203]: cluster 2020-09-08T15:39:19.532265+0000 osd.1 (osd.1) 27 : cluster [DBG] 2.9 scrub ok 2020-09-08T15:39:20.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:20 smithi104 bash[9203]: cluster 2020-09-08T15:39:19.729226+0000 osd.5 (osd.5) 19 : cluster [DBG] 2.2 scrub starts 2020-09-08T15:39:20.960 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:39:20.968 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:20 smithi166 bash[9446]: cluster 2020-09-08T15:39:19.264669+0000 osd.8 (osd.8) 14 : cluster [DBG] 2.b scrub ok 2020-09-08T15:39:20.968 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:20 smithi166 bash[9446]: cluster 2020-09-08T15:39:19.532265+0000 osd.1 (osd.1) 27 : cluster [DBG] 2.9 scrub ok 2020-09-08T15:39:20.969 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:20 smithi166 bash[9446]: cluster 2020-09-08T15:39:19.729226+0000 osd.5 (osd.5) 19 : cluster [DBG] 2.2 scrub starts 2020-09-08T15:39:21.085 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:39:21.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:39:20 smithi055 bash[19003]: debug 2020-09-08T15:39:20.856+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:21.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:20 smithi055 bash[9296]: cluster 2020-09-08T15:39:19.264669+0000 osd.8 (osd.8) 14 : cluster [DBG] 2.b scrub ok 2020-09-08T15:39:21.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:20 smithi055 bash[9296]: cluster 2020-09-08T15:39:19.532265+0000 osd.1 (osd.1) 27 : cluster [DBG] 2.9 scrub ok 2020-09-08T15:39:21.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:20 smithi055 bash[9296]: cluster 2020-09-08T15:39:19.729226+0000 osd.5 (osd.5) 19 : cluster [DBG] 2.2 scrub starts 2020-09-08T15:39:21.190 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:39:21.290 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:39:21 smithi166 bash[16593]: debug 2020-09-08T15:39:21.079+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:21.291 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:39:21.397 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:21.614 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:39:21 smithi166 bash[16593]: debug 2020-09-08T15:39:21.291+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:21.847 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:21.949 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:22.047 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:39:22.047 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:39:22.079 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:21 smithi055 bash[9296]: cluster 2020-09-08T15:39:20.292970+0000 osd.3 (osd.3) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:22.080 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:21 smithi055 bash[9296]: cluster 2020-09-08T15:39:20.293051+0000 osd.3 (osd.3) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:22.080 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:21 smithi055 bash[9296]: cluster 2020-09-08T15:39:20.720523+0000 mon.a (mon.0) 989 : cluster [DBG] osdmap e168: 12 total, 12 up, 12 in 2020-09-08T15:39:22.080 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:21 smithi055 bash[9296]: cluster 2020-09-08T15:39:21.619285+0000 mgr.x (mgr.24118) 355 : cluster [DBG] pgmap v445: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 7.7 MiB/s wr, 680 op/s 2020-09-08T15:39:22.081 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:22.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:21 smithi166 bash[9446]: cluster 2020-09-08T15:39:20.292970+0000 osd.3 (osd.3) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:22.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:21 smithi166 bash[9446]: cluster 2020-09-08T15:39:20.293051+0000 osd.3 (osd.3) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:22.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:21 smithi166 bash[9446]: cluster 2020-09-08T15:39:20.720523+0000 mon.a (mon.0) 989 : cluster [DBG] osdmap e168: 12 total, 12 up, 12 in 2020-09-08T15:39:22.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:21 smithi166 bash[9446]: cluster 2020-09-08T15:39:21.619285+0000 mgr.x (mgr.24118) 355 : cluster [DBG] pgmap v445: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 7.7 MiB/s wr, 680 op/s 2020-09-08T15:39:22.177 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:39:21 smithi104 bash[22305]: debug 2020-09-08T15:39:21.850+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:22.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:21 smithi104 bash[9203]: cluster 2020-09-08T15:39:20.292970+0000 osd.3 (osd.3) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:22.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:21 smithi104 bash[9203]: cluster 2020-09-08T15:39:20.293051+0000 osd.3 (osd.3) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:22.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:21 smithi104 bash[9203]: cluster 2020-09-08T15:39:20.720523+0000 mon.a (mon.0) 989 : cluster [DBG] osdmap e168: 12 total, 12 up, 12 in 2020-09-08T15:39:22.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:21 smithi104 bash[9203]: cluster 2020-09-08T15:39:21.619285+0000 mgr.x (mgr.24118) 355 : cluster [DBG] pgmap v445: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 7.7 MiB/s wr, 680 op/s 2020-09-08T15:39:22.184 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:22.300 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:22.404 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:39:22.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:39:22 smithi055 bash[14301]: debug 2020-09-08T15:39:22.080+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:22.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:39:22 smithi055 bash[11956]: debug 2020-09-08T15:39:22.300+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:22.719 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:39:22.823 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:39:23.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:22 smithi166 bash[9446]: cluster 2020-09-08T15:39:21.176865+0000 osd.2 (osd.2) 22 : cluster [DBG] 2.1 scrub starts 2020-09-08T15:39:23.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:22 smithi055 bash[9296]: cluster 2020-09-08T15:39:21.176865+0000 osd.2 (osd.2) 22 : cluster [DBG] 2.1 scrub starts 2020-09-08T15:39:23.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:22 smithi104 bash[9203]: cluster 2020-09-08T15:39:21.176865+0000 osd.2 (osd.2) 22 : cluster [DBG] 2.1 scrub starts 2020-09-08T15:39:23.178 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:39:22 smithi104 bash[15273]: debug 2020-09-08T15:39:22.718+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:24.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:23 smithi166 bash[9446]: cluster 2020-09-08T15:39:22.287337+0000 osd.5 (osd.5) 20 : cluster [DBG] 2.2 scrub ok 2020-09-08T15:39:24.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:23 smithi166 bash[9446]: cluster 2020-09-08T15:39:23.619908+0000 mgr.x (mgr.24118) 356 : cluster [DBG] pgmap v446: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 148 B/s rd, 3.2 MiB/s wr, 1.12k op/s 2020-09-08T15:39:24.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:23 smithi055 bash[9296]: cluster 2020-09-08T15:39:22.287337+0000 osd.5 (osd.5) 20 : cluster [DBG] 2.2 scrub ok 2020-09-08T15:39:24.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:23 smithi055 bash[9296]: cluster 2020-09-08T15:39:23.619908+0000 mgr.x (mgr.24118) 356 : cluster [DBG] pgmap v446: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 148 B/s rd, 3.2 MiB/s wr, 1.12k op/s 2020-09-08T15:39:24.160 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:39:24.161 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:23 smithi104 bash[9203]: cluster 2020-09-08T15:39:22.287337+0000 osd.5 (osd.5) 20 : cluster [DBG] 2.2 scrub ok 2020-09-08T15:39:24.162 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:23 smithi104 bash[9203]: cluster 2020-09-08T15:39:23.619908+0000 mgr.x (mgr.24118) 356 : cluster [DBG] pgmap v446: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 148 B/s rd, 3.2 MiB/s wr, 1.12k op/s 2020-09-08T15:39:24.162 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:39:24 smithi104 bash[17588]: debug 2020-09-08T15:39:24.158+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:24.264 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:39:24.395 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:39:24.498 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:24.615 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:39:24 smithi166 bash[18960]: debug 2020-09-08T15:39:24.395+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:24.623 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:24.728 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:24.850 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:24.956 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:25.061 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:24 smithi055 bash[9296]: cluster 2020-09-08T15:39:23.528434+0000 osd.1 (osd.1) 28 : cluster [DBG] 2.4 scrub starts 2020-09-08T15:39:25.062 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:39:24 smithi055 bash[14301]: debug 2020-09-08T15:39:24.852+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:25.062 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:39:25 smithi055 bash[14301]: debug 2020-09-08T15:39:25.060+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:25.063 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:25.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:24 smithi166 bash[9446]: cluster 2020-09-08T15:39:23.528434+0000 osd.1 (osd.1) 28 : cluster [DBG] 2.4 scrub starts 2020-09-08T15:39:25.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:39:24 smithi166 bash[11928]: debug 2020-09-08T15:39:24.623+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:25.165 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:25.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:24 smithi104 bash[9203]: cluster 2020-09-08T15:39:23.528434+0000 osd.1 (osd.1) 28 : cluster [DBG] 2.4 scrub starts 2020-09-08T15:39:25.277 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:25.390 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:25.392 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2020-09-08T15:39:25.403 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:39:25 smithi055 bash[16646]: debug 2020-09-08T15:39:25.280+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:25.504 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:25.559 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:39:25.611 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:39:25.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:39:25 smithi055 bash[14301]: debug 2020-09-08T15:39:25.504+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:25.731 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:39:25.848 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:25.954 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:26.056 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:39:26.065 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:25 smithi166 bash[9446]: cluster 2020-09-08T15:39:24.190361+0000 osd.8 (osd.8) 15 : cluster [DBG] 2.f scrub starts 2020-09-08T15:39:26.065 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:25 smithi166 bash[9446]: cluster 2020-09-08T15:39:24.331709+0000 osd.2 (osd.2) 23 : cluster [DBG] 2.1 scrub ok 2020-09-08T15:39:26.065 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:25 smithi166 bash[9446]: cluster 2020-09-08T15:39:25.620459+0000 mgr.x (mgr.24118) 357 : cluster [DBG] pgmap v447: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 112 KiB/s wr, 808 op/s 2020-09-08T15:39:26.066 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:39:25 smithi166 bash[18960]: debug 2020-09-08T15:39:25.731+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:26.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:39:25 smithi055 bash[19003]: debug 2020-09-08T15:39:25.940+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:26.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:25 smithi055 bash[9296]: cluster 2020-09-08T15:39:24.190361+0000 osd.8 (osd.8) 15 : cluster [DBG] 2.f scrub starts 2020-09-08T15:39:26.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:25 smithi055 bash[9296]: cluster 2020-09-08T15:39:24.331709+0000 osd.2 (osd.2) 23 : cluster [DBG] 2.1 scrub ok 2020-09-08T15:39:26.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:25 smithi055 bash[9296]: cluster 2020-09-08T15:39:25.620459+0000 mgr.x (mgr.24118) 357 : cluster [DBG] pgmap v447: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 112 KiB/s wr, 808 op/s 2020-09-08T15:39:26.165 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:39:26.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:25 smithi104 bash[9203]: cluster 2020-09-08T15:39:24.190361+0000 osd.8 (osd.8) 15 : cluster [DBG] 2.f scrub starts 2020-09-08T15:39:26.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:25 smithi104 bash[9203]: cluster 2020-09-08T15:39:24.331709+0000 osd.2 (osd.2) 23 : cluster [DBG] 2.1 scrub ok 2020-09-08T15:39:26.185 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:25 smithi104 bash[9203]: cluster 2020-09-08T15:39:25.620459+0000 mgr.x (mgr.24118) 357 : cluster [DBG] pgmap v447: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 112 KiB/s wr, 808 op/s 2020-09-08T15:39:26.269 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:39:26.365 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:39:26 smithi166 bash[18960]: debug 2020-09-08T15:39:26.163+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:26.387 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:39:26.494 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:39:26.619 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:39:26.724 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:26.830 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:26.865 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:39:26 smithi166 bash[18960]: debug 2020-09-08T15:39:26.387+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:26.865 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:39:26 smithi166 bash[18960]: debug 2020-09-08T15:39:26.623+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:26.933 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:39:27.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:39:26 smithi055 bash[11956]: debug 2020-09-08T15:39:26.832+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:27.204 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:39:27.204 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:39:27 smithi104 bash[17588]: debug 2020-09-08T15:39:27.201+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:27.305 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:27.421 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:27.528 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:27.615 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:39:27 smithi166 bash[11928]: debug 2020-09-08T15:39:27.423+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:27.640 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:27.743 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:39:27.849 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:39:27.906 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:39:27 smithi055 bash[11956]: debug 2020-09-08T15:39:27.640+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:27.952 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:28.058 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:39:28 smithi166 bash[11928]: debug 2020-09-08T15:39:28.059+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:28.059 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:39:27 smithi166 bash[14254]: debug 2020-09-08T15:39:27.851+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:28.060 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:28.164 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:39:28.291 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:39:28.347 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:39:28.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:28 smithi166 bash[9446]: cluster 2020-09-08T15:39:26.488525+0000 osd.1 (osd.1) 29 : cluster [DBG] 2.4 scrub ok 2020-09-08T15:39:28.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:28 smithi166 bash[9446]: cluster 2020-09-08T15:39:26.660644+0000 osd.8 (osd.8) 16 : cluster [DBG] 2.f scrub ok 2020-09-08T15:39:28.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:28 smithi166 bash[9446]: cluster 2020-09-08T15:39:27.036378+0000 osd.8 (osd.8) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:28.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:28 smithi166 bash[9446]: cluster 2020-09-08T15:39:27.036468+0000 osd.8 (osd.8) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:28.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:28 smithi166 bash[9446]: cluster 2020-09-08T15:39:27.091860+0000 mon.a (mon.0) 990 : cluster [DBG] osdmap e169: 12 total, 12 up, 12 in 2020-09-08T15:39:28.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:28 smithi166 bash[9446]: cluster 2020-09-08T15:39:27.620977+0000 mgr.x (mgr.24118) 358 : cluster [DBG] pgmap v449: 17 pgs: 17 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.07k op/s 2020-09-08T15:39:28.386 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:39:28.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:28 smithi055 bash[9296]: cluster 2020-09-08T15:39:26.488525+0000 osd.1 (osd.1) 29 : cluster [DBG] 2.4 scrub ok 2020-09-08T15:39:28.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:28 smithi055 bash[9296]: cluster 2020-09-08T15:39:26.660644+0000 osd.8 (osd.8) 16 : cluster [DBG] 2.f scrub ok 2020-09-08T15:39:28.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:28 smithi055 bash[9296]: cluster 2020-09-08T15:39:27.036378+0000 osd.8 (osd.8) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:28.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:28 smithi055 bash[9296]: cluster 2020-09-08T15:39:27.036468+0000 osd.8 (osd.8) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:28.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:28 smithi055 bash[9296]: cluster 2020-09-08T15:39:27.091860+0000 mon.a (mon.0) 990 : cluster [DBG] osdmap e169: 12 total, 12 up, 12 in 2020-09-08T15:39:28.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:28 smithi055 bash[9296]: cluster 2020-09-08T15:39:27.620977+0000 mgr.x (mgr.24118) 358 : cluster [DBG] pgmap v449: 17 pgs: 17 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.07k op/s 2020-09-08T15:39:28.418 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:39:28.438 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:39:28.475 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:39:28.514 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:39:28.521 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:39:28.534 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:28 smithi104 bash[9203]: cluster 2020-09-08T15:39:26.488525+0000 osd.1 (osd.1) 29 : cluster [DBG] 2.4 scrub ok 2020-09-08T15:39:28.534 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:28 smithi104 bash[9203]: cluster 2020-09-08T15:39:26.660644+0000 osd.8 (osd.8) 16 : cluster [DBG] 2.f scrub ok 2020-09-08T15:39:28.534 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:28 smithi104 bash[9203]: cluster 2020-09-08T15:39:27.036378+0000 osd.8 (osd.8) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:28.534 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:28 smithi104 bash[9203]: cluster 2020-09-08T15:39:27.036468+0000 osd.8 (osd.8) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:28.535 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:28 smithi104 bash[9203]: cluster 2020-09-08T15:39:27.091860+0000 mon.a (mon.0) 990 : cluster [DBG] osdmap e169: 12 total, 12 up, 12 in 2020-09-08T15:39:28.535 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:28 smithi104 bash[9203]: cluster 2020-09-08T15:39:27.620977+0000 mgr.x (mgr.24118) 358 : cluster [DBG] pgmap v449: 17 pgs: 17 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.07k op/s 2020-09-08T15:39:28.535 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:39:28 smithi104 bash[15273]: debug 2020-09-08T15:39:28.417+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:28.559 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:39:28.599 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:39:28.647 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:39:28.650 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:39:28.681 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:39:28.718 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:39:28.751 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:39:28.754 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:39:28.755 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:39:28.755 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:39:28.756 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:39:28.756 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:39:28.756 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:39:28.756 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:39:28.756 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:39:28.757 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:39:28.757 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:39:28.757 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:39:28.757 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:39:28.759 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:39:28.864 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:39:28 smithi166 bash[14254]: debug 2020-09-08T15:39:28.647+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:28.864 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:39:28 smithi166 bash[14254]: debug 2020-09-08T15:39:28.863+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:28.865 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:39:28.976 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:29.092 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:29.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:29 smithi166 bash[9446]: cluster 2020-09-08T15:39:28.203358+0000 osd.8 (osd.8) 19 : cluster [DBG] 2.5 scrub starts 2020-09-08T15:39:29.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:29 smithi166 bash[9446]: audit 2020-09-08T15:39:28.258195+0000 mon.a (mon.0) 991 : audit [DBG] from='client.? v1:172.21.15.104:0/2893387924' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:39:29.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:29 smithi166 bash[9446]: audit 2020-09-08T15:39:28.587260+0000 mon.a (mon.0) 992 : audit [INF] from='client.? v1:172.21.15.104:0/600670540' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:39:29.116 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:39:29.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:29 smithi104 bash[9203]: cluster 2020-09-08T15:39:28.203358+0000 osd.8 (osd.8) 19 : cluster [DBG] 2.5 scrub starts 2020-09-08T15:39:29.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:29 smithi104 bash[9203]: audit 2020-09-08T15:39:28.258195+0000 mon.a (mon.0) 991 : audit [DBG] from='client.? v1:172.21.15.104:0/2893387924' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:39:29.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:29 smithi104 bash[9203]: audit 2020-09-08T15:39:28.587260+0000 mon.a (mon.0) 992 : audit [INF] from='client.? v1:172.21.15.104:0/600670540' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:39:29.197 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:39:29.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:39:29 smithi055 bash[19003]: debug 2020-09-08T15:39:29.092+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:29.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:29 smithi055 bash[9296]: cluster 2020-09-08T15:39:28.203358+0000 osd.8 (osd.8) 19 : cluster [DBG] 2.5 scrub starts 2020-09-08T15:39:29.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:29 smithi055 bash[9296]: audit 2020-09-08T15:39:28.258195+0000 mon.a (mon.0) 991 : audit [DBG] from='client.? v1:172.21.15.104:0/2893387924' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:39:29.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:29 smithi055 bash[9296]: audit 2020-09-08T15:39:28.587260+0000 mon.a (mon.0) 992 : audit [INF] from='client.? v1:172.21.15.104:0/600670540' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:39:29.429 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:39:29.533 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:29.645 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:29.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:39:29 smithi104 bash[19922]: debug 2020-09-08T15:39:29.425+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:29.748 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:29.906 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:29.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:39:29 smithi055 bash[19003]: debug 2020-09-08T15:39:29.648+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:30.010 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:39:30.098 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:39:29 smithi166 bash[11928]: debug 2020-09-08T15:39:29.906+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:30.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:30 smithi166 bash[9446]: cluster 2020-09-08T15:39:29.099371+0000 mon.a (mon.0) 993 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:39:30.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:30 smithi166 bash[9446]: cluster 2020-09-08T15:39:29.099408+0000 mon.a (mon.0) 994 : cluster [INF] Cluster is now healthy 2020-09-08T15:39:30.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:30 smithi166 bash[9446]: audit 2020-09-08T15:39:29.111015+0000 mon.a (mon.0) 995 : audit [INF] from='client.? v1:172.21.15.104:0/600670540' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:39:30.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:30 smithi166 bash[9446]: cluster 2020-09-08T15:39:29.111058+0000 mon.a (mon.0) 996 : cluster [DBG] osdmap e170: 12 total, 12 up, 12 in 2020-09-08T15:39:30.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:30 smithi166 bash[9446]: cluster 2020-09-08T15:39:29.621661+0000 mgr.x (mgr.24118) 359 : cluster [DBG] pgmap v451: 17 pgs: 17 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.27k op/s 2020-09-08T15:39:30.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:30 smithi055 bash[9296]: cluster 2020-09-08T15:39:29.099371+0000 mon.a (mon.0) 993 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:39:30.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:30 smithi055 bash[9296]: cluster 2020-09-08T15:39:29.099408+0000 mon.a (mon.0) 994 : cluster [INF] Cluster is now healthy 2020-09-08T15:39:30.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:30 smithi055 bash[9296]: audit 2020-09-08T15:39:29.111015+0000 mon.a (mon.0) 995 : audit [INF] from='client.? v1:172.21.15.104:0/600670540' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:39:30.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:30 smithi055 bash[9296]: cluster 2020-09-08T15:39:29.111058+0000 mon.a (mon.0) 996 : cluster [DBG] osdmap e170: 12 total, 12 up, 12 in 2020-09-08T15:39:30.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:30 smithi055 bash[9296]: cluster 2020-09-08T15:39:29.621661+0000 mgr.x (mgr.24118) 359 : cluster [DBG] pgmap v451: 17 pgs: 17 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.27k op/s 2020-09-08T15:39:30.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:30 smithi104 bash[9203]: cluster 2020-09-08T15:39:29.099371+0000 mon.a (mon.0) 993 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:39:30.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:30 smithi104 bash[9203]: cluster 2020-09-08T15:39:29.099408+0000 mon.a (mon.0) 994 : cluster [INF] Cluster is now healthy 2020-09-08T15:39:30.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:30 smithi104 bash[9203]: audit 2020-09-08T15:39:29.111015+0000 mon.a (mon.0) 995 : audit [INF] from='client.? v1:172.21.15.104:0/600670540' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:39:30.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:30 smithi104 bash[9203]: cluster 2020-09-08T15:39:29.111058+0000 mon.a (mon.0) 996 : cluster [DBG] osdmap e170: 12 total, 12 up, 12 in 2020-09-08T15:39:30.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:30 smithi104 bash[9203]: cluster 2020-09-08T15:39:29.621661+0000 mgr.x (mgr.24118) 359 : cluster [DBG] pgmap v451: 17 pgs: 17 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.27k op/s 2020-09-08T15:39:30.665 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:39:30.769 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:30.896 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:30.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:39:30 smithi104 bash[19922]: debug 2020-09-08T15:39:30.661+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:31.002 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:31.110 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:39:30 smithi055 bash[19003]: debug 2020-09-08T15:39:30.896+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:31.111 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:39:31 smithi055 bash[16646]: debug 2020-09-08T15:39:31.112+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:31.111 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:31.214 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:39:31.432 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:39:31.538 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:31.660 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:31.680 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:39:31 smithi104 bash[17588]: debug 2020-09-08T15:39:31.433+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:31.766 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:39:31.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:39:31 smithi055 bash[16646]: debug 2020-09-08T15:39:31.660+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:31.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:31 smithi055 bash[9296]: cluster 2020-09-08T15:39:31.622117+0000 mgr.x (mgr.24118) 360 : cluster [DBG] pgmap v452: 17 pgs: 17 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 622 op/s 2020-09-08T15:39:32.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:31 smithi166 bash[9446]: cluster 2020-09-08T15:39:31.622117+0000 mgr.x (mgr.24118) 360 : cluster [DBG] pgmap v452: 17 pgs: 17 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 622 op/s 2020-09-08T15:39:32.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:31 smithi104 bash[9203]: cluster 2020-09-08T15:39:31.622117+0000 mgr.x (mgr.24118) 360 : cluster [DBG] pgmap v452: 17 pgs: 17 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 622 op/s 2020-09-08T15:39:32.278 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:39:32.383 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:32.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:39:32 smithi104 bash[19922]: debug 2020-09-08T15:39:32.273+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:32.701 INFO:teuthology.orchestra.run.smithi104.stderr:reweighted osd.5 to 1 (10000) 2020-09-08T15:39:32.832 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:32.941 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:32.948 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:32 smithi055 bash[9296]: cluster 2020-09-08T15:39:31.634019+0000 osd.8 (osd.8) 20 : cluster [DBG] 2.5 scrub ok 2020-09-08T15:39:32.949 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:32 smithi055 bash[9296]: audit 2020-09-08T15:39:32.033205+0000 mon.a (mon.0) 997 : audit [INF] from='client.24853 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2020-09-08T15:39:32.949 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:32 smithi055 bash[9296]: audit 2020-09-08T15:39:32.033226+0000 mon.b (mon.1) 77 : audit [INF] from='client.? v1:172.21.15.104:0/972434960' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2020-09-08T15:39:32.975 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:39:32.986 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:39:32 smithi104 bash[22305]: debug 2020-09-08T15:39:32.829+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:32.987 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:32 smithi104 bash[9203]: cluster 2020-09-08T15:39:31.634019+0000 osd.8 (osd.8) 20 : cluster [DBG] 2.5 scrub ok 2020-09-08T15:39:32.987 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:32 smithi104 bash[9203]: audit 2020-09-08T15:39:32.033205+0000 mon.a (mon.0) 997 : audit [INF] from='client.24853 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2020-09-08T15:39:32.987 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:32 smithi104 bash[9203]: audit 2020-09-08T15:39:32.033226+0000 mon.b (mon.1) 77 : audit [INF] from='client.? v1:172.21.15.104:0/972434960' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2020-09-08T15:39:33.050 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:33.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:32 smithi166 bash[9446]: cluster 2020-09-08T15:39:31.634019+0000 osd.8 (osd.8) 20 : cluster [DBG] 2.5 scrub ok 2020-09-08T15:39:33.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:32 smithi166 bash[9446]: audit 2020-09-08T15:39:32.033205+0000 mon.a (mon.0) 997 : audit [INF] from='client.24853 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2020-09-08T15:39:33.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:32 smithi166 bash[9446]: audit 2020-09-08T15:39:32.033226+0000 mon.b (mon.1) 77 : audit [INF] from='client.? v1:172.21.15.104:0/972434960' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2020-09-08T15:39:33.130 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:39:33.159 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:39:33.315 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:39:33.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:39:33 smithi055 bash[16646]: debug 2020-09-08T15:39:33.052+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:33.420 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:33.451 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:39:33.615 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:39:33 smithi166 bash[16593]: debug 2020-09-08T15:39:33.318+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:33.634 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:39:34.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:33 smithi166 bash[9446]: audit 2020-09-08T15:39:32.698785+0000 mon.a (mon.0) 998 : audit [INF] from='client.24853 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2020-09-08T15:39:34.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:33 smithi166 bash[9446]: cluster 2020-09-08T15:39:32.698829+0000 mon.a (mon.0) 999 : cluster [DBG] osdmap e171: 12 total, 12 up, 12 in 2020-09-08T15:39:34.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:33 smithi166 bash[9446]: cluster 2020-09-08T15:39:33.209299+0000 osd.1 (osd.1) 30 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:34.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:33 smithi166 bash[9446]: cluster 2020-09-08T15:39:33.209412+0000 osd.1 (osd.1) 31 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:34.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:33 smithi166 bash[9446]: cluster 2020-09-08T15:39:33.622698+0000 mgr.x (mgr.24118) 361 : cluster [DBG] pgmap v454: 17 pgs: 17 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 991 op/s 2020-09-08T15:39:34.139 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:33 smithi104 bash[9203]: audit 2020-09-08T15:39:32.698785+0000 mon.a (mon.0) 998 : audit [INF] from='client.24853 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2020-09-08T15:39:34.139 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:33 smithi104 bash[9203]: cluster 2020-09-08T15:39:32.698829+0000 mon.a (mon.0) 999 : cluster [DBG] osdmap e171: 12 total, 12 up, 12 in 2020-09-08T15:39:34.139 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:33 smithi104 bash[9203]: cluster 2020-09-08T15:39:33.209299+0000 osd.1 (osd.1) 30 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:34.140 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:33 smithi104 bash[9203]: cluster 2020-09-08T15:39:33.209412+0000 osd.1 (osd.1) 31 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:34.140 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:33 smithi104 bash[9203]: cluster 2020-09-08T15:39:33.622698+0000 mgr.x (mgr.24118) 361 : cluster [DBG] pgmap v454: 17 pgs: 17 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 991 op/s 2020-09-08T15:39:34.144 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:34.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:33 smithi055 bash[9296]: audit 2020-09-08T15:39:32.698785+0000 mon.a (mon.0) 998 : audit [INF] from='client.24853 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2020-09-08T15:39:34.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:33 smithi055 bash[9296]: cluster 2020-09-08T15:39:32.698829+0000 mon.a (mon.0) 999 : cluster [DBG] osdmap e171: 12 total, 12 up, 12 in 2020-09-08T15:39:34.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:33 smithi055 bash[9296]: cluster 2020-09-08T15:39:33.209299+0000 osd.1 (osd.1) 30 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:34.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:33 smithi055 bash[9296]: cluster 2020-09-08T15:39:33.209412+0000 osd.1 (osd.1) 31 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:34.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:33 smithi055 bash[9296]: cluster 2020-09-08T15:39:33.622698+0000 mgr.x (mgr.24118) 361 : cluster [DBG] pgmap v454: 17 pgs: 17 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 991 op/s 2020-09-08T15:39:34.246 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:34.427 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:39:34 smithi104 bash[22305]: debug 2020-09-08T15:39:34.141+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:34.673 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:39:34.674 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T15:39:34.674 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2020-09-08T15:39:34.674 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2020-09-08T15:39:34.674 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:39:34.819 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd reweight 6 1 2020-09-08T15:39:34.987 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:39:34.988 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:34 smithi104 bash[9203]: cluster 2020-09-08T15:39:33.708545+0000 mon.a (mon.0) 1000 : cluster [DBG] osdmap e172: 12 total, 12 up, 12 in 2020-09-08T15:39:35.022 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:35 smithi055 bash[9296]: cluster 2020-09-08T15:39:33.708545+0000 mon.a (mon.0) 1000 : cluster [DBG] osdmap e172: 12 total, 12 up, 12 in 2020-09-08T15:39:35.049 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:35.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:34 smithi166 bash[9446]: cluster 2020-09-08T15:39:33.708545+0000 mon.a (mon.0) 1000 : cluster [DBG] osdmap e172: 12 total, 12 up, 12 in 2020-09-08T15:39:35.154 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:35.269 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:35.373 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:39:35.411 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:39:35 smithi055 bash[16646]: debug 2020-09-08T15:39:35.052+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:35.411 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:39:35 smithi055 bash[14301]: debug 2020-09-08T15:39:35.272+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:36.064 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:35 smithi104 bash[9203]: cluster 2020-09-08T15:39:35.623117+0000 mgr.x (mgr.24118) 362 : cluster [DBG] pgmap v456: 17 pgs: 17 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 751 op/s 2020-09-08T15:39:36.065 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:39:36.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:35 smithi166 bash[9446]: cluster 2020-09-08T15:39:35.623117+0000 mgr.x (mgr.24118) 362 : cluster [DBG] pgmap v456: 17 pgs: 17 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 751 op/s 2020-09-08T15:39:36.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:35 smithi055 bash[9296]: cluster 2020-09-08T15:39:35.623117+0000 mgr.x (mgr.24118) 362 : cluster [DBG] pgmap v456: 17 pgs: 17 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 751 op/s 2020-09-08T15:39:36.166 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:36.296 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:36.399 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:36.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:39:36 smithi104 bash[17588]: debug 2020-09-08T15:39:36.049+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:36.493 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:39:36 smithi055 bash[19003]: debug 2020-09-08T15:39:36.296+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:36.496 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:36.599 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:36.693 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:36.810 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:36.822 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:39:36 smithi055 bash[16646]: debug 2020-09-08T15:39:36.492+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:36.823 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:39:36 smithi055 bash[16646]: debug 2020-09-08T15:39:36.696+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:36.922 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:37.029 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:39:37.090 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2020-09-08T15:39:37.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:39:36 smithi055 bash[16646]: debug 2020-09-08T15:39:36.924+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:37.202 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:39:37.255 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:39:37.306 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:39:37.365 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:39:37 smithi166 bash[14254]: debug 2020-09-08T15:39:37.202+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:37.954 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:37 smithi104 bash[9203]: cluster 2020-09-08T15:39:37.623635+0000 mgr.x (mgr.24118) 363 : cluster [DBG] pgmap v457: 17 pgs: 17 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T15:39:37.989 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:39:38.092 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:39:38.099 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:37 smithi166 bash[9446]: cluster 2020-09-08T15:39:37.623635+0000 mgr.x (mgr.24118) 363 : cluster [DBG] pgmap v457: 17 pgs: 17 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T15:39:38.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:37 smithi055 bash[9296]: cluster 2020-09-08T15:39:37.623635+0000 mgr.x (mgr.24118) 363 : cluster [DBG] pgmap v457: 17 pgs: 17 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T15:39:38.310 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:39:38.365 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:39:38 smithi166 bash[14254]: debug 2020-09-08T15:39:38.306+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:38.417 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:39:38.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:39:37 smithi104 bash[19922]: debug 2020-09-08T15:39:37.985+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:38.535 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:39:38.641 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:39:38.759 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:39:38 smithi166 bash[18960]: debug 2020-09-08T15:39:38.538+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:38.759 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:39:38 smithi166 bash[18960]: debug 2020-09-08T15:39:38.758+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:38.760 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:39:38.863 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:38.998 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:39.104 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:39.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:39:39 smithi055 bash[19003]: debug 2020-09-08T15:39:39.000+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:39.234 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:39.335 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:39.365 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:39:39 smithi166 bash[11928]: debug 2020-09-08T15:39:39.226+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:39.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:39 smithi166 bash[9446]: cluster 2020-09-08T15:39:39.247369+0000 osd.6 (osd.6) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:39.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:39 smithi166 bash[9446]: cluster 2020-09-08T15:39:39.247480+0000 osd.6 (osd.6) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:39.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:39 smithi166 bash[9446]: cluster 2020-09-08T15:39:39.306736+0000 mon.a (mon.0) 1001 : cluster [DBG] osdmap e173: 12 total, 12 up, 12 in 2020-09-08T15:39:39.471 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:39.574 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:39.584 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:39 smithi104 bash[9203]: cluster 2020-09-08T15:39:39.247369+0000 osd.6 (osd.6) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:39.584 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:39 smithi104 bash[9203]: cluster 2020-09-08T15:39:39.247480+0000 osd.6 (osd.6) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:39.584 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:39 smithi104 bash[9203]: cluster 2020-09-08T15:39:39.306736+0000 mon.a (mon.0) 1001 : cluster [DBG] osdmap e173: 12 total, 12 up, 12 in 2020-09-08T15:39:39.656 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:39:39 smithi055 bash[14301]: debug 2020-09-08T15:39:39.472+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:39.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:39 smithi055 bash[9296]: cluster 2020-09-08T15:39:39.247369+0000 osd.6 (osd.6) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:39.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:39 smithi055 bash[9296]: cluster 2020-09-08T15:39:39.247480+0000 osd.6 (osd.6) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:39.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:39 smithi055 bash[9296]: cluster 2020-09-08T15:39:39.306736+0000 mon.a (mon.0) 1001 : cluster [DBG] osdmap e173: 12 total, 12 up, 12 in 2020-09-08T15:39:40.026 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:40.131 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:40.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:39:40 smithi104 bash[22305]: debug 2020-09-08T15:39:40.025+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:40.840 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:40.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:40 smithi166 bash[9446]: cluster 2020-09-08T15:39:39.624113+0000 mgr.x (mgr.24118) 364 : cluster [DBG] pgmap v459: 17 pgs: 17 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 850 op/s 2020-09-08T15:39:40.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:40 smithi055 bash[9296]: cluster 2020-09-08T15:39:39.624113+0000 mgr.x (mgr.24118) 364 : cluster [DBG] pgmap v459: 17 pgs: 17 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 850 op/s 2020-09-08T15:39:40.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:40 smithi104 bash[9203]: cluster 2020-09-08T15:39:39.624113+0000 mgr.x (mgr.24118) 364 : cluster [DBG] pgmap v459: 17 pgs: 17 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 850 op/s 2020-09-08T15:39:40.927 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:39:40 smithi104 bash[22305]: debug 2020-09-08T15:39:40.833+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:40.941 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:42.082 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:41 smithi104 bash[9203]: cluster 2020-09-08T15:39:41.624519+0000 mgr.x (mgr.24118) 365 : cluster [DBG] pgmap v460: 17 pgs: 17 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 620 op/s 2020-09-08T15:39:42.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:41 smithi166 bash[9446]: cluster 2020-09-08T15:39:41.624519+0000 mgr.x (mgr.24118) 365 : cluster [DBG] pgmap v460: 17 pgs: 17 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 620 op/s 2020-09-08T15:39:42.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:41 smithi055 bash[9296]: cluster 2020-09-08T15:39:41.624519+0000 mgr.x (mgr.24118) 365 : cluster [DBG] pgmap v460: 17 pgs: 17 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 620 op/s 2020-09-08T15:39:43.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:42 smithi166 bash[9446]: cluster 2020-09-08T15:39:41.640820+0000 osd.11 (osd.11) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:43.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:42 smithi166 bash[9446]: cluster 2020-09-08T15:39:41.640921+0000 osd.11 (osd.11) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:43.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:42 smithi166 bash[9446]: cluster 2020-09-08T15:39:41.699549+0000 mon.a (mon.0) 1002 : cluster [DBG] osdmap e174: 12 total, 12 up, 12 in 2020-09-08T15:39:43.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:42 smithi055 bash[9296]: cluster 2020-09-08T15:39:41.640820+0000 osd.11 (osd.11) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:43.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:42 smithi055 bash[9296]: cluster 2020-09-08T15:39:41.640921+0000 osd.11 (osd.11) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:43.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:42 smithi055 bash[9296]: cluster 2020-09-08T15:39:41.699549+0000 mon.a (mon.0) 1002 : cluster [DBG] osdmap e174: 12 total, 12 up, 12 in 2020-09-08T15:39:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:42 smithi104 bash[9203]: cluster 2020-09-08T15:39:41.640820+0000 osd.11 (osd.11) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:43.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:42 smithi104 bash[9203]: cluster 2020-09-08T15:39:41.640921+0000 osd.11 (osd.11) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:43.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:42 smithi104 bash[9203]: cluster 2020-09-08T15:39:41.699549+0000 mon.a (mon.0) 1002 : cluster [DBG] osdmap e174: 12 total, 12 up, 12 in 2020-09-08T15:39:44.092 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:43 smithi055 bash[9296]: cluster 2020-09-08T15:39:42.674676+0000 osd.7 (osd.7) 29 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:39:44.093 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:43 smithi055 bash[9296]: cluster 2020-09-08T15:39:42.676358+0000 osd.7 (osd.7) 30 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:39:44.093 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:43 smithi055 bash[9296]: cluster 2020-09-08T15:39:43.625017+0000 mgr.x (mgr.24118) 366 : cluster [DBG] pgmap v462: 17 pgs: 17 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T15:39:44.093 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:44.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:43 smithi166 bash[9446]: cluster 2020-09-08T15:39:42.674676+0000 osd.7 (osd.7) 29 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:39:44.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:43 smithi166 bash[9446]: cluster 2020-09-08T15:39:42.676358+0000 osd.7 (osd.7) 30 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:39:44.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:43 smithi166 bash[9446]: cluster 2020-09-08T15:39:43.625017+0000 mgr.x (mgr.24118) 366 : cluster [DBG] pgmap v462: 17 pgs: 17 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T15:39:44.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:43 smithi104 bash[9203]: cluster 2020-09-08T15:39:42.674676+0000 osd.7 (osd.7) 29 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:39:44.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:43 smithi104 bash[9203]: cluster 2020-09-08T15:39:42.676358+0000 osd.7 (osd.7) 30 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:39:44.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:43 smithi104 bash[9203]: cluster 2020-09-08T15:39:43.625017+0000 mgr.x (mgr.24118) 366 : cluster [DBG] pgmap v462: 17 pgs: 17 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T15:39:44.196 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:39:44.338 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:39:44.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:39:44 smithi055 bash[11956]: debug 2020-09-08T15:39:44.096+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:44.446 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:44.561 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:44.615 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:39:44 smithi166 bash[16593]: debug 2020-09-08T15:39:44.338+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:44.665 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:39:44.789 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:39:44.897 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:44.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:39:44 smithi055 bash[11956]: debug 2020-09-08T15:39:44.564+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:45.000 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:45.108 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:45.114 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:39:44 smithi166 bash[16593]: debug 2020-09-08T15:39:44.786+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:45.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:39:45 smithi055 bash[16646]: debug 2020-09-08T15:39:45.000+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:45.165 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:39:45.212 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:39:45.252 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:39:45.288 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:39:45.318 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:39:45.347 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:39:45.369 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:39:45.403 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:39:45.431 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:45.444 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:39:45.481 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:39:45.529 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:39:45.534 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:39:45.568 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:39:45.587 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:39:45.587 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:39:45.588 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:39:45.588 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:39:45.588 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:39:45.588 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:39:45.588 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:39:45.589 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:39:45.589 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:39:45.589 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:39:45.590 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:39:45.590 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:39:45.662 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:39:45.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:39:45 smithi104 bash[22305]: debug 2020-09-08T15:39:45.429+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:45.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:45 smithi104 bash[9203]: audit 2020-09-08T15:39:45.113925+0000 mon.a (mon.0) 1003 : audit [DBG] from='client.? v1:172.21.15.104:0/2809656173' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:39:45.763 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:39:45.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:45 smithi166 bash[9446]: audit 2020-09-08T15:39:45.113925+0000 mon.a (mon.0) 1003 : audit [DBG] from='client.? v1:172.21.15.104:0/2809656173' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:39:45.867 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:39:45 smithi166 bash[16593]: debug 2020-09-08T15:39:45.650+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:45.876 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:39:45.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:45 smithi055 bash[9296]: audit 2020-09-08T15:39:45.113925+0000 mon.a (mon.0) 1003 : audit [DBG] from='client.? v1:172.21.15.104:0/2809656173' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:39:45.981 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:46.098 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:46.204 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:46.365 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:39:46 smithi166 bash[11928]: debug 2020-09-08T15:39:46.094+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:46.365 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:39:45 smithi166 bash[16593]: debug 2020-09-08T15:39:45.878+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:46.559 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:46.608 INFO:teuthology.orchestra.run.smithi104.stderr:reweighted osd.6 to 1 (10000) 2020-09-08T15:39:46.614 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:39:46.663 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:39:46.669 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:39:46.670 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:39:46.833 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:39:46 smithi104 bash[22305]: debug 2020-09-08T15:39:46.553+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:46.834 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:46 smithi104 bash[9203]: cluster 2020-09-08T15:39:45.441902+0000 osd.4 (osd.4) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:46.834 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:46 smithi104 bash[9203]: cluster 2020-09-08T15:39:45.442089+0000 osd.4 (osd.4) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:46.834 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:46 smithi104 bash[9203]: cluster 2020-09-08T15:39:45.600255+0000 mon.a (mon.0) 1004 : cluster [DBG] osdmap e175: 12 total, 12 up, 12 in 2020-09-08T15:39:46.835 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:46 smithi104 bash[9203]: cluster 2020-09-08T15:39:45.625518+0000 mgr.x (mgr.24118) 367 : cluster [DBG] pgmap v464: 17 pgs: 17 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 781 op/s 2020-09-08T15:39:46.835 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:46 smithi104 bash[9203]: audit 2020-09-08T15:39:45.966192+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? v1:172.21.15.104:0/1985464322' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:39:46.835 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:46 smithi104 bash[9203]: audit 2020-09-08T15:39:46.026311+0000 mon.a (mon.0) 1006 : audit [INF] from='client.? v1:172.21.15.104:0/2693578549' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2020-09-08T15:39:46.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:46 smithi166 bash[9446]: cluster 2020-09-08T15:39:45.441902+0000 osd.4 (osd.4) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:46.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:46 smithi166 bash[9446]: cluster 2020-09-08T15:39:45.442089+0000 osd.4 (osd.4) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:46.888 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:46 smithi166 bash[9446]: cluster 2020-09-08T15:39:45.600255+0000 mon.a (mon.0) 1004 : cluster [DBG] osdmap e175: 12 total, 12 up, 12 in 2020-09-08T15:39:46.888 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:46 smithi166 bash[9446]: cluster 2020-09-08T15:39:45.625518+0000 mgr.x (mgr.24118) 367 : cluster [DBG] pgmap v464: 17 pgs: 17 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 781 op/s 2020-09-08T15:39:46.888 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:46 smithi166 bash[9446]: audit 2020-09-08T15:39:45.966192+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? v1:172.21.15.104:0/1985464322' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:39:46.888 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:46 smithi166 bash[9446]: audit 2020-09-08T15:39:46.026311+0000 mon.a (mon.0) 1006 : audit [INF] from='client.? v1:172.21.15.104:0/2693578549' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2020-09-08T15:39:46.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:46 smithi055 bash[9296]: cluster 2020-09-08T15:39:45.441902+0000 osd.4 (osd.4) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:46.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:46 smithi055 bash[9296]: cluster 2020-09-08T15:39:45.442089+0000 osd.4 (osd.4) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:46.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:46 smithi055 bash[9296]: cluster 2020-09-08T15:39:45.600255+0000 mon.a (mon.0) 1004 : cluster [DBG] osdmap e175: 12 total, 12 up, 12 in 2020-09-08T15:39:46.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:46 smithi055 bash[9296]: cluster 2020-09-08T15:39:45.625518+0000 mgr.x (mgr.24118) 367 : cluster [DBG] pgmap v464: 17 pgs: 17 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 781 op/s 2020-09-08T15:39:46.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:46 smithi055 bash[9296]: audit 2020-09-08T15:39:45.966192+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? v1:172.21.15.104:0/1985464322' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:39:46.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:46 smithi055 bash[9296]: audit 2020-09-08T15:39:46.026311+0000 mon.a (mon.0) 1006 : audit [INF] from='client.? v1:172.21.15.104:0/2693578549' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2020-09-08T15:39:47.009 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:39:47.111 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:39:47.177 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:39:47 smithi104 bash[17588]: debug 2020-09-08T15:39:47.005+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:47.252 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:39:47.354 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:39:47.455 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:39:47 smithi166 bash[16593]: debug 2020-09-08T15:39:47.250+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:47.455 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:39:47.558 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:39:47.865 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:39:47 smithi166 bash[14254]: debug 2020-09-08T15:39:47.454+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:47.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:47 smithi166 bash[9446]: cluster 2020-09-08T15:39:46.597455+0000 mon.a (mon.0) 1007 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:39:47.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:47 smithi166 bash[9446]: audit 2020-09-08T15:39:46.610501+0000 mon.a (mon.0) 1008 : audit [INF] from='client.? v1:172.21.15.104:0/1985464322' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:39:47.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:47 smithi166 bash[9446]: audit 2020-09-08T15:39:46.610546+0000 mon.a (mon.0) 1009 : audit [INF] from='client.? v1:172.21.15.104:0/2693578549' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2020-09-08T15:39:47.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:47 smithi166 bash[9446]: cluster 2020-09-08T15:39:46.610598+0000 mon.a (mon.0) 1010 : cluster [DBG] osdmap e176: 12 total, 12 up, 12 in 2020-09-08T15:39:47.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:47 smithi055 bash[9296]: cluster 2020-09-08T15:39:46.597455+0000 mon.a (mon.0) 1007 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:39:47.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:47 smithi055 bash[9296]: audit 2020-09-08T15:39:46.610501+0000 mon.a (mon.0) 1008 : audit [INF] from='client.? v1:172.21.15.104:0/1985464322' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:39:47.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:47 smithi055 bash[9296]: audit 2020-09-08T15:39:46.610546+0000 mon.a (mon.0) 1009 : audit [INF] from='client.? v1:172.21.15.104:0/2693578549' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2020-09-08T15:39:47.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:47 smithi055 bash[9296]: cluster 2020-09-08T15:39:46.610598+0000 mon.a (mon.0) 1010 : cluster [DBG] osdmap e176: 12 total, 12 up, 12 in 2020-09-08T15:39:47.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:47 smithi104 bash[9203]: cluster 2020-09-08T15:39:46.597455+0000 mon.a (mon.0) 1007 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:39:47.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:47 smithi104 bash[9203]: audit 2020-09-08T15:39:46.610501+0000 mon.a (mon.0) 1008 : audit [INF] from='client.? v1:172.21.15.104:0/1985464322' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:39:47.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:47 smithi104 bash[9203]: audit 2020-09-08T15:39:46.610546+0000 mon.a (mon.0) 1009 : audit [INF] from='client.? v1:172.21.15.104:0/2693578549' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2020-09-08T15:39:47.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:47 smithi104 bash[9203]: cluster 2020-09-08T15:39:46.610598+0000 mon.a (mon.0) 1010 : cluster [DBG] osdmap e176: 12 total, 12 up, 12 in 2020-09-08T15:39:48.184 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:39:48.287 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:48.620 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:48 smithi104 bash[9203]: cluster 2020-09-08T15:39:47.625993+0000 mgr.x (mgr.24118) 368 : cluster [DBG] pgmap v466: 17 pgs: 17 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.39k op/s 2020-09-08T15:39:48.621 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:39:48 smithi104 bash[15273]: debug 2020-09-08T15:39:48.185+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:48.758 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd reweight 7 1 2020-09-08T15:39:48.778 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:39:48.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:48 smithi166 bash[9446]: cluster 2020-09-08T15:39:47.625993+0000 mgr.x (mgr.24118) 368 : cluster [DBG] pgmap v466: 17 pgs: 17 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.39k op/s 2020-09-08T15:39:48.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:48 smithi055 bash[9296]: cluster 2020-09-08T15:39:47.625993+0000 mgr.x (mgr.24118) 368 : cluster [DBG] pgmap v466: 17 pgs: 17 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.39k op/s 2020-09-08T15:39:48.928 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2020-09-08T15:39:48.933 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:39:48.975 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:39:49.088 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:39:50.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:50 smithi166 bash[9446]: cluster 2020-09-08T15:39:49.626468+0000 mgr.x (mgr.24118) 369 : cluster [DBG] pgmap v467: 17 pgs: 17 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:39:50.878 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:50 smithi104 bash[9203]: cluster 2020-09-08T15:39:49.626468+0000 mgr.x (mgr.24118) 369 : cluster [DBG] pgmap v467: 17 pgs: 17 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:39:50.880 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:39:50.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:50 smithi055 bash[9296]: cluster 2020-09-08T15:39:49.626468+0000 mgr.x (mgr.24118) 369 : cluster [DBG] pgmap v467: 17 pgs: 17 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:39:51.045 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:39:52.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:51 smithi166 bash[9446]: cluster 2020-09-08T15:39:51.626896+0000 mgr.x (mgr.24118) 370 : cluster [DBG] pgmap v468: 17 pgs: 17 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 616 op/s 2020-09-08T15:39:52.174 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:51 smithi055 bash[9296]: cluster 2020-09-08T15:39:51.626896+0000 mgr.x (mgr.24118) 370 : cluster [DBG] pgmap v468: 17 pgs: 17 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 616 op/s 2020-09-08T15:39:52.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:51 smithi104 bash[9203]: cluster 2020-09-08T15:39:51.626896+0000 mgr.x (mgr.24118) 370 : cluster [DBG] pgmap v468: 17 pgs: 17 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 616 op/s 2020-09-08T15:39:52.600 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:52.705 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:39:52.865 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:39:52 smithi166 bash[11928]: debug 2020-09-08T15:39:52.602+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:53.806 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:39:53.806 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:39:53.883 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:39:53.885 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:53 smithi055 bash[9296]: cluster 2020-09-08T15:39:53.627394+0000 mgr.x (mgr.24118) 371 : cluster [DBG] pgmap v469: 17 pgs: 17 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.23k op/s 2020-09-08T15:39:53.987 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:54.115 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:39:53 smithi166 bash[14254]: debug 2020-09-08T15:39:53.882+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:54.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:53 smithi166 bash[9446]: cluster 2020-09-08T15:39:53.627394+0000 mgr.x (mgr.24118) 371 : cluster [DBG] pgmap v469: 17 pgs: 17 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.23k op/s 2020-09-08T15:39:54.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:53 smithi104 bash[9203]: cluster 2020-09-08T15:39:53.627394+0000 mgr.x (mgr.24118) 371 : cluster [DBG] pgmap v469: 17 pgs: 17 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.23k op/s 2020-09-08T15:39:54.516 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:39:54.620 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:39:54.732 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2020-09-08T15:39:54.844 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:39:54.845 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:54 smithi104 bash[9203]: cluster 2020-09-08T15:39:53.720618+0000 osd.9 (osd.9) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:54.845 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:54 smithi104 bash[9203]: cluster 2020-09-08T15:39:53.720710+0000 osd.9 (osd.9) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:54.845 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:39:54 smithi104 bash[15273]: debug 2020-09-08T15:39:54.841+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:54.871 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:39:54.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:54 smithi055 bash[9296]: cluster 2020-09-08T15:39:53.720618+0000 osd.9 (osd.9) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:54.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:54 smithi055 bash[9296]: cluster 2020-09-08T15:39:53.720710+0000 osd.9 (osd.9) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:54.910 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:39:54 smithi055 bash[11956]: debug 2020-09-08T15:39:54.516+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:54.947 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:39:54.955 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:54 smithi166 bash[9446]: cluster 2020-09-08T15:39:53.720618+0000 osd.9 (osd.9) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:54.955 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:54 smithi166 bash[9446]: cluster 2020-09-08T15:39:53.720710+0000 osd.9 (osd.9) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:55.119 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:39:55.221 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:55.229 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:39:55 smithi166 bash[18960]: debug 2020-09-08T15:39:55.118+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:55.334 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:55.443 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:55.563 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:55.615 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:39:55 smithi166 bash[11928]: debug 2020-09-08T15:39:55.334+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:55.681 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:55.682 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:39:55.712 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:39:55 smithi055 bash[19003]: debug 2020-09-08T15:39:55.564+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:55.715 INFO:teuthology.orchestra.run.smithi104.stderr:reweighted osd.7 to 1 (10000) 2020-09-08T15:39:55.730 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:39:55.768 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:39:55.806 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:39:55.839 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:39:55.877 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:39:55.914 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:39:55.951 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:39:55.988 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:39:56.028 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:39:56.062 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:39:56.069 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:39:56 smithi055 bash[19003]: debug 2020-09-08T15:39:56.072+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:56.070 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:55 smithi055 bash[9296]: cluster 2020-09-08T15:39:54.701128+0000 mon.a (mon.0) 1011 : cluster [DBG] osdmap e177: 12 total, 12 up, 12 in 2020-09-08T15:39:56.070 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:55 smithi055 bash[9296]: audit 2020-09-08T15:39:55.140515+0000 mon.c (mon.2) 39 : audit [INF] from='client.? v1:172.21.15.104:0/1879209654' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2020-09-08T15:39:56.070 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:55 smithi055 bash[9296]: audit 2020-09-08T15:39:55.140965+0000 mon.a (mon.0) 1012 : audit [INF] from='client.34862 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2020-09-08T15:39:56.070 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:55 smithi055 bash[9296]: cluster 2020-09-08T15:39:55.627841+0000 mgr.x (mgr.24118) 372 : cluster [DBG] pgmap v471: 17 pgs: 17 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T15:39:56.071 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:55 smithi055 bash[9296]: audit 2020-09-08T15:39:55.645262+0000 mon.b (mon.1) 78 : audit [DBG] from='client.? v1:172.21.15.104:0/4091817800' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:39:56.071 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:39:56.098 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:39:56.103 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:39:56.103 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:39:56.104 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:39:56.104 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:39:56.105 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:39:56.105 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:39:56.105 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:39:56.106 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:39:56.106 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:39:56.106 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:39:56.106 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:39:56.107 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:39:56.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:55 smithi166 bash[9446]: cluster 2020-09-08T15:39:54.701128+0000 mon.a (mon.0) 1011 : cluster [DBG] osdmap e177: 12 total, 12 up, 12 in 2020-09-08T15:39:56.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:55 smithi166 bash[9446]: audit 2020-09-08T15:39:55.140515+0000 mon.c (mon.2) 39 : audit [INF] from='client.? v1:172.21.15.104:0/1879209654' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2020-09-08T15:39:56.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:55 smithi166 bash[9446]: audit 2020-09-08T15:39:55.140965+0000 mon.a (mon.0) 1012 : audit [INF] from='client.34862 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2020-09-08T15:39:56.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:55 smithi166 bash[9446]: cluster 2020-09-08T15:39:55.627841+0000 mgr.x (mgr.24118) 372 : cluster [DBG] pgmap v471: 17 pgs: 17 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T15:39:56.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:55 smithi166 bash[9446]: audit 2020-09-08T15:39:55.645262+0000 mon.b (mon.1) 78 : audit [DBG] from='client.? v1:172.21.15.104:0/4091817800' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:39:56.175 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:56.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:55 smithi104 bash[9203]: cluster 2020-09-08T15:39:54.701128+0000 mon.a (mon.0) 1011 : cluster [DBG] osdmap e177: 12 total, 12 up, 12 in 2020-09-08T15:39:56.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:55 smithi104 bash[9203]: audit 2020-09-08T15:39:55.140515+0000 mon.c (mon.2) 39 : audit [INF] from='client.? v1:172.21.15.104:0/1879209654' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2020-09-08T15:39:56.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:55 smithi104 bash[9203]: audit 2020-09-08T15:39:55.140965+0000 mon.a (mon.0) 1012 : audit [INF] from='client.34862 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2020-09-08T15:39:56.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:55 smithi104 bash[9203]: cluster 2020-09-08T15:39:55.627841+0000 mgr.x (mgr.24118) 372 : cluster [DBG] pgmap v471: 17 pgs: 17 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T15:39:56.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:55 smithi104 bash[9203]: audit 2020-09-08T15:39:55.645262+0000 mon.b (mon.1) 78 : audit [DBG] from='client.? v1:172.21.15.104:0/4091817800' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:39:56.322 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:39:56 smithi055 bash[16646]: debug 2020-09-08T15:39:56.324+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:56.323 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:56.427 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:39:57.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:56 smithi166 bash[9446]: audit 2020-09-08T15:39:55.712474+0000 mon.a (mon.0) 1013 : audit [INF] from='client.34862 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2020-09-08T15:39:57.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:56 smithi166 bash[9446]: cluster 2020-09-08T15:39:55.712562+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e178: 12 total, 12 up, 12 in 2020-09-08T15:39:57.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:56 smithi055 bash[9296]: audit 2020-09-08T15:39:55.712474+0000 mon.a (mon.0) 1013 : audit [INF] from='client.34862 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2020-09-08T15:39:57.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:56 smithi055 bash[9296]: cluster 2020-09-08T15:39:55.712562+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e178: 12 total, 12 up, 12 in 2020-09-08T15:39:57.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:56 smithi104 bash[9203]: audit 2020-09-08T15:39:55.712474+0000 mon.a (mon.0) 1013 : audit [INF] from='client.34862 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2020-09-08T15:39:57.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:56 smithi104 bash[9203]: cluster 2020-09-08T15:39:55.712562+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e178: 12 total, 12 up, 12 in 2020-09-08T15:39:57.231 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:39:57.334 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:39:57.464 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd reweight 8 1 2020-09-08T15:39:57.520 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:39:57.520 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:39:57 smithi104 bash[17588]: debug 2020-09-08T15:39:57.521+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:57.521 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:39:57 smithi104 bash[19922]: debug 2020-09-08T15:39:57.229+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:57.624 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:39:57.635 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:39:57.911 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:39:57.913 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:57 smithi104 bash[9203]: cluster 2020-09-08T15:39:56.213001+0000 osd.2 (osd.2) 24 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:57.913 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:57 smithi104 bash[9203]: cluster 2020-09-08T15:39:56.213097+0000 osd.2 (osd.2) 25 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:57.913 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:57 smithi104 bash[9203]: cluster 2020-09-08T15:39:56.728435+0000 mon.a (mon.0) 1015 : cluster [DBG] osdmap e179: 12 total, 12 up, 12 in 2020-09-08T15:39:57.914 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:57 smithi104 bash[9203]: cluster 2020-09-08T15:39:57.628420+0000 mgr.x (mgr.24118) 373 : cluster [DBG] pgmap v474: 17 pgs: 17 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.33k op/s 2020-09-08T15:39:58.022 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:58.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:57 smithi166 bash[9446]: cluster 2020-09-08T15:39:56.213001+0000 osd.2 (osd.2) 24 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:58.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:57 smithi166 bash[9446]: cluster 2020-09-08T15:39:56.213097+0000 osd.2 (osd.2) 25 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:58.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:57 smithi166 bash[9446]: cluster 2020-09-08T15:39:56.728435+0000 mon.a (mon.0) 1015 : cluster [DBG] osdmap e179: 12 total, 12 up, 12 in 2020-09-08T15:39:58.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:57 smithi166 bash[9446]: cluster 2020-09-08T15:39:57.628420+0000 mgr.x (mgr.24118) 373 : cluster [DBG] pgmap v474: 17 pgs: 17 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.33k op/s 2020-09-08T15:39:58.164 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:57 smithi055 bash[9296]: cluster 2020-09-08T15:39:56.213001+0000 osd.2 (osd.2) 24 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:39:58.165 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:57 smithi055 bash[9296]: cluster 2020-09-08T15:39:56.213097+0000 osd.2 (osd.2) 25 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:39:58.165 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:57 smithi055 bash[9296]: cluster 2020-09-08T15:39:56.728435+0000 mon.a (mon.0) 1015 : cluster [DBG] osdmap e179: 12 total, 12 up, 12 in 2020-09-08T15:39:58.165 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:57 smithi055 bash[9296]: cluster 2020-09-08T15:39:57.628420+0000 mgr.x (mgr.24118) 373 : cluster [DBG] pgmap v474: 17 pgs: 17 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.33k op/s 2020-09-08T15:39:58.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:39:58 smithi104 bash[15273]: debug 2020-09-08T15:39:57.909+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:58.399 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:58.503 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:58.677 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:39:58 smithi104 bash[22305]: debug 2020-09-08T15:39:58.397+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:58.678 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:39:58.781 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:39:58.828 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:39:58 smithi055 bash[14301]: debug 2020-09-08T15:39:58.676+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:58.981 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:39:58.982 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T15:39:58.982 INFO:teuthology.orchestra.run.smithi055.stdout: "complaint_time": 30, 2020-09-08T15:39:58.982 INFO:teuthology.orchestra.run.smithi055.stdout: "num_blocked_ops": 0 2020-09-08T15:39:58.982 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:39:58.999 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:39:59.035 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:39:59.104 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:59.126 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:39:59 smithi104 bash[17588]: debug 2020-09-08T15:39:58.997+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:59.126 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:59 smithi104 bash[9203]: audit 2020-09-08T15:39:58.981537+0000 mon.a (mon.0) 1016 : audit [INF] from='client.? v1:172.21.15.104:0/4241269494' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:39:59.127 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:39:59 smithi104 bash[9203]: cluster 2020-09-08T15:39:59.031987+0000 mon.a (mon.0) 1017 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:39:59.305 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:59 smithi055 bash[9296]: audit 2020-09-08T15:39:58.981537+0000 mon.a (mon.0) 1016 : audit [INF] from='client.? v1:172.21.15.104:0/4241269494' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:39:59.306 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:39:59 smithi055 bash[9296]: cluster 2020-09-08T15:39:59.031987+0000 mon.a (mon.0) 1017 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:39:59.306 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:39:59.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:59 smithi166 bash[9446]: audit 2020-09-08T15:39:58.981537+0000 mon.a (mon.0) 1016 : audit [INF] from='client.? v1:172.21.15.104:0/4241269494' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:39:59.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:39:59 smithi166 bash[9446]: cluster 2020-09-08T15:39:59.031987+0000 mon.a (mon.0) 1017 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:39:59.411 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:59.570 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:39:59.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:39:59 smithi055 bash[16646]: debug 2020-09-08T15:39:59.304+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:59.673 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:39:59.681 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:39:59.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:39:59 smithi166 bash[11928]: debug 2020-09-08T15:39:59.570+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:39:59.883 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:40:00.010 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2020-09-08T15:40:00.040 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:00 smithi104 bash[9203]: audit 2020-09-08T15:39:59.038318+0000 mon.a (mon.0) 1018 : audit [INF] from='client.? v1:172.21.15.104:0/4241269494' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:40:00.041 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:00 smithi104 bash[9203]: cluster 2020-09-08T15:39:59.038357+0000 mon.a (mon.0) 1019 : cluster [DBG] osdmap e180: 12 total, 12 up, 12 in 2020-09-08T15:40:00.041 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:00 smithi104 bash[9203]: cluster 2020-09-08T15:39:59.628797+0000 mgr.x (mgr.24118) 374 : cluster [DBG] pgmap v476: 17 pgs: 17 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 931 op/s 2020-09-08T15:40:00.160 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:40:00.296 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:00.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:00 smithi166 bash[9446]: audit 2020-09-08T15:39:59.038318+0000 mon.a (mon.0) 1018 : audit [INF] from='client.? v1:172.21.15.104:0/4241269494' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:40:00.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:00 smithi166 bash[9446]: cluster 2020-09-08T15:39:59.038357+0000 mon.a (mon.0) 1019 : cluster [DBG] osdmap e180: 12 total, 12 up, 12 in 2020-09-08T15:40:00.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:00 smithi166 bash[9446]: cluster 2020-09-08T15:39:59.628797+0000 mgr.x (mgr.24118) 374 : cluster [DBG] pgmap v476: 17 pgs: 17 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 931 op/s 2020-09-08T15:40:00.399 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:00.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:00 smithi055 bash[9296]: audit 2020-09-08T15:39:59.038318+0000 mon.a (mon.0) 1018 : audit [INF] from='client.? v1:172.21.15.104:0/4241269494' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:40:00.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:00 smithi055 bash[9296]: cluster 2020-09-08T15:39:59.038357+0000 mon.a (mon.0) 1019 : cluster [DBG] osdmap e180: 12 total, 12 up, 12 in 2020-09-08T15:40:00.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:00 smithi055 bash[9296]: cluster 2020-09-08T15:39:59.628797+0000 mgr.x (mgr.24118) 374 : cluster [DBG] pgmap v476: 17 pgs: 17 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 931 op/s 2020-09-08T15:40:00.664 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:40:00 smithi104 bash[22305]: debug 2020-09-08T15:40:00.297+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:00.746 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:40:00 smithi055 bash[16646]: debug 2020-09-08T15:40:00.748+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:00.746 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:00.851 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:01.342 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:01.445 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:01.598 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:01.651 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:40:01 smithi055 bash[16646]: debug 2020-09-08T15:40:01.340+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:01.710 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:40:01.822 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:40:01.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:01 smithi166 bash[9446]: cluster 2020-09-08T15:40:01.629185+0000 mgr.x (mgr.24118) 375 : cluster [DBG] pgmap v477: 17 pgs: 17 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 764 op/s 2020-09-08T15:40:01.865 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:40:01 smithi166 bash[11928]: debug 2020-09-08T15:40:01.598+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:01.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:40:01 smithi055 bash[19003]: debug 2020-09-08T15:40:01.824+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:01.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:01 smithi055 bash[9296]: cluster 2020-09-08T15:40:01.629185+0000 mgr.x (mgr.24118) 375 : cluster [DBG] pgmap v477: 17 pgs: 17 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 764 op/s 2020-09-08T15:40:01.928 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:01.937 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:01 smithi104 bash[9203]: cluster 2020-09-08T15:40:01.629185+0000 mgr.x (mgr.24118) 375 : cluster [DBG] pgmap v477: 17 pgs: 17 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 764 op/s 2020-09-08T15:40:02.135 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:02.239 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:40:02.250 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:40:02 smithi104 bash[19922]: debug 2020-09-08T15:40:02.133+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:02.665 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:40:02.718 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:40:02.821 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:02.836 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:40:02.927 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:40:02 smithi104 bash[17588]: debug 2020-09-08T15:40:02.713+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:03.711 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:03.815 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:03.974 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:40:03 smithi166 bash[11928]: debug 2020-09-08T15:40:03.970+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:03.975 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:03 smithi166 bash[9446]: cluster 2020-09-08T15:40:03.629959+0000 mgr.x (mgr.24118) 376 : cluster [DBG] pgmap v478: 17 pgs: 17 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.17k op/s 2020-09-08T15:40:03.975 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:04.020 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:40:04.020 INFO:teuthology.orchestra.run.smithi055.stdout: "size": 20, 2020-09-08T15:40:04.020 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 600, 2020-09-08T15:40:04.020 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T15:40:04.021 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.021 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:89959 2.d e145/126 2:b0f3281f:::benchmark_data_smithi166_20551_object89958:head v 145'5544, mlcod=145'5542)", 2020-09-08T15:40:04.021 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.615696+0000", 2020-09-08T15:40:04.021 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 99.406671223000004, 2020-09-08T15:40:04.022 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.041290546999999997, 2020-09-08T15:40:04.022 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:04.022 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:04.022 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:04.022 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.023 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:04.023 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.615696+0000", 2020-09-08T15:40:04.023 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:04.023 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.023 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.024 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:04.024 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.615696+0000", 2020-09-08T15:40:04.024 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.7872000000000002e-05 2020-09-08T15:40:04.024 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.024 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.024 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:04.025 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.615784+0000", 2020-09-08T15:40:04.025 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2560000000000002e-06 2020-09-08T15:40:04.025 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.025 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.025 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:04.026 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.615792+0000", 2020-09-08T15:40:04.026 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6212e-05 2020-09-08T15:40:04.026 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.026 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.026 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:04.027 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.615808+0000", 2020-09-08T15:40:04.027 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.2050000000000008e-06 2020-09-08T15:40:04.027 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.027 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.027 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:04.028 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.615817+0000", 2020-09-08T15:40:04.028 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.4191999999999998e-05 2020-09-08T15:40:04.028 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.029 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.029 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:04.029 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.615882+0000", 2020-09-08T15:40:04.029 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.0769999999999998e-05 2020-09-08T15:40:04.029 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.030 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.030 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:04.030 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.615922+0000", 2020-09-08T15:40:04.030 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.040969764999999998 2020-09-08T15:40:04.030 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.031 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.031 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:04.031 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.656892+0000", 2020-09-08T15:40:04.031 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4275000000000001e-05 2020-09-08T15:40:04.031 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.032 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.032 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:04.032 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.656986+0000", 2020-09-08T15:40:04.032 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.041290546999999997 2020-09-08T15:40:04.032 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.032 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.033 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.033 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.033 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.033 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:96972 2.d e147/126 2:b28078ee:::benchmark_data_smithi166_20551_object96971:head v 147'6004, mlcod=147'6002)", 2020-09-08T15:40:04.033 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:29.627111+0000", 2020-09-08T15:40:04.034 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 94.395256012000004, 2020-09-08T15:40:04.034 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044209323000000002, 2020-09-08T15:40:04.034 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:04.034 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:04.034 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:04.035 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.035 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:04.035 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.627111+0000", 2020-09-08T15:40:04.035 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:04.035 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.036 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.036 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:04.036 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.627111+0000", 2020-09-08T15:40:04.036 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012088599999999999 2020-09-08T15:40:04.036 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.036 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.037 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:04.037 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.627232+0000", 2020-09-08T15:40:04.037 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.29e-05 2020-09-08T15:40:04.037 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.037 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.038 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:04.038 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.627245+0000", 2020-09-08T15:40:04.038 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.1520999999999999e-05 2020-09-08T15:40:04.038 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.039 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.039 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:04.039 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.627266+0000", 2020-09-08T15:40:04.039 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1144999999999999e-05 2020-09-08T15:40:04.040 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.040 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.040 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:04.040 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.627278+0000", 2020-09-08T15:40:04.040 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.8328000000000001e-05 2020-09-08T15:40:04.040 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.041 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.041 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:04.041 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.627336+0000", 2020-09-08T15:40:04.041 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.9403e-05 2020-09-08T15:40:04.041 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.042 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.042 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:04.042 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.627365+0000", 2020-09-08T15:40:04.042 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043826854999999998 2020-09-08T15:40:04.042 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.043 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.043 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:04.043 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.671192+0000", 2020-09-08T15:40:04.043 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012828500000000001 2020-09-08T15:40:04.043 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.044 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.044 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:04.044 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.671320+0000", 2020-09-08T15:40:04.044 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044209323000000002 2020-09-08T15:40:04.044 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.044 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.045 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.045 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.045 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.045 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:96977 2.d e147/126 2:b572cc18:::benchmark_data_smithi166_20551_object96976:head v 147'6005, mlcod=147'6002)", 2020-09-08T15:40:04.045 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:29.636838+0000", 2020-09-08T15:40:04.046 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 94.385529109999993, 2020-09-08T15:40:04.046 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.034581226, 2020-09-08T15:40:04.046 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:04.046 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:04.046 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:04.047 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.047 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:04.047 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.636838+0000", 2020-09-08T15:40:04.047 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:04.047 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.048 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.048 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:04.048 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.636838+0000", 2020-09-08T15:40:04.048 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.0309999999999995e-05 2020-09-08T15:40:04.048 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.048 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.053 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:04.053 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.636918+0000", 2020-09-08T15:40:04.053 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.5719999999999999e-06 2020-09-08T15:40:04.054 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.054 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.054 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:04.054 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.636925+0000", 2020-09-08T15:40:04.054 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3035e-05 2020-09-08T15:40:04.055 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.055 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.055 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:04.055 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.636938+0000", 2020-09-08T15:40:04.055 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.5660000000000007e-06 2020-09-08T15:40:04.056 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.056 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.056 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:04.056 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.636947+0000", 2020-09-08T15:40:04.056 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.2966000000000007e-05 2020-09-08T15:40:04.057 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.057 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.057 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:04.057 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.637010+0000", 2020-09-08T15:40:04.057 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.3374999999999999e-05 2020-09-08T15:40:04.058 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.058 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.058 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:04.058 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.637033+0000", 2020-09-08T15:40:04.059 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.034292727000000002 2020-09-08T15:40:04.059 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.059 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.059 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:04.059 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.671326+0000", 2020-09-08T15:40:04.060 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3675e-05 2020-09-08T15:40:04.060 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.060 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.060 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:04.060 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.671419+0000", 2020-09-08T15:40:04.061 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.034581226 2020-09-08T15:40:04.061 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.061 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.061 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.061 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.062 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.062 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:104231 2.d e149/126 2:bddc5f0a:::benchmark_data_smithi166_20551_object104230:head v 149'6446, mlcod=149'6439)", 2020-09-08T15:40:04.062 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:34.993420+0000", 2020-09-08T15:40:04.062 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 89.028947251999995, 2020-09-08T15:40:04.062 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.022610866, 2020-09-08T15:40:04.063 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:04.063 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:04.063 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:04.064 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.064 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:04.064 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.993420+0000", 2020-09-08T15:40:04.064 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:04.064 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.064 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.065 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:04.065 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.993420+0000", 2020-09-08T15:40:04.065 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000242363 2020-09-08T15:40:04.065 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.065 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.066 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:04.066 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.993662+0000", 2020-09-08T15:40:04.066 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1253999999999999e-05 2020-09-08T15:40:04.066 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.066 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.067 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:04.067 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.993674+0000", 2020-09-08T15:40:04.067 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.8977000000000001e-05 2020-09-08T15:40:04.067 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.067 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.068 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:04.068 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.993703+0000", 2020-09-08T15:40:04.068 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.968e-06 2020-09-08T15:40:04.068 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.068 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.069 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:04.069 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.993709+0000", 2020-09-08T15:40:04.069 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.013083892 2020-09-08T15:40:04.069 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.069 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.069 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:04.070 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.006793+0000", 2020-09-08T15:40:04.070 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6475000000000001e-05 2020-09-08T15:40:04.070 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.070 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.070 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:04.071 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.006810+0000", 2020-09-08T15:40:04.071 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0091412019999999993 2020-09-08T15:40:04.071 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.071 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.071 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:04.072 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.015951+0000", 2020-09-08T15:40:04.072 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.9734999999999995e-05 2020-09-08T15:40:04.072 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.072 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.072 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:04.073 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.016031+0000", 2020-09-08T15:40:04.073 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.022610866 2020-09-08T15:40:04.073 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.073 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.074 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.074 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.074 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.074 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:104233 2.d e149/126 2:b4b13966:::benchmark_data_smithi166_20551_object104232:head v 149'6447, mlcod=149'6439)", 2020-09-08T15:40:04.074 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:34.993778+0000", 2020-09-08T15:40:04.075 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 89.028589538999995, 2020-09-08T15:40:04.075 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.022335487000000001, 2020-09-08T15:40:04.075 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:04.075 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:04.075 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:04.076 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.076 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:04.076 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.993778+0000", 2020-09-08T15:40:04.076 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:04.076 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.077 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.077 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:04.077 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.993778+0000", 2020-09-08T15:40:04.077 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00026113299999999998 2020-09-08T15:40:04.077 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.077 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.078 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:04.078 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.994039+0000", 2020-09-08T15:40:04.078 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1606e-05 2020-09-08T15:40:04.078 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.078 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.079 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:04.079 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.994050+0000", 2020-09-08T15:40:04.079 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.7152999999999999e-05 2020-09-08T15:40:04.079 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.079 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.080 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:04.080 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.994118+0000", 2020-09-08T15:40:04.080 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.3000000000000004e-06 2020-09-08T15:40:04.080 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.080 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.081 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:04.081 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.994125+0000", 2020-09-08T15:40:04.081 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.014502477999999999 2020-09-08T15:40:04.081 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.081 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.081 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:04.082 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.008627+0000", 2020-09-08T15:40:04.082 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6634e-05 2020-09-08T15:40:04.082 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.082 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.082 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:04.083 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.008644+0000", 2020-09-08T15:40:04.083 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0073952710000000001 2020-09-08T15:40:04.083 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.083 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.083 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:04.084 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.016039+0000", 2020-09-08T15:40:04.084 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.3912000000000007e-05 2020-09-08T15:40:04.084 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.084 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.085 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:04.085 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.016113+0000", 2020-09-08T15:40:04.085 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.022335487000000001 2020-09-08T15:40:04.086 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.086 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.086 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.086 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.086 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.087 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:104260 2.d e149/126 2:bd9b5e79:::benchmark_data_smithi166_20551_object104259:head v 149'6448, mlcod=149'6439)", 2020-09-08T15:40:04.087 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:34.994188+0000", 2020-09-08T15:40:04.087 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 89.028179030999993, 2020-09-08T15:40:04.087 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.026010376000000002, 2020-09-08T15:40:04.087 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:04.087 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:04.088 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:04.088 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.088 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:04.088 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.994188+0000", 2020-09-08T15:40:04.088 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:04.089 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.089 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.089 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:04.089 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.994188+0000", 2020-09-08T15:40:04.089 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00024107999999999999 2020-09-08T15:40:04.090 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.090 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.090 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:04.090 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.994429+0000", 2020-09-08T15:40:04.090 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1285e-05 2020-09-08T15:40:04.090 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.091 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.091 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:04.091 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.994441+0000", 2020-09-08T15:40:04.091 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.6755999999999996e-05 2020-09-08T15:40:04.091 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.092 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.092 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:04.092 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.994507+0000", 2020-09-08T15:40:04.092 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.8520000000000004e-06 2020-09-08T15:40:04.092 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.093 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.093 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:04.093 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.994515+0000", 2020-09-08T15:40:04.093 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.015744522 2020-09-08T15:40:04.093 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.094 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.094 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:04.094 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.010260+0000", 2020-09-08T15:40:04.094 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.8202e-05 2020-09-08T15:40:04.095 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.095 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.095 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:04.095 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.010278+0000", 2020-09-08T15:40:04.095 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0098263650000000001 2020-09-08T15:40:04.096 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.096 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.096 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:04.096 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.020104+0000", 2020-09-08T15:40:04.096 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4314000000000005e-05 2020-09-08T15:40:04.097 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.097 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.097 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:04.097 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.020199+0000", 2020-09-08T15:40:04.097 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.026010376000000002 2020-09-08T15:40:04.097 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.098 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.098 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.098 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.098 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.098 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:104267 2.d e149/126 2:b7ac92f7:::benchmark_data_smithi166_20551_object104266:head v 149'6449, mlcod=149'6439)", 2020-09-08T15:40:04.099 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:34.994584+0000", 2020-09-08T15:40:04.099 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 89.027783322000005, 2020-09-08T15:40:04.099 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.025678671, 2020-09-08T15:40:04.099 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:04.099 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:04.100 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:04.100 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.100 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:04.100 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.994584+0000", 2020-09-08T15:40:04.100 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:04.100 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.101 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.101 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:04.101 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.994584+0000", 2020-09-08T15:40:04.101 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00030630399999999997 2020-09-08T15:40:04.102 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.102 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.102 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:04.102 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.994890+0000", 2020-09-08T15:40:04.102 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2534000000000001e-05 2020-09-08T15:40:04.102 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.103 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.103 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:04.103 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.994903+0000", 2020-09-08T15:40:04.103 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.4751000000000001e-05 2020-09-08T15:40:04.103 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.104 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.104 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:04.104 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.994927+0000", 2020-09-08T15:40:04.104 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.3599999999999996e-06 2020-09-08T15:40:04.105 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.105 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.105 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:04.105 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.994936+0000", 2020-09-08T15:40:04.105 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.017308318999999999 2020-09-08T15:40:04.106 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.106 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.106 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:04.106 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.012244+0000", 2020-09-08T15:40:04.106 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.2624999999999999e-05 2020-09-08T15:40:04.107 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.107 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.107 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:04.107 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.012287+0000", 2020-09-08T15:40:04.107 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0079193110000000001 2020-09-08T15:40:04.108 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.108 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.108 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:04.108 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.020206+0000", 2020-09-08T15:40:04.108 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.6467e-05 2020-09-08T15:40:04.109 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.109 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.109 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:04.109 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.020263+0000", 2020-09-08T15:40:04.109 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.025678671 2020-09-08T15:40:04.110 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.110 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.110 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.110 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.110 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.110 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:104279 2.d e149/126 2:be860736:::benchmark_data_smithi166_20551_object104278:head v 149'6450, mlcod=149'6439)", 2020-09-08T15:40:04.111 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:34.995097+0000", 2020-09-08T15:40:04.111 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 89.027270384000005, 2020-09-08T15:40:04.111 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.030753697999999999, 2020-09-08T15:40:04.111 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:04.111 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:04.112 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:04.112 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.112 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:04.112 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.995097+0000", 2020-09-08T15:40:04.112 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:04.113 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.113 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.113 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:04.113 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.995097+0000", 2020-09-08T15:40:04.113 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000223923 2020-09-08T15:40:04.114 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.114 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.114 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:04.114 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.995321+0000", 2020-09-08T15:40:04.115 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.7250000000000003e-06 2020-09-08T15:40:04.115 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.115 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.115 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:04.115 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.995329+0000", 2020-09-08T15:40:04.116 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3207000000000001e-05 2020-09-08T15:40:04.116 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.116 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.116 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:04.116 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.995343+0000", 2020-09-08T15:40:04.116 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9909999999999999e-06 2020-09-08T15:40:04.117 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.117 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.117 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:04.117 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.995349+0000", 2020-09-08T15:40:04.117 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.019053697000000001 2020-09-08T15:40:04.118 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.118 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.118 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:04.119 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.014402+0000", 2020-09-08T15:40:04.119 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.7359e-05 2020-09-08T15:40:04.119 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.119 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.119 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:04.120 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.014430+0000", 2020-09-08T15:40:04.120 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.011291832999999999 2020-09-08T15:40:04.120 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.120 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.120 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:04.120 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.025722+0000", 2020-09-08T15:40:04.121 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012896300000000001 2020-09-08T15:40:04.121 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.121 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.121 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:04.121 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.025850+0000", 2020-09-08T15:40:04.122 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.030753697999999999 2020-09-08T15:40:04.122 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.122 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.122 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.122 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.123 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.123 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:104307 2.d e149/126 2:b08df55e:::benchmark_data_smithi166_20551_object104306:head v 149'6451, mlcod=149'6439)", 2020-09-08T15:40:04.123 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:34.995424+0000", 2020-09-08T15:40:04.123 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 89.0269428, 2020-09-08T15:40:04.123 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.030524948, 2020-09-08T15:40:04.124 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:04.124 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:04.124 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:04.124 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.124 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:04.124 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.995424+0000", 2020-09-08T15:40:04.125 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:04.125 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.125 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.126 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:04.126 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.995424+0000", 2020-09-08T15:40:04.126 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000140233 2020-09-08T15:40:04.126 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.126 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.126 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:04.127 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.995565+0000", 2020-09-08T15:40:04.127 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.0499999999999992e-06 2020-09-08T15:40:04.127 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.127 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.127 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:04.128 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.995573+0000", 2020-09-08T15:40:04.128 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2272999999999999e-05 2020-09-08T15:40:04.128 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.128 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.128 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:04.129 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.995585+0000", 2020-09-08T15:40:04.129 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.7949999999999998e-06 2020-09-08T15:40:04.129 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.129 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.129 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:04.130 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.995590+0000", 2020-09-08T15:40:04.130 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.020582907000000001 2020-09-08T15:40:04.130 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.130 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.130 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:04.130 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.016173+0000", 2020-09-08T15:40:04.131 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.7429000000000001e-05 2020-09-08T15:40:04.131 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.131 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.131 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:04.131 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.016200+0000", 2020-09-08T15:40:04.132 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0096557469999999992 2020-09-08T15:40:04.132 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.132 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.132 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:04.132 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.025856+0000", 2020-09-08T15:40:04.132 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3513999999999999e-05 2020-09-08T15:40:04.133 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.133 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.133 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:04.133 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.025949+0000", 2020-09-08T15:40:04.133 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.030524948 2020-09-08T15:40:04.134 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.134 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.134 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.134 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.134 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.135 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:104309 2.d e149/126 2:bb0c4cab:::benchmark_data_smithi166_20551_object104308:head v 149'6452, mlcod=149'6439)", 2020-09-08T15:40:04.135 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:34.995629+0000", 2020-09-08T15:40:04.135 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 89.026737897999993, 2020-09-08T15:40:04.135 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.030411852999999999, 2020-09-08T15:40:04.136 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:04.136 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:04.136 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:04.136 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.136 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:04.137 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.995629+0000", 2020-09-08T15:40:04.137 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:04.137 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.137 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.137 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:04.138 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.995629+0000", 2020-09-08T15:40:04.138 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017498899999999999 2020-09-08T15:40:04.138 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.138 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.138 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:04.138 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.995804+0000", 2020-09-08T15:40:04.139 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.1119999999999996e-06 2020-09-08T15:40:04.139 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.139 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.139 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:04.139 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.995812+0000", 2020-09-08T15:40:04.140 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2656e-05 2020-09-08T15:40:04.140 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.140 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.140 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:04.140 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.995825+0000", 2020-09-08T15:40:04.141 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.6080000000000001e-06 2020-09-08T15:40:04.141 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.141 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.141 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:04.141 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.995832+0000", 2020-09-08T15:40:04.142 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.021973504000000001 2020-09-08T15:40:04.142 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.142 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.142 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:04.142 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.017805+0000", 2020-09-08T15:40:04.143 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.1520999999999999e-05 2020-09-08T15:40:04.143 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.143 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.143 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:04.143 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.017837+0000", 2020-09-08T15:40:04.143 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0081181350000000003 2020-09-08T15:40:04.144 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.144 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.144 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:04.144 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.025955+0000", 2020-09-08T15:40:04.144 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.6328000000000004e-05 2020-09-08T15:40:04.145 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.145 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.145 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:04.145 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.026041+0000", 2020-09-08T15:40:04.146 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.030411852999999999 2020-09-08T15:40:04.146 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.146 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.146 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.146 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.147 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.147 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:104328 2.d e149/126 2:b07827f8:::benchmark_data_smithi166_20551_object104327:head v 149'6453, mlcod=149'6439)", 2020-09-08T15:40:04.147 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:34.995871+0000", 2020-09-08T15:40:04.147 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 89.026495781999998, 2020-09-08T15:40:04.147 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.030253828, 2020-09-08T15:40:04.148 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:04.148 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:04.148 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:04.148 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.148 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:04.149 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.995871+0000", 2020-09-08T15:40:04.149 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:04.149 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.149 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.149 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:04.149 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.995871+0000", 2020-09-08T15:40:04.150 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017462399999999999 2020-09-08T15:40:04.150 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.150 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.150 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:04.150 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.996046+0000", 2020-09-08T15:40:04.151 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2390000000000002e-06 2020-09-08T15:40:04.151 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.151 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.151 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:04.151 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.996054+0000", 2020-09-08T15:40:04.152 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2329e-05 2020-09-08T15:40:04.153 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.153 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.153 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:04.153 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.996067+0000", 2020-09-08T15:40:04.154 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.6630000000000004e-06 2020-09-08T15:40:04.154 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.154 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.154 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:04.155 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.996073+0000", 2020-09-08T15:40:04.155 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.024259109000000001 2020-09-08T15:40:04.155 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.155 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.156 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:04.156 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.020332+0000", 2020-09-08T15:40:04.156 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.3978e-05 2020-09-08T15:40:04.156 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.156 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.157 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:04.157 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.020356+0000", 2020-09-08T15:40:04.157 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.005688673 2020-09-08T15:40:04.157 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.157 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.158 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:04.158 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.026045+0000", 2020-09-08T15:40:04.158 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.0212999999999999e-05 2020-09-08T15:40:04.158 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.158 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.159 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:04.159 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.026125+0000", 2020-09-08T15:40:04.159 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.030253828 2020-09-08T15:40:04.159 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.160 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.160 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.160 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.160 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.160 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:104332 2.d e149/126 2:b9ce8de0:::benchmark_data_smithi166_20551_object104331:head v 149'6454, mlcod=149'6439)", 2020-09-08T15:40:04.161 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:34.996113+0000", 2020-09-08T15:40:04.161 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 89.026253988999997, 2020-09-08T15:40:04.161 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.036544348999999997, 2020-09-08T15:40:04.161 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:04.161 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:04.162 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:04.162 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.162 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:04.162 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.996113+0000", 2020-09-08T15:40:04.162 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:04.163 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.163 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.163 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:04.163 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.996113+0000", 2020-09-08T15:40:04.163 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000121329 2020-09-08T15:40:04.163 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.164 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.164 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:04.164 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.996235+0000", 2020-09-08T15:40:04.164 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.3250000000000002e-06 2020-09-08T15:40:04.164 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.165 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.165 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:04.165 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.996242+0000", 2020-09-08T15:40:04.165 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1863e-05 2020-09-08T15:40:04.165 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.166 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.166 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:04.166 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.996254+0000", 2020-09-08T15:40:04.166 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.2359999999999998e-06 2020-09-08T15:40:04.167 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.167 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.167 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:04.167 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:34.996258+0000", 2020-09-08T15:40:04.167 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.026807951 2020-09-08T15:40:04.168 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.168 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.168 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:04.168 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.023066+0000", 2020-09-08T15:40:04.168 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.0833e-05 2020-09-08T15:40:04.169 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.169 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.169 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:04.169 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.023107+0000", 2020-09-08T15:40:04.169 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0094870379999999997 2020-09-08T15:40:04.170 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.170 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.170 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:04.170 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.032594+0000", 2020-09-08T15:40:04.170 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3774000000000003e-05 2020-09-08T15:40:04.170 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.171 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.171 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:04.171 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:35.032658+0000", 2020-09-08T15:40:04.171 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.036544348999999997 2020-09-08T15:40:04.171 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.172 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.172 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.172 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.172 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.172 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:128176 2.d e159/126 2:b14ea5a4:::benchmark_data_smithi166_20551_object128175:head v 159'8033, mlcod=159'8031)", 2020-09-08T15:40:04.173 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:54.620593+0000", 2020-09-08T15:40:04.173 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 69.401774099999997, 2020-09-08T15:40:04.173 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043179051000000003, 2020-09-08T15:40:04.173 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:04.173 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:04.174 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:04.174 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.174 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:04.174 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.620593+0000", 2020-09-08T15:40:04.174 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:04.174 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.175 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.175 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:04.175 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.620593+0000", 2020-09-08T15:40:04.175 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00027082799999999998 2020-09-08T15:40:04.175 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.176 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.176 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:04.176 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.620864+0000", 2020-09-08T15:40:04.176 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3526000000000001e-05 2020-09-08T15:40:04.177 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.177 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.177 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:04.177 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.620877+0000", 2020-09-08T15:40:04.177 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.0452e-05 2020-09-08T15:40:04.178 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.178 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.178 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:04.178 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.620898+0000", 2020-09-08T15:40:04.178 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0383e-05 2020-09-08T15:40:04.179 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.179 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.179 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:04.179 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.620908+0000", 2020-09-08T15:40:04.179 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.1581999999999993e-05 2020-09-08T15:40:04.180 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.180 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.180 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:04.180 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.620980+0000", 2020-09-08T15:40:04.180 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.0502999999999999e-05 2020-09-08T15:40:04.181 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.181 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.181 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:04.181 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.621010+0000", 2020-09-08T15:40:04.181 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.042675626000000001 2020-09-08T15:40:04.181 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.182 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.182 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:04.182 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.663686+0000", 2020-09-08T15:40:04.182 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.6150999999999996e-05 2020-09-08T15:40:04.182 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.183 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.183 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:04.183 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.663772+0000", 2020-09-08T15:40:04.183 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043179051000000003 2020-09-08T15:40:04.183 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.184 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.184 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.184 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.184 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.184 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:128188 2.d e159/126 2:b453b1b7:::benchmark_data_smithi166_20551_object128187:head v 159'8034, mlcod=159'8031)", 2020-09-08T15:40:04.185 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:54.637631+0000", 2020-09-08T15:40:04.185 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 69.384736520000004, 2020-09-08T15:40:04.185 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.026904603999999999, 2020-09-08T15:40:04.185 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:04.185 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:04.186 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:04.186 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.186 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:04.187 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.637631+0000", 2020-09-08T15:40:04.187 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:04.187 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.187 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.187 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:04.188 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.637631+0000", 2020-09-08T15:40:04.188 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010555 2020-09-08T15:40:04.188 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.188 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.188 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:04.189 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.637736+0000", 2020-09-08T15:40:04.189 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4679999999999992e-06 2020-09-08T15:40:04.189 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.189 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.189 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:04.189 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.637745+0000", 2020-09-08T15:40:04.190 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6390999999999999e-05 2020-09-08T15:40:04.190 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.190 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.190 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:04.190 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.637761+0000", 2020-09-08T15:40:04.191 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4200000000000007e-06 2020-09-08T15:40:04.191 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.191 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.191 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:04.191 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.637769+0000", 2020-09-08T15:40:04.192 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0273999999999999e-05 2020-09-08T15:40:04.192 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.192 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.192 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:04.192 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.637830+0000", 2020-09-08T15:40:04.192 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.5939000000000003e-05 2020-09-08T15:40:04.193 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.193 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.193 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:04.193 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.637866+0000", 2020-09-08T15:40:04.193 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.026440608000000001 2020-09-08T15:40:04.194 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.194 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.194 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:04.194 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.664306+0000", 2020-09-08T15:40:04.194 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00022895399999999999 2020-09-08T15:40:04.195 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.195 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.195 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:04.195 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.664535+0000", 2020-09-08T15:40:04.195 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.026904603999999999 2020-09-08T15:40:04.196 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.196 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.196 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.196 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.196 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.197 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:134538 2.d e160/126 2:b094cd69:::benchmark_data_smithi166_20551_object134537:head v 160'8449, mlcod=160'8445)", 2020-09-08T15:40:04.197 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:59.959309+0000", 2020-09-08T15:40:04.197 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 64.063058429999998, 2020-09-08T15:40:04.197 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.022414020999999999, 2020-09-08T15:40:04.198 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:04.198 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:04.198 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:04.198 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.198 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:04.198 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:59.959309+0000", 2020-09-08T15:40:04.199 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:04.199 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.199 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.199 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:04.199 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:59.959309+0000", 2020-09-08T15:40:04.200 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000130247 2020-09-08T15:40:04.200 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.200 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.200 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:04.200 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:59.959439+0000", 2020-09-08T15:40:04.201 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.1179999999999992e-06 2020-09-08T15:40:04.201 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.201 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.201 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:04.201 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:59.959447+0000", 2020-09-08T15:40:04.202 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.292e-05 2020-09-08T15:40:04.202 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.202 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.202 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:04.202 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:59.959460+0000", 2020-09-08T15:40:04.202 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3280000000000002e-06 2020-09-08T15:40:04.203 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.203 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.203 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:04.203 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:59.959466+0000", 2020-09-08T15:40:04.203 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.0935999999999999e-05 2020-09-08T15:40:04.204 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.204 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.204 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:04.204 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:59.959547+0000", 2020-09-08T15:40:04.204 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.5998000000000002e-05 2020-09-08T15:40:04.205 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.205 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.205 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:04.205 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:59.959593+0000", 2020-09-08T15:40:04.205 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.022020580000000001 2020-09-08T15:40:04.205 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.206 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.206 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:04.206 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:59.981614+0000", 2020-09-08T15:40:04.206 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000108894 2020-09-08T15:40:04.206 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.207 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.207 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:04.209 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:59.981723+0000", 2020-09-08T15:40:04.209 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.022414020999999999 2020-09-08T15:40:04.210 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.210 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.210 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.210 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.211 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.211 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:143289 2.d e163/126 2:bbe86b89:::benchmark_data_smithi166_20551_object143288:head v 163'9029, mlcod=163'9025)", 2020-09-08T15:40:04.211 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:08.017537+0000", 2020-09-08T15:40:04.211 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 56.004829870999998, 2020-09-08T15:40:04.211 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.023724347999999999, 2020-09-08T15:40:04.212 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:04.212 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:04.213 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:04.213 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.213 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:04.213 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.017537+0000", 2020-09-08T15:40:04.213 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:04.214 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.214 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.215 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:04.215 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.017537+0000", 2020-09-08T15:40:04.215 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017966400000000001 2020-09-08T15:40:04.215 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.215 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.216 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:04.216 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.017717+0000", 2020-09-08T15:40:04.216 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2303999999999999e-05 2020-09-08T15:40:04.216 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.216 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.217 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:04.217 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.017729+0000", 2020-09-08T15:40:04.217 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.4139e-05 2020-09-08T15:40:04.217 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.218 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.218 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:04.218 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.017763+0000", 2020-09-08T15:40:04.218 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.0169999999999996e-06 2020-09-08T15:40:04.219 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.219 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.219 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:04.219 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.017771+0000", 2020-09-08T15:40:04.219 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0039160319999999998 2020-09-08T15:40:04.220 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.220 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.220 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:04.220 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.021687+0000", 2020-09-08T15:40:04.221 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.7452e-05 2020-09-08T15:40:04.221 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.221 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.221 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:04.222 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.021725+0000", 2020-09-08T15:40:04.222 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.019462690000000001 2020-09-08T15:40:04.222 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.222 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.222 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:04.223 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.041188+0000", 2020-09-08T15:40:04.223 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.4049999999999997e-05 2020-09-08T15:40:04.223 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.223 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.223 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:04.224 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.041262+0000", 2020-09-08T15:40:04.224 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.023724347999999999 2020-09-08T15:40:04.224 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.224 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.224 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.225 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.225 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.225 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:169658 2.d e172/126 2:b75af67b:::benchmark_data_smithi166_20551_object22038:head v 172'10685, mlcod=172'10682)", 2020-09-08T15:40:04.225 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:35.897217+0000", 2020-09-08T15:40:04.226 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 28.125150394999999, 2020-09-08T15:40:04.226 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.025305820999999999, 2020-09-08T15:40:04.226 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:04.226 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:04.227 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:04.227 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.228 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:04.228 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:35.897217+0000", 2020-09-08T15:40:04.228 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:04.228 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.229 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.229 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:04.229 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:35.897217+0000", 2020-09-08T15:40:04.229 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017033400000000001 2020-09-08T15:40:04.230 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.230 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.230 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:04.230 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:35.897387+0000", 2020-09-08T15:40:04.231 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6821999999999999e-05 2020-09-08T15:40:04.231 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.231 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.232 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:04.232 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:35.897404+0000", 2020-09-08T15:40:04.232 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.1797000000000001e-05 2020-09-08T15:40:04.232 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.232 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.233 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:04.233 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:35.897436+0000", 2020-09-08T15:40:04.233 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5829999999999999e-05 2020-09-08T15:40:04.233 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.234 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.234 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:04.234 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:35.897452+0000", 2020-09-08T15:40:04.234 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000101273 2020-09-08T15:40:04.234 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.235 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.235 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:04.235 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:35.897553+0000", 2020-09-08T15:40:04.235 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.1261999999999999e-05 2020-09-08T15:40:04.236 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.236 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.236 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:04.236 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:35.897594+0000", 2020-09-08T15:40:04.236 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.024864001 2020-09-08T15:40:04.237 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.237 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.237 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:04.237 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:35.922458+0000", 2020-09-08T15:40:04.237 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.4502e-05 2020-09-08T15:40:04.238 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.239 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.239 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:04.239 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:35.922523+0000", 2020-09-08T15:40:04.239 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.025305820999999999 2020-09-08T15:40:04.240 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.240 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.240 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.240 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.240 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.241 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:169662 2.d e172/126 2:bac2d8a9:::benchmark_data_smithi166_20551_object22042:head v 172'10686, mlcod=172'10682)", 2020-09-08T15:40:04.241 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:35.897595+0000", 2020-09-08T15:40:04.241 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 28.124772197999999, 2020-09-08T15:40:04.242 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.025002475, 2020-09-08T15:40:04.242 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:04.242 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:04.242 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:04.242 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.242 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:04.243 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:35.897595+0000", 2020-09-08T15:40:04.243 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:04.243 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.243 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.243 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:04.244 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:35.897595+0000", 2020-09-08T15:40:04.244 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.7431999999999999e-05 2020-09-08T15:40:04.244 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.244 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.244 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:04.245 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:35.897682+0000", 2020-09-08T15:40:04.245 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3730000000000001e-05 2020-09-08T15:40:04.245 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.245 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.245 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:04.246 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:35.897696+0000", 2020-09-08T15:40:04.246 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.7311999999999998e-05 2020-09-08T15:40:04.246 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.246 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.247 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:04.247 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:35.897733+0000", 2020-09-08T15:40:04.247 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.8800000000000002e-06 2020-09-08T15:40:04.248 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.248 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.248 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:04.248 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:35.897740+0000", 2020-09-08T15:40:04.248 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0058620499999999997 2020-09-08T15:40:04.249 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.249 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.249 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:04.249 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:35.903602+0000", 2020-09-08T15:40:04.249 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.6554999999999998e-05 2020-09-08T15:40:04.250 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.250 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.250 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:04.250 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:35.903639+0000", 2020-09-08T15:40:04.250 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.018886785999999999 2020-09-08T15:40:04.251 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.251 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.251 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:04.251 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:35.922526+0000", 2020-09-08T15:40:04.251 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.1730000000000006e-05 2020-09-08T15:40:04.251 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.252 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.252 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:04.252 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:35.922597+0000", 2020-09-08T15:40:04.252 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.025002475 2020-09-08T15:40:04.252 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.253 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.253 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.253 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.253 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.253 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:175840 2.d e174/126 2:bc210f65:::benchmark_data_smithi166_20551_object28220:head v 174'11066, mlcod=174'11064)", 2020-09-08T15:40:04.254 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:42.053051+0000", 2020-09-08T15:40:04.254 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 21.969316250999999, 2020-09-08T15:40:04.254 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.021616442, 2020-09-08T15:40:04.254 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:04.255 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:04.255 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:04.255 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.255 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:04.255 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.053051+0000", 2020-09-08T15:40:04.256 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:04.256 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.256 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.256 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:04.256 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.053051+0000", 2020-09-08T15:40:04.257 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000112308 2020-09-08T15:40:04.257 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.257 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.257 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:04.259 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.053163+0000", 2020-09-08T15:40:04.259 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.8564e-05 2020-09-08T15:40:04.260 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.260 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.260 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:04.260 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.053192+0000", 2020-09-08T15:40:04.260 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.7634e-05 2020-09-08T15:40:04.261 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.261 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.261 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:04.261 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.053219+0000", 2020-09-08T15:40:04.262 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3328e-05 2020-09-08T15:40:04.262 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.262 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.262 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:04.262 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.053233+0000", 2020-09-08T15:40:04.263 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4870000000000006e-05 2020-09-08T15:40:04.263 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.263 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.263 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:04.263 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.053318+0000", 2020-09-08T15:40:04.264 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.8670000000000001e-05 2020-09-08T15:40:04.264 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.264 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.264 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:04.264 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.053356+0000", 2020-09-08T15:40:04.264 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.021230837999999998 2020-09-08T15:40:04.265 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.265 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.265 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:04.265 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.074587+0000", 2020-09-08T15:40:04.265 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.0229999999999996e-05 2020-09-08T15:40:04.266 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.266 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.266 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:04.267 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.074667+0000", 2020-09-08T15:40:04.267 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.021616442 2020-09-08T15:40:04.267 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.267 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.267 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.268 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.268 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.268 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:175850 2.d e174/126 2:b4e10947:::benchmark_data_smithi166_20551_object28230:head v 174'11068, mlcod=174'11064)", 2020-09-08T15:40:04.268 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:42.057201+0000", 2020-09-08T15:40:04.269 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 21.965166622000002, 2020-09-08T15:40:04.269 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.021684001000000001, 2020-09-08T15:40:04.269 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:04.269 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:04.270 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:04.270 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.270 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:04.270 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.057201+0000", 2020-09-08T15:40:04.270 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:04.271 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.271 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.271 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:04.271 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.057201+0000", 2020-09-08T15:40:04.272 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.7158999999999999e-05 2020-09-08T15:40:04.272 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.272 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.272 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:04.273 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.057268+0000", 2020-09-08T15:40:04.273 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.9120000000000004e-06 2020-09-08T15:40:04.273 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.273 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.273 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:04.274 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.057277+0000", 2020-09-08T15:40:04.274 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4031e-05 2020-09-08T15:40:04.274 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.275 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.275 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:04.275 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.057291+0000", 2020-09-08T15:40:04.275 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.8360000000000003e-06 2020-09-08T15:40:04.275 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.276 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.276 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:04.276 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.057297+0000", 2020-09-08T15:40:04.276 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0086411049999999996 2020-09-08T15:40:04.276 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.277 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.277 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:04.277 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.065939+0000", 2020-09-08T15:40:04.277 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.0397e-05 2020-09-08T15:40:04.277 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.278 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.278 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:04.278 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.065979+0000", 2020-09-08T15:40:04.280 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.012841197 2020-09-08T15:40:04.280 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.280 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.281 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:04.281 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.078820+0000", 2020-09-08T15:40:04.281 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.4363999999999996e-05 2020-09-08T15:40:04.281 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:04.282 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:04.282 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:04.282 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.078885+0000", 2020-09-08T15:40:04.282 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.021684001000000001 2020-09-08T15:40:04.283 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.283 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.283 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.283 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:04.284 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:04.284 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:40:04.284 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:04.286 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:40:03 smithi104 bash[22305]: debug 2020-09-08T15:40:03.709+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:04.286 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:03 smithi104 bash[9203]: cluster 2020-09-08T15:40:03.629959+0000 mgr.x (mgr.24118) 376 : cluster [DBG] pgmap v478: 17 pgs: 17 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.17k op/s 2020-09-08T15:40:04.287 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:03 smithi055 bash[9296]: cluster 2020-09-08T15:40:03.629959+0000 mgr.x (mgr.24118) 376 : cluster [DBG] pgmap v478: 17 pgs: 17 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.17k op/s 2020-09-08T15:40:04.756 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:04.859 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:40:04.875 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2020-09-08T15:40:05.014 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:40:05.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:40:04 smithi055 bash[16646]: debug 2020-09-08T15:40:04.755+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:05.259 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:40:05.363 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:05.673 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:05.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:40:05 smithi104 bash[17588]: debug 2020-09-08T15:40:05.253+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:05.699 INFO:teuthology.orchestra.run.smithi104.stderr:reweighted osd.8 to 1 (10000) 2020-09-08T15:40:05.776 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:06.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:05 smithi166 bash[9446]: cluster 2020-09-08T15:40:04.549410+0000 osd.7 (osd.7) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:06.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:05 smithi166 bash[9446]: cluster 2020-09-08T15:40:04.549529+0000 osd.7 (osd.7) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:06.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:05 smithi166 bash[9446]: cluster 2020-09-08T15:40:04.694906+0000 mon.a (mon.0) 1020 : cluster [DBG] osdmap e181: 12 total, 12 up, 12 in 2020-09-08T15:40:06.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:05 smithi166 bash[9446]: cluster 2020-09-08T15:40:05.630391+0000 mgr.x (mgr.24118) 377 : cluster [DBG] pgmap v480: 17 pgs: 17 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 765 op/s 2020-09-08T15:40:06.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:05 smithi166 bash[9446]: audit 2020-09-08T15:40:05.675906+0000 mon.a (mon.0) 1021 : audit [INF] from='client.? v1:172.21.15.104:0/2370317436' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 8, "weight": 1.0}]: dispatch 2020-09-08T15:40:06.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:40:05 smithi055 bash[11956]: debug 2020-09-08T15:40:05.675+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:06.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:05 smithi055 bash[9296]: cluster 2020-09-08T15:40:04.549410+0000 osd.7 (osd.7) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:06.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:05 smithi055 bash[9296]: cluster 2020-09-08T15:40:04.549529+0000 osd.7 (osd.7) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:06.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:05 smithi055 bash[9296]: cluster 2020-09-08T15:40:04.694906+0000 mon.a (mon.0) 1020 : cluster [DBG] osdmap e181: 12 total, 12 up, 12 in 2020-09-08T15:40:06.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:05 smithi055 bash[9296]: cluster 2020-09-08T15:40:05.630391+0000 mgr.x (mgr.24118) 377 : cluster [DBG] pgmap v480: 17 pgs: 17 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 765 op/s 2020-09-08T15:40:06.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:05 smithi055 bash[9296]: audit 2020-09-08T15:40:05.675906+0000 mon.a (mon.0) 1021 : audit [INF] from='client.? v1:172.21.15.104:0/2370317436' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 8, "weight": 1.0}]: dispatch 2020-09-08T15:40:06.173 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:05 smithi104 bash[9203]: cluster 2020-09-08T15:40:04.549410+0000 osd.7 (osd.7) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:06.174 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:05 smithi104 bash[9203]: cluster 2020-09-08T15:40:04.549529+0000 osd.7 (osd.7) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:06.174 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:05 smithi104 bash[9203]: cluster 2020-09-08T15:40:04.694906+0000 mon.a (mon.0) 1020 : cluster [DBG] osdmap e181: 12 total, 12 up, 12 in 2020-09-08T15:40:06.174 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:05 smithi104 bash[9203]: cluster 2020-09-08T15:40:05.630391+0000 mgr.x (mgr.24118) 377 : cluster [DBG] pgmap v480: 17 pgs: 17 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 765 op/s 2020-09-08T15:40:06.174 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:05 smithi104 bash[9203]: audit 2020-09-08T15:40:05.675906+0000 mon.a (mon.0) 1021 : audit [INF] from='client.? v1:172.21.15.104:0/2370317436' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 8, "weight": 1.0}]: dispatch 2020-09-08T15:40:06.280 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:06.384 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:06.527 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:06.565 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:40:06 smithi055 bash[16646]: debug 2020-09-08T15:40:06.279+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:06.634 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:40:06.700 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:40:06 smithi166 bash[11928]: debug 2020-09-08T15:40:06.522+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:06.747 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:40:06.841 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:06 smithi055 bash[9296]: audit 2020-09-08T15:40:05.696437+0000 mon.a (mon.0) 1022 : audit [INF] from='client.? v1:172.21.15.104:0/2370317436' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 8, "weight": 1.0}]': finished 2020-09-08T15:40:06.841 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:06 smithi055 bash[9296]: cluster 2020-09-08T15:40:05.696516+0000 mon.a (mon.0) 1023 : cluster [DBG] osdmap e182: 12 total, 12 up, 12 in 2020-09-08T15:40:06.853 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:07.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:06 smithi166 bash[9446]: audit 2020-09-08T15:40:05.696437+0000 mon.a (mon.0) 1022 : audit [INF] from='client.? v1:172.21.15.104:0/2370317436' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 8, "weight": 1.0}]': finished 2020-09-08T15:40:07.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:06 smithi166 bash[9446]: cluster 2020-09-08T15:40:05.696516+0000 mon.a (mon.0) 1023 : cluster [DBG] osdmap e182: 12 total, 12 up, 12 in 2020-09-08T15:40:07.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:40:06 smithi166 bash[18960]: debug 2020-09-08T15:40:06.746+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:07.166 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:06 smithi104 bash[9203]: audit 2020-09-08T15:40:05.696437+0000 mon.a (mon.0) 1022 : audit [INF] from='client.? v1:172.21.15.104:0/2370317436' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 8, "weight": 1.0}]': finished 2020-09-08T15:40:07.166 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:06 smithi104 bash[9203]: cluster 2020-09-08T15:40:05.696516+0000 mon.a (mon.0) 1023 : cluster [DBG] osdmap e182: 12 total, 12 up, 12 in 2020-09-08T15:40:07.169 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:07.272 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:07.432 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:40:07 smithi104 bash[19922]: debug 2020-09-08T15:40:07.169+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:07.584 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd reweight 9 1 2020-09-08T15:40:07.746 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:40:07.831 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:40:07.863 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:40:07.892 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:40:07.921 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:40:07.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:07 smithi104 bash[9203]: cluster 2020-09-08T15:40:06.135420+0000 osd.0 (osd.0) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:07.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:07 smithi104 bash[9203]: cluster 2020-09-08T15:40:06.135510+0000 osd.0 (osd.0) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:07.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:07 smithi104 bash[9203]: cluster 2020-09-08T15:40:06.714844+0000 mon.a (mon.0) 1024 : cluster [DBG] osdmap e183: 12 total, 12 up, 12 in 2020-09-08T15:40:07.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:07 smithi104 bash[9203]: cluster 2020-09-08T15:40:07.630895+0000 mgr.x (mgr.24118) 378 : cluster [DBG] pgmap v483: 17 pgs: 17 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.36k op/s 2020-09-08T15:40:07.951 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:40:07.993 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:40:08.015 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:08.027 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:40:08.067 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:40:08.102 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:40:08.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:07 smithi166 bash[9446]: cluster 2020-09-08T15:40:06.135420+0000 osd.0 (osd.0) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:08.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:07 smithi166 bash[9446]: cluster 2020-09-08T15:40:06.135510+0000 osd.0 (osd.0) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:08.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:07 smithi166 bash[9446]: cluster 2020-09-08T15:40:06.714844+0000 mon.a (mon.0) 1024 : cluster [DBG] osdmap e183: 12 total, 12 up, 12 in 2020-09-08T15:40:08.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:07 smithi166 bash[9446]: cluster 2020-09-08T15:40:07.630895+0000 mgr.x (mgr.24118) 378 : cluster [DBG] pgmap v483: 17 pgs: 17 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.36k op/s 2020-09-08T15:40:08.119 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:40:08.141 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:40:08.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:07 smithi055 bash[9296]: cluster 2020-09-08T15:40:06.135420+0000 osd.0 (osd.0) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:08.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:07 smithi055 bash[9296]: cluster 2020-09-08T15:40:06.135510+0000 osd.0 (osd.0) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:08.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:07 smithi055 bash[9296]: cluster 2020-09-08T15:40:06.714844+0000 mon.a (mon.0) 1024 : cluster [DBG] osdmap e183: 12 total, 12 up, 12 in 2020-09-08T15:40:08.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:07 smithi055 bash[9296]: cluster 2020-09-08T15:40:07.630895+0000 mgr.x (mgr.24118) 378 : cluster [DBG] pgmap v483: 17 pgs: 17 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.36k op/s 2020-09-08T15:40:08.172 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:40:08.203 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:40:08.207 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:40:08.208 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:40:08.208 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:40:08.208 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:40:08.208 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:40:08.208 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:40:08.209 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:40:08.209 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:40:08.209 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:40:08.209 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:40:08.209 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:40:08.210 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:40:08.239 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:40:08.354 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:40:08.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:40:08 smithi104 bash[22305]: debug 2020-09-08T15:40:08.009+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:08.615 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:40:08 smithi166 bash[14254]: debug 2020-09-08T15:40:08.238+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:08.715 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:08 smithi104 bash[9203]: audit 2020-09-08T15:40:07.793925+0000 mon.c (mon.2) 40 : audit [DBG] from='client.? v1:172.21.15.104:0/3904012169' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:40:08.843 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:40:08.946 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:08.976 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:08 smithi055 bash[9296]: audit 2020-09-08T15:40:07.793925+0000 mon.c (mon.2) 40 : audit [DBG] from='client.? v1:172.21.15.104:0/3904012169' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:40:09.099 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:09.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:08 smithi166 bash[9446]: audit 2020-09-08T15:40:07.793925+0000 mon.c (mon.2) 40 : audit [DBG] from='client.? v1:172.21.15.104:0/3904012169' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:40:09.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:40:08 smithi166 bash[18960]: debug 2020-09-08T15:40:08.846+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:09.138 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:40:09.139 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:40:09.203 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:09.315 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:09.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:40:09 smithi055 bash[11956]: debug 2020-09-08T15:40:09.099+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:09.417 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:40:09.539 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:40:09.540 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:40:09 smithi166 bash[16593]: debug 2020-09-08T15:40:09.314+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:09.646 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:09.762 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:09.865 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:40:09 smithi166 bash[14254]: debug 2020-09-08T15:40:09.538+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:09.865 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:40:09 smithi166 bash[16593]: debug 2020-09-08T15:40:09.762+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:09.867 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:09.996 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:10.089 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2020-09-08T15:40:10.104 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:40:10.253 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:40:10.296 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:40:10.357 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:10.364 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:40:10 smithi055 bash[14301]: debug 2020-09-08T15:40:10.251+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:10.365 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:40:09 smithi166 bash[16593]: debug 2020-09-08T15:40:09.994+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:10.662 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:10 smithi055 bash[9296]: cluster 2020-09-08T15:40:09.631376+0000 mgr.x (mgr.24118) 379 : cluster [DBG] pgmap v484: 17 pgs: 17 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 854 op/s 2020-09-08T15:40:10.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:10 smithi166 bash[9446]: cluster 2020-09-08T15:40:09.631376+0000 mgr.x (mgr.24118) 379 : cluster [DBG] pgmap v484: 17 pgs: 17 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 854 op/s 2020-09-08T15:40:10.892 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:10.922 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:10 smithi104 bash[9203]: cluster 2020-09-08T15:40:09.631376+0000 mgr.x (mgr.24118) 379 : cluster [DBG] pgmap v484: 17 pgs: 17 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 854 op/s 2020-09-08T15:40:10.997 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:40:11.127 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:40:11.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:40:10 smithi055 bash[11956]: debug 2020-09-08T15:40:10.891+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:11.232 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:11.347 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:40:11 smithi166 bash[18960]: debug 2020-09-08T15:40:11.126+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:11.350 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:11.454 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:40:11.558 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:40:11.615 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:40:11 smithi166 bash[11928]: debug 2020-09-08T15:40:11.350+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:11.615 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:40:11 smithi166 bash[18960]: debug 2020-09-08T15:40:11.558+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:11.665 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:40:11.805 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:40:11.909 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:40:12.033 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:11 smithi166 bash[9446]: cluster 2020-09-08T15:40:11.631758+0000 mgr.x (mgr.24118) 380 : cluster [DBG] pgmap v485: 17 pgs: 17 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 738 op/s 2020-09-08T15:40:12.034 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:40:12.078 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:40:12.086 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:11 smithi104 bash[9203]: cluster 2020-09-08T15:40:11.631758+0000 mgr.x (mgr.24118) 380 : cluster [DBG] pgmap v485: 17 pgs: 17 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 738 op/s 2020-09-08T15:40:12.091 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:40:11 smithi055 bash[19003]: debug 2020-09-08T15:40:11.803+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:12.092 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:11 smithi055 bash[9296]: cluster 2020-09-08T15:40:11.631758+0000 mgr.x (mgr.24118) 380 : cluster [DBG] pgmap v485: 17 pgs: 17 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 738 op/s 2020-09-08T15:40:12.141 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:40:12.241 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:40:12.279 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:40:12.365 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:40:12 smithi166 bash[18960]: debug 2020-09-08T15:40:12.034+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:12.382 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:12.392 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:40:12 smithi055 bash[14301]: debug 2020-09-08T15:40:12.279+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:12.707 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:40:12.880 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:12.991 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:40:13.007 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:12 smithi166 bash[9446]: audit 2020-09-08T15:40:11.919466+0000 mon.b (mon.1) 79 : audit [INF] from='client.? v1:172.21.15.104:0/955041181' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:40:13.007 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:12 smithi166 bash[9446]: audit 2020-09-08T15:40:11.919668+0000 mon.a (mon.0) 1025 : audit [INF] from='client.24895 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:40:13.008 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:12 smithi166 bash[9446]: cluster 2020-09-08T15:40:12.236113+0000 osd.5 (osd.5) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:13.008 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:12 smithi166 bash[9446]: cluster 2020-09-08T15:40:12.236234+0000 osd.5 (osd.5) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:13.146 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:40:13.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:40:12 smithi055 bash[11956]: debug 2020-09-08T15:40:12.879+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:13.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:12 smithi055 bash[9296]: audit 2020-09-08T15:40:11.919466+0000 mon.b (mon.1) 79 : audit [INF] from='client.? v1:172.21.15.104:0/955041181' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:40:13.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:12 smithi055 bash[9296]: audit 2020-09-08T15:40:11.919668+0000 mon.a (mon.0) 1025 : audit [INF] from='client.24895 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:40:13.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:12 smithi055 bash[9296]: cluster 2020-09-08T15:40:12.236113+0000 osd.5 (osd.5) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:13.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:12 smithi055 bash[9296]: cluster 2020-09-08T15:40:12.236234+0000 osd.5 (osd.5) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:13.188 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:12 smithi104 bash[9203]: audit 2020-09-08T15:40:11.919466+0000 mon.b (mon.1) 79 : audit [INF] from='client.? v1:172.21.15.104:0/955041181' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:40:13.188 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:12 smithi104 bash[9203]: audit 2020-09-08T15:40:11.919668+0000 mon.a (mon.0) 1025 : audit [INF] from='client.24895 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:40:13.188 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:12 smithi104 bash[9203]: cluster 2020-09-08T15:40:12.236113+0000 osd.5 (osd.5) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:13.189 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:12 smithi104 bash[9203]: cluster 2020-09-08T15:40:12.236234+0000 osd.5 (osd.5) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:13.252 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:40:13.365 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:40:13 smithi166 bash[18960]: debug 2020-09-08T15:40:13.141+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:13.627 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:40:13.730 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:40:13.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:40:13 smithi055 bash[19003]: debug 2020-09-08T15:40:13.627+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:13.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:13 smithi055 bash[9296]: cluster 2020-09-08T15:40:12.694931+0000 mon.a (mon.0) 1026 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:40:13.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:13 smithi055 bash[9296]: audit 2020-09-08T15:40:12.708460+0000 mon.a (mon.0) 1027 : audit [INF] from='client.24895 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:40:13.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:13 smithi055 bash[9296]: cluster 2020-09-08T15:40:12.708528+0000 mon.a (mon.0) 1028 : cluster [DBG] osdmap e184: 12 total, 12 up, 12 in 2020-09-08T15:40:13.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:13 smithi055 bash[9296]: cluster 2020-09-08T15:40:13.632240+0000 mgr.x (mgr.24118) 381 : cluster [DBG] pgmap v487: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.28k op/s 2020-09-08T15:40:14.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:13 smithi166 bash[9446]: cluster 2020-09-08T15:40:12.694931+0000 mon.a (mon.0) 1026 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:40:14.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:13 smithi166 bash[9446]: audit 2020-09-08T15:40:12.708460+0000 mon.a (mon.0) 1027 : audit [INF] from='client.24895 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:40:14.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:13 smithi166 bash[9446]: cluster 2020-09-08T15:40:12.708528+0000 mon.a (mon.0) 1028 : cluster [DBG] osdmap e184: 12 total, 12 up, 12 in 2020-09-08T15:40:14.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:13 smithi166 bash[9446]: cluster 2020-09-08T15:40:13.632240+0000 mgr.x (mgr.24118) 381 : cluster [DBG] pgmap v487: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.28k op/s 2020-09-08T15:40:14.158 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:13 smithi104 bash[9203]: cluster 2020-09-08T15:40:12.694931+0000 mon.a (mon.0) 1026 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:40:14.158 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:13 smithi104 bash[9203]: audit 2020-09-08T15:40:12.708460+0000 mon.a (mon.0) 1027 : audit [INF] from='client.24895 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:40:14.159 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:13 smithi104 bash[9203]: cluster 2020-09-08T15:40:12.708528+0000 mon.a (mon.0) 1028 : cluster [DBG] osdmap e184: 12 total, 12 up, 12 in 2020-09-08T15:40:14.159 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:13 smithi104 bash[9203]: cluster 2020-09-08T15:40:13.632240+0000 mgr.x (mgr.24118) 381 : cluster [DBG] pgmap v487: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.28k op/s 2020-09-08T15:40:14.408 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:40:14.513 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:14.646 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:14.710 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:40:14 smithi055 bash[19003]: debug 2020-09-08T15:40:14.407+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:14.752 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:14.877 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:14 smithi104 bash[9203]: cluster 2020-09-08T15:40:12.784365+0000 osd.7 (osd.7) 33 : cluster [DBG] 2.0 scrub starts 2020-09-08T15:40:14.878 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:14 smithi104 bash[9203]: cluster 2020-09-08T15:40:13.564748+0000 osd.11 (osd.11) 17 : cluster [DBG] 2.c scrub starts 2020-09-08T15:40:14.878 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:14 smithi166 bash[9446]: cluster 2020-09-08T15:40:12.784365+0000 osd.7 (osd.7) 33 : cluster [DBG] 2.0 scrub starts 2020-09-08T15:40:14.878 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:14 smithi166 bash[9446]: cluster 2020-09-08T15:40:13.564748+0000 osd.11 (osd.11) 17 : cluster [DBG] 2.c scrub starts 2020-09-08T15:40:14.879 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:40:14 smithi166 bash[16593]: debug 2020-09-08T15:40:14.645+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:14.899 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:14.968 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:40:14.969 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:40:15.003 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:15.058 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:40:14 smithi055 bash[16646]: debug 2020-09-08T15:40:14.895+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:15.059 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:14 smithi055 bash[9296]: cluster 2020-09-08T15:40:12.784365+0000 osd.7 (osd.7) 33 : cluster [DBG] 2.0 scrub starts 2020-09-08T15:40:15.059 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:14 smithi055 bash[9296]: cluster 2020-09-08T15:40:13.564748+0000 osd.11 (osd.11) 17 : cluster [DBG] 2.c scrub starts 2020-09-08T15:40:15.126 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:15.143 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:40:15 smithi104 bash[15273]: debug 2020-09-08T15:40:15.125+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:15.230 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:40:15.719 INFO:teuthology.orchestra.run.smithi104.stderr:reweighted osd.9 to 1 (10000) 2020-09-08T15:40:15.934 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:40:16.037 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:16.047 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:15 smithi166 bash[9446]: cluster 2020-09-08T15:40:14.389371+0000 osd.7 (osd.7) 34 : cluster [DBG] 2.0 scrub ok 2020-09-08T15:40:16.047 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:15 smithi166 bash[9446]: audit 2020-09-08T15:40:15.112788+0000 mon.a (mon.0) 1029 : audit [INF] from='client.? v1:172.21.15.104:0/149327661' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 9, "weight": 1.0}]: dispatch 2020-09-08T15:40:16.049 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:15 smithi166 bash[9446]: cluster 2020-09-08T15:40:15.632760+0000 mgr.x (mgr.24118) 382 : cluster [DBG] pgmap v488: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 923 op/s 2020-09-08T15:40:16.093 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:40:15 smithi055 bash[19003]: debug 2020-09-08T15:40:15.935+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:16.093 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:15 smithi055 bash[9296]: cluster 2020-09-08T15:40:14.389371+0000 osd.7 (osd.7) 34 : cluster [DBG] 2.0 scrub ok 2020-09-08T15:40:16.093 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:15 smithi055 bash[9296]: audit 2020-09-08T15:40:15.112788+0000 mon.a (mon.0) 1029 : audit [INF] from='client.? v1:172.21.15.104:0/149327661' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 9, "weight": 1.0}]: dispatch 2020-09-08T15:40:16.094 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:15 smithi055 bash[9296]: cluster 2020-09-08T15:40:15.632760+0000 mgr.x (mgr.24118) 382 : cluster [DBG] pgmap v488: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 923 op/s 2020-09-08T15:40:16.094 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2020-09-08T15:40:16.160 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:40:16.162 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:16.170 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:15 smithi104 bash[9203]: cluster 2020-09-08T15:40:14.389371+0000 osd.7 (osd.7) 34 : cluster [DBG] 2.0 scrub ok 2020-09-08T15:40:16.171 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:15 smithi104 bash[9203]: audit 2020-09-08T15:40:15.112788+0000 mon.a (mon.0) 1029 : audit [INF] from='client.? v1:172.21.15.104:0/149327661' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 9, "weight": 1.0}]: dispatch 2020-09-08T15:40:16.171 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:15 smithi104 bash[9203]: cluster 2020-09-08T15:40:15.632760+0000 mgr.x (mgr.24118) 382 : cluster [DBG] pgmap v488: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 923 op/s 2020-09-08T15:40:16.251 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:40:16.266 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:16.338 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:40:16.365 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:40:16 smithi166 bash[16593]: debug 2020-09-08T15:40:16.157+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:16.943 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:17.045 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:17.053 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:16 smithi104 bash[9203]: cluster 2020-09-08T15:40:15.718175+0000 osd.11 (osd.11) 18 : cluster [DBG] 2.c scrub ok 2020-09-08T15:40:17.053 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:16 smithi104 bash[9203]: audit 2020-09-08T15:40:15.721661+0000 mon.a (mon.0) 1030 : audit [INF] from='client.? v1:172.21.15.104:0/149327661' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 9, "weight": 1.0}]': finished 2020-09-08T15:40:17.054 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:16 smithi104 bash[9203]: cluster 2020-09-08T15:40:15.721703+0000 mon.a (mon.0) 1031 : cluster [DBG] osdmap e185: 12 total, 12 up, 12 in 2020-09-08T15:40:17.054 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:40:16 smithi104 bash[15273]: debug 2020-09-08T15:40:16.945+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:17.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:16 smithi166 bash[9446]: cluster 2020-09-08T15:40:15.718175+0000 osd.11 (osd.11) 18 : cluster [DBG] 2.c scrub ok 2020-09-08T15:40:17.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:16 smithi166 bash[9446]: audit 2020-09-08T15:40:15.721661+0000 mon.a (mon.0) 1030 : audit [INF] from='client.? v1:172.21.15.104:0/149327661' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 9, "weight": 1.0}]': finished 2020-09-08T15:40:17.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:16 smithi166 bash[9446]: cluster 2020-09-08T15:40:15.721703+0000 mon.a (mon.0) 1031 : cluster [DBG] osdmap e185: 12 total, 12 up, 12 in 2020-09-08T15:40:17.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:16 smithi055 bash[9296]: cluster 2020-09-08T15:40:15.718175+0000 osd.11 (osd.11) 18 : cluster [DBG] 2.c scrub ok 2020-09-08T15:40:17.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:16 smithi055 bash[9296]: audit 2020-09-08T15:40:15.721661+0000 mon.a (mon.0) 1030 : audit [INF] from='client.? v1:172.21.15.104:0/149327661' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 9, "weight": 1.0}]': finished 2020-09-08T15:40:17.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:16 smithi055 bash[9296]: cluster 2020-09-08T15:40:15.721703+0000 mon.a (mon.0) 1031 : cluster [DBG] osdmap e185: 12 total, 12 up, 12 in 2020-09-08T15:40:17.527 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd reweight 10 1 2020-09-08T15:40:17.539 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:17.642 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:40:17.695 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:40:17.775 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:40:17.883 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:17.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:17 smithi104 bash[9203]: cluster 2020-09-08T15:40:17.633261+0000 mgr.x (mgr.24118) 383 : cluster [DBG] pgmap v490: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.06k op/s 2020-09-08T15:40:17.927 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:40:17 smithi104 bash[19922]: debug 2020-09-08T15:40:17.537+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:18.000 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:17 smithi055 bash[9296]: cluster 2020-09-08T15:40:17.633261+0000 mgr.x (mgr.24118) 383 : cluster [DBG] pgmap v490: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.06k op/s 2020-09-08T15:40:18.001 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:18.105 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:40:18.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:17 smithi166 bash[9446]: cluster 2020-09-08T15:40:17.633261+0000 mgr.x (mgr.24118) 383 : cluster [DBG] pgmap v490: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.06k op/s 2020-09-08T15:40:18.114 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:40:17 smithi166 bash[14254]: debug 2020-09-08T15:40:17.773+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:18.226 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:40:18.329 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:18.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:40:18 smithi166 bash[14254]: debug 2020-09-08T15:40:18.221+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:18.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:40:18 smithi055 bash[16646]: debug 2020-09-08T15:40:18.003+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:18.659 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:18.763 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:18.876 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:18.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:40:18 smithi104 bash[15273]: debug 2020-09-08T15:40:18.657+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:18.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:18 smithi104 bash[9203]: cluster 2020-09-08T15:40:18.017706+0000 osd.10 (osd.10) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:18.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:18 smithi104 bash[9203]: cluster 2020-09-08T15:40:18.017785+0000 osd.10 (osd.10) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:18.986 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:19.104 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:18 smithi166 bash[9446]: cluster 2020-09-08T15:40:18.017706+0000 osd.10 (osd.10) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:19.104 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:18 smithi166 bash[9446]: cluster 2020-09-08T15:40:18.017785+0000 osd.10 (osd.10) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:19.105 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:19.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:18 smithi055 bash[9296]: cluster 2020-09-08T15:40:18.017706+0000 osd.10 (osd.10) 13 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:19.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:18 smithi055 bash[9296]: cluster 2020-09-08T15:40:18.017785+0000 osd.10 (osd.10) 14 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:19.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:40:18 smithi055 bash[11956]: debug 2020-09-08T15:40:18.867+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:19.209 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:40:19.365 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:40:19 smithi166 bash[11928]: debug 2020-09-08T15:40:19.105+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:19.420 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:40:19.525 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:19.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:40:19 smithi055 bash[14301]: debug 2020-09-08T15:40:19.419+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:19.726 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:19.734 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:40:19.767 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:40:19.806 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:40:19.830 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:19.854 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:40:19.893 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:40:19.927 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:40:19.938 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:19.973 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:40:20.016 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:40:20.043 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:40:20.050 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:40:19 smithi166 bash[11928]: debug 2020-09-08T15:40:19.937+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:20.050 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:19 smithi166 bash[9446]: cluster 2020-09-08T15:40:18.751695+0000 mon.a (mon.0) 1032 : cluster [DBG] osdmap e186: 12 total, 12 up, 12 in 2020-09-08T15:40:20.050 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:19 smithi166 bash[9446]: cluster 2020-09-08T15:40:19.633929+0000 mgr.x (mgr.24118) 384 : cluster [DBG] pgmap v492: 17 pgs: 17 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 957 op/s 2020-09-08T15:40:20.051 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:19 smithi166 bash[9446]: audit 2020-09-08T15:40:19.684614+0000 mon.a (mon.0) 1033 : audit [DBG] from='client.? v1:172.21.15.104:0/3970646638' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:40:20.058 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:40:20.091 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:40:20.119 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:40:20.148 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:40:20.155 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:40:20.156 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:40:20.156 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:40:20.156 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:40:20.156 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:40:20.157 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:40:20.157 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:40:20.157 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:40:20.157 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:40:20.157 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:40:20.157 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:40:20.158 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:40:20.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:19 smithi055 bash[9296]: cluster 2020-09-08T15:40:18.751695+0000 mon.a (mon.0) 1032 : cluster [DBG] osdmap e186: 12 total, 12 up, 12 in 2020-09-08T15:40:20.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:19 smithi055 bash[9296]: cluster 2020-09-08T15:40:19.633929+0000 mgr.x (mgr.24118) 384 : cluster [DBG] pgmap v492: 17 pgs: 17 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 957 op/s 2020-09-08T15:40:20.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:19 smithi055 bash[9296]: audit 2020-09-08T15:40:19.684614+0000 mon.a (mon.0) 1033 : audit [DBG] from='client.? v1:172.21.15.104:0/3970646638' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:40:20.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:40:19 smithi104 bash[22305]: debug 2020-09-08T15:40:19.721+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:20.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:19 smithi104 bash[9203]: cluster 2020-09-08T15:40:18.751695+0000 mon.a (mon.0) 1032 : cluster [DBG] osdmap e186: 12 total, 12 up, 12 in 2020-09-08T15:40:20.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:19 smithi104 bash[9203]: cluster 2020-09-08T15:40:19.633929+0000 mgr.x (mgr.24118) 384 : cluster [DBG] pgmap v492: 17 pgs: 17 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 957 op/s 2020-09-08T15:40:20.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:19 smithi104 bash[9203]: audit 2020-09-08T15:40:19.684614+0000 mon.a (mon.0) 1033 : audit [DBG] from='client.? v1:172.21.15.104:0/3970646638' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:40:20.179 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:40:20.285 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:40:20.365 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:40:20 smithi166 bash[18960]: debug 2020-09-08T15:40:20.181+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:20.405 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:40:20.510 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:40:20.528 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:40:20 smithi055 bash[14301]: debug 2020-09-08T15:40:20.407+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:20.539 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:40:20.539 INFO:teuthology.orchestra.run.smithi055.stdout: "size": 20, 2020-09-08T15:40:20.539 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 600, 2020-09-08T15:40:20.539 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T15:40:20.540 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.540 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:45921 2.2 2:428cf780:::benchmark_data_smithi166_20551_object45920:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e136)", 2020-09-08T15:40:20.540 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:37:58.341925+0000", 2020-09-08T15:40:20.540 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 142.192224372, 2020-09-08T15:40:20.540 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.34525313499999999, 2020-09-08T15:40:20.541 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:20.541 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:20.541 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:20.541 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:20.541 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:20.547 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 45921 2020-09-08T15:40:20.547 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.548 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:20.548 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.548 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:20.548 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.341925+0000", 2020-09-08T15:40:20.548 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:20.549 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.549 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.549 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:20.549 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.341925+0000", 2020-09-08T15:40:20.549 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999681 2020-09-08T15:40:20.550 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.550 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.550 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:20.551 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.341893+0000", 2020-09-08T15:40:20.551 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012937300000000001 2020-09-08T15:40:20.551 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.551 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.552 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:20.552 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.342022+0000", 2020-09-08T15:40:20.552 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.7824000000000001e-05 2020-09-08T15:40:20.552 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.552 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.553 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:20.553 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.342040+0000", 2020-09-08T15:40:20.553 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4800000000000001e-06 2020-09-08T15:40:20.553 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.553 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.554 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:20.554 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.342048+0000", 2020-09-08T15:40:20.554 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4997999999999998e-05 2020-09-08T15:40:20.554 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.554 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.555 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:20.555 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.342103+0000", 2020-09-08T15:40:20.555 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00021354800000000001 2020-09-08T15:40:20.555 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.555 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.556 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:20.556 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.342317+0000", 2020-09-08T15:40:20.556 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00025859800000000002 2020-09-08T15:40:20.556 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.556 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.556 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:40:20.557 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.342575+0000", 2020-09-08T15:40:20.557 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0031463870000000001 2020-09-08T15:40:20.557 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.557 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.558 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:20.558 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.345722+0000", 2020-09-08T15:40:20.558 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.34120681800000002 2020-09-08T15:40:20.558 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.558 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.559 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:20.559 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.686929+0000", 2020-09-08T15:40:20.559 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012220199999999999 2020-09-08T15:40:20.559 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.559 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.560 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:20.560 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.687051+0000", 2020-09-08T15:40:20.560 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012694799999999999 2020-09-08T15:40:20.560 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.560 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.560 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:20.561 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.687178+0000", 2020-09-08T15:40:20.561 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.34525313499999999 2020-09-08T15:40:20.561 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.561 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.561 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.562 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.562 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.562 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:45923 2.2 2:4d7d5ef1:::benchmark_data_smithi166_20551_object45922:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e136)", 2020-09-08T15:40:20.562 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:37:58.342878+0000", 2020-09-08T15:40:20.562 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 142.19127084499999, 2020-09-08T15:40:20.563 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.35287102100000001, 2020-09-08T15:40:20.563 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:20.563 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:20.563 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:20.563 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:20.564 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:20.564 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 45923 2020-09-08T15:40:20.564 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.564 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:20.564 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.564 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:20.565 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.342878+0000", 2020-09-08T15:40:20.565 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:20.565 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.565 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.565 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:20.566 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.342878+0000", 2020-09-08T15:40:20.566 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9993744 2020-09-08T15:40:20.566 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.566 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.566 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:20.567 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.342252+0000", 2020-09-08T15:40:20.567 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00073652699999999995 2020-09-08T15:40:20.567 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.568 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.568 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:20.569 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.342989+0000", 2020-09-08T15:40:20.569 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.4348000000000001e-05 2020-09-08T15:40:20.569 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.569 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.569 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:20.570 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.343013+0000", 2020-09-08T15:40:20.570 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.5230000000000001e-06 2020-09-08T15:40:20.570 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.570 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.570 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:20.570 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.343023+0000", 2020-09-08T15:40:20.571 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0014988110000000001 2020-09-08T15:40:20.571 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.571 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.571 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:20.571 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.344522+0000", 2020-09-08T15:40:20.572 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000135344 2020-09-08T15:40:20.572 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.572 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.572 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:20.572 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.344657+0000", 2020-09-08T15:40:20.573 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000145191 2020-09-08T15:40:20.573 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.573 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.573 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:40:20.573 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.344802+0000", 2020-09-08T15:40:20.574 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0024681 2020-09-08T15:40:20.574 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.574 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.574 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:20.574 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.347270+0000", 2020-09-08T15:40:20.574 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.34815605999999999 2020-09-08T15:40:20.575 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.575 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.575 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:20.575 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.695426+0000", 2020-09-08T15:40:20.575 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017602699999999999 2020-09-08T15:40:20.576 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.576 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.576 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:20.576 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.695602+0000", 2020-09-08T15:40:20.576 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014683099999999999 2020-09-08T15:40:20.577 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.577 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.577 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:20.577 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.695749+0000", 2020-09-08T15:40:20.577 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.35287102100000001 2020-09-08T15:40:20.577 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.578 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.578 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.578 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.578 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.578 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:45929 2.2 2:414a7c36:::benchmark_data_smithi166_20551_object45928:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e136)", 2020-09-08T15:40:20.579 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:37:58.344758+0000", 2020-09-08T15:40:20.579 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 142.18939054699999, 2020-09-08T15:40:20.579 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.35285140999999998, 2020-09-08T15:40:20.579 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:20.579 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:20.580 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:20.580 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:20.580 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:20.580 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 45929 2020-09-08T15:40:20.580 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.581 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:20.581 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.581 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:20.581 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.344758+0000", 2020-09-08T15:40:20.581 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:20.581 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.582 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.582 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:20.582 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.344758+0000", 2020-09-08T15:40:20.582 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999633 2020-09-08T15:40:20.582 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.583 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.583 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:20.583 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.344722+0000", 2020-09-08T15:40:20.583 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015230199999999999 2020-09-08T15:40:20.583 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.584 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.584 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:20.584 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.344874+0000", 2020-09-08T15:40:20.584 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4406999999999999e-05 2020-09-08T15:40:20.584 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.585 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.585 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:20.585 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.344888+0000", 2020-09-08T15:40:20.585 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.6190000000000002e-06 2020-09-08T15:40:20.585 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.586 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.586 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:20.586 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.344896+0000", 2020-09-08T15:40:20.586 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00086708599999999999 2020-09-08T15:40:20.586 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.586 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.587 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:20.587 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.345763+0000", 2020-09-08T15:40:20.587 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000152671 2020-09-08T15:40:20.588 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.588 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.588 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:20.588 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.345916+0000", 2020-09-08T15:40:20.588 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017866599999999999 2020-09-08T15:40:20.589 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.589 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.589 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:40:20.589 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.346094+0000", 2020-09-08T15:40:20.589 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0012776910000000001 2020-09-08T15:40:20.590 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.590 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.590 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:20.590 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.347372+0000", 2020-09-08T15:40:20.591 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.35008186099999999 2020-09-08T15:40:20.591 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.591 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.591 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:20.591 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.697454+0000", 2020-09-08T15:40:20.591 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4142999999999997e-05 2020-09-08T15:40:20.592 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.592 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.592 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:20.592 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.697538+0000", 2020-09-08T15:40:20.592 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.1858000000000002e-05 2020-09-08T15:40:20.593 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.593 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.593 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:20.593 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.697610+0000", 2020-09-08T15:40:20.593 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.35285140999999998 2020-09-08T15:40:20.594 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.594 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.594 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.594 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.594 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.595 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:45938 2.2 2:4bf87e99:::benchmark_data_smithi166_20551_object45937:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e136)", 2020-09-08T15:40:20.595 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:37:58.350110+0000", 2020-09-08T15:40:20.595 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 142.184038921, 2020-09-08T15:40:20.595 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.34775478599999998, 2020-09-08T15:40:20.595 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:20.595 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:20.596 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:20.596 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:20.596 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:20.596 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 45938 2020-09-08T15:40:20.596 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.597 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:20.597 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.597 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:20.597 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.350110+0000", 2020-09-08T15:40:20.597 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:20.598 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.598 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.598 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:20.598 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.350110+0000", 2020-09-08T15:40:20.598 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999647 2020-09-08T15:40:20.598 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.599 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.599 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:20.599 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.350075+0000", 2020-09-08T15:40:20.599 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000136365 2020-09-08T15:40:20.599 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.600 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.600 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:20.600 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.350211+0000", 2020-09-08T15:40:20.601 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4902999999999999e-05 2020-09-08T15:40:20.601 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.601 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.601 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:20.602 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.350226+0000", 2020-09-08T15:40:20.602 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.6490000000000003e-06 2020-09-08T15:40:20.602 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.602 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.602 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:20.602 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.350236+0000", 2020-09-08T15:40:20.603 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.6338000000000002e-05 2020-09-08T15:40:20.603 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.603 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.603 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:20.603 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.350292+0000", 2020-09-08T15:40:20.604 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000142752 2020-09-08T15:40:20.604 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.604 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.604 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:20.604 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.350435+0000", 2020-09-08T15:40:20.605 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014585100000000001 2020-09-08T15:40:20.605 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.605 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.605 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:40:20.605 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.350581+0000", 2020-09-08T15:40:20.606 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0026425709999999998 2020-09-08T15:40:20.606 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.606 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.606 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:20.606 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.353223+0000", 2020-09-08T15:40:20.606 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.34447684699999997 2020-09-08T15:40:20.607 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.607 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.607 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:20.607 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.697700+0000", 2020-09-08T15:40:20.607 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.1796e-05 2020-09-08T15:40:20.608 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.608 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.608 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:20.608 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.697792+0000", 2020-09-08T15:40:20.608 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.2999999999999999e-05 2020-09-08T15:40:20.609 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.609 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.609 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:20.609 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.697865+0000", 2020-09-08T15:40:20.609 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.34775478599999998 2020-09-08T15:40:20.610 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.610 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.610 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.610 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.611 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.611 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:45939 2.2 2:4f582c4a:::benchmark_data_smithi166_20551_object45938:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e136)", 2020-09-08T15:40:20.611 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:37:58.351462+0000", 2020-09-08T15:40:20.611 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 142.18268730700001, 2020-09-08T15:40:20.611 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.34665726200000002, 2020-09-08T15:40:20.612 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:20.612 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:20.612 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:20.612 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:20.612 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:20.612 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 45939 2020-09-08T15:40:20.613 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.613 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:20.613 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.613 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:20.613 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.351462+0000", 2020-09-08T15:40:20.614 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:20.614 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.614 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.614 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:20.614 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.351462+0000", 2020-09-08T15:40:20.615 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999194 2020-09-08T15:40:20.615 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.615 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.615 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:20.615 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.351381+0000", 2020-09-08T15:40:20.615 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00026436 2020-09-08T15:40:20.616 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.616 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.616 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:20.616 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.351645+0000", 2020-09-08T15:40:20.616 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.3751999999999999e-05 2020-09-08T15:40:20.617 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.617 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.617 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:20.617 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.351669+0000", 2020-09-08T15:40:20.617 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1116e-05 2020-09-08T15:40:20.618 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.618 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.618 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:20.618 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.351680+0000", 2020-09-08T15:40:20.618 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.5058e-05 2020-09-08T15:40:20.618 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.619 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.619 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:20.619 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.351745+0000", 2020-09-08T15:40:20.619 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00020902599999999999 2020-09-08T15:40:20.619 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.620 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.620 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:20.620 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.351954+0000", 2020-09-08T15:40:20.620 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015292699999999999 2020-09-08T15:40:20.620 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.621 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.621 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:40:20.621 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.352107+0000", 2020-09-08T15:40:20.621 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0026418280000000001 2020-09-08T15:40:20.621 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.622 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.622 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:20.622 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.354749+0000", 2020-09-08T15:40:20.622 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.34320125000000001 2020-09-08T15:40:20.622 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.622 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.623 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:20.623 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.697950+0000", 2020-09-08T15:40:20.623 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.0161999999999996e-05 2020-09-08T15:40:20.623 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.623 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.624 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:20.624 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.698030+0000", 2020-09-08T15:40:20.624 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.8504999999999994e-05 2020-09-08T15:40:20.624 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.624 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.625 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:20.626 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.698119+0000", 2020-09-08T15:40:20.626 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.34665726200000002 2020-09-08T15:40:20.626 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.627 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.627 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.627 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.627 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.627 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:45940 2.2 2:4a1e3731:::benchmark_data_smithi166_20551_object45939:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e136)", 2020-09-08T15:40:20.628 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:37:58.351759+0000", 2020-09-08T15:40:20.628 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 142.182390231, 2020-09-08T15:40:20.628 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.34664142599999997, 2020-09-08T15:40:20.628 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:20.628 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:20.629 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:20.630 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:20.630 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:20.630 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 45940 2020-09-08T15:40:20.631 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.631 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:20.631 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.631 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:20.632 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.351759+0000", 2020-09-08T15:40:20.633 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:20.633 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.633 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.633 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:20.633 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.351759+0000", 2020-09-08T15:40:20.634 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999652 2020-09-08T15:40:20.634 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.634 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.634 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:20.634 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.351724+0000", 2020-09-08T15:40:20.635 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000231566 2020-09-08T15:40:20.635 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.635 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.635 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:20.635 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.351955+0000", 2020-09-08T15:40:20.636 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2204999999999999e-05 2020-09-08T15:40:20.636 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.636 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.636 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:20.636 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.351968+0000", 2020-09-08T15:40:20.637 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.5260000000000001e-06 2020-09-08T15:40:20.637 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.637 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.637 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:20.637 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.351973+0000", 2020-09-08T15:40:20.638 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0013005969999999999 2020-09-08T15:40:20.638 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.638 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.638 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:20.638 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.353274+0000", 2020-09-08T15:40:20.638 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000204849 2020-09-08T15:40:20.639 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.639 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.639 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:20.639 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.353479+0000", 2020-09-08T15:40:20.639 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019514200000000001 2020-09-08T15:40:20.640 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.640 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.640 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:40:20.640 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.353674+0000", 2020-09-08T15:40:20.640 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0012793590000000001 2020-09-08T15:40:20.641 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.641 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.641 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:20.641 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.354953+0000", 2020-09-08T15:40:20.641 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.34324395299999999 2020-09-08T15:40:20.642 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.642 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.643 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:20.643 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.698197+0000", 2020-09-08T15:40:20.643 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000133535 2020-09-08T15:40:20.643 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.644 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.644 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:20.644 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.698331+0000", 2020-09-08T15:40:20.644 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.9550999999999999e-05 2020-09-08T15:40:20.644 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.645 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.645 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:20.646 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:58.698400+0000", 2020-09-08T15:40:20.646 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.34664142599999997 2020-09-08T15:40:20.646 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.646 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.646 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.647 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.647 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.647 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:143831 2.3 2:c699b655:::benchmark_data_smithi166_20551_object143830:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e163)", 2020-09-08T15:40:20.647 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:08.509577+0000", 2020-09-08T15:40:20.647 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 72.024572054000004, 2020-09-08T15:40:20.648 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.39917955500000002, 2020-09-08T15:40:20.648 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:20.648 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:20.648 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:20.648 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:20.649 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:20.649 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 143831 2020-09-08T15:40:20.649 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.649 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:20.649 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.650 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:20.650 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.509577+0000", 2020-09-08T15:40:20.650 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:20.650 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.650 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.651 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:20.651 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.509577+0000", 2020-09-08T15:40:20.651 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999561 2020-09-08T15:40:20.651 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.651 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.651 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:20.652 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.509533+0000", 2020-09-08T15:40:20.652 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014324199999999999 2020-09-08T15:40:20.652 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.652 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.652 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:20.653 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.509676+0000", 2020-09-08T15:40:20.653 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.0876000000000001e-05 2020-09-08T15:40:20.653 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.653 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.654 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:20.654 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.509697+0000", 2020-09-08T15:40:20.655 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.1769999999999998e-06 2020-09-08T15:40:20.655 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.655 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.655 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:20.655 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.509705+0000", 2020-09-08T15:40:20.656 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.5145999999999997e-05 2020-09-08T15:40:20.656 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.656 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.656 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:20.656 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.509760+0000", 2020-09-08T15:40:20.656 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00018251000000000001 2020-09-08T15:40:20.657 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.657 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.657 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:20.657 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.509943+0000", 2020-09-08T15:40:20.657 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000176625 2020-09-08T15:40:20.658 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.658 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.658 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 2", 2020-09-08T15:40:20.658 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.510119+0000", 2020-09-08T15:40:20.658 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0042263220000000002 2020-09-08T15:40:20.659 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.659 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.659 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:20.659 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.514346+0000", 2020-09-08T15:40:20.659 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.39420186899999998 2020-09-08T15:40:20.660 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.660 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.660 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:20.660 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.908548+0000", 2020-09-08T15:40:20.660 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012506400000000001 2020-09-08T15:40:20.660 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.661 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.661 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:20.661 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.908673+0000", 2020-09-08T15:40:20.661 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.3801000000000006e-05 2020-09-08T15:40:20.661 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.662 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.662 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:20.662 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.908756+0000", 2020-09-08T15:40:20.662 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.39917955500000002 2020-09-08T15:40:20.662 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.663 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.663 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.663 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.663 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.663 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:143853 2.3 2:c90fbbc2:::benchmark_data_smithi166_20551_object143852:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e163)", 2020-09-08T15:40:20.664 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:08.526203+0000", 2020-09-08T15:40:20.664 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 72.007945984000003, 2020-09-08T15:40:20.664 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40916183099999998, 2020-09-08T15:40:20.665 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:20.665 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:20.665 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:20.665 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:20.665 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:20.666 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 143853 2020-09-08T15:40:20.666 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.666 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:20.666 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.666 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:20.666 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.526203+0000", 2020-09-08T15:40:20.667 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:20.667 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.667 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.667 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:20.667 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.526203+0000", 2020-09-08T15:40:20.668 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999704 2020-09-08T15:40:20.668 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.668 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.668 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:20.668 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.526173+0000", 2020-09-08T15:40:20.669 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010233700000000001 2020-09-08T15:40:20.669 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.669 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.669 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:20.670 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.526276+0000", 2020-09-08T15:40:20.670 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3071000000000001e-05 2020-09-08T15:40:20.670 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.670 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.670 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:20.671 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.526289+0000", 2020-09-08T15:40:20.671 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.664e-05 2020-09-08T15:40:20.671 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.671 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.671 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:20.671 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.526305+0000", 2020-09-08T15:40:20.672 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9234000000000002e-05 2020-09-08T15:40:20.672 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.672 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.672 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:20.672 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.526365+0000", 2020-09-08T15:40:20.673 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015939200000000001 2020-09-08T15:40:20.673 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.673 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.673 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:20.674 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.526524+0000", 2020-09-08T15:40:20.674 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016764500000000001 2020-09-08T15:40:20.674 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.674 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.674 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 2", 2020-09-08T15:40:20.675 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.526692+0000", 2020-09-08T15:40:20.675 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0047147889999999996 2020-09-08T15:40:20.675 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.675 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.675 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:20.676 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.531406+0000", 2020-09-08T15:40:20.676 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.403609889 2020-09-08T15:40:20.676 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.676 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.676 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:20.676 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.935016+0000", 2020-09-08T15:40:20.677 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000198202 2020-09-08T15:40:20.677 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.677 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.677 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:20.677 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.935215+0000", 2020-09-08T15:40:20.678 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015023700000000001 2020-09-08T15:40:20.678 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.678 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.678 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:20.678 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.935365+0000", 2020-09-08T15:40:20.679 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40916183099999998 2020-09-08T15:40:20.679 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.679 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.679 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.679 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.680 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.680 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:143921 2.3 2:c8fe75ff:::benchmark_data_smithi166_20551_object143920:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e163)", 2020-09-08T15:40:20.680 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:08.614372+0000", 2020-09-08T15:40:20.680 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 71.919777341, 2020-09-08T15:40:20.680 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.32143718500000001, 2020-09-08T15:40:20.681 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:20.681 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:20.681 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:20.681 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:20.681 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:20.681 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 143921 2020-09-08T15:40:20.682 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.682 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:20.682 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.682 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:20.682 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.614372+0000", 2020-09-08T15:40:20.683 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:20.683 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.683 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.683 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:20.683 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.614372+0000", 2020-09-08T15:40:20.684 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999599 2020-09-08T15:40:20.684 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.684 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.684 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:20.684 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.614331+0000", 2020-09-08T15:40:20.685 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013648600000000001 2020-09-08T15:40:20.685 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.685 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.685 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:20.685 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.614468+0000", 2020-09-08T15:40:20.686 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.8504e-05 2020-09-08T15:40:20.686 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.686 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.686 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:20.686 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.614496+0000", 2020-09-08T15:40:20.686 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.668e-05 2020-09-08T15:40:20.687 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.687 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.687 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:20.687 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.614513+0000", 2020-09-08T15:40:20.687 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012556199999999999 2020-09-08T15:40:20.688 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.688 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.688 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:20.688 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.614639+0000", 2020-09-08T15:40:20.688 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00040463599999999999 2020-09-08T15:40:20.689 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.689 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.689 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:20.689 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.615043+0000", 2020-09-08T15:40:20.689 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00051743299999999998 2020-09-08T15:40:20.689 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.690 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.690 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 2", 2020-09-08T15:40:20.690 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.615561+0000", 2020-09-08T15:40:20.690 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.017989267 2020-09-08T15:40:20.690 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.691 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.691 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:20.691 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.633550+0000", 2020-09-08T15:40:20.691 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.30194837800000002 2020-09-08T15:40:20.691 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.692 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.692 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:20.692 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.935498+0000", 2020-09-08T15:40:20.692 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000109498 2020-09-08T15:40:20.692 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.692 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.693 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:20.693 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.935608+0000", 2020-09-08T15:40:20.693 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000200862 2020-09-08T15:40:20.693 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.693 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.694 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:20.694 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.935809+0000", 2020-09-08T15:40:20.694 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.32143718500000001 2020-09-08T15:40:20.694 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.694 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.695 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.695 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.695 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.695 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:143923 2.3 2:c86bfb94:::benchmark_data_smithi166_20551_object143922:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e163)", 2020-09-08T15:40:20.695 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:08.618419+0000", 2020-09-08T15:40:20.696 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 71.915730038999996, 2020-09-08T15:40:20.696 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.317962721, 2020-09-08T15:40:20.696 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:20.696 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:20.697 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:20.697 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:20.697 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:20.697 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 143923 2020-09-08T15:40:20.698 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.698 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:20.698 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.698 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:20.698 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.618419+0000", 2020-09-08T15:40:20.699 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:20.699 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.699 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.699 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:20.699 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.618419+0000", 2020-09-08T15:40:20.700 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999628 2020-09-08T15:40:20.700 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.700 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.700 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:20.700 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.618382+0000", 2020-09-08T15:40:20.701 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015882200000000001 2020-09-08T15:40:20.701 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.701 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.701 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:20.701 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.618541+0000", 2020-09-08T15:40:20.701 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5101e-05 2020-09-08T15:40:20.702 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.702 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.702 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:20.702 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.618556+0000", 2020-09-08T15:40:20.702 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.986e-06 2020-09-08T15:40:20.703 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.703 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.703 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:20.703 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.618565+0000", 2020-09-08T15:40:20.703 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0026802810000000001 2020-09-08T15:40:20.704 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.704 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.704 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:20.704 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.621245+0000", 2020-09-08T15:40:20.704 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000195997 2020-09-08T15:40:20.705 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.705 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.705 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:20.705 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.621441+0000", 2020-09-08T15:40:20.706 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000230187 2020-09-08T15:40:20.706 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.706 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.706 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 2", 2020-09-08T15:40:20.706 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.621671+0000", 2020-09-08T15:40:20.706 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.011900124999999999 2020-09-08T15:40:20.707 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.707 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.707 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:20.707 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.633571+0000", 2020-09-08T15:40:20.707 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.30245320399999998 2020-09-08T15:40:20.708 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.708 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.708 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:20.708 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.936024+0000", 2020-09-08T15:40:20.708 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019684600000000001 2020-09-08T15:40:20.709 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.709 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.709 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:20.709 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.936221+0000", 2020-09-08T15:40:20.709 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016038000000000001 2020-09-08T15:40:20.709 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.710 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.710 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:20.710 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.936382+0000", 2020-09-08T15:40:20.710 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.317962721 2020-09-08T15:40:20.710 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.711 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.711 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.711 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.711 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.711 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:143924 2.3 2:cd2ec23b:::benchmark_data_smithi166_20551_object143923:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e163)", 2020-09-08T15:40:20.712 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:08.618626+0000", 2020-09-08T15:40:20.712 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 71.915523242000006, 2020-09-08T15:40:20.712 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.318211947, 2020-09-08T15:40:20.712 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:20.712 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:20.712 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:20.713 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:20.713 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:20.713 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 143924 2020-09-08T15:40:20.713 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.714 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:20.714 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.714 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:20.714 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.618626+0000", 2020-09-08T15:40:20.714 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:20.715 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.715 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.715 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:20.715 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.618626+0000", 2020-09-08T15:40:20.715 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999676 2020-09-08T15:40:20.715 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.716 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.716 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:20.716 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.618593+0000", 2020-09-08T15:40:20.716 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014246199999999999 2020-09-08T15:40:20.716 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.717 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.717 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:20.717 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.618736+0000", 2020-09-08T15:40:20.717 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3930999999999999e-05 2020-09-08T15:40:20.717 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.718 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.718 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:20.718 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.618750+0000", 2020-09-08T15:40:20.718 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.6620000000000002e-06 2020-09-08T15:40:20.718 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.718 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.719 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:20.719 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.618755+0000", 2020-09-08T15:40:20.719 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0069612190000000003 2020-09-08T15:40:20.719 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.719 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.720 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:20.720 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.625717+0000", 2020-09-08T15:40:20.720 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00024434199999999999 2020-09-08T15:40:20.720 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.720 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.721 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:20.721 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.625961+0000", 2020-09-08T15:40:20.721 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000322825 2020-09-08T15:40:20.721 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.721 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.722 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 2", 2020-09-08T15:40:20.722 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.626284+0000", 2020-09-08T15:40:20.722 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0072978189999999997 2020-09-08T15:40:20.722 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.722 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.723 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:20.723 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.633582+0000", 2020-09-08T15:40:20.723 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.302937024 2020-09-08T15:40:20.723 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.724 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.724 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:20.724 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.936519+0000", 2020-09-08T15:40:20.724 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012115700000000001 2020-09-08T15:40:20.724 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.724 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.725 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:20.725 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.936640+0000", 2020-09-08T15:40:20.725 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019781900000000001 2020-09-08T15:40:20.725 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.725 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.726 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:20.726 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.936838+0000", 2020-09-08T15:40:20.726 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.318211947 2020-09-08T15:40:20.726 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.726 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.727 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.727 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.727 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.727 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:143926 2.3 2:c2e6e20d:::benchmark_data_smithi166_20551_object143925:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e163)", 2020-09-08T15:40:20.727 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:08.619296+0000", 2020-09-08T15:40:20.727 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 71.914852960000005, 2020-09-08T15:40:20.728 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.31948112299999998, 2020-09-08T15:40:20.728 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:20.728 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:20.728 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:20.728 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:20.729 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:20.729 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 143926 2020-09-08T15:40:20.729 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.729 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:20.729 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.730 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:20.730 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.619296+0000", 2020-09-08T15:40:20.730 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:20.730 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.730 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.730 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:20.731 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.619296+0000", 2020-09-08T15:40:20.731 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999518 2020-09-08T15:40:20.731 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.731 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.731 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:20.732 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.619248+0000", 2020-09-08T15:40:20.732 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014545699999999999 2020-09-08T15:40:20.732 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.732 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.732 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:20.733 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.619393+0000", 2020-09-08T15:40:20.733 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.7099999999999999e-05 2020-09-08T15:40:20.733 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.733 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.733 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:20.734 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.619411+0000", 2020-09-08T15:40:20.734 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3679999999999998e-06 2020-09-08T15:40:20.740 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.740 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.740 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:20.740 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.619417+0000", 2020-09-08T15:40:20.740 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.009672613 2020-09-08T15:40:20.741 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.741 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.741 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:20.741 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.629090+0000", 2020-09-08T15:40:20.742 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000283596 2020-09-08T15:40:20.742 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.742 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.742 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:20.742 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.629373+0000", 2020-09-08T15:40:20.742 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019577900000000001 2020-09-08T15:40:20.743 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.743 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.743 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 2", 2020-09-08T15:40:20.743 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.629569+0000", 2020-09-08T15:40:20.743 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0043432330000000002 2020-09-08T15:40:20.744 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.744 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.744 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:20.744 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.633912+0000", 2020-09-08T15:40:20.744 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.30454112500000002 2020-09-08T15:40:20.745 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.745 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.745 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:20.745 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.938453+0000", 2020-09-08T15:40:20.745 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000184573 2020-09-08T15:40:20.746 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.746 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.746 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:20.746 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.938638+0000", 2020-09-08T15:40:20.746 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000139247 2020-09-08T15:40:20.746 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.747 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.747 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:20.747 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.938777+0000", 2020-09-08T15:40:20.747 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.31948112299999998 2020-09-08T15:40:20.747 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.748 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.748 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.748 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.748 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.748 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:143936 2.3 2:c9931022:::benchmark_data_smithi166_20551_object143935:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e163)", 2020-09-08T15:40:20.749 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:08.684988+0000", 2020-09-08T15:40:20.749 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 71.849161074999998, 2020-09-08T15:40:20.749 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.254349086, 2020-09-08T15:40:20.749 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:20.749 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:20.749 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:20.750 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:20.750 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:20.750 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 143936 2020-09-08T15:40:20.750 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.750 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:20.751 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.751 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:20.751 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.684988+0000", 2020-09-08T15:40:20.751 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:20.751 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.752 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.752 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:20.752 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.684988+0000", 2020-09-08T15:40:20.752 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9998612 2020-09-08T15:40:20.752 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.752 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.753 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:20.753 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.684849+0000", 2020-09-08T15:40:20.753 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00040744499999999999 2020-09-08T15:40:20.753 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.753 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.754 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:20.754 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.685257+0000", 2020-09-08T15:40:20.754 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.5055000000000001e-05 2020-09-08T15:40:20.754 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.754 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.755 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:20.755 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.685312+0000", 2020-09-08T15:40:20.755 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.6193000000000001e-05 2020-09-08T15:40:20.755 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.755 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.755 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:20.756 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.685338+0000", 2020-09-08T15:40:20.756 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000100753 2020-09-08T15:40:20.756 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.756 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.756 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:20.757 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.685439+0000", 2020-09-08T15:40:20.757 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00024207499999999999 2020-09-08T15:40:20.757 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.757 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.757 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:20.758 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.685681+0000", 2020-09-08T15:40:20.758 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00024862899999999999 2020-09-08T15:40:20.758 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.758 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.758 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 2", 2020-09-08T15:40:20.759 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.685929+0000", 2020-09-08T15:40:20.759 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.005849389 2020-09-08T15:40:20.759 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.759 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.759 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:20.759 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.691779+0000", 2020-09-08T15:40:20.760 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.24719170900000001 2020-09-08T15:40:20.760 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.760 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.760 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:20.760 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.938971+0000", 2020-09-08T15:40:20.761 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00020030800000000001 2020-09-08T15:40:20.761 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.761 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.761 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:20.761 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.939171+0000", 2020-09-08T15:40:20.762 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000166107 2020-09-08T15:40:20.762 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.762 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.762 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:20.762 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:08.939337+0000", 2020-09-08T15:40:20.762 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.254349086 2020-09-08T15:40:20.763 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.763 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.763 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.763 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.763 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.764 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:176588 2.3 2:c75a7bdb:::benchmark_data_smithi166_20551_object28968:head [delete] snapc 0=[] ondisk+write+known_if_redirected e174)", 2020-09-08T15:40:20.764 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:42.725699+0000", 2020-09-08T15:40:20.764 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 37.808450014000002, 2020-09-08T15:40:20.764 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.25792180100000001, 2020-09-08T15:40:20.764 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:20.765 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:20.765 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:20.765 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:20.765 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:20.765 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 176588 2020-09-08T15:40:20.765 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.766 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:20.766 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.766 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:20.766 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.725699+0000", 2020-09-08T15:40:20.766 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:20.767 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.767 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.767 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:20.767 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.725699+0000", 2020-09-08T15:40:20.767 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999948 2020-09-08T15:40:20.768 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.768 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.768 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:20.768 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.725647+0000", 2020-09-08T15:40:20.768 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000110955 2020-09-08T15:40:20.768 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.769 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.769 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:20.769 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.725758+0000", 2020-09-08T15:40:20.769 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.0043000000000001e-05 2020-09-08T15:40:20.769 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.770 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.770 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:20.770 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.725778+0000", 2020-09-08T15:40:20.770 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.6439999999999995e-06 2020-09-08T15:40:20.770 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.771 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.771 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:20.771 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.725787+0000", 2020-09-08T15:40:20.771 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.7221999999999999e-05 2020-09-08T15:40:20.771 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.771 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.772 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:20.772 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.725875+0000", 2020-09-08T15:40:20.772 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015761400000000001 2020-09-08T15:40:20.772 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.772 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.773 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:20.773 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.726032+0000", 2020-09-08T15:40:20.773 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016568900000000001 2020-09-08T15:40:20.773 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.773 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.774 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 2", 2020-09-08T15:40:20.774 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.726198+0000", 2020-09-08T15:40:20.774 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0044395859999999997 2020-09-08T15:40:20.774 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.775 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.775 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:20.775 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.730638+0000", 2020-09-08T15:40:20.775 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.25275620900000001 2020-09-08T15:40:20.775 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.776 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.776 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:20.776 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.983394+0000", 2020-09-08T15:40:20.776 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014038399999999999 2020-09-08T15:40:20.776 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.777 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.777 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:20.777 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.983534+0000", 2020-09-08T15:40:20.777 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.6581000000000003e-05 2020-09-08T15:40:20.777 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.778 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.778 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:20.778 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.983621+0000", 2020-09-08T15:40:20.778 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.25792180100000001 2020-09-08T15:40:20.778 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.778 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.779 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.779 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.779 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.779 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:176627 2.3 2:cf143bcc:::benchmark_data_smithi166_20551_object29007:head [delete] snapc 0=[] ondisk+write+known_if_redirected e174)", 2020-09-08T15:40:20.779 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:42.769354+0000", 2020-09-08T15:40:20.780 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 37.764795292999999, 2020-09-08T15:40:20.780 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.23388630899999999, 2020-09-08T15:40:20.780 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:20.780 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:20.780 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:20.781 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:20.781 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:20.781 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 176627 2020-09-08T15:40:20.781 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.781 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:20.782 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.782 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:20.782 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.769354+0000", 2020-09-08T15:40:20.782 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:20.782 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.782 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.783 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:20.783 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.769354+0000", 2020-09-08T15:40:20.783 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999299 2020-09-08T15:40:20.783 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.783 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.784 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:20.784 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.769283+0000", 2020-09-08T15:40:20.784 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00020425599999999999 2020-09-08T15:40:20.784 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.784 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.785 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:20.785 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.769488+0000", 2020-09-08T15:40:20.785 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.5191000000000002e-05 2020-09-08T15:40:20.785 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.785 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.785 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:20.786 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.769533+0000", 2020-09-08T15:40:20.786 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.9371000000000001e-05 2020-09-08T15:40:20.786 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.786 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.786 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:20.787 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.769552+0000", 2020-09-08T15:40:20.787 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000112899 2020-09-08T15:40:20.787 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.787 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.787 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:20.788 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.769665+0000", 2020-09-08T15:40:20.788 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00023767300000000001 2020-09-08T15:40:20.788 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.788 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.788 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:20.788 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.769903+0000", 2020-09-08T15:40:20.789 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019026399999999999 2020-09-08T15:40:20.789 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.789 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.789 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 2", 2020-09-08T15:40:20.789 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.770093+0000", 2020-09-08T15:40:20.790 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0050153649999999999 2020-09-08T15:40:20.790 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.790 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.790 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:20.790 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.775108+0000", 2020-09-08T15:40:20.791 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.22793265400000001 2020-09-08T15:40:20.791 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.791 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.791 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:20.791 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:43.003041+0000", 2020-09-08T15:40:20.791 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000113705 2020-09-08T15:40:20.792 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.792 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.792 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:20.792 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:43.003155+0000", 2020-09-08T15:40:20.792 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.5167000000000006e-05 2020-09-08T15:40:20.793 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.793 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.793 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:20.793 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:43.003240+0000", 2020-09-08T15:40:20.793 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.23388630899999999 2020-09-08T15:40:20.794 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.794 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.794 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.794 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.794 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.794 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:176676 2.3 2:cd06dc43:::benchmark_data_smithi166_20551_object29056:head [delete] snapc 0=[] ondisk+write+known_if_redirected e174)", 2020-09-08T15:40:20.795 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:42.842710+0000", 2020-09-08T15:40:20.795 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 37.691438738000002, 2020-09-08T15:40:20.795 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.16888868100000001, 2020-09-08T15:40:20.795 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:20.795 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:20.796 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:20.796 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:20.796 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:20.796 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 176676 2020-09-08T15:40:20.796 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.797 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:20.797 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.797 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:20.797 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.842710+0000", 2020-09-08T15:40:20.798 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:20.798 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.798 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.798 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:20.799 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.842710+0000", 2020-09-08T15:40:20.799 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999256 2020-09-08T15:40:20.799 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.799 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.799 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:20.800 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.842636+0000", 2020-09-08T15:40:20.800 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016423100000000001 2020-09-08T15:40:20.800 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.800 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.800 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:20.801 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.842800+0000", 2020-09-08T15:40:20.801 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.9850000000000001e-05 2020-09-08T15:40:20.801 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.801 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.801 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:20.802 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.842830+0000", 2020-09-08T15:40:20.802 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5532e-05 2020-09-08T15:40:20.802 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.802 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.802 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:20.802 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.842845+0000", 2020-09-08T15:40:20.803 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.4586999999999996e-05 2020-09-08T15:40:20.803 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.803 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.803 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:20.803 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.842920+0000", 2020-09-08T15:40:20.804 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000113557 2020-09-08T15:40:20.804 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.804 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.804 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:20.804 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.843033+0000", 2020-09-08T15:40:20.805 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3097999999999995e-05 2020-09-08T15:40:20.805 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.805 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.805 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 2", 2020-09-08T15:40:20.805 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.843126+0000", 2020-09-08T15:40:20.806 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0058555270000000001 2020-09-08T15:40:20.806 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.806 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.806 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:20.807 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.848982+0000", 2020-09-08T15:40:20.807 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.16240581400000001 2020-09-08T15:40:20.807 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.807 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.807 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:20.807 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:43.011388+0000", 2020-09-08T15:40:20.808 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000113961 2020-09-08T15:40:20.808 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.808 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.808 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:20.808 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:43.011502+0000", 2020-09-08T15:40:20.809 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.7137000000000002e-05 2020-09-08T15:40:20.809 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.809 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.809 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:20.809 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:43.011599+0000", 2020-09-08T15:40:20.810 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.16888868100000001 2020-09-08T15:40:20.810 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.810 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.810 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.810 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.810 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.811 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:176680 2.3 2:c4f74e2b:::benchmark_data_smithi166_20551_object29060:head [delete] snapc 0=[] ondisk+write+known_if_redirected e174)", 2020-09-08T15:40:20.811 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:42.850001+0000", 2020-09-08T15:40:20.811 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 37.684147891000002, 2020-09-08T15:40:20.811 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.161816238, 2020-09-08T15:40:20.811 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:20.812 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:20.812 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:20.812 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:20.812 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:20.812 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 176680 2020-09-08T15:40:20.813 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.813 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:20.813 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.813 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:20.814 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.850001+0000", 2020-09-08T15:40:20.814 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:20.814 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.814 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.814 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:20.815 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.850001+0000", 2020-09-08T15:40:20.815 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999356 2020-09-08T15:40:20.815 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.815 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.815 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:20.815 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.849937+0000", 2020-09-08T15:40:20.816 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014115899999999999 2020-09-08T15:40:20.816 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.816 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.816 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:20.816 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.850078+0000", 2020-09-08T15:40:20.817 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.8433999999999999e-05 2020-09-08T15:40:20.817 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.817 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.817 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:20.817 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.850106+0000", 2020-09-08T15:40:20.818 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3395e-05 2020-09-08T15:40:20.818 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.818 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.818 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:20.818 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.850120+0000", 2020-09-08T15:40:20.818 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000122309 2020-09-08T15:40:20.819 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.819 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.819 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:20.819 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.850242+0000", 2020-09-08T15:40:20.819 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00031170299999999998 2020-09-08T15:40:20.820 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.820 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.820 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:20.820 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.850554+0000", 2020-09-08T15:40:20.820 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000330255 2020-09-08T15:40:20.821 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.821 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.821 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 2", 2020-09-08T15:40:20.821 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.850884+0000", 2020-09-08T15:40:20.821 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0068315140000000003 2020-09-08T15:40:20.821 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.822 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.822 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:20.822 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:42.857715+0000", 2020-09-08T15:40:20.822 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.15400303800000001 2020-09-08T15:40:20.822 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.823 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.823 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:20.823 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:43.011718+0000", 2020-09-08T15:40:20.823 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.2408000000000003e-05 2020-09-08T15:40:20.823 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.824 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.824 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:20.824 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:43.011781+0000", 2020-09-08T15:40:20.824 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.6528e-05 2020-09-08T15:40:20.825 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.825 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.825 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:20.825 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:43.011817+0000", 2020-09-08T15:40:20.825 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.161816238 2020-09-08T15:40:20.826 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.826 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.826 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.826 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.826 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.827 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:194376 2.2 2:44b1a1ee:::benchmark_data_smithi166_20551_object46756:head [delete] snapc 0=[] ondisk+write+known_if_redirected e180)", 2020-09-08T15:40:20.827 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:40:01.360970+0000", 2020-09-08T15:40:20.827 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 19.173178997000001, 2020-09-08T15:40:20.827 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.26244880300000001, 2020-09-08T15:40:20.827 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:20.827 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:20.828 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:20.828 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:20.828 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:20.828 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 194376 2020-09-08T15:40:20.828 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.829 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:20.829 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.829 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:20.829 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.360970+0000", 2020-09-08T15:40:20.829 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:20.830 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.830 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.830 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:20.830 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.360970+0000", 2020-09-08T15:40:20.830 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999719 2020-09-08T15:40:20.831 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.831 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.831 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:20.831 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.360942+0000", 2020-09-08T15:40:20.831 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.7704000000000001e-05 2020-09-08T15:40:20.831 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.832 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.832 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:20.832 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.361009+0000", 2020-09-08T15:40:20.832 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.6600000000000007e-06 2020-09-08T15:40:20.832 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.833 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.833 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:20.833 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.361019+0000", 2020-09-08T15:40:20.833 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.0869999999999997e-06 2020-09-08T15:40:20.833 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.834 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.834 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:20.834 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.361024+0000", 2020-09-08T15:40:20.834 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.4873999999999998e-05 2020-09-08T15:40:20.834 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.834 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.835 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:20.835 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.361069+0000", 2020-09-08T15:40:20.835 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000100969 2020-09-08T15:40:20.835 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.835 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.836 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:20.836 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.361170+0000", 2020-09-08T15:40:20.836 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.1829000000000005e-05 2020-09-08T15:40:20.836 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.836 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.837 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:40:20.837 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.361262+0000", 2020-09-08T15:40:20.837 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0041011629999999997 2020-09-08T15:40:20.837 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.837 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.837 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:20.838 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.365363+0000", 2020-09-08T15:40:20.838 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.25780853300000001 2020-09-08T15:40:20.838 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.838 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.838 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:20.839 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.623171+0000", 2020-09-08T15:40:20.839 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000133588 2020-09-08T15:40:20.839 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.839 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.839 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:20.840 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.623305+0000", 2020-09-08T15:40:20.840 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011367 2020-09-08T15:40:20.840 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.840 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.841 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:20.841 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.623419+0000", 2020-09-08T15:40:20.841 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.26244880300000001 2020-09-08T15:40:20.841 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.841 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.842 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.842 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.842 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.842 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:194386 2.2 2:425d78f2:::benchmark_data_smithi166_20551_object46766:head [delete] snapc 0=[] ondisk+write+known_if_redirected e180)", 2020-09-08T15:40:20.842 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:40:01.369088+0000", 2020-09-08T15:40:20.842 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 19.165060704999998, 2020-09-08T15:40:20.843 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.254636635, 2020-09-08T15:40:20.843 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:20.843 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:20.843 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:20.843 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:20.844 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:20.844 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 194386 2020-09-08T15:40:20.844 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.844 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:20.844 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.845 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:20.845 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.369088+0000", 2020-09-08T15:40:20.845 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:20.845 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.845 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.845 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:20.846 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.369088+0000", 2020-09-08T15:40:20.846 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999642 2020-09-08T15:40:20.846 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.846 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.846 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:20.847 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.369052+0000", 2020-09-08T15:40:20.847 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.4836e-05 2020-09-08T15:40:20.847 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.847 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.847 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:20.848 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.369127+0000", 2020-09-08T15:40:20.848 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5599000000000001e-05 2020-09-08T15:40:20.848 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.848 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.848 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:20.848 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.369143+0000", 2020-09-08T15:40:20.849 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.5429999999999994e-06 2020-09-08T15:40:20.849 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.849 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.849 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:20.849 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.369151+0000", 2020-09-08T15:40:20.850 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.7185999999999999e-05 2020-09-08T15:40:20.850 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.850 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.850 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:20.850 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.369198+0000", 2020-09-08T15:40:20.850 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000141237 2020-09-08T15:40:20.851 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.851 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.851 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:20.851 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.369340+0000", 2020-09-08T15:40:20.851 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013973999999999999 2020-09-08T15:40:20.852 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.852 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.852 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:40:20.852 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.369479+0000", 2020-09-08T15:40:20.852 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.004709413 2020-09-08T15:40:20.853 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.853 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.853 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:20.853 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.374189+0000", 2020-09-08T15:40:20.853 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.24933007300000001 2020-09-08T15:40:20.854 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.854 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.854 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:20.854 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.623519+0000", 2020-09-08T15:40:20.854 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000115024 2020-09-08T15:40:20.854 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.855 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.855 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:20.855 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.623634+0000", 2020-09-08T15:40:20.855 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.0964999999999994e-05 2020-09-08T15:40:20.855 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.856 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.856 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:20.856 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.623725+0000", 2020-09-08T15:40:20.856 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.254636635 2020-09-08T15:40:20.856 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.857 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.857 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.857 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.857 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.857 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:194394 2.2 2:49f3b51e:::benchmark_data_smithi166_20551_object46774:head [delete] snapc 0=[] ondisk+write+known_if_redirected e180)", 2020-09-08T15:40:20.858 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:40:01.381132+0000", 2020-09-08T15:40:20.858 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 19.153016683000001, 2020-09-08T15:40:20.858 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.247967297, 2020-09-08T15:40:20.858 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:20.858 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:20.858 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:20.859 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:20.859 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:20.859 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 194394 2020-09-08T15:40:20.859 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.859 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:20.860 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.860 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:20.860 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.381132+0000", 2020-09-08T15:40:20.860 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:20.860 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.861 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.861 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:20.861 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.381132+0000", 2020-09-08T15:40:20.861 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999599 2020-09-08T15:40:20.861 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.861 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.862 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:20.862 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.381092+0000", 2020-09-08T15:40:20.862 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4733000000000003e-05 2020-09-08T15:40:20.862 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.862 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.863 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:20.863 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.381187+0000", 2020-09-08T15:40:20.863 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6259000000000001e-05 2020-09-08T15:40:20.863 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.863 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.864 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:20.864 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.381203+0000", 2020-09-08T15:40:20.864 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3659999999999997e-06 2020-09-08T15:40:20.864 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.864 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.864 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:20.865 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.381210+0000", 2020-09-08T15:40:20.865 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000137513 2020-09-08T15:40:20.865 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.865 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.865 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:20.866 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.381347+0000", 2020-09-08T15:40:20.866 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00021692599999999999 2020-09-08T15:40:20.866 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.866 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.866 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:20.867 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.381564+0000", 2020-09-08T15:40:20.867 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013845400000000001 2020-09-08T15:40:20.867 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.867 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.867 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:40:20.867 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.381702+0000", 2020-09-08T15:40:20.868 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0051013430000000004 2020-09-08T15:40:20.868 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.868 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.868 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:20.868 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.386804+0000", 2020-09-08T15:40:20.869 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.24201513099999999 2020-09-08T15:40:20.869 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.869 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.869 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:20.869 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.628819+0000", 2020-09-08T15:40:20.870 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000172127 2020-09-08T15:40:20.870 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.870 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.870 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:20.870 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.628991+0000", 2020-09-08T15:40:20.870 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010851 2020-09-08T15:40:20.871 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:20.871 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:20.871 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:20.871 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:01.629100+0000", 2020-09-08T15:40:20.871 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.247967297 2020-09-08T15:40:20.872 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.872 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.872 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.872 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:20.872 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:20.872 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:40:20.874 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:40:20.906 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:40:20 smithi055 bash[19003]: debug 2020-09-08T15:40:20.619+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:20.975 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:21.417 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:21.519 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:21.546 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2020-09-08T15:40:21.690 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:40:21.906 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:40:21 smithi055 bash[11956]: debug 2020-09-08T15:40:21.415+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:22.083 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:22.186 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:22.389 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:22.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:40:22 smithi104 bash[22305]: debug 2020-09-08T15:40:22.081+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:22.492 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:22.600 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:40:22 smithi055 bash[16646]: debug 2020-09-08T15:40:22.391+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:22.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:22 smithi166 bash[9446]: cluster 2020-09-08T15:40:21.192289+0000 osd.3 (osd.3) 19 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:22.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:22 smithi166 bash[9446]: cluster 2020-09-08T15:40:21.192383+0000 osd.3 (osd.3) 20 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:22.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:22 smithi166 bash[9446]: cluster 2020-09-08T15:40:21.603381+0000 mon.a (mon.0) 1034 : cluster [DBG] osdmap e187: 12 total, 12 up, 12 in 2020-09-08T15:40:22.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:22 smithi166 bash[9446]: cluster 2020-09-08T15:40:21.634471+0000 mgr.x (mgr.24118) 385 : cluster [DBG] pgmap v494: 17 pgs: 17 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 841 op/s 2020-09-08T15:40:22.877 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:22 smithi104 bash[9203]: cluster 2020-09-08T15:40:21.192289+0000 osd.3 (osd.3) 19 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:22.878 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:22 smithi104 bash[9203]: cluster 2020-09-08T15:40:21.192383+0000 osd.3 (osd.3) 20 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:22.878 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:22 smithi104 bash[9203]: cluster 2020-09-08T15:40:21.603381+0000 mon.a (mon.0) 1034 : cluster [DBG] osdmap e187: 12 total, 12 up, 12 in 2020-09-08T15:40:22.878 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:22 smithi104 bash[9203]: cluster 2020-09-08T15:40:21.634471+0000 mgr.x (mgr.24118) 385 : cluster [DBG] pgmap v494: 17 pgs: 17 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 841 op/s 2020-09-08T15:40:22.879 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:22.917 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:22 smithi055 bash[9296]: cluster 2020-09-08T15:40:21.192289+0000 osd.3 (osd.3) 19 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:22.917 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:22 smithi055 bash[9296]: cluster 2020-09-08T15:40:21.192383+0000 osd.3 (osd.3) 20 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:22.918 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:22 smithi055 bash[9296]: cluster 2020-09-08T15:40:21.603381+0000 mon.a (mon.0) 1034 : cluster [DBG] osdmap e187: 12 total, 12 up, 12 in 2020-09-08T15:40:22.918 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:22 smithi055 bash[9296]: cluster 2020-09-08T15:40:21.634471+0000 mgr.x (mgr.24118) 385 : cluster [DBG] pgmap v494: 17 pgs: 17 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 841 op/s 2020-09-08T15:40:22.982 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:23.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:40:22 smithi104 bash[19922]: debug 2020-09-08T15:40:22.877+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:23.289 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:23.393 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:23.524 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:23.605 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:40:23 smithi104 bash[19922]: debug 2020-09-08T15:40:23.289+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:23.629 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:23.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:23 smithi166 bash[9446]: cluster 2020-09-08T15:40:22.471718+0000 osd.6 (osd.6) 19 : cluster [DBG] 2.7 scrub starts 2020-09-08T15:40:23.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:23 smithi055 bash[9296]: cluster 2020-09-08T15:40:22.471718+0000 osd.6 (osd.6) 19 : cluster [DBG] 2.7 scrub starts 2020-09-08T15:40:23.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:40:23 smithi055 bash[16646]: debug 2020-09-08T15:40:23.527+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:23.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:23 smithi104 bash[9203]: cluster 2020-09-08T15:40:22.471718+0000 osd.6 (osd.6) 19 : cluster [DBG] 2.7 scrub starts 2020-09-08T15:40:24.029 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:24.086 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:40:24.131 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:24.249 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:40:24.381 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:24.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:40:24 smithi055 bash[16646]: debug 2020-09-08T15:40:24.375+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:24.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:40:24 smithi104 bash[22305]: debug 2020-09-08T15:40:24.021+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:24.485 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:24.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:24 smithi166 bash[9446]: cluster 2020-09-08T15:40:23.634962+0000 mgr.x (mgr.24118) 386 : cluster [DBG] pgmap v495: 17 pgs: 17 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.28k op/s 2020-09-08T15:40:24.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:24 smithi166 bash[9446]: cluster 2020-09-08T15:40:23.781737+0000 osd.6 (osd.6) 20 : cluster [DBG] 2.7 scrub ok 2020-09-08T15:40:24.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:24 smithi055 bash[9296]: cluster 2020-09-08T15:40:23.634962+0000 mgr.x (mgr.24118) 386 : cluster [DBG] pgmap v495: 17 pgs: 17 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.28k op/s 2020-09-08T15:40:24.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:24 smithi055 bash[9296]: cluster 2020-09-08T15:40:23.781737+0000 osd.6 (osd.6) 20 : cluster [DBG] 2.7 scrub ok 2020-09-08T15:40:24.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:24 smithi104 bash[9203]: cluster 2020-09-08T15:40:23.634962+0000 mgr.x (mgr.24118) 386 : cluster [DBG] pgmap v495: 17 pgs: 17 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.28k op/s 2020-09-08T15:40:24.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:24 smithi104 bash[9203]: cluster 2020-09-08T15:40:23.781737+0000 osd.6 (osd.6) 20 : cluster [DBG] 2.7 scrub ok 2020-09-08T15:40:25.630 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:40:25.635 INFO:teuthology.orchestra.run.smithi104.stderr:reweighted osd.10 to 1 (10000) 2020-09-08T15:40:25.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:25 smithi055 bash[9296]: audit 2020-09-08T15:40:24.751006+0000 mon.a (mon.0) 1035 : audit [INF] from='client.? v1:172.21.15.104:0/3662148493' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:40:25.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:25 smithi055 bash[9296]: cluster 2020-09-08T15:40:24.860883+0000 osd.1 (osd.1) 32 : cluster [DBG] 2.6 scrub starts 2020-09-08T15:40:25.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:25 smithi055 bash[9296]: audit 2020-09-08T15:40:25.589480+0000 mon.a (mon.0) 1036 : audit [INF] from='client.? v1:172.21.15.104:0/2576598796' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 10, "weight": 1.0}]: dispatch 2020-09-08T15:40:25.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:25 smithi104 bash[9203]: audit 2020-09-08T15:40:24.751006+0000 mon.a (mon.0) 1035 : audit [INF] from='client.? v1:172.21.15.104:0/3662148493' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:40:25.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:25 smithi104 bash[9203]: cluster 2020-09-08T15:40:24.860883+0000 osd.1 (osd.1) 32 : cluster [DBG] 2.6 scrub starts 2020-09-08T15:40:25.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:25 smithi104 bash[9203]: audit 2020-09-08T15:40:25.589480+0000 mon.a (mon.0) 1036 : audit [INF] from='client.? v1:172.21.15.104:0/2576598796' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 10, "weight": 1.0}]: dispatch 2020-09-08T15:40:26.067 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:40:26.067 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T15:40:26.068 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:26.068 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:219194 2.7 2:e6907a2a:::benchmark_data_smithi166_20551_object71574:head [delete] snapc 0=[] ondisk+write+known_if_redirected e188)", 2020-09-08T15:40:26.068 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:40:26.063534+0000", 2020-09-08T15:40:26.068 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.0073604789999999996, 2020-09-08T15:40:26.069 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0074034779999999998, 2020-09-08T15:40:26.069 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:26.069 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:40:26.069 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:26.069 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:26.070 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:26.070 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 219194 2020-09-08T15:40:26.070 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:26.070 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:26.070 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:26.071 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:26.071 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:26.063534+0000", 2020-09-08T15:40:26.071 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:26.071 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:26.071 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:26.072 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:26.072 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:26.063534+0000", 2020-09-08T15:40:26.072 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999599 2020-09-08T15:40:26.072 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:26.073 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:26.073 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:26.073 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:26.063493+0000", 2020-09-08T15:40:26.074 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.5445999999999996e-05 2020-09-08T15:40:26.074 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:26.074 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:26.074 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:26.074 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:26.063569+0000", 2020-09-08T15:40:26.075 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2296000000000001e-05 2020-09-08T15:40:26.075 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:26.075 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:26.075 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:26.075 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:26.063581+0000", 2020-09-08T15:40:26.076 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.9820000000000002e-06 2020-09-08T15:40:26.076 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:26.076 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:26.076 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:26.076 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:26.063588+0000", 2020-09-08T15:40:26.076 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.1536000000000002e-05 2020-09-08T15:40:26.077 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:26.077 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:26.077 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:26.077 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:26.063650+0000", 2020-09-08T15:40:26.077 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015198100000000001 2020-09-08T15:40:26.078 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:26.078 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:26.078 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:26.079 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:26.063802+0000", 2020-09-08T15:40:26.079 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019650900000000001 2020-09-08T15:40:26.079 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:26.079 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:26.079 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:40:26.080 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:26.063998+0000", 2020-09-08T15:40:26.080 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00046466699999999999 2020-09-08T15:40:26.080 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:26.080 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:26.080 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:26.080 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:26.081 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:26.081 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:219198 2.7 2:e8b7371f:::benchmark_data_smithi166_20551_object71578:head [delete] snapc 0=[] ondisk+write+known_if_redirected e188)", 2020-09-08T15:40:26.081 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:40:26.065244+0000", 2020-09-08T15:40:26.081 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.0056498090000000004, 2020-09-08T15:40:26.081 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.005779452, 2020-09-08T15:40:26.082 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:26.082 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:40:26.082 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:26.082 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:26.082 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:26.083 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 219198 2020-09-08T15:40:26.083 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:26.083 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:26.083 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:26.083 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:26.084 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:26.065244+0000", 2020-09-08T15:40:26.084 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:26.084 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:26.084 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:26.084 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:26.084 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:26.065244+0000", 2020-09-08T15:40:26.085 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999561 2020-09-08T15:40:26.085 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:26.085 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:26.085 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:26.085 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:26.065201+0000", 2020-09-08T15:40:26.086 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.8905000000000003e-05 2020-09-08T15:40:26.086 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:26.086 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:26.086 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:26.086 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:26.065299+0000", 2020-09-08T15:40:26.087 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.8345000000000001e-05 2020-09-08T15:40:26.087 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:26.087 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:26.087 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:26.087 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:26.065318+0000", 2020-09-08T15:40:26.088 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.6369999999999994e-06 2020-09-08T15:40:26.088 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:26.088 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:26.088 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:26.089 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:26.065327+0000", 2020-09-08T15:40:26.089 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0031077660000000001 2020-09-08T15:40:26.089 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:26.089 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:26.089 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:26.090 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:26.068435+0000", 2020-09-08T15:40:26.090 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015410599999999999 2020-09-08T15:40:26.090 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:26.090 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:26.090 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:26.091 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:26.068589+0000", 2020-09-08T15:40:26.091 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015506 2020-09-08T15:40:26.091 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:26.091 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:26.091 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:40:26.092 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:26.068744+0000", 2020-09-08T15:40:26.092 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0035001310000000001 2020-09-08T15:40:26.092 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:26.092 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:26.092 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:26.092 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:26.093 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2020-09-08T15:40:26.093 INFO:teuthology.orchestra.run.smithi055.stdout: "num_ops": 2 2020-09-08T15:40:26.093 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:40:26.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:25 smithi166 bash[9446]: audit 2020-09-08T15:40:24.751006+0000 mon.a (mon.0) 1035 : audit [INF] from='client.? v1:172.21.15.104:0/3662148493' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:40:26.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:25 smithi166 bash[9446]: cluster 2020-09-08T15:40:24.860883+0000 osd.1 (osd.1) 32 : cluster [DBG] 2.6 scrub starts 2020-09-08T15:40:26.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:25 smithi166 bash[9446]: audit 2020-09-08T15:40:25.589480+0000 mon.a (mon.0) 1036 : audit [INF] from='client.? v1:172.21.15.104:0/2576598796' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 10, "weight": 1.0}]: dispatch 2020-09-08T15:40:26.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:26 smithi055 bash[9296]: cluster 2020-09-08T15:40:25.619840+0000 mon.a (mon.0) 1037 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:40:26.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:26 smithi055 bash[9296]: cluster 2020-09-08T15:40:25.619871+0000 mon.a (mon.0) 1038 : cluster [INF] Cluster is now healthy 2020-09-08T15:40:26.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:26 smithi055 bash[9296]: audit 2020-09-08T15:40:25.628830+0000 mon.a (mon.0) 1039 : audit [INF] from='client.? v1:172.21.15.104:0/3662148493' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:40:26.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:26 smithi055 bash[9296]: audit 2020-09-08T15:40:25.628916+0000 mon.a (mon.0) 1040 : audit [INF] from='client.? v1:172.21.15.104:0/2576598796' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 10, "weight": 1.0}]': finished 2020-09-08T15:40:26.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:26 smithi055 bash[9296]: cluster 2020-09-08T15:40:25.629088+0000 mon.a (mon.0) 1041 : cluster [DBG] osdmap e188: 12 total, 12 up, 12 in 2020-09-08T15:40:26.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:26 smithi055 bash[9296]: cluster 2020-09-08T15:40:25.635324+0000 mgr.x (mgr.24118) 387 : cluster [DBG] pgmap v497: 17 pgs: 17 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 740 op/s 2020-09-08T15:40:26.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:26 smithi055 bash[9296]: cluster 2020-09-08T15:40:26.436600+0000 osd.1 (osd.1) 33 : cluster [DBG] 2.6 scrub ok 2020-09-08T15:40:26.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:26 smithi104 bash[9203]: cluster 2020-09-08T15:40:25.619840+0000 mon.a (mon.0) 1037 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:40:26.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:26 smithi104 bash[9203]: cluster 2020-09-08T15:40:25.619871+0000 mon.a (mon.0) 1038 : cluster [INF] Cluster is now healthy 2020-09-08T15:40:26.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:26 smithi104 bash[9203]: audit 2020-09-08T15:40:25.628830+0000 mon.a (mon.0) 1039 : audit [INF] from='client.? v1:172.21.15.104:0/3662148493' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:40:26.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:26 smithi104 bash[9203]: audit 2020-09-08T15:40:25.628916+0000 mon.a (mon.0) 1040 : audit [INF] from='client.? v1:172.21.15.104:0/2576598796' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 10, "weight": 1.0}]': finished 2020-09-08T15:40:26.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:26 smithi104 bash[9203]: cluster 2020-09-08T15:40:25.629088+0000 mon.a (mon.0) 1041 : cluster [DBG] osdmap e188: 12 total, 12 up, 12 in 2020-09-08T15:40:26.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:26 smithi104 bash[9203]: cluster 2020-09-08T15:40:25.635324+0000 mgr.x (mgr.24118) 387 : cluster [DBG] pgmap v497: 17 pgs: 17 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 740 op/s 2020-09-08T15:40:26.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:26 smithi104 bash[9203]: cluster 2020-09-08T15:40:26.436600+0000 osd.1 (osd.1) 33 : cluster [DBG] 2.6 scrub ok 2020-09-08T15:40:26.953 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2020-09-08T15:40:27.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:26 smithi166 bash[9446]: cluster 2020-09-08T15:40:25.619840+0000 mon.a (mon.0) 1037 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:40:27.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:26 smithi166 bash[9446]: cluster 2020-09-08T15:40:25.619871+0000 mon.a (mon.0) 1038 : cluster [INF] Cluster is now healthy 2020-09-08T15:40:27.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:26 smithi166 bash[9446]: audit 2020-09-08T15:40:25.628830+0000 mon.a (mon.0) 1039 : audit [INF] from='client.? v1:172.21.15.104:0/3662148493' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:40:27.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:26 smithi166 bash[9446]: audit 2020-09-08T15:40:25.628916+0000 mon.a (mon.0) 1040 : audit [INF] from='client.? v1:172.21.15.104:0/2576598796' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 10, "weight": 1.0}]': finished 2020-09-08T15:40:27.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:26 smithi166 bash[9446]: cluster 2020-09-08T15:40:25.629088+0000 mon.a (mon.0) 1041 : cluster [DBG] osdmap e188: 12 total, 12 up, 12 in 2020-09-08T15:40:27.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:26 smithi166 bash[9446]: cluster 2020-09-08T15:40:25.635324+0000 mgr.x (mgr.24118) 387 : cluster [DBG] pgmap v497: 17 pgs: 17 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 740 op/s 2020-09-08T15:40:27.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:26 smithi166 bash[9446]: cluster 2020-09-08T15:40:26.436600+0000 osd.1 (osd.1) 33 : cluster [DBG] 2.6 scrub ok 2020-09-08T15:40:27.146 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:40:27.402 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd reweight 11 1 2020-09-08T15:40:27.612 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:40:27.631 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:27 smithi104 bash[9203]: cluster 2020-09-08T15:40:27.086767+0000 osd.8 (osd.8) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:27.632 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:27 smithi104 bash[9203]: cluster 2020-09-08T15:40:27.086849+0000 osd.8 (osd.8) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:27.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:27 smithi055 bash[9296]: cluster 2020-09-08T15:40:27.086767+0000 osd.8 (osd.8) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:27.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:27 smithi055 bash[9296]: cluster 2020-09-08T15:40:27.086849+0000 osd.8 (osd.8) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:28.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:27 smithi166 bash[9446]: cluster 2020-09-08T15:40:27.086767+0000 osd.8 (osd.8) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:28.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:27 smithi166 bash[9446]: cluster 2020-09-08T15:40:27.086849+0000 osd.8 (osd.8) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:28.848 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:28.867 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:28 smithi055 bash[9296]: cluster 2020-09-08T15:40:27.635899+0000 mgr.x (mgr.24118) 388 : cluster [DBG] pgmap v498: 17 pgs: 17 active+clean; 4.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:40:28.868 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:28 smithi055 bash[9296]: cluster 2020-09-08T15:40:27.645066+0000 mon.a (mon.0) 1042 : cluster [DBG] osdmap e189: 12 total, 12 up, 12 in 2020-09-08T15:40:28.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:28 smithi104 bash[9203]: cluster 2020-09-08T15:40:27.635899+0000 mgr.x (mgr.24118) 388 : cluster [DBG] pgmap v498: 17 pgs: 17 active+clean; 4.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:40:28.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:28 smithi104 bash[9203]: cluster 2020-09-08T15:40:27.645066+0000 mon.a (mon.0) 1042 : cluster [DBG] osdmap e189: 12 total, 12 up, 12 in 2020-09-08T15:40:28.953 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:28.961 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:28 smithi166 bash[9446]: cluster 2020-09-08T15:40:27.635899+0000 mgr.x (mgr.24118) 388 : cluster [DBG] pgmap v498: 17 pgs: 17 active+clean; 4.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:40:28.961 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:28 smithi166 bash[9446]: cluster 2020-09-08T15:40:27.645066+0000 mon.a (mon.0) 1042 : cluster [DBG] osdmap e189: 12 total, 12 up, 12 in 2020-09-08T15:40:28.962 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:40:28 smithi166 bash[11928]: debug 2020-09-08T15:40:28.849+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:29.048 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:29.153 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:29.365 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:40:29 smithi166 bash[11928]: debug 2020-09-08T15:40:29.049+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:29.380 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:40:29.548 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:40:29.576 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:29.682 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:29.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:40:29 smithi104 bash[19922]: debug 2020-09-08T15:40:29.577+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:29.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:29 smithi104 bash[9203]: cluster 2020-09-08T15:40:28.561266+0000 osd.11 (osd.11) 19 : cluster [DBG] 2.a scrub starts 2020-09-08T15:40:29.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:29 smithi104 bash[9203]: cluster 2020-09-08T15:40:28.894453+0000 osd.1 (osd.1) 34 : cluster [DBG] 2.9 scrub starts 2020-09-08T15:40:29.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:29 smithi104 bash[9203]: cluster 2020-09-08T15:40:29.636357+0000 mgr.x (mgr.24118) 389 : cluster [DBG] pgmap v500: 17 pgs: 17 active+clean; 4.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T15:40:30.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:29 smithi166 bash[9446]: cluster 2020-09-08T15:40:28.561266+0000 osd.11 (osd.11) 19 : cluster [DBG] 2.a scrub starts 2020-09-08T15:40:30.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:29 smithi166 bash[9446]: cluster 2020-09-08T15:40:28.894453+0000 osd.1 (osd.1) 34 : cluster [DBG] 2.9 scrub starts 2020-09-08T15:40:30.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:29 smithi166 bash[9446]: cluster 2020-09-08T15:40:29.636357+0000 mgr.x (mgr.24118) 389 : cluster [DBG] pgmap v500: 17 pgs: 17 active+clean; 4.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T15:40:30.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:29 smithi055 bash[9296]: cluster 2020-09-08T15:40:28.561266+0000 osd.11 (osd.11) 19 : cluster [DBG] 2.a scrub starts 2020-09-08T15:40:30.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:29 smithi055 bash[9296]: cluster 2020-09-08T15:40:28.894453+0000 osd.1 (osd.1) 34 : cluster [DBG] 2.9 scrub starts 2020-09-08T15:40:30.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:29 smithi055 bash[9296]: cluster 2020-09-08T15:40:29.636357+0000 mgr.x (mgr.24118) 389 : cluster [DBG] pgmap v500: 17 pgs: 17 active+clean; 4.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T15:40:30.429 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:30.532 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:40:30.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:40:30 smithi104 bash[19922]: debug 2020-09-08T15:40:30.421+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:31.254 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:40:31.255 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T15:40:31.255 INFO:teuthology.orchestra.run.smithi055.stdout: "complaint_time": 30, 2020-09-08T15:40:31.255 INFO:teuthology.orchestra.run.smithi055.stdout: "num_blocked_ops": 0 2020-09-08T15:40:31.255 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:40:31.386 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:40:31.490 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:31.619 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:31.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:40:31 smithi104 bash[17588]: debug 2020-09-08T15:40:31.381+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:31.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:31 smithi104 bash[9203]: cluster 2020-09-08T15:40:30.575699+0000 osd.11 (osd.11) 20 : cluster [DBG] 2.a scrub ok 2020-09-08T15:40:31.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:31 smithi104 bash[9203]: audit 2020-09-08T15:40:31.558143+0000 mon.a (mon.0) 1043 : audit [DBG] from='client.? v1:172.21.15.104:0/3216620731' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:40:31.733 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:31.791 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:40:31.834 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:40:31.865 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:40:31 smithi166 bash[11928]: debug 2020-09-08T15:40:31.621+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:31.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:31 smithi166 bash[9446]: cluster 2020-09-08T15:40:30.575699+0000 osd.11 (osd.11) 20 : cluster [DBG] 2.a scrub ok 2020-09-08T15:40:31.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:31 smithi166 bash[9446]: audit 2020-09-08T15:40:31.558143+0000 mon.a (mon.0) 1043 : audit [DBG] from='client.? v1:172.21.15.104:0/3216620731' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:40:31.872 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:40:31.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:31 smithi055 bash[9296]: cluster 2020-09-08T15:40:30.575699+0000 osd.11 (osd.11) 20 : cluster [DBG] 2.a scrub ok 2020-09-08T15:40:31.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:31 smithi055 bash[9296]: audit 2020-09-08T15:40:31.558143+0000 mon.a (mon.0) 1043 : audit [DBG] from='client.? v1:172.21.15.104:0/3216620731' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:40:31.908 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:40:31.945 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:40:31.976 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:40:32.000 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:40:32.025 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:40:32.035 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2020-09-08T15:40:32.080 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:32.080 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:40:32.122 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:40:32.155 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:40:32.182 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:40:32.196 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:40:32.197 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:40:32.208 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:40:32.209 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:40:32.209 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:40:32.209 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:40:32.209 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:40:32.210 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:40:32.210 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:40:32.210 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:40:32.210 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:40:32.210 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:40:32.211 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:40:32.211 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:40:32.446 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:40:32 smithi104 bash[19922]: debug 2020-09-08T15:40:32.073+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:32.553 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:40:32.656 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:32.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:32 smithi166 bash[9446]: cluster 2020-09-08T15:40:31.090292+0000 osd.1 (osd.1) 35 : cluster [DBG] 2.9 scrub ok 2020-09-08T15:40:32.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:32 smithi166 bash[9446]: cluster 2020-09-08T15:40:31.636917+0000 mgr.x (mgr.24118) 390 : cluster [DBG] pgmap v501: 17 pgs: 17 active+clean; 4.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 557 op/s 2020-09-08T15:40:32.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:40:32 smithi055 bash[14301]: debug 2020-09-08T15:40:32.551+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:32.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:32 smithi055 bash[9296]: cluster 2020-09-08T15:40:31.090292+0000 osd.1 (osd.1) 35 : cluster [DBG] 2.9 scrub ok 2020-09-08T15:40:32.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:32 smithi055 bash[9296]: cluster 2020-09-08T15:40:31.636917+0000 mgr.x (mgr.24118) 390 : cluster [DBG] pgmap v501: 17 pgs: 17 active+clean; 4.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 557 op/s 2020-09-08T15:40:32.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:32 smithi104 bash[9203]: cluster 2020-09-08T15:40:31.090292+0000 osd.1 (osd.1) 35 : cluster [DBG] 2.9 scrub ok 2020-09-08T15:40:32.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:32 smithi104 bash[9203]: cluster 2020-09-08T15:40:31.636917+0000 mgr.x (mgr.24118) 390 : cluster [DBG] pgmap v501: 17 pgs: 17 active+clean; 4.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 557 op/s 2020-09-08T15:40:32.967 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:33.071 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:33.311 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:33.408 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:40:33 smithi055 bash[11956]: debug 2020-09-08T15:40:33.311+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:33.416 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:40:33.478 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:40:32 smithi104 bash[15273]: debug 2020-09-08T15:40:32.965+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:33.582 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:40:33.688 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:40:33.689 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:40:33 smithi055 bash[19003]: debug 2020-09-08T15:40:33.583+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:33.815 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:40:33.925 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:34.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:33 smithi166 bash[9446]: cluster 2020-09-08T15:40:33.637416+0000 mgr.x (mgr.24118) 391 : cluster [DBG] pgmap v502: 17 pgs: 17 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T15:40:34.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:33 smithi055 bash[9296]: cluster 2020-09-08T15:40:33.637416+0000 mgr.x (mgr.24118) 391 : cluster [DBG] pgmap v502: 17 pgs: 17 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T15:40:34.159 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:40:33 smithi055 bash[14301]: debug 2020-09-08T15:40:33.815+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:34.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:33 smithi104 bash[9203]: cluster 2020-09-08T15:40:33.637416+0000 mgr.x (mgr.24118) 391 : cluster [DBG] pgmap v502: 17 pgs: 17 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T15:40:34.220 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:34.323 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:40:34.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:40:34 smithi104 bash[22305]: debug 2020-09-08T15:40:34.213+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:34.684 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:40:34.787 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:35.095 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:35.098 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:34 smithi104 bash[9203]: cluster 2020-09-08T15:40:33.790578+0000 osd.2 (osd.2) 26 : cluster [DBG] 2.1 scrub starts 2020-09-08T15:40:35.099 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:34 smithi104 bash[9203]: cluster 2020-09-08T15:40:34.045099+0000 osd.1 (osd.1) 36 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:35.099 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:34 smithi104 bash[9203]: cluster 2020-09-08T15:40:34.045198+0000 osd.1 (osd.1) 37 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:35.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:34 smithi166 bash[9446]: cluster 2020-09-08T15:40:33.790578+0000 osd.2 (osd.2) 26 : cluster [DBG] 2.1 scrub starts 2020-09-08T15:40:35.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:34 smithi166 bash[9446]: cluster 2020-09-08T15:40:34.045099+0000 osd.1 (osd.1) 36 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:35.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:34 smithi166 bash[9446]: cluster 2020-09-08T15:40:34.045198+0000 osd.1 (osd.1) 37 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:35.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:40:34 smithi055 bash[19003]: debug 2020-09-08T15:40:34.683+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:34 smithi055 bash[9296]: cluster 2020-09-08T15:40:33.790578+0000 osd.2 (osd.2) 26 : cluster [DBG] 2.1 scrub starts 2020-09-08T15:40:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:34 smithi055 bash[9296]: cluster 2020-09-08T15:40:34.045099+0000 osd.1 (osd.1) 36 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:35.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:34 smithi055 bash[9296]: cluster 2020-09-08T15:40:34.045198+0000 osd.1 (osd.1) 37 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:35.201 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:35.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:40:35 smithi104 bash[22305]: debug 2020-09-08T15:40:35.093+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:35.790 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:35.894 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:36.051 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:36.052 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:35 smithi055 bash[9296]: cluster 2020-09-08T15:40:34.706511+0000 mon.a (mon.0) 1044 : cluster [DBG] osdmap e190: 12 total, 12 up, 12 in 2020-09-08T15:40:36.052 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:35 smithi055 bash[9296]: cluster 2020-09-08T15:40:35.157056+0000 osd.5 (osd.5) 23 : cluster [DBG] 2.2 scrub starts 2020-09-08T15:40:36.052 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:35 smithi055 bash[9296]: cluster 2020-09-08T15:40:35.637873+0000 mgr.x (mgr.24118) 392 : cluster [DBG] pgmap v504: 17 pgs: 17 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 701 op/s 2020-09-08T15:40:36.105 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:35 smithi104 bash[9203]: cluster 2020-09-08T15:40:34.706511+0000 mon.a (mon.0) 1044 : cluster [DBG] osdmap e190: 12 total, 12 up, 12 in 2020-09-08T15:40:36.105 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:35 smithi104 bash[9203]: cluster 2020-09-08T15:40:35.157056+0000 osd.5 (osd.5) 23 : cluster [DBG] 2.2 scrub starts 2020-09-08T15:40:36.105 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:35 smithi104 bash[9203]: cluster 2020-09-08T15:40:35.637873+0000 mgr.x (mgr.24118) 392 : cluster [DBG] pgmap v504: 17 pgs: 17 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 701 op/s 2020-09-08T15:40:36.105 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:40:35 smithi104 bash[19922]: debug 2020-09-08T15:40:35.789+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:36.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:35 smithi166 bash[9446]: cluster 2020-09-08T15:40:34.706511+0000 mon.a (mon.0) 1044 : cluster [DBG] osdmap e190: 12 total, 12 up, 12 in 2020-09-08T15:40:36.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:35 smithi166 bash[9446]: cluster 2020-09-08T15:40:35.157056+0000 osd.5 (osd.5) 23 : cluster [DBG] 2.2 scrub starts 2020-09-08T15:40:36.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:35 smithi166 bash[9446]: cluster 2020-09-08T15:40:35.637873+0000 mgr.x (mgr.24118) 392 : cluster [DBG] pgmap v504: 17 pgs: 17 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 701 op/s 2020-09-08T15:40:36.156 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:40:36.262 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:40:36.290 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:40:36.394 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:36.402 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:40:36 smithi055 bash[11956]: debug 2020-09-08T15:40:36.051+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:36.428 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:40:36.450 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:40:36.451 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:40:36.505 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:36.611 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:40:36.615 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:40:36 smithi166 bash[18960]: debug 2020-09-08T15:40:36.289+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:36.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:40:36 smithi055 bash[11956]: debug 2020-09-08T15:40:36.507+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:36.719 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:40:36.723 INFO:teuthology.orchestra.run.smithi104.stderr:reweighted osd.11 to 1 (10000) 2020-09-08T15:40:36.731 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:40:36.833 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:36.958 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:36 smithi166 bash[9446]: cluster 2020-09-08T15:40:35.304759+0000 osd.2 (osd.2) 27 : cluster [DBG] 2.1 scrub ok 2020-09-08T15:40:36.959 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:36 smithi166 bash[9446]: audit 2020-09-08T15:40:35.815085+0000 mon.a (mon.0) 1045 : audit [INF] from='client.? v1:172.21.15.104:0/2434471136' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 11, "weight": 1.0}]: dispatch 2020-09-08T15:40:36.959 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:36 smithi166 bash[9446]: audit 2020-09-08T15:40:36.635678+0000 mon.c (mon.2) 41 : audit [INF] from='client.? v1:172.21.15.104:0/328113702' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:40:36.959 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:36 smithi166 bash[9446]: audit 2020-09-08T15:40:36.636637+0000 mon.a (mon.0) 1046 : audit [INF] from='client.34919 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:40:36.960 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:40:36 smithi166 bash[14254]: debug 2020-09-08T15:40:36.721+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:36.964 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:37.067 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:40:37.081 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:36 smithi055 bash[9296]: cluster 2020-09-08T15:40:35.304759+0000 osd.2 (osd.2) 27 : cluster [DBG] 2.1 scrub ok 2020-09-08T15:40:37.081 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:36 smithi055 bash[9296]: audit 2020-09-08T15:40:35.815085+0000 mon.a (mon.0) 1045 : audit [INF] from='client.? v1:172.21.15.104:0/2434471136' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 11, "weight": 1.0}]: dispatch 2020-09-08T15:40:37.081 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:36 smithi055 bash[9296]: audit 2020-09-08T15:40:36.635678+0000 mon.c (mon.2) 41 : audit [INF] from='client.? v1:172.21.15.104:0/328113702' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:40:37.081 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:36 smithi055 bash[9296]: audit 2020-09-08T15:40:36.636637+0000 mon.a (mon.0) 1046 : audit [INF] from='client.34919 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:40:37.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:36 smithi104 bash[9203]: cluster 2020-09-08T15:40:35.304759+0000 osd.2 (osd.2) 27 : cluster [DBG] 2.1 scrub ok 2020-09-08T15:40:37.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:36 smithi104 bash[9203]: audit 2020-09-08T15:40:35.815085+0000 mon.a (mon.0) 1045 : audit [INF] from='client.? v1:172.21.15.104:0/2434471136' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 11, "weight": 1.0}]: dispatch 2020-09-08T15:40:37.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:36 smithi104 bash[9203]: audit 2020-09-08T15:40:36.635678+0000 mon.c (mon.2) 41 : audit [INF] from='client.? v1:172.21.15.104:0/328113702' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:40:37.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:36 smithi104 bash[9203]: audit 2020-09-08T15:40:36.636637+0000 mon.a (mon.0) 1046 : audit [INF] from='client.34919 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:40:37.365 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:40:36 smithi166 bash[16593]: debug 2020-09-08T15:40:36.957+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:37.404 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:40:37 smithi055 bash[19003]: debug 2020-09-08T15:40:37.407+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:37.408 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:40:37.512 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:40:37.532 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2020-09-08T15:40:37.628 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:40:37.680 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:40:37.734 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:40:37.855 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:37 smithi166 bash[9446]: cluster 2020-09-08T15:40:36.687463+0000 osd.5 (osd.5) 24 : cluster [DBG] 2.2 scrub ok 2020-09-08T15:40:37.855 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:37 smithi166 bash[9446]: cluster 2020-09-08T15:40:36.712006+0000 mon.a (mon.0) 1047 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:40:37.855 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:37 smithi166 bash[9446]: audit 2020-09-08T15:40:36.725532+0000 mon.a (mon.0) 1048 : audit [INF] from='client.? v1:172.21.15.104:0/2434471136' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 11, "weight": 1.0}]': finished 2020-09-08T15:40:37.856 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:37 smithi166 bash[9446]: audit 2020-09-08T15:40:36.725568+0000 mon.a (mon.0) 1049 : audit [INF] from='client.34919 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:40:37.856 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:37 smithi166 bash[9446]: cluster 2020-09-08T15:40:36.725595+0000 mon.a (mon.0) 1050 : cluster [DBG] osdmap e191: 12 total, 12 up, 12 in 2020-09-08T15:40:37.856 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:37 smithi166 bash[9446]: cluster 2020-09-08T15:40:37.638416+0000 mgr.x (mgr.24118) 393 : cluster [DBG] pgmap v506: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 928 op/s 2020-09-08T15:40:37.857 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:40:37 smithi166 bash[14254]: debug 2020-09-08T15:40:37.625+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:37.857 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:40:37 smithi166 bash[18960]: debug 2020-09-08T15:40:37.853+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:37.858 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:40:37.961 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:38.061 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:38.115 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:40:38 smithi166 bash[11928]: debug 2020-09-08T15:40:38.061+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:38.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:37 smithi055 bash[9296]: cluster 2020-09-08T15:40:36.687463+0000 osd.5 (osd.5) 24 : cluster [DBG] 2.2 scrub ok 2020-09-08T15:40:38.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:37 smithi055 bash[9296]: cluster 2020-09-08T15:40:36.712006+0000 mon.a (mon.0) 1047 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:40:38.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:37 smithi055 bash[9296]: audit 2020-09-08T15:40:36.725532+0000 mon.a (mon.0) 1048 : audit [INF] from='client.? v1:172.21.15.104:0/2434471136' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 11, "weight": 1.0}]': finished 2020-09-08T15:40:38.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:37 smithi055 bash[9296]: audit 2020-09-08T15:40:36.725568+0000 mon.a (mon.0) 1049 : audit [INF] from='client.34919 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:40:38.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:37 smithi055 bash[9296]: cluster 2020-09-08T15:40:36.725595+0000 mon.a (mon.0) 1050 : cluster [DBG] osdmap e191: 12 total, 12 up, 12 in 2020-09-08T15:40:38.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:37 smithi055 bash[9296]: cluster 2020-09-08T15:40:37.638416+0000 mgr.x (mgr.24118) 393 : cluster [DBG] pgmap v506: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 928 op/s 2020-09-08T15:40:38.165 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:38.172 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:37 smithi104 bash[9203]: cluster 2020-09-08T15:40:36.687463+0000 osd.5 (osd.5) 24 : cluster [DBG] 2.2 scrub ok 2020-09-08T15:40:38.172 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:37 smithi104 bash[9203]: cluster 2020-09-08T15:40:36.712006+0000 mon.a (mon.0) 1047 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:40:38.172 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:37 smithi104 bash[9203]: audit 2020-09-08T15:40:36.725532+0000 mon.a (mon.0) 1048 : audit [INF] from='client.? v1:172.21.15.104:0/2434471136' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 11, "weight": 1.0}]': finished 2020-09-08T15:40:38.173 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:37 smithi104 bash[9203]: audit 2020-09-08T15:40:36.725568+0000 mon.a (mon.0) 1049 : audit [INF] from='client.34919 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:40:38.173 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:37 smithi104 bash[9203]: cluster 2020-09-08T15:40:36.725595+0000 mon.a (mon.0) 1050 : cluster [DBG] osdmap e191: 12 total, 12 up, 12 in 2020-09-08T15:40:38.173 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:37 smithi104 bash[9203]: cluster 2020-09-08T15:40:37.638416+0000 mgr.x (mgr.24118) 393 : cluster [DBG] pgmap v506: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 928 op/s 2020-09-08T15:40:38.603 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:38.706 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:38.788 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2020-09-08T15:40:38.788 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph pg dump --format=json 2020-09-08T15:40:38.864 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:40:38 smithi104 bash[19922]: debug 2020-09-08T15:40:38.601+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:38.913 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:38.955 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:40:39.016 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:39.129 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:40:38 smithi055 bash[11956]: debug 2020-09-08T15:40:38.911+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:39.151 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:39.256 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:39.365 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:40:39 smithi166 bash[11928]: debug 2020-09-08T15:40:39.145+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:39.413 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:39.516 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:39.611 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:39.718 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:39.727 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:40:39 smithi055 bash[16646]: debug 2020-09-08T15:40:39.415+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:39.865 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:40:39 smithi166 bash[11928]: debug 2020-09-08T15:40:39.613+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:40.159 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:40.262 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:40.383 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:40.486 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:40.597 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:40:40 smithi166 bash[16593]: debug 2020-09-08T15:40:40.381+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:40.598 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:40.618 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:40:40 smithi055 bash[16646]: debug 2020-09-08T15:40:40.159+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:40.618 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:40 smithi055 bash[9296]: cluster 2020-09-08T15:40:39.639002+0000 mgr.x (mgr.24118) 394 : cluster [DBG] pgmap v507: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T15:40:40.618 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:40 smithi055 bash[9296]: cluster 2020-09-08T15:40:40.153618+0000 osd.6 (osd.6) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:40.619 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:40 smithi055 bash[9296]: cluster 2020-09-08T15:40:40.153717+0000 osd.6 (osd.6) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:40.705 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:40.865 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:40:40 smithi166 bash[16593]: debug 2020-09-08T15:40:40.597+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:40.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:40 smithi166 bash[9446]: cluster 2020-09-08T15:40:39.639002+0000 mgr.x (mgr.24118) 394 : cluster [DBG] pgmap v507: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T15:40:40.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:40 smithi166 bash[9446]: cluster 2020-09-08T15:40:40.153618+0000 osd.6 (osd.6) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:40.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:40 smithi166 bash[9446]: cluster 2020-09-08T15:40:40.153717+0000 osd.6 (osd.6) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:40.868 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:40:40.890 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:40 smithi104 bash[9203]: cluster 2020-09-08T15:40:39.639002+0000 mgr.x (mgr.24118) 394 : cluster [DBG] pgmap v507: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T15:40:40.890 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:40 smithi104 bash[9203]: cluster 2020-09-08T15:40:40.153618+0000 osd.6 (osd.6) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:40.891 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:40 smithi104 bash[9203]: cluster 2020-09-08T15:40:40.153717+0000 osd.6 (osd.6) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:40.900 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:40:40 smithi055 bash[11956]: debug 2020-09-08T15:40:40.899+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:40.901 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:41.008 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:41.058 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:40:41.358 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:41.461 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:41.645 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:41.653 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:40:41 smithi104 bash[15273]: debug 2020-09-08T15:40:41.357+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:41.747 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:40:41.829 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:40:41 smithi055 bash[11956]: debug 2020-09-08T15:40:41.635+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:41.829 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:41 smithi055 bash[9296]: cluster 2020-09-08T15:40:40.627959+0000 mon.a (mon.0) 1051 : cluster [DBG] osdmap e192: 12 total, 12 up, 12 in 2020-09-08T15:40:41.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:41 smithi104 bash[9203]: cluster 2020-09-08T15:40:40.627959+0000 mon.a (mon.0) 1051 : cluster [DBG] osdmap e192: 12 total, 12 up, 12 in 2020-09-08T15:40:42.070 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:40:42.071 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:40:42.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:41 smithi166 bash[9446]: cluster 2020-09-08T15:40:40.627959+0000 mon.a (mon.0) 1051 : cluster [DBG] osdmap e192: 12 total, 12 up, 12 in 2020-09-08T15:40:42.450 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:40:42.554 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:40:42.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:42 smithi055 bash[9296]: cluster 2020-09-08T15:40:41.639481+0000 mgr.x (mgr.24118) 395 : cluster [DBG] pgmap v509: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 690 op/s 2020-09-08T15:40:42.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:42 smithi104 bash[9203]: cluster 2020-09-08T15:40:41.639481+0000 mgr.x (mgr.24118) 395 : cluster [DBG] pgmap v509: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 690 op/s 2020-09-08T15:40:42.927 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:40:42 smithi104 bash[17588]: debug 2020-09-08T15:40:42.449+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:42.978 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:40:43.081 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:43.117 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2020-09-08T15:40:43.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:42 smithi166 bash[9446]: cluster 2020-09-08T15:40:41.639481+0000 mgr.x (mgr.24118) 395 : cluster [DBG] pgmap v509: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 690 op/s 2020-09-08T15:40:43.215 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:43.267 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:40:43.322 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:43.365 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:40:43 smithi166 bash[16593]: debug 2020-09-08T15:40:43.213+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:43.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:40:42 smithi055 bash[14301]: debug 2020-09-08T15:40:42.979+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:43.637 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:43 smithi104 bash[9203]: cluster 2020-09-08T15:40:42.450125+0000 osd.11 (osd.11) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:43.637 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:43 smithi104 bash[9203]: cluster 2020-09-08T15:40:42.450222+0000 osd.11 (osd.11) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:43.637 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:43 smithi104 bash[9203]: cluster 2020-09-08T15:40:42.639936+0000 mon.a (mon.0) 1052 : cluster [DBG] osdmap e193: 12 total, 12 up, 12 in 2020-09-08T15:40:43.782 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:43.887 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:44.372 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:40:43 smithi104 bash[19922]: debug 2020-09-08T15:40:43.781+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:44.373 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:43 smithi166 bash[9446]: cluster 2020-09-08T15:40:42.450125+0000 osd.11 (osd.11) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:44.374 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:43 smithi166 bash[9446]: cluster 2020-09-08T15:40:42.450222+0000 osd.11 (osd.11) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:44.374 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:43 smithi166 bash[9446]: cluster 2020-09-08T15:40:42.639936+0000 mon.a (mon.0) 1052 : cluster [DBG] osdmap e193: 12 total, 12 up, 12 in 2020-09-08T15:40:44.374 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:43 smithi055 bash[9296]: cluster 2020-09-08T15:40:42.450125+0000 osd.11 (osd.11) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:44.374 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:43 smithi055 bash[9296]: cluster 2020-09-08T15:40:42.450222+0000 osd.11 (osd.11) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:44.375 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:43 smithi055 bash[9296]: cluster 2020-09-08T15:40:42.639936+0000 mon.a (mon.0) 1052 : cluster [DBG] osdmap e193: 12 total, 12 up, 12 in 2020-09-08T15:40:44.502 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:44.586 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:40:44.604 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:40:44.625 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:40:44.640 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:44 smithi166 bash[9446]: cluster 2020-09-08T15:40:43.639962+0000 mgr.x (mgr.24118) 396 : cluster [DBG] pgmap v511: 17 pgs: 17 active+clean; 3.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.30k op/s 2020-09-08T15:40:44.640 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:44 smithi166 bash[9446]: audit 2020-09-08T15:40:44.560372+0000 mon.b (mon.1) 80 : audit [DBG] from='client.? v1:172.21.15.104:0/3101577771' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:40:44.641 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:40:44 smithi166 bash[11928]: debug 2020-09-08T15:40:44.497+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:44.671 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:40:44.707 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:40:44.721 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:40:44.748 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:40:44.782 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:40:44.823 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:44.825 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:40:44.864 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:40:44.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:44 smithi055 bash[9296]: cluster 2020-09-08T15:40:43.639962+0000 mgr.x (mgr.24118) 396 : cluster [DBG] pgmap v511: 17 pgs: 17 active+clean; 3.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.30k op/s 2020-09-08T15:40:44.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:44 smithi055 bash[9296]: audit 2020-09-08T15:40:44.560372+0000 mon.b (mon.1) 80 : audit [DBG] from='client.? v1:172.21.15.104:0/3101577771' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:40:44.908 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:40:44 smithi055 bash[14301]: debug 2020-09-08T15:40:44.711+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:44.908 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:40:44.941 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:40:44.948 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:40:44.954 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:44 smithi104 bash[9203]: cluster 2020-09-08T15:40:43.639962+0000 mgr.x (mgr.24118) 396 : cluster [DBG] pgmap v511: 17 pgs: 17 active+clean; 3.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.30k op/s 2020-09-08T15:40:44.955 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:44 smithi104 bash[9203]: audit 2020-09-08T15:40:44.560372+0000 mon.b (mon.1) 80 : audit [DBG] from='client.? v1:172.21.15.104:0/3101577771' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:40:44.987 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:40:45.025 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:40:45.027 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:40:45.027 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:40:45.027 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:40:45.028 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:40:45.028 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:40:45.028 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:40:45.028 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:40:45.028 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:40:45.029 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:40:45.029 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:40:45.029 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:40:45.029 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:40:45.048 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:45.178 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:45.282 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:45.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:40:45 smithi166 bash[11928]: debug 2020-09-08T15:40:45.173+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:45.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:40:44 smithi055 bash[11956]: debug 2020-09-08T15:40:44.939+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:45.802 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:45.906 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:46.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:40:45 smithi104 bash[19922]: debug 2020-09-08T15:40:45.797+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:46.206 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T15:40:46.223 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2020-09-08T15:40:46.327 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:40:46.428 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:46.603 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:40:46 smithi104 bash[19922]: debug 2020-09-08T15:40:46.325+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:46.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:46 smithi055 bash[9296]: cluster 2020-09-08T15:40:45.640467+0000 mgr.x (mgr.24118) 397 : cluster [DBG] pgmap v512: 17 pgs: 17 active+clean; 3.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 836 op/s 2020-09-08T15:40:46.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:46 smithi104 bash[9203]: cluster 2020-09-08T15:40:45.640467+0000 mgr.x (mgr.24118) 397 : cluster [DBG] pgmap v512: 17 pgs: 17 active+clean; 3.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 836 op/s 2020-09-08T15:40:47.015 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:47.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:46 smithi166 bash[9446]: cluster 2020-09-08T15:40:45.640467+0000 mgr.x (mgr.24118) 397 : cluster [DBG] pgmap v512: 17 pgs: 17 active+clean; 3.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 836 op/s 2020-09-08T15:40:47.119 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:47.217 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:40:47.218 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T15:40:47.218 INFO:teuthology.orchestra.run.smithi055.stdout: "complaint_time": 30, 2020-09-08T15:40:47.219 INFO:teuthology.orchestra.run.smithi055.stdout: "num_blocked_ops": 0 2020-09-08T15:40:47.219 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:40:47.427 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:40:47 smithi104 bash[22305]: debug 2020-09-08T15:40:47.021+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:47.529 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:47.631 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:47.645 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:47 smithi055 bash[9296]: audit 2020-09-08T15:40:46.194849+0000 mgr.x (mgr.24118) 398 : audit [DBG] from='client.15048 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:40:47.645 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:47 smithi055 bash[9296]: cluster 2020-09-08T15:40:46.317634+0000 osd.4 (osd.4) 19 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:47.645 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:47 smithi055 bash[9296]: cluster 2020-09-08T15:40:46.317709+0000 osd.4 (osd.4) 20 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:47.646 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:47 smithi055 bash[9296]: cluster 2020-09-08T15:40:46.656019+0000 mon.a (mon.0) 1053 : cluster [DBG] osdmap e194: 12 total, 12 up, 12 in 2020-09-08T15:40:47.646 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:47 smithi055 bash[9296]: cluster 2020-09-08T15:40:47.641116+0000 mgr.x (mgr.24118) 399 : cluster [DBG] pgmap v514: 17 pgs: 17 active+clean; 3.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.22k op/s 2020-09-08T15:40:47.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:47 smithi104 bash[9203]: audit 2020-09-08T15:40:46.194849+0000 mgr.x (mgr.24118) 398 : audit [DBG] from='client.15048 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:40:47.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:47 smithi104 bash[9203]: cluster 2020-09-08T15:40:46.317634+0000 osd.4 (osd.4) 19 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:47.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:47 smithi104 bash[9203]: cluster 2020-09-08T15:40:46.317709+0000 osd.4 (osd.4) 20 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:47.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:47 smithi104 bash[9203]: cluster 2020-09-08T15:40:46.656019+0000 mon.a (mon.0) 1053 : cluster [DBG] osdmap e194: 12 total, 12 up, 12 in 2020-09-08T15:40:47.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:47 smithi104 bash[9203]: cluster 2020-09-08T15:40:47.641116+0000 mgr.x (mgr.24118) 399 : cluster [DBG] pgmap v514: 17 pgs: 17 active+clean; 3.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.22k op/s 2020-09-08T15:40:47.928 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:40:47 smithi104 bash[15273]: debug 2020-09-08T15:40:47.525+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:48.060 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2020-09-08T15:40:48.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:47 smithi166 bash[9446]: audit 2020-09-08T15:40:46.194849+0000 mgr.x (mgr.24118) 398 : audit [DBG] from='client.15048 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:40:48.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:47 smithi166 bash[9446]: cluster 2020-09-08T15:40:46.317634+0000 osd.4 (osd.4) 19 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:48.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:47 smithi166 bash[9446]: cluster 2020-09-08T15:40:46.317709+0000 osd.4 (osd.4) 20 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:48.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:47 smithi166 bash[9446]: cluster 2020-09-08T15:40:46.656019+0000 mon.a (mon.0) 1053 : cluster [DBG] osdmap e194: 12 total, 12 up, 12 in 2020-09-08T15:40:48.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:47 smithi166 bash[9446]: cluster 2020-09-08T15:40:47.641116+0000 mgr.x (mgr.24118) 399 : cluster [DBG] pgmap v514: 17 pgs: 17 active+clean; 3.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.22k op/s 2020-09-08T15:40:48.120 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2020-09-08T15:40:45.640214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4021813272,"num_objects":61372,"num_object_clones":0,"num_object_copies":122744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61372,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":233870,"num_write_kb":9447492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160873,"ondisk_log_size":160873,"up":34,"acting":34,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":8521528,"kb_used_data":7940472,"kb_used_omap":0,"kb_used_meta":579456,"kb_avail":1116420296,"statfs":{"total":1151940427776,"available":1143214383104,"internally_reserved":0,"allocated":8131043328,"data_stored":8128128839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":593362944},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-438566912,"num_objects":-6692,"num_object_clones":0,"num_object_copies":-13384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6692,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002134"},"pg_stats":[{"pgid":"2.f","version":"192'14369","reported_seq":"14445","reported_epoch":"192","state":"active+clean","last_fresh":"2020-09-08T15:40:41.773800+0000","last_change":"2020-09-08T15:39:26.660710+0000","last_active":"2020-09-08T15:40:41.773800+0000","last_peered":"2020-09-08T15:40:41.773800+0000","last_clean":"2020-09-08T15:40:41.773800+0000","last_became_active":"2020-09-08T15:37:29.882656+0000","last_became_peered":"2020-09-08T15:37:29.882656+0000","last_unstale":"2020-09-08T15:40:41.773800+0000","last_undegraded":"2020-09-08T15:40:41.773800+0000","last_fullsized":"2020-09-08T15:40:41.773800+0000","mapping_epoch":126,"log_start":"140'4300","ondisk_log_start":"140'4300","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"168'9930","last_scrub_stamp":"2020-09-08T15:39:26.660646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:39:26.660646+0000","log_size":10069,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":251461632,"num_objects":3837,"num_object_clones":0,"num_object_copies":7674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14369,"num_write_kb":582592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9],"acting":[8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"192'14646","reported_seq":"14722","reported_epoch":"192","state":"active+clean","last_fresh":"2020-09-08T15:40:41.764712+0000","last_change":"2020-09-08T15:39:26.488598+0000","last_active":"2020-09-08T15:40:41.764712+0000","last_peered":"2020-09-08T15:40:41.764712+0000","last_clean":"2020-09-08T15:40:41.764712+0000","last_became_active":"2020-09-08T15:37:30.089439+0000","last_became_peered":"2020-09-08T15:37:30.089439+0000","last_unstale":"2020-09-08T15:40:41.764712+0000","last_undegraded":"2020-09-08T15:40:41.764712+0000","last_fullsized":"2020-09-08T15:40:41.764712+0000","mapping_epoch":126,"log_start":"141'4600","ondisk_log_start":"141'4600","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"168'10008","last_scrub_stamp":"2020-09-08T15:39:26.488528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:39:26.488528+0000","log_size":10046,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":249692160,"num_objects":3810,"num_object_clones":0,"num_object_copies":7620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14646,"num_write_kb":590592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"192'14520","reported_seq":"14599","reported_epoch":"192","state":"active+clean","last_fresh":"2020-09-08T15:40:41.643368+0000","last_change":"2020-09-08T15:40:36.687544+0000","last_active":"2020-09-08T15:40:41.643368+0000","last_peered":"2020-09-08T15:40:41.643368+0000","last_clean":"2020-09-08T15:40:41.643368+0000","last_became_active":"2020-09-08T15:37:29.876533+0000","last_became_peered":"2020-09-08T15:37:29.876533+0000","last_unstale":"2020-09-08T15:40:41.643368+0000","last_undegraded":"2020-09-08T15:40:41.643368+0000","last_fullsized":"2020-09-08T15:40:41.643368+0000","mapping_epoch":126,"log_start":"141'4500","ondisk_log_start":"141'4500","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"190'14202","last_scrub_stamp":"2020-09-08T15:40:36.687465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:36.687465+0000","log_size":10020,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256114688,"num_objects":3908,"num_object_clones":0,"num_object_copies":7816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14520,"num_write_kb":589696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"192'14817","reported_seq":"14896","reported_epoch":"192","state":"active+clean","last_fresh":"2020-09-08T15:40:41.838338+0000","last_change":"2020-09-08T15:40:35.304847+0000","last_active":"2020-09-08T15:40:41.838338+0000","last_peered":"2020-09-08T15:40:41.838338+0000","last_clean":"2020-09-08T15:40:41.838338+0000","last_became_active":"2020-09-08T15:37:30.091928+0000","last_became_peered":"2020-09-08T15:37:30.091928+0000","last_unstale":"2020-09-08T15:40:41.838338+0000","last_undegraded":"2020-09-08T15:40:41.838338+0000","last_fullsized":"2020-09-08T15:40:41.838338+0000","mapping_epoch":126,"log_start":"142'4800","ondisk_log_start":"142'4800","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"190'14394","last_scrub_stamp":"2020-09-08T15:40:35.304761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:35.304761+0000","log_size":10017,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":254345216,"num_objects":3881,"num_object_clones":0,"num_object_copies":7762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14817,"num_write_kb":598336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"192'14676","reported_seq":"14752","reported_epoch":"192","state":"active+clean","last_fresh":"2020-09-08T15:40:41.630643+0000","last_change":"2020-09-08T15:39:12.946359+0000","last_active":"2020-09-08T15:40:41.630643+0000","last_peered":"2020-09-08T15:40:41.630643+0000","last_clean":"2020-09-08T15:40:41.630643+0000","last_became_active":"2020-09-08T15:37:29.875715+0000","last_became_peered":"2020-09-08T15:37:29.875715+0000","last_unstale":"2020-09-08T15:40:41.630643+0000","last_undegraded":"2020-09-08T15:40:41.630643+0000","last_fullsized":"2020-09-08T15:40:41.630643+0000","mapping_epoch":126,"log_start":"141'4600","ondisk_log_start":"141'4600","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"165'9189","last_scrub_stamp":"2020-09-08T15:39:12.946278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:39:12.946278+0000","log_size":10076,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":242614272,"num_objects":3702,"num_object_clones":0,"num_object_copies":7404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14676,"num_write_kb":588096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"99'6","reported_seq":"206","reported_epoch":"192","state":"active+clean","last_fresh":"2020-09-08T15:40:40.634918+0000","last_change":"2020-09-08T15:39:42.676434+0000","last_active":"2020-09-08T15:40:40.634918+0000","last_peered":"2020-09-08T15:40:40.634918+0000","last_clean":"2020-09-08T15:40:40.634918+0000","last_became_active":"2020-09-08T15:34:43.031697+0000","last_became_peered":"2020-09-08T15:34:43.031697+0000","last_unstale":"2020-09-08T15:40:40.634918+0000","last_undegraded":"2020-09-08T15:40:40.634918+0000","last_fullsized":"2020-09-08T15:40:40.634918+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":2,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'6","last_scrub_stamp":"2020-09-08T15:39:42.676360+0000","last_deep_scrub":"99'6","last_deep_scrub_stamp":"2020-09-08T15:35:23.132465+0000","last_clean_scrub_stamp":"2020-09-08T15:39:42.676360+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"193'14738","reported_seq":"14818","reported_epoch":"193","state":"active+clean","last_fresh":"2020-09-08T15:40:43.610157+0000","last_change":"2020-09-08T15:40:14.389420+0000","last_active":"2020-09-08T15:40:43.610157+0000","last_peered":"2020-09-08T15:40:43.610157+0000","last_clean":"2020-09-08T15:40:43.610157+0000","last_became_active":"2020-09-08T15:37:30.094918+0000","last_became_peered":"2020-09-08T15:37:30.094918+0000","last_unstale":"2020-09-08T15:40:43.610157+0000","last_undegraded":"2020-09-08T15:40:43.610157+0000","last_fullsized":"2020-09-08T15:40:43.610157+0000","mapping_epoch":126,"log_start":"142'4700","ondisk_log_start":"142'4700","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'12937","last_scrub_stamp":"2020-09-08T15:40:14.389373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:14.389373+0000","log_size":10038,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":246808576,"num_objects":3766,"num_object_clones":0,"num_object_copies":7532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14738,"num_write_kb":592128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"192'14433","reported_seq":"14509","reported_epoch":"192","state":"active+clean","last_fresh":"2020-09-08T15:40:41.787172+0000","last_change":"2020-09-08T15:39:31.634079+0000","last_active":"2020-09-08T15:40:41.787172+0000","last_peered":"2020-09-08T15:40:41.787172+0000","last_clean":"2020-09-08T15:40:41.787172+0000","last_became_active":"2020-09-08T15:37:30.090699+0000","last_became_peered":"2020-09-08T15:37:30.090699+0000","last_unstale":"2020-09-08T15:40:41.787172+0000","last_undegraded":"2020-09-08T15:40:41.787172+0000","last_fullsized":"2020-09-08T15:40:41.787172+0000","mapping_epoch":126,"log_start":"140'4400","ondisk_log_start":"140'4400","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"170'10244","last_scrub_stamp":"2020-09-08T15:39:31.634020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:39:31.634020+0000","log_size":10033,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":251592704,"num_objects":3839,"num_object_clones":0,"num_object_copies":7678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14433,"num_write_kb":584704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.6","version":"192'14584","reported_seq":"14665","reported_epoch":"192","state":"active+clean","last_fresh":"2020-09-08T15:40:41.723076+0000","last_change":"2020-09-08T15:40:26.436714+0000","last_active":"2020-09-08T15:40:41.723076+0000","last_peered":"2020-09-08T15:40:41.723076+0000","last_clean":"2020-09-08T15:40:41.723076+0000","last_became_active":"2020-09-08T15:37:29.889471+0000","last_became_peered":"2020-09-08T15:37:29.889471+0000","last_unstale":"2020-09-08T15:40:41.723076+0000","last_undegraded":"2020-09-08T15:40:41.723076+0000","last_fullsized":"2020-09-08T15:40:41.723076+0000","mapping_epoch":126,"log_start":"141'4500","ondisk_log_start":"141'4500","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'13713","last_scrub_stamp":"2020-09-08T15:40:26.436603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:26.436603+0000","log_size":10084,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":252313600,"num_objects":3850,"num_object_clones":0,"num_object_copies":7700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14584,"num_write_kb":589888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"192'14637","reported_seq":"14716","reported_epoch":"192","state":"active+clean","last_fresh":"2020-09-08T15:40:41.676167+0000","last_change":"2020-09-08T15:40:23.781832+0000","last_active":"2020-09-08T15:40:41.676167+0000","last_peered":"2020-09-08T15:40:41.676167+0000","last_clean":"2020-09-08T15:40:41.676167+0000","last_became_active":"2020-09-08T15:37:30.095291+0000","last_became_peered":"2020-09-08T15:37:30.095291+0000","last_unstale":"2020-09-08T15:40:41.676167+0000","last_undegraded":"2020-09-08T15:40:41.676167+0000","last_fullsized":"2020-09-08T15:40:41.676167+0000","mapping_epoch":126,"log_start":"141'4600","ondisk_log_start":"141'4600","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'13605","last_scrub_stamp":"2020-09-08T15:40:23.781740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:23.781740+0000","log_size":10037,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":255000576,"num_objects":3891,"num_object_clones":0,"num_object_copies":7782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14637,"num_write_kb":592896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.8","version":"193'14597","reported_seq":"14674","reported_epoch":"193","state":"active+clean","last_fresh":"2020-09-08T15:40:43.613781+0000","last_change":"2020-09-08T15:39:15.907759+0000","last_active":"2020-09-08T15:40:43.613781+0000","last_peered":"2020-09-08T15:40:43.613781+0000","last_clean":"2020-09-08T15:40:43.613781+0000","last_became_active":"2020-09-08T15:37:29.881507+0000","last_became_peered":"2020-09-08T15:37:29.881507+0000","last_unstale":"2020-09-08T15:40:43.613781+0000","last_undegraded":"2020-09-08T15:40:43.613781+0000","last_fullsized":"2020-09-08T15:40:43.613781+0000","mapping_epoch":126,"log_start":"141'4500","ondisk_log_start":"141'4500","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"165'9281","last_scrub_stamp":"2020-09-08T15:39:15.907669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:39:15.907669+0000","log_size":10097,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":243597312,"num_objects":3717,"num_object_clones":0,"num_object_copies":7434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14597,"num_write_kb":586048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9],"acting":[7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.9","version":"192'14460","reported_seq":"14539","reported_epoch":"192","state":"active+clean","last_fresh":"2020-09-08T15:40:41.752653+0000","last_change":"2020-09-08T15:40:31.090354+0000","last_active":"2020-09-08T15:40:41.752653+0000","last_peered":"2020-09-08T15:40:41.752653+0000","last_clean":"2020-09-08T15:40:41.752653+0000","last_became_active":"2020-09-08T15:37:29.879357+0000","last_became_peered":"2020-09-08T15:37:29.879357+0000","last_unstale":"2020-09-08T15:40:41.752653+0000","last_undegraded":"2020-09-08T15:40:41.752653+0000","last_fullsized":"2020-09-08T15:40:41.752653+0000","mapping_epoch":126,"log_start":"141'4400","ondisk_log_start":"141'4400","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'13830","last_scrub_stamp":"2020-09-08T15:40:31.090293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:31.090293+0000","log_size":10060,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":253493248,"num_objects":3868,"num_object_clones":0,"num_object_copies":7736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14460,"num_write_kb":586496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"193'14721","reported_seq":"14801","reported_epoch":"193","state":"active+clean","last_fresh":"2020-09-08T15:40:42.701002+0000","last_change":"2020-09-08T15:40:30.575755+0000","last_active":"2020-09-08T15:40:42.701002+0000","last_peered":"2020-09-08T15:40:42.701002+0000","last_clean":"2020-09-08T15:40:42.701002+0000","last_became_active":"2020-09-08T15:37:30.094030+0000","last_became_peered":"2020-09-08T15:37:30.094030+0000","last_unstale":"2020-09-08T15:40:42.701002+0000","last_undegraded":"2020-09-08T15:40:42.701002+0000","last_fullsized":"2020-09-08T15:40:42.701002+0000","mapping_epoch":126,"log_start":"142'4700","ondisk_log_start":"142'4700","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'13996","last_scrub_stamp":"2020-09-08T15:40:30.575701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:30.575701+0000","log_size":10021,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":247922688,"num_objects":3783,"num_object_clones":0,"num_object_copies":7566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14721,"num_write_kb":592128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3],"acting":[11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.b","version":"192'14557","reported_seq":"14633","reported_epoch":"192","state":"active+clean","last_fresh":"2020-09-08T15:40:41.772688+0000","last_change":"2020-09-08T15:39:19.264726+0000","last_active":"2020-09-08T15:40:41.772688+0000","last_peered":"2020-09-08T15:40:41.772688+0000","last_clean":"2020-09-08T15:40:41.772688+0000","last_became_active":"2020-09-08T15:37:29.879505+0000","last_became_peered":"2020-09-08T15:37:29.879505+0000","last_unstale":"2020-09-08T15:40:41.772688+0000","last_undegraded":"2020-09-08T15:40:41.772688+0000","last_fullsized":"2020-09-08T15:40:41.772688+0000","mapping_epoch":126,"log_start":"141'4500","ondisk_log_start":"141'4500","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'9555","last_scrub_stamp":"2020-09-08T15:39:19.264670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:39:19.264670+0000","log_size":10057,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256573440,"num_objects":3915,"num_object_clones":0,"num_object_copies":7830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14557,"num_write_kb":591104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.c","version":"193'14576","reported_seq":"14656","reported_epoch":"193","state":"active+clean","last_fresh":"2020-09-08T15:40:42.743724+0000","last_change":"2020-09-08T15:40:15.718317+0000","last_active":"2020-09-08T15:40:42.743724+0000","last_peered":"2020-09-08T15:40:42.743724+0000","last_clean":"2020-09-08T15:40:42.743724+0000","last_became_active":"2020-09-08T15:37:30.093315+0000","last_became_peered":"2020-09-08T15:37:30.093315+0000","last_unstale":"2020-09-08T15:40:42.743724+0000","last_undegraded":"2020-09-08T15:40:42.743724+0000","last_fullsized":"2020-09-08T15:40:42.743724+0000","mapping_epoch":126,"log_start":"141'4500","ondisk_log_start":"141'4500","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'12953","last_scrub_stamp":"2020-09-08T15:40:15.718177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:15.718177+0000","log_size":10076,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":244449280,"num_objects":3730,"num_object_clones":0,"num_object_copies":7460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14576,"num_write_kb":585792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0],"acting":[11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.d","version":"193'14684","reported_seq":"14762","reported_epoch":"193","state":"active+clean","last_fresh":"2020-09-08T15:40:42.834722+0000","last_change":"2020-09-08T15:39:13.390802+0000","last_active":"2020-09-08T15:40:42.834722+0000","last_peered":"2020-09-08T15:40:42.834722+0000","last_clean":"2020-09-08T15:40:42.834722+0000","last_became_active":"2020-09-08T15:37:30.092910+0000","last_became_peered":"2020-09-08T15:37:30.092910+0000","last_unstale":"2020-09-08T15:40:42.834722+0000","last_undegraded":"2020-09-08T15:40:42.834722+0000","last_fullsized":"2020-09-08T15:40:42.834722+0000","mapping_epoch":126,"log_start":"142'4600","ondisk_log_start":"142'4600","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"165'9318","last_scrub_stamp":"2020-09-08T15:39:13.390657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:39:13.390657+0000","log_size":10084,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":259063832,"num_objects":3954,"num_object_clones":0,"num_object_copies":7908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3954,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14684,"num_write_kb":596353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.e","version":"192'14852","reported_seq":"14929","reported_epoch":"192","state":"active+clean","last_fresh":"2020-09-08T15:40:41.817717+0000","last_change":"2020-09-08T15:39:18.552789+0000","last_active":"2020-09-08T15:40:41.817717+0000","last_peered":"2020-09-08T15:40:41.817717+0000","last_clean":"2020-09-08T15:40:41.817717+0000","last_became_active":"2020-09-08T15:37:30.093099+0000","last_became_peered":"2020-09-08T15:37:30.093099+0000","last_unstale":"2020-09-08T15:40:41.817717+0000","last_undegraded":"2020-09-08T15:40:41.817717+0000","last_fullsized":"2020-09-08T15:40:41.817717+0000","mapping_epoch":126,"log_start":"142'4800","ondisk_log_start":"142'4800","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'9707","last_scrub_stamp":"2020-09-08T15:39:18.552716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:39:18.552716+0000","log_size":10052,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256770048,"num_objects":3918,"num_object_clones":0,"num_object_copies":7836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14852,"num_write_kb":600640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4021813272,"num_objects":61369,"num_object_clones":0,"num_object_copies":122738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":233867,"num_write_kb":9447489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8046845952,"data_stored":8046837808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160867,"ondisk_log_size":160867,"up":32,"acting":32,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":2,"acting":2,"num_store_stats":7}],"osd_stats":[{"osd":11,"up_from":0,"seq":76,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":779332,"kb_used_data":732472,"kb_used_omap":0,"kb_used_meta":46720,"kb_avail":92965820,"statfs":{"total":95995035648,"available":95196999680,"internally_reserved":0,"allocated":750051328,"data_stored":749813544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":47841280},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:39:53 2020","interfaces":[{"interface":"back","average":{"1min":2.3860000000000001,"5min":2.0529999999999999,"15min":1.917},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":6.7679999999999998,"5min":6.7679999999999998,"15min":6.7679999999999998},"last":1.087},{"interface":"front","average":{"1min":2.8690000000000002,"5min":2.9990000000000001,"15min":3.2069999999999999},"min":{"1min":1.081,"5min":1.022,"15min":1.022},"max":{"1min":7.2889999999999997,"5min":7.2889999999999997,"15min":7.2889999999999997},"last":2.2919999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:39:53 2020","interfaces":[{"interface":"back","average":{"1min":3.9420000000000002,"5min":2.5419999999999998,"15min":2.2200000000000002},"min":{"1min":0.80700000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":34.723999999999997,"5min":34.723999999999997,"15min":34.723999999999997},"last":1.589},{"interface":"front","average":{"1min":5.3680000000000003,"5min":3.02,"15min":2.7679999999999998},"min":{"1min":0.92700000000000005,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":38.625999999999998,"5min":38.625999999999998,"15min":38.625999999999998},"last":2.4540000000000002}]},{"osd":2,"last update":"Tue Sep 8 15:39:53 2020","interfaces":[{"interface":"back","average":{"1min":5.4500000000000002,"5min":2.859,"15min":2.3410000000000002},"min":{"1min":0.83899999999999997,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":36.637999999999998,"5min":36.637999999999998,"15min":36.637999999999998},"last":1.125},{"interface":"front","average":{"1min":4.798,"5min":3.3620000000000001,"15min":3.032},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":36.694000000000003,"5min":36.694000000000003,"15min":36.694000000000003},"last":1.1850000000000001}]},{"osd":3,"last update":"Tue Sep 8 15:39:53 2020","interfaces":[{"interface":"back","average":{"1min":2.1899999999999999,"5min":4.0599999999999996,"15min":2.9060000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":6.3380000000000001,"5min":147.49100000000001,"15min":147.49100000000001},"last":2.23},{"interface":"front","average":{"1min":3.6379999999999999,"5min":4.5650000000000004,"15min":3.613},"min":{"1min":1.252,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":10.131,"5min":150.32400000000001,"15min":150.32400000000001},"last":2.5840000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:39:53 2020","interfaces":[{"interface":"back","average":{"1min":2.8220000000000001,"5min":3.1379999999999999,"15min":3.452},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":8.0289999999999999,"5min":8.0289999999999999,"15min":8.0289999999999999},"last":2.7090000000000001},{"interface":"front","average":{"1min":2.9209999999999998,"5min":2.7650000000000001,"15min":2.7919999999999998},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":7.7679999999999998,"5min":7.7679999999999998,"15min":7.7679999999999998},"last":1.9910000000000001}]},{"osd":5,"last update":"Tue Sep 8 15:39:53 2020","interfaces":[{"interface":"back","average":{"1min":2.3719999999999999,"5min":3.0680000000000001,"15min":3.3130000000000002},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":6.8499999999999996,"5min":6.8499999999999996,"15min":6.8499999999999996},"last":2.4990000000000001},{"interface":"front","average":{"1min":3.0510000000000002,"5min":3.4049999999999998,"15min":3.8010000000000002},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":8.5289999999999999,"5min":8.5289999999999999,"15min":8.5289999999999999},"last":3.069}]},{"osd":6,"last update":"Tue Sep 8 15:39:53 2020","interfaces":[{"interface":"back","average":{"1min":2.3460000000000001,"5min":3.0720000000000001,"15min":3.3140000000000001},"min":{"1min":1.196,"5min":1.036,"15min":1.036},"max":{"1min":6.6210000000000004,"5min":7.8479999999999999,"15min":7.8479999999999999},"last":2.1040000000000001},{"interface":"front","average":{"1min":3.1669999999999998,"5min":3.4329999999999998,"15min":3.5579999999999998},"min":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"max":{"1min":8.2110000000000003,"5min":8.2110000000000003,"15min":8.2110000000000003},"last":2.8300000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:39:53 2020","interfaces":[{"interface":"back","average":{"1min":3.4889999999999999,"5min":3.496,"15min":3.6720000000000002},"min":{"1min":1.22,"5min":1.22,"15min":1.22},"max":{"1min":8.4610000000000003,"5min":8.4610000000000003,"15min":8.4610000000000003},"last":2.331},{"interface":"front","average":{"1min":2.532,"5min":3.5369999999999999,"15min":3.7730000000000001},"min":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"max":{"1min":6.492,"5min":7.9420000000000002,"15min":7.9420000000000002},"last":2.3650000000000002}]},{"osd":8,"last update":"Tue Sep 8 15:39:53 2020","interfaces":[{"interface":"back","average":{"1min":3.5019999999999998,"5min":3.7349999999999999,"15min":4.0030000000000001},"min":{"1min":1.744,"5min":1.335,"15min":1.335},"max":{"1min":8.5370000000000008,"5min":8.5370000000000008,"15min":8.5370000000000008},"last":3.1880000000000002},{"interface":"front","average":{"1min":4.133,"5min":3.7320000000000002,"15min":3.8479999999999999},"min":{"1min":1.575,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":8.9580000000000002,"5min":8.9580000000000002,"15min":8.9580000000000002},"last":3.302}]},{"osd":10,"last update":"Tue Sep 8 15:39:53 2020","interfaces":[{"interface":"back","average":{"1min":3.3180000000000001,"5min":3.8399999999999999,"15min":4.2779999999999996},"min":{"1min":1.498,"5min":1.498,"15min":1.498},"max":{"1min":8.3469999999999995,"5min":8.3469999999999995,"15min":8.3469999999999995},"last":2.9489999999999998},{"interface":"front","average":{"1min":3.7509999999999999,"5min":3.8260000000000001,"15min":4.1360000000000001},"min":{"1min":1.5880000000000001,"5min":1.107,"15min":1.107},"max":{"1min":8.6600000000000001,"5min":8.6600000000000001,"15min":8.6600000000000001},"last":2.1880000000000002}]}]},{"osd":10,"up_from":0,"seq":83,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":531076,"kb_used_data":497720,"kb_used_omap":0,"kb_used_meta":33216,"kb_avail":93214076,"statfs":{"total":95995035648,"available":95451213824,"internally_reserved":0,"allocated":509665280,"data_stored":509427496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":34013184},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7330000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":5.9669999999999996,"5min":2.8799999999999999,"15min":2.5659999999999998},"min":{"1min":0.66900000000000004,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":64.471999999999994,"5min":64.471999999999994,"15min":64.471999999999994},"last":0.81699999999999995},{"interface":"front","average":{"1min":6.6349999999999998,"5min":3.3250000000000002,"15min":2.8130000000000002},"min":{"1min":0.70099999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":64.707999999999998,"5min":64.707999999999998,"15min":64.707999999999998},"last":0.88900000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":4.0949999999999998,"5min":2.593,"15min":2.363},"min":{"1min":0.80800000000000005,"5min":0.745,"15min":0.745},"max":{"1min":40.453000000000003,"5min":40.453000000000003,"15min":40.453000000000003},"last":1.2210000000000001},{"interface":"front","average":{"1min":3.7730000000000001,"5min":2.4500000000000002,"15min":2.6970000000000001},"min":{"1min":0.88900000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":37.209000000000003,"5min":37.209000000000003,"15min":37.209000000000003},"last":1.3029999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":3.7250000000000001,"5min":4.5549999999999997,"15min":3.04},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":37.298999999999999,"5min":232.87299999999999,"15min":232.87299999999999},"last":1.728},{"interface":"front","average":{"1min":4.1539999999999999,"5min":5.1710000000000003,"15min":3.7829999999999999},"min":{"1min":1.0760000000000001,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":36.137,"5min":237.75800000000001,"15min":237.75800000000001},"last":1.454}]},{"osd":4,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":3.5009999999999999,"5min":2.859,"15min":3.2650000000000001},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":38.237000000000002,"5min":38.237000000000002,"15min":38.237000000000002},"last":1.5940000000000001},{"interface":"front","average":{"1min":3.6179999999999999,"5min":2.855,"15min":3.0089999999999999},"min":{"1min":1.0009999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":37.786000000000001,"5min":37.786000000000001,"15min":37.786000000000001},"last":2.1499999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":3.8879999999999999,"5min":3.0750000000000002,"15min":3.448},"min":{"1min":0.996,"5min":0.996,"15min":0.996},"max":{"1min":36.493000000000002,"5min":36.493000000000002,"15min":36.493000000000002},"last":1.7769999999999999},{"interface":"front","average":{"1min":3.859,"5min":3.25,"15min":3.657},"min":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"max":{"1min":38.957999999999998,"5min":38.957999999999998,"15min":38.957999999999998},"last":1.956}]},{"osd":6,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":3.8410000000000002,"5min":3.1150000000000002,"15min":3.4289999999999998},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":38.537999999999997,"5min":38.537999999999997,"15min":38.537999999999997},"last":2.2919999999999998},{"interface":"front","average":{"1min":4.0890000000000004,"5min":3.3140000000000001,"15min":3.71},"min":{"1min":1.266,"5min":1.1950000000000001,"15min":1.1950000000000001},"max":{"1min":37.698999999999998,"5min":37.698999999999998,"15min":37.698999999999998},"last":2.036}]},{"osd":7,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":4.319,"5min":3.4369999999999998,"15min":3.778},"min":{"1min":1.2410000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"max":{"1min":39.100000000000001,"5min":39.100000000000001,"15min":39.100000000000001},"last":2.427},{"interface":"front","average":{"1min":4.532,"5min":3.5470000000000002,"15min":3.9289999999999998},"min":{"1min":1.3440000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"max":{"1min":38.131999999999998,"5min":38.131999999999998,"15min":38.131999999999998},"last":1.9159999999999999}]},{"osd":8,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":4.4509999999999996,"5min":3.7679999999999998,"15min":4.0419999999999998},"min":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"max":{"1min":42.031999999999996,"5min":42.031999999999996,"15min":42.031999999999996},"last":2.3849999999999998},{"interface":"front","average":{"1min":4.4669999999999996,"5min":3.7850000000000001,"15min":3.9430000000000001},"min":{"1min":1.1850000000000001,"5min":1.1639999999999999,"15min":1.1639999999999999},"max":{"1min":39.512999999999998,"5min":39.512999999999998,"15min":39.512999999999998},"last":2.831}]},{"osd":9,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":4.7549999999999999,"5min":3.9790000000000001,"15min":4.2720000000000002},"min":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":42.603999999999999,"5min":42.603999999999999,"15min":42.603999999999999},"last":2.246},{"interface":"front","average":{"1min":4.5919999999999996,"5min":3.8650000000000002,"15min":4.0899999999999999},"min":{"1min":1.5189999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"max":{"1min":40.029000000000003,"5min":40.029000000000003,"15min":40.029000000000003},"last":6.2050000000000001}]},{"osd":11,"last update":"Tue Sep 8 15:39:58 2020","interfaces":[{"interface":"back","average":{"1min":5.6070000000000002,"5min":4.391,"15min":4.3890000000000002},"min":{"1min":1.6619999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"max":{"1min":39.271000000000001,"5min":39.271000000000001,"15min":39.271000000000001},"last":3.972},{"interface":"front","average":{"1min":4.6269999999999998,"5min":4.1500000000000004,"15min":4.3319999999999999},"min":{"1min":1.4650000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"max":{"1min":42.329999999999998,"5min":42.329999999999998,"15min":42.329999999999998},"last":2.1219999999999999}]}]},{"osd":9,"up_from":0,"seq":86,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":790468,"kb_used_data":743804,"kb_used_omap":0,"kb_used_meta":46656,"kb_avail":92954684,"statfs":{"total":95995035648,"available":95185596416,"internally_reserved":0,"allocated":761655296,"data_stored":761413440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":47775744},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:40:05 2020","interfaces":[{"interface":"back","average":{"1min":1.8240000000000001,"5min":2.0470000000000002,"15min":1.921},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":5.3959999999999999,"5min":9.0809999999999995,"15min":9.0809999999999995},"last":3.1890000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.8779999999999999,"15min":1.8939999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.597,"5min":5.1440000000000001,"15min":5.1440000000000001},"last":1.5800000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:40:05 2020","interfaces":[{"interface":"back","average":{"1min":2.0510000000000002,"5min":2.1499999999999999,"15min":2.0150000000000001},"min":{"1min":0.97499999999999998,"5min":0.755,"15min":0.755},"max":{"1min":3.8769999999999998,"5min":9.5519999999999996,"15min":9.5519999999999996},"last":0.85099999999999998},{"interface":"front","average":{"1min":2.3799999999999999,"5min":2.4260000000000002,"15min":2.3319999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":6.6230000000000002,"5min":9.4250000000000007,"15min":9.4250000000000007},"last":0.96799999999999997}]},{"osd":2,"last update":"Tue Sep 8 15:40:05 2020","interfaces":[{"interface":"back","average":{"1min":2.0089999999999999,"5min":2.3570000000000002,"15min":2.4340000000000002},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":6.2169999999999996,"5min":6.2169999999999996,"15min":6.2169999999999996},"last":2.5089999999999999},{"interface":"front","average":{"1min":1.9259999999999999,"5min":2.4409999999999998,"15min":2.3330000000000002},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":5.7060000000000004,"5min":9.4770000000000003,"15min":9.4770000000000003},"last":1.0189999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:40:05 2020","interfaces":[{"interface":"back","average":{"1min":2.1629999999999998,"5min":2.6789999999999998,"15min":2.4529999999999998},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":4.7699999999999996,"5min":24.562000000000001,"15min":24.562000000000001},"last":1.7190000000000001},{"interface":"front","average":{"1min":2.3359999999999999,"5min":2.8780000000000001,"15min":2.722},"min":{"1min":1.1799999999999999,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":5.1749999999999998,"5min":23.184000000000001,"15min":23.184000000000001},"last":1.8280000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:40:05 2020","interfaces":[{"interface":"back","average":{"1min":2.177,"5min":2.7370000000000001,"15min":2.702},"min":{"1min":1.087,"5min":1.087,"15min":1.087},"max":{"1min":6.7400000000000002,"5min":6.7400000000000002,"15min":6.7400000000000002},"last":2.637},{"interface":"front","average":{"1min":2.2250000000000001,"5min":2.9769999999999999,"15min":2.9889999999999999},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":3.7850000000000001,"5min":10.201000000000001,"15min":10.201000000000001},"last":2.1190000000000002}]},{"osd":5,"last update":"Tue Sep 8 15:40:05 2020","interfaces":[{"interface":"back","average":{"1min":2.2919999999999998,"5min":2.9449999999999998,"15min":3.085},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":5.8310000000000004,"5min":11.097,"15min":11.097},"last":0.89700000000000002},{"interface":"front","average":{"1min":2.6560000000000001,"5min":3.097,"15min":3.1429999999999998},"min":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0589999999999999},"max":{"1min":5.8780000000000001,"5min":9.9280000000000008,"15min":9.9280000000000008},"last":2.4420000000000002}]},{"osd":6,"last update":"Tue Sep 8 15:40:05 2020","interfaces":[{"interface":"back","average":{"1min":2.2909999999999999,"5min":3.1819999999999999,"15min":3.194},"min":{"1min":1.2809999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":5.9509999999999996,"5min":10.798,"15min":10.798},"last":1.1339999999999999},{"interface":"front","average":{"1min":2.5259999999999998,"5min":3.2109999999999999,"15min":3.2240000000000002},"min":{"1min":1.242,"5min":1.242,"15min":1.242},"max":{"1min":6.8730000000000002,"5min":11.766,"15min":11.766},"last":2.9420000000000002}]},{"osd":7,"last update":"Tue Sep 8 15:40:05 2020","interfaces":[{"interface":"back","average":{"1min":2.6070000000000002,"5min":3.4449999999999998,"15min":3.4089999999999998},"min":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"max":{"1min":4.1890000000000001,"5min":10.31,"15min":10.31},"last":2.2309999999999999},{"interface":"front","average":{"1min":2.8340000000000001,"5min":3.6520000000000001,"15min":3.6240000000000001},"min":{"1min":1.593,"5min":1.593,"15min":1.593},"max":{"1min":4.3280000000000003,"5min":10.837999999999999,"15min":10.837999999999999},"last":2.569}]},{"osd":8,"last update":"Tue Sep 8 15:40:05 2020","interfaces":[{"interface":"back","average":{"1min":3.2650000000000001,"5min":3.5470000000000002,"15min":3.2909999999999999},"min":{"1min":1.351,"5min":1.351,"15min":1.3180000000000001},"max":{"1min":11,"5min":11,"15min":11},"last":2.8210000000000002},{"interface":"front","average":{"1min":3.3380000000000001,"5min":3.754,"15min":3.407},"min":{"1min":1.373,"5min":1.373,"15min":1.373},"max":{"1min":8.9580000000000002,"5min":10.977,"15min":10.977},"last":2.3370000000000002}]},{"osd":10,"last update":"Tue Sep 8 15:40:24 2020","interfaces":[{"interface":"back","average":{"1min":2.8879999999999999,"5min":3.3780000000000001,"15min":3.7469999999999999},"min":{"1min":1.5329999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"max":{"1min":8.8200000000000003,"5min":8.8200000000000003,"15min":8.8200000000000003},"last":3.246},{"interface":"front","average":{"1min":2.0790000000000002,"5min":3.3620000000000001,"15min":3.6459999999999999},"min":{"1min":1.196,"5min":1.196,"15min":1.196},"max":{"1min":3.282,"5min":11.625,"15min":11.625},"last":1.6850000000000001}]}]},{"osd":8,"up_from":0,"seq":91,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1064836,"kb_used_data":1000504,"kb_used_omap":0,"kb_used_meta":64128,"kb_avail":92680316,"statfs":{"total":95995035648,"available":94904643584,"internally_reserved":0,"allocated":1024516096,"data_stored":1024278312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65667072},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":2.0009999999999999,"5min":2.141,"15min":2.1709999999999998},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":8.6069999999999993,"5min":8.6069999999999993,"15min":8.6069999999999993},"last":4.0519999999999996},{"interface":"front","average":{"1min":3.5049999999999999,"5min":2.472,"15min":2.4460000000000002},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":13.773999999999999,"5min":13.773999999999999,"15min":13.773999999999999},"last":1.6919999999999999}]},{"osd":1,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.492,"5min":2.0659999999999998,"15min":1.8839999999999999},"min":{"1min":1.4690000000000001,"5min":1.137,"15min":1.137},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":1.4350000000000001},{"interface":"front","average":{"1min":2.7200000000000002,"5min":2.427,"15min":2.347},"min":{"1min":1.351,"5min":1.046,"15min":1.046},"max":{"1min":4.444,"5min":4.444,"15min":4.444},"last":1.8600000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":10.316000000000001,"5min":4.3819999999999997,"15min":2.677},"min":{"1min":0.79500000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":145.893,"5min":145.893,"15min":145.893},"last":9.5050000000000008},{"interface":"front","average":{"1min":9.9939999999999998,"5min":3.9420000000000002,"15min":2.9580000000000002},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":153.91900000000001,"5min":153.91900000000001,"15min":153.91900000000001},"last":4.2750000000000004}]},{"osd":3,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":2.5739999999999998,"5min":4.0869999999999997,"15min":3.1779999999999999},"min":{"1min":0.73799999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":9.2989999999999995,"5min":125.111,"15min":125.111},"last":4.194},{"interface":"front","average":{"1min":3.1190000000000002,"5min":3.952,"15min":2.9060000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":13.505000000000001,"5min":126.124,"15min":126.124},"last":1.823}]},{"osd":4,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":2.5329999999999999,"5min":2.9830000000000001,"15min":2.907},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":9.3800000000000008,"5min":9.7599999999999998,"15min":9.7599999999999998},"last":4.3470000000000004},{"interface":"front","average":{"1min":3.343,"5min":2.7320000000000002,"15min":2.6110000000000002},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":13.999000000000001,"5min":13.999000000000001,"15min":13.999000000000001},"last":1.9450000000000001}]},{"osd":5,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":2.6720000000000002,"5min":3.2690000000000001,"15min":3.1890000000000001},"min":{"1min":0.90500000000000003,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":9.4399999999999995,"5min":9.4399999999999995,"15min":9.4399999999999995},"last":9.6349999999999998},{"interface":"front","average":{"1min":2.6600000000000001,"5min":3.323,"15min":3.3380000000000001},"min":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":9.6820000000000004,"5min":9.6820000000000004,"15min":9.6820000000000004},"last":4.492}]},{"osd":6,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":3.343,"5min":3.552,"15min":3.3450000000000002},"min":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"max":{"1min":13.887,"5min":13.887,"15min":13.887},"last":2.194},{"interface":"front","average":{"1min":2.9550000000000001,"5min":2.9470000000000001,"15min":2.923},"min":{"1min":1.153,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":12.502000000000001,"5min":12.502000000000001,"15min":12.502000000000001},"last":4.4210000000000003}]},{"osd":7,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":3.3119999999999998,"5min":3.7360000000000002,"15min":3.6469999999999998},"min":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"max":{"1min":9.4469999999999992,"5min":10.236000000000001,"15min":10.236000000000001},"last":4.5609999999999999},{"interface":"front","average":{"1min":3.948,"5min":3.4289999999999998,"15min":3.2959999999999998},"min":{"1min":1.3620000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"max":{"1min":14.109999999999999,"5min":14.109999999999999,"15min":14.109999999999999},"last":2.3170000000000002}]},{"osd":9,"last update":"Tue Sep 8 15:40:04 2020","interfaces":[{"interface":"back","average":{"1min":4.0369999999999999,"5min":4.0270000000000001,"15min":3.5979999999999999},"min":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.157},"max":{"1min":13.506,"5min":13.506,"15min":13.506},"last":9.3580000000000005},{"interface":"front","average":{"1min":3.2450000000000001,"5min":3.8849999999999998,"15min":3.4889999999999999},"min":{"1min":1.294,"5min":1.294,"15min":1.1919999999999999},"max":{"1min":9.5259999999999998,"5min":10.122999999999999,"15min":10.122999999999999},"last":6.3550000000000004}]},{"osd":10,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":3.6379999999999999,"5min":3.3490000000000002,"15min":3.6070000000000002},"min":{"1min":1.423,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":13.656000000000001,"5min":13.656000000000001,"15min":13.656000000000001},"last":2.0739999999999998},{"interface":"front","average":{"1min":2.9729999999999999,"5min":3.8140000000000001,"15min":3.988},"min":{"1min":1.296,"5min":1.296,"15min":1.296},"max":{"1min":9.6649999999999991,"5min":11.051,"15min":11.051},"last":13.638}]},{"osd":11,"last update":"Tue Sep 8 15:40:35 2020","interfaces":[{"interface":"back","average":{"1min":3.512,"5min":3.665,"15min":3.9239999999999999},"min":{"1min":1.214,"5min":1.214,"15min":1.214},"max":{"1min":12.646000000000001,"5min":12.646000000000001,"15min":12.646000000000001},"last":2.4359999999999999},{"interface":"front","average":{"1min":3.4830000000000001,"5min":4.4210000000000003,"15min":4.7939999999999996},"min":{"1min":1.248,"5min":1.248,"15min":1.248},"max":{"1min":10.667999999999999,"5min":11.186999999999999,"15min":11.186999999999999},"last":9.7599999999999998}]}]},{"osd":7,"up_from":0,"seq":97,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":776332,"kb_used_data":721028,"kb_used_omap":0,"kb_used_meta":55296,"kb_avail":92968820,"statfs":{"total":95995035648,"available":95200071680,"internally_reserved":0,"allocated":738332672,"data_stored":738090500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":56623104},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:02 2020","interfaces":[{"interface":"back","average":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"min":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"max":{"1min":4.0190000000000001,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":2.2189999999999999},{"interface":"front","average":{"1min":2.6259999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"min":{"1min":1.542,"5min":1.542,"15min":1.542},"max":{"1min":4.5229999999999997,"5min":4.5229999999999997,"15min":4.5229999999999997},"last":2.3079999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":2.133,"15min":2.2759999999999998},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":4.8600000000000003,"5min":6.6219999999999999,"15min":6.6219999999999999},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.802,"5min":2.1030000000000002,"15min":2.2869999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":5.3159999999999998,"5min":5.3159999999999998,"15min":5.3159999999999998},"last":1.423}]},{"osd":2,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":2.3879999999999999,"5min":2.5350000000000001,"15min":2.4860000000000002},"min":{"1min":1.0649999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":6.234,"5min":6.9779999999999998,"15min":6.9779999999999998},"last":1.8360000000000001},{"interface":"front","average":{"1min":2.569,"5min":2.4870000000000001,"15min":2.7509999999999999},"min":{"1min":1.335,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":6.6879999999999997,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":3.3260000000000001}]},{"osd":3,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":2.3109999999999999,"5min":2.899,"15min":2.9129999999999998},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":4.593,"5min":37.801000000000002,"15min":37.801000000000002},"last":3.2000000000000002},{"interface":"front","average":{"1min":2.661,"5min":3.6030000000000002,"15min":3.387},"min":{"1min":1.2430000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":12.099,"5min":45.284999999999997,"15min":45.284999999999997},"last":3.0990000000000002}]},{"osd":4,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":1.794,"5min":2.5089999999999999,"15min":2.7989999999999999},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":3.1930000000000001,"5min":8.5779999999999994,"15min":8.5779999999999994},"last":1.9990000000000001},{"interface":"front","average":{"1min":2.2320000000000002,"5min":3.004,"15min":3.2229999999999999},"min":{"1min":1.246,"5min":1.1319999999999999,"15min":1.1319999999999999},"max":{"1min":4.4009999999999998,"5min":7.3570000000000002,"15min":7.3570000000000002},"last":2.9279999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":2.3149999999999999,"5min":2.956,"15min":3.464},"min":{"1min":1.228,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":4.5540000000000003,"5min":9.0800000000000001,"15min":9.0800000000000001},"last":2.6539999999999999},{"interface":"front","average":{"1min":2.3159999999999998,"5min":2.9449999999999998,"15min":3.339},"min":{"1min":1.208,"5min":1.1779999999999999,"15min":1.1779999999999999},"max":{"1min":4.2869999999999999,"5min":6.0229999999999997,"15min":6.0229999999999997},"last":3.4580000000000002}]},{"osd":6,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":2.5219999999999998,"5min":3.1749999999999998,"15min":3.7080000000000002},"min":{"1min":1.304,"5min":1.304,"15min":1.304},"max":{"1min":5.1210000000000004,"5min":8.8300000000000001,"15min":8.8300000000000001},"last":2.9750000000000001},{"interface":"front","average":{"1min":2.6259999999999999,"5min":3.359,"15min":3.7450000000000001},"min":{"1min":1.7,"5min":1.3939999999999999,"15min":1.3939999999999999},"max":{"1min":4.2910000000000004,"5min":7.681,"15min":7.681},"last":3.5510000000000002}]},{"osd":8,"last update":"Tue Sep 8 15:39:42 2020","interfaces":[{"interface":"back","average":{"1min":2.9180000000000001,"5min":3.8170000000000002,"15min":4.0259999999999998},"min":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"max":{"1min":6.4880000000000004,"5min":6.9530000000000003,"15min":6.9530000000000003},"last":3.645},{"interface":"front","average":{"1min":3.4510000000000001,"5min":4.0129999999999999,"15min":4.1520000000000001},"min":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"max":{"1min":9.2119999999999997,"5min":9.2119999999999997,"15min":9.2119999999999997},"last":2.3540000000000001}]},{"osd":9,"last update":"Tue Sep 8 15:40:01 2020","interfaces":[{"interface":"back","average":{"1min":3.2519999999999998,"5min":4.0720000000000001,"15min":4.54},"min":{"1min":1.9039999999999999,"5min":1.7170000000000001,"15min":1.7170000000000001},"max":{"1min":6.8140000000000001,"5min":7.476,"15min":7.476},"last":3.6989999999999998},{"interface":"front","average":{"1min":2.5619999999999998,"5min":3.5350000000000001,"15min":4.149},"min":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"max":{"1min":4.008,"5min":6.274,"15min":6.274},"last":7.7460000000000004}]},{"osd":10,"last update":"Tue Sep 8 15:40:31 2020","interfaces":[{"interface":"back","average":{"1min":3.2040000000000002,"5min":3.7370000000000001,"15min":4.4119999999999999},"min":{"1min":2.0299999999999998,"5min":1.5309999999999999,"15min":1.5309999999999999},"max":{"1min":7.3739999999999997,"5min":9.5009999999999994,"15min":9.5009999999999994},"last":3.25},{"interface":"front","average":{"1min":2.7810000000000001,"5min":3.8860000000000001,"15min":4.75},"min":{"1min":1.583,"5min":1.583,"15min":1.583},"max":{"1min":4.8079999999999998,"5min":7.5720000000000001,"15min":8.3650000000000002},"last":2.1859999999999999}]},{"osd":11,"last update":"Tue Sep 8 15:39:50 2020","interfaces":[{"interface":"back","average":{"1min":3.0819999999999999,"5min":4.2460000000000004,"15min":4.6399999999999997},"min":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"max":{"1min":5.2279999999999998,"5min":9.1790000000000003,"15min":9.1790000000000003},"last":2.52},{"interface":"front","average":{"1min":3.7890000000000001,"5min":4.5659999999999998,"15min":5.0659999999999998},"min":{"1min":1.756,"5min":1.756,"15min":1.756},"max":{"1min":6.5069999999999997,"5min":7.7629999999999999,"15min":7.7629999999999999},"last":3.403}]}]},{"osd":6,"up_from":0,"seq":101,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":279748,"kb_used_data":251576,"kb_used_omap":0,"kb_used_meta":28032,"kb_avail":93465404,"statfs":{"total":95995035648,"available":95708573696,"internally_reserved":0,"allocated":257613824,"data_stored":257376060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28704768},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:39:54 2020","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.9039999999999999,"15min":1.877},"min":{"1min":0.94199999999999995,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.8809999999999998,"5min":3.8809999999999998,"15min":3.8809999999999998},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.984,"15min":1.966},"min":{"1min":1.0429999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.847,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":1.3180000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:39:54 2020","interfaces":[{"interface":"back","average":{"1min":2.3500000000000001,"5min":2.226,"15min":2.1899999999999999},"min":{"1min":0.98199999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":4.4409999999999998,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":1.103},{"interface":"front","average":{"1min":2.2789999999999999,"5min":2.625,"15min":2.4820000000000002},"min":{"1min":0.98699999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":7.6849999999999996,"5min":7.6849999999999996,"15min":7.6849999999999996},"last":1.073}]},{"osd":2,"last update":"Tue Sep 8 15:39:54 2020","interfaces":[{"interface":"back","average":{"1min":2.2280000000000002,"5min":2.7210000000000001,"15min":2.4990000000000001},"min":{"1min":1.333,"5min":1.3240000000000001,"15min":1.3240000000000001},"max":{"1min":5.5039999999999996,"5min":5.5039999999999996,"15min":5.5039999999999996},"last":8.577},{"interface":"front","average":{"1min":2.589,"5min":2.7999999999999998,"15min":2.5},"min":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":5.9299999999999997,"5min":5.9299999999999997,"15min":5.9299999999999997},"last":8.5589999999999993}]},{"osd":3,"last update":"Tue Sep 8 15:39:54 2020","interfaces":[{"interface":"back","average":{"1min":2.5430000000000001,"5min":3.863,"15min":2.8860000000000001},"min":{"1min":1.2549999999999999,"5min":1.224,"15min":1.224},"max":{"1min":4.2240000000000002,"5min":109.65300000000001,"15min":109.65300000000001},"last":2.605},{"interface":"front","average":{"1min":2.3260000000000001,"5min":4.1269999999999998,"15min":3.2589999999999999},"min":{"1min":1.278,"5min":1.278,"15min":1.278},"max":{"1min":3.9830000000000001,"5min":109.51600000000001,"15min":109.51600000000001},"last":1.357}]},{"osd":4,"last update":"Tue Sep 8 15:39:54 2020","interfaces":[{"interface":"back","average":{"1min":2.129,"5min":2.9729999999999999,"15min":2.7370000000000001},"min":{"1min":1.4379999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":4.3079999999999998,"5min":4.9980000000000002,"15min":4.9980000000000002},"last":1.5640000000000001},{"interface":"front","average":{"1min":2.3170000000000002,"5min":3.2120000000000002,"15min":2.9790000000000001},"min":{"1min":1.4630000000000001,"5min":1.052,"15min":1.052},"max":{"1min":3.8679999999999999,"5min":22.062000000000001,"15min":22.062000000000001},"last":2.7759999999999998}]},{"osd":5,"last update":"Tue Sep 8 15:39:54 2020","interfaces":[{"interface":"back","average":{"1min":2.5640000000000001,"5min":3.3490000000000002,"15min":3.181},"min":{"1min":1.613,"5min":1.1879999999999999,"15min":1.1879999999999999},"max":{"1min":4.5979999999999999,"5min":5.8460000000000001,"15min":6.1269999999999998},"last":2.3610000000000002},{"interface":"front","average":{"1min":2.9340000000000002,"5min":3.7250000000000001,"15min":3.3159999999999998},"min":{"1min":2.0249999999999999,"5min":1.149,"15min":1.149},"max":{"1min":7.1120000000000001,"5min":21.228999999999999,"15min":21.228999999999999},"last":2.9740000000000002}]},{"osd":7,"last update":"Tue Sep 8 15:40:14 2020","interfaces":[{"interface":"back","average":{"1min":2.7200000000000002,"5min":3.5979999999999999,"15min":3.6699999999999999},"min":{"1min":1.196,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":4.665,"5min":22.911999999999999,"15min":22.911999999999999},"last":4.7359999999999998},{"interface":"front","average":{"1min":2.8769999999999998,"5min":3.9950000000000001,"15min":3.855},"min":{"1min":1.887,"5min":1.5609999999999999,"15min":1.5609999999999999},"max":{"1min":5.032,"5min":26.056000000000001,"15min":26.056000000000001},"last":4.7930000000000001}]},{"osd":8,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":2.71,"5min":3.4449999999999998,"15min":3.7709999999999999},"min":{"1min":1.609,"5min":1.454,"15min":1.454},"max":{"1min":3.9140000000000001,"5min":6.3479999999999999,"15min":6.3479999999999999},"last":2.8809999999999998},{"interface":"front","average":{"1min":3.0219999999999998,"5min":3.472,"15min":3.952},"min":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"max":{"1min":8.6850000000000005,"5min":8.6850000000000005,"15min":8.6850000000000005},"last":1.4690000000000001}]},{"osd":9,"last update":"Tue Sep 8 15:40:03 2020","interfaces":[{"interface":"back","average":{"1min":3.3079999999999998,"5min":4.2060000000000004,"15min":4.3499999999999996},"min":{"1min":2.165,"5min":1.5569999999999999,"15min":1.5569999999999999},"max":{"1min":7.1020000000000003,"5min":21.716000000000001,"15min":21.716000000000001},"last":3.2360000000000002},{"interface":"front","average":{"1min":2.9710000000000001,"5min":3.8530000000000002,"15min":4.298},"min":{"1min":1.736,"5min":1.736,"15min":1.736},"max":{"1min":7.0250000000000004,"5min":7.0250000000000004,"15min":7.0250000000000004},"last":1.8839999999999999}]},{"osd":10,"last update":"Tue Sep 8 15:40:27 2020","interfaces":[{"interface":"back","average":{"1min":2.5030000000000001,"5min":3.855,"15min":4.415},"min":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"max":{"1min":3.3860000000000001,"5min":21.331,"15min":21.331},"last":1.7310000000000001},{"interface":"front","average":{"1min":3.2879999999999998,"5min":4.0179999999999998,"15min":4.4249999999999998},"min":{"1min":2.2290000000000001,"5min":1.915,"15min":1.915},"max":{"1min":6.0979999999999999,"5min":21.515999999999998,"15min":21.515999999999998},"last":3.1419999999999999}]}]},{"osd":1,"up_from":0,"seq":125,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1072516,"kb_used_data":999416,"kb_used_omap":0,"kb_used_meta":72896,"kb_avail":92672636,"statfs":{"total":95995035648,"available":94896779264,"internally_reserved":0,"allocated":1023401984,"data_stored":1023164220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":74645504},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:39:50 2020","interfaces":[{"interface":"back","average":{"1min":7.2910000000000004,"5min":3.2210000000000001,"15min":1.9930000000000001},"min":{"1min":0.80800000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":45.085000000000001,"5min":45.085000000000001,"15min":45.085000000000001},"last":2.149},{"interface":"front","average":{"1min":7.3099999999999996,"5min":3.5030000000000001,"15min":2.1949999999999998},"min":{"1min":0.88500000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":45.765999999999998,"5min":45.765999999999998,"15min":45.765999999999998},"last":2.7650000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":5.992,"5min":3.5139999999999998,"15min":2.3149999999999999},"min":{"1min":0.84699999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":44.039000000000001,"5min":44.039000000000001,"15min":44.039000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":6.3090000000000002,"5min":3.79,"15min":2.5059999999999998},"min":{"1min":0.71799999999999997,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":44.841000000000001,"5min":44.841000000000001,"15min":44.841000000000001},"last":3.7069999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:40:40 2020","interfaces":[{"interface":"back","average":{"1min":3.4700000000000002,"5min":3.5870000000000002,"15min":2.7149999999999999},"min":{"1min":0.98999999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":12.49,"5min":44.768999999999998,"15min":44.768999999999998},"last":2.0369999999999999},{"interface":"front","average":{"1min":4.1289999999999996,"5min":4.0570000000000004,"15min":2.964},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":21.399999999999999,"5min":45.892000000000003,"15min":45.892000000000003},"last":2.8940000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:39:56 2020","interfaces":[{"interface":"back","average":{"1min":6.1619999999999999,"5min":3.2069999999999999,"15min":2.5030000000000001},"min":{"1min":1.0329999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":44.970999999999997,"5min":44.970999999999997,"15min":44.970999999999997},"last":0.749},{"interface":"front","average":{"1min":4.8799999999999999,"5min":3.5110000000000001,"15min":2.9910000000000001},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":51.289000000000001,"5min":51.289000000000001,"15min":51.289000000000001},"last":3.4260000000000002}]},{"osd":5,"last update":"Tue Sep 8 15:40:25 2020","interfaces":[{"interface":"back","average":{"1min":4.8380000000000001,"5min":3.819,"15min":3.246},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":45.289000000000001,"5min":45.289000000000001,"15min":45.289000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":5.8029999999999999,"5min":3.3580000000000001,"15min":2.75},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":51.499000000000002,"5min":51.499000000000002,"15min":51.499000000000002},"last":3.109}]},{"osd":6,"last update":"Tue Sep 8 15:39:50 2020","interfaces":[{"interface":"back","average":{"1min":6.3940000000000001,"5min":4.0910000000000002,"15min":3.7639999999999998},"min":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"max":{"1min":44.932000000000002,"5min":44.932000000000002,"15min":44.932000000000002},"last":1.1040000000000001},{"interface":"front","average":{"1min":6.9009999999999998,"5min":4.3360000000000003,"15min":3.806},"min":{"1min":1.6830000000000001,"5min":1.2949999999999999,"15min":1.2949999999999999},"max":{"1min":45.159999999999997,"5min":45.159999999999997,"15min":45.159999999999997},"last":3.282}]},{"osd":7,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":5.5090000000000003,"5min":4.5990000000000002,"15min":3.6619999999999999},"min":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"max":{"1min":45.409999999999997,"5min":45.409999999999997,"15min":45.409999999999997},"last":3.5659999999999998},{"interface":"front","average":{"1min":4.9790000000000001,"5min":4.3410000000000002,"15min":4.0350000000000001},"min":{"1min":1.238,"5min":1.238,"15min":1.238},"max":{"1min":45.375,"5min":45.375,"15min":45.375},"last":1.238}]},{"osd":8,"last update":"Tue Sep 8 15:40:40 2020","interfaces":[{"interface":"back","average":{"1min":3.6720000000000002,"5min":4.7400000000000002,"15min":4.4390000000000001},"min":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"max":{"1min":7.9850000000000003,"5min":45.640999999999998,"15min":45.640999999999998},"last":3.8460000000000001},{"interface":"front","average":{"1min":3.5409999999999999,"5min":4.7389999999999999,"15min":4.2889999999999997},"min":{"1min":1.153,"5min":1.153,"15min":1.153},"max":{"1min":11.839,"5min":45.235999999999997,"15min":45.235999999999997},"last":1.2150000000000001}]},{"osd":9,"last update":"Tue Sep 8 15:40:10 2020","interfaces":[{"interface":"back","average":{"1min":5.6619999999999999,"5min":3.6520000000000001,"15min":3.3769999999999998},"min":{"1min":1.103,"5min":1.0860000000000001,"15min":1.0860000000000001},"max":{"1min":51.698,"5min":51.698,"15min":51.698},"last":1.329},{"interface":"front","average":{"1min":5.375,"5min":4.6189999999999998,"15min":4.359},"min":{"1min":1.45,"5min":1.45,"15min":1.45},"max":{"1min":45.515000000000001,"5min":45.515000000000001,"15min":45.515000000000001},"last":1.548}]},{"osd":10,"last update":"Tue Sep 8 15:40:25 2020","interfaces":[{"interface":"back","average":{"1min":5.6440000000000001,"5min":4.6870000000000003,"15min":4.3689999999999998},"min":{"1min":1.587,"5min":1.4159999999999999,"15min":1.4159999999999999},"max":{"1min":46.039999999999999,"5min":46.039999999999999,"15min":46.039999999999999},"last":3.992},{"interface":"front","average":{"1min":5.6280000000000001,"5min":3.7879999999999998,"15min":3.593},"min":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":51.892000000000003,"5min":51.892000000000003,"15min":51.892000000000003},"last":1.411}]}]},{"osd":0,"up_from":0,"seq":130,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":818424,"kb_used_data":759024,"kb_used_omap":0,"kb_used_meta":59136,"kb_avail":92926728,"statfs":{"total":95995035648,"available":95156969472,"internally_reserved":0,"allocated":777240576,"data_stored":776958531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":60555264},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 8 15:39:49 2020","interfaces":[{"interface":"back","average":{"1min":2.9119999999999999,"5min":2.327,"15min":1.6719999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":10.579000000000001,"5min":10.579000000000001,"15min":10.579000000000001},"last":1.294},{"interface":"front","average":{"1min":3.5139999999999998,"5min":2.3599999999999999,"15min":1.7290000000000001},"min":{"1min":0.78100000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":13.589,"5min":13.589,"15min":13.589},"last":1.3939999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:34:05 2020","interfaces":[{"interface":"back","average":{"1min":1.9950000000000001,"5min":1.621,"15min":1.5189999999999999},"min":{"1min":1.3999999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.8519999999999999,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":2.6589999999999998},{"interface":"front","average":{"1min":2.629,"5min":1.956,"15min":1.72},"min":{"1min":1.6419999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":2.4049999999999998}]},{"osd":3,"last update":"Tue Sep 8 15:39:39 2020","interfaces":[{"interface":"back","average":{"1min":2.125,"5min":2.3479999999999999,"15min":2.7989999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":7.4420000000000002,"5min":7.4420000000000002,"15min":27.298999999999999},"last":1.8440000000000001},{"interface":"front","average":{"1min":2.4710000000000001,"5min":3.093,"15min":3.4550000000000001},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":7.7990000000000004,"5min":7.7990000000000004,"15min":28.314},"last":1.9319999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:40:05 2020","interfaces":[{"interface":"back","average":{"1min":2.04,"5min":2.6070000000000002,"15min":2.5329999999999999},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":6.8849999999999998,"5min":6.8849999999999998,"15min":6.8849999999999998},"last":2.0739999999999998},{"interface":"front","average":{"1min":6.1180000000000003,"5min":3.6669999999999998,"15min":3.0590000000000002},"min":{"1min":0.995,"5min":0.995,"15min":0.995},"max":{"1min":73.802999999999997,"5min":73.802999999999997,"15min":73.802999999999997},"last":4.8339999999999996}]},{"osd":5,"last update":"Tue Sep 8 15:40:23 2020","interfaces":[{"interface":"back","average":{"1min":2.774,"5min":3.157,"15min":3.0489999999999999},"min":{"1min":0.83999999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":9.5730000000000004,"5min":9.5730000000000004,"15min":9.5730000000000004},"last":2.4420000000000002},{"interface":"front","average":{"1min":2.0329999999999999,"5min":2.343,"15min":2.9359999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":3.923,"5min":12.890000000000001,"15min":12.890000000000001},"last":2.1339999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:39:49 2020","interfaces":[{"interface":"back","average":{"1min":2.77,"5min":2.5510000000000002,"15min":3.2759999999999998},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":13.151999999999999,"5min":13.151999999999999,"15min":13.151999999999999},"last":2.3300000000000001},{"interface":"front","average":{"1min":3.2000000000000002,"5min":2.8889999999999998,"15min":3.6659999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":13.48,"5min":13.48,"15min":13.48},"last":2.472}]},{"osd":7,"last update":"Tue Sep 8 15:40:20 2020","interfaces":[{"interface":"back","average":{"1min":7.6299999999999999,"5min":4.0199999999999996,"15min":4.2519999999999998},"min":{"1min":1.605,"5min":1.302,"15min":1.302},"max":{"1min":71.167000000000002,"5min":71.167000000000002,"15min":71.167000000000002},"last":2.746},{"interface":"front","average":{"1min":8.0850000000000009,"5min":4.4649999999999999,"15min":4.234},"min":{"1min":1.488,"5min":1.3240000000000001,"15min":1.3240000000000001},"max":{"1min":74.480000000000004,"5min":74.480000000000004,"15min":74.480000000000004},"last":2.5950000000000002}]},{"osd":8,"last update":"Tue Sep 8 15:39:39 2020","interfaces":[{"interface":"back","average":{"1min":3.044,"5min":3.8780000000000001,"15min":3.8719999999999999},"min":{"1min":1.093,"5min":1.093,"15min":1.093},"max":{"1min":7.173,"5min":7.173,"15min":7.173},"last":2.1709999999999998},{"interface":"front","average":{"1min":10.484999999999999,"5min":4.7400000000000002,"15min":4.4119999999999999},"min":{"1min":1.319,"5min":1.319,"15min":1.319},"max":{"1min":131.958,"5min":131.958,"15min":131.958},"last":4.2480000000000002}]},{"osd":9,"last update":"Tue Sep 8 15:40:04 2020","interfaces":[{"interface":"back","average":{"1min":3.5070000000000001,"5min":3.9550000000000001,"15min":4.1749999999999998},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":13.259,"5min":13.259,"15min":13.259},"last":2.8839999999999999},{"interface":"front","average":{"1min":3.2400000000000002,"5min":3.4039999999999999,"15min":3.8900000000000001},"min":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":7.3399999999999999,"5min":7.3399999999999999,"15min":7.3399999999999999},"last":2.718}]},{"osd":10,"last update":"Tue Sep 8 15:40:23 2020","interfaces":[{"interface":"back","average":{"1min":2.5630000000000002,"5min":2.9220000000000002,"15min":3.0539999999999998},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":4.7489999999999997,"5min":6.923,"15min":6.923},"last":1.7050000000000001},{"interface":"front","average":{"1min":7.468,"5min":4.7030000000000003,"15min":4.4909999999999997},"min":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"max":{"1min":73.444000000000003,"5min":73.444000000000003,"15min":73.444000000000003},"last":2.2949999999999999}]},{"osd":11,"last update":"Tue Sep 8 15:39:49 2020","interfaces":[{"interface":"back","average":{"1min":3.9609999999999999,"5min":3.7570000000000001,"15min":3.8450000000000002},"min":{"1min":1.476,"5min":1.476,"15min":1.476},"max":{"1min":12.486000000000001,"5min":12.486000000000001,"15min":12.486000000000001},"last":3.75},{"interface":"front","average":{"1min":3.4369999999999998,"5min":4.181,"15min":4.6340000000000003},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":13.369999999999999,"5min":13.369999999999999,"15min":13.369999999999999},"last":2.621}]}]},{"osd":2,"up_from":0,"seq":122,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":786820,"kb_used_data":739000,"kb_used_omap":0,"kb_used_meta":47680,"kb_avail":92958332,"statfs":{"total":95995035648,"available":95189331968,"internally_reserved":0,"allocated":756736000,"data_stored":756498236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":48824320},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:40:10 2020","interfaces":[{"interface":"back","average":{"1min":8.4199999999999999,"5min":3.3599999999999999,"15min":2.2029999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":162.83799999999999,"5min":162.83799999999999,"15min":162.83799999999999},"last":1.4730000000000001},{"interface":"front","average":{"1min":8.4369999999999994,"5min":3.4209999999999998,"15min":2.2869999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":162.63499999999999,"5min":162.63499999999999,"15min":162.63499999999999},"last":6.0300000000000002}]},{"osd":1,"last update":"Tue Sep 8 15:40:10 2020","interfaces":[{"interface":"back","average":{"1min":5.2939999999999996,"5min":2.9350000000000001,"15min":2.1099999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":63.887,"5min":63.887,"15min":63.887},"last":2.1960000000000002},{"interface":"front","average":{"1min":2.5259999999999998,"5min":2.597,"15min":2.2669999999999999},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":10.519,"5min":10.519,"15min":10.519},"last":1.204}]},{"osd":3,"last update":"Tue Sep 8 15:40:36 2020","interfaces":[{"interface":"back","average":{"1min":3.0019999999999998,"5min":2.8570000000000002,"15min":2.5449999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":11.315,"5min":11.315,"15min":11.315},"last":9.8149999999999995},{"interface":"front","average":{"1min":2.923,"5min":2.645,"15min":2.4609999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":11.737,"5min":11.737,"15min":11.737},"last":6.367}]},{"osd":4,"last update":"Tue Sep 8 15:40:04 2020","interfaces":[{"interface":"back","average":{"1min":2.548,"5min":2.7309999999999999,"15min":2.5350000000000001},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":7.1100000000000003,"5min":7.2530000000000001,"15min":7.2530000000000001},"last":2.1720000000000002},{"interface":"front","average":{"1min":5.1070000000000002,"5min":3.4369999999999998,"15min":2.7330000000000001},"min":{"1min":1.04,"5min":1.04,"15min":1.04},"max":{"1min":64.284999999999997,"5min":64.284999999999997,"15min":64.284999999999997},"last":2.4649999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:40:25 2020","interfaces":[{"interface":"back","average":{"1min":5.7460000000000004,"5min":3.875,"15min":3.2469999999999999},"min":{"1min":1.222,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":63.509,"5min":63.509,"15min":63.509},"last":2.7090000000000001},{"interface":"front","average":{"1min":3.0230000000000001,"5min":3.2469999999999999,"15min":3.0579999999999998},"min":{"1min":1.2689999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"max":{"1min":11.055999999999999,"5min":11.055999999999999,"15min":11.055999999999999},"last":1.7549999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:39:48 2020","interfaces":[{"interface":"back","average":{"1min":3.5960000000000001,"5min":3.5219999999999998,"15min":3.258},"min":{"1min":1.4179999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"max":{"1min":7.9630000000000001,"5min":12.112,"15min":12.112},"last":2.581},{"interface":"front","average":{"1min":3.093,"5min":3.54,"15min":3.528},"min":{"1min":1.399,"5min":1.105,"15min":1.105},"max":{"1min":11.234999999999999,"5min":11.234999999999999,"15min":11.234999999999999},"last":1.8819999999999999}]},{"osd":7,"last update":"Tue Sep 8 15:40:21 2020","interfaces":[{"interface":"back","average":{"1min":3.3780000000000001,"5min":3.8180000000000001,"15min":3.9569999999999999},"min":{"1min":1.298,"5min":1.298,"15min":1.298},"max":{"1min":11.436,"5min":11.436,"15min":11.436},"last":2.1320000000000001},{"interface":"front","average":{"1min":3.0670000000000002,"5min":3.3940000000000001,"15min":3.5950000000000002},"min":{"1min":1.016,"5min":1.006,"15min":1.006},"max":{"1min":7.2270000000000003,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":5.9160000000000004}]},{"osd":8,"last update":"Tue Sep 8 15:40:30 2020","interfaces":[{"interface":"back","average":{"1min":6.0899999999999999,"5min":4.0759999999999996,"15min":3.802},"min":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":64.116,"5min":64.116,"15min":64.116},"last":3.0169999999999999},{"interface":"front","average":{"1min":5.9539999999999997,"5min":4.2370000000000001,"15min":4.0620000000000003},"min":{"1min":1.532,"5min":1.218,"15min":1.218},"max":{"1min":63.299999999999997,"5min":63.299999999999997,"15min":63.299999999999997},"last":2.0110000000000001}]},{"osd":9,"last update":"Tue Sep 8 15:40:10 2020","interfaces":[{"interface":"back","average":{"1min":6.4059999999999997,"5min":4.4589999999999996,"15min":4.1829999999999998},"min":{"1min":1.591,"5min":1.591,"15min":1.591},"max":{"1min":64.004000000000005,"5min":64.004000000000005,"15min":64.004000000000005},"last":6.2370000000000001},{"interface":"front","average":{"1min":3.5760000000000001,"5min":3.7879999999999998,"15min":4.1299999999999999},"min":{"1min":1.401,"5min":1.401,"15min":1.401},"max":{"1min":11.704000000000001,"5min":11.704000000000001,"15min":11.704000000000001},"last":4.2270000000000003}]},{"osd":10,"last update":"Tue Sep 8 15:40:25 2020","interfaces":[{"interface":"back","average":{"1min":6.4800000000000004,"5min":4.5209999999999999,"15min":4.1890000000000001},"min":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"max":{"1min":63.743000000000002,"5min":63.743000000000002,"15min":63.743000000000002},"last":3.1259999999999999},{"interface":"front","average":{"1min":3.0800000000000001,"5min":3.2559999999999998,"15min":3.6139999999999999},"min":{"1min":1.159,"5min":1.159,"15min":1.159},"max":{"1min":6.9690000000000003,"5min":7.9960000000000004,"15min":7.9960000000000004},"last":5.7800000000000002}]}]},{"osd":3,"up_from":0,"seq":115,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":806724,"kb_used_data":754808,"kb_used_omap":0,"kb_used_meta":51712,"kb_avail":92938428,"statfs":{"total":95995035648,"available":95168950272,"internally_reserved":0,"allocated":772923392,"data_stored":772685628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":52953088},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:31 2020","interfaces":[{"interface":"back","average":{"1min":2.1379999999999999,"5min":1.7210000000000001,"15min":1.629},"min":{"1min":1.79,"5min":1.1419999999999999,"15min":1.1419999999999999},"max":{"1min":2.7930000000000001,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":3.169},{"interface":"front","average":{"1min":2.6800000000000002,"5min":2.2309999999999999,"15min":2.0990000000000002},"min":{"1min":2.0270000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"max":{"1min":3.5409999999999999,"5min":3.5409999999999999,"15min":3.5409999999999999},"last":2.5579999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:39:41 2020","interfaces":[{"interface":"back","average":{"1min":4.5709999999999997,"5min":2.6440000000000001,"15min":2.117},"min":{"1min":0.81100000000000005,"5min":0.628,"15min":0.628},"max":{"1min":44.109999999999999,"5min":44.109999999999999,"15min":44.109999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":4.6980000000000004,"5min":3.629,"15min":2.4209999999999998},"min":{"1min":0.753,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":45.209000000000003,"5min":45.618000000000002,"15min":45.618000000000002},"last":0.97299999999999998}]},{"osd":2,"last update":"Tue Sep 8 15:39:41 2020","interfaces":[{"interface":"back","average":{"1min":2.8300000000000001,"5min":3.1880000000000002,"15min":2.4700000000000002},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":12.276999999999999,"5min":45.774000000000001,"15min":45.774000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":2.8260000000000001,"5min":2.444,"15min":2.3149999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":6.7359999999999998,"5min":13.948,"15min":13.948},"last":5.2830000000000004}]},{"osd":4,"last update":"Tue Sep 8 15:40:01 2020","interfaces":[{"interface":"back","average":{"1min":2.7839999999999998,"5min":2.5390000000000001,"15min":2.5459999999999998},"min":{"1min":1.085,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":5.9530000000000003,"5min":5.9530000000000003,"15min":5.9530000000000003},"last":1.2809999999999999},{"interface":"front","average":{"1min":2.1949999999999998,"5min":2.3580000000000001,"15min":2.452},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":11.298,"5min":11.298,"15min":11.298},"last":1.466}]},{"osd":5,"last update":"Tue Sep 8 15:40:27 2020","interfaces":[{"interface":"back","average":{"1min":3.2200000000000002,"5min":3.1469999999999998,"15min":2.964},"min":{"1min":1.0609999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"max":{"1min":27.007999999999999,"5min":45.442999999999998,"15min":45.442999999999998},"last":1.5},{"interface":"front","average":{"1min":4.556,"5min":3.3500000000000001,"15min":3.2789999999999999},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":23.565000000000001,"5min":23.565000000000001,"15min":23.565000000000001},"last":1.53}]},{"osd":6,"last update":"Tue Sep 8 15:39:52 2020","interfaces":[{"interface":"back","average":{"1min":2.46,"5min":2.976,"15min":3.2360000000000002},"min":{"1min":1.1759999999999999,"5min":1.127,"15min":1.127},"max":{"1min":6.2880000000000003,"5min":6.2880000000000003,"15min":6.2880000000000003},"last":2.46},{"interface":"front","average":{"1min":2.2349999999999999,"5min":3.4809999999999999,"15min":3.524},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":5.4359999999999999,"5min":47.457999999999998,"15min":47.457999999999998},"last":2.1749999999999998}]},{"osd":7,"last update":"Tue Sep 8 15:40:27 2020","interfaces":[{"interface":"back","average":{"1min":5.0999999999999996,"5min":4.0739999999999998,"15min":4.1959999999999997},"min":{"1min":1.3899999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"max":{"1min":26.035,"5min":26.035,"15min":26.035},"last":2.0270000000000001},{"interface":"front","average":{"1min":3.7749999999999999,"5min":3.2509999999999999,"15min":3.4990000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":25.521000000000001,"5min":25.521000000000001,"15min":25.521000000000001},"last":1.6899999999999999}]},{"osd":8,"last update":"Tue Sep 8 15:40:39 2020","interfaces":[{"interface":"back","average":{"1min":2.6099999999999999,"5min":3.5870000000000002,"15min":3.7069999999999999},"min":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":8.4529999999999994,"5min":46.917999999999999,"15min":46.917999999999999},"last":2.3799999999999999},{"interface":"front","average":{"1min":4.1689999999999996,"5min":4.6900000000000004,"15min":4.4249999999999998},"min":{"1min":1.272,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":11.534000000000001,"5min":44.893000000000001,"15min":44.893000000000001},"last":2.1579999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:40:01 2020","interfaces":[{"interface":"back","average":{"1min":2.681,"5min":3.181,"15min":3.7280000000000002},"min":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":10.872999999999999,"5min":10.872999999999999,"15min":10.872999999999999},"last":1.6140000000000001},{"interface":"front","average":{"1min":3.5,"5min":3.9870000000000001,"15min":4.2709999999999999},"min":{"1min":1.4790000000000001,"5min":1.4279999999999999,"15min":1.4279999999999999},"max":{"1min":12.752000000000001,"5min":46.665999999999997,"15min":46.665999999999997},"last":1.9790000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:40:27 2020","interfaces":[{"interface":"back","average":{"1min":3.9780000000000002,"5min":4.3280000000000003,"15min":4.3739999999999997},"min":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"max":{"1min":11.694000000000001,"5min":39.283000000000001,"15min":39.283000000000001},"last":1.8819999999999999},{"interface":"front","average":{"1min":2.9279999999999999,"5min":3.3010000000000002,"15min":3.6080000000000001},"min":{"1min":1.081,"5min":1.081,"15min":1.081},"max":{"1min":11.005000000000001,"5min":11.005000000000001,"15min":11.005000000000001},"last":1.8069999999999999}]},{"osd":11,"last update":"Tue Sep 8 15:39:52 2020","interfaces":[{"interface":"back","average":{"1min":3.7709999999999999,"5min":4.3129999999999997,"15min":4.6479999999999997},"min":{"1min":1.196,"5min":1.196,"15min":1.196},"max":{"1min":11.156000000000001,"5min":13.882999999999999,"15min":13.882999999999999},"last":1.9450000000000001},{"interface":"front","average":{"1min":5.8010000000000002,"5min":5.226,"15min":4.5060000000000002},"min":{"1min":1.577,"5min":1.577,"15min":1.577},"max":{"1min":44.722999999999999,"5min":47.234999999999999,"15min":47.234999999999999},"last":2.3010000000000002}]}]},{"osd":4,"up_from":0,"seq":112,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":278288,"kb_used_data":250056,"kb_used_omap":0,"kb_used_meta":28224,"kb_avail":93466864,"statfs":{"total":95995035648,"available":95710068736,"internally_reserved":0,"allocated":256057344,"data_stored":255811100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28901376},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:39:58 2020","interfaces":[{"interface":"back","average":{"1min":1.661,"5min":1.9490000000000001,"15min":1.871},"min":{"1min":0.80900000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":5.6210000000000004,"5min":5.7130000000000001,"15min":5.7130000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.5529999999999999,"5min":2.0609999999999999,"15min":1.8680000000000001},"min":{"1min":0.85099999999999998,"5min":0.71299999999999997,"15min":0.70799999999999996},"max":{"1min":4.1059999999999999,"5min":4.3650000000000002,"15min":4.3650000000000002},"last":1.0529999999999999}]},{"osd":1,"last update":"Tue Sep 8 15:39:58 2020","interfaces":[{"interface":"back","average":{"1min":2.577,"5min":3.5510000000000002,"15min":2.3929999999999998},"min":{"1min":1.179,"5min":0.996,"15min":0.996},"max":{"1min":15.336,"5min":74.947999999999993,"15min":74.947999999999993},"last":2.9660000000000002},{"interface":"front","average":{"1min":3.109,"5min":3.8969999999999998,"15min":2.665},"min":{"1min":1.087,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":16.388999999999999,"5min":71.108999999999995,"15min":71.108999999999995},"last":6.665}]},{"osd":2,"last update":"Tue Sep 8 15:39:58 2020","interfaces":[{"interface":"back","average":{"1min":2.3580000000000001,"5min":3.9079999999999999,"15min":2.6619999999999999},"min":{"1min":1.2270000000000001,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":3.714,"5min":74.504000000000005,"15min":74.504000000000005},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.9279999999999999,"5min":3.8079999999999998,"15min":2.7480000000000002},"min":{"1min":1.2010000000000001,"5min":1.155,"15min":1.155},"max":{"1min":2.8530000000000002,"5min":73.885000000000005,"15min":73.885000000000005},"last":1.5920000000000001}]},{"osd":3,"last update":"Tue Sep 8 15:39:58 2020","interfaces":[{"interface":"back","average":{"1min":2.46,"5min":2.8879999999999999,"15min":2.5750000000000002},"min":{"1min":1.091,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":6.1870000000000003,"5min":6.3280000000000003,"15min":6.3280000000000003},"last":1.1240000000000001},{"interface":"front","average":{"1min":2.681,"5min":2.839,"15min":2.4750000000000001},"min":{"1min":1.5489999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"max":{"1min":5.2809999999999997,"5min":7.4900000000000002,"15min":7.4900000000000002},"last":1.504}]},{"osd":5,"last update":"Tue Sep 8 15:40:28 2020","interfaces":[{"interface":"back","average":{"1min":2.351,"5min":2.9860000000000002,"15min":2.8220000000000001},"min":{"1min":1.2709999999999999,"5min":1.024,"15min":1.014},"max":{"1min":3.8759999999999999,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":5.2240000000000002},{"interface":"front","average":{"1min":2.161,"5min":3.0310000000000001,"15min":2.8759999999999999},"min":{"1min":1.274,"5min":1.2190000000000001,"15min":1.129},"max":{"1min":3.6949999999999998,"5min":7.5359999999999996,"15min":7.5359999999999996},"last":5.1459999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:39:45 2020","interfaces":[{"interface":"back","average":{"1min":2.859,"5min":3.5299999999999998,"15min":3.3399999999999999},"min":{"1min":1.4179999999999999,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":6.0830000000000002,"5min":6.8650000000000002,"15min":6.8650000000000002},"last":1.9319999999999999},{"interface":"front","average":{"1min":2.6200000000000001,"5min":3.4340000000000002,"15min":3.3559999999999999},"min":{"1min":1.365,"5min":1.1659999999999999,"15min":1.1659999999999999},"max":{"1min":5.931,"5min":6.827,"15min":6.827},"last":1.4450000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:40:08 2020","interfaces":[{"interface":"back","average":{"1min":3.1339999999999999,"5min":3.6110000000000002,"15min":3.782},"min":{"1min":1.8660000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"max":{"1min":5.3280000000000003,"5min":7.4699999999999998,"15min":7.4699999999999998},"last":2.4830000000000001},{"interface":"front","average":{"1min":3.137,"5min":3.6629999999999998,"15min":3.8799999999999999},"min":{"1min":1.546,"5min":1.546,"15min":1.546},"max":{"1min":6.4269999999999996,"5min":7.1459999999999999,"15min":7.1459999999999999},"last":4.8449999999999998}]},{"osd":8,"last update":"Tue Sep 8 15:40:39 2020","interfaces":[{"interface":"back","average":{"1min":2.9449999999999998,"5min":5.2080000000000002,"15min":4.3559999999999999},"min":{"1min":1.276,"5min":1.276,"15min":1.276},"max":{"1min":7.8140000000000001,"5min":109.608,"15min":109.608},"last":1.9630000000000001},{"interface":"front","average":{"1min":3.222,"5min":5.3799999999999999,"15min":4.7039999999999997},"min":{"1min":1.889,"5min":1.49,"15min":1.49},"max":{"1min":6.4039999999999999,"5min":116.52500000000001,"15min":116.52500000000001},"last":2.0600000000000001}]},{"osd":9,"last update":"Tue Sep 8 15:40:04 2020","interfaces":[{"interface":"back","average":{"1min":2.7890000000000001,"5min":5.5419999999999998,"15min":4.7629999999999999},"min":{"1min":1.546,"5min":1.546,"15min":1.546},"max":{"1min":6.4080000000000004,"5min":118.312,"15min":118.312},"last":1.6259999999999999},{"interface":"front","average":{"1min":2.782,"5min":5.3120000000000003,"15min":4.7229999999999999},"min":{"1min":1.7689999999999999,"5min":1.647,"15min":1.647},"max":{"1min":4.3470000000000004,"5min":116.871,"15min":116.871},"last":2.1829999999999998}]},{"osd":10,"last update":"Tue Sep 8 15:40:28 2020","interfaces":[{"interface":"back","average":{"1min":2.593,"5min":3.5710000000000002,"15min":4.4829999999999997},"min":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"max":{"1min":6.5170000000000003,"5min":7.9279999999999999,"15min":7.9279999999999999},"last":2.0990000000000002},{"interface":"front","average":{"1min":2.5950000000000002,"5min":3.8330000000000002,"15min":4.5629999999999997},"min":{"1min":1.702,"5min":1.702,"15min":1.702},"max":{"1min":4.5609999999999999,"5min":7.9870000000000001,"15min":7.9870000000000001},"last":3.1499999999999999}]}]},{"osd":5,"up_from":0,"seq":105,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":536964,"kb_used_data":491064,"kb_used_omap":0,"kb_used_meta":45760,"kb_avail":93208188,"statfs":{"total":95995035648,"available":95445184512,"internally_reserved":0,"allocated":502849536,"data_stored":502611772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46858240},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:40:29 2020","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.8720000000000001,"15min":1.881},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":5.4370000000000003,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.7350000000000001,"5min":2.3210000000000002,"15min":2.0790000000000002},"min":{"1min":0.90800000000000003,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":4.4859999999999998,"5min":7.1340000000000003,"15min":7.1340000000000003},"last":1.5649999999999999}]},{"osd":1,"last update":"Tue Sep 8 15:40:29 2020","interfaces":[{"interface":"back","average":{"1min":4.0940000000000003,"5min":2.8610000000000002,"15min":2.492},"min":{"1min":0.97799999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":35.908999999999999,"5min":35.908999999999999,"15min":35.908999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":4.79,"5min":3.3860000000000001,"15min":2.464},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":36.561999999999998,"5min":36.561999999999998,"15min":36.561999999999998},"last":4.125}]},{"osd":2,"last update":"Tue Sep 8 15:40:29 2020","interfaces":[{"interface":"back","average":{"1min":2.649,"5min":2.7970000000000002,"15min":2.3530000000000002},"min":{"1min":0.97499999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":6.8250000000000002,"5min":31.021000000000001,"15min":31.021000000000001},"last":1.905},{"interface":"front","average":{"1min":2.766,"5min":3.2770000000000001,"15min":2.903},"min":{"1min":1.1100000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"max":{"1min":5.5229999999999997,"5min":37.658999999999999,"15min":37.658999999999999},"last":3.9900000000000002}]},{"osd":3,"last update":"Tue Sep 8 15:40:29 2020","interfaces":[{"interface":"back","average":{"1min":2.6659999999999999,"5min":3.3599999999999999,"15min":3.0030000000000001},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":8.4220000000000006,"5min":8.4220000000000006,"15min":8.4220000000000006},"last":1.95},{"interface":"front","average":{"1min":2.3479999999999999,"5min":2.8149999999999999,"15min":2.8860000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":6.819,"5min":6.819,"15min":6.819},"last":1.4590000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:40:29 2020","interfaces":[{"interface":"back","average":{"1min":2.339,"5min":2.5070000000000001,"15min":2.3610000000000002},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":6.9180000000000001,"5min":6.9180000000000001,"15min":6.9180000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":2.1080000000000001,"5min":2.899,"15min":2.738},"min":{"1min":1.1990000000000001,"5min":1.1819999999999999,"15min":1.1819999999999999},"max":{"1min":3.3889999999999998,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":2.2559999999999998}]},{"osd":6,"last update":"Tue Sep 8 15:39:57 2020","interfaces":[{"interface":"back","average":{"1min":3.4729999999999999,"5min":3.782,"15min":3.6160000000000001},"min":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"max":{"1min":8.7710000000000008,"5min":8.7710000000000008,"15min":8.7710000000000008},"last":2.5419999999999998},{"interface":"front","average":{"1min":3.911,"5min":4.2850000000000001,"15min":3.3860000000000001},"min":{"1min":1.73,"5min":1.73,"15min":1.73},"max":{"1min":9.359,"5min":9.359,"15min":9.359},"last":2.6379999999999999}]},{"osd":7,"last update":"Tue Sep 8 15:40:13 2020","interfaces":[{"interface":"back","average":{"1min":2.996,"5min":3.5680000000000001,"15min":3.5609999999999999},"min":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"max":{"1min":6.9909999999999997,"5min":6.9909999999999997,"15min":6.9909999999999997},"last":2.1680000000000001},{"interface":"front","average":{"1min":4.0149999999999997,"5min":4.1539999999999999,"15min":4.0709999999999997},"min":{"1min":2.2789999999999999,"5min":1.532,"15min":1.532},"max":{"1min":9.2129999999999992,"5min":9.2129999999999992,"15min":9.2129999999999992},"last":2.746}]},{"osd":8,"last update":"Tue Sep 8 15:40:39 2020","interfaces":[{"interface":"back","average":{"1min":2.8279999999999998,"5min":3.2719999999999998,"15min":3.4990000000000001},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":5.8330000000000002,"5min":7.8140000000000001,"15min":7.8140000000000001},"last":1.6830000000000001},{"interface":"front","average":{"1min":2.641,"5min":3.2719999999999998,"15min":3.794},"min":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"max":{"1min":10.138999999999999,"5min":10.138999999999999,"15min":10.138999999999999},"last":2.1230000000000002}]},{"osd":9,"last update":"Tue Sep 8 15:40:06 2020","interfaces":[{"interface":"back","average":{"1min":3.569,"5min":3.931,"15min":4.3479999999999999},"min":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"max":{"1min":7.8959999999999999,"5min":7.8959999999999999,"15min":7.8959999999999999},"last":2.0680000000000001},{"interface":"front","average":{"1min":4.1109999999999998,"5min":4.2169999999999996,"15min":4.218},"min":{"1min":2.4319999999999999,"5min":1.992,"15min":1.992},"max":{"1min":8.9149999999999991,"5min":8.9149999999999991,"15min":8.9149999999999991},"last":2.2890000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:40:29 2020","interfaces":[{"interface":"back","average":{"1min":2.4300000000000002,"5min":3.3130000000000002,"15min":3.9289999999999998},"min":{"1min":1.431,"5min":1.431,"15min":1.431},"max":{"1min":4.4690000000000003,"5min":5.7309999999999999,"15min":5.7309999999999999},"last":2.3860000000000001},{"interface":"front","average":{"1min":3.762,"5min":4.4089999999999998,"15min":4.6609999999999996},"min":{"1min":2.2610000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"max":{"1min":9.0670000000000002,"5min":9.0670000000000002,"15min":9.0670000000000002},"last":2.4319999999999999}]}]}]}} 2020-09-08T15:40:48.127 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph pg dump --format=json 2020-09-08T15:40:48.212 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:40:48.278 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:48.283 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:40:48.380 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:48.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:40:48 smithi104 bash[15273]: debug 2020-09-08T15:40:48.277+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:48.517 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:48.616 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:40:48.627 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:48.784 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:40:48.865 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:40:48 smithi166 bash[16593]: debug 2020-09-08T15:40:48.513+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:49.398 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:49.502 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:49.653 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:49.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:40:49 smithi104 bash[15273]: debug 2020-09-08T15:40:49.397+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:49.758 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:49.865 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:40:49 smithi166 bash[11928]: debug 2020-09-08T15:40:49.653+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:49.909 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:40:50.012 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:50.163 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:40:49 smithi055 bash[16646]: debug 2020-09-08T15:40:49.906+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:50.229 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:50.333 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:40:50.500 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:40:50 smithi104 bash[22305]: debug 2020-09-08T15:40:50.229+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:50.501 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:40:50.604 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:50.627 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:40:50.743 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:50.754 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:50 smithi104 bash[9203]: cluster 2020-09-08T15:40:49.641688+0000 mgr.x (mgr.24118) 400 : cluster [DBG] pgmap v515: 17 pgs: 17 active+clean; 3.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:40:50.754 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:50 smithi104 bash[9203]: audit 2020-09-08T15:40:50.461771+0000 mon.a (mon.0) 1054 : audit [INF] from='client.? v1:172.21.15.104:0/1543508356' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:40:50.755 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:40:50 smithi104 bash[17588]: debug 2020-09-08T15:40:50.501+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:50.849 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:50.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:50 smithi166 bash[9446]: cluster 2020-09-08T15:40:49.641688+0000 mgr.x (mgr.24118) 400 : cluster [DBG] pgmap v515: 17 pgs: 17 active+clean; 3.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:40:50.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:50 smithi166 bash[9446]: audit 2020-09-08T15:40:50.461771+0000 mon.a (mon.0) 1054 : audit [INF] from='client.? v1:172.21.15.104:0/1543508356' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:40:50.867 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:40:50 smithi166 bash[16593]: debug 2020-09-08T15:40:50.741+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:50.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:50 smithi055 bash[9296]: cluster 2020-09-08T15:40:49.641688+0000 mgr.x (mgr.24118) 400 : cluster [DBG] pgmap v515: 17 pgs: 17 active+clean; 3.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:40:50.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:50 smithi055 bash[9296]: audit 2020-09-08T15:40:50.461771+0000 mon.a (mon.0) 1054 : audit [INF] from='client.? v1:172.21.15.104:0/1543508356' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:40:51.540 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:51.625 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:40:51 smithi104 bash[22305]: debug 2020-09-08T15:40:51.545+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:51.626 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:51 smithi104 bash[9203]: cluster 2020-09-08T15:40:50.615878+0000 mon.a (mon.0) 1055 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:40:51.626 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:51 smithi104 bash[9203]: audit 2020-09-08T15:40:50.627823+0000 mon.a (mon.0) 1056 : audit [INF] from='client.? v1:172.21.15.104:0/1543508356' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:40:51.626 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:51 smithi104 bash[9203]: cluster 2020-09-08T15:40:50.627865+0000 mon.a (mon.0) 1057 : cluster [DBG] osdmap e195: 12 total, 12 up, 12 in 2020-09-08T15:40:51.643 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:40:51.786 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:40:51.888 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:51.902 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:51 smithi166 bash[9446]: cluster 2020-09-08T15:40:50.615878+0000 mon.a (mon.0) 1055 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:40:51.903 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:51 smithi166 bash[9446]: audit 2020-09-08T15:40:50.627823+0000 mon.a (mon.0) 1056 : audit [INF] from='client.? v1:172.21.15.104:0/1543508356' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:40:51.903 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:51 smithi166 bash[9446]: cluster 2020-09-08T15:40:50.627865+0000 mon.a (mon.0) 1057 : cluster [DBG] osdmap e195: 12 total, 12 up, 12 in 2020-09-08T15:40:51.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:40:51 smithi055 bash[14301]: debug 2020-09-08T15:40:51.786+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:51.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:51 smithi055 bash[9296]: cluster 2020-09-08T15:40:50.615878+0000 mon.a (mon.0) 1055 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:40:51.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:51 smithi055 bash[9296]: audit 2020-09-08T15:40:50.627823+0000 mon.a (mon.0) 1056 : audit [INF] from='client.? v1:172.21.15.104:0/1543508356' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:40:51.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:51 smithi055 bash[9296]: cluster 2020-09-08T15:40:50.627865+0000 mon.a (mon.0) 1057 : cluster [DBG] osdmap e195: 12 total, 12 up, 12 in 2020-09-08T15:40:52.035 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:52.106 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:40:52.106 INFO:teuthology.orchestra.run.smithi055.stdout: "size": 20, 2020-09-08T15:40:52.108 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 600, 2020-09-08T15:40:52.108 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T15:40:52.110 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.110 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89439 2.0 2:0e36dded:::benchmark_data_smithi166_20551_object89438:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:40:52.110 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.174161+0000", 2020-09-08T15:40:52.110 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 147.93299846100001, 2020-09-08T15:40:52.110 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.21937716400000001, 2020-09-08T15:40:52.111 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:52.111 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:52.111 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:52.111 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:52.111 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:52.112 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89439 2020-09-08T15:40:52.112 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.112 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:52.114 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.114 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:52.114 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.174161+0000", 2020-09-08T15:40:52.115 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:52.115 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.115 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.115 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:52.115 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.174161+0000", 2020-09-08T15:40:52.116 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999681 2020-09-08T15:40:52.116 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.116 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.116 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:52.117 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.174129+0000", 2020-09-08T15:40:52.118 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000103976 2020-09-08T15:40:52.118 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.119 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.119 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:52.119 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.174233+0000", 2020-09-08T15:40:52.119 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.396e-05 2020-09-08T15:40:52.119 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.120 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.120 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:52.120 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.174247+0000", 2020-09-08T15:40:52.120 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9159999999999998e-06 2020-09-08T15:40:52.120 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.121 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.121 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:52.121 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.174253+0000", 2020-09-08T15:40:52.121 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0038480929999999999 2020-09-08T15:40:52.121 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.122 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.122 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:52.122 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.178101+0000", 2020-09-08T15:40:52.122 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00028278999999999999 2020-09-08T15:40:52.122 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.123 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.123 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:52.123 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.178383+0000", 2020-09-08T15:40:52.123 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00034010699999999997 2020-09-08T15:40:52.123 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.124 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.124 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:40:52.124 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.178724+0000", 2020-09-08T15:40:52.124 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0053716789999999999 2020-09-08T15:40:52.124 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.125 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.126 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:52.126 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.184095+0000", 2020-09-08T15:40:52.127 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.20911371300000001 2020-09-08T15:40:52.127 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.127 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.128 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:52.129 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.393209+0000", 2020-09-08T15:40:52.129 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00018925499999999999 2020-09-08T15:40:52.129 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.130 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.130 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:52.130 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.393398+0000", 2020-09-08T15:40:52.131 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013971799999999999 2020-09-08T15:40:52.131 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.131 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.132 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:52.132 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.393538+0000", 2020-09-08T15:40:52.132 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.21937716400000001 2020-09-08T15:40:52.133 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.133 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.133 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.133 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.133 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.134 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89441 2.0 2:0bc9b942:::benchmark_data_smithi166_20551_object89440:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:40:52.134 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.175434+0000", 2020-09-08T15:40:52.134 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 147.93172530000001, 2020-09-08T15:40:52.134 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.21857064800000001, 2020-09-08T15:40:52.134 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:52.135 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:52.135 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:52.135 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:52.135 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:52.135 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89441 2020-09-08T15:40:52.136 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.136 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:52.137 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.137 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:52.137 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.175434+0000", 2020-09-08T15:40:52.138 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:52.138 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.139 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.139 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:52.139 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.175434+0000", 2020-09-08T15:40:52.140 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999762 2020-09-08T15:40:52.140 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.140 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.140 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:52.140 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.175410+0000", 2020-09-08T15:40:52.141 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3329e-05 2020-09-08T15:40:52.141 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.141 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.141 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:52.142 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.175503+0000", 2020-09-08T15:40:52.143 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.9150000000000008e-06 2020-09-08T15:40:52.143 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.143 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.143 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:52.143 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.175513+0000", 2020-09-08T15:40:52.144 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2959999999999998e-06 2020-09-08T15:40:52.144 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.144 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.144 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:52.144 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.175518+0000", 2020-09-08T15:40:52.145 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0059908849999999996 2020-09-08T15:40:52.145 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.145 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.145 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:52.146 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.181509+0000", 2020-09-08T15:40:52.146 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011476699999999999 2020-09-08T15:40:52.146 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.146 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.147 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:52.147 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.181624+0000", 2020-09-08T15:40:52.147 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000119091 2020-09-08T15:40:52.147 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.148 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.148 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:40:52.148 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.181743+0000", 2020-09-08T15:40:52.148 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0087421660000000009 2020-09-08T15:40:52.148 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.149 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.149 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:52.150 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.190485+0000", 2020-09-08T15:40:52.150 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.20319680900000001 2020-09-08T15:40:52.150 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.150 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.150 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:52.151 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.393682+0000", 2020-09-08T15:40:52.151 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00021175400000000001 2020-09-08T15:40:52.151 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.151 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.151 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:52.152 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.393894+0000", 2020-09-08T15:40:52.152 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000110592 2020-09-08T15:40:52.152 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.152 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.152 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:52.153 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.394005+0000", 2020-09-08T15:40:52.153 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.21857064800000001 2020-09-08T15:40:52.153 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.153 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.153 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.154 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.154 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.154 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89446 2.0 2:0947ac70:::benchmark_data_smithi166_20551_object89445:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:40:52.154 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.179773+0000", 2020-09-08T15:40:52.154 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 147.927386353, 2020-09-08T15:40:52.155 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.226094445, 2020-09-08T15:40:52.155 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:52.155 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:52.155 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:52.155 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:52.156 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:52.156 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89446 2020-09-08T15:40:52.156 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.156 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:52.156 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.156 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:52.157 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.179773+0000", 2020-09-08T15:40:52.158 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:52.158 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.158 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.159 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:52.159 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.179773+0000", 2020-09-08T15:40:52.159 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999599 2020-09-08T15:40:52.160 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.160 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.160 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:52.161 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.179733+0000", 2020-09-08T15:40:52.161 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000139274 2020-09-08T15:40:52.161 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.161 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.163 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:52.163 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.179872+0000", 2020-09-08T15:40:52.164 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.7482999999999998e-05 2020-09-08T15:40:52.164 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.164 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.164 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:52.164 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.179889+0000", 2020-09-08T15:40:52.165 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.3039999999999999e-06 2020-09-08T15:40:52.165 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.165 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.165 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:52.165 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.179897+0000", 2020-09-08T15:40:52.166 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0042542550000000002 2020-09-08T15:40:52.166 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.166 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.166 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:52.166 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.184151+0000", 2020-09-08T15:40:52.166 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019428300000000001 2020-09-08T15:40:52.167 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.167 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.167 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:52.167 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.184345+0000", 2020-09-08T15:40:52.167 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000216963 2020-09-08T15:40:52.168 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.168 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.168 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:40:52.168 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.184562+0000", 2020-09-08T15:40:52.168 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0059322400000000001 2020-09-08T15:40:52.169 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.169 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.169 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:52.169 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.190494+0000", 2020-09-08T15:40:52.169 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.21496723700000001 2020-09-08T15:40:52.170 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.170 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.170 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:52.170 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.405462+0000", 2020-09-08T15:40:52.170 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00021017400000000001 2020-09-08T15:40:52.171 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.171 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.171 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:52.171 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.405672+0000", 2020-09-08T15:40:52.171 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019537999999999999 2020-09-08T15:40:52.172 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.172 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.172 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:52.173 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.405867+0000", 2020-09-08T15:40:52.173 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.226094445 2020-09-08T15:40:52.173 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.173 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.173 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.174 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.174 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.174 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89460 2.0 2:086076a4:::benchmark_data_smithi166_20551_object89459:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:40:52.174 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.186439+0000", 2020-09-08T15:40:52.174 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 147.92072050100001, 2020-09-08T15:40:52.175 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.22460018800000001, 2020-09-08T15:40:52.175 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:52.175 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:52.175 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:52.175 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:52.176 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:52.176 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89460 2020-09-08T15:40:52.176 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.176 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:52.176 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.176 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:52.177 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.186439+0000", 2020-09-08T15:40:52.177 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:52.177 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.177 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.177 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:52.178 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.186439+0000", 2020-09-08T15:40:52.178 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999571 2020-09-08T15:40:52.178 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.178 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.178 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:52.179 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.186396+0000", 2020-09-08T15:40:52.179 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000159894 2020-09-08T15:40:52.179 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.179 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.179 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:52.180 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.186556+0000", 2020-09-08T15:40:52.180 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6770999999999999e-05 2020-09-08T15:40:52.180 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.180 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.180 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:52.181 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.186572+0000", 2020-09-08T15:40:52.181 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0795000000000001e-05 2020-09-08T15:40:52.181 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.181 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.181 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:52.182 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.186583+0000", 2020-09-08T15:40:52.182 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00075302799999999999 2020-09-08T15:40:52.182 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.182 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.182 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:52.183 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.187336+0000", 2020-09-08T15:40:52.183 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016681399999999999 2020-09-08T15:40:52.183 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.183 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.183 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:52.184 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.187503+0000", 2020-09-08T15:40:52.184 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00021730600000000001 2020-09-08T15:40:52.184 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.184 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.184 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:40:52.184 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.187720+0000", 2020-09-08T15:40:52.185 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0031015919999999998 2020-09-08T15:40:52.185 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.185 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.185 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:52.186 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.190822+0000", 2020-09-08T15:40:52.186 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.21988956600000001 2020-09-08T15:40:52.186 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.186 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.186 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:52.186 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.410712+0000", 2020-09-08T15:40:52.187 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000173163 2020-09-08T15:40:52.187 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.187 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.187 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:52.187 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.410885+0000", 2020-09-08T15:40:52.188 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015414399999999999 2020-09-08T15:40:52.188 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.188 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.188 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:52.188 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.411039+0000", 2020-09-08T15:40:52.189 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.22460018800000001 2020-09-08T15:40:52.189 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.189 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.189 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.189 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.190 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.190 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89516 2.0 2:088ea7fe:::benchmark_data_smithi166_20551_object89515:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:40:52.190 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.223866+0000", 2020-09-08T15:40:52.190 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 147.88329320599999, 2020-09-08T15:40:52.190 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.18755557, 2020-09-08T15:40:52.191 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:52.191 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:52.191 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:52.191 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:52.191 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:52.191 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89516 2020-09-08T15:40:52.192 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.192 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:52.192 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.192 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:52.192 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.223866+0000", 2020-09-08T15:40:52.193 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:52.193 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.193 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.193 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:52.193 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.223866+0000", 2020-09-08T15:40:52.194 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999723 2020-09-08T15:40:52.194 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.194 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.194 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:52.194 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.223838+0000", 2020-09-08T15:40:52.195 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3925000000000006e-05 2020-09-08T15:40:52.195 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.195 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.195 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:52.195 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.223932+0000", 2020-09-08T15:40:52.195 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.0628999999999998e-05 2020-09-08T15:40:52.196 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.196 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.196 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:52.196 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.223953+0000", 2020-09-08T15:40:52.197 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.1099999999999992e-06 2020-09-08T15:40:52.197 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.197 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.197 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:52.197 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.223962+0000", 2020-09-08T15:40:52.198 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.8196999999999999e-05 2020-09-08T15:40:52.198 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.198 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.198 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:52.198 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.224010+0000", 2020-09-08T15:40:52.199 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012655399999999999 2020-09-08T15:40:52.199 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.199 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.199 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:52.199 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.224137+0000", 2020-09-08T15:40:52.200 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000121923 2020-09-08T15:40:52.200 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.200 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.200 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:40:52.200 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.224259+0000", 2020-09-08T15:40:52.201 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0066709400000000002 2020-09-08T15:40:52.201 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.201 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.201 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:52.201 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.230930+0000", 2020-09-08T15:40:52.202 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.180219667 2020-09-08T15:40:52.202 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.202 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.202 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:52.202 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.411149+0000", 2020-09-08T15:40:52.203 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000127995 2020-09-08T15:40:52.203 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.203 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.203 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:52.203 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.411277+0000", 2020-09-08T15:40:52.204 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000144291 2020-09-08T15:40:52.204 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.204 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.204 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:52.205 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.411422+0000", 2020-09-08T15:40:52.205 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.18755557 2020-09-08T15:40:52.205 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.205 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.206 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.206 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.206 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.206 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89520 2.0 2:044f3cb9:::benchmark_data_smithi166_20551_object89519:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:40:52.207 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.225510+0000", 2020-09-08T15:40:52.207 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 147.88164899500001, 2020-09-08T15:40:52.207 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.189580258, 2020-09-08T15:40:52.207 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:52.207 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:52.207 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:52.208 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:52.208 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:52.208 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89520 2020-09-08T15:40:52.208 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.208 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:52.209 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.209 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:52.209 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.225510+0000", 2020-09-08T15:40:52.209 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:52.209 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.210 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.210 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:52.210 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.225510+0000", 2020-09-08T15:40:52.210 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999599 2020-09-08T15:40:52.210 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.211 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.211 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:52.211 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.225470+0000", 2020-09-08T15:40:52.211 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000151206 2020-09-08T15:40:52.211 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.211 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.212 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:52.212 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.225621+0000", 2020-09-08T15:40:52.212 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.9643000000000001e-05 2020-09-08T15:40:52.212 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.212 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.213 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:52.213 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.225641+0000", 2020-09-08T15:40:52.213 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.1949999999999997e-06 2020-09-08T15:40:52.213 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.213 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.214 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:52.214 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.225648+0000", 2020-09-08T15:40:52.214 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.001303238 2020-09-08T15:40:52.214 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.214 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.215 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:52.215 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.226951+0000", 2020-09-08T15:40:52.215 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000112038 2020-09-08T15:40:52.215 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.215 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.216 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:52.216 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.227063+0000", 2020-09-08T15:40:52.216 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019811000000000001 2020-09-08T15:40:52.216 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.216 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.217 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:40:52.217 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.227261+0000", 2020-09-08T15:40:52.217 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0038368679999999998 2020-09-08T15:40:52.217 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.217 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.218 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:52.218 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.231098+0000", 2020-09-08T15:40:52.218 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.18374184299999999 2020-09-08T15:40:52.218 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.218 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.219 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:52.219 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.414840+0000", 2020-09-08T15:40:52.219 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000107272 2020-09-08T15:40:52.219 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.219 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.220 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:52.220 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.414947+0000", 2020-09-08T15:40:52.220 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014302799999999999 2020-09-08T15:40:52.220 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.220 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.221 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:52.221 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.415090+0000", 2020-09-08T15:40:52.221 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.189580258 2020-09-08T15:40:52.221 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.222 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.222 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.222 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.222 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.223 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89523 2.0 2:08e116f9:::benchmark_data_smithi166_20551_object89522:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:40:52.223 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.226111+0000", 2020-09-08T15:40:52.223 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 147.88104843799999, 2020-09-08T15:40:52.223 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.189455454, 2020-09-08T15:40:52.223 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:52.224 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:52.224 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:52.224 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:52.224 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:52.224 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89523 2020-09-08T15:40:52.225 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.225 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:52.225 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.225 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:52.225 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.226111+0000", 2020-09-08T15:40:52.226 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:52.226 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.226 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.226 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:52.226 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.226111+0000", 2020-09-08T15:40:52.226 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999418 2020-09-08T15:40:52.227 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.227 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.227 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:52.227 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.226052+0000", 2020-09-08T15:40:52.227 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000161218 2020-09-08T15:40:52.228 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.228 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.228 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:52.228 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.226214+0000", 2020-09-08T15:40:52.228 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.7329e-05 2020-09-08T15:40:52.229 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.229 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.229 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:52.229 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.226231+0000", 2020-09-08T15:40:52.229 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.6839999999999999e-06 2020-09-08T15:40:52.230 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.230 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.230 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:52.230 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.226238+0000", 2020-09-08T15:40:52.230 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0027857310000000001 2020-09-08T15:40:52.230 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.231 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.231 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:52.231 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.229023+0000", 2020-09-08T15:40:52.231 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017312600000000001 2020-09-08T15:40:52.236 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.236 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.237 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:52.238 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.229196+0000", 2020-09-08T15:40:52.238 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016157899999999999 2020-09-08T15:40:52.238 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.238 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.239 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:40:52.239 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.229358+0000", 2020-09-08T15:40:52.239 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0017535249999999999 2020-09-08T15:40:52.239 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.239 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.240 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:52.240 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.231112+0000", 2020-09-08T15:40:52.240 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.18410831599999999 2020-09-08T15:40:52.240 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.240 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.241 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:52.241 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.415220+0000", 2020-09-08T15:40:52.241 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00020349800000000001 2020-09-08T15:40:52.241 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.241 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.242 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:52.242 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.415423+0000", 2020-09-08T15:40:52.242 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000142812 2020-09-08T15:40:52.242 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.242 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.243 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:52.243 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.415566+0000", 2020-09-08T15:40:52.243 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.189455454 2020-09-08T15:40:52.243 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.243 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.244 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.244 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.244 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.244 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89537 2.0 2:018d1f16:::benchmark_data_smithi166_20551_object89536:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:40:52.244 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.239835+0000", 2020-09-08T15:40:52.245 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 147.86732452000001, 2020-09-08T15:40:52.245 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.17835358600000001, 2020-09-08T15:40:52.245 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:52.245 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:52.245 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:52.246 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:52.246 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:52.246 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89537 2020-09-08T15:40:52.246 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.246 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:52.246 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.247 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:52.247 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.239835+0000", 2020-09-08T15:40:52.248 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:52.248 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.248 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.248 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:52.248 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.239835+0000", 2020-09-08T15:40:52.249 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999733 2020-09-08T15:40:52.249 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.250 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.250 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:52.250 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.239808+0000", 2020-09-08T15:40:52.250 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.7369999999999999e-05 2020-09-08T15:40:52.251 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.252 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.252 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:52.252 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.239895+0000", 2020-09-08T15:40:52.252 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3844e-05 2020-09-08T15:40:52.252 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.253 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.253 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:52.253 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.239909+0000", 2020-09-08T15:40:52.253 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.5729999999999998e-06 2020-09-08T15:40:52.253 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.254 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.254 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:52.254 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.239915+0000", 2020-09-08T15:40:52.254 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.1159000000000003e-05 2020-09-08T15:40:52.254 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.255 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.255 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:52.255 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.239966+0000", 2020-09-08T15:40:52.255 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013670500000000001 2020-09-08T15:40:52.255 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.256 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.256 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:52.256 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.240103+0000", 2020-09-08T15:40:52.256 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013091099999999999 2020-09-08T15:40:52.256 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.257 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.257 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:40:52.257 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.240234+0000", 2020-09-08T15:40:52.257 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0028162170000000002 2020-09-08T15:40:52.257 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.258 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.258 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:52.258 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.243050+0000", 2020-09-08T15:40:52.258 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.17495691099999999 2020-09-08T15:40:52.258 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.259 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.259 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:52.259 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.418007+0000", 2020-09-08T15:40:52.259 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000104721 2020-09-08T15:40:52.259 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.259 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.260 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:52.260 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.418111+0000", 2020-09-08T15:40:52.260 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.6829000000000006e-05 2020-09-08T15:40:52.260 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.260 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.261 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:52.261 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.418188+0000", 2020-09-08T15:40:52.261 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.17835358600000001 2020-09-08T15:40:52.261 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.261 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.262 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.262 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.262 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.262 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89587 2.0 2:0c8f5aee:::benchmark_data_smithi166_20551_object89586:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:40:52.262 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.281537+0000", 2020-09-08T15:40:52.263 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 147.82562210699999, 2020-09-08T15:40:52.263 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.137134652, 2020-09-08T15:40:52.263 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:52.263 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:52.263 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:52.263 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:52.264 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:52.264 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89587 2020-09-08T15:40:52.264 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.264 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:52.264 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.265 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:52.265 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.281537+0000", 2020-09-08T15:40:52.265 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:52.265 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.266 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.266 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:52.266 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.281537+0000", 2020-09-08T15:40:52.266 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999719 2020-09-08T15:40:52.266 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.267 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.267 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:52.267 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.281509+0000", 2020-09-08T15:40:52.267 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4314000000000005e-05 2020-09-08T15:40:52.267 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.268 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.268 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:52.268 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.281603+0000", 2020-09-08T15:40:52.268 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0509999999999999e-05 2020-09-08T15:40:52.268 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.268 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.269 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:52.269 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.281614+0000", 2020-09-08T15:40:52.269 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.1270000000000002e-06 2020-09-08T15:40:52.269 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.269 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.270 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:52.270 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.281619+0000", 2020-09-08T15:40:52.270 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3501000000000001e-05 2020-09-08T15:40:52.270 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.270 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.271 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:52.271 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.281682+0000", 2020-09-08T15:40:52.271 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014942699999999999 2020-09-08T15:40:52.271 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.271 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.272 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:52.272 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.281832+0000", 2020-09-08T15:40:52.272 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013368399999999999 2020-09-08T15:40:52.272 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.272 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.272 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:40:52.273 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.281966+0000", 2020-09-08T15:40:52.273 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.002819097 2020-09-08T15:40:52.273 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.273 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.273 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:52.274 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.284785+0000", 2020-09-08T15:40:52.274 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13357950299999999 2020-09-08T15:40:52.274 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.274 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.274 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:52.275 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.418364+0000", 2020-09-08T15:40:52.275 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000184484 2020-09-08T15:40:52.275 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.275 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.275 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:52.275 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.418549+0000", 2020-09-08T15:40:52.276 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012305199999999999 2020-09-08T15:40:52.276 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.276 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.276 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:52.276 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.418672+0000", 2020-09-08T15:40:52.277 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.137134652 2020-09-08T15:40:52.277 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.277 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.277 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.277 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.278 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.278 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89602 2.0 2:0b0345ed:::benchmark_data_smithi166_20551_object89601:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:40:52.278 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.295553+0000", 2020-09-08T15:40:52.278 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 147.81160610500001, 2020-09-08T15:40:52.278 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.12688089399999999, 2020-09-08T15:40:52.279 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:52.279 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:52.279 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:52.279 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:52.279 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:52.279 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89602 2020-09-08T15:40:52.280 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.280 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:52.280 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.280 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:52.280 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.295553+0000", 2020-09-08T15:40:52.281 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:52.281 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.281 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.281 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:52.281 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.295553+0000", 2020-09-08T15:40:52.282 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999695 2020-09-08T15:40:52.282 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.282 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.282 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:52.282 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.295523+0000", 2020-09-08T15:40:52.283 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000117009 2020-09-08T15:40:52.283 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.283 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.283 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:52.283 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.295640+0000", 2020-09-08T15:40:52.284 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6603999999999999e-05 2020-09-08T15:40:52.284 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.284 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.284 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:52.284 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.295656+0000", 2020-09-08T15:40:52.284 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.02e-06 2020-09-08T15:40:52.285 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.285 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.285 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:52.285 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.295662+0000", 2020-09-08T15:40:52.285 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.5550000000000003e-05 2020-09-08T15:40:52.286 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.286 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.286 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:52.286 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.295708+0000", 2020-09-08T15:40:52.286 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000144191 2020-09-08T15:40:52.287 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.287 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.287 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:52.287 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.295852+0000", 2020-09-08T15:40:52.287 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017592399999999999 2020-09-08T15:40:52.287 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.288 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.288 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:40:52.288 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.296028+0000", 2020-09-08T15:40:52.288 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0030969019999999999 2020-09-08T15:40:52.288 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.289 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.289 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:52.289 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.299125+0000", 2020-09-08T15:40:52.289 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.12320244900000001 2020-09-08T15:40:52.289 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.290 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.290 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:52.290 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.422327+0000", 2020-09-08T15:40:52.290 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3899000000000006e-05 2020-09-08T15:40:52.290 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.291 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.291 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:52.291 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.422391+0000", 2020-09-08T15:40:52.291 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.2871999999999999e-05 2020-09-08T15:40:52.291 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.291 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.292 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:52.292 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.422434+0000", 2020-09-08T15:40:52.292 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.12688089399999999 2020-09-08T15:40:52.292 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.292 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.293 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.293 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.293 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.293 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89607 2.0 2:022da93d:::benchmark_data_smithi166_20551_object89606:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:40:52.293 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.299434+0000", 2020-09-08T15:40:52.294 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 147.80772468999999, 2020-09-08T15:40:52.294 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.123131773, 2020-09-08T15:40:52.294 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:52.294 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:52.294 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:52.294 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:52.295 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:52.295 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89607 2020-09-08T15:40:52.295 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.295 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:52.295 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.296 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:52.296 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.299434+0000", 2020-09-08T15:40:52.296 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:52.296 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.296 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.297 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:52.297 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.299434+0000", 2020-09-08T15:40:52.297 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999342 2020-09-08T15:40:52.297 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.297 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.298 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:52.298 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.299369+0000", 2020-09-08T15:40:52.298 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00023050100000000001 2020-09-08T15:40:52.298 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.298 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.298 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:52.299 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.299599+0000", 2020-09-08T15:40:52.299 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.1597999999999999e-05 2020-09-08T15:40:52.299 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.299 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.299 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:52.300 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.299621+0000", 2020-09-08T15:40:52.300 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.136e-05 2020-09-08T15:40:52.300 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.300 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.300 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:52.301 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.299632+0000", 2020-09-08T15:40:52.301 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.117e-05 2020-09-08T15:40:52.301 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.301 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.301 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:52.301 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.299683+0000", 2020-09-08T15:40:52.302 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015463599999999999 2020-09-08T15:40:52.302 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.302 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.302 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:52.302 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.299838+0000", 2020-09-08T15:40:52.303 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00030745099999999998 2020-09-08T15:40:52.303 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.303 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.303 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:40:52.303 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.300145+0000", 2020-09-08T15:40:52.304 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0034436800000000002 2020-09-08T15:40:52.304 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.304 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.304 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:52.304 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.303589+0000", 2020-09-08T15:40:52.304 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.118888599 2020-09-08T15:40:52.305 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.305 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.305 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:52.305 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.422478+0000", 2020-09-08T15:40:52.305 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.0747e-05 2020-09-08T15:40:52.306 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.306 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.306 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:52.306 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.422528+0000", 2020-09-08T15:40:52.306 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.7802999999999997e-05 2020-09-08T15:40:52.307 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.307 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.307 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:52.307 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.422566+0000", 2020-09-08T15:40:52.307 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.123131773 2020-09-08T15:40:52.307 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.308 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.308 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.308 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.308 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.308 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89617 2.0 2:0a66c03e:::benchmark_data_smithi166_20551_object89616:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:40:52.309 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.311993+0000", 2020-09-08T15:40:52.309 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 147.79516643400001, 2020-09-08T15:40:52.309 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.11077023499999999, 2020-09-08T15:40:52.309 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:52.309 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:52.310 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:52.310 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:52.310 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:52.310 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89617 2020-09-08T15:40:52.310 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.311 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:52.311 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.311 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:52.311 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.311993+0000", 2020-09-08T15:40:52.311 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:52.311 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.312 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.312 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:52.312 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.311993+0000", 2020-09-08T15:40:52.312 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999032 2020-09-08T15:40:52.312 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.313 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.313 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:52.313 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.311896+0000", 2020-09-08T15:40:52.313 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00030494799999999998 2020-09-08T15:40:52.313 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.314 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.314 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:52.315 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.312201+0000", 2020-09-08T15:40:52.315 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.3696999999999999e-05 2020-09-08T15:40:52.315 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.315 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.315 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:52.316 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.312235+0000", 2020-09-08T15:40:52.316 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6943999999999999e-05 2020-09-08T15:40:52.316 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.316 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.316 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:52.317 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.312252+0000", 2020-09-08T15:40:52.317 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.2341999999999994e-05 2020-09-08T15:40:52.317 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.317 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.317 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:52.318 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.312314+0000", 2020-09-08T15:40:52.318 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010929099999999999 2020-09-08T15:40:52.318 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.318 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.318 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:52.319 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.312423+0000", 2020-09-08T15:40:52.319 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000111549 2020-09-08T15:40:52.319 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.320 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.320 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:40:52.320 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.312535+0000", 2020-09-08T15:40:52.320 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0026448869999999998 2020-09-08T15:40:52.320 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.321 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.321 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:52.321 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.315180+0000", 2020-09-08T15:40:52.321 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.107495172 2020-09-08T15:40:52.321 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.322 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.322 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:52.322 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.422675+0000", 2020-09-08T15:40:52.322 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.0086000000000001e-05 2020-09-08T15:40:52.322 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.323 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.323 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:52.323 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.422725+0000", 2020-09-08T15:40:52.324 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.8000000000000002e-05 2020-09-08T15:40:52.324 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.324 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.324 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:52.324 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.422763+0000", 2020-09-08T15:40:52.325 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.11077023499999999 2020-09-08T15:40:52.326 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.326 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.326 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.326 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.326 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.327 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89644 2.0 2:04eb915d:::benchmark_data_smithi166_20551_object89643:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:40:52.327 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.353047+0000", 2020-09-08T15:40:52.327 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 147.75411177999999, 2020-09-08T15:40:52.327 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.069886246999999999, 2020-09-08T15:40:52.327 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:52.328 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:52.328 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:52.328 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:52.328 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:52.328 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89644 2020-09-08T15:40:52.329 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.329 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:52.329 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.329 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:52.329 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.353047+0000", 2020-09-08T15:40:52.330 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:52.330 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.330 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.331 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:52.331 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.353047+0000", 2020-09-08T15:40:52.331 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999361 2020-09-08T15:40:52.331 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.331 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.331 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:52.332 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.352983+0000", 2020-09-08T15:40:52.332 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000182475 2020-09-08T15:40:52.332 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.332 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.332 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:52.333 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.353166+0000", 2020-09-08T15:40:52.333 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.9805e-05 2020-09-08T15:40:52.334 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.334 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.334 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:52.334 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.353196+0000", 2020-09-08T15:40:52.335 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.9230000000000002e-06 2020-09-08T15:40:52.335 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.335 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.336 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:52.336 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.353204+0000", 2020-09-08T15:40:52.336 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.1116e-05 2020-09-08T15:40:52.336 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.336 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.337 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:52.337 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.353295+0000", 2020-09-08T15:40:52.337 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000235765 2020-09-08T15:40:52.337 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.338 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.338 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:52.338 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.353530+0000", 2020-09-08T15:40:52.338 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00025074199999999998 2020-09-08T15:40:52.338 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.339 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.339 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:40:52.339 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.353781+0000", 2020-09-08T15:40:52.339 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0075869609999999997 2020-09-08T15:40:52.339 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.340 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.340 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:52.341 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.361368+0000", 2020-09-08T15:40:52.341 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.061480008000000003 2020-09-08T15:40:52.341 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.341 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.342 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:52.342 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.422848+0000", 2020-09-08T15:40:52.342 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.7945000000000003e-05 2020-09-08T15:40:52.342 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.343 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.343 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:52.343 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.422896+0000", 2020-09-08T15:40:52.343 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.7537999999999999e-05 2020-09-08T15:40:52.343 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.344 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.344 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:52.344 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.422934+0000", 2020-09-08T15:40:52.344 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.069886246999999999 2020-09-08T15:40:52.344 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.345 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.346 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.346 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.346 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.346 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89645 2.0 2:0274707b:::benchmark_data_smithi166_20551_object89644:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:40:52.346 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.354188+0000", 2020-09-08T15:40:52.347 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 147.752971022, 2020-09-08T15:40:52.347 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.068920411000000001, 2020-09-08T15:40:52.347 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:52.347 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:52.347 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:52.348 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:52.348 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:52.348 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89645 2020-09-08T15:40:52.348 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.348 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:52.349 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.349 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:52.349 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.354188+0000", 2020-09-08T15:40:52.349 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:52.349 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.350 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.350 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:52.350 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.354188+0000", 2020-09-08T15:40:52.350 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999908 2020-09-08T15:40:52.350 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.351 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.351 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:52.351 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.354096+0000", 2020-09-08T15:40:52.351 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00033920400000000001 2020-09-08T15:40:52.351 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.352 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.352 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:52.352 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.354435+0000", 2020-09-08T15:40:52.352 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.3025000000000003e-05 2020-09-08T15:40:52.352 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.352 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.353 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:52.353 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.354468+0000", 2020-09-08T15:40:52.353 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3743e-05 2020-09-08T15:40:52.353 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.353 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.354 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:52.354 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.354482+0000", 2020-09-08T15:40:52.354 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.003284899 2020-09-08T15:40:52.354 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.354 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.355 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:52.355 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.357767+0000", 2020-09-08T15:40:52.355 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00029920899999999999 2020-09-08T15:40:52.355 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.355 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.356 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:52.356 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.358066+0000", 2020-09-08T15:40:52.357 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00029829800000000001 2020-09-08T15:40:52.357 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.357 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.357 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:40:52.358 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.358364+0000", 2020-09-08T15:40:52.358 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0032520719999999999 2020-09-08T15:40:52.358 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.358 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.358 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:52.359 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.361616+0000", 2020-09-08T15:40:52.359 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.061409819999999997 2020-09-08T15:40:52.359 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.359 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.359 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:52.359 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.423026+0000", 2020-09-08T15:40:52.360 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.7165e-05 2020-09-08T15:40:52.360 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.360 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.360 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:52.360 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.423073+0000", 2020-09-08T15:40:52.361 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.5145999999999999e-05 2020-09-08T15:40:52.361 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.361 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.361 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:52.361 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.423109+0000", 2020-09-08T15:40:52.362 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.068920411000000001 2020-09-08T15:40:52.362 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.362 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.362 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.362 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.363 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.363 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89649 2.0 2:00923499:::benchmark_data_smithi166_20551_object89648:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:40:52.363 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.367450+0000", 2020-09-08T15:40:52.363 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 147.73970935599999, 2020-09-08T15:40:52.363 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.055827651999999998, 2020-09-08T15:40:52.364 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:52.364 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:52.364 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:52.364 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:52.364 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:52.365 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89649 2020-09-08T15:40:52.365 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.365 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:52.365 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.365 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:52.366 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.367450+0000", 2020-09-08T15:40:52.366 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:52.366 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.366 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.366 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:52.367 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.367450+0000", 2020-09-08T15:40:52.367 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999917 2020-09-08T15:40:52.367 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.367 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.367 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:52.367 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.367367+0000", 2020-09-08T15:40:52.368 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00023770799999999999 2020-09-08T15:40:52.368 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.368 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.368 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:52.368 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.367605+0000", 2020-09-08T15:40:52.369 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.4606e-05 2020-09-08T15:40:52.369 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.369 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.369 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:52.369 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.367639+0000", 2020-09-08T15:40:52.370 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6537000000000001e-05 2020-09-08T15:40:52.370 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.370 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.370 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:52.370 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.367656+0000", 2020-09-08T15:40:52.370 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012123799999999999 2020-09-08T15:40:52.371 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.371 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.371 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:52.371 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.367777+0000", 2020-09-08T15:40:52.371 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00029653399999999999 2020-09-08T15:40:52.372 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.372 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.372 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:52.372 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.368074+0000", 2020-09-08T15:40:52.372 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00030816399999999999 2020-09-08T15:40:52.373 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.373 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.373 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:40:52.373 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.368382+0000", 2020-09-08T15:40:52.373 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.003565988 2020-09-08T15:40:52.373 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.374 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.374 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:52.374 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.371948+0000", 2020-09-08T15:40:52.374 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.051251810000000002 2020-09-08T15:40:52.374 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.375 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.375 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:52.375 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.423200+0000", 2020-09-08T15:40:52.375 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.4635000000000002e-05 2020-09-08T15:40:52.375 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.376 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.376 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:52.377 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.423244+0000", 2020-09-08T15:40:52.378 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.3219000000000002e-05 2020-09-08T15:40:52.378 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.378 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.378 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:52.379 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.423277+0000", 2020-09-08T15:40:52.379 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.055827651999999998 2020-09-08T15:40:52.379 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.379 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.379 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.380 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.380 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.380 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:163598 2.8 2:18d67c1d:::benchmark_data_smithi166_20551_object15978:head [delete] snapc 0=[] ondisk+write+known_if_redirected e170)", 2020-09-08T15:40:52.380 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:29.622448+0000", 2020-09-08T15:40:52.380 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 82.484711586000003, 2020-09-08T15:40:52.381 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.054119552000000001, 2020-09-08T15:40:52.381 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:52.381 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:52.381 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:52.381 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:52.382 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:52.382 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 163598 2020-09-08T15:40:52.382 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.382 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:52.382 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.383 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:52.383 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.622448+0000", 2020-09-08T15:40:52.383 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:52.383 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.383 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.384 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:52.384 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.622448+0000", 2020-09-08T15:40:52.384 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999356 2020-09-08T15:40:52.384 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.384 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.385 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:52.385 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.622383+0000", 2020-09-08T15:40:52.385 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013599700000000001 2020-09-08T15:40:52.385 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.385 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.386 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:52.386 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.622519+0000", 2020-09-08T15:40:52.386 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.2985000000000001e-05 2020-09-08T15:40:52.386 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.386 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.387 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:52.387 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.622542+0000", 2020-09-08T15:40:52.387 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3186999999999999e-05 2020-09-08T15:40:52.388 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.389 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.389 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:52.389 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.622556+0000", 2020-09-08T15:40:52.389 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0063137540000000004 2020-09-08T15:40:52.390 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.390 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.390 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:52.390 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.628869+0000", 2020-09-08T15:40:52.390 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00020320599999999999 2020-09-08T15:40:52.391 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.391 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.391 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:52.391 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.629072+0000", 2020-09-08T15:40:52.391 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000176952 2020-09-08T15:40:52.391 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.392 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.392 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 9", 2020-09-08T15:40:52.392 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.629249+0000", 2020-09-08T15:40:52.392 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.037862382 2020-09-08T15:40:52.392 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.393 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.393 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:52.393 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.667112+0000", 2020-09-08T15:40:52.393 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0092788030000000004 2020-09-08T15:40:52.393 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.394 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.394 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:52.394 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.676391+0000", 2020-09-08T15:40:52.395 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010282599999999999 2020-09-08T15:40:52.395 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.395 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.395 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:52.395 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.676493+0000", 2020-09-08T15:40:52.396 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.3707000000000004e-05 2020-09-08T15:40:52.396 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.396 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.396 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:52.396 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.676567+0000", 2020-09-08T15:40:52.396 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.054119552000000001 2020-09-08T15:40:52.397 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.397 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.397 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.397 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.397 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.398 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:175128 2.8 2:149b75c1:::benchmark_data_smithi166_20551_object27508:head [delete] snapc 0=[] ondisk+write+known_if_redirected e173)", 2020-09-08T15:40:52.398 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:41.251866+0000", 2020-09-08T15:40:52.398 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 70.855293055000004, 2020-09-08T15:40:52.398 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.107354433, 2020-09-08T15:40:52.398 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:52.399 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:52.399 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:52.399 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:52.399 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:52.399 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 175128 2020-09-08T15:40:52.400 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.400 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:52.400 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.400 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:52.400 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.251866+0000", 2020-09-08T15:40:52.401 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:52.401 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.401 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.401 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:52.401 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.251866+0000", 2020-09-08T15:40:52.402 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999547 2020-09-08T15:40:52.403 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.403 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.403 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:52.403 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.251821+0000", 2020-09-08T15:40:52.404 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000101625 2020-09-08T15:40:52.404 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.404 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.404 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:52.404 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.251922+0000", 2020-09-08T15:40:52.405 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6382999999999999e-05 2020-09-08T15:40:52.405 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.405 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.405 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:52.405 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.251939+0000", 2020-09-08T15:40:52.406 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.2339999999999997e-06 2020-09-08T15:40:52.406 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.406 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.406 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:52.406 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.251946+0000", 2020-09-08T15:40:52.407 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.6387e-05 2020-09-08T15:40:52.407 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.407 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.407 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:52.407 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.252012+0000", 2020-09-08T15:40:52.408 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015116500000000001 2020-09-08T15:40:52.408 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.408 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.408 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:52.408 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.252163+0000", 2020-09-08T15:40:52.409 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012295599999999999 2020-09-08T15:40:52.409 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.409 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.409 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 9", 2020-09-08T15:40:52.409 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.252286+0000", 2020-09-08T15:40:52.409 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0091166770000000001 2020-09-08T15:40:52.410 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.410 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.410 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:52.410 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.261403+0000", 2020-09-08T15:40:52.410 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.097473291000000004 2020-09-08T15:40:52.411 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.411 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.411 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:52.411 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.358876+0000", 2020-09-08T15:40:52.411 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00018616600000000001 2020-09-08T15:40:52.412 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.412 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.412 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:52.412 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.359062+0000", 2020-09-08T15:40:52.412 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015807000000000001 2020-09-08T15:40:52.413 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.413 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.413 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:52.413 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.359221+0000", 2020-09-08T15:40:52.413 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.107354433 2020-09-08T15:40:52.414 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.414 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.414 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.414 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.414 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.415 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:175131 2.8 2:19c3e01e:::benchmark_data_smithi166_20551_object27511:head [delete] snapc 0=[] ondisk+write+known_if_redirected e173)", 2020-09-08T15:40:52.415 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:41.252973+0000", 2020-09-08T15:40:52.415 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 70.854186221999996, 2020-09-08T15:40:52.415 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.106997694, 2020-09-08T15:40:52.415 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:52.416 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:52.416 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:52.416 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:52.416 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:52.416 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 175131 2020-09-08T15:40:52.417 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.417 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:52.417 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.417 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:52.417 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.252973+0000", 2020-09-08T15:40:52.418 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:52.418 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.418 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.418 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:52.418 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.252973+0000", 2020-09-08T15:40:52.419 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999714 2020-09-08T15:40:52.419 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.419 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.419 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:52.419 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.252945+0000", 2020-09-08T15:40:52.419 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.4818000000000001e-05 2020-09-08T15:40:52.420 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.420 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.420 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:52.420 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.253019+0000", 2020-09-08T15:40:52.420 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1229999999999999e-05 2020-09-08T15:40:52.421 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.421 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.421 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:52.421 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.253031+0000", 2020-09-08T15:40:52.421 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.169e-06 2020-09-08T15:40:52.422 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.422 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.422 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:52.422 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.253036+0000", 2020-09-08T15:40:52.422 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.003938548 2020-09-08T15:40:52.423 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.423 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.424 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:52.424 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.256974+0000", 2020-09-08T15:40:52.424 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011123 2020-09-08T15:40:52.425 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.425 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.425 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:52.425 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.257086+0000", 2020-09-08T15:40:52.425 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000107394 2020-09-08T15:40:52.426 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.426 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.426 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 9", 2020-09-08T15:40:52.426 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.257193+0000", 2020-09-08T15:40:52.426 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.102161761 2020-09-08T15:40:52.426 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.427 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.427 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:52.427 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.359355+0000", 2020-09-08T15:40:52.427 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2814999999999999e-05 2020-09-08T15:40:52.427 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.428 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.428 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:52.428 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.359437+0000", 2020-09-08T15:40:52.428 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000370999 2020-09-08T15:40:52.428 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.429 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.429 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:52.429 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.359808+0000", 2020-09-08T15:40:52.429 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016218999999999999 2020-09-08T15:40:52.429 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.430 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.430 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:52.430 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.359971+0000", 2020-09-08T15:40:52.430 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.106997694 2020-09-08T15:40:52.430 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.431 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.431 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.431 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.431 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.431 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:175135 2.8 2:103c3040:::benchmark_data_smithi166_20551_object27515:head [delete] snapc 0=[] ondisk+write+known_if_redirected e173)", 2020-09-08T15:40:52.431 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:41.257657+0000", 2020-09-08T15:40:52.432 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 70.849502595999994, 2020-09-08T15:40:52.432 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.104412821, 2020-09-08T15:40:52.432 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:52.432 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:52.432 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:40:52.433 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:40:52.433 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:40:52.433 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 175135 2020-09-08T15:40:52.433 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.433 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:52.434 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.434 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:52.434 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.257657+0000", 2020-09-08T15:40:52.434 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:52.434 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.435 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.435 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:52.435 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.257657+0000", 2020-09-08T15:40:52.435 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999642 2020-09-08T15:40:52.435 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.435 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.436 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:52.436 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.257621+0000", 2020-09-08T15:40:52.436 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.9930000000000002e-05 2020-09-08T15:40:52.436 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.436 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.437 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:52.437 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.257701+0000", 2020-09-08T15:40:52.437 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4738e-05 2020-09-08T15:40:52.437 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.437 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.438 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:52.438 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.257716+0000", 2020-09-08T15:40:52.438 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.9260000000000003e-06 2020-09-08T15:40:52.438 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.438 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.438 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:52.439 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.257723+0000", 2020-09-08T15:40:52.439 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.003709581 2020-09-08T15:40:52.439 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.439 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.439 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:52.440 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.261432+0000", 2020-09-08T15:40:52.440 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000107734 2020-09-08T15:40:52.440 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.440 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.440 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:52.441 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.261540+0000", 2020-09-08T15:40:52.441 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.5105999999999994e-05 2020-09-08T15:40:52.441 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.441 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.441 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 9", 2020-09-08T15:40:52.442 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.261635+0000", 2020-09-08T15:40:52.442 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.098462913999999999 2020-09-08T15:40:52.442 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.442 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.442 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:40:52.442 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.360098+0000", 2020-09-08T15:40:52.443 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.6285999999999999e-05 2020-09-08T15:40:52.443 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.443 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.443 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:40:52.444 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.360124+0000", 2020-09-08T15:40:52.445 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000155391 2020-09-08T15:40:52.445 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.445 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.445 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:52.446 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.360280+0000", 2020-09-08T15:40:52.446 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.001789892 2020-09-08T15:40:52.446 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.446 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.446 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:52.447 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.362069+0000", 2020-09-08T15:40:52.447 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.104412821 2020-09-08T15:40:52.447 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.447 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.447 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.448 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.448 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.448 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:175136 2.9 e173/126 2:9d176888:::benchmark_data_smithi166_20551_object27516:head v 173'10796, mlcod=173'10794)", 2020-09-08T15:40:52.448 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:41.258295+0000", 2020-09-08T15:40:52.448 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 70.848863847000004, 2020-09-08T15:40:52.449 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.100905498, 2020-09-08T15:40:52.449 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:40:52.449 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:40:52.449 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:40:52.450 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.450 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:40:52.450 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.258295+0000", 2020-09-08T15:40:52.450 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:40:52.450 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.451 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.451 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:40:52.451 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.258295+0000", 2020-09-08T15:40:52.451 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.771e-05 2020-09-08T15:40:52.451 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.452 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.452 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:40:52.452 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.258373+0000", 2020-09-08T15:40:52.452 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.9400000000000008e-06 2020-09-08T15:40:52.452 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.453 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.453 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:40:52.453 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.258382+0000", 2020-09-08T15:40:52.453 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.437e-05 2020-09-08T15:40:52.453 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.454 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.454 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:40:52.454 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.258396+0000", 2020-09-08T15:40:52.454 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.4250000000000003e-06 2020-09-08T15:40:52.454 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.455 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.455 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:40:52.455 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.258403+0000", 2020-09-08T15:40:52.455 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3078999999999997e-05 2020-09-08T15:40:52.455 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.456 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.456 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:40:52.456 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.258466+0000", 2020-09-08T15:40:52.456 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.7182000000000002e-05 2020-09-08T15:40:52.456 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.457 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.457 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:40:52.457 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.258493+0000", 2020-09-08T15:40:52.457 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.099979720999999994 2020-09-08T15:40:52.457 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.458 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.458 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:40:52.458 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.358473+0000", 2020-09-08T15:40:52.458 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00072807099999999995 2020-09-08T15:40:52.458 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:40:52.459 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:40:52.459 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:40:52.459 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:41.359201+0000", 2020-09-08T15:40:52.459 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.100905498 2020-09-08T15:40:52.459 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.460 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.460 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.460 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:40:52.460 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:40:52.460 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:40:52.461 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:52.462 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:40:52 smithi166 bash[16593]: debug 2020-09-08T15:40:52.037+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:52.634 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:52 smithi055 bash[9296]: cluster 2020-09-08T15:40:51.642170+0000 mgr.x (mgr.24118) 401 : cluster [DBG] pgmap v517: 17 pgs: 17 active+clean; 3.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 626 op/s 2020-09-08T15:40:52.680 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:52.782 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:52.792 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:52 smithi104 bash[9203]: cluster 2020-09-08T15:40:51.642170+0000 mgr.x (mgr.24118) 401 : cluster [DBG] pgmap v517: 17 pgs: 17 active+clean; 3.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 626 op/s 2020-09-08T15:40:52.793 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:40:52 smithi104 bash[15273]: debug 2020-09-08T15:40:52.677+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:53.095 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2020-09-08T15:40:53.104 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:52 smithi166 bash[9446]: cluster 2020-09-08T15:40:51.642170+0000 mgr.x (mgr.24118) 401 : cluster [DBG] pgmap v517: 17 pgs: 17 active+clean; 3.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 626 op/s 2020-09-08T15:40:53.257 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:40:53.286 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:53.390 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:53.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:40:53 smithi104 bash[22305]: debug 2020-09-08T15:40:53.285+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:53.702 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:53 smithi166 bash[9446]: cluster 2020-09-08T15:40:53.642755+0000 mgr.x (mgr.24118) 402 : cluster [DBG] pgmap v518: 17 pgs: 17 active+clean; 3.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:40:53.827 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:40:53.994 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:40:54.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:53 smithi055 bash[9296]: cluster 2020-09-08T15:40:53.642755+0000 mgr.x (mgr.24118) 402 : cluster [DBG] pgmap v518: 17 pgs: 17 active+clean; 3.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:40:54.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:53 smithi104 bash[9203]: cluster 2020-09-08T15:40:53.642755+0000 mgr.x (mgr.24118) 402 : cluster [DBG] pgmap v518: 17 pgs: 17 active+clean; 3.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:40:55.503 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:40:55.531 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:40:55.567 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:40:55.605 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:40:55.642 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:40:55.698 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:40:55.732 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:40:55.733 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:40:55.774 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:40:55.799 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:55 smithi104 bash[9203]: cluster 2020-09-08T15:40:54.670152+0000 osd.9 (osd.9) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:55.799 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:55 smithi104 bash[9203]: cluster 2020-09-08T15:40:54.670241+0000 osd.9 (osd.9) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:55.800 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:55 smithi104 bash[9203]: audit 2020-09-08T15:40:55.469790+0000 mon.a (mon.0) 1058 : audit [DBG] from='client.? v1:172.21.15.104:0/785676983' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:40:55.818 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:40:55.835 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:55.862 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:40:55.870 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:40:55 smithi166 bash[11928]: debug 2020-09-08T15:40:55.733+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:55.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:55 smithi166 bash[9446]: cluster 2020-09-08T15:40:54.670152+0000 osd.9 (osd.9) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:55.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:55 smithi166 bash[9446]: cluster 2020-09-08T15:40:54.670241+0000 osd.9 (osd.9) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:55.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:55 smithi166 bash[9446]: audit 2020-09-08T15:40:55.469790+0000 mon.a (mon.0) 1058 : audit [DBG] from='client.? v1:172.21.15.104:0/785676983' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:40:55.902 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:40:55.915 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:55 smithi055 bash[9296]: cluster 2020-09-08T15:40:54.670152+0000 osd.9 (osd.9) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:55.915 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:55 smithi055 bash[9296]: cluster 2020-09-08T15:40:54.670241+0000 osd.9 (osd.9) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:55.915 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:55 smithi055 bash[9296]: audit 2020-09-08T15:40:55.469790+0000 mon.a (mon.0) 1058 : audit [DBG] from='client.? v1:172.21.15.104:0/785676983' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:40:55.944 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:40:55.951 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:40:55.951 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:40:55.951 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:40:55.951 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:40:55.952 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:40:55.952 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:40:55.952 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:40:55.952 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:40:55.952 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:40:55.953 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:40:55.953 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:40:55.953 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:40:55.990 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:40:56.052 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:40:55 smithi104 bash[22305]: debug 2020-09-08T15:40:55.985+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:56.097 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:40:56.107 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T15:40:56.111 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2020-09-08T15:40:56.216 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:40:56.322 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:56.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:40:56 smithi055 bash[19003]: debug 2020-09-08T15:40:56.214+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:56.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:56 smithi055 bash[9296]: cluster 2020-09-08T15:40:55.626526+0000 mon.a (mon.0) 1059 : cluster [DBG] osdmap e196: 12 total, 12 up, 12 in 2020-09-08T15:40:56.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:56 smithi055 bash[9296]: cluster 2020-09-08T15:40:55.643294+0000 mgr.x (mgr.24118) 403 : cluster [DBG] pgmap v520: 17 pgs: 17 active+clean; 3.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 828 op/s 2020-09-08T15:40:56.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:56 smithi104 bash[9203]: cluster 2020-09-08T15:40:55.626526+0000 mon.a (mon.0) 1059 : cluster [DBG] osdmap e196: 12 total, 12 up, 12 in 2020-09-08T15:40:56.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:56 smithi104 bash[9203]: cluster 2020-09-08T15:40:55.643294+0000 mgr.x (mgr.24118) 403 : cluster [DBG] pgmap v520: 17 pgs: 17 active+clean; 3.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 828 op/s 2020-09-08T15:40:57.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:56 smithi166 bash[9446]: cluster 2020-09-08T15:40:55.626526+0000 mon.a (mon.0) 1059 : cluster [DBG] osdmap e196: 12 total, 12 up, 12 in 2020-09-08T15:40:57.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:56 smithi166 bash[9446]: cluster 2020-09-08T15:40:55.643294+0000 mgr.x (mgr.24118) 403 : cluster [DBG] pgmap v520: 17 pgs: 17 active+clean; 3.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 828 op/s 2020-09-08T15:40:57.674 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:57.781 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:40:57.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:57 smithi055 bash[9296]: audit 2020-09-08T15:40:56.099339+0000 mgr.x (mgr.24118) 404 : audit [DBG] from='client.15066 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:40:57.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:57 smithi055 bash[9296]: cluster 2020-09-08T15:40:56.946591+0000 osd.2 (osd.2) 28 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:57.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:57 smithi055 bash[9296]: cluster 2020-09-08T15:40:56.946671+0000 osd.2 (osd.2) 29 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:57.935 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:57 smithi104 bash[9203]: audit 2020-09-08T15:40:56.099339+0000 mgr.x (mgr.24118) 404 : audit [DBG] from='client.15066 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:40:57.935 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:57 smithi104 bash[9203]: cluster 2020-09-08T15:40:56.946591+0000 osd.2 (osd.2) 28 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:57.935 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:57 smithi104 bash[9203]: cluster 2020-09-08T15:40:56.946671+0000 osd.2 (osd.2) 29 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:57.936 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:40:57 smithi104 bash[15273]: debug 2020-09-08T15:40:57.673+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:57.999 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:40:58.114 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:40:58.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:57 smithi166 bash[9446]: audit 2020-09-08T15:40:56.099339+0000 mgr.x (mgr.24118) 404 : audit [DBG] from='client.15066 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:40:58.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:57 smithi166 bash[9446]: cluster 2020-09-08T15:40:56.946591+0000 osd.2 (osd.2) 28 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:40:58.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:57 smithi166 bash[9446]: cluster 2020-09-08T15:40:56.946671+0000 osd.2 (osd.2) 29 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:40:58.257 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:40:58 smithi104 bash[17588]: debug 2020-09-08T15:40:58.001+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:58.257 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2020-09-08T15:40:55.643043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3360227352,"num_objects":51277,"num_object_clones":0,"num_object_copies":102554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51277,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":243965,"num_write_kb":9447492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":34,"acting":34,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":7255212,"kb_used_data":6640140,"kb_used_omap":0,"kb_used_meta":614848,"kb_avail":1117686612,"statfs":{"total":1151940427776,"available":1144511090688,"internally_reserved":0,"allocated":6799503360,"data_stored":6796487138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":629604352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,9,10,8,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-434438144,"num_objects":-6629,"num_object_clones":0,"num_object_copies":-13258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6629,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002319"},"pg_stats":[{"pgid":"2.f","version":"195'15014","reported_seq":"15093","reported_epoch":"195","state":"active+clean","last_fresh":"2020-09-08T15:40:51.789927+0000","last_change":"2020-09-08T15:39:26.660710+0000","last_active":"2020-09-08T15:40:51.789927+0000","last_peered":"2020-09-08T15:40:51.789927+0000","last_clean":"2020-09-08T15:40:51.789927+0000","last_became_active":"2020-09-08T15:37:29.882656+0000","last_became_peered":"2020-09-08T15:37:29.882656+0000","last_unstale":"2020-09-08T15:40:51.789927+0000","last_undegraded":"2020-09-08T15:40:51.789927+0000","last_fullsized":"2020-09-08T15:40:51.789927+0000","mapping_epoch":126,"log_start":"143'5000","ondisk_log_start":"143'5000","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"168'9930","last_scrub_stamp":"2020-09-08T15:39:26.660646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:39:26.660646+0000","log_size":10014,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":209190912,"num_objects":3192,"num_object_clones":0,"num_object_copies":6384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15014,"num_write_kb":582592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9],"acting":[8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"195'15234","reported_seq":"15313","reported_epoch":"195","state":"active+clean","last_fresh":"2020-09-08T15:40:51.754478+0000","last_change":"2020-09-08T15:39:26.488598+0000","last_active":"2020-09-08T15:40:51.754478+0000","last_peered":"2020-09-08T15:40:51.754478+0000","last_clean":"2020-09-08T15:40:51.754478+0000","last_became_active":"2020-09-08T15:37:30.089439+0000","last_became_peered":"2020-09-08T15:37:30.089439+0000","last_unstale":"2020-09-08T15:40:51.754478+0000","last_undegraded":"2020-09-08T15:40:51.754478+0000","last_fullsized":"2020-09-08T15:40:51.754478+0000","mapping_epoch":126,"log_start":"143'5200","ondisk_log_start":"143'5200","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"168'10008","last_scrub_stamp":"2020-09-08T15:39:26.488528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:39:26.488528+0000","log_size":10034,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":211156992,"num_objects":3222,"num_object_clones":0,"num_object_copies":6444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15234,"num_write_kb":590592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"195'15163","reported_seq":"15245","reported_epoch":"195","state":"active+clean","last_fresh":"2020-09-08T15:40:51.651370+0000","last_change":"2020-09-08T15:40:36.687544+0000","last_active":"2020-09-08T15:40:51.651370+0000","last_peered":"2020-09-08T15:40:51.651370+0000","last_clean":"2020-09-08T15:40:51.651370+0000","last_became_active":"2020-09-08T15:37:29.876533+0000","last_became_peered":"2020-09-08T15:37:29.876533+0000","last_unstale":"2020-09-08T15:40:51.651370+0000","last_undegraded":"2020-09-08T15:40:51.651370+0000","last_fullsized":"2020-09-08T15:40:51.651370+0000","mapping_epoch":126,"log_start":"144'5100","ondisk_log_start":"144'5100","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"190'14202","last_scrub_stamp":"2020-09-08T15:40:36.687465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:36.687465+0000","log_size":10063,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":213975040,"num_objects":3265,"num_object_clones":0,"num_object_copies":6530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15163,"num_write_kb":589696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"195'15479","reported_seq":"15561","reported_epoch":"195","state":"active+clean","last_fresh":"2020-09-08T15:40:51.839260+0000","last_change":"2020-09-08T15:40:35.304847+0000","last_active":"2020-09-08T15:40:51.839260+0000","last_peered":"2020-09-08T15:40:51.839260+0000","last_clean":"2020-09-08T15:40:51.839260+0000","last_became_active":"2020-09-08T15:37:30.091928+0000","last_became_peered":"2020-09-08T15:37:30.091928+0000","last_unstale":"2020-09-08T15:40:51.839260+0000","last_undegraded":"2020-09-08T15:40:51.839260+0000","last_fullsized":"2020-09-08T15:40:51.839260+0000","mapping_epoch":126,"log_start":"145'5400","ondisk_log_start":"145'5400","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"190'14394","last_scrub_stamp":"2020-09-08T15:40:35.304761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:35.304761+0000","log_size":10079,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":210960384,"num_objects":3219,"num_object_clones":0,"num_object_copies":6438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15479,"num_write_kb":598336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"195'15276","reported_seq":"15355","reported_epoch":"195","state":"active+clean","last_fresh":"2020-09-08T15:40:51.640966+0000","last_change":"2020-09-08T15:39:12.946359+0000","last_active":"2020-09-08T15:40:51.640966+0000","last_peered":"2020-09-08T15:40:51.640966+0000","last_clean":"2020-09-08T15:40:51.640966+0000","last_became_active":"2020-09-08T15:37:29.875715+0000","last_became_peered":"2020-09-08T15:37:29.875715+0000","last_unstale":"2020-09-08T15:40:51.640966+0000","last_undegraded":"2020-09-08T15:40:51.640966+0000","last_fullsized":"2020-09-08T15:40:51.640966+0000","mapping_epoch":126,"log_start":"143'5200","ondisk_log_start":"143'5200","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"165'9189","last_scrub_stamp":"2020-09-08T15:39:12.946278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:39:12.946278+0000","log_size":10076,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":203292672,"num_objects":3102,"num_object_clones":0,"num_object_copies":6204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15276,"num_write_kb":588096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"99'6","reported_seq":"208","reported_epoch":"194","state":"active+clean","last_fresh":"2020-09-08T15:40:46.661020+0000","last_change":"2020-09-08T15:39:42.676434+0000","last_active":"2020-09-08T15:40:46.661020+0000","last_peered":"2020-09-08T15:40:46.661020+0000","last_clean":"2020-09-08T15:40:46.661020+0000","last_became_active":"2020-09-08T15:34:43.031697+0000","last_became_peered":"2020-09-08T15:34:43.031697+0000","last_unstale":"2020-09-08T15:40:46.661020+0000","last_undegraded":"2020-09-08T15:40:46.661020+0000","last_fullsized":"2020-09-08T15:40:46.661020+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":2,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'6","last_scrub_stamp":"2020-09-08T15:39:42.676360+0000","last_deep_scrub":"99'6","last_deep_scrub_stamp":"2020-09-08T15:35:23.132465+0000","last_clean_scrub_stamp":"2020-09-08T15:39:42.676360+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"195'15330","reported_seq":"15412","reported_epoch":"195","state":"active+clean","last_fresh":"2020-09-08T15:40:53.601071+0000","last_change":"2020-09-08T15:40:14.389420+0000","last_active":"2020-09-08T15:40:53.601071+0000","last_peered":"2020-09-08T15:40:53.601071+0000","last_clean":"2020-09-08T15:40:53.601071+0000","last_became_active":"2020-09-08T15:37:30.094918+0000","last_became_peered":"2020-09-08T15:37:30.094918+0000","last_unstale":"2020-09-08T15:40:53.601071+0000","last_undegraded":"2020-09-08T15:40:53.601071+0000","last_fullsized":"2020-09-08T15:40:53.601071+0000","mapping_epoch":126,"log_start":"145'5300","ondisk_log_start":"145'5300","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'12937","last_scrub_stamp":"2020-09-08T15:40:14.389373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:14.389373+0000","log_size":10030,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":208011264,"num_objects":3174,"num_object_clones":0,"num_object_copies":6348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15330,"num_write_kb":592128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"195'15057","reported_seq":"15136","reported_epoch":"195","state":"active+clean","last_fresh":"2020-09-08T15:40:51.794762+0000","last_change":"2020-09-08T15:39:31.634079+0000","last_active":"2020-09-08T15:40:51.794762+0000","last_peered":"2020-09-08T15:40:51.794762+0000","last_clean":"2020-09-08T15:40:51.794762+0000","last_became_active":"2020-09-08T15:37:30.090699+0000","last_became_peered":"2020-09-08T15:37:30.090699+0000","last_unstale":"2020-09-08T15:40:51.794762+0000","last_undegraded":"2020-09-08T15:40:51.794762+0000","last_fullsized":"2020-09-08T15:40:51.794762+0000","mapping_epoch":126,"log_start":"143'5000","ondisk_log_start":"143'5000","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"170'10244","last_scrub_stamp":"2020-09-08T15:39:31.634020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:39:31.634020+0000","log_size":10057,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":210698240,"num_objects":3215,"num_object_clones":0,"num_object_copies":6430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15057,"num_write_kb":584704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.6","version":"195'15217","reported_seq":"15301","reported_epoch":"195","state":"active+clean","last_fresh":"2020-09-08T15:40:51.749503+0000","last_change":"2020-09-08T15:40:26.436714+0000","last_active":"2020-09-08T15:40:51.749503+0000","last_peered":"2020-09-08T15:40:51.749503+0000","last_clean":"2020-09-08T15:40:51.749503+0000","last_became_active":"2020-09-08T15:37:29.889471+0000","last_became_peered":"2020-09-08T15:37:29.889471+0000","last_unstale":"2020-09-08T15:40:51.749503+0000","last_undegraded":"2020-09-08T15:40:51.749503+0000","last_fullsized":"2020-09-08T15:40:51.749503+0000","mapping_epoch":126,"log_start":"144'5200","ondisk_log_start":"144'5200","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'13713","last_scrub_stamp":"2020-09-08T15:40:26.436603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:26.436603+0000","log_size":10017,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":210829312,"num_objects":3217,"num_object_clones":0,"num_object_copies":6434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15217,"num_write_kb":589888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"195'15273","reported_seq":"15355","reported_epoch":"195","state":"active+clean","last_fresh":"2020-09-08T15:40:51.660023+0000","last_change":"2020-09-08T15:40:23.781832+0000","last_active":"2020-09-08T15:40:51.660023+0000","last_peered":"2020-09-08T15:40:51.660023+0000","last_clean":"2020-09-08T15:40:51.660023+0000","last_became_active":"2020-09-08T15:37:30.095291+0000","last_became_peered":"2020-09-08T15:37:30.095291+0000","last_unstale":"2020-09-08T15:40:51.660023+0000","last_undegraded":"2020-09-08T15:40:51.660023+0000","last_fullsized":"2020-09-08T15:40:51.660023+0000","mapping_epoch":126,"log_start":"144'5200","ondisk_log_start":"144'5200","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'13605","last_scrub_stamp":"2020-09-08T15:40:23.781740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:23.781740+0000","log_size":10073,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":213319680,"num_objects":3255,"num_object_clones":0,"num_object_copies":6510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15273,"num_write_kb":592896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.8","version":"195'15256","reported_seq":"15335","reported_epoch":"195","state":"active+clean","last_fresh":"2020-09-08T15:40:53.609548+0000","last_change":"2020-09-08T15:39:15.907759+0000","last_active":"2020-09-08T15:40:53.609548+0000","last_peered":"2020-09-08T15:40:53.609548+0000","last_clean":"2020-09-08T15:40:53.609548+0000","last_became_active":"2020-09-08T15:37:29.881507+0000","last_became_peered":"2020-09-08T15:37:29.881507+0000","last_unstale":"2020-09-08T15:40:53.609548+0000","last_undegraded":"2020-09-08T15:40:53.609548+0000","last_fullsized":"2020-09-08T15:40:53.609548+0000","mapping_epoch":126,"log_start":"144'5200","ondisk_log_start":"144'5200","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"165'9281","last_scrub_stamp":"2020-09-08T15:39:15.907669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:39:15.907669+0000","log_size":10056,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":200409088,"num_objects":3058,"num_object_clones":0,"num_object_copies":6116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15256,"num_write_kb":586048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9],"acting":[7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.9","version":"195'15078","reported_seq":"15160","reported_epoch":"195","state":"active+clean","last_fresh":"2020-09-08T15:40:51.761400+0000","last_change":"2020-09-08T15:40:31.090354+0000","last_active":"2020-09-08T15:40:51.761400+0000","last_peered":"2020-09-08T15:40:51.761400+0000","last_clean":"2020-09-08T15:40:51.761400+0000","last_became_active":"2020-09-08T15:37:29.879357+0000","last_became_peered":"2020-09-08T15:37:29.879357+0000","last_unstale":"2020-09-08T15:40:51.761400+0000","last_undegraded":"2020-09-08T15:40:51.761400+0000","last_fullsized":"2020-09-08T15:40:51.761400+0000","mapping_epoch":126,"log_start":"143'5000","ondisk_log_start":"143'5000","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'13830","last_scrub_stamp":"2020-09-08T15:40:31.090293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:31.090293+0000","log_size":10078,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":212992000,"num_objects":3250,"num_object_clones":0,"num_object_copies":6500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15078,"num_write_kb":586496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"195'15362","reported_seq":"15444","reported_epoch":"195","state":"active+clean","last_fresh":"2020-09-08T15:40:52.740542+0000","last_change":"2020-09-08T15:40:30.575755+0000","last_active":"2020-09-08T15:40:52.740542+0000","last_peered":"2020-09-08T15:40:52.740542+0000","last_clean":"2020-09-08T15:40:52.740542+0000","last_became_active":"2020-09-08T15:37:30.094030+0000","last_became_peered":"2020-09-08T15:37:30.094030+0000","last_unstale":"2020-09-08T15:40:52.740542+0000","last_undegraded":"2020-09-08T15:40:52.740542+0000","last_fullsized":"2020-09-08T15:40:52.740542+0000","mapping_epoch":126,"log_start":"145'5300","ondisk_log_start":"145'5300","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'13996","last_scrub_stamp":"2020-09-08T15:40:30.575701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:30.575701+0000","log_size":10062,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":205914112,"num_objects":3142,"num_object_clones":0,"num_object_copies":6284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15362,"num_write_kb":592128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3],"acting":[11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.b","version":"195'15236","reported_seq":"15314","reported_epoch":"195","state":"active+clean","last_fresh":"2020-09-08T15:40:51.796727+0000","last_change":"2020-09-08T15:39:19.264726+0000","last_active":"2020-09-08T15:40:51.796727+0000","last_peered":"2020-09-08T15:40:51.796727+0000","last_clean":"2020-09-08T15:40:51.796727+0000","last_became_active":"2020-09-08T15:37:29.879505+0000","last_became_peered":"2020-09-08T15:37:29.879505+0000","last_unstale":"2020-09-08T15:40:51.796727+0000","last_undegraded":"2020-09-08T15:40:51.796727+0000","last_fullsized":"2020-09-08T15:40:51.796727+0000","mapping_epoch":126,"log_start":"144'5200","ondisk_log_start":"144'5200","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'9555","last_scrub_stamp":"2020-09-08T15:39:19.264670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:39:19.264670+0000","log_size":10036,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":212074496,"num_objects":3236,"num_object_clones":0,"num_object_copies":6472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15236,"num_write_kb":591104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.c","version":"195'15208","reported_seq":"15290","reported_epoch":"195","state":"active+clean","last_fresh":"2020-09-08T15:40:52.716174+0000","last_change":"2020-09-08T15:40:15.718317+0000","last_active":"2020-09-08T15:40:52.716174+0000","last_peered":"2020-09-08T15:40:52.716174+0000","last_clean":"2020-09-08T15:40:52.716174+0000","last_became_active":"2020-09-08T15:37:30.093315+0000","last_became_peered":"2020-09-08T15:37:30.093315+0000","last_unstale":"2020-09-08T15:40:52.716174+0000","last_undegraded":"2020-09-08T15:40:52.716174+0000","last_fullsized":"2020-09-08T15:40:52.716174+0000","mapping_epoch":126,"log_start":"144'5200","ondisk_log_start":"144'5200","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'12953","last_scrub_stamp":"2020-09-08T15:40:15.718177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:15.718177+0000","log_size":10008,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":203030528,"num_objects":3098,"num_object_clones":0,"num_object_copies":6196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15208,"num_write_kb":585792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0],"acting":[11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.d","version":"195'15328","reported_seq":"15408","reported_epoch":"195","state":"active+clean","last_fresh":"2020-09-08T15:40:52.814694+0000","last_change":"2020-09-08T15:39:13.390802+0000","last_active":"2020-09-08T15:40:52.814694+0000","last_peered":"2020-09-08T15:40:52.814694+0000","last_clean":"2020-09-08T15:40:52.814694+0000","last_became_active":"2020-09-08T15:37:30.092910+0000","last_became_peered":"2020-09-08T15:37:30.092910+0000","last_unstale":"2020-09-08T15:40:52.814694+0000","last_undegraded":"2020-09-08T15:40:52.814694+0000","last_fullsized":"2020-09-08T15:40:52.814694+0000","mapping_epoch":126,"log_start":"145'5300","ondisk_log_start":"145'5300","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"165'9318","last_scrub_stamp":"2020-09-08T15:39:13.390657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:39:13.390657+0000","log_size":10028,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":216858648,"num_objects":3310,"num_object_clones":0,"num_object_copies":6620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3310,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15328,"num_write_kb":596353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.e","version":"195'15451","reported_seq":"15532","reported_epoch":"195","state":"active+clean","last_fresh":"2020-09-08T15:40:51.789818+0000","last_change":"2020-09-08T15:39:18.552789+0000","last_active":"2020-09-08T15:40:51.789818+0000","last_peered":"2020-09-08T15:40:51.789818+0000","last_clean":"2020-09-08T15:40:51.789818+0000","last_became_active":"2020-09-08T15:37:30.093099+0000","last_became_peered":"2020-09-08T15:37:30.093099+0000","last_unstale":"2020-09-08T15:40:51.789818+0000","last_undegraded":"2020-09-08T15:40:51.789818+0000","last_fullsized":"2020-09-08T15:40:51.789818+0000","mapping_epoch":126,"log_start":"145'5400","ondisk_log_start":"145'5400","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'9707","last_scrub_stamp":"2020-09-08T15:39:18.552716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:39:18.552716+0000","log_size":10051,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":217513984,"num_objects":3319,"num_object_clones":0,"num_object_copies":6638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15451,"num_write_kb":600640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3360227352,"num_objects":51274,"num_object_clones":0,"num_object_copies":102548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51274,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":243962,"num_write_kb":9447489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6720987136,"data_stored":6720978992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160762,"ondisk_log_size":160762,"up":32,"acting":32,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":2,"acting":2,"num_store_stats":7}],"osd_stats":[{"osd":11,"up_from":0,"seq":78,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":653028,"kb_used_data":604828,"kb_used_omap":0,"kb_used_meta":48192,"kb_avail":93092124,"statfs":{"total":95995035648,"available":95326334976,"internally_reserved":0,"allocated":619343872,"data_stored":619092814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":49348608},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:39:53 2020","interfaces":[{"interface":"back","average":{"1min":2.3860000000000001,"5min":2.0529999999999999,"15min":1.917},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":6.7679999999999998,"5min":6.7679999999999998,"15min":6.7679999999999998},"last":1.597},{"interface":"front","average":{"1min":2.8690000000000002,"5min":2.9990000000000001,"15min":3.2069999999999999},"min":{"1min":1.081,"5min":1.022,"15min":1.022},"max":{"1min":7.2889999999999997,"5min":7.2889999999999997,"15min":7.2889999999999997},"last":1.907}]},{"osd":1,"last update":"Tue Sep 8 15:39:53 2020","interfaces":[{"interface":"back","average":{"1min":3.9420000000000002,"5min":2.5419999999999998,"15min":2.2200000000000002},"min":{"1min":0.80700000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":34.723999999999997,"5min":34.723999999999997,"15min":34.723999999999997},"last":1.0920000000000001},{"interface":"front","average":{"1min":5.3680000000000003,"5min":3.02,"15min":2.7679999999999998},"min":{"1min":0.92700000000000005,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":38.625999999999998,"5min":38.625999999999998,"15min":38.625999999999998},"last":1.7250000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:39:53 2020","interfaces":[{"interface":"back","average":{"1min":5.4500000000000002,"5min":2.859,"15min":2.3410000000000002},"min":{"1min":0.83899999999999997,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":36.637999999999998,"5min":36.637999999999998,"15min":36.637999999999998},"last":2.1850000000000001},{"interface":"front","average":{"1min":4.798,"5min":3.3620000000000001,"15min":3.032},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":36.694000000000003,"5min":36.694000000000003,"15min":36.694000000000003},"last":1.3620000000000001}]},{"osd":3,"last update":"Tue Sep 8 15:39:53 2020","interfaces":[{"interface":"back","average":{"1min":2.1899999999999999,"5min":4.0599999999999996,"15min":2.9060000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":6.3380000000000001,"5min":147.49100000000001,"15min":147.49100000000001},"last":1.4770000000000001},{"interface":"front","average":{"1min":3.6379999999999999,"5min":4.5650000000000004,"15min":3.613},"min":{"1min":1.252,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":10.131,"5min":150.32400000000001,"15min":150.32400000000001},"last":2.0219999999999998}]},{"osd":4,"last update":"Tue Sep 8 15:39:53 2020","interfaces":[{"interface":"back","average":{"1min":2.8220000000000001,"5min":3.1379999999999999,"15min":3.452},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":8.0289999999999999,"5min":8.0289999999999999,"15min":8.0289999999999999},"last":2.1360000000000001},{"interface":"front","average":{"1min":2.9209999999999998,"5min":2.7650000000000001,"15min":2.7919999999999998},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":7.7679999999999998,"5min":7.7679999999999998,"15min":7.7679999999999998},"last":1.6240000000000001}]},{"osd":5,"last update":"Tue Sep 8 15:39:53 2020","interfaces":[{"interface":"back","average":{"1min":2.3719999999999999,"5min":3.0680000000000001,"15min":3.3130000000000002},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":6.8499999999999996,"5min":6.8499999999999996,"15min":6.8499999999999996},"last":1.6719999999999999},{"interface":"front","average":{"1min":3.0510000000000002,"5min":3.4049999999999998,"15min":3.8010000000000002},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":8.5289999999999999,"5min":8.5289999999999999,"15min":8.5289999999999999},"last":2.3639999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:39:53 2020","interfaces":[{"interface":"back","average":{"1min":2.3460000000000001,"5min":3.0720000000000001,"15min":3.3140000000000001},"min":{"1min":1.196,"5min":1.036,"15min":1.036},"max":{"1min":6.6210000000000004,"5min":7.8479999999999999,"15min":7.8479999999999999},"last":1.8580000000000001},{"interface":"front","average":{"1min":3.1669999999999998,"5min":3.4329999999999998,"15min":3.5579999999999998},"min":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"max":{"1min":8.2110000000000003,"5min":8.2110000000000003,"15min":8.2110000000000003},"last":2.2509999999999999}]},{"osd":7,"last update":"Tue Sep 8 15:39:53 2020","interfaces":[{"interface":"back","average":{"1min":3.4889999999999999,"5min":3.496,"15min":3.6720000000000002},"min":{"1min":1.22,"5min":1.22,"15min":1.22},"max":{"1min":8.4610000000000003,"5min":8.4610000000000003,"15min":8.4610000000000003},"last":2.0569999999999999},{"interface":"front","average":{"1min":2.532,"5min":3.5369999999999999,"15min":3.7730000000000001},"min":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"max":{"1min":6.492,"5min":7.9420000000000002,"15min":7.9420000000000002},"last":1.97}]},{"osd":8,"last update":"Tue Sep 8 15:39:53 2020","interfaces":[{"interface":"back","average":{"1min":3.5019999999999998,"5min":3.7349999999999999,"15min":4.0030000000000001},"min":{"1min":1.744,"5min":1.335,"15min":1.335},"max":{"1min":8.5370000000000008,"5min":8.5370000000000008,"15min":8.5370000000000008},"last":6.7770000000000001},{"interface":"front","average":{"1min":4.133,"5min":3.7320000000000002,"15min":3.8479999999999999},"min":{"1min":1.575,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":8.9580000000000002,"5min":8.9580000000000002,"15min":8.9580000000000002},"last":6.8499999999999996}]},{"osd":10,"last update":"Tue Sep 8 15:39:53 2020","interfaces":[{"interface":"back","average":{"1min":3.3180000000000001,"5min":3.8399999999999999,"15min":4.2779999999999996},"min":{"1min":1.498,"5min":1.498,"15min":1.498},"max":{"1min":8.3469999999999995,"5min":8.3469999999999995,"15min":8.3469999999999995},"last":5.5039999999999996},{"interface":"front","average":{"1min":3.7509999999999999,"5min":3.8260000000000001,"15min":4.1360000000000001},"min":{"1min":1.5880000000000001,"5min":1.107,"15min":1.107},"max":{"1min":8.6600000000000001,"5min":8.6600000000000001,"15min":8.6600000000000001},"last":2.2989999999999999}]}]},{"osd":10,"up_from":0,"seq":85,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":463652,"kb_used_data":417180,"kb_used_omap":0,"kb_used_meta":46464,"kb_avail":93281500,"statfs":{"total":95995035648,"available":95520256000,"internally_reserved":0,"allocated":427192320,"data_stored":426941262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":47579136},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7330000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":5.9669999999999996,"5min":2.8799999999999999,"15min":2.5659999999999998},"min":{"1min":0.66900000000000004,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":64.471999999999994,"5min":64.471999999999994,"15min":64.471999999999994},"last":3.6779999999999999},{"interface":"front","average":{"1min":6.6349999999999998,"5min":3.3250000000000002,"15min":2.8130000000000002},"min":{"1min":0.70099999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":64.707999999999998,"5min":64.707999999999998,"15min":64.707999999999998},"last":0.73499999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":4.0949999999999998,"5min":2.593,"15min":2.363},"min":{"1min":0.80800000000000005,"5min":0.745,"15min":0.745},"max":{"1min":40.453000000000003,"5min":40.453000000000003,"15min":40.453000000000003},"last":1.071},{"interface":"front","average":{"1min":3.7730000000000001,"5min":2.4500000000000002,"15min":2.6970000000000001},"min":{"1min":0.88900000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":37.209000000000003,"5min":37.209000000000003,"15min":37.209000000000003},"last":1.169}]},{"osd":3,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":3.7250000000000001,"5min":4.5549999999999997,"15min":3.04},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":37.298999999999999,"5min":232.87299999999999,"15min":232.87299999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":4.1539999999999999,"5min":5.1710000000000003,"15min":3.7829999999999999},"min":{"1min":1.0760000000000001,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":36.137,"5min":237.75800000000001,"15min":237.75800000000001},"last":1.02}]},{"osd":4,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":3.5009999999999999,"5min":2.859,"15min":3.2650000000000001},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":38.237000000000002,"5min":38.237000000000002,"15min":38.237000000000002},"last":2.2170000000000001},{"interface":"front","average":{"1min":3.6179999999999999,"5min":2.855,"15min":3.0089999999999999},"min":{"1min":1.0009999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":37.786000000000001,"5min":37.786000000000001,"15min":37.786000000000001},"last":0.78600000000000003}]},{"osd":5,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":3.8879999999999999,"5min":3.0750000000000002,"15min":3.448},"min":{"1min":0.996,"5min":0.996,"15min":0.996},"max":{"1min":36.493000000000002,"5min":36.493000000000002,"15min":36.493000000000002},"last":1.298},{"interface":"front","average":{"1min":3.859,"5min":3.25,"15min":3.657},"min":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"max":{"1min":38.957999999999998,"5min":38.957999999999998,"15min":38.957999999999998},"last":1.833}]},{"osd":6,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":3.8410000000000002,"5min":3.1150000000000002,"15min":3.4289999999999998},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":38.537999999999997,"5min":38.537999999999997,"15min":38.537999999999997},"last":1.0960000000000001},{"interface":"front","average":{"1min":4.0890000000000004,"5min":3.3140000000000001,"15min":3.71},"min":{"1min":1.266,"5min":1.1950000000000001,"15min":1.1950000000000001},"max":{"1min":37.698999999999998,"5min":37.698999999999998,"15min":37.698999999999998},"last":1.6870000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":4.319,"5min":3.4369999999999998,"15min":3.778},"min":{"1min":1.2410000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"max":{"1min":39.100000000000001,"5min":39.100000000000001,"15min":39.100000000000001},"last":1.2030000000000001},{"interface":"front","average":{"1min":4.532,"5min":3.5470000000000002,"15min":3.9289999999999998},"min":{"1min":1.3440000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"max":{"1min":38.131999999999998,"5min":38.131999999999998,"15min":38.131999999999998},"last":1.4350000000000001}]},{"osd":8,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":4.4509999999999996,"5min":3.7679999999999998,"15min":4.0419999999999998},"min":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"max":{"1min":42.031999999999996,"5min":42.031999999999996,"15min":42.031999999999996},"last":2.0179999999999998},{"interface":"front","average":{"1min":4.4669999999999996,"5min":3.7850000000000001,"15min":3.9430000000000001},"min":{"1min":1.1850000000000001,"5min":1.1639999999999999,"15min":1.1639999999999999},"max":{"1min":39.512999999999998,"5min":39.512999999999998,"15min":39.512999999999998},"last":1.3260000000000001}]},{"osd":9,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":4.7549999999999999,"5min":3.9790000000000001,"15min":4.2720000000000002},"min":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":42.603999999999999,"5min":42.603999999999999,"15min":42.603999999999999},"last":1.4830000000000001},{"interface":"front","average":{"1min":4.5919999999999996,"5min":3.8650000000000002,"15min":4.0899999999999999},"min":{"1min":1.5189999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"max":{"1min":40.029000000000003,"5min":40.029000000000003,"15min":40.029000000000003},"last":1.4570000000000001}]},{"osd":11,"last update":"Tue Sep 8 15:39:58 2020","interfaces":[{"interface":"back","average":{"1min":5.6070000000000002,"5min":4.391,"15min":4.3890000000000002},"min":{"1min":1.6619999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"max":{"1min":39.271000000000001,"5min":39.271000000000001,"15min":39.271000000000001},"last":4.5679999999999996},{"interface":"front","average":{"1min":4.6269999999999998,"5min":4.1500000000000004,"15min":4.3319999999999999},"min":{"1min":1.4650000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"max":{"1min":42.329999999999998,"5min":42.329999999999998,"15min":42.329999999999998},"last":1.5940000000000001}]}]},{"osd":9,"up_from":0,"seq":88,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":680552,"kb_used_data":615968,"kb_used_omap":0,"kb_used_meta":64512,"kb_avail":93064600,"statfs":{"total":95995035648,"available":95298150400,"internally_reserved":0,"allocated":630751232,"data_stored":630496102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66060288},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:40:05 2020","interfaces":[{"interface":"back","average":{"1min":1.8240000000000001,"5min":2.0470000000000002,"15min":1.921},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":5.3959999999999999,"5min":9.0809999999999995,"15min":9.0809999999999995},"last":2.2189999999999999},{"interface":"front","average":{"1min":1.393,"5min":1.8779999999999999,"15min":1.8939999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.597,"5min":5.1440000000000001,"15min":5.1440000000000001},"last":0.67500000000000004}]},{"osd":1,"last update":"Tue Sep 8 15:40:05 2020","interfaces":[{"interface":"back","average":{"1min":2.0510000000000002,"5min":2.1499999999999999,"15min":2.0150000000000001},"min":{"1min":0.97499999999999998,"5min":0.755,"15min":0.755},"max":{"1min":3.8769999999999998,"5min":9.5519999999999996,"15min":9.5519999999999996},"last":0.70099999999999996},{"interface":"front","average":{"1min":2.3799999999999999,"5min":2.4260000000000002,"15min":2.3319999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":6.6230000000000002,"5min":9.4250000000000007,"15min":9.4250000000000007},"last":0.94499999999999995}]},{"osd":2,"last update":"Tue Sep 8 15:40:05 2020","interfaces":[{"interface":"back","average":{"1min":2.0089999999999999,"5min":2.3570000000000002,"15min":2.4340000000000002},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":6.2169999999999996,"5min":6.2169999999999996,"15min":6.2169999999999996},"last":1.7729999999999999},{"interface":"front","average":{"1min":1.9259999999999999,"5min":2.4409999999999998,"15min":2.3330000000000002},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":5.7060000000000004,"5min":9.4770000000000003,"15min":9.4770000000000003},"last":2.3319999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:40:05 2020","interfaces":[{"interface":"back","average":{"1min":2.1629999999999998,"5min":2.6789999999999998,"15min":2.4529999999999998},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":4.7699999999999996,"5min":24.562000000000001,"15min":24.562000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":2.3359999999999999,"5min":2.8780000000000001,"15min":2.722},"min":{"1min":1.1799999999999999,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":5.1749999999999998,"5min":23.184000000000001,"15min":23.184000000000001},"last":1.1539999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:40:05 2020","interfaces":[{"interface":"back","average":{"1min":2.177,"5min":2.7370000000000001,"15min":2.702},"min":{"1min":1.087,"5min":1.087,"15min":1.087},"max":{"1min":6.7400000000000002,"5min":6.7400000000000002,"15min":6.7400000000000002},"last":6.524},{"interface":"front","average":{"1min":2.2250000000000001,"5min":2.9769999999999999,"15min":2.9889999999999999},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":3.7850000000000001,"5min":10.201000000000001,"15min":10.201000000000001},"last":4.383}]},{"osd":5,"last update":"Tue Sep 8 15:40:05 2020","interfaces":[{"interface":"back","average":{"1min":2.2919999999999998,"5min":2.9449999999999998,"15min":3.085},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":5.8310000000000004,"5min":11.097,"15min":11.097},"last":4.4359999999999999},{"interface":"front","average":{"1min":2.6560000000000001,"5min":3.097,"15min":3.1429999999999998},"min":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0589999999999999},"max":{"1min":5.8780000000000001,"5min":9.9280000000000008,"15min":9.9280000000000008},"last":1.3779999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:40:05 2020","interfaces":[{"interface":"back","average":{"1min":2.2909999999999999,"5min":3.1819999999999999,"15min":3.194},"min":{"1min":1.2809999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":5.9509999999999996,"5min":10.798,"15min":10.798},"last":2.4830000000000001},{"interface":"front","average":{"1min":2.5259999999999998,"5min":3.2109999999999999,"15min":3.2240000000000002},"min":{"1min":1.242,"5min":1.242,"15min":1.242},"max":{"1min":6.8730000000000002,"5min":11.766,"15min":11.766},"last":5.5}]},{"osd":7,"last update":"Tue Sep 8 15:40:05 2020","interfaces":[{"interface":"back","average":{"1min":2.6070000000000002,"5min":3.4449999999999998,"15min":3.4089999999999998},"min":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"max":{"1min":4.1890000000000001,"5min":10.31,"15min":10.31},"last":5.907},{"interface":"front","average":{"1min":2.8340000000000001,"5min":3.6520000000000001,"15min":3.6240000000000001},"min":{"1min":1.593,"5min":1.593,"15min":1.593},"max":{"1min":4.3280000000000003,"5min":10.837999999999999,"15min":10.837999999999999},"last":6.016}]},{"osd":8,"last update":"Tue Sep 8 15:40:05 2020","interfaces":[{"interface":"back","average":{"1min":3.2650000000000001,"5min":3.5470000000000002,"15min":3.2909999999999999},"min":{"1min":1.351,"5min":1.351,"15min":1.3180000000000001},"max":{"1min":11,"5min":11,"15min":11},"last":5.6420000000000003},{"interface":"front","average":{"1min":3.3380000000000001,"5min":3.754,"15min":3.407},"min":{"1min":1.373,"5min":1.373,"15min":1.373},"max":{"1min":8.9580000000000002,"5min":10.977,"15min":10.977},"last":1.262}]},{"osd":10,"last update":"Tue Sep 8 15:40:24 2020","interfaces":[{"interface":"back","average":{"1min":2.8879999999999999,"5min":3.3780000000000001,"15min":3.7469999999999999},"min":{"1min":1.5329999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"max":{"1min":8.8200000000000003,"5min":8.8200000000000003,"15min":8.8200000000000003},"last":5.7800000000000002},{"interface":"front","average":{"1min":2.0790000000000002,"5min":3.3620000000000001,"15min":3.6459999999999999},"min":{"1min":1.196,"5min":1.196,"15min":1.196},"max":{"1min":3.282,"5min":11.625,"15min":11.625},"last":2.629}]}]},{"osd":8,"up_from":0,"seq":93,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":900580,"kb_used_data":838684,"kb_used_omap":0,"kb_used_meta":61888,"kb_avail":92844572,"statfs":{"total":95995035648,"available":95072841728,"internally_reserved":0,"allocated":858812416,"data_stored":858561358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":63373312},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":2.0009999999999999,"5min":2.141,"15min":2.1709999999999998},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":8.6069999999999993,"5min":8.6069999999999993,"15min":8.6069999999999993},"last":11.154999999999999},{"interface":"front","average":{"1min":3.5049999999999999,"5min":2.472,"15min":2.4460000000000002},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":13.773999999999999,"5min":13.773999999999999,"15min":13.773999999999999},"last":1.468}]},{"osd":1,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.492,"5min":2.0659999999999998,"15min":1.8839999999999999},"min":{"1min":1.4690000000000001,"5min":1.137,"15min":1.137},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":1.4350000000000001},{"interface":"front","average":{"1min":2.7200000000000002,"5min":2.427,"15min":2.347},"min":{"1min":1.351,"5min":1.046,"15min":1.046},"max":{"1min":4.444,"5min":4.444,"15min":4.444},"last":1.8600000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":10.316000000000001,"5min":4.3819999999999997,"15min":2.677},"min":{"1min":0.79500000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":145.893,"5min":145.893,"15min":145.893},"last":5.6890000000000001},{"interface":"front","average":{"1min":9.9939999999999998,"5min":3.9420000000000002,"15min":2.9580000000000002},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":153.91900000000001,"5min":153.91900000000001,"15min":153.91900000000001},"last":12.141}]},{"osd":3,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":2.5739999999999998,"5min":4.0869999999999997,"15min":3.1779999999999999},"min":{"1min":0.73799999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":9.2989999999999995,"5min":125.111,"15min":125.111},"last":11.539999999999999},{"interface":"front","average":{"1min":3.1190000000000002,"5min":3.952,"15min":2.9060000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":13.505000000000001,"5min":126.124,"15min":126.124},"last":1.8939999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":2.5329999999999999,"5min":2.9830000000000001,"15min":2.907},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":9.3800000000000008,"5min":9.7599999999999998,"15min":9.7599999999999998},"last":11.404999999999999},{"interface":"front","average":{"1min":3.343,"5min":2.7320000000000002,"15min":2.6110000000000002},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":13.999000000000001,"5min":13.999000000000001,"15min":13.999000000000001},"last":2.5790000000000002}]},{"osd":5,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":2.6720000000000002,"5min":3.2690000000000001,"15min":3.1890000000000001},"min":{"1min":0.90500000000000003,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":9.4399999999999995,"5min":9.4399999999999995,"15min":9.4399999999999995},"last":6.0119999999999996},{"interface":"front","average":{"1min":2.6600000000000001,"5min":3.323,"15min":3.3380000000000001},"min":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":9.6820000000000004,"5min":9.6820000000000004,"15min":9.6820000000000004},"last":11.885999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":3.343,"5min":3.552,"15min":3.3450000000000002},"min":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"max":{"1min":13.887,"5min":13.887,"15min":13.887},"last":2.363},{"interface":"front","average":{"1min":2.9550000000000001,"5min":2.9470000000000001,"15min":2.923},"min":{"1min":1.153,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":12.502000000000001,"5min":12.502000000000001,"15min":12.502000000000001},"last":11.775}]},{"osd":7,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":3.3119999999999998,"5min":3.7360000000000002,"15min":3.6469999999999998},"min":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"max":{"1min":9.4469999999999992,"5min":10.236000000000001,"15min":10.236000000000001},"last":11.661},{"interface":"front","average":{"1min":3.948,"5min":3.4289999999999998,"15min":3.2959999999999998},"min":{"1min":1.3620000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"max":{"1min":14.109999999999999,"5min":14.109999999999999,"15min":14.109999999999999},"last":2.2410000000000001}]},{"osd":9,"last update":"Tue Sep 8 15:40:04 2020","interfaces":[{"interface":"back","average":{"1min":4.0369999999999999,"5min":4.0270000000000001,"15min":3.5979999999999999},"min":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.157},"max":{"1min":13.506,"5min":13.506,"15min":13.506},"last":11.913},{"interface":"front","average":{"1min":3.2450000000000001,"5min":3.8849999999999998,"15min":3.4889999999999999},"min":{"1min":1.294,"5min":1.294,"15min":1.1919999999999999},"max":{"1min":9.5259999999999998,"5min":10.122999999999999,"15min":10.122999999999999},"last":12.052}]},{"osd":10,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":3.6379999999999999,"5min":3.3490000000000002,"15min":3.6070000000000002},"min":{"1min":1.423,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":13.656000000000001,"5min":13.656000000000001,"15min":13.656000000000001},"last":2.6819999999999999},{"interface":"front","average":{"1min":2.9729999999999999,"5min":3.8140000000000001,"15min":3.988},"min":{"1min":1.296,"5min":1.296,"15min":1.296},"max":{"1min":9.6649999999999991,"5min":11.051,"15min":11.051},"last":5.8150000000000004}]},{"osd":11,"last update":"Tue Sep 8 15:40:35 2020","interfaces":[{"interface":"back","average":{"1min":3.512,"5min":3.665,"15min":3.9239999999999999},"min":{"1min":1.214,"5min":1.214,"15min":1.214},"max":{"1min":12.646000000000001,"5min":12.646000000000001,"15min":12.646000000000001},"last":2.4700000000000002},{"interface":"front","average":{"1min":3.4830000000000001,"5min":4.4210000000000003,"15min":4.7939999999999996},"min":{"1min":1.248,"5min":1.248,"15min":1.248},"max":{"1min":10.667999999999999,"5min":11.186999999999999,"15min":11.186999999999999},"last":6.8129999999999997}]}]},{"osd":7,"up_from":0,"seq":99,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":659236,"kb_used_data":602588,"kb_used_omap":0,"kb_used_meta":56640,"kb_avail":93085916,"statfs":{"total":95995035648,"available":95319977984,"internally_reserved":0,"allocated":617050112,"data_stored":616799074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":57999360},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:02 2020","interfaces":[{"interface":"back","average":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"min":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"max":{"1min":4.0190000000000001,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":2.2189999999999999},{"interface":"front","average":{"1min":2.6259999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"min":{"1min":1.542,"5min":1.542,"15min":1.542},"max":{"1min":4.5229999999999997,"5min":4.5229999999999997,"15min":4.5229999999999997},"last":2.3079999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":2.133,"15min":2.2759999999999998},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":4.8600000000000003,"5min":6.6219999999999999,"15min":6.6219999999999999},"last":1.331},{"interface":"front","average":{"1min":1.802,"5min":2.1030000000000002,"15min":2.2869999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":5.3159999999999998,"5min":5.3159999999999998,"15min":5.3159999999999998},"last":1.371}]},{"osd":2,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":2.3879999999999999,"5min":2.5350000000000001,"15min":2.4860000000000002},"min":{"1min":1.0649999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":6.234,"5min":6.9779999999999998,"15min":6.9779999999999998},"last":3.9140000000000001},{"interface":"front","average":{"1min":2.569,"5min":2.4870000000000001,"15min":2.7509999999999999},"min":{"1min":1.335,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":6.6879999999999997,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":2.1280000000000001}]},{"osd":3,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":2.3109999999999999,"5min":2.899,"15min":2.9129999999999998},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":4.593,"5min":37.801000000000002,"15min":37.801000000000002},"last":3.4399999999999999},{"interface":"front","average":{"1min":2.661,"5min":3.6030000000000002,"15min":3.387},"min":{"1min":1.2430000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":12.099,"5min":45.284999999999997,"15min":45.284999999999997},"last":3.6520000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":1.794,"5min":2.5089999999999999,"15min":2.7989999999999999},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":3.1930000000000001,"5min":8.5779999999999994,"15min":8.5779999999999994},"last":1.976},{"interface":"front","average":{"1min":2.2320000000000002,"5min":3.004,"15min":3.2229999999999999},"min":{"1min":1.246,"5min":1.1319999999999999,"15min":1.1319999999999999},"max":{"1min":4.4009999999999998,"5min":7.3570000000000002,"15min":7.3570000000000002},"last":4.8390000000000004}]},{"osd":5,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":2.3149999999999999,"5min":2.956,"15min":3.464},"min":{"1min":1.228,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":4.5540000000000003,"5min":9.0800000000000001,"15min":9.0800000000000001},"last":2.7469999999999999},{"interface":"front","average":{"1min":2.3159999999999998,"5min":2.9449999999999998,"15min":3.339},"min":{"1min":1.208,"5min":1.1779999999999999,"15min":1.1779999999999999},"max":{"1min":4.2869999999999999,"5min":6.0229999999999997,"15min":6.0229999999999997},"last":2.3090000000000002}]},{"osd":6,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":2.5219999999999998,"5min":3.1749999999999998,"15min":3.7080000000000002},"min":{"1min":1.304,"5min":1.304,"15min":1.304},"max":{"1min":5.1210000000000004,"5min":8.8300000000000001,"15min":8.8300000000000001},"last":2.2650000000000001},{"interface":"front","average":{"1min":2.6259999999999999,"5min":3.359,"15min":3.7450000000000001},"min":{"1min":1.7,"5min":1.3939999999999999,"15min":1.3939999999999999},"max":{"1min":4.2910000000000004,"5min":7.681,"15min":7.681},"last":5.8700000000000001}]},{"osd":8,"last update":"Tue Sep 8 15:40:43 2020","interfaces":[{"interface":"back","average":{"1min":3.3420000000000001,"5min":3.5739999999999998,"15min":3.9729999999999999},"min":{"1min":1.5009999999999999,"5min":1.3340000000000001,"15min":1.3340000000000001},"max":{"1min":8.2260000000000009,"5min":8.2260000000000009,"15min":8.2260000000000009},"last":2.4460000000000002},{"interface":"front","average":{"1min":3.1840000000000002,"5min":3.665,"15min":4.0830000000000002},"min":{"1min":1.677,"5min":1.6319999999999999,"15min":1.6319999999999999},"max":{"1min":8.4220000000000006,"5min":9.2119999999999997,"15min":9.2119999999999997},"last":2.5049999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:40:01 2020","interfaces":[{"interface":"back","average":{"1min":3.2519999999999998,"5min":4.0720000000000001,"15min":4.54},"min":{"1min":1.9039999999999999,"5min":1.7170000000000001,"15min":1.7170000000000001},"max":{"1min":6.8140000000000001,"5min":7.476,"15min":7.476},"last":6.1130000000000004},{"interface":"front","average":{"1min":2.5619999999999998,"5min":3.5350000000000001,"15min":4.149},"min":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"max":{"1min":4.008,"5min":6.274,"15min":6.274},"last":2.581}]},{"osd":10,"last update":"Tue Sep 8 15:40:31 2020","interfaces":[{"interface":"back","average":{"1min":3.2040000000000002,"5min":3.7370000000000001,"15min":4.4119999999999999},"min":{"1min":2.0299999999999998,"5min":1.5309999999999999,"15min":1.5309999999999999},"max":{"1min":7.3739999999999997,"5min":9.5009999999999994,"15min":9.5009999999999994},"last":6.4790000000000001},{"interface":"front","average":{"1min":2.7810000000000001,"5min":3.8860000000000001,"15min":4.75},"min":{"1min":1.583,"5min":1.583,"15min":1.583},"max":{"1min":4.8079999999999998,"5min":7.5720000000000001,"15min":8.3650000000000002},"last":3.012}]},{"osd":11,"last update":"Tue Sep 8 15:40:50 2020","interfaces":[{"interface":"back","average":{"1min":3.1509999999999998,"5min":3.9089999999999998,"15min":4.5279999999999996},"min":{"1min":1.8109999999999999,"5min":1.7010000000000001,"15min":1.7010000000000001},"max":{"1min":7.0410000000000004,"5min":9.1790000000000003,"15min":9.1790000000000003},"last":3.2629999999999999},{"interface":"front","average":{"1min":3.9249999999999998,"5min":4.2869999999999999,"15min":4.9729999999999999},"min":{"1min":2.1819999999999999,"5min":1.756,"15min":1.756},"max":{"1min":7.181,"5min":7.7629999999999999,"15min":7.7629999999999999},"last":6.9029999999999996}]}]},{"osd":6,"up_from":0,"seq":103,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":243492,"kb_used_data":210908,"kb_used_omap":0,"kb_used_meta":32576,"kb_avail":93501660,"statfs":{"total":95995035648,"available":95745699840,"internally_reserved":0,"allocated":215969792,"data_stored":215718754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":33357824},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:39:54 2020","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.9039999999999999,"15min":1.877},"min":{"1min":0.94199999999999995,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.8809999999999998,"5min":3.8809999999999998,"15min":3.8809999999999998},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.984,"15min":1.966},"min":{"1min":1.0429999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.847,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":1.3160000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:39:54 2020","interfaces":[{"interface":"back","average":{"1min":2.3500000000000001,"5min":2.226,"15min":2.1899999999999999},"min":{"1min":0.98199999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":4.4409999999999998,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":2.2789999999999999,"5min":2.625,"15min":2.4820000000000002},"min":{"1min":0.98699999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":7.6849999999999996,"5min":7.6849999999999996,"15min":7.6849999999999996},"last":2.3919999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:39:54 2020","interfaces":[{"interface":"back","average":{"1min":2.2280000000000002,"5min":2.7210000000000001,"15min":2.4990000000000001},"min":{"1min":1.333,"5min":1.3240000000000001,"15min":1.3240000000000001},"max":{"1min":5.5039999999999996,"5min":5.5039999999999996,"15min":5.5039999999999996},"last":1.5780000000000001},{"interface":"front","average":{"1min":2.589,"5min":2.7999999999999998,"15min":2.5},"min":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":5.9299999999999997,"5min":5.9299999999999997,"15min":5.9299999999999997},"last":1.758}]},{"osd":3,"last update":"Tue Sep 8 15:39:54 2020","interfaces":[{"interface":"back","average":{"1min":2.5430000000000001,"5min":3.863,"15min":2.8860000000000001},"min":{"1min":1.2549999999999999,"5min":1.224,"15min":1.224},"max":{"1min":4.2240000000000002,"5min":109.65300000000001,"15min":109.65300000000001},"last":1.51},{"interface":"front","average":{"1min":2.3260000000000001,"5min":4.1269999999999998,"15min":3.2589999999999999},"min":{"1min":1.278,"5min":1.278,"15min":1.278},"max":{"1min":3.9830000000000001,"5min":109.51600000000001,"15min":109.51600000000001},"last":1.3640000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:39:54 2020","interfaces":[{"interface":"back","average":{"1min":2.129,"5min":2.9729999999999999,"15min":2.7370000000000001},"min":{"1min":1.4379999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":4.3079999999999998,"5min":4.9980000000000002,"15min":4.9980000000000002},"last":1.952},{"interface":"front","average":{"1min":2.3170000000000002,"5min":3.2120000000000002,"15min":2.9790000000000001},"min":{"1min":1.4630000000000001,"5min":1.052,"15min":1.052},"max":{"1min":3.8679999999999999,"5min":22.062000000000001,"15min":22.062000000000001},"last":1.712}]},{"osd":5,"last update":"Tue Sep 8 15:39:54 2020","interfaces":[{"interface":"back","average":{"1min":2.5640000000000001,"5min":3.3490000000000002,"15min":3.181},"min":{"1min":1.613,"5min":1.1879999999999999,"15min":1.1879999999999999},"max":{"1min":4.5979999999999999,"5min":5.8460000000000001,"15min":6.1269999999999998},"last":2.262},{"interface":"front","average":{"1min":2.9340000000000002,"5min":3.7250000000000001,"15min":3.3159999999999998},"min":{"1min":2.0249999999999999,"5min":1.149,"15min":1.149},"max":{"1min":7.1120000000000001,"5min":21.228999999999999,"15min":21.228999999999999},"last":2.359}]},{"osd":7,"last update":"Tue Sep 8 15:40:14 2020","interfaces":[{"interface":"back","average":{"1min":2.7200000000000002,"5min":3.5979999999999999,"15min":3.6699999999999999},"min":{"1min":1.196,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":4.665,"5min":22.911999999999999,"15min":22.911999999999999},"last":2.5920000000000001},{"interface":"front","average":{"1min":2.8769999999999998,"5min":3.9950000000000001,"15min":3.855},"min":{"1min":1.887,"5min":1.5609999999999999,"15min":1.5609999999999999},"max":{"1min":5.032,"5min":26.056000000000001,"15min":26.056000000000001},"last":2.746}]},{"osd":8,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":2.71,"5min":3.4449999999999998,"15min":3.7709999999999999},"min":{"1min":1.609,"5min":1.454,"15min":1.454},"max":{"1min":3.9140000000000001,"5min":6.3479999999999999,"15min":6.3479999999999999},"last":1.893},{"interface":"front","average":{"1min":3.0219999999999998,"5min":3.472,"15min":3.952},"min":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"max":{"1min":8.6850000000000005,"5min":8.6850000000000005,"15min":8.6850000000000005},"last":2.206}]},{"osd":9,"last update":"Tue Sep 8 15:40:03 2020","interfaces":[{"interface":"back","average":{"1min":3.3079999999999998,"5min":4.2060000000000004,"15min":4.3499999999999996},"min":{"1min":2.165,"5min":1.5569999999999999,"15min":1.5569999999999999},"max":{"1min":7.1020000000000003,"5min":21.716000000000001,"15min":21.716000000000001},"last":2.101},{"interface":"front","average":{"1min":2.9710000000000001,"5min":3.8530000000000002,"15min":4.298},"min":{"1min":1.736,"5min":1.736,"15min":1.736},"max":{"1min":7.0250000000000004,"5min":7.0250000000000004,"15min":7.0250000000000004},"last":2.452}]},{"osd":10,"last update":"Tue Sep 8 15:40:27 2020","interfaces":[{"interface":"back","average":{"1min":2.5030000000000001,"5min":3.855,"15min":4.415},"min":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"max":{"1min":3.3860000000000001,"5min":21.331,"15min":21.331},"last":2.6419999999999999},{"interface":"front","average":{"1min":3.2879999999999998,"5min":4.0179999999999998,"15min":4.4249999999999998},"min":{"1min":2.2290000000000001,"5min":1.915,"15min":1.915},"max":{"1min":6.0979999999999999,"5min":21.515999999999998,"15min":21.515999999999998},"last":2.2360000000000002}]}]},{"osd":1,"up_from":0,"seq":127,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":910756,"kb_used_data":840092,"kb_used_omap":0,"kb_used_meta":70656,"kb_avail":92834396,"statfs":{"total":95995035648,"available":95062421504,"internally_reserved":0,"allocated":860254208,"data_stored":860003170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72351744},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:40:51 2020","interfaces":[{"interface":"back","average":{"1min":2.4079999999999999,"5min":3.2639999999999998,"15min":2.073},"min":{"1min":0.57399999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":7.7450000000000001,"5min":45.085000000000001,"15min":45.085000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":3.2490000000000001,"5min":3.6669999999999998,"15min":2.3250000000000002},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":13.231,"5min":45.765999999999998,"15min":45.765999999999998},"last":0.98399999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":5.992,"5min":3.5139999999999998,"15min":2.3149999999999999},"min":{"1min":0.84699999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":44.039000000000001,"5min":44.039000000000001,"15min":44.039000000000001},"last":1.7170000000000001},{"interface":"front","average":{"1min":6.3090000000000002,"5min":3.79,"15min":2.5059999999999998},"min":{"1min":0.71799999999999997,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":44.841000000000001,"5min":44.841000000000001,"15min":44.841000000000001},"last":2.4039999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:40:40 2020","interfaces":[{"interface":"back","average":{"1min":3.4700000000000002,"5min":3.5870000000000002,"15min":2.7149999999999999},"min":{"1min":0.98999999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":12.49,"5min":44.768999999999998,"15min":44.768999999999998},"last":2.169},{"interface":"front","average":{"1min":4.1289999999999996,"5min":4.0570000000000004,"15min":2.964},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":21.399999999999999,"5min":45.892000000000003,"15min":45.892000000000003},"last":5.9500000000000002}]},{"osd":4,"last update":"Tue Sep 8 15:39:56 2020","interfaces":[{"interface":"back","average":{"1min":6.1619999999999999,"5min":3.2069999999999999,"15min":2.5030000000000001},"min":{"1min":1.0329999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":44.970999999999997,"5min":44.970999999999997,"15min":44.970999999999997},"last":5.867},{"interface":"front","average":{"1min":4.8799999999999999,"5min":3.5110000000000001,"15min":2.9910000000000001},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":51.289000000000001,"5min":51.289000000000001,"15min":51.289000000000001},"last":1.895}]},{"osd":5,"last update":"Tue Sep 8 15:40:25 2020","interfaces":[{"interface":"back","average":{"1min":4.8380000000000001,"5min":3.819,"15min":3.246},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":45.289000000000001,"5min":45.289000000000001,"15min":45.289000000000001},"last":5.9880000000000004},{"interface":"front","average":{"1min":5.8029999999999999,"5min":3.3580000000000001,"15min":2.75},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":51.499000000000002,"5min":51.499000000000002,"15min":51.499000000000002},"last":1.7490000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:40:51 2020","interfaces":[{"interface":"back","average":{"1min":3.375,"5min":4.0309999999999997,"15min":3.7519999999999998},"min":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"max":{"1min":11.973000000000001,"5min":44.932000000000002,"15min":44.932000000000002},"last":2.3319999999999999},{"interface":"front","average":{"1min":3.3769999999999998,"5min":4.2869999999999999,"15min":3.7959999999999998},"min":{"1min":1.6060000000000001,"5min":1.2949999999999999,"15min":1.2949999999999999},"max":{"1min":7.9980000000000002,"5min":45.159999999999997,"15min":45.159999999999997},"last":2.0459999999999998}]},{"osd":7,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":5.5090000000000003,"5min":4.5990000000000002,"15min":3.6619999999999999},"min":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"max":{"1min":45.409999999999997,"5min":45.409999999999997,"15min":45.409999999999997},"last":2.3109999999999999},{"interface":"front","average":{"1min":4.9790000000000001,"5min":4.3410000000000002,"15min":4.0350000000000001},"min":{"1min":1.238,"5min":1.238,"15min":1.238},"max":{"1min":45.375,"5min":45.375,"15min":45.375},"last":6.1580000000000004}]},{"osd":8,"last update":"Tue Sep 8 15:40:40 2020","interfaces":[{"interface":"back","average":{"1min":3.6720000000000002,"5min":4.7400000000000002,"15min":4.4390000000000001},"min":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"max":{"1min":7.9850000000000003,"5min":45.640999999999998,"15min":45.640999999999998},"last":2.198},{"interface":"front","average":{"1min":3.5409999999999999,"5min":4.7389999999999999,"15min":4.2889999999999997},"min":{"1min":1.153,"5min":1.153,"15min":1.153},"max":{"1min":11.839,"5min":45.235999999999997,"15min":45.235999999999997},"last":1.873}]},{"osd":9,"last update":"Tue Sep 8 15:40:10 2020","interfaces":[{"interface":"back","average":{"1min":5.6619999999999999,"5min":3.6520000000000001,"15min":3.3769999999999998},"min":{"1min":1.103,"5min":1.0860000000000001,"15min":1.0860000000000001},"max":{"1min":51.698,"5min":51.698,"15min":51.698},"last":2.0230000000000001},{"interface":"front","average":{"1min":5.375,"5min":4.6189999999999998,"15min":4.359},"min":{"1min":1.45,"5min":1.45,"15min":1.45},"max":{"1min":45.515000000000001,"5min":45.515000000000001,"15min":45.515000000000001},"last":6.0890000000000004}]},{"osd":10,"last update":"Tue Sep 8 15:40:25 2020","interfaces":[{"interface":"back","average":{"1min":5.6440000000000001,"5min":4.6870000000000003,"15min":4.3689999999999998},"min":{"1min":1.587,"5min":1.4159999999999999,"15min":1.4159999999999999},"max":{"1min":46.039999999999999,"5min":46.039999999999999,"15min":46.039999999999999},"last":2.4900000000000002},{"interface":"front","average":{"1min":5.6280000000000001,"5min":3.7879999999999998,"15min":3.593},"min":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":51.892000000000003,"5min":51.892000000000003,"15min":51.892000000000003},"last":6.3120000000000003}]}]},{"osd":0,"up_from":0,"seq":132,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":698520,"kb_used_data":637968,"kb_used_omap":0,"kb_used_meta":60544,"kb_avail":93046632,"statfs":{"total":95995035648,"available":95279751168,"internally_reserved":0,"allocated":653279232,"data_stored":653032652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":61997056},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 8 15:39:49 2020","interfaces":[{"interface":"back","average":{"1min":2.9119999999999999,"5min":2.327,"15min":1.6719999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":10.579000000000001,"5min":10.579000000000001,"15min":10.579000000000001},"last":10.125},{"interface":"front","average":{"1min":3.5139999999999998,"5min":2.3599999999999999,"15min":1.7290000000000001},"min":{"1min":0.78100000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":13.589,"5min":13.589,"15min":13.589},"last":1.034}]},{"osd":2,"last update":"Tue Sep 8 15:34:05 2020","interfaces":[{"interface":"back","average":{"1min":1.9950000000000001,"5min":1.621,"15min":1.5189999999999999},"min":{"1min":1.3999999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.8519999999999999,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":2.6589999999999998},{"interface":"front","average":{"1min":2.629,"5min":1.956,"15min":1.72},"min":{"1min":1.6419999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":2.4049999999999998}]},{"osd":3,"last update":"Tue Sep 8 15:40:41 2020","interfaces":[{"interface":"back","average":{"1min":3.02,"5min":2.4660000000000002,"15min":2.7879999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":9.8670000000000009,"5min":9.8670000000000009,"15min":27.298999999999999},"last":3.125},{"interface":"front","average":{"1min":7.2450000000000001,"5min":3.8839999999999999,"15min":3.6880000000000002},"min":{"1min":1.167,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":73.075000000000003,"5min":73.075000000000003,"15min":73.075000000000003},"last":3.5270000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:40:05 2020","interfaces":[{"interface":"back","average":{"1min":2.04,"5min":2.6070000000000002,"15min":2.5329999999999999},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":6.8849999999999998,"5min":6.8849999999999998,"15min":6.8849999999999998},"last":2.3180000000000001},{"interface":"front","average":{"1min":6.1180000000000003,"5min":3.6669999999999998,"15min":3.0590000000000002},"min":{"1min":0.995,"5min":0.995,"15min":0.995},"max":{"1min":73.802999999999997,"5min":73.802999999999997,"15min":73.802999999999997},"last":2.7069999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:40:23 2020","interfaces":[{"interface":"back","average":{"1min":2.774,"5min":3.157,"15min":3.0489999999999999},"min":{"1min":0.83999999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":9.5730000000000004,"5min":9.5730000000000004,"15min":9.5730000000000004},"last":5.0359999999999996},{"interface":"front","average":{"1min":2.0329999999999999,"5min":2.343,"15min":2.9359999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":3.923,"5min":12.890000000000001,"15min":12.890000000000001},"last":2.2759999999999998}]},{"osd":6,"last update":"Tue Sep 8 15:39:49 2020","interfaces":[{"interface":"back","average":{"1min":2.77,"5min":2.5510000000000002,"15min":3.2759999999999998},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":13.151999999999999,"5min":13.151999999999999,"15min":13.151999999999999},"last":2.3940000000000001},{"interface":"front","average":{"1min":3.2000000000000002,"5min":2.8889999999999998,"15min":3.6659999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":13.48,"5min":13.48,"15min":13.48},"last":3.0449999999999999}]},{"osd":7,"last update":"Tue Sep 8 15:40:20 2020","interfaces":[{"interface":"back","average":{"1min":7.6299999999999999,"5min":4.0199999999999996,"15min":4.2519999999999998},"min":{"1min":1.605,"5min":1.302,"15min":1.302},"max":{"1min":71.167000000000002,"5min":71.167000000000002,"15min":71.167000000000002},"last":2.9350000000000001},{"interface":"front","average":{"1min":8.0850000000000009,"5min":4.4649999999999999,"15min":4.234},"min":{"1min":1.488,"5min":1.3240000000000001,"15min":1.3240000000000001},"max":{"1min":74.480000000000004,"5min":74.480000000000004,"15min":74.480000000000004},"last":9.8650000000000002}]},{"osd":8,"last update":"Tue Sep 8 15:40:41 2020","interfaces":[{"interface":"back","average":{"1min":2.6989999999999998,"5min":3.5419999999999998,"15min":3.794},"min":{"1min":1.1020000000000001,"5min":1.093,"15min":1.093},"max":{"1min":6.5359999999999996,"5min":7.173,"15min":7.173},"last":10.004},{"interface":"front","average":{"1min":7.0700000000000003,"5min":5.4770000000000003,"15min":4.5999999999999996},"min":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"max":{"1min":71.025000000000006,"5min":131.958,"15min":131.958},"last":2.633}]},{"osd":9,"last update":"Tue Sep 8 15:40:04 2020","interfaces":[{"interface":"back","average":{"1min":3.5070000000000001,"5min":3.9550000000000001,"15min":4.1749999999999998},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":13.259,"5min":13.259,"15min":13.259},"last":2.8180000000000001},{"interface":"front","average":{"1min":3.2400000000000002,"5min":3.4039999999999999,"15min":3.8900000000000001},"min":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":7.3399999999999999,"5min":7.3399999999999999,"15min":7.3399999999999999},"last":4.3449999999999998}]},{"osd":10,"last update":"Tue Sep 8 15:40:23 2020","interfaces":[{"interface":"back","average":{"1min":2.5630000000000002,"5min":2.9220000000000002,"15min":3.0539999999999998},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":4.7489999999999997,"5min":6.923,"15min":6.923},"last":9.2330000000000005},{"interface":"front","average":{"1min":7.468,"5min":4.7030000000000003,"15min":4.4909999999999997},"min":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"max":{"1min":73.444000000000003,"5min":73.444000000000003,"15min":73.444000000000003},"last":3.9009999999999998}]},{"osd":11,"last update":"Tue Sep 8 15:39:49 2020","interfaces":[{"interface":"back","average":{"1min":3.9609999999999999,"5min":3.7570000000000001,"15min":3.8450000000000002},"min":{"1min":1.476,"5min":1.476,"15min":1.476},"max":{"1min":12.486000000000001,"5min":12.486000000000001,"15min":12.486000000000001},"last":4.6950000000000003},{"interface":"front","average":{"1min":3.4369999999999998,"5min":4.181,"15min":4.6340000000000003},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":13.369999999999999,"5min":13.369999999999999,"15min":13.369999999999999},"last":2.5059999999999998}]}]},{"osd":2,"up_from":0,"seq":124,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":669412,"kb_used_data":620380,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":93075740,"statfs":{"total":95995035648,"available":95309557760,"internally_reserved":0,"allocated":635269120,"data_stored":635018082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:40:10 2020","interfaces":[{"interface":"back","average":{"1min":8.4199999999999999,"5min":3.3599999999999999,"15min":2.2029999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":162.83799999999999,"5min":162.83799999999999,"15min":162.83799999999999},"last":1.3819999999999999},{"interface":"front","average":{"1min":8.4369999999999994,"5min":3.4209999999999998,"15min":2.2869999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":162.63499999999999,"5min":162.63499999999999,"15min":162.63499999999999},"last":1.0580000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:40:10 2020","interfaces":[{"interface":"back","average":{"1min":5.2939999999999996,"5min":2.9350000000000001,"15min":2.1099999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":63.887,"5min":63.887,"15min":63.887},"last":2.0110000000000001},{"interface":"front","average":{"1min":2.5259999999999998,"5min":2.597,"15min":2.2669999999999999},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":10.519,"5min":10.519,"15min":10.519},"last":1.7190000000000001}]},{"osd":3,"last update":"Tue Sep 8 15:40:36 2020","interfaces":[{"interface":"back","average":{"1min":3.0019999999999998,"5min":2.8570000000000002,"15min":2.5449999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":11.315,"5min":11.315,"15min":11.315},"last":5.1280000000000001},{"interface":"front","average":{"1min":2.923,"5min":2.645,"15min":2.4609999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":11.737,"5min":11.737,"15min":11.737},"last":1.7669999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:40:04 2020","interfaces":[{"interface":"back","average":{"1min":2.548,"5min":2.7309999999999999,"15min":2.5350000000000001},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":7.1100000000000003,"5min":7.2530000000000001,"15min":7.2530000000000001},"last":1.1519999999999999},{"interface":"front","average":{"1min":5.1070000000000002,"5min":3.4369999999999998,"15min":2.7330000000000001},"min":{"1min":1.04,"5min":1.04,"15min":1.04},"max":{"1min":64.284999999999997,"5min":64.284999999999997,"15min":64.284999999999997},"last":2.9039999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:40:25 2020","interfaces":[{"interface":"back","average":{"1min":5.7460000000000004,"5min":3.875,"15min":3.2469999999999999},"min":{"1min":1.222,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":63.509,"5min":63.509,"15min":63.509},"last":5.6029999999999998},{"interface":"front","average":{"1min":3.0230000000000001,"5min":3.2469999999999999,"15min":3.0579999999999998},"min":{"1min":1.2689999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"max":{"1min":11.055999999999999,"5min":11.055999999999999,"15min":11.055999999999999},"last":2.2989999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:40:48 2020","interfaces":[{"interface":"back","average":{"1min":6.1989999999999998,"5min":4.0620000000000003,"15min":3.4660000000000002},"min":{"1min":1.3129999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"max":{"1min":63.631,"5min":63.631,"15min":63.631},"last":4.9509999999999996},{"interface":"front","average":{"1min":2.9369999999999998,"5min":3.3340000000000001,"15min":3.4929999999999999},"min":{"1min":1.1870000000000001,"5min":1.105,"15min":1.105},"max":{"1min":8.0120000000000005,"5min":11.234999999999999,"15min":11.234999999999999},"last":2.6280000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:40:21 2020","interfaces":[{"interface":"back","average":{"1min":3.3780000000000001,"5min":3.8180000000000001,"15min":3.9569999999999999},"min":{"1min":1.298,"5min":1.298,"15min":1.298},"max":{"1min":11.436,"5min":11.436,"15min":11.436},"last":2.181},{"interface":"front","average":{"1min":3.0670000000000002,"5min":3.3940000000000001,"15min":3.5950000000000002},"min":{"1min":1.016,"5min":1.006,"15min":1.006},"max":{"1min":7.2270000000000003,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":5.181}]},{"osd":8,"last update":"Tue Sep 8 15:40:30 2020","interfaces":[{"interface":"back","average":{"1min":6.0899999999999999,"5min":4.0759999999999996,"15min":3.802},"min":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":64.116,"5min":64.116,"15min":64.116},"last":2.6659999999999999},{"interface":"front","average":{"1min":5.9539999999999997,"5min":4.2370000000000001,"15min":4.0620000000000003},"min":{"1min":1.532,"5min":1.218,"15min":1.218},"max":{"1min":63.299999999999997,"5min":63.299999999999997,"15min":63.299999999999997},"last":2.4199999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:40:10 2020","interfaces":[{"interface":"back","average":{"1min":6.4059999999999997,"5min":4.4589999999999996,"15min":4.1829999999999998},"min":{"1min":1.591,"5min":1.591,"15min":1.591},"max":{"1min":64.004000000000005,"5min":64.004000000000005,"15min":64.004000000000005},"last":5.1029999999999998},{"interface":"front","average":{"1min":3.5760000000000001,"5min":3.7879999999999998,"15min":4.1299999999999999},"min":{"1min":1.401,"5min":1.401,"15min":1.401},"max":{"1min":11.704000000000001,"5min":11.704000000000001,"15min":11.704000000000001},"last":4.8380000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:40:25 2020","interfaces":[{"interface":"back","average":{"1min":6.4800000000000004,"5min":4.5209999999999999,"15min":4.1890000000000001},"min":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"max":{"1min":63.743000000000002,"5min":63.743000000000002,"15min":63.743000000000002},"last":5.2329999999999997},{"interface":"front","average":{"1min":3.0800000000000001,"5min":3.2559999999999998,"15min":3.6139999999999999},"min":{"1min":1.159,"5min":1.159,"15min":1.159},"max":{"1min":6.9690000000000003,"5min":7.9960000000000004,"15min":7.9960000000000004},"last":1.6499999999999999}]}]},{"osd":3,"up_from":0,"seq":117,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":682020,"kb_used_data":632988,"kb_used_omap":0,"kb_used_meta":49024,"kb_avail":93063132,"statfs":{"total":95995035648,"available":95296647168,"internally_reserved":0,"allocated":648179712,"data_stored":647928674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50200576},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:31 2020","interfaces":[{"interface":"back","average":{"1min":2.1379999999999999,"5min":1.7210000000000001,"15min":1.629},"min":{"1min":1.79,"5min":1.1419999999999999,"15min":1.1419999999999999},"max":{"1min":2.7930000000000001,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":3.169},{"interface":"front","average":{"1min":2.6800000000000002,"5min":2.2309999999999999,"15min":2.0990000000000002},"min":{"1min":2.0270000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"max":{"1min":3.5409999999999999,"5min":3.5409999999999999,"15min":3.5409999999999999},"last":2.5579999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:40:42 2020","interfaces":[{"interface":"back","average":{"1min":3.4740000000000002,"5min":2.9100000000000001,"15min":2.2309999999999999},"min":{"1min":0.76000000000000001,"5min":0.628,"15min":0.628},"max":{"1min":11.423999999999999,"5min":44.109999999999999,"15min":44.109999999999999},"last":1.8700000000000001},{"interface":"front","average":{"1min":3.0030000000000001,"5min":3.7810000000000001,"15min":2.508},"min":{"1min":0.97299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":7.532,"5min":45.618000000000002,"15min":45.618000000000002},"last":2.98}]},{"osd":2,"last update":"Tue Sep 8 15:40:42 2020","interfaces":[{"interface":"back","average":{"1min":2.605,"5min":3.222,"15min":2.5089999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":10.340999999999999,"5min":45.774000000000001,"15min":45.774000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":3.036,"5min":2.548,"15min":2.375},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":12.098000000000001,"5min":13.948,"15min":13.948},"last":2.1600000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:40:01 2020","interfaces":[{"interface":"back","average":{"1min":2.7839999999999998,"5min":2.5390000000000001,"15min":2.5459999999999998},"min":{"1min":1.085,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":5.9530000000000003,"5min":5.9530000000000003,"15min":5.9530000000000003},"last":4.8860000000000001},{"interface":"front","average":{"1min":2.1949999999999998,"5min":2.3580000000000001,"15min":2.452},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":11.298,"5min":11.298,"15min":11.298},"last":1.7250000000000001}]},{"osd":5,"last update":"Tue Sep 8 15:40:27 2020","interfaces":[{"interface":"back","average":{"1min":3.2200000000000002,"5min":3.1469999999999998,"15min":2.964},"min":{"1min":1.0609999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"max":{"1min":27.007999999999999,"5min":45.442999999999998,"15min":45.442999999999998},"last":2.444},{"interface":"front","average":{"1min":4.556,"5min":3.3500000000000001,"15min":3.2789999999999999},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":23.565000000000001,"5min":23.565000000000001,"15min":23.565000000000001},"last":5.5599999999999996}]},{"osd":6,"last update":"Tue Sep 8 15:39:52 2020","interfaces":[{"interface":"back","average":{"1min":2.46,"5min":2.976,"15min":3.2360000000000002},"min":{"1min":1.1759999999999999,"5min":1.127,"15min":1.127},"max":{"1min":6.2880000000000003,"5min":6.2880000000000003,"15min":6.2880000000000003},"last":3.1299999999999999},{"interface":"front","average":{"1min":2.2349999999999999,"5min":3.4809999999999999,"15min":3.524},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":5.4359999999999999,"5min":47.457999999999998,"15min":47.457999999999998},"last":2.5739999999999998}]},{"osd":7,"last update":"Tue Sep 8 15:40:27 2020","interfaces":[{"interface":"back","average":{"1min":5.0999999999999996,"5min":4.0739999999999998,"15min":4.1959999999999997},"min":{"1min":1.3899999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"max":{"1min":26.035,"5min":26.035,"15min":26.035},"last":5.2460000000000004},{"interface":"front","average":{"1min":3.7749999999999999,"5min":3.2509999999999999,"15min":3.4990000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":25.521000000000001,"5min":25.521000000000001,"15min":25.521000000000001},"last":1.587}]},{"osd":8,"last update":"Tue Sep 8 15:40:39 2020","interfaces":[{"interface":"back","average":{"1min":2.6099999999999999,"5min":3.5870000000000002,"15min":3.7069999999999999},"min":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":8.4529999999999994,"5min":46.917999999999999,"15min":46.917999999999999},"last":2.673},{"interface":"front","average":{"1min":4.1689999999999996,"5min":4.6900000000000004,"15min":4.4249999999999998},"min":{"1min":1.272,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":11.534000000000001,"5min":44.893000000000001,"15min":44.893000000000001},"last":5.4089999999999998}]},{"osd":9,"last update":"Tue Sep 8 15:40:01 2020","interfaces":[{"interface":"back","average":{"1min":2.681,"5min":3.181,"15min":3.7280000000000002},"min":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":10.872999999999999,"5min":10.872999999999999,"15min":10.872999999999999},"last":1.272},{"interface":"front","average":{"1min":3.5,"5min":3.9870000000000001,"15min":4.2709999999999999},"min":{"1min":1.4790000000000001,"5min":1.4279999999999999,"15min":1.4279999999999999},"max":{"1min":12.752000000000001,"5min":46.665999999999997,"15min":46.665999999999997},"last":2.778}]},{"osd":10,"last update":"Tue Sep 8 15:40:27 2020","interfaces":[{"interface":"back","average":{"1min":3.9780000000000002,"5min":4.3280000000000003,"15min":4.3739999999999997},"min":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"max":{"1min":11.694000000000001,"5min":39.283000000000001,"15min":39.283000000000001},"last":5.0709999999999997},{"interface":"front","average":{"1min":2.9279999999999999,"5min":3.3010000000000002,"15min":3.6080000000000001},"min":{"1min":1.081,"5min":1.081,"15min":1.081},"max":{"1min":11.005000000000001,"5min":11.005000000000001,"15min":11.005000000000001},"last":1.4319999999999999}]},{"osd":11,"last update":"Tue Sep 8 15:39:52 2020","interfaces":[{"interface":"back","average":{"1min":3.7709999999999999,"5min":4.3129999999999997,"15min":4.6479999999999997},"min":{"1min":1.196,"5min":1.196,"15min":1.196},"max":{"1min":11.156000000000001,"5min":13.882999999999999,"15min":13.882999999999999},"last":2.0329999999999999},{"interface":"front","average":{"1min":5.8010000000000002,"5min":5.226,"15min":4.5060000000000002},"min":{"1min":1.577,"5min":1.577,"15min":1.577},"max":{"1min":44.722999999999999,"5min":47.234999999999999,"15min":47.234999999999999},"last":2.863}]}]},{"osd":4,"up_from":0,"seq":114,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":241704,"kb_used_data":208864,"kb_used_omap":0,"kb_used_meta":32768,"kb_avail":93503448,"statfs":{"total":95995035648,"available":95747530752,"internally_reserved":0,"allocated":213876736,"data_stored":213621626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":33554432},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:39:58 2020","interfaces":[{"interface":"back","average":{"1min":1.661,"5min":1.9490000000000001,"15min":1.871},"min":{"1min":0.80900000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":5.6210000000000004,"5min":5.7130000000000001,"15min":5.7130000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.5529999999999999,"5min":2.0609999999999999,"15min":1.8680000000000001},"min":{"1min":0.85099999999999998,"5min":0.71299999999999997,"15min":0.70799999999999996},"max":{"1min":4.1059999999999999,"5min":4.3650000000000002,"15min":4.3650000000000002},"last":1.3979999999999999}]},{"osd":1,"last update":"Tue Sep 8 15:39:58 2020","interfaces":[{"interface":"back","average":{"1min":2.577,"5min":3.5510000000000002,"15min":2.3929999999999998},"min":{"1min":1.179,"5min":0.996,"15min":0.996},"max":{"1min":15.336,"5min":74.947999999999993,"15min":74.947999999999993},"last":4.8719999999999999},{"interface":"front","average":{"1min":3.109,"5min":3.8969999999999998,"15min":2.665},"min":{"1min":1.087,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":16.388999999999999,"5min":71.108999999999995,"15min":71.108999999999995},"last":1.5720000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:39:58 2020","interfaces":[{"interface":"back","average":{"1min":2.3580000000000001,"5min":3.9079999999999999,"15min":2.6619999999999999},"min":{"1min":1.2270000000000001,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":3.714,"5min":74.504000000000005,"15min":74.504000000000005},"last":4.2169999999999996},{"interface":"front","average":{"1min":1.9279999999999999,"5min":3.8079999999999998,"15min":2.7480000000000002},"min":{"1min":1.2010000000000001,"5min":1.155,"15min":1.155},"max":{"1min":2.8530000000000002,"5min":73.885000000000005,"15min":73.885000000000005},"last":1.27}]},{"osd":3,"last update":"Tue Sep 8 15:39:58 2020","interfaces":[{"interface":"back","average":{"1min":2.46,"5min":2.8879999999999999,"15min":2.5750000000000002},"min":{"1min":1.091,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":6.1870000000000003,"5min":6.3280000000000003,"15min":6.3280000000000003},"last":2.6669999999999998},{"interface":"front","average":{"1min":2.681,"5min":2.839,"15min":2.4750000000000001},"min":{"1min":1.5489999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"max":{"1min":5.2809999999999997,"5min":7.4900000000000002,"15min":7.4900000000000002},"last":7.5590000000000002}]},{"osd":5,"last update":"Tue Sep 8 15:40:28 2020","interfaces":[{"interface":"back","average":{"1min":2.351,"5min":2.9860000000000002,"15min":2.8220000000000001},"min":{"1min":1.2709999999999999,"5min":1.024,"15min":1.014},"max":{"1min":3.8759999999999999,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":2.3159999999999998},{"interface":"front","average":{"1min":2.161,"5min":3.0310000000000001,"15min":2.8759999999999999},"min":{"1min":1.274,"5min":1.2190000000000001,"15min":1.129},"max":{"1min":3.6949999999999998,"5min":7.5359999999999996,"15min":7.5359999999999996},"last":4.2800000000000002}]},{"osd":6,"last update":"Tue Sep 8 15:40:46 2020","interfaces":[{"interface":"back","average":{"1min":2.8260000000000001,"5min":3.2639999999999998,"15min":3.3159999999999998},"min":{"1min":1.645,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":6.6420000000000003,"5min":6.8650000000000002,"15min":6.8650000000000002},"last":4.6710000000000003},{"interface":"front","average":{"1min":2.3940000000000001,"5min":3.1549999999999998,"15min":3.2999999999999998},"min":{"1min":1.4450000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"max":{"1min":6.8849999999999998,"5min":6.8849999999999998,"15min":6.8849999999999998},"last":3.4710000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:40:08 2020","interfaces":[{"interface":"back","average":{"1min":3.1339999999999999,"5min":3.6110000000000002,"15min":3.782},"min":{"1min":1.8660000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"max":{"1min":5.3280000000000003,"5min":7.4699999999999998,"15min":7.4699999999999998},"last":2.8490000000000002},{"interface":"front","average":{"1min":3.137,"5min":3.6629999999999998,"15min":3.8799999999999999},"min":{"1min":1.546,"5min":1.546,"15min":1.546},"max":{"1min":6.4269999999999996,"5min":7.1459999999999999,"15min":7.1459999999999999},"last":2.819}]},{"osd":8,"last update":"Tue Sep 8 15:40:39 2020","interfaces":[{"interface":"back","average":{"1min":2.9449999999999998,"5min":5.2080000000000002,"15min":4.3559999999999999},"min":{"1min":1.276,"5min":1.276,"15min":1.276},"max":{"1min":7.8140000000000001,"5min":109.608,"15min":109.608},"last":2.524},{"interface":"front","average":{"1min":3.222,"5min":5.3799999999999999,"15min":4.7039999999999997},"min":{"1min":1.889,"5min":1.49,"15min":1.49},"max":{"1min":6.4039999999999999,"5min":116.52500000000001,"15min":116.52500000000001},"last":4.3929999999999998}]},{"osd":9,"last update":"Tue Sep 8 15:40:04 2020","interfaces":[{"interface":"back","average":{"1min":2.7890000000000001,"5min":5.5419999999999998,"15min":4.7629999999999999},"min":{"1min":1.546,"5min":1.546,"15min":1.546},"max":{"1min":6.4080000000000004,"5min":118.312,"15min":118.312},"last":3.0739999999999998},{"interface":"front","average":{"1min":2.782,"5min":5.3120000000000003,"15min":4.7229999999999999},"min":{"1min":1.7689999999999999,"5min":1.647,"15min":1.647},"max":{"1min":4.3470000000000004,"5min":116.871,"15min":116.871},"last":4.4829999999999997}]},{"osd":10,"last update":"Tue Sep 8 15:40:28 2020","interfaces":[{"interface":"back","average":{"1min":2.593,"5min":3.5710000000000002,"15min":4.4829999999999997},"min":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"max":{"1min":6.5170000000000003,"5min":7.9279999999999999,"15min":7.9279999999999999},"last":2.6960000000000002},{"interface":"front","average":{"1min":2.5950000000000002,"5min":3.8330000000000002,"15min":4.5629999999999997},"min":{"1min":1.702,"5min":1.702,"15min":1.702},"max":{"1min":4.5609999999999999,"5min":7.9870000000000001,"15min":7.9870000000000001},"last":3.3399999999999999}]}]},{"osd":5,"up_from":0,"seq":107,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":452260,"kb_used_data":409692,"kb_used_omap":0,"kb_used_meta":42560,"kb_avail":93292892,"statfs":{"total":95995035648,"available":95531921408,"internally_reserved":0,"allocated":419524608,"data_stored":419273570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":43581440},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:40:29 2020","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.8720000000000001,"15min":1.881},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":5.4370000000000003,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":1.571},{"interface":"front","average":{"1min":1.7350000000000001,"5min":2.3210000000000002,"15min":2.0790000000000002},"min":{"1min":0.90800000000000003,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":4.4859999999999998,"5min":7.1340000000000003,"15min":7.1340000000000003},"last":2.9769999999999999}]},{"osd":1,"last update":"Tue Sep 8 15:40:29 2020","interfaces":[{"interface":"back","average":{"1min":4.0940000000000003,"5min":2.8610000000000002,"15min":2.492},"min":{"1min":0.97799999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":35.908999999999999,"5min":35.908999999999999,"15min":35.908999999999999},"last":1.109},{"interface":"front","average":{"1min":4.79,"5min":3.3860000000000001,"15min":2.464},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":36.561999999999998,"5min":36.561999999999998,"15min":36.561999999999998},"last":2.4460000000000002}]},{"osd":2,"last update":"Tue Sep 8 15:40:29 2020","interfaces":[{"interface":"back","average":{"1min":2.649,"5min":2.7970000000000002,"15min":2.3530000000000002},"min":{"1min":0.97499999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":6.8250000000000002,"5min":31.021000000000001,"15min":31.021000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":2.766,"5min":3.2770000000000001,"15min":2.903},"min":{"1min":1.1100000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"max":{"1min":5.5229999999999997,"5min":37.658999999999999,"15min":37.658999999999999},"last":3.2469999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:40:29 2020","interfaces":[{"interface":"back","average":{"1min":2.6659999999999999,"5min":3.3599999999999999,"15min":3.0030000000000001},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":8.4220000000000006,"5min":8.4220000000000006,"15min":8.4220000000000006},"last":2.718},{"interface":"front","average":{"1min":2.3479999999999999,"5min":2.8149999999999999,"15min":2.8860000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":6.819,"5min":6.819,"15min":6.819},"last":1.4059999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:40:29 2020","interfaces":[{"interface":"back","average":{"1min":2.339,"5min":2.5070000000000001,"15min":2.3610000000000002},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":6.9180000000000001,"5min":6.9180000000000001,"15min":6.9180000000000001},"last":1.6319999999999999},{"interface":"front","average":{"1min":2.1080000000000001,"5min":2.899,"15min":2.738},"min":{"1min":1.1990000000000001,"5min":1.1819999999999999,"15min":1.1819999999999999},"max":{"1min":3.3889999999999998,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":1.7390000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:39:57 2020","interfaces":[{"interface":"back","average":{"1min":3.4729999999999999,"5min":3.782,"15min":3.6160000000000001},"min":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"max":{"1min":8.7710000000000008,"5min":8.7710000000000008,"15min":8.7710000000000008},"last":3.52},{"interface":"front","average":{"1min":3.911,"5min":4.2850000000000001,"15min":3.3860000000000001},"min":{"1min":1.73,"5min":1.73,"15min":1.73},"max":{"1min":9.359,"5min":9.359,"15min":9.359},"last":4.6669999999999998}]},{"osd":7,"last update":"Tue Sep 8 15:40:13 2020","interfaces":[{"interface":"back","average":{"1min":2.996,"5min":3.5680000000000001,"15min":3.5609999999999999},"min":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"max":{"1min":6.9909999999999997,"5min":6.9909999999999997,"15min":6.9909999999999997},"last":3.7959999999999998},{"interface":"front","average":{"1min":4.0149999999999997,"5min":4.1539999999999999,"15min":4.0709999999999997},"min":{"1min":2.2789999999999999,"5min":1.532,"15min":1.532},"max":{"1min":9.2129999999999992,"5min":9.2129999999999992,"15min":9.2129999999999992},"last":4.9980000000000002}]},{"osd":8,"last update":"Tue Sep 8 15:40:39 2020","interfaces":[{"interface":"back","average":{"1min":2.8279999999999998,"5min":3.2719999999999998,"15min":3.4990000000000001},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":5.8330000000000002,"5min":7.8140000000000001,"15min":7.8140000000000001},"last":1.8},{"interface":"front","average":{"1min":2.641,"5min":3.2719999999999998,"15min":3.794},"min":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"max":{"1min":10.138999999999999,"5min":10.138999999999999,"15min":10.138999999999999},"last":2.0009999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:40:06 2020","interfaces":[{"interface":"back","average":{"1min":3.569,"5min":3.931,"15min":4.3479999999999999},"min":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"max":{"1min":7.8959999999999999,"5min":7.8959999999999999,"15min":7.8959999999999999},"last":1.96},{"interface":"front","average":{"1min":4.1109999999999998,"5min":4.2169999999999996,"15min":4.218},"min":{"1min":2.4319999999999999,"5min":1.992,"15min":1.992},"max":{"1min":8.9149999999999991,"5min":8.9149999999999991,"15min":8.9149999999999991},"last":4.4029999999999996}]},{"osd":10,"last update":"Tue Sep 8 15:40:29 2020","interfaces":[{"interface":"back","average":{"1min":2.4300000000000002,"5min":3.3130000000000002,"15min":3.9289999999999998},"min":{"1min":1.431,"5min":1.431,"15min":1.431},"max":{"1min":4.4690000000000003,"5min":5.7309999999999999,"15min":5.7309999999999999},"last":2.2250000000000001},{"interface":"front","average":{"1min":3.762,"5min":4.4089999999999998,"15min":4.6609999999999996},"min":{"1min":2.2610000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"max":{"1min":9.0670000000000002,"5min":9.0670000000000002,"15min":9.0670000000000002},"last":4.8470000000000004}]}]}]}} 2020-09-08T15:40:58.264 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2020-09-08T15:40:58.265 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-09-08T15:40:58.265 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd dump -f json-pretty 2020-09-08T15:40:58.371 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:40:58.374 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:40:58 smithi166 bash[18960]: debug 2020-09-08T15:40:58.368+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:58.427 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:40:58.474 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:40:58.644 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:58 smithi166 bash[9446]: cluster 2020-09-08T15:40:57.643630+0000 mgr.x (mgr.24118) 405 : cluster [DBG] pgmap v522: 17 pgs: 17 active+clean; 2.9 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.21k op/s 2020-09-08T15:40:58.644 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:40:58 smithi166 bash[9446]: cluster 2020-09-08T15:40:57.643937+0000 mon.a (mon.0) 1060 : cluster [DBG] osdmap e197: 12 total, 12 up, 12 in 2020-09-08T15:40:58.646 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:58 smithi104 bash[9203]: cluster 2020-09-08T15:40:57.643630+0000 mgr.x (mgr.24118) 405 : cluster [DBG] pgmap v522: 17 pgs: 17 active+clean; 2.9 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.21k op/s 2020-09-08T15:40:58.647 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:40:58 smithi104 bash[9203]: cluster 2020-09-08T15:40:57.643937+0000 mon.a (mon.0) 1060 : cluster [DBG] osdmap e197: 12 total, 12 up, 12 in 2020-09-08T15:40:58.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:58 smithi055 bash[9296]: cluster 2020-09-08T15:40:57.643630+0000 mgr.x (mgr.24118) 405 : cluster [DBG] pgmap v522: 17 pgs: 17 active+clean; 2.9 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.21k op/s 2020-09-08T15:40:58.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:40:58 smithi055 bash[9296]: cluster 2020-09-08T15:40:57.643937+0000 mon.a (mon.0) 1060 : cluster [DBG] osdmap e197: 12 total, 12 up, 12 in 2020-09-08T15:40:59.026 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:40:59.129 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:59.430 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:40:59 smithi104 bash[17588]: debug 2020-09-08T15:40:59.025+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:59.574 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:40:59.678 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:59.830 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:40:59.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:40:59 smithi104 bash[15273]: debug 2020-09-08T15:40:59.573+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:40:59.932 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:00.030 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:41:00.038 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:00.118 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:40:59 smithi166 bash[16593]: debug 2020-09-08T15:40:59.808+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:00.158 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:00.195 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:41:00.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:41:00 smithi055 bash[11956]: debug 2020-09-08T15:41:00.038+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:00.579 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:41:00 smithi166 bash[16593]: debug 2020-09-08T15:41:00.580+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:00.580 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:00.693 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:41:00.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:00 smithi166 bash[9446]: cluster 2020-09-08T15:40:59.644177+0000 mgr.x (mgr.24118) 406 : cluster [DBG] pgmap v523: 17 pgs: 17 active+clean; 2.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T15:41:00.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:00 smithi055 bash[9296]: cluster 2020-09-08T15:40:59.644177+0000 mgr.x (mgr.24118) 406 : cluster [DBG] pgmap v523: 17 pgs: 17 active+clean; 2.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T15:41:00.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:00 smithi104 bash[9203]: cluster 2020-09-08T15:40:59.644177+0000 mgr.x (mgr.24118) 406 : cluster [DBG] pgmap v523: 17 pgs: 17 active+clean; 2.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T15:41:01.182 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:41:01.286 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:01.610 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:41:01 smithi166 bash[11928]: debug 2020-09-08T15:41:01.180+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:01.633 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:41:01.789 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:01.887 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:01 smithi166 bash[9446]: audit 2020-09-08T15:41:00.778224+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? v1:172.21.15.104:0/2529654271' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:41:01.888 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:41:01 smithi166 bash[16593]: debug 2020-09-08T15:41:01.788+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:01.892 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:01.903 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:01 smithi055 bash[9296]: audit 2020-09-08T15:41:00.778224+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? v1:172.21.15.104:0/2529654271' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:41:01.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:01 smithi104 bash[9203]: audit 2020-09-08T15:41:00.778224+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? v1:172.21.15.104:0/2529654271' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:41:02.006 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:02.111 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:02.161 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:41:02 smithi055 bash[14301]: debug 2020-09-08T15:41:02.002+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:02.477 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:41:02.478 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:41:02.515 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:02.618 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:02.717 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:02.822 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:02.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:02 smithi166 bash[9446]: cluster 2020-09-08T15:41:01.614062+0000 mon.a (mon.0) 1062 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:41:02.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:02 smithi166 bash[9446]: audit 2020-09-08T15:41:01.632555+0000 mon.a (mon.0) 1063 : audit [INF] from='client.? v1:172.21.15.104:0/2529654271' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:41:02.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:02 smithi166 bash[9446]: cluster 2020-09-08T15:41:01.632657+0000 mon.a (mon.0) 1064 : cluster [DBG] osdmap e198: 12 total, 12 up, 12 in 2020-09-08T15:41:02.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:02 smithi166 bash[9446]: cluster 2020-09-08T15:41:01.644610+0000 mgr.x (mgr.24118) 407 : cluster [DBG] pgmap v525: 17 pgs: 17 active+clean; 2.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 798 op/s 2020-09-08T15:41:02.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:02 smithi166 bash[9446]: cluster 2020-09-08T15:41:02.321018+0000 osd.5 (osd.5) 25 : cluster [DBG] 2.3 scrub starts 2020-09-08T15:41:02.911 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:41:02 smithi055 bash[16646]: debug 2020-09-08T15:41:02.718+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:02.912 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:02 smithi055 bash[9296]: cluster 2020-09-08T15:41:01.614062+0000 mon.a (mon.0) 1062 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:41:02.912 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:02 smithi055 bash[9296]: audit 2020-09-08T15:41:01.632555+0000 mon.a (mon.0) 1063 : audit [INF] from='client.? v1:172.21.15.104:0/2529654271' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:41:02.912 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:02 smithi055 bash[9296]: cluster 2020-09-08T15:41:01.632657+0000 mon.a (mon.0) 1064 : cluster [DBG] osdmap e198: 12 total, 12 up, 12 in 2020-09-08T15:41:02.912 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:02 smithi055 bash[9296]: cluster 2020-09-08T15:41:01.644610+0000 mgr.x (mgr.24118) 407 : cluster [DBG] pgmap v525: 17 pgs: 17 active+clean; 2.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 798 op/s 2020-09-08T15:41:02.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:02 smithi055 bash[9296]: cluster 2020-09-08T15:41:02.321018+0000 osd.5 (osd.5) 25 : cluster [DBG] 2.3 scrub starts 2020-09-08T15:41:02.925 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:02.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:41:02 smithi104 bash[19922]: debug 2020-09-08T15:41:02.513+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:02.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:02 smithi104 bash[9203]: cluster 2020-09-08T15:41:01.614062+0000 mon.a (mon.0) 1062 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:41:02.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:02 smithi104 bash[9203]: audit 2020-09-08T15:41:01.632555+0000 mon.a (mon.0) 1063 : audit [INF] from='client.? v1:172.21.15.104:0/2529654271' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:41:02.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:02 smithi104 bash[9203]: cluster 2020-09-08T15:41:01.632657+0000 mon.a (mon.0) 1064 : cluster [DBG] osdmap e198: 12 total, 12 up, 12 in 2020-09-08T15:41:02.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:02 smithi104 bash[9203]: cluster 2020-09-08T15:41:01.644610+0000 mgr.x (mgr.24118) 407 : cluster [DBG] pgmap v525: 17 pgs: 17 active+clean; 2.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 798 op/s 2020-09-08T15:41:02.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:02 smithi104 bash[9203]: cluster 2020-09-08T15:41:02.321018+0000 osd.5 (osd.5) 25 : cluster [DBG] 2.3 scrub starts 2020-09-08T15:41:03.029 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:03.124 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:03.232 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:03.240 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:41:03 smithi055 bash[16646]: debug 2020-09-08T15:41:03.122+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:03.240 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:41:02 smithi055 bash[14301]: debug 2020-09-08T15:41:02.926+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:03.350 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:03.456 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:03.510 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2020-09-08T15:41:03.603 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:03.632 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:03 smithi055 bash[9296]: cluster 2020-09-08T15:41:02.466890+0000 osd.7 (osd.7) 35 : cluster [DBG] 2.8 scrub starts 2020-09-08T15:41:03.633 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:03 smithi055 bash[9296]: cluster 2020-09-08T15:41:03.134238+0000 osd.5 (osd.5) 26 : cluster [DBG] 2.3 scrub ok 2020-09-08T15:41:03.633 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:41:03 smithi055 bash[14301]: debug 2020-09-08T15:41:03.346+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:03.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:03 smithi104 bash[9203]: cluster 2020-09-08T15:41:02.466890+0000 osd.7 (osd.7) 35 : cluster [DBG] 2.8 scrub starts 2020-09-08T15:41:03.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:03 smithi104 bash[9203]: cluster 2020-09-08T15:41:03.134238+0000 osd.5 (osd.5) 26 : cluster [DBG] 2.3 scrub ok 2020-09-08T15:41:03.678 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:41:03.706 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:03.820 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:03.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:03 smithi166 bash[9446]: cluster 2020-09-08T15:41:02.466890+0000 osd.7 (osd.7) 35 : cluster [DBG] 2.8 scrub starts 2020-09-08T15:41:03.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:03 smithi166 bash[9446]: cluster 2020-09-08T15:41:03.134238+0000 osd.5 (osd.5) 26 : cluster [DBG] 2.3 scrub ok 2020-09-08T15:41:03.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:41:03 smithi166 bash[18960]: debug 2020-09-08T15:41:03.600+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:03.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:41:03 smithi055 bash[14301]: debug 2020-09-08T15:41:03.810+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:03.921 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:04.403 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:04.506 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:04.835 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:04 smithi104 bash[9203]: cluster 2020-09-08T15:41:03.288747+0000 osd.7 (osd.7) 36 : cluster [DBG] 2.8 scrub ok 2020-09-08T15:41:04.836 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:04 smithi104 bash[9203]: cluster 2020-09-08T15:41:03.489117+0000 osd.7 (osd.7) 37 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:41:04.836 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:04 smithi104 bash[9203]: cluster 2020-09-08T15:41:03.497246+0000 osd.7 (osd.7) 38 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:41:04.836 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:04 smithi104 bash[9203]: cluster 2020-09-08T15:41:03.645143+0000 mgr.x (mgr.24118) 408 : cluster [DBG] pgmap v526: 17 pgs: 17 active+clean; 2.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.22k op/s 2020-09-08T15:41:04.837 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:41:04 smithi104 bash[15273]: debug 2020-09-08T15:41:04.833+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:04.837 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:04.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:41:04 smithi166 bash[18960]: debug 2020-09-08T15:41:04.400+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:04.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:04 smithi166 bash[9446]: cluster 2020-09-08T15:41:03.288747+0000 osd.7 (osd.7) 36 : cluster [DBG] 2.8 scrub ok 2020-09-08T15:41:04.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:04 smithi166 bash[9446]: cluster 2020-09-08T15:41:03.489117+0000 osd.7 (osd.7) 37 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:41:04.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:04 smithi166 bash[9446]: cluster 2020-09-08T15:41:03.497246+0000 osd.7 (osd.7) 38 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:41:04.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:04 smithi166 bash[9446]: cluster 2020-09-08T15:41:03.645143+0000 mgr.x (mgr.24118) 408 : cluster [DBG] pgmap v526: 17 pgs: 17 active+clean; 2.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.22k op/s 2020-09-08T15:41:04.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:04 smithi055 bash[9296]: cluster 2020-09-08T15:41:03.288747+0000 osd.7 (osd.7) 36 : cluster [DBG] 2.8 scrub ok 2020-09-08T15:41:04.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:04 smithi055 bash[9296]: cluster 2020-09-08T15:41:03.489117+0000 osd.7 (osd.7) 37 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:41:04.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:04 smithi055 bash[9296]: cluster 2020-09-08T15:41:03.497246+0000 osd.7 (osd.7) 38 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:41:04.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:04 smithi055 bash[9296]: cluster 2020-09-08T15:41:03.645143+0000 mgr.x (mgr.24118) 408 : cluster [DBG] pgmap v526: 17 pgs: 17 active+clean; 2.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.22k op/s 2020-09-08T15:41:04.939 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:05.199 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:05.200 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:41:05 smithi166 bash[16593]: debug 2020-09-08T15:41:05.196+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:05.302 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:41:05.413 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:41:05.415 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:41:05.517 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:05.584 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:41:05.637 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:41:05 smithi055 bash[19003]: debug 2020-09-08T15:41:05.410+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:05.641 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:05.655 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T15:41:05.656 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:41:05.656 INFO:teuthology.orchestra.run.smithi104.stdout: "epoch": 199, 2020-09-08T15:41:05.656 INFO:teuthology.orchestra.run.smithi104.stdout: "fsid": "759b27b6-f1e7-11ea-a080-001a4aab830c", 2020-09-08T15:41:05.657 INFO:teuthology.orchestra.run.smithi104.stdout: "created": "2020-09-08T15:25:41.788365+0000", 2020-09-08T15:41:05.657 INFO:teuthology.orchestra.run.smithi104.stdout: "modified": "2020-09-08T15:41:05.638807+0000", 2020-09-08T15:41:05.657 INFO:teuthology.orchestra.run.smithi104.stdout: "last_up_change": "2020-09-08T15:34:41.591462+0000", 2020-09-08T15:41:05.657 INFO:teuthology.orchestra.run.smithi104.stdout: "last_in_change": "2020-09-08T15:34:41.591462+0000", 2020-09-08T15:41:05.658 INFO:teuthology.orchestra.run.smithi104.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-09-08T15:41:05.658 INFO:teuthology.orchestra.run.smithi104.stdout: "flags_num": 5804032, 2020-09-08T15:41:05.658 INFO:teuthology.orchestra.run.smithi104.stdout: "flags_set": [ 2020-09-08T15:41:05.658 INFO:teuthology.orchestra.run.smithi104.stdout: "nodeep-scrub", 2020-09-08T15:41:05.658 INFO:teuthology.orchestra.run.smithi104.stdout: "pglog_hardlimit", 2020-09-08T15:41:05.659 INFO:teuthology.orchestra.run.smithi104.stdout: "purged_snapdirs", 2020-09-08T15:41:05.659 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_deletes", 2020-09-08T15:41:05.659 INFO:teuthology.orchestra.run.smithi104.stdout: "sortbitwise" 2020-09-08T15:41:05.659 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T15:41:05.660 INFO:teuthology.orchestra.run.smithi104.stdout: "crush_version": 27, 2020-09-08T15:41:05.660 INFO:teuthology.orchestra.run.smithi104.stdout: "full_ratio": 0.94999998807907104, 2020-09-08T15:41:05.660 INFO:teuthology.orchestra.run.smithi104.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-09-08T15:41:05.660 INFO:teuthology.orchestra.run.smithi104.stdout: "nearfull_ratio": 0.85000002384185791, 2020-09-08T15:41:05.661 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_snapshot": "", 2020-09-08T15:41:05.661 INFO:teuthology.orchestra.run.smithi104.stdout: "pool_max": 2, 2020-09-08T15:41:05.661 INFO:teuthology.orchestra.run.smithi104.stdout: "max_osd": 12, 2020-09-08T15:41:05.661 INFO:teuthology.orchestra.run.smithi104.stdout: "require_min_compat_client": "luminous", 2020-09-08T15:41:05.661 INFO:teuthology.orchestra.run.smithi104.stdout: "min_compat_client": "jewel", 2020-09-08T15:41:05.662 INFO:teuthology.orchestra.run.smithi104.stdout: "require_osd_release": "pacific", 2020-09-08T15:41:05.662 INFO:teuthology.orchestra.run.smithi104.stdout: "pools": [ 2020-09-08T15:41:05.662 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.662 INFO:teuthology.orchestra.run.smithi104.stdout: "pool": 1, 2020-09-08T15:41:05.663 INFO:teuthology.orchestra.run.smithi104.stdout: "pool_name": "device_health_metrics", 2020-09-08T15:41:05.663 INFO:teuthology.orchestra.run.smithi104.stdout: "create_time": "2020-09-08T15:25:56.314385+0000", 2020-09-08T15:41:05.663 INFO:teuthology.orchestra.run.smithi104.stdout: "flags": 1, 2020-09-08T15:41:05.663 INFO:teuthology.orchestra.run.smithi104.stdout: "flags_names": "hashpspool", 2020-09-08T15:41:05.663 INFO:teuthology.orchestra.run.smithi104.stdout: "type": 1, 2020-09-08T15:41:05.664 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 2, 2020-09-08T15:41:05.664 INFO:teuthology.orchestra.run.smithi104.stdout: "min_size": 2, 2020-09-08T15:41:05.664 INFO:teuthology.orchestra.run.smithi104.stdout: "crush_rule": 0, 2020-09-08T15:41:05.664 INFO:teuthology.orchestra.run.smithi104.stdout: "object_hash": 2, 2020-09-08T15:41:05.665 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_autoscale_mode": "off", 2020-09-08T15:41:05.665 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num": 1, 2020-09-08T15:41:05.665 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_placement_num": 1, 2020-09-08T15:41:05.665 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_placement_num_target": 1, 2020-09-08T15:41:05.666 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num_target": 1, 2020-09-08T15:41:05.666 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num_pending": 1, 2020-09-08T15:41:05.666 INFO:teuthology.orchestra.run.smithi104.stdout: "last_pg_merge_meta": { 2020-09-08T15:41:05.667 INFO:teuthology.orchestra.run.smithi104.stdout: "source_pgid": "0.0", 2020-09-08T15:41:05.667 INFO:teuthology.orchestra.run.smithi104.stdout: "ready_epoch": 0, 2020-09-08T15:41:05.667 INFO:teuthology.orchestra.run.smithi104.stdout: "last_epoch_started": 0, 2020-09-08T15:41:05.667 INFO:teuthology.orchestra.run.smithi104.stdout: "last_epoch_clean": 0, 2020-09-08T15:41:05.668 INFO:teuthology.orchestra.run.smithi104.stdout: "source_version": "0'0", 2020-09-08T15:41:05.668 INFO:teuthology.orchestra.run.smithi104.stdout: "target_version": "0'0" 2020-09-08T15:41:05.668 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.668 INFO:teuthology.orchestra.run.smithi104.stdout: "last_change": "13", 2020-09-08T15:41:05.669 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend": "0", 2020-09-08T15:41:05.669 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend_prenautilus": "0", 2020-09-08T15:41:05.670 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend_preluminous": "0", 2020-09-08T15:41:05.670 INFO:teuthology.orchestra.run.smithi104.stdout: "auid": 0, 2020-09-08T15:41:05.670 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_mode": "selfmanaged", 2020-09-08T15:41:05.670 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_seq": 0, 2020-09-08T15:41:05.671 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_epoch": 0, 2020-09-08T15:41:05.671 INFO:teuthology.orchestra.run.smithi104.stdout: "pool_snaps": [], 2020-09-08T15:41:05.671 INFO:teuthology.orchestra.run.smithi104.stdout: "removed_snaps": "[]", 2020-09-08T15:41:05.671 INFO:teuthology.orchestra.run.smithi104.stdout: "quota_max_bytes": 0, 2020-09-08T15:41:05.671 INFO:teuthology.orchestra.run.smithi104.stdout: "quota_max_objects": 0, 2020-09-08T15:41:05.672 INFO:teuthology.orchestra.run.smithi104.stdout: "tiers": [], 2020-09-08T15:41:05.672 INFO:teuthology.orchestra.run.smithi104.stdout: "tier_of": -1, 2020-09-08T15:41:05.672 INFO:teuthology.orchestra.run.smithi104.stdout: "read_tier": -1, 2020-09-08T15:41:05.672 INFO:teuthology.orchestra.run.smithi104.stdout: "write_tier": -1, 2020-09-08T15:41:05.672 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_mode": "none", 2020-09-08T15:41:05.673 INFO:teuthology.orchestra.run.smithi104.stdout: "target_max_bytes": 0, 2020-09-08T15:41:05.673 INFO:teuthology.orchestra.run.smithi104.stdout: "target_max_objects": 0, 2020-09-08T15:41:05.673 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-09-08T15:41:05.673 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-09-08T15:41:05.674 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_full_ratio_micro": 800000, 2020-09-08T15:41:05.674 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_min_flush_age": 0, 2020-09-08T15:41:05.674 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_min_evict_age": 0, 2020-09-08T15:41:05.674 INFO:teuthology.orchestra.run.smithi104.stdout: "erasure_code_profile": "", 2020-09-08T15:41:05.675 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_params": { 2020-09-08T15:41:05.675 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "none" 2020-09-08T15:41:05.675 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.675 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_period": 0, 2020-09-08T15:41:05.675 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_count": 0, 2020-09-08T15:41:05.676 INFO:teuthology.orchestra.run.smithi104.stdout: "use_gmt_hitset": true, 2020-09-08T15:41:05.676 INFO:teuthology.orchestra.run.smithi104.stdout: "min_read_recency_for_promote": 0, 2020-09-08T15:41:05.676 INFO:teuthology.orchestra.run.smithi104.stdout: "min_write_recency_for_promote": 0, 2020-09-08T15:41:05.676 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_grade_decay_rate": 0, 2020-09-08T15:41:05.676 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_search_last_n": 0, 2020-09-08T15:41:05.677 INFO:teuthology.orchestra.run.smithi104.stdout: "grade_table": [], 2020-09-08T15:41:05.677 INFO:teuthology.orchestra.run.smithi104.stdout: "stripe_width": 0, 2020-09-08T15:41:05.677 INFO:teuthology.orchestra.run.smithi104.stdout: "expected_num_objects": 0, 2020-09-08T15:41:05.678 INFO:teuthology.orchestra.run.smithi104.stdout: "fast_read": false, 2020-09-08T15:41:05.678 INFO:teuthology.orchestra.run.smithi104.stdout: "options": { 2020-09-08T15:41:05.678 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num_min": 1 2020-09-08T15:41:05.679 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.679 INFO:teuthology.orchestra.run.smithi104.stdout: "application_metadata": { 2020-09-08T15:41:05.679 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_devicehealth": {} 2020-09-08T15:41:05.679 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.679 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.680 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.680 INFO:teuthology.orchestra.run.smithi104.stdout: "pool": 2, 2020-09-08T15:41:05.680 INFO:teuthology.orchestra.run.smithi104.stdout: "pool_name": "unique_pool_0", 2020-09-08T15:41:05.680 INFO:teuthology.orchestra.run.smithi104.stdout: "create_time": "2020-09-08T15:37:28.858589+0000", 2020-09-08T15:41:05.680 INFO:teuthology.orchestra.run.smithi104.stdout: "flags": 1, 2020-09-08T15:41:05.681 INFO:teuthology.orchestra.run.smithi104.stdout: "flags_names": "hashpspool", 2020-09-08T15:41:05.681 INFO:teuthology.orchestra.run.smithi104.stdout: "type": 1, 2020-09-08T15:41:05.681 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 2, 2020-09-08T15:41:05.682 INFO:teuthology.orchestra.run.smithi104.stdout: "min_size": 2, 2020-09-08T15:41:05.682 INFO:teuthology.orchestra.run.smithi104.stdout: "crush_rule": 0, 2020-09-08T15:41:05.682 INFO:teuthology.orchestra.run.smithi104.stdout: "object_hash": 2, 2020-09-08T15:41:05.682 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_autoscale_mode": "off", 2020-09-08T15:41:05.682 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num": 16, 2020-09-08T15:41:05.683 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_placement_num": 16, 2020-09-08T15:41:05.683 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_placement_num_target": 16, 2020-09-08T15:41:05.683 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num_target": 16, 2020-09-08T15:41:05.683 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num_pending": 16, 2020-09-08T15:41:05.684 INFO:teuthology.orchestra.run.smithi104.stdout: "last_pg_merge_meta": { 2020-09-08T15:41:05.684 INFO:teuthology.orchestra.run.smithi104.stdout: "source_pgid": "0.0", 2020-09-08T15:41:05.684 INFO:teuthology.orchestra.run.smithi104.stdout: "ready_epoch": 0, 2020-09-08T15:41:05.684 INFO:teuthology.orchestra.run.smithi104.stdout: "last_epoch_started": 0, 2020-09-08T15:41:05.685 INFO:teuthology.orchestra.run.smithi104.stdout: "last_epoch_clean": 0, 2020-09-08T15:41:05.685 INFO:teuthology.orchestra.run.smithi104.stdout: "source_version": "0'0", 2020-09-08T15:41:05.685 INFO:teuthology.orchestra.run.smithi104.stdout: "target_version": "0'0" 2020-09-08T15:41:05.686 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.686 INFO:teuthology.orchestra.run.smithi104.stdout: "last_change": "131", 2020-09-08T15:41:05.686 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend": "0", 2020-09-08T15:41:05.686 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend_prenautilus": "0", 2020-09-08T15:41:05.687 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend_preluminous": "0", 2020-09-08T15:41:05.687 INFO:teuthology.orchestra.run.smithi104.stdout: "auid": 0, 2020-09-08T15:41:05.687 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_mode": "selfmanaged", 2020-09-08T15:41:05.688 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_seq": 0, 2020-09-08T15:41:05.688 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_epoch": 0, 2020-09-08T15:41:05.688 INFO:teuthology.orchestra.run.smithi104.stdout: "pool_snaps": [], 2020-09-08T15:41:05.688 INFO:teuthology.orchestra.run.smithi104.stdout: "removed_snaps": "[]", 2020-09-08T15:41:05.688 INFO:teuthology.orchestra.run.smithi104.stdout: "quota_max_bytes": 0, 2020-09-08T15:41:05.689 INFO:teuthology.orchestra.run.smithi104.stdout: "quota_max_objects": 0, 2020-09-08T15:41:05.689 INFO:teuthology.orchestra.run.smithi104.stdout: "tiers": [], 2020-09-08T15:41:05.689 INFO:teuthology.orchestra.run.smithi104.stdout: "tier_of": -1, 2020-09-08T15:41:05.689 INFO:teuthology.orchestra.run.smithi104.stdout: "read_tier": -1, 2020-09-08T15:41:05.689 INFO:teuthology.orchestra.run.smithi104.stdout: "write_tier": -1, 2020-09-08T15:41:05.690 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_mode": "none", 2020-09-08T15:41:05.690 INFO:teuthology.orchestra.run.smithi104.stdout: "target_max_bytes": 0, 2020-09-08T15:41:05.690 INFO:teuthology.orchestra.run.smithi104.stdout: "target_max_objects": 0, 2020-09-08T15:41:05.690 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-09-08T15:41:05.690 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-09-08T15:41:05.691 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_full_ratio_micro": 800000, 2020-09-08T15:41:05.691 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_min_flush_age": 0, 2020-09-08T15:41:05.691 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_min_evict_age": 0, 2020-09-08T15:41:05.691 INFO:teuthology.orchestra.run.smithi104.stdout: "erasure_code_profile": "", 2020-09-08T15:41:05.692 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_params": { 2020-09-08T15:41:05.692 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "none" 2020-09-08T15:41:05.692 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.692 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_period": 0, 2020-09-08T15:41:05.692 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_count": 0, 2020-09-08T15:41:05.693 INFO:teuthology.orchestra.run.smithi104.stdout: "use_gmt_hitset": true, 2020-09-08T15:41:05.693 INFO:teuthology.orchestra.run.smithi104.stdout: "min_read_recency_for_promote": 0, 2020-09-08T15:41:05.693 INFO:teuthology.orchestra.run.smithi104.stdout: "min_write_recency_for_promote": 0, 2020-09-08T15:41:05.693 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_grade_decay_rate": 0, 2020-09-08T15:41:05.693 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_search_last_n": 0, 2020-09-08T15:41:05.694 INFO:teuthology.orchestra.run.smithi104.stdout: "grade_table": [], 2020-09-08T15:41:05.694 INFO:teuthology.orchestra.run.smithi104.stdout: "stripe_width": 0, 2020-09-08T15:41:05.694 INFO:teuthology.orchestra.run.smithi104.stdout: "expected_num_objects": 0, 2020-09-08T15:41:05.694 INFO:teuthology.orchestra.run.smithi104.stdout: "fast_read": false, 2020-09-08T15:41:05.695 INFO:teuthology.orchestra.run.smithi104.stdout: "options": {}, 2020-09-08T15:41:05.695 INFO:teuthology.orchestra.run.smithi104.stdout: "application_metadata": { 2020-09-08T15:41:05.695 INFO:teuthology.orchestra.run.smithi104.stdout: "rados": {} 2020-09-08T15:41:05.695 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.695 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.696 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T15:41:05.696 INFO:teuthology.orchestra.run.smithi104.stdout: "osds": [ 2020-09-08T15:41:05.696 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.696 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 0, 2020-09-08T15:41:05.697 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "e49a7341-7c75-4883-b061-3e8fe9467e6e", 2020-09-08T15:41:05.697 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:05.697 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:05.698 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:05.698 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:05.698 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:05.698 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:05.698 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 11, 2020-09-08T15:41:05.699 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 55, 2020-09-08T15:41:05.699 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:05.699 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:05.699 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:05.700 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.700 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.700 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.700 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6801", 2020-09-08T15:41:05.701 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:41:05.701 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.701 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.701 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.701 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:05.702 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.702 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.702 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.702 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6802", 2020-09-08T15:41:05.703 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:41:05.703 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.703 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.703 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.704 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6803", 2020-09-08T15:41:05.704 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:41:05.704 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.704 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.704 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.705 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:05.705 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.705 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.706 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.706 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6806", 2020-09-08T15:41:05.706 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:41:05.706 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.707 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.707 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.707 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6807", 2020-09-08T15:41:05.707 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:41:05.707 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.708 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.708 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.708 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:05.709 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.709 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.709 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.709 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6804", 2020-09-08T15:41:05.710 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:41:05.710 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.710 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.710 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.710 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6805", 2020-09-08T15:41:05.711 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:41:05.711 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.711 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.711 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.712 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.104:6801/82470352", 2020-09-08T15:41:05.712 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.104:6803/82470352", 2020-09-08T15:41:05.712 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.104:6807/82470352", 2020-09-08T15:41:05.712 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.104:6805/82470352", 2020-09-08T15:41:05.712 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:05.713 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:05.713 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:05.713 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.713 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.713 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.714 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 1, 2020-09-08T15:41:05.714 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "a83df412-110d-440a-b411-ad2a7c36f81a", 2020-09-08T15:41:05.714 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:05.714 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:05.715 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:05.715 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:05.715 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:05.715 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:05.715 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 17, 2020-09-08T15:41:05.716 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 126, 2020-09-08T15:41:05.716 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:05.716 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:05.716 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:05.717 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.717 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.717 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.717 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6809", 2020-09-08T15:41:05.717 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:41:05.719 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.719 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.719 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.720 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:05.720 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.720 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.720 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.721 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6810", 2020-09-08T15:41:05.721 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:41:05.721 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.721 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.722 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.722 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6811", 2020-09-08T15:41:05.722 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:41:05.722 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.723 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.723 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.723 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:05.723 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.723 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.724 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.724 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6814", 2020-09-08T15:41:05.724 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:41:05.724 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.725 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.725 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.725 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6815", 2020-09-08T15:41:05.726 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:41:05.726 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.726 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.726 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.727 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:05.727 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.727 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.727 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.728 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6812", 2020-09-08T15:41:05.728 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:41:05.728 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.729 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.729 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.729 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6813", 2020-09-08T15:41:05.729 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:41:05.730 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.730 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.730 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.730 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.104:6809/1421798992", 2020-09-08T15:41:05.730 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.104:6811/1421798992", 2020-09-08T15:41:05.731 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.104:6815/1421798992", 2020-09-08T15:41:05.731 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.104:6813/1421798992", 2020-09-08T15:41:05.731 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:05.731 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:05.731 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:05.731 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.732 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.732 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.732 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 2, 2020-09-08T15:41:05.732 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "ca6da1cc-1d8d-494b-93e6-e065ff70d434", 2020-09-08T15:41:05.732 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:05.733 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:05.733 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:05.733 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:05.733 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:05.733 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:05.734 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 23, 2020-09-08T15:41:05.734 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 126, 2020-09-08T15:41:05.734 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:05.734 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:05.734 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:05.735 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.735 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.735 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.735 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6817", 2020-09-08T15:41:05.735 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:41:05.735 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.736 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.736 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.737 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:05.737 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.737 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.738 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.738 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6818", 2020-09-08T15:41:05.738 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:41:05.738 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.738 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6819", 2020-09-08T15:41:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:41:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.740 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.740 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:05.740 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.740 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.740 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.741 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6822", 2020-09-08T15:41:05.741 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:41:05.741 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.741 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.741 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.742 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6823", 2020-09-08T15:41:05.742 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:41:05.742 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.742 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6820", 2020-09-08T15:41:05.744 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:41:05.744 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.744 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.744 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.744 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6821", 2020-09-08T15:41:05.745 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:41:05.745 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.745 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.746 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.746 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.104:6817/2713413369", 2020-09-08T15:41:05.746 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.104:6819/2713413369", 2020-09-08T15:41:05.746 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.104:6823/2713413369", 2020-09-08T15:41:05.746 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.104:6821/2713413369", 2020-09-08T15:41:05.746 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:05.747 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:05.747 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:05.747 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.747 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.747 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.748 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 3, 2020-09-08T15:41:05.748 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e", 2020-09-08T15:41:05.748 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:05.748 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:05.748 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:05.749 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:05.749 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:05.749 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:05.749 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 29, 2020-09-08T15:41:05.749 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 0, 2020-09-08T15:41:05.750 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:05.750 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:05.750 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:05.750 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.750 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.750 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.751 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6825", 2020-09-08T15:41:05.751 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:41:05.751 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.751 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.751 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.752 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:05.752 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.752 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.752 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.752 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6826", 2020-09-08T15:41:05.752 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:41:05.753 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.753 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.753 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.753 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6827", 2020-09-08T15:41:05.753 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:41:05.754 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.754 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.754 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.754 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:05.754 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.755 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.755 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.755 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6830", 2020-09-08T15:41:05.755 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:41:05.755 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.756 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.756 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.756 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6831", 2020-09-08T15:41:05.756 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:41:05.756 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.756 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.757 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.757 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:05.757 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.757 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.757 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.758 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6828", 2020-09-08T15:41:05.758 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:41:05.758 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.758 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.758 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6829", 2020-09-08T15:41:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:41:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.104:6825/1495946248", 2020-09-08T15:41:05.760 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.104:6827/1495946248", 2020-09-08T15:41:05.760 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.104:6831/1495946248", 2020-09-08T15:41:05.760 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.104:6829/1495946248", 2020-09-08T15:41:05.760 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:05.761 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:05.761 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:05.761 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.761 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.761 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.762 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 4, 2020-09-08T15:41:05.762 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "d8334c70-f776-4652-944d-aed35a8e43f7", 2020-09-08T15:41:05.762 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:05.762 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:05.762 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:05.763 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 0, 2020-09-08T15:41:05.763 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:05.763 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:05.763 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 35, 2020-09-08T15:41:05.763 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 0, 2020-09-08T15:41:05.764 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:05.764 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:05.764 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:05.764 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.764 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.764 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.765 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6803", 2020-09-08T15:41:05.765 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:41:05.765 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.765 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.765 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.766 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:05.766 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.766 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.766 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.766 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6804", 2020-09-08T15:41:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:41:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6805", 2020-09-08T15:41:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:41:05.768 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.768 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.768 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.768 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:05.768 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.769 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.769 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.770 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6808", 2020-09-08T15:41:05.770 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:41:05.770 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.770 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.770 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.771 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6809", 2020-09-08T15:41:05.771 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:41:05.771 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.771 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.772 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.772 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:05.772 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.772 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.772 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.773 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6806", 2020-09-08T15:41:05.773 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:41:05.773 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.773 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.773 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.773 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6807", 2020-09-08T15:41:05.774 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:41:05.774 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.774 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.774 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.774 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.55:6803/1173103260", 2020-09-08T15:41:05.775 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.55:6805/1173103260", 2020-09-08T15:41:05.775 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.55:6809/1173103260", 2020-09-08T15:41:05.775 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.55:6807/1173103260", 2020-09-08T15:41:05.775 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:05.775 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:05.776 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:05.776 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.776 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.776 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.776 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 5, 2020-09-08T15:41:05.776 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "6789abd2-9e9b-4c04-81fb-0ce2d2849e39", 2020-09-08T15:41:05.777 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:05.777 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:05.777 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:05.777 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:05.777 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:05.778 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:05.778 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 42, 2020-09-08T15:41:05.778 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 126, 2020-09-08T15:41:05.778 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:05.778 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:05.779 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:05.779 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.779 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.779 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.779 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6811", 2020-09-08T15:41:05.779 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:41:05.780 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.780 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.780 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.780 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:05.780 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.781 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.781 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.781 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6812", 2020-09-08T15:41:05.781 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:41:05.782 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.788 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.788 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.788 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6813", 2020-09-08T15:41:05.788 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:41:05.789 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.789 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.789 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.789 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:05.790 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.790 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.790 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.790 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6816", 2020-09-08T15:41:05.790 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:41:05.791 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.791 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.791 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.791 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6817", 2020-09-08T15:41:05.791 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:41:05.792 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.792 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.792 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.792 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:05.792 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.793 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.793 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.793 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6814", 2020-09-08T15:41:05.793 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:41:05.793 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.793 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.794 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.794 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6815", 2020-09-08T15:41:05.794 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:41:05.794 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.794 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.795 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.795 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.55:6811/1047046376", 2020-09-08T15:41:05.795 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.55:6813/1047046376", 2020-09-08T15:41:05.795 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.55:6817/1047046376", 2020-09-08T15:41:05.795 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.55:6815/1047046376", 2020-09-08T15:41:05.796 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:05.796 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:05.796 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:05.796 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.796 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.797 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.797 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 6, 2020-09-08T15:41:05.797 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "ad9a93fe-0ca0-4052-bfa9-e2abb268829e", 2020-09-08T15:41:05.797 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:05.797 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:05.797 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:05.798 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:05.798 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:05.798 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:05.798 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 49, 2020-09-08T15:41:05.798 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 126, 2020-09-08T15:41:05.799 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:05.799 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:05.799 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:05.799 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.799 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.800 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.800 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6819", 2020-09-08T15:41:05.800 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:41:05.800 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.800 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.800 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.801 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:05.801 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.801 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.801 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.801 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6820", 2020-09-08T15:41:05.802 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:41:05.802 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.802 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.802 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.802 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6821", 2020-09-08T15:41:05.803 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:41:05.803 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.804 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.804 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.804 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:05.804 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.805 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.805 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.805 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6824", 2020-09-08T15:41:05.805 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:41:05.805 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.806 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.806 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.806 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6825", 2020-09-08T15:41:05.806 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:41:05.806 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.807 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.807 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.807 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:05.807 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.807 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.808 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.808 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6822", 2020-09-08T15:41:05.808 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:41:05.808 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.808 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.808 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.809 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6823", 2020-09-08T15:41:05.809 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:41:05.809 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.810 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.810 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.810 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.55:6819/2501844793", 2020-09-08T15:41:05.810 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.55:6821/2501844793", 2020-09-08T15:41:05.810 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.55:6825/2501844793", 2020-09-08T15:41:05.811 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.55:6823/2501844793", 2020-09-08T15:41:05.811 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:05.811 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:05.811 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:05.811 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.812 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.812 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.812 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 7, 2020-09-08T15:41:05.812 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "f3795c86-4540-442c-a151-b2769fa577e4", 2020-09-08T15:41:05.812 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:05.812 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:05.813 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:05.813 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:05.813 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:05.813 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:05.813 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 56, 2020-09-08T15:41:05.814 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 126, 2020-09-08T15:41:05.814 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:05.814 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:05.814 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:05.814 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.815 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.815 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.815 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6827", 2020-09-08T15:41:05.815 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:41:05.815 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.815 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.816 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.816 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:05.816 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.816 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.816 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.817 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6828", 2020-09-08T15:41:05.817 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:41:05.817 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.817 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.817 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.818 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6829", 2020-09-08T15:41:05.818 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:41:05.818 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.818 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.818 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.819 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:05.819 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.819 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.819 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.820 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6832", 2020-09-08T15:41:05.820 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:41:05.820 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.820 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.820 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.821 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6833", 2020-09-08T15:41:05.821 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:41:05.821 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.821 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.821 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.822 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:05.822 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.822 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.822 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.822 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6830", 2020-09-08T15:41:05.823 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:41:05.823 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.823 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.823 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.823 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6831", 2020-09-08T15:41:05.823 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:41:05.824 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.824 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.824 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.824 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.55:6827/128909153", 2020-09-08T15:41:05.824 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.55:6829/128909153", 2020-09-08T15:41:05.825 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.55:6833/128909153", 2020-09-08T15:41:05.825 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.55:6831/128909153", 2020-09-08T15:41:05.825 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:05.825 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:05.825 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:05.826 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.826 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.826 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.826 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 8, 2020-09-08T15:41:05.826 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "3b0ac380-878e-4c1c-9278-80ce67ca2df4", 2020-09-08T15:41:05.827 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:05.827 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:05.827 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:05.827 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:05.827 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:05.828 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:05.828 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 64, 2020-09-08T15:41:05.828 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 126, 2020-09-08T15:41:05.828 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:05.828 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:05.828 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:05.829 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.829 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.829 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.829 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6801", 2020-09-08T15:41:05.829 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:41:05.830 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.830 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.830 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.830 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:05.831 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.831 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.831 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.831 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6802", 2020-09-08T15:41:05.831 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:41:05.832 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.832 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.832 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.832 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6803", 2020-09-08T15:41:05.832 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:41:05.833 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.833 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.833 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.833 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:05.833 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.834 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.834 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.834 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6806", 2020-09-08T15:41:05.834 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:41:05.834 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.834 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.835 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.835 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6807", 2020-09-08T15:41:05.835 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:41:05.835 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.835 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.836 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.836 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:05.836 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.836 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.836 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.836 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6804", 2020-09-08T15:41:05.837 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:41:05.837 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.837 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.837 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.837 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6805", 2020-09-08T15:41:05.838 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:41:05.838 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.838 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.838 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.838 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.166:6801/2108117574", 2020-09-08T15:41:05.839 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.166:6803/2108117574", 2020-09-08T15:41:05.839 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.166:6807/2108117574", 2020-09-08T15:41:05.839 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.166:6805/2108117574", 2020-09-08T15:41:05.839 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:05.839 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:05.839 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:05.840 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.840 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.840 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.841 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 9, 2020-09-08T15:41:05.841 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08", 2020-09-08T15:41:05.841 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:05.841 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:05.841 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:05.842 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:05.842 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:05.842 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:05.842 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 72, 2020-09-08T15:41:05.842 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 126, 2020-09-08T15:41:05.843 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:05.843 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:05.843 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:05.843 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.843 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.843 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.844 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6809", 2020-09-08T15:41:05.844 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:41:05.844 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.844 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.844 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.845 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:05.845 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.845 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.845 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.845 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6810", 2020-09-08T15:41:05.846 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:41:05.846 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.846 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.846 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.846 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6811", 2020-09-08T15:41:05.846 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:41:05.847 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.847 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.847 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.847 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:05.847 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.848 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.848 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.848 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6814", 2020-09-08T15:41:05.848 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:41:05.848 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.848 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.849 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.849 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6815", 2020-09-08T15:41:05.849 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:41:05.849 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.849 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.850 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.850 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:05.850 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.851 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.851 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.851 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6812", 2020-09-08T15:41:05.851 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:41:05.851 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.852 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.852 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.853 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6813", 2020-09-08T15:41:05.853 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:41:05.853 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.853 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.853 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.854 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.166:6809/1458870080", 2020-09-08T15:41:05.854 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.166:6811/1458870080", 2020-09-08T15:41:05.854 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.166:6815/1458870080", 2020-09-08T15:41:05.854 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.166:6813/1458870080", 2020-09-08T15:41:05.854 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:05.854 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:05.855 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:05.855 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.855 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.855 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.855 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 10, 2020-09-08T15:41:05.856 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "c194210e-bd89-4a1f-98fb-6bf44f0b324c", 2020-09-08T15:41:05.856 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:05.856 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:05.856 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:05.856 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:05.857 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:05.857 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:05.857 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 81, 2020-09-08T15:41:05.857 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 0, 2020-09-08T15:41:05.857 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:05.858 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:05.858 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:05.858 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.858 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.858 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.858 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6817", 2020-09-08T15:41:05.859 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:41:05.859 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.859 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.859 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.859 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:05.860 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.860 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.860 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.860 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6818", 2020-09-08T15:41:05.861 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:41:05.861 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.861 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.862 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.862 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6819", 2020-09-08T15:41:05.862 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:41:05.862 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.862 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.863 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.863 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:05.863 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.863 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.863 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.863 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6822", 2020-09-08T15:41:05.864 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:41:05.864 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.864 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.864 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.864 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6823", 2020-09-08T15:41:05.865 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:41:05.865 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.865 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.865 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.865 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:05.866 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.866 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.866 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.866 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6820", 2020-09-08T15:41:05.866 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:41:05.867 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.867 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.867 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.867 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6821", 2020-09-08T15:41:05.868 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:41:05.868 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.868 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.868 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.868 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.166:6817/62517848", 2020-09-08T15:41:05.869 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.166:6819/62517848", 2020-09-08T15:41:05.869 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.166:6823/62517848", 2020-09-08T15:41:05.869 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.166:6821/62517848", 2020-09-08T15:41:05.869 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:05.869 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:05.870 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:05.870 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.870 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.870 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.870 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 11, 2020-09-08T15:41:05.870 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "121301fe-a41c-40c0-ad09-b00de37a55e1", 2020-09-08T15:41:05.871 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:05.871 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:05.871 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:05.872 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:05.872 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:05.872 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:05.872 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 90, 2020-09-08T15:41:05.872 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 126, 2020-09-08T15:41:05.872 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:05.873 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:05.874 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:05.874 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.874 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.874 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.874 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6825", 2020-09-08T15:41:05.874 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:41:05.875 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.875 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.875 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.875 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:05.875 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.876 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.876 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.876 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6826", 2020-09-08T15:41:05.876 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:41:05.876 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.877 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.877 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.877 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6827", 2020-09-08T15:41:05.877 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:41:05.877 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.877 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.878 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.878 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:05.878 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.878 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.878 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.879 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6830", 2020-09-08T15:41:05.879 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:41:05.879 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.879 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.879 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.880 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6831", 2020-09-08T15:41:05.880 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:41:05.880 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.880 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.880 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.881 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:05.881 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:05.881 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.882 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:05.882 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6828", 2020-09-08T15:41:05.882 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:41:05.882 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.882 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.883 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:05.883 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6829", 2020-09-08T15:41:05.883 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:41:05.883 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.883 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.884 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.884 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.166:6825/691735125", 2020-09-08T15:41:05.884 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.166:6827/691735125", 2020-09-08T15:41:05.884 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.166:6831/691735125", 2020-09-08T15:41:05.884 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.166:6829/691735125", 2020-09-08T15:41:05.885 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:05.885 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:05.885 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:05.885 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:05.885 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.886 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T15:41:05.886 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_xinfo": [ 2020-09-08T15:41:05.886 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.886 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 0, 2020-09-08T15:41:05.886 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:05.886 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:05.887 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:05.887 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:05.887 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:05.887 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:40:06.135513+0000", 2020-09-08T15:41:05.887 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:05.888 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.888 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.888 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 1, 2020-09-08T15:41:05.888 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:05.888 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:05.889 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:05.889 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:05.889 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:05.889 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:40:34.045201+0000", 2020-09-08T15:41:05.889 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:05.889 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.890 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.890 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 2, 2020-09-08T15:41:05.890 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:05.890 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:05.890 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:05.891 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:05.891 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:05.891 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:40:56.946674+0000", 2020-09-08T15:41:05.892 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:05.892 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.892 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.892 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 3, 2020-09-08T15:41:05.892 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:05.893 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:05.893 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:05.893 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:05.893 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:05.893 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:40:21.192386+0000", 2020-09-08T15:41:05.893 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:05.894 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.894 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.894 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 4, 2020-09-08T15:41:05.894 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:05.894 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:05.895 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:05.895 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:05.895 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:05.895 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:40:46.317711+0000", 2020-09-08T15:41:05.895 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:05.896 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.896 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.896 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 5, 2020-09-08T15:41:05.896 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:05.896 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:05.896 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:05.897 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:05.897 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:05.897 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:40:12.236239+0000", 2020-09-08T15:41:05.897 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:05.897 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.898 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.898 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 6, 2020-09-08T15:41:05.898 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:05.898 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:05.898 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:05.899 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:05.899 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:05.899 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:40:40.153723+0000", 2020-09-08T15:41:05.899 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:05.899 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.899 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.900 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 7, 2020-09-08T15:41:05.900 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:05.900 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:05.900 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:05.900 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:05.901 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:05.901 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:41:05.533799+0000", 2020-09-08T15:41:05.901 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:05.901 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.901 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.902 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 8, 2020-09-08T15:41:05.902 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:05.902 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:05.902 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:05.903 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:05.903 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:05.903 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:40:27.086853+0000", 2020-09-08T15:41:05.903 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:05.903 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.904 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.904 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 9, 2020-09-08T15:41:05.904 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:05.904 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:05.904 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:05.905 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:05.905 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:05.905 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:40:54.670244+0000", 2020-09-08T15:41:05.905 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:05.906 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.906 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.906 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 10, 2020-09-08T15:41:05.906 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:05.906 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:05.907 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:05.907 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:05.907 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:05.907 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:40:18.017790+0000", 2020-09-08T15:41:05.907 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:05.908 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.908 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:05.908 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 11, 2020-09-08T15:41:05.908 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:05.908 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:05.908 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:05.909 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:05.909 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:05.909 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:40:42.450226+0000", 2020-09-08T15:41:05.909 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:05.909 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.910 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T15:41:05.910 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_upmap": [], 2020-09-08T15:41:05.910 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_upmap_items": [], 2020-09-08T15:41:05.910 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_temp": [], 2020-09-08T15:41:05.910 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_temp": [], 2020-09-08T15:41:05.911 INFO:teuthology.orchestra.run.smithi104.stdout: "blocklist": { 2020-09-08T15:41:05.911 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/1257365091": "2020-09-09T15:29:39.430456+0000", 2020-09-08T15:41:05.911 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/1399123455": "2020-09-09T15:29:39.430456+0000", 2020-09-08T15:41:05.911 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/2810441605": "2020-09-09T15:29:39.430456+0000", 2020-09-08T15:41:05.911 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6800/182241824": "2020-09-09T15:29:39.430456+0000", 2020-09-08T15:41:05.911 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6801/182241824": "2020-09-09T15:29:39.430456+0000", 2020-09-08T15:41:05.912 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/2021933044": "2020-09-09T15:26:33.248197+0000", 2020-09-08T15:41:05.912 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6800/3129754347": "2020-09-09T15:26:33.248197+0000", 2020-09-08T15:41:05.912 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/1270031705": "2020-09-09T15:26:33.248197+0000", 2020-09-08T15:41:05.912 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/2568432190": "2020-09-09T15:26:33.248197+0000", 2020-09-08T15:41:05.912 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6800/1138921844": "2020-09-09T15:26:09.867236+0000", 2020-09-08T15:41:05.913 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/3827328474": "2020-09-09T15:26:09.867236+0000", 2020-09-08T15:41:05.913 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6801/1138921844": "2020-09-09T15:26:09.867236+0000", 2020-09-08T15:41:05.913 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/1662035381": "2020-09-09T15:26:09.867236+0000", 2020-09-08T15:41:05.913 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6801/3129754347": "2020-09-09T15:26:33.248197+0000", 2020-09-08T15:41:05.913 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/2169968102": "2020-09-09T15:26:09.867236+0000" 2020-09-08T15:41:05.914 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.914 INFO:teuthology.orchestra.run.smithi104.stdout: "erasure_code_profiles": { 2020-09-08T15:41:05.914 INFO:teuthology.orchestra.run.smithi104.stdout: "default": { 2020-09-08T15:41:05.914 INFO:teuthology.orchestra.run.smithi104.stdout: "crush-failure-domain": "osd", 2020-09-08T15:41:05.914 INFO:teuthology.orchestra.run.smithi104.stdout: "k": "2", 2020-09-08T15:41:05.915 INFO:teuthology.orchestra.run.smithi104.stdout: "m": "1", 2020-09-08T15:41:05.915 INFO:teuthology.orchestra.run.smithi104.stdout: "plugin": "jerasure", 2020-09-08T15:41:05.915 INFO:teuthology.orchestra.run.smithi104.stdout: "ruleset-failure-domain": "osd", 2020-09-08T15:41:05.915 INFO:teuthology.orchestra.run.smithi104.stdout: "technique": "reed_sol_van" 2020-09-08T15:41:05.915 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:05.915 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:05.916 INFO:teuthology.orchestra.run.smithi104.stdout: "removed_snaps_queue": [], 2020-09-08T15:41:05.916 INFO:teuthology.orchestra.run.smithi104.stdout: "new_removed_snaps": [], 2020-09-08T15:41:05.916 INFO:teuthology.orchestra.run.smithi104.stdout: "new_purged_snaps": [], 2020-09-08T15:41:05.916 INFO:teuthology.orchestra.run.smithi104.stdout: "crush_node_flags": {}, 2020-09-08T15:41:05.916 INFO:teuthology.orchestra.run.smithi104.stdout: "device_class_flags": {} 2020-09-08T15:41:05.917 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:41:05.917 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:41:05.919 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:05 smithi055 bash[9296]: cluster 2020-09-08T15:41:04.164454+0000 osd.1 (osd.1) 38 : cluster [DBG] 2.4 scrub starts 2020-09-08T15:41:05.919 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:05 smithi166 bash[9446]: cluster 2020-09-08T15:41:04.164454+0000 osd.1 (osd.1) 38 : cluster [DBG] 2.4 scrub starts 2020-09-08T15:41:05.919 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:41:05 smithi166 bash[14254]: debug 2020-09-08T15:41:05.636+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:05.920 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:05 smithi104 bash[9203]: cluster 2020-09-08T15:41:04.164454+0000 osd.1 (osd.1) 38 : cluster [DBG] 2.4 scrub starts 2020-09-08T15:41:06.029 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:41:06.137 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:06.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:41:06 smithi055 bash[19003]: debug 2020-09-08T15:41:06.026+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:06.552 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:41:06 smithi166 bash[18960]: debug 2020-09-08T15:41:06.552+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:06.555 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:06.662 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:41:06.775 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:41:06.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:06 smithi166 bash[9446]: cluster 2020-09-08T15:41:05.185402+0000 osd.1 (osd.1) 39 : cluster [DBG] 2.4 scrub ok 2020-09-08T15:41:06.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:06 smithi166 bash[9446]: cluster 2020-09-08T15:41:05.533704+0000 osd.7 (osd.7) 39 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:06.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:06 smithi166 bash[9446]: cluster 2020-09-08T15:41:05.533796+0000 osd.7 (osd.7) 40 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:06.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:06 smithi166 bash[9446]: cluster 2020-09-08T15:41:05.645662+0000 mgr.x (mgr.24118) 409 : cluster [DBG] pgmap v527: 17 pgs: 17 active+clean; 2.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 786 op/s 2020-09-08T15:41:06.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:06 smithi166 bash[9446]: cluster 2020-09-08T15:41:05.651031+0000 mon.a (mon.0) 1065 : cluster [DBG] osdmap e199: 12 total, 12 up, 12 in 2020-09-08T15:41:06.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:06 smithi166 bash[9446]: audit 2020-09-08T15:41:05.651385+0000 mon.a (mon.0) 1066 : audit [DBG] from='client.? v1:172.21.15.104:0/2326363792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2020-09-08T15:41:06.880 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:06.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:41:06 smithi055 bash[19003]: debug 2020-09-08T15:41:06.774+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:06.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:06 smithi055 bash[9296]: cluster 2020-09-08T15:41:05.185402+0000 osd.1 (osd.1) 39 : cluster [DBG] 2.4 scrub ok 2020-09-08T15:41:06.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:06 smithi055 bash[9296]: cluster 2020-09-08T15:41:05.533704+0000 osd.7 (osd.7) 39 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:06.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:06 smithi055 bash[9296]: cluster 2020-09-08T15:41:05.533796+0000 osd.7 (osd.7) 40 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:06.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:06 smithi055 bash[9296]: cluster 2020-09-08T15:41:05.645662+0000 mgr.x (mgr.24118) 409 : cluster [DBG] pgmap v527: 17 pgs: 17 active+clean; 2.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 786 op/s 2020-09-08T15:41:06.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:06 smithi055 bash[9296]: cluster 2020-09-08T15:41:05.651031+0000 mon.a (mon.0) 1065 : cluster [DBG] osdmap e199: 12 total, 12 up, 12 in 2020-09-08T15:41:06.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:06 smithi055 bash[9296]: audit 2020-09-08T15:41:05.651385+0000 mon.a (mon.0) 1066 : audit [DBG] from='client.? v1:172.21.15.104:0/2326363792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2020-09-08T15:41:06.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:06 smithi104 bash[9203]: cluster 2020-09-08T15:41:05.185402+0000 osd.1 (osd.1) 39 : cluster [DBG] 2.4 scrub ok 2020-09-08T15:41:06.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:06 smithi104 bash[9203]: cluster 2020-09-08T15:41:05.533704+0000 osd.7 (osd.7) 39 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:06.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:06 smithi104 bash[9203]: cluster 2020-09-08T15:41:05.533796+0000 osd.7 (osd.7) 40 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:06.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:06 smithi104 bash[9203]: cluster 2020-09-08T15:41:05.645662+0000 mgr.x (mgr.24118) 409 : cluster [DBG] pgmap v527: 17 pgs: 17 active+clean; 2.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 786 op/s 2020-09-08T15:41:06.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:06 smithi104 bash[9203]: cluster 2020-09-08T15:41:05.651031+0000 mon.a (mon.0) 1065 : cluster [DBG] osdmap e199: 12 total, 12 up, 12 in 2020-09-08T15:41:06.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:06 smithi104 bash[9203]: audit 2020-09-08T15:41:05.651385+0000 mon.a (mon.0) 1066 : audit [DBG] from='client.? v1:172.21.15.104:0/2326363792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2020-09-08T15:41:07.101 INFO:tasks.thrashosds.thrasher:j is {'epoch': 199, 'fsid': '759b27b6-f1e7-11ea-a080-001a4aab830c', 'created': '2020-09-08T15:25:41.788365+0000', 'modified': '2020-09-08T15:41:05.638807+0000', 'last_up_change': '2020-09-08T15:34:41.591462+0000', 'last_in_change': '2020-09-08T15:34:41.591462+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 27, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 12, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-09-08T15:25:56.314385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'unique_pool_0', 'create_time': '2020-09-08T15:37:28.858589+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'e49a7341-7c75-4883-b061-3e8fe9467e6e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6801', 'nonce': 82470352}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6802', 'nonce': 82470352}, {'type': 'v1', 'addr': '172.21.15.104:6803', 'nonce': 82470352}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6806', 'nonce': 82470352}, {'type': 'v1', 'addr': '172.21.15.104:6807', 'nonce': 82470352}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6804', 'nonce': 82470352}, {'type': 'v1', 'addr': '172.21.15.104:6805', 'nonce': 82470352}]}, 'public_addr': '172.21.15.104:6801/82470352', 'cluster_addr': '172.21.15.104:6803/82470352', 'heartbeat_back_addr': '172.21.15.104:6807/82470352', 'heartbeat_front_addr': '172.21.15.104:6805/82470352', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a83df412-110d-440a-b411-ad2a7c36f81a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6809', 'nonce': 1421798992}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6810', 'nonce': 1421798992}, {'type': 'v1', 'addr': '172.21.15.104:6811', 'nonce': 1421798992}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6814', 'nonce': 1421798992}, {'type': 'v1', 'addr': '172.21.15.104:6815', 'nonce': 1421798992}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6812', 'nonce': 1421798992}, {'type': 'v1', 'addr': '172.21.15.104:6813', 'nonce': 1421798992}]}, 'public_addr': '172.21.15.104:6809/1421798992', 'cluster_addr': '172.21.15.104:6811/1421798992', 'heartbeat_back_addr': '172.21.15.104:6815/1421798992', 'heartbeat_front_addr': '172.21.15.104:6813/1421798992', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'ca6da1cc-1d8d-494b-93e6-e065ff70d434', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6817', 'nonce': 2713413369}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6818', 'nonce': 2713413369}, {'type': 'v1', 'addr': '172.21.15.104:6819', 'nonce': 2713413369}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6822', 'nonce': 2713413369}, {'type': 'v1', 'addr': '172.21.15.104:6823', 'nonce': 2713413369}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6820', 'nonce': 2713413369}, {'type': 'v1', 'addr': '172.21.15.104:6821', 'nonce': 2713413369}]}, 'public_addr': '172.21.15.104:6817/2713413369', 'cluster_addr': '172.21.15.104:6819/2713413369', 'heartbeat_back_addr': '172.21.15.104:6823/2713413369', 'heartbeat_front_addr': '172.21.15.104:6821/2713413369', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 29, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6825', 'nonce': 1495946248}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6826', 'nonce': 1495946248}, {'type': 'v1', 'addr': '172.21.15.104:6827', 'nonce': 1495946248}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6830', 'nonce': 1495946248}, {'type': 'v1', 'addr': '172.21.15.104:6831', 'nonce': 1495946248}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6828', 'nonce': 1495946248}, {'type': 'v1', 'addr': '172.21.15.104:6829', 'nonce': 1495946248}]}, 'public_addr': '172.21.15.104:6825/1495946248', 'cluster_addr': '172.21.15.104:6827/1495946248', 'heartbeat_back_addr': '172.21.15.104:6831/1495946248', 'heartbeat_front_addr': '172.21.15.104:6829/1495946248', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd8334c70-f776-4652-944d-aed35a8e43f7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 35, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.55:6803', 'nonce': 1173103260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6804', 'nonce': 1173103260}, {'type': 'v1', 'addr': '172.21.15.55:6805', 'nonce': 1173103260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6808', 'nonce': 1173103260}, {'type': 'v1', 'addr': '172.21.15.55:6809', 'nonce': 1173103260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6806', 'nonce': 1173103260}, {'type': 'v1', 'addr': '172.21.15.55:6807', 'nonce': 1173103260}]}, 'public_addr': '172.21.15.55:6803/1173103260', 'cluster_addr': '172.21.15.55:6805/1173103260', 'heartbeat_back_addr': '172.21.15.55:6809/1173103260', 'heartbeat_front_addr': '172.21.15.55:6807/1173103260', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '6789abd2-9e9b-4c04-81fb-0ce2d2849e39', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 42, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.55:6811', 'nonce': 1047046376}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6812', 'nonce': 1047046376}, {'type': 'v1', 'addr': '172.21.15.55:6813', 'nonce': 1047046376}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6816', 'nonce': 1047046376}, {'type': 'v1', 'addr': '172.21.15.55:6817', 'nonce': 1047046376}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6814', 'nonce': 1047046376}, {'type': 'v1', 'addr': '172.21.15.55:6815', 'nonce': 1047046376}]}, 'public_addr': '172.21.15.55:6811/1047046376', 'cluster_addr': '172.21.15.55:6813/1047046376', 'heartbeat_back_addr': '172.21.15.55:6817/1047046376', 'heartbeat_front_addr': '172.21.15.55:6815/1047046376', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ad9a93fe-0ca0-4052-bfa9-e2abb268829e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 49, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.55:6819', 'nonce': 2501844793}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6820', 'nonce': 2501844793}, {'type': 'v1', 'addr': '172.21.15.55:6821', 'nonce': 2501844793}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6824', 'nonce': 2501844793}, {'type': 'v1', 'addr': '172.21.15.55:6825', 'nonce': 2501844793}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6822', 'nonce': 2501844793}, {'type': 'v1', 'addr': '172.21.15.55:6823', 'nonce': 2501844793}]}, 'public_addr': '172.21.15.55:6819/2501844793', 'cluster_addr': '172.21.15.55:6821/2501844793', 'heartbeat_back_addr': '172.21.15.55:6825/2501844793', 'heartbeat_front_addr': '172.21.15.55:6823/2501844793', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'f3795c86-4540-442c-a151-b2769fa577e4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 56, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.55:6827', 'nonce': 128909153}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6828', 'nonce': 128909153}, {'type': 'v1', 'addr': '172.21.15.55:6829', 'nonce': 128909153}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6832', 'nonce': 128909153}, {'type': 'v1', 'addr': '172.21.15.55:6833', 'nonce': 128909153}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6830', 'nonce': 128909153}, {'type': 'v1', 'addr': '172.21.15.55:6831', 'nonce': 128909153}]}, 'public_addr': '172.21.15.55:6827/128909153', 'cluster_addr': '172.21.15.55:6829/128909153', 'heartbeat_back_addr': '172.21.15.55:6833/128909153', 'heartbeat_front_addr': '172.21.15.55:6831/128909153', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '3b0ac380-878e-4c1c-9278-80ce67ca2df4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 64, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.166:6801', 'nonce': 2108117574}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6802', 'nonce': 2108117574}, {'type': 'v1', 'addr': '172.21.15.166:6803', 'nonce': 2108117574}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6806', 'nonce': 2108117574}, {'type': 'v1', 'addr': '172.21.15.166:6807', 'nonce': 2108117574}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6804', 'nonce': 2108117574}, {'type': 'v1', 'addr': '172.21.15.166:6805', 'nonce': 2108117574}]}, 'public_addr': '172.21.15.166:6801/2108117574', 'cluster_addr': '172.21.15.166:6803/2108117574', 'heartbeat_back_addr': '172.21.15.166:6807/2108117574', 'heartbeat_front_addr': '172.21.15.166:6805/2108117574', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': 'ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 72, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.166:6809', 'nonce': 1458870080}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6810', 'nonce': 1458870080}, {'type': 'v1', 'addr': '172.21.15.166:6811', 'nonce': 1458870080}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6814', 'nonce': 1458870080}, {'type': 'v1', 'addr': '172.21.15.166:6815', 'nonce': 1458870080}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6812', 'nonce': 1458870080}, {'type': 'v1', 'addr': '172.21.15.166:6813', 'nonce': 1458870080}]}, 'public_addr': '172.21.15.166:6809/1458870080', 'cluster_addr': '172.21.15.166:6811/1458870080', 'heartbeat_back_addr': '172.21.15.166:6815/1458870080', 'heartbeat_front_addr': '172.21.15.166:6813/1458870080', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': 'c194210e-bd89-4a1f-98fb-6bf44f0b324c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 81, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.166:6817', 'nonce': 62517848}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6818', 'nonce': 62517848}, {'type': 'v1', 'addr': '172.21.15.166:6819', 'nonce': 62517848}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6822', 'nonce': 62517848}, {'type': 'v1', 'addr': '172.21.15.166:6823', 'nonce': 62517848}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6820', 'nonce': 62517848}, {'type': 'v1', 'addr': '172.21.15.166:6821', 'nonce': 62517848}]}, 'public_addr': '172.21.15.166:6817/62517848', 'cluster_addr': '172.21.15.166:6819/62517848', 'heartbeat_back_addr': '172.21.15.166:6823/62517848', 'heartbeat_front_addr': '172.21.15.166:6821/62517848', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '121301fe-a41c-40c0-ad09-b00de37a55e1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 90, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.166:6825', 'nonce': 691735125}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6826', 'nonce': 691735125}, {'type': 'v1', 'addr': '172.21.15.166:6827', 'nonce': 691735125}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6830', 'nonce': 691735125}, {'type': 'v1', 'addr': '172.21.15.166:6831', 'nonce': 691735125}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6828', 'nonce': 691735125}, {'type': 'v1', 'addr': '172.21.15.166:6829', 'nonce': 691735125}]}, 'public_addr': '172.21.15.166:6825/691735125', 'cluster_addr': '172.21.15.166:6827/691735125', 'heartbeat_back_addr': '172.21.15.166:6831/691735125', 'heartbeat_front_addr': '172.21.15.166:6829/691735125', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:40:06.135513+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:40:34.045201+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:40:56.946674+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:40:21.192386+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:40:46.317711+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:40:12.236239+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:40:40.153723+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:41:05.533799+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:40:27.086853+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:40:54.670244+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:40:18.017790+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:40:42.450226+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.104:0/1257365091': '2020-09-09T15:29:39.430456+0000', '172.21.15.104:0/1399123455': '2020-09-09T15:29:39.430456+0000', '172.21.15.104:0/2810441605': '2020-09-09T15:29:39.430456+0000', '172.21.15.104:6800/182241824': '2020-09-09T15:29:39.430456+0000', '172.21.15.104:6801/182241824': '2020-09-09T15:29:39.430456+0000', '172.21.15.104:0/2021933044': '2020-09-09T15:26:33.248197+0000', '172.21.15.104:6800/3129754347': '2020-09-09T15:26:33.248197+0000', '172.21.15.104:0/1270031705': '2020-09-09T15:26:33.248197+0000', '172.21.15.104:0/2568432190': '2020-09-09T15:26:33.248197+0000', '172.21.15.104:6800/1138921844': '2020-09-09T15:26:09.867236+0000', '172.21.15.104:0/3827328474': '2020-09-09T15:26:09.867236+0000', '172.21.15.104:6801/1138921844': '2020-09-09T15:26:09.867236+0000', '172.21.15.104:0/1662035381': '2020-09-09T15:26:09.867236+0000', '172.21.15.104:6801/3129754347': '2020-09-09T15:26:33.248197+0000', '172.21.15.104:0/2169968102': '2020-09-09T15:26:09.867236+0000'}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}} 2020-09-08T15:41:07.102 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph pg dump --format=json 2020-09-08T15:41:07.272 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:41:07.305 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:07.308 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:41:07 smithi166 bash[14254]: debug 2020-09-08T15:41:07.304+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:07.409 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:07.532 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:07.636 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:07.740 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:41:07 smithi055 bash[11956]: debug 2020-09-08T15:41:07.530+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:07.740 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:07 smithi055 bash[9296]: cluster 2020-09-08T15:41:06.141578+0000 osd.0 (osd.0) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:07.741 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:07 smithi055 bash[9296]: cluster 2020-09-08T15:41:06.141664+0000 osd.0 (osd.0) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:07.741 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:07 smithi055 bash[9296]: cluster 2020-09-08T15:41:06.659174+0000 mon.a (mon.0) 1067 : cluster [DBG] osdmap e200: 12 total, 12 up, 12 in 2020-09-08T15:41:07.741 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:07 smithi055 bash[9296]: cluster 2020-09-08T15:41:06.730951+0000 osd.2 (osd.2) 30 : cluster [DBG] 2.e scrub starts 2020-09-08T15:41:07.741 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:07 smithi055 bash[9296]: cluster 2020-09-08T15:41:06.978267+0000 osd.8 (osd.8) 23 : cluster [DBG] 2.f scrub starts 2020-09-08T15:41:07.742 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:07 smithi055 bash[9296]: cluster 2020-09-08T15:41:07.646154+0000 mgr.x (mgr.24118) 410 : cluster [DBG] pgmap v530: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.36k op/s 2020-09-08T15:41:07.742 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:07.848 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:07.962 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:07 smithi104 bash[9203]: cluster 2020-09-08T15:41:06.141578+0000 osd.0 (osd.0) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:07.963 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:07 smithi104 bash[9203]: cluster 2020-09-08T15:41:06.141664+0000 osd.0 (osd.0) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:07.963 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:07 smithi104 bash[9203]: cluster 2020-09-08T15:41:06.659174+0000 mon.a (mon.0) 1067 : cluster [DBG] osdmap e200: 12 total, 12 up, 12 in 2020-09-08T15:41:07.963 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:07 smithi104 bash[9203]: cluster 2020-09-08T15:41:06.730951+0000 osd.2 (osd.2) 30 : cluster [DBG] 2.e scrub starts 2020-09-08T15:41:08.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:07 smithi166 bash[9446]: cluster 2020-09-08T15:41:06.141578+0000 osd.0 (osd.0) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:08.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:07 smithi166 bash[9446]: cluster 2020-09-08T15:41:06.141664+0000 osd.0 (osd.0) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:08.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:07 smithi166 bash[9446]: cluster 2020-09-08T15:41:06.659174+0000 mon.a (mon.0) 1067 : cluster [DBG] osdmap e200: 12 total, 12 up, 12 in 2020-09-08T15:41:08.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:07 smithi166 bash[9446]: cluster 2020-09-08T15:41:06.730951+0000 osd.2 (osd.2) 30 : cluster [DBG] 2.e scrub starts 2020-09-08T15:41:08.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:07 smithi166 bash[9446]: cluster 2020-09-08T15:41:06.978267+0000 osd.8 (osd.8) 23 : cluster [DBG] 2.f scrub starts 2020-09-08T15:41:08.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:07 smithi166 bash[9446]: cluster 2020-09-08T15:41:07.646154+0000 mgr.x (mgr.24118) 410 : cluster [DBG] pgmap v530: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.36k op/s 2020-09-08T15:41:08.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:41:07 smithi055 bash[16646]: debug 2020-09-08T15:41:07.738+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:08.164 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:08.245 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:08 smithi104 bash[9203]: cluster 2020-09-08T15:41:06.978267+0000 osd.8 (osd.8) 23 : cluster [DBG] 2.f scrub starts 2020-09-08T15:41:08.246 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:08 smithi104 bash[9203]: cluster 2020-09-08T15:41:07.646154+0000 mgr.x (mgr.24118) 410 : cluster [DBG] pgmap v530: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.36k op/s 2020-09-08T15:41:08.246 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:41:08 smithi104 bash[15273]: debug 2020-09-08T15:41:08.165+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:08.273 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:08.494 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:41:08.494 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:41:08.552 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:41:08.581 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:41:08.617 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:41:08.646 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:41:08.676 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:41:08.724 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:41:08.755 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:41:08.793 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:41:08.826 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:41:08.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:08 smithi166 bash[9446]: cluster 2020-09-08T15:41:07.596109+0000 osd.2 (osd.2) 31 : cluster [DBG] 2.e scrub ok 2020-09-08T15:41:08.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:08 smithi166 bash[9446]: cluster 2020-09-08T15:41:07.772893+0000 osd.8 (osd.8) 24 : cluster [DBG] 2.f scrub ok 2020-09-08T15:41:08.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:08 smithi166 bash[9446]: audit 2020-09-08T15:41:08.532385+0000 mon.c (mon.2) 42 : audit [DBG] from='client.? v1:172.21.15.104:0/201979069' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:41:08.868 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:41:08.879 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:08.904 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:41:08.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:08 smithi104 bash[9203]: cluster 2020-09-08T15:41:07.596109+0000 osd.2 (osd.2) 31 : cluster [DBG] 2.e scrub ok 2020-09-08T15:41:08.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:08 smithi104 bash[9203]: cluster 2020-09-08T15:41:07.772893+0000 osd.8 (osd.8) 24 : cluster [DBG] 2.f scrub ok 2020-09-08T15:41:08.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:08 smithi104 bash[9203]: audit 2020-09-08T15:41:08.532385+0000 mon.c (mon.2) 42 : audit [DBG] from='client.? v1:172.21.15.104:0/201979069' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:41:08.927 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:41:08 smithi104 bash[15273]: debug 2020-09-08T15:41:08.877+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:08.938 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:41:08.949 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:41:08.950 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:41:08.950 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:41:08.950 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:41:08.950 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:41:08.951 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:41:08.951 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:41:08.951 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:41:08.952 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:41:08.952 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:41:08.952 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:41:08.952 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:41:08.982 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:08.996 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:08 smithi055 bash[9296]: cluster 2020-09-08T15:41:07.596109+0000 osd.2 (osd.2) 31 : cluster [DBG] 2.e scrub ok 2020-09-08T15:41:08.997 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:08 smithi055 bash[9296]: cluster 2020-09-08T15:41:07.772893+0000 osd.8 (osd.8) 24 : cluster [DBG] 2.f scrub ok 2020-09-08T15:41:08.997 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:08 smithi055 bash[9296]: audit 2020-09-08T15:41:08.532385+0000 mon.c (mon.2) 42 : audit [DBG] from='client.? v1:172.21.15.104:0/201979069' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:41:09.116 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:09.219 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:09.381 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:09.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:41:09 smithi055 bash[11956]: debug 2020-09-08T15:41:09.114+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:09.454 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2020-09-08T15:41:09.482 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:09.526 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:41:09 smithi166 bash[14254]: debug 2020-09-08T15:41:09.376+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:09.620 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:41:09.651 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:09.754 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:09.778 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:41:09 smithi166 bash[14254]: debug 2020-09-08T15:41:09.648+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:10.270 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:10.376 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:10.498 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:10.602 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:10.614 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:10 smithi166 bash[9446]: cluster 2020-09-08T15:41:09.646675+0000 mgr.x (mgr.24118) 411 : cluster [DBG] pgmap v531: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.20k op/s 2020-09-08T15:41:10.614 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:41:10 smithi166 bash[16593]: debug 2020-09-08T15:41:10.268+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:10.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:10 smithi055 bash[9296]: cluster 2020-09-08T15:41:09.646675+0000 mgr.x (mgr.24118) 411 : cluster [DBG] pgmap v531: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.20k op/s 2020-09-08T15:41:10.657 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:41:10 smithi055 bash[11956]: debug 2020-09-08T15:41:10.494+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:10.922 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:10 smithi104 bash[9203]: cluster 2020-09-08T15:41:09.646675+0000 mgr.x (mgr.24118) 411 : cluster [DBG] pgmap v531: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.20k op/s 2020-09-08T15:41:10.985 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:11.088 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:41:11.197 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:41:11.301 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:11.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:41:11 smithi055 bash[19003]: debug 2020-09-08T15:41:11.194+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:11.423 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:11.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:41:10 smithi104 bash[19922]: debug 2020-09-08T15:41:10.981+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:11.496 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:41:11 smithi166 bash[14254]: debug 2020-09-08T15:41:11.420+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:11.530 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:41:11.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:11 smithi166 bash[9446]: cluster 2020-09-08T15:41:10.947942+0000 osd.8 (osd.8) 25 : cluster [DBG] 2.b scrub starts 2020-09-08T15:41:11.894 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:11 smithi104 bash[9203]: cluster 2020-09-08T15:41:10.947942+0000 osd.8 (osd.8) 25 : cluster [DBG] 2.b scrub starts 2020-09-08T15:41:11.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:11 smithi055 bash[9296]: cluster 2020-09-08T15:41:10.947942+0000 osd.8 (osd.8) 25 : cluster [DBG] 2.b scrub starts 2020-09-08T15:41:11.957 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:41:12.061 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:12.181 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:12.282 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:12.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:41:12 smithi055 bash[11956]: debug 2020-09-08T15:41:12.174+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:12.415 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:12.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:41:11 smithi104 bash[17588]: debug 2020-09-08T15:41:11.953+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:12.521 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:12.609 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:41:12.615 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:41:12 smithi166 bash[16593]: debug 2020-09-08T15:41:12.412+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:12.641 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:41:12.642 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:12.743 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:12.781 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:41:12.854 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:12 smithi055 bash[9296]: cluster 2020-09-08T15:41:11.551893+0000 osd.9 (osd.9) 19 : cluster [DBG] 2.d scrub starts 2020-09-08T15:41:12.854 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:12 smithi055 bash[9296]: cluster 2020-09-08T15:41:11.647081+0000 mgr.x (mgr.24118) 412 : cluster [DBG] pgmap v532: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 587 op/s 2020-09-08T15:41:12.854 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:12 smithi055 bash[9296]: cluster 2020-09-08T15:41:11.741631+0000 osd.8 (osd.8) 26 : cluster [DBG] 2.b scrub ok 2020-09-08T15:41:12.855 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:12 smithi055 bash[9296]: cluster 2020-09-08T15:41:11.934511+0000 osd.8 (osd.8) 27 : cluster [DBG] 2.5 scrub starts 2020-09-08T15:41:12.855 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:12 smithi055 bash[9296]: cluster 2020-09-08T15:41:12.336826+0000 osd.5 (osd.5) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:12.855 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:12 smithi055 bash[9296]: cluster 2020-09-08T15:41:12.336978+0000 osd.5 (osd.5) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:12.855 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:12 smithi055 bash[9296]: audit 2020-09-08T15:41:12.347045+0000 mon.a (mon.0) 1068 : audit [INF] from='client.? v1:172.21.15.104:0/143514507' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:41:12.856 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:41:12 smithi055 bash[14301]: debug 2020-09-08T15:41:12.634+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:12.856 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:12.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:12 smithi104 bash[9203]: cluster 2020-09-08T15:41:11.551893+0000 osd.9 (osd.9) 19 : cluster [DBG] 2.d scrub starts 2020-09-08T15:41:12.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:12 smithi104 bash[9203]: cluster 2020-09-08T15:41:11.647081+0000 mgr.x (mgr.24118) 412 : cluster [DBG] pgmap v532: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 587 op/s 2020-09-08T15:41:12.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:12 smithi104 bash[9203]: cluster 2020-09-08T15:41:11.741631+0000 osd.8 (osd.8) 26 : cluster [DBG] 2.b scrub ok 2020-09-08T15:41:12.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:12 smithi104 bash[9203]: cluster 2020-09-08T15:41:11.934511+0000 osd.8 (osd.8) 27 : cluster [DBG] 2.5 scrub starts 2020-09-08T15:41:12.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:12 smithi104 bash[9203]: cluster 2020-09-08T15:41:12.336826+0000 osd.5 (osd.5) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:12.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:12 smithi104 bash[9203]: cluster 2020-09-08T15:41:12.336978+0000 osd.5 (osd.5) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:12.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:12 smithi104 bash[9203]: audit 2020-09-08T15:41:12.347045+0000 mon.a (mon.0) 1068 : audit [INF] from='client.? v1:172.21.15.104:0/143514507' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:41:12.959 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:41:12.968 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:12 smithi166 bash[9446]: cluster 2020-09-08T15:41:11.551893+0000 osd.9 (osd.9) 19 : cluster [DBG] 2.d scrub starts 2020-09-08T15:41:12.968 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:12 smithi166 bash[9446]: cluster 2020-09-08T15:41:11.647081+0000 mgr.x (mgr.24118) 412 : cluster [DBG] pgmap v532: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 587 op/s 2020-09-08T15:41:12.968 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:12 smithi166 bash[9446]: cluster 2020-09-08T15:41:11.741631+0000 osd.8 (osd.8) 26 : cluster [DBG] 2.b scrub ok 2020-09-08T15:41:12.969 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:12 smithi166 bash[9446]: cluster 2020-09-08T15:41:11.934511+0000 osd.8 (osd.8) 27 : cluster [DBG] 2.5 scrub starts 2020-09-08T15:41:12.969 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:12 smithi166 bash[9446]: cluster 2020-09-08T15:41:12.336826+0000 osd.5 (osd.5) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:12.969 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:12 smithi166 bash[9446]: cluster 2020-09-08T15:41:12.336978+0000 osd.5 (osd.5) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:12.969 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:12 smithi166 bash[9446]: audit 2020-09-08T15:41:12.347045+0000 mon.a (mon.0) 1068 : audit [INF] from='client.? v1:172.21.15.104:0/143514507' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:41:13.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:41:12 smithi055 bash[11956]: debug 2020-09-08T15:41:12.854+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:13.239 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:41:13.343 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:41:13.615 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:41:13 smithi166 bash[11928]: debug 2020-09-08T15:41:13.236+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:13.885 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:13 smithi055 bash[9296]: cluster 2020-09-08T15:41:12.199006+0000 osd.9 (osd.9) 20 : cluster [DBG] 2.d scrub ok 2020-09-08T15:41:13.886 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:13 smithi055 bash[9296]: cluster 2020-09-08T15:41:12.620839+0000 mon.a (mon.0) 1069 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:41:13.886 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:13 smithi055 bash[9296]: cluster 2020-09-08T15:41:12.620873+0000 mon.a (mon.0) 1070 : cluster [INF] Cluster is now healthy 2020-09-08T15:41:13.886 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:13 smithi055 bash[9296]: audit 2020-09-08T15:41:12.639105+0000 mon.a (mon.0) 1071 : audit [INF] from='client.? v1:172.21.15.104:0/143514507' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:41:13.886 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:13 smithi055 bash[9296]: cluster 2020-09-08T15:41:12.639191+0000 mon.a (mon.0) 1072 : cluster [DBG] osdmap e201: 12 total, 12 up, 12 in 2020-09-08T15:41:13.887 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:13 smithi055 bash[9296]: cluster 2020-09-08T15:41:12.748342+0000 osd.8 (osd.8) 28 : cluster [DBG] 2.5 scrub ok 2020-09-08T15:41:13.887 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:41:13.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:13 smithi104 bash[9203]: cluster 2020-09-08T15:41:12.199006+0000 osd.9 (osd.9) 20 : cluster [DBG] 2.d scrub ok 2020-09-08T15:41:13.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:13 smithi104 bash[9203]: cluster 2020-09-08T15:41:12.620839+0000 mon.a (mon.0) 1069 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:41:13.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:13 smithi104 bash[9203]: cluster 2020-09-08T15:41:12.620873+0000 mon.a (mon.0) 1070 : cluster [INF] Cluster is now healthy 2020-09-08T15:41:13.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:13 smithi104 bash[9203]: audit 2020-09-08T15:41:12.639105+0000 mon.a (mon.0) 1071 : audit [INF] from='client.? v1:172.21.15.104:0/143514507' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:41:13.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:13 smithi104 bash[9203]: cluster 2020-09-08T15:41:12.639191+0000 mon.a (mon.0) 1072 : cluster [DBG] osdmap e201: 12 total, 12 up, 12 in 2020-09-08T15:41:13.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:13 smithi104 bash[9203]: cluster 2020-09-08T15:41:12.748342+0000 osd.8 (osd.8) 28 : cluster [DBG] 2.5 scrub ok 2020-09-08T15:41:13.990 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:13.998 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:13 smithi166 bash[9446]: cluster 2020-09-08T15:41:12.199006+0000 osd.9 (osd.9) 20 : cluster [DBG] 2.d scrub ok 2020-09-08T15:41:13.999 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:13 smithi166 bash[9446]: cluster 2020-09-08T15:41:12.620839+0000 mon.a (mon.0) 1069 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:41:13.999 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:13 smithi166 bash[9446]: cluster 2020-09-08T15:41:12.620873+0000 mon.a (mon.0) 1070 : cluster [INF] Cluster is now healthy 2020-09-08T15:41:13.999 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:13 smithi166 bash[9446]: audit 2020-09-08T15:41:12.639105+0000 mon.a (mon.0) 1071 : audit [INF] from='client.? v1:172.21.15.104:0/143514507' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:41:13.999 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:13 smithi166 bash[9446]: cluster 2020-09-08T15:41:12.639191+0000 mon.a (mon.0) 1072 : cluster [DBG] osdmap e201: 12 total, 12 up, 12 in 2020-09-08T15:41:14.000 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:13 smithi166 bash[9446]: cluster 2020-09-08T15:41:12.748342+0000 osd.8 (osd.8) 28 : cluster [DBG] 2.5 scrub ok 2020-09-08T15:41:14.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:41:13 smithi055 bash[19003]: debug 2020-09-08T15:41:13.886+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:14.161 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:14.263 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:41:14.314 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:41:14 smithi166 bash[18960]: debug 2020-09-08T15:41:14.160+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:14.498 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:41:14.521 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:41:14.522 INFO:teuthology.orchestra.run.smithi166.stdout: "size": 20, 2020-09-08T15:41:14.522 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 600, 2020-09-08T15:41:14.522 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T15:41:14.523 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.523 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:168405 2.5 2:a65aa17e:::benchmark_data_smithi166_20551_object20785:head [delete] snapc 0=[] ondisk+write+known_if_redirected e172)", 2020-09-08T15:41:14.523 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:34.484279+0000", 2020-09-08T15:41:14.523 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 100.03798816, 2020-09-08T15:41:14.523 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.264170609, 2020-09-08T15:41:14.524 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:14.524 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:14.524 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:14.524 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:14.524 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:14.525 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 168405 2020-09-08T15:41:14.525 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.525 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:14.525 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.526 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:14.526 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484279+0000", 2020-09-08T15:41:14.526 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:14.526 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.526 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.527 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:14.527 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484279+0000", 2020-09-08T15:41:14.527 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999642 2020-09-08T15:41:14.527 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.527 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.528 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:14.528 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484243+0000", 2020-09-08T15:41:14.529 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.0168999999999998e-05 2020-09-08T15:41:14.530 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.530 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.530 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:14.530 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484324+0000", 2020-09-08T15:41:14.531 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3794999999999999e-05 2020-09-08T15:41:14.531 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.531 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.531 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:14.532 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484337+0000", 2020-09-08T15:41:14.532 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7969e-05 2020-09-08T15:41:14.532 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.532 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.532 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:14.533 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484355+0000", 2020-09-08T15:41:14.533 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7426e-05 2020-09-08T15:41:14.533 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.534 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.534 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:14.534 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484433+0000", 2020-09-08T15:41:14.535 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00028967899999999999 2020-09-08T15:41:14.535 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.535 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.535 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:14.536 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484722+0000", 2020-09-08T15:41:14.537 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013431700000000001 2020-09-08T15:41:14.537 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.537 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.538 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:14.538 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484857+0000", 2020-09-08T15:41:14.538 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.010789607 2020-09-08T15:41:14.538 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.538 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.539 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:14.539 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.495646+0000", 2020-09-08T15:41:14.539 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25252755100000002 2020-09-08T15:41:14.539 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.539 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.540 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:14.540 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.748174+0000", 2020-09-08T15:41:14.540 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018818 2020-09-08T15:41:14.540 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.540 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.541 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:14.541 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.748362+0000", 2020-09-08T15:41:14.541 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.7688000000000005e-05 2020-09-08T15:41:14.542 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.542 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.542 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:14.542 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.748450+0000", 2020-09-08T15:41:14.542 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.264170609 2020-09-08T15:41:14.543 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.543 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.543 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.543 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.544 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.544 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:168406 2.5 2:adf3a2f3:::benchmark_data_smithi166_20551_object20786:head [delete] snapc 0=[] ondisk+write+known_if_redirected e172)", 2020-09-08T15:41:14.544 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:34.484624+0000", 2020-09-08T15:41:14.544 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 100.037642944, 2020-09-08T15:41:14.545 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.27197898700000001, 2020-09-08T15:41:14.545 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:14.545 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:14.545 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:14.545 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:14.546 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:14.546 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 168406 2020-09-08T15:41:14.546 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.546 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:14.546 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.547 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:14.547 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484624+0000", 2020-09-08T15:41:14.547 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:14.547 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.547 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.547 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:14.548 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484624+0000", 2020-09-08T15:41:14.548 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999623 2020-09-08T15:41:14.548 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.548 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.548 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:14.549 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484587+0000", 2020-09-08T15:41:14.549 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.3590000000000004e-05 2020-09-08T15:41:14.549 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.549 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.549 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:14.550 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484670+0000", 2020-09-08T15:41:14.550 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4182000000000001e-05 2020-09-08T15:41:14.550 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.550 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.550 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:14.551 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484684+0000", 2020-09-08T15:41:14.551 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6139999999999997e-06 2020-09-08T15:41:14.551 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.551 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.551 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:14.552 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484691+0000", 2020-09-08T15:41:14.552 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0058046310000000002 2020-09-08T15:41:14.552 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.552 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.552 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:14.553 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.490496+0000", 2020-09-08T15:41:14.553 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018183399999999999 2020-09-08T15:41:14.553 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.554 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.554 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:14.554 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.490678+0000", 2020-09-08T15:41:14.554 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017738500000000001 2020-09-08T15:41:14.554 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.555 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.555 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:14.555 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.490855+0000", 2020-09-08T15:41:14.555 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0051368480000000003 2020-09-08T15:41:14.555 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.556 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.556 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:14.556 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.495992+0000", 2020-09-08T15:41:14.556 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.26042808899999997 2020-09-08T15:41:14.556 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.557 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.557 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:14.557 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.756420+0000", 2020-09-08T15:41:14.557 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010587800000000001 2020-09-08T15:41:14.557 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.558 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.558 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:14.558 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.756526+0000", 2020-09-08T15:41:14.558 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7712000000000004e-05 2020-09-08T15:41:14.558 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.559 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.559 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:14.559 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.756603+0000", 2020-09-08T15:41:14.559 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.27197898700000001 2020-09-08T15:41:14.559 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.560 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.560 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.560 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.560 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.560 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:168416 2.5 2:ad81a029:::benchmark_data_smithi166_20551_object20796:head [delete] snapc 0=[] ondisk+write+known_if_redirected e172)", 2020-09-08T15:41:14.561 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:34.498712+0000", 2020-09-08T15:41:14.561 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 100.02355505, 2020-09-08T15:41:14.561 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.26572992400000001, 2020-09-08T15:41:14.561 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:14.561 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:14.562 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:14.562 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:14.562 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:14.562 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 168416 2020-09-08T15:41:14.562 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.563 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:14.563 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.563 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:14.563 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.498712+0000", 2020-09-08T15:41:14.563 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:14.565 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.565 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.565 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:14.565 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.498712+0000", 2020-09-08T15:41:14.566 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999766 2020-09-08T15:41:14.566 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.566 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.566 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:14.567 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.498689+0000", 2020-09-08T15:41:14.567 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.2722000000000001e-05 2020-09-08T15:41:14.567 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.567 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.567 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:14.568 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.498742+0000", 2020-09-08T15:41:14.568 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2274e-05 2020-09-08T15:41:14.568 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.568 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.568 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:14.569 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.498754+0000", 2020-09-08T15:41:14.569 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.4699999999999996e-06 2020-09-08T15:41:14.569 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.569 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.570 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:14.570 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.498761+0000", 2020-09-08T15:41:14.570 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1848999999999998e-05 2020-09-08T15:41:14.570 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.570 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.571 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:14.571 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.498813+0000", 2020-09-08T15:41:14.571 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000123433 2020-09-08T15:41:14.571 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.571 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.572 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:14.572 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.498937+0000", 2020-09-08T15:41:14.572 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013201200000000001 2020-09-08T15:41:14.572 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.572 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.573 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:14.573 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.499069+0000", 2020-09-08T15:41:14.573 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0054701159999999997 2020-09-08T15:41:14.573 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.573 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.574 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:14.574 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.504539+0000", 2020-09-08T15:41:14.574 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25956900100000002 2020-09-08T15:41:14.574 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.575 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.575 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:14.575 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.764108+0000", 2020-09-08T15:41:14.575 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018175299999999999 2020-09-08T15:41:14.576 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.576 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.576 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:14.576 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.764290+0000", 2020-09-08T15:41:14.576 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000152657 2020-09-08T15:41:14.576 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.577 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.577 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:14.577 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.764442+0000", 2020-09-08T15:41:14.577 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.26572992400000001 2020-09-08T15:41:14.577 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.578 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.578 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.578 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.578 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.578 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:168429 2.5 2:a057a1df:::benchmark_data_smithi166_20551_object20809:head [delete] snapc 0=[] ondisk+write+known_if_redirected e172)", 2020-09-08T15:41:14.579 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:34.509989+0000", 2020-09-08T15:41:14.579 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 100.01227824, 2020-09-08T15:41:14.579 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25484900199999999, 2020-09-08T15:41:14.579 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:14.579 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:14.580 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:14.580 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:14.580 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:14.580 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 168429 2020-09-08T15:41:14.580 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.581 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:14.581 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.581 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:14.581 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.509989+0000", 2020-09-08T15:41:14.581 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:14.582 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.582 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.582 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:14.582 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.509989+0000", 2020-09-08T15:41:14.582 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999571 2020-09-08T15:41:14.583 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.583 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.583 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:14.583 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.509946+0000", 2020-09-08T15:41:14.583 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.6836000000000006e-05 2020-09-08T15:41:14.584 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.584 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.584 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:14.584 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510043+0000", 2020-09-08T15:41:14.584 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6231000000000001e-05 2020-09-08T15:41:14.585 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.585 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.585 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:14.585 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510059+0000", 2020-09-08T15:41:14.586 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6209999999999998e-06 2020-09-08T15:41:14.586 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.586 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.586 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:14.586 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510066+0000", 2020-09-08T15:41:14.587 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00027533400000000002 2020-09-08T15:41:14.587 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.587 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.587 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:14.587 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510341+0000", 2020-09-08T15:41:14.588 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012089100000000001 2020-09-08T15:41:14.588 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.588 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.588 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:14.588 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510462+0000", 2020-09-08T15:41:14.589 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000100579 2020-09-08T15:41:14.589 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.589 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.589 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:14.589 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510563+0000", 2020-09-08T15:41:14.590 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0099092620000000003 2020-09-08T15:41:14.590 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.590 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.590 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:14.590 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.520472+0000", 2020-09-08T15:41:14.591 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.244098331 2020-09-08T15:41:14.592 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.592 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.593 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:14.593 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.764570+0000", 2020-09-08T15:41:14.593 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000185859 2020-09-08T15:41:14.593 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.593 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.594 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:14.594 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.764756+0000", 2020-09-08T15:41:14.594 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.2008000000000005e-05 2020-09-08T15:41:14.594 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.594 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.595 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:14.595 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.764838+0000", 2020-09-08T15:41:14.595 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25484900199999999 2020-09-08T15:41:14.596 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.596 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.596 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.596 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.597 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.597 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:168430 2.5 2:adddae02:::benchmark_data_smithi166_20551_object20810:head [delete] snapc 0=[] ondisk+write+known_if_redirected e172)", 2020-09-08T15:41:14.597 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:34.510140+0000", 2020-09-08T15:41:14.597 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 100.01212701, 2020-09-08T15:41:14.597 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25847826400000001, 2020-09-08T15:41:14.599 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:14.599 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:14.600 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:14.600 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:14.600 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:14.600 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 168430 2020-09-08T15:41:14.600 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.601 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:14.601 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.601 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:14.601 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510140+0000", 2020-09-08T15:41:14.601 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:14.602 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.602 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.602 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:14.602 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510140+0000", 2020-09-08T15:41:14.602 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999599 2020-09-08T15:41:14.603 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.603 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.603 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:14.603 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510100+0000", 2020-09-08T15:41:14.603 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8372e-05 2020-09-08T15:41:14.604 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.604 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.604 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:14.604 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510189+0000", 2020-09-08T15:41:14.604 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3943e-05 2020-09-08T15:41:14.604 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.605 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.605 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:14.605 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510203+0000", 2020-09-08T15:41:14.605 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1479999999999997e-06 2020-09-08T15:41:14.605 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.606 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.606 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:14.606 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510208+0000", 2020-09-08T15:41:14.607 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0057710879999999997 2020-09-08T15:41:14.607 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.607 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.607 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:14.607 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.515979+0000", 2020-09-08T15:41:14.608 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000161865 2020-09-08T15:41:14.608 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.608 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.608 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:14.608 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.516141+0000", 2020-09-08T15:41:14.608 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012483500000000001 2020-09-08T15:41:14.609 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.609 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.609 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:14.609 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.516265+0000", 2020-09-08T15:41:14.609 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0043580379999999998 2020-09-08T15:41:14.610 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.610 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.610 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:14.610 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.520624+0000", 2020-09-08T15:41:14.611 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.24768425499999999 2020-09-08T15:41:14.611 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.611 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.612 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:14.612 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.768308+0000", 2020-09-08T15:41:14.612 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016047300000000001 2020-09-08T15:41:14.612 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.612 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.613 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:14.613 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.768468+0000", 2020-09-08T15:41:14.613 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0001504 2020-09-08T15:41:14.613 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.613 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.614 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:14.614 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.768619+0000", 2020-09-08T15:41:14.614 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25847826400000001 2020-09-08T15:41:14.614 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.615 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.615 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.615 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.615 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.615 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:168457 2.5 2:a50f2b9e:::benchmark_data_smithi166_20551_object20837:head [delete] snapc 0=[] ondisk+write+known_if_redirected e172)", 2020-09-08T15:41:14.615 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:34.545652+0000", 2020-09-08T15:41:14.616 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 99.976615439, 2020-09-08T15:41:14.616 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22335165500000001, 2020-09-08T15:41:14.616 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:14.616 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:14.616 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:14.617 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:14.617 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:14.617 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 168457 2020-09-08T15:41:14.617 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.617 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:14.618 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.618 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:14.618 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.545652+0000", 2020-09-08T15:41:14.618 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:14.618 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.619 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.619 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:14.619 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.545652+0000", 2020-09-08T15:41:14.619 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999709 2020-09-08T15:41:14.619 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.619 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.620 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:14.620 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.545623+0000", 2020-09-08T15:41:14.620 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6372999999999996e-05 2020-09-08T15:41:14.620 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.620 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.621 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:14.621 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.545689+0000", 2020-09-08T15:41:14.621 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4678000000000001e-05 2020-09-08T15:41:14.621 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.621 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.622 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:14.622 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.545704+0000", 2020-09-08T15:41:14.622 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.3829999999999992e-06 2020-09-08T15:41:14.622 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.622 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.623 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:14.623 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.545712+0000", 2020-09-08T15:41:14.623 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.6777000000000003e-05 2020-09-08T15:41:14.623 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.623 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.623 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:14.624 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.545759+0000", 2020-09-08T15:41:14.624 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013097400000000001 2020-09-08T15:41:14.624 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.625 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.626 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:14.626 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.545890+0000", 2020-09-08T15:41:14.626 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011462499999999999 2020-09-08T15:41:14.626 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.626 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.627 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:14.627 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.546004+0000", 2020-09-08T15:41:14.627 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0044159000000000004 2020-09-08T15:41:14.627 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.627 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.628 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:14.628 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.550420+0000", 2020-09-08T15:41:14.628 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21836440400000001 2020-09-08T15:41:14.628 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.628 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.629 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:14.629 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.768785+0000", 2020-09-08T15:41:14.629 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7306000000000007e-05 2020-09-08T15:41:14.629 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.629 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.630 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:14.630 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.768882+0000", 2020-09-08T15:41:14.630 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000121558 2020-09-08T15:41:14.630 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.630 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.631 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:14.631 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.769004+0000", 2020-09-08T15:41:14.631 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22335165500000001 2020-09-08T15:41:14.631 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.631 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.632 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.632 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.632 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.632 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:190014 2.5 2:a63b47c4:::benchmark_data_smithi166_20551_object42394:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:41:14.633 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:56.732376+0000", 2020-09-08T15:41:14.633 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 77.789891323000006, 2020-09-08T15:41:14.633 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.24903745499999999, 2020-09-08T15:41:14.634 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:14.634 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:14.634 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:14.634 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:14.634 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:14.635 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 190014 2020-09-08T15:41:14.635 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.635 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:14.635 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.635 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:14.636 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.732376+0000", 2020-09-08T15:41:14.636 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:14.636 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.636 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.636 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:14.637 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.732376+0000", 2020-09-08T15:41:14.637 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999585 2020-09-08T15:41:14.637 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.637 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.637 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:14.638 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.732335+0000", 2020-09-08T15:41:14.638 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.1034999999999998e-05 2020-09-08T15:41:14.638 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.638 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.638 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:14.639 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.732426+0000", 2020-09-08T15:41:14.639 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5119e-05 2020-09-08T15:41:14.639 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.639 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.639 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:14.640 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.732441+0000", 2020-09-08T15:41:14.640 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.7560000000000002e-06 2020-09-08T15:41:14.640 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.640 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.641 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:14.641 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.732448+0000", 2020-09-08T15:41:14.641 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9833000000000002e-05 2020-09-08T15:41:14.641 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.641 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.642 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:14.642 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.732497+0000", 2020-09-08T15:41:14.642 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015706199999999999 2020-09-08T15:41:14.642 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.642 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.642 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:14.643 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.732655+0000", 2020-09-08T15:41:14.643 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000177307 2020-09-08T15:41:14.643 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.643 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.643 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:14.644 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.732832+0000", 2020-09-08T15:41:14.644 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.011138075000000001 2020-09-08T15:41:14.644 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.644 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.644 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:14.645 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.743970+0000", 2020-09-08T15:41:14.645 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.237051084 2020-09-08T15:41:14.645 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.645 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.645 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:14.646 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.981021+0000", 2020-09-08T15:41:14.646 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000189249 2020-09-08T15:41:14.646 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.646 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.646 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:14.647 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.981210+0000", 2020-09-08T15:41:14.647 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00020328300000000001 2020-09-08T15:41:14.647 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.647 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.647 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:14.648 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.981414+0000", 2020-09-08T15:41:14.648 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.24903745499999999 2020-09-08T15:41:14.648 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.648 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.648 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.649 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.649 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.649 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:190017 2.5 2:a3e31b0f:::benchmark_data_smithi166_20551_object42397:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:41:14.650 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:56.743914+0000", 2020-09-08T15:41:14.650 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 77.778353038999995, 2020-09-08T15:41:14.650 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.23789671500000001, 2020-09-08T15:41:14.650 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:14.650 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:14.650 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:14.651 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:14.651 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:14.651 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 190017 2020-09-08T15:41:14.651 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.651 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:14.652 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.652 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:14.652 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.743914+0000", 2020-09-08T15:41:14.652 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:14.652 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.653 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.653 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:14.653 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.743914+0000", 2020-09-08T15:41:14.653 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.999958 2020-09-08T15:41:14.653 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.654 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.654 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:14.654 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.743872+0000", 2020-09-08T15:41:14.654 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.1538000000000003e-05 2020-09-08T15:41:14.654 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.654 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.655 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:14.655 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.743964+0000", 2020-09-08T15:41:14.655 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4995e-05 2020-09-08T15:41:14.655 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.655 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.656 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:14.656 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.743979+0000", 2020-09-08T15:41:14.656 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.4249999999999998e-06 2020-09-08T15:41:14.656 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.656 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.657 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:14.657 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.743987+0000", 2020-09-08T15:41:14.657 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1610000000000002e-05 2020-09-08T15:41:14.657 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.657 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.658 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:14.658 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.744039+0000", 2020-09-08T15:41:14.658 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000184709 2020-09-08T15:41:14.658 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.658 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.658 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:14.659 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.744224+0000", 2020-09-08T15:41:14.659 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0002028 2020-09-08T15:41:14.659 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.659 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.659 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:14.660 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.744426+0000", 2020-09-08T15:41:14.660 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0046687500000000002 2020-09-08T15:41:14.660 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.660 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.660 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:14.661 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.749095+0000", 2020-09-08T15:41:14.662 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.23249661399999999 2020-09-08T15:41:14.662 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.662 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.662 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:14.663 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.981592+0000", 2020-09-08T15:41:14.663 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010754100000000001 2020-09-08T15:41:14.663 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.663 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.663 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:14.664 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.981699+0000", 2020-09-08T15:41:14.664 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000111724 2020-09-08T15:41:14.664 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.664 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.664 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:14.664 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.981811+0000", 2020-09-08T15:41:14.665 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.23789671500000001 2020-09-08T15:41:14.665 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.665 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.665 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.665 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.666 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.666 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:190048 2.5 2:aada8004:::benchmark_data_smithi166_20551_object42428:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:41:14.666 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:56.766453+0000", 2020-09-08T15:41:14.666 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 77.755814697999995, 2020-09-08T15:41:14.666 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.238099114, 2020-09-08T15:41:14.667 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:14.667 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:14.667 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:14.667 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:14.667 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:14.668 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 190048 2020-09-08T15:41:14.668 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.668 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:14.668 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.668 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:14.668 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.766453+0000", 2020-09-08T15:41:14.669 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:14.669 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.669 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.669 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:14.669 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.766453+0000", 2020-09-08T15:41:14.670 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999776 2020-09-08T15:41:14.670 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.670 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.670 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:14.670 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.766430+0000", 2020-09-08T15:41:14.671 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9654000000000003e-05 2020-09-08T15:41:14.671 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.671 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.671 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:14.671 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.766480+0000", 2020-09-08T15:41:14.672 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1321999999999999e-05 2020-09-08T15:41:14.672 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.672 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.672 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:14.672 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.766491+0000", 2020-09-08T15:41:14.672 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.2060000000000001e-06 2020-09-08T15:41:14.673 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.673 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.673 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:14.673 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.766499+0000", 2020-09-08T15:41:14.673 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.4616000000000002e-05 2020-09-08T15:41:14.674 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.674 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.674 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:14.674 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.766533+0000", 2020-09-08T15:41:14.674 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013506999999999999 2020-09-08T15:41:14.675 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.675 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.675 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:14.675 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.766668+0000", 2020-09-08T15:41:14.675 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.5450000000000003e-05 2020-09-08T15:41:14.676 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.676 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.676 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:14.676 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.766764+0000", 2020-09-08T15:41:14.676 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.004176155 2020-09-08T15:41:14.676 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.677 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.677 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:14.677 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.770940+0000", 2020-09-08T15:41:14.677 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.23336604499999999 2020-09-08T15:41:14.677 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.678 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.678 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:14.678 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.004306+0000", 2020-09-08T15:41:14.678 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000123845 2020-09-08T15:41:14.678 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.679 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.679 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:14.679 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.004430+0000", 2020-09-08T15:41:14.679 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000121987 2020-09-08T15:41:14.679 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.679 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.680 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:14.680 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.004552+0000", 2020-09-08T15:41:14.680 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.238099114 2020-09-08T15:41:14.680 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.680 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.681 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.681 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.681 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.681 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:190078 2.5 2:a346554c:::benchmark_data_smithi166_20551_object42458:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:41:14.681 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:56.806910+0000", 2020-09-08T15:41:14.682 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 77.715357409999996, 2020-09-08T15:41:14.682 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19808690300000001, 2020-09-08T15:41:14.682 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:14.682 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:14.682 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:14.683 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:14.683 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:14.683 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 190078 2020-09-08T15:41:14.683 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.683 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:14.683 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.684 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:14.684 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.806910+0000", 2020-09-08T15:41:14.684 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:14.684 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.684 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.685 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:14.685 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.806910+0000", 2020-09-08T15:41:14.685 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999633 2020-09-08T15:41:14.685 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.686 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.686 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:14.687 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.806873+0000", 2020-09-08T15:41:14.687 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.8899999999999993e-05 2020-09-08T15:41:14.687 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.687 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.687 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:14.688 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.806952+0000", 2020-09-08T15:41:14.688 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7453999999999999e-05 2020-09-08T15:41:14.688 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.688 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.688 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:14.689 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.806970+0000", 2020-09-08T15:41:14.689 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.473e-06 2020-09-08T15:41:14.689 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.689 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.690 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:14.690 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.806978+0000", 2020-09-08T15:41:14.690 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0358999999999997e-05 2020-09-08T15:41:14.690 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.690 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.691 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:14.691 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.807029+0000", 2020-09-08T15:41:14.691 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000171743 2020-09-08T15:41:14.691 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.691 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.691 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:14.692 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.807200+0000", 2020-09-08T15:41:14.692 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018509099999999999 2020-09-08T15:41:14.693 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.693 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.693 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:14.693 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.807385+0000", 2020-09-08T15:41:14.694 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0092689729999999998 2020-09-08T15:41:14.694 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.694 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.694 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:14.694 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.816654+0000", 2020-09-08T15:41:14.694 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18805126799999999 2020-09-08T15:41:14.695 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.695 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.695 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:14.695 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.004706+0000", 2020-09-08T15:41:14.695 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014980899999999999 2020-09-08T15:41:14.696 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.696 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.696 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:14.696 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.004855+0000", 2020-09-08T15:41:14.696 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000141433 2020-09-08T15:41:14.697 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.697 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.697 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:14.697 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.004997+0000", 2020-09-08T15:41:14.697 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19808690300000001 2020-09-08T15:41:14.698 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.698 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.698 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.698 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.698 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.699 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:190080 2.5 2:ad471823:::benchmark_data_smithi166_20551_object42460:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:41:14.699 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:56.809412+0000", 2020-09-08T15:41:14.699 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 77.71285537, 2020-09-08T15:41:14.699 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19592162900000001, 2020-09-08T15:41:14.700 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:14.700 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:14.700 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:14.700 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:14.700 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:14.700 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 190080 2020-09-08T15:41:14.701 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.701 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:14.701 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.701 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:14.702 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.809412+0000", 2020-09-08T15:41:14.702 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:14.702 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.702 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.702 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:14.702 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.809412+0000", 2020-09-08T15:41:14.703 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999661 2020-09-08T15:41:14.703 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.703 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.703 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:14.703 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.809378+0000", 2020-09-08T15:41:14.704 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7377000000000001e-05 2020-09-08T15:41:14.704 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.704 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.704 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:14.704 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.809455+0000", 2020-09-08T15:41:14.705 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6597999999999999e-05 2020-09-08T15:41:14.705 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.705 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.705 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:14.705 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.809472+0000", 2020-09-08T15:41:14.706 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5200000000000003e-06 2020-09-08T15:41:14.706 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.706 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.706 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:14.706 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.809478+0000", 2020-09-08T15:41:14.706 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0020781829999999999 2020-09-08T15:41:14.707 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.707 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.707 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:14.707 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.811557+0000", 2020-09-08T15:41:14.707 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000173401 2020-09-08T15:41:14.708 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.709 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.709 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:14.709 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.811730+0000", 2020-09-08T15:41:14.710 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000195064 2020-09-08T15:41:14.710 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.710 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.710 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:14.710 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.811925+0000", 2020-09-08T15:41:14.711 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0049081790000000004 2020-09-08T15:41:14.711 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.711 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.711 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:14.711 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.816833+0000", 2020-09-08T15:41:14.712 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.188264762 2020-09-08T15:41:14.712 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.712 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.712 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:14.712 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.005098+0000", 2020-09-08T15:41:14.713 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000141617 2020-09-08T15:41:14.713 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.713 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.713 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:14.713 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.005240+0000", 2020-09-08T15:41:14.714 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.3998000000000004e-05 2020-09-08T15:41:14.714 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.714 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.714 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:14.714 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.005334+0000", 2020-09-08T15:41:14.715 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19592162900000001 2020-09-08T15:41:14.715 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.715 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.715 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.715 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.715 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.716 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:190996 2.5 2:accb40be:::benchmark_data_smithi166_20551_object43376:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:41:14.716 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:57.853842+0000", 2020-09-08T15:41:14.716 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 76.668425553000006, 2020-09-08T15:41:14.716 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.226087756, 2020-09-08T15:41:14.716 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:14.717 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:14.717 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:14.717 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:14.717 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:14.717 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 190996 2020-09-08T15:41:14.718 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.718 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:14.718 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.718 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:14.718 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.853842+0000", 2020-09-08T15:41:14.719 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:14.719 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.719 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.719 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:14.719 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.853842+0000", 2020-09-08T15:41:14.720 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999614 2020-09-08T15:41:14.720 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.720 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.720 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:14.720 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.853803+0000", 2020-09-08T15:41:14.722 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8703999999999996e-05 2020-09-08T15:41:14.722 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.722 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.722 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:14.722 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.853892+0000", 2020-09-08T15:41:14.723 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6443000000000002e-05 2020-09-08T15:41:14.723 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.723 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.723 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:14.723 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.853908+0000", 2020-09-08T15:41:14.724 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.083e-06 2020-09-08T15:41:14.724 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.724 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.724 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:14.724 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.853914+0000", 2020-09-08T15:41:14.725 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012618199999999999 2020-09-08T15:41:14.725 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.725 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.725 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:14.725 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.854041+0000", 2020-09-08T15:41:14.726 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000176226 2020-09-08T15:41:14.726 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.726 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.726 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:14.726 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.854217+0000", 2020-09-08T15:41:14.727 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013297 2020-09-08T15:41:14.727 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.727 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.727 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:14.729 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.854350+0000", 2020-09-08T15:41:14.729 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0042987959999999997 2020-09-08T15:41:14.729 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.729 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.730 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:14.730 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.858649+0000", 2020-09-08T15:41:14.730 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.220929282 2020-09-08T15:41:14.730 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.731 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.731 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:14.731 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.079578+0000", 2020-09-08T15:41:14.731 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016881200000000001 2020-09-08T15:41:14.732 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.732 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.732 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:14.732 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.079747+0000", 2020-09-08T15:41:14.732 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018289599999999999 2020-09-08T15:41:14.733 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.733 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.733 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:14.733 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.079930+0000", 2020-09-08T15:41:14.734 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.226087756 2020-09-08T15:41:14.734 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.734 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.734 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.734 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.735 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.735 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:191002 2.5 2:a497d67f:::benchmark_data_smithi166_20551_object43382:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:41:14.735 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:57.861883+0000", 2020-09-08T15:41:14.735 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 76.660384269000005, 2020-09-08T15:41:14.736 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21849718600000001, 2020-09-08T15:41:14.736 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:14.737 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:14.737 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:14.737 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:14.737 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:14.738 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 191002 2020-09-08T15:41:14.738 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.738 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:14.739 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.739 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:14.739 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.861883+0000", 2020-09-08T15:41:14.739 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:14.740 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.740 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.740 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:14.740 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.861883+0000", 2020-09-08T15:41:14.740 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999547 2020-09-08T15:41:14.741 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.741 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.741 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:14.741 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.861838+0000", 2020-09-08T15:41:14.741 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011824600000000001 2020-09-08T15:41:14.742 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.742 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.742 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:14.742 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.861956+0000", 2020-09-08T15:41:14.743 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.1251e-05 2020-09-08T15:41:14.743 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.743 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.743 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:14.744 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.861977+0000", 2020-09-08T15:41:14.744 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.1940000000000001e-06 2020-09-08T15:41:14.744 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.744 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.744 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:14.745 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.861985+0000", 2020-09-08T15:41:14.745 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.3094999999999999e-05 2020-09-08T15:41:14.745 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.745 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.745 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:14.746 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862038+0000", 2020-09-08T15:41:14.746 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012477500000000001 2020-09-08T15:41:14.746 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.746 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.747 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:14.747 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862162+0000", 2020-09-08T15:41:14.747 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016471899999999999 2020-09-08T15:41:14.747 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.747 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.748 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:14.748 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862327+0000", 2020-09-08T15:41:14.748 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.010215866000000001 2020-09-08T15:41:14.748 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.749 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.749 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:14.749 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.872543+0000", 2020-09-08T15:41:14.749 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20754151300000001 2020-09-08T15:41:14.750 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.750 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.750 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:14.750 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.080085+0000", 2020-09-08T15:41:14.750 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019456599999999999 2020-09-08T15:41:14.751 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.751 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.751 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:14.751 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.080279+0000", 2020-09-08T15:41:14.751 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000101184 2020-09-08T15:41:14.752 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.752 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.752 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:14.752 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.080380+0000", 2020-09-08T15:41:14.753 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21849718600000001 2020-09-08T15:41:14.753 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.753 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.753 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.753 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.754 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.754 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:191006 2.5 2:ab629002:::benchmark_data_smithi166_20551_object43386:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:41:14.754 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:57.862749+0000", 2020-09-08T15:41:14.754 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 76.659518523000003, 2020-09-08T15:41:14.755 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.224825307, 2020-09-08T15:41:14.755 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:14.755 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:14.755 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:14.755 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:14.756 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:14.756 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 191006 2020-09-08T15:41:14.756 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.757 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:14.757 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.757 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:14.757 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862749+0000", 2020-09-08T15:41:14.758 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:14.758 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.758 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.759 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:14.759 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862749+0000", 2020-09-08T15:41:14.759 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999537 2020-09-08T15:41:14.759 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.759 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.760 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:14.760 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862702+0000", 2020-09-08T15:41:14.760 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000106022 2020-09-08T15:41:14.760 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.761 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.761 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:14.761 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862808+0000", 2020-09-08T15:41:14.761 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.9309000000000001e-05 2020-09-08T15:41:14.761 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.762 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.762 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:14.762 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862828+0000", 2020-09-08T15:41:14.762 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2070000000000001e-06 2020-09-08T15:41:14.762 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.763 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.763 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:14.763 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862837+0000", 2020-09-08T15:41:14.763 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0051266419999999998 2020-09-08T15:41:14.764 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.764 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.764 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:14.764 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.867964+0000", 2020-09-08T15:41:14.764 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014680499999999999 2020-09-08T15:41:14.765 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.765 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.765 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:14.765 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.868110+0000", 2020-09-08T15:41:14.766 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019938700000000001 2020-09-08T15:41:14.766 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.766 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.766 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:14.766 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.868310+0000", 2020-09-08T15:41:14.767 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0043860180000000002 2020-09-08T15:41:14.767 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.767 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.767 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:14.768 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.872696+0000", 2020-09-08T15:41:14.768 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21478683100000001 2020-09-08T15:41:14.768 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.768 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.768 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:14.769 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.087483+0000", 2020-09-08T15:41:14.769 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.3461000000000002e-05 2020-09-08T15:41:14.769 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.769 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.770 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:14.770 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.087536+0000", 2020-09-08T15:41:14.770 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.8077999999999997e-05 2020-09-08T15:41:14.770 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.770 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.771 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:14.771 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.087574+0000", 2020-09-08T15:41:14.771 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.224825307 2020-09-08T15:41:14.771 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.772 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.772 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.772 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.772 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.772 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:191024 2.5 2:a481b27b:::benchmark_data_smithi166_20551_object43404:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:41:14.773 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:57.900157+0000", 2020-09-08T15:41:14.773 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 76.622110754999994, 2020-09-08T15:41:14.773 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18752587100000001, 2020-09-08T15:41:14.773 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:14.774 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:14.774 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:14.774 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:14.774 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:14.774 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 191024 2020-09-08T15:41:14.775 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.775 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:14.775 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.775 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:14.776 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.900157+0000", 2020-09-08T15:41:14.776 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:14.776 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.776 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.776 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:14.777 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.900157+0000", 2020-09-08T15:41:14.777 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999723 2020-09-08T15:41:14.777 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.777 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.778 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:14.778 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.900129+0000", 2020-09-08T15:41:14.778 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.1223999999999995e-05 2020-09-08T15:41:14.778 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.778 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.779 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:14.779 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.900190+0000", 2020-09-08T15:41:14.779 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3706999999999999e-05 2020-09-08T15:41:14.779 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.780 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.780 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:14.780 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.900204+0000", 2020-09-08T15:41:14.780 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.49e-06 2020-09-08T15:41:14.780 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.781 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.781 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:14.781 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.900209+0000", 2020-09-08T15:41:14.781 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8096999999999994e-05 2020-09-08T15:41:14.782 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.782 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.782 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:14.782 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.900298+0000", 2020-09-08T15:41:14.783 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000202022 2020-09-08T15:41:14.783 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.783 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.784 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:14.784 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.900500+0000", 2020-09-08T15:41:14.784 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00020663999999999999 2020-09-08T15:41:14.784 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.785 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.785 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:14.785 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.900706+0000", 2020-09-08T15:41:14.785 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0064340400000000002 2020-09-08T15:41:14.785 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.786 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.786 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:14.786 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.907140+0000", 2020-09-08T15:41:14.786 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18047359700000001 2020-09-08T15:41:14.787 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.787 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.787 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:14.787 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.087614+0000", 2020-09-08T15:41:14.787 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.9319000000000001e-05 2020-09-08T15:41:14.788 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.788 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.788 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:14.788 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.087653+0000", 2020-09-08T15:41:14.789 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.932e-05 2020-09-08T15:41:14.789 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.789 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.789 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:14.790 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.087682+0000", 2020-09-08T15:41:14.790 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18752587100000001 2020-09-08T15:41:14.790 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.790 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.791 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.791 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.791 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.792 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223538 2.5 2:af9447da:::benchmark_data_smithi166_20551_object75918:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:41:14.792 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.454811+0000", 2020-09-08T15:41:14.792 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 43.067456761999999, 2020-09-08T15:41:14.792 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.31664958199999999, 2020-09-08T15:41:14.793 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:14.793 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:14.793 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:14.793 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:14.793 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:14.794 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223538 2020-09-08T15:41:14.794 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.794 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:14.794 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.795 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:14.795 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454811+0000", 2020-09-08T15:41:14.795 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:14.795 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.795 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.796 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:14.796 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454811+0000", 2020-09-08T15:41:14.796 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.999958 2020-09-08T15:41:14.796 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.797 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.797 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:14.797 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454768+0000", 2020-09-08T15:41:14.797 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.0786999999999997e-05 2020-09-08T15:41:14.798 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.798 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.798 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:14.798 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454859+0000", 2020-09-08T15:41:14.798 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.0344999999999999e-05 2020-09-08T15:41:14.799 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.799 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.799 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:14.799 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454880+0000", 2020-09-08T15:41:14.800 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.5410000000000003e-06 2020-09-08T15:41:14.800 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.800 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.800 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:14.800 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454887+0000", 2020-09-08T15:41:14.801 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.7349999999999999e-05 2020-09-08T15:41:14.801 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.801 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.801 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:14.802 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454935+0000", 2020-09-08T15:41:14.802 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000126422 2020-09-08T15:41:14.802 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.802 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.803 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:14.803 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.455061+0000", 2020-09-08T15:41:14.803 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013292600000000001 2020-09-08T15:41:14.803 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.803 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.804 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:14.804 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.455194+0000", 2020-09-08T15:41:14.804 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.010390197 2020-09-08T15:41:14.804 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.805 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.805 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:14.805 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.465584+0000", 2020-09-08T15:41:14.805 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.305744764 2020-09-08T15:41:14.805 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.806 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.806 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:14.806 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771329+0000", 2020-09-08T15:41:14.806 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.0704999999999995e-05 2020-09-08T15:41:14.807 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.807 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.807 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:14.807 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771400+0000", 2020-09-08T15:41:14.807 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.0677999999999997e-05 2020-09-08T15:41:14.808 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.808 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.808 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:14.808 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771460+0000", 2020-09-08T15:41:14.810 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.31664958199999999 2020-09-08T15:41:14.811 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.811 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.811 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.811 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.812 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.812 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223539 2.5 2:ac3e761c:::benchmark_data_smithi166_20551_object75919:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:41:14.812 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.455043+0000", 2020-09-08T15:41:14.812 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 43.067224193999998, 2020-09-08T15:41:14.813 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.316615533, 2020-09-08T15:41:14.813 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:14.813 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:14.813 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:14.813 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:14.814 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:14.814 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223539 2020-09-08T15:41:14.814 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.814 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:14.814 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.815 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:14.815 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.455043+0000", 2020-09-08T15:41:14.815 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:14.815 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.815 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.816 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:14.816 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.455043+0000", 2020-09-08T15:41:14.816 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999542 2020-09-08T15:41:14.816 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.817 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.817 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:14.817 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454997+0000", 2020-09-08T15:41:14.818 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000103986 2020-09-08T15:41:14.818 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.818 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.818 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:14.818 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.455101+0000", 2020-09-08T15:41:14.819 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6912999999999999e-05 2020-09-08T15:41:14.819 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.819 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.819 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:14.819 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.455118+0000", 2020-09-08T15:41:14.820 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.9060000000000002e-06 2020-09-08T15:41:14.820 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.820 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.820 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:14.821 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.455126+0000", 2020-09-08T15:41:14.821 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0052929229999999997 2020-09-08T15:41:14.821 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.821 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.822 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:14.822 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.460419+0000", 2020-09-08T15:41:14.822 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018929099999999999 2020-09-08T15:41:14.822 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.822 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.823 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:14.823 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.460608+0000", 2020-09-08T15:41:14.823 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000212525 2020-09-08T15:41:14.823 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.823 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.824 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:14.824 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.460821+0000", 2020-09-08T15:41:14.824 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0049404239999999997 2020-09-08T15:41:14.824 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.825 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.825 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:14.825 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.465761+0000", 2020-09-08T15:41:14.825 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.30577196099999998 2020-09-08T15:41:14.825 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.826 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.826 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:14.826 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771533+0000", 2020-09-08T15:41:14.826 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.7619000000000004e-05 2020-09-08T15:41:14.827 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.827 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.827 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:14.827 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771601+0000", 2020-09-08T15:41:14.827 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7948999999999997e-05 2020-09-08T15:41:14.828 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.828 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.828 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:14.828 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771659+0000", 2020-09-08T15:41:14.829 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.316615533 2020-09-08T15:41:14.829 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.829 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.829 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.829 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.830 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.830 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223553 2.5 2:af0eb85e:::benchmark_data_smithi166_20551_object75933:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:41:14.830 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.472693+0000", 2020-09-08T15:41:14.830 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 43.049574051, 2020-09-08T15:41:14.831 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.299064581, 2020-09-08T15:41:14.831 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:14.831 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:14.831 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:14.831 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:14.832 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:14.832 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223553 2020-09-08T15:41:14.832 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.832 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:14.832 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.833 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:14.833 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.472693+0000", 2020-09-08T15:41:14.833 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:14.833 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.834 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.834 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:14.834 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.472693+0000", 2020-09-08T15:41:14.834 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999433 2020-09-08T15:41:14.834 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.835 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.835 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:14.835 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.472636+0000", 2020-09-08T15:41:14.835 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000126057 2020-09-08T15:41:14.835 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.836 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.836 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:14.836 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.472762+0000", 2020-09-08T15:41:14.836 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.4116000000000001e-05 2020-09-08T15:41:14.837 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.837 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.837 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:14.837 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.472787+0000", 2020-09-08T15:41:14.838 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1021e-05 2020-09-08T15:41:14.838 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.838 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.838 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:14.839 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.472798+0000", 2020-09-08T15:41:14.839 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8405000000000001e-05 2020-09-08T15:41:14.839 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.839 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.840 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:14.840 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.472856+0000", 2020-09-08T15:41:14.840 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000129591 2020-09-08T15:41:14.840 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.841 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.841 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:14.841 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.472986+0000", 2020-09-08T15:41:14.841 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000115479 2020-09-08T15:41:14.841 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.842 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.842 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:14.842 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.473101+0000", 2020-09-08T15:41:14.842 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.010937853000000001 2020-09-08T15:41:14.842 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.843 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.843 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:14.843 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.484039+0000", 2020-09-08T15:41:14.843 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.28766051399999998 2020-09-08T15:41:14.844 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.844 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.844 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:14.844 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771699+0000", 2020-09-08T15:41:14.845 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.1099000000000002e-05 2020-09-08T15:41:14.845 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.845 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.845 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:14.845 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771731+0000", 2020-09-08T15:41:14.846 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.7396e-05 2020-09-08T15:41:14.846 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.846 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.846 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:14.847 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771758+0000", 2020-09-08T15:41:14.847 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.299064581 2020-09-08T15:41:14.847 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.847 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.847 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.848 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.848 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.848 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223558 2.5 2:ae593298:::benchmark_data_smithi166_20551_object75938:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:41:14.848 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.475216+0000", 2020-09-08T15:41:14.848 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 43.047051084000003, 2020-09-08T15:41:14.849 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.29670734500000001, 2020-09-08T15:41:14.849 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:14.849 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:14.850 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:14.850 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:14.851 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:14.851 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223558 2020-09-08T15:41:14.851 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.851 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:14.852 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.852 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:14.852 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.475216+0000", 2020-09-08T15:41:14.852 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:14.852 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.853 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.853 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:14.853 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.475216+0000", 2020-09-08T15:41:14.853 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999781 2020-09-08T15:41:14.853 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.854 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.854 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:14.854 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.475194+0000", 2020-09-08T15:41:14.854 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0077000000000002e-05 2020-09-08T15:41:14.854 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.855 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.855 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:14.855 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.475245+0000", 2020-09-08T15:41:14.855 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4603999999999999e-05 2020-09-08T15:41:14.855 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.856 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.856 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:14.856 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.475259+0000", 2020-09-08T15:41:14.856 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0710000000000002e-06 2020-09-08T15:41:14.856 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.857 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.858 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:14.858 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.475264+0000", 2020-09-08T15:41:14.858 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0035202369999999998 2020-09-08T15:41:14.858 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.859 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.859 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:14.859 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.478784+0000", 2020-09-08T15:41:14.859 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016699 2020-09-08T15:41:14.859 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.859 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.860 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:14.860 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.478951+0000", 2020-09-08T15:41:14.860 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000185355 2020-09-08T15:41:14.860 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.860 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.861 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:14.861 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.479137+0000", 2020-09-08T15:41:14.861 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0050839780000000003 2020-09-08T15:41:14.861 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.861 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.862 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:14.862 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.484221+0000", 2020-09-08T15:41:14.862 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.28760382800000001 2020-09-08T15:41:14.862 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.863 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.863 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:14.863 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771825+0000", 2020-09-08T15:41:14.863 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.2598e-05 2020-09-08T15:41:14.863 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.864 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.864 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:14.864 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771877+0000", 2020-09-08T15:41:14.865 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.6451e-05 2020-09-08T15:41:14.865 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.865 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.865 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:14.865 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771924+0000", 2020-09-08T15:41:14.866 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.29670734500000001 2020-09-08T15:41:14.866 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.866 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.866 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.866 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.867 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.867 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223583 2.5 2:a056f7fe:::benchmark_data_smithi166_20551_object75963:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:41:14.867 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.520982+0000", 2020-09-08T15:41:14.867 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 43.001284922000004, 2020-09-08T15:41:14.867 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25551785999999999, 2020-09-08T15:41:14.868 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:14.868 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:14.868 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:14.868 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:14.868 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:14.869 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223583 2020-09-08T15:41:14.869 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.869 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:14.869 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.869 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:14.870 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.520982+0000", 2020-09-08T15:41:14.870 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:14.870 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.870 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.870 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:14.870 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.520982+0000", 2020-09-08T15:41:14.871 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999781 2020-09-08T15:41:14.871 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.871 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.871 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:14.871 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.520960+0000", 2020-09-08T15:41:14.872 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0510999999999998e-05 2020-09-08T15:41:14.872 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.874 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.874 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:14.874 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.521011+0000", 2020-09-08T15:41:14.874 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.6080000000000002e-06 2020-09-08T15:41:14.875 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.875 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.875 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:14.875 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.521020+0000", 2020-09-08T15:41:14.875 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1560000000000003e-06 2020-09-08T15:41:14.876 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.876 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.876 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:14.876 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.521026+0000", 2020-09-08T15:41:14.876 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.5790000000000005e-05 2020-09-08T15:41:14.877 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.877 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.877 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:14.877 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.521101+0000", 2020-09-08T15:41:14.877 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00025211500000000002 2020-09-08T15:41:14.878 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.878 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.878 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:14.878 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.521354+0000", 2020-09-08T15:41:14.878 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000217529 2020-09-08T15:41:14.879 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.879 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.879 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:14.879 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.521571+0000", 2020-09-08T15:41:14.879 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0049845890000000002 2020-09-08T15:41:14.880 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.880 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.880 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:14.880 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.526556+0000", 2020-09-08T15:41:14.880 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.24970619599999999 2020-09-08T15:41:14.880 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.881 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.881 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:14.881 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.776262+0000", 2020-09-08T15:41:14.881 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000159863 2020-09-08T15:41:14.881 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.882 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.882 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:14.882 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.776422+0000", 2020-09-08T15:41:14.882 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.8588e-05 2020-09-08T15:41:14.882 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:14.883 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:14.883 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:14.883 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.776500+0000", 2020-09-08T15:41:14.883 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25551785999999999 2020-09-08T15:41:14.883 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.884 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.884 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.884 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:14.884 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:14.884 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:41:14.885 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:14.887 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:41:14 smithi104 bash[22305]: debug 2020-09-08T15:41:14.497+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:14.888 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:14 smithi104 bash[9203]: cluster 2020-09-08T15:41:13.647513+0000 mgr.x (mgr.24118) 413 : cluster [DBG] pgmap v534: 17 pgs: 17 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.18k op/s 2020-09-08T15:41:14.888 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:14 smithi166 bash[9446]: cluster 2020-09-08T15:41:13.647513+0000 mgr.x (mgr.24118) 413 : cluster [DBG] pgmap v534: 17 pgs: 17 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.18k op/s 2020-09-08T15:41:15.158 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T15:41:15.306 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:14 smithi055 bash[9296]: cluster 2020-09-08T15:41:13.647513+0000 mgr.x (mgr.24118) 413 : cluster [DBG] pgmap v534: 17 pgs: 17 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.18k op/s 2020-09-08T15:41:15.306 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2020-09-08T15:41:15.399 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:15.502 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:15.545 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_ops_in_flight 2020-09-08T15:41:15.632 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:15.689 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:15 smithi166 bash[9446]: audit 2020-09-08T15:41:15.144706+0000 mgr.x (mgr.24118) 414 : audit [DBG] from='client.34973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:41:15.689 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:15 smithi166 bash[9446]: cluster 2020-09-08T15:41:15.648017+0000 mgr.x (mgr.24118) 415 : cluster [DBG] pgmap v535: 17 pgs: 17 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T15:41:15.690 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:41:15 smithi166 bash[16593]: debug 2020-09-08T15:41:15.400+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:15.690 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:41:15 smithi166 bash[16593]: debug 2020-09-08T15:41:15.624+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:15.734 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:15.750 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:41:16.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:15 smithi055 bash[9296]: audit 2020-09-08T15:41:15.144706+0000 mgr.x (mgr.24118) 414 : audit [DBG] from='client.34973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:41:16.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:15 smithi055 bash[9296]: cluster 2020-09-08T15:41:15.648017+0000 mgr.x (mgr.24118) 415 : cluster [DBG] pgmap v535: 17 pgs: 17 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T15:41:16.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:15 smithi104 bash[9203]: audit 2020-09-08T15:41:15.144706+0000 mgr.x (mgr.24118) 414 : audit [DBG] from='client.34973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:41:16.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:15 smithi104 bash[9203]: cluster 2020-09-08T15:41:15.648017+0000 mgr.x (mgr.24118) 415 : cluster [DBG] pgmap v535: 17 pgs: 17 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T15:41:16.227 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:16.334 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:16.495 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:16.599 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:16.607 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:41:16 smithi104 bash[15273]: debug 2020-09-08T15:41:16.229+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:16.615 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:41:16 smithi166 bash[16593]: debug 2020-09-08T15:41:16.488+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:16.815 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:16.890 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:41:16.899 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:41:16 smithi104 bash[19922]: debug 2020-09-08T15:41:16.813+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:16.921 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":534,"stamp":"2020-09-08T15:41:13.647263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2103836696,"num_objects":32106,"num_object_clones":0,"num_object_copies":64212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32106,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263136,"num_write_kb":9447492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":34,"acting":34,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":4834912,"kb_used_data":4191232,"kb_used_omap":0,"kb_used_meta":643200,"kb_avail":1120106912,"statfs":{"total":1151940427776,"available":1146989477888,"internally_reserved":0,"allocated":4291821568,"data_stored":4288526992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":658636800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,4,7,7,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-619053056,"num_objects":-9446,"num_object_clones":0,"num_object_copies":-18892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-9446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9446,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.996442"},"pg_stats":[{"pgid":"2.f","version":"200'16215","reported_seq":"16302","reported_epoch":"200","state":"active+clean","last_fresh":"2020-09-08T15:41:11.770075+0000","last_change":"2020-09-08T15:41:07.772966+0000","last_active":"2020-09-08T15:41:11.770075+0000","last_peered":"2020-09-08T15:41:11.770075+0000","last_clean":"2020-09-08T15:41:11.770075+0000","last_became_active":"2020-09-08T15:37:29.882656+0000","last_became_peered":"2020-09-08T15:37:29.882656+0000","last_unstale":"2020-09-08T15:41:11.770075+0000","last_undegraded":"2020-09-08T15:41:11.770075+0000","last_fullsized":"2020-09-08T15:41:11.770075+0000","mapping_epoch":126,"log_start":"147'6200","ondisk_log_start":"147'6200","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15972","last_scrub_stamp":"2020-09-08T15:41:07.772896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:41:07.772896+0000","log_size":10015,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":130482176,"num_objects":1991,"num_object_clones":0,"num_object_copies":3982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16215,"num_write_kb":582592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9],"acting":[8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"200'16417","reported_seq":"16504","reported_epoch":"200","state":"active+clean","last_fresh":"2020-09-08T15:41:11.815593+0000","last_change":"2020-09-08T15:41:05.185480+0000","last_active":"2020-09-08T15:41:11.815593+0000","last_peered":"2020-09-08T15:41:11.815593+0000","last_clean":"2020-09-08T15:41:11.815593+0000","last_became_active":"2020-09-08T15:37:30.089439+0000","last_became_peered":"2020-09-08T15:37:30.089439+0000","last_unstale":"2020-09-08T15:41:11.815593+0000","last_undegraded":"2020-09-08T15:41:11.815593+0000","last_fullsized":"2020-09-08T15:41:11.815593+0000","mapping_epoch":126,"log_start":"148'6400","ondisk_log_start":"148'6400","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'16039","last_scrub_stamp":"2020-09-08T15:41:05.185405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:41:05.185405+0000","log_size":10017,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":133627904,"num_objects":2039,"num_object_clones":0,"num_object_copies":4078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16417,"num_write_kb":590592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"200'16380","reported_seq":"16467","reported_epoch":"200","state":"active+clean","last_fresh":"2020-09-08T15:41:11.562648+0000","last_change":"2020-09-08T15:40:36.687544+0000","last_active":"2020-09-08T15:41:11.562648+0000","last_peered":"2020-09-08T15:41:11.562648+0000","last_clean":"2020-09-08T15:41:11.562648+0000","last_became_active":"2020-09-08T15:37:29.876533+0000","last_became_peered":"2020-09-08T15:37:29.876533+0000","last_unstale":"2020-09-08T15:41:11.562648+0000","last_undegraded":"2020-09-08T15:41:11.562648+0000","last_fullsized":"2020-09-08T15:41:11.562648+0000","mapping_epoch":126,"log_start":"147'6300","ondisk_log_start":"147'6300","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"190'14202","last_scrub_stamp":"2020-09-08T15:40:36.687465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:36.687465+0000","log_size":10080,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":134217728,"num_objects":2048,"num_object_clones":0,"num_object_copies":4096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16380,"num_write_kb":589696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"200'16666","reported_seq":"16753","reported_epoch":"200","state":"active+clean","last_fresh":"2020-09-08T15:41:11.772924+0000","last_change":"2020-09-08T15:40:35.304847+0000","last_active":"2020-09-08T15:41:11.772924+0000","last_peered":"2020-09-08T15:41:11.772924+0000","last_clean":"2020-09-08T15:41:11.772924+0000","last_became_active":"2020-09-08T15:37:30.091928+0000","last_became_peered":"2020-09-08T15:37:30.091928+0000","last_unstale":"2020-09-08T15:41:11.772924+0000","last_undegraded":"2020-09-08T15:41:11.772924+0000","last_fullsized":"2020-09-08T15:41:11.772924+0000","mapping_epoch":126,"log_start":"149'6600","ondisk_log_start":"149'6600","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"190'14394","last_scrub_stamp":"2020-09-08T15:40:35.304761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:35.304761+0000","log_size":10066,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":133169152,"num_objects":2032,"num_object_clones":0,"num_object_copies":4064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16666,"num_write_kb":598336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"200'16480","reported_seq":"16567","reported_epoch":"200","state":"active+clean","last_fresh":"2020-09-08T15:41:11.667109+0000","last_change":"2020-09-08T15:41:03.134319+0000","last_active":"2020-09-08T15:41:11.667109+0000","last_peered":"2020-09-08T15:41:11.667109+0000","last_clean":"2020-09-08T15:41:11.667109+0000","last_became_active":"2020-09-08T15:37:29.875715+0000","last_became_peered":"2020-09-08T15:37:29.875715+0000","last_unstale":"2020-09-08T15:41:11.667109+0000","last_undegraded":"2020-09-08T15:41:11.667109+0000","last_fullsized":"2020-09-08T15:41:11.667109+0000","mapping_epoch":126,"log_start":"147'6400","ondisk_log_start":"147'6400","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'15995","last_scrub_stamp":"2020-09-08T15:41:03.134240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:41:03.134240+0000","log_size":10080,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":124387328,"num_objects":1898,"num_object_clones":0,"num_object_copies":3796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16480,"num_write_kb":588096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"99'6","reported_seq":"217","reported_epoch":"200","state":"active+clean","last_fresh":"2020-09-08T15:41:06.663531+0000","last_change":"2020-09-08T15:41:03.497368+0000","last_active":"2020-09-08T15:41:06.663531+0000","last_peered":"2020-09-08T15:41:06.663531+0000","last_clean":"2020-09-08T15:41:06.663531+0000","last_became_active":"2020-09-08T15:34:43.031697+0000","last_became_peered":"2020-09-08T15:34:43.031697+0000","last_unstale":"2020-09-08T15:41:06.663531+0000","last_undegraded":"2020-09-08T15:41:06.663531+0000","last_fullsized":"2020-09-08T15:41:06.663531+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":2,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'6","last_scrub_stamp":"2020-09-08T15:41:03.497248+0000","last_deep_scrub":"99'6","last_deep_scrub_stamp":"2020-09-08T15:35:23.132465+0000","last_clean_scrub_stamp":"2020-09-08T15:41:03.497248+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"201'16523","reported_seq":"16611","reported_epoch":"201","state":"active+clean","last_fresh":"2020-09-08T15:41:13.617054+0000","last_change":"2020-09-08T15:40:14.389420+0000","last_active":"2020-09-08T15:41:13.617054+0000","last_peered":"2020-09-08T15:41:13.617054+0000","last_clean":"2020-09-08T15:41:13.617054+0000","last_became_active":"2020-09-08T15:37:30.094918+0000","last_became_peered":"2020-09-08T15:37:30.094918+0000","last_unstale":"2020-09-08T15:41:13.617054+0000","last_undegraded":"2020-09-08T15:41:13.617054+0000","last_fullsized":"2020-09-08T15:41:13.617054+0000","mapping_epoch":126,"log_start":"149'6500","ondisk_log_start":"149'6500","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'12937","last_scrub_stamp":"2020-09-08T15:40:14.389373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:14.389373+0000","log_size":10023,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":129826816,"num_objects":1981,"num_object_clones":0,"num_object_copies":3962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16523,"num_write_kb":592128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"200'16239","reported_seq":"16323","reported_epoch":"200","state":"active+clean","last_fresh":"2020-09-08T15:41:11.779968+0000","last_change":"2020-09-08T15:39:31.634079+0000","last_active":"2020-09-08T15:41:11.779968+0000","last_peered":"2020-09-08T15:41:11.779968+0000","last_clean":"2020-09-08T15:41:11.779968+0000","last_became_active":"2020-09-08T15:37:30.090699+0000","last_became_peered":"2020-09-08T15:37:30.090699+0000","last_unstale":"2020-09-08T15:41:11.779968+0000","last_undegraded":"2020-09-08T15:41:11.779968+0000","last_fullsized":"2020-09-08T15:41:11.779968+0000","mapping_epoch":126,"log_start":"147'6200","ondisk_log_start":"147'6200","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"170'10244","last_scrub_stamp":"2020-09-08T15:39:31.634020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:39:31.634020+0000","log_size":10039,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":133234688,"num_objects":2033,"num_object_clones":0,"num_object_copies":4066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16239,"num_write_kb":584704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.6","version":"200'16438","reported_seq":"16527","reported_epoch":"200","state":"active+clean","last_fresh":"2020-09-08T15:41:11.795416+0000","last_change":"2020-09-08T15:40:26.436714+0000","last_active":"2020-09-08T15:41:11.795416+0000","last_peered":"2020-09-08T15:41:11.795416+0000","last_clean":"2020-09-08T15:41:11.795416+0000","last_became_active":"2020-09-08T15:37:29.889471+0000","last_became_peered":"2020-09-08T15:37:29.889471+0000","last_unstale":"2020-09-08T15:41:11.795416+0000","last_undegraded":"2020-09-08T15:41:11.795416+0000","last_fullsized":"2020-09-08T15:41:11.795416+0000","mapping_epoch":126,"log_start":"148'6400","ondisk_log_start":"148'6400","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'13713","last_scrub_stamp":"2020-09-08T15:40:26.436603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:26.436603+0000","log_size":10038,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":130809856,"num_objects":1996,"num_object_clones":0,"num_object_copies":3992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16438,"num_write_kb":589888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"200'16488","reported_seq":"16575","reported_epoch":"200","state":"active+clean","last_fresh":"2020-09-08T15:41:11.659398+0000","last_change":"2020-09-08T15:40:23.781832+0000","last_active":"2020-09-08T15:41:11.659398+0000","last_peered":"2020-09-08T15:41:11.659398+0000","last_clean":"2020-09-08T15:41:11.659398+0000","last_became_active":"2020-09-08T15:37:30.095291+0000","last_became_peered":"2020-09-08T15:37:30.095291+0000","last_unstale":"2020-09-08T15:41:11.659398+0000","last_undegraded":"2020-09-08T15:41:11.659398+0000","last_fullsized":"2020-09-08T15:41:11.659398+0000","mapping_epoch":126,"log_start":"147'6400","ondisk_log_start":"147'6400","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'13605","last_scrub_stamp":"2020-09-08T15:40:23.781740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:23.781740+0000","log_size":10088,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":133693440,"num_objects":2040,"num_object_clones":0,"num_object_copies":4080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16488,"num_write_kb":592896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.8","version":"201'16408","reported_seq":"16496","reported_epoch":"201","state":"active+clean","last_fresh":"2020-09-08T15:41:13.597429+0000","last_change":"2020-09-08T15:41:03.288848+0000","last_active":"2020-09-08T15:41:13.597429+0000","last_peered":"2020-09-08T15:41:13.597429+0000","last_clean":"2020-09-08T15:41:13.597429+0000","last_became_active":"2020-09-08T15:37:29.881507+0000","last_became_peered":"2020-09-08T15:37:29.881507+0000","last_unstale":"2020-09-08T15:41:13.597429+0000","last_undegraded":"2020-09-08T15:41:13.597429+0000","last_fullsized":"2020-09-08T15:41:13.597429+0000","mapping_epoch":126,"log_start":"148'6400","ondisk_log_start":"148'6400","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'15829","last_scrub_stamp":"2020-09-08T15:41:03.288750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:41:03.288750+0000","log_size":10008,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":124911616,"num_objects":1906,"num_object_clones":0,"num_object_copies":3812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16408,"num_write_kb":586048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9],"acting":[7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.9","version":"200'16263","reported_seq":"16348","reported_epoch":"200","state":"active+clean","last_fresh":"2020-09-08T15:41:11.800303+0000","last_change":"2020-09-08T15:40:31.090354+0000","last_active":"2020-09-08T15:41:11.800303+0000","last_peered":"2020-09-08T15:41:11.800303+0000","last_clean":"2020-09-08T15:41:11.800303+0000","last_became_active":"2020-09-08T15:37:29.879357+0000","last_became_peered":"2020-09-08T15:37:29.879357+0000","last_unstale":"2020-09-08T15:41:11.800303+0000","last_undegraded":"2020-09-08T15:41:11.800303+0000","last_fullsized":"2020-09-08T15:41:11.800303+0000","mapping_epoch":126,"log_start":"147'6200","ondisk_log_start":"147'6200","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'13830","last_scrub_stamp":"2020-09-08T15:40:31.090293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:31.090293+0000","log_size":10063,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":135331840,"num_objects":2065,"num_object_clones":0,"num_object_copies":4130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16263,"num_write_kb":586496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"201'16536","reported_seq":"16624","reported_epoch":"201","state":"active+clean","last_fresh":"2020-09-08T15:41:12.774900+0000","last_change":"2020-09-08T15:40:30.575755+0000","last_active":"2020-09-08T15:41:12.774900+0000","last_peered":"2020-09-08T15:41:12.774900+0000","last_clean":"2020-09-08T15:41:12.774900+0000","last_became_active":"2020-09-08T15:37:30.094030+0000","last_became_peered":"2020-09-08T15:37:30.094030+0000","last_unstale":"2020-09-08T15:41:12.774900+0000","last_undegraded":"2020-09-08T15:41:12.774900+0000","last_fullsized":"2020-09-08T15:41:12.774900+0000","mapping_epoch":126,"log_start":"148'6500","ondisk_log_start":"148'6500","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'13996","last_scrub_stamp":"2020-09-08T15:40:30.575701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:30.575701+0000","log_size":10036,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128974848,"num_objects":1968,"num_object_clones":0,"num_object_copies":3936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16536,"num_write_kb":592128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3],"acting":[11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.b","version":"200'16433","reported_seq":"16520","reported_epoch":"200","state":"active+clean","last_fresh":"2020-09-08T15:41:11.785245+0000","last_change":"2020-09-08T15:41:11.741706+0000","last_active":"2020-09-08T15:41:11.785245+0000","last_peered":"2020-09-08T15:41:11.785245+0000","last_clean":"2020-09-08T15:41:11.785245+0000","last_became_active":"2020-09-08T15:37:29.879505+0000","last_became_peered":"2020-09-08T15:37:29.879505+0000","last_unstale":"2020-09-08T15:41:11.785245+0000","last_undegraded":"2020-09-08T15:41:11.785245+0000","last_fullsized":"2020-09-08T15:41:11.785245+0000","mapping_epoch":126,"log_start":"148'6400","ondisk_log_start":"148'6400","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'16428","last_scrub_stamp":"2020-09-08T15:41:11.741633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:41:11.741633+0000","log_size":10033,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":133627904,"num_objects":2039,"num_object_clones":0,"num_object_copies":4078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16433,"num_write_kb":591104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.c","version":"201'16334","reported_seq":"16422","reported_epoch":"201","state":"active+clean","last_fresh":"2020-09-08T15:41:12.752509+0000","last_change":"2020-09-08T15:40:15.718317+0000","last_active":"2020-09-08T15:41:12.752509+0000","last_peered":"2020-09-08T15:41:12.752509+0000","last_clean":"2020-09-08T15:41:12.752509+0000","last_became_active":"2020-09-08T15:37:30.093315+0000","last_became_peered":"2020-09-08T15:37:30.093315+0000","last_unstale":"2020-09-08T15:41:12.752509+0000","last_undegraded":"2020-09-08T15:41:12.752509+0000","last_fullsized":"2020-09-08T15:41:12.752509+0000","mapping_epoch":126,"log_start":"147'6300","ondisk_log_start":"147'6300","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'12953","last_scrub_stamp":"2020-09-08T15:40:15.718177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:15.718177+0000","log_size":10034,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":129236992,"num_objects":1972,"num_object_clones":0,"num_object_copies":3944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16334,"num_write_kb":585792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0],"acting":[11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.d","version":"201'16551","reported_seq":"16640","reported_epoch":"201","state":"active+clean","last_fresh":"2020-09-08T15:41:12.860556+0000","last_change":"2020-09-08T15:41:12.199062+0000","last_active":"2020-09-08T15:41:12.860556+0000","last_peered":"2020-09-08T15:41:12.860556+0000","last_clean":"2020-09-08T15:41:12.860556+0000","last_became_active":"2020-09-08T15:37:30.092910+0000","last_became_peered":"2020-09-08T15:37:30.092910+0000","last_unstale":"2020-09-08T15:41:12.860556+0000","last_undegraded":"2020-09-08T15:41:12.860556+0000","last_fullsized":"2020-09-08T15:41:12.860556+0000","mapping_epoch":126,"log_start":"149'6500","ondisk_log_start":"149'6500","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'16508","last_scrub_stamp":"2020-09-08T15:41:12.199008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:41:12.199008+0000","log_size":10051,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":136708120,"num_objects":2087,"num_object_clones":0,"num_object_copies":4174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2087,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16551,"num_write_kb":596353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.e","version":"200'16762","reported_seq":"16851","reported_epoch":"200","state":"active+clean","last_fresh":"2020-09-08T15:41:11.834555+0000","last_change":"2020-09-08T15:41:07.596185+0000","last_active":"2020-09-08T15:41:11.834555+0000","last_peered":"2020-09-08T15:41:11.834555+0000","last_clean":"2020-09-08T15:41:11.834555+0000","last_became_active":"2020-09-08T15:37:30.093099+0000","last_became_peered":"2020-09-08T15:37:30.093099+0000","last_unstale":"2020-09-08T15:41:11.834555+0000","last_undegraded":"2020-09-08T15:41:11.834555+0000","last_fullsized":"2020-09-08T15:41:11.834555+0000","mapping_epoch":126,"log_start":"149'6700","ondisk_log_start":"149'6700","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'16496","last_scrub_stamp":"2020-09-08T15:41:07.596111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:41:07.596111+0000","log_size":10062,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":131596288,"num_objects":2008,"num_object_clones":0,"num_object_copies":4016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16762,"num_write_kb":600640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":2103836696,"num_objects":32103,"num_object_clones":0,"num_object_copies":64206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32103,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263133,"num_write_kb":9447489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4228849664,"data_stored":4228841520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160733,"ondisk_log_size":160733,"up":32,"acting":32,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":2,"acting":2,"num_store_stats":7}],"osd_stats":[{"osd":11,"up_from":0,"seq":82,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":444268,"kb_used_data":381156,"kb_used_omap":0,"kb_used_meta":63104,"kb_avail":93300884,"statfs":{"total":95995035648,"available":95540105216,"internally_reserved":0,"allocated":390303744,"data_stored":390026188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64618496},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":2.9119999999999999,"5min":2.2650000000000001,"15min":1.988},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":33.677999999999997,"5min":33.677999999999997,"15min":33.677999999999997},"last":0.87},{"interface":"front","average":{"1min":3.6440000000000001,"5min":3.0649999999999999,"15min":3.2290000000000001},"min":{"1min":1.1930000000000001,"5min":1.022,"15min":1.022},"max":{"1min":34.058,"5min":34.058,"15min":34.058},"last":1.248}]},{"osd":1,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":10.811,"5min":4.2930000000000001,"15min":2.8039999999999998},"min":{"1min":0.78800000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":163.51499999999999,"5min":163.51499999999999,"15min":163.51499999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":11.222,"5min":4.7359999999999998,"15min":3.3399999999999999},"min":{"1min":1.2789999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":166.94300000000001,"5min":166.94300000000001,"15min":166.94300000000001},"last":2.7690000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":3.5569999999999999,"5min":3.1539999999999999,"15min":2.4390000000000001},"min":{"1min":0.84599999999999997,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":19.966999999999999,"5min":36.637999999999998,"15min":36.637999999999998},"last":3.8500000000000001},{"interface":"front","average":{"1min":3.3780000000000001,"5min":3.464,"15min":3.0659999999999998},"min":{"1min":1.0309999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":20.390000000000001,"5min":36.694000000000003,"15min":36.694000000000003},"last":5.9370000000000003}]},{"osd":3,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":2.2989999999999999,"5min":4.0540000000000003,"15min":2.9039999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":4.6520000000000001,"5min":147.49100000000001,"15min":147.49100000000001},"last":1.155},{"interface":"front","average":{"1min":2.5,"5min":4.4370000000000003,"15min":3.5699999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":5.3460000000000001,"5min":150.32400000000001,"15min":150.32400000000001},"last":3.2109999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":2.2610000000000001,"5min":2.8679999999999999,"15min":3.3620000000000001},"min":{"1min":0.95299999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":4.9009999999999998,"5min":8.0289999999999999,"15min":8.0289999999999999},"last":1.452},{"interface":"front","average":{"1min":2.6739999999999999,"5min":2.738,"15min":2.7829999999999999},"min":{"1min":0.92300000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":7.6609999999999996,"5min":7.7679999999999998,"15min":7.7679999999999998},"last":0.94399999999999995}]},{"osd":5,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":2.5870000000000002,"5min":2.8980000000000001,"15min":3.2559999999999998},"min":{"1min":1.1699999999999999,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":5.2370000000000001,"5min":6.8499999999999996,"15min":6.8499999999999996},"last":1.3680000000000001},{"interface":"front","average":{"1min":2.968,"5min":3.198,"15min":3.7320000000000002},"min":{"1min":1.262,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":6.6079999999999997,"5min":8.5289999999999999,"15min":8.5289999999999999},"last":1.7010000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":2.9300000000000002,"5min":2.9700000000000002,"15min":3.2799999999999998},"min":{"1min":1.167,"5min":1.036,"15min":1.036},"max":{"1min":5.8109999999999999,"5min":7.8479999999999999,"15min":7.8479999999999999},"last":1.478},{"interface":"front","average":{"1min":2.6579999999999999,"5min":3.2400000000000002,"15min":3.4940000000000002},"min":{"1min":1.125,"5min":1.125,"15min":1.125},"max":{"1min":5.7800000000000002,"5min":8.2110000000000003,"15min":8.2110000000000003},"last":2.488}]},{"osd":7,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":3.0339999999999998,"5min":3.351,"15min":3.6230000000000002},"min":{"1min":1.2749999999999999,"5min":1.22,"15min":1.22},"max":{"1min":8.3100000000000005,"5min":8.4610000000000003,"15min":8.4610000000000003},"last":1.131},{"interface":"front","average":{"1min":3.5350000000000001,"5min":3.4660000000000002,"15min":3.7490000000000001},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":9.1820000000000004,"5min":9.1820000000000004,"15min":9.1820000000000004},"last":1.718}]},{"osd":8,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":4.2619999999999996,"5min":3.7599999999999998,"15min":4.0119999999999996},"min":{"1min":1.246,"5min":1.246,"15min":1.246},"max":{"1min":13.018000000000001,"5min":13.018000000000001,"15min":13.018000000000001},"last":2.9910000000000001},{"interface":"front","average":{"1min":4.5970000000000004,"5min":3.8700000000000001,"15min":3.8940000000000001},"min":{"1min":1.5069999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":13.592000000000001,"5min":13.592000000000001,"15min":13.592000000000001},"last":3.1019999999999999}]},{"osd":10,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":3.355,"5min":3.6110000000000002,"15min":4.202},"min":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"max":{"1min":9.7439999999999998,"5min":9.7439999999999998,"15min":9.7439999999999998},"last":2.641},{"interface":"front","average":{"1min":2.9950000000000001,"5min":3.5670000000000002,"15min":4.0490000000000004},"min":{"1min":1.085,"5min":1.085,"15min":1.085},"max":{"1min":8.4260000000000002,"5min":8.6600000000000001,"15min":8.6600000000000001},"last":1.4239999999999999}]}]},{"osd":10,"up_from":0,"seq":89,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314848,"kb_used_data":262488,"kb_used_omap":0,"kb_used_meta":52352,"kb_avail":93430304,"statfs":{"total":95995035648,"available":95672631296,"internally_reserved":0,"allocated":268787712,"data_stored":268514564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":53608448},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7330000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":5.9669999999999996,"5min":2.8799999999999999,"15min":2.5659999999999998},"min":{"1min":0.66900000000000004,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":64.471999999999994,"5min":64.471999999999994,"15min":64.471999999999994},"last":1.343},{"interface":"front","average":{"1min":6.6349999999999998,"5min":3.3250000000000002,"15min":2.8130000000000002},"min":{"1min":0.70099999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":64.707999999999998,"5min":64.707999999999998,"15min":64.707999999999998},"last":1.911}]},{"osd":2,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":4.0949999999999998,"5min":2.593,"15min":2.363},"min":{"1min":0.80800000000000005,"5min":0.745,"15min":0.745},"max":{"1min":40.453000000000003,"5min":40.453000000000003,"15min":40.453000000000003},"last":2.3439999999999999},{"interface":"front","average":{"1min":3.7730000000000001,"5min":2.4500000000000002,"15min":2.6970000000000001},"min":{"1min":0.88900000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":37.209000000000003,"5min":37.209000000000003,"15min":37.209000000000003},"last":1.6930000000000001}]},{"osd":3,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":3.7250000000000001,"5min":4.5549999999999997,"15min":3.04},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":37.298999999999999,"5min":232.87299999999999,"15min":232.87299999999999},"last":1.95},{"interface":"front","average":{"1min":4.1539999999999999,"5min":5.1710000000000003,"15min":3.7829999999999999},"min":{"1min":1.0760000000000001,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":36.137,"5min":237.75800000000001,"15min":237.75800000000001},"last":1.8600000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":3.5009999999999999,"5min":2.859,"15min":3.2650000000000001},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":38.237000000000002,"5min":38.237000000000002,"15min":38.237000000000002},"last":2.5},{"interface":"front","average":{"1min":3.6179999999999999,"5min":2.855,"15min":3.0089999999999999},"min":{"1min":1.0009999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":37.786000000000001,"5min":37.786000000000001,"15min":37.786000000000001},"last":2.8029999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":3.8879999999999999,"5min":3.0750000000000002,"15min":3.448},"min":{"1min":0.996,"5min":0.996,"15min":0.996},"max":{"1min":36.493000000000002,"5min":36.493000000000002,"15min":36.493000000000002},"last":2.431},{"interface":"front","average":{"1min":3.859,"5min":3.25,"15min":3.657},"min":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"max":{"1min":38.957999999999998,"5min":38.957999999999998,"15min":38.957999999999998},"last":3.048}]},{"osd":6,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":3.8410000000000002,"5min":3.1150000000000002,"15min":3.4289999999999998},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":38.537999999999997,"5min":38.537999999999997,"15min":38.537999999999997},"last":3.3279999999999998},{"interface":"front","average":{"1min":4.0890000000000004,"5min":3.3140000000000001,"15min":3.71},"min":{"1min":1.266,"5min":1.1950000000000001,"15min":1.1950000000000001},"max":{"1min":37.698999999999998,"5min":37.698999999999998,"15min":37.698999999999998},"last":2.6389999999999998}]},{"osd":7,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":4.319,"5min":3.4369999999999998,"15min":3.778},"min":{"1min":1.2410000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"max":{"1min":39.100000000000001,"5min":39.100000000000001,"15min":39.100000000000001},"last":3.089},{"interface":"front","average":{"1min":4.532,"5min":3.5470000000000002,"15min":3.9289999999999998},"min":{"1min":1.3440000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"max":{"1min":38.131999999999998,"5min":38.131999999999998,"15min":38.131999999999998},"last":2.2280000000000002}]},{"osd":8,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":4.4509999999999996,"5min":3.7679999999999998,"15min":4.0419999999999998},"min":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"max":{"1min":42.031999999999996,"5min":42.031999999999996,"15min":42.031999999999996},"last":3.8439999999999999},{"interface":"front","average":{"1min":4.4669999999999996,"5min":3.7850000000000001,"15min":3.9430000000000001},"min":{"1min":1.1850000000000001,"5min":1.1639999999999999,"15min":1.1639999999999999},"max":{"1min":39.512999999999998,"5min":39.512999999999998,"15min":39.512999999999998},"last":4.048}]},{"osd":9,"last update":"Tue Sep 8 15:40:32 2020","interfaces":[{"interface":"back","average":{"1min":4.7549999999999999,"5min":3.9790000000000001,"15min":4.2720000000000002},"min":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":42.603999999999999,"5min":42.603999999999999,"15min":42.603999999999999},"last":2.6989999999999998},{"interface":"front","average":{"1min":4.5919999999999996,"5min":3.8650000000000002,"15min":4.0899999999999999},"min":{"1min":1.5189999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"max":{"1min":40.029000000000003,"5min":40.029000000000003,"15min":40.029000000000003},"last":3.9510000000000001}]},{"osd":11,"last update":"Tue Sep 8 15:41:02 2020","interfaces":[{"interface":"back","average":{"1min":2.956,"5min":4.1040000000000001,"15min":4.2939999999999996},"min":{"1min":1.3520000000000001,"5min":1.3009999999999999,"15min":1.3009999999999999},"max":{"1min":8.9510000000000005,"5min":39.271000000000001,"15min":39.271000000000001},"last":4.1310000000000002},{"interface":"front","average":{"1min":2.7559999999999998,"5min":3.8170000000000002,"15min":4.2210000000000001},"min":{"1min":1.385,"5min":1.3779999999999999,"15min":1.3779999999999999},"max":{"1min":5.4930000000000003,"5min":42.329999999999998,"15min":42.329999999999998},"last":2.8769999999999998}]}]},{"osd":9,"up_from":0,"seq":92,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":444964,"kb_used_data":385884,"kb_used_omap":0,"kb_used_meta":58944,"kb_avail":93300188,"statfs":{"total":95995035648,"available":95539392512,"internally_reserved":0,"allocated":395145216,"data_stored":394867996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":60358656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:41:09 2020","interfaces":[{"interface":"back","average":{"1min":11.460000000000001,"5min":3.9140000000000001,"15min":2.5609999999999999},"min":{"1min":0.746,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":180.84899999999999,"5min":180.84899999999999,"15min":180.84899999999999},"last":4.3399999999999999},{"interface":"front","average":{"1min":11.324,"5min":3.7530000000000001,"15min":2.5230000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":175.077,"5min":175.077,"15min":175.077},"last":0.95699999999999996}]},{"osd":1,"last update":"Tue Sep 8 15:41:09 2020","interfaces":[{"interface":"back","average":{"1min":2.0880000000000001,"5min":2.133,"15min":2.024},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":8.8249999999999993,"5min":9.5519999999999996,"15min":9.5519999999999996},"last":1.054},{"interface":"front","average":{"1min":2.6219999999999999,"5min":2.4529999999999998,"15min":2.3540000000000001},"min":{"1min":0.94499999999999995,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":7.5439999999999996,"5min":9.4250000000000007,"15min":9.4250000000000007},"last":1.2749999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:41:09 2020","interfaces":[{"interface":"back","average":{"1min":1.968,"5min":2.2069999999999999,"15min":2.3999999999999999},"min":{"1min":1.113,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":6.5830000000000002,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":1.605},{"interface":"front","average":{"1min":1.8740000000000001,"5min":2.306,"15min":2.306},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":4.9530000000000003,"5min":9.4770000000000003,"15min":9.4770000000000003},"last":4.1050000000000004}]},{"osd":3,"last update":"Tue Sep 8 15:41:11 2020","interfaces":[{"interface":"back","average":{"1min":1.8740000000000001,"5min":2.5750000000000002,"15min":2.4220000000000002},"min":{"1min":0.97799999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":5.3079999999999998,"5min":24.562000000000001,"15min":24.562000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":2.0819999999999999,"5min":2.7690000000000001,"15min":2.6840000000000002},"min":{"1min":1.004,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":4.9029999999999996,"5min":23.184000000000001,"15min":23.184000000000001},"last":1.3859999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:41:09 2020","interfaces":[{"interface":"back","average":{"1min":2.5150000000000001,"5min":2.6309999999999998,"15min":2.6909999999999998},"min":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"max":{"1min":6.8129999999999997,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.722},{"interface":"front","average":{"1min":2.266,"5min":2.722,"15min":2.9409999999999998},"min":{"1min":1.3340000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":4.383,"5min":10.201000000000001,"15min":10.201000000000001},"last":1.6859999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:41:09 2020","interfaces":[{"interface":"back","average":{"1min":2.2610000000000001,"5min":2.7069999999999999,"15min":3.0259999999999998},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":5.0999999999999996,"5min":11.097,"15min":11.097},"last":9.5869999999999997},{"interface":"front","average":{"1min":2.1699999999999999,"5min":2.8399999999999999,"15min":3.077},"min":{"1min":1.22,"5min":1.0860000000000001,"15min":1.0589999999999999},"max":{"1min":3.4580000000000002,"5min":9.9280000000000008,"15min":9.9280000000000008},"last":2.2599999999999998}]},{"osd":6,"last update":"Tue Sep 8 15:41:09 2020","interfaces":[{"interface":"back","average":{"1min":2.266,"5min":2.8839999999999999,"15min":3.1320000000000001},"min":{"1min":1.1339999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":5.6459999999999999,"5min":10.798,"15min":10.798},"last":4.5890000000000004},{"interface":"front","average":{"1min":2.625,"5min":3.0110000000000001,"15min":3.1829999999999998},"min":{"1min":1.4059999999999999,"5min":1.242,"15min":1.242},"max":{"1min":6.7389999999999999,"5min":11.766,"15min":11.766},"last":4.7450000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:41:09 2020","interfaces":[{"interface":"back","average":{"1min":2.5299999999999998,"5min":3.1629999999999998,"15min":3.351},"min":{"1min":1.458,"5min":1.2110000000000001,"15min":1.2110000000000001},"max":{"1min":5.907,"5min":10.31,"15min":10.31},"last":2.3809999999999998},{"interface":"front","average":{"1min":2.7789999999999999,"5min":3.3959999999999999,"15min":3.5680000000000001},"min":{"1min":1.669,"5min":1.593,"15min":1.593},"max":{"1min":6.016,"5min":10.837999999999999,"15min":10.837999999999999},"last":2.4940000000000002}]},{"osd":8,"last update":"Tue Sep 8 15:41:09 2020","interfaces":[{"interface":"back","average":{"1min":2.8039999999999998,"5min":3.3599999999999999,"15min":3.2669999999999999},"min":{"1min":1.264,"5min":1.264,"15min":1.264},"max":{"1min":6.4409999999999998,"5min":11,"15min":11},"last":1.944},{"interface":"front","average":{"1min":2.552,"5min":3.4540000000000002,"15min":3.3620000000000001},"min":{"1min":1.262,"5min":1.262,"15min":1.262},"max":{"1min":4.0090000000000003,"5min":10.977,"15min":10.977},"last":2.6190000000000002}]},{"osd":10,"last update":"Tue Sep 8 15:40:24 2020","interfaces":[{"interface":"back","average":{"1min":2.8879999999999999,"5min":3.3780000000000001,"15min":3.7469999999999999},"min":{"1min":1.5329999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"max":{"1min":8.8200000000000003,"5min":8.8200000000000003,"15min":8.8200000000000003},"last":1.841},{"interface":"front","average":{"1min":2.0790000000000002,"5min":3.3620000000000001,"15min":3.6459999999999999},"min":{"1min":1.196,"5min":1.196,"15min":1.196},"max":{"1min":3.282,"5min":11.625,"15min":11.625},"last":4.4669999999999996}]}]},{"osd":8,"up_from":0,"seq":97,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":602848,"kb_used_data":529176,"kb_used_omap":0,"kb_used_meta":73600,"kb_avail":93142304,"statfs":{"total":95995035648,"available":95377719296,"internally_reserved":0,"allocated":541876224,"data_stored":541603076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":75366400},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":2.0009999999999999,"5min":2.141,"15min":2.1709999999999998},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":8.6069999999999993,"5min":8.6069999999999993,"15min":8.6069999999999993},"last":1.296},{"interface":"front","average":{"1min":3.5049999999999999,"5min":2.472,"15min":2.4460000000000002},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":13.773999999999999,"5min":13.773999999999999,"15min":13.773999999999999},"last":1.5600000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.492,"5min":2.0659999999999998,"15min":1.8839999999999999},"min":{"1min":1.4690000000000001,"5min":1.137,"15min":1.137},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":1.4350000000000001},{"interface":"front","average":{"1min":2.7200000000000002,"5min":2.427,"15min":2.347},"min":{"1min":1.351,"5min":1.046,"15min":1.046},"max":{"1min":4.444,"5min":4.444,"15min":4.444},"last":1.8600000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":10.316000000000001,"5min":4.3819999999999997,"15min":2.677},"min":{"1min":0.79500000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":145.893,"5min":145.893,"15min":145.893},"last":4.7930000000000001},{"interface":"front","average":{"1min":9.9939999999999998,"5min":3.9420000000000002,"15min":2.9580000000000002},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":153.91900000000001,"5min":153.91900000000001,"15min":153.91900000000001},"last":1.9430000000000001}]},{"osd":3,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":2.5739999999999998,"5min":4.0869999999999997,"15min":3.1779999999999999},"min":{"1min":0.73799999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":9.2989999999999995,"5min":125.111,"15min":125.111},"last":2.3450000000000002},{"interface":"front","average":{"1min":3.1190000000000002,"5min":3.952,"15min":2.9060000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":13.505000000000001,"5min":126.124,"15min":126.124},"last":4.2880000000000003}]},{"osd":4,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":2.5329999999999999,"5min":2.9830000000000001,"15min":2.907},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":9.3800000000000008,"5min":9.7599999999999998,"15min":9.7599999999999998},"last":2.7090000000000001},{"interface":"front","average":{"1min":3.343,"5min":2.7320000000000002,"15min":2.6110000000000002},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":13.999000000000001,"5min":13.999000000000001,"15min":13.999000000000001},"last":1.71}]},{"osd":5,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":2.6720000000000002,"5min":3.2690000000000001,"15min":3.1890000000000001},"min":{"1min":0.90500000000000003,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":9.4399999999999995,"5min":9.4399999999999995,"15min":9.4399999999999995},"last":7.0730000000000004},{"interface":"front","average":{"1min":2.6600000000000001,"5min":3.323,"15min":3.3380000000000001},"min":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":9.6820000000000004,"5min":9.6820000000000004,"15min":9.6820000000000004},"last":2.5510000000000002}]},{"osd":6,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":3.343,"5min":3.552,"15min":3.3450000000000002},"min":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"max":{"1min":13.887,"5min":13.887,"15min":13.887},"last":3.016},{"interface":"front","average":{"1min":2.9550000000000001,"5min":2.9470000000000001,"15min":2.923},"min":{"1min":1.153,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":12.502000000000001,"5min":12.502000000000001,"15min":12.502000000000001},"last":2.9780000000000002}]},{"osd":7,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":3.3119999999999998,"5min":3.7360000000000002,"15min":3.6469999999999998},"min":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"max":{"1min":9.4469999999999992,"5min":10.236000000000001,"15min":10.236000000000001},"last":2.847},{"interface":"front","average":{"1min":3.948,"5min":3.4289999999999998,"15min":3.2959999999999998},"min":{"1min":1.3620000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"max":{"1min":14.109999999999999,"5min":14.109999999999999,"15min":14.109999999999999},"last":3.161}]},{"osd":9,"last update":"Tue Sep 8 15:41:07 2020","interfaces":[{"interface":"back","average":{"1min":3.1499999999999999,"5min":3.8650000000000002,"15min":3.5819999999999999},"min":{"1min":1.095,"5min":1.095,"15min":1.095},"max":{"1min":11.913,"5min":13.506,"15min":13.506},"last":6.9320000000000004},{"interface":"front","average":{"1min":3.883,"5min":3.8290000000000002,"15min":3.5289999999999999},"min":{"1min":1.004,"5min":1.004,"15min":1.004},"max":{"1min":12.052,"5min":12.052,"15min":12.052},"last":4.9020000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":3.6379999999999999,"5min":3.3490000000000002,"15min":3.6070000000000002},"min":{"1min":1.423,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":13.656000000000001,"5min":13.656000000000001,"15min":13.656000000000001},"last":5.7859999999999996},{"interface":"front","average":{"1min":2.9729999999999999,"5min":3.8140000000000001,"15min":3.988},"min":{"1min":1.296,"5min":1.296,"15min":1.296},"max":{"1min":9.6649999999999991,"5min":11.051,"15min":11.051},"last":6.7990000000000004}]},{"osd":11,"last update":"Tue Sep 8 15:40:35 2020","interfaces":[{"interface":"back","average":{"1min":3.512,"5min":3.665,"15min":3.9239999999999999},"min":{"1min":1.214,"5min":1.214,"15min":1.214},"max":{"1min":12.646000000000001,"5min":12.646000000000001,"15min":12.646000000000001},"last":6.4489999999999998},{"interface":"front","average":{"1min":3.4830000000000001,"5min":4.4210000000000003,"15min":4.7939999999999996},"min":{"1min":1.248,"5min":1.248,"15min":1.248},"max":{"1min":10.667999999999999,"5min":11.186999999999999,"15min":11.186999999999999},"last":7.3419999999999996}]}]},{"osd":7,"up_from":0,"seq":103,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":431596,"kb_used_data":376420,"kb_used_omap":0,"kb_used_meta":55168,"kb_avail":93313556,"statfs":{"total":95995035648,"available":95553081344,"internally_reserved":0,"allocated":385454080,"data_stored":385176544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":56492032},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:02 2020","interfaces":[{"interface":"back","average":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"min":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"max":{"1min":4.0190000000000001,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":2.2189999999999999},{"interface":"front","average":{"1min":2.6259999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"min":{"1min":1.542,"5min":1.542,"15min":1.542},"max":{"1min":4.5229999999999997,"5min":4.5229999999999997,"15min":4.5229999999999997},"last":2.3079999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":2.133,"15min":2.2759999999999998},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":4.8600000000000003,"5min":6.6219999999999999,"15min":6.6219999999999999},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.802,"5min":2.1030000000000002,"15min":2.2869999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":5.3159999999999998,"5min":5.3159999999999998,"15min":5.3159999999999998},"last":1.867}]},{"osd":2,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":2.3879999999999999,"5min":2.5350000000000001,"15min":2.4860000000000002},"min":{"1min":1.0649999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":6.234,"5min":6.9779999999999998,"15min":6.9779999999999998},"last":1.774},{"interface":"front","average":{"1min":2.569,"5min":2.4870000000000001,"15min":2.7509999999999999},"min":{"1min":1.335,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":6.6879999999999997,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":1.5009999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":2.3109999999999999,"5min":2.899,"15min":2.9129999999999998},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":4.593,"5min":37.801000000000002,"15min":37.801000000000002},"last":7.1399999999999997},{"interface":"front","average":{"1min":2.661,"5min":3.6030000000000002,"15min":3.387},"min":{"1min":1.2430000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":12.099,"5min":45.284999999999997,"15min":45.284999999999997},"last":1.946}]},{"osd":4,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":1.794,"5min":2.5089999999999999,"15min":2.7989999999999999},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":3.1930000000000001,"5min":8.5779999999999994,"15min":8.5779999999999994},"last":1.897},{"interface":"front","average":{"1min":2.2320000000000002,"5min":3.004,"15min":3.2229999999999999},"min":{"1min":1.246,"5min":1.1319999999999999,"15min":1.1319999999999999},"max":{"1min":4.4009999999999998,"5min":7.3570000000000002,"15min":7.3570000000000002},"last":2.0910000000000002}]},{"osd":5,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":2.3149999999999999,"5min":2.956,"15min":3.464},"min":{"1min":1.228,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":4.5540000000000003,"5min":9.0800000000000001,"15min":9.0800000000000001},"last":2.048},{"interface":"front","average":{"1min":2.3159999999999998,"5min":2.9449999999999998,"15min":3.339},"min":{"1min":1.208,"5min":1.1779999999999999,"15min":1.1779999999999999},"max":{"1min":4.2869999999999999,"5min":6.0229999999999997,"15min":6.0229999999999997},"last":2.1299999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":2.5219999999999998,"5min":3.1749999999999998,"15min":3.7080000000000002},"min":{"1min":1.304,"5min":1.304,"15min":1.304},"max":{"1min":5.1210000000000004,"5min":8.8300000000000001,"15min":8.8300000000000001},"last":2.1859999999999999},{"interface":"front","average":{"1min":2.6259999999999999,"5min":3.359,"15min":3.7450000000000001},"min":{"1min":1.7,"5min":1.3939999999999999,"15min":1.3939999999999999},"max":{"1min":4.2910000000000004,"5min":7.681,"15min":7.681},"last":2.3519999999999999}]},{"osd":8,"last update":"Tue Sep 8 15:40:43 2020","interfaces":[{"interface":"back","average":{"1min":3.3420000000000001,"5min":3.5739999999999998,"15min":3.9729999999999999},"min":{"1min":1.5009999999999999,"5min":1.3340000000000001,"15min":1.3340000000000001},"max":{"1min":8.2260000000000009,"5min":8.2260000000000009,"15min":8.2260000000000009},"last":2.0129999999999999},{"interface":"front","average":{"1min":3.1840000000000002,"5min":3.665,"15min":4.0830000000000002},"min":{"1min":1.677,"5min":1.6319999999999999,"15min":1.6319999999999999},"max":{"1min":8.4220000000000006,"5min":9.2119999999999997,"15min":9.2119999999999997},"last":2.6709999999999998}]},{"osd":9,"last update":"Tue Sep 8 15:41:02 2020","interfaces":[{"interface":"back","average":{"1min":3.589,"5min":3.8210000000000002,"15min":4.4610000000000003},"min":{"1min":2.1339999999999999,"5min":1.7170000000000001,"15min":1.7170000000000001},"max":{"1min":6.9800000000000004,"5min":7.476,"15min":7.476},"last":2.2280000000000002},{"interface":"front","average":{"1min":3.617,"5min":3.4009999999999998,"15min":4.093},"min":{"1min":2.0459999999999998,"5min":1.4570000000000001,"15min":1.4570000000000001},"max":{"1min":7.7460000000000004,"5min":7.7460000000000004,"15min":7.7460000000000004},"last":2.25}]},{"osd":10,"last update":"Tue Sep 8 15:40:31 2020","interfaces":[{"interface":"back","average":{"1min":3.2040000000000002,"5min":3.7370000000000001,"15min":4.4119999999999999},"min":{"1min":2.0299999999999998,"5min":1.5309999999999999,"15min":1.5309999999999999},"max":{"1min":7.3739999999999997,"5min":9.5009999999999994,"15min":9.5009999999999994},"last":2.641},{"interface":"front","average":{"1min":2.7810000000000001,"5min":3.8860000000000001,"15min":4.75},"min":{"1min":1.583,"5min":1.583,"15min":1.583},"max":{"1min":4.8079999999999998,"5min":7.5720000000000001,"15min":8.3650000000000002},"last":2.762}]},{"osd":11,"last update":"Tue Sep 8 15:40:50 2020","interfaces":[{"interface":"back","average":{"1min":3.1509999999999998,"5min":3.9089999999999998,"15min":4.5279999999999996},"min":{"1min":1.8109999999999999,"5min":1.7010000000000001,"15min":1.7010000000000001},"max":{"1min":7.0410000000000004,"5min":9.1790000000000003,"15min":9.1790000000000003},"last":2.8319999999999999},{"interface":"front","average":{"1min":3.9249999999999998,"5min":4.2869999999999999,"15min":4.9729999999999999},"min":{"1min":2.1819999999999999,"5min":1.756,"15min":1.756},"max":{"1min":7.181,"5min":7.7629999999999999,"15min":7.7629999999999999},"last":2.5110000000000001}]}]},{"osd":6,"up_from":0,"seq":107,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":154592,"kb_used_data":133336,"kb_used_omap":0,"kb_used_meta":21248,"kb_avail":93590560,"statfs":{"total":95995035648,"available":95836733440,"internally_reserved":0,"allocated":136536064,"data_stored":136262936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21757952},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.6799999999999999,"15min":1.851},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":3.399,"5min":3.8809999999999998,"15min":3.8809999999999998},"last":1.012},{"interface":"front","average":{"1min":1.581,"5min":1.853,"15min":1.9430000000000001},"min":{"1min":0.89900000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.8570000000000002,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":1.7130000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":10.734,"5min":3.9239999999999999,"15min":2.7599999999999998},"min":{"1min":0.86699999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":149.34200000000001,"5min":149.34200000000001,"15min":149.34200000000001},"last":2.0609999999999999},{"interface":"front","average":{"1min":10.590999999999999,"5min":4.1479999999999997,"15min":3.0299999999999998},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":149.03899999999999,"5min":149.03899999999999,"15min":149.03899999999999},"last":4.952}]},{"osd":2,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":32.973999999999997,"5min":8.6989999999999998,"15min":4.5410000000000004},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":549.75999999999999,"5min":549.75999999999999,"15min":549.75999999999999},"last":5.1379999999999999},{"interface":"front","average":{"1min":33.076000000000001,"5min":8.7970000000000006,"15min":4.5519999999999996},"min":{"1min":1.4379999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":549.85900000000004,"5min":549.85900000000004,"15min":549.85900000000004},"last":1.508}]},{"osd":3,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":2.2570000000000001,"5min":3.6539999999999999,"15min":2.8809999999999998},"min":{"1min":1.2250000000000001,"5min":1.224,"15min":1.224},"max":{"1min":3.8100000000000001,"5min":109.65300000000001,"15min":109.65300000000001},"last":1.921},{"interface":"front","average":{"1min":2.0390000000000001,"5min":3.7959999999999998,"15min":3.2120000000000002},"min":{"1min":1.24,"5min":1.24,"15min":1.24},"max":{"1min":4.4870000000000001,"5min":109.51600000000001,"15min":109.51600000000001},"last":5.5540000000000003}]},{"osd":4,"last update":"Tue Sep 8 15:40:58 2020","interfaces":[{"interface":"back","average":{"1min":2.1480000000000001,"5min":2.6779999999999999,"15min":2.7109999999999999},"min":{"1min":0.92700000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":4.0129999999999999,"5min":4.9980000000000002,"15min":4.9980000000000002},"last":5.048},{"interface":"front","average":{"1min":2.351,"5min":2.9590000000000001,"15min":2.9489999999999998},"min":{"1min":1.1910000000000001,"5min":1.052,"15min":1.052},"max":{"1min":3.9220000000000002,"5min":22.062000000000001,"15min":22.062000000000001},"last":5.1749999999999998}]},{"osd":5,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":2.778,"5min":3.1110000000000002,"15min":3.1640000000000001},"min":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"max":{"1min":7.1710000000000003,"5min":7.1710000000000003,"15min":7.1710000000000003},"last":1.7669999999999999},{"interface":"front","average":{"1min":2.6509999999999998,"5min":3.444,"15min":3.29},"min":{"1min":1.7230000000000001,"5min":1.149,"15min":1.149},"max":{"1min":4.1619999999999999,"5min":21.228999999999999,"15min":21.228999999999999},"last":4.8330000000000002}]},{"osd":7,"last update":"Tue Sep 8 15:40:14 2020","interfaces":[{"interface":"back","average":{"1min":2.7200000000000002,"5min":3.5979999999999999,"15min":3.6699999999999999},"min":{"1min":1.196,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":4.665,"5min":22.911999999999999,"15min":22.911999999999999},"last":5.2969999999999997},{"interface":"front","average":{"1min":2.8769999999999998,"5min":3.9950000000000001,"15min":3.855},"min":{"1min":1.887,"5min":1.5609999999999999,"15min":1.5609999999999999},"max":{"1min":5.032,"5min":26.056000000000001,"15min":26.056000000000001},"last":2.121}]},{"osd":8,"last update":"Tue Sep 8 15:40:38 2020","interfaces":[{"interface":"back","average":{"1min":2.71,"5min":3.4449999999999998,"15min":3.7709999999999999},"min":{"1min":1.609,"5min":1.454,"15min":1.454},"max":{"1min":3.9140000000000001,"5min":6.3479999999999999,"15min":6.3479999999999999},"last":2.3399999999999999},{"interface":"front","average":{"1min":3.0219999999999998,"5min":3.472,"15min":3.952},"min":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"max":{"1min":8.6850000000000005,"5min":8.6850000000000005,"15min":8.6850000000000005},"last":5.3959999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:41:04 2020","interfaces":[{"interface":"back","average":{"1min":2.855,"5min":3.8180000000000001,"15min":4.2460000000000004},"min":{"1min":1.974,"5min":1.5569999999999999,"15min":1.5569999999999999},"max":{"1min":4.1710000000000003,"5min":21.716000000000001,"15min":21.716000000000001},"last":2.1760000000000002},{"interface":"front","average":{"1min":2.5950000000000002,"5min":3.4710000000000001,"15min":4.1699999999999999},"min":{"1min":1.224,"5min":1.224,"15min":1.224},"max":{"1min":4.4320000000000004,"5min":7.0250000000000004,"15min":7.0250000000000004},"last":1.9710000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:40:27 2020","interfaces":[{"interface":"back","average":{"1min":2.5030000000000001,"5min":3.855,"15min":4.415},"min":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"max":{"1min":3.3860000000000001,"5min":21.331,"15min":21.331},"last":2.2719999999999998},{"interface":"front","average":{"1min":3.2879999999999998,"5min":4.0179999999999998,"15min":4.4249999999999998},"min":{"1min":2.2290000000000001,"5min":1.915,"15min":1.915},"max":{"1min":6.0979999999999999,"5min":21.515999999999998,"15min":21.515999999999998},"last":2.4500000000000002}]}]},{"osd":1,"up_from":0,"seq":131,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":599648,"kb_used_data":533592,"kb_used_omap":0,"kb_used_meta":65984,"kb_avail":93145504,"statfs":{"total":95995035648,"available":95380996096,"internally_reserved":0,"allocated":546398208,"data_stored":546125080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":67567616},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:40:51 2020","interfaces":[{"interface":"back","average":{"1min":2.4079999999999999,"5min":3.2639999999999998,"15min":2.073},"min":{"1min":0.57399999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":7.7450000000000001,"5min":45.085000000000001,"15min":45.085000000000001},"last":1.579},{"interface":"front","average":{"1min":3.2490000000000001,"5min":3.6669999999999998,"15min":2.3250000000000002},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":13.231,"5min":45.765999999999998,"15min":45.765999999999998},"last":2.2029999999999998}]},{"osd":2,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":5.992,"5min":3.5139999999999998,"15min":2.3149999999999999},"min":{"1min":0.84699999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":44.039000000000001,"5min":44.039000000000001,"15min":44.039000000000001},"last":1.9690000000000001},{"interface":"front","average":{"1min":6.3090000000000002,"5min":3.79,"15min":2.5059999999999998},"min":{"1min":0.71799999999999997,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":44.841000000000001,"5min":44.841000000000001,"15min":44.841000000000001},"last":2.3820000000000001}]},{"osd":3,"last update":"Tue Sep 8 15:40:40 2020","interfaces":[{"interface":"back","average":{"1min":3.4700000000000002,"5min":3.5870000000000002,"15min":2.7149999999999999},"min":{"1min":0.98999999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":12.49,"5min":44.768999999999998,"15min":44.768999999999998},"last":3.4039999999999999},{"interface":"front","average":{"1min":4.1289999999999996,"5min":4.0570000000000004,"15min":2.964},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":21.399999999999999,"5min":45.892000000000003,"15min":45.892000000000003},"last":2.3079999999999998}]},{"osd":4,"last update":"Tue Sep 8 15:40:59 2020","interfaces":[{"interface":"back","average":{"1min":2.448,"5min":3.2770000000000001,"15min":2.5310000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":7.5599999999999996,"5min":44.970999999999997,"15min":44.970999999999997},"last":0.77100000000000002},{"interface":"front","average":{"1min":3.1579999999999999,"5min":3.484,"15min":3.0230000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":7.5940000000000003,"5min":51.289000000000001,"15min":51.289000000000001},"last":2.4590000000000001}]},{"osd":5,"last update":"Tue Sep 8 15:40:25 2020","interfaces":[{"interface":"back","average":{"1min":4.8380000000000001,"5min":3.819,"15min":3.246},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":45.289000000000001,"5min":45.289000000000001,"15min":45.289000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":5.8029999999999999,"5min":3.3580000000000001,"15min":2.75},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":51.499000000000002,"5min":51.499000000000002,"15min":51.499000000000002},"last":2.6000000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:40:51 2020","interfaces":[{"interface":"back","average":{"1min":3.375,"5min":4.0309999999999997,"15min":3.7519999999999998},"min":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"max":{"1min":11.973000000000001,"5min":44.932000000000002,"15min":44.932000000000002},"last":2.1629999999999998},{"interface":"front","average":{"1min":3.3769999999999998,"5min":4.2869999999999999,"15min":3.7959999999999998},"min":{"1min":1.6060000000000001,"5min":1.2949999999999999,"15min":1.2949999999999999},"max":{"1min":7.9980000000000002,"5min":45.159999999999997,"15min":45.159999999999997},"last":2.5350000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:40:17 2020","interfaces":[{"interface":"back","average":{"1min":5.5090000000000003,"5min":4.5990000000000002,"15min":3.6619999999999999},"min":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"max":{"1min":45.409999999999997,"5min":45.409999999999997,"15min":45.409999999999997},"last":3.0089999999999999},{"interface":"front","average":{"1min":4.9790000000000001,"5min":4.3410000000000002,"15min":4.0350000000000001},"min":{"1min":1.238,"5min":1.238,"15min":1.238},"max":{"1min":45.375,"5min":45.375,"15min":45.375},"last":1.3500000000000001}]},{"osd":8,"last update":"Tue Sep 8 15:40:40 2020","interfaces":[{"interface":"back","average":{"1min":3.6720000000000002,"5min":4.7400000000000002,"15min":4.4390000000000001},"min":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"max":{"1min":7.9850000000000003,"5min":45.640999999999998,"15min":45.640999999999998},"last":2.7210000000000001},{"interface":"front","average":{"1min":3.5409999999999999,"5min":4.7389999999999999,"15min":4.2889999999999997},"min":{"1min":1.153,"5min":1.153,"15min":1.153},"max":{"1min":11.839,"5min":45.235999999999997,"15min":45.235999999999997},"last":1.8460000000000001}]},{"osd":9,"last update":"Tue Sep 8 15:40:10 2020","interfaces":[{"interface":"back","average":{"1min":5.6619999999999999,"5min":3.6520000000000001,"15min":3.3769999999999998},"min":{"1min":1.103,"5min":1.0860000000000001,"15min":1.0860000000000001},"max":{"1min":51.698,"5min":51.698,"15min":51.698},"last":3.7890000000000001},{"interface":"front","average":{"1min":5.375,"5min":4.6189999999999998,"15min":4.359},"min":{"1min":1.45,"5min":1.45,"15min":1.45},"max":{"1min":45.515000000000001,"5min":45.515000000000001,"15min":45.515000000000001},"last":1.111}]},{"osd":10,"last update":"Tue Sep 8 15:40:25 2020","interfaces":[{"interface":"back","average":{"1min":5.6440000000000001,"5min":4.6870000000000003,"15min":4.3689999999999998},"min":{"1min":1.587,"5min":1.4159999999999999,"15min":1.4159999999999999},"max":{"1min":46.039999999999999,"5min":46.039999999999999,"15min":46.039999999999999},"last":2.8559999999999999},{"interface":"front","average":{"1min":5.6280000000000001,"5min":3.7879999999999998,"15min":3.593},"min":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":51.892000000000003,"5min":51.892000000000003,"15min":51.892000000000003},"last":1.2330000000000001}]}]},{"osd":0,"up_from":0,"seq":136,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":464608,"kb_used_data":405464,"kb_used_omap":0,"kb_used_meta":59136,"kb_avail":93280544,"statfs":{"total":95995035648,"available":95519277056,"internally_reserved":0,"allocated":415195136,"data_stored":414922008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":60555264},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 8 15:40:51 2020","interfaces":[{"interface":"back","average":{"1min":7.2130000000000001,"5min":3.347,"15min":2.0750000000000002},"min":{"1min":0.79700000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":74.831999999999994,"5min":74.831999999999994,"15min":74.831999999999994},"last":4.4189999999999996},{"interface":"front","average":{"1min":7.0250000000000004,"5min":3.371,"15min":2.1160000000000001},"min":{"1min":0.94999999999999996,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":72.611000000000004,"5min":72.611000000000004,"15min":72.611000000000004},"last":1.347}]},{"osd":2,"last update":"Tue Sep 8 15:34:05 2020","interfaces":[{"interface":"back","average":{"1min":1.9950000000000001,"5min":1.621,"15min":1.5189999999999999},"min":{"1min":1.3999999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.8519999999999999,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":2.6589999999999998},{"interface":"front","average":{"1min":2.629,"5min":1.956,"15min":1.72},"min":{"1min":1.6419999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":2.4049999999999998}]},{"osd":3,"last update":"Tue Sep 8 15:40:41 2020","interfaces":[{"interface":"back","average":{"1min":3.02,"5min":2.4660000000000002,"15min":2.7879999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":9.8670000000000009,"5min":9.8670000000000009,"15min":27.298999999999999},"last":0.999},{"interface":"front","average":{"1min":7.2450000000000001,"5min":3.8839999999999999,"15min":3.6880000000000002},"min":{"1min":1.167,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":73.075000000000003,"5min":73.075000000000003,"15min":73.075000000000003},"last":1.389}]},{"osd":4,"last update":"Tue Sep 8 15:41:05 2020","interfaces":[{"interface":"back","average":{"1min":6.1950000000000003,"5min":3.2440000000000002,"15min":2.7919999999999998},"min":{"1min":0.98599999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":67.105999999999995,"5min":67.105999999999995,"15min":67.105999999999995},"last":1.0820000000000001},{"interface":"front","average":{"1min":6.0279999999999996,"5min":4.1980000000000004,"15min":3.2879999999999998},"min":{"1min":1.3380000000000001,"5min":0.995,"15min":0.995},"max":{"1min":54.067999999999998,"5min":73.802999999999997,"15min":73.802999999999997},"last":1.1459999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:40:23 2020","interfaces":[{"interface":"back","average":{"1min":2.774,"5min":3.157,"15min":3.0489999999999999},"min":{"1min":0.83999999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":9.5730000000000004,"5min":9.5730000000000004,"15min":9.5730000000000004},"last":1.587},{"interface":"front","average":{"1min":2.0329999999999999,"5min":2.343,"15min":2.9359999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":3.923,"5min":12.890000000000001,"15min":12.890000000000001},"last":1.0589999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:40:51 2020","interfaces":[{"interface":"back","average":{"1min":2.1360000000000001,"5min":2.4510000000000001,"15min":3.1760000000000002},"min":{"1min":0.94699999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":5.71,"5min":13.151999999999999,"15min":13.151999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":2.3500000000000001,"5min":2.7669999999999999,"15min":3.5529999999999999},"min":{"1min":0.86899999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":5.923,"5min":13.48,"15min":13.48},"last":1.474}]},{"osd":7,"last update":"Tue Sep 8 15:40:20 2020","interfaces":[{"interface":"back","average":{"1min":7.6299999999999999,"5min":4.0199999999999996,"15min":4.2519999999999998},"min":{"1min":1.605,"5min":1.302,"15min":1.302},"max":{"1min":71.167000000000002,"5min":71.167000000000002,"15min":71.167000000000002},"last":1.681},{"interface":"front","average":{"1min":8.0850000000000009,"5min":4.4649999999999999,"15min":4.234},"min":{"1min":1.488,"5min":1.3240000000000001,"15min":1.3240000000000001},"max":{"1min":74.480000000000004,"5min":74.480000000000004,"15min":74.480000000000004},"last":2.137}]},{"osd":8,"last update":"Tue Sep 8 15:40:41 2020","interfaces":[{"interface":"back","average":{"1min":2.6989999999999998,"5min":3.5419999999999998,"15min":3.794},"min":{"1min":1.1020000000000001,"5min":1.093,"15min":1.093},"max":{"1min":6.5359999999999996,"5min":7.173,"15min":7.173},"last":1.7230000000000001},{"interface":"front","average":{"1min":7.0700000000000003,"5min":5.4770000000000003,"15min":4.5999999999999996},"min":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"max":{"1min":71.025000000000006,"5min":131.958,"15min":131.958},"last":4.9409999999999998}]},{"osd":9,"last update":"Tue Sep 8 15:41:05 2020","interfaces":[{"interface":"back","average":{"1min":6.0979999999999999,"5min":4.3170000000000002,"15min":4.2960000000000003},"min":{"1min":1.337,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":54.292999999999999,"5min":54.292999999999999,"15min":54.292999999999999},"last":1.851},{"interface":"front","average":{"1min":6.9370000000000003,"5min":4.0970000000000004,"15min":4.077},"min":{"1min":1.1559999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":67.659999999999997,"5min":67.659999999999997,"15min":67.659999999999997},"last":2.0070000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:40:23 2020","interfaces":[{"interface":"back","average":{"1min":2.5630000000000002,"5min":2.9220000000000002,"15min":3.0539999999999998},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":4.7489999999999997,"5min":6.923,"15min":6.923},"last":2.7160000000000002},{"interface":"front","average":{"1min":7.468,"5min":4.7030000000000003,"15min":4.4909999999999997},"min":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"max":{"1min":73.444000000000003,"5min":73.444000000000003,"15min":73.444000000000003},"last":1.659}]},{"osd":11,"last update":"Tue Sep 8 15:40:51 2020","interfaces":[{"interface":"back","average":{"1min":8.3390000000000004,"5min":4.6470000000000002,"15min":4.141},"min":{"1min":2.1190000000000002,"5min":1.476,"15min":1.476},"max":{"1min":74.147000000000006,"5min":74.147000000000006,"15min":74.147000000000006},"last":1.784},{"interface":"front","average":{"1min":3.3090000000000002,"5min":3.8700000000000001,"15min":4.5300000000000002},"min":{"1min":1.052,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":9.4559999999999995,"5min":13.369999999999999,"15min":13.369999999999999},"last":1.5680000000000001}]}]},{"osd":2,"up_from":0,"seq":128,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":445024,"kb_used_data":380952,"kb_used_omap":0,"kb_used_meta":64064,"kb_avail":93300128,"statfs":{"total":95995035648,"available":95539331072,"internally_reserved":0,"allocated":390094848,"data_stored":389821720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65601536},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:41:10 2020","interfaces":[{"interface":"back","average":{"1min":2.8519999999999999,"5min":3.4910000000000001,"15min":2.2949999999999999},"min":{"1min":0.67700000000000005,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":11.571,"5min":162.83799999999999,"15min":162.83799999999999},"last":4.7519999999999998},{"interface":"front","average":{"1min":3.0449999999999999,"5min":3.6299999999999999,"15min":2.3849999999999998},"min":{"1min":1.0109999999999999,"5min":0.624,"15min":0.624},"max":{"1min":11.326000000000001,"5min":162.63499999999999,"15min":162.63499999999999},"last":1.0109999999999999}]},{"osd":1,"last update":"Tue Sep 8 15:41:10 2020","interfaces":[{"interface":"back","average":{"1min":6.5910000000000002,"5min":3.8460000000000001,"15min":2.444},"min":{"1min":1.026,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":78.728999999999999,"5min":78.728999999999999,"15min":78.728999999999999},"last":1.272},{"interface":"front","average":{"1min":6.7409999999999997,"5min":3.4129999999999998,"15min":2.5880000000000001},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":80.103999999999999,"5min":80.103999999999999,"15min":80.103999999999999},"last":1.107}]},{"osd":3,"last update":"Tue Sep 8 15:40:36 2020","interfaces":[{"interface":"back","average":{"1min":3.0019999999999998,"5min":2.8570000000000002,"15min":2.5449999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":11.315,"5min":11.315,"15min":11.315},"last":4.6390000000000002},{"interface":"front","average":{"1min":2.923,"5min":2.645,"15min":2.4609999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":11.737,"5min":11.737,"15min":11.737},"last":6.0999999999999996}]},{"osd":4,"last update":"Tue Sep 8 15:41:07 2020","interfaces":[{"interface":"back","average":{"1min":2.7040000000000002,"5min":2.6850000000000001,"15min":2.5600000000000001},"min":{"1min":0.97699999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":7.6550000000000002,"5min":7.6550000000000002,"15min":7.6550000000000002},"last":1.4299999999999999},{"interface":"front","average":{"1min":3.2989999999999999,"5min":3.5,"15min":2.8029999999999999},"min":{"1min":1.1000000000000001,"5min":1.04,"15min":1.04},"max":{"1min":9.7539999999999996,"5min":64.284999999999997,"15min":64.284999999999997},"last":1.927}]},{"osd":5,"last update":"Tue Sep 8 15:40:25 2020","interfaces":[{"interface":"back","average":{"1min":5.7460000000000004,"5min":3.875,"15min":3.2469999999999999},"min":{"1min":1.222,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":63.509,"5min":63.509,"15min":63.509},"last":1.4079999999999999},{"interface":"front","average":{"1min":3.0230000000000001,"5min":3.2469999999999999,"15min":3.0579999999999998},"min":{"1min":1.2689999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"max":{"1min":11.055999999999999,"5min":11.055999999999999,"15min":11.055999999999999},"last":1.823}]},{"osd":6,"last update":"Tue Sep 8 15:40:48 2020","interfaces":[{"interface":"back","average":{"1min":6.1989999999999998,"5min":4.0620000000000003,"15min":3.4660000000000002},"min":{"1min":1.3129999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"max":{"1min":63.631,"5min":63.631,"15min":63.631},"last":1.7969999999999999},{"interface":"front","average":{"1min":2.9369999999999998,"5min":3.3340000000000001,"15min":3.4929999999999999},"min":{"1min":1.1870000000000001,"5min":1.105,"15min":1.105},"max":{"1min":8.0120000000000005,"5min":11.234999999999999,"15min":11.234999999999999},"last":1.9590000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:40:21 2020","interfaces":[{"interface":"back","average":{"1min":3.3780000000000001,"5min":3.8180000000000001,"15min":3.9569999999999999},"min":{"1min":1.298,"5min":1.298,"15min":1.298},"max":{"1min":11.436,"5min":11.436,"15min":11.436},"last":2.2559999999999998},{"interface":"front","average":{"1min":3.0670000000000002,"5min":3.3940000000000001,"15min":3.5950000000000002},"min":{"1min":1.016,"5min":1.006,"15min":1.006},"max":{"1min":7.2270000000000003,"5min":7.8659999999999997,"15min":7.8659999999999997},"last":2.085}]},{"osd":8,"last update":"Tue Sep 8 15:40:30 2020","interfaces":[{"interface":"back","average":{"1min":6.0899999999999999,"5min":4.0759999999999996,"15min":3.802},"min":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":64.116,"5min":64.116,"15min":64.116},"last":1.5429999999999999},{"interface":"front","average":{"1min":5.9539999999999997,"5min":4.2370000000000001,"15min":4.0620000000000003},"min":{"1min":1.532,"5min":1.218,"15min":1.218},"max":{"1min":63.299999999999997,"5min":63.299999999999997,"15min":63.299999999999997},"last":3.3340000000000001}]},{"osd":9,"last update":"Tue Sep 8 15:41:10 2020","interfaces":[{"interface":"back","average":{"1min":3.7749999999999999,"5min":4.4000000000000004,"15min":4.165},"min":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"max":{"1min":10.406000000000001,"5min":64.004000000000005,"15min":64.004000000000005},"last":2.0539999999999998},{"interface":"front","average":{"1min":3.629,"5min":3.669,"15min":4.085},"min":{"1min":1.706,"5min":1.401,"15min":1.401},"max":{"1min":11.869,"5min":11.869,"15min":11.869},"last":1.706}]},{"osd":10,"last update":"Tue Sep 8 15:40:25 2020","interfaces":[{"interface":"back","average":{"1min":6.4800000000000004,"5min":4.5209999999999999,"15min":4.1890000000000001},"min":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"max":{"1min":63.743000000000002,"5min":63.743000000000002,"15min":63.743000000000002},"last":1.671},{"interface":"front","average":{"1min":3.0800000000000001,"5min":3.2559999999999998,"15min":3.6139999999999999},"min":{"1min":1.159,"5min":1.159,"15min":1.159},"max":{"1min":6.9690000000000003,"5min":7.9960000000000004,"15min":7.9960000000000004},"last":1.5660000000000001}]}]},{"osd":3,"up_from":0,"seq":121,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":461472,"kb_used_data":397400,"kb_used_omap":0,"kb_used_meta":64000,"kb_avail":93283680,"statfs":{"total":95995035648,"available":95522488320,"internally_reserved":0,"allocated":406937600,"data_stored":406664472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65536000},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:31 2020","interfaces":[{"interface":"back","average":{"1min":2.1379999999999999,"5min":1.7210000000000001,"15min":1.629},"min":{"1min":1.79,"5min":1.1419999999999999,"15min":1.1419999999999999},"max":{"1min":2.7930000000000001,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":3.169},{"interface":"front","average":{"1min":2.6800000000000002,"5min":2.2309999999999999,"15min":2.0990000000000002},"min":{"1min":2.0270000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"max":{"1min":3.5409999999999999,"5min":3.5409999999999999,"15min":3.5409999999999999},"last":2.5579999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:40:42 2020","interfaces":[{"interface":"back","average":{"1min":3.4740000000000002,"5min":2.9100000000000001,"15min":2.2309999999999999},"min":{"1min":0.76000000000000001,"5min":0.628,"15min":0.628},"max":{"1min":11.423999999999999,"5min":44.109999999999999,"15min":44.109999999999999},"last":1.198},{"interface":"front","average":{"1min":3.0030000000000001,"5min":3.7810000000000001,"15min":2.508},"min":{"1min":0.97299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":7.532,"5min":45.618000000000002,"15min":45.618000000000002},"last":1.643}]},{"osd":2,"last update":"Tue Sep 8 15:40:42 2020","interfaces":[{"interface":"back","average":{"1min":2.605,"5min":3.222,"15min":2.5089999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":10.340999999999999,"5min":45.774000000000001,"15min":45.774000000000001},"last":1.109},{"interface":"front","average":{"1min":3.036,"5min":2.548,"15min":2.375},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":12.098000000000001,"5min":13.948,"15min":13.948},"last":1.143}]},{"osd":4,"last update":"Tue Sep 8 15:41:03 2020","interfaces":[{"interface":"back","average":{"1min":2.746,"5min":2.5169999999999999,"15min":2.5659999999999998},"min":{"1min":1.018,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":10.523,"5min":10.523,"15min":10.523},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.9510000000000001,"5min":2.1880000000000002,"15min":2.4239999999999999},"min":{"1min":0.91100000000000003,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":5.3460000000000001,"5min":11.298,"15min":11.298},"last":1.758}]},{"osd":5,"last update":"Tue Sep 8 15:40:27 2020","interfaces":[{"interface":"back","average":{"1min":3.2200000000000002,"5min":3.1469999999999998,"15min":2.964},"min":{"1min":1.0609999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"max":{"1min":27.007999999999999,"5min":45.442999999999998,"15min":45.442999999999998},"last":1.5029999999999999},{"interface":"front","average":{"1min":4.556,"5min":3.3500000000000001,"15min":3.2789999999999999},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":23.565000000000001,"5min":23.565000000000001,"15min":23.565000000000001},"last":1.7849999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:40:53 2020","interfaces":[{"interface":"back","average":{"1min":2.7999999999999998,"5min":2.8889999999999998,"15min":3.2029999999999998},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":7.7969999999999997,"5min":7.7969999999999997,"15min":7.7969999999999997},"last":1.3560000000000001},{"interface":"front","average":{"1min":2.3940000000000001,"5min":3.214,"15min":3.452},"min":{"1min":1.3560000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":5.9859999999999998,"5min":47.457999999999998,"15min":47.457999999999998},"last":5.7690000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:40:27 2020","interfaces":[{"interface":"back","average":{"1min":5.0999999999999996,"5min":4.0739999999999998,"15min":4.1959999999999997},"min":{"1min":1.3899999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"max":{"1min":26.035,"5min":26.035,"15min":26.035},"last":6.2249999999999996},{"interface":"front","average":{"1min":3.7749999999999999,"5min":3.2509999999999999,"15min":3.4990000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":25.521000000000001,"5min":25.521000000000001,"15min":25.521000000000001},"last":1.476}]},{"osd":8,"last update":"Tue Sep 8 15:40:39 2020","interfaces":[{"interface":"back","average":{"1min":2.6099999999999999,"5min":3.5870000000000002,"15min":3.7069999999999999},"min":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":8.4529999999999994,"5min":46.917999999999999,"15min":46.917999999999999},"last":1.9219999999999999},{"interface":"front","average":{"1min":4.1689999999999996,"5min":4.6900000000000004,"15min":4.4249999999999998},"min":{"1min":1.272,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":11.534000000000001,"5min":44.893000000000001,"15min":44.893000000000001},"last":1.8160000000000001}]},{"osd":9,"last update":"Tue Sep 8 15:41:03 2020","interfaces":[{"interface":"back","average":{"1min":2.7610000000000001,"5min":2.9750000000000001,"15min":3.645},"min":{"1min":1.272,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":7.6669999999999998,"5min":10.872999999999999,"15min":10.872999999999999},"last":1.333},{"interface":"front","average":{"1min":2.605,"5min":3.7909999999999999,"15min":4.1509999999999998},"min":{"1min":1.5409999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"max":{"1min":5.5679999999999996,"5min":46.665999999999997,"15min":46.665999999999997},"last":1.9570000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:40:27 2020","interfaces":[{"interface":"back","average":{"1min":3.9780000000000002,"5min":4.3280000000000003,"15min":4.3739999999999997},"min":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"max":{"1min":11.694000000000001,"5min":39.283000000000001,"15min":39.283000000000001},"last":2.0640000000000001},{"interface":"front","average":{"1min":2.9279999999999999,"5min":3.3010000000000002,"15min":3.6080000000000001},"min":{"1min":1.081,"5min":1.081,"15min":1.081},"max":{"1min":11.005000000000001,"5min":11.005000000000001,"15min":11.005000000000001},"last":3.7799999999999998}]},{"osd":11,"last update":"Tue Sep 8 15:40:53 2020","interfaces":[{"interface":"back","average":{"1min":2.5209999999999999,"5min":3.8540000000000001,"15min":4.4950000000000001},"min":{"1min":1.5549999999999999,"5min":1.196,"15min":1.196},"max":{"1min":5.8390000000000004,"5min":13.882999999999999,"15min":13.882999999999999},"last":1.609},{"interface":"front","average":{"1min":2.7010000000000001,"5min":4.9370000000000003,"15min":4.4089999999999998},"min":{"1min":1.677,"5min":1.577,"15min":1.577},"max":{"1min":7.6639999999999997,"5min":47.234999999999999,"15min":47.234999999999999},"last":2.0939999999999999}]}]},{"osd":4,"up_from":0,"seq":117,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":170852,"kb_used_data":149532,"kb_used_omap":0,"kb_used_meta":21248,"kb_avail":93574300,"statfs":{"total":95995035648,"available":95820083200,"internally_reserved":0,"allocated":153120768,"data_stored":152843568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21757952},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:41:00 2020","interfaces":[{"interface":"back","average":{"1min":1.6830000000000001,"5min":1.8320000000000001,"15min":1.865},"min":{"1min":0.81100000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":4.7850000000000001,"5min":5.7130000000000001,"15min":5.7130000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":2.3700000000000001,"5min":2.0230000000000001,"15min":1.915},"min":{"1min":0.83699999999999997,"5min":0.71299999999999997,"15min":0.70799999999999996},"max":{"1min":10.471,"5min":10.471,"15min":10.471},"last":1.948}]},{"osd":1,"last update":"Tue Sep 8 15:41:00 2020","interfaces":[{"interface":"back","average":{"1min":3.2559999999999998,"5min":3.7210000000000001,"15min":2.4929999999999999},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":14.127000000000001,"5min":74.947999999999993,"15min":74.947999999999993},"last":1.0880000000000001},{"interface":"front","average":{"1min":2.992,"5min":3.9289999999999998,"15min":2.738},"min":{"1min":1.339,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":6.665,"5min":71.108999999999995,"15min":71.108999999999995},"last":1.1279999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:41:00 2020","interfaces":[{"interface":"back","average":{"1min":2.8980000000000001,"5min":3.8769999999999998,"15min":2.7309999999999999},"min":{"1min":1.5680000000000001,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":7.1790000000000003,"5min":74.504000000000005,"15min":74.504000000000005},"last":2.5539999999999998},{"interface":"front","average":{"1min":2.6349999999999998,"5min":3.6949999999999998,"15min":2.786},"min":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"max":{"1min":6.8630000000000004,"5min":73.885000000000005,"15min":73.885000000000005},"last":4.1790000000000003}]},{"osd":3,"last update":"Tue Sep 8 15:41:00 2020","interfaces":[{"interface":"back","average":{"1min":1.9670000000000001,"5min":2.6080000000000001,"15min":2.556},"min":{"1min":1.0169999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":5.7240000000000002,"5min":6.3280000000000003,"15min":6.3280000000000003},"last":2.5219999999999998},{"interface":"front","average":{"1min":3.4670000000000001,"5min":2.9260000000000002,"15min":2.5630000000000002},"min":{"1min":1.379,"5min":1.1539999999999999,"15min":1.1539999999999999},"max":{"1min":9.5069999999999997,"5min":9.5069999999999997,"15min":9.5069999999999997},"last":2.2559999999999998}]},{"osd":5,"last update":"Tue Sep 8 15:40:28 2020","interfaces":[{"interface":"back","average":{"1min":2.351,"5min":2.9860000000000002,"15min":2.8220000000000001},"min":{"1min":1.2709999999999999,"5min":1.024,"15min":1.014},"max":{"1min":3.8759999999999999,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":2.3330000000000002},{"interface":"front","average":{"1min":2.161,"5min":3.0310000000000001,"15min":2.8759999999999999},"min":{"1min":1.274,"5min":1.2190000000000001,"15min":1.129},"max":{"1min":3.6949999999999998,"5min":7.5359999999999996,"15min":7.5359999999999996},"last":2.4300000000000002}]},{"osd":6,"last update":"Tue Sep 8 15:40:46 2020","interfaces":[{"interface":"back","average":{"1min":2.8260000000000001,"5min":3.2639999999999998,"15min":3.3159999999999998},"min":{"1min":1.645,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":6.6420000000000003,"5min":6.8650000000000002,"15min":6.8650000000000002},"last":1.413},{"interface":"front","average":{"1min":2.3940000000000001,"5min":3.1549999999999998,"15min":3.2999999999999998},"min":{"1min":1.4450000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"max":{"1min":6.8849999999999998,"5min":6.8849999999999998,"15min":6.8849999999999998},"last":2.0129999999999999}]},{"osd":7,"last update":"Tue Sep 8 15:41:08 2020","interfaces":[{"interface":"back","average":{"1min":2.6240000000000001,"5min":3.3119999999999998,"15min":3.7029999999999998},"min":{"1min":1.6890000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"max":{"1min":6.4470000000000001,"5min":7.2969999999999997,"15min":7.4699999999999998},"last":1.6890000000000001},{"interface":"front","average":{"1min":3.133,"5min":3.4809999999999999,"15min":3.8260000000000001},"min":{"1min":1.8260000000000001,"5min":1.546,"15min":1.546},"max":{"1min":6.0199999999999996,"5min":7.1459999999999999,"15min":7.1459999999999999},"last":1.8260000000000001}]},{"osd":8,"last update":"Tue Sep 8 15:40:39 2020","interfaces":[{"interface":"back","average":{"1min":2.9449999999999998,"5min":5.2080000000000002,"15min":4.3559999999999999},"min":{"1min":1.276,"5min":1.276,"15min":1.276},"max":{"1min":7.8140000000000001,"5min":109.608,"15min":109.608},"last":6.5060000000000002},{"interface":"front","average":{"1min":3.222,"5min":5.3799999999999999,"15min":4.7039999999999997},"min":{"1min":1.889,"5min":1.49,"15min":1.49},"max":{"1min":6.4039999999999999,"5min":116.52500000000001,"15min":116.52500000000001},"last":7.1479999999999997}]},{"osd":9,"last update":"Tue Sep 8 15:41:06 2020","interfaces":[{"interface":"back","average":{"1min":2.8759999999999999,"5min":5.2300000000000004,"15min":4.6630000000000003},"min":{"1min":1.6259999999999999,"5min":1.546,"15min":1.546},"max":{"1min":6.3470000000000004,"5min":118.312,"15min":118.312},"last":1.9059999999999999},{"interface":"front","average":{"1min":3.375,"5min":5.1280000000000001,"15min":4.6529999999999996},"min":{"1min":2.0449999999999999,"5min":1.647,"15min":1.647},"max":{"1min":6.524,"5min":116.871,"15min":116.871},"last":1.7270000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:40:28 2020","interfaces":[{"interface":"back","average":{"1min":2.593,"5min":3.5710000000000002,"15min":4.4829999999999997},"min":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"max":{"1min":6.5170000000000003,"5min":7.9279999999999999,"15min":7.9279999999999999},"last":1.7689999999999999},{"interface":"front","average":{"1min":2.5950000000000002,"5min":3.8330000000000002,"15min":4.5629999999999997},"min":{"1min":1.702,"5min":1.702,"15min":1.702},"max":{"1min":4.5609999999999999,"5min":7.9870000000000001,"15min":7.9870000000000001},"last":2.0870000000000002}]}]},{"osd":5,"up_from":0,"seq":111,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":300192,"kb_used_data":255832,"kb_used_omap":0,"kb_used_meta":44352,"kb_avail":93444960,"statfs":{"total":95995035648,"available":95687639040,"internally_reserved":0,"allocated":261971968,"data_stored":261698840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":45416448},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:40:29 2020","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.8720000000000001,"15min":1.881},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":5.4370000000000003,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":1.165},{"interface":"front","average":{"1min":1.7350000000000001,"5min":2.3210000000000002,"15min":2.0790000000000002},"min":{"1min":0.90800000000000003,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":4.4859999999999998,"5min":7.1340000000000003,"15min":7.1340000000000003},"last":2.528}]},{"osd":1,"last update":"Tue Sep 8 15:40:29 2020","interfaces":[{"interface":"back","average":{"1min":4.0940000000000003,"5min":2.8610000000000002,"15min":2.492},"min":{"1min":0.97799999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":35.908999999999999,"5min":35.908999999999999,"15min":35.908999999999999},"last":1.212},{"interface":"front","average":{"1min":4.79,"5min":3.3860000000000001,"15min":2.464},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":36.561999999999998,"5min":36.561999999999998,"15min":36.561999999999998},"last":2.7890000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:40:29 2020","interfaces":[{"interface":"back","average":{"1min":2.649,"5min":2.7970000000000002,"15min":2.3530000000000002},"min":{"1min":0.97499999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":6.8250000000000002,"5min":31.021000000000001,"15min":31.021000000000001},"last":1.266},{"interface":"front","average":{"1min":2.766,"5min":3.2770000000000001,"15min":2.903},"min":{"1min":1.1100000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"max":{"1min":5.5229999999999997,"5min":37.658999999999999,"15min":37.658999999999999},"last":3.1059999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:40:29 2020","interfaces":[{"interface":"back","average":{"1min":2.6659999999999999,"5min":3.3599999999999999,"15min":3.0030000000000001},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":8.4220000000000006,"5min":8.4220000000000006,"15min":8.4220000000000006},"last":2.972},{"interface":"front","average":{"1min":2.3479999999999999,"5min":2.8149999999999999,"15min":2.8860000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":6.819,"5min":6.819,"15min":6.819},"last":1.575}]},{"osd":4,"last update":"Tue Sep 8 15:40:29 2020","interfaces":[{"interface":"back","average":{"1min":2.339,"5min":2.5070000000000001,"15min":2.3610000000000002},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":6.9180000000000001,"5min":6.9180000000000001,"15min":6.9180000000000001},"last":1.7929999999999999},{"interface":"front","average":{"1min":2.1080000000000001,"5min":2.899,"15min":2.738},"min":{"1min":1.1990000000000001,"5min":1.1819999999999999,"15min":1.1819999999999999},"max":{"1min":3.3889999999999998,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":1.5529999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:40:58 2020","interfaces":[{"interface":"back","average":{"1min":2.5499999999999998,"5min":3.456,"15min":3.552},"min":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"max":{"1min":4.3810000000000002,"5min":8.7710000000000008,"15min":8.7710000000000008},"last":3.2360000000000002},{"interface":"front","average":{"1min":2.9980000000000002,"5min":4.024,"15min":3.3929999999999998},"min":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"max":{"1min":4.6669999999999998,"5min":9.359,"15min":9.359},"last":3.3639999999999999}]},{"osd":7,"last update":"Tue Sep 8 15:40:13 2020","interfaces":[{"interface":"back","average":{"1min":2.996,"5min":3.5680000000000001,"15min":3.5609999999999999},"min":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"max":{"1min":6.9909999999999997,"5min":6.9909999999999997,"15min":6.9909999999999997},"last":3.7759999999999998},{"interface":"front","average":{"1min":4.0149999999999997,"5min":4.1539999999999999,"15min":4.0709999999999997},"min":{"1min":2.2789999999999999,"5min":1.532,"15min":1.532},"max":{"1min":9.2129999999999992,"5min":9.2129999999999992,"15min":9.2129999999999992},"last":3.9049999999999998}]},{"osd":8,"last update":"Tue Sep 8 15:40:39 2020","interfaces":[{"interface":"back","average":{"1min":2.8279999999999998,"5min":3.2719999999999998,"15min":3.4990000000000001},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":5.8330000000000002,"5min":7.8140000000000001,"15min":7.8140000000000001},"last":2.214},{"interface":"front","average":{"1min":2.641,"5min":3.2719999999999998,"15min":3.794},"min":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"max":{"1min":10.138999999999999,"5min":10.138999999999999,"15min":10.138999999999999},"last":1.9790000000000001}]},{"osd":9,"last update":"Tue Sep 8 15:41:09 2020","interfaces":[{"interface":"back","average":{"1min":2.8620000000000001,"5min":3.4620000000000002,"15min":4.2350000000000003},"min":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"max":{"1min":8.1980000000000004,"5min":8.1980000000000004,"15min":8.1980000000000004},"last":2.0070000000000001},{"interface":"front","average":{"1min":3.3889999999999998,"5min":4.0519999999999996,"15min":4.1630000000000003},"min":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"max":{"1min":8.2560000000000002,"5min":8.9149999999999991,"15min":8.9149999999999991},"last":3.613}]},{"osd":10,"last update":"Tue Sep 8 15:40:29 2020","interfaces":[{"interface":"back","average":{"1min":2.4300000000000002,"5min":3.3130000000000002,"15min":3.9289999999999998},"min":{"1min":1.431,"5min":1.431,"15min":1.431},"max":{"1min":4.4690000000000003,"5min":5.7309999999999999,"15min":5.7309999999999999},"last":2.1190000000000002},{"interface":"front","average":{"1min":3.762,"5min":4.4089999999999998,"15min":4.6609999999999996},"min":{"1min":2.2610000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"max":{"1min":9.0670000000000002,"5min":9.0670000000000002,"15min":9.0670000000000002},"last":4.0590000000000002}]}]}]}} 2020-09-08T15:41:17.002 INFO:tasks.thrashosds.thrasher:Setting 2.a to [10, 8, 1, 2] 2020-09-08T15:41:17.002 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.a', '10', '8', '1', '2'] 2020-09-08T15:41:17.003 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pg-upmap-items 2.a 10 8 1 2 2020-09-08T15:41:17.004 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:41:17.076 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:41:17.198 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:41:17.575 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:41:17.679 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:41:17.853 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:41:17.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:17 smithi166 bash[9446]: cluster 2020-09-08T15:41:17.648544+0000 mgr.x (mgr.24118) 416 : cluster [DBG] pgmap v536: 17 pgs: 17 active+clean; 1.8 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 961 op/s 2020-09-08T15:41:17.885 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:41:17 smithi104 bash[22305]: debug 2020-09-08T15:41:17.849+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:17.886 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:41:17 smithi104 bash[17588]: debug 2020-09-08T15:41:17.573+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:17.886 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:17 smithi104 bash[9203]: cluster 2020-09-08T15:41:17.648544+0000 mgr.x (mgr.24118) 416 : cluster [DBG] pgmap v536: 17 pgs: 17 active+clean; 1.8 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 961 op/s 2020-09-08T15:41:17.956 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:41:18.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:17 smithi055 bash[9296]: cluster 2020-09-08T15:41:17.648544+0000 mgr.x (mgr.24118) 416 : cluster [DBG] pgmap v536: 17 pgs: 17 active+clean; 1.8 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 961 op/s 2020-09-08T15:41:18.311 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:41:18.415 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:18.602 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:18.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:41:18 smithi166 bash[11928]: debug 2020-09-08T15:41:18.312+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:18.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:41:18 smithi104 bash[19922]: debug 2020-09-08T15:41:18.597+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:18.704 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:18.847 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:18.952 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:19.005 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:18 smithi055 bash[9296]: cluster 2020-09-08T15:41:18.090763+0000 osd.10 (osd.10) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:19.006 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:18 smithi055 bash[9296]: cluster 2020-09-08T15:41:18.090860+0000 osd.10 (osd.10) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:19.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:18 smithi166 bash[9446]: cluster 2020-09-08T15:41:18.090763+0000 osd.10 (osd.10) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:19.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:18 smithi166 bash[9446]: cluster 2020-09-08T15:41:18.090860+0000 osd.10 (osd.10) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:19.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:41:18 smithi104 bash[19922]: debug 2020-09-08T15:41:18.845+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:19.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:18 smithi104 bash[9203]: cluster 2020-09-08T15:41:18.090763+0000 osd.10 (osd.10) 15 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:19.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:18 smithi104 bash[9203]: cluster 2020-09-08T15:41:18.090860+0000 osd.10 (osd.10) 16 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:19.735 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:41:19.776 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:41:19.821 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:41:19.858 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:41:19.899 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:41:19.925 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:41:19.958 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:41:19.987 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:41:19.999 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:19 smithi166 bash[9446]: cluster 2020-09-08T15:41:18.717127+0000 mon.a (mon.0) 1073 : cluster [DBG] osdmap e202: 12 total, 12 up, 12 in 2020-09-08T15:41:19.999 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:19 smithi166 bash[9446]: cluster 2020-09-08T15:41:19.649354+0000 mgr.x (mgr.24118) 417 : cluster [DBG] pgmap v538: 17 pgs: 17 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.22k op/s 2020-09-08T15:41:20.000 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:19 smithi166 bash[9446]: audit 2020-09-08T15:41:19.691648+0000 mon.a (mon.0) 1074 : audit [DBG] from='client.? v1:172.21.15.104:0/646105093' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:41:20.014 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:19 smithi104 bash[9203]: cluster 2020-09-08T15:41:18.717127+0000 mon.a (mon.0) 1073 : cluster [DBG] osdmap e202: 12 total, 12 up, 12 in 2020-09-08T15:41:20.015 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:19 smithi104 bash[9203]: cluster 2020-09-08T15:41:19.649354+0000 mgr.x (mgr.24118) 417 : cluster [DBG] pgmap v538: 17 pgs: 17 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.22k op/s 2020-09-08T15:41:20.015 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:19 smithi104 bash[9203]: audit 2020-09-08T15:41:19.691648+0000 mon.a (mon.0) 1074 : audit [DBG] from='client.? v1:172.21.15.104:0/646105093' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:41:20.027 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:41:20.053 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:41:20.079 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:41:20.101 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:41:20.103 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:41:20.104 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:41:20.104 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:41:20.104 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:41:20.104 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:41:20.105 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:41:20.105 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:41:20.105 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:41:20.105 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:41:20.105 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:41:20.106 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:41:20.106 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:41:20.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:19 smithi055 bash[9296]: cluster 2020-09-08T15:41:18.717127+0000 mon.a (mon.0) 1073 : cluster [DBG] osdmap e202: 12 total, 12 up, 12 in 2020-09-08T15:41:20.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:19 smithi055 bash[9296]: cluster 2020-09-08T15:41:19.649354+0000 mgr.x (mgr.24118) 417 : cluster [DBG] pgmap v538: 17 pgs: 17 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.22k op/s 2020-09-08T15:41:20.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:19 smithi055 bash[9296]: audit 2020-09-08T15:41:19.691648+0000 mon.a (mon.0) 1074 : audit [DBG] from='client.? v1:172.21.15.104:0/646105093' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:41:20.185 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:41:20.186 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:41:21.156 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_blocked_ops 2020-09-08T15:41:21.319 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:41:21.377 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:21.478 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:21.658 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:41:21 smithi055 bash[14301]: debug 2020-09-08T15:41:21.362+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:21.767 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:21.878 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:21.997 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:22.100 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:22.108 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:41:21 smithi104 bash[19922]: debug 2020-09-08T15:41:21.769+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:22.162 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:41:22 smithi055 bash[14301]: debug 2020-09-08T15:41:21.994+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:22.535 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:22.639 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:41:22.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:22 smithi104 bash[9203]: cluster 2020-09-08T15:41:21.650008+0000 mgr.x (mgr.24118) 418 : cluster [DBG] pgmap v539: 17 pgs: 17 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 719 op/s 2020-09-08T15:41:22.927 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:41:22 smithi104 bash[15273]: debug 2020-09-08T15:41:22.533+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:22.956 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:22 smithi166 bash[9446]: cluster 2020-09-08T15:41:21.650008+0000 mgr.x (mgr.24118) 418 : cluster [DBG] pgmap v539: 17 pgs: 17 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 719 op/s 2020-09-08T15:41:23.058 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:41:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:22 smithi055 bash[9296]: cluster 2020-09-08T15:41:21.650008+0000 mgr.x (mgr.24118) 418 : cluster [DBG] pgmap v539: 17 pgs: 17 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 719 op/s 2020-09-08T15:41:23.162 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:23.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:41:23 smithi104 bash[17588]: debug 2020-09-08T15:41:23.057+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:23.527 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:23.629 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:23.733 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:23.838 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:23.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:41:23 smithi055 bash[16646]: debug 2020-09-08T15:41:23.726+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:23.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:23 smithi055 bash[9296]: cluster 2020-09-08T15:41:21.987685+0000 osd.3 (osd.3) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:23.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:23 smithi055 bash[9296]: cluster 2020-09-08T15:41:21.987767+0000 osd.3 (osd.3) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:23.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:23 smithi055 bash[9296]: cluster 2020-09-08T15:41:22.713750+0000 mon.a (mon.0) 1075 : cluster [DBG] osdmap e203: 12 total, 12 up, 12 in 2020-09-08T15:41:23.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:41:23 smithi104 bash[19922]: debug 2020-09-08T15:41:23.517+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:23.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:23 smithi104 bash[9203]: cluster 2020-09-08T15:41:21.987685+0000 osd.3 (osd.3) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:23.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:23 smithi104 bash[9203]: cluster 2020-09-08T15:41:21.987767+0000 osd.3 (osd.3) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:23.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:23 smithi104 bash[9203]: cluster 2020-09-08T15:41:22.713750+0000 mon.a (mon.0) 1075 : cluster [DBG] osdmap e203: 12 total, 12 up, 12 in 2020-09-08T15:41:24.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:23 smithi166 bash[9446]: cluster 2020-09-08T15:41:21.987685+0000 osd.3 (osd.3) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:24.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:23 smithi166 bash[9446]: cluster 2020-09-08T15:41:21.987767+0000 osd.3 (osd.3) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:24.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:23 smithi166 bash[9446]: cluster 2020-09-08T15:41:22.713750+0000 mon.a (mon.0) 1075 : cluster [DBG] osdmap e203: 12 total, 12 up, 12 in 2020-09-08T15:41:24.278 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:41:24.334 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:24.437 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:24.451 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:41:24.547 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:24.615 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:41:24 smithi166 bash[14254]: debug 2020-09-08T15:41:24.332+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:24.651 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:24.909 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:41:24 smithi055 bash[11956]: debug 2020-09-08T15:41:24.550+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:25.266 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:25.370 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:25.511 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:25.553 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:41:25.553 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:41:25.616 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:41:25.617 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:25 smithi104 bash[9203]: cluster 2020-09-08T15:41:23.650494+0000 mgr.x (mgr.24118) 419 : cluster [DBG] pgmap v541: 17 pgs: 17 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T15:41:25.618 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:25 smithi104 bash[9203]: audit 2020-09-08T15:41:24.626365+0000 mon.c (mon.2) 43 : audit [INF] from='client.? v1:172.21.15.104:0/2873220198' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:41:25.618 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:41:25 smithi104 bash[15273]: debug 2020-09-08T15:41:25.265+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:25.618 INFO:teuthology.orchestra.run.smithi104.stderr:set 2.a pg_upmap_items mapping to [10->8,1->2] 2020-09-08T15:41:25.623 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:41:25.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:25 smithi166 bash[9446]: cluster 2020-09-08T15:41:23.650494+0000 mgr.x (mgr.24118) 419 : cluster [DBG] pgmap v541: 17 pgs: 17 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T15:41:25.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:25 smithi166 bash[9446]: audit 2020-09-08T15:41:24.626365+0000 mon.c (mon.2) 43 : audit [INF] from='client.? v1:172.21.15.104:0/2873220198' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:41:25.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:25 smithi166 bash[9446]: audit 2020-09-08T15:41:24.629329+0000 mon.a (mon.0) 1076 : audit [INF] from='client.34982 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:41:25.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:25 smithi166 bash[9446]: audit 2020-09-08T15:41:24.657496+0000 mon.a (mon.0) 1077 : audit [INF] from='client.? v1:172.21.15.104:0/1568343815' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "2.a", "id": [10, 8, 1, 2]}]: dispatch 2020-09-08T15:41:25.867 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:41:25 smithi166 bash[14254]: debug 2020-09-08T15:41:25.508+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:25.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:25 smithi055 bash[9296]: cluster 2020-09-08T15:41:23.650494+0000 mgr.x (mgr.24118) 419 : cluster [DBG] pgmap v541: 17 pgs: 17 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T15:41:25.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:25 smithi055 bash[9296]: audit 2020-09-08T15:41:24.626365+0000 mon.c (mon.2) 43 : audit [INF] from='client.? v1:172.21.15.104:0/2873220198' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:41:25.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:25 smithi055 bash[9296]: audit 2020-09-08T15:41:24.629329+0000 mon.a (mon.0) 1076 : audit [INF] from='client.34982 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:41:25.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:25 smithi055 bash[9296]: audit 2020-09-08T15:41:24.657496+0000 mon.a (mon.0) 1077 : audit [INF] from='client.? v1:172.21.15.104:0/1568343815' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "2.a", "id": [10, 8, 1, 2]}]: dispatch 2020-09-08T15:41:25.919 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:41:25.920 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:25 smithi104 bash[9203]: audit 2020-09-08T15:41:24.629329+0000 mon.a (mon.0) 1076 : audit [INF] from='client.34982 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:41:25.920 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:25 smithi104 bash[9203]: audit 2020-09-08T15:41:24.657496+0000 mon.a (mon.0) 1077 : audit [INF] from='client.? v1:172.21.15.104:0/1568343815' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "2.a", "id": [10, 8, 1, 2]}]: dispatch 2020-09-08T15:41:26.025 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:26.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:41:25 smithi104 bash[22305]: debug 2020-09-08T15:41:25.921+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:26.435 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:26.539 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:26.558 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_historic_ops 2020-09-08T15:41:26.726 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:41:26.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:26 smithi166 bash[9446]: cluster 2020-09-08T15:41:25.612576+0000 mon.a (mon.0) 1078 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:41:26.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:26 smithi166 bash[9446]: audit 2020-09-08T15:41:25.617146+0000 mon.a (mon.0) 1079 : audit [INF] from='client.34982 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:41:26.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:26 smithi166 bash[9446]: audit 2020-09-08T15:41:25.617202+0000 mon.a (mon.0) 1080 : audit [INF] from='client.? v1:172.21.15.104:0/1568343815' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "2.a", "id": [10, 8, 1, 2]}]': finished 2020-09-08T15:41:26.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:26 smithi166 bash[9446]: cluster 2020-09-08T15:41:25.617254+0000 mon.a (mon.0) 1081 : cluster [DBG] osdmap e204: 12 total, 12 up, 12 in 2020-09-08T15:41:26.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:41:26 smithi166 bash[18960]: debug 2020-09-08T15:41:26.432+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:26.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:26 smithi055 bash[9296]: cluster 2020-09-08T15:41:25.612576+0000 mon.a (mon.0) 1078 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:41:26.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:26 smithi055 bash[9296]: audit 2020-09-08T15:41:25.617146+0000 mon.a (mon.0) 1079 : audit [INF] from='client.34982 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:41:26.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:26 smithi055 bash[9296]: audit 2020-09-08T15:41:25.617202+0000 mon.a (mon.0) 1080 : audit [INF] from='client.? v1:172.21.15.104:0/1568343815' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "2.a", "id": [10, 8, 1, 2]}]': finished 2020-09-08T15:41:26.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:26 smithi055 bash[9296]: cluster 2020-09-08T15:41:25.617254+0000 mon.a (mon.0) 1081 : cluster [DBG] osdmap e204: 12 total, 12 up, 12 in 2020-09-08T15:41:26.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:26 smithi104 bash[9203]: cluster 2020-09-08T15:41:25.612576+0000 mon.a (mon.0) 1078 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:41:26.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:26 smithi104 bash[9203]: audit 2020-09-08T15:41:25.617146+0000 mon.a (mon.0) 1079 : audit [INF] from='client.34982 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:41:26.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:26 smithi104 bash[9203]: audit 2020-09-08T15:41:25.617202+0000 mon.a (mon.0) 1080 : audit [INF] from='client.? v1:172.21.15.104:0/1568343815' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "2.a", "id": [10, 8, 1, 2]}]': finished 2020-09-08T15:41:26.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:26 smithi104 bash[9203]: cluster 2020-09-08T15:41:25.617254+0000 mon.a (mon.0) 1081 : cluster [DBG] osdmap e204: 12 total, 12 up, 12 in 2020-09-08T15:41:27.087 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:27.190 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:41:27.372 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:41:27 smithi104 bash[19922]: debug 2020-09-08T15:41:27.089+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:27.375 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:41:27.494 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:27.623 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:41:27 smithi104 bash[22305]: debug 2020-09-08T15:41:27.373+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:27.623 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:27 smithi104 bash[9203]: cluster 2020-09-08T15:41:25.650934+0000 mgr.x (mgr.24118) 420 : cluster [DBG] pgmap v543: 17 pgs: 17 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 704 op/s 2020-09-08T15:41:27.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:27 smithi166 bash[9446]: cluster 2020-09-08T15:41:25.650934+0000 mgr.x (mgr.24118) 420 : cluster [DBG] pgmap v543: 17 pgs: 17 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 704 op/s 2020-09-08T15:41:27.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:27 smithi055 bash[9296]: cluster 2020-09-08T15:41:25.650934+0000 mgr.x (mgr.24118) 420 : cluster [DBG] pgmap v543: 17 pgs: 17 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 704 op/s 2020-09-08T15:41:28.041 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:28.154 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:28.294 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:41:28 smithi166 bash[18960]: debug 2020-09-08T15:41:28.040+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:28.487 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:28.590 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:28.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:28 smithi166 bash[9446]: cluster 2020-09-08T15:41:27.381674+0000 osd.8 (osd.8) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:28.872 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:28 smithi166 bash[9446]: cluster 2020-09-08T15:41:27.381764+0000 osd.8 (osd.8) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:28.872 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:28 smithi166 bash[9446]: cluster 2020-09-08T15:41:27.638843+0000 mon.a (mon.0) 1082 : cluster [DBG] osdmap e205: 12 total, 12 up, 12 in 2020-09-08T15:41:28.872 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:28 smithi166 bash[9446]: audit 2020-09-08T15:41:27.649806+0000 mon.a (mon.0) 1083 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:41:28.872 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:28 smithi166 bash[9446]: audit 2020-09-08T15:41:28.040928+0000 mon.a (mon.0) 1084 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:41:28.035442\"}"}]: dispatch 2020-09-08T15:41:28.873 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:28 smithi166 bash[9446]: audit 2020-09-08T15:41:28.044217+0000 mon.a (mon.0) 1085 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:41:28.035442\"}"}]': finished 2020-09-08T15:41:28.873 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:28 smithi166 bash[9446]: audit 2020-09-08T15:41:28.048794+0000 mon.a (mon.0) 1086 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:41:28.046885\"}"}]: dispatch 2020-09-08T15:41:28.873 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:28 smithi166 bash[9446]: audit 2020-09-08T15:41:28.053893+0000 mon.a (mon.0) 1087 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:41:28.046885\"}"}]': finished 2020-09-08T15:41:28.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:28 smithi055 bash[9296]: cluster 2020-09-08T15:41:27.381674+0000 osd.8 (osd.8) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:28.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:28 smithi055 bash[9296]: cluster 2020-09-08T15:41:27.381764+0000 osd.8 (osd.8) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:28.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:28 smithi055 bash[9296]: cluster 2020-09-08T15:41:27.638843+0000 mon.a (mon.0) 1082 : cluster [DBG] osdmap e205: 12 total, 12 up, 12 in 2020-09-08T15:41:28.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:28 smithi055 bash[9296]: audit 2020-09-08T15:41:27.649806+0000 mon.a (mon.0) 1083 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:41:28.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:28 smithi055 bash[9296]: audit 2020-09-08T15:41:28.040928+0000 mon.a (mon.0) 1084 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:41:28.035442\"}"}]: dispatch 2020-09-08T15:41:28.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:28 smithi055 bash[9296]: audit 2020-09-08T15:41:28.044217+0000 mon.a (mon.0) 1085 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:41:28.035442\"}"}]': finished 2020-09-08T15:41:28.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:28 smithi055 bash[9296]: audit 2020-09-08T15:41:28.048794+0000 mon.a (mon.0) 1086 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:41:28.046885\"}"}]: dispatch 2020-09-08T15:41:28.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:28 smithi055 bash[9296]: audit 2020-09-08T15:41:28.053893+0000 mon.a (mon.0) 1087 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:41:28.046885\"}"}]': finished 2020-09-08T15:41:28.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:28 smithi104 bash[9203]: cluster 2020-09-08T15:41:27.381674+0000 osd.8 (osd.8) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:28.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:28 smithi104 bash[9203]: cluster 2020-09-08T15:41:27.381764+0000 osd.8 (osd.8) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:28.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:28 smithi104 bash[9203]: cluster 2020-09-08T15:41:27.638843+0000 mon.a (mon.0) 1082 : cluster [DBG] osdmap e205: 12 total, 12 up, 12 in 2020-09-08T15:41:28.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:28 smithi104 bash[9203]: audit 2020-09-08T15:41:27.649806+0000 mon.a (mon.0) 1083 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:41:28.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:28 smithi104 bash[9203]: audit 2020-09-08T15:41:28.040928+0000 mon.a (mon.0) 1084 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:41:28.035442\"}"}]: dispatch 2020-09-08T15:41:28.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:28 smithi104 bash[9203]: audit 2020-09-08T15:41:28.044217+0000 mon.a (mon.0) 1085 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:41:28.035442\"}"}]': finished 2020-09-08T15:41:28.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:28 smithi104 bash[9203]: audit 2020-09-08T15:41:28.048794+0000 mon.a (mon.0) 1086 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:41:28.046885\"}"}]: dispatch 2020-09-08T15:41:28.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:28 smithi104 bash[9203]: audit 2020-09-08T15:41:28.053893+0000 mon.a (mon.0) 1087 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:41:28.046885\"}"}]': finished 2020-09-08T15:41:28.929 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:41:28 smithi104 bash[15273]: debug 2020-09-08T15:41:28.489+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:29.040 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:29.141 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:29.265 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:29.368 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:41:29.375 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:41:29 smithi104 bash[15273]: debug 2020-09-08T15:41:29.037+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:29.405 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:41:29.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:41:29 smithi055 bash[16646]: debug 2020-09-08T15:41:29.266+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:29.570 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:41:29.748 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:41:29.852 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:29.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:29 smithi055 bash[9296]: cluster 2020-09-08T15:41:27.651382+0000 mgr.x (mgr.24118) 421 : cluster [DBG] pgmap v545: 17 pgs: 17 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.38k op/s 2020-09-08T15:41:29.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:29 smithi055 bash[9296]: audit 2020-09-08T15:41:28.870776+0000 mon.a (mon.0) 1088 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868402\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868897\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869135\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869362\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:41:28.869649\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:36:29.049062\"}"}]: dispatch 2020-09-08T15:41:29.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:29 smithi055 bash[9296]: audit 2020-09-08T15:41:28.875530+0000 mon.a (mon.0) 1089 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868402\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868897\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869135\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869362\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:41:28.869649\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:36:29.049062\"}"}]': finished 2020-09-08T15:41:29.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:29 smithi055 bash[9296]: audit 2020-09-08T15:41:28.878247+0000 mon.a (mon.0) 1090 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:41:29.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:29 smithi055 bash[9296]: audit 2020-09-08T15:41:28.882019+0000 mon.a (mon.0) 1091 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:41:29.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:29 smithi104 bash[9203]: cluster 2020-09-08T15:41:27.651382+0000 mgr.x (mgr.24118) 421 : cluster [DBG] pgmap v545: 17 pgs: 17 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.38k op/s 2020-09-08T15:41:29.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:29 smithi104 bash[9203]: audit 2020-09-08T15:41:28.870776+0000 mon.a (mon.0) 1088 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868402\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868897\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869135\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869362\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:41:28.869649\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:36:29.049062\"}"}]: dispatch 2020-09-08T15:41:29.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:29 smithi104 bash[9203]: audit 2020-09-08T15:41:28.875530+0000 mon.a (mon.0) 1089 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868402\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868897\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869135\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869362\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:41:28.869649\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:36:29.049062\"}"}]': finished 2020-09-08T15:41:29.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:29 smithi104 bash[9203]: audit 2020-09-08T15:41:28.878247+0000 mon.a (mon.0) 1090 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:41:29.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:29 smithi104 bash[9203]: audit 2020-09-08T15:41:28.882019+0000 mon.a (mon.0) 1091 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:41:29.930 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:41:29 smithi104 bash[17588]: debug 2020-09-08T15:41:29.741+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:29.965 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:30.067 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:30.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:29 smithi166 bash[9446]: cluster 2020-09-08T15:41:27.651382+0000 mgr.x (mgr.24118) 421 : cluster [DBG] pgmap v545: 17 pgs: 17 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.38k op/s 2020-09-08T15:41:30.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:29 smithi166 bash[9446]: audit 2020-09-08T15:41:28.870776+0000 mon.a (mon.0) 1088 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868402\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868897\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869135\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869362\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:41:28.869649\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:36:29.049062\"}"}]: dispatch 2020-09-08T15:41:30.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:29 smithi166 bash[9446]: audit 2020-09-08T15:41:28.875530+0000 mon.a (mon.0) 1089 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868402\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868897\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869135\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869362\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:41:28.869649\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:36:29.049062\"}"}]': finished 2020-09-08T15:41:30.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:29 smithi166 bash[9446]: audit 2020-09-08T15:41:28.878247+0000 mon.a (mon.0) 1090 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:41:30.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:29 smithi166 bash[9446]: audit 2020-09-08T15:41:28.882019+0000 mon.a (mon.0) 1091 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:41:30.283 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:30.388 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:30.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:41:29 smithi055 bash[11956]: debug 2020-09-08T15:41:29.958+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:30.430 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:41:30 smithi104 bash[19922]: debug 2020-09-08T15:41:30.277+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:30.614 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:30.717 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:41:30.810 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:41:30 smithi166 bash[16593]: debug 2020-09-08T15:41:30.612+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:31.015 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:41:31.015 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:41:31 smithi104 bash[17588]: debug 2020-09-08T15:41:31.013+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:31.092 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:41:31.092 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:41:31.118 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:41:31.257 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:41:31.317 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:41:31.354 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:41:31.363 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:31.387 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:41:31.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:41:31 smithi055 bash[19003]: debug 2020-09-08T15:41:31.258+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:31.424 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:41:31.456 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:41:31.498 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:41:31.537 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:41:31.567 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:41:31.605 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:41:31.635 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:41:31.676 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:41:31.723 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:41:31.723 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:41:31.723 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:41:31.724 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:41:31.724 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:41:31.724 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:41:31.724 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:41:31.725 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:41:31.725 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:41:31.725 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:41:31.726 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:41:31.726 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:41:31.726 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:41:31.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:31 smithi055 bash[9296]: cluster 2020-09-08T15:41:29.651863+0000 mgr.x (mgr.24118) 422 : cluster [DBG] pgmap v546: 17 pgs: 17 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 942 op/s 2020-09-08T15:41:31.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:31 smithi055 bash[9296]: audit 2020-09-08T15:41:31.288457+0000 mon.b (mon.1) 81 : audit [DBG] from='client.? v1:172.21.15.104:0/2414235332' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:41:31.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:31 smithi104 bash[9203]: cluster 2020-09-08T15:41:29.651863+0000 mgr.x (mgr.24118) 422 : cluster [DBG] pgmap v546: 17 pgs: 17 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 942 op/s 2020-09-08T15:41:31.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:31 smithi104 bash[9203]: audit 2020-09-08T15:41:31.288457+0000 mon.b (mon.1) 81 : audit [DBG] from='client.? v1:172.21.15.104:0/2414235332' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:41:32.008 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:31 smithi166 bash[9446]: cluster 2020-09-08T15:41:29.651863+0000 mgr.x (mgr.24118) 422 : cluster [DBG] pgmap v546: 17 pgs: 17 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 942 op/s 2020-09-08T15:41:32.009 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:31 smithi166 bash[9446]: audit 2020-09-08T15:41:31.288457+0000 mon.b (mon.1) 81 : audit [DBG] from='client.? v1:172.21.15.104:0/2414235332' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:41:32.010 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:32.119 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:41:32.128 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2020-09-08T15:41:32.306 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:41:32.307 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:41:32 smithi166 bash[11928]: debug 2020-09-08T15:41:32.304+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:32.307 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:41:32 smithi166 bash[14254]: debug 2020-09-08T15:41:32.012+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:32.326 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2020-09-08T15:41:32.326 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-09-08T15:41:32.327 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd dump -f json-pretty 2020-09-08T15:41:32.334 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:41:32.408 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:41:32.494 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:41:32.544 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:41:32.648 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:32.766 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:41:32 smithi055 bash[19003]: debug 2020-09-08T15:41:32.546+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:32.766 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:41:32 smithi055 bash[14301]: debug 2020-09-08T15:41:32.770+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:32.767 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:32.874 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:32.984 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:33.087 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:41:33.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:41:32 smithi055 bash[14301]: debug 2020-09-08T15:41:32.986+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:33.515 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:41:33.619 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:41:33.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:33 smithi166 bash[9446]: cluster 2020-09-08T15:41:31.652338+0000 mgr.x (mgr.24118) 423 : cluster [DBG] pgmap v547: 17 pgs: 17 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 626 op/s 2020-09-08T15:41:33.868 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:41:33 smithi166 bash[11928]: debug 2020-09-08T15:41:33.512+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:33.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:33 smithi055 bash[9296]: cluster 2020-09-08T15:41:31.652338+0000 mgr.x (mgr.24118) 423 : cluster [DBG] pgmap v547: 17 pgs: 17 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 626 op/s 2020-09-08T15:41:33.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:33 smithi104 bash[9203]: cluster 2020-09-08T15:41:31.652338+0000 mgr.x (mgr.24118) 423 : cluster [DBG] pgmap v547: 17 pgs: 17 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 626 op/s 2020-09-08T15:41:34.216 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:41:34.317 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:34.510 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:41:34 smithi104 bash[22305]: debug 2020-09-08T15:41:34.209+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:34.986 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:35.089 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:35.199 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:35.305 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:35.312 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:41:35 smithi166 bash[14254]: debug 2020-09-08T15:41:35.200+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:35.312 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:41:34 smithi166 bash[18960]: debug 2020-09-08T15:41:34.988+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:35.492 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:41:35.608 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:35 smithi166 bash[9446]: cluster 2020-09-08T15:41:33.652814+0000 mgr.x (mgr.24118) 424 : cluster [DBG] pgmap v548: 17 pgs: 17 active+clean; 805 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.16k op/s 2020-09-08T15:41:35.609 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:35 smithi166 bash[9446]: cluster 2020-09-08T15:41:34.305190+0000 osd.1 (osd.1) 40 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:35.609 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:35 smithi166 bash[9446]: cluster 2020-09-08T15:41:34.305295+0000 osd.1 (osd.1) 41 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:35.609 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:35 smithi166 bash[9446]: cluster 2020-09-08T15:41:34.713562+0000 mon.a (mon.0) 1092 : cluster [DBG] osdmap e206: 12 total, 12 up, 12 in 2020-09-08T15:41:35.610 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:35 smithi166 bash[9446]: audit 2020-09-08T15:41:35.331613+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? v1:172.21.15.104:0/2065512358' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:41:35.657 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:41:35.716 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:41:35.800 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:35 smithi104 bash[9203]: cluster 2020-09-08T15:41:33.652814+0000 mgr.x (mgr.24118) 424 : cluster [DBG] pgmap v548: 17 pgs: 17 active+clean; 805 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.16k op/s 2020-09-08T15:41:35.800 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:35 smithi104 bash[9203]: cluster 2020-09-08T15:41:34.305190+0000 osd.1 (osd.1) 40 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:35.800 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:35 smithi104 bash[9203]: cluster 2020-09-08T15:41:34.305295+0000 osd.1 (osd.1) 41 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:35.800 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:35 smithi104 bash[9203]: cluster 2020-09-08T15:41:34.713562+0000 mon.a (mon.0) 1092 : cluster [DBG] osdmap e206: 12 total, 12 up, 12 in 2020-09-08T15:41:35.801 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:35 smithi104 bash[9203]: audit 2020-09-08T15:41:35.331613+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? v1:172.21.15.104:0/2065512358' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:41:35.803 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:35.865 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:41:35 smithi166 bash[18960]: debug 2020-09-08T15:41:35.804+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:35.911 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:41:35.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:35 smithi055 bash[9296]: cluster 2020-09-08T15:41:33.652814+0000 mgr.x (mgr.24118) 424 : cluster [DBG] pgmap v548: 17 pgs: 17 active+clean; 805 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.16k op/s 2020-09-08T15:41:35.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:35 smithi055 bash[9296]: cluster 2020-09-08T15:41:34.305190+0000 osd.1 (osd.1) 40 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:35.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:35 smithi055 bash[9296]: cluster 2020-09-08T15:41:34.305295+0000 osd.1 (osd.1) 41 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:35.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:35 smithi055 bash[9296]: cluster 2020-09-08T15:41:34.713562+0000 mon.a (mon.0) 1092 : cluster [DBG] osdmap e206: 12 total, 12 up, 12 in 2020-09-08T15:41:35.914 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:35 smithi055 bash[9296]: audit 2020-09-08T15:41:35.331613+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? v1:172.21.15.104:0/2065512358' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:41:36.439 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:41:36.542 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:36.682 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:36.689 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:41:36 smithi166 bash[11928]: debug 2020-09-08T15:41:36.440+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:36.689 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:36 smithi166 bash[9446]: cluster 2020-09-08T15:41:35.710665+0000 mon.a (mon.0) 1094 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:41:36.689 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:36 smithi166 bash[9446]: audit 2020-09-08T15:41:35.715645+0000 mon.a (mon.0) 1095 : audit [INF] from='client.? v1:172.21.15.104:0/2065512358' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:41:36.690 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:36 smithi166 bash[9446]: cluster 2020-09-08T15:41:35.715801+0000 mon.a (mon.0) 1096 : cluster [DBG] osdmap e207: 12 total, 12 up, 12 in 2020-09-08T15:41:36.690 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:41:36 smithi166 bash[16593]: debug 2020-09-08T15:41:36.684+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:36.794 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:36.915 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:36 smithi055 bash[9296]: cluster 2020-09-08T15:41:35.710665+0000 mon.a (mon.0) 1094 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:41:36.915 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:36 smithi055 bash[9296]: audit 2020-09-08T15:41:35.715645+0000 mon.a (mon.0) 1095 : audit [INF] from='client.? v1:172.21.15.104:0/2065512358' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:41:36.915 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:36 smithi055 bash[9296]: cluster 2020-09-08T15:41:35.715801+0000 mon.a (mon.0) 1096 : cluster [DBG] osdmap e207: 12 total, 12 up, 12 in 2020-09-08T15:41:36.933 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:36 smithi104 bash[9203]: cluster 2020-09-08T15:41:35.710665+0000 mon.a (mon.0) 1094 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:41:36.933 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:36 smithi104 bash[9203]: audit 2020-09-08T15:41:35.715645+0000 mon.a (mon.0) 1095 : audit [INF] from='client.? v1:172.21.15.104:0/2065512358' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:41:36.933 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:36 smithi104 bash[9203]: cluster 2020-09-08T15:41:35.715801+0000 mon.a (mon.0) 1096 : cluster [DBG] osdmap e207: 12 total, 12 up, 12 in 2020-09-08T15:41:37.038 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:41:37.039 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T15:41:37.039 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.039 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.34742.0:287548 2.7 e207/126 2:e55d3d8f:::benchmark_data_smithi166_20551_object139928:head v 207'18039, mlcod=207'18037)", 2020-09-08T15:41:37.041 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:41:37.033390+0000", 2020-09-08T15:41:37.041 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 0.010218830999999999, 2020-09-08T15:41:37.042 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.010244870999999999, 2020-09-08T15:41:37.042 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:37.042 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "started", 2020-09-08T15:41:37.042 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:37.042 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.043 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:37.043 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.033390+0000", 2020-09-08T15:41:37.043 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:37.043 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.043 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.044 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:37.044 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.033390+0000", 2020-09-08T15:41:37.044 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.3424999999999997e-05 2020-09-08T15:41:37.044 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.044 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.045 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:37.045 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.033454+0000", 2020-09-08T15:41:37.045 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.8649999999999996e-06 2020-09-08T15:41:37.045 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.045 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.046 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:37.046 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.033464+0000", 2020-09-08T15:41:37.046 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3552e-05 2020-09-08T15:41:37.048 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.048 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.049 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:37.049 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.033477+0000", 2020-09-08T15:41:37.049 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6579999999999996e-06 2020-09-08T15:41:37.049 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.049 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.050 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:37.050 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.033484+0000", 2020-09-08T15:41:37.050 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.4422000000000002e-05 2020-09-08T15:41:37.053 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.053 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.054 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:37.054 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.033538+0000", 2020-09-08T15:41:37.054 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.7651999999999999e-05 2020-09-08T15:41:37.055 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.055 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.055 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:37.055 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.033566+0000", 2020-09-08T15:41:37.056 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017557400000000001 2020-09-08T15:41:37.056 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:37.056 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:37.056 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:37.056 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.057 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.057 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.34742.0:287549 2.7 e207/126 2:eaf9e08e:::benchmark_data_smithi166_20551_object139929:head v 207'18040, mlcod=207'18037)", 2020-09-08T15:41:37.057 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:41:37.033994+0000", 2020-09-08T15:41:37.057 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 0.0096154650000000001, 2020-09-08T15:41:37.057 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0096900030000000009, 2020-09-08T15:41:37.058 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:37.058 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "started", 2020-09-08T15:41:37.058 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:37.059 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.059 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:37.059 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.033994+0000", 2020-09-08T15:41:37.059 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:37.060 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.060 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.060 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:37.061 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.033994+0000", 2020-09-08T15:41:37.061 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.3793000000000004e-05 2020-09-08T15:41:37.062 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.062 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.062 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:37.062 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.034057+0000", 2020-09-08T15:41:37.062 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.695e-06 2020-09-08T15:41:37.063 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.063 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.063 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:37.063 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.034065+0000", 2020-09-08T15:41:37.063 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5509e-05 2020-09-08T15:41:37.063 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.064 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.064 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:37.064 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.034081+0000", 2020-09-08T15:41:37.064 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.135e-06 2020-09-08T15:41:37.064 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.065 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.065 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:37.065 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.034086+0000", 2020-09-08T15:41:37.065 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0057185730000000002 2020-09-08T15:41:37.065 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.066 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.066 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:37.066 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.039804+0000", 2020-09-08T15:41:37.066 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.0034000000000001e-05 2020-09-08T15:41:37.067 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.067 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.067 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:37.067 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.039834+0000", 2020-09-08T15:41:37.067 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0058407390000000002 2020-09-08T15:41:37.068 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:37.068 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:37.068 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:37.068 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.068 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.069 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.34742.0:287550 2.7 e207/126)", 2020-09-08T15:41:37.069 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:41:37.038659+0000", 2020-09-08T15:41:37.070 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 0.0049499449999999999, 2020-09-08T15:41:37.070 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0050650440000000003, 2020-09-08T15:41:37.070 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:37.070 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "queued for pg", 2020-09-08T15:41:37.071 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:37.071 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.071 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:37.072 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.038659+0000", 2020-09-08T15:41:37.072 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:37.072 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.072 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.073 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:37.073 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.038659+0000", 2020-09-08T15:41:37.073 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.7046999999999995e-05 2020-09-08T15:41:37.074 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.074 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.074 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:37.074 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.038746+0000", 2020-09-08T15:41:37.074 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2455e-05 2020-09-08T15:41:37.075 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.075 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.075 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:37.075 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.038759+0000", 2020-09-08T15:41:37.075 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.0632000000000002e-05 2020-09-08T15:41:37.076 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.076 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.076 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:37.076 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.038779+0000", 2020-09-08T15:41:37.077 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.9869999999999994e-06 2020-09-08T15:41:37.077 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:37.077 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:37.077 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:37.077 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:41:37.038789+0000", 2020-09-08T15:41:37.078 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000130121 2020-09-08T15:41:37.078 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:37.078 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:37.078 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:37.079 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:37.079 INFO:teuthology.orchestra.run.smithi166.stdout: ], 2020-09-08T15:41:37.079 INFO:teuthology.orchestra.run.smithi166.stdout: "num_ops": 3 2020-09-08T15:41:37.080 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:41:37.154 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:37.256 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:37.392 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:37.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:41:37 smithi104 bash[19922]: debug 2020-09-08T15:41:37.148+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:37.495 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:37.596 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:41:37 smithi055 bash[16646]: debug 2020-09-08T15:41:37.393+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:37.596 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:41:37 smithi055 bash[16646]: debug 2020-09-08T15:41:37.597+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:37.597 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:37.703 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:37.818 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:37.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:37 smithi166 bash[9446]: cluster 2020-09-08T15:41:35.653204+0000 mgr.x (mgr.24118) 425 : cluster [DBG] pgmap v550: 17 pgs: 17 active+clean; 766 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 813 op/s 2020-09-08T15:41:37.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:37 smithi055 bash[9296]: cluster 2020-09-08T15:41:35.653204+0000 mgr.x (mgr.24118) 425 : cluster [DBG] pgmap v550: 17 pgs: 17 active+clean; 766 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 813 op/s 2020-09-08T15:41:37.908 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:41:37 smithi055 bash[16646]: debug 2020-09-08T15:41:37.821+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:37.924 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:37.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:37 smithi104 bash[9203]: cluster 2020-09-08T15:41:35.653204+0000 mgr.x (mgr.24118) 425 : cluster [DBG] pgmap v550: 17 pgs: 17 active+clean; 766 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 813 op/s 2020-09-08T15:41:38.064 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:38.090 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2020-09-08T15:41:38.168 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:38.173 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T15:41:38.173 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:41:38.173 INFO:teuthology.orchestra.run.smithi104.stdout: "epoch": 207, 2020-09-08T15:41:38.174 INFO:teuthology.orchestra.run.smithi104.stdout: "fsid": "759b27b6-f1e7-11ea-a080-001a4aab830c", 2020-09-08T15:41:38.174 INFO:teuthology.orchestra.run.smithi104.stdout: "created": "2020-09-08T15:25:41.788365+0000", 2020-09-08T15:41:38.174 INFO:teuthology.orchestra.run.smithi104.stdout: "modified": "2020-09-08T15:41:35.710453+0000", 2020-09-08T15:41:38.174 INFO:teuthology.orchestra.run.smithi104.stdout: "last_up_change": "2020-09-08T15:34:41.591462+0000", 2020-09-08T15:41:38.174 INFO:teuthology.orchestra.run.smithi104.stdout: "last_in_change": "2020-09-08T15:34:41.591462+0000", 2020-09-08T15:41:38.175 INFO:teuthology.orchestra.run.smithi104.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-09-08T15:41:38.175 INFO:teuthology.orchestra.run.smithi104.stdout: "flags_num": 5806080, 2020-09-08T15:41:38.175 INFO:teuthology.orchestra.run.smithi104.stdout: "flags_set": [ 2020-09-08T15:41:38.175 INFO:teuthology.orchestra.run.smithi104.stdout: "nodeep-scrub", 2020-09-08T15:41:38.175 INFO:teuthology.orchestra.run.smithi104.stdout: "noscrub", 2020-09-08T15:41:38.176 INFO:teuthology.orchestra.run.smithi104.stdout: "pglog_hardlimit", 2020-09-08T15:41:38.176 INFO:teuthology.orchestra.run.smithi104.stdout: "purged_snapdirs", 2020-09-08T15:41:38.176 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_deletes", 2020-09-08T15:41:38.176 INFO:teuthology.orchestra.run.smithi104.stdout: "sortbitwise" 2020-09-08T15:41:38.176 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T15:41:38.177 INFO:teuthology.orchestra.run.smithi104.stdout: "crush_version": 27, 2020-09-08T15:41:38.177 INFO:teuthology.orchestra.run.smithi104.stdout: "full_ratio": 0.94999998807907104, 2020-09-08T15:41:38.177 INFO:teuthology.orchestra.run.smithi104.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-09-08T15:41:38.177 INFO:teuthology.orchestra.run.smithi104.stdout: "nearfull_ratio": 0.85000002384185791, 2020-09-08T15:41:38.177 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_snapshot": "", 2020-09-08T15:41:38.178 INFO:teuthology.orchestra.run.smithi104.stdout: "pool_max": 2, 2020-09-08T15:41:38.178 INFO:teuthology.orchestra.run.smithi104.stdout: "max_osd": 12, 2020-09-08T15:41:38.178 INFO:teuthology.orchestra.run.smithi104.stdout: "require_min_compat_client": "luminous", 2020-09-08T15:41:38.178 INFO:teuthology.orchestra.run.smithi104.stdout: "min_compat_client": "jewel", 2020-09-08T15:41:38.178 INFO:teuthology.orchestra.run.smithi104.stdout: "require_osd_release": "pacific", 2020-09-08T15:41:38.179 INFO:teuthology.orchestra.run.smithi104.stdout: "pools": [ 2020-09-08T15:41:38.179 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.179 INFO:teuthology.orchestra.run.smithi104.stdout: "pool": 1, 2020-09-08T15:41:38.179 INFO:teuthology.orchestra.run.smithi104.stdout: "pool_name": "device_health_metrics", 2020-09-08T15:41:38.179 INFO:teuthology.orchestra.run.smithi104.stdout: "create_time": "2020-09-08T15:25:56.314385+0000", 2020-09-08T15:41:38.180 INFO:teuthology.orchestra.run.smithi104.stdout: "flags": 1, 2020-09-08T15:41:38.180 INFO:teuthology.orchestra.run.smithi104.stdout: "flags_names": "hashpspool", 2020-09-08T15:41:38.180 INFO:teuthology.orchestra.run.smithi104.stdout: "type": 1, 2020-09-08T15:41:38.180 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 2, 2020-09-08T15:41:38.180 INFO:teuthology.orchestra.run.smithi104.stdout: "min_size": 2, 2020-09-08T15:41:38.181 INFO:teuthology.orchestra.run.smithi104.stdout: "crush_rule": 0, 2020-09-08T15:41:38.181 INFO:teuthology.orchestra.run.smithi104.stdout: "object_hash": 2, 2020-09-08T15:41:38.181 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_autoscale_mode": "off", 2020-09-08T15:41:38.182 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num": 1, 2020-09-08T15:41:38.182 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_placement_num": 1, 2020-09-08T15:41:38.182 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_placement_num_target": 1, 2020-09-08T15:41:38.182 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num_target": 1, 2020-09-08T15:41:38.183 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num_pending": 1, 2020-09-08T15:41:38.183 INFO:teuthology.orchestra.run.smithi104.stdout: "last_pg_merge_meta": { 2020-09-08T15:41:38.183 INFO:teuthology.orchestra.run.smithi104.stdout: "source_pgid": "0.0", 2020-09-08T15:41:38.183 INFO:teuthology.orchestra.run.smithi104.stdout: "ready_epoch": 0, 2020-09-08T15:41:38.183 INFO:teuthology.orchestra.run.smithi104.stdout: "last_epoch_started": 0, 2020-09-08T15:41:38.184 INFO:teuthology.orchestra.run.smithi104.stdout: "last_epoch_clean": 0, 2020-09-08T15:41:38.184 INFO:teuthology.orchestra.run.smithi104.stdout: "source_version": "0'0", 2020-09-08T15:41:38.184 INFO:teuthology.orchestra.run.smithi104.stdout: "target_version": "0'0" 2020-09-08T15:41:38.184 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.185 INFO:teuthology.orchestra.run.smithi104.stdout: "last_change": "13", 2020-09-08T15:41:38.185 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend": "0", 2020-09-08T15:41:38.185 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend_prenautilus": "0", 2020-09-08T15:41:38.185 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend_preluminous": "0", 2020-09-08T15:41:38.185 INFO:teuthology.orchestra.run.smithi104.stdout: "auid": 0, 2020-09-08T15:41:38.186 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_mode": "selfmanaged", 2020-09-08T15:41:38.186 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_seq": 0, 2020-09-08T15:41:38.186 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_epoch": 0, 2020-09-08T15:41:38.186 INFO:teuthology.orchestra.run.smithi104.stdout: "pool_snaps": [], 2020-09-08T15:41:38.186 INFO:teuthology.orchestra.run.smithi104.stdout: "removed_snaps": "[]", 2020-09-08T15:41:38.187 INFO:teuthology.orchestra.run.smithi104.stdout: "quota_max_bytes": 0, 2020-09-08T15:41:38.187 INFO:teuthology.orchestra.run.smithi104.stdout: "quota_max_objects": 0, 2020-09-08T15:41:38.187 INFO:teuthology.orchestra.run.smithi104.stdout: "tiers": [], 2020-09-08T15:41:38.187 INFO:teuthology.orchestra.run.smithi104.stdout: "tier_of": -1, 2020-09-08T15:41:38.187 INFO:teuthology.orchestra.run.smithi104.stdout: "read_tier": -1, 2020-09-08T15:41:38.188 INFO:teuthology.orchestra.run.smithi104.stdout: "write_tier": -1, 2020-09-08T15:41:38.188 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_mode": "none", 2020-09-08T15:41:38.188 INFO:teuthology.orchestra.run.smithi104.stdout: "target_max_bytes": 0, 2020-09-08T15:41:38.188 INFO:teuthology.orchestra.run.smithi104.stdout: "target_max_objects": 0, 2020-09-08T15:41:38.188 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-09-08T15:41:38.189 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-09-08T15:41:38.189 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_full_ratio_micro": 800000, 2020-09-08T15:41:38.189 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_min_flush_age": 0, 2020-09-08T15:41:38.189 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_min_evict_age": 0, 2020-09-08T15:41:38.189 INFO:teuthology.orchestra.run.smithi104.stdout: "erasure_code_profile": "", 2020-09-08T15:41:38.190 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_params": { 2020-09-08T15:41:38.191 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "none" 2020-09-08T15:41:38.191 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.192 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_period": 0, 2020-09-08T15:41:38.192 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_count": 0, 2020-09-08T15:41:38.192 INFO:teuthology.orchestra.run.smithi104.stdout: "use_gmt_hitset": true, 2020-09-08T15:41:38.192 INFO:teuthology.orchestra.run.smithi104.stdout: "min_read_recency_for_promote": 0, 2020-09-08T15:41:38.192 INFO:teuthology.orchestra.run.smithi104.stdout: "min_write_recency_for_promote": 0, 2020-09-08T15:41:38.193 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_grade_decay_rate": 0, 2020-09-08T15:41:38.193 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_search_last_n": 0, 2020-09-08T15:41:38.193 INFO:teuthology.orchestra.run.smithi104.stdout: "grade_table": [], 2020-09-08T15:41:38.193 INFO:teuthology.orchestra.run.smithi104.stdout: "stripe_width": 0, 2020-09-08T15:41:38.193 INFO:teuthology.orchestra.run.smithi104.stdout: "expected_num_objects": 0, 2020-09-08T15:41:38.194 INFO:teuthology.orchestra.run.smithi104.stdout: "fast_read": false, 2020-09-08T15:41:38.194 INFO:teuthology.orchestra.run.smithi104.stdout: "options": { 2020-09-08T15:41:38.194 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num_min": 1 2020-09-08T15:41:38.194 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.194 INFO:teuthology.orchestra.run.smithi104.stdout: "application_metadata": { 2020-09-08T15:41:38.195 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_devicehealth": {} 2020-09-08T15:41:38.195 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.195 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.196 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.196 INFO:teuthology.orchestra.run.smithi104.stdout: "pool": 2, 2020-09-08T15:41:38.197 INFO:teuthology.orchestra.run.smithi104.stdout: "pool_name": "unique_pool_0", 2020-09-08T15:41:38.197 INFO:teuthology.orchestra.run.smithi104.stdout: "create_time": "2020-09-08T15:37:28.858589+0000", 2020-09-08T15:41:38.197 INFO:teuthology.orchestra.run.smithi104.stdout: "flags": 1, 2020-09-08T15:41:38.197 INFO:teuthology.orchestra.run.smithi104.stdout: "flags_names": "hashpspool", 2020-09-08T15:41:38.198 INFO:teuthology.orchestra.run.smithi104.stdout: "type": 1, 2020-09-08T15:41:38.198 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 2, 2020-09-08T15:41:38.198 INFO:teuthology.orchestra.run.smithi104.stdout: "min_size": 2, 2020-09-08T15:41:38.198 INFO:teuthology.orchestra.run.smithi104.stdout: "crush_rule": 0, 2020-09-08T15:41:38.198 INFO:teuthology.orchestra.run.smithi104.stdout: "object_hash": 2, 2020-09-08T15:41:38.199 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_autoscale_mode": "off", 2020-09-08T15:41:38.199 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num": 16, 2020-09-08T15:41:38.199 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_placement_num": 16, 2020-09-08T15:41:38.199 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_placement_num_target": 16, 2020-09-08T15:41:38.199 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num_target": 16, 2020-09-08T15:41:38.200 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num_pending": 16, 2020-09-08T15:41:38.200 INFO:teuthology.orchestra.run.smithi104.stdout: "last_pg_merge_meta": { 2020-09-08T15:41:38.200 INFO:teuthology.orchestra.run.smithi104.stdout: "source_pgid": "0.0", 2020-09-08T15:41:38.200 INFO:teuthology.orchestra.run.smithi104.stdout: "ready_epoch": 0, 2020-09-08T15:41:38.200 INFO:teuthology.orchestra.run.smithi104.stdout: "last_epoch_started": 0, 2020-09-08T15:41:38.201 INFO:teuthology.orchestra.run.smithi104.stdout: "last_epoch_clean": 0, 2020-09-08T15:41:38.201 INFO:teuthology.orchestra.run.smithi104.stdout: "source_version": "0'0", 2020-09-08T15:41:38.201 INFO:teuthology.orchestra.run.smithi104.stdout: "target_version": "0'0" 2020-09-08T15:41:38.202 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.202 INFO:teuthology.orchestra.run.smithi104.stdout: "last_change": "131", 2020-09-08T15:41:38.202 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend": "0", 2020-09-08T15:41:38.203 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend_prenautilus": "0", 2020-09-08T15:41:38.203 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend_preluminous": "0", 2020-09-08T15:41:38.203 INFO:teuthology.orchestra.run.smithi104.stdout: "auid": 0, 2020-09-08T15:41:38.203 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_mode": "selfmanaged", 2020-09-08T15:41:38.203 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_seq": 0, 2020-09-08T15:41:38.203 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_epoch": 0, 2020-09-08T15:41:38.204 INFO:teuthology.orchestra.run.smithi104.stdout: "pool_snaps": [], 2020-09-08T15:41:38.204 INFO:teuthology.orchestra.run.smithi104.stdout: "removed_snaps": "[]", 2020-09-08T15:41:38.204 INFO:teuthology.orchestra.run.smithi104.stdout: "quota_max_bytes": 0, 2020-09-08T15:41:38.204 INFO:teuthology.orchestra.run.smithi104.stdout: "quota_max_objects": 0, 2020-09-08T15:41:38.204 INFO:teuthology.orchestra.run.smithi104.stdout: "tiers": [], 2020-09-08T15:41:38.205 INFO:teuthology.orchestra.run.smithi104.stdout: "tier_of": -1, 2020-09-08T15:41:38.205 INFO:teuthology.orchestra.run.smithi104.stdout: "read_tier": -1, 2020-09-08T15:41:38.205 INFO:teuthology.orchestra.run.smithi104.stdout: "write_tier": -1, 2020-09-08T15:41:38.205 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_mode": "none", 2020-09-08T15:41:38.205 INFO:teuthology.orchestra.run.smithi104.stdout: "target_max_bytes": 0, 2020-09-08T15:41:38.206 INFO:teuthology.orchestra.run.smithi104.stdout: "target_max_objects": 0, 2020-09-08T15:41:38.206 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-09-08T15:41:38.206 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-09-08T15:41:38.206 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_full_ratio_micro": 800000, 2020-09-08T15:41:38.206 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_min_flush_age": 0, 2020-09-08T15:41:38.207 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_min_evict_age": 0, 2020-09-08T15:41:38.207 INFO:teuthology.orchestra.run.smithi104.stdout: "erasure_code_profile": "", 2020-09-08T15:41:38.207 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_params": { 2020-09-08T15:41:38.207 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "none" 2020-09-08T15:41:38.207 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.208 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_period": 0, 2020-09-08T15:41:38.208 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_count": 0, 2020-09-08T15:41:38.208 INFO:teuthology.orchestra.run.smithi104.stdout: "use_gmt_hitset": true, 2020-09-08T15:41:38.208 INFO:teuthology.orchestra.run.smithi104.stdout: "min_read_recency_for_promote": 0, 2020-09-08T15:41:38.208 INFO:teuthology.orchestra.run.smithi104.stdout: "min_write_recency_for_promote": 0, 2020-09-08T15:41:38.208 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_grade_decay_rate": 0, 2020-09-08T15:41:38.209 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_search_last_n": 0, 2020-09-08T15:41:38.209 INFO:teuthology.orchestra.run.smithi104.stdout: "grade_table": [], 2020-09-08T15:41:38.209 INFO:teuthology.orchestra.run.smithi104.stdout: "stripe_width": 0, 2020-09-08T15:41:38.209 INFO:teuthology.orchestra.run.smithi104.stdout: "expected_num_objects": 0, 2020-09-08T15:41:38.209 INFO:teuthology.orchestra.run.smithi104.stdout: "fast_read": false, 2020-09-08T15:41:38.210 INFO:teuthology.orchestra.run.smithi104.stdout: "options": {}, 2020-09-08T15:41:38.210 INFO:teuthology.orchestra.run.smithi104.stdout: "application_metadata": { 2020-09-08T15:41:38.210 INFO:teuthology.orchestra.run.smithi104.stdout: "rados": {} 2020-09-08T15:41:38.210 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.210 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.211 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T15:41:38.211 INFO:teuthology.orchestra.run.smithi104.stdout: "osds": [ 2020-09-08T15:41:38.211 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.211 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 0, 2020-09-08T15:41:38.211 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "e49a7341-7c75-4883-b061-3e8fe9467e6e", 2020-09-08T15:41:38.212 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:38.212 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:38.212 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:38.213 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:38.213 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:38.213 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:38.213 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 11, 2020-09-08T15:41:38.213 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 55, 2020-09-08T15:41:38.214 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:38.214 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:38.214 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:38.214 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.214 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.215 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.215 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6801", 2020-09-08T15:41:38.215 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:41:38.215 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.215 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.215 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.216 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:38.216 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.216 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.216 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.216 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6802", 2020-09-08T15:41:38.217 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:41:38.217 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.217 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.217 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.217 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6803", 2020-09-08T15:41:38.218 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:41:38.218 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.218 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.218 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.218 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:38.219 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.219 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.219 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.219 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6806", 2020-09-08T15:41:38.219 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:41:38.219 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.220 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.220 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.220 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6807", 2020-09-08T15:41:38.220 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:41:38.220 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.221 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.221 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.221 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:38.221 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.221 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.222 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.222 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6804", 2020-09-08T15:41:38.222 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:41:38.222 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.223 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.223 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.223 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6805", 2020-09-08T15:41:38.223 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:41:38.224 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.224 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.224 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.104:6801/82470352", 2020-09-08T15:41:38.224 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.104:6803/82470352", 2020-09-08T15:41:38.225 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.104:6807/82470352", 2020-09-08T15:41:38.225 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.104:6805/82470352", 2020-09-08T15:41:38.225 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:38.225 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:38.225 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:38.225 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.226 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.226 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.226 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 1, 2020-09-08T15:41:38.226 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "a83df412-110d-440a-b411-ad2a7c36f81a", 2020-09-08T15:41:38.226 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:38.227 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:38.227 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:38.227 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:38.227 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:38.227 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:38.228 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 17, 2020-09-08T15:41:38.228 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 126, 2020-09-08T15:41:38.228 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:38.228 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:38.228 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:38.229 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.229 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.229 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.229 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6809", 2020-09-08T15:41:38.229 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:41:38.230 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.230 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.230 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.230 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:38.230 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.231 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.231 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.231 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6810", 2020-09-08T15:41:38.231 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:41:38.231 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.231 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.232 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.232 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6811", 2020-09-08T15:41:38.232 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:41:38.233 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.233 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.233 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.234 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:38.234 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.234 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.234 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.234 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6814", 2020-09-08T15:41:38.235 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:41:38.235 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.235 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.235 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.235 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6815", 2020-09-08T15:41:38.236 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:41:38.236 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.236 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.236 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.236 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:38.237 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.237 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.237 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.237 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6812", 2020-09-08T15:41:38.237 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:41:38.237 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.238 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.238 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.238 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6813", 2020-09-08T15:41:38.238 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:41:38.238 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.239 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.239 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.239 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.104:6809/1421798992", 2020-09-08T15:41:38.239 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.104:6811/1421798992", 2020-09-08T15:41:38.239 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.104:6815/1421798992", 2020-09-08T15:41:38.240 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.104:6813/1421798992", 2020-09-08T15:41:38.240 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:38.240 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:38.240 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:38.240 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.241 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.241 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.241 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 2, 2020-09-08T15:41:38.241 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "ca6da1cc-1d8d-494b-93e6-e065ff70d434", 2020-09-08T15:41:38.241 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:38.242 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:38.242 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:38.242 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:38.242 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:38.242 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:38.242 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 23, 2020-09-08T15:41:38.243 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 126, 2020-09-08T15:41:38.243 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:38.243 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:38.244 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:38.244 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.244 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.244 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.245 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6817", 2020-09-08T15:41:38.245 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:41:38.246 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.246 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.246 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.246 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:38.247 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.247 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.247 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.247 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6818", 2020-09-08T15:41:38.248 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:41:38.248 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.248 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.248 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.248 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6819", 2020-09-08T15:41:38.249 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:41:38.249 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.249 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.249 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.249 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:38.250 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.250 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.250 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.250 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6822", 2020-09-08T15:41:38.250 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:41:38.251 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.251 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.251 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.251 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6823", 2020-09-08T15:41:38.251 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:41:38.252 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.252 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.252 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.252 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:38.252 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.253 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.253 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.253 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6820", 2020-09-08T15:41:38.253 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:41:38.253 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.253 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.254 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.254 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6821", 2020-09-08T15:41:38.254 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:41:38.254 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.254 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.255 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.256 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.104:6817/2713413369", 2020-09-08T15:41:38.257 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.104:6819/2713413369", 2020-09-08T15:41:38.257 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.104:6823/2713413369", 2020-09-08T15:41:38.257 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.104:6821/2713413369", 2020-09-08T15:41:38.258 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:38.260 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:38.260 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:38.260 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.260 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.261 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.261 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 3, 2020-09-08T15:41:38.261 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e", 2020-09-08T15:41:38.261 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:38.261 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:38.262 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:38.262 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:38.262 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:38.262 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:38.262 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 29, 2020-09-08T15:41:38.263 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 0, 2020-09-08T15:41:38.263 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:38.263 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:38.263 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:38.263 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.264 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.264 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.264 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6825", 2020-09-08T15:41:38.264 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:41:38.264 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.265 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.265 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.265 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:38.265 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.265 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.266 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.266 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6826", 2020-09-08T15:41:38.266 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:41:38.266 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.266 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.267 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.267 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6827", 2020-09-08T15:41:38.267 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:41:38.267 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.268 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.268 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.269 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:38.270 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.270 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.270 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.270 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6830", 2020-09-08T15:41:38.271 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:41:38.271 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.271 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.271 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.271 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6831", 2020-09-08T15:41:38.271 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:41:38.272 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.272 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.272 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.272 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:38.272 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.273 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.273 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.273 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6828", 2020-09-08T15:41:38.273 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:41:38.273 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.274 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.274 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.274 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6829", 2020-09-08T15:41:38.274 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:41:38.274 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.275 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.275 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.275 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.104:6825/1495946248", 2020-09-08T15:41:38.275 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.104:6827/1495946248", 2020-09-08T15:41:38.275 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.104:6831/1495946248", 2020-09-08T15:41:38.276 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.104:6829/1495946248", 2020-09-08T15:41:38.276 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:38.276 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:38.276 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:38.276 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.276 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.277 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.277 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 4, 2020-09-08T15:41:38.277 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "d8334c70-f776-4652-944d-aed35a8e43f7", 2020-09-08T15:41:38.277 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:38.278 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:38.278 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:38.278 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 0, 2020-09-08T15:41:38.278 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:38.278 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:38.279 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 35, 2020-09-08T15:41:38.279 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 0, 2020-09-08T15:41:38.279 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:38.279 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:38.280 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:38.280 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.280 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.280 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.281 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6803", 2020-09-08T15:41:38.281 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:41:38.281 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.281 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.281 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.282 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:38.282 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.282 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.282 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.282 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6804", 2020-09-08T15:41:38.283 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:41:38.283 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.283 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.283 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.283 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6805", 2020-09-08T15:41:38.284 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:41:38.284 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.284 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.284 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.284 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:38.284 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.285 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.285 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.285 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6808", 2020-09-08T15:41:38.285 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:41:38.286 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.286 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.286 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.286 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6809", 2020-09-08T15:41:38.286 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:41:38.286 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.287 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.287 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.287 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:38.287 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.287 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.288 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.288 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6806", 2020-09-08T15:41:38.288 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:41:38.288 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.288 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.289 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.289 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6807", 2020-09-08T15:41:38.289 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:41:38.289 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.290 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.290 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.291 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.55:6803/1173103260", 2020-09-08T15:41:38.291 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.55:6805/1173103260", 2020-09-08T15:41:38.291 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.55:6809/1173103260", 2020-09-08T15:41:38.291 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.55:6807/1173103260", 2020-09-08T15:41:38.291 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:38.292 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:38.292 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:38.292 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.292 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.292 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.293 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 5, 2020-09-08T15:41:38.293 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "6789abd2-9e9b-4c04-81fb-0ce2d2849e39", 2020-09-08T15:41:38.293 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:38.293 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:38.293 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:38.294 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:38.294 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:38.294 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:38.294 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 42, 2020-09-08T15:41:38.294 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 126, 2020-09-08T15:41:38.295 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:38.295 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:38.295 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:38.295 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.295 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.295 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.296 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6811", 2020-09-08T15:41:38.296 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:41:38.296 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.296 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.296 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.297 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:38.297 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.297 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.297 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.297 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6812", 2020-09-08T15:41:38.298 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:41:38.298 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.298 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.298 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.298 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6813", 2020-09-08T15:41:38.299 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:41:38.299 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.299 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.299 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.299 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:38.300 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.300 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.300 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.301 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6816", 2020-09-08T15:41:38.301 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:41:38.302 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.302 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.302 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.302 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6817", 2020-09-08T15:41:38.302 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:41:38.303 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.304 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.304 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.304 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:38.304 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.305 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.305 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.305 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6814", 2020-09-08T15:41:38.305 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:41:38.305 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.306 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.306 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.306 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6815", 2020-09-08T15:41:38.306 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:41:38.307 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.307 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.307 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.308 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.55:6811/1047046376", 2020-09-08T15:41:38.308 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.55:6813/1047046376", 2020-09-08T15:41:38.308 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.55:6817/1047046376", 2020-09-08T15:41:38.308 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.55:6815/1047046376", 2020-09-08T15:41:38.308 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:38.309 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:38.309 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:38.309 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.309 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.310 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.310 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 6, 2020-09-08T15:41:38.310 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "ad9a93fe-0ca0-4052-bfa9-e2abb268829e", 2020-09-08T15:41:38.310 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:38.311 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:38.311 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:38.311 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:38.311 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:38.312 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:38.312 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 49, 2020-09-08T15:41:38.312 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 126, 2020-09-08T15:41:38.312 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:38.312 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:38.313 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:38.313 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.313 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.313 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.314 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6819", 2020-09-08T15:41:38.314 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:41:38.315 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.315 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.315 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.315 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:38.316 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.316 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.316 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.316 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6820", 2020-09-08T15:41:38.317 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:41:38.317 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.317 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.317 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.318 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6821", 2020-09-08T15:41:38.318 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:41:38.318 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.318 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.319 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.319 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:38.319 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.319 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.319 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.320 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6824", 2020-09-08T15:41:38.320 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:41:38.320 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.320 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.320 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.321 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6825", 2020-09-08T15:41:38.321 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:41:38.321 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.321 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.322 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.322 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:38.322 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.322 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.323 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.323 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6822", 2020-09-08T15:41:38.323 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:41:38.323 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.323 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.324 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.324 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6823", 2020-09-08T15:41:38.325 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:41:38.325 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.325 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.325 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.326 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.55:6819/2501844793", 2020-09-08T15:41:38.326 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.55:6821/2501844793", 2020-09-08T15:41:38.326 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.55:6825/2501844793", 2020-09-08T15:41:38.326 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.55:6823/2501844793", 2020-09-08T15:41:38.327 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:38.327 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:38.327 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:38.327 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.327 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.328 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.328 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 7, 2020-09-08T15:41:38.328 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "f3795c86-4540-442c-a151-b2769fa577e4", 2020-09-08T15:41:38.328 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:38.329 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:38.329 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:38.329 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:38.329 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:38.329 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:38.330 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 56, 2020-09-08T15:41:38.330 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 126, 2020-09-08T15:41:38.330 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:38.330 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:38.331 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:38.331 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.331 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.331 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.331 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6827", 2020-09-08T15:41:38.332 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:41:38.332 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.332 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.332 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.332 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:38.333 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.333 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.333 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.333 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6828", 2020-09-08T15:41:38.333 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:41:38.334 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.334 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.334 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.334 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6829", 2020-09-08T15:41:38.334 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:41:38.334 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.335 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.335 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.335 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:38.335 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.335 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.336 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.336 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6832", 2020-09-08T15:41:38.336 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:41:38.336 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.336 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.337 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.337 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6833", 2020-09-08T15:41:38.337 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:41:38.337 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.337 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.338 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.338 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:38.338 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.338 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.338 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.339 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6830", 2020-09-08T15:41:38.339 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:41:38.339 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.339 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.339 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.340 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6831", 2020-09-08T15:41:38.340 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:41:38.340 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.340 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.341 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.341 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.55:6827/128909153", 2020-09-08T15:41:38.341 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.55:6829/128909153", 2020-09-08T15:41:38.341 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.55:6833/128909153", 2020-09-08T15:41:38.341 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.55:6831/128909153", 2020-09-08T15:41:38.342 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:38.342 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:38.342 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:38.342 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.342 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.343 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.343 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 8, 2020-09-08T15:41:38.343 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "3b0ac380-878e-4c1c-9278-80ce67ca2df4", 2020-09-08T15:41:38.343 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:38.343 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:38.344 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:38.344 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:38.344 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:38.344 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:38.344 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 64, 2020-09-08T15:41:38.345 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 126, 2020-09-08T15:41:38.345 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:38.345 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:38.345 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:38.345 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.346 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.346 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.346 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6801", 2020-09-08T15:41:38.346 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:41:38.346 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.347 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.347 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.347 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:38.347 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.347 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.348 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.348 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6802", 2020-09-08T15:41:38.348 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:41:38.348 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.348 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.348 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.349 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6803", 2020-09-08T15:41:38.349 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:41:38.349 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.349 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.349 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.350 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:38.350 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.350 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.350 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.350 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6806", 2020-09-08T15:41:38.351 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:41:38.351 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.351 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.353 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.353 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6807", 2020-09-08T15:41:38.353 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:41:38.361 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.361 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.361 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.362 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:38.362 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.362 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.362 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.362 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6804", 2020-09-08T15:41:38.363 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:41:38.363 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.363 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.363 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.363 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6805", 2020-09-08T15:41:38.364 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:41:38.364 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.364 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.364 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.365 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.166:6801/2108117574", 2020-09-08T15:41:38.365 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.166:6803/2108117574", 2020-09-08T15:41:38.365 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.166:6807/2108117574", 2020-09-08T15:41:38.365 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.166:6805/2108117574", 2020-09-08T15:41:38.365 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:38.366 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:38.366 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:38.366 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.366 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.366 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.367 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 9, 2020-09-08T15:41:38.367 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08", 2020-09-08T15:41:38.367 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:38.367 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:38.367 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:38.368 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:38.368 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:38.368 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:38.368 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 72, 2020-09-08T15:41:38.368 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 126, 2020-09-08T15:41:38.369 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:38.369 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:38.369 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:38.369 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.369 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.370 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.370 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6809", 2020-09-08T15:41:38.370 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:41:38.370 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.370 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.371 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.371 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:38.371 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.371 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.371 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.372 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6810", 2020-09-08T15:41:38.372 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:41:38.372 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.372 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.372 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.373 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6811", 2020-09-08T15:41:38.373 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:41:38.373 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.373 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.373 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.374 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:38.374 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.374 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.374 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.374 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6814", 2020-09-08T15:41:38.375 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:41:38.375 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.375 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.375 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.375 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6815", 2020-09-08T15:41:38.376 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:41:38.376 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.376 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.376 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.376 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:38.376 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.377 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.377 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.377 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6812", 2020-09-08T15:41:38.377 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:41:38.377 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.378 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.378 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.378 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6813", 2020-09-08T15:41:38.378 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:41:38.378 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.379 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.379 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.379 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.166:6809/1458870080", 2020-09-08T15:41:38.379 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.166:6811/1458870080", 2020-09-08T15:41:38.379 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.166:6815/1458870080", 2020-09-08T15:41:38.380 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.166:6813/1458870080", 2020-09-08T15:41:38.380 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:38.380 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:38.380 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:38.380 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.381 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.381 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.381 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 10, 2020-09-08T15:41:38.381 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "c194210e-bd89-4a1f-98fb-6bf44f0b324c", 2020-09-08T15:41:38.381 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:38.382 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:38.382 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:38.382 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:38.382 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:38.382 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:38.382 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 81, 2020-09-08T15:41:38.383 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 0, 2020-09-08T15:41:38.383 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:38.383 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:38.383 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:38.383 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.384 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.384 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.384 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6817", 2020-09-08T15:41:38.384 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:41:38.385 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.385 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.385 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.385 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:38.386 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.386 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.386 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.386 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6818", 2020-09-08T15:41:38.386 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:41:38.387 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.387 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.387 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.387 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6819", 2020-09-08T15:41:38.387 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:41:38.388 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.388 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.388 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.388 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:38.388 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.389 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.389 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.389 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6822", 2020-09-08T15:41:38.389 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:41:38.389 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.390 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.390 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.390 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6823", 2020-09-08T15:41:38.390 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:41:38.390 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.391 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.391 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.391 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:38.391 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.391 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.392 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.392 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6820", 2020-09-08T15:41:38.392 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:41:38.392 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.392 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.392 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.393 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6821", 2020-09-08T15:41:38.393 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:41:38.393 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.393 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.393 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.394 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.166:6817/62517848", 2020-09-08T15:41:38.394 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.166:6819/62517848", 2020-09-08T15:41:38.394 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.166:6823/62517848", 2020-09-08T15:41:38.394 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.166:6821/62517848", 2020-09-08T15:41:38.394 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:38.395 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:38.395 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:38.395 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.395 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.395 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.396 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 11, 2020-09-08T15:41:38.396 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "121301fe-a41c-40c0-ad09-b00de37a55e1", 2020-09-08T15:41:38.396 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:41:38.396 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:41:38.396 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:41:38.397 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:41:38.397 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:41:38.397 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:41:38.397 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 90, 2020-09-08T15:41:38.397 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 126, 2020-09-08T15:41:38.398 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:41:38.398 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:41:38.398 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:41:38.398 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.398 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.399 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.399 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6825", 2020-09-08T15:41:38.399 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:41:38.399 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.399 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.399 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.400 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:41:38.400 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.400 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.400 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.400 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6826", 2020-09-08T15:41:38.401 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:41:38.401 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.401 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.401 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.401 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6827", 2020-09-08T15:41:38.402 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:41:38.402 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.402 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.403 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.403 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:41:38.403 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.403 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.403 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.404 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6830", 2020-09-08T15:41:38.404 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:41:38.404 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.404 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.404 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.405 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6831", 2020-09-08T15:41:38.405 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:41:38.405 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.405 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.405 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.406 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:41:38.406 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:41:38.406 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.406 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:41:38.406 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6828", 2020-09-08T15:41:38.407 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:41:38.407 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.407 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.407 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:41:38.407 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6829", 2020-09-08T15:41:38.407 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:41:38.408 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.408 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.408 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.408 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.166:6825/691735125", 2020-09-08T15:41:38.409 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.166:6827/691735125", 2020-09-08T15:41:38.409 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.166:6831/691735125", 2020-09-08T15:41:38.409 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.166:6829/691735125", 2020-09-08T15:41:38.409 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:41:38.409 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:41:38.410 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:41:38.410 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:41:38.410 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.410 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T15:41:38.410 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_xinfo": [ 2020-09-08T15:41:38.410 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.411 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 0, 2020-09-08T15:41:38.411 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:38.411 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:38.412 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:38.412 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:38.412 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:38.412 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:41:06.141666+0000", 2020-09-08T15:41:38.412 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:38.413 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.413 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.413 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 1, 2020-09-08T15:41:38.413 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:38.413 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:38.414 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:38.414 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:38.414 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:38.414 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:41:34.305299+0000", 2020-09-08T15:41:38.414 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:38.414 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.415 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.415 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 2, 2020-09-08T15:41:38.415 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:38.415 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:38.415 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:38.416 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:38.416 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:38.416 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:40:56.946674+0000", 2020-09-08T15:41:38.416 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:38.416 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.417 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.417 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 3, 2020-09-08T15:41:38.417 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:38.417 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:38.417 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:38.418 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:38.418 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:38.418 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:41:21.987770+0000", 2020-09-08T15:41:38.418 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:38.418 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.419 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.419 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 4, 2020-09-08T15:41:38.419 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:38.419 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:38.419 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:38.420 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:38.420 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:38.420 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:40:46.317711+0000", 2020-09-08T15:41:38.420 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:38.420 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.421 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.421 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 5, 2020-09-08T15:41:38.421 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:38.421 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:38.421 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:38.422 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:38.422 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:38.422 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:41:12.336983+0000", 2020-09-08T15:41:38.422 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:38.422 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.422 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.423 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 6, 2020-09-08T15:41:38.423 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:38.423 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:38.423 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:38.423 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:38.424 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:38.424 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:40:40.153723+0000", 2020-09-08T15:41:38.424 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:38.424 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.424 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.425 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 7, 2020-09-08T15:41:38.425 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:38.425 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:38.425 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:38.425 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:38.426 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:38.426 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:41:05.533799+0000", 2020-09-08T15:41:38.426 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:38.426 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.427 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.427 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 8, 2020-09-08T15:41:38.427 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:38.427 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:38.427 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:38.428 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:38.428 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:38.428 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:41:27.381767+0000", 2020-09-08T15:41:38.428 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:38.428 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.429 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.429 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 9, 2020-09-08T15:41:38.429 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:38.429 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:38.430 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:38.430 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:38.430 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:38.430 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:40:54.670244+0000", 2020-09-08T15:41:38.430 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:38.431 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.431 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.431 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 10, 2020-09-08T15:41:38.431 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:38.431 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:38.432 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:38.432 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:38.432 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:38.432 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:41:18.090862+0000", 2020-09-08T15:41:38.432 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:38.433 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.433 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:41:38.433 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 11, 2020-09-08T15:41:38.433 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:41:38.433 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:41:38.434 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:41:38.434 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:41:38.434 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:41:38.434 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:40:42.450226+0000", 2020-09-08T15:41:38.434 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:41:38.435 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.435 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T15:41:38.435 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_upmap": [], 2020-09-08T15:41:38.435 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_upmap_items": [], 2020-09-08T15:41:38.435 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_temp": [], 2020-09-08T15:41:38.436 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_temp": [], 2020-09-08T15:41:38.436 INFO:teuthology.orchestra.run.smithi104.stdout: "blocklist": { 2020-09-08T15:41:38.436 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/1257365091": "2020-09-09T15:29:39.430456+0000", 2020-09-08T15:41:38.436 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/1399123455": "2020-09-09T15:29:39.430456+0000", 2020-09-08T15:41:38.436 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/2810441605": "2020-09-09T15:29:39.430456+0000", 2020-09-08T15:41:38.436 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6800/182241824": "2020-09-09T15:29:39.430456+0000", 2020-09-08T15:41:38.437 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6801/182241824": "2020-09-09T15:29:39.430456+0000", 2020-09-08T15:41:38.437 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/2021933044": "2020-09-09T15:26:33.248197+0000", 2020-09-08T15:41:38.437 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6800/3129754347": "2020-09-09T15:26:33.248197+0000", 2020-09-08T15:41:38.437 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/1270031705": "2020-09-09T15:26:33.248197+0000", 2020-09-08T15:41:38.437 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/2568432190": "2020-09-09T15:26:33.248197+0000", 2020-09-08T15:41:38.438 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6800/1138921844": "2020-09-09T15:26:09.867236+0000", 2020-09-08T15:41:38.438 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/3827328474": "2020-09-09T15:26:09.867236+0000", 2020-09-08T15:41:38.438 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6801/1138921844": "2020-09-09T15:26:09.867236+0000", 2020-09-08T15:41:38.439 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/1662035381": "2020-09-09T15:26:09.867236+0000", 2020-09-08T15:41:38.439 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6801/3129754347": "2020-09-09T15:26:33.248197+0000", 2020-09-08T15:41:38.440 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/2169968102": "2020-09-09T15:26:09.867236+0000" 2020-09-08T15:41:38.440 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.440 INFO:teuthology.orchestra.run.smithi104.stdout: "erasure_code_profiles": { 2020-09-08T15:41:38.440 INFO:teuthology.orchestra.run.smithi104.stdout: "default": { 2020-09-08T15:41:38.440 INFO:teuthology.orchestra.run.smithi104.stdout: "crush-failure-domain": "osd", 2020-09-08T15:41:38.441 INFO:teuthology.orchestra.run.smithi104.stdout: "k": "2", 2020-09-08T15:41:38.441 INFO:teuthology.orchestra.run.smithi104.stdout: "m": "1", 2020-09-08T15:41:38.441 INFO:teuthology.orchestra.run.smithi104.stdout: "plugin": "jerasure", 2020-09-08T15:41:38.441 INFO:teuthology.orchestra.run.smithi104.stdout: "ruleset-failure-domain": "osd", 2020-09-08T15:41:38.441 INFO:teuthology.orchestra.run.smithi104.stdout: "technique": "reed_sol_van" 2020-09-08T15:41:38.442 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:41:38.442 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:41:38.442 INFO:teuthology.orchestra.run.smithi104.stdout: "removed_snaps_queue": [], 2020-09-08T15:41:38.442 INFO:teuthology.orchestra.run.smithi104.stdout: "new_removed_snaps": [], 2020-09-08T15:41:38.442 INFO:teuthology.orchestra.run.smithi104.stdout: "new_purged_snaps": [], 2020-09-08T15:41:38.443 INFO:teuthology.orchestra.run.smithi104.stdout: "crush_node_flags": {}, 2020-09-08T15:41:38.443 INFO:teuthology.orchestra.run.smithi104.stdout: "device_class_flags": {} 2020-09-08T15:41:38.443 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:41:38.444 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:41:38.445 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:38.445 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:41:38 smithi166 bash[16593]: debug 2020-09-08T15:41:38.312+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:38.446 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:41:38 smithi104 bash[19922]: debug 2020-09-08T15:41:38.064+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:38.547 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:38.626 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:38 smithi104 bash[9203]: audit 2020-09-08T15:41:38.172549+0000 mon.a (mon.0) 1097 : audit [DBG] from='client.? v1:172.21.15.104:0/2542979935' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2020-09-08T15:41:38.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:38 smithi166 bash[9446]: audit 2020-09-08T15:41:38.172549+0000 mon.a (mon.0) 1097 : audit [DBG] from='client.? v1:172.21.15.104:0/2542979935' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2020-09-08T15:41:38.894 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:38.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:38 smithi055 bash[9296]: audit 2020-09-08T15:41:38.172549+0000 mon.a (mon.0) 1097 : audit [DBG] from='client.? v1:172.21.15.104:0/2542979935' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2020-09-08T15:41:38.998 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:39.286 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:41:39.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:41:38 smithi166 bash[18960]: debug 2020-09-08T15:41:38.892+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:39.449 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:41:39.522 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:39.627 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:41:39.629 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:39 smithi104 bash[9203]: cluster 2020-09-08T15:41:37.653710+0000 mgr.x (mgr.24118) 426 : cluster [DBG] pgmap v552: 17 pgs: 17 active+clean; 564 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:41:39.746 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:41:39.842 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:41:39 smithi166 bash[18960]: debug 2020-09-08T15:41:39.524+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:39.842 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:39 smithi166 bash[9446]: cluster 2020-09-08T15:41:37.653710+0000 mgr.x (mgr.24118) 426 : cluster [DBG] pgmap v552: 17 pgs: 17 active+clean; 564 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:41:39.856 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:41:39.922 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:39 smithi055 bash[9296]: cluster 2020-09-08T15:41:37.653710+0000 mgr.x (mgr.24118) 426 : cluster [DBG] pgmap v552: 17 pgs: 17 active+clean; 564 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:41:39.923 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:41:39 smithi055 bash[19003]: debug 2020-09-08T15:41:39.745+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:39.965 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:41:40.070 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:40.165 INFO:tasks.thrashosds.thrasher:j is {'epoch': 207, 'fsid': '759b27b6-f1e7-11ea-a080-001a4aab830c', 'created': '2020-09-08T15:25:41.788365+0000', 'modified': '2020-09-08T15:41:35.710453+0000', 'last_up_change': '2020-09-08T15:34:41.591462+0000', 'last_in_change': '2020-09-08T15:34:41.591462+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 27, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 12, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-09-08T15:25:56.314385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'unique_pool_0', 'create_time': '2020-09-08T15:37:28.858589+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'e49a7341-7c75-4883-b061-3e8fe9467e6e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6801', 'nonce': 82470352}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6802', 'nonce': 82470352}, {'type': 'v1', 'addr': '172.21.15.104:6803', 'nonce': 82470352}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6806', 'nonce': 82470352}, {'type': 'v1', 'addr': '172.21.15.104:6807', 'nonce': 82470352}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6804', 'nonce': 82470352}, {'type': 'v1', 'addr': '172.21.15.104:6805', 'nonce': 82470352}]}, 'public_addr': '172.21.15.104:6801/82470352', 'cluster_addr': '172.21.15.104:6803/82470352', 'heartbeat_back_addr': '172.21.15.104:6807/82470352', 'heartbeat_front_addr': '172.21.15.104:6805/82470352', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a83df412-110d-440a-b411-ad2a7c36f81a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6809', 'nonce': 1421798992}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6810', 'nonce': 1421798992}, {'type': 'v1', 'addr': '172.21.15.104:6811', 'nonce': 1421798992}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6814', 'nonce': 1421798992}, {'type': 'v1', 'addr': '172.21.15.104:6815', 'nonce': 1421798992}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6812', 'nonce': 1421798992}, {'type': 'v1', 'addr': '172.21.15.104:6813', 'nonce': 1421798992}]}, 'public_addr': '172.21.15.104:6809/1421798992', 'cluster_addr': '172.21.15.104:6811/1421798992', 'heartbeat_back_addr': '172.21.15.104:6815/1421798992', 'heartbeat_front_addr': '172.21.15.104:6813/1421798992', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'ca6da1cc-1d8d-494b-93e6-e065ff70d434', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6817', 'nonce': 2713413369}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6818', 'nonce': 2713413369}, {'type': 'v1', 'addr': '172.21.15.104:6819', 'nonce': 2713413369}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6822', 'nonce': 2713413369}, {'type': 'v1', 'addr': '172.21.15.104:6823', 'nonce': 2713413369}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6820', 'nonce': 2713413369}, {'type': 'v1', 'addr': '172.21.15.104:6821', 'nonce': 2713413369}]}, 'public_addr': '172.21.15.104:6817/2713413369', 'cluster_addr': '172.21.15.104:6819/2713413369', 'heartbeat_back_addr': '172.21.15.104:6823/2713413369', 'heartbeat_front_addr': '172.21.15.104:6821/2713413369', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 29, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6825', 'nonce': 1495946248}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6826', 'nonce': 1495946248}, {'type': 'v1', 'addr': '172.21.15.104:6827', 'nonce': 1495946248}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6830', 'nonce': 1495946248}, {'type': 'v1', 'addr': '172.21.15.104:6831', 'nonce': 1495946248}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6828', 'nonce': 1495946248}, {'type': 'v1', 'addr': '172.21.15.104:6829', 'nonce': 1495946248}]}, 'public_addr': '172.21.15.104:6825/1495946248', 'cluster_addr': '172.21.15.104:6827/1495946248', 'heartbeat_back_addr': '172.21.15.104:6831/1495946248', 'heartbeat_front_addr': '172.21.15.104:6829/1495946248', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd8334c70-f776-4652-944d-aed35a8e43f7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 35, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.55:6803', 'nonce': 1173103260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6804', 'nonce': 1173103260}, {'type': 'v1', 'addr': '172.21.15.55:6805', 'nonce': 1173103260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6808', 'nonce': 1173103260}, {'type': 'v1', 'addr': '172.21.15.55:6809', 'nonce': 1173103260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6806', 'nonce': 1173103260}, {'type': 'v1', 'addr': '172.21.15.55:6807', 'nonce': 1173103260}]}, 'public_addr': '172.21.15.55:6803/1173103260', 'cluster_addr': '172.21.15.55:6805/1173103260', 'heartbeat_back_addr': '172.21.15.55:6809/1173103260', 'heartbeat_front_addr': '172.21.15.55:6807/1173103260', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '6789abd2-9e9b-4c04-81fb-0ce2d2849e39', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 42, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.55:6811', 'nonce': 1047046376}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6812', 'nonce': 1047046376}, {'type': 'v1', 'addr': '172.21.15.55:6813', 'nonce': 1047046376}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6816', 'nonce': 1047046376}, {'type': 'v1', 'addr': '172.21.15.55:6817', 'nonce': 1047046376}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6814', 'nonce': 1047046376}, {'type': 'v1', 'addr': '172.21.15.55:6815', 'nonce': 1047046376}]}, 'public_addr': '172.21.15.55:6811/1047046376', 'cluster_addr': '172.21.15.55:6813/1047046376', 'heartbeat_back_addr': '172.21.15.55:6817/1047046376', 'heartbeat_front_addr': '172.21.15.55:6815/1047046376', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ad9a93fe-0ca0-4052-bfa9-e2abb268829e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 49, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.55:6819', 'nonce': 2501844793}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6820', 'nonce': 2501844793}, {'type': 'v1', 'addr': '172.21.15.55:6821', 'nonce': 2501844793}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6824', 'nonce': 2501844793}, {'type': 'v1', 'addr': '172.21.15.55:6825', 'nonce': 2501844793}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6822', 'nonce': 2501844793}, {'type': 'v1', 'addr': '172.21.15.55:6823', 'nonce': 2501844793}]}, 'public_addr': '172.21.15.55:6819/2501844793', 'cluster_addr': '172.21.15.55:6821/2501844793', 'heartbeat_back_addr': '172.21.15.55:6825/2501844793', 'heartbeat_front_addr': '172.21.15.55:6823/2501844793', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'f3795c86-4540-442c-a151-b2769fa577e4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 56, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.55:6827', 'nonce': 128909153}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6828', 'nonce': 128909153}, {'type': 'v1', 'addr': '172.21.15.55:6829', 'nonce': 128909153}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6832', 'nonce': 128909153}, {'type': 'v1', 'addr': '172.21.15.55:6833', 'nonce': 128909153}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6830', 'nonce': 128909153}, {'type': 'v1', 'addr': '172.21.15.55:6831', 'nonce': 128909153}]}, 'public_addr': '172.21.15.55:6827/128909153', 'cluster_addr': '172.21.15.55:6829/128909153', 'heartbeat_back_addr': '172.21.15.55:6833/128909153', 'heartbeat_front_addr': '172.21.15.55:6831/128909153', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '3b0ac380-878e-4c1c-9278-80ce67ca2df4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 64, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.166:6801', 'nonce': 2108117574}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6802', 'nonce': 2108117574}, {'type': 'v1', 'addr': '172.21.15.166:6803', 'nonce': 2108117574}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6806', 'nonce': 2108117574}, {'type': 'v1', 'addr': '172.21.15.166:6807', 'nonce': 2108117574}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6804', 'nonce': 2108117574}, {'type': 'v1', 'addr': '172.21.15.166:6805', 'nonce': 2108117574}]}, 'public_addr': '172.21.15.166:6801/2108117574', 'cluster_addr': '172.21.15.166:6803/2108117574', 'heartbeat_back_addr': '172.21.15.166:6807/2108117574', 'heartbeat_front_addr': '172.21.15.166:6805/2108117574', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': 'ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 72, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.166:6809', 'nonce': 1458870080}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6810', 'nonce': 1458870080}, {'type': 'v1', 'addr': '172.21.15.166:6811', 'nonce': 1458870080}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6814', 'nonce': 1458870080}, {'type': 'v1', 'addr': '172.21.15.166:6815', 'nonce': 1458870080}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6812', 'nonce': 1458870080}, {'type': 'v1', 'addr': '172.21.15.166:6813', 'nonce': 1458870080}]}, 'public_addr': '172.21.15.166:6809/1458870080', 'cluster_addr': '172.21.15.166:6811/1458870080', 'heartbeat_back_addr': '172.21.15.166:6815/1458870080', 'heartbeat_front_addr': '172.21.15.166:6813/1458870080', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': 'c194210e-bd89-4a1f-98fb-6bf44f0b324c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 81, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.166:6817', 'nonce': 62517848}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6818', 'nonce': 62517848}, {'type': 'v1', 'addr': '172.21.15.166:6819', 'nonce': 62517848}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6822', 'nonce': 62517848}, {'type': 'v1', 'addr': '172.21.15.166:6823', 'nonce': 62517848}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6820', 'nonce': 62517848}, {'type': 'v1', 'addr': '172.21.15.166:6821', 'nonce': 62517848}]}, 'public_addr': '172.21.15.166:6817/62517848', 'cluster_addr': '172.21.15.166:6819/62517848', 'heartbeat_back_addr': '172.21.15.166:6823/62517848', 'heartbeat_front_addr': '172.21.15.166:6821/62517848', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '121301fe-a41c-40c0-ad09-b00de37a55e1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 90, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.166:6825', 'nonce': 691735125}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6826', 'nonce': 691735125}, {'type': 'v1', 'addr': '172.21.15.166:6827', 'nonce': 691735125}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6830', 'nonce': 691735125}, {'type': 'v1', 'addr': '172.21.15.166:6831', 'nonce': 691735125}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6828', 'nonce': 691735125}, {'type': 'v1', 'addr': '172.21.15.166:6829', 'nonce': 691735125}]}, 'public_addr': '172.21.15.166:6825/691735125', 'cluster_addr': '172.21.15.166:6827/691735125', 'heartbeat_back_addr': '172.21.15.166:6831/691735125', 'heartbeat_front_addr': '172.21.15.166:6829/691735125', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:41:06.141666+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:41:34.305299+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:40:56.946674+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:41:21.987770+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:40:46.317711+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:41:12.336983+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:40:40.153723+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:41:05.533799+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:41:27.381767+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:40:54.670244+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:41:18.090862+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:40:42.450226+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.104:0/1257365091': '2020-09-09T15:29:39.430456+0000', '172.21.15.104:0/1399123455': '2020-09-09T15:29:39.430456+0000', '172.21.15.104:0/2810441605': '2020-09-09T15:29:39.430456+0000', '172.21.15.104:6800/182241824': '2020-09-09T15:29:39.430456+0000', '172.21.15.104:6801/182241824': '2020-09-09T15:29:39.430456+0000', '172.21.15.104:0/2021933044': '2020-09-09T15:26:33.248197+0000', '172.21.15.104:6800/3129754347': '2020-09-09T15:26:33.248197+0000', '172.21.15.104:0/1270031705': '2020-09-09T15:26:33.248197+0000', '172.21.15.104:0/2568432190': '2020-09-09T15:26:33.248197+0000', '172.21.15.104:6800/1138921844': '2020-09-09T15:26:09.867236+0000', '172.21.15.104:0/3827328474': '2020-09-09T15:26:09.867236+0000', '172.21.15.104:6801/1138921844': '2020-09-09T15:26:09.867236+0000', '172.21.15.104:0/1662035381': '2020-09-09T15:26:09.867236+0000', '172.21.15.104:6801/3129754347': '2020-09-09T15:26:33.248197+0000', '172.21.15.104:0/2169968102': '2020-09-09T15:26:09.867236+0000'}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}} 2020-09-08T15:41:40.166 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph pg dump --format=json 2020-09-08T15:41:40.336 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:41:40.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:41:39 smithi055 bash[19003]: debug 2020-09-08T15:41:39.965+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:40.554 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:40.657 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:40.765 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:40.875 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:41:40.890 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:41:40 smithi104 bash[19922]: debug 2020-09-08T15:41:40.552+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:41.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:41:40 smithi055 bash[11956]: debug 2020-09-08T15:41:40.765+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:41.599 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:41:41.704 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:41.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:41 smithi055 bash[9296]: cluster 2020-09-08T15:41:39.654112+0000 mgr.x (mgr.24118) 427 : cluster [DBG] pgmap v553: 17 pgs: 17 active+clean; 472 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.20k op/s 2020-09-08T15:41:41.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:41 smithi055 bash[9296]: cluster 2020-09-08T15:41:40.422508+0000 osd.6 (osd.6) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:41.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:41 smithi055 bash[9296]: cluster 2020-09-08T15:41:40.422584+0000 osd.6 (osd.6) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:41.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:41 smithi055 bash[9296]: cluster 2020-09-08T15:41:40.646120+0000 mon.a (mon.0) 1098 : cluster [DBG] osdmap e208: 12 total, 12 up, 12 in 2020-09-08T15:41:41.928 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:41:41 smithi104 bash[22305]: debug 2020-09-08T15:41:41.596+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:41.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:41 smithi104 bash[9203]: cluster 2020-09-08T15:41:39.654112+0000 mgr.x (mgr.24118) 427 : cluster [DBG] pgmap v553: 17 pgs: 17 active+clean; 472 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.20k op/s 2020-09-08T15:41:41.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:41 smithi104 bash[9203]: cluster 2020-09-08T15:41:40.422508+0000 osd.6 (osd.6) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:41.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:41 smithi104 bash[9203]: cluster 2020-09-08T15:41:40.422584+0000 osd.6 (osd.6) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:41.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:41 smithi104 bash[9203]: cluster 2020-09-08T15:41:40.646120+0000 mon.a (mon.0) 1098 : cluster [DBG] osdmap e208: 12 total, 12 up, 12 in 2020-09-08T15:41:41.992 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:41 smithi166 bash[9446]: cluster 2020-09-08T15:41:39.654112+0000 mgr.x (mgr.24118) 427 : cluster [DBG] pgmap v553: 17 pgs: 17 active+clean; 472 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.20k op/s 2020-09-08T15:41:41.993 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:41 smithi166 bash[9446]: cluster 2020-09-08T15:41:40.422508+0000 osd.6 (osd.6) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:41.993 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:41 smithi166 bash[9446]: cluster 2020-09-08T15:41:40.422584+0000 osd.6 (osd.6) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:41.996 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:41 smithi166 bash[9446]: cluster 2020-09-08T15:41:40.646120+0000 mon.a (mon.0) 1098 : cluster [DBG] osdmap e208: 12 total, 12 up, 12 in 2020-09-08T15:41:41.997 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:42.099 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:42.283 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:41:41 smithi166 bash[16593]: debug 2020-09-08T15:41:41.996+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:42.417 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:41:42 smithi104 bash[19922]: debug 2020-09-08T15:41:42.416+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:42.418 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:42.530 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:42.596 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:41:42.597 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T15:41:42.597 INFO:teuthology.orchestra.run.smithi166.stdout: "complaint_time": 30, 2020-09-08T15:41:42.597 INFO:teuthology.orchestra.run.smithi166.stdout: "num_blocked_ops": 0 2020-09-08T15:41:42.597 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:41:42.666 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:42.767 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:41:42.869 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:41:42 smithi166 bash[16593]: debug 2020-09-08T15:41:42.659+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:43.654 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2020-09-08T15:41:43.656 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:41:43.760 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:43.811 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:41:43.835 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:41:43.875 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:41:43.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:43 smithi055 bash[9296]: cluster 2020-09-08T15:41:41.654610+0000 mgr.x (mgr.24118) 428 : cluster [DBG] pgmap v555: 17 pgs: 17 active+clean; 472 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 679 op/s 2020-09-08T15:41:43.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:43 smithi055 bash[9296]: cluster 2020-09-08T15:41:42.473577+0000 osd.11 (osd.11) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:43.913 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:43 smithi055 bash[9296]: cluster 2020-09-08T15:41:42.473670+0000 osd.11 (osd.11) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:43.914 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:43 smithi055 bash[9296]: cluster 2020-09-08T15:41:42.659058+0000 mon.a (mon.0) 1099 : cluster [DBG] osdmap e209: 12 total, 12 up, 12 in 2020-09-08T15:41:43.914 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:41:43.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:43 smithi104 bash[9203]: cluster 2020-09-08T15:41:41.654610+0000 mgr.x (mgr.24118) 428 : cluster [DBG] pgmap v555: 17 pgs: 17 active+clean; 472 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 679 op/s 2020-09-08T15:41:43.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:43 smithi104 bash[9203]: cluster 2020-09-08T15:41:42.473577+0000 osd.11 (osd.11) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:43.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:43 smithi104 bash[9203]: cluster 2020-09-08T15:41:42.473670+0000 osd.11 (osd.11) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:43.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:43 smithi104 bash[9203]: cluster 2020-09-08T15:41:42.659058+0000 mon.a (mon.0) 1099 : cluster [DBG] osdmap e209: 12 total, 12 up, 12 in 2020-09-08T15:41:43.932 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:41:43 smithi104 bash[17588]: debug 2020-09-08T15:41:43.652+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:43.958 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:41:43.990 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:41:44.026 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:41:44.058 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:41:44.089 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:41:44.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:43 smithi166 bash[9446]: cluster 2020-09-08T15:41:41.654610+0000 mgr.x (mgr.24118) 428 : cluster [DBG] pgmap v555: 17 pgs: 17 active+clean; 472 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 679 op/s 2020-09-08T15:41:44.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:43 smithi166 bash[9446]: cluster 2020-09-08T15:41:42.473577+0000 osd.11 (osd.11) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:44.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:43 smithi166 bash[9446]: cluster 2020-09-08T15:41:42.473670+0000 osd.11 (osd.11) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:44.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:43 smithi166 bash[9446]: cluster 2020-09-08T15:41:42.659058+0000 mon.a (mon.0) 1099 : cluster [DBG] osdmap e209: 12 total, 12 up, 12 in 2020-09-08T15:41:44.124 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:41:44.159 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:41:44.194 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:41:44.224 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:41:44.232 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:41:44.233 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:41:44.233 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:41:44.234 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:41:44.234 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:41:44.234 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:41:44.234 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:41:44.234 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:41:44.235 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:41:44.235 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:41:44.235 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:41:44.235 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:41:44.317 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:44.422 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:44.547 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:44.652 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:41:44.674 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:41:44 smithi104 bash[15273]: debug 2020-09-08T15:41:44.316+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:44.705 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:41:44 smithi055 bash[14301]: debug 2020-09-08T15:41:44.549+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:44.844 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Removed 147617 objects 2020-09-08T15:41:44.844 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Clean up completed and total clean up time :151.058 2020-09-08T15:41:44.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:44 smithi104 bash[9203]: cluster 2020-09-08T15:41:43.655076+0000 mgr.x (mgr.24118) 429 : cluster [DBG] pgmap v557: 17 pgs: 17 active+clean; 201 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T15:41:44.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:44 smithi104 bash[9203]: audit 2020-09-08T15:41:43.797912+0000 mon.b (mon.1) 82 : audit [DBG] from='client.? v1:172.21.15.104:0/1429754144' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:41:45.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:44 smithi166 bash[9446]: cluster 2020-09-08T15:41:43.655076+0000 mgr.x (mgr.24118) 429 : cluster [DBG] pgmap v557: 17 pgs: 17 active+clean; 201 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T15:41:45.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:44 smithi166 bash[9446]: audit 2020-09-08T15:41:43.797912+0000 mon.b (mon.1) 82 : audit [DBG] from='client.? v1:172.21.15.104:0/1429754144' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:41:45.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:44 smithi055 bash[9296]: cluster 2020-09-08T15:41:43.655076+0000 mgr.x (mgr.24118) 429 : cluster [DBG] pgmap v557: 17 pgs: 17 active+clean; 201 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T15:41:45.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:44 smithi055 bash[9296]: audit 2020-09-08T15:41:43.797912+0000 mon.b (mon.1) 82 : audit [DBG] from='client.? v1:172.21.15.104:0/1429754144' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:41:45.242 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:41:45.344 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:45.475 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:45.582 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:45.596 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:41:45 smithi104 bash[17588]: debug 2020-09-08T15:41:45.240+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:45.659 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:41:45 smithi055 bash[11956]: debug 2020-09-08T15:41:45.477+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:45.876 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:46.102 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:46.179 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:41:45 smithi104 bash[15273]: debug 2020-09-08T15:41:45.876+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:46.348 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:46.454 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:41:46.646 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:41:46 smithi104 bash[19922]: debug 2020-09-08T15:41:46.348+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:46.647 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:41:46.750 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:46.804 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_0 2020-09-08T15:41:46.805 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2020-09-08T15:41:46.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:41:46 smithi104 bash[17588]: debug 2020-09-08T15:41:46.644+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:46.934 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:46 smithi104 bash[9203]: cluster 2020-09-08T15:41:45.655717+0000 mgr.x (mgr.24118) 430 : cluster [DBG] pgmap v558: 17 pgs: 17 active+clean; 161 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 804 op/s 2020-09-08T15:41:46.950 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:41:47.035 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:46 smithi166 bash[9446]: cluster 2020-09-08T15:41:45.655717+0000 mgr.x (mgr.24118) 430 : cluster [DBG] pgmap v558: 17 pgs: 17 active+clean; 161 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 804 op/s 2020-09-08T15:41:47.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:46 smithi055 bash[9296]: cluster 2020-09-08T15:41:45.655717+0000 mgr.x (mgr.24118) 430 : cluster [DBG] pgmap v558: 17 pgs: 17 active+clean; 161 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 804 op/s 2020-09-08T15:41:47.225 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T15:41:47.226 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2020-09-08T15:41:47.299 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:41:47.299 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:41:47.737 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:41:47.975 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2020-09-08T15:41:47.989 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:47 smithi166 bash[9446]: cluster 2020-09-08T15:41:46.565475+0000 osd.4 (osd.4) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:47.989 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:47 smithi166 bash[9446]: cluster 2020-09-08T15:41:46.565601+0000 osd.4 (osd.4) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:47.990 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:47 smithi166 bash[9446]: cluster 2020-09-08T15:41:46.726405+0000 mon.a (mon.0) 1100 : cluster [DBG] osdmap e210: 12 total, 12 up, 12 in 2020-09-08T15:41:47.990 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:47 smithi166 bash[9446]: audit 2020-09-08T15:41:46.967087+0000 mon.a (mon.0) 1101 : audit [INF] from='client.? v1:172.21.15.104:0/3792835812' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:41:47.990 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:47 smithi166 bash[9446]: audit 2020-09-08T15:41:47.211033+0000 mgr.x (mgr.24118) 431 : audit [DBG] from='client.15141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:41:47.996 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:41:48.009 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:47 smithi104 bash[9203]: cluster 2020-09-08T15:41:46.565475+0000 osd.4 (osd.4) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:48.009 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:47 smithi104 bash[9203]: cluster 2020-09-08T15:41:46.565601+0000 osd.4 (osd.4) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:48.010 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:47 smithi104 bash[9203]: cluster 2020-09-08T15:41:46.726405+0000 mon.a (mon.0) 1100 : cluster [DBG] osdmap e210: 12 total, 12 up, 12 in 2020-09-08T15:41:48.010 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:47 smithi104 bash[9203]: audit 2020-09-08T15:41:46.967087+0000 mon.a (mon.0) 1101 : audit [INF] from='client.? v1:172.21.15.104:0/3792835812' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:41:48.010 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:47 smithi104 bash[9203]: audit 2020-09-08T15:41:47.211033+0000 mgr.x (mgr.24118) 431 : audit [DBG] from='client.15141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:41:48.101 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:41:48.131 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:41:48.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:47 smithi055 bash[9296]: cluster 2020-09-08T15:41:46.565475+0000 osd.4 (osd.4) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:48.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:47 smithi055 bash[9296]: cluster 2020-09-08T15:41:46.565601+0000 osd.4 (osd.4) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:48.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:47 smithi055 bash[9296]: cluster 2020-09-08T15:41:46.726405+0000 mon.a (mon.0) 1100 : cluster [DBG] osdmap e210: 12 total, 12 up, 12 in 2020-09-08T15:41:48.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:47 smithi055 bash[9296]: audit 2020-09-08T15:41:46.967087+0000 mon.a (mon.0) 1101 : audit [INF] from='client.? v1:172.21.15.104:0/3792835812' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:41:48.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:47 smithi055 bash[9296]: audit 2020-09-08T15:41:47.211033+0000 mgr.x (mgr.24118) 431 : audit [DBG] from='client.15141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:41:48.222 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:48.330 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:48.593 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:48.603 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:41:48 smithi166 bash[18960]: debug 2020-09-08T15:41:48.595+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:48.657 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:41:48 smithi055 bash[11956]: debug 2020-09-08T15:41:48.225+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:48.683 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":559,"stamp":"2020-09-08T15:41:46.724412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":132317208,"num_objects":2023,"num_object_clones":0,"num_object_copies":4046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2023,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293219,"num_write_kb":9447492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160922,"ondisk_log_size":160922,"up":34,"acting":34,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":972784,"kb_used_data":280336,"kb_used_omap":0,"kb_used_meta":692352,"kb_avail":1123969040,"statfs":{"total":1151940427776,"available":1150944296960,"internally_reserved":0,"allocated":287064064,"data_stored":283859460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":708968448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,6,1,4,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-362151936,"num_objects":-5526,"num_object_clones":0,"num_object_copies":-11052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5526,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.070517"},"pg_stats":[{"pgid":"2.f","version":"208'18013","reported_seq":"18108","reported_epoch":"208","state":"active+clean","last_fresh":"2020-09-08T15:41:41.879944+0000","last_change":"2020-09-08T15:41:07.772966+0000","last_active":"2020-09-08T15:41:41.879944+0000","last_peered":"2020-09-08T15:41:41.879944+0000","last_clean":"2020-09-08T15:41:41.879944+0000","last_became_active":"2020-09-08T15:37:29.882656+0000","last_became_peered":"2020-09-08T15:37:29.882656+0000","last_unstale":"2020-09-08T15:41:41.879944+0000","last_undegraded":"2020-09-08T15:41:41.879944+0000","last_fullsized":"2020-09-08T15:41:41.879944+0000","mapping_epoch":126,"log_start":"159'8000","ondisk_log_start":"159'8000","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15972","last_scrub_stamp":"2020-09-08T15:41:07.772896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:41:07.772896+0000","log_size":10013,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12648448,"num_objects":193,"num_object_clones":0,"num_object_copies":386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18013,"num_write_kb":582592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9],"acting":[8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"208'18259","reported_seq":"18354","reported_epoch":"208","state":"active+clean","last_fresh":"2020-09-08T15:41:41.839122+0000","last_change":"2020-09-08T15:41:05.185480+0000","last_active":"2020-09-08T15:41:41.839122+0000","last_peered":"2020-09-08T15:41:41.839122+0000","last_clean":"2020-09-08T15:41:41.839122+0000","last_became_active":"2020-09-08T15:37:30.089439+0000","last_became_peered":"2020-09-08T15:37:30.089439+0000","last_unstale":"2020-09-08T15:41:41.839122+0000","last_undegraded":"2020-09-08T15:41:41.839122+0000","last_fullsized":"2020-09-08T15:41:41.839122+0000","mapping_epoch":126,"log_start":"160'8200","ondisk_log_start":"160'8200","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'16039","last_scrub_stamp":"2020-09-08T15:41:05.185405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:41:05.185405+0000","log_size":10059,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12910592,"num_objects":197,"num_object_clones":0,"num_object_copies":394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18259,"num_write_kb":590592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"209'18428","reported_seq":"18524","reported_epoch":"209","state":"active+clean","last_fresh":"2020-09-08T15:41:44.790651+0000","last_change":"2020-09-08T15:40:36.687544+0000","last_active":"2020-09-08T15:41:44.790651+0000","last_peered":"2020-09-08T15:41:44.790651+0000","last_clean":"2020-09-08T15:41:44.790651+0000","last_became_active":"2020-09-08T15:37:29.876533+0000","last_became_peered":"2020-09-08T15:37:29.876533+0000","last_unstale":"2020-09-08T15:41:44.790651+0000","last_undegraded":"2020-09-08T15:41:44.790651+0000","last_fullsized":"2020-09-08T15:41:44.790651+0000","mapping_epoch":126,"log_start":"160'8400","ondisk_log_start":"160'8400","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"190'14202","last_scrub_stamp":"2020-09-08T15:40:36.687465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:36.687465+0000","log_size":10028,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18428,"num_write_kb":589696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"208'18497","reported_seq":"18592","reported_epoch":"208","state":"active+clean","last_fresh":"2020-09-08T15:41:41.820343+0000","last_change":"2020-09-08T15:40:35.304847+0000","last_active":"2020-09-08T15:41:41.820343+0000","last_peered":"2020-09-08T15:41:41.820343+0000","last_clean":"2020-09-08T15:41:41.820343+0000","last_became_active":"2020-09-08T15:37:30.091928+0000","last_became_peered":"2020-09-08T15:37:30.091928+0000","last_unstale":"2020-09-08T15:41:41.820343+0000","last_undegraded":"2020-09-08T15:41:41.820343+0000","last_fullsized":"2020-09-08T15:41:41.820343+0000","mapping_epoch":126,"log_start":"160'8400","ondisk_log_start":"160'8400","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"190'14394","last_scrub_stamp":"2020-09-08T15:40:35.304761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:35.304761+0000","log_size":10097,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13172736,"num_objects":201,"num_object_clones":0,"num_object_copies":402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18497,"num_write_kb":598336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"209'18378","reported_seq":"18474","reported_epoch":"209","state":"active+clean","last_fresh":"2020-09-08T15:41:44.840241+0000","last_change":"2020-09-08T15:41:03.134319+0000","last_active":"2020-09-08T15:41:44.840241+0000","last_peered":"2020-09-08T15:41:44.840241+0000","last_clean":"2020-09-08T15:41:44.840241+0000","last_became_active":"2020-09-08T15:37:29.875715+0000","last_became_peered":"2020-09-08T15:37:29.875715+0000","last_unstale":"2020-09-08T15:41:44.840241+0000","last_undegraded":"2020-09-08T15:41:44.840241+0000","last_fullsized":"2020-09-08T15:41:44.840241+0000","mapping_epoch":126,"log_start":"160'8300","ondisk_log_start":"160'8300","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'15995","last_scrub_stamp":"2020-09-08T15:41:03.134240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:41:03.134240+0000","log_size":10078,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18378,"num_write_kb":588096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"99'6","reported_seq":"225","reported_epoch":"208","state":"active+clean","last_fresh":"2020-09-08T15:41:40.651393+0000","last_change":"2020-09-08T15:41:03.497368+0000","last_active":"2020-09-08T15:41:40.651393+0000","last_peered":"2020-09-08T15:41:40.651393+0000","last_clean":"2020-09-08T15:41:40.651393+0000","last_became_active":"2020-09-08T15:34:43.031697+0000","last_became_peered":"2020-09-08T15:34:43.031697+0000","last_unstale":"2020-09-08T15:41:40.651393+0000","last_undegraded":"2020-09-08T15:41:40.651393+0000","last_fullsized":"2020-09-08T15:41:40.651393+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":2,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'6","last_scrub_stamp":"2020-09-08T15:41:03.497248+0000","last_deep_scrub":"99'6","last_deep_scrub_stamp":"2020-09-08T15:35:23.132465+0000","last_clean_scrub_stamp":"2020-09-08T15:41:03.497248+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"209'18419","reported_seq":"18515","reported_epoch":"209","state":"active+clean","last_fresh":"2020-09-08T15:41:43.640662+0000","last_change":"2020-09-08T15:40:14.389420+0000","last_active":"2020-09-08T15:41:43.640662+0000","last_peered":"2020-09-08T15:41:43.640662+0000","last_clean":"2020-09-08T15:41:43.640662+0000","last_became_active":"2020-09-08T15:37:30.094918+0000","last_became_peered":"2020-09-08T15:37:30.094918+0000","last_unstale":"2020-09-08T15:41:43.640662+0000","last_undegraded":"2020-09-08T15:41:43.640662+0000","last_fullsized":"2020-09-08T15:41:43.640662+0000","mapping_epoch":126,"log_start":"160'8400","ondisk_log_start":"160'8400","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'12937","last_scrub_stamp":"2020-09-08T15:40:14.389373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:14.389373+0000","log_size":10019,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18419,"num_write_kb":592128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"208'18093","reported_seq":"18188","reported_epoch":"208","state":"active+clean","last_fresh":"2020-09-08T15:41:41.866001+0000","last_change":"2020-09-08T15:41:12.748428+0000","last_active":"2020-09-08T15:41:41.866001+0000","last_peered":"2020-09-08T15:41:41.866001+0000","last_clean":"2020-09-08T15:41:41.866001+0000","last_became_active":"2020-09-08T15:37:30.090699+0000","last_became_peered":"2020-09-08T15:37:30.090699+0000","last_unstale":"2020-09-08T15:41:41.866001+0000","last_undegraded":"2020-09-08T15:41:41.866001+0000","last_fullsized":"2020-09-08T15:41:41.866001+0000","mapping_epoch":126,"log_start":"159'8000","ondisk_log_start":"159'8000","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'16309","last_scrub_stamp":"2020-09-08T15:41:12.748343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:41:12.748343+0000","log_size":10093,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11730944,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18093,"num_write_kb":584704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.6","version":"208'18256","reported_seq":"18353","reported_epoch":"208","state":"active+clean","last_fresh":"2020-09-08T15:41:41.853873+0000","last_change":"2020-09-08T15:40:26.436714+0000","last_active":"2020-09-08T15:41:41.853873+0000","last_peered":"2020-09-08T15:41:41.853873+0000","last_clean":"2020-09-08T15:41:41.853873+0000","last_became_active":"2020-09-08T15:37:29.889471+0000","last_became_peered":"2020-09-08T15:37:29.889471+0000","last_unstale":"2020-09-08T15:41:41.853873+0000","last_undegraded":"2020-09-08T15:41:41.853873+0000","last_fullsized":"2020-09-08T15:41:41.853873+0000","mapping_epoch":126,"log_start":"160'8200","ondisk_log_start":"160'8200","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'13713","last_scrub_stamp":"2020-09-08T15:40:26.436603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:26.436603+0000","log_size":10056,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11665408,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18256,"num_write_kb":589888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"209'18528","reported_seq":"18624","reported_epoch":"209","state":"active+clean","last_fresh":"2020-09-08T15:41:44.807591+0000","last_change":"2020-09-08T15:40:23.781832+0000","last_active":"2020-09-08T15:41:44.807591+0000","last_peered":"2020-09-08T15:41:44.807591+0000","last_clean":"2020-09-08T15:41:44.807591+0000","last_became_active":"2020-09-08T15:37:30.095291+0000","last_became_peered":"2020-09-08T15:37:30.095291+0000","last_unstale":"2020-09-08T15:41:44.807591+0000","last_undegraded":"2020-09-08T15:41:44.807591+0000","last_fullsized":"2020-09-08T15:41:44.807591+0000","mapping_epoch":126,"log_start":"160'8500","ondisk_log_start":"160'8500","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'13605","last_scrub_stamp":"2020-09-08T15:40:23.781740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:23.781740+0000","log_size":10028,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18528,"num_write_kb":592896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.8","version":"209'18239","reported_seq":"18335","reported_epoch":"209","state":"active+clean","last_fresh":"2020-09-08T15:41:43.649019+0000","last_change":"2020-09-08T15:41:03.288848+0000","last_active":"2020-09-08T15:41:43.649019+0000","last_peered":"2020-09-08T15:41:43.649019+0000","last_clean":"2020-09-08T15:41:43.649019+0000","last_became_active":"2020-09-08T15:37:29.881507+0000","last_became_peered":"2020-09-08T15:37:29.881507+0000","last_unstale":"2020-09-08T15:41:43.649019+0000","last_undegraded":"2020-09-08T15:41:43.649019+0000","last_fullsized":"2020-09-08T15:41:43.649019+0000","mapping_epoch":126,"log_start":"160'8200","ondisk_log_start":"160'8200","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'15829","last_scrub_stamp":"2020-09-08T15:41:03.288750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:41:03.288750+0000","log_size":10039,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18239,"num_write_kb":586048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9],"acting":[7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.9","version":"208'18140","reported_seq":"18235","reported_epoch":"208","state":"active+clean","last_fresh":"2020-09-08T15:41:41.813102+0000","last_change":"2020-09-08T15:40:31.090354+0000","last_active":"2020-09-08T15:41:41.813102+0000","last_peered":"2020-09-08T15:41:41.813102+0000","last_clean":"2020-09-08T15:41:41.813102+0000","last_became_active":"2020-09-08T15:37:29.879357+0000","last_became_peered":"2020-09-08T15:37:29.879357+0000","last_unstale":"2020-09-08T15:41:41.813102+0000","last_undegraded":"2020-09-08T15:41:41.813102+0000","last_fullsized":"2020-09-08T15:41:41.813102+0000","mapping_epoch":126,"log_start":"160'8100","ondisk_log_start":"160'8100","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'13830","last_scrub_stamp":"2020-09-08T15:40:31.090293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:31.090293+0000","log_size":10040,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12320768,"num_objects":188,"num_object_clones":0,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":586496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"209'18385","reported_seq":"18481","reported_epoch":"209","state":"active+clean","last_fresh":"2020-09-08T15:41:42.733161+0000","last_change":"2020-09-08T15:40:30.575755+0000","last_active":"2020-09-08T15:41:42.733161+0000","last_peered":"2020-09-08T15:41:42.733161+0000","last_clean":"2020-09-08T15:41:42.733161+0000","last_became_active":"2020-09-08T15:37:30.094030+0000","last_became_peered":"2020-09-08T15:37:30.094030+0000","last_unstale":"2020-09-08T15:41:42.733161+0000","last_undegraded":"2020-09-08T15:41:42.733161+0000","last_fullsized":"2020-09-08T15:41:42.733161+0000","mapping_epoch":126,"log_start":"160'8300","ondisk_log_start":"160'8300","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'13996","last_scrub_stamp":"2020-09-08T15:40:30.575701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:30.575701+0000","log_size":10085,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7798784,"num_objects":119,"num_object_clones":0,"num_object_copies":238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18385,"num_write_kb":592128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3],"acting":[11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.b","version":"208'18300","reported_seq":"18395","reported_epoch":"208","state":"active+clean","last_fresh":"2020-09-08T15:41:41.880332+0000","last_change":"2020-09-08T15:41:11.741706+0000","last_active":"2020-09-08T15:41:41.880332+0000","last_peered":"2020-09-08T15:41:41.880332+0000","last_clean":"2020-09-08T15:41:41.880332+0000","last_became_active":"2020-09-08T15:37:29.879505+0000","last_became_peered":"2020-09-08T15:37:29.879505+0000","last_unstale":"2020-09-08T15:41:41.880332+0000","last_undegraded":"2020-09-08T15:41:41.880332+0000","last_fullsized":"2020-09-08T15:41:41.880332+0000","mapping_epoch":126,"log_start":"159'8200","ondisk_log_start":"159'8200","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'16428","last_scrub_stamp":"2020-09-08T15:41:11.741633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:41:11.741633+0000","log_size":10100,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11272192,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18300,"num_write_kb":591104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.c","version":"209'18169","reported_seq":"18264","reported_epoch":"209","state":"active+clean","last_fresh":"2020-09-08T15:41:42.777098+0000","last_change":"2020-09-08T15:40:15.718317+0000","last_active":"2020-09-08T15:41:42.777098+0000","last_peered":"2020-09-08T15:41:42.777098+0000","last_clean":"2020-09-08T15:41:42.777098+0000","last_became_active":"2020-09-08T15:37:30.093315+0000","last_became_peered":"2020-09-08T15:37:30.093315+0000","last_unstale":"2020-09-08T15:41:42.777098+0000","last_undegraded":"2020-09-08T15:41:42.777098+0000","last_fullsized":"2020-09-08T15:41:42.777098+0000","mapping_epoch":126,"log_start":"160'8100","ondisk_log_start":"160'8100","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'12953","last_scrub_stamp":"2020-09-08T15:40:15.718177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:40:15.718177+0000","log_size":10069,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8978432,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18169,"num_write_kb":585792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0],"acting":[11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.d","version":"209'18530","reported_seq":"18627","reported_epoch":"209","state":"active+clean","last_fresh":"2020-09-08T15:41:42.888726+0000","last_change":"2020-09-08T15:41:12.199062+0000","last_active":"2020-09-08T15:41:42.888726+0000","last_peered":"2020-09-08T15:41:42.888726+0000","last_clean":"2020-09-08T15:41:42.888726+0000","last_became_active":"2020-09-08T15:37:30.092910+0000","last_became_peered":"2020-09-08T15:37:30.092910+0000","last_unstale":"2020-09-08T15:41:42.888726+0000","last_undegraded":"2020-09-08T15:41:42.888726+0000","last_fullsized":"2020-09-08T15:41:42.888726+0000","mapping_epoch":126,"log_start":"160'8500","ondisk_log_start":"160'8500","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'16508","last_scrub_stamp":"2020-09-08T15:41:12.199008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:41:12.199008+0000","log_size":10030,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7012376,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18530,"num_write_kb":596353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.e","version":"208'18582","reported_seq":"18679","reported_epoch":"208","state":"active+clean","last_fresh":"2020-09-08T15:41:41.852284+0000","last_change":"2020-09-08T15:41:07.596185+0000","last_active":"2020-09-08T15:41:41.852284+0000","last_peered":"2020-09-08T15:41:41.852284+0000","last_clean":"2020-09-08T15:41:41.852284+0000","last_became_active":"2020-09-08T15:37:30.093099+0000","last_became_peered":"2020-09-08T15:37:30.093099+0000","last_unstale":"2020-09-08T15:41:41.852284+0000","last_undegraded":"2020-09-08T15:41:41.852284+0000","last_fullsized":"2020-09-08T15:41:41.852284+0000","mapping_epoch":126,"log_start":"160'8500","ondisk_log_start":"160'8500","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'16496","last_scrub_stamp":"2020-09-08T15:41:07.596111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:37:28.858710+0000","last_clean_scrub_stamp":"2020-09-08T15:41:07.596111+0000","log_size":10082,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12320768,"num_objects":188,"num_object_clones":0,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18582,"num_write_kb":600640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":132317208,"num_objects":2020,"num_object_clones":0,"num_object_copies":4040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2020,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293216,"num_write_kb":9447489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":214638592,"data_stored":214630448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":32,"acting":32,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":2,"acting":2,"num_store_stats":7}],"osd_stats":[{"osd":11,"up_from":0,"seq":88,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":93448,"kb_used_data":34560,"kb_used_omap":0,"kb_used_meta":58880,"kb_avail":93651704,"statfs":{"total":95995035648,"available":95899344896,"internally_reserved":0,"allocated":35389440,"data_stored":35125259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":60293120},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":2.9119999999999999,"5min":2.2650000000000001,"15min":1.988},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":33.677999999999997,"5min":33.677999999999997,"15min":33.677999999999997},"last":0.871},{"interface":"front","average":{"1min":3.6440000000000001,"5min":3.0649999999999999,"15min":3.2290000000000001},"min":{"1min":1.1930000000000001,"5min":1.022,"15min":1.022},"max":{"1min":34.058,"5min":34.058,"15min":34.058},"last":1.47}]},{"osd":1,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":10.811,"5min":4.2930000000000001,"15min":2.8039999999999998},"min":{"1min":0.78800000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":163.51499999999999,"5min":163.51499999999999,"15min":163.51499999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":11.222,"5min":4.7359999999999998,"15min":3.3399999999999999},"min":{"1min":1.2789999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":166.94300000000001,"5min":166.94300000000001,"15min":166.94300000000001},"last":1.839}]},{"osd":2,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":3.5569999999999999,"5min":3.1539999999999999,"15min":2.4390000000000001},"min":{"1min":0.84599999999999997,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":19.966999999999999,"5min":36.637999999999998,"15min":36.637999999999998},"last":1.9910000000000001},{"interface":"front","average":{"1min":3.3780000000000001,"5min":3.464,"15min":3.0659999999999998},"min":{"1min":1.0309999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":20.390000000000001,"5min":36.694000000000003,"15min":36.694000000000003},"last":1.72}]},{"osd":3,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":2.2989999999999999,"5min":4.0540000000000003,"15min":2.9039999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":4.6520000000000001,"5min":147.49100000000001,"15min":147.49100000000001},"last":1.177},{"interface":"front","average":{"1min":2.5,"5min":4.4370000000000003,"15min":3.5699999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":5.3460000000000001,"5min":150.32400000000001,"15min":150.32400000000001},"last":2.1709999999999998}]},{"osd":4,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":2.2610000000000001,"5min":2.8679999999999999,"15min":3.3620000000000001},"min":{"1min":0.95299999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":4.9009999999999998,"5min":8.0289999999999999,"15min":8.0289999999999999},"last":1.663},{"interface":"front","average":{"1min":2.6739999999999999,"5min":2.738,"15min":2.7829999999999999},"min":{"1min":0.92300000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":7.6609999999999996,"5min":7.7679999999999998,"15min":7.7679999999999998},"last":1.3560000000000001}]},{"osd":5,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":2.5870000000000002,"5min":2.8980000000000001,"15min":3.2559999999999998},"min":{"1min":1.1699999999999999,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":5.2370000000000001,"5min":6.8499999999999996,"15min":6.8499999999999996},"last":1.3200000000000001},{"interface":"front","average":{"1min":2.968,"5min":3.198,"15min":3.7320000000000002},"min":{"1min":1.262,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":6.6079999999999997,"5min":8.5289999999999999,"15min":8.5289999999999999},"last":2.04}]},{"osd":6,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":2.9300000000000002,"5min":2.9700000000000002,"15min":3.2799999999999998},"min":{"1min":1.167,"5min":1.036,"15min":1.036},"max":{"1min":5.8109999999999999,"5min":7.8479999999999999,"15min":7.8479999999999999},"last":1.502},{"interface":"front","average":{"1min":2.6579999999999999,"5min":3.2400000000000002,"15min":3.4940000000000002},"min":{"1min":1.125,"5min":1.125,"15min":1.125},"max":{"1min":5.7800000000000002,"5min":8.2110000000000003,"15min":8.2110000000000003},"last":2.573}]},{"osd":7,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":3.0339999999999998,"5min":3.351,"15min":3.6230000000000002},"min":{"1min":1.2749999999999999,"5min":1.22,"15min":1.22},"max":{"1min":8.3100000000000005,"5min":8.4610000000000003,"15min":8.4610000000000003},"last":1.6180000000000001},{"interface":"front","average":{"1min":3.5350000000000001,"5min":3.4660000000000002,"15min":3.7490000000000001},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":9.1820000000000004,"5min":9.1820000000000004,"15min":9.1820000000000004},"last":1.8740000000000001}]},{"osd":8,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":4.2619999999999996,"5min":3.7599999999999998,"15min":4.0119999999999996},"min":{"1min":1.246,"5min":1.246,"15min":1.246},"max":{"1min":13.018000000000001,"5min":13.018000000000001,"15min":13.018000000000001},"last":2.4449999999999998},{"interface":"front","average":{"1min":4.5970000000000004,"5min":3.8700000000000001,"15min":3.8940000000000001},"min":{"1min":1.5069999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":13.592000000000001,"5min":13.592000000000001,"15min":13.592000000000001},"last":2.7080000000000002}]},{"osd":10,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":3.355,"5min":3.6110000000000002,"15min":4.202},"min":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"max":{"1min":9.7439999999999998,"5min":9.7439999999999998,"15min":9.7439999999999998},"last":2.3050000000000002},{"interface":"front","average":{"1min":2.9950000000000001,"5min":3.5670000000000002,"15min":4.0490000000000004},"min":{"1min":1.085,"5min":1.085,"15min":1.085},"max":{"1min":8.4260000000000002,"5min":8.6600000000000001,"15min":8.6600000000000001},"last":1.75}]}]},{"osd":10,"up_from":0,"seq":95,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":72776,"kb_used_data":21760,"kb_used_omap":0,"kb_used_meta":51008,"kb_avail":93672376,"statfs":{"total":95995035648,"available":95920513024,"internally_reserved":0,"allocated":22282240,"data_stored":22018059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":52232192},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7330000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:41:35 2020","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":2.7959999999999998,"15min":2.5059999999999998},"min":{"1min":0.64600000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":6.3970000000000002,"5min":64.471999999999994,"15min":64.471999999999994},"last":1.792},{"interface":"front","average":{"1min":1.891,"5min":3.1739999999999999,"15min":2.7690000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":5.8650000000000002,"5min":64.707999999999998,"15min":64.707999999999998},"last":0.96999999999999997}]},{"osd":2,"last update":"Tue Sep 8 15:41:35 2020","interfaces":[{"interface":"back","average":{"1min":6.4059999999999997,"5min":3.4119999999999999,"15min":2.6400000000000001},"min":{"1min":0.76000000000000001,"5min":0.745,"15min":0.745},"max":{"1min":95.197999999999993,"5min":95.197999999999993,"15min":95.197999999999993},"last":1.339},{"interface":"front","average":{"1min":6.2480000000000002,"5min":3.2599999999999998,"15min":2.9249999999999998},"min":{"1min":0.99199999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":94.778999999999996,"5min":94.778999999999996,"15min":94.778999999999996},"last":1.1599999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:41:35 2020","interfaces":[{"interface":"back","average":{"1min":1.7490000000000001,"5min":4.5220000000000002,"15min":3.004},"min":{"1min":0.80800000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":4.0830000000000002,"5min":232.87299999999999,"15min":232.87299999999999},"last":3.1200000000000001},{"interface":"front","average":{"1min":2.6859999999999999,"5min":5.165,"15min":3.7570000000000001},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":9.1929999999999996,"5min":237.75800000000001,"15min":237.75800000000001},"last":1.038}]},{"osd":4,"last update":"Tue Sep 8 15:41:35 2020","interfaces":[{"interface":"back","average":{"1min":1.919,"5min":2.6139999999999999,"15min":3.1619999999999999},"min":{"1min":1.026,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":4.7480000000000002,"5min":38.237000000000002,"15min":38.237000000000002},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.95,"5min":2.649,"15min":2.9329999999999998},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":3.7250000000000001,"5min":37.786000000000001,"15min":37.786000000000001},"last":5.4649999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:41:35 2020","interfaces":[{"interface":"back","average":{"1min":2.0590000000000002,"5min":2.843,"15min":3.343},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":6.2709999999999999,"5min":36.493000000000002,"15min":36.493000000000002},"last":1.276},{"interface":"front","average":{"1min":2.2160000000000002,"5min":2.9609999999999999,"15min":3.5470000000000002},"min":{"1min":1.145,"5min":1.145,"15min":1.145},"max":{"1min":4.8869999999999996,"5min":38.957999999999998,"15min":38.957999999999998},"last":1.585}]},{"osd":6,"last update":"Tue Sep 8 15:41:35 2020","interfaces":[{"interface":"back","average":{"1min":2.1829999999999998,"5min":2.8860000000000001,"15min":3.335},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":4.54,"5min":38.537999999999997,"15min":38.537999999999997},"last":5.1820000000000004},{"interface":"front","average":{"1min":2.343,"5min":3.0470000000000002,"15min":3.605},"min":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"max":{"1min":7.3570000000000002,"5min":37.698999999999998,"15min":37.698999999999998},"last":1.385}]},{"osd":7,"last update":"Tue Sep 8 15:41:35 2020","interfaces":[{"interface":"back","average":{"1min":2.3849999999999998,"5min":3.1659999999999999,"15min":3.6739999999999999},"min":{"1min":1.026,"5min":1.026,"15min":1.026},"max":{"1min":4.8579999999999997,"5min":39.100000000000001,"15min":39.100000000000001},"last":5.6719999999999997},{"interface":"front","average":{"1min":2.4790000000000001,"5min":3.2629999999999999,"15min":3.8199999999999998},"min":{"1min":1.3939999999999999,"5min":1.3140000000000001,"15min":1.3140000000000001},"max":{"1min":7.4790000000000001,"5min":38.131999999999998,"15min":38.131999999999998},"last":5.0640000000000001}]},{"osd":8,"last update":"Tue Sep 8 15:41:35 2020","interfaces":[{"interface":"back","average":{"1min":2.4940000000000002,"5min":3.464,"15min":3.9289999999999998},"min":{"1min":1.181,"5min":1.181,"15min":1.181},"max":{"1min":5.6680000000000001,"5min":42.031999999999996,"15min":42.031999999999996},"last":2.2210000000000001},{"interface":"front","average":{"1min":2.98,"5min":3.5680000000000001,"15min":3.8740000000000001},"min":{"1min":1.113,"5min":1.113,"15min":1.113},"max":{"1min":5.1070000000000002,"5min":39.512999999999998,"15min":39.512999999999998},"last":5.9009999999999998}]},{"osd":9,"last update":"Tue Sep 8 15:41:35 2020","interfaces":[{"interface":"back","average":{"1min":2.569,"5min":3.6499999999999999,"15min":4.149},"min":{"1min":1.4830000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":5.5359999999999996,"5min":42.603999999999999,"15min":42.603999999999999},"last":5.3559999999999999},{"interface":"front","average":{"1min":2.7330000000000001,"5min":3.5590000000000002,"15min":3.992},"min":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"max":{"1min":6.2050000000000001,"5min":40.029000000000003,"15min":40.029000000000003},"last":2.5179999999999998}]},{"osd":11,"last update":"Tue Sep 8 15:41:02 2020","interfaces":[{"interface":"back","average":{"1min":2.956,"5min":4.1040000000000001,"15min":4.2939999999999996},"min":{"1min":1.3520000000000001,"5min":1.3009999999999999,"15min":1.3009999999999999},"max":{"1min":8.9510000000000005,"5min":39.271000000000001,"15min":39.271000000000001},"last":5.79},{"interface":"front","average":{"1min":2.7559999999999998,"5min":3.8170000000000002,"15min":4.2210000000000001},"min":{"1min":1.385,"5min":1.3779999999999999,"15min":1.3779999999999999},"max":{"1min":5.4930000000000003,"5min":42.329999999999998,"15min":42.329999999999998},"last":1.948}]}]},{"osd":9,"up_from":0,"seq":98,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":86092,"kb_used_data":27076,"kb_used_omap":0,"kb_used_meta":59008,"kb_avail":93659060,"statfs":{"total":95995035648,"available":95906877440,"internally_reserved":0,"allocated":27725824,"data_stored":27457571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":60424192},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:41:09 2020","interfaces":[{"interface":"back","average":{"1min":11.460000000000001,"5min":3.9140000000000001,"15min":2.5609999999999999},"min":{"1min":0.746,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":180.84899999999999,"5min":180.84899999999999,"15min":180.84899999999999},"last":1.651},{"interface":"front","average":{"1min":11.324,"5min":3.7530000000000001,"15min":2.5230000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":175.077,"5min":175.077,"15min":175.077},"last":1.1559999999999999}]},{"osd":1,"last update":"Tue Sep 8 15:41:09 2020","interfaces":[{"interface":"back","average":{"1min":2.0880000000000001,"5min":2.133,"15min":2.024},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":8.8249999999999993,"5min":9.5519999999999996,"15min":9.5519999999999996},"last":1.802},{"interface":"front","average":{"1min":2.6219999999999999,"5min":2.4529999999999998,"15min":2.3540000000000001},"min":{"1min":0.94499999999999995,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":7.5439999999999996,"5min":9.4250000000000007,"15min":9.4250000000000007},"last":1.4019999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:41:09 2020","interfaces":[{"interface":"back","average":{"1min":1.968,"5min":2.2069999999999999,"15min":2.3999999999999999},"min":{"1min":1.113,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":6.5830000000000002,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":1.8959999999999999},{"interface":"front","average":{"1min":1.8740000000000001,"5min":2.306,"15min":2.306},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":4.9530000000000003,"5min":9.4770000000000003,"15min":9.4770000000000003},"last":1.9450000000000001}]},{"osd":3,"last update":"Tue Sep 8 15:41:11 2020","interfaces":[{"interface":"back","average":{"1min":1.8740000000000001,"5min":2.5750000000000002,"15min":2.4220000000000002},"min":{"1min":0.97799999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":5.3079999999999998,"5min":24.562000000000001,"15min":24.562000000000001},"last":2.2429999999999999},{"interface":"front","average":{"1min":2.0819999999999999,"5min":2.7690000000000001,"15min":2.6840000000000002},"min":{"1min":1.004,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":4.9029999999999996,"5min":23.184000000000001,"15min":23.184000000000001},"last":1.278}]},{"osd":4,"last update":"Tue Sep 8 15:41:09 2020","interfaces":[{"interface":"back","average":{"1min":2.5150000000000001,"5min":2.6309999999999998,"15min":2.6909999999999998},"min":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"max":{"1min":6.8129999999999997,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.694},{"interface":"front","average":{"1min":2.266,"5min":2.722,"15min":2.9409999999999998},"min":{"1min":1.3340000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":4.383,"5min":10.201000000000001,"15min":10.201000000000001},"last":1.528}]},{"osd":5,"last update":"Tue Sep 8 15:41:09 2020","interfaces":[{"interface":"back","average":{"1min":2.2610000000000001,"5min":2.7069999999999999,"15min":3.0259999999999998},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":5.0999999999999996,"5min":11.097,"15min":11.097},"last":2.27},{"interface":"front","average":{"1min":2.1699999999999999,"5min":2.8399999999999999,"15min":3.077},"min":{"1min":1.22,"5min":1.0860000000000001,"15min":1.0589999999999999},"max":{"1min":3.4580000000000002,"5min":9.9280000000000008,"15min":9.9280000000000008},"last":2.3690000000000002}]},{"osd":6,"last update":"Tue Sep 8 15:41:09 2020","interfaces":[{"interface":"back","average":{"1min":2.266,"5min":2.8839999999999999,"15min":3.1320000000000001},"min":{"1min":1.1339999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":5.6459999999999999,"5min":10.798,"15min":10.798},"last":2.1160000000000001},{"interface":"front","average":{"1min":2.625,"5min":3.0110000000000001,"15min":3.1829999999999998},"min":{"1min":1.4059999999999999,"5min":1.242,"15min":1.242},"max":{"1min":6.7389999999999999,"5min":11.766,"15min":11.766},"last":2.3929999999999998}]},{"osd":7,"last update":"Tue Sep 8 15:41:09 2020","interfaces":[{"interface":"back","average":{"1min":2.5299999999999998,"5min":3.1629999999999998,"15min":3.351},"min":{"1min":1.458,"5min":1.2110000000000001,"15min":1.2110000000000001},"max":{"1min":5.907,"5min":10.31,"15min":10.31},"last":2.4809999999999999},{"interface":"front","average":{"1min":2.7789999999999999,"5min":3.3959999999999999,"15min":3.5680000000000001},"min":{"1min":1.669,"5min":1.593,"15min":1.593},"max":{"1min":6.016,"5min":10.837999999999999,"15min":10.837999999999999},"last":2.5449999999999999}]},{"osd":8,"last update":"Tue Sep 8 15:41:09 2020","interfaces":[{"interface":"back","average":{"1min":2.8039999999999998,"5min":3.3599999999999999,"15min":3.2669999999999999},"min":{"1min":1.264,"5min":1.264,"15min":1.264},"max":{"1min":6.4409999999999998,"5min":11,"15min":11},"last":9.0429999999999993},{"interface":"front","average":{"1min":2.552,"5min":3.4540000000000002,"15min":3.3620000000000001},"min":{"1min":1.262,"5min":1.262,"15min":1.262},"max":{"1min":4.0090000000000003,"5min":10.977,"15min":10.977},"last":9.3520000000000003}]},{"osd":10,"last update":"Tue Sep 8 15:41:26 2020","interfaces":[{"interface":"back","average":{"1min":3.363,"5min":3.2610000000000001,"15min":3.7090000000000001},"min":{"1min":1.518,"5min":1.2529999999999999,"15min":1.2529999999999999},"max":{"1min":7.0090000000000003,"5min":8.8200000000000003,"15min":8.8200000000000003},"last":2.0089999999999999},{"interface":"front","average":{"1min":2.6749999999999998,"5min":3.1040000000000001,"15min":3.5720000000000001},"min":{"1min":1.165,"5min":1.165,"15min":1.165},"max":{"1min":5.2489999999999997,"5min":11.625,"15min":11.625},"last":2.1469999999999998}]}]},{"osd":8,"up_from":0,"seq":104,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":141640,"kb_used_data":62656,"kb_used_omap":0,"kb_used_meta":78976,"kb_avail":93603512,"statfs":{"total":95995035648,"available":95849996288,"internally_reserved":0,"allocated":64159744,"data_stored":63895563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":80871424},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:41:40 2020","interfaces":[{"interface":"back","average":{"1min":2.5579999999999998,"5min":2.1970000000000001,"15min":2.1960000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":11.154999999999999,"5min":11.154999999999999,"15min":11.154999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.968,"5min":2.3559999999999999,"15min":2.4180000000000001},"min":{"1min":0.73399999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":5.0709999999999997,"5min":13.773999999999999,"15min":13.773999999999999},"last":2.3570000000000002}]},{"osd":1,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.492,"5min":2.0659999999999998,"15min":1.8839999999999999},"min":{"1min":1.4690000000000001,"5min":1.137,"15min":1.137},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":1.4350000000000001},{"interface":"front","average":{"1min":2.7200000000000002,"5min":2.427,"15min":2.347},"min":{"1min":1.351,"5min":1.046,"15min":1.046},"max":{"1min":4.444,"5min":4.444,"15min":4.444},"last":1.8600000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:41:40 2020","interfaces":[{"interface":"back","average":{"1min":8.2219999999999995,"5min":5.5800000000000001,"15min":3.1059999999999999},"min":{"1min":0.79700000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":115.011,"5min":145.893,"15min":145.893},"last":2.7080000000000002},{"interface":"front","average":{"1min":8.3119999999999994,"5min":5.1070000000000002,"15min":3.3500000000000001},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":114.161,"5min":153.91900000000001,"15min":153.91900000000001},"last":0.97299999999999998}]},{"osd":3,"last update":"Tue Sep 8 15:41:40 2020","interfaces":[{"interface":"back","average":{"1min":3.3980000000000001,"5min":4.1719999999999997,"15min":3.2240000000000002},"min":{"1min":1.0169999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":11.539999999999999,"5min":125.111,"15min":125.111},"last":1.1339999999999999},{"interface":"front","average":{"1min":2.0249999999999999,"5min":3.8639999999999999,"15min":2.883},"min":{"1min":0.91000000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":5.181,"5min":126.124,"15min":126.124},"last":2.8460000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:41:40 2020","interfaces":[{"interface":"back","average":{"1min":2.8380000000000001,"5min":2.9209999999999998,"15min":2.9089999999999998},"min":{"1min":0.94899999999999995,"5min":0.872,"15min":0.872},"max":{"1min":11.404999999999999,"5min":11.404999999999999,"15min":11.404999999999999},"last":1.262},{"interface":"front","average":{"1min":2.0760000000000001,"5min":2.5710000000000002,"15min":2.5840000000000001},"min":{"1min":1.0880000000000001,"5min":0.874,"15min":0.874},"max":{"1min":4.0469999999999997,"5min":13.999000000000001,"15min":13.999000000000001},"last":3.1160000000000001}]},{"osd":5,"last update":"Tue Sep 8 15:41:40 2020","interfaces":[{"interface":"back","average":{"1min":2.907,"5min":3.1400000000000001,"15min":3.177},"min":{"1min":1.02,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":9.6349999999999998,"5min":9.6349999999999998,"15min":9.6349999999999998},"last":2.5760000000000001},{"interface":"front","average":{"1min":3.0899999999999999,"5min":3.194,"15min":3.3199999999999998},"min":{"1min":1.071,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":11.885999999999999,"5min":11.885999999999999,"15min":11.885999999999999},"last":1.3839999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:41:40 2020","interfaces":[{"interface":"back","average":{"1min":2.6629999999999998,"5min":3.323,"15min":3.3100000000000001},"min":{"1min":1.238,"5min":1.1299999999999999,"15min":1.1299999999999999},"max":{"1min":9.968,"5min":13.887,"15min":13.887},"last":2.9780000000000002},{"interface":"front","average":{"1min":3.2429999999999999,"5min":2.9390000000000001,"15min":2.9489999999999998},"min":{"1min":1.359,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":11.775,"5min":12.502000000000001,"15min":12.502000000000001},"last":1.4990000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:41:40 2020","interfaces":[{"interface":"back","average":{"1min":3.3130000000000002,"5min":3.5910000000000002,"15min":3.6299999999999999},"min":{"1min":1.3029999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"max":{"1min":11.661,"5min":11.661,"15min":11.661},"last":1.7430000000000001},{"interface":"front","average":{"1min":2.823,"5min":3.2650000000000001,"15min":3.2719999999999998},"min":{"1min":1.4770000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"max":{"1min":10.234,"5min":14.109999999999999,"15min":14.109999999999999},"last":3.242}]},{"osd":9,"last update":"Tue Sep 8 15:41:07 2020","interfaces":[{"interface":"back","average":{"1min":3.1499999999999999,"5min":3.8650000000000002,"15min":3.5819999999999999},"min":{"1min":1.095,"5min":1.095,"15min":1.095},"max":{"1min":11.913,"5min":13.506,"15min":13.506},"last":4.2140000000000004},{"interface":"front","average":{"1min":3.883,"5min":3.8290000000000002,"15min":3.5289999999999999},"min":{"1min":1.004,"5min":1.004,"15min":1.004},"max":{"1min":12.052,"5min":12.052,"15min":12.052},"last":1.865}]},{"osd":10,"last update":"Tue Sep 8 15:41:40 2020","interfaces":[{"interface":"back","average":{"1min":3.02,"5min":3.2090000000000001,"15min":3.5590000000000002},"min":{"1min":1.4730000000000001,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":10.101000000000001,"5min":13.656000000000001,"15min":13.656000000000001},"last":3.355},{"interface":"front","average":{"1min":3.681,"5min":3.726,"15min":3.9609999999999999},"min":{"1min":1.3149999999999999,"5min":1.296,"15min":1.296},"max":{"1min":13.638,"5min":13.638,"15min":13.638},"last":2.8109999999999999}]},{"osd":11,"last update":"Tue Sep 8 15:41:36 2020","interfaces":[{"interface":"back","average":{"1min":4.0460000000000003,"5min":3.6629999999999998,"15min":3.923},"min":{"1min":1.591,"5min":1.214,"15min":1.214},"max":{"1min":14.221,"5min":14.221,"15min":14.221},"last":3.4239999999999999},{"interface":"front","average":{"1min":3.8439999999999999,"5min":4.194,"15min":4.718},"min":{"1min":1.405,"5min":1.248,"15min":1.248},"max":{"1min":9.7599999999999998,"5min":11.186999999999999,"15min":11.186999999999999},"last":2.9550000000000001}]}]},{"osd":7,"up_from":0,"seq":109,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":94996,"kb_used_data":27660,"kb_used_omap":0,"kb_used_meta":67328,"kb_avail":93650156,"statfs":{"total":95995035648,"available":95897759744,"internally_reserved":0,"allocated":28323840,"data_stored":28055271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68943872},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:02 2020","interfaces":[{"interface":"back","average":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"min":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"max":{"1min":4.0190000000000001,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":2.2189999999999999},{"interface":"front","average":{"1min":2.6259999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"min":{"1min":1.542,"5min":1.542,"15min":1.542},"max":{"1min":4.5229999999999997,"5min":4.5229999999999997,"15min":4.5229999999999997},"last":2.3079999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:41:19 2020","interfaces":[{"interface":"back","average":{"1min":2.0369999999999999,"5min":2.0089999999999999,"15min":2.258},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":6.2220000000000004,"5min":6.6219999999999999,"15min":6.6219999999999999},"last":1.77},{"interface":"front","average":{"1min":1.895,"5min":2.048,"15min":2.2559999999999998},"min":{"1min":0.85799999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":3.0720000000000001,"5min":5.3159999999999998,"15min":5.3159999999999998},"last":1.6519999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:41:19 2020","interfaces":[{"interface":"back","average":{"1min":18.271000000000001,"5min":5.7089999999999996,"15min":3.544},"min":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":324.77100000000002,"5min":324.77100000000002,"15min":324.77100000000002},"last":0.96999999999999997},{"interface":"front","average":{"1min":18.981000000000002,"5min":5.79,"15min":3.8279999999999998},"min":{"1min":1.02,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":321.48500000000001,"5min":321.48500000000001,"15min":321.48500000000001},"last":0.93500000000000005}]},{"osd":3,"last update":"Tue Sep 8 15:41:19 2020","interfaces":[{"interface":"back","average":{"1min":2.7069999999999999,"5min":2.883,"15min":2.9039999999999999},"min":{"1min":1.208,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":7.1399999999999997,"5min":37.801000000000002,"15min":37.801000000000002},"last":1.5309999999999999},{"interface":"front","average":{"1min":3.0840000000000001,"5min":3.6560000000000001,"15min":3.3780000000000001},"min":{"1min":0.85599999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":6.2450000000000001,"5min":45.284999999999997,"15min":45.284999999999997},"last":2.5110000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:41:19 2020","interfaces":[{"interface":"back","average":{"1min":2.3420000000000001,"5min":2.3740000000000001,"15min":2.7629999999999999},"min":{"1min":1.1839999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":4.0739999999999998,"5min":8.5779999999999994,"15min":8.5779999999999994},"last":1.6859999999999999},{"interface":"front","average":{"1min":2.9460000000000002,"5min":2.9180000000000001,"15min":3.2040000000000002},"min":{"1min":1.127,"5min":1.127,"15min":1.127},"max":{"1min":6.6740000000000004,"5min":7.3570000000000002,"15min":7.3570000000000002},"last":2.3450000000000002}]},{"osd":5,"last update":"Tue Sep 8 15:41:19 2020","interfaces":[{"interface":"back","average":{"1min":2.726,"5min":2.806,"15min":3.4020000000000001},"min":{"1min":1.373,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":4.327,"5min":9.0800000000000001,"15min":9.0800000000000001},"last":2.7200000000000002},{"interface":"front","average":{"1min":2.9049999999999998,"5min":2.851,"15min":3.2999999999999998},"min":{"1min":1.762,"5min":1.1779999999999999,"15min":1.1779999999999999},"max":{"1min":4.2610000000000001,"5min":6.0229999999999997,"15min":6.0229999999999997},"last":1.845}]},{"osd":6,"last update":"Tue Sep 8 15:41:19 2020","interfaces":[{"interface":"back","average":{"1min":3.0739999999999998,"5min":3.0790000000000002,"15min":3.6499999999999999},"min":{"1min":1.5329999999999999,"5min":1.304,"15min":1.304},"max":{"1min":6.6360000000000001,"5min":8.8300000000000001,"15min":8.8300000000000001},"last":2.1520000000000001},{"interface":"front","average":{"1min":3.4700000000000002,"5min":3.2719999999999998,"15min":3.7189999999999999},"min":{"1min":1.7749999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"max":{"1min":6.4740000000000002,"5min":7.681,"15min":7.681},"last":2.8319999999999999}]},{"osd":8,"last update":"Tue Sep 8 15:40:43 2020","interfaces":[{"interface":"back","average":{"1min":3.3420000000000001,"5min":3.5739999999999998,"15min":3.9729999999999999},"min":{"1min":1.5009999999999999,"5min":1.3340000000000001,"15min":1.3340000000000001},"max":{"1min":8.2260000000000009,"5min":8.2260000000000009,"15min":8.2260000000000009},"last":1.986},{"interface":"front","average":{"1min":3.1840000000000002,"5min":3.665,"15min":4.0830000000000002},"min":{"1min":1.677,"5min":1.6319999999999999,"15min":1.6319999999999999},"max":{"1min":8.4220000000000006,"5min":9.2119999999999997,"15min":9.2119999999999997},"last":2.2829999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:41:02 2020","interfaces":[{"interface":"back","average":{"1min":3.589,"5min":3.8210000000000002,"15min":4.4610000000000003},"min":{"1min":2.1339999999999999,"5min":1.7170000000000001,"15min":1.7170000000000001},"max":{"1min":6.9800000000000004,"5min":7.476,"15min":7.476},"last":3.1200000000000001},{"interface":"front","average":{"1min":3.617,"5min":3.4009999999999998,"15min":4.093},"min":{"1min":2.0459999999999998,"5min":1.4570000000000001,"15min":1.4570000000000001},"max":{"1min":7.7460000000000004,"5min":7.7460000000000004,"15min":7.7460000000000004},"last":3.6829999999999998}]},{"osd":10,"last update":"Tue Sep 8 15:41:31 2020","interfaces":[{"interface":"back","average":{"1min":3.2469999999999999,"5min":3.5369999999999999,"15min":4.3120000000000003},"min":{"1min":2.052,"5min":1.5309999999999999,"15min":1.5309999999999999},"max":{"1min":6.4790000000000001,"5min":9.5009999999999994,"15min":9.5009999999999994},"last":3.3130000000000002},{"interface":"front","average":{"1min":2.9780000000000002,"5min":3.5049999999999999,"15min":4.6029999999999998},"min":{"1min":1.6639999999999999,"5min":1.583,"15min":1.583},"max":{"1min":4.4770000000000003,"5min":7.5720000000000001,"15min":8.3650000000000002},"last":2.4319999999999999}]},{"osd":11,"last update":"Tue Sep 8 15:40:50 2020","interfaces":[{"interface":"back","average":{"1min":3.1509999999999998,"5min":3.9089999999999998,"15min":4.5279999999999996},"min":{"1min":1.8109999999999999,"5min":1.7010000000000001,"15min":1.7010000000000001},"max":{"1min":7.0410000000000004,"5min":9.1790000000000003,"15min":9.1790000000000003},"last":3.4910000000000001},{"interface":"front","average":{"1min":3.9249999999999998,"5min":4.2869999999999999,"15min":4.9729999999999999},"min":{"1min":2.1819999999999999,"5min":1.756,"15min":1.756},"max":{"1min":7.181,"5min":7.7629999999999999,"15min":7.7629999999999999},"last":2.5830000000000002}]}]},{"osd":6,"up_from":0,"seq":114,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31764,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":93713388,"statfs":{"total":95995035648,"available":95962509312,"internally_reserved":0,"allocated":733184,"data_stored":464615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.6799999999999999,"15min":1.851},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":3.399,"5min":3.8809999999999998,"15min":3.8809999999999998},"last":2.984},{"interface":"front","average":{"1min":1.581,"5min":1.853,"15min":1.9430000000000001},"min":{"1min":0.89900000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.8570000000000002,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":3.5979999999999999}]},{"osd":1,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":10.734,"5min":3.9239999999999999,"15min":2.7599999999999998},"min":{"1min":0.86699999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":149.34200000000001,"5min":149.34200000000001,"15min":149.34200000000001},"last":1.992},{"interface":"front","average":{"1min":10.590999999999999,"5min":4.1479999999999997,"15min":3.0299999999999998},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":149.03899999999999,"5min":149.03899999999999,"15min":149.03899999999999},"last":2.2069999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":32.973999999999997,"5min":8.6989999999999998,"15min":4.5410000000000004},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":549.75999999999999,"5min":549.75999999999999,"15min":549.75999999999999},"last":2.6419999999999999},{"interface":"front","average":{"1min":33.076000000000001,"5min":8.7970000000000006,"15min":4.5519999999999996},"min":{"1min":1.4379999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":549.85900000000004,"5min":549.85900000000004,"15min":549.85900000000004},"last":3.29}]},{"osd":3,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":2.2570000000000001,"5min":3.6539999999999999,"15min":2.8809999999999998},"min":{"1min":1.2250000000000001,"5min":1.224,"15min":1.224},"max":{"1min":3.8100000000000001,"5min":109.65300000000001,"15min":109.65300000000001},"last":4.1180000000000003},{"interface":"front","average":{"1min":2.0390000000000001,"5min":3.7959999999999998,"15min":3.2120000000000002},"min":{"1min":1.24,"5min":1.24,"15min":1.24},"max":{"1min":4.4870000000000001,"5min":109.51600000000001,"15min":109.51600000000001},"last":3.173}]},{"osd":4,"last update":"Tue Sep 8 15:40:58 2020","interfaces":[{"interface":"back","average":{"1min":2.1480000000000001,"5min":2.6779999999999999,"15min":2.7109999999999999},"min":{"1min":0.92700000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":4.0129999999999999,"5min":4.9980000000000002,"15min":4.9980000000000002},"last":2.5329999999999999},{"interface":"front","average":{"1min":2.351,"5min":2.9590000000000001,"15min":2.9489999999999998},"min":{"1min":1.1910000000000001,"5min":1.052,"15min":1.052},"max":{"1min":3.9220000000000002,"5min":22.062000000000001,"15min":22.062000000000001},"last":2.4550000000000001}]},{"osd":5,"last update":"Tue Sep 8 15:40:55 2020","interfaces":[{"interface":"back","average":{"1min":2.778,"5min":3.1110000000000002,"15min":3.1640000000000001},"min":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"max":{"1min":7.1710000000000003,"5min":7.1710000000000003,"15min":7.1710000000000003},"last":3.5230000000000001},{"interface":"front","average":{"1min":2.6509999999999998,"5min":3.444,"15min":3.29},"min":{"1min":1.7230000000000001,"5min":1.149,"15min":1.149},"max":{"1min":4.1619999999999999,"5min":21.228999999999999,"15min":21.228999999999999},"last":2.8460000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:41:14 2020","interfaces":[{"interface":"back","average":{"1min":2.782,"5min":3.343,"15min":3.6120000000000001},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":5.2969999999999997,"5min":22.911999999999999,"15min":22.911999999999999},"last":3.8690000000000002},{"interface":"front","average":{"1min":2.9129999999999998,"5min":3.6709999999999998,"15min":3.8029999999999999},"min":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"max":{"1min":4.8239999999999998,"5min":26.056000000000001,"15min":26.056000000000001},"last":3.766}]},{"osd":8,"last update":"Tue Sep 8 15:41:39 2020","interfaces":[{"interface":"back","average":{"1min":2.4390000000000001,"5min":3.0609999999999999,"15min":3.6739999999999999},"min":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"max":{"1min":3.9489999999999998,"5min":6.3479999999999999,"15min":6.3479999999999999},"last":4.5170000000000003},{"interface":"front","average":{"1min":2.8050000000000002,"5min":3.1779999999999999,"15min":3.8610000000000002},"min":{"1min":1.341,"5min":1.1299999999999999,"15min":1.1299999999999999},"max":{"1min":5.3959999999999999,"5min":8.6850000000000005,"15min":8.6850000000000005},"last":4.4349999999999996}]},{"osd":9,"last update":"Tue Sep 8 15:41:04 2020","interfaces":[{"interface":"back","average":{"1min":2.855,"5min":3.8180000000000001,"15min":4.2460000000000004},"min":{"1min":1.974,"5min":1.5569999999999999,"15min":1.5569999999999999},"max":{"1min":4.1710000000000003,"5min":21.716000000000001,"15min":21.716000000000001},"last":4.7450000000000001},{"interface":"front","average":{"1min":2.5950000000000002,"5min":3.4710000000000001,"15min":4.1699999999999999},"min":{"1min":1.224,"5min":1.224,"15min":1.224},"max":{"1min":4.4320000000000004,"5min":7.0250000000000004,"15min":7.0250000000000004},"last":4.0339999999999998}]},{"osd":10,"last update":"Tue Sep 8 15:41:30 2020","interfaces":[{"interface":"back","average":{"1min":2.7410000000000001,"5min":3.4649999999999999,"15min":4.2839999999999998},"min":{"1min":1.569,"5min":1.5529999999999999,"15min":1.5529999999999999},"max":{"1min":5.5570000000000004,"5min":21.331,"15min":21.331},"last":4.2770000000000001},{"interface":"front","average":{"1min":2.8769999999999998,"5min":3.6699999999999999,"15min":4.3079999999999998},"min":{"1min":1.9299999999999999,"5min":1.915,"15min":1.915},"max":{"1min":4.3529999999999998,"5min":21.515999999999998,"15min":21.515999999999998},"last":4.9530000000000003}]}]},{"osd":1,"up_from":0,"seq":137,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":134344,"kb_used_data":58816,"kb_used_omap":0,"kb_used_meta":75520,"kb_avail":93610808,"statfs":{"total":95995035648,"available":95857467392,"internally_reserved":0,"allocated":60227584,"data_stored":59963423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77332480},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:40:51 2020","interfaces":[{"interface":"back","average":{"1min":2.4079999999999999,"5min":3.2639999999999998,"15min":2.073},"min":{"1min":0.57399999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":7.7450000000000001,"5min":45.085000000000001,"15min":45.085000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":3.2490000000000001,"5min":3.6669999999999998,"15min":2.3250000000000002},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":13.231,"5min":45.765999999999998,"15min":45.765999999999998},"last":1.4339999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:41:20 2020","interfaces":[{"interface":"back","average":{"1min":9.4450000000000003,"5min":4.8929999999999998,"15min":2.8460000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":153.63499999999999,"5min":153.63499999999999,"15min":153.63499999999999},"last":1.327},{"interface":"front","average":{"1min":9.5470000000000006,"5min":5.1529999999999996,"15min":3.0310000000000001},"min":{"1min":0.69399999999999995,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":153.864,"5min":153.864,"15min":153.864},"last":1.77}]},{"osd":3,"last update":"Tue Sep 8 15:40:40 2020","interfaces":[{"interface":"back","average":{"1min":3.4700000000000002,"5min":3.5870000000000002,"15min":2.7149999999999999},"min":{"1min":0.98999999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":12.49,"5min":44.768999999999998,"15min":44.768999999999998},"last":1.8129999999999999},{"interface":"front","average":{"1min":4.1289999999999996,"5min":4.0570000000000004,"15min":2.964},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":21.399999999999999,"5min":45.892000000000003,"15min":45.892000000000003},"last":1.2070000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:40:59 2020","interfaces":[{"interface":"back","average":{"1min":2.448,"5min":3.2770000000000001,"15min":2.5310000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":7.5599999999999996,"5min":44.970999999999997,"15min":44.970999999999997},"last":1.3879999999999999},{"interface":"front","average":{"1min":3.1579999999999999,"5min":3.484,"15min":3.0230000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":7.5940000000000003,"5min":51.289000000000001,"15min":51.289000000000001},"last":2.4969999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:41:27 2020","interfaces":[{"interface":"back","average":{"1min":2.496,"5min":3.7010000000000001,"15min":3.2210000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":7.1349999999999998,"5min":45.289000000000001,"15min":45.289000000000001},"last":1.5229999999999999},{"interface":"front","average":{"1min":2.9670000000000001,"5min":3.431,"15min":2.7890000000000001},"min":{"1min":1.353,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":7.3049999999999997,"5min":51.499000000000002,"15min":51.499000000000002},"last":2.6160000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:40:51 2020","interfaces":[{"interface":"back","average":{"1min":3.375,"5min":4.0309999999999997,"15min":3.7519999999999998},"min":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"max":{"1min":11.973000000000001,"5min":44.932000000000002,"15min":44.932000000000002},"last":1.9890000000000001},{"interface":"front","average":{"1min":3.3769999999999998,"5min":4.2869999999999999,"15min":3.7959999999999998},"min":{"1min":1.6060000000000001,"5min":1.2949999999999999,"15min":1.2949999999999999},"max":{"1min":7.9980000000000002,"5min":45.159999999999997,"15min":45.159999999999997},"last":2.379}]},{"osd":7,"last update":"Tue Sep 8 15:41:20 2020","interfaces":[{"interface":"back","average":{"1min":3.8250000000000002,"5min":4.5650000000000004,"15min":3.71},"min":{"1min":1.607,"5min":1.4690000000000001,"15min":1.4690000000000001},"max":{"1min":13.747,"5min":45.409999999999997,"15min":45.409999999999997},"last":2.738},{"interface":"front","average":{"1min":3.4129999999999998,"5min":4.234,"15min":4.0039999999999996},"min":{"1min":1.085,"5min":1.085,"15min":1.085},"max":{"1min":12.09,"5min":45.375,"15min":45.375},"last":1.724}]},{"osd":8,"last update":"Tue Sep 8 15:40:40 2020","interfaces":[{"interface":"back","average":{"1min":3.6720000000000002,"5min":4.7400000000000002,"15min":4.4390000000000001},"min":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"max":{"1min":7.9850000000000003,"5min":45.640999999999998,"15min":45.640999999999998},"last":7.2629999999999999},{"interface":"front","average":{"1min":3.5409999999999999,"5min":4.7389999999999999,"15min":4.2889999999999997},"min":{"1min":1.153,"5min":1.153,"15min":1.153},"max":{"1min":11.839,"5min":45.235999999999997,"15min":45.235999999999997},"last":6.6470000000000002}]},{"osd":9,"last update":"Tue Sep 8 15:41:12 2020","interfaces":[{"interface":"back","average":{"1min":2.8319999999999999,"5min":3.5190000000000001,"15min":3.3500000000000001},"min":{"1min":1.1599999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"max":{"1min":8.0890000000000004,"5min":51.698,"15min":51.698},"last":2.1890000000000001},{"interface":"front","average":{"1min":3.069,"5min":4.3600000000000003,"15min":4.2809999999999997},"min":{"1min":1.111,"5min":1.111,"15min":1.111},"max":{"1min":12.215999999999999,"5min":45.515000000000001,"15min":45.515000000000001},"last":1.889}]},{"osd":10,"last update":"Tue Sep 8 15:41:27 2020","interfaces":[{"interface":"back","average":{"1min":3.0579999999999998,"5min":4.4340000000000002,"15min":4.2919999999999998},"min":{"1min":1.5009999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"max":{"1min":7.8780000000000001,"5min":46.039999999999999,"15min":46.039999999999999},"last":6.4850000000000003},{"interface":"front","average":{"1min":2.9500000000000002,"5min":3.6859999999999999,"15min":3.5569999999999999},"min":{"1min":1.2330000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":8.0869999999999997,"5min":51.892000000000003,"15min":51.892000000000003},"last":3.8650000000000002}]}]},{"osd":0,"up_from":0,"seq":143,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":64468,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":63744,"kb_avail":93680684,"statfs":{"total":95995035648,"available":95929020416,"internally_reserved":0,"allocated":733184,"data_stored":464615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65273856},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 8 15:40:51 2020","interfaces":[{"interface":"back","average":{"1min":7.2130000000000001,"5min":3.347,"15min":2.0750000000000002},"min":{"1min":0.79700000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":74.831999999999994,"5min":74.831999999999994,"15min":74.831999999999994},"last":0.83999999999999997},{"interface":"front","average":{"1min":7.0250000000000004,"5min":3.371,"15min":2.1160000000000001},"min":{"1min":0.94999999999999996,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":72.611000000000004,"5min":72.611000000000004,"15min":72.611000000000004},"last":2.5369999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:34:05 2020","interfaces":[{"interface":"back","average":{"1min":1.9950000000000001,"5min":1.621,"15min":1.5189999999999999},"min":{"1min":1.3999999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.8519999999999999,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":2.6589999999999998},{"interface":"front","average":{"1min":2.629,"5min":1.956,"15min":1.72},"min":{"1min":1.6419999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":2.4049999999999998}]},{"osd":3,"last update":"Tue Sep 8 15:41:41 2020","interfaces":[{"interface":"back","average":{"1min":6.3630000000000004,"5min":3.2040000000000002,"15min":3},"min":{"1min":0.999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":53.308,"5min":53.308,"15min":53.308},"last":1.7390000000000001},{"interface":"front","average":{"1min":7.4500000000000002,"5min":4.6879999999999997,"15min":3.9350000000000001},"min":{"1min":1.3480000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":67.244,"5min":73.075000000000003,"15min":73.075000000000003},"last":1.869}]},{"osd":4,"last update":"Tue Sep 8 15:41:05 2020","interfaces":[{"interface":"back","average":{"1min":6.1950000000000003,"5min":3.2440000000000002,"15min":2.7919999999999998},"min":{"1min":0.98599999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":67.105999999999995,"5min":67.105999999999995,"15min":67.105999999999995},"last":1.28},{"interface":"front","average":{"1min":6.0279999999999996,"5min":4.1980000000000004,"15min":3.2879999999999998},"min":{"1min":1.3380000000000001,"5min":0.995,"15min":0.995},"max":{"1min":54.067999999999998,"5min":73.802999999999997,"15min":73.802999999999997},"last":1.585}]},{"osd":5,"last update":"Tue Sep 8 15:41:24 2020","interfaces":[{"interface":"back","average":{"1min":7.0679999999999996,"5min":3.8090000000000002,"15min":3.331},"min":{"1min":1.248,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":66.950999999999993,"5min":66.950999999999993,"15min":66.950999999999993},"last":1.4219999999999999},{"interface":"front","average":{"1min":5.6710000000000003,"5min":3,"15min":3.097},"min":{"1min":1.0589999999999999,"5min":0.75,"15min":0.75},"max":{"1min":66.435000000000002,"5min":66.435000000000002,"15min":66.435000000000002},"last":1.298}]},{"osd":6,"last update":"Tue Sep 8 15:40:51 2020","interfaces":[{"interface":"back","average":{"1min":2.1360000000000001,"5min":2.4510000000000001,"15min":3.1760000000000002},"min":{"1min":0.94699999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":5.71,"5min":13.151999999999999,"15min":13.151999999999999},"last":1.772},{"interface":"front","average":{"1min":2.3500000000000001,"5min":2.7669999999999999,"15min":3.5529999999999999},"min":{"1min":0.86899999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":5.923,"5min":13.48,"15min":13.48},"last":1.5}]},{"osd":7,"last update":"Tue Sep 8 15:41:24 2020","interfaces":[{"interface":"back","average":{"1min":5.6689999999999996,"5min":4.5010000000000003,"15min":4.3339999999999996},"min":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"max":{"1min":55.093000000000004,"5min":71.167000000000002,"15min":71.167000000000002},"last":2.121},{"interface":"front","average":{"1min":7.2409999999999997,"5min":5.1269999999999998,"15min":4.4420000000000002},"min":{"1min":2.069,"5min":1.3240000000000001,"15min":1.3240000000000001},"max":{"1min":67.378,"5min":74.480000000000004,"15min":74.480000000000004},"last":2.1600000000000001}]},{"osd":8,"last update":"Tue Sep 8 15:41:41 2020","interfaces":[{"interface":"back","average":{"1min":7.8209999999999997,"5min":4.1920000000000002,"15min":4.0570000000000004},"min":{"1min":1.7230000000000001,"5min":1.093,"15min":1.093},"max":{"1min":67.528000000000006,"5min":67.528000000000006,"15min":67.528000000000006},"last":5.8680000000000003},{"interface":"front","average":{"1min":5.625,"5min":5.923,"15min":4.6920000000000002},"min":{"1min":1.46,"5min":1.2070000000000001,"15min":1.2070000000000001},"max":{"1min":53.405000000000001,"5min":131.958,"15min":131.958},"last":2.0270000000000001}]},{"osd":9,"last update":"Tue Sep 8 15:41:05 2020","interfaces":[{"interface":"back","average":{"1min":6.0979999999999999,"5min":4.3170000000000002,"15min":4.2960000000000003},"min":{"1min":1.337,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":54.292999999999999,"5min":54.292999999999999,"15min":54.292999999999999},"last":2.2490000000000001},{"interface":"front","average":{"1min":6.9370000000000003,"5min":4.0970000000000004,"15min":4.077},"min":{"1min":1.1559999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":67.659999999999997,"5min":67.659999999999997,"15min":67.659999999999997},"last":1.901}]},{"osd":10,"last update":"Tue Sep 8 15:41:24 2020","interfaces":[{"interface":"back","average":{"1min":6.516,"5min":3.5649999999999999,"15min":3.2810000000000001},"min":{"1min":1.145,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":55.441000000000003,"5min":55.441000000000003,"15min":55.441000000000003},"last":1.2490000000000001},{"interface":"front","average":{"1min":7.1950000000000003,"5min":5.1529999999999996,"15min":4.6779999999999999},"min":{"1min":1.5980000000000001,"5min":1.1739999999999999,"15min":1.1739999999999999},"max":{"1min":67.923000000000002,"5min":73.444000000000003,"15min":73.444000000000003},"last":2.0030000000000001}]},{"osd":11,"last update":"Tue Sep 8 15:40:51 2020","interfaces":[{"interface":"back","average":{"1min":8.3390000000000004,"5min":4.6470000000000002,"15min":4.141},"min":{"1min":2.1190000000000002,"5min":1.476,"15min":1.476},"max":{"1min":74.147000000000006,"5min":74.147000000000006,"15min":74.147000000000006},"last":5.7460000000000004},{"interface":"front","average":{"1min":3.3090000000000002,"5min":3.8700000000000001,"15min":4.5300000000000002},"min":{"1min":1.052,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":9.4559999999999995,"5min":13.369999999999999,"15min":13.369999999999999},"last":1.6259999999999999}]}]},{"osd":2,"up_from":0,"seq":134,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":100936,"kb_used_data":41152,"kb_used_omap":0,"kb_used_meta":59776,"kb_avail":93644216,"statfs":{"total":95995035648,"available":95891677184,"internally_reserved":0,"allocated":42139648,"data_stored":41875487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":61210624},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:41:10 2020","interfaces":[{"interface":"back","average":{"1min":2.8519999999999999,"5min":3.4910000000000001,"15min":2.2949999999999999},"min":{"1min":0.67700000000000005,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":11.571,"5min":162.83799999999999,"15min":162.83799999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":3.0449999999999999,"5min":3.6299999999999999,"15min":2.3849999999999998},"min":{"1min":1.0109999999999999,"5min":0.624,"15min":0.624},"max":{"1min":11.326000000000001,"5min":162.63499999999999,"15min":162.63499999999999},"last":0.83999999999999997}]},{"osd":1,"last update":"Tue Sep 8 15:41:10 2020","interfaces":[{"interface":"back","average":{"1min":6.5910000000000002,"5min":3.8460000000000001,"15min":2.444},"min":{"1min":1.026,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":78.728999999999999,"5min":78.728999999999999,"15min":78.728999999999999},"last":1.2310000000000001},{"interface":"front","average":{"1min":6.7409999999999997,"5min":3.4129999999999998,"15min":2.5880000000000001},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":80.103999999999999,"5min":80.103999999999999,"15min":80.103999999999999},"last":1.5389999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:41:40 2020","interfaces":[{"interface":"back","average":{"1min":3.617,"5min":2.992,"15min":2.6379999999999999},"min":{"1min":1.0660000000000001,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":11.643000000000001,"5min":11.643000000000001,"15min":11.643000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":3.4289999999999998,"5min":2.7810000000000001,"15min":2.5379999999999998},"min":{"1min":0.82299999999999995,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":10.925000000000001,"5min":11.737,"15min":11.737},"last":1.2789999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:41:07 2020","interfaces":[{"interface":"back","average":{"1min":2.7040000000000002,"5min":2.6850000000000001,"15min":2.5600000000000001},"min":{"1min":0.97699999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":7.6550000000000002,"5min":7.6550000000000002,"15min":7.6550000000000002},"last":1.119},{"interface":"front","average":{"1min":3.2989999999999999,"5min":3.5,"15min":2.8029999999999999},"min":{"1min":1.1000000000000001,"5min":1.04,"15min":1.04},"max":{"1min":9.7539999999999996,"5min":64.284999999999997,"15min":64.284999999999997},"last":0.877}]},{"osd":5,"last update":"Tue Sep 8 15:41:28 2020","interfaces":[{"interface":"back","average":{"1min":2.8250000000000002,"5min":3.7290000000000001,"15min":3.2490000000000001},"min":{"1min":0.95699999999999996,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":6.0750000000000002,"5min":63.509,"15min":63.509},"last":1.091},{"interface":"front","average":{"1min":2.7370000000000001,"5min":3.0419999999999998,"15min":3.052},"min":{"1min":1.427,"5min":1.0529999999999999,"15min":1.0529999999999999},"max":{"1min":11.117000000000001,"5min":11.117000000000001,"15min":11.117000000000001},"last":1.4139999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:40:48 2020","interfaces":[{"interface":"back","average":{"1min":6.1989999999999998,"5min":4.0620000000000003,"15min":3.4660000000000002},"min":{"1min":1.3129999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"max":{"1min":63.631,"5min":63.631,"15min":63.631},"last":1.496},{"interface":"front","average":{"1min":2.9369999999999998,"5min":3.3340000000000001,"15min":3.4929999999999999},"min":{"1min":1.1870000000000001,"5min":1.105,"15min":1.105},"max":{"1min":8.0120000000000005,"5min":11.234999999999999,"15min":11.234999999999999},"last":1.645}]},{"osd":7,"last update":"Tue Sep 8 15:41:23 2020","interfaces":[{"interface":"back","average":{"1min":3.105,"5min":3.5449999999999999,"15min":3.899},"min":{"1min":1.504,"5min":1.298,"15min":1.298},"max":{"1min":11.247,"5min":11.436,"15min":11.436},"last":1.4359999999999999},{"interface":"front","average":{"1min":3.5179999999999998,"5min":3.2890000000000001,"15min":3.5819999999999999},"min":{"1min":1.264,"5min":1.006,"15min":1.006},"max":{"1min":10.784000000000001,"5min":10.784000000000001,"15min":10.784000000000001},"last":2.1320000000000001}]},{"osd":8,"last update":"Tue Sep 8 15:41:34 2020","interfaces":[{"interface":"back","average":{"1min":3.7440000000000002,"5min":4.0270000000000001,"15min":3.8079999999999998},"min":{"1min":1.2350000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":10.026,"5min":64.116,"15min":64.116},"last":1.3600000000000001},{"interface":"front","average":{"1min":3.0350000000000001,"5min":3.996,"15min":4},"min":{"1min":1.3700000000000001,"5min":1.218,"15min":1.218},"max":{"1min":11.372999999999999,"5min":63.299999999999997,"15min":63.299999999999997},"last":1.7430000000000001}]},{"osd":9,"last update":"Tue Sep 8 15:41:10 2020","interfaces":[{"interface":"back","average":{"1min":3.7749999999999999,"5min":4.4000000000000004,"15min":4.165},"min":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"max":{"1min":10.406000000000001,"5min":64.004000000000005,"15min":64.004000000000005},"last":1.6200000000000001},{"interface":"front","average":{"1min":3.629,"5min":3.669,"15min":4.085},"min":{"1min":1.706,"5min":1.401,"15min":1.401},"max":{"1min":11.869,"5min":11.869,"15min":11.869},"last":2.3220000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:41:28 2020","interfaces":[{"interface":"back","average":{"1min":3.597,"5min":4.3460000000000001,"15min":4.1600000000000001},"min":{"1min":1.671,"5min":1.4339999999999999,"15min":1.4339999999999999},"max":{"1min":10.555999999999999,"5min":63.743000000000002,"15min":63.743000000000002},"last":1.762},{"interface":"front","average":{"1min":3.758,"5min":3.2290000000000001,"15min":3.6120000000000001},"min":{"1min":1.337,"5min":1.159,"15min":1.159},"max":{"1min":11.067,"5min":11.067,"15min":11.067},"last":6.2729999999999997}]}]},{"osd":3,"up_from":0,"seq":128,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":69268,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":68544,"kb_avail":93675884,"statfs":{"total":95995035648,"available":95924105216,"internally_reserved":0,"allocated":733184,"data_stored":464615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":70189056},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:31 2020","interfaces":[{"interface":"back","average":{"1min":2.1379999999999999,"5min":1.7210000000000001,"15min":1.629},"min":{"1min":1.79,"5min":1.1419999999999999,"15min":1.1419999999999999},"max":{"1min":2.7930000000000001,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":3.169},{"interface":"front","average":{"1min":2.6800000000000002,"5min":2.2309999999999999,"15min":2.0990000000000002},"min":{"1min":2.0270000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"max":{"1min":3.5409999999999999,"5min":3.5409999999999999,"15min":3.5409999999999999},"last":2.5579999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:41:44 2020","interfaces":[{"interface":"back","average":{"1min":2.7229999999999999,"5min":3.028,"15min":2.2949999999999999},"min":{"1min":0.77400000000000002,"5min":0.628,"15min":0.628},"max":{"1min":5.4189999999999996,"5min":44.109999999999999,"15min":44.109999999999999},"last":2.3340000000000001},{"interface":"front","average":{"1min":3.1230000000000002,"5min":3.9710000000000001,"15min":2.6030000000000002},"min":{"1min":0.96499999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":7.6139999999999999,"5min":45.618000000000002,"15min":45.618000000000002},"last":2.8220000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:41:44 2020","interfaces":[{"interface":"back","average":{"1min":2.5910000000000002,"5min":3.2280000000000002,"15min":2.5459999999999998},"min":{"1min":0.84099999999999997,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":7.9160000000000004,"5min":45.774000000000001,"15min":45.774000000000001},"last":2.3969999999999998},{"interface":"front","average":{"1min":2.859,"5min":2.6819999999999999,"15min":2.4239999999999999},"min":{"1min":0.81000000000000005,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":7.7279999999999998,"5min":13.948,"15min":13.948},"last":2.5099999999999998}]},{"osd":4,"last update":"Tue Sep 8 15:41:03 2020","interfaces":[{"interface":"back","average":{"1min":2.746,"5min":2.5169999999999999,"15min":2.5659999999999998},"min":{"1min":1.018,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":10.523,"5min":10.523,"15min":10.523},"last":3.3319999999999999},{"interface":"front","average":{"1min":1.9510000000000001,"5min":2.1880000000000002,"15min":2.4239999999999999},"min":{"1min":0.91100000000000003,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":5.3460000000000001,"5min":11.298,"15min":11.298},"last":3.0169999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:41:29 2020","interfaces":[{"interface":"back","average":{"1min":1.982,"5min":2.9580000000000002,"15min":2.9100000000000001},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":4.4020000000000001,"5min":45.442999999999998,"15min":45.442999999999998},"last":3.0960000000000001},{"interface":"front","average":{"1min":2.7450000000000001,"5min":3.2080000000000002,"15min":3.2519999999999998},"min":{"1min":1.53,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":5.5599999999999996,"5min":23.565000000000001,"15min":23.565000000000001},"last":3.9350000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:40:53 2020","interfaces":[{"interface":"back","average":{"1min":2.7999999999999998,"5min":2.8889999999999998,"15min":3.2029999999999998},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":7.7969999999999997,"5min":7.7969999999999997,"15min":7.7969999999999997},"last":3.5840000000000001},{"interface":"front","average":{"1min":2.3940000000000001,"5min":3.214,"15min":3.452},"min":{"1min":1.3560000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":5.9859999999999998,"5min":47.457999999999998,"15min":47.457999999999998},"last":3.847}]},{"osd":7,"last update":"Tue Sep 8 15:41:29 2020","interfaces":[{"interface":"back","average":{"1min":3.2970000000000002,"5min":3.887,"15min":4.133},"min":{"1min":1.5109999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"max":{"1min":7.1200000000000001,"5min":26.035,"15min":26.035},"last":4.3250000000000002},{"interface":"front","average":{"1min":2.8690000000000002,"5min":3.109,"15min":3.4500000000000002},"min":{"1min":1.3240000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":7.851,"5min":25.521000000000001,"15min":25.521000000000001},"last":3.2879999999999998}]},{"osd":8,"last update":"Tue Sep 8 15:41:41 2020","interfaces":[{"interface":"back","average":{"1min":4.0899999999999999,"5min":3.6269999999999998,"15min":3.7269999999999999},"min":{"1min":1.5720000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":21.329000000000001,"5min":46.917999999999999,"15min":46.917999999999999},"last":5.0720000000000001},{"interface":"front","average":{"1min":4.3609999999999998,"5min":4.6580000000000004,"15min":4.4290000000000003},"min":{"1min":1.3260000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":19.02,"5min":44.893000000000001,"15min":44.893000000000001},"last":4.1420000000000003}]},{"osd":9,"last update":"Tue Sep 8 15:41:03 2020","interfaces":[{"interface":"back","average":{"1min":2.7610000000000001,"5min":2.9750000000000001,"15min":3.645},"min":{"1min":1.272,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":7.6669999999999998,"5min":10.872999999999999,"15min":10.872999999999999},"last":3.5459999999999998},{"interface":"front","average":{"1min":2.605,"5min":3.7909999999999999,"15min":4.1509999999999998},"min":{"1min":1.5409999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"max":{"1min":5.5679999999999996,"5min":46.665999999999997,"15min":46.665999999999997},"last":4.4119999999999999}]},{"osd":10,"last update":"Tue Sep 8 15:41:29 2020","interfaces":[{"interface":"back","average":{"1min":2.899,"5min":3.9809999999999999,"15min":4.274},"min":{"1min":1.371,"5min":1.3520000000000001,"15min":1.3520000000000001},"max":{"1min":5.5229999999999997,"5min":39.283000000000001,"15min":39.283000000000001},"last":4.71},{"interface":"front","average":{"1min":2.4889999999999999,"5min":2.9790000000000001,"15min":3.5230000000000001},"min":{"1min":1.4319999999999999,"5min":1.081,"15min":1.081},"max":{"1min":4.4859999999999998,"5min":11.005000000000001,"15min":11.005000000000001},"last":3.7909999999999999}]},{"osd":11,"last update":"Tue Sep 8 15:40:53 2020","interfaces":[{"interface":"back","average":{"1min":2.5209999999999999,"5min":3.8540000000000001,"15min":4.4950000000000001},"min":{"1min":1.5549999999999999,"5min":1.196,"15min":1.196},"max":{"1min":5.8390000000000004,"5min":13.882999999999999,"15min":13.882999999999999},"last":4.0369999999999999},{"interface":"front","average":{"1min":2.7010000000000001,"5min":4.9370000000000003,"15min":4.4089999999999998},"min":{"1min":1.677,"5min":1.577,"15min":1.577},"max":{"1min":7.6639999999999997,"5min":47.234999999999999,"15min":47.234999999999999},"last":4.6349999999999998}]}]},{"osd":4,"up_from":0,"seq":124,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":34968,"kb_used_data":3792,"kb_used_omap":0,"kb_used_meta":31168,"kb_avail":93710184,"statfs":{"total":95995035648,"available":95959228416,"internally_reserved":0,"allocated":3883008,"data_stored":3610367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31916032},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:41:00 2020","interfaces":[{"interface":"back","average":{"1min":1.6830000000000001,"5min":1.8320000000000001,"15min":1.865},"min":{"1min":0.81100000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":4.7850000000000001,"5min":5.7130000000000001,"15min":5.7130000000000001},"last":1.2649999999999999},{"interface":"front","average":{"1min":2.3700000000000001,"5min":2.0230000000000001,"15min":1.915},"min":{"1min":0.83699999999999997,"5min":0.71299999999999997,"15min":0.70799999999999996},"max":{"1min":10.471,"5min":10.471,"15min":10.471},"last":0.873}]},{"osd":1,"last update":"Tue Sep 8 15:41:00 2020","interfaces":[{"interface":"back","average":{"1min":3.2559999999999998,"5min":3.7210000000000001,"15min":2.4929999999999999},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":14.127000000000001,"5min":74.947999999999993,"15min":74.947999999999993},"last":4.8070000000000004},{"interface":"front","average":{"1min":2.992,"5min":3.9289999999999998,"15min":2.738},"min":{"1min":1.339,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":6.665,"5min":71.108999999999995,"15min":71.108999999999995},"last":2.609}]},{"osd":2,"last update":"Tue Sep 8 15:41:00 2020","interfaces":[{"interface":"back","average":{"1min":2.8980000000000001,"5min":3.8769999999999998,"15min":2.7309999999999999},"min":{"1min":1.5680000000000001,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":7.1790000000000003,"5min":74.504000000000005,"15min":74.504000000000005},"last":3.992},{"interface":"front","average":{"1min":2.6349999999999998,"5min":3.6949999999999998,"15min":2.786},"min":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"max":{"1min":6.8630000000000004,"5min":73.885000000000005,"15min":73.885000000000005},"last":2.0219999999999998}]},{"osd":3,"last update":"Tue Sep 8 15:41:00 2020","interfaces":[{"interface":"back","average":{"1min":1.9670000000000001,"5min":2.6080000000000001,"15min":2.556},"min":{"1min":1.0169999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":5.7240000000000002,"5min":6.3280000000000003,"15min":6.3280000000000003},"last":1.1859999999999999},{"interface":"front","average":{"1min":3.4670000000000001,"5min":2.9260000000000002,"15min":2.5630000000000002},"min":{"1min":1.379,"5min":1.1539999999999999,"15min":1.1539999999999999},"max":{"1min":9.5069999999999997,"5min":9.5069999999999997,"15min":9.5069999999999997},"last":1.3129999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:41:32 2020","interfaces":[{"interface":"back","average":{"1min":3.0609999999999999,"5min":2.8069999999999999,"15min":2.8610000000000002},"min":{"1min":1.556,"5min":1.024,"15min":1.014},"max":{"1min":6.2930000000000001,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":1.0289999999999999},{"interface":"front","average":{"1min":3.0030000000000001,"5min":2.8420000000000001,"15min":2.903},"min":{"1min":1.304,"5min":1.2190000000000001,"15min":1.129},"max":{"1min":6.5499999999999998,"5min":7.5359999999999996,"15min":7.5359999999999996},"last":1.427}]},{"osd":6,"last update":"Tue Sep 8 15:40:46 2020","interfaces":[{"interface":"back","average":{"1min":2.8260000000000001,"5min":3.2639999999999998,"15min":3.3159999999999998},"min":{"1min":1.645,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":6.6420000000000003,"5min":6.8650000000000002,"15min":6.8650000000000002},"last":1.1399999999999999},{"interface":"front","average":{"1min":2.3940000000000001,"5min":3.1549999999999998,"15min":3.2999999999999998},"min":{"1min":1.4450000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"max":{"1min":6.8849999999999998,"5min":6.8849999999999998,"15min":6.8849999999999998},"last":1.6759999999999999}]},{"osd":7,"last update":"Tue Sep 8 15:41:08 2020","interfaces":[{"interface":"back","average":{"1min":2.6240000000000001,"5min":3.3119999999999998,"15min":3.7029999999999998},"min":{"1min":1.6890000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"max":{"1min":6.4470000000000001,"5min":7.2969999999999997,"15min":7.4699999999999998},"last":1.8400000000000001},{"interface":"front","average":{"1min":3.133,"5min":3.4809999999999999,"15min":3.8260000000000001},"min":{"1min":1.8260000000000001,"5min":1.546,"15min":1.546},"max":{"1min":6.0199999999999996,"5min":7.1459999999999999,"15min":7.1459999999999999},"last":2.2749999999999999}]},{"osd":8,"last update":"Tue Sep 8 15:41:40 2020","interfaces":[{"interface":"back","average":{"1min":4.157,"5min":5.2140000000000004,"15min":4.3719999999999999},"min":{"1min":1.413,"5min":1.276,"15min":1.276},"max":{"1min":8.0909999999999993,"5min":109.608,"15min":109.608},"last":5.3689999999999998},{"interface":"front","average":{"1min":2.972,"5min":5.0380000000000003,"15min":4.6130000000000004},"min":{"1min":1.609,"5min":1.49,"15min":1.49},"max":{"1min":7.1479999999999997,"5min":116.52500000000001,"15min":116.52500000000001},"last":1.609}]},{"osd":9,"last update":"Tue Sep 8 15:41:06 2020","interfaces":[{"interface":"back","average":{"1min":2.8759999999999999,"5min":5.2300000000000004,"15min":4.6630000000000003},"min":{"1min":1.6259999999999999,"5min":1.546,"15min":1.546},"max":{"1min":6.3470000000000004,"5min":118.312,"15min":118.312},"last":2.1320000000000001},{"interface":"front","average":{"1min":3.375,"5min":5.1280000000000001,"15min":4.6529999999999996},"min":{"1min":2.0449999999999999,"5min":1.647,"15min":1.647},"max":{"1min":6.524,"5min":116.871,"15min":116.871},"last":1.4650000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:41:32 2020","interfaces":[{"interface":"back","average":{"1min":3.2170000000000001,"5min":3.2519999999999998,"15min":4.3680000000000003},"min":{"1min":1.7689999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"max":{"1min":6.6070000000000002,"5min":6.6070000000000002,"15min":7.9279999999999999},"last":1.6439999999999999},{"interface":"front","average":{"1min":3.3889999999999998,"5min":3.5419999999999998,"15min":4.46},"min":{"1min":1.861,"5min":1.702,"15min":1.702},"max":{"1min":6.7439999999999998,"5min":7.9870000000000001,"15min":7.9870000000000001},"last":1.9790000000000001}]}]},{"osd":5,"up_from":0,"seq":119,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":48084,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":47360,"kb_avail":93697068,"statfs":{"total":95995035648,"available":95945797632,"internally_reserved":0,"allocated":733184,"data_stored":464615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":48496640},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:41:31 2020","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.673,"15min":1.849},"min":{"1min":0.88700000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.3759999999999999,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":5.4400000000000004},{"interface":"front","average":{"1min":2.169,"5min":2.2389999999999999,"15min":2.101},"min":{"1min":1.0469999999999999,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":4.4950000000000001,"5min":7.1340000000000003,"15min":7.1340000000000003},"last":1.9750000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:41:31 2020","interfaces":[{"interface":"back","average":{"1min":8.6010000000000009,"5min":3.9809999999999999,"15min":2.9159999999999999},"min":{"1min":0.96399999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":128.488,"5min":128.488,"15min":128.488},"last":4.2939999999999996},{"interface":"front","average":{"1min":9.3789999999999996,"5min":4.8259999999999996,"15min":2.9580000000000002},"min":{"1min":1.105,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":129.63800000000001,"5min":129.63800000000001,"15min":129.63800000000001},"last":2.0499999999999998}]},{"osd":2,"last update":"Tue Sep 8 15:41:31 2020","interfaces":[{"interface":"back","average":{"1min":8.7520000000000007,"5min":4.056,"15min":2.7989999999999999},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":127.498,"5min":127.498,"15min":127.498},"last":2.1240000000000001},{"interface":"front","average":{"1min":8.7390000000000008,"5min":4.3250000000000002,"15min":3.3109999999999999},"min":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":126.673,"5min":126.673,"15min":126.673},"last":3.8439999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:41:31 2020","interfaces":[{"interface":"back","average":{"1min":2.6949999999999998,"5min":3.1459999999999999,"15min":3.0070000000000001},"min":{"1min":1.571,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":4.101,"5min":8.4220000000000006,"15min":8.4220000000000006},"last":4.6989999999999998},{"interface":"front","average":{"1min":2.02,"5min":2.5739999999999998,"15min":2.8330000000000002},"min":{"1min":0.97599999999999998,"5min":0.875,"15min":0.875},"max":{"1min":3.911,"5min":6.819,"15min":6.819},"last":3.069}]},{"osd":4,"last update":"Tue Sep 8 15:41:31 2020","interfaces":[{"interface":"back","average":{"1min":1.821,"5min":2.2749999999999999,"15min":2.3370000000000002},"min":{"1min":0.98399999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":3.976,"5min":6.9180000000000001,"15min":6.9180000000000001},"last":2.7029999999999998},{"interface":"front","average":{"1min":1.8779999999999999,"5min":2.5310000000000001,"15min":2.698},"min":{"1min":1.135,"5min":1.135,"15min":1.135},"max":{"1min":3.9390000000000001,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":2.7610000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:40:58 2020","interfaces":[{"interface":"back","average":{"1min":2.5499999999999998,"5min":3.456,"15min":3.552},"min":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"max":{"1min":4.3810000000000002,"5min":8.7710000000000008,"15min":8.7710000000000008},"last":5.0880000000000001},{"interface":"front","average":{"1min":2.9980000000000002,"5min":4.024,"15min":3.3929999999999998},"min":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"max":{"1min":4.6669999999999998,"5min":9.359,"15min":9.359},"last":5.8310000000000004}]},{"osd":7,"last update":"Tue Sep 8 15:41:15 2020","interfaces":[{"interface":"back","average":{"1min":3.0289999999999999,"5min":3.2029999999999998,"15min":3.532},"min":{"1min":1.26,"5min":1.26,"15min":1.26},"max":{"1min":8.9610000000000003,"5min":8.9610000000000003,"15min":8.9610000000000003},"last":6.2169999999999996},{"interface":"front","average":{"1min":3.488,"5min":4.0979999999999999,"15min":4.0350000000000001},"min":{"1min":1.5549999999999999,"5min":1.532,"15min":1.532},"max":{"1min":9.1769999999999996,"5min":9.2129999999999992,"15min":9.2129999999999992},"last":6.617}]},{"osd":8,"last update":"Tue Sep 8 15:41:41 2020","interfaces":[{"interface":"back","average":{"1min":3.3439999999999999,"5min":3.2170000000000001,"15min":3.4849999999999999},"min":{"1min":1.6659999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":9.3190000000000008,"5min":9.3190000000000008,"15min":9.3190000000000008},"last":6.9960000000000004},{"interface":"front","average":{"1min":2.4500000000000002,"5min":2.9510000000000001,"15min":3.6869999999999998},"min":{"1min":1.278,"5min":1.1479999999999999,"15min":1.1479999999999999},"max":{"1min":9.1099999999999994,"5min":10.138999999999999,"15min":10.138999999999999},"last":3.7959999999999998}]},{"osd":9,"last update":"Tue Sep 8 15:41:09 2020","interfaces":[{"interface":"back","average":{"1min":2.8620000000000001,"5min":3.4620000000000002,"15min":4.2350000000000003},"min":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"max":{"1min":8.1980000000000004,"5min":8.1980000000000004,"15min":8.1980000000000004},"last":3.4119999999999999},{"interface":"front","average":{"1min":3.3889999999999998,"5min":4.0519999999999996,"15min":4.1630000000000003},"min":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"max":{"1min":8.2560000000000002,"5min":8.9149999999999991,"15min":8.9149999999999991},"last":7.3920000000000003}]},{"osd":10,"last update":"Tue Sep 8 15:41:31 2020","interfaces":[{"interface":"back","average":{"1min":2.4449999999999998,"5min":2.9510000000000001,"15min":3.8090000000000002},"min":{"1min":1.536,"5min":1.431,"15min":1.431},"max":{"1min":5.8959999999999999,"5min":5.8959999999999999,"15min":5.8959999999999999},"last":3.1179999999999999},{"interface":"front","average":{"1min":3.5070000000000001,"5min":4.0170000000000003,"15min":4.5750000000000002},"min":{"1min":1.99,"5min":1.8660000000000001,"15min":1.8660000000000001},"max":{"1min":6.2270000000000003,"5min":9.0670000000000002,"15min":9.0670000000000002},"last":7.766}]}]}]}} 2020-09-08T15:41:48.694 INFO:tasks.thrashosds.thrasher:Setting 2.2 to [0, 7, 6, 2] 2020-09-08T15:41:48.694 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.2', '0', '7', '6', '2'] 2020-09-08T15:41:48.694 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pg-upmap-items 2.2 0 7 6 2 2020-09-08T15:41:48.713 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:48.849 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:48.858 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:41:48.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:48 smithi166 bash[9446]: cluster 2020-09-08T15:41:47.656418+0000 mgr.x (mgr.24118) 432 : cluster [DBG] pgmap v560: 17 pgs: 17 active+clean; 33 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T15:41:48.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:48 smithi166 bash[9446]: cluster 2020-09-08T15:41:47.724987+0000 mon.a (mon.0) 1102 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:41:48.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:48 smithi166 bash[9446]: audit 2020-09-08T15:41:47.738746+0000 mon.a (mon.0) 1103 : audit [INF] from='client.? v1:172.21.15.104:0/3792835812' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:41:48.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:48 smithi166 bash[9446]: cluster 2020-09-08T15:41:47.738831+0000 mon.a (mon.0) 1104 : cluster [DBG] osdmap e211: 12 total, 12 up, 12 in 2020-09-08T15:41:48.955 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:48.990 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:48 smithi104 bash[9203]: cluster 2020-09-08T15:41:47.656418+0000 mgr.x (mgr.24118) 432 : cluster [DBG] pgmap v560: 17 pgs: 17 active+clean; 33 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T15:41:48.991 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:48 smithi104 bash[9203]: cluster 2020-09-08T15:41:47.724987+0000 mon.a (mon.0) 1102 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:41:48.991 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:48 smithi104 bash[9203]: audit 2020-09-08T15:41:47.738746+0000 mon.a (mon.0) 1103 : audit [INF] from='client.? v1:172.21.15.104:0/3792835812' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:41:48.991 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:48 smithi104 bash[9203]: cluster 2020-09-08T15:41:47.738831+0000 mon.a (mon.0) 1104 : cluster [DBG] osdmap e211: 12 total, 12 up, 12 in 2020-09-08T15:41:49.078 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:41:48 smithi055 bash[11956]: debug 2020-09-08T15:41:48.853+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:49.078 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:48 smithi055 bash[9296]: cluster 2020-09-08T15:41:47.656418+0000 mgr.x (mgr.24118) 432 : cluster [DBG] pgmap v560: 17 pgs: 17 active+clean; 33 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T15:41:49.078 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:48 smithi055 bash[9296]: cluster 2020-09-08T15:41:47.724987+0000 mon.a (mon.0) 1102 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:41:49.079 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:48 smithi055 bash[9296]: audit 2020-09-08T15:41:47.738746+0000 mon.a (mon.0) 1103 : audit [INF] from='client.? v1:172.21.15.104:0/3792835812' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:41:49.079 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:48 smithi055 bash[9296]: cluster 2020-09-08T15:41:47.738831+0000 mon.a (mon.0) 1104 : cluster [DBG] osdmap e211: 12 total, 12 up, 12 in 2020-09-08T15:41:49.079 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:49.183 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:49.283 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:49.388 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:49.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:41:49 smithi055 bash[16646]: debug 2020-09-08T15:41:49.081+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:49.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:41:49 smithi055 bash[16646]: debug 2020-09-08T15:41:49.285+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:49.499 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:49.570 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:41:49 smithi166 bash[16593]: debug 2020-09-08T15:41:49.503+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:49.603 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:49.877 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:49 smithi166 bash[9446]: cluster 2020-09-08T15:41:48.221843+0000 osd.1 (osd.1) 42 : cluster [DBG] 2.9 scrub starts 2020-09-08T15:41:49.878 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:49 smithi166 bash[9446]: cluster 2020-09-08T15:41:48.231878+0000 osd.1 (osd.1) 43 : cluster [DBG] 2.9 scrub ok 2020-09-08T15:41:49.878 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:49 smithi166 bash[9446]: cluster 2020-09-08T15:41:48.818647+0000 osd.11 (osd.11) 25 : cluster [DBG] 2.c scrub starts 2020-09-08T15:41:49.878 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:49 smithi166 bash[9446]: cluster 2020-09-08T15:41:48.837357+0000 osd.11 (osd.11) 26 : cluster [DBG] 2.c scrub ok 2020-09-08T15:41:49.879 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:49 smithi166 bash[9446]: cluster 2020-09-08T15:41:49.034828+0000 osd.6 (osd.6) 25 : cluster [DBG] 2.7 scrub starts 2020-09-08T15:41:49.879 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:49 smithi166 bash[9446]: cluster 2020-09-08T15:41:49.044624+0000 osd.6 (osd.6) 26 : cluster [DBG] 2.7 scrub ok 2020-09-08T15:41:49.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:49 smithi104 bash[9203]: cluster 2020-09-08T15:41:48.221843+0000 osd.1 (osd.1) 42 : cluster [DBG] 2.9 scrub starts 2020-09-08T15:41:49.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:49 smithi104 bash[9203]: cluster 2020-09-08T15:41:48.231878+0000 osd.1 (osd.1) 43 : cluster [DBG] 2.9 scrub ok 2020-09-08T15:41:49.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:49 smithi104 bash[9203]: cluster 2020-09-08T15:41:48.818647+0000 osd.11 (osd.11) 25 : cluster [DBG] 2.c scrub starts 2020-09-08T15:41:49.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:49 smithi104 bash[9203]: cluster 2020-09-08T15:41:48.837357+0000 osd.11 (osd.11) 26 : cluster [DBG] 2.c scrub ok 2020-09-08T15:41:49.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:49 smithi104 bash[9203]: cluster 2020-09-08T15:41:49.034828+0000 osd.6 (osd.6) 25 : cluster [DBG] 2.7 scrub starts 2020-09-08T15:41:49.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:49 smithi104 bash[9203]: cluster 2020-09-08T15:41:49.044624+0000 osd.6 (osd.6) 26 : cluster [DBG] 2.7 scrub ok 2020-09-08T15:41:49.954 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:50.058 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:41:50.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:49 smithi055 bash[9296]: cluster 2020-09-08T15:41:48.221843+0000 osd.1 (osd.1) 42 : cluster [DBG] 2.9 scrub starts 2020-09-08T15:41:50.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:49 smithi055 bash[9296]: cluster 2020-09-08T15:41:48.231878+0000 osd.1 (osd.1) 43 : cluster [DBG] 2.9 scrub ok 2020-09-08T15:41:50.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:49 smithi055 bash[9296]: cluster 2020-09-08T15:41:48.818647+0000 osd.11 (osd.11) 25 : cluster [DBG] 2.c scrub starts 2020-09-08T15:41:50.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:49 smithi055 bash[9296]: cluster 2020-09-08T15:41:48.837357+0000 osd.11 (osd.11) 26 : cluster [DBG] 2.c scrub ok 2020-09-08T15:41:50.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:49 smithi055 bash[9296]: cluster 2020-09-08T15:41:49.034828+0000 osd.6 (osd.6) 25 : cluster [DBG] 2.7 scrub starts 2020-09-08T15:41:50.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:49 smithi055 bash[9296]: cluster 2020-09-08T15:41:49.044624+0000 osd.6 (osd.6) 26 : cluster [DBG] 2.7 scrub ok 2020-09-08T15:41:50.538 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:41:50.602 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:41:49 smithi104 bash[15273]: debug 2020-09-08T15:41:49.952+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:50.640 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:50.773 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:50.882 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:50.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:41:50 smithi055 bash[14301]: debug 2020-09-08T15:41:50.777+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:50.930 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:41:50 smithi104 bash[17588]: debug 2020-09-08T15:41:50.532+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:50.993 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:41:51.080 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:41:51.122 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:41:51.199 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:51.226 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:41:51 smithi166 bash[16593]: debug 2020-09-08T15:41:51.083+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:51.315 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:51.418 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:51.438 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:41:51.439 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:41:51.519 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:41:51 smithi055 bash[14301]: debug 2020-09-08T15:41:51.309+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:51.519 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:51.635 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:51.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:51 smithi166 bash[9446]: cluster 2020-09-08T15:41:49.657104+0000 mgr.x (mgr.24118) 433 : cluster [DBG] pgmap v562: 17 pgs: 17 active+clean; 0 B data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 597 op/s 2020-09-08T15:41:51.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:51 smithi166 bash[9446]: cluster 2020-09-08T15:41:50.200039+0000 osd.1 (osd.1) 44 : cluster [DBG] 2.6 scrub starts 2020-09-08T15:41:51.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:51 smithi166 bash[9446]: cluster 2020-09-08T15:41:50.218406+0000 osd.1 (osd.1) 45 : cluster [DBG] 2.6 scrub ok 2020-09-08T15:41:51.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:51 smithi166 bash[9446]: cluster 2020-09-08T15:41:50.648657+0000 osd.7 (osd.7) 41 : cluster [DBG] 2.0 scrub starts 2020-09-08T15:41:51.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:51 smithi166 bash[9446]: cluster 2020-09-08T15:41:50.669628+0000 osd.7 (osd.7) 42 : cluster [DBG] 2.0 scrub ok 2020-09-08T15:41:51.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:51 smithi166 bash[9446]: cluster 2020-09-08T15:41:50.874107+0000 osd.11 (osd.11) 27 : cluster [DBG] 2.a scrub starts 2020-09-08T15:41:51.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:51 smithi166 bash[9446]: cluster 2020-09-08T15:41:50.891249+0000 osd.11 (osd.11) 28 : cluster [DBG] 2.a scrub ok 2020-09-08T15:41:51.908 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:41:51 smithi055 bash[11956]: debug 2020-09-08T15:41:51.521+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:51.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:51 smithi055 bash[9296]: cluster 2020-09-08T15:41:49.657104+0000 mgr.x (mgr.24118) 433 : cluster [DBG] pgmap v562: 17 pgs: 17 active+clean; 0 B data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 597 op/s 2020-09-08T15:41:51.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:51 smithi055 bash[9296]: cluster 2020-09-08T15:41:50.200039+0000 osd.1 (osd.1) 44 : cluster [DBG] 2.6 scrub starts 2020-09-08T15:41:51.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:51 smithi055 bash[9296]: cluster 2020-09-08T15:41:50.218406+0000 osd.1 (osd.1) 45 : cluster [DBG] 2.6 scrub ok 2020-09-08T15:41:51.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:51 smithi055 bash[9296]: cluster 2020-09-08T15:41:50.648657+0000 osd.7 (osd.7) 41 : cluster [DBG] 2.0 scrub starts 2020-09-08T15:41:51.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:51 smithi055 bash[9296]: cluster 2020-09-08T15:41:50.669628+0000 osd.7 (osd.7) 42 : cluster [DBG] 2.0 scrub ok 2020-09-08T15:41:51.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:51 smithi055 bash[9296]: cluster 2020-09-08T15:41:50.874107+0000 osd.11 (osd.11) 27 : cluster [DBG] 2.a scrub starts 2020-09-08T15:41:51.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:51 smithi055 bash[9296]: cluster 2020-09-08T15:41:50.891249+0000 osd.11 (osd.11) 28 : cluster [DBG] 2.a scrub ok 2020-09-08T15:41:51.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:51 smithi104 bash[9203]: cluster 2020-09-08T15:41:49.657104+0000 mgr.x (mgr.24118) 433 : cluster [DBG] pgmap v562: 17 pgs: 17 active+clean; 0 B data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 597 op/s 2020-09-08T15:41:51.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:51 smithi104 bash[9203]: cluster 2020-09-08T15:41:50.200039+0000 osd.1 (osd.1) 44 : cluster [DBG] 2.6 scrub starts 2020-09-08T15:41:51.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:51 smithi104 bash[9203]: cluster 2020-09-08T15:41:50.218406+0000 osd.1 (osd.1) 45 : cluster [DBG] 2.6 scrub ok 2020-09-08T15:41:51.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:51 smithi104 bash[9203]: cluster 2020-09-08T15:41:50.648657+0000 osd.7 (osd.7) 41 : cluster [DBG] 2.0 scrub starts 2020-09-08T15:41:51.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:51 smithi104 bash[9203]: cluster 2020-09-08T15:41:50.669628+0000 osd.7 (osd.7) 42 : cluster [DBG] 2.0 scrub ok 2020-09-08T15:41:51.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:51 smithi104 bash[9203]: cluster 2020-09-08T15:41:50.874107+0000 osd.11 (osd.11) 27 : cluster [DBG] 2.a scrub starts 2020-09-08T15:41:51.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:51 smithi104 bash[9203]: cluster 2020-09-08T15:41:50.891249+0000 osd.11 (osd.11) 28 : cluster [DBG] 2.a scrub ok 2020-09-08T15:41:52.090 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:52.140 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2020-09-08T15:41:52.199 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:41:52.271 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:41:52.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:41:52 smithi104 bash[19922]: debug 2020-09-08T15:41:52.088+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:52.579 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:41:52.684 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:52.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:41:52 smithi104 bash[22305]: debug 2020-09-08T15:41:52.576+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:53.006 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:53.115 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:53.248 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:41:53.356 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:53.429 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:41:53 smithi104 bash[15273]: debug 2020-09-08T15:41:53.004+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:53.489 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:41:53 smithi166 bash[18960]: debug 2020-09-08T15:41:53.247+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:53.634 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:53.741 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:53.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:53 smithi166 bash[9446]: cluster 2020-09-08T15:41:51.657761+0000 mgr.x (mgr.24118) 434 : cluster [DBG] pgmap v563: 17 pgs: 17 active+clean; 0 B data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 402 op/s 2020-09-08T15:41:53.910 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:41:53 smithi055 bash[11956]: debug 2020-09-08T15:41:53.633+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:53.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:53 smithi055 bash[9296]: cluster 2020-09-08T15:41:51.657761+0000 mgr.x (mgr.24118) 434 : cluster [DBG] pgmap v563: 17 pgs: 17 active+clean; 0 B data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 402 op/s 2020-09-08T15:41:53.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:53 smithi104 bash[9203]: cluster 2020-09-08T15:41:51.657761+0000 mgr.x (mgr.24118) 434 : cluster [DBG] pgmap v563: 17 pgs: 17 active+clean; 0 B data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 402 op/s 2020-09-08T15:41:53.993 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:54.098 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:54.221 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:54.325 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:54.338 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:41:54 smithi055 bash[14301]: debug 2020-09-08T15:41:54.225+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:54.338 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:41:53 smithi055 bash[16646]: debug 2020-09-08T15:41:53.997+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:54.559 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:54.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:41:54 smithi055 bash[14301]: debug 2020-09-08T15:41:54.561+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:54.674 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:54.868 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:54.973 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:54.977 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:41:54.999 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:41:54 smithi166 bash[14254]: debug 2020-09-08T15:41:54.867+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:55.000 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:41:55.028 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:41:55.055 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:41:55.080 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:41:55.101 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:41:55.111 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:41:55.138 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:41:55.162 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:41:55.199 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:41:55.210 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:55.221 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:41:55.222 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:41:55.224 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:41:55.254 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:41:55.288 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:41:55.294 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:41:55.294 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:41:55.294 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:41:55.294 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:41:55.295 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:41:55.295 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:41:55.295 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:41:55.295 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:41:55.296 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:41:55.296 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:41:55.296 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:41:55.296 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:41:55.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:41:55 smithi055 bash[11956]: debug 2020-09-08T15:41:55.105+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:55.413 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:55.498 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:41:55 smithi104 bash[19922]: debug 2020-09-08T15:41:55.412+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:55.527 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:55.647 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'unique_pool_0' removed 2020-09-08T15:41:55.853 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:55 smithi166 bash[9446]: cluster 2020-09-08T15:41:53.658435+0000 mgr.x (mgr.24118) 435 : cluster [DBG] pgmap v564: 17 pgs: 17 active+clean; 0 B data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 322 op/s 2020-09-08T15:41:55.854 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:55 smithi166 bash[9446]: audit 2020-09-08T15:41:54.783594+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? v1:172.21.15.104:0/1994204231' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T15:41:55.854 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:55 smithi166 bash[9446]: audit 2020-09-08T15:41:54.959705+0000 mon.a (mon.0) 1106 : audit [DBG] from='client.? v1:172.21.15.104:0/193682566' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:41:55.854 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:55.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:55 smithi055 bash[9296]: cluster 2020-09-08T15:41:53.658435+0000 mgr.x (mgr.24118) 435 : cluster [DBG] pgmap v564: 17 pgs: 17 active+clean; 0 B data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 322 op/s 2020-09-08T15:41:55.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:55 smithi055 bash[9296]: audit 2020-09-08T15:41:54.783594+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? v1:172.21.15.104:0/1994204231' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T15:41:55.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:55 smithi055 bash[9296]: audit 2020-09-08T15:41:54.959705+0000 mon.a (mon.0) 1106 : audit [DBG] from='client.? v1:172.21.15.104:0/193682566' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:41:55.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:55 smithi104 bash[9203]: cluster 2020-09-08T15:41:53.658435+0000 mgr.x (mgr.24118) 435 : cluster [DBG] pgmap v564: 17 pgs: 17 active+clean; 0 B data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 322 op/s 2020-09-08T15:41:55.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:55 smithi104 bash[9203]: audit 2020-09-08T15:41:54.783594+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? v1:172.21.15.104:0/1994204231' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T15:41:55.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:55 smithi104 bash[9203]: audit 2020-09-08T15:41:54.959705+0000 mon.a (mon.0) 1106 : audit [DBG] from='client.? v1:172.21.15.104:0/193682566' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:41:55.961 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:55.974 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2020-09-08T15:41:55.980 INFO:teuthology.orchestra.run.smithi104.stderr:Error ENOENT: pg 2.2 does not exist 2020-09-08T15:41:56.077 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:56.105 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:41:56.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:41:55 smithi166 bash[14254]: debug 2020-09-08T15:41:55.855+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:56.185 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:56.413 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:41:56 smithi055 bash[16646]: debug 2020-09-08T15:41:56.081+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:56.575 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:56.681 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:56.801 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:41:56.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:41:56 smithi166 bash[14254]: debug 2020-09-08T15:41:56.575+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:56.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:56 smithi166 bash[9446]: cluster 2020-09-08T15:41:55.191979+0000 osd.9 (osd.9) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:56.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:56 smithi166 bash[9446]: cluster 2020-09-08T15:41:55.192147+0000 osd.9 (osd.9) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:56.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:56 smithi166 bash[9446]: audit 2020-09-08T15:41:55.648090+0000 mon.a (mon.0) 1107 : audit [INF] from='client.? v1:172.21.15.104:0/1994204231' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T15:41:56.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:56 smithi166 bash[9446]: cluster 2020-09-08T15:41:55.648157+0000 mon.a (mon.0) 1108 : cluster [DBG] osdmap e212: 12 total, 12 up, 12 in 2020-09-08T15:41:56.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:56 smithi166 bash[9446]: audit 2020-09-08T15:41:55.985568+0000 mon.a (mon.0) 1109 : audit [INF] from='client.? v1:172.21.15.104:0/271500251' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "2.2", "id": [0, 7, 6, 2]}]: dispatch 2020-09-08T15:41:56.903 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:41:56.916 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:56 smithi104 bash[9203]: cluster 2020-09-08T15:41:55.191979+0000 osd.9 (osd.9) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:56.916 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:56 smithi104 bash[9203]: cluster 2020-09-08T15:41:55.192147+0000 osd.9 (osd.9) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:56.917 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:56 smithi104 bash[9203]: audit 2020-09-08T15:41:55.648090+0000 mon.a (mon.0) 1107 : audit [INF] from='client.? v1:172.21.15.104:0/1994204231' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T15:41:56.917 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:56 smithi104 bash[9203]: cluster 2020-09-08T15:41:55.648157+0000 mon.a (mon.0) 1108 : cluster [DBG] osdmap e212: 12 total, 12 up, 12 in 2020-09-08T15:41:56.917 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:56 smithi104 bash[9203]: audit 2020-09-08T15:41:55.985568+0000 mon.a (mon.0) 1109 : audit [INF] from='client.? v1:172.21.15.104:0/271500251' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "2.2", "id": [0, 7, 6, 2]}]: dispatch 2020-09-08T15:41:57.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:41:56 smithi055 bash[14301]: debug 2020-09-08T15:41:56.797+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:57.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:56 smithi055 bash[9296]: cluster 2020-09-08T15:41:55.191979+0000 osd.9 (osd.9) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:57.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:56 smithi055 bash[9296]: cluster 2020-09-08T15:41:55.192147+0000 osd.9 (osd.9) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:57.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:56 smithi055 bash[9296]: audit 2020-09-08T15:41:55.648090+0000 mon.a (mon.0) 1107 : audit [INF] from='client.? v1:172.21.15.104:0/1994204231' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T15:41:57.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:56 smithi055 bash[9296]: cluster 2020-09-08T15:41:55.648157+0000 mon.a (mon.0) 1108 : cluster [DBG] osdmap e212: 12 total, 12 up, 12 in 2020-09-08T15:41:57.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:56 smithi055 bash[9296]: audit 2020-09-08T15:41:55.985568+0000 mon.a (mon.0) 1109 : audit [INF] from='client.? v1:172.21.15.104:0/271500251' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "2.2", "id": [0, 7, 6, 2]}]: dispatch 2020-09-08T15:41:57.258 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:41:57.363 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:57.434 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2020-09-08T15:41:57.435 INFO:tasks.radosbench:Beginning radosbench... 2020-09-08T15:41:57.436 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_1 2020-09-08T15:41:57.436 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool create unique_pool_1 16 2020-09-08T15:41:57.491 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:57.530 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:41:57 smithi104 bash[22305]: debug 2020-09-08T15:41:57.256+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:57.531 DEBUG:teuthology.orchestra.run:got remote process result: 2 2020-09-08T15:41:57.532 INFO:tasks.thrashosds.thrasher:Failed to rm-pg-upmap-items, ignoring 2020-09-08T15:41:57.567 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:41:57.585 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:41:57 smithi166 bash[14254]: debug 2020-09-08T15:41:57.491+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:57.596 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:57.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:57 smithi166 bash[9446]: cluster 2020-09-08T15:41:55.658889+0000 mgr.x (mgr.24118) 436 : cluster [DBG] pgmap v566: 1 pgs: 1 active+clean; 0 B data, 696 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:41:57.902 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:57 smithi104 bash[9203]: cluster 2020-09-08T15:41:55.658889+0000 mgr.x (mgr.24118) 436 : cluster [DBG] pgmap v566: 1 pgs: 1 active+clean; 0 B data, 696 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:41:57.903 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:41:58.005 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:58.162 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:57 smithi055 bash[9296]: cluster 2020-09-08T15:41:55.658889+0000 mgr.x (mgr.24118) 436 : cluster [DBG] pgmap v566: 1 pgs: 1 active+clean; 0 B data, 696 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:41:58.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:41:57 smithi104 bash[19922]: debug 2020-09-08T15:41:57.900+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:58.281 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:41:58.387 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:41:58.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:41:58 smithi104 bash[15273]: debug 2020-09-08T15:41:58.280+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:58.680 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:41:58.688 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:41:58.715 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:41:58.814 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:41:58.825 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:41:59.033 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:58 smithi166 bash[9446]: cluster 2020-09-08T15:41:57.659486+0000 mgr.x (mgr.24118) 437 : cluster [DBG] pgmap v567: 1 pgs: 1 active+clean; 0 B data, 704 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:41:59.034 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:58 smithi166 bash[9446]: cluster 2020-09-08T15:41:57.794582+0000 osd.2 (osd.2) 32 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:59.034 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:58 smithi166 bash[9446]: cluster 2020-09-08T15:41:57.794695+0000 osd.2 (osd.2) 33 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:59.034 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:58 smithi166 bash[9446]: audit 2020-09-08T15:41:58.006119+0000 mon.a (mon.0) 1110 : audit [INF] from='client.? v1:172.21.15.104:0/3827662356' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:41:59.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:58 smithi055 bash[9296]: cluster 2020-09-08T15:41:57.659486+0000 mgr.x (mgr.24118) 437 : cluster [DBG] pgmap v567: 1 pgs: 1 active+clean; 0 B data, 704 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:41:59.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:58 smithi055 bash[9296]: cluster 2020-09-08T15:41:57.794582+0000 osd.2 (osd.2) 32 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:59.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:58 smithi055 bash[9296]: cluster 2020-09-08T15:41:57.794695+0000 osd.2 (osd.2) 33 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:59.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:58 smithi055 bash[9296]: audit 2020-09-08T15:41:58.006119+0000 mon.a (mon.0) 1110 : audit [INF] from='client.? v1:172.21.15.104:0/3827662356' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:41:59.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:41:58 smithi104 bash[22305]: debug 2020-09-08T15:41:58.716+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:59.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:58 smithi104 bash[9203]: cluster 2020-09-08T15:41:57.659486+0000 mgr.x (mgr.24118) 437 : cluster [DBG] pgmap v567: 1 pgs: 1 active+clean; 0 B data, 704 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:41:59.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:58 smithi104 bash[9203]: cluster 2020-09-08T15:41:57.794582+0000 osd.2 (osd.2) 32 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:41:59.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:58 smithi104 bash[9203]: cluster 2020-09-08T15:41:57.794695+0000 osd.2 (osd.2) 33 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:41:59.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:58 smithi104 bash[9203]: audit 2020-09-08T15:41:58.006119+0000 mon.a (mon.0) 1110 : audit [INF] from='client.? v1:172.21.15.104:0/3827662356' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:41:59.223 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:41:59.286 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:41:59.287 INFO:teuthology.orchestra.run.smithi166.stdout: "size": 20, 2020-09-08T15:41:59.288 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 600, 2020-09-08T15:41:59.291 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T15:41:59.291 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.293 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:35079 2.c 2:398e8dbe:::benchmark_data_smithi166_20551_object35078:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e135)", 2020-09-08T15:41:59.293 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:37:53.800754+0000", 2020-09-08T15:41:59.294 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 245.486945416, 2020-09-08T15:41:59.294 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.213459918, 2020-09-08T15:41:59.294 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:59.295 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:59.295 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:59.296 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:59.296 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:59.296 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 35079 2020-09-08T15:41:59.296 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.297 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:59.297 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.297 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:59.297 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:53.800754+0000", 2020-09-08T15:41:59.297 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:59.298 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.298 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.299 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:59.299 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:53.800754+0000", 2020-09-08T15:41:59.299 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999633 2020-09-08T15:41:59.299 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.300 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.300 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:59.300 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:53.800717+0000", 2020-09-08T15:41:59.300 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012784600000000001 2020-09-08T15:41:59.300 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.301 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.301 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:59.301 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:53.800845+0000", 2020-09-08T15:41:59.301 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3692e-05 2020-09-08T15:41:59.302 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.302 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.302 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:59.302 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:53.800859+0000", 2020-09-08T15:41:59.302 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7869999999999998e-06 2020-09-08T15:41:59.305 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.306 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.306 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:59.306 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:53.800865+0000", 2020-09-08T15:41:59.306 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0017377270000000001 2020-09-08T15:41:59.306 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.307 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.307 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:59.307 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:53.802603+0000", 2020-09-08T15:41:59.307 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015147500000000001 2020-09-08T15:41:59.307 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.308 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.308 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:59.308 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:53.802754+0000", 2020-09-08T15:41:59.308 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016630900000000001 2020-09-08T15:41:59.308 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.309 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.309 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:59.309 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:53.802920+0000", 2020-09-08T15:41:59.309 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0025625439999999999 2020-09-08T15:41:59.310 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.310 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.310 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:59.310 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:53.805483+0000", 2020-09-08T15:41:59.310 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.208501978 2020-09-08T15:41:59.311 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.311 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.311 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:59.311 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:54.013985+0000", 2020-09-08T15:41:59.311 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000130343 2020-09-08T15:41:59.312 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.312 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.313 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:59.313 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:54.014115+0000", 2020-09-08T15:41:59.315 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.9028000000000001e-05 2020-09-08T15:41:59.315 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.315 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.316 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:59.316 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:54.014214+0000", 2020-09-08T15:41:59.316 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.213459918 2020-09-08T15:41:59.316 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.317 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.317 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.317 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.317 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.318 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:35080 2.c 2:35792630:::benchmark_data_smithi166_20551_object35079:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e135)", 2020-09-08T15:41:59.318 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:37:53.800924+0000", 2020-09-08T15:41:59.318 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 245.48677579700001, 2020-09-08T15:41:59.318 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.213513066, 2020-09-08T15:41:59.318 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:59.319 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:59.319 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:59.319 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:59.319 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:59.320 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 35080 2020-09-08T15:41:59.321 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.321 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:59.321 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.321 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:59.322 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:53.800924+0000", 2020-09-08T15:41:59.322 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:59.322 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.322 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.322 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:59.323 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:53.800924+0000", 2020-09-08T15:41:59.323 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999642 2020-09-08T15:41:59.323 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.323 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.324 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:59.325 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:53.800888+0000", 2020-09-08T15:41:59.325 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012226500000000001 2020-09-08T15:41:59.325 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.326 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.326 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:59.326 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:53.801011+0000", 2020-09-08T15:41:59.326 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3175e-05 2020-09-08T15:41:59.327 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.327 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.327 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:59.328 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:53.801024+0000", 2020-09-08T15:41:59.329 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.6280000000000002e-06 2020-09-08T15:41:59.329 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.329 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.329 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:59.330 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:53.801029+0000", 2020-09-08T15:41:59.330 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.002948553 2020-09-08T15:41:59.330 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.330 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.331 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:59.331 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:53.803978+0000", 2020-09-08T15:41:59.331 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000147238 2020-09-08T15:41:59.331 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.331 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.331 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:59.332 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:53.804125+0000", 2020-09-08T15:41:59.332 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00020102400000000001 2020-09-08T15:41:59.332 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.332 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.332 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:59.333 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:53.804326+0000", 2020-09-08T15:41:59.333 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0012480029999999999 2020-09-08T15:41:59.333 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.333 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.333 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:59.334 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:53.805574+0000", 2020-09-08T15:41:59.334 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20870664999999999 2020-09-08T15:41:59.334 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.336 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.336 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:59.336 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:54.014281+0000", 2020-09-08T15:41:59.337 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6367999999999999e-05 2020-09-08T15:41:59.337 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.337 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.337 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:59.337 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:54.014347+0000", 2020-09-08T15:41:59.338 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.9747999999999995e-05 2020-09-08T15:41:59.338 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.338 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.338 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:59.338 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:37:54.014437+0000", 2020-09-08T15:41:59.339 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.213513066 2020-09-08T15:41:59.339 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.339 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.339 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.339 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.340 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.340 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:168424 2.c 2:3d6437cf:::benchmark_data_smithi166_20551_object20804:head [delete] snapc 0=[] ondisk+write+known_if_redirected e172)", 2020-09-08T15:41:59.340 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:34.504428+0000", 2020-09-08T15:41:59.340 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 144.78327150600001, 2020-09-08T15:41:59.340 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25930227700000003, 2020-09-08T15:41:59.341 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:59.341 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:59.341 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:59.341 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:59.342 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:59.342 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 168424 2020-09-08T15:41:59.342 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.342 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:59.342 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.343 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:59.343 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.504428+0000", 2020-09-08T15:41:59.343 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:59.343 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.343 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.344 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:59.344 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.504428+0000", 2020-09-08T15:41:59.344 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999576 2020-09-08T15:41:59.344 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.346 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.346 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:59.347 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.504386+0000", 2020-09-08T15:41:59.347 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000101994 2020-09-08T15:41:59.347 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.347 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.347 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:59.348 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.504488+0000", 2020-09-08T15:41:59.348 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7666e-05 2020-09-08T15:41:59.348 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.348 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.348 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:59.349 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.504505+0000", 2020-09-08T15:41:59.349 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.579e-06 2020-09-08T15:41:59.349 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.349 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.349 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:59.350 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.504512+0000", 2020-09-08T15:41:59.350 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6767000000000006e-05 2020-09-08T15:41:59.350 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.350 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.350 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:59.351 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.504579+0000", 2020-09-08T15:41:59.351 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000147458 2020-09-08T15:41:59.351 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.351 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.352 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:59.352 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.504726+0000", 2020-09-08T15:41:59.352 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017433300000000001 2020-09-08T15:41:59.352 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.352 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.354 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:59.354 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.504900+0000", 2020-09-08T15:41:59.354 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0086762360000000004 2020-09-08T15:41:59.354 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.355 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.355 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:59.355 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.513577+0000", 2020-09-08T15:41:59.356 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.249556533 2020-09-08T15:41:59.356 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.356 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.356 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:59.356 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.763133+0000", 2020-09-08T15:41:59.357 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00045789300000000002 2020-09-08T15:41:59.357 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.357 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.357 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:59.357 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.763591+0000", 2020-09-08T15:41:59.358 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013938700000000001 2020-09-08T15:41:59.358 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.358 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.358 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:59.358 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.763730+0000", 2020-09-08T15:41:59.359 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25930227700000003 2020-09-08T15:41:59.359 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.359 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.359 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.360 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.360 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.360 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:168426 2.c 2:3c2555c5:::benchmark_data_smithi166_20551_object20806:head [delete] snapc 0=[] ondisk+write+known_if_redirected e172)", 2020-09-08T15:41:59.360 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:34.507744+0000", 2020-09-08T15:41:59.360 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 144.77995561399999, 2020-09-08T15:41:59.361 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.26420644399999998, 2020-09-08T15:41:59.361 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:59.361 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:59.361 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:59.362 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:59.362 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:59.362 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 168426 2020-09-08T15:41:59.362 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.363 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:59.363 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.363 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:59.363 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.507744+0000", 2020-09-08T15:41:59.363 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:59.363 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.364 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.364 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:59.364 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.507744+0000", 2020-09-08T15:41:59.364 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999552 2020-09-08T15:41:59.364 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.365 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.365 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:59.365 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.507699+0000", 2020-09-08T15:41:59.365 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000105511 2020-09-08T15:41:59.365 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.367 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.367 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:59.367 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.507805+0000", 2020-09-08T15:41:59.368 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.2659000000000001e-05 2020-09-08T15:41:59.368 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.368 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.368 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:59.369 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.507828+0000", 2020-09-08T15:41:59.369 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1388e-05 2020-09-08T15:41:59.369 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.369 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.370 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:59.370 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.507839+0000", 2020-09-08T15:41:59.370 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0015584349999999999 2020-09-08T15:41:59.378 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.378 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.378 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:59.379 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.509397+0000", 2020-09-08T15:41:59.379 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013745599999999999 2020-09-08T15:41:59.379 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.379 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.379 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:59.380 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.509535+0000", 2020-09-08T15:41:59.380 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.6337999999999998e-05 2020-09-08T15:41:59.380 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.380 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.380 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:59.381 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.509631+0000", 2020-09-08T15:41:59.381 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0041092610000000003 2020-09-08T15:41:59.381 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.381 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.381 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:59.382 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.513740+0000", 2020-09-08T15:41:59.382 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25808060700000002 2020-09-08T15:41:59.382 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.382 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.383 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:59.383 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.771821+0000", 2020-09-08T15:41:59.383 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.2149000000000003e-05 2020-09-08T15:41:59.384 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.384 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.384 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:59.384 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.771893+0000", 2020-09-08T15:41:59.384 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7364000000000002e-05 2020-09-08T15:41:59.384 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.385 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.385 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:59.385 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.771951+0000", 2020-09-08T15:41:59.385 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.26420644399999998 2020-09-08T15:41:59.385 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.386 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.386 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.386 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.386 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.386 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:168441 2.c 2:309281aa:::benchmark_data_smithi166_20551_object20821:head [delete] snapc 0=[] ondisk+write+known_if_redirected e172)", 2020-09-08T15:41:59.387 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:34.521913+0000", 2020-09-08T15:41:59.387 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 144.76578714999999, 2020-09-08T15:41:59.387 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25015406200000001, 2020-09-08T15:41:59.387 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:59.387 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:59.388 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:59.388 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:59.388 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:59.388 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 168441 2020-09-08T15:41:59.388 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.389 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:59.389 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.389 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:59.389 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.521913+0000", 2020-09-08T15:41:59.389 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:59.389 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.390 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.390 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:59.390 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.521913+0000", 2020-09-08T15:41:59.390 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999542 2020-09-08T15:41:59.390 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.391 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.391 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:59.391 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.521867+0000", 2020-09-08T15:41:59.391 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7751000000000007e-05 2020-09-08T15:41:59.391 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.392 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.392 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:59.392 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.521964+0000", 2020-09-08T15:41:59.392 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4474000000000001e-05 2020-09-08T15:41:59.392 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.392 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.393 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:59.393 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.521979+0000", 2020-09-08T15:41:59.394 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7729999999999996e-06 2020-09-08T15:41:59.394 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.394 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.394 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:59.394 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.521985+0000", 2020-09-08T15:41:59.394 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.4146e-05 2020-09-08T15:41:59.395 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.395 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.395 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:59.395 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.522039+0000", 2020-09-08T15:41:59.395 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016305099999999999 2020-09-08T15:41:59.396 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.396 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.396 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:59.397 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.522202+0000", 2020-09-08T15:41:59.397 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000175203 2020-09-08T15:41:59.397 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.398 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.398 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:59.398 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.522377+0000", 2020-09-08T15:41:59.398 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0062686959999999998 2020-09-08T15:41:59.399 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.399 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.399 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:59.399 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.528646+0000", 2020-09-08T15:41:59.401 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.24335162499999999 2020-09-08T15:41:59.401 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.401 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.401 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:59.402 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.771997+0000", 2020-09-08T15:41:59.402 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.2258000000000002e-05 2020-09-08T15:41:59.402 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.402 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.403 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:59.403 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.772030+0000", 2020-09-08T15:41:59.403 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.7060000000000001e-05 2020-09-08T15:41:59.403 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.403 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.404 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:59.404 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.772067+0000", 2020-09-08T15:41:59.404 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25015406200000001 2020-09-08T15:41:59.404 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.404 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.405 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.405 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.405 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.405 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:168448 2.c 2:338e2e15:::benchmark_data_smithi166_20551_object20828:head [delete] snapc 0=[] ondisk+write+known_if_redirected e172)", 2020-09-08T15:41:59.405 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:34.531525+0000", 2020-09-08T15:41:59.406 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 144.756174493, 2020-09-08T15:41:59.406 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25337866399999998, 2020-09-08T15:41:59.406 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:59.406 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:59.406 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:59.407 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:59.407 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:59.407 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 168448 2020-09-08T15:41:59.407 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.408 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:59.408 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.408 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:59.408 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.531525+0000", 2020-09-08T15:41:59.409 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:59.409 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.410 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.410 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:59.410 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.531525+0000", 2020-09-08T15:41:59.410 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999585 2020-09-08T15:41:59.411 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.411 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.411 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:59.411 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.531484+0000", 2020-09-08T15:41:59.411 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.5507999999999996e-05 2020-09-08T15:41:59.412 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.412 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.412 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:59.412 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.531559+0000", 2020-09-08T15:41:59.412 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3113e-05 2020-09-08T15:41:59.413 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.413 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.413 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:59.413 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.531572+0000", 2020-09-08T15:41:59.413 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.4839999999999998e-06 2020-09-08T15:41:59.414 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.414 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.415 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:59.415 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.531580+0000", 2020-09-08T15:41:59.415 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.558e-05 2020-09-08T15:41:59.415 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.415 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.416 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:59.416 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.531635+0000", 2020-09-08T15:41:59.416 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000183771 2020-09-08T15:41:59.416 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.416 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.416 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:59.417 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.531819+0000", 2020-09-08T15:41:59.417 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000199085 2020-09-08T15:41:59.417 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.417 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.417 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:59.418 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.532018+0000", 2020-09-08T15:41:59.418 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0042823269999999998 2020-09-08T15:41:59.418 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.418 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.418 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:59.419 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.536301+0000", 2020-09-08T15:41:59.419 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.24852070700000001 2020-09-08T15:41:59.419 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.419 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.419 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:59.420 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.784821+0000", 2020-09-08T15:41:59.420 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.6830999999999999e-05 2020-09-08T15:41:59.420 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.420 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.420 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:59.421 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.784868+0000", 2020-09-08T15:41:59.421 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.5642000000000002e-05 2020-09-08T15:41:59.421 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.421 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.421 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:59.422 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.784904+0000", 2020-09-08T15:41:59.422 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25337866399999998 2020-09-08T15:41:59.422 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.422 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.422 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.422 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.423 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.423 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:190035 2.c 2:3fcfe93a:::benchmark_data_smithi166_20551_object42415:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:41:59.423 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:56.753626+0000", 2020-09-08T15:41:59.423 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 122.534073479, 2020-09-08T15:41:59.423 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25899984999999998, 2020-09-08T15:41:59.424 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:59.424 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:59.424 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:59.424 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:59.425 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:59.425 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 190035 2020-09-08T15:41:59.425 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.425 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:59.426 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.426 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:59.426 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.753626+0000", 2020-09-08T15:41:59.426 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:59.426 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.427 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.427 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:59.427 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.753626+0000", 2020-09-08T15:41:59.427 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999628 2020-09-08T15:41:59.427 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.428 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.428 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:59.428 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.753589+0000", 2020-09-08T15:41:59.428 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.5816000000000006e-05 2020-09-08T15:41:59.428 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.429 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.429 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:59.429 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.753675+0000", 2020-09-08T15:41:59.429 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1996e-05 2020-09-08T15:41:59.429 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.430 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.430 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:59.430 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.753687+0000", 2020-09-08T15:41:59.430 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1240000000000004e-06 2020-09-08T15:41:59.430 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.431 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.431 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:59.431 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.753692+0000", 2020-09-08T15:41:59.431 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.2862999999999999e-05 2020-09-08T15:41:59.431 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.431 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.432 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:59.432 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.753745+0000", 2020-09-08T15:41:59.432 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000170662 2020-09-08T15:41:59.432 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.432 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.433 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:59.433 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.753916+0000", 2020-09-08T15:41:59.433 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000180192 2020-09-08T15:41:59.433 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.433 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.434 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:59.434 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.754096+0000", 2020-09-08T15:41:59.434 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0057858969999999999 2020-09-08T15:41:59.434 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.434 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.434 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:59.435 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.759882+0000", 2020-09-08T15:41:59.435 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25262357000000002 2020-09-08T15:41:59.436 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.436 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.436 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:59.436 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.012505+0000", 2020-09-08T15:41:59.436 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.8390000000000001e-05 2020-09-08T15:41:59.437 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.437 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.437 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:59.437 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.012574+0000", 2020-09-08T15:41:59.437 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.2312000000000003e-05 2020-09-08T15:41:59.438 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.438 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.438 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:59.438 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.012626+0000", 2020-09-08T15:41:59.438 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25899984999999998 2020-09-08T15:41:59.439 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.439 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.439 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.439 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.439 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.440 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:190069 2.c 2:34d5f635:::benchmark_data_smithi166_20551_object42449:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:41:59.440 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:56.794135+0000", 2020-09-08T15:41:59.440 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 122.493564632, 2020-09-08T15:41:59.440 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21871511399999999, 2020-09-08T15:41:59.440 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:59.441 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:59.441 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:59.441 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:59.441 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:59.441 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 190069 2020-09-08T15:41:59.442 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.442 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:59.442 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.442 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:59.443 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.794135+0000", 2020-09-08T15:41:59.443 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:59.443 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.443 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.444 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:59.444 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.794135+0000", 2020-09-08T15:41:59.444 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999542 2020-09-08T15:41:59.444 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.445 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.445 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:59.445 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.794089+0000", 2020-09-08T15:41:59.446 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000106322 2020-09-08T15:41:59.446 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.446 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.446 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:59.447 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.794196+0000", 2020-09-08T15:41:59.447 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.1535e-05 2020-09-08T15:41:59.447 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.447 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.447 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:59.447 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.794217+0000", 2020-09-08T15:41:59.448 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.798e-06 2020-09-08T15:41:59.448 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.448 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.448 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:59.448 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.794227+0000", 2020-09-08T15:41:59.449 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.0865000000000003e-05 2020-09-08T15:41:59.449 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.449 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.449 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:59.449 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.794268+0000", 2020-09-08T15:41:59.450 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010985900000000001 2020-09-08T15:41:59.450 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.450 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.450 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:59.450 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.794378+0000", 2020-09-08T15:41:59.451 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000103013 2020-09-08T15:41:59.451 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.451 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.451 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:59.451 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.794481+0000", 2020-09-08T15:41:59.452 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0049067039999999996 2020-09-08T15:41:59.452 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.452 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.452 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:59.452 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.799387+0000", 2020-09-08T15:41:59.452 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21332652299999999 2020-09-08T15:41:59.453 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.453 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.453 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:59.453 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.012714+0000", 2020-09-08T15:41:59.453 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8010000000000002e-05 2020-09-08T15:41:59.454 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.454 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.454 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:59.454 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.012772+0000", 2020-09-08T15:41:59.454 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.8300000000000006e-05 2020-09-08T15:41:59.455 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.455 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.455 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:59.455 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.012850+0000", 2020-09-08T15:41:59.455 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21871511399999999 2020-09-08T15:41:59.456 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.456 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.457 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.457 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.457 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.457 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:190956 2.c 2:3977f071:::benchmark_data_smithi166_20551_object43336:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:41:59.457 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:57.813900+0000", 2020-09-08T15:41:59.458 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 121.473799445, 2020-09-08T15:41:59.458 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25090004500000002, 2020-09-08T15:41:59.458 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:59.458 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:59.458 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:59.458 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:59.459 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:59.459 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 190956 2020-09-08T15:41:59.459 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.459 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:59.459 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.460 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:59.460 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.813900+0000", 2020-09-08T15:41:59.460 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:59.460 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.460 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.461 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:59.461 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.813900+0000", 2020-09-08T15:41:59.461 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999676 2020-09-08T15:41:59.461 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.461 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.462 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:59.462 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.813868+0000", 2020-09-08T15:41:59.462 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5728000000000005e-05 2020-09-08T15:41:59.462 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.462 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.462 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:59.463 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.813934+0000", 2020-09-08T15:41:59.463 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8980000000000002e-06 2020-09-08T15:41:59.463 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.463 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.463 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:59.464 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.813942+0000", 2020-09-08T15:41:59.464 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.5050000000000001e-06 2020-09-08T15:41:59.464 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.464 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.464 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:59.465 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.813947+0000", 2020-09-08T15:41:59.465 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.8933000000000001e-05 2020-09-08T15:41:59.465 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.465 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.465 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:59.466 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.813986+0000", 2020-09-08T15:41:59.466 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000113168 2020-09-08T15:41:59.466 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.466 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.467 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:59.467 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.814099+0000", 2020-09-08T15:41:59.467 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00036050500000000001 2020-09-08T15:41:59.467 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.468 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.468 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:59.468 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.814460+0000", 2020-09-08T15:41:59.468 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.010845913 2020-09-08T15:41:59.468 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.469 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.469 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:59.469 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.825305+0000", 2020-09-08T15:41:59.469 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.239296599 2020-09-08T15:41:59.469 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.470 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.470 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:59.470 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.064602+0000", 2020-09-08T15:41:59.470 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010784400000000001 2020-09-08T15:41:59.470 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.470 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.471 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:59.471 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.064710+0000", 2020-09-08T15:41:59.471 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.0341999999999997e-05 2020-09-08T15:41:59.471 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.471 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.472 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:59.472 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.064800+0000", 2020-09-08T15:41:59.475 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25090004500000002 2020-09-08T15:41:59.475 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.475 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.475 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.475 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.476 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.476 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:190977 2.c 2:353f5957:::benchmark_data_smithi166_20551_object43357:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:41:59.476 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:57.833153+0000", 2020-09-08T15:41:59.476 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 121.454546695, 2020-09-08T15:41:59.476 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.27689429799999998, 2020-09-08T15:41:59.477 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:59.477 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:59.478 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:59.478 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:59.478 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:59.478 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 190977 2020-09-08T15:41:59.478 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.479 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:59.479 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.479 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:59.479 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.833153+0000", 2020-09-08T15:41:59.479 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:59.480 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.480 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.480 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:59.480 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.833153+0000", 2020-09-08T15:41:59.480 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999781 2020-09-08T15:41:59.480 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.481 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.481 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:59.481 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.833131+0000", 2020-09-08T15:41:59.481 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0523999999999999e-05 2020-09-08T15:41:59.481 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.482 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.482 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:59.482 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.833181+0000", 2020-09-08T15:41:59.482 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2450000000000004e-06 2020-09-08T15:41:59.482 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.483 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.483 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:59.483 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.833191+0000", 2020-09-08T15:41:59.483 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.4150000000000001e-06 2020-09-08T15:41:59.483 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.484 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.484 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:59.484 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.833195+0000", 2020-09-08T15:41:59.484 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.6110999999999999e-05 2020-09-08T15:41:59.484 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.484 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.485 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:59.485 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.833241+0000", 2020-09-08T15:41:59.485 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014592500000000001 2020-09-08T15:41:59.485 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.485 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.486 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:59.486 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.833387+0000", 2020-09-08T15:41:59.486 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016657600000000001 2020-09-08T15:41:59.486 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.486 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.487 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:59.487 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.833554+0000", 2020-09-08T15:41:59.487 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0054606749999999999 2020-09-08T15:41:59.488 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.488 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.488 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:59.488 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.839014+0000", 2020-09-08T15:41:59.488 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.27090976 2020-09-08T15:41:59.489 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.489 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.489 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:59.489 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.109924+0000", 2020-09-08T15:41:59.489 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.1544000000000005e-05 2020-09-08T15:41:59.490 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.490 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.490 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:59.490 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.109996+0000", 2020-09-08T15:41:59.490 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1629999999999999e-05 2020-09-08T15:41:59.491 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.491 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.491 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:59.491 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.110047+0000", 2020-09-08T15:41:59.491 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.27689429799999998 2020-09-08T15:41:59.492 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.492 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.492 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.492 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.492 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.493 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:190986 2.c 2:3a92fc8e:::benchmark_data_smithi166_20551_object43366:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:41:59.493 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:57.844166+0000", 2020-09-08T15:41:59.493 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 121.44353334500001, 2020-09-08T15:41:59.493 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.26633395199999998, 2020-09-08T15:41:59.493 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:59.494 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:59.494 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:59.494 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:59.494 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:59.494 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 190986 2020-09-08T15:41:59.495 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.495 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:59.495 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.495 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:59.495 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.844166+0000", 2020-09-08T15:41:59.495 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:59.496 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.496 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.496 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:59.496 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.844166+0000", 2020-09-08T15:41:59.496 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999423 2020-09-08T15:41:59.497 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.497 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.497 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:59.497 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.844109+0000", 2020-09-08T15:41:59.497 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000114329 2020-09-08T15:41:59.498 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.498 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.498 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:59.499 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.844223+0000", 2020-09-08T15:41:59.499 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.4448e-05 2020-09-08T15:41:59.499 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.499 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.499 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:59.500 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.844258+0000", 2020-09-08T15:41:59.500 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.1486999999999999e-05 2020-09-08T15:41:59.500 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.500 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.500 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:59.501 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.844289+0000", 2020-09-08T15:41:59.501 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.4245e-05 2020-09-08T15:41:59.501 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.501 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.501 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:59.501 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.844343+0000", 2020-09-08T15:41:59.502 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000118576 2020-09-08T15:41:59.502 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.502 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.502 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:59.502 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.844462+0000", 2020-09-08T15:41:59.503 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015451199999999999 2020-09-08T15:41:59.503 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.503 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.503 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:59.503 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.844616+0000", 2020-09-08T15:41:59.504 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0046295770000000002 2020-09-08T15:41:59.504 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.504 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.504 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:59.504 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.849246+0000", 2020-09-08T15:41:59.505 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.26095940899999998 2020-09-08T15:41:59.505 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.505 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.505 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:59.505 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.110205+0000", 2020-09-08T15:41:59.505 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000241635 2020-09-08T15:41:59.506 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.506 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.506 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:59.506 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.110447+0000", 2020-09-08T15:41:59.506 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.3251000000000002e-05 2020-09-08T15:41:59.507 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.507 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.507 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:59.507 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.110500+0000", 2020-09-08T15:41:59.507 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.26633395199999998 2020-09-08T15:41:59.508 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.508 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.508 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.509 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.509 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.509 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:190993 2.c 2:3c0228a4:::benchmark_data_smithi166_20551_object43373:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:41:59.509 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:57.853648+0000", 2020-09-08T15:41:59.509 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 121.43405184, 2020-09-08T15:41:59.510 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25717041800000001, 2020-09-08T15:41:59.510 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:59.510 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:59.510 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:59.510 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:59.511 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:59.511 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 190993 2020-09-08T15:41:59.511 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.511 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:59.511 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.512 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:59.512 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.853648+0000", 2020-09-08T15:41:59.512 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:59.512 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.512 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.513 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:59.513 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.853648+0000", 2020-09-08T15:41:59.513 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.999958 2020-09-08T15:41:59.513 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.513 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.514 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:59.514 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.853606+0000", 2020-09-08T15:41:59.514 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7428000000000005e-05 2020-09-08T15:41:59.514 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.514 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.515 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:59.515 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.853683+0000", 2020-09-08T15:41:59.515 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.3149999999999992e-06 2020-09-08T15:41:59.515 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.515 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.515 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:59.516 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.853692+0000", 2020-09-08T15:41:59.516 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.2200000000000003e-06 2020-09-08T15:41:59.516 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.516 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.516 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:59.517 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.853696+0000", 2020-09-08T15:41:59.517 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.7911000000000003e-05 2020-09-08T15:41:59.517 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.517 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.517 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:59.518 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.853744+0000", 2020-09-08T15:41:59.518 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016690300000000001 2020-09-08T15:41:59.518 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.518 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.518 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:59.518 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.853911+0000", 2020-09-08T15:41:59.519 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000175969 2020-09-08T15:41:59.519 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.520 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.520 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:59.520 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.854087+0000", 2020-09-08T15:41:59.520 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0068571120000000003 2020-09-08T15:41:59.520 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.521 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.521 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:59.521 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.860944+0000", 2020-09-08T15:41:59.521 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.24959942399999999 2020-09-08T15:41:59.521 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.522 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.522 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:59.522 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.110543+0000", 2020-09-08T15:41:59.522 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000212523 2020-09-08T15:41:59.522 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.523 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.523 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:59.523 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.110756+0000", 2020-09-08T15:41:59.523 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.2508000000000005e-05 2020-09-08T15:41:59.523 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.523 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.524 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:59.524 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.110818+0000", 2020-09-08T15:41:59.524 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25717041800000001 2020-09-08T15:41:59.524 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.524 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.525 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.525 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.525 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.525 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:191004 2.c 2:340ca489:::benchmark_data_smithi166_20551_object43384:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:41:59.525 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:57.862433+0000", 2020-09-08T15:41:59.526 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 121.425266307, 2020-09-08T15:41:59.526 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.24891711999999999, 2020-09-08T15:41:59.526 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:59.526 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:59.526 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:59.527 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:59.527 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:59.527 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 191004 2020-09-08T15:41:59.527 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.527 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:59.527 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.528 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:59.528 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862433+0000", 2020-09-08T15:41:59.528 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:59.528 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.528 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.529 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:59.529 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862433+0000", 2020-09-08T15:41:59.529 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9998598 2020-09-08T15:41:59.530 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.530 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.530 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:59.530 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862293+0000", 2020-09-08T15:41:59.531 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019292800000000001 2020-09-08T15:41:59.531 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.531 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.531 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:59.531 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862486+0000", 2020-09-08T15:41:59.532 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5061e-05 2020-09-08T15:41:59.532 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.532 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.532 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:59.532 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862501+0000", 2020-09-08T15:41:59.532 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.3040000000000001e-06 2020-09-08T15:41:59.533 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.533 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.533 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:59.533 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862507+0000", 2020-09-08T15:41:59.533 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.3510000000000006e-05 2020-09-08T15:41:59.534 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.534 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.534 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:59.534 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862581+0000", 2020-09-08T15:41:59.534 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000134311 2020-09-08T15:41:59.535 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.535 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.535 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:59.535 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862715+0000", 2020-09-08T15:41:59.535 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013944700000000001 2020-09-08T15:41:59.535 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.536 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.536 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:59.536 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862855+0000", 2020-09-08T15:41:59.536 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0054684429999999999 2020-09-08T15:41:59.536 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.537 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.537 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:59.537 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.868323+0000", 2020-09-08T15:41:59.537 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.242747723 2020-09-08T15:41:59.537 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.538 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.538 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:59.538 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.111071+0000", 2020-09-08T15:41:59.538 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000222062 2020-09-08T15:41:59.538 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.539 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.539 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:59.539 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.111293+0000", 2020-09-08T15:41:59.539 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7626e-05 2020-09-08T15:41:59.539 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.539 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.540 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:59.540 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.111350+0000", 2020-09-08T15:41:59.541 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.24891711999999999 2020-09-08T15:41:59.541 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.541 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.541 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.541 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.542 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.542 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223524 2.c 2:3ad6268a:::benchmark_data_smithi166_20551_object75904:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:41:59.542 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.443068+0000", 2020-09-08T15:41:59.542 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 87.844632136000001, 2020-09-08T15:41:59.542 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.312334479, 2020-09-08T15:41:59.543 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:59.543 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:59.543 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:59.543 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:59.544 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:59.544 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223524 2020-09-08T15:41:59.544 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.544 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:59.545 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.545 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:59.545 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.443068+0000", 2020-09-08T15:41:59.545 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:59.546 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.546 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.546 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:59.546 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.443068+0000", 2020-09-08T15:41:59.546 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999752 2020-09-08T15:41:59.547 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.547 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.547 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:59.547 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.443043+0000", 2020-09-08T15:41:59.547 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.0220999999999998e-05 2020-09-08T15:41:59.547 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.548 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.548 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:59.548 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.443103+0000", 2020-09-08T15:41:59.548 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2289999999999992e-06 2020-09-08T15:41:59.548 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.549 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.549 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:59.549 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.443112+0000", 2020-09-08T15:41:59.549 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9359999999999998e-06 2020-09-08T15:41:59.549 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.550 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.550 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:59.551 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.443117+0000", 2020-09-08T15:41:59.551 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0022104389999999998 2020-09-08T15:41:59.551 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.551 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.551 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:59.552 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.445328+0000", 2020-09-08T15:41:59.552 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015247200000000001 2020-09-08T15:41:59.552 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.552 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.552 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:59.552 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.445480+0000", 2020-09-08T15:41:59.553 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016037700000000001 2020-09-08T15:41:59.553 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.553 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.553 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:59.553 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.445640+0000", 2020-09-08T15:41:59.554 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0047941390000000002 2020-09-08T15:41:59.554 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.554 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.554 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:59.555 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.450435+0000", 2020-09-08T15:41:59.555 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.30471783000000002 2020-09-08T15:41:59.555 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.555 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.555 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:59.555 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.755152+0000", 2020-09-08T15:41:59.556 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011526100000000001 2020-09-08T15:41:59.556 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.556 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.556 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:59.556 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.755268+0000", 2020-09-08T15:41:59.557 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000134411 2020-09-08T15:41:59.557 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.557 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.557 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:59.557 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.755402+0000", 2020-09-08T15:41:59.558 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.312334479 2020-09-08T15:41:59.558 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.558 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.558 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.558 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.558 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.559 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223535 2.c 2:3b35efb1:::benchmark_data_smithi166_20551_object75915:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:41:59.559 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.454088+0000", 2020-09-08T15:41:59.559 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 87.833611429000001, 2020-09-08T15:41:59.559 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.30184969299999997, 2020-09-08T15:41:59.559 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:59.560 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:59.560 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:59.560 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:59.560 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:59.561 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223535 2020-09-08T15:41:59.561 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.561 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:59.561 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.562 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:59.562 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454088+0000", 2020-09-08T15:41:59.562 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:59.562 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.562 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.563 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:59.563 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454088+0000", 2020-09-08T15:41:59.563 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999619 2020-09-08T15:41:59.564 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.564 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.564 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:59.564 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454050+0000", 2020-09-08T15:41:59.564 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7121999999999996e-05 2020-09-08T15:41:59.565 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.565 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.565 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:59.565 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454147+0000", 2020-09-08T15:41:59.565 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.0890000000000002e-05 2020-09-08T15:41:59.565 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.566 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.566 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:59.566 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454168+0000", 2020-09-08T15:41:59.566 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0044000000000001e-05 2020-09-08T15:41:59.566 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.567 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.567 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:59.567 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454178+0000", 2020-09-08T15:41:59.567 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.1668999999999995e-05 2020-09-08T15:41:59.567 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.568 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.568 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:59.568 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454240+0000", 2020-09-08T15:41:59.568 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00020094000000000001 2020-09-08T15:41:59.568 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.568 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.569 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:59.569 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454441+0000", 2020-09-08T15:41:59.569 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000217865 2020-09-08T15:41:59.569 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.569 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.570 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:59.570 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454658+0000", 2020-09-08T15:41:59.570 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0053479290000000004 2020-09-08T15:41:59.570 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.570 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.571 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:59.571 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.460006+0000", 2020-09-08T15:41:59.571 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.29555245899999999 2020-09-08T15:41:59.572 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.572 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.572 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:59.572 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.755559+0000", 2020-09-08T15:41:59.572 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000214642 2020-09-08T15:41:59.573 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.573 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.573 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:59.573 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.755773+0000", 2020-09-08T15:41:59.573 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000164463 2020-09-08T15:41:59.574 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.574 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.574 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:59.574 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.755938+0000", 2020-09-08T15:41:59.575 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.30184969299999997 2020-09-08T15:41:59.575 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.575 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.576 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.576 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.576 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.576 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223545 2.c 2:3ce5f60b:::benchmark_data_smithi166_20551_object75925:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:41:59.576 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.463157+0000", 2020-09-08T15:41:59.577 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 87.824543156999994, 2020-09-08T15:41:59.577 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.31968120500000002, 2020-09-08T15:41:59.577 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:59.577 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:59.577 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:59.578 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:59.578 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:59.578 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223545 2020-09-08T15:41:59.578 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.578 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:59.579 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.579 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:59.579 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.463157+0000", 2020-09-08T15:41:59.579 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:59.579 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.579 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.580 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:59.580 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.463157+0000", 2020-09-08T15:41:59.580 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999375 2020-09-08T15:41:59.580 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.580 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.581 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:59.581 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.463094+0000", 2020-09-08T15:41:59.581 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013685800000000001 2020-09-08T15:41:59.581 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.582 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.582 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:59.583 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.463231+0000", 2020-09-08T15:41:59.583 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.6152999999999998e-05 2020-09-08T15:41:59.583 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.583 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.584 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:59.584 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.463257+0000", 2020-09-08T15:41:59.584 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1065e-05 2020-09-08T15:41:59.584 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.584 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.585 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:59.585 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.463268+0000", 2020-09-08T15:41:59.585 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.2441000000000002e-05 2020-09-08T15:41:59.585 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.585 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.586 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:59.586 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.463311+0000", 2020-09-08T15:41:59.586 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.9934999999999998e-05 2020-09-08T15:41:59.586 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.586 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.586 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:59.587 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.463411+0000", 2020-09-08T15:41:59.587 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.4406000000000004e-05 2020-09-08T15:41:59.587 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.587 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.587 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:59.588 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.463505+0000", 2020-09-08T15:41:59.588 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0050517319999999998 2020-09-08T15:41:59.588 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.588 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.588 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:59.589 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.468557+0000", 2020-09-08T15:41:59.589 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.31419107800000001 2020-09-08T15:41:59.589 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.589 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.589 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:59.590 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.782748+0000", 2020-09-08T15:41:59.591 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.2206999999999997e-05 2020-09-08T15:41:59.591 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.592 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.592 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:59.592 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.782800+0000", 2020-09-08T15:41:59.592 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.7558999999999997e-05 2020-09-08T15:41:59.593 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.593 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.593 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:59.593 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.782838+0000", 2020-09-08T15:41:59.593 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.31968120500000002 2020-09-08T15:41:59.594 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.594 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.594 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.594 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.594 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.595 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223556 2.c 2:3a873e3a:::benchmark_data_smithi166_20551_object75936:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:41:59.595 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.473508+0000", 2020-09-08T15:41:59.595 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 87.814191816000005, 2020-09-08T15:41:59.595 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.30949394699999999, 2020-09-08T15:41:59.595 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:59.595 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:59.596 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:59.596 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:59.596 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:59.596 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223556 2020-09-08T15:41:59.596 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.597 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:59.597 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.597 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:59.597 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.473508+0000", 2020-09-08T15:41:59.597 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:59.598 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.598 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.598 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:59.598 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.473508+0000", 2020-09-08T15:41:59.598 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999657 2020-09-08T15:41:59.599 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.599 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.599 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:59.599 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.473474+0000", 2020-09-08T15:41:59.599 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.2657000000000005e-05 2020-09-08T15:41:59.599 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.600 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.600 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:59.600 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.473556+0000", 2020-09-08T15:41:59.600 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.2762e-05 2020-09-08T15:41:59.600 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.601 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.601 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:59.601 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.473579+0000", 2020-09-08T15:41:59.601 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.2389999999999996e-06 2020-09-08T15:41:59.601 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.602 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.602 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:59.602 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.473586+0000", 2020-09-08T15:41:59.602 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.3835999999999998e-05 2020-09-08T15:41:59.602 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.603 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.603 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:59.603 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.473640+0000", 2020-09-08T15:41:59.603 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015941100000000001 2020-09-08T15:41:59.603 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.604 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.604 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:59.604 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.473799+0000", 2020-09-08T15:41:59.604 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000184205 2020-09-08T15:41:59.604 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.605 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.605 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:59.605 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.473984+0000", 2020-09-08T15:41:59.605 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0057273860000000001 2020-09-08T15:41:59.605 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.606 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.606 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:59.606 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.479711+0000", 2020-09-08T15:41:59.606 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.30318046300000001 2020-09-08T15:41:59.606 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.606 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.607 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:59.607 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.782891+0000", 2020-09-08T15:41:59.607 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6531000000000004e-05 2020-09-08T15:41:59.607 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.607 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.608 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:59.608 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.782958+0000", 2020-09-08T15:41:59.608 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.3819999999999997e-05 2020-09-08T15:41:59.608 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.608 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.609 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:59.609 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.783002+0000", 2020-09-08T15:41:59.609 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.30949394699999999 2020-09-08T15:41:59.609 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.610 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.610 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.610 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.610 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.610 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223569 2.c 2:33a4eac1:::benchmark_data_smithi166_20551_object75949:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:41:59.611 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.491343+0000", 2020-09-08T15:41:59.611 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 87.796357056999994, 2020-09-08T15:41:59.611 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.29718613599999999, 2020-09-08T15:41:59.611 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:59.611 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:59.612 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:59.612 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:59.612 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:59.612 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223569 2020-09-08T15:41:59.612 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.613 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:59.613 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.613 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:59.613 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.491343+0000", 2020-09-08T15:41:59.613 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:59.614 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.614 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.614 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:59.614 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.491343+0000", 2020-09-08T15:41:59.614 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999657 2020-09-08T15:41:59.614 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.615 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.615 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:59.615 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.491308+0000", 2020-09-08T15:41:59.615 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.0702000000000001e-05 2020-09-08T15:41:59.615 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.616 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.616 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:59.616 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.491389+0000", 2020-09-08T15:41:59.616 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.8026999999999999e-05 2020-09-08T15:41:59.616 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.617 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.617 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:59.617 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.491407+0000", 2020-09-08T15:41:59.617 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.4380000000000001e-06 2020-09-08T15:41:59.617 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.618 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.618 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:59.618 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.491417+0000", 2020-09-08T15:41:59.618 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.0940999999999998e-05 2020-09-08T15:41:59.618 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.619 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.619 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:59.619 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.491478+0000", 2020-09-08T15:41:59.619 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016345800000000001 2020-09-08T15:41:59.619 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.619 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.620 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:59.620 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.491641+0000", 2020-09-08T15:41:59.620 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016969199999999999 2020-09-08T15:41:59.620 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.620 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.621 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:59.621 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.491811+0000", 2020-09-08T15:41:59.621 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.010406499 2020-09-08T15:41:59.621 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.621 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.622 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:59.622 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.502217+0000", 2020-09-08T15:41:59.622 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.286036083 2020-09-08T15:41:59.622 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.622 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.623 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:59.623 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.788253+0000", 2020-09-08T15:41:59.623 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015021899999999999 2020-09-08T15:41:59.623 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.623 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.624 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:59.624 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.788404+0000", 2020-09-08T15:41:59.625 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012526200000000001 2020-09-08T15:41:59.625 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.625 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.626 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:59.626 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.788529+0000", 2020-09-08T15:41:59.626 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.29718613599999999 2020-09-08T15:41:59.626 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.626 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.627 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.627 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.627 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.627 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223572 2.c 2:350e5a44:::benchmark_data_smithi166_20551_object75952:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:41:59.627 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.494958+0000", 2020-09-08T15:41:59.627 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 87.792742128, 2020-09-08T15:41:59.628 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.29377856200000002, 2020-09-08T15:41:59.628 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:59.628 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:59.628 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:59.628 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:59.629 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:59.629 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223572 2020-09-08T15:41:59.629 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.629 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:59.629 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.630 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:59.630 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.494958+0000", 2020-09-08T15:41:59.630 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:59.630 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.630 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.631 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:59.631 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.494958+0000", 2020-09-08T15:41:59.631 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999638 2020-09-08T15:41:59.631 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.631 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.632 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:59.632 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.494921+0000", 2020-09-08T15:41:59.633 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.4671000000000004e-05 2020-09-08T15:41:59.634 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.634 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.634 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:59.635 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.494996+0000", 2020-09-08T15:41:59.635 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6960999999999999e-05 2020-09-08T15:41:59.635 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.636 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.637 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:59.637 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.495013+0000", 2020-09-08T15:41:59.638 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.276e-06 2020-09-08T15:41:59.638 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.638 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.638 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:59.638 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.495021+0000", 2020-09-08T15:41:59.639 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0018827360000000001 2020-09-08T15:41:59.639 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.639 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.639 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:59.639 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.496904+0000", 2020-09-08T15:41:59.640 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000124506 2020-09-08T15:41:59.640 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.640 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.640 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:59.640 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.497028+0000", 2020-09-08T15:41:59.641 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010885100000000001 2020-09-08T15:41:59.641 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.641 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.641 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:59.641 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.497137+0000", 2020-09-08T15:41:59.642 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.005206759 2020-09-08T15:41:59.642 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.642 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.642 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:59.642 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.502344+0000", 2020-09-08T15:41:59.643 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.28627132700000002 2020-09-08T15:41:59.643 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.643 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.643 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:59.643 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.788615+0000", 2020-09-08T15:41:59.644 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5796000000000005e-05 2020-09-08T15:41:59.644 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.644 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.644 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:59.644 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.788681+0000", 2020-09-08T15:41:59.644 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.5087999999999998e-05 2020-09-08T15:41:59.645 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.645 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.645 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:59.645 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.788736+0000", 2020-09-08T15:41:59.645 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.29377856200000002 2020-09-08T15:41:59.646 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.646 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.646 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.647 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.647 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.647 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223588 2.c 2:389e62f7:::benchmark_data_smithi166_20551_object75968:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:41:59.650 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.533630+0000", 2020-09-08T15:41:59.650 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 87.754069866999998, 2020-09-08T15:41:59.650 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.26923397700000001, 2020-09-08T15:41:59.650 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:41:59.651 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:41:59.651 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:41:59.651 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:41:59.651 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:41:59.651 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223588 2020-09-08T15:41:59.652 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.652 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:41:59.652 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.652 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:41:59.652 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.533630+0000", 2020-09-08T15:41:59.653 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:41:59.653 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.653 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.654 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:41:59.654 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.533630+0000", 2020-09-08T15:41:59.655 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999485 2020-09-08T15:41:59.655 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.655 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.655 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:41:59.656 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.533578+0000", 2020-09-08T15:41:59.656 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0001155 2020-09-08T15:41:59.656 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.657 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.657 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:41:59.657 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.533694+0000", 2020-09-08T15:41:59.657 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.0339999999999998e-05 2020-09-08T15:41:59.657 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.658 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.658 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:41:59.658 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.533714+0000", 2020-09-08T15:41:59.659 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.8649999999999996e-06 2020-09-08T15:41:59.659 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.659 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.659 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:41:59.660 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.533724+0000", 2020-09-08T15:41:59.660 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.905e-05 2020-09-08T15:41:59.660 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.660 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.660 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:41:59.661 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.533773+0000", 2020-09-08T15:41:59.661 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014188800000000001 2020-09-08T15:41:59.661 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.661 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.661 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:41:59.662 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.533915+0000", 2020-09-08T15:41:59.662 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013381800000000001 2020-09-08T15:41:59.662 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.662 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.662 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:41:59.663 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.534049+0000", 2020-09-08T15:41:59.663 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.004890569 2020-09-08T15:41:59.663 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.663 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.663 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:41:59.664 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.538939+0000", 2020-09-08T15:41:59.664 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.26370252500000002 2020-09-08T15:41:59.664 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.665 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.665 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:41:59.666 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.802642+0000", 2020-09-08T15:41:59.666 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7527000000000005e-05 2020-09-08T15:41:59.666 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.666 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.666 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:41:59.667 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.802719+0000", 2020-09-08T15:41:59.667 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014438299999999999 2020-09-08T15:41:59.667 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:41:59.667 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:41:59.668 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:41:59.668 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.802864+0000", 2020-09-08T15:41:59.668 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.26923397700000001 2020-09-08T15:41:59.668 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.669 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.669 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.669 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:41:59.669 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:41:59.669 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:41:59.671 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:59.679 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:41:59 smithi104 bash[17588]: debug 2020-09-08T15:41:59.220+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:41:59.786 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:41:59.892 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:41:59.966 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:59 smithi166 bash[9446]: cluster 2020-09-08T15:41:58.673028+0000 mon.a (mon.0) 1111 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:41:59.966 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:59 smithi166 bash[9446]: cluster 2020-09-08T15:41:58.673093+0000 mon.a (mon.0) 1112 : cluster [INF] Cluster is now healthy 2020-09-08T15:41:59.967 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:59 smithi166 bash[9446]: audit 2020-09-08T15:41:58.680577+0000 mon.a (mon.0) 1113 : audit [INF] from='client.? v1:172.21.15.104:0/3827662356' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:41:59.967 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:41:59 smithi166 bash[9446]: cluster 2020-09-08T15:41:58.680669+0000 mon.a (mon.0) 1114 : cluster [DBG] osdmap e213: 12 total, 12 up, 12 in 2020-09-08T15:41:59.968 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2020-09-08T15:41:59.982 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:59 smithi104 bash[9203]: cluster 2020-09-08T15:41:58.673028+0000 mon.a (mon.0) 1111 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:41:59.982 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:59 smithi104 bash[9203]: cluster 2020-09-08T15:41:58.673093+0000 mon.a (mon.0) 1112 : cluster [INF] Cluster is now healthy 2020-09-08T15:41:59.982 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:59 smithi104 bash[9203]: audit 2020-09-08T15:41:58.680577+0000 mon.a (mon.0) 1113 : audit [INF] from='client.? v1:172.21.15.104:0/3827662356' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:41:59.984 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:41:59 smithi104 bash[9203]: cluster 2020-09-08T15:41:58.680669+0000 mon.a (mon.0) 1114 : cluster [DBG] osdmap e213: 12 total, 12 up, 12 in 2020-09-08T15:42:00.015 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:42:00.116 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:42:00.121 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:00.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:41:59 smithi055 bash[16646]: debug 2020-09-08T15:41:59.789+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:00.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:59 smithi055 bash[9296]: cluster 2020-09-08T15:41:58.673028+0000 mon.a (mon.0) 1111 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:42:00.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:59 smithi055 bash[9296]: cluster 2020-09-08T15:41:58.673093+0000 mon.a (mon.0) 1112 : cluster [INF] Cluster is now healthy 2020-09-08T15:42:00.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:59 smithi055 bash[9296]: audit 2020-09-08T15:41:58.680577+0000 mon.a (mon.0) 1113 : audit [INF] from='client.? v1:172.21.15.104:0/3827662356' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:42:00.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:41:59 smithi055 bash[9296]: cluster 2020-09-08T15:41:58.680669+0000 mon.a (mon.0) 1114 : cluster [DBG] osdmap e213: 12 total, 12 up, 12 in 2020-09-08T15:42:00.235 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:42:00 smithi166 bash[14254]: debug 2020-09-08T15:42:00.015+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:00.236 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:00.342 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:00.475 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:00.582 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:00.615 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:42:00 smithi166 bash[16593]: debug 2020-09-08T15:42:00.239+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:00.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:42:00 smithi166 bash[16593]: debug 2020-09-08T15:42:00.479+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:00.688 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:00.794 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:00.908 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:42:00 smithi055 bash[16646]: debug 2020-09-08T15:42:00.689+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:00.914 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:01.019 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:01.149 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:01.262 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:42:01.408 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:42:00 smithi055 bash[11956]: debug 2020-09-08T15:42:00.917+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:01.408 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:42:01 smithi055 bash[11956]: debug 2020-09-08T15:42:01.149+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:01.544 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:42:01.652 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:01.813 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:01.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:01 smithi055 bash[9296]: cluster 2020-09-08T15:41:59.660089+0000 mgr.x (mgr.24118) 438 : cluster [DBG] pgmap v569: 1 pgs: 1 active+clean; 0 B data, 709 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:01.920 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:01.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:01 smithi104 bash[9203]: cluster 2020-09-08T15:41:59.660089+0000 mgr.x (mgr.24118) 438 : cluster [DBG] pgmap v569: 1 pgs: 1 active+clean; 0 B data, 709 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:01.933 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:42:01 smithi104 bash[15273]: debug 2020-09-08T15:42:01.544+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:02.018 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:02.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:01 smithi166 bash[9446]: cluster 2020-09-08T15:41:59.660089+0000 mgr.x (mgr.24118) 438 : cluster [DBG] pgmap v569: 1 pgs: 1 active+clean; 0 B data, 709 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:02.120 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:42:01 smithi166 bash[11928]: debug 2020-09-08T15:42:01.815+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:02.123 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:02.248 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:42:02 smithi055 bash[14301]: debug 2020-09-08T15:42:02.021+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:02.249 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:02.354 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:02.492 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:02.534 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2020-09-08T15:42:02.534 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-09-08T15:42:02.534 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2020-09-08T15:42:02.535 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd out 1 2020-09-08T15:42:02.564 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:42:02.598 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:02.658 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:42:02 smithi055 bash[11956]: debug 2020-09-08T15:42:02.253+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:02.658 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:42:02 smithi055 bash[19003]: debug 2020-09-08T15:42:02.493+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:02.677 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:42:02.723 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:42:02.724 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:02.831 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:02.865 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:42:02 smithi166 bash[11928]: debug 2020-09-08T15:42:02.727+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:02.955 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:03.065 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:03.160 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:42:02 smithi055 bash[19003]: debug 2020-09-08T15:42:02.957+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:03.391 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:03.493 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:03.627 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:03.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:42:03 smithi104 bash[17588]: debug 2020-09-08T15:42:03.388+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:03.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:03 smithi104 bash[9203]: cluster 2020-09-08T15:42:01.660690+0000 mgr.x (mgr.24118) 439 : cluster [DBG] pgmap v570: 1 pgs: 1 active+clean; 0 B data, 710 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:03.733 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:03.867 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:03.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:03 smithi055 bash[9296]: cluster 2020-09-08T15:42:01.660690+0000 mgr.x (mgr.24118) 439 : cluster [DBG] pgmap v570: 1 pgs: 1 active+clean; 0 B data, 710 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:03.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:42:03 smithi055 bash[14301]: debug 2020-09-08T15:42:03.629+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:03.977 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:03.991 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:03 smithi166 bash[9446]: cluster 2020-09-08T15:42:01.660690+0000 mgr.x (mgr.24118) 439 : cluster [DBG] pgmap v570: 1 pgs: 1 active+clean; 0 B data, 710 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:03.991 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:42:03 smithi166 bash[11928]: debug 2020-09-08T15:42:03.867+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:04.108 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:04.212 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:04.344 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:04.365 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:42:04 smithi166 bash[16593]: debug 2020-09-08T15:42:04.107+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:04.449 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:04.570 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:04.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:42:04 smithi055 bash[14301]: debug 2020-09-08T15:42:04.345+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:04.675 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:04.865 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:42:04 smithi166 bash[18960]: debug 2020-09-08T15:42:04.571+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:04.972 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:05.075 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:05.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:42:04 smithi104 bash[22305]: debug 2020-09-08T15:42:04.972+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:05.485 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:05.589 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:05.862 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:42:05 smithi104 bash[19922]: debug 2020-09-08T15:42:05.484+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:05.957 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'unique_pool_1' created 2020-09-08T15:42:05.975 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:06.076 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:06.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:05 smithi055 bash[9296]: cluster 2020-09-08T15:42:03.661345+0000 mgr.x (mgr.24118) 440 : cluster [DBG] pgmap v571: 1 pgs: 1 active+clean; 0 B data, 709 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:06.183 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:05 smithi104 bash[9203]: cluster 2020-09-08T15:42:03.661345+0000 mgr.x (mgr.24118) 440 : cluster [DBG] pgmap v571: 1 pgs: 1 active+clean; 0 B data, 709 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:06.183 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:42:05 smithi104 bash[19922]: debug 2020-09-08T15:42:05.964+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:06.222 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:06.325 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:06.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:05 smithi166 bash[9446]: cluster 2020-09-08T15:42:03.661345+0000 mgr.x (mgr.24118) 440 : cluster [DBG] pgmap v571: 1 pgs: 1 active+clean; 0 B data, 709 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:06.465 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:06.567 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:06.653 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:42:06.658 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:42:06 smithi055 bash[14301]: debug 2020-09-08T15:42:06.221+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:06.660 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:42:06 smithi055 bash[16646]: debug 2020-09-08T15:42:06.453+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:06.684 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:42:06.711 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:42:06.734 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:42:06.765 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:42:06.793 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:42:06.821 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:42:06.854 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:42:06.888 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:42:06.921 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:42:06.948 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:42:06.971 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:06.978 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:42:06.979 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:42:06.979 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:42:06.980 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:42:06.980 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:42:06.980 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:42:06.980 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:42:06.980 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:42:06.981 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:42:06.981 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:42:06.981 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:42:06.981 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:42:06.981 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:42:07.073 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:07.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:06 smithi055 bash[9296]: audit 2020-09-08T15:42:05.453070+0000 mon.a (mon.0) 1115 : audit [INF] from='client.? v1:172.21.15.104:0/146160674' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2020-09-08T15:42:07.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:06 smithi055 bash[9296]: cluster 2020-09-08T15:42:05.661986+0000 mgr.x (mgr.24118) 441 : cluster [DBG] pgmap v572: 1 pgs: 1 active+clean; 0 B data, 709 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:07.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:06 smithi055 bash[9296]: audit 2020-09-08T15:42:05.958833+0000 mon.a (mon.0) 1116 : audit [INF] from='client.? v1:172.21.15.104:0/146160674' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2020-09-08T15:42:07.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:06 smithi055 bash[9296]: cluster 2020-09-08T15:42:05.958927+0000 mon.a (mon.0) 1117 : cluster [DBG] osdmap e214: 12 total, 12 up, 12 in 2020-09-08T15:42:07.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:06 smithi055 bash[9296]: cluster 2020-09-08T15:42:06.536111+0000 osd.7 (osd.7) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:07.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:06 smithi055 bash[9296]: cluster 2020-09-08T15:42:06.536233+0000 osd.7 (osd.7) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:07.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:06 smithi055 bash[9296]: audit 2020-09-08T15:42:06.633039+0000 mon.a (mon.0) 1118 : audit [DBG] from='client.? v1:172.21.15.104:0/1775796129' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:42:07.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:42:06 smithi104 bash[19922]: debug 2020-09-08T15:42:06.964+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:07.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:06 smithi104 bash[9203]: audit 2020-09-08T15:42:05.453070+0000 mon.a (mon.0) 1115 : audit [INF] from='client.? v1:172.21.15.104:0/146160674' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2020-09-08T15:42:07.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:06 smithi104 bash[9203]: cluster 2020-09-08T15:42:05.661986+0000 mgr.x (mgr.24118) 441 : cluster [DBG] pgmap v572: 1 pgs: 1 active+clean; 0 B data, 709 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:07.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:06 smithi104 bash[9203]: audit 2020-09-08T15:42:05.958833+0000 mon.a (mon.0) 1116 : audit [INF] from='client.? v1:172.21.15.104:0/146160674' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2020-09-08T15:42:07.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:06 smithi104 bash[9203]: cluster 2020-09-08T15:42:05.958927+0000 mon.a (mon.0) 1117 : cluster [DBG] osdmap e214: 12 total, 12 up, 12 in 2020-09-08T15:42:07.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:06 smithi104 bash[9203]: cluster 2020-09-08T15:42:06.536111+0000 osd.7 (osd.7) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:07.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:06 smithi104 bash[9203]: cluster 2020-09-08T15:42:06.536233+0000 osd.7 (osd.7) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:07.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:06 smithi104 bash[9203]: audit 2020-09-08T15:42:06.633039+0000 mon.a (mon.0) 1118 : audit [DBG] from='client.? v1:172.21.15.104:0/1775796129' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:42:07.226 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:06 smithi166 bash[9446]: audit 2020-09-08T15:42:05.453070+0000 mon.a (mon.0) 1115 : audit [INF] from='client.? v1:172.21.15.104:0/146160674' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2020-09-08T15:42:07.226 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:06 smithi166 bash[9446]: cluster 2020-09-08T15:42:05.661986+0000 mgr.x (mgr.24118) 441 : cluster [DBG] pgmap v572: 1 pgs: 1 active+clean; 0 B data, 709 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:07.226 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:06 smithi166 bash[9446]: audit 2020-09-08T15:42:05.958833+0000 mon.a (mon.0) 1116 : audit [INF] from='client.? v1:172.21.15.104:0/146160674' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2020-09-08T15:42:07.227 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:06 smithi166 bash[9446]: cluster 2020-09-08T15:42:05.958927+0000 mon.a (mon.0) 1117 : cluster [DBG] osdmap e214: 12 total, 12 up, 12 in 2020-09-08T15:42:07.227 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:06 smithi166 bash[9446]: cluster 2020-09-08T15:42:06.536111+0000 osd.7 (osd.7) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:07.227 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:06 smithi166 bash[9446]: cluster 2020-09-08T15:42:06.536233+0000 osd.7 (osd.7) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:07.227 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:06 smithi166 bash[9446]: audit 2020-09-08T15:42:06.633039+0000 mon.a (mon.0) 1118 : audit [DBG] from='client.? v1:172.21.15.104:0/1775796129' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:42:07.228 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:07.331 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:07.455 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:07.562 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:42:07.622 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:42:07 smithi166 bash[18960]: debug 2020-09-08T15:42:07.227+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:07.623 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:42:07 smithi166 bash[18960]: debug 2020-09-08T15:42:07.459+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:07.706 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool application enable unique_pool_1 rados --yes-i-really-mean-it || true 2020-09-08T15:42:07.893 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:42:07.894 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:42:07.894 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:42:07.966 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:42:08.080 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:08.191 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:08.296 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:42:08.313 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:07 smithi166 bash[9446]: cluster 2020-09-08T15:42:06.966865+0000 mon.a (mon.0) 1119 : cluster [DBG] osdmap e215: 12 total, 12 up, 12 in 2020-09-08T15:42:08.313 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:07 smithi166 bash[9446]: cluster 2020-09-08T15:42:06.994066+0000 osd.0 (osd.0) 25 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:08.313 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:07 smithi166 bash[9446]: cluster 2020-09-08T15:42:06.994154+0000 osd.0 (osd.0) 26 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:08.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:42:08 smithi055 bash[14301]: debug 2020-09-08T15:42:08.193+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:08.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:07 smithi055 bash[9296]: cluster 2020-09-08T15:42:06.966865+0000 mon.a (mon.0) 1119 : cluster [DBG] osdmap e215: 12 total, 12 up, 12 in 2020-09-08T15:42:08.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:07 smithi055 bash[9296]: cluster 2020-09-08T15:42:06.994066+0000 osd.0 (osd.0) 25 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:08.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:07 smithi055 bash[9296]: cluster 2020-09-08T15:42:06.994154+0000 osd.0 (osd.0) 26 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:08.427 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:42:07 smithi104 bash[15273]: debug 2020-09-08T15:42:07.964+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:08.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:07 smithi104 bash[9203]: cluster 2020-09-08T15:42:06.966865+0000 mon.a (mon.0) 1119 : cluster [DBG] osdmap e215: 12 total, 12 up, 12 in 2020-09-08T15:42:08.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:07 smithi104 bash[9203]: cluster 2020-09-08T15:42:06.994066+0000 osd.0 (osd.0) 25 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:08.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:07 smithi104 bash[9203]: cluster 2020-09-08T15:42:06.994154+0000 osd.0 (osd.0) 26 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:08.433 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:42:08.539 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:42:08.618 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:42:08 smithi166 bash[14254]: debug 2020-09-08T15:42:08.435+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:08.667 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:42:08.779 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:08.974 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:42:08 smithi166 bash[14254]: debug 2020-09-08T15:42:08.667+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:09.122 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:08 smithi104 bash[9203]: cluster 2020-09-08T15:42:07.662676+0000 mgr.x (mgr.24118) 442 : cluster [DBG] pgmap v575: 17 pgs: 8 creating+peering, 1 active+clean, 8 unknown; 0 B data, 709 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:09.135 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:09.242 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:09.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:08 smithi166 bash[9446]: cluster 2020-09-08T15:42:07.662676+0000 mgr.x (mgr.24118) 442 : cluster [DBG] pgmap v575: 17 pgs: 8 creating+peering, 1 active+clean, 8 unknown; 0 B data, 709 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:09.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:08 smithi166 bash[9446]: cluster 2020-09-08T15:42:07.975556+0000 mon.a (mon.0) 1120 : cluster [DBG] osdmap e216: 12 total, 12 up, 12 in 2020-09-08T15:42:09.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:08 smithi055 bash[9296]: cluster 2020-09-08T15:42:07.662676+0000 mgr.x (mgr.24118) 442 : cluster [DBG] pgmap v575: 17 pgs: 8 creating+peering, 1 active+clean, 8 unknown; 0 B data, 709 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:09.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:08 smithi055 bash[9296]: cluster 2020-09-08T15:42:07.975556+0000 mon.a (mon.0) 1120 : cluster [DBG] osdmap e216: 12 total, 12 up, 12 in 2020-09-08T15:42:09.427 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:42:09 smithi104 bash[17588]: debug 2020-09-08T15:42:09.136+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:09.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:09 smithi104 bash[9203]: cluster 2020-09-08T15:42:07.975556+0000 mon.a (mon.0) 1120 : cluster [DBG] osdmap e216: 12 total, 12 up, 12 in 2020-09-08T15:42:09.657 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:09.759 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:09.774 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:42:09 smithi104 bash[17588]: debug 2020-09-08T15:42:09.656+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:10.095 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:10.106 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2020-09-08T15:42:10.202 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:10.285 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:42:10.350 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:10.430 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:42:10 smithi104 bash[19922]: debug 2020-09-08T15:42:10.096+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:10.455 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:10.572 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:10.615 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:42:10 smithi166 bash[11928]: debug 2020-09-08T15:42:10.351+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:10.677 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:10.783 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:42:10 smithi055 bash[11956]: debug 2020-09-08T15:42:10.573+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:10.783 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:10.891 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:42:11.002 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:42:11.112 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:11.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:42:10 smithi055 bash[16646]: debug 2020-09-08T15:42:10.785+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:11.243 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:42:11 smithi166 bash[14254]: debug 2020-09-08T15:42:11.003+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:11.243 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:11.351 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:11.467 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:42:11.487 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:11.596 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:11.612 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:42:11.615 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:42:11 smithi166 bash[11928]: debug 2020-09-08T15:42:11.243+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:11.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:42:11 smithi166 bash[18960]: debug 2020-09-08T15:42:11.487+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:11.723 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:11.824 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:11.837 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:11 smithi104 bash[9203]: cluster 2020-09-08T15:42:09.663355+0000 mgr.x (mgr.24118) 443 : cluster [DBG] pgmap v577: 17 pgs: 8 creating+peering, 6 active+clean, 3 unknown; 0 B data, 709 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:11.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:11 smithi055 bash[9296]: cluster 2020-09-08T15:42:09.663355+0000 mgr.x (mgr.24118) 443 : cluster [DBG] pgmap v577: 17 pgs: 8 creating+peering, 6 active+clean, 3 unknown; 0 B data, 709 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:11.908 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:42:11 smithi055 bash[11956]: debug 2020-09-08T15:42:11.721+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:12.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:11 smithi166 bash[9446]: cluster 2020-09-08T15:42:09.663355+0000 mgr.x (mgr.24118) 443 : cluster [DBG] pgmap v577: 17 pgs: 8 creating+peering, 6 active+clean, 3 unknown; 0 B data, 709 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:12.220 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:12.331 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:12.460 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:12.566 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:42:12.615 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:42:12 smithi166 bash[18960]: debug 2020-09-08T15:42:12.463+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:12.642 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:42:12 smithi104 bash[17588]: debug 2020-09-08T15:42:12.220+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:12.648 INFO:teuthology.orchestra.run.smithi104.stderr:marked out osd.1. 2020-09-08T15:42:12.696 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:42:12.802 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:42:12.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:12 smithi055 bash[9296]: audit 2020-09-08T15:42:12.528991+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? v1:172.21.15.104:0/218413222' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["1"]}]: dispatch 2020-09-08T15:42:12.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:12 smithi055 bash[9296]: cluster 2020-09-08T15:42:12.529166+0000 mon.a (mon.0) 1122 : cluster [INF] Client client.admin marked osd.1 out, while it was still marked up 2020-09-08T15:42:13.014 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:12 smithi104 bash[9203]: debug 2020-09-08T15:42:12.524+0000 7efd04a72700 -1 mon.a@0(leader).osd e216 definitely_dead 0 2020-09-08T15:42:13.015 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:12 smithi104 bash[9203]: audit 2020-09-08T15:42:12.528991+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? v1:172.21.15.104:0/218413222' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["1"]}]: dispatch 2020-09-08T15:42:13.015 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:12 smithi104 bash[9203]: cluster 2020-09-08T15:42:12.529166+0000 mon.a (mon.0) 1122 : cluster [INF] Client client.admin marked osd.1 out, while it was still marked up 2020-09-08T15:42:13.115 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:42:12 smithi166 bash[14254]: debug 2020-09-08T15:42:12.699+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:13.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:12 smithi166 bash[9446]: audit 2020-09-08T15:42:12.528991+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? v1:172.21.15.104:0/218413222' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["1"]}]: dispatch 2020-09-08T15:42:13.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:12 smithi166 bash[9446]: cluster 2020-09-08T15:42:12.529166+0000 mon.a (mon.0) 1122 : cluster [INF] Client client.admin marked osd.1 out, while it was still marked up 2020-09-08T15:42:13.170 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:42:13.273 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:13.408 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:13.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:42:13 smithi104 bash[15273]: debug 2020-09-08T15:42:13.160+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:13.514 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:13.648 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:42:13.655 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:42:13 smithi055 bash[16646]: debug 2020-09-08T15:42:13.409+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:13.889 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:13.889 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:13 smithi104 bash[9203]: cluster 2020-09-08T15:42:11.663907+0000 mgr.x (mgr.24118) 444 : cluster [DBG] pgmap v578: 17 pgs: 8 creating+peering, 9 active+clean; 0 B data, 709 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:13.889 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:13 smithi104 bash[9203]: audit 2020-09-08T15:42:12.649054+0000 mon.a (mon.0) 1123 : audit [INF] from='client.? v1:172.21.15.104:0/218413222' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["1"]}]': finished 2020-09-08T15:42:13.890 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:13 smithi104 bash[9203]: cluster 2020-09-08T15:42:12.649128+0000 mon.a (mon.0) 1124 : cluster [DBG] osdmap e217: 12 total, 12 up, 11 in 2020-09-08T15:42:13.890 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:13 smithi104 bash[9203]: audit 2020-09-08T15:42:12.906708+0000 mon.a (mon.0) 1125 : audit [INF] from='client.? v1:172.21.15.104:0/647591925' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:42:13.890 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:13 smithi104 bash[9203]: cluster 2020-09-08T15:42:13.282970+0000 osd.5 (osd.5) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:13.890 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:13 smithi104 bash[9203]: cluster 2020-09-08T15:42:13.283094+0000 osd.5 (osd.5) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:13.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:13 smithi055 bash[9296]: cluster 2020-09-08T15:42:11.663907+0000 mgr.x (mgr.24118) 444 : cluster [DBG] pgmap v578: 17 pgs: 8 creating+peering, 9 active+clean; 0 B data, 709 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:13.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:13 smithi055 bash[9296]: audit 2020-09-08T15:42:12.649054+0000 mon.a (mon.0) 1123 : audit [INF] from='client.? v1:172.21.15.104:0/218413222' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["1"]}]': finished 2020-09-08T15:42:13.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:13 smithi055 bash[9296]: cluster 2020-09-08T15:42:12.649128+0000 mon.a (mon.0) 1124 : cluster [DBG] osdmap e217: 12 total, 12 up, 11 in 2020-09-08T15:42:13.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:13 smithi055 bash[9296]: audit 2020-09-08T15:42:12.906708+0000 mon.a (mon.0) 1125 : audit [INF] from='client.? v1:172.21.15.104:0/647591925' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:42:13.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:13 smithi055 bash[9296]: cluster 2020-09-08T15:42:13.282970+0000 osd.5 (osd.5) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:13.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:13 smithi055 bash[9296]: cluster 2020-09-08T15:42:13.283094+0000 osd.5 (osd.5) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:13.992 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:14.120 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:14.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:13 smithi166 bash[9446]: cluster 2020-09-08T15:42:11.663907+0000 mgr.x (mgr.24118) 444 : cluster [DBG] pgmap v578: 17 pgs: 8 creating+peering, 9 active+clean; 0 B data, 709 MiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:42:14.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:13 smithi166 bash[9446]: audit 2020-09-08T15:42:12.649054+0000 mon.a (mon.0) 1123 : audit [INF] from='client.? v1:172.21.15.104:0/218413222' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["1"]}]': finished 2020-09-08T15:42:14.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:13 smithi166 bash[9446]: cluster 2020-09-08T15:42:12.649128+0000 mon.a (mon.0) 1124 : cluster [DBG] osdmap e217: 12 total, 12 up, 11 in 2020-09-08T15:42:14.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:13 smithi166 bash[9446]: audit 2020-09-08T15:42:12.906708+0000 mon.a (mon.0) 1125 : audit [INF] from='client.? v1:172.21.15.104:0/647591925' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:42:14.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:13 smithi166 bash[9446]: cluster 2020-09-08T15:42:13.282970+0000 osd.5 (osd.5) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:14.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:13 smithi166 bash[9446]: cluster 2020-09-08T15:42:13.283094+0000 osd.5 (osd.5) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:14.178 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:42:13 smithi104 bash[17588]: debug 2020-09-08T15:42:13.888+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:14.230 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:14.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:42:14 smithi055 bash[14301]: debug 2020-09-08T15:42:14.121+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:14.613 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:14.715 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:14.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:14 smithi104 bash[9203]: cluster 2020-09-08T15:42:13.644868+0000 mon.a (mon.0) 1126 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:42:14.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:14 smithi104 bash[9203]: audit 2020-09-08T15:42:13.650479+0000 mon.a (mon.0) 1127 : audit [INF] from='client.? v1:172.21.15.104:0/647591925' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:42:14.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:14 smithi104 bash[9203]: cluster 2020-09-08T15:42:13.650530+0000 mon.a (mon.0) 1128 : cluster [DBG] osdmap e218: 12 total, 12 up, 11 in 2020-09-08T15:42:14.929 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:42:14 smithi104 bash[19922]: debug 2020-09-08T15:42:14.608+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:15.124 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:15.125 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:14 smithi166 bash[9446]: cluster 2020-09-08T15:42:13.644868+0000 mon.a (mon.0) 1126 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:42:15.125 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:14 smithi166 bash[9446]: audit 2020-09-08T15:42:13.650479+0000 mon.a (mon.0) 1127 : audit [INF] from='client.? v1:172.21.15.104:0/647591925' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:42:15.126 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:14 smithi166 bash[9446]: cluster 2020-09-08T15:42:13.650530+0000 mon.a (mon.0) 1128 : cluster [DBG] osdmap e218: 12 total, 12 up, 11 in 2020-09-08T15:42:15.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:14 smithi055 bash[9296]: cluster 2020-09-08T15:42:13.644868+0000 mon.a (mon.0) 1126 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:42:15.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:14 smithi055 bash[9296]: audit 2020-09-08T15:42:13.650479+0000 mon.a (mon.0) 1127 : audit [INF] from='client.? v1:172.21.15.104:0/647591925' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:42:15.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:14 smithi055 bash[9296]: cluster 2020-09-08T15:42:13.650530+0000 mon.a (mon.0) 1128 : cluster [DBG] osdmap e218: 12 total, 12 up, 11 in 2020-09-08T15:42:15.228 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:15.402 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:15.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:42:15 smithi104 bash[19922]: debug 2020-09-08T15:42:15.120+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:15.512 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:15.618 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:42:15 smithi166 bash[16593]: debug 2020-09-08T15:42:15.403+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:15.736 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:15 smithi104 bash[9203]: cluster 2020-09-08T15:42:13.664503+0000 mgr.x (mgr.24118) 445 : cluster [DBG] pgmap v581: 17 pgs: 2 peering, 15 active+clean; 0 B data, 625 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:42:15.736 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:15.843 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:15.969 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:15 smithi166 bash[9446]: cluster 2020-09-08T15:42:13.664503+0000 mgr.x (mgr.24118) 445 : cluster [DBG] pgmap v581: 17 pgs: 2 peering, 15 active+clean; 0 B data, 625 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:42:15.970 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:15.988 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:42:15 smithi104 bash[19922]: debug 2020-09-08T15:42:15.736+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:16.076 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:16.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:15 smithi055 bash[9296]: cluster 2020-09-08T15:42:13.664503+0000 mgr.x (mgr.24118) 445 : cluster [DBG] pgmap v581: 17 pgs: 2 peering, 15 active+clean; 0 B data, 625 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:42:16.205 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:16.307 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:16.365 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:42:15 smithi166 bash[11928]: debug 2020-09-08T15:42:15.971+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:16.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:42:16 smithi166 bash[11928]: debug 2020-09-08T15:42:16.199+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:16.462 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:16.568 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:16.582 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:42:16 smithi055 bash[11956]: debug 2020-09-08T15:42:16.461+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:16.690 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:16.793 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:16.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:16 smithi055 bash[9296]: cluster 2020-09-08T15:42:15.665196+0000 mgr.x (mgr.24118) 446 : cluster [DBG] pgmap v582: 17 pgs: 2 peering, 15 active+clean; 0 B data, 625 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:42:16.911 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:42:16 smithi055 bash[14301]: debug 2020-09-08T15:42:16.689+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:16.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:16 smithi104 bash[9203]: cluster 2020-09-08T15:42:15.665196+0000 mgr.x (mgr.24118) 446 : cluster [DBG] pgmap v582: 17 pgs: 2 peering, 15 active+clean; 0 B data, 625 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:42:17.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:16 smithi166 bash[9446]: cluster 2020-09-08T15:42:15.665196+0000 mgr.x (mgr.24118) 446 : cluster [DBG] pgmap v582: 17 pgs: 2 peering, 15 active+clean; 0 B data, 625 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:42:17.149 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:17.259 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:17.402 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:17.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:42:17 smithi104 bash[17588]: debug 2020-09-08T15:42:17.148+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:17.509 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:17.531 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:42:17 smithi055 bash[14301]: debug 2020-09-08T15:42:17.405+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:17.643 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:17.699 INFO:teuthology.orchestra.run.smithi104.stderr:enabled application 'rados' on pool 'unique_pool_1' 2020-09-08T15:42:17.750 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:17.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:17 smithi055 bash[9296]: audit 2020-09-08T15:42:17.590951+0000 mon.b (mon.1) 83 : audit [INF] from='client.? v1:172.21.15.104:0/1467930059' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T15:42:17.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:17 smithi055 bash[9296]: audit 2020-09-08T15:42:17.591813+0000 mon.a (mon.0) 1129 : audit [INF] from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T15:42:17.908 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:42:17 smithi055 bash[19003]: debug 2020-09-08T15:42:17.645+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:17.958 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:17 smithi104 bash[9203]: audit 2020-09-08T15:42:17.590951+0000 mon.b (mon.1) 83 : audit [INF] from='client.? v1:172.21.15.104:0/1467930059' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T15:42:17.959 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:17 smithi104 bash[9203]: audit 2020-09-08T15:42:17.591813+0000 mon.a (mon.0) 1129 : audit [INF] from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T15:42:17.959 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:17.991 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:42:17.994 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:42:17.994 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:42:18.070 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:18.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:17 smithi166 bash[9446]: audit 2020-09-08T15:42:17.590951+0000 mon.b (mon.1) 83 : audit [INF] from='client.? v1:172.21.15.104:0/1467930059' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T15:42:18.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:17 smithi166 bash[9446]: audit 2020-09-08T15:42:17.591813+0000 mon.a (mon.0) 1129 : audit [INF] from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T15:42:18.122 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:42:18.230 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:42:17 smithi104 bash[19922]: debug 2020-09-08T15:42:17.956+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:18.248 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:18.354 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:18.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:42:18 smithi055 bash[14301]: debug 2020-09-08T15:42:18.249+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:18.502 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:18.605 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:18.689 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:42:18 smithi166 bash[18960]: debug 2020-09-08T15:42:18.499+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:18.833 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:18.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:18 smithi104 bash[9203]: cluster 2020-09-08T15:42:17.665985+0000 mgr.x (mgr.24118) 447 : cluster [DBG] pgmap v583: 17 pgs: 2 peering, 15 active+clean; 0 B data, 625 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:42:18.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:18 smithi104 bash[9203]: audit 2020-09-08T15:42:17.695137+0000 mon.a (mon.0) 1130 : audit [INF] from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T15:42:18.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:18 smithi104 bash[9203]: cluster 2020-09-08T15:42:17.695209+0000 mon.a (mon.0) 1131 : cluster [DBG] osdmap e219: 12 total, 12 up, 11 in 2020-09-08T15:42:18.927 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:42:18 smithi104 bash[17588]: debug 2020-09-08T15:42:18.836+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:18.945 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:19.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:18 smithi166 bash[9446]: cluster 2020-09-08T15:42:17.665985+0000 mgr.x (mgr.24118) 447 : cluster [DBG] pgmap v583: 17 pgs: 2 peering, 15 active+clean; 0 B data, 625 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:42:19.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:18 smithi166 bash[9446]: audit 2020-09-08T15:42:17.695137+0000 mon.a (mon.0) 1130 : audit [INF] from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T15:42:19.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:18 smithi166 bash[9446]: cluster 2020-09-08T15:42:17.695209+0000 mon.a (mon.0) 1131 : cluster [DBG] osdmap e219: 12 total, 12 up, 11 in 2020-09-08T15:42:19.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:18 smithi055 bash[9296]: cluster 2020-09-08T15:42:17.665985+0000 mgr.x (mgr.24118) 447 : cluster [DBG] pgmap v583: 17 pgs: 2 peering, 15 active+clean; 0 B data, 625 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:42:19.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:18 smithi055 bash[9296]: audit 2020-09-08T15:42:17.695137+0000 mon.a (mon.0) 1130 : audit [INF] from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T15:42:19.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:18 smithi055 bash[9296]: cluster 2020-09-08T15:42:17.695209+0000 mon.a (mon.0) 1131 : cluster [DBG] osdmap e219: 12 total, 12 up, 11 in 2020-09-08T15:42:19.307 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:19.378 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2020-09-08T15:42:19.379 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-09-08T15:42:19.379 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd dump -f json-pretty 2020-09-08T15:42:19.411 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:19.520 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:42:19.557 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:19.653 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:42:19 smithi104 bash[22305]: debug 2020-09-08T15:42:19.308+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:19.654 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2020-09-08T15:42:19.668 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:19.688 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:42:19.698 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:42:19 smithi166 bash[16593]: debug 2020-09-08T15:42:19.555+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:19.713 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:42:19.744 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:42:19.774 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:42:19.798 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:42:19.799 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:42:19.830 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:42:19.855 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:42:19.882 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:42:19.906 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:42:19.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:19 smithi104 bash[9203]: cluster 2020-09-08T15:42:18.419403+0000 osd.10 (osd.10) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:19.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:19 smithi104 bash[9203]: cluster 2020-09-08T15:42:18.419486+0000 osd.10 (osd.10) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:19.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:19 smithi104 bash[9203]: cluster 2020-09-08T15:42:18.703319+0000 mon.a (mon.0) 1132 : cluster [DBG] osdmap e220: 12 total, 12 up, 11 in 2020-09-08T15:42:19.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:19 smithi104 bash[9203]: audit 2020-09-08T15:42:19.669323+0000 mon.a (mon.0) 1133 : audit [DBG] from='client.? v1:172.21.15.104:0/3216819043' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:42:19.941 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:42:19.977 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:42:20.000 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:42:20.002 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:42:20.002 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:42:20.002 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:42:20.003 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:42:20.003 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:42:20.003 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:42:20.003 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:42:20.003 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:42:20.004 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:42:20.004 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:42:20.004 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:42:20.004 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:42:20.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:19 smithi166 bash[9446]: cluster 2020-09-08T15:42:18.419403+0000 osd.10 (osd.10) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:20.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:19 smithi166 bash[9446]: cluster 2020-09-08T15:42:18.419486+0000 osd.10 (osd.10) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:20.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:19 smithi166 bash[9446]: cluster 2020-09-08T15:42:18.703319+0000 mon.a (mon.0) 1132 : cluster [DBG] osdmap e220: 12 total, 12 up, 11 in 2020-09-08T15:42:20.123 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:19 smithi166 bash[9446]: audit 2020-09-08T15:42:19.669323+0000 mon.a (mon.0) 1133 : audit [DBG] from='client.? v1:172.21.15.104:0/3216819043' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:42:20.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:19 smithi055 bash[9296]: cluster 2020-09-08T15:42:18.419403+0000 osd.10 (osd.10) 17 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:20.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:19 smithi055 bash[9296]: cluster 2020-09-08T15:42:18.419486+0000 osd.10 (osd.10) 18 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:20.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:19 smithi055 bash[9296]: cluster 2020-09-08T15:42:18.703319+0000 mon.a (mon.0) 1132 : cluster [DBG] osdmap e220: 12 total, 12 up, 11 in 2020-09-08T15:42:20.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:19 smithi055 bash[9296]: audit 2020-09-08T15:42:19.669323+0000 mon.a (mon.0) 1133 : audit [DBG] from='client.? v1:172.21.15.104:0/3216819043' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:42:20.168 INFO:teuthology.orchestra.run.smithi166:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.2 -b 65536 --object-size 65536 -p unique_pool_1 bench 90 write' 2020-09-08T15:42:20.171 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2020-09-08T15:42:20.171 INFO:teuthology.orchestra.run:waiting for 3000 2020-09-08T15:42:20.212 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:hints = 1 2020-09-08T15:42:20.213 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2020-09-08T15:42:20.213 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Object prefix: benchmark_data_smithi166_26097 2020-09-08T15:42:20.213 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:42:20.214 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 0 11 11 0 0 0 - 0 2020-09-08T15:42:20.278 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:20.389 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:20.428 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:42:20 smithi104 bash[17588]: debug 2020-09-08T15:42:20.280+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:20.500 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:20.604 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:20.657 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:42:20 smithi055 bash[11956]: debug 2020-09-08T15:42:20.501+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:20.726 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:20.831 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:20.865 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:42:20 smithi166 bash[16593]: debug 2020-09-08T15:42:20.727+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:21.217 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 1 16 3997 3981 248.63 248.812 0.00376757 0.00401216 2020-09-08T15:42:21.309 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:21.422 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:21.528 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:21.631 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:21.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:21 smithi166 bash[9446]: cluster 2020-09-08T15:42:19.666641+0000 mgr.x (mgr.24118) 448 : cluster [DBG] pgmap v586: 17 pgs: 17 active+clean; 0 B data, 625 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:42:21.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:42:21 smithi166 bash[16593]: debug 2020-09-08T15:42:21.531+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:21.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:21 smithi055 bash[9296]: cluster 2020-09-08T15:42:19.666641+0000 mgr.x (mgr.24118) 448 : cluster [DBG] pgmap v586: 17 pgs: 17 active+clean; 0 B data, 625 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:42:22.214 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 2 15 7280 7265 226.935 205.25 0.0048247 0.00440001 2020-09-08T15:42:22.401 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:42:21 smithi104 bash[22305]: debug 2020-09-08T15:42:21.312+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:22.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:22 smithi104 bash[9203]: cluster 2020-09-08T15:42:19.666641+0000 mgr.x (mgr.24118) 448 : cluster [DBG] pgmap v586: 17 pgs: 17 active+clean; 0 B data, 625 MiB used, 983 GiB / 983 GiB avail 2020-09-08T15:42:22.727 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:22.830 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:22.967 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:23.071 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:23.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:42:22 smithi166 bash[16593]: debug 2020-09-08T15:42:22.967+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:23.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:42:22 smithi104 bash[19922]: debug 2020-09-08T15:42:22.732+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:23.198 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:23.214 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 3 16 10347 10331 215.162 191.625 0.0025221 0.00464316 2020-09-08T15:42:23.308 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:23.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:42:23 smithi055 bash[19003]: debug 2020-09-08T15:42:23.201+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:23.783 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:42:23.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:23 smithi055 bash[9296]: cluster 2020-09-08T15:42:21.667110+0000 mgr.x (mgr.24118) 449 : cluster [DBG] pgmap v587: 17 pgs: 17 active+clean; 22 MiB data, 626 MiB used, 983 GiB / 983 GiB avail; 2.8 MiB/s wr, 44 op/s 2020-09-08T15:42:23.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:23 smithi055 bash[9296]: cluster 2020-09-08T15:42:22.114154+0000 osd.3 (osd.3) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:23.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:23 smithi055 bash[9296]: cluster 2020-09-08T15:42:22.114227+0000 osd.3 (osd.3) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:23.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:23 smithi055 bash[9296]: cluster 2020-09-08T15:42:22.638820+0000 mon.a (mon.0) 1134 : cluster [DBG] osdmap e221: 12 total, 12 up, 11 in 2020-09-08T15:42:23.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:23 smithi104 bash[9203]: cluster 2020-09-08T15:42:21.667110+0000 mgr.x (mgr.24118) 449 : cluster [DBG] pgmap v587: 17 pgs: 17 active+clean; 22 MiB data, 626 MiB used, 983 GiB / 983 GiB avail; 2.8 MiB/s wr, 44 op/s 2020-09-08T15:42:23.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:23 smithi104 bash[9203]: cluster 2020-09-08T15:42:22.114154+0000 osd.3 (osd.3) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:23.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:23 smithi104 bash[9203]: cluster 2020-09-08T15:42:22.114227+0000 osd.3 (osd.3) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:23.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:23 smithi104 bash[9203]: cluster 2020-09-08T15:42:22.638820+0000 mon.a (mon.0) 1134 : cluster [DBG] osdmap e221: 12 total, 12 up, 11 in 2020-09-08T15:42:23.951 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:42:24.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:23 smithi166 bash[9446]: cluster 2020-09-08T15:42:21.667110+0000 mgr.x (mgr.24118) 449 : cluster [DBG] pgmap v587: 17 pgs: 17 active+clean; 22 MiB data, 626 MiB used, 983 GiB / 983 GiB avail; 2.8 MiB/s wr, 44 op/s 2020-09-08T15:42:24.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:23 smithi166 bash[9446]: cluster 2020-09-08T15:42:22.114154+0000 osd.3 (osd.3) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:24.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:23 smithi166 bash[9446]: cluster 2020-09-08T15:42:22.114227+0000 osd.3 (osd.3) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:24.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:23 smithi166 bash[9446]: cluster 2020-09-08T15:42:22.638820+0000 mon.a (mon.0) 1134 : cluster [DBG] osdmap e221: 12 total, 12 up, 11 in 2020-09-08T15:42:24.214 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 4 16 13862 13846 216.288 219.688 0.00236654 0.00461923 2020-09-08T15:42:24.438 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:24.546 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:42:24.682 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:42:24.793 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:24.805 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:42:24 smithi104 bash[19922]: debug 2020-09-08T15:42:24.440+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:24.865 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:42:24 smithi166 bash[14254]: debug 2020-09-08T15:42:24.683+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:25.214 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 5 16 16135 16119 201.443 142.062 0.00242003 0.00492489 2020-09-08T15:42:25.254 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:25.357 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:25.506 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:25.610 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:25.637 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:42:25 smithi104 bash[22305]: debug 2020-09-08T15:42:25.256+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:25.638 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:42:25 smithi166 bash[11928]: debug 2020-09-08T15:42:25.507+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:25.735 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:25.838 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:25.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:42:25 smithi055 bash[16646]: debug 2020-09-08T15:42:25.737+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:25.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:25 smithi055 bash[9296]: cluster 2020-09-08T15:42:23.667556+0000 mgr.x (mgr.24118) 450 : cluster [DBG] pgmap v589: 17 pgs: 17 active+clean; 310 MiB data, 1.0 GiB used, 982 GiB / 983 GiB avail; 52 MiB/s wr, 827 op/s 2020-09-08T15:42:25.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:25 smithi055 bash[9296]: audit 2020-09-08T15:42:24.760079+0000 mon.a (mon.0) 1135 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}, {\"id\": \"448d5c57-f6de-443a-ab88-169d81b78450\", \"message\": \"Rebalancing after osd.11 marked in\", \"refs\": [[\"osd\", 11]], \"started_at\": 1599579281.6064675, \"finished_at\": 1599579283.5990696}, {\"id\": \"4bd2c04f-3aed-4914-b08b-6531f7786cca\", \"message\": \"Rebalancing after osd.1 marked out\", \"refs\": [[\"osd\", 1]], \"started_at\": 1599579732.6504169, \"finished_at\": 1599579743.653669}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:42:25.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:25 smithi055 bash[9296]: audit 2020-09-08T15:42:24.764414+0000 mon.a (mon.0) 1136 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}, {\"id\": \"448d5c57-f6de-443a-ab88-169d81b78450\", \"message\": \"Rebalancing after osd.11 marked in\", \"refs\": [[\"osd\", 11]], \"started_at\": 1599579281.6064675, \"finished_at\": 1599579283.5990696}, {\"id\": \"4bd2c04f-3aed-4914-b08b-6531f7786cca\", \"message\": \"Rebalancing after osd.1 marked out\", \"refs\": [[\"osd\", 1]], \"started_at\": 1599579732.6504169, \"finished_at\": 1599579743.653669}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:42:25.986 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:25 smithi104 bash[9203]: cluster 2020-09-08T15:42:23.667556+0000 mgr.x (mgr.24118) 450 : cluster [DBG] pgmap v589: 17 pgs: 17 active+clean; 310 MiB data, 1.0 GiB used, 982 GiB / 983 GiB avail; 52 MiB/s wr, 827 op/s 2020-09-08T15:42:25.986 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:25 smithi104 bash[9203]: audit 2020-09-08T15:42:24.760079+0000 mon.a (mon.0) 1135 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}, {\"id\": \"448d5c57-f6de-443a-ab88-169d81b78450\", \"message\": \"Rebalancing after osd.11 marked in\", \"refs\": [[\"osd\", 11]], \"started_at\": 1599579281.6064675, \"finished_at\": 1599579283.5990696}, {\"id\": \"4bd2c04f-3aed-4914-b08b-6531f7786cca\", \"message\": \"Rebalancing after osd.1 marked out\", \"refs\": [[\"osd\", 1]], \"started_at\": 1599579732.6504169, \"finished_at\": 1599579743.653669}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:42:25.987 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:25 smithi104 bash[9203]: audit 2020-09-08T15:42:24.764414+0000 mon.a (mon.0) 1136 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}, {\"id\": \"448d5c57-f6de-443a-ab88-169d81b78450\", \"message\": \"Rebalancing after osd.11 marked in\", \"refs\": [[\"osd\", 11]], \"started_at\": 1599579281.6064675, \"finished_at\": 1599579283.5990696}, {\"id\": \"4bd2c04f-3aed-4914-b08b-6531f7786cca\", \"message\": \"Rebalancing after osd.1 marked out\", \"refs\": [[\"osd\", 1]], \"started_at\": 1599579732.6504169, \"finished_at\": 1599579743.653669}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:42:26.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:25 smithi166 bash[9446]: cluster 2020-09-08T15:42:23.667556+0000 mgr.x (mgr.24118) 450 : cluster [DBG] pgmap v589: 17 pgs: 17 active+clean; 310 MiB data, 1.0 GiB used, 982 GiB / 983 GiB avail; 52 MiB/s wr, 827 op/s 2020-09-08T15:42:26.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:25 smithi166 bash[9446]: audit 2020-09-08T15:42:24.760079+0000 mon.a (mon.0) 1135 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}, {\"id\": \"448d5c57-f6de-443a-ab88-169d81b78450\", \"message\": \"Rebalancing after osd.11 marked in\", \"refs\": [[\"osd\", 11]], \"started_at\": 1599579281.6064675, \"finished_at\": 1599579283.5990696}, {\"id\": \"4bd2c04f-3aed-4914-b08b-6531f7786cca\", \"message\": \"Rebalancing after osd.1 marked out\", \"refs\": [[\"osd\", 1]], \"started_at\": 1599579732.6504169, \"finished_at\": 1599579743.653669}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:42:26.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:25 smithi166 bash[9446]: audit 2020-09-08T15:42:24.764414+0000 mon.a (mon.0) 1136 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}, {\"id\": \"448d5c57-f6de-443a-ab88-169d81b78450\", \"message\": \"Rebalancing after osd.11 marked in\", \"refs\": [[\"osd\", 11]], \"started_at\": 1599579281.6064675, \"finished_at\": 1599579283.5990696}, {\"id\": \"4bd2c04f-3aed-4914-b08b-6531f7786cca\", \"message\": \"Rebalancing after osd.1 marked out\", \"refs\": [[\"osd\", 1]], \"started_at\": 1599579732.6504169, \"finished_at\": 1599579743.653669}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:42:26.142 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:26.214 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 6 16 18941 18925 197.096 175.375 0.00347454 0.00506863 2020-09-08T15:42:26.245 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:26.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:42:26 smithi104 bash[17588]: debug 2020-09-08T15:42:26.144+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:27.214 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 7 16 22272 22256 198.678 208.188 0.00266948 0.00502988 2020-09-08T15:42:27.827 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:27.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:27 smithi055 bash[9296]: cluster 2020-09-08T15:42:25.668055+0000 mgr.x (mgr.24118) 451 : cluster [DBG] pgmap v590: 17 pgs: 17 active+clean; 407 MiB data, 1.4 GiB used, 982 GiB / 983 GiB avail; 51 MiB/s wr, 815 op/s 2020-09-08T15:42:27.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:42:27 smithi104 bash[17588]: debug 2020-09-08T15:42:27.828+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:27.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:27 smithi104 bash[9203]: cluster 2020-09-08T15:42:25.668055+0000 mgr.x (mgr.24118) 451 : cluster [DBG] pgmap v590: 17 pgs: 17 active+clean; 407 MiB data, 1.4 GiB used, 982 GiB / 983 GiB avail; 51 MiB/s wr, 815 op/s 2020-09-08T15:42:27.933 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:27.942 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:27 smithi166 bash[9446]: cluster 2020-09-08T15:42:25.668055+0000 mgr.x (mgr.24118) 451 : cluster [DBG] pgmap v590: 17 pgs: 17 active+clean; 407 MiB data, 1.4 GiB used, 982 GiB / 983 GiB avail; 51 MiB/s wr, 815 op/s 2020-09-08T15:42:28.072 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:28.175 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:28.214 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 8 16 25391 25375 198.208 194.938 0.00716336 0.00504231 2020-09-08T15:42:28.365 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:42:28 smithi166 bash[16593]: debug 2020-09-08T15:42:28.074+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:28.446 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:42:28 smithi104 bash[19922]: debug 2020-09-08T15:42:28.444+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:28.447 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:28.554 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:29.214 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 9 16 27775 27759 192.74 149 0.00263432 0.00518469 2020-09-08T15:42:29.261 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:29.364 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:29.502 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:29.606 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:29.621 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:42:29 smithi055 bash[11956]: debug 2020-09-08T15:42:29.264+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:29.649 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:42:29 smithi166 bash[16593]: debug 2020-09-08T15:42:29.502+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:29.665 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:42:29.720 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:29.829 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:29.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:29 smithi055 bash[9296]: cluster 2020-09-08T15:42:27.668552+0000 mgr.x (mgr.24118) 452 : cluster [DBG] pgmap v591: 17 pgs: 17 active+clean; 1004 MiB data, 2.3 GiB used, 981 GiB / 983 GiB avail; 112 MiB/s wr, 1.79k op/s 2020-09-08T15:42:29.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:29 smithi055 bash[9296]: cluster 2020-09-08T15:42:28.387121+0000 osd.8 (osd.8) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:29.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:29 smithi055 bash[9296]: cluster 2020-09-08T15:42:28.387213+0000 osd.8 (osd.8) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:29.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:29 smithi055 bash[9296]: cluster 2020-09-08T15:42:28.655119+0000 mon.a (mon.0) 1137 : cluster [DBG] osdmap e222: 12 total, 12 up, 11 in 2020-09-08T15:42:29.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:29 smithi055 bash[9296]: audit 2020-09-08T15:42:28.811424+0000 mon.a (mon.0) 1138 : audit [INF] from='client.? v1:172.21.15.104:0/2326522733' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:42:29.909 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:42:29 smithi055 bash[14301]: debug 2020-09-08T15:42:29.720+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:29.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:29 smithi104 bash[9203]: cluster 2020-09-08T15:42:27.668552+0000 mgr.x (mgr.24118) 452 : cluster [DBG] pgmap v591: 17 pgs: 17 active+clean; 1004 MiB data, 2.3 GiB used, 981 GiB / 983 GiB avail; 112 MiB/s wr, 1.79k op/s 2020-09-08T15:42:29.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:29 smithi104 bash[9203]: cluster 2020-09-08T15:42:28.387121+0000 osd.8 (osd.8) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:29.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:29 smithi104 bash[9203]: cluster 2020-09-08T15:42:28.387213+0000 osd.8 (osd.8) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:29.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:29 smithi104 bash[9203]: cluster 2020-09-08T15:42:28.655119+0000 mon.a (mon.0) 1137 : cluster [DBG] osdmap e222: 12 total, 12 up, 11 in 2020-09-08T15:42:29.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:29 smithi104 bash[9203]: audit 2020-09-08T15:42:28.811424+0000 mon.a (mon.0) 1138 : audit [INF] from='client.? v1:172.21.15.104:0/2326522733' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:42:30.097 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:30.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:29 smithi166 bash[9446]: cluster 2020-09-08T15:42:27.668552+0000 mgr.x (mgr.24118) 452 : cluster [DBG] pgmap v591: 17 pgs: 17 active+clean; 1004 MiB data, 2.3 GiB used, 981 GiB / 983 GiB avail; 112 MiB/s wr, 1.79k op/s 2020-09-08T15:42:30.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:29 smithi166 bash[9446]: cluster 2020-09-08T15:42:28.387121+0000 osd.8 (osd.8) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:30.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:29 smithi166 bash[9446]: cluster 2020-09-08T15:42:28.387213+0000 osd.8 (osd.8) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:30.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:29 smithi166 bash[9446]: cluster 2020-09-08T15:42:28.655119+0000 mon.a (mon.0) 1137 : cluster [DBG] osdmap e222: 12 total, 12 up, 11 in 2020-09-08T15:42:30.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:29 smithi166 bash[9446]: audit 2020-09-08T15:42:28.811424+0000 mon.a (mon.0) 1138 : audit [INF] from='client.? v1:172.21.15.104:0/2326522733' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:42:30.202 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:30.214 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 10 16 30029 30013 187.552 140.875 0.00345216 0.00532766 2020-09-08T15:42:30.427 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:42:30 smithi104 bash[22305]: debug 2020-09-08T15:42:30.100+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:30.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:30 smithi055 bash[9296]: cluster 2020-09-08T15:42:29.653289+0000 mon.a (mon.0) 1139 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:42:30.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:30 smithi055 bash[9296]: audit 2020-09-08T15:42:29.665537+0000 mon.a (mon.0) 1140 : audit [INF] from='client.? v1:172.21.15.104:0/2326522733' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:42:30.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:30 smithi055 bash[9296]: cluster 2020-09-08T15:42:29.665743+0000 mon.a (mon.0) 1141 : cluster [DBG] osdmap e223: 12 total, 12 up, 11 in 2020-09-08T15:42:30.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:30 smithi104 bash[9203]: cluster 2020-09-08T15:42:29.653289+0000 mon.a (mon.0) 1139 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:42:30.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:30 smithi104 bash[9203]: audit 2020-09-08T15:42:29.665537+0000 mon.a (mon.0) 1140 : audit [INF] from='client.? v1:172.21.15.104:0/2326522733' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:42:30.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:30 smithi104 bash[9203]: cluster 2020-09-08T15:42:29.665743+0000 mon.a (mon.0) 1141 : cluster [DBG] osdmap e223: 12 total, 12 up, 11 in 2020-09-08T15:42:30.972 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:42:31.075 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:31.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:30 smithi166 bash[9446]: cluster 2020-09-08T15:42:29.653289+0000 mon.a (mon.0) 1139 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:42:31.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:30 smithi166 bash[9446]: audit 2020-09-08T15:42:29.665537+0000 mon.a (mon.0) 1140 : audit [INF] from='client.? v1:172.21.15.104:0/2326522733' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:42:31.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:30 smithi166 bash[9446]: cluster 2020-09-08T15:42:29.665743+0000 mon.a (mon.0) 1141 : cluster [DBG] osdmap e223: 12 total, 12 up, 11 in 2020-09-08T15:42:31.222 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 11 15 31879 31864 181.019 115.688 0.0042959 0.0055219 2020-09-08T15:42:31.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:42:30 smithi104 bash[19922]: debug 2020-09-08T15:42:30.976+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:31.479 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:31.583 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:31.713 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:31.742 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:31 smithi104 bash[9203]: cluster 2020-09-08T15:42:29.669601+0000 mgr.x (mgr.24118) 453 : cluster [DBG] pgmap v594: 17 pgs: 17 active+clean; 1.3 GiB data, 3.3 GiB used, 980 GiB / 983 GiB avail; 169 MiB/s wr, 2.70k op/s 2020-09-08T15:42:31.743 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:42:31 smithi104 bash[22305]: debug 2020-09-08T15:42:31.480+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:31.816 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:31.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:31 smithi055 bash[9296]: cluster 2020-09-08T15:42:29.669601+0000 mgr.x (mgr.24118) 453 : cluster [DBG] pgmap v594: 17 pgs: 17 active+clean; 1.3 GiB data, 3.3 GiB used, 980 GiB / 983 GiB avail; 169 MiB/s wr, 2.70k op/s 2020-09-08T15:42:31.908 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:42:31 smithi055 bash[16646]: debug 2020-09-08T15:42:31.716+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:32.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:31 smithi166 bash[9446]: cluster 2020-09-08T15:42:29.669601+0000 mgr.x (mgr.24118) 453 : cluster [DBG] pgmap v594: 17 pgs: 17 active+clean; 1.3 GiB data, 3.3 GiB used, 980 GiB / 983 GiB avail; 169 MiB/s wr, 2.70k op/s 2020-09-08T15:42:32.218 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 12 16 34002 33986 176.985 132.625 0.00735748 0.00564817 2020-09-08T15:42:32.934 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:32 smithi104 bash[9203]: cluster 2020-09-08T15:42:31.670031+0000 mgr.x (mgr.24118) 454 : cluster [DBG] pgmap v595: 17 pgs: 17 active+clean; 1.5 GiB data, 3.5 GiB used, 980 GiB / 983 GiB avail; 150 MiB/s wr, 2.40k op/s 2020-09-08T15:42:33.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:32 smithi166 bash[9446]: cluster 2020-09-08T15:42:31.670031+0000 mgr.x (mgr.24118) 454 : cluster [DBG] pgmap v595: 17 pgs: 17 active+clean; 1.5 GiB data, 3.5 GiB used, 980 GiB / 983 GiB avail; 150 MiB/s wr, 2.40k op/s 2020-09-08T15:42:33.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:32 smithi055 bash[9296]: cluster 2020-09-08T15:42:31.670031+0000 mgr.x (mgr.24118) 454 : cluster [DBG] pgmap v595: 17 pgs: 17 active+clean; 1.5 GiB data, 3.5 GiB used, 980 GiB / 983 GiB avail; 150 MiB/s wr, 2.40k op/s 2020-09-08T15:42:33.215 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 13 16 36420 36404 174.995 151.125 0.00717698 0.00571053 2020-09-08T15:42:33.322 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:42:33.323 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2020-09-08T15:42:33.323 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2020-09-08T15:42:33.323 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T15:42:33.323 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.324 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:32395 2.5 e135/126 2:ae2af7d8:::benchmark_data_smithi166_20551_object32394:head v 135'2041, mlcod=135'2039)", 2020-09-08T15:42:33.324 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:37:52.720459+0000", 2020-09-08T15:42:33.324 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 280.59801793100002, 2020-09-08T15:42:33.324 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16542880800000001, 2020-09-08T15:42:33.325 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:42:33.325 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:42:33.325 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:42:33.325 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.325 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:42:33.326 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.720459+0000", 2020-09-08T15:42:33.326 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:42:33.327 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.327 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.327 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:42:33.327 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.720459+0000", 2020-09-08T15:42:33.327 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000116075 2020-09-08T15:42:33.328 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.328 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.328 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:42:33.328 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.720575+0000", 2020-09-08T15:42:33.328 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8030000000000002e-06 2020-09-08T15:42:33.329 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.329 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.329 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:42:33.329 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.720580+0000", 2020-09-08T15:42:33.330 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6640000000000004e-06 2020-09-08T15:42:33.330 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.331 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.331 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:42:33.331 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.720589+0000", 2020-09-08T15:42:33.331 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8950000000000003e-06 2020-09-08T15:42:33.331 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.332 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.332 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:42:33.332 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.720595+0000", 2020-09-08T15:42:33.332 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15654379299999999 2020-09-08T15:42:33.332 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.333 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.333 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:42:33.333 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.877139+0000", 2020-09-08T15:42:33.333 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9963000000000007e-05 2020-09-08T15:42:33.333 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.334 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.334 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:42:33.334 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.877219+0000", 2020-09-08T15:42:33.334 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0084794959999999996 2020-09-08T15:42:33.335 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.335 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.335 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:42:33.335 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.885698+0000", 2020-09-08T15:42:33.335 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018911899999999999 2020-09-08T15:42:33.336 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.336 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.336 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:42:33.337 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:37:52.885887+0000", 2020-09-08T15:42:33.337 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16542880800000001 2020-09-08T15:42:33.338 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.338 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.338 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.338 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.338 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.339 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:168405 2.5 e172/126 2:a65aa17e:::benchmark_data_smithi166_20551_object20785:head v 172'10423, mlcod=172'10421)", 2020-09-08T15:42:33.339 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:34.490923+0000", 2020-09-08T15:42:33.339 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 178.82755334699999, 2020-09-08T15:42:33.339 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.25628678799999999, 2020-09-08T15:42:33.339 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:42:33.342 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:42:33.342 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:42:33.342 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.342 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:42:33.343 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.490923+0000", 2020-09-08T15:42:33.343 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:42:33.343 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.343 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.343 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:42:33.343 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.490923+0000", 2020-09-08T15:42:33.344 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6544999999999994e-05 2020-09-08T15:42:33.344 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.344 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.344 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:42:33.344 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.490990+0000", 2020-09-08T15:42:33.345 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2035000000000001e-05 2020-09-08T15:42:33.345 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.345 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.345 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:42:33.345 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491002+0000", 2020-09-08T15:42:33.346 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9689000000000001e-05 2020-09-08T15:42:33.346 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.346 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.346 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:42:33.346 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491021+0000", 2020-09-08T15:42:33.347 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0033e-05 2020-09-08T15:42:33.347 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.348 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.348 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:42:33.348 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491031+0000", 2020-09-08T15:42:33.348 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5401999999999995e-05 2020-09-08T15:42:33.348 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.349 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.349 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:42:33.349 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491097+0000", 2020-09-08T15:42:33.349 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0103999999999998e-05 2020-09-08T15:42:33.349 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.350 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.350 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:42:33.350 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491127+0000", 2020-09-08T15:42:33.350 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.25595135000000002 2020-09-08T15:42:33.350 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.350 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.351 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:42:33.352 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.747078+0000", 2020-09-08T15:42:33.352 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013163000000000001 2020-09-08T15:42:33.352 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.352 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.352 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:42:33.353 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.747210+0000", 2020-09-08T15:42:33.353 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.25628678799999999 2020-09-08T15:42:33.353 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.353 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.353 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.354 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.354 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.354 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:168406 2.5 e172/126 2:adf3a2f3:::benchmark_data_smithi166_20551_object20786:head v 172'10424, mlcod=172'10421)", 2020-09-08T15:42:33.354 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:34.491101+0000", 2020-09-08T15:42:33.354 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 178.82737585699999, 2020-09-08T15:42:33.355 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26448300600000002, 2020-09-08T15:42:33.355 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:42:33.355 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:42:33.355 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:42:33.355 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.356 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:42:33.356 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491101+0000", 2020-09-08T15:42:33.356 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:42:33.356 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.357 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.357 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:42:33.357 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491101+0000", 2020-09-08T15:42:33.358 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3269000000000005e-05 2020-09-08T15:42:33.358 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.358 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.358 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:42:33.359 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491184+0000", 2020-09-08T15:42:33.359 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0890000000000006e-06 2020-09-08T15:42:33.359 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.359 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.360 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:42:33.360 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491193+0000", 2020-09-08T15:42:33.360 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5333e-05 2020-09-08T15:42:33.360 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.360 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.361 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:42:33.361 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491208+0000", 2020-09-08T15:42:33.361 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3520000000000003e-06 2020-09-08T15:42:33.361 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.362 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.362 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:42:33.362 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491215+0000", 2020-09-08T15:42:33.363 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0056763029999999997 2020-09-08T15:42:33.363 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.363 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.363 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:42:33.363 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.496891+0000", 2020-09-08T15:42:33.364 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0164000000000001e-05 2020-09-08T15:42:33.364 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.364 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.364 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:42:33.364 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.496921+0000", 2020-09-08T15:42:33.365 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.258572301 2020-09-08T15:42:33.365 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.365 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.365 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:42:33.365 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.755493+0000", 2020-09-08T15:42:33.366 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0195e-05 2020-09-08T15:42:33.366 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.366 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.366 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:42:33.366 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.755584+0000", 2020-09-08T15:42:33.366 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26448300600000002 2020-09-08T15:42:33.367 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.367 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.367 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.367 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.367 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.368 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:168416 2.5 e172/126 2:ad81a029:::benchmark_data_smithi166_20551_object20796:head v 172'10425, mlcod=172'10421)", 2020-09-08T15:42:33.368 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:34.499124+0000", 2020-09-08T15:42:33.368 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 178.81935225999999, 2020-09-08T15:42:33.369 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26367184900000001, 2020-09-08T15:42:33.369 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:42:33.369 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:42:33.369 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:42:33.370 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.370 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:42:33.370 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.499124+0000", 2020-09-08T15:42:33.370 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:42:33.370 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.371 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.371 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:42:33.371 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.499124+0000", 2020-09-08T15:42:33.371 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3675000000000003e-05 2020-09-08T15:42:33.372 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.372 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.372 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:42:33.373 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.499188+0000", 2020-09-08T15:42:33.373 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7679999999999997e-06 2020-09-08T15:42:33.373 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.373 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.374 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:42:33.374 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.499197+0000", 2020-09-08T15:42:33.374 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4082e-05 2020-09-08T15:42:33.374 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.374 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.375 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:42:33.375 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.499211+0000", 2020-09-08T15:42:33.375 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.691e-06 2020-09-08T15:42:33.375 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.375 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.375 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:42:33.376 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.499217+0000", 2020-09-08T15:42:33.376 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.24804995799999999 2020-09-08T15:42:33.376 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.376 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.376 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:42:33.377 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.747267+0000", 2020-09-08T15:42:33.377 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2809999999999998e-05 2020-09-08T15:42:33.377 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.377 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.377 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:42:33.378 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.747310+0000", 2020-09-08T15:42:33.378 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.015420059 2020-09-08T15:42:33.378 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.379 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.379 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:42:33.379 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.762730+0000", 2020-09-08T15:42:33.379 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5806e-05 2020-09-08T15:42:33.379 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.380 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.380 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:42:33.380 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.762796+0000", 2020-09-08T15:42:33.380 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26367184900000001 2020-09-08T15:42:33.380 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.381 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.382 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.382 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.382 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.382 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:190014 2.5 e179/126 2:a63b47c4:::benchmark_data_smithi166_20551_object42394:head v 179'11794, mlcod=178'11792)", 2020-09-08T15:42:33.383 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:56.744397+0000", 2020-09-08T15:42:33.383 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 156.57407929600001, 2020-09-08T15:42:33.383 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.23354170399999999, 2020-09-08T15:42:33.383 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:42:33.383 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:42:33.384 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:42:33.384 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.384 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:42:33.384 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.744397+0000", 2020-09-08T15:42:33.384 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:42:33.385 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.385 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.385 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:42:33.385 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.744397+0000", 2020-09-08T15:42:33.385 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6326000000000001e-05 2020-09-08T15:42:33.386 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.386 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.386 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:42:33.386 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.744474+0000", 2020-09-08T15:42:33.386 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8599999999999999e-06 2020-09-08T15:42:33.387 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.387 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.387 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:42:33.387 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.744482+0000", 2020-09-08T15:42:33.387 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1197e-05 2020-09-08T15:42:33.388 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.388 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.388 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:42:33.388 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.744514+0000", 2020-09-08T15:42:33.388 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7800000000000001e-06 2020-09-08T15:42:33.389 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.389 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.389 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:42:33.389 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.744521+0000", 2020-09-08T15:42:33.389 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9327999999999998e-05 2020-09-08T15:42:33.389 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.390 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.390 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:42:33.390 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.744581+0000", 2020-09-08T15:42:33.390 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1537000000000001e-05 2020-09-08T15:42:33.390 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.391 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.391 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:42:33.391 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.744602+0000", 2020-09-08T15:42:33.391 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.233170874 2020-09-08T15:42:33.391 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.392 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.392 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:42:33.392 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.977773+0000", 2020-09-08T15:42:33.392 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000165802 2020-09-08T15:42:33.392 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.393 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.393 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:42:33.393 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.977939+0000", 2020-09-08T15:42:33.393 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.23354170399999999 2020-09-08T15:42:33.394 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.394 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.394 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.394 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.394 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.395 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:190017 2.5 e179/126 2:a3e31b0f:::benchmark_data_smithi166_20551_object42397:head v 179'11795, mlcod=178'11792)", 2020-09-08T15:42:33.395 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:56.744661+0000", 2020-09-08T15:42:33.395 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 156.57381529599999, 2020-09-08T15:42:33.395 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.233401841, 2020-09-08T15:42:33.395 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:42:33.396 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:42:33.396 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:42:33.396 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.396 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:42:33.396 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.744661+0000", 2020-09-08T15:42:33.396 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:42:33.397 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.397 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.397 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:42:33.397 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.744661+0000", 2020-09-08T15:42:33.397 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0197999999999993e-05 2020-09-08T15:42:33.398 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.398 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.398 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:42:33.398 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.744741+0000", 2020-09-08T15:42:33.398 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7090000000000008e-06 2020-09-08T15:42:33.399 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.399 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.399 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:42:33.399 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.744750+0000", 2020-09-08T15:42:33.399 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4688000000000001e-05 2020-09-08T15:42:33.400 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.400 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.400 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:42:33.400 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.744765+0000", 2020-09-08T15:42:33.401 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9880000000000004e-06 2020-09-08T15:42:33.401 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.401 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.401 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:42:33.401 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.744770+0000", 2020-09-08T15:42:33.402 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0058076789999999996 2020-09-08T15:42:33.402 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.402 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.402 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:42:33.402 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.750577+0000", 2020-09-08T15:42:33.402 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.7305999999999999e-05 2020-09-08T15:42:33.403 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.403 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.403 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:42:33.403 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.750605+0000", 2020-09-08T15:42:33.403 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.22734027300000001 2020-09-08T15:42:33.404 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.404 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.404 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:42:33.404 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.977945+0000", 2020-09-08T15:42:33.404 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000118 2020-09-08T15:42:33.405 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.405 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.405 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:42:33.405 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.978063+0000", 2020-09-08T15:42:33.405 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.233401841 2020-09-08T15:42:33.405 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.406 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.406 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.406 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.406 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.406 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:190024 2.4 e179/126 2:223e1daa:::benchmark_data_smithi166_20551_object42404:head v 179'11903, mlcod=178'11901)", 2020-09-08T15:42:33.407 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:56.751495+0000", 2020-09-08T15:42:33.407 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 156.56698131900001, 2020-09-08T15:42:33.407 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.24695220000000001, 2020-09-08T15:42:33.407 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:42:33.407 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:42:33.408 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:42:33.408 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.408 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:42:33.408 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.751495+0000", 2020-09-08T15:42:33.408 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:42:33.408 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.409 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.409 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:42:33.409 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.751495+0000", 2020-09-08T15:42:33.409 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1027999999999999e-05 2020-09-08T15:42:33.409 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.410 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.410 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:42:33.410 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.751556+0000", 2020-09-08T15:42:33.410 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4340000000000003e-06 2020-09-08T15:42:33.410 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.411 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.411 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:42:33.411 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.751564+0000", 2020-09-08T15:42:33.411 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2662e-05 2020-09-08T15:42:33.411 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.412 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.412 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:42:33.412 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.751576+0000", 2020-09-08T15:42:33.412 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3640000000000004e-06 2020-09-08T15:42:33.412 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.413 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.413 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:42:33.413 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.751583+0000", 2020-09-08T15:42:33.413 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.22617980100000001 2020-09-08T15:42:33.413 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.414 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.414 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:42:33.414 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.977762+0000", 2020-09-08T15:42:33.414 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9385000000000001e-05 2020-09-08T15:42:33.415 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.415 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.415 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:42:33.415 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.977792+0000", 2020-09-08T15:42:33.415 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.020575709000000001 2020-09-08T15:42:33.416 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.416 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.416 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:42:33.416 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.998368+0000", 2020-09-08T15:42:33.416 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9816999999999998e-05 2020-09-08T15:42:33.417 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.417 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.417 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:42:33.417 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.998447+0000", 2020-09-08T15:42:33.417 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.24695220000000001 2020-09-08T15:42:33.418 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.418 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.418 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.418 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.419 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.419 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:190950 2.4 e179/126 2:238df183:::benchmark_data_smithi166_20551_object43330:head v 179'11957, mlcod=179'11953)", 2020-09-08T15:42:33.420 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:57.816566+0000", 2020-09-08T15:42:33.420 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 155.50191008799999, 2020-09-08T15:42:33.420 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.24666176100000001, 2020-09-08T15:42:33.420 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:42:33.421 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:42:33.421 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:42:33.421 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.421 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:42:33.421 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.816566+0000", 2020-09-08T15:42:33.422 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:42:33.422 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.422 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.422 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:42:33.422 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.816566+0000", 2020-09-08T15:42:33.422 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2438000000000001e-05 2020-09-08T15:42:33.423 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.423 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.423 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:42:33.423 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.816629+0000", 2020-09-08T15:42:33.423 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4800000000000001e-06 2020-09-08T15:42:33.424 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.424 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.424 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:42:33.424 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.816637+0000", 2020-09-08T15:42:33.424 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2876e-05 2020-09-08T15:42:33.425 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.425 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.425 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:42:33.425 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.816650+0000", 2020-09-08T15:42:33.425 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2620000000000004e-06 2020-09-08T15:42:33.426 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.426 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.426 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:42:33.426 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.816656+0000", 2020-09-08T15:42:33.426 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0235000000000003e-05 2020-09-08T15:42:33.427 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.427 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.427 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:42:33.427 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.816707+0000", 2020-09-08T15:42:33.427 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6855e-05 2020-09-08T15:42:33.428 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.428 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.428 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:42:33.428 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.816734+0000", 2020-09-08T15:42:33.428 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.246288009 2020-09-08T15:42:33.429 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.429 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.429 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:42:33.429 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.063022+0000", 2020-09-08T15:42:33.429 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020660599999999999 2020-09-08T15:42:33.430 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.430 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.430 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:42:33.430 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.063228+0000", 2020-09-08T15:42:33.430 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.24666176100000001 2020-09-08T15:42:33.431 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.431 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.431 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.431 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.431 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.432 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:190956 2.c e179/126 2:3977f071:::benchmark_data_smithi166_20551_object43336:head v 179'11882, mlcod=179'11879)", 2020-09-08T15:42:33.432 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:57.817105+0000", 2020-09-08T15:42:33.432 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 155.50137157500001, 2020-09-08T15:42:33.432 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.246211079, 2020-09-08T15:42:33.432 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:42:33.432 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:42:33.433 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:42:33.433 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.433 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:42:33.433 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.817105+0000", 2020-09-08T15:42:33.433 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:42:33.434 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.434 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.434 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:42:33.434 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.817105+0000", 2020-09-08T15:42:33.434 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1634e-05 2020-09-08T15:42:33.435 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.435 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.435 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:42:33.435 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.817167+0000", 2020-09-08T15:42:33.435 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6590000000000007e-06 2020-09-08T15:42:33.436 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.436 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.436 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:42:33.436 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.817174+0000", 2020-09-08T15:42:33.436 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2143e-05 2020-09-08T15:42:33.436 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.437 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.437 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:42:33.437 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.817186+0000", 2020-09-08T15:42:33.437 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2000000000000002e-06 2020-09-08T15:42:33.437 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.438 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.438 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:42:33.438 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.817192+0000", 2020-09-08T15:42:33.438 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003370184 2020-09-08T15:42:33.438 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.439 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.439 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:42:33.439 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.820562+0000", 2020-09-08T15:42:33.439 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.7878999999999999e-05 2020-09-08T15:42:33.439 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.440 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.440 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:42:33.440 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.820590+0000", 2020-09-08T15:42:33.440 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.242655176 2020-09-08T15:42:33.440 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.440 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.441 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:42:33.441 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.063245+0000", 2020-09-08T15:42:33.441 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1204000000000005e-05 2020-09-08T15:42:33.441 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.441 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.442 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:42:33.442 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.063316+0000", 2020-09-08T15:42:33.442 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.246211079 2020-09-08T15:42:33.442 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.442 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.443 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.443 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.443 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.443 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:190953 2.4 e179/126 2:2ee0e6e8:::benchmark_data_smithi166_20551_object43333:head v 179'11958, mlcod=179'11953)", 2020-09-08T15:42:33.443 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:57.820699+0000", 2020-09-08T15:42:33.444 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 155.49777795, 2020-09-08T15:42:33.444 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.265174093, 2020-09-08T15:42:33.444 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:42:33.444 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:42:33.445 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:42:33.445 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.445 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:42:33.445 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.820699+0000", 2020-09-08T15:42:33.445 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:42:33.446 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.446 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.446 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:42:33.446 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.820699+0000", 2020-09-08T15:42:33.446 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.24242719200000001 2020-09-08T15:42:33.447 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.447 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.447 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:42:33.447 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.063126+0000", 2020-09-08T15:42:33.447 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3751999999999999e-05 2020-09-08T15:42:33.447 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.448 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.448 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:42:33.448 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.063149+0000", 2020-09-08T15:42:33.448 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6261999999999999e-05 2020-09-08T15:42:33.449 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.449 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.449 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:42:33.449 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.063186+0000", 2020-09-08T15:42:33.449 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.632e-05 2020-09-08T15:42:33.450 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.450 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.450 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:42:33.450 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.063202+0000", 2020-09-08T15:42:33.450 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3277999999999999e-05 2020-09-08T15:42:33.451 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.451 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.451 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:42:33.451 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.063275+0000", 2020-09-08T15:42:33.451 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1987000000000001e-05 2020-09-08T15:42:33.451 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.452 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.452 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:42:33.452 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.063297+0000", 2020-09-08T15:42:33.452 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.022480079 2020-09-08T15:42:33.452 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.453 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.453 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:42:33.453 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.085777+0000", 2020-09-08T15:42:33.453 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5223000000000006e-05 2020-09-08T15:42:33.453 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.454 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.454 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:42:33.454 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.085873+0000", 2020-09-08T15:42:33.454 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.265174093 2020-09-08T15:42:33.454 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.455 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.455 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.455 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.455 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.455 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:223524 2.c e189/126 2:3ad6268a:::benchmark_data_smithi166_20551_object75904:head v 189'13910, mlcod=189'13906)", 2020-09-08T15:42:33.456 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:40:31.454160+0000", 2020-09-08T15:42:33.456 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 121.864316923, 2020-09-08T15:42:33.456 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.30000275999999998, 2020-09-08T15:42:33.456 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:42:33.456 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:42:33.456 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:42:33.457 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.457 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:42:33.457 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.454160+0000", 2020-09-08T15:42:33.457 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:42:33.457 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.458 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.458 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:42:33.458 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.454160+0000", 2020-09-08T15:42:33.458 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2197000000000001e-05 2020-09-08T15:42:33.458 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.459 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.459 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:42:33.459 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.454222+0000", 2020-09-08T15:42:33.459 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2730000000000002e-06 2020-09-08T15:42:33.459 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.460 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.460 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:42:33.460 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.454230+0000", 2020-09-08T15:42:33.460 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3343000000000001e-05 2020-09-08T15:42:33.460 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.461 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.461 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:42:33.461 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.454243+0000", 2020-09-08T15:42:33.461 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.105e-06 2020-09-08T15:42:33.461 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.461 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.462 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:42:33.462 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.454249+0000", 2020-09-08T15:42:33.462 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5810999999999999e-05 2020-09-08T15:42:33.462 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.462 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.463 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:42:33.463 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.454295+0000", 2020-09-08T15:42:33.463 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3672e-05 2020-09-08T15:42:33.463 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.463 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.464 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:42:33.464 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.454319+0000", 2020-09-08T15:42:33.464 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29974217399999997 2020-09-08T15:42:33.464 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.464 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.465 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:42:33.465 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.754061+0000", 2020-09-08T15:42:33.465 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000101185 2020-09-08T15:42:33.465 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.465 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.466 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:42:33.466 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.754162+0000", 2020-09-08T15:42:33.466 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.30000275999999998 2020-09-08T15:42:33.466 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.466 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.466 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.467 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.467 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.467 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:223535 2.c e189/126 2:3b35efb1:::benchmark_data_smithi166_20551_object75915:head v 189'13911, mlcod=189'13908)", 2020-09-08T15:42:33.467 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:40:31.455185+0000", 2020-09-08T15:42:33.467 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 121.863291222, 2020-09-08T15:42:33.468 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29904997799999999, 2020-09-08T15:42:33.468 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:42:33.468 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:42:33.468 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:42:33.468 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.469 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:42:33.470 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455185+0000", 2020-09-08T15:42:33.470 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:42:33.470 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.470 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.471 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:42:33.471 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455185+0000", 2020-09-08T15:42:33.471 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7972000000000006e-05 2020-09-08T15:42:33.471 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.471 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.472 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:42:33.472 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455253+0000", 2020-09-08T15:42:33.472 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6969999999999999e-06 2020-09-08T15:42:33.472 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.472 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.472 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:42:33.473 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455262+0000", 2020-09-08T15:42:33.473 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4432e-05 2020-09-08T15:42:33.473 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.473 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.473 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:42:33.474 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455276+0000", 2020-09-08T15:42:33.474 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4259999999999996e-06 2020-09-08T15:42:33.474 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.474 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.474 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:42:33.475 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455281+0000", 2020-09-08T15:42:33.475 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0042721829999999997 2020-09-08T15:42:33.475 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.475 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.475 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:42:33.476 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.459553+0000", 2020-09-08T15:42:33.476 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6232999999999999e-05 2020-09-08T15:42:33.476 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.476 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.477 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:42:33.477 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.459569+0000", 2020-09-08T15:42:33.478 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29460175999999999 2020-09-08T15:42:33.478 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.479 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.479 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:42:33.479 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.754171+0000", 2020-09-08T15:42:33.479 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4275000000000004e-05 2020-09-08T15:42:33.479 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.480 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.480 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:42:33.480 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.754235+0000", 2020-09-08T15:42:33.480 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29904997799999999 2020-09-08T15:42:33.480 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.481 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.481 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.481 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.481 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.481 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:223538 2.5 e189/126 2:af9447da:::benchmark_data_smithi166_20551_object75918:head v 189'13833, mlcod=189'13831)", 2020-09-08T15:42:33.482 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:40:31.455706+0000", 2020-09-08T15:42:33.482 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 121.862770235, 2020-09-08T15:42:33.482 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.31563575399999999, 2020-09-08T15:42:33.482 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:42:33.482 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:42:33.483 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:42:33.483 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.483 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:42:33.483 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455706+0000", 2020-09-08T15:42:33.483 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:42:33.484 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.484 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.484 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:42:33.485 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455706+0000", 2020-09-08T15:42:33.485 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7342999999999995e-05 2020-09-08T15:42:33.485 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.485 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.485 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:42:33.486 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455774+0000", 2020-09-08T15:42:33.486 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6449999999999994e-06 2020-09-08T15:42:33.486 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.486 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.486 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:42:33.487 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455782+0000", 2020-09-08T15:42:33.487 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3685e-05 2020-09-08T15:42:33.487 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.487 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.487 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:42:33.488 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455796+0000", 2020-09-08T15:42:33.488 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3369999999999996e-06 2020-09-08T15:42:33.489 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.489 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.489 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:42:33.490 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455802+0000", 2020-09-08T15:42:33.490 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000106783 2020-09-08T15:42:33.490 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.490 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.491 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:42:33.491 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455909+0000", 2020-09-08T15:42:33.491 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.357e-05 2020-09-08T15:42:33.491 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.492 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.492 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:42:33.492 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455933+0000", 2020-09-08T15:42:33.493 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.31528879999999998 2020-09-08T15:42:33.493 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.493 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.493 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:42:33.494 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.771221+0000", 2020-09-08T15:42:33.494 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000120591 2020-09-08T15:42:33.494 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.494 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.494 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:42:33.495 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.771342+0000", 2020-09-08T15:42:33.495 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.31563575399999999 2020-09-08T15:42:33.495 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.495 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.495 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.496 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.496 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.496 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:223539 2.5 e189/126 2:ac3e761c:::benchmark_data_smithi166_20551_object75919:head v 189'13834, mlcod=189'13831)", 2020-09-08T15:42:33.496 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:40:31.461367+0000", 2020-09-08T15:42:33.496 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 121.857109737, 2020-09-08T15:42:33.497 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.31003701900000002, 2020-09-08T15:42:33.497 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:42:33.497 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:42:33.497 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:42:33.497 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.498 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:42:33.498 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.461367+0000", 2020-09-08T15:42:33.498 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:42:33.498 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.498 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.499 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:42:33.499 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.461367+0000", 2020-09-08T15:42:33.499 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4367000000000002e-05 2020-09-08T15:42:33.499 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.500 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.500 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:42:33.500 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.461431+0000", 2020-09-08T15:42:33.500 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1689999999999998e-06 2020-09-08T15:42:33.500 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.500 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.501 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:42:33.501 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.461440+0000", 2020-09-08T15:42:33.501 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3594000000000001e-05 2020-09-08T15:42:33.502 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.502 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.502 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:42:33.502 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.461454+0000", 2020-09-08T15:42:33.502 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8560000000000003e-06 2020-09-08T15:42:33.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.503 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.503 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:42:33.503 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.461460+0000", 2020-09-08T15:42:33.503 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29267275199999998 2020-09-08T15:42:33.504 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.504 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.504 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:42:33.504 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.754133+0000", 2020-09-08T15:42:33.505 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5941999999999999e-05 2020-09-08T15:42:33.505 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.505 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.505 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:42:33.505 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.754208+0000", 2020-09-08T15:42:33.506 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.017145581 2020-09-08T15:42:33.506 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.506 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.506 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:42:33.506 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.771354+0000", 2020-09-08T15:42:33.507 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9758e-05 2020-09-08T15:42:33.507 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.507 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.507 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:42:33.507 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.771404+0000", 2020-09-08T15:42:33.508 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.31003701900000002 2020-09-08T15:42:33.508 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.508 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.508 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.508 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.510 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.510 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:5402 3.3 3:c3410f15:::benchmark_data_smithi166_26097_object5401:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e220)", 2020-09-08T15:42:33.510 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:42:21.601358+0000", 2020-09-08T15:42:33.511 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 11.717118765, 2020-09-08T15:42:33.511 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15635258199999999, 2020-09-08T15:42:33.511 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:42:33.511 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:42:33.511 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:42:33.512 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:42:33.512 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:42:33.512 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 5402 2020-09-08T15:42:33.512 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.512 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:42:33.513 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.513 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:42:33.513 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:21.601358+0000", 2020-09-08T15:42:33.513 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:42:33.513 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.514 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.514 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:42:33.514 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:21.601358+0000", 2020-09-08T15:42:33.514 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999576 2020-09-08T15:42:33.514 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.515 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.515 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:42:33.515 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:21.601315+0000", 2020-09-08T15:42:33.515 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020960299999999999 2020-09-08T15:42:33.515 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.516 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.516 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:42:33.516 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:21.601525+0000", 2020-09-08T15:42:33.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5636000000000001e-05 2020-09-08T15:42:33.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:42:33.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:21.601541+0000", 2020-09-08T15:42:33.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4919999999999998e-06 2020-09-08T15:42:33.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:42:33.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:21.601547+0000", 2020-09-08T15:42:33.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00067876999999999998 2020-09-08T15:42:33.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:42:33.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:21.602226+0000", 2020-09-08T15:42:33.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016995000000000001 2020-09-08T15:42:33.521 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.522 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.522 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:42:33.522 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:21.602396+0000", 2020-09-08T15:42:33.522 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019151300000000001 2020-09-08T15:42:33.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T15:42:33.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:21.602587+0000", 2020-09-08T15:42:33.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000715432 2020-09-08T15:42:33.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.524 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:42:33.524 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:21.603303+0000", 2020-09-08T15:42:33.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15313569899999999 2020-09-08T15:42:33.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:42:33.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:21.756438+0000", 2020-09-08T15:42:33.525 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0011282149999999999 2020-09-08T15:42:33.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:42:33.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:21.757567+0000", 2020-09-08T15:42:33.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000143738 2020-09-08T15:42:33.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.527 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.527 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:42:33.527 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:21.757710+0000", 2020-09-08T15:42:33.527 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15635258199999999 2020-09-08T15:42:33.527 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.528 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.528 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.528 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:14002 3.6 3:6d1cec01:::benchmark_data_smithi166_26097_object14001:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e221)", 2020-09-08T15:42:33.529 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:42:24.259085+0000", 2020-09-08T15:42:33.529 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 9.0593918840000001, 2020-09-08T15:42:33.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.183730859, 2020-09-08T15:42:33.529 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:42:33.529 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:42:33.530 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:42:33.531 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:42:33.531 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:42:33.531 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 14002 2020-09-08T15:42:33.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.531 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:42:33.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.532 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:42:33.532 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.259085+0000", 2020-09-08T15:42:33.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:42:33.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.533 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.533 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:42:33.533 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.259085+0000", 2020-09-08T15:42:33.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999413 2020-09-08T15:42:33.533 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.534 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:42:33.536 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.259026+0000", 2020-09-08T15:42:33.537 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000159576 2020-09-08T15:42:33.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.538 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:42:33.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.259186+0000", 2020-09-08T15:42:33.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5400000000000002e-05 2020-09-08T15:42:33.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.539 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:42:33.539 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.259201+0000", 2020-09-08T15:42:33.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0870000000000003e-06 2020-09-08T15:42:33.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:42:33.541 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.259207+0000", 2020-09-08T15:42:33.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4575e-05 2020-09-08T15:42:33.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:42:33.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.259242+0000", 2020-09-08T15:42:33.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017866400000000001 2020-09-08T15:42:33.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.543 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:42:33.543 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.259420+0000", 2020-09-08T15:42:33.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000215045 2020-09-08T15:42:33.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T15:42:33.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.259635+0000", 2020-09-08T15:42:33.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00089183600000000002 2020-09-08T15:42:33.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:42:33.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.260527+0000", 2020-09-08T15:42:33.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18216494699999999 2020-09-08T15:42:33.546 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.546 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:42:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.442692+0000", 2020-09-08T15:42:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7516999999999997e-05 2020-09-08T15:42:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:42:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.442760+0000", 2020-09-08T15:42:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5727999999999999e-05 2020-09-08T15:42:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:42:33.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.442815+0000", 2020-09-08T15:42:33.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.183730859 2020-09-08T15:42:33.550 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.550 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.550 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.550 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.551 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:14005 3.1 3:8b5c2c2d:::benchmark_data_smithi166_26097_object14004:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e221)", 2020-09-08T15:42:33.551 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:42:24.259465+0000", 2020-09-08T15:42:33.551 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 9.0590114239999995, 2020-09-08T15:42:33.551 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.188065965, 2020-09-08T15:42:33.552 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:42:33.552 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:42:33.552 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:42:33.552 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:42:33.552 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:42:33.553 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 14005 2020-09-08T15:42:33.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.553 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:42:33.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.554 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:42:33.554 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.259465+0000", 2020-09-08T15:42:33.554 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:42:33.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.555 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:42:33.555 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.259465+0000", 2020-09-08T15:42:33.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999638 2020-09-08T15:42:33.555 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.555 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.556 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:42:33.556 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.259429+0000", 2020-09-08T15:42:33.556 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013092600000000001 2020-09-08T15:42:33.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.557 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.557 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:42:33.557 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.259560+0000", 2020-09-08T15:42:33.557 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2999e-05 2020-09-08T15:42:33.557 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.559 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:42:33.559 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.259573+0000", 2020-09-08T15:42:33.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2790000000000004e-06 2020-09-08T15:42:33.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.561 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:42:33.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.259579+0000", 2020-09-08T15:42:33.561 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1312e-05 2020-09-08T15:42:33.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.562 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.562 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:42:33.562 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.259621+0000", 2020-09-08T15:42:33.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013084199999999999 2020-09-08T15:42:33.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:42:33.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.259751+0000", 2020-09-08T15:42:33.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018228 2020-09-08T15:42:33.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.564 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T15:42:33.565 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.259934+0000", 2020-09-08T15:42:33.565 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001025916 2020-09-08T15:42:33.565 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.565 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.566 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:42:33.566 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.260960+0000", 2020-09-08T15:42:33.566 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18643510799999999 2020-09-08T15:42:33.566 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.567 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.567 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:42:33.567 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.447395+0000", 2020-09-08T15:42:33.567 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1904999999999997e-05 2020-09-08T15:42:33.567 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.568 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.568 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:42:33.568 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.447467+0000", 2020-09-08T15:42:33.569 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4444999999999997e-05 2020-09-08T15:42:33.569 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.569 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.569 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:42:33.570 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.447531+0000", 2020-09-08T15:42:33.570 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.188065965 2020-09-08T15:42:33.570 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.571 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.571 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.571 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.571 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.571 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:14010 3.3 3:cf43ff66:::benchmark_data_smithi166_26097_object14009:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e221)", 2020-09-08T15:42:33.572 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:42:24.261218+0000", 2020-09-08T15:42:33.572 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 9.0572582340000007, 2020-09-08T15:42:33.572 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18736646700000001, 2020-09-08T15:42:33.572 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:42:33.573 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:42:33.573 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:42:33.573 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:42:33.573 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:42:33.574 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 14010 2020-09-08T15:42:33.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.574 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:42:33.574 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:42:33.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261218+0000", 2020-09-08T15:42:33.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:42:33.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:42:33.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261218+0000", 2020-09-08T15:42:33.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999595 2020-09-08T15:42:33.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.578 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.578 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:42:33.578 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261178+0000", 2020-09-08T15:42:33.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014161999999999999 2020-09-08T15:42:33.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.579 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.579 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:42:33.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261319+0000", 2020-09-08T15:42:33.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.361e-05 2020-09-08T15:42:33.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:42:33.581 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261333+0000", 2020-09-08T15:42:33.581 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9610000000000002e-06 2020-09-08T15:42:33.581 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.581 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.581 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:42:33.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261339+0000", 2020-09-08T15:42:33.582 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5729000000000002e-05 2020-09-08T15:42:33.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:42:33.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261385+0000", 2020-09-08T15:42:33.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000145444 2020-09-08T15:42:33.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:42:33.583 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261530+0000", 2020-09-08T15:42:33.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017737100000000001 2020-09-08T15:42:33.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.584 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T15:42:33.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261707+0000", 2020-09-08T15:42:33.585 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.179901792 2020-09-08T15:42:33.585 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.585 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.585 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:42:33.585 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.441609+0000", 2020-09-08T15:42:33.586 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0068064780000000004 2020-09-08T15:42:33.586 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.586 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:42:33.586 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.448416+0000", 2020-09-08T15:42:33.587 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000102224 2020-09-08T15:42:33.588 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.588 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.588 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:42:33.588 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.448518+0000", 2020-09-08T15:42:33.588 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6742000000000006e-05 2020-09-08T15:42:33.589 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.589 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.589 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:42:33.589 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.448585+0000", 2020-09-08T15:42:33.589 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18736646700000001 2020-09-08T15:42:33.590 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.590 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.590 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.590 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.590 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.591 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:14011 3.6 3:66cd5184:::benchmark_data_smithi166_26097_object14010:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e221)", 2020-09-08T15:42:33.591 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:42:24.261410+0000", 2020-09-08T15:42:33.591 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 9.0570664010000002, 2020-09-08T15:42:33.591 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.19494124299999999, 2020-09-08T15:42:33.591 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:42:33.592 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:42:33.592 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:42:33.592 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:42:33.592 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:42:33.592 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 14011 2020-09-08T15:42:33.593 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.593 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:42:33.593 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:42:33.593 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261410+0000", 2020-09-08T15:42:33.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:42:33.594 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.594 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:42:33.594 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261410+0000", 2020-09-08T15:42:33.595 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999614 2020-09-08T15:42:33.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.595 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.595 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:42:33.595 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261371+0000", 2020-09-08T15:42:33.595 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000137634 2020-09-08T15:42:33.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:42:33.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261509+0000", 2020-09-08T15:42:33.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0011e-05 2020-09-08T15:42:33.597 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.597 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:42:33.597 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261519+0000", 2020-09-08T15:42:33.597 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7349999999999998e-06 2020-09-08T15:42:33.598 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.598 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.598 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:42:33.598 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261523+0000", 2020-09-08T15:42:33.598 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1743000000000003e-05 2020-09-08T15:42:33.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.599 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.599 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:42:33.599 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261564+0000", 2020-09-08T15:42:33.599 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014421700000000001 2020-09-08T15:42:33.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:42:33.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261709+0000", 2020-09-08T15:42:33.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17987958700000001 2020-09-08T15:42:33.600 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T15:42:33.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.441588+0000", 2020-09-08T15:42:33.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0013824709999999999 2020-09-08T15:42:33.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:42:33.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.442971+0000", 2020-09-08T15:42:33.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013315309000000001 2020-09-08T15:42:33.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:42:33.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.456286+0000", 2020-09-08T15:42:33.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6254000000000002e-05 2020-09-08T15:42:33.604 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:42:33.604 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.456322+0000", 2020-09-08T15:42:33.604 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9110000000000001e-05 2020-09-08T15:42:33.605 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.605 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.605 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:42:33.605 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.456351+0000", 2020-09-08T15:42:33.605 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.19494124299999999 2020-09-08T15:42:33.606 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.606 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.606 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.606 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.606 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.607 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop_reply(client.35087.0:14002 3.6 e221/217 ondisk, result = 0)", 2020-09-08T15:42:33.607 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:42:24.261608+0000", 2020-09-08T15:42:33.607 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 9.0568688749999993, 2020-09-08T15:42:33.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.181215599, 2020-09-08T15:42:33.607 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:42:33.607 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2020-09-08T15:42:33.608 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:42:33.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:42:33.608 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261608+0000", 2020-09-08T15:42:33.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:42:33.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.609 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.609 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:42:33.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261608+0000", 2020-09-08T15:42:33.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6563e-05 2020-09-08T15:42:33.610 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.610 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.610 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:42:33.610 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261634+0000", 2020-09-08T15:42:33.610 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7190000000000004e-06 2020-09-08T15:42:33.611 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.611 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.611 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:42:33.611 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261641+0000", 2020-09-08T15:42:33.612 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.047e-05 2020-09-08T15:42:33.612 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.612 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.612 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:42:33.612 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261651+0000", 2020-09-08T15:42:33.613 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3059999999999997e-06 2020-09-08T15:42:33.613 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.613 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.613 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:42:33.613 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.261657+0000", 2020-09-08T15:42:33.614 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18101355599999999 2020-09-08T15:42:33.614 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.614 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:42:33.614 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.442670+0000", 2020-09-08T15:42:33.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1695000000000001e-05 2020-09-08T15:42:33.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.615 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:42:33.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.442682+0000", 2020-09-08T15:42:33.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014129 2020-09-08T15:42:33.616 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.616 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.616 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:42:33.616 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:24.442823+0000", 2020-09-08T15:42:33.616 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.181215599 2020-09-08T15:42:33.617 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.617 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.617 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.617 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.617 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.618 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:42:33.619 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T15:42:33.620 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:42:33.620 INFO:teuthology.orchestra.run.smithi104.stdout: "epoch": 223, 2020-09-08T15:42:33.620 INFO:teuthology.orchestra.run.smithi104.stdout: "fsid": "759b27b6-f1e7-11ea-a080-001a4aab830c", 2020-09-08T15:42:33.620 INFO:teuthology.orchestra.run.smithi104.stdout: "created": "2020-09-08T15:25:41.788365+0000", 2020-09-08T15:42:33.620 INFO:teuthology.orchestra.run.smithi104.stdout: "modified": "2020-09-08T15:42:29.653098+0000", 2020-09-08T15:42:33.621 INFO:teuthology.orchestra.run.smithi104.stdout: "last_up_change": "2020-09-08T15:34:41.591462+0000", 2020-09-08T15:42:33.621 INFO:teuthology.orchestra.run.smithi104.stdout: "last_in_change": "2020-09-08T15:42:12.636980+0000", 2020-09-08T15:42:33.621 INFO:teuthology.orchestra.run.smithi104.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-09-08T15:42:33.621 INFO:teuthology.orchestra.run.smithi104.stdout: "flags_num": 5806080, 2020-09-08T15:42:33.621 INFO:teuthology.orchestra.run.smithi104.stdout: "flags_set": [ 2020-09-08T15:42:33.622 INFO:teuthology.orchestra.run.smithi104.stdout: "nodeep-scrub", 2020-09-08T15:42:33.622 INFO:teuthology.orchestra.run.smithi104.stdout: "noscrub", 2020-09-08T15:42:33.622 INFO:teuthology.orchestra.run.smithi104.stdout: "pglog_hardlimit", 2020-09-08T15:42:33.622 INFO:teuthology.orchestra.run.smithi104.stdout: "purged_snapdirs", 2020-09-08T15:42:33.622 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_deletes", 2020-09-08T15:42:33.623 INFO:teuthology.orchestra.run.smithi104.stdout: "sortbitwise" 2020-09-08T15:42:33.623 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T15:42:33.623 INFO:teuthology.orchestra.run.smithi104.stdout: "crush_version": 27, 2020-09-08T15:42:33.623 INFO:teuthology.orchestra.run.smithi104.stdout: "full_ratio": 0.94999998807907104, 2020-09-08T15:42:33.623 INFO:teuthology.orchestra.run.smithi104.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-09-08T15:42:33.624 INFO:teuthology.orchestra.run.smithi104.stdout: "nearfull_ratio": 0.85000002384185791, 2020-09-08T15:42:33.624 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_snapshot": "", 2020-09-08T15:42:33.624 INFO:teuthology.orchestra.run.smithi104.stdout: "pool_max": 3, 2020-09-08T15:42:33.624 INFO:teuthology.orchestra.run.smithi104.stdout: "max_osd": 12, 2020-09-08T15:42:33.624 INFO:teuthology.orchestra.run.smithi104.stdout: "require_min_compat_client": "luminous", 2020-09-08T15:42:33.625 INFO:teuthology.orchestra.run.smithi104.stdout: "min_compat_client": "jewel", 2020-09-08T15:42:33.625 INFO:teuthology.orchestra.run.smithi104.stdout: "require_osd_release": "pacific", 2020-09-08T15:42:33.625 INFO:teuthology.orchestra.run.smithi104.stdout: "pools": [ 2020-09-08T15:42:33.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.626 INFO:teuthology.orchestra.run.smithi104.stdout: "pool": 1, 2020-09-08T15:42:33.626 INFO:teuthology.orchestra.run.smithi104.stdout: "pool_name": "device_health_metrics", 2020-09-08T15:42:33.626 INFO:teuthology.orchestra.run.smithi104.stdout: "create_time": "2020-09-08T15:25:56.314385+0000", 2020-09-08T15:42:33.626 INFO:teuthology.orchestra.run.smithi104.stdout: "flags": 1, 2020-09-08T15:42:33.626 INFO:teuthology.orchestra.run.smithi104.stdout: "flags_names": "hashpspool", 2020-09-08T15:42:33.627 INFO:teuthology.orchestra.run.smithi104.stdout: "type": 1, 2020-09-08T15:42:33.627 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 2, 2020-09-08T15:42:33.627 INFO:teuthology.orchestra.run.smithi104.stdout: "min_size": 2, 2020-09-08T15:42:33.627 INFO:teuthology.orchestra.run.smithi104.stdout: "crush_rule": 0, 2020-09-08T15:42:33.628 INFO:teuthology.orchestra.run.smithi104.stdout: "object_hash": 2, 2020-09-08T15:42:33.628 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_autoscale_mode": "off", 2020-09-08T15:42:33.628 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num": 1, 2020-09-08T15:42:33.628 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_placement_num": 1, 2020-09-08T15:42:33.628 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_placement_num_target": 1, 2020-09-08T15:42:33.629 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num_target": 1, 2020-09-08T15:42:33.629 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num_pending": 1, 2020-09-08T15:42:33.629 INFO:teuthology.orchestra.run.smithi104.stdout: "last_pg_merge_meta": { 2020-09-08T15:42:33.629 INFO:teuthology.orchestra.run.smithi104.stdout: "source_pgid": "0.0", 2020-09-08T15:42:33.629 INFO:teuthology.orchestra.run.smithi104.stdout: "ready_epoch": 0, 2020-09-08T15:42:33.630 INFO:teuthology.orchestra.run.smithi104.stdout: "last_epoch_started": 0, 2020-09-08T15:42:33.630 INFO:teuthology.orchestra.run.smithi104.stdout: "last_epoch_clean": 0, 2020-09-08T15:42:33.630 INFO:teuthology.orchestra.run.smithi104.stdout: "source_version": "0'0", 2020-09-08T15:42:33.630 INFO:teuthology.orchestra.run.smithi104.stdout: "target_version": "0'0" 2020-09-08T15:42:33.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.631 INFO:teuthology.orchestra.run.smithi104.stdout: "last_change": "13", 2020-09-08T15:42:33.631 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend": "0", 2020-09-08T15:42:33.631 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend_prenautilus": "0", 2020-09-08T15:42:33.631 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend_preluminous": "0", 2020-09-08T15:42:33.631 INFO:teuthology.orchestra.run.smithi104.stdout: "auid": 0, 2020-09-08T15:42:33.631 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_mode": "selfmanaged", 2020-09-08T15:42:33.632 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_seq": 0, 2020-09-08T15:42:33.632 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_epoch": 0, 2020-09-08T15:42:33.632 INFO:teuthology.orchestra.run.smithi104.stdout: "pool_snaps": [], 2020-09-08T15:42:33.632 INFO:teuthology.orchestra.run.smithi104.stdout: "removed_snaps": "[]", 2020-09-08T15:42:33.632 INFO:teuthology.orchestra.run.smithi104.stdout: "quota_max_bytes": 0, 2020-09-08T15:42:33.633 INFO:teuthology.orchestra.run.smithi104.stdout: "quota_max_objects": 0, 2020-09-08T15:42:33.633 INFO:teuthology.orchestra.run.smithi104.stdout: "tiers": [], 2020-09-08T15:42:33.633 INFO:teuthology.orchestra.run.smithi104.stdout: "tier_of": -1, 2020-09-08T15:42:33.633 INFO:teuthology.orchestra.run.smithi104.stdout: "read_tier": -1, 2020-09-08T15:42:33.634 INFO:teuthology.orchestra.run.smithi104.stdout: "write_tier": -1, 2020-09-08T15:42:33.634 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_mode": "none", 2020-09-08T15:42:33.634 INFO:teuthology.orchestra.run.smithi104.stdout: "target_max_bytes": 0, 2020-09-08T15:42:33.634 INFO:teuthology.orchestra.run.smithi104.stdout: "target_max_objects": 0, 2020-09-08T15:42:33.634 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-09-08T15:42:33.634 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-09-08T15:42:33.635 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_full_ratio_micro": 800000, 2020-09-08T15:42:33.635 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_min_flush_age": 0, 2020-09-08T15:42:33.635 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_min_evict_age": 0, 2020-09-08T15:42:33.635 INFO:teuthology.orchestra.run.smithi104.stdout: "erasure_code_profile": "", 2020-09-08T15:42:33.635 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_params": { 2020-09-08T15:42:33.636 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "none" 2020-09-08T15:42:33.636 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.636 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_period": 0, 2020-09-08T15:42:33.636 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_count": 0, 2020-09-08T15:42:33.636 INFO:teuthology.orchestra.run.smithi104.stdout: "use_gmt_hitset": true, 2020-09-08T15:42:33.637 INFO:teuthology.orchestra.run.smithi104.stdout: "min_read_recency_for_promote": 0, 2020-09-08T15:42:33.637 INFO:teuthology.orchestra.run.smithi104.stdout: "min_write_recency_for_promote": 0, 2020-09-08T15:42:33.637 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_grade_decay_rate": 0, 2020-09-08T15:42:33.637 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_search_last_n": 0, 2020-09-08T15:42:33.637 INFO:teuthology.orchestra.run.smithi104.stdout: "grade_table": [], 2020-09-08T15:42:33.638 INFO:teuthology.orchestra.run.smithi104.stdout: "stripe_width": 0, 2020-09-08T15:42:33.638 INFO:teuthology.orchestra.run.smithi104.stdout: "expected_num_objects": 0, 2020-09-08T15:42:33.638 INFO:teuthology.orchestra.run.smithi104.stdout: "fast_read": false, 2020-09-08T15:42:33.638 INFO:teuthology.orchestra.run.smithi104.stdout: "options": { 2020-09-08T15:42:33.638 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num_min": 1 2020-09-08T15:42:33.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.639 INFO:teuthology.orchestra.run.smithi104.stdout: "application_metadata": { 2020-09-08T15:42:33.639 INFO:teuthology.orchestra.run.smithi104.stdout: "mgr_devicehealth": {} 2020-09-08T15:42:33.640 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.640 INFO:teuthology.orchestra.run.smithi104.stdout: "pool": 3, 2020-09-08T15:42:33.641 INFO:teuthology.orchestra.run.smithi104.stdout: "pool_name": "unique_pool_1", 2020-09-08T15:42:33.641 INFO:teuthology.orchestra.run.smithi104.stdout: "create_time": "2020-09-08T15:42:05.902210+0000", 2020-09-08T15:42:33.641 INFO:teuthology.orchestra.run.smithi104.stdout: "flags": 1, 2020-09-08T15:42:33.641 INFO:teuthology.orchestra.run.smithi104.stdout: "flags_names": "hashpspool", 2020-09-08T15:42:33.641 INFO:teuthology.orchestra.run.smithi104.stdout: "type": 1, 2020-09-08T15:42:33.642 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 2, 2020-09-08T15:42:33.642 INFO:teuthology.orchestra.run.smithi104.stdout: "min_size": 2, 2020-09-08T15:42:33.642 INFO:teuthology.orchestra.run.smithi104.stdout: "crush_rule": 0, 2020-09-08T15:42:33.642 INFO:teuthology.orchestra.run.smithi104.stdout: "object_hash": 2, 2020-09-08T15:42:33.642 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_autoscale_mode": "off", 2020-09-08T15:42:33.643 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num": 16, 2020-09-08T15:42:33.643 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_placement_num": 16, 2020-09-08T15:42:33.643 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_placement_num_target": 16, 2020-09-08T15:42:33.643 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num_target": 16, 2020-09-08T15:42:33.644 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_num_pending": 16, 2020-09-08T15:42:33.644 INFO:teuthology.orchestra.run.smithi104.stdout: "last_pg_merge_meta": { 2020-09-08T15:42:33.644 INFO:teuthology.orchestra.run.smithi104.stdout: "source_pgid": "0.0", 2020-09-08T15:42:33.644 INFO:teuthology.orchestra.run.smithi104.stdout: "ready_epoch": 0, 2020-09-08T15:42:33.645 INFO:teuthology.orchestra.run.smithi104.stdout: "last_epoch_started": 0, 2020-09-08T15:42:33.645 INFO:teuthology.orchestra.run.smithi104.stdout: "last_epoch_clean": 0, 2020-09-08T15:42:33.645 INFO:teuthology.orchestra.run.smithi104.stdout: "source_version": "0'0", 2020-09-08T15:42:33.645 INFO:teuthology.orchestra.run.smithi104.stdout: "target_version": "0'0" 2020-09-08T15:42:33.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.646 INFO:teuthology.orchestra.run.smithi104.stdout: "last_change": "219", 2020-09-08T15:42:33.646 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend": "0", 2020-09-08T15:42:33.646 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend_prenautilus": "0", 2020-09-08T15:42:33.646 INFO:teuthology.orchestra.run.smithi104.stdout: "last_force_op_resend_preluminous": "0", 2020-09-08T15:42:33.646 INFO:teuthology.orchestra.run.smithi104.stdout: "auid": 0, 2020-09-08T15:42:33.646 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_mode": "selfmanaged", 2020-09-08T15:42:33.647 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_seq": 0, 2020-09-08T15:42:33.647 INFO:teuthology.orchestra.run.smithi104.stdout: "snap_epoch": 0, 2020-09-08T15:42:33.647 INFO:teuthology.orchestra.run.smithi104.stdout: "pool_snaps": [], 2020-09-08T15:42:33.647 INFO:teuthology.orchestra.run.smithi104.stdout: "removed_snaps": "[]", 2020-09-08T15:42:33.647 INFO:teuthology.orchestra.run.smithi104.stdout: "quota_max_bytes": 0, 2020-09-08T15:42:33.648 INFO:teuthology.orchestra.run.smithi104.stdout: "quota_max_objects": 0, 2020-09-08T15:42:33.648 INFO:teuthology.orchestra.run.smithi104.stdout: "tiers": [], 2020-09-08T15:42:33.648 INFO:teuthology.orchestra.run.smithi104.stdout: "tier_of": -1, 2020-09-08T15:42:33.650 INFO:teuthology.orchestra.run.smithi104.stdout: "read_tier": -1, 2020-09-08T15:42:33.650 INFO:teuthology.orchestra.run.smithi104.stdout: "write_tier": -1, 2020-09-08T15:42:33.650 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_mode": "none", 2020-09-08T15:42:33.650 INFO:teuthology.orchestra.run.smithi104.stdout: "target_max_bytes": 0, 2020-09-08T15:42:33.650 INFO:teuthology.orchestra.run.smithi104.stdout: "target_max_objects": 0, 2020-09-08T15:42:33.651 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-09-08T15:42:33.651 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-09-08T15:42:33.651 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_target_full_ratio_micro": 800000, 2020-09-08T15:42:33.651 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_min_flush_age": 0, 2020-09-08T15:42:33.651 INFO:teuthology.orchestra.run.smithi104.stdout: "cache_min_evict_age": 0, 2020-09-08T15:42:33.651 INFO:teuthology.orchestra.run.smithi104.stdout: "erasure_code_profile": "", 2020-09-08T15:42:33.652 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_params": { 2020-09-08T15:42:33.652 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "none" 2020-09-08T15:42:33.652 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.652 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_period": 0, 2020-09-08T15:42:33.652 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_count": 0, 2020-09-08T15:42:33.653 INFO:teuthology.orchestra.run.smithi104.stdout: "use_gmt_hitset": true, 2020-09-08T15:42:33.653 INFO:teuthology.orchestra.run.smithi104.stdout: "min_read_recency_for_promote": 0, 2020-09-08T15:42:33.653 INFO:teuthology.orchestra.run.smithi104.stdout: "min_write_recency_for_promote": 0, 2020-09-08T15:42:33.653 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_grade_decay_rate": 0, 2020-09-08T15:42:33.653 INFO:teuthology.orchestra.run.smithi104.stdout: "hit_set_search_last_n": 0, 2020-09-08T15:42:33.654 INFO:teuthology.orchestra.run.smithi104.stdout: "grade_table": [], 2020-09-08T15:42:33.654 INFO:teuthology.orchestra.run.smithi104.stdout: "stripe_width": 0, 2020-09-08T15:42:33.654 INFO:teuthology.orchestra.run.smithi104.stdout: "expected_num_objects": 0, 2020-09-08T15:42:33.654 INFO:teuthology.orchestra.run.smithi104.stdout: "fast_read": false, 2020-09-08T15:42:33.654 INFO:teuthology.orchestra.run.smithi104.stdout: "options": {}, 2020-09-08T15:42:33.655 INFO:teuthology.orchestra.run.smithi104.stdout: "application_metadata": { 2020-09-08T15:42:33.655 INFO:teuthology.orchestra.run.smithi104.stdout: "rados": {} 2020-09-08T15:42:33.655 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.655 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.655 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T15:42:33.656 INFO:teuthology.orchestra.run.smithi104.stdout: "osds": [ 2020-09-08T15:42:33.656 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.656 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 0, 2020-09-08T15:42:33.656 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "e49a7341-7c75-4883-b061-3e8fe9467e6e", 2020-09-08T15:42:33.656 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:42:33.657 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:42:33.657 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:42:33.657 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:42:33.657 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:42:33.658 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:42:33.658 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 11, 2020-09-08T15:42:33.658 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 217, 2020-09-08T15:42:33.659 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:42:33.659 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:42:33.659 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:42:33.659 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.659 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.660 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.660 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6801", 2020-09-08T15:42:33.660 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:42:33.660 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.660 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.661 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.661 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:42:33.661 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.661 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.662 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.662 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6802", 2020-09-08T15:42:33.662 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:42:33.662 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.664 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.664 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.665 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6803", 2020-09-08T15:42:33.665 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:42:33.665 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.666 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.666 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.666 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:42:33.666 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.667 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.667 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.668 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6806", 2020-09-08T15:42:33.668 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:42:33.668 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.668 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.669 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.669 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6807", 2020-09-08T15:42:33.669 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:42:33.670 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.670 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.670 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.670 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:42:33.671 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.671 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.671 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.671 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6804", 2020-09-08T15:42:33.671 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:42:33.672 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.672 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.672 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.672 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6805", 2020-09-08T15:42:33.673 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 82470352 2020-09-08T15:42:33.673 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.673 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.673 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.673 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.104:6801/82470352", 2020-09-08T15:42:33.674 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.104:6803/82470352", 2020-09-08T15:42:33.674 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.104:6807/82470352", 2020-09-08T15:42:33.675 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.104:6805/82470352", 2020-09-08T15:42:33.675 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:42:33.675 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:42:33.675 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:42:33.676 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.676 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.677 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.677 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 1, 2020-09-08T15:42:33.677 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "a83df412-110d-440a-b411-ad2a7c36f81a", 2020-09-08T15:42:33.678 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:42:33.678 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 0, 2020-09-08T15:42:33.678 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 0, 2020-09-08T15:42:33.679 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:42:33.679 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:42:33.679 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:42:33.680 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 17, 2020-09-08T15:42:33.680 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 214, 2020-09-08T15:42:33.680 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:42:33.681 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:42:33.681 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:42:33.681 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.681 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.682 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.682 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6809", 2020-09-08T15:42:33.682 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:42:33.682 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.682 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.683 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.683 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:42:33.683 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.683 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.683 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.684 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6810", 2020-09-08T15:42:33.684 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:42:33.685 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.685 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.685 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.685 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6811", 2020-09-08T15:42:33.686 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:42:33.686 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.686 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.686 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.686 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:42:33.687 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.687 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.687 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.687 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6814", 2020-09-08T15:42:33.687 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:42:33.688 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.688 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.688 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.688 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6815", 2020-09-08T15:42:33.689 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:42:33.689 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.689 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.689 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.689 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:42:33.690 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.690 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.690 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.690 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6812", 2020-09-08T15:42:33.690 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:42:33.691 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.691 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.691 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.691 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6813", 2020-09-08T15:42:33.692 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1421798992 2020-09-08T15:42:33.692 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.692 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.692 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.692 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.104:6809/1421798992", 2020-09-08T15:42:33.693 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.104:6811/1421798992", 2020-09-08T15:42:33.693 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.104:6815/1421798992", 2020-09-08T15:42:33.693 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.104:6813/1421798992", 2020-09-08T15:42:33.693 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:42:33.694 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:42:33.694 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:42:33.694 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.694 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.694 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.695 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 2, 2020-09-08T15:42:33.695 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "ca6da1cc-1d8d-494b-93e6-e065ff70d434", 2020-09-08T15:42:33.696 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:42:33.696 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:42:33.696 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:42:33.696 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:42:33.696 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:42:33.697 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:42:33.697 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 23, 2020-09-08T15:42:33.697 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 126, 2020-09-08T15:42:33.697 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:42:33.698 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:42:33.698 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:42:33.698 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.698 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.698 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.699 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6817", 2020-09-08T15:42:33.699 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:42:33.699 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.699 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.700 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.700 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:42:33.700 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.700 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.700 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.701 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6818", 2020-09-08T15:42:33.701 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:42:33.701 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.701 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.702 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.702 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6819", 2020-09-08T15:42:33.702 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:42:33.702 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.703 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.703 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.703 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:42:33.703 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.704 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.704 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.704 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6822", 2020-09-08T15:42:33.704 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:42:33.704 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.705 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.705 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.706 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6823", 2020-09-08T15:42:33.706 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:42:33.706 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.706 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.707 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.707 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:42:33.707 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.707 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.707 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.708 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6820", 2020-09-08T15:42:33.708 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:42:33.708 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.708 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.708 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.709 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6821", 2020-09-08T15:42:33.709 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2713413369 2020-09-08T15:42:33.709 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.709 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.710 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.710 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.104:6817/2713413369", 2020-09-08T15:42:33.710 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.104:6819/2713413369", 2020-09-08T15:42:33.710 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.104:6823/2713413369", 2020-09-08T15:42:33.710 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.104:6821/2713413369", 2020-09-08T15:42:33.711 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:42:33.714 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:42:33.714 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:42:33.714 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.715 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.715 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.715 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 3, 2020-09-08T15:42:33.715 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e", 2020-09-08T15:42:33.716 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:42:33.716 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:42:33.717 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:42:33.717 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:42:33.717 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:42:33.717 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:42:33.717 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 29, 2020-09-08T15:42:33.718 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 214, 2020-09-08T15:42:33.718 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:42:33.718 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:42:33.718 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:42:33.719 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.719 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.719 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.720 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6825", 2020-09-08T15:42:33.720 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:42:33.720 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.720 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.721 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.721 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:42:33.722 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.722 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.723 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.723 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6826", 2020-09-08T15:42:33.723 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:42:33.724 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.724 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.724 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.724 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6827", 2020-09-08T15:42:33.724 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:42:33.725 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.725 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.725 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.725 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:42:33.725 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.726 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.726 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.726 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6830", 2020-09-08T15:42:33.726 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:42:33.727 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.727 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.727 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.727 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6831", 2020-09-08T15:42:33.727 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:42:33.727 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.728 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.728 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.728 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:42:33.728 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.728 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.729 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.729 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6828", 2020-09-08T15:42:33.729 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:42:33.729 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.729 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.730 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.730 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.104:6829", 2020-09-08T15:42:33.730 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1495946248 2020-09-08T15:42:33.730 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.730 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.731 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.731 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.104:6825/1495946248", 2020-09-08T15:42:33.731 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.104:6827/1495946248", 2020-09-08T15:42:33.731 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.104:6831/1495946248", 2020-09-08T15:42:33.731 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.104:6829/1495946248", 2020-09-08T15:42:33.732 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:42:33.732 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:42:33.732 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:42:33.732 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.732 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.733 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.733 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 4, 2020-09-08T15:42:33.733 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "d8334c70-f776-4652-944d-aed35a8e43f7", 2020-09-08T15:42:33.733 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:42:33.733 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:42:33.733 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:42:33.734 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 0, 2020-09-08T15:42:33.734 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:42:33.734 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:42:33.734 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 35, 2020-09-08T15:42:33.734 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 0, 2020-09-08T15:42:33.735 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:42:33.735 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:42:33.735 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:42:33.735 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.735 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.736 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.736 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6803", 2020-09-08T15:42:33.736 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:42:33.736 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.736 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.736 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.737 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:42:33.737 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.737 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.737 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.737 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6804", 2020-09-08T15:42:33.738 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:42:33.738 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.738 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.738 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.738 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6805", 2020-09-08T15:42:33.739 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:42:33.739 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.739 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.740 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.740 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:42:33.740 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.740 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.741 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.741 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6808", 2020-09-08T15:42:33.741 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:42:33.741 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.741 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.742 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.742 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6809", 2020-09-08T15:42:33.742 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:42:33.742 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.742 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.743 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.743 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:42:33.743 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.743 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.744 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.744 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6806", 2020-09-08T15:42:33.744 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:42:33.744 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.744 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.744 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.745 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6807", 2020-09-08T15:42:33.745 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1173103260 2020-09-08T15:42:33.745 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.745 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.745 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.746 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.55:6803/1173103260", 2020-09-08T15:42:33.746 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.55:6805/1173103260", 2020-09-08T15:42:33.746 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.55:6809/1173103260", 2020-09-08T15:42:33.746 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.55:6807/1173103260", 2020-09-08T15:42:33.746 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:42:33.747 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:42:33.747 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:42:33.747 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.748 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.748 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.748 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 5, 2020-09-08T15:42:33.748 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "6789abd2-9e9b-4c04-81fb-0ce2d2849e39", 2020-09-08T15:42:33.748 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:42:33.749 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:42:33.749 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:42:33.749 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:42:33.749 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:42:33.749 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:42:33.749 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 42, 2020-09-08T15:42:33.750 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 214, 2020-09-08T15:42:33.750 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:42:33.750 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:42:33.750 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:42:33.750 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.751 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.751 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.751 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6811", 2020-09-08T15:42:33.751 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:42:33.751 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.752 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.752 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.752 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:42:33.752 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.752 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.752 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.753 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6812", 2020-09-08T15:42:33.753 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:42:33.753 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.753 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.753 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.754 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6813", 2020-09-08T15:42:33.754 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:42:33.754 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.754 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.754 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.755 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:42:33.755 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.755 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.755 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.755 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6816", 2020-09-08T15:42:33.756 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:42:33.756 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.756 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.756 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.756 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6817", 2020-09-08T15:42:33.757 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:42:33.757 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.757 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.757 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.757 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:42:33.757 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.758 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.758 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.758 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6814", 2020-09-08T15:42:33.758 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:42:33.758 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.759 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.759 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.759 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6815", 2020-09-08T15:42:33.759 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1047046376 2020-09-08T15:42:33.759 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.759 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.760 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.760 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.55:6811/1047046376", 2020-09-08T15:42:33.760 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.55:6813/1047046376", 2020-09-08T15:42:33.760 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.55:6817/1047046376", 2020-09-08T15:42:33.760 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.55:6815/1047046376", 2020-09-08T15:42:33.761 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:42:33.761 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:42:33.761 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:42:33.761 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.761 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.762 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.762 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 6, 2020-09-08T15:42:33.762 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "ad9a93fe-0ca0-4052-bfa9-e2abb268829e", 2020-09-08T15:42:33.762 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:42:33.762 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:42:33.763 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:42:33.763 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:42:33.763 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:42:33.763 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:42:33.763 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 49, 2020-09-08T15:42:33.763 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 214, 2020-09-08T15:42:33.764 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:42:33.764 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:42:33.764 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:42:33.764 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.764 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.765 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.765 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6819", 2020-09-08T15:42:33.765 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:42:33.765 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.765 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.766 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.766 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:42:33.766 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.766 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.766 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.766 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6820", 2020-09-08T15:42:33.767 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:42:33.767 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.767 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.767 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.767 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6821", 2020-09-08T15:42:33.768 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:42:33.768 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.768 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.768 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.768 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:42:33.768 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.769 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.769 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.769 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6824", 2020-09-08T15:42:33.769 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:42:33.769 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.770 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.770 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.770 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6825", 2020-09-08T15:42:33.770 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:42:33.770 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.771 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.771 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.771 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:42:33.771 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.771 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.772 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.772 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6822", 2020-09-08T15:42:33.772 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:42:33.772 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.772 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.772 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.773 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6823", 2020-09-08T15:42:33.773 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2501844793 2020-09-08T15:42:33.773 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.773 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.773 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.774 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.55:6819/2501844793", 2020-09-08T15:42:33.774 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.55:6821/2501844793", 2020-09-08T15:42:33.774 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.55:6825/2501844793", 2020-09-08T15:42:33.774 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.55:6823/2501844793", 2020-09-08T15:42:33.774 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:42:33.775 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:42:33.775 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:42:33.775 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.775 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.776 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.776 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 7, 2020-09-08T15:42:33.776 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "f3795c86-4540-442c-a151-b2769fa577e4", 2020-09-08T15:42:33.776 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:42:33.776 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:42:33.777 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:42:33.777 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:42:33.777 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:42:33.777 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:42:33.777 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 56, 2020-09-08T15:42:33.778 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 214, 2020-09-08T15:42:33.778 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:42:33.778 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:42:33.778 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:42:33.778 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.778 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.779 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.779 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6827", 2020-09-08T15:42:33.779 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:42:33.779 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.779 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.780 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.780 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:42:33.780 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.780 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.780 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.781 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6828", 2020-09-08T15:42:33.781 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:42:33.781 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.781 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.781 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.781 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6829", 2020-09-08T15:42:33.782 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:42:33.782 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.782 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.782 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.782 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:42:33.783 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.783 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.783 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.783 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6832", 2020-09-08T15:42:33.783 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:42:33.784 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.784 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.784 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.784 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6833", 2020-09-08T15:42:33.784 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:42:33.784 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.785 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.785 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.785 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:42:33.785 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.785 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.786 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.786 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6830", 2020-09-08T15:42:33.786 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:42:33.786 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.786 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.787 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.787 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.55:6831", 2020-09-08T15:42:33.787 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 128909153 2020-09-08T15:42:33.787 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.787 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.787 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.788 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.55:6827/128909153", 2020-09-08T15:42:33.788 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.55:6829/128909153", 2020-09-08T15:42:33.788 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.55:6833/128909153", 2020-09-08T15:42:33.788 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.55:6831/128909153", 2020-09-08T15:42:33.788 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:42:33.789 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:42:33.789 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:42:33.789 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.789 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.789 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.790 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 8, 2020-09-08T15:42:33.790 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "3b0ac380-878e-4c1c-9278-80ce67ca2df4", 2020-09-08T15:42:33.790 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:42:33.790 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:42:33.790 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:42:33.791 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:42:33.791 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:42:33.791 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:42:33.791 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 64, 2020-09-08T15:42:33.791 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 214, 2020-09-08T15:42:33.792 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:42:33.792 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:42:33.792 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:42:33.792 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.792 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.792 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.793 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6801", 2020-09-08T15:42:33.793 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:42:33.793 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.793 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.793 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.794 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:42:33.794 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.794 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.794 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.794 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6802", 2020-09-08T15:42:33.795 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:42:33.795 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.795 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.795 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.795 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6803", 2020-09-08T15:42:33.795 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:42:33.796 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.796 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.796 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.796 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:42:33.796 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.797 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.797 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.797 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6806", 2020-09-08T15:42:33.797 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:42:33.797 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.798 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.798 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.798 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6807", 2020-09-08T15:42:33.798 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:42:33.798 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.798 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.799 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.799 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:42:33.799 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.799 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.799 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.800 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6804", 2020-09-08T15:42:33.800 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:42:33.800 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.800 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.800 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.801 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6805", 2020-09-08T15:42:33.801 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 2108117574 2020-09-08T15:42:33.801 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.801 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.801 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.801 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.166:6801/2108117574", 2020-09-08T15:42:33.802 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.166:6803/2108117574", 2020-09-08T15:42:33.802 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.166:6807/2108117574", 2020-09-08T15:42:33.802 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.166:6805/2108117574", 2020-09-08T15:42:33.802 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:42:33.802 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:42:33.803 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:42:33.803 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.803 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.803 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.803 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 9, 2020-09-08T15:42:33.804 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08", 2020-09-08T15:42:33.804 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:42:33.804 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:42:33.804 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:42:33.804 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:42:33.804 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:42:33.805 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:42:33.805 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 72, 2020-09-08T15:42:33.805 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 126, 2020-09-08T15:42:33.805 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:42:33.805 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:42:33.806 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:42:33.806 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.806 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.806 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.806 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6809", 2020-09-08T15:42:33.807 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:42:33.807 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.807 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.807 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.807 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:42:33.808 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.808 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.808 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.808 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6810", 2020-09-08T15:42:33.808 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:42:33.808 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.809 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.809 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.809 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6811", 2020-09-08T15:42:33.809 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:42:33.809 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.810 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.810 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.810 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:42:33.810 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.810 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.811 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.811 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6814", 2020-09-08T15:42:33.811 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:42:33.811 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.811 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.811 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.812 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6815", 2020-09-08T15:42:33.812 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:42:33.812 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.812 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.812 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.813 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:42:33.813 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.813 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.813 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.813 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6812", 2020-09-08T15:42:33.814 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:42:33.814 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.814 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.814 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.814 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6813", 2020-09-08T15:42:33.814 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 1458870080 2020-09-08T15:42:33.815 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.815 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.815 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.815 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.166:6809/1458870080", 2020-09-08T15:42:33.816 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.166:6811/1458870080", 2020-09-08T15:42:33.816 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.166:6815/1458870080", 2020-09-08T15:42:33.816 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.166:6813/1458870080", 2020-09-08T15:42:33.816 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:42:33.816 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:42:33.817 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:42:33.817 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.817 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.817 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.817 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 10, 2020-09-08T15:42:33.818 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "c194210e-bd89-4a1f-98fb-6bf44f0b324c", 2020-09-08T15:42:33.818 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:42:33.818 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:42:33.818 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:42:33.818 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:42:33.819 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:42:33.819 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:42:33.819 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 81, 2020-09-08T15:42:33.819 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 0, 2020-09-08T15:42:33.820 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:42:33.820 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:42:33.820 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:42:33.820 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.820 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.821 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.821 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6817", 2020-09-08T15:42:33.821 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:42:33.821 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.821 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.822 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.822 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:42:33.822 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.822 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.822 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.823 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6818", 2020-09-08T15:42:33.823 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:42:33.823 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.823 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.823 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.824 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6819", 2020-09-08T15:42:33.824 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:42:33.824 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.824 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.824 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.825 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:42:33.825 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.825 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.825 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.825 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6822", 2020-09-08T15:42:33.826 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:42:33.826 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.826 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.826 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.826 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6823", 2020-09-08T15:42:33.827 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:42:33.827 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.827 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.827 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.827 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:42:33.828 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.828 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.828 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.828 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6820", 2020-09-08T15:42:33.828 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:42:33.828 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.829 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.829 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.829 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6821", 2020-09-08T15:42:33.829 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 62517848 2020-09-08T15:42:33.829 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.830 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.830 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.830 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.166:6817/62517848", 2020-09-08T15:42:33.830 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.166:6819/62517848", 2020-09-08T15:42:33.830 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.166:6823/62517848", 2020-09-08T15:42:33.831 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.166:6821/62517848", 2020-09-08T15:42:33.831 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:42:33.831 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:42:33.831 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:42:33.831 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.831 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.832 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.833 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 11, 2020-09-08T15:42:33.833 INFO:teuthology.orchestra.run.smithi104.stdout: "uuid": "121301fe-a41c-40c0-ad09-b00de37a55e1", 2020-09-08T15:42:33.834 INFO:teuthology.orchestra.run.smithi104.stdout: "up": 1, 2020-09-08T15:42:33.834 INFO:teuthology.orchestra.run.smithi104.stdout: "in": 1, 2020-09-08T15:42:33.834 INFO:teuthology.orchestra.run.smithi104.stdout: "weight": 1, 2020-09-08T15:42:33.834 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_affinity": 1, 2020-09-08T15:42:33.835 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_begin": 0, 2020-09-08T15:42:33.835 INFO:teuthology.orchestra.run.smithi104.stdout: "last_clean_end": 0, 2020-09-08T15:42:33.835 INFO:teuthology.orchestra.run.smithi104.stdout: "up_from": 90, 2020-09-08T15:42:33.835 INFO:teuthology.orchestra.run.smithi104.stdout: "up_thru": 217, 2020-09-08T15:42:33.835 INFO:teuthology.orchestra.run.smithi104.stdout: "down_at": 0, 2020-09-08T15:42:33.836 INFO:teuthology.orchestra.run.smithi104.stdout: "lost_at": 0, 2020-09-08T15:42:33.836 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addrs": { 2020-09-08T15:42:33.836 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.836 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.837 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.837 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6825", 2020-09-08T15:42:33.837 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:42:33.837 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.838 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.838 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.838 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addrs": { 2020-09-08T15:42:33.838 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.839 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.839 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.839 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6826", 2020-09-08T15:42:33.840 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:42:33.841 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.841 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.841 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.841 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6827", 2020-09-08T15:42:33.842 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:42:33.842 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.842 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.842 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.842 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addrs": { 2020-09-08T15:42:33.843 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.843 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.843 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.843 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6830", 2020-09-08T15:42:33.843 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:42:33.844 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.844 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.844 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.844 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6831", 2020-09-08T15:42:33.844 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:42:33.845 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.845 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.845 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.845 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addrs": { 2020-09-08T15:42:33.846 INFO:teuthology.orchestra.run.smithi104.stdout: "addrvec": [ 2020-09-08T15:42:33.846 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.846 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v2", 2020-09-08T15:42:33.846 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6828", 2020-09-08T15:42:33.847 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:42:33.847 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.847 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.847 INFO:teuthology.orchestra.run.smithi104.stdout: "type": "v1", 2020-09-08T15:42:33.847 INFO:teuthology.orchestra.run.smithi104.stdout: "addr": "172.21.15.166:6829", 2020-09-08T15:42:33.848 INFO:teuthology.orchestra.run.smithi104.stdout: "nonce": 691735125 2020-09-08T15:42:33.848 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.848 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.848 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.848 INFO:teuthology.orchestra.run.smithi104.stdout: "public_addr": "172.21.15.166:6825/691735125", 2020-09-08T15:42:33.849 INFO:teuthology.orchestra.run.smithi104.stdout: "cluster_addr": "172.21.15.166:6827/691735125", 2020-09-08T15:42:33.849 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_back_addr": "172.21.15.166:6831/691735125", 2020-09-08T15:42:33.849 INFO:teuthology.orchestra.run.smithi104.stdout: "heartbeat_front_addr": "172.21.15.166:6829/691735125", 2020-09-08T15:42:33.849 INFO:teuthology.orchestra.run.smithi104.stdout: "state": [ 2020-09-08T15:42:33.849 INFO:teuthology.orchestra.run.smithi104.stdout: "exists", 2020-09-08T15:42:33.850 INFO:teuthology.orchestra.run.smithi104.stdout: "up" 2020-09-08T15:42:33.850 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:42:33.850 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.850 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T15:42:33.850 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_xinfo": [ 2020-09-08T15:42:33.851 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.851 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 0, 2020-09-08T15:42:33.851 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:42:33.851 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:42:33.851 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:42:33.852 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:42:33.852 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:42:33.852 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:42:06.994157+0000", 2020-09-08T15:42:33.852 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:42:33.852 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.852 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.853 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 1, 2020-09-08T15:42:33.853 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:42:33.853 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:42:33.853 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:42:33.853 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:42:33.854 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 65536, 2020-09-08T15:42:33.854 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:41:34.305299+0000", 2020-09-08T15:42:33.854 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:42:33.854 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.854 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.855 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 2, 2020-09-08T15:42:33.855 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:42:33.855 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:42:33.855 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:42:33.855 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:42:33.856 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:42:33.856 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:41:57.794699+0000", 2020-09-08T15:42:33.856 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:42:33.856 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.856 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.856 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 3, 2020-09-08T15:42:33.857 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:42:33.857 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:42:33.857 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:42:33.857 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:42:33.857 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:42:33.858 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:42:22.114232+0000", 2020-09-08T15:42:33.858 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:42:33.858 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.858 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.858 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 4, 2020-09-08T15:42:33.859 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:42:33.859 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:42:33.859 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:42:33.860 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:42:33.860 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:42:33.861 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:41:46.565606+0000", 2020-09-08T15:42:33.861 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:42:33.861 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.861 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.861 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 5, 2020-09-08T15:42:33.862 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:42:33.862 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:42:33.862 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:42:33.862 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:42:33.862 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:42:33.863 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:42:13.283099+0000", 2020-09-08T15:42:33.863 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:42:33.863 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.863 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.863 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 6, 2020-09-08T15:42:33.864 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:42:33.864 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:42:33.864 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:42:33.864 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:42:33.864 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:42:33.865 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:41:40.422586+0000", 2020-09-08T15:42:33.865 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:42:33.865 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.865 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.865 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 7, 2020-09-08T15:42:33.866 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:42:33.866 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:42:33.866 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:42:33.866 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:42:33.866 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:42:33.867 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:42:06.536241+0000", 2020-09-08T15:42:33.867 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:42:33.867 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.867 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.867 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 8, 2020-09-08T15:42:33.868 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:42:33.868 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:42:33.868 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:42:33.868 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:42:33.868 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:42:33.868 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:42:28.387217+0000", 2020-09-08T15:42:33.869 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:42:33.869 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.869 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.869 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 9, 2020-09-08T15:42:33.869 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:42:33.870 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:42:33.870 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:42:33.870 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:42:33.870 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:42:33.870 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:41:55.192154+0000", 2020-09-08T15:42:33.871 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:42:33.871 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.871 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.871 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 10, 2020-09-08T15:42:33.871 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:42:33.872 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:42:33.872 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:42:33.872 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:42:33.872 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:42:33.872 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:42:18.419489+0000", 2020-09-08T15:42:33.872 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:42:33.873 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.873 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:42:33.873 INFO:teuthology.orchestra.run.smithi104.stdout: "osd": 11, 2020-09-08T15:42:33.873 INFO:teuthology.orchestra.run.smithi104.stdout: "down_stamp": "0.000000", 2020-09-08T15:42:33.873 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_probability": 0, 2020-09-08T15:42:33.874 INFO:teuthology.orchestra.run.smithi104.stdout: "laggy_interval": 0, 2020-09-08T15:42:33.874 INFO:teuthology.orchestra.run.smithi104.stdout: "features": 4540138292837744639, 2020-09-08T15:42:33.874 INFO:teuthology.orchestra.run.smithi104.stdout: "old_weight": 0, 2020-09-08T15:42:33.874 INFO:teuthology.orchestra.run.smithi104.stdout: "last_purged_snaps_scrub": "2020-09-08T15:41:42.473673+0000", 2020-09-08T15:42:33.874 INFO:teuthology.orchestra.run.smithi104.stdout: "dead_epoch": 0 2020-09-08T15:42:33.875 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.875 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T15:42:33.875 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_upmap": [], 2020-09-08T15:42:33.876 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_upmap_items": [], 2020-09-08T15:42:33.876 INFO:teuthology.orchestra.run.smithi104.stdout: "pg_temp": [], 2020-09-08T15:42:33.876 INFO:teuthology.orchestra.run.smithi104.stdout: "primary_temp": [], 2020-09-08T15:42:33.876 INFO:teuthology.orchestra.run.smithi104.stdout: "blocklist": { 2020-09-08T15:42:33.877 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/1257365091": "2020-09-09T15:29:39.430456+0000", 2020-09-08T15:42:33.877 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/1399123455": "2020-09-09T15:29:39.430456+0000", 2020-09-08T15:42:33.877 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/2810441605": "2020-09-09T15:29:39.430456+0000", 2020-09-08T15:42:33.877 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6800/182241824": "2020-09-09T15:29:39.430456+0000", 2020-09-08T15:42:33.877 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6801/182241824": "2020-09-09T15:29:39.430456+0000", 2020-09-08T15:42:33.878 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/2021933044": "2020-09-09T15:26:33.248197+0000", 2020-09-08T15:42:33.878 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6800/3129754347": "2020-09-09T15:26:33.248197+0000", 2020-09-08T15:42:33.878 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/1270031705": "2020-09-09T15:26:33.248197+0000", 2020-09-08T15:42:33.878 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/2568432190": "2020-09-09T15:26:33.248197+0000", 2020-09-08T15:42:33.878 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6800/1138921844": "2020-09-09T15:26:09.867236+0000", 2020-09-08T15:42:33.879 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/3827328474": "2020-09-09T15:26:09.867236+0000", 2020-09-08T15:42:33.879 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6801/1138921844": "2020-09-09T15:26:09.867236+0000", 2020-09-08T15:42:33.879 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/1662035381": "2020-09-09T15:26:09.867236+0000", 2020-09-08T15:42:33.879 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:6801/3129754347": "2020-09-09T15:26:33.248197+0000", 2020-09-08T15:42:33.879 INFO:teuthology.orchestra.run.smithi104.stdout: "172.21.15.104:0/2169968102": "2020-09-09T15:26:09.867236+0000" 2020-09-08T15:42:33.880 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.880 INFO:teuthology.orchestra.run.smithi104.stdout: "erasure_code_profiles": { 2020-09-08T15:42:33.880 INFO:teuthology.orchestra.run.smithi104.stdout: "default": { 2020-09-08T15:42:33.880 INFO:teuthology.orchestra.run.smithi104.stdout: "crush-failure-domain": "osd", 2020-09-08T15:42:33.880 INFO:teuthology.orchestra.run.smithi104.stdout: "k": "2", 2020-09-08T15:42:33.881 INFO:teuthology.orchestra.run.smithi104.stdout: "m": "1", 2020-09-08T15:42:33.881 INFO:teuthology.orchestra.run.smithi104.stdout: "plugin": "jerasure", 2020-09-08T15:42:33.881 INFO:teuthology.orchestra.run.smithi104.stdout: "ruleset-failure-domain": "osd", 2020-09-08T15:42:33.881 INFO:teuthology.orchestra.run.smithi104.stdout: "technique": "reed_sol_van" 2020-09-08T15:42:33.881 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:42:33.882 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:42:33.882 INFO:teuthology.orchestra.run.smithi104.stdout: "removed_snaps_queue": [], 2020-09-08T15:42:33.882 INFO:teuthology.orchestra.run.smithi104.stdout: "new_removed_snaps": [], 2020-09-08T15:42:33.882 INFO:teuthology.orchestra.run.smithi104.stdout: "new_purged_snaps": [], 2020-09-08T15:42:33.882 INFO:teuthology.orchestra.run.smithi104.stdout: "crush_node_flags": {}, 2020-09-08T15:42:33.883 INFO:teuthology.orchestra.run.smithi104.stdout: "device_class_flags": {} 2020-09-08T15:42:33.883 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:42:33.884 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:42:34.035 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:42:34.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:33 smithi166 bash[9446]: audit 2020-09-08T15:42:33.552802+0000 mon.b (mon.1) 84 : audit [DBG] from='client.? v1:172.21.15.104:0/3353922390' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2020-09-08T15:42:34.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:33 smithi055 bash[9296]: audit 2020-09-08T15:42:33.552802+0000 mon.b (mon.1) 84 : audit [DBG] from='client.? v1:172.21.15.104:0/3353922390' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2020-09-08T15:42:34.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:33 smithi104 bash[9203]: audit 2020-09-08T15:42:33.552802+0000 mon.b (mon.1) 84 : audit [DBG] from='client.? v1:172.21.15.104:0/3353922390' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2020-09-08T15:42:34.215 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 14 16 38206 38190 170.468 111.625 0.00401702 0.00584205 2020-09-08T15:42:34.219 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:34.324 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:34.615 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:42:34 smithi166 bash[18960]: debug 2020-09-08T15:42:34.218+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:35.215 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 15 16 40677 40661 169.399 154.438 0.00684324 0.00590055 2020-09-08T15:42:35.240 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:35.343 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:35.609 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:35.648 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:42:35 smithi104 bash[22305]: debug 2020-09-08T15:42:35.244+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:35.711 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:35.853 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:35.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:35 smithi055 bash[9296]: cluster 2020-09-08T15:42:33.670600+0000 mgr.x (mgr.24118) 455 : cluster [DBG] pgmap v596: 17 pgs: 17 active+clean; 2.0 GiB data, 4.5 GiB used, 979 GiB / 983 GiB avail; 203 MiB/s wr, 3.24k op/s 2020-09-08T15:42:35.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:35 smithi104 bash[9203]: cluster 2020-09-08T15:42:33.670600+0000 mgr.x (mgr.24118) 455 : cluster [DBG] pgmap v596: 17 pgs: 17 active+clean; 2.0 GiB data, 4.5 GiB used, 979 GiB / 983 GiB avail; 203 MiB/s wr, 3.24k op/s 2020-09-08T15:42:35.930 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:42:35 smithi104 bash[17588]: debug 2020-09-08T15:42:35.604+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:35.956 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:36.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:35 smithi166 bash[9446]: cluster 2020-09-08T15:42:33.670600+0000 mgr.x (mgr.24118) 455 : cluster [DBG] pgmap v596: 17 pgs: 17 active+clean; 2.0 GiB data, 4.5 GiB used, 979 GiB / 983 GiB avail; 203 MiB/s wr, 3.24k op/s 2020-09-08T15:42:36.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:42:35 smithi166 bash[18960]: debug 2020-09-08T15:42:35.854+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:36.215 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 16 16 42906 42890 167.518 139.312 0.00274 0.00596214 2020-09-08T15:42:36.464 INFO:tasks.thrashosds.thrasher:j is {'epoch': 223, 'fsid': '759b27b6-f1e7-11ea-a080-001a4aab830c', 'created': '2020-09-08T15:25:41.788365+0000', 'modified': '2020-09-08T15:42:29.653098+0000', 'last_up_change': '2020-09-08T15:34:41.591462+0000', 'last_in_change': '2020-09-08T15:42:12.636980+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 27, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 12, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-09-08T15:25:56.314385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 3, 'pool_name': 'unique_pool_1', 'create_time': '2020-09-08T15:42:05.902210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '219', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'e49a7341-7c75-4883-b061-3e8fe9467e6e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 217, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6801', 'nonce': 82470352}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6802', 'nonce': 82470352}, {'type': 'v1', 'addr': '172.21.15.104:6803', 'nonce': 82470352}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6806', 'nonce': 82470352}, {'type': 'v1', 'addr': '172.21.15.104:6807', 'nonce': 82470352}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6804', 'nonce': 82470352}, {'type': 'v1', 'addr': '172.21.15.104:6805', 'nonce': 82470352}]}, 'public_addr': '172.21.15.104:6801/82470352', 'cluster_addr': '172.21.15.104:6803/82470352', 'heartbeat_back_addr': '172.21.15.104:6807/82470352', 'heartbeat_front_addr': '172.21.15.104:6805/82470352', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a83df412-110d-440a-b411-ad2a7c36f81a', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 214, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6809', 'nonce': 1421798992}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6810', 'nonce': 1421798992}, {'type': 'v1', 'addr': '172.21.15.104:6811', 'nonce': 1421798992}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6814', 'nonce': 1421798992}, {'type': 'v1', 'addr': '172.21.15.104:6815', 'nonce': 1421798992}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6812', 'nonce': 1421798992}, {'type': 'v1', 'addr': '172.21.15.104:6813', 'nonce': 1421798992}]}, 'public_addr': '172.21.15.104:6809/1421798992', 'cluster_addr': '172.21.15.104:6811/1421798992', 'heartbeat_back_addr': '172.21.15.104:6815/1421798992', 'heartbeat_front_addr': '172.21.15.104:6813/1421798992', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'ca6da1cc-1d8d-494b-93e6-e065ff70d434', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6817', 'nonce': 2713413369}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6818', 'nonce': 2713413369}, {'type': 'v1', 'addr': '172.21.15.104:6819', 'nonce': 2713413369}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6822', 'nonce': 2713413369}, {'type': 'v1', 'addr': '172.21.15.104:6823', 'nonce': 2713413369}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6820', 'nonce': 2713413369}, {'type': 'v1', 'addr': '172.21.15.104:6821', 'nonce': 2713413369}]}, 'public_addr': '172.21.15.104:6817/2713413369', 'cluster_addr': '172.21.15.104:6819/2713413369', 'heartbeat_back_addr': '172.21.15.104:6823/2713413369', 'heartbeat_front_addr': '172.21.15.104:6821/2713413369', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 29, 'up_thru': 214, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6825', 'nonce': 1495946248}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6826', 'nonce': 1495946248}, {'type': 'v1', 'addr': '172.21.15.104:6827', 'nonce': 1495946248}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6830', 'nonce': 1495946248}, {'type': 'v1', 'addr': '172.21.15.104:6831', 'nonce': 1495946248}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6828', 'nonce': 1495946248}, {'type': 'v1', 'addr': '172.21.15.104:6829', 'nonce': 1495946248}]}, 'public_addr': '172.21.15.104:6825/1495946248', 'cluster_addr': '172.21.15.104:6827/1495946248', 'heartbeat_back_addr': '172.21.15.104:6831/1495946248', 'heartbeat_front_addr': '172.21.15.104:6829/1495946248', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd8334c70-f776-4652-944d-aed35a8e43f7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 35, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.55:6803', 'nonce': 1173103260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6804', 'nonce': 1173103260}, {'type': 'v1', 'addr': '172.21.15.55:6805', 'nonce': 1173103260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6808', 'nonce': 1173103260}, {'type': 'v1', 'addr': '172.21.15.55:6809', 'nonce': 1173103260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6806', 'nonce': 1173103260}, {'type': 'v1', 'addr': '172.21.15.55:6807', 'nonce': 1173103260}]}, 'public_addr': '172.21.15.55:6803/1173103260', 'cluster_addr': '172.21.15.55:6805/1173103260', 'heartbeat_back_addr': '172.21.15.55:6809/1173103260', 'heartbeat_front_addr': '172.21.15.55:6807/1173103260', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '6789abd2-9e9b-4c04-81fb-0ce2d2849e39', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 42, 'up_thru': 214, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.55:6811', 'nonce': 1047046376}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6812', 'nonce': 1047046376}, {'type': 'v1', 'addr': '172.21.15.55:6813', 'nonce': 1047046376}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6816', 'nonce': 1047046376}, {'type': 'v1', 'addr': '172.21.15.55:6817', 'nonce': 1047046376}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6814', 'nonce': 1047046376}, {'type': 'v1', 'addr': '172.21.15.55:6815', 'nonce': 1047046376}]}, 'public_addr': '172.21.15.55:6811/1047046376', 'cluster_addr': '172.21.15.55:6813/1047046376', 'heartbeat_back_addr': '172.21.15.55:6817/1047046376', 'heartbeat_front_addr': '172.21.15.55:6815/1047046376', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ad9a93fe-0ca0-4052-bfa9-e2abb268829e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 49, 'up_thru': 214, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.55:6819', 'nonce': 2501844793}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6820', 'nonce': 2501844793}, {'type': 'v1', 'addr': '172.21.15.55:6821', 'nonce': 2501844793}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6824', 'nonce': 2501844793}, {'type': 'v1', 'addr': '172.21.15.55:6825', 'nonce': 2501844793}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6822', 'nonce': 2501844793}, {'type': 'v1', 'addr': '172.21.15.55:6823', 'nonce': 2501844793}]}, 'public_addr': '172.21.15.55:6819/2501844793', 'cluster_addr': '172.21.15.55:6821/2501844793', 'heartbeat_back_addr': '172.21.15.55:6825/2501844793', 'heartbeat_front_addr': '172.21.15.55:6823/2501844793', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'f3795c86-4540-442c-a151-b2769fa577e4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 56, 'up_thru': 214, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.55:6827', 'nonce': 128909153}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6828', 'nonce': 128909153}, {'type': 'v1', 'addr': '172.21.15.55:6829', 'nonce': 128909153}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6832', 'nonce': 128909153}, {'type': 'v1', 'addr': '172.21.15.55:6833', 'nonce': 128909153}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6830', 'nonce': 128909153}, {'type': 'v1', 'addr': '172.21.15.55:6831', 'nonce': 128909153}]}, 'public_addr': '172.21.15.55:6827/128909153', 'cluster_addr': '172.21.15.55:6829/128909153', 'heartbeat_back_addr': '172.21.15.55:6833/128909153', 'heartbeat_front_addr': '172.21.15.55:6831/128909153', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '3b0ac380-878e-4c1c-9278-80ce67ca2df4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 64, 'up_thru': 214, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.166:6801', 'nonce': 2108117574}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6802', 'nonce': 2108117574}, {'type': 'v1', 'addr': '172.21.15.166:6803', 'nonce': 2108117574}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6806', 'nonce': 2108117574}, {'type': 'v1', 'addr': '172.21.15.166:6807', 'nonce': 2108117574}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6804', 'nonce': 2108117574}, {'type': 'v1', 'addr': '172.21.15.166:6805', 'nonce': 2108117574}]}, 'public_addr': '172.21.15.166:6801/2108117574', 'cluster_addr': '172.21.15.166:6803/2108117574', 'heartbeat_back_addr': '172.21.15.166:6807/2108117574', 'heartbeat_front_addr': '172.21.15.166:6805/2108117574', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': 'ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 72, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.166:6809', 'nonce': 1458870080}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6810', 'nonce': 1458870080}, {'type': 'v1', 'addr': '172.21.15.166:6811', 'nonce': 1458870080}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6814', 'nonce': 1458870080}, {'type': 'v1', 'addr': '172.21.15.166:6815', 'nonce': 1458870080}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6812', 'nonce': 1458870080}, {'type': 'v1', 'addr': '172.21.15.166:6813', 'nonce': 1458870080}]}, 'public_addr': '172.21.15.166:6809/1458870080', 'cluster_addr': '172.21.15.166:6811/1458870080', 'heartbeat_back_addr': '172.21.15.166:6815/1458870080', 'heartbeat_front_addr': '172.21.15.166:6813/1458870080', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': 'c194210e-bd89-4a1f-98fb-6bf44f0b324c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 81, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.166:6817', 'nonce': 62517848}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6818', 'nonce': 62517848}, {'type': 'v1', 'addr': '172.21.15.166:6819', 'nonce': 62517848}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6822', 'nonce': 62517848}, {'type': 'v1', 'addr': '172.21.15.166:6823', 'nonce': 62517848}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6820', 'nonce': 62517848}, {'type': 'v1', 'addr': '172.21.15.166:6821', 'nonce': 62517848}]}, 'public_addr': '172.21.15.166:6817/62517848', 'cluster_addr': '172.21.15.166:6819/62517848', 'heartbeat_back_addr': '172.21.15.166:6823/62517848', 'heartbeat_front_addr': '172.21.15.166:6821/62517848', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '121301fe-a41c-40c0-ad09-b00de37a55e1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 90, 'up_thru': 217, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.166:6825', 'nonce': 691735125}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6826', 'nonce': 691735125}, {'type': 'v1', 'addr': '172.21.15.166:6827', 'nonce': 691735125}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6830', 'nonce': 691735125}, {'type': 'v1', 'addr': '172.21.15.166:6831', 'nonce': 691735125}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.166:6828', 'nonce': 691735125}, {'type': 'v1', 'addr': '172.21.15.166:6829', 'nonce': 691735125}]}, 'public_addr': '172.21.15.166:6825/691735125', 'cluster_addr': '172.21.15.166:6827/691735125', 'heartbeat_back_addr': '172.21.15.166:6831/691735125', 'heartbeat_front_addr': '172.21.15.166:6829/691735125', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:42:06.994157+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 65536, 'last_purged_snaps_scrub': '2020-09-08T15:41:34.305299+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:41:57.794699+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:42:22.114232+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:41:46.565606+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:42:13.283099+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:41:40.422586+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:42:06.536241+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:42:28.387217+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:41:55.192154+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:42:18.419489+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-09-08T15:41:42.473673+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.104:0/1257365091': '2020-09-09T15:29:39.430456+0000', '172.21.15.104:0/1399123455': '2020-09-09T15:29:39.430456+0000', '172.21.15.104:0/2810441605': '2020-09-09T15:29:39.430456+0000', '172.21.15.104:6800/182241824': '2020-09-09T15:29:39.430456+0000', '172.21.15.104:6801/182241824': '2020-09-09T15:29:39.430456+0000', '172.21.15.104:0/2021933044': '2020-09-09T15:26:33.248197+0000', '172.21.15.104:6800/3129754347': '2020-09-09T15:26:33.248197+0000', '172.21.15.104:0/1270031705': '2020-09-09T15:26:33.248197+0000', '172.21.15.104:0/2568432190': '2020-09-09T15:26:33.248197+0000', '172.21.15.104:6800/1138921844': '2020-09-09T15:26:09.867236+0000', '172.21.15.104:0/3827328474': '2020-09-09T15:26:09.867236+0000', '172.21.15.104:6801/1138921844': '2020-09-09T15:26:09.867236+0000', '172.21.15.104:0/1662035381': '2020-09-09T15:26:09.867236+0000', '172.21.15.104:6801/3129754347': '2020-09-09T15:26:33.248197+0000', '172.21.15.104:0/2169968102': '2020-09-09T15:26:09.867236+0000'}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}} 2020-09-08T15:42:36.465 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph pg dump --format=json 2020-09-08T15:42:36.470 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2020-09-08T15:42:36.624 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:42:36.672 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:42:36.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:36 smithi055 bash[9296]: cluster 2020-09-08T15:42:34.997514+0000 osd.1 (osd.1) 46 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:36.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:36 smithi055 bash[9296]: cluster 2020-09-08T15:42:34.997630+0000 osd.1 (osd.1) 47 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:36.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:36 smithi055 bash[9296]: cluster 2020-09-08T15:42:35.634183+0000 mon.a (mon.0) 1142 : cluster [DBG] osdmap e224: 12 total, 12 up, 11 in 2020-09-08T15:42:36.918 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:36 smithi104 bash[9203]: cluster 2020-09-08T15:42:34.997514+0000 osd.1 (osd.1) 46 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:36.918 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:36 smithi104 bash[9203]: cluster 2020-09-08T15:42:34.997630+0000 osd.1 (osd.1) 47 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:36.918 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:36 smithi104 bash[9203]: cluster 2020-09-08T15:42:35.634183+0000 mon.a (mon.0) 1142 : cluster [DBG] osdmap e224: 12 total, 12 up, 11 in 2020-09-08T15:42:37.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:36 smithi166 bash[9446]: cluster 2020-09-08T15:42:34.997514+0000 osd.1 (osd.1) 46 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:37.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:36 smithi166 bash[9446]: cluster 2020-09-08T15:42:34.997630+0000 osd.1 (osd.1) 47 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:37.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:36 smithi166 bash[9446]: cluster 2020-09-08T15:42:35.634183+0000 mon.a (mon.0) 1142 : cluster [DBG] osdmap e224: 12 total, 12 up, 11 in 2020-09-08T15:42:37.216 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 17 16 43919 43903 161.388 63.3125 0.0151294 0.00619332 2020-09-08T15:42:37.627 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:42:37.658 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:42:37.680 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:42:37.736 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:42:37.771 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:42:37.807 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:42:37.840 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:42:37.870 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:42:37.902 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:42:37.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:37 smithi055 bash[9296]: cluster 2020-09-08T15:42:35.671311+0000 mgr.x (mgr.24118) 456 : cluster [DBG] pgmap v598: 17 pgs: 17 active+clean; 2.1 GiB data, 4.8 GiB used, 979 GiB / 983 GiB avail; 124 MiB/s wr, 1.98k op/s 2020-09-08T15:42:37.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:37 smithi055 bash[9296]: audit 2020-09-08T15:42:37.604429+0000 mon.a (mon.0) 1143 : audit [DBG] from='client.? v1:172.21.15.104:0/3332464561' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:42:37.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:37 smithi104 bash[9203]: cluster 2020-09-08T15:42:35.671311+0000 mgr.x (mgr.24118) 456 : cluster [DBG] pgmap v598: 17 pgs: 17 active+clean; 2.1 GiB data, 4.8 GiB used, 979 GiB / 983 GiB avail; 124 MiB/s wr, 1.98k op/s 2020-09-08T15:42:37.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:37 smithi104 bash[9203]: audit 2020-09-08T15:42:37.604429+0000 mon.a (mon.0) 1143 : audit [DBG] from='client.? v1:172.21.15.104:0/3332464561' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:42:37.945 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:42:37.979 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:42:38.012 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:42:38.015 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:42:38.015 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:42:38.015 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:42:38.015 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:42:38.016 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:42:38.016 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:42:38.016 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:42:38.016 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:42:38.017 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:42:38.017 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:42:38.017 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:42:38.017 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:42:38.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:37 smithi166 bash[9446]: cluster 2020-09-08T15:42:35.671311+0000 mgr.x (mgr.24118) 456 : cluster [DBG] pgmap v598: 17 pgs: 17 active+clean; 2.1 GiB data, 4.8 GiB used, 979 GiB / 983 GiB avail; 124 MiB/s wr, 1.98k op/s 2020-09-08T15:42:38.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:37 smithi166 bash[9446]: audit 2020-09-08T15:42:37.604429+0000 mon.a (mon.0) 1143 : audit [DBG] from='client.? v1:172.21.15.104:0/3332464561' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:42:38.215 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 18 16 45986 45970 159.598 129.188 0.0042344 0.00626314 2020-09-08T15:42:39.215 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 19 16 47826 47810 157.251 115 0.00681845 0.00635617 2020-09-08T15:42:39.645 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:39 smithi104 bash[9203]: cluster 2020-09-08T15:42:37.671805+0000 mgr.x (mgr.24118) 457 : cluster [DBG] pgmap v599: 17 pgs: 17 active+clean; 2.5 GiB data, 5.5 GiB used, 978 GiB / 983 GiB avail; 147 MiB/s wr, 2.35k op/s 2020-09-08T15:42:39.836 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:39.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:39 smithi055 bash[9296]: cluster 2020-09-08T15:42:37.671805+0000 mgr.x (mgr.24118) 457 : cluster [DBG] pgmap v599: 17 pgs: 17 active+clean; 2.5 GiB data, 5.5 GiB used, 978 GiB / 983 GiB avail; 147 MiB/s wr, 2.35k op/s 2020-09-08T15:42:39.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:42:39 smithi055 bash[19003]: debug 2020-09-08T15:42:39.836+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:39.940 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:39.949 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:39 smithi166 bash[9446]: cluster 2020-09-08T15:42:37.671805+0000 mgr.x (mgr.24118) 457 : cluster [DBG] pgmap v599: 17 pgs: 17 active+clean; 2.5 GiB data, 5.5 GiB used, 978 GiB / 983 GiB avail; 147 MiB/s wr, 2.35k op/s 2020-09-08T15:42:40.072 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:40.185 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:40.215 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 15:42:40.221175 min lat: 0.00154046 max lat: 0.419999 avg lat: 0.00642066 2020-09-08T15:42:40.216 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:42:40.216 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 20 16 49822 49806 155.625 124.75 0.0030345 0.00642066 2020-09-08T15:42:40.365 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:42:40 smithi166 bash[11928]: debug 2020-09-08T15:42:40.074+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:40.393 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:40.499 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:40.512 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:42:40 smithi104 bash[22305]: debug 2020-09-08T15:42:40.396+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:41.216 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 21 16 51880 51864 154.339 128.625 0.0144314 0.00647689 2020-09-08T15:42:41.353 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:42:41.374 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:41.480 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:42:41.513 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:42:41.654 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:42:41 smithi104 bash[17588]: debug 2020-09-08T15:42:41.380+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:41.654 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:41 smithi104 bash[9203]: cluster 2020-09-08T15:42:39.672311+0000 mgr.x (mgr.24118) 458 : cluster [DBG] pgmap v600: 17 pgs: 17 active+clean; 2.7 GiB data, 6.1 GiB used, 977 GiB / 983 GiB avail; 140 MiB/s wr, 2.23k op/s 2020-09-08T15:42:41.655 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:41 smithi104 bash[9203]: cluster 2020-09-08T15:42:40.573995+0000 osd.6 (osd.6) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:41.655 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:41 smithi104 bash[9203]: cluster 2020-09-08T15:42:40.574093+0000 osd.6 (osd.6) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:41.655 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:41 smithi104 bash[9203]: cluster 2020-09-08T15:42:40.660589+0000 mon.a (mon.0) 1144 : cluster [DBG] osdmap e225: 12 total, 12 up, 11 in 2020-09-08T15:42:41.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:41 smithi055 bash[9296]: cluster 2020-09-08T15:42:39.672311+0000 mgr.x (mgr.24118) 458 : cluster [DBG] pgmap v600: 17 pgs: 17 active+clean; 2.7 GiB data, 6.1 GiB used, 977 GiB / 983 GiB avail; 140 MiB/s wr, 2.23k op/s 2020-09-08T15:42:41.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:41 smithi055 bash[9296]: cluster 2020-09-08T15:42:40.573995+0000 osd.6 (osd.6) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:41.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:41 smithi055 bash[9296]: cluster 2020-09-08T15:42:40.574093+0000 osd.6 (osd.6) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:41.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:41 smithi055 bash[9296]: cluster 2020-09-08T15:42:40.660589+0000 mon.a (mon.0) 1144 : cluster [DBG] osdmap e225: 12 total, 12 up, 11 in 2020-09-08T15:42:42.124 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:41 smithi166 bash[9446]: cluster 2020-09-08T15:42:39.672311+0000 mgr.x (mgr.24118) 458 : cluster [DBG] pgmap v600: 17 pgs: 17 active+clean; 2.7 GiB data, 6.1 GiB used, 977 GiB / 983 GiB avail; 140 MiB/s wr, 2.23k op/s 2020-09-08T15:42:42.125 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:41 smithi166 bash[9446]: cluster 2020-09-08T15:42:40.573995+0000 osd.6 (osd.6) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:42.125 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:41 smithi166 bash[9446]: cluster 2020-09-08T15:42:40.574093+0000 osd.6 (osd.6) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:42.125 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:41 smithi166 bash[9446]: cluster 2020-09-08T15:42:40.660589+0000 mon.a (mon.0) 1144 : cluster [DBG] osdmap e225: 12 total, 12 up, 11 in 2020-09-08T15:42:42.215 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 22 16 54026 54010 153.42 134.125 0.00395908 0.0065156 2020-09-08T15:42:42.236 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:42:42.341 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:42.522 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:42.615 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:42:42 smithi166 bash[11928]: debug 2020-09-08T15:42:42.522+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:42.626 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:42.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:42:42 smithi104 bash[15273]: debug 2020-09-08T15:42:42.240+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:42.747 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:42.852 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:42.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:42:42 smithi055 bash[19003]: debug 2020-09-08T15:42:42.748+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:43.215 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 23 16 55709 55693 151.322 105.188 0.0101282 0.00660614 2020-09-08T15:42:43.596 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:43.701 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:42:43.836 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:42:43.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:42:43 smithi104 bash[22305]: debug 2020-09-08T15:42:43.600+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:43.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:43 smithi104 bash[9203]: cluster 2020-09-08T15:42:41.672718+0000 mgr.x (mgr.24118) 459 : cluster [DBG] pgmap v602: 17 pgs: 17 active+clean; 2.8 GiB data, 6.3 GiB used, 977 GiB / 983 GiB avail; 104 MiB/s wr, 1.66k op/s 2020-09-08T15:42:43.941 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:42:43.956 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:43 smithi166 bash[9446]: cluster 2020-09-08T15:42:41.672718+0000 mgr.x (mgr.24118) 459 : cluster [DBG] pgmap v602: 17 pgs: 17 active+clean; 2.8 GiB data, 6.3 GiB used, 977 GiB / 983 GiB avail; 104 MiB/s wr, 1.66k op/s 2020-09-08T15:42:43.956 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:42:43 smithi166 bash[14254]: debug 2020-09-08T15:42:43.838+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:44.059 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:42:44.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:43 smithi055 bash[9296]: cluster 2020-09-08T15:42:41.672718+0000 mgr.x (mgr.24118) 459 : cluster [DBG] pgmap v602: 17 pgs: 17 active+clean; 2.8 GiB data, 6.3 GiB used, 977 GiB / 983 GiB avail; 104 MiB/s wr, 1.66k op/s 2020-09-08T15:42:44.162 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:44.216 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 24 15 57483 57468 149.639 110.938 0.00850496 0.00668089 2020-09-08T15:42:44.288 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:44.365 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:42:44 smithi166 bash[14254]: debug 2020-09-08T15:42:44.062+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:44.391 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:44.494 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:44.597 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:44.607 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:42:44 smithi055 bash[19003]: debug 2020-09-08T15:42:44.288+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:44.673 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:42:44 smithi166 bash[18960]: debug 2020-09-08T15:42:44.494+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:44.673 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:44 smithi104 bash[9203]: cluster 2020-09-08T15:42:43.207029+0000 osd.11 (osd.11) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:44.674 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:44 smithi104 bash[9203]: cluster 2020-09-08T15:42:43.207115+0000 osd.11 (osd.11) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:44.674 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:44 smithi104 bash[9203]: cluster 2020-09-08T15:42:43.671504+0000 mon.a (mon.0) 1145 : cluster [DBG] osdmap e226: 12 total, 12 up, 11 in 2020-09-08T15:42:44.674 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:44 smithi104 bash[9203]: cluster 2020-09-08T15:42:43.673125+0000 mgr.x (mgr.24118) 460 : cluster [DBG] pgmap v604: 17 pgs: 17 active+clean; 3.2 GiB data, 7.0 GiB used, 976 GiB / 983 GiB avail; 146 MiB/s wr, 2.34k op/s 2020-09-08T15:42:44.674 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:44 smithi104 bash[9203]: cluster 2020-09-08T15:42:44.634517+0000 mon.a (mon.0) 1146 : cluster [DBG] osdmap e227: 12 total, 12 up, 11 in 2020-09-08T15:42:44.706 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:44.809 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:44.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:44 smithi055 bash[9296]: cluster 2020-09-08T15:42:43.207029+0000 osd.11 (osd.11) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:44.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:44 smithi055 bash[9296]: cluster 2020-09-08T15:42:43.207115+0000 osd.11 (osd.11) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:44.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:44 smithi055 bash[9296]: cluster 2020-09-08T15:42:43.671504+0000 mon.a (mon.0) 1145 : cluster [DBG] osdmap e226: 12 total, 12 up, 11 in 2020-09-08T15:42:44.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:44 smithi055 bash[9296]: cluster 2020-09-08T15:42:43.673125+0000 mgr.x (mgr.24118) 460 : cluster [DBG] pgmap v604: 17 pgs: 17 active+clean; 3.2 GiB data, 7.0 GiB used, 976 GiB / 983 GiB avail; 146 MiB/s wr, 2.34k op/s 2020-09-08T15:42:44.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:44 smithi055 bash[9296]: cluster 2020-09-08T15:42:44.634517+0000 mon.a (mon.0) 1146 : cluster [DBG] osdmap e227: 12 total, 12 up, 11 in 2020-09-08T15:42:44.909 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:42:44 smithi055 bash[16646]: debug 2020-09-08T15:42:44.704+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:44.929 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:45.033 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:45.046 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:44 smithi166 bash[9446]: cluster 2020-09-08T15:42:43.207029+0000 osd.11 (osd.11) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:45.046 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:44 smithi166 bash[9446]: cluster 2020-09-08T15:42:43.207115+0000 osd.11 (osd.11) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:45.047 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:44 smithi166 bash[9446]: cluster 2020-09-08T15:42:43.671504+0000 mon.a (mon.0) 1145 : cluster [DBG] osdmap e226: 12 total, 12 up, 11 in 2020-09-08T15:42:45.047 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:44 smithi166 bash[9446]: cluster 2020-09-08T15:42:43.673125+0000 mgr.x (mgr.24118) 460 : cluster [DBG] pgmap v604: 17 pgs: 17 active+clean; 3.2 GiB data, 7.0 GiB used, 976 GiB / 983 GiB avail; 146 MiB/s wr, 2.34k op/s 2020-09-08T15:42:45.047 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:44 smithi166 bash[9446]: cluster 2020-09-08T15:42:44.634517+0000 mon.a (mon.0) 1146 : cluster [DBG] osdmap e227: 12 total, 12 up, 11 in 2020-09-08T15:42:45.149 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:45.216 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 25 16 59410 59394 148.468 120.375 0.00490731 0.00673299 2020-09-08T15:42:45.254 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:45.362 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:42:45 smithi166 bash[18960]: debug 2020-09-08T15:42:45.146+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:45.363 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:45.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:42:44 smithi055 bash[16646]: debug 2020-09-08T15:42:44.928+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:45.468 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:45.590 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:45.615 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:42:45 smithi166 bash[11928]: debug 2020-09-08T15:42:45.362+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:45.687 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:42:45.691 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:45.807 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:45.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:45 smithi055 bash[9296]: audit 2020-09-08T15:42:45.157747+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? v1:172.21.15.104:0/2944779211' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:42:45.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:42:45 smithi055 bash[16646]: debug 2020-09-08T15:42:45.584+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:45.911 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:45.923 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:45 smithi104 bash[9203]: audit 2020-09-08T15:42:45.157747+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? v1:172.21.15.104:0/2944779211' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:42:46.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:45 smithi166 bash[9446]: audit 2020-09-08T15:42:45.157747+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? v1:172.21.15.104:0/2944779211' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:42:46.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:42:45 smithi166 bash[16593]: debug 2020-09-08T15:42:45.806+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:46.194 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:42:46.201 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:42:46.216 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 26 16 61326 61310 147.363 119.75 0.00387517 0.00678353 2020-09-08T15:42:46.724 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:46.827 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:46.939 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:46 smithi166 bash[9446]: cluster 2020-09-08T15:42:45.673645+0000 mgr.x (mgr.24118) 461 : cluster [DBG] pgmap v606: 17 pgs: 17 active+clean; 3.3 GiB data, 7.3 GiB used, 976 GiB / 983 GiB avail; 101 MiB/s wr, 1.62k op/s 2020-09-08T15:42:46.940 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:46 smithi166 bash[9446]: cluster 2020-09-08T15:42:45.678362+0000 mon.a (mon.0) 1148 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:42:46.940 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:46 smithi166 bash[9446]: audit 2020-09-08T15:42:45.690313+0000 mon.a (mon.0) 1149 : audit [INF] from='client.? v1:172.21.15.104:0/2944779211' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:42:46.940 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:46 smithi166 bash[9446]: cluster 2020-09-08T15:42:45.690360+0000 mon.a (mon.0) 1150 : cluster [DBG] osdmap e228: 12 total, 12 up, 11 in 2020-09-08T15:42:46.941 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:46.949 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:46 smithi104 bash[9203]: cluster 2020-09-08T15:42:45.673645+0000 mgr.x (mgr.24118) 461 : cluster [DBG] pgmap v606: 17 pgs: 17 active+clean; 3.3 GiB data, 7.3 GiB used, 976 GiB / 983 GiB avail; 101 MiB/s wr, 1.62k op/s 2020-09-08T15:42:46.950 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:46 smithi104 bash[9203]: cluster 2020-09-08T15:42:45.678362+0000 mon.a (mon.0) 1148 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:42:46.950 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:46 smithi104 bash[9203]: audit 2020-09-08T15:42:45.690313+0000 mon.a (mon.0) 1149 : audit [INF] from='client.? v1:172.21.15.104:0/2944779211' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:42:46.950 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:46 smithi104 bash[9203]: cluster 2020-09-08T15:42:45.690360+0000 mon.a (mon.0) 1150 : cluster [DBG] osdmap e228: 12 total, 12 up, 11 in 2020-09-08T15:42:46.952 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:42:46 smithi104 bash[17588]: debug 2020-09-08T15:42:46.724+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:47.046 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:47.158 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:47.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:46 smithi055 bash[9296]: cluster 2020-09-08T15:42:45.673645+0000 mgr.x (mgr.24118) 461 : cluster [DBG] pgmap v606: 17 pgs: 17 active+clean; 3.3 GiB data, 7.3 GiB used, 976 GiB / 983 GiB avail; 101 MiB/s wr, 1.62k op/s 2020-09-08T15:42:47.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:46 smithi055 bash[9296]: cluster 2020-09-08T15:42:45.678362+0000 mon.a (mon.0) 1148 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:42:47.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:46 smithi055 bash[9296]: audit 2020-09-08T15:42:45.690313+0000 mon.a (mon.0) 1149 : audit [INF] from='client.? v1:172.21.15.104:0/2944779211' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:42:47.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:46 smithi055 bash[9296]: cluster 2020-09-08T15:42:45.690360+0000 mon.a (mon.0) 1150 : cluster [DBG] osdmap e228: 12 total, 12 up, 11 in 2020-09-08T15:42:47.216 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 27 16 63198 63182 146.238 117 0.0108271 0.00683667 2020-09-08T15:42:47.262 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:47.365 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:42:47 smithi166 bash[16593]: debug 2020-09-08T15:42:47.158+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:47.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:42:46 smithi166 bash[18960]: debug 2020-09-08T15:42:46.938+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:47.393 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:47.496 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:47.570 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T15:42:47.574 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2020-09-08T15:42:47.648 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:47.649 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:42:47 smithi055 bash[19003]: debug 2020-09-08T15:42:47.392+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:47.649 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:42:47 smithi055 bash[19003]: debug 2020-09-08T15:42:47.648+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:47.752 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:47.883 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:47.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:47 smithi055 bash[9296]: cluster 2020-09-08T15:42:46.715143+0000 osd.4 (osd.4) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:47.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:47 smithi055 bash[9296]: cluster 2020-09-08T15:42:46.715238+0000 osd.4 (osd.4) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:47.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:47 smithi055 bash[9296]: audit 2020-09-08T15:42:47.564705+0000 mgr.x (mgr.24118) 462 : audit [DBG] from='client.35114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:42:47.908 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:42:47 smithi055 bash[14301]: debug 2020-09-08T15:42:47.884+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:47.987 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:42:47.995 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:47 smithi166 bash[9446]: cluster 2020-09-08T15:42:46.715143+0000 osd.4 (osd.4) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:47.996 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:47 smithi166 bash[9446]: cluster 2020-09-08T15:42:46.715238+0000 osd.4 (osd.4) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:47.997 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:47 smithi166 bash[9446]: audit 2020-09-08T15:42:47.564705+0000 mgr.x (mgr.24118) 462 : audit [DBG] from='client.35114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:42:48.101 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:42:48.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:47 smithi104 bash[9203]: cluster 2020-09-08T15:42:46.715143+0000 osd.4 (osd.4) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:48.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:47 smithi104 bash[9203]: cluster 2020-09-08T15:42:46.715238+0000 osd.4 (osd.4) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:48.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:47 smithi104 bash[9203]: audit 2020-09-08T15:42:47.564705+0000 mgr.x (mgr.24118) 462 : audit [DBG] from='client.35114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:42:48.208 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:48.216 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 28 16 65072 65056 145.198 117.125 0.0083458 0.00688528 2020-09-08T15:42:48.318 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:42:48 smithi166 bash[14254]: debug 2020-09-08T15:42:48.102+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:48.451 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:48.554 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:48.615 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:42:48 smithi166 bash[11928]: debug 2020-09-08T15:42:48.318+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:48.669 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:48.670 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2020-09-08T15:42:48.770 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:48.857 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:42:48.905 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:48.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:48 smithi104 bash[9203]: cluster 2020-09-08T15:42:47.674116+0000 mgr.x (mgr.24118) 463 : cluster [DBG] pgmap v608: 17 pgs: 17 active+clean; 3.6 GiB data, 7.8 GiB used, 976 GiB / 983 GiB avail; 145 MiB/s wr, 2.32k op/s 2020-09-08T15:42:48.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:48 smithi104 bash[9203]: cluster 2020-09-08T15:42:47.692171+0000 mon.a (mon.0) 1151 : cluster [DBG] osdmap e229: 12 total, 12 up, 11 in 2020-09-08T15:42:49.010 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:49.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:48 smithi166 bash[9446]: cluster 2020-09-08T15:42:47.674116+0000 mgr.x (mgr.24118) 463 : cluster [DBG] pgmap v608: 17 pgs: 17 active+clean; 3.6 GiB data, 7.8 GiB used, 976 GiB / 983 GiB avail; 145 MiB/s wr, 2.32k op/s 2020-09-08T15:42:49.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:48 smithi166 bash[9446]: cluster 2020-09-08T15:42:47.692171+0000 mon.a (mon.0) 1151 : cluster [DBG] osdmap e229: 12 total, 12 up, 11 in 2020-09-08T15:42:49.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:42:48 smithi166 bash[18960]: debug 2020-09-08T15:42:48.658+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:49.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:48 smithi055 bash[9296]: cluster 2020-09-08T15:42:47.674116+0000 mgr.x (mgr.24118) 463 : cluster [DBG] pgmap v608: 17 pgs: 17 active+clean; 3.6 GiB data, 7.8 GiB used, 976 GiB / 983 GiB avail; 145 MiB/s wr, 2.32k op/s 2020-09-08T15:42:49.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:48 smithi055 bash[9296]: cluster 2020-09-08T15:42:47.692171+0000 mon.a (mon.0) 1151 : cluster [DBG] osdmap e229: 12 total, 12 up, 11 in 2020-09-08T15:42:49.158 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:42:48 smithi055 bash[14301]: debug 2020-09-08T15:42:48.908+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:49.216 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 29 16 66886 66870 144.101 113.375 0.00523491 0.00693715 2020-09-08T15:42:49.521 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:49.624 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:49.745 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:49.849 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:49.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:42:49 smithi055 bash[14301]: debug 2020-09-08T15:42:49.744+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:49.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:42:49 smithi104 bash[17588]: debug 2020-09-08T15:42:49.524+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:49.987 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:50.092 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:50.115 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:42:49 smithi166 bash[16593]: debug 2020-09-08T15:42:49.986+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:50.204 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:50.206 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:42:50.216 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 30 16 68635 68619 142.941 109.312 0.00765209 0.00699364 2020-09-08T15:42:50.309 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:50.364 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:42:50.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:42:50 smithi055 bash[16646]: debug 2020-09-08T15:42:50.204+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:50.414 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":607,"stamp":"2020-09-08T15:42:45.687731+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3523739648,"num_objects":53771,"num_object_clones":0,"num_object_copies":107542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53771,"num_write_kb":3441155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53774,"ondisk_log_size":53774,"up":34,"acting":34,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":11,"num_per_pool_osds":9,"num_per_pool_omap_osds":9,"kb":1031196672,"kb_used":7624468,"kb_used_data":6869372,"kb_used_omap":0,"kb_used_meta":755008,"kb_avail":1023572204,"statfs":{"total":1055945392128,"available":1048137936896,"internally_reserved":0,"allocated":7034236928,"data_stored":7030959923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":773128192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":635699200,"num_objects":9700,"num_object_clones":0,"num_object_copies":19400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9700,"num_write_kb":620800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.030049"},"pg_stats":[{"pgid":"3.f","version":"225'3597","reported_seq":"3615","reported_epoch":"225","state":"active+clean","last_fresh":"2020-09-08T15:42:43.669309+0000","last_change":"2020-09-08T15:42:06.978216+0000","last_active":"2020-09-08T15:42:43.669309+0000","last_peered":"2020-09-08T15:42:43.669309+0000","last_clean":"2020-09-08T15:42:43.669309+0000","last_became_active":"2020-09-08T15:42:06.977841+0000","last_became_peered":"2020-09-08T15:42:06.977841+0000","last_unstale":"2020-09-08T15:42:43.669309+0000","last_undegraded":"2020-09-08T15:42:43.669309+0000","last_fullsized":"2020-09-08T15:42:43.669309+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":3597,"ondisk_log_size":3597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":235732992,"num_objects":3597,"num_object_clones":0,"num_object_copies":7194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3597,"num_write_kb":230208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"225'3415","reported_seq":"3432","reported_epoch":"225","state":"active+clean","last_fresh":"2020-09-08T15:42:42.694214+0000","last_change":"2020-09-08T15:42:07.110906+0000","last_active":"2020-09-08T15:42:42.694214+0000","last_peered":"2020-09-08T15:42:42.694214+0000","last_clean":"2020-09-08T15:42:42.694214+0000","last_became_active":"2020-09-08T15:42:07.110465+0000","last_became_peered":"2020-09-08T15:42:07.110465+0000","last_unstale":"2020-09-08T15:42:42.694214+0000","last_undegraded":"2020-09-08T15:42:42.694214+0000","last_fullsized":"2020-09-08T15:42:42.694214+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":3415,"ondisk_log_size":3415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":223805440,"num_objects":3415,"num_object_clones":0,"num_object_copies":6830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3415,"num_write_kb":218560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4],"acting":[11,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.d","version":"225'3529","reported_seq":"3547","reported_epoch":"225","state":"active+clean","last_fresh":"2020-09-08T15:42:43.675987+0000","last_change":"2020-09-08T15:42:06.977801+0000","last_active":"2020-09-08T15:42:43.675987+0000","last_peered":"2020-09-08T15:42:43.675987+0000","last_clean":"2020-09-08T15:42:43.675987+0000","last_became_active":"2020-09-08T15:42:06.976987+0000","last_became_peered":"2020-09-08T15:42:06.976987+0000","last_unstale":"2020-09-08T15:42:43.675987+0000","last_undegraded":"2020-09-08T15:42:43.675987+0000","last_fullsized":"2020-09-08T15:42:43.675987+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":3529,"ondisk_log_size":3529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":231276544,"num_objects":3529,"num_object_clones":0,"num_object_copies":7058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3529,"num_write_kb":225856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"225'3319","reported_seq":"3335","reported_epoch":"225","state":"active+clean","last_fresh":"2020-09-08T15:42:41.702085+0000","last_change":"2020-09-08T15:42:06.974957+0000","last_active":"2020-09-08T15:42:41.702085+0000","last_peered":"2020-09-08T15:42:41.702085+0000","last_clean":"2020-09-08T15:42:41.702085+0000","last_became_active":"2020-09-08T15:42:06.974012+0000","last_became_peered":"2020-09-08T15:42:06.974012+0000","last_unstale":"2020-09-08T15:42:41.702085+0000","last_undegraded":"2020-09-08T15:42:41.702085+0000","last_fullsized":"2020-09-08T15:42:41.702085+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":3319,"ondisk_log_size":3319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":217513984,"num_objects":3319,"num_object_clones":0,"num_object_copies":6638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3319,"num_write_kb":212416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.b","version":"225'3312","reported_seq":"3330","reported_epoch":"225","state":"active+clean","last_fresh":"2020-09-08T15:42:42.159232+0000","last_change":"2020-09-08T15:42:06.973383+0000","last_active":"2020-09-08T15:42:42.159232+0000","last_peered":"2020-09-08T15:42:42.159232+0000","last_clean":"2020-09-08T15:42:42.159232+0000","last_became_active":"2020-09-08T15:42:06.972976+0000","last_became_peered":"2020-09-08T15:42:06.972976+0000","last_unstale":"2020-09-08T15:42:42.159232+0000","last_undegraded":"2020-09-08T15:42:42.159232+0000","last_fullsized":"2020-09-08T15:42:42.159232+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":3312,"ondisk_log_size":3312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":217055232,"num_objects":3312,"num_object_clones":0,"num_object_copies":6624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3312,"num_write_kb":211968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"225'3338","reported_seq":"3355","reported_epoch":"225","state":"active+clean","last_fresh":"2020-09-08T15:42:41.698695+0000","last_change":"2020-09-08T15:42:07.112085+0000","last_active":"2020-09-08T15:42:41.698695+0000","last_peered":"2020-09-08T15:42:41.698695+0000","last_clean":"2020-09-08T15:42:41.698695+0000","last_became_active":"2020-09-08T15:42:07.110953+0000","last_became_peered":"2020-09-08T15:42:07.110953+0000","last_unstale":"2020-09-08T15:42:41.698695+0000","last_undegraded":"2020-09-08T15:42:41.698695+0000","last_fullsized":"2020-09-08T15:42:41.698695+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":3338,"ondisk_log_size":3338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":218759168,"num_objects":3338,"num_object_clones":0,"num_object_copies":6676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3338,"num_write_kb":213632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"225'3423","reported_seq":"3440","reported_epoch":"225","state":"active+clean","last_fresh":"2020-09-08T15:42:41.885277+0000","last_change":"2020-09-08T15:42:06.975878+0000","last_active":"2020-09-08T15:42:41.885277+0000","last_peered":"2020-09-08T15:42:41.885277+0000","last_clean":"2020-09-08T15:42:41.885277+0000","last_became_active":"2020-09-08T15:42:06.975234+0000","last_became_peered":"2020-09-08T15:42:06.975234+0000","last_unstale":"2020-09-08T15:42:41.885277+0000","last_undegraded":"2020-09-08T15:42:41.885277+0000","last_fullsized":"2020-09-08T15:42:41.885277+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":3423,"ondisk_log_size":3423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":224329728,"num_objects":3423,"num_object_clones":0,"num_object_copies":6846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3423,"num_write_kb":219072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.8","version":"225'3344","reported_seq":"3369","reported_epoch":"225","state":"active+clean","last_fresh":"2020-09-08T15:42:42.720074+0000","last_change":"2020-09-08T15:42:13.914115+0000","last_active":"2020-09-08T15:42:42.720074+0000","last_peered":"2020-09-08T15:42:42.720074+0000","last_clean":"2020-09-08T15:42:42.720074+0000","last_became_active":"2020-09-08T15:42:13.912746+0000","last_became_peered":"2020-09-08T15:42:13.912746+0000","last_unstale":"2020-09-08T15:42:42.720074+0000","last_undegraded":"2020-09-08T15:42:42.720074+0000","last_fullsized":"2020-09-08T15:42:42.720074+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":3344,"ondisk_log_size":3344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":219152384,"num_objects":3344,"num_object_clones":0,"num_object_copies":6688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3344,"num_write_kb":214016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4],"acting":[11,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.7","version":"225'3320","reported_seq":"3338","reported_epoch":"225","state":"active+clean","last_fresh":"2020-09-08T15:42:42.156704+0000","last_change":"2020-09-08T15:42:06.977271+0000","last_active":"2020-09-08T15:42:42.156704+0000","last_peered":"2020-09-08T15:42:42.156704+0000","last_clean":"2020-09-08T15:42:42.156704+0000","last_became_active":"2020-09-08T15:42:06.976841+0000","last_became_peered":"2020-09-08T15:42:06.976841+0000","last_unstale":"2020-09-08T15:42:42.156704+0000","last_undegraded":"2020-09-08T15:42:42.156704+0000","last_fullsized":"2020-09-08T15:42:42.156704+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":3320,"ondisk_log_size":3320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":217579520,"num_objects":3320,"num_object_clones":0,"num_object_copies":6640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3320,"num_write_kb":212480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11],"acting":[3,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"225'3203","reported_seq":"3228","reported_epoch":"225","state":"active+clean","last_fresh":"2020-09-08T15:42:40.728349+0000","last_change":"2020-09-08T15:42:13.913591+0000","last_active":"2020-09-08T15:42:40.728349+0000","last_peered":"2020-09-08T15:42:40.728349+0000","last_clean":"2020-09-08T15:42:40.728349+0000","last_became_active":"2020-09-08T15:42:13.913210+0000","last_became_peered":"2020-09-08T15:42:13.913210+0000","last_unstale":"2020-09-08T15:42:40.728349+0000","last_undegraded":"2020-09-08T15:42:40.728349+0000","last_fullsized":"2020-09-08T15:42:40.728349+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":3203,"ondisk_log_size":3203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":209911808,"num_objects":3203,"num_object_clones":0,"num_object_copies":6406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3203,"num_write_kb":204992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"225'3371","reported_seq":"3388","reported_epoch":"225","state":"active+clean","last_fresh":"2020-09-08T15:42:41.695343+0000","last_change":"2020-09-08T15:42:06.976571+0000","last_active":"2020-09-08T15:42:41.695343+0000","last_peered":"2020-09-08T15:42:41.695343+0000","last_clean":"2020-09-08T15:42:41.695343+0000","last_became_active":"2020-09-08T15:42:06.976051+0000","last_became_peered":"2020-09-08T15:42:06.976051+0000","last_unstale":"2020-09-08T15:42:41.695343+0000","last_undegraded":"2020-09-08T15:42:41.695343+0000","last_fullsized":"2020-09-08T15:42:41.695343+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":3371,"ondisk_log_size":3371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220921856,"num_objects":3371,"num_object_clones":0,"num_object_copies":6742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3371,"num_write_kb":215744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.4","version":"225'3405","reported_seq":"3422","reported_epoch":"225","state":"active+clean","last_fresh":"2020-09-08T15:42:42.734533+0000","last_change":"2020-09-08T15:42:13.914073+0000","last_active":"2020-09-08T15:42:42.734533+0000","last_peered":"2020-09-08T15:42:42.734533+0000","last_clean":"2020-09-08T15:42:42.734533+0000","last_became_active":"2020-09-08T15:42:13.913363+0000","last_became_peered":"2020-09-08T15:42:13.913363+0000","last_unstale":"2020-09-08T15:42:42.734533+0000","last_undegraded":"2020-09-08T15:42:42.734533+0000","last_fullsized":"2020-09-08T15:42:42.734533+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":3405,"ondisk_log_size":3405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":223150080,"num_objects":3405,"num_object_clones":0,"num_object_copies":6810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3405,"num_write_kb":217920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5],"acting":[11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3","version":"225'3157","reported_seq":"3174","reported_epoch":"225","state":"active+clean","last_fresh":"2020-09-08T15:42:40.745654+0000","last_change":"2020-09-08T15:42:07.111837+0000","last_active":"2020-09-08T15:42:40.745654+0000","last_peered":"2020-09-08T15:42:40.745654+0000","last_clean":"2020-09-08T15:42:40.745654+0000","last_became_active":"2020-09-08T15:42:07.110780+0000","last_became_peered":"2020-09-08T15:42:07.110780+0000","last_unstale":"2020-09-08T15:42:40.745654+0000","last_undegraded":"2020-09-08T15:42:40.745654+0000","last_fullsized":"2020-09-08T15:42:40.745654+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":3157,"ondisk_log_size":3157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":206897152,"num_objects":3157,"num_object_clones":0,"num_object_copies":6314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3157,"num_write_kb":202048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"225'3142","reported_seq":"3159","reported_epoch":"225","state":"active+clean","last_fresh":"2020-09-08T15:42:40.744323+0000","last_change":"2020-09-08T15:42:07.113446+0000","last_active":"2020-09-08T15:42:40.744323+0000","last_peered":"2020-09-08T15:42:40.744323+0000","last_clean":"2020-09-08T15:42:40.744323+0000","last_became_active":"2020-09-08T15:42:07.111969+0000","last_became_peered":"2020-09-08T15:42:07.111969+0000","last_unstale":"2020-09-08T15:42:40.744323+0000","last_undegraded":"2020-09-08T15:42:40.744323+0000","last_fullsized":"2020-09-08T15:42:40.744323+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":3142,"ondisk_log_size":3142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":205914112,"num_objects":3142,"num_object_clones":0,"num_object_copies":6284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3142,"num_write_kb":201088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"225'3522","reported_seq":"3539","reported_epoch":"225","state":"active+clean","last_fresh":"2020-09-08T15:42:42.737971+0000","last_change":"2020-09-08T15:42:07.110130+0000","last_active":"2020-09-08T15:42:42.737971+0000","last_peered":"2020-09-08T15:42:42.737971+0000","last_clean":"2020-09-08T15:42:42.737971+0000","last_became_active":"2020-09-08T15:42:07.109409+0000","last_became_peered":"2020-09-08T15:42:07.109409+0000","last_unstale":"2020-09-08T15:42:42.737971+0000","last_undegraded":"2020-09-08T15:42:42.737971+0000","last_fullsized":"2020-09-08T15:42:42.737971+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":3522,"ondisk_log_size":3522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":230817792,"num_objects":3522,"num_object_clones":0,"num_object_copies":7044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3522,"num_write_kb":225408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9],"acting":[11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2","version":"225'3371","reported_seq":"3389","reported_epoch":"225","state":"active+clean","last_fresh":"2020-09-08T15:42:42.154099+0000","last_change":"2020-09-08T15:42:07.110869+0000","last_active":"2020-09-08T15:42:42.154099+0000","last_peered":"2020-09-08T15:42:42.154099+0000","last_clean":"2020-09-08T15:42:42.154099+0000","last_became_active":"2020-09-08T15:42:07.110316+0000","last_became_peered":"2020-09-08T15:42:07.110316+0000","last_unstale":"2020-09-08T15:42:42.154099+0000","last_undegraded":"2020-09-08T15:42:42.154099+0000","last_fullsized":"2020-09-08T15:42:42.154099+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":3371,"ondisk_log_size":3371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220921856,"num_objects":3371,"num_object_clones":0,"num_object_copies":6742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3371,"num_write_kb":215744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"99'6","reported_seq":"241","reported_epoch":"225","state":"active+clean","last_fresh":"2020-09-08T15:42:40.665649+0000","last_change":"2020-09-08T15:41:03.497368+0000","last_active":"2020-09-08T15:42:40.665649+0000","last_peered":"2020-09-08T15:42:40.665649+0000","last_clean":"2020-09-08T15:42:40.665649+0000","last_became_active":"2020-09-08T15:34:43.031697+0000","last_became_peered":"2020-09-08T15:34:43.031697+0000","last_unstale":"2020-09-08T15:42:40.665649+0000","last_undegraded":"2020-09-08T15:42:40.665649+0000","last_fullsized":"2020-09-08T15:42:40.665649+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":2,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'6","last_scrub_stamp":"2020-09-08T15:41:03.497248+0000","last_deep_scrub":"99'6","last_deep_scrub_stamp":"2020-09-08T15:35:23.132465+0000","last_clean_scrub_stamp":"2020-09-08T15:41:03.497248+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":3523739648,"num_objects":53768,"num_object_clones":0,"num_object_copies":107536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53768,"num_write_kb":3441152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7164198912,"data_stored":7164198912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53768,"ondisk_log_size":53768,"up":32,"acting":32,"num_store_stats":10},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":2,"acting":2,"num_store_stats":7}],"osd_stats":[{"osd":11,"up_from":0,"seq":100,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1622176,"kb_used_data":1531992,"kb_used_omap":0,"kb_used_meta":90176,"kb_avail":92122976,"statfs":{"total":95995035648,"available":94333927424,"internally_reserved":0,"allocated":1568759808,"data_stored":1568461485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":92340224},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:42:00 2020","interfaces":[{"interface":"back","average":{"1min":2.758,"5min":2.3690000000000002,"15min":2.0489999999999999},"min":{"1min":0.70399999999999996,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":16.419,"5min":33.677999999999997,"15min":33.677999999999997},"last":2.1899999999999999},{"interface":"front","average":{"1min":3.6299999999999999,"5min":3.109,"15min":3.2509999999999999},"min":{"1min":1.085,"5min":1.022,"15min":1.022},"max":{"1min":22.658999999999999,"5min":34.058,"15min":34.058},"last":2.444}]},{"osd":1,"last update":"Tue Sep 8 15:42:00 2020","interfaces":[{"interface":"back","average":{"1min":1.835,"5min":4.2569999999999997,"15min":2.7890000000000001},"min":{"1min":0.74099999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":6.1970000000000001,"5min":163.51499999999999,"15min":163.51499999999999},"last":0.871},{"interface":"front","average":{"1min":2.431,"5min":4.7569999999999997,"15min":3.3260000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":7.0609999999999999,"5min":166.94300000000001,"15min":166.94300000000001},"last":2.3170000000000002}]},{"osd":2,"last update":"Tue Sep 8 15:42:00 2020","interfaces":[{"interface":"back","average":{"1min":2.9500000000000002,"5min":3.2679999999999998,"15min":2.4969999999999999},"min":{"1min":1.038,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":7.5460000000000003,"5min":36.637999999999998,"15min":36.637999999999998},"last":2.0329999999999999},{"interface":"front","average":{"1min":2.6589999999999998,"5min":3.3860000000000001,"15min":3.052},"min":{"1min":1.0069999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":7.5949999999999998,"5min":36.694000000000003,"15min":36.694000000000003},"last":2.113}]},{"osd":3,"last update":"Tue Sep 8 15:42:00 2020","interfaces":[{"interface":"back","average":{"1min":1.887,"5min":3.8759999999999999,"15min":2.8740000000000001},"min":{"1min":0.81299999999999994,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":7.6070000000000002,"5min":147.49100000000001,"15min":147.49100000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":3.081,"5min":4.4160000000000004,"15min":3.5659999999999998},"min":{"1min":1.014,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":7.274,"5min":150.32400000000001,"15min":150.32400000000001},"last":2.673}]},{"osd":4,"last update":"Tue Sep 8 15:42:00 2020","interfaces":[{"interface":"back","average":{"1min":2.6539999999999999,"5min":2.7160000000000002,"15min":3.2989999999999999},"min":{"1min":1.2470000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":7.5590000000000002,"5min":8.0289999999999999,"15min":8.0289999999999999},"last":2.8149999999999999},{"interface":"front","average":{"1min":2.6360000000000001,"5min":2.6720000000000002,"15min":2.7719999999999998},"min":{"1min":0.94399999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":6.3959999999999999,"5min":7.7679999999999998,"15min":7.7679999999999998},"last":5.383}]},{"osd":5,"last update":"Tue Sep 8 15:42:00 2020","interfaces":[{"interface":"back","average":{"1min":2.0990000000000002,"5min":2.633,"15min":3.1669999999999998},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":4.0970000000000004,"5min":6.8499999999999996,"15min":6.8499999999999996},"last":2.9620000000000002},{"interface":"front","average":{"1min":2.9319999999999999,"5min":3.0270000000000001,"15min":3.661},"min":{"1min":1.474,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":7.7510000000000003,"5min":8.5289999999999999,"15min":8.5289999999999999},"last":3.0539999999999998}]},{"osd":6,"last update":"Tue Sep 8 15:42:00 2020","interfaces":[{"interface":"back","average":{"1min":2.125,"5min":2.6840000000000002,"15min":3.1930000000000001},"min":{"1min":1.109,"5min":1.036,"15min":1.036},"max":{"1min":3.839,"5min":7.8479999999999999,"15min":7.8479999999999999},"last":3.0910000000000002},{"interface":"front","average":{"1min":3,"5min":3.0750000000000002,"15min":3.452},"min":{"1min":1.3819999999999999,"5min":1.125,"15min":1.125},"max":{"1min":7.8179999999999996,"5min":8.2110000000000003,"15min":8.2110000000000003},"last":5.2130000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:42:00 2020","interfaces":[{"interface":"back","average":{"1min":2.798,"5min":3.1499999999999999,"15min":3.5590000000000002},"min":{"1min":1.131,"5min":1.131,"15min":1.131},"max":{"1min":6.6239999999999997,"5min":8.4610000000000003,"15min":8.4610000000000003},"last":5.8970000000000002},{"interface":"front","average":{"1min":2.4380000000000002,"5min":3.161,"15min":3.6520000000000001},"min":{"1min":1.411,"5min":0.997,"15min":0.997},"max":{"1min":4.2130000000000001,"5min":9.1820000000000004,"15min":9.1820000000000004},"last":3.2290000000000001}]},{"osd":8,"last update":"Tue Sep 8 15:42:00 2020","interfaces":[{"interface":"back","average":{"1min":4.1029999999999998,"5min":3.7389999999999999,"15min":4.0090000000000003},"min":{"1min":1.4930000000000001,"5min":1.246,"15min":1.246},"max":{"1min":9.8740000000000006,"5min":13.018000000000001,"15min":13.018000000000001},"last":2.9329999999999998},{"interface":"front","average":{"1min":4.3609999999999998,"5min":3.9700000000000002,"15min":3.9249999999999998},"min":{"1min":1.8460000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":9.9939999999999998,"5min":13.592000000000001,"15min":13.592000000000001},"last":3.452}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.2569999999999997}]},{"osd":10,"last update":"Tue Sep 8 15:42:00 2020","interfaces":[{"interface":"back","average":{"1min":3.161,"5min":3.3700000000000001,"15min":4.1130000000000004},"min":{"1min":1.71,"5min":1.3260000000000001,"15min":1.3260000000000001},"max":{"1min":7.6769999999999996,"5min":9.7439999999999998,"15min":9.7439999999999998},"last":3.1909999999999998},{"interface":"front","average":{"1min":3.0219999999999998,"5min":3.306,"15min":3.9649999999999999},"min":{"1min":1.4239999999999999,"5min":1.085,"15min":1.085},"max":{"1min":6.8170000000000002,"5min":8.6600000000000001,"15min":8.6600000000000001},"last":5.766}]}]},{"osd":10,"up_from":0,"seq":107,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":57184,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":56384,"kb_avail":93687968,"statfs":{"total":95995035648,"available":95936479232,"internally_reserved":0,"allocated":811008,"data_stored":512685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":57737216},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7330000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:42:35 2020","interfaces":[{"interface":"back","average":{"1min":1.994,"5min":2.806,"15min":2.4780000000000002},"min":{"1min":0.73299999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":5.5810000000000004,"5min":64.471999999999994,"15min":64.471999999999994},"last":1.1259999999999999},{"interface":"front","average":{"1min":2.0600000000000001,"5min":3.121,"15min":2.7360000000000002},"min":{"1min":0.91100000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":5.516,"5min":64.707999999999998,"15min":64.707999999999998},"last":1.202}]},{"osd":2,"last update":"Tue Sep 8 15:42:35 2020","interfaces":[{"interface":"back","average":{"1min":2.2130000000000001,"5min":3.3580000000000001,"15min":2.637},"min":{"1min":0.753,"5min":0.745,"15min":0.745},"max":{"1min":3.968,"5min":95.197999999999993,"15min":95.197999999999993},"last":1.5609999999999999},{"interface":"front","average":{"1min":2.351,"5min":3.2400000000000002,"15min":2.8940000000000001},"min":{"1min":1.0469999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":6.3899999999999997,"5min":94.778999999999996,"15min":94.778999999999996},"last":1.347}]},{"osd":3,"last update":"Tue Sep 8 15:42:35 2020","interfaces":[{"interface":"back","average":{"1min":2.0219999999999998,"5min":4.5449999999999999,"15min":2.9870000000000001},"min":{"1min":1.1180000000000001,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":3.1200000000000001,"5min":232.87299999999999,"15min":232.87299999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":2.484,"5min":5.1260000000000003,"15min":3.7160000000000002},"min":{"1min":1.038,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":5.7480000000000002,"5min":237.75800000000001,"15min":237.75800000000001},"last":1.6910000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:42:35 2020","interfaces":[{"interface":"back","average":{"1min":2.6989999999999998,"5min":2.4969999999999999,"15min":3.1110000000000002},"min":{"1min":1.069,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":4.4050000000000002,"5min":38.237000000000002,"15min":38.237000000000002},"last":1.7250000000000001},{"interface":"front","average":{"1min":2.5720000000000001,"5min":2.5899999999999999,"15min":2.899},"min":{"1min":1.0109999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":5.4649999999999999,"5min":37.786000000000001,"15min":37.786000000000001},"last":1.599}]},{"osd":5,"last update":"Tue Sep 8 15:42:35 2020","interfaces":[{"interface":"back","average":{"1min":3.0640000000000001,"5min":2.7959999999999998,"15min":3.3050000000000002},"min":{"1min":1.276,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":6.2240000000000002,"5min":36.493000000000002,"15min":36.493000000000002},"last":2.2570000000000001},{"interface":"front","average":{"1min":3.0150000000000001,"5min":2.847,"15min":3.4910000000000001},"min":{"1min":1.204,"5min":1.145,"15min":1.145},"max":{"1min":5.7699999999999996,"5min":38.957999999999998,"15min":38.957999999999998},"last":2.1080000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:42:35 2020","interfaces":[{"interface":"back","average":{"1min":2.8759999999999999,"5min":2.7570000000000001,"15min":3.2879999999999998},"min":{"1min":1.149,"5min":0.998,"15min":0.998},"max":{"1min":5.1820000000000004,"5min":38.537999999999997,"15min":38.537999999999997},"last":1.756},{"interface":"front","average":{"1min":2.8180000000000001,"5min":2.8929999999999998,"15min":3.5329999999999999},"min":{"1min":1.373,"5min":1.1359999999999999,"15min":1.1359999999999999},"max":{"1min":5.202,"5min":37.698999999999998,"15min":37.698999999999998},"last":1.9730000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:42:35 2020","interfaces":[{"interface":"back","average":{"1min":3.4729999999999999,"5min":3.153,"15min":3.6419999999999999},"min":{"1min":1.917,"5min":1.026,"15min":1.026},"max":{"1min":5.6719999999999997,"5min":39.100000000000001,"15min":39.100000000000001},"last":1.877},{"interface":"front","average":{"1min":3.649,"5min":3.2210000000000001,"15min":3.7879999999999998},"min":{"1min":1.8400000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"max":{"1min":6.0279999999999996,"5min":38.131999999999998,"15min":38.131999999999998},"last":2.2330000000000001}]},{"osd":8,"last update":"Tue Sep 8 15:42:35 2020","interfaces":[{"interface":"back","average":{"1min":3.375,"5min":3.3540000000000001,"15min":3.8759999999999999},"min":{"1min":1.3720000000000001,"5min":1.181,"15min":1.181},"max":{"1min":4.7830000000000004,"5min":42.031999999999996,"15min":42.031999999999996},"last":2.4060000000000001},{"interface":"front","average":{"1min":3.9260000000000002,"5min":3.5550000000000002,"15min":3.867},"min":{"1min":1.2789999999999999,"5min":1.113,"15min":1.113},"max":{"1min":9.4600000000000009,"5min":39.512999999999998,"15min":39.512999999999998},"last":2.52}]},{"osd":9,"last update":"Tue Sep 8 15:42:35 2020","interfaces":[{"interface":"back","average":{"1min":4.1189999999999998,"5min":3.6259999999999999,"15min":4.1289999999999996},"min":{"1min":1.411,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":9.3089999999999993,"5min":42.603999999999999,"15min":42.603999999999999},"last":3.3410000000000002},{"interface":"front","average":{"1min":3.734,"5min":3.4630000000000001,"15min":3.9609999999999999},"min":{"1min":1.5029999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"max":{"1min":6.5629999999999997,"5min":40.029000000000003,"15min":40.029000000000003},"last":2.5550000000000002}]},{"osd":11,"last update":"Tue Sep 8 15:42:07 2020","interfaces":[{"interface":"back","average":{"1min":3.266,"5min":3.879,"15min":4.2190000000000003},"min":{"1min":1.264,"5min":1.264,"15min":1.264},"max":{"1min":5.9740000000000002,"5min":39.271000000000001,"15min":39.271000000000001},"last":2.6680000000000001},{"interface":"front","average":{"1min":2.9260000000000002,"5min":3.4900000000000002,"15min":4.1210000000000004},"min":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"max":{"1min":5.2690000000000001,"5min":42.329999999999998,"15min":42.329999999999998},"last":2.7250000000000001}]}]},{"osd":9,"up_from":0,"seq":110,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":507424,"kb_used_data":431320,"kb_used_omap":0,"kb_used_meta":76096,"kb_avail":93237728,"statfs":{"total":95995035648,"available":95475433472,"internally_reserved":0,"allocated":441671680,"data_stored":441373357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77922304},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:42:09 2020","interfaces":[{"interface":"back","average":{"1min":1.988,"5min":3.8820000000000001,"15min":2.5699999999999998},"min":{"1min":0.755,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":4.9950000000000001,"5min":180.84899999999999,"15min":180.84899999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":2.387,"5min":3.8199999999999998,"15min":2.5550000000000002},"min":{"1min":0.72699999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":6.1699999999999999,"5min":175.077,"15min":175.077},"last":1.837}]},{"osd":1,"last update":"Tue Sep 8 15:42:09 2020","interfaces":[{"interface":"back","average":{"1min":1.847,"5min":2.0800000000000001,"15min":2.0179999999999998},"min":{"1min":0.91800000000000004,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":4.0010000000000003,"5min":9.5519999999999996,"15min":9.5519999999999996},"last":0.98199999999999998},{"interface":"front","average":{"1min":2.3660000000000001,"5min":2.431,"15min":2.3599999999999999},"min":{"1min":0.82899999999999996,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":4.1639999999999997,"5min":9.4250000000000007,"15min":9.4250000000000007},"last":1.234}]},{"osd":2,"last update":"Tue Sep 8 15:42:09 2020","interfaces":[{"interface":"back","average":{"1min":3.3610000000000002,"5min":2.4180000000000001,"15min":2.46},"min":{"1min":0.80000000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":14.298,"5min":14.298,"15min":14.298},"last":1.075},{"interface":"front","average":{"1min":2.8820000000000001,"5min":2.3780000000000001,"15min":2.3460000000000001},"min":{"1min":1.1639999999999999,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":6.407,"5min":9.4770000000000003,"15min":9.4770000000000003},"last":1.4890000000000001}]},{"osd":3,"last update":"Tue Sep 8 15:42:12 2020","interfaces":[{"interface":"back","average":{"1min":3.3460000000000001,"5min":2.7240000000000002,"15min":2.4889999999999999},"min":{"1min":1.411,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":9.0180000000000007,"5min":24.562000000000001,"15min":24.562000000000001},"last":2.133},{"interface":"front","average":{"1min":3.0019999999999998,"5min":2.7799999999999998,"15min":2.7080000000000002},"min":{"1min":1.0609999999999999,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":8.5999999999999996,"5min":23.184000000000001,"15min":23.184000000000001},"last":2.3290000000000002}]},{"osd":4,"last update":"Tue Sep 8 15:42:09 2020","interfaces":[{"interface":"back","average":{"1min":3.153,"5min":2.6709999999999998,"15min":2.722},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":6.6150000000000002,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.5489999999999999},{"interface":"front","average":{"1min":3.0270000000000001,"5min":2.6179999999999999,"15min":2.9430000000000001},"min":{"1min":1.4399999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":4.8719999999999999,"5min":10.201000000000001,"15min":10.201000000000001},"last":1.9350000000000001}]},{"osd":5,"last update":"Tue Sep 8 15:42:09 2020","interfaces":[{"interface":"back","average":{"1min":3.3490000000000002,"5min":2.7200000000000002,"15min":3.0390000000000001},"min":{"1min":1.0649999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":9.5869999999999997,"5min":11.097,"15min":11.097},"last":2.476},{"interface":"front","average":{"1min":3.1549999999999998,"5min":2.7709999999999999,"15min":3.0760000000000001},"min":{"1min":1.5680000000000001,"5min":1.0860000000000001,"15min":1.0589999999999999},"max":{"1min":5.0869999999999997,"5min":9.9280000000000008,"15min":9.9280000000000008},"last":1.641}]},{"osd":6,"last update":"Tue Sep 8 15:42:09 2020","interfaces":[{"interface":"back","average":{"1min":3.0339999999999998,"5min":2.746,"15min":3.1200000000000001},"min":{"1min":1.2050000000000001,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":4.5890000000000004,"5min":10.798,"15min":10.798},"last":1.788},{"interface":"front","average":{"1min":3.6179999999999999,"5min":3.0379999999999998,"15min":3.2090000000000001},"min":{"1min":1.2709999999999999,"5min":1.242,"15min":1.242},"max":{"1min":6.726,"5min":11.766,"15min":11.766},"last":2.4209999999999998}]},{"osd":7,"last update":"Tue Sep 8 15:42:09 2020","interfaces":[{"interface":"back","average":{"1min":3.4399999999999999,"5min":3.0510000000000002,"15min":3.355},"min":{"1min":1.627,"5min":1.2110000000000001,"15min":1.2110000000000001},"max":{"1min":5.3209999999999997,"5min":10.31,"15min":10.31},"last":86.463999999999999},{"interface":"front","average":{"1min":3.786,"5min":3.323,"15min":3.5800000000000001},"min":{"1min":1.9079999999999999,"5min":1.593,"15min":1.593},"max":{"1min":5.5890000000000004,"5min":10.837999999999999,"15min":10.837999999999999},"last":86.656000000000006}]},{"osd":8,"last update":"Tue Sep 8 15:42:09 2020","interfaces":[{"interface":"back","average":{"1min":4.1269999999999998,"5min":3.4660000000000002,"15min":3.331},"min":{"1min":1.595,"5min":1.264,"15min":1.264},"max":{"1min":9.0429999999999993,"5min":11,"15min":11},"last":2.169},{"interface":"front","average":{"1min":3.9620000000000002,"5min":3.3999999999999999,"15min":3.4100000000000001},"min":{"1min":1.778,"5min":1.262,"15min":1.262},"max":{"1min":9.3520000000000003,"5min":10.977,"15min":10.977},"last":2.581}]},{"osd":10,"last update":"Tue Sep 8 15:42:31 2020","interfaces":[{"interface":"back","average":{"1min":3.6099999999999999,"5min":3.1309999999999998,"15min":3.6880000000000002},"min":{"1min":1.5760000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"max":{"1min":7.7939999999999996,"5min":8.8200000000000003,"15min":8.8200000000000003},"last":2.6219999999999999},{"interface":"front","average":{"1min":3.3069999999999999,"5min":2.956,"15min":3.54},"min":{"1min":1.6599999999999999,"5min":1.165,"15min":1.165},"max":{"1min":5.2720000000000002,"5min":11.625,"15min":11.625},"last":2.0800000000000001}]}]},{"osd":8,"up_from":0,"seq":116,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308320,"kb_used_data":219032,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":93436832,"statfs":{"total":95995035648,"available":95679315968,"internally_reserved":0,"allocated":224288768,"data_stored":223990445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:41:40 2020","interfaces":[{"interface":"back","average":{"1min":2.5579999999999998,"5min":2.1970000000000001,"15min":2.1960000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":11.154999999999999,"5min":11.154999999999999,"15min":11.154999999999999},"last":1.7909999999999999},{"interface":"front","average":{"1min":1.968,"5min":2.3559999999999999,"15min":2.4180000000000001},"min":{"1min":0.73399999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":5.0709999999999997,"5min":13.773999999999999,"15min":13.773999999999999},"last":1.224}]},{"osd":1,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.492,"5min":2.0659999999999998,"15min":1.8839999999999999},"min":{"1min":1.4690000000000001,"5min":1.137,"15min":1.137},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":1.4350000000000001},{"interface":"front","average":{"1min":2.7200000000000002,"5min":2.427,"15min":2.347},"min":{"1min":1.351,"5min":1.046,"15min":1.046},"max":{"1min":4.444,"5min":4.444,"15min":4.444},"last":1.8600000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:41:40 2020","interfaces":[{"interface":"back","average":{"1min":8.2219999999999995,"5min":5.5800000000000001,"15min":3.1059999999999999},"min":{"1min":0.79700000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":115.011,"5min":145.893,"15min":145.893},"last":0.79700000000000004},{"interface":"front","average":{"1min":8.3119999999999994,"5min":5.1070000000000002,"15min":3.3500000000000001},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":114.161,"5min":153.91900000000001,"15min":153.91900000000001},"last":1.6539999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:41:40 2020","interfaces":[{"interface":"back","average":{"1min":3.3980000000000001,"5min":4.1719999999999997,"15min":3.2240000000000002},"min":{"1min":1.0169999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":11.539999999999999,"5min":125.111,"15min":125.111},"last":2.0579999999999998},{"interface":"front","average":{"1min":2.0249999999999999,"5min":3.8639999999999999,"15min":2.883},"min":{"1min":0.91000000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":5.181,"5min":126.124,"15min":126.124},"last":1.75}]},{"osd":4,"last update":"Tue Sep 8 15:41:40 2020","interfaces":[{"interface":"back","average":{"1min":2.8380000000000001,"5min":2.9209999999999998,"15min":2.9089999999999998},"min":{"1min":0.94899999999999995,"5min":0.872,"15min":0.872},"max":{"1min":11.404999999999999,"5min":11.404999999999999,"15min":11.404999999999999},"last":2.4609999999999999},{"interface":"front","average":{"1min":2.0760000000000001,"5min":2.5710000000000002,"15min":2.5840000000000001},"min":{"1min":1.0880000000000001,"5min":0.874,"15min":0.874},"max":{"1min":4.0469999999999997,"5min":13.999000000000001,"15min":13.999000000000001},"last":1.905}]},{"osd":5,"last update":"Tue Sep 8 15:41:40 2020","interfaces":[{"interface":"back","average":{"1min":2.907,"5min":3.1400000000000001,"15min":3.177},"min":{"1min":1.02,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":9.6349999999999998,"5min":9.6349999999999998,"15min":9.6349999999999998},"last":1.1639999999999999},{"interface":"front","average":{"1min":3.0899999999999999,"5min":3.194,"15min":3.3199999999999998},"min":{"1min":1.071,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":11.885999999999999,"5min":11.885999999999999,"15min":11.885999999999999},"last":1.9239999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:41:40 2020","interfaces":[{"interface":"back","average":{"1min":2.6629999999999998,"5min":3.323,"15min":3.3100000000000001},"min":{"1min":1.238,"5min":1.1299999999999999,"15min":1.1299999999999999},"max":{"1min":9.968,"5min":13.887,"15min":13.887},"last":2.0310000000000001},{"interface":"front","average":{"1min":3.2429999999999999,"5min":2.9390000000000001,"15min":2.9489999999999998},"min":{"1min":1.359,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":11.775,"5min":12.502000000000001,"15min":12.502000000000001},"last":2.3199999999999998}]},{"osd":7,"last update":"Tue Sep 8 15:41:40 2020","interfaces":[{"interface":"back","average":{"1min":3.3130000000000002,"5min":3.5910000000000002,"15min":3.6299999999999999},"min":{"1min":1.3029999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"max":{"1min":11.661,"5min":11.661,"15min":11.661},"last":2.798},{"interface":"front","average":{"1min":2.823,"5min":3.2650000000000001,"15min":3.2719999999999998},"min":{"1min":1.4770000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"max":{"1min":10.234,"5min":14.109999999999999,"15min":14.109999999999999},"last":2.4409999999999998}]},{"osd":9,"last update":"Tue Sep 8 15:42:07 2020","interfaces":[{"interface":"back","average":{"1min":3.6099999999999999,"5min":3.8109999999999999,"15min":3.597},"min":{"1min":1.5549999999999999,"5min":1.095,"15min":1.095},"max":{"1min":6.9320000000000004,"5min":13.506,"15min":13.506},"last":1.304},{"interface":"front","average":{"1min":3.403,"5min":3.6539999999999999,"15min":3.536},"min":{"1min":1.208,"5min":1.004,"15min":1.004},"max":{"1min":5.2519999999999998,"5min":12.052,"15min":12.052},"last":2.1949999999999998}]},{"osd":10,"last update":"Tue Sep 8 15:41:40 2020","interfaces":[{"interface":"back","average":{"1min":3.02,"5min":3.2090000000000001,"15min":3.5590000000000002},"min":{"1min":1.4730000000000001,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":10.101000000000001,"5min":13.656000000000001,"15min":13.656000000000001},"last":2.1600000000000001},{"interface":"front","average":{"1min":3.681,"5min":3.726,"15min":3.9609999999999999},"min":{"1min":1.3149999999999999,"5min":1.296,"15min":1.296},"max":{"1min":13.638,"5min":13.638,"15min":13.638},"last":1.3959999999999999}]},{"osd":11,"last update":"Tue Sep 8 15:42:38 2020","interfaces":[{"interface":"back","average":{"1min":3.738,"5min":3.6000000000000001,"15min":3.9020000000000001},"min":{"1min":1.9419999999999999,"5min":1.214,"15min":1.214},"max":{"1min":5.6689999999999996,"5min":14.221,"15min":14.221},"last":2.2959999999999998},{"interface":"front","average":{"1min":3.7759999999999998,"5min":3.9529999999999998,"15min":4.6379999999999999},"min":{"1min":2.6269999999999998,"5min":1.248,"15min":1.248},"max":{"1min":5.5609999999999999,"5min":11.186999999999999,"15min":11.186999999999999},"last":1.4630000000000001}]}]},{"osd":7,"up_from":0,"seq":121,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":511456,"kb_used_data":443224,"kb_used_omap":0,"kb_used_meta":68224,"kb_avail":93233696,"statfs":{"total":95995035648,"available":95471304704,"internally_reserved":0,"allocated":453861376,"data_stored":453563073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":69861376},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:02 2020","interfaces":[{"interface":"back","average":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"min":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"max":{"1min":4.0190000000000001,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":2.2189999999999999},{"interface":"front","average":{"1min":2.6259999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"min":{"1min":1.542,"5min":1.542,"15min":1.542},"max":{"1min":4.5229999999999997,"5min":4.5229999999999997,"15min":4.5229999999999997},"last":2.3079999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:42:21 2020","interfaces":[{"interface":"back","average":{"1min":6.359,"5min":2.8220000000000001,"15min":2.5289999999999999},"min":{"1min":0.78700000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":109.97,"5min":109.97,"15min":109.97},"last":1.861},{"interface":"front","average":{"1min":6.7140000000000004,"5min":2.9260000000000002,"15min":2.5459999999999998},"min":{"1min":1.0880000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":107.093,"5min":107.093,"15min":107.093},"last":1.3220000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:42:21 2020","interfaces":[{"interface":"back","average":{"1min":1.9390000000000001,"5min":5.5700000000000003,"15min":3.5139999999999998},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":3.9700000000000002,"5min":324.77100000000002,"15min":324.77100000000002},"last":3.379},{"interface":"front","average":{"1min":2.2970000000000002,"5min":5.694,"15min":3.794},"min":{"1min":0.93500000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":4.3419999999999996,"5min":321.48500000000001,"15min":321.48500000000001},"last":1.633}]},{"osd":3,"last update":"Tue Sep 8 15:42:21 2020","interfaces":[{"interface":"back","average":{"1min":2.4929999999999999,"5min":2.831,"15min":2.8809999999999998},"min":{"1min":1.1439999999999999,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":3.8889999999999998,"5min":37.801000000000002,"15min":37.801000000000002},"last":4.0839999999999996},{"interface":"front","average":{"1min":2.597,"5min":3.5870000000000002,"15min":3.3359999999999999},"min":{"1min":1.3340000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":4.1230000000000002,"5min":45.284999999999997,"15min":45.284999999999997},"last":4.1520000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:42:21 2020","interfaces":[{"interface":"back","average":{"1min":2.7069999999999999,"5min":2.27,"15min":2.7509999999999999},"min":{"1min":1.321,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":4.1150000000000002,"5min":8.5779999999999994,"15min":8.5779999999999994},"last":1.4530000000000001},{"interface":"front","average":{"1min":3.1680000000000001,"5min":2.8330000000000002,"15min":3.1989999999999998},"min":{"1min":1.4970000000000001,"5min":1.127,"15min":1.127},"max":{"1min":5.3719999999999999,"5min":7.3570000000000002,"15min":7.3570000000000002},"last":3.218}]},{"osd":5,"last update":"Tue Sep 8 15:42:21 2020","interfaces":[{"interface":"back","average":{"1min":3.1539999999999999,"5min":2.7189999999999999,"15min":3.3679999999999999},"min":{"1min":1.452,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":5.0899999999999999,"5min":9.0800000000000001,"15min":9.0800000000000001},"last":1.3640000000000001},{"interface":"front","average":{"1min":3.0329999999999999,"5min":2.758,"15min":3.2690000000000001},"min":{"1min":1.708,"5min":1.1779999999999999,"15min":1.1779999999999999},"max":{"1min":4.7290000000000001,"5min":6.0229999999999997,"15min":6.0229999999999997},"last":1.8089999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:42:21 2020","interfaces":[{"interface":"back","average":{"1min":3.2370000000000001,"5min":2.9510000000000001,"15min":3.6030000000000002},"min":{"1min":1.8180000000000001,"5min":1.304,"15min":1.304},"max":{"1min":4.5970000000000004,"5min":8.8300000000000001,"15min":8.8300000000000001},"last":1.569},{"interface":"front","average":{"1min":3.4540000000000002,"5min":3.1549999999999998,"15min":3.6909999999999998},"min":{"1min":1.694,"5min":1.3939999999999999,"15min":1.3939999999999999},"max":{"1min":5.2149999999999999,"5min":7.681,"15min":7.681},"last":3.5390000000000001}]},{"osd":8,"last update":"Tue Sep 8 15:41:44 2020","interfaces":[{"interface":"back","average":{"1min":2.6909999999999998,"5min":3.2799999999999998,"15min":3.8769999999999998},"min":{"1min":1.9019999999999999,"5min":1.3340000000000001,"15min":1.3340000000000001},"max":{"1min":4.399,"5min":8.2260000000000009,"15min":8.2260000000000009},"last":2.4900000000000002},{"interface":"front","average":{"1min":2.75,"5min":3.327,"15min":3.9849999999999999},"min":{"1min":1.994,"5min":1.6319999999999999,"15min":1.6319999999999999},"max":{"1min":3.9860000000000002,"5min":9.2119999999999997,"15min":9.2119999999999997},"last":2.29}]},{"osd":9,"last update":"Tue Sep 8 15:42:07 2020","interfaces":[{"interface":"back","average":{"1min":3.6099999999999999,"5min":3.5960000000000001,"15min":4.383},"min":{"1min":1.8819999999999999,"5min":1.7170000000000001,"15min":1.7170000000000001},"max":{"1min":6.1100000000000003,"5min":7.476,"15min":7.476},"last":3.7010000000000001},{"interface":"front","average":{"1min":3.399,"5min":3.2789999999999999,"15min":4.0220000000000002},"min":{"1min":2.0430000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"max":{"1min":5.2770000000000001,"5min":7.7460000000000004,"15min":7.7460000000000004},"last":1.968}]},{"osd":10,"last update":"Tue Sep 8 15:42:34 2020","interfaces":[{"interface":"back","average":{"1min":3.798,"5min":3.4340000000000002,"15min":4.2480000000000002},"min":{"1min":2.0190000000000001,"5min":1.5309999999999999,"15min":1.5309999999999999},"max":{"1min":5.3470000000000004,"5min":9.5009999999999994,"15min":9.5009999999999994},"last":3.839},{"interface":"front","average":{"1min":3.7250000000000001,"5min":3.3380000000000001,"15min":4.5060000000000002},"min":{"1min":2.0579999999999998,"5min":1.583,"15min":1.583},"max":{"1min":5.3730000000000002,"5min":7.5720000000000001,"15min":8.3650000000000002},"last":2.4420000000000002}]},{"osd":11,"last update":"Tue Sep 8 15:41:54 2020","interfaces":[{"interface":"back","average":{"1min":3.5960000000000001,"5min":3.7400000000000002,"15min":4.4450000000000003},"min":{"1min":1.8,"5min":1.7010000000000001,"15min":1.7010000000000001},"max":{"1min":5.944,"5min":9.1790000000000003,"15min":9.1790000000000003},"last":4.0049999999999999},{"interface":"front","average":{"1min":3.7189999999999999,"5min":4.0289999999999999,"15min":4.867},"min":{"1min":2.1259999999999999,"5min":1.756,"15min":1.756},"max":{"1min":7.5359999999999996,"5min":7.7629999999999999,"15min":7.7629999999999999},"last":2.1259999999999999}]}]},{"osd":6,"up_from":0,"seq":125,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":240288,"kb_used_data":209304,"kb_used_omap":0,"kb_used_meta":30976,"kb_avail":93504864,"statfs":{"total":95995035648,"available":95748980736,"internally_reserved":0,"allocated":214327296,"data_stored":214028993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31719424},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:41:58 2020","interfaces":[{"interface":"back","average":{"1min":1.673,"5min":1.5600000000000001,"15min":1.841},"min":{"1min":0.78000000000000003,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":4.665,"5min":4.665,"15min":4.665},"last":1.927},{"interface":"front","average":{"1min":2.23,"5min":1.8220000000000001,"15min":1.964},"min":{"1min":0.89800000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":4.7329999999999997,"5min":4.7329999999999997,"15min":4.7329999999999997},"last":3.7320000000000002}]},{"osd":1,"last update":"Tue Sep 8 15:41:58 2020","interfaces":[{"interface":"back","average":{"1min":2.4689999999999999,"5min":3.9700000000000002,"15min":2.7799999999999998},"min":{"1min":0.98999999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":5.9489999999999998,"5min":149.34200000000001,"15min":149.34200000000001},"last":1.8640000000000001},{"interface":"front","average":{"1min":2.2629999999999999,"5min":4.016,"15min":3.024},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":6.1619999999999999,"5min":149.03899999999999,"15min":149.03899999999999},"last":2.028}]},{"osd":2,"last update":"Tue Sep 8 15:41:58 2020","interfaces":[{"interface":"back","average":{"1min":2.2480000000000002,"5min":8.5939999999999994,"15min":4.5350000000000001},"min":{"1min":1.0329999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":5.1379999999999999,"5min":549.75999999999999,"15min":549.75999999999999},"last":2.3460000000000001},{"interface":"front","average":{"1min":2.391,"5min":8.6679999999999993,"15min":4.5590000000000002},"min":{"1min":1.1719999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":4.6840000000000002,"5min":549.85900000000004,"15min":549.85900000000004},"last":3.1760000000000002}]},{"osd":3,"last update":"Tue Sep 8 15:41:58 2020","interfaces":[{"interface":"back","average":{"1min":2.883,"5min":3.6200000000000001,"15min":2.9180000000000001},"min":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"max":{"1min":4.6449999999999996,"5min":109.65300000000001,"15min":109.65300000000001},"last":170.35400000000001},{"interface":"front","average":{"1min":2.9079999999999999,"5min":3.6749999999999998,"15min":3.222},"min":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"max":{"1min":5.5540000000000003,"5min":109.51600000000001,"15min":109.51600000000001},"last":170.31899999999999}]},{"osd":4,"last update":"Tue Sep 8 15:42:01 2020","interfaces":[{"interface":"back","average":{"1min":2.6059999999999999,"5min":2.5190000000000001,"15min":2.7160000000000002},"min":{"1min":1.248,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":5.048,"5min":5.048,"15min":5.048},"last":4.9109999999999996},{"interface":"front","average":{"1min":2.8860000000000001,"5min":2.8300000000000001,"15min":2.9540000000000002},"min":{"1min":1.4790000000000001,"5min":1.052,"15min":1.052},"max":{"1min":5.2149999999999999,"5min":22.062000000000001,"15min":22.062000000000001},"last":3.6440000000000001}]},{"osd":5,"last update":"Tue Sep 8 15:41:58 2020","interfaces":[{"interface":"back","average":{"1min":3.1779999999999999,"5min":2.9399999999999999,"15min":3.173},"min":{"1min":1.1140000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"max":{"1min":7.2560000000000002,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":4.2149999999999999},{"interface":"front","average":{"1min":3.149,"5min":3.3029999999999999,"15min":3.2970000000000002},"min":{"1min":1.1499999999999999,"5min":1.149,"15min":1.149},"max":{"1min":5.6029999999999998,"5min":21.228999999999999,"15min":21.228999999999999},"last":4.327}]},{"osd":7,"last update":"Tue Sep 8 15:42:15 2020","interfaces":[{"interface":"back","average":{"1min":3.1150000000000002,"5min":3.2109999999999999,"15min":3.577},"min":{"1min":1.1200000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":5.0510000000000002,"5min":22.911999999999999,"15min":22.911999999999999},"last":3.0390000000000001},{"interface":"front","average":{"1min":3.609,"5min":3.4929999999999999,"15min":3.7970000000000002},"min":{"1min":1.6870000000000001,"5min":1.3919999999999999,"15min":1.3919999999999999},"max":{"1min":5.9530000000000003,"5min":26.056000000000001,"15min":26.056000000000001},"last":4.7519999999999998}]},{"osd":8,"last update":"Tue Sep 8 15:41:39 2020","interfaces":[{"interface":"back","average":{"1min":2.4390000000000001,"5min":3.0609999999999999,"15min":3.6739999999999999},"min":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"max":{"1min":3.9489999999999998,"5min":6.3479999999999999,"15min":6.3479999999999999},"last":5.569},{"interface":"front","average":{"1min":2.8050000000000002,"5min":3.1779999999999999,"15min":3.8610000000000002},"min":{"1min":1.341,"5min":1.1299999999999999,"15min":1.1299999999999999},"max":{"1min":5.3959999999999999,"5min":8.6850000000000005,"15min":8.6850000000000005},"last":3.5409999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:42:06 2020","interfaces":[{"interface":"back","average":{"1min":3.476,"5min":3.5630000000000002,"15min":4.1829999999999998},"min":{"1min":1.577,"5min":1.5569999999999999,"15min":1.5569999999999999},"max":{"1min":5.7889999999999997,"5min":21.716000000000001,"15min":21.716000000000001},"last":6.3390000000000004},{"interface":"front","average":{"1min":3.1219999999999999,"5min":3.1669999999999998,"15min":4.0759999999999996},"min":{"1min":1.4590000000000001,"5min":1.224,"15min":1.224},"max":{"1min":6.5940000000000003,"5min":7.0250000000000004,"15min":7.0250000000000004},"last":5.3689999999999998}]},{"osd":10,"last update":"Tue Sep 8 15:42:31 2020","interfaces":[{"interface":"back","average":{"1min":7.5179999999999998,"5min":4.0060000000000002,"15min":4.4729999999999999},"min":{"1min":1.7529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"max":{"1min":84.159000000000006,"5min":84.159000000000006,"15min":84.159000000000006},"last":5.9400000000000004},{"interface":"front","average":{"1min":7.9000000000000004,"5min":4.2930000000000001,"15min":4.5259999999999998},"min":{"1min":1.9850000000000001,"5min":1.915,"15min":1.915},"max":{"1min":85.475999999999999,"5min":85.475999999999999,"15min":85.475999999999999},"last":6.0890000000000004}]}]},{"osd":1,"up_from":0,"seq":149,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":0,"seq":154,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":862036,"kb_used_data":787852,"kb_used_omap":0,"kb_used_meta":74176,"kb_avail":92883116,"statfs":{"total":95995035648,"available":95112310784,"internally_reserved":0,"allocated":806760448,"data_stored":806466553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":75956224},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 8 15:41:51 2020","interfaces":[{"interface":"back","average":{"1min":7.5579999999999998,"5min":4.3769999999999998,"15min":2.5},"min":{"1min":0.77600000000000002,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":69.259,"5min":74.831999999999994,"15min":74.831999999999994},"last":2.2130000000000001},{"interface":"front","average":{"1min":6.1280000000000001,"5min":4.1719999999999997,"15min":2.4430000000000001},"min":{"1min":0.80200000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":66.754999999999995,"5min":72.611000000000004,"15min":72.611000000000004},"last":0.73499999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:34:05 2020","interfaces":[{"interface":"back","average":{"1min":1.9950000000000001,"5min":1.621,"15min":1.5189999999999999},"min":{"1min":1.3999999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.8519999999999999,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":2.6589999999999998},{"interface":"front","average":{"1min":2.629,"5min":1.956,"15min":1.72},"min":{"1min":1.6419999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":2.4049999999999998}]},{"osd":3,"last update":"Tue Sep 8 15:41:41 2020","interfaces":[{"interface":"back","average":{"1min":6.3630000000000004,"5min":3.2040000000000002,"15min":3},"min":{"1min":0.999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":53.308,"5min":53.308,"15min":53.308},"last":3.093},{"interface":"front","average":{"1min":7.4500000000000002,"5min":4.6879999999999997,"15min":3.9350000000000001},"min":{"1min":1.3480000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":67.244,"5min":73.075000000000003,"15min":73.075000000000003},"last":3.238}]},{"osd":4,"last update":"Tue Sep 8 15:42:05 2020","interfaces":[{"interface":"back","average":{"1min":2.5190000000000001,"5min":3.0939999999999999,"15min":2.806},"min":{"1min":1.0820000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":5.1369999999999996,"5min":67.105999999999995,"15min":67.105999999999995},"last":3.3490000000000002},{"interface":"front","average":{"1min":2.7000000000000002,"5min":4.0010000000000003,"15min":3.2949999999999999},"min":{"1min":1.107,"5min":0.995,"15min":0.995},"max":{"1min":5.2069999999999999,"5min":73.802999999999997,"15min":73.802999999999997},"last":2.3690000000000002}]},{"osd":5,"last update":"Tue Sep 8 15:42:27 2020","interfaces":[{"interface":"back","average":{"1min":2.6139999999999999,"5min":3.548,"15min":3.3149999999999999},"min":{"1min":1.4219999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":4.7329999999999997,"5min":66.950999999999993,"15min":66.950999999999993},"last":2.552},{"interface":"front","average":{"1min":2.1099999999999999,"5min":2.923,"15min":3.02},"min":{"1min":1.0720000000000001,"5min":0.75,"15min":0.75},"max":{"1min":3.3050000000000002,"5min":66.435000000000002,"15min":66.435000000000002},"last":1.1439999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:41:51 2020","interfaces":[{"interface":"back","average":{"1min":6.8620000000000001,"5min":3.25,"15min":3.3900000000000001},"min":{"1min":1.2569999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":54.665999999999997,"5min":54.665999999999997,"15min":54.665999999999997},"last":1.282},{"interface":"front","average":{"1min":6.851,"5min":3.4990000000000001,"15min":3.7400000000000002},"min":{"1min":1.1379999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":53.789000000000001,"5min":53.789000000000001,"15min":53.789000000000001},"last":2.0600000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:42:27 2020","interfaces":[{"interface":"back","average":{"1min":2.9929999999999999,"5min":4.4500000000000002,"15min":4.2380000000000004},"min":{"1min":1.6240000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"max":{"1min":5.1159999999999997,"5min":71.167000000000002,"15min":71.167000000000002},"last":2.472},{"interface":"front","average":{"1min":2.6509999999999998,"5min":4.8040000000000003,"15min":4.3449999999999998},"min":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"max":{"1min":4.6989999999999998,"5min":74.480000000000004,"15min":74.480000000000004},"last":1.4099999999999999}]},{"osd":8,"last update":"Tue Sep 8 15:41:41 2020","interfaces":[{"interface":"back","average":{"1min":7.8209999999999997,"5min":4.1920000000000002,"15min":4.0570000000000004},"min":{"1min":1.7230000000000001,"5min":1.093,"15min":1.093},"max":{"1min":67.528000000000006,"5min":67.528000000000006,"15min":67.528000000000006},"last":2.7040000000000002},{"interface":"front","average":{"1min":5.625,"5min":5.923,"15min":4.6920000000000002},"min":{"1min":1.46,"5min":1.2070000000000001,"15min":1.2070000000000001},"max":{"1min":53.405000000000001,"5min":131.958,"15min":131.958},"last":1.593}]},{"osd":9,"last update":"Tue Sep 8 15:42:05 2020","interfaces":[{"interface":"back","average":{"1min":3.1909999999999998,"5min":3.9929999999999999,"15min":4.2229999999999999},"min":{"1min":1.577,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":6.0410000000000004,"5min":54.292999999999999,"15min":54.292999999999999},"last":2.827},{"interface":"front","average":{"1min":3.516,"5min":4.0369999999999999,"15min":4.0350000000000001},"min":{"1min":1.7849999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":6.0739999999999998,"5min":67.659999999999997,"15min":67.659999999999997},"last":2.1890000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:42:27 2020","interfaces":[{"interface":"back","average":{"1min":2.1499999999999999,"5min":3.2970000000000002,"15min":3.2160000000000002},"min":{"1min":1.0600000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":4.0149999999999997,"5min":55.441000000000003,"15min":55.441000000000003},"last":1.111},{"interface":"front","average":{"1min":3.331,"5min":4.8650000000000002,"15min":4.6079999999999997},"min":{"1min":1.679,"5min":1.1739999999999999,"15min":1.1739999999999999},"max":{"1min":5.1050000000000004,"5min":73.444000000000003,"15min":73.444000000000003},"last":2.6259999999999999}]},{"osd":11,"last update":"Tue Sep 8 15:41:51 2020","interfaces":[{"interface":"back","average":{"1min":7.9290000000000003,"5min":5.4050000000000002,"15min":4.4109999999999996},"min":{"1min":1.784,"5min":1.476,"15min":1.476},"max":{"1min":67.801000000000002,"5min":74.147000000000006,"15min":74.147000000000006},"last":2.956},{"interface":"front","average":{"1min":6.4429999999999996,"5min":4.1859999999999999,"15min":4.6360000000000001},"min":{"1min":1.5680000000000001,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":66.613,"5min":66.613,"15min":66.613},"last":2.3210000000000002}]}]},{"osd":2,"up_from":0,"seq":146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":66580,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":65792,"kb_avail":93678572,"statfs":{"total":95995035648,"available":95926857728,"internally_reserved":0,"allocated":798720,"data_stored":504825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":67371008},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:42:12 2020","interfaces":[{"interface":"back","average":{"1min":1.8799999999999999,"5min":3.4159999999999999,"15min":2.3220000000000001},"min":{"1min":0.80400000000000005,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":3.25,"5min":162.83799999999999,"15min":162.83799999999999},"last":1.855},{"interface":"front","average":{"1min":1.9359999999999999,"5min":3.5859999999999999,"15min":2.4079999999999999},"min":{"1min":0.78400000000000003,"5min":0.624,"15min":0.624},"max":{"1min":5.3650000000000002,"5min":162.63499999999999,"15min":162.63499999999999},"last":2.5019999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:42:12 2020","interfaces":[{"interface":"back","average":{"1min":2.0009999999999999,"5min":3.8100000000000001,"15min":2.4729999999999999},"min":{"1min":0.876,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":9.016,"5min":78.728999999999999,"15min":78.728999999999999},"last":1.9179999999999999},{"interface":"front","average":{"1min":3.1099999999999999,"5min":3.496,"15min":2.6680000000000001},"min":{"1min":0.99299999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":9.0090000000000003,"5min":80.103999999999999,"15min":80.103999999999999},"last":2.4100000000000001}]},{"osd":3,"last update":"Tue Sep 8 15:41:40 2020","interfaces":[{"interface":"back","average":{"1min":3.617,"5min":2.992,"15min":2.6379999999999999},"min":{"1min":1.0660000000000001,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":11.643000000000001,"5min":11.643000000000001,"15min":11.643000000000001},"last":276.34399999999999},{"interface":"front","average":{"1min":3.4289999999999998,"5min":2.7810000000000001,"15min":2.5379999999999998},"min":{"1min":0.82299999999999995,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":10.925000000000001,"5min":11.737,"15min":11.737},"last":276.60000000000002}]},{"osd":4,"last update":"Tue Sep 8 15:42:07 2020","interfaces":[{"interface":"back","average":{"1min":2.6930000000000001,"5min":2.6019999999999999,"15min":2.5840000000000001},"min":{"1min":1.0580000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":6.9500000000000002,"5min":7.6550000000000002,"15min":7.6550000000000002},"last":2.056},{"interface":"front","average":{"1min":2.101,"5min":3.2930000000000001,"15min":2.7930000000000001},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":3.1699999999999999,"5min":64.284999999999997,"15min":64.284999999999997},"last":2.863}]},{"osd":5,"last update":"Tue Sep 8 15:42:32 2020","interfaces":[{"interface":"back","average":{"1min":2.29,"5min":3.4980000000000002,"15min":3.2149999999999999},"min":{"1min":1.091,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":4.0860000000000003,"5min":63.509,"15min":63.509},"last":2.6160000000000001},{"interface":"front","average":{"1min":2.6539999999999999,"5min":2.8250000000000002,"15min":3.0409999999999999},"min":{"1min":1.1899999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"max":{"1min":4.2329999999999997,"5min":11.117000000000001,"15min":11.117000000000001},"last":2.9950000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:41:51 2020","interfaces":[{"interface":"back","average":{"1min":2.6269999999999998,"5min":3.819,"15min":3.4369999999999998},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":5.6280000000000001,"5min":63.631,"15min":63.631},"last":3.1110000000000002},{"interface":"front","average":{"1min":3.3170000000000002,"5min":3.2109999999999999,"15min":3.4830000000000001},"min":{"1min":1.1240000000000001,"5min":1.105,"15min":1.105},"max":{"1min":11.492000000000001,"5min":11.492000000000001,"15min":11.492000000000001},"last":2.7879999999999998}]},{"osd":7,"last update":"Tue Sep 8 15:42:26 2020","interfaces":[{"interface":"back","average":{"1min":2.8900000000000001,"5min":3.2930000000000001,"15min":3.8260000000000001},"min":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"max":{"1min":6.1749999999999998,"5min":11.436,"15min":11.436},"last":2.738},{"interface":"front","average":{"1min":3.5409999999999999,"5min":3.2949999999999999,"15min":3.5710000000000002},"min":{"1min":1.714,"5min":1.006,"15min":1.006},"max":{"1min":7.3789999999999996,"5min":10.784000000000001,"15min":10.784000000000001},"last":3.3940000000000001}]},{"osd":8,"last update":"Tue Sep 8 15:42:36 2020","interfaces":[{"interface":"back","average":{"1min":2.52,"5min":3.8260000000000001,"15min":3.7320000000000002},"min":{"1min":1.3600000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":3.7759999999999998,"5min":64.116,"15min":64.116},"last":3.3359999999999999},{"interface":"front","average":{"1min":3.355,"5min":3.847,"15min":3.9590000000000001},"min":{"1min":1.7430000000000001,"5min":1.218,"15min":1.218},"max":{"1min":5.2960000000000003,"5min":63.299999999999997,"15min":63.299999999999997},"last":3.1960000000000002}]},{"osd":9,"last update":"Tue Sep 8 15:42:12 2020","interfaces":[{"interface":"back","average":{"1min":3.0270000000000001,"5min":4.1900000000000004,"15min":4.0970000000000004},"min":{"1min":1.6160000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"max":{"1min":7.3849999999999998,"5min":64.004000000000005,"15min":64.004000000000005},"last":3.5419999999999998},{"interface":"front","average":{"1min":3.6539999999999999,"5min":3.5680000000000001,"15min":4.0419999999999998},"min":{"1min":1.556,"5min":1.401,"15min":1.401},"max":{"1min":8.0239999999999991,"5min":11.869,"15min":11.869},"last":3.5800000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:42:32 2020","interfaces":[{"interface":"back","average":{"1min":2.7549999999999999,"5min":4.0800000000000001,"15min":4.0759999999999996},"min":{"1min":1.5860000000000001,"5min":1.4339999999999999,"15min":1.4339999999999999},"max":{"1min":4.4509999999999996,"5min":63.743000000000002,"15min":63.743000000000002},"last":3.7330000000000001},{"interface":"front","average":{"1min":3.0390000000000001,"5min":3.1520000000000001,"15min":3.5619999999999998},"min":{"1min":1.47,"5min":1.159,"15min":1.159},"max":{"1min":6.2729999999999997,"5min":11.067,"15min":11.067},"last":3.2389999999999999}]}]},{"osd":3,"up_from":0,"seq":139,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1113248,"kb_used_data":1034264,"kb_used_omap":0,"kb_used_meta":78976,"kb_avail":92631904,"statfs":{"total":95995035648,"available":94855069696,"internally_reserved":0,"allocated":1059086336,"data_stored":1058788033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":80871424},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:31 2020","interfaces":[{"interface":"back","average":{"1min":2.1379999999999999,"5min":1.7210000000000001,"15min":1.629},"min":{"1min":1.79,"5min":1.1419999999999999,"15min":1.1419999999999999},"max":{"1min":2.7930000000000001,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":1.167},{"interface":"front","average":{"1min":2.6800000000000002,"5min":2.2309999999999999,"15min":2.0990000000000002},"min":{"1min":2.0270000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"max":{"1min":3.5409999999999999,"5min":3.5409999999999999,"15min":3.5409999999999999},"last":1.7290000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:41:44 2020","interfaces":[{"interface":"back","average":{"1min":2.7229999999999999,"5min":3.028,"15min":2.2949999999999999},"min":{"1min":0.77400000000000002,"5min":0.628,"15min":0.628},"max":{"1min":5.4189999999999996,"5min":44.109999999999999,"15min":44.109999999999999},"last":2.4470000000000001},{"interface":"front","average":{"1min":3.1230000000000002,"5min":3.9710000000000001,"15min":2.6030000000000002},"min":{"1min":0.96499999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":7.6139999999999999,"5min":45.618000000000002,"15min":45.618000000000002},"last":3.871}]},{"osd":2,"last update":"Tue Sep 8 15:41:44 2020","interfaces":[{"interface":"back","average":{"1min":2.5910000000000002,"5min":3.2280000000000002,"15min":2.5459999999999998},"min":{"1min":0.84099999999999997,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":7.9160000000000004,"5min":45.774000000000001,"15min":45.774000000000001},"last":1.508},{"interface":"front","average":{"1min":2.859,"5min":2.6819999999999999,"15min":2.4239999999999999},"min":{"1min":0.81000000000000005,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":7.7279999999999998,"5min":13.948,"15min":13.948},"last":1.0649999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:42:03 2020","interfaces":[{"interface":"back","average":{"1min":2.3540000000000001,"5min":2.4420000000000002,"15min":2.5590000000000002},"min":{"1min":1.1579999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":3.4399999999999999,"5min":10.523,"15min":10.523},"last":1.0309999999999999},{"interface":"front","average":{"1min":2.3999999999999999,"5min":2.1099999999999999,"15min":2.4260000000000002},"min":{"1min":1.0029999999999999,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":3.431,"5min":11.298,"15min":11.298},"last":8.8970000000000002}]},{"osd":5,"last update":"Tue Sep 8 15:42:33 2020","interfaces":[{"interface":"back","average":{"1min":3.2069999999999999,"5min":2.972,"15min":2.9380000000000002},"min":{"1min":1.6240000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":4.9039999999999999,"5min":45.442999999999998,"15min":45.442999999999998},"last":2.2080000000000002},{"interface":"front","average":{"1min":2.8660000000000001,"5min":3.1299999999999999,"15min":3.2320000000000002},"min":{"1min":1.331,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":4.2320000000000002,"5min":23.565000000000001,"15min":23.565000000000001},"last":1.7549999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:41:55 2020","interfaces":[{"interface":"back","average":{"1min":2.222,"5min":2.6480000000000001,"15min":3.1309999999999998},"min":{"1min":1.056,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":3.7080000000000002,"5min":7.7969999999999997,"15min":7.7969999999999997},"last":2.5459999999999998},{"interface":"front","average":{"1min":2.6629999999999998,"5min":3.0379999999999998,"15min":3.3969999999999998},"min":{"1min":1.3700000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":5.7690000000000001,"5min":47.457999999999998,"15min":47.457999999999998},"last":2.3900000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:42:33 2020","interfaces":[{"interface":"back","average":{"1min":3.363,"5min":3.7669999999999999,"15min":4.0739999999999998},"min":{"1min":1.51,"5min":1.1160000000000001,"15min":1.1160000000000001},"max":{"1min":5,"5min":26.035,"15min":26.035},"last":3.1469999999999998},{"interface":"front","average":{"1min":2.9220000000000002,"5min":3.0089999999999999,"15min":3.4039999999999999},"min":{"1min":0.86099999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":4.1360000000000001,"5min":25.521000000000001,"15min":25.521000000000001},"last":7.6820000000000004}]},{"osd":8,"last update":"Tue Sep 8 15:42:41 2020","interfaces":[{"interface":"back","average":{"1min":4.3310000000000004,"5min":3.7610000000000001,"15min":3.7629999999999999},"min":{"1min":1.992,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":16.812000000000001,"5min":46.917999999999999,"15min":46.917999999999999},"last":3.2290000000000001},{"interface":"front","average":{"1min":7.6550000000000002,"5min":5.3949999999999996,"15min":4.6529999999999996},"min":{"1min":2.032,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":78.808999999999997,"5min":78.808999999999997,"15min":78.808999999999997},"last":2.7029999999999998}]},{"osd":9,"last update":"Tue Sep 8 15:42:04 2020","interfaces":[{"interface":"back","average":{"1min":2.77,"5min":2.7909999999999999,"15min":3.5630000000000002},"min":{"1min":1.2529999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":4.2629999999999999,"5min":10.872999999999999,"15min":10.872999999999999},"last":7.4100000000000001},{"interface":"front","average":{"1min":2.8719999999999999,"5min":3.5470000000000002,"15min":4.048},"min":{"1min":1.23,"5min":1.23,"15min":1.23},"max":{"1min":6.1360000000000001,"5min":46.665999999999997,"15min":46.665999999999997},"last":2.8450000000000002}]},{"osd":10,"last update":"Tue Sep 8 15:42:33 2020","interfaces":[{"interface":"back","average":{"1min":3.4969999999999999,"5min":3.8149999999999999,"15min":4.2140000000000004},"min":{"1min":1.7190000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"max":{"1min":4.9829999999999997,"5min":39.283000000000001,"15min":39.283000000000001},"last":3.3279999999999998},{"interface":"front","average":{"1min":3.2450000000000001,"5min":2.8540000000000001,"15min":3.4889999999999999},"min":{"1min":1.0840000000000001,"5min":1.081,"15min":1.081},"max":{"1min":4.6479999999999997,"5min":11.005000000000001,"15min":11.005000000000001},"last":8.3200000000000003}]},{"osd":11,"last update":"Tue Sep 8 15:41:55 2020","interfaces":[{"interface":"back","average":{"1min":3.2000000000000002,"5min":3.5579999999999998,"15min":4.3869999999999996},"min":{"1min":1.609,"5min":1.196,"15min":1.196},"max":{"1min":5.0860000000000003,"5min":13.882999999999999,"15min":13.882999999999999},"last":9.0559999999999992},{"interface":"front","average":{"1min":2.952,"5min":4.6840000000000002,"15min":4.3300000000000001},"min":{"1min":1.6359999999999999,"5min":1.577,"15min":1.577},"max":{"1min":4.6349999999999998,"5min":47.234999999999999,"15min":47.234999999999999},"last":5.0650000000000004}]}]},{"osd":4,"up_from":0,"seq":136,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1439852,"kb_used_data":1377060,"kb_used_omap":0,"kb_used_meta":62784,"kb_avail":92305300,"statfs":{"total":95995035648,"available":94520627200,"internally_reserved":0,"allocated":1410109440,"data_stored":1409806729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64290816},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:42:05 2020","interfaces":[{"interface":"back","average":{"1min":2.056,"5min":1.7569999999999999,"15min":1.8839999999999999},"min":{"1min":0.98699999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":4.8010000000000002,"5min":5.6210000000000004,"15min":5.7130000000000001},"last":1.599},{"interface":"front","average":{"1min":2.3450000000000002,"5min":1.984,"15min":1.9610000000000001},"min":{"1min":0.873,"5min":0.71299999999999997,"15min":0.70799999999999996},"max":{"1min":4.1429999999999998,"5min":10.471,"15min":10.471},"last":1.5109999999999999}]},{"osd":1,"last update":"Tue Sep 8 15:42:05 2020","interfaces":[{"interface":"back","average":{"1min":2.3519999999999999,"5min":3.722,"15min":2.532},"min":{"1min":0.89100000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":5.3780000000000001,"5min":74.947999999999993,"15min":74.947999999999993},"last":2.0790000000000002},{"interface":"front","average":{"1min":2.8849999999999998,"5min":3.9260000000000002,"15min":2.8039999999999998},"min":{"1min":1,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":9.4090000000000007,"5min":71.108999999999995,"15min":71.108999999999995},"last":2.597}]},{"osd":2,"last update":"Tue Sep 8 15:42:05 2020","interfaces":[{"interface":"back","average":{"1min":2.7589999999999999,"5min":3.819,"15min":2.7909999999999999},"min":{"1min":1.2709999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":4.9450000000000003,"5min":74.504000000000005,"15min":74.504000000000005},"last":1.843},{"interface":"front","average":{"1min":2.6840000000000002,"5min":3.5640000000000001,"15min":2.8279999999999998},"min":{"1min":1.014,"5min":1.014,"15min":1.014},"max":{"1min":5.0039999999999996,"5min":73.885000000000005,"15min":73.885000000000005},"last":2.169}]},{"osd":3,"last update":"Tue Sep 8 15:42:05 2020","interfaces":[{"interface":"back","average":{"1min":2.5299999999999998,"5min":2.4100000000000001,"15min":2.5750000000000002},"min":{"1min":1.1859999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":5.1420000000000003,"5min":6.3280000000000003,"15min":6.3280000000000003},"last":63.783000000000001},{"interface":"front","average":{"1min":2.7690000000000001,"5min":2.8460000000000001,"15min":2.605},"min":{"1min":1.093,"5min":1.093,"15min":1.093},"max":{"1min":4.9409999999999998,"5min":9.5069999999999997,"15min":9.5069999999999997},"last":60.545999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:42:36 2020","interfaces":[{"interface":"back","average":{"1min":9.1929999999999996,"5min":3.8690000000000002,"15min":3.3079999999999998},"min":{"1min":1.0289999999999999,"5min":1.024,"15min":1.014},"max":{"1min":125.51600000000001,"5min":125.51600000000001,"15min":125.51600000000001},"last":2.548},{"interface":"front","average":{"1min":9.391,"5min":3.9740000000000002,"15min":3.3559999999999999},"min":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.129},"max":{"1min":125.816,"5min":125.816,"15min":125.816},"last":3.645}]},{"osd":6,"last update":"Tue Sep 8 15:41:46 2020","interfaces":[{"interface":"back","average":{"1min":2.6440000000000001,"5min":2.9790000000000001,"15min":3.2799999999999998},"min":{"1min":1.1399999999999999,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":4.6710000000000003,"5min":6.8650000000000002,"15min":6.8650000000000002},"last":3.0310000000000001},{"interface":"front","average":{"1min":2.6920000000000002,"5min":2.9319999999999999,"15min":3.2629999999999999},"min":{"1min":1.639,"5min":1.1659999999999999,"15min":1.1659999999999999},"max":{"1min":5.8179999999999996,"5min":6.8849999999999998,"15min":6.8849999999999998},"last":3.2360000000000002}]},{"osd":7,"last update":"Tue Sep 8 15:42:11 2020","interfaces":[{"interface":"back","average":{"1min":3.5219999999999998,"5min":3.1749999999999998,"15min":3.6840000000000002},"min":{"1min":1.8400000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"max":{"1min":6.6059999999999999,"5min":7.2969999999999997,"15min":7.4699999999999998},"last":2.8370000000000002},{"interface":"front","average":{"1min":3.9529999999999998,"5min":3.4140000000000001,"15min":3.827},"min":{"1min":1.6779999999999999,"5min":1.546,"15min":1.546},"max":{"1min":6.8440000000000003,"5min":7.1459999999999999,"15min":7.1459999999999999},"last":3.9830000000000001}]},{"osd":8,"last update":"Tue Sep 8 15:42:41 2020","interfaces":[{"interface":"back","average":{"1min":3.6720000000000002,"5min":5.056,"15min":4.3570000000000002},"min":{"1min":1.476,"5min":1.276,"15min":1.276},"max":{"1min":5.7510000000000003,"5min":109.608,"15min":109.608},"last":3.3250000000000002},{"interface":"front","average":{"1min":4.0679999999999996,"5min":4.9340000000000002,"15min":4.5949999999999998},"min":{"1min":1.6080000000000001,"5min":1.49,"15min":1.49},"max":{"1min":6.532,"5min":116.52500000000001,"15min":116.52500000000001},"last":3.5630000000000002}]},{"osd":9,"last update":"Tue Sep 8 15:42:11 2020","interfaces":[{"interface":"back","average":{"1min":3.8599999999999999,"5min":5.0540000000000003,"15min":4.6289999999999996},"min":{"1min":1.9059999999999999,"5min":1.546,"15min":1.546},"max":{"1min":6.609,"5min":118.312,"15min":118.312},"last":4.2919999999999998},{"interface":"front","average":{"1min":3.6819999999999999,"5min":4.9720000000000004,"15min":4.6029999999999998},"min":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"max":{"1min":6.1479999999999997,"5min":116.871,"15min":116.871},"last":3.7890000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:42:36 2020","interfaces":[{"interface":"back","average":{"1min":3.4769999999999999,"5min":3.004,"15min":4.2699999999999996},"min":{"1min":1.6439999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"max":{"1min":4.7009999999999996,"5min":6.6070000000000002,"15min":7.9279999999999999},"last":2.915},{"interface":"front","average":{"1min":4.2539999999999996,"5min":3.4380000000000002,"15min":4.415},"min":{"1min":1.9790000000000001,"5min":1.702,"15min":1.702},"max":{"1min":6.5060000000000002,"5min":7.9870000000000001,"15min":7.9870000000000001},"last":4.5960000000000001}]}]},{"osd":5,"up_from":0,"seq":130,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":895904,"kb_used_data":833752,"kb_used_omap":0,"kb_used_meta":62144,"kb_avail":92849248,"statfs":{"total":95995035648,"available":95077629952,"internally_reserved":0,"allocated":853762048,"data_stored":853463745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":63635456},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:42:35 2020","interfaces":[{"interface":"back","average":{"1min":2.0920000000000001,"5min":1.629,"15min":1.871},"min":{"1min":0.77500000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":5.4400000000000004,"5min":5.4400000000000004,"15min":5.4400000000000004},"last":77.744},{"interface":"front","average":{"1min":2.5139999999999998,"5min":2.246,"15min":2.1469999999999998},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":4.9320000000000004,"5min":7.1340000000000003,"15min":7.1340000000000003},"last":81.403000000000006}]},{"osd":1,"last update":"Tue Sep 8 15:42:35 2020","interfaces":[{"interface":"back","average":{"1min":3.0129999999999999,"5min":4.1070000000000002,"15min":2.968},"min":{"1min":1.2050000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":4.9119999999999999,"5min":128.488,"15min":128.488},"last":82.390000000000001},{"interface":"front","average":{"1min":1.8720000000000001,"5min":4.5380000000000003,"15min":2.9510000000000001},"min":{"1min":0.82599999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":3.4780000000000002,"5min":129.63800000000001,"15min":129.63800000000001},"last":74.873999999999995}]},{"osd":2,"last update":"Tue Sep 8 15:42:35 2020","interfaces":[{"interface":"back","average":{"1min":2.0619999999999998,"5min":3.903,"15min":2.7999999999999998},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":6.8789999999999996,"5min":127.498,"15min":127.498},"last":74.340999999999994},{"interface":"front","average":{"1min":3.056,"5min":4.3049999999999997,"15min":3.3410000000000002},"min":{"1min":1.661,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":4.5579999999999998,"5min":126.673,"15min":126.673},"last":82.730000000000004}]},{"osd":3,"last update":"Tue Sep 8 15:42:35 2020","interfaces":[{"interface":"back","average":{"1min":4.7649999999999997,"5min":3.29,"15min":3.149},"min":{"1min":1.3480000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":23.974,"5min":23.974,"15min":23.974},"last":82.040999999999997},{"interface":"front","average":{"1min":3.8100000000000001,"5min":2.6739999999999999,"15min":2.8999999999999999},"min":{"1min":1.097,"5min":0.875,"15min":0.875},"max":{"1min":24.187999999999999,"5min":24.187999999999999,"15min":24.187999999999999},"last":75.313999999999993}]},{"osd":4,"last update":"Tue Sep 8 15:42:35 2020","interfaces":[{"interface":"back","average":{"1min":2.9510000000000001,"5min":2.2629999999999999,"15min":2.3900000000000001},"min":{"1min":1.04,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":6.109,"5min":6.9180000000000001,"15min":6.9180000000000001},"last":80.195999999999998},{"interface":"front","average":{"1min":2.6779999999999999,"5min":2.3460000000000001,"15min":2.7109999999999999},"min":{"1min":1.127,"5min":1.127,"15min":1.127},"max":{"1min":4.6749999999999998,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":79.810000000000002}]},{"osd":6,"last update":"Tue Sep 8 15:42:03 2020","interfaces":[{"interface":"back","average":{"1min":3.3199999999999998,"5min":3.2970000000000002,"15min":3.54},"min":{"1min":1.863,"5min":1.0720000000000001,"15min":1.0720000000000001},"max":{"1min":5.1349999999999998,"5min":8.7710000000000008,"15min":8.7710000000000008},"last":220.636},{"interface":"front","average":{"1min":3.7749999999999999,"5min":3.8279999999999998,"15min":3.452},"min":{"1min":2.2629999999999999,"5min":1.3580000000000001,"15min":1.3580000000000001},"max":{"1min":5.8310000000000004,"5min":9.359,"15min":9.359},"last":221.00700000000001}]},{"osd":7,"last update":"Tue Sep 8 15:42:19 2020","interfaces":[{"interface":"back","average":{"1min":3.992,"5min":3.2650000000000001,"15min":3.5680000000000001},"min":{"1min":2.1429999999999998,"5min":1.26,"15min":1.26},"max":{"1min":6.7329999999999997,"5min":8.9610000000000003,"15min":8.9610000000000003},"last":83.299999999999997},{"interface":"front","average":{"1min":4.4480000000000004,"5min":4.016,"15min":4.0640000000000001},"min":{"1min":2.5350000000000001,"5min":1.532,"15min":1.532},"max":{"1min":6.9829999999999997,"5min":9.2129999999999992,"15min":9.2129999999999992},"last":220.34399999999999}]},{"osd":8,"last update":"Tue Sep 8 15:41:41 2020","interfaces":[{"interface":"back","average":{"1min":3.3439999999999999,"5min":3.2170000000000001,"15min":3.4849999999999999},"min":{"1min":1.6659999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":9.3190000000000008,"5min":9.3190000000000008,"15min":9.3190000000000008},"last":83.019000000000005},{"interface":"front","average":{"1min":2.4500000000000002,"5min":2.9510000000000001,"15min":3.6869999999999998},"min":{"1min":1.278,"5min":1.1479999999999999,"15min":1.1479999999999999},"max":{"1min":9.1099999999999994,"5min":10.138999999999999,"15min":10.138999999999999},"last":75.641999999999996}]},{"osd":9,"last update":"Tue Sep 8 15:42:13 2020","interfaces":[{"interface":"back","average":{"1min":2.8530000000000002,"5min":3.1269999999999998,"15min":4.1210000000000004},"min":{"1min":1.504,"5min":1.2350000000000001,"15min":1.2350000000000001},"max":{"1min":4.3540000000000001,"5min":8.1980000000000004,"15min":8.1980000000000004},"last":76.382999999999996},{"interface":"front","average":{"1min":4.1719999999999997,"5min":3.9380000000000002,"15min":4.1600000000000001},"min":{"1min":2.3540000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"max":{"1min":7.3920000000000003,"5min":8.9149999999999991,"15min":8.9149999999999991},"last":83.159999999999997}]},{"osd":10,"last update":"Tue Sep 8 15:42:35 2020","interfaces":[{"interface":"back","average":{"1min":2.7330000000000001,"5min":2.6659999999999999,"15min":3.7090000000000001},"min":{"1min":1.5820000000000001,"5min":1.431,"15min":1.431},"max":{"1min":3.8700000000000001,"5min":5.8959999999999999,"15min":5.8959999999999999},"last":77.087999999999994},{"interface":"front","average":{"1min":4.6500000000000004,"5min":3.8969999999999998,"15min":4.5659999999999998},"min":{"1min":2.2869999999999999,"5min":1.8660000000000001,"15min":1.8660000000000001},"max":{"1min":7.766,"5min":9.0670000000000002,"15min":9.0670000000000002},"last":219.92599999999999}]}]}]}} 2020-09-08T15:42:50.445 INFO:tasks.thrashosds.thrasher:Setting 3.c to [11, 8] 2020-09-08T15:42:50.446 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.c', '11', '8'] 2020-09-08T15:42:50.446 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pg-upmap 3.c 11 8 2020-09-08T15:42:50.448 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:50.550 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:50.607 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:42:50.615 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:42:50 smithi166 bash[16593]: debug 2020-09-08T15:42:50.422+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:50.631 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:50 smithi104 bash[9203]: cluster 2020-09-08T15:42:48.935998+0000 osd.7 (osd.7) 45 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:42:50.631 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:50 smithi104 bash[9203]: cluster 2020-09-08T15:42:48.963433+0000 osd.7 (osd.7) 46 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:42:50.632 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:50 smithi104 bash[9203]: cluster 2020-09-08T15:42:49.633923+0000 mon.a (mon.0) 1152 : cluster [DBG] osdmap e230: 12 total, 12 up, 11 in 2020-09-08T15:42:50.663 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:50.766 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:50.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:50 smithi055 bash[9296]: cluster 2020-09-08T15:42:48.935998+0000 osd.7 (osd.7) 45 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:42:50.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:50 smithi055 bash[9296]: cluster 2020-09-08T15:42:48.963433+0000 osd.7 (osd.7) 46 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:42:50.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:50 smithi055 bash[9296]: cluster 2020-09-08T15:42:49.633923+0000 mon.a (mon.0) 1152 : cluster [DBG] osdmap e230: 12 total, 12 up, 11 in 2020-09-08T15:42:50.915 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:51.019 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:51.027 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:50 smithi166 bash[9446]: cluster 2020-09-08T15:42:48.935998+0000 osd.7 (osd.7) 45 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:42:51.027 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:50 smithi166 bash[9446]: cluster 2020-09-08T15:42:48.963433+0000 osd.7 (osd.7) 46 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:42:51.027 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:50 smithi166 bash[9446]: cluster 2020-09-08T15:42:49.633923+0000 mon.a (mon.0) 1152 : cluster [DBG] osdmap e230: 12 total, 12 up, 11 in 2020-09-08T15:42:51.028 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:42:50 smithi166 bash[16593]: debug 2020-09-08T15:42:50.662+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:51.135 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:51.216 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 31 16 70421 70405 141.93 111.625 0.00724352 0.00704345 2020-09-08T15:42:51.236 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:51.365 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:42:51 smithi166 bash[11928]: debug 2020-09-08T15:42:51.130+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:51.370 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:42:51.408 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:42:50 smithi055 bash[19003]: debug 2020-09-08T15:42:50.912+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:51.471 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:51.574 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:51.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:42:51 smithi166 bash[11928]: debug 2020-09-08T15:42:51.366+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:51.679 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:51.795 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:51.796 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:51 smithi055 bash[9296]: cluster 2020-09-08T15:42:49.674636+0000 mgr.x (mgr.24118) 464 : cluster [DBG] pgmap v611: 17 pgs: 17 active+clean; 3.8 GiB data, 8.4 GiB used, 975 GiB / 983 GiB avail; 115 MiB/s wr, 1.84k op/s 2020-09-08T15:42:51.797 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:42:51 smithi055 bash[11956]: debug 2020-09-08T15:42:51.576+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:51.899 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:51.909 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:51 smithi166 bash[9446]: cluster 2020-09-08T15:42:49.674636+0000 mgr.x (mgr.24118) 464 : cluster [DBG] pgmap v611: 17 pgs: 17 active+clean; 3.8 GiB data, 8.4 GiB used, 975 GiB / 983 GiB avail; 115 MiB/s wr, 1.84k op/s 2020-09-08T15:42:51.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:51 smithi104 bash[9203]: cluster 2020-09-08T15:42:49.674636+0000 mgr.x (mgr.24118) 464 : cluster [DBG] pgmap v611: 17 pgs: 17 active+clean; 3.8 GiB data, 8.4 GiB used, 975 GiB / 983 GiB avail; 115 MiB/s wr, 1.84k op/s 2020-09-08T15:42:52.008 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:52.113 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:52.122 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:42:51 smithi055 bash[14301]: debug 2020-09-08T15:42:51.796+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:52.218 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 32 16 71795 71779 140.178 85.875 0.00884186 0.00713169 2020-09-08T15:42:52.234 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:52.335 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:52.343 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:42:52 smithi166 bash[16593]: debug 2020-09-08T15:42:52.010+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:52.411 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:42:52 smithi055 bash[11956]: debug 2020-09-08T15:42:52.232+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:52.446 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:52.549 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:52.615 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:42:52 smithi166 bash[18960]: debug 2020-09-08T15:42:52.446+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:52.655 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:52.760 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:52.814 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:42:52.842 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:42:52.871 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:42:52.879 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:42:52 smithi055 bash[16646]: debug 2020-09-08T15:42:52.656+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:52.880 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:52.945 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:42:52.980 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:42:52.995 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:53.014 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:42:53.047 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:42:53.081 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:42:53.114 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:42:53.145 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:42:53.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:42:52 smithi055 bash[11956]: debug 2020-09-08T15:42:52.880+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:53.174 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:42:53.201 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:42:53.203 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:42:53.203 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:42:53.203 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:42:53.204 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:42:53.204 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:42:53.204 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:42:53.204 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:42:53.204 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:42:53.204 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:42:53.205 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:42:53.205 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:42:53.205 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:42:53.216 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 33 16 73472 73456 139.106 104.812 0.00555652 0.00718688 2020-09-08T15:42:53.747 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:53.851 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:53.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:53 smithi055 bash[9296]: cluster 2020-09-08T15:42:51.675254+0000 mgr.x (mgr.24118) 465 : cluster [DBG] pgmap v612: 17 pgs: 17 active+clean; 4.0 GiB data, 8.6 GiB used, 975 GiB / 983 GiB avail; 114 MiB/s wr, 1.83k op/s 2020-09-08T15:42:53.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:53 smithi055 bash[9296]: audit 2020-09-08T15:42:52.786671+0000 mon.c (mon.2) 44 : audit [DBG] from='client.? v1:172.21.15.104:0/3247400356' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:42:53.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:42:53 smithi104 bash[17588]: debug 2020-09-08T15:42:53.748+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:53.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:53 smithi104 bash[9203]: cluster 2020-09-08T15:42:51.675254+0000 mgr.x (mgr.24118) 465 : cluster [DBG] pgmap v612: 17 pgs: 17 active+clean; 4.0 GiB data, 8.6 GiB used, 975 GiB / 983 GiB avail; 114 MiB/s wr, 1.83k op/s 2020-09-08T15:42:53.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:53 smithi104 bash[9203]: audit 2020-09-08T15:42:52.786671+0000 mon.c (mon.2) 44 : audit [DBG] from='client.? v1:172.21.15.104:0/3247400356' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:42:54.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:53 smithi166 bash[9446]: cluster 2020-09-08T15:42:51.675254+0000 mgr.x (mgr.24118) 465 : cluster [DBG] pgmap v612: 17 pgs: 17 active+clean; 4.0 GiB data, 8.6 GiB used, 975 GiB / 983 GiB avail; 114 MiB/s wr, 1.83k op/s 2020-09-08T15:42:54.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:53 smithi166 bash[9446]: audit 2020-09-08T15:42:52.786671+0000 mon.c (mon.2) 44 : audit [DBG] from='client.? v1:172.21.15.104:0/3247400356' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:42:54.216 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 34 16 75071 75055 137.954 99.9375 0.0172395 0.00724658 2020-09-08T15:42:54.588 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:54.679 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:42:54 smithi104 bash[22305]: debug 2020-09-08T15:42:54.588+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:54.690 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:54.815 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:42:54.918 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:55.158 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:42:54 smithi055 bash[19003]: debug 2020-09-08T15:42:54.816+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:55.217 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 35 16 76683 76667 136.891 100.75 0.00700104 0.00730327 2020-09-08T15:42:55.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:55 smithi055 bash[9296]: cluster 2020-09-08T15:42:53.676111+0000 mgr.x (mgr.24118) 466 : cluster [DBG] pgmap v613: 17 pgs: 17 active+clean; 4.3 GiB data, 9.3 GiB used, 974 GiB / 983 GiB avail; 133 MiB/s wr, 2.12k op/s 2020-09-08T15:42:55.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:55 smithi055 bash[9296]: cluster 2020-09-08T15:42:54.636205+0000 mon.a (mon.0) 1153 : cluster [DBG] osdmap e231: 12 total, 12 up, 11 in 2020-09-08T15:42:55.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:55 smithi104 bash[9203]: cluster 2020-09-08T15:42:53.676111+0000 mgr.x (mgr.24118) 466 : cluster [DBG] pgmap v613: 17 pgs: 17 active+clean; 4.3 GiB data, 9.3 GiB used, 974 GiB / 983 GiB avail; 133 MiB/s wr, 2.12k op/s 2020-09-08T15:42:55.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:55 smithi104 bash[9203]: cluster 2020-09-08T15:42:54.636205+0000 mon.a (mon.0) 1153 : cluster [DBG] osdmap e231: 12 total, 12 up, 11 in 2020-09-08T15:42:56.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:55 smithi166 bash[9446]: cluster 2020-09-08T15:42:53.676111+0000 mgr.x (mgr.24118) 466 : cluster [DBG] pgmap v613: 17 pgs: 17 active+clean; 4.3 GiB data, 9.3 GiB used, 974 GiB / 983 GiB avail; 133 MiB/s wr, 2.12k op/s 2020-09-08T15:42:56.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:55 smithi166 bash[9446]: cluster 2020-09-08T15:42:54.636205+0000 mon.a (mon.0) 1153 : cluster [DBG] osdmap e231: 12 total, 12 up, 11 in 2020-09-08T15:42:56.155 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:42:56.217 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 36 16 78398 78382 136.066 107.188 0.00774528 0.00734703 2020-09-08T15:42:56.259 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:56.615 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:42:56 smithi166 bash[18960]: debug 2020-09-08T15:42:56.154+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:56.895 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:56.998 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:57.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:42:56 smithi104 bash[22305]: debug 2020-09-08T15:42:56.900+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:57.217 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 37 16 80072 80056 135.216 104.625 0.00775387 0.00739345 2020-09-08T15:42:57.220 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:42:57.342 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:57.357 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:42:57.550 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:42:57.638 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:42:57 smithi104 bash[22305]: debug 2020-09-08T15:42:57.224+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:57.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:57 smithi055 bash[9296]: cluster 2020-09-08T15:42:55.676626+0000 mgr.x (mgr.24118) 467 : cluster [DBG] pgmap v615: 17 pgs: 17 active+clean; 4.4 GiB data, 9.5 GiB used, 974 GiB / 983 GiB avail; 95 MiB/s wr, 1.51k op/s 2020-09-08T15:42:57.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:57 smithi055 bash[9296]: cluster 2020-09-08T15:42:55.971998+0000 osd.9 (osd.9) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:57.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:57 smithi055 bash[9296]: cluster 2020-09-08T15:42:55.972070+0000 osd.9 (osd.9) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:57.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:57 smithi055 bash[9296]: cluster 2020-09-08T15:42:56.642516+0000 mon.a (mon.0) 1154 : cluster [DBG] osdmap e232: 12 total, 12 up, 11 in 2020-09-08T15:42:57.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:57 smithi104 bash[9203]: cluster 2020-09-08T15:42:55.676626+0000 mgr.x (mgr.24118) 467 : cluster [DBG] pgmap v615: 17 pgs: 17 active+clean; 4.4 GiB data, 9.5 GiB used, 974 GiB / 983 GiB avail; 95 MiB/s wr, 1.51k op/s 2020-09-08T15:42:57.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:57 smithi104 bash[9203]: cluster 2020-09-08T15:42:55.971998+0000 osd.9 (osd.9) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:57.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:57 smithi104 bash[9203]: cluster 2020-09-08T15:42:55.972070+0000 osd.9 (osd.9) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:57.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:57 smithi104 bash[9203]: cluster 2020-09-08T15:42:56.642516+0000 mon.a (mon.0) 1154 : cluster [DBG] osdmap e232: 12 total, 12 up, 11 in 2020-09-08T15:42:58.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:57 smithi166 bash[9446]: cluster 2020-09-08T15:42:55.676626+0000 mgr.x (mgr.24118) 467 : cluster [DBG] pgmap v615: 17 pgs: 17 active+clean; 4.4 GiB data, 9.5 GiB used, 974 GiB / 983 GiB avail; 95 MiB/s wr, 1.51k op/s 2020-09-08T15:42:58.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:57 smithi166 bash[9446]: cluster 2020-09-08T15:42:55.971998+0000 osd.9 (osd.9) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:58.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:57 smithi166 bash[9446]: cluster 2020-09-08T15:42:55.972070+0000 osd.9 (osd.9) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:58.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:57 smithi166 bash[9446]: cluster 2020-09-08T15:42:56.642516+0000 mon.a (mon.0) 1154 : cluster [DBG] osdmap e232: 12 total, 12 up, 11 in 2020-09-08T15:42:58.217 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 38 16 81765 81749 134.442 105.812 0.00897397 0.00743627 2020-09-08T15:42:58.295 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:42:58.399 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:58.526 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:42:58.565 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:42:58.565 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T15:42:58.566 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2020-09-08T15:42:58.566 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2020-09-08T15:42:58.566 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:42:58.630 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:58.734 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:42:58 smithi104 bash[17588]: debug 2020-09-08T15:42:58.300+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:58.739 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:58 smithi055 bash[9296]: cluster 2020-09-08T15:42:58.000757+0000 osd.2 (osd.2) 34 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:58.740 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:58 smithi055 bash[9296]: cluster 2020-09-08T15:42:58.000838+0000 osd.2 (osd.2) 35 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:58.740 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:42:58 smithi055 bash[14301]: debug 2020-09-08T15:42:58.524+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:58.740 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:42:58.846 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:58.968 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:42:59.072 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:42:59.080 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:58 smithi104 bash[9203]: cluster 2020-09-08T15:42:58.000757+0000 osd.2 (osd.2) 34 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:59.080 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:58 smithi104 bash[9203]: cluster 2020-09-08T15:42:58.000838+0000 osd.2 (osd.2) 35 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:59.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:58 smithi166 bash[9446]: cluster 2020-09-08T15:42:58.000757+0000 osd.2 (osd.2) 34 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:42:59.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:58 smithi166 bash[9446]: cluster 2020-09-08T15:42:58.000838+0000 osd.2 (osd.2) 35 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:42:59.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:42:58 smithi055 bash[11956]: debug 2020-09-08T15:42:58.740+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:59.158 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:42:58 smithi055 bash[16646]: debug 2020-09-08T15:42:58.968+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:59.218 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 39 16 83361 83345 133.552 99.75 0.00685907 0.00748593 2020-09-08T15:42:59.647 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:42:59.751 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:59.886 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:42:59.917 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:59 smithi055 bash[9296]: cluster 2020-09-08T15:42:57.677144+0000 mgr.x (mgr.24118) 468 : cluster [DBG] pgmap v617: 17 pgs: 17 active+clean; 4.7 GiB data, 10 GiB used, 973 GiB / 983 GiB avail; 107 MiB/s wr, 1.72k op/s 2020-09-08T15:42:59.917 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:59 smithi055 bash[9296]: cluster 2020-09-08T15:42:58.654610+0000 mon.a (mon.0) 1155 : cluster [DBG] osdmap e233: 12 total, 12 up, 11 in 2020-09-08T15:42:59.918 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:42:59 smithi055 bash[9296]: cluster 2020-09-08T15:42:59.637463+0000 mon.a (mon.0) 1156 : cluster [DBG] osdmap e234: 12 total, 12 up, 11 in 2020-09-08T15:42:59.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:59 smithi104 bash[9203]: cluster 2020-09-08T15:42:57.677144+0000 mgr.x (mgr.24118) 468 : cluster [DBG] pgmap v617: 17 pgs: 17 active+clean; 4.7 GiB data, 10 GiB used, 973 GiB / 983 GiB avail; 107 MiB/s wr, 1.72k op/s 2020-09-08T15:42:59.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:59 smithi104 bash[9203]: cluster 2020-09-08T15:42:58.654610+0000 mon.a (mon.0) 1155 : cluster [DBG] osdmap e233: 12 total, 12 up, 11 in 2020-09-08T15:42:59.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:42:59 smithi104 bash[9203]: cluster 2020-09-08T15:42:59.637463+0000 mon.a (mon.0) 1156 : cluster [DBG] osdmap e234: 12 total, 12 up, 11 in 2020-09-08T15:42:59.927 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:42:59 smithi104 bash[15273]: debug 2020-09-08T15:42:59.648+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:42:59.990 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:43:00.113 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:43:00.115 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:42:59 smithi166 bash[16593]: debug 2020-09-08T15:42:59.886+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:00.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:59 smithi166 bash[9446]: cluster 2020-09-08T15:42:57.677144+0000 mgr.x (mgr.24118) 468 : cluster [DBG] pgmap v617: 17 pgs: 17 active+clean; 4.7 GiB data, 10 GiB used, 973 GiB / 983 GiB avail; 107 MiB/s wr, 1.72k op/s 2020-09-08T15:43:00.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:59 smithi166 bash[9446]: cluster 2020-09-08T15:42:58.654610+0000 mon.a (mon.0) 1155 : cluster [DBG] osdmap e233: 12 total, 12 up, 11 in 2020-09-08T15:43:00.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:42:59 smithi166 bash[9446]: cluster 2020-09-08T15:42:59.637463+0000 mon.a (mon.0) 1156 : cluster [DBG] osdmap e234: 12 total, 12 up, 11 in 2020-09-08T15:43:00.215 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:43:00.217 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 15:43:00.222910 min lat: 0.00154046 max lat: 0.419999 avg lat: 0.00756018 2020-09-08T15:43:00.217 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:43:00.218 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 40 16 84656 84640 132.236 80.9375 0.00850483 0.00756018 2020-09-08T15:43:00.329 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:43:00 smithi055 bash[14301]: debug 2020-09-08T15:43:00.108+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:00.330 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:43:00.420 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2020-09-08T15:43:00.435 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:43:00.545 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:43:00.585 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:43:00.652 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:43:00.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:43:00 smithi055 bash[16646]: debug 2020-09-08T15:43:00.328+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:00.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:43:00 smithi055 bash[16646]: debug 2020-09-08T15:43:00.544+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:01.218 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 41 16 86224 86208 131.401 98 0.0077719 0.0076082 2020-09-08T15:43:01.450 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:43:01.554 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:43:01.671 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:43:01.774 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:43:01.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:01 smithi055 bash[9296]: cluster 2020-09-08T15:42:59.677563+0000 mgr.x (mgr.24118) 469 : cluster [DBG] pgmap v620: 17 pgs: 17 active+clean; 4.9 GiB data, 11 GiB used, 973 GiB / 983 GiB avail; 102 MiB/s wr, 1.63k op/s 2020-09-08T15:43:01.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:43:01 smithi055 bash[14301]: debug 2020-09-08T15:43:01.672+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:01.924 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:01 smithi104 bash[9203]: cluster 2020-09-08T15:42:59.677563+0000 mgr.x (mgr.24118) 469 : cluster [DBG] pgmap v620: 17 pgs: 17 active+clean; 4.9 GiB data, 11 GiB used, 973 GiB / 983 GiB avail; 102 MiB/s wr, 1.63k op/s 2020-09-08T15:43:01.924 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:43:01 smithi104 bash[19922]: debug 2020-09-08T15:43:01.451+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:02.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:01 smithi166 bash[9446]: cluster 2020-09-08T15:42:59.677563+0000 mgr.x (mgr.24118) 469 : cluster [DBG] pgmap v620: 17 pgs: 17 active+clean; 4.9 GiB data, 11 GiB used, 973 GiB / 983 GiB avail; 102 MiB/s wr, 1.63k op/s 2020-09-08T15:43:02.217 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 42 16 87452 87436 130.1 76.75 0.00946931 0.00768442 2020-09-08T15:43:02.665 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:43:02.671 INFO:teuthology.orchestra.run.smithi104.stderr:set 3.c pg_upmap mapping to [11,8] 2020-09-08T15:43:02.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:02 smithi055 bash[9296]: audit 2020-09-08T15:43:02.619582+0000 mon.a (mon.0) 1157 : audit [INF] from='client.? v1:172.21.15.104:0/1593787504' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:43:02.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:02 smithi055 bash[9296]: audit 2020-09-08T15:43:02.622661+0000 mon.c (mon.2) 45 : audit [INF] from='client.? v1:172.21.15.104:0/3564061198' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [11, 8]}]: dispatch 2020-09-08T15:43:02.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:02 smithi055 bash[9296]: audit 2020-09-08T15:43:02.623873+0000 mon.a (mon.0) 1158 : audit [INF] from='client.35129 ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [11, 8]}]: dispatch 2020-09-08T15:43:02.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:02 smithi104 bash[9203]: audit 2020-09-08T15:43:02.619582+0000 mon.a (mon.0) 1157 : audit [INF] from='client.? v1:172.21.15.104:0/1593787504' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:43:02.936 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:02 smithi104 bash[9203]: audit 2020-09-08T15:43:02.622661+0000 mon.c (mon.2) 45 : audit [INF] from='client.? v1:172.21.15.104:0/3564061198' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [11, 8]}]: dispatch 2020-09-08T15:43:02.936 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:02 smithi104 bash[9203]: audit 2020-09-08T15:43:02.623873+0000 mon.a (mon.0) 1158 : audit [INF] from='client.35129 ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [11, 8]}]: dispatch 2020-09-08T15:43:03.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:02 smithi166 bash[9446]: audit 2020-09-08T15:43:02.619582+0000 mon.a (mon.0) 1157 : audit [INF] from='client.? v1:172.21.15.104:0/1593787504' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:43:03.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:02 smithi166 bash[9446]: audit 2020-09-08T15:43:02.622661+0000 mon.c (mon.2) 45 : audit [INF] from='client.? v1:172.21.15.104:0/3564061198' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [11, 8]}]: dispatch 2020-09-08T15:43:03.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:02 smithi166 bash[9446]: audit 2020-09-08T15:43:02.623873+0000 mon.a (mon.0) 1158 : audit [INF] from='client.35129 ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [11, 8]}]: dispatch 2020-09-08T15:43:03.217 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 43 16 88298 88282 128.304 52.875 0.00854968 0.00771386 2020-09-08T15:43:04.028 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:03 smithi104 bash[9203]: cluster 2020-09-08T15:43:01.678038+0000 mgr.x (mgr.24118) 470 : cluster [DBG] pgmap v621: 17 pgs: 17 active+clean; 5.0 GiB data, 11 GiB used, 973 GiB / 983 GiB avail; 101 MiB/s wr, 1.61k op/s 2020-09-08T15:43:04.029 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:03 smithi104 bash[9203]: cluster 2020-09-08T15:43:02.658001+0000 mon.a (mon.0) 1159 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:43:04.029 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:03 smithi104 bash[9203]: cluster 2020-09-08T15:43:02.658040+0000 mon.a (mon.0) 1160 : cluster [INF] Cluster is now healthy 2020-09-08T15:43:04.029 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:03 smithi104 bash[9203]: audit 2020-09-08T15:43:02.667545+0000 mon.a (mon.0) 1161 : audit [INF] from='client.? v1:172.21.15.104:0/1593787504' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:43:04.030 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:03 smithi104 bash[9203]: audit 2020-09-08T15:43:02.667594+0000 mon.a (mon.0) 1162 : audit [INF] from='client.35129 ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [11, 8]}]': finished 2020-09-08T15:43:04.030 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:03 smithi104 bash[9203]: cluster 2020-09-08T15:43:02.667636+0000 mon.a (mon.0) 1163 : cluster [DBG] osdmap e235: 12 total, 12 up, 11 in 2020-09-08T15:43:04.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:03 smithi166 bash[9446]: cluster 2020-09-08T15:43:01.678038+0000 mgr.x (mgr.24118) 470 : cluster [DBG] pgmap v621: 17 pgs: 17 active+clean; 5.0 GiB data, 11 GiB used, 973 GiB / 983 GiB avail; 101 MiB/s wr, 1.61k op/s 2020-09-08T15:43:04.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:03 smithi166 bash[9446]: cluster 2020-09-08T15:43:02.658001+0000 mon.a (mon.0) 1159 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:43:04.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:03 smithi166 bash[9446]: cluster 2020-09-08T15:43:02.658040+0000 mon.a (mon.0) 1160 : cluster [INF] Cluster is now healthy 2020-09-08T15:43:04.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:03 smithi166 bash[9446]: audit 2020-09-08T15:43:02.667545+0000 mon.a (mon.0) 1161 : audit [INF] from='client.? v1:172.21.15.104:0/1593787504' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:43:04.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:03 smithi166 bash[9446]: audit 2020-09-08T15:43:02.667594+0000 mon.a (mon.0) 1162 : audit [INF] from='client.35129 ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [11, 8]}]': finished 2020-09-08T15:43:04.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:03 smithi166 bash[9446]: cluster 2020-09-08T15:43:02.667636+0000 mon.a (mon.0) 1163 : cluster [DBG] osdmap e235: 12 total, 12 up, 11 in 2020-09-08T15:43:04.167 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:03 smithi055 bash[9296]: cluster 2020-09-08T15:43:01.678038+0000 mgr.x (mgr.24118) 470 : cluster [DBG] pgmap v621: 17 pgs: 17 active+clean; 5.0 GiB data, 11 GiB used, 973 GiB / 983 GiB avail; 101 MiB/s wr, 1.61k op/s 2020-09-08T15:43:04.167 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:03 smithi055 bash[9296]: cluster 2020-09-08T15:43:02.658001+0000 mon.a (mon.0) 1159 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:43:04.167 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:03 smithi055 bash[9296]: cluster 2020-09-08T15:43:02.658040+0000 mon.a (mon.0) 1160 : cluster [INF] Cluster is now healthy 2020-09-08T15:43:04.167 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:03 smithi055 bash[9296]: audit 2020-09-08T15:43:02.667545+0000 mon.a (mon.0) 1161 : audit [INF] from='client.? v1:172.21.15.104:0/1593787504' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:43:04.168 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:03 smithi055 bash[9296]: audit 2020-09-08T15:43:02.667594+0000 mon.a (mon.0) 1162 : audit [INF] from='client.35129 ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [11, 8]}]': finished 2020-09-08T15:43:04.168 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:03 smithi055 bash[9296]: cluster 2020-09-08T15:43:02.667636+0000 mon.a (mon.0) 1163 : cluster [DBG] osdmap e235: 12 total, 12 up, 11 in 2020-09-08T15:43:04.218 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 44 16 88298 88282 125.387 0 - 0.00771386 2020-09-08T15:43:04.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:04 smithi104 bash[9203]: cluster 2020-09-08T15:43:03.674431+0000 mon.a (mon.0) 1164 : cluster [DBG] osdmap e236: 12 total, 12 up, 11 in 2020-09-08T15:43:05.020 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:04 smithi166 bash[9446]: cluster 2020-09-08T15:43:03.674431+0000 mon.a (mon.0) 1164 : cluster [DBG] osdmap e236: 12 total, 12 up, 11 in 2020-09-08T15:43:05.021 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:43:05.126 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:43:05.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:04 smithi055 bash[9296]: cluster 2020-09-08T15:43:03.674431+0000 mon.a (mon.0) 1164 : cluster [DBG] osdmap e236: 12 total, 12 up, 11 in 2020-09-08T15:43:05.225 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 45 16 88298 88282 122.601 0 - 0.00771386 2020-09-08T15:43:05.365 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:43:05 smithi166 bash[11928]: debug 2020-09-08T15:43:05.022+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:05.793 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:43:05.824 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:43:05.850 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:43:05.882 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:43:05.919 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:43:05.948 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:43:05.982 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:43:06.007 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:43:06.042 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:43:06.069 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:43:06.099 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:43:06.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:05 smithi166 bash[9446]: cluster 2020-09-08T15:43:03.678647+0000 mgr.x (mgr.24118) 471 : cluster [DBG] pgmap v624: 17 pgs: 17 active+clean; 5.3 GiB data, 11 GiB used, 972 GiB / 983 GiB avail; 95 MiB/s wr, 1.51k op/s 2020-09-08T15:43:06.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:05 smithi166 bash[9446]: cluster 2020-09-08T15:43:04.682996+0000 mon.a (mon.0) 1165 : cluster [DBG] osdmap e237: 12 total, 12 up, 11 in 2020-09-08T15:43:06.132 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:43:06.134 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:43:06.134 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:43:06.135 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:43:06.135 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:43:06.135 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:43:06.135 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:43:06.135 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:43:06.136 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:43:06.136 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:43:06.136 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:43:06.136 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:43:06.136 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:43:06.145 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:43:06.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:05 smithi055 bash[9296]: cluster 2020-09-08T15:43:03.678647+0000 mgr.x (mgr.24118) 471 : cluster [DBG] pgmap v624: 17 pgs: 17 active+clean; 5.3 GiB data, 11 GiB used, 972 GiB / 983 GiB avail; 95 MiB/s wr, 1.51k op/s 2020-09-08T15:43:06.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:05 smithi055 bash[9296]: cluster 2020-09-08T15:43:04.682996+0000 mon.a (mon.0) 1165 : cluster [DBG] osdmap e237: 12 total, 12 up, 11 in 2020-09-08T15:43:06.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:05 smithi104 bash[9203]: cluster 2020-09-08T15:43:03.678647+0000 mgr.x (mgr.24118) 471 : cluster [DBG] pgmap v624: 17 pgs: 17 active+clean; 5.3 GiB data, 11 GiB used, 972 GiB / 983 GiB avail; 95 MiB/s wr, 1.51k op/s 2020-09-08T15:43:06.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:05 smithi104 bash[9203]: cluster 2020-09-08T15:43:04.682996+0000 mon.a (mon.0) 1165 : cluster [DBG] osdmap e237: 12 total, 12 up, 11 in 2020-09-08T15:43:06.218 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 46 16 88801 88785 120.619 10.4792 0.00587948 0.00828848 2020-09-08T15:43:06.249 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:43:06.369 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:43:06.474 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:43:06.615 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:43:06 smithi166 bash[14254]: debug 2020-09-08T15:43:06.146+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:06.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:43:06 smithi055 bash[14301]: debug 2020-09-08T15:43:06.372+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:06.775 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:43:06.783 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:06 smithi104 bash[9203]: cluster 2020-09-08T15:43:05.679221+0000 mgr.x (mgr.24118) 472 : cluster [DBG] pgmap v626: 17 pgs: 17 active+clean; 5.3 GiB data, 11 GiB used, 972 GiB / 983 GiB avail; 76 MiB/s wr, 1.22k op/s 2020-09-08T15:43:06.783 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:06 smithi104 bash[9203]: audit 2020-09-08T15:43:05.767559+0000 mon.a (mon.0) 1166 : audit [DBG] from='client.? v1:172.21.15.104:0/4252302166' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:43:06.929 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:43:07.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:06 smithi166 bash[9446]: cluster 2020-09-08T15:43:05.679221+0000 mgr.x (mgr.24118) 472 : cluster [DBG] pgmap v626: 17 pgs: 17 active+clean; 5.3 GiB data, 11 GiB used, 972 GiB / 983 GiB avail; 76 MiB/s wr, 1.22k op/s 2020-09-08T15:43:07.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:06 smithi166 bash[9446]: audit 2020-09-08T15:43:05.767559+0000 mon.a (mon.0) 1166 : audit [DBG] from='client.? v1:172.21.15.104:0/4252302166' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:43:07.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:06 smithi055 bash[9296]: cluster 2020-09-08T15:43:05.679221+0000 mgr.x (mgr.24118) 472 : cluster [DBG] pgmap v626: 17 pgs: 17 active+clean; 5.3 GiB data, 11 GiB used, 972 GiB / 983 GiB avail; 76 MiB/s wr, 1.22k op/s 2020-09-08T15:43:07.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:06 smithi055 bash[9296]: audit 2020-09-08T15:43:05.767559+0000 mon.a (mon.0) 1166 : audit [DBG] from='client.? v1:172.21.15.104:0/4252302166' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:43:07.218 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 47 16 90077 90061 119.749 79.75 0.00626938 0.00834422 2020-09-08T15:43:07.481 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:43:07.583 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:43:07.723 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:43:07.828 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:43:07.865 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:43:07 smithi166 bash[11928]: debug 2020-09-08T15:43:07.722+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:07.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:43:07 smithi104 bash[19922]: debug 2020-09-08T15:43:07.479+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:07.947 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:43:08.051 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:43:08.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:43:07 smithi055 bash[11956]: debug 2020-09-08T15:43:07.948+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:08.171 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:43:08.218 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 48 16 91022 91006 118.485 59.0625 0.0105396 0.00843811 2020-09-08T15:43:08.276 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:08.365 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:43:08 smithi166 bash[11928]: debug 2020-09-08T15:43:08.170+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:08.390 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:08.497 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:08.732 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:08 smithi166 bash[9446]: cluster 2020-09-08T15:43:06.805024+0000 osd.7 (osd.7) 47 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:08.733 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:08 smithi166 bash[9446]: cluster 2020-09-08T15:43:06.805126+0000 osd.7 (osd.7) 48 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:08.733 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:08 smithi166 bash[9446]: cluster 2020-09-08T15:43:07.679714+0000 mgr.x (mgr.24118) 473 : cluster [DBG] pgmap v627: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 5.4 GiB data, 12 GiB used, 972 GiB / 983 GiB avail; 79 MiB/s wr, 1.26k op/s; 11092/177892 objects misplaced (6.235%); 352 KiB/s, 5 objects/s recovering 2020-09-08T15:43:08.733 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:08 smithi166 bash[9446]: cluster 2020-09-08T15:43:07.739063+0000 mon.a (mon.0) 1167 : cluster [DBG] osdmap e238: 12 total, 12 up, 11 in 2020-09-08T15:43:08.734 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:08 smithi166 bash[9446]: cluster 2020-09-08T15:43:07.821709+0000 osd.0 (osd.0) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:08.734 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:08 smithi166 bash[9446]: cluster 2020-09-08T15:43:07.821803+0000 osd.0 (osd.0) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:08.734 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:43:08 smithi166 bash[16593]: debug 2020-09-08T15:43:08.390+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:08.868 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:08 smithi104 bash[9203]: cluster 2020-09-08T15:43:06.805024+0000 osd.7 (osd.7) 47 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:08.868 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:08 smithi104 bash[9203]: cluster 2020-09-08T15:43:06.805126+0000 osd.7 (osd.7) 48 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:08.869 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:08 smithi104 bash[9203]: cluster 2020-09-08T15:43:07.679714+0000 mgr.x (mgr.24118) 473 : cluster [DBG] pgmap v627: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 5.4 GiB data, 12 GiB used, 972 GiB / 983 GiB avail; 79 MiB/s wr, 1.26k op/s; 11092/177892 objects misplaced (6.235%); 352 KiB/s, 5 objects/s recovering 2020-09-08T15:43:08.869 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:08 smithi104 bash[9203]: cluster 2020-09-08T15:43:07.739063+0000 mon.a (mon.0) 1167 : cluster [DBG] osdmap e238: 12 total, 12 up, 11 in 2020-09-08T15:43:08.869 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:08 smithi104 bash[9203]: cluster 2020-09-08T15:43:07.821709+0000 osd.0 (osd.0) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:08.869 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:08 smithi104 bash[9203]: cluster 2020-09-08T15:43:07.821803+0000 osd.0 (osd.0) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:08.875 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:08.978 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:09.116 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:09.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:08 smithi055 bash[9296]: cluster 2020-09-08T15:43:06.805024+0000 osd.7 (osd.7) 47 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:09.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:08 smithi055 bash[9296]: cluster 2020-09-08T15:43:06.805126+0000 osd.7 (osd.7) 48 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:09.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:08 smithi055 bash[9296]: cluster 2020-09-08T15:43:07.679714+0000 mgr.x (mgr.24118) 473 : cluster [DBG] pgmap v627: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 5.4 GiB data, 12 GiB used, 972 GiB / 983 GiB avail; 79 MiB/s wr, 1.26k op/s; 11092/177892 objects misplaced (6.235%); 352 KiB/s, 5 objects/s recovering 2020-09-08T15:43:09.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:08 smithi055 bash[9296]: cluster 2020-09-08T15:43:07.739063+0000 mon.a (mon.0) 1167 : cluster [DBG] osdmap e238: 12 total, 12 up, 11 in 2020-09-08T15:43:09.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:08 smithi055 bash[9296]: cluster 2020-09-08T15:43:07.821709+0000 osd.0 (osd.0) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:09.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:08 smithi055 bash[9296]: cluster 2020-09-08T15:43:07.821803+0000 osd.0 (osd.0) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:09.177 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:43:08 smithi104 bash[22305]: debug 2020-09-08T15:43:08.875+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:09.219 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 49 16 92214 92198 117.587 74.5 0.0136359 0.00850129 2020-09-08T15:43:09.220 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:43:09.227 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:43:09.228 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:43:09.368 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:43:09.368 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:43:09 smithi166 bash[18960]: debug 2020-09-08T15:43:09.114+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:09.395 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2020-09-08T15:43:09.395 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-09-08T15:43:09.396 INFO:tasks.thrashosds.thrasher:Adding osd 1 2020-09-08T15:43:09.396 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd in 1 2020-09-08T15:43:09.470 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:09.559 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:43:09.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:43:09 smithi055 bash[14301]: debug 2020-09-08T15:43:09.364+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:10.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:09 smithi166 bash[9446]: cluster 2020-09-08T15:43:08.745179+0000 mon.a (mon.0) 1168 : cluster [DBG] osdmap e239: 12 total, 12 up, 11 in 2020-09-08T15:43:10.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:09 smithi166 bash[9446]: cluster 2020-09-08T15:43:09.641236+0000 mon.a (mon.0) 1169 : cluster [DBG] osdmap e240: 12 total, 12 up, 11 in 2020-09-08T15:43:10.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:09 smithi055 bash[9296]: cluster 2020-09-08T15:43:08.745179+0000 mon.a (mon.0) 1168 : cluster [DBG] osdmap e239: 12 total, 12 up, 11 in 2020-09-08T15:43:10.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:09 smithi055 bash[9296]: cluster 2020-09-08T15:43:09.641236+0000 mon.a (mon.0) 1169 : cluster [DBG] osdmap e240: 12 total, 12 up, 11 in 2020-09-08T15:43:10.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:09 smithi104 bash[9203]: cluster 2020-09-08T15:43:08.745179+0000 mon.a (mon.0) 1168 : cluster [DBG] osdmap e239: 12 total, 12 up, 11 in 2020-09-08T15:43:10.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:09 smithi104 bash[9203]: cluster 2020-09-08T15:43:09.641236+0000 mon.a (mon.0) 1169 : cluster [DBG] osdmap e240: 12 total, 12 up, 11 in 2020-09-08T15:43:10.222 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 50 16 93032 93016 116.258 51.125 0.0166153 0.00859931 2020-09-08T15:43:10.384 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:10.488 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:43:10.618 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:43:10.723 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:10.744 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:43:10 smithi104 bash[22305]: debug 2020-09-08T15:43:10.383+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:10.865 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:43:10 smithi166 bash[11928]: debug 2020-09-08T15:43:10.618+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:10.871 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:43:11.020 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:43:11.218 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 51 16 94146 94130 115.343 69.625 0.006442 0.00866767 2020-09-08T15:43:11.484 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2020-09-08T15:43:11.581 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:11.643 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:43:11.683 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:11.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:11 smithi055 bash[9296]: cluster 2020-09-08T15:43:09.680097+0000 mgr.x (mgr.24118) 474 : cluster [DBG] pgmap v631: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 5.5 GiB data, 12 GiB used, 972 GiB / 983 GiB avail; 34 MiB/s wr, 550 op/s; 11092/179996 objects misplaced (6.162%); 422 KiB/s, 6 objects/s recovering 2020-09-08T15:43:11.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:43:11 smithi104 bash[22305]: debug 2020-09-08T15:43:11.579+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:11.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:11 smithi104 bash[9203]: cluster 2020-09-08T15:43:09.680097+0000 mgr.x (mgr.24118) 474 : cluster [DBG] pgmap v631: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 5.5 GiB data, 12 GiB used, 972 GiB / 983 GiB avail; 34 MiB/s wr, 550 op/s; 11092/179996 objects misplaced (6.162%); 422 KiB/s, 6 objects/s recovering 2020-09-08T15:43:12.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:11 smithi166 bash[9446]: cluster 2020-09-08T15:43:09.680097+0000 mgr.x (mgr.24118) 474 : cluster [DBG] pgmap v631: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 5.5 GiB data, 12 GiB used, 972 GiB / 983 GiB avail; 34 MiB/s wr, 550 op/s; 11092/179996 objects misplaced (6.162%); 422 KiB/s, 6 objects/s recovering 2020-09-08T15:43:12.218 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 52 16 95426 95410 114.664 80 0.00536993 0.00871741 2020-09-08T15:43:12.894 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:12.997 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:43:13.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:43:12 smithi104 bash[15273]: debug 2020-09-08T15:43:12.895+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:13.219 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 53 16 96318 96302 113.552 55.75 0.0131957 0.0087976 2020-09-08T15:43:13.672 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:43:13.778 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:13.816 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:13 smithi104 bash[9203]: cluster 2020-09-08T15:43:11.680647+0000 mgr.x (mgr.24118) 475 : cluster [DBG] pgmap v632: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 5.6 GiB data, 12 GiB used, 972 GiB / 983 GiB avail; 39 MiB/s wr, 627 op/s; 11092/182016 objects misplaced (6.094%); 352 KiB/s, 5 objects/s recovering 2020-09-08T15:43:13.816 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:43:13 smithi104 bash[19922]: debug 2020-09-08T15:43:13.675+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:13.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:13 smithi055 bash[9296]: cluster 2020-09-08T15:43:11.680647+0000 mgr.x (mgr.24118) 475 : cluster [DBG] pgmap v632: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 5.6 GiB data, 12 GiB used, 972 GiB / 983 GiB avail; 39 MiB/s wr, 627 op/s; 11092/182016 objects misplaced (6.094%); 352 KiB/s, 5 objects/s recovering 2020-09-08T15:43:14.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:13 smithi166 bash[9446]: cluster 2020-09-08T15:43:11.680647+0000 mgr.x (mgr.24118) 475 : cluster [DBG] pgmap v632: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 5.6 GiB data, 12 GiB used, 972 GiB / 983 GiB avail; 39 MiB/s wr, 627 op/s; 11092/182016 objects misplaced (6.094%); 352 KiB/s, 5 objects/s recovering 2020-09-08T15:43:14.219 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 54 16 97475 97459 112.788 72.3125 0.0346967 0.00886132 2020-09-08T15:43:14.514 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:14.618 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:14.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:43:14 smithi104 bash[22305]: debug 2020-09-08T15:43:14.515+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:14.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:14 smithi104 bash[9203]: cluster 2020-09-08T15:43:13.449455+0000 osd.5 (osd.5) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:14.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:14 smithi104 bash[9203]: cluster 2020-09-08T15:43:13.449564+0000 osd.5 (osd.5) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:14.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:14 smithi104 bash[9203]: cluster 2020-09-08T15:43:13.653647+0000 mon.a (mon.0) 1170 : cluster [DBG] osdmap e241: 12 total, 12 up, 11 in 2020-09-08T15:43:14.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:14 smithi104 bash[9203]: cluster 2020-09-08T15:43:14.643389+0000 mon.a (mon.0) 1171 : cluster [DBG] osdmap e242: 12 total, 12 up, 11 in 2020-09-08T15:43:15.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:14 smithi166 bash[9446]: cluster 2020-09-08T15:43:13.449455+0000 osd.5 (osd.5) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:15.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:14 smithi166 bash[9446]: cluster 2020-09-08T15:43:13.449564+0000 osd.5 (osd.5) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:15.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:14 smithi166 bash[9446]: cluster 2020-09-08T15:43:13.653647+0000 mon.a (mon.0) 1170 : cluster [DBG] osdmap e241: 12 total, 12 up, 11 in 2020-09-08T15:43:15.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:14 smithi166 bash[9446]: cluster 2020-09-08T15:43:14.643389+0000 mon.a (mon.0) 1171 : cluster [DBG] osdmap e242: 12 total, 12 up, 11 in 2020-09-08T15:43:15.163 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:14 smithi055 bash[9296]: cluster 2020-09-08T15:43:13.449455+0000 osd.5 (osd.5) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:15.163 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:14 smithi055 bash[9296]: cluster 2020-09-08T15:43:13.449564+0000 osd.5 (osd.5) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:15.163 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:14 smithi055 bash[9296]: cluster 2020-09-08T15:43:13.653647+0000 mon.a (mon.0) 1170 : cluster [DBG] osdmap e241: 12 total, 12 up, 11 in 2020-09-08T15:43:15.164 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:14 smithi055 bash[9296]: cluster 2020-09-08T15:43:14.643389+0000 mon.a (mon.0) 1171 : cluster [DBG] osdmap e242: 12 total, 12 up, 11 in 2020-09-08T15:43:15.222 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 55 16 97813 97797 111.121 21.125 0.00660107 0.00891357 2020-09-08T15:43:15.866 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:15.930 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:43:15 smithi104 bash[22305]: debug 2020-09-08T15:43:15.867+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:15.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:15 smithi104 bash[9203]: cluster 2020-09-08T15:43:13.681190+0000 mgr.x (mgr.24118) 476 : cluster [DBG] pgmap v634: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 5.8 GiB data, 12 GiB used, 971 GiB / 983 GiB avail; 60 MiB/s wr, 963 op/s; 10914/189348 objects misplaced (5.764%); 958 KiB/s, 14 objects/s recovering 2020-09-08T15:43:15.974 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:43:15.985 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:15 smithi055 bash[9296]: cluster 2020-09-08T15:43:13.681190+0000 mgr.x (mgr.24118) 476 : cluster [DBG] pgmap v634: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 5.8 GiB data, 12 GiB used, 971 GiB / 983 GiB avail; 60 MiB/s wr, 963 op/s; 10914/189348 objects misplaced (5.764%); 958 KiB/s, 14 objects/s recovering 2020-09-08T15:43:16.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:15 smithi166 bash[9446]: cluster 2020-09-08T15:43:13.681190+0000 mgr.x (mgr.24118) 476 : cluster [DBG] pgmap v634: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 5.8 GiB data, 12 GiB used, 971 GiB / 983 GiB avail; 60 MiB/s wr, 963 op/s; 10914/189348 objects misplaced (5.764%); 958 KiB/s, 14 objects/s recovering 2020-09-08T15:43:16.219 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 56 16 98820 98804 110.261 62.9375 0.0050728 0.00906572 2020-09-08T15:43:17.114 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:43:17.216 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:43:17.219 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 57 16 100011 99995 109.632 74.4375 0.00494114 0.00911591 2020-09-08T15:43:17.366 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:43:17.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:43:17 smithi055 bash[14301]: debug 2020-09-08T15:43:17.112+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:17.481 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:17.618 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:43:17 smithi166 bash[11928]: debug 2020-09-08T15:43:17.361+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:17.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:17 smithi104 bash[9203]: cluster 2020-09-08T15:43:15.681639+0000 mgr.x (mgr.24118) 477 : cluster [DBG] pgmap v636: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 5.8 GiB data, 12 GiB used, 971 GiB / 983 GiB avail; 55 MiB/s wr, 875 op/s; 10914/190578 objects misplaced (5.727%); 943 KiB/s, 14 objects/s recovering 2020-09-08T15:43:18.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:17 smithi166 bash[9446]: cluster 2020-09-08T15:43:15.681639+0000 mgr.x (mgr.24118) 477 : cluster [DBG] pgmap v636: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 5.8 GiB data, 12 GiB used, 971 GiB / 983 GiB avail; 55 MiB/s wr, 875 op/s; 10914/190578 objects misplaced (5.727%); 943 KiB/s, 14 objects/s recovering 2020-09-08T15:43:18.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:17 smithi055 bash[9296]: cluster 2020-09-08T15:43:15.681639+0000 mgr.x (mgr.24118) 477 : cluster [DBG] pgmap v636: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 5.8 GiB data, 12 GiB used, 971 GiB / 983 GiB avail; 55 MiB/s wr, 875 op/s; 10914/190578 objects misplaced (5.727%); 943 KiB/s, 14 objects/s recovering 2020-09-08T15:43:18.221 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 58 16 100553 100537 108.326 33.875 0.0289424 0.0092291 2020-09-08T15:43:18.458 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:18.562 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:43:18.697 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:18 smithi104 bash[9203]: cluster 2020-09-08T15:43:16.339853+0000 osd.0 (osd.0) 29 : cluster [DBG] 3.3 scrub starts 2020-09-08T15:43:18.697 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:43:18 smithi104 bash[15273]: debug 2020-09-08T15:43:18.459+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:18.703 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:43:18.806 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:18.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:18 smithi166 bash[9446]: cluster 2020-09-08T15:43:16.339853+0000 osd.0 (osd.0) 29 : cluster [DBG] 3.3 scrub starts 2020-09-08T15:43:18.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:43:18 smithi166 bash[11928]: debug 2020-09-08T15:43:18.697+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:18.950 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:19.058 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:43:19.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:18 smithi055 bash[9296]: cluster 2020-09-08T15:43:16.339853+0000 osd.0 (osd.0) 29 : cluster [DBG] 3.3 scrub starts 2020-09-08T15:43:19.222 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 59 16 101679 101663 107.683 70.375 0.0110957 0.00928367 2020-09-08T15:43:19.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:43:18 smithi166 bash[18960]: debug 2020-09-08T15:43:18.945+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:19.646 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:43:20.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:19 smithi166 bash[9446]: cluster 2020-09-08T15:43:17.682391+0000 mgr.x (mgr.24118) 478 : cluster [DBG] pgmap v637: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 6.0 GiB data, 13 GiB used, 971 GiB / 983 GiB avail; 64 MiB/s wr, 1.03k op/s; 10688/196456 objects misplaced (5.440%); 1.6 MiB/s, 25 objects/s recovering 2020-09-08T15:43:20.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:19 smithi166 bash[9446]: cluster 2020-09-08T15:43:18.597090+0000 osd.10 (osd.10) 19 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:20.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:19 smithi166 bash[9446]: cluster 2020-09-08T15:43:18.597175+0000 osd.10 (osd.10) 20 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:20.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:19 smithi166 bash[9446]: cluster 2020-09-08T15:43:18.686485+0000 mon.a (mon.0) 1172 : cluster [DBG] osdmap e243: 12 total, 12 up, 11 in 2020-09-08T15:43:20.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:19 smithi166 bash[9446]: cluster 2020-09-08T15:43:18.764825+0000 osd.0 (osd.0) 30 : cluster [DBG] 3.3 scrub ok 2020-09-08T15:43:20.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:19 smithi166 bash[9446]: audit 2020-09-08T15:43:18.778416+0000 mon.a (mon.0) 1173 : audit [INF] from='client.? v1:172.21.15.104:0/1017723941' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:43:20.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:19 smithi166 bash[9446]: cluster 2020-09-08T15:43:19.639053+0000 mon.a (mon.0) 1174 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:43:20.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:19 smithi166 bash[9446]: audit 2020-09-08T15:43:19.646420+0000 mon.a (mon.0) 1175 : audit [INF] from='client.? v1:172.21.15.104:0/1017723941' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:43:20.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:19 smithi166 bash[9446]: cluster 2020-09-08T15:43:19.646473+0000 mon.a (mon.0) 1176 : cluster [DBG] osdmap e244: 12 total, 12 up, 11 in 2020-09-08T15:43:20.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:19 smithi055 bash[9296]: cluster 2020-09-08T15:43:17.682391+0000 mgr.x (mgr.24118) 478 : cluster [DBG] pgmap v637: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 6.0 GiB data, 13 GiB used, 971 GiB / 983 GiB avail; 64 MiB/s wr, 1.03k op/s; 10688/196456 objects misplaced (5.440%); 1.6 MiB/s, 25 objects/s recovering 2020-09-08T15:43:20.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:19 smithi055 bash[9296]: cluster 2020-09-08T15:43:18.597090+0000 osd.10 (osd.10) 19 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:20.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:19 smithi055 bash[9296]: cluster 2020-09-08T15:43:18.597175+0000 osd.10 (osd.10) 20 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:20.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:19 smithi055 bash[9296]: cluster 2020-09-08T15:43:18.686485+0000 mon.a (mon.0) 1172 : cluster [DBG] osdmap e243: 12 total, 12 up, 11 in 2020-09-08T15:43:20.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:19 smithi055 bash[9296]: cluster 2020-09-08T15:43:18.764825+0000 osd.0 (osd.0) 30 : cluster [DBG] 3.3 scrub ok 2020-09-08T15:43:20.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:19 smithi055 bash[9296]: audit 2020-09-08T15:43:18.778416+0000 mon.a (mon.0) 1173 : audit [INF] from='client.? v1:172.21.15.104:0/1017723941' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:43:20.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:19 smithi055 bash[9296]: cluster 2020-09-08T15:43:19.639053+0000 mon.a (mon.0) 1174 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:43:20.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:19 smithi055 bash[9296]: audit 2020-09-08T15:43:19.646420+0000 mon.a (mon.0) 1175 : audit [INF] from='client.? v1:172.21.15.104:0/1017723941' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:43:20.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:19 smithi055 bash[9296]: cluster 2020-09-08T15:43:19.646473+0000 mon.a (mon.0) 1176 : cluster [DBG] osdmap e244: 12 total, 12 up, 11 in 2020-09-08T15:43:20.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:19 smithi104 bash[9203]: cluster 2020-09-08T15:43:17.682391+0000 mgr.x (mgr.24118) 478 : cluster [DBG] pgmap v637: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 6.0 GiB data, 13 GiB used, 971 GiB / 983 GiB avail; 64 MiB/s wr, 1.03k op/s; 10688/196456 objects misplaced (5.440%); 1.6 MiB/s, 25 objects/s recovering 2020-09-08T15:43:20.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:19 smithi104 bash[9203]: cluster 2020-09-08T15:43:18.597090+0000 osd.10 (osd.10) 19 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:20.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:19 smithi104 bash[9203]: cluster 2020-09-08T15:43:18.597175+0000 osd.10 (osd.10) 20 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:20.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:19 smithi104 bash[9203]: cluster 2020-09-08T15:43:18.686485+0000 mon.a (mon.0) 1172 : cluster [DBG] osdmap e243: 12 total, 12 up, 11 in 2020-09-08T15:43:20.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:19 smithi104 bash[9203]: cluster 2020-09-08T15:43:18.764825+0000 osd.0 (osd.0) 30 : cluster [DBG] 3.3 scrub ok 2020-09-08T15:43:20.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:19 smithi104 bash[9203]: audit 2020-09-08T15:43:18.778416+0000 mon.a (mon.0) 1173 : audit [INF] from='client.? v1:172.21.15.104:0/1017723941' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:43:20.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:19 smithi104 bash[9203]: cluster 2020-09-08T15:43:19.639053+0000 mon.a (mon.0) 1174 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:43:20.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:19 smithi104 bash[9203]: audit 2020-09-08T15:43:19.646420+0000 mon.a (mon.0) 1175 : audit [INF] from='client.? v1:172.21.15.104:0/1017723941' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:43:20.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:19 smithi104 bash[9203]: cluster 2020-09-08T15:43:19.646473+0000 mon.a (mon.0) 1176 : cluster [DBG] osdmap e244: 12 total, 12 up, 11 in 2020-09-08T15:43:20.219 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 15:43:20.224934 min lat: 0.00154046 max lat: 2.89809 avg lat: 0.00934377 2020-09-08T15:43:20.219 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:43:20.220 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 60 16 102749 102733 107.003 66.875 0.0106712 0.00934377 2020-09-08T15:43:20.334 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:43:20.436 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:43:20.589 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:43:20.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:43:20 smithi104 bash[19922]: debug 2020-09-08T15:43:20.335+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:20.698 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:20.877 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:20.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:43:20 smithi055 bash[16646]: debug 2020-09-08T15:43:20.587+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:20.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:20 smithi055 bash[9296]: cluster 2020-09-08T15:43:19.683001+0000 mgr.x (mgr.24118) 479 : cluster [DBG] pgmap v640: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 6.1 GiB data, 13 GiB used, 970 GiB / 983 GiB avail; 53 MiB/s wr, 843 op/s; 10688/199474 objects misplaced (5.358%); 1.2 MiB/s, 18 objects/s recovering 2020-09-08T15:43:20.983 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:43:21.101 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:43:21.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:20 smithi166 bash[9446]: cluster 2020-09-08T15:43:19.683001+0000 mgr.x (mgr.24118) 479 : cluster [DBG] pgmap v640: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 6.1 GiB data, 13 GiB used, 970 GiB / 983 GiB avail; 53 MiB/s wr, 843 op/s; 10688/199474 objects misplaced (5.358%); 1.2 MiB/s, 18 objects/s recovering 2020-09-08T15:43:21.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:43:20 smithi166 bash[18960]: debug 2020-09-08T15:43:20.877+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:21.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:20 smithi104 bash[9203]: cluster 2020-09-08T15:43:19.683001+0000 mgr.x (mgr.24118) 479 : cluster [DBG] pgmap v640: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 6.1 GiB data, 13 GiB used, 970 GiB / 983 GiB avail; 53 MiB/s wr, 843 op/s; 10688/199474 objects misplaced (5.358%); 1.2 MiB/s, 18 objects/s recovering 2020-09-08T15:43:21.208 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:21.219 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 61 15 103773 103758 106.299 64.0625 0.0243882 0.0094041 2020-09-08T15:43:21.334 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:21.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:43:21 smithi055 bash[19003]: debug 2020-09-08T15:43:21.095+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:21.436 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:21.615 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:43:21 smithi166 bash[18960]: debug 2020-09-08T15:43:21.325+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:22.219 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 62 16 104918 104902 105.737 71.5 0.015083 0.00945548 2020-09-08T15:43:22.250 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:22.358 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:43:22.507 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:43:22.580 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:43:22 smithi104 bash[22305]: debug 2020-09-08T15:43:22.251+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:22.581 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:22 smithi104 bash[9203]: debug 2020-09-08T15:43:22.583+0000 7efd04a72700 -1 mon.a@0(leader).osd e244 definitely_dead 0 2020-09-08T15:43:22.612 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:22.615 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:43:22 smithi166 bash[14254]: debug 2020-09-08T15:43:22.509+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:22.712 INFO:teuthology.orchestra.run.smithi104.stderr:marked in osd.1. 2020-09-08T15:43:22.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:22 smithi104 bash[9203]: cluster 2020-09-08T15:43:21.683506+0000 mgr.x (mgr.24118) 480 : cluster [DBG] pgmap v641: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 6.1 GiB data, 13 GiB used, 970 GiB / 983 GiB avail; 48 MiB/s wr, 768 op/s; 10688/201406 objects misplaced (5.307%); 1.0 MiB/s, 16 objects/s recovering 2020-09-08T15:43:22.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:22 smithi104 bash[9203]: audit 2020-09-08T15:43:22.585265+0000 mon.b (mon.1) 85 : audit [DBG] from='client.? v1:172.21.15.104:0/326974886' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2020-09-08T15:43:22.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:22 smithi104 bash[9203]: audit 2020-09-08T15:43:22.585927+0000 mon.a (mon.0) 1177 : audit [DBG] from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2020-09-08T15:43:23.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:22 smithi166 bash[9446]: cluster 2020-09-08T15:43:21.683506+0000 mgr.x (mgr.24118) 480 : cluster [DBG] pgmap v641: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 6.1 GiB data, 13 GiB used, 970 GiB / 983 GiB avail; 48 MiB/s wr, 768 op/s; 10688/201406 objects misplaced (5.307%); 1.0 MiB/s, 16 objects/s recovering 2020-09-08T15:43:23.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:22 smithi166 bash[9446]: audit 2020-09-08T15:43:22.585265+0000 mon.b (mon.1) 85 : audit [DBG] from='client.? v1:172.21.15.104:0/326974886' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2020-09-08T15:43:23.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:22 smithi166 bash[9446]: audit 2020-09-08T15:43:22.585927+0000 mon.a (mon.0) 1177 : audit [DBG] from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2020-09-08T15:43:23.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:22 smithi055 bash[9296]: cluster 2020-09-08T15:43:21.683506+0000 mgr.x (mgr.24118) 480 : cluster [DBG] pgmap v641: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 6.1 GiB data, 13 GiB used, 970 GiB / 983 GiB avail; 48 MiB/s wr, 768 op/s; 10688/201406 objects misplaced (5.307%); 1.0 MiB/s, 16 objects/s recovering 2020-09-08T15:43:23.162 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:22 smithi055 bash[9296]: audit 2020-09-08T15:43:22.585265+0000 mon.b (mon.1) 85 : audit [DBG] from='client.? v1:172.21.15.104:0/326974886' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2020-09-08T15:43:23.162 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:22 smithi055 bash[9296]: audit 2020-09-08T15:43:22.585927+0000 mon.a (mon.0) 1177 : audit [DBG] from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2020-09-08T15:43:23.220 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 63 16 105463 105447 104.599 34.0625 0.00593576 0.00948784 2020-09-08T15:43:23.328 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:23.434 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:43:23.579 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:43:23.678 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:43:23 smithi104 bash[15273]: debug 2020-09-08T15:43:23.331+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:23.687 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:43:23.705 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:43:23 smithi055 bash[19003]: debug 2020-09-08T15:43:23.579+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:23.854 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:43:23.958 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:43:23.971 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:23 smithi055 bash[9296]: cluster 2020-09-08T15:43:22.362875+0000 osd.3 (osd.3) 25 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:23.972 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:23 smithi055 bash[9296]: cluster 2020-09-08T15:43:22.362948+0000 osd.3 (osd.3) 26 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:23.972 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:23 smithi055 bash[9296]: audit 2020-09-08T15:43:22.704723+0000 mon.a (mon.0) 1178 : audit [INF] from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["1"]}]': finished 2020-09-08T15:43:23.972 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:23 smithi055 bash[9296]: cluster 2020-09-08T15:43:22.704776+0000 mon.a (mon.0) 1179 : cluster [DBG] osdmap e245: 12 total, 12 up, 12 in 2020-09-08T15:43:24.080 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:43:24.115 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:43:23 smithi166 bash[14254]: debug 2020-09-08T15:43:23.853+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:24.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:23 smithi166 bash[9446]: cluster 2020-09-08T15:43:22.362875+0000 osd.3 (osd.3) 25 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:24.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:23 smithi166 bash[9446]: cluster 2020-09-08T15:43:22.362948+0000 osd.3 (osd.3) 26 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:24.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:23 smithi166 bash[9446]: audit 2020-09-08T15:43:22.704723+0000 mon.a (mon.0) 1178 : audit [INF] from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["1"]}]': finished 2020-09-08T15:43:24.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:23 smithi166 bash[9446]: cluster 2020-09-08T15:43:22.704776+0000 mon.a (mon.0) 1179 : cluster [DBG] osdmap e245: 12 total, 12 up, 12 in 2020-09-08T15:43:24.143 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:23 smithi104 bash[9203]: cluster 2020-09-08T15:43:22.362875+0000 osd.3 (osd.3) 25 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:24.144 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:23 smithi104 bash[9203]: cluster 2020-09-08T15:43:22.362948+0000 osd.3 (osd.3) 26 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:24.144 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:23 smithi104 bash[9203]: audit 2020-09-08T15:43:22.704723+0000 mon.a (mon.0) 1178 : audit [INF] from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["1"]}]': finished 2020-09-08T15:43:24.144 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:23 smithi104 bash[9203]: cluster 2020-09-08T15:43:22.704776+0000 mon.a (mon.0) 1179 : cluster [DBG] osdmap e245: 12 total, 12 up, 12 in 2020-09-08T15:43:24.186 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:24.219 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 64 16 105575 105559 103.074 7 0.0239612 0.00963762 2020-09-08T15:43:24.262 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:43:24.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:43:24 smithi055 bash[16646]: debug 2020-09-08T15:43:24.079+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:24.416 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:43:24.492 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:24.538 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:43:24.563 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:43:24.596 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:43:24.598 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:43:24.623 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:43:24.653 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:43:24.684 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:43:24.707 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:43:24.708 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:43:24.739 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:43:24.772 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:43:24.796 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:43:24.812 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:43:24.824 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:43:24.831 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:24 smithi104 bash[9203]: cluster 2020-09-08T15:43:23.684253+0000 mgr.x (mgr.24118) 481 : cluster [DBG] pgmap v643: 17 pgs: 1 peering, 1 active+recovering+remapped, 15 active+clean; 6.3 GiB data, 14 GiB used, 970 GiB / 983 GiB avail; 58 MiB/s wr, 931 op/s; 10482/207636 objects misplaced (5.048%); 1.1 MiB/s, 17 objects/s recovering 2020-09-08T15:43:24.832 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:24 smithi104 bash[9203]: cluster 2020-09-08T15:43:23.714447+0000 mon.a (mon.0) 1180 : cluster [DBG] osdmap e246: 12 total, 12 up, 12 in 2020-09-08T15:43:24.832 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:24 smithi104 bash[9203]: audit 2020-09-08T15:43:24.511504+0000 mon.a (mon.0) 1181 : audit [DBG] from='client.? v1:172.21.15.104:0/1532484084' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:43:24.832 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:43:24 smithi104 bash[15273]: debug 2020-09-08T15:43:24.495+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:24.865 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:43:24.867 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:43:24.867 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:43:24.867 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:43:24.868 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:43:24.868 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:43:24.868 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:43:24.868 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:43:24.869 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:43:24.869 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:43:24.869 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:43:24.870 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:43:24.870 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:43:24.908 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:43:24 smithi055 bash[19003]: debug 2020-09-08T15:43:24.707+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:24.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:24 smithi055 bash[9296]: cluster 2020-09-08T15:43:23.684253+0000 mgr.x (mgr.24118) 481 : cluster [DBG] pgmap v643: 17 pgs: 1 peering, 1 active+recovering+remapped, 15 active+clean; 6.3 GiB data, 14 GiB used, 970 GiB / 983 GiB avail; 58 MiB/s wr, 931 op/s; 10482/207636 objects misplaced (5.048%); 1.1 MiB/s, 17 objects/s recovering 2020-09-08T15:43:24.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:24 smithi055 bash[9296]: cluster 2020-09-08T15:43:23.714447+0000 mon.a (mon.0) 1180 : cluster [DBG] osdmap e246: 12 total, 12 up, 12 in 2020-09-08T15:43:24.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:24 smithi055 bash[9296]: audit 2020-09-08T15:43:24.511504+0000 mon.a (mon.0) 1181 : audit [DBG] from='client.? v1:172.21.15.104:0/1532484084' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:43:24.917 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:43:25.022 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:25.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:24 smithi166 bash[9446]: cluster 2020-09-08T15:43:23.684253+0000 mgr.x (mgr.24118) 481 : cluster [DBG] pgmap v643: 17 pgs: 1 peering, 1 active+recovering+remapped, 15 active+clean; 6.3 GiB data, 14 GiB used, 970 GiB / 983 GiB avail; 58 MiB/s wr, 931 op/s; 10482/207636 objects misplaced (5.048%); 1.1 MiB/s, 17 objects/s recovering 2020-09-08T15:43:25.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:24 smithi166 bash[9446]: cluster 2020-09-08T15:43:23.714447+0000 mon.a (mon.0) 1180 : cluster [DBG] osdmap e246: 12 total, 12 up, 12 in 2020-09-08T15:43:25.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:24 smithi166 bash[9446]: audit 2020-09-08T15:43:24.511504+0000 mon.a (mon.0) 1181 : audit [DBG] from='client.? v1:172.21.15.104:0/1532484084' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:43:25.220 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 65 16 105713 105697 101.621 8.625 0.00863065 0.00983355 2020-09-08T15:43:25.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:43:24 smithi055 bash[11956]: debug 2020-09-08T15:43:24.915+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:25.765 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:25.784 INFO:tasks.thrashosds.thrasher:Added osd 1 2020-09-08T15:43:25.869 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:43:25.970 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:43:25.970 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T15:43:25.971 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 0 2020-09-08T15:43:25.971 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:43:26.006 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:25 smithi055 bash[9296]: cluster 2020-09-08T15:43:24.737329+0000 mon.a (mon.0) 1182 : cluster [DBG] osdmap e247: 12 total, 12 up, 12 in 2020-09-08T15:43:26.006 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:43:26.109 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:26.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:25 smithi166 bash[9446]: cluster 2020-09-08T15:43:24.737329+0000 mon.a (mon.0) 1182 : cluster [DBG] osdmap e247: 12 total, 12 up, 12 in 2020-09-08T15:43:26.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:43:25 smithi104 bash[15273]: debug 2020-09-08T15:43:25.767+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:26.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:25 smithi104 bash[9203]: cluster 2020-09-08T15:43:24.737329+0000 mon.a (mon.0) 1182 : cluster [DBG] osdmap e247: 12 total, 12 up, 12 in 2020-09-08T15:43:26.220 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 66 16 106471 106455 100.799 47.375 0.00574019 0.00991571 2020-09-08T15:43:26.409 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:43:26 smithi055 bash[19003]: debug 2020-09-08T15:43:26.007+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:26.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:26 smithi104 bash[9203]: cluster 2020-09-08T15:43:25.684809+0000 mgr.x (mgr.24118) 482 : cluster [DBG] pgmap v646: 17 pgs: 1 peering, 1 active+recovering+remapped, 15 active+clean; 6.4 GiB data, 14 GiB used, 970 GiB / 983 GiB avail; 48 MiB/s wr, 770 op/s; 10482/208720 objects misplaced (5.022%); 1.1 MiB/s, 17 objects/s recovering 2020-09-08T15:43:27.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:26 smithi166 bash[9446]: cluster 2020-09-08T15:43:25.684809+0000 mgr.x (mgr.24118) 482 : cluster [DBG] pgmap v646: 17 pgs: 1 peering, 1 active+recovering+remapped, 15 active+clean; 6.4 GiB data, 14 GiB used, 970 GiB / 983 GiB avail; 48 MiB/s wr, 770 op/s; 10482/208720 objects misplaced (5.022%); 1.1 MiB/s, 17 objects/s recovering 2020-09-08T15:43:27.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:26 smithi055 bash[9296]: cluster 2020-09-08T15:43:25.684809+0000 mgr.x (mgr.24118) 482 : cluster [DBG] pgmap v646: 17 pgs: 1 peering, 1 active+recovering+remapped, 15 active+clean; 6.4 GiB data, 14 GiB used, 970 GiB / 983 GiB avail; 48 MiB/s wr, 770 op/s; 10482/208720 objects misplaced (5.022%); 1.1 MiB/s, 17 objects/s recovering 2020-09-08T15:43:27.158 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:27.220 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 67 16 107194 107178 99.9693 45.1875 0.0118387 0.01 2020-09-08T15:43:27.261 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:27.386 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:27.490 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:27.500 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:43:27 smithi166 bash[18960]: debug 2020-09-08T15:43:27.157+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:27.500 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:43:27 smithi166 bash[18960]: debug 2020-09-08T15:43:27.385+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:27.594 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:27.705 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:27.824 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:43:27 smithi166 bash[18960]: debug 2020-09-08T15:43:27.593+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:27.824 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:43:27 smithi166 bash[18960]: debug 2020-09-08T15:43:27.825+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:27.827 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:27.933 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:43:28.077 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:43:28 smithi166 bash[14254]: debug 2020-09-08T15:43:28.077+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:28.078 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:43:28.184 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:28.220 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 68 16 107997 107981 99.2372 50.1875 0.00979756 0.0100732 2020-09-08T15:43:28.292 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:28.365 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:43:28 smithi166 bash[18960]: debug 2020-09-08T15:43:28.289+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:28.402 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:28.527 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:28.633 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:28.748 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:28 smithi166 bash[9446]: cluster 2020-09-08T15:43:27.685421+0000 mgr.x (mgr.24118) 483 : cluster [DBG] pgmap v647: 17 pgs: 1 peering, 2 active+recovering+remapped, 1 active+recovering, 13 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s wr, 940 op/s; 23326/212700 objects misplaced (10.967%); 3.3 MiB/s, 53 objects/s recovering 2020-09-08T15:43:28.749 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:43:28 smithi166 bash[16593]: debug 2020-09-08T15:43:28.521+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:28.859 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:28 smithi104 bash[9203]: cluster 2020-09-08T15:43:27.685421+0000 mgr.x (mgr.24118) 483 : cluster [DBG] pgmap v647: 17 pgs: 1 peering, 2 active+recovering+remapped, 1 active+recovering, 13 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s wr, 940 op/s; 23326/212700 objects misplaced (10.967%); 3.3 MiB/s, 53 objects/s recovering 2020-09-08T15:43:28.862 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:28.884 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2020-09-08T15:43:28.974 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:43:29.090 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:43:29.097 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:43:29.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:28 smithi055 bash[9296]: cluster 2020-09-08T15:43:27.685421+0000 mgr.x (mgr.24118) 483 : cluster [DBG] pgmap v647: 17 pgs: 1 peering, 2 active+recovering+remapped, 1 active+recovering, 13 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s wr, 940 op/s; 23326/212700 objects misplaced (10.967%); 3.3 MiB/s, 53 objects/s recovering 2020-09-08T15:43:29.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:43:28 smithi104 bash[15273]: debug 2020-09-08T15:43:28.859+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:29.192 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:43:29.220 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 69 16 108722 108706 98.4556 45.3125 0.00654531 0.0101539 2020-09-08T15:43:29.333 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:43:29.365 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:43:29 smithi166 bash[11928]: debug 2020-09-08T15:43:29.089+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:29.439 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:43:29.662 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:43:29 smithi055 bash[19003]: debug 2020-09-08T15:43:29.331+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:29.751 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:29 smithi104 bash[9203]: cluster 2020-09-08T15:43:29.139950+0000 osd.8 (osd.8) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:29.752 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:29 smithi104 bash[9203]: cluster 2020-09-08T15:43:29.140053+0000 osd.8 (osd.8) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:30.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:29 smithi166 bash[9446]: cluster 2020-09-08T15:43:29.139950+0000 osd.8 (osd.8) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:30.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:29 smithi166 bash[9446]: cluster 2020-09-08T15:43:29.140053+0000 osd.8 (osd.8) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:30.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:29 smithi055 bash[9296]: cluster 2020-09-08T15:43:29.139950+0000 osd.8 (osd.8) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:30.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:29 smithi055 bash[9296]: cluster 2020-09-08T15:43:29.140053+0000 osd.8 (osd.8) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:30.220 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 70 16 109494 109478 97.7384 48.25 0.00637522 0.0102276 2020-09-08T15:43:30.584 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:43:30.598 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:43:30.598 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:43:30 smithi104 bash[17588]: debug 2020-09-08T15:43:30.591+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:30.700 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:43:30.782 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:43:30.785 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2020-09-08T15:43:30.785 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-09-08T15:43:30.787 INFO:tasks.thrashosds.thrasher:fixing pg num pool device_health_metrics 2020-09-08T15:43:30.788 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph pg dump --format=json 2020-09-08T15:43:30.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:30 smithi104 bash[9203]: cluster 2020-09-08T15:43:29.686029+0000 mgr.x (mgr.24118) 484 : cluster [DBG] pgmap v648: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s wr, 615 op/s; 29768/214320 objects misplaced (13.890%); 2.8 MiB/s, 45 objects/s recovering 2020-09-08T15:43:30.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:30 smithi104 bash[9203]: cluster 2020-09-08T15:43:29.765398+0000 mon.a (mon.0) 1183 : cluster [DBG] osdmap e248: 12 total, 12 up, 12 in 2020-09-08T15:43:30.968 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:43:31.126 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:30 smithi166 bash[9446]: cluster 2020-09-08T15:43:29.686029+0000 mgr.x (mgr.24118) 484 : cluster [DBG] pgmap v648: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s wr, 615 op/s; 29768/214320 objects misplaced (13.890%); 2.8 MiB/s, 45 objects/s recovering 2020-09-08T15:43:31.126 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:30 smithi166 bash[9446]: cluster 2020-09-08T15:43:29.765398+0000 mon.a (mon.0) 1183 : cluster [DBG] osdmap e248: 12 total, 12 up, 12 in 2020-09-08T15:43:31.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:30 smithi055 bash[9296]: cluster 2020-09-08T15:43:29.686029+0000 mgr.x (mgr.24118) 484 : cluster [DBG] pgmap v648: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s wr, 615 op/s; 29768/214320 objects misplaced (13.890%); 2.8 MiB/s, 45 objects/s recovering 2020-09-08T15:43:31.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:30 smithi055 bash[9296]: cluster 2020-09-08T15:43:29.765398+0000 mon.a (mon.0) 1183 : cluster [DBG] osdmap e248: 12 total, 12 up, 12 in 2020-09-08T15:43:31.220 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 71 16 110263 110247 97.0386 48.0625 0.00697441 0.0103011 2020-09-08T15:43:32.038 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:43:32.145 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:32.224 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 72 16 111063 111047 96.3853 50 0.0102336 0.0103709 2020-09-08T15:43:32.531 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:43:32 smithi104 bash[17588]: debug 2020-09-08T15:43:32.035+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:33.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:32 smithi166 bash[9446]: cluster 2020-09-08T15:43:31.686621+0000 mgr.x (mgr.24118) 485 : cluster [DBG] pgmap v650: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s wr, 439 op/s; 29673/215736 objects misplaced (13.754%); 3.2 MiB/s, 51 objects/s recovering 2020-09-08T15:43:33.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:32 smithi055 bash[9296]: cluster 2020-09-08T15:43:31.686621+0000 mgr.x (mgr.24118) 485 : cluster [DBG] pgmap v650: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s wr, 439 op/s; 29673/215736 objects misplaced (13.754%); 3.2 MiB/s, 51 objects/s recovering 2020-09-08T15:43:33.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:32 smithi104 bash[9203]: cluster 2020-09-08T15:43:31.686621+0000 mgr.x (mgr.24118) 485 : cluster [DBG] pgmap v650: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s wr, 439 op/s; 29673/215736 objects misplaced (13.754%); 3.2 MiB/s, 51 objects/s recovering 2020-09-08T15:43:33.220 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 73 16 111831 111815 95.7224 48 0.00974185 0.0104449 2020-09-08T15:43:34.226 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 74 16 112594 112578 95.0733 47.6875 0.017721 0.0105137 2020-09-08T15:43:34.434 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:34.538 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:34.865 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:43:34 smithi166 bash[16593]: debug 2020-09-08T15:43:34.433+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:35.221 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 75 16 113289 113273 94.3847 43.4375 0.0071631 0.0105911 2020-09-08T15:43:35.511 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:35.614 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:43:35.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:35 smithi166 bash[9446]: cluster 2020-09-08T15:43:33.687174+0000 mgr.x (mgr.24118) 486 : cluster [DBG] pgmap v651: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s wr, 688 op/s; 29221/221046 objects misplaced (13.219%); 5.3 MiB/s, 85 objects/s recovering 2020-09-08T15:43:35.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:43:35 smithi166 bash[16593]: debug 2020-09-08T15:43:35.509+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:35.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:35 smithi055 bash[9296]: cluster 2020-09-08T15:43:33.687174+0000 mgr.x (mgr.24118) 486 : cluster [DBG] pgmap v651: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s wr, 688 op/s; 29221/221046 objects misplaced (13.219%); 5.3 MiB/s, 85 objects/s recovering 2020-09-08T15:43:35.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:35 smithi104 bash[9203]: cluster 2020-09-08T15:43:33.687174+0000 mgr.x (mgr.24118) 486 : cluster [DBG] pgmap v651: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s wr, 688 op/s; 29221/221046 objects misplaced (13.219%); 5.3 MiB/s, 85 objects/s recovering 2020-09-08T15:43:36.221 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 76 16 113970 113954 93.7028 42.5625 0.00784403 0.0106662 2020-09-08T15:43:36.528 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:43:36.631 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:43:36.809 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:43:36 smithi104 bash[17588]: debug 2020-09-08T15:43:36.527+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:36.810 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:43:36.915 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:37.055 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:37.164 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:43:37.178 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:43:36 smithi104 bash[17588]: debug 2020-09-08T15:43:36.803+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:37.221 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 77 16 114752 114736 93.1206 48.875 0.00943035 0.0107348 2020-09-08T15:43:37.365 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:43:37 smithi166 bash[16593]: debug 2020-09-08T15:43:37.053+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:37.390 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:43:37.503 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:37.618 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:37.654 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:43:37 smithi055 bash[16646]: debug 2020-09-08T15:43:37.391+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:37.655 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:37 smithi055 bash[9296]: cluster 2020-09-08T15:43:35.687678+0000 mgr.x (mgr.24118) 487 : cluster [DBG] pgmap v652: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s wr, 661 op/s; 29221/221956 objects misplaced (13.165%); 4.8 MiB/s, 76 objects/s recovering 2020-09-08T15:43:37.655 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:37 smithi055 bash[9296]: cluster 2020-09-08T15:43:35.869854+0000 osd.1 (osd.1) 48 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:37.655 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:37 smithi055 bash[9296]: cluster 2020-09-08T15:43:35.869953+0000 osd.1 (osd.1) 49 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:37.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:37 smithi055 bash[9296]: cluster 2020-09-08T15:43:36.660855+0000 mon.a (mon.0) 1184 : cluster [DBG] osdmap e249: 12 total, 12 up, 12 in 2020-09-08T15:43:37.724 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:43:37.837 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:43:37.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:37 smithi166 bash[9446]: cluster 2020-09-08T15:43:35.687678+0000 mgr.x (mgr.24118) 487 : cluster [DBG] pgmap v652: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s wr, 661 op/s; 29221/221956 objects misplaced (13.165%); 4.8 MiB/s, 76 objects/s recovering 2020-09-08T15:43:37.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:37 smithi166 bash[9446]: cluster 2020-09-08T15:43:35.869854+0000 osd.1 (osd.1) 48 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:37.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:37 smithi166 bash[9446]: cluster 2020-09-08T15:43:35.869953+0000 osd.1 (osd.1) 49 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:37.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:37 smithi166 bash[9446]: cluster 2020-09-08T15:43:36.660855+0000 mon.a (mon.0) 1184 : cluster [DBG] osdmap e249: 12 total, 12 up, 12 in 2020-09-08T15:43:37.867 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:43:37 smithi166 bash[16593]: debug 2020-09-08T15:43:37.617+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:37.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:43:37 smithi055 bash[19003]: debug 2020-09-08T15:43:37.835+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:37.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:37 smithi104 bash[9203]: cluster 2020-09-08T15:43:35.687678+0000 mgr.x (mgr.24118) 487 : cluster [DBG] pgmap v652: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s wr, 661 op/s; 29221/221956 objects misplaced (13.165%); 4.8 MiB/s, 76 objects/s recovering 2020-09-08T15:43:37.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:37 smithi104 bash[9203]: cluster 2020-09-08T15:43:35.869854+0000 osd.1 (osd.1) 48 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:37.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:37 smithi104 bash[9203]: cluster 2020-09-08T15:43:35.869953+0000 osd.1 (osd.1) 49 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:37.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:37 smithi104 bash[9203]: cluster 2020-09-08T15:43:36.660855+0000 mon.a (mon.0) 1184 : cluster [DBG] osdmap e249: 12 total, 12 up, 12 in 2020-09-08T15:43:37.948 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:43:38.069 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:43:38.175 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:38.226 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 78 16 115462 115446 92.4956 44.375 0.0640547 0.0108086 2020-09-08T15:43:38.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:43:38 smithi055 bash[19003]: debug 2020-09-08T15:43:38.067+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:39.084 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:39.191 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:39.225 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 79 16 116268 116252 91.9624 50.375 0.0112991 0.0108693 2020-09-08T15:43:39.386 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:39.403 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:43:39 smithi104 bash[22305]: debug 2020-09-08T15:43:39.087+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:39.490 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:43:39.629 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:43:39.656 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:43:39 smithi104 bash[15273]: debug 2020-09-08T15:43:39.379+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:39.656 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:39 smithi104 bash[9203]: cluster 2020-09-08T15:43:37.688134+0000 mgr.x (mgr.24118) 488 : cluster [DBG] pgmap v654: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s wr, 796 op/s; 28784/227062 objects misplaced (12.677%); 6.1 MiB/s, 97 objects/s recovering 2020-09-08T15:43:39.734 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:43:39.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:39 smithi166 bash[9446]: cluster 2020-09-08T15:43:37.688134+0000 mgr.x (mgr.24118) 488 : cluster [DBG] pgmap v654: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s wr, 796 op/s; 28784/227062 objects misplaced (12.677%); 6.1 MiB/s, 97 objects/s recovering 2020-09-08T15:43:39.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:43:39 smithi166 bash[14254]: debug 2020-09-08T15:43:39.629+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:39.874 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:43:39.908 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:43:39 smithi055 bash[11956]: debug 2020-09-08T15:43:39.875+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:39.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:39 smithi055 bash[9296]: cluster 2020-09-08T15:43:37.688134+0000 mgr.x (mgr.24118) 488 : cluster [DBG] pgmap v654: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s wr, 796 op/s; 28784/227062 objects misplaced (12.677%); 6.1 MiB/s, 97 objects/s recovering 2020-09-08T15:43:39.982 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:43:40.105 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:43:40.221 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 15:43:40.226726 min lat: 0.00154046 max lat: 2.89809 avg lat: 0.0109379 2020-09-08T15:43:40.222 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:43:40.222 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 80 16 117017 117001 91.398 46.8125 0.0130393 0.0109379 2020-09-08T15:43:40.231 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:40.367 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:40.412 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:43:40 smithi055 bash[11956]: debug 2020-09-08T15:43:40.087+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:40.472 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:43:40.577 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:43:40 smithi166 bash[14254]: debug 2020-09-08T15:43:40.577+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:40.577 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:43:40 smithi166 bash[16593]: debug 2020-09-08T15:43:40.365+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:40.586 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:43:40.679 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:43:40.688 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:40.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:40 smithi166 bash[9446]: audit 2020-09-08T15:43:40.606864+0000 mon.a (mon.0) 1185 : audit [INF] from='client.? v1:172.21.15.104:0/908233471' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:43:40.968 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:40 smithi104 bash[9203]: audit 2020-09-08T15:43:40.606864+0000 mon.a (mon.0) 1185 : audit [INF] from='client.? v1:172.21.15.104:0/908233471' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:43:41.027 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:41.130 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:43:41.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:40 smithi055 bash[9296]: audit 2020-09-08T15:43:40.606864+0000 mon.a (mon.0) 1185 : audit [INF] from='client.? v1:172.21.15.104:0/908233471' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:43:41.229 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 81 16 117682 117666 90.781 41.5625 0.0173015 0.0110126 2020-09-08T15:43:41.255 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:43:41.356 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:43:41.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:43:41 smithi104 bash[22305]: debug 2020-09-08T15:43:41.023+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:41.484 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:43:41 smithi166 bash[14254]: debug 2020-09-08T15:43:41.253+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:41.485 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:43:41.590 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:43:41.714 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:43:41.819 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:41.865 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:43:41 smithi166 bash[14254]: debug 2020-09-08T15:43:41.485+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:41.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:41 smithi166 bash[9446]: cluster 2020-09-08T15:43:39.688631+0000 mgr.x (mgr.24118) 489 : cluster [DBG] pgmap v655: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s wr, 758 op/s; 28663/229382 objects misplaced (12.496%); 5.7 MiB/s, 90 objects/s recovering 2020-09-08T15:43:41.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:41 smithi166 bash[9446]: cluster 2020-09-08T15:43:40.662110+0000 mon.a (mon.0) 1186 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:43:41.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:41 smithi166 bash[9446]: audit 2020-09-08T15:43:40.680093+0000 mon.a (mon.0) 1187 : audit [INF] from='client.? v1:172.21.15.104:0/908233471' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:43:41.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:41 smithi166 bash[9446]: cluster 2020-09-08T15:43:40.680142+0000 mon.a (mon.0) 1188 : cluster [DBG] osdmap e250: 12 total, 12 up, 12 in 2020-09-08T15:43:41.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:41 smithi166 bash[9446]: cluster 2020-09-08T15:43:40.786071+0000 osd.6 (osd.6) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:41.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:41 smithi166 bash[9446]: cluster 2020-09-08T15:43:40.786211+0000 osd.6 (osd.6) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:41.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:43:41 smithi055 bash[11956]: debug 2020-09-08T15:43:41.715+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:41.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:41 smithi055 bash[9296]: cluster 2020-09-08T15:43:39.688631+0000 mgr.x (mgr.24118) 489 : cluster [DBG] pgmap v655: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s wr, 758 op/s; 28663/229382 objects misplaced (12.496%); 5.7 MiB/s, 90 objects/s recovering 2020-09-08T15:43:41.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:41 smithi055 bash[9296]: cluster 2020-09-08T15:43:40.662110+0000 mon.a (mon.0) 1186 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:43:41.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:41 smithi055 bash[9296]: audit 2020-09-08T15:43:40.680093+0000 mon.a (mon.0) 1187 : audit [INF] from='client.? v1:172.21.15.104:0/908233471' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:43:41.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:41 smithi055 bash[9296]: cluster 2020-09-08T15:43:40.680142+0000 mon.a (mon.0) 1188 : cluster [DBG] osdmap e250: 12 total, 12 up, 12 in 2020-09-08T15:43:41.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:41 smithi055 bash[9296]: cluster 2020-09-08T15:43:40.786071+0000 osd.6 (osd.6) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:41.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:41 smithi055 bash[9296]: cluster 2020-09-08T15:43:40.786211+0000 osd.6 (osd.6) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:41.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:41 smithi104 bash[9203]: cluster 2020-09-08T15:43:39.688631+0000 mgr.x (mgr.24118) 489 : cluster [DBG] pgmap v655: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s wr, 758 op/s; 28663/229382 objects misplaced (12.496%); 5.7 MiB/s, 90 objects/s recovering 2020-09-08T15:43:41.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:41 smithi104 bash[9203]: cluster 2020-09-08T15:43:40.662110+0000 mon.a (mon.0) 1186 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:43:41.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:41 smithi104 bash[9203]: audit 2020-09-08T15:43:40.680093+0000 mon.a (mon.0) 1187 : audit [INF] from='client.? v1:172.21.15.104:0/908233471' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:43:41.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:41 smithi104 bash[9203]: cluster 2020-09-08T15:43:40.680142+0000 mon.a (mon.0) 1188 : cluster [DBG] osdmap e250: 12 total, 12 up, 12 in 2020-09-08T15:43:41.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:41 smithi104 bash[9203]: cluster 2020-09-08T15:43:40.786071+0000 osd.6 (osd.6) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:41.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:41 smithi104 bash[9203]: cluster 2020-09-08T15:43:40.786211+0000 osd.6 (osd.6) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:42.221 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:42.222 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 82 16 118386 118370 90.2105 44 0.00849685 0.0110758 2020-09-08T15:43:42.323 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:42.448 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:42.557 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:42.615 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:43:42 smithi166 bash[16593]: debug 2020-09-08T15:43:42.445+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:42.670 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:42.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:43:42 smithi104 bash[22305]: debug 2020-09-08T15:43:42.219+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:42.774 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:43.026 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:42 smithi104 bash[9203]: cluster 2020-09-08T15:43:41.683513+0000 mon.a (mon.0) 1189 : cluster [DBG] osdmap e251: 12 total, 12 up, 12 in 2020-09-08T15:43:43.027 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:42 smithi104 bash[9203]: cluster 2020-09-08T15:43:41.689101+0000 mgr.x (mgr.24118) 490 : cluster [DBG] pgmap v658: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s wr, 734 op/s; 28558/230778 objects misplaced (12.375%); 5.8 MiB/s, 93 objects/s recovering 2020-09-08T15:43:43.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:42 smithi166 bash[9446]: cluster 2020-09-08T15:43:41.683513+0000 mon.a (mon.0) 1189 : cluster [DBG] osdmap e251: 12 total, 12 up, 12 in 2020-09-08T15:43:43.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:42 smithi166 bash[9446]: cluster 2020-09-08T15:43:41.689101+0000 mgr.x (mgr.24118) 490 : cluster [DBG] pgmap v658: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s wr, 734 op/s; 28558/230778 objects misplaced (12.375%); 5.8 MiB/s, 93 objects/s recovering 2020-09-08T15:43:43.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:43:42 smithi166 bash[16593]: debug 2020-09-08T15:43:42.665+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:43.123 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:43.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:42 smithi055 bash[9296]: cluster 2020-09-08T15:43:41.683513+0000 mon.a (mon.0) 1189 : cluster [DBG] osdmap e251: 12 total, 12 up, 12 in 2020-09-08T15:43:43.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:42 smithi055 bash[9296]: cluster 2020-09-08T15:43:41.689101+0000 mgr.x (mgr.24118) 490 : cluster [DBG] pgmap v658: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s wr, 734 op/s; 28558/230778 objects misplaced (12.375%); 5.8 MiB/s, 93 objects/s recovering 2020-09-08T15:43:43.223 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 83 16 119109 119093 89.668 45.1875 0.00565411 0.0111475 2020-09-08T15:43:43.233 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:43:43.306 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:43:43.306 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T15:43:43.306 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2020-09-08T15:43:43.306 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2020-09-08T15:43:43.307 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:43:43.413 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:43:43 smithi104 bash[22305]: debug 2020-09-08T15:43:43.119+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:43.495 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:43:43.603 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:43.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:43:43 smithi104 bash[17588]: debug 2020-09-08T15:43:43.495+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:43.840 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:43.943 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:44.097 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:44.177 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:43:43 smithi104 bash[15273]: debug 2020-09-08T15:43:43.843+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:44.200 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:43:44.223 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 84 16 119766 119750 89.0894 41.0625 0.00550934 0.0112218 2020-09-08T15:43:44.300 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:43:44.368 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:43:44 smithi166 bash[18960]: debug 2020-09-08T15:43:44.089+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:44.403 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:44.515 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:44.618 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:43:44.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:43:44 smithi055 bash[19003]: debug 2020-09-08T15:43:44.299+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:44.747 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:43:44 smithi166 bash[18960]: debug 2020-09-08T15:43:44.513+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:45.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:44 smithi166 bash[9446]: cluster 2020-09-08T15:43:43.689730+0000 mgr.x (mgr.24118) 491 : cluster [DBG] pgmap v659: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s wr, 882 op/s; 28073/235774 objects misplaced (11.907%); 7.4 MiB/s, 118 objects/s recovering 2020-09-08T15:43:45.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:44 smithi166 bash[9446]: cluster 2020-09-08T15:43:44.018531+0000 osd.11 (osd.11) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:45.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:44 smithi166 bash[9446]: cluster 2020-09-08T15:43:44.018618+0000 osd.11 (osd.11) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:45.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:44 smithi055 bash[9296]: cluster 2020-09-08T15:43:43.689730+0000 mgr.x (mgr.24118) 491 : cluster [DBG] pgmap v659: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s wr, 882 op/s; 28073/235774 objects misplaced (11.907%); 7.4 MiB/s, 118 objects/s recovering 2020-09-08T15:43:45.311 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:44 smithi055 bash[9296]: cluster 2020-09-08T15:43:44.018531+0000 osd.11 (osd.11) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:45.312 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:44 smithi055 bash[9296]: cluster 2020-09-08T15:43:44.018618+0000 osd.11 (osd.11) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:45.313 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 85 16 120441 120425 88.5376 42.1875 0.0167445 0.0112862 2020-09-08T15:43:45.313 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:44 smithi104 bash[9203]: cluster 2020-09-08T15:43:43.689730+0000 mgr.x (mgr.24118) 491 : cluster [DBG] pgmap v659: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s wr, 882 op/s; 28073/235774 objects misplaced (11.907%); 7.4 MiB/s, 118 objects/s recovering 2020-09-08T15:43:45.314 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:44 smithi104 bash[9203]: cluster 2020-09-08T15:43:44.018531+0000 osd.11 (osd.11) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:45.314 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:44 smithi104 bash[9203]: cluster 2020-09-08T15:43:44.018618+0000 osd.11 (osd.11) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:45.328 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:43:45.431 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:43:45.482 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:43:45.488 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:43:45.490 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2020-09-08T15:43:45.536 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:43:45.577 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:43:45.633 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:43:45.673 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:43:45.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:43:45 smithi104 bash[17588]: debug 2020-09-08T15:43:45.679+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:45.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:43:45 smithi104 bash[19922]: debug 2020-09-08T15:43:45.319+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:45.682 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:43:45.715 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:43:45.730 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:43:45.744 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:43:45.793 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:45.800 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:43:45.803 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T15:43:45.804 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2020-09-08T15:43:45.818 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:43:45.858 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:43:45.896 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:43:45.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:45 smithi104 bash[9203]: cluster 2020-09-08T15:43:44.748774+0000 mon.a (mon.0) 1190 : cluster [DBG] osdmap e252: 12 total, 12 up, 12 in 2020-09-08T15:43:45.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:45 smithi104 bash[9203]: audit 2020-09-08T15:43:45.416824+0000 mon.a (mon.0) 1191 : audit [DBG] from='client.? v1:172.21.15.104:0/3616486980' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:43:45.936 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:43:45.977 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:43:45.991 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:43:45.992 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:43:45.992 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:43:45.992 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:43:45.992 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:43:45.993 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:43:45.993 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:43:45.993 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:43:45.994 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:43:45.994 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:43:45.994 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:43:45.994 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:43:46.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:45 smithi166 bash[9446]: cluster 2020-09-08T15:43:44.748774+0000 mon.a (mon.0) 1190 : cluster [DBG] osdmap e252: 12 total, 12 up, 12 in 2020-09-08T15:43:46.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:45 smithi166 bash[9446]: audit 2020-09-08T15:43:45.416824+0000 mon.a (mon.0) 1191 : audit [DBG] from='client.? v1:172.21.15.104:0/3616486980' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:43:46.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:45 smithi055 bash[9296]: cluster 2020-09-08T15:43:44.748774+0000 mon.a (mon.0) 1190 : cluster [DBG] osdmap e252: 12 total, 12 up, 12 in 2020-09-08T15:43:46.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:45 smithi055 bash[9296]: audit 2020-09-08T15:43:45.416824+0000 mon.a (mon.0) 1191 : audit [DBG] from='client.? v1:172.21.15.104:0/3616486980' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:43:46.223 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 86 16 121081 121065 87.9732 40 0.00586792 0.0113643 2020-09-08T15:43:47.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:46 smithi166 bash[9446]: cluster 2020-09-08T15:43:45.690232+0000 mgr.x (mgr.24118) 492 : cluster [DBG] pgmap v661: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s wr, 607 op/s; 28073/236680 objects misplaced (11.861%); 5.0 MiB/s, 79 objects/s recovering 2020-09-08T15:43:47.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:46 smithi166 bash[9446]: audit 2020-09-08T15:43:45.759420+0000 mgr.x (mgr.24118) 493 : audit [DBG] from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:43:47.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:46 smithi055 bash[9296]: cluster 2020-09-08T15:43:45.690232+0000 mgr.x (mgr.24118) 492 : cluster [DBG] pgmap v661: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s wr, 607 op/s; 28073/236680 objects misplaced (11.861%); 5.0 MiB/s, 79 objects/s recovering 2020-09-08T15:43:47.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:46 smithi055 bash[9296]: audit 2020-09-08T15:43:45.759420+0000 mgr.x (mgr.24118) 493 : audit [DBG] from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:43:47.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:46 smithi104 bash[9203]: cluster 2020-09-08T15:43:45.690232+0000 mgr.x (mgr.24118) 492 : cluster [DBG] pgmap v661: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s wr, 607 op/s; 28073/236680 objects misplaced (11.861%); 5.0 MiB/s, 79 objects/s recovering 2020-09-08T15:43:47.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:46 smithi104 bash[9203]: audit 2020-09-08T15:43:45.759420+0000 mgr.x (mgr.24118) 493 : audit [DBG] from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T15:43:47.223 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 87 16 121860 121844 87.5216 48.6875 0.00942878 0.0114225 2020-09-08T15:43:48.151 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:48.223 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 88 16 122625 122609 87.0703 47.8125 0.00836366 0.0114815 2020-09-08T15:43:48.255 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:43:48.397 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:43:48.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:43:48 smithi104 bash[22305]: debug 2020-09-08T15:43:48.151+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:48.501 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:48.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:43:48 smithi166 bash[11928]: debug 2020-09-08T15:43:48.397+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:48.777 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:48 smithi104 bash[9203]: cluster 2020-09-08T15:43:47.174822+0000 osd.4 (osd.4) 25 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:48.777 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:48 smithi104 bash[9203]: cluster 2020-09-08T15:43:47.174901+0000 osd.4 (osd.4) 26 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:48.778 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:48 smithi104 bash[9203]: cluster 2020-09-08T15:43:47.690805+0000 mgr.x (mgr.24118) 494 : cluster [DBG] pgmap v662: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s wr, 862 op/s; 27653/241478 objects misplaced (11.452%); 7.1 MiB/s, 113 objects/s recovering 2020-09-08T15:43:48.778 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:48 smithi104 bash[9203]: cluster 2020-09-08T15:43:47.783357+0000 mon.a (mon.0) 1192 : cluster [DBG] osdmap e253: 12 total, 12 up, 12 in 2020-09-08T15:43:49.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:48 smithi166 bash[9446]: cluster 2020-09-08T15:43:47.174822+0000 osd.4 (osd.4) 25 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:49.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:48 smithi166 bash[9446]: cluster 2020-09-08T15:43:47.174901+0000 osd.4 (osd.4) 26 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:49.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:48 smithi166 bash[9446]: cluster 2020-09-08T15:43:47.690805+0000 mgr.x (mgr.24118) 494 : cluster [DBG] pgmap v662: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s wr, 862 op/s; 27653/241478 objects misplaced (11.452%); 7.1 MiB/s, 113 objects/s recovering 2020-09-08T15:43:49.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:48 smithi166 bash[9446]: cluster 2020-09-08T15:43:47.783357+0000 mon.a (mon.0) 1192 : cluster [DBG] osdmap e253: 12 total, 12 up, 12 in 2020-09-08T15:43:49.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:48 smithi055 bash[9296]: cluster 2020-09-08T15:43:47.174822+0000 osd.4 (osd.4) 25 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:49.162 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:48 smithi055 bash[9296]: cluster 2020-09-08T15:43:47.174901+0000 osd.4 (osd.4) 26 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:49.162 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:48 smithi055 bash[9296]: cluster 2020-09-08T15:43:47.690805+0000 mgr.x (mgr.24118) 494 : cluster [DBG] pgmap v662: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s wr, 862 op/s; 27653/241478 objects misplaced (11.452%); 7.1 MiB/s, 113 objects/s recovering 2020-09-08T15:43:49.162 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:48 smithi055 bash[9296]: cluster 2020-09-08T15:43:47.783357+0000 mon.a (mon.0) 1192 : cluster [DBG] osdmap e253: 12 total, 12 up, 12 in 2020-09-08T15:43:49.224 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 89 15 123432 123417 86.6593 50.5 0.0240214 0.0115364 2020-09-08T15:43:49.280 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":661,"stamp":"2020-09-08T15:43:45.689955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7755333632,"num_objects":118340,"num_object_clones":0,"num_object_copies":236680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":28073,"num_objects_unfound":0,"num_objects_dirty":118340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":118340,"num_write_kb":7573571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2043,"num_bytes_recovered":133300224,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118343,"ondisk_log_size":118343,"up":34,"acting":34,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":12,"num_per_pool_osds":10,"num_per_pool_omap_osds":10,"kb":1124941824,"kb_used":16511836,"kb_used_data":15498364,"kb_used_omap":0,"kb_used_meta":1013376,"kb_avail":1108429988,"statfs":{"total":1151940427776,"available":1135032307712,"internally_reserved":0,"allocated":15870324736,"data_stored":15867266511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1037697024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,6,8,4,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":239140864,"num_objects":3649,"num_object_clones":0,"num_object_copies":7298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-590,"num_objects_unfound":0,"num_objects_dirty":3649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3649,"num_write_kb":233536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":479,"num_bytes_recovered":31391744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001646"},"pg_stats":[{"pgid":"3.f","version":"251'7503","reported_seq":"7546","reported_epoch":"251","state":"active+clean","last_fresh":"2020-09-08T15:43:43.612649+0000","last_change":"2020-09-08T15:42:06.978216+0000","last_active":"2020-09-08T15:43:43.612649+0000","last_peered":"2020-09-08T15:43:43.612649+0000","last_clean":"2020-09-08T15:43:43.612649+0000","last_became_active":"2020-09-08T15:42:06.977841+0000","last_became_peered":"2020-09-08T15:42:06.977841+0000","last_unstale":"2020-09-08T15:43:43.612649+0000","last_undegraded":"2020-09-08T15:43:43.612649+0000","last_fullsized":"2020-09-08T15:43:43.612649+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":7503,"ondisk_log_size":7503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":491716608,"num_objects":7503,"num_object_clones":0,"num_object_copies":15006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7503,"num_write_kb":480192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"251'7448","reported_seq":"7491","reported_epoch":"251","state":"active+clean","last_fresh":"2020-09-08T15:43:42.878400+0000","last_change":"2020-09-08T15:42:07.110906+0000","last_active":"2020-09-08T15:43:42.878400+0000","last_peered":"2020-09-08T15:43:42.878400+0000","last_clean":"2020-09-08T15:43:42.878400+0000","last_became_active":"2020-09-08T15:42:07.110465+0000","last_became_peered":"2020-09-08T15:42:07.110465+0000","last_unstale":"2020-09-08T15:43:42.878400+0000","last_undegraded":"2020-09-08T15:43:42.878400+0000","last_fullsized":"2020-09-08T15:43:42.878400+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":7448,"ondisk_log_size":7448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":488112128,"num_objects":7448,"num_object_clones":0,"num_object_copies":14896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7448,"num_write_kb":476672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4],"acting":[11,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.d","version":"251'7422","reported_seq":"7465","reported_epoch":"251","state":"active+clean","last_fresh":"2020-09-08T15:43:43.670548+0000","last_change":"2020-09-08T15:42:06.977801+0000","last_active":"2020-09-08T15:43:43.670548+0000","last_peered":"2020-09-08T15:43:43.670548+0000","last_clean":"2020-09-08T15:43:43.670548+0000","last_became_active":"2020-09-08T15:42:06.976987+0000","last_became_peered":"2020-09-08T15:42:06.976987+0000","last_unstale":"2020-09-08T15:43:43.670548+0000","last_undegraded":"2020-09-08T15:43:43.670548+0000","last_fullsized":"2020-09-08T15:43:43.670548+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":7422,"ondisk_log_size":7422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":486408192,"num_objects":7422,"num_object_clones":0,"num_object_copies":14844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7422,"num_write_kb":475008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"251'7479","reported_seq":"9876","reported_epoch":"251","state":"active+recovering+remapped","last_fresh":"2020-09-08T15:43:42.240160+0000","last_change":"2020-09-08T15:43:05.656882+0000","last_active":"2020-09-08T15:43:42.240160+0000","last_peered":"2020-09-08T15:43:42.240160+0000","last_clean":"2020-09-08T15:43:02.584638+0000","last_became_active":"2020-09-08T15:43:05.551667+0000","last_became_peered":"2020-09-08T15:43:05.551667+0000","last_unstale":"2020-09-08T15:43:42.240160+0000","last_undegraded":"2020-09-08T15:43:42.240160+0000","last_fullsized":"2020-09-08T15:43:42.240160+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":7479,"ondisk_log_size":7479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":490143744,"num_objects":7479,"num_object_clones":0,"num_object_copies":14958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":9591,"num_objects_unfound":0,"num_objects_dirty":7479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7479,"num_write_kb":478656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":783,"num_bytes_recovered":51314688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8],"acting":[3,8],"avail_no_missing":["3","5"],"object_location_counts":[{"shards":"3,5","objects":4795},{"shards":"3,5,8","objects":1},{"shards":"3,8","objects":2683}],"blocked_by":[],"up_primary":11,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"251'7323","reported_seq":"7367","reported_epoch":"251","state":"active+clean","last_fresh":"2020-09-08T15:43:42.186789+0000","last_change":"2020-09-08T15:42:06.973383+0000","last_active":"2020-09-08T15:43:42.186789+0000","last_peered":"2020-09-08T15:43:42.186789+0000","last_clean":"2020-09-08T15:43:42.186789+0000","last_became_active":"2020-09-08T15:42:06.972976+0000","last_became_peered":"2020-09-08T15:42:06.972976+0000","last_unstale":"2020-09-08T15:43:42.186789+0000","last_undegraded":"2020-09-08T15:43:42.186789+0000","last_fullsized":"2020-09-08T15:43:42.186789+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":7323,"ondisk_log_size":7323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":479920128,"num_objects":7323,"num_object_clones":0,"num_object_copies":14646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7323,"num_write_kb":468672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"250'7350","reported_seq":"7394","reported_epoch":"250","state":"active+clean","last_fresh":"2020-09-08T15:43:41.654354+0000","last_change":"2020-09-08T15:42:07.112085+0000","last_active":"2020-09-08T15:43:41.654354+0000","last_peered":"2020-09-08T15:43:41.654354+0000","last_clean":"2020-09-08T15:43:41.654354+0000","last_became_active":"2020-09-08T15:42:07.110953+0000","last_became_peered":"2020-09-08T15:42:07.110953+0000","last_unstale":"2020-09-08T15:43:41.654354+0000","last_undegraded":"2020-09-08T15:43:41.654354+0000","last_fullsized":"2020-09-08T15:43:41.654354+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":7350,"ondisk_log_size":7350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":481689600,"num_objects":7350,"num_object_clones":0,"num_object_copies":14700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7350,"num_write_kb":470400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"251'7521","reported_seq":"7564","reported_epoch":"251","state":"active+clean","last_fresh":"2020-09-08T15:43:41.911455+0000","last_change":"2020-09-08T15:42:06.975878+0000","last_active":"2020-09-08T15:43:41.911455+0000","last_peered":"2020-09-08T15:43:41.911455+0000","last_clean":"2020-09-08T15:43:41.911455+0000","last_became_active":"2020-09-08T15:42:06.975234+0000","last_became_peered":"2020-09-08T15:42:06.975234+0000","last_unstale":"2020-09-08T15:43:41.911455+0000","last_undegraded":"2020-09-08T15:43:41.911455+0000","last_fullsized":"2020-09-08T15:43:41.911455+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":7521,"ondisk_log_size":7521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":492896256,"num_objects":7521,"num_object_clones":0,"num_object_copies":15042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7521,"num_write_kb":481344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.8","version":"251'7366","reported_seq":"8340","reported_epoch":"251","state":"active+recovering","last_fresh":"2020-09-08T15:43:42.846563+0000","last_change":"2020-09-08T15:43:24.032684+0000","last_active":"2020-09-08T15:43:42.846563+0000","last_peered":"2020-09-08T15:43:42.846563+0000","last_clean":"2020-09-08T15:43:22.686424+0000","last_became_active":"2020-09-08T15:43:23.964446+0000","last_became_peered":"2020-09-08T15:43:23.964446+0000","last_unstale":"2020-09-08T15:43:42.846563+0000","last_undegraded":"2020-09-08T15:43:42.846563+0000","last_fullsized":"2020-09-08T15:43:42.846563+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":7366,"ondisk_log_size":7366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":482738176,"num_objects":7366,"num_object_clones":0,"num_object_copies":14732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6081,"num_objects_unfound":0,"num_objects_dirty":7366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7366,"num_write_kb":471424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":460,"num_bytes_recovered":30146560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1],"acting":[11,1],"avail_no_missing":["11","4"],"object_location_counts":[{"shards":"1,11","objects":1285},{"shards":"4,11","objects":6081}],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.7","version":"251'7348","reported_seq":"7392","reported_epoch":"251","state":"active+clean","last_fresh":"2020-09-08T15:43:42.223434+0000","last_change":"2020-09-08T15:42:06.977271+0000","last_active":"2020-09-08T15:43:42.223434+0000","last_peered":"2020-09-08T15:43:42.223434+0000","last_clean":"2020-09-08T15:43:42.223434+0000","last_became_active":"2020-09-08T15:42:06.976841+0000","last_became_peered":"2020-09-08T15:42:06.976841+0000","last_unstale":"2020-09-08T15:43:42.223434+0000","last_undegraded":"2020-09-08T15:43:42.223434+0000","last_fullsized":"2020-09-08T15:43:42.223434+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":7348,"ondisk_log_size":7348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":481558528,"num_objects":7348,"num_object_clones":0,"num_object_copies":14696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7348,"num_write_kb":470272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11],"acting":[3,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"250'7418","reported_seq":"8172","reported_epoch":"250","state":"active+recovering","last_fresh":"2020-09-08T15:43:40.835319+0000","last_change":"2020-09-08T15:43:24.073268+0000","last_active":"2020-09-08T15:43:40.835319+0000","last_peered":"2020-09-08T15:43:40.835319+0000","last_clean":"2020-09-08T15:43:22.700138+0000","last_became_active":"2020-09-08T15:43:23.966584+0000","last_became_peered":"2020-09-08T15:43:23.966584+0000","last_unstale":"2020-09-08T15:43:40.835319+0000","last_undegraded":"2020-09-08T15:43:40.835319+0000","last_fullsized":"2020-09-08T15:43:40.835319+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":7418,"ondisk_log_size":7418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":486146048,"num_objects":7418,"num_object_clones":0,"num_object_copies":14836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6318,"num_objects_unfound":0,"num_objects_dirty":7418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7418,"num_write_kb":474752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":350,"num_bytes_recovered":22937600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","4"],"object_location_counts":[{"shards":"0,1","objects":1100},{"shards":"0,4","objects":6318}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"251'7469","reported_seq":"7512","reported_epoch":"251","state":"active+clean","last_fresh":"2020-09-08T15:43:41.687081+0000","last_change":"2020-09-08T15:42:06.976571+0000","last_active":"2020-09-08T15:43:41.687081+0000","last_peered":"2020-09-08T15:43:41.687081+0000","last_clean":"2020-09-08T15:43:41.687081+0000","last_became_active":"2020-09-08T15:42:06.976051+0000","last_became_peered":"2020-09-08T15:42:06.976051+0000","last_unstale":"2020-09-08T15:43:41.687081+0000","last_undegraded":"2020-09-08T15:43:41.687081+0000","last_fullsized":"2020-09-08T15:43:41.687081+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":7469,"ondisk_log_size":7469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":489488384,"num_objects":7469,"num_object_clones":0,"num_object_copies":14938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7469,"num_write_kb":478016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.4","version":"251'7329","reported_seq":"8257","reported_epoch":"251","state":"active+recovering+remapped","last_fresh":"2020-09-08T15:43:41.747069+0000","last_change":"2020-09-08T15:43:25.089261+0000","last_active":"2020-09-08T15:43:41.747069+0000","last_peered":"2020-09-08T15:43:41.747069+0000","last_clean":"2020-09-08T15:43:22.677557+0000","last_became_active":"2020-09-08T15:43:25.026290+0000","last_became_peered":"2020-09-08T15:43:25.026290+0000","last_unstale":"2020-09-08T15:43:41.747069+0000","last_undegraded":"2020-09-08T15:43:41.747069+0000","last_fullsized":"2020-09-08T15:43:41.747069+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":7329,"ondisk_log_size":7329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":480313344,"num_objects":7329,"num_object_clones":0,"num_object_copies":14658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6083,"num_objects_unfound":0,"num_objects_dirty":7329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7329,"num_write_kb":469056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":441,"num_bytes_recovered":28901376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11],"acting":[5,11],"avail_no_missing":["5","11"],"object_location_counts":[{"shards":"5,11","objects":1246}],"blocked_by":[],"up_primary":1,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"250'7292","reported_seq":"7336","reported_epoch":"250","state":"active+clean","last_fresh":"2020-09-08T15:43:40.843307+0000","last_change":"2020-09-08T15:43:18.764889+0000","last_active":"2020-09-08T15:43:40.843307+0000","last_peered":"2020-09-08T15:43:40.843307+0000","last_clean":"2020-09-08T15:43:40.843307+0000","last_became_active":"2020-09-08T15:42:07.110780+0000","last_became_peered":"2020-09-08T15:42:07.110780+0000","last_unstale":"2020-09-08T15:43:40.843307+0000","last_undegraded":"2020-09-08T15:43:40.843307+0000","last_fullsized":"2020-09-08T15:43:40.843307+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"243'6295","last_scrub_stamp":"2020-09-08T15:43:18.764827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:43:18.764827+0000","log_size":7292,"ondisk_log_size":7292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":477888512,"num_objects":7292,"num_object_clones":0,"num_object_copies":14584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7292,"num_write_kb":466688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"250'7208","reported_seq":"7250","reported_epoch":"250","state":"active+clean","last_fresh":"2020-09-08T15:43:40.758543+0000","last_change":"2020-09-08T15:42:07.113446+0000","last_active":"2020-09-08T15:43:40.758543+0000","last_peered":"2020-09-08T15:43:40.758543+0000","last_clean":"2020-09-08T15:43:40.758543+0000","last_became_active":"2020-09-08T15:42:07.111969+0000","last_became_peered":"2020-09-08T15:42:07.111969+0000","last_unstale":"2020-09-08T15:43:40.758543+0000","last_undegraded":"2020-09-08T15:43:40.758543+0000","last_fullsized":"2020-09-08T15:43:40.758543+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":7208,"ondisk_log_size":7208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":472383488,"num_objects":7208,"num_object_clones":0,"num_object_copies":14416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7208,"num_write_kb":461312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"251'7539","reported_seq":"7582","reported_epoch":"251","state":"active+clean","last_fresh":"2020-09-08T15:43:42.877897+0000","last_change":"2020-09-08T15:42:07.110130+0000","last_active":"2020-09-08T15:43:42.877897+0000","last_peered":"2020-09-08T15:43:42.877897+0000","last_clean":"2020-09-08T15:43:42.877897+0000","last_became_active":"2020-09-08T15:42:07.109409+0000","last_became_peered":"2020-09-08T15:42:07.109409+0000","last_unstale":"2020-09-08T15:43:42.877897+0000","last_undegraded":"2020-09-08T15:43:42.877897+0000","last_fullsized":"2020-09-08T15:43:42.877897+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":7539,"ondisk_log_size":7539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":494075904,"num_objects":7539,"num_object_clones":0,"num_object_copies":15078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7539,"num_write_kb":482496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9],"acting":[11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2","version":"251'7322","reported_seq":"7366","reported_epoch":"251","state":"active+clean","last_fresh":"2020-09-08T15:43:42.192292+0000","last_change":"2020-09-08T15:42:07.110869+0000","last_active":"2020-09-08T15:43:42.192292+0000","last_peered":"2020-09-08T15:43:42.192292+0000","last_clean":"2020-09-08T15:43:42.192292+0000","last_became_active":"2020-09-08T15:42:07.110316+0000","last_became_peered":"2020-09-08T15:42:07.110316+0000","last_unstale":"2020-09-08T15:43:42.192292+0000","last_undegraded":"2020-09-08T15:43:42.192292+0000","last_fullsized":"2020-09-08T15:43:42.192292+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-09-08T15:42:05.952386+0000","last_clean_scrub_stamp":"2020-09-08T15:42:05.952386+0000","log_size":7322,"ondisk_log_size":7322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":479854592,"num_objects":7322,"num_object_clones":0,"num_object_copies":14644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7322,"num_write_kb":468608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"99'6","reported_seq":"269","reported_epoch":"250","state":"active+clean","last_fresh":"2020-09-08T15:43:40.684255+0000","last_change":"2020-09-08T15:42:48.963507+0000","last_active":"2020-09-08T15:43:40.684255+0000","last_peered":"2020-09-08T15:43:40.684255+0000","last_clean":"2020-09-08T15:43:40.684255+0000","last_became_active":"2020-09-08T15:34:43.031697+0000","last_became_peered":"2020-09-08T15:34:43.031697+0000","last_unstale":"2020-09-08T15:43:40.684255+0000","last_undegraded":"2020-09-08T15:43:40.684255+0000","last_fullsized":"2020-09-08T15:43:40.684255+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":2,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'6","last_scrub_stamp":"2020-09-08T15:42:48.963435+0000","last_deep_scrub":"99'6","last_deep_scrub_stamp":"2020-09-08T15:35:23.132465+0000","last_clean_scrub_stamp":"2020-09-08T15:42:48.963435+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":7755333632,"num_objects":118337,"num_object_clones":0,"num_object_copies":236674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":28073,"num_objects_unfound":0,"num_objects_dirty":118337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":118337,"num_write_kb":7573568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2034,"num_bytes_recovered":133300224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15911682048,"data_stored":15911682048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118337,"ondisk_log_size":118337,"up":32,"acting":32,"num_store_stats":10},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":2,"acting":2,"num_store_stats":7}],"osd_stats":[{"osd":11,"up_from":0,"seq":112,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3629556,"kb_used_data":3505580,"kb_used_omap":0,"kb_used_meta":123968,"kb_avail":90115596,"statfs":{"total":95995035648,"available":92278370304,"internally_reserved":0,"allocated":3589713920,"data_stored":3589456534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":126943232},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:43:02 2020","interfaces":[{"interface":"back","average":{"1min":1.9039999999999999,"5min":2.3759999999999999,"15min":2.052},"min":{"1min":0.745,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":4.2009999999999996,"5min":33.677999999999997,"15min":33.677999999999997},"last":0.96299999999999997},{"interface":"front","average":{"1min":3.415,"5min":3.1800000000000002,"15min":3.2570000000000001},"min":{"1min":1.5309999999999999,"5min":1.022,"15min":1.022},"max":{"1min":5.1390000000000002,"5min":34.058,"15min":34.058},"last":2.6379999999999999}]},{"osd":1,"last update":"Tue Sep 8 15:43:02 2020","interfaces":[{"interface":"back","average":{"1min":1.77,"5min":4.2460000000000004,"15min":2.77},"min":{"1min":0.70399999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":4.2389999999999999,"5min":163.51499999999999,"15min":163.51499999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":2.8319999999999999,"5min":4.8559999999999999,"15min":3.3380000000000001},"min":{"1min":1.0649999999999999,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":4.8099999999999996,"5min":166.94300000000001,"15min":166.94300000000001},"last":2.5249999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:42:00 2020","interfaces":[{"interface":"back","average":{"1min":2.9500000000000002,"5min":3.2679999999999998,"15min":2.4969999999999999},"min":{"1min":1.038,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":7.5460000000000003,"5min":36.637999999999998,"15min":36.637999999999998},"last":2.0329999999999999},{"interface":"front","average":{"1min":2.6589999999999998,"5min":3.3860000000000001,"15min":3.052},"min":{"1min":1.0069999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":7.5949999999999998,"5min":36.694000000000003,"15min":36.694000000000003},"last":2.113}]},{"osd":3,"last update":"Tue Sep 8 15:43:02 2020","interfaces":[{"interface":"back","average":{"1min":7.7850000000000001,"5min":4.9530000000000003,"15min":3.238},"min":{"1min":0.89200000000000002,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":150.54900000000001,"5min":150.54900000000001,"15min":150.54900000000001},"last":2.0550000000000002},{"interface":"front","average":{"1min":9.3059999999999992,"5min":5.702,"15min":3.9780000000000002},"min":{"1min":1.3420000000000001,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":158.34299999999999,"5min":158.34299999999999,"15min":158.34299999999999},"last":2.7469999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:43:02 2020","interfaces":[{"interface":"back","average":{"1min":3.5139999999999998,"5min":2.7109999999999999,"15min":3.2919999999999998},"min":{"1min":1.2,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":8.4939999999999998,"5min":8.4939999999999998,"15min":8.4939999999999998},"last":2.8530000000000002},{"interface":"front","average":{"1min":2.3290000000000002,"5min":2.6160000000000001,"15min":2.7400000000000002},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":5.383,"5min":7.7679999999999998,"15min":7.7679999999999998},"last":9.375}]},{"osd":5,"last update":"Tue Sep 8 15:43:02 2020","interfaces":[{"interface":"back","average":{"1min":2.6640000000000001,"5min":2.5739999999999998,"15min":3.1160000000000001},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":4.899,"5min":6.8499999999999996,"15min":6.8499999999999996},"last":2.9569999999999999},{"interface":"front","average":{"1min":3.6720000000000002,"5min":3.016,"15min":3.6389999999999998},"min":{"1min":1.337,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":5.8490000000000002,"5min":8.5289999999999999,"15min":8.5289999999999999},"last":1.0840000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:43:02 2020","interfaces":[{"interface":"back","average":{"1min":7.2930000000000001,"5min":3.5590000000000002,"15min":3.4500000000000002},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":110.82299999999999,"5min":110.82299999999999,"15min":110.82299999999999},"last":2.1619999999999999},{"interface":"front","average":{"1min":7.0650000000000004,"5min":3.8359999999999999,"15min":3.6819999999999999},"min":{"1min":1.125,"5min":1.125,"15min":1.125},"max":{"1min":105.249,"5min":105.249,"15min":105.249},"last":2.2690000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:43:02 2020","interfaces":[{"interface":"back","average":{"1min":2.738,"5min":3.0449999999999999,"15min":3.4910000000000001},"min":{"1min":1.155,"5min":1.131,"15min":1.131},"max":{"1min":5.8970000000000002,"5min":8.4610000000000003,"15min":8.4610000000000003},"last":9.1449999999999996},{"interface":"front","average":{"1min":3.698,"5min":3.1800000000000002,"15min":3.6389999999999998},"min":{"1min":1.2170000000000001,"5min":0.997,"15min":0.997},"max":{"1min":6.1189999999999998,"5min":9.1820000000000004,"15min":9.1820000000000004},"last":2.4940000000000002}]},{"osd":8,"last update":"Tue Sep 8 15:43:02 2020","interfaces":[{"interface":"back","average":{"1min":3.7120000000000002,"5min":3.673,"15min":3.9809999999999999},"min":{"1min":1.462,"5min":1.246,"15min":1.246},"max":{"1min":5.4390000000000001,"5min":13.018000000000001,"15min":13.018000000000001},"last":3.0640000000000001},{"interface":"front","average":{"1min":4.149,"5min":4.1200000000000001,"15min":3.9409999999999998},"min":{"1min":1.76,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":6.0330000000000004,"5min":13.592000000000001,"15min":13.592000000000001},"last":3.1880000000000002}]},{"osd":9,"last update":"Tue Sep 8 15:43:10 2020","interfaces":[{"interface":"back","average":{"1min":2.6869999999999998,"5min":2.6869999999999998,"15min":2.6869999999999998},"min":{"1min":1.022,"5min":1.022,"15min":1.022},"max":{"1min":6.2569999999999997,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":9},{"interface":"front","average":{"1min":3.6070000000000002,"5min":3.6070000000000002,"15min":3.6070000000000002},"min":{"1min":1.123,"5min":1.123,"15min":1.123},"max":{"1min":6.3049999999999997,"5min":6.3049999999999997,"15min":6.3049999999999997},"last":2.3889999999999998}]},{"osd":10,"last update":"Tue Sep 8 15:43:02 2020","interfaces":[{"interface":"back","average":{"1min":4.0460000000000003,"5min":3.3410000000000002,"15min":4.0830000000000002},"min":{"1min":1.7529999999999999,"5min":1.3260000000000001,"15min":1.3260000000000001},"max":{"1min":5.8200000000000003,"5min":9.7439999999999998,"15min":9.7439999999999998},"last":3.3010000000000002},{"interface":"front","average":{"1min":3.222,"5min":3.226,"15min":3.8940000000000001},"min":{"1min":1.381,"5min":1.085,"15min":1.085},"max":{"1min":5.766,"5min":8.6600000000000001,"15min":8.6600000000000001},"last":9.2639999999999993}]}]},{"osd":10,"up_from":0,"seq":119,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":57332,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":56640,"kb_avail":93687820,"statfs":{"total":95995035648,"available":95936327680,"internally_reserved":0,"allocated":700416,"data_stored":443030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":57999360},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7330000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:43:39 2020","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":2.609,"15min":2.4119999999999999},"min":{"1min":0.751,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":3.4239999999999999,"5min":64.471999999999994,"15min":64.471999999999994},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.645,"5min":2.9689999999999999,"15min":2.6749999999999998},"min":{"1min":0.77900000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":3.8260000000000001,"5min":64.707999999999998,"15min":64.707999999999998},"last":1.3560000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:43:39 2020","interfaces":[{"interface":"back","average":{"1min":1.9370000000000001,"5min":3.3069999999999999,"15min":2.617},"min":{"1min":1.0509999999999999,"5min":0.745,"15min":0.745},"max":{"1min":4.4100000000000001,"5min":95.197999999999993,"15min":95.197999999999993},"last":1.9079999999999999},{"interface":"front","average":{"1min":1.8169999999999999,"5min":3.1930000000000001,"15min":2.827},"min":{"1min":0.876,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":4.4470000000000001,"5min":94.778999999999996,"15min":94.778999999999996},"last":0.876}]},{"osd":3,"last update":"Tue Sep 8 15:43:39 2020","interfaces":[{"interface":"back","average":{"1min":20.431000000000001,"5min":8.282,"15min":4.1970000000000001},"min":{"1min":0.81399999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":353.03699999999998,"5min":353.03699999999998,"15min":353.03699999999998},"last":0.94099999999999995},{"interface":"front","average":{"1min":20.777999999999999,"5min":8.8010000000000002,"15min":4.8949999999999996},"min":{"1min":1.1319999999999999,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":354.35599999999999,"5min":354.35599999999999,"15min":354.35599999999999},"last":1.1319999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:43:39 2020","interfaces":[{"interface":"back","average":{"1min":2.238,"5min":2.464,"15min":3.0289999999999999},"min":{"1min":1.361,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":4.6349999999999998,"5min":38.237000000000002,"15min":38.237000000000002},"last":2.0499999999999998},{"interface":"front","average":{"1min":2.5139999999999998,"5min":2.6080000000000001,"15min":2.8610000000000002},"min":{"1min":1.331,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":4.3029999999999999,"5min":37.786000000000001,"15min":37.786000000000001},"last":1.331}]},{"osd":5,"last update":"Tue Sep 8 15:43:39 2020","interfaces":[{"interface":"back","average":{"1min":3.5009999999999999,"5min":2.9529999999999998,"15min":3.2959999999999998},"min":{"1min":1.2809999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":23.193999999999999,"5min":36.493000000000002,"15min":36.493000000000002},"last":2.73},{"interface":"front","average":{"1min":3.5219999999999998,"5min":3.0089999999999999,"15min":3.468},"min":{"1min":1.5529999999999999,"5min":1.145,"15min":1.145},"max":{"1min":23.614999999999998,"5min":38.957999999999998,"15min":38.957999999999998},"last":1.7430000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:43:39 2020","interfaces":[{"interface":"back","average":{"1min":2.645,"5min":2.7629999999999999,"15min":3.2250000000000001},"min":{"1min":1.216,"5min":0.998,"15min":0.998},"max":{"1min":6.3799999999999999,"5min":38.537999999999997,"15min":38.537999999999997},"last":1.216},{"interface":"front","average":{"1min":2.5329999999999999,"5min":2.7669999999999999,"15min":3.4409999999999998},"min":{"1min":1.2849999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"max":{"1min":5.8609999999999998,"5min":37.698999999999998,"15min":37.698999999999998},"last":1.591}]},{"osd":7,"last update":"Tue Sep 8 15:43:39 2020","interfaces":[{"interface":"back","average":{"1min":2.7240000000000002,"5min":3.097,"15min":3.5600000000000001},"min":{"1min":1.397,"5min":1.026,"15min":1.026},"max":{"1min":6.4939999999999998,"5min":39.100000000000001,"15min":39.100000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":2.681,"5min":3.1059999999999999,"15min":3.6920000000000002},"min":{"1min":1.409,"5min":1.325,"15min":1.3140000000000001},"max":{"1min":5.5709999999999997,"5min":38.131999999999998,"15min":38.131999999999998},"last":1.867}]},{"osd":8,"last update":"Tue Sep 8 15:43:39 2020","interfaces":[{"interface":"back","average":{"1min":2.8380000000000001,"5min":3.2709999999999999,"15min":3.786},"min":{"1min":1.506,"5min":1.181,"15min":1.181},"max":{"1min":5.5839999999999996,"5min":42.031999999999996,"15min":42.031999999999996},"last":2.8639999999999999},{"interface":"front","average":{"1min":3.0760000000000001,"5min":3.488,"15min":3.8039999999999998},"min":{"1min":1.452,"5min":1.113,"15min":1.113},"max":{"1min":6.7290000000000001,"5min":39.512999999999998,"15min":39.512999999999998},"last":2.1909999999999998}]},{"osd":9,"last update":"Tue Sep 8 15:43:39 2020","interfaces":[{"interface":"back","average":{"1min":3.173,"5min":3.5409999999999999,"15min":4.0460000000000003},"min":{"1min":1.581,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":7.71,"5min":42.603999999999999,"15min":42.603999999999999},"last":1.6910000000000001},{"interface":"front","average":{"1min":3.0710000000000002,"5min":3.3799999999999999,"15min":3.8849999999999998},"min":{"1min":1.47,"5min":1.3109999999999999,"15min":1.3109999999999999},"max":{"1min":5.3470000000000004,"5min":40.029000000000003,"15min":40.029000000000003},"last":2.976}]},{"osd":11,"last update":"Tue Sep 8 15:43:11 2020","interfaces":[{"interface":"back","average":{"1min":3.8380000000000001,"5min":3.831,"15min":4.1820000000000004},"min":{"1min":1.581,"5min":1.264,"15min":1.264},"max":{"1min":10.603999999999999,"5min":39.271000000000001,"15min":39.271000000000001},"last":1.8149999999999999},{"interface":"front","average":{"1min":3.8959999999999999,"5min":3.4420000000000002,"15min":4.0860000000000003},"min":{"1min":1.9390000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"max":{"1min":6.0229999999999997,"5min":42.329999999999998,"15min":42.329999999999998},"last":3.0859999999999999}]}]},{"osd":9,"up_from":0,"seq":122,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1034484,"kb_used_data":953580,"kb_used_omap":0,"kb_used_meta":80896,"kb_avail":92710668,"statfs":{"total":95995035648,"available":94935724032,"internally_reserved":0,"allocated":976465920,"data_stored":976208534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":82837504},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:43:10 2020","interfaces":[{"interface":"back","average":{"1min":2.4300000000000002,"5min":3.9910000000000001,"15min":2.6080000000000001},"min":{"1min":0.85599999999999998,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":21.074999999999999,"5min":180.84899999999999,"15min":180.84899999999999},"last":9.3040000000000003},{"interface":"front","average":{"1min":2.5720000000000001,"5min":3.9510000000000001,"15min":2.5990000000000002},"min":{"1min":0.92200000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":19.824000000000002,"5min":175.077,"15min":175.077},"last":7.6399999999999997}]},{"osd":1,"last update":"Tue Sep 8 15:43:10 2020","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.97,"15min":1.988},"min":{"1min":0.82299999999999995,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.8799999999999999,"5min":9.5519999999999996,"15min":9.5519999999999996},"last":12.824},{"interface":"front","average":{"1min":1.911,"5min":2.327,"15min":2.335},"min":{"1min":1.157,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":3.0510000000000002,"5min":9.4250000000000007,"15min":9.4250000000000007},"last":12.284000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:43:10 2020","interfaces":[{"interface":"back","average":{"1min":1.8220000000000001,"5min":2.2989999999999999,"15min":2.4159999999999999},"min":{"1min":1.075,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":3.1720000000000002,"5min":14.298,"15min":14.298},"last":1.276},{"interface":"front","average":{"1min":1.7030000000000001,"5min":2.2050000000000001,"15min":2.3069999999999999},"min":{"1min":1.0569999999999999,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.6800000000000002,"5min":9.4770000000000003,"15min":9.4770000000000003},"last":1.343}]},{"osd":3,"last update":"Tue Sep 8 15:43:16 2020","interfaces":[{"interface":"back","average":{"1min":3.093,"5min":2.7999999999999998,"15min":2.5390000000000001},"min":{"1min":1.3740000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":18.913,"5min":24.562000000000001,"15min":24.562000000000001},"last":10.087},{"interface":"front","average":{"1min":3.8079999999999998,"5min":2.923,"15min":2.786},"min":{"1min":1.591,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":27.045000000000002,"5min":27.045000000000002,"15min":27.045000000000002},"last":17.864000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:43:10 2020","interfaces":[{"interface":"back","average":{"1min":2.1930000000000001,"5min":2.5350000000000001,"15min":2.6890000000000001},"min":{"1min":1.0640000000000001,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":4.2999999999999998,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":1.6739999999999999},{"interface":"front","average":{"1min":2.4590000000000001,"5min":2.4550000000000001,"15min":2.907},"min":{"1min":1.4199999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":3.9180000000000001,"5min":10.201000000000001,"15min":10.201000000000001},"last":2.3319999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:43:10 2020","interfaces":[{"interface":"back","average":{"1min":2.0680000000000001,"5min":2.5430000000000001,"15min":2.9660000000000002},"min":{"1min":0.997,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":3.8340000000000001,"5min":11.097,"15min":11.097},"last":1.996},{"interface":"front","average":{"1min":3.1259999999999999,"5min":2.726,"15min":3.073},"min":{"1min":1.6120000000000001,"5min":1.0860000000000001,"15min":1.0589999999999999},"max":{"1min":9.8399999999999999,"5min":9.9280000000000008,"15min":9.9280000000000008},"last":2.194}]},{"osd":6,"last update":"Tue Sep 8 15:43:10 2020","interfaces":[{"interface":"back","average":{"1min":2.202,"5min":2.5510000000000002,"15min":3.0539999999999998},"min":{"1min":1.0089999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":4.2069999999999999,"5min":10.798,"15min":10.798},"last":1.7410000000000001},{"interface":"front","average":{"1min":2.4380000000000002,"5min":2.879,"15min":3.1560000000000001},"min":{"1min":1.3640000000000001,"5min":1.242,"15min":1.242},"max":{"1min":4.0629999999999997,"5min":11.766,"15min":11.766},"last":1.9470000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:43:10 2020","interfaces":[{"interface":"back","average":{"1min":8.1549999999999994,"5min":3.9350000000000001,"15min":3.6720000000000002},"min":{"1min":1.9590000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"max":{"1min":86.463999999999999,"5min":86.463999999999999,"15min":86.463999999999999},"last":2.044},{"interface":"front","average":{"1min":8.5809999999999995,"5min":4.2469999999999999,"15min":3.911},"min":{"1min":2.0859999999999999,"5min":1.593,"15min":1.593},"max":{"1min":86.656000000000006,"5min":86.656000000000006,"15min":86.656000000000006},"last":2.468}]},{"osd":8,"last update":"Tue Sep 8 15:43:10 2020","interfaces":[{"interface":"back","average":{"1min":2.6669999999999998,"5min":3.2650000000000001,"15min":3.298},"min":{"1min":1.9099999999999999,"5min":1.264,"15min":1.264},"max":{"1min":4.5259999999999998,"5min":11,"15min":11},"last":2.2389999999999999},{"interface":"front","average":{"1min":3.137,"5min":3.226,"15min":3.4039999999999999},"min":{"1min":1.8029999999999999,"5min":1.262,"15min":1.262},"max":{"1min":4.9939999999999998,"5min":10.977,"15min":10.977},"last":1.827}]},{"osd":10,"last update":"Tue Sep 8 15:43:32 2020","interfaces":[{"interface":"back","average":{"1min":2.4820000000000002,"5min":3.044,"15min":3.5910000000000002},"min":{"1min":1.6299999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"max":{"1min":4.2640000000000002,"5min":8.8200000000000003,"15min":8.8200000000000003},"last":2.427},{"interface":"front","average":{"1min":2.1379999999999999,"5min":2.7029999999999998,"15min":3.4300000000000002},"min":{"1min":1.2989999999999999,"5min":1.165,"15min":1.165},"max":{"1min":4.7779999999999996,"5min":11.625,"15min":11.625},"last":2.1440000000000001}]}]},{"osd":8,"up_from":0,"seq":128,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":747176,"kb_used_data":646560,"kb_used_omap":0,"kb_used_meta":100608,"kb_avail":92997976,"statfs":{"total":95995035648,"available":95229927424,"internally_reserved":0,"allocated":662077440,"data_stored":661824375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":103022592},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:42:45 2020","interfaces":[{"interface":"back","average":{"1min":2.2040000000000002,"5min":2.2130000000000001,"15min":2.1970000000000001},"min":{"1min":0.81200000000000006,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":4.141,"5min":11.154999999999999,"15min":11.154999999999999},"last":1.8129999999999999},{"interface":"front","average":{"1min":2.3679999999999999,"5min":2.4089999999999998,"15min":2.4169999999999998},"min":{"1min":0.93899999999999995,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":3.7320000000000002,"5min":13.773999999999999,"15min":13.773999999999999},"last":4.2839999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.492,"5min":2.0659999999999998,"15min":1.8839999999999999},"min":{"1min":1.4690000000000001,"5min":1.137,"15min":1.137},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":1.4350000000000001},{"interface":"front","average":{"1min":2.7200000000000002,"5min":2.427,"15min":2.347},"min":{"1min":1.351,"5min":1.046,"15min":1.046},"max":{"1min":4.444,"5min":4.444,"15min":4.444},"last":1.8600000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:42:45 2020","interfaces":[{"interface":"back","average":{"1min":1.7549999999999999,"5min":5.3520000000000003,"15min":3.1030000000000002},"min":{"1min":0.79000000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.6970000000000001,"5min":145.893,"15min":145.893},"last":1.661},{"interface":"front","average":{"1min":1.9390000000000001,"5min":5,"15min":3.3159999999999998},"min":{"1min":0.88600000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.8969999999999998,"5min":153.91900000000001,"15min":153.91900000000001},"last":1.0780000000000001}]},{"osd":3,"last update":"Tue Sep 8 15:42:45 2020","interfaces":[{"interface":"back","average":{"1min":2.4649999999999999,"5min":4.1029999999999998,"15min":3.2069999999999999},"min":{"1min":1.1559999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":3.758,"5min":125.111,"15min":125.111},"last":2.4009999999999998},{"interface":"front","average":{"1min":2.0419999999999998,"5min":3.7509999999999999,"15min":2.8620000000000001},"min":{"1min":0.88200000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":4.3029999999999999,"5min":126.124,"15min":126.124},"last":4.5940000000000003}]},{"osd":4,"last update":"Tue Sep 8 15:42:45 2020","interfaces":[{"interface":"back","average":{"1min":2.8719999999999999,"5min":2.8130000000000002,"15min":2.9119999999999999},"min":{"1min":1.3680000000000001,"5min":0.872,"15min":0.872},"max":{"1min":4.4980000000000002,"5min":11.404999999999999,"15min":11.404999999999999},"last":1.4259999999999999},{"interface":"front","average":{"1min":2.6139999999999999,"5min":2.5600000000000001,"15min":2.5920000000000001},"min":{"1min":1.1419999999999999,"5min":0.874,"15min":0.874},"max":{"1min":4.0910000000000002,"5min":13.999000000000001,"15min":13.999000000000001},"last":4.4660000000000002}]},{"osd":5,"last update":"Tue Sep 8 15:42:45 2020","interfaces":[{"interface":"back","average":{"1min":2.6030000000000002,"5min":2.9630000000000001,"15min":3.1440000000000001},"min":{"1min":1.0880000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":4.2290000000000001,"5min":9.6349999999999998,"15min":9.6349999999999998},"last":1.373},{"interface":"front","average":{"1min":3.0950000000000002,"5min":3.0590000000000002,"15min":3.3029999999999999},"min":{"1min":1.173,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":4.9489999999999998,"5min":11.885999999999999,"15min":11.885999999999999},"last":1.7090000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:42:45 2020","interfaces":[{"interface":"back","average":{"1min":2.992,"5min":3.133,"15min":3.2970000000000002},"min":{"1min":1.2310000000000001,"5min":1.1299999999999999,"15min":1.1299999999999999},"max":{"1min":4.5789999999999997,"5min":13.887,"15min":13.887},"last":5.1900000000000004},{"interface":"front","average":{"1min":3.2109999999999999,"5min":2.988,"15min":2.9729999999999999},"min":{"1min":1.4059999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":4.8780000000000001,"5min":12.502000000000001,"15min":12.502000000000001},"last":1.5820000000000001}]},{"osd":7,"last update":"Tue Sep 8 15:42:45 2020","interfaces":[{"interface":"back","average":{"1min":3.5550000000000002,"5min":3.4580000000000002,"15min":3.629},"min":{"1min":1.905,"5min":1.2310000000000001,"15min":1.2310000000000001},"max":{"1min":5.1989999999999998,"5min":11.661,"15min":11.661},"last":4.9340000000000002},{"interface":"front","average":{"1min":3.2050000000000001,"5min":3.2480000000000002,"15min":3.2730000000000001},"min":{"1min":1.6799999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"max":{"1min":4.7290000000000001,"5min":14.109999999999999,"15min":14.109999999999999},"last":2.0990000000000002}]},{"osd":9,"last update":"Tue Sep 8 15:43:09 2020","interfaces":[{"interface":"back","average":{"1min":2.2650000000000001,"5min":3.5270000000000001,"15min":3.5230000000000001},"min":{"1min":1.304,"5min":1.095,"15min":1.095},"max":{"1min":6.2249999999999996,"5min":13.506,"15min":13.506},"last":1.1950000000000001},{"interface":"front","average":{"1min":2.6579999999999999,"5min":3.3999999999999999,"15min":3.4940000000000002},"min":{"1min":1.706,"5min":1.004,"15min":1.004},"max":{"1min":4.5430000000000001,"5min":12.052,"15min":12.052},"last":1.968}]},{"osd":10,"last update":"Tue Sep 8 15:42:45 2020","interfaces":[{"interface":"back","average":{"1min":3.2949999999999999,"5min":3.1699999999999999,"15min":3.5299999999999998},"min":{"1min":1.7849999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":5.0449999999999999,"5min":13.656000000000001,"15min":13.656000000000001},"last":4.7270000000000003},{"interface":"front","average":{"1min":3.2789999999999999,"5min":3.5649999999999999,"15min":3.9079999999999999},"min":{"1min":1.3959999999999999,"5min":1.296,"15min":1.296},"max":{"1min":4.556,"5min":13.638,"15min":13.638},"last":1.514}]},{"osd":11,"last update":"Tue Sep 8 15:43:41 2020","interfaces":[{"interface":"back","average":{"1min":3.2389999999999999,"5min":3.4369999999999998,"15min":3.8479999999999999},"min":{"1min":1.173,"5min":1.173,"15min":1.173},"max":{"1min":7.0629999999999997,"5min":14.221,"15min":14.221},"last":5.0529999999999999},{"interface":"front","average":{"1min":2.774,"5min":3.5110000000000001,"15min":4.4909999999999997},"min":{"1min":1.4630000000000001,"5min":1.248,"15min":1.248},"max":{"1min":5.415,"5min":11.186999999999999,"15min":11.186999999999999},"last":1.7909999999999999}]}]},{"osd":7,"up_from":0,"seq":133,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1041396,"kb_used_data":952108,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":92703756,"statfs":{"total":95995035648,"available":94928646144,"internally_reserved":0,"allocated":974958592,"data_stored":974701226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:02 2020","interfaces":[{"interface":"back","average":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"min":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"max":{"1min":4.0190000000000001,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":2.2189999999999999},{"interface":"front","average":{"1min":2.6259999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"min":{"1min":1.542,"5min":1.542,"15min":1.542},"max":{"1min":4.5229999999999997,"5min":4.5229999999999997,"15min":4.5229999999999997},"last":2.3079999999999998}]},{"osd":1,"last update":"Tue Sep 8 15:43:26 2020","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":2.6930000000000001,"15min":2.4830000000000001},"min":{"1min":0.78600000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":4.0910000000000002,"5min":109.97,"15min":109.97},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.5369999999999999,"5min":2.8239999999999998,"15min":2.4910000000000001},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":3.5739999999999998,"5min":107.093,"15min":107.093},"last":1.3480000000000001}]},{"osd":2,"last update":"Tue Sep 8 15:43:26 2020","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":5.4269999999999996,"15min":3.4540000000000002},"min":{"1min":0.89000000000000001,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":3.379,"5min":324.77100000000002,"15min":324.77100000000002},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.7310000000000001,"5min":5.5730000000000004,"15min":3.7210000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":3.1019999999999999,"5min":321.48500000000001,"15min":321.48500000000001},"last":1.0649999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:43:26 2020","interfaces":[{"interface":"back","average":{"1min":9.5500000000000007,"5min":4.2439999999999998,"15min":3.3279999999999998},"min":{"1min":1.145,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":149.07499999999999,"5min":149.07499999999999,"15min":149.07499999999999},"last":1.982},{"interface":"front","average":{"1min":9.7840000000000007,"5min":5.0289999999999999,"15min":3.7730000000000001},"min":{"1min":0.87,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":152.61600000000001,"5min":152.61600000000001,"15min":152.61600000000001},"last":1.9239999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:43:26 2020","interfaces":[{"interface":"back","average":{"1min":2.1739999999999999,"5min":2.137,"15min":2.7029999999999998},"min":{"1min":0.995,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":4.2939999999999996,"5min":4.2939999999999996,"15min":8.5779999999999994},"last":1.5960000000000001},{"interface":"front","average":{"1min":2.5499999999999998,"5min":2.7010000000000001,"15min":3.153},"min":{"1min":1.212,"5min":1.127,"15min":1.127},"max":{"1min":4.6029999999999998,"5min":7.3570000000000002,"15min":7.3570000000000002},"last":2.0760000000000001}]},{"osd":5,"last update":"Tue Sep 8 15:43:26 2020","interfaces":[{"interface":"back","average":{"1min":2.4969999999999999,"5min":2.5430000000000001,"15min":3.2909999999999999},"min":{"1min":1.22,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":5.5369999999999999,"5min":5.5369999999999999,"15min":9.0800000000000001},"last":1.72},{"interface":"front","average":{"1min":2.641,"5min":2.6549999999999998,"15min":3.2130000000000001},"min":{"1min":1.153,"5min":1.153,"15min":1.153},"max":{"1min":5.4219999999999997,"5min":6.0229999999999997,"15min":6.0229999999999997},"last":1.649}]},{"osd":6,"last update":"Tue Sep 8 15:43:26 2020","interfaces":[{"interface":"back","average":{"1min":2.4390000000000001,"5min":2.7130000000000001,"15min":3.5030000000000001},"min":{"1min":1.27,"5min":1.27,"15min":1.27},"max":{"1min":4.5830000000000002,"5min":6.6360000000000001,"15min":8.8300000000000001},"last":2.2509999999999999},{"interface":"front","average":{"1min":2.8159999999999998,"5min":3.056,"15min":3.621},"min":{"1min":1.242,"5min":1.242,"15min":1.242},"max":{"1min":4.391,"5min":7.681,"15min":7.681},"last":2.1699999999999999}]},{"osd":8,"last update":"Tue Sep 8 15:42:47 2020","interfaces":[{"interface":"back","average":{"1min":3.3050000000000002,"5min":3.1080000000000001,"15min":3.8220000000000001},"min":{"1min":1.3919999999999999,"5min":1.3340000000000001,"15min":1.3340000000000001},"max":{"1min":5.2009999999999996,"5min":8.2260000000000009,"15min":8.2260000000000009},"last":1.5169999999999999},{"interface":"front","average":{"1min":3.4649999999999999,"5min":3.169,"15min":3.9340000000000002},"min":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"max":{"1min":5.7969999999999997,"5min":9.2119999999999997,"15min":9.2119999999999997},"last":1.843}]},{"osd":9,"last update":"Tue Sep 8 15:43:10 2020","interfaces":[{"interface":"back","average":{"1min":3.1040000000000001,"5min":3.3809999999999998,"15min":4.2720000000000002},"min":{"1min":1.361,"5min":1.361,"15min":1.361},"max":{"1min":5.5800000000000001,"5min":7.476,"15min":7.476},"last":2.4079999999999999},{"interface":"front","average":{"1min":3.0550000000000002,"5min":3.1600000000000001,"15min":3.9289999999999998},"min":{"1min":1.4990000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"max":{"1min":6.101,"5min":7.7460000000000004,"15min":7.7460000000000004},"last":1.8160000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:43:38 2020","interfaces":[{"interface":"back","average":{"1min":3.1579999999999999,"5min":3.3130000000000002,"15min":4.1420000000000003},"min":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"max":{"1min":5.407,"5min":7.3739999999999997,"15min":9.5009999999999994},"last":2.5710000000000002},{"interface":"front","average":{"1min":2.8420000000000001,"5min":3.1190000000000002,"15min":4.3499999999999996},"min":{"1min":1.6060000000000001,"5min":1.583,"15min":1.583},"max":{"1min":5.5650000000000004,"5min":7.5720000000000001,"15min":8.3650000000000002},"last":2.3679999999999999}]},{"osd":11,"last update":"Tue Sep 8 15:42:55 2020","interfaces":[{"interface":"back","average":{"1min":3.669,"5min":3.552,"15min":4.367},"min":{"1min":1.714,"5min":1.7010000000000001,"15min":1.7010000000000001},"max":{"1min":5.7690000000000001,"5min":9.1790000000000003,"15min":9.1790000000000003},"last":2.7149999999999999},{"interface":"front","average":{"1min":3.5649999999999999,"5min":3.7989999999999999,"15min":4.75},"min":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"max":{"1min":5.9450000000000003,"5min":7.7629999999999999,"15min":7.7629999999999999},"last":2.476}]}]},{"osd":6,"up_from":0,"seq":137,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":505320,"kb_used_data":469792,"kb_used_omap":0,"kb_used_meta":35520,"kb_avail":93239832,"statfs":{"total":95995035648,"available":95477587968,"internally_reserved":0,"allocated":481067008,"data_stored":480813963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36372480},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:42:58 2020","interfaces":[{"interface":"back","average":{"1min":1.698,"5min":1.51,"15min":1.8320000000000001},"min":{"1min":0.91600000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":3.1080000000000001,"5min":4.665,"15min":4.665},"last":1.052},{"interface":"front","average":{"1min":2.5390000000000001,"5min":1.8919999999999999,"15min":2.0049999999999999},"min":{"1min":1.0329999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":4.3239999999999998,"5min":4.7329999999999997,"15min":4.7329999999999997},"last":1.1950000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:42:58 2020","interfaces":[{"interface":"back","average":{"1min":1.978,"5min":3.8820000000000001,"15min":2.766},"min":{"1min":0.96999999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":4.7880000000000003,"5min":149.34200000000001,"15min":149.34200000000001},"last":1.375},{"interface":"front","average":{"1min":1.8799999999999999,"5min":3.8319999999999999,"15min":2.9910000000000001},"min":{"1min":0.95499999999999996,"5min":0.746,"15min":0.746},"max":{"1min":3.762,"5min":149.03899999999999,"15min":149.03899999999999},"last":1.71}]},{"osd":2,"last update":"Tue Sep 8 15:42:58 2020","interfaces":[{"interface":"back","average":{"1min":2.2090000000000001,"5min":8.4269999999999996,"15min":4.5259999999999998},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":4.0220000000000002,"5min":549.75999999999999,"15min":549.75999999999999},"last":2.2639999999999998},{"interface":"front","average":{"1min":2.391,"5min":8.5449999999999999,"15min":4.5650000000000004},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":4.4790000000000001,"5min":549.85900000000004,"15min":549.85900000000004},"last":1.5129999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:42:58 2020","interfaces":[{"interface":"back","average":{"1min":15.821,"5min":6.2039999999999997,"15min":3.8170000000000002},"min":{"1min":1.46,"5min":1.1499999999999999,"15min":1.1499999999999999},"max":{"1min":170.35400000000001,"5min":170.35400000000001,"15min":170.35400000000001},"last":2.3170000000000002},{"interface":"front","average":{"1min":11.353,"5min":5.2450000000000001,"15min":3.7959999999999998},"min":{"1min":1.4299999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"max":{"1min":170.31899999999999,"5min":170.31899999999999,"15min":170.31899999999999},"last":2.464}]},{"osd":4,"last update":"Tue Sep 8 15:43:02 2020","interfaces":[{"interface":"back","average":{"1min":7.8499999999999996,"5min":3.3940000000000001,"15min":3.0699999999999998},"min":{"1min":1.4970000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":89.180999999999997,"5min":89.180999999999997,"15min":89.180999999999997},"last":2.6709999999999998},{"interface":"front","average":{"1min":8.3490000000000002,"5min":3.8170000000000002,"15min":3.3239999999999998},"min":{"1min":1.4490000000000001,"5min":1.052,"15min":1.052},"max":{"1min":89.596000000000004,"5min":89.596000000000004,"15min":89.596000000000004},"last":1.665}]},{"osd":5,"last update":"Tue Sep 8 15:42:58 2020","interfaces":[{"interface":"back","average":{"1min":7.7130000000000001,"5min":3.73,"15min":3.4849999999999999},"min":{"1min":1.2909999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"max":{"1min":89.049000000000007,"5min":89.049000000000007,"15min":89.049000000000007},"last":2.7040000000000002},{"interface":"front","average":{"1min":8.0139999999999993,"5min":4.1040000000000001,"15min":3.629},"min":{"1min":1.4199999999999999,"5min":1.149,"15min":1.149},"max":{"1min":86.313999999999993,"5min":86.313999999999993,"15min":86.313999999999993},"last":3.1699999999999999}]},{"osd":7,"last update":"Tue Sep 8 15:43:20 2020","interfaces":[{"interface":"back","average":{"1min":6.343,"5min":3.5409999999999999,"15min":3.7559999999999998},"min":{"1min":1.246,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":85.143000000000001,"5min":85.143000000000001,"15min":85.143000000000001},"last":2.931},{"interface":"front","average":{"1min":2.9630000000000001,"5min":3.0750000000000002,"15min":3.7490000000000001},"min":{"1min":1.222,"5min":1.222,"15min":1.222},"max":{"1min":5.8849999999999998,"5min":5.9530000000000003,"15min":26.056000000000001},"last":2.867}]},{"osd":8,"last update":"Tue Sep 8 15:42:42 2020","interfaces":[{"interface":"back","average":{"1min":8.1890000000000001,"5min":3.8199999999999998,"15min":3.9609999999999999},"min":{"1min":2.048,"5min":1.3680000000000001,"15min":1.3680000000000001},"max":{"1min":84.995999999999995,"5min":84.995999999999995,"15min":84.995999999999995},"last":2.8250000000000002},{"interface":"front","average":{"1min":7.3659999999999997,"5min":3.7749999999999999,"15min":4.0739999999999998},"min":{"1min":1.506,"5min":1.1299999999999999,"15min":1.1299999999999999},"max":{"1min":85.762,"5min":85.762,"15min":85.762},"last":2.605}]},{"osd":9,"last update":"Tue Sep 8 15:43:10 2020","interfaces":[{"interface":"back","average":{"1min":7.7169999999999996,"5min":4.2290000000000001,"15min":4.4020000000000001},"min":{"1min":1.54,"5min":1.54,"15min":1.54},"max":{"1min":85.912999999999997,"5min":85.912999999999997,"15min":85.912999999999997},"last":3.0070000000000001},{"interface":"front","average":{"1min":7.532,"5min":3.839,"15min":4.2770000000000001},"min":{"1min":1.6200000000000001,"5min":1.224,"15min":1.224},"max":{"1min":84.652000000000001,"5min":84.652000000000001,"15min":84.652000000000001},"last":2.3500000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:43:32 2020","interfaces":[{"interface":"back","average":{"1min":2.9620000000000002,"5min":3.7200000000000002,"15min":4.3570000000000002},"min":{"1min":1.7789999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"max":{"1min":5.9400000000000004,"5min":84.159000000000006,"15min":84.159000000000006},"last":2.5},{"interface":"front","average":{"1min":3.581,"5min":4.1619999999999999,"15min":4.4560000000000004},"min":{"1min":2.1739999999999999,"5min":1.9299999999999999,"15min":1.915},"max":{"1min":6.1280000000000001,"5min":85.475999999999999,"15min":85.475999999999999},"last":3.323}]}]},{"osd":1,"up_from":0,"seq":161,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":322664,"kb_used_data":229216,"kb_used_omap":0,"kb_used_meta":93440,"kb_avail":93422488,"statfs":{"total":95995035648,"available":95664627712,"internally_reserved":0,"allocated":234717184,"data_stored":234464139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95682560},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:42:53 2020","interfaces":[{"interface":"back","average":{"1min":1.8240000000000001,"5min":3.2090000000000001,"15min":2.1920000000000002},"min":{"1min":0.77700000000000002,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":3.742,"5min":45.085000000000001,"15min":45.085000000000001},"last":5.609},{"interface":"front","average":{"1min":2.3639999999999999,"5min":3.5790000000000002,"15min":2.4660000000000002},"min":{"1min":0.997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":4.0069999999999997,"5min":45.765999999999998,"15min":45.765999999999998},"last":5.6349999999999998}]},{"osd":2,"last update":"Tue Sep 8 15:43:21 2020","interfaces":[{"interface":"back","average":{"1min":2.012,"5min":4.7969999999999997,"15min":2.9660000000000002},"min":{"1min":1.0740000000000001,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":3.2789999999999999,"5min":153.63499999999999,"15min":153.63499999999999},"last":6.7530000000000001},{"interface":"front","average":{"1min":2.1179999999999999,"5min":5.0250000000000004,"15min":3.1560000000000001},"min":{"1min":1.167,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":3.629,"5min":153.864,"15min":153.864},"last":6.8929999999999998}]},{"osd":3,"last update":"Tue Sep 8 15:41:44 2020","interfaces":[{"interface":"back","average":{"1min":2.7949999999999999,"5min":3.5960000000000001,"15min":2.7570000000000001},"min":{"1min":1.0209999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":5.6820000000000004,"5min":44.768999999999998,"15min":44.768999999999998},"last":2.9830000000000001},{"interface":"front","average":{"1min":2.8260000000000001,"5min":4.1040000000000001,"15min":2.996},"min":{"1min":0.90900000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":14.608000000000001,"5min":45.892000000000003,"15min":45.892000000000003},"last":2.8700000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:43:06 2020","interfaces":[{"interface":"back","average":{"1min":1.6459999999999999,"5min":3.1739999999999999,"15min":2.5270000000000001},"min":{"1min":0.84199999999999997,"5min":0.749,"15min":0.749},"max":{"1min":2.8260000000000001,"5min":44.970999999999997,"15min":44.970999999999997},"last":5.7130000000000001},{"interface":"front","average":{"1min":1.96,"5min":3.1059999999999999,"15min":2.968},"min":{"1min":0.84099999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":3.1709999999999998,"5min":51.289000000000001,"15min":51.289000000000001},"last":7.2789999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:43:31 2020","interfaces":[{"interface":"back","average":{"1min":2.3300000000000001,"5min":3.4980000000000002,"15min":3.1800000000000002},"min":{"1min":0.95199999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":4.8200000000000003,"5min":45.289000000000001,"15min":45.289000000000001},"last":7.0439999999999996},{"interface":"front","average":{"1min":1.976,"5min":3.3069999999999999,"15min":2.7719999999999998},"min":{"1min":0.99299999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":4.7809999999999997,"5min":51.499000000000002,"15min":51.499000000000002},"last":6.431}]},{"osd":6,"last update":"Tue Sep 8 15:42:53 2020","interfaces":[{"interface":"back","average":{"1min":2.5019999999999998,"5min":3.6779999999999999,"15min":3.6509999999999998},"min":{"1min":1.3129999999999999,"5min":1.1040000000000001,"15min":1.1040000000000001},"max":{"1min":3.625,"5min":44.932000000000002,"15min":44.932000000000002},"last":6.9800000000000004},{"interface":"front","average":{"1min":2.5409999999999999,"5min":3.9510000000000001,"15min":3.6989999999999998},"min":{"1min":1.115,"5min":1.115,"15min":1.115},"max":{"1min":4.0190000000000001,"5min":45.159999999999997,"15min":45.159999999999997},"last":7.1299999999999999}]},{"osd":7,"last update":"Tue Sep 8 15:43:21 2020","interfaces":[{"interface":"back","average":{"1min":9.7710000000000008,"5min":5.8410000000000002,"15min":4.2290000000000001},"min":{"1min":1.3280000000000001,"5min":1.2969999999999999,"15min":1.2969999999999999},"max":{"1min":130.70099999999999,"5min":130.70099999999999,"15min":130.70099999999999},"last":7.476},{"interface":"front","average":{"1min":9.6379999999999999,"5min":5.2590000000000003,"15min":4.327},"min":{"1min":1.296,"5min":1.085,"15min":1.085},"max":{"1min":137.398,"5min":137.398,"15min":137.398},"last":6.016}]},{"osd":8,"last update":"Tue Sep 8 15:42:47 2020","interfaces":[{"interface":"back","average":{"1min":3.1499999999999999,"5min":4.556,"15min":4.335},"min":{"1min":0.995,"5min":0.995,"15min":0.995},"max":{"1min":5.3079999999999998,"5min":45.640999999999998,"15min":45.640999999999998},"last":7.202},{"interface":"front","average":{"1min":2.9220000000000002,"5min":4.4610000000000003,"15min":4.1879999999999997},"min":{"1min":1.468,"5min":1.153,"15min":1.153},"max":{"1min":4.4260000000000002,"5min":45.235999999999997,"15min":45.235999999999997},"last":7.1459999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:43:16 2020","interfaces":[{"interface":"back","average":{"1min":2.6989999999999998,"5min":3.52,"15min":3.335},"min":{"1min":1.677,"5min":1.0860000000000001,"15min":1.0860000000000001},"max":{"1min":4.407,"5min":51.698,"15min":51.698},"last":7.5380000000000003},{"interface":"front","average":{"1min":2.4740000000000002,"5min":3.9950000000000001,"15min":4.1120000000000001},"min":{"1min":1.4330000000000001,"5min":1.111,"15min":1.111},"max":{"1min":8.4640000000000004,"5min":45.515000000000001,"15min":45.515000000000001},"last":6.8129999999999997}]},{"osd":10,"last update":"Tue Sep 8 15:43:31 2020","interfaces":[{"interface":"back","average":{"1min":3.113,"5min":4.2400000000000002,"15min":4.2240000000000002},"min":{"1min":1.506,"5min":1.4159999999999999,"15min":1.4159999999999999},"max":{"1min":8.7219999999999995,"5min":46.039999999999999,"15min":46.039999999999999},"last":6.5579999999999998},{"interface":"front","average":{"1min":2.4369999999999998,"5min":3.2789999999999999,"15min":3.4420000000000002},"min":{"1min":1.6859999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":3.931,"5min":51.892000000000003,"15min":51.892000000000003},"last":7.2939999999999996}]},{"osd":11,"last update":"Tue Sep 8 15:43:06 2020","interfaces":[{"interface":"back","average":{"1min":2.855,"5min":2.855,"15min":2.855},"min":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"max":{"1min":4.8209999999999997,"5min":4.8209999999999997,"15min":4.8209999999999997},"last":6.8799999999999999},{"interface":"front","average":{"1min":3.1640000000000001,"5min":3.1640000000000001,"15min":3.1640000000000001},"min":{"1min":1.849,"5min":1.849,"15min":1.849},"max":{"1min":4.8869999999999996,"5min":4.8869999999999996,"15min":4.8869999999999996},"last":7.4160000000000004}]}]},{"osd":0,"up_from":0,"seq":166,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1964648,"kb_used_data":1867808,"kb_used_omap":0,"kb_used_meta":96832,"kb_avail":91780504,"statfs":{"total":95995035648,"available":93983236096,"internally_reserved":0,"allocated":1912635392,"data_stored":1912382347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":99155968},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 8 15:42:52 2020","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":4.2430000000000003,"15min":2.5259999999999998},"min":{"1min":0.85999999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.331,"5min":74.831999999999994,"15min":74.831999999999994},"last":2.4630000000000001},{"interface":"front","average":{"1min":1.536,"5min":4.0709999999999997,"15min":2.464},"min":{"1min":0.73499999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.3519999999999999,"5min":72.611000000000004,"15min":72.611000000000004},"last":3.1339999999999999}]},{"osd":2,"last update":"Tue Sep 8 15:34:05 2020","interfaces":[{"interface":"back","average":{"1min":1.9950000000000001,"5min":1.621,"15min":1.5189999999999999},"min":{"1min":1.3999999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.8519999999999999,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":2.6589999999999998},{"interface":"front","average":{"1min":2.629,"5min":1.956,"15min":1.72},"min":{"1min":1.6419999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":2.4049999999999998}]},{"osd":3,"last update":"Tue Sep 8 15:42:41 2020","interfaces":[{"interface":"back","average":{"1min":2.0350000000000001,"5min":3.0720000000000001,"15min":2.9239999999999999},"min":{"1min":0.89900000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":3.1520000000000001,"5min":53.308,"15min":53.308},"last":2.0779999999999998},{"interface":"front","average":{"1min":2.5419999999999998,"5min":4.4409999999999998,"15min":3.8540000000000001},"min":{"1min":1.448,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":3.8069999999999999,"5min":73.075000000000003,"15min":73.075000000000003},"last":2.2050000000000001}]},{"osd":4,"last update":"Tue Sep 8 15:43:06 2020","interfaces":[{"interface":"back","average":{"1min":2.2749999999999999,"5min":2.948,"15min":2.8039999999999998},"min":{"1min":1.1279999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":3.3490000000000002,"5min":67.105999999999995,"15min":67.105999999999995},"last":0.94099999999999995},{"interface":"front","average":{"1min":2.331,"5min":3.8319999999999999,"15min":3.2770000000000001},"min":{"1min":1.268,"5min":0.995,"15min":0.995},"max":{"1min":3.2589999999999999,"5min":73.802999999999997,"15min":73.802999999999997},"last":1.4059999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:43:29 2020","interfaces":[{"interface":"back","average":{"1min":3.794,"5min":3.7240000000000002,"15min":3.379},"min":{"1min":1.069,"5min":0.83999999999999997,"15min":0.76900000000000002},"max":{"1min":24.497,"5min":66.950999999999993,"15min":66.950999999999993},"last":1.081},{"interface":"front","average":{"1min":3.2090000000000001,"5min":3.1560000000000001,"15min":3.016},"min":{"1min":0.83599999999999997,"5min":0.75,"15min":0.75},"max":{"1min":24.564,"5min":66.435000000000002,"15min":66.435000000000002},"last":3.6259999999999999}]},{"osd":6,"last update":"Tue Sep 8 15:42:52 2020","interfaces":[{"interface":"back","average":{"1min":2.0419999999999998,"5min":3.1400000000000001,"15min":3.2829999999999999},"min":{"1min":1.0800000000000001,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":3.6600000000000001,"5min":54.665999999999997,"15min":54.665999999999997},"last":3.7290000000000001},{"interface":"front","average":{"1min":2.4729999999999999,"5min":3.4009999999999998,"15min":3.6349999999999998},"min":{"1min":1.5309999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":4.2220000000000004,"5min":53.789000000000001,"15min":53.789000000000001},"last":4.202}]},{"osd":7,"last update":"Tue Sep 8 15:43:29 2020","interfaces":[{"interface":"back","average":{"1min":3.9449999999999998,"5min":4.6799999999999997,"15min":4.2060000000000004},"min":{"1min":1.448,"5min":1.1899999999999999,"15min":1.1899999999999999},"max":{"1min":24.797000000000001,"5min":71.167000000000002,"15min":71.167000000000002},"last":1.294},{"interface":"front","average":{"1min":3.6219999999999999,"5min":4.8159999999999998,"15min":4.3120000000000003},"min":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"max":{"1min":26.872,"5min":74.480000000000004,"15min":74.480000000000004},"last":3.823}]},{"osd":8,"last update":"Tue Sep 8 15:42:41 2020","interfaces":[{"interface":"back","average":{"1min":3.1699999999999999,"5min":3.9169999999999998,"15min":4.0110000000000001},"min":{"1min":1.8120000000000001,"5min":1.093,"15min":1.093},"max":{"1min":5.3049999999999997,"5min":67.528000000000006,"15min":67.528000000000006},"last":1.526},{"interface":"front","average":{"1min":2.7789999999999999,"5min":5.806,"15min":4.5940000000000003},"min":{"1min":1.3799999999999999,"5min":1.2070000000000001,"15min":1.2070000000000001},"max":{"1min":5.0170000000000003,"5min":131.958,"15min":131.958},"last":3.9159999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:43:06 2020","interfaces":[{"interface":"back","average":{"1min":2.9359999999999999,"5min":3.6899999999999999,"15min":4.133},"min":{"1min":2.0190000000000001,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":4.125,"5min":54.292999999999999,"15min":54.292999999999999},"last":1.8120000000000001},{"interface":"front","average":{"1min":2.4289999999999998,"5min":3.8069999999999999,"15min":3.9220000000000002},"min":{"1min":1.6879999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":3.6299999999999999,"5min":67.659999999999997,"15min":67.659999999999997},"last":4.0110000000000001}]},{"osd":10,"last update":"Tue Sep 8 15:43:29 2020","interfaces":[{"interface":"back","average":{"1min":3.387,"5min":3.4060000000000001,"15min":3.234},"min":{"1min":1.111,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":25.012,"5min":55.441000000000003,"15min":55.441000000000003},"last":1.173},{"interface":"front","average":{"1min":4.4139999999999997,"5min":5.0039999999999996,"15min":4.6100000000000003},"min":{"1min":1.744,"5min":1.1739999999999999,"15min":1.1739999999999999},"max":{"1min":25.295999999999999,"5min":73.444000000000003,"15min":73.444000000000003},"last":2.3380000000000001}]},{"osd":11,"last update":"Tue Sep 8 15:42:52 2020","interfaces":[{"interface":"back","average":{"1min":3.4239999999999999,"5min":5.2910000000000004,"15min":4.3799999999999999},"min":{"1min":1.8100000000000001,"5min":1.476,"15min":1.476},"max":{"1min":5.5060000000000002,"5min":74.147000000000006,"15min":74.147000000000006},"last":1.9390000000000001},{"interface":"front","average":{"1min":2.8719999999999999,"5min":3.778,"15min":4.5030000000000001},"min":{"1min":1.7729999999999999,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":4.6269999999999998,"5min":66.613,"15min":66.613},"last":4.3460000000000001}]}]},{"osd":2,"up_from":0,"seq":158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":66728,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":66048,"kb_avail":93678424,"statfs":{"total":95995035648,"available":95926706176,"internally_reserved":0,"allocated":688128,"data_stored":435083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":67633152},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:43:13 2020","interfaces":[{"interface":"back","average":{"1min":14.311,"5min":5.8559999999999999,"15min":3.177},"min":{"1min":0.79200000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":189.82599999999999,"5min":189.82599999999999,"15min":189.82599999999999},"last":1.012},{"interface":"front","average":{"1min":14.292999999999999,"5min":5.9660000000000002,"15min":3.2559999999999998},"min":{"1min":0.81899999999999995,"5min":0.624,"15min":0.624},"max":{"1min":185.15799999999999,"5min":185.15799999999999,"15min":185.15799999999999},"last":1.4670000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:43:13 2020","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":3.6469999999999998,"15min":2.4660000000000002},"min":{"1min":0.877,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.5699999999999998,"5min":78.728999999999999,"15min":78.728999999999999},"last":1.9710000000000001},{"interface":"front","average":{"1min":2.4729999999999999,"5min":3.448,"15min":2.7050000000000001},"min":{"1min":1.008,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":6.6390000000000002,"5min":80.103999999999999,"15min":80.103999999999999},"last":2.0070000000000001}]},{"osd":3,"last update":"Tue Sep 8 15:42:42 2020","interfaces":[{"interface":"back","average":{"1min":22.422000000000001,"5min":6.8550000000000004,"15min":3.9849999999999999},"min":{"1min":1.8680000000000001,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":276.34399999999999,"5min":276.34399999999999,"15min":276.34399999999999},"last":2.153},{"interface":"front","average":{"1min":21.667999999999999,"5min":6.6020000000000003,"15min":3.831},"min":{"1min":1.016,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":276.60000000000002,"5min":276.60000000000002,"15min":276.60000000000002},"last":2.9729999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:43:10 2020","interfaces":[{"interface":"back","average":{"1min":2.3029999999999999,"5min":2.5169999999999999,"15min":2.5830000000000002},"min":{"1min":0.90600000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":6.8970000000000002,"5min":7.6550000000000002,"15min":7.6550000000000002},"last":3.2869999999999999},{"interface":"front","average":{"1min":2.4940000000000002,"5min":3.177,"15min":2.8100000000000001},"min":{"1min":1.25,"5min":0.877,"15min":0.877},"max":{"1min":4.1189999999999998,"5min":64.284999999999997,"15min":64.284999999999997},"last":1.0429999999999999}]},{"osd":5,"last update":"Tue Sep 8 15:43:33 2020","interfaces":[{"interface":"back","average":{"1min":2.3210000000000002,"5min":3.3050000000000002,"15min":3.1829999999999998},"min":{"1min":0.94499999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":4.383,"5min":63.509,"15min":63.509},"last":1.6080000000000001},{"interface":"front","average":{"1min":2.677,"5min":2.7690000000000001,"15min":3.0310000000000001},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":5.8940000000000001,"5min":11.117000000000001,"15min":11.117000000000001},"last":2.302}]},{"osd":6,"last update":"Tue Sep 8 15:42:52 2020","interfaces":[{"interface":"back","average":{"1min":2.9990000000000001,"5min":3.71,"15min":3.4319999999999999},"min":{"1min":1.264,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":7.0199999999999996,"5min":63.631,"15min":63.631},"last":1.242},{"interface":"front","average":{"1min":2.8860000000000001,"5min":2.9929999999999999,"15min":3.444},"min":{"1min":1.429,"5min":1.105,"15min":1.105},"max":{"1min":4.7990000000000004,"5min":11.492000000000001,"15min":11.492000000000001},"last":2.6099999999999999}]},{"osd":7,"last update":"Tue Sep 8 15:43:28 2020","interfaces":[{"interface":"back","average":{"1min":2.839,"5min":3.1280000000000001,"15min":3.7509999999999999},"min":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"max":{"1min":8.0259999999999998,"5min":11.436,"15min":11.436},"last":3.7229999999999999},{"interface":"front","average":{"1min":3.1800000000000002,"5min":3.2970000000000002,"15min":3.536},"min":{"1min":1.544,"5min":1.016,"15min":1.006},"max":{"1min":6.0730000000000004,"5min":10.784000000000001,"15min":10.784000000000001},"last":2.9260000000000002}]},{"osd":8,"last update":"Tue Sep 8 15:43:36 2020","interfaces":[{"interface":"back","average":{"1min":2.8079999999999998,"5min":3.7490000000000001,"15min":3.6749999999999998},"min":{"1min":1.5289999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":5.8209999999999997,"5min":64.116,"15min":64.116},"last":1.3480000000000001},{"interface":"front","average":{"1min":3.1579999999999999,"5min":3.8159999999999998,"15min":3.9060000000000001},"min":{"1min":1.5680000000000001,"5min":1.218,"15min":1.218},"max":{"1min":6.25,"5min":63.299999999999997,"15min":63.299999999999997},"last":2.4590000000000001}]},{"osd":9,"last update":"Tue Sep 8 15:43:13 2020","interfaces":[{"interface":"back","average":{"1min":3.097,"5min":4.0289999999999999,"15min":4.0339999999999998},"min":{"1min":1.5940000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"max":{"1min":6.2889999999999997,"5min":64.004000000000005,"15min":64.004000000000005},"last":1.5049999999999999},{"interface":"front","average":{"1min":3.6120000000000001,"5min":3.5179999999999998,"15min":3.996},"min":{"1min":1.79,"5min":1.401,"15min":1.401},"max":{"1min":6.3780000000000001,"5min":11.869,"15min":11.869},"last":2.7709999999999999}]},{"osd":10,"last update":"Tue Sep 8 15:43:33 2020","interfaces":[{"interface":"back","average":{"1min":3.181,"5min":4.0170000000000003,"15min":4.0199999999999996},"min":{"1min":1.98,"5min":1.4339999999999999,"15min":1.4339999999999999},"max":{"1min":5.9800000000000004,"5min":63.743000000000002,"15min":63.743000000000002},"last":1.7470000000000001},{"interface":"front","average":{"1min":2.7400000000000002,"5min":3.0939999999999999,"15min":3.4910000000000001},"min":{"1min":1.5660000000000001,"5min":1.159,"15min":1.159},"max":{"1min":7.4779999999999998,"5min":11.067,"15min":11.067},"last":3.1389999999999998}]}]},{"osd":3,"up_from":0,"seq":151,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2469160,"kb_used_data":2358944,"kb_used_omap":0,"kb_used_meta":110208,"kb_avail":91275992,"statfs":{"total":95995035648,"available":93466615808,"internally_reserved":0,"allocated":2415558656,"data_stored":2415305611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":112852992},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:43:12 2020","interfaces":[{"interface":"back","average":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":8.0299999999999994,"5min":8.0299999999999994,"15min":8.0299999999999994},"last":3.7130000000000001},{"interface":"front","average":{"1min":5.6970000000000001,"5min":5.6970000000000001,"15min":5.6970000000000001},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":79.247,"5min":79.247,"15min":79.247},"last":1.978}]},{"osd":1,"last update":"Tue Sep 8 15:41:44 2020","interfaces":[{"interface":"back","average":{"1min":2.7229999999999999,"5min":3.028,"15min":2.2949999999999999},"min":{"1min":0.77400000000000002,"5min":0.628,"15min":0.628},"max":{"1min":5.4189999999999996,"5min":44.109999999999999,"15min":44.109999999999999},"last":2.4470000000000001},{"interface":"front","average":{"1min":3.1230000000000002,"5min":3.9710000000000001,"15min":2.6030000000000002},"min":{"1min":0.96499999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":7.6139999999999999,"5min":45.618000000000002,"15min":45.618000000000002},"last":3.871}]},{"osd":2,"last update":"Tue Sep 8 15:42:45 2020","interfaces":[{"interface":"back","average":{"1min":2.605,"5min":3.2799999999999998,"15min":2.585},"min":{"1min":0.93899999999999995,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":15.911,"5min":45.774000000000001,"15min":45.774000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":2.6499999999999999,"5min":2.8069999999999999,"15min":2.4590000000000001},"min":{"1min":1.0649999999999999,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":9.3599999999999994,"5min":13.948,"15min":13.948},"last":1.7589999999999999}]},{"osd":4,"last update":"Tue Sep 8 15:43:05 2020","interfaces":[{"interface":"back","average":{"1min":2.9940000000000002,"5min":2.5720000000000001,"15min":2.5960000000000001},"min":{"1min":1.0309999999999999,"5min":0.97599999999999998,"15min":0.94099999999999995},"max":{"1min":16.068000000000001,"5min":16.068000000000001,"15min":16.068000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":3.0209999999999999,"5min":2.2240000000000002,"15min":2.4700000000000002},"min":{"1min":0.78300000000000003,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":8.8970000000000002,"5min":11.298,"15min":11.298},"last":3.8330000000000002}]},{"osd":5,"last update":"Tue Sep 8 15:43:34 2020","interfaces":[{"interface":"back","average":{"1min":2.9700000000000002,"5min":2.7050000000000001,"15min":2.9500000000000002},"min":{"1min":1.4650000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":8.6289999999999996,"5min":27.007999999999999,"15min":45.442999999999998},"last":2.323},{"interface":"front","average":{"1min":3.415,"5min":3.2759999999999998,"15min":3.25},"min":{"1min":1.4339999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":16.998999999999999,"5min":23.565000000000001,"15min":23.565000000000001},"last":1.3320000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:42:57 2020","interfaces":[{"interface":"back","average":{"1min":7.2329999999999997,"5min":3.4430000000000001,"15min":3.3929999999999998},"min":{"1min":1.599,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":78.659000000000006,"5min":78.659000000000006,"15min":78.659000000000006},"last":2.2170000000000001},{"interface":"front","average":{"1min":7.4089999999999998,"5min":3.9009999999999998,"15min":3.6589999999999998},"min":{"1min":1.774,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":78.953999999999994,"5min":78.953999999999994,"15min":78.953999999999994},"last":2.851}]},{"osd":7,"last update":"Tue Sep 8 15:43:34 2020","interfaces":[{"interface":"back","average":{"1min":6.5899999999999999,"5min":4.3220000000000001,"15min":4.2300000000000004},"min":{"1min":1.5309999999999999,"5min":1.3220000000000001,"15min":1.1160000000000001},"max":{"1min":57.622,"5min":57.622,"15min":57.622},"last":1.6180000000000001},{"interface":"front","average":{"1min":6.2519999999999998,"5min":3.661,"15min":3.5800000000000001},"min":{"1min":1.2829999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":57.034999999999997,"5min":57.034999999999997,"15min":57.034999999999997},"last":3.9500000000000002}]},{"osd":8,"last update":"Tue Sep 8 15:43:41 2020","interfaces":[{"interface":"back","average":{"1min":2.847,"5min":3.3260000000000001,"15min":3.6989999999999998},"min":{"1min":1.4239999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":5.5839999999999996,"5min":21.329000000000001,"15min":46.917999999999999},"last":1.514},{"interface":"front","average":{"1min":3.2229999999999999,"5min":5.016,"15min":4.5810000000000004},"min":{"1min":1.8959999999999999,"5min":1.272,"15min":1.0780000000000001},"max":{"1min":9.093,"5min":78.808999999999997,"15min":78.808999999999997},"last":2.6150000000000002}]},{"osd":9,"last update":"Tue Sep 8 15:43:05 2020","interfaces":[{"interface":"back","average":{"1min":3.673,"5min":2.8069999999999999,"15min":3.5409999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":10.097,"5min":10.872999999999999,"15min":10.872999999999999},"last":4.3360000000000003},{"interface":"front","average":{"1min":7.6310000000000002,"5min":4.3449999999999998,"15min":4.2619999999999996},"min":{"1min":2.1309999999999998,"5min":1.23,"15min":1.23},"max":{"1min":79.096999999999994,"5min":79.096999999999994,"15min":79.096999999999994},"last":4.4480000000000004}]},{"osd":10,"last update":"Tue Sep 8 15:43:34 2020","interfaces":[{"interface":"back","average":{"1min":3.827,"5min":3.5099999999999998,"15min":4.1760000000000002},"min":{"1min":1.659,"5min":1.3520000000000001,"15min":1.3520000000000001},"max":{"1min":17.161000000000001,"5min":17.161000000000001,"15min":39.283000000000001},"last":1.881},{"interface":"front","average":{"1min":3.9590000000000001,"5min":3.032,"15min":3.5019999999999998},"min":{"1min":1.7050000000000001,"5min":1.081,"15min":1.081},"max":{"1min":9.7579999999999991,"5min":11.005000000000001,"15min":11.005000000000001},"last":4.0860000000000003}]},{"osd":11,"last update":"Tue Sep 8 15:42:57 2020","interfaces":[{"interface":"back","average":{"1min":4.2469999999999999,"5min":3.5739999999999998,"15min":4.3490000000000002},"min":{"1min":1.2170000000000001,"5min":1.196,"15min":1.196},"max":{"1min":9.452,"5min":13.882999999999999,"15min":13.882999999999999},"last":4.2229999999999999},{"interface":"front","average":{"1min":8.2300000000000004,"5min":5.5579999999999998,"15min":4.6020000000000003},"min":{"1min":1.9370000000000001,"5min":1.577,"15min":1.577},"max":{"1min":79.403999999999996,"5min":79.403999999999996,"15min":79.403999999999996},"last":2.7370000000000001}]}]},{"osd":4,"up_from":0,"seq":148,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2826292,"kb_used_data":2744620,"kb_used_omap":0,"kb_used_meta":81664,"kb_avail":90918860,"statfs":{"total":95995035648,"available":93100912640,"internally_reserved":0,"allocated":2810490880,"data_stored":2810233514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":83623936},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:43:07 2020","interfaces":[{"interface":"back","average":{"1min":2.1989999999999998,"5min":1.77,"15min":1.9119999999999999},"min":{"1min":0.94799999999999995,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":4.8559999999999999,"5min":5.6210000000000004,"15min":5.7130000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.7829999999999999,"5min":1.8580000000000001,"15min":1.97},"min":{"1min":0.90700000000000003,"5min":0.71299999999999997,"15min":0.70799999999999996},"max":{"1min":4.4249999999999998,"5min":10.471,"15min":10.471},"last":2.21}]},{"osd":1,"last update":"Tue Sep 8 15:43:07 2020","interfaces":[{"interface":"back","average":{"1min":1.9339999999999999,"5min":3.6339999999999999,"15min":2.544},"min":{"1min":1.002,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":3.2559999999999998,"5min":74.947999999999993,"15min":74.947999999999993},"last":1.0569999999999999},{"interface":"front","average":{"1min":2.1949999999999998,"5min":3.8109999999999999,"15min":2.8239999999999998},"min":{"1min":1.032,"5min":1,"15min":0.90700000000000003},"max":{"1min":3.9809999999999999,"5min":71.108999999999995,"15min":71.108999999999995},"last":1.147}]},{"osd":2,"last update":"Tue Sep 8 15:43:07 2020","interfaces":[{"interface":"back","average":{"1min":2.3959999999999999,"5min":3.7290000000000001,"15min":2.827},"min":{"1min":1.2729999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":4.2869999999999999,"5min":74.504000000000005,"15min":74.504000000000005},"last":2.0030000000000001},{"interface":"front","average":{"1min":2.5219999999999998,"5min":3.5009999999999999,"15min":2.859},"min":{"1min":1.179,"5min":1.014,"15min":1.014},"max":{"1min":5.5330000000000004,"5min":73.885000000000005,"15min":73.885000000000005},"last":1.373}]},{"osd":3,"last update":"Tue Sep 8 15:43:08 2020","interfaces":[{"interface":"back","average":{"1min":10.741,"5min":3.9470000000000001,"15min":3.141},"min":{"1min":1.5860000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":104.419,"5min":104.419,"15min":104.419},"last":1.2689999999999999},{"interface":"front","average":{"1min":10.894,"5min":4.4340000000000002,"15min":3.1890000000000001},"min":{"1min":1.4079999999999999,"5min":1.093,"15min":1.093},"max":{"1min":107.218,"5min":107.218,"15min":107.218},"last":2.3969999999999998}]},{"osd":5,"last update":"Tue Sep 8 15:43:37 2020","interfaces":[{"interface":"back","average":{"1min":7.7309999999999999,"5min":4.8890000000000002,"15min":3.6579999999999999},"min":{"1min":1.04,"5min":1.0289999999999999,"15min":1.014},"max":{"1min":110.30200000000001,"5min":125.51600000000001,"15min":125.51600000000001},"last":1.6599999999999999},{"interface":"front","average":{"1min":8.0180000000000007,"5min":4.9790000000000001,"15min":3.718},"min":{"1min":1.262,"5min":1.1559999999999999,"15min":1.129},"max":{"1min":109.89,"5min":125.816,"15min":125.816},"last":1.3500000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:42:54 2020","interfaces":[{"interface":"back","average":{"1min":3.4249999999999998,"5min":2.839,"15min":3.2959999999999998},"min":{"1min":1.575,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":5.343,"5min":6.6420000000000003,"15min":6.8650000000000002},"last":1.131},{"interface":"front","average":{"1min":3.4420000000000002,"5min":2.7709999999999999,"15min":3.2759999999999998},"min":{"1min":1.4470000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"max":{"1min":5.6369999999999996,"5min":6.8849999999999998,"15min":6.8849999999999998},"last":3.0329999999999999}]},{"osd":7,"last update":"Tue Sep 8 15:43:13 2020","interfaces":[{"interface":"back","average":{"1min":3.2360000000000002,"5min":3.0830000000000002,"15min":3.645},"min":{"1min":1.631,"5min":1.3460000000000001,"15min":1.3460000000000001},"max":{"1min":6.9359999999999999,"5min":6.9359999999999999,"15min":7.4699999999999998},"last":1.427},{"interface":"front","average":{"1min":3.4209999999999998,"5min":3.3279999999999998,"15min":3.7919999999999998},"min":{"1min":1.8089999999999999,"5min":1.546,"15min":1.546},"max":{"1min":5.8940000000000001,"5min":6.8440000000000003,"15min":7.1459999999999999},"last":1.504}]},{"osd":8,"last update":"Tue Sep 8 15:43:44 2020","interfaces":[{"interface":"back","average":{"1min":3.73,"5min":4.8449999999999998,"15min":4.3449999999999998},"min":{"1min":1.663,"5min":1.276,"15min":1.276},"max":{"1min":13.656000000000001,"5min":109.608,"15min":109.608},"last":2.8479999999999999},{"interface":"front","average":{"1min":3.8730000000000002,"5min":4.6870000000000003,"15min":4.5640000000000001},"min":{"1min":1.8260000000000001,"5min":1.6080000000000001,"15min":1.49},"max":{"1min":13.789999999999999,"5min":116.52500000000001,"15min":116.52500000000001},"last":3.1899999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:43:13 2020","interfaces":[{"interface":"back","average":{"1min":3.3599999999999999,"5min":4.8929999999999998,"15min":4.5609999999999999},"min":{"1min":1.974,"5min":1.546,"15min":1.546},"max":{"1min":5.1680000000000001,"5min":118.312,"15min":118.312},"last":1.635},{"interface":"front","average":{"1min":3.3980000000000001,"5min":4.8550000000000004,"15min":4.5339999999999998},"min":{"1min":1.7909999999999999,"5min":1.4650000000000001,"15min":1.4650000000000001},"max":{"1min":6.2210000000000001,"5min":116.871,"15min":116.871},"last":2.6030000000000002}]},{"osd":10,"last update":"Tue Sep 8 15:43:37 2020","interfaces":[{"interface":"back","average":{"1min":3.3839999999999999,"5min":3.0369999999999999,"15min":4.1669999999999998},"min":{"1min":1.337,"5min":1.337,"15min":1.337},"max":{"1min":7.7300000000000004,"5min":7.7300000000000004,"15min":7.9279999999999999},"last":1.9419999999999999},{"interface":"front","average":{"1min":4.0090000000000003,"5min":3.4950000000000001,"15min":4.3540000000000001},"min":{"1min":2.1150000000000002,"5min":1.702,"15min":1.702},"max":{"1min":13.731999999999999,"5min":13.731999999999999,"15min":13.731999999999999},"last":1.8}]}]},{"osd":5,"up_from":0,"seq":142,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1847080,"kb_used_data":1768800,"kb_used_omap":0,"kb_used_meta":78272,"kb_avail":91898072,"statfs":{"total":95995035648,"available":94103625728,"internally_reserved":0,"allocated":1811251200,"data_stored":1810998155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":80150528},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:42:35 2020","interfaces":[{"interface":"back","average":{"1min":2.0920000000000001,"5min":1.629,"15min":1.871},"min":{"1min":0.77500000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":5.4400000000000004,"5min":5.4400000000000004,"15min":5.4400000000000004},"last":0.84399999999999997},{"interface":"front","average":{"1min":2.5139999999999998,"5min":2.246,"15min":2.1469999999999998},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":4.9320000000000004,"5min":7.1340000000000003,"15min":7.1340000000000003},"last":2.258}]},{"osd":1,"last update":"Tue Sep 8 15:43:38 2020","interfaces":[{"interface":"back","average":{"1min":7.0880000000000001,"5min":4.8710000000000004,"15min":3.2909999999999999},"min":{"1min":1.054,"5min":0.91300000000000003,"15min":0.88700000000000001},"max":{"1min":82.390000000000001,"5min":128.488,"15min":128.488},"last":1.718},{"interface":"front","average":{"1min":6.508,"5min":5.0209999999999999,"15min":3.2530000000000001},"min":{"1min":0.80900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":74.873999999999995,"5min":129.63800000000001,"15min":129.63800000000001},"last":1.766}]},{"osd":2,"last update":"Tue Sep 8 15:43:38 2020","interfaces":[{"interface":"back","average":{"1min":6.1230000000000002,"5min":4.3849999999999998,"15min":3.0710000000000002},"min":{"1min":0.90000000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":74.340999999999994,"5min":127.498,"15min":127.498},"last":1.806},{"interface":"front","average":{"1min":7.8230000000000004,"5min":4.9690000000000003,"15min":3.6880000000000002},"min":{"1min":1.5149999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":82.730000000000004,"5min":126.673,"15min":126.673},"last":3.8679999999999999}]},{"osd":3,"last update":"Tue Sep 8 15:43:38 2020","interfaces":[{"interface":"back","average":{"1min":8.4120000000000008,"5min":4.3550000000000004,"15min":3.5329999999999999},"min":{"1min":1.712,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":82.040999999999997,"5min":82.040999999999997,"15min":82.040999999999997},"last":4.1130000000000004},{"interface":"front","average":{"1min":7.3959999999999999,"5min":3.6560000000000001,"15min":3.2050000000000001},"min":{"1min":1.4610000000000001,"5min":0.875,"15min":0.875},"max":{"1min":75.313999999999993,"5min":75.313999999999993,"15min":75.313999999999993},"last":2.0419999999999998}]},{"osd":4,"last update":"Tue Sep 8 15:43:38 2020","interfaces":[{"interface":"back","average":{"1min":7.6109999999999998,"5min":3.3140000000000001,"15min":2.7530000000000001},"min":{"1min":0.96399999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":80.195999999999998,"5min":80.195999999999998,"15min":80.195999999999998},"last":5.9109999999999996},{"interface":"front","average":{"1min":7.6340000000000003,"5min":3.2690000000000001,"15min":3.0539999999999998},"min":{"1min":1.276,"5min":1.127,"15min":1.127},"max":{"1min":79.810000000000002,"5min":79.810000000000002,"15min":79.810000000000002},"last":4.9470000000000001}]},{"osd":6,"last update":"Tue Sep 8 15:43:06 2020","interfaces":[{"interface":"back","average":{"1min":16.760999999999999,"5min":5.8010000000000002,"15min":4.4240000000000004},"min":{"1min":1.3540000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"max":{"1min":220.636,"5min":220.636,"15min":220.636},"last":4.7549999999999999},{"interface":"front","average":{"1min":17.245000000000001,"5min":6.2809999999999997,"15min":4.4089999999999998},"min":{"1min":1.8759999999999999,"5min":1.3580000000000001,"15min":1.3580000000000001},"max":{"1min":221.00700000000001,"5min":221.00700000000001,"15min":221.00700000000001},"last":5.2199999999999998}]},{"osd":7,"last update":"Tue Sep 8 15:43:24 2020","interfaces":[{"interface":"back","average":{"1min":8.2469999999999999,"5min":4.2060000000000004,"15min":3.887},"min":{"1min":1.7290000000000001,"5min":1.26,"15min":1.26},"max":{"1min":83.299999999999997,"5min":83.299999999999997,"15min":83.299999999999997},"last":5.6760000000000002},{"interface":"front","average":{"1min":16.658000000000001,"5min":6.3869999999999996,"15min":4.9059999999999997},"min":{"1min":2.2170000000000001,"5min":1.532,"15min":1.532},"max":{"1min":220.34399999999999,"5min":220.34399999999999,"15min":220.34399999999999},"last":7.0099999999999998}]},{"osd":8,"last update":"Tue Sep 8 15:42:44 2020","interfaces":[{"interface":"back","average":{"1min":8.7479999999999993,"5min":4.1790000000000003,"15min":3.8300000000000001},"min":{"1min":2.4060000000000001,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":83.019000000000005,"5min":83.019000000000005,"15min":83.019000000000005},"last":6.0419999999999998},{"interface":"front","average":{"1min":7.1449999999999996,"5min":3.5739999999999998,"15min":3.8940000000000001},"min":{"1min":1.708,"5min":1.1479999999999999,"15min":1.1479999999999999},"max":{"1min":75.641999999999996,"5min":75.641999999999996,"15min":75.641999999999996},"last":2.3359999999999999}]},{"osd":9,"last update":"Tue Sep 8 15:43:17 2020","interfaces":[{"interface":"back","average":{"1min":7.4219999999999997,"5min":3.8639999999999999,"15min":4.3120000000000003},"min":{"1min":1.4079999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"max":{"1min":76.382999999999996,"5min":76.382999999999996,"15min":76.382999999999996},"last":2.52},{"interface":"front","average":{"1min":8.8629999999999995,"5min":4.7690000000000001,"15min":4.4690000000000003},"min":{"1min":2.5670000000000002,"5min":1.4770000000000001,"15min":1.4770000000000001},"max":{"1min":83.159999999999997,"5min":83.159999999999997,"15min":83.159999999999997},"last":6.2949999999999999}]},{"osd":10,"last update":"Tue Sep 8 15:43:38 2020","interfaces":[{"interface":"back","average":{"1min":6.9320000000000004,"5min":3.4020000000000001,"15min":3.8889999999999998},"min":{"1min":1.488,"5min":1.431,"15min":1.431},"max":{"1min":77.087999999999994,"5min":77.087999999999994,"15min":77.087999999999994},"last":2.6890000000000001},{"interface":"front","average":{"1min":18.085000000000001,"5min":6.7009999999999996,"15min":5.4530000000000003},"min":{"1min":2.8210000000000002,"5min":1.8660000000000001,"15min":1.8660000000000001},"max":{"1min":219.92599999999999,"5min":219.92599999999999,"15min":219.92599999999999},"last":6.5469999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4660000000000002}]}]}]}} 2020-09-08T15:43:49.287 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-08-0934/qa/tasks/ceph_manager.py", line 118, in wrapper return func(self) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-08-0934/qa/tasks/ceph_manager.py", line 1204, in _do_thrash self.choose_action()() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-08-0934/qa/tasks/ceph_manager.py", line 782, in fix_pgp_num if self.ceph_manager.set_pool_pgpnum(pool, force): File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-08-0934/qa/tasks/ceph_manager.py", line 2078, in set_pool_pgpnum self.set_pool_property(pool_name, 'pgp_num', self.pools[pool_name]) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-08-0934/qa/tasks/ceph_manager.py", line 1997, in set_pool_property str(val)) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-08-0934/qa/tasks/ceph_manager.py", line 1365, in raw_cluster_cmd_result check_status=False).existatus File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-08-0934/qa/tasks/ceph_manager.py", line 51, in shell ] + args, TypeError: can only concatenate list (not "tuple") to list 2020-09-08T15:43:49.288 ERROR:tasks.thrashosds.thrasher:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-08-0934/qa/tasks/ceph_manager.py", line 1072, in do_thrash self._do_thrash() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-08-0934/qa/tasks/ceph_manager.py", line 118, in wrapper return func(self) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-08-0934/qa/tasks/ceph_manager.py", line 1204, in _do_thrash self.choose_action()() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-08-0934/qa/tasks/ceph_manager.py", line 782, in fix_pgp_num if self.ceph_manager.set_pool_pgpnum(pool, force): File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-08-0934/qa/tasks/ceph_manager.py", line 2078, in set_pool_pgpnum self.set_pool_property(pool_name, 'pgp_num', self.pools[pool_name]) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-08-0934/qa/tasks/ceph_manager.py", line 1997, in set_pool_property str(val)) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-08-0934/qa/tasks/ceph_manager.py", line 1365, in raw_cluster_cmd_result check_status=False).existatus File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-09-08-0934/qa/tasks/ceph_manager.py", line 51, in shell ] + args, TypeError: can only concatenate list (not "tuple") to list 2020-09-08T15:43:49.299 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:49.412 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:43:49.609 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:43:49 smithi104 bash[15273]: debug 2020-09-08T15:43:49.299+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:49.610 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:43:49.722 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:43:49.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:43:49 smithi104 bash[17588]: debug 2020-09-08T15:43:49.611+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:50.223 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 90 16 124158 124142 86.1999 45.3125 0.0132763 0.0115969 2020-09-08T15:43:51.224 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Total time run: 90.0544 2020-09-08T15:43:51.224 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Total writes made: 124158 2020-09-08T15:43:51.224 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Write size: 65536 2020-09-08T15:43:51.225 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Object size: 65536 2020-09-08T15:43:51.225 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Bandwidth (MB/sec): 86.1687 2020-09-08T15:43:51.225 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev Bandwidth: 51.6608 2020-09-08T15:43:51.225 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max bandwidth (MB/sec): 248.812 2020-09-08T15:43:51.226 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min bandwidth (MB/sec): 0 2020-09-08T15:43:51.226 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Average IOPS: 1378 2020-09-08T15:43:51.226 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev IOPS: 826 2020-09-08T15:43:51.226 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max IOPS: 3981 2020-09-08T15:43:51.226 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min IOPS: 0 2020-09-08T15:43:51.227 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Average Latency(s): 0.0116039 2020-09-08T15:43:51.227 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev Latency(s): 0.0425272 2020-09-08T15:43:51.228 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max latency(s): 2.89809 2020-09-08T15:43:51.228 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min latency(s): 0.00154046 2020-09-08T15:43:51.241 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Cleaning up (deleting benchmark objects) 2020-09-08T15:43:51.415 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:43:51.595 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:43:51.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:51 smithi104 bash[9203]: cluster 2020-09-08T15:43:49.691295+0000 mgr.x (mgr.24118) 495 : cluster [DBG] pgmap v664: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s wr, 807 op/s; 27539/243700 objects misplaced (11.300%); 6.3 MiB/s, 100 objects/s recovering 2020-09-08T15:43:51.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:51 smithi055 bash[9296]: cluster 2020-09-08T15:43:49.691295+0000 mgr.x (mgr.24118) 495 : cluster [DBG] pgmap v664: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s wr, 807 op/s; 27539/243700 objects misplaced (11.300%); 6.3 MiB/s, 100 objects/s recovering 2020-09-08T15:43:52.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:51 smithi166 bash[9446]: cluster 2020-09-08T15:43:49.691295+0000 mgr.x (mgr.24118) 495 : cluster [DBG] pgmap v664: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s wr, 807 op/s; 27539/243700 objects misplaced (11.300%); 6.3 MiB/s, 100 objects/s recovering 2020-09-08T15:43:52.517 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:43:52.619 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:52.755 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:52.859 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:52.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:43:52 smithi055 bash[16646]: debug 2020-09-08T15:43:52.515+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:52.981 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:43:52 smithi166 bash[16593]: debug 2020-09-08T15:43:52.753+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:52.983 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:53.087 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:53.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:43:52 smithi166 bash[18960]: debug 2020-09-08T15:43:52.981+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:53.385 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:53.494 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:53.655 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:53 smithi104 bash[9203]: cluster 2020-09-08T15:43:51.691774+0000 mgr.x (mgr.24118) 496 : cluster [DBG] pgmap v665: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s wr, 570 op/s; 27425/244900 objects misplaced (11.198%); 4.2 MiB/s, 67 objects/s recovering 2020-09-08T15:43:53.655 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:43:53 smithi104 bash[15273]: debug 2020-09-08T15:43:53.387+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:53.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:53 smithi055 bash[9296]: cluster 2020-09-08T15:43:51.691774+0000 mgr.x (mgr.24118) 496 : cluster [DBG] pgmap v665: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s wr, 570 op/s; 27425/244900 objects misplaced (11.198%); 4.2 MiB/s, 67 objects/s recovering 2020-09-08T15:43:54.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:53 smithi166 bash[9446]: cluster 2020-09-08T15:43:51.691774+0000 mgr.x (mgr.24118) 496 : cluster [DBG] pgmap v665: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s wr, 570 op/s; 27425/244900 objects misplaced (11.198%); 4.2 MiB/s, 67 objects/s recovering 2020-09-08T15:43:54.242 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:54.346 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:43:54.476 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:43:54.582 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:54.600 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:43:54 smithi104 bash[15273]: debug 2020-09-08T15:43:54.239+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:54.615 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:43:54 smithi166 bash[14254]: debug 2020-09-08T15:43:54.477+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:55.141 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:43:55.246 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:55.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:43:55 smithi104 bash[15273]: debug 2020-09-08T15:43:55.143+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:55.897 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:43:55.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:55 smithi104 bash[9203]: cluster 2020-09-08T15:43:53.692280+0000 mgr.x (mgr.24118) 497 : cluster [DBG] pgmap v666: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s wr, 694 op/s; 26895/246894 objects misplaced (10.893%); 6.7 MiB/s, 106 objects/s recovering 2020-09-08T15:43:56.001 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:43:56.010 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:55 smithi055 bash[9296]: cluster 2020-09-08T15:43:53.692280+0000 mgr.x (mgr.24118) 497 : cluster [DBG] pgmap v666: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s wr, 694 op/s; 26895/246894 objects misplaced (10.893%); 6.7 MiB/s, 106 objects/s recovering 2020-09-08T15:43:56.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:55 smithi166 bash[9446]: cluster 2020-09-08T15:43:53.692280+0000 mgr.x (mgr.24118) 497 : cluster [DBG] pgmap v666: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s wr, 694 op/s; 26895/246894 objects misplaced (10.893%); 6.7 MiB/s, 106 objects/s recovering 2020-09-08T15:43:56.117 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:43:55 smithi166 bash[18960]: debug 2020-09-08T15:43:55.897+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:56.145 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:43:56.249 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:43:56.262 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:43:56 smithi055 bash[16646]: debug 2020-09-08T15:43:56.143+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:56.375 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:43:56.480 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:56.631 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:43:56.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:43:56 smithi055 bash[14301]: debug 2020-09-08T15:43:56.375+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:56.732 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:43:56.742 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:43:56 smithi166 bash[16593]: debug 2020-09-08T15:43:56.629+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:56.846 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:43:56.950 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:43:57.062 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:43:57.115 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:43:56 smithi166 bash[11928]: debug 2020-09-08T15:43:56.845+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:57.166 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:43:57.278 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:43:57 smithi055 bash[11956]: debug 2020-09-08T15:43:57.059+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:57.279 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:43:57.306 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:43:57.307 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:43:57.385 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:57.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:43:57 smithi055 bash[19003]: debug 2020-09-08T15:43:57.279+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:57.666 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:57 smithi104 bash[9203]: cluster 2020-09-08T15:43:55.692745+0000 mgr.x (mgr.24118) 498 : cluster [DBG] pgmap v667: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s rd, 36 MiB/s wr, 662 op/s; 26895/246708 objects misplaced (10.902%); 6.0 MiB/s, 95 objects/s recovering 2020-09-08T15:43:57.666 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:57 smithi104 bash[9203]: cluster 2020-09-08T15:43:56.355524+0000 osd.9 (osd.9) 25 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:57.667 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:57 smithi104 bash[9203]: cluster 2020-09-08T15:43:56.355608+0000 osd.9 (osd.9) 26 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:57.667 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:57 smithi104 bash[9203]: cluster 2020-09-08T15:43:56.674352+0000 mon.a (mon.0) 1193 : cluster [DBG] osdmap e254: 12 total, 12 up, 12 in 2020-09-08T15:43:57.667 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:57 smithi104 bash[9203]: audit 2020-09-08T15:43:57.474325+0000 mon.c (mon.2) 46 : audit [INF] from='client.? v1:172.21.15.104:0/258296615' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:43:57.667 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:57 smithi104 bash[9203]: audit 2020-09-08T15:43:57.475685+0000 mon.a (mon.0) 1194 : audit [INF] from='client.35171 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:43:57.679 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:43:58.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:57 smithi166 bash[9446]: cluster 2020-09-08T15:43:55.692745+0000 mgr.x (mgr.24118) 498 : cluster [DBG] pgmap v667: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s rd, 36 MiB/s wr, 662 op/s; 26895/246708 objects misplaced (10.902%); 6.0 MiB/s, 95 objects/s recovering 2020-09-08T15:43:58.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:57 smithi166 bash[9446]: cluster 2020-09-08T15:43:56.355524+0000 osd.9 (osd.9) 25 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:58.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:57 smithi166 bash[9446]: cluster 2020-09-08T15:43:56.355608+0000 osd.9 (osd.9) 26 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:58.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:57 smithi166 bash[9446]: cluster 2020-09-08T15:43:56.674352+0000 mon.a (mon.0) 1193 : cluster [DBG] osdmap e254: 12 total, 12 up, 12 in 2020-09-08T15:43:58.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:57 smithi166 bash[9446]: audit 2020-09-08T15:43:57.474325+0000 mon.c (mon.2) 46 : audit [INF] from='client.? v1:172.21.15.104:0/258296615' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:43:58.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:57 smithi166 bash[9446]: audit 2020-09-08T15:43:57.475685+0000 mon.a (mon.0) 1194 : audit [INF] from='client.35171 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:43:58.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:57 smithi055 bash[9296]: cluster 2020-09-08T15:43:55.692745+0000 mgr.x (mgr.24118) 498 : cluster [DBG] pgmap v667: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s rd, 36 MiB/s wr, 662 op/s; 26895/246708 objects misplaced (10.902%); 6.0 MiB/s, 95 objects/s recovering 2020-09-08T15:43:58.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:57 smithi055 bash[9296]: cluster 2020-09-08T15:43:56.355524+0000 osd.9 (osd.9) 25 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:58.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:57 smithi055 bash[9296]: cluster 2020-09-08T15:43:56.355608+0000 osd.9 (osd.9) 26 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:58.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:57 smithi055 bash[9296]: cluster 2020-09-08T15:43:56.674352+0000 mon.a (mon.0) 1193 : cluster [DBG] osdmap e254: 12 total, 12 up, 12 in 2020-09-08T15:43:58.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:57 smithi055 bash[9296]: audit 2020-09-08T15:43:57.474325+0000 mon.c (mon.2) 46 : audit [INF] from='client.? v1:172.21.15.104:0/258296615' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:43:58.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:57 smithi055 bash[9296]: audit 2020-09-08T15:43:57.475685+0000 mon.a (mon.0) 1194 : audit [INF] from='client.35171 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:43:58.227 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:58.330 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:58.345 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:43:58 smithi104 bash[22305]: debug 2020-09-08T15:43:58.227+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:58.673 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:58 smithi104 bash[9203]: cluster 2020-09-08T15:43:57.670395+0000 mon.a (mon.0) 1195 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:43:58.674 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:58 smithi104 bash[9203]: audit 2020-09-08T15:43:57.680026+0000 mon.a (mon.0) 1196 : audit [INF] from='client.35171 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:43:58.674 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:58 smithi104 bash[9203]: cluster 2020-09-08T15:43:57.680073+0000 mon.a (mon.0) 1197 : cluster [DBG] osdmap e255: 12 total, 12 up, 12 in 2020-09-08T15:43:59.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:58 smithi166 bash[9446]: cluster 2020-09-08T15:43:57.670395+0000 mon.a (mon.0) 1195 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:43:59.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:58 smithi166 bash[9446]: audit 2020-09-08T15:43:57.680026+0000 mon.a (mon.0) 1196 : audit [INF] from='client.35171 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:43:59.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:58 smithi166 bash[9446]: cluster 2020-09-08T15:43:57.680073+0000 mon.a (mon.0) 1197 : cluster [DBG] osdmap e255: 12 total, 12 up, 12 in 2020-09-08T15:43:59.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:58 smithi055 bash[9296]: cluster 2020-09-08T15:43:57.670395+0000 mon.a (mon.0) 1195 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:43:59.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:58 smithi055 bash[9296]: audit 2020-09-08T15:43:57.680026+0000 mon.a (mon.0) 1196 : audit [INF] from='client.35171 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:43:59.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:58 smithi055 bash[9296]: cluster 2020-09-08T15:43:57.680073+0000 mon.a (mon.0) 1197 : cluster [DBG] osdmap e255: 12 total, 12 up, 12 in 2020-09-08T15:43:59.788 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:43:59.906 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:43:59.931 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:43:59 smithi104 bash[22305]: debug 2020-09-08T15:43:59.787+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:43:59.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:59 smithi104 bash[9203]: cluster 2020-09-08T15:43:57.693197+0000 mgr.x (mgr.24118) 499 : cluster [DBG] pgmap v670: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 18 MiB/s wr, 706 op/s; 26425/241640 objects misplaced (10.936%); 6.9 MiB/s, 110 objects/s recovering 2020-09-08T15:43:59.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:59 smithi104 bash[9203]: cluster 2020-09-08T15:43:58.525969+0000 osd.2 (osd.2) 36 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:43:59.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:59 smithi104 bash[9203]: cluster 2020-09-08T15:43:58.526066+0000 osd.2 (osd.2) 37 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:43:59.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:43:59 smithi104 bash[9203]: cluster 2020-09-08T15:43:58.685668+0000 mon.a (mon.0) 1198 : cluster [DBG] osdmap e256: 12 total, 12 up, 12 in 2020-09-08T15:44:00.123 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:59 smithi166 bash[9446]: cluster 2020-09-08T15:43:57.693197+0000 mgr.x (mgr.24118) 499 : cluster [DBG] pgmap v670: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 18 MiB/s wr, 706 op/s; 26425/241640 objects misplaced (10.936%); 6.9 MiB/s, 110 objects/s recovering 2020-09-08T15:44:00.128 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:59 smithi166 bash[9446]: cluster 2020-09-08T15:43:58.525969+0000 osd.2 (osd.2) 36 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:00.128 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:59 smithi166 bash[9446]: cluster 2020-09-08T15:43:58.526066+0000 osd.2 (osd.2) 37 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:00.128 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:43:59 smithi166 bash[9446]: cluster 2020-09-08T15:43:58.685668+0000 mon.a (mon.0) 1198 : cluster [DBG] osdmap e256: 12 total, 12 up, 12 in 2020-09-08T15:44:00.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:59 smithi055 bash[9296]: cluster 2020-09-08T15:43:57.693197+0000 mgr.x (mgr.24118) 499 : cluster [DBG] pgmap v670: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 18 MiB/s wr, 706 op/s; 26425/241640 objects misplaced (10.936%); 6.9 MiB/s, 110 objects/s recovering 2020-09-08T15:44:00.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:59 smithi055 bash[9296]: cluster 2020-09-08T15:43:58.525969+0000 osd.2 (osd.2) 36 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:00.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:59 smithi055 bash[9296]: cluster 2020-09-08T15:43:58.526066+0000 osd.2 (osd.2) 37 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:00.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:43:59 smithi055 bash[9296]: cluster 2020-09-08T15:43:58.685668+0000 mon.a (mon.0) 1198 : cluster [DBG] osdmap e256: 12 total, 12 up, 12 in 2020-09-08T15:44:00.489 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:44:00.575 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2020-09-08T15:44:00.592 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:00.748 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:44:00.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:44:00 smithi104 bash[19922]: debug 2020-09-08T15:44:00.487+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:00.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:00 smithi104 bash[9203]: cluster 2020-09-08T15:43:59.594773+0000 osd.6 (osd.6) 31 : cluster [DBG] 3.a scrub starts 2020-09-08T15:44:01.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:00 smithi166 bash[9446]: cluster 2020-09-08T15:43:59.594773+0000 osd.6 (osd.6) 31 : cluster [DBG] 3.a scrub starts 2020-09-08T15:44:01.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:00 smithi055 bash[9296]: cluster 2020-09-08T15:43:59.594773+0000 osd.6 (osd.6) 31 : cluster [DBG] 3.a scrub starts 2020-09-08T15:44:01.445 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:01.549 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:44:01.682 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:44:01.790 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:44:01.801 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:01 smithi104 bash[9203]: cluster 2020-09-08T15:43:59.693664+0000 mgr.x (mgr.24118) 500 : cluster [DBG] pgmap v672: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 1.7 MiB/s wr, 687 op/s; 26299/239304 objects misplaced (10.990%); 5.0 MiB/s, 80 objects/s recovering 2020-09-08T15:44:01.801 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:01 smithi104 bash[9203]: cluster 2020-09-08T15:44:01.042482+0000 osd.6 (osd.6) 32 : cluster [DBG] 3.a scrub ok 2020-09-08T15:44:01.802 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:44:01 smithi104 bash[22305]: debug 2020-09-08T15:44:01.447+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:01.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:44:01 smithi055 bash[19003]: debug 2020-09-08T15:44:01.683+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:01.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:01 smithi055 bash[9296]: cluster 2020-09-08T15:43:59.693664+0000 mgr.x (mgr.24118) 500 : cluster [DBG] pgmap v672: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 1.7 MiB/s wr, 687 op/s; 26299/239304 objects misplaced (10.990%); 5.0 MiB/s, 80 objects/s recovering 2020-09-08T15:44:01.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:01 smithi055 bash[9296]: cluster 2020-09-08T15:44:01.042482+0000 osd.6 (osd.6) 32 : cluster [DBG] 3.a scrub ok 2020-09-08T15:44:02.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:01 smithi166 bash[9446]: cluster 2020-09-08T15:43:59.693664+0000 mgr.x (mgr.24118) 500 : cluster [DBG] pgmap v672: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 1.7 MiB/s wr, 687 op/s; 26299/239304 objects misplaced (10.990%); 5.0 MiB/s, 80 objects/s recovering 2020-09-08T15:44:02.124 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:01 smithi166 bash[9446]: cluster 2020-09-08T15:44:01.042482+0000 osd.6 (osd.6) 32 : cluster [DBG] 3.a scrub ok 2020-09-08T15:44:02.439 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:44:02.547 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:44:02.578 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:44:02.703 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:02 smithi104 bash[9203]: cluster 2020-09-08T15:44:01.694181+0000 mgr.x (mgr.24118) 501 : cluster [DBG] pgmap v673: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 721 op/s; 26208/238046 objects misplaced (11.010%); 6.0 MiB/s, 95 objects/s recovering 2020-09-08T15:44:02.704 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:44:02 smithi104 bash[15273]: debug 2020-09-08T15:44:02.435+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:02.749 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:44:02.926 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:44:02.954 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:44:02.963 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:02 smithi166 bash[9446]: cluster 2020-09-08T15:44:01.694181+0000 mgr.x (mgr.24118) 501 : cluster [DBG] pgmap v673: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 721 op/s; 26208/238046 objects misplaced (11.010%); 6.0 MiB/s, 95 objects/s recovering 2020-09-08T15:44:02.992 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:44:03.024 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:44:03.061 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:44:03.102 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:44:03.146 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:44:03.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:02 smithi055 bash[9296]: cluster 2020-09-08T15:44:01.694181+0000 mgr.x (mgr.24118) 501 : cluster [DBG] pgmap v673: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 721 op/s; 26208/238046 objects misplaced (11.010%); 6.0 MiB/s, 95 objects/s recovering 2020-09-08T15:44:03.189 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:44:03.224 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:44:03.262 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:44:03.290 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:44:03.330 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:44:03.336 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:44:03.336 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:44:03.336 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:44:03.336 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:44:03.337 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:44:03.337 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:44:03.337 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:44:03.337 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:44:03.338 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:44:03.338 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:44:03.338 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:44:03.338 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:44:04.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:03 smithi166 bash[9446]: audit 2020-09-08T15:44:02.899946+0000 mon.a (mon.0) 1199 : audit [DBG] from='client.? v1:172.21.15.104:0/644083019' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:44:04.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:03 smithi055 bash[9296]: audit 2020-09-08T15:44:02.899946+0000 mon.a (mon.0) 1199 : audit [DBG] from='client.? v1:172.21.15.104:0/644083019' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:44:04.190 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:03 smithi104 bash[9203]: audit 2020-09-08T15:44:02.899946+0000 mon.a (mon.0) 1199 : audit [DBG] from='client.? v1:172.21.15.104:0/644083019' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:44:04.848 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:44:04.953 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:05.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:44:04 smithi166 bash[11928]: debug 2020-09-08T15:44:04.844+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:05.720 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:05.822 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:44:05.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:05 smithi104 bash[9203]: cluster 2020-09-08T15:44:03.694637+0000 mgr.x (mgr.24118) 502 : cluster [DBG] pgmap v674: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 856 op/s; 25776/233416 objects misplaced (11.043%); 7.1 MiB/s, 113 objects/s recovering 2020-09-08T15:44:05.931 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:44:05 smithi104 bash[17588]: debug 2020-09-08T15:44:05.719+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:06.054 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:44:06.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:05 smithi166 bash[9446]: cluster 2020-09-08T15:44:03.694637+0000 mgr.x (mgr.24118) 502 : cluster [DBG] pgmap v674: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 856 op/s; 25776/233416 objects misplaced (11.043%); 7.1 MiB/s, 113 objects/s recovering 2020-09-08T15:44:06.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:05 smithi055 bash[9296]: cluster 2020-09-08T15:44:03.694637+0000 mgr.x (mgr.24118) 502 : cluster [DBG] pgmap v674: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 856 op/s; 25776/233416 objects misplaced (11.043%); 7.1 MiB/s, 113 objects/s recovering 2020-09-08T15:44:06.164 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:44:06.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:44:06 smithi104 bash[15273]: debug 2020-09-08T15:44:06.051+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:06.937 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:44:07.042 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:07.368 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:44:06 smithi166 bash[18960]: debug 2020-09-08T15:44:06.932+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:07.778 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:07.880 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:44:07.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:44:07 smithi104 bash[17588]: debug 2020-09-08T15:44:07.775+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:07.938 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:07 smithi104 bash[9203]: cluster 2020-09-08T15:44:05.695069+0000 mgr.x (mgr.24118) 503 : cluster [DBG] pgmap v675: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 569 op/s; 25776/232510 objects misplaced (11.086%); 4.3 MiB/s, 68 objects/s recovering 2020-09-08T15:44:08.074 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:44:08.094 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:44:08.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:07 smithi166 bash[9446]: cluster 2020-09-08T15:44:05.695069+0000 mgr.x (mgr.24118) 503 : cluster [DBG] pgmap v675: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 569 op/s; 25776/232510 objects misplaced (11.086%); 4.3 MiB/s, 68 objects/s recovering 2020-09-08T15:44:08.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:07 smithi055 bash[9296]: cluster 2020-09-08T15:44:05.695069+0000 mgr.x (mgr.24118) 503 : cluster [DBG] pgmap v675: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 569 op/s; 25776/232510 objects misplaced (11.086%); 4.3 MiB/s, 68 objects/s recovering 2020-09-08T15:44:08.199 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:44:08.244 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:44:08.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:44:08 smithi104 bash[19922]: debug 2020-09-08T15:44:08.091+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:08.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:08 smithi104 bash[9203]: cluster 2020-09-08T15:44:06.961442+0000 osd.7 (osd.7) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:08.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:08 smithi104 bash[9203]: cluster 2020-09-08T15:44:06.961539+0000 osd.7 (osd.7) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:08.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:08 smithi104 bash[9203]: cluster 2020-09-08T15:44:07.667371+0000 mon.a (mon.0) 1200 : cluster [DBG] osdmap e257: 12 total, 12 up, 12 in 2020-09-08T15:44:09.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:08 smithi166 bash[9446]: cluster 2020-09-08T15:44:06.961442+0000 osd.7 (osd.7) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:09.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:08 smithi166 bash[9446]: cluster 2020-09-08T15:44:06.961539+0000 osd.7 (osd.7) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:09.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:08 smithi166 bash[9446]: cluster 2020-09-08T15:44:07.667371+0000 mon.a (mon.0) 1200 : cluster [DBG] osdmap e257: 12 total, 12 up, 12 in 2020-09-08T15:44:09.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:08 smithi055 bash[9296]: cluster 2020-09-08T15:44:06.961442+0000 osd.7 (osd.7) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:09.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:08 smithi055 bash[9296]: cluster 2020-09-08T15:44:06.961539+0000 osd.7 (osd.7) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:09.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:08 smithi055 bash[9296]: cluster 2020-09-08T15:44:07.667371+0000 mon.a (mon.0) 1200 : cluster [DBG] osdmap e257: 12 total, 12 up, 12 in 2020-09-08T15:44:09.604 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:44:09.712 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:09.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:09 smithi104 bash[9203]: cluster 2020-09-08T15:44:07.695596+0000 mgr.x (mgr.24118) 504 : cluster [DBG] pgmap v677: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 698 op/s; 25304/227654 objects misplaced (11.115%); 5.5 MiB/s, 87 objects/s recovering 2020-09-08T15:44:09.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:09 smithi104 bash[9203]: cluster 2020-09-08T15:44:08.812451+0000 osd.0 (osd.0) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:09.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:09 smithi104 bash[9203]: cluster 2020-09-08T15:44:08.812552+0000 osd.0 (osd.0) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:09.927 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:44:09 smithi104 bash[15273]: debug 2020-09-08T15:44:09.607+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:10.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:09 smithi166 bash[9446]: cluster 2020-09-08T15:44:07.695596+0000 mgr.x (mgr.24118) 504 : cluster [DBG] pgmap v677: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 698 op/s; 25304/227654 objects misplaced (11.115%); 5.5 MiB/s, 87 objects/s recovering 2020-09-08T15:44:10.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:09 smithi166 bash[9446]: cluster 2020-09-08T15:44:08.812451+0000 osd.0 (osd.0) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:10.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:09 smithi166 bash[9446]: cluster 2020-09-08T15:44:08.812552+0000 osd.0 (osd.0) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:10.162 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:09 smithi055 bash[9296]: cluster 2020-09-08T15:44:07.695596+0000 mgr.x (mgr.24118) 504 : cluster [DBG] pgmap v677: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 698 op/s; 25304/227654 objects misplaced (11.115%); 5.5 MiB/s, 87 objects/s recovering 2020-09-08T15:44:10.163 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:09 smithi055 bash[9296]: cluster 2020-09-08T15:44:08.812451+0000 osd.0 (osd.0) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:10.163 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:09 smithi055 bash[9296]: cluster 2020-09-08T15:44:08.812552+0000 osd.0 (osd.0) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:10.591 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:10.695 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:44:10.719 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:44:10.719 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T15:44:10.720 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.720 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:138314 3.c 3:3a32a9db:::benchmark_data_smithi166_26097_object14153:head [delete] snapc 0=[] ondisk+write+known_if_redirected e258)", 2020-09-08T15:44:10.720 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:10.687180+0000", 2020-09-08T15:44:10.720 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.031305351000000002, 2020-09-08T15:44:10.721 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.031341029999999999, 2020-09-08T15:44:10.721 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:10.721 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:44:10.721 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:10.723 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:10.723 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:10.724 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 138314 2020-09-08T15:44:10.724 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.724 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:10.724 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.725 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:10.725 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.687180+0000", 2020-09-08T15:44:10.725 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:10.725 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.725 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.726 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:10.726 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.687180+0000", 2020-09-08T15:44:10.726 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999561 2020-09-08T15:44:10.726 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.726 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.727 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:10.727 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.687136+0000", 2020-09-08T15:44:10.727 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8716999999999997e-05 2020-09-08T15:44:10.727 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.727 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.728 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:10.728 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.687235+0000", 2020-09-08T15:44:10.728 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6076999999999999e-05 2020-09-08T15:44:10.728 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.728 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.729 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:10.729 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.687251+0000", 2020-09-08T15:44:10.729 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9910000000000002e-06 2020-09-08T15:44:10.729 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.731 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.731 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:10.732 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.687259+0000", 2020-09-08T15:44:10.732 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0209e-05 2020-09-08T15:44:10.732 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.736 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.738 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:10.738 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.687309+0000", 2020-09-08T15:44:10.739 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00053509100000000002 2020-09-08T15:44:10.739 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.739 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.739 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:10.740 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.687844+0000", 2020-09-08T15:44:10.740 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000191819 2020-09-08T15:44:10.740 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.740 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.740 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:44:10.741 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.688036+0000", 2020-09-08T15:44:10.741 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.015004939 2020-09-08T15:44:10.741 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.741 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.741 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:10.742 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.703041+0000", 2020-09-08T15:44:10.742 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002076558 2020-09-08T15:44:10.742 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.742 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.742 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:10.743 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.705117+0000", 2020-09-08T15:44:10.743 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.017937616 2020-09-08T15:44:10.743 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.743 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:10.743 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.744 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.744 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.744 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:138316 3.2 3:44fb4324:::benchmark_data_smithi166_26097_object14155:head [delete] snapc 0=[] ondisk+write+known_if_redirected e258)", 2020-09-08T15:44:10.744 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:10.692623+0000", 2020-09-08T15:44:10.744 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.025862434, 2020-09-08T15:44:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.026005734999999999, 2020-09-08T15:44:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:44:10.745 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 138316 2020-09-08T15:44:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.746 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.692623+0000", 2020-09-08T15:44:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:10.747 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.692623+0000", 2020-09-08T15:44:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999958 2020-09-08T15:44:10.748 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.692581+0000", 2020-09-08T15:44:10.749 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9735999999999995e-05 2020-09-08T15:44:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.692680+0000", 2020-09-08T15:44:10.750 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7473999999999999e-05 2020-09-08T15:44:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:10.751 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.692698+0000", 2020-09-08T15:44:10.754 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1440000000000003e-06 2020-09-08T15:44:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:10.755 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.692706+0000", 2020-09-08T15:44:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00097460000000000005 2020-09-08T15:44:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.756 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:10.760 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.693681+0000", 2020-09-08T15:44:10.760 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00043044600000000002 2020-09-08T15:44:10.760 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.760 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.760 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:10.761 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.694111+0000", 2020-09-08T15:44:10.761 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00089655900000000001 2020-09-08T15:44:10.761 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.761 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.762 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 9", 2020-09-08T15:44:10.762 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.695008+0000", 2020-09-08T15:44:10.762 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0045383530000000002 2020-09-08T15:44:10.762 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.764 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.764 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:10.764 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.699546+0000", 2020-09-08T15:44:10.766 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.006923312 2020-09-08T15:44:10.766 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.766 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:10.766 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.767 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.767 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.767 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:138320 3.c 3:3d91cab1:::benchmark_data_smithi166_26097_object14159:head [delete] snapc 0=[] ondisk+write+known_if_redirected e258)", 2020-09-08T15:44:10.767 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:10.700285+0000", 2020-09-08T15:44:10.768 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.018199749000000001, 2020-09-08T15:44:10.768 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.018437433, 2020-09-08T15:44:10.768 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:10.768 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:44:10.769 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:10.769 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:10.769 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:10.769 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 138320 2020-09-08T15:44:10.770 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.770 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:10.770 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.771 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:10.771 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.700285+0000", 2020-09-08T15:44:10.771 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:10.772 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.772 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.772 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:10.772 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.700285+0000", 2020-09-08T15:44:10.772 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999528 2020-09-08T15:44:10.773 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.773 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.773 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:10.774 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.700238+0000", 2020-09-08T15:44:10.774 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000106308 2020-09-08T15:44:10.775 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.775 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.775 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:10.775 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.700344+0000", 2020-09-08T15:44:10.775 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8434e-05 2020-09-08T15:44:10.776 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.776 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.776 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:10.776 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.700363+0000", 2020-09-08T15:44:10.776 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2039999999999997e-06 2020-09-08T15:44:10.777 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.777 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.777 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:10.777 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.700371+0000", 2020-09-08T15:44:10.778 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0047871620000000002 2020-09-08T15:44:10.778 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.778 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.778 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:10.779 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.705158+0000", 2020-09-08T15:44:10.779 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000206267 2020-09-08T15:44:10.779 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.779 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.779 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:10.780 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.705365+0000", 2020-09-08T15:44:10.780 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000200891 2020-09-08T15:44:10.780 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.780 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.780 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:44:10.781 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.705565+0000", 2020-09-08T15:44:10.781 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0052802090000000001 2020-09-08T15:44:10.781 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.781 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:10.781 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.782 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.782 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.782 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:138322 3.2 3:454bf566:::benchmark_data_smithi166_26097_object14161:head [delete] snapc 0=[] ondisk+write+known_if_redirected e258)", 2020-09-08T15:44:10.782 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:10.701666+0000", 2020-09-08T15:44:10.782 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.016818796, 2020-09-08T15:44:10.783 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.017143265000000001, 2020-09-08T15:44:10.783 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:10.783 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:44:10.783 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:10.783 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:10.783 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:10.784 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 138322 2020-09-08T15:44:10.784 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.784 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:10.784 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.784 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:10.785 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.701666+0000", 2020-09-08T15:44:10.785 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:10.785 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.785 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.785 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:10.786 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.701666+0000", 2020-09-08T15:44:10.786 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999614 2020-09-08T15:44:10.786 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.786 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.787 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:10.787 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.701627+0000", 2020-09-08T15:44:10.787 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6188999999999998e-05 2020-09-08T15:44:10.787 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.787 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.788 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:10.788 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.701714+0000", 2020-09-08T15:44:10.788 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3738e-05 2020-09-08T15:44:10.788 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.788 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.789 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:10.789 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.701727+0000", 2020-09-08T15:44:10.789 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6180000000000003e-06 2020-09-08T15:44:10.789 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.789 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.790 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:10.790 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.701733+0000", 2020-09-08T15:44:10.790 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5858e-05 2020-09-08T15:44:10.791 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.791 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.791 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:10.791 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.701779+0000", 2020-09-08T15:44:10.791 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000145952 2020-09-08T15:44:10.792 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.792 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.792 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:10.792 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.701925+0000", 2020-09-08T15:44:10.792 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000134303 2020-09-08T15:44:10.793 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.793 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.793 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 9", 2020-09-08T15:44:10.793 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.702059+0000", 2020-09-08T15:44:10.793 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0048583890000000003 2020-09-08T15:44:10.794 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.794 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.794 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:10.794 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.706917+0000", 2020-09-08T15:44:10.794 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0052512119999999999 2020-09-08T15:44:10.794 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.795 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:10.795 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.795 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.795 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.795 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:138324 3.b 3:d53d89e1:::benchmark_data_smithi166_26097_object14163:head [delete] snapc 0=[] ondisk+write+known_if_redirected e258)", 2020-09-08T15:44:10.796 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:10.707027+0000", 2020-09-08T15:44:10.798 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.011457765999999999, 2020-09-08T15:44:10.799 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.011880161, 2020-09-08T15:44:10.799 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:10.799 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:44:10.799 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:10.800 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:10.800 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:10.800 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 138324 2020-09-08T15:44:10.800 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.807 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:10.807 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.807 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:10.808 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.707027+0000", 2020-09-08T15:44:10.808 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:10.808 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.808 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.808 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:10.809 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.707027+0000", 2020-09-08T15:44:10.809 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999959 2020-09-08T15:44:10.809 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.809 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.810 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:10.810 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.706986+0000", 2020-09-08T15:44:10.810 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000101356 2020-09-08T15:44:10.810 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.811 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.811 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:10.811 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.707088+0000", 2020-09-08T15:44:10.811 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1217e-05 2020-09-08T15:44:10.811 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.812 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.812 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:10.812 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.707109+0000", 2020-09-08T15:44:10.812 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0876999999999999e-05 2020-09-08T15:44:10.812 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.812 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.813 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:10.813 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.707120+0000", 2020-09-08T15:44:10.813 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2292999999999995e-05 2020-09-08T15:44:10.813 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.814 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.814 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:10.814 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.707182+0000", 2020-09-08T15:44:10.814 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015757799999999999 2020-09-08T15:44:10.814 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.814 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.815 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:10.815 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.707340+0000", 2020-09-08T15:44:10.815 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000176701 2020-09-08T15:44:10.815 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.815 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.816 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0", 2020-09-08T15:44:10.816 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.707516+0000", 2020-09-08T15:44:10.816 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0051231870000000004 2020-09-08T15:44:10.816 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.816 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.817 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:10.817 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.712639+0000", 2020-09-08T15:44:10.817 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0056121909999999999 2020-09-08T15:44:10.817 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.817 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:10.818 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.819 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.819 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.819 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:138328 3.2 3:4ece648f:::benchmark_data_smithi166_26097_object14167:head [delete] snapc 0=[] ondisk+write+known_if_redirected e258)", 2020-09-08T15:44:10.819 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:10.707224+0000", 2020-09-08T15:44:10.819 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.011261092, 2020-09-08T15:44:10.820 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.011776211999999999, 2020-09-08T15:44:10.820 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:10.820 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:44:10.820 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:10.820 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:10.821 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:10.821 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 138328 2020-09-08T15:44:10.821 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.821 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:10.821 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.822 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:10.822 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.707224+0000", 2020-09-08T15:44:10.822 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:10.822 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.823 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.823 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:10.823 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.707224+0000", 2020-09-08T15:44:10.823 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999595 2020-09-08T15:44:10.823 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.824 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.824 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:10.824 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.707183+0000", 2020-09-08T15:44:10.824 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8760999999999999e-05 2020-09-08T15:44:10.824 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.825 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.825 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:10.825 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.707272+0000", 2020-09-08T15:44:10.825 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3634e-05 2020-09-08T15:44:10.825 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.825 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.826 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:10.826 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.707286+0000", 2020-09-08T15:44:10.826 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3800000000000002e-06 2020-09-08T15:44:10.826 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.826 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.827 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:10.827 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.707291+0000", 2020-09-08T15:44:10.827 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00029669100000000002 2020-09-08T15:44:10.827 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.827 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.828 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:10.828 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.707588+0000", 2020-09-08T15:44:10.828 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000163979 2020-09-08T15:44:10.828 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.828 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.829 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:10.829 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.707752+0000", 2020-09-08T15:44:10.829 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014083499999999999 2020-09-08T15:44:10.829 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.829 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.830 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 9", 2020-09-08T15:44:10.830 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.707892+0000", 2020-09-08T15:44:10.830 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0052204740000000001 2020-09-08T15:44:10.830 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.830 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.831 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:10.831 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.713113+0000", 2020-09-08T15:44:10.831 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0058890000000000001 2020-09-08T15:44:10.831 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.831 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:10.832 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.832 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.832 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.832 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop_reply(client.35087.0:138314 3.c e258/236)", 2020-09-08T15:44:10.832 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:10.716812+0000", 2020-09-08T15:44:10.832 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.001672939, 2020-09-08T15:44:10.833 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0022788460000000002, 2020-09-08T15:44:10.833 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:10.833 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "queued for pg", 2020-09-08T15:44:10.833 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:10.833 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.836 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:10.836 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.716812+0000", 2020-09-08T15:44:10.837 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:10.837 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.838 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.838 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:10.838 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.716812+0000", 2020-09-08T15:44:10.839 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9614999999999998e-05 2020-09-08T15:44:10.839 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.839 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.839 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:10.839 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.716862+0000", 2020-09-08T15:44:10.840 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1499999999999999e-06 2020-09-08T15:44:10.840 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.840 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.840 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:10.840 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.716870+0000", 2020-09-08T15:44:10.841 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4518000000000001e-05 2020-09-08T15:44:10.841 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.841 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.841 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:10.841 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.716884+0000", 2020-09-08T15:44:10.843 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6760000000000001e-06 2020-09-08T15:44:10.843 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.843 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.843 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:10.843 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.716891+0000", 2020-09-08T15:44:10.844 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.8959000000000001e-05 2020-09-08T15:44:10.844 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.844 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:10.844 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.844 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.845 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.845 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop_reply(client.35087.0:138320 3.c e258/236)", 2020-09-08T15:44:10.845 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:10.716993+0000", 2020-09-08T15:44:10.845 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0014921789999999999, 2020-09-08T15:44:10.845 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0021498250000000002, 2020-09-08T15:44:10.846 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:10.846 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "queued for pg", 2020-09-08T15:44:10.846 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:10.846 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.846 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:10.847 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.716993+0000", 2020-09-08T15:44:10.847 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:10.847 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.847 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.847 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:10.847 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.716993+0000", 2020-09-08T15:44:10.848 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6631000000000001e-05 2020-09-08T15:44:10.848 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.848 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.848 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:10.848 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.717039+0000", 2020-09-08T15:44:10.849 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6760000000000001e-06 2020-09-08T15:44:10.849 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.849 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.849 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:10.849 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.717046+0000", 2020-09-08T15:44:10.850 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1875999999999999e-05 2020-09-08T15:44:10.850 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.850 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.850 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:10.850 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.717058+0000", 2020-09-08T15:44:10.851 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6689999999999997e-06 2020-09-08T15:44:10.851 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.851 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.851 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:10.851 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.717063+0000", 2020-09-08T15:44:10.852 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9851999999999995e-05 2020-09-08T15:44:10.852 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.852 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:10.852 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.852 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.852 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.853 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:138332 3.c 3.39261b1c (undecoded) ondisk+write+known_if_redirected e258)", 2020-09-08T15:44:10.853 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:10.717160+0000", 2020-09-08T15:44:10.853 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.00132525, 2020-09-08T15:44:10.853 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020367779999999999, 2020-09-08T15:44:10.853 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:10.854 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "queued for pg", 2020-09-08T15:44:10.854 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:10.854 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:10.854 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:10.854 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 138332 2020-09-08T15:44:10.855 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.855 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:10.855 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.855 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:10.855 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.717160+0000", 2020-09-08T15:44:10.856 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:10.856 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.856 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.856 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:10.856 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.717160+0000", 2020-09-08T15:44:10.856 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999609 2020-09-08T15:44:10.857 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.857 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.857 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:10.857 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.717120+0000", 2020-09-08T15:44:10.858 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1484999999999996e-05 2020-09-08T15:44:10.858 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.859 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.859 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:10.859 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.717212+0000", 2020-09-08T15:44:10.859 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5804999999999999e-05 2020-09-08T15:44:10.860 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.860 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.860 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:10.860 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.717228+0000", 2020-09-08T15:44:10.861 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9719999999999997e-06 2020-09-08T15:44:10.861 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.861 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.861 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:10.861 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.717234+0000", 2020-09-08T15:44:10.862 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3932999999999998e-05 2020-09-08T15:44:10.862 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.862 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:10.862 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.862 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.863 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.863 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:138333 3.2 3:4dbf82cb:::benchmark_data_smithi166_26097_object14172:head [delete] snapc 0=[] ondisk+write+known_if_redirected e258)", 2020-09-08T15:44:10.863 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:10.717303+0000", 2020-09-08T15:44:10.863 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.001182237, 2020-09-08T15:44:10.863 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0019586970000000001, 2020-09-08T15:44:10.864 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:10.864 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:44:10.864 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:10.864 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:10.865 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:10.865 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 138333 2020-09-08T15:44:10.865 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.865 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:10.865 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.866 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:10.866 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.717303+0000", 2020-09-08T15:44:10.866 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:10.866 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.866 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.868 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:10.868 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.717303+0000", 2020-09-08T15:44:10.868 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999958 2020-09-08T15:44:10.868 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.869 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.870 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:10.870 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.717261+0000", 2020-09-08T15:44:10.870 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2676000000000005e-05 2020-09-08T15:44:10.870 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.871 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.872 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:10.872 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.717353+0000", 2020-09-08T15:44:10.872 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3179e-05 2020-09-08T15:44:10.873 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.873 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.873 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:10.874 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.717366+0000", 2020-09-08T15:44:10.874 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8380000000000001e-06 2020-09-08T15:44:10.874 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.874 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.874 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:10.875 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.717371+0000", 2020-09-08T15:44:10.875 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00029857100000000003 2020-09-08T15:44:10.875 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.875 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.876 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:10.876 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.717670+0000", 2020-09-08T15:44:10.876 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000133831 2020-09-08T15:44:10.876 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.876 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.877 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:10.877 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.717804+0000", 2020-09-08T15:44:10.877 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013138099999999999 2020-09-08T15:44:10.877 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.877 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.878 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 9", 2020-09-08T15:44:10.878 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.717935+0000", 2020-09-08T15:44:10.878 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00063232299999999996 2020-09-08T15:44:10.878 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.879 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:10.879 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.879 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.879 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.879 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:138334 3.2 3.db6f0ca2 (undecoded) ondisk+write+known_if_redirected e258)", 2020-09-08T15:44:10.880 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:10.718515+0000", 2020-09-08T15:44:10.880 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 4294967295.9999704, 2020-09-08T15:44:10.880 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00084145200000000002, 2020-09-08T15:44:10.880 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:10.880 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "queued for pg", 2020-09-08T15:44:10.881 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:10.881 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:10.881 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:10.881 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 138334 2020-09-08T15:44:10.881 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.882 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:10.882 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.882 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:10.883 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.718515+0000", 2020-09-08T15:44:10.883 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:10.883 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.883 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.883 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:10.884 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.718515+0000", 2020-09-08T15:44:10.884 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999576 2020-09-08T15:44:10.884 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.884 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.884 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:10.885 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.718472+0000", 2020-09-08T15:44:10.885 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5204999999999994e-05 2020-09-08T15:44:10.885 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.885 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.885 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:10.886 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.718567+0000", 2020-09-08T15:44:10.886 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4732000000000001e-05 2020-09-08T15:44:10.886 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.886 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.886 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:10.887 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.718582+0000", 2020-09-08T15:44:10.887 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1079999999999998e-06 2020-09-08T15:44:10.887 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.887 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.888 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:10.888 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.718588+0000", 2020-09-08T15:44:10.888 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3522999999999994e-05 2020-09-08T15:44:10.888 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.888 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:10.889 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.889 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.889 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.889 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:138336 3.b 3:d6200f88:::benchmark_data_smithi166_26097_object14175:head [delete] snapc 0=[] ondisk+write+known_if_redirected e258)", 2020-09-08T15:44:10.889 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:10.718664+0000", 2020-09-08T15:44:10.890 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 4294967295.9998212, 2020-09-08T15:44:10.890 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00076351399999999997, 2020-09-08T15:44:10.891 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:10.891 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:44:10.891 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:10.892 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:10.892 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:10.892 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 138336 2020-09-08T15:44:10.893 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.893 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:10.893 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.893 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:10.893 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.718664+0000", 2020-09-08T15:44:10.894 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:10.894 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.894 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.894 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:10.895 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.718664+0000", 2020-09-08T15:44:10.895 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999599 2020-09-08T15:44:10.895 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.895 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.895 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:10.896 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.718623+0000", 2020-09-08T15:44:10.896 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0093000000000006e-05 2020-09-08T15:44:10.896 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.897 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.897 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:10.897 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.718714+0000", 2020-09-08T15:44:10.897 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2311e-05 2020-09-08T15:44:10.898 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.898 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.898 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:10.898 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.718726+0000", 2020-09-08T15:44:10.898 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4909999999999999e-06 2020-09-08T15:44:10.899 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.899 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.899 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:10.899 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.718730+0000", 2020-09-08T15:44:10.899 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6193000000000003e-05 2020-09-08T15:44:10.900 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.900 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.900 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:10.900 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.718777+0000", 2020-09-08T15:44:10.900 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015559699999999999 2020-09-08T15:44:10.901 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.901 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.901 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:10.901 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.718932+0000", 2020-09-08T15:44:10.901 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014840599999999999 2020-09-08T15:44:10.902 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.902 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.902 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0", 2020-09-08T15:44:10.902 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.719081+0000", 2020-09-08T15:44:10.903 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00041693500000000001 2020-09-08T15:44:10.903 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.903 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:10.903 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.903 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.904 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.904 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop_reply(client.35087.0:138316 3.2 e258/214)", 2020-09-08T15:44:10.904 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:10.718808+0000", 2020-09-08T15:44:10.904 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 4294967295.9996777, 2020-09-08T15:44:10.904 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00071315199999999999, 2020-09-08T15:44:10.905 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:10.905 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "queued for pg", 2020-09-08T15:44:10.906 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:10.906 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.906 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:10.906 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.718808+0000", 2020-09-08T15:44:10.906 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:10.907 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.907 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.907 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:10.907 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.718808+0000", 2020-09-08T15:44:10.907 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4976999999999999e-05 2020-09-08T15:44:10.908 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.908 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.908 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:10.908 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.718862+0000", 2020-09-08T15:44:10.909 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2870000000000004e-06 2020-09-08T15:44:10.910 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.910 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.910 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:10.910 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.718871+0000", 2020-09-08T15:44:10.911 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6308999999999999e-05 2020-09-08T15:44:10.911 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.911 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.911 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:10.912 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.718887+0000", 2020-09-08T15:44:10.912 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9280000000000001e-06 2020-09-08T15:44:10.912 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.912 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.913 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:10.913 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.718892+0000", 2020-09-08T15:44:10.913 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4500999999999996e-05 2020-09-08T15:44:10.913 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.913 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:10.914 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.914 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.914 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.914 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop_reply(client.35087.0:138322 3.2 e258/214)", 2020-09-08T15:44:10.914 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:10.719003+0000", 2020-09-08T15:44:10.915 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 4294967295.9994817, 2020-09-08T15:44:10.915 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00056864000000000003, 2020-09-08T15:44:10.915 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:10.915 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "queued for pg", 2020-09-08T15:44:10.916 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:10.916 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.916 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:10.916 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.719003+0000", 2020-09-08T15:44:10.916 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:10.917 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.917 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.917 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:10.917 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.719003+0000", 2020-09-08T15:44:10.917 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1384000000000001e-05 2020-09-08T15:44:10.918 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.918 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.918 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:10.918 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.719055+0000", 2020-09-08T15:44:10.919 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6520000000000006e-06 2020-09-08T15:44:10.919 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.919 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.919 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:10.919 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.719062+0000", 2020-09-08T15:44:10.920 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2969999999999999e-05 2020-09-08T15:44:10.920 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.920 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.920 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:10.920 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.719075+0000", 2020-09-08T15:44:10.921 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0080000000000004e-06 2020-09-08T15:44:10.921 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:10.921 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:10.921 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:10.922 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:10.719079+0000", 2020-09-08T15:44:10.922 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6013999999999994e-05 2020-09-08T15:44:10.922 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.922 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:10.923 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.923 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:10.923 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T15:44:10.923 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 14 2020-09-08T15:44:10.923 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:44:10.925 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:44:10.927 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:44:10 smithi104 bash[17588]: debug 2020-09-08T15:44:10.591+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:10.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:10 smithi104 bash[9203]: cluster 2020-09-08T15:44:09.684037+0000 mon.a (mon.0) 1201 : cluster [DBG] osdmap e258: 12 total, 12 up, 12 in 2020-09-08T15:44:10.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:10 smithi104 bash[9203]: cluster 2020-09-08T15:44:09.696121+0000 mgr.x (mgr.24118) 505 : cluster [DBG] pgmap v679: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 795 op/s; 25191/225314 objects misplaced (11.180%); 6.3 MiB/s, 101 objects/s recovering 2020-09-08T15:44:11.026 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:11.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:10 smithi166 bash[9446]: cluster 2020-09-08T15:44:09.684037+0000 mon.a (mon.0) 1201 : cluster [DBG] osdmap e258: 12 total, 12 up, 12 in 2020-09-08T15:44:11.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:10 smithi166 bash[9446]: cluster 2020-09-08T15:44:09.696121+0000 mgr.x (mgr.24118) 505 : cluster [DBG] pgmap v679: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 795 op/s; 25191/225314 objects misplaced (11.180%); 6.3 MiB/s, 101 objects/s recovering 2020-09-08T15:44:11.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:44:10 smithi166 bash[16593]: debug 2020-09-08T15:44:10.828+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:11.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:10 smithi055 bash[9296]: cluster 2020-09-08T15:44:09.684037+0000 mon.a (mon.0) 1201 : cluster [DBG] osdmap e258: 12 total, 12 up, 12 in 2020-09-08T15:44:11.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:10 smithi055 bash[9296]: cluster 2020-09-08T15:44:09.696121+0000 mgr.x (mgr.24118) 505 : cluster [DBG] pgmap v679: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 795 op/s; 25191/225314 objects misplaced (11.180%); 6.3 MiB/s, 101 objects/s recovering 2020-09-08T15:44:11.272 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:11.390 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:44:11.430 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:44:11 smithi104 bash[22305]: debug 2020-09-08T15:44:11.263+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:11.514 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:44:11.619 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:44:11.660 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:44:11 smithi055 bash[11956]: debug 2020-09-08T15:44:11.510+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:11.744 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:44:11.849 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:44:11.951 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:44:12.056 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:44:12.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:44:11 smithi055 bash[14301]: debug 2020-09-08T15:44:11.950+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:12.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:44:11 smithi055 bash[19003]: debug 2020-09-08T15:44:11.742+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:12.207 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:44:12.311 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:44:12.321 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:44:12 smithi166 bash[11928]: debug 2020-09-08T15:44:12.204+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:12.443 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:44:12.552 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:44:12.618 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:44:12 smithi166 bash[11928]: debug 2020-09-08T15:44:12.440+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:12.671 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:44:12.775 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:44:12.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:44:12 smithi055 bash[16646]: debug 2020-09-08T15:44:12.670+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:12.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:12 smithi055 bash[9296]: cluster 2020-09-08T15:44:11.696565+0000 mgr.x (mgr.24118) 506 : cluster [DBG] pgmap v680: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 599 op/s; 25093/223822 objects misplaced (11.211%); 4.5 MiB/s, 71 objects/s recovering 2020-09-08T15:44:12.927 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:44:13.034 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:44:13.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:12 smithi166 bash[9446]: cluster 2020-09-08T15:44:11.696565+0000 mgr.x (mgr.24118) 506 : cluster [DBG] pgmap v680: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 599 op/s; 25093/223822 objects misplaced (11.211%); 4.5 MiB/s, 71 objects/s recovering 2020-09-08T15:44:13.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:44:12 smithi166 bash[14254]: debug 2020-09-08T15:44:12.924+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:13.128 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:12 smithi104 bash[9203]: cluster 2020-09-08T15:44:11.696565+0000 mgr.x (mgr.24118) 506 : cluster [DBG] pgmap v680: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 599 op/s; 25093/223822 objects misplaced (11.211%); 4.5 MiB/s, 71 objects/s recovering 2020-09-08T15:44:13.148 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:44:13.250 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:44:13.369 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:44:13.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:44:13 smithi055 bash[16646]: debug 2020-09-08T15:44:13.138+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:13.431 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2020-09-08T15:44:13.474 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:44:13.593 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:44:13.646 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:44:13 smithi166 bash[16593]: debug 2020-09-08T15:44:13.368+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:13.647 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:44:13.754 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:44:13.883 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:44:13.908 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:44:13 smithi055 bash[11956]: debug 2020-09-08T15:44:13.590+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:13.985 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:44:14.092 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:44:13 smithi166 bash[16593]: debug 2020-09-08T15:44:13.872+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:14.094 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:44:14.197 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:44:14.302 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:44:14.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:44:14 smithi166 bash[14254]: debug 2020-09-08T15:44:14.300+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:14.368 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:44:14 smithi166 bash[18960]: debug 2020-09-08T15:44:14.092+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:14.409 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:14.774 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:44:14.815 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:14.919 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:44:15.035 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:14 smithi166 bash[9446]: cluster 2020-09-08T15:44:13.697135+0000 mgr.x (mgr.24118) 507 : cluster [DBG] pgmap v681: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 866 op/s; 24602/218646 objects misplaced (11.252%); 7.5 MiB/s, 119 objects/s recovering 2020-09-08T15:44:15.035 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:14 smithi166 bash[9446]: audit 2020-09-08T15:44:14.123843+0000 mon.c (mon.2) 47 : audit [INF] from='client.? v1:172.21.15.104:0/1188765200' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:44:15.036 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:14 smithi166 bash[9446]: audit 2020-09-08T15:44:14.125269+0000 mon.a (mon.0) 1202 : audit [INF] from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:44:15.036 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:14 smithi166 bash[9446]: cluster 2020-09-08T15:44:14.458334+0000 osd.5 (osd.5) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:15.037 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:14 smithi166 bash[9446]: cluster 2020-09-08T15:44:14.458490+0000 osd.5 (osd.5) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:15.037 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:44:15.141 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:44:15.149 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:14 smithi055 bash[9296]: cluster 2020-09-08T15:44:13.697135+0000 mgr.x (mgr.24118) 507 : cluster [DBG] pgmap v681: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 866 op/s; 24602/218646 objects misplaced (11.252%); 7.5 MiB/s, 119 objects/s recovering 2020-09-08T15:44:15.150 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:14 smithi055 bash[9296]: audit 2020-09-08T15:44:14.123843+0000 mon.c (mon.2) 47 : audit [INF] from='client.? v1:172.21.15.104:0/1188765200' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:44:15.150 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:14 smithi055 bash[9296]: audit 2020-09-08T15:44:14.125269+0000 mon.a (mon.0) 1202 : audit [INF] from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:44:15.150 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:14 smithi055 bash[9296]: cluster 2020-09-08T15:44:14.458334+0000 osd.5 (osd.5) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:15.150 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:14 smithi055 bash[9296]: cluster 2020-09-08T15:44:14.458490+0000 osd.5 (osd.5) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:15.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:44:14 smithi104 bash[22305]: debug 2020-09-08T15:44:14.815+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:15.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:14 smithi104 bash[9203]: cluster 2020-09-08T15:44:13.697135+0000 mgr.x (mgr.24118) 507 : cluster [DBG] pgmap v681: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 866 op/s; 24602/218646 objects misplaced (11.252%); 7.5 MiB/s, 119 objects/s recovering 2020-09-08T15:44:15.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:14 smithi104 bash[9203]: audit 2020-09-08T15:44:14.123843+0000 mon.c (mon.2) 47 : audit [INF] from='client.? v1:172.21.15.104:0/1188765200' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:44:15.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:14 smithi104 bash[9203]: audit 2020-09-08T15:44:14.125269+0000 mon.a (mon.0) 1202 : audit [INF] from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:44:15.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:14 smithi104 bash[9203]: cluster 2020-09-08T15:44:14.458334+0000 osd.5 (osd.5) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:15.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:14 smithi104 bash[9203]: cluster 2020-09-08T15:44:14.458490+0000 osd.5 (osd.5) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:15.260 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:44:15.363 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:15.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:44:15 smithi166 bash[11928]: debug 2020-09-08T15:44:15.032+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:15.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:44:15 smithi055 bash[19003]: debug 2020-09-08T15:44:15.254+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:15.903 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:16.017 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:16.026 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:15 smithi104 bash[9203]: cluster 2020-09-08T15:44:14.756279+0000 mon.a (mon.0) 1203 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:44:16.027 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:15 smithi104 bash[9203]: cluster 2020-09-08T15:44:14.756303+0000 mon.a (mon.0) 1204 : cluster [INF] Cluster is now healthy 2020-09-08T15:44:16.028 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:15 smithi104 bash[9203]: audit 2020-09-08T15:44:14.768976+0000 mon.a (mon.0) 1205 : audit [INF] from='client.35183 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:44:16.028 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:15 smithi104 bash[9203]: cluster 2020-09-08T15:44:14.769040+0000 mon.a (mon.0) 1206 : cluster [DBG] osdmap e259: 12 total, 12 up, 12 in 2020-09-08T15:44:16.028 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:44:15 smithi104 bash[17588]: debug 2020-09-08T15:44:15.903+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:16.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:15 smithi166 bash[9446]: cluster 2020-09-08T15:44:14.756279+0000 mon.a (mon.0) 1203 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:44:16.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:15 smithi166 bash[9446]: cluster 2020-09-08T15:44:14.756303+0000 mon.a (mon.0) 1204 : cluster [INF] Cluster is now healthy 2020-09-08T15:44:16.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:15 smithi166 bash[9446]: audit 2020-09-08T15:44:14.768976+0000 mon.a (mon.0) 1205 : audit [INF] from='client.35183 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:44:16.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:15 smithi166 bash[9446]: cluster 2020-09-08T15:44:14.769040+0000 mon.a (mon.0) 1206 : cluster [DBG] osdmap e259: 12 total, 12 up, 12 in 2020-09-08T15:44:16.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:15 smithi055 bash[9296]: cluster 2020-09-08T15:44:14.756279+0000 mon.a (mon.0) 1203 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:44:16.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:15 smithi055 bash[9296]: cluster 2020-09-08T15:44:14.756303+0000 mon.a (mon.0) 1204 : cluster [INF] Cluster is now healthy 2020-09-08T15:44:16.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:15 smithi055 bash[9296]: audit 2020-09-08T15:44:14.768976+0000 mon.a (mon.0) 1205 : audit [INF] from='client.35183 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:44:16.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:15 smithi055 bash[9296]: cluster 2020-09-08T15:44:14.769040+0000 mon.a (mon.0) 1206 : cluster [DBG] osdmap e259: 12 total, 12 up, 12 in 2020-09-08T15:44:16.683 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:16.788 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:44:16.919 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:44:17.022 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:44:17.031 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:44:16 smithi055 bash[19003]: debug 2020-09-08T15:44:16.914+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:17.032 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:16 smithi055 bash[9296]: cluster 2020-09-08T15:44:15.697543+0000 mgr.x (mgr.24118) 508 : cluster [DBG] pgmap v683: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 615 op/s; 24602/217806 objects misplaced (11.295%); 4.6 MiB/s, 73 objects/s recovering 2020-09-08T15:44:17.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:16 smithi166 bash[9446]: cluster 2020-09-08T15:44:15.697543+0000 mgr.x (mgr.24118) 508 : cluster [DBG] pgmap v683: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 615 op/s; 24602/217806 objects misplaced (11.295%); 4.6 MiB/s, 73 objects/s recovering 2020-09-08T15:44:17.131 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:44:17.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:16 smithi104 bash[9203]: cluster 2020-09-08T15:44:15.697543+0000 mgr.x (mgr.24118) 508 : cluster [DBG] pgmap v683: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 615 op/s; 24602/217806 objects misplaced (11.295%); 4.6 MiB/s, 73 objects/s recovering 2020-09-08T15:44:17.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:44:16 smithi104 bash[22305]: debug 2020-09-08T15:44:16.683+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:17.235 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:17.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:44:17 smithi055 bash[16646]: debug 2020-09-08T15:44:17.130+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:17.935 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:18.038 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:44:18.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:44:17 smithi104 bash[22305]: debug 2020-09-08T15:44:17.935+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:18.216 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:44:18.320 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:44:18.337 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:44:18 smithi166 bash[11928]: debug 2020-09-08T15:44:18.212+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:18.449 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:44:18.553 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:44:18.594 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:44:18.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:44:18 smithi166 bash[18960]: debug 2020-09-08T15:44:18.444+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:18.637 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:44:18.665 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:44:18.692 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:44:18.734 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:44:18.776 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:44:18.810 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:44:18.846 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:44:18.885 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:44:18.918 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:44:18.944 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:44:18.983 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:44:18.993 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:44:18.993 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:44:18.993 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:44:18.994 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:44:18.994 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:44:18.994 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:44:18.994 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:44:18.994 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:44:18.995 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:44:18.995 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:44:18.995 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:44:18.998 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:44:19.063 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:18 smithi104 bash[9203]: cluster 2020-09-08T15:44:17.697969+0000 mgr.x (mgr.24118) 509 : cluster [DBG] pgmap v684: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 741 op/s; 24025/213426 objects misplaced (11.257%); 7.3 MiB/s, 116 objects/s recovering 2020-09-08T15:44:19.064 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:18 smithi104 bash[9203]: audit 2020-09-08T15:44:18.564827+0000 mon.b (mon.1) 86 : audit [DBG] from='client.? v1:172.21.15.104:0/3664285144' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:44:19.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:18 smithi166 bash[9446]: cluster 2020-09-08T15:44:17.697969+0000 mgr.x (mgr.24118) 509 : cluster [DBG] pgmap v684: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 741 op/s; 24025/213426 objects misplaced (11.257%); 7.3 MiB/s, 116 objects/s recovering 2020-09-08T15:44:19.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:18 smithi166 bash[9446]: audit 2020-09-08T15:44:18.564827+0000 mon.b (mon.1) 86 : audit [DBG] from='client.? v1:172.21.15.104:0/3664285144' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:44:19.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:18 smithi055 bash[9296]: cluster 2020-09-08T15:44:17.697969+0000 mgr.x (mgr.24118) 509 : cluster [DBG] pgmap v684: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 741 op/s; 24025/213426 objects misplaced (11.257%); 7.3 MiB/s, 116 objects/s recovering 2020-09-08T15:44:19.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:18 smithi055 bash[9296]: audit 2020-09-08T15:44:18.564827+0000 mon.b (mon.1) 86 : audit [DBG] from='client.? v1:172.21.15.104:0/3664285144' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:44:19.393 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:44:19 smithi104 bash[15273]: debug 2020-09-08T15:44:19.395+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:19.398 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:44:19.500 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:44:19.925 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:44:20.083 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:44:20.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:19 smithi166 bash[9446]: cluster 2020-09-08T15:44:18.906258+0000 osd.10 (osd.10) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:20.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:19 smithi166 bash[9446]: cluster 2020-09-08T15:44:18.906329+0000 osd.10 (osd.10) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:20.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:19 smithi055 bash[9296]: cluster 2020-09-08T15:44:18.906258+0000 osd.10 (osd.10) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:20.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:19 smithi055 bash[9296]: cluster 2020-09-08T15:44:18.906329+0000 osd.10 (osd.10) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:20.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:19 smithi104 bash[9203]: cluster 2020-09-08T15:44:18.906258+0000 osd.10 (osd.10) 21 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:20.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:19 smithi104 bash[9203]: cluster 2020-09-08T15:44:18.906329+0000 osd.10 (osd.10) 22 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:20.921 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:44:21.038 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:44:21.052 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:20 smithi166 bash[9446]: cluster 2020-09-08T15:44:19.698461+0000 mgr.x (mgr.24118) 510 : cluster [DBG] pgmap v685: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 688 op/s; 23873/211548 objects misplaced (11.285%); 6.8 MiB/s, 108 objects/s recovering 2020-09-08T15:44:21.052 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:20 smithi166 bash[9446]: cluster 2020-09-08T15:44:19.788678+0000 mon.a (mon.0) 1207 : cluster [DBG] osdmap e260: 12 total, 12 up, 12 in 2020-09-08T15:44:21.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:44:20 smithi055 bash[11956]: debug 2020-09-08T15:44:20.918+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:21.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:20 smithi055 bash[9296]: cluster 2020-09-08T15:44:19.698461+0000 mgr.x (mgr.24118) 510 : cluster [DBG] pgmap v685: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 688 op/s; 23873/211548 objects misplaced (11.285%); 6.8 MiB/s, 108 objects/s recovering 2020-09-08T15:44:21.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:20 smithi055 bash[9296]: cluster 2020-09-08T15:44:19.788678+0000 mon.a (mon.0) 1207 : cluster [DBG] osdmap e260: 12 total, 12 up, 12 in 2020-09-08T15:44:21.175 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:44:21.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:20 smithi104 bash[9203]: cluster 2020-09-08T15:44:19.698461+0000 mgr.x (mgr.24118) 510 : cluster [DBG] pgmap v685: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 688 op/s; 23873/211548 objects misplaced (11.285%); 6.8 MiB/s, 108 objects/s recovering 2020-09-08T15:44:21.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:20 smithi104 bash[9203]: cluster 2020-09-08T15:44:19.788678+0000 mon.a (mon.0) 1207 : cluster [DBG] osdmap e260: 12 total, 12 up, 12 in 2020-09-08T15:44:21.280 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:44:21.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:44:21 smithi166 bash[18960]: debug 2020-09-08T15:44:21.172+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:21.380 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:44:21.482 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:44:21.583 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:44:21.687 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:44:21.803 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:44:21.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:44:21 smithi166 bash[16593]: debug 2020-09-08T15:44:21.580+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:21.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:44:21 smithi166 bash[18960]: debug 2020-09-08T15:44:21.376+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:21.909 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:44:22.023 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:44:21 smithi055 bash[14301]: debug 2020-09-08T15:44:21.802+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:22.024 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:44:22.125 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:44:22.240 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:44:22.344 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:44:22.353 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:44:22 smithi055 bash[14301]: debug 2020-09-08T15:44:22.018+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:22.353 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:44:22 smithi055 bash[14301]: debug 2020-09-08T15:44:22.238+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:22.454 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:44:22.559 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:22.658 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:44:22 smithi055 bash[16646]: debug 2020-09-08T15:44:22.454+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:23.024 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:22 smithi104 bash[9203]: cluster 2020-09-08T15:44:21.698942+0000 mgr.x (mgr.24118) 511 : cluster [DBG] pgmap v687: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 510 op/s; 23685/210474 objects misplaced (11.253%); 6.2 MiB/s, 98 objects/s recovering 2020-09-08T15:44:23.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:22 smithi166 bash[9446]: cluster 2020-09-08T15:44:21.698942+0000 mgr.x (mgr.24118) 511 : cluster [DBG] pgmap v687: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 510 op/s; 23685/210474 objects misplaced (11.253%); 6.2 MiB/s, 98 objects/s recovering 2020-09-08T15:44:23.136 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:22 smithi055 bash[9296]: cluster 2020-09-08T15:44:21.698942+0000 mgr.x (mgr.24118) 511 : cluster [DBG] pgmap v687: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 510 op/s; 23685/210474 objects misplaced (11.253%); 6.2 MiB/s, 98 objects/s recovering 2020-09-08T15:44:23.242 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:44:23.362 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:44:23.432 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:44:23 smithi104 bash[17588]: debug 2020-09-08T15:44:23.135+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:23.467 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:44:23.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:44:23 smithi055 bash[19003]: debug 2020-09-08T15:44:23.362+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:23.662 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:44:23.766 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:44:23.782 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:44:23 smithi104 bash[15273]: debug 2020-09-08T15:44:23.663+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:23.881 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:44:23.973 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:44:23.974 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:44:23.986 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:24.158 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:44:23 smithi055 bash[11956]: debug 2020-09-08T15:44:23.882+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:24.187 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:24.236 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:44:24.245 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:44:24 smithi104 bash[17588]: debug 2020-09-08T15:44:24.183+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:24.290 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:24.391 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:44:25.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:24 smithi166 bash[9446]: cluster 2020-09-08T15:44:23.248102+0000 osd.3 (osd.3) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:25.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:24 smithi166 bash[9446]: cluster 2020-09-08T15:44:23.248176+0000 osd.3 (osd.3) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:25.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:24 smithi166 bash[9446]: cluster 2020-09-08T15:44:23.699563+0000 mgr.x (mgr.24118) 512 : cluster [DBG] pgmap v688: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 628 op/s; 23115/206572 objects misplaced (11.190%); 8.6 MiB/s, 137 objects/s recovering 2020-09-08T15:44:25.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:24 smithi166 bash[9446]: cluster 2020-09-08T15:44:23.801869+0000 mon.a (mon.0) 1208 : cluster [DBG] osdmap e261: 12 total, 12 up, 12 in 2020-09-08T15:44:25.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:24 smithi055 bash[9296]: cluster 2020-09-08T15:44:23.248102+0000 osd.3 (osd.3) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:25.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:24 smithi055 bash[9296]: cluster 2020-09-08T15:44:23.248176+0000 osd.3 (osd.3) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:25.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:24 smithi055 bash[9296]: cluster 2020-09-08T15:44:23.699563+0000 mgr.x (mgr.24118) 512 : cluster [DBG] pgmap v688: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 628 op/s; 23115/206572 objects misplaced (11.190%); 8.6 MiB/s, 137 objects/s recovering 2020-09-08T15:44:25.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:24 smithi055 bash[9296]: cluster 2020-09-08T15:44:23.801869+0000 mon.a (mon.0) 1208 : cluster [DBG] osdmap e261: 12 total, 12 up, 12 in 2020-09-08T15:44:25.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:24 smithi104 bash[9203]: cluster 2020-09-08T15:44:23.248102+0000 osd.3 (osd.3) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:25.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:24 smithi104 bash[9203]: cluster 2020-09-08T15:44:23.248176+0000 osd.3 (osd.3) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:25.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:24 smithi104 bash[9203]: cluster 2020-09-08T15:44:23.699563+0000 mgr.x (mgr.24118) 512 : cluster [DBG] pgmap v688: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 628 op/s; 23115/206572 objects misplaced (11.190%); 8.6 MiB/s, 137 objects/s recovering 2020-09-08T15:44:25.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:24 smithi104 bash[9203]: cluster 2020-09-08T15:44:23.801869+0000 mon.a (mon.0) 1208 : cluster [DBG] osdmap e261: 12 total, 12 up, 12 in 2020-09-08T15:44:26.022 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:26.126 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:44:26.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:44:26 smithi104 bash[17588]: debug 2020-09-08T15:44:26.023+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:27.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:26 smithi166 bash[9446]: cluster 2020-09-08T15:44:25.699965+0000 mgr.x (mgr.24118) 513 : cluster [DBG] pgmap v690: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 475 op/s; 23115/205818 objects misplaced (11.231%); 6.1 MiB/s, 97 objects/s recovering 2020-09-08T15:44:27.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:26 smithi055 bash[9296]: cluster 2020-09-08T15:44:25.699965+0000 mgr.x (mgr.24118) 513 : cluster [DBG] pgmap v690: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 475 op/s; 23115/205818 objects misplaced (11.231%); 6.1 MiB/s, 97 objects/s recovering 2020-09-08T15:44:27.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:26 smithi104 bash[9203]: cluster 2020-09-08T15:44:25.699965+0000 mgr.x (mgr.24118) 513 : cluster [DBG] pgmap v690: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 475 op/s; 23115/205818 objects misplaced (11.231%); 6.1 MiB/s, 97 objects/s recovering 2020-09-08T15:44:27.466 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2020-09-08T15:44:27.632 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:44:29.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:28 smithi166 bash[9446]: cluster 2020-09-08T15:44:27.700519+0000 mgr.x (mgr.24118) 514 : cluster [DBG] pgmap v691: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 631 op/s; 22498/201444 objects misplaced (11.168%); 8.7 MiB/s, 139 objects/s recovering 2020-09-08T15:44:29.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:28 smithi055 bash[9296]: cluster 2020-09-08T15:44:27.700519+0000 mgr.x (mgr.24118) 514 : cluster [DBG] pgmap v691: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 631 op/s; 22498/201444 objects misplaced (11.168%); 8.7 MiB/s, 139 objects/s recovering 2020-09-08T15:44:29.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:28 smithi104 bash[9203]: cluster 2020-09-08T15:44:27.700519+0000 mgr.x (mgr.24118) 514 : cluster [DBG] pgmap v691: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 631 op/s; 22498/201444 objects misplaced (11.168%); 8.7 MiB/s, 139 objects/s recovering 2020-09-08T15:44:29.636 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:44:29.741 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:44:29.875 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:44:29.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:44:29 smithi055 bash[16646]: debug 2020-09-08T15:44:29.634+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:29.978 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:44:30.109 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:44:29 smithi166 bash[14254]: debug 2020-09-08T15:44:29.872+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:30.109 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:44:30 smithi166 bash[16593]: debug 2020-09-08T15:44:30.108+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:30.109 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:44:30.214 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:30.814 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:44:31.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:30 smithi166 bash[9446]: cluster 2020-09-08T15:44:29.178183+0000 osd.8 (osd.8) 35 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:31.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:30 smithi166 bash[9446]: cluster 2020-09-08T15:44:29.178255+0000 osd.8 (osd.8) 36 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:31.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:30 smithi166 bash[9446]: cluster 2020-09-08T15:44:29.700968+0000 mgr.x (mgr.24118) 515 : cluster [DBG] pgmap v692: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 612 op/s; 22358/199394 objects misplaced (11.213%); 7.9 MiB/s, 126 objects/s recovering 2020-09-08T15:44:31.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:30 smithi166 bash[9446]: cluster 2020-09-08T15:44:29.816108+0000 mon.a (mon.0) 1209 : cluster [DBG] osdmap e262: 12 total, 12 up, 12 in 2020-09-08T15:44:31.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:30 smithi166 bash[9446]: audit 2020-09-08T15:44:30.564128+0000 mon.c (mon.2) 48 : audit [INF] from='client.? v1:172.21.15.104:0/310397403' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:44:31.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:30 smithi166 bash[9446]: audit 2020-09-08T15:44:30.565648+0000 mon.a (mon.0) 1210 : audit [INF] from='client.35192 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:44:31.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:30 smithi055 bash[9296]: cluster 2020-09-08T15:44:29.178183+0000 osd.8 (osd.8) 35 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:31.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:30 smithi055 bash[9296]: cluster 2020-09-08T15:44:29.178255+0000 osd.8 (osd.8) 36 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:31.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:30 smithi055 bash[9296]: cluster 2020-09-08T15:44:29.700968+0000 mgr.x (mgr.24118) 515 : cluster [DBG] pgmap v692: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 612 op/s; 22358/199394 objects misplaced (11.213%); 7.9 MiB/s, 126 objects/s recovering 2020-09-08T15:44:31.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:30 smithi055 bash[9296]: cluster 2020-09-08T15:44:29.816108+0000 mon.a (mon.0) 1209 : cluster [DBG] osdmap e262: 12 total, 12 up, 12 in 2020-09-08T15:44:31.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:30 smithi055 bash[9296]: audit 2020-09-08T15:44:30.564128+0000 mon.c (mon.2) 48 : audit [INF] from='client.? v1:172.21.15.104:0/310397403' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:44:31.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:30 smithi055 bash[9296]: audit 2020-09-08T15:44:30.565648+0000 mon.a (mon.0) 1210 : audit [INF] from='client.35192 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:44:31.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:30 smithi104 bash[9203]: cluster 2020-09-08T15:44:29.178183+0000 osd.8 (osd.8) 35 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:31.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:30 smithi104 bash[9203]: cluster 2020-09-08T15:44:29.178255+0000 osd.8 (osd.8) 36 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:31.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:30 smithi104 bash[9203]: cluster 2020-09-08T15:44:29.700968+0000 mgr.x (mgr.24118) 515 : cluster [DBG] pgmap v692: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 612 op/s; 22358/199394 objects misplaced (11.213%); 7.9 MiB/s, 126 objects/s recovering 2020-09-08T15:44:31.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:30 smithi104 bash[9203]: cluster 2020-09-08T15:44:29.816108+0000 mon.a (mon.0) 1209 : cluster [DBG] osdmap e262: 12 total, 12 up, 12 in 2020-09-08T15:44:31.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:30 smithi104 bash[9203]: audit 2020-09-08T15:44:30.564128+0000 mon.c (mon.2) 48 : audit [INF] from='client.? v1:172.21.15.104:0/310397403' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:44:31.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:30 smithi104 bash[9203]: audit 2020-09-08T15:44:30.565648+0000 mon.a (mon.0) 1210 : audit [INF] from='client.35192 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:44:31.263 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:31.366 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:31.526 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:44:31 smithi104 bash[17588]: debug 2020-09-08T15:44:31.263+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:31.554 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:31.662 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:44:31.817 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:44:31 smithi104 bash[22305]: debug 2020-09-08T15:44:31.555+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:31.817 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:31 smithi104 bash[9203]: cluster 2020-09-08T15:44:30.811387+0000 mon.a (mon.0) 1211 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:44:31.817 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:31 smithi104 bash[9203]: audit 2020-09-08T15:44:30.815424+0000 mon.a (mon.0) 1212 : audit [INF] from='client.35192 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:44:31.818 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:31 smithi104 bash[9203]: cluster 2020-09-08T15:44:30.815471+0000 mon.a (mon.0) 1213 : cluster [DBG] osdmap e263: 12 total, 12 up, 12 in 2020-09-08T15:44:32.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:31 smithi166 bash[9446]: cluster 2020-09-08T15:44:30.811387+0000 mon.a (mon.0) 1211 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:44:32.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:31 smithi166 bash[9446]: audit 2020-09-08T15:44:30.815424+0000 mon.a (mon.0) 1212 : audit [INF] from='client.35192 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:44:32.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:31 smithi166 bash[9446]: cluster 2020-09-08T15:44:30.815471+0000 mon.a (mon.0) 1213 : cluster [DBG] osdmap e263: 12 total, 12 up, 12 in 2020-09-08T15:44:32.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:31 smithi055 bash[9296]: cluster 2020-09-08T15:44:30.811387+0000 mon.a (mon.0) 1211 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:44:32.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:31 smithi055 bash[9296]: audit 2020-09-08T15:44:30.815424+0000 mon.a (mon.0) 1212 : audit [INF] from='client.35192 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:44:32.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:31 smithi055 bash[9296]: cluster 2020-09-08T15:44:30.815471+0000 mon.a (mon.0) 1213 : cluster [DBG] osdmap e263: 12 total, 12 up, 12 in 2020-09-08T15:44:32.694 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:44:32.798 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:44:32.938 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:32 smithi104 bash[9203]: cluster 2020-09-08T15:44:31.701553+0000 mgr.x (mgr.24118) 516 : cluster [DBG] pgmap v695: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 478 op/s; 22171/198254 objects misplaced (11.183%); 6.4 MiB/s, 103 objects/s recovering 2020-09-08T15:44:32.939 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:44:32 smithi104 bash[15273]: debug 2020-09-08T15:44:32.691+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:32.971 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:44:33.083 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:44:33.092 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:32 smithi055 bash[9296]: cluster 2020-09-08T15:44:31.701553+0000 mgr.x (mgr.24118) 516 : cluster [DBG] pgmap v695: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 478 op/s; 22171/198254 objects misplaced (11.183%); 6.4 MiB/s, 103 objects/s recovering 2020-09-08T15:44:33.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:32 smithi166 bash[9446]: cluster 2020-09-08T15:44:31.701553+0000 mgr.x (mgr.24118) 516 : cluster [DBG] pgmap v695: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 478 op/s; 22171/198254 objects misplaced (11.183%); 6.4 MiB/s, 103 objects/s recovering 2020-09-08T15:44:33.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:44:32 smithi166 bash[11928]: debug 2020-09-08T15:44:32.972+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:33.198 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:44:33.302 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:44:33.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:44:33 smithi055 bash[16646]: debug 2020-09-08T15:44:33.198+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:33.408 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:44:33.514 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:44:33.914 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:44:33 smithi055 bash[19003]: debug 2020-09-08T15:44:33.406+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:33.934 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:44:34.039 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:34.177 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:44:33 smithi104 bash[19922]: debug 2020-09-08T15:44:33.931+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:34.273 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:34.376 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:44:34.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:44:34 smithi104 bash[17588]: debug 2020-09-08T15:44:34.267+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:35.454 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:44:35.552 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:44:35.557 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:44:35.599 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:44:35.626 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:44:35.670 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:44:35.679 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:44:35.699 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:44:35.735 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:44:35.769 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:44:35.784 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:44:35.801 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:44:35.834 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:44:35.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:35 smithi166 bash[9446]: cluster 2020-09-08T15:44:33.702207+0000 mgr.x (mgr.24118) 517 : cluster [DBG] pgmap v696: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 741 op/s; 21615/193946 objects misplaced (11.145%); 9.7 MiB/s, 154 objects/s recovering 2020-09-08T15:44:35.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:35 smithi166 bash[9446]: audit 2020-09-08T15:44:35.523731+0000 mon.a (mon.0) 1214 : audit [DBG] from='client.? v1:172.21.15.104:0/2711964153' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:44:35.867 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:44:35 smithi166 bash[14254]: debug 2020-09-08T15:44:35.680+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:35.868 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:44:35.894 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:44:35.898 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:44:35.903 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:44:35 smithi104 bash[19922]: debug 2020-09-08T15:44:35.450+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:35.904 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:35 smithi104 bash[9203]: cluster 2020-09-08T15:44:33.702207+0000 mgr.x (mgr.24118) 517 : cluster [DBG] pgmap v696: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 741 op/s; 21615/193946 objects misplaced (11.145%); 9.7 MiB/s, 154 objects/s recovering 2020-09-08T15:44:35.904 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:35 smithi104 bash[9203]: audit 2020-09-08T15:44:35.523731+0000 mon.a (mon.0) 1214 : audit [DBG] from='client.? v1:172.21.15.104:0/2711964153' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:44:35.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:35 smithi055 bash[9296]: cluster 2020-09-08T15:44:33.702207+0000 mgr.x (mgr.24118) 517 : cluster [DBG] pgmap v696: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 741 op/s; 21615/193946 objects misplaced (11.145%); 9.7 MiB/s, 154 objects/s recovering 2020-09-08T15:44:35.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:35 smithi055 bash[9296]: audit 2020-09-08T15:44:35.523731+0000 mon.a (mon.0) 1214 : audit [DBG] from='client.? v1:172.21.15.104:0/2711964153' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:44:35.917 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:44:35.930 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:44:35.935 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:44:35.935 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:44:35.935 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:44:35.936 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:44:35.936 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:44:35.936 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:44:35.937 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:44:35.937 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:44:35.937 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:44:35.938 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:44:35.938 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:44:35.938 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:44:36.026 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:44:36.059 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:44:36.143 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:44:36.243 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:44:36.252 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:44:35 smithi166 bash[14254]: debug 2020-09-08T15:44:35.920+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:36.252 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:44:36 smithi166 bash[18960]: debug 2020-09-08T15:44:36.136+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:36.367 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:44:36.471 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:36.615 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:44:36 smithi166 bash[14254]: debug 2020-09-08T15:44:36.360+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:37.370 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:37.472 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:44:37.713 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:44:37 smithi104 bash[22305]: debug 2020-09-08T15:44:37.370+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:37.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:37 smithi166 bash[9446]: cluster 2020-09-08T15:44:35.702681+0000 mgr.x (mgr.24118) 518 : cluster [DBG] pgmap v697: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 517 op/s; 21615/193166 objects misplaced (11.190%); 5.9 MiB/s, 93 objects/s recovering 2020-09-08T15:44:37.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:37 smithi166 bash[9446]: cluster 2020-09-08T15:44:36.856821+0000 osd.1 (osd.1) 50 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:37.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:37 smithi166 bash[9446]: cluster 2020-09-08T15:44:36.856953+0000 osd.1 (osd.1) 51 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:37.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:37 smithi055 bash[9296]: cluster 2020-09-08T15:44:35.702681+0000 mgr.x (mgr.24118) 518 : cluster [DBG] pgmap v697: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 517 op/s; 21615/193166 objects misplaced (11.190%); 5.9 MiB/s, 93 objects/s recovering 2020-09-08T15:44:37.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:37 smithi055 bash[9296]: cluster 2020-09-08T15:44:36.856821+0000 osd.1 (osd.1) 50 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:37.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:37 smithi055 bash[9296]: cluster 2020-09-08T15:44:36.856953+0000 osd.1 (osd.1) 51 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:38.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:37 smithi104 bash[9203]: cluster 2020-09-08T15:44:35.702681+0000 mgr.x (mgr.24118) 518 : cluster [DBG] pgmap v697: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 517 op/s; 21615/193166 objects misplaced (11.190%); 5.9 MiB/s, 93 objects/s recovering 2020-09-08T15:44:38.183 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:37 smithi104 bash[9203]: cluster 2020-09-08T15:44:36.856821+0000 osd.1 (osd.1) 50 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:38.183 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:37 smithi104 bash[9203]: cluster 2020-09-08T15:44:36.856953+0000 osd.1 (osd.1) 51 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:38.692 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:38 smithi104 bash[9203]: cluster 2020-09-08T15:44:37.668259+0000 mon.a (mon.0) 1215 : cluster [DBG] osdmap e264: 12 total, 12 up, 12 in 2020-09-08T15:44:38.960 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:44:38.961 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2020-09-08T15:44:38.961 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2020-09-08T15:44:38.961 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T15:44:38.962 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:38.962 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:29998 3.7 3:eee3627c:::benchmark_data_smithi166_26097_object29997:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e223)", 2020-09-08T15:44:38.962 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:42:30.208171+0000", 2020-09-08T15:44:38.963 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 128.750241905, 2020-09-08T15:44:38.963 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.232170193, 2020-09-08T15:44:38.963 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:38.964 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:44:38.964 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:38.964 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:38.964 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:38.965 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 29998 2020-09-08T15:44:38.965 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:38.965 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:38.965 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:38.970 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:38.970 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.208171+0000", 2020-09-08T15:44:38.970 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:38.970 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:38.970 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:38.971 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:38.971 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.208171+0000", 2020-09-08T15:44:38.971 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999969 2020-09-08T15:44:38.971 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:38.971 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:38.972 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:38.972 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.208140+0000", 2020-09-08T15:44:38.972 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011421000000000001 2020-09-08T15:44:38.972 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:38.972 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:38.973 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:38.973 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.208254+0000", 2020-09-08T15:44:38.973 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1049e-05 2020-09-08T15:44:38.974 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:38.974 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:38.974 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:38.974 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.208265+0000", 2020-09-08T15:44:38.975 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3650000000000003e-06 2020-09-08T15:44:38.975 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:38.975 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:38.975 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:38.975 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.208271+0000", 2020-09-08T15:44:38.976 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6675999999999998e-05 2020-09-08T15:44:38.976 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:38.976 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:38.976 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:38.976 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.208317+0000", 2020-09-08T15:44:38.977 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015322299999999999 2020-09-08T15:44:38.978 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:38.978 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:38.979 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:38.979 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.208471+0000", 2020-09-08T15:44:38.979 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019431099999999999 2020-09-08T15:44:38.979 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:38.979 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:38.980 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T15:44:38.980 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.208665+0000", 2020-09-08T15:44:38.980 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001639738 2020-09-08T15:44:38.980 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:38.981 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:38.981 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:38.982 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.210305+0000", 2020-09-08T15:44:38.982 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.22974133799999999 2020-09-08T15:44:38.982 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:38.983 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:38.983 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:38.984 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.440046+0000", 2020-09-08T15:44:38.984 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000168319 2020-09-08T15:44:38.985 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:38.985 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:38.985 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:44:38.986 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.440214+0000", 2020-09-08T15:44:38.986 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000127181 2020-09-08T15:44:38.986 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:38.987 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:38.993 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:44:38.993 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.440342+0000", 2020-09-08T15:44:38.994 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.232170193 2020-09-08T15:44:38.995 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:38.995 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:38.995 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:38.996 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:38.996 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:38.996 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:29999 3.b 3:d990a5a9:::benchmark_data_smithi166_26097_object29998:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e223)", 2020-09-08T15:44:38.997 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:42:30.209673+0000", 2020-09-08T15:44:38.997 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 128.748740083, 2020-09-08T15:44:38.997 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.219638259, 2020-09-08T15:44:38.998 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:38.998 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:44:38.998 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:38.999 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:38.999 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:39.000 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 29999 2020-09-08T15:44:39.000 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.000 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:39.001 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.001 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:39.002 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.209673+0000", 2020-09-08T15:44:39.002 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:39.003 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.003 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.003 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:39.004 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.209673+0000", 2020-09-08T15:44:39.004 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999948 2020-09-08T15:44:39.005 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.006 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.006 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:39.007 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.209621+0000", 2020-09-08T15:44:39.007 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000186195 2020-09-08T15:44:39.007 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.008 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.008 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:39.008 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.209807+0000", 2020-09-08T15:44:39.009 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6696000000000001e-05 2020-09-08T15:44:39.009 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.009 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.010 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:39.010 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.209824+0000", 2020-09-08T15:44:39.010 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.2830000000000004e-06 2020-09-08T15:44:39.011 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.011 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.012 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:39.012 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.209831+0000", 2020-09-08T15:44:39.013 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000139097 2020-09-08T15:44:39.013 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.013 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.014 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.014 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.209970+0000", 2020-09-08T15:44:39.014 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013665800000000001 2020-09-08T15:44:39.015 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.015 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.016 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:39.016 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.210107+0000", 2020-09-08T15:44:39.017 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000143654 2020-09-08T15:44:39.017 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.017 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.018 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0", 2020-09-08T15:44:39.018 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.210251+0000", 2020-09-08T15:44:39.018 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.218271621 2020-09-08T15:44:39.019 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.019 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.019 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:39.020 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.428522+0000", 2020-09-08T15:44:39.020 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000444485 2020-09-08T15:44:39.020 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.021 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.021 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.021 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.428967+0000", 2020-09-08T15:44:39.022 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019558800000000001 2020-09-08T15:44:39.022 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.022 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.023 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:44:39.023 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.429162+0000", 2020-09-08T15:44:39.023 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000149192 2020-09-08T15:44:39.024 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.024 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.024 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:44:39.025 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:30.429311+0000", 2020-09-08T15:44:39.025 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.219638259 2020-09-08T15:44:39.025 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.027 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:39.027 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.027 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.028 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.028 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:43780 3.2 3:4163bbab:::benchmark_data_smithi166_26097_object43779:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e224)", 2020-09-08T15:44:39.028 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:42:36.759490+0000", 2020-09-08T15:44:39.029 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 122.198923259, 2020-09-08T15:44:39.029 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.41321206799999999, 2020-09-08T15:44:39.029 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:39.030 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:44:39.030 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:39.030 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:39.031 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:39.031 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 43780 2020-09-08T15:44:39.031 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.032 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:39.032 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.032 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:39.033 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.759490+0000", 2020-09-08T15:44:39.033 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:39.033 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.034 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.034 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:39.034 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.759490+0000", 2020-09-08T15:44:39.035 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999547 2020-09-08T15:44:39.035 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.036 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.037 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:39.038 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.759445+0000", 2020-09-08T15:44:39.038 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014533100000000001 2020-09-08T15:44:39.038 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.039 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.039 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:39.039 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.759590+0000", 2020-09-08T15:44:39.040 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6682000000000001e-05 2020-09-08T15:44:39.040 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.041 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.041 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:39.041 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.759607+0000", 2020-09-08T15:44:39.042 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7320000000000001e-06 2020-09-08T15:44:39.042 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.043 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.043 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:39.043 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.759613+0000", 2020-09-08T15:44:39.044 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6366999999999998e-05 2020-09-08T15:44:39.044 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.044 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.045 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.045 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.759670+0000", 2020-09-08T15:44:39.045 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015800100000000001 2020-09-08T15:44:39.046 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.046 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.046 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:39.047 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.759828+0000", 2020-09-08T15:44:39.047 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000194437 2020-09-08T15:44:39.048 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.048 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.049 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 9", 2020-09-08T15:44:39.049 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.760022+0000", 2020-09-08T15:44:39.049 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.40266279900000002 2020-09-08T15:44:39.050 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.050 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.050 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:39.051 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:37.162685+0000", 2020-09-08T15:44:39.051 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.009906616 2020-09-08T15:44:39.052 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.052 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.053 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.053 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:37.172592+0000", 2020-09-08T15:44:39.053 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8109000000000003e-05 2020-09-08T15:44:39.054 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.054 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.054 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:44:39.055 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:37.172640+0000", 2020-09-08T15:44:39.055 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2464000000000004e-05 2020-09-08T15:44:39.055 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.056 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.056 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:44:39.056 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:37.172702+0000", 2020-09-08T15:44:39.057 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.41321206799999999 2020-09-08T15:44:39.057 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.057 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:39.061 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.061 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.061 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.062 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.35087.0:43778 3.5 e224/214 3:a04cf8c5:::benchmark_data_smithi166_26097_object43777:head v 224'2767, mlcod=224'2765)", 2020-09-08T15:44:39.062 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:42:36.760017+0000", 2020-09-08T15:44:39.062 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 122.198396587, 2020-09-08T15:44:39.063 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.40266761400000001, 2020-09-08T15:44:39.063 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:39.063 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:44:39.064 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:39.064 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.065 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:39.065 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.760017+0000", 2020-09-08T15:44:39.065 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:39.066 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.066 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.066 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:39.067 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.760017+0000", 2020-09-08T15:44:39.067 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5453999999999998e-05 2020-09-08T15:44:39.067 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.068 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.068 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:39.068 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.760112+0000", 2020-09-08T15:44:39.068 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5340000000000002e-06 2020-09-08T15:44:39.069 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.069 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.070 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:39.070 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.760120+0000", 2020-09-08T15:44:39.070 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2197e-05 2020-09-08T15:44:39.071 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.072 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.072 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:39.072 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.760132+0000", 2020-09-08T15:44:39.073 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1399999999999999e-06 2020-09-08T15:44:39.073 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.073 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.074 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:39.074 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.760137+0000", 2020-09-08T15:44:39.074 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7766000000000003e-05 2020-09-08T15:44:39.075 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.075 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.075 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.076 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.760185+0000", 2020-09-08T15:44:39.076 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8325000000000002e-05 2020-09-08T15:44:39.076 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.077 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.077 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:39.077 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.760203+0000", 2020-09-08T15:44:39.078 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.40228635600000001 2020-09-08T15:44:39.078 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.078 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.079 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:44:39.079 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:37.162489+0000", 2020-09-08T15:44:39.079 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000194842 2020-09-08T15:44:39.081 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.082 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.082 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:44:39.082 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:37.162684+0000", 2020-09-08T15:44:39.083 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.40266761400000001 2020-09-08T15:44:39.083 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.084 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:39.084 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.084 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.084 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.085 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88121 3.c 3:3274f1b2:::benchmark_data_smithi166_26097_object88120:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:44:39.085 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.847115+0000", 2020-09-08T15:44:39.085 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 94.111298242999993, 2020-09-08T15:44:39.085 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.75263644100000004, 2020-09-08T15:44:39.085 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:39.086 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:44:39.086 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:39.086 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:39.086 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:39.087 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88121 2020-09-08T15:44:39.087 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.087 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:39.087 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.087 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:39.088 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847115+0000", 2020-09-08T15:44:39.088 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:39.088 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.088 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.088 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:39.089 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847115+0000", 2020-09-08T15:44:39.089 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9998198 2020-09-08T15:44:39.089 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.089 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.089 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:39.090 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.846935+0000", 2020-09-08T15:44:39.090 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00042954899999999999 2020-09-08T15:44:39.090 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.090 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.091 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:39.091 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847364+0000", 2020-09-08T15:44:39.091 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4828999999999999e-05 2020-09-08T15:44:39.091 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.091 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.092 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:39.092 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847389+0000", 2020-09-08T15:44:39.093 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2669e-05 2020-09-08T15:44:39.093 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.093 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.093 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:39.093 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847402+0000", 2020-09-08T15:44:39.094 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00049592600000000005 2020-09-08T15:44:39.094 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.094 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.094 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.094 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847898+0000", 2020-09-08T15:44:39.095 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6884999999999998e-05 2020-09-08T15:44:39.095 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.095 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.095 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:44:39.095 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847915+0000", 2020-09-08T15:44:39.096 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.71660728200000001 2020-09-08T15:44:39.096 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.096 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.096 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.096 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.564522+0000", 2020-09-08T15:44:39.097 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0018229349999999999 2020-09-08T15:44:39.097 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.097 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.097 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:39.098 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.566345+0000", 2020-09-08T15:44:39.098 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018965599999999999 2020-09-08T15:44:39.098 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.098 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.098 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:44:39.099 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.566535+0000", 2020-09-08T15:44:39.099 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.015621836 2020-09-08T15:44:39.099 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.099 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.099 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.100 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.582156+0000", 2020-09-08T15:44:39.100 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0068253599999999999 2020-09-08T15:44:39.100 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.100 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.100 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:39.101 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.588982+0000", 2020-09-08T15:44:39.101 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0088333649999999993 2020-09-08T15:44:39.101 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.101 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.101 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.102 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.597815+0000", 2020-09-08T15:44:39.102 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4142999999999998e-05 2020-09-08T15:44:39.102 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.103 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.103 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:44:39.103 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.597889+0000", 2020-09-08T15:44:39.104 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0018622479999999999 2020-09-08T15:44:39.104 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.104 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.104 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:44:39.104 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.599752+0000", 2020-09-08T15:44:39.105 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.75263644100000004 2020-09-08T15:44:39.105 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.105 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:39.105 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.105 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.106 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.106 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88124 3.c 3:3aadf346:::benchmark_data_smithi166_26097_object88123:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:44:39.106 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.847493+0000", 2020-09-08T15:44:39.106 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 94.110920082999996, 2020-09-08T15:44:39.106 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.75358171299999999, 2020-09-08T15:44:39.107 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:39.107 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:44:39.107 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:39.107 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:39.107 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:39.108 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88124 2020-09-08T15:44:39.108 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.108 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:39.108 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.108 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:39.109 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847493+0000", 2020-09-08T15:44:39.109 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:39.109 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.109 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.109 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:39.110 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847493+0000", 2020-09-08T15:44:39.110 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999495 2020-09-08T15:44:39.110 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.110 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.111 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:39.111 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847443+0000", 2020-09-08T15:44:39.111 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000182323 2020-09-08T15:44:39.111 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.111 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.112 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:39.112 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847625+0000", 2020-09-08T15:44:39.112 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6303e-05 2020-09-08T15:44:39.113 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.113 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.114 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:39.114 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847641+0000", 2020-09-08T15:44:39.114 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5647000000000001e-05 2020-09-08T15:44:39.115 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.115 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.115 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:39.115 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847657+0000", 2020-09-08T15:44:39.115 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00031383799999999999 2020-09-08T15:44:39.116 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.116 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.116 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.116 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847971+0000", 2020-09-08T15:44:39.116 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2159e-05 2020-09-08T15:44:39.117 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.117 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.117 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:44:39.117 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847983+0000", 2020-09-08T15:44:39.117 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.72442674699999998 2020-09-08T15:44:39.118 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.118 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.118 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.118 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.572410+0000", 2020-09-08T15:44:39.119 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017684800000000001 2020-09-08T15:44:39.119 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.119 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.119 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:39.120 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.572587+0000", 2020-09-08T15:44:39.120 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018562599999999999 2020-09-08T15:44:39.120 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.120 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.120 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:44:39.121 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.572772+0000", 2020-09-08T15:44:39.121 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.016268680000000001 2020-09-08T15:44:39.121 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.121 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.121 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.122 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.589041+0000", 2020-09-08T15:44:39.122 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1363999999999998e-05 2020-09-08T15:44:39.122 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.122 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.122 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:39.123 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.589092+0000", 2020-09-08T15:44:39.123 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010730368000000001 2020-09-08T15:44:39.123 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.124 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.124 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.124 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.599823+0000", 2020-09-08T15:44:39.124 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2834000000000002e-05 2020-09-08T15:44:39.125 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.125 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.125 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:44:39.125 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.599885+0000", 2020-09-08T15:44:39.125 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0011894430000000001 2020-09-08T15:44:39.126 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.126 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.126 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:44:39.127 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.601075+0000", 2020-09-08T15:44:39.127 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.75358171299999999 2020-09-08T15:44:39.127 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.127 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:39.128 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.128 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.128 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.128 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88161 3.c 3:38648e2f:::benchmark_data_smithi166_26097_object88160:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:44:39.128 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.847735+0000", 2020-09-08T15:44:39.129 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 94.110677895999999, 2020-09-08T15:44:39.129 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.76368154799999999, 2020-09-08T15:44:39.129 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:39.129 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:44:39.129 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:39.130 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:39.130 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:39.130 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88161 2020-09-08T15:44:39.130 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.130 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847735+0000", 2020-09-08T15:44:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.132 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.132 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:39.132 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847735+0000", 2020-09-08T15:44:39.132 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999537 2020-09-08T15:44:39.132 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.133 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.133 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:39.133 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847689+0000", 2020-09-08T15:44:39.133 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015866 2020-09-08T15:44:39.133 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.134 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.134 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:39.135 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847848+0000", 2020-09-08T15:44:39.135 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5380000000000002e-06 2020-09-08T15:44:39.135 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.135 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.136 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:39.136 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847856+0000", 2020-09-08T15:44:39.136 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9319999999999998e-06 2020-09-08T15:44:39.136 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.136 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.137 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:39.137 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847860+0000", 2020-09-08T15:44:39.137 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015259200000000001 2020-09-08T15:44:39.137 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.137 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.138 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.138 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848013+0000", 2020-09-08T15:44:39.138 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0889e-05 2020-09-08T15:44:39.138 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.138 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.139 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:44:39.139 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848024+0000", 2020-09-08T15:44:39.139 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.72736553100000001 2020-09-08T15:44:39.139 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.139 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.140 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.140 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.575389+0000", 2020-09-08T15:44:39.140 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00037211999999999998 2020-09-08T15:44:39.140 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.140 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.141 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:39.141 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.575761+0000", 2020-09-08T15:44:39.141 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00042625200000000001 2020-09-08T15:44:39.141 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.141 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.142 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:44:39.142 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.576187+0000", 2020-09-08T15:44:39.142 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.012893685 2020-09-08T15:44:39.142 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.143 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.143 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.143 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.589081+0000", 2020-09-08T15:44:39.143 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0710999999999999e-05 2020-09-08T15:44:39.143 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.144 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.144 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:39.144 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.589102+0000", 2020-09-08T15:44:39.145 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.020542207999999999 2020-09-08T15:44:39.145 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.146 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.146 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.147 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.609644+0000", 2020-09-08T15:44:39.147 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1699000000000007e-05 2020-09-08T15:44:39.147 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.147 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.147 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:44:39.148 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.609706+0000", 2020-09-08T15:44:39.148 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001711202 2020-09-08T15:44:39.148 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.148 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.148 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:44:39.149 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.611417+0000", 2020-09-08T15:44:39.149 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.76368154799999999 2020-09-08T15:44:39.149 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.149 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:39.149 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.150 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.150 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.150 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88165 3.c 3:3ddcf4f9:::benchmark_data_smithi166_26097_object88164:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:44:39.150 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.847901+0000", 2020-09-08T15:44:39.150 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 94.110512474000004, 2020-09-08T15:44:39.151 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.77547376999999995, 2020-09-08T15:44:39.151 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:39.151 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:44:39.151 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:39.151 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:39.152 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:39.152 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88165 2020-09-08T15:44:39.152 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.152 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:39.152 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.153 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:39.153 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847901+0000", 2020-09-08T15:44:39.153 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:39.153 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.153 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.154 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:39.154 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847901+0000", 2020-09-08T15:44:39.154 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999762 2020-09-08T15:44:39.154 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847877+0000", 2020-09-08T15:44:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000129016 2020-09-08T15:44:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.156 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.156 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:39.156 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848006+0000", 2020-09-08T15:44:39.156 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2859999999999999e-06 2020-09-08T15:44:39.156 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.157 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.157 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:39.157 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848014+0000", 2020-09-08T15:44:39.157 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6950000000000002e-06 2020-09-08T15:44:39.157 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.158 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.158 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:39.158 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848018+0000", 2020-09-08T15:44:39.158 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7629999999999997e-05 2020-09-08T15:44:39.158 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.159 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.159 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.159 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848056+0000", 2020-09-08T15:44:39.159 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4609999999999997e-06 2020-09-08T15:44:39.159 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.160 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.160 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:44:39.160 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848065+0000", 2020-09-08T15:44:39.160 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.73443109100000004 2020-09-08T15:44:39.160 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.161 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.161 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.161 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.582496+0000", 2020-09-08T15:44:39.161 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000291484 2020-09-08T15:44:39.161 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.162 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.162 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:39.162 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.582788+0000", 2020-09-08T15:44:39.162 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00036355799999999999 2020-09-08T15:44:39.162 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.163 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.163 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:44:39.163 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.583151+0000", 2020-09-08T15:44:39.163 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0059636890000000003 2020-09-08T15:44:39.163 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.164 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.164 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:39.164 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.589115+0000", 2020-09-08T15:44:39.164 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.012338839000000001 2020-09-08T15:44:39.164 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.164 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.165 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.165 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.601454+0000", 2020-09-08T15:44:39.165 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.019415953999999999 2020-09-08T15:44:39.165 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.165 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.166 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.166 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.620870+0000", 2020-09-08T15:44:39.166 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000130216 2020-09-08T15:44:39.166 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.166 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:44:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.621000+0000", 2020-09-08T15:44:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002374733 2020-09-08T15:44:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.168 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:44:39.168 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.623375+0000", 2020-09-08T15:44:39.168 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.77547376999999995 2020-09-08T15:44:39.168 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.168 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:39.168 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.169 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.169 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.169 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88180 3.c 3:39dfcfe5:::benchmark_data_smithi166_26097_object88179:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:44:39.169 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.848069+0000", 2020-09-08T15:44:39.169 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 94.110343876000002, 2020-09-08T15:44:39.170 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.77707152300000004, 2020-09-08T15:44:39.170 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:39.170 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:44:39.170 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:39.170 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:39.171 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:39.171 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88180 2020-09-08T15:44:39.171 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.171 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:39.171 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.172 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:39.172 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848069+0000", 2020-09-08T15:44:39.172 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:39.172 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.172 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.173 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:39.173 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848069+0000", 2020-09-08T15:44:39.173 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999728 2020-09-08T15:44:39.173 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.173 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.173 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:39.174 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848042+0000", 2020-09-08T15:44:39.174 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000114635 2020-09-08T15:44:39.174 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.174 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.174 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:39.175 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848157+0000", 2020-09-08T15:44:39.175 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6072000000000001e-05 2020-09-08T15:44:39.175 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.175 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.175 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:39.176 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848173+0000", 2020-09-08T15:44:39.176 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.079e-06 2020-09-08T15:44:39.176 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.176 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.176 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:39.176 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848178+0000", 2020-09-08T15:44:39.177 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4790999999999999e-05 2020-09-08T15:44:39.177 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.177 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.177 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.177 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848223+0000", 2020-09-08T15:44:39.178 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0448999999999999e-05 2020-09-08T15:44:39.178 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.178 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.178 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:44:39.178 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848233+0000", 2020-09-08T15:44:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.73823174300000005 2020-09-08T15:44:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.586465+0000", 2020-09-08T15:44:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000170996 2020-09-08T15:44:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.586636+0000", 2020-09-08T15:44:39.181 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000164415 2020-09-08T15:44:39.181 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.181 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.181 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:44:39.181 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.586800+0000", 2020-09-08T15:44:39.181 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010925416 2020-09-08T15:44:39.182 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.182 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.182 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:39.182 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.597726+0000", 2020-09-08T15:44:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.011842284999999999 2020-09-08T15:44:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.609568+0000", 2020-09-08T15:44:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013896431000000001 2020-09-08T15:44:39.184 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.184 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.184 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.184 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.623465+0000", 2020-09-08T15:44:39.184 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5380000000000001e-05 2020-09-08T15:44:39.185 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.185 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.185 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:44:39.185 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.623530+0000", 2020-09-08T15:44:39.185 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001610956 2020-09-08T15:44:39.186 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.186 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.186 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:44:39.186 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.625141+0000", 2020-09-08T15:44:39.186 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.77707152300000004 2020-09-08T15:44:39.187 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.187 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:39.187 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.187 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.187 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.188 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88185 3.c 3:32cc51c5:::benchmark_data_smithi166_26097_object88184:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:44:39.188 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.848234+0000", 2020-09-08T15:44:39.188 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 94.110179187, 2020-09-08T15:44:39.188 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.78053202799999999, 2020-09-08T15:44:39.188 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:39.188 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:44:39.189 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:39.189 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:39.189 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:39.189 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88185 2020-09-08T15:44:39.189 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.190 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:39.190 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.190 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:39.190 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848234+0000", 2020-09-08T15:44:39.190 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:39.191 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.191 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.196 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:39.196 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848234+0000", 2020-09-08T15:44:39.196 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999728 2020-09-08T15:44:39.196 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.197 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.197 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:39.197 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848207+0000", 2020-09-08T15:44:39.197 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010272000000000001 2020-09-08T15:44:39.197 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.198 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.198 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:39.198 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848310+0000", 2020-09-08T15:44:39.198 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0137e-05 2020-09-08T15:44:39.198 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848320+0000", 2020-09-08T15:44:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.535e-06 2020-09-08T15:44:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.200 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.200 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:39.200 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848324+0000", 2020-09-08T15:44:39.201 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5059999999999999e-05 2020-09-08T15:44:39.201 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.201 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.201 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.201 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848369+0000", 2020-09-08T15:44:39.201 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.044e-05 2020-09-08T15:44:39.202 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.202 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.202 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:44:39.202 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848379+0000", 2020-09-08T15:44:39.202 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.74076494900000001 2020-09-08T15:44:39.203 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.203 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.203 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.203 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.589144+0000", 2020-09-08T15:44:39.203 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000137644 2020-09-08T15:44:39.204 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.204 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.204 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:39.204 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.589282+0000", 2020-09-08T15:44:39.204 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014025999999999999 2020-09-08T15:44:39.205 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.205 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.205 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:44:39.205 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.589422+0000", 2020-09-08T15:44:39.205 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0083156619999999997 2020-09-08T15:44:39.206 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.206 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.206 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:39.206 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.597738+0000", 2020-09-08T15:44:39.206 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.027475880000000001 2020-09-08T15:44:39.207 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.207 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.207 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.207 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.625213+0000", 2020-09-08T15:44:39.207 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0021174570000000001 2020-09-08T15:44:39.207 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.208 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.208 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.208 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.627331+0000", 2020-09-08T15:44:39.208 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2708999999999999e-05 2020-09-08T15:44:39.208 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.209 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.209 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:44:39.209 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.627394+0000", 2020-09-08T15:44:39.209 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001372559 2020-09-08T15:44:39.209 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.210 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.210 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:44:39.210 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.628766+0000", 2020-09-08T15:44:39.211 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.78053202799999999 2020-09-08T15:44:39.211 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.211 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:39.211 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.211 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.212 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.212 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88206 3.c 3:3799269c:::benchmark_data_smithi166_26097_object88205:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:44:39.212 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.848395+0000", 2020-09-08T15:44:39.212 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 94.110018648999997, 2020-09-08T15:44:39.212 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.79597148600000001, 2020-09-08T15:44:39.212 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:39.213 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:44:39.213 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:39.213 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:39.213 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:39.213 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88206 2020-09-08T15:44:39.214 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.214 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:39.214 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.214 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:39.214 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848395+0000", 2020-09-08T15:44:39.215 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:39.215 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.215 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.215 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:39.215 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848395+0000", 2020-09-08T15:44:39.216 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999628 2020-09-08T15:44:39.216 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.216 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.216 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:39.216 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848357+0000", 2020-09-08T15:44:39.216 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000106314 2020-09-08T15:44:39.217 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.217 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.217 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:39.217 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848464+0000", 2020-09-08T15:44:39.217 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0295000000000001e-05 2020-09-08T15:44:39.218 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.218 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.218 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:39.218 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848474+0000", 2020-09-08T15:44:39.218 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.568e-06 2020-09-08T15:44:39.219 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.219 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.219 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:39.219 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848478+0000", 2020-09-08T15:44:39.219 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6579999999999997e-05 2020-09-08T15:44:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848534+0000", 2020-09-08T15:44:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2971e-05 2020-09-08T15:44:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.221 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.221 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:44:39.221 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848547+0000", 2020-09-08T15:44:39.221 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.74238484699999996 2020-09-08T15:44:39.221 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.222 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.222 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.222 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.590932+0000", 2020-09-08T15:44:39.223 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000127992 2020-09-08T15:44:39.223 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.223 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.223 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:39.223 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.591060+0000", 2020-09-08T15:44:39.223 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000138567 2020-09-08T15:44:39.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.224 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.224 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:44:39.224 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.591199+0000", 2020-09-08T15:44:39.224 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010183153 2020-09-08T15:44:39.225 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.225 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.225 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:39.225 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.601382+0000", 2020-09-08T15:44:39.225 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.023873466999999999 2020-09-08T15:44:39.226 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.226 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.226 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.226 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.625255+0000", 2020-09-08T15:44:39.226 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.015987699000000001 2020-09-08T15:44:39.227 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.227 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.227 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.227 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.641243+0000", 2020-09-08T15:44:39.227 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012196700000000001 2020-09-08T15:44:39.228 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.228 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.228 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:44:39.228 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.641365+0000", 2020-09-08T15:44:39.228 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0030013349999999999 2020-09-08T15:44:39.228 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.229 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.229 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:44:39.229 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.644366+0000", 2020-09-08T15:44:39.229 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.79597148600000001 2020-09-08T15:44:39.229 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.230 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:39.230 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.230 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.230 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.230 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88213 3.c 3:3d413ed8:::benchmark_data_smithi166_26097_object88212:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:44:39.231 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.848546+0000", 2020-09-08T15:44:39.231 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 94.109867796000003, 2020-09-08T15:44:39.231 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.79807563500000001, 2020-09-08T15:44:39.231 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:39.231 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:44:39.232 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:39.232 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:39.232 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:39.232 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88213 2020-09-08T15:44:39.232 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.232 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:39.233 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.233 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:39.233 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848546+0000", 2020-09-08T15:44:39.233 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:39.233 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.234 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.234 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:39.234 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848546+0000", 2020-09-08T15:44:39.234 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999743 2020-09-08T15:44:39.234 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.235 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.235 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:39.235 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848520+0000", 2020-09-08T15:44:39.235 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1861999999999996e-05 2020-09-08T15:44:39.235 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.236 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.236 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:39.236 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848612+0000", 2020-09-08T15:44:39.236 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0196e-05 2020-09-08T15:44:39.236 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.237 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.237 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:39.237 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848622+0000", 2020-09-08T15:44:39.237 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.49e-06 2020-09-08T15:44:39.237 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.238 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.238 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:39.238 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848625+0000", 2020-09-08T15:44:39.238 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0156e-05 2020-09-08T15:44:39.238 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.239 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.239 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.239 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848666+0000", 2020-09-08T15:44:39.239 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0519e-05 2020-09-08T15:44:39.239 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.239 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.240 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:44:39.240 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848676+0000", 2020-09-08T15:44:39.240 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.74410123500000003 2020-09-08T15:44:39.240 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.240 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.241 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.241 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.592777+0000", 2020-09-08T15:44:39.241 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013514399999999999 2020-09-08T15:44:39.241 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.241 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.242 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:39.242 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.592912+0000", 2020-09-08T15:44:39.242 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014323800000000001 2020-09-08T15:44:39.243 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.243 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.243 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:44:39.243 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.593056+0000", 2020-09-08T15:44:39.243 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0083361330000000008 2020-09-08T15:44:39.243 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.244 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.244 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:39.244 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.601392+0000", 2020-09-08T15:44:39.244 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.030220244 2020-09-08T15:44:39.244 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.245 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.245 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.245 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.631612+0000", 2020-09-08T15:44:39.246 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.012851883 2020-09-08T15:44:39.246 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.246 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.246 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.246 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.644464+0000", 2020-09-08T15:44:39.247 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012559800000000001 2020-09-08T15:44:39.247 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.247 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.247 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:44:39.247 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.644589+0000", 2020-09-08T15:44:39.248 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020316470000000001 2020-09-08T15:44:39.248 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.248 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.248 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:44:39.248 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.646621+0000", 2020-09-08T15:44:39.249 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.79807563500000001 2020-09-08T15:44:39.249 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.249 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:39.249 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.249 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.249 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.250 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88217 3.c 3:30993993:::benchmark_data_smithi166_26097_object88216:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:44:39.250 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.848691+0000", 2020-09-08T15:44:39.250 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 94.109721918000005, 2020-09-08T15:44:39.250 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.80980842799999997, 2020-09-08T15:44:39.250 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:39.251 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:44:39.251 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:39.251 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:39.251 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:39.251 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88217 2020-09-08T15:44:39.252 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.252 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:39.252 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.252 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:39.252 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848691+0000", 2020-09-08T15:44:39.253 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:39.253 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.253 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.253 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:39.253 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848691+0000", 2020-09-08T15:44:39.254 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999642 2020-09-08T15:44:39.254 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.254 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.255 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:39.255 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848656+0000", 2020-09-08T15:44:39.257 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010443 2020-09-08T15:44:39.257 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.257 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.258 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:39.258 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848760+0000", 2020-09-08T15:44:39.258 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0158e-05 2020-09-08T15:44:39.258 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.258 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.259 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:39.259 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848770+0000", 2020-09-08T15:44:39.259 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3409999999999999e-06 2020-09-08T15:44:39.259 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.259 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.260 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:39.260 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848774+0000", 2020-09-08T15:44:39.260 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5544000000000003e-05 2020-09-08T15:44:39.260 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.260 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.261 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.261 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848819+0000", 2020-09-08T15:44:39.261 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2342e-05 2020-09-08T15:44:39.261 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.261 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.262 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:44:39.262 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848832+0000", 2020-09-08T15:44:39.262 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.746899388 2020-09-08T15:44:39.263 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.263 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.263 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.263 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.595731+0000", 2020-09-08T15:44:39.263 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000158994 2020-09-08T15:44:39.264 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.264 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.264 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:39.264 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.595890+0000", 2020-09-08T15:44:39.264 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000178278 2020-09-08T15:44:39.265 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.265 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.265 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:44:39.265 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.596068+0000", 2020-09-08T15:44:39.265 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0077743609999999996 2020-09-08T15:44:39.266 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.266 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.266 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:39.266 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.603843+0000", 2020-09-08T15:44:39.266 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.027843777 2020-09-08T15:44:39.267 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.267 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.267 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.267 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.631686+0000", 2020-09-08T15:44:39.267 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.025082657000000001 2020-09-08T15:44:39.268 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.268 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.268 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.268 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.656769+0000", 2020-09-08T15:44:39.268 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011135 2020-09-08T15:44:39.268 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.269 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.269 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:44:39.269 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.656880+0000", 2020-09-08T15:44:39.269 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0016193970000000001 2020-09-08T15:44:39.270 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.270 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.270 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:44:39.271 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.658500+0000", 2020-09-08T15:44:39.271 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.80980842799999997 2020-09-08T15:44:39.271 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.271 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:39.271 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.272 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.272 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.272 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88229 3.c 3:3fa93278:::benchmark_data_smithi166_26097_object88228:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:44:39.272 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.848825+0000", 2020-09-08T15:44:39.272 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 94.109588013000007, 2020-09-08T15:44:39.272 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.81162889800000004, 2020-09-08T15:44:39.273 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:39.273 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:44:39.273 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:39.273 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:39.274 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:39.274 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88229 2020-09-08T15:44:39.276 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.276 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:39.276 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.277 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:39.277 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848825+0000", 2020-09-08T15:44:39.277 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:39.277 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.278 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.278 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:39.278 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848825+0000", 2020-09-08T15:44:39.278 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999728 2020-09-08T15:44:39.279 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.279 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.279 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:39.279 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848798+0000", 2020-09-08T15:44:39.279 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5902000000000004e-05 2020-09-08T15:44:39.279 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.280 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.280 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:39.280 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848894+0000", 2020-09-08T15:44:39.280 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9150000000000008e-06 2020-09-08T15:44:39.280 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.281 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.281 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:39.281 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848904+0000", 2020-09-08T15:44:39.281 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1769999999999998e-06 2020-09-08T15:44:39.281 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.282 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.282 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:39.282 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848907+0000", 2020-09-08T15:44:39.282 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8699999999999999e-05 2020-09-08T15:44:39.282 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.283 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.283 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.283 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848946+0000", 2020-09-08T15:44:39.283 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9720000000000004e-06 2020-09-08T15:44:39.284 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.284 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.284 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:44:39.284 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848956+0000", 2020-09-08T15:44:39.285 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.75253392399999997 2020-09-08T15:44:39.285 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.285 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.285 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.285 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.601490+0000", 2020-09-08T15:44:39.286 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014808099999999999 2020-09-08T15:44:39.286 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.286 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.286 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:39.286 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.601638+0000", 2020-09-08T15:44:39.287 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017448400000000001 2020-09-08T15:44:39.287 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.287 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.287 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:44:39.287 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.601812+0000", 2020-09-08T15:44:39.287 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0077719160000000002 2020-09-08T15:44:39.288 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.288 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.288 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:39.288 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.609584+0000", 2020-09-08T15:44:39.288 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.027487385 2020-09-08T15:44:39.289 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.289 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.289 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.289 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.637072+0000", 2020-09-08T15:44:39.289 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.021532775000000001 2020-09-08T15:44:39.290 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.290 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.291 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.291 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.658604+0000", 2020-09-08T15:44:39.291 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000112063 2020-09-08T15:44:39.291 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.291 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.292 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:44:39.292 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.658716+0000", 2020-09-08T15:44:39.292 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001737717 2020-09-08T15:44:39.292 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.292 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.293 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:44:39.293 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.660454+0000", 2020-09-08T15:44:39.293 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.81162889800000004 2020-09-08T15:44:39.293 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.293 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:39.294 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.294 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.294 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.294 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88244 3.c 3:32754c75:::benchmark_data_smithi166_26097_object88243:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:44:39.294 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.848971+0000", 2020-09-08T15:44:39.295 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 94.109442450000003, 2020-09-08T15:44:39.295 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83018217599999999, 2020-09-08T15:44:39.295 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:39.295 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:44:39.295 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:39.296 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:39.296 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:39.296 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88244 2020-09-08T15:44:39.296 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.296 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:39.297 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.297 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:39.297 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848971+0000", 2020-09-08T15:44:39.298 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:39.298 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.298 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.298 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:39.298 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848971+0000", 2020-09-08T15:44:39.299 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999638 2020-09-08T15:44:39.299 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.299 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.299 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:39.299 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848935+0000", 2020-09-08T15:44:39.300 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000103946 2020-09-08T15:44:39.300 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.300 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.300 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:39.300 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849039+0000", 2020-09-08T15:44:39.301 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3365e-05 2020-09-08T15:44:39.301 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.301 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.301 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:39.301 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849052+0000", 2020-09-08T15:44:39.303 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4489999999999999e-06 2020-09-08T15:44:39.304 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.304 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.304 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:39.304 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849056+0000", 2020-09-08T15:44:39.304 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5677999999999999e-05 2020-09-08T15:44:39.305 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.305 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.305 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.305 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849101+0000", 2020-09-08T15:44:39.305 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2026e-05 2020-09-08T15:44:39.306 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.306 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.306 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:44:39.306 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849113+0000", 2020-09-08T15:44:39.306 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.754773735 2020-09-08T15:44:39.307 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.307 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.308 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.308 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.603887+0000", 2020-09-08T15:44:39.308 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016359499999999999 2020-09-08T15:44:39.308 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.308 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.309 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:39.309 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.604051+0000", 2020-09-08T15:44:39.309 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000182654 2020-09-08T15:44:39.309 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.309 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.310 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:44:39.310 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.604233+0000", 2020-09-08T15:44:39.310 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0053636939999999996 2020-09-08T15:44:39.310 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.310 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.311 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:39.311 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.609597+0000", 2020-09-08T15:44:39.311 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.02768924 2020-09-08T15:44:39.312 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.312 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.312 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.312 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.637286+0000", 2020-09-08T15:44:39.313 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.039558932999999998 2020-09-08T15:44:39.313 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.313 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.313 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.313 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.676845+0000", 2020-09-08T15:44:39.314 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8937000000000006e-05 2020-09-08T15:44:39.314 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.314 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.314 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:44:39.315 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.676934+0000", 2020-09-08T15:44:39.315 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0022190230000000001 2020-09-08T15:44:39.315 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.315 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.316 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:44:39.316 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.679153+0000", 2020-09-08T15:44:39.316 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83018217599999999 2020-09-08T15:44:39.317 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.317 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:39.317 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.317 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.318 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.318 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88252 3.c 3:3ca3108e:::benchmark_data_smithi166_26097_object88251:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:44:39.318 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.849107+0000", 2020-09-08T15:44:39.319 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 94.109306466000007, 2020-09-08T15:44:39.319 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83242581000000004, 2020-09-08T15:44:39.319 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:39.319 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:44:39.319 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:39.320 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:39.320 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:39.320 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88252 2020-09-08T15:44:39.320 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.320 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:39.321 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.321 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:39.321 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849107+0000", 2020-09-08T15:44:39.321 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:39.321 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.322 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.322 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:39.322 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849107+0000", 2020-09-08T15:44:39.322 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999733 2020-09-08T15:44:39.322 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.322 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.323 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:39.323 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849080+0000", 2020-09-08T15:44:39.323 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3986000000000004e-05 2020-09-08T15:44:39.323 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.323 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.324 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:39.324 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849174+0000", 2020-09-08T15:44:39.324 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7790000000000007e-06 2020-09-08T15:44:39.324 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.324 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.325 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:39.325 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849184+0000", 2020-09-08T15:44:39.325 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.14e-06 2020-09-08T15:44:39.325 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.325 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.326 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:39.326 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849187+0000", 2020-09-08T15:44:39.326 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7882000000000001e-05 2020-09-08T15:44:39.326 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.326 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.326 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.327 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849225+0000", 2020-09-08T15:44:39.327 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0234999999999999e-05 2020-09-08T15:44:39.327 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.327 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.327 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:44:39.328 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849235+0000", 2020-09-08T15:44:39.328 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.75657893499999995 2020-09-08T15:44:39.328 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.328 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.328 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.329 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.605814+0000", 2020-09-08T15:44:39.329 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015115199999999999 2020-09-08T15:44:39.329 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.329 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.329 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:39.330 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.605965+0000", 2020-09-08T15:44:39.330 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000181583 2020-09-08T15:44:39.330 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.330 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.330 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:44:39.331 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.606147+0000", 2020-09-08T15:44:39.331 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0086315590000000005 2020-09-08T15:44:39.331 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.331 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.331 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:39.332 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.614778+0000", 2020-09-08T15:44:39.332 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.031948953000000002 2020-09-08T15:44:39.332 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.332 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.332 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.333 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.646727+0000", 2020-09-08T15:44:39.333 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.032630385999999997 2020-09-08T15:44:39.333 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.334 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.334 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.334 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.679358+0000", 2020-09-08T15:44:39.334 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7763000000000006e-05 2020-09-08T15:44:39.336 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.336 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.336 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:44:39.336 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.679455+0000", 2020-09-08T15:44:39.337 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020773549999999999 2020-09-08T15:44:39.337 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.337 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.337 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:44:39.338 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.681533+0000", 2020-09-08T15:44:39.338 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83242581000000004 2020-09-08T15:44:39.338 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.338 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:39.338 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.339 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.339 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.339 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88264 3.c 3:3a7639d5:::benchmark_data_smithi166_26097_object88263:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:44:39.340 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.849254+0000", 2020-09-08T15:44:39.340 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 94.109159661000007, 2020-09-08T15:44:39.340 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83457435899999999, 2020-09-08T15:44:39.340 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:39.341 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:44:39.341 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:39.341 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:39.341 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:39.341 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88264 2020-09-08T15:44:39.342 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.342 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:39.342 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.342 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:39.342 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849254+0000", 2020-09-08T15:44:39.343 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:39.343 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.343 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.345 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:39.346 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849254+0000", 2020-09-08T15:44:39.346 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999599 2020-09-08T15:44:39.346 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.346 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.347 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:39.347 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849214+0000", 2020-09-08T15:44:39.347 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000108477 2020-09-08T15:44:39.347 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.347 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.347 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:39.348 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849322+0000", 2020-09-08T15:44:39.348 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0156999999999999e-05 2020-09-08T15:44:39.348 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.348 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.348 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:39.349 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849332+0000", 2020-09-08T15:44:39.349 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2880000000000001e-06 2020-09-08T15:44:39.349 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.349 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.349 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:39.350 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849336+0000", 2020-09-08T15:44:39.350 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5487e-05 2020-09-08T15:44:39.351 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.351 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.351 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.351 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849381+0000", 2020-09-08T15:44:39.351 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2632e-05 2020-09-08T15:44:39.352 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.352 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.352 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:44:39.352 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849394+0000", 2020-09-08T15:44:39.358 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.75832558800000005 2020-09-08T15:44:39.358 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.358 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.359 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.359 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.607719+0000", 2020-09-08T15:44:39.359 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015323299999999999 2020-09-08T15:44:39.359 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.359 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.360 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:39.360 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.607872+0000", 2020-09-08T15:44:39.360 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017337799999999999 2020-09-08T15:44:39.360 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.360 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.361 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:44:39.361 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.608046+0000", 2020-09-08T15:44:39.361 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0095075239999999998 2020-09-08T15:44:39.361 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.361 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.362 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:39.362 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.617553+0000", 2020-09-08T15:44:39.362 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.036108202999999998 2020-09-08T15:44:39.362 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.362 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.363 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.363 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.653662+0000", 2020-09-08T15:44:39.363 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.028012091999999999 2020-09-08T15:44:39.363 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.363 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.363 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.364 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.681674+0000", 2020-09-08T15:44:39.364 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5779999999999995e-05 2020-09-08T15:44:39.364 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.364 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.374 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:44:39.374 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.681759+0000", 2020-09-08T15:44:39.374 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020685320000000001 2020-09-08T15:44:39.375 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.375 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.375 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:44:39.375 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.683828+0000", 2020-09-08T15:44:39.375 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83457435899999999 2020-09-08T15:44:39.376 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.376 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:39.376 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.376 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.376 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.378 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88277 3.c 3:3b3cc411:::benchmark_data_smithi166_26097_object88276:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:44:39.378 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.849387+0000", 2020-09-08T15:44:39.378 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 94.109026150000005, 2020-09-08T15:44:39.384 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83678120199999995, 2020-09-08T15:44:39.384 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:39.385 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:44:39.385 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:39.385 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:39.385 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:39.386 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88277 2020-09-08T15:44:39.386 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.386 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:39.386 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.386 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:39.387 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849387+0000", 2020-09-08T15:44:39.387 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:39.387 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.387 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.387 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:39.388 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849387+0000", 2020-09-08T15:44:39.388 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999719 2020-09-08T15:44:39.388 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.388 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.388 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:39.388 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849359+0000", 2020-09-08T15:44:39.389 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3432999999999998e-05 2020-09-08T15:44:39.389 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.389 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.389 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:39.397 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849453+0000", 2020-09-08T15:44:39.398 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8660000000000001e-06 2020-09-08T15:44:39.398 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.398 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.398 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:39.398 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849462+0000", 2020-09-08T15:44:39.399 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.287e-06 2020-09-08T15:44:39.399 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.399 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.399 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:39.399 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849466+0000", 2020-09-08T15:44:39.400 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8856000000000001e-05 2020-09-08T15:44:39.400 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.400 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.400 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.400 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849515+0000", 2020-09-08T15:44:39.401 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0053e-05 2020-09-08T15:44:39.401 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.401 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.401 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:44:39.407 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849525+0000", 2020-09-08T15:44:39.408 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.76201480799999999 2020-09-08T15:44:39.408 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.408 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.408 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.408 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.611539+0000", 2020-09-08T15:44:39.409 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00026357000000000003 2020-09-08T15:44:39.409 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.409 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.409 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:39.409 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.611803+0000", 2020-09-08T15:44:39.410 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00029939000000000002 2020-09-08T15:44:39.410 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.410 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.410 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:44:39.410 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.612102+0000", 2020-09-08T15:44:39.411 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013058955000000001 2020-09-08T15:44:39.413 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.414 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.414 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:39.414 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.625161+0000", 2020-09-08T15:44:39.414 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.051603609000000002 2020-09-08T15:44:39.415 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.415 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.415 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.415 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.676765+0000", 2020-09-08T15:44:39.415 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0073178669999999996 2020-09-08T15:44:39.418 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.418 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.419 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.419 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.684083+0000", 2020-09-08T15:44:39.419 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6081000000000005e-05 2020-09-08T15:44:39.419 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.419 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.420 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:44:39.420 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.684169+0000", 2020-09-08T15:44:39.420 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0019995009999999999 2020-09-08T15:44:39.420 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.420 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.421 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:44:39.421 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.686168+0000", 2020-09-08T15:44:39.421 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83678120199999995 2020-09-08T15:44:39.421 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.421 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:39.426 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.426 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.426 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.426 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88294 3.c 3:3f7cd38d:::benchmark_data_smithi166_26097_object88293:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:44:39.427 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.849540+0000", 2020-09-08T15:44:39.427 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 94.108873799999998, 2020-09-08T15:44:39.427 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.84994455800000002, 2020-09-08T15:44:39.427 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:39.427 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:44:39.428 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:39.428 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:39.428 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:39.428 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88294 2020-09-08T15:44:39.428 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.429 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:39.429 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.429 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:39.429 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849540+0000", 2020-09-08T15:44:39.429 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:39.430 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.430 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.430 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:39.431 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849540+0000", 2020-09-08T15:44:39.431 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999614 2020-09-08T15:44:39.431 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.431 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.431 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:39.432 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849501+0000", 2020-09-08T15:44:39.432 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000103999 2020-09-08T15:44:39.432 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.432 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.432 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:39.433 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849605+0000", 2020-09-08T15:44:39.433 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0275000000000001e-05 2020-09-08T15:44:39.433 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.433 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.433 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:39.435 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849615+0000", 2020-09-08T15:44:39.435 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5099999999999999e-06 2020-09-08T15:44:39.436 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.436 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.436 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:39.436 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849619+0000", 2020-09-08T15:44:39.436 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3949999999999998e-05 2020-09-08T15:44:39.437 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.437 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.437 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.437 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849663+0000", 2020-09-08T15:44:39.437 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2666999999999999e-05 2020-09-08T15:44:39.446 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.446 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.447 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:44:39.448 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849675+0000", 2020-09-08T15:44:39.448 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.76514351999999997 2020-09-08T15:44:39.448 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.448 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.449 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.449 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.614819+0000", 2020-09-08T15:44:39.449 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00022547600000000001 2020-09-08T15:44:39.449 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.450 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.450 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:39.450 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.615044+0000", 2020-09-08T15:44:39.450 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00023015000000000001 2020-09-08T15:44:39.450 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.451 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.451 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:44:39.454 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.615275+0000", 2020-09-08T15:44:39.454 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0099885300000000007 2020-09-08T15:44:39.454 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.455 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.455 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:39.455 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.625263+0000", 2020-09-08T15:44:39.455 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.053987439999999998 2020-09-08T15:44:39.455 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.456 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.456 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.456 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.679251+0000", 2020-09-08T15:44:39.456 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.017811098000000001 2020-09-08T15:44:39.456 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.457 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.457 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.457 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.697062+0000", 2020-09-08T15:44:39.457 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3387999999999995e-05 2020-09-08T15:44:39.457 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.470 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.470 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:44:39.470 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.697145+0000", 2020-09-08T15:44:39.471 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0023390310000000001 2020-09-08T15:44:39.471 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.471 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.471 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:44:39.471 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.699484+0000", 2020-09-08T15:44:39.472 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.84994455800000002 2020-09-08T15:44:39.472 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.472 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:39.472 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.472 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.473 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.473 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88298 3.c 3:3f320a61:::benchmark_data_smithi166_26097_object88297:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:44:39.473 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.849669+0000", 2020-09-08T15:44:39.473 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 94.108744212999994, 2020-09-08T15:44:39.473 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.85250447500000004, 2020-09-08T15:44:39.474 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:44:39.474 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:44:39.474 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:44:39.474 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:44:39.474 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:44:39.475 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88298 2020-09-08T15:44:39.475 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.475 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:44:39.475 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.475 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:44:39.476 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849669+0000", 2020-09-08T15:44:39.476 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:44:39.479 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.480 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.480 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:44:39.480 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849669+0000", 2020-09-08T15:44:39.480 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999728 2020-09-08T15:44:39.480 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.481 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.481 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:44:39.481 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849642+0000", 2020-09-08T15:44:39.481 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000112197 2020-09-08T15:44:39.486 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.486 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.487 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:44:39.487 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849754+0000", 2020-09-08T15:44:39.487 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7410000000000004e-06 2020-09-08T15:44:39.487 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.487 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.488 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:44:39.488 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849764+0000", 2020-09-08T15:44:39.488 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6109999999999998e-06 2020-09-08T15:44:39.491 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.491 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.491 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:44:39.492 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849768+0000", 2020-09-08T15:44:39.492 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8702000000000003e-05 2020-09-08T15:44:39.492 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.492 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.492 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.493 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849806+0000", 2020-09-08T15:44:39.493 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5669999999999995e-06 2020-09-08T15:44:39.493 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.493 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.493 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:44:39.494 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849815+0000", 2020-09-08T15:44:39.494 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.76780925 2020-09-08T15:44:39.494 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.494 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.494 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:44:39.495 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.617624+0000", 2020-09-08T15:44:39.496 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000225837 2020-09-08T15:44:39.496 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.496 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.496 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:44:39.497 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.617850+0000", 2020-09-08T15:44:39.498 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00021239699999999999 2020-09-08T15:44:39.498 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.498 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.498 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:44:39.499 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.618062+0000", 2020-09-08T15:44:39.499 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0072071269999999998 2020-09-08T15:44:39.499 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.499 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.499 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:44:39.500 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.625270+0000", 2020-09-08T15:44:39.500 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.056336343999999997 2020-09-08T15:44:39.500 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.500 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.501 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.501 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.681606+0000", 2020-09-08T15:44:39.501 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.017974381000000001 2020-09-08T15:44:39.501 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.502 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.502 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:44:39.502 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.699580+0000", 2020-09-08T15:44:39.502 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8785999999999999e-05 2020-09-08T15:44:39.502 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.503 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.503 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:44:39.503 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.699679+0000", 2020-09-08T15:44:39.503 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0024944770000000002 2020-09-08T15:44:39.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:44:39.504 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:44:39.504 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:44:39.505 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.702174+0000", 2020-09-08T15:44:39.505 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.85250447500000004 2020-09-08T15:44:39.506 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.506 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:39.506 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.506 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:44:39.507 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:44:39.507 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:44:39.508 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:38 smithi055 bash[9296]: cluster 2020-09-08T15:44:37.668259+0000 mon.a (mon.0) 1215 : cluster [DBG] osdmap e264: 12 total, 12 up, 12 in 2020-09-08T15:44:39.508 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:38 smithi166 bash[9446]: cluster 2020-09-08T15:44:37.668259+0000 mon.a (mon.0) 1215 : cluster [DBG] osdmap e264: 12 total, 12 up, 12 in 2020-09-08T15:44:39.670 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:39 smithi104 bash[9203]: cluster 2020-09-08T15:44:37.703057+0000 mgr.x (mgr.24118) 519 : cluster [DBG] pgmap v699: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 664 op/s; 20992/188904 objects misplaced (11.113%); 8.7 MiB/s, 139 objects/s recovering 2020-09-08T15:44:39.868 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:44:39.971 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:44:39.983 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:39 smithi055 bash[9296]: cluster 2020-09-08T15:44:37.703057+0000 mgr.x (mgr.24118) 519 : cluster [DBG] pgmap v699: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 664 op/s; 20992/188904 objects misplaced (11.113%); 8.7 MiB/s, 139 objects/s recovering 2020-09-08T15:44:40.105 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:44:40.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:39 smithi166 bash[9446]: cluster 2020-09-08T15:44:37.703057+0000 mgr.x (mgr.24118) 519 : cluster [DBG] pgmap v699: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 664 op/s; 20992/188904 objects misplaced (11.113%); 8.7 MiB/s, 139 objects/s recovering 2020-09-08T15:44:40.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:44:39 smithi166 bash[16593]: debug 2020-09-08T15:44:39.872+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:40.216 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:44:40.410 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:44:40 smithi055 bash[14301]: debug 2020-09-08T15:44:40.106+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:40.436 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:44:40.625 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:44:41.402 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:44:41.508 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:44:41.674 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:44:41 smithi104 bash[15273]: debug 2020-09-08T15:44:41.402+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:41.865 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:41 smithi166 bash[9446]: cluster 2020-09-08T15:44:39.703552+0000 mgr.x (mgr.24118) 520 : cluster [DBG] pgmap v700: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 694 op/s; 20834/187040 objects misplaced (11.139%); 8.9 MiB/s, 141 objects/s recovering 2020-09-08T15:44:41.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:41 smithi104 bash[9203]: cluster 2020-09-08T15:44:39.703552+0000 mgr.x (mgr.24118) 520 : cluster [DBG] pgmap v700: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 694 op/s; 20834/187040 objects misplaced (11.139%); 8.9 MiB/s, 141 objects/s recovering 2020-09-08T15:44:42.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:41 smithi055 bash[9296]: cluster 2020-09-08T15:44:39.703552+0000 mgr.x (mgr.24118) 520 : cluster [DBG] pgmap v700: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 694 op/s; 20834/187040 objects misplaced (11.139%); 8.9 MiB/s, 141 objects/s recovering 2020-09-08T15:44:42.786 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2020-09-08T15:44:42.922 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:44:43.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:42 smithi166 bash[9446]: cluster 2020-09-08T15:44:41.695526+0000 osd.6 (osd.6) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:43.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:42 smithi166 bash[9446]: cluster 2020-09-08T15:44:41.695662+0000 osd.6 (osd.6) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:43.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:42 smithi055 bash[9296]: cluster 2020-09-08T15:44:41.695526+0000 osd.6 (osd.6) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:43.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:42 smithi055 bash[9296]: cluster 2020-09-08T15:44:41.695662+0000 osd.6 (osd.6) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:42 smithi104 bash[9203]: cluster 2020-09-08T15:44:41.695526+0000 osd.6 (osd.6) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:42 smithi104 bash[9203]: cluster 2020-09-08T15:44:41.695662+0000 osd.6 (osd.6) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:43.812 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:44:43.978 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:43 smithi104 bash[9203]: cluster 2020-09-08T15:44:41.704245+0000 mgr.x (mgr.24118) 521 : cluster [DBG] pgmap v701: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 618 op/s; 20634/185886 objects misplaced (11.100%); 8.0 MiB/s, 127 objects/s recovering 2020-09-08T15:44:43.978 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:43 smithi104 bash[9203]: cluster 2020-09-08T15:44:42.689138+0000 mon.a (mon.0) 1216 : cluster [DBG] osdmap e265: 12 total, 12 up, 12 in 2020-09-08T15:44:44.082 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:44.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:43 smithi166 bash[9446]: cluster 2020-09-08T15:44:41.704245+0000 mgr.x (mgr.24118) 521 : cluster [DBG] pgmap v701: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 618 op/s; 20634/185886 objects misplaced (11.100%); 8.0 MiB/s, 127 objects/s recovering 2020-09-08T15:44:44.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:43 smithi166 bash[9446]: cluster 2020-09-08T15:44:42.689138+0000 mon.a (mon.0) 1216 : cluster [DBG] osdmap e265: 12 total, 12 up, 12 in 2020-09-08T15:44:44.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:44:43 smithi166 bash[16593]: debug 2020-09-08T15:44:43.815+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:44.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:43 smithi055 bash[9296]: cluster 2020-09-08T15:44:41.704245+0000 mgr.x (mgr.24118) 521 : cluster [DBG] pgmap v701: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 618 op/s; 20634/185886 objects misplaced (11.100%); 8.0 MiB/s, 127 objects/s recovering 2020-09-08T15:44:44.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:43 smithi055 bash[9296]: cluster 2020-09-08T15:44:42.689138+0000 mon.a (mon.0) 1216 : cluster [DBG] osdmap e265: 12 total, 12 up, 12 in 2020-09-08T15:44:44.233 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:44.336 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:44:44.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:44:44 smithi104 bash[22305]: debug 2020-09-08T15:44:44.210+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:45.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:44 smithi166 bash[9446]: cluster 2020-09-08T15:44:43.704967+0000 mgr.x (mgr.24118) 522 : cluster [DBG] pgmap v703: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 703 op/s; 20052/181900 objects misplaced (11.024%); 10 MiB/s, 162 objects/s recovering 2020-09-08T15:44:45.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:44 smithi166 bash[9446]: cluster 2020-09-08T15:44:44.072064+0000 osd.11 (osd.11) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:45.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:44 smithi166 bash[9446]: cluster 2020-09-08T15:44:44.072159+0000 osd.11 (osd.11) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:45.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:44 smithi055 bash[9296]: cluster 2020-09-08T15:44:43.704967+0000 mgr.x (mgr.24118) 522 : cluster [DBG] pgmap v703: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 703 op/s; 20052/181900 objects misplaced (11.024%); 10 MiB/s, 162 objects/s recovering 2020-09-08T15:44:45.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:44 smithi055 bash[9296]: cluster 2020-09-08T15:44:44.072064+0000 osd.11 (osd.11) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:45.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:44 smithi055 bash[9296]: cluster 2020-09-08T15:44:44.072159+0000 osd.11 (osd.11) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:45.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:44 smithi104 bash[9203]: cluster 2020-09-08T15:44:43.704967+0000 mgr.x (mgr.24118) 522 : cluster [DBG] pgmap v703: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 703 op/s; 20052/181900 objects misplaced (11.024%); 10 MiB/s, 162 objects/s recovering 2020-09-08T15:44:45.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:44 smithi104 bash[9203]: cluster 2020-09-08T15:44:44.072064+0000 osd.11 (osd.11) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:45.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:44 smithi104 bash[9203]: cluster 2020-09-08T15:44:44.072159+0000 osd.11 (osd.11) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:45.711 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:44:46.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:45 smithi166 bash[9446]: cluster 2020-09-08T15:44:44.698479+0000 mon.a (mon.0) 1217 : cluster [DBG] osdmap e266: 12 total, 12 up, 12 in 2020-09-08T15:44:46.123 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:45 smithi166 bash[9446]: audit 2020-09-08T15:44:44.719459+0000 mon.a (mon.0) 1218 : audit [INF] from='client.? v1:172.21.15.104:0/2118047690' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:44:46.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:45 smithi055 bash[9296]: cluster 2020-09-08T15:44:44.698479+0000 mon.a (mon.0) 1217 : cluster [DBG] osdmap e266: 12 total, 12 up, 12 in 2020-09-08T15:44:46.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:45 smithi055 bash[9296]: audit 2020-09-08T15:44:44.719459+0000 mon.a (mon.0) 1218 : audit [INF] from='client.? v1:172.21.15.104:0/2118047690' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:44:46.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:45 smithi104 bash[9203]: cluster 2020-09-08T15:44:44.698479+0000 mon.a (mon.0) 1217 : cluster [DBG] osdmap e266: 12 total, 12 up, 12 in 2020-09-08T15:44:46.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:45 smithi104 bash[9203]: audit 2020-09-08T15:44:44.719459+0000 mon.a (mon.0) 1218 : audit [INF] from='client.? v1:172.21.15.104:0/2118047690' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:44:46.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:46 smithi104 bash[9203]: cluster 2020-09-08T15:44:45.695865+0000 mon.a (mon.0) 1219 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:44:46.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:46 smithi104 bash[9203]: cluster 2020-09-08T15:44:45.705415+0000 mgr.x (mgr.24118) 523 : cluster [DBG] pgmap v706: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 486 op/s; 20052/181200 objects misplaced (11.066%); 6.8 MiB/s, 108 objects/s recovering 2020-09-08T15:44:46.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:46 smithi104 bash[9203]: audit 2020-09-08T15:44:45.707378+0000 mon.a (mon.0) 1220 : audit [INF] from='client.? v1:172.21.15.104:0/2118047690' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:44:46.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:46 smithi104 bash[9203]: cluster 2020-09-08T15:44:45.707557+0000 mon.a (mon.0) 1221 : cluster [DBG] osdmap e267: 12 total, 12 up, 12 in 2020-09-08T15:44:47.002 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:46 smithi055 bash[9296]: cluster 2020-09-08T15:44:45.695865+0000 mon.a (mon.0) 1219 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:44:47.003 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:46 smithi055 bash[9296]: cluster 2020-09-08T15:44:45.705415+0000 mgr.x (mgr.24118) 523 : cluster [DBG] pgmap v706: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 486 op/s; 20052/181200 objects misplaced (11.066%); 6.8 MiB/s, 108 objects/s recovering 2020-09-08T15:44:47.003 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:46 smithi055 bash[9296]: audit 2020-09-08T15:44:45.707378+0000 mon.a (mon.0) 1220 : audit [INF] from='client.? v1:172.21.15.104:0/2118047690' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:44:47.003 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:46 smithi055 bash[9296]: cluster 2020-09-08T15:44:45.707557+0000 mon.a (mon.0) 1221 : cluster [DBG] osdmap e267: 12 total, 12 up, 12 in 2020-09-08T15:44:47.003 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:44:47.112 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:44:47.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:46 smithi166 bash[9446]: cluster 2020-09-08T15:44:45.695865+0000 mon.a (mon.0) 1219 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:44:47.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:46 smithi166 bash[9446]: cluster 2020-09-08T15:44:45.705415+0000 mgr.x (mgr.24118) 523 : cluster [DBG] pgmap v706: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 486 op/s; 20052/181200 objects misplaced (11.066%); 6.8 MiB/s, 108 objects/s recovering 2020-09-08T15:44:47.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:46 smithi166 bash[9446]: audit 2020-09-08T15:44:45.707378+0000 mon.a (mon.0) 1220 : audit [INF] from='client.? v1:172.21.15.104:0/2118047690' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:44:47.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:46 smithi166 bash[9446]: cluster 2020-09-08T15:44:45.707557+0000 mon.a (mon.0) 1221 : cluster [DBG] osdmap e267: 12 total, 12 up, 12 in 2020-09-08T15:44:47.255 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:44:47.361 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:44:47.410 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:44:47 smithi055 bash[16646]: debug 2020-09-08T15:44:47.002+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:47.543 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:44:47.615 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:44:47 smithi166 bash[18960]: debug 2020-09-08T15:44:47.251+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:47.640 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:44:47 smithi104 bash[15273]: debug 2020-09-08T15:44:47.542+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:47.666 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:44:47.938 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:44:47.982 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:44:48.024 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:44:48.057 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:44:48.086 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:44:48.122 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:44:48.158 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:44:48.193 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:44:48.240 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:44:48.280 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:44:48.291 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:44:48.317 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:44:48.357 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:44:48.360 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:44:48.360 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:44:48.361 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:44:48.361 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:44:48.361 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:44:48.362 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:44:48.362 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:44:48.362 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:44:48.362 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:44:48.363 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:44:48.363 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:44:48.363 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:44:48.394 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:44:48.544 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:44:48 smithi055 bash[11956]: debug 2020-09-08T15:44:48.294+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:48.544 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:44:48.649 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:48.908 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:44:48 smithi055 bash[16646]: debug 2020-09-08T15:44:48.546+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:48.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:48 smithi055 bash[9296]: cluster 2020-09-08T15:44:47.705904+0000 mgr.x (mgr.24118) 524 : cluster [DBG] pgmap v707: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 721 op/s; 19421/177226 objects misplaced (10.958%); 10 MiB/s, 160 objects/s recovering 2020-09-08T15:44:48.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:48 smithi055 bash[9296]: audit 2020-09-08T15:44:47.894184+0000 mon.a (mon.0) 1222 : audit [DBG] from='client.? v1:172.21.15.104:0/3988945733' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:44:49.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:48 smithi166 bash[9446]: cluster 2020-09-08T15:44:47.705904+0000 mgr.x (mgr.24118) 524 : cluster [DBG] pgmap v707: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 721 op/s; 19421/177226 objects misplaced (10.958%); 10 MiB/s, 160 objects/s recovering 2020-09-08T15:44:49.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:48 smithi166 bash[9446]: audit 2020-09-08T15:44:47.894184+0000 mon.a (mon.0) 1222 : audit [DBG] from='client.? v1:172.21.15.104:0/3988945733' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:44:49.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:48 smithi104 bash[9203]: cluster 2020-09-08T15:44:47.705904+0000 mgr.x (mgr.24118) 524 : cluster [DBG] pgmap v707: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 721 op/s; 19421/177226 objects misplaced (10.958%); 10 MiB/s, 160 objects/s recovering 2020-09-08T15:44:49.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:48 smithi104 bash[9203]: audit 2020-09-08T15:44:47.894184+0000 mon.a (mon.0) 1222 : audit [DBG] from='client.? v1:172.21.15.104:0/3988945733' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:44:49.466 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:44:49.634 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:44:49.738 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:49 smithi104 bash[9203]: cluster 2020-09-08T15:44:48.172208+0000 osd.4 (osd.4) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:49.739 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:49 smithi104 bash[9203]: cluster 2020-09-08T15:44:48.172293+0000 osd.4 (osd.4) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:49.739 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:49 smithi104 bash[9203]: cluster 2020-09-08T15:44:48.726912+0000 mon.a (mon.0) 1223 : cluster [DBG] osdmap e268: 12 total, 12 up, 12 in 2020-09-08T15:44:50.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:49 smithi166 bash[9446]: cluster 2020-09-08T15:44:48.172208+0000 osd.4 (osd.4) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:50.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:49 smithi166 bash[9446]: cluster 2020-09-08T15:44:48.172293+0000 osd.4 (osd.4) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:50.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:49 smithi166 bash[9446]: cluster 2020-09-08T15:44:48.726912+0000 mon.a (mon.0) 1223 : cluster [DBG] osdmap e268: 12 total, 12 up, 12 in 2020-09-08T15:44:50.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:49 smithi055 bash[9296]: cluster 2020-09-08T15:44:48.172208+0000 osd.4 (osd.4) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:50.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:49 smithi055 bash[9296]: cluster 2020-09-08T15:44:48.172293+0000 osd.4 (osd.4) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:50.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:49 smithi055 bash[9296]: cluster 2020-09-08T15:44:48.726912+0000 mon.a (mon.0) 1223 : cluster [DBG] osdmap e268: 12 total, 12 up, 12 in 2020-09-08T15:44:50.386 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:50.489 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:44:50.673 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:44:50.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:44:50 smithi104 bash[22305]: debug 2020-09-08T15:44:50.382+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:50.776 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:44:50.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:44:50 smithi166 bash[16593]: debug 2020-09-08T15:44:50.675+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:51.676 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:44:51.768 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:44:51.769 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:44:51.780 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:44:51.788 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:51 smithi055 bash[9296]: cluster 2020-09-08T15:44:49.706422+0000 mgr.x (mgr.24118) 525 : cluster [DBG] pgmap v709: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 548 op/s; 19227/175312 objects misplaced (10.967%); 7.3 MiB/s, 117 objects/s recovering 2020-09-08T15:44:51.922 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:44:51.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:44:51 smithi104 bash[19922]: debug 2020-09-08T15:44:51.678+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:51.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:51 smithi104 bash[9203]: cluster 2020-09-08T15:44:49.706422+0000 mgr.x (mgr.24118) 525 : cluster [DBG] pgmap v709: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 548 op/s; 19227/175312 objects misplaced (10.967%); 7.3 MiB/s, 117 objects/s recovering 2020-09-08T15:44:52.024 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:44:52.031 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:51 smithi166 bash[9446]: cluster 2020-09-08T15:44:49.706422+0000 mgr.x (mgr.24118) 525 : cluster [DBG] pgmap v709: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 548 op/s; 19227/175312 objects misplaced (10.967%); 7.3 MiB/s, 117 objects/s recovering 2020-09-08T15:44:52.194 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:44:52.270 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:44:51 smithi055 bash[11956]: debug 2020-09-08T15:44:51.922+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:52.298 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:44:52.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:44:52 smithi166 bash[18960]: debug 2020-09-08T15:44:52.195+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:52.509 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:44:52.613 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:44:52.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:44:52 smithi104 bash[19922]: debug 2020-09-08T15:44:52.502+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:52.812 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:44:52.837 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:44:52.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:44:52 smithi055 bash[11956]: debug 2020-09-08T15:44:52.838+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:52.942 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:44:52.967 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2020-09-08T15:44:52.975 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:44:53.063 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:44:53.158 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:44:53.393 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:53.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:44:53 smithi055 bash[11956]: debug 2020-09-08T15:44:53.066+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:54.070 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:53 smithi104 bash[9203]: cluster 2020-09-08T15:44:51.706848+0000 mgr.x (mgr.24118) 526 : cluster [DBG] pgmap v710: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 503 op/s; 19025/174134 objects misplaced (10.925%); 8.1 MiB/s, 129 objects/s recovering 2020-09-08T15:44:54.071 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:44:54 smithi104 bash[17588]: debug 2020-09-08T15:44:54.070+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:54.075 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:54.124 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:53 smithi166 bash[9446]: cluster 2020-09-08T15:44:51.706848+0000 mgr.x (mgr.24118) 526 : cluster [DBG] pgmap v710: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 503 op/s; 19025/174134 objects misplaced (10.925%); 8.1 MiB/s, 129 objects/s recovering 2020-09-08T15:44:54.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:53 smithi055 bash[9296]: cluster 2020-09-08T15:44:51.706848+0000 mgr.x (mgr.24118) 526 : cluster [DBG] pgmap v710: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 503 op/s; 19025/174134 objects misplaced (10.925%); 8.1 MiB/s, 129 objects/s recovering 2020-09-08T15:44:54.177 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:44:54.709 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:44:54.814 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:44:54.948 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:44:55.030 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:44:54 smithi055 bash[11956]: debug 2020-09-08T15:44:54.714+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:55.053 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:44:55.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:44:54 smithi166 bash[18960]: debug 2020-09-08T15:44:54.947+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:55.182 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:44:55.286 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:44:55.301 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:44:55 smithi055 bash[11956]: debug 2020-09-08T15:44:55.182+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:55.482 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:44:55.585 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:44:55.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:44:55 smithi055 bash[14301]: debug 2020-09-08T15:44:55.478+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:55.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:55 smithi104 bash[9203]: cluster 2020-09-08T15:44:53.707334+0000 mgr.x (mgr.24118) 527 : cluster [DBG] pgmap v711: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 721 op/s; 18305/169658 objects misplaced (10.789%); 11 MiB/s, 179 objects/s recovering 2020-09-08T15:44:56.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:55 smithi166 bash[9446]: cluster 2020-09-08T15:44:53.707334+0000 mgr.x (mgr.24118) 527 : cluster [DBG] pgmap v711: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 721 op/s; 18305/169658 objects misplaced (10.789%); 11 MiB/s, 179 objects/s recovering 2020-09-08T15:44:56.163 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:55 smithi055 bash[9296]: cluster 2020-09-08T15:44:53.707334+0000 mgr.x (mgr.24118) 527 : cluster [DBG] pgmap v711: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 721 op/s; 18305/169658 objects misplaced (10.789%); 11 MiB/s, 179 objects/s recovering 2020-09-08T15:44:56.164 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:44:56.269 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:44:56.404 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:44:56.430 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:44:56 smithi104 bash[15273]: debug 2020-09-08T15:44:56.166+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:56.514 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:56.618 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:44:56 smithi166 bash[18960]: debug 2020-09-08T15:44:56.403+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:56.693 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:56.795 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:44:56.930 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:44:56 smithi104 bash[22305]: debug 2020-09-08T15:44:56.686+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:57.071 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:44:57.172 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:44:57.285 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:44:57.388 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:57.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:44:57 smithi055 bash[14301]: debug 2020-09-08T15:44:57.070+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:57.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:44:57 smithi166 bash[11928]: debug 2020-09-08T15:44:57.287+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:57.676 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:44:58.029 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:57 smithi055 bash[9296]: cluster 2020-09-08T15:44:55.707863+0000 mgr.x (mgr.24118) 528 : cluster [DBG] pgmap v712: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 618 op/s; 18305/168830 objects misplaced (10.842%); 9.0 MiB/s, 143 objects/s recovering 2020-09-08T15:44:58.029 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:57 smithi055 bash[9296]: audit 2020-09-08T15:44:57.166984+0000 mon.a (mon.0) 1224 : audit [INF] from='client.? v1:172.21.15.104:0/1063650639' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:44:58.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:57 smithi166 bash[9446]: cluster 2020-09-08T15:44:55.707863+0000 mgr.x (mgr.24118) 528 : cluster [DBG] pgmap v712: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 618 op/s; 18305/168830 objects misplaced (10.842%); 9.0 MiB/s, 143 objects/s recovering 2020-09-08T15:44:58.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:57 smithi166 bash[9446]: audit 2020-09-08T15:44:57.166984+0000 mon.a (mon.0) 1224 : audit [INF] from='client.? v1:172.21.15.104:0/1063650639' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:44:58.262 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:44:58.263 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:44:58.302 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:57 smithi104 bash[9203]: cluster 2020-09-08T15:44:55.707863+0000 mgr.x (mgr.24118) 528 : cluster [DBG] pgmap v712: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 618 op/s; 18305/168830 objects misplaced (10.842%); 9.0 MiB/s, 143 objects/s recovering 2020-09-08T15:44:58.302 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:57 smithi104 bash[9203]: audit 2020-09-08T15:44:57.166984+0000 mon.a (mon.0) 1224 : audit [INF] from='client.? v1:172.21.15.104:0/1063650639' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:44:58.355 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:44:58.462 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:58.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:44:58 smithi104 bash[17588]: debug 2020-09-08T15:44:58.350+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:59.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:58 smithi166 bash[9446]: cluster 2020-09-08T15:44:57.187035+0000 osd.9 (osd.9) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:59.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:58 smithi166 bash[9446]: cluster 2020-09-08T15:44:57.187109+0000 osd.9 (osd.9) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:59.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:58 smithi166 bash[9446]: cluster 2020-09-08T15:44:57.670560+0000 mon.a (mon.0) 1225 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:44:59.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:58 smithi166 bash[9446]: audit 2020-09-08T15:44:57.675190+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? v1:172.21.15.104:0/1063650639' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:44:59.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:58 smithi166 bash[9446]: cluster 2020-09-08T15:44:57.675236+0000 mon.a (mon.0) 1227 : cluster [DBG] osdmap e269: 12 total, 12 up, 12 in 2020-09-08T15:44:59.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:58 smithi055 bash[9296]: cluster 2020-09-08T15:44:57.187035+0000 osd.9 (osd.9) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:59.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:58 smithi055 bash[9296]: cluster 2020-09-08T15:44:57.187109+0000 osd.9 (osd.9) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:59.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:58 smithi055 bash[9296]: cluster 2020-09-08T15:44:57.670560+0000 mon.a (mon.0) 1225 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:44:59.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:58 smithi055 bash[9296]: audit 2020-09-08T15:44:57.675190+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? v1:172.21.15.104:0/1063650639' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:44:59.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:58 smithi055 bash[9296]: cluster 2020-09-08T15:44:57.675236+0000 mon.a (mon.0) 1227 : cluster [DBG] osdmap e269: 12 total, 12 up, 12 in 2020-09-08T15:44:59.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:58 smithi104 bash[9203]: cluster 2020-09-08T15:44:57.187035+0000 osd.9 (osd.9) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:59.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:58 smithi104 bash[9203]: cluster 2020-09-08T15:44:57.187109+0000 osd.9 (osd.9) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:59.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:58 smithi104 bash[9203]: cluster 2020-09-08T15:44:57.670560+0000 mon.a (mon.0) 1225 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:44:59.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:58 smithi104 bash[9203]: audit 2020-09-08T15:44:57.675190+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? v1:172.21.15.104:0/1063650639' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:44:59.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:58 smithi104 bash[9203]: cluster 2020-09-08T15:44:57.675236+0000 mon.a (mon.0) 1227 : cluster [DBG] osdmap e269: 12 total, 12 up, 12 in 2020-09-08T15:44:59.309 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2020-09-08T15:44:59.447 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:44:59.466 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:44:59.554 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:44:59.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:59 smithi055 bash[9296]: cluster 2020-09-08T15:44:57.708361+0000 mgr.x (mgr.24118) 529 : cluster [DBG] pgmap v714: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 649 op/s; 17547/164378 objects misplaced (10.675%); 9.1 MiB/s, 144 objects/s recovering 2020-09-08T15:44:59.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:59 smithi055 bash[9296]: cluster 2020-09-08T15:44:59.313820+0000 osd.2 (osd.2) 38 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:59.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:44:59 smithi055 bash[9296]: cluster 2020-09-08T15:44:59.313899+0000 osd.2 (osd.2) 39 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:44:59.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:44:59 smithi104 bash[22305]: debug 2020-09-08T15:44:59.446+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:44:59.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:59 smithi104 bash[9203]: cluster 2020-09-08T15:44:57.708361+0000 mgr.x (mgr.24118) 529 : cluster [DBG] pgmap v714: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 649 op/s; 17547/164378 objects misplaced (10.675%); 9.1 MiB/s, 144 objects/s recovering 2020-09-08T15:44:59.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:59 smithi104 bash[9203]: cluster 2020-09-08T15:44:59.313820+0000 osd.2 (osd.2) 38 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:44:59.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:44:59 smithi104 bash[9203]: cluster 2020-09-08T15:44:59.313899+0000 osd.2 (osd.2) 39 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:00.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:59 smithi166 bash[9446]: cluster 2020-09-08T15:44:57.708361+0000 mgr.x (mgr.24118) 529 : cluster [DBG] pgmap v714: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 649 op/s; 17547/164378 objects misplaced (10.675%); 9.1 MiB/s, 144 objects/s recovering 2020-09-08T15:45:00.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:59 smithi166 bash[9446]: cluster 2020-09-08T15:44:59.313820+0000 osd.2 (osd.2) 38 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:00.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:44:59 smithi166 bash[9446]: cluster 2020-09-08T15:44:59.313899+0000 osd.2 (osd.2) 39 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:00.196 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:00.299 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:00.440 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:00.542 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:45:00.552 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:45:00 smithi055 bash[14301]: debug 2020-09-08T15:45:00.198+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:00.616 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:45:00 smithi166 bash[14254]: debug 2020-09-08T15:45:00.439+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:00.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:00 smithi055 bash[9296]: cluster 2020-09-08T15:44:59.701796+0000 mon.a (mon.0) 1228 : cluster [DBG] osdmap e270: 12 total, 12 up, 12 in 2020-09-08T15:45:00.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:00 smithi055 bash[9296]: cluster 2020-09-08T15:44:59.708818+0000 mgr.x (mgr.24118) 530 : cluster [DBG] pgmap v716: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 737 op/s; 17340/162338 objects misplaced (10.681%); 10 MiB/s, 163 objects/s recovering 2020-09-08T15:45:00.918 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:45:01.021 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:01.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:00 smithi166 bash[9446]: cluster 2020-09-08T15:44:59.701796+0000 mon.a (mon.0) 1228 : cluster [DBG] osdmap e270: 12 total, 12 up, 12 in 2020-09-08T15:45:01.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:00 smithi166 bash[9446]: cluster 2020-09-08T15:44:59.708818+0000 mgr.x (mgr.24118) 530 : cluster [DBG] pgmap v716: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 737 op/s; 17340/162338 objects misplaced (10.681%); 10 MiB/s, 163 objects/s recovering 2020-09-08T15:45:01.171 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:45:00 smithi055 bash[16646]: debug 2020-09-08T15:45:00.922+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:01.172 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:01.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:00 smithi104 bash[9203]: cluster 2020-09-08T15:44:59.701796+0000 mon.a (mon.0) 1228 : cluster [DBG] osdmap e270: 12 total, 12 up, 12 in 2020-09-08T15:45:01.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:00 smithi104 bash[9203]: cluster 2020-09-08T15:44:59.708818+0000 mgr.x (mgr.24118) 530 : cluster [DBG] pgmap v716: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 5.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 737 op/s; 17340/162338 objects misplaced (10.681%); 10 MiB/s, 163 objects/s recovering 2020-09-08T15:45:01.277 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:01.373 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:01.478 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:01.534 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:45:01 smithi055 bash[14301]: debug 2020-09-08T15:45:01.174+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:01.535 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:45:01 smithi055 bash[14301]: debug 2020-09-08T15:45:01.378+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:01.618 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:01.723 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:45:01.819 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:45:01.856 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:45:01.873 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:45:01 smithi166 bash[16593]: debug 2020-09-08T15:45:01.619+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:01.892 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:45:01.921 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:45:01.955 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:45:01.993 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:45:02.023 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:45:02.062 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:45:02.095 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:45:02.131 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:45:02.164 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:45:02.210 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:45:02.234 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:45:02.234 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:45:02.235 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:45:02.235 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:45:02.235 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:45:02.235 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:45:02.235 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:45:02.236 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:45:02.236 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:45:02.236 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:45:02.236 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:45:02.236 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:45:02.332 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:45:02.503 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:45:02.532 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:45:02.635 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:45:02.910 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:45:02 smithi055 bash[16646]: debug 2020-09-08T15:45:02.534+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:02.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:02 smithi055 bash[9296]: cluster 2020-09-08T15:45:01.709261+0000 mgr.x (mgr.24118) 531 : cluster [DBG] pgmap v717: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 532 op/s; 17141/161134 objects misplaced (10.638%); 7.6 MiB/s, 121 objects/s recovering 2020-09-08T15:45:02.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:02 smithi055 bash[9296]: audit 2020-09-08T15:45:01.777144+0000 mon.a (mon.0) 1229 : audit [DBG] from='client.? v1:172.21.15.104:0/1747410093' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:45:02.958 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:02 smithi104 bash[9203]: cluster 2020-09-08T15:45:01.709261+0000 mgr.x (mgr.24118) 531 : cluster [DBG] pgmap v717: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 532 op/s; 17141/161134 objects misplaced (10.638%); 7.6 MiB/s, 121 objects/s recovering 2020-09-08T15:45:02.958 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:02 smithi104 bash[9203]: audit 2020-09-08T15:45:01.777144+0000 mon.a (mon.0) 1229 : audit [DBG] from='client.? v1:172.21.15.104:0/1747410093' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:45:03.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:02 smithi166 bash[9446]: cluster 2020-09-08T15:45:01.709261+0000 mgr.x (mgr.24118) 531 : cluster [DBG] pgmap v717: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 532 op/s; 17141/161134 objects misplaced (10.638%); 7.6 MiB/s, 121 objects/s recovering 2020-09-08T15:45:03.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:02 smithi166 bash[9446]: audit 2020-09-08T15:45:01.777144+0000 mon.a (mon.0) 1229 : audit [DBG] from='client.? v1:172.21.15.104:0/1747410093' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:45:04.415 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:45:04.524 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:04.644 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:04.647 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:45:04.647 INFO:teuthology.orchestra.run.smithi055.stdout: "size": 20, 2020-09-08T15:45:04.647 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 600, 2020-09-08T15:45:04.648 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T15:45:04.648 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.648 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:89959 2.d e145/126 2:b0f3281f:::benchmark_data_smithi166_20551_object89958:head v 145'5544, mlcod=145'5542)", 2020-09-08T15:45:04.648 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.615696+0000", 2020-09-08T15:45:04.648 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 400.03382761, 2020-09-08T15:45:04.649 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.041290546999999997, 2020-09-08T15:45:04.649 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:04.649 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:04.649 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:04.649 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.651 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:04.651 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.615696+0000", 2020-09-08T15:45:04.651 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:04.651 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.652 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.652 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:04.652 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.615696+0000", 2020-09-08T15:45:04.652 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.7872000000000002e-05 2020-09-08T15:45:04.652 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.653 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.653 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:04.653 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.615784+0000", 2020-09-08T15:45:04.653 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2560000000000002e-06 2020-09-08T15:45:04.653 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.654 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.654 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:04.654 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.615792+0000", 2020-09-08T15:45:04.654 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6212e-05 2020-09-08T15:45:04.655 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.655 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.655 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:04.655 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.615808+0000", 2020-09-08T15:45:04.655 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.2050000000000008e-06 2020-09-08T15:45:04.656 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.656 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.656 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:04.657 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.615817+0000", 2020-09-08T15:45:04.657 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.4191999999999998e-05 2020-09-08T15:45:04.657 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.658 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.658 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:04.658 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.615882+0000", 2020-09-08T15:45:04.658 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.0769999999999998e-05 2020-09-08T15:45:04.658 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.659 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.661 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:04.662 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.615922+0000", 2020-09-08T15:45:04.662 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.040969764999999998 2020-09-08T15:45:04.662 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.662 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.662 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:04.663 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.656892+0000", 2020-09-08T15:45:04.663 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4275000000000001e-05 2020-09-08T15:45:04.663 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.663 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.663 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:04.664 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.656986+0000", 2020-09-08T15:45:04.664 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.041290546999999997 2020-09-08T15:45:04.664 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.664 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:04.664 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.665 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.665 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.665 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:96972 2.d e147/126 2:b28078ee:::benchmark_data_smithi166_20551_object96971:head v 147'6004, mlcod=147'6002)", 2020-09-08T15:45:04.666 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:29.627111+0000", 2020-09-08T15:45:04.666 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 395.02241239900002, 2020-09-08T15:45:04.666 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044209323000000002, 2020-09-08T15:45:04.666 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:04.666 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:04.667 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:04.667 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.667 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:04.668 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.627111+0000", 2020-09-08T15:45:04.668 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:04.668 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.668 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.668 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:04.668 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.627111+0000", 2020-09-08T15:45:04.669 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012088599999999999 2020-09-08T15:45:04.669 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.669 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.669 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:04.669 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.627232+0000", 2020-09-08T15:45:04.670 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.29e-05 2020-09-08T15:45:04.670 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.670 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.670 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:04.670 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.627245+0000", 2020-09-08T15:45:04.671 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.1520999999999999e-05 2020-09-08T15:45:04.671 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.671 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.671 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:04.672 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.627266+0000", 2020-09-08T15:45:04.672 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1144999999999999e-05 2020-09-08T15:45:04.672 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.673 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.673 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:04.673 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.627278+0000", 2020-09-08T15:45:04.673 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.8328000000000001e-05 2020-09-08T15:45:04.673 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.674 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.674 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:04.674 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.627336+0000", 2020-09-08T15:45:04.674 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.9403e-05 2020-09-08T15:45:04.674 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.675 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.675 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:04.675 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.627365+0000", 2020-09-08T15:45:04.675 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043826854999999998 2020-09-08T15:45:04.675 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.675 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.676 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:04.676 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.671192+0000", 2020-09-08T15:45:04.676 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012828500000000001 2020-09-08T15:45:04.676 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.676 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.677 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:04.677 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:29.671320+0000", 2020-09-08T15:45:04.677 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044209323000000002 2020-09-08T15:45:04.677 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.677 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:04.687 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.687 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.687 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.687 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.34742.0:128176 2.d e159/126 2:b14ea5a4:::benchmark_data_smithi166_20551_object128175:head v 159'8033, mlcod=159'8031)", 2020-09-08T15:45:04.688 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:54.620593+0000", 2020-09-08T15:45:04.688 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 370.02893048700003, 2020-09-08T15:45:04.688 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043179051000000003, 2020-09-08T15:45:04.688 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:04.689 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:04.689 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:04.689 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.689 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:04.689 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.620593+0000", 2020-09-08T15:45:04.690 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:04.690 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.690 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.690 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:04.690 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.620593+0000", 2020-09-08T15:45:04.691 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00027082799999999998 2020-09-08T15:45:04.691 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.691 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.691 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:04.692 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.620864+0000", 2020-09-08T15:45:04.692 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3526000000000001e-05 2020-09-08T15:45:04.692 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.692 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.692 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:04.693 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.620877+0000", 2020-09-08T15:45:04.693 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.0452e-05 2020-09-08T15:45:04.693 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.694 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.694 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:04.694 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.620898+0000", 2020-09-08T15:45:04.694 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0383e-05 2020-09-08T15:45:04.694 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.695 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.695 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:04.695 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.620908+0000", 2020-09-08T15:45:04.695 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.1581999999999993e-05 2020-09-08T15:45:04.695 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.696 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.696 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:04.696 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.620980+0000", 2020-09-08T15:45:04.706 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.0502999999999999e-05 2020-09-08T15:45:04.706 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.706 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.707 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:04.707 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.621010+0000", 2020-09-08T15:45:04.707 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.042675626000000001 2020-09-08T15:45:04.707 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.707 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.708 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:04.708 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.663686+0000", 2020-09-08T15:45:04.708 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.6150999999999996e-05 2020-09-08T15:45:04.708 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.708 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.709 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:04.709 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:54.663772+0000", 2020-09-08T15:45:04.709 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043179051000000003 2020-09-08T15:45:04.709 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.717 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:04.718 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.718 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.718 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.719 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:14757 3.1 e221/214 3:8dd7aeda:::benchmark_data_smithi166_26097_object14756:head v 221'909, mlcod=221'906)", 2020-09-08T15:45:04.719 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:24.675897+0000", 2020-09-08T15:45:04.719 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 159.97362626099999, 2020-09-08T15:45:04.719 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.042680911000000002, 2020-09-08T15:45:04.719 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:04.720 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:04.720 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:04.720 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.720 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:04.721 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.675897+0000", 2020-09-08T15:45:04.721 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:04.721 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.721 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.721 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:04.723 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.675897+0000", 2020-09-08T15:45:04.723 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010288799999999999 2020-09-08T15:45:04.723 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.723 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.724 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:04.724 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.676000+0000", 2020-09-08T15:45:04.724 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.9490000000000008e-06 2020-09-08T15:45:04.724 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.724 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.725 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:04.725 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.676010+0000", 2020-09-08T15:45:04.725 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6002e-05 2020-09-08T15:45:04.725 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.726 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.726 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:04.726 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.676026+0000", 2020-09-08T15:45:04.726 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.703e-06 2020-09-08T15:45:04.726 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.727 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.727 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:04.727 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.676033+0000", 2020-09-08T15:45:04.727 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000121105 2020-09-08T15:45:04.727 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.728 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.728 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:04.728 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.676154+0000", 2020-09-08T15:45:04.728 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.1246999999999998e-05 2020-09-08T15:45:04.728 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.729 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.729 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:04.729 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.676175+0000", 2020-09-08T15:45:04.729 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.042256384000000001 2020-09-08T15:45:04.729 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.737 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.737 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:04.738 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.718432+0000", 2020-09-08T15:45:04.738 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014663299999999999 2020-09-08T15:45:04.738 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.738 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.738 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:04.739 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.718578+0000", 2020-09-08T15:45:04.739 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.042680911000000002 2020-09-08T15:45:04.739 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.739 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:04.739 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.740 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.740 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.740 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:14762 3.1 e221/214 3:87c90742:::benchmark_data_smithi166_26097_object14761:head v 221'910, mlcod=221'906)", 2020-09-08T15:45:04.740 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:24.676910+0000", 2020-09-08T15:45:04.742 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 159.97261321400001, 2020-09-08T15:45:04.742 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.042098704000000001, 2020-09-08T15:45:04.742 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:04.743 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:04.743 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:04.743 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.743 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:04.743 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.676910+0000", 2020-09-08T15:45:04.744 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:04.744 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.745 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.745 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:04.745 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.676910+0000", 2020-09-08T15:45:04.745 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000103045 2020-09-08T15:45:04.746 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.746 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.753 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:04.753 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677013+0000", 2020-09-08T15:45:04.754 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2190000000000004e-06 2020-09-08T15:45:04.754 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.754 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.754 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:04.754 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677022+0000", 2020-09-08T15:45:04.755 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5857000000000001e-05 2020-09-08T15:45:04.755 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.756 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.756 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:04.756 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677037+0000", 2020-09-08T15:45:04.756 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4070000000000001e-06 2020-09-08T15:45:04.757 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.757 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.757 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:04.757 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677043+0000", 2020-09-08T15:45:04.757 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000109727 2020-09-08T15:45:04.758 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.758 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.758 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:04.758 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677153+0000", 2020-09-08T15:45:04.758 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.9534e-05 2020-09-08T15:45:04.759 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.759 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.759 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:04.759 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677172+0000", 2020-09-08T15:45:04.759 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.041712301 2020-09-08T15:45:04.760 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.760 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.760 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:04.760 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.718884+0000", 2020-09-08T15:45:04.760 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000124614 2020-09-08T15:45:04.761 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.761 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.761 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:04.761 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.719009+0000", 2020-09-08T15:45:04.762 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.042098704000000001 2020-09-08T15:45:04.762 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.763 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:04.763 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.763 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.763 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.763 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:14761 3.a e221/214 3:57b84887:::benchmark_data_smithi166_26097_object14760:head v 221'901, mlcod=221'899)", 2020-09-08T15:45:04.764 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:24.677165+0000", 2020-09-08T15:45:04.764 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 159.97235813, 2020-09-08T15:45:04.764 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.041849460999999998, 2020-09-08T15:45:04.764 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:04.765 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:04.765 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:04.765 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.766 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:04.766 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677165+0000", 2020-09-08T15:45:04.766 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:04.767 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.767 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.767 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:04.767 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677165+0000", 2020-09-08T15:45:04.768 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.2948999999999994e-05 2020-09-08T15:45:04.768 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.768 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.769 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:04.769 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677258+0000", 2020-09-08T15:45:04.769 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.1210000000000007e-06 2020-09-08T15:45:04.769 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.769 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.770 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:04.770 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677266+0000", 2020-09-08T15:45:04.770 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6892999999999999e-05 2020-09-08T15:45:04.770 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.770 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.771 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:04.771 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677283+0000", 2020-09-08T15:45:04.771 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4029999999999998e-06 2020-09-08T15:45:04.771 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.771 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.772 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:04.772 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677289+0000", 2020-09-08T15:45:04.772 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.5768e-05 2020-09-08T15:45:04.772 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.772 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.773 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:04.773 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677335+0000", 2020-09-08T15:45:04.773 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.7956000000000001e-05 2020-09-08T15:45:04.773 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.774 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.774 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:04.774 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677363+0000", 2020-09-08T15:45:04.774 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.041515683999999997 2020-09-08T15:45:04.774 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.775 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.775 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:04.775 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.718878+0000", 2020-09-08T15:45:04.775 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000136687 2020-09-08T15:45:04.775 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.776 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.776 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:04.777 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.719015+0000", 2020-09-08T15:45:04.777 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.041849460999999998 2020-09-08T15:45:04.777 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.777 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:04.777 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.778 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.778 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.778 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:26823 3.a e222/214 3:5134e458:::benchmark_data_smithi166_26097_object26822:head v 222'1667, mlcod=222'1665)", 2020-09-08T15:45:04.778 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:28.723749+0000", 2020-09-08T15:45:04.778 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 155.92577465599999, 2020-09-08T15:45:04.779 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.17094826399999999, 2020-09-08T15:45:04.779 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:04.779 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:04.779 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:04.779 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.780 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:04.780 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723749+0000", 2020-09-08T15:45:04.780 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:04.780 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.780 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.781 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:04.781 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723749+0000", 2020-09-08T15:45:04.781 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.5164000000000002e-05 2020-09-08T15:45:04.781 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.782 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.782 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:04.782 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723814+0000", 2020-09-08T15:45:04.782 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.4159999999999998e-06 2020-09-08T15:45:04.782 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.783 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.783 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:04.783 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723821+0000", 2020-09-08T15:45:04.784 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.7540000000000001e-06 2020-09-08T15:45:04.784 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.784 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.784 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:04.785 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723831+0000", 2020-09-08T15:45:04.785 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.3930000000000001e-06 2020-09-08T15:45:04.785 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.785 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.785 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:04.786 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723836+0000", 2020-09-08T15:45:04.786 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.0776000000000003e-05 2020-09-08T15:45:04.787 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.788 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.788 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:04.788 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723886+0000", 2020-09-08T15:45:04.788 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.16580175899999999 2020-09-08T15:45:04.789 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.789 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.789 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:04.789 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.889688+0000", 2020-09-08T15:45:04.790 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0049295679999999996 2020-09-08T15:45:04.790 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.790 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.790 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:04.790 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.894618+0000", 2020-09-08T15:45:04.791 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.9433999999999999e-05 2020-09-08T15:45:04.791 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.791 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.791 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:04.791 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.894697+0000", 2020-09-08T15:45:04.792 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.17094826399999999 2020-09-08T15:45:04.792 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.792 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:04.792 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.792 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.793 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.793 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:29031 3.a e223/214 3:5b3c5f49:::benchmark_data_smithi166_26097_object29030:head v 223'1790, mlcod=222'1788)", 2020-09-08T15:45:04.793 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:29.670585+0000", 2020-09-08T15:45:04.794 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 154.97893890200001, 2020-09-08T15:45:04.794 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.038081961999999997, 2020-09-08T15:45:04.794 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:04.794 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:04.794 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:04.795 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.795 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:04.795 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.670585+0000", 2020-09-08T15:45:04.795 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:04.795 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.796 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.796 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:04.796 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.670585+0000", 2020-09-08T15:45:04.796 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.7557000000000001e-05 2020-09-08T15:45:04.796 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.797 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.797 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:04.797 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.670682+0000", 2020-09-08T15:45:04.797 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3090000000000001e-06 2020-09-08T15:45:04.798 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.798 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.798 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:04.799 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.670692+0000", 2020-09-08T15:45:04.799 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.487e-05 2020-09-08T15:45:04.799 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.799 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.800 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:04.800 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.670706+0000", 2020-09-08T15:45:04.800 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.5310000000000004e-06 2020-09-08T15:45:04.800 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.800 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.801 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:04.801 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.670714+0000", 2020-09-08T15:45:04.801 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00133851 2020-09-08T15:45:04.801 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.801 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.802 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:04.802 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.672052+0000", 2020-09-08T15:45:04.802 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.1322999999999999e-05 2020-09-08T15:45:04.802 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.802 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.803 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:04.803 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.672074+0000", 2020-09-08T15:45:04.803 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.036502716999999997 2020-09-08T15:45:04.803 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.804 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.804 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:04.804 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.708576+0000", 2020-09-08T15:45:04.804 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.0144999999999998e-05 2020-09-08T15:45:04.804 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.805 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.805 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:04.805 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.708667+0000", 2020-09-08T15:45:04.805 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.038081961999999997 2020-09-08T15:45:04.805 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.806 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:04.806 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.806 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.806 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.806 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:29032 3.a e223/214 3:5db72d8b:::benchmark_data_smithi166_26097_object29031:head v 223'1791, mlcod=222'1788)", 2020-09-08T15:45:04.807 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:29.670769+0000", 2020-09-08T15:45:04.807 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 154.978754353, 2020-09-08T15:45:04.807 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.037990932999999998, 2020-09-08T15:45:04.807 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:04.807 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:04.808 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:04.808 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.809 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:04.809 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.670769+0000", 2020-09-08T15:45:04.809 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:04.809 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.809 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.810 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:04.810 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.670769+0000", 2020-09-08T15:45:04.810 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00030112500000000001 2020-09-08T15:45:04.810 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.811 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.811 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:04.811 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.671070+0000", 2020-09-08T15:45:04.811 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4589999999999998e-06 2020-09-08T15:45:04.811 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.812 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.812 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:04.812 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.671079+0000", 2020-09-08T15:45:04.812 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4221e-05 2020-09-08T15:45:04.812 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.813 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.813 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:04.813 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.671093+0000", 2020-09-08T15:45:04.813 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.6879999999999996e-06 2020-09-08T15:45:04.813 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.814 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.814 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:04.814 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.671099+0000", 2020-09-08T15:45:04.814 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0017955779999999999 2020-09-08T15:45:04.814 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.815 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.815 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:04.815 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.672894+0000", 2020-09-08T15:45:04.815 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.8587999999999999e-05 2020-09-08T15:45:04.815 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.816 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.816 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:04.816 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.672913+0000", 2020-09-08T15:45:04.816 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.035760185999999999 2020-09-08T15:45:04.816 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.817 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.817 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:04.817 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.708673+0000", 2020-09-08T15:45:04.817 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.7088000000000004e-05 2020-09-08T15:45:04.817 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.818 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.818 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:04.818 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.708760+0000", 2020-09-08T15:45:04.818 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.037990932999999998 2020-09-08T15:45:04.819 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.822 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:04.822 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.822 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.823 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.823 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:33161 3.6 e223/217 3:6cd77548:::benchmark_data_smithi166_26097_object33160:head v 223'2100, mlcod=223'2096)", 2020-09-08T15:45:04.823 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:31.718886+0000", 2020-09-08T15:45:04.823 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 152.93063737, 2020-09-08T15:45:04.823 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.038692445999999998, 2020-09-08T15:45:04.824 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:04.824 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:04.824 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:04.824 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.824 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:04.825 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:31.718886+0000", 2020-09-08T15:45:04.825 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:04.825 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.825 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.825 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:04.826 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:31.718886+0000", 2020-09-08T15:45:04.826 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.6160000000000003e-05 2020-09-08T15:45:04.826 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.826 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.826 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:04.827 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:31.718962+0000", 2020-09-08T15:45:04.827 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2639999999999999e-06 2020-09-08T15:45:04.827 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.827 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.828 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:04.828 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:31.718968+0000", 2020-09-08T15:45:04.828 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0034000000000001e-05 2020-09-08T15:45:04.828 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.828 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.829 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:04.829 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:31.718978+0000", 2020-09-08T15:45:04.829 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.4910000000000001e-06 2020-09-08T15:45:04.829 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.829 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.830 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:04.830 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:31.718981+0000", 2020-09-08T15:45:04.830 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.6041000000000002e-05 2020-09-08T15:45:04.830 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.830 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.831 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:04.831 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:31.719027+0000", 2020-09-08T15:45:04.831 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.3002000000000001e-05 2020-09-08T15:45:04.831 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.839 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.839 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:04.839 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:31.719050+0000", 2020-09-08T15:45:04.839 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.038280806000000001 2020-09-08T15:45:04.840 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.840 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.840 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:04.851 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:31.757331+0000", 2020-09-08T15:45:04.851 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000247648 2020-09-08T15:45:04.851 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.851 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.852 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:04.852 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:31.757579+0000", 2020-09-08T15:45:04.852 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.038692445999999998 2020-09-08T15:45:04.852 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.853 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:04.853 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.853 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.853 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.854 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:42772 3.a e224/214 3:593c3494:::benchmark_data_smithi166_26097_object42771:head v 224'2705, mlcod=224'2700)", 2020-09-08T15:45:04.854 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.103175+0000", 2020-09-08T15:45:04.854 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 148.54634856499999, 2020-09-08T15:45:04.855 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.063953390999999998, 2020-09-08T15:45:04.855 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:04.855 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:04.855 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:04.855 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.856 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:04.856 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103175+0000", 2020-09-08T15:45:04.856 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:04.856 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.856 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.857 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:04.857 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103175+0000", 2020-09-08T15:45:04.857 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.9434999999999999e-05 2020-09-08T15:45:04.857 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.857 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.863 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:04.864 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103274+0000", 2020-09-08T15:45:04.864 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.5959999999999993e-06 2020-09-08T15:45:04.864 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.864 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.865 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:04.865 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103284+0000", 2020-09-08T15:45:04.865 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5735000000000001e-05 2020-09-08T15:45:04.865 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.866 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.866 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:04.866 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103300+0000", 2020-09-08T15:45:04.866 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.5139999999999998e-06 2020-09-08T15:45:04.867 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.868 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.869 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:04.870 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103306+0000", 2020-09-08T15:45:04.872 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2772000000000002e-05 2020-09-08T15:45:04.872 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.872 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.873 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:04.873 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103359+0000", 2020-09-08T15:45:04.873 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.2966e-05 2020-09-08T15:45:04.874 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.874 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.883 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:04.883 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103382+0000", 2020-09-08T15:45:04.884 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.063606357000000002 2020-09-08T15:45:04.884 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.884 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.884 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:04.884 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.166988+0000", 2020-09-08T15:45:04.885 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000140016 2020-09-08T15:45:04.885 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.885 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.885 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:04.886 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.167128+0000", 2020-09-08T15:45:04.886 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.063953390999999998 2020-09-08T15:45:04.886 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.886 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:04.887 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.887 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.887 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.887 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:42775 3.6 e224/217 3:625fe670:::benchmark_data_smithi166_26097_object42774:head v 224'2718, mlcod=224'2716)", 2020-09-08T15:45:04.887 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.103618+0000", 2020-09-08T15:45:04.888 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 148.54590542299999, 2020-09-08T15:45:04.888 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.066506083999999993, 2020-09-08T15:45:04.888 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:04.888 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:04.888 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:04.889 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.889 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:04.889 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103618+0000", 2020-09-08T15:45:04.890 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:04.890 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.890 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.890 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:04.890 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103618+0000", 2020-09-08T15:45:04.891 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011588 2020-09-08T15:45:04.891 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.891 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.891 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:04.892 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103734+0000", 2020-09-08T15:45:04.892 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.5589999999999994e-06 2020-09-08T15:45:04.892 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.892 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.892 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:04.893 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103744+0000", 2020-09-08T15:45:04.893 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5424e-05 2020-09-08T15:45:04.893 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.893 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.893 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:04.894 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103759+0000", 2020-09-08T15:45:04.894 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.64e-06 2020-09-08T15:45:04.894 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.894 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.894 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:04.895 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103766+0000", 2020-09-08T15:45:04.895 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.0469000000000006e-05 2020-09-08T15:45:04.895 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.895 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.895 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:04.896 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103836+0000", 2020-09-08T15:45:04.896 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.7387000000000001e-05 2020-09-08T15:45:04.896 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.896 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.896 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:04.897 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103863+0000", 2020-09-08T15:45:04.897 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.066159327000000004 2020-09-08T15:45:04.897 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.897 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.897 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:04.898 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.170023+0000", 2020-09-08T15:45:04.898 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010139799999999999 2020-09-08T15:45:04.898 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.898 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.898 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:04.899 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.170124+0000", 2020-09-08T15:45:04.899 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.066506083999999993 2020-09-08T15:45:04.899 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.899 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:04.899 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.900 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.900 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.900 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:42778 3.a e224/214 3:57632573:::benchmark_data_smithi166_26097_object42777:head v 224'2706, mlcod=224'2703)", 2020-09-08T15:45:04.900 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.105027+0000", 2020-09-08T15:45:04.900 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 148.544496439, 2020-09-08T15:45:04.900 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.067041475000000003, 2020-09-08T15:45:04.901 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:04.901 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:04.901 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:04.901 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.902 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:04.902 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.105027+0000", 2020-09-08T15:45:04.902 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:04.902 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.903 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.903 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:04.903 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.105027+0000", 2020-09-08T15:45:04.903 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4012999999999996e-05 2020-09-08T15:45:04.903 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.904 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.904 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:04.904 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.105121+0000", 2020-09-08T15:45:04.904 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.2180000000000005e-06 2020-09-08T15:45:04.904 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.905 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.905 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:04.905 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.105130+0000", 2020-09-08T15:45:04.905 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4474999999999999e-05 2020-09-08T15:45:04.905 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.906 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.906 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:04.906 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.105145+0000", 2020-09-08T15:45:04.906 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.6489999999999996e-06 2020-09-08T15:45:04.906 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.907 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.907 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:04.907 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.105150+0000", 2020-09-08T15:45:04.907 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.3319000000000003e-05 2020-09-08T15:45:04.908 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.908 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.908 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:04.908 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.105194+0000", 2020-09-08T15:45:04.908 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.1750999999999999e-05 2020-09-08T15:45:04.909 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.909 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.911 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:04.911 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.105216+0000", 2020-09-08T15:45:04.911 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.066681024000000005 2020-09-08T15:45:04.911 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.911 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.912 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:04.912 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.171897+0000", 2020-09-08T15:45:04.912 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017202600000000001 2020-09-08T15:45:04.912 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.912 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.913 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:04.913 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172069+0000", 2020-09-08T15:45:04.913 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.067041475000000003 2020-09-08T15:45:04.913 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.913 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:04.914 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.914 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.914 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.914 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:42782 3.6 e224/217 3:6e3106f1:::benchmark_data_smithi166_26097_object42781:head v 224'2719, mlcod=224'2716)", 2020-09-08T15:45:04.914 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.106056+0000", 2020-09-08T15:45:04.915 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 148.54346720300001, 2020-09-08T15:45:04.915 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.064212001000000005, 2020-09-08T15:45:04.915 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:04.915 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:04.915 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:04.916 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.916 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:04.916 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106056+0000", 2020-09-08T15:45:04.916 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:04.916 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.917 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.917 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:04.917 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106056+0000", 2020-09-08T15:45:04.917 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000108381 2020-09-08T15:45:04.917 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.918 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.918 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:04.918 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106165+0000", 2020-09-08T15:45:04.918 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.8169999999999999e-06 2020-09-08T15:45:04.918 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.918 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.919 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:04.919 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106173+0000", 2020-09-08T15:45:04.919 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4012999999999999e-05 2020-09-08T15:45:04.919 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.919 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.920 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:04.920 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106187+0000", 2020-09-08T15:45:04.920 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.8949999999999997e-06 2020-09-08T15:45:04.920 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.920 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.921 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:04.921 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106191+0000", 2020-09-08T15:45:04.921 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.8047999999999998e-05 2020-09-08T15:45:04.921 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.921 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.922 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:04.922 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106240+0000", 2020-09-08T15:45:04.922 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.4159999999999999e-05 2020-09-08T15:45:04.922 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.923 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.923 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:04.923 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106264+0000", 2020-09-08T15:45:04.923 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.063872915000000002 2020-09-08T15:45:04.923 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.924 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.924 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:04.924 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.170137+0000", 2020-09-08T15:45:04.924 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000131772 2020-09-08T15:45:04.924 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.925 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.925 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:04.925 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.170268+0000", 2020-09-08T15:45:04.925 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.064212001000000005 2020-09-08T15:45:04.925 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.925 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:04.926 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.926 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.926 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.928 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:42774 3.e e224/214 3:7fbb735a:::benchmark_data_smithi166_26097_object42773:head v 224'2632, mlcod=224'2629)", 2020-09-08T15:45:04.928 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.106371+0000", 2020-09-08T15:45:04.928 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 148.54315216099999, 2020-09-08T15:45:04.928 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.065972292000000002, 2020-09-08T15:45:04.929 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:04.929 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:04.929 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:04.929 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.930 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:04.930 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106371+0000", 2020-09-08T15:45:04.930 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:04.930 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.930 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.931 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:04.931 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106371+0000", 2020-09-08T15:45:04.931 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000102574 2020-09-08T15:45:04.931 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.932 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.932 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:04.932 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106474+0000", 2020-09-08T15:45:04.933 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.2750000000000001e-06 2020-09-08T15:45:04.933 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.933 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.934 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:04.934 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106483+0000", 2020-09-08T15:45:04.934 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4969e-05 2020-09-08T15:45:04.934 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.934 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.935 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:04.935 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106498+0000", 2020-09-08T15:45:04.935 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9750000000000004e-06 2020-09-08T15:45:04.935 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.935 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.936 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:04.936 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106504+0000", 2020-09-08T15:45:04.936 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00099841199999999995 2020-09-08T15:45:04.936 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.943 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.943 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:04.943 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107503+0000", 2020-09-08T15:45:04.944 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.4423e-05 2020-09-08T15:45:04.944 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.944 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.944 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:04.945 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107527+0000", 2020-09-08T15:45:04.945 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.064517089999999999 2020-09-08T15:45:04.945 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.945 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.945 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:04.946 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172044+0000", 2020-09-08T15:45:04.946 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00029957399999999999 2020-09-08T15:45:04.948 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.948 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.949 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:04.950 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172344+0000", 2020-09-08T15:45:04.958 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.065972292000000002 2020-09-08T15:45:04.958 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.958 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:04.959 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.959 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.959 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.959 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:42785 3.a e224/214 3:5993af2f:::benchmark_data_smithi166_26097_object42784:head v 224'2707, mlcod=224'2703)", 2020-09-08T15:45:04.959 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.106902+0000", 2020-09-08T15:45:04.960 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 148.54262205200001, 2020-09-08T15:45:04.960 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.065297044999999998, 2020-09-08T15:45:04.960 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:04.960 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:04.960 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:04.961 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.961 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:04.961 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106902+0000", 2020-09-08T15:45:04.961 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:04.961 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.962 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.962 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:04.962 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106902+0000", 2020-09-08T15:45:04.963 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000100908 2020-09-08T15:45:04.963 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.963 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.963 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:04.963 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107002+0000", 2020-09-08T15:45:04.964 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.1149999999999994e-06 2020-09-08T15:45:04.964 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.964 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.964 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:04.964 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107011+0000", 2020-09-08T15:45:04.964 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3278999999999999e-05 2020-09-08T15:45:04.965 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.965 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.965 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:04.965 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107024+0000", 2020-09-08T15:45:04.966 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4759999999999997e-06 2020-09-08T15:45:04.966 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.966 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.966 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:04.967 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107029+0000", 2020-09-08T15:45:04.967 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.057070444999999997 2020-09-08T15:45:04.967 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.967 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.967 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:04.968 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.164100+0000", 2020-09-08T15:45:04.968 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9882e-05 2020-09-08T15:45:04.968 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.968 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.968 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:04.969 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.164160+0000", 2020-09-08T15:45:04.969 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0079172059999999995 2020-09-08T15:45:04.969 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.969 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.969 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:04.970 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172077+0000", 2020-09-08T15:45:04.970 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000121734 2020-09-08T15:45:04.970 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.970 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.970 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:04.971 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172199+0000", 2020-09-08T15:45:04.971 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.065297044999999998 2020-09-08T15:45:04.971 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.971 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:04.971 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.971 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.972 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.972 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:42788 3.e e224/214 3:731858e7:::benchmark_data_smithi166_26097_object42787:head v 224'2633, mlcod=224'2630)", 2020-09-08T15:45:04.973 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.107430+0000", 2020-09-08T15:45:04.973 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 148.542093709, 2020-09-08T15:45:04.974 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.065101027000000006, 2020-09-08T15:45:04.974 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:04.974 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:04.974 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:04.974 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.975 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:04.975 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107430+0000", 2020-09-08T15:45:04.975 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:04.975 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.975 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.975 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:04.976 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107430+0000", 2020-09-08T15:45:04.976 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000105549 2020-09-08T15:45:04.976 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.976 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.976 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:04.977 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107535+0000", 2020-09-08T15:45:04.977 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4570000000000005e-06 2020-09-08T15:45:04.977 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.977 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.977 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:04.978 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107544+0000", 2020-09-08T15:45:04.978 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3541e-05 2020-09-08T15:45:04.979 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.979 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.979 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:04.979 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107557+0000", 2020-09-08T15:45:04.979 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.784e-06 2020-09-08T15:45:04.980 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.988 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.988 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:04.988 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107563+0000", 2020-09-08T15:45:04.988 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.001228966 2020-09-08T15:45:04.988 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.989 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.989 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:04.991 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.108792+0000", 2020-09-08T15:45:04.992 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4925e-05 2020-09-08T15:45:04.992 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.992 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.993 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:04.993 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.108807+0000", 2020-09-08T15:45:04.993 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.063545818000000004 2020-09-08T15:45:04.993 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.994 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.994 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:04.994 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172353+0000", 2020-09-08T15:45:04.994 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000177987 2020-09-08T15:45:04.994 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.995 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.995 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:04.995 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172531+0000", 2020-09-08T15:45:04.995 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.065101027000000006 2020-09-08T15:45:04.995 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.996 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:04.996 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:04.996 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.996 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.996 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:42789 3.a e224/214 3:58a5da28:::benchmark_data_smithi166_26097_object42788:head v 224'2708, mlcod=224'2703)", 2020-09-08T15:45:04.997 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.108694+0000", 2020-09-08T15:45:04.997 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 148.540829258, 2020-09-08T15:45:04.997 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.063628563999999999, 2020-09-08T15:45:04.997 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:04.997 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:04.997 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:04.998 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.998 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:04.998 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.108694+0000", 2020-09-08T15:45:04.998 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:04.998 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:04.999 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:04.999 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:04.999 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.108694+0000", 2020-09-08T15:45:04.999 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.8534000000000003e-05 2020-09-08T15:45:04.999 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.000 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.000 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:05.000 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.108783+0000", 2020-09-08T15:45:05.000 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.5259999999999993e-06 2020-09-08T15:45:05.000 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.000 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.001 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:05.001 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.108791+0000", 2020-09-08T15:45:05.001 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4341e-05 2020-09-08T15:45:05.001 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.002 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.002 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:05.002 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.108806+0000", 2020-09-08T15:45:05.002 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9630000000000003e-06 2020-09-08T15:45:05.002 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.002 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.003 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:05.003 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.108812+0000", 2020-09-08T15:45:05.003 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.058410995 2020-09-08T15:45:05.003 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.003 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.004 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:05.004 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.167223+0000", 2020-09-08T15:45:05.004 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4601999999999997e-05 2020-09-08T15:45:05.004 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.004 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.005 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:05.005 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.167277+0000", 2020-09-08T15:45:05.005 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0049246059999999998 2020-09-08T15:45:05.005 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.005 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.006 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:05.006 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172202+0000", 2020-09-08T15:45:05.006 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012099699999999999 2020-09-08T15:45:05.006 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.007 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.007 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:05.007 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172323+0000", 2020-09-08T15:45:05.007 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.063628563999999999 2020-09-08T15:45:05.007 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:05.008 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:05.008 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:05.008 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.008 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.008 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:120120 3.e e251/214 3:75c7de04:::benchmark_data_smithi166_26097_object120119:head v 251'7514, mlcod=251'7512)", 2020-09-08T15:45:05.008 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:43:44.687062+0000", 2020-09-08T15:45:05.009 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 79.962461347000001, 2020-09-08T15:45:05.009 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043140563999999999, 2020-09-08T15:45:05.009 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:05.009 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:05.009 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:05.010 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.010 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:05.010 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:43:44.687062+0000", 2020-09-08T15:45:05.010 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:05.010 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.011 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.011 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:05.011 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:43:44.687062+0000", 2020-09-08T15:45:05.011 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013374700000000001 2020-09-08T15:45:05.011 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.011 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.012 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:05.013 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:43:44.687196+0000", 2020-09-08T15:45:05.013 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1669e-05 2020-09-08T15:45:05.013 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.014 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.014 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:05.014 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:43:44.687208+0000", 2020-09-08T15:45:05.014 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.5409999999999999e-05 2020-09-08T15:45:05.014 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.015 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.015 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:05.015 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:43:44.687233+0000", 2020-09-08T15:45:05.015 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1007e-05 2020-09-08T15:45:05.015 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.016 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.016 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:05.016 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:43:44.687244+0000", 2020-09-08T15:45:05.016 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3690999999999997e-05 2020-09-08T15:45:05.016 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.017 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.017 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:05.017 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:43:44.687308+0000", 2020-09-08T15:45:05.017 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.8229e-05 2020-09-08T15:45:05.017 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.018 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.018 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:05.019 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:43:44.687336+0000", 2020-09-08T15:45:05.019 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.042700637999999999 2020-09-08T15:45:05.019 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.019 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.019 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:05.020 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:43:44.730037+0000", 2020-09-08T15:45:05.020 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016617300000000001 2020-09-08T15:45:05.020 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.020 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.024 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:05.025 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:43:44.730203+0000", 2020-09-08T15:45:05.025 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043140563999999999 2020-09-08T15:45:05.025 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:05.025 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:05.025 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:05.026 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.026 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.026 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:156269 3.3 e264/214 3:c5007835:::benchmark_data_smithi166_26097_object32108:head v 264'9757, mlcod=264'9755)", 2020-09-08T15:45:05.026 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:44:39.699256+0000", 2020-09-08T15:45:05.026 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 24.950267082, 2020-09-08T15:45:05.027 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044242320000000002, 2020-09-08T15:45:05.027 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:05.027 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:05.027 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:05.027 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.028 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:05.028 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699256+0000", 2020-09-08T15:45:05.028 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:05.028 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.028 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.029 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:05.029 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699256+0000", 2020-09-08T15:45:05.029 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.9709000000000005e-05 2020-09-08T15:45:05.029 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.029 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.030 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:05.030 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699346+0000", 2020-09-08T15:45:05.030 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.9023999999999999e-05 2020-09-08T15:45:05.030 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.031 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.031 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:05.031 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699365+0000", 2020-09-08T15:45:05.031 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.1735e-05 2020-09-08T15:45:05.031 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.031 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.032 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:05.032 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699397+0000", 2020-09-08T15:45:05.032 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5481e-05 2020-09-08T15:45:05.032 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.032 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.033 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:05.033 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699412+0000", 2020-09-08T15:45:05.033 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.9234000000000007e-05 2020-09-08T15:45:05.034 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.034 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.034 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:05.034 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699502+0000", 2020-09-08T15:45:05.035 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2763000000000003e-05 2020-09-08T15:45:05.035 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.035 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.035 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:05.035 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699554+0000", 2020-09-08T15:45:05.036 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043752211999999999 2020-09-08T15:45:05.036 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.036 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.036 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:05.036 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.743307+0000", 2020-09-08T15:45:05.036 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019216200000000001 2020-09-08T15:45:05.037 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:05.037 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:05.037 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:05.037 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.743499+0000", 2020-09-08T15:45:05.037 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044242320000000002 2020-09-08T15:45:05.038 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:05.038 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:05.038 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:05.038 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:05.038 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:05.039 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:45:05.040 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:45:04 smithi104 bash[15273]: debug 2020-09-08T15:45:04.418+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:05.040 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:45:04 smithi055 bash[11956]: debug 2020-09-08T15:45:04.641+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:05.140 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:45:05.427 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:45:05.535 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:05.703 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:05.768 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:05 smithi104 bash[9203]: cluster 2020-09-08T15:45:03.709692+0000 mgr.x (mgr.24118) 532 : cluster [DBG] pgmap v718: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 765 op/s; 16357/156576 objects misplaced (10.447%); 12 MiB/s, 192 objects/s recovering 2020-09-08T15:45:05.769 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:45:05 smithi104 bash[15273]: debug 2020-09-08T15:45:05.434+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:05.791 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2020-09-08T15:45:05.805 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:05.807 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:05 smithi055 bash[9296]: cluster 2020-09-08T15:45:03.709692+0000 mgr.x (mgr.24118) 532 : cluster [DBG] pgmap v718: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 765 op/s; 16357/156576 objects misplaced (10.447%); 12 MiB/s, 192 objects/s recovering 2020-09-08T15:45:05.807 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:45:05 smithi055 bash[14301]: debug 2020-09-08T15:45:05.705+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:05.942 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:05.969 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:45:06.044 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:06.058 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:05 smithi166 bash[9446]: cluster 2020-09-08T15:45:03.709692+0000 mgr.x (mgr.24118) 532 : cluster [DBG] pgmap v718: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 765 op/s; 16357/156576 objects misplaced (10.447%); 12 MiB/s, 192 objects/s recovering 2020-09-08T15:45:06.158 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:45:05 smithi055 bash[11956]: debug 2020-09-08T15:45:05.941+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:06.213 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:06.317 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:45:06.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:45:06 smithi166 bash[14254]: debug 2020-09-08T15:45:06.215+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:07.306 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:45:07.482 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:45:07.858 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:07 smithi055 bash[9296]: cluster 2020-09-08T15:45:05.710136+0000 mgr.x (mgr.24118) 533 : cluster [DBG] pgmap v719: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 534 op/s; 16357/155794 objects misplaced (10.499%); 7.6 MiB/s, 121 objects/s recovering 2020-09-08T15:45:07.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:07 smithi104 bash[9203]: cluster 2020-09-08T15:45:05.710136+0000 mgr.x (mgr.24118) 533 : cluster [DBG] pgmap v719: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 534 op/s; 16357/155794 objects misplaced (10.499%); 7.6 MiB/s, 121 objects/s recovering 2020-09-08T15:45:08.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:07 smithi166 bash[9446]: cluster 2020-09-08T15:45:05.710136+0000 mgr.x (mgr.24118) 533 : cluster [DBG] pgmap v719: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 534 op/s; 16357/155794 objects misplaced (10.499%); 7.6 MiB/s, 121 objects/s recovering 2020-09-08T15:45:08.374 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:45:08.488 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:45:08.678 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:45:08 smithi104 bash[17588]: debug 2020-09-08T15:45:08.374+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:08.678 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:45:08 smithi104 bash[17588]: debug 2020-09-08T15:45:08.670+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:08.678 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:45:08.779 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:45:08.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:08 smithi104 bash[9203]: cluster 2020-09-08T15:45:07.511305+0000 osd.7 (osd.7) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:08.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:08 smithi104 bash[9203]: cluster 2020-09-08T15:45:07.511411+0000 osd.7 (osd.7) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:08.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:08 smithi104 bash[9203]: cluster 2020-09-08T15:45:07.670685+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e271: 12 total, 12 up, 12 in 2020-09-08T15:45:09.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:08 smithi166 bash[9446]: cluster 2020-09-08T15:45:07.511305+0000 osd.7 (osd.7) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:09.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:08 smithi166 bash[9446]: cluster 2020-09-08T15:45:07.511411+0000 osd.7 (osd.7) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:09.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:08 smithi166 bash[9446]: cluster 2020-09-08T15:45:07.670685+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e271: 12 total, 12 up, 12 in 2020-09-08T15:45:09.162 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:08 smithi055 bash[9296]: cluster 2020-09-08T15:45:07.511305+0000 osd.7 (osd.7) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:09.163 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:08 smithi055 bash[9296]: cluster 2020-09-08T15:45:07.511411+0000 osd.7 (osd.7) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:09.163 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:08 smithi055 bash[9296]: cluster 2020-09-08T15:45:07.670685+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e271: 12 total, 12 up, 12 in 2020-09-08T15:45:09.861 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:45:09.965 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:09.976 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:09 smithi166 bash[9446]: cluster 2020-09-08T15:45:07.710656+0000 mgr.x (mgr.24118) 534 : cluster [DBG] pgmap v721: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 702 op/s; 15532/151086 objects misplaced (10.280%); 11 MiB/s, 172 objects/s recovering 2020-09-08T15:45:09.977 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:09 smithi166 bash[9446]: cluster 2020-09-08T15:45:08.919175+0000 osd.0 (osd.0) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:09.977 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:09 smithi166 bash[9446]: cluster 2020-09-08T15:45:08.919250+0000 osd.0 (osd.0) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:10.102 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:10.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:09 smithi055 bash[9296]: cluster 2020-09-08T15:45:07.710656+0000 mgr.x (mgr.24118) 534 : cluster [DBG] pgmap v721: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 702 op/s; 15532/151086 objects misplaced (10.280%); 11 MiB/s, 172 objects/s recovering 2020-09-08T15:45:10.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:09 smithi055 bash[9296]: cluster 2020-09-08T15:45:08.919175+0000 osd.0 (osd.0) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:10.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:09 smithi055 bash[9296]: cluster 2020-09-08T15:45:08.919250+0000 osd.0 (osd.0) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:10.211 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:10.326 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:10.340 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:45:09 smithi104 bash[17588]: debug 2020-09-08T15:45:09.862+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:10.343 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:09 smithi104 bash[9203]: cluster 2020-09-08T15:45:07.710656+0000 mgr.x (mgr.24118) 534 : cluster [DBG] pgmap v721: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 702 op/s; 15532/151086 objects misplaced (10.280%); 11 MiB/s, 172 objects/s recovering 2020-09-08T15:45:10.344 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:09 smithi104 bash[9203]: cluster 2020-09-08T15:45:08.919175+0000 osd.0 (osd.0) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:10.345 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:09 smithi104 bash[9203]: cluster 2020-09-08T15:45:08.919250+0000 osd.0 (osd.0) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:10.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:45:10 smithi166 bash[14254]: debug 2020-09-08T15:45:10.099+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:10.430 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:10.564 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:45:10 smithi055 bash[11956]: debug 2020-09-08T15:45:10.329+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:10.565 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:10.618 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:45:10.619 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T15:45:10.619 INFO:teuthology.orchestra.run.smithi055.stdout: "num_ops": 0 2020-09-08T15:45:10.619 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:45:10.670 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:45:10.910 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:45:10 smithi055 bash[19003]: debug 2020-09-08T15:45:10.565+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:10.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:10 smithi055 bash[9296]: cluster 2020-09-08T15:45:09.699137+0000 mon.a (mon.0) 1231 : cluster [DBG] osdmap e272: 12 total, 12 up, 12 in 2020-09-08T15:45:10.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:10 smithi055 bash[9296]: cluster 2020-09-08T15:45:09.711079+0000 mgr.x (mgr.24118) 535 : cluster [DBG] pgmap v723: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 761 op/s; 15325/148946 objects misplaced (10.289%); 11 MiB/s, 173 objects/s recovering 2020-09-08T15:45:11.023 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:10 smithi104 bash[9203]: cluster 2020-09-08T15:45:09.699137+0000 mon.a (mon.0) 1231 : cluster [DBG] osdmap e272: 12 total, 12 up, 12 in 2020-09-08T15:45:11.023 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:10 smithi104 bash[9203]: cluster 2020-09-08T15:45:09.711079+0000 mgr.x (mgr.24118) 535 : cluster [DBG] pgmap v723: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 761 op/s; 15325/148946 objects misplaced (10.289%); 11 MiB/s, 173 objects/s recovering 2020-09-08T15:45:11.026 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:45:11.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:10 smithi166 bash[9446]: cluster 2020-09-08T15:45:09.699137+0000 mon.a (mon.0) 1231 : cluster [DBG] osdmap e272: 12 total, 12 up, 12 in 2020-09-08T15:45:11.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:10 smithi166 bash[9446]: cluster 2020-09-08T15:45:09.711079+0000 mgr.x (mgr.24118) 535 : cluster [DBG] pgmap v723: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 761 op/s; 15325/148946 objects misplaced (10.289%); 11 MiB/s, 173 objects/s recovering 2020-09-08T15:45:11.130 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:11.430 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:45:11 smithi104 bash[15273]: debug 2020-09-08T15:45:11.026+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:11.657 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:11.763 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:45:11.773 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2020-09-08T15:45:11.908 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:45:11 smithi055 bash[14301]: debug 2020-09-08T15:45:11.657+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:11.919 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:45:12.755 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:45:12.780 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:45:12.859 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:13.037 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:45:13 smithi166 bash[18960]: debug 2020-09-08T15:45:13.039+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:13.038 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:12 smithi166 bash[9446]: cluster 2020-09-08T15:45:11.711566+0000 mgr.x (mgr.24118) 536 : cluster [DBG] pgmap v724: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 556 op/s; 15131/147670 objects misplaced (10.246%); 7.9 MiB/s, 126 objects/s recovering 2020-09-08T15:45:13.038 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:12 smithi166 bash[9446]: audit 2020-09-08T15:45:12.067448+0000 mon.a (mon.0) 1232 : audit [INF] from='client.? v1:172.21.15.104:0/412861333' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:45:13.038 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:13.141 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:13.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:12 smithi055 bash[9296]: cluster 2020-09-08T15:45:11.711566+0000 mgr.x (mgr.24118) 536 : cluster [DBG] pgmap v724: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 556 op/s; 15131/147670 objects misplaced (10.246%); 7.9 MiB/s, 126 objects/s recovering 2020-09-08T15:45:13.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:12 smithi055 bash[9296]: audit 2020-09-08T15:45:12.067448+0000 mon.a (mon.0) 1232 : audit [INF] from='client.? v1:172.21.15.104:0/412861333' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:45:13.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:12 smithi104 bash[9203]: cluster 2020-09-08T15:45:11.711566+0000 mgr.x (mgr.24118) 536 : cluster [DBG] pgmap v724: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 556 op/s; 15131/147670 objects misplaced (10.246%); 7.9 MiB/s, 126 objects/s recovering 2020-09-08T15:45:13.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:12 smithi104 bash[9203]: audit 2020-09-08T15:45:12.067448+0000 mon.a (mon.0) 1232 : audit [INF] from='client.? v1:172.21.15.104:0/412861333' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:45:13.178 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:45:12 smithi104 bash[19922]: debug 2020-09-08T15:45:12.754+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:13.305 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:13.407 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:45:13.417 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:45:13 smithi055 bash[14301]: debug 2020-09-08T15:45:13.305+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:13.538 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:45:13.643 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:13.746 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:45:13 smithi055 bash[16646]: debug 2020-09-08T15:45:13.541+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:13.747 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:13.863 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:14.023 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:45:13 smithi055 bash[14301]: debug 2020-09-08T15:45:13.749+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:14.023 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:13 smithi055 bash[9296]: cluster 2020-09-08T15:45:12.765469+0000 mon.a (mon.0) 1233 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:45:14.023 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:13 smithi055 bash[9296]: cluster 2020-09-08T15:45:12.765508+0000 mon.a (mon.0) 1234 : cluster [INF] Cluster is now healthy 2020-09-08T15:45:14.024 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:13 smithi055 bash[9296]: audit 2020-09-08T15:45:12.776931+0000 mon.a (mon.0) 1235 : audit [INF] from='client.? v1:172.21.15.104:0/412861333' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:45:14.024 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:13 smithi055 bash[9296]: cluster 2020-09-08T15:45:12.777034+0000 mon.a (mon.0) 1236 : cluster [DBG] osdmap e273: 12 total, 12 up, 12 in 2020-09-08T15:45:14.024 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:14.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:13 smithi166 bash[9446]: cluster 2020-09-08T15:45:12.765469+0000 mon.a (mon.0) 1233 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:45:14.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:13 smithi166 bash[9446]: cluster 2020-09-08T15:45:12.765508+0000 mon.a (mon.0) 1234 : cluster [INF] Cluster is now healthy 2020-09-08T15:45:14.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:13 smithi166 bash[9446]: audit 2020-09-08T15:45:12.776931+0000 mon.a (mon.0) 1235 : audit [INF] from='client.? v1:172.21.15.104:0/412861333' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:45:14.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:13 smithi166 bash[9446]: cluster 2020-09-08T15:45:12.777034+0000 mon.a (mon.0) 1236 : cluster [DBG] osdmap e273: 12 total, 12 up, 12 in 2020-09-08T15:45:14.126 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:14.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:13 smithi104 bash[9203]: cluster 2020-09-08T15:45:12.765469+0000 mon.a (mon.0) 1233 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:45:14.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:13 smithi104 bash[9203]: cluster 2020-09-08T15:45:12.765508+0000 mon.a (mon.0) 1234 : cluster [INF] Cluster is now healthy 2020-09-08T15:45:14.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:13 smithi104 bash[9203]: audit 2020-09-08T15:45:12.776931+0000 mon.a (mon.0) 1235 : audit [INF] from='client.? v1:172.21.15.104:0/412861333' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:45:14.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:13 smithi104 bash[9203]: cluster 2020-09-08T15:45:12.777034+0000 mon.a (mon.0) 1236 : cluster [DBG] osdmap e273: 12 total, 12 up, 12 in 2020-09-08T15:45:14.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:45:14 smithi055 bash[14301]: debug 2020-09-08T15:45:14.021+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:15.885 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:45:15.917 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:45:15.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:15 smithi104 bash[9203]: cluster 2020-09-08T15:45:13.712057+0000 mgr.x (mgr.24118) 537 : cluster [DBG] pgmap v726: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 681 op/s; 14337/142846 objects misplaced (10.037%); 10 MiB/s, 164 objects/s recovering 2020-09-08T15:45:15.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:15 smithi104 bash[9203]: cluster 2020-09-08T15:45:14.676991+0000 osd.5 (osd.5) 35 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:15.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:15 smithi104 bash[9203]: cluster 2020-09-08T15:45:14.677075+0000 osd.5 (osd.5) 36 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:15.952 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:45:15.989 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:45:16.023 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:45:16.061 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:45:16.118 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:45:16.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:15 smithi166 bash[9446]: cluster 2020-09-08T15:45:13.712057+0000 mgr.x (mgr.24118) 537 : cluster [DBG] pgmap v726: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 681 op/s; 14337/142846 objects misplaced (10.037%); 10 MiB/s, 164 objects/s recovering 2020-09-08T15:45:16.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:15 smithi166 bash[9446]: cluster 2020-09-08T15:45:14.676991+0000 osd.5 (osd.5) 35 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:16.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:15 smithi166 bash[9446]: cluster 2020-09-08T15:45:14.677075+0000 osd.5 (osd.5) 36 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:16.129 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:45:16.153 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:15 smithi055 bash[9296]: cluster 2020-09-08T15:45:13.712057+0000 mgr.x (mgr.24118) 537 : cluster [DBG] pgmap v726: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 681 op/s; 14337/142846 objects misplaced (10.037%); 10 MiB/s, 164 objects/s recovering 2020-09-08T15:45:16.154 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:15 smithi055 bash[9296]: cluster 2020-09-08T15:45:14.676991+0000 osd.5 (osd.5) 35 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:16.154 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:15 smithi055 bash[9296]: cluster 2020-09-08T15:45:14.677075+0000 osd.5 (osd.5) 36 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:16.156 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:45:16.189 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:45:16.219 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:45:16.247 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:45:16.249 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:45:16.249 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:45:16.249 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:45:16.250 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:45:16.250 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:45:16.250 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:45:16.250 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:45:16.251 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:45:16.251 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:45:16.251 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:45:16.251 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:45:16.251 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:45:16.328 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:16.427 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:45:16.428 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:45:16.438 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:16.576 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:16.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:45:16 smithi166 bash[18960]: debug 2020-09-08T15:45:16.331+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:16.682 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:45:16.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:16 smithi055 bash[9296]: cluster 2020-09-08T15:45:15.678041+0000 mon.a (mon.0) 1237 : cluster [DBG] osdmap e274: 12 total, 12 up, 12 in 2020-09-08T15:45:16.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:16 smithi055 bash[9296]: audit 2020-09-08T15:45:15.848330+0000 mon.a (mon.0) 1238 : audit [DBG] from='client.? v1:172.21.15.104:0/3395453001' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:45:16.908 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:45:16 smithi055 bash[14301]: debug 2020-09-08T15:45:16.577+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:17.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:16 smithi166 bash[9446]: cluster 2020-09-08T15:45:15.678041+0000 mon.a (mon.0) 1237 : cluster [DBG] osdmap e274: 12 total, 12 up, 12 in 2020-09-08T15:45:17.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:16 smithi166 bash[9446]: audit 2020-09-08T15:45:15.848330+0000 mon.a (mon.0) 1238 : audit [DBG] from='client.? v1:172.21.15.104:0/3395453001' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:45:17.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:16 smithi104 bash[9203]: cluster 2020-09-08T15:45:15.678041+0000 mon.a (mon.0) 1237 : cluster [DBG] osdmap e274: 12 total, 12 up, 12 in 2020-09-08T15:45:17.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:16 smithi104 bash[9203]: audit 2020-09-08T15:45:15.848330+0000 mon.a (mon.0) 1238 : audit [DBG] from='client.? v1:172.21.15.104:0/3395453001' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:45:17.271 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:45:17.445 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:45:17.508 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2020-09-08T15:45:17.751 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:45:17.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:17 smithi055 bash[9296]: cluster 2020-09-08T15:45:15.712586+0000 mgr.x (mgr.24118) 538 : cluster [DBG] pgmap v728: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 580 op/s; 14337/141958 objects misplaced (10.099%); 8.2 MiB/s, 130 objects/s recovering 2020-09-08T15:45:18.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:17 smithi166 bash[9446]: cluster 2020-09-08T15:45:15.712586+0000 mgr.x (mgr.24118) 538 : cluster [DBG] pgmap v728: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 580 op/s; 14337/141958 objects misplaced (10.099%); 8.2 MiB/s, 130 objects/s recovering 2020-09-08T15:45:18.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:17 smithi104 bash[9203]: cluster 2020-09-08T15:45:15.712586+0000 mgr.x (mgr.24118) 538 : cluster [DBG] pgmap v728: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 580 op/s; 14337/141958 objects misplaced (10.099%); 8.2 MiB/s, 130 objects/s recovering 2020-09-08T15:45:18.735 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:45:18.839 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:45:19.090 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:18 smithi104 bash[9203]: cluster 2020-09-08T15:45:17.713184+0000 mgr.x (mgr.24118) 539 : cluster [DBG] pgmap v729: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 697 op/s; 13554/137788 objects misplaced (9.837%); 11 MiB/s, 172 objects/s recovering 2020-09-08T15:45:19.091 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:45:19 smithi104 bash[19922]: debug 2020-09-08T15:45:19.090+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:19.092 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:45:19.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:18 smithi166 bash[9446]: cluster 2020-09-08T15:45:17.713184+0000 mgr.x (mgr.24118) 539 : cluster [DBG] pgmap v729: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 697 op/s; 13554/137788 objects misplaced (9.837%); 11 MiB/s, 172 objects/s recovering 2020-09-08T15:45:19.117 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:45:18 smithi166 bash[11928]: debug 2020-09-08T15:45:18.735+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:19.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:18 smithi055 bash[9296]: cluster 2020-09-08T15:45:17.713184+0000 mgr.x (mgr.24118) 539 : cluster [DBG] pgmap v729: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 697 op/s; 13554/137788 objects misplaced (9.837%); 11 MiB/s, 172 objects/s recovering 2020-09-08T15:45:19.195 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:45:19.366 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:45:19 smithi104 bash[15273]: debug 2020-09-08T15:45:19.366+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:19.367 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:45:19.474 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:19.613 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:19.716 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:19.964 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:45:19 smithi055 bash[19003]: debug 2020-09-08T15:45:19.613+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:19.964 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:19 smithi055 bash[9296]: cluster 2020-09-08T15:45:19.038907+0000 osd.10 (osd.10) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:19.964 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:19 smithi055 bash[9296]: cluster 2020-09-08T15:45:19.038983+0000 osd.10 (osd.10) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:20.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:19 smithi166 bash[9446]: cluster 2020-09-08T15:45:19.038907+0000 osd.10 (osd.10) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:20.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:19 smithi166 bash[9446]: cluster 2020-09-08T15:45:19.038983+0000 osd.10 (osd.10) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:20.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:19 smithi104 bash[9203]: cluster 2020-09-08T15:45:19.038907+0000 osd.10 (osd.10) 23 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:20.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:19 smithi104 bash[9203]: cluster 2020-09-08T15:45:19.038983+0000 osd.10 (osd.10) 24 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:20.373 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:45:20.431 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:20.515 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:45:20.534 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:45:20.655 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:45:20.750 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:45:20 smithi166 bash[11928]: debug 2020-09-08T15:45:20.655+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:20.751 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:20 smithi166 bash[9446]: cluster 2020-09-08T15:45:19.713749+0000 mgr.x (mgr.24118) 540 : cluster [DBG] pgmap v730: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 739 op/s; 13354/135836 objects misplaced (9.831%); 11 MiB/s, 173 objects/s recovering 2020-09-08T15:45:20.751 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:20 smithi166 bash[9446]: cluster 2020-09-08T15:45:19.751310+0000 mon.a (mon.0) 1239 : cluster [DBG] osdmap e275: 12 total, 12 up, 12 in 2020-09-08T15:45:20.751 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:45:20 smithi166 bash[18960]: debug 2020-09-08T15:45:20.431+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:20.760 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:20.879 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:20.987 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:45:21.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:45:20 smithi166 bash[16593]: debug 2020-09-08T15:45:20.879+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:21.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:20 smithi055 bash[9296]: cluster 2020-09-08T15:45:19.713749+0000 mgr.x (mgr.24118) 540 : cluster [DBG] pgmap v730: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 739 op/s; 13354/135836 objects misplaced (9.831%); 11 MiB/s, 173 objects/s recovering 2020-09-08T15:45:21.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:20 smithi055 bash[9296]: cluster 2020-09-08T15:45:19.751310+0000 mon.a (mon.0) 1239 : cluster [DBG] osdmap e275: 12 total, 12 up, 12 in 2020-09-08T15:45:21.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:20 smithi104 bash[9203]: cluster 2020-09-08T15:45:19.713749+0000 mgr.x (mgr.24118) 540 : cluster [DBG] pgmap v730: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 739 op/s; 13354/135836 objects misplaced (9.831%); 11 MiB/s, 173 objects/s recovering 2020-09-08T15:45:21.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:20 smithi104 bash[9203]: cluster 2020-09-08T15:45:19.751310+0000 mon.a (mon.0) 1239 : cluster [DBG] osdmap e275: 12 total, 12 up, 12 in 2020-09-08T15:45:21.195 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:45:21.299 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:21.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:45:21 smithi055 bash[16646]: debug 2020-09-08T15:45:21.197+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:21.843 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:21.947 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:22.116 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:45:21 smithi055 bash[19003]: debug 2020-09-08T15:45:21.845+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:22.117 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:22.221 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:22.365 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:22.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:45:22 smithi055 bash[19003]: debug 2020-09-08T15:45:22.121+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:22.436 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:45:22.437 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:45:22.467 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:22.490 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:45:22 smithi166 bash[16593]: debug 2020-09-08T15:45:22.359+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:22.611 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:22.715 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:22.761 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:22 smithi166 bash[9446]: cluster 2020-09-08T15:45:21.714387+0000 mgr.x (mgr.24118) 541 : cluster [DBG] pgmap v732: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 516 op/s; 13148/134578 objects misplaced (9.770%); 7.8 MiB/s, 125 objects/s recovering 2020-09-08T15:45:22.761 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:45:22 smithi166 bash[16593]: debug 2020-09-08T15:45:22.607+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:22.824 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:22.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:22 smithi055 bash[9296]: cluster 2020-09-08T15:45:21.714387+0000 mgr.x (mgr.24118) 541 : cluster [DBG] pgmap v732: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 516 op/s; 13148/134578 objects misplaced (9.770%); 7.8 MiB/s, 125 objects/s recovering 2020-09-08T15:45:22.930 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:23.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:45:22 smithi166 bash[16593]: debug 2020-09-08T15:45:22.823+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:23.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:22 smithi104 bash[9203]: cluster 2020-09-08T15:45:21.714387+0000 mgr.x (mgr.24118) 541 : cluster [DBG] pgmap v732: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 516 op/s; 13148/134578 objects misplaced (9.770%); 7.8 MiB/s, 125 objects/s recovering 2020-09-08T15:45:23.401 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:23.505 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:45:23.537 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2020-09-08T15:45:23.639 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:45:23.658 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:45:23 smithi055 bash[16646]: debug 2020-09-08T15:45:23.641+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:23.659 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:45:23 smithi055 bash[19003]: debug 2020-09-08T15:45:23.401+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:23.718 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:45:23.750 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:45:23.933 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:45:24.036 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:24.045 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:45:23 smithi166 bash[11928]: debug 2020-09-08T15:45:23.931+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:24.167 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:24.270 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:24.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:45:24 smithi166 bash[14254]: debug 2020-09-08T15:45:24.167+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:24.387 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:24.493 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:24.633 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:24.636 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:45:24 smithi166 bash[16593]: debug 2020-09-08T15:45:24.387+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:24.636 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:45:24 smithi166 bash[18960]: debug 2020-09-08T15:45:24.635+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:24.738 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:24.871 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:24.975 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:24.983 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:24 smithi166 bash[9446]: cluster 2020-09-08T15:45:23.553719+0000 osd.3 (osd.3) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:24.983 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:24 smithi166 bash[9446]: cluster 2020-09-08T15:45:23.553810+0000 osd.3 (osd.3) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:24.984 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:24 smithi166 bash[9446]: cluster 2020-09-08T15:45:23.714919+0000 mgr.x (mgr.24118) 542 : cluster [DBG] pgmap v733: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 746 op/s; 12229/129950 objects misplaced (9.411%); 13 MiB/s, 209 objects/s recovering 2020-09-08T15:45:24.984 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:24 smithi166 bash[9446]: cluster 2020-09-08T15:45:23.780098+0000 mon.a (mon.0) 1240 : cluster [DBG] osdmap e276: 12 total, 12 up, 12 in 2020-09-08T15:45:24.984 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:24 smithi166 bash[9446]: audit 2020-09-08T15:45:24.677638+0000 mon.a (mon.0) 1241 : audit [INF] from='client.? v1:172.21.15.104:0/2323219170' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:45:24.985 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:45:24 smithi166 bash[14254]: debug 2020-09-08T15:45:24.871+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:25.095 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:25.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:24 smithi055 bash[9296]: cluster 2020-09-08T15:45:23.553719+0000 osd.3 (osd.3) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:25.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:24 smithi055 bash[9296]: cluster 2020-09-08T15:45:23.553810+0000 osd.3 (osd.3) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:25.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:24 smithi055 bash[9296]: cluster 2020-09-08T15:45:23.714919+0000 mgr.x (mgr.24118) 542 : cluster [DBG] pgmap v733: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 746 op/s; 12229/129950 objects misplaced (9.411%); 13 MiB/s, 209 objects/s recovering 2020-09-08T15:45:25.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:24 smithi055 bash[9296]: cluster 2020-09-08T15:45:23.780098+0000 mon.a (mon.0) 1240 : cluster [DBG] osdmap e276: 12 total, 12 up, 12 in 2020-09-08T15:45:25.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:24 smithi055 bash[9296]: audit 2020-09-08T15:45:24.677638+0000 mon.a (mon.0) 1241 : audit [INF] from='client.? v1:172.21.15.104:0/2323219170' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:45:25.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:24 smithi104 bash[9203]: cluster 2020-09-08T15:45:23.553719+0000 osd.3 (osd.3) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:25.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:24 smithi104 bash[9203]: cluster 2020-09-08T15:45:23.553810+0000 osd.3 (osd.3) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:25.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:24 smithi104 bash[9203]: cluster 2020-09-08T15:45:23.714919+0000 mgr.x (mgr.24118) 542 : cluster [DBG] pgmap v733: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 4.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 746 op/s; 12229/129950 objects misplaced (9.411%); 13 MiB/s, 209 objects/s recovering 2020-09-08T15:45:25.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:24 smithi104 bash[9203]: cluster 2020-09-08T15:45:23.780098+0000 mon.a (mon.0) 1240 : cluster [DBG] osdmap e276: 12 total, 12 up, 12 in 2020-09-08T15:45:25.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:24 smithi104 bash[9203]: audit 2020-09-08T15:45:24.677638+0000 mon.a (mon.0) 1241 : audit [INF] from='client.? v1:172.21.15.104:0/2323219170' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:45:25.201 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:25.323 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:25.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:45:25 smithi166 bash[18960]: debug 2020-09-08T15:45:25.095+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:25.426 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:45:25.532 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:45:25 smithi055 bash[19003]: debug 2020-09-08T15:45:25.321+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:25.538 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:45:25.642 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:45:25.672 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:45:25.746 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:45:25 smithi166 bash[11928]: debug 2020-09-08T15:45:25.539+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:25.746 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:45:25 smithi166 bash[11928]: debug 2020-09-08T15:45:25.747+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:25.753 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:45:25.859 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:25.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:25 smithi055 bash[9296]: cluster 2020-09-08T15:45:24.592047+0000 osd.6 (osd.6) 35 : cluster [DBG] 3.a scrub starts 2020-09-08T15:45:25.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:25 smithi055 bash[9296]: cluster 2020-09-08T15:45:25.669177+0000 mon.a (mon.0) 1242 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:45:25.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:25 smithi055 bash[9296]: audit 2020-09-08T15:45:25.675229+0000 mon.a (mon.0) 1243 : audit [INF] from='client.? v1:172.21.15.104:0/2323219170' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:45:25.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:25 smithi055 bash[9296]: cluster 2020-09-08T15:45:25.675268+0000 mon.a (mon.0) 1244 : cluster [DBG] osdmap e277: 12 total, 12 up, 12 in 2020-09-08T15:45:25.954 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:26.062 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:26.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:25 smithi166 bash[9446]: cluster 2020-09-08T15:45:24.592047+0000 osd.6 (osd.6) 35 : cluster [DBG] 3.a scrub starts 2020-09-08T15:45:26.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:25 smithi166 bash[9446]: cluster 2020-09-08T15:45:25.669177+0000 mon.a (mon.0) 1242 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:45:26.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:25 smithi166 bash[9446]: audit 2020-09-08T15:45:25.675229+0000 mon.a (mon.0) 1243 : audit [INF] from='client.? v1:172.21.15.104:0/2323219170' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:45:26.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:25 smithi166 bash[9446]: cluster 2020-09-08T15:45:25.675268+0000 mon.a (mon.0) 1244 : cluster [DBG] osdmap e277: 12 total, 12 up, 12 in 2020-09-08T15:45:26.118 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:45:25 smithi166 bash[18960]: debug 2020-09-08T15:45:25.955+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:26.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:25 smithi104 bash[9203]: cluster 2020-09-08T15:45:24.592047+0000 osd.6 (osd.6) 35 : cluster [DBG] 3.a scrub starts 2020-09-08T15:45:26.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:25 smithi104 bash[9203]: cluster 2020-09-08T15:45:25.669177+0000 mon.a (mon.0) 1242 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:45:26.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:25 smithi104 bash[9203]: audit 2020-09-08T15:45:25.675229+0000 mon.a (mon.0) 1243 : audit [INF] from='client.? v1:172.21.15.104:0/2323219170' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:45:26.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:25 smithi104 bash[9203]: cluster 2020-09-08T15:45:25.675268+0000 mon.a (mon.0) 1244 : cluster [DBG] osdmap e277: 12 total, 12 up, 12 in 2020-09-08T15:45:26.709 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:26.811 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:27.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:26 smithi166 bash[9446]: cluster 2020-09-08T15:45:25.401452+0000 osd.6 (osd.6) 36 : cluster [DBG] 3.a scrub ok 2020-09-08T15:45:27.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:26 smithi166 bash[9446]: cluster 2020-09-08T15:45:25.715375+0000 mgr.x (mgr.24118) 543 : cluster [DBG] pgmap v736: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.9 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 555 op/s; 12229/129172 objects misplaced (9.467%); 9.2 MiB/s, 147 objects/s recovering 2020-09-08T15:45:27.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:45:26 smithi055 bash[11956]: debug 2020-09-08T15:45:26.713+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:27.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:26 smithi055 bash[9296]: cluster 2020-09-08T15:45:25.401452+0000 osd.6 (osd.6) 36 : cluster [DBG] 3.a scrub ok 2020-09-08T15:45:27.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:26 smithi055 bash[9296]: cluster 2020-09-08T15:45:25.715375+0000 mgr.x (mgr.24118) 543 : cluster [DBG] pgmap v736: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.9 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 555 op/s; 12229/129172 objects misplaced (9.467%); 9.2 MiB/s, 147 objects/s recovering 2020-09-08T15:45:27.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:26 smithi104 bash[9203]: cluster 2020-09-08T15:45:25.401452+0000 osd.6 (osd.6) 36 : cluster [DBG] 3.a scrub ok 2020-09-08T15:45:27.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:26 smithi104 bash[9203]: cluster 2020-09-08T15:45:25.715375+0000 mgr.x (mgr.24118) 543 : cluster [DBG] pgmap v736: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.9 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 555 op/s; 12229/129172 objects misplaced (9.467%); 9.2 MiB/s, 147 objects/s recovering 2020-09-08T15:45:27.325 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:27.432 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:45:27.604 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:45:27 smithi055 bash[11956]: debug 2020-09-08T15:45:27.329+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:27.605 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:45:27.706 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:27.851 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:27.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:45:27 smithi055 bash[16646]: debug 2020-09-08T15:45:27.593+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:27.957 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:27.965 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:45:27 smithi166 bash[16593]: debug 2020-09-08T15:45:27.851+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:28.080 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:28.188 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:45:28.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:45:28 smithi166 bash[18960]: debug 2020-09-08T15:45:28.083+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:28.448 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:45:28.552 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:28.674 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:28.704 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:45:28.705 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:45:28.780 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:45:28 smithi104 bash[17588]: debug 2020-09-08T15:45:28.442+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:28.781 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:28 smithi104 bash[9203]: cluster 2020-09-08T15:45:27.716074+0000 mgr.x (mgr.24118) 544 : cluster [DBG] pgmap v737: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.8 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 715 op/s; 11376/124438 objects misplaced (9.142%); 12 MiB/s, 191 objects/s recovering 2020-09-08T15:45:28.783 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:28.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:28 smithi166 bash[9446]: cluster 2020-09-08T15:45:27.716074+0000 mgr.x (mgr.24118) 544 : cluster [DBG] pgmap v737: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.8 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 715 op/s; 11376/124438 objects misplaced (9.142%); 12 MiB/s, 191 objects/s recovering 2020-09-08T15:45:28.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:45:28 smithi166 bash[18960]: debug 2020-09-08T15:45:28.667+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:28.868 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:45:28.910 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:45:28.940 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:28.948 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:45:28.983 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:45:29.022 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:45:29.044 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:45:29.063 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:45:29.104 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:45:29.134 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:45:29.174 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:28 smithi055 bash[9296]: cluster 2020-09-08T15:45:27.716074+0000 mgr.x (mgr.24118) 544 : cluster [DBG] pgmap v737: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.8 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 715 op/s; 11376/124438 objects misplaced (9.142%); 12 MiB/s, 191 objects/s recovering 2020-09-08T15:45:29.175 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:45:28 smithi055 bash[11956]: debug 2020-09-08T15:45:28.941+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:29.175 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:45:29.204 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:45:29.238 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:45:29.274 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:45:29.275 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:45:29.275 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:45:29.275 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:45:29.275 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:45:29.275 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:45:29.276 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:45:29.276 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:45:29.276 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:45:29.276 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:45:29.276 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:45:29.277 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:45:29.277 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:45:29.624 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:45:29.728 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:29.816 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2020-09-08T15:45:29.866 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:29.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:29 smithi055 bash[9296]: audit 2020-09-08T15:45:28.821672+0000 mon.a (mon.0) 1245 : audit [DBG] from='client.? v1:172.21.15.104:0/3912733159' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:45:29.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:45:29 smithi055 bash[19003]: debug 2020-09-08T15:45:29.857+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:29.908 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:45:29 smithi055 bash[16646]: debug 2020-09-08T15:45:29.625+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:29.968 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:29.981 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:45:30.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:29 smithi166 bash[9446]: audit 2020-09-08T15:45:28.821672+0000 mon.a (mon.0) 1245 : audit [DBG] from='client.? v1:172.21.15.104:0/3912733159' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:45:30.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:29 smithi104 bash[9203]: audit 2020-09-08T15:45:28.821672+0000 mon.a (mon.0) 1245 : audit [DBG] from='client.? v1:172.21.15.104:0/3912733159' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:45:30.191 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:45:30.367 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:45:30.479 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:30.582 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:45:30.785 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:45:30 smithi166 bash[18960]: debug 2020-09-08T15:45:30.478+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:30.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:30 smithi104 bash[9203]: cluster 2020-09-08T15:45:29.716679+0000 mgr.x (mgr.24118) 545 : cluster [DBG] pgmap v738: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 769 op/s; 11153/122266 objects misplaced (9.122%); 12 MiB/s, 192 objects/s recovering 2020-09-08T15:45:30.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:30 smithi104 bash[9203]: cluster 2020-09-08T15:45:30.113664+0000 osd.8 (osd.8) 37 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:30.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:30 smithi104 bash[9203]: cluster 2020-09-08T15:45:30.113751+0000 osd.8 (osd.8) 38 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:31.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:30 smithi166 bash[9446]: cluster 2020-09-08T15:45:29.716679+0000 mgr.x (mgr.24118) 545 : cluster [DBG] pgmap v738: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 769 op/s; 11153/122266 objects misplaced (9.122%); 12 MiB/s, 192 objects/s recovering 2020-09-08T15:45:31.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:30 smithi166 bash[9446]: cluster 2020-09-08T15:45:30.113664+0000 osd.8 (osd.8) 37 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:31.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:30 smithi166 bash[9446]: cluster 2020-09-08T15:45:30.113751+0000 osd.8 (osd.8) 38 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:31.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:30 smithi055 bash[9296]: cluster 2020-09-08T15:45:29.716679+0000 mgr.x (mgr.24118) 545 : cluster [DBG] pgmap v738: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 769 op/s; 11153/122266 objects misplaced (9.122%); 12 MiB/s, 192 objects/s recovering 2020-09-08T15:45:31.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:30 smithi055 bash[9296]: cluster 2020-09-08T15:45:30.113664+0000 osd.8 (osd.8) 37 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:31.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:30 smithi055 bash[9296]: cluster 2020-09-08T15:45:30.113751+0000 osd.8 (osd.8) 38 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:31.227 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:45:31.329 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:31.495 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:31.596 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:45:31.604 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:45:31 smithi055 bash[19003]: debug 2020-09-08T15:45:31.489+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:31.627 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:45:31 smithi104 bash[15273]: debug 2020-09-08T15:45:31.222+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:31.716 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:45:31.820 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:45:31.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:45:31 smithi055 bash[16646]: debug 2020-09-08T15:45:31.713+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:31.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:31 smithi055 bash[9296]: cluster 2020-09-08T15:45:30.800267+0000 mon.a (mon.0) 1246 : cluster [DBG] osdmap e278: 12 total, 12 up, 12 in 2020-09-08T15:45:31.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:31 smithi104 bash[9203]: cluster 2020-09-08T15:45:30.800267+0000 mon.a (mon.0) 1246 : cluster [DBG] osdmap e278: 12 total, 12 up, 12 in 2020-09-08T15:45:31.968 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:45:32.074 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:45:32.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:31 smithi166 bash[9446]: cluster 2020-09-08T15:45:30.800267+0000 mon.a (mon.0) 1246 : cluster [DBG] osdmap e278: 12 total, 12 up, 12 in 2020-09-08T15:45:32.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:45:31 smithi166 bash[11928]: debug 2020-09-08T15:45:31.970+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:32.338 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:45:32.338 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:45:32 smithi104 bash[15273]: debug 2020-09-08T15:45:32.338+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:32.441 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:45:32.639 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:45:32.747 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:45:32.889 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:45:32.892 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:45:32 smithi104 bash[17588]: debug 2020-09-08T15:45:32.630+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:32.892 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:32 smithi104 bash[9203]: cluster 2020-09-08T15:45:31.717112+0000 mgr.x (mgr.24118) 546 : cluster [DBG] pgmap v740: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 522 op/s; 10914/120868 objects misplaced (9.030%); 8.6 MiB/s, 138 objects/s recovering 2020-09-08T15:45:32.892 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:45:32 smithi104 bash[15273]: debug 2020-09-08T15:45:32.886+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:32.994 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:33.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:32 smithi166 bash[9446]: cluster 2020-09-08T15:45:31.717112+0000 mgr.x (mgr.24118) 546 : cluster [DBG] pgmap v740: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 522 op/s; 10914/120868 objects misplaced (9.030%); 8.6 MiB/s, 138 objects/s recovering 2020-09-08T15:45:33.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:32 smithi055 bash[9296]: cluster 2020-09-08T15:45:31.717112+0000 mgr.x (mgr.24118) 546 : cluster [DBG] pgmap v740: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 522 op/s; 10914/120868 objects misplaced (9.030%); 8.6 MiB/s, 138 objects/s recovering 2020-09-08T15:45:33.630 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:45:33.664 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:33.768 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:33.802 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:45:33.899 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:34.005 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:45:34.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:45:33 smithi166 bash[16593]: debug 2020-09-08T15:45:33.898+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:34.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:45:33 smithi055 bash[11956]: debug 2020-09-08T15:45:33.669+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:34.687 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:45:34.790 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:45:34.851 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:45:34.851 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T15:45:34.851 INFO:teuthology.orchestra.run.smithi055.stdout: "complaint_time": 30, 2020-09-08T15:45:34.852 INFO:teuthology.orchestra.run.smithi055.stdout: "num_blocked_ops": 0 2020-09-08T15:45:34.852 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:45:35.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:45:34 smithi104 bash[22305]: debug 2020-09-08T15:45:34.686+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:35.538 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:45:35.642 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:45:35.845 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2020-09-08T15:45:35.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:35 smithi104 bash[9203]: cluster 2020-09-08T15:45:33.717652+0000 mgr.x (mgr.24118) 547 : cluster [DBG] pgmap v741: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 829 op/s; 9991/115830 objects misplaced (8.626%); 14 MiB/s, 221 objects/s recovering 2020-09-08T15:45:35.927 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:45:35 smithi104 bash[15273]: debug 2020-09-08T15:45:35.534+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:35.998 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:45:36.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:35 smithi166 bash[9446]: cluster 2020-09-08T15:45:33.717652+0000 mgr.x (mgr.24118) 547 : cluster [DBG] pgmap v741: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 829 op/s; 9991/115830 objects misplaced (8.626%); 14 MiB/s, 221 objects/s recovering 2020-09-08T15:45:36.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:35 smithi055 bash[9296]: cluster 2020-09-08T15:45:33.717652+0000 mgr.x (mgr.24118) 547 : cluster [DBG] pgmap v741: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 829 op/s; 9991/115830 objects misplaced (8.626%); 14 MiB/s, 221 objects/s recovering 2020-09-08T15:45:36.636 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:45:36.738 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:36.881 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:36.928 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:45:36 smithi104 bash[17588]: debug 2020-09-08T15:45:36.634+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:36.986 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:45:37.158 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:45:36 smithi055 bash[19003]: debug 2020-09-08T15:45:36.885+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:37.186 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:45:37.290 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:45:37.292 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:45:37 smithi104 bash[19922]: debug 2020-09-08T15:45:37.186+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:37.464 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:45:37.574 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:37.672 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:37 smithi104 bash[9203]: cluster 2020-09-08T15:45:35.718225+0000 mgr.x (mgr.24118) 548 : cluster [DBG] pgmap v742: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 712 op/s; 9991/114912 objects misplaced (8.694%); 11 MiB/s, 177 objects/s recovering 2020-09-08T15:45:37.672 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:45:37 smithi104 bash[17588]: debug 2020-09-08T15:45:37.466+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:37.749 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:37.860 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:37.869 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:37 smithi055 bash[9296]: cluster 2020-09-08T15:45:35.718225+0000 mgr.x (mgr.24118) 548 : cluster [DBG] pgmap v742: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 712 op/s; 9991/114912 objects misplaced (8.694%); 11 MiB/s, 177 objects/s recovering 2020-09-08T15:45:37.870 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:45:37 smithi055 bash[14301]: debug 2020-09-08T15:45:37.753+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:37.978 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:38.084 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:38.092 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:37 smithi166 bash[9446]: cluster 2020-09-08T15:45:35.718225+0000 mgr.x (mgr.24118) 548 : cluster [DBG] pgmap v742: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 712 op/s; 9991/114912 objects misplaced (8.694%); 11 MiB/s, 177 objects/s recovering 2020-09-08T15:45:38.155 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:45:37 smithi055 bash[19003]: debug 2020-09-08T15:45:37.973+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:38.220 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:38.325 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:38.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:45:38 smithi166 bash[14254]: debug 2020-09-08T15:45:38.222+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:38.692 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:45:39.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:38 smithi166 bash[9446]: cluster 2020-09-08T15:45:36.963559+0000 osd.1 (osd.1) 52 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:39.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:38 smithi166 bash[9446]: cluster 2020-09-08T15:45:36.963650+0000 osd.1 (osd.1) 53 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:39.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:38 smithi166 bash[9446]: cluster 2020-09-08T15:45:37.688833+0000 mon.a (mon.0) 1247 : cluster [DBG] osdmap e279: 12 total, 12 up, 12 in 2020-09-08T15:45:39.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:38 smithi166 bash[9446]: audit 2020-09-08T15:45:37.847771+0000 mon.a (mon.0) 1248 : audit [INF] from='client.? v1:172.21.15.104:0/3753148680' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:45:39.118 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:38 smithi055 bash[9296]: cluster 2020-09-08T15:45:36.963559+0000 osd.1 (osd.1) 52 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:39.118 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:38 smithi055 bash[9296]: cluster 2020-09-08T15:45:36.963650+0000 osd.1 (osd.1) 53 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:39.119 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:38 smithi055 bash[9296]: cluster 2020-09-08T15:45:37.688833+0000 mon.a (mon.0) 1247 : cluster [DBG] osdmap e279: 12 total, 12 up, 12 in 2020-09-08T15:45:39.121 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:38 smithi055 bash[9296]: audit 2020-09-08T15:45:37.847771+0000 mon.a (mon.0) 1248 : audit [INF] from='client.? v1:172.21.15.104:0/3753148680' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:45:39.123 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:39.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:38 smithi104 bash[9203]: cluster 2020-09-08T15:45:36.963559+0000 osd.1 (osd.1) 52 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:39.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:38 smithi104 bash[9203]: cluster 2020-09-08T15:45:36.963650+0000 osd.1 (osd.1) 53 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:39.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:38 smithi104 bash[9203]: cluster 2020-09-08T15:45:37.688833+0000 mon.a (mon.0) 1247 : cluster [DBG] osdmap e279: 12 total, 12 up, 12 in 2020-09-08T15:45:39.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:38 smithi104 bash[9203]: audit 2020-09-08T15:45:37.847771+0000 mon.a (mon.0) 1248 : audit [INF] from='client.? v1:172.21.15.104:0/3753148680' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:45:39.224 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:45:39.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:45:39 smithi055 bash[19003]: debug 2020-09-08T15:45:39.121+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:39.675 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:45:39 smithi104 bash[15273]: debug 2020-09-08T15:45:39.674+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:39.677 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:45:39.782 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:45:39.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:39 smithi104 bash[9203]: cluster 2020-09-08T15:45:37.718823+0000 mgr.x (mgr.24118) 549 : cluster [DBG] pgmap v744: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 784 op/s; 9097/109708 objects misplaced (8.292%); 12 MiB/s, 198 objects/s recovering 2020-09-08T15:45:39.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:39 smithi104 bash[9203]: cluster 2020-09-08T15:45:38.685454+0000 mon.a (mon.0) 1249 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:45:39.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:39 smithi104 bash[9203]: audit 2020-09-08T15:45:38.694268+0000 mon.a (mon.0) 1250 : audit [INF] from='client.? v1:172.21.15.104:0/3753148680' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:45:39.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:39 smithi104 bash[9203]: cluster 2020-09-08T15:45:38.694306+0000 mon.a (mon.0) 1251 : cluster [DBG] osdmap e280: 12 total, 12 up, 12 in 2020-09-08T15:45:40.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:39 smithi166 bash[9446]: cluster 2020-09-08T15:45:37.718823+0000 mgr.x (mgr.24118) 549 : cluster [DBG] pgmap v744: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 784 op/s; 9097/109708 objects misplaced (8.292%); 12 MiB/s, 198 objects/s recovering 2020-09-08T15:45:40.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:39 smithi166 bash[9446]: cluster 2020-09-08T15:45:38.685454+0000 mon.a (mon.0) 1249 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:45:40.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:39 smithi166 bash[9446]: audit 2020-09-08T15:45:38.694268+0000 mon.a (mon.0) 1250 : audit [INF] from='client.? v1:172.21.15.104:0/3753148680' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:45:40.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:39 smithi166 bash[9446]: cluster 2020-09-08T15:45:38.694306+0000 mon.a (mon.0) 1251 : cluster [DBG] osdmap e280: 12 total, 12 up, 12 in 2020-09-08T15:45:40.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:39 smithi055 bash[9296]: cluster 2020-09-08T15:45:37.718823+0000 mgr.x (mgr.24118) 549 : cluster [DBG] pgmap v744: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 784 op/s; 9097/109708 objects misplaced (8.292%); 12 MiB/s, 198 objects/s recovering 2020-09-08T15:45:40.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:39 smithi055 bash[9296]: cluster 2020-09-08T15:45:38.685454+0000 mon.a (mon.0) 1249 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:45:40.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:39 smithi055 bash[9296]: audit 2020-09-08T15:45:38.694268+0000 mon.a (mon.0) 1250 : audit [INF] from='client.? v1:172.21.15.104:0/3753148680' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:45:40.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:39 smithi055 bash[9296]: cluster 2020-09-08T15:45:38.694306+0000 mon.a (mon.0) 1251 : cluster [DBG] osdmap e280: 12 total, 12 up, 12 in 2020-09-08T15:45:40.408 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:45:40.520 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:40.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:45:40 smithi104 bash[19922]: debug 2020-09-08T15:45:40.410+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:40.780 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:40.783 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:40 smithi166 bash[9446]: audit 2020-09-08T15:45:39.778089+0000 mon.a (mon.0) 1252 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}, {\"id\": \"448d5c57-f6de-443a-ab88-169d81b78450\", \"message\": \"Rebalancing after osd.11 marked in\", \"refs\": [[\"osd\", 11]], \"started_at\": 1599579281.6064675, \"finished_at\": 1599579283.5990696}, {\"id\": \"4bd2c04f-3aed-4914-b08b-6531f7786cca\", \"message\": \"Rebalancing after osd.1 marked out\", \"refs\": [[\"osd\", 1]], \"started_at\": 1599579732.6504169, \"finished_at\": 1599579743.653669}, {\"id\": \"1ab6909d-95da-4336-aa7d-9565e86a361d\", \"message\": \"Rebalancing after osd.1 marked in\", \"refs\": [[\"osd\", 1]], \"started_at\": 1599579802.706257, \"finished_at\": 1599579935.6808252}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:45:40.784 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:40 smithi166 bash[9446]: audit 2020-09-08T15:45:39.781833+0000 mon.a (mon.0) 1253 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}, {\"id\": \"448d5c57-f6de-443a-ab88-169d81b78450\", \"message\": \"Rebalancing after osd.11 marked in\", \"refs\": [[\"osd\", 11]], \"started_at\": 1599579281.6064675, \"finished_at\": 1599579283.5990696}, {\"id\": \"4bd2c04f-3aed-4914-b08b-6531f7786cca\", \"message\": \"Rebalancing after osd.1 marked out\", \"refs\": [[\"osd\", 1]], \"started_at\": 1599579732.6504169, \"finished_at\": 1599579743.653669}, {\"id\": \"1ab6909d-95da-4336-aa7d-9565e86a361d\", \"message\": \"Rebalancing after osd.1 marked in\", \"refs\": [[\"osd\", 1]], \"started_at\": 1599579802.706257, \"finished_at\": 1599579935.6808252}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:45:40.784 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:45:40 smithi166 bash[16593]: debug 2020-09-08T15:45:40.774+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:40.885 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:45:40.966 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:40 smithi055 bash[9296]: audit 2020-09-08T15:45:39.778089+0000 mon.a (mon.0) 1252 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}, {\"id\": \"448d5c57-f6de-443a-ab88-169d81b78450\", \"message\": \"Rebalancing after osd.11 marked in\", \"refs\": [[\"osd\", 11]], \"started_at\": 1599579281.6064675, \"finished_at\": 1599579283.5990696}, {\"id\": \"4bd2c04f-3aed-4914-b08b-6531f7786cca\", \"message\": \"Rebalancing after osd.1 marked out\", \"refs\": [[\"osd\", 1]], \"started_at\": 1599579732.6504169, \"finished_at\": 1599579743.653669}, {\"id\": \"1ab6909d-95da-4336-aa7d-9565e86a361d\", \"message\": \"Rebalancing after osd.1 marked in\", \"refs\": [[\"osd\", 1]], \"started_at\": 1599579802.706257, \"finished_at\": 1599579935.6808252}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:45:40.966 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:40 smithi055 bash[9296]: audit 2020-09-08T15:45:39.781833+0000 mon.a (mon.0) 1253 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}, {\"id\": \"448d5c57-f6de-443a-ab88-169d81b78450\", \"message\": \"Rebalancing after osd.11 marked in\", \"refs\": [[\"osd\", 11]], \"started_at\": 1599579281.6064675, \"finished_at\": 1599579283.5990696}, {\"id\": \"4bd2c04f-3aed-4914-b08b-6531f7786cca\", \"message\": \"Rebalancing after osd.1 marked out\", \"refs\": [[\"osd\", 1]], \"started_at\": 1599579732.6504169, \"finished_at\": 1599579743.653669}, {\"id\": \"1ab6909d-95da-4336-aa7d-9565e86a361d\", \"message\": \"Rebalancing after osd.1 marked in\", \"refs\": [[\"osd\", 1]], \"started_at\": 1599579802.706257, \"finished_at\": 1599579935.6808252}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:45:41.145 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:45:41.146 INFO:teuthology.orchestra.run.smithi055.stdout: "size": 20, 2020-09-08T15:45:41.146 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 600, 2020-09-08T15:45:41.146 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T15:45:41.146 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.146 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:23594 2.7 2:e1de44fd:::benchmark_data_smithi166_20551_object23593:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e134)", 2020-09-08T15:45:41.147 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:37:49.619598+0000", 2020-09-08T15:45:41.147 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 471.52378418299998, 2020-09-08T15:45:41.148 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.051753980999999998, 2020-09-08T15:45:41.148 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:41.148 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:41.148 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:41.148 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:45:41.149 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:45:41.149 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 23594 2020-09-08T15:45:41.149 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.149 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:41.149 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.150 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:41.150 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:49.619598+0000", 2020-09-08T15:45:41.150 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:41.150 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.150 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.151 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:41.151 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:49.619598+0000", 2020-09-08T15:45:41.151 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999475 2020-09-08T15:45:41.151 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.151 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.152 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:41.152 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:49.619546+0000", 2020-09-08T15:45:41.152 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019299599999999999 2020-09-08T15:45:41.152 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.152 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.152 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:41.153 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:49.619739+0000", 2020-09-08T15:45:41.153 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.9857999999999999e-05 2020-09-08T15:45:41.153 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.153 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.153 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:41.154 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:49.619759+0000", 2020-09-08T15:45:41.154 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.9899999999999992e-06 2020-09-08T15:45:41.154 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.154 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.155 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:41.155 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:49.619769+0000", 2020-09-08T15:45:41.155 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.9665000000000003e-05 2020-09-08T15:45:41.155 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.156 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.156 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.156 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:49.619858+0000", 2020-09-08T15:45:41.156 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00022408700000000001 2020-09-08T15:45:41.157 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.157 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.157 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:41.163 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:49.620082+0000", 2020-09-08T15:45:41.163 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00027346800000000002 2020-09-08T15:45:41.163 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.163 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.163 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:45:41.164 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:49.620356+0000", 2020-09-08T15:45:41.164 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.001421528 2020-09-08T15:45:41.164 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.164 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.164 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:41.165 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:49.621777+0000", 2020-09-08T15:45:41.165 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.049354953999999999 2020-09-08T15:45:41.165 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.165 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.166 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:41.167 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:49.671132+0000", 2020-09-08T15:45:41.167 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015814799999999999 2020-09-08T15:45:41.167 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.167 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.167 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:41.168 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:49.671290+0000", 2020-09-08T15:45:41.168 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.1824999999999998e-05 2020-09-08T15:45:41.168 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.168 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.169 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:41.170 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:37:49.671352+0000", 2020-09-08T15:45:41.170 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.051753980999999998 2020-09-08T15:45:41.170 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.170 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.170 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.171 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.171 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.171 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:144624 2.7 2:ef4201fc:::benchmark_data_smithi166_20551_object144623:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e163)", 2020-09-08T15:45:41.172 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:09.620840+0000", 2020-09-08T15:45:41.172 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 391.522542241, 2020-09-08T15:45:41.172 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044412262000000001, 2020-09-08T15:45:41.172 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:41.173 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:41.173 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:41.173 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:45:41.173 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:45:41.174 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 144624 2020-09-08T15:45:41.174 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.174 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:41.174 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.175 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:41.175 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:09.620840+0000", 2020-09-08T15:45:41.175 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:41.175 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.175 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.176 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:41.176 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:09.620840+0000", 2020-09-08T15:45:41.176 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999518 2020-09-08T15:45:41.176 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.177 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.177 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:41.177 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:09.620792+0000", 2020-09-08T15:45:41.177 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000168021 2020-09-08T15:45:41.178 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.178 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.178 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:41.178 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:09.620960+0000", 2020-09-08T15:45:41.178 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.3193999999999998e-05 2020-09-08T15:45:41.180 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.180 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.180 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:41.181 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:09.620983+0000", 2020-09-08T15:45:41.181 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0956e-05 2020-09-08T15:45:41.182 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.182 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.182 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:41.182 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:09.620994+0000", 2020-09-08T15:45:41.183 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010328299999999999 2020-09-08T15:45:41.183 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.183 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.183 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.183 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:09.621098+0000", 2020-09-08T15:45:41.184 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00027880400000000001 2020-09-08T15:45:41.184 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.184 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.184 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:41.185 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:09.621376+0000", 2020-09-08T15:45:41.185 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00030058899999999998 2020-09-08T15:45:41.185 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.185 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.185 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:45:41.186 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:09.621677+0000", 2020-09-08T15:45:41.186 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0058806290000000001 2020-09-08T15:45:41.186 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.186 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.186 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:41.187 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:09.627558+0000", 2020-09-08T15:45:41.187 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.037398389999999997 2020-09-08T15:45:41.187 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.187 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.188 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:41.188 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:09.664956+0000", 2020-09-08T15:45:41.188 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000169722 2020-09-08T15:45:41.188 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.188 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.189 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:41.189 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:09.665126+0000", 2020-09-08T15:45:41.189 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000126706 2020-09-08T15:45:41.189 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.190 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.191 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:41.193 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:09.665252+0000", 2020-09-08T15:45:41.193 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044412262000000001 2020-09-08T15:45:41.193 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.194 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.194 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.194 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.194 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.194 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:163665 2.7 2:e20ae007:::benchmark_data_smithi166_20551_object16045:head [delete] snapc 0=[] ondisk+write+known_if_redirected e170)", 2020-09-08T15:45:41.195 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:29.728808+0000", 2020-09-08T15:45:41.195 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 371.41457396999999, 2020-09-08T15:45:41.195 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.050650082999999999, 2020-09-08T15:45:41.195 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:41.196 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:41.196 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:41.196 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:45:41.196 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:45:41.197 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 163665 2020-09-08T15:45:41.197 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.197 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:41.197 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.198 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:41.198 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.728808+0000", 2020-09-08T15:45:41.198 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:41.198 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.198 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.199 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:41.199 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.728808+0000", 2020-09-08T15:45:41.199 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999576 2020-09-08T15:45:41.199 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.200 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.200 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:41.200 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.728766+0000", 2020-09-08T15:45:41.201 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3676000000000002e-05 2020-09-08T15:45:41.201 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.202 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.202 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:41.202 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.728860+0000", 2020-09-08T15:45:41.202 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4722999999999999e-05 2020-09-08T15:45:41.202 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.203 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.203 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:41.203 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.728875+0000", 2020-09-08T15:45:41.203 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.5440000000000004e-06 2020-09-08T15:45:41.203 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.204 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.204 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:41.204 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.728881+0000", 2020-09-08T15:45:41.204 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0095777129999999999 2020-09-08T15:45:41.205 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.205 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.205 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.205 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.738459+0000", 2020-09-08T15:45:41.205 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.6648e-05 2020-09-08T15:45:41.206 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.206 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.206 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:41.206 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.738555+0000", 2020-09-08T15:45:41.207 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4963000000000006e-05 2020-09-08T15:45:41.207 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.207 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.207 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:45:41.208 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.738640+0000", 2020-09-08T15:45:41.208 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.030408659000000001 2020-09-08T15:45:41.208 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.208 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.208 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:41.209 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.769049+0000", 2020-09-08T15:45:41.209 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.010256013 2020-09-08T15:45:41.209 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.209 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.209 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:41.210 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.779305+0000", 2020-09-08T15:45:41.210 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.1128000000000002e-05 2020-09-08T15:45:41.210 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.210 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.211 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:41.211 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.779366+0000", 2020-09-08T15:45:41.212 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.2305000000000005e-05 2020-09-08T15:45:41.213 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.213 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.213 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:41.213 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:29.779459+0000", 2020-09-08T15:45:41.214 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.050650082999999999 2020-09-08T15:45:41.214 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.225 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.225 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.225 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.225 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.226 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:192572 2.7 2:ec0aeb2e:::benchmark_data_smithi166_20551_object44952:head [delete] snapc 0=[] ondisk+write+known_if_redirected e180)", 2020-09-08T15:45:41.226 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:39:59.630037+0000", 2020-09-08T15:45:41.226 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 341.51334501999997, 2020-09-08T15:45:41.226 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044697788000000002, 2020-09-08T15:45:41.227 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:41.227 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:41.227 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:41.227 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:45:41.227 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:45:41.227 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 192572 2020-09-08T15:45:41.228 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.228 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:41.228 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.228 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:41.228 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:59.630037+0000", 2020-09-08T15:45:41.229 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:41.229 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.229 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.229 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:41.229 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:59.630037+0000", 2020-09-08T15:45:41.230 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999485 2020-09-08T15:45:41.230 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.230 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.230 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:41.231 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:59.629986+0000", 2020-09-08T15:45:41.231 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013451899999999999 2020-09-08T15:45:41.231 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.231 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.231 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:41.231 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:59.630121+0000", 2020-09-08T15:45:41.232 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.6579000000000001e-05 2020-09-08T15:45:41.232 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.232 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.236 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:41.236 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:59.630147+0000", 2020-09-08T15:45:41.237 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3991e-05 2020-09-08T15:45:41.237 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.237 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.242 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:41.243 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:59.630161+0000", 2020-09-08T15:45:41.243 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4425999999999994e-05 2020-09-08T15:45:41.243 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.243 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.244 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.244 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:59.630246+0000", 2020-09-08T15:45:41.244 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000197441 2020-09-08T15:45:41.244 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.244 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.245 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:41.245 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:59.630443+0000", 2020-09-08T15:45:41.245 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000199507 2020-09-08T15:45:41.245 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.246 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.246 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:45:41.246 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:59.630642+0000", 2020-09-08T15:45:41.246 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0068329929999999999 2020-09-08T15:45:41.247 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.247 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.247 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:41.247 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:59.637475+0000", 2020-09-08T15:45:41.247 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.037028612000000002 2020-09-08T15:45:41.248 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.248 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.248 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:41.248 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:59.674504+0000", 2020-09-08T15:45:41.248 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013764699999999999 2020-09-08T15:45:41.249 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.249 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.249 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:41.250 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:59.674642+0000", 2020-09-08T15:45:41.250 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3449999999999995e-05 2020-09-08T15:45:41.250 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.250 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.250 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:41.251 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:39:59.674735+0000", 2020-09-08T15:45:41.251 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044697788000000002 2020-09-08T15:45:41.252 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.252 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.252 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.253 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.253 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.253 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:198245 2.7 2:e0315abc:::benchmark_data_smithi166_20551_object50625:head [delete] snapc 0=[] ondisk+write+known_if_redirected e181)", 2020-09-08T15:45:41.253 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:40:05.333534+0000", 2020-09-08T15:45:41.253 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 335.80984891000003, 2020-09-08T15:45:41.254 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044309279, 2020-09-08T15:45:41.254 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:41.254 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:41.254 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:41.254 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:45:41.255 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:45:41.255 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 198245 2020-09-08T15:45:41.255 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.255 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:41.255 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.256 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:41.256 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:05.333534+0000", 2020-09-08T15:45:41.256 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:41.257 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.257 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.257 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:41.257 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:05.333534+0000", 2020-09-08T15:45:41.257 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999695 2020-09-08T15:45:41.258 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.258 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.258 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:41.258 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:05.333503+0000", 2020-09-08T15:45:41.258 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.8291000000000001e-05 2020-09-08T15:45:41.259 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.259 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.259 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:41.259 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:05.333571+0000", 2020-09-08T15:45:41.259 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5248e-05 2020-09-08T15:45:41.260 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.260 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.260 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:41.260 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:05.333587+0000", 2020-09-08T15:45:41.260 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3230000000000004e-06 2020-09-08T15:45:41.261 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.261 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.261 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:41.261 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:05.333596+0000", 2020-09-08T15:45:41.261 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.007512594 2020-09-08T15:45:41.262 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.262 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.262 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.262 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:05.341108+0000", 2020-09-08T15:45:41.263 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015279700000000001 2020-09-08T15:45:41.263 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.263 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.263 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:41.263 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:05.341261+0000", 2020-09-08T15:45:41.264 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000215777 2020-09-08T15:45:41.264 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.264 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.264 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:45:41.264 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:05.341477+0000", 2020-09-08T15:45:41.265 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0044837940000000001 2020-09-08T15:45:41.265 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.265 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.265 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:41.265 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:05.345961+0000", 2020-09-08T15:45:41.266 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.031715398999999998 2020-09-08T15:45:41.266 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.266 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.267 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:41.267 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:05.377676+0000", 2020-09-08T15:45:41.267 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.8537999999999997e-05 2020-09-08T15:45:41.267 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.268 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.268 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:41.268 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:05.377775+0000", 2020-09-08T15:45:41.268 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.8020000000000003e-05 2020-09-08T15:45:41.268 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.269 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.269 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:41.269 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:40:05.377843+0000", 2020-09-08T15:45:41.269 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044309279 2020-09-08T15:45:41.269 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.270 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.270 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.270 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.270 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.270 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:269918 2.7 2:e334a9c7:::benchmark_data_smithi166_20551_object122298:head [delete] snapc 0=[] ondisk+write+known_if_redirected e202)", 2020-09-08T15:45:41.271 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:41:18.955142+0000", 2020-09-08T15:45:41.271 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 262.188240286, 2020-09-08T15:45:41.271 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.067176626000000003, 2020-09-08T15:45:41.271 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:41.271 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:41.272 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:41.272 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:45:41.272 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:45:41.272 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 269918 2020-09-08T15:45:41.272 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.273 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:41.273 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.273 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:41.273 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:41:18.955142+0000", 2020-09-08T15:45:41.273 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:41.274 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.274 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.274 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:41.274 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:41:18.955142+0000", 2020-09-08T15:45:41.274 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999969 2020-09-08T15:45:41.274 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.275 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.275 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:41.275 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:41:18.955111+0000", 2020-09-08T15:45:41.275 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.0947999999999999e-05 2020-09-08T15:45:41.275 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.276 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.276 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:41.276 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:41:18.955182+0000", 2020-09-08T15:45:41.276 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4727e-05 2020-09-08T15:45:41.276 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.277 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.277 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:41.277 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:41:18.955197+0000", 2020-09-08T15:45:41.278 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.2920000000000001e-06 2020-09-08T15:45:41.278 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.278 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.278 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:41.278 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:41:18.955203+0000", 2020-09-08T15:45:41.279 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.0248999999999998e-05 2020-09-08T15:45:41.279 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.279 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.279 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.279 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:41:18.955273+0000", 2020-09-08T15:45:41.280 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000121687 2020-09-08T15:45:41.280 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.280 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.280 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:41.280 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:41:18.955395+0000", 2020-09-08T15:45:41.280 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011500700000000001 2020-09-08T15:45:41.281 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.281 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.281 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:45:41.281 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:41:18.955510+0000", 2020-09-08T15:45:41.281 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.062775873999999995 2020-09-08T15:45:41.282 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.282 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.282 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:41.282 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:41:19.018286+0000", 2020-09-08T15:45:41.282 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0039301420000000002 2020-09-08T15:45:41.283 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.283 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.283 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:41.283 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:41:19.022216+0000", 2020-09-08T15:45:41.283 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.7726000000000003e-05 2020-09-08T15:45:41.284 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.284 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.284 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:41.284 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:41:19.022274+0000", 2020-09-08T15:45:41.284 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.5030000000000001e-05 2020-09-08T15:45:41.285 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.285 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.285 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:41.285 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:41:19.022319+0000", 2020-09-08T15:45:41.285 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.067176626000000003 2020-09-08T15:45:41.286 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.286 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.286 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.286 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.286 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.286 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:14761 3.a 3:57b84887:::benchmark_data_smithi166_26097_object14760:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e221)", 2020-09-08T15:45:41.287 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:24.675505+0000", 2020-09-08T15:45:41.287 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 196.467877351, 2020-09-08T15:45:41.289 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044319269000000001, 2020-09-08T15:45:41.289 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:41.289 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:41.289 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:41.290 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:45:41.290 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:45:41.290 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 14761 2020-09-08T15:45:41.290 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.290 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:41.291 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.291 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:41.291 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.675505+0000", 2020-09-08T15:45:41.291 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:41.291 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.292 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.292 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:41.292 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.675505+0000", 2020-09-08T15:45:41.292 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999958 2020-09-08T15:45:41.292 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.293 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.293 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:41.293 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.675463+0000", 2020-09-08T15:45:41.293 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014143000000000001 2020-09-08T15:45:41.293 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.293 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.294 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:41.294 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.675605+0000", 2020-09-08T15:45:41.294 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.1827999999999998e-05 2020-09-08T15:45:41.294 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.294 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.295 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:41.295 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.675626+0000", 2020-09-08T15:45:41.297 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2640000000000008e-06 2020-09-08T15:45:41.297 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.298 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.299 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:41.299 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.675635+0000", 2020-09-08T15:45:41.299 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.7168000000000004e-05 2020-09-08T15:45:41.299 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.299 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.300 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.300 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.675712+0000", 2020-09-08T15:45:41.300 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00021245699999999999 2020-09-08T15:45:41.300 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.300 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.300 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:41.301 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.675924+0000", 2020-09-08T15:45:41.301 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000237641 2020-09-08T15:45:41.301 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.301 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.301 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 4", 2020-09-08T15:45:41.302 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.676162+0000", 2020-09-08T15:45:41.302 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.033971207000000003 2020-09-08T15:45:41.302 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.302 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.302 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:41.303 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.710133+0000", 2020-09-08T15:45:41.303 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0095706479999999993 2020-09-08T15:45:41.303 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.303 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.303 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:41.304 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.719704+0000", 2020-09-08T15:45:41.304 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.0976000000000006e-05 2020-09-08T15:45:41.304 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.304 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.304 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:41.305 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.719775+0000", 2020-09-08T15:45:41.305 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.9620999999999998e-05 2020-09-08T15:45:41.305 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.305 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.305 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:41.305 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.719824+0000", 2020-09-08T15:45:41.306 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044319269000000001 2020-09-08T15:45:41.306 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.306 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.306 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.307 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.307 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.307 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:26823 3.a 3:5134e458:::benchmark_data_smithi166_26097_object26822:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e222)", 2020-09-08T15:45:41.307 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:28.723231+0000", 2020-09-08T15:45:41.307 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 192.420151524, 2020-09-08T15:45:41.307 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.173002815, 2020-09-08T15:45:41.309 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:41.309 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:41.310 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:41.310 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:45:41.310 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:45:41.310 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 26823 2020-09-08T15:45:41.310 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.311 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:41.311 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.311 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:41.311 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723231+0000", 2020-09-08T15:45:41.311 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:41.312 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.312 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.312 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:41.312 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723231+0000", 2020-09-08T15:45:41.312 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999733 2020-09-08T15:45:41.313 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.313 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.313 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:41.313 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723204+0000", 2020-09-08T15:45:41.313 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.1558000000000007e-05 2020-09-08T15:45:41.314 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.314 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.314 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:41.314 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723296+0000", 2020-09-08T15:45:41.314 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.6749999999999997e-06 2020-09-08T15:45:41.314 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.315 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.315 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:41.315 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723306+0000", 2020-09-08T15:45:41.315 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.7600000000000002e-06 2020-09-08T15:45:41.315 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.316 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.316 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:41.316 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723310+0000", 2020-09-08T15:45:41.316 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.7574000000000002e-05 2020-09-08T15:45:41.316 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.317 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.317 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.317 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723358+0000", 2020-09-08T15:45:41.317 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011018600000000001 2020-09-08T15:45:41.317 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.318 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.318 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:41.318 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723468+0000", 2020-09-08T15:45:41.318 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000115451 2020-09-08T15:45:41.318 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.318 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.319 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 4", 2020-09-08T15:45:41.319 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723584+0000", 2020-09-08T15:45:41.319 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00099073499999999997 2020-09-08T15:45:41.320 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.320 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.320 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:41.320 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.724574+0000", 2020-09-08T15:45:41.321 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.171517118 2020-09-08T15:45:41.321 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.321 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.321 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:41.322 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.896091+0000", 2020-09-08T15:45:41.322 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.1619999999999994e-05 2020-09-08T15:45:41.322 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.323 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.323 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:41.323 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.896173+0000", 2020-09-08T15:45:41.324 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0693000000000003e-05 2020-09-08T15:45:41.325 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.325 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.325 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:41.325 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.896234+0000", 2020-09-08T15:45:41.326 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.173002815 2020-09-08T15:45:41.326 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.326 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.326 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.326 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.327 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.327 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:29031 3.a 3:5b3c5f49:::benchmark_data_smithi166_26097_object29030:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e223)", 2020-09-08T15:45:41.327 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:29.665438+0000", 2020-09-08T15:45:41.327 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 191.47794422800001, 2020-09-08T15:45:41.327 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043933514999999999, 2020-09-08T15:45:41.328 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:41.328 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:41.328 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:41.328 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:45:41.328 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:45:41.329 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 29031 2020-09-08T15:45:41.329 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.329 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:41.330 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.330 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:41.330 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.665438+0000", 2020-09-08T15:45:41.330 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:41.330 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.331 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.331 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:41.331 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.665438+0000", 2020-09-08T15:45:41.331 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999657 2020-09-08T15:45:41.331 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.331 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.332 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:41.332 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.665404+0000", 2020-09-08T15:45:41.332 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014944000000000001 2020-09-08T15:45:41.332 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.333 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.333 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:41.333 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.665553+0000", 2020-09-08T15:45:41.333 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.2054000000000002e-05 2020-09-08T15:45:41.333 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.333 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.334 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:41.334 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.665585+0000", 2020-09-08T15:45:41.334 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3781000000000001e-05 2020-09-08T15:45:41.334 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.334 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.335 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:41.335 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.665599+0000", 2020-09-08T15:45:41.335 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4825000000000002e-05 2020-09-08T15:45:41.335 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.335 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.336 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.336 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.665684+0000", 2020-09-08T15:45:41.337 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2323999999999999e-05 2020-09-08T15:45:41.337 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.338 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.339 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op must wait for map", 2020-09-08T15:45:41.339 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.665696+0000", 2020-09-08T15:45:41.339 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0022901359999999999 2020-09-08T15:45:41.339 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.339 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.340 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.340 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.667986+0000", 2020-09-08T15:45:41.340 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00018592500000000001 2020-09-08T15:45:41.340 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.341 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.341 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:41.341 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.668172+0000", 2020-09-08T15:45:41.341 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000217891 2020-09-08T15:45:41.341 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.342 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.342 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 4", 2020-09-08T15:45:41.342 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.668390+0000", 2020-09-08T15:45:41.342 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.002740532 2020-09-08T15:45:41.342 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.343 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.343 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:41.343 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.671131+0000", 2020-09-08T15:45:41.343 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.038015502999999999 2020-09-08T15:45:41.343 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.344 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.344 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:41.344 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.709146+0000", 2020-09-08T15:45:41.344 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014694 2020-09-08T15:45:41.344 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.345 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.345 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:41.345 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.709293+0000", 2020-09-08T15:45:41.345 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.8662e-05 2020-09-08T15:45:41.345 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.346 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.346 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:41.346 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:29.709372+0000", 2020-09-08T15:45:41.346 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043933514999999999 2020-09-08T15:45:41.346 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.346 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.347 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.347 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.347 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.347 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:37967 3.a 3:5b11dbb8:::benchmark_data_smithi166_26097_object37966:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e223)", 2020-09-08T15:45:41.347 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:34.094063+0000", 2020-09-08T15:45:41.348 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 187.049319357, 2020-09-08T15:45:41.348 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13139339999999999, 2020-09-08T15:45:41.348 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:41.348 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:41.348 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:41.349 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:45:41.349 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:45:41.349 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 37967 2020-09-08T15:45:41.349 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.349 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:41.350 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.350 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:41.350 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:34.094063+0000", 2020-09-08T15:45:41.351 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:41.351 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.351 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.351 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:41.351 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:34.094063+0000", 2020-09-08T15:45:41.352 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999747 2020-09-08T15:45:41.352 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.352 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.352 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:41.352 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:34.094038+0000", 2020-09-08T15:45:41.352 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010946300000000001 2020-09-08T15:45:41.353 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.353 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.353 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:41.353 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:34.094147+0000", 2020-09-08T15:45:41.353 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.8408999999999999e-05 2020-09-08T15:45:41.354 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.354 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.354 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:41.354 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:34.094166+0000", 2020-09-08T15:45:41.354 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.1470000000000001e-06 2020-09-08T15:45:41.355 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.355 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.355 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:41.355 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:34.094174+0000", 2020-09-08T15:45:41.355 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.9567999999999995e-05 2020-09-08T15:45:41.356 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.356 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.356 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.356 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:34.094243+0000", 2020-09-08T15:45:41.356 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000161259 2020-09-08T15:45:41.357 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.357 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.357 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:41.357 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:34.094405+0000", 2020-09-08T15:45:41.357 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000170853 2020-09-08T15:45:41.358 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.358 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.358 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 4", 2020-09-08T15:45:41.358 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:34.094575+0000", 2020-09-08T15:45:41.358 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0015390079999999999 2020-09-08T15:45:41.358 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.359 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.359 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:41.359 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:34.096114+0000", 2020-09-08T15:45:41.359 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.12920846999999999 2020-09-08T15:45:41.359 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.360 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.360 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:41.360 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:34.225323+0000", 2020-09-08T15:45:41.361 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.8606e-05 2020-09-08T15:45:41.361 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.361 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.361 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:41.361 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:34.225391+0000", 2020-09-08T15:45:41.362 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.4969999999999996e-05 2020-09-08T15:45:41.362 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.362 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.362 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:41.363 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:34.225456+0000", 2020-09-08T15:45:41.363 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13139339999999999 2020-09-08T15:45:41.363 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.363 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.363 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.364 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.364 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.364 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:42772 3.a 3:593c3494:::benchmark_data_smithi166_26097_object42771:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e224)", 2020-09-08T15:45:41.364 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.101773+0000", 2020-09-08T15:45:41.364 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 185.041609701, 2020-09-08T15:45:41.365 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.066228087000000005, 2020-09-08T15:45:41.365 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:41.365 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:41.365 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:41.365 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:45:41.366 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:45:41.366 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 42772 2020-09-08T15:45:41.366 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.366 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:41.366 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.367 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:41.367 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.101773+0000", 2020-09-08T15:45:41.367 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:41.367 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.367 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.368 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:41.368 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.101773+0000", 2020-09-08T15:45:41.368 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999518 2020-09-08T15:45:41.368 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.368 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.368 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:41.369 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.101724+0000", 2020-09-08T15:45:41.369 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015315399999999999 2020-09-08T15:45:41.369 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.369 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.369 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:41.370 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.101878+0000", 2020-09-08T15:45:41.370 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5485000000000002e-05 2020-09-08T15:45:41.370 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.370 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.370 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:41.371 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.101893+0000", 2020-09-08T15:45:41.371 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.7660000000000001e-06 2020-09-08T15:45:41.371 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.372 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.372 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:41.372 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.101900+0000", 2020-09-08T15:45:41.372 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00065410999999999996 2020-09-08T15:45:41.372 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.372 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.373 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.373 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.102554+0000", 2020-09-08T15:45:41.373 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000144647 2020-09-08T15:45:41.373 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.373 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.374 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:41.374 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.102699+0000", 2020-09-08T15:45:41.374 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015233 2020-09-08T15:45:41.374 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.374 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.375 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 4", 2020-09-08T15:45:41.375 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.102851+0000", 2020-09-08T15:45:41.375 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0033049310000000001 2020-09-08T15:45:41.375 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.375 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.376 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:41.376 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106156+0000", 2020-09-08T15:45:41.376 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.061523526000000002 2020-09-08T15:45:41.376 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.376 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.376 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:41.377 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.167679+0000", 2020-09-08T15:45:41.377 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000157672 2020-09-08T15:45:41.377 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.377 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.377 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:41.378 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.167837+0000", 2020-09-08T15:45:41.378 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016381999999999999 2020-09-08T15:45:41.378 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.378 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.378 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:41.379 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.168001+0000", 2020-09-08T15:45:41.379 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.066228087000000005 2020-09-08T15:45:41.379 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.379 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.379 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.380 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.380 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.380 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:42778 3.a 3:57632573:::benchmark_data_smithi166_26097_object42777:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e224)", 2020-09-08T15:45:41.380 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.103653+0000", 2020-09-08T15:45:41.380 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 185.03972935499999, 2020-09-08T15:45:41.381 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.069003771000000005, 2020-09-08T15:45:41.381 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:41.381 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:41.381 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:41.382 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:45:41.382 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:45:41.382 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 42778 2020-09-08T15:45:41.382 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.382 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:41.383 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.383 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:41.383 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103653+0000", 2020-09-08T15:45:41.383 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:41.383 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.384 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.384 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:41.384 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103653+0000", 2020-09-08T15:45:41.384 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999537 2020-09-08T15:45:41.384 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.385 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.385 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:41.385 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103607+0000", 2020-09-08T15:45:41.385 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000144344 2020-09-08T15:45:41.385 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.385 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.386 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:41.386 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103751+0000", 2020-09-08T15:45:41.386 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6113e-05 2020-09-08T15:45:41.386 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.386 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.387 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:41.387 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103767+0000", 2020-09-08T15:45:41.387 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.2029999999999997e-06 2020-09-08T15:45:41.387 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.387 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.388 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:41.388 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103774+0000", 2020-09-08T15:45:41.388 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00065702299999999996 2020-09-08T15:45:41.388 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.388 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.389 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.389 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.104431+0000", 2020-09-08T15:45:41.389 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013896399999999999 2020-09-08T15:45:41.389 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.389 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.390 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:41.390 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.104570+0000", 2020-09-08T15:45:41.390 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000213401 2020-09-08T15:45:41.390 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.390 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.390 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 4", 2020-09-08T15:45:41.391 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.104783+0000", 2020-09-08T15:45:41.391 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0034062910000000001 2020-09-08T15:45:41.391 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.392 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.392 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:41.392 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.108189+0000", 2020-09-08T15:45:41.392 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.064285290999999994 2020-09-08T15:45:41.392 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.393 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.393 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:41.393 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172475+0000", 2020-09-08T15:45:41.393 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010238600000000001 2020-09-08T15:45:41.393 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.394 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.394 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:41.394 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172577+0000", 2020-09-08T15:45:41.394 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.9835999999999999e-05 2020-09-08T15:45:41.394 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.395 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.395 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:41.395 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172657+0000", 2020-09-08T15:45:41.395 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.069003771000000005 2020-09-08T15:45:41.395 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.396 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.396 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.396 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.396 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.396 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:42785 3.a 3:5993af2f:::benchmark_data_smithi166_26097_object42784:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e224)", 2020-09-08T15:45:41.397 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.104749+0000", 2020-09-08T15:45:41.397 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 185.03863338100001, 2020-09-08T15:45:41.397 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.068135928999999998, 2020-09-08T15:45:41.397 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:41.397 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:41.397 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:41.398 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:45:41.398 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:45:41.398 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 42785 2020-09-08T15:45:41.398 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.398 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:41.399 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.399 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:41.399 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.104749+0000", 2020-09-08T15:45:41.399 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:41.399 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.400 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.400 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:41.400 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.104749+0000", 2020-09-08T15:45:41.400 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999576 2020-09-08T15:45:41.400 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.401 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.401 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:41.401 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.104707+0000", 2020-09-08T15:45:41.401 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000147155 2020-09-08T15:45:41.401 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.401 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.402 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:41.402 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.104854+0000", 2020-09-08T15:45:41.403 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5511000000000001e-05 2020-09-08T15:45:41.404 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.404 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.404 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:41.404 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.104869+0000", 2020-09-08T15:45:41.404 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.8449999999999997e-06 2020-09-08T15:45:41.405 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.405 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.405 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:41.405 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.104876+0000", 2020-09-08T15:45:41.405 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0014107849999999999 2020-09-08T15:45:41.406 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.406 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.406 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.406 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106287+0000", 2020-09-08T15:45:41.406 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000181559 2020-09-08T15:45:41.406 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.407 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.407 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:41.407 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106469+0000", 2020-09-08T15:45:41.407 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017944799999999999 2020-09-08T15:45:41.407 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.408 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.408 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 4", 2020-09-08T15:45:41.408 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106648+0000", 2020-09-08T15:45:41.408 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0028664519999999998 2020-09-08T15:45:41.408 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.409 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.409 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:41.409 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.109515+0000", 2020-09-08T15:45:41.409 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.063208075000000002 2020-09-08T15:45:41.409 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.410 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.410 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:41.410 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172723+0000", 2020-09-08T15:45:41.410 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.0462000000000002e-05 2020-09-08T15:45:41.410 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.411 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.411 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:41.411 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172813+0000", 2020-09-08T15:45:41.411 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.1915000000000005e-05 2020-09-08T15:45:41.411 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.411 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.412 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:41.412 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172885+0000", 2020-09-08T15:45:41.412 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.068135928999999998 2020-09-08T15:45:41.413 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.413 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.413 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.413 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.413 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.414 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:42789 3.a 3:58a5da28:::benchmark_data_smithi166_26097_object42788:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e224)", 2020-09-08T15:45:41.414 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.107775+0000", 2020-09-08T15:45:41.414 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 185.03560749499999, 2020-09-08T15:45:41.414 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.065336321000000003, 2020-09-08T15:45:41.414 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:41.415 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:41.415 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:41.415 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:45:41.415 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:45:41.415 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 42789 2020-09-08T15:45:41.416 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.416 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:41.416 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.416 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:41.416 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107775+0000", 2020-09-08T15:45:41.417 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:41.417 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.417 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.417 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:41.417 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107775+0000", 2020-09-08T15:45:41.426 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999633 2020-09-08T15:45:41.427 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.427 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.427 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:41.427 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107738+0000", 2020-09-08T15:45:41.427 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000121884 2020-09-08T15:45:41.428 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.428 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.428 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:41.428 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107860+0000", 2020-09-08T15:45:41.429 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4270000000000001e-05 2020-09-08T15:45:41.429 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.429 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.429 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:41.429 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107874+0000", 2020-09-08T15:45:41.430 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.2890000000000003e-06 2020-09-08T15:45:41.430 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.430 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.430 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:41.430 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107880+0000", 2020-09-08T15:45:41.430 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00033471299999999999 2020-09-08T15:45:41.431 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.431 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.431 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.431 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.108215+0000", 2020-09-08T15:45:41.431 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015263599999999999 2020-09-08T15:45:41.432 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.432 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.432 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:41.432 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.108368+0000", 2020-09-08T15:45:41.432 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012440500000000001 2020-09-08T15:45:41.433 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.433 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.433 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 4", 2020-09-08T15:45:41.433 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.108492+0000", 2020-09-08T15:45:41.433 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0012517279999999999 2020-09-08T15:45:41.434 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.434 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.434 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:41.434 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.109744+0000", 2020-09-08T15:45:41.434 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.063207990000000006 2020-09-08T15:45:41.435 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.435 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.435 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:41.435 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172952+0000", 2020-09-08T15:45:41.435 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.5667000000000005e-05 2020-09-08T15:45:41.436 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.436 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.436 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:41.437 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.173038+0000", 2020-09-08T15:45:41.437 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.3626000000000003e-05 2020-09-08T15:45:41.437 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.437 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.437 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:41.438 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.173111+0000", 2020-09-08T15:45:41.438 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.065336321000000003 2020-09-08T15:45:41.438 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.438 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.438 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.438 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.439 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.439 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:42798 3.a 3:5f026ae2:::benchmark_data_smithi166_26097_object42797:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e224)", 2020-09-08T15:45:41.439 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.111958+0000", 2020-09-08T15:45:41.439 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 185.03142470500001, 2020-09-08T15:45:41.439 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.061450103999999998, 2020-09-08T15:45:41.440 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:41.440 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:41.440 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:41.440 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:45:41.440 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:45:41.441 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 42798 2020-09-08T15:45:41.441 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.441 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:41.441 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.441 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:41.442 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.111958+0000", 2020-09-08T15:45:41.442 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:41.442 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.442 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.442 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:41.443 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.111958+0000", 2020-09-08T15:45:41.443 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999757 2020-09-08T15:45:41.443 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.443 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.443 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:41.443 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.111933+0000", 2020-09-08T15:45:41.444 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.6658999999999998e-05 2020-09-08T15:45:41.444 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.444 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.444 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:41.444 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.112020+0000", 2020-09-08T15:45:41.445 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.55e-06 2020-09-08T15:45:41.445 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.445 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.445 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:41.445 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.112030+0000", 2020-09-08T15:45:41.446 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.8269999999999997e-06 2020-09-08T15:45:41.446 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.446 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.447 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:41.447 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.112035+0000", 2020-09-08T15:45:41.447 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3275999999999995e-05 2020-09-08T15:45:41.447 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.447 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.448 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.448 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.112098+0000", 2020-09-08T15:45:41.448 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014654699999999999 2020-09-08T15:45:41.448 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.448 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.448 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:41.449 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.112244+0000", 2020-09-08T15:45:41.449 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017536200000000001 2020-09-08T15:45:41.449 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.449 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.449 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 4", 2020-09-08T15:45:41.450 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.112420+0000", 2020-09-08T15:45:41.450 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0014296039999999999 2020-09-08T15:45:41.450 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.450 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.450 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:41.451 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.113849+0000", 2020-09-08T15:45:41.451 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.059375710999999998 2020-09-08T15:45:41.451 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.451 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.451 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:41.452 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.173225+0000", 2020-09-08T15:45:41.452 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000105862 2020-09-08T15:45:41.452 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.452 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.452 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:41.452 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.173331+0000", 2020-09-08T15:45:41.453 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.6934999999999994e-05 2020-09-08T15:45:41.453 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.453 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.453 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:41.453 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.173408+0000", 2020-09-08T15:45:41.454 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.061450103999999998 2020-09-08T15:45:41.454 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.454 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.454 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.454 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.455 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.455 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:174393 3.a 3:51162694:::benchmark_data_smithi166_26097_object50232:head [delete] snapc 0=[] ondisk+write+known_if_redirected e271)", 2020-09-08T15:45:41.455 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:45:08.112972+0000", 2020-09-08T15:45:41.455 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 33.030410629000002, 2020-09-08T15:45:41.455 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.087428169999999999, 2020-09-08T15:45:41.456 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:41.456 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:41.456 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:41.456 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:45:41.457 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:45:41.457 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 174393 2020-09-08T15:45:41.457 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.457 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:41.458 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.458 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:41.458 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.112972+0000", 2020-09-08T15:45:41.458 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:41.458 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.459 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.459 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:41.459 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.112972+0000", 2020-09-08T15:45:41.459 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999461 2020-09-08T15:45:41.459 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.459 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.460 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:41.460 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.112918+0000", 2020-09-08T15:45:41.460 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000121241 2020-09-08T15:45:41.460 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.460 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.461 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:41.461 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.113039+0000", 2020-09-08T15:45:41.461 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.2413999999999999e-05 2020-09-08T15:45:41.461 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.461 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.462 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:41.462 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.113062+0000", 2020-09-08T15:45:41.462 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0879999999999999e-05 2020-09-08T15:45:41.462 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.462 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.463 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:41.463 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.113072+0000", 2020-09-08T15:45:41.463 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.3271999999999998e-05 2020-09-08T15:45:41.463 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.464 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.464 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.464 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.113156+0000", 2020-09-08T15:45:41.464 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000156942 2020-09-08T15:45:41.464 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.465 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.465 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:41.465 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.113313+0000", 2020-09-08T15:45:41.465 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00085245999999999998 2020-09-08T15:45:41.465 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.465 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.466 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 4", 2020-09-08T15:45:41.466 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.114165+0000", 2020-09-08T15:45:41.466 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0093092599999999998 2020-09-08T15:45:41.466 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.466 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.467 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:41.467 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.123474+0000", 2020-09-08T15:45:41.467 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.076785161000000005 2020-09-08T15:45:41.468 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.468 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.468 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:41.468 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.200260+0000", 2020-09-08T15:45:41.469 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.9531999999999997e-05 2020-09-08T15:45:41.469 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.469 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.469 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:41.469 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.200339+0000", 2020-09-08T15:45:41.469 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0890000000000001e-05 2020-09-08T15:45:41.470 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.470 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.470 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:41.471 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.200400+0000", 2020-09-08T15:45:41.471 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.087428169999999999 2020-09-08T15:45:41.472 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.472 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.472 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.472 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.473 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.473 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:174398 3.a 3:54917550:::benchmark_data_smithi166_26097_object50237:head [delete] snapc 0=[] ondisk+write+known_if_redirected e271)", 2020-09-08T15:45:41.473 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:45:08.125075+0000", 2020-09-08T15:45:41.473 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 33.018307434999997, 2020-09-08T15:45:41.473 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.095072001000000003, 2020-09-08T15:45:41.474 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:41.475 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:41.475 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:41.476 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:45:41.476 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:45:41.476 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 174398 2020-09-08T15:45:41.476 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.476 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:41.477 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.477 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:41.477 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.125075+0000", 2020-09-08T15:45:41.477 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:41.477 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.478 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.478 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:41.478 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.125075+0000", 2020-09-08T15:45:41.478 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999409 2020-09-08T15:45:41.478 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.479 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.479 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:41.479 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.125016+0000", 2020-09-08T15:45:41.479 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.8330000000000002e-05 2020-09-08T15:45:41.479 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.480 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.480 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:41.480 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.125114+0000", 2020-09-08T15:45:41.480 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3135e-05 2020-09-08T15:45:41.480 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.481 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.481 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:41.481 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.125127+0000", 2020-09-08T15:45:41.481 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.9260000000000003e-06 2020-09-08T15:45:41.481 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.482 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.482 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:41.482 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.125134+0000", 2020-09-08T15:45:41.482 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000106635 2020-09-08T15:45:41.482 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.483 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.483 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.483 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.125241+0000", 2020-09-08T15:45:41.483 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000231456 2020-09-08T15:45:41.483 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.483 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.484 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:41.484 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.125472+0000", 2020-09-08T15:45:41.484 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00025269300000000001 2020-09-08T15:45:41.484 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.484 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.485 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 4", 2020-09-08T15:45:41.485 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.125725+0000", 2020-09-08T15:45:41.485 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0064473300000000002 2020-09-08T15:45:41.485 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.485 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.486 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:41.486 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.132172+0000", 2020-09-08T15:45:41.486 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.087866023000000001 2020-09-08T15:45:41.486 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.486 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.487 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:41.487 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.220038+0000", 2020-09-08T15:45:41.487 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.8022999999999996e-05 2020-09-08T15:45:41.487 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.487 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.488 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:41.488 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.220106+0000", 2020-09-08T15:45:41.488 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.0525000000000002e-05 2020-09-08T15:45:41.488 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.488 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.488 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:41.489 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.220147+0000", 2020-09-08T15:45:41.489 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.095072001000000003 2020-09-08T15:45:41.489 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.489 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.489 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.490 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.490 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.490 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:174412 3.a 3:529e2d5b:::benchmark_data_smithi166_26097_object50251:head [delete] snapc 0=[] ondisk+write+known_if_redirected e271)", 2020-09-08T15:45:41.490 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:45:08.177446+0000", 2020-09-08T15:45:41.490 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 32.965936368999998, 2020-09-08T15:45:41.491 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044083682999999999, 2020-09-08T15:45:41.491 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:41.491 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:41.491 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:41.491 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:45:41.492 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:45:41.492 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 174412 2020-09-08T15:45:41.492 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.492 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:41.492 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.493 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:41.493 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.177446+0000", 2020-09-08T15:45:41.493 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:41.493 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.493 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.494 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:41.494 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.177446+0000", 2020-09-08T15:45:41.494 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999428 2020-09-08T15:45:41.494 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.494 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.494 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:41.495 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.177389+0000", 2020-09-08T15:45:41.495 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012853800000000001 2020-09-08T15:45:41.495 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.495 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.495 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:41.496 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.177517+0000", 2020-09-08T15:45:41.496 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.3802999999999999e-05 2020-09-08T15:45:41.496 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.496 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.496 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:41.497 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.177541+0000", 2020-09-08T15:45:41.497 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2622e-05 2020-09-08T15:45:41.497 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.497 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.497 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:41.498 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.177554+0000", 2020-09-08T15:45:41.498 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000128491 2020-09-08T15:45:41.498 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.498 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.498 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.498 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.177682+0000", 2020-09-08T15:45:41.499 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00022078500000000001 2020-09-08T15:45:41.499 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.499 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.499 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:41.499 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.177903+0000", 2020-09-08T15:45:41.500 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000240604 2020-09-08T15:45:41.500 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.500 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.500 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 4", 2020-09-08T15:45:41.500 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.178144+0000", 2020-09-08T15:45:41.501 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0076425979999999996 2020-09-08T15:45:41.501 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.501 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.501 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:41.501 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.185786+0000", 2020-09-08T15:45:41.502 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.035609057999999999 2020-09-08T15:45:41.502 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.502 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.502 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:41.502 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.221395+0000", 2020-09-08T15:45:41.502 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.6772000000000003e-05 2020-09-08T15:45:41.503 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.503 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.503 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:41.503 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.221472+0000", 2020-09-08T15:45:41.503 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.7751000000000003e-05 2020-09-08T15:45:41.504 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.504 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.504 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:41.504 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.221530+0000", 2020-09-08T15:45:41.504 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044083682999999999 2020-09-08T15:45:41.505 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.505 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.505 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.505 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.505 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.506 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:178736 3.a 3:55ded067:::benchmark_data_smithi166_26097_object54575:head [delete] snapc 0=[] ondisk+write+known_if_redirected e273)", 2020-09-08T15:45:41.506 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:45:14.701091+0000", 2020-09-08T15:45:41.506 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 26.442291857000001, 2020-09-08T15:45:41.506 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.047337088999999999, 2020-09-08T15:45:41.506 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:41.506 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:41.507 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:41.507 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:45:41.507 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:45:41.507 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 178736 2020-09-08T15:45:41.507 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.508 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:41.508 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.508 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:41.508 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.701091+0000", 2020-09-08T15:45:41.508 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:41.509 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.509 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.509 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:41.509 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.701091+0000", 2020-09-08T15:45:41.509 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999671 2020-09-08T15:45:41.509 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.510 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.510 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:41.510 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.701058+0000", 2020-09-08T15:45:41.510 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2015000000000007e-05 2020-09-08T15:45:41.510 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.511 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.511 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:41.511 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.701140+0000", 2020-09-08T15:45:41.511 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4667e-05 2020-09-08T15:45:41.511 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.512 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.512 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:41.512 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.701154+0000", 2020-09-08T15:45:41.512 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.8420000000000001e-06 2020-09-08T15:45:41.512 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.513 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.513 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:41.513 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.701160+0000", 2020-09-08T15:45:41.513 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0044443410000000001 2020-09-08T15:45:41.513 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.514 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.514 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.514 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.705604+0000", 2020-09-08T15:45:41.514 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000120851 2020-09-08T15:45:41.514 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.514 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.515 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:41.515 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.705725+0000", 2020-09-08T15:45:41.515 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000106871 2020-09-08T15:45:41.515 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.515 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.516 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 4", 2020-09-08T15:45:41.516 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.705832+0000", 2020-09-08T15:45:41.516 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.026846339 2020-09-08T15:45:41.516 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.516 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.517 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:41.517 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.732678+0000", 2020-09-08T15:45:41.517 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.015610247000000001 2020-09-08T15:45:41.517 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.517 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.517 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:41.518 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.748289+0000", 2020-09-08T15:45:41.518 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.1470000000000004e-05 2020-09-08T15:45:41.518 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.518 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.518 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:41.519 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.748370+0000", 2020-09-08T15:45:41.519 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.7448000000000003e-05 2020-09-08T15:45:41.519 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.519 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.519 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:41.520 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.748428+0000", 2020-09-08T15:45:41.520 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.047337088999999999 2020-09-08T15:45:41.520 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.520 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.520 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.520 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.521 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.521 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:188883 3.a 3:50405509:::benchmark_data_smithi166_26097_object64722:head [delete] snapc 0=[] ondisk+write+known_if_redirected e277)", 2020-09-08T15:45:41.521 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:45:29.716018+0000", 2020-09-08T15:45:41.521 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 11.427364389999999, 2020-09-08T15:45:41.521 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.054819528999999999, 2020-09-08T15:45:41.522 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:41.522 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:41.522 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:41.522 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:45:41.522 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:45:41.523 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 188883 2020-09-08T15:45:41.523 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.523 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:41.523 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.523 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:41.524 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.716018+0000", 2020-09-08T15:45:41.524 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:41.524 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.524 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.524 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:41.525 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.716018+0000", 2020-09-08T15:45:41.525 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999671 2020-09-08T15:45:41.525 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.525 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.525 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:41.526 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.715985+0000", 2020-09-08T15:45:41.526 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.4047000000000004e-05 2020-09-08T15:45:41.526 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.526 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.526 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:41.526 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.716059+0000", 2020-09-08T15:45:41.527 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.42e-05 2020-09-08T15:45:41.527 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.527 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.527 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:41.527 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.716074+0000", 2020-09-08T15:45:41.528 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.7260000000000002e-06 2020-09-08T15:45:41.528 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.528 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.528 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:41.528 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.716081+0000", 2020-09-08T15:45:41.529 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.0110000000000003e-05 2020-09-08T15:45:41.529 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.529 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.529 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:41.529 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.716171+0000", 2020-09-08T15:45:41.530 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00026574200000000002 2020-09-08T15:45:41.530 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.530 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.530 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:41.530 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.716437+0000", 2020-09-08T15:45:41.530 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00031028000000000002 2020-09-08T15:45:41.531 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.531 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.531 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 4", 2020-09-08T15:45:41.531 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.716747+0000", 2020-09-08T15:45:41.531 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.047940958999999998 2020-09-08T15:45:41.532 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.532 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.532 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:41.532 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.764688+0000", 2020-09-08T15:45:41.532 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0060245009999999998 2020-09-08T15:45:41.533 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.533 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.533 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:41.533 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.770713+0000", 2020-09-08T15:45:41.533 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.6680999999999994e-05 2020-09-08T15:45:41.533 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.534 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.534 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:41.534 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.770780+0000", 2020-09-08T15:45:41.534 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.7975e-05 2020-09-08T15:45:41.534 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:41.535 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:41.535 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:41.535 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.770838+0000", 2020-09-08T15:45:41.535 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.054819528999999999 2020-09-08T15:45:41.535 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.536 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.536 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.536 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:41.536 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:41.536 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:45:41.538 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:40 smithi104 bash[9203]: audit 2020-09-08T15:45:39.778089+0000 mon.a (mon.0) 1252 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}, {\"id\": \"448d5c57-f6de-443a-ab88-169d81b78450\", \"message\": \"Rebalancing after osd.11 marked in\", \"refs\": [[\"osd\", 11]], \"started_at\": 1599579281.6064675, \"finished_at\": 1599579283.5990696}, {\"id\": \"4bd2c04f-3aed-4914-b08b-6531f7786cca\", \"message\": \"Rebalancing after osd.1 marked out\", \"refs\": [[\"osd\", 1]], \"started_at\": 1599579732.6504169, \"finished_at\": 1599579743.653669}, {\"id\": \"1ab6909d-95da-4336-aa7d-9565e86a361d\", \"message\": \"Rebalancing after osd.1 marked in\", \"refs\": [[\"osd\", 1]], \"started_at\": 1599579802.706257, \"finished_at\": 1599579935.6808252}], \"version\": 2, \"compat_version\": 2}"}]: dispatch 2020-09-08T15:45:41.538 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:40 smithi104 bash[9203]: audit 2020-09-08T15:45:39.781833+0000 mon.a (mon.0) 1253 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/progress/completed","val":"{\"events\": [{\"id\": \"c159de0d-d6af-4738-942f-3dd503333f6a\", \"message\": \"Rebalancing after osd.5 marked in\", \"refs\": [[\"osd\", 5]], \"started_at\": 1599579130.5725398, \"finished_at\": 1599579146.7706537}, {\"id\": \"3b998b99-5156-4e6a-9d8e-2ec4f6473474\", \"message\": \"Rebalancing after osd.6 marked in\", \"refs\": [[\"osd\", 6]], \"started_at\": 1599579156.583228, \"finished_at\": 1599579173.4079}, {\"id\": \"2a666e0f-43bc-46e5-babf-7ddeff93137c\", \"message\": \"Rebalancing after osd.7 marked in\", \"refs\": [[\"osd\", 7]], \"started_at\": 1599579195.8454256, \"finished_at\": 1599579203.594702}, {\"id\": \"6a14d0d0-6d63-4722-8194-baf75a1ada8d\", \"message\": \"Rebalancing after osd.8 marked in\", \"refs\": [[\"osd\", 8]], \"started_at\": 1599579205.6062355, \"finished_at\": 1599579221.1546183}, {\"id\": \"448d5c57-f6de-443a-ab88-169d81b78450\", \"message\": \"Rebalancing after osd.11 marked in\", \"refs\": [[\"osd\", 11]], \"started_at\": 1599579281.6064675, \"finished_at\": 1599579283.5990696}, {\"id\": \"4bd2c04f-3aed-4914-b08b-6531f7786cca\", \"message\": \"Rebalancing after osd.1 marked out\", \"refs\": [[\"osd\", 1]], \"started_at\": 1599579732.6504169, \"finished_at\": 1599579743.653669}, {\"id\": \"1ab6909d-95da-4336-aa7d-9565e86a361d\", \"message\": \"Rebalancing after osd.1 marked in\", \"refs\": [[\"osd\", 1]], \"started_at\": 1599579802.706257, \"finished_at\": 1599579935.6808252}], \"version\": 2, \"compat_version\": 2}"}]': finished 2020-09-08T15:45:41.539 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:45:41.581 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:45:41 smithi104 bash[17588]: debug 2020-09-08T15:45:41.246+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:41.640 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:45:41.878 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:45:41.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:41 smithi055 bash[9296]: cluster 2020-09-08T15:45:39.719311+0000 mgr.x (mgr.24118) 550 : cluster [DBG] pgmap v746: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 847 op/s; 8857/107300 objects misplaced (8.254%); 12 MiB/s, 198 objects/s recovering 2020-09-08T15:45:41.912 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:45:41.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:41 smithi104 bash[9203]: cluster 2020-09-08T15:45:39.719311+0000 mgr.x (mgr.24118) 550 : cluster [DBG] pgmap v746: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 847 op/s; 8857/107300 objects misplaced (8.254%); 12 MiB/s, 198 objects/s recovering 2020-09-08T15:45:41.952 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:45:41.994 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:45:42.037 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:45:42.064 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:45:42.085 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:45:42.099 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:45:42.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:41 smithi166 bash[9446]: cluster 2020-09-08T15:45:39.719311+0000 mgr.x (mgr.24118) 550 : cluster [DBG] pgmap v746: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 847 op/s; 8857/107300 objects misplaced (8.254%); 12 MiB/s, 198 objects/s recovering 2020-09-08T15:45:42.130 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:45:42.172 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:45:42.189 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:42.211 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:45:42.233 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2020-09-08T15:45:42.250 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:45:42.288 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:45:42.291 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:45:42.292 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:45:42.292 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:45:42.292 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:45:42.292 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:45:42.293 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:45:42.293 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:45:42.293 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:45:42.293 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:45:42.293 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:45:42.294 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:45:42.294 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:45:42.327 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:42.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:45:42 smithi055 bash[16646]: debug 2020-09-08T15:45:42.089+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:42.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:45:42 smithi055 bash[19003]: debug 2020-09-08T15:45:42.329+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:42.429 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:45:42.431 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:42.579 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:42.682 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:42.692 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:45:42 smithi166 bash[18960]: debug 2020-09-08T15:45:42.578+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:42.798 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:42.902 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:43.025 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:45:42 smithi166 bash[16593]: debug 2020-09-08T15:45:42.798+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:43.026 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:42 smithi166 bash[9446]: cluster 2020-09-08T15:45:41.719852+0000 mgr.x (mgr.24118) 551 : cluster [DBG] pgmap v747: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 617 op/s; 8620/105944 objects misplaced (8.136%); 8.9 MiB/s, 142 objects/s recovering 2020-09-08T15:45:43.026 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:42 smithi166 bash[9446]: audit 2020-09-08T15:45:41.845576+0000 mon.a (mon.0) 1254 : audit [DBG] from='client.? v1:172.21.15.104:0/1050547873' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:45:43.027 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:42 smithi166 bash[9446]: cluster 2020-09-08T15:45:42.041167+0000 osd.6 (osd.6) 37 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:43.027 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:42 smithi166 bash[9446]: cluster 2020-09-08T15:45:42.041263+0000 osd.6 (osd.6) 38 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:43.027 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:45:43 smithi166 bash[14254]: debug 2020-09-08T15:45:43.022+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:43.028 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:43.129 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:43.139 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:42 smithi055 bash[9296]: cluster 2020-09-08T15:45:41.719852+0000 mgr.x (mgr.24118) 551 : cluster [DBG] pgmap v747: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 617 op/s; 8620/105944 objects misplaced (8.136%); 8.9 MiB/s, 142 objects/s recovering 2020-09-08T15:45:43.140 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:42 smithi055 bash[9296]: audit 2020-09-08T15:45:41.845576+0000 mon.a (mon.0) 1254 : audit [DBG] from='client.? v1:172.21.15.104:0/1050547873' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:45:43.141 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:42 smithi055 bash[9296]: cluster 2020-09-08T15:45:42.041167+0000 osd.6 (osd.6) 37 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:43.141 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:42 smithi055 bash[9296]: cluster 2020-09-08T15:45:42.041263+0000 osd.6 (osd.6) 38 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:42 smithi104 bash[9203]: cluster 2020-09-08T15:45:41.719852+0000 mgr.x (mgr.24118) 551 : cluster [DBG] pgmap v747: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 617 op/s; 8620/105944 objects misplaced (8.136%); 8.9 MiB/s, 142 objects/s recovering 2020-09-08T15:45:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:42 smithi104 bash[9203]: audit 2020-09-08T15:45:41.845576+0000 mon.a (mon.0) 1254 : audit [DBG] from='client.? v1:172.21.15.104:0/1050547873' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:45:43.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:42 smithi104 bash[9203]: cluster 2020-09-08T15:45:42.041167+0000 osd.6 (osd.6) 37 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:43.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:42 smithi104 bash[9203]: cluster 2020-09-08T15:45:42.041263+0000 osd.6 (osd.6) 38 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:43.331 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:43.408 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:45:43 smithi055 bash[14301]: debug 2020-09-08T15:45:43.333+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:43.434 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:43.540 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:45:43.707 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:43 smithi055 bash[9296]: cluster 2020-09-08T15:45:42.714915+0000 mon.a (mon.0) 1255 : cluster [DBG] osdmap e281: 12 total, 12 up, 12 in 2020-09-08T15:45:43.712 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:45:43.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:43 smithi104 bash[9203]: cluster 2020-09-08T15:45:42.714915+0000 mon.a (mon.0) 1255 : cluster [DBG] osdmap e281: 12 total, 12 up, 12 in 2020-09-08T15:45:44.025 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:44.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:43 smithi166 bash[9446]: cluster 2020-09-08T15:45:42.714915+0000 mon.a (mon.0) 1255 : cluster [DBG] osdmap e281: 12 total, 12 up, 12 in 2020-09-08T15:45:44.128 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:44.263 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:44.320 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:45:44 smithi055 bash[19003]: debug 2020-09-08T15:45:44.025+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:44.365 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:44.476 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:44.580 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:45:44.616 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:45:44 smithi166 bash[14254]: debug 2020-09-08T15:45:44.258+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:44.625 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:45:44 smithi055 bash[19003]: debug 2020-09-08T15:45:44.473+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:44.803 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:45:44.906 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:44.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:45:44 smithi104 bash[15273]: debug 2020-09-08T15:45:44.802+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:45.038 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:45.142 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:45:45.249 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:45:45 smithi166 bash[18960]: debug 2020-09-08T15:45:45.034+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:45.254 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:45:45.372 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:45.475 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:45.582 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:45.592 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:45:45 smithi166 bash[11928]: debug 2020-09-08T15:45:45.250+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:45.592 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:45:45 smithi166 bash[18960]: debug 2020-09-08T15:45:45.470+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:45.689 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:45:45.706 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:45.809 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:45.860 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:45:45.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:45 smithi166 bash[9446]: cluster 2020-09-08T15:45:43.720391+0000 mgr.x (mgr.24118) 552 : cluster [DBG] pgmap v749: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 731 op/s; 7612/100886 objects misplaced (7.545%); 13 MiB/s, 203 objects/s recovering 2020-09-08T15:45:45.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:45 smithi166 bash[9446]: cluster 2020-09-08T15:45:44.267772+0000 osd.11 (osd.11) 35 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:45.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:45 smithi166 bash[9446]: cluster 2020-09-08T15:45:44.267860+0000 osd.11 (osd.11) 36 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:45.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:45 smithi166 bash[9446]: cluster 2020-09-08T15:45:44.755598+0000 mon.a (mon.0) 1256 : cluster [DBG] osdmap e282: 12 total, 12 up, 12 in 2020-09-08T15:45:45.867 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:45:45 smithi166 bash[14254]: debug 2020-09-08T15:45:45.702+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:46.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:45 smithi055 bash[9296]: cluster 2020-09-08T15:45:43.720391+0000 mgr.x (mgr.24118) 552 : cluster [DBG] pgmap v749: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 731 op/s; 7612/100886 objects misplaced (7.545%); 13 MiB/s, 203 objects/s recovering 2020-09-08T15:45:46.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:45 smithi055 bash[9296]: cluster 2020-09-08T15:45:44.267772+0000 osd.11 (osd.11) 35 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:46.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:45 smithi055 bash[9296]: cluster 2020-09-08T15:45:44.267860+0000 osd.11 (osd.11) 36 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:46.162 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:45 smithi055 bash[9296]: cluster 2020-09-08T15:45:44.755598+0000 mon.a (mon.0) 1256 : cluster [DBG] osdmap e282: 12 total, 12 up, 12 in 2020-09-08T15:45:46.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:45 smithi104 bash[9203]: cluster 2020-09-08T15:45:43.720391+0000 mgr.x (mgr.24118) 552 : cluster [DBG] pgmap v749: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 731 op/s; 7612/100886 objects misplaced (7.545%); 13 MiB/s, 203 objects/s recovering 2020-09-08T15:45:46.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:45 smithi104 bash[9203]: cluster 2020-09-08T15:45:44.267772+0000 osd.11 (osd.11) 35 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:46.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:45 smithi104 bash[9203]: cluster 2020-09-08T15:45:44.267860+0000 osd.11 (osd.11) 36 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:46.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:45 smithi104 bash[9203]: cluster 2020-09-08T15:45:44.755598+0000 mon.a (mon.0) 1256 : cluster [DBG] osdmap e282: 12 total, 12 up, 12 in 2020-09-08T15:45:46.355 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:46.457 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:46.662 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:45:46 smithi055 bash[11956]: debug 2020-09-08T15:45:46.353+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:47.096 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:47.210 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:47.366 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:45:47 smithi055 bash[19003]: debug 2020-09-08T15:45:47.097+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:47.367 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:47.471 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:47.605 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:47.658 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:45:47 smithi055 bash[11956]: debug 2020-09-08T15:45:47.357+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:47.711 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:45:47.713 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:45:47.714 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:45:47.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:45:47 smithi166 bash[14254]: debug 2020-09-08T15:45:47.606+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:47.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:47 smithi166 bash[9446]: cluster 2020-09-08T15:45:45.720855+0000 mgr.x (mgr.24118) 553 : cluster [DBG] pgmap v751: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 587 op/s; 7612/99982 objects misplaced (7.613%); 8.8 MiB/s, 140 objects/s recovering 2020-09-08T15:45:47.989 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:47 smithi104 bash[9203]: cluster 2020-09-08T15:45:45.720855+0000 mgr.x (mgr.24118) 553 : cluster [DBG] pgmap v751: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 587 op/s; 7612/99982 objects misplaced (7.613%); 8.8 MiB/s, 140 objects/s recovering 2020-09-08T15:45:47.990 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:45:48.093 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:45:48.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:47 smithi055 bash[9296]: cluster 2020-09-08T15:45:45.720855+0000 mgr.x (mgr.24118) 553 : cluster [DBG] pgmap v751: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 3.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 587 op/s; 7612/99982 objects misplaced (7.613%); 8.8 MiB/s, 140 objects/s recovering 2020-09-08T15:45:48.217 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:45:48.323 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:45:48.335 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:45:47 smithi104 bash[17588]: debug 2020-09-08T15:45:47.990+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:48.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:45:48 smithi166 bash[11928]: debug 2020-09-08T15:45:48.214+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:48.815 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2020-09-08T15:45:48.966 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:45:49.102 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:45:49.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:45:49 smithi104 bash[22305]: debug 2020-09-08T15:45:49.102+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:49.204 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:45:50.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:49 smithi166 bash[9446]: cluster 2020-09-08T15:45:47.721303+0000 mgr.x (mgr.24118) 554 : cluster [DBG] pgmap v752: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 794 op/s; 6638/94586 objects misplaced (7.018%); 13 MiB/s, 214 objects/s recovering 2020-09-08T15:45:50.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:49 smithi166 bash[9446]: cluster 2020-09-08T15:45:48.299883+0000 osd.4 (osd.4) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:50.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:49 smithi166 bash[9446]: cluster 2020-09-08T15:45:48.299988+0000 osd.4 (osd.4) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:50.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:49 smithi166 bash[9446]: cluster 2020-09-08T15:45:48.698086+0000 mon.a (mon.0) 1257 : cluster [DBG] osdmap e283: 12 total, 12 up, 12 in 2020-09-08T15:45:50.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:49 smithi166 bash[9446]: cluster 2020-09-08T15:45:49.676341+0000 mon.a (mon.0) 1258 : cluster [DBG] osdmap e284: 12 total, 12 up, 12 in 2020-09-08T15:45:50.144 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:45:50.145 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:49 smithi104 bash[9203]: cluster 2020-09-08T15:45:47.721303+0000 mgr.x (mgr.24118) 554 : cluster [DBG] pgmap v752: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 794 op/s; 6638/94586 objects misplaced (7.018%); 13 MiB/s, 214 objects/s recovering 2020-09-08T15:45:50.145 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:49 smithi104 bash[9203]: cluster 2020-09-08T15:45:48.299883+0000 osd.4 (osd.4) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:50.145 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:49 smithi104 bash[9203]: cluster 2020-09-08T15:45:48.299988+0000 osd.4 (osd.4) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:50.146 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:49 smithi104 bash[9203]: cluster 2020-09-08T15:45:48.698086+0000 mon.a (mon.0) 1257 : cluster [DBG] osdmap e283: 12 total, 12 up, 12 in 2020-09-08T15:45:50.146 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:49 smithi104 bash[9203]: cluster 2020-09-08T15:45:49.676341+0000 mon.a (mon.0) 1258 : cluster [DBG] osdmap e284: 12 total, 12 up, 12 in 2020-09-08T15:45:50.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:49 smithi055 bash[9296]: cluster 2020-09-08T15:45:47.721303+0000 mgr.x (mgr.24118) 554 : cluster [DBG] pgmap v752: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 794 op/s; 6638/94586 objects misplaced (7.018%); 13 MiB/s, 214 objects/s recovering 2020-09-08T15:45:50.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:49 smithi055 bash[9296]: cluster 2020-09-08T15:45:48.299883+0000 osd.4 (osd.4) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:50.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:49 smithi055 bash[9296]: cluster 2020-09-08T15:45:48.299988+0000 osd.4 (osd.4) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:50.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:49 smithi055 bash[9296]: cluster 2020-09-08T15:45:48.698086+0000 mon.a (mon.0) 1257 : cluster [DBG] osdmap e283: 12 total, 12 up, 12 in 2020-09-08T15:45:50.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:49 smithi055 bash[9296]: cluster 2020-09-08T15:45:49.676341+0000 mon.a (mon.0) 1258 : cluster [DBG] osdmap e284: 12 total, 12 up, 12 in 2020-09-08T15:45:50.247 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:50.371 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:50.429 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:45:50 smithi104 bash[17588]: debug 2020-09-08T15:45:50.146+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:50.474 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:50.588 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:50.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:45:50 smithi166 bash[14254]: debug 2020-09-08T15:45:50.370+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:50.693 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:50.706 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:45:50 smithi055 bash[14301]: debug 2020-09-08T15:45:50.593+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:50.838 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:50.944 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:51.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:45:50 smithi055 bash[19003]: debug 2020-09-08T15:45:50.841+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:51.697 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:51.800 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:45:51.973 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:51 smithi104 bash[9203]: cluster 2020-09-08T15:45:49.721659+0000 mgr.x (mgr.24118) 555 : cluster [DBG] pgmap v755: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 737 op/s; 6395/92038 objects misplaced (6.948%); 10 MiB/s, 161 objects/s recovering 2020-09-08T15:45:51.974 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:45:52.078 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:45:52.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:51 smithi166 bash[9446]: cluster 2020-09-08T15:45:49.721659+0000 mgr.x (mgr.24118) 555 : cluster [DBG] pgmap v755: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 737 op/s; 6395/92038 objects misplaced (6.948%); 10 MiB/s, 161 objects/s recovering 2020-09-08T15:45:52.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:45:51 smithi055 bash[11956]: debug 2020-09-08T15:45:51.697+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:52.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:51 smithi055 bash[9296]: cluster 2020-09-08T15:45:49.721659+0000 mgr.x (mgr.24118) 555 : cluster [DBG] pgmap v755: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 737 op/s; 6395/92038 objects misplaced (6.948%); 10 MiB/s, 161 objects/s recovering 2020-09-08T15:45:52.432 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:45:51 smithi104 bash[19922]: debug 2020-09-08T15:45:51.974+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:52.606 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:45:52.709 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:52.711 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:45:52.927 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:45:52 smithi104 bash[19922]: debug 2020-09-08T15:45:52.606+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:52.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:52 smithi104 bash[9203]: cluster 2020-09-08T15:45:51.722108+0000 mgr.x (mgr.24118) 556 : cluster [DBG] pgmap v756: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 679 op/s; 6129/90508 objects misplaced (6.772%); 11 MiB/s, 177 objects/s recovering 2020-09-08T15:45:52.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:52 smithi104 bash[9203]: audit 2020-09-08T15:45:52.296960+0000 mon.a (mon.0) 1259 : audit [INF] from='client.? v1:172.21.15.104:0/584683585' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:45:53.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:52 smithi166 bash[9446]: cluster 2020-09-08T15:45:51.722108+0000 mgr.x (mgr.24118) 556 : cluster [DBG] pgmap v756: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 679 op/s; 6129/90508 objects misplaced (6.772%); 11 MiB/s, 177 objects/s recovering 2020-09-08T15:45:53.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:52 smithi166 bash[9446]: audit 2020-09-08T15:45:52.296960+0000 mon.a (mon.0) 1259 : audit [INF] from='client.? v1:172.21.15.104:0/584683585' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:45:53.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:52 smithi055 bash[9296]: cluster 2020-09-08T15:45:51.722108+0000 mgr.x (mgr.24118) 556 : cluster [DBG] pgmap v756: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 679 op/s; 6129/90508 objects misplaced (6.772%); 11 MiB/s, 177 objects/s recovering 2020-09-08T15:45:53.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:52 smithi055 bash[9296]: audit 2020-09-08T15:45:52.296960+0000 mon.a (mon.0) 1259 : audit [INF] from='client.? v1:172.21.15.104:0/584683585' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:45:53.523 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:45:53.556 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:45:53.597 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:45:53.635 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:45:53.666 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:45:53.667 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:45:53.676 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:45:53.699 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:53 smithi055 bash[9296]: cluster 2020-09-08T15:45:52.700799+0000 mon.a (mon.0) 1260 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:45:53.699 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:53 smithi055 bash[9296]: audit 2020-09-08T15:45:52.712701+0000 mon.a (mon.0) 1261 : audit [INF] from='client.? v1:172.21.15.104:0/584683585' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:45:53.699 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:53 smithi055 bash[9296]: cluster 2020-09-08T15:45:52.712750+0000 mon.a (mon.0) 1262 : cluster [DBG] osdmap e285: 12 total, 12 up, 12 in 2020-09-08T15:45:53.710 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:53 smithi055 bash[9296]: audit 2020-09-08T15:45:53.473308+0000 mon.a (mon.0) 1263 : audit [DBG] from='client.? v1:172.21.15.104:0/3823034523' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:45:53.710 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:53 smithi055 bash[9296]: audit 2020-09-08T15:45:53.479849+0000 mon.c (mon.2) 49 : audit [DBG] from='client.15381 v1:172.21.15.104:0/3823034523' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:45:53.721 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:45:53.755 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:45:53.784 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:45:53.824 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:45:53.860 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:45:53.901 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:45:53.947 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:45:53.948 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:45:53.948 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:45:53.948 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:45:53.949 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:45:53.949 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:45:53.949 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:45:53.949 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:45:53.949 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:45:53.950 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:45:53.951 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:45:53.951 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:45:53.952 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:45:54.025 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:53 smithi104 bash[9203]: cluster 2020-09-08T15:45:52.700799+0000 mon.a (mon.0) 1260 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:45:54.025 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:53 smithi104 bash[9203]: audit 2020-09-08T15:45:52.712701+0000 mon.a (mon.0) 1261 : audit [INF] from='client.? v1:172.21.15.104:0/584683585' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:45:54.025 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:53 smithi104 bash[9203]: cluster 2020-09-08T15:45:52.712750+0000 mon.a (mon.0) 1262 : cluster [DBG] osdmap e285: 12 total, 12 up, 12 in 2020-09-08T15:45:54.026 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:53 smithi104 bash[9203]: audit 2020-09-08T15:45:53.473308+0000 mon.a (mon.0) 1263 : audit [DBG] from='client.? v1:172.21.15.104:0/3823034523' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:45:54.026 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:53 smithi104 bash[9203]: audit 2020-09-08T15:45:53.479849+0000 mon.c (mon.2) 49 : audit [DBG] from='client.15381 v1:172.21.15.104:0/3823034523' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:45:54.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:53 smithi166 bash[9446]: cluster 2020-09-08T15:45:52.700799+0000 mon.a (mon.0) 1260 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:45:54.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:53 smithi166 bash[9446]: audit 2020-09-08T15:45:52.712701+0000 mon.a (mon.0) 1261 : audit [INF] from='client.? v1:172.21.15.104:0/584683585' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:45:54.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:53 smithi166 bash[9446]: cluster 2020-09-08T15:45:52.712750+0000 mon.a (mon.0) 1262 : cluster [DBG] osdmap e285: 12 total, 12 up, 12 in 2020-09-08T15:45:54.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:53 smithi166 bash[9446]: audit 2020-09-08T15:45:53.473308+0000 mon.a (mon.0) 1263 : audit [DBG] from='client.? v1:172.21.15.104:0/3823034523' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:45:54.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:53 smithi166 bash[9446]: audit 2020-09-08T15:45:53.479849+0000 mon.c (mon.2) 49 : audit [DBG] from='client.15381 v1:172.21.15.104:0/3823034523' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:45:54.635 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2020-09-08T15:45:54.795 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:45:54.923 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:55.024 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:55.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:45:54 smithi166 bash[14254]: debug 2020-09-08T15:45:54.914+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:55.488 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:45:55.592 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:55.910 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:45:55 smithi055 bash[11956]: debug 2020-09-08T15:45:55.489+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:55.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:55 smithi055 bash[9296]: cluster 2020-09-08T15:45:53.722708+0000 mgr.x (mgr.24118) 557 : cluster [DBG] pgmap v758: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 781 op/s; 5172/85210 objects misplaced (6.070%); 13 MiB/s, 205 objects/s recovering 2020-09-08T15:45:55.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:55 smithi055 bash[9296]: cluster 2020-09-08T15:45:54.677760+0000 mon.a (mon.0) 1264 : cluster [DBG] osdmap e286: 12 total, 12 up, 12 in 2020-09-08T15:45:55.936 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:55 smithi104 bash[9203]: cluster 2020-09-08T15:45:53.722708+0000 mgr.x (mgr.24118) 557 : cluster [DBG] pgmap v758: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 781 op/s; 5172/85210 objects misplaced (6.070%); 13 MiB/s, 205 objects/s recovering 2020-09-08T15:45:55.936 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:55 smithi104 bash[9203]: cluster 2020-09-08T15:45:54.677760+0000 mon.a (mon.0) 1264 : cluster [DBG] osdmap e286: 12 total, 12 up, 12 in 2020-09-08T15:45:56.062 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:56.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:55 smithi166 bash[9446]: cluster 2020-09-08T15:45:53.722708+0000 mgr.x (mgr.24118) 557 : cluster [DBG] pgmap v758: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 781 op/s; 5172/85210 objects misplaced (6.070%); 13 MiB/s, 205 objects/s recovering 2020-09-08T15:45:56.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:55 smithi166 bash[9446]: cluster 2020-09-08T15:45:54.677760+0000 mon.a (mon.0) 1264 : cluster [DBG] osdmap e286: 12 total, 12 up, 12 in 2020-09-08T15:45:56.160 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:45:56.166 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:45:56.324 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:45:56.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:45:56 smithi055 bash[14301]: debug 2020-09-08T15:45:56.065+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:56.507 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:45:56.609 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:56.732 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:56.836 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:45:56.850 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:45:56 smithi055 bash[14301]: debug 2020-09-08T15:45:56.733+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:56.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:45:56 smithi166 bash[11928]: debug 2020-09-08T15:45:56.506+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:57.101 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:45:57.268 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:45:57.446 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:45:57.550 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:45:57.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:57 smithi104 bash[9203]: cluster 2020-09-08T15:45:55.723184+0000 mgr.x (mgr.24118) 558 : cluster [DBG] pgmap v760: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 642 op/s; 5172/84268 objects misplaced (6.138%); 10 MiB/s, 163 objects/s recovering 2020-09-08T15:45:57.927 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:45:57 smithi104 bash[19922]: debug 2020-09-08T15:45:57.446+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:58.089 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:45:58.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:57 smithi166 bash[9446]: cluster 2020-09-08T15:45:55.723184+0000 mgr.x (mgr.24118) 558 : cluster [DBG] pgmap v760: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 642 op/s; 5172/84268 objects misplaced (6.138%); 10 MiB/s, 163 objects/s recovering 2020-09-08T15:45:58.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:57 smithi055 bash[9296]: cluster 2020-09-08T15:45:55.723184+0000 mgr.x (mgr.24118) 558 : cluster [DBG] pgmap v760: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 642 op/s; 5172/84268 objects misplaced (6.138%); 10 MiB/s, 163 objects/s recovering 2020-09-08T15:45:58.200 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:58.434 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:45:58 smithi104 bash[19922]: debug 2020-09-08T15:45:58.090+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:58.594 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:45:58.698 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:58.843 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:45:58.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:58 smithi055 bash[9296]: cluster 2020-09-08T15:45:57.351918+0000 osd.9 (osd.9) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:58.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:58 smithi055 bash[9296]: cluster 2020-09-08T15:45:57.352001+0000 osd.9 (osd.9) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:58.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:58 smithi055 bash[9296]: cluster 2020-09-08T15:45:57.688257+0000 mon.a (mon.0) 1265 : cluster [DBG] osdmap e287: 12 total, 12 up, 12 in 2020-09-08T15:45:58.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:45:58 smithi055 bash[14301]: debug 2020-09-08T15:45:58.597+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:58.943 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:59.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:58 smithi166 bash[9446]: cluster 2020-09-08T15:45:57.351918+0000 osd.9 (osd.9) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:59.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:58 smithi166 bash[9446]: cluster 2020-09-08T15:45:57.352001+0000 osd.9 (osd.9) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:59.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:58 smithi166 bash[9446]: cluster 2020-09-08T15:45:57.688257+0000 mon.a (mon.0) 1265 : cluster [DBG] osdmap e287: 12 total, 12 up, 12 in 2020-09-08T15:45:59.117 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:45:58 smithi166 bash[14254]: debug 2020-09-08T15:45:58.826+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:59.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:58 smithi104 bash[9203]: cluster 2020-09-08T15:45:57.351918+0000 osd.9 (osd.9) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:45:59.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:58 smithi104 bash[9203]: cluster 2020-09-08T15:45:57.352001+0000 osd.9 (osd.9) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:45:59.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:58 smithi104 bash[9203]: cluster 2020-09-08T15:45:57.688257+0000 mon.a (mon.0) 1265 : cluster [DBG] osdmap e287: 12 total, 12 up, 12 in 2020-09-08T15:45:59.276 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:45:59.378 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:59.498 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:45:59.560 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:45:59 smithi055 bash[19003]: debug 2020-09-08T15:45:59.272+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:59.605 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:59.700 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:45:59 smithi166 bash[18960]: debug 2020-09-08T15:45:59.494+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:45:59.700 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:59 smithi166 bash[9446]: cluster 2020-09-08T15:45:57.723710+0000 mgr.x (mgr.24118) 559 : cluster [DBG] pgmap v762: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 942 op/s; 4118/79192 objects misplaced (5.200%); 16 MiB/s, 253 objects/s recovering 2020-09-08T15:45:59.701 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:45:59 smithi166 bash[9446]: cluster 2020-09-08T15:45:59.681421+0000 mon.a (mon.0) 1266 : cluster [DBG] osdmap e288: 12 total, 12 up, 12 in 2020-09-08T15:45:59.728 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:45:59.815 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:45:59.816 INFO:teuthology.orchestra.run.smithi055.stdout: "size": 20, 2020-09-08T15:45:59.816 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 600, 2020-09-08T15:45:59.816 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T15:45:59.817 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.817 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89439 2.0 2:0e36dded:::benchmark_data_smithi166_20551_object89438:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:45:59.818 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.174161+0000", 2020-09-08T15:45:59.818 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 455.640884035, 2020-09-08T15:45:59.819 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.21937716400000001, 2020-09-08T15:45:59.819 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:59.819 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:59.820 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:59.820 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:45:59.820 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:45:59.820 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89439 2020-09-08T15:45:59.821 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.821 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:59.821 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.821 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:59.822 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.174161+0000", 2020-09-08T15:45:59.823 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:59.823 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.823 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.823 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:59.824 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.174161+0000", 2020-09-08T15:45:59.824 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999681 2020-09-08T15:45:59.824 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.824 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.825 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:59.825 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.174129+0000", 2020-09-08T15:45:59.826 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000103976 2020-09-08T15:45:59.826 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.826 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.827 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:59.827 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.174233+0000", 2020-09-08T15:45:59.833 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.396e-05 2020-09-08T15:45:59.834 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.834 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.834 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:59.835 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.174247+0000", 2020-09-08T15:45:59.835 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9159999999999998e-06 2020-09-08T15:45:59.835 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.836 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.836 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:59.836 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.174253+0000", 2020-09-08T15:45:59.837 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0038480929999999999 2020-09-08T15:45:59.837 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.837 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.838 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:59.838 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.178101+0000", 2020-09-08T15:45:59.838 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00028278999999999999 2020-09-08T15:45:59.838 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.839 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.840 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:59.840 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.178383+0000", 2020-09-08T15:45:59.840 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00034010699999999997 2020-09-08T15:45:59.849 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.850 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.851 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:45:59.851 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.178724+0000", 2020-09-08T15:45:59.851 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0053716789999999999 2020-09-08T15:45:59.852 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.852 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.852 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:59.853 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.184095+0000", 2020-09-08T15:45:59.855 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.20911371300000001 2020-09-08T15:45:59.856 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.856 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.859 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:59.859 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.393209+0000", 2020-09-08T15:45:59.859 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00018925499999999999 2020-09-08T15:45:59.860 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.860 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.860 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:59.860 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.393398+0000", 2020-09-08T15:45:59.861 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013971799999999999 2020-09-08T15:45:59.861 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.861 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.861 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:59.862 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.393538+0000", 2020-09-08T15:45:59.862 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.21937716400000001 2020-09-08T15:45:59.862 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:59.862 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:59.863 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:59.863 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.863 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.863 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89441 2.0 2:0bc9b942:::benchmark_data_smithi166_20551_object89440:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:45:59.863 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.175434+0000", 2020-09-08T15:45:59.864 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 455.63961087400003, 2020-09-08T15:45:59.864 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.21857064800000001, 2020-09-08T15:45:59.864 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:59.864 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:59.864 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:59.865 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:45:59.865 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:45:59.865 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89441 2020-09-08T15:45:59.865 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.865 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:59.866 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.866 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:59.866 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.175434+0000", 2020-09-08T15:45:59.867 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:59.867 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.870 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.870 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:59.870 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.175434+0000", 2020-09-08T15:45:59.870 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999762 2020-09-08T15:45:59.870 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.871 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.871 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:59.871 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.175410+0000", 2020-09-08T15:45:59.871 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3329e-05 2020-09-08T15:45:59.871 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.872 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.872 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:59.872 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.175503+0000", 2020-09-08T15:45:59.872 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.9150000000000008e-06 2020-09-08T15:45:59.872 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.873 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.873 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:59.873 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.175513+0000", 2020-09-08T15:45:59.873 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2959999999999998e-06 2020-09-08T15:45:59.873 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.874 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.874 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:59.874 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.175518+0000", 2020-09-08T15:45:59.874 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0059908849999999996 2020-09-08T15:45:59.875 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.875 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.875 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:59.875 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.181509+0000", 2020-09-08T15:45:59.875 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011476699999999999 2020-09-08T15:45:59.876 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.876 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.876 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:59.876 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.181624+0000", 2020-09-08T15:45:59.876 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000119091 2020-09-08T15:45:59.876 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.877 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.877 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:45:59.877 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.181743+0000", 2020-09-08T15:45:59.877 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0087421660000000009 2020-09-08T15:45:59.878 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.878 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.878 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:59.878 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.190485+0000", 2020-09-08T15:45:59.879 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.20319680900000001 2020-09-08T15:45:59.879 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.879 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.879 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:59.879 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.393682+0000", 2020-09-08T15:45:59.880 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00021175400000000001 2020-09-08T15:45:59.880 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.880 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.880 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:59.880 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.393894+0000", 2020-09-08T15:45:59.881 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000110592 2020-09-08T15:45:59.881 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.881 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.881 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:59.881 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.394005+0000", 2020-09-08T15:45:59.882 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.21857064800000001 2020-09-08T15:45:59.882 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:59.882 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:59.882 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:59.882 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.883 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.883 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89446 2.0 2:0947ac70:::benchmark_data_smithi166_20551_object89445:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:45:59.883 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.179773+0000", 2020-09-08T15:45:59.883 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 455.63527192700002, 2020-09-08T15:45:59.883 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.226094445, 2020-09-08T15:45:59.884 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:59.884 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:59.884 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:59.884 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:45:59.884 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:45:59.885 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89446 2020-09-08T15:45:59.885 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.885 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:59.885 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.885 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:59.886 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.179773+0000", 2020-09-08T15:45:59.886 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:59.886 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.893 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.895 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:59.895 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.179773+0000", 2020-09-08T15:45:59.895 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999599 2020-09-08T15:45:59.895 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.896 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.896 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:59.896 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.179733+0000", 2020-09-08T15:45:59.896 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000139274 2020-09-08T15:45:59.896 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.897 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.897 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:59.897 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.179872+0000", 2020-09-08T15:45:59.897 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.7482999999999998e-05 2020-09-08T15:45:59.897 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.898 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.898 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:59.898 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.179889+0000", 2020-09-08T15:45:59.898 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.3039999999999999e-06 2020-09-08T15:45:59.898 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.899 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.899 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:59.899 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.179897+0000", 2020-09-08T15:45:59.899 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0042542550000000002 2020-09-08T15:45:59.899 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.900 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.900 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:59.900 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.184151+0000", 2020-09-08T15:45:59.900 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019428300000000001 2020-09-08T15:45:59.900 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.901 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.901 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:59.901 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.184345+0000", 2020-09-08T15:45:59.901 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000216963 2020-09-08T15:45:59.901 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.902 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.902 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:45:59.902 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.184562+0000", 2020-09-08T15:45:59.902 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0059322400000000001 2020-09-08T15:45:59.903 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.903 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.903 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:59.903 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.190494+0000", 2020-09-08T15:45:59.903 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.21496723700000001 2020-09-08T15:45:59.904 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.904 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.904 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:59.904 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.405462+0000", 2020-09-08T15:45:59.904 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00021017400000000001 2020-09-08T15:45:59.905 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.905 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.905 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:59.905 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.405672+0000", 2020-09-08T15:45:59.905 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019537999999999999 2020-09-08T15:45:59.907 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.907 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.907 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:59.908 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.405867+0000", 2020-09-08T15:45:59.908 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.226094445 2020-09-08T15:45:59.908 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:59.908 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:59.908 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:59.909 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.909 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.909 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89460 2.0 2:086076a4:::benchmark_data_smithi166_20551_object89459:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:45:59.909 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.186439+0000", 2020-09-08T15:45:59.909 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 455.62860607499999, 2020-09-08T15:45:59.910 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.22460018800000001, 2020-09-08T15:45:59.910 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:59.910 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:59.910 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:59.910 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:45:59.911 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:45:59.911 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89460 2020-09-08T15:45:59.911 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.911 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:59.911 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.912 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:59.912 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.186439+0000", 2020-09-08T15:45:59.912 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:59.912 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.912 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.913 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:59.913 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.186439+0000", 2020-09-08T15:45:59.913 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999571 2020-09-08T15:45:59.913 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.913 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.914 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:59.914 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.186396+0000", 2020-09-08T15:45:59.914 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000159894 2020-09-08T15:45:59.914 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.914 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.915 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:59.915 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.186556+0000", 2020-09-08T15:45:59.915 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6770999999999999e-05 2020-09-08T15:45:59.915 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.915 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.916 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:59.916 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.186572+0000", 2020-09-08T15:45:59.916 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0795000000000001e-05 2020-09-08T15:45:59.916 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.916 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.917 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:59.917 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.186583+0000", 2020-09-08T15:45:59.917 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00075302799999999999 2020-09-08T15:45:59.917 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.917 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.918 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:59.918 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.187336+0000", 2020-09-08T15:45:59.918 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016681399999999999 2020-09-08T15:45:59.918 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.918 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.919 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:59.919 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.187503+0000", 2020-09-08T15:45:59.919 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00021730600000000001 2020-09-08T15:45:59.919 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.919 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.920 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:45:59.920 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.187720+0000", 2020-09-08T15:45:59.920 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0031015919999999998 2020-09-08T15:45:59.920 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.920 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.921 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:59.921 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.190822+0000", 2020-09-08T15:45:59.921 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.21988956600000001 2020-09-08T15:45:59.921 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.921 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.922 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:59.922 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.410712+0000", 2020-09-08T15:45:59.922 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000173163 2020-09-08T15:45:59.922 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.922 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.923 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:59.923 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.410885+0000", 2020-09-08T15:45:59.923 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015414399999999999 2020-09-08T15:45:59.924 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.924 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.924 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:59.924 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.411039+0000", 2020-09-08T15:45:59.924 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.22460018800000001 2020-09-08T15:45:59.924 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:59.925 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:59.925 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:59.925 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.925 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.925 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89516 2.0 2:088ea7fe:::benchmark_data_smithi166_20551_object89515:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:45:59.926 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.223866+0000", 2020-09-08T15:45:59.926 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 455.59117878000001, 2020-09-08T15:45:59.926 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.18755557, 2020-09-08T15:45:59.926 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:59.926 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:59.927 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:59.927 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:45:59.927 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:45:59.927 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89516 2020-09-08T15:45:59.927 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.928 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:59.928 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.928 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:59.928 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.223866+0000", 2020-09-08T15:45:59.928 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:59.929 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.929 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.929 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:59.929 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.223866+0000", 2020-09-08T15:45:59.929 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999723 2020-09-08T15:45:59.930 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.930 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.930 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:59.930 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.223838+0000", 2020-09-08T15:45:59.930 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3925000000000006e-05 2020-09-08T15:45:59.931 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.931 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.931 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:59.931 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.223932+0000", 2020-09-08T15:45:59.931 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.0628999999999998e-05 2020-09-08T15:45:59.932 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.932 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.932 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:59.932 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.223953+0000", 2020-09-08T15:45:59.932 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.1099999999999992e-06 2020-09-08T15:45:59.933 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.933 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.933 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:59.933 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.223962+0000", 2020-09-08T15:45:59.934 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.8196999999999999e-05 2020-09-08T15:45:59.934 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.934 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.936 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:59.936 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.224010+0000", 2020-09-08T15:45:59.937 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012655399999999999 2020-09-08T15:45:59.937 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.937 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.937 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:59.937 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.224137+0000", 2020-09-08T15:45:59.938 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000121923 2020-09-08T15:45:59.938 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.938 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.938 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:45:59.938 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.224259+0000", 2020-09-08T15:45:59.939 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0066709400000000002 2020-09-08T15:45:59.940 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.940 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.940 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:59.940 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.230930+0000", 2020-09-08T15:45:59.941 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.180219667 2020-09-08T15:45:59.941 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.941 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.941 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:59.941 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.411149+0000", 2020-09-08T15:45:59.942 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000127995 2020-09-08T15:45:59.942 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.942 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.944 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:59.944 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.411277+0000", 2020-09-08T15:45:59.945 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000144291 2020-09-08T15:45:59.945 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.945 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.945 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:59.945 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.411422+0000", 2020-09-08T15:45:59.946 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.18755557 2020-09-08T15:45:59.946 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:59.946 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:59.946 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:59.946 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.947 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.947 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89520 2.0 2:044f3cb9:::benchmark_data_smithi166_20551_object89519:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:45:59.947 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.225510+0000", 2020-09-08T15:45:59.947 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 455.58953456900002, 2020-09-08T15:45:59.947 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.189580258, 2020-09-08T15:45:59.948 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:59.948 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:59.948 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:59.948 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:45:59.948 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:45:59.949 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89520 2020-09-08T15:45:59.949 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.949 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:59.949 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.949 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:59.950 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.225510+0000", 2020-09-08T15:45:59.950 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:59.950 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.950 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.950 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:59.951 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.225510+0000", 2020-09-08T15:45:59.951 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999599 2020-09-08T15:45:59.951 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.951 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.951 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:59.952 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.225470+0000", 2020-09-08T15:45:59.952 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000151206 2020-09-08T15:45:59.952 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.952 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.952 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:59.952 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.225621+0000", 2020-09-08T15:45:59.953 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.9643000000000001e-05 2020-09-08T15:45:59.953 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.953 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.953 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:59.953 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.225641+0000", 2020-09-08T15:45:59.954 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.1949999999999997e-06 2020-09-08T15:45:59.954 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.954 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.954 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:59.954 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.225648+0000", 2020-09-08T15:45:59.955 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.001303238 2020-09-08T15:45:59.955 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.955 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.955 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:59.955 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.226951+0000", 2020-09-08T15:45:59.956 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000112038 2020-09-08T15:45:59.956 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.956 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.956 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:59.956 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.227063+0000", 2020-09-08T15:45:59.957 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019811000000000001 2020-09-08T15:45:59.957 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.957 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.957 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:45:59.957 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.227261+0000", 2020-09-08T15:45:59.958 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0038368679999999998 2020-09-08T15:45:59.958 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.958 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.958 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:59.958 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.231098+0000", 2020-09-08T15:45:59.959 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.18374184299999999 2020-09-08T15:45:59.959 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.959 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.959 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:59.959 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.414840+0000", 2020-09-08T15:45:59.960 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000107272 2020-09-08T15:45:59.960 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.960 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.960 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:59.960 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.414947+0000", 2020-09-08T15:45:59.960 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014302799999999999 2020-09-08T15:45:59.961 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.961 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.961 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:59.961 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.415090+0000", 2020-09-08T15:45:59.961 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.189580258 2020-09-08T15:45:59.962 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:59.962 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:59.962 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:59.962 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.962 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.963 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.34742.0:89523 2.0 2:08e116f9:::benchmark_data_smithi166_20551_object89522:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e145)", 2020-09-08T15:45:59.963 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:38:24.226111+0000", 2020-09-08T15:45:59.963 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 455.58893401199998, 2020-09-08T15:45:59.963 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.189455454, 2020-09-08T15:45:59.963 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:59.964 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:59.964 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:59.964 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.34742", 2020-09-08T15:45:59.964 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:45:59.964 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 89523 2020-09-08T15:45:59.965 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.965 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:59.965 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.965 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:59.965 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.226111+0000", 2020-09-08T15:45:59.966 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:59.966 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.966 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.966 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:59.966 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.226111+0000", 2020-09-08T15:45:59.967 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999418 2020-09-08T15:45:59.967 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.967 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.968 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:59.968 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.226052+0000", 2020-09-08T15:45:59.968 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000161218 2020-09-08T15:45:59.968 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.969 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.969 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:59.969 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.226214+0000", 2020-09-08T15:45:59.969 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.7329e-05 2020-09-08T15:45:59.969 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.970 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.970 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:59.970 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.226231+0000", 2020-09-08T15:45:59.970 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.6839999999999999e-06 2020-09-08T15:45:59.970 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.971 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.971 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:59.971 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.226238+0000", 2020-09-08T15:45:59.971 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0027857310000000001 2020-09-08T15:45:59.971 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.972 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.972 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:59.972 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.229023+0000", 2020-09-08T15:45:59.972 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017312600000000001 2020-09-08T15:45:59.972 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.973 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.973 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:59.973 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.229196+0000", 2020-09-08T15:45:59.973 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016157899999999999 2020-09-08T15:45:59.973 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.974 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.974 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T15:45:59.974 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.229358+0000", 2020-09-08T15:45:59.974 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0017535249999999999 2020-09-08T15:45:59.974 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.975 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.975 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:59.975 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.231112+0000", 2020-09-08T15:45:59.975 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.18410831599999999 2020-09-08T15:45:59.975 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.976 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.976 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:59.976 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.415220+0000", 2020-09-08T15:45:59.976 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00020349800000000001 2020-09-08T15:45:59.976 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.976 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.977 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:59.977 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.415423+0000", 2020-09-08T15:45:59.977 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000142812 2020-09-08T15:45:59.977 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.978 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.978 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:59.978 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:38:24.415566+0000", 2020-09-08T15:45:59.978 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.189455454 2020-09-08T15:45:59.978 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:59.979 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:59.979 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:59.979 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.979 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.980 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:26812 3.d 3:b6ecaaea:::benchmark_data_smithi166_26097_object26811:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e222)", 2020-09-08T15:45:59.980 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:28.717066+0000", 2020-09-08T15:45:59.980 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 211.097979165, 2020-09-08T15:45:59.980 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.185898073, 2020-09-08T15:45:59.981 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:59.981 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:59.981 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:45:59.982 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:45:59.982 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:45:59.982 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 26812 2020-09-08T15:45:59.982 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.982 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:45:59.982 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.983 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:45:59.983 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.717066+0000", 2020-09-08T15:45:59.983 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:45:59.983 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.984 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.984 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:45:59.985 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.717066+0000", 2020-09-08T15:45:59.986 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999766 2020-09-08T15:45:59.986 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.986 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.986 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:45:59.986 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.717042+0000", 2020-09-08T15:45:59.987 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.7430999999999996e-05 2020-09-08T15:45:59.987 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.987 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.987 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:45:59.987 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.717140+0000", 2020-09-08T15:45:59.988 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2182e-05 2020-09-08T15:45:59.988 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.988 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.988 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:45:59.988 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.717152+0000", 2020-09-08T15:45:59.989 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.0609999999999998e-06 2020-09-08T15:45:59.989 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.989 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.989 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:45:59.989 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.717159+0000", 2020-09-08T15:45:59.990 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.3402999999999998e-05 2020-09-08T15:45:59.990 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.990 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.990 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:45:59.990 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.717202+0000", 2020-09-08T15:45:59.991 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000114305 2020-09-08T15:45:59.991 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.991 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.991 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:45:59.991 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.717317+0000", 2020-09-08T15:45:59.992 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00036377399999999998 2020-09-08T15:45:59.992 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.992 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.992 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 5", 2020-09-08T15:45:59.992 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.717680+0000", 2020-09-08T15:45:59.993 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00079849599999999999 2020-09-08T15:45:59.993 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.993 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.993 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:45:59.993 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.718479+0000", 2020-09-08T15:45:59.994 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.18440601600000001 2020-09-08T15:45:59.994 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.994 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.995 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:45:59.995 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.902885+0000", 2020-09-08T15:45:59.995 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.5849000000000001e-05 2020-09-08T15:45:59.995 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.995 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.996 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:45:59.996 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.902931+0000", 2020-09-08T15:45:59.996 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.2818000000000003e-05 2020-09-08T15:45:59.996 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.996 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.997 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:45:59.997 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.902964+0000", 2020-09-08T15:45:59.997 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.185898073 2020-09-08T15:45:59.997 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:59.997 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:45:59.998 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:45:59.998 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:45:59.998 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:45:59.998 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:36951 3.d 3:b56e5fc3:::benchmark_data_smithi166_26097_object36950:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e223)", 2020-09-08T15:45:59.998 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:33.432484+0000", 2020-09-08T15:45:59.999 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 206.38256037400001, 2020-09-08T15:45:59.999 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.20940113199999999, 2020-09-08T15:45:59.999 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:45:59.999 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:45:59.999 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:46:00.000 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:46:00.000 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:46:00.000 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 36951 2020-09-08T15:46:00.000 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.000 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:46:00.001 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.001 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:46:00.001 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.432484+0000", 2020-09-08T15:46:00.001 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:46:00.001 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.002 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.002 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:46:00.002 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.432484+0000", 2020-09-08T15:46:00.002 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999638 2020-09-08T15:46:00.002 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.003 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.003 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:46:00.003 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.432448+0000", 2020-09-08T15:46:00.003 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000146126 2020-09-08T15:46:00.003 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.003 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.004 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:46:00.004 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.432594+0000", 2020-09-08T15:46:00.004 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5352000000000001e-05 2020-09-08T15:46:00.004 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.004 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.005 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:46:00.005 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.432610+0000", 2020-09-08T15:46:00.005 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.477e-06 2020-09-08T15:46:00.005 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.006 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.006 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:46:00.006 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.432616+0000", 2020-09-08T15:46:00.006 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.4796999999999996e-05 2020-09-08T15:46:00.006 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.007 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.007 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:46:00.007 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.432681+0000", 2020-09-08T15:46:00.007 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000156441 2020-09-08T15:46:00.007 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.008 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.008 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:46:00.008 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.432837+0000", 2020-09-08T15:46:00.008 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00020998999999999999 2020-09-08T15:46:00.008 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.009 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.009 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 5", 2020-09-08T15:46:00.009 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.433047+0000", 2020-09-08T15:46:00.010 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00330767 2020-09-08T15:46:00.010 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.010 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.010 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:46:00.010 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.436355+0000", 2020-09-08T15:46:00.011 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.20532682099999999 2020-09-08T15:46:00.011 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.011 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.011 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:00.011 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.641682+0000", 2020-09-08T15:46:00.012 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011547100000000001 2020-09-08T15:46:00.012 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.012 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.012 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:46:00.012 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.641797+0000", 2020-09-08T15:46:00.012 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.8197999999999998e-05 2020-09-08T15:46:00.013 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.013 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.013 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:46:00.013 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.641886+0000", 2020-09-08T15:46:00.013 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.20940113199999999 2020-09-08T15:46:00.014 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.014 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:46:00.014 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.014 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.014 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.015 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:36958 3.d 3:bd3c6e00:::benchmark_data_smithi166_26097_object36957:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e223)", 2020-09-08T15:46:00.015 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:33.434592+0000", 2020-09-08T15:46:00.015 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 206.38045313999999, 2020-09-08T15:46:00.015 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.217482596, 2020-09-08T15:46:00.015 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:46:00.016 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:00.016 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:46:00.016 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:46:00.016 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:46:00.016 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 36958 2020-09-08T15:46:00.017 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.017 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:46:00.017 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.017 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:46:00.017 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.434592+0000", 2020-09-08T15:46:00.022 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:46:00.023 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.023 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.023 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:46:00.028 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.434592+0000", 2020-09-08T15:46:00.028 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999958 2020-09-08T15:46:00.028 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.029 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.029 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:46:00.029 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.434550+0000", 2020-09-08T15:46:00.029 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000150006 2020-09-08T15:46:00.029 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.041 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.042 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:46:00.042 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.434700+0000", 2020-09-08T15:46:00.042 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.451e-05 2020-09-08T15:46:00.043 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.043 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.043 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:46:00.043 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.434714+0000", 2020-09-08T15:46:00.044 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.285e-06 2020-09-08T15:46:00.044 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.044 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.044 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:46:00.044 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.434721+0000", 2020-09-08T15:46:00.045 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.0732000000000002e-05 2020-09-08T15:46:00.045 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.045 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.045 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:46:00.045 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.434761+0000", 2020-09-08T15:46:00.046 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000112055 2020-09-08T15:46:00.047 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.047 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.047 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:46:00.047 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.434873+0000", 2020-09-08T15:46:00.048 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019727699999999999 2020-09-08T15:46:00.048 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.048 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.048 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 5", 2020-09-08T15:46:00.048 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.435071+0000", 2020-09-08T15:46:00.049 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0014893840000000001 2020-09-08T15:46:00.049 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.049 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.049 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:46:00.050 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.436560+0000", 2020-09-08T15:46:00.050 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.21538986399999999 2020-09-08T15:46:00.050 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.050 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.050 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:00.051 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.651950+0000", 2020-09-08T15:46:00.051 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.5977999999999997e-05 2020-09-08T15:46:00.051 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.051 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.051 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:46:00.052 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.652016+0000", 2020-09-08T15:46:00.052 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.8285000000000002e-05 2020-09-08T15:46:00.052 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.052 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.052 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:46:00.053 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.652074+0000", 2020-09-08T15:46:00.053 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.217482596 2020-09-08T15:46:00.053 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.053 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:46:00.053 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.054 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.054 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.054 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:36990 3.d 3:bb64c0f8:::benchmark_data_smithi166_26097_object36989:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e223)", 2020-09-08T15:46:00.054 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:33.446759+0000", 2020-09-08T15:46:00.054 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 206.36828586499999, 2020-09-08T15:46:00.055 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.205601806, 2020-09-08T15:46:00.055 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:46:00.055 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:00.056 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:46:00.056 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:46:00.056 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:46:00.056 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 36990 2020-09-08T15:46:00.056 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.057 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:46:00.057 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.057 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:46:00.057 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.446759+0000", 2020-09-08T15:46:00.057 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:46:00.058 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.058 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.058 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:46:00.058 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.446759+0000", 2020-09-08T15:46:00.058 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999585 2020-09-08T15:46:00.059 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.059 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.059 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:46:00.059 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.446717+0000", 2020-09-08T15:46:00.060 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000162461 2020-09-08T15:46:00.060 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.060 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.060 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:46:00.060 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.446880+0000", 2020-09-08T15:46:00.061 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4861999999999999e-05 2020-09-08T15:46:00.061 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.061 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.061 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:46:00.062 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.446895+0000", 2020-09-08T15:46:00.062 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.6250000000000002e-06 2020-09-08T15:46:00.062 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.063 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.063 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:46:00.063 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.446904+0000", 2020-09-08T15:46:00.063 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.6867999999999999e-05 2020-09-08T15:46:00.063 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.064 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.064 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:46:00.064 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.446951+0000", 2020-09-08T15:46:00.064 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012854899999999999 2020-09-08T15:46:00.064 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.065 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.065 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:46:00.065 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.447080+0000", 2020-09-08T15:46:00.065 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019507500000000001 2020-09-08T15:46:00.065 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.066 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.067 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 5", 2020-09-08T15:46:00.067 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.447275+0000", 2020-09-08T15:46:00.067 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.001664586 2020-09-08T15:46:00.068 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.068 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.068 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:46:00.069 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.448939+0000", 2020-09-08T15:46:00.069 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.20321997999999999 2020-09-08T15:46:00.069 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.070 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.070 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:00.070 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.652159+0000", 2020-09-08T15:46:00.070 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000112502 2020-09-08T15:46:00.070 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.071 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.071 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:46:00.071 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.652272+0000", 2020-09-08T15:46:00.071 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.8962999999999995e-05 2020-09-08T15:46:00.071 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.072 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.072 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:46:00.072 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:33.652361+0000", 2020-09-08T15:46:00.072 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.205601806 2020-09-08T15:46:00.073 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.073 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:46:00.073 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.073 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.073 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.074 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:155728 3.f 3:fdbde2cb:::benchmark_data_smithi166_26097_object31567:head [delete] snapc 0=[] ondisk+write+known_if_redirected e264)", 2020-09-08T15:46:00.074 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:44:38.734787+0000", 2020-09-08T15:46:00.074 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 81.080257341999996, 2020-09-08T15:46:00.074 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.25803799599999999, 2020-09-08T15:46:00.074 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:46:00.075 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:00.075 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:46:00.075 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:46:00.075 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:46:00.075 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 155728 2020-09-08T15:46:00.076 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.076 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:46:00.076 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.076 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:46:00.076 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.734787+0000", 2020-09-08T15:46:00.077 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:46:00.077 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.078 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.078 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:46:00.078 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.734787+0000", 2020-09-08T15:46:00.078 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999175 2020-09-08T15:46:00.078 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.079 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.079 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:46:00.079 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.734705+0000", 2020-09-08T15:46:00.079 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00018019800000000001 2020-09-08T15:46:00.079 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.080 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.080 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:46:00.080 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.734885+0000", 2020-09-08T15:46:00.080 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.7936000000000002e-05 2020-09-08T15:46:00.080 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.081 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.081 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:46:00.081 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.734913+0000", 2020-09-08T15:46:00.081 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2904e-05 2020-09-08T15:46:00.081 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.082 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.082 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:46:00.082 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.734926+0000", 2020-09-08T15:46:00.085 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0063388029999999996 2020-09-08T15:46:00.085 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.085 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.086 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:46:00.086 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.741265+0000", 2020-09-08T15:46:00.086 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00069344899999999995 2020-09-08T15:46:00.086 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.086 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.087 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:46:00.087 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.741958+0000", 2020-09-08T15:46:00.087 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00024322600000000001 2020-09-08T15:46:00.087 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.087 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.088 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 11", 2020-09-08T15:46:00.088 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.742201+0000", 2020-09-08T15:46:00.088 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.018419073000000001 2020-09-08T15:46:00.088 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.088 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.089 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:46:00.089 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.760620+0000", 2020-09-08T15:46:00.089 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.231868511 2020-09-08T15:46:00.089 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.090 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.090 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:00.090 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.992489+0000", 2020-09-08T15:46:00.090 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000148818 2020-09-08T15:46:00.090 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.094 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.094 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:46:00.094 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.992638+0000", 2020-09-08T15:46:00.094 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00018780000000000001 2020-09-08T15:46:00.095 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.095 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.095 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:46:00.095 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.992825+0000", 2020-09-08T15:46:00.095 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.25803799599999999 2020-09-08T15:46:00.096 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.096 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:46:00.096 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.096 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.096 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.097 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:155730 3.f 3:f438a699:::benchmark_data_smithi166_26097_object31569:head [delete] snapc 0=[] ondisk+write+known_if_redirected e264)", 2020-09-08T15:46:00.097 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:44:38.737408+0000", 2020-09-08T15:46:00.097 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 81.077636583, 2020-09-08T15:46:00.097 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.27755827799999999, 2020-09-08T15:46:00.097 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:46:00.098 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:00.098 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:46:00.098 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:46:00.098 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:46:00.099 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 155730 2020-09-08T15:46:00.099 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.099 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:46:00.099 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.100 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:46:00.100 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.737408+0000", 2020-09-08T15:46:00.100 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:46:00.100 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.100 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.101 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:46:00.101 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.737408+0000", 2020-09-08T15:46:00.101 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999647 2020-09-08T15:46:00.101 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.101 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.102 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:46:00.102 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.737373+0000", 2020-09-08T15:46:00.102 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000110678 2020-09-08T15:46:00.102 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.102 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.103 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:46:00.103 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.737484+0000", 2020-09-08T15:46:00.103 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.9789999999999999e-05 2020-09-08T15:46:00.103 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.103 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.103 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:46:00.104 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.737503+0000", 2020-09-08T15:46:00.104 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.8769999999999999e-06 2020-09-08T15:46:00.104 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.104 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.104 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:46:00.105 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.737512+0000", 2020-09-08T15:46:00.105 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.012819496 2020-09-08T15:46:00.105 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.105 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.105 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:46:00.106 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.750332+0000", 2020-09-08T15:46:00.106 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010244899999999999 2020-09-08T15:46:00.106 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.106 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.106 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:46:00.107 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.750434+0000", 2020-09-08T15:46:00.107 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.6415000000000005e-05 2020-09-08T15:46:00.107 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.107 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.107 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 11", 2020-09-08T15:46:00.107 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.750521+0000", 2020-09-08T15:46:00.108 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.010111466 2020-09-08T15:46:00.108 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.108 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.108 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:46:00.108 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.760632+0000", 2020-09-08T15:46:00.109 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.25423113600000002 2020-09-08T15:46:00.109 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.109 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.109 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:00.109 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.014863+0000", 2020-09-08T15:46:00.110 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2602000000000002e-05 2020-09-08T15:46:00.110 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.110 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.111 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:46:00.111 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.014916+0000", 2020-09-08T15:46:00.111 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.0688e-05 2020-09-08T15:46:00.111 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.111 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.112 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:46:00.112 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.014966+0000", 2020-09-08T15:46:00.112 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.27755827799999999 2020-09-08T15:46:00.112 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.112 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:46:00.113 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.113 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.113 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.113 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:155738 3.f 3:f6595afa:::benchmark_data_smithi166_26097_object31577:head [delete] snapc 0=[] ondisk+write+known_if_redirected e264)", 2020-09-08T15:46:00.113 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:44:38.747451+0000", 2020-09-08T15:46:00.114 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 81.067593959999996, 2020-09-08T15:46:00.114 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.26767150699999998, 2020-09-08T15:46:00.114 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:46:00.114 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:00.115 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:46:00.115 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:46:00.115 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:46:00.115 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 155738 2020-09-08T15:46:00.115 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.116 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:46:00.116 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.116 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:46:00.116 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.747451+0000", 2020-09-08T15:46:00.116 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:46:00.116 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.117 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.117 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:46:00.117 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.747451+0000", 2020-09-08T15:46:00.117 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999342 2020-09-08T15:46:00.117 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.118 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.118 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:46:00.118 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.747385+0000", 2020-09-08T15:46:00.118 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014450899999999999 2020-09-08T15:46:00.118 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.119 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.119 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:46:00.119 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.747530+0000", 2020-09-08T15:46:00.119 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.7336999999999999e-05 2020-09-08T15:46:00.119 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.120 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.120 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:46:00.120 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.747557+0000", 2020-09-08T15:46:00.120 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2167999999999999e-05 2020-09-08T15:46:00.120 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.121 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.121 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:46:00.121 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.747569+0000", 2020-09-08T15:46:00.121 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.007673114 2020-09-08T15:46:00.121 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.121 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.122 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:46:00.122 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.755242+0000", 2020-09-08T15:46:00.122 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000165878 2020-09-08T15:46:00.122 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.122 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.123 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:46:00.123 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.755408+0000", 2020-09-08T15:46:00.123 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000154449 2020-09-08T15:46:00.123 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.123 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.124 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 11", 2020-09-08T15:46:00.124 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.755563+0000", 2020-09-08T15:46:00.124 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0052431980000000001 2020-09-08T15:46:00.124 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.124 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.124 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:46:00.125 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.760806+0000", 2020-09-08T15:46:00.125 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.25422231000000001 2020-09-08T15:46:00.125 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.125 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.125 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:00.126 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.015028+0000", 2020-09-08T15:46:00.126 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9865999999999999e-05 2020-09-08T15:46:00.126 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.126 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.126 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:46:00.127 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.015088+0000", 2020-09-08T15:46:00.127 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.4246999999999999e-05 2020-09-08T15:46:00.127 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.127 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.127 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:46:00.127 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.015122+0000", 2020-09-08T15:46:00.129 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.26767150699999998 2020-09-08T15:46:00.129 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.129 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:46:00.129 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.131 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.131 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.131 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:155753 3.f 3:f7bf1398:::benchmark_data_smithi166_26097_object31592:head [delete] snapc 0=[] ondisk+write+known_if_redirected e264)", 2020-09-08T15:46:00.131 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:44:38.777041+0000", 2020-09-08T15:46:00.131 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 81.038003418000002, 2020-09-08T15:46:00.132 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.25282072100000003, 2020-09-08T15:46:00.132 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:46:00.132 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:00.133 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:46:00.133 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:46:00.133 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:46:00.133 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 155753 2020-09-08T15:46:00.133 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.133 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:46:00.134 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.134 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:46:00.134 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.777041+0000", 2020-09-08T15:46:00.134 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:46:00.134 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.135 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.135 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:46:00.135 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.777041+0000", 2020-09-08T15:46:00.135 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999547 2020-09-08T15:46:00.135 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.136 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.136 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:46:00.136 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.776996+0000", 2020-09-08T15:46:00.136 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010441 2020-09-08T15:46:00.136 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.137 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.137 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:46:00.137 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.777100+0000", 2020-09-08T15:46:00.137 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.9528e-05 2020-09-08T15:46:00.138 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.138 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.138 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:46:00.138 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.777120+0000", 2020-09-08T15:46:00.138 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0614e-05 2020-09-08T15:46:00.139 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.139 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.139 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:46:00.139 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.777131+0000", 2020-09-08T15:46:00.139 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0909000000000002e-05 2020-09-08T15:46:00.140 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.140 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.140 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:46:00.140 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.777192+0000", 2020-09-08T15:46:00.140 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016369299999999999 2020-09-08T15:46:00.141 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.141 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.141 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:46:00.141 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.777355+0000", 2020-09-08T15:46:00.141 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000141848 2020-09-08T15:46:00.142 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.142 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.142 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 11", 2020-09-08T15:46:00.142 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.777497+0000", 2020-09-08T15:46:00.142 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.010555151 2020-09-08T15:46:00.143 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.143 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.143 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:46:00.143 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.788052+0000", 2020-09-08T15:46:00.143 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.24166077799999999 2020-09-08T15:46:00.143 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.144 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.144 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:00.144 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.029713+0000", 2020-09-08T15:46:00.144 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.8423000000000004e-05 2020-09-08T15:46:00.144 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.145 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.145 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:46:00.145 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.029801+0000", 2020-09-08T15:46:00.145 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0656999999999999e-05 2020-09-08T15:46:00.145 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.146 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.146 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:46:00.146 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.029862+0000", 2020-09-08T15:46:00.146 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.25282072100000003 2020-09-08T15:46:00.146 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.147 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:46:00.147 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.147 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.147 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.147 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:155754 3.f 3:f7a78bf2:::benchmark_data_smithi166_26097_object31593:head [delete] snapc 0=[] ondisk+write+known_if_redirected e264)", 2020-09-08T15:46:00.147 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:44:38.778055+0000", 2020-09-08T15:46:00.148 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 81.036989348999995, 2020-09-08T15:46:00.148 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.25198914300000003, 2020-09-08T15:46:00.148 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:46:00.148 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:00.148 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:46:00.149 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:46:00.149 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:46:00.149 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 155754 2020-09-08T15:46:00.149 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.149 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:46:00.150 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.150 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:46:00.150 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.778055+0000", 2020-09-08T15:46:00.150 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:46:00.150 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.151 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.151 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:46:00.151 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.778055+0000", 2020-09-08T15:46:00.151 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999409 2020-09-08T15:46:00.151 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.151 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.152 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:46:00.152 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.777996+0000", 2020-09-08T15:46:00.152 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000132476 2020-09-08T15:46:00.152 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.152 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.153 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:46:00.153 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.778129+0000", 2020-09-08T15:46:00.153 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.3096e-05 2020-09-08T15:46:00.153 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.153 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.154 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:46:00.154 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.778152+0000", 2020-09-08T15:46:00.155 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.5250000000000005e-06 2020-09-08T15:46:00.155 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.155 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.156 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:46:00.156 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.778160+0000", 2020-09-08T15:46:00.156 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0040231970000000001 2020-09-08T15:46:00.156 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.156 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.157 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:46:00.157 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.782183+0000", 2020-09-08T15:46:00.157 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00025253999999999999 2020-09-08T15:46:00.157 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.157 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.158 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:46:00.158 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.782436+0000", 2020-09-08T15:46:00.158 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000245799 2020-09-08T15:46:00.159 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.159 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.159 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 11", 2020-09-08T15:46:00.160 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.782682+0000", 2020-09-08T15:46:00.160 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0056363280000000003 2020-09-08T15:46:00.160 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.160 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.160 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:46:00.160 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:38.788318+0000", 2020-09-08T15:46:00.161 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.24160298399999999 2020-09-08T15:46:00.161 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.161 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.161 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:00.161 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.029921+0000", 2020-09-08T15:46:00.162 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.7688999999999995e-05 2020-09-08T15:46:00.162 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.162 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.162 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:46:00.162 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.029989+0000", 2020-09-08T15:46:00.163 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.5819000000000002e-05 2020-09-08T15:46:00.163 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.163 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.163 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:46:00.163 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.030045+0000", 2020-09-08T15:46:00.164 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.25198914300000003 2020-09-08T15:46:00.164 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.164 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:46:00.164 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.164 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.164 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.165 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:208536 3.f 3:f7ceb391:::benchmark_data_smithi166_26097_object84375:head [delete] snapc 0=[] ondisk+write+known_if_redirected e286)", 2020-09-08T15:46:00.166 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:45:55.477763+0000", 2020-09-08T15:46:00.167 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 4.3372819390000004, 2020-09-08T15:46:00.167 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.199352584, 2020-09-08T15:46:00.167 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:46:00.167 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:00.168 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:46:00.168 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:46:00.168 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:46:00.168 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 208536 2020-09-08T15:46:00.168 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.169 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:46:00.169 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.169 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:46:00.169 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.477763+0000", 2020-09-08T15:46:00.169 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:46:00.170 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.170 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.170 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:46:00.171 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.477763+0000", 2020-09-08T15:46:00.171 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999633 2020-09-08T15:46:00.171 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.171 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.171 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:46:00.171 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.477726+0000", 2020-09-08T15:46:00.172 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.8820999999999997e-05 2020-09-08T15:46:00.172 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.172 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.172 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:46:00.172 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.477795+0000", 2020-09-08T15:46:00.173 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.6840000000000002e-06 2020-09-08T15:46:00.173 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.173 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.173 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:46:00.173 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.477804+0000", 2020-09-08T15:46:00.174 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.0380000000000001e-06 2020-09-08T15:46:00.174 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.174 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.174 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:46:00.174 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.477808+0000", 2020-09-08T15:46:00.175 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0076090919999999996 2020-09-08T15:46:00.175 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.175 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.175 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:46:00.175 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.485417+0000", 2020-09-08T15:46:00.176 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015112799999999999 2020-09-08T15:46:00.176 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.176 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.176 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:46:00.176 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.485568+0000", 2020-09-08T15:46:00.177 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000147806 2020-09-08T15:46:00.177 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.177 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.178 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 11", 2020-09-08T15:46:00.178 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.485716+0000", 2020-09-08T15:46:00.178 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.011448043 2020-09-08T15:46:00.178 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.178 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.179 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:46:00.179 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.497164+0000", 2020-09-08T15:46:00.179 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.17965792999999999 2020-09-08T15:46:00.179 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.179 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.179 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:00.180 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.676822+0000", 2020-09-08T15:46:00.180 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00021258200000000001 2020-09-08T15:46:00.180 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.180 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.180 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:46:00.181 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.677034+0000", 2020-09-08T15:46:00.181 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.1284000000000003e-05 2020-09-08T15:46:00.181 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.181 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.181 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:46:00.182 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.677115+0000", 2020-09-08T15:46:00.182 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.199352584 2020-09-08T15:46:00.182 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.182 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:46:00.182 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.183 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.183 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.183 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:208537 3.f 3:f6ad3e2f:::benchmark_data_smithi166_26097_object84376:head [delete] snapc 0=[] ondisk+write+known_if_redirected e286)", 2020-09-08T15:46:00.183 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:45:55.479916+0000", 2020-09-08T15:46:00.183 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 4.3351284860000003, 2020-09-08T15:46:00.184 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.22410005, 2020-09-08T15:46:00.184 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:46:00.184 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:00.185 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:46:00.185 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:46:00.186 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:46:00.186 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 208537 2020-09-08T15:46:00.187 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.187 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:46:00.187 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.187 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:46:00.187 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.479916+0000", 2020-09-08T15:46:00.188 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:46:00.188 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.188 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.188 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:46:00.189 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.479916+0000", 2020-09-08T15:46:00.189 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999723 2020-09-08T15:46:00.189 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.190 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.190 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:46:00.190 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.479888+0000", 2020-09-08T15:46:00.190 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0723000000000001e-05 2020-09-08T15:46:00.190 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.191 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.191 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:46:00.191 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.479949+0000", 2020-09-08T15:46:00.191 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3055e-05 2020-09-08T15:46:00.191 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.192 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.192 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:46:00.192 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.479962+0000", 2020-09-08T15:46:00.192 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.4320000000000002e-06 2020-09-08T15:46:00.192 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.192 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.193 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:46:00.193 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.479970+0000", 2020-09-08T15:46:00.193 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.011522687 2020-09-08T15:46:00.193 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.194 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.195 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:46:00.195 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.491492+0000", 2020-09-08T15:46:00.195 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016333499999999999 2020-09-08T15:46:00.195 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.196 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.196 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:46:00.196 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.491656+0000", 2020-09-08T15:46:00.196 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016177300000000001 2020-09-08T15:46:00.196 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.196 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.197 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 11", 2020-09-08T15:46:00.197 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.491817+0000", 2020-09-08T15:46:00.198 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0055552079999999998 2020-09-08T15:46:00.198 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.199 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.199 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:46:00.199 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.497373+0000", 2020-09-08T15:46:00.199 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.206546966 2020-09-08T15:46:00.199 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.200 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.200 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:00.200 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.703920+0000", 2020-09-08T15:46:00.200 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.1433e-05 2020-09-08T15:46:00.200 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.201 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.201 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:46:00.201 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.703971+0000", 2020-09-08T15:46:00.201 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.5262999999999997e-05 2020-09-08T15:46:00.201 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.202 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.202 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:46:00.202 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.704016+0000", 2020-09-08T15:46:00.203 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.22410005 2020-09-08T15:46:00.203 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.203 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:46:00.203 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.203 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.203 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.204 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:208565 3.f 3:f2168262:::benchmark_data_smithi166_26097_object84404:head [delete] snapc 0=[] ondisk+write+known_if_redirected e286)", 2020-09-08T15:46:00.204 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:45:55.509249+0000", 2020-09-08T15:46:00.204 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 4.3057953089999996, 2020-09-08T15:46:00.204 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.19489281, 2020-09-08T15:46:00.204 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:46:00.205 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:00.205 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:46:00.205 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:46:00.205 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:46:00.205 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 208565 2020-09-08T15:46:00.206 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.206 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:46:00.206 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.206 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:46:00.206 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.509249+0000", 2020-09-08T15:46:00.207 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:46:00.207 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.207 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.207 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:46:00.207 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.509249+0000", 2020-09-08T15:46:00.207 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999781 2020-09-08T15:46:00.208 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.208 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.208 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:46:00.208 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.509227+0000", 2020-09-08T15:46:00.208 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.0701999999999997e-05 2020-09-08T15:46:00.209 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.209 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.209 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:46:00.209 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.509278+0000", 2020-09-08T15:46:00.209 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.2439999999999996e-06 2020-09-08T15:46:00.210 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.210 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.210 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:46:00.210 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.509285+0000", 2020-09-08T15:46:00.210 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.2040000000000002e-06 2020-09-08T15:46:00.210 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.211 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.211 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:46:00.211 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.509289+0000", 2020-09-08T15:46:00.211 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.3139999999999997e-05 2020-09-08T15:46:00.211 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.212 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.212 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:46:00.212 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.509332+0000", 2020-09-08T15:46:00.212 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.1519000000000002e-05 2020-09-08T15:46:00.212 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.213 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.213 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:46:00.213 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.509423+0000", 2020-09-08T15:46:00.213 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.1447999999999996e-05 2020-09-08T15:46:00.213 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.214 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.214 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 11", 2020-09-08T15:46:00.214 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.509515+0000", 2020-09-08T15:46:00.214 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.013084887 2020-09-08T15:46:00.214 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.215 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.215 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:46:00.215 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.522600+0000", 2020-09-08T15:46:00.215 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.181461703 2020-09-08T15:46:00.215 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.215 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.216 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:00.216 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.704061+0000", 2020-09-08T15:46:00.216 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.7003999999999999e-05 2020-09-08T15:46:00.217 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.217 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.217 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:46:00.218 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.704108+0000", 2020-09-08T15:46:00.218 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.4048000000000003e-05 2020-09-08T15:46:00.218 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.218 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.218 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:46:00.219 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.704142+0000", 2020-09-08T15:46:00.219 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.19489281 2020-09-08T15:46:00.219 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.219 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:46:00.219 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.220 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.220 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.220 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35087.0:208567 3.f 3:f1fccc57:::benchmark_data_smithi166_26097_object84406:head [delete] snapc 0=[] ondisk+write+known_if_redirected e286)", 2020-09-08T15:46:00.220 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:45:55.510472+0000", 2020-09-08T15:46:00.220 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 4.3045722959999999, 2020-09-08T15:46:00.221 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.193791992, 2020-09-08T15:46:00.221 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:46:00.221 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:00.221 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:46:00.221 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35087", 2020-09-08T15:46:00.222 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:46:00.222 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 208567 2020-09-08T15:46:00.222 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.222 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:46:00.222 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.223 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:46:00.223 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.510472+0000", 2020-09-08T15:46:00.223 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:46:00.223 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.223 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.223 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:46:00.224 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.510472+0000", 2020-09-08T15:46:00.224 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999557 2020-09-08T15:46:00.224 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.224 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.224 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:46:00.225 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.510428+0000", 2020-09-08T15:46:00.225 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.5594000000000007e-05 2020-09-08T15:46:00.225 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.225 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.225 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:46:00.226 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.510524+0000", 2020-09-08T15:46:00.226 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.7075000000000002e-05 2020-09-08T15:46:00.226 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.226 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.227 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:46:00.227 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.510541+0000", 2020-09-08T15:46:00.227 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.4719999999999998e-06 2020-09-08T15:46:00.227 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.227 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.228 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:46:00.228 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.510548+0000", 2020-09-08T15:46:00.228 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0050884859999999997 2020-09-08T15:46:00.228 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.228 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.228 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:46:00.229 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.515637+0000", 2020-09-08T15:46:00.229 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019650099999999999 2020-09-08T15:46:00.229 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.229 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.229 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:46:00.230 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.515833+0000", 2020-09-08T15:46:00.230 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00018778600000000001 2020-09-08T15:46:00.230 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.230 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.230 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 11", 2020-09-08T15:46:00.231 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.516021+0000", 2020-09-08T15:46:00.231 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0068217599999999996 2020-09-08T15:46:00.231 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.231 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.231 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:46:00.231 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.522843+0000", 2020-09-08T15:46:00.232 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.181355345 2020-09-08T15:46:00.232 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.232 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.232 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:00.232 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.704198+0000", 2020-09-08T15:46:00.233 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.6908e-05 2020-09-08T15:46:00.233 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.233 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.233 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:46:00.233 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.704235+0000", 2020-09-08T15:46:00.234 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.9609999999999999e-05 2020-09-08T15:46:00.235 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:46:00.235 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:46:00.235 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:46:00.235 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:55.704264+0000", 2020-09-08T15:46:00.236 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.193791992 2020-09-08T15:46:00.236 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.236 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:46:00.236 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.236 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:46:00.237 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:46:00.237 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:46:00.237 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:00.245 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:45:59 smithi166 bash[16593]: debug 2020-09-08T15:45:59.730+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:00.246 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:59 smithi104 bash[9203]: cluster 2020-09-08T15:45:57.723710+0000 mgr.x (mgr.24118) 559 : cluster [DBG] pgmap v762: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 942 op/s; 4118/79192 objects misplaced (5.200%); 16 MiB/s, 253 objects/s recovering 2020-09-08T15:46:00.246 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:45:59 smithi104 bash[9203]: cluster 2020-09-08T15:45:59.681421+0000 mon.a (mon.0) 1266 : cluster [DBG] osdmap e288: 12 total, 12 up, 12 in 2020-09-08T15:46:00.246 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:59 smithi055 bash[9296]: cluster 2020-09-08T15:45:57.723710+0000 mgr.x (mgr.24118) 559 : cluster [DBG] pgmap v762: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 942 op/s; 4118/79192 objects misplaced (5.200%); 16 MiB/s, 253 objects/s recovering 2020-09-08T15:46:00.247 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:45:59 smithi055 bash[9296]: cluster 2020-09-08T15:45:59.681421+0000 mon.a (mon.0) 1266 : cluster [DBG] osdmap e288: 12 total, 12 up, 12 in 2020-09-08T15:46:00.374 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:00.477 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:00.582 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:46:00 smithi166 bash[16593]: debug 2020-09-08T15:46:00.366+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:00.583 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:00.583 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:46:00 smithi166 bash[18960]: debug 2020-09-08T15:46:00.582+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:00.691 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:00.825 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:00.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:00 smithi166 bash[9446]: cluster 2020-09-08T15:45:59.376220+0000 osd.2 (osd.2) 40 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:00.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:00 smithi166 bash[9446]: cluster 2020-09-08T15:45:59.376295+0000 osd.2 (osd.2) 41 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:00.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:00 smithi166 bash[9446]: cluster 2020-09-08T15:45:59.724205+0000 mgr.x (mgr.24118) 560 : cluster [DBG] pgmap v764: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 704 op/s; 3885/76748 objects misplaced (5.062%); 11 MiB/s, 171 objects/s recovering 2020-09-08T15:46:00.901 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2020-09-08T15:46:00.930 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:01.074 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:46:01.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:46:00 smithi055 bash[14301]: debug 2020-09-08T15:46:00.816+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:01.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:00 smithi055 bash[9296]: cluster 2020-09-08T15:45:59.376220+0000 osd.2 (osd.2) 40 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:01.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:00 smithi055 bash[9296]: cluster 2020-09-08T15:45:59.376295+0000 osd.2 (osd.2) 41 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:01.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:00 smithi055 bash[9296]: cluster 2020-09-08T15:45:59.724205+0000 mgr.x (mgr.24118) 560 : cluster [DBG] pgmap v764: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 704 op/s; 3885/76748 objects misplaced (5.062%); 11 MiB/s, 171 objects/s recovering 2020-09-08T15:46:01.183 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:00 smithi104 bash[9203]: cluster 2020-09-08T15:45:59.376220+0000 osd.2 (osd.2) 40 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:01.184 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:00 smithi104 bash[9203]: cluster 2020-09-08T15:45:59.376295+0000 osd.2 (osd.2) 41 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:01.184 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:00 smithi104 bash[9203]: cluster 2020-09-08T15:45:59.724205+0000 mgr.x (mgr.24118) 560 : cluster [DBG] pgmap v764: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 704 op/s; 3885/76748 objects misplaced (5.062%); 11 MiB/s, 171 objects/s recovering 2020-09-08T15:46:02.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:01 smithi166 bash[9446]: cluster 2020-09-08T15:46:00.720830+0000 mon.a (mon.0) 1267 : cluster [DBG] osdmap e289: 12 total, 12 up, 12 in 2020-09-08T15:46:02.170 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:01 smithi055 bash[9296]: cluster 2020-09-08T15:46:00.720830+0000 mon.a (mon.0) 1267 : cluster [DBG] osdmap e289: 12 total, 12 up, 12 in 2020-09-08T15:46:02.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:01 smithi104 bash[9203]: cluster 2020-09-08T15:46:00.720830+0000 mon.a (mon.0) 1267 : cluster [DBG] osdmap e289: 12 total, 12 up, 12 in 2020-09-08T15:46:02.399 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:02.505 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:03.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:02 smithi055 bash[9296]: cluster 2020-09-08T15:46:01.719183+0000 mon.a (mon.0) 1268 : cluster [DBG] osdmap e290: 12 total, 12 up, 12 in 2020-09-08T15:46:03.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:02 smithi055 bash[9296]: cluster 2020-09-08T15:46:01.724687+0000 mgr.x (mgr.24118) 561 : cluster [DBG] pgmap v767: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 475 op/s; 3601/75348 objects misplaced (4.779%); 8.0 MiB/s, 128 objects/s recovering 2020-09-08T15:46:03.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:02 smithi104 bash[9203]: cluster 2020-09-08T15:46:01.719183+0000 mon.a (mon.0) 1268 : cluster [DBG] osdmap e290: 12 total, 12 up, 12 in 2020-09-08T15:46:03.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:02 smithi104 bash[9203]: cluster 2020-09-08T15:46:01.724687+0000 mgr.x (mgr.24118) 561 : cluster [DBG] pgmap v767: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 475 op/s; 3601/75348 objects misplaced (4.779%); 8.0 MiB/s, 128 objects/s recovering 2020-09-08T15:46:03.293 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:46:02 smithi166 bash[16593]: debug 2020-09-08T15:46:02.398+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:03.505 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:03.520 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:46:03.542 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:46:03.568 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:46:03.605 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:46:03.611 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:03.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:46:03 smithi166 bash[16593]: debug 2020-09-08T15:46:03.506+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:03.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:03 smithi166 bash[9446]: cluster 2020-09-08T15:46:01.719183+0000 mon.a (mon.0) 1268 : cluster [DBG] osdmap e290: 12 total, 12 up, 12 in 2020-09-08T15:46:03.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:03 smithi166 bash[9446]: cluster 2020-09-08T15:46:01.724687+0000 mgr.x (mgr.24118) 561 : cluster [DBG] pgmap v767: 17 pgs: 2 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 475 op/s; 3601/75348 objects misplaced (4.779%); 8.0 MiB/s, 128 objects/s recovering 2020-09-08T15:46:03.641 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:46:03.673 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:46:03.717 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:46:03.734 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:03.751 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:46:03.784 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:46:03.816 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:46:03.849 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:03.850 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:46:03.879 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:46:03.885 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:46:03.885 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:46:03.889 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:46:03.889 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:46:03.889 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:46:03.889 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:46:03.890 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:46:03.890 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:46:03.890 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:46:03.890 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:46:03.891 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:46:03.891 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:46:03.935 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:46:03 smithi104 bash[22305]: debug 2020-09-08T15:46:03.734+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:03.935 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:03 smithi104 bash[9203]: audit 2020-09-08T15:46:03.488855+0000 mon.a (mon.0) 1269 : audit [DBG] from='client.? v1:172.21.15.104:0/4220938846' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:46:03.989 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:03.992 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:03 smithi055 bash[9296]: audit 2020-09-08T15:46:03.488855+0000 mon.a (mon.0) 1269 : audit [DBG] from='client.? v1:172.21.15.104:0/4220938846' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:46:03.993 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:46:03 smithi055 bash[11956]: debug 2020-09-08T15:46:03.988+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:04.096 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:04.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:03 smithi166 bash[9446]: audit 2020-09-08T15:46:03.488855+0000 mon.a (mon.0) 1269 : audit [DBG] from='client.? v1:172.21.15.104:0/4220938846' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:46:04.213 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:04.318 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:04.339 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:46:04 smithi055 bash[14301]: debug 2020-09-08T15:46:04.216+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:04.440 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:04.545 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:04.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:46:04 smithi055 bash[11956]: debug 2020-09-08T15:46:04.440+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:04.661 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:04.770 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:04.786 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:46:04.995 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:05.006 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:04 smithi166 bash[9446]: cluster 2020-09-08T15:46:03.725518+0000 mgr.x (mgr.24118) 562 : cluster [DBG] pgmap v768: 17 pgs: 1 active+clean+wait, 1 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 678 op/s; 2708/71048 objects misplaced (3.812%); 13 MiB/s, 213 objects/s recovering 2020-09-08T15:46:05.007 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:04 smithi166 bash[9446]: audit 2020-09-08T15:46:03.870695+0000 mon.a (mon.0) 1270 : audit [INF] from='client.? v1:172.21.15.104:0/1681790507' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:46:05.098 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:05.113 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:46:04 smithi055 bash[14301]: debug 2020-09-08T15:46:04.664+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:05.113 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:04 smithi055 bash[9296]: cluster 2020-09-08T15:46:03.725518+0000 mgr.x (mgr.24118) 562 : cluster [DBG] pgmap v768: 17 pgs: 1 active+clean+wait, 1 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 678 op/s; 2708/71048 objects misplaced (3.812%); 13 MiB/s, 213 objects/s recovering 2020-09-08T15:46:05.113 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:04 smithi055 bash[9296]: audit 2020-09-08T15:46:03.870695+0000 mon.a (mon.0) 1270 : audit [INF] from='client.? v1:172.21.15.104:0/1681790507' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:46:05.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:04 smithi104 bash[9203]: cluster 2020-09-08T15:46:03.725518+0000 mgr.x (mgr.24118) 562 : cluster [DBG] pgmap v768: 17 pgs: 1 active+clean+wait, 1 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 678 op/s; 2708/71048 objects misplaced (3.812%); 13 MiB/s, 213 objects/s recovering 2020-09-08T15:46:05.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:04 smithi104 bash[9203]: audit 2020-09-08T15:46:03.870695+0000 mon.a (mon.0) 1270 : audit [INF] from='client.? v1:172.21.15.104:0/1681790507' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:46:05.177 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:46:05 smithi104 bash[17588]: debug 2020-09-08T15:46:04.994+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:05.213 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:05.318 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:05.408 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:46:05 smithi055 bash[14301]: debug 2020-09-08T15:46:05.216+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:05.646 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:05.751 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:05.883 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:05.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:46:05 smithi104 bash[22305]: debug 2020-09-08T15:46:05.646+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:05.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:05 smithi104 bash[9203]: cluster 2020-09-08T15:46:04.756017+0000 mon.a (mon.0) 1271 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:46:05.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:05 smithi104 bash[9203]: cluster 2020-09-08T15:46:04.756095+0000 mon.a (mon.0) 1272 : cluster [INF] Cluster is now healthy 2020-09-08T15:46:05.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:05 smithi104 bash[9203]: audit 2020-09-08T15:46:04.782293+0000 mon.a (mon.0) 1273 : audit [INF] from='client.? v1:172.21.15.104:0/1681790507' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:46:05.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:05 smithi104 bash[9203]: cluster 2020-09-08T15:46:04.782334+0000 mon.a (mon.0) 1274 : cluster [DBG] osdmap e291: 12 total, 12 up, 12 in 2020-09-08T15:46:05.989 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:06.108 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:46:05 smithi055 bash[19003]: debug 2020-09-08T15:46:05.884+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:06.108 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:05 smithi055 bash[9296]: cluster 2020-09-08T15:46:04.756017+0000 mon.a (mon.0) 1271 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:46:06.109 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:05 smithi055 bash[9296]: cluster 2020-09-08T15:46:04.756095+0000 mon.a (mon.0) 1272 : cluster [INF] Cluster is now healthy 2020-09-08T15:46:06.109 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:05 smithi055 bash[9296]: audit 2020-09-08T15:46:04.782293+0000 mon.a (mon.0) 1273 : audit [INF] from='client.? v1:172.21.15.104:0/1681790507' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:46:06.109 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:05 smithi055 bash[9296]: cluster 2020-09-08T15:46:04.782334+0000 mon.a (mon.0) 1274 : cluster [DBG] osdmap e291: 12 total, 12 up, 12 in 2020-09-08T15:46:06.109 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:06.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:05 smithi166 bash[9446]: cluster 2020-09-08T15:46:04.756017+0000 mon.a (mon.0) 1271 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:46:06.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:05 smithi166 bash[9446]: cluster 2020-09-08T15:46:04.756095+0000 mon.a (mon.0) 1272 : cluster [INF] Cluster is now healthy 2020-09-08T15:46:06.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:05 smithi166 bash[9446]: audit 2020-09-08T15:46:04.782293+0000 mon.a (mon.0) 1273 : audit [INF] from='client.? v1:172.21.15.104:0/1681790507' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:46:06.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:05 smithi166 bash[9446]: cluster 2020-09-08T15:46:04.782334+0000 mon.a (mon.0) 1274 : cluster [DBG] osdmap e291: 12 total, 12 up, 12 in 2020-09-08T15:46:06.214 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:06.364 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:06.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:46:06 smithi055 bash[16646]: debug 2020-09-08T15:46:06.108+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:06.468 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:46:06.477 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:46:06 smithi104 bash[22305]: debug 2020-09-08T15:46:06.366+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:06.580 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:46:06.685 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:46:06.779 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:46:06 smithi104 bash[19922]: debug 2020-09-08T15:46:06.582+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:06.779 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:06 smithi104 bash[9203]: cluster 2020-09-08T15:46:05.726325+0000 mgr.x (mgr.24118) 563 : cluster [DBG] pgmap v770: 17 pgs: 1 active+clean+wait, 1 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 520 op/s; 2708/70506 objects misplaced (3.841%); 11 MiB/s, 175 objects/s recovering 2020-09-08T15:46:06.815 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:46:06.926 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:46:06.983 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:46:07.067 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:06 smithi166 bash[9446]: cluster 2020-09-08T15:46:05.726325+0000 mgr.x (mgr.24118) 563 : cluster [DBG] pgmap v770: 17 pgs: 1 active+clean+wait, 1 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 520 op/s; 2708/70506 objects misplaced (3.841%); 11 MiB/s, 175 objects/s recovering 2020-09-08T15:46:07.068 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:46:07.079 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:46:07.080 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:46:07.134 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:46:07.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:06 smithi055 bash[9296]: cluster 2020-09-08T15:46:05.726325+0000 mgr.x (mgr.24118) 563 : cluster [DBG] pgmap v770: 17 pgs: 1 active+clean+wait, 1 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 520 op/s; 2708/70506 objects misplaced (3.841%); 11 MiB/s, 175 objects/s recovering 2020-09-08T15:46:07.176 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:07.177 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:46:06 smithi104 bash[19922]: debug 2020-09-08T15:46:06.814+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:07.390 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:46:07 smithi166 bash[11928]: debug 2020-09-08T15:46:07.070+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:07.764 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:07.782 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:46:07.867 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:07.925 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:46:07.948 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2020-09-08T15:46:07.996 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:08.087 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:46:08.103 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:46:08.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:46:07 smithi104 bash[22305]: debug 2020-09-08T15:46:07.766+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:08.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:46:08 smithi166 bash[18960]: debug 2020-09-08T15:46:07.998+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:08.479 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:46:08.582 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:46:08.782 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:46:08 smithi166 bash[11928]: debug 2020-09-08T15:46:08.478+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:08.984 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:46:09.088 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:46:09.102 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:08 smithi166 bash[9446]: cluster 2020-09-08T15:46:07.727193+0000 mgr.x (mgr.24118) 564 : cluster [DBG] pgmap v771: 17 pgs: 1 active+recovering+remapped, 1 active+clean+wait, 2 active+recovering, 13 active+clean; 2.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 451 op/s; 1710/70420 objects misplaced (2.428%); 18 MiB/s, 292 objects/s recovering 2020-09-08T15:46:09.102 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:46:08 smithi166 bash[14254]: debug 2020-09-08T15:46:08.982+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:09.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:08 smithi055 bash[9296]: cluster 2020-09-08T15:46:07.727193+0000 mgr.x (mgr.24118) 564 : cluster [DBG] pgmap v771: 17 pgs: 1 active+recovering+remapped, 1 active+clean+wait, 2 active+recovering, 13 active+clean; 2.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 451 op/s; 1710/70420 objects misplaced (2.428%); 18 MiB/s, 292 objects/s recovering 2020-09-08T15:46:09.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:08 smithi104 bash[9203]: cluster 2020-09-08T15:46:07.727193+0000 mgr.x (mgr.24118) 564 : cluster [DBG] pgmap v771: 17 pgs: 1 active+recovering+remapped, 1 active+clean+wait, 2 active+recovering, 13 active+clean; 2.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 451 op/s; 1710/70420 objects misplaced (2.428%); 18 MiB/s, 292 objects/s recovering 2020-09-08T15:46:09.276 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:46:09.375 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:46:09 smithi166 bash[11928]: debug 2020-09-08T15:46:09.278+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:09.381 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:09.503 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:09.604 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:09.658 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:46:09 smithi055 bash[19003]: debug 2020-09-08T15:46:09.492+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:09.721 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:09.825 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:09.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:09 smithi104 bash[9203]: cluster 2020-09-08T15:46:08.173346+0000 osd.7 (osd.7) 53 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:09.937 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:09 smithi104 bash[9203]: cluster 2020-09-08T15:46:08.173470+0000 osd.7 (osd.7) 54 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:09.942 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:09 smithi104 bash[9203]: cluster 2020-09-08T15:46:08.797857+0000 mon.a (mon.0) 1275 : cluster [DBG] osdmap e292: 12 total, 12 up, 12 in 2020-09-08T15:46:10.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:09 smithi166 bash[9446]: cluster 2020-09-08T15:46:08.173346+0000 osd.7 (osd.7) 53 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:10.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:09 smithi166 bash[9446]: cluster 2020-09-08T15:46:08.173470+0000 osd.7 (osd.7) 54 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:10.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:09 smithi166 bash[9446]: cluster 2020-09-08T15:46:08.797857+0000 mon.a (mon.0) 1275 : cluster [DBG] osdmap e292: 12 total, 12 up, 12 in 2020-09-08T15:46:10.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:09 smithi055 bash[9296]: cluster 2020-09-08T15:46:08.173346+0000 osd.7 (osd.7) 53 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:10.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:09 smithi055 bash[9296]: cluster 2020-09-08T15:46:08.173470+0000 osd.7 (osd.7) 54 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:10.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:09 smithi055 bash[9296]: cluster 2020-09-08T15:46:08.797857+0000 mon.a (mon.0) 1275 : cluster [DBG] osdmap e292: 12 total, 12 up, 12 in 2020-09-08T15:46:10.158 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:46:09 smithi055 bash[14301]: debug 2020-09-08T15:46:09.724+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:10.348 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:10.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:46:10 smithi104 bash[22305]: debug 2020-09-08T15:46:10.350+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:10.451 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:10.546 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:10.649 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:10.798 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:46:10 smithi055 bash[16646]: debug 2020-09-08T15:46:10.548+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:10.918 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:10 smithi166 bash[9446]: cluster 2020-09-08T15:46:09.727733+0000 mgr.x (mgr.24118) 565 : cluster [DBG] pgmap v773: 17 pgs: 1 active+clean+wait, 1 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 307 op/s; 1198/70420 objects misplaced (1.701%); 18 MiB/s, 284 objects/s recovering 2020-09-08T15:46:10.919 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:10 smithi166 bash[9446]: cluster 2020-09-08T15:46:09.753334+0000 osd.0 (osd.0) 35 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:10.919 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:10 smithi166 bash[9446]: cluster 2020-09-08T15:46:09.753448+0000 osd.0 (osd.0) 36 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:10.919 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:10 smithi166 bash[9446]: cluster 2020-09-08T15:46:10.695549+0000 mon.a (mon.0) 1276 : cluster [DBG] osdmap e293: 12 total, 12 up, 12 in 2020-09-08T15:46:10.920 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:11.022 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:11.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:10 smithi055 bash[9296]: cluster 2020-09-08T15:46:09.727733+0000 mgr.x (mgr.24118) 565 : cluster [DBG] pgmap v773: 17 pgs: 1 active+clean+wait, 1 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 307 op/s; 1198/70420 objects misplaced (1.701%); 18 MiB/s, 284 objects/s recovering 2020-09-08T15:46:11.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:10 smithi055 bash[9296]: cluster 2020-09-08T15:46:09.753334+0000 osd.0 (osd.0) 35 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:11.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:10 smithi055 bash[9296]: cluster 2020-09-08T15:46:09.753448+0000 osd.0 (osd.0) 36 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:11.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:10 smithi055 bash[9296]: cluster 2020-09-08T15:46:10.695549+0000 mon.a (mon.0) 1276 : cluster [DBG] osdmap e293: 12 total, 12 up, 12 in 2020-09-08T15:46:11.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:10 smithi104 bash[9203]: cluster 2020-09-08T15:46:09.727733+0000 mgr.x (mgr.24118) 565 : cluster [DBG] pgmap v773: 17 pgs: 1 active+clean+wait, 1 active+recovering+remapped, 2 active+recovering, 13 active+clean; 2.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 307 op/s; 1198/70420 objects misplaced (1.701%); 18 MiB/s, 284 objects/s recovering 2020-09-08T15:46:11.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:10 smithi104 bash[9203]: cluster 2020-09-08T15:46:09.753334+0000 osd.0 (osd.0) 35 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:11.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:10 smithi104 bash[9203]: cluster 2020-09-08T15:46:09.753448+0000 osd.0 (osd.0) 36 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:11.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:10 smithi104 bash[9203]: cluster 2020-09-08T15:46:10.695549+0000 mon.a (mon.0) 1276 : cluster [DBG] osdmap e293: 12 total, 12 up, 12 in 2020-09-08T15:46:11.366 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:46:10 smithi166 bash[16593]: debug 2020-09-08T15:46:10.918+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:11.509 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:11.613 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:46:11.773 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:46:11 smithi166 bash[18960]: debug 2020-09-08T15:46:11.506+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:11.774 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:46:11.878 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:12.019 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:12.025 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:11 smithi166 bash[9446]: cluster 2020-09-08T15:46:10.410444+0000 osd.3 (osd.3) 31 : cluster [DBG] 3.b scrub starts 2020-09-08T15:46:12.025 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:11 smithi166 bash[9446]: cluster 2020-09-08T15:46:11.044254+0000 osd.3 (osd.3) 32 : cluster [DBG] 3.b scrub ok 2020-09-08T15:46:12.026 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:46:11 smithi166 bash[14254]: debug 2020-09-08T15:46:11.774+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:12.026 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:46:12 smithi166 bash[16593]: debug 2020-09-08T15:46:12.018+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:12.127 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:12.137 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:11 smithi055 bash[9296]: cluster 2020-09-08T15:46:10.410444+0000 osd.3 (osd.3) 31 : cluster [DBG] 3.b scrub starts 2020-09-08T15:46:12.138 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:11 smithi055 bash[9296]: cluster 2020-09-08T15:46:11.044254+0000 osd.3 (osd.3) 32 : cluster [DBG] 3.b scrub ok 2020-09-08T15:46:12.154 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:46:12.154 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:46:12.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:11 smithi104 bash[9203]: cluster 2020-09-08T15:46:10.410444+0000 osd.3 (osd.3) 31 : cluster [DBG] 3.b scrub starts 2020-09-08T15:46:12.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:11 smithi104 bash[9203]: cluster 2020-09-08T15:46:11.044254+0000 osd.3 (osd.3) 32 : cluster [DBG] 3.b scrub ok 2020-09-08T15:46:12.262 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:12.367 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:12.408 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:46:12 smithi055 bash[16646]: debug 2020-09-08T15:46:12.260+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:12.614 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:12.693 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:46:12 smithi104 bash[22305]: debug 2020-09-08T15:46:12.614+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:12.718 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:13.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:12 smithi166 bash[9446]: cluster 2020-09-08T15:46:11.728216+0000 mgr.x (mgr.24118) 566 : cluster [DBG] pgmap v775: 17 pgs: 1 active+clean+wait, 1 active+recovering+remapped, 1 active+recovering, 14 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 40 op/s; 783/69946 objects misplaced (1.119%); 17 MiB/s, 277 objects/s recovering 2020-09-08T15:46:13.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:12 smithi055 bash[9296]: cluster 2020-09-08T15:46:11.728216+0000 mgr.x (mgr.24118) 566 : cluster [DBG] pgmap v775: 17 pgs: 1 active+clean+wait, 1 active+recovering+remapped, 1 active+recovering, 14 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 40 op/s; 783/69946 objects misplaced (1.119%); 17 MiB/s, 277 objects/s recovering 2020-09-08T15:46:13.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:12 smithi104 bash[9203]: cluster 2020-09-08T15:46:11.728216+0000 mgr.x (mgr.24118) 566 : cluster [DBG] pgmap v775: 17 pgs: 1 active+clean+wait, 1 active+recovering+remapped, 1 active+recovering, 14 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 40 op/s; 783/69946 objects misplaced (1.119%); 17 MiB/s, 277 objects/s recovering 2020-09-08T15:46:13.187 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:13.291 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:13.321 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2020-09-08T15:46:13.458 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:46:13.486 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:46:13 smithi104 bash[17588]: debug 2020-09-08T15:46:13.190+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:13.509 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:13.612 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:13.640 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:46:13.662 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:46:13.684 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:46:13.707 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:46:13.733 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:46:13.761 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:46:13.794 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:46:13.806 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:46:13 smithi104 bash[15273]: debug 2020-09-08T15:46:13.510+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:13.825 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:46:13.867 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:46:13.898 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:46:13.922 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:46:13.950 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:46:13.952 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:46:13.952 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:46:13.952 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:46:13.952 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:46:13.953 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:46:13.953 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:46:13.953 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:46:13.953 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:46:13.954 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:46:13.954 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:46:13.954 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:46:13.954 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:46:14.047 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:13 smithi166 bash[9446]: cluster 2020-09-08T15:46:12.503104+0000 osd.8 (osd.8) 39 : cluster [DBG] 3.9 scrub starts 2020-09-08T15:46:14.048 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:13 smithi166 bash[9446]: cluster 2020-09-08T15:46:12.820062+0000 mon.a (mon.0) 1277 : cluster [DBG] osdmap e294: 12 total, 12 up, 12 in 2020-09-08T15:46:14.048 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:13 smithi166 bash[9446]: cluster 2020-09-08T15:46:13.059585+0000 osd.8 (osd.8) 40 : cluster [DBG] 3.9 scrub ok 2020-09-08T15:46:14.048 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:13 smithi166 bash[9446]: audit 2020-09-08T15:46:13.618680+0000 mon.c (mon.2) 50 : audit [DBG] from='client.? v1:172.21.15.104:0/475015848' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:46:14.048 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:14.160 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:14.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:13 smithi055 bash[9296]: cluster 2020-09-08T15:46:12.503104+0000 osd.8 (osd.8) 39 : cluster [DBG] 3.9 scrub starts 2020-09-08T15:46:14.162 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:13 smithi055 bash[9296]: cluster 2020-09-08T15:46:12.820062+0000 mon.a (mon.0) 1277 : cluster [DBG] osdmap e294: 12 total, 12 up, 12 in 2020-09-08T15:46:14.162 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:13 smithi055 bash[9296]: cluster 2020-09-08T15:46:13.059585+0000 osd.8 (osd.8) 40 : cluster [DBG] 3.9 scrub ok 2020-09-08T15:46:14.162 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:13 smithi055 bash[9296]: audit 2020-09-08T15:46:13.618680+0000 mon.c (mon.2) 50 : audit [DBG] from='client.? v1:172.21.15.104:0/475015848' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:46:14.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:13 smithi104 bash[9203]: cluster 2020-09-08T15:46:12.503104+0000 osd.8 (osd.8) 39 : cluster [DBG] 3.9 scrub starts 2020-09-08T15:46:14.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:13 smithi104 bash[9203]: cluster 2020-09-08T15:46:12.820062+0000 mon.a (mon.0) 1277 : cluster [DBG] osdmap e294: 12 total, 12 up, 12 in 2020-09-08T15:46:14.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:13 smithi104 bash[9203]: cluster 2020-09-08T15:46:13.059585+0000 osd.8 (osd.8) 40 : cluster [DBG] 3.9 scrub ok 2020-09-08T15:46:14.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:13 smithi104 bash[9203]: audit 2020-09-08T15:46:13.618680+0000 mon.c (mon.2) 50 : audit [DBG] from='client.? v1:172.21.15.104:0/475015848' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:46:14.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:46:14 smithi166 bash[18960]: debug 2020-09-08T15:46:14.046+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:14.514 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:14.617 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:46:14.686 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:46:14.816 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:46:14 smithi166 bash[18960]: debug 2020-09-08T15:46:14.514+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:14.816 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:46:14.922 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:15.070 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:15.072 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:46:15 smithi055 bash[16646]: debug 2020-09-08T15:46:15.072+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:15.072 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:14 smithi055 bash[9296]: cluster 2020-09-08T15:46:13.728885+0000 mgr.x (mgr.24118) 567 : cluster [DBG] pgmap v777: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 348 op/s; 33/66240 objects misplaced (0.050%); 17 MiB/s, 279 objects/s recovering 2020-09-08T15:46:15.073 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:14 smithi055 bash[9296]: cluster 2020-09-08T15:46:13.829536+0000 mon.a (mon.0) 1278 : cluster [DBG] osdmap e295: 12 total, 12 up, 12 in 2020-09-08T15:46:15.073 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:14 smithi055 bash[9296]: audit 2020-09-08T15:46:14.139736+0000 mon.a (mon.0) 1279 : audit [INF] from='client.? v1:172.21.15.104:0/2743848409' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:46:15.073 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:14 smithi055 bash[9296]: cluster 2020-09-08T15:46:14.677605+0000 mon.a (mon.0) 1280 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:46:15.073 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:14 smithi055 bash[9296]: audit 2020-09-08T15:46:14.682464+0000 mon.a (mon.0) 1281 : audit [INF] from='client.? v1:172.21.15.104:0/2743848409' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:46:15.074 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:14 smithi055 bash[9296]: cluster 2020-09-08T15:46:14.682513+0000 mon.a (mon.0) 1282 : cluster [DBG] osdmap e296: 12 total, 12 up, 12 in 2020-09-08T15:46:15.102 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:46:14 smithi166 bash[11928]: debug 2020-09-08T15:46:14.818+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:15.102 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:14 smithi166 bash[9446]: cluster 2020-09-08T15:46:13.728885+0000 mgr.x (mgr.24118) 567 : cluster [DBG] pgmap v777: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 348 op/s; 33/66240 objects misplaced (0.050%); 17 MiB/s, 279 objects/s recovering 2020-09-08T15:46:15.103 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:14 smithi166 bash[9446]: cluster 2020-09-08T15:46:13.829536+0000 mon.a (mon.0) 1278 : cluster [DBG] osdmap e295: 12 total, 12 up, 12 in 2020-09-08T15:46:15.103 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:14 smithi166 bash[9446]: audit 2020-09-08T15:46:14.139736+0000 mon.a (mon.0) 1279 : audit [INF] from='client.? v1:172.21.15.104:0/2743848409' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:46:15.103 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:14 smithi166 bash[9446]: cluster 2020-09-08T15:46:14.677605+0000 mon.a (mon.0) 1280 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:46:15.103 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:14 smithi166 bash[9446]: audit 2020-09-08T15:46:14.682464+0000 mon.a (mon.0) 1281 : audit [INF] from='client.? v1:172.21.15.104:0/2743848409' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:46:15.103 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:14 smithi166 bash[9446]: cluster 2020-09-08T15:46:14.682513+0000 mon.a (mon.0) 1282 : cluster [DBG] osdmap e296: 12 total, 12 up, 12 in 2020-09-08T15:46:15.176 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:46:15.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:14 smithi104 bash[9203]: cluster 2020-09-08T15:46:13.728885+0000 mgr.x (mgr.24118) 567 : cluster [DBG] pgmap v777: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 348 op/s; 33/66240 objects misplaced (0.050%); 17 MiB/s, 279 objects/s recovering 2020-09-08T15:46:15.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:14 smithi104 bash[9203]: cluster 2020-09-08T15:46:13.829536+0000 mon.a (mon.0) 1278 : cluster [DBG] osdmap e295: 12 total, 12 up, 12 in 2020-09-08T15:46:15.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:14 smithi104 bash[9203]: audit 2020-09-08T15:46:14.139736+0000 mon.a (mon.0) 1279 : audit [INF] from='client.? v1:172.21.15.104:0/2743848409' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:46:15.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:14 smithi104 bash[9203]: cluster 2020-09-08T15:46:14.677605+0000 mon.a (mon.0) 1280 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:46:15.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:14 smithi104 bash[9203]: audit 2020-09-08T15:46:14.682464+0000 mon.a (mon.0) 1281 : audit [INF] from='client.? v1:172.21.15.104:0/2743848409' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:46:15.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:14 smithi104 bash[9203]: cluster 2020-09-08T15:46:14.682513+0000 mon.a (mon.0) 1282 : cluster [DBG] osdmap e296: 12 total, 12 up, 12 in 2020-09-08T15:46:15.374 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:46:15.474 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:46:15 smithi104 bash[19922]: debug 2020-09-08T15:46:15.374+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:15.491 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:15.618 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:15.721 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:15.824 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:46:15 smithi055 bash[11956]: debug 2020-09-08T15:46:15.620+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:15.840 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:15.945 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:46:16.072 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:46:15 smithi104 bash[17588]: debug 2020-09-08T15:46:15.841+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:16.072 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:15 smithi104 bash[9203]: cluster 2020-09-08T15:46:14.137185+0000 osd.7 (osd.7) 55 : cluster [DBG] 3.f scrub starts 2020-09-08T15:46:16.073 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:15 smithi104 bash[9203]: cluster 2020-09-08T15:46:14.566847+0000 osd.7 (osd.7) 56 : cluster [DBG] 3.f scrub ok 2020-09-08T15:46:16.073 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:15 smithi104 bash[9203]: cluster 2020-09-08T15:46:14.694020+0000 osd.1 (osd.1) 54 : cluster [DBG] 3.4 deep-scrub starts 2020-09-08T15:46:16.073 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:15 smithi104 bash[9203]: cluster 2020-09-08T15:46:14.743713+0000 osd.5 (osd.5) 37 : cluster [DBG] 3.5 scrub starts 2020-09-08T15:46:16.073 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:15 smithi104 bash[9203]: cluster 2020-09-08T15:46:15.153933+0000 osd.5 (osd.5) 38 : cluster [DBG] 3.5 scrub ok 2020-09-08T15:46:16.074 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:46:16.123 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:15 smithi166 bash[9446]: cluster 2020-09-08T15:46:14.137185+0000 osd.7 (osd.7) 55 : cluster [DBG] 3.f scrub starts 2020-09-08T15:46:16.123 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:15 smithi166 bash[9446]: cluster 2020-09-08T15:46:14.566847+0000 osd.7 (osd.7) 56 : cluster [DBG] 3.f scrub ok 2020-09-08T15:46:16.124 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:15 smithi166 bash[9446]: cluster 2020-09-08T15:46:14.694020+0000 osd.1 (osd.1) 54 : cluster [DBG] 3.4 deep-scrub starts 2020-09-08T15:46:16.124 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:15 smithi166 bash[9446]: cluster 2020-09-08T15:46:14.743713+0000 osd.5 (osd.5) 37 : cluster [DBG] 3.5 scrub starts 2020-09-08T15:46:16.124 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:15 smithi166 bash[9446]: cluster 2020-09-08T15:46:15.153933+0000 osd.5 (osd.5) 38 : cluster [DBG] 3.5 scrub ok 2020-09-08T15:46:16.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:15 smithi055 bash[9296]: cluster 2020-09-08T15:46:14.137185+0000 osd.7 (osd.7) 55 : cluster [DBG] 3.f scrub starts 2020-09-08T15:46:16.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:15 smithi055 bash[9296]: cluster 2020-09-08T15:46:14.566847+0000 osd.7 (osd.7) 56 : cluster [DBG] 3.f scrub ok 2020-09-08T15:46:16.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:15 smithi055 bash[9296]: cluster 2020-09-08T15:46:14.694020+0000 osd.1 (osd.1) 54 : cluster [DBG] 3.4 deep-scrub starts 2020-09-08T15:46:16.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:15 smithi055 bash[9296]: cluster 2020-09-08T15:46:14.743713+0000 osd.5 (osd.5) 37 : cluster [DBG] 3.5 scrub starts 2020-09-08T15:46:16.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:15 smithi055 bash[9296]: cluster 2020-09-08T15:46:15.153933+0000 osd.5 (osd.5) 38 : cluster [DBG] 3.5 scrub ok 2020-09-08T15:46:16.177 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:46:16.298 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:46:16.404 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:16.427 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:46:16 smithi104 bash[19922]: debug 2020-09-08T15:46:16.073+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:16.427 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:46:16 smithi104 bash[19922]: debug 2020-09-08T15:46:16.297+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:16.545 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:16.650 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:16.780 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:46:16 smithi055 bash[11956]: debug 2020-09-08T15:46:16.548+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:16.781 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:16.785 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:46:16.888 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:16.925 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:46:17.023 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:17.123 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:46:17.133 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:16 smithi166 bash[9446]: cluster 2020-09-08T15:46:15.639129+0000 osd.5 (osd.5) 39 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:17.133 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:16 smithi166 bash[9446]: cluster 2020-09-08T15:46:15.639286+0000 osd.5 (osd.5) 40 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:17.133 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:16 smithi166 bash[9446]: cluster 2020-09-08T15:46:15.729671+0000 mgr.x (mgr.24118) 568 : cluster [DBG] pgmap v780: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 491 op/s; 33/65472 objects misplaced (0.050%); 14 MiB/s, 231 objects/s recovering 2020-09-08T15:46:17.133 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:16 smithi166 bash[9446]: cluster 2020-09-08T15:46:15.842199+0000 mon.a (mon.0) 1283 : cluster [DBG] osdmap e297: 12 total, 12 up, 12 in 2020-09-08T15:46:17.134 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:46:17 smithi166 bash[18960]: debug 2020-09-08T15:46:17.017+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:17.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:46:16 smithi055 bash[19003]: debug 2020-09-08T15:46:16.784+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:17.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:16 smithi055 bash[9296]: cluster 2020-09-08T15:46:15.639129+0000 osd.5 (osd.5) 39 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:17.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:16 smithi055 bash[9296]: cluster 2020-09-08T15:46:15.639286+0000 osd.5 (osd.5) 40 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:17.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:16 smithi055 bash[9296]: cluster 2020-09-08T15:46:15.729671+0000 mgr.x (mgr.24118) 568 : cluster [DBG] pgmap v780: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 491 op/s; 33/65472 objects misplaced (0.050%); 14 MiB/s, 231 objects/s recovering 2020-09-08T15:46:17.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:16 smithi055 bash[9296]: cluster 2020-09-08T15:46:15.842199+0000 mon.a (mon.0) 1283 : cluster [DBG] osdmap e297: 12 total, 12 up, 12 in 2020-09-08T15:46:17.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:16 smithi104 bash[9203]: cluster 2020-09-08T15:46:15.639129+0000 osd.5 (osd.5) 39 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:17.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:16 smithi104 bash[9203]: cluster 2020-09-08T15:46:15.639286+0000 osd.5 (osd.5) 40 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:17.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:16 smithi104 bash[9203]: cluster 2020-09-08T15:46:15.729671+0000 mgr.x (mgr.24118) 568 : cluster [DBG] pgmap v780: 17 pgs: 1 active+recovering+remapped, 16 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 491 op/s; 33/65472 objects misplaced (0.050%); 14 MiB/s, 231 objects/s recovering 2020-09-08T15:46:17.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:16 smithi104 bash[9203]: cluster 2020-09-08T15:46:15.842199+0000 mon.a (mon.0) 1283 : cluster [DBG] osdmap e297: 12 total, 12 up, 12 in 2020-09-08T15:46:17.182 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:46:17.183 INFO:teuthology.orchestra.run.smithi166.stdout: "size": 20, 2020-09-08T15:46:17.183 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 600, 2020-09-08T15:46:17.183 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T15:46:17.183 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.184 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:168405 2.5 2:a65aa17e:::benchmark_data_smithi166_20551_object20785:head [delete] snapc 0=[] ondisk+write+known_if_redirected e172)", 2020-09-08T15:46:17.184 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:34.484279+0000", 2020-09-08T15:46:17.184 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 402.699562077, 2020-09-08T15:46:17.184 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.264170609, 2020-09-08T15:46:17.184 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:17.185 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:17.185 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:46:17.185 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:46:17.185 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:46:17.185 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 168405 2020-09-08T15:46:17.186 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.186 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:17.186 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.186 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:17.186 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484279+0000", 2020-09-08T15:46:17.186 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:17.187 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.187 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.187 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:17.187 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484279+0000", 2020-09-08T15:46:17.187 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999642 2020-09-08T15:46:17.188 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.188 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.188 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:17.188 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484243+0000", 2020-09-08T15:46:17.188 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.0168999999999998e-05 2020-09-08T15:46:17.189 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.189 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.189 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:17.189 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484324+0000", 2020-09-08T15:46:17.189 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3794999999999999e-05 2020-09-08T15:46:17.190 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.190 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.190 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:17.190 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484337+0000", 2020-09-08T15:46:17.190 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7969e-05 2020-09-08T15:46:17.190 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.191 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.191 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:17.191 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484355+0000", 2020-09-08T15:46:17.191 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7426e-05 2020-09-08T15:46:17.191 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.193 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.194 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:17.194 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484433+0000", 2020-09-08T15:46:17.194 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00028967899999999999 2020-09-08T15:46:17.194 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.195 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.195 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:17.196 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484722+0000", 2020-09-08T15:46:17.197 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013431700000000001 2020-09-08T15:46:17.198 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.198 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.198 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:46:17.199 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484857+0000", 2020-09-08T15:46:17.199 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.010789607 2020-09-08T15:46:17.201 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.201 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.201 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:46:17.201 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.495646+0000", 2020-09-08T15:46:17.202 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25252755100000002 2020-09-08T15:46:17.202 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.202 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.202 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:17.202 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.748174+0000", 2020-09-08T15:46:17.202 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018818 2020-09-08T15:46:17.203 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.203 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.203 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:46:17.203 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.748362+0000", 2020-09-08T15:46:17.203 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.7688000000000005e-05 2020-09-08T15:46:17.204 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.204 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.205 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:46:17.206 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.748450+0000", 2020-09-08T15:46:17.206 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.264170609 2020-09-08T15:46:17.206 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.206 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.206 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.207 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.207 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.207 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:168406 2.5 2:adf3a2f3:::benchmark_data_smithi166_20551_object20786:head [delete] snapc 0=[] ondisk+write+known_if_redirected e172)", 2020-09-08T15:46:17.207 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:34.484624+0000", 2020-09-08T15:46:17.207 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 402.69921686100002, 2020-09-08T15:46:17.208 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.27197898700000001, 2020-09-08T15:46:17.208 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:17.208 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:17.208 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:46:17.208 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:46:17.209 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:46:17.210 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 168406 2020-09-08T15:46:17.210 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.210 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:17.211 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.211 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:17.211 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484624+0000", 2020-09-08T15:46:17.211 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:17.211 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.212 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.212 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:17.212 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484624+0000", 2020-09-08T15:46:17.212 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999623 2020-09-08T15:46:17.212 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.212 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.213 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:17.214 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484587+0000", 2020-09-08T15:46:17.214 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.3590000000000004e-05 2020-09-08T15:46:17.214 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.214 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.215 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:17.215 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484670+0000", 2020-09-08T15:46:17.215 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4182000000000001e-05 2020-09-08T15:46:17.216 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.216 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.216 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:17.216 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484684+0000", 2020-09-08T15:46:17.217 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6139999999999997e-06 2020-09-08T15:46:17.217 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.217 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.217 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:17.217 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.484691+0000", 2020-09-08T15:46:17.218 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0058046310000000002 2020-09-08T15:46:17.218 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.218 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.218 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:17.218 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.490496+0000", 2020-09-08T15:46:17.219 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018183399999999999 2020-09-08T15:46:17.219 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.219 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.219 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:17.219 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.490678+0000", 2020-09-08T15:46:17.220 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017738500000000001 2020-09-08T15:46:17.220 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.220 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.221 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:46:17.221 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.490855+0000", 2020-09-08T15:46:17.221 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0051368480000000003 2020-09-08T15:46:17.222 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.222 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.222 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:46:17.222 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.495992+0000", 2020-09-08T15:46:17.222 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.26042808899999997 2020-09-08T15:46:17.222 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.223 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.223 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:17.223 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.756420+0000", 2020-09-08T15:46:17.223 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010587800000000001 2020-09-08T15:46:17.223 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.224 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.224 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:46:17.224 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.756526+0000", 2020-09-08T15:46:17.224 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7712000000000004e-05 2020-09-08T15:46:17.224 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.225 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.225 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:46:17.225 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.756603+0000", 2020-09-08T15:46:17.225 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.27197898700000001 2020-09-08T15:46:17.225 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.226 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.226 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.226 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.226 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.226 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:168416 2.5 2:ad81a029:::benchmark_data_smithi166_20551_object20796:head [delete] snapc 0=[] ondisk+write+known_if_redirected e172)", 2020-09-08T15:46:17.226 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:34.498712+0000", 2020-09-08T15:46:17.227 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 402.68512896700003, 2020-09-08T15:46:17.227 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.26572992400000001, 2020-09-08T15:46:17.227 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:17.227 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:17.227 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:46:17.228 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:46:17.228 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:46:17.228 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 168416 2020-09-08T15:46:17.228 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.228 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:17.229 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.229 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:17.229 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.498712+0000", 2020-09-08T15:46:17.229 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:17.229 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.230 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.230 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:17.230 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.498712+0000", 2020-09-08T15:46:17.230 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999766 2020-09-08T15:46:17.230 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.230 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.231 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:17.231 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.498689+0000", 2020-09-08T15:46:17.231 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.2722000000000001e-05 2020-09-08T15:46:17.232 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.232 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.232 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:17.232 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.498742+0000", 2020-09-08T15:46:17.240 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2274e-05 2020-09-08T15:46:17.243 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.243 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.243 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:17.243 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.498754+0000", 2020-09-08T15:46:17.244 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.4699999999999996e-06 2020-09-08T15:46:17.244 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.244 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.244 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:17.244 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.498761+0000", 2020-09-08T15:46:17.244 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1848999999999998e-05 2020-09-08T15:46:17.245 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.245 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.245 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:17.245 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.498813+0000", 2020-09-08T15:46:17.245 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000123433 2020-09-08T15:46:17.246 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.246 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.246 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:17.246 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.498937+0000", 2020-09-08T15:46:17.246 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013201200000000001 2020-09-08T15:46:17.247 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.247 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.247 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:46:17.247 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.499069+0000", 2020-09-08T15:46:17.247 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0054701159999999997 2020-09-08T15:46:17.248 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.248 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.248 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:46:17.248 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.504539+0000", 2020-09-08T15:46:17.248 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25956900100000002 2020-09-08T15:46:17.248 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.249 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.249 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:17.249 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.764108+0000", 2020-09-08T15:46:17.249 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018175299999999999 2020-09-08T15:46:17.249 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.250 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.250 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:46:17.250 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.764290+0000", 2020-09-08T15:46:17.250 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000152657 2020-09-08T15:46:17.250 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.251 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.251 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:46:17.252 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.764442+0000", 2020-09-08T15:46:17.252 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.26572992400000001 2020-09-08T15:46:17.252 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.252 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.252 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.253 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.253 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.253 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:168429 2.5 2:a057a1df:::benchmark_data_smithi166_20551_object20809:head [delete] snapc 0=[] ondisk+write+known_if_redirected e172)", 2020-09-08T15:46:17.253 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:34.509989+0000", 2020-09-08T15:46:17.254 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 402.673852157, 2020-09-08T15:46:17.256 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25484900199999999, 2020-09-08T15:46:17.256 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:17.256 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:17.256 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:46:17.256 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:46:17.257 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:46:17.257 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 168429 2020-09-08T15:46:17.257 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.258 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:17.258 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.258 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:17.258 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.509989+0000", 2020-09-08T15:46:17.258 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:17.259 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.259 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.259 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:17.259 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.509989+0000", 2020-09-08T15:46:17.259 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999571 2020-09-08T15:46:17.260 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.260 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.260 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:17.260 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.509946+0000", 2020-09-08T15:46:17.260 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.6836000000000006e-05 2020-09-08T15:46:17.261 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.261 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.261 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:17.261 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510043+0000", 2020-09-08T15:46:17.261 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6231000000000001e-05 2020-09-08T15:46:17.261 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.262 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.262 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:17.262 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510059+0000", 2020-09-08T15:46:17.262 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6209999999999998e-06 2020-09-08T15:46:17.262 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.263 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.263 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:17.263 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510066+0000", 2020-09-08T15:46:17.263 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00027533400000000002 2020-09-08T15:46:17.263 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.264 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.264 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:17.264 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510341+0000", 2020-09-08T15:46:17.264 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012089100000000001 2020-09-08T15:46:17.264 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.265 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.265 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:17.266 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510462+0000", 2020-09-08T15:46:17.266 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000100579 2020-09-08T15:46:17.266 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.266 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.267 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:46:17.267 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510563+0000", 2020-09-08T15:46:17.267 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0099092620000000003 2020-09-08T15:46:17.267 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.267 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.268 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:46:17.268 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.520472+0000", 2020-09-08T15:46:17.268 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.244098331 2020-09-08T15:46:17.268 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.268 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.269 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:17.269 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.764570+0000", 2020-09-08T15:46:17.269 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000185859 2020-09-08T15:46:17.269 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.269 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.270 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:46:17.270 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.764756+0000", 2020-09-08T15:46:17.270 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.2008000000000005e-05 2020-09-08T15:46:17.270 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.270 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.271 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:46:17.271 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.764838+0000", 2020-09-08T15:46:17.271 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25484900199999999 2020-09-08T15:46:17.271 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.271 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.272 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.272 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.272 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.272 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:168430 2.5 2:adddae02:::benchmark_data_smithi166_20551_object20810:head [delete] snapc 0=[] ondisk+write+known_if_redirected e172)", 2020-09-08T15:46:17.272 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:34.510140+0000", 2020-09-08T15:46:17.273 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 402.67370092700003, 2020-09-08T15:46:17.273 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25847826400000001, 2020-09-08T15:46:17.273 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:17.273 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:17.274 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:46:17.274 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:46:17.274 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:46:17.274 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 168430 2020-09-08T15:46:17.274 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.275 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:17.275 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.275 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:17.275 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510140+0000", 2020-09-08T15:46:17.275 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:17.276 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.276 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.276 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:17.276 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510140+0000", 2020-09-08T15:46:17.276 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999599 2020-09-08T15:46:17.276 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.277 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.277 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:17.277 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510100+0000", 2020-09-08T15:46:17.277 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8372e-05 2020-09-08T15:46:17.277 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.278 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.278 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:17.278 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510189+0000", 2020-09-08T15:46:17.278 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3943e-05 2020-09-08T15:46:17.278 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.279 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.279 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:17.279 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510203+0000", 2020-09-08T15:46:17.279 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1479999999999997e-06 2020-09-08T15:46:17.279 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.280 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.280 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:17.280 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.510208+0000", 2020-09-08T15:46:17.280 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0057710879999999997 2020-09-08T15:46:17.280 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.280 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.281 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:17.281 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.515979+0000", 2020-09-08T15:46:17.281 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000161865 2020-09-08T15:46:17.281 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.281 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.282 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:17.282 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.516141+0000", 2020-09-08T15:46:17.282 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012483500000000001 2020-09-08T15:46:17.283 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.283 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.283 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:46:17.283 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.516265+0000", 2020-09-08T15:46:17.283 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0043580379999999998 2020-09-08T15:46:17.284 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.284 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.284 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:46:17.284 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.520624+0000", 2020-09-08T15:46:17.284 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.24768425499999999 2020-09-08T15:46:17.285 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.285 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.285 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:17.285 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.768308+0000", 2020-09-08T15:46:17.285 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016047300000000001 2020-09-08T15:46:17.285 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.286 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.286 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:46:17.286 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.768468+0000", 2020-09-08T15:46:17.286 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0001504 2020-09-08T15:46:17.286 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.287 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.288 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:46:17.288 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.768619+0000", 2020-09-08T15:46:17.288 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25847826400000001 2020-09-08T15:46:17.288 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.289 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.289 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.289 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.289 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.289 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:168457 2.5 2:a50f2b9e:::benchmark_data_smithi166_20551_object20837:head [delete] snapc 0=[] ondisk+write+known_if_redirected e172)", 2020-09-08T15:46:17.290 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:34.545652+0000", 2020-09-08T15:46:17.290 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 402.638189356, 2020-09-08T15:46:17.290 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22335165500000001, 2020-09-08T15:46:17.291 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:17.291 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:17.291 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:46:17.291 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:46:17.291 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:46:17.292 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 168457 2020-09-08T15:46:17.292 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.292 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:17.292 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.293 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:17.293 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.545652+0000", 2020-09-08T15:46:17.293 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:17.293 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.294 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.294 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:17.294 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.545652+0000", 2020-09-08T15:46:17.294 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999709 2020-09-08T15:46:17.294 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.295 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.295 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:17.295 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.545623+0000", 2020-09-08T15:46:17.295 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6372999999999996e-05 2020-09-08T15:46:17.296 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.296 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.296 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:17.296 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.545689+0000", 2020-09-08T15:46:17.296 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4678000000000001e-05 2020-09-08T15:46:17.296 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.297 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.297 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:17.297 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.545704+0000", 2020-09-08T15:46:17.297 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.3829999999999992e-06 2020-09-08T15:46:17.298 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.298 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.298 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:17.298 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.545712+0000", 2020-09-08T15:46:17.298 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.6777000000000003e-05 2020-09-08T15:46:17.299 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.299 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.299 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:17.299 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.545759+0000", 2020-09-08T15:46:17.300 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013097400000000001 2020-09-08T15:46:17.300 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.300 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.300 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:17.300 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.545890+0000", 2020-09-08T15:46:17.301 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011462499999999999 2020-09-08T15:46:17.301 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.301 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.301 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:46:17.302 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.546004+0000", 2020-09-08T15:46:17.302 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0044159000000000004 2020-09-08T15:46:17.302 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.302 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.302 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:46:17.303 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.550420+0000", 2020-09-08T15:46:17.303 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21836440400000001 2020-09-08T15:46:17.303 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.303 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.303 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:17.304 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.768785+0000", 2020-09-08T15:46:17.304 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7306000000000007e-05 2020-09-08T15:46:17.304 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.305 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.305 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:46:17.305 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.768882+0000", 2020-09-08T15:46:17.305 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000121558 2020-09-08T15:46:17.305 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.306 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.306 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:46:17.306 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:34.769004+0000", 2020-09-08T15:46:17.306 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22335165500000001 2020-09-08T15:46:17.307 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.307 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.307 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.307 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.307 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.308 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:190014 2.5 2:a63b47c4:::benchmark_data_smithi166_20551_object42394:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:46:17.308 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:56.732376+0000", 2020-09-08T15:46:17.308 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 380.45146524, 2020-09-08T15:46:17.308 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.24903745499999999, 2020-09-08T15:46:17.309 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:17.309 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:17.309 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:46:17.309 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:46:17.310 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:46:17.310 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 190014 2020-09-08T15:46:17.310 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.310 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:17.310 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.311 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:17.311 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.732376+0000", 2020-09-08T15:46:17.312 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:17.312 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.312 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.312 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:17.312 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.732376+0000", 2020-09-08T15:46:17.313 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999585 2020-09-08T15:46:17.313 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.313 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.313 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:17.313 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.732335+0000", 2020-09-08T15:46:17.314 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.1034999999999998e-05 2020-09-08T15:46:17.314 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.314 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.314 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:17.314 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.732426+0000", 2020-09-08T15:46:17.315 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5119e-05 2020-09-08T15:46:17.315 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.315 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.315 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:17.315 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.732441+0000", 2020-09-08T15:46:17.316 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.7560000000000002e-06 2020-09-08T15:46:17.316 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.316 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.316 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:17.317 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.732448+0000", 2020-09-08T15:46:17.317 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9833000000000002e-05 2020-09-08T15:46:17.317 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.317 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.317 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:17.318 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.732497+0000", 2020-09-08T15:46:17.318 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015706199999999999 2020-09-08T15:46:17.318 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.318 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.318 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:17.319 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.732655+0000", 2020-09-08T15:46:17.319 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000177307 2020-09-08T15:46:17.319 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.319 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.319 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:46:17.320 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.732832+0000", 2020-09-08T15:46:17.320 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.011138075000000001 2020-09-08T15:46:17.320 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.320 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.320 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:46:17.321 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.743970+0000", 2020-09-08T15:46:17.321 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.237051084 2020-09-08T15:46:17.321 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.321 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.321 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:17.322 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.981021+0000", 2020-09-08T15:46:17.322 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000189249 2020-09-08T15:46:17.322 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.322 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.323 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:46:17.323 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.981210+0000", 2020-09-08T15:46:17.323 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00020328300000000001 2020-09-08T15:46:17.323 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.323 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.324 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:46:17.324 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.981414+0000", 2020-09-08T15:46:17.324 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.24903745499999999 2020-09-08T15:46:17.324 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.325 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.325 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.325 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.325 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.325 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:190017 2.5 2:a3e31b0f:::benchmark_data_smithi166_20551_object42397:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:46:17.326 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:56.743914+0000", 2020-09-08T15:46:17.326 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 380.43992695600002, 2020-09-08T15:46:17.326 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.23789671500000001, 2020-09-08T15:46:17.326 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:17.327 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:17.327 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:46:17.327 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:46:17.327 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:46:17.327 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 190017 2020-09-08T15:46:17.328 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.328 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:17.328 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.328 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:17.329 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.743914+0000", 2020-09-08T15:46:17.329 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:17.329 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.330 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.330 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:17.330 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.743914+0000", 2020-09-08T15:46:17.330 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.999958 2020-09-08T15:46:17.330 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.331 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.331 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:17.331 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.743872+0000", 2020-09-08T15:46:17.331 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.1538000000000003e-05 2020-09-08T15:46:17.332 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.332 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.332 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:17.332 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.743964+0000", 2020-09-08T15:46:17.332 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4995e-05 2020-09-08T15:46:17.333 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.333 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.333 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:17.333 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.743979+0000", 2020-09-08T15:46:17.334 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.4249999999999998e-06 2020-09-08T15:46:17.334 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.334 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.334 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:17.334 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.743987+0000", 2020-09-08T15:46:17.335 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1610000000000002e-05 2020-09-08T15:46:17.335 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.335 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.335 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:17.336 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.744039+0000", 2020-09-08T15:46:17.336 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000184709 2020-09-08T15:46:17.336 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.336 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.337 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:17.337 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.744224+0000", 2020-09-08T15:46:17.337 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0002028 2020-09-08T15:46:17.337 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.337 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.338 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:46:17.338 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.744426+0000", 2020-09-08T15:46:17.338 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0046687500000000002 2020-09-08T15:46:17.338 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.339 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.339 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:46:17.339 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.749095+0000", 2020-09-08T15:46:17.339 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.23249661399999999 2020-09-08T15:46:17.339 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.340 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.340 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:17.340 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.981592+0000", 2020-09-08T15:46:17.340 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010754100000000001 2020-09-08T15:46:17.340 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.341 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.341 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:46:17.341 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.981699+0000", 2020-09-08T15:46:17.341 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000111724 2020-09-08T15:46:17.342 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.342 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.342 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:46:17.343 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.981811+0000", 2020-09-08T15:46:17.343 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.23789671500000001 2020-09-08T15:46:17.343 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.343 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.343 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.344 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.344 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.344 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:190048 2.5 2:aada8004:::benchmark_data_smithi166_20551_object42428:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:46:17.344 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:56.766453+0000", 2020-09-08T15:46:17.344 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 380.41738861499999, 2020-09-08T15:46:17.345 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.238099114, 2020-09-08T15:46:17.345 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:17.345 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:17.345 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:46:17.346 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:46:17.347 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:46:17.347 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 190048 2020-09-08T15:46:17.347 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.348 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:17.348 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.348 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:17.348 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.766453+0000", 2020-09-08T15:46:17.348 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:17.349 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.349 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.349 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:17.349 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.766453+0000", 2020-09-08T15:46:17.349 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999776 2020-09-08T15:46:17.350 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.350 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.350 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:17.350 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.766430+0000", 2020-09-08T15:46:17.350 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9654000000000003e-05 2020-09-08T15:46:17.351 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.353 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.353 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:17.353 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.766480+0000", 2020-09-08T15:46:17.353 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1321999999999999e-05 2020-09-08T15:46:17.353 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.354 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.354 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:17.354 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.766491+0000", 2020-09-08T15:46:17.354 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.2060000000000001e-06 2020-09-08T15:46:17.354 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.355 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.355 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:17.355 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.766499+0000", 2020-09-08T15:46:17.355 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.4616000000000002e-05 2020-09-08T15:46:17.355 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.356 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.356 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:17.356 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.766533+0000", 2020-09-08T15:46:17.356 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013506999999999999 2020-09-08T15:46:17.356 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.357 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.357 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:17.357 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.766668+0000", 2020-09-08T15:46:17.357 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.5450000000000003e-05 2020-09-08T15:46:17.357 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.358 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.358 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:46:17.358 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.766764+0000", 2020-09-08T15:46:17.358 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.004176155 2020-09-08T15:46:17.358 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.358 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.359 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:46:17.359 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.770940+0000", 2020-09-08T15:46:17.359 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.23336604499999999 2020-09-08T15:46:17.361 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.361 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.362 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:17.362 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.004306+0000", 2020-09-08T15:46:17.362 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000123845 2020-09-08T15:46:17.362 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.362 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.363 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:46:17.363 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.004430+0000", 2020-09-08T15:46:17.363 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000121987 2020-09-08T15:46:17.363 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.363 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.364 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:46:17.364 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.004552+0000", 2020-09-08T15:46:17.364 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.238099114 2020-09-08T15:46:17.364 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.364 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.365 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.365 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.365 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.365 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:190078 2.5 2:a346554c:::benchmark_data_smithi166_20551_object42458:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:46:17.366 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:56.806910+0000", 2020-09-08T15:46:17.366 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 380.37693132700002, 2020-09-08T15:46:17.366 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19808690300000001, 2020-09-08T15:46:17.366 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:17.366 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:17.367 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:46:17.367 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:46:17.367 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:46:17.367 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 190078 2020-09-08T15:46:17.367 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.367 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:17.368 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.368 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:17.368 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.806910+0000", 2020-09-08T15:46:17.368 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:17.369 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.370 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.370 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:17.370 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.806910+0000", 2020-09-08T15:46:17.370 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999633 2020-09-08T15:46:17.370 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.371 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.371 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:17.371 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.806873+0000", 2020-09-08T15:46:17.371 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.8899999999999993e-05 2020-09-08T15:46:17.371 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.372 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.372 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:17.372 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.806952+0000", 2020-09-08T15:46:17.372 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7453999999999999e-05 2020-09-08T15:46:17.372 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.373 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.373 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:17.373 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.806970+0000", 2020-09-08T15:46:17.373 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.473e-06 2020-09-08T15:46:17.373 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.374 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.374 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:17.374 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.806978+0000", 2020-09-08T15:46:17.374 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0358999999999997e-05 2020-09-08T15:46:17.375 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.375 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.375 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:17.375 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.807029+0000", 2020-09-08T15:46:17.375 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000171743 2020-09-08T15:46:17.376 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.376 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.376 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:17.376 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.807200+0000", 2020-09-08T15:46:17.376 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018509099999999999 2020-09-08T15:46:17.377 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.377 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.377 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:46:17.377 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.807385+0000", 2020-09-08T15:46:17.377 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0092689729999999998 2020-09-08T15:46:17.378 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.378 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.378 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:46:17.378 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.816654+0000", 2020-09-08T15:46:17.378 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18805126799999999 2020-09-08T15:46:17.379 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.379 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.379 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:17.379 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.004706+0000", 2020-09-08T15:46:17.379 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014980899999999999 2020-09-08T15:46:17.379 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.380 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.380 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:46:17.380 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.004855+0000", 2020-09-08T15:46:17.380 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000141433 2020-09-08T15:46:17.380 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.381 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.381 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:46:17.381 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.004997+0000", 2020-09-08T15:46:17.381 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19808690300000001 2020-09-08T15:46:17.381 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.382 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.382 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.382 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.382 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.383 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:190080 2.5 2:ad471823:::benchmark_data_smithi166_20551_object42460:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:46:17.383 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:56.809412+0000", 2020-09-08T15:46:17.383 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 380.374429287, 2020-09-08T15:46:17.384 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19592162900000001, 2020-09-08T15:46:17.384 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:17.384 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:17.385 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:46:17.385 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:46:17.385 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:46:17.386 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 190080 2020-09-08T15:46:17.386 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.386 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:17.387 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.387 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:17.387 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.809412+0000", 2020-09-08T15:46:17.388 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:17.388 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.388 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.389 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:17.389 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.809412+0000", 2020-09-08T15:46:17.389 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999661 2020-09-08T15:46:17.390 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.390 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.390 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:17.391 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.809378+0000", 2020-09-08T15:46:17.391 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7377000000000001e-05 2020-09-08T15:46:17.391 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.392 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.392 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:17.392 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.809455+0000", 2020-09-08T15:46:17.393 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6597999999999999e-05 2020-09-08T15:46:17.393 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.393 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.394 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:17.394 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.809472+0000", 2020-09-08T15:46:17.394 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5200000000000003e-06 2020-09-08T15:46:17.395 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.395 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.395 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:17.396 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.809478+0000", 2020-09-08T15:46:17.396 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0020781829999999999 2020-09-08T15:46:17.396 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.397 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.397 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:17.397 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.811557+0000", 2020-09-08T15:46:17.398 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000173401 2020-09-08T15:46:17.398 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.398 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.399 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:17.399 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.811730+0000", 2020-09-08T15:46:17.399 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000195064 2020-09-08T15:46:17.400 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.400 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.400 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:46:17.401 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.811925+0000", 2020-09-08T15:46:17.401 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0049081790000000004 2020-09-08T15:46:17.402 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.402 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.402 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:46:17.403 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:56.816833+0000", 2020-09-08T15:46:17.403 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.188264762 2020-09-08T15:46:17.404 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.404 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.405 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:17.405 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.005098+0000", 2020-09-08T15:46:17.405 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000141617 2020-09-08T15:46:17.406 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.406 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.406 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:46:17.407 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.005240+0000", 2020-09-08T15:46:17.407 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.3998000000000004e-05 2020-09-08T15:46:17.407 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.408 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.408 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:46:17.408 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.005334+0000", 2020-09-08T15:46:17.409 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19592162900000001 2020-09-08T15:46:17.409 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.409 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.410 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.410 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.410 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.411 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:190996 2.5 2:accb40be:::benchmark_data_smithi166_20551_object43376:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:46:17.411 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:57.853842+0000", 2020-09-08T15:46:17.411 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 379.32999947000002, 2020-09-08T15:46:17.412 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.226087756, 2020-09-08T15:46:17.412 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:17.412 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:17.413 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:46:17.413 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:46:17.413 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:46:17.414 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 190996 2020-09-08T15:46:17.414 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.414 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:17.415 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.415 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:17.415 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.853842+0000", 2020-09-08T15:46:17.416 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:17.416 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.416 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.417 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:17.417 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.853842+0000", 2020-09-08T15:46:17.417 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999614 2020-09-08T15:46:17.418 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.418 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.418 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:17.419 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.853803+0000", 2020-09-08T15:46:17.419 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8703999999999996e-05 2020-09-08T15:46:17.419 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.420 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.420 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:17.420 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.853892+0000", 2020-09-08T15:46:17.421 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6443000000000002e-05 2020-09-08T15:46:17.421 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.421 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.422 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:17.422 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.853908+0000", 2020-09-08T15:46:17.422 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.083e-06 2020-09-08T15:46:17.423 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.423 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.423 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:17.424 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.853914+0000", 2020-09-08T15:46:17.424 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012618199999999999 2020-09-08T15:46:17.424 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.425 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.425 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:17.425 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.854041+0000", 2020-09-08T15:46:17.425 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000176226 2020-09-08T15:46:17.425 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.426 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.426 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:17.426 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.854217+0000", 2020-09-08T15:46:17.426 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013297 2020-09-08T15:46:17.427 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.427 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.427 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:46:17.428 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.854350+0000", 2020-09-08T15:46:17.428 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0042987959999999997 2020-09-08T15:46:17.428 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.429 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.429 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:46:17.429 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.858649+0000", 2020-09-08T15:46:17.430 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.220929282 2020-09-08T15:46:17.430 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.430 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.431 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:17.431 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.079578+0000", 2020-09-08T15:46:17.431 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016881200000000001 2020-09-08T15:46:17.432 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.432 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.432 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:46:17.433 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.079747+0000", 2020-09-08T15:46:17.433 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018289599999999999 2020-09-08T15:46:17.433 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.434 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.434 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:46:17.434 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.079930+0000", 2020-09-08T15:46:17.435 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.226087756 2020-09-08T15:46:17.435 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.435 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.436 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.436 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.436 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.437 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:191002 2.5 2:a497d67f:::benchmark_data_smithi166_20551_object43382:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:46:17.437 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:57.861883+0000", 2020-09-08T15:46:17.437 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 379.32195818600002, 2020-09-08T15:46:17.438 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21849718600000001, 2020-09-08T15:46:17.438 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:17.438 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:17.439 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:46:17.439 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:46:17.439 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:46:17.440 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 191002 2020-09-08T15:46:17.440 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.440 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:17.441 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.441 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:17.442 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.861883+0000", 2020-09-08T15:46:17.442 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:17.443 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.443 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.443 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:17.444 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.861883+0000", 2020-09-08T15:46:17.444 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999547 2020-09-08T15:46:17.444 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.445 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.445 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:17.445 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.861838+0000", 2020-09-08T15:46:17.446 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011824600000000001 2020-09-08T15:46:17.446 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.446 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.447 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:17.447 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.861956+0000", 2020-09-08T15:46:17.447 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.1251e-05 2020-09-08T15:46:17.448 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.448 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.449 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:17.449 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.861977+0000", 2020-09-08T15:46:17.449 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.1940000000000001e-06 2020-09-08T15:46:17.449 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.449 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.450 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:17.450 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.861985+0000", 2020-09-08T15:46:17.450 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.3094999999999999e-05 2020-09-08T15:46:17.450 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.450 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.451 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:17.451 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862038+0000", 2020-09-08T15:46:17.451 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012477500000000001 2020-09-08T15:46:17.451 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.451 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.452 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:17.452 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862162+0000", 2020-09-08T15:46:17.452 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016471899999999999 2020-09-08T15:46:17.452 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.452 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.453 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:46:17.453 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862327+0000", 2020-09-08T15:46:17.453 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.010215866000000001 2020-09-08T15:46:17.453 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.453 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.454 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:46:17.454 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.872543+0000", 2020-09-08T15:46:17.454 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20754151300000001 2020-09-08T15:46:17.454 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.454 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.454 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:17.455 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.080085+0000", 2020-09-08T15:46:17.455 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019456599999999999 2020-09-08T15:46:17.455 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.455 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.455 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:46:17.456 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.080279+0000", 2020-09-08T15:46:17.456 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000101184 2020-09-08T15:46:17.456 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.456 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.456 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:46:17.457 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.080380+0000", 2020-09-08T15:46:17.457 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21849718600000001 2020-09-08T15:46:17.457 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.457 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.457 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.458 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.458 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.458 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:191006 2.5 2:ab629002:::benchmark_data_smithi166_20551_object43386:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:46:17.458 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:57.862749+0000", 2020-09-08T15:46:17.458 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 379.32109243999997, 2020-09-08T15:46:17.458 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.224825307, 2020-09-08T15:46:17.459 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:17.459 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:17.459 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:46:17.459 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:46:17.459 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:46:17.460 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 191006 2020-09-08T15:46:17.460 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.460 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:17.460 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.460 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:17.461 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862749+0000", 2020-09-08T15:46:17.461 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:17.461 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.461 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.461 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:17.461 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862749+0000", 2020-09-08T15:46:17.462 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999537 2020-09-08T15:46:17.462 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.462 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.462 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:17.462 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862702+0000", 2020-09-08T15:46:17.463 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000106022 2020-09-08T15:46:17.463 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.463 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.463 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:17.463 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862808+0000", 2020-09-08T15:46:17.464 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.9309000000000001e-05 2020-09-08T15:46:17.464 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.464 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.464 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:17.464 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862828+0000", 2020-09-08T15:46:17.464 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2070000000000001e-06 2020-09-08T15:46:17.465 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.465 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.465 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:17.465 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.862837+0000", 2020-09-08T15:46:17.465 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0051266419999999998 2020-09-08T15:46:17.466 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.466 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.466 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:17.466 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.867964+0000", 2020-09-08T15:46:17.466 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014680499999999999 2020-09-08T15:46:17.467 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.467 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.467 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:17.467 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.868110+0000", 2020-09-08T15:46:17.467 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019938700000000001 2020-09-08T15:46:17.467 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.468 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.468 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:46:17.468 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.868310+0000", 2020-09-08T15:46:17.468 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0043860180000000002 2020-09-08T15:46:17.468 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.469 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.469 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:46:17.469 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.872696+0000", 2020-09-08T15:46:17.469 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21478683100000001 2020-09-08T15:46:17.469 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.470 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.470 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:17.470 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.087483+0000", 2020-09-08T15:46:17.470 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.3461000000000002e-05 2020-09-08T15:46:17.470 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.470 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.471 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:46:17.471 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.087536+0000", 2020-09-08T15:46:17.471 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.8077999999999997e-05 2020-09-08T15:46:17.471 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.471 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.472 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:46:17.472 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.087574+0000", 2020-09-08T15:46:17.472 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.224825307 2020-09-08T15:46:17.472 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.472 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.472 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.473 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.473 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.473 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:191024 2.5 2:a481b27b:::benchmark_data_smithi166_20551_object43404:head [delete] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-09-08T15:46:17.473 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:39:57.900157+0000", 2020-09-08T15:46:17.473 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 379.28368467199999, 2020-09-08T15:46:17.474 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18752587100000001, 2020-09-08T15:46:17.474 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:17.474 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:17.474 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:46:17.474 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:46:17.475 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:46:17.475 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 191024 2020-09-08T15:46:17.475 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.475 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:17.475 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.476 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:17.476 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.900157+0000", 2020-09-08T15:46:17.476 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:17.476 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.476 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.476 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:17.477 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.900157+0000", 2020-09-08T15:46:17.477 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999723 2020-09-08T15:46:17.477 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.477 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.477 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:17.478 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.900129+0000", 2020-09-08T15:46:17.478 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.1223999999999995e-05 2020-09-08T15:46:17.478 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.478 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.478 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:17.479 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.900190+0000", 2020-09-08T15:46:17.479 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3706999999999999e-05 2020-09-08T15:46:17.479 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.479 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.479 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:17.480 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.900204+0000", 2020-09-08T15:46:17.480 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.49e-06 2020-09-08T15:46:17.480 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.480 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.480 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:17.480 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.900209+0000", 2020-09-08T15:46:17.481 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8096999999999994e-05 2020-09-08T15:46:17.481 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.481 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.481 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:17.481 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.900298+0000", 2020-09-08T15:46:17.482 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000202022 2020-09-08T15:46:17.482 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.482 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.482 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:17.482 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.900500+0000", 2020-09-08T15:46:17.483 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00020663999999999999 2020-09-08T15:46:17.483 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.483 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.483 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:46:17.483 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.900706+0000", 2020-09-08T15:46:17.483 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0064340400000000002 2020-09-08T15:46:17.484 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.484 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.484 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:46:17.484 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:57.907140+0000", 2020-09-08T15:46:17.484 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18047359700000001 2020-09-08T15:46:17.485 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.485 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.485 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:17.485 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.087614+0000", 2020-09-08T15:46:17.485 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.9319000000000001e-05 2020-09-08T15:46:17.486 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.486 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.486 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:46:17.486 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.087653+0000", 2020-09-08T15:46:17.486 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.932e-05 2020-09-08T15:46:17.486 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.487 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.487 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:46:17.487 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:39:58.087682+0000", 2020-09-08T15:46:17.487 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18752587100000001 2020-09-08T15:46:17.487 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.488 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.488 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.488 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.488 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.488 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223538 2.5 2:af9447da:::benchmark_data_smithi166_20551_object75918:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:46:17.489 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.454811+0000", 2020-09-08T15:46:17.489 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 345.729030679, 2020-09-08T15:46:17.489 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.31664958199999999, 2020-09-08T15:46:17.489 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:17.489 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:17.490 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:46:17.490 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:46:17.490 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:46:17.490 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223538 2020-09-08T15:46:17.490 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.490 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:17.491 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.491 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:17.491 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454811+0000", 2020-09-08T15:46:17.491 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:17.491 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.492 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.492 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:17.492 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454811+0000", 2020-09-08T15:46:17.492 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.999958 2020-09-08T15:46:17.492 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.492 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.493 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:17.493 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454768+0000", 2020-09-08T15:46:17.493 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.0786999999999997e-05 2020-09-08T15:46:17.493 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.493 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.494 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:17.494 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454859+0000", 2020-09-08T15:46:17.494 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.0344999999999999e-05 2020-09-08T15:46:17.494 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.494 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.495 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:17.495 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454880+0000", 2020-09-08T15:46:17.495 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.5410000000000003e-06 2020-09-08T15:46:17.495 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.495 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.495 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:17.496 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454887+0000", 2020-09-08T15:46:17.496 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.7349999999999999e-05 2020-09-08T15:46:17.496 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.496 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.496 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:17.497 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454935+0000", 2020-09-08T15:46:17.497 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000126422 2020-09-08T15:46:17.497 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.497 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.497 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:17.498 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.455061+0000", 2020-09-08T15:46:17.498 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013292600000000001 2020-09-08T15:46:17.498 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.498 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.498 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:46:17.498 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.455194+0000", 2020-09-08T15:46:17.499 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.010390197 2020-09-08T15:46:17.499 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.499 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.499 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:46:17.499 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.465584+0000", 2020-09-08T15:46:17.500 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.305744764 2020-09-08T15:46:17.500 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.500 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.500 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:17.500 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771329+0000", 2020-09-08T15:46:17.501 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.0704999999999995e-05 2020-09-08T15:46:17.501 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.501 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.501 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:46:17.501 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771400+0000", 2020-09-08T15:46:17.501 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.0677999999999997e-05 2020-09-08T15:46:17.502 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.502 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.502 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:46:17.502 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771460+0000", 2020-09-08T15:46:17.502 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.31664958199999999 2020-09-08T15:46:17.503 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.503 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.503 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.504 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.504 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.504 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223539 2.5 2:ac3e761c:::benchmark_data_smithi166_20551_object75919:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:46:17.504 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.455043+0000", 2020-09-08T15:46:17.504 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 345.728798111, 2020-09-08T15:46:17.505 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.316615533, 2020-09-08T15:46:17.505 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:17.505 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:17.505 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:46:17.506 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:46:17.506 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:46:17.506 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223539 2020-09-08T15:46:17.506 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.506 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:17.507 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.507 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:17.507 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.455043+0000", 2020-09-08T15:46:17.507 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:17.507 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.508 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.508 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:17.508 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.455043+0000", 2020-09-08T15:46:17.508 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999542 2020-09-08T15:46:17.508 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.508 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.509 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:17.509 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454997+0000", 2020-09-08T15:46:17.509 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000103986 2020-09-08T15:46:17.509 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.509 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.510 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:17.510 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.455101+0000", 2020-09-08T15:46:17.510 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6912999999999999e-05 2020-09-08T15:46:17.510 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.510 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.511 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:17.511 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.455118+0000", 2020-09-08T15:46:17.511 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.9060000000000002e-06 2020-09-08T15:46:17.511 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.511 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.511 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:17.512 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.455126+0000", 2020-09-08T15:46:17.512 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0052929229999999997 2020-09-08T15:46:17.512 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.512 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.512 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:17.513 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.460419+0000", 2020-09-08T15:46:17.513 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018929099999999999 2020-09-08T15:46:17.513 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.513 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.513 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:17.514 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.460608+0000", 2020-09-08T15:46:17.514 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000212525 2020-09-08T15:46:17.514 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.514 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.514 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:46:17.515 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.460821+0000", 2020-09-08T15:46:17.515 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0049404239999999997 2020-09-08T15:46:17.515 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.515 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.515 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:46:17.515 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.465761+0000", 2020-09-08T15:46:17.516 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.30577196099999998 2020-09-08T15:46:17.516 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.516 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.516 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:17.516 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771533+0000", 2020-09-08T15:46:17.517 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.7619000000000004e-05 2020-09-08T15:46:17.517 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.517 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.517 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:46:17.517 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771601+0000", 2020-09-08T15:46:17.518 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7948999999999997e-05 2020-09-08T15:46:17.518 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.518 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.518 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:46:17.518 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771659+0000", 2020-09-08T15:46:17.518 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.316615533 2020-09-08T15:46:17.519 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.519 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.519 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.519 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.519 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.520 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223553 2.5 2:af0eb85e:::benchmark_data_smithi166_20551_object75933:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:46:17.520 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.472693+0000", 2020-09-08T15:46:17.520 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 345.71114796799998, 2020-09-08T15:46:17.520 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.299064581, 2020-09-08T15:46:17.520 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:17.521 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:17.521 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:46:17.521 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:46:17.521 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:46:17.521 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223553 2020-09-08T15:46:17.521 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.522 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:17.522 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.522 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:17.522 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.472693+0000", 2020-09-08T15:46:17.522 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:17.523 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.523 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.523 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:17.523 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.472693+0000", 2020-09-08T15:46:17.523 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999433 2020-09-08T15:46:17.524 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.524 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.524 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:17.524 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.472636+0000", 2020-09-08T15:46:17.524 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000126057 2020-09-08T15:46:17.524 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.525 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.525 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:17.525 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.472762+0000", 2020-09-08T15:46:17.525 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.4116000000000001e-05 2020-09-08T15:46:17.525 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.526 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.526 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:17.526 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.472787+0000", 2020-09-08T15:46:17.526 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1021e-05 2020-09-08T15:46:17.526 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.527 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.527 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:17.527 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.472798+0000", 2020-09-08T15:46:17.527 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8405000000000001e-05 2020-09-08T15:46:17.527 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.527 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.528 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:17.528 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.472856+0000", 2020-09-08T15:46:17.528 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000129591 2020-09-08T15:46:17.528 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.528 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.529 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:17.529 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.472986+0000", 2020-09-08T15:46:17.529 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000115479 2020-09-08T15:46:17.529 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.529 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.530 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:46:17.530 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.473101+0000", 2020-09-08T15:46:17.530 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.010937853000000001 2020-09-08T15:46:17.530 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.530 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.530 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:46:17.531 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.484039+0000", 2020-09-08T15:46:17.531 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.28766051399999998 2020-09-08T15:46:17.531 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.531 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.531 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:17.532 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771699+0000", 2020-09-08T15:46:17.532 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.1099000000000002e-05 2020-09-08T15:46:17.532 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.532 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.532 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:46:17.532 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771731+0000", 2020-09-08T15:46:17.533 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.7396e-05 2020-09-08T15:46:17.533 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.533 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.533 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:46:17.533 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771758+0000", 2020-09-08T15:46:17.534 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.299064581 2020-09-08T15:46:17.534 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.534 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.534 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.534 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.535 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.535 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223558 2.5 2:ae593298:::benchmark_data_smithi166_20551_object75938:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:46:17.535 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.475216+0000", 2020-09-08T15:46:17.535 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 345.70862500099997, 2020-09-08T15:46:17.535 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.29670734500000001, 2020-09-08T15:46:17.535 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:17.536 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:17.536 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:46:17.536 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:46:17.536 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:46:17.536 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223558 2020-09-08T15:46:17.537 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.537 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:17.537 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.537 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:17.537 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.475216+0000", 2020-09-08T15:46:17.538 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:17.538 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.538 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.538 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:17.538 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.475216+0000", 2020-09-08T15:46:17.538 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999781 2020-09-08T15:46:17.539 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.539 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.539 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:17.539 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.475194+0000", 2020-09-08T15:46:17.539 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0077000000000002e-05 2020-09-08T15:46:17.540 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.540 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.540 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:17.540 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.475245+0000", 2020-09-08T15:46:17.540 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4603999999999999e-05 2020-09-08T15:46:17.541 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.541 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.541 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:17.541 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.475259+0000", 2020-09-08T15:46:17.541 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0710000000000002e-06 2020-09-08T15:46:17.542 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.542 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.542 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:17.542 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.475264+0000", 2020-09-08T15:46:17.543 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0035202369999999998 2020-09-08T15:46:17.543 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.543 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.543 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:17.543 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.478784+0000", 2020-09-08T15:46:17.544 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016699 2020-09-08T15:46:17.544 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.544 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.544 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:17.544 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.478951+0000", 2020-09-08T15:46:17.544 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000185355 2020-09-08T15:46:17.545 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.545 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.545 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:46:17.545 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.479137+0000", 2020-09-08T15:46:17.545 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0050839780000000003 2020-09-08T15:46:17.546 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.546 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.546 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:46:17.546 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.484221+0000", 2020-09-08T15:46:17.546 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.28760382800000001 2020-09-08T15:46:17.547 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.547 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.547 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:17.547 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771825+0000", 2020-09-08T15:46:17.547 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.2598e-05 2020-09-08T15:46:17.548 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.548 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.548 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:46:17.548 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771877+0000", 2020-09-08T15:46:17.548 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.6451e-05 2020-09-08T15:46:17.548 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.549 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.549 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:46:17.549 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.771924+0000", 2020-09-08T15:46:17.550 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.29670734500000001 2020-09-08T15:46:17.550 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.550 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.550 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.550 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.550 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.551 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223583 2.5 2:a056f7fe:::benchmark_data_smithi166_20551_object75963:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:46:17.551 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.520982+0000", 2020-09-08T15:46:17.551 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 345.66285883900002, 2020-09-08T15:46:17.551 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25551785999999999, 2020-09-08T15:46:17.551 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:17.552 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:46:17.552 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:46:17.552 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:46:17.552 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:46:17.552 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223583 2020-09-08T15:46:17.553 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.553 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:17.553 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.553 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:17.553 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.520982+0000", 2020-09-08T15:46:17.553 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:17.554 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.554 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.554 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:17.554 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.520982+0000", 2020-09-08T15:46:17.554 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999781 2020-09-08T15:46:17.555 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.555 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.555 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:17.555 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.520960+0000", 2020-09-08T15:46:17.555 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0510999999999998e-05 2020-09-08T15:46:17.556 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.556 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.556 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:17.556 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.521011+0000", 2020-09-08T15:46:17.556 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.6080000000000002e-06 2020-09-08T15:46:17.556 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.557 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.557 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:17.557 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.521020+0000", 2020-09-08T15:46:17.557 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1560000000000003e-06 2020-09-08T15:46:17.557 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.558 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.558 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:17.558 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.521026+0000", 2020-09-08T15:46:17.558 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.5790000000000005e-05 2020-09-08T15:46:17.558 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.558 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.559 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:17.559 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.521101+0000", 2020-09-08T15:46:17.559 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00025211500000000002 2020-09-08T15:46:17.559 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.559 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.560 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:17.560 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.521354+0000", 2020-09-08T15:46:17.560 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000217529 2020-09-08T15:46:17.560 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.560 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.561 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:46:17.561 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.521571+0000", 2020-09-08T15:46:17.561 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0049845890000000002 2020-09-08T15:46:17.561 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.561 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.561 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:46:17.562 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.526556+0000", 2020-09-08T15:46:17.562 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.24970619599999999 2020-09-08T15:46:17.562 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.562 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.562 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:46:17.563 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.776262+0000", 2020-09-08T15:46:17.563 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000159863 2020-09-08T15:46:17.563 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.563 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.563 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:46:17.564 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.776422+0000", 2020-09-08T15:46:17.564 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.8588e-05 2020-09-08T15:46:17.564 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:17.564 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:17.564 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:46:17.564 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.776500+0000", 2020-09-08T15:46:17.565 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25551785999999999 2020-09-08T15:46:17.565 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.565 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.565 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.565 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:17.566 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:17.566 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:46:17.566 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:46:17.568 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:46:17.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:46:17 smithi166 bash[11928]: debug 2020-09-08T15:46:17.221+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:17.670 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:17.698 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:46:17.840 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:17.946 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:17.963 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:46:17 smithi055 bash[11956]: debug 2020-09-08T15:46:17.844+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:18.021 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_ops_in_flight 2020-09-08T15:46:18.082 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:18.161 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:46:18.190 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:18.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:46:18 smithi055 bash[19003]: debug 2020-09-08T15:46:18.084+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:18.549 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:18.661 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:18.836 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:46:18 smithi104 bash[22305]: debug 2020-09-08T15:46:18.549+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:18.851 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:18.955 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:46:19.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:18 smithi166 bash[9446]: cluster 2020-09-08T15:46:17.730508+0000 mgr.x (mgr.24118) 569 : cluster [DBG] pgmap v782: 17 pgs: 1 active+clean+scrubbing+deep+wait, 16 active+clean; 2.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 404 op/s; 3.8 MiB/s, 66 objects/s recovering 2020-09-08T15:46:19.139 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:46:18 smithi104 bash[22305]: debug 2020-09-08T15:46:18.853+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:19.139 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:18 smithi104 bash[9203]: cluster 2020-09-08T15:46:17.730508+0000 mgr.x (mgr.24118) 569 : cluster [DBG] pgmap v782: 17 pgs: 1 active+clean+scrubbing+deep+wait, 16 active+clean; 2.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 404 op/s; 3.8 MiB/s, 66 objects/s recovering 2020-09-08T15:46:19.139 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:46:19.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:18 smithi055 bash[9296]: cluster 2020-09-08T15:46:17.730508+0000 mgr.x (mgr.24118) 569 : cluster [DBG] pgmap v782: 17 pgs: 1 active+clean+scrubbing+deep+wait, 16 active+clean; 2.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 404 op/s; 3.8 MiB/s, 66 objects/s recovering 2020-09-08T15:46:19.242 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:19.386 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:46:19 smithi104 bash[19922]: debug 2020-09-08T15:46:19.137+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:19.402 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:19.507 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:19.644 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:19.710 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:46:19 smithi104 bash[15273]: debug 2020-09-08T15:46:19.401+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:19.750 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:19.840 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:46:19 smithi166 bash[16593]: debug 2020-09-08T15:46:19.641+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:19.895 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:20.002 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:20.021 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:46:19 smithi055 bash[11956]: debug 2020-09-08T15:46:19.896+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:20.108 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:19 smithi055 bash[9296]: cluster 2020-09-08T15:46:19.239940+0000 osd.10 (osd.10) 25 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:20.109 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:19 smithi055 bash[9296]: cluster 2020-09-08T15:46:19.240027+0000 osd.10 (osd.10) 26 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:20.109 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:19 smithi055 bash[9296]: cluster 2020-09-08T15:46:19.686589+0000 mon.a (mon.0) 1284 : cluster [DBG] osdmap e298: 12 total, 12 up, 12 in 2020-09-08T15:46:20.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:19 smithi166 bash[9446]: cluster 2020-09-08T15:46:19.239940+0000 osd.10 (osd.10) 25 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:20.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:19 smithi166 bash[9446]: cluster 2020-09-08T15:46:19.240027+0000 osd.10 (osd.10) 26 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:20.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:19 smithi166 bash[9446]: cluster 2020-09-08T15:46:19.686589+0000 mon.a (mon.0) 1284 : cluster [DBG] osdmap e298: 12 total, 12 up, 12 in 2020-09-08T15:46:20.118 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:20.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:19 smithi104 bash[9203]: cluster 2020-09-08T15:46:19.239940+0000 osd.10 (osd.10) 25 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:20.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:19 smithi104 bash[9203]: cluster 2020-09-08T15:46:19.240027+0000 osd.10 (osd.10) 26 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:20.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:19 smithi104 bash[9203]: cluster 2020-09-08T15:46:19.686589+0000 mon.a (mon.0) 1284 : cluster [DBG] osdmap e298: 12 total, 12 up, 12 in 2020-09-08T15:46:20.223 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:20.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:46:20 smithi055 bash[14301]: debug 2020-09-08T15:46:20.120+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:20.645 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:20.749 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:21.118 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:46:20 smithi166 bash[16593]: debug 2020-09-08T15:46:20.645+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:21.126 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:21.234 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:21.353 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:21.445 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:46:21 smithi104 bash[22305]: debug 2020-09-08T15:46:21.129+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:21.462 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:21.575 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:21.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:46:21 smithi166 bash[18960]: debug 2020-09-08T15:46:21.569+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:21.619 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:46:21.619 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T15:46:21.619 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:21.619 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.35087.0:216677 3.2 e298/214 3:4e8f25f8:::benchmark_data_smithi166_26097_object92516:head v 298'13467, mlcod=298'13465)", 2020-09-08T15:46:21.620 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:46:21.619973+0000", 2020-09-08T15:46:21.620 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 0.0034784239999999999, 2020-09-08T15:46:21.620 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0035125209999999998, 2020-09-08T15:46:21.620 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:46:21.620 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "started", 2020-09-08T15:46:21.621 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:46:21.621 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:21.621 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:46:21.621 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:46:21.619973+0000", 2020-09-08T15:46:21.621 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:46:21.622 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:21.622 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:21.622 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:46:21.622 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:46:21.619973+0000", 2020-09-08T15:46:21.622 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.8361000000000004e-05 2020-09-08T15:46:21.623 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:21.623 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:21.623 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:46:21.623 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:46:21.620052+0000", 2020-09-08T15:46:21.623 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.799e-06 2020-09-08T15:46:21.624 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:21.624 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:21.624 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:46:21.624 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:46:21.620061+0000", 2020-09-08T15:46:21.624 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3047999999999999e-05 2020-09-08T15:46:21.625 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:21.625 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:21.625 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:46:21.625 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:46:21.620074+0000", 2020-09-08T15:46:21.625 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.993e-06 2020-09-08T15:46:21.626 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:21.626 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:21.626 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:46:21.626 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:46:21.620080+0000", 2020-09-08T15:46:21.626 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7967000000000003e-05 2020-09-08T15:46:21.627 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:21.627 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:21.627 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:46:21.627 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:46:21.620138+0000", 2020-09-08T15:46:21.627 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.1794000000000001e-05 2020-09-08T15:46:21.627 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:46:21.628 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:46:21.628 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:46:21.628 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:46:21.620169+0000", 2020-09-08T15:46:21.628 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019596199999999999 2020-09-08T15:46:21.628 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:21.629 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:46:21.629 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:21.629 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:46:21.629 INFO:teuthology.orchestra.run.smithi166.stdout: ], 2020-09-08T15:46:21.629 INFO:teuthology.orchestra.run.smithi166.stdout: "num_ops": 1 2020-09-08T15:46:21.630 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:46:21.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:46:21 smithi055 bash[16646]: debug 2020-09-08T15:46:21.348+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:21.681 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:46:21.873 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:46:21.976 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:21.986 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:21 smithi104 bash[9203]: cluster 2020-09-08T15:46:19.731176+0000 mgr.x (mgr.24118) 570 : cluster [DBG] pgmap v784: 17 pgs: 1 active+clean+scrubbing+deep+wait, 16 active+clean; 1.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 133 op/s; 0 B/s, 5 objects/s recovering 2020-09-08T15:46:22.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:46:21 smithi166 bash[14254]: debug 2020-09-08T15:46:21.869+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:22.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:21 smithi166 bash[9446]: cluster 2020-09-08T15:46:19.731176+0000 mgr.x (mgr.24118) 570 : cluster [DBG] pgmap v784: 17 pgs: 1 active+clean+scrubbing+deep+wait, 16 active+clean; 1.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 133 op/s; 0 B/s, 5 objects/s recovering 2020-09-08T15:46:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:21 smithi055 bash[9296]: cluster 2020-09-08T15:46:19.731176+0000 mgr.x (mgr.24118) 570 : cluster [DBG] pgmap v784: 17 pgs: 1 active+clean+scrubbing+deep+wait, 16 active+clean; 1.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 133 op/s; 0 B/s, 5 objects/s recovering 2020-09-08T15:46:22.206 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:22.310 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:22.313 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:46:22 smithi104 bash[15273]: debug 2020-09-08T15:46:22.205+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:22.443 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:22.543 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_blocked_ops 2020-09-08T15:46:22.564 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:22.612 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:46:22.640 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:46:22.665 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:46:22.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:46:22 smithi104 bash[22305]: debug 2020-09-08T15:46:22.441+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:22.677 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:22.689 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:46:22.693 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:46:22.720 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:46:22.748 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:46:22.782 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:22.786 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:46:22.822 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:46:22.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:22 smithi166 bash[9446]: audit 2020-09-08T15:46:22.574044+0000 mon.a (mon.0) 1285 : audit [DBG] from='client.? v1:172.21.15.104:0/704239882' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:46:22.868 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:46:22.895 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:46:22 smithi055 bash[19003]: debug 2020-09-08T15:46:22.680+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:22.895 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:22 smithi055 bash[9296]: audit 2020-09-08T15:46:22.574044+0000 mon.a (mon.0) 1285 : audit [DBG] from='client.? v1:172.21.15.104:0/704239882' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:46:22.896 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:22.897 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:46:22.930 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:46:22.958 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:46:22.960 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:46:22.961 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:46:22.961 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:46:22.961 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:46:22.961 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:46:22.962 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:46:22.962 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:46:22.962 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:46:22.962 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:46:22.962 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:46:22.963 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:46:22.963 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:46:22.998 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:23.034 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:22 smithi104 bash[9203]: audit 2020-09-08T15:46:22.574044+0000 mon.a (mon.0) 1285 : audit [DBG] from='client.? v1:172.21.15.104:0/704239882' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:46:23.097 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:23.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:46:23 smithi055 bash[19003]: debug 2020-09-08T15:46:23.100+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:23.158 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:46:22 smithi055 bash[14301]: debug 2020-09-08T15:46:22.892+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:23.200 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:23.711 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:46:24.010 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:24.013 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:23 smithi104 bash[9203]: cluster 2020-09-08T15:46:21.731861+0000 mgr.x (mgr.24118) 571 : cluster [DBG] pgmap v785: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 1.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 120 op/s; 0 B/s, 4 objects/s recovering 2020-09-08T15:46:24.013 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:23 smithi104 bash[9203]: audit 2020-09-08T15:46:22.942711+0000 mon.a (mon.0) 1286 : audit [INF] from='client.? v1:172.21.15.104:0/4220567352' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:46:24.115 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:46:24.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:23 smithi166 bash[9446]: cluster 2020-09-08T15:46:21.731861+0000 mgr.x (mgr.24118) 571 : cluster [DBG] pgmap v785: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 1.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 120 op/s; 0 B/s, 4 objects/s recovering 2020-09-08T15:46:24.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:23 smithi166 bash[9446]: audit 2020-09-08T15:46:22.942711+0000 mon.a (mon.0) 1286 : audit [INF] from='client.? v1:172.21.15.104:0/4220567352' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:46:24.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:23 smithi055 bash[9296]: cluster 2020-09-08T15:46:21.731861+0000 mgr.x (mgr.24118) 571 : cluster [DBG] pgmap v785: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 1.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 120 op/s; 0 B/s, 4 objects/s recovering 2020-09-08T15:46:24.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:23 smithi055 bash[9296]: audit 2020-09-08T15:46:22.942711+0000 mon.a (mon.0) 1286 : audit [INF] from='client.? v1:172.21.15.104:0/4220567352' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:46:24.294 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:46:24.397 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:46:24.419 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:46:24 smithi166 bash[14254]: debug 2020-09-08T15:46:24.293+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:24.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:46:24 smithi104 bash[17588]: debug 2020-09-08T15:46:24.009+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:24.700 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:24 smithi166 bash[9446]: cluster 2020-09-08T15:46:23.702432+0000 mon.a (mon.0) 1287 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:46:24.701 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:24 smithi166 bash[9446]: audit 2020-09-08T15:46:23.714331+0000 mon.a (mon.0) 1288 : audit [INF] from='client.? v1:172.21.15.104:0/4220567352' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:46:24.701 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:24 smithi166 bash[9446]: cluster 2020-09-08T15:46:23.714362+0000 mon.a (mon.0) 1289 : cluster [DBG] osdmap e299: 12 total, 12 up, 12 in 2020-09-08T15:46:24.701 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:24 smithi166 bash[9446]: cluster 2020-09-08T15:46:24.685922+0000 mon.a (mon.0) 1290 : cluster [DBG] osdmap e300: 12 total, 12 up, 12 in 2020-09-08T15:46:24.838 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:46:24.940 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:25.048 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:25.049 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:24 smithi055 bash[9296]: cluster 2020-09-08T15:46:23.702432+0000 mon.a (mon.0) 1287 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:46:25.049 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:24 smithi055 bash[9296]: audit 2020-09-08T15:46:23.714331+0000 mon.a (mon.0) 1288 : audit [INF] from='client.? v1:172.21.15.104:0/4220567352' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:46:25.049 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:24 smithi055 bash[9296]: cluster 2020-09-08T15:46:23.714362+0000 mon.a (mon.0) 1289 : cluster [DBG] osdmap e299: 12 total, 12 up, 12 in 2020-09-08T15:46:25.049 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:24 smithi055 bash[9296]: cluster 2020-09-08T15:46:24.685922+0000 mon.a (mon.0) 1290 : cluster [DBG] osdmap e300: 12 total, 12 up, 12 in 2020-09-08T15:46:25.050 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:46:25 smithi055 bash[14301]: debug 2020-09-08T15:46:25.048+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:25.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:46:24 smithi166 bash[14254]: debug 2020-09-08T15:46:24.833+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:25.162 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:25.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:24 smithi104 bash[9203]: cluster 2020-09-08T15:46:23.702432+0000 mon.a (mon.0) 1287 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:46:25.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:24 smithi104 bash[9203]: audit 2020-09-08T15:46:23.714331+0000 mon.a (mon.0) 1288 : audit [INF] from='client.? v1:172.21.15.104:0/4220567352' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:46:25.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:24 smithi104 bash[9203]: cluster 2020-09-08T15:46:23.714362+0000 mon.a (mon.0) 1289 : cluster [DBG] osdmap e299: 12 total, 12 up, 12 in 2020-09-08T15:46:25.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:24 smithi104 bash[9203]: cluster 2020-09-08T15:46:24.685922+0000 mon.a (mon.0) 1290 : cluster [DBG] osdmap e300: 12 total, 12 up, 12 in 2020-09-08T15:46:25.281 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:25.390 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:25.408 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:46:25 smithi055 bash[19003]: debug 2020-09-08T15:46:25.284+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:25.527 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:25.633 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:46:25.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:46:25 smithi104 bash[17588]: debug 2020-09-08T15:46:25.525+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:26.011 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:46:26.011 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:25 smithi166 bash[9446]: cluster 2020-09-08T15:46:23.732347+0000 mgr.x (mgr.24118) 572 : cluster [DBG] pgmap v787: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 1.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 166 op/s; 0 B/s, 4 objects/s recovering 2020-09-08T15:46:26.012 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:25 smithi166 bash[9446]: cluster 2020-09-08T15:46:24.512708+0000 osd.3 (osd.3) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:26.012 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:25 smithi166 bash[9446]: cluster 2020-09-08T15:46:24.512802+0000 osd.3 (osd.3) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:26.116 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:26.126 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:25 smithi104 bash[9203]: cluster 2020-09-08T15:46:23.732347+0000 mgr.x (mgr.24118) 572 : cluster [DBG] pgmap v787: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 1.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 166 op/s; 0 B/s, 4 objects/s recovering 2020-09-08T15:46:26.127 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:25 smithi104 bash[9203]: cluster 2020-09-08T15:46:24.512708+0000 osd.3 (osd.3) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:26.127 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:25 smithi104 bash[9203]: cluster 2020-09-08T15:46:24.512802+0000 osd.3 (osd.3) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:26.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:25 smithi055 bash[9296]: cluster 2020-09-08T15:46:23.732347+0000 mgr.x (mgr.24118) 572 : cluster [DBG] pgmap v787: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 1.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 166 op/s; 0 B/s, 4 objects/s recovering 2020-09-08T15:46:26.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:25 smithi055 bash[9296]: cluster 2020-09-08T15:46:24.512708+0000 osd.3 (osd.3) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:26.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:25 smithi055 bash[9296]: cluster 2020-09-08T15:46:24.512802+0000 osd.3 (osd.3) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:26.235 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:46:26.250 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:26.359 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:26.360 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:46:26 smithi166 bash[14254]: debug 2020-09-08T15:46:26.009+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:26.392 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:46:26.427 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:46:26 smithi104 bash[17588]: debug 2020-09-08T15:46:26.245+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:26.482 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:26.541 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:46:26.542 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:46:26.589 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:26.877 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:46:26.886 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:46:26 smithi104 bash[17588]: debug 2020-09-08T15:46:26.473+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:26.888 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:26 smithi104 bash[9203]: cluster 2020-09-08T15:46:25.732878+0000 mgr.x (mgr.24118) 573 : cluster [DBG] pgmap v789: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 1.9 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 121 op/s 2020-09-08T15:46:27.039 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:46:27.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:26 smithi166 bash[9446]: cluster 2020-09-08T15:46:25.732878+0000 mgr.x (mgr.24118) 573 : cluster [DBG] pgmap v789: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 1.9 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 121 op/s 2020-09-08T15:46:27.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:26 smithi055 bash[9296]: cluster 2020-09-08T15:46:25.732878+0000 mgr.x (mgr.24118) 573 : cluster [DBG] pgmap v789: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 1.9 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 121 op/s 2020-09-08T15:46:27.494 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_historic_ops 2020-09-08T15:46:27.637 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:27.651 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:46:27.742 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:46:27.927 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:46:27 smithi104 bash[22305]: debug 2020-09-08T15:46:27.637+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:28.244 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:46:28.349 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:46:28.616 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:46:28 smithi166 bash[14254]: debug 2020-09-08T15:46:28.241+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:28.845 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:46:28.947 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:29.096 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:28 smithi104 bash[9203]: cluster 2020-09-08T15:46:27.733392+0000 mgr.x (mgr.24118) 574 : cluster [DBG] pgmap v790: 17 pgs: 17 active+clean; 1.7 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 425 op/s 2020-09-08T15:46:29.096 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:29.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:46:28 smithi166 bash[11928]: debug 2020-09-08T15:46:28.841+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:29.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:28 smithi166 bash[9446]: cluster 2020-09-08T15:46:27.733392+0000 mgr.x (mgr.24118) 574 : cluster [DBG] pgmap v790: 17 pgs: 17 active+clean; 1.7 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 425 op/s 2020-09-08T15:46:29.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:28 smithi055 bash[9296]: cluster 2020-09-08T15:46:27.733392+0000 mgr.x (mgr.24118) 574 : cluster [DBG] pgmap v790: 17 pgs: 17 active+clean; 1.7 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 425 op/s 2020-09-08T15:46:29.199 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:29.299 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:29.404 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:29.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:46:29 smithi104 bash[17588]: debug 2020-09-08T15:46:29.085+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:29.514 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:46:29 smithi055 bash[16646]: debug 2020-09-08T15:46:29.300+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:29.536 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:29.638 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:29.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:46:29 smithi055 bash[16646]: debug 2020-09-08T15:46:29.512+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:30.212 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:30.315 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:30.418 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:30.523 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:30.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:46:30 smithi166 bash[16593]: debug 2020-09-08T15:46:30.209+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:30.626 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:46:30 smithi055 bash[11956]: debug 2020-09-08T15:46:30.420+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:30.627 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:30.731 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:46:30.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:46:30 smithi055 bash[19003]: debug 2020-09-08T15:46:30.628+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:30.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:30 smithi055 bash[9296]: cluster 2020-09-08T15:46:30.137845+0000 osd.8 (osd.8) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:30.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:30 smithi055 bash[9296]: cluster 2020-09-08T15:46:30.137939+0000 osd.8 (osd.8) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:31.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:30 smithi166 bash[9446]: cluster 2020-09-08T15:46:30.137845+0000 osd.8 (osd.8) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:31.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:30 smithi166 bash[9446]: cluster 2020-09-08T15:46:30.137939+0000 osd.8 (osd.8) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:31.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:30 smithi104 bash[9203]: cluster 2020-09-08T15:46:30.137845+0000 osd.8 (osd.8) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:31.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:30 smithi104 bash[9203]: cluster 2020-09-08T15:46:30.137939+0000 osd.8 (osd.8) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:31.303 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:46:31.412 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:31.520 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:31.624 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:31.678 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:46:31 smithi104 bash[19922]: debug 2020-09-08T15:46:31.301+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:31.695 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:46:31 smithi055 bash[19003]: debug 2020-09-08T15:46:31.520+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:31.696 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:31 smithi055 bash[9296]: cluster 2020-09-08T15:46:29.733811+0000 mgr.x (mgr.24118) 575 : cluster [DBG] pgmap v791: 17 pgs: 17 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 642 op/s 2020-09-08T15:46:31.696 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:31 smithi055 bash[9296]: cluster 2020-09-08T15:46:30.702744+0000 mon.a (mon.0) 1291 : cluster [DBG] osdmap e301: 12 total, 12 up, 12 in 2020-09-08T15:46:31.737 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:31.843 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:31.954 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:46:31.955 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:46:31.956 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:46:31 smithi055 bash[19003]: debug 2020-09-08T15:46:31.960+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:31.957 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:31.959 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:46:31 smithi055 bash[16646]: debug 2020-09-08T15:46:31.740+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:32.036 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:31 smithi166 bash[9446]: cluster 2020-09-08T15:46:29.733811+0000 mgr.x (mgr.24118) 575 : cluster [DBG] pgmap v791: 17 pgs: 17 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 642 op/s 2020-09-08T15:46:32.037 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:31 smithi166 bash[9446]: cluster 2020-09-08T15:46:30.702744+0000 mon.a (mon.0) 1291 : cluster [DBG] osdmap e301: 12 total, 12 up, 12 in 2020-09-08T15:46:32.062 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:46:32.071 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:31 smithi104 bash[9203]: cluster 2020-09-08T15:46:29.733811+0000 mgr.x (mgr.24118) 575 : cluster [DBG] pgmap v791: 17 pgs: 17 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 642 op/s 2020-09-08T15:46:32.071 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:31 smithi104 bash[9203]: cluster 2020-09-08T15:46:30.702744+0000 mon.a (mon.0) 1291 : cluster [DBG] osdmap e301: 12 total, 12 up, 12 in 2020-09-08T15:46:32.227 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:46:32.331 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:46:32.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:46:32 smithi104 bash[19922]: debug 2020-09-08T15:46:32.229+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:32.805 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:46:32.842 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:46:32.843 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:46:32.885 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:46:32.945 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:32.950 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:46:32.970 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2020-09-08T15:46:32.996 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:46:33.026 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:46:33.056 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:46:33.080 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:33.096 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:46:33.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:46:32 smithi166 bash[14254]: debug 2020-09-08T15:46:32.837+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:33.124 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:46:33.127 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:46:33.157 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:46:33.183 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:33.193 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:46:33.232 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:46:33.254 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:46:33.255 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:46:33.255 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:46:33.256 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:46:33.256 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:46:33.256 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:46:33.257 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:46:33.258 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:46:33.258 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:46:33.258 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:46:33.258 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:46:33.259 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:46:33.346 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:46:33 smithi104 bash[15273]: debug 2020-09-08T15:46:33.081+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:33.378 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:33.481 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:33.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:46:33 smithi055 bash[16646]: debug 2020-09-08T15:46:33.380+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:33.704 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:46:34.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:33 smithi166 bash[9446]: cluster 2020-09-08T15:46:31.734272+0000 mgr.x (mgr.24118) 576 : cluster [DBG] pgmap v793: 17 pgs: 17 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 867 op/s 2020-09-08T15:46:34.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:33 smithi166 bash[9446]: audit 2020-09-08T15:46:32.782493+0000 mon.a (mon.0) 1292 : audit [DBG] from='client.? v1:172.21.15.104:0/631081529' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:46:34.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:33 smithi166 bash[9446]: audit 2020-09-08T15:46:33.334959+0000 mon.a (mon.0) 1293 : audit [INF] from='client.? v1:172.21.15.104:0/3886750148' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:46:34.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:33 smithi055 bash[9296]: cluster 2020-09-08T15:46:31.734272+0000 mgr.x (mgr.24118) 576 : cluster [DBG] pgmap v793: 17 pgs: 17 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 867 op/s 2020-09-08T15:46:34.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:33 smithi055 bash[9296]: audit 2020-09-08T15:46:32.782493+0000 mon.a (mon.0) 1292 : audit [DBG] from='client.? v1:172.21.15.104:0/631081529' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:46:34.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:33 smithi055 bash[9296]: audit 2020-09-08T15:46:33.334959+0000 mon.a (mon.0) 1293 : audit [INF] from='client.? v1:172.21.15.104:0/3886750148' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:46:34.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:33 smithi104 bash[9203]: cluster 2020-09-08T15:46:31.734272+0000 mgr.x (mgr.24118) 576 : cluster [DBG] pgmap v793: 17 pgs: 17 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 867 op/s 2020-09-08T15:46:34.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:33 smithi104 bash[9203]: audit 2020-09-08T15:46:32.782493+0000 mon.a (mon.0) 1292 : audit [DBG] from='client.? v1:172.21.15.104:0/631081529' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:46:34.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:33 smithi104 bash[9203]: audit 2020-09-08T15:46:33.334959+0000 mon.a (mon.0) 1293 : audit [INF] from='client.? v1:172.21.15.104:0/3886750148' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:46:34.314 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:34.417 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:34.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:46:34 smithi104 bash[17588]: debug 2020-09-08T15:46:34.313+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:34.852 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:34 smithi166 bash[9446]: cluster 2020-09-08T15:46:33.701684+0000 mon.a (mon.0) 1294 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:46:34.853 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:34 smithi166 bash[9446]: audit 2020-09-08T15:46:33.705465+0000 mon.a (mon.0) 1295 : audit [INF] from='client.? v1:172.21.15.104:0/3886750148' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:46:34.853 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:34 smithi166 bash[9446]: cluster 2020-09-08T15:46:33.705509+0000 mon.a (mon.0) 1296 : cluster [DBG] osdmap e302: 12 total, 12 up, 12 in 2020-09-08T15:46:34.853 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:34 smithi166 bash[9446]: cluster 2020-09-08T15:46:33.734772+0000 mgr.x (mgr.24118) 577 : cluster [DBG] pgmap v795: 17 pgs: 17 active+clean; 1.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T15:46:34.854 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:34 smithi166 bash[9446]: cluster 2020-09-08T15:46:33.875031+0000 osd.1 (osd.1) 55 : cluster [DBG] 3.4 scrub starts 2020-09-08T15:46:35.099 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:34 smithi104 bash[9203]: cluster 2020-09-08T15:46:33.701684+0000 mon.a (mon.0) 1294 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:46:35.100 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:34 smithi104 bash[9203]: audit 2020-09-08T15:46:33.705465+0000 mon.a (mon.0) 1295 : audit [INF] from='client.? v1:172.21.15.104:0/3886750148' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:46:35.100 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:34 smithi104 bash[9203]: cluster 2020-09-08T15:46:33.705509+0000 mon.a (mon.0) 1296 : cluster [DBG] osdmap e302: 12 total, 12 up, 12 in 2020-09-08T15:46:35.100 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:34 smithi104 bash[9203]: cluster 2020-09-08T15:46:33.734772+0000 mgr.x (mgr.24118) 577 : cluster [DBG] pgmap v795: 17 pgs: 17 active+clean; 1.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T15:46:35.100 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:34 smithi104 bash[9203]: cluster 2020-09-08T15:46:33.875031+0000 osd.1 (osd.1) 55 : cluster [DBG] 3.4 scrub starts 2020-09-08T15:46:35.101 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:46:35 smithi104 bash[15273]: debug 2020-09-08T15:46:35.101+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:35.101 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:35.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:34 smithi055 bash[9296]: cluster 2020-09-08T15:46:33.701684+0000 mon.a (mon.0) 1294 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:46:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:34 smithi055 bash[9296]: audit 2020-09-08T15:46:33.705465+0000 mon.a (mon.0) 1295 : audit [INF] from='client.? v1:172.21.15.104:0/3886750148' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:46:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:34 smithi055 bash[9296]: cluster 2020-09-08T15:46:33.705509+0000 mon.a (mon.0) 1296 : cluster [DBG] osdmap e302: 12 total, 12 up, 12 in 2020-09-08T15:46:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:34 smithi055 bash[9296]: cluster 2020-09-08T15:46:33.734772+0000 mgr.x (mgr.24118) 577 : cluster [DBG] pgmap v795: 17 pgs: 17 active+clean; 1.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T15:46:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:34 smithi055 bash[9296]: cluster 2020-09-08T15:46:33.875031+0000 osd.1 (osd.1) 55 : cluster [DBG] 3.4 scrub starts 2020-09-08T15:46:35.205 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:35.583 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:35.687 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:35.838 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:35.868 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:46:35 smithi166 bash[18960]: debug 2020-09-08T15:46:35.581+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:35.943 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:36.076 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:36.178 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:46:35 smithi104 bash[15273]: debug 2020-09-08T15:46:35.829+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:36.183 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:36.298 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:46:36 smithi055 bash[16646]: debug 2020-09-08T15:46:36.076+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:36.299 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:46:36 smithi055 bash[11956]: debug 2020-09-08T15:46:36.296+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:36.299 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:36.402 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:36.526 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:36.639 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:36.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:46:36 smithi104 bash[15273]: debug 2020-09-08T15:46:36.525+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:36.840 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:36.909 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:36 smithi166 bash[9446]: cluster 2020-09-08T15:46:35.031512+0000 osd.7 (osd.7) 57 : cluster [DBG] 3.d scrub starts 2020-09-08T15:46:36.910 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:36 smithi166 bash[9446]: cluster 2020-09-08T15:46:35.292178+0000 osd.7 (osd.7) 58 : cluster [DBG] 3.d scrub ok 2020-09-08T15:46:36.910 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:36 smithi166 bash[9446]: cluster 2020-09-08T15:46:35.735220+0000 mgr.x (mgr.24118) 578 : cluster [DBG] pgmap v796: 17 pgs: 17 active+clean; 1.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 865 op/s 2020-09-08T15:46:36.910 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:46:36 smithi166 bash[16593]: debug 2020-09-08T15:46:36.837+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:36.943 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:37.085 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:36 smithi104 bash[9203]: cluster 2020-09-08T15:46:35.031512+0000 osd.7 (osd.7) 57 : cluster [DBG] 3.d scrub starts 2020-09-08T15:46:37.086 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:36 smithi104 bash[9203]: cluster 2020-09-08T15:46:35.292178+0000 osd.7 (osd.7) 58 : cluster [DBG] 3.d scrub ok 2020-09-08T15:46:37.086 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:36 smithi104 bash[9203]: cluster 2020-09-08T15:46:35.735220+0000 mgr.x (mgr.24118) 578 : cluster [DBG] pgmap v796: 17 pgs: 17 active+clean; 1.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 865 op/s 2020-09-08T15:46:37.089 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:37.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:36 smithi055 bash[9296]: cluster 2020-09-08T15:46:35.031512+0000 osd.7 (osd.7) 57 : cluster [DBG] 3.d scrub starts 2020-09-08T15:46:37.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:36 smithi055 bash[9296]: cluster 2020-09-08T15:46:35.292178+0000 osd.7 (osd.7) 58 : cluster [DBG] 3.d scrub ok 2020-09-08T15:46:37.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:36 smithi055 bash[9296]: cluster 2020-09-08T15:46:35.735220+0000 mgr.x (mgr.24118) 578 : cluster [DBG] pgmap v796: 17 pgs: 17 active+clean; 1.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 865 op/s 2020-09-08T15:46:37.193 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:37.213 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:46:37.214 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T15:46:37.214 INFO:teuthology.orchestra.run.smithi166.stdout: "num_ops": 0 2020-09-08T15:46:37.214 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:46:37.274 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:46:37.309 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:37.415 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:37.430 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:46:37.432 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:46:37 smithi104 bash[22305]: debug 2020-09-08T15:46:37.085+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:37.433 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:46:37.579 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:46:37.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:46:37 smithi055 bash[11956]: debug 2020-09-08T15:46:37.308+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:38.149 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2020-09-08T15:46:38.282 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:38.318 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:46:38.387 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:38.511 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:38.613 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:38.656 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:46:38 smithi055 bash[14301]: debug 2020-09-08T15:46:38.508+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:38.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:46:38 smithi104 bash[17588]: debug 2020-09-08T15:46:38.281+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:38.943 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:38 smithi166 bash[9446]: cluster 2020-09-08T15:46:37.735690+0000 mgr.x (mgr.24118) 579 : cluster [DBG] pgmap v797: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:46:38.944 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:38 smithi166 bash[9446]: cluster 2020-09-08T15:46:37.838066+0000 osd.1 (osd.1) 56 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:38.944 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:38 smithi166 bash[9446]: cluster 2020-09-08T15:46:37.838151+0000 osd.1 (osd.1) 57 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:38.944 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:46:38 smithi166 bash[16593]: debug 2020-09-08T15:46:38.941+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:38.947 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:39.052 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:39.060 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:38 smithi055 bash[9296]: cluster 2020-09-08T15:46:37.735690+0000 mgr.x (mgr.24118) 579 : cluster [DBG] pgmap v797: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:46:39.061 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:38 smithi055 bash[9296]: cluster 2020-09-08T15:46:37.838066+0000 osd.1 (osd.1) 56 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:39.061 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:38 smithi055 bash[9296]: cluster 2020-09-08T15:46:37.838151+0000 osd.1 (osd.1) 57 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:39.146 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:39.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:38 smithi104 bash[9203]: cluster 2020-09-08T15:46:37.735690+0000 mgr.x (mgr.24118) 579 : cluster [DBG] pgmap v797: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:46:39.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:38 smithi104 bash[9203]: cluster 2020-09-08T15:46:37.838066+0000 osd.1 (osd.1) 56 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:39.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:38 smithi104 bash[9203]: cluster 2020-09-08T15:46:37.838151+0000 osd.1 (osd.1) 57 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:39.251 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:39.363 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:46:39 smithi055 bash[14301]: debug 2020-09-08T15:46:39.144+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:39.363 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:39.470 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:39.586 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:39.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:46:39 smithi055 bash[16646]: debug 2020-09-08T15:46:39.364+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:39.658 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:46:39 smithi055 bash[19003]: debug 2020-09-08T15:46:39.584+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:39.691 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:39.878 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:39.981 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:40.022 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:46:39 smithi104 bash[22305]: debug 2020-09-08T15:46:39.877+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:40.023 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:39 smithi104 bash[9203]: cluster 2020-09-08T15:46:38.774343+0000 osd.6 (osd.6) 39 : cluster [DBG] 3.a scrub starts 2020-09-08T15:46:40.023 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:39 smithi104 bash[9203]: cluster 2020-09-08T15:46:38.794739+0000 mon.a (mon.0) 1297 : cluster [DBG] osdmap e303: 12 total, 12 up, 12 in 2020-09-08T15:46:40.023 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:39 smithi104 bash[9203]: cluster 2020-09-08T15:46:39.092275+0000 osd.6 (osd.6) 40 : cluster [DBG] 3.a scrub ok 2020-09-08T15:46:40.023 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:39 smithi104 bash[9203]: cluster 2020-09-08T15:46:39.692171+0000 mon.a (mon.0) 1298 : cluster [DBG] osdmap e304: 12 total, 12 up, 12 in 2020-09-08T15:46:40.082 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:39 smithi166 bash[9446]: cluster 2020-09-08T15:46:38.774343+0000 osd.6 (osd.6) 39 : cluster [DBG] 3.a scrub starts 2020-09-08T15:46:40.083 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:39 smithi166 bash[9446]: cluster 2020-09-08T15:46:38.794739+0000 mon.a (mon.0) 1297 : cluster [DBG] osdmap e303: 12 total, 12 up, 12 in 2020-09-08T15:46:40.083 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:39 smithi166 bash[9446]: cluster 2020-09-08T15:46:39.092275+0000 osd.6 (osd.6) 40 : cluster [DBG] 3.a scrub ok 2020-09-08T15:46:40.083 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:39 smithi166 bash[9446]: cluster 2020-09-08T15:46:39.692171+0000 mon.a (mon.0) 1298 : cluster [DBG] osdmap e304: 12 total, 12 up, 12 in 2020-09-08T15:46:40.154 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:40.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:39 smithi055 bash[9296]: cluster 2020-09-08T15:46:38.774343+0000 osd.6 (osd.6) 39 : cluster [DBG] 3.a scrub starts 2020-09-08T15:46:40.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:39 smithi055 bash[9296]: cluster 2020-09-08T15:46:38.794739+0000 mon.a (mon.0) 1297 : cluster [DBG] osdmap e303: 12 total, 12 up, 12 in 2020-09-08T15:46:40.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:39 smithi055 bash[9296]: cluster 2020-09-08T15:46:39.092275+0000 osd.6 (osd.6) 40 : cluster [DBG] 3.a scrub ok 2020-09-08T15:46:40.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:39 smithi055 bash[9296]: cluster 2020-09-08T15:46:39.692171+0000 mon.a (mon.0) 1298 : cluster [DBG] osdmap e304: 12 total, 12 up, 12 in 2020-09-08T15:46:40.255 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:40.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:46:40 smithi166 bash[18960]: debug 2020-09-08T15:46:40.133+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:40.946 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:41.049 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:41.164 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:41.273 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:41.282 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:46:40 smithi104 bash[17588]: debug 2020-09-08T15:46:40.945+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:41.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:46:41 smithi055 bash[11956]: debug 2020-09-08T15:46:41.164+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:41.598 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:46:41 smithi104 bash[22305]: debug 2020-09-08T15:46:41.597+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:41.598 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:41.702 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:41.816 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:41.924 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:41.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:41 smithi104 bash[9203]: cluster 2020-09-08T15:46:39.736142+0000 mgr.x (mgr.24118) 580 : cluster [DBG] pgmap v800: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 883 op/s 2020-09-08T15:46:42.008 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:41 smithi166 bash[9446]: cluster 2020-09-08T15:46:39.736142+0000 mgr.x (mgr.24118) 580 : cluster [DBG] pgmap v800: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 883 op/s 2020-09-08T15:46:42.050 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:41 smithi055 bash[9296]: cluster 2020-09-08T15:46:39.736142+0000 mgr.x (mgr.24118) 580 : cluster [DBG] pgmap v800: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 883 op/s 2020-09-08T15:46:42.051 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:46:41 smithi055 bash[19003]: debug 2020-09-08T15:46:41.816+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:42.051 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:42.051 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:46:42 smithi055 bash[14301]: debug 2020-09-08T15:46:42.052+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:42.154 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:42.281 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:46:42.281 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T15:46:42.281 INFO:teuthology.orchestra.run.smithi166.stdout: "complaint_time": 30, 2020-09-08T15:46:42.282 INFO:teuthology.orchestra.run.smithi166.stdout: "num_blocked_ops": 0 2020-09-08T15:46:42.282 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:46:42.421 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:42.524 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:42.630 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:42.679 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:46:42 smithi104 bash[22305]: debug 2020-09-08T15:46:42.421+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:42.733 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:42.835 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:46:42 smithi055 bash[16646]: debug 2020-09-08T15:46:42.632+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:42.836 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:42.937 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:43.136 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:43.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:46:42 smithi055 bash[16646]: debug 2020-09-08T15:46:42.832+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:43.225 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:46:43 smithi166 bash[16593]: debug 2020-09-08T15:46:43.133+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:43.226 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2020-09-08T15:46:43.240 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:46:43.388 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:46:43.409 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:46:43.515 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:43.619 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:46:43 smithi166 bash[11928]: debug 2020-09-08T15:46:43.401+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:43.625 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:43.699 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:43 smithi104 bash[9203]: cluster 2020-09-08T15:46:41.736667+0000 mgr.x (mgr.24118) 581 : cluster [DBG] pgmap v801: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 924 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 930 op/s 2020-09-08T15:46:43.699 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:43 smithi104 bash[9203]: cluster 2020-09-08T15:46:42.913595+0000 osd.6 (osd.6) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:43.700 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:43 smithi104 bash[9203]: cluster 2020-09-08T15:46:42.913680+0000 osd.6 (osd.6) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:43.728 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:43.788 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:46:43.830 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:46:43.878 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:46:43.885 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:43.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:46:43 smithi055 bash[19003]: debug 2020-09-08T15:46:43.624+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:43.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:43 smithi055 bash[9296]: cluster 2020-09-08T15:46:41.736667+0000 mgr.x (mgr.24118) 581 : cluster [DBG] pgmap v801: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 924 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 930 op/s 2020-09-08T15:46:43.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:43 smithi055 bash[9296]: cluster 2020-09-08T15:46:42.913595+0000 osd.6 (osd.6) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:43.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:43 smithi055 bash[9296]: cluster 2020-09-08T15:46:42.913680+0000 osd.6 (osd.6) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:43.922 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:46:43.958 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:46:43.990 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:43.998 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:46:44.000 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:46:43 smithi104 bash[17588]: debug 2020-09-08T15:46:43.885+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:44.037 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:46:44.077 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:46:44.104 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:46:44.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:43 smithi166 bash[9446]: cluster 2020-09-08T15:46:41.736667+0000 mgr.x (mgr.24118) 581 : cluster [DBG] pgmap v801: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 924 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 930 op/s 2020-09-08T15:46:44.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:43 smithi166 bash[9446]: cluster 2020-09-08T15:46:42.913595+0000 osd.6 (osd.6) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:44.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:43 smithi166 bash[9446]: cluster 2020-09-08T15:46:42.913680+0000 osd.6 (osd.6) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:44.117 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:44.154 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:46:44.174 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:46:44.221 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:44.279 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:46:44.286 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:46:44.286 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:46:44.287 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:46:44.287 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:46:44.287 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:46:44.287 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:46:44.287 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:46:44.288 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:46:44.288 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:46:44.288 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:46:44.288 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:46:44.288 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:46:44.344 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:44.379 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:46:44 smithi104 bash[17588]: debug 2020-09-08T15:46:44.117+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:44.454 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:44.658 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:46:44 smithi055 bash[11956]: debug 2020-09-08T15:46:44.344+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:44.687 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:44.696 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:46:44.792 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:45.007 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:44 smithi166 bash[9446]: cluster 2020-09-08T15:46:43.713631+0000 mon.a (mon.0) 1299 : cluster [DBG] osdmap e305: 12 total, 12 up, 12 in 2020-09-08T15:46:45.008 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:44 smithi166 bash[9446]: cluster 2020-09-08T15:46:43.737175+0000 mgr.x (mgr.24118) 582 : cluster [DBG] pgmap v803: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 789 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:46:45.008 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:44 smithi166 bash[9446]: audit 2020-09-08T15:46:43.752379+0000 mon.c (mon.2) 51 : audit [DBG] from='client.? v1:172.21.15.104:0/3001057874' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:46:45.008 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:44 smithi166 bash[9446]: audit 2020-09-08T15:46:43.990965+0000 mon.a (mon.0) 1300 : audit [INF] from='client.? v1:172.21.15.104:0/3519003789' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:46:45.008 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:44 smithi166 bash[9446]: cluster 2020-09-08T15:46:44.687135+0000 mon.a (mon.0) 1301 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:46:45.009 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:44 smithi166 bash[9446]: cluster 2020-09-08T15:46:44.687183+0000 mon.a (mon.0) 1302 : cluster [INF] Cluster is now healthy 2020-09-08T15:46:45.009 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:44 smithi166 bash[9446]: audit 2020-09-08T15:46:44.694177+0000 mon.a (mon.0) 1303 : audit [INF] from='client.? v1:172.21.15.104:0/3519003789' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:46:45.009 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:44 smithi166 bash[9446]: cluster 2020-09-08T15:46:44.694252+0000 mon.a (mon.0) 1304 : cluster [DBG] osdmap e306: 12 total, 12 up, 12 in 2020-09-08T15:46:45.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:46:44 smithi055 bash[16646]: debug 2020-09-08T15:46:44.688+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:45.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:44 smithi055 bash[9296]: cluster 2020-09-08T15:46:43.713631+0000 mon.a (mon.0) 1299 : cluster [DBG] osdmap e305: 12 total, 12 up, 12 in 2020-09-08T15:46:45.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:44 smithi055 bash[9296]: cluster 2020-09-08T15:46:43.737175+0000 mgr.x (mgr.24118) 582 : cluster [DBG] pgmap v803: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 789 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:46:45.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:44 smithi055 bash[9296]: audit 2020-09-08T15:46:43.752379+0000 mon.c (mon.2) 51 : audit [DBG] from='client.? v1:172.21.15.104:0/3001057874' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:46:45.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:44 smithi055 bash[9296]: audit 2020-09-08T15:46:43.990965+0000 mon.a (mon.0) 1300 : audit [INF] from='client.? v1:172.21.15.104:0/3519003789' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:46:45.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:44 smithi055 bash[9296]: cluster 2020-09-08T15:46:44.687135+0000 mon.a (mon.0) 1301 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:46:45.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:44 smithi055 bash[9296]: cluster 2020-09-08T15:46:44.687183+0000 mon.a (mon.0) 1302 : cluster [INF] Cluster is now healthy 2020-09-08T15:46:45.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:44 smithi055 bash[9296]: audit 2020-09-08T15:46:44.694177+0000 mon.a (mon.0) 1303 : audit [INF] from='client.? v1:172.21.15.104:0/3519003789' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:46:45.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:44 smithi055 bash[9296]: cluster 2020-09-08T15:46:44.694252+0000 mon.a (mon.0) 1304 : cluster [DBG] osdmap e306: 12 total, 12 up, 12 in 2020-09-08T15:46:45.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:44 smithi104 bash[9203]: cluster 2020-09-08T15:46:43.713631+0000 mon.a (mon.0) 1299 : cluster [DBG] osdmap e305: 12 total, 12 up, 12 in 2020-09-08T15:46:45.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:44 smithi104 bash[9203]: cluster 2020-09-08T15:46:43.737175+0000 mgr.x (mgr.24118) 582 : cluster [DBG] pgmap v803: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 789 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:46:45.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:44 smithi104 bash[9203]: audit 2020-09-08T15:46:43.752379+0000 mon.c (mon.2) 51 : audit [DBG] from='client.? v1:172.21.15.104:0/3001057874' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:46:45.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:44 smithi104 bash[9203]: audit 2020-09-08T15:46:43.990965+0000 mon.a (mon.0) 1300 : audit [INF] from='client.? v1:172.21.15.104:0/3519003789' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:46:45.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:44 smithi104 bash[9203]: cluster 2020-09-08T15:46:44.687135+0000 mon.a (mon.0) 1301 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:46:45.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:44 smithi104 bash[9203]: cluster 2020-09-08T15:46:44.687183+0000 mon.a (mon.0) 1302 : cluster [INF] Cluster is now healthy 2020-09-08T15:46:45.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:44 smithi104 bash[9203]: audit 2020-09-08T15:46:44.694177+0000 mon.a (mon.0) 1303 : audit [INF] from='client.? v1:172.21.15.104:0/3519003789' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:46:45.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:44 smithi104 bash[9203]: cluster 2020-09-08T15:46:44.694252+0000 mon.a (mon.0) 1304 : cluster [DBG] osdmap e306: 12 total, 12 up, 12 in 2020-09-08T15:46:45.572 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:45.675 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:45.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:46:45 smithi104 bash[17588]: debug 2020-09-08T15:46:45.573+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:46.084 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:46.187 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:46.308 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:46.415 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:46.423 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:46:46 smithi104 bash[15273]: debug 2020-09-08T15:46:46.081+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:46.527 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:46.648 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:46:46.656 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:46:46 smithi055 bash[14301]: debug 2020-09-08T15:46:46.308+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:46.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:46:46 smithi104 bash[15273]: debug 2020-09-08T15:46:46.525+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:47.032 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:46:47.033 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:46 smithi166 bash[9446]: cluster 2020-09-08T15:46:45.266630+0000 osd.11 (osd.11) 37 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:47.033 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:46 smithi166 bash[9446]: cluster 2020-09-08T15:46:45.266712+0000 osd.11 (osd.11) 38 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:47.033 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:46 smithi166 bash[9446]: cluster 2020-09-08T15:46:45.723300+0000 mon.a (mon.0) 1305 : cluster [DBG] osdmap e307: 12 total, 12 up, 12 in 2020-09-08T15:46:47.033 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:46 smithi166 bash[9446]: cluster 2020-09-08T15:46:45.737608+0000 mgr.x (mgr.24118) 583 : cluster [DBG] pgmap v806: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 753 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 809 op/s 2020-09-08T15:46:47.135 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:47.144 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:46 smithi104 bash[9203]: cluster 2020-09-08T15:46:45.266630+0000 osd.11 (osd.11) 37 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:47.144 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:46 smithi104 bash[9203]: cluster 2020-09-08T15:46:45.266712+0000 osd.11 (osd.11) 38 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:47.144 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:46 smithi104 bash[9203]: cluster 2020-09-08T15:46:45.723300+0000 mon.a (mon.0) 1305 : cluster [DBG] osdmap e307: 12 total, 12 up, 12 in 2020-09-08T15:46:47.145 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:46 smithi104 bash[9203]: cluster 2020-09-08T15:46:45.737608+0000 mgr.x (mgr.24118) 583 : cluster [DBG] pgmap v806: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 753 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 809 op/s 2020-09-08T15:46:47.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:46 smithi055 bash[9296]: cluster 2020-09-08T15:46:45.266630+0000 osd.11 (osd.11) 37 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:47.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:46 smithi055 bash[9296]: cluster 2020-09-08T15:46:45.266712+0000 osd.11 (osd.11) 38 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:47.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:46 smithi055 bash[9296]: cluster 2020-09-08T15:46:45.723300+0000 mon.a (mon.0) 1305 : cluster [DBG] osdmap e307: 12 total, 12 up, 12 in 2020-09-08T15:46:47.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:46 smithi055 bash[9296]: cluster 2020-09-08T15:46:45.737608+0000 mgr.x (mgr.24118) 583 : cluster [DBG] pgmap v806: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 753 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 809 op/s 2020-09-08T15:46:47.253 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:47.319 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:46:47 smithi166 bash[11928]: debug 2020-09-08T15:46:47.033+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:47.356 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:47.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:46:47 smithi104 bash[15273]: debug 2020-09-08T15:46:47.249+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:47.496 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:47.600 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:46:47.611 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:46:47.611 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:46:47.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:46:47 smithi055 bash[16646]: debug 2020-09-08T15:46:47.484+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:47.719 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:46:47.826 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:47.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:46:47 smithi166 bash[11928]: debug 2020-09-08T15:46:47.717+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:47.936 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:48.039 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:48.172 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:46:48 smithi104 bash[22305]: debug 2020-09-08T15:46:48.169+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:48.172 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:48.173 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:46:47 smithi104 bash[17588]: debug 2020-09-08T15:46:47.937+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:48.187 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:46:48.259 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:46:48.277 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:48.367 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:46:48.389 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:48.435 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:46:48.497 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:48.640 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2020-09-08T15:46:48.656 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:46:48 smithi055 bash[14301]: debug 2020-09-08T15:46:48.392+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:48.657 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:48.760 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:48.794 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:46:48 smithi166 bash[16593]: debug 2020-09-08T15:46:48.657+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:48.810 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:46:49.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:48 smithi166 bash[9446]: cluster 2020-09-08T15:46:47.738135+0000 mgr.x (mgr.24118) 584 : cluster [DBG] pgmap v807: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 571 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 940 op/s 2020-09-08T15:46:49.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:48 smithi055 bash[9296]: cluster 2020-09-08T15:46:47.738135+0000 mgr.x (mgr.24118) 584 : cluster [DBG] pgmap v807: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 571 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 940 op/s 2020-09-08T15:46:49.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:48 smithi104 bash[9203]: cluster 2020-09-08T15:46:47.738135+0000 mgr.x (mgr.24118) 584 : cluster [DBG] pgmap v807: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 571 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 940 op/s 2020-09-08T15:46:49.465 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:49.568 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:49.788 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:46:49 smithi104 bash[22305]: debug 2020-09-08T15:46:49.465+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:49.788 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:49 smithi104 bash[9203]: cluster 2020-09-08T15:46:49.128494+0000 osd.4 (osd.4) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:49.789 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:49 smithi104 bash[9203]: cluster 2020-09-08T15:46:49.128591+0000 osd.4 (osd.4) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:49.789 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:49 smithi104 bash[9203]: cluster 2020-09-08T15:46:49.692362+0000 mon.a (mon.0) 1306 : cluster [DBG] osdmap e308: 12 total, 12 up, 12 in 2020-09-08T15:46:49.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:49 smithi166 bash[9446]: cluster 2020-09-08T15:46:49.128494+0000 osd.4 (osd.4) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:49.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:49 smithi166 bash[9446]: cluster 2020-09-08T15:46:49.128591+0000 osd.4 (osd.4) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:49.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:49 smithi166 bash[9446]: cluster 2020-09-08T15:46:49.692362+0000 mon.a (mon.0) 1306 : cluster [DBG] osdmap e308: 12 total, 12 up, 12 in 2020-09-08T15:46:49.895 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:49.997 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:50.130 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:49 smithi055 bash[9296]: cluster 2020-09-08T15:46:49.128494+0000 osd.4 (osd.4) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:50.130 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:49 smithi055 bash[9296]: cluster 2020-09-08T15:46:49.128591+0000 osd.4 (osd.4) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:50.131 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:49 smithi055 bash[9296]: cluster 2020-09-08T15:46:49.692362+0000 mon.a (mon.0) 1306 : cluster [DBG] osdmap e308: 12 total, 12 up, 12 in 2020-09-08T15:46:50.132 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:50.183 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:46:49 smithi166 bash[16593]: debug 2020-09-08T15:46:49.885+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:50.236 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:46:50.379 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:46:50.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:46:50 smithi055 bash[14301]: debug 2020-09-08T15:46:50.132+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:50.482 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:50.483 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:46:50 smithi166 bash[14254]: debug 2020-09-08T15:46:50.365+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:50.590 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:50.699 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:50.841 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:46:50 smithi055 bash[16646]: debug 2020-09-08T15:46:50.592+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:50.842 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:50.947 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:51.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:46:50 smithi055 bash[14301]: debug 2020-09-08T15:46:50.844+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:51.279 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:51.384 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:51.619 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:46:51 smithi166 bash[18960]: debug 2020-09-08T15:46:51.277+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:51.702 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:51 smithi104 bash[9203]: cluster 2020-09-08T15:46:49.738627+0000 mgr.x (mgr.24118) 585 : cluster [DBG] pgmap v809: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 458 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 883 op/s 2020-09-08T15:46:52.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:51 smithi166 bash[9446]: cluster 2020-09-08T15:46:49.738627+0000 mgr.x (mgr.24118) 585 : cluster [DBG] pgmap v809: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 458 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 883 op/s 2020-09-08T15:46:52.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:51 smithi055 bash[9296]: cluster 2020-09-08T15:46:49.738627+0000 mgr.x (mgr.24118) 585 : cluster [DBG] pgmap v809: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 458 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 883 op/s 2020-09-08T15:46:52.531 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:52.634 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:52.820 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:46:52.923 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:52.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:46:52 smithi104 bash[15273]: debug 2020-09-08T15:46:52.529+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:52.935 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:46:52 smithi166 bash[18960]: debug 2020-09-08T15:46:52.817+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:53.126 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:46:53.127 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:46:54.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:53 smithi166 bash[9446]: cluster 2020-09-08T15:46:51.739101+0000 mgr.x (mgr.24118) 586 : cluster [DBG] pgmap v810: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 320 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 983 op/s 2020-09-08T15:46:54.148 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2020-09-08T15:46:54.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:53 smithi055 bash[9296]: cluster 2020-09-08T15:46:51.739101+0000 mgr.x (mgr.24118) 586 : cluster [DBG] pgmap v810: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 320 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 983 op/s 2020-09-08T15:46:54.187 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:53 smithi104 bash[9203]: cluster 2020-09-08T15:46:51.739101+0000 mgr.x (mgr.24118) 586 : cluster [DBG] pgmap v810: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 320 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 983 op/s 2020-09-08T15:46:54.189 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:54.310 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:46:54.318 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:54.534 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:46:54.602 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Removed 124158 objects 2020-09-08T15:46:54.602 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Clean up completed and total clean up time :183.362 2020-09-08T15:46:54.614 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:46:54 smithi104 bash[17588]: debug 2020-09-08T15:46:54.533+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:54.638 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:54.662 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:46:54 smithi055 bash[11956]: debug 2020-09-08T15:46:54.191+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:54.768 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:46:54.777 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:54.794 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:46:54.821 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:46:54.851 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:46:54.876 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:46:54.881 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:54.902 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:46:54.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:46:54 smithi104 bash[22305]: debug 2020-09-08T15:46:54.777+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:54.928 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:46:54.954 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:46:54.981 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:46:55.006 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:55.007 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:46:55.038 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:46:55.067 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:46:55.067 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:46:55.067 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:46:55.068 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:46:55.068 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:46:55.068 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:46:55.068 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:46:55.068 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:46:55.068 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:46:55.069 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:46:55.069 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:46:55.069 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:46:55.069 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:46:55.111 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:46:55.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:46:55 smithi055 bash[19003]: debug 2020-09-08T15:46:55.007+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:55.173 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_1 2020-09-08T15:46:55.174 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool rm unique_pool_1 unique_pool_1 --yes-i-really-really-mean-it 2020-09-08T15:46:55.322 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:46:55.335 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:46:55.427 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:55.526 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:46:55 smithi166 bash[11928]: debug 2020-09-08T15:46:55.321+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:55.726 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:46:55.789 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:55 smithi166 bash[9446]: cluster 2020-09-08T15:46:53.739527+0000 mgr.x (mgr.24118) 587 : cluster [DBG] pgmap v811: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 185 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T15:46:55.789 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:55 smithi166 bash[9446]: audit 2020-09-08T15:46:54.748116+0000 mon.a (mon.0) 1307 : audit [DBG] from='client.? v1:172.21.15.104:0/2776497809' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:46:55.790 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:55 smithi166 bash[9446]: audit 2020-09-08T15:46:54.998799+0000 mon.a (mon.0) 1308 : audit [INF] from='client.? v1:172.21.15.104:0/2300930493' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:46:56.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:55 smithi055 bash[9296]: cluster 2020-09-08T15:46:53.739527+0000 mgr.x (mgr.24118) 587 : cluster [DBG] pgmap v811: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 185 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T15:46:56.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:55 smithi055 bash[9296]: audit 2020-09-08T15:46:54.748116+0000 mon.a (mon.0) 1307 : audit [DBG] from='client.? v1:172.21.15.104:0/2776497809' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:46:56.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:55 smithi055 bash[9296]: audit 2020-09-08T15:46:54.998799+0000 mon.a (mon.0) 1308 : audit [INF] from='client.? v1:172.21.15.104:0/2300930493' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:46:56.241 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:55 smithi104 bash[9203]: cluster 2020-09-08T15:46:53.739527+0000 mgr.x (mgr.24118) 587 : cluster [DBG] pgmap v811: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 185 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T15:46:56.241 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:55 smithi104 bash[9203]: audit 2020-09-08T15:46:54.748116+0000 mon.a (mon.0) 1307 : audit [DBG] from='client.? v1:172.21.15.104:0/2776497809' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:46:56.242 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:55 smithi104 bash[9203]: audit 2020-09-08T15:46:54.998799+0000 mon.a (mon.0) 1308 : audit [INF] from='client.? v1:172.21.15.104:0/2300930493' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:46:56.323 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:46:56.429 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:46:56.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:46:56 smithi104 bash[15273]: debug 2020-09-08T15:46:56.325+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:56.788 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:46:56.894 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:57.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:56 smithi166 bash[9446]: cluster 2020-09-08T15:46:55.717011+0000 mon.a (mon.0) 1309 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:46:57.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:56 smithi166 bash[9446]: audit 2020-09-08T15:46:55.725812+0000 mon.a (mon.0) 1310 : audit [INF] from='client.? v1:172.21.15.104:0/2300930493' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:46:57.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:56 smithi166 bash[9446]: cluster 2020-09-08T15:46:55.725898+0000 mon.a (mon.0) 1311 : cluster [DBG] osdmap e309: 12 total, 12 up, 12 in 2020-09-08T15:46:57.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:56 smithi055 bash[9296]: cluster 2020-09-08T15:46:55.717011+0000 mon.a (mon.0) 1309 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:46:57.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:56 smithi055 bash[9296]: audit 2020-09-08T15:46:55.725812+0000 mon.a (mon.0) 1310 : audit [INF] from='client.? v1:172.21.15.104:0/2300930493' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:46:57.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:56 smithi055 bash[9296]: cluster 2020-09-08T15:46:55.725898+0000 mon.a (mon.0) 1311 : cluster [DBG] osdmap e309: 12 total, 12 up, 12 in 2020-09-08T15:46:57.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:46:56 smithi104 bash[19922]: debug 2020-09-08T15:46:56.789+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:57.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:56 smithi104 bash[9203]: cluster 2020-09-08T15:46:55.717011+0000 mon.a (mon.0) 1309 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:46:57.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:56 smithi104 bash[9203]: audit 2020-09-08T15:46:55.725812+0000 mon.a (mon.0) 1310 : audit [INF] from='client.? v1:172.21.15.104:0/2300930493' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:46:57.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:56 smithi104 bash[9203]: cluster 2020-09-08T15:46:55.725898+0000 mon.a (mon.0) 1311 : cluster [DBG] osdmap e309: 12 total, 12 up, 12 in 2020-09-08T15:46:57.207 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:46:57.309 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:57.459 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:46:57 smithi166 bash[16593]: debug 2020-09-08T15:46:57.193+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:57.483 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:46:57.590 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:57.612 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:46:57.612 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T15:46:57.613 INFO:teuthology.orchestra.run.smithi166.stdout: "complaint_time": 30, 2020-09-08T15:46:57.613 INFO:teuthology.orchestra.run.smithi166.stdout: "num_blocked_ops": 0 2020-09-08T15:46:57.613 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:46:57.721 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:46:57.823 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:57 smithi104 bash[9203]: cluster 2020-09-08T15:46:55.740127+0000 mgr.x (mgr.24118) 588 : cluster [DBG] pgmap v813: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 147 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 849 op/s 2020-09-08T15:46:57.823 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:46:57 smithi104 bash[22305]: debug 2020-09-08T15:46:57.485+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:57.828 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:57.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:46:57 smithi055 bash[16646]: debug 2020-09-08T15:46:57.723+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:57.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:57 smithi055 bash[9296]: cluster 2020-09-08T15:46:55.740127+0000 mgr.x (mgr.24118) 588 : cluster [DBG] pgmap v813: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 147 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 849 op/s 2020-09-08T15:46:57.958 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:46:58.066 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:58.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:57 smithi166 bash[9446]: cluster 2020-09-08T15:46:55.740127+0000 mgr.x (mgr.24118) 588 : cluster [DBG] pgmap v813: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 147 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 849 op/s 2020-09-08T15:46:58.191 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:46:58.301 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:46:58.320 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2020-09-08T15:46:58.321 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:46:57 smithi055 bash[11956]: debug 2020-09-08T15:46:57.959+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:58.331 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:46:58 smithi055 bash[14301]: debug 2020-09-08T15:46:58.191+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:46:58.466 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:46:58.856 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:46:58.993 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:46:59.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:58 smithi166 bash[9446]: cluster 2020-09-08T15:46:57.545694+0000 osd.9 (osd.9) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:59.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:58 smithi166 bash[9446]: cluster 2020-09-08T15:46:57.545829+0000 osd.9 (osd.9) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:59.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:58 smithi166 bash[9446]: cluster 2020-09-08T15:46:57.727796+0000 mon.a (mon.0) 1312 : cluster [DBG] osdmap e310: 12 total, 12 up, 12 in 2020-09-08T15:46:59.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:46:58 smithi166 bash[9446]: cluster 2020-09-08T15:46:57.740735+0000 mgr.x (mgr.24118) 589 : cluster [DBG] pgmap v815: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 27 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 861 op/s 2020-09-08T15:46:59.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:58 smithi055 bash[9296]: cluster 2020-09-08T15:46:57.545694+0000 osd.9 (osd.9) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:59.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:58 smithi055 bash[9296]: cluster 2020-09-08T15:46:57.545829+0000 osd.9 (osd.9) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:59.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:58 smithi055 bash[9296]: cluster 2020-09-08T15:46:57.727796+0000 mon.a (mon.0) 1312 : cluster [DBG] osdmap e310: 12 total, 12 up, 12 in 2020-09-08T15:46:59.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:46:58 smithi055 bash[9296]: cluster 2020-09-08T15:46:57.740735+0000 mgr.x (mgr.24118) 589 : cluster [DBG] pgmap v815: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 27 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 861 op/s 2020-09-08T15:46:59.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:58 smithi104 bash[9203]: cluster 2020-09-08T15:46:57.545694+0000 osd.9 (osd.9) 31 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:46:59.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:58 smithi104 bash[9203]: cluster 2020-09-08T15:46:57.545829+0000 osd.9 (osd.9) 32 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:46:59.183 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:58 smithi104 bash[9203]: cluster 2020-09-08T15:46:57.727796+0000 mon.a (mon.0) 1312 : cluster [DBG] osdmap e310: 12 total, 12 up, 12 in 2020-09-08T15:46:59.183 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:46:58 smithi104 bash[9203]: cluster 2020-09-08T15:46:57.740735+0000 mgr.x (mgr.24118) 589 : cluster [DBG] pgmap v815: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 27 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 861 op/s 2020-09-08T15:46:59.208 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:46:59.342 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:47:00.706 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'unique_pool_1' removed 2020-09-08T15:47:00.875 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:47:00.980 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:00.994 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:00 smithi055 bash[9296]: audit 2020-09-08T15:47:00.661540+0000 mon.a (mon.0) 1313 : audit [INF] from='client.? v1:172.21.15.104:0/2314420089' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T15:47:00.995 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:47:00 smithi055 bash[19003]: debug 2020-09-08T15:47:00.875+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:01.004 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:00 smithi104 bash[9203]: audit 2020-09-08T15:47:00.661540+0000 mon.a (mon.0) 1313 : audit [INF] from='client.? v1:172.21.15.104:0/2314420089' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T15:47:01.096 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:01.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:00 smithi166 bash[9446]: audit 2020-09-08T15:47:00.661540+0000 mon.a (mon.0) 1313 : audit [INF] from='client.? v1:172.21.15.104:0/2314420089' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T15:47:01.200 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:01.305 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:47:01 smithi055 bash[14301]: debug 2020-09-08T15:47:01.095+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:01.305 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:01.409 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:01.516 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:01.622 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:47:01.622 INFO:teuthology.orchestra.run.smithi166.stdout: "size": 20, 2020-09-08T15:47:01.623 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 600, 2020-09-08T15:47:01.623 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T15:47:01.623 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.624 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223524 2.c 2:3ad6268a:::benchmark_data_smithi166_20551_object75904:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:47:01.624 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.443068+0000", 2020-09-08T15:47:01.624 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 390.178278971, 2020-09-08T15:47:01.624 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.312334479, 2020-09-08T15:47:01.625 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:47:01.625 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:01.625 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:47:01.625 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:47:01.625 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:47:01.626 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223524 2020-09-08T15:47:01.626 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.626 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:47:01.626 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.627 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:47:01.627 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.443068+0000", 2020-09-08T15:47:01.627 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:47:01.627 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.628 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.628 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:47:01.628 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.443068+0000", 2020-09-08T15:47:01.628 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999752 2020-09-08T15:47:01.628 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.629 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.629 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:47:01.629 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.443043+0000", 2020-09-08T15:47:01.629 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.0220999999999998e-05 2020-09-08T15:47:01.641 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.641 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.641 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:47:01.641 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.443103+0000", 2020-09-08T15:47:01.642 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2289999999999992e-06 2020-09-08T15:47:01.642 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.642 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.643 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:47:01.643 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.443112+0000", 2020-09-08T15:47:01.643 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9359999999999998e-06 2020-09-08T15:47:01.644 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.644 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.644 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:47:01.644 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.443117+0000", 2020-09-08T15:47:01.644 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0022104389999999998 2020-09-08T15:47:01.645 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.645 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.645 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.645 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.445328+0000", 2020-09-08T15:47:01.645 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015247200000000001 2020-09-08T15:47:01.646 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.647 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.647 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:47:01.647 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.445480+0000", 2020-09-08T15:47:01.647 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016037700000000001 2020-09-08T15:47:01.648 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.648 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.648 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:47:01.648 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.445640+0000", 2020-09-08T15:47:01.650 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0047941390000000002 2020-09-08T15:47:01.650 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.651 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.651 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:47:01.651 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.450435+0000", 2020-09-08T15:47:01.651 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.30471783000000002 2020-09-08T15:47:01.651 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.652 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.652 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:47:01.652 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.755152+0000", 2020-09-08T15:47:01.653 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011526100000000001 2020-09-08T15:47:01.653 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.653 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.662 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:47:01.663 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.755268+0000", 2020-09-08T15:47:01.663 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000134411 2020-09-08T15:47:01.663 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.664 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.664 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:47:01.664 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.755402+0000", 2020-09-08T15:47:01.665 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.312334479 2020-09-08T15:47:01.665 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.665 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.665 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.666 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.666 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.666 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223535 2.c 2:3b35efb1:::benchmark_data_smithi166_20551_object75915:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:47:01.666 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.454088+0000", 2020-09-08T15:47:01.667 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 390.167258264, 2020-09-08T15:47:01.667 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.30184969299999997, 2020-09-08T15:47:01.667 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:47:01.667 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:01.668 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:47:01.668 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:47:01.668 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:47:01.668 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223535 2020-09-08T15:47:01.668 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.669 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:47:01.670 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.670 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:47:01.670 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454088+0000", 2020-09-08T15:47:01.670 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:47:01.671 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.671 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.671 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:47:01.671 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454088+0000", 2020-09-08T15:47:01.671 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999619 2020-09-08T15:47:01.672 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.672 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.672 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:47:01.672 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454050+0000", 2020-09-08T15:47:01.673 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7121999999999996e-05 2020-09-08T15:47:01.673 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.686 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.686 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:47:01.686 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454147+0000", 2020-09-08T15:47:01.687 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.0890000000000002e-05 2020-09-08T15:47:01.687 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.687 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.687 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:47:01.688 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454168+0000", 2020-09-08T15:47:01.688 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0044000000000001e-05 2020-09-08T15:47:01.688 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.688 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.689 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:47:01.689 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454178+0000", 2020-09-08T15:47:01.689 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.1668999999999995e-05 2020-09-08T15:47:01.689 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.691 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.692 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.693 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454240+0000", 2020-09-08T15:47:01.693 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00020094000000000001 2020-09-08T15:47:01.693 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.693 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.694 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:47:01.694 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454441+0000", 2020-09-08T15:47:01.694 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000217865 2020-09-08T15:47:01.695 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.695 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.695 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:47:01.695 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.454658+0000", 2020-09-08T15:47:01.695 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0053479290000000004 2020-09-08T15:47:01.696 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.696 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.696 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:47:01.696 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.460006+0000", 2020-09-08T15:47:01.697 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.29555245899999999 2020-09-08T15:47:01.697 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.697 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.697 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:47:01.698 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.755559+0000", 2020-09-08T15:47:01.698 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000214642 2020-09-08T15:47:01.698 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.698 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.699 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:47:01.699 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.755773+0000", 2020-09-08T15:47:01.699 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000164463 2020-09-08T15:47:01.699 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.700 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.700 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:47:01.700 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.755938+0000", 2020-09-08T15:47:01.701 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.30184969299999997 2020-09-08T15:47:01.701 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.702 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.702 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.702 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.702 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.702 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223545 2.c 2:3ce5f60b:::benchmark_data_smithi166_20551_object75925:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:47:01.704 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.463157+0000", 2020-09-08T15:47:01.705 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 390.15818999200002, 2020-09-08T15:47:01.705 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.31968120500000002, 2020-09-08T15:47:01.705 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:47:01.706 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:01.706 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:47:01.706 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:47:01.706 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:47:01.706 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223545 2020-09-08T15:47:01.707 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.712 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:47:01.712 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.712 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:47:01.713 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.463157+0000", 2020-09-08T15:47:01.713 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:47:01.713 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.713 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.714 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:47:01.714 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.463157+0000", 2020-09-08T15:47:01.714 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999375 2020-09-08T15:47:01.714 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.714 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.715 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:47:01.715 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.463094+0000", 2020-09-08T15:47:01.715 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013685800000000001 2020-09-08T15:47:01.715 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.715 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.716 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:47:01.716 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.463231+0000", 2020-09-08T15:47:01.716 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.6152999999999998e-05 2020-09-08T15:47:01.716 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.716 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.717 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:47:01.717 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.463257+0000", 2020-09-08T15:47:01.717 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1065e-05 2020-09-08T15:47:01.717 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.717 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.718 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:47:01.718 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.463268+0000", 2020-09-08T15:47:01.718 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.2441000000000002e-05 2020-09-08T15:47:01.718 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.718 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.718 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.719 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.463311+0000", 2020-09-08T15:47:01.719 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.9934999999999998e-05 2020-09-08T15:47:01.719 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.719 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.719 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:47:01.720 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.463411+0000", 2020-09-08T15:47:01.720 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.4406000000000004e-05 2020-09-08T15:47:01.720 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.720 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.720 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:47:01.721 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.463505+0000", 2020-09-08T15:47:01.721 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0050517319999999998 2020-09-08T15:47:01.721 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.721 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.721 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:47:01.722 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.468557+0000", 2020-09-08T15:47:01.722 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.31419107800000001 2020-09-08T15:47:01.722 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.722 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.722 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:47:01.722 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.782748+0000", 2020-09-08T15:47:01.723 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.2206999999999997e-05 2020-09-08T15:47:01.723 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.723 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.723 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:47:01.723 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.782800+0000", 2020-09-08T15:47:01.724 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.7558999999999997e-05 2020-09-08T15:47:01.724 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.724 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.724 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:47:01.724 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.782838+0000", 2020-09-08T15:47:01.725 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.31968120500000002 2020-09-08T15:47:01.725 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.725 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.725 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.725 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.726 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.726 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.34742.0:223556 2.c 2:3a873e3a:::benchmark_data_smithi166_20551_object75936:head [delete] snapc 0=[] ondisk+write+known_if_redirected e189)", 2020-09-08T15:47:01.726 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:40:31.473508+0000", 2020-09-08T15:47:01.726 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 390.14783865099997, 2020-09-08T15:47:01.726 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.30949394699999999, 2020-09-08T15:47:01.726 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:47:01.727 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:01.727 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:47:01.727 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.34742", 2020-09-08T15:47:01.727 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2945525739", 2020-09-08T15:47:01.727 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223556 2020-09-08T15:47:01.728 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.728 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:47:01.728 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.728 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:47:01.728 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.473508+0000", 2020-09-08T15:47:01.729 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:47:01.729 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.729 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.729 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:47:01.729 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.473508+0000", 2020-09-08T15:47:01.730 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999657 2020-09-08T15:47:01.730 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.730 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.730 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:47:01.730 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.473474+0000", 2020-09-08T15:47:01.730 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.2657000000000005e-05 2020-09-08T15:47:01.731 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.733 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.733 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:47:01.734 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.473556+0000", 2020-09-08T15:47:01.734 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.2762e-05 2020-09-08T15:47:01.734 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.734 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.735 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:47:01.735 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.473579+0000", 2020-09-08T15:47:01.735 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.2389999999999996e-06 2020-09-08T15:47:01.735 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.736 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.736 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:47:01.736 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.473586+0000", 2020-09-08T15:47:01.736 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.3835999999999998e-05 2020-09-08T15:47:01.737 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.737 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.737 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.737 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.473640+0000", 2020-09-08T15:47:01.738 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015941100000000001 2020-09-08T15:47:01.738 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.738 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.739 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:47:01.739 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.473799+0000", 2020-09-08T15:47:01.739 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000184205 2020-09-08T15:47:01.739 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.740 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.740 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:47:01.740 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.473984+0000", 2020-09-08T15:47:01.740 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0057273860000000001 2020-09-08T15:47:01.741 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.741 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.741 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:47:01.742 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.479711+0000", 2020-09-08T15:47:01.742 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.30318046300000001 2020-09-08T15:47:01.742 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.742 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.743 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:47:01.746 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.782891+0000", 2020-09-08T15:47:01.746 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6531000000000004e-05 2020-09-08T15:47:01.746 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.746 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.747 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:47:01.747 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.782958+0000", 2020-09-08T15:47:01.747 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.3819999999999997e-05 2020-09-08T15:47:01.747 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.747 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.748 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:47:01.748 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:40:31.783002+0000", 2020-09-08T15:47:01.748 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.30949394699999999 2020-09-08T15:47:01.748 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.748 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.749 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.749 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.749 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.749 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88121 3.c 3.4d8f2e4c (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:47:01.749 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.700024+0000", 2020-09-08T15:47:01.750 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 238.92132290800001, 2020-09-08T15:47:01.750 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.97689150199999997, 2020-09-08T15:47:01.750 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:47:01.750 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:47:01.750 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:47:01.751 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:47:01.751 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:01.751 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88121 2020-09-08T15:47:01.751 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.751 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:47:01.752 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.752 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:47:01.752 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.700024+0000", 2020-09-08T15:47:01.752 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:47:01.752 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.753 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.753 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:47:01.753 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.700024+0000", 2020-09-08T15:47:01.753 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999032 2020-09-08T15:47:01.753 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.754 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.754 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:47:01.754 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.699927+0000", 2020-09-08T15:47:01.754 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018412100000000001 2020-09-08T15:47:01.754 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.755 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.755 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:47:01.755 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.700111+0000", 2020-09-08T15:47:01.755 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4973999999999999e-05 2020-09-08T15:47:01.755 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.756 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.756 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:47:01.756 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.700126+0000", 2020-09-08T15:47:01.756 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.4910000000000002e-06 2020-09-08T15:47:01.756 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.757 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.757 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:47:01.757 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.700133+0000", 2020-09-08T15:47:01.757 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6443000000000001e-05 2020-09-08T15:47:01.757 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.758 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.758 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.758 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.700199+0000", 2020-09-08T15:47:01.758 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.6340000000000007e-06 2020-09-08T15:47:01.758 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.758 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.759 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:47:01.759 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.700208+0000", 2020-09-08T15:47:01.759 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.97667827799999996 2020-09-08T15:47:01.759 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.759 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.760 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.760 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.676886+0000", 2020-09-08T15:47:01.760 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.9227999999999999e-05 2020-09-08T15:47:01.760 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.760 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.761 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:47:01.761 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.676915+0000", 2020-09-08T15:47:01.761 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.97689150199999997 2020-09-08T15:47:01.761 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.761 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.762 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.762 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.762 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.762 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88124 3.c 3.62cfb55c (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:47:01.763 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.701342+0000", 2020-09-08T15:47:01.763 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 238.92000448900001, 2020-09-08T15:47:01.763 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.97560956499999996, 2020-09-08T15:47:01.763 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:47:01.763 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:47:01.764 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:47:01.764 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:47:01.764 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:01.764 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88124 2020-09-08T15:47:01.764 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.764 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:47:01.765 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.765 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:47:01.765 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.701342+0000", 2020-09-08T15:47:01.765 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:47:01.765 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.766 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.766 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:47:01.766 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.701342+0000", 2020-09-08T15:47:01.766 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999804 2020-09-08T15:47:01.766 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.767 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.767 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:47:01.767 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.701322+0000", 2020-09-08T15:47:01.767 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.8744000000000005e-05 2020-09-08T15:47:01.767 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.768 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.768 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:47:01.768 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.701391+0000", 2020-09-08T15:47:01.768 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.3339999999999996e-06 2020-09-08T15:47:01.768 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.768 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.769 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:47:01.769 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.701398+0000", 2020-09-08T15:47:01.769 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.0599999999999999e-06 2020-09-08T15:47:01.769 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.769 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.770 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:47:01.770 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.701401+0000", 2020-09-08T15:47:01.770 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.8257999999999998e-05 2020-09-08T15:47:01.770 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.770 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.771 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.771 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.701450+0000", 2020-09-08T15:47:01.771 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0329000000000001e-05 2020-09-08T15:47:01.771 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.771 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.772 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:47:01.772 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.701460+0000", 2020-09-08T15:47:01.772 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.97547700800000003 2020-09-08T15:47:01.772 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.772 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.773 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.773 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.676937+0000", 2020-09-08T15:47:01.773 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4528999999999999e-05 2020-09-08T15:47:01.773 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.773 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.773 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:47:01.774 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.676952+0000", 2020-09-08T15:47:01.774 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.97560956499999996 2020-09-08T15:47:01.774 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.774 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.775 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.775 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.775 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.775 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88161 3.c 3.f471261c (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:47:01.776 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.730886+0000", 2020-09-08T15:47:01.776 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 238.89046027500001, 2020-09-08T15:47:01.776 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.94609881299999998, 2020-09-08T15:47:01.776 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:47:01.776 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:47:01.777 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:47:01.777 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:47:01.777 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:01.777 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88161 2020-09-08T15:47:01.777 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.778 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:47:01.778 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.778 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:47:01.778 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.730886+0000", 2020-09-08T15:47:01.778 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:47:01.778 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.779 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.779 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:47:01.779 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.730886+0000", 2020-09-08T15:47:01.779 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999766 2020-09-08T15:47:01.779 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.780 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.780 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:47:01.780 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.730863+0000", 2020-09-08T15:47:01.780 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.6293999999999997e-05 2020-09-08T15:47:01.780 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.781 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.781 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:47:01.781 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.730959+0000", 2020-09-08T15:47:01.781 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.1086000000000001e-05 2020-09-08T15:47:01.781 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.782 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.782 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:47:01.782 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.730980+0000", 2020-09-08T15:47:01.782 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.0809999999999996e-06 2020-09-08T15:47:01.782 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.783 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.783 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:47:01.783 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.730987+0000", 2020-09-08T15:47:01.783 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6370000000000003e-05 2020-09-08T15:47:01.783 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.784 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.784 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.784 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.731054+0000", 2020-09-08T15:47:01.784 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.7700000000000007e-06 2020-09-08T15:47:01.784 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.784 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.785 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:47:01.785 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.731062+0000", 2020-09-08T15:47:01.785 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.94591566199999999 2020-09-08T15:47:01.785 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.785 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.786 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.786 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.676978+0000", 2020-09-08T15:47:01.786 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.9609999999999999e-06 2020-09-08T15:47:01.786 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.786 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.787 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:47:01.787 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.676985+0000", 2020-09-08T15:47:01.787 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.94609881299999998 2020-09-08T15:47:01.787 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.787 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.788 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.788 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.788 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.788 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88165 3.c 3.9f2f3bbc (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:47:01.789 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.732971+0000", 2020-09-08T15:47:01.789 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 238.888375454, 2020-09-08T15:47:01.789 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.94403996000000001, 2020-09-08T15:47:01.789 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:47:01.789 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:47:01.790 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:47:01.790 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:47:01.790 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:01.790 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88165 2020-09-08T15:47:01.790 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.791 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:47:01.791 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.791 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:47:01.791 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.732971+0000", 2020-09-08T15:47:01.791 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:47:01.791 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.792 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.792 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:47:01.792 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.732971+0000", 2020-09-08T15:47:01.792 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999523 2020-09-08T15:47:01.792 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.793 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.793 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:47:01.793 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.732923+0000", 2020-09-08T15:47:01.793 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015177800000000001 2020-09-08T15:47:01.793 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.794 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.794 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:47:01.794 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.733075+0000", 2020-09-08T15:47:01.794 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6350000000000001e-05 2020-09-08T15:47:01.794 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.794 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.795 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:47:01.795 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.733092+0000", 2020-09-08T15:47:01.795 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.1280000000000002e-06 2020-09-08T15:47:01.795 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.795 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.796 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:47:01.796 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.733099+0000", 2020-09-08T15:47:01.796 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5586000000000005e-05 2020-09-08T15:47:01.796 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.796 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.797 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.797 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.733164+0000", 2020-09-08T15:47:01.797 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.013e-05 2020-09-08T15:47:01.797 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.797 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.798 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:47:01.798 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.733174+0000", 2020-09-08T15:47:01.798 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.943830006 2020-09-08T15:47:01.798 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.798 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.799 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.799 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677004+0000", 2020-09-08T15:47:01.799 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.596e-06 2020-09-08T15:47:01.799 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.799 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.800 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:47:01.800 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677011+0000", 2020-09-08T15:47:01.800 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.94403996000000001 2020-09-08T15:47:01.800 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.800 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.801 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.801 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.801 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.801 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88180 3.c 3.a7f3fb9c (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:47:01.801 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.746178+0000", 2020-09-08T15:47:01.801 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 238.875168583, 2020-09-08T15:47:01.802 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.93086361399999995, 2020-09-08T15:47:01.802 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:47:01.802 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:47:01.802 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:47:01.802 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:47:01.803 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:01.803 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88180 2020-09-08T15:47:01.803 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.803 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:47:01.803 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.804 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:47:01.804 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.746178+0000", 2020-09-08T15:47:01.804 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:47:01.804 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.804 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.804 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:47:01.805 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.746178+0000", 2020-09-08T15:47:01.805 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999104 2020-09-08T15:47:01.805 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.805 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.805 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:47:01.806 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.746088+0000", 2020-09-08T15:47:01.806 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00020530400000000001 2020-09-08T15:47:01.806 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.806 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.806 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:47:01.807 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.746294+0000", 2020-09-08T15:47:01.807 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.1090999999999998e-05 2020-09-08T15:47:01.807 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.807 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.807 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:47:01.808 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.746315+0000", 2020-09-08T15:47:01.808 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.5599999999999994e-06 2020-09-08T15:47:01.808 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.808 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.808 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:47:01.808 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.746323+0000", 2020-09-08T15:47:01.809 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.5589000000000001e-05 2020-09-08T15:47:01.809 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.809 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.809 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.809 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.746359+0000", 2020-09-08T15:47:01.810 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.0600000000000008e-06 2020-09-08T15:47:01.810 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.810 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.810 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:47:01.810 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.746367+0000", 2020-09-08T15:47:01.811 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.93066255799999997 2020-09-08T15:47:01.811 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.811 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.811 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.811 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677030+0000", 2020-09-08T15:47:01.811 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.203e-05 2020-09-08T15:47:01.812 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.812 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.812 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:47:01.812 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677042+0000", 2020-09-08T15:47:01.812 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.93086361399999995 2020-09-08T15:47:01.813 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.813 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.813 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.813 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.813 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.814 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88185 3.c 3.a38a334c (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:47:01.814 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.747647+0000", 2020-09-08T15:47:01.814 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 238.873699528, 2020-09-08T15:47:01.814 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.92947780899999999, 2020-09-08T15:47:01.815 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:47:01.815 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:47:01.815 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:47:01.816 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:47:01.816 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:01.816 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88185 2020-09-08T15:47:01.816 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.816 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:47:01.817 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.817 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:47:01.817 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.747647+0000", 2020-09-08T15:47:01.817 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:47:01.817 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.818 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.818 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:47:01.819 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.747647+0000", 2020-09-08T15:47:01.819 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999719 2020-09-08T15:47:01.819 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.819 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.820 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:47:01.820 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.747619+0000", 2020-09-08T15:47:01.820 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010487199999999999 2020-09-08T15:47:01.820 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.820 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.821 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:47:01.821 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.747724+0000", 2020-09-08T15:47:01.821 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0954e-05 2020-09-08T15:47:01.821 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.821 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.822 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:47:01.822 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.747735+0000", 2020-09-08T15:47:01.822 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.2039999999999996e-06 2020-09-08T15:47:01.822 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.822 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.823 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:47:01.823 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.747740+0000", 2020-09-08T15:47:01.823 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.1072000000000002e-05 2020-09-08T15:47:01.823 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.823 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.824 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.824 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.747781+0000", 2020-09-08T15:47:01.824 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.2149999999999998e-06 2020-09-08T15:47:01.824 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.824 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.824 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:47:01.825 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.747787+0000", 2020-09-08T15:47:01.825 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.92931792700000004 2020-09-08T15:47:01.825 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.825 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.825 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.826 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677105+0000", 2020-09-08T15:47:01.826 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.9596999999999999e-05 2020-09-08T15:47:01.826 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.826 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.826 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:47:01.827 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677125+0000", 2020-09-08T15:47:01.827 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.92947780899999999 2020-09-08T15:47:01.827 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.827 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.827 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.827 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.828 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.828 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88206 3.c 3.396499ec (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:47:01.828 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.766105+0000", 2020-09-08T15:47:01.828 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 238.85524174400001, 2020-09-08T15:47:01.828 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.911052847, 2020-09-08T15:47:01.829 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:47:01.829 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:47:01.829 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:47:01.829 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:47:01.830 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:01.830 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88206 2020-09-08T15:47:01.830 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.830 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:47:01.831 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.831 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:47:01.831 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.766105+0000", 2020-09-08T15:47:01.831 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:47:01.831 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.832 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.832 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:47:01.832 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.766105+0000", 2020-09-08T15:47:01.832 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999504 2020-09-08T15:47:01.832 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.833 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.833 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:47:01.833 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.766055+0000", 2020-09-08T15:47:01.833 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000161633 2020-09-08T15:47:01.833 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.833 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.834 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:47:01.834 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.766217+0000", 2020-09-08T15:47:01.839 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.5154e-05 2020-09-08T15:47:01.839 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.840 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.840 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:47:01.840 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.766242+0000", 2020-09-08T15:47:01.840 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.9680000000000001e-06 2020-09-08T15:47:01.840 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.841 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.841 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:47:01.841 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.766252+0000", 2020-09-08T15:47:01.841 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.5038999999999997e-05 2020-09-08T15:47:01.841 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.842 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.842 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.842 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.766337+0000", 2020-09-08T15:47:01.842 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.1067999999999999e-05 2020-09-08T15:47:01.842 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.843 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.843 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:47:01.843 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.766358+0000", 2020-09-08T15:47:01.843 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.91078725599999999 2020-09-08T15:47:01.843 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.844 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.844 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.844 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677145+0000", 2020-09-08T15:47:01.844 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2483999999999999e-05 2020-09-08T15:47:01.844 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.844 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.845 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:47:01.845 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677158+0000", 2020-09-08T15:47:01.845 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.911052847 2020-09-08T15:47:01.845 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.845 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.846 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.846 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.846 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.846 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88213 3.c 3.1b7c82bc (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:47:01.846 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.770524+0000", 2020-09-08T15:47:01.847 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 238.850822247, 2020-09-08T15:47:01.847 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.90669372299999995, 2020-09-08T15:47:01.847 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:47:01.847 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:47:01.847 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:47:01.848 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:47:01.848 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:01.848 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88213 2020-09-08T15:47:01.848 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.848 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:47:01.849 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.849 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:47:01.849 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.770524+0000", 2020-09-08T15:47:01.849 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:47:01.849 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.850 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.850 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:47:01.850 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.770524+0000", 2020-09-08T15:47:01.850 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999757 2020-09-08T15:47:01.850 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.851 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.851 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:47:01.851 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.770500+0000", 2020-09-08T15:47:01.851 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.1895000000000001e-05 2020-09-08T15:47:01.851 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.851 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.852 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:47:01.852 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.770582+0000", 2020-09-08T15:47:01.852 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.9410000000000002e-06 2020-09-08T15:47:01.852 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.852 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.853 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:47:01.853 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.770592+0000", 2020-09-08T15:47:01.853 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.3240000000000002e-06 2020-09-08T15:47:01.853 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.853 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.854 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:47:01.854 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.770597+0000", 2020-09-08T15:47:01.854 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.5640000000000003e-05 2020-09-08T15:47:01.854 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.854 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.855 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.855 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.770653+0000", 2020-09-08T15:47:01.855 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.8029999999999999e-06 2020-09-08T15:47:01.855 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.855 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.856 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:47:01.856 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.770659+0000", 2020-09-08T15:47:01.856 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.90654226699999996 2020-09-08T15:47:01.856 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.856 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.856 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.857 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677202+0000", 2020-09-08T15:47:01.857 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6291e-05 2020-09-08T15:47:01.857 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.857 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.857 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:47:01.858 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677218+0000", 2020-09-08T15:47:01.858 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.90669372299999995 2020-09-08T15:47:01.858 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.858 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.858 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.859 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.859 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.859 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88217 3.c 3.c99c990c (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:47:01.859 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.773207+0000", 2020-09-08T15:47:01.859 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 238.84813944999999, 2020-09-08T15:47:01.860 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.90404103099999999, 2020-09-08T15:47:01.860 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:47:01.860 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:47:01.860 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:47:01.860 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:47:01.860 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:01.861 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88217 2020-09-08T15:47:01.861 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.861 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:47:01.861 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.861 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:47:01.862 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.773207+0000", 2020-09-08T15:47:01.862 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:47:01.862 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.862 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.862 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:47:01.863 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.773207+0000", 2020-09-08T15:47:01.863 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999504 2020-09-08T15:47:01.863 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.863 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.863 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:47:01.863 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.773157+0000", 2020-09-08T15:47:01.864 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015484 2020-09-08T15:47:01.864 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.864 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.864 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:47:01.864 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.773312+0000", 2020-09-08T15:47:01.865 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5645e-05 2020-09-08T15:47:01.865 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.865 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.865 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:47:01.865 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.773328+0000", 2020-09-08T15:47:01.866 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.3879999999999997e-06 2020-09-08T15:47:01.866 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.866 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.866 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:47:01.866 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.773334+0000", 2020-09-08T15:47:01.867 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0011813629999999999 2020-09-08T15:47:01.867 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.867 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.867 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.867 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.774515+0000", 2020-09-08T15:47:01.868 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1753000000000001e-05 2020-09-08T15:47:01.868 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.868 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.868 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:47:01.868 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.774527+0000", 2020-09-08T15:47:01.869 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.90270882500000005 2020-09-08T15:47:01.869 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.869 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.869 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.869 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677236+0000", 2020-09-08T15:47:01.869 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2037e-05 2020-09-08T15:47:01.870 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.870 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.870 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:47:01.870 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677248+0000", 2020-09-08T15:47:01.870 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.90404103099999999 2020-09-08T15:47:01.871 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.871 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.871 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.871 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.871 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.872 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88229 3.c 3.1e4c95fc (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:47:01.872 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.783938+0000", 2020-09-08T15:47:01.872 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 238.83740857800001, 2020-09-08T15:47:01.872 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.89337602900000002, 2020-09-08T15:47:01.872 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:47:01.873 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:47:01.873 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:47:01.873 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:47:01.873 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:01.873 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88229 2020-09-08T15:47:01.874 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.874 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:47:01.874 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.874 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:47:01.874 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.783938+0000", 2020-09-08T15:47:01.875 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:47:01.875 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.875 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.875 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:47:01.875 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.783938+0000", 2020-09-08T15:47:01.876 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999452 2020-09-08T15:47:01.876 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.876 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.877 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:47:01.877 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.783883+0000", 2020-09-08T15:47:01.877 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016782400000000001 2020-09-08T15:47:01.877 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.877 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.877 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:47:01.878 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.784051+0000", 2020-09-08T15:47:01.878 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7544e-05 2020-09-08T15:47:01.878 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.878 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.878 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:47:01.879 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.784068+0000", 2020-09-08T15:47:01.879 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.3669999999999999e-06 2020-09-08T15:47:01.879 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.879 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.879 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:47:01.880 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.784076+0000", 2020-09-08T15:47:01.880 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9112e-05 2020-09-08T15:47:01.880 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.880 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.880 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.881 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.784125+0000", 2020-09-08T15:47:01.881 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.611e-06 2020-09-08T15:47:01.881 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.881 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.881 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:47:01.882 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.784134+0000", 2020-09-08T15:47:01.882 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.89315457099999995 2020-09-08T15:47:01.882 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.882 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.882 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.882 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677289+0000", 2020-09-08T15:47:01.883 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.4997000000000001e-05 2020-09-08T15:47:01.884 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.884 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.884 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:47:01.884 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677314+0000", 2020-09-08T15:47:01.885 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.89337602900000002 2020-09-08T15:47:01.885 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.885 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.885 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.885 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.886 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.887 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88244 3.c 3.ae32ae4c (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:47:01.888 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.801246+0000", 2020-09-08T15:47:01.888 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 238.82010057400001, 2020-09-08T15:47:01.889 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.87618958400000002, 2020-09-08T15:47:01.889 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:47:01.889 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:47:01.889 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:47:01.889 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:47:01.890 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:01.890 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88244 2020-09-08T15:47:01.892 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.892 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:47:01.892 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.893 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:47:01.893 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.801246+0000", 2020-09-08T15:47:01.893 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:47:01.893 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.894 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.894 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:47:01.894 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.801246+0000", 2020-09-08T15:47:01.894 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9998989 2020-09-08T15:47:01.894 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.894 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.895 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:47:01.895 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.801145+0000", 2020-09-08T15:47:01.895 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000188982 2020-09-08T15:47:01.895 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.895 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.896 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:47:01.896 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.801334+0000", 2020-09-08T15:47:01.896 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7173999999999999e-05 2020-09-08T15:47:01.896 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.896 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.897 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:47:01.897 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.801351+0000", 2020-09-08T15:47:01.897 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2329999999999995e-06 2020-09-08T15:47:01.897 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.897 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.898 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:47:01.898 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.801360+0000", 2020-09-08T15:47:01.898 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.1766000000000004e-05 2020-09-08T15:47:01.898 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.898 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.899 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.899 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.801432+0000", 2020-09-08T15:47:01.899 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5519000000000002e-05 2020-09-08T15:47:01.899 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.899 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.900 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:47:01.900 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.801447+0000", 2020-09-08T15:47:01.900 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.87588484700000002 2020-09-08T15:47:01.900 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.900 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.901 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.901 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677332+0000", 2020-09-08T15:47:01.901 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000103272 2020-09-08T15:47:01.901 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.901 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.902 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:47:01.902 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677436+0000", 2020-09-08T15:47:01.902 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.87618958400000002 2020-09-08T15:47:01.902 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.902 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.902 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.903 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.903 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.903 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88252 3.c 3.7108c53c (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:47:01.903 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.811533+0000", 2020-09-08T15:47:01.903 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 238.809813142, 2020-09-08T15:47:01.904 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.86589799099999998, 2020-09-08T15:47:01.904 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:47:01.904 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:47:01.904 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:47:01.904 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:47:01.905 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:01.905 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88252 2020-09-08T15:47:01.905 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.905 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:47:01.905 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.906 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:47:01.906 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.811533+0000", 2020-09-08T15:47:01.906 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:47:01.906 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.906 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.907 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:47:01.907 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.811533+0000", 2020-09-08T15:47:01.907 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999599 2020-09-08T15:47:01.907 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.908 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.908 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:47:01.908 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.811493+0000", 2020-09-08T15:47:01.909 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013593500000000001 2020-09-08T15:47:01.909 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.909 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.909 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:47:01.909 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.811629+0000", 2020-09-08T15:47:01.910 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7910999999999998e-05 2020-09-08T15:47:01.910 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.910 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.910 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:47:01.910 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.811647+0000", 2020-09-08T15:47:01.911 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5679999999999996e-06 2020-09-08T15:47:01.911 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.911 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.911 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:47:01.911 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.811654+0000", 2020-09-08T15:47:01.912 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6841999999999995e-05 2020-09-08T15:47:01.912 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.912 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.912 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.912 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.811720+0000", 2020-09-08T15:47:01.912 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.037e-06 2020-09-08T15:47:01.913 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.913 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.913 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:47:01.913 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.811729+0000", 2020-09-08T15:47:01.913 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.86563225099999996 2020-09-08T15:47:01.914 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.914 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.914 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.914 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677362+0000", 2020-09-08T15:47:01.914 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.9690000000000005e-05 2020-09-08T15:47:01.915 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.915 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.915 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:47:01.915 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677431+0000", 2020-09-08T15:47:01.915 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.86589799099999998 2020-09-08T15:47:01.916 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.916 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.916 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.916 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.916 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.917 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88264 3.c 3.ab9c6e5c (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:47:01.917 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.826418+0000", 2020-09-08T15:47:01.917 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 238.79492825299999, 2020-09-08T15:47:01.917 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.85108039700000004, 2020-09-08T15:47:01.917 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:47:01.918 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:47:01.918 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:47:01.918 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:47:01.918 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:01.918 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88264 2020-09-08T15:47:01.918 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.919 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:47:01.919 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.919 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:47:01.919 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.826418+0000", 2020-09-08T15:47:01.919 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:47:01.920 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.920 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.920 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:47:01.920 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.826418+0000", 2020-09-08T15:47:01.920 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999714 2020-09-08T15:47:01.921 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.921 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.921 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:47:01.921 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.826390+0000", 2020-09-08T15:47:01.921 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.0500000000000004e-05 2020-09-08T15:47:01.922 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.922 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.922 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:47:01.922 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.826480+0000", 2020-09-08T15:47:01.923 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.5810000000000003e-06 2020-09-08T15:47:01.923 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.924 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.924 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:47:01.924 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.826490+0000", 2020-09-08T15:47:01.925 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.7269999999999998e-06 2020-09-08T15:47:01.925 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.925 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.926 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:47:01.926 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.826495+0000", 2020-09-08T15:47:01.926 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.4483e-05 2020-09-08T15:47:01.926 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.927 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.927 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.927 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.826549+0000", 2020-09-08T15:47:01.927 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.3500000000000003e-06 2020-09-08T15:47:01.927 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.928 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.928 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:47:01.928 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.826558+0000", 2020-09-08T15:47:01.930 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.85092367300000005 2020-09-08T15:47:01.930 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.930 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.930 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.930 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677482+0000", 2020-09-08T15:47:01.931 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6571000000000001e-05 2020-09-08T15:47:01.931 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.931 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.931 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:47:01.931 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677499+0000", 2020-09-08T15:47:01.932 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.85108039700000004 2020-09-08T15:47:01.932 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.932 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.932 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.932 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.933 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.933 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88277 3.c 3.88233cdc (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:47:01.933 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.850783+0000", 2020-09-08T15:47:01.933 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 238.77056373900001, 2020-09-08T15:47:01.933 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.82674549799999997, 2020-09-08T15:47:01.934 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:47:01.934 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:47:01.934 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:47:01.934 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:47:01.934 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:01.935 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88277 2020-09-08T15:47:01.935 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.935 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:47:01.935 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.935 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:47:01.936 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.850783+0000", 2020-09-08T15:47:01.936 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:47:01.936 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.936 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.936 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:47:01.936 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.850783+0000", 2020-09-08T15:47:01.937 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999638 2020-09-08T15:47:01.937 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.937 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.937 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:47:01.937 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.850747+0000", 2020-09-08T15:47:01.938 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012006399999999999 2020-09-08T15:47:01.938 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.938 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.938 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:47:01.938 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.850867+0000", 2020-09-08T15:47:01.939 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7493e-05 2020-09-08T15:47:01.939 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.939 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.939 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:47:01.939 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.850884+0000", 2020-09-08T15:47:01.940 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7729999999999994e-06 2020-09-08T15:47:01.940 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.940 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.940 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:47:01.940 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.850894+0000", 2020-09-08T15:47:01.941 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.1898999999999997e-05 2020-09-08T15:47:01.941 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.941 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.941 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.941 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.850966+0000", 2020-09-08T15:47:01.942 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5427e-05 2020-09-08T15:47:01.942 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.942 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.942 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:47:01.942 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.850981+0000", 2020-09-08T15:47:01.943 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.82653671299999998 2020-09-08T15:47:01.943 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.943 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.943 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.943 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677518+0000", 2020-09-08T15:47:01.944 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0154e-05 2020-09-08T15:47:01.944 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.944 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.944 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:47:01.944 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677528+0000", 2020-09-08T15:47:01.944 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.82674549799999997 2020-09-08T15:47:01.945 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.945 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.945 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.945 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.945 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.946 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88294 3.c 3.b1cb3efc (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:47:01.946 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.909023+0000", 2020-09-08T15:47:01.946 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 238.71232366199999, 2020-09-08T15:47:01.946 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.76856889299999998, 2020-09-08T15:47:01.946 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:47:01.947 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:47:01.947 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:47:01.947 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:47:01.947 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:01.947 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88294 2020-09-08T15:47:01.948 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.948 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:47:01.948 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.948 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:47:01.948 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.909023+0000", 2020-09-08T15:47:01.948 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:47:01.949 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.949 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.949 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:47:01.949 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.909023+0000", 2020-09-08T15:47:01.949 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999738 2020-09-08T15:47:01.950 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.950 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.950 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:47:01.950 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.908997+0000", 2020-09-08T15:47:01.950 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2019000000000001e-05 2020-09-08T15:47:01.951 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.951 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.951 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:47:01.951 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.909089+0000", 2020-09-08T15:47:01.951 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4188e-05 2020-09-08T15:47:01.952 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.952 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.952 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:47:01.952 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.909103+0000", 2020-09-08T15:47:01.952 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.6540000000000004e-06 2020-09-08T15:47:01.952 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.953 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.953 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:47:01.953 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.909109+0000", 2020-09-08T15:47:01.953 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7284000000000002e-05 2020-09-08T15:47:01.953 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.954 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.954 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.954 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.909166+0000", 2020-09-08T15:47:01.954 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8189999999999998e-06 2020-09-08T15:47:01.954 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.955 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.955 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:47:01.955 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.909175+0000", 2020-09-08T15:47:01.955 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.76839657900000002 2020-09-08T15:47:01.955 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.956 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.956 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.956 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677571+0000", 2020-09-08T15:47:01.956 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.0534e-05 2020-09-08T15:47:01.956 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.956 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.957 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:47:01.957 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677592+0000", 2020-09-08T15:47:01.957 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.76856889299999998 2020-09-08T15:47:01.957 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.957 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.958 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.958 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.958 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.958 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88298 3.c 3.86504cfc (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:47:01.958 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.934302+0000", 2020-09-08T15:47:01.959 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 238.68704429100001, 2020-09-08T15:47:01.959 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.74331687800000001, 2020-09-08T15:47:01.959 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:47:01.959 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:47:01.959 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:47:01.960 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:47:01.960 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:01.960 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88298 2020-09-08T15:47:01.960 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.960 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:47:01.960 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.961 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:47:01.961 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.934302+0000", 2020-09-08T15:47:01.961 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:47:01.961 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.961 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.962 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:47:01.962 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.934302+0000", 2020-09-08T15:47:01.962 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999776 2020-09-08T15:47:01.962 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.962 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.963 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:47:01.963 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.934280+0000", 2020-09-08T15:47:01.963 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.1067000000000002e-05 2020-09-08T15:47:01.963 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.963 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.963 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:47:01.964 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.934361+0000", 2020-09-08T15:47:01.964 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.031e-05 2020-09-08T15:47:01.964 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.964 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.964 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:47:01.965 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.934371+0000", 2020-09-08T15:47:01.965 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.7739999999999999e-06 2020-09-08T15:47:01.965 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.965 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.965 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:47:01.966 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.934378+0000", 2020-09-08T15:47:01.966 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.2311000000000005e-05 2020-09-08T15:47:01.966 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.966 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.966 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.966 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.934460+0000", 2020-09-08T15:47:01.967 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1685000000000001e-05 2020-09-08T15:47:01.967 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.967 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.967 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:47:01.967 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.934472+0000", 2020-09-08T15:47:01.968 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.743139683 2020-09-08T15:47:01.968 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.968 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.968 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:47:01.968 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677611+0000", 2020-09-08T15:47:01.969 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.6290000000000001e-06 2020-09-08T15:47:01.969 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:47:01.969 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:47:01.969 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:47:01.969 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677619+0000", 2020-09-08T15:47:01.970 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.74331687800000001 2020-09-08T15:47:01.970 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.970 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.970 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.970 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:47:01.970 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:47:01.971 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:47:01.971 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:01.973 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:01 smithi104 bash[9203]: cluster 2020-09-08T15:46:59.741394+0000 mgr.x (mgr.24118) 590 : cluster [DBG] pgmap v816: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 639 op/s 2020-09-08T15:47:01.973 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:01 smithi104 bash[9203]: cluster 2020-09-08T15:47:00.387971+0000 osd.2 (osd.2) 42 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:01.973 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:01 smithi104 bash[9203]: cluster 2020-09-08T15:47:00.388099+0000 osd.2 (osd.2) 43 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:01.974 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:01 smithi104 bash[9203]: audit 2020-09-08T15:47:00.707060+0000 mon.a (mon.0) 1314 : audit [INF] from='client.? v1:172.21.15.104:0/2314420089' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T15:47:01.974 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:01 smithi104 bash[9203]: cluster 2020-09-08T15:47:00.707174+0000 mon.a (mon.0) 1315 : cluster [DBG] osdmap e311: 12 total, 12 up, 12 in 2020-09-08T15:47:01.974 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:47:01 smithi055 bash[16646]: debug 2020-09-08T15:47:01.307+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:01.975 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:47:01 smithi166 bash[11928]: debug 2020-09-08T15:47:01.512+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:01.975 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:01 smithi166 bash[9446]: cluster 2020-09-08T15:46:59.741394+0000 mgr.x (mgr.24118) 590 : cluster [DBG] pgmap v816: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 639 op/s 2020-09-08T15:47:01.975 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:01 smithi166 bash[9446]: cluster 2020-09-08T15:47:00.387971+0000 osd.2 (osd.2) 42 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:01.975 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:01 smithi166 bash[9446]: cluster 2020-09-08T15:47:00.388099+0000 osd.2 (osd.2) 43 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:01.976 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:01 smithi166 bash[9446]: audit 2020-09-08T15:47:00.707060+0000 mon.a (mon.0) 1314 : audit [INF] from='client.? v1:172.21.15.104:0/2314420089' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T15:47:01.986 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:01 smithi055 bash[9296]: cluster 2020-09-08T15:46:59.741394+0000 mgr.x (mgr.24118) 590 : cluster [DBG] pgmap v816: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 639 op/s 2020-09-08T15:47:01.986 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:01 smithi055 bash[9296]: cluster 2020-09-08T15:47:00.387971+0000 osd.2 (osd.2) 42 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:01.987 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:01 smithi055 bash[9296]: cluster 2020-09-08T15:47:00.388099+0000 osd.2 (osd.2) 43 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:01.987 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:01 smithi055 bash[9296]: audit 2020-09-08T15:47:00.707060+0000 mon.a (mon.0) 1314 : audit [INF] from='client.? v1:172.21.15.104:0/2314420089' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T15:47:01.987 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:01 smithi055 bash[9296]: cluster 2020-09-08T15:47:00.707174+0000 mon.a (mon.0) 1315 : cluster [DBG] osdmap e311: 12 total, 12 up, 12 in 2020-09-08T15:47:02.066 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2020-09-08T15:47:02.068 INFO:tasks.radosbench:Beginning radosbench... 2020-09-08T15:47:02.068 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_2 2020-09-08T15:47:02.069 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool create unique_pool_2 16 2020-09-08T15:47:02.086 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:02.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:01 smithi166 bash[9446]: cluster 2020-09-08T15:47:00.707174+0000 mon.a (mon.0) 1315 : cluster [DBG] osdmap e311: 12 total, 12 up, 12 in 2020-09-08T15:47:02.192 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:02.198 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:47:02.299 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:47:02 smithi055 bash[14301]: debug 2020-09-08T15:47:02.087+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:02.300 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:02.379 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2020-09-08T15:47:02.405 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:02.517 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:47:02.526 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:02.631 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:02.642 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:47:02 smithi166 bash[18960]: debug 2020-09-08T15:47:02.524+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:02.658 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:47:02 smithi055 bash[16646]: debug 2020-09-08T15:47:02.299+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:02.759 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:02.865 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:02.992 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:47:02 smithi166 bash[14254]: debug 2020-09-08T15:47:02.756+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:02.993 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:03.099 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:47:03.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:47:02 smithi166 bash[11928]: debug 2020-09-08T15:47:02.992+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:03.527 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:47:03.631 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:03.744 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:03.850 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:47:03.867 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:03 smithi104 bash[9203]: cluster 2020-09-08T15:47:01.741998+0000 mgr.x (mgr.24118) 591 : cluster [DBG] pgmap v818: 1 pgs: 1 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:03.868 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:47:03 smithi104 bash[17588]: debug 2020-09-08T15:47:03.529+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:04.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:03 smithi166 bash[9446]: cluster 2020-09-08T15:47:01.741998+0000 mgr.x (mgr.24118) 591 : cluster [DBG] pgmap v818: 1 pgs: 1 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:04.123 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:47:03 smithi166 bash[14254]: debug 2020-09-08T15:47:03.744+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:04.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:03 smithi055 bash[9296]: cluster 2020-09-08T15:47:01.741998+0000 mgr.x (mgr.24118) 591 : cluster [DBG] pgmap v818: 1 pgs: 1 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:04.427 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:47:04.533 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:04.652 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:04.714 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:47:04 smithi104 bash[19922]: debug 2020-09-08T15:47:04.429+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:04.758 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:04.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:47:04 smithi166 bash[18960]: debug 2020-09-08T15:47:04.652+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:04.871 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:04.977 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:47:05.098 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:47:05.204 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:05.218 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:47:04 smithi166 bash[14254]: debug 2020-09-08T15:47:04.868+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:05.322 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:05.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:47:05 smithi055 bash[19003]: debug 2020-09-08T15:47:05.099+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:05.426 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:05.580 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:47:05 smithi166 bash[14254]: debug 2020-09-08T15:47:05.320+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:05.581 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:05.594 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:47:05.618 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:47:05.643 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:47:05.666 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:47:05.687 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:05.692 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:47:05.731 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:47:05.746 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:47:05.778 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:47:05.799 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:05.814 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:47:05.842 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:47:05.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:47:05 smithi166 bash[11928]: debug 2020-09-08T15:47:05.580+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:05.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:05 smithi166 bash[9446]: cluster 2020-09-08T15:47:03.742655+0000 mgr.x (mgr.24118) 592 : cluster [DBG] pgmap v819: 1 pgs: 1 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:05.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:05 smithi166 bash[9446]: cluster 2020-09-08T15:47:04.697782+0000 mon.a (mon.0) 1316 : cluster [DBG] osdmap e312: 12 total, 12 up, 12 in 2020-09-08T15:47:05.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:05 smithi166 bash[9446]: audit 2020-09-08T15:47:05.573734+0000 mon.a (mon.0) 1317 : audit [DBG] from='client.? v1:172.21.15.104:0/119432260' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:47:05.872 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:47:05.901 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:47:05.903 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:47:05.904 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:47:05.904 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:47:05.904 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:47:05.907 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:47:05.908 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:47:05.909 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:47:05.909 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:47:05.909 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:47:05.909 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:47:05.909 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:47:05.910 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:47:05.911 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:05.966 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:05 smithi104 bash[9203]: cluster 2020-09-08T15:47:03.742655+0000 mgr.x (mgr.24118) 592 : cluster [DBG] pgmap v819: 1 pgs: 1 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:05.967 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:05 smithi104 bash[9203]: cluster 2020-09-08T15:47:04.697782+0000 mon.a (mon.0) 1316 : cluster [DBG] osdmap e312: 12 total, 12 up, 12 in 2020-09-08T15:47:05.967 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:05 smithi104 bash[9203]: audit 2020-09-08T15:47:05.573734+0000 mon.a (mon.0) 1317 : audit [DBG] from='client.? v1:172.21.15.104:0/119432260' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:47:06.038 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:47:05 smithi055 bash[11956]: debug 2020-09-08T15:47:05.799+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:06.038 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:05 smithi055 bash[9296]: cluster 2020-09-08T15:47:03.742655+0000 mgr.x (mgr.24118) 592 : cluster [DBG] pgmap v819: 1 pgs: 1 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:06.039 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:05 smithi055 bash[9296]: cluster 2020-09-08T15:47:04.697782+0000 mon.a (mon.0) 1316 : cluster [DBG] osdmap e312: 12 total, 12 up, 12 in 2020-09-08T15:47:06.039 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:05 smithi055 bash[9296]: audit 2020-09-08T15:47:05.573734+0000 mon.a (mon.0) 1317 : audit [DBG] from='client.? v1:172.21.15.104:0/119432260' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:47:06.039 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:06.145 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:06.316 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:06.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:47:06 smithi055 bash[11956]: debug 2020-09-08T15:47:06.039+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:06.421 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:06.435 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:47:06 smithi166 bash[16593]: debug 2020-09-08T15:47:06.312+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:06.550 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:06.655 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:06.783 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:47:06 smithi166 bash[14254]: debug 2020-09-08T15:47:06.548+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:06.783 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:06.889 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:47:07.032 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:47:07.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:47:06 smithi166 bash[16593]: debug 2020-09-08T15:47:06.780+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:07.138 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:07.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:47:07 smithi104 bash[22305]: debug 2020-09-08T15:47:07.033+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:07.256 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:07.362 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:07.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:47:07 smithi055 bash[14301]: debug 2020-09-08T15:47:07.255+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:07.486 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:07.592 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:07.702 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:07.710 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:47:07.711 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:07 smithi104 bash[9203]: cluster 2020-09-08T15:47:05.743369+0000 mgr.x (mgr.24118) 593 : cluster [DBG] pgmap v821: 1 pgs: 1 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:07.711 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:07 smithi104 bash[9203]: audit 2020-09-08T15:47:06.781562+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? v1:172.21.15.104:0/165741603' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:47:07.812 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:07.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:47:07 smithi055 bash[14301]: debug 2020-09-08T15:47:07.703+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:07.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:07 smithi055 bash[9296]: cluster 2020-09-08T15:47:05.743369+0000 mgr.x (mgr.24118) 593 : cluster [DBG] pgmap v821: 1 pgs: 1 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:07.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:07 smithi055 bash[9296]: audit 2020-09-08T15:47:06.781562+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? v1:172.21.15.104:0/165741603' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:47:07.908 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:47:07 smithi055 bash[11956]: debug 2020-09-08T15:47:07.487+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:07.965 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:07 smithi166 bash[9446]: cluster 2020-09-08T15:47:05.743369+0000 mgr.x (mgr.24118) 593 : cluster [DBG] pgmap v821: 1 pgs: 1 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:07.965 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:07 smithi166 bash[9446]: audit 2020-09-08T15:47:06.781562+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? v1:172.21.15.104:0/165741603' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:47:07.966 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:08.070 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:08.192 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:08.297 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:08.309 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:47:07 smithi166 bash[11928]: debug 2020-09-08T15:47:07.960+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:08.310 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:47:08 smithi166 bash[14254]: debug 2020-09-08T15:47:08.192+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:08.428 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:08.536 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:08.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:47:08 smithi166 bash[16593]: debug 2020-09-08T15:47:08.428+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:08.662 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:08.768 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:47:08.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:47:08 smithi055 bash[14301]: debug 2020-09-08T15:47:08.663+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:08.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:08 smithi055 bash[9296]: cluster 2020-09-08T15:47:07.699868+0000 mon.a (mon.0) 1319 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:47:08.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:08 smithi055 bash[9296]: audit 2020-09-08T15:47:07.707133+0000 mon.a (mon.0) 1320 : audit [INF] from='client.? v1:172.21.15.104:0/165741603' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:47:08.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:08 smithi055 bash[9296]: cluster 2020-09-08T15:47:07.707208+0000 mon.a (mon.0) 1321 : cluster [DBG] osdmap e313: 12 total, 12 up, 12 in 2020-09-08T15:47:08.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:08 smithi055 bash[9296]: cluster 2020-09-08T15:47:07.744110+0000 mgr.x (mgr.24118) 594 : cluster [DBG] pgmap v823: 1 pgs: 1 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:08.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:08 smithi104 bash[9203]: cluster 2020-09-08T15:47:07.699868+0000 mon.a (mon.0) 1319 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:47:08.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:08 smithi104 bash[9203]: audit 2020-09-08T15:47:07.707133+0000 mon.a (mon.0) 1320 : audit [INF] from='client.? v1:172.21.15.104:0/165741603' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:47:08.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:08 smithi104 bash[9203]: cluster 2020-09-08T15:47:07.707208+0000 mon.a (mon.0) 1321 : cluster [DBG] osdmap e313: 12 total, 12 up, 12 in 2020-09-08T15:47:08.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:08 smithi104 bash[9203]: cluster 2020-09-08T15:47:07.744110+0000 mgr.x (mgr.24118) 594 : cluster [DBG] pgmap v823: 1 pgs: 1 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:08.976 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:47:09.081 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:47:09.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:08 smithi166 bash[9446]: cluster 2020-09-08T15:47:07.699868+0000 mon.a (mon.0) 1319 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:47:09.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:08 smithi166 bash[9446]: audit 2020-09-08T15:47:07.707133+0000 mon.a (mon.0) 1320 : audit [INF] from='client.? v1:172.21.15.104:0/165741603' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:47:09.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:08 smithi166 bash[9446]: cluster 2020-09-08T15:47:07.707208+0000 mon.a (mon.0) 1321 : cluster [DBG] osdmap e313: 12 total, 12 up, 12 in 2020-09-08T15:47:09.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:08 smithi166 bash[9446]: cluster 2020-09-08T15:47:07.744110+0000 mgr.x (mgr.24118) 594 : cluster [DBG] pgmap v823: 1 pgs: 1 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:09.165 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:47:09.166 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:47:09.202 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:47:08 smithi104 bash[17588]: debug 2020-09-08T15:47:08.977+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:09.202 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:47:09.304 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:09.444 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:47:09.454 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:09.458 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:47:09 smithi104 bash[22305]: debug 2020-09-08T15:47:09.197+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:09.560 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:09.575 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:47:09.577 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:47:09 smithi166 bash[16593]: debug 2020-09-08T15:47:09.452+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:09.690 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:09.701 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'unique_pool_2' created 2020-09-08T15:47:09.795 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:09.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:47:09 smithi166 bash[14254]: debug 2020-09-08T15:47:09.684+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:09.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:09 smithi166 bash[9446]: cluster 2020-09-08T15:47:08.825797+0000 osd.7 (osd.7) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:09.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:09 smithi166 bash[9446]: cluster 2020-09-08T15:47:08.825925+0000 osd.7 (osd.7) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:09.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:09 smithi166 bash[9446]: audit 2020-09-08T15:47:09.131700+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? v1:172.21.15.104:0/3244986967' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2020-09-08T15:47:09.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:09 smithi166 bash[9446]: audit 2020-09-08T15:47:09.700736+0000 mon.a (mon.0) 1323 : audit [INF] from='client.? v1:172.21.15.104:0/3244986967' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2020-09-08T15:47:09.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:09 smithi166 bash[9446]: cluster 2020-09-08T15:47:09.700785+0000 mon.a (mon.0) 1324 : cluster [DBG] osdmap e314: 12 total, 12 up, 12 in 2020-09-08T15:47:09.909 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:10.015 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:10.027 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:47:09 smithi055 bash[11956]: debug 2020-09-08T15:47:09.911+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:10.028 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:09 smithi055 bash[9296]: cluster 2020-09-08T15:47:08.825797+0000 osd.7 (osd.7) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:10.028 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:09 smithi055 bash[9296]: cluster 2020-09-08T15:47:08.825925+0000 osd.7 (osd.7) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:10.028 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:09 smithi055 bash[9296]: audit 2020-09-08T15:47:09.131700+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? v1:172.21.15.104:0/3244986967' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2020-09-08T15:47:10.029 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:09 smithi055 bash[9296]: audit 2020-09-08T15:47:09.700736+0000 mon.a (mon.0) 1323 : audit [INF] from='client.? v1:172.21.15.104:0/3244986967' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2020-09-08T15:47:10.029 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:09 smithi055 bash[9296]: cluster 2020-09-08T15:47:09.700785+0000 mon.a (mon.0) 1324 : cluster [DBG] osdmap e314: 12 total, 12 up, 12 in 2020-09-08T15:47:10.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:09 smithi104 bash[9203]: cluster 2020-09-08T15:47:08.825797+0000 osd.7 (osd.7) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:10.186 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:09 smithi104 bash[9203]: cluster 2020-09-08T15:47:08.825925+0000 osd.7 (osd.7) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:10.187 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:09 smithi104 bash[9203]: audit 2020-09-08T15:47:09.131700+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? v1:172.21.15.104:0/3244986967' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2020-09-08T15:47:10.187 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:09 smithi104 bash[9203]: audit 2020-09-08T15:47:09.700736+0000 mon.a (mon.0) 1323 : audit [INF] from='client.? v1:172.21.15.104:0/3244986967' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2020-09-08T15:47:10.187 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:09 smithi104 bash[9203]: cluster 2020-09-08T15:47:09.700785+0000 mon.a (mon.0) 1324 : cluster [DBG] osdmap e314: 12 total, 12 up, 12 in 2020-09-08T15:47:10.187 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:10.289 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:10.407 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:10.408 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:47:10 smithi055 bash[11956]: debug 2020-09-08T15:47:10.175+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:10.512 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:10.613 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2020-09-08T15:47:10.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:47:10 smithi166 bash[18960]: debug 2020-09-08T15:47:10.404+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:10.629 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:10.731 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:10.791 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:47:10.851 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:10.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:10 smithi104 bash[9203]: cluster 2020-09-08T15:47:09.744792+0000 mgr.x (mgr.24118) 595 : cluster [DBG] pgmap v825: 17 pgs: 16 unknown, 1 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:10.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:10 smithi104 bash[9203]: cluster 2020-09-08T15:47:09.911424+0000 osd.0 (osd.0) 37 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:10.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:10 smithi104 bash[9203]: cluster 2020-09-08T15:47:09.911500+0000 osd.0 (osd.0) 38 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:10.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:10 smithi104 bash[9203]: cluster 2020-09-08T15:47:10.703031+0000 mon.a (mon.0) 1325 : cluster [DBG] osdmap e315: 12 total, 12 up, 12 in 2020-09-08T15:47:10.962 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:10.969 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:47:10.974 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:47:10 smithi166 bash[14254]: debug 2020-09-08T15:47:10.844+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:10.974 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:10 smithi166 bash[9446]: cluster 2020-09-08T15:47:09.744792+0000 mgr.x (mgr.24118) 595 : cluster [DBG] pgmap v825: 17 pgs: 16 unknown, 1 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:10.975 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:10 smithi166 bash[9446]: cluster 2020-09-08T15:47:09.911424+0000 osd.0 (osd.0) 37 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:10.975 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:10 smithi166 bash[9446]: cluster 2020-09-08T15:47:09.911500+0000 osd.0 (osd.0) 38 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:10.975 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:10 smithi166 bash[9446]: cluster 2020-09-08T15:47:10.703031+0000 mon.a (mon.0) 1325 : cluster [DBG] osdmap e315: 12 total, 12 up, 12 in 2020-09-08T15:47:10.975 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:47:10 smithi166 bash[16593]: debug 2020-09-08T15:47:10.624+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:11.087 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:11.103 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:47:11.135 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool application enable unique_pool_2 rados --yes-i-really-mean-it || true 2020-09-08T15:47:11.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:10 smithi055 bash[9296]: cluster 2020-09-08T15:47:09.744792+0000 mgr.x (mgr.24118) 595 : cluster [DBG] pgmap v825: 17 pgs: 16 unknown, 1 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:11.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:10 smithi055 bash[9296]: cluster 2020-09-08T15:47:09.911424+0000 osd.0 (osd.0) 37 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:11.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:10 smithi055 bash[9296]: cluster 2020-09-08T15:47:09.911500+0000 osd.0 (osd.0) 38 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:11.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:10 smithi055 bash[9296]: cluster 2020-09-08T15:47:10.703031+0000 mon.a (mon.0) 1325 : cluster [DBG] osdmap e315: 12 total, 12 up, 12 in 2020-09-08T15:47:11.190 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:11.257 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:47:11.315 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:11.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:47:11 smithi166 bash[14254]: debug 2020-09-08T15:47:11.080+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:11.420 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:47:11.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:47:11 smithi055 bash[14301]: debug 2020-09-08T15:47:11.307+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:11.765 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:47:11.874 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:12.004 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:12.113 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:12.158 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:47:12 smithi055 bash[16646]: debug 2020-09-08T15:47:12.003+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:12.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:47:11 smithi104 bash[19922]: debug 2020-09-08T15:47:11.765+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:12.265 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:12.373 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:47:12.598 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:47:12.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:47:12 smithi166 bash[16593]: debug 2020-09-08T15:47:12.264+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:12.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:47:12 smithi104 bash[22305]: debug 2020-09-08T15:47:12.597+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:12.703 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:12.820 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:12.927 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:13.009 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:12 smithi104 bash[9203]: cluster 2020-09-08T15:47:11.706744+0000 mon.a (mon.0) 1326 : cluster [DBG] osdmap e316: 12 total, 12 up, 12 in 2020-09-08T15:47:13.052 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:12 smithi055 bash[9296]: cluster 2020-09-08T15:47:11.706744+0000 mon.a (mon.0) 1326 : cluster [DBG] osdmap e316: 12 total, 12 up, 12 in 2020-09-08T15:47:13.052 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:13.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:12 smithi166 bash[9446]: cluster 2020-09-08T15:47:11.706744+0000 mon.a (mon.0) 1326 : cluster [DBG] osdmap e316: 12 total, 12 up, 12 in 2020-09-08T15:47:13.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:47:12 smithi166 bash[16593]: debug 2020-09-08T15:47:12.820+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:13.163 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:13.279 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:13.384 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:47:13.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:47:13 smithi055 bash[11956]: debug 2020-09-08T15:47:13.051+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:13.614 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:47:13.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:47:13 smithi166 bash[18960]: debug 2020-09-08T15:47:13.276+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:13.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:47:13 smithi104 bash[17588]: debug 2020-09-08T15:47:13.613+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:13.721 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:47:14.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:13 smithi166 bash[9446]: cluster 2020-09-08T15:47:11.745319+0000 mgr.x (mgr.24118) 596 : cluster [DBG] pgmap v828: 17 pgs: 10 unknown, 7 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:14.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:13 smithi055 bash[9296]: cluster 2020-09-08T15:47:11.745319+0000 mgr.x (mgr.24118) 596 : cluster [DBG] pgmap v828: 17 pgs: 10 unknown, 7 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:14.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:13 smithi104 bash[9203]: cluster 2020-09-08T15:47:11.745319+0000 mgr.x (mgr.24118) 596 : cluster [DBG] pgmap v828: 17 pgs: 10 unknown, 7 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:14.199 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:47:14.300 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:14.561 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:47:14 smithi104 bash[22305]: debug 2020-09-08T15:47:14.197+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:14.562 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:14.666 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:14.841 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:47:14 smithi104 bash[15273]: debug 2020-09-08T15:47:14.561+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:15.151 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:15.256 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:15.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:47:15 smithi104 bash[15273]: debug 2020-09-08T15:47:15.153+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:15.691 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:15.793 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:16.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:15 smithi166 bash[9446]: cluster 2020-09-08T15:47:13.745953+0000 mgr.x (mgr.24118) 597 : cluster [DBG] pgmap v829: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:16.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:15 smithi055 bash[9296]: cluster 2020-09-08T15:47:13.745953+0000 mgr.x (mgr.24118) 597 : cluster [DBG] pgmap v829: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:16.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:15 smithi104 bash[9203]: cluster 2020-09-08T15:47:13.745953+0000 mgr.x (mgr.24118) 597 : cluster [DBG] pgmap v829: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:16.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:47:15 smithi104 bash[15273]: debug 2020-09-08T15:47:15.689+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:16.435 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:16.540 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:16.694 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:16.802 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:16.813 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:47:16 smithi104 bash[15273]: debug 2020-09-08T15:47:16.437+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:16.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:47:16 smithi166 bash[16593]: debug 2020-09-08T15:47:16.692+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:17.160 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:17.265 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:17.388 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:17.494 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:17.606 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:47:17 smithi055 bash[11956]: debug 2020-09-08T15:47:17.391+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:17.607 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:17.689 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:47:17.714 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:47:17.721 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:47:17.744 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:47:17 smithi104 bash[15273]: debug 2020-09-08T15:47:17.161+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:17.745 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:47:17.769 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:47:17.803 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:47:17.839 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:47:17.862 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:47:17.891 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:47:17.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:47:17 smithi055 bash[11956]: debug 2020-09-08T15:47:17.607+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:17.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:17 smithi055 bash[9296]: cluster 2020-09-08T15:47:15.746605+0000 mgr.x (mgr.24118) 598 : cluster [DBG] pgmap v830: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:17.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:17 smithi055 bash[9296]: cluster 2020-09-08T15:47:16.591392+0000 osd.5 (osd.5) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:17.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:17 smithi055 bash[9296]: cluster 2020-09-08T15:47:16.591550+0000 osd.5 (osd.5) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:17.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:17 smithi055 bash[9296]: cluster 2020-09-08T15:47:16.713924+0000 mon.a (mon.0) 1327 : cluster [DBG] osdmap e317: 12 total, 12 up, 12 in 2020-09-08T15:47:17.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:17 smithi055 bash[9296]: audit 2020-09-08T15:47:17.666537+0000 mon.a (mon.0) 1328 : audit [DBG] from='client.? v1:172.21.15.104:0/4151037710' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:47:17.917 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:47:17.954 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:47:17.985 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:47:18.009 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:17 smithi104 bash[9203]: cluster 2020-09-08T15:47:15.746605+0000 mgr.x (mgr.24118) 598 : cluster [DBG] pgmap v830: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:18.010 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:17 smithi104 bash[9203]: cluster 2020-09-08T15:47:16.591392+0000 osd.5 (osd.5) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:18.010 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:17 smithi104 bash[9203]: cluster 2020-09-08T15:47:16.591550+0000 osd.5 (osd.5) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:18.010 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:17 smithi104 bash[9203]: cluster 2020-09-08T15:47:16.713924+0000 mon.a (mon.0) 1327 : cluster [DBG] osdmap e317: 12 total, 12 up, 12 in 2020-09-08T15:47:18.010 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:17 smithi104 bash[9203]: audit 2020-09-08T15:47:17.666537+0000 mon.a (mon.0) 1328 : audit [DBG] from='client.? v1:172.21.15.104:0/4151037710' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:47:18.011 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:47:18.014 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:47:18.014 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:47:18.015 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:47:18.015 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:47:18.015 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:47:18.016 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:47:18.016 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:47:18.016 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:47:18.016 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:47:18.017 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:47:18.018 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:47:18.019 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:47:18.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:17 smithi166 bash[9446]: cluster 2020-09-08T15:47:15.746605+0000 mgr.x (mgr.24118) 598 : cluster [DBG] pgmap v830: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:18.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:17 smithi166 bash[9446]: cluster 2020-09-08T15:47:16.591392+0000 osd.5 (osd.5) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:18.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:17 smithi166 bash[9446]: cluster 2020-09-08T15:47:16.591550+0000 osd.5 (osd.5) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:18.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:17 smithi166 bash[9446]: cluster 2020-09-08T15:47:16.713924+0000 mon.a (mon.0) 1327 : cluster [DBG] osdmap e317: 12 total, 12 up, 12 in 2020-09-08T15:47:18.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:17 smithi166 bash[9446]: audit 2020-09-08T15:47:17.666537+0000 mon.a (mon.0) 1328 : audit [DBG] from='client.? v1:172.21.15.104:0/4151037710' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:47:18.132 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:47:18.152 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:47:18.152 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T15:47:18.152 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2020-09-08T15:47:18.152 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2020-09-08T15:47:18.154 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:47:18.234 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:18.427 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:47:18 smithi104 bash[19922]: debug 2020-09-08T15:47:18.125+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:18.851 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:18.956 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:19.106 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:18 smithi166 bash[9446]: cluster 2020-09-08T15:47:17.747329+0000 mgr.x (mgr.24118) 599 : cluster [DBG] pgmap v832: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:19.108 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:19.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:18 smithi055 bash[9296]: cluster 2020-09-08T15:47:17.747329+0000 mgr.x (mgr.24118) 599 : cluster [DBG] pgmap v832: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:19.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:18 smithi104 bash[9203]: cluster 2020-09-08T15:47:17.747329+0000 mgr.x (mgr.24118) 599 : cluster [DBG] pgmap v832: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:19.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:47:18 smithi104 bash[15273]: debug 2020-09-08T15:47:18.853+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:19.212 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:19.345 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:19.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:47:19 smithi166 bash[11928]: debug 2020-09-08T15:47:19.104+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:19.366 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:47:19 smithi166 bash[16593]: debug 2020-09-08T15:47:19.344+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:19.451 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:19.670 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:19.770 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:47:19 smithi104 bash[15273]: debug 2020-09-08T15:47:19.669+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:19.772 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2020-09-08T15:47:19.773 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:19.884 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:19.903 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:47:19.990 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:47:20.113 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:47:19 smithi055 bash[11956]: debug 2020-09-08T15:47:19.883+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:20.114 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:47:20.220 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:20.367 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:20.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:47:20 smithi055 bash[19003]: debug 2020-09-08T15:47:20.115+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:20.472 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:20.618 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:47:20 smithi166 bash[14254]: debug 2020-09-08T15:47:20.364+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:20.658 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:20.718 INFO:teuthology.orchestra.run.smithi104.stderr:enabled application 'rados' on pool 'unique_pool_2' 2020-09-08T15:47:20.762 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:20.793 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:20 smithi104 bash[9203]: cluster 2020-09-08T15:47:19.707368+0000 mon.a (mon.0) 1329 : cluster [DBG] osdmap e318: 12 total, 12 up, 12 in 2020-09-08T15:47:20.794 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:20 smithi104 bash[9203]: cluster 2020-09-08T15:47:19.729966+0000 osd.10 (osd.10) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:20.794 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:20 smithi104 bash[9203]: cluster 2020-09-08T15:47:19.730090+0000 osd.10 (osd.10) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:20.794 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:20 smithi104 bash[9203]: audit 2020-09-08T15:47:20.253105+0000 mon.a (mon.0) 1330 : audit [INF] from='client.? v1:172.21.15.104:0/849462952' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T15:47:20.794 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:47:20 smithi104 bash[15273]: debug 2020-09-08T15:47:20.657+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:20.884 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:20.988 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:21.000 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:20 smithi055 bash[9296]: cluster 2020-09-08T15:47:19.707368+0000 mon.a (mon.0) 1329 : cluster [DBG] osdmap e318: 12 total, 12 up, 12 in 2020-09-08T15:47:21.000 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:20 smithi055 bash[9296]: cluster 2020-09-08T15:47:19.729966+0000 osd.10 (osd.10) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:21.001 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:20 smithi055 bash[9296]: cluster 2020-09-08T15:47:19.730090+0000 osd.10 (osd.10) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:21.001 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:20 smithi055 bash[9296]: audit 2020-09-08T15:47:20.253105+0000 mon.a (mon.0) 1330 : audit [INF] from='client.? v1:172.21.15.104:0/849462952' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T15:47:21.094 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:21.118 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:47:20 smithi166 bash[18960]: debug 2020-09-08T15:47:20.880+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:21.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:20 smithi166 bash[9446]: cluster 2020-09-08T15:47:19.707368+0000 mon.a (mon.0) 1329 : cluster [DBG] osdmap e318: 12 total, 12 up, 12 in 2020-09-08T15:47:21.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:20 smithi166 bash[9446]: cluster 2020-09-08T15:47:19.729966+0000 osd.10 (osd.10) 27 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:21.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:20 smithi166 bash[9446]: cluster 2020-09-08T15:47:19.730090+0000 osd.10 (osd.10) 28 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:21.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:20 smithi166 bash[9446]: audit 2020-09-08T15:47:20.253105+0000 mon.a (mon.0) 1330 : audit [INF] from='client.? v1:172.21.15.104:0/849462952' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T15:47:21.201 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:47:21.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:47:21 smithi055 bash[16646]: debug 2020-09-08T15:47:21.095+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:21.544 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:47:21.674 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:47:21.692 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:47:21.724 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:47:21.794 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:22.048 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:47:21 smithi104 bash[22305]: debug 2020-09-08T15:47:21.693+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:22.049 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:21 smithi104 bash[9203]: cluster 2020-09-08T15:47:19.747950+0000 mgr.x (mgr.24118) 600 : cluster [DBG] pgmap v834: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:22.049 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:21 smithi104 bash[9203]: audit 2020-09-08T15:47:20.720806+0000 mon.a (mon.0) 1331 : audit [INF] from='client.? v1:172.21.15.104:0/849462952' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T15:47:22.049 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:21 smithi104 bash[9203]: cluster 2020-09-08T15:47:20.720865+0000 mon.a (mon.0) 1332 : cluster [DBG] osdmap e319: 12 total, 12 up, 12 in 2020-09-08T15:47:22.057 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:21 smithi104 bash[9203]: audit 2020-09-08T15:47:20.763423+0000 mon.a (mon.0) 1333 : audit [INF] from='client.? v1:172.21.15.104:0/272557050' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:47:22.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:21 smithi166 bash[9446]: cluster 2020-09-08T15:47:19.747950+0000 mgr.x (mgr.24118) 600 : cluster [DBG] pgmap v834: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:22.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:21 smithi166 bash[9446]: audit 2020-09-08T15:47:20.720806+0000 mon.a (mon.0) 1331 : audit [INF] from='client.? v1:172.21.15.104:0/849462952' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T15:47:22.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:21 smithi166 bash[9446]: cluster 2020-09-08T15:47:20.720865+0000 mon.a (mon.0) 1332 : cluster [DBG] osdmap e319: 12 total, 12 up, 12 in 2020-09-08T15:47:22.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:21 smithi166 bash[9446]: audit 2020-09-08T15:47:20.763423+0000 mon.a (mon.0) 1333 : audit [INF] from='client.? v1:172.21.15.104:0/272557050' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:47:22.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:21 smithi055 bash[9296]: cluster 2020-09-08T15:47:19.747950+0000 mgr.x (mgr.24118) 600 : cluster [DBG] pgmap v834: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:21 smithi055 bash[9296]: audit 2020-09-08T15:47:20.720806+0000 mon.a (mon.0) 1331 : audit [INF] from='client.? v1:172.21.15.104:0/849462952' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T15:47:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:21 smithi055 bash[9296]: cluster 2020-09-08T15:47:20.720865+0000 mon.a (mon.0) 1332 : cluster [DBG] osdmap e319: 12 total, 12 up, 12 in 2020-09-08T15:47:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:21 smithi055 bash[9296]: audit 2020-09-08T15:47:20.763423+0000 mon.a (mon.0) 1333 : audit [INF] from='client.? v1:172.21.15.104:0/272557050' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:47:22.373 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:22.479 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:22.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:47:22 smithi104 bash[15273]: debug 2020-09-08T15:47:22.373+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:22.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:22 smithi166 bash[9446]: cluster 2020-09-08T15:47:21.718753+0000 mon.a (mon.0) 1334 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:47:22.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:22 smithi166 bash[9446]: audit 2020-09-08T15:47:21.724519+0000 mon.a (mon.0) 1335 : audit [INF] from='client.? v1:172.21.15.104:0/272557050' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:47:22.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:22 smithi166 bash[9446]: cluster 2020-09-08T15:47:21.724564+0000 mon.a (mon.0) 1336 : cluster [DBG] osdmap e320: 12 total, 12 up, 12 in 2020-09-08T15:47:22.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:22 smithi166 bash[9446]: cluster 2020-09-08T15:47:21.748670+0000 mgr.x (mgr.24118) 601 : cluster [DBG] pgmap v837: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:22.899 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:22.974 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:22 smithi104 bash[9203]: cluster 2020-09-08T15:47:21.718753+0000 mon.a (mon.0) 1334 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:47:22.975 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:22 smithi104 bash[9203]: audit 2020-09-08T15:47:21.724519+0000 mon.a (mon.0) 1335 : audit [INF] from='client.? v1:172.21.15.104:0/272557050' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:47:22.975 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:22 smithi104 bash[9203]: cluster 2020-09-08T15:47:21.724564+0000 mon.a (mon.0) 1336 : cluster [DBG] osdmap e320: 12 total, 12 up, 12 in 2020-09-08T15:47:22.976 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:22 smithi104 bash[9203]: cluster 2020-09-08T15:47:21.748670+0000 mgr.x (mgr.24118) 601 : cluster [DBG] pgmap v837: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:23.013 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:22 smithi055 bash[9296]: cluster 2020-09-08T15:47:21.718753+0000 mon.a (mon.0) 1334 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:47:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:22 smithi055 bash[9296]: audit 2020-09-08T15:47:21.724519+0000 mon.a (mon.0) 1335 : audit [INF] from='client.? v1:172.21.15.104:0/272557050' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:47:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:22 smithi055 bash[9296]: cluster 2020-09-08T15:47:21.724564+0000 mon.a (mon.0) 1336 : cluster [DBG] osdmap e320: 12 total, 12 up, 12 in 2020-09-08T15:47:23.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:22 smithi055 bash[9296]: cluster 2020-09-08T15:47:21.748670+0000 mgr.x (mgr.24118) 601 : cluster [DBG] pgmap v837: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:23.194 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:23.299 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:23.335 INFO:teuthology.orchestra.run.smithi166:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.2 -b 65536 --object-size 65536 -p unique_pool_2 bench 90 write' 2020-09-08T15:47:23.339 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2020-09-08T15:47:23.340 INFO:teuthology.orchestra.run:waiting for 3000 2020-09-08T15:47:23.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:47:22 smithi166 bash[18960]: debug 2020-09-08T15:47:22.896+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:23.379 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:hints = 1 2020-09-08T15:47:23.379 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2020-09-08T15:47:23.379 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Object prefix: benchmark_data_smithi166_32054 2020-09-08T15:47:23.380 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:47:23.381 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 0 0 0 0 0 0 - 0 2020-09-08T15:47:23.382 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:47:23 smithi104 bash[15273]: debug 2020-09-08T15:47:23.193+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:23.424 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:23.529 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:23.646 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:23.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:47:23 smithi055 bash[16646]: debug 2020-09-08T15:47:23.423+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:23.751 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:23.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:47:23 smithi166 bash[14254]: debug 2020-09-08T15:47:23.644+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:23.882 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:23.986 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:24.108 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:47:23 smithi055 bash[14301]: debug 2020-09-08T15:47:23.883+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:24.109 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:47:24 smithi055 bash[11956]: debug 2020-09-08T15:47:24.111+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:24.109 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:24.213 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:24.335 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:24.382 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 1 16 3964 3948 246.738 246.75 0.00384244 0.00403714 2020-09-08T15:47:24.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:47:24 smithi055 bash[14301]: debug 2020-09-08T15:47:24.335+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:24.440 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:47:24.798 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:24 smithi104 bash[9203]: cluster 2020-09-08T15:47:22.978960+0000 osd.7 (osd.7) 61 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:47:24.798 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:24 smithi104 bash[9203]: cluster 2020-09-08T15:47:23.036912+0000 osd.7 (osd.7) 62 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:47:24.799 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:24 smithi104 bash[9203]: cluster 2020-09-08T15:47:23.749354+0000 mgr.x (mgr.24118) 602 : cluster [DBG] pgmap v838: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:25.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:24 smithi166 bash[9446]: cluster 2020-09-08T15:47:22.978960+0000 osd.7 (osd.7) 61 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:47:25.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:24 smithi166 bash[9446]: cluster 2020-09-08T15:47:23.036912+0000 osd.7 (osd.7) 62 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:47:25.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:24 smithi166 bash[9446]: cluster 2020-09-08T15:47:23.749354+0000 mgr.x (mgr.24118) 602 : cluster [DBG] pgmap v838: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:25.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:24 smithi055 bash[9296]: cluster 2020-09-08T15:47:22.978960+0000 osd.7 (osd.7) 61 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:47:25.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:24 smithi055 bash[9296]: cluster 2020-09-08T15:47:23.036912+0000 osd.7 (osd.7) 62 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:47:25.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:24 smithi055 bash[9296]: cluster 2020-09-08T15:47:23.749354+0000 mgr.x (mgr.24118) 602 : cluster [DBG] pgmap v838: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:25.163 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:47:25.267 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:47:25.380 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 2 16 7832 7816 244.23 241.75 0.00279977 0.00408556 2020-09-08T15:47:25.385 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:47:25.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:47:25 smithi104 bash[19922]: debug 2020-09-08T15:47:25.169+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:25.563 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:47:26.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:25 smithi166 bash[9446]: cluster 2020-09-08T15:47:25.061663+0000 osd.3 (osd.3) 35 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:26.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:25 smithi166 bash[9446]: cluster 2020-09-08T15:47:25.061731+0000 osd.3 (osd.3) 36 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:26.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:25 smithi055 bash[9296]: cluster 2020-09-08T15:47:25.061663+0000 osd.3 (osd.3) 35 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:26.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:25 smithi055 bash[9296]: cluster 2020-09-08T15:47:25.061731+0000 osd.3 (osd.3) 36 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:26.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:25 smithi104 bash[9203]: cluster 2020-09-08T15:47:25.061663+0000 osd.3 (osd.3) 35 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:26.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:25 smithi104 bash[9203]: cluster 2020-09-08T15:47:25.061731+0000 osd.3 (osd.3) 36 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:26.382 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 3 16 11651 11635 242.376 238.688 0.00311515 0.00411975 2020-09-08T15:47:27.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:26 smithi166 bash[9446]: cluster 2020-09-08T15:47:25.750033+0000 mgr.x (mgr.24118) 603 : cluster [DBG] pgmap v839: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:27.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:26 smithi166 bash[9446]: cluster 2020-09-08T15:47:25.815155+0000 mon.a (mon.0) 1337 : cluster [DBG] osdmap e321: 12 total, 12 up, 12 in 2020-09-08T15:47:27.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:26 smithi055 bash[9296]: cluster 2020-09-08T15:47:25.750033+0000 mgr.x (mgr.24118) 603 : cluster [DBG] pgmap v839: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:27.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:26 smithi055 bash[9296]: cluster 2020-09-08T15:47:25.815155+0000 mon.a (mon.0) 1337 : cluster [DBG] osdmap e321: 12 total, 12 up, 12 in 2020-09-08T15:47:27.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:26 smithi104 bash[9203]: cluster 2020-09-08T15:47:25.750033+0000 mgr.x (mgr.24118) 603 : cluster [DBG] pgmap v839: 17 pgs: 17 active+clean; 0 B data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:47:27.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:26 smithi104 bash[9203]: cluster 2020-09-08T15:47:25.815155+0000 mon.a (mon.0) 1337 : cluster [DBG] osdmap e321: 12 total, 12 up, 12 in 2020-09-08T15:47:27.380 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 4 16 14350 14334 223.95 168.688 0.00386245 0.00446106 2020-09-08T15:47:27.556 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:47:27.661 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:27.811 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:27.916 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:27.941 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:47:27 smithi104 bash[17588]: debug 2020-09-08T15:47:27.557+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:28.027 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:28.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:47:27 smithi166 bash[14254]: debug 2020-09-08T15:47:27.808+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:28.131 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:28.255 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:28.358 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:28.382 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 5 16 16954 16938 211.649 162.75 0.00337687 0.00472011 2020-09-08T15:47:28.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:47:28 smithi055 bash[14301]: debug 2020-09-08T15:47:28.027+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:28.462 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:47:28 smithi166 bash[11928]: debug 2020-09-08T15:47:28.252+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:28.463 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:28.569 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:47:28.807 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:28 smithi166 bash[9446]: cluster 2020-09-08T15:47:27.750604+0000 mgr.x (mgr.24118) 604 : cluster [DBG] pgmap v841: 17 pgs: 17 active+clean; 446 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 MiB/s wr, 1.01k op/s 2020-09-08T15:47:28.808 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:47:28 smithi166 bash[11928]: debug 2020-09-08T15:47:28.460+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:28.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:28 smithi104 bash[9203]: cluster 2020-09-08T15:47:27.750604+0000 mgr.x (mgr.24118) 604 : cluster [DBG] pgmap v841: 17 pgs: 17 active+clean; 446 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 MiB/s wr, 1.01k op/s 2020-09-08T15:47:29.126 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:47:29.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:28 smithi055 bash[9296]: cluster 2020-09-08T15:47:27.750604+0000 mgr.x (mgr.24118) 604 : cluster [DBG] pgmap v841: 17 pgs: 17 active+clean; 446 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 MiB/s wr, 1.01k op/s 2020-09-08T15:47:29.229 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:29.382 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 6 16 20051 20035 208.632 193.562 0.00311216 0.00478854 2020-09-08T15:47:29.399 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:47:29 smithi104 bash[17588]: debug 2020-09-08T15:47:29.125+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:30.254 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:30.358 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:30.382 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 7 16 22608 22592 201.657 159.812 0.00513682 0.00495405 2020-09-08T15:47:30.503 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:30.603 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:47:30.604 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2020-09-08T15:47:30.604 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2020-09-08T15:47:30.604 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T15:47:30.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.604 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:168405 2.5 e172/126 2:a65aa17e:::benchmark_data_smithi166_20551_object20785:head v 172'10423, mlcod=172'10421)", 2020-09-08T15:47:30.605 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:34.490923+0000", 2020-09-08T15:47:30.605 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 476.11561929700002, 2020-09-08T15:47:30.605 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.25628678799999999, 2020-09-08T15:47:30.605 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:47:30.606 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:30.606 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:47:30.606 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.606 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:47:30.606 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.490923+0000", 2020-09-08T15:47:30.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:47:30.607 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.607 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:47:30.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.490923+0000", 2020-09-08T15:47:30.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6544999999999994e-05 2020-09-08T15:47:30.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:47:30.608 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.490990+0000", 2020-09-08T15:47:30.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2035000000000001e-05 2020-09-08T15:47:30.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.609 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.609 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:47:30.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491002+0000", 2020-09-08T15:47:30.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9689000000000001e-05 2020-09-08T15:47:30.610 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.610 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.610 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:47:30.610 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491021+0000", 2020-09-08T15:47:30.610 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0033e-05 2020-09-08T15:47:30.611 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.611 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.611 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:47:30.611 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491031+0000", 2020-09-08T15:47:30.611 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5401999999999995e-05 2020-09-08T15:47:30.612 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.612 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.612 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:47:30.612 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491097+0000", 2020-09-08T15:47:30.612 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0103999999999998e-05 2020-09-08T15:47:30.613 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.613 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.613 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:47:30.613 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491127+0000", 2020-09-08T15:47:30.613 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.25595135000000002 2020-09-08T15:47:30.614 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.614 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:47:30.614 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.747078+0000", 2020-09-08T15:47:30.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013163000000000001 2020-09-08T15:47:30.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.615 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:47:30.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.747210+0000", 2020-09-08T15:47:30.616 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.25628678799999999 2020-09-08T15:47:30.616 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.616 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.616 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.616 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.617 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.617 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:168406 2.5 e172/126 2:adf3a2f3:::benchmark_data_smithi166_20551_object20786:head v 172'10424, mlcod=172'10421)", 2020-09-08T15:47:30.617 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:34.491101+0000", 2020-09-08T15:47:30.617 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 476.11544180700002, 2020-09-08T15:47:30.617 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26448300600000002, 2020-09-08T15:47:30.617 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:47:30.618 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:30.618 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:47:30.618 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.618 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:47:30.618 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491101+0000", 2020-09-08T15:47:30.619 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:47:30.619 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:47:30.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491101+0000", 2020-09-08T15:47:30.620 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3269000000000005e-05 2020-09-08T15:47:30.620 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.620 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.620 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:47:30.620 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491184+0000", 2020-09-08T15:47:30.621 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0890000000000006e-06 2020-09-08T15:47:30.621 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.621 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.621 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:47:30.621 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491193+0000", 2020-09-08T15:47:30.621 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5333e-05 2020-09-08T15:47:30.622 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:47:30.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491208+0000", 2020-09-08T15:47:30.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3520000000000003e-06 2020-09-08T15:47:30.623 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.623 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.623 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:47:30.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.491215+0000", 2020-09-08T15:47:30.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0056763029999999997 2020-09-08T15:47:30.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.624 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:47:30.624 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.496891+0000", 2020-09-08T15:47:30.624 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0164000000000001e-05 2020-09-08T15:47:30.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.625 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:47:30.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.496921+0000", 2020-09-08T15:47:30.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.258572301 2020-09-08T15:47:30.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:47:30.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.755493+0000", 2020-09-08T15:47:30.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0195e-05 2020-09-08T15:47:30.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.627 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:47:30.627 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.755584+0000", 2020-09-08T15:47:30.627 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26448300600000002 2020-09-08T15:47:30.627 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.628 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.628 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.629 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.630 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.630 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:168416 2.5 e172/126 2:ad81a029:::benchmark_data_smithi166_20551_object20796:head v 172'10425, mlcod=172'10421)", 2020-09-08T15:47:30.630 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:34.499124+0000", 2020-09-08T15:47:30.630 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 476.10741820999999, 2020-09-08T15:47:30.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26367184900000001, 2020-09-08T15:47:30.631 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:47:30.631 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:30.632 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:47:30.632 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.632 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:47:30.632 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.499124+0000", 2020-09-08T15:47:30.632 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:47:30.633 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.633 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:47:30.633 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.499124+0000", 2020-09-08T15:47:30.633 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3675000000000003e-05 2020-09-08T15:47:30.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.634 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.634 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:47:30.634 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.499188+0000", 2020-09-08T15:47:30.635 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7679999999999997e-06 2020-09-08T15:47:30.635 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.635 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.635 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:47:30.635 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.499197+0000", 2020-09-08T15:47:30.636 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4082e-05 2020-09-08T15:47:30.636 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.636 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.636 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:47:30.636 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.499211+0000", 2020-09-08T15:47:30.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.691e-06 2020-09-08T15:47:30.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.637 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:47:30.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.499217+0000", 2020-09-08T15:47:30.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.24804995799999999 2020-09-08T15:47:30.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.638 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:47:30.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.747267+0000", 2020-09-08T15:47:30.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2809999999999998e-05 2020-09-08T15:47:30.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:47:30.639 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.747310+0000", 2020-09-08T15:47:30.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.015420059 2020-09-08T15:47:30.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.640 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:47:30.640 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.762730+0000", 2020-09-08T15:47:30.640 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5806e-05 2020-09-08T15:47:30.641 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.641 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.641 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:47:30.641 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:34.762796+0000", 2020-09-08T15:47:30.641 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26367184900000001 2020-09-08T15:47:30.642 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.642 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.643 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.644 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.645 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:190024 2.4 e179/126 2:223e1daa:::benchmark_data_smithi166_20551_object42404:head v 179'11903, mlcod=178'11901)", 2020-09-08T15:47:30.645 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:56.751495+0000", 2020-09-08T15:47:30.645 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 453.85504726900001, 2020-09-08T15:47:30.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.24695220000000001, 2020-09-08T15:47:30.646 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:47:30.646 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:30.646 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:47:30.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:47:30.647 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.751495+0000", 2020-09-08T15:47:30.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:47:30.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:47:30.648 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.751495+0000", 2020-09-08T15:47:30.648 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1027999999999999e-05 2020-09-08T15:47:30.648 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.648 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.648 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:47:30.649 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.751556+0000", 2020-09-08T15:47:30.649 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4340000000000003e-06 2020-09-08T15:47:30.649 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.649 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.649 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:47:30.650 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.751564+0000", 2020-09-08T15:47:30.650 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2662e-05 2020-09-08T15:47:30.650 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.650 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.650 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:47:30.651 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.751576+0000", 2020-09-08T15:47:30.651 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3640000000000004e-06 2020-09-08T15:47:30.651 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.651 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.651 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:47:30.651 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.751583+0000", 2020-09-08T15:47:30.652 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.22617980100000001 2020-09-08T15:47:30.652 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.652 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.652 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:47:30.652 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.977762+0000", 2020-09-08T15:47:30.653 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9385000000000001e-05 2020-09-08T15:47:30.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.653 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.653 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:47:30.653 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.977792+0000", 2020-09-08T15:47:30.654 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.020575709000000001 2020-09-08T15:47:30.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:47:30.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.998368+0000", 2020-09-08T15:47:30.655 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9816999999999998e-05 2020-09-08T15:47:30.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.656 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.656 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:47:30.656 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:56.998447+0000", 2020-09-08T15:47:30.656 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.24695220000000001 2020-09-08T15:47:30.657 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.657 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.657 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.657 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.657 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.658 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:190950 2.4 e179/126 2:238df183:::benchmark_data_smithi166_20551_object43330:head v 179'11957, mlcod=179'11953)", 2020-09-08T15:47:30.658 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:57.816566+0000", 2020-09-08T15:47:30.658 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 452.78997603800002, 2020-09-08T15:47:30.658 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.24666176100000001, 2020-09-08T15:47:30.658 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:47:30.658 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:30.659 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:47:30.659 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.659 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:47:30.659 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.816566+0000", 2020-09-08T15:47:30.659 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:47:30.660 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.660 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.660 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:47:30.660 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.816566+0000", 2020-09-08T15:47:30.660 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2438000000000001e-05 2020-09-08T15:47:30.661 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.661 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.661 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:47:30.661 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.816629+0000", 2020-09-08T15:47:30.663 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4800000000000001e-06 2020-09-08T15:47:30.663 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.664 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.664 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:47:30.664 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.816637+0000", 2020-09-08T15:47:30.664 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2876e-05 2020-09-08T15:47:30.664 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.665 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.665 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:47:30.665 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.816650+0000", 2020-09-08T15:47:30.665 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2620000000000004e-06 2020-09-08T15:47:30.665 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.666 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.672 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:47:30.672 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.816656+0000", 2020-09-08T15:47:30.672 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0235000000000003e-05 2020-09-08T15:47:30.672 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.673 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.673 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:47:30.673 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.816707+0000", 2020-09-08T15:47:30.673 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6855e-05 2020-09-08T15:47:30.673 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.674 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.674 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:47:30.674 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.816734+0000", 2020-09-08T15:47:30.674 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.246288009 2020-09-08T15:47:30.674 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.675 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.675 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:47:30.675 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.063022+0000", 2020-09-08T15:47:30.675 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020660599999999999 2020-09-08T15:47:30.675 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.676 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.676 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:47:30.676 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.063228+0000", 2020-09-08T15:47:30.676 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.24666176100000001 2020-09-08T15:47:30.676 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.676 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.677 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.677 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.677 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.677 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:190956 2.c e179/126 2:3977f071:::benchmark_data_smithi166_20551_object43336:head v 179'11882, mlcod=179'11879)", 2020-09-08T15:47:30.677 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:57.817105+0000", 2020-09-08T15:47:30.678 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 452.78943752499998, 2020-09-08T15:47:30.678 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.246211079, 2020-09-08T15:47:30.678 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:47:30.678 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:30.678 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:47:30.679 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.679 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:47:30.679 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.817105+0000", 2020-09-08T15:47:30.679 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:47:30.679 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.680 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.680 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:47:30.680 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.817105+0000", 2020-09-08T15:47:30.680 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1634e-05 2020-09-08T15:47:30.680 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.681 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.681 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:47:30.681 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.817167+0000", 2020-09-08T15:47:30.681 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6590000000000007e-06 2020-09-08T15:47:30.681 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.681 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.682 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:47:30.682 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.817174+0000", 2020-09-08T15:47:30.682 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2143e-05 2020-09-08T15:47:30.682 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.682 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.683 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:47:30.683 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.817186+0000", 2020-09-08T15:47:30.683 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2000000000000002e-06 2020-09-08T15:47:30.683 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.683 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.684 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:47:30.684 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.817192+0000", 2020-09-08T15:47:30.684 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003370184 2020-09-08T15:47:30.684 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.684 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.684 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:47:30.685 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.820562+0000", 2020-09-08T15:47:30.685 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.7878999999999999e-05 2020-09-08T15:47:30.685 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.685 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.685 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:47:30.686 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.820590+0000", 2020-09-08T15:47:30.686 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.242655176 2020-09-08T15:47:30.686 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.686 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.686 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:47:30.687 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.063245+0000", 2020-09-08T15:47:30.687 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1204000000000005e-05 2020-09-08T15:47:30.687 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.687 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.687 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:47:30.688 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.063316+0000", 2020-09-08T15:47:30.688 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.246211079 2020-09-08T15:47:30.688 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.688 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.688 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.688 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.689 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.689 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:190953 2.4 e179/126 2:2ee0e6e8:::benchmark_data_smithi166_20551_object43333:head v 179'11958, mlcod=179'11953)", 2020-09-08T15:47:30.689 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:39:57.820699+0000", 2020-09-08T15:47:30.689 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 452.78584389999997, 2020-09-08T15:47:30.689 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.265174093, 2020-09-08T15:47:30.690 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:47:30.690 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:30.690 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:47:30.691 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.691 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:47:30.691 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.820699+0000", 2020-09-08T15:47:30.691 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:47:30.691 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.692 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.692 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:47:30.693 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:57.820699+0000", 2020-09-08T15:47:30.693 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.24242719200000001 2020-09-08T15:47:30.693 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.693 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.693 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:47:30.694 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.063126+0000", 2020-09-08T15:47:30.694 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3751999999999999e-05 2020-09-08T15:47:30.694 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.694 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.695 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:47:30.695 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.063149+0000", 2020-09-08T15:47:30.695 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6261999999999999e-05 2020-09-08T15:47:30.695 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.695 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.696 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:47:30.696 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.063186+0000", 2020-09-08T15:47:30.696 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.632e-05 2020-09-08T15:47:30.696 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.696 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.697 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:47:30.697 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.063202+0000", 2020-09-08T15:47:30.697 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3277999999999999e-05 2020-09-08T15:47:30.697 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.697 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.698 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:47:30.698 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.063275+0000", 2020-09-08T15:47:30.698 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1987000000000001e-05 2020-09-08T15:47:30.698 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.698 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.699 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:47:30.699 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.063297+0000", 2020-09-08T15:47:30.699 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.022480079 2020-09-08T15:47:30.699 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.699 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.699 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:47:30.700 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.085777+0000", 2020-09-08T15:47:30.700 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5223000000000006e-05 2020-09-08T15:47:30.700 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.700 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.700 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:47:30.701 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:39:58.085873+0000", 2020-09-08T15:47:30.701 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.265174093 2020-09-08T15:47:30.701 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.701 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.701 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.702 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.702 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.702 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:223524 2.c e189/126 2:3ad6268a:::benchmark_data_smithi166_20551_object75904:head v 189'13910, mlcod=189'13906)", 2020-09-08T15:47:30.702 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:40:31.454160+0000", 2020-09-08T15:47:30.703 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 419.15238287300002, 2020-09-08T15:47:30.703 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.30000275999999998, 2020-09-08T15:47:30.703 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:47:30.703 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:30.703 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:47:30.704 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.704 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:47:30.704 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.454160+0000", 2020-09-08T15:47:30.704 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:47:30.704 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.704 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.705 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:47:30.705 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.454160+0000", 2020-09-08T15:47:30.705 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2197000000000001e-05 2020-09-08T15:47:30.705 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.705 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.706 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:47:30.706 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.454222+0000", 2020-09-08T15:47:30.706 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2730000000000002e-06 2020-09-08T15:47:30.706 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.706 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.707 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:47:30.707 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.454230+0000", 2020-09-08T15:47:30.707 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3343000000000001e-05 2020-09-08T15:47:30.707 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.707 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.708 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:47:30.708 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.454243+0000", 2020-09-08T15:47:30.708 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.105e-06 2020-09-08T15:47:30.708 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.708 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.708 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:47:30.709 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.454249+0000", 2020-09-08T15:47:30.709 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5810999999999999e-05 2020-09-08T15:47:30.709 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.709 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.709 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:47:30.710 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.454295+0000", 2020-09-08T15:47:30.710 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3672e-05 2020-09-08T15:47:30.710 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.710 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.710 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:47:30.711 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.454319+0000", 2020-09-08T15:47:30.711 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29974217399999997 2020-09-08T15:47:30.711 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.711 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.711 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:47:30.712 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.754061+0000", 2020-09-08T15:47:30.712 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000101185 2020-09-08T15:47:30.712 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.713 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.714 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:47:30.714 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.754162+0000", 2020-09-08T15:47:30.714 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.30000275999999998 2020-09-08T15:47:30.714 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.714 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.715 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.715 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.715 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.715 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:223535 2.c e189/126 2:3b35efb1:::benchmark_data_smithi166_20551_object75915:head v 189'13911, mlcod=189'13908)", 2020-09-08T15:47:30.715 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:40:31.455185+0000", 2020-09-08T15:47:30.716 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 419.15135717200002, 2020-09-08T15:47:30.716 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29904997799999999, 2020-09-08T15:47:30.716 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:47:30.716 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:30.716 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:47:30.717 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.717 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:47:30.717 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455185+0000", 2020-09-08T15:47:30.717 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:47:30.717 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.717 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.718 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:47:30.718 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455185+0000", 2020-09-08T15:47:30.718 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7972000000000006e-05 2020-09-08T15:47:30.718 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.718 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.719 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:47:30.719 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455253+0000", 2020-09-08T15:47:30.719 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6969999999999999e-06 2020-09-08T15:47:30.719 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.719 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.720 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:47:30.720 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455262+0000", 2020-09-08T15:47:30.720 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4432e-05 2020-09-08T15:47:30.720 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.720 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.720 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:47:30.721 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455276+0000", 2020-09-08T15:47:30.721 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4259999999999996e-06 2020-09-08T15:47:30.721 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.721 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.721 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:47:30.722 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455281+0000", 2020-09-08T15:47:30.722 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0042721829999999997 2020-09-08T15:47:30.722 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.722 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.722 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:47:30.723 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.459553+0000", 2020-09-08T15:47:30.723 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6232999999999999e-05 2020-09-08T15:47:30.723 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.723 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.723 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:47:30.724 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.459569+0000", 2020-09-08T15:47:30.724 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29460175999999999 2020-09-08T15:47:30.724 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.724 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.724 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:47:30.724 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.754171+0000", 2020-09-08T15:47:30.725 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4275000000000004e-05 2020-09-08T15:47:30.725 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.725 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.725 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:47:30.725 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.754235+0000", 2020-09-08T15:47:30.726 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29904997799999999 2020-09-08T15:47:30.726 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.726 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.726 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.726 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.727 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.727 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:223538 2.5 e189/126 2:af9447da:::benchmark_data_smithi166_20551_object75918:head v 189'13833, mlcod=189'13831)", 2020-09-08T15:47:30.727 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:40:31.455706+0000", 2020-09-08T15:47:30.727 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 419.150836185, 2020-09-08T15:47:30.727 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.31563575399999999, 2020-09-08T15:47:30.728 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:47:30.728 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:30.728 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:47:30.728 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.728 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:47:30.729 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455706+0000", 2020-09-08T15:47:30.729 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:47:30.729 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.729 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.730 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:47:30.730 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455706+0000", 2020-09-08T15:47:30.730 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7342999999999995e-05 2020-09-08T15:47:30.730 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.731 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.731 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:47:30.731 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455774+0000", 2020-09-08T15:47:30.731 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6449999999999994e-06 2020-09-08T15:47:30.731 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.731 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.732 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:47:30.732 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455782+0000", 2020-09-08T15:47:30.732 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3685e-05 2020-09-08T15:47:30.732 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.732 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.733 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:47:30.733 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455796+0000", 2020-09-08T15:47:30.733 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3369999999999996e-06 2020-09-08T15:47:30.733 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.733 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.734 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:47:30.734 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455802+0000", 2020-09-08T15:47:30.734 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000106783 2020-09-08T15:47:30.734 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.734 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.735 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:47:30.735 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455909+0000", 2020-09-08T15:47:30.736 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.357e-05 2020-09-08T15:47:30.736 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.736 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.736 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:47:30.737 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.455933+0000", 2020-09-08T15:47:30.737 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.31528879999999998 2020-09-08T15:47:30.737 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.737 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.737 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:47:30.738 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.771221+0000", 2020-09-08T15:47:30.738 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000120591 2020-09-08T15:47:30.738 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.738 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.738 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:47:30.739 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.771342+0000", 2020-09-08T15:47:30.739 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.31563575399999999 2020-09-08T15:47:30.739 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.739 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.739 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.740 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.740 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.740 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.34742.0:223539 2.5 e189/126 2:ac3e761c:::benchmark_data_smithi166_20551_object75919:head v 189'13834, mlcod=189'13831)", 2020-09-08T15:47:30.740 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:40:31.461367+0000", 2020-09-08T15:47:30.740 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 419.14517568700001, 2020-09-08T15:47:30.740 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.31003701900000002, 2020-09-08T15:47:30.741 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:47:30.741 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:30.741 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:47:30.741 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.741 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:47:30.742 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.461367+0000", 2020-09-08T15:47:30.742 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:47:30.742 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.742 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.742 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:47:30.743 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.461367+0000", 2020-09-08T15:47:30.743 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4367000000000002e-05 2020-09-08T15:47:30.743 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.743 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.743 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:47:30.744 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.461431+0000", 2020-09-08T15:47:30.744 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1689999999999998e-06 2020-09-08T15:47:30.744 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.744 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.744 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:47:30.744 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.461440+0000", 2020-09-08T15:47:30.745 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3594000000000001e-05 2020-09-08T15:47:30.745 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.745 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.745 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:47:30.745 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.461454+0000", 2020-09-08T15:47:30.746 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8560000000000003e-06 2020-09-08T15:47:30.746 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.746 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.746 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:47:30.746 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.461460+0000", 2020-09-08T15:47:30.747 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29267275199999998 2020-09-08T15:47:30.747 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.747 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.747 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:47:30.747 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.754133+0000", 2020-09-08T15:47:30.748 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5941999999999999e-05 2020-09-08T15:47:30.748 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.748 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.748 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:47:30.748 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.754208+0000", 2020-09-08T15:47:30.748 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.017145581 2020-09-08T15:47:30.749 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.749 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.749 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:47:30.749 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.771354+0000", 2020-09-08T15:47:30.749 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9758e-05 2020-09-08T15:47:30.750 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.750 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.750 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:47:30.750 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:40:31.771404+0000", 2020-09-08T15:47:30.750 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.31003701900000002 2020-09-08T15:47:30.751 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.751 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.751 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.751 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.751 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.751 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:154541 3.6 3:6d1688dd:::benchmark_data_smithi166_26097_object30380:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e263)", 2020-09-08T15:47:30.752 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:36.806581+0000", 2020-09-08T15:47:30.752 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 173.799961059, 2020-09-08T15:47:30.752 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.25021634799999998, 2020-09-08T15:47:30.752 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:47:30.752 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:30.753 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:47:30.753 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:47:30.753 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:30.753 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 154541 2020-09-08T15:47:30.753 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.754 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:47:30.754 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.755 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:47:30.755 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:36.806581+0000", 2020-09-08T15:47:30.756 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:47:30.756 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.756 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.756 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:47:30.756 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:36.806581+0000", 2020-09-08T15:47:30.757 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999781 2020-09-08T15:47:30.757 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.757 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.757 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:47:30.757 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:36.806559+0000", 2020-09-08T15:47:30.758 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9808999999999997e-05 2020-09-08T15:47:30.758 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.758 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.758 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:47:30.758 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:36.806609+0000", 2020-09-08T15:47:30.758 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9230000000000008e-06 2020-09-08T15:47:30.759 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.759 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.759 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:47:30.759 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:36.806618+0000", 2020-09-08T15:47:30.759 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.216e-06 2020-09-08T15:47:30.760 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.760 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.760 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:47:30.760 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:36.806622+0000", 2020-09-08T15:47:30.760 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18533844299999999 2020-09-08T15:47:30.761 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.761 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.761 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:47:30.761 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:36.991961+0000", 2020-09-08T15:47:30.761 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012709700000000001 2020-09-08T15:47:30.762 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.762 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.762 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:47:30.762 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:36.992088+0000", 2020-09-08T15:47:30.762 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018045100000000001 2020-09-08T15:47:30.762 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.763 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.763 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:47:30.763 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:36.992268+0000", 2020-09-08T15:47:30.763 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013989474999999999 2020-09-08T15:47:30.763 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.764 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.764 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:47:30.764 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:37.006258+0000", 2020-09-08T15:47:30.764 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.047477640000000002 2020-09-08T15:47:30.764 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.765 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.765 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:47:30.765 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:37.053735+0000", 2020-09-08T15:47:30.765 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6804999999999994e-05 2020-09-08T15:47:30.765 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.765 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.766 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:47:30.766 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:37.053812+0000", 2020-09-08T15:47:30.766 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0029855609999999999 2020-09-08T15:47:30.766 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.766 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.767 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:47:30.767 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:37.056798+0000", 2020-09-08T15:47:30.767 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.25021634799999998 2020-09-08T15:47:30.767 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.767 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.768 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.768 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.768 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.768 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:159518 3.6 3:630d1e06:::benchmark_data_smithi166_26097_object35357:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e266)", 2020-09-08T15:47:30.768 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:45.532156+0000", 2020-09-08T15:47:30.769 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 165.074386121, 2020-09-08T15:47:30.769 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.27662112599999999, 2020-09-08T15:47:30.769 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:47:30.769 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:30.769 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:47:30.769 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:47:30.770 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:30.770 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 159518 2020-09-08T15:47:30.770 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.770 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:47:30.771 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.771 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:47:30.771 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.532156+0000", 2020-09-08T15:47:30.771 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:47:30.772 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.772 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.772 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:47:30.772 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.532156+0000", 2020-09-08T15:47:30.772 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999557 2020-09-08T15:47:30.772 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.773 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.773 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:47:30.773 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.532112+0000", 2020-09-08T15:47:30.773 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0600999999999995e-05 2020-09-08T15:47:30.773 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.774 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.774 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:47:30.774 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.532203+0000", 2020-09-08T15:47:30.774 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4331000000000001e-05 2020-09-08T15:47:30.774 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.775 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.775 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:47:30.775 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.532217+0000", 2020-09-08T15:47:30.775 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3280000000000002e-06 2020-09-08T15:47:30.775 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.776 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.776 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:47:30.776 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.532223+0000", 2020-09-08T15:47:30.776 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21526457099999999 2020-09-08T15:47:30.776 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.776 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.777 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:47:30.777 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.747488+0000", 2020-09-08T15:47:30.777 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012507100000000001 2020-09-08T15:47:30.777 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.777 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.778 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:47:30.778 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.747613+0000", 2020-09-08T15:47:30.778 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019435500000000001 2020-09-08T15:47:30.778 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.778 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.779 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:47:30.779 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.747807+0000", 2020-09-08T15:47:30.779 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.029396507999999998 2020-09-08T15:47:30.779 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.779 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.779 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:47:30.780 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.777204+0000", 2020-09-08T15:47:30.780 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.028517411999999999 2020-09-08T15:47:30.780 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.780 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.780 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:47:30.781 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.805721+0000", 2020-09-08T15:47:30.781 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9368000000000004e-05 2020-09-08T15:47:30.781 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.781 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.781 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:47:30.782 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.805790+0000", 2020-09-08T15:47:30.782 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0029869940000000002 2020-09-08T15:47:30.782 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.782 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.782 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:47:30.783 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.808777+0000", 2020-09-08T15:47:30.783 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.27662112599999999 2020-09-08T15:47:30.783 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.783 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.783 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.783 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.784 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.784 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:159584 3.6 3:6026c71b:::benchmark_data_smithi166_26097_object35423:head [delete] snapc 0=[] ondisk+write+known_if_redirected e266)", 2020-09-08T15:47:30.784 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:45.537206+0000", 2020-09-08T15:47:30.784 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 165.06933659000001, 2020-09-08T15:47:30.784 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.27456241100000001, 2020-09-08T15:47:30.785 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:47:30.785 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:30.785 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:47:30.785 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:47:30.785 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:30.786 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 159584 2020-09-08T15:47:30.786 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.786 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:47:30.786 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.786 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:47:30.787 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.537206+0000", 2020-09-08T15:47:30.787 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:47:30.787 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.787 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.787 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:47:30.787 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.537206+0000", 2020-09-08T15:47:30.788 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999633 2020-09-08T15:47:30.790 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.790 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.790 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:47:30.791 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.537169+0000", 2020-09-08T15:47:30.791 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2824000000000005e-05 2020-09-08T15:47:30.791 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.791 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.792 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:47:30.792 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.537252+0000", 2020-09-08T15:47:30.792 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3548e-05 2020-09-08T15:47:30.792 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.793 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.793 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:47:30.793 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.537266+0000", 2020-09-08T15:47:30.793 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4180000000000002e-06 2020-09-08T15:47:30.794 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.794 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.794 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:47:30.794 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.537272+0000", 2020-09-08T15:47:30.794 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21571037300000001 2020-09-08T15:47:30.795 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.795 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.795 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:47:30.795 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.752982+0000", 2020-09-08T15:47:30.795 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010925900000000001 2020-09-08T15:47:30.796 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.796 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.796 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:47:30.796 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.753092+0000", 2020-09-08T15:47:30.796 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012448599999999999 2020-09-08T15:47:30.797 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.797 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.797 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:47:30.797 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.753216+0000", 2020-09-08T15:47:30.797 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.023998478 2020-09-08T15:47:30.798 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.798 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.798 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:47:30.798 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.777215+0000", 2020-09-08T15:47:30.798 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.031656517000000002 2020-09-08T15:47:30.799 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.799 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.799 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:47:30.799 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.808871+0000", 2020-09-08T15:47:30.799 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1524000000000001e-05 2020-09-08T15:47:30.800 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.800 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.800 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:47:30.800 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.808953+0000", 2020-09-08T15:47:30.800 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0028157099999999999 2020-09-08T15:47:30.800 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.801 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.801 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:47:30.801 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.811768+0000", 2020-09-08T15:47:30.801 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.27456241100000001 2020-09-08T15:47:30.801 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.802 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.802 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.802 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.802 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.802 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:159544 3.6 3:6e79caa0:::benchmark_data_smithi166_26097_object35383:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e266)", 2020-09-08T15:47:30.803 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:45.547487+0000", 2020-09-08T15:47:30.803 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 165.05905551999999, 2020-09-08T15:47:30.803 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26711421800000001, 2020-09-08T15:47:30.803 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:47:30.803 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:30.804 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:47:30.804 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:47:30.804 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:30.804 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 159544 2020-09-08T15:47:30.804 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.804 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:47:30.805 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.805 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:47:30.805 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.547487+0000", 2020-09-08T15:47:30.805 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:47:30.805 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.806 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.806 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:47:30.806 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.547487+0000", 2020-09-08T15:47:30.806 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999781 2020-09-08T15:47:30.806 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.807 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.807 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:47:30.807 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.547465+0000", 2020-09-08T15:47:30.807 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9867000000000002e-05 2020-09-08T15:47:30.807 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.807 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.808 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:47:30.808 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.547515+0000", 2020-09-08T15:47:30.808 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8439999999999993e-06 2020-09-08T15:47:30.808 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.808 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.809 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:47:30.809 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.547525+0000", 2020-09-08T15:47:30.809 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3600000000000004e-06 2020-09-08T15:47:30.809 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.809 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.810 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:47:30.810 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.547530+0000", 2020-09-08T15:47:30.810 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21145528899999999 2020-09-08T15:47:30.810 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.810 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.811 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:47:30.811 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.758985+0000", 2020-09-08T15:47:30.811 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000184446 2020-09-08T15:47:30.811 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.811 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.812 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:47:30.812 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.759170+0000", 2020-09-08T15:47:30.812 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00024394799999999999 2020-09-08T15:47:30.812 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.812 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.812 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:47:30.813 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.759414+0000", 2020-09-08T15:47:30.813 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.023618573 2020-09-08T15:47:30.813 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.813 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.813 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:47:30.814 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.783032+0000", 2020-09-08T15:47:30.814 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.028794650000000001 2020-09-08T15:47:30.814 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.814 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.814 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:47:30.815 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.811827+0000", 2020-09-08T15:47:30.815 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0613999999999999e-05 2020-09-08T15:47:30.815 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.815 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.815 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:47:30.816 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.811888+0000", 2020-09-08T15:47:30.816 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0027136370000000001 2020-09-08T15:47:30.816 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.816 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.816 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:47:30.816 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.814601+0000", 2020-09-08T15:47:30.817 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26711421800000001 2020-09-08T15:47:30.817 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.817 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.817 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.817 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.818 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.818 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:159589 3.6 3:6e217afc:::benchmark_data_smithi166_26097_object35428:head [delete] snapc 0=[] ondisk+write+known_if_redirected e266)", 2020-09-08T15:47:30.818 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:45.550419+0000", 2020-09-08T15:47:30.818 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 165.056123523, 2020-09-08T15:47:30.818 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26680337999999998, 2020-09-08T15:47:30.819 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:47:30.819 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:30.819 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:47:30.819 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:47:30.819 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:30.820 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 159589 2020-09-08T15:47:30.820 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.820 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:47:30.820 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.820 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:47:30.820 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.550419+0000", 2020-09-08T15:47:30.821 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:47:30.821 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.821 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.821 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:47:30.821 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.550419+0000", 2020-09-08T15:47:30.822 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999762 2020-09-08T15:47:30.822 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.822 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.822 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:47:30.822 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.550395+0000", 2020-09-08T15:47:30.823 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2611999999999997e-05 2020-09-08T15:47:30.823 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.823 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.823 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:47:30.823 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.550448+0000", 2020-09-08T15:47:30.823 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0429999999999996e-06 2020-09-08T15:47:30.824 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.824 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.824 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:47:30.824 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.550457+0000", 2020-09-08T15:47:30.824 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5759999999999999e-06 2020-09-08T15:47:30.825 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.825 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.825 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:47:30.825 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.550461+0000", 2020-09-08T15:47:30.825 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.215614627 2020-09-08T15:47:30.826 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.826 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.826 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:47:30.826 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.766076+0000", 2020-09-08T15:47:30.826 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013089300000000001 2020-09-08T15:47:30.827 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.827 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.827 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:47:30.827 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.766207+0000", 2020-09-08T15:47:30.827 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000177202 2020-09-08T15:47:30.827 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.828 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.828 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:47:30.828 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.766384+0000", 2020-09-08T15:47:30.828 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.016659257 2020-09-08T15:47:30.828 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.829 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.829 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:47:30.829 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.783043+0000", 2020-09-08T15:47:30.830 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.031611454999999997 2020-09-08T15:47:30.831 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.831 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.831 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:47:30.832 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.814655+0000", 2020-09-08T15:47:30.832 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5247999999999998e-05 2020-09-08T15:47:30.832 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.832 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.832 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:47:30.833 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.814700+0000", 2020-09-08T15:47:30.833 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0025223730000000001 2020-09-08T15:47:30.834 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.834 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.834 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:47:30.834 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.817222+0000", 2020-09-08T15:47:30.834 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26680337999999998 2020-09-08T15:47:30.835 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.835 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.835 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.835 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.835 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.836 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:159591 3.6 3:6f2d1afc:::benchmark_data_smithi166_26097_object35430:head [delete] snapc 0=[] ondisk+write+known_if_redirected e266)", 2020-09-08T15:47:30.836 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:45.556028+0000", 2020-09-08T15:47:30.837 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 165.05051465400001, 2020-09-08T15:47:30.837 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26371128599999999, 2020-09-08T15:47:30.838 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:47:30.838 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:30.838 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:47:30.838 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:47:30.839 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:30.839 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 159591 2020-09-08T15:47:30.839 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.839 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:47:30.840 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.840 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:47:30.840 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.556028+0000", 2020-09-08T15:47:30.840 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:47:30.840 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.841 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.841 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:47:30.841 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.556028+0000", 2020-09-08T15:47:30.841 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999762 2020-09-08T15:47:30.841 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.842 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.842 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:47:30.842 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.556004+0000", 2020-09-08T15:47:30.842 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2824000000000001e-05 2020-09-08T15:47:30.843 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.843 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.843 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:47:30.843 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.556057+0000", 2020-09-08T15:47:30.843 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0103e-05 2020-09-08T15:47:30.844 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.844 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.844 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:47:30.844 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.556067+0000", 2020-09-08T15:47:30.844 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6940000000000003e-06 2020-09-08T15:47:30.845 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.845 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.845 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:47:30.845 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.556072+0000", 2020-09-08T15:47:30.845 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21557869800000001 2020-09-08T15:47:30.846 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.846 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.846 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:47:30.846 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.771650+0000", 2020-09-08T15:47:30.846 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000133487 2020-09-08T15:47:30.847 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.847 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.847 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:47:30.847 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.771784+0000", 2020-09-08T15:47:30.847 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018799800000000001 2020-09-08T15:47:30.848 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.848 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.848 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:47:30.848 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.771972+0000", 2020-09-08T15:47:30.848 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.011247484 2020-09-08T15:47:30.849 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.849 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.849 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:47:30.849 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.783219+0000", 2020-09-08T15:47:30.849 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.034083690999999999 2020-09-08T15:47:30.850 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.850 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.850 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:47:30.850 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.817303+0000", 2020-09-08T15:47:30.850 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3438999999999999e-05 2020-09-08T15:47:30.851 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.851 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.851 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:47:30.851 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.817377+0000", 2020-09-08T15:47:30.851 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0023625270000000001 2020-09-08T15:47:30.852 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.852 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.852 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:47:30.852 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.819739+0000", 2020-09-08T15:47:30.852 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26371128599999999 2020-09-08T15:47:30.853 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.853 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.853 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.853 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.853 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.854 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:159557 3.6 3:6ef60a98:::benchmark_data_smithi166_26097_object35396:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e266)", 2020-09-08T15:47:30.854 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:45.566679+0000", 2020-09-08T15:47:30.854 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 165.03986338199999, 2020-09-08T15:47:30.854 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26237816400000002, 2020-09-08T15:47:30.854 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:47:30.855 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:30.855 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:47:30.855 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:47:30.855 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:30.855 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 159557 2020-09-08T15:47:30.856 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.856 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:47:30.856 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.856 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:47:30.856 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.566679+0000", 2020-09-08T15:47:30.857 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:47:30.857 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.857 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.857 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:47:30.857 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.566679+0000", 2020-09-08T15:47:30.858 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999938 2020-09-08T15:47:30.858 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.858 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.858 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:47:30.858 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.566617+0000", 2020-09-08T15:47:30.859 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011448 2020-09-08T15:47:30.859 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.859 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.859 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:47:30.859 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.566731+0000", 2020-09-08T15:47:30.860 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4863e-05 2020-09-08T15:47:30.860 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.860 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.860 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:47:30.861 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.566746+0000", 2020-09-08T15:47:30.861 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4710000000000004e-06 2020-09-08T15:47:30.861 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.861 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.862 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:47:30.862 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.566753+0000", 2020-09-08T15:47:30.862 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.210512002 2020-09-08T15:47:30.862 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.862 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.863 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:47:30.863 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.777265+0000", 2020-09-08T15:47:30.863 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019517300000000001 2020-09-08T15:47:30.863 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.863 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.864 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:47:30.864 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.777460+0000", 2020-09-08T15:47:30.864 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012716999999999999 2020-09-08T15:47:30.864 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.864 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.865 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:47:30.865 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.777587+0000", 2020-09-08T15:47:30.865 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0056399199999999997 2020-09-08T15:47:30.865 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.865 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.866 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:47:30.866 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.783227+0000", 2020-09-08T15:47:30.866 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.043802909000000001 2020-09-08T15:47:30.866 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.866 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.867 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:47:30.867 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.827030+0000", 2020-09-08T15:47:30.867 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7509000000000001e-05 2020-09-08T15:47:30.867 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.867 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.868 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:47:30.868 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.827087+0000", 2020-09-08T15:47:30.868 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0019697650000000001 2020-09-08T15:47:30.868 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.868 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.869 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:47:30.869 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.829057+0000", 2020-09-08T15:47:30.869 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26237816400000002 2020-09-08T15:47:30.870 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.870 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.870 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.870 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.871 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.871 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:210004 3.1 3:8d06d744:::benchmark_data_smithi166_26097_object85843:head [delete] snapc 0=[] ondisk+write+known_if_redirected e286)", 2020-09-08T15:47:30.871 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:45:57.366855+0000", 2020-09-08T15:47:30.871 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 93.239687809000003, 2020-09-08T15:47:30.871 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.23795509100000001, 2020-09-08T15:47:30.872 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:47:30.872 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:30.872 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:47:30.872 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:47:30.873 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:30.873 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 210004 2020-09-08T15:47:30.873 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.873 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:47:30.873 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.874 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:47:30.874 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.366855+0000", 2020-09-08T15:47:30.874 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:47:30.874 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.874 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.875 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:47:30.875 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.366855+0000", 2020-09-08T15:47:30.875 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999585 2020-09-08T15:47:30.875 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.876 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.876 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:47:30.878 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.366813+0000", 2020-09-08T15:47:30.878 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3678000000000007e-05 2020-09-08T15:47:30.878 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.878 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.878 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:47:30.879 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.366907+0000", 2020-09-08T15:47:30.879 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5118e-05 2020-09-08T15:47:30.879 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.879 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.879 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:47:30.880 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.366922+0000", 2020-09-08T15:47:30.880 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6880000000000002e-06 2020-09-08T15:47:30.880 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.880 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.880 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:47:30.881 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.366929+0000", 2020-09-08T15:47:30.881 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0086118610000000002 2020-09-08T15:47:30.881 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.881 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.881 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:47:30.882 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.375541+0000", 2020-09-08T15:47:30.882 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010226999999999999 2020-09-08T15:47:30.882 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.882 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.882 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:47:30.883 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.375643+0000", 2020-09-08T15:47:30.883 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000106678 2020-09-08T15:47:30.883 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.883 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.883 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T15:47:30.883 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.375750+0000", 2020-09-08T15:47:30.884 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.224626567 2020-09-08T15:47:30.884 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.884 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.884 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:47:30.884 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.600376+0000", 2020-09-08T15:47:30.885 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2773000000000004e-05 2020-09-08T15:47:30.885 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.885 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.886 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:47:30.886 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.600439+0000", 2020-09-08T15:47:30.886 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8282000000000006e-05 2020-09-08T15:47:30.886 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.886 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.887 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:47:30.887 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.600527+0000", 2020-09-08T15:47:30.887 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0042825010000000002 2020-09-08T15:47:30.887 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.887 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.888 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:47:30.888 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.604810+0000", 2020-09-08T15:47:30.888 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.23795509100000001 2020-09-08T15:47:30.888 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.888 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.889 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.889 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.889 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.889 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:210010 3.1 3:87751333:::benchmark_data_smithi166_26097_object85849:head [delete] snapc 0=[] ondisk+write+known_if_redirected e286)", 2020-09-08T15:47:30.889 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:45:57.370481+0000", 2020-09-08T15:47:30.890 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 93.236061194000001, 2020-09-08T15:47:30.890 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.24020914400000001, 2020-09-08T15:47:30.890 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:47:30.890 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:47:30.890 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:47:30.891 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:47:30.891 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:47:30.891 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 210010 2020-09-08T15:47:30.891 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.891 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:47:30.892 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.892 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:47:30.892 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.370481+0000", 2020-09-08T15:47:30.892 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:47:30.892 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.893 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.893 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:47:30.893 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.370481+0000", 2020-09-08T15:47:30.893 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999628 2020-09-08T15:47:30.893 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.893 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.894 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:47:30.894 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.370444+0000", 2020-09-08T15:47:30.894 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6192000000000004e-05 2020-09-08T15:47:30.894 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.894 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.895 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:47:30.895 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.370530+0000", 2020-09-08T15:47:30.895 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.312e-05 2020-09-08T15:47:30.895 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.895 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.896 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:47:30.896 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.370544+0000", 2020-09-08T15:47:30.896 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.195e-06 2020-09-08T15:47:30.897 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.897 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.897 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:47:30.897 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.370550+0000", 2020-09-08T15:47:30.897 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010056516999999999 2020-09-08T15:47:30.898 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.898 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.898 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:47:30.898 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.380606+0000", 2020-09-08T15:47:30.898 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013948899999999999 2020-09-08T15:47:30.899 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.899 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.899 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:47:30.899 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.380746+0000", 2020-09-08T15:47:30.899 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014580600000000001 2020-09-08T15:47:30.900 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.900 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.900 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T15:47:30.900 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.380892+0000", 2020-09-08T15:47:30.900 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.22394421 2020-09-08T15:47:30.901 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.901 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.901 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:47:30.901 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.604836+0000", 2020-09-08T15:47:30.901 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0057373220000000004 2020-09-08T15:47:30.901 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.902 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.902 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:47:30.902 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.610573+0000", 2020-09-08T15:47:30.902 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5308000000000006e-05 2020-09-08T15:47:30.902 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.903 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.903 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:47:30.903 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.610638+0000", 2020-09-08T15:47:30.903 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1950999999999998e-05 2020-09-08T15:47:30.903 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:47:30.904 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:47:30.904 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:47:30.904 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.610690+0000", 2020-09-08T15:47:30.904 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.24020914400000001 2020-09-08T15:47:30.904 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.905 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.905 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.905 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:47:30.905 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:47:30.905 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:47:30.906 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:30.908 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:30 smithi166 bash[9446]: cluster 2020-09-08T15:47:29.705783+0000 mon.a (mon.0) 1338 : cluster [DBG] osdmap e322: 12 total, 12 up, 12 in 2020-09-08T15:47:30.908 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:47:30 smithi166 bash[16593]: debug 2020-09-08T15:47:30.500+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:30.908 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:47:30 smithi104 bash[15273]: debug 2020-09-08T15:47:30.253+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:31.014 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:31.121 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:31.132 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:30 smithi055 bash[9296]: cluster 2020-09-08T15:47:29.705783+0000 mon.a (mon.0) 1338 : cluster [DBG] osdmap e322: 12 total, 12 up, 12 in 2020-09-08T15:47:31.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:30 smithi104 bash[9203]: cluster 2020-09-08T15:47:29.705783+0000 mon.a (mon.0) 1338 : cluster [DBG] osdmap e322: 12 total, 12 up, 12 in 2020-09-08T15:47:31.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:47:31 smithi166 bash[18960]: debug 2020-09-08T15:47:31.012+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:31.382 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 8 15 25380 25365 198.113 173.312 0.00425768 0.00504414 2020-09-08T15:47:32.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:31 smithi166 bash[9446]: cluster 2020-09-08T15:47:29.750997+0000 mgr.x (mgr.24118) 605 : cluster [DBG] pgmap v843: 17 pgs: 17 active+clean; 898 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 112 MiB/s wr, 1.79k op/s 2020-09-08T15:47:32.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:31 smithi166 bash[9446]: cluster 2020-09-08T15:47:30.257463+0000 osd.8 (osd.8) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:32.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:31 smithi166 bash[9446]: cluster 2020-09-08T15:47:30.257555+0000 osd.8 (osd.8) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:32.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:31 smithi166 bash[9446]: cluster 2020-09-08T15:47:30.723022+0000 mon.a (mon.0) 1339 : cluster [DBG] osdmap e323: 12 total, 12 up, 12 in 2020-09-08T15:47:32.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:31 smithi055 bash[9296]: cluster 2020-09-08T15:47:29.750997+0000 mgr.x (mgr.24118) 605 : cluster [DBG] pgmap v843: 17 pgs: 17 active+clean; 898 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 112 MiB/s wr, 1.79k op/s 2020-09-08T15:47:32.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:31 smithi055 bash[9296]: cluster 2020-09-08T15:47:30.257463+0000 osd.8 (osd.8) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:32.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:31 smithi055 bash[9296]: cluster 2020-09-08T15:47:30.257555+0000 osd.8 (osd.8) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:32.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:31 smithi055 bash[9296]: cluster 2020-09-08T15:47:30.723022+0000 mon.a (mon.0) 1339 : cluster [DBG] osdmap e323: 12 total, 12 up, 12 in 2020-09-08T15:47:32.275 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:31 smithi104 bash[9203]: cluster 2020-09-08T15:47:29.750997+0000 mgr.x (mgr.24118) 605 : cluster [DBG] pgmap v843: 17 pgs: 17 active+clean; 898 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 112 MiB/s wr, 1.79k op/s 2020-09-08T15:47:32.275 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:31 smithi104 bash[9203]: cluster 2020-09-08T15:47:30.257463+0000 osd.8 (osd.8) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:32.275 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:31 smithi104 bash[9203]: cluster 2020-09-08T15:47:30.257555+0000 osd.8 (osd.8) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:32.276 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:31 smithi104 bash[9203]: cluster 2020-09-08T15:47:30.723022+0000 mon.a (mon.0) 1339 : cluster [DBG] osdmap e323: 12 total, 12 up, 12 in 2020-09-08T15:47:32.382 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 9 16 27762 27746 192.635 148.812 0.00639386 0.00518795 2020-09-08T15:47:32.983 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:33.086 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:33.097 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:32 smithi055 bash[9296]: cluster 2020-09-08T15:47:31.751463+0000 mgr.x (mgr.24118) 606 : cluster [DBG] pgmap v845: 17 pgs: 17 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 203 MiB/s wr, 3.24k op/s 2020-09-08T15:47:33.098 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:47:32 smithi055 bash[14301]: debug 2020-09-08T15:47:32.983+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:33.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:32 smithi166 bash[9446]: cluster 2020-09-08T15:47:31.751463+0000 mgr.x (mgr.24118) 606 : cluster [DBG] pgmap v845: 17 pgs: 17 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 203 MiB/s wr, 3.24k op/s 2020-09-08T15:47:33.123 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:32 smithi104 bash[9203]: cluster 2020-09-08T15:47:31.751463+0000 mgr.x (mgr.24118) 606 : cluster [DBG] pgmap v845: 17 pgs: 17 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 203 MiB/s wr, 3.24k op/s 2020-09-08T15:47:33.124 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2020-09-08T15:47:33.194 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:33.299 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:47:33.382 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 10 16 30614 30598 191.195 178.25 0.00840003 0.00522722 2020-09-08T15:47:33.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:47:33 smithi055 bash[16646]: debug 2020-09-08T15:47:33.195+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:34.158 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:47:34.382 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 11 15 33385 33370 189.563 173.25 0.00821469 0.00527213 2020-09-08T15:47:35.155 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:47:35.260 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:47:35.382 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 12 16 36042 36026 187.598 166 0.00557492 0.00532771 2020-09-08T15:47:35.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:47:35 smithi104 bash[19922]: debug 2020-09-08T15:47:35.157+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:35.436 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:47:35.453 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:47:35.489 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:47:35.522 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:47:35.539 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:35.567 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:47:35.605 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:47:35.641 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:47:35.653 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:35.684 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:47:35.699 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:35 smithi104 bash[9203]: cluster 2020-09-08T15:47:33.751894+0000 mgr.x (mgr.24118) 607 : cluster [DBG] pgmap v846: 17 pgs: 17 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 220 MiB/s wr, 3.52k op/s 2020-09-08T15:47:35.700 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:35 smithi104 bash[9203]: cluster 2020-09-08T15:47:34.706512+0000 mon.a (mon.0) 1340 : cluster [DBG] osdmap e324: 12 total, 12 up, 12 in 2020-09-08T15:47:35.700 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:35 smithi104 bash[9203]: audit 2020-09-08T15:47:35.416117+0000 mon.c (mon.2) 52 : audit [DBG] from='client.? v1:172.21.15.104:0/2814919740' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:47:35.700 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:47:35 smithi104 bash[22305]: debug 2020-09-08T15:47:35.433+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:35.713 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:47:35.747 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:47:35.758 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:35.786 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:47:35.829 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:47:35.866 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:47:35.871 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:47:35.871 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:47:35.871 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:47:35.871 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:47:35.872 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:47:35.872 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:47:35.872 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:47:35.872 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:47:35.872 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:47:35.873 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:47:35.873 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:47:35.873 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:47:35.880 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:35.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:35 smithi055 bash[9296]: cluster 2020-09-08T15:47:33.751894+0000 mgr.x (mgr.24118) 607 : cluster [DBG] pgmap v846: 17 pgs: 17 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 220 MiB/s wr, 3.52k op/s 2020-09-08T15:47:35.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:35 smithi055 bash[9296]: cluster 2020-09-08T15:47:34.706512+0000 mon.a (mon.0) 1340 : cluster [DBG] osdmap e324: 12 total, 12 up, 12 in 2020-09-08T15:47:35.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:35 smithi055 bash[9296]: audit 2020-09-08T15:47:35.416117+0000 mon.c (mon.2) 52 : audit [DBG] from='client.? v1:172.21.15.104:0/2814919740' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:47:35.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:47:35 smithi055 bash[14301]: debug 2020-09-08T15:47:35.651+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:35.984 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:47:36.095 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:47:36.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:47:35 smithi166 bash[18960]: debug 2020-09-08T15:47:35.884+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:36.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:35 smithi166 bash[9446]: cluster 2020-09-08T15:47:33.751894+0000 mgr.x (mgr.24118) 607 : cluster [DBG] pgmap v846: 17 pgs: 17 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 220 MiB/s wr, 3.52k op/s 2020-09-08T15:47:36.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:35 smithi166 bash[9446]: cluster 2020-09-08T15:47:34.706512+0000 mon.a (mon.0) 1340 : cluster [DBG] osdmap e324: 12 total, 12 up, 12 in 2020-09-08T15:47:36.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:35 smithi166 bash[9446]: audit 2020-09-08T15:47:35.416117+0000 mon.c (mon.2) 52 : audit [DBG] from='client.? v1:172.21.15.104:0/2814919740' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:47:36.199 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:36.315 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:36.383 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 13 16 38404 38388 184.522 147.625 0.00310372 0.00539724 2020-09-08T15:47:36.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:47:36 smithi055 bash[19003]: debug 2020-09-08T15:47:36.095+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:36.419 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:36.550 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:47:36 smithi166 bash[16593]: debug 2020-09-08T15:47:36.316+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:36.551 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:47:36 smithi166 bash[18960]: debug 2020-09-08T15:47:36.552+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:36.551 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:36.656 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:36.770 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:36.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:47:36 smithi166 bash[18960]: debug 2020-09-08T15:47:36.768+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:36.875 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:36.992 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:37.097 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:37.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:47:36 smithi055 bash[16646]: debug 2020-09-08T15:47:36.991+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:37.261 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:37.364 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:37.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:47:37 smithi166 bash[18960]: debug 2020-09-08T15:47:37.260+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:37.383 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 14 16 40750 40734 181.815 146.625 0.00550887 0.00549652 2020-09-08T15:47:37.466 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:37.572 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:47:37.703 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:37 smithi166 bash[9446]: cluster 2020-09-08T15:47:35.752298+0000 mgr.x (mgr.24118) 608 : cluster [DBG] pgmap v848: 17 pgs: 17 active+clean; 1.7 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 140 MiB/s wr, 2.24k op/s 2020-09-08T15:47:37.703 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:47:37 smithi166 bash[14254]: debug 2020-09-08T15:47:37.468+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:37.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:37 smithi104 bash[9203]: cluster 2020-09-08T15:47:35.752298+0000 mgr.x (mgr.24118) 608 : cluster [DBG] pgmap v848: 17 pgs: 17 active+clean; 1.7 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 140 MiB/s wr, 2.24k op/s 2020-09-08T15:47:38.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:37 smithi055 bash[9296]: cluster 2020-09-08T15:47:35.752298+0000 mgr.x (mgr.24118) 608 : cluster [DBG] pgmap v848: 17 pgs: 17 active+clean; 1.7 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 140 MiB/s wr, 2.24k op/s 2020-09-08T15:47:38.386 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 15 16 43233 43217 180.039 155.188 0.00842503 0.00555191 2020-09-08T15:47:38.390 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:47:38.494 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:38.607 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:47:38 smithi104 bash[17588]: debug 2020-09-08T15:47:38.389+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:38.650 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:38.753 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:38.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:47:38 smithi055 bash[14301]: debug 2020-09-08T15:47:38.646+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:39.383 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 16 16 45206 45190 176.493 123.312 0.00530622 0.00566333 2020-09-08T15:47:39.707 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:47:40.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:39 smithi166 bash[9446]: cluster 2020-09-08T15:47:37.752859+0000 mgr.x (mgr.24118) 609 : cluster [DBG] pgmap v849: 17 pgs: 17 active+clean; 2.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 162 MiB/s wr, 2.60k op/s 2020-09-08T15:47:40.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:39 smithi166 bash[9446]: cluster 2020-09-08T15:47:38.460535+0000 osd.1 (osd.1) 58 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:40.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:39 smithi166 bash[9446]: cluster 2020-09-08T15:47:38.460635+0000 osd.1 (osd.1) 59 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:40.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:39 smithi166 bash[9446]: cluster 2020-09-08T15:47:38.719996+0000 mon.a (mon.0) 1341 : cluster [DBG] osdmap e325: 12 total, 12 up, 12 in 2020-09-08T15:47:40.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:39 smithi166 bash[9446]: audit 2020-09-08T15:47:39.445589+0000 mon.a (mon.0) 1342 : audit [INF] from='client.? v1:172.21.15.104:0/4075903659' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:47:40.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:39 smithi166 bash[9446]: cluster 2020-09-08T15:47:39.703426+0000 mon.a (mon.0) 1343 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:47:40.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:39 smithi166 bash[9446]: cluster 2020-09-08T15:47:39.703470+0000 mon.a (mon.0) 1344 : cluster [INF] Cluster is now healthy 2020-09-08T15:47:40.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:39 smithi166 bash[9446]: audit 2020-09-08T15:47:39.709468+0000 mon.a (mon.0) 1345 : audit [INF] from='client.? v1:172.21.15.104:0/4075903659' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:47:40.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:39 smithi166 bash[9446]: cluster 2020-09-08T15:47:39.709516+0000 mon.a (mon.0) 1346 : cluster [DBG] osdmap e326: 12 total, 12 up, 12 in 2020-09-08T15:47:40.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:39 smithi055 bash[9296]: cluster 2020-09-08T15:47:37.752859+0000 mgr.x (mgr.24118) 609 : cluster [DBG] pgmap v849: 17 pgs: 17 active+clean; 2.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 162 MiB/s wr, 2.60k op/s 2020-09-08T15:47:40.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:39 smithi055 bash[9296]: cluster 2020-09-08T15:47:38.460535+0000 osd.1 (osd.1) 58 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:40.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:39 smithi055 bash[9296]: cluster 2020-09-08T15:47:38.460635+0000 osd.1 (osd.1) 59 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:40.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:39 smithi055 bash[9296]: cluster 2020-09-08T15:47:38.719996+0000 mon.a (mon.0) 1341 : cluster [DBG] osdmap e325: 12 total, 12 up, 12 in 2020-09-08T15:47:40.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:39 smithi055 bash[9296]: audit 2020-09-08T15:47:39.445589+0000 mon.a (mon.0) 1342 : audit [INF] from='client.? v1:172.21.15.104:0/4075903659' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:47:40.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:39 smithi055 bash[9296]: cluster 2020-09-08T15:47:39.703426+0000 mon.a (mon.0) 1343 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:47:40.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:39 smithi055 bash[9296]: cluster 2020-09-08T15:47:39.703470+0000 mon.a (mon.0) 1344 : cluster [INF] Cluster is now healthy 2020-09-08T15:47:40.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:39 smithi055 bash[9296]: audit 2020-09-08T15:47:39.709468+0000 mon.a (mon.0) 1345 : audit [INF] from='client.? v1:172.21.15.104:0/4075903659' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:47:40.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:39 smithi055 bash[9296]: cluster 2020-09-08T15:47:39.709516+0000 mon.a (mon.0) 1346 : cluster [DBG] osdmap e326: 12 total, 12 up, 12 in 2020-09-08T15:47:40.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:39 smithi104 bash[9203]: cluster 2020-09-08T15:47:37.752859+0000 mgr.x (mgr.24118) 609 : cluster [DBG] pgmap v849: 17 pgs: 17 active+clean; 2.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 162 MiB/s wr, 2.60k op/s 2020-09-08T15:47:40.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:39 smithi104 bash[9203]: cluster 2020-09-08T15:47:38.460535+0000 osd.1 (osd.1) 58 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:40.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:39 smithi104 bash[9203]: cluster 2020-09-08T15:47:38.460635+0000 osd.1 (osd.1) 59 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:40.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:39 smithi104 bash[9203]: cluster 2020-09-08T15:47:38.719996+0000 mon.a (mon.0) 1341 : cluster [DBG] osdmap e325: 12 total, 12 up, 12 in 2020-09-08T15:47:40.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:39 smithi104 bash[9203]: audit 2020-09-08T15:47:39.445589+0000 mon.a (mon.0) 1342 : audit [INF] from='client.? v1:172.21.15.104:0/4075903659' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:47:40.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:39 smithi104 bash[9203]: cluster 2020-09-08T15:47:39.703426+0000 mon.a (mon.0) 1343 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:47:40.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:39 smithi104 bash[9203]: cluster 2020-09-08T15:47:39.703470+0000 mon.a (mon.0) 1344 : cluster [INF] Cluster is now healthy 2020-09-08T15:47:40.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:39 smithi104 bash[9203]: audit 2020-09-08T15:47:39.709468+0000 mon.a (mon.0) 1345 : audit [INF] from='client.? v1:172.21.15.104:0/4075903659' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:47:40.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:39 smithi104 bash[9203]: cluster 2020-09-08T15:47:39.709516+0000 mon.a (mon.0) 1346 : cluster [DBG] osdmap e326: 12 total, 12 up, 12 in 2020-09-08T15:47:40.359 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:47:40.383 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 17 16 47540 47524 174.691 145.875 0.00675629 0.00572227 2020-09-08T15:47:40.533 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:47:41.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:40 smithi166 bash[9446]: cluster 2020-09-08T15:47:39.753326+0000 mgr.x (mgr.24118) 610 : cluster [DBG] pgmap v852: 17 pgs: 17 active+clean; 2.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 133 MiB/s wr, 2.13k op/s 2020-09-08T15:47:41.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:40 smithi055 bash[9296]: cluster 2020-09-08T15:47:39.753326+0000 mgr.x (mgr.24118) 610 : cluster [DBG] pgmap v852: 17 pgs: 17 active+clean; 2.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 133 MiB/s wr, 2.13k op/s 2020-09-08T15:47:41.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:40 smithi104 bash[9203]: cluster 2020-09-08T15:47:39.753326+0000 mgr.x (mgr.24118) 610 : cluster [DBG] pgmap v852: 17 pgs: 17 active+clean; 2.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 133 MiB/s wr, 2.13k op/s 2020-09-08T15:47:41.383 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 18 16 49833 49817 172.948 143.312 0.0046808 0.00577878 2020-09-08T15:47:42.384 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 19 16 51676 51660 169.907 115.188 0.0041989 0.00588247 2020-09-08T15:47:43.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:42 smithi166 bash[9446]: cluster 2020-09-08T15:47:41.753879+0000 mgr.x (mgr.24118) 611 : cluster [DBG] pgmap v853: 17 pgs: 17 active+clean; 2.7 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 151 MiB/s wr, 2.41k op/s 2020-09-08T15:47:43.150 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:42 smithi104 bash[9203]: cluster 2020-09-08T15:47:41.753879+0000 mgr.x (mgr.24118) 611 : cluster [DBG] pgmap v853: 17 pgs: 17 active+clean; 2.7 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 151 MiB/s wr, 2.41k op/s 2020-09-08T15:47:43.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:42 smithi055 bash[9296]: cluster 2020-09-08T15:47:41.753879+0000 mgr.x (mgr.24118) 611 : cluster [DBG] pgmap v853: 17 pgs: 17 active+clean; 2.7 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 151 MiB/s wr, 2.41k op/s 2020-09-08T15:47:43.175 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:47:43.175 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:47:43.383 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 15:47:43.388352 min lat: 0.00163485 max lat: 0.224228 avg lat: 0.00594755 2020-09-08T15:47:43.384 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:47:43.384 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 20 16 53641 53625 167.552 122.812 0.00318336 0.00594755 2020-09-08T15:47:43.489 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:47:43.653 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:47:43.963 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:44.065 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:44.190 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:47:44.295 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:47:44.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:47:43 smithi166 bash[11928]: debug 2020-09-08T15:47:43.955+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:44.383 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 21 16 54551 54535 162.282 56.875 0.00305905 0.00615923 2020-09-08T15:47:44.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:47:44 smithi055 bash[14301]: debug 2020-09-08T15:47:44.190+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:44.854 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:47:44.958 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:45.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:44 smithi166 bash[9446]: cluster 2020-09-08T15:47:43.754723+0000 mgr.x (mgr.24118) 612 : cluster [DBG] pgmap v854: 17 pgs: 17 active+clean; 3.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 186 MiB/s wr, 2.98k op/s 2020-09-08T15:47:45.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:44 smithi166 bash[9446]: cluster 2020-09-08T15:47:43.859285+0000 osd.6 (osd.6) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:45.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:44 smithi166 bash[9446]: cluster 2020-09-08T15:47:43.859425+0000 osd.6 (osd.6) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:45.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:44 smithi055 bash[9296]: cluster 2020-09-08T15:47:43.754723+0000 mgr.x (mgr.24118) 612 : cluster [DBG] pgmap v854: 17 pgs: 17 active+clean; 3.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 186 MiB/s wr, 2.98k op/s 2020-09-08T15:47:45.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:44 smithi055 bash[9296]: cluster 2020-09-08T15:47:43.859285+0000 osd.6 (osd.6) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:45.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:44 smithi055 bash[9296]: cluster 2020-09-08T15:47:43.859425+0000 osd.6 (osd.6) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:45.177 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:47:44 smithi104 bash[22305]: debug 2020-09-08T15:47:44.853+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:45.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:44 smithi104 bash[9203]: cluster 2020-09-08T15:47:43.754723+0000 mgr.x (mgr.24118) 612 : cluster [DBG] pgmap v854: 17 pgs: 17 active+clean; 3.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 186 MiB/s wr, 2.98k op/s 2020-09-08T15:47:45.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:44 smithi104 bash[9203]: cluster 2020-09-08T15:47:43.859285+0000 osd.6 (osd.6) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:45.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:44 smithi104 bash[9203]: cluster 2020-09-08T15:47:43.859425+0000 osd.6 (osd.6) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:45.384 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 22 16 56506 56490 160.459 122.188 0.0037821 0.00622051 2020-09-08T15:47:45.721 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2020-09-08T15:47:45.891 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:47:46.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:45 smithi166 bash[9446]: cluster 2020-09-08T15:47:44.816617+0000 mon.a (mon.0) 1347 : cluster [DBG] osdmap e327: 12 total, 12 up, 12 in 2020-09-08T15:47:46.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:45 smithi055 bash[9296]: cluster 2020-09-08T15:47:44.816617+0000 mon.a (mon.0) 1347 : cluster [DBG] osdmap e327: 12 total, 12 up, 12 in 2020-09-08T15:47:46.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:45 smithi104 bash[9203]: cluster 2020-09-08T15:47:44.816617+0000 mon.a (mon.0) 1347 : cluster [DBG] osdmap e327: 12 total, 12 up, 12 in 2020-09-08T15:47:46.385 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 23 16 58005 57989 157.555 93.6875 0.00438027 0.00629532 2020-09-08T15:47:47.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:46 smithi166 bash[9446]: cluster 2020-09-08T15:47:45.315458+0000 osd.11 (osd.11) 39 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:47.123 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:46 smithi166 bash[9446]: cluster 2020-09-08T15:47:45.315550+0000 osd.11 (osd.11) 40 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:47.123 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:46 smithi166 bash[9446]: cluster 2020-09-08T15:47:45.755176+0000 mgr.x (mgr.24118) 613 : cluster [DBG] pgmap v856: 17 pgs: 17 active+clean; 3.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 MiB/s wr, 1.57k op/s 2020-09-08T15:47:47.124 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:46 smithi166 bash[9446]: cluster 2020-09-08T15:47:45.832027+0000 mon.a (mon.0) 1348 : cluster [DBG] osdmap e328: 12 total, 12 up, 12 in 2020-09-08T15:47:47.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:46 smithi055 bash[9296]: cluster 2020-09-08T15:47:45.315458+0000 osd.11 (osd.11) 39 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:47.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:46 smithi055 bash[9296]: cluster 2020-09-08T15:47:45.315550+0000 osd.11 (osd.11) 40 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:47.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:46 smithi055 bash[9296]: cluster 2020-09-08T15:47:45.755176+0000 mgr.x (mgr.24118) 613 : cluster [DBG] pgmap v856: 17 pgs: 17 active+clean; 3.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 MiB/s wr, 1.57k op/s 2020-09-08T15:47:47.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:46 smithi055 bash[9296]: cluster 2020-09-08T15:47:45.832027+0000 mon.a (mon.0) 1348 : cluster [DBG] osdmap e328: 12 total, 12 up, 12 in 2020-09-08T15:47:47.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:46 smithi104 bash[9203]: cluster 2020-09-08T15:47:45.315458+0000 osd.11 (osd.11) 39 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:47.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:46 smithi104 bash[9203]: cluster 2020-09-08T15:47:45.315550+0000 osd.11 (osd.11) 40 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:47.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:46 smithi104 bash[9203]: cluster 2020-09-08T15:47:45.755176+0000 mgr.x (mgr.24118) 613 : cluster [DBG] pgmap v856: 17 pgs: 17 active+clean; 3.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 MiB/s wr, 1.57k op/s 2020-09-08T15:47:47.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:46 smithi104 bash[9203]: cluster 2020-09-08T15:47:45.832027+0000 mon.a (mon.0) 1348 : cluster [DBG] osdmap e328: 12 total, 12 up, 12 in 2020-09-08T15:47:47.305 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:47.394 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 24 16 59933 59917 156.011 120.5 0.00504012 0.00640737 2020-09-08T15:47:47.410 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:47.523 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:47.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:47:47 smithi166 bash[18960]: debug 2020-09-08T15:47:47.307+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:47.628 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:47:47.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:47:47 smithi055 bash[16646]: debug 2020-09-08T15:47:47.522+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:47.935 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:47:48.038 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:48.164 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:48.214 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:47:47 smithi104 bash[17588]: debug 2020-09-08T15:47:47.933+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:48.267 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:48.366 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:47:48 smithi166 bash[16593]: debug 2020-09-08T15:47:48.163+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:48.384 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 25 16 61749 61733 154.31 113.5 0.023627 0.00647547 2020-09-08T15:47:48.388 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:47:48.399 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:48.423 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:47:48.456 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:47:48.490 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:47:48.507 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:47:48.525 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:47:48.561 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:47:48.593 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:47:48.621 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:47:48.625 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:47:48.645 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:47:48.672 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:47:48.694 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:47:48.731 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:48.734 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:47:48.737 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:47:48.737 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:47:48.737 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:47:48.738 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:47:48.738 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:47:48.738 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:47:48.738 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:47:48.738 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:47:48.739 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:47:48.739 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:47:48.739 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:47:48.739 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:47:48.744 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:47:48 smithi166 bash[11928]: debug 2020-09-08T15:47:48.399+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:48.827 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:47:48 smithi055 bash[19003]: debug 2020-09-08T15:47:48.626+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:48.827 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:48 smithi055 bash[9296]: cluster 2020-09-08T15:47:47.755603+0000 mgr.x (mgr.24118) 614 : cluster [DBG] pgmap v858: 17 pgs: 17 active+clean; 3.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 122 MiB/s wr, 1.95k op/s 2020-09-08T15:47:48.828 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:48 smithi055 bash[9296]: audit 2020-09-08T15:47:48.364880+0000 mon.a (mon.0) 1349 : audit [DBG] from='client.? v1:172.21.15.104:0/841806752' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:47:48.857 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:48.964 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:49.083 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:49.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:48 smithi166 bash[9446]: cluster 2020-09-08T15:47:47.755603+0000 mgr.x (mgr.24118) 614 : cluster [DBG] pgmap v858: 17 pgs: 17 active+clean; 3.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 122 MiB/s wr, 1.95k op/s 2020-09-08T15:47:49.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:48 smithi166 bash[9446]: audit 2020-09-08T15:47:48.364880+0000 mon.a (mon.0) 1349 : audit [DBG] from='client.? v1:172.21.15.104:0/841806752' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:47:49.117 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:47:48 smithi166 bash[14254]: debug 2020-09-08T15:47:48.859+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:49.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:48 smithi104 bash[9203]: cluster 2020-09-08T15:47:47.755603+0000 mgr.x (mgr.24118) 614 : cluster [DBG] pgmap v858: 17 pgs: 17 active+clean; 3.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 122 MiB/s wr, 1.95k op/s 2020-09-08T15:47:49.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:48 smithi104 bash[9203]: audit 2020-09-08T15:47:48.364880+0000 mon.a (mon.0) 1349 : audit [DBG] from='client.? v1:172.21.15.104:0/841806752' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:47:49.187 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:49.304 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:49.384 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 26 16 63351 63335 152.226 100.125 0.00711802 0.00656701 2020-09-08T15:47:49.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:47:49 smithi055 bash[16646]: debug 2020-09-08T15:47:49.082+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:49.408 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:49.529 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:47:49 smithi166 bash[16593]: debug 2020-09-08T15:47:49.307+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:49.530 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:49.634 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:47:49.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:47:49 smithi166 bash[16593]: debug 2020-09-08T15:47:49.531+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:50.384 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 27 16 65205 65189 150.879 115.875 0.00521241 0.00662412 2020-09-08T15:47:50.869 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:47:50.973 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:51.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:50 smithi166 bash[9446]: cluster 2020-09-08T15:47:49.482183+0000 osd.4 (osd.4) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:51.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:50 smithi166 bash[9446]: cluster 2020-09-08T15:47:49.482270+0000 osd.4 (osd.4) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:51.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:50 smithi166 bash[9446]: cluster 2020-09-08T15:47:49.756073+0000 mgr.x (mgr.24118) 615 : cluster [DBG] pgmap v859: 17 pgs: 17 active+clean; 3.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 MiB/s wr, 1.87k op/s 2020-09-08T15:47:51.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:50 smithi166 bash[9446]: cluster 2020-09-08T15:47:49.837439+0000 mon.a (mon.0) 1350 : cluster [DBG] osdmap e329: 12 total, 12 up, 12 in 2020-09-08T15:47:51.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:50 smithi055 bash[9296]: cluster 2020-09-08T15:47:49.482183+0000 osd.4 (osd.4) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:51.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:50 smithi055 bash[9296]: cluster 2020-09-08T15:47:49.482270+0000 osd.4 (osd.4) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:51.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:50 smithi055 bash[9296]: cluster 2020-09-08T15:47:49.756073+0000 mgr.x (mgr.24118) 615 : cluster [DBG] pgmap v859: 17 pgs: 17 active+clean; 3.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 MiB/s wr, 1.87k op/s 2020-09-08T15:47:51.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:50 smithi055 bash[9296]: cluster 2020-09-08T15:47:49.837439+0000 mon.a (mon.0) 1350 : cluster [DBG] osdmap e329: 12 total, 12 up, 12 in 2020-09-08T15:47:51.190 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:47:50 smithi104 bash[19922]: debug 2020-09-08T15:47:50.869+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:51.191 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:50 smithi104 bash[9203]: cluster 2020-09-08T15:47:49.482183+0000 osd.4 (osd.4) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:51.191 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:50 smithi104 bash[9203]: cluster 2020-09-08T15:47:49.482270+0000 osd.4 (osd.4) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:51.191 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:50 smithi104 bash[9203]: cluster 2020-09-08T15:47:49.756073+0000 mgr.x (mgr.24118) 615 : cluster [DBG] pgmap v859: 17 pgs: 17 active+clean; 3.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 MiB/s wr, 1.87k op/s 2020-09-08T15:47:51.191 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:50 smithi104 bash[9203]: cluster 2020-09-08T15:47:49.837439+0000 mon.a (mon.0) 1350 : cluster [DBG] osdmap e329: 12 total, 12 up, 12 in 2020-09-08T15:47:51.384 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 28 16 66397 66381 148.151 74.5 0.00987368 0.00674754 2020-09-08T15:47:51.621 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:51.725 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:47:51.861 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:47:51.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:47:51 smithi104 bash[17588]: debug 2020-09-08T15:47:51.853+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:51.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:47:51 smithi104 bash[15273]: debug 2020-09-08T15:47:51.617+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:51.969 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:52.116 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:52.222 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:47:52.365 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:47:52.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:47:52 smithi166 bash[11928]: debug 2020-09-08T15:47:52.115+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:52.384 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 29 16 68263 68247 147.064 116.625 0.0119966 0.00679745 2020-09-08T15:47:52.469 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:52.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:47:52 smithi055 bash[19003]: debug 2020-09-08T15:47:52.362+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:52.822 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:52.926 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:53.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:52 smithi166 bash[9446]: cluster 2020-09-08T15:47:51.756676+0000 mgr.x (mgr.24118) 616 : cluster [DBG] pgmap v861: 17 pgs: 17 active+clean; 3.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 101 MiB/s wr, 1.62k op/s 2020-09-08T15:47:53.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:52 smithi055 bash[9296]: cluster 2020-09-08T15:47:51.756676+0000 mgr.x (mgr.24118) 616 : cluster [DBG] pgmap v861: 17 pgs: 17 active+clean; 3.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 101 MiB/s wr, 1.62k op/s 2020-09-08T15:47:53.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:52 smithi104 bash[9203]: cluster 2020-09-08T15:47:51.756676+0000 mgr.x (mgr.24118) 616 : cluster [DBG] pgmap v861: 17 pgs: 17 active+clean; 3.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 101 MiB/s wr, 1.62k op/s 2020-09-08T15:47:53.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:47:52 smithi104 bash[15273]: debug 2020-09-08T15:47:52.821+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:53.384 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 30 16 70112 70096 146.014 115.562 0.00912366 0.00684657 2020-09-08T15:47:53.712 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:53.722 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:47:53.816 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:53.906 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:47:53.948 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:54.052 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:47:54.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:47:53 smithi055 bash[11956]: debug 2020-09-08T15:47:53.950+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:54.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:47:53 smithi104 bash[15273]: debug 2020-09-08T15:47:53.713+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:54.177 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:47:54.284 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:54.384 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 31 16 71547 71531 144.196 89.6875 0.0152148 0.00693178 2020-09-08T15:47:54.417 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:47:54.521 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:54.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:47:54 smithi166 bash[18960]: debug 2020-09-08T15:47:54.419+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:54.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:47:54 smithi055 bash[19003]: debug 2020-09-08T15:47:54.178+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:54.660 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:54.709 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:47:54.765 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:54.861 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:54.968 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:47:55.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:47:54 smithi166 bash[11928]: debug 2020-09-08T15:47:54.663+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:55.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:54 smithi166 bash[9446]: cluster 2020-09-08T15:47:53.757326+0000 mgr.x (mgr.24118) 617 : cluster [DBG] pgmap v862: 17 pgs: 17 active+clean; 4.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 MiB/s wr, 2.05k op/s 2020-09-08T15:47:55.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:54 smithi166 bash[9446]: audit 2020-09-08T15:47:54.388361+0000 mon.a (mon.0) 1351 : audit [INF] from='client.? v1:172.21.15.104:0/644889313' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:47:55.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:54 smithi166 bash[9446]: cluster 2020-09-08T15:47:54.707413+0000 mon.a (mon.0) 1352 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:47:55.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:54 smithi166 bash[9446]: audit 2020-09-08T15:47:54.711902+0000 mon.a (mon.0) 1353 : audit [INF] from='client.? v1:172.21.15.104:0/644889313' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:47:55.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:54 smithi166 bash[9446]: cluster 2020-09-08T15:47:54.711969+0000 mon.a (mon.0) 1354 : cluster [DBG] osdmap e330: 12 total, 12 up, 12 in 2020-09-08T15:47:55.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:47:54 smithi055 bash[16646]: debug 2020-09-08T15:47:54.862+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:55.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:54 smithi055 bash[9296]: cluster 2020-09-08T15:47:53.757326+0000 mgr.x (mgr.24118) 617 : cluster [DBG] pgmap v862: 17 pgs: 17 active+clean; 4.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 MiB/s wr, 2.05k op/s 2020-09-08T15:47:55.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:54 smithi055 bash[9296]: audit 2020-09-08T15:47:54.388361+0000 mon.a (mon.0) 1351 : audit [INF] from='client.? v1:172.21.15.104:0/644889313' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:47:55.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:54 smithi055 bash[9296]: cluster 2020-09-08T15:47:54.707413+0000 mon.a (mon.0) 1352 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:47:55.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:54 smithi055 bash[9296]: audit 2020-09-08T15:47:54.711902+0000 mon.a (mon.0) 1353 : audit [INF] from='client.? v1:172.21.15.104:0/644889313' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:47:55.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:54 smithi055 bash[9296]: cluster 2020-09-08T15:47:54.711969+0000 mon.a (mon.0) 1354 : cluster [DBG] osdmap e330: 12 total, 12 up, 12 in 2020-09-08T15:47:55.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:54 smithi104 bash[9203]: cluster 2020-09-08T15:47:53.757326+0000 mgr.x (mgr.24118) 617 : cluster [DBG] pgmap v862: 17 pgs: 17 active+clean; 4.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 MiB/s wr, 2.05k op/s 2020-09-08T15:47:55.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:54 smithi104 bash[9203]: audit 2020-09-08T15:47:54.388361+0000 mon.a (mon.0) 1351 : audit [INF] from='client.? v1:172.21.15.104:0/644889313' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:47:55.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:54 smithi104 bash[9203]: cluster 2020-09-08T15:47:54.707413+0000 mon.a (mon.0) 1352 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:47:55.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:54 smithi104 bash[9203]: audit 2020-09-08T15:47:54.711902+0000 mon.a (mon.0) 1353 : audit [INF] from='client.? v1:172.21.15.104:0/644889313' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:47:55.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:54 smithi104 bash[9203]: cluster 2020-09-08T15:47:54.711969+0000 mon.a (mon.0) 1354 : cluster [DBG] osdmap e330: 12 total, 12 up, 12 in 2020-09-08T15:47:55.384 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 32 16 73241 73225 142.999 105.875 0.0120669 0.00699071 2020-09-08T15:47:55.630 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:47:55.734 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:47:55.905 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:47:55.905 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T15:47:55.906 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2020-09-08T15:47:55.906 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2020-09-08T15:47:55.906 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:47:55.931 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:47:55 smithi104 bash[17588]: debug 2020-09-08T15:47:55.632+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:56.332 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:47:56.386 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 33 15 74999 74984 141.997 109.938 0.00919127 0.00704026 2020-09-08T15:47:56.436 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:56.585 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:47:56 smithi104 bash[22305]: debug 2020-09-08T15:47:56.332+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:56.595 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:47:56.706 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:56.846 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:47:56 smithi055 bash[16646]: debug 2020-09-08T15:47:56.594+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:56.846 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:56 smithi055 bash[9296]: cluster 2020-09-08T15:47:55.757795+0000 mgr.x (mgr.24118) 618 : cluster [DBG] pgmap v864: 17 pgs: 17 active+clean; 4.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.48k op/s 2020-09-08T15:47:56.878 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:47:56.981 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:56.996 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:47:56 smithi166 bash[11928]: debug 2020-09-08T15:47:56.879+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:56.997 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:56 smithi166 bash[9446]: cluster 2020-09-08T15:47:55.757795+0000 mgr.x (mgr.24118) 618 : cluster [DBG] pgmap v864: 17 pgs: 17 active+clean; 4.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.48k op/s 2020-09-08T15:47:57.099 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:47:57.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:56 smithi104 bash[9203]: cluster 2020-09-08T15:47:55.757795+0000 mgr.x (mgr.24118) 618 : cluster [DBG] pgmap v864: 17 pgs: 17 active+clean; 4.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.48k op/s 2020-09-08T15:47:57.205 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:57.368 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:47:57 smithi166 bash[16593]: debug 2020-09-08T15:47:57.099+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:57.385 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 34 16 76571 76555 140.708 98.1875 0.0108318 0.0071049 2020-09-08T15:47:57.430 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:47:57.472 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:47:57 smithi104 bash[15273]: debug 2020-09-08T15:47:57.428+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:57.534 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:57.557 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2020-09-08T15:47:57.645 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:57.730 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:47:57.754 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:47:57.909 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:47:57 smithi055 bash[11956]: debug 2020-09-08T15:47:57.646+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:58.385 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 35 16 78019 78003 139.273 90.5 0.00922515 0.0071771 2020-09-08T15:47:58.578 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:47:58.586 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:47:58.683 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:47:58.759 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:47:58.853 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:47:58 smithi104 bash[19922]: debug 2020-09-08T15:47:58.580+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:59.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:58 smithi166 bash[9446]: cluster 2020-09-08T15:47:57.758314+0000 mgr.x (mgr.24118) 619 : cluster [DBG] pgmap v865: 17 pgs: 17 active+clean; 4.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 102 MiB/s wr, 1.63k op/s 2020-09-08T15:47:59.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:58 smithi055 bash[9296]: cluster 2020-09-08T15:47:57.758314+0000 mgr.x (mgr.24118) 619 : cluster [DBG] pgmap v865: 17 pgs: 17 active+clean; 4.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 102 MiB/s wr, 1.63k op/s 2020-09-08T15:47:59.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:58 smithi104 bash[9203]: cluster 2020-09-08T15:47:57.758314+0000 mgr.x (mgr.24118) 619 : cluster [DBG] pgmap v865: 17 pgs: 17 active+clean; 4.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 102 MiB/s wr, 1.63k op/s 2020-09-08T15:47:59.385 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 36 16 79719 79703 138.356 106.25 0.00659882 0.00722522 2020-09-08T15:47:59.405 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:47:59.507 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:59.633 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:47:59.687 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:47:59 smithi104 bash[22305]: debug 2020-09-08T15:47:59.400+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:47:59.739 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:47:59.855 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:59 smithi166 bash[9446]: cluster 2020-09-08T15:47:58.082303+0000 osd.9 (osd.9) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:47:59.856 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:59 smithi166 bash[9446]: cluster 2020-09-08T15:47:58.082386+0000 osd.9 (osd.9) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:47:59.856 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:59 smithi166 bash[9446]: cluster 2020-09-08T15:47:58.856207+0000 mon.a (mon.0) 1355 : cluster [DBG] osdmap e331: 12 total, 12 up, 12 in 2020-09-08T15:47:59.856 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:47:59 smithi166 bash[9446]: cluster 2020-09-08T15:47:59.714476+0000 mon.a (mon.0) 1356 : cluster [DBG] osdmap e332: 12 total, 12 up, 12 in 2020-09-08T15:47:59.857 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:47:59 smithi166 bash[14254]: debug 2020-09-08T15:47:59.635+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:00.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:59 smithi055 bash[9296]: cluster 2020-09-08T15:47:58.082303+0000 osd.9 (osd.9) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:00.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:59 smithi055 bash[9296]: cluster 2020-09-08T15:47:58.082386+0000 osd.9 (osd.9) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:00.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:59 smithi055 bash[9296]: cluster 2020-09-08T15:47:58.856207+0000 mon.a (mon.0) 1355 : cluster [DBG] osdmap e331: 12 total, 12 up, 12 in 2020-09-08T15:48:00.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:47:59 smithi055 bash[9296]: cluster 2020-09-08T15:47:59.714476+0000 mon.a (mon.0) 1356 : cluster [DBG] osdmap e332: 12 total, 12 up, 12 in 2020-09-08T15:48:00.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:59 smithi104 bash[9203]: cluster 2020-09-08T15:47:58.082303+0000 osd.9 (osd.9) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:00.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:59 smithi104 bash[9203]: cluster 2020-09-08T15:47:58.082386+0000 osd.9 (osd.9) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:00.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:59 smithi104 bash[9203]: cluster 2020-09-08T15:47:58.856207+0000 mon.a (mon.0) 1355 : cluster [DBG] osdmap e331: 12 total, 12 up, 12 in 2020-09-08T15:48:00.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:47:59 smithi104 bash[9203]: cluster 2020-09-08T15:47:59.714476+0000 mon.a (mon.0) 1356 : cluster [DBG] osdmap e332: 12 total, 12 up, 12 in 2020-09-08T15:48:00.388 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 37 16 81011 80995 136.799 80.75 0.00960657 0.00730798 2020-09-08T15:48:01.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:00 smithi166 bash[9446]: cluster 2020-09-08T15:47:59.758775+0000 mgr.x (mgr.24118) 620 : cluster [DBG] pgmap v868: 17 pgs: 17 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s wr, 1.38k op/s 2020-09-08T15:48:01.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:00 smithi055 bash[9296]: cluster 2020-09-08T15:47:59.758775+0000 mgr.x (mgr.24118) 620 : cluster [DBG] pgmap v868: 17 pgs: 17 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s wr, 1.38k op/s 2020-09-08T15:48:01.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:00 smithi104 bash[9203]: cluster 2020-09-08T15:47:59.758775+0000 mgr.x (mgr.24118) 620 : cluster [DBG] pgmap v868: 17 pgs: 17 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s wr, 1.38k op/s 2020-09-08T15:48:01.385 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 38 16 82468 82452 135.595 91.0625 0.00478409 0.00737274 2020-09-08T15:48:02.131 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:01 smithi104 bash[9203]: cluster 2020-09-08T15:48:00.619694+0000 osd.2 (osd.2) 44 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:02.131 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:01 smithi104 bash[9203]: cluster 2020-09-08T15:48:00.619806+0000 osd.2 (osd.2) 45 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:02.131 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:01 smithi104 bash[9203]: cluster 2020-09-08T15:48:00.865295+0000 mon.a (mon.0) 1357 : cluster [DBG] osdmap e333: 12 total, 12 up, 12 in 2020-09-08T15:48:02.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:01 smithi055 bash[9296]: cluster 2020-09-08T15:48:00.619694+0000 osd.2 (osd.2) 44 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:02.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:01 smithi055 bash[9296]: cluster 2020-09-08T15:48:00.619806+0000 osd.2 (osd.2) 45 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:02.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:01 smithi055 bash[9296]: cluster 2020-09-08T15:48:00.865295+0000 mon.a (mon.0) 1357 : cluster [DBG] osdmap e333: 12 total, 12 up, 12 in 2020-09-08T15:48:02.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:01 smithi166 bash[9446]: cluster 2020-09-08T15:48:00.619694+0000 osd.2 (osd.2) 44 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:02.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:01 smithi166 bash[9446]: cluster 2020-09-08T15:48:00.619806+0000 osd.2 (osd.2) 45 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:02.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:01 smithi166 bash[9446]: cluster 2020-09-08T15:48:00.865295+0000 mon.a (mon.0) 1357 : cluster [DBG] osdmap e333: 12 total, 12 up, 12 in 2020-09-08T15:48:02.385 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 39 16 84064 84048 134.676 99.75 0.00504845 0.00742297 2020-09-08T15:48:02.568 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:48:02.600 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:48:02.634 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:48:02.669 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:48:02.707 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:48:02.746 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:48:02.782 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:48:02.810 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:48:02.848 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:48:02.891 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:48:02.923 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:48:02.924 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:48:02.955 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:48:02.965 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:48:02.965 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:48:02.965 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:48:02.966 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:48:02.966 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:48:02.966 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:48:02.966 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:48:02.966 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:48:02.967 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:48:02.967 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:48:02.967 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:48:02.967 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:48:03.030 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:48:03.147 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:02 smithi055 bash[9296]: cluster 2020-09-08T15:48:01.759300+0000 mgr.x (mgr.24118) 621 : cluster [DBG] pgmap v870: 17 pgs: 17 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 116 MiB/s wr, 1.86k op/s 2020-09-08T15:48:03.148 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:02 smithi055 bash[9296]: audit 2020-09-08T15:48:02.536848+0000 mon.a (mon.0) 1358 : audit [DBG] from='client.? v1:172.21.15.104:0/2823129675' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:48:03.148 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:48:02 smithi055 bash[11956]: debug 2020-09-08T15:48:02.922+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:03.148 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:48:03 smithi055 bash[11956]: debug 2020-09-08T15:48:03.146+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:03.149 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:48:03.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:02 smithi104 bash[9203]: cluster 2020-09-08T15:48:01.759300+0000 mgr.x (mgr.24118) 621 : cluster [DBG] pgmap v870: 17 pgs: 17 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 116 MiB/s wr, 1.86k op/s 2020-09-08T15:48:03.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:02 smithi104 bash[9203]: audit 2020-09-08T15:48:02.536848+0000 mon.a (mon.0) 1358 : audit [DBG] from='client.? v1:172.21.15.104:0/2823129675' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:48:03.251 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:03.261 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:02 smithi166 bash[9446]: cluster 2020-09-08T15:48:01.759300+0000 mgr.x (mgr.24118) 621 : cluster [DBG] pgmap v870: 17 pgs: 17 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 116 MiB/s wr, 1.86k op/s 2020-09-08T15:48:03.262 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:02 smithi166 bash[9446]: audit 2020-09-08T15:48:02.536848+0000 mon.a (mon.0) 1358 : audit [DBG] from='client.? v1:172.21.15.104:0/2823129675' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:48:03.386 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 15:48:03.390153 min lat: 0.00163485 max lat: 0.556499 avg lat: 0.00747493 2020-09-08T15:48:03.387 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:48:03.387 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 40 16 85619 85603 133.738 97.1875 0.00472395 0.00747493 2020-09-08T15:48:03.389 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:03.492 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:03.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:48:03 smithi166 bash[18960]: debug 2020-09-08T15:48:03.391+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:03.620 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:03.727 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:03.834 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:03.937 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:04.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:48:03 smithi166 bash[11928]: debug 2020-09-08T15:48:03.619+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:04.117 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:48:03 smithi166 bash[18960]: debug 2020-09-08T15:48:03.835+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:04.385 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 41 15 87119 87104 132.764 93.8125 0.0148043 0.00753019 2020-09-08T15:48:04.734 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:04.838 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:48:05.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:48:04 smithi104 bash[19922]: debug 2020-09-08T15:48:04.732+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:05.386 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 42 16 88622 88606 131.838 93.875 0.00799623 0.00758271 2020-09-08T15:48:05.611 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:48:05.718 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:05.882 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:05 smithi104 bash[9203]: cluster 2020-09-08T15:48:03.759775+0000 mgr.x (mgr.24118) 622 : cluster [DBG] pgmap v871: 17 pgs: 17 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 116 MiB/s wr, 1.85k op/s 2020-09-08T15:48:05.882 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:05 smithi104 bash[9203]: cluster 2020-09-08T15:48:04.714618+0000 mon.a (mon.0) 1359 : cluster [DBG] osdmap e334: 12 total, 12 up, 12 in 2020-09-08T15:48:05.882 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:48:05 smithi104 bash[22305]: debug 2020-09-08T15:48:05.608+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:05.883 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:05.984 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:05.993 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:05 smithi166 bash[9446]: cluster 2020-09-08T15:48:03.759775+0000 mgr.x (mgr.24118) 622 : cluster [DBG] pgmap v871: 17 pgs: 17 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 116 MiB/s wr, 1.85k op/s 2020-09-08T15:48:05.993 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:05 smithi166 bash[9446]: cluster 2020-09-08T15:48:04.714618+0000 mon.a (mon.0) 1359 : cluster [DBG] osdmap e334: 12 total, 12 up, 12 in 2020-09-08T15:48:06.128 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:06.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:05 smithi055 bash[9296]: cluster 2020-09-08T15:48:03.759775+0000 mgr.x (mgr.24118) 622 : cluster [DBG] pgmap v871: 17 pgs: 17 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 116 MiB/s wr, 1.85k op/s 2020-09-08T15:48:06.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:05 smithi055 bash[9296]: cluster 2020-09-08T15:48:04.714618+0000 mon.a (mon.0) 1359 : cluster [DBG] osdmap e334: 12 total, 12 up, 12 in 2020-09-08T15:48:06.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:48:05 smithi104 bash[19922]: debug 2020-09-08T15:48:05.876+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:06.232 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:48:06.344 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:48:06.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:48:06 smithi166 bash[18960]: debug 2020-09-08T15:48:06.131+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:06.385 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 43 16 90179 90163 131.035 97.3125 0.00820777 0.0076292 2020-09-08T15:48:06.445 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:06.657 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:48:06 smithi055 bash[11956]: debug 2020-09-08T15:48:06.338+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:06.695 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:06.800 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:06.809 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:48:06 smithi104 bash[19922]: debug 2020-09-08T15:48:06.692+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:06.953 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:48:06.972 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:07.085 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:48:07.155 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:48:07.179 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:48:06 smithi104 bash[19922]: debug 2020-09-08T15:48:06.968+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:07.203 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:48:07.306 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:07.385 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 44 16 91673 91657 130.179 93.375 0.00776818 0.00767929 2020-09-08T15:48:07.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:48:07 smithi055 bash[16646]: debug 2020-09-08T15:48:07.202+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:08.022 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:07 smithi104 bash[9203]: cluster 2020-09-08T15:48:05.760252+0000 mgr.x (mgr.24118) 623 : cluster [DBG] pgmap v873: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T15:48:08.067 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:08.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:07 smithi166 bash[9446]: cluster 2020-09-08T15:48:05.760252+0000 mgr.x (mgr.24118) 623 : cluster [DBG] pgmap v873: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T15:48:08.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:07 smithi055 bash[9296]: cluster 2020-09-08T15:48:05.760252+0000 mgr.x (mgr.24118) 623 : cluster [DBG] pgmap v873: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T15:48:08.171 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:08.213 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:48:08.213 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:48:08.353 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:08.385 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 45 16 93167 93151 129.361 93.375 0.00678017 0.00772807 2020-09-08T15:48:08.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:48:08 smithi104 bash[19922]: debug 2020-09-08T15:48:08.064+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:08.463 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:48:08.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:48:08 smithi166 bash[18960]: debug 2020-09-08T15:48:08.355+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:08.731 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:48:08.803 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:08 smithi104 bash[9203]: audit 2020-09-08T15:48:08.508449+0000 mon.b (mon.1) 87 : audit [INF] from='client.? v1:172.21.15.104:0/407843346' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:48:08.803 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:08 smithi104 bash[9203]: audit 2020-09-08T15:48:08.509048+0000 mon.a (mon.0) 1360 : audit [INF] from='client.25405 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:48:09.017 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:48:09.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:08 smithi166 bash[9446]: audit 2020-09-08T15:48:08.508449+0000 mon.b (mon.1) 87 : audit [INF] from='client.? v1:172.21.15.104:0/407843346' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:48:09.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:08 smithi166 bash[9446]: audit 2020-09-08T15:48:08.509048+0000 mon.a (mon.0) 1360 : audit [INF] from='client.25405 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:48:09.120 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:48:09.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:08 smithi055 bash[9296]: audit 2020-09-08T15:48:08.508449+0000 mon.b (mon.1) 87 : audit [INF] from='client.? v1:172.21.15.104:0/407843346' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:48:09.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:08 smithi055 bash[9296]: audit 2020-09-08T15:48:08.509048+0000 mon.a (mon.0) 1360 : audit [INF] from='client.25405 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:48:09.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:48:09 smithi104 bash[15273]: debug 2020-09-08T15:48:09.016+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:09.229 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:48:09.336 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:48:09.386 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 46 16 94498 94482 128.357 83.1875 0.0170413 0.00778839 2020-09-08T15:48:09.447 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:48:09.554 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:48:09.562 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:48:09 smithi166 bash[16593]: debug 2020-09-08T15:48:09.231+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:09.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:48:09 smithi055 bash[16646]: debug 2020-09-08T15:48:09.450+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:09.678 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:48:09.783 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:09.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:48:09 smithi166 bash[16593]: debug 2020-09-08T15:48:09.679+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:09.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:09 smithi166 bash[9446]: cluster 2020-09-08T15:48:07.760816+0000 mgr.x (mgr.24118) 624 : cluster [DBG] pgmap v874: 17 pgs: 17 active+clean; 5.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 93 MiB/s wr, 1.49k op/s 2020-09-08T15:48:09.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:09 smithi166 bash[9446]: cluster 2020-09-08T15:48:08.718576+0000 mon.a (mon.0) 1361 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:48:09.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:09 smithi166 bash[9446]: audit 2020-09-08T15:48:08.730020+0000 mon.a (mon.0) 1362 : audit [INF] from='client.25405 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:48:09.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:09 smithi166 bash[9446]: cluster 2020-09-08T15:48:08.730068+0000 mon.a (mon.0) 1363 : cluster [DBG] osdmap e335: 12 total, 12 up, 12 in 2020-09-08T15:48:09.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:09 smithi166 bash[9446]: cluster 2020-09-08T15:48:08.920613+0000 osd.7 (osd.7) 63 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:09.878 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:09 smithi166 bash[9446]: cluster 2020-09-08T15:48:08.920735+0000 osd.7 (osd.7) 64 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:09.878 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:09 smithi166 bash[9446]: cluster 2020-09-08T15:48:09.715136+0000 mon.a (mon.0) 1364 : cluster [DBG] osdmap e336: 12 total, 12 up, 12 in 2020-09-08T15:48:10.082 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:09 smithi104 bash[9203]: cluster 2020-09-08T15:48:07.760816+0000 mgr.x (mgr.24118) 624 : cluster [DBG] pgmap v874: 17 pgs: 17 active+clean; 5.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 93 MiB/s wr, 1.49k op/s 2020-09-08T15:48:10.083 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:09 smithi104 bash[9203]: cluster 2020-09-08T15:48:08.718576+0000 mon.a (mon.0) 1361 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:48:10.083 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:09 smithi104 bash[9203]: audit 2020-09-08T15:48:08.730020+0000 mon.a (mon.0) 1362 : audit [INF] from='client.25405 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:48:10.083 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:09 smithi104 bash[9203]: cluster 2020-09-08T15:48:08.730068+0000 mon.a (mon.0) 1363 : cluster [DBG] osdmap e335: 12 total, 12 up, 12 in 2020-09-08T15:48:10.083 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:09 smithi104 bash[9203]: cluster 2020-09-08T15:48:08.920613+0000 osd.7 (osd.7) 63 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:10.084 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:09 smithi104 bash[9203]: cluster 2020-09-08T15:48:08.920735+0000 osd.7 (osd.7) 64 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:10.084 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:09 smithi104 bash[9203]: cluster 2020-09-08T15:48:09.715136+0000 mon.a (mon.0) 1364 : cluster [DBG] osdmap e336: 12 total, 12 up, 12 in 2020-09-08T15:48:10.084 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:10.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:09 smithi055 bash[9296]: cluster 2020-09-08T15:48:07.760816+0000 mgr.x (mgr.24118) 624 : cluster [DBG] pgmap v874: 17 pgs: 17 active+clean; 5.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 93 MiB/s wr, 1.49k op/s 2020-09-08T15:48:10.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:09 smithi055 bash[9296]: cluster 2020-09-08T15:48:08.718576+0000 mon.a (mon.0) 1361 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:48:10.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:09 smithi055 bash[9296]: audit 2020-09-08T15:48:08.730020+0000 mon.a (mon.0) 1362 : audit [INF] from='client.25405 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:48:10.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:09 smithi055 bash[9296]: cluster 2020-09-08T15:48:08.730068+0000 mon.a (mon.0) 1363 : cluster [DBG] osdmap e335: 12 total, 12 up, 12 in 2020-09-08T15:48:10.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:09 smithi055 bash[9296]: cluster 2020-09-08T15:48:08.920613+0000 osd.7 (osd.7) 63 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:10.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:09 smithi055 bash[9296]: cluster 2020-09-08T15:48:08.920735+0000 osd.7 (osd.7) 64 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:10.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:09 smithi055 bash[9296]: cluster 2020-09-08T15:48:09.715136+0000 mon.a (mon.0) 1364 : cluster [DBG] osdmap e336: 12 total, 12 up, 12 in 2020-09-08T15:48:10.186 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:10.312 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:10.349 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:48:10 smithi104 bash[19922]: debug 2020-09-08T15:48:10.080+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:10.350 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2020-09-08T15:48:10.386 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 47 16 95932 95916 127.533 89.625 0.00553988 0.00783902 2020-09-08T15:48:10.418 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:10.541 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:10.589 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:48:10.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:48:10 smithi166 bash[11928]: debug 2020-09-08T15:48:10.311+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:10.648 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:10.683 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:48:10 smithi104 bash[17588]: debug 2020-09-08T15:48:10.536+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:10.755 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:10.865 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:11.013 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:48:10 smithi166 bash[11928]: debug 2020-09-08T15:48:10.755+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:11.013 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:10 smithi166 bash[9446]: cluster 2020-09-08T15:48:09.761226+0000 mgr.x (mgr.24118) 625 : cluster [DBG] pgmap v877: 17 pgs: 17 active+clean; 5.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s wr, 1.27k op/s 2020-09-08T15:48:11.014 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:11.122 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:48:11.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:10 smithi055 bash[9296]: cluster 2020-09-08T15:48:09.761226+0000 mgr.x (mgr.24118) 625 : cluster [DBG] pgmap v877: 17 pgs: 17 active+clean; 5.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s wr, 1.27k op/s 2020-09-08T15:48:11.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:10 smithi104 bash[9203]: cluster 2020-09-08T15:48:09.761226+0000 mgr.x (mgr.24118) 625 : cluster [DBG] pgmap v877: 17 pgs: 17 active+clean; 5.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s wr, 1.27k op/s 2020-09-08T15:48:11.251 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:48:11.361 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:11.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:48:11 smithi166 bash[11928]: debug 2020-09-08T15:48:11.015+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:11.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:48:11 smithi166 bash[14254]: debug 2020-09-08T15:48:11.239+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:11.386 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 48 15 97281 97266 126.634 84.375 0.0079226 0.00789486 2020-09-08T15:48:11.484 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:11.598 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:11.724 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:11.734 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:48:11 smithi166 bash[18960]: debug 2020-09-08T15:48:11.487+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:11.832 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:48:11.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:48:11 smithi055 bash[19003]: debug 2020-09-08T15:48:11.726+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:11.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:11 smithi055 bash[9296]: cluster 2020-09-08T15:48:10.160849+0000 osd.0 (osd.0) 39 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:11.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:11 smithi055 bash[9296]: cluster 2020-09-08T15:48:10.160928+0000 osd.0 (osd.0) 40 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:11.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:11 smithi055 bash[9296]: cluster 2020-09-08T15:48:10.741900+0000 mon.a (mon.0) 1365 : cluster [DBG] osdmap e337: 12 total, 12 up, 12 in 2020-09-08T15:48:11.934 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:48:12.039 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:12.047 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:11 smithi104 bash[9203]: cluster 2020-09-08T15:48:10.160849+0000 osd.0 (osd.0) 39 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:12.048 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:11 smithi104 bash[9203]: cluster 2020-09-08T15:48:10.160928+0000 osd.0 (osd.0) 40 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:12.048 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:11 smithi104 bash[9203]: cluster 2020-09-08T15:48:10.741900+0000 mon.a (mon.0) 1365 : cluster [DBG] osdmap e337: 12 total, 12 up, 12 in 2020-09-08T15:48:12.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:11 smithi166 bash[9446]: cluster 2020-09-08T15:48:10.160849+0000 osd.0 (osd.0) 39 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:12.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:11 smithi166 bash[9446]: cluster 2020-09-08T15:48:10.160928+0000 osd.0 (osd.0) 40 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:12.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:11 smithi166 bash[9446]: cluster 2020-09-08T15:48:10.741900+0000 mon.a (mon.0) 1365 : cluster [DBG] osdmap e337: 12 total, 12 up, 12 in 2020-09-08T15:48:12.386 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 49 16 98691 98675 125.846 88.0625 0.00853994 0.00794368 2020-09-08T15:48:12.399 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:48:12.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:48:11 smithi055 bash[14301]: debug 2020-09-08T15:48:11.938+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:12.561 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:48:12.721 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:12.825 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:48:12.980 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:48:13.081 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:13.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:12 smithi166 bash[9446]: cluster 2020-09-08T15:48:11.761782+0000 mgr.x (mgr.24118) 626 : cluster [DBG] pgmap v879: 17 pgs: 17 active+clean; 5.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 106 MiB/s wr, 1.70k op/s 2020-09-08T15:48:13.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:48:12 smithi055 bash[11956]: debug 2020-09-08T15:48:12.966+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:13.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:12 smithi055 bash[9296]: cluster 2020-09-08T15:48:11.761782+0000 mgr.x (mgr.24118) 626 : cluster [DBG] pgmap v879: 17 pgs: 17 active+clean; 5.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 106 MiB/s wr, 1.70k op/s 2020-09-08T15:48:13.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:48:12 smithi104 bash[17588]: debug 2020-09-08T15:48:12.720+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:13.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:12 smithi104 bash[9203]: cluster 2020-09-08T15:48:11.761782+0000 mgr.x (mgr.24118) 626 : cluster [DBG] pgmap v879: 17 pgs: 17 active+clean; 5.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 106 MiB/s wr, 1.70k op/s 2020-09-08T15:48:13.386 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 50 16 100097 100081 125.087 87.875 0.00835232 0.00799179 2020-09-08T15:48:13.406 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:13.512 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:48:13.624 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:48:13.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:48:13 smithi104 bash[17588]: debug 2020-09-08T15:48:13.400+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:13.729 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:48:13.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:48:13 smithi055 bash[16646]: debug 2020-09-08T15:48:13.626+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:14.386 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 51 16 101191 101175 123.975 68.375 0.0138339 0.0080641 2020-09-08T15:48:15.386 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 52 16 102302 102286 122.926 69.4375 0.00922422 0.0081328 2020-09-08T15:48:16.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:15 smithi055 bash[9296]: cluster 2020-09-08T15:48:13.762441+0000 mgr.x (mgr.24118) 627 : cluster [DBG] pgmap v880: 17 pgs: 17 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 107 MiB/s wr, 1.71k op/s 2020-09-08T15:48:16.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:15 smithi055 bash[9296]: cluster 2020-09-08T15:48:14.716249+0000 mon.a (mon.0) 1366 : cluster [DBG] osdmap e338: 12 total, 12 up, 12 in 2020-09-08T15:48:16.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:15 smithi104 bash[9203]: cluster 2020-09-08T15:48:13.762441+0000 mgr.x (mgr.24118) 627 : cluster [DBG] pgmap v880: 17 pgs: 17 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 107 MiB/s wr, 1.71k op/s 2020-09-08T15:48:16.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:15 smithi104 bash[9203]: cluster 2020-09-08T15:48:14.716249+0000 mon.a (mon.0) 1366 : cluster [DBG] osdmap e338: 12 total, 12 up, 12 in 2020-09-08T15:48:16.210 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:15 smithi166 bash[9446]: cluster 2020-09-08T15:48:13.762441+0000 mgr.x (mgr.24118) 627 : cluster [DBG] pgmap v880: 17 pgs: 17 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 107 MiB/s wr, 1.71k op/s 2020-09-08T15:48:16.210 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:15 smithi166 bash[9446]: cluster 2020-09-08T15:48:14.716249+0000 mon.a (mon.0) 1366 : cluster [DBG] osdmap e338: 12 total, 12 up, 12 in 2020-09-08T15:48:16.386 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 53 15 103510 103495 122.032 75.5625 0.00905614 0.00819233 2020-09-08T15:48:16.938 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:48:16.969 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:48:17.007 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:48:17.050 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:48:17.092 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:48:17.121 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:48:17.156 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:48:17.196 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:48:17.230 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:48:17.262 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:48:17.300 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:48:17.344 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:48:17.349 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:48:17.349 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:48:17.349 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:48:17.349 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:48:17.350 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:48:17.350 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:48:17.350 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:48:17.350 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:48:17.351 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:48:17.351 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:48:17.351 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:48:17.351 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:48:17.389 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 54 16 104849 104833 121.321 83.625 0.0265291 0.0082407 2020-09-08T15:48:17.726 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:17 smithi104 bash[9203]: cluster 2020-09-08T15:48:15.762829+0000 mgr.x (mgr.24118) 628 : cluster [DBG] pgmap v882: 17 pgs: 17 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s wr, 1.16k op/s 2020-09-08T15:48:17.727 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:17 smithi104 bash[9203]: cluster 2020-09-08T15:48:16.822976+0000 osd.5 (osd.5) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:17.727 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:17 smithi104 bash[9203]: cluster 2020-09-08T15:48:16.823061+0000 osd.5 (osd.5) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:17.727 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:17 smithi104 bash[9203]: audit 2020-09-08T15:48:16.913686+0000 mon.a (mon.0) 1367 : audit [DBG] from='client.? v1:172.21.15.104:0/3569556584' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:48:18.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:17 smithi166 bash[9446]: cluster 2020-09-08T15:48:15.762829+0000 mgr.x (mgr.24118) 628 : cluster [DBG] pgmap v882: 17 pgs: 17 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s wr, 1.16k op/s 2020-09-08T15:48:18.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:17 smithi166 bash[9446]: cluster 2020-09-08T15:48:16.822976+0000 osd.5 (osd.5) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:18.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:17 smithi166 bash[9446]: cluster 2020-09-08T15:48:16.823061+0000 osd.5 (osd.5) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:18.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:17 smithi166 bash[9446]: audit 2020-09-08T15:48:16.913686+0000 mon.a (mon.0) 1367 : audit [DBG] from='client.? v1:172.21.15.104:0/3569556584' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:48:18.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:17 smithi055 bash[9296]: cluster 2020-09-08T15:48:15.762829+0000 mgr.x (mgr.24118) 628 : cluster [DBG] pgmap v882: 17 pgs: 17 active+clean; 6.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s wr, 1.16k op/s 2020-09-08T15:48:18.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:17 smithi055 bash[9296]: cluster 2020-09-08T15:48:16.822976+0000 osd.5 (osd.5) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:18.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:17 smithi055 bash[9296]: cluster 2020-09-08T15:48:16.823061+0000 osd.5 (osd.5) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:18.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:17 smithi055 bash[9296]: audit 2020-09-08T15:48:16.913686+0000 mon.a (mon.0) 1367 : audit [DBG] from='client.? v1:172.21.15.104:0/3569556584' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:48:18.386 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 55 16 106067 106051 120.499 76.125 0.00575922 0.00829675 2020-09-08T15:48:18.678 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:48:18.788 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:19.070 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:18 smithi104 bash[9203]: cluster 2020-09-08T15:48:17.724061+0000 mon.a (mon.0) 1368 : cluster [DBG] osdmap e339: 12 total, 12 up, 12 in 2020-09-08T15:48:19.070 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:18 smithi104 bash[9203]: cluster 2020-09-08T15:48:17.763221+0000 mgr.x (mgr.24118) 629 : cluster [DBG] pgmap v884: 17 pgs: 17 active+clean; 6.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 94 MiB/s wr, 1.50k op/s 2020-09-08T15:48:19.118 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:48:18 smithi166 bash[14254]: debug 2020-09-08T15:48:18.679+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:19.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:18 smithi166 bash[9446]: cluster 2020-09-08T15:48:17.724061+0000 mon.a (mon.0) 1368 : cluster [DBG] osdmap e339: 12 total, 12 up, 12 in 2020-09-08T15:48:19.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:18 smithi166 bash[9446]: cluster 2020-09-08T15:48:17.763221+0000 mgr.x (mgr.24118) 629 : cluster [DBG] pgmap v884: 17 pgs: 17 active+clean; 6.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 94 MiB/s wr, 1.50k op/s 2020-09-08T15:48:19.126 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:48:19.126 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T15:48:19.126 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:48:19.127 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.35381.0:106990 4.4 e339/314 4:2fb2abdf:::benchmark_data_smithi166_32054_object106989:head v 339'6654, mlcod=339'6652)", 2020-09-08T15:48:19.127 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:48:19.128210+0000", 2020-09-08T15:48:19.127 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0015152469999999999, 2020-09-08T15:48:19.127 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001546892, 2020-09-08T15:48:19.127 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:48:19.128 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2020-09-08T15:48:19.128 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:48:19.128 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:48:19.128 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:48:19.128 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:48:19.128210+0000", 2020-09-08T15:48:19.129 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:48:19.129 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:48:19.129 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:48:19.129 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:48:19.129 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:48:19.128210+0000", 2020-09-08T15:48:19.130 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019213499999999999 2020-09-08T15:48:19.130 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:48:19.130 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:48:19.131 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:48:19.131 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:48:19.128402+0000", 2020-09-08T15:48:19.131 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1509999999999993e-06 2020-09-08T15:48:19.131 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:48:19.131 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:48:19.132 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:48:19.132 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:48:19.128411+0000", 2020-09-08T15:48:19.132 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5330000000000001e-05 2020-09-08T15:48:19.132 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:48:19.132 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:48:19.132 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:48:19.133 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:48:19.128426+0000", 2020-09-08T15:48:19.133 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0609999999999996e-06 2020-09-08T15:48:19.133 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:48:19.133 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:48:19.133 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:48:19.139 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:48:19.128434+0000", 2020-09-08T15:48:19.139 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5624999999999995e-05 2020-09-08T15:48:19.139 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:48:19.139 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:48:19.139 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:48:19.140 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:48:19.128510+0000", 2020-09-08T15:48:19.140 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4763999999999999e-05 2020-09-08T15:48:19.140 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:48:19.140 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:48:19.140 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:48:19.141 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:48:19.128535+0000", 2020-09-08T15:48:19.141 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00032506599999999999 2020-09-08T15:48:19.141 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:48:19.141 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:48:19.141 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:48:19.142 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:48:19.142 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T15:48:19.142 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 1 2020-09-08T15:48:19.142 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:48:19.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:18 smithi055 bash[9296]: cluster 2020-09-08T15:48:17.724061+0000 mon.a (mon.0) 1368 : cluster [DBG] osdmap e339: 12 total, 12 up, 12 in 2020-09-08T15:48:19.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:18 smithi055 bash[9296]: cluster 2020-09-08T15:48:17.763221+0000 mgr.x (mgr.24118) 629 : cluster [DBG] pgmap v884: 17 pgs: 17 active+clean; 6.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 94 MiB/s wr, 1.50k op/s 2020-09-08T15:48:19.388 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 56 16 107334 107318 119.761 79.1875 0.00888714 0.00834778 2020-09-08T15:48:19.394 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:19.497 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:19.609 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:19.713 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:19.721 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:48:19 smithi166 bash[11928]: debug 2020-09-08T15:48:19.391+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:19.721 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:48:19.835 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:19.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:48:19 smithi055 bash[19003]: debug 2020-09-08T15:48:19.614+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:19.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:19 smithi055 bash[9296]: audit 2020-09-08T15:48:19.564295+0000 mon.a (mon.0) 1369 : audit [INF] from='client.? v1:172.21.15.104:0/2980028175' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:48:19.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:19 smithi055 bash[9296]: cluster 2020-09-08T15:48:19.713832+0000 mon.a (mon.0) 1370 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:48:19.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:19 smithi055 bash[9296]: audit 2020-09-08T15:48:19.720935+0000 mon.a (mon.0) 1371 : audit [INF] from='client.? v1:172.21.15.104:0/2980028175' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:48:19.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:19 smithi055 bash[9296]: cluster 2020-09-08T15:48:19.720981+0000 mon.a (mon.0) 1372 : cluster [DBG] osdmap e340: 12 total, 12 up, 12 in 2020-09-08T15:48:19.940 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:48:19.983 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:19 smithi104 bash[9203]: audit 2020-09-08T15:48:19.564295+0000 mon.a (mon.0) 1369 : audit [INF] from='client.? v1:172.21.15.104:0/2980028175' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:48:19.983 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:19 smithi104 bash[9203]: cluster 2020-09-08T15:48:19.713832+0000 mon.a (mon.0) 1370 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:48:19.983 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:19 smithi104 bash[9203]: audit 2020-09-08T15:48:19.720935+0000 mon.a (mon.0) 1371 : audit [INF] from='client.? v1:172.21.15.104:0/2980028175' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:48:19.984 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:19 smithi104 bash[9203]: cluster 2020-09-08T15:48:19.720981+0000 mon.a (mon.0) 1372 : cluster [DBG] osdmap e340: 12 total, 12 up, 12 in 2020-09-08T15:48:20.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:48:19 smithi166 bash[18960]: debug 2020-09-08T15:48:19.835+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:20.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:19 smithi166 bash[9446]: audit 2020-09-08T15:48:19.564295+0000 mon.a (mon.0) 1369 : audit [INF] from='client.? v1:172.21.15.104:0/2980028175' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:48:20.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:19 smithi166 bash[9446]: cluster 2020-09-08T15:48:19.713832+0000 mon.a (mon.0) 1370 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:48:20.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:19 smithi166 bash[9446]: audit 2020-09-08T15:48:19.720935+0000 mon.a (mon.0) 1371 : audit [INF] from='client.? v1:172.21.15.104:0/2980028175' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:48:20.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:19 smithi166 bash[9446]: cluster 2020-09-08T15:48:19.720981+0000 mon.a (mon.0) 1372 : cluster [DBG] osdmap e340: 12 total, 12 up, 12 in 2020-09-08T15:48:20.284 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:48:20.387 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 57 16 108566 108550 119.011 77 0.0241647 0.00840058 2020-09-08T15:48:20.389 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:48:20.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:48:20 smithi104 bash[22305]: debug 2020-09-08T15:48:20.280+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:20.788 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:48:20.870 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2020-09-08T15:48:20.971 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:48:20.979 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:48:21.058 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:48:21.086 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:21.095 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:20 smithi166 bash[9446]: cluster 2020-09-08T15:48:19.763618+0000 mgr.x (mgr.24118) 630 : cluster [DBG] pgmap v886: 17 pgs: 17 active+clean; 6.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 63 MiB/s wr, 1.01k op/s 2020-09-08T15:48:21.096 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:20 smithi166 bash[9446]: cluster 2020-09-08T15:48:19.785610+0000 osd.10 (osd.10) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:21.096 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:20 smithi166 bash[9446]: cluster 2020-09-08T15:48:19.785702+0000 osd.10 (osd.10) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:21.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:20 smithi055 bash[9296]: cluster 2020-09-08T15:48:19.763618+0000 mgr.x (mgr.24118) 630 : cluster [DBG] pgmap v886: 17 pgs: 17 active+clean; 6.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 63 MiB/s wr, 1.01k op/s 2020-09-08T15:48:21.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:20 smithi055 bash[9296]: cluster 2020-09-08T15:48:19.785610+0000 osd.10 (osd.10) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:21.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:20 smithi055 bash[9296]: cluster 2020-09-08T15:48:19.785702+0000 osd.10 (osd.10) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:21.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:20 smithi104 bash[9203]: cluster 2020-09-08T15:48:19.763618+0000 mgr.x (mgr.24118) 630 : cluster [DBG] pgmap v886: 17 pgs: 17 active+clean; 6.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 63 MiB/s wr, 1.01k op/s 2020-09-08T15:48:21.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:20 smithi104 bash[9203]: cluster 2020-09-08T15:48:19.785610+0000 osd.10 (osd.10) 29 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:21.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:20 smithi104 bash[9203]: cluster 2020-09-08T15:48:19.785702+0000 osd.10 (osd.10) 30 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:21.180 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:48:20 smithi104 bash[22305]: debug 2020-09-08T15:48:20.980+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:21.220 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:21.324 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:21.371 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:48:21 smithi166 bash[11928]: debug 2020-09-08T15:48:21.219+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:21.387 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 58 16 109756 109740 118.241 74.375 0.0157954 0.00845521 2020-09-08T15:48:21.434 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:21.539 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:48:21.738 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:48:21 smithi166 bash[18960]: debug 2020-09-08T15:48:21.435+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:21.961 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:21 smithi104 bash[9203]: cluster 2020-09-08T15:48:20.091436+0000 osd.1 (osd.1) 60 : cluster [DBG] 4.4 scrub starts 2020-09-08T15:48:21.961 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:21 smithi104 bash[9203]: cluster 2020-09-08T15:48:20.579920+0000 osd.3 (osd.3) 37 : cluster [DBG] 4.c scrub starts 2020-09-08T15:48:21.962 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:21 smithi104 bash[9203]: cluster 2020-09-08T15:48:20.745802+0000 mon.a (mon.0) 1373 : cluster [DBG] osdmap e341: 12 total, 12 up, 12 in 2020-09-08T15:48:21.962 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:21 smithi104 bash[9203]: cluster 2020-09-08T15:48:20.819321+0000 osd.11 (osd.11) 41 : cluster [DBG] 4.1 scrub starts 2020-09-08T15:48:22.061 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:48:22.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:21 smithi166 bash[9446]: cluster 2020-09-08T15:48:20.091436+0000 osd.1 (osd.1) 60 : cluster [DBG] 4.4 scrub starts 2020-09-08T15:48:22.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:21 smithi166 bash[9446]: cluster 2020-09-08T15:48:20.579920+0000 osd.3 (osd.3) 37 : cluster [DBG] 4.c scrub starts 2020-09-08T15:48:22.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:21 smithi166 bash[9446]: cluster 2020-09-08T15:48:20.745802+0000 mon.a (mon.0) 1373 : cluster [DBG] osdmap e341: 12 total, 12 up, 12 in 2020-09-08T15:48:22.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:21 smithi166 bash[9446]: cluster 2020-09-08T15:48:20.819321+0000 osd.11 (osd.11) 41 : cluster [DBG] 4.1 scrub starts 2020-09-08T15:48:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:21 smithi055 bash[9296]: cluster 2020-09-08T15:48:20.091436+0000 osd.1 (osd.1) 60 : cluster [DBG] 4.4 scrub starts 2020-09-08T15:48:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:21 smithi055 bash[9296]: cluster 2020-09-08T15:48:20.579920+0000 osd.3 (osd.3) 37 : cluster [DBG] 4.c scrub starts 2020-09-08T15:48:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:21 smithi055 bash[9296]: cluster 2020-09-08T15:48:20.745802+0000 mon.a (mon.0) 1373 : cluster [DBG] osdmap e341: 12 total, 12 up, 12 in 2020-09-08T15:48:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:21 smithi055 bash[9296]: cluster 2020-09-08T15:48:20.819321+0000 osd.11 (osd.11) 41 : cluster [DBG] 4.1 scrub starts 2020-09-08T15:48:22.167 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:22.387 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 59 16 110838 110822 117.383 67.625 0.00790458 0.00851671 2020-09-08T15:48:22.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:48:22 smithi104 bash[15273]: debug 2020-09-08T15:48:22.060+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:22.922 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:23.024 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:23.033 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:22 smithi055 bash[9296]: cluster 2020-09-08T15:48:21.764099+0000 mgr.x (mgr.24118) 631 : cluster [DBG] pgmap v888: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s wr, 1.21k op/s 2020-09-08T15:48:23.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:22 smithi166 bash[9446]: cluster 2020-09-08T15:48:21.764099+0000 mgr.x (mgr.24118) 631 : cluster [DBG] pgmap v888: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s wr, 1.21k op/s 2020-09-08T15:48:23.172 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:23.254 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:22 smithi104 bash[9203]: cluster 2020-09-08T15:48:21.764099+0000 mgr.x (mgr.24118) 631 : cluster [DBG] pgmap v888: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s wr, 1.21k op/s 2020-09-08T15:48:23.254 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:48:22 smithi104 bash[19922]: debug 2020-09-08T15:48:22.920+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:23.356 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:48:23.363 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:48:23 smithi055 bash[19003]: debug 2020-09-08T15:48:23.170+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:23.387 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 15:48:23.391902 min lat: 0.00163485 max lat: 0.556499 avg lat: 0.00856308 2020-09-08T15:48:23.387 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:48:23.388 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 60 16 112103 112087 116.744 79.0625 0.00630556 0.00856308 2020-09-08T15:48:23.473 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:48:23.579 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:48:23.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:48:23 smithi055 bash[11956]: debug 2020-09-08T15:48:23.478+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:23.709 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:48:23.818 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:23.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:48:23 smithi104 bash[22305]: debug 2020-09-08T15:48:23.708+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:23.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:23 smithi104 bash[9203]: cluster 2020-09-08T15:48:22.105098+0000 osd.1 (osd.1) 61 : cluster [DBG] 4.4 scrub ok 2020-09-08T15:48:23.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:23 smithi104 bash[9203]: cluster 2020-09-08T15:48:22.536319+0000 osd.3 (osd.3) 38 : cluster [DBG] 4.c scrub ok 2020-09-08T15:48:23.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:23 smithi104 bash[9203]: cluster 2020-09-08T15:48:22.584715+0000 osd.6 (osd.6) 45 : cluster [DBG] 4.a scrub starts 2020-09-08T15:48:23.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:23 smithi104 bash[9203]: cluster 2020-09-08T15:48:22.586166+0000 osd.3 (osd.3) 39 : cluster [DBG] 4.0 scrub starts 2020-09-08T15:48:23.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:23 smithi104 bash[9203]: cluster 2020-09-08T15:48:22.597386+0000 osd.10 (osd.10) 31 : cluster [DBG] 4.e scrub starts 2020-09-08T15:48:23.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:23 smithi104 bash[9203]: cluster 2020-09-08T15:48:22.981554+0000 osd.11 (osd.11) 42 : cluster [DBG] 4.1 scrub ok 2020-09-08T15:48:23.943 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:23.992 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:48:24.047 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:24.056 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:23 smithi166 bash[9446]: cluster 2020-09-08T15:48:22.105098+0000 osd.1 (osd.1) 61 : cluster [DBG] 4.4 scrub ok 2020-09-08T15:48:24.057 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:23 smithi166 bash[9446]: cluster 2020-09-08T15:48:22.536319+0000 osd.3 (osd.3) 38 : cluster [DBG] 4.c scrub ok 2020-09-08T15:48:24.057 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:23 smithi166 bash[9446]: cluster 2020-09-08T15:48:22.584715+0000 osd.6 (osd.6) 45 : cluster [DBG] 4.a scrub starts 2020-09-08T15:48:24.057 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:23 smithi166 bash[9446]: cluster 2020-09-08T15:48:22.586166+0000 osd.3 (osd.3) 39 : cluster [DBG] 4.0 scrub starts 2020-09-08T15:48:24.057 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:23 smithi166 bash[9446]: cluster 2020-09-08T15:48:22.597386+0000 osd.10 (osd.10) 31 : cluster [DBG] 4.e scrub starts 2020-09-08T15:48:24.057 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:23 smithi166 bash[9446]: cluster 2020-09-08T15:48:22.981554+0000 osd.11 (osd.11) 42 : cluster [DBG] 4.1 scrub ok 2020-09-08T15:48:24.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:48:23 smithi055 bash[19003]: debug 2020-09-08T15:48:23.946+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:24.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:23 smithi055 bash[9296]: cluster 2020-09-08T15:48:22.105098+0000 osd.1 (osd.1) 61 : cluster [DBG] 4.4 scrub ok 2020-09-08T15:48:24.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:23 smithi055 bash[9296]: cluster 2020-09-08T15:48:22.536319+0000 osd.3 (osd.3) 38 : cluster [DBG] 4.c scrub ok 2020-09-08T15:48:24.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:23 smithi055 bash[9296]: cluster 2020-09-08T15:48:22.584715+0000 osd.6 (osd.6) 45 : cluster [DBG] 4.a scrub starts 2020-09-08T15:48:24.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:23 smithi055 bash[9296]: cluster 2020-09-08T15:48:22.586166+0000 osd.3 (osd.3) 39 : cluster [DBG] 4.0 scrub starts 2020-09-08T15:48:24.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:23 smithi055 bash[9296]: cluster 2020-09-08T15:48:22.597386+0000 osd.10 (osd.10) 31 : cluster [DBG] 4.e scrub starts 2020-09-08T15:48:24.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:23 smithi055 bash[9296]: cluster 2020-09-08T15:48:22.981554+0000 osd.11 (osd.11) 42 : cluster [DBG] 4.1 scrub ok 2020-09-08T15:48:24.159 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:48:24.200 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:24.303 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:48:24.311 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:48:24 smithi166 bash[18960]: debug 2020-09-08T15:48:24.198+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:24.394 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 61 16 113280 113264 116.036 73.5625 0.0148427 0.00861597 2020-09-08T15:48:24.413 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:48:24.519 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:48:24.616 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:48:24 smithi166 bash[14254]: debug 2020-09-08T15:48:24.410+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:24.652 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:48:24.754 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:24.906 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:48:24 smithi055 bash[14301]: debug 2020-09-08T15:48:24.649+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:25.390 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 62 16 114309 114293 115.202 64.3125 0.0165424 0.00867805 2020-09-08T15:48:25.874 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:25.982 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:48:25.998 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:25 smithi055 bash[9296]: cluster 2020-09-08T15:48:23.764615+0000 mgr.x (mgr.24118) 632 : cluster [DBG] pgmap v889: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 MiB/s wr, 1.45k op/s 2020-09-08T15:48:25.998 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:25 smithi055 bash[9296]: cluster 2020-09-08T15:48:24.504203+0000 osd.10 (osd.10) 32 : cluster [DBG] 4.e scrub ok 2020-09-08T15:48:25.999 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:25 smithi055 bash[9296]: cluster 2020-09-08T15:48:24.687796+0000 osd.9 (osd.9) 35 : cluster [DBG] 4.d scrub starts 2020-09-08T15:48:25.999 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:25 smithi055 bash[9296]: cluster 2020-09-08T15:48:24.719782+0000 mon.a (mon.0) 1374 : cluster [DBG] osdmap e342: 12 total, 12 up, 12 in 2020-09-08T15:48:25.999 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:25 smithi055 bash[9296]: cluster 2020-09-08T15:48:24.945073+0000 osd.3 (osd.3) 40 : cluster [DBG] 4.0 scrub ok 2020-09-08T15:48:25.999 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:25 smithi055 bash[9296]: cluster 2020-09-08T15:48:25.130612+0000 osd.6 (osd.6) 46 : cluster [DBG] 4.a scrub ok 2020-09-08T15:48:26.000 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:25 smithi055 bash[9296]: cluster 2020-09-08T15:48:25.258443+0000 osd.3 (osd.3) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:26.000 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:25 smithi055 bash[9296]: cluster 2020-09-08T15:48:25.258531+0000 osd.3 (osd.3) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:26.097 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:48:26.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:25 smithi166 bash[9446]: cluster 2020-09-08T15:48:23.764615+0000 mgr.x (mgr.24118) 632 : cluster [DBG] pgmap v889: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 MiB/s wr, 1.45k op/s 2020-09-08T15:48:26.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:25 smithi166 bash[9446]: cluster 2020-09-08T15:48:24.504203+0000 osd.10 (osd.10) 32 : cluster [DBG] 4.e scrub ok 2020-09-08T15:48:26.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:25 smithi166 bash[9446]: cluster 2020-09-08T15:48:24.687796+0000 osd.9 (osd.9) 35 : cluster [DBG] 4.d scrub starts 2020-09-08T15:48:26.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:25 smithi166 bash[9446]: cluster 2020-09-08T15:48:24.719782+0000 mon.a (mon.0) 1374 : cluster [DBG] osdmap e342: 12 total, 12 up, 12 in 2020-09-08T15:48:26.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:25 smithi166 bash[9446]: cluster 2020-09-08T15:48:24.945073+0000 osd.3 (osd.3) 40 : cluster [DBG] 4.0 scrub ok 2020-09-08T15:48:26.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:25 smithi166 bash[9446]: cluster 2020-09-08T15:48:25.130612+0000 osd.6 (osd.6) 46 : cluster [DBG] 4.a scrub ok 2020-09-08T15:48:26.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:25 smithi166 bash[9446]: cluster 2020-09-08T15:48:25.258443+0000 osd.3 (osd.3) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:26.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:25 smithi166 bash[9446]: cluster 2020-09-08T15:48:25.258531+0000 osd.3 (osd.3) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:26.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:48:25 smithi104 bash[17588]: debug 2020-09-08T15:48:25.872+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:26.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:25 smithi104 bash[9203]: cluster 2020-09-08T15:48:23.764615+0000 mgr.x (mgr.24118) 632 : cluster [DBG] pgmap v889: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 MiB/s wr, 1.45k op/s 2020-09-08T15:48:26.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:25 smithi104 bash[9203]: cluster 2020-09-08T15:48:24.504203+0000 osd.10 (osd.10) 32 : cluster [DBG] 4.e scrub ok 2020-09-08T15:48:26.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:25 smithi104 bash[9203]: cluster 2020-09-08T15:48:24.687796+0000 osd.9 (osd.9) 35 : cluster [DBG] 4.d scrub starts 2020-09-08T15:48:26.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:25 smithi104 bash[9203]: cluster 2020-09-08T15:48:24.719782+0000 mon.a (mon.0) 1374 : cluster [DBG] osdmap e342: 12 total, 12 up, 12 in 2020-09-08T15:48:26.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:25 smithi104 bash[9203]: cluster 2020-09-08T15:48:24.945073+0000 osd.3 (osd.3) 40 : cluster [DBG] 4.0 scrub ok 2020-09-08T15:48:26.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:25 smithi104 bash[9203]: cluster 2020-09-08T15:48:25.130612+0000 osd.6 (osd.6) 46 : cluster [DBG] 4.a scrub ok 2020-09-08T15:48:26.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:25 smithi104 bash[9203]: cluster 2020-09-08T15:48:25.258443+0000 osd.3 (osd.3) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:26.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:25 smithi104 bash[9203]: cluster 2020-09-08T15:48:25.258531+0000 osd.3 (osd.3) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:26.212 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:26.388 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 63 16 115351 115335 114.407 65.125 0.00788154 0.00871463 2020-09-08T15:48:26.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:48:26 smithi055 bash[14301]: debug 2020-09-08T15:48:26.097+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:26.721 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:26.825 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:48:26.954 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:48:27.059 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:27.072 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:26 smithi055 bash[9296]: cluster 2020-09-08T15:48:25.159728+0000 osd.1 (osd.1) 62 : cluster [DBG] 4.7 scrub starts 2020-09-08T15:48:27.073 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:26 smithi055 bash[9296]: cluster 2020-09-08T15:48:25.760909+0000 mon.a (mon.0) 1375 : cluster [DBG] osdmap e343: 12 total, 12 up, 12 in 2020-09-08T15:48:27.073 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:26 smithi055 bash[9296]: cluster 2020-09-08T15:48:25.764963+0000 mgr.x (mgr.24118) 633 : cluster [DBG] pgmap v892: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 MiB/s wr, 1.02k op/s 2020-09-08T15:48:27.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:26 smithi166 bash[9446]: cluster 2020-09-08T15:48:25.159728+0000 osd.1 (osd.1) 62 : cluster [DBG] 4.7 scrub starts 2020-09-08T15:48:27.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:26 smithi166 bash[9446]: cluster 2020-09-08T15:48:25.760909+0000 mon.a (mon.0) 1375 : cluster [DBG] osdmap e343: 12 total, 12 up, 12 in 2020-09-08T15:48:27.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:26 smithi166 bash[9446]: cluster 2020-09-08T15:48:25.764963+0000 mgr.x (mgr.24118) 633 : cluster [DBG] pgmap v892: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 MiB/s wr, 1.02k op/s 2020-09-08T15:48:27.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:48:26 smithi166 bash[16593]: debug 2020-09-08T15:48:26.954+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:27.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:26 smithi104 bash[9203]: cluster 2020-09-08T15:48:25.159728+0000 osd.1 (osd.1) 62 : cluster [DBG] 4.7 scrub starts 2020-09-08T15:48:27.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:26 smithi104 bash[9203]: cluster 2020-09-08T15:48:25.760909+0000 mon.a (mon.0) 1375 : cluster [DBG] osdmap e343: 12 total, 12 up, 12 in 2020-09-08T15:48:27.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:26 smithi104 bash[9203]: cluster 2020-09-08T15:48:25.764963+0000 mgr.x (mgr.24118) 633 : cluster [DBG] pgmap v892: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 MiB/s wr, 1.02k op/s 2020-09-08T15:48:27.178 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:48:26 smithi104 bash[17588]: debug 2020-09-08T15:48:26.716+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:27.208 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:27.312 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:27.388 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 64 16 116113 116097 113.363 47.625 0.00648397 0.00881889 2020-09-08T15:48:27.419 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:48:27 smithi055 bash[19003]: debug 2020-09-08T15:48:27.209+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:27.702 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:27.812 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:28.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:27 smithi166 bash[9446]: cluster 2020-09-08T15:48:26.462891+0000 osd.9 (osd.9) 36 : cluster [DBG] 4.d scrub ok 2020-09-08T15:48:28.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:27 smithi166 bash[9446]: cluster 2020-09-08T15:48:26.698253+0000 osd.9 (osd.9) 37 : cluster [DBG] 4.2 scrub starts 2020-09-08T15:48:28.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:27 smithi055 bash[9296]: cluster 2020-09-08T15:48:26.462891+0000 osd.9 (osd.9) 36 : cluster [DBG] 4.d scrub ok 2020-09-08T15:48:28.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:27 smithi055 bash[9296]: cluster 2020-09-08T15:48:26.698253+0000 osd.9 (osd.9) 37 : cluster [DBG] 4.2 scrub starts 2020-09-08T15:48:28.179 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:48:27 smithi104 bash[17588]: debug 2020-09-08T15:48:27.704+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:28.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:27 smithi104 bash[9203]: cluster 2020-09-08T15:48:26.462891+0000 osd.9 (osd.9) 36 : cluster [DBG] 4.d scrub ok 2020-09-08T15:48:28.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:27 smithi104 bash[9203]: cluster 2020-09-08T15:48:26.698253+0000 osd.9 (osd.9) 37 : cluster [DBG] 4.2 scrub starts 2020-09-08T15:48:28.388 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 65 16 117145 117129 112.611 64.5 0.00737679 0.00887847 2020-09-08T15:48:28.407 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:28.510 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:48:28.645 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:48:28.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:48:28 smithi104 bash[19922]: debug 2020-09-08T15:48:28.408+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:28.749 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:28.813 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:48:28 smithi166 bash[14254]: debug 2020-09-08T15:48:28.646+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:28.814 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:28 smithi166 bash[9446]: cluster 2020-09-08T15:48:27.110966+0000 osd.1 (osd.1) 63 : cluster [DBG] 4.7 scrub ok 2020-09-08T15:48:28.814 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:28 smithi166 bash[9446]: cluster 2020-09-08T15:48:27.765523+0000 mgr.x (mgr.24118) 634 : cluster [DBG] pgmap v893: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s wr, 1.30k op/s 2020-09-08T15:48:28.872 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:28.976 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:48:29.086 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:48:29.087 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:48:29 smithi166 bash[16593]: debug 2020-09-08T15:48:29.086+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:29.139 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:28 smithi104 bash[9203]: cluster 2020-09-08T15:48:27.110966+0000 osd.1 (osd.1) 63 : cluster [DBG] 4.7 scrub ok 2020-09-08T15:48:29.139 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:28 smithi104 bash[9203]: cluster 2020-09-08T15:48:27.765523+0000 mgr.x (mgr.24118) 634 : cluster [DBG] pgmap v893: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s wr, 1.30k op/s 2020-09-08T15:48:29.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:48:28 smithi055 bash[19003]: debug 2020-09-08T15:48:28.873+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:29.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:28 smithi055 bash[9296]: cluster 2020-09-08T15:48:27.110966+0000 osd.1 (osd.1) 63 : cluster [DBG] 4.7 scrub ok 2020-09-08T15:48:29.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:28 smithi055 bash[9296]: cluster 2020-09-08T15:48:27.765523+0000 mgr.x (mgr.24118) 634 : cluster [DBG] pgmap v893: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s wr, 1.30k op/s 2020-09-08T15:48:29.190 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:29.307 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:29.388 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 66 16 118375 118359 112.07 76.875 0.022824 0.0089205 2020-09-08T15:48:29.411 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:29.433 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:48:29.524 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:48:29.560 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:48:29.590 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:48:29.623 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:48:29.658 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:48:29.659 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:48:29 smithi055 bash[19003]: debug 2020-09-08T15:48:29.309+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:29.690 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:48:29.730 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:48:29.764 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:48:29.789 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:48:29.816 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:29 smithi104 bash[9203]: cluster 2020-09-08T15:48:28.377716+0000 osd.9 (osd.9) 38 : cluster [DBG] 4.2 scrub ok 2020-09-08T15:48:29.816 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:29 smithi104 bash[9203]: audit 2020-09-08T15:48:29.407147+0000 mon.a (mon.0) 1376 : audit [DBG] from='client.? v1:172.21.15.104:0/3615440680' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:48:29.817 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:29 smithi104 bash[9203]: cluster 2020-09-08T15:48:29.722386+0000 mon.a (mon.0) 1377 : cluster [DBG] osdmap e344: 12 total, 12 up, 12 in 2020-09-08T15:48:29.824 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:48:29.856 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:48:29.858 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:48:29.858 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:48:29.858 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:48:29.859 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:48:29.859 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:48:29.859 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:48:29.859 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:48:29.860 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:48:29.860 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:48:29.860 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:48:29.861 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:48:29.861 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:48:30.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:29 smithi166 bash[9446]: cluster 2020-09-08T15:48:28.377716+0000 osd.9 (osd.9) 38 : cluster [DBG] 4.2 scrub ok 2020-09-08T15:48:30.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:29 smithi166 bash[9446]: audit 2020-09-08T15:48:29.407147+0000 mon.a (mon.0) 1376 : audit [DBG] from='client.? v1:172.21.15.104:0/3615440680' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:48:30.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:29 smithi166 bash[9446]: cluster 2020-09-08T15:48:29.722386+0000 mon.a (mon.0) 1377 : cluster [DBG] osdmap e344: 12 total, 12 up, 12 in 2020-09-08T15:48:30.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:29 smithi055 bash[9296]: cluster 2020-09-08T15:48:28.377716+0000 osd.9 (osd.9) 38 : cluster [DBG] 4.2 scrub ok 2020-09-08T15:48:30.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:29 smithi055 bash[9296]: audit 2020-09-08T15:48:29.407147+0000 mon.a (mon.0) 1376 : audit [DBG] from='client.? v1:172.21.15.104:0/3615440680' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:48:30.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:29 smithi055 bash[9296]: cluster 2020-09-08T15:48:29.722386+0000 mon.a (mon.0) 1377 : cluster [DBG] osdmap e344: 12 total, 12 up, 12 in 2020-09-08T15:48:30.294 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:30.389 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 67 16 119325 119309 111.283 59.375 0.0100443 0.00898428 2020-09-08T15:48:30.390 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:48:30.390 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:48:30.402 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:30.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:48:30 smithi104 bash[17588]: debug 2020-09-08T15:48:30.292+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:31.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:30 smithi166 bash[9446]: cluster 2020-09-08T15:48:29.765944+0000 mgr.x (mgr.24118) 635 : cluster [DBG] pgmap v895: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s wr, 862 op/s 2020-09-08T15:48:31.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:30 smithi055 bash[9296]: cluster 2020-09-08T15:48:29.765944+0000 mgr.x (mgr.24118) 635 : cluster [DBG] pgmap v895: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s wr, 862 op/s 2020-09-08T15:48:31.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:30 smithi104 bash[9203]: cluster 2020-09-08T15:48:29.765944+0000 mgr.x (mgr.24118) 635 : cluster [DBG] pgmap v895: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s wr, 862 op/s 2020-09-08T15:48:31.262 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:31.374 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:31.394 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 68 16 120595 120579 110.814 79.375 0.00875381 0.009022 2020-09-08T15:48:31.505 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:31.610 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:48:31.658 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:48:31 smithi055 bash[19003]: debug 2020-09-08T15:48:31.505+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:31.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:48:31 smithi104 bash[17588]: debug 2020-09-08T15:48:31.260+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:31.716 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:48:31.823 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:31.964 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:32.070 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:48:32.078 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:31 smithi055 bash[9296]: cluster 2020-09-08T15:48:30.536232+0000 osd.6 (osd.6) 47 : cluster [DBG] 4.5 scrub starts 2020-09-08T15:48:32.078 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:31 smithi055 bash[9296]: cluster 2020-09-08T15:48:30.925179+0000 osd.8 (osd.8) 45 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:32.078 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:31 smithi055 bash[9296]: cluster 2020-09-08T15:48:30.925276+0000 osd.8 (osd.8) 46 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:32.079 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:48:31 smithi055 bash[14301]: debug 2020-09-08T15:48:31.717+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:32.103 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:31 smithi104 bash[9203]: cluster 2020-09-08T15:48:30.536232+0000 osd.6 (osd.6) 47 : cluster [DBG] 4.5 scrub starts 2020-09-08T15:48:32.103 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:31 smithi104 bash[9203]: cluster 2020-09-08T15:48:30.925179+0000 osd.8 (osd.8) 45 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:32.104 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:31 smithi104 bash[9203]: cluster 2020-09-08T15:48:30.925276+0000 osd.8 (osd.8) 46 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:32.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:48:31 smithi166 bash[11928]: debug 2020-09-08T15:48:31.962+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:32.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:31 smithi166 bash[9446]: cluster 2020-09-08T15:48:30.536232+0000 osd.6 (osd.6) 47 : cluster [DBG] 4.5 scrub starts 2020-09-08T15:48:32.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:31 smithi166 bash[9446]: cluster 2020-09-08T15:48:30.925179+0000 osd.8 (osd.8) 45 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:32.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:31 smithi166 bash[9446]: cluster 2020-09-08T15:48:30.925276+0000 osd.8 (osd.8) 46 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:32.181 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:48:32.287 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:32.388 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 69 16 121646 121630 110.16 65.6875 0.00737914 0.00905501 2020-09-08T15:48:32.426 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:48:32 smithi055 bash[14301]: debug 2020-09-08T15:48:32.185+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:32.637 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2020-09-08T15:48:32.797 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:32.906 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:32.916 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:48:32.917 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:32 smithi104 bash[9203]: cluster 2020-09-08T15:48:31.766411+0000 mgr.x (mgr.24118) 636 : cluster [DBG] pgmap v896: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s wr, 879 op/s 2020-09-08T15:48:32.917 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:32 smithi104 bash[9203]: cluster 2020-09-08T15:48:31.836587+0000 mon.a (mon.0) 1378 : cluster [DBG] osdmap e345: 12 total, 12 up, 12 in 2020-09-08T15:48:32.917 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:48:32 smithi104 bash[17588]: debug 2020-09-08T15:48:32.800+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:33.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:32 smithi166 bash[9446]: cluster 2020-09-08T15:48:31.766411+0000 mgr.x (mgr.24118) 636 : cluster [DBG] pgmap v896: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s wr, 879 op/s 2020-09-08T15:48:33.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:32 smithi166 bash[9446]: cluster 2020-09-08T15:48:31.836587+0000 mon.a (mon.0) 1378 : cluster [DBG] osdmap e345: 12 total, 12 up, 12 in 2020-09-08T15:48:33.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:32 smithi055 bash[9296]: cluster 2020-09-08T15:48:31.766411+0000 mgr.x (mgr.24118) 636 : cluster [DBG] pgmap v896: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s wr, 879 op/s 2020-09-08T15:48:33.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:32 smithi055 bash[9296]: cluster 2020-09-08T15:48:31.836587+0000 mon.a (mon.0) 1378 : cluster [DBG] osdmap e345: 12 total, 12 up, 12 in 2020-09-08T15:48:33.388 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 70 16 122793 122777 109.61 71.6875 0.00590285 0.00911534 2020-09-08T15:48:33.492 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:33.597 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:33.736 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:33.836 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:48:33 smithi104 bash[19922]: debug 2020-09-08T15:48:33.492+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:33.836 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:33 smithi104 bash[9203]: cluster 2020-09-08T15:48:32.490903+0000 osd.6 (osd.6) 48 : cluster [DBG] 4.5 scrub ok 2020-09-08T15:48:33.839 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:48:33.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:33 smithi166 bash[9446]: cluster 2020-09-08T15:48:32.490903+0000 osd.6 (osd.6) 48 : cluster [DBG] 4.5 scrub ok 2020-09-08T15:48:33.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:48:33 smithi166 bash[18960]: debug 2020-09-08T15:48:33.734+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:34.105 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:48:34.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:33 smithi055 bash[9296]: cluster 2020-09-08T15:48:32.490903+0000 osd.6 (osd.6) 48 : cluster [DBG] 4.5 scrub ok 2020-09-08T15:48:34.277 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:48:34.388 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 71 16 123752 123736 108.91 59.9375 0.00520455 0.0091783 2020-09-08T15:48:34.731 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:48:35.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:34 smithi166 bash[9446]: cluster 2020-09-08T15:48:33.766929+0000 mgr.x (mgr.24118) 637 : cluster [DBG] pgmap v898: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s wr, 1.35k op/s 2020-09-08T15:48:35.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:34 smithi166 bash[9446]: cluster 2020-09-08T15:48:33.931709+0000 osd.11 (osd.11) 43 : cluster [DBG] 4.3 scrub starts 2020-09-08T15:48:35.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:34 smithi166 bash[9446]: audit 2020-09-08T15:48:34.272749+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? v1:172.21.15.104:0/2237502311' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:48:35.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:34 smithi166 bash[9446]: cluster 2020-09-08T15:48:34.717935+0000 mon.a (mon.0) 1380 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:48:35.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:34 smithi166 bash[9446]: cluster 2020-09-08T15:48:34.717972+0000 mon.a (mon.0) 1381 : cluster [INF] Cluster is now healthy 2020-09-08T15:48:35.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:34 smithi166 bash[9446]: audit 2020-09-08T15:48:34.722988+0000 mon.a (mon.0) 1382 : audit [INF] from='client.? v1:172.21.15.104:0/2237502311' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:48:35.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:34 smithi166 bash[9446]: cluster 2020-09-08T15:48:34.723046+0000 mon.a (mon.0) 1383 : cluster [DBG] osdmap e346: 12 total, 12 up, 12 in 2020-09-08T15:48:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:34 smithi055 bash[9296]: cluster 2020-09-08T15:48:33.766929+0000 mgr.x (mgr.24118) 637 : cluster [DBG] pgmap v898: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s wr, 1.35k op/s 2020-09-08T15:48:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:34 smithi055 bash[9296]: cluster 2020-09-08T15:48:33.931709+0000 osd.11 (osd.11) 43 : cluster [DBG] 4.3 scrub starts 2020-09-08T15:48:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:34 smithi055 bash[9296]: audit 2020-09-08T15:48:34.272749+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? v1:172.21.15.104:0/2237502311' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:48:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:34 smithi055 bash[9296]: cluster 2020-09-08T15:48:34.717935+0000 mon.a (mon.0) 1380 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:48:35.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:34 smithi055 bash[9296]: cluster 2020-09-08T15:48:34.717972+0000 mon.a (mon.0) 1381 : cluster [INF] Cluster is now healthy 2020-09-08T15:48:35.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:34 smithi055 bash[9296]: audit 2020-09-08T15:48:34.722988+0000 mon.a (mon.0) 1382 : audit [INF] from='client.? v1:172.21.15.104:0/2237502311' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:48:35.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:34 smithi055 bash[9296]: cluster 2020-09-08T15:48:34.723046+0000 mon.a (mon.0) 1383 : cluster [DBG] osdmap e346: 12 total, 12 up, 12 in 2020-09-08T15:48:35.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:34 smithi104 bash[9203]: cluster 2020-09-08T15:48:33.766929+0000 mgr.x (mgr.24118) 637 : cluster [DBG] pgmap v898: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s wr, 1.35k op/s 2020-09-08T15:48:35.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:34 smithi104 bash[9203]: cluster 2020-09-08T15:48:33.931709+0000 osd.11 (osd.11) 43 : cluster [DBG] 4.3 scrub starts 2020-09-08T15:48:35.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:34 smithi104 bash[9203]: audit 2020-09-08T15:48:34.272749+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? v1:172.21.15.104:0/2237502311' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:48:35.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:34 smithi104 bash[9203]: cluster 2020-09-08T15:48:34.717935+0000 mon.a (mon.0) 1380 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:48:35.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:34 smithi104 bash[9203]: cluster 2020-09-08T15:48:34.717972+0000 mon.a (mon.0) 1381 : cluster [INF] Cluster is now healthy 2020-09-08T15:48:35.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:34 smithi104 bash[9203]: audit 2020-09-08T15:48:34.722988+0000 mon.a (mon.0) 1382 : audit [INF] from='client.? v1:172.21.15.104:0/2237502311' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:48:35.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:34 smithi104 bash[9203]: cluster 2020-09-08T15:48:34.723046+0000 mon.a (mon.0) 1383 : cluster [DBG] osdmap e346: 12 total, 12 up, 12 in 2020-09-08T15:48:35.388 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 72 16 124932 124916 108.422 73.75 0.00652262 0.009221 2020-09-08T15:48:36.388 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 73 16 126122 126106 107.956 74.375 0.00908883 0.00926117 2020-09-08T15:48:37.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:36 smithi166 bash[9446]: cluster 2020-09-08T15:48:35.767404+0000 mgr.x (mgr.24118) 638 : cluster [DBG] pgmap v900: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s wr, 932 op/s 2020-09-08T15:48:37.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:36 smithi166 bash[9446]: cluster 2020-09-08T15:48:36.227749+0000 osd.11 (osd.11) 44 : cluster [DBG] 4.3 scrub ok 2020-09-08T15:48:37.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:36 smithi055 bash[9296]: cluster 2020-09-08T15:48:35.767404+0000 mgr.x (mgr.24118) 638 : cluster [DBG] pgmap v900: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s wr, 932 op/s 2020-09-08T15:48:37.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:36 smithi055 bash[9296]: cluster 2020-09-08T15:48:36.227749+0000 osd.11 (osd.11) 44 : cluster [DBG] 4.3 scrub ok 2020-09-08T15:48:37.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:36 smithi104 bash[9203]: cluster 2020-09-08T15:48:35.767404+0000 mgr.x (mgr.24118) 638 : cluster [DBG] pgmap v900: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s wr, 932 op/s 2020-09-08T15:48:37.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:36 smithi104 bash[9203]: cluster 2020-09-08T15:48:36.227749+0000 osd.11 (osd.11) 44 : cluster [DBG] 4.3 scrub ok 2020-09-08T15:48:37.388 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 74 16 127157 127141 107.371 64.6875 0.00525754 0.0093104 2020-09-08T15:48:38.306 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:48:38.388 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 75 16 128361 128345 106.942 75.25 0.0154465 0.00934895 2020-09-08T15:48:38.485 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:48:38.538 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:48:38.641 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:48:38.779 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:48:38.844 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:38 smithi055 bash[9296]: cluster 2020-09-08T15:48:37.279897+0000 osd.1 (osd.1) 64 : cluster [DBG] 4.f scrub starts 2020-09-08T15:48:38.845 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:38 smithi055 bash[9296]: cluster 2020-09-08T15:48:37.767865+0000 mgr.x (mgr.24118) 639 : cluster [DBG] pgmap v901: 17 pgs: 17 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s wr, 1.09k op/s 2020-09-08T15:48:38.845 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:48:38 smithi055 bash[14301]: debug 2020-09-08T15:48:38.533+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:38.884 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:48:39.009 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:48:39.112 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:48:39.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:38 smithi166 bash[9446]: cluster 2020-09-08T15:48:37.279897+0000 osd.1 (osd.1) 64 : cluster [DBG] 4.f scrub starts 2020-09-08T15:48:39.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:38 smithi166 bash[9446]: cluster 2020-09-08T15:48:37.767865+0000 mgr.x (mgr.24118) 639 : cluster [DBG] pgmap v901: 17 pgs: 17 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s wr, 1.09k op/s 2020-09-08T15:48:39.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:48:38 smithi166 bash[16593]: debug 2020-09-08T15:48:38.778+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:39.156 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:48:39 smithi055 bash[14301]: debug 2020-09-08T15:48:39.013+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:39.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:38 smithi104 bash[9203]: cluster 2020-09-08T15:48:37.279897+0000 osd.1 (osd.1) 64 : cluster [DBG] 4.f scrub starts 2020-09-08T15:48:39.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:38 smithi104 bash[9203]: cluster 2020-09-08T15:48:37.767865+0000 mgr.x (mgr.24118) 639 : cluster [DBG] pgmap v901: 17 pgs: 17 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s wr, 1.09k op/s 2020-09-08T15:48:39.255 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:48:39.360 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:48:39.389 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 76 16 129517 129501 106.486 72.25 0.00568028 0.00938906 2020-09-08T15:48:39.466 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:48:39.569 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:39.582 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:48:39 smithi166 bash[14254]: debug 2020-09-08T15:48:39.250+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:39.657 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:48:39 smithi055 bash[11956]: debug 2020-09-08T15:48:39.469+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:39.692 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:39.800 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:39.847 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:48:39 smithi166 bash[18960]: debug 2020-09-08T15:48:39.694+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:39.919 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:40.029 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:40.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:39 smithi166 bash[9446]: cluster 2020-09-08T15:48:38.892701+0000 osd.1 (osd.1) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:40.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:39 smithi166 bash[9446]: cluster 2020-09-08T15:48:38.892893+0000 osd.1 (osd.1) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:40.134 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:48:39 smithi055 bash[19003]: debug 2020-09-08T15:48:39.921+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:40.135 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:48:40 smithi055 bash[19003]: debug 2020-09-08T15:48:40.137+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:40.135 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:39 smithi055 bash[9296]: cluster 2020-09-08T15:48:38.892701+0000 osd.1 (osd.1) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:40.135 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:39 smithi055 bash[9296]: cluster 2020-09-08T15:48:38.892893+0000 osd.1 (osd.1) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:40.136 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:40.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:39 smithi104 bash[9203]: cluster 2020-09-08T15:48:38.892701+0000 osd.1 (osd.1) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:40.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:39 smithi104 bash[9203]: cluster 2020-09-08T15:48:38.892893+0000 osd.1 (osd.1) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:40.241 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:40.389 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 77 16 130154 130138 105.62 39.8125 0.0109801 0.00946543 2020-09-08T15:48:40.793 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:40.898 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:48:40.985 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:48:40.985 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:48:41.020 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:48:41.094 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:40 smithi104 bash[9203]: cluster 2020-09-08T15:48:39.768312+0000 mgr.x (mgr.24118) 640 : cluster [DBG] pgmap v902: 17 pgs: 17 active+clean; 7.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 79 MiB/s wr, 1.27k op/s 2020-09-08T15:48:41.095 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:40 smithi104 bash[9203]: cluster 2020-09-08T15:48:39.865485+0000 mon.a (mon.0) 1384 : cluster [DBG] osdmap e347: 12 total, 12 up, 12 in 2020-09-08T15:48:41.095 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:48:40 smithi104 bash[17588]: debug 2020-09-08T15:48:40.796+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:41.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:40 smithi166 bash[9446]: cluster 2020-09-08T15:48:39.768312+0000 mgr.x (mgr.24118) 640 : cluster [DBG] pgmap v902: 17 pgs: 17 active+clean; 7.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 79 MiB/s wr, 1.27k op/s 2020-09-08T15:48:41.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:40 smithi166 bash[9446]: cluster 2020-09-08T15:48:39.865485+0000 mon.a (mon.0) 1384 : cluster [DBG] osdmap e347: 12 total, 12 up, 12 in 2020-09-08T15:48:41.122 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:48:41.158 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:48:41 smithi055 bash[16646]: debug 2020-09-08T15:48:41.017+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:41.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:40 smithi055 bash[9296]: cluster 2020-09-08T15:48:39.768312+0000 mgr.x (mgr.24118) 640 : cluster [DBG] pgmap v902: 17 pgs: 17 active+clean; 7.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 79 MiB/s wr, 1.27k op/s 2020-09-08T15:48:41.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:40 smithi055 bash[9296]: cluster 2020-09-08T15:48:39.865485+0000 mon.a (mon.0) 1384 : cluster [DBG] osdmap e347: 12 total, 12 up, 12 in 2020-09-08T15:48:41.334 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:48:41.366 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:48:41 smithi104 bash[15273]: debug 2020-09-08T15:48:41.328+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:41.389 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 78 16 131269 131253 105.159 69.6875 0.00557558 0.0095059 2020-09-08T15:48:41.438 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:48:41.536 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:48:41.643 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:41.785 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:48:41.822 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:48:41.865 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:48:41 smithi055 bash[16646]: debug 2020-09-08T15:48:41.537+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:41.866 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:41 smithi055 bash[9296]: cluster 2020-09-08T15:48:41.070640+0000 osd.11 (osd.11) 45 : cluster [DBG] 4.8 deep-scrub starts 2020-09-08T15:48:41.866 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:41 smithi055 bash[9296]: audit 2020-09-08T15:48:41.761246+0000 mon.a (mon.0) 1385 : audit [DBG] from='client.? v1:172.21.15.104:0/1193517982' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:48:41.870 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:48:41.910 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:48:41.936 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:48:41.967 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:48:42.001 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:48:42.043 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:48:42.073 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:48:42.109 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:48:42.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:41 smithi166 bash[9446]: cluster 2020-09-08T15:48:41.070640+0000 osd.11 (osd.11) 45 : cluster [DBG] 4.8 deep-scrub starts 2020-09-08T15:48:42.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:41 smithi166 bash[9446]: audit 2020-09-08T15:48:41.761246+0000 mon.a (mon.0) 1385 : audit [DBG] from='client.? v1:172.21.15.104:0/1193517982' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:48:42.140 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:48:42.145 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:41 smithi104 bash[9203]: cluster 2020-09-08T15:48:41.070640+0000 osd.11 (osd.11) 45 : cluster [DBG] 4.8 deep-scrub starts 2020-09-08T15:48:42.145 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:41 smithi104 bash[9203]: audit 2020-09-08T15:48:41.761246+0000 mon.a (mon.0) 1385 : audit [DBG] from='client.? v1:172.21.15.104:0/1193517982' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:48:42.176 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:48:42.179 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:48:42.179 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:48:42.180 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:48:42.180 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:48:42.180 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:48:42.180 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:48:42.180 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:48:42.181 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:48:42.181 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:48:42.181 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:48:42.182 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:48:42.182 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:48:42.322 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:42.389 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 79 16 132229 132213 104.588 60 0.0156087 0.00955914 2020-09-08T15:48:42.426 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:42.427 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:48:42 smithi104 bash[19922]: debug 2020-09-08T15:48:42.324+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:42.568 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:42.675 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:48:42.796 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:48:42.862 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:48:42 smithi166 bash[11928]: debug 2020-09-08T15:48:42.566+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:42.900 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:48:43.022 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:42 smithi055 bash[9296]: cluster 2020-09-08T15:48:41.285200+0000 osd.1 (osd.1) 67 : cluster [DBG] 4.f scrub ok 2020-09-08T15:48:43.022 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:42 smithi055 bash[9296]: cluster 2020-09-08T15:48:41.768745+0000 mgr.x (mgr.24118) 641 : cluster [DBG] pgmap v904: 17 pgs: 17 active+clean; 7.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 MiB/s wr, 811 op/s 2020-09-08T15:48:43.023 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:48:42 smithi055 bash[14301]: debug 2020-09-08T15:48:42.797+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:43.023 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:48:43.060 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2020-09-08T15:48:43.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:42 smithi166 bash[9446]: cluster 2020-09-08T15:48:41.285200+0000 osd.1 (osd.1) 67 : cluster [DBG] 4.f scrub ok 2020-09-08T15:48:43.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:42 smithi166 bash[9446]: cluster 2020-09-08T15:48:41.768745+0000 mgr.x (mgr.24118) 641 : cluster [DBG] pgmap v904: 17 pgs: 17 active+clean; 7.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 MiB/s wr, 811 op/s 2020-09-08T15:48:43.124 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:48:43.134 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:42 smithi104 bash[9203]: cluster 2020-09-08T15:48:41.285200+0000 osd.1 (osd.1) 67 : cluster [DBG] 4.f scrub ok 2020-09-08T15:48:43.134 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:42 smithi104 bash[9203]: cluster 2020-09-08T15:48:41.768745+0000 mgr.x (mgr.24118) 641 : cluster [DBG] pgmap v904: 17 pgs: 17 active+clean; 7.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 MiB/s wr, 811 op/s 2020-09-08T15:48:43.341 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:48:43.389 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 15:48:43.393911 min lat: 0.00163485 max lat: 0.556499 avg lat: 0.00960874 2020-09-08T15:48:43.389 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:48:43.390 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 80 16 133199 133183 104.038 60.625 0.00637951 0.00960874 2020-09-08T15:48:43.414 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:48:43 smithi055 bash[16646]: debug 2020-09-08T15:48:43.017+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:43.729 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:48:43.836 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:44.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:48:43 smithi104 bash[15273]: debug 2020-09-08T15:48:43.728+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:44.394 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 81 15 134162 134147 103.497 60.25 0.0120985 0.00965729 2020-09-08T15:48:44.911 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:45.013 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:48:45.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:44 smithi055 bash[9296]: cluster 2020-09-08T15:48:43.769271+0000 mgr.x (mgr.24118) 642 : cluster [DBG] pgmap v905: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s wr, 1.16k op/s 2020-09-08T15:48:45.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:44 smithi055 bash[9296]: cluster 2020-09-08T15:48:43.885387+0000 osd.9 (osd.9) 39 : cluster [DBG] 4.6 scrub starts 2020-09-08T15:48:45.164 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:48:44 smithi166 bash[11928]: debug 2020-09-08T15:48:44.902+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:45.165 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:44 smithi166 bash[9446]: cluster 2020-09-08T15:48:43.769271+0000 mgr.x (mgr.24118) 642 : cluster [DBG] pgmap v905: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s wr, 1.16k op/s 2020-09-08T15:48:45.165 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:44 smithi166 bash[9446]: cluster 2020-09-08T15:48:43.885387+0000 osd.9 (osd.9) 39 : cluster [DBG] 4.6 scrub starts 2020-09-08T15:48:45.165 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:48:45.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:44 smithi104 bash[9203]: cluster 2020-09-08T15:48:43.769271+0000 mgr.x (mgr.24118) 642 : cluster [DBG] pgmap v905: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s wr, 1.16k op/s 2020-09-08T15:48:45.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:44 smithi104 bash[9203]: cluster 2020-09-08T15:48:43.885387+0000 osd.9 (osd.9) 39 : cluster [DBG] 4.6 scrub starts 2020-09-08T15:48:45.268 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:48:45.394 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 82 16 135027 135011 102.894 54 0.00702953 0.00971661 2020-09-08T15:48:45.616 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:48:45 smithi166 bash[14254]: debug 2020-09-08T15:48:45.162+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:45.717 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:48:45.823 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:48:46.006 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:45 smithi104 bash[9203]: cluster 2020-09-08T15:48:44.426487+0000 osd.6 (osd.6) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:46.006 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:45 smithi104 bash[9203]: cluster 2020-09-08T15:48:44.426590+0000 osd.6 (osd.6) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:46.006 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:45 smithi104 bash[9203]: cluster 2020-09-08T15:48:44.873008+0000 mon.a (mon.0) 1386 : cluster [DBG] osdmap e348: 12 total, 12 up, 12 in 2020-09-08T15:48:46.006 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:48:45 smithi104 bash[15273]: debug 2020-09-08T15:48:45.720+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:46.007 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:48:46 smithi104 bash[15273]: debug 2020-09-08T15:48:46.008+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:46.012 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:48:46.116 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:46.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:45 smithi055 bash[9296]: cluster 2020-09-08T15:48:44.426487+0000 osd.6 (osd.6) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:46.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:45 smithi055 bash[9296]: cluster 2020-09-08T15:48:44.426590+0000 osd.6 (osd.6) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:46.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:45 smithi055 bash[9296]: cluster 2020-09-08T15:48:44.873008+0000 mon.a (mon.0) 1386 : cluster [DBG] osdmap e348: 12 total, 12 up, 12 in 2020-09-08T15:48:46.276 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:48:46 smithi104 bash[19922]: debug 2020-09-08T15:48:46.280+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:46.278 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:46.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:45 smithi166 bash[9446]: cluster 2020-09-08T15:48:44.426487+0000 osd.6 (osd.6) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:46.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:45 smithi166 bash[9446]: cluster 2020-09-08T15:48:44.426590+0000 osd.6 (osd.6) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:46.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:45 smithi166 bash[9446]: cluster 2020-09-08T15:48:44.873008+0000 mon.a (mon.0) 1386 : cluster [DBG] osdmap e348: 12 total, 12 up, 12 in 2020-09-08T15:48:46.382 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:48:46.389 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 83 16 135954 135938 102.352 57.9375 0.00831073 0.00976659 2020-09-08T15:48:46.526 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:48:46.630 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:46.749 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:48:46 smithi166 bash[14254]: debug 2020-09-08T15:48:46.526+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:46.749 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:48:46 smithi166 bash[11928]: debug 2020-09-08T15:48:46.750+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:46.750 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:46.855 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:48:46.870 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:48:46.888 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:48:46.976 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:48:47.007 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:47 smithi104 bash[9203]: cluster 2020-09-08T15:48:45.769743+0000 mgr.x (mgr.24118) 643 : cluster [DBG] pgmap v907: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s wr, 921 op/s 2020-09-08T15:48:47.007 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:47 smithi104 bash[9203]: cluster 2020-09-08T15:48:46.087355+0000 osd.9 (osd.9) 40 : cluster [DBG] 4.6 scrub ok 2020-09-08T15:48:47.008 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:47 smithi104 bash[9203]: audit 2020-09-08T15:48:46.724586+0000 mon.a (mon.0) 1387 : audit [INF] from='client.? v1:172.21.15.104:0/374930513' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:48:47.026 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:48:47.082 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:47.096 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:46 smithi166 bash[9446]: cluster 2020-09-08T15:48:45.769743+0000 mgr.x (mgr.24118) 643 : cluster [DBG] pgmap v907: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s wr, 921 op/s 2020-09-08T15:48:47.096 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:46 smithi166 bash[9446]: cluster 2020-09-08T15:48:46.087355+0000 osd.9 (osd.9) 40 : cluster [DBG] 4.6 scrub ok 2020-09-08T15:48:47.097 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:46 smithi166 bash[9446]: audit 2020-09-08T15:48:46.724586+0000 mon.a (mon.0) 1387 : audit [INF] from='client.? v1:172.21.15.104:0/374930513' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:48:47.097 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:48:46 smithi166 bash[14254]: debug 2020-09-08T15:48:46.974+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:47.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:46 smithi055 bash[9296]: cluster 2020-09-08T15:48:45.769743+0000 mgr.x (mgr.24118) 643 : cluster [DBG] pgmap v907: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s wr, 921 op/s 2020-09-08T15:48:47.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:46 smithi055 bash[9296]: cluster 2020-09-08T15:48:46.087355+0000 osd.9 (osd.9) 40 : cluster [DBG] 4.6 scrub ok 2020-09-08T15:48:47.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:46 smithi055 bash[9296]: audit 2020-09-08T15:48:46.724586+0000 mon.a (mon.0) 1387 : audit [INF] from='client.? v1:172.21.15.104:0/374930513' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:48:47.196 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:47.302 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:47.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:48:47 smithi166 bash[11928]: debug 2020-09-08T15:48:47.194+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:47.389 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 84 16 136894 136878 101.833 58.75 0.00767889 0.00981532 2020-09-08T15:48:47.764 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:47.867 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:47.981 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:48:48.085 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:48.095 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:47 smithi104 bash[9203]: cluster 2020-09-08T15:48:46.318352+0000 osd.11 (osd.11) 46 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:48.097 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:47 smithi104 bash[9203]: cluster 2020-09-08T15:48:46.318409+0000 osd.11 (osd.11) 47 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:48.097 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:47 smithi104 bash[9203]: cluster 2020-09-08T15:48:46.885435+0000 mon.a (mon.0) 1388 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:48:48.097 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:47 smithi104 bash[9203]: audit 2020-09-08T15:48:46.890238+0000 mon.a (mon.0) 1389 : audit [INF] from='client.? v1:172.21.15.104:0/374930513' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:48:48.099 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:47 smithi104 bash[9203]: cluster 2020-09-08T15:48:46.890317+0000 mon.a (mon.0) 1390 : cluster [DBG] osdmap e349: 12 total, 12 up, 12 in 2020-09-08T15:48:48.099 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:48:47 smithi104 bash[17588]: debug 2020-09-08T15:48:47.768+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:48.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:48:47 smithi055 bash[19003]: debug 2020-09-08T15:48:47.981+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:48.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:47 smithi055 bash[9296]: cluster 2020-09-08T15:48:46.318352+0000 osd.11 (osd.11) 46 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:48.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:47 smithi055 bash[9296]: cluster 2020-09-08T15:48:46.318409+0000 osd.11 (osd.11) 47 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:48.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:47 smithi055 bash[9296]: cluster 2020-09-08T15:48:46.885435+0000 mon.a (mon.0) 1388 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:48:48.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:47 smithi055 bash[9296]: audit 2020-09-08T15:48:46.890238+0000 mon.a (mon.0) 1389 : audit [INF] from='client.? v1:172.21.15.104:0/374930513' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:48:48.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:47 smithi055 bash[9296]: cluster 2020-09-08T15:48:46.890317+0000 mon.a (mon.0) 1390 : cluster [DBG] osdmap e349: 12 total, 12 up, 12 in 2020-09-08T15:48:48.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:47 smithi166 bash[9446]: cluster 2020-09-08T15:48:46.318352+0000 osd.11 (osd.11) 46 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:48.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:47 smithi166 bash[9446]: cluster 2020-09-08T15:48:46.318409+0000 osd.11 (osd.11) 47 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:48.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:47 smithi166 bash[9446]: cluster 2020-09-08T15:48:46.885435+0000 mon.a (mon.0) 1388 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:48:48.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:47 smithi166 bash[9446]: audit 2020-09-08T15:48:46.890238+0000 mon.a (mon.0) 1389 : audit [INF] from='client.? v1:172.21.15.104:0/374930513' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:48:48.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:47 smithi166 bash[9446]: cluster 2020-09-08T15:48:46.890317+0000 mon.a (mon.0) 1390 : cluster [DBG] osdmap e349: 12 total, 12 up, 12 in 2020-09-08T15:48:48.390 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 85 16 137795 137779 101.297 56.3125 0.0162665 0.00986842 2020-09-08T15:48:48.709 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:48.813 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:48:48.924 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:48:49.028 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:49.152 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:48 smithi166 bash[9446]: cluster 2020-09-08T15:48:47.770171+0000 mgr.x (mgr.24118) 644 : cluster [DBG] pgmap v909: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s wr, 958 op/s 2020-09-08T15:48:49.156 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:49.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:48:48 smithi055 bash[11956]: debug 2020-09-08T15:48:48.925+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:49.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:48 smithi055 bash[9296]: cluster 2020-09-08T15:48:47.770171+0000 mgr.x (mgr.24118) 644 : cluster [DBG] pgmap v909: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s wr, 958 op/s 2020-09-08T15:48:49.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:48 smithi104 bash[9203]: cluster 2020-09-08T15:48:47.770171+0000 mgr.x (mgr.24118) 644 : cluster [DBG] pgmap v909: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s wr, 958 op/s 2020-09-08T15:48:49.177 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:48:48 smithi104 bash[17588]: debug 2020-09-08T15:48:48.712+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:49.263 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:48:49.371 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:48:49.390 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 86 16 138705 138689 100.781 56.875 0.0190157 0.00992006 2020-09-08T15:48:49.471 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:48:49.583 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:48:49 smithi055 bash[14301]: debug 2020-09-08T15:48:49.365+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:49.583 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:48:49 smithi055 bash[16646]: debug 2020-09-08T15:48:49.585+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:49.583 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:48:49.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:48:49 smithi166 bash[11928]: debug 2020-09-08T15:48:49.154+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:49.689 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:48:49.819 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:48:49.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:48:49 smithi055 bash[11956]: debug 2020-09-08T15:48:49.817+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:49.924 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:50.183 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:48:50.183 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T15:48:50.183 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 0 2020-09-08T15:48:50.183 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:48:50.390 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 87 16 139527 139511 100.213 51.375 0.00670585 0.00997631 2020-09-08T15:48:50.927 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:48:51.072 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:48:51.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:50 smithi055 bash[9296]: cluster 2020-09-08T15:48:49.577043+0000 osd.4 (osd.4) 35 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:51.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:50 smithi055 bash[9296]: cluster 2020-09-08T15:48:49.577121+0000 osd.4 (osd.4) 36 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:51.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:50 smithi055 bash[9296]: cluster 2020-09-08T15:48:49.770661+0000 mgr.x (mgr.24118) 645 : cluster [DBG] pgmap v910: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s wr, 1.08k op/s 2020-09-08T15:48:51.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:50 smithi055 bash[9296]: cluster 2020-09-08T15:48:49.904384+0000 mon.a (mon.0) 1391 : cluster [DBG] osdmap e350: 12 total, 12 up, 12 in 2020-09-08T15:48:51.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:50 smithi104 bash[9203]: cluster 2020-09-08T15:48:49.577043+0000 osd.4 (osd.4) 35 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:51.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:50 smithi104 bash[9203]: cluster 2020-09-08T15:48:49.577121+0000 osd.4 (osd.4) 36 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:51.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:50 smithi104 bash[9203]: cluster 2020-09-08T15:48:49.770661+0000 mgr.x (mgr.24118) 645 : cluster [DBG] pgmap v910: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s wr, 1.08k op/s 2020-09-08T15:48:51.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:50 smithi104 bash[9203]: cluster 2020-09-08T15:48:49.904384+0000 mon.a (mon.0) 1391 : cluster [DBG] osdmap e350: 12 total, 12 up, 12 in 2020-09-08T15:48:51.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:50 smithi166 bash[9446]: cluster 2020-09-08T15:48:49.577043+0000 osd.4 (osd.4) 35 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:48:51.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:50 smithi166 bash[9446]: cluster 2020-09-08T15:48:49.577121+0000 osd.4 (osd.4) 36 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:48:51.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:50 smithi166 bash[9446]: cluster 2020-09-08T15:48:49.770661+0000 mgr.x (mgr.24118) 645 : cluster [DBG] pgmap v910: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s wr, 1.08k op/s 2020-09-08T15:48:51.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:50 smithi166 bash[9446]: cluster 2020-09-08T15:48:49.904384+0000 mon.a (mon.0) 1391 : cluster [DBG] osdmap e350: 12 total, 12 up, 12 in 2020-09-08T15:48:51.390 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 88 16 140392 140376 99.6883 54.0625 0.00989183 0.0100288 2020-09-08T15:48:52.012 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2020-09-08T15:48:52.165 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:48:52.390 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 89 16 141324 141308 99.2226 58.25 0.0122868 0.010075 2020-09-08T15:48:53.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:52 smithi055 bash[9296]: cluster 2020-09-08T15:48:51.771120+0000 mgr.x (mgr.24118) 646 : cluster [DBG] pgmap v912: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s wr, 800 op/s 2020-09-08T15:48:53.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:52 smithi104 bash[9203]: cluster 2020-09-08T15:48:51.771120+0000 mgr.x (mgr.24118) 646 : cluster [DBG] pgmap v912: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s wr, 800 op/s 2020-09-08T15:48:53.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:52 smithi166 bash[9446]: cluster 2020-09-08T15:48:51.771120+0000 mgr.x (mgr.24118) 646 : cluster [DBG] pgmap v912: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s wr, 800 op/s 2020-09-08T15:48:53.390 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 90 16 142139 142123 98.6861 50.9375 0.00803037 0.01013 2020-09-08T15:48:54.258 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:48:54.283 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:48:54.311 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:48:54.333 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:48:54.361 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:48:54.389 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:48:54.390 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Total time run: 90.0518 2020-09-08T15:48:54.390 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Total writes made: 142139 2020-09-08T15:48:54.391 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Write size: 65536 2020-09-08T15:48:54.391 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Object size: 65536 2020-09-08T15:48:54.391 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Bandwidth (MB/sec): 98.6508 2020-09-08T15:48:54.391 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev Bandwidth: 44.4051 2020-09-08T15:48:54.391 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max bandwidth (MB/sec): 246.75 2020-09-08T15:48:54.392 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min bandwidth (MB/sec): 39.8125 2020-09-08T15:48:54.392 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Average IOPS: 1578 2020-09-08T15:48:54.392 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev IOPS: 710 2020-09-08T15:48:54.392 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max IOPS: 3948 2020-09-08T15:48:54.392 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min IOPS: 637 2020-09-08T15:48:54.393 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Average Latency(s): 0.0101324 2020-09-08T15:48:54.393 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev Latency(s): 0.0149273 2020-09-08T15:48:54.393 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max latency(s): 0.556499 2020-09-08T15:48:54.394 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min latency(s): 0.00163485 2020-09-08T15:48:54.402 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Cleaning up (deleting benchmark objects) 2020-09-08T15:48:54.421 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:48:54.459 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:48:54.498 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:48:54.526 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:48:54.562 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:48:54.590 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:48:54.591 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:48:54.591 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:48:54.591 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:48:54.591 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:48:54.591 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:48:54.592 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:48:54.592 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:48:54.592 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:48:54.592 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:48:54.592 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:48:54.593 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:48:54.593 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:48:54.952 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:55.055 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:55.062 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:54 smithi104 bash[9203]: cluster 2020-09-08T15:48:53.771793+0000 mgr.x (mgr.24118) 647 : cluster [DBG] pgmap v913: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s wr, 1.14k op/s 2020-09-08T15:48:55.062 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:54 smithi104 bash[9203]: audit 2020-09-08T15:48:54.235906+0000 mon.a (mon.0) 1392 : audit [DBG] from='client.? v1:172.21.15.104:0/3152587555' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:48:55.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:54 smithi166 bash[9446]: cluster 2020-09-08T15:48:53.771793+0000 mgr.x (mgr.24118) 647 : cluster [DBG] pgmap v913: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s wr, 1.14k op/s 2020-09-08T15:48:55.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:54 smithi166 bash[9446]: audit 2020-09-08T15:48:54.235906+0000 mon.a (mon.0) 1392 : audit [DBG] from='client.? v1:172.21.15.104:0/3152587555' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:48:55.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:48:54 smithi166 bash[18960]: debug 2020-09-08T15:48:54.950+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:55.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:54 smithi055 bash[9296]: cluster 2020-09-08T15:48:53.771793+0000 mgr.x (mgr.24118) 647 : cluster [DBG] pgmap v913: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s wr, 1.14k op/s 2020-09-08T15:48:55.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:54 smithi055 bash[9296]: audit 2020-09-08T15:48:54.235906+0000 mon.a (mon.0) 1392 : audit [DBG] from='client.? v1:172.21.15.104:0/3152587555' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:48:55.783 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:55.886 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:56.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:48:55 smithi104 bash[19922]: debug 2020-09-08T15:48:55.788+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:56.207 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:48:56.310 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:56.431 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:48:56.535 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:48:56.644 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:48:56.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:48:56 smithi104 bash[17588]: debug 2020-09-08T15:48:56.432+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:56.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:48:56 smithi104 bash[19922]: debug 2020-09-08T15:48:56.208+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:56.748 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:48:56.861 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:48:56 smithi055 bash[14301]: debug 2020-09-08T15:48:56.645+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:56.862 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:48:56.966 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:57.092 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:48:57.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:56 smithi055 bash[9296]: cluster 2020-09-08T15:48:55.772393+0000 mgr.x (mgr.24118) 648 : cluster [DBG] pgmap v914: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 MiB/s wr, 763 op/s 2020-09-08T15:48:57.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:48:56 smithi055 bash[16646]: debug 2020-09-08T15:48:56.865+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:57.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:56 smithi104 bash[9203]: cluster 2020-09-08T15:48:55.772393+0000 mgr.x (mgr.24118) 648 : cluster [DBG] pgmap v914: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 MiB/s wr, 763 op/s 2020-09-08T15:48:57.199 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:48:57.208 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:56 smithi166 bash[9446]: cluster 2020-09-08T15:48:55.772393+0000 mgr.x (mgr.24118) 648 : cluster [DBG] pgmap v914: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 MiB/s wr, 763 op/s 2020-09-08T15:48:57.208 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:48:57 smithi166 bash[11928]: debug 2020-09-08T15:48:57.090+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:57.324 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:48:57.429 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:57.536 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:48:57 smithi166 bash[14254]: debug 2020-09-08T15:48:57.322+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:57.540 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:48:57.645 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:48:57.868 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:48:57 smithi166 bash[18960]: debug 2020-09-08T15:48:57.538+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:58.104 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:48:58.207 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:48:58.334 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:48:58.394 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:48:58.403 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:48:58 smithi104 bash[22305]: debug 2020-09-08T15:48:58.108+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:58.435 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:48:58.555 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:48:58.585 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:48:58.618 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:48:58 smithi166 bash[14254]: debug 2020-09-08T15:48:58.330+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:58.660 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:48:58.787 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:48:58.788 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:48:58.908 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:48:58 smithi055 bash[11956]: debug 2020-09-08T15:48:58.557+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:58.925 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:48:58.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:58 smithi104 bash[9203]: cluster 2020-09-08T15:48:57.772848+0000 mgr.x (mgr.24118) 649 : cluster [DBG] pgmap v915: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s wr, 843 op/s 2020-09-08T15:48:58.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:58 smithi104 bash[9203]: audit 2020-09-08T15:48:58.592778+0000 mon.a (mon.0) 1393 : audit [INF] from='client.? v1:172.21.15.104:0/751156208' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:48:59.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:58 smithi166 bash[9446]: cluster 2020-09-08T15:48:57.772848+0000 mgr.x (mgr.24118) 649 : cluster [DBG] pgmap v915: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s wr, 843 op/s 2020-09-08T15:48:59.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:58 smithi166 bash[9446]: audit 2020-09-08T15:48:58.592778+0000 mon.a (mon.0) 1393 : audit [INF] from='client.? v1:172.21.15.104:0/751156208' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:48:59.382 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:48:59.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:58 smithi055 bash[9296]: cluster 2020-09-08T15:48:57.772848+0000 mgr.x (mgr.24118) 649 : cluster [DBG] pgmap v915: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s wr, 843 op/s 2020-09-08T15:48:59.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:58 smithi055 bash[9296]: audit 2020-09-08T15:48:58.592778+0000 mon.a (mon.0) 1393 : audit [INF] from='client.? v1:172.21.15.104:0/751156208' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:48:59.487 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:48:59.617 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:48:59.679 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:48:59 smithi104 bash[15273]: debug 2020-09-08T15:48:59.380+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:59.724 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:48:59.838 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:48:59.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:48:59 smithi166 bash[16593]: debug 2020-09-08T15:48:59.614+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:48:59.941 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:00.049 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:00.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:48:59 smithi055 bash[16646]: debug 2020-09-08T15:48:59.841+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:00.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:59 smithi055 bash[9296]: cluster 2020-09-08T15:48:58.491508+0000 osd.9 (osd.9) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:00.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:59 smithi055 bash[9296]: cluster 2020-09-08T15:48:58.491606+0000 osd.9 (osd.9) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:00.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:59 smithi055 bash[9296]: cluster 2020-09-08T15:48:58.917376+0000 mon.a (mon.0) 1394 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:49:00.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:59 smithi055 bash[9296]: audit 2020-09-08T15:48:58.924456+0000 mon.a (mon.0) 1395 : audit [INF] from='client.? v1:172.21.15.104:0/751156208' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:49:00.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:48:59 smithi055 bash[9296]: cluster 2020-09-08T15:48:58.924512+0000 mon.a (mon.0) 1396 : cluster [DBG] osdmap e351: 12 total, 12 up, 12 in 2020-09-08T15:49:00.161 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:49:00.174 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:59 smithi166 bash[9446]: cluster 2020-09-08T15:48:58.491508+0000 osd.9 (osd.9) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:00.174 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:59 smithi166 bash[9446]: cluster 2020-09-08T15:48:58.491606+0000 osd.9 (osd.9) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:00.174 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:59 smithi166 bash[9446]: cluster 2020-09-08T15:48:58.917376+0000 mon.a (mon.0) 1394 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:49:00.175 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:59 smithi166 bash[9446]: audit 2020-09-08T15:48:58.924456+0000 mon.a (mon.0) 1395 : audit [INF] from='client.? v1:172.21.15.104:0/751156208' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:49:00.175 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:48:59 smithi166 bash[9446]: cluster 2020-09-08T15:48:58.924512+0000 mon.a (mon.0) 1396 : cluster [DBG] osdmap e351: 12 total, 12 up, 12 in 2020-09-08T15:49:00.175 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:49:00 smithi166 bash[16593]: debug 2020-09-08T15:49:00.050+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:00.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:59 smithi104 bash[9203]: cluster 2020-09-08T15:48:58.491508+0000 osd.9 (osd.9) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:00.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:59 smithi104 bash[9203]: cluster 2020-09-08T15:48:58.491606+0000 osd.9 (osd.9) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:00.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:59 smithi104 bash[9203]: cluster 2020-09-08T15:48:58.917376+0000 mon.a (mon.0) 1394 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:49:00.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:59 smithi104 bash[9203]: audit 2020-09-08T15:48:58.924456+0000 mon.a (mon.0) 1395 : audit [INF] from='client.? v1:172.21.15.104:0/751156208' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:49:00.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:48:59 smithi104 bash[9203]: cluster 2020-09-08T15:48:58.924512+0000 mon.a (mon.0) 1396 : cluster [DBG] osdmap e351: 12 total, 12 up, 12 in 2020-09-08T15:49:00.280 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:49:00.387 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:00.501 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:00.603 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:49:00.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:49:00 smithi166 bash[18960]: debug 2020-09-08T15:49:00.274+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:00.658 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:49:00 smithi055 bash[11956]: debug 2020-09-08T15:49:00.501+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:00.698 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2020-09-08T15:49:00.727 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:49:00.830 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:00.885 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:49:00.970 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:01.080 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:01.089 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:49:00 smithi055 bash[19003]: debug 2020-09-08T15:49:00.729+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:01.204 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:01.312 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:01.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:49:01 smithi055 bash[11956]: debug 2020-09-08T15:49:01.201+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:01.423 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:01.430 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:49:00 smithi104 bash[15273]: debug 2020-09-08T15:49:00.972+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:01.532 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:01.708 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:01.726 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:01 smithi055 bash[9296]: cluster 2020-09-08T15:48:59.773224+0000 mgr.x (mgr.24118) 650 : cluster [DBG] pgmap v917: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s rd, 33 MiB/s wr, 813 op/s 2020-09-08T15:49:01.726 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:49:01 smithi055 bash[16646]: debug 2020-09-08T15:49:01.425+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:01.812 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:49:01.872 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:01 smithi166 bash[9446]: cluster 2020-09-08T15:48:59.773224+0000 mgr.x (mgr.24118) 650 : cluster [DBG] pgmap v917: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s rd, 33 MiB/s wr, 813 op/s 2020-09-08T15:49:01.872 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:49:01 smithi166 bash[14254]: debug 2020-09-08T15:49:01.706+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:01.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:01 smithi104 bash[9203]: cluster 2020-09-08T15:48:59.773224+0000 mgr.x (mgr.24118) 650 : cluster [DBG] pgmap v917: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s rd, 33 MiB/s wr, 813 op/s 2020-09-08T15:49:01.932 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:49:02.036 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:02.141 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:02.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:49:01 smithi055 bash[19003]: debug 2020-09-08T15:49:01.933+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:02.244 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:02.350 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:02.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:49:02 smithi166 bash[11928]: debug 2020-09-08T15:49:02.138+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:02.452 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:02.584 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:02.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:49:02 smithi055 bash[16646]: debug 2020-09-08T15:49:02.345+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:02.691 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:02.732 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:02 smithi166 bash[9446]: cluster 2020-09-08T15:49:01.596750+0000 osd.2 (osd.2) 46 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:02.732 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:02 smithi166 bash[9446]: cluster 2020-09-08T15:49:01.596841+0000 osd.2 (osd.2) 47 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:02.732 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:02 smithi166 bash[9446]: cluster 2020-09-08T15:49:01.737865+0000 mon.a (mon.0) 1397 : cluster [DBG] osdmap e352: 12 total, 12 up, 12 in 2020-09-08T15:49:02.733 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:02 smithi166 bash[9446]: cluster 2020-09-08T15:49:01.773642+0000 mgr.x (mgr.24118) 651 : cluster [DBG] pgmap v919: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 6.3 MiB/s wr, 555 op/s 2020-09-08T15:49:02.733 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:49:02 smithi166 bash[14254]: debug 2020-09-08T15:49:02.582+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:02.771 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:49:02.817 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:02.924 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:02.928 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:49:03.117 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:49:02 smithi166 bash[14254]: debug 2020-09-08T15:49:02.818+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:03.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:02 smithi055 bash[9296]: cluster 2020-09-08T15:49:01.596750+0000 osd.2 (osd.2) 46 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:03.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:02 smithi055 bash[9296]: cluster 2020-09-08T15:49:01.596841+0000 osd.2 (osd.2) 47 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:03.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:02 smithi055 bash[9296]: cluster 2020-09-08T15:49:01.737865+0000 mon.a (mon.0) 1397 : cluster [DBG] osdmap e352: 12 total, 12 up, 12 in 2020-09-08T15:49:03.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:02 smithi055 bash[9296]: cluster 2020-09-08T15:49:01.773642+0000 mgr.x (mgr.24118) 651 : cluster [DBG] pgmap v919: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 6.3 MiB/s wr, 555 op/s 2020-09-08T15:49:03.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:02 smithi104 bash[9203]: cluster 2020-09-08T15:49:01.596750+0000 osd.2 (osd.2) 46 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:03.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:02 smithi104 bash[9203]: cluster 2020-09-08T15:49:01.596841+0000 osd.2 (osd.2) 47 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:03.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:02 smithi104 bash[9203]: cluster 2020-09-08T15:49:01.737865+0000 mon.a (mon.0) 1397 : cluster [DBG] osdmap e352: 12 total, 12 up, 12 in 2020-09-08T15:49:03.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:02 smithi104 bash[9203]: cluster 2020-09-08T15:49:01.773642+0000 mgr.x (mgr.24118) 651 : cluster [DBG] pgmap v919: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 6.3 MiB/s wr, 555 op/s 2020-09-08T15:49:03.697 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:03.800 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:03.953 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:04.055 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:49:04.063 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:49:03 smithi166 bash[11928]: debug 2020-09-08T15:49:03.950+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:04.168 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:49:04.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:49:03 smithi104 bash[15273]: debug 2020-09-08T15:49:03.696+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:04.272 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:04.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:49:04 smithi166 bash[18960]: debug 2020-09-08T15:49:04.166+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:04.375 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:04.478 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:04.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:49:04 smithi055 bash[16646]: debug 2020-09-08T15:49:04.377+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:05.059 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:05.163 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:05.312 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:05.419 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:49:05.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:49:05 smithi104 bash[15273]: debug 2020-09-08T15:49:05.060+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:05.542 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:49:05.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:49:05 smithi166 bash[16593]: debug 2020-09-08T15:49:05.310+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:05.643 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:05.722 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:49:05 smithi055 bash[19003]: debug 2020-09-08T15:49:05.541+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:05.748 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:05.854 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:05.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:05 smithi104 bash[9203]: cluster 2020-09-08T15:49:03.774152+0000 mgr.x (mgr.24118) 652 : cluster [DBG] pgmap v920: 17 pgs: 17 active+clean; 8.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 6.3 MiB/s wr, 1.05k op/s 2020-09-08T15:49:05.978 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:05 smithi166 bash[9446]: cluster 2020-09-08T15:49:03.774152+0000 mgr.x (mgr.24118) 652 : cluster [DBG] pgmap v920: 17 pgs: 17 active+clean; 8.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 6.3 MiB/s wr, 1.05k op/s 2020-09-08T15:49:05.978 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:49:05 smithi166 bash[11928]: debug 2020-09-08T15:49:05.749+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:05.979 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:06.081 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:06.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:05 smithi055 bash[9296]: cluster 2020-09-08T15:49:03.774152+0000 mgr.x (mgr.24118) 652 : cluster [DBG] pgmap v920: 17 pgs: 17 active+clean; 8.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 6.3 MiB/s wr, 1.05k op/s 2020-09-08T15:49:06.356 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:49:06.366 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:49:05 smithi166 bash[16593]: debug 2020-09-08T15:49:05.973+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:06.384 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:49:06 smithi104 bash[15273]: debug 2020-09-08T15:49:06.380+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:06.384 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:06.409 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:49:06.446 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:49:06.486 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:49:06.492 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:49:06.525 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:49:06.564 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:49:06.593 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:49:06.610 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:49:06.626 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:49:06.664 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:49:06.684 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:49:06.714 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:06.727 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:49:06.762 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:49:06.770 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:49:06.770 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:49:06.770 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:49:06.771 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:49:06.771 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:49:06.771 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:49:06.771 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:49:06.771 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:49:06.772 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:49:06.772 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:49:06.772 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:49:06.772 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:49:06.821 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:06.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:06 smithi166 bash[9446]: audit 2020-09-08T15:49:06.323689+0000 mon.c (mon.2) 53 : audit [DBG] from='client.? v1:172.21.15.104:0/2740204471' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:49:06.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:49:06 smithi166 bash[18960]: debug 2020-09-08T15:49:06.609+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:06.926 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:49:07.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:49:06 smithi055 bash[11956]: debug 2020-09-08T15:49:06.825+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:07.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:06 smithi055 bash[9296]: audit 2020-09-08T15:49:06.323689+0000 mon.c (mon.2) 53 : audit [DBG] from='client.? v1:172.21.15.104:0/2740204471' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:49:07.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:06 smithi104 bash[9203]: audit 2020-09-08T15:49:06.323689+0000 mon.c (mon.2) 53 : audit [DBG] from='client.? v1:172.21.15.104:0/2740204471' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:49:07.556 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:49:07.657 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:07.812 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:07.914 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:07.922 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:07 smithi166 bash[9446]: cluster 2020-09-08T15:49:05.774781+0000 mgr.x (mgr.24118) 653 : cluster [DBG] pgmap v921: 17 pgs: 17 active+clean; 8.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 400 KiB/s wr, 808 op/s 2020-09-08T15:49:07.923 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:49:07 smithi166 bash[11928]: debug 2020-09-08T15:49:07.805+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:07.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:49:07 smithi104 bash[22305]: debug 2020-09-08T15:49:07.552+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:07.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:07 smithi104 bash[9203]: cluster 2020-09-08T15:49:05.774781+0000 mgr.x (mgr.24118) 653 : cluster [DBG] pgmap v921: 17 pgs: 17 active+clean; 8.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 400 KiB/s wr, 808 op/s 2020-09-08T15:49:08.027 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:08.131 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:08.140 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:07 smithi055 bash[9296]: cluster 2020-09-08T15:49:05.774781+0000 mgr.x (mgr.24118) 653 : cluster [DBG] pgmap v921: 17 pgs: 17 active+clean; 8.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 400 KiB/s wr, 808 op/s 2020-09-08T15:49:08.264 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:08.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:49:08 smithi166 bash[14254]: debug 2020-09-08T15:49:08.025+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:08.371 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:49:08.408 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:49:08 smithi055 bash[11956]: debug 2020-09-08T15:49:08.261+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:08.967 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:49:08.967 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2020-09-08T15:49:08.967 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2020-09-08T15:49:08.968 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T15:49:08.968 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.968 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:43780 3.2 3:4163bbab:::benchmark_data_smithi166_26097_object43779:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e224)", 2020-09-08T15:49:08.968 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:42:36.759490+0000", 2020-09-08T15:49:08.969 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 392.20725875900001, 2020-09-08T15:49:08.969 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.41321206799999999, 2020-09-08T15:49:08.969 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:49:08.969 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:08.969 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:49:08.970 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:49:08.970 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:49:08.970 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 43780 2020-09-08T15:49:08.970 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.971 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:49:08.971 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.971 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:49:08.971 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.759490+0000", 2020-09-08T15:49:08.971 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:49:08.972 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.972 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.972 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:49:08.972 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.759490+0000", 2020-09-08T15:49:08.972 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999547 2020-09-08T15:49:08.973 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.973 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.973 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:49:08.973 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.759445+0000", 2020-09-08T15:49:08.973 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014533100000000001 2020-09-08T15:49:08.974 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.975 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.975 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:49:08.975 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.759590+0000", 2020-09-08T15:49:08.975 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6682000000000001e-05 2020-09-08T15:49:08.975 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.976 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.976 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:49:08.976 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.759607+0000", 2020-09-08T15:49:08.976 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7320000000000001e-06 2020-09-08T15:49:08.977 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.977 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.977 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:49:08.977 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.759613+0000", 2020-09-08T15:49:08.977 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6366999999999998e-05 2020-09-08T15:49:08.978 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.978 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.978 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:08.978 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.759670+0000", 2020-09-08T15:49:08.978 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015800100000000001 2020-09-08T15:49:08.979 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.979 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.979 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:49:08.979 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.759828+0000", 2020-09-08T15:49:08.979 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000194437 2020-09-08T15:49:08.980 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.980 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.980 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 9", 2020-09-08T15:49:08.980 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.760022+0000", 2020-09-08T15:49:08.980 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.40266279900000002 2020-09-08T15:49:08.981 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.981 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.981 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:49:08.981 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:37.162685+0000", 2020-09-08T15:49:08.981 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.009906616 2020-09-08T15:49:08.982 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.982 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.982 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:08.982 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:37.172592+0000", 2020-09-08T15:49:08.982 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8109000000000003e-05 2020-09-08T15:49:08.983 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.983 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.983 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:49:08.983 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:37.172640+0000", 2020-09-08T15:49:08.983 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2464000000000004e-05 2020-09-08T15:49:08.983 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.984 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.984 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:49:08.984 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:37.172702+0000", 2020-09-08T15:49:08.984 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.41321206799999999 2020-09-08T15:49:08.984 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:08.985 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:08.985 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:08.985 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.985 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.985 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.35087.0:43778 3.5 e224/214 3:a04cf8c5:::benchmark_data_smithi166_26097_object43777:head v 224'2767, mlcod=224'2765)", 2020-09-08T15:49:08.986 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:42:36.760017+0000", 2020-09-08T15:49:08.986 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 392.20673208699998, 2020-09-08T15:49:08.986 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.40266761400000001, 2020-09-08T15:49:08.986 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:49:08.986 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:08.987 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:49:08.987 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.987 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:49:08.987 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.760017+0000", 2020-09-08T15:49:08.987 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:49:08.988 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.988 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.988 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:49:08.988 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.760017+0000", 2020-09-08T15:49:08.988 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5453999999999998e-05 2020-09-08T15:49:08.989 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.989 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.989 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:49:08.989 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.760112+0000", 2020-09-08T15:49:08.989 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5340000000000002e-06 2020-09-08T15:49:08.989 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.990 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.990 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:49:08.990 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.760120+0000", 2020-09-08T15:49:08.990 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2197e-05 2020-09-08T15:49:08.990 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.991 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.991 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:49:08.991 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.760132+0000", 2020-09-08T15:49:08.991 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1399999999999999e-06 2020-09-08T15:49:08.991 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.992 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.992 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:49:08.992 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.760137+0000", 2020-09-08T15:49:08.993 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7766000000000003e-05 2020-09-08T15:49:08.993 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.993 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.993 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:08.994 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.760185+0000", 2020-09-08T15:49:08.994 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8325000000000002e-05 2020-09-08T15:49:08.994 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.994 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.994 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:49:08.995 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:36.760203+0000", 2020-09-08T15:49:08.995 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.40228635600000001 2020-09-08T15:49:08.995 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.995 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.995 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:49:08.996 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:37.162489+0000", 2020-09-08T15:49:08.996 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000194842 2020-09-08T15:49:08.996 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.996 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.996 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:49:08.997 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:42:37.162684+0000", 2020-09-08T15:49:08.997 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.40266761400000001 2020-09-08T15:49:08.997 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:08.997 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:08.997 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:08.998 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:08.998 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:08.998 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88121 3.c 3:3274f1b2:::benchmark_data_smithi166_26097_object88120:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:49:08.998 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.847115+0000", 2020-09-08T15:49:08.998 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 364.11963374300001, 2020-09-08T15:49:08.999 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.75263644100000004, 2020-09-08T15:49:08.999 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:49:08.999 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:08.999 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:49:08.999 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:49:09.000 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:49:09.000 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88121 2020-09-08T15:49:09.000 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.000 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:49:09.000 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.000 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:49:09.001 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847115+0000", 2020-09-08T15:49:09.001 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:49:09.001 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.001 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.001 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:49:09.002 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847115+0000", 2020-09-08T15:49:09.002 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9998198 2020-09-08T15:49:09.002 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.002 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.002 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:49:09.003 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.846935+0000", 2020-09-08T15:49:09.003 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00042954899999999999 2020-09-08T15:49:09.003 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.003 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.003 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:49:09.004 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847364+0000", 2020-09-08T15:49:09.004 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4828999999999999e-05 2020-09-08T15:49:09.005 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.005 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.005 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:49:09.005 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847389+0000", 2020-09-08T15:49:09.005 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2669e-05 2020-09-08T15:49:09.006 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.006 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.006 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:49:09.006 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847402+0000", 2020-09-08T15:49:09.006 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00049592600000000005 2020-09-08T15:49:09.007 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.007 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.007 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.007 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847898+0000", 2020-09-08T15:49:09.007 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6884999999999998e-05 2020-09-08T15:49:09.008 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.008 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.008 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:49:09.008 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847915+0000", 2020-09-08T15:49:09.008 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.71660728200000001 2020-09-08T15:49:09.009 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.009 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.009 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.009 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.564522+0000", 2020-09-08T15:49:09.009 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0018229349999999999 2020-09-08T15:49:09.010 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.010 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.010 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:49:09.010 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.566345+0000", 2020-09-08T15:49:09.010 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018965599999999999 2020-09-08T15:49:09.011 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.011 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.011 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:49:09.011 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.566535+0000", 2020-09-08T15:49:09.011 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.015621836 2020-09-08T15:49:09.012 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.012 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.012 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.012 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.582156+0000", 2020-09-08T15:49:09.013 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0068253599999999999 2020-09-08T15:49:09.013 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.013 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.013 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:49:09.013 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.588982+0000", 2020-09-08T15:49:09.014 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0088333649999999993 2020-09-08T15:49:09.014 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.014 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.014 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.014 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.597815+0000", 2020-09-08T15:49:09.016 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4142999999999998e-05 2020-09-08T15:49:09.016 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.016 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.017 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:49:09.017 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.597889+0000", 2020-09-08T15:49:09.017 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0018622479999999999 2020-09-08T15:49:09.017 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.018 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.018 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:49:09.018 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.599752+0000", 2020-09-08T15:49:09.018 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.75263644100000004 2020-09-08T15:49:09.018 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.019 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:09.019 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.019 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.019 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.019 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88124 3.c 3:3aadf346:::benchmark_data_smithi166_26097_object88123:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:49:09.020 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.847493+0000", 2020-09-08T15:49:09.020 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 364.11925558299998, 2020-09-08T15:49:09.020 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.75358171299999999, 2020-09-08T15:49:09.020 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:49:09.020 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:09.021 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:49:09.021 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:49:09.021 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:49:09.021 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88124 2020-09-08T15:49:09.022 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.022 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:49:09.022 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.022 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:49:09.023 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847493+0000", 2020-09-08T15:49:09.023 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:49:09.023 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.023 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.023 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:49:09.024 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847493+0000", 2020-09-08T15:49:09.024 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999495 2020-09-08T15:49:09.024 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.024 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.024 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:49:09.025 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847443+0000", 2020-09-08T15:49:09.025 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000182323 2020-09-08T15:49:09.025 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.025 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.025 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:49:09.026 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847625+0000", 2020-09-08T15:49:09.026 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6303e-05 2020-09-08T15:49:09.026 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.026 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.026 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:49:09.027 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847641+0000", 2020-09-08T15:49:09.027 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5647000000000001e-05 2020-09-08T15:49:09.027 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.027 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.027 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:49:09.028 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847657+0000", 2020-09-08T15:49:09.028 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00031383799999999999 2020-09-08T15:49:09.028 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.028 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.028 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.029 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847971+0000", 2020-09-08T15:49:09.029 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2159e-05 2020-09-08T15:49:09.029 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.029 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.029 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:49:09.030 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847983+0000", 2020-09-08T15:49:09.030 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.72442674699999998 2020-09-08T15:49:09.030 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.030 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.030 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.031 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.572410+0000", 2020-09-08T15:49:09.031 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017684800000000001 2020-09-08T15:49:09.031 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.031 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.031 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:49:09.032 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.572587+0000", 2020-09-08T15:49:09.032 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018562599999999999 2020-09-08T15:49:09.032 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.032 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.032 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:49:09.032 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.572772+0000", 2020-09-08T15:49:09.033 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.016268680000000001 2020-09-08T15:49:09.033 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.034 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.034 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.035 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.589041+0000", 2020-09-08T15:49:09.035 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1363999999999998e-05 2020-09-08T15:49:09.035 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.035 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.035 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:49:09.036 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.589092+0000", 2020-09-08T15:49:09.036 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010730368000000001 2020-09-08T15:49:09.036 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.036 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.036 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.036 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.599823+0000", 2020-09-08T15:49:09.037 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2834000000000002e-05 2020-09-08T15:49:09.037 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.037 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.037 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:49:09.037 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.599885+0000", 2020-09-08T15:49:09.038 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0011894430000000001 2020-09-08T15:49:09.038 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.038 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.038 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:49:09.038 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.601075+0000", 2020-09-08T15:49:09.039 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.75358171299999999 2020-09-08T15:49:09.039 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.039 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:09.039 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.039 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.040 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.040 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88161 3.c 3:38648e2f:::benchmark_data_smithi166_26097_object88160:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:49:09.040 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.847735+0000", 2020-09-08T15:49:09.040 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 364.11901339600001, 2020-09-08T15:49:09.040 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.76368154799999999, 2020-09-08T15:49:09.041 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:49:09.041 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:09.041 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:49:09.041 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:49:09.042 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:49:09.043 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88161 2020-09-08T15:49:09.043 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.043 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:49:09.043 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.043 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:49:09.044 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847735+0000", 2020-09-08T15:49:09.044 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:49:09.044 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.044 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.044 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:49:09.045 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847735+0000", 2020-09-08T15:49:09.045 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999537 2020-09-08T15:49:09.045 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.045 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.045 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:49:09.045 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847689+0000", 2020-09-08T15:49:09.046 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015866 2020-09-08T15:49:09.046 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.046 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.046 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:49:09.047 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847848+0000", 2020-09-08T15:49:09.047 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5380000000000002e-06 2020-09-08T15:49:09.047 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.047 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.047 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:49:09.048 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847856+0000", 2020-09-08T15:49:09.048 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9319999999999998e-06 2020-09-08T15:49:09.048 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.048 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.049 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:49:09.049 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847860+0000", 2020-09-08T15:49:09.049 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015259200000000001 2020-09-08T15:49:09.049 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.049 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.050 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.050 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848013+0000", 2020-09-08T15:49:09.050 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0889e-05 2020-09-08T15:49:09.050 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.050 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.050 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:49:09.051 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848024+0000", 2020-09-08T15:49:09.051 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.72736553100000001 2020-09-08T15:49:09.051 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.051 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.051 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.052 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.575389+0000", 2020-09-08T15:49:09.052 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00037211999999999998 2020-09-08T15:49:09.052 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.052 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.053 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:49:09.053 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.575761+0000", 2020-09-08T15:49:09.053 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00042625200000000001 2020-09-08T15:49:09.053 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.053 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.054 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:49:09.054 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.576187+0000", 2020-09-08T15:49:09.054 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.012893685 2020-09-08T15:49:09.054 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.054 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.054 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.055 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.589081+0000", 2020-09-08T15:49:09.055 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0710999999999999e-05 2020-09-08T15:49:09.055 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.055 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.055 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:49:09.056 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.589102+0000", 2020-09-08T15:49:09.056 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.020542207999999999 2020-09-08T15:49:09.056 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.056 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.056 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.057 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.609644+0000", 2020-09-08T15:49:09.057 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1699000000000007e-05 2020-09-08T15:49:09.057 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.057 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.057 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:49:09.058 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.609706+0000", 2020-09-08T15:49:09.058 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001711202 2020-09-08T15:49:09.058 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.058 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.058 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:49:09.059 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.611417+0000", 2020-09-08T15:49:09.059 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.76368154799999999 2020-09-08T15:49:09.059 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.059 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:09.059 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.060 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.060 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.060 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88165 3.c 3:3ddcf4f9:::benchmark_data_smithi166_26097_object88164:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:49:09.060 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.847901+0000", 2020-09-08T15:49:09.060 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 364.118847974, 2020-09-08T15:49:09.061 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.77547376999999995, 2020-09-08T15:49:09.061 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:49:09.061 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:09.061 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:49:09.061 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:49:09.062 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:49:09.062 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88165 2020-09-08T15:49:09.062 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.062 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:49:09.062 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.063 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:49:09.063 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847901+0000", 2020-09-08T15:49:09.063 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:49:09.063 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.064 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.064 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:49:09.064 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847901+0000", 2020-09-08T15:49:09.064 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999762 2020-09-08T15:49:09.064 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.065 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.065 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:49:09.065 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.847877+0000", 2020-09-08T15:49:09.065 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000129016 2020-09-08T15:49:09.066 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.066 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.066 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:49:09.066 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848006+0000", 2020-09-08T15:49:09.066 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2859999999999999e-06 2020-09-08T15:49:09.066 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.067 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.067 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:49:09.067 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848014+0000", 2020-09-08T15:49:09.067 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6950000000000002e-06 2020-09-08T15:49:09.067 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.068 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.068 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:49:09.068 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848018+0000", 2020-09-08T15:49:09.068 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7629999999999997e-05 2020-09-08T15:49:09.068 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.069 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.069 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.069 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848056+0000", 2020-09-08T15:49:09.069 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4609999999999997e-06 2020-09-08T15:49:09.069 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.070 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.070 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:49:09.070 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848065+0000", 2020-09-08T15:49:09.070 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.73443109100000004 2020-09-08T15:49:09.070 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.071 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.071 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.071 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.582496+0000", 2020-09-08T15:49:09.071 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000291484 2020-09-08T15:49:09.071 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.071 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.072 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:49:09.072 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.582788+0000", 2020-09-08T15:49:09.072 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00036355799999999999 2020-09-08T15:49:09.072 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.072 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.073 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:49:09.073 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.583151+0000", 2020-09-08T15:49:09.073 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0059636890000000003 2020-09-08T15:49:09.073 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.073 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.074 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:49:09.074 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.589115+0000", 2020-09-08T15:49:09.074 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.012338839000000001 2020-09-08T15:49:09.074 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.074 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.075 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.075 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.601454+0000", 2020-09-08T15:49:09.075 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.019415953999999999 2020-09-08T15:49:09.075 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.075 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.075 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.076 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.620870+0000", 2020-09-08T15:49:09.076 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000130216 2020-09-08T15:49:09.076 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.076 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.076 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:49:09.077 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.621000+0000", 2020-09-08T15:49:09.077 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002374733 2020-09-08T15:49:09.077 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.077 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.077 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:49:09.078 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.623375+0000", 2020-09-08T15:49:09.078 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.77547376999999995 2020-09-08T15:49:09.078 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.078 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:09.078 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.078 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.079 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.079 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88180 3.c 3:39dfcfe5:::benchmark_data_smithi166_26097_object88179:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:49:09.079 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.848069+0000", 2020-09-08T15:49:09.079 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 364.11867937599999, 2020-09-08T15:49:09.079 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.77707152300000004, 2020-09-08T15:49:09.080 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:49:09.080 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:09.080 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:49:09.080 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:49:09.080 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:49:09.081 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88180 2020-09-08T15:49:09.081 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.081 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:49:09.081 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.081 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:49:09.082 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848069+0000", 2020-09-08T15:49:09.082 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:49:09.082 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.082 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.082 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:49:09.083 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848069+0000", 2020-09-08T15:49:09.083 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999728 2020-09-08T15:49:09.083 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.083 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.083 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:49:09.083 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848042+0000", 2020-09-08T15:49:09.084 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000114635 2020-09-08T15:49:09.084 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.084 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.084 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:49:09.084 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848157+0000", 2020-09-08T15:49:09.085 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6072000000000001e-05 2020-09-08T15:49:09.085 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.085 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.086 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:49:09.086 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848173+0000", 2020-09-08T15:49:09.086 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.079e-06 2020-09-08T15:49:09.086 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.086 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.087 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:49:09.087 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848178+0000", 2020-09-08T15:49:09.087 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4790999999999999e-05 2020-09-08T15:49:09.087 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.087 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.088 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.088 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848223+0000", 2020-09-08T15:49:09.088 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0448999999999999e-05 2020-09-08T15:49:09.088 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.088 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.089 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:49:09.089 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848233+0000", 2020-09-08T15:49:09.089 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.73823174300000005 2020-09-08T15:49:09.089 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.089 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.090 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.090 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.586465+0000", 2020-09-08T15:49:09.090 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000170996 2020-09-08T15:49:09.090 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.090 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.091 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:49:09.091 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.586636+0000", 2020-09-08T15:49:09.091 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000164415 2020-09-08T15:49:09.091 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.092 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.092 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:49:09.092 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.586800+0000", 2020-09-08T15:49:09.092 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010925416 2020-09-08T15:49:09.093 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.093 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.093 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:49:09.093 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.597726+0000", 2020-09-08T15:49:09.093 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.011842284999999999 2020-09-08T15:49:09.094 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.094 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.094 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.094 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.609568+0000", 2020-09-08T15:49:09.095 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013896431000000001 2020-09-08T15:49:09.095 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.095 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.095 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.095 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.623465+0000", 2020-09-08T15:49:09.096 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5380000000000001e-05 2020-09-08T15:49:09.096 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.096 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.096 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:49:09.096 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.623530+0000", 2020-09-08T15:49:09.097 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001610956 2020-09-08T15:49:09.097 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.097 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.097 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:49:09.097 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.625141+0000", 2020-09-08T15:49:09.098 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.77707152300000004 2020-09-08T15:49:09.098 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.098 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:09.098 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.098 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.099 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.099 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88185 3.c 3:32cc51c5:::benchmark_data_smithi166_26097_object88184:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:49:09.099 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.848234+0000", 2020-09-08T15:49:09.099 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 364.11851468700002, 2020-09-08T15:49:09.099 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.78053202799999999, 2020-09-08T15:49:09.099 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:49:09.100 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:09.100 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:49:09.100 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:49:09.100 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:49:09.100 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88185 2020-09-08T15:49:09.101 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.101 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:49:09.101 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.101 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:49:09.101 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848234+0000", 2020-09-08T15:49:09.102 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:49:09.102 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.102 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.102 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:49:09.102 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848234+0000", 2020-09-08T15:49:09.103 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999728 2020-09-08T15:49:09.103 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.103 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.103 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:49:09.103 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848207+0000", 2020-09-08T15:49:09.104 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010272000000000001 2020-09-08T15:49:09.104 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.104 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.104 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:49:09.104 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848310+0000", 2020-09-08T15:49:09.105 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0137e-05 2020-09-08T15:49:09.105 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.105 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.105 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:49:09.105 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848320+0000", 2020-09-08T15:49:09.106 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.535e-06 2020-09-08T15:49:09.106 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.106 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.106 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:49:09.106 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848324+0000", 2020-09-08T15:49:09.107 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5059999999999999e-05 2020-09-08T15:49:09.107 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.107 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.107 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.107 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848369+0000", 2020-09-08T15:49:09.108 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.044e-05 2020-09-08T15:49:09.108 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.108 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.108 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:49:09.108 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848379+0000", 2020-09-08T15:49:09.108 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.74076494900000001 2020-09-08T15:49:09.109 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.109 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.109 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.109 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.589144+0000", 2020-09-08T15:49:09.109 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000137644 2020-09-08T15:49:09.110 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.110 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.110 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:49:09.110 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.589282+0000", 2020-09-08T15:49:09.110 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014025999999999999 2020-09-08T15:49:09.111 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.111 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.111 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:49:09.111 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.589422+0000", 2020-09-08T15:49:09.111 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0083156619999999997 2020-09-08T15:49:09.112 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.112 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.112 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:49:09.112 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.597738+0000", 2020-09-08T15:49:09.112 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.027475880000000001 2020-09-08T15:49:09.112 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.113 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.113 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.113 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.625213+0000", 2020-09-08T15:49:09.113 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0021174570000000001 2020-09-08T15:49:09.113 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.114 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.114 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.114 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.627331+0000", 2020-09-08T15:49:09.114 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2708999999999999e-05 2020-09-08T15:49:09.114 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.115 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.115 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:49:09.115 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.627394+0000", 2020-09-08T15:49:09.115 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001372559 2020-09-08T15:49:09.116 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.116 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.116 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:49:09.116 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.628766+0000", 2020-09-08T15:49:09.117 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.78053202799999999 2020-09-08T15:49:09.117 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.117 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:09.117 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.117 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.118 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.118 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88206 3.c 3:3799269c:::benchmark_data_smithi166_26097_object88205:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:49:09.118 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.848395+0000", 2020-09-08T15:49:09.118 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 364.11835414900003, 2020-09-08T15:49:09.118 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.79597148600000001, 2020-09-08T15:49:09.119 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:49:09.119 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:09.119 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:49:09.119 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:49:09.119 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:49:09.120 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88206 2020-09-08T15:49:09.120 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.120 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:49:09.120 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.120 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:49:09.121 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848395+0000", 2020-09-08T15:49:09.121 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:49:09.121 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.122 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.122 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:49:09.122 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848395+0000", 2020-09-08T15:49:09.122 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999628 2020-09-08T15:49:09.122 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.122 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.123 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:49:09.123 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848357+0000", 2020-09-08T15:49:09.123 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000106314 2020-09-08T15:49:09.123 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.123 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.124 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:49:09.124 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848464+0000", 2020-09-08T15:49:09.124 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0295000000000001e-05 2020-09-08T15:49:09.124 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.124 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.125 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:49:09.125 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848474+0000", 2020-09-08T15:49:09.125 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.568e-06 2020-09-08T15:49:09.125 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.125 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.126 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:49:09.126 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848478+0000", 2020-09-08T15:49:09.126 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6579999999999997e-05 2020-09-08T15:49:09.126 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.126 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.127 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.127 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848534+0000", 2020-09-08T15:49:09.127 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2971e-05 2020-09-08T15:49:09.127 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.127 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.127 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:49:09.128 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848547+0000", 2020-09-08T15:49:09.128 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.74238484699999996 2020-09-08T15:49:09.128 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.128 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.129 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.129 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.590932+0000", 2020-09-08T15:49:09.129 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000127992 2020-09-08T15:49:09.129 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.129 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.130 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:49:09.130 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.591060+0000", 2020-09-08T15:49:09.130 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000138567 2020-09-08T15:49:09.130 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.130 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.130 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:49:09.131 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.591199+0000", 2020-09-08T15:49:09.131 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010183153 2020-09-08T15:49:09.131 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.131 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.131 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:49:09.132 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.601382+0000", 2020-09-08T15:49:09.132 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.023873466999999999 2020-09-08T15:49:09.132 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.132 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.132 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.133 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.625255+0000", 2020-09-08T15:49:09.133 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.015987699000000001 2020-09-08T15:49:09.133 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.133 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.133 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.134 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.641243+0000", 2020-09-08T15:49:09.134 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012196700000000001 2020-09-08T15:49:09.134 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.134 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.134 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:49:09.135 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.641365+0000", 2020-09-08T15:49:09.135 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0030013349999999999 2020-09-08T15:49:09.135 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.135 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.135 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:49:09.136 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.644366+0000", 2020-09-08T15:49:09.136 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.79597148600000001 2020-09-08T15:49:09.136 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.136 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:09.136 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.137 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.137 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.137 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88213 3.c 3:3d413ed8:::benchmark_data_smithi166_26097_object88212:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:49:09.137 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.848546+0000", 2020-09-08T15:49:09.137 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 364.11820329599999, 2020-09-08T15:49:09.138 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.79807563500000001, 2020-09-08T15:49:09.138 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:49:09.138 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:09.138 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:49:09.138 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:49:09.139 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:49:09.139 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88213 2020-09-08T15:49:09.139 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.139 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:49:09.139 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.140 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:49:09.140 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848546+0000", 2020-09-08T15:49:09.140 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:49:09.140 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.140 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.141 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:49:09.141 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848546+0000", 2020-09-08T15:49:09.141 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999743 2020-09-08T15:49:09.141 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.142 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.142 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:49:09.142 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848520+0000", 2020-09-08T15:49:09.142 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1861999999999996e-05 2020-09-08T15:49:09.143 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.143 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.143 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:49:09.143 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848612+0000", 2020-09-08T15:49:09.143 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0196e-05 2020-09-08T15:49:09.144 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.144 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.144 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:49:09.144 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848622+0000", 2020-09-08T15:49:09.144 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.49e-06 2020-09-08T15:49:09.145 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.145 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.145 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:49:09.145 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848625+0000", 2020-09-08T15:49:09.145 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0156e-05 2020-09-08T15:49:09.146 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.146 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.146 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.146 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848666+0000", 2020-09-08T15:49:09.146 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0519e-05 2020-09-08T15:49:09.146 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.147 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.147 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:49:09.147 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848676+0000", 2020-09-08T15:49:09.147 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.74410123500000003 2020-09-08T15:49:09.147 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.148 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.148 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.148 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.592777+0000", 2020-09-08T15:49:09.148 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013514399999999999 2020-09-08T15:49:09.148 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.149 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.149 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:49:09.149 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.592912+0000", 2020-09-08T15:49:09.149 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014323800000000001 2020-09-08T15:49:09.149 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.149 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.150 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:49:09.150 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.593056+0000", 2020-09-08T15:49:09.150 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0083361330000000008 2020-09-08T15:49:09.150 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.150 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.151 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:49:09.151 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.601392+0000", 2020-09-08T15:49:09.152 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.030220244 2020-09-08T15:49:09.152 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.152 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.152 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.153 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.631612+0000", 2020-09-08T15:49:09.153 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.012851883 2020-09-08T15:49:09.153 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.154 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.154 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.154 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.644464+0000", 2020-09-08T15:49:09.156 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012559800000000001 2020-09-08T15:49:09.156 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.156 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.156 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:49:09.157 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.644589+0000", 2020-09-08T15:49:09.157 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020316470000000001 2020-09-08T15:49:09.157 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.157 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.157 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:49:09.158 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.646621+0000", 2020-09-08T15:49:09.158 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.79807563500000001 2020-09-08T15:49:09.158 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.158 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:09.158 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.159 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.159 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.159 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88217 3.c 3:30993993:::benchmark_data_smithi166_26097_object88216:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:49:09.159 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.848691+0000", 2020-09-08T15:49:09.159 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 364.11805741799998, 2020-09-08T15:49:09.160 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.80980842799999997, 2020-09-08T15:49:09.160 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:49:09.160 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:09.160 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:49:09.160 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:49:09.161 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:49:09.161 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88217 2020-09-08T15:49:09.161 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.162 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:49:09.162 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.162 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:49:09.162 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848691+0000", 2020-09-08T15:49:09.162 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:49:09.163 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.163 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.163 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:49:09.163 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848691+0000", 2020-09-08T15:49:09.164 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999642 2020-09-08T15:49:09.164 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.166 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:08 smithi055 bash[9296]: cluster 2020-09-08T15:49:07.775436+0000 mgr.x (mgr.24118) 654 : cluster [DBG] pgmap v922: 17 pgs: 17 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 887 op/s 2020-09-08T15:49:09.166 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:08 smithi166 bash[9446]: cluster 2020-09-08T15:49:07.775436+0000 mgr.x (mgr.24118) 654 : cluster [DBG] pgmap v922: 17 pgs: 17 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 887 op/s 2020-09-08T15:49:09.166 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.166 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:49:09.167 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848656+0000", 2020-09-08T15:49:09.168 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010443 2020-09-08T15:49:09.168 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.168 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.168 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:49:09.168 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848760+0000", 2020-09-08T15:49:09.169 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0158e-05 2020-09-08T15:49:09.169 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.169 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.169 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:49:09.170 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848770+0000", 2020-09-08T15:49:09.170 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3409999999999999e-06 2020-09-08T15:49:09.170 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.170 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.170 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:49:09.170 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848774+0000", 2020-09-08T15:49:09.171 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5544000000000003e-05 2020-09-08T15:49:09.171 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.171 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.171 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.171 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848819+0000", 2020-09-08T15:49:09.172 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2342e-05 2020-09-08T15:49:09.172 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.172 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.172 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:49:09.172 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848832+0000", 2020-09-08T15:49:09.173 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.746899388 2020-09-08T15:49:09.173 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.173 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.173 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.173 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.595731+0000", 2020-09-08T15:49:09.174 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000158994 2020-09-08T15:49:09.174 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.174 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.174 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:49:09.174 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.595890+0000", 2020-09-08T15:49:09.175 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000178278 2020-09-08T15:49:09.175 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.175 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.175 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:49:09.175 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.596068+0000", 2020-09-08T15:49:09.176 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0077743609999999996 2020-09-08T15:49:09.176 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.176 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.176 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:49:09.176 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.603843+0000", 2020-09-08T15:49:09.177 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.027843777 2020-09-08T15:49:09.177 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.177 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.177 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.178 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.631686+0000", 2020-09-08T15:49:09.178 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.025082657000000001 2020-09-08T15:49:09.178 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.178 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.178 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.179 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.656769+0000", 2020-09-08T15:49:09.179 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011135 2020-09-08T15:49:09.179 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.179 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.179 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:49:09.180 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.656880+0000", 2020-09-08T15:49:09.180 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0016193970000000001 2020-09-08T15:49:09.180 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.180 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.180 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:49:09.181 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.658500+0000", 2020-09-08T15:49:09.181 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.80980842799999997 2020-09-08T15:49:09.181 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.181 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:09.181 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.182 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.182 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.182 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88229 3.c 3:3fa93278:::benchmark_data_smithi166_26097_object88228:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:49:09.182 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.848825+0000", 2020-09-08T15:49:09.182 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 364.11792351299999, 2020-09-08T15:49:09.183 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.81162889800000004, 2020-09-08T15:49:09.183 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:49:09.183 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:09.183 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:49:09.183 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:49:09.184 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:49:09.184 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88229 2020-09-08T15:49:09.184 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.184 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:49:09.184 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.184 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:49:09.185 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848825+0000", 2020-09-08T15:49:09.185 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:49:09.185 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.185 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.185 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:49:09.186 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848825+0000", 2020-09-08T15:49:09.186 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999728 2020-09-08T15:49:09.186 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.186 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.186 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:49:09.187 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848798+0000", 2020-09-08T15:49:09.187 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5902000000000004e-05 2020-09-08T15:49:09.187 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.187 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.187 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:49:09.188 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848894+0000", 2020-09-08T15:49:09.188 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9150000000000008e-06 2020-09-08T15:49:09.188 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.188 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.188 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:49:09.189 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848904+0000", 2020-09-08T15:49:09.189 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1769999999999998e-06 2020-09-08T15:49:09.189 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.189 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.189 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:49:09.189 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848907+0000", 2020-09-08T15:49:09.190 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8699999999999999e-05 2020-09-08T15:49:09.190 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.190 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.190 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.190 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848946+0000", 2020-09-08T15:49:09.191 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9720000000000004e-06 2020-09-08T15:49:09.191 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.191 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.191 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:49:09.191 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848956+0000", 2020-09-08T15:49:09.192 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.75253392399999997 2020-09-08T15:49:09.192 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.192 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.192 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.192 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.601490+0000", 2020-09-08T15:49:09.193 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014808099999999999 2020-09-08T15:49:09.193 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.193 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.193 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:49:09.193 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.601638+0000", 2020-09-08T15:49:09.194 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017448400000000001 2020-09-08T15:49:09.194 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.194 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.194 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:49:09.195 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.601812+0000", 2020-09-08T15:49:09.195 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0077719160000000002 2020-09-08T15:49:09.195 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.196 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.196 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:49:09.196 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.609584+0000", 2020-09-08T15:49:09.196 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.027487385 2020-09-08T15:49:09.196 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.196 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.197 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.197 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.637072+0000", 2020-09-08T15:49:09.197 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.021532775000000001 2020-09-08T15:49:09.197 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.197 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.198 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.198 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.658604+0000", 2020-09-08T15:49:09.198 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000112063 2020-09-08T15:49:09.198 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.198 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.199 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:49:09.199 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.658716+0000", 2020-09-08T15:49:09.199 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001737717 2020-09-08T15:49:09.199 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.199 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.200 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:49:09.200 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.660454+0000", 2020-09-08T15:49:09.200 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.81162889800000004 2020-09-08T15:49:09.200 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.200 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:09.200 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.201 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.201 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.201 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88244 3.c 3:32754c75:::benchmark_data_smithi166_26097_object88243:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:49:09.201 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.848971+0000", 2020-09-08T15:49:09.201 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 364.11777795, 2020-09-08T15:49:09.202 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83018217599999999, 2020-09-08T15:49:09.202 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:49:09.202 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:09.202 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:49:09.202 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:49:09.203 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:49:09.203 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88244 2020-09-08T15:49:09.203 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.203 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:49:09.203 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.204 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:49:09.204 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848971+0000", 2020-09-08T15:49:09.204 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:49:09.204 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.204 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.205 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:49:09.205 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848971+0000", 2020-09-08T15:49:09.205 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999638 2020-09-08T15:49:09.205 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.206 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.206 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:49:09.206 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.848935+0000", 2020-09-08T15:49:09.206 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000103946 2020-09-08T15:49:09.206 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.206 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.207 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:49:09.207 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849039+0000", 2020-09-08T15:49:09.207 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3365e-05 2020-09-08T15:49:09.207 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.207 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.208 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:49:09.208 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849052+0000", 2020-09-08T15:49:09.208 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4489999999999999e-06 2020-09-08T15:49:09.208 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.208 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.209 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:49:09.209 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849056+0000", 2020-09-08T15:49:09.209 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5677999999999999e-05 2020-09-08T15:49:09.209 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.209 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.210 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.210 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849101+0000", 2020-09-08T15:49:09.210 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2026e-05 2020-09-08T15:49:09.210 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.210 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.210 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:49:09.211 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849113+0000", 2020-09-08T15:49:09.211 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.754773735 2020-09-08T15:49:09.211 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.211 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.211 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.212 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.603887+0000", 2020-09-08T15:49:09.212 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016359499999999999 2020-09-08T15:49:09.212 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.212 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.212 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:49:09.213 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.604051+0000", 2020-09-08T15:49:09.213 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000182654 2020-09-08T15:49:09.213 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.213 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.213 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:49:09.214 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.604233+0000", 2020-09-08T15:49:09.214 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0053636939999999996 2020-09-08T15:49:09.214 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.214 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.214 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:49:09.214 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.609597+0000", 2020-09-08T15:49:09.215 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.02768924 2020-09-08T15:49:09.215 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.215 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.215 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.215 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.637286+0000", 2020-09-08T15:49:09.216 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.039558932999999998 2020-09-08T15:49:09.216 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.216 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.216 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.216 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.676845+0000", 2020-09-08T15:49:09.217 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8937000000000006e-05 2020-09-08T15:49:09.218 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.218 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.218 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:49:09.219 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.676934+0000", 2020-09-08T15:49:09.219 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0022190230000000001 2020-09-08T15:49:09.219 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.219 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.219 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:49:09.220 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.679153+0000", 2020-09-08T15:49:09.220 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83018217599999999 2020-09-08T15:49:09.220 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.220 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:09.220 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.221 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.221 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.221 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88252 3.c 3:3ca3108e:::benchmark_data_smithi166_26097_object88251:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:49:09.221 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.849107+0000", 2020-09-08T15:49:09.221 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 364.11764196600001, 2020-09-08T15:49:09.222 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83242581000000004, 2020-09-08T15:49:09.222 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:49:09.222 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:09.222 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:49:09.222 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:49:09.223 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:49:09.223 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88252 2020-09-08T15:49:09.223 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.223 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:49:09.223 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.224 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:49:09.224 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849107+0000", 2020-09-08T15:49:09.224 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:49:09.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.224 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.224 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:49:09.225 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849107+0000", 2020-09-08T15:49:09.225 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999733 2020-09-08T15:49:09.225 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.225 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.225 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:49:09.226 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849080+0000", 2020-09-08T15:49:09.226 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3986000000000004e-05 2020-09-08T15:49:09.226 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.226 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.226 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:49:09.227 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849174+0000", 2020-09-08T15:49:09.227 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7790000000000007e-06 2020-09-08T15:49:09.227 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.227 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.227 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:49:09.228 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849184+0000", 2020-09-08T15:49:09.228 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.14e-06 2020-09-08T15:49:09.228 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.228 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.228 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:49:09.228 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849187+0000", 2020-09-08T15:49:09.229 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7882000000000001e-05 2020-09-08T15:49:09.229 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.229 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.229 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.229 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849225+0000", 2020-09-08T15:49:09.230 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0234999999999999e-05 2020-09-08T15:49:09.230 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.230 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.230 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:49:09.230 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849235+0000", 2020-09-08T15:49:09.231 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.75657893499999995 2020-09-08T15:49:09.231 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.231 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.231 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.231 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.605814+0000", 2020-09-08T15:49:09.232 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015115199999999999 2020-09-08T15:49:09.232 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.232 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.232 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:49:09.232 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.605965+0000", 2020-09-08T15:49:09.233 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000181583 2020-09-08T15:49:09.233 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.233 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.233 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:49:09.233 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.606147+0000", 2020-09-08T15:49:09.234 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0086315590000000005 2020-09-08T15:49:09.234 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.234 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.234 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:49:09.234 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.614778+0000", 2020-09-08T15:49:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.031948953000000002 2020-09-08T15:49:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.646727+0000", 2020-09-08T15:49:09.236 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.032630385999999997 2020-09-08T15:49:09.236 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.236 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.236 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.236 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.679358+0000", 2020-09-08T15:49:09.236 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7763000000000006e-05 2020-09-08T15:49:09.237 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.237 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.237 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:49:09.237 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.679455+0000", 2020-09-08T15:49:09.237 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020773549999999999 2020-09-08T15:49:09.238 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.238 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.238 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:49:09.238 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.681533+0000", 2020-09-08T15:49:09.238 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83242581000000004 2020-09-08T15:49:09.239 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.239 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:09.239 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.239 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.239 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.240 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88264 3.c 3:3a7639d5:::benchmark_data_smithi166_26097_object88263:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:49:09.240 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.849254+0000", 2020-09-08T15:49:09.240 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 364.11749516100002, 2020-09-08T15:49:09.240 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83457435899999999, 2020-09-08T15:49:09.240 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:49:09.241 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:09.241 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:49:09.241 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:49:09.241 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:49:09.241 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88264 2020-09-08T15:49:09.242 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.242 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:49:09.242 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.242 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:49:09.242 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849254+0000", 2020-09-08T15:49:09.243 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:49:09.243 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.243 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.243 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:49:09.243 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849254+0000", 2020-09-08T15:49:09.244 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999599 2020-09-08T15:49:09.244 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.244 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.244 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:49:09.244 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849214+0000", 2020-09-08T15:49:09.245 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000108477 2020-09-08T15:49:09.245 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.245 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.245 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:49:09.245 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849322+0000", 2020-09-08T15:49:09.246 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0156999999999999e-05 2020-09-08T15:49:09.251 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.251 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.252 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:49:09.252 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849332+0000", 2020-09-08T15:49:09.252 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2880000000000001e-06 2020-09-08T15:49:09.252 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.254 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.255 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:49:09.256 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849336+0000", 2020-09-08T15:49:09.256 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5487e-05 2020-09-08T15:49:09.256 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.257 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.257 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.257 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849381+0000", 2020-09-08T15:49:09.257 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2632e-05 2020-09-08T15:49:09.257 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.258 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.258 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:49:09.258 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849394+0000", 2020-09-08T15:49:09.258 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.75832558800000005 2020-09-08T15:49:09.258 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.259 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.259 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.259 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.607719+0000", 2020-09-08T15:49:09.259 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015323299999999999 2020-09-08T15:49:09.259 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.260 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.260 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:49:09.260 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.607872+0000", 2020-09-08T15:49:09.260 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017337799999999999 2020-09-08T15:49:09.260 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.261 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.261 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:49:09.261 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.608046+0000", 2020-09-08T15:49:09.261 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0095075239999999998 2020-09-08T15:49:09.261 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.262 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.262 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:49:09.263 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.617553+0000", 2020-09-08T15:49:09.263 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.036108202999999998 2020-09-08T15:49:09.263 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.263 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.263 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.264 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.653662+0000", 2020-09-08T15:49:09.264 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.028012091999999999 2020-09-08T15:49:09.264 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.264 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.264 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.265 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.681674+0000", 2020-09-08T15:49:09.265 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5779999999999995e-05 2020-09-08T15:49:09.265 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.265 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.265 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:49:09.266 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.681759+0000", 2020-09-08T15:49:09.266 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020685320000000001 2020-09-08T15:49:09.266 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.266 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.266 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:49:09.267 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.683828+0000", 2020-09-08T15:49:09.267 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83457435899999999 2020-09-08T15:49:09.267 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.268 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:09.268 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.268 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.269 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.269 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88277 3.c 3:3b3cc411:::benchmark_data_smithi166_26097_object88276:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:49:09.269 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.849387+0000", 2020-09-08T15:49:09.269 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 364.11736165000002, 2020-09-08T15:49:09.270 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83678120199999995, 2020-09-08T15:49:09.270 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:49:09.270 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:09.270 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:49:09.270 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:49:09.271 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:49:09.271 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88277 2020-09-08T15:49:09.271 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.271 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:49:09.271 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.272 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:49:09.272 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849387+0000", 2020-09-08T15:49:09.272 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:49:09.272 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.272 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.273 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:49:09.273 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849387+0000", 2020-09-08T15:49:09.273 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999719 2020-09-08T15:49:09.273 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.273 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.274 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:49:09.274 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849359+0000", 2020-09-08T15:49:09.274 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3432999999999998e-05 2020-09-08T15:49:09.274 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.274 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.275 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:49:09.275 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849453+0000", 2020-09-08T15:49:09.275 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8660000000000001e-06 2020-09-08T15:49:09.275 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.276 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.276 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:49:09.276 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849462+0000", 2020-09-08T15:49:09.277 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.287e-06 2020-09-08T15:49:09.277 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.278 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.278 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:49:09.278 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849466+0000", 2020-09-08T15:49:09.278 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8856000000000001e-05 2020-09-08T15:49:09.278 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.279 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.279 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.279 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849515+0000", 2020-09-08T15:49:09.279 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0053e-05 2020-09-08T15:49:09.279 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.280 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.280 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:49:09.280 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849525+0000", 2020-09-08T15:49:09.280 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.76201480799999999 2020-09-08T15:49:09.280 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.281 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.281 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.281 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.611539+0000", 2020-09-08T15:49:09.281 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00026357000000000003 2020-09-08T15:49:09.281 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.281 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.282 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:49:09.282 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.611803+0000", 2020-09-08T15:49:09.282 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00029939000000000002 2020-09-08T15:49:09.282 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.282 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.283 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:49:09.283 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.612102+0000", 2020-09-08T15:49:09.283 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013058955000000001 2020-09-08T15:49:09.283 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.283 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.284 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:49:09.284 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.625161+0000", 2020-09-08T15:49:09.284 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.051603609000000002 2020-09-08T15:49:09.284 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.284 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.285 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.285 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.676765+0000", 2020-09-08T15:49:09.285 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0073178669999999996 2020-09-08T15:49:09.285 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.285 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.286 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.286 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.684083+0000", 2020-09-08T15:49:09.286 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6081000000000005e-05 2020-09-08T15:49:09.286 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.286 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.287 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:49:09.287 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.684169+0000", 2020-09-08T15:49:09.287 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0019995009999999999 2020-09-08T15:49:09.287 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.287 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.287 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:49:09.288 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.686168+0000", 2020-09-08T15:49:09.288 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83678120199999995 2020-09-08T15:49:09.288 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.288 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:09.288 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.289 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.289 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.289 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88294 3.c 3:3f7cd38d:::benchmark_data_smithi166_26097_object88293:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:49:09.289 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.849540+0000", 2020-09-08T15:49:09.289 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 364.11720930000001, 2020-09-08T15:49:09.290 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.84994455800000002, 2020-09-08T15:49:09.290 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:49:09.290 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:09.290 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:49:09.290 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:49:09.291 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:49:09.291 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88294 2020-09-08T15:49:09.291 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.291 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:49:09.291 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.292 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:49:09.292 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849540+0000", 2020-09-08T15:49:09.292 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:49:09.292 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.292 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.293 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:49:09.293 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849540+0000", 2020-09-08T15:49:09.293 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999614 2020-09-08T15:49:09.293 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.293 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.294 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:49:09.294 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849501+0000", 2020-09-08T15:49:09.294 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000103999 2020-09-08T15:49:09.294 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.294 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.295 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:49:09.295 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849605+0000", 2020-09-08T15:49:09.295 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0275000000000001e-05 2020-09-08T15:49:09.296 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.296 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.297 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:49:09.297 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849615+0000", 2020-09-08T15:49:09.298 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5099999999999999e-06 2020-09-08T15:49:09.298 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.298 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.298 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:49:09.298 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849619+0000", 2020-09-08T15:49:09.299 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3949999999999998e-05 2020-09-08T15:49:09.299 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.299 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.299 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.299 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849663+0000", 2020-09-08T15:49:09.300 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2666999999999999e-05 2020-09-08T15:49:09.300 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.300 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.300 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:49:09.300 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849675+0000", 2020-09-08T15:49:09.301 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.76514351999999997 2020-09-08T15:49:09.301 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.301 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.301 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.301 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.614819+0000", 2020-09-08T15:49:09.302 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00022547600000000001 2020-09-08T15:49:09.302 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.302 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.302 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:49:09.302 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.615044+0000", 2020-09-08T15:49:09.303 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00023015000000000001 2020-09-08T15:49:09.303 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.303 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.303 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:49:09.303 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.615275+0000", 2020-09-08T15:49:09.304 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0099885300000000007 2020-09-08T15:49:09.304 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.304 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.304 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:49:09.304 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.625263+0000", 2020-09-08T15:49:09.304 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.053987439999999998 2020-09-08T15:49:09.305 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.305 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.305 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.305 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.679251+0000", 2020-09-08T15:49:09.305 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.017811098000000001 2020-09-08T15:49:09.306 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.306 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.306 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.306 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.697062+0000", 2020-09-08T15:49:09.307 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3387999999999995e-05 2020-09-08T15:49:09.307 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.307 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.307 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:49:09.307 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.697145+0000", 2020-09-08T15:49:09.308 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0023390310000000001 2020-09-08T15:49:09.308 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.308 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.308 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:49:09.308 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.699484+0000", 2020-09-08T15:49:09.309 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.84994455800000002 2020-09-08T15:49:09.309 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.309 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:09.309 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.309 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.310 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.310 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:88298 3.c 3:3f320a61:::benchmark_data_smithi166_26097_object88297:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e236)", 2020-09-08T15:49:09.310 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:43:04.849669+0000", 2020-09-08T15:49:09.310 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 364.11707971300001, 2020-09-08T15:49:09.310 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.85250447500000004, 2020-09-08T15:49:09.311 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:49:09.311 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:09.311 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:49:09.312 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:49:09.312 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:49:09.312 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 88298 2020-09-08T15:49:09.312 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.312 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:49:09.313 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.313 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:49:09.313 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849669+0000", 2020-09-08T15:49:09.313 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:49:09.314 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.314 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.314 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:49:09.314 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849669+0000", 2020-09-08T15:49:09.314 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999728 2020-09-08T15:49:09.315 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.315 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.315 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:49:09.315 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849642+0000", 2020-09-08T15:49:09.315 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000112197 2020-09-08T15:49:09.316 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.316 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.316 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:49:09.316 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849754+0000", 2020-09-08T15:49:09.316 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7410000000000004e-06 2020-09-08T15:49:09.316 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.317 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.317 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:49:09.317 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849764+0000", 2020-09-08T15:49:09.317 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6109999999999998e-06 2020-09-08T15:49:09.317 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.318 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.318 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:49:09.318 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849768+0000", 2020-09-08T15:49:09.318 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8702000000000003e-05 2020-09-08T15:49:09.318 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.319 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.319 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.319 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849806+0000", 2020-09-08T15:49:09.319 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5669999999999995e-06 2020-09-08T15:49:09.319 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.320 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.320 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2020-09-08T15:49:09.320 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:04.849815+0000", 2020-09-08T15:49:09.320 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.76780925 2020-09-08T15:49:09.320 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.320 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.321 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.321 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.617624+0000", 2020-09-08T15:49:09.321 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000225837 2020-09-08T15:49:09.321 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.321 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.322 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:49:09.322 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.617850+0000", 2020-09-08T15:49:09.322 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00021239699999999999 2020-09-08T15:49:09.322 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.322 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.323 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8,11", 2020-09-08T15:49:09.323 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.618062+0000", 2020-09-08T15:49:09.323 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0072071269999999998 2020-09-08T15:49:09.323 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.323 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.324 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:49:09.324 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.625270+0000", 2020-09-08T15:49:09.324 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.056336343999999997 2020-09-08T15:49:09.324 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.324 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.324 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.325 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.681606+0000", 2020-09-08T15:49:09.325 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.017974381000000001 2020-09-08T15:49:09.325 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.325 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.325 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.326 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.699580+0000", 2020-09-08T15:49:09.326 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8785999999999999e-05 2020-09-08T15:49:09.326 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.326 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.326 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:49:09.327 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.699679+0000", 2020-09-08T15:49:09.327 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0024944770000000002 2020-09-08T15:49:09.327 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.327 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.327 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:49:09.327 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:43:05.702174+0000", 2020-09-08T15:49:09.328 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.85250447500000004 2020-09-08T15:49:09.328 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.328 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:09.328 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.328 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.329 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.329 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:229038 3.7 3:e3b8f503:::benchmark_data_smithi166_26097_object104877:head [delete] snapc 0=[] ondisk+write+known_if_redirected e302)", 2020-09-08T15:49:09.329 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:46:34.567563+0000", 2020-09-08T15:49:09.329 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 154.39918601400001, 2020-09-08T15:49:09.329 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.35880672899999999, 2020-09-08T15:49:09.330 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:49:09.330 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:09.330 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:49:09.330 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:49:09.330 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:49:09.331 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 229038 2020-09-08T15:49:09.331 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.331 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:49:09.331 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.331 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:49:09.332 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.567563+0000", 2020-09-08T15:49:09.332 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:49:09.332 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.332 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.332 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:49:09.333 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.567563+0000", 2020-09-08T15:49:09.333 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999595 2020-09-08T15:49:09.333 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.333 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.333 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:49:09.333 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.567522+0000", 2020-09-08T15:49:09.334 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9178000000000006e-05 2020-09-08T15:49:09.334 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.334 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.334 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:49:09.334 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.567611+0000", 2020-09-08T15:49:09.335 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4585e-05 2020-09-08T15:49:09.335 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.335 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.335 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:49:09.335 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.567626+0000", 2020-09-08T15:49:09.336 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1970000000000001e-06 2020-09-08T15:49:09.336 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.336 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.336 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:49:09.336 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.567632+0000", 2020-09-08T15:49:09.337 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4849999999999999e-05 2020-09-08T15:49:09.337 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.337 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.337 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.337 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.567677+0000", 2020-09-08T15:49:09.337 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9438999999999994e-05 2020-09-08T15:49:09.338 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.339 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.339 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:49:09.339 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.567776+0000", 2020-09-08T15:49:09.339 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000108504 2020-09-08T15:49:09.340 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.340 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.340 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T15:49:09.340 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.567885+0000", 2020-09-08T15:49:09.340 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0085057699999999993 2020-09-08T15:49:09.341 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.341 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.341 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:49:09.341 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.576391+0000", 2020-09-08T15:49:09.341 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.34984064999999998 2020-09-08T15:49:09.342 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.342 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.342 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.342 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.926231+0000", 2020-09-08T15:49:09.342 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3153999999999997e-05 2020-09-08T15:49:09.343 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.343 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.343 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:49:09.343 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.926315+0000", 2020-09-08T15:49:09.343 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4970999999999999e-05 2020-09-08T15:49:09.344 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.344 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.344 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:49:09.344 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.926370+0000", 2020-09-08T15:49:09.344 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.35880672899999999 2020-09-08T15:49:09.344 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.345 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:09.345 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.345 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.345 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.345 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:229046 3.2 3:40f39515:::benchmark_data_smithi166_26097_object104885:head [delete] snapc 0=[] ondisk+write+known_if_redirected e302)", 2020-09-08T15:49:09.346 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:46:34.576921+0000", 2020-09-08T15:49:09.346 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 154.38982777800001, 2020-09-08T15:49:09.346 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.360092568, 2020-09-08T15:49:09.346 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:49:09.346 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:09.347 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:49:09.347 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:49:09.347 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:49:09.347 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 229046 2020-09-08T15:49:09.347 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.348 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:49:09.348 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.348 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:49:09.348 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.576921+0000", 2020-09-08T15:49:09.348 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:49:09.348 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.349 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.349 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:49:09.349 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.576921+0000", 2020-09-08T15:49:09.349 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999743 2020-09-08T15:49:09.349 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.350 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.350 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:49:09.350 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.576895+0000", 2020-09-08T15:49:09.350 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8717e-05 2020-09-08T15:49:09.350 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.351 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.351 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:49:09.351 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.576954+0000", 2020-09-08T15:49:09.351 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9539999999999999e-06 2020-09-08T15:49:09.351 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.352 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.352 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:49:09.352 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.576964+0000", 2020-09-08T15:49:09.352 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3789999999999999e-06 2020-09-08T15:49:09.352 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.352 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.353 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:49:09.353 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.576968+0000", 2020-09-08T15:49:09.353 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0744000000000002e-05 2020-09-08T15:49:09.353 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.353 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.354 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:49:09.354 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.577009+0000", 2020-09-08T15:49:09.354 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6898999999999995e-05 2020-09-08T15:49:09.354 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.354 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.355 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:49:09.355 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.577106+0000", 2020-09-08T15:49:09.355 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1438000000000001e-05 2020-09-08T15:49:09.355 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.355 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.356 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 9", 2020-09-08T15:49:09.356 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.577197+0000", 2020-09-08T15:49:09.356 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0042720830000000003 2020-09-08T15:49:09.356 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.356 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.356 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:49:09.357 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.581469+0000", 2020-09-08T15:49:09.357 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.355284883 2020-09-08T15:49:09.357 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.357 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.357 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:09.358 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.936754+0000", 2020-09-08T15:49:09.358 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014192499999999999 2020-09-08T15:49:09.358 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.358 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.358 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:49:09.359 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.936896+0000", 2020-09-08T15:49:09.359 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000117531 2020-09-08T15:49:09.359 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:49:09.359 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:49:09.359 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:49:09.359 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:46:34.937014+0000", 2020-09-08T15:49:09.360 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.360092568 2020-09-08T15:49:09.360 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.360 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:09.360 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.360 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:49:09.361 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:49:09.361 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:49:09.361 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:08 smithi104 bash[9203]: cluster 2020-09-08T15:49:07.775436+0000 mgr.x (mgr.24118) 654 : cluster [DBG] pgmap v922: 17 pgs: 17 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 887 op/s 2020-09-08T15:49:10.253 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2020-09-08T15:49:10.387 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:49:10.413 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:49:10.568 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:49:11.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:10 smithi166 bash[9446]: cluster 2020-09-08T15:49:09.061216+0000 osd.7 (osd.7) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:11.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:10 smithi166 bash[9446]: cluster 2020-09-08T15:49:09.061281+0000 osd.7 (osd.7) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:11.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:10 smithi166 bash[9446]: cluster 2020-09-08T15:49:09.746402+0000 mon.a (mon.0) 1398 : cluster [DBG] osdmap e353: 12 total, 12 up, 12 in 2020-09-08T15:49:11.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:10 smithi166 bash[9446]: cluster 2020-09-08T15:49:09.775789+0000 mgr.x (mgr.24118) 655 : cluster [DBG] pgmap v924: 17 pgs: 17 active+clean; 7.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T15:49:11.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:10 smithi055 bash[9296]: cluster 2020-09-08T15:49:09.061216+0000 osd.7 (osd.7) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:11.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:10 smithi055 bash[9296]: cluster 2020-09-08T15:49:09.061281+0000 osd.7 (osd.7) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:11.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:10 smithi055 bash[9296]: cluster 2020-09-08T15:49:09.746402+0000 mon.a (mon.0) 1398 : cluster [DBG] osdmap e353: 12 total, 12 up, 12 in 2020-09-08T15:49:11.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:10 smithi055 bash[9296]: cluster 2020-09-08T15:49:09.775789+0000 mgr.x (mgr.24118) 655 : cluster [DBG] pgmap v924: 17 pgs: 17 active+clean; 7.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T15:49:11.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:10 smithi104 bash[9203]: cluster 2020-09-08T15:49:09.061216+0000 osd.7 (osd.7) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:11.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:10 smithi104 bash[9203]: cluster 2020-09-08T15:49:09.061281+0000 osd.7 (osd.7) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:11.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:10 smithi104 bash[9203]: cluster 2020-09-08T15:49:09.746402+0000 mon.a (mon.0) 1398 : cluster [DBG] osdmap e353: 12 total, 12 up, 12 in 2020-09-08T15:49:11.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:10 smithi104 bash[9203]: cluster 2020-09-08T15:49:09.775789+0000 mgr.x (mgr.24118) 655 : cluster [DBG] pgmap v924: 17 pgs: 17 active+clean; 7.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T15:49:11.759 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:49:12.053 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:11 smithi104 bash[9203]: cluster 2020-09-08T15:49:10.234309+0000 osd.0 (osd.0) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:12.054 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:11 smithi104 bash[9203]: cluster 2020-09-08T15:49:10.234395+0000 osd.0 (osd.0) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:12.054 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:11 smithi104 bash[9203]: cluster 2020-09-08T15:49:10.752949+0000 mon.a (mon.0) 1399 : cluster [DBG] osdmap e354: 12 total, 12 up, 12 in 2020-09-08T15:49:12.054 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:11 smithi104 bash[9203]: audit 2020-09-08T15:49:10.847682+0000 mon.c (mon.2) 54 : audit [INF] from='client.? v1:172.21.15.104:0/299143658' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:49:12.055 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:11 smithi104 bash[9203]: audit 2020-09-08T15:49:10.850104+0000 mon.a (mon.0) 1400 : audit [INF] from='client.35468 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:49:12.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:11 smithi166 bash[9446]: cluster 2020-09-08T15:49:10.234309+0000 osd.0 (osd.0) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:12.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:11 smithi166 bash[9446]: cluster 2020-09-08T15:49:10.234395+0000 osd.0 (osd.0) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:12.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:11 smithi166 bash[9446]: cluster 2020-09-08T15:49:10.752949+0000 mon.a (mon.0) 1399 : cluster [DBG] osdmap e354: 12 total, 12 up, 12 in 2020-09-08T15:49:12.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:11 smithi166 bash[9446]: audit 2020-09-08T15:49:10.847682+0000 mon.c (mon.2) 54 : audit [INF] from='client.? v1:172.21.15.104:0/299143658' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:49:12.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:11 smithi166 bash[9446]: audit 2020-09-08T15:49:10.850104+0000 mon.a (mon.0) 1400 : audit [INF] from='client.35468 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:49:12.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:11 smithi055 bash[9296]: cluster 2020-09-08T15:49:10.234309+0000 osd.0 (osd.0) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:12.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:11 smithi055 bash[9296]: cluster 2020-09-08T15:49:10.234395+0000 osd.0 (osd.0) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:12.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:11 smithi055 bash[9296]: cluster 2020-09-08T15:49:10.752949+0000 mon.a (mon.0) 1399 : cluster [DBG] osdmap e354: 12 total, 12 up, 12 in 2020-09-08T15:49:12.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:11 smithi055 bash[9296]: audit 2020-09-08T15:49:10.847682+0000 mon.c (mon.2) 54 : audit [INF] from='client.? v1:172.21.15.104:0/299143658' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:49:12.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:11 smithi055 bash[9296]: audit 2020-09-08T15:49:10.850104+0000 mon.a (mon.0) 1400 : audit [INF] from='client.35468 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:49:12.323 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:49:12.427 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:12.574 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:12.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:49:12 smithi055 bash[19003]: debug 2020-09-08T15:49:12.321+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:12.676 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:12.747 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:12 smithi166 bash[9446]: cluster 2020-09-08T15:49:11.750005+0000 mon.a (mon.0) 1401 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:49:12.748 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:12 smithi166 bash[9446]: audit 2020-09-08T15:49:11.762116+0000 mon.a (mon.0) 1402 : audit [INF] from='client.35468 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:49:12.748 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:12 smithi166 bash[9446]: cluster 2020-09-08T15:49:11.762161+0000 mon.a (mon.0) 1403 : cluster [DBG] osdmap e355: 12 total, 12 up, 12 in 2020-09-08T15:49:12.748 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:12 smithi166 bash[9446]: cluster 2020-09-08T15:49:11.776175+0000 mgr.x (mgr.24118) 656 : cluster [DBG] pgmap v927: 17 pgs: 17 active+clean; 7.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:49:12.748 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:49:12 smithi166 bash[16593]: debug 2020-09-08T15:49:12.569+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:12.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:12 smithi104 bash[9203]: cluster 2020-09-08T15:49:11.750005+0000 mon.a (mon.0) 1401 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:49:12.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:12 smithi104 bash[9203]: audit 2020-09-08T15:49:11.762116+0000 mon.a (mon.0) 1402 : audit [INF] from='client.35468 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:49:12.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:12 smithi104 bash[9203]: cluster 2020-09-08T15:49:11.762161+0000 mon.a (mon.0) 1403 : cluster [DBG] osdmap e355: 12 total, 12 up, 12 in 2020-09-08T15:49:12.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:12 smithi104 bash[9203]: cluster 2020-09-08T15:49:11.776175+0000 mgr.x (mgr.24118) 656 : cluster [DBG] pgmap v927: 17 pgs: 17 active+clean; 7.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:49:12.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:12 smithi104 bash[9203]: cluster 2020-09-08T15:49:12.240829+0000 osd.11 (osd.11) 48 : cluster [DBG] 4.8 scrub starts 2020-09-08T15:49:12.967 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:12 smithi055 bash[9296]: cluster 2020-09-08T15:49:11.750005+0000 mon.a (mon.0) 1401 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:49:12.968 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:12 smithi055 bash[9296]: audit 2020-09-08T15:49:11.762116+0000 mon.a (mon.0) 1402 : audit [INF] from='client.35468 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:49:12.968 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:12 smithi055 bash[9296]: cluster 2020-09-08T15:49:11.762161+0000 mon.a (mon.0) 1403 : cluster [DBG] osdmap e355: 12 total, 12 up, 12 in 2020-09-08T15:49:12.968 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:12 smithi055 bash[9296]: cluster 2020-09-08T15:49:11.776175+0000 mgr.x (mgr.24118) 656 : cluster [DBG] pgmap v927: 17 pgs: 17 active+clean; 7.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:49:12.968 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:12 smithi055 bash[9296]: cluster 2020-09-08T15:49:12.240829+0000 osd.11 (osd.11) 48 : cluster [DBG] 4.8 scrub starts 2020-09-08T15:49:12.969 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:13.071 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:49:13.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:49:12 smithi055 bash[16646]: debug 2020-09-08T15:49:12.969+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:13.684 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:49:13.787 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:13.916 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:13.991 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:13 smithi055 bash[9296]: cluster 2020-09-08T15:49:13.096252+0000 osd.0 (osd.0) 43 : cluster [DBG] 4.b scrub starts 2020-09-08T15:49:13.992 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:49:13 smithi055 bash[14301]: debug 2020-09-08T15:49:13.684+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:14.022 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:14.042 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:49:13 smithi166 bash[11928]: debug 2020-09-08T15:49:13.913+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:14.042 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:12 smithi166 bash[9446]: cluster 2020-09-08T15:49:12.240829+0000 osd.11 (osd.11) 48 : cluster [DBG] 4.8 scrub starts 2020-09-08T15:49:14.043 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:13 smithi166 bash[9446]: cluster 2020-09-08T15:49:13.096252+0000 osd.0 (osd.0) 43 : cluster [DBG] 4.b scrub starts 2020-09-08T15:49:14.169 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:14.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:13 smithi104 bash[9203]: cluster 2020-09-08T15:49:13.096252+0000 osd.0 (osd.0) 43 : cluster [DBG] 4.b scrub starts 2020-09-08T15:49:14.274 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:49:14.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:49:14 smithi166 bash[11928]: debug 2020-09-08T15:49:14.161+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:14.535 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:49:14 smithi104 bash[19922]: debug 2020-09-08T15:49:14.539+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:14.538 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:49:14.643 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:14.806 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:49:14 smithi104 bash[17588]: debug 2020-09-08T15:49:14.807+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:14.812 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:14.817 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:49:14.915 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:49:14.986 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:49:15.032 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:49:15.138 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:15.248 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:49:15.260 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:49:15 smithi166 bash[18960]: debug 2020-09-08T15:49:15.029+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:15.261 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:15.287 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:49:15.324 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:49:15.362 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:49:15.367 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:49:15.402 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:49:15.458 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:49:15.480 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:49:15.513 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:49:15.555 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:49:15.583 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:49:15.615 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:49:15.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:49:15 smithi166 bash[16593]: debug 2020-09-08T15:49:15.257+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:15.649 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:49:15.664 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:49:15.664 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:49:15.664 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:49:15.665 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:49:15.665 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:49:15.665 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:49:15.665 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:49:15.665 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:49:15.666 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:49:15.666 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:49:15.666 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:49:15.677 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:49:15.974 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:49:15.978 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:49:15 smithi104 bash[19922]: debug 2020-09-08T15:49:15.975+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:15.978 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:15 smithi104 bash[9203]: cluster 2020-09-08T15:49:13.776647+0000 mgr.x (mgr.24118) 657 : cluster [DBG] pgmap v928: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T15:49:15.978 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:15 smithi104 bash[9203]: cluster 2020-09-08T15:49:14.496137+0000 osd.11 (osd.11) 49 : cluster [DBG] 4.8 scrub ok 2020-09-08T15:49:15.978 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:15 smithi104 bash[9203]: audit 2020-09-08T15:49:15.217482+0000 mon.a (mon.0) 1404 : audit [DBG] from='client.? v1:172.21.15.104:0/104479375' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:49:16.080 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:16.087 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:15 smithi166 bash[9446]: cluster 2020-09-08T15:49:13.776647+0000 mgr.x (mgr.24118) 657 : cluster [DBG] pgmap v928: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T15:49:16.087 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:15 smithi166 bash[9446]: cluster 2020-09-08T15:49:14.496137+0000 osd.11 (osd.11) 49 : cluster [DBG] 4.8 scrub ok 2020-09-08T15:49:16.088 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:15 smithi166 bash[9446]: audit 2020-09-08T15:49:15.217482+0000 mon.a (mon.0) 1404 : audit [DBG] from='client.? v1:172.21.15.104:0/104479375' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:49:16.164 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:15 smithi055 bash[9296]: cluster 2020-09-08T15:49:13.776647+0000 mgr.x (mgr.24118) 657 : cluster [DBG] pgmap v928: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T15:49:16.165 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:15 smithi055 bash[9296]: cluster 2020-09-08T15:49:14.496137+0000 osd.11 (osd.11) 49 : cluster [DBG] 4.8 scrub ok 2020-09-08T15:49:16.165 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:15 smithi055 bash[9296]: audit 2020-09-08T15:49:15.217482+0000 mon.a (mon.0) 1404 : audit [DBG] from='client.? v1:172.21.15.104:0/104479375' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:49:16.209 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:16.311 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:16.366 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:49:16 smithi166 bash[16593]: debug 2020-09-08T15:49:16.201+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:16.435 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:16.545 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:16.607 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:49:16.607 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:49:16.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:49:16 smithi055 bash[11956]: debug 2020-09-08T15:49:16.436+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:16.689 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:16.791 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:16.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:49:16 smithi166 bash[11928]: debug 2020-09-08T15:49:16.681+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:16.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:16 smithi166 bash[9446]: cluster 2020-09-08T15:49:15.618169+0000 osd.0 (osd.0) 44 : cluster [DBG] 4.b scrub ok 2020-09-08T15:49:16.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:16 smithi166 bash[9446]: cluster 2020-09-08T15:49:15.657368+0000 osd.7 (osd.7) 67 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:49:16.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:16 smithi166 bash[9446]: cluster 2020-09-08T15:49:15.715492+0000 osd.7 (osd.7) 68 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:49:16.919 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:17.028 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:17.040 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:16 smithi104 bash[9203]: cluster 2020-09-08T15:49:15.618169+0000 osd.0 (osd.0) 44 : cluster [DBG] 4.b scrub ok 2020-09-08T15:49:17.041 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:16 smithi104 bash[9203]: cluster 2020-09-08T15:49:15.657368+0000 osd.7 (osd.7) 67 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:49:17.041 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:16 smithi104 bash[9203]: cluster 2020-09-08T15:49:15.715492+0000 osd.7 (osd.7) 68 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:49:17.148 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:17.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:49:16 smithi055 bash[16646]: debug 2020-09-08T15:49:16.916+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:17.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:16 smithi055 bash[9296]: cluster 2020-09-08T15:49:15.618169+0000 osd.0 (osd.0) 44 : cluster [DBG] 4.b scrub ok 2020-09-08T15:49:17.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:16 smithi055 bash[9296]: cluster 2020-09-08T15:49:15.657368+0000 osd.7 (osd.7) 67 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:49:17.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:16 smithi055 bash[9296]: cluster 2020-09-08T15:49:15.715492+0000 osd.7 (osd.7) 68 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:49:17.253 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:49:17.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:49:17 smithi166 bash[14254]: debug 2020-09-08T15:49:17.145+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:17.481 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:49:17.585 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:17.606 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:49:17 smithi055 bash[14301]: debug 2020-09-08T15:49:17.480+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:17.607 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2020-09-08T15:49:17.707 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:17.766 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:49:17.811 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:49:17.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:17 smithi166 bash[9446]: cluster 2020-09-08T15:49:15.777076+0000 mgr.x (mgr.24118) 658 : cluster [DBG] pgmap v929: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 823 op/s 2020-09-08T15:49:17.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:17 smithi166 bash[9446]: cluster 2020-09-08T15:49:17.145395+0000 osd.5 (osd.5) 45 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:17.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:17 smithi166 bash[9446]: cluster 2020-09-08T15:49:17.145474+0000 osd.5 (osd.5) 46 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:17.867 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:49:17 smithi166 bash[16593]: debug 2020-09-08T15:49:17.701+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:17.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:17 smithi055 bash[9296]: cluster 2020-09-08T15:49:15.777076+0000 mgr.x (mgr.24118) 658 : cluster [DBG] pgmap v929: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 823 op/s 2020-09-08T15:49:17.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:17 smithi055 bash[9296]: cluster 2020-09-08T15:49:17.145395+0000 osd.5 (osd.5) 45 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:17.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:17 smithi055 bash[9296]: cluster 2020-09-08T15:49:17.145474+0000 osd.5 (osd.5) 46 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:17.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:17 smithi104 bash[9203]: cluster 2020-09-08T15:49:15.777076+0000 mgr.x (mgr.24118) 658 : cluster [DBG] pgmap v929: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 823 op/s 2020-09-08T15:49:17.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:17 smithi104 bash[9203]: cluster 2020-09-08T15:49:17.145395+0000 osd.5 (osd.5) 45 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:17.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:17 smithi104 bash[9203]: cluster 2020-09-08T15:49:17.145474+0000 osd.5 (osd.5) 46 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:18.315 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:49:18.419 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:18.574 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:49:18 smithi055 bash[19003]: debug 2020-09-08T15:49:18.316+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:18.575 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:18.679 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:18.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:49:18 smithi055 bash[11956]: debug 2020-09-08T15:49:18.576+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:18.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:18 smithi055 bash[9296]: cluster 2020-09-08T15:49:17.746763+0000 mon.a (mon.0) 1405 : cluster [DBG] osdmap e356: 12 total, 12 up, 12 in 2020-09-08T15:49:18.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:18 smithi055 bash[9296]: cluster 2020-09-08T15:49:17.777450+0000 mgr.x (mgr.24118) 659 : cluster [DBG] pgmap v931: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.11k op/s 2020-09-08T15:49:19.042 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:49:19.050 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:18 smithi104 bash[9203]: cluster 2020-09-08T15:49:17.746763+0000 mon.a (mon.0) 1405 : cluster [DBG] osdmap e356: 12 total, 12 up, 12 in 2020-09-08T15:49:19.051 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:18 smithi104 bash[9203]: cluster 2020-09-08T15:49:17.777450+0000 mgr.x (mgr.24118) 659 : cluster [DBG] pgmap v931: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.11k op/s 2020-09-08T15:49:19.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:18 smithi166 bash[9446]: cluster 2020-09-08T15:49:17.746763+0000 mon.a (mon.0) 1405 : cluster [DBG] osdmap e356: 12 total, 12 up, 12 in 2020-09-08T15:49:19.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:18 smithi166 bash[9446]: cluster 2020-09-08T15:49:17.777450+0000 mgr.x (mgr.24118) 659 : cluster [DBG] pgmap v931: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.11k op/s 2020-09-08T15:49:19.140 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:19.202 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:49:19.243 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:19.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:49:19 smithi055 bash[11956]: debug 2020-09-08T15:49:19.136+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:19.411 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:19.513 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:49:19.775 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:49:19 smithi166 bash[16593]: debug 2020-09-08T15:49:19.409+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:19.979 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:49:19 smithi055 bash[14301]: debug 2020-09-08T15:49:19.980+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:19.979 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:49:20.085 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:20.188 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:20.293 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:49:20.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:49:20 smithi166 bash[14254]: debug 2020-09-08T15:49:20.185+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:20.578 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:49:20.683 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:20.827 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:49:20 smithi104 bash[19922]: debug 2020-09-08T15:49:20.579+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:20.827 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:20 smithi104 bash[9203]: cluster 2020-09-08T15:49:19.278033+0000 osd.11 (osd.11) 50 : cluster [DBG] 4.9 scrub starts 2020-09-08T15:49:20.828 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:20 smithi104 bash[9203]: cluster 2020-09-08T15:49:20.404197+0000 osd.10 (osd.10) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:20.828 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:20 smithi104 bash[9203]: cluster 2020-09-08T15:49:20.404365+0000 osd.10 (osd.10) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:20.829 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:20.933 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:20.943 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:20 smithi166 bash[9446]: cluster 2020-09-08T15:49:19.278033+0000 osd.11 (osd.11) 50 : cluster [DBG] 4.9 scrub starts 2020-09-08T15:49:20.944 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:20 smithi166 bash[9446]: cluster 2020-09-08T15:49:20.404197+0000 osd.10 (osd.10) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:20.944 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:20 smithi166 bash[9446]: cluster 2020-09-08T15:49:20.404365+0000 osd.10 (osd.10) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:20.944 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:49:20 smithi166 bash[14254]: debug 2020-09-08T15:49:20.829+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:21.047 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:21.155 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:49:21.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:20 smithi055 bash[9296]: cluster 2020-09-08T15:49:19.278033+0000 osd.11 (osd.11) 50 : cluster [DBG] 4.9 scrub starts 2020-09-08T15:49:21.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:20 smithi055 bash[9296]: cluster 2020-09-08T15:49:20.404197+0000 osd.10 (osd.10) 33 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:21.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:20 smithi055 bash[9296]: cluster 2020-09-08T15:49:20.404365+0000 osd.10 (osd.10) 34 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:21.341 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:49:21.366 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:49:21 smithi166 bash[16593]: debug 2020-09-08T15:49:21.045+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:21.445 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:21.600 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:21.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:49:21 smithi055 bash[14301]: debug 2020-09-08T15:49:21.344+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:21.706 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:49:21.715 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:49:21 smithi166 bash[14254]: debug 2020-09-08T15:49:21.601+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:21.757 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:49:22.092 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:21 smithi166 bash[9446]: cluster 2020-09-08T15:49:19.777942+0000 mgr.x (mgr.24118) 660 : cluster [DBG] pgmap v932: 17 pgs: 17 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T15:49:22.093 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:21 smithi166 bash[9446]: cluster 2020-09-08T15:49:20.747063+0000 mon.a (mon.0) 1406 : cluster [DBG] osdmap e357: 12 total, 12 up, 12 in 2020-09-08T15:49:22.093 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:21 smithi166 bash[9446]: audit 2020-09-08T15:49:21.289433+0000 mon.b (mon.1) 88 : audit [INF] from='client.? v1:172.21.15.104:0/2675210739' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:49:22.093 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:21 smithi166 bash[9446]: audit 2020-09-08T15:49:21.289988+0000 mon.a (mon.0) 1407 : audit [INF] from='client.25471 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:49:22.109 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:49:22 smithi166 bash[18960]: debug 2020-09-08T15:49:22.085+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:22.109 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:49:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:21 smithi055 bash[9296]: cluster 2020-09-08T15:49:19.777942+0000 mgr.x (mgr.24118) 660 : cluster [DBG] pgmap v932: 17 pgs: 17 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T15:49:22.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:21 smithi055 bash[9296]: cluster 2020-09-08T15:49:20.747063+0000 mon.a (mon.0) 1406 : cluster [DBG] osdmap e357: 12 total, 12 up, 12 in 2020-09-08T15:49:22.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:21 smithi055 bash[9296]: audit 2020-09-08T15:49:21.289433+0000 mon.b (mon.1) 88 : audit [INF] from='client.? v1:172.21.15.104:0/2675210739' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:49:22.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:21 smithi055 bash[9296]: audit 2020-09-08T15:49:21.289988+0000 mon.a (mon.0) 1407 : audit [INF] from='client.25471 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:49:22.183 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:21 smithi104 bash[9203]: cluster 2020-09-08T15:49:19.777942+0000 mgr.x (mgr.24118) 660 : cluster [DBG] pgmap v932: 17 pgs: 17 active+clean; 7.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T15:49:22.183 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:21 smithi104 bash[9203]: cluster 2020-09-08T15:49:20.747063+0000 mon.a (mon.0) 1406 : cluster [DBG] osdmap e357: 12 total, 12 up, 12 in 2020-09-08T15:49:22.184 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:21 smithi104 bash[9203]: audit 2020-09-08T15:49:21.289433+0000 mon.b (mon.1) 88 : audit [INF] from='client.? v1:172.21.15.104:0/2675210739' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:49:22.184 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:21 smithi104 bash[9203]: audit 2020-09-08T15:49:21.289988+0000 mon.a (mon.0) 1407 : audit [INF] from='client.25471 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:49:22.211 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:49:22.323 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:49:22.427 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:49:22.430 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:49:22.430 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:49:22.516 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:49:22 smithi055 bash[14301]: debug 2020-09-08T15:49:22.324+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:22.552 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:49:22.657 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:22.741 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:49:22 smithi166 bash[18960]: debug 2020-09-08T15:49:22.549+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:22.741 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:22 smithi166 bash[9446]: cluster 2020-09-08T15:49:21.743924+0000 mon.a (mon.0) 1408 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:49:22.741 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:22 smithi166 bash[9446]: cluster 2020-09-08T15:49:21.743953+0000 mon.a (mon.0) 1409 : cluster [INF] Cluster is now healthy 2020-09-08T15:49:22.742 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:22 smithi166 bash[9446]: audit 2020-09-08T15:49:21.754380+0000 mon.a (mon.0) 1410 : audit [INF] from='client.25471 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:49:22.742 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:22 smithi166 bash[9446]: cluster 2020-09-08T15:49:21.754439+0000 mon.a (mon.0) 1411 : cluster [DBG] osdmap e358: 12 total, 12 up, 12 in 2020-09-08T15:49:22.742 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:22 smithi166 bash[9446]: cluster 2020-09-08T15:49:21.778339+0000 mgr.x (mgr.24118) 661 : cluster [DBG] pgmap v935: 17 pgs: 17 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:49:22.777 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:22.880 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:22.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:22 smithi055 bash[9296]: cluster 2020-09-08T15:49:21.743924+0000 mon.a (mon.0) 1408 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:49:22.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:22 smithi055 bash[9296]: cluster 2020-09-08T15:49:21.743953+0000 mon.a (mon.0) 1409 : cluster [INF] Cluster is now healthy 2020-09-08T15:49:22.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:22 smithi055 bash[9296]: audit 2020-09-08T15:49:21.754380+0000 mon.a (mon.0) 1410 : audit [INF] from='client.25471 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:49:22.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:22 smithi055 bash[9296]: cluster 2020-09-08T15:49:21.754439+0000 mon.a (mon.0) 1411 : cluster [DBG] osdmap e358: 12 total, 12 up, 12 in 2020-09-08T15:49:22.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:22 smithi055 bash[9296]: cluster 2020-09-08T15:49:21.778339+0000 mgr.x (mgr.24118) 661 : cluster [DBG] pgmap v935: 17 pgs: 17 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:49:23.054 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:23.074 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:22 smithi104 bash[9203]: cluster 2020-09-08T15:49:21.743924+0000 mon.a (mon.0) 1408 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:49:23.075 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:22 smithi104 bash[9203]: cluster 2020-09-08T15:49:21.743953+0000 mon.a (mon.0) 1409 : cluster [INF] Cluster is now healthy 2020-09-08T15:49:23.075 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:22 smithi104 bash[9203]: audit 2020-09-08T15:49:21.754380+0000 mon.a (mon.0) 1410 : audit [INF] from='client.25471 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:49:23.075 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:22 smithi104 bash[9203]: cluster 2020-09-08T15:49:21.754439+0000 mon.a (mon.0) 1411 : cluster [DBG] osdmap e358: 12 total, 12 up, 12 in 2020-09-08T15:49:23.075 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:22 smithi104 bash[9203]: cluster 2020-09-08T15:49:21.778339+0000 mgr.x (mgr.24118) 661 : cluster [DBG] pgmap v935: 17 pgs: 17 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:49:23.076 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:49:23 smithi104 bash[17588]: debug 2020-09-08T15:49:23.055+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:23.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:49:22 smithi166 bash[14254]: debug 2020-09-08T15:49:22.777+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:23.163 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:23.371 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2020-09-08T15:49:23.524 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:49:23.568 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:23.675 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:23.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:49:23 smithi104 bash[15273]: debug 2020-09-08T15:49:23.567+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:24.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:23 smithi166 bash[9446]: cluster 2020-09-08T15:49:22.804298+0000 osd.11 (osd.11) 51 : cluster [DBG] 4.9 scrub ok 2020-09-08T15:49:24.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:23 smithi055 bash[9296]: cluster 2020-09-08T15:49:22.804298+0000 osd.11 (osd.11) 51 : cluster [DBG] 4.9 scrub ok 2020-09-08T15:49:24.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:23 smithi104 bash[9203]: cluster 2020-09-08T15:49:22.804298+0000 osd.11 (osd.11) 51 : cluster [DBG] 4.9 scrub ok 2020-09-08T15:49:24.185 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:24.288 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:24.437 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:24.543 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:24.555 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:49:24 smithi055 bash[11956]: debug 2020-09-08T15:49:24.188+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:24.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:49:24 smithi166 bash[11928]: debug 2020-09-08T15:49:24.437+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:24.821 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:49:24.845 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:49:24 smithi055 bash[11956]: debug 2020-09-08T15:49:24.840+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:24.846 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:24.851 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:49:24.885 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:49:24.918 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:49:24.949 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:49:24.964 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:49:25.005 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:49:25.039 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:49:25.075 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:49:25.076 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:49:25.113 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:49:25.145 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:49:25.179 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:25.180 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:49:25.218 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:49:25.218 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:49:25.224 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:49:25.224 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:49:25.225 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:49:25.225 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:49:25.225 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:49:25.226 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:49:25.226 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:49:25.226 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:49:25.226 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:49:25.227 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:49:25.227 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:49:25.308 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:25.330 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:49:25 smithi104 bash[19922]: debug 2020-09-08T15:49:25.075+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:25.413 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:49:25.438 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:49:25.562 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:49:25.605 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:49:25.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:49:25 smithi166 bash[16593]: debug 2020-09-08T15:49:25.305+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:25.665 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:25.772 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:49:25 smithi055 bash[19003]: debug 2020-09-08T15:49:25.564+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:25.799 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:25 smithi104 bash[9203]: cluster 2020-09-08T15:49:23.778798+0000 mgr.x (mgr.24118) 662 : cluster [DBG] pgmap v936: 17 pgs: 17 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.22k op/s 2020-09-08T15:49:25.799 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:25 smithi104 bash[9203]: audit 2020-09-08T15:49:24.792653+0000 mon.a (mon.0) 1412 : audit [DBG] from='client.? v1:172.21.15.104:0/1008780341' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:49:26.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:25 smithi166 bash[9446]: cluster 2020-09-08T15:49:23.778798+0000 mgr.x (mgr.24118) 662 : cluster [DBG] pgmap v936: 17 pgs: 17 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.22k op/s 2020-09-08T15:49:26.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:25 smithi166 bash[9446]: audit 2020-09-08T15:49:24.792653+0000 mon.a (mon.0) 1412 : audit [DBG] from='client.? v1:172.21.15.104:0/1008780341' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:49:26.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:25 smithi055 bash[9296]: cluster 2020-09-08T15:49:23.778798+0000 mgr.x (mgr.24118) 662 : cluster [DBG] pgmap v936: 17 pgs: 17 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.22k op/s 2020-09-08T15:49:26.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:25 smithi055 bash[9296]: audit 2020-09-08T15:49:24.792653+0000 mon.a (mon.0) 1412 : audit [DBG] from='client.? v1:172.21.15.104:0/1008780341' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:49:26.191 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:26.296 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:26.448 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:26.550 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:26.560 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:49:26 smithi055 bash[11956]: debug 2020-09-08T15:49:26.448+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:26.680 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:49:26 smithi104 bash[17588]: debug 2020-09-08T15:49:26.191+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:26.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:26 smithi055 bash[9296]: cluster 2020-09-08T15:49:25.578550+0000 osd.3 (osd.3) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:26.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:26 smithi055 bash[9296]: cluster 2020-09-08T15:49:25.578632+0000 osd.3 (osd.3) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:26.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:26 smithi055 bash[9296]: cluster 2020-09-08T15:49:25.758926+0000 mon.a (mon.0) 1413 : cluster [DBG] osdmap e359: 12 total, 12 up, 12 in 2020-09-08T15:49:26.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:26 smithi055 bash[9296]: cluster 2020-09-08T15:49:25.779672+0000 mgr.x (mgr.24118) 663 : cluster [DBG] pgmap v938: 17 pgs: 17 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 846 op/s 2020-09-08T15:49:27.103 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:27.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:26 smithi166 bash[9446]: cluster 2020-09-08T15:49:25.578550+0000 osd.3 (osd.3) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:27.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:26 smithi166 bash[9446]: cluster 2020-09-08T15:49:25.578632+0000 osd.3 (osd.3) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:27.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:26 smithi166 bash[9446]: cluster 2020-09-08T15:49:25.758926+0000 mon.a (mon.0) 1413 : cluster [DBG] osdmap e359: 12 total, 12 up, 12 in 2020-09-08T15:49:27.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:26 smithi166 bash[9446]: cluster 2020-09-08T15:49:25.779672+0000 mgr.x (mgr.24118) 663 : cluster [DBG] pgmap v938: 17 pgs: 17 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 846 op/s 2020-09-08T15:49:27.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:26 smithi104 bash[9203]: cluster 2020-09-08T15:49:25.578550+0000 osd.3 (osd.3) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:27.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:26 smithi104 bash[9203]: cluster 2020-09-08T15:49:25.578632+0000 osd.3 (osd.3) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:27.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:26 smithi104 bash[9203]: cluster 2020-09-08T15:49:25.758926+0000 mon.a (mon.0) 1413 : cluster [DBG] osdmap e359: 12 total, 12 up, 12 in 2020-09-08T15:49:27.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:26 smithi104 bash[9203]: cluster 2020-09-08T15:49:25.779672+0000 mgr.x (mgr.24118) 663 : cluster [DBG] pgmap v938: 17 pgs: 17 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 846 op/s 2020-09-08T15:49:27.206 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:49:27.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:49:27 smithi055 bash[11956]: debug 2020-09-08T15:49:27.104+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:27.815 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:49:27.919 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:49:28.042 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:49:28.138 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:49:27 smithi055 bash[14301]: debug 2020-09-08T15:49:27.812+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:28.139 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:49:28 smithi055 bash[14301]: debug 2020-09-08T15:49:28.044+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:28.146 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:49:28.398 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:49:28.399 INFO:teuthology.orchestra.run.smithi055.stdout: "size": 20, 2020-09-08T15:49:28.399 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 600, 2020-09-08T15:49:28.399 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T15:49:28.400 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.400 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:14757 3.1 e221/214 3:8dd7aeda:::benchmark_data_smithi166_26097_object14756:head v 221'909, mlcod=221'906)", 2020-09-08T15:49:28.400 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:24.675897+0000", 2020-09-08T15:49:28.400 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 423.72602951300001, 2020-09-08T15:49:28.401 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.042680911000000002, 2020-09-08T15:49:28.401 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:28.402 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:28.402 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:28.402 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.403 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:28.403 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.675897+0000", 2020-09-08T15:49:28.403 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:28.403 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.404 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.404 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:28.404 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.675897+0000", 2020-09-08T15:49:28.404 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010288799999999999 2020-09-08T15:49:28.404 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.405 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.405 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:28.405 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.676000+0000", 2020-09-08T15:49:28.405 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.9490000000000008e-06 2020-09-08T15:49:28.405 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.405 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.406 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:28.406 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.676010+0000", 2020-09-08T15:49:28.406 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6002e-05 2020-09-08T15:49:28.406 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.406 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.407 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:28.407 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.676026+0000", 2020-09-08T15:49:28.407 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.703e-06 2020-09-08T15:49:28.407 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.407 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.408 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:28.408 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.676033+0000", 2020-09-08T15:49:28.408 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000121105 2020-09-08T15:49:28.408 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.408 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.409 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:28.409 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.676154+0000", 2020-09-08T15:49:28.409 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.1246999999999998e-05 2020-09-08T15:49:28.409 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.409 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.410 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:28.410 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.676175+0000", 2020-09-08T15:49:28.410 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.042256384000000001 2020-09-08T15:49:28.410 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.410 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.410 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:49:28.411 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.718432+0000", 2020-09-08T15:49:28.411 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014663299999999999 2020-09-08T15:49:28.411 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.411 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.411 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:49:28.412 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.718578+0000", 2020-09-08T15:49:28.413 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.042680911000000002 2020-09-08T15:49:28.413 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.413 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.413 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.413 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.414 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.414 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:14762 3.1 e221/214 3:87c90742:::benchmark_data_smithi166_26097_object14761:head v 221'910, mlcod=221'906)", 2020-09-08T15:49:28.414 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:24.676910+0000", 2020-09-08T15:49:28.414 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 423.725016466, 2020-09-08T15:49:28.414 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.042098704000000001, 2020-09-08T15:49:28.415 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:28.415 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:28.415 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:28.415 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.415 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:28.416 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.676910+0000", 2020-09-08T15:49:28.416 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:28.416 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.416 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.416 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:28.417 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.676910+0000", 2020-09-08T15:49:28.417 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000103045 2020-09-08T15:49:28.417 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.417 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.417 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:28.418 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677013+0000", 2020-09-08T15:49:28.418 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2190000000000004e-06 2020-09-08T15:49:28.418 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.418 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.418 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:28.419 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677022+0000", 2020-09-08T15:49:28.419 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5857000000000001e-05 2020-09-08T15:49:28.419 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.419 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.419 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:28.420 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677037+0000", 2020-09-08T15:49:28.420 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4070000000000001e-06 2020-09-08T15:49:28.420 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.420 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.421 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:28.421 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677043+0000", 2020-09-08T15:49:28.421 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000109727 2020-09-08T15:49:28.421 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.421 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.422 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:28.422 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677153+0000", 2020-09-08T15:49:28.423 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.9534e-05 2020-09-08T15:49:28.423 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.423 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.423 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:28.424 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677172+0000", 2020-09-08T15:49:28.424 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.041712301 2020-09-08T15:49:28.424 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.424 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.424 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:49:28.425 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.718884+0000", 2020-09-08T15:49:28.425 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000124614 2020-09-08T15:49:28.425 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.425 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.425 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:49:28.426 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.719009+0000", 2020-09-08T15:49:28.426 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.042098704000000001 2020-09-08T15:49:28.426 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.426 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.426 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.427 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.427 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.427 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:14761 3.a e221/214 3:57b84887:::benchmark_data_smithi166_26097_object14760:head v 221'901, mlcod=221'899)", 2020-09-08T15:49:28.427 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:24.677165+0000", 2020-09-08T15:49:28.427 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 423.724761382, 2020-09-08T15:49:28.427 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.041849460999999998, 2020-09-08T15:49:28.428 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:28.428 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:28.428 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:28.428 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.428 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:28.429 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677165+0000", 2020-09-08T15:49:28.429 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:28.429 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.429 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.429 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:28.430 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677165+0000", 2020-09-08T15:49:28.430 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.2948999999999994e-05 2020-09-08T15:49:28.430 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.430 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.430 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:28.431 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677258+0000", 2020-09-08T15:49:28.431 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.1210000000000007e-06 2020-09-08T15:49:28.431 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.431 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.431 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:28.431 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677266+0000", 2020-09-08T15:49:28.432 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6892999999999999e-05 2020-09-08T15:49:28.432 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.432 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.432 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:28.433 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677283+0000", 2020-09-08T15:49:28.434 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4029999999999998e-06 2020-09-08T15:49:28.434 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.434 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.434 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:28.435 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677289+0000", 2020-09-08T15:49:28.435 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.5768e-05 2020-09-08T15:49:28.435 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.435 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.435 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:28.436 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677335+0000", 2020-09-08T15:49:28.436 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.7956000000000001e-05 2020-09-08T15:49:28.436 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.436 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.436 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:28.437 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.677363+0000", 2020-09-08T15:49:28.437 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.041515683999999997 2020-09-08T15:49:28.437 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.437 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.437 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:49:28.437 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.718878+0000", 2020-09-08T15:49:28.438 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000136687 2020-09-08T15:49:28.438 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.438 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.438 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:49:28.438 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:24.719015+0000", 2020-09-08T15:49:28.439 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.041849460999999998 2020-09-08T15:49:28.439 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.439 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.439 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.439 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.440 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.440 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:26823 3.a e222/214 3:5134e458:::benchmark_data_smithi166_26097_object26822:head v 222'1667, mlcod=222'1665)", 2020-09-08T15:49:28.440 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:28.723749+0000", 2020-09-08T15:49:28.440 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 419.67817790800001, 2020-09-08T15:49:28.440 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.17094826399999999, 2020-09-08T15:49:28.441 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:28.441 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:28.441 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:28.441 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.441 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:28.442 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723749+0000", 2020-09-08T15:49:28.442 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:28.442 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.442 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.442 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:28.443 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723749+0000", 2020-09-08T15:49:28.443 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.5164000000000002e-05 2020-09-08T15:49:28.443 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.444 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.444 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:28.444 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723814+0000", 2020-09-08T15:49:28.445 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.4159999999999998e-06 2020-09-08T15:49:28.445 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.445 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.445 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:28.445 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723821+0000", 2020-09-08T15:49:28.446 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.7540000000000001e-06 2020-09-08T15:49:28.446 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.446 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.446 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:28.446 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723831+0000", 2020-09-08T15:49:28.447 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.3930000000000001e-06 2020-09-08T15:49:28.447 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.447 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.447 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:28.447 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723836+0000", 2020-09-08T15:49:28.447 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.0776000000000003e-05 2020-09-08T15:49:28.448 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.448 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.448 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:28.448 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.723886+0000", 2020-09-08T15:49:28.448 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.16580175899999999 2020-09-08T15:49:28.449 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.449 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.449 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:28.449 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.889688+0000", 2020-09-08T15:49:28.449 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0049295679999999996 2020-09-08T15:49:28.450 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.450 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.450 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:49:28.450 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.894618+0000", 2020-09-08T15:49:28.450 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.9433999999999999e-05 2020-09-08T15:49:28.451 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.451 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.451 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:49:28.451 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:28.894697+0000", 2020-09-08T15:49:28.451 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.17094826399999999 2020-09-08T15:49:28.451 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.452 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.452 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.452 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.452 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.452 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:42772 3.a e224/214 3:593c3494:::benchmark_data_smithi166_26097_object42771:head v 224'2705, mlcod=224'2700)", 2020-09-08T15:49:28.453 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.103175+0000", 2020-09-08T15:49:28.453 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 412.29875181699998, 2020-09-08T15:49:28.453 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.063953390999999998, 2020-09-08T15:49:28.453 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:28.454 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:28.455 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:28.455 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.455 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:28.455 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103175+0000", 2020-09-08T15:49:28.455 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:28.456 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.456 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.456 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:28.456 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103175+0000", 2020-09-08T15:49:28.457 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.9434999999999999e-05 2020-09-08T15:49:28.457 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.457 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.457 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:28.457 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103274+0000", 2020-09-08T15:49:28.458 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.5959999999999993e-06 2020-09-08T15:49:28.458 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.458 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.458 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:28.458 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103284+0000", 2020-09-08T15:49:28.459 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5735000000000001e-05 2020-09-08T15:49:28.459 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.459 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.459 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:28.459 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103300+0000", 2020-09-08T15:49:28.460 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.5139999999999998e-06 2020-09-08T15:49:28.460 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.460 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.460 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:28.460 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103306+0000", 2020-09-08T15:49:28.460 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2772000000000002e-05 2020-09-08T15:49:28.461 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.461 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.461 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:28.461 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103359+0000", 2020-09-08T15:49:28.461 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.2966e-05 2020-09-08T15:49:28.462 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.462 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.462 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:28.462 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103382+0000", 2020-09-08T15:49:28.462 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.063606357000000002 2020-09-08T15:49:28.463 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.463 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.463 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:49:28.463 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.166988+0000", 2020-09-08T15:49:28.463 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000140016 2020-09-08T15:49:28.464 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.465 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.465 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:49:28.465 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.167128+0000", 2020-09-08T15:49:28.465 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.063953390999999998 2020-09-08T15:49:28.466 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.466 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.466 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.466 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.466 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.467 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:42775 3.6 e224/217 3:625fe670:::benchmark_data_smithi166_26097_object42774:head v 224'2718, mlcod=224'2716)", 2020-09-08T15:49:28.467 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.103618+0000", 2020-09-08T15:49:28.467 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 412.29830867499999, 2020-09-08T15:49:28.467 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.066506083999999993, 2020-09-08T15:49:28.467 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:28.468 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:28.468 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:28.468 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.468 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:28.468 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103618+0000", 2020-09-08T15:49:28.468 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:28.469 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.469 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.469 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:28.469 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103618+0000", 2020-09-08T15:49:28.469 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011588 2020-09-08T15:49:28.470 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.470 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.470 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:28.470 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103734+0000", 2020-09-08T15:49:28.470 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.5589999999999994e-06 2020-09-08T15:49:28.471 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.471 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.471 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:28.471 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103744+0000", 2020-09-08T15:49:28.471 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5424e-05 2020-09-08T15:49:28.472 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.472 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.472 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:28.472 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103759+0000", 2020-09-08T15:49:28.472 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.64e-06 2020-09-08T15:49:28.472 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.473 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.473 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:28.473 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103766+0000", 2020-09-08T15:49:28.473 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.0469000000000006e-05 2020-09-08T15:49:28.473 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.474 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.474 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:28.474 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103836+0000", 2020-09-08T15:49:28.475 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.7387000000000001e-05 2020-09-08T15:49:28.475 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.475 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.476 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:28.476 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.103863+0000", 2020-09-08T15:49:28.476 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.066159327000000004 2020-09-08T15:49:28.476 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.476 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.477 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:49:28.478 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.170023+0000", 2020-09-08T15:49:28.478 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010139799999999999 2020-09-08T15:49:28.478 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.478 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.483 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:49:28.483 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.170124+0000", 2020-09-08T15:49:28.483 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.066506083999999993 2020-09-08T15:49:28.483 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.483 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.484 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.484 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.484 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.484 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:42778 3.a e224/214 3:57632573:::benchmark_data_smithi166_26097_object42777:head v 224'2706, mlcod=224'2703)", 2020-09-08T15:49:28.484 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.105027+0000", 2020-09-08T15:49:28.486 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 412.29689969100002, 2020-09-08T15:49:28.486 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.067041475000000003, 2020-09-08T15:49:28.486 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:28.487 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:28.487 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:28.487 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.487 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:28.488 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.105027+0000", 2020-09-08T15:49:28.488 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:28.488 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.488 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.489 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:28.489 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.105027+0000", 2020-09-08T15:49:28.489 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4012999999999996e-05 2020-09-08T15:49:28.489 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.489 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.490 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:28.490 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.105121+0000", 2020-09-08T15:49:28.490 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.2180000000000005e-06 2020-09-08T15:49:28.490 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.491 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.491 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:28.491 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.105130+0000", 2020-09-08T15:49:28.491 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4474999999999999e-05 2020-09-08T15:49:28.491 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.492 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.492 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:28.493 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.105145+0000", 2020-09-08T15:49:28.493 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.6489999999999996e-06 2020-09-08T15:49:28.493 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.493 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.493 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:28.494 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.105150+0000", 2020-09-08T15:49:28.494 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.3319000000000003e-05 2020-09-08T15:49:28.494 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.494 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.494 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:28.495 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.105194+0000", 2020-09-08T15:49:28.495 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.1750999999999999e-05 2020-09-08T15:49:28.495 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.495 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.495 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:28.496 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.105216+0000", 2020-09-08T15:49:28.496 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.066681024000000005 2020-09-08T15:49:28.496 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.496 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.496 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:49:28.497 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.171897+0000", 2020-09-08T15:49:28.497 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017202600000000001 2020-09-08T15:49:28.497 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.497 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.497 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:49:28.498 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172069+0000", 2020-09-08T15:49:28.498 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.067041475000000003 2020-09-08T15:49:28.498 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.498 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.498 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.499 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.499 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.499 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:42782 3.6 e224/217 3:6e3106f1:::benchmark_data_smithi166_26097_object42781:head v 224'2719, mlcod=224'2716)", 2020-09-08T15:49:28.499 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.106056+0000", 2020-09-08T15:49:28.499 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 412.295870455, 2020-09-08T15:49:28.500 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.064212001000000005, 2020-09-08T15:49:28.500 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:28.500 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:28.500 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:28.500 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.501 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:28.501 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106056+0000", 2020-09-08T15:49:28.501 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:28.501 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.501 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.502 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:28.502 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106056+0000", 2020-09-08T15:49:28.502 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000108381 2020-09-08T15:49:28.502 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.502 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.503 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:28.503 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106165+0000", 2020-09-08T15:49:28.503 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.8169999999999999e-06 2020-09-08T15:49:28.504 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.504 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.504 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:28.504 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106173+0000", 2020-09-08T15:49:28.504 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4012999999999999e-05 2020-09-08T15:49:28.505 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.505 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.505 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:28.505 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106187+0000", 2020-09-08T15:49:28.505 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.8949999999999997e-06 2020-09-08T15:49:28.506 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.506 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.506 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:28.506 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106191+0000", 2020-09-08T15:49:28.506 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.8047999999999998e-05 2020-09-08T15:49:28.507 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.507 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.507 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:28.507 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106240+0000", 2020-09-08T15:49:28.507 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.4159999999999999e-05 2020-09-08T15:49:28.508 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.508 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.508 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:28.508 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106264+0000", 2020-09-08T15:49:28.508 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.063872915000000002 2020-09-08T15:49:28.509 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.509 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.509 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:49:28.509 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.170137+0000", 2020-09-08T15:49:28.509 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000131772 2020-09-08T15:49:28.510 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.510 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.510 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:49:28.510 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.170268+0000", 2020-09-08T15:49:28.510 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.064212001000000005 2020-09-08T15:49:28.511 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.511 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.511 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.511 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.512 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.512 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:42774 3.e e224/214 3:7fbb735a:::benchmark_data_smithi166_26097_object42773:head v 224'2632, mlcod=224'2629)", 2020-09-08T15:49:28.512 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.106371+0000", 2020-09-08T15:49:28.513 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 412.29555541299999, 2020-09-08T15:49:28.513 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.065972292000000002, 2020-09-08T15:49:28.513 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:28.513 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:28.513 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:28.514 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.514 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:28.514 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106371+0000", 2020-09-08T15:49:28.514 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:28.514 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.515 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.515 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:28.515 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106371+0000", 2020-09-08T15:49:28.515 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000102574 2020-09-08T15:49:28.515 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.516 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.516 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:28.516 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106474+0000", 2020-09-08T15:49:28.516 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.2750000000000001e-06 2020-09-08T15:49:28.516 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.517 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.517 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:28.517 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106483+0000", 2020-09-08T15:49:28.518 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4969e-05 2020-09-08T15:49:28.518 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.518 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.518 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:28.518 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106498+0000", 2020-09-08T15:49:28.519 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9750000000000004e-06 2020-09-08T15:49:28.519 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.519 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.520 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:28.520 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106504+0000", 2020-09-08T15:49:28.520 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00099841199999999995 2020-09-08T15:49:28.520 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.520 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.521 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:28.521 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107503+0000", 2020-09-08T15:49:28.521 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.4423e-05 2020-09-08T15:49:28.521 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.521 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.522 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:28.522 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107527+0000", 2020-09-08T15:49:28.522 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.064517089999999999 2020-09-08T15:49:28.522 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.522 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.523 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:49:28.523 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172044+0000", 2020-09-08T15:49:28.523 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00029957399999999999 2020-09-08T15:49:28.523 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.523 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.523 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:49:28.524 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172344+0000", 2020-09-08T15:49:28.524 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.065972292000000002 2020-09-08T15:49:28.524 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.524 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.524 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.525 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.525 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.525 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:42785 3.a e224/214 3:5993af2f:::benchmark_data_smithi166_26097_object42784:head v 224'2707, mlcod=224'2703)", 2020-09-08T15:49:28.526 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.106902+0000", 2020-09-08T15:49:28.526 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 412.29502530399998, 2020-09-08T15:49:28.526 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.065297044999999998, 2020-09-08T15:49:28.526 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:28.526 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:28.527 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:28.527 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.527 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:28.527 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106902+0000", 2020-09-08T15:49:28.527 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:28.528 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.528 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.528 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:28.528 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.106902+0000", 2020-09-08T15:49:28.528 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000100908 2020-09-08T15:49:28.529 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.529 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.529 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:28.529 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107002+0000", 2020-09-08T15:49:28.529 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.1149999999999994e-06 2020-09-08T15:49:28.529 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.530 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.530 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:28.530 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107011+0000", 2020-09-08T15:49:28.530 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3278999999999999e-05 2020-09-08T15:49:28.531 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.531 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.531 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:28.531 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107024+0000", 2020-09-08T15:49:28.532 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4759999999999997e-06 2020-09-08T15:49:28.532 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.532 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.532 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:28.532 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107029+0000", 2020-09-08T15:49:28.533 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.057070444999999997 2020-09-08T15:49:28.533 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.533 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.533 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:28.533 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.164100+0000", 2020-09-08T15:49:28.534 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9882e-05 2020-09-08T15:49:28.534 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.534 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.534 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:28.534 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.164160+0000", 2020-09-08T15:49:28.535 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0079172059999999995 2020-09-08T15:49:28.535 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.535 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.535 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:49:28.535 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172077+0000", 2020-09-08T15:49:28.536 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000121734 2020-09-08T15:49:28.536 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.536 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.536 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:49:28.536 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172199+0000", 2020-09-08T15:49:28.537 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.065297044999999998 2020-09-08T15:49:28.537 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.537 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.537 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.537 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.538 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.538 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:42788 3.e e224/214 3:731858e7:::benchmark_data_smithi166_26097_object42787:head v 224'2633, mlcod=224'2630)", 2020-09-08T15:49:28.538 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.107430+0000", 2020-09-08T15:49:28.538 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 412.29449696099999, 2020-09-08T15:49:28.538 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.065101027000000006, 2020-09-08T15:49:28.539 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:28.539 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:28.539 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:28.539 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.539 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:28.540 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107430+0000", 2020-09-08T15:49:28.540 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:28.540 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.540 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.540 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:28.541 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107430+0000", 2020-09-08T15:49:28.541 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000105549 2020-09-08T15:49:28.541 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.541 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.541 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:28.542 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107535+0000", 2020-09-08T15:49:28.542 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4570000000000005e-06 2020-09-08T15:49:28.542 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.542 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.542 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:28.543 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107544+0000", 2020-09-08T15:49:28.543 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3541e-05 2020-09-08T15:49:28.543 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.543 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.543 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:28.544 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107557+0000", 2020-09-08T15:49:28.544 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.784e-06 2020-09-08T15:49:28.544 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.544 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.544 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:28.545 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.107563+0000", 2020-09-08T15:49:28.545 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.001228966 2020-09-08T15:49:28.545 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.545 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.545 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:28.546 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.108792+0000", 2020-09-08T15:49:28.546 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4925e-05 2020-09-08T15:49:28.546 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.546 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.546 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:28.546 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.108807+0000", 2020-09-08T15:49:28.547 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.063545818000000004 2020-09-08T15:49:28.547 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.547 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.547 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:49:28.547 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172353+0000", 2020-09-08T15:49:28.548 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000177987 2020-09-08T15:49:28.548 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.548 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.548 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:49:28.548 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172531+0000", 2020-09-08T15:49:28.549 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.065101027000000006 2020-09-08T15:49:28.549 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.549 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.549 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.549 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.550 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.550 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:42789 3.a e224/214 3:58a5da28:::benchmark_data_smithi166_26097_object42788:head v 224'2708, mlcod=224'2703)", 2020-09-08T15:49:28.550 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:42:36.108694+0000", 2020-09-08T15:49:28.550 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 412.29323251, 2020-09-08T15:49:28.550 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.063628563999999999, 2020-09-08T15:49:28.551 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:28.551 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:28.551 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:28.551 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.551 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:28.552 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.108694+0000", 2020-09-08T15:49:28.552 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:28.552 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.552 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.552 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:28.552 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.108694+0000", 2020-09-08T15:49:28.553 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.8534000000000003e-05 2020-09-08T15:49:28.553 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.553 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.553 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:28.553 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.108783+0000", 2020-09-08T15:49:28.554 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.5259999999999993e-06 2020-09-08T15:49:28.554 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.554 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.554 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:28.554 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.108791+0000", 2020-09-08T15:49:28.555 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4341e-05 2020-09-08T15:49:28.555 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.555 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.555 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:28.555 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.108806+0000", 2020-09-08T15:49:28.556 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9630000000000003e-06 2020-09-08T15:49:28.556 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.556 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.556 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:28.556 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.108812+0000", 2020-09-08T15:49:28.557 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.058410995 2020-09-08T15:49:28.557 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.557 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.557 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:28.557 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.167223+0000", 2020-09-08T15:49:28.558 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4601999999999997e-05 2020-09-08T15:49:28.558 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.558 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.558 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:28.558 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.167277+0000", 2020-09-08T15:49:28.559 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0049246059999999998 2020-09-08T15:49:28.559 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.559 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.559 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:49:28.559 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172202+0000", 2020-09-08T15:49:28.559 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012099699999999999 2020-09-08T15:49:28.560 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.560 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.560 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:49:28.560 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:42:36.172323+0000", 2020-09-08T15:49:28.560 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.063628563999999999 2020-09-08T15:49:28.561 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.561 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.561 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.561 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.561 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.562 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:120120 3.e e251/214 3:75c7de04:::benchmark_data_smithi166_26097_object120119:head v 251'7514, mlcod=251'7512)", 2020-09-08T15:49:28.562 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:43:44.687062+0000", 2020-09-08T15:49:28.562 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 343.71486459900001, 2020-09-08T15:49:28.562 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043140563999999999, 2020-09-08T15:49:28.562 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:28.563 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:28.563 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:28.563 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.563 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:28.563 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:43:44.687062+0000", 2020-09-08T15:49:28.564 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:28.564 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.564 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.564 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:28.564 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:43:44.687062+0000", 2020-09-08T15:49:28.565 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013374700000000001 2020-09-08T15:49:28.565 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.565 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.565 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:28.565 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:43:44.687196+0000", 2020-09-08T15:49:28.566 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1669e-05 2020-09-08T15:49:28.566 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.566 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.566 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:28.566 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:43:44.687208+0000", 2020-09-08T15:49:28.566 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.5409999999999999e-05 2020-09-08T15:49:28.567 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.567 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.567 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:28.567 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:43:44.687233+0000", 2020-09-08T15:49:28.567 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1007e-05 2020-09-08T15:49:28.568 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.568 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.568 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:28.568 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:43:44.687244+0000", 2020-09-08T15:49:28.568 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3690999999999997e-05 2020-09-08T15:49:28.569 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.569 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.569 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:28.569 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:43:44.687308+0000", 2020-09-08T15:49:28.569 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.8229e-05 2020-09-08T15:49:28.570 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.570 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.570 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:28.570 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:43:44.687336+0000", 2020-09-08T15:49:28.570 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.042700637999999999 2020-09-08T15:49:28.571 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.571 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.571 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:49:28.571 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:43:44.730037+0000", 2020-09-08T15:49:28.571 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016617300000000001 2020-09-08T15:49:28.572 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.572 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.572 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:49:28.572 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:43:44.730203+0000", 2020-09-08T15:49:28.572 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043140563999999999 2020-09-08T15:49:28.573 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.573 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.573 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.573 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.573 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.574 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:156269 3.3 e264/214 3:c5007835:::benchmark_data_smithi166_26097_object32108:head v 264'9757, mlcod=264'9755)", 2020-09-08T15:49:28.574 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:44:39.699256+0000", 2020-09-08T15:49:28.574 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 288.702670334, 2020-09-08T15:49:28.575 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044242320000000002, 2020-09-08T15:49:28.575 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:28.575 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:28.575 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:28.575 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.576 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:28.576 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699256+0000", 2020-09-08T15:49:28.576 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:28.576 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.576 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.577 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:28.577 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699256+0000", 2020-09-08T15:49:28.577 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.9709000000000005e-05 2020-09-08T15:49:28.577 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.577 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.578 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:28.578 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699346+0000", 2020-09-08T15:49:28.578 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.9023999999999999e-05 2020-09-08T15:49:28.578 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.578 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.578 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:28.579 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699365+0000", 2020-09-08T15:49:28.579 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.1735e-05 2020-09-08T15:49:28.579 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.579 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.579 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:28.580 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699397+0000", 2020-09-08T15:49:28.580 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5481e-05 2020-09-08T15:49:28.580 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.580 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.580 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:28.581 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699412+0000", 2020-09-08T15:49:28.581 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.9234000000000007e-05 2020-09-08T15:49:28.581 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.581 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.581 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:28.582 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699502+0000", 2020-09-08T15:49:28.582 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2763000000000003e-05 2020-09-08T15:49:28.582 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.582 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.582 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:28.583 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699554+0000", 2020-09-08T15:49:28.583 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043752211999999999 2020-09-08T15:49:28.583 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.583 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.583 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:49:28.584 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.743307+0000", 2020-09-08T15:49:28.584 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019216200000000001 2020-09-08T15:49:28.584 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.584 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.585 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:49:28.585 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.743499+0000", 2020-09-08T15:49:28.585 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044242320000000002 2020-09-08T15:49:28.585 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.585 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.586 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.586 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.586 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.586 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:174393 3.a e271/214 3:51162694:::benchmark_data_smithi166_26097_object50232:head v 271'10901, mlcod=271'10899)", 2020-09-08T15:49:28.586 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:45:08.114501+0000", 2020-09-08T15:49:28.587 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 260.28742617, 2020-09-08T15:49:28.589 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.084128171000000002, 2020-09-08T15:49:28.589 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:28.589 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:28.589 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:28.589 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.590 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:28.590 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.114501+0000", 2020-09-08T15:49:28.590 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:28.590 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.590 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.591 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:28.591 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.114501+0000", 2020-09-08T15:49:28.591 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.0494999999999993e-05 2020-09-08T15:49:28.591 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.591 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.592 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:28.592 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.114591+0000", 2020-09-08T15:49:28.592 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2646000000000001e-05 2020-09-08T15:49:28.592 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.593 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.593 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:28.593 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.114604+0000", 2020-09-08T15:49:28.593 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.0993999999999999e-05 2020-09-08T15:49:28.594 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.594 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.594 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:28.594 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.114625+0000", 2020-09-08T15:49:28.594 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4939999999999992e-06 2020-09-08T15:49:28.595 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.595 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.595 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:28.595 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.114634+0000", 2020-09-08T15:49:28.595 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.9614e-05 2020-09-08T15:49:28.596 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.596 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.596 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:28.596 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.114724+0000", 2020-09-08T15:49:28.596 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.2058999999999998e-05 2020-09-08T15:49:28.597 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.597 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.597 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:28.597 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.114766+0000", 2020-09-08T15:49:28.597 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0091200340000000008 2020-09-08T15:49:28.597 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.598 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.599 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:49:28.599 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.123886+0000", 2020-09-08T15:49:28.599 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.074742834999999994 2020-09-08T15:49:28.600 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.600 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.600 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:49:28.600 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.198629+0000", 2020-09-08T15:49:28.601 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.084128171000000002 2020-09-08T15:49:28.601 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.601 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.601 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.601 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.602 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.602 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:178736 3.a e273/214 3:55ded067:::benchmark_data_smithi166_26097_object54575:head v 273'11171, mlcod=273'11167)", 2020-09-08T15:49:28.603 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:45:14.706046+0000", 2020-09-08T15:49:28.603 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 253.695880695, 2020-09-08T15:49:28.603 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.041815702000000003, 2020-09-08T15:49:28.604 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:28.604 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:28.604 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:28.604 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.605 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:28.605 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.706046+0000", 2020-09-08T15:49:28.605 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:28.605 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.605 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.606 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:28.606 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.706046+0000", 2020-09-08T15:49:28.606 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.0309000000000003e-05 2020-09-08T15:49:28.606 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.606 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.607 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:28.607 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.706086+0000", 2020-09-08T15:49:28.607 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.2119999999999997e-06 2020-09-08T15:49:28.607 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.607 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.608 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:28.608 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.706094+0000", 2020-09-08T15:49:28.608 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.7219999999999994e-06 2020-09-08T15:49:28.608 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.608 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.608 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:28.609 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.706103+0000", 2020-09-08T15:49:28.609 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.5419999999999996e-06 2020-09-08T15:49:28.609 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.609 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.609 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:28.610 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.706109+0000", 2020-09-08T15:49:28.610 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.2021000000000003e-05 2020-09-08T15:49:28.610 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.610 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.610 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:28.611 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.706151+0000", 2020-09-08T15:49:28.611 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.0727e-05 2020-09-08T15:49:28.611 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.611 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.611 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:28.612 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.706172+0000", 2020-09-08T15:49:28.612 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.041602410999999999 2020-09-08T15:49:28.612 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.612 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.612 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:49:28.613 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.747774+0000", 2020-09-08T15:49:28.613 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.7757999999999996e-05 2020-09-08T15:49:28.613 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.614 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.614 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:49:28.614 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:14.747862+0000", 2020-09-08T15:49:28.614 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.041815702000000003 2020-09-08T15:49:28.614 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.615 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.615 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.615 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.615 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.615 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:188875 3.e e277/214 3:78e0085b:::benchmark_data_smithi166_26097_object64714:head v 277'11767, mlcod=277'11764)", 2020-09-08T15:49:28.615 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:45:29.700758+0000", 2020-09-08T15:49:28.616 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 238.701168704, 2020-09-08T15:49:28.616 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.051517127000000003, 2020-09-08T15:49:28.616 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:28.616 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:28.616 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:28.617 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.617 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:28.617 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.700758+0000", 2020-09-08T15:49:28.617 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:28.617 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.618 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.618 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:28.618 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.700758+0000", 2020-09-08T15:49:28.618 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000105216 2020-09-08T15:49:28.619 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.619 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.619 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:28.619 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.700863+0000", 2020-09-08T15:49:28.619 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6963999999999999e-05 2020-09-08T15:49:28.620 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.620 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.620 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:28.620 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.700880+0000", 2020-09-08T15:49:28.621 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.6429000000000001e-05 2020-09-08T15:49:28.621 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.621 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.621 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:28.621 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.700907+0000", 2020-09-08T15:49:28.621 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1688000000000001e-05 2020-09-08T15:49:28.622 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.622 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.622 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:28.622 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.700918+0000", 2020-09-08T15:49:28.622 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.9370999999999997e-05 2020-09-08T15:49:28.623 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.623 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.623 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:28.623 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.700988+0000", 2020-09-08T15:49:28.623 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.7089000000000001e-05 2020-09-08T15:49:28.624 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.624 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.624 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:28.624 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.701015+0000", 2020-09-08T15:49:28.624 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.051182631999999999 2020-09-08T15:49:28.624 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.625 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.625 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:49:28.625 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.752197+0000", 2020-09-08T15:49:28.625 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.7737999999999993e-05 2020-09-08T15:49:28.625 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.626 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.626 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:49:28.626 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.752275+0000", 2020-09-08T15:49:28.626 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.051517127000000003 2020-09-08T15:49:28.626 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.627 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.627 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.627 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.627 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.627 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:209805 3.a e286/214 3:5f9a82ac:::benchmark_data_smithi166_26097_object85644:head v 286'13119, mlcod=286'13117)", 2020-09-08T15:49:28.628 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:45:57.115910+0000", 2020-09-08T15:49:28.628 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 211.28601695399999, 2020-09-08T15:49:28.628 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.053311198999999997, 2020-09-08T15:49:28.628 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:28.628 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:28.629 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:28.629 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.629 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:28.629 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:57.115910+0000", 2020-09-08T15:49:28.629 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:28.630 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.630 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.630 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:28.630 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:57.115910+0000", 2020-09-08T15:49:28.630 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.9079999999999998e-05 2020-09-08T15:49:28.630 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.631 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.631 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:28.631 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:57.115959+0000", 2020-09-08T15:49:28.631 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.3579999999999997e-06 2020-09-08T15:49:28.631 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.632 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.632 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:28.632 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:57.115966+0000", 2020-09-08T15:49:28.632 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1231e-05 2020-09-08T15:49:28.633 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.633 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.633 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:28.633 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:57.115978+0000", 2020-09-08T15:49:28.633 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.463e-06 2020-09-08T15:49:28.634 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.634 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.634 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:28.634 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:57.115983+0000", 2020-09-08T15:49:28.634 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4694999999999997e-05 2020-09-08T15:49:28.635 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.635 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.635 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:28.635 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:57.116038+0000", 2020-09-08T15:49:28.635 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.6264000000000001e-05 2020-09-08T15:49:28.636 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.636 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.636 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:28.636 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:57.116064+0000", 2020-09-08T15:49:28.636 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.053111716000000003 2020-09-08T15:49:28.636 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.637 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.637 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:49:28.637 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:57.169176+0000", 2020-09-08T15:49:28.637 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.5392000000000002e-05 2020-09-08T15:49:28.637 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.638 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.638 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:49:28.638 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:57.169221+0000", 2020-09-08T15:49:28.638 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.053311198999999997 2020-09-08T15:49:28.639 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.639 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.639 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.639 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.639 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.639 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35381.0:158549 4.d e354/314 4:bce4dc56:::benchmark_data_smithi166_32054_object16407:head v 354'9897, mlcod=354'9895)", 2020-09-08T15:49:28.640 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:49:10.974090+0000", 2020-09-08T15:49:28.640 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 17.427837211, 2020-09-08T15:49:28.640 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.112964493, 2020-09-08T15:49:28.640 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:28.640 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:28.641 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:28.641 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.641 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:28.641 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974090+0000", 2020-09-08T15:49:28.641 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:28.642 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.642 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.642 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:28.642 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974090+0000", 2020-09-08T15:49:28.642 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.7972000000000002e-05 2020-09-08T15:49:28.643 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.643 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.643 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:28.643 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974128+0000", 2020-09-08T15:49:28.643 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0639999999999999e-06 2020-09-08T15:49:28.643 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.644 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.644 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:28.644 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974134+0000", 2020-09-08T15:49:28.644 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.1800000000000002e-06 2020-09-08T15:49:28.644 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.645 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.645 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:28.645 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974143+0000", 2020-09-08T15:49:28.645 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.8539999999999996e-06 2020-09-08T15:49:28.645 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.646 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.646 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:28.646 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974148+0000", 2020-09-08T15:49:28.646 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9451999999999999e-05 2020-09-08T15:49:28.646 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.646 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.647 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:28.647 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974207+0000", 2020-09-08T15:49:28.647 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.6696e-05 2020-09-08T15:49:28.647 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.647 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.648 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:28.648 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974234+0000", 2020-09-08T15:49:28.648 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0053667009999999998 2020-09-08T15:49:28.648 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.648 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.649 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:49:28.649 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.979601+0000", 2020-09-08T15:49:28.649 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.107453574 2020-09-08T15:49:28.649 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.649 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.650 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:49:28.650 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:11.087054+0000", 2020-09-08T15:49:28.650 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.112964493 2020-09-08T15:49:28.650 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.650 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.650 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.651 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.651 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.651 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35381.0:173566 4.d e358/314 4:b1bda86a:::benchmark_data_smithi166_32054_object31424:head v 358'10850, mlcod=358'10848)", 2020-09-08T15:49:28.651 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:49:25.745617+0000", 2020-09-08T15:49:28.651 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 2.6563099210000001, 2020-09-08T15:49:28.652 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.047025483999999999, 2020-09-08T15:49:28.652 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:28.652 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:49:28.652 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:28.652 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.653 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:28.653 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:25.745617+0000", 2020-09-08T15:49:28.653 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:28.653 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.653 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.654 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:28.654 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:25.745617+0000", 2020-09-08T15:49:28.654 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.7506999999999999e-05 2020-09-08T15:49:28.654 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.654 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.654 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:28.655 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:25.745654+0000", 2020-09-08T15:49:28.655 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0630000000000002e-06 2020-09-08T15:49:28.655 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.655 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.655 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:28.656 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:25.745660+0000", 2020-09-08T15:49:28.656 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.0829999999999993e-06 2020-09-08T15:49:28.656 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.656 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.656 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:28.657 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:25.745670+0000", 2020-09-08T15:49:28.657 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.6099999999999999e-06 2020-09-08T15:49:28.657 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.657 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.657 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:28.657 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:25.745674+0000", 2020-09-08T15:49:28.658 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.033268842 2020-09-08T15:49:28.658 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.658 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.658 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:28.658 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:25.778943+0000", 2020-09-08T15:49:28.659 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.1858000000000003e-05 2020-09-08T15:49:28.659 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.659 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.659 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:28.659 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:25.779005+0000", 2020-09-08T15:49:28.660 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.013550797999999999 2020-09-08T15:49:28.660 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.660 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.660 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:49:28.660 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:25.792556+0000", 2020-09-08T15:49:28.660 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.6723000000000003e-05 2020-09-08T15:49:28.661 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:28.661 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:28.661 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:49:28.661 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:25.792642+0000", 2020-09-08T15:49:28.661 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.047025483999999999 2020-09-08T15:49:28.662 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.662 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.662 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.662 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:28.662 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:28.663 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:49:28.711 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:49:28.816 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:28.833 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:28 smithi055 bash[9296]: cluster 2020-09-08T15:49:27.780181+0000 mgr.x (mgr.24118) 664 : cluster [DBG] pgmap v939: 17 pgs: 17 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T15:49:29.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:28 smithi166 bash[9446]: cluster 2020-09-08T15:49:27.780181+0000 mgr.x (mgr.24118) 664 : cluster [DBG] pgmap v939: 17 pgs: 17 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T15:49:29.178 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:49:28 smithi104 bash[22305]: debug 2020-09-08T15:49:28.707+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:29.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:28 smithi104 bash[9203]: cluster 2020-09-08T15:49:27.780181+0000 mgr.x (mgr.24118) 664 : cluster [DBG] pgmap v939: 17 pgs: 17 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T15:49:29.190 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:49:29.346 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:49:29.418 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2020-09-08T15:49:29.574 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:49:31.739 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:49:31.845 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:31 smithi104 bash[9203]: cluster 2020-09-08T15:49:29.780720+0000 mgr.x (mgr.24118) 665 : cluster [DBG] pgmap v940: 17 pgs: 17 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.15k op/s 2020-09-08T15:49:31.846 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:31 smithi104 bash[9203]: audit 2020-09-08T15:49:31.516531+0000 mon.a (mon.0) 1414 : audit [INF] from='client.? v1:172.21.15.104:0/4117662959' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:49:32.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:31 smithi166 bash[9446]: cluster 2020-09-08T15:49:29.780720+0000 mgr.x (mgr.24118) 665 : cluster [DBG] pgmap v940: 17 pgs: 17 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.15k op/s 2020-09-08T15:49:32.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:31 smithi166 bash[9446]: audit 2020-09-08T15:49:31.516531+0000 mon.a (mon.0) 1414 : audit [INF] from='client.? v1:172.21.15.104:0/4117662959' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:49:32.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:31 smithi055 bash[9296]: cluster 2020-09-08T15:49:29.780720+0000 mgr.x (mgr.24118) 665 : cluster [DBG] pgmap v940: 17 pgs: 17 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.15k op/s 2020-09-08T15:49:32.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:31 smithi055 bash[9296]: audit 2020-09-08T15:49:31.516531+0000 mon.a (mon.0) 1414 : audit [INF] from='client.? v1:172.21.15.104:0/4117662959' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:49:33.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:32 smithi166 bash[9446]: cluster 2020-09-08T15:49:31.624449+0000 osd.8 (osd.8) 47 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:33.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:32 smithi166 bash[9446]: cluster 2020-09-08T15:49:31.624543+0000 osd.8 (osd.8) 48 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:33.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:32 smithi166 bash[9446]: cluster 2020-09-08T15:49:31.736106+0000 mon.a (mon.0) 1415 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:49:33.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:32 smithi166 bash[9446]: audit 2020-09-08T15:49:31.741242+0000 mon.a (mon.0) 1416 : audit [INF] from='client.? v1:172.21.15.104:0/4117662959' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:49:33.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:32 smithi166 bash[9446]: cluster 2020-09-08T15:49:31.741304+0000 mon.a (mon.0) 1417 : cluster [DBG] osdmap e360: 12 total, 12 up, 12 in 2020-09-08T15:49:33.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:32 smithi166 bash[9446]: cluster 2020-09-08T15:49:31.781184+0000 mgr.x (mgr.24118) 666 : cluster [DBG] pgmap v942: 17 pgs: 17 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 760 op/s 2020-09-08T15:49:33.140 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:33.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:32 smithi055 bash[9296]: cluster 2020-09-08T15:49:31.624449+0000 osd.8 (osd.8) 47 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:33.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:32 smithi055 bash[9296]: cluster 2020-09-08T15:49:31.624543+0000 osd.8 (osd.8) 48 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:33.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:32 smithi055 bash[9296]: cluster 2020-09-08T15:49:31.736106+0000 mon.a (mon.0) 1415 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:49:33.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:32 smithi055 bash[9296]: audit 2020-09-08T15:49:31.741242+0000 mon.a (mon.0) 1416 : audit [INF] from='client.? v1:172.21.15.104:0/4117662959' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:49:33.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:32 smithi055 bash[9296]: cluster 2020-09-08T15:49:31.741304+0000 mon.a (mon.0) 1417 : cluster [DBG] osdmap e360: 12 total, 12 up, 12 in 2020-09-08T15:49:33.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:32 smithi055 bash[9296]: cluster 2020-09-08T15:49:31.781184+0000 mgr.x (mgr.24118) 666 : cluster [DBG] pgmap v942: 17 pgs: 17 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 760 op/s 2020-09-08T15:49:33.184 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:32 smithi104 bash[9203]: cluster 2020-09-08T15:49:31.624449+0000 osd.8 (osd.8) 47 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:33.185 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:32 smithi104 bash[9203]: cluster 2020-09-08T15:49:31.624543+0000 osd.8 (osd.8) 48 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:33.185 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:32 smithi104 bash[9203]: cluster 2020-09-08T15:49:31.736106+0000 mon.a (mon.0) 1415 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:49:33.185 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:32 smithi104 bash[9203]: audit 2020-09-08T15:49:31.741242+0000 mon.a (mon.0) 1416 : audit [INF] from='client.? v1:172.21.15.104:0/4117662959' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:49:33.186 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:32 smithi104 bash[9203]: cluster 2020-09-08T15:49:31.741304+0000 mon.a (mon.0) 1417 : cluster [DBG] osdmap e360: 12 total, 12 up, 12 in 2020-09-08T15:49:33.186 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:32 smithi104 bash[9203]: cluster 2020-09-08T15:49:31.781184+0000 mgr.x (mgr.24118) 666 : cluster [DBG] pgmap v942: 17 pgs: 17 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 760 op/s 2020-09-08T15:49:33.245 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:33.616 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:49:33 smithi166 bash[14254]: debug 2020-09-08T15:49:33.129+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:33.866 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:49:33.867 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T15:49:33.867 INFO:teuthology.orchestra.run.smithi055.stdout: "num_ops": 0 2020-09-08T15:49:33.867 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:49:34.006 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:34.108 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:34.338 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:34.369 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:49:34 smithi166 bash[16593]: debug 2020-09-08T15:49:34.001+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:34.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:49:34 smithi104 bash[15273]: debug 2020-09-08T15:49:34.339+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:34.442 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:34.576 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:34.684 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:34.772 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:49:34.797 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2020-09-08T15:49:34.800 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:49:34.812 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:34.830 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:49:34 smithi104 bash[17588]: debug 2020-09-08T15:49:34.575+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:34.830 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:34 smithi104 bash[9203]: cluster 2020-09-08T15:49:33.781698+0000 mgr.x (mgr.24118) 667 : cluster [DBG] pgmap v943: 17 pgs: 17 active+clean; 6.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:49:34.831 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:34 smithi104 bash[9203]: audit 2020-09-08T15:49:34.731408+0000 mon.a (mon.0) 1418 : audit [DBG] from='client.? v1:172.21.15.104:0/349956843' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:49:34.832 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:49:34.864 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:49:34.901 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:49:34.916 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:49:34.939 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:49:34.956 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:49:34.974 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:49:35.010 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:49:35.041 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:49:35.047 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:49:35.075 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:49:35.101 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:49:35.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:34 smithi166 bash[9446]: cluster 2020-09-08T15:49:33.781698+0000 mgr.x (mgr.24118) 667 : cluster [DBG] pgmap v943: 17 pgs: 17 active+clean; 6.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:49:35.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:34 smithi166 bash[9446]: audit 2020-09-08T15:49:34.731408+0000 mon.a (mon.0) 1418 : audit [DBG] from='client.? v1:172.21.15.104:0/349956843' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:49:35.133 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:49:35.135 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:49:35.135 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:49:35.136 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:49:35.136 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:49:35.136 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:49:35.136 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:49:35.137 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:49:35.137 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:49:35.137 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:49:35.137 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:49:35.138 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:49:35.138 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:49:35.151 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:35.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:49:34 smithi055 bash[11956]: debug 2020-09-08T15:49:34.812+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:34 smithi055 bash[9296]: cluster 2020-09-08T15:49:33.781698+0000 mgr.x (mgr.24118) 667 : cluster [DBG] pgmap v943: 17 pgs: 17 active+clean; 6.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:49:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:34 smithi055 bash[9296]: audit 2020-09-08T15:49:34.731408+0000 mon.a (mon.0) 1418 : audit [DBG] from='client.? v1:172.21.15.104:0/349956843' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:49:35.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:49:35 smithi104 bash[22305]: debug 2020-09-08T15:49:35.051+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:35.188 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:49:35.296 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:35.353 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:49:35.410 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:49:35.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:49:35 smithi166 bash[11928]: debug 2020-09-08T15:49:35.289+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:36.085 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:49:36.190 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:36.311 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:36.415 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:36.422 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:49:36 smithi104 bash[19922]: debug 2020-09-08T15:49:36.087+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:36.476 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:49:36 smithi055 bash[16646]: debug 2020-09-08T15:49:36.312+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:36.917 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:37.021 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:49:37.030 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:36 smithi055 bash[9296]: cluster 2020-09-08T15:49:35.782145+0000 mgr.x (mgr.24118) 668 : cluster [DBG] pgmap v944: 17 pgs: 17 active+clean; 6.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:49:37.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:36 smithi166 bash[9446]: cluster 2020-09-08T15:49:35.782145+0000 mgr.x (mgr.24118) 668 : cluster [DBG] pgmap v944: 17 pgs: 17 active+clean; 6.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:49:37.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:49:36 smithi104 bash[15273]: debug 2020-09-08T15:49:36.919+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:37.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:36 smithi104 bash[9203]: cluster 2020-09-08T15:49:35.782145+0000 mgr.x (mgr.24118) 668 : cluster [DBG] pgmap v944: 17 pgs: 17 active+clean; 6.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:49:37.503 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:49:37.605 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:49:37.785 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:49:37.889 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:49:37.905 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:49:37 smithi104 bash[19922]: debug 2020-09-08T15:49:37.783+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:37.906 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:49:37 smithi055 bash[19003]: debug 2020-09-08T15:49:37.504+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:38.558 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:49:38.664 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:49:38.782 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:49:38.843 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:49:38 smithi104 bash[22305]: debug 2020-09-08T15:49:38.779+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:38.843 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:49:38 smithi104 bash[19922]: debug 2020-09-08T15:49:38.559+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:38.891 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:38.909 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:49:39.032 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:39.069 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:49:39.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:38 smithi166 bash[9446]: cluster 2020-09-08T15:49:37.782666+0000 mgr.x (mgr.24118) 669 : cluster [DBG] pgmap v945: 17 pgs: 17 active+clean; 6.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:49:39.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:49:39 smithi166 bash[16593]: debug 2020-09-08T15:49:39.025+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:39.135 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:39.140 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:38 smithi055 bash[9296]: cluster 2020-09-08T15:49:37.782666+0000 mgr.x (mgr.24118) 669 : cluster [DBG] pgmap v945: 17 pgs: 17 active+clean; 6.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:49:39.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:38 smithi104 bash[9203]: cluster 2020-09-08T15:49:37.782666+0000 mgr.x (mgr.24118) 669 : cluster [DBG] pgmap v945: 17 pgs: 17 active+clean; 6.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:49:39.287 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:39.328 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:49:39.328 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:49:39.391 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:39.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:49:39 smithi055 bash[16646]: debug 2020-09-08T15:49:39.288+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:39.520 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:39.628 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:39.811 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:39.869 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:49:39 smithi166 bash[11928]: debug 2020-09-08T15:49:39.513+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:39.914 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:39.934 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:49:39 smithi104 bash[17588]: debug 2020-09-08T15:49:39.815+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:40.040 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:40.146 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:40.231 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2020-09-08T15:49:40.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:49:40 smithi166 bash[14254]: debug 2020-09-08T15:49:40.037+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:40.441 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:49:40.524 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:40.626 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:40.839 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:49:40 smithi104 bash[17588]: debug 2020-09-08T15:49:40.519+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:40.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:40 smithi055 bash[9296]: cluster 2020-09-08T15:49:39.295541+0000 osd.1 (osd.1) 68 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:40.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:40 smithi055 bash[9296]: cluster 2020-09-08T15:49:39.295638+0000 osd.1 (osd.1) 69 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:40.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:40 smithi055 bash[9296]: cluster 2020-09-08T15:49:39.783068+0000 mgr.x (mgr.24118) 670 : cluster [DBG] pgmap v946: 17 pgs: 17 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T15:49:40.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:40 smithi055 bash[9296]: cluster 2020-09-08T15:49:39.847732+0000 mon.a (mon.0) 1419 : cluster [DBG] osdmap e361: 12 total, 12 up, 12 in 2020-09-08T15:49:41.089 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:41.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:40 smithi166 bash[9446]: cluster 2020-09-08T15:49:39.295541+0000 osd.1 (osd.1) 68 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:41.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:40 smithi166 bash[9446]: cluster 2020-09-08T15:49:39.295638+0000 osd.1 (osd.1) 69 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:41.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:40 smithi166 bash[9446]: cluster 2020-09-08T15:49:39.783068+0000 mgr.x (mgr.24118) 670 : cluster [DBG] pgmap v946: 17 pgs: 17 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T15:49:41.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:40 smithi166 bash[9446]: cluster 2020-09-08T15:49:39.847732+0000 mon.a (mon.0) 1419 : cluster [DBG] osdmap e361: 12 total, 12 up, 12 in 2020-09-08T15:49:41.154 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:40 smithi104 bash[9203]: cluster 2020-09-08T15:49:39.295541+0000 osd.1 (osd.1) 68 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:41.154 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:40 smithi104 bash[9203]: cluster 2020-09-08T15:49:39.295638+0000 osd.1 (osd.1) 69 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:41.154 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:40 smithi104 bash[9203]: cluster 2020-09-08T15:49:39.783068+0000 mgr.x (mgr.24118) 670 : cluster [DBG] pgmap v946: 17 pgs: 17 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T15:49:41.155 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:40 smithi104 bash[9203]: cluster 2020-09-08T15:49:39.847732+0000 mon.a (mon.0) 1419 : cluster [DBG] osdmap e361: 12 total, 12 up, 12 in 2020-09-08T15:49:41.194 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:41.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:49:41 smithi055 bash[11956]: debug 2020-09-08T15:49:41.088+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:41.751 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:41.849 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:41 smithi104 bash[9203]: audit 2020-09-08T15:49:41.692159+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? v1:172.21.15.104:0/3898704344' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:49:41.849 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:41 smithi104 bash[9203]: audit 2020-09-08T15:49:41.694969+0000 mon.b (mon.1) 89 : audit [INF] from='client.15615 v1:172.21.15.104:0/3898704344' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:49:41.849 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:41 smithi104 bash[9203]: audit 2020-09-08T15:49:41.695438+0000 mon.a (mon.0) 1421 : audit [INF] from='client.15615 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:49:41.855 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:49:41.856 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:49:42.049 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:49:41 smithi055 bash[16646]: debug 2020-09-08T15:49:41.752+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:42.050 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:41 smithi055 bash[9296]: audit 2020-09-08T15:49:41.692159+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? v1:172.21.15.104:0/3898704344' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:49:42.050 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:41 smithi055 bash[9296]: audit 2020-09-08T15:49:41.694969+0000 mon.b (mon.1) 89 : audit [INF] from='client.15615 v1:172.21.15.104:0/3898704344' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:49:42.050 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:41 smithi055 bash[9296]: audit 2020-09-08T15:49:41.695438+0000 mon.a (mon.0) 1421 : audit [INF] from='client.15615 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:49:42.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:41 smithi166 bash[9446]: audit 2020-09-08T15:49:41.692159+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? v1:172.21.15.104:0/3898704344' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:49:42.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:41 smithi166 bash[9446]: audit 2020-09-08T15:49:41.694969+0000 mon.b (mon.1) 89 : audit [INF] from='client.15615 v1:172.21.15.104:0/3898704344' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:49:42.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:41 smithi166 bash[9446]: audit 2020-09-08T15:49:41.695438+0000 mon.a (mon.0) 1421 : audit [INF] from='client.15615 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:49:42.650 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:49:42.757 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:49:42.894 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:49:42.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:49:42 smithi104 bash[22305]: debug 2020-09-08T15:49:42.651+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:42.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:42 smithi104 bash[9203]: cluster 2020-09-08T15:49:41.783633+0000 mgr.x (mgr.24118) 671 : cluster [DBG] pgmap v948: 17 pgs: 17 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T15:49:42.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:42 smithi104 bash[9203]: cluster 2020-09-08T15:49:41.844933+0000 mon.a (mon.0) 1422 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:49:42.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:42 smithi104 bash[9203]: audit 2020-09-08T15:49:41.849615+0000 mon.a (mon.0) 1423 : audit [INF] from='client.15615 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:49:42.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:42 smithi104 bash[9203]: cluster 2020-09-08T15:49:41.849654+0000 mon.a (mon.0) 1424 : cluster [DBG] osdmap e362: 12 total, 12 up, 12 in 2020-09-08T15:49:43.000 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:43.106 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:42 smithi166 bash[9446]: cluster 2020-09-08T15:49:41.783633+0000 mgr.x (mgr.24118) 671 : cluster [DBG] pgmap v948: 17 pgs: 17 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T15:49:43.107 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:42 smithi166 bash[9446]: cluster 2020-09-08T15:49:41.844933+0000 mon.a (mon.0) 1422 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:49:43.107 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:42 smithi166 bash[9446]: audit 2020-09-08T15:49:41.849615+0000 mon.a (mon.0) 1423 : audit [INF] from='client.15615 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:49:43.107 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:42 smithi166 bash[9446]: cluster 2020-09-08T15:49:41.849654+0000 mon.a (mon.0) 1424 : cluster [DBG] osdmap e362: 12 total, 12 up, 12 in 2020-09-08T15:49:43.107 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:49:42 smithi166 bash[18960]: debug 2020-09-08T15:49:42.893+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:43.109 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:43.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:42 smithi055 bash[9296]: cluster 2020-09-08T15:49:41.783633+0000 mgr.x (mgr.24118) 671 : cluster [DBG] pgmap v948: 17 pgs: 17 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T15:49:43.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:42 smithi055 bash[9296]: cluster 2020-09-08T15:49:41.844933+0000 mon.a (mon.0) 1422 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:49:43.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:42 smithi055 bash[9296]: audit 2020-09-08T15:49:41.849615+0000 mon.a (mon.0) 1423 : audit [INF] from='client.15615 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:49:43.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:42 smithi055 bash[9296]: cluster 2020-09-08T15:49:41.849654+0000 mon.a (mon.0) 1424 : cluster [DBG] osdmap e362: 12 total, 12 up, 12 in 2020-09-08T15:49:43.213 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:49:43.364 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:49:43.370 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:49:43 smithi166 bash[14254]: debug 2020-09-08T15:49:43.105+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:43.471 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:43.480 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:49:43 smithi104 bash[19922]: debug 2020-09-08T15:49:43.367+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:43.941 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:44.046 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:49:44.430 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:49:43 smithi104 bash[15273]: debug 2020-09-08T15:49:43.939+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:44.591 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:49:44.647 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:49:44.647 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:49:44.698 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:44.918 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:49:44 smithi104 bash[22305]: debug 2020-09-08T15:49:44.595+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:44.919 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:45.024 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:45.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:49:44 smithi104 bash[15273]: debug 2020-09-08T15:49:44.919+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:45.189 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:45.248 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:49:45.275 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:49:45.291 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:49:45.305 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:49:45.340 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:49:45.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:49:45 smithi166 bash[14254]: debug 2020-09-08T15:49:45.185+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:45.379 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:49:45.414 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:49:45.448 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:49:45.450 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:49:45.456 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:49:45.483 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:49:45.518 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:49:45.558 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:49:45.559 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:49:45 smithi055 bash[14301]: debug 2020-09-08T15:49:45.456+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:45.561 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2020-09-08T15:49:45.562 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:45.598 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:49:45.618 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:49:45.638 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:49:45.652 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:49:45.653 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:49:45.653 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:49:45.653 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:49:45.653 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:49:45.654 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:49:45.654 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:49:45.654 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:49:45.654 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:49:45.654 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:49:45.655 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:49:45.655 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:49:45.710 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:45.712 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:49:45.803 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:49:45 smithi104 bash[17588]: debug 2020-09-08T15:49:45.703+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:45.823 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:45.835 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:45 smithi055 bash[9296]: cluster 2020-09-08T15:49:43.784181+0000 mgr.x (mgr.24118) 672 : cluster [DBG] pgmap v950: 17 pgs: 17 active+clean; 5.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:49:45.835 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:45 smithi055 bash[9296]: cluster 2020-09-08T15:49:44.461476+0000 osd.6 (osd.6) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:45.835 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:45 smithi055 bash[9296]: cluster 2020-09-08T15:49:44.461567+0000 osd.6 (osd.6) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:45.835 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:45 smithi055 bash[9296]: cluster 2020-09-08T15:49:44.868434+0000 mon.a (mon.0) 1425 : cluster [DBG] osdmap e363: 12 total, 12 up, 12 in 2020-09-08T15:49:45.836 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:45 smithi055 bash[9296]: audit 2020-09-08T15:49:45.214476+0000 mon.b (mon.1) 90 : audit [DBG] from='client.? v1:172.21.15.104:0/1153988522' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:49:46.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:45 smithi166 bash[9446]: cluster 2020-09-08T15:49:43.784181+0000 mgr.x (mgr.24118) 672 : cluster [DBG] pgmap v950: 17 pgs: 17 active+clean; 5.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:49:46.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:45 smithi166 bash[9446]: cluster 2020-09-08T15:49:44.461476+0000 osd.6 (osd.6) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:46.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:45 smithi166 bash[9446]: cluster 2020-09-08T15:49:44.461567+0000 osd.6 (osd.6) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:46.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:45 smithi166 bash[9446]: cluster 2020-09-08T15:49:44.868434+0000 mon.a (mon.0) 1425 : cluster [DBG] osdmap e363: 12 total, 12 up, 12 in 2020-09-08T15:49:46.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:45 smithi166 bash[9446]: audit 2020-09-08T15:49:45.214476+0000 mon.b (mon.1) 90 : audit [DBG] from='client.? v1:172.21.15.104:0/1153988522' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:49:46.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:45 smithi104 bash[9203]: cluster 2020-09-08T15:49:43.784181+0000 mgr.x (mgr.24118) 672 : cluster [DBG] pgmap v950: 17 pgs: 17 active+clean; 5.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:49:46.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:45 smithi104 bash[9203]: cluster 2020-09-08T15:49:44.461476+0000 osd.6 (osd.6) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:46.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:45 smithi104 bash[9203]: cluster 2020-09-08T15:49:44.461567+0000 osd.6 (osd.6) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:46.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:45 smithi104 bash[9203]: cluster 2020-09-08T15:49:44.868434+0000 mon.a (mon.0) 1425 : cluster [DBG] osdmap e363: 12 total, 12 up, 12 in 2020-09-08T15:49:46.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:45 smithi104 bash[9203]: audit 2020-09-08T15:49:45.214476+0000 mon.b (mon.1) 90 : audit [DBG] from='client.? v1:172.21.15.104:0/1153988522' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:49:46.353 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:46.455 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:46.608 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:46.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:49:46 smithi055 bash[11956]: debug 2020-09-08T15:49:46.352+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:46.712 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:49:46.720 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:49:46 smithi166 bash[11928]: debug 2020-09-08T15:49:46.605+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:46.864 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:49:46.968 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:49:47.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:46 smithi166 bash[9446]: cluster 2020-09-08T15:49:45.784553+0000 mgr.x (mgr.24118) 673 : cluster [DBG] pgmap v952: 17 pgs: 17 active+clean; 5.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 841 op/s 2020-09-08T15:49:47.117 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:49:46 smithi166 bash[18960]: debug 2020-09-08T15:49:46.861+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:47.133 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:46 smithi055 bash[9296]: cluster 2020-09-08T15:49:45.784553+0000 mgr.x (mgr.24118) 673 : cluster [DBG] pgmap v952: 17 pgs: 17 active+clean; 5.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 841 op/s 2020-09-08T15:49:47.133 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:49:47 smithi055 bash[14301]: debug 2020-09-08T15:49:47.136+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:47.134 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:49:47.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:46 smithi104 bash[9203]: cluster 2020-09-08T15:49:45.784553+0000 mgr.x (mgr.24118) 673 : cluster [DBG] pgmap v952: 17 pgs: 17 active+clean; 5.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 841 op/s 2020-09-08T15:49:47.237 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:47.396 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:47.500 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:49:47.654 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:49:47 smithi055 bash[16646]: debug 2020-09-08T15:49:47.396+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:47.735 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:49:47.792 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:47 smithi104 bash[9203]: cluster 2020-09-08T15:49:46.535451+0000 osd.11 (osd.11) 52 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:47.792 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:47 smithi104 bash[9203]: cluster 2020-09-08T15:49:46.535528+0000 osd.11 (osd.11) 53 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:47.793 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:47 smithi104 bash[9203]: cluster 2020-09-08T15:49:46.765804+0000 mon.a (mon.0) 1426 : cluster [DBG] osdmap e364: 12 total, 12 up, 12 in 2020-09-08T15:49:47.793 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:49:47 smithi104 bash[19922]: debug 2020-09-08T15:49:47.739+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:47.839 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:49:47.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:47 smithi055 bash[9296]: cluster 2020-09-08T15:49:46.535451+0000 osd.11 (osd.11) 52 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:47.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:47 smithi055 bash[9296]: cluster 2020-09-08T15:49:46.535528+0000 osd.11 (osd.11) 53 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:47.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:47 smithi055 bash[9296]: cluster 2020-09-08T15:49:46.765804+0000 mon.a (mon.0) 1426 : cluster [DBG] osdmap e364: 12 total, 12 up, 12 in 2020-09-08T15:49:47.974 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:49:48.077 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:48.090 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:47 smithi166 bash[9446]: cluster 2020-09-08T15:49:46.535451+0000 osd.11 (osd.11) 52 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:48.090 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:47 smithi166 bash[9446]: cluster 2020-09-08T15:49:46.535528+0000 osd.11 (osd.11) 53 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:48.091 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:47 smithi166 bash[9446]: cluster 2020-09-08T15:49:46.765804+0000 mon.a (mon.0) 1426 : cluster [DBG] osdmap e364: 12 total, 12 up, 12 in 2020-09-08T15:49:48.091 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:49:47 smithi166 bash[18960]: debug 2020-09-08T15:49:47.969+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:48.189 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:48.292 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:48.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:49:48 smithi166 bash[14254]: debug 2020-09-08T15:49:48.189+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:48.494 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:48.600 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:48.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:49:48 smithi104 bash[17588]: debug 2020-09-08T15:49:48.495+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:48.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:48 smithi055 bash[9296]: cluster 2020-09-08T15:49:47.784958+0000 mgr.x (mgr.24118) 674 : cluster [DBG] pgmap v954: 17 pgs: 17 active+clean; 5.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.16k op/s 2020-09-08T15:49:49.051 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:49.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:48 smithi166 bash[9446]: cluster 2020-09-08T15:49:47.784958+0000 mgr.x (mgr.24118) 674 : cluster [DBG] pgmap v954: 17 pgs: 17 active+clean; 5.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.16k op/s 2020-09-08T15:49:49.154 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:49.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:48 smithi104 bash[9203]: cluster 2020-09-08T15:49:47.784958+0000 mgr.x (mgr.24118) 674 : cluster [DBG] pgmap v954: 17 pgs: 17 active+clean; 5.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.16k op/s 2020-09-08T15:49:49.294 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:49.398 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:49.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:49:49 smithi055 bash[11956]: debug 2020-09-08T15:49:49.052+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:49.516 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:49.517 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:49:49 smithi166 bash[16593]: debug 2020-09-08T15:49:49.289+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:49.622 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:49.756 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:49.792 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:49:49.861 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:49.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:49:49 smithi166 bash[14254]: debug 2020-09-08T15:49:49.513+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:49.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:49:49 smithi166 bash[14254]: debug 2020-09-08T15:49:49.749+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:49.969 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:49:50.432 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:49:50.433 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T15:49:50.433 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.433 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35381.0:198627 4.a 4:53970362:::benchmark_data_smithi166_32054_object56485:head [delete] snapc 0=[] ondisk+write+known_if_redirected e365)", 2020-09-08T15:49:50.433 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:49:50.428970+0000", 2020-09-08T15:49:50.434 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.0089474459999999995, 2020-09-08T15:49:50.434 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0089866410000000001, 2020-09-08T15:49:50.434 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:50.434 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:49:50.434 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:49:50.435 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35381", 2020-09-08T15:49:50.435 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:49:50.435 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 198627 2020-09-08T15:49:50.435 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:50.436 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:50.436 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.436 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:50.436 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:50.428970+0000", 2020-09-08T15:49:50.436 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:50.437 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:50.437 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.437 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:50.437 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:50.428970+0000", 2020-09-08T15:49:50.437 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999752 2020-09-08T15:49:50.438 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:50.438 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.438 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:50.438 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:50.428945+0000", 2020-09-08T15:49:50.439 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.0510000000000001e-05 2020-09-08T15:49:50.439 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:50.439 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.439 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:50.439 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:50.429016+0000", 2020-09-08T15:49:50.440 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.0701000000000001e-05 2020-09-08T15:49:50.440 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:50.440 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.440 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:50.440 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:50.429036+0000", 2020-09-08T15:49:50.440 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1428e-05 2020-09-08T15:49:50.441 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:50.441 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.441 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:50.441 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:50.429048+0000", 2020-09-08T15:49:50.441 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.9074999999999998e-05 2020-09-08T15:49:50.442 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:50.442 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.442 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:50.442 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:50.429117+0000", 2020-09-08T15:49:50.442 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00020060100000000001 2020-09-08T15:49:50.443 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:50.443 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.443 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:50.443 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:50.429317+0000", 2020-09-08T15:49:50.443 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017865200000000001 2020-09-08T15:49:50.444 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:50.444 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.444 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:49:50.444 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:50.429496+0000", 2020-09-08T15:49:50.444 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0080747360000000008 2020-09-08T15:49:50.445 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:50.445 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.445 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:49:50.445 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:50.437571+0000", 2020-09-08T15:49:50.445 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00027712199999999998 2020-09-08T15:49:50.447 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:50.447 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.448 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:49:50.448 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:50.437848+0000", 2020-09-08T15:49:50.448 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0088781050000000007 2020-09-08T15:49:50.448 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:50.449 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:50.449 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:50.449 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:50.449 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.450 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop_reply(client.35381.0:198627 4.a e365/314 ondisk, result = 0)", 2020-09-08T15:49:50.450 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:49:50.437715+0000", 2020-09-08T15:49:50.450 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.00020250800000000001, 2020-09-08T15:49:50.450 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00034271900000000001, 2020-09-08T15:49:50.450 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:49:50.450 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "started", 2020-09-08T15:49:50.453 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:49:50.453 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.453 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:49:50.453 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:50.437715+0000", 2020-09-08T15:49:50.453 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:49:50.454 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:50.454 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.454 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:49:50.454 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:50.437715+0000", 2020-09-08T15:49:50.454 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.3318999999999997e-05 2020-09-08T15:49:50.455 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:50.455 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.455 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:49:50.455 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:50.437748+0000", 2020-09-08T15:49:50.455 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.1059999999999996e-06 2020-09-08T15:49:50.456 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:50.456 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.457 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:49:50.457 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:50.437754+0000", 2020-09-08T15:49:50.457 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4149e-05 2020-09-08T15:49:50.457 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:50.457 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.458 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:49:50.458 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:50.437768+0000", 2020-09-08T15:49:50.458 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.8060000000000001e-06 2020-09-08T15:49:50.458 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:50.459 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.459 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:49:50.459 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:50.437777+0000", 2020-09-08T15:49:50.459 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.3980999999999998e-05 2020-09-08T15:49:50.459 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:50.460 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.460 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:49:50.460 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:50.437821+0000", 2020-09-08T15:49:50.460 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4304999999999999e-05 2020-09-08T15:49:50.460 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:49:50.461 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:49:50.461 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:49:50.461 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:50.437835+0000", 2020-09-08T15:49:50.461 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000120666 2020-09-08T15:49:50.461 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:50.462 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:49:50.462 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:50.462 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:49:50.462 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2020-09-08T15:49:50.462 INFO:teuthology.orchestra.run.smithi055.stdout: "num_ops": 2 2020-09-08T15:49:50.463 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:49:50.464 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:50.570 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:50.798 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:49:50 smithi104 bash[15273]: debug 2020-09-08T15:49:50.447+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:50.798 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:50 smithi104 bash[9203]: cluster 2020-09-08T15:49:49.719237+0000 osd.4 (osd.4) 37 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:50.799 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:50 smithi104 bash[9203]: cluster 2020-09-08T15:49:49.719304+0000 osd.4 (osd.4) 38 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:50.799 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:50 smithi104 bash[9203]: cluster 2020-09-08T15:49:49.785340+0000 mgr.x (mgr.24118) 675 : cluster [DBG] pgmap v955: 17 pgs: 17 active+clean; 5.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T15:49:50.799 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:50 smithi104 bash[9203]: cluster 2020-09-08T15:49:49.804116+0000 mon.a (mon.0) 1427 : cluster [DBG] osdmap e365: 12 total, 12 up, 12 in 2020-09-08T15:49:51.022 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:51.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:50 smithi166 bash[9446]: cluster 2020-09-08T15:49:49.719237+0000 osd.4 (osd.4) 37 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:51.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:50 smithi166 bash[9446]: cluster 2020-09-08T15:49:49.719304+0000 osd.4 (osd.4) 38 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:51.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:50 smithi166 bash[9446]: cluster 2020-09-08T15:49:49.785340+0000 mgr.x (mgr.24118) 675 : cluster [DBG] pgmap v955: 17 pgs: 17 active+clean; 5.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T15:49:51.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:50 smithi166 bash[9446]: cluster 2020-09-08T15:49:49.804116+0000 mon.a (mon.0) 1427 : cluster [DBG] osdmap e365: 12 total, 12 up, 12 in 2020-09-08T15:49:51.141 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:51.149 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:49:51 smithi104 bash[17588]: debug 2020-09-08T15:49:51.023+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:51.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:50 smithi055 bash[9296]: cluster 2020-09-08T15:49:49.719237+0000 osd.4 (osd.4) 37 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:51.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:50 smithi055 bash[9296]: cluster 2020-09-08T15:49:49.719304+0000 osd.4 (osd.4) 38 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:51.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:50 smithi055 bash[9296]: cluster 2020-09-08T15:49:49.785340+0000 mgr.x (mgr.24118) 675 : cluster [DBG] pgmap v955: 17 pgs: 17 active+clean; 5.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T15:49:51.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:50 smithi055 bash[9296]: cluster 2020-09-08T15:49:49.804116+0000 mon.a (mon.0) 1427 : cluster [DBG] osdmap e365: 12 total, 12 up, 12 in 2020-09-08T15:49:51.310 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2020-09-08T15:49:51.466 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:49:51.614 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:51.719 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:51.882 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:51.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:49:51 smithi104 bash[15273]: debug 2020-09-08T15:49:51.615+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:51.986 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:51.993 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:49:51 smithi166 bash[16593]: debug 2020-09-08T15:49:51.881+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:52.095 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:52.201 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:52.322 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:49:52 smithi166 bash[14254]: debug 2020-09-08T15:49:52.093+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:52.322 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:49:52.426 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:52.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:49:52 smithi166 bash[16593]: debug 2020-09-08T15:49:52.321+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:52.816 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:49:53.059 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:52 smithi104 bash[9203]: cluster 2020-09-08T15:49:51.785734+0000 mgr.x (mgr.24118) 676 : cluster [DBG] pgmap v957: 17 pgs: 17 active+clean; 5.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 884 op/s 2020-09-08T15:49:53.059 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:52 smithi104 bash[9203]: audit 2020-09-08T15:49:52.518973+0000 mon.a (mon.0) 1428 : audit [INF] from='client.? v1:172.21.15.104:0/1933055522' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:49:53.071 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:52 smithi055 bash[9296]: cluster 2020-09-08T15:49:51.785734+0000 mgr.x (mgr.24118) 676 : cluster [DBG] pgmap v957: 17 pgs: 17 active+clean; 5.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 884 op/s 2020-09-08T15:49:53.071 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:52 smithi055 bash[9296]: audit 2020-09-08T15:49:52.518973+0000 mon.a (mon.0) 1428 : audit [INF] from='client.? v1:172.21.15.104:0/1933055522' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:49:53.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:52 smithi166 bash[9446]: cluster 2020-09-08T15:49:51.785734+0000 mgr.x (mgr.24118) 676 : cluster [DBG] pgmap v957: 17 pgs: 17 active+clean; 5.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 884 op/s 2020-09-08T15:49:53.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:52 smithi166 bash[9446]: audit 2020-09-08T15:49:52.518973+0000 mon.a (mon.0) 1428 : audit [INF] from='client.? v1:172.21.15.104:0/1933055522' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:49:53.170 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:53.277 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:53.449 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:49:53 smithi104 bash[17588]: debug 2020-09-08T15:49:53.167+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:53.821 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:49:53.925 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:49:54.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:53 smithi166 bash[9446]: cluster 2020-09-08T15:49:52.807715+0000 mon.a (mon.0) 1429 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:49:54.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:53 smithi166 bash[9446]: audit 2020-09-08T15:49:52.818746+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? v1:172.21.15.104:0/1933055522' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:49:54.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:53 smithi166 bash[9446]: cluster 2020-09-08T15:49:52.818796+0000 mon.a (mon.0) 1431 : cluster [DBG] osdmap e366: 12 total, 12 up, 12 in 2020-09-08T15:49:54.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:49:53 smithi055 bash[11956]: debug 2020-09-08T15:49:53.824+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:54.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:53 smithi055 bash[9296]: cluster 2020-09-08T15:49:52.807715+0000 mon.a (mon.0) 1429 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:49:54.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:53 smithi055 bash[9296]: audit 2020-09-08T15:49:52.818746+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? v1:172.21.15.104:0/1933055522' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:49:54.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:53 smithi055 bash[9296]: cluster 2020-09-08T15:49:52.818796+0000 mon.a (mon.0) 1431 : cluster [DBG] osdmap e366: 12 total, 12 up, 12 in 2020-09-08T15:49:54.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:53 smithi104 bash[9203]: cluster 2020-09-08T15:49:52.807715+0000 mon.a (mon.0) 1429 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:49:54.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:53 smithi104 bash[9203]: audit 2020-09-08T15:49:52.818746+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? v1:172.21.15.104:0/1933055522' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:49:54.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:53 smithi104 bash[9203]: cluster 2020-09-08T15:49:52.818796+0000 mon.a (mon.0) 1431 : cluster [DBG] osdmap e366: 12 total, 12 up, 12 in 2020-09-08T15:49:54.369 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:49:54.472 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:54.658 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:49:54 smithi055 bash[19003]: debug 2020-09-08T15:49:54.368+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:54.719 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:54.826 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:54.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:54 smithi104 bash[9203]: cluster 2020-09-08T15:49:53.623935+0000 osd.3 (osd.3) 45 : cluster [DBG] 4.c scrub starts 2020-09-08T15:49:54.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:54 smithi104 bash[9203]: cluster 2020-09-08T15:49:53.786207+0000 mgr.x (mgr.24118) 677 : cluster [DBG] pgmap v959: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.31k op/s 2020-09-08T15:49:54.987 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:55.091 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:55.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:54 smithi166 bash[9446]: cluster 2020-09-08T15:49:53.623935+0000 osd.3 (osd.3) 45 : cluster [DBG] 4.c scrub starts 2020-09-08T15:49:55.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:54 smithi166 bash[9446]: cluster 2020-09-08T15:49:53.786207+0000 mgr.x (mgr.24118) 677 : cluster [DBG] pgmap v959: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.31k op/s 2020-09-08T15:49:55.117 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:49:54 smithi166 bash[14254]: debug 2020-09-08T15:49:54.984+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:55.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:49:54 smithi055 bash[16646]: debug 2020-09-08T15:49:54.712+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:55.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:54 smithi055 bash[9296]: cluster 2020-09-08T15:49:53.623935+0000 osd.3 (osd.3) 45 : cluster [DBG] 4.c scrub starts 2020-09-08T15:49:55.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:54 smithi055 bash[9296]: cluster 2020-09-08T15:49:53.786207+0000 mgr.x (mgr.24118) 677 : cluster [DBG] pgmap v959: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.31k op/s 2020-09-08T15:49:55.400 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:55.503 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:55.681 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:49:55 smithi104 bash[15273]: debug 2020-09-08T15:49:55.403+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:55.688 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:55.783 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:49:55 smithi055 bash[16646]: debug 2020-09-08T15:49:55.688+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:55.793 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:49:55.933 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:49:55.978 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:49:56.004 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:49:56.036 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:56.042 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:49:56.057 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:49:56.058 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T15:49:56.058 INFO:teuthology.orchestra.run.smithi055.stdout: "complaint_time": 30, 2020-09-08T15:49:56.058 INFO:teuthology.orchestra.run.smithi055.stdout: "num_blocked_ops": 0 2020-09-08T15:49:56.059 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:49:56.076 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:49:56.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:55 smithi166 bash[9446]: cluster 2020-09-08T15:49:54.541144+0000 osd.6 (osd.6) 53 : cluster [DBG] 4.a scrub starts 2020-09-08T15:49:56.118 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:49:56.144 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:55 smithi055 bash[9296]: cluster 2020-09-08T15:49:54.541144+0000 osd.6 (osd.6) 53 : cluster [DBG] 4.a scrub starts 2020-09-08T15:49:56.145 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:49:55 smithi055 bash[19003]: debug 2020-09-08T15:49:55.932+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:56.148 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:49:56.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:55 smithi104 bash[9203]: cluster 2020-09-08T15:49:54.541144+0000 osd.6 (osd.6) 53 : cluster [DBG] 4.a scrub starts 2020-09-08T15:49:56.181 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:49:56.185 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:56.221 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:49:56.264 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:49:56.288 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:49:56.299 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:49:56.335 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:49:56.370 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:49:56.371 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:49:56.372 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:49:56.372 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:49:56.372 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:49:56.373 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:49:56.373 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:49:56.373 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:49:56.373 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:49:56.373 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:49:56.374 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:49:56.374 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:49:56.374 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:49:56.384 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:49:56.490 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:56.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:49:56 smithi166 bash[11928]: debug 2020-09-08T15:49:56.184+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:56.617 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:49:56 smithi166 bash[18960]: debug 2020-09-08T15:49:56.380+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:56.670 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:49:56.727 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:49:56 smithi104 bash[17588]: debug 2020-09-08T15:49:56.727+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:56.728 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:49:56.831 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:56.841 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:49:56.941 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:49:57.047 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:57.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:56 smithi166 bash[9446]: cluster 2020-09-08T15:49:55.490337+0000 osd.3 (osd.3) 46 : cluster [DBG] 4.c scrub ok 2020-09-08T15:49:57.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:56 smithi166 bash[9446]: cluster 2020-09-08T15:49:55.624605+0000 osd.3 (osd.3) 47 : cluster [DBG] 4.0 scrub starts 2020-09-08T15:49:57.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:56 smithi166 bash[9446]: cluster 2020-09-08T15:49:55.786665+0000 mgr.x (mgr.24118) 678 : cluster [DBG] pgmap v960: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 916 op/s 2020-09-08T15:49:57.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:56 smithi166 bash[9446]: cluster 2020-09-08T15:49:55.882400+0000 osd.9 (osd.9) 43 : cluster [DBG] 4.d scrub starts 2020-09-08T15:49:57.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:56 smithi166 bash[9446]: audit 2020-09-08T15:49:55.948417+0000 mon.b (mon.1) 91 : audit [DBG] from='client.? v1:172.21.15.104:0/2278169633' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:49:57.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:56 smithi166 bash[9446]: cluster 2020-09-08T15:49:56.310355+0000 osd.6 (osd.6) 54 : cluster [DBG] 4.a scrub ok 2020-09-08T15:49:57.121 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:49:56 smithi166 bash[14254]: debug 2020-09-08T15:49:56.940+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:57.150 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:57.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:56 smithi055 bash[9296]: cluster 2020-09-08T15:49:55.490337+0000 osd.3 (osd.3) 46 : cluster [DBG] 4.c scrub ok 2020-09-08T15:49:57.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:56 smithi055 bash[9296]: cluster 2020-09-08T15:49:55.624605+0000 osd.3 (osd.3) 47 : cluster [DBG] 4.0 scrub starts 2020-09-08T15:49:57.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:56 smithi055 bash[9296]: cluster 2020-09-08T15:49:55.786665+0000 mgr.x (mgr.24118) 678 : cluster [DBG] pgmap v960: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 916 op/s 2020-09-08T15:49:57.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:56 smithi055 bash[9296]: cluster 2020-09-08T15:49:55.882400+0000 osd.9 (osd.9) 43 : cluster [DBG] 4.d scrub starts 2020-09-08T15:49:57.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:56 smithi055 bash[9296]: audit 2020-09-08T15:49:55.948417+0000 mon.b (mon.1) 91 : audit [DBG] from='client.? v1:172.21.15.104:0/2278169633' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:49:57.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:56 smithi055 bash[9296]: cluster 2020-09-08T15:49:56.310355+0000 osd.6 (osd.6) 54 : cluster [DBG] 4.a scrub ok 2020-09-08T15:49:57.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:56 smithi104 bash[9203]: cluster 2020-09-08T15:49:55.490337+0000 osd.3 (osd.3) 46 : cluster [DBG] 4.c scrub ok 2020-09-08T15:49:57.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:56 smithi104 bash[9203]: cluster 2020-09-08T15:49:55.624605+0000 osd.3 (osd.3) 47 : cluster [DBG] 4.0 scrub starts 2020-09-08T15:49:57.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:56 smithi104 bash[9203]: cluster 2020-09-08T15:49:55.786665+0000 mgr.x (mgr.24118) 678 : cluster [DBG] pgmap v960: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 916 op/s 2020-09-08T15:49:57.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:56 smithi104 bash[9203]: cluster 2020-09-08T15:49:55.882400+0000 osd.9 (osd.9) 43 : cluster [DBG] 4.d scrub starts 2020-09-08T15:49:57.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:56 smithi104 bash[9203]: audit 2020-09-08T15:49:55.948417+0000 mon.b (mon.1) 91 : audit [DBG] from='client.? v1:172.21.15.104:0/2278169633' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:49:57.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:56 smithi104 bash[9203]: cluster 2020-09-08T15:49:56.310355+0000 osd.6 (osd.6) 54 : cluster [DBG] 4.a scrub ok 2020-09-08T15:49:57.254 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:57.350 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2020-09-08T15:49:57.409 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:57.511 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:57.520 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:49:57 smithi166 bash[11928]: debug 2020-09-08T15:49:57.148+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:57.521 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:49:57.624 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:57.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:49:57 smithi055 bash[16646]: debug 2020-09-08T15:49:57.404+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:57.731 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:57.829 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:49:57 smithi166 bash[11928]: debug 2020-09-08T15:49:57.620+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:57.829 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:57 smithi166 bash[9446]: cluster 2020-09-08T15:49:56.561636+0000 osd.6 (osd.6) 55 : cluster [DBG] 4.5 scrub starts 2020-09-08T15:49:57.830 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:57 smithi166 bash[9446]: cluster 2020-09-08T15:49:56.628541+0000 osd.1 (osd.1) 70 : cluster [DBG] 4.4 scrub starts 2020-09-08T15:49:57.830 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:57 smithi166 bash[9446]: cluster 2020-09-08T15:49:57.348168+0000 osd.9 (osd.9) 44 : cluster [DBG] 4.d scrub ok 2020-09-08T15:49:58.027 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:49:58 smithi104 bash[15273]: debug 2020-09-08T15:49:58.027+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:58.027 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:57 smithi104 bash[9203]: cluster 2020-09-08T15:49:56.561636+0000 osd.6 (osd.6) 55 : cluster [DBG] 4.5 scrub starts 2020-09-08T15:49:58.027 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:57 smithi104 bash[9203]: cluster 2020-09-08T15:49:56.628541+0000 osd.1 (osd.1) 70 : cluster [DBG] 4.4 scrub starts 2020-09-08T15:49:58.027 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:57 smithi104 bash[9203]: cluster 2020-09-08T15:49:57.348168+0000 osd.9 (osd.9) 44 : cluster [DBG] 4.d scrub ok 2020-09-08T15:49:58.028 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:49:58.131 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:49:58.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:57 smithi055 bash[9296]: cluster 2020-09-08T15:49:56.561636+0000 osd.6 (osd.6) 55 : cluster [DBG] 4.5 scrub starts 2020-09-08T15:49:58.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:57 smithi055 bash[9296]: cluster 2020-09-08T15:49:56.628541+0000 osd.1 (osd.1) 70 : cluster [DBG] 4.4 scrub starts 2020-09-08T15:49:58.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:57 smithi055 bash[9296]: cluster 2020-09-08T15:49:57.348168+0000 osd.9 (osd.9) 44 : cluster [DBG] 4.d scrub ok 2020-09-08T15:49:58.907 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:49:59.010 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:59.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:58 smithi166 bash[9446]: cluster 2020-09-08T15:49:57.550354+0000 osd.3 (osd.3) 48 : cluster [DBG] 4.0 scrub ok 2020-09-08T15:49:59.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:58 smithi166 bash[9446]: cluster 2020-09-08T15:49:57.787104+0000 mgr.x (mgr.24118) 679 : cluster [DBG] pgmap v961: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 4.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 971 op/s 2020-09-08T15:49:59.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:58 smithi166 bash[9446]: cluster 2020-09-08T15:49:57.950051+0000 osd.9 (osd.9) 45 : cluster [DBG] 4.2 scrub starts 2020-09-08T15:49:59.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:58 smithi166 bash[9446]: cluster 2020-09-08T15:49:57.993343+0000 osd.6 (osd.6) 56 : cluster [DBG] 4.5 scrub ok 2020-09-08T15:49:59.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:58 smithi055 bash[9296]: cluster 2020-09-08T15:49:57.550354+0000 osd.3 (osd.3) 48 : cluster [DBG] 4.0 scrub ok 2020-09-08T15:49:59.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:58 smithi055 bash[9296]: cluster 2020-09-08T15:49:57.787104+0000 mgr.x (mgr.24118) 679 : cluster [DBG] pgmap v961: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 4.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 971 op/s 2020-09-08T15:49:59.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:58 smithi055 bash[9296]: cluster 2020-09-08T15:49:57.950051+0000 osd.9 (osd.9) 45 : cluster [DBG] 4.2 scrub starts 2020-09-08T15:49:59.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:58 smithi055 bash[9296]: cluster 2020-09-08T15:49:57.993343+0000 osd.6 (osd.6) 56 : cluster [DBG] 4.5 scrub ok 2020-09-08T15:49:59.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:49:58 smithi104 bash[19922]: debug 2020-09-08T15:49:58.907+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:49:59.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:58 smithi104 bash[9203]: cluster 2020-09-08T15:49:57.550354+0000 osd.3 (osd.3) 48 : cluster [DBG] 4.0 scrub ok 2020-09-08T15:49:59.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:58 smithi104 bash[9203]: cluster 2020-09-08T15:49:57.787104+0000 mgr.x (mgr.24118) 679 : cluster [DBG] pgmap v961: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 4.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 971 op/s 2020-09-08T15:49:59.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:58 smithi104 bash[9203]: cluster 2020-09-08T15:49:57.950051+0000 osd.9 (osd.9) 45 : cluster [DBG] 4.2 scrub starts 2020-09-08T15:49:59.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:58 smithi104 bash[9203]: cluster 2020-09-08T15:49:57.993343+0000 osd.6 (osd.6) 56 : cluster [DBG] 4.5 scrub ok 2020-09-08T15:49:59.835 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:49:59.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:59 smithi104 bash[9203]: cluster 2020-09-08T15:49:58.273700+0000 osd.1 (osd.1) 71 : cluster [DBG] 4.4 scrub ok 2020-09-08T15:49:59.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:59 smithi104 bash[9203]: cluster 2020-09-08T15:49:58.569707+0000 osd.1 (osd.1) 72 : cluster [DBG] 4.7 scrub starts 2020-09-08T15:49:59.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:59 smithi104 bash[9203]: cluster 2020-09-08T15:49:58.988709+0000 osd.9 (osd.9) 46 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:59.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:59 smithi104 bash[9203]: cluster 2020-09-08T15:49:58.988775+0000 osd.9 (osd.9) 47 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:59.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:49:59 smithi104 bash[9203]: cluster 2020-09-08T15:49:59.280967+0000 osd.9 (osd.9) 48 : cluster [DBG] 4.2 scrub ok 2020-09-08T15:49:59.938 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:49:59.947 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:59 smithi055 bash[9296]: cluster 2020-09-08T15:49:58.273700+0000 osd.1 (osd.1) 71 : cluster [DBG] 4.4 scrub ok 2020-09-08T15:49:59.947 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:59 smithi055 bash[9296]: cluster 2020-09-08T15:49:58.569707+0000 osd.1 (osd.1) 72 : cluster [DBG] 4.7 scrub starts 2020-09-08T15:49:59.947 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:59 smithi055 bash[9296]: cluster 2020-09-08T15:49:58.988709+0000 osd.9 (osd.9) 46 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:49:59.948 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:59 smithi055 bash[9296]: cluster 2020-09-08T15:49:58.988775+0000 osd.9 (osd.9) 47 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:49:59.948 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:49:59 smithi055 bash[9296]: cluster 2020-09-08T15:49:59.280967+0000 osd.9 (osd.9) 48 : cluster [DBG] 4.2 scrub ok 2020-09-08T15:50:00.107 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:50:00.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:49:59 smithi166 bash[11928]: debug 2020-09-08T15:49:59.832+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:00.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:59 smithi166 bash[9446]: cluster 2020-09-08T15:49:58.273700+0000 osd.1 (osd.1) 71 : cluster [DBG] 4.4 scrub ok 2020-09-08T15:50:00.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:59 smithi166 bash[9446]: cluster 2020-09-08T15:49:58.569707+0000 osd.1 (osd.1) 72 : cluster [DBG] 4.7 scrub starts 2020-09-08T15:50:00.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:59 smithi166 bash[9446]: cluster 2020-09-08T15:49:58.988709+0000 osd.9 (osd.9) 46 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:00.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:59 smithi166 bash[9446]: cluster 2020-09-08T15:49:58.988775+0000 osd.9 (osd.9) 47 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:00.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:49:59 smithi166 bash[9446]: cluster 2020-09-08T15:49:59.280967+0000 osd.9 (osd.9) 48 : cluster [DBG] 4.2 scrub ok 2020-09-08T15:50:00.259 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:50:00.512 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:50:00.619 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:00.744 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:00.841 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:00 smithi055 bash[9296]: cluster 2020-09-08T15:49:59.787497+0000 mgr.x (mgr.24118) 680 : cluster [DBG] pgmap v962: 17 pgs: 4 active+clean+scrubbing, 13 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 994 op/s 2020-09-08T15:50:00.841 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:00 smithi055 bash[9296]: cluster 2020-09-08T15:49:59.838002+0000 mon.a (mon.0) 1432 : cluster [DBG] osdmap e367: 12 total, 12 up, 12 in 2020-09-08T15:50:00.842 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:50:00 smithi055 bash[16646]: debug 2020-09-08T15:50:00.512+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:00.860 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:01.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:00 smithi166 bash[9446]: cluster 2020-09-08T15:49:59.787497+0000 mgr.x (mgr.24118) 680 : cluster [DBG] pgmap v962: 17 pgs: 4 active+clean+scrubbing, 13 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 994 op/s 2020-09-08T15:50:01.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:00 smithi166 bash[9446]: cluster 2020-09-08T15:49:59.838002+0000 mon.a (mon.0) 1432 : cluster [DBG] osdmap e367: 12 total, 12 up, 12 in 2020-09-08T15:50:01.118 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:50:00 smithi166 bash[18960]: debug 2020-09-08T15:50:00.736+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:01.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:00 smithi104 bash[9203]: cluster 2020-09-08T15:49:59.787497+0000 mgr.x (mgr.24118) 680 : cluster [DBG] pgmap v962: 17 pgs: 4 active+clean+scrubbing, 13 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 994 op/s 2020-09-08T15:50:01.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:00 smithi104 bash[9203]: cluster 2020-09-08T15:49:59.838002+0000 mon.a (mon.0) 1432 : cluster [DBG] osdmap e367: 12 total, 12 up, 12 in 2020-09-08T15:50:01.482 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:01.586 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:01.866 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:50:01 smithi055 bash[11956]: debug 2020-09-08T15:50:01.868+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:01.866 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:01.929 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:50:01 smithi104 bash[15273]: debug 2020-09-08T15:50:01.483+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:01.969 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:02.422 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:50:02.422 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:50:02.648 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:02.752 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:50:02.853 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:02 smithi055 bash[9296]: cluster 2020-09-08T15:50:01.273398+0000 osd.1 (osd.1) 73 : cluster [DBG] 4.7 scrub ok 2020-09-08T15:50:02.853 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:02 smithi055 bash[9296]: cluster 2020-09-08T15:50:01.296950+0000 osd.11 (osd.11) 54 : cluster [DBG] 4.3 scrub starts 2020-09-08T15:50:02.853 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:02 smithi055 bash[9296]: cluster 2020-09-08T15:50:01.560148+0000 osd.1 (osd.1) 74 : cluster [DBG] 4.f scrub starts 2020-09-08T15:50:02.853 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:02 smithi055 bash[9296]: cluster 2020-09-08T15:50:01.649317+0000 osd.2 (osd.2) 48 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:02.854 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:02 smithi055 bash[9296]: cluster 2020-09-08T15:50:01.649391+0000 osd.2 (osd.2) 49 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:02.854 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:02 smithi055 bash[9296]: cluster 2020-09-08T15:50:01.788138+0000 mgr.x (mgr.24118) 681 : cluster [DBG] pgmap v964: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 898 op/s 2020-09-08T15:50:02.854 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:02 smithi055 bash[9296]: cluster 2020-09-08T15:50:01.854535+0000 mon.a (mon.0) 1433 : cluster [DBG] osdmap e368: 12 total, 12 up, 12 in 2020-09-08T15:50:02.909 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:50:02.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:02 smithi104 bash[9203]: cluster 2020-09-08T15:50:01.273398+0000 osd.1 (osd.1) 73 : cluster [DBG] 4.7 scrub ok 2020-09-08T15:50:02.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:02 smithi104 bash[9203]: cluster 2020-09-08T15:50:01.296950+0000 osd.11 (osd.11) 54 : cluster [DBG] 4.3 scrub starts 2020-09-08T15:50:02.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:02 smithi104 bash[9203]: cluster 2020-09-08T15:50:01.560148+0000 osd.1 (osd.1) 74 : cluster [DBG] 4.f scrub starts 2020-09-08T15:50:02.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:02 smithi104 bash[9203]: cluster 2020-09-08T15:50:01.649317+0000 osd.2 (osd.2) 48 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:02.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:02 smithi104 bash[9203]: cluster 2020-09-08T15:50:01.649391+0000 osd.2 (osd.2) 49 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:02.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:02 smithi104 bash[9203]: cluster 2020-09-08T15:50:01.788138+0000 mgr.x (mgr.24118) 681 : cluster [DBG] pgmap v964: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 898 op/s 2020-09-08T15:50:02.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:02 smithi104 bash[9203]: cluster 2020-09-08T15:50:01.854535+0000 mon.a (mon.0) 1433 : cluster [DBG] osdmap e368: 12 total, 12 up, 12 in 2020-09-08T15:50:02.928 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:50:02 smithi104 bash[15273]: debug 2020-09-08T15:50:02.651+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:03.011 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:50:03.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:50:02 smithi166 bash[11928]: debug 2020-09-08T15:50:02.904+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:03.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:02 smithi166 bash[9446]: cluster 2020-09-08T15:50:01.273398+0000 osd.1 (osd.1) 73 : cluster [DBG] 4.7 scrub ok 2020-09-08T15:50:03.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:02 smithi166 bash[9446]: cluster 2020-09-08T15:50:01.296950+0000 osd.11 (osd.11) 54 : cluster [DBG] 4.3 scrub starts 2020-09-08T15:50:03.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:02 smithi166 bash[9446]: cluster 2020-09-08T15:50:01.560148+0000 osd.1 (osd.1) 74 : cluster [DBG] 4.f scrub starts 2020-09-08T15:50:03.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:02 smithi166 bash[9446]: cluster 2020-09-08T15:50:01.649317+0000 osd.2 (osd.2) 48 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:03.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:02 smithi166 bash[9446]: cluster 2020-09-08T15:50:01.649391+0000 osd.2 (osd.2) 49 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:03.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:02 smithi166 bash[9446]: cluster 2020-09-08T15:50:01.788138+0000 mgr.x (mgr.24118) 681 : cluster [DBG] pgmap v964: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 898 op/s 2020-09-08T15:50:03.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:02 smithi166 bash[9446]: cluster 2020-09-08T15:50:01.854535+0000 mon.a (mon.0) 1433 : cluster [DBG] osdmap e368: 12 total, 12 up, 12 in 2020-09-08T15:50:03.351 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:50:03.454 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:03.489 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2020-09-08T15:50:03.562 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:03.660 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:50:03 smithi055 bash[16646]: debug 2020-09-08T15:50:03.352+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:03.661 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:50:03 smithi055 bash[11956]: debug 2020-09-08T15:50:03.560+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:03.670 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:50:03.696 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:50:03.778 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:50:03.865 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:50:03.882 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:50:04.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:50:03 smithi166 bash[16593]: debug 2020-09-08T15:50:03.776+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:04.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:03 smithi166 bash[9446]: cluster 2020-09-08T15:50:02.527457+0000 osd.11 (osd.11) 55 : cluster [DBG] 4.3 scrub ok 2020-09-08T15:50:04.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:03 smithi166 bash[9446]: cluster 2020-09-08T15:50:03.240977+0000 osd.11 (osd.11) 56 : cluster [DBG] 4.1 scrub starts 2020-09-08T15:50:04.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:03 smithi166 bash[9446]: cluster 2020-09-08T15:50:03.276558+0000 osd.10 (osd.10) 35 : cluster [DBG] 4.e scrub starts 2020-09-08T15:50:04.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:03 smithi166 bash[9446]: audit 2020-09-08T15:50:03.743393+0000 mon.a (mon.0) 1434 : audit [INF] from='client.? v1:172.21.15.104:0/525945876' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:50:04.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:03 smithi055 bash[9296]: cluster 2020-09-08T15:50:02.527457+0000 osd.11 (osd.11) 55 : cluster [DBG] 4.3 scrub ok 2020-09-08T15:50:04.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:03 smithi055 bash[9296]: cluster 2020-09-08T15:50:03.240977+0000 osd.11 (osd.11) 56 : cluster [DBG] 4.1 scrub starts 2020-09-08T15:50:04.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:03 smithi055 bash[9296]: cluster 2020-09-08T15:50:03.276558+0000 osd.10 (osd.10) 35 : cluster [DBG] 4.e scrub starts 2020-09-08T15:50:04.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:03 smithi055 bash[9296]: audit 2020-09-08T15:50:03.743393+0000 mon.a (mon.0) 1434 : audit [INF] from='client.? v1:172.21.15.104:0/525945876' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:50:04.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:03 smithi104 bash[9203]: cluster 2020-09-08T15:50:02.527457+0000 osd.11 (osd.11) 55 : cluster [DBG] 4.3 scrub ok 2020-09-08T15:50:04.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:03 smithi104 bash[9203]: cluster 2020-09-08T15:50:03.240977+0000 osd.11 (osd.11) 56 : cluster [DBG] 4.1 scrub starts 2020-09-08T15:50:04.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:03 smithi104 bash[9203]: cluster 2020-09-08T15:50:03.276558+0000 osd.10 (osd.10) 35 : cluster [DBG] 4.e scrub starts 2020-09-08T15:50:04.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:03 smithi104 bash[9203]: audit 2020-09-08T15:50:03.743393+0000 mon.a (mon.0) 1434 : audit [INF] from='client.? v1:172.21.15.104:0/525945876' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:50:04.184 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:50:04.288 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:04.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:50:04 smithi166 bash[11928]: debug 2020-09-08T15:50:04.180+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:04.993 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:05.097 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:05.110 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:50:04 smithi104 bash[17588]: debug 2020-09-08T15:50:04.995+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:05.111 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:04 smithi104 bash[9203]: cluster 2020-09-08T15:50:03.788705+0000 mgr.x (mgr.24118) 682 : cluster [DBG] pgmap v966: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.23k op/s 2020-09-08T15:50:05.111 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:04 smithi104 bash[9203]: cluster 2020-09-08T15:50:03.857511+0000 mon.a (mon.0) 1435 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:50:05.111 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:04 smithi104 bash[9203]: cluster 2020-09-08T15:50:03.857547+0000 mon.a (mon.0) 1436 : cluster [INF] Cluster is now healthy 2020-09-08T15:50:05.111 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:04 smithi104 bash[9203]: audit 2020-09-08T15:50:03.868300+0000 mon.a (mon.0) 1437 : audit [INF] from='client.? v1:172.21.15.104:0/525945876' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:50:05.111 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:04 smithi104 bash[9203]: cluster 2020-09-08T15:50:03.868349+0000 mon.a (mon.0) 1438 : cluster [DBG] osdmap e369: 12 total, 12 up, 12 in 2020-09-08T15:50:05.112 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:04 smithi104 bash[9203]: cluster 2020-09-08T15:50:03.870710+0000 osd.1 (osd.1) 75 : cluster [DBG] 4.f scrub ok 2020-09-08T15:50:05.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:04 smithi166 bash[9446]: cluster 2020-09-08T15:50:03.788705+0000 mgr.x (mgr.24118) 682 : cluster [DBG] pgmap v966: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.23k op/s 2020-09-08T15:50:05.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:04 smithi166 bash[9446]: cluster 2020-09-08T15:50:03.857511+0000 mon.a (mon.0) 1435 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:50:05.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:04 smithi166 bash[9446]: cluster 2020-09-08T15:50:03.857547+0000 mon.a (mon.0) 1436 : cluster [INF] Cluster is now healthy 2020-09-08T15:50:05.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:04 smithi166 bash[9446]: audit 2020-09-08T15:50:03.868300+0000 mon.a (mon.0) 1437 : audit [INF] from='client.? v1:172.21.15.104:0/525945876' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:50:05.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:04 smithi166 bash[9446]: cluster 2020-09-08T15:50:03.868349+0000 mon.a (mon.0) 1438 : cluster [DBG] osdmap e369: 12 total, 12 up, 12 in 2020-09-08T15:50:05.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:04 smithi166 bash[9446]: cluster 2020-09-08T15:50:03.870710+0000 osd.1 (osd.1) 75 : cluster [DBG] 4.f scrub ok 2020-09-08T15:50:05.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:04 smithi055 bash[9296]: cluster 2020-09-08T15:50:03.788705+0000 mgr.x (mgr.24118) 682 : cluster [DBG] pgmap v966: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.23k op/s 2020-09-08T15:50:05.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:04 smithi055 bash[9296]: cluster 2020-09-08T15:50:03.857511+0000 mon.a (mon.0) 1435 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:50:05.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:04 smithi055 bash[9296]: cluster 2020-09-08T15:50:03.857547+0000 mon.a (mon.0) 1436 : cluster [INF] Cluster is now healthy 2020-09-08T15:50:05.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:04 smithi055 bash[9296]: audit 2020-09-08T15:50:03.868300+0000 mon.a (mon.0) 1437 : audit [INF] from='client.? v1:172.21.15.104:0/525945876' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:50:05.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:04 smithi055 bash[9296]: cluster 2020-09-08T15:50:03.868349+0000 mon.a (mon.0) 1438 : cluster [DBG] osdmap e369: 12 total, 12 up, 12 in 2020-09-08T15:50:05.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:04 smithi055 bash[9296]: cluster 2020-09-08T15:50:03.870710+0000 osd.1 (osd.1) 75 : cluster [DBG] 4.f scrub ok 2020-09-08T15:50:05.487 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:05.591 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:50:05.716 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:50:05.819 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:50:05.867 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:50:05 smithi104 bash[15273]: debug 2020-09-08T15:50:05.487+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:06.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:05 smithi166 bash[9446]: cluster 2020-09-08T15:50:04.594634+0000 osd.10 (osd.10) 36 : cluster [DBG] 4.e scrub ok 2020-09-08T15:50:06.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:05 smithi166 bash[9446]: cluster 2020-09-08T15:50:04.672227+0000 osd.11 (osd.11) 57 : cluster [DBG] 4.1 scrub ok 2020-09-08T15:50:06.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:05 smithi055 bash[9296]: cluster 2020-09-08T15:50:04.594634+0000 osd.10 (osd.10) 36 : cluster [DBG] 4.e scrub ok 2020-09-08T15:50:06.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:05 smithi055 bash[9296]: cluster 2020-09-08T15:50:04.672227+0000 osd.11 (osd.11) 57 : cluster [DBG] 4.1 scrub ok 2020-09-08T15:50:06.158 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:50:05 smithi055 bash[14301]: debug 2020-09-08T15:50:05.715+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:06.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:05 smithi104 bash[9203]: cluster 2020-09-08T15:50:04.594634+0000 osd.10 (osd.10) 36 : cluster [DBG] 4.e scrub ok 2020-09-08T15:50:06.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:05 smithi104 bash[9203]: cluster 2020-09-08T15:50:04.672227+0000 osd.11 (osd.11) 57 : cluster [DBG] 4.1 scrub ok 2020-09-08T15:50:06.752 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:50:06.782 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:50:06.822 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:50:06.849 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:50:06.869 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:06 smithi104 bash[9203]: cluster 2020-09-08T15:50:05.789092+0000 mgr.x (mgr.24118) 683 : cluster [DBG] pgmap v968: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 820 op/s 2020-09-08T15:50:06.869 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:06 smithi104 bash[9203]: cluster 2020-09-08T15:50:05.924701+0000 osd.9 (osd.9) 49 : cluster [DBG] 4.6 scrub starts 2020-09-08T15:50:06.869 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:06 smithi104 bash[9203]: audit 2020-09-08T15:50:06.715900+0000 mon.b (mon.1) 92 : audit [DBG] from='client.? v1:172.21.15.104:0/3410798620' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:50:06.890 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:50:06.930 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:50:06.964 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:50:07.002 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:50:07.033 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:50:07.076 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:50:07.112 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:50:07.147 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:50:07.152 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:50:07.152 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:50:07.152 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:50:07.152 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:50:07.153 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:50:07.153 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:50:07.153 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:50:07.153 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:50:07.154 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:50:07.154 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:50:07.154 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:50:07.154 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:50:07.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:06 smithi055 bash[9296]: cluster 2020-09-08T15:50:05.789092+0000 mgr.x (mgr.24118) 683 : cluster [DBG] pgmap v968: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 820 op/s 2020-09-08T15:50:07.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:06 smithi055 bash[9296]: cluster 2020-09-08T15:50:05.924701+0000 osd.9 (osd.9) 49 : cluster [DBG] 4.6 scrub starts 2020-09-08T15:50:07.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:06 smithi055 bash[9296]: audit 2020-09-08T15:50:06.715900+0000 mon.b (mon.1) 92 : audit [DBG] from='client.? v1:172.21.15.104:0/3410798620' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:50:07.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:06 smithi166 bash[9446]: cluster 2020-09-08T15:50:05.789092+0000 mgr.x (mgr.24118) 683 : cluster [DBG] pgmap v968: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 820 op/s 2020-09-08T15:50:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:06 smithi166 bash[9446]: cluster 2020-09-08T15:50:05.924701+0000 osd.9 (osd.9) 49 : cluster [DBG] 4.6 scrub starts 2020-09-08T15:50:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:06 smithi166 bash[9446]: audit 2020-09-08T15:50:06.715900+0000 mon.b (mon.1) 92 : audit [DBG] from='client.? v1:172.21.15.104:0/3410798620' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:50:07.562 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:50:07.780 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:50:08.123 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:50:08.123 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T15:50:08.123 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:50:08.123 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35381.0:215829 4.8 e369/314 4:145efad9:::benchmark_data_smithi166_32054_object73687:head v 369'15630, mlcod=369'15628)", 2020-09-08T15:50:08.124 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:50:08.127844+0000", 2020-09-08T15:50:08.124 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.000872403, 2020-09-08T15:50:08.124 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00089457700000000002, 2020-09-08T15:50:08.124 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:50:08.124 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "started", 2020-09-08T15:50:08.125 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:50:08.125 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:50:08.125 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:50:08.125 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.127844+0000", 2020-09-08T15:50:08.125 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:50:08.126 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:50:08.126 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:50:08.126 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:50:08.126 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.127844+0000", 2020-09-08T15:50:08.126 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.8845000000000007e-05 2020-09-08T15:50:08.127 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:50:08.127 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:50:08.127 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:50:08.127 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.127942+0000", 2020-09-08T15:50:08.127 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1956e-05 2020-09-08T15:50:08.128 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:50:08.128 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:50:08.128 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:50:08.128 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.127954+0000", 2020-09-08T15:50:08.128 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.0656999999999999e-05 2020-09-08T15:50:08.129 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:50:08.129 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:50:08.129 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:50:08.129 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.127975+0000", 2020-09-08T15:50:08.129 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.6320000000000008e-06 2020-09-08T15:50:08.130 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:50:08.130 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:50:08.130 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:50:08.130 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.127983+0000", 2020-09-08T15:50:08.130 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4379000000000002e-05 2020-09-08T15:50:08.131 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:50:08.131 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:50:08.131 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:50:08.131 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.128037+0000", 2020-09-08T15:50:08.131 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.8708000000000001e-05 2020-09-08T15:50:08.131 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:50:08.132 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:50:08.132 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:50:08.133 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.128056+0000", 2020-09-08T15:50:08.133 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000212177 2020-09-08T15:50:08.133 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:50:08.133 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:50:08.134 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:50:08.134 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:50:08.134 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2020-09-08T15:50:08.134 INFO:teuthology.orchestra.run.smithi055.stdout: "num_ops": 1 2020-09-08T15:50:08.134 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:50:08.999 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2020-09-08T15:50:09.172 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:08 smithi055 bash[9296]: cluster 2020-09-08T15:50:07.643850+0000 osd.9 (osd.9) 50 : cluster [DBG] 4.6 scrub ok 2020-09-08T15:50:09.173 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:08 smithi055 bash[9296]: cluster 2020-09-08T15:50:07.789536+0000 mgr.x (mgr.24118) 684 : cluster [DBG] pgmap v969: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 954 op/s 2020-09-08T15:50:09.173 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:50:09.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:08 smithi104 bash[9203]: cluster 2020-09-08T15:50:07.643850+0000 osd.9 (osd.9) 50 : cluster [DBG] 4.6 scrub ok 2020-09-08T15:50:09.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:08 smithi104 bash[9203]: cluster 2020-09-08T15:50:07.789536+0000 mgr.x (mgr.24118) 684 : cluster [DBG] pgmap v969: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 954 op/s 2020-09-08T15:50:09.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:08 smithi166 bash[9446]: cluster 2020-09-08T15:50:07.643850+0000 osd.9 (osd.9) 50 : cluster [DBG] 4.6 scrub ok 2020-09-08T15:50:09.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:08 smithi166 bash[9446]: cluster 2020-09-08T15:50:07.789536+0000 mgr.x (mgr.24118) 684 : cluster [DBG] pgmap v969: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 954 op/s 2020-09-08T15:50:09.937 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:50:10.040 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:10.228 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:50:09 smithi166 bash[14254]: debug 2020-09-08T15:50:09.936+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:10.229 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:50:10 smithi166 bash[18960]: debug 2020-09-08T15:50:10.228+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:10.229 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:10.332 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:10.441 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:10.542 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:10.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:50:10 smithi166 bash[18960]: debug 2020-09-08T15:50:10.436+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:10.666 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:10.771 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:50:10.798 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:50:10 smithi055 bash[11956]: debug 2020-09-08T15:50:10.659+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:10.800 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:50:10.904 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:50:10.973 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:50:11.010 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:11.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:10 smithi055 bash[9296]: cluster 2020-09-08T15:50:09.486171+0000 osd.7 (osd.7) 69 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:11.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:10 smithi055 bash[9296]: cluster 2020-09-08T15:50:09.486250+0000 osd.7 (osd.7) 70 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:11.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:10 smithi055 bash[9296]: cluster 2020-09-08T15:50:09.789907+0000 mgr.x (mgr.24118) 685 : cluster [DBG] pgmap v970: 17 pgs: 17 active+clean; 4.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T15:50:11.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:10 smithi055 bash[9296]: cluster 2020-09-08T15:50:09.884349+0000 mon.a (mon.0) 1439 : cluster [DBG] osdmap e370: 12 total, 12 up, 12 in 2020-09-08T15:50:11.178 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:50:10 smithi104 bash[22305]: debug 2020-09-08T15:50:10.903+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:11.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:10 smithi104 bash[9203]: cluster 2020-09-08T15:50:09.486171+0000 osd.7 (osd.7) 69 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:11.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:10 smithi104 bash[9203]: cluster 2020-09-08T15:50:09.486250+0000 osd.7 (osd.7) 70 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:11.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:10 smithi104 bash[9203]: cluster 2020-09-08T15:50:09.789907+0000 mgr.x (mgr.24118) 685 : cluster [DBG] pgmap v970: 17 pgs: 17 active+clean; 4.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T15:50:11.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:10 smithi104 bash[9203]: cluster 2020-09-08T15:50:09.884349+0000 mon.a (mon.0) 1439 : cluster [DBG] osdmap e370: 12 total, 12 up, 12 in 2020-09-08T15:50:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:10 smithi166 bash[9446]: cluster 2020-09-08T15:50:09.486171+0000 osd.7 (osd.7) 69 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:10 smithi166 bash[9446]: cluster 2020-09-08T15:50:09.486250+0000 osd.7 (osd.7) 70 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:10 smithi166 bash[9446]: cluster 2020-09-08T15:50:09.789907+0000 mgr.x (mgr.24118) 685 : cluster [DBG] pgmap v970: 17 pgs: 17 active+clean; 4.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T15:50:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:10 smithi166 bash[9446]: cluster 2020-09-08T15:50:09.884349+0000 mon.a (mon.0) 1439 : cluster [DBG] osdmap e370: 12 total, 12 up, 12 in 2020-09-08T15:50:11.592 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:11.695 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:11.908 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:50:11 smithi055 bash[11956]: debug 2020-09-08T15:50:11.591+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:12.123 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:12.227 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:12.365 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:12.472 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:50:12 smithi055 bash[11956]: debug 2020-09-08T15:50:12.123+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:12.474 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:12.581 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:50:12 smithi166 bash[18960]: debug 2020-09-08T15:50:12.360+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:12.581 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:50:12 smithi166 bash[18960]: debug 2020-09-08T15:50:12.580+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:12.584 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:12.689 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:12.801 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:12.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:50:12 smithi166 bash[18960]: debug 2020-09-08T15:50:12.796+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:12.905 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:50:13.019 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:50:13.122 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:13.131 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:12 smithi104 bash[9203]: cluster 2020-09-08T15:50:11.200820+0000 osd.0 (osd.0) 45 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:13.131 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:12 smithi104 bash[9203]: cluster 2020-09-08T15:50:11.200898+0000 osd.0 (osd.0) 46 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:13.131 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:12 smithi104 bash[9203]: cluster 2020-09-08T15:50:11.790287+0000 mgr.x (mgr.24118) 686 : cluster [DBG] pgmap v972: 17 pgs: 17 active+clean; 4.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 710 op/s 2020-09-08T15:50:13.131 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:12 smithi104 bash[9203]: cluster 2020-09-08T15:50:11.895943+0000 mon.a (mon.0) 1440 : cluster [DBG] osdmap e371: 12 total, 12 up, 12 in 2020-09-08T15:50:13.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:12 smithi055 bash[9296]: cluster 2020-09-08T15:50:11.200820+0000 osd.0 (osd.0) 45 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:13.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:12 smithi055 bash[9296]: cluster 2020-09-08T15:50:11.200898+0000 osd.0 (osd.0) 46 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:13.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:12 smithi055 bash[9296]: cluster 2020-09-08T15:50:11.790287+0000 mgr.x (mgr.24118) 686 : cluster [DBG] pgmap v972: 17 pgs: 17 active+clean; 4.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 710 op/s 2020-09-08T15:50:13.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:12 smithi055 bash[9296]: cluster 2020-09-08T15:50:11.895943+0000 mon.a (mon.0) 1440 : cluster [DBG] osdmap e371: 12 total, 12 up, 12 in 2020-09-08T15:50:13.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:12 smithi166 bash[9446]: cluster 2020-09-08T15:50:11.200820+0000 osd.0 (osd.0) 45 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:13.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:12 smithi166 bash[9446]: cluster 2020-09-08T15:50:11.200898+0000 osd.0 (osd.0) 46 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:13.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:12 smithi166 bash[9446]: cluster 2020-09-08T15:50:11.790287+0000 mgr.x (mgr.24118) 686 : cluster [DBG] pgmap v972: 17 pgs: 17 active+clean; 4.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 710 op/s 2020-09-08T15:50:13.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:12 smithi166 bash[9446]: cluster 2020-09-08T15:50:11.895943+0000 mon.a (mon.0) 1440 : cluster [DBG] osdmap e371: 12 total, 12 up, 12 in 2020-09-08T15:50:13.368 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:50:13 smithi166 bash[16593]: debug 2020-09-08T15:50:13.016+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:13.498 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:50:13.499 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T15:50:13.499 INFO:teuthology.orchestra.run.smithi055.stdout: "complaint_time": 30, 2020-09-08T15:50:13.499 INFO:teuthology.orchestra.run.smithi055.stdout: "num_blocked_ops": 0 2020-09-08T15:50:13.499 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:50:13.771 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:13.875 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:50:13.906 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:50:14.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:13 smithi055 bash[9296]: audit 2020-09-08T15:50:13.043444+0000 mon.a (mon.0) 1441 : audit [INF] from='client.? v1:172.21.15.104:0/2458527193' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:50:14.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:13 smithi104 bash[9203]: audit 2020-09-08T15:50:13.043444+0000 mon.a (mon.0) 1441 : audit [INF] from='client.? v1:172.21.15.104:0/2458527193' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:50:14.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:50:13 smithi104 bash[17588]: debug 2020-09-08T15:50:13.771+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:14.351 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:50:14.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:13 smithi166 bash[9446]: audit 2020-09-08T15:50:13.043444+0000 mon.a (mon.0) 1441 : audit [INF] from='client.? v1:172.21.15.104:0/2458527193' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:50:14.454 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:14.471 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2020-09-08T15:50:14.630 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:50:14.658 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:50:14 smithi055 bash[14301]: debug 2020-09-08T15:50:14.347+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:14.771 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:50:14 smithi104 bash[17588]: debug 2020-09-08T15:50:14.775+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:14.772 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:14.876 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:50:14.999 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:50:15.102 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:50:15.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:14 smithi055 bash[9296]: cluster 2020-09-08T15:50:13.790694+0000 mgr.x (mgr.24118) 687 : cluster [DBG] pgmap v974: 17 pgs: 17 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.21k op/s 2020-09-08T15:50:15.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:14 smithi055 bash[9296]: cluster 2020-09-08T15:50:13.898411+0000 mon.a (mon.0) 1442 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:50:15.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:14 smithi055 bash[9296]: audit 2020-09-08T15:50:13.908707+0000 mon.a (mon.0) 1443 : audit [INF] from='client.? v1:172.21.15.104:0/2458527193' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:50:15.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:14 smithi055 bash[9296]: cluster 2020-09-08T15:50:13.908768+0000 mon.a (mon.0) 1444 : cluster [DBG] osdmap e372: 12 total, 12 up, 12 in 2020-09-08T15:50:15.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:14 smithi104 bash[9203]: cluster 2020-09-08T15:50:13.790694+0000 mgr.x (mgr.24118) 687 : cluster [DBG] pgmap v974: 17 pgs: 17 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.21k op/s 2020-09-08T15:50:15.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:14 smithi104 bash[9203]: cluster 2020-09-08T15:50:13.898411+0000 mon.a (mon.0) 1442 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:50:15.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:14 smithi104 bash[9203]: audit 2020-09-08T15:50:13.908707+0000 mon.a (mon.0) 1443 : audit [INF] from='client.? v1:172.21.15.104:0/2458527193' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:50:15.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:14 smithi104 bash[9203]: cluster 2020-09-08T15:50:13.908768+0000 mon.a (mon.0) 1444 : cluster [DBG] osdmap e372: 12 total, 12 up, 12 in 2020-09-08T15:50:15.365 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:50:15.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:50:15 smithi166 bash[14254]: debug 2020-09-08T15:50:14.996+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:15.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:14 smithi166 bash[9446]: cluster 2020-09-08T15:50:13.790694+0000 mgr.x (mgr.24118) 687 : cluster [DBG] pgmap v974: 17 pgs: 17 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.21k op/s 2020-09-08T15:50:15.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:14 smithi166 bash[9446]: cluster 2020-09-08T15:50:13.898411+0000 mon.a (mon.0) 1442 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:50:15.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:14 smithi166 bash[9446]: audit 2020-09-08T15:50:13.908707+0000 mon.a (mon.0) 1443 : audit [INF] from='client.? v1:172.21.15.104:0/2458527193' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:50:15.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:14 smithi166 bash[9446]: cluster 2020-09-08T15:50:13.908768+0000 mon.a (mon.0) 1444 : cluster [DBG] osdmap e372: 12 total, 12 up, 12 in 2020-09-08T15:50:15.470 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:15.592 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:15.658 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:50:15 smithi055 bash[14301]: debug 2020-09-08T15:50:15.363+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:15.694 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:15.811 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:50:15 smithi166 bash[18960]: debug 2020-09-08T15:50:15.588+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:15.811 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:15.919 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:16.101 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:16.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:50:15 smithi166 bash[18960]: debug 2020-09-08T15:50:15.804+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:16.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:50:16 smithi104 bash[17588]: debug 2020-09-08T15:50:16.099+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:16.204 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:16.323 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:16.429 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:16.512 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:50:16 smithi055 bash[11956]: debug 2020-09-08T15:50:16.323+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:16.548 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:16.550 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:50:16.582 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:50:16.630 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:50:16.652 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:50:16.666 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:50:16.706 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:50:16.746 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:50:16.780 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:50:16.781 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:50:16.807 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:50:16.842 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:50:16.881 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:50:16.883 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:16.893 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:50:16 smithi104 bash[17588]: debug 2020-09-08T15:50:16.551+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:16.898 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:16 smithi055 bash[9296]: cluster 2020-09-08T15:50:15.791186+0000 mgr.x (mgr.24118) 688 : cluster [DBG] pgmap v976: 17 pgs: 17 active+clean; 3.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 810 op/s 2020-09-08T15:50:16.899 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:16 smithi055 bash[9296]: audit 2020-09-08T15:50:16.508018+0000 mon.b (mon.1) 93 : audit [DBG] from='client.? v1:172.21.15.104:0/1375263638' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:50:16.925 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:50:16.974 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:50:16.976 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:50:16.977 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:50:16.977 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:50:16.977 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:50:16.977 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:50:16.978 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:50:16.978 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:50:16.978 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:50:16.978 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:50:16.978 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:50:16.979 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:50:16.979 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:50:17.030 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:17.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:50:16 smithi166 bash[14254]: debug 2020-09-08T15:50:16.776+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:17.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:16 smithi166 bash[9446]: cluster 2020-09-08T15:50:15.791186+0000 mgr.x (mgr.24118) 688 : cluster [DBG] pgmap v976: 17 pgs: 17 active+clean; 3.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 810 op/s 2020-09-08T15:50:17.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:16 smithi166 bash[9446]: audit 2020-09-08T15:50:16.508018+0000 mon.b (mon.1) 93 : audit [DBG] from='client.? v1:172.21.15.104:0/1375263638' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:50:17.142 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:17.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:16 smithi104 bash[9203]: cluster 2020-09-08T15:50:15.791186+0000 mgr.x (mgr.24118) 688 : cluster [DBG] pgmap v976: 17 pgs: 17 active+clean; 3.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 810 op/s 2020-09-08T15:50:17.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:16 smithi104 bash[9203]: audit 2020-09-08T15:50:16.508018+0000 mon.b (mon.1) 93 : audit [DBG] from='client.? v1:172.21.15.104:0/1375263638' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:50:17.177 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:50:17 smithi104 bash[15273]: debug 2020-09-08T15:50:17.031+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:17.294 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:17.397 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:50:17.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:50:17 smithi055 bash[11956]: debug 2020-09-08T15:50:17.295+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:17.517 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:50:17.622 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:50:17.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:50:17 smithi166 bash[11928]: debug 2020-09-08T15:50:17.516+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:17.927 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:50:17.959 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:50:18.062 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:18.087 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:50:18.177 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:18.282 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:50:18.291 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:50:17 smithi055 bash[16646]: debug 2020-09-08T15:50:17.959+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:18.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:50:18 smithi166 bash[18960]: debug 2020-09-08T15:50:18.176+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:18.650 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:50:18.754 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:18.906 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:50:18 smithi055 bash[19003]: debug 2020-09-08T15:50:18.647+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:19.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:18 smithi104 bash[9203]: cluster 2020-09-08T15:50:17.791693+0000 mgr.x (mgr.24118) 689 : cluster [DBG] pgmap v977: 17 pgs: 17 active+clean; 3.7 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 979 op/s 2020-09-08T15:50:19.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:18 smithi104 bash[9203]: cluster 2020-09-08T15:50:18.045045+0000 osd.5 (osd.5) 47 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:19.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:18 smithi104 bash[9203]: cluster 2020-09-08T15:50:18.045145+0000 osd.5 (osd.5) 48 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:19.210 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:19.214 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:50:19.214 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:50:19.287 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:18 smithi055 bash[9296]: cluster 2020-09-08T15:50:17.791693+0000 mgr.x (mgr.24118) 689 : cluster [DBG] pgmap v977: 17 pgs: 17 active+clean; 3.7 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 979 op/s 2020-09-08T15:50:19.288 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:18 smithi055 bash[9296]: cluster 2020-09-08T15:50:18.045045+0000 osd.5 (osd.5) 47 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:19.288 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:18 smithi055 bash[9296]: cluster 2020-09-08T15:50:18.045145+0000 osd.5 (osd.5) 48 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:19.315 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:19.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:18 smithi166 bash[9446]: cluster 2020-09-08T15:50:17.791693+0000 mgr.x (mgr.24118) 689 : cluster [DBG] pgmap v977: 17 pgs: 17 active+clean; 3.7 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 979 op/s 2020-09-08T15:50:19.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:18 smithi166 bash[9446]: cluster 2020-09-08T15:50:18.045045+0000 osd.5 (osd.5) 47 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:19.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:18 smithi166 bash[9446]: cluster 2020-09-08T15:50:18.045145+0000 osd.5 (osd.5) 48 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:19.498 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:50:19 smithi104 bash[15273]: debug 2020-09-08T15:50:19.211+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:19.502 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:19.606 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:19.802 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:50:19 smithi104 bash[17588]: debug 2020-09-08T15:50:19.499+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:20.063 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:20.169 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:50:20.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:19 smithi104 bash[9203]: cluster 2020-09-08T15:50:18.911699+0000 mon.a (mon.0) 1445 : cluster [DBG] osdmap e373: 12 total, 12 up, 12 in 2020-09-08T15:50:20.178 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:19 smithi166 bash[9446]: cluster 2020-09-08T15:50:18.911699+0000 mon.a (mon.0) 1445 : cluster [DBG] osdmap e373: 12 total, 12 up, 12 in 2020-09-08T15:50:20.198 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:19 smithi055 bash[9296]: cluster 2020-09-08T15:50:18.911699+0000 mon.a (mon.0) 1445 : cluster [DBG] osdmap e373: 12 total, 12 up, 12 in 2020-09-08T15:50:20.199 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:50:20 smithi055 bash[11956]: debug 2020-09-08T15:50:20.063+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:20.199 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2020-09-08T15:50:20.281 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:50:20.325 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:50:20.338 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:50:20.423 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:50:20.430 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:20.436 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:50:20 smithi166 bash[11928]: debug 2020-09-08T15:50:20.324+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:21 smithi055 bash[9296]: cluster 2020-09-08T15:50:19.792120+0000 mgr.x (mgr.24118) 690 : cluster [DBG] pgmap v979: 17 pgs: 17 active+clean; 3.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.07k op/s 2020-09-08T15:50:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:21 smithi055 bash[9296]: cluster 2020-09-08T15:50:20.496529+0000 osd.10 (osd.10) 37 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:21 smithi055 bash[9296]: cluster 2020-09-08T15:50:20.496618+0000 osd.10 (osd.10) 38 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:22.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:21 smithi055 bash[9296]: cluster 2020-09-08T15:50:20.919381+0000 mon.a (mon.0) 1446 : cluster [DBG] osdmap e374: 12 total, 12 up, 12 in 2020-09-08T15:50:22.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:21 smithi104 bash[9203]: cluster 2020-09-08T15:50:19.792120+0000 mgr.x (mgr.24118) 690 : cluster [DBG] pgmap v979: 17 pgs: 17 active+clean; 3.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.07k op/s 2020-09-08T15:50:22.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:21 smithi104 bash[9203]: cluster 2020-09-08T15:50:20.496529+0000 osd.10 (osd.10) 37 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:22.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:21 smithi104 bash[9203]: cluster 2020-09-08T15:50:20.496618+0000 osd.10 (osd.10) 38 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:22.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:21 smithi104 bash[9203]: cluster 2020-09-08T15:50:20.919381+0000 mon.a (mon.0) 1446 : cluster [DBG] osdmap e374: 12 total, 12 up, 12 in 2020-09-08T15:50:22.233 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:21 smithi166 bash[9446]: cluster 2020-09-08T15:50:19.792120+0000 mgr.x (mgr.24118) 690 : cluster [DBG] pgmap v979: 17 pgs: 17 active+clean; 3.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.07k op/s 2020-09-08T15:50:22.234 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:21 smithi166 bash[9446]: cluster 2020-09-08T15:50:20.496529+0000 osd.10 (osd.10) 37 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:22.234 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:21 smithi166 bash[9446]: cluster 2020-09-08T15:50:20.496618+0000 osd.10 (osd.10) 38 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:22.234 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:21 smithi166 bash[9446]: cluster 2020-09-08T15:50:20.919381+0000 mon.a (mon.0) 1446 : cluster [DBG] osdmap e374: 12 total, 12 up, 12 in 2020-09-08T15:50:23.532 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:23.635 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:23.886 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:23 smithi166 bash[9446]: cluster 2020-09-08T15:50:21.792530+0000 mgr.x (mgr.24118) 691 : cluster [DBG] pgmap v981: 17 pgs: 17 active+clean; 3.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 768 op/s 2020-09-08T15:50:23.886 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:50:23 smithi166 bash[18960]: debug 2020-09-08T15:50:23.528+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:24.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:23 smithi055 bash[9296]: cluster 2020-09-08T15:50:21.792530+0000 mgr.x (mgr.24118) 691 : cluster [DBG] pgmap v981: 17 pgs: 17 active+clean; 3.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 768 op/s 2020-09-08T15:50:24.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:23 smithi104 bash[9203]: cluster 2020-09-08T15:50:21.792530+0000 mgr.x (mgr.24118) 691 : cluster [DBG] pgmap v981: 17 pgs: 17 active+clean; 3.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 768 op/s 2020-09-08T15:50:24.304 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:24.408 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:50:24.515 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:50:24.618 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:50:24.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:50:24 smithi104 bash[15273]: debug 2020-09-08T15:50:24.307+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:24.732 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:50:24 smithi055 bash[16646]: debug 2020-09-08T15:50:24.515+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:24.734 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:50:24.758 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:50:24.839 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:50:24.964 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:50:25.070 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:50:25.072 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:24 smithi166 bash[9446]: audit 2020-09-08T15:50:23.780770+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? v1:172.21.15.104:0/1514255881' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:50:25.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:24 smithi055 bash[9296]: audit 2020-09-08T15:50:23.780770+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? v1:172.21.15.104:0/1514255881' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:50:25.158 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:50:24 smithi055 bash[19003]: debug 2020-09-08T15:50:24.731+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:25.158 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:50:24 smithi055 bash[14301]: debug 2020-09-08T15:50:24.963+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:25.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:24 smithi104 bash[9203]: audit 2020-09-08T15:50:23.780770+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? v1:172.21.15.104:0/1514255881' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:50:25.261 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:50:25.365 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:50:25.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:50:25 smithi166 bash[14254]: debug 2020-09-08T15:50:25.260+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:25.873 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:50:25.977 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:26.092 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:25 smithi055 bash[9296]: cluster 2020-09-08T15:50:23.792970+0000 mgr.x (mgr.24118) 692 : cluster [DBG] pgmap v982: 17 pgs: 17 active+clean; 3.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:50:26.092 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:25 smithi055 bash[9296]: cluster 2020-09-08T15:50:24.750441+0000 mon.a (mon.0) 1448 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:50:26.093 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:25 smithi055 bash[9296]: audit 2020-09-08T15:50:24.761133+0000 mon.a (mon.0) 1449 : audit [INF] from='client.? v1:172.21.15.104:0/1514255881' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:50:26.093 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:25 smithi055 bash[9296]: cluster 2020-09-08T15:50:24.761167+0000 mon.a (mon.0) 1450 : cluster [DBG] osdmap e375: 12 total, 12 up, 12 in 2020-09-08T15:50:26.096 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:26.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:25 smithi166 bash[9446]: cluster 2020-09-08T15:50:23.792970+0000 mgr.x (mgr.24118) 692 : cluster [DBG] pgmap v982: 17 pgs: 17 active+clean; 3.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:50:26.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:25 smithi166 bash[9446]: cluster 2020-09-08T15:50:24.750441+0000 mon.a (mon.0) 1448 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:50:26.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:25 smithi166 bash[9446]: audit 2020-09-08T15:50:24.761133+0000 mon.a (mon.0) 1449 : audit [INF] from='client.? v1:172.21.15.104:0/1514255881' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:50:26.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:25 smithi166 bash[9446]: cluster 2020-09-08T15:50:24.761167+0000 mon.a (mon.0) 1450 : cluster [DBG] osdmap e375: 12 total, 12 up, 12 in 2020-09-08T15:50:26.163 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:50:25 smithi104 bash[19922]: debug 2020-09-08T15:50:25.875+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:26.163 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:25 smithi104 bash[9203]: cluster 2020-09-08T15:50:23.792970+0000 mgr.x (mgr.24118) 692 : cluster [DBG] pgmap v982: 17 pgs: 17 active+clean; 3.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:50:26.164 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:25 smithi104 bash[9203]: cluster 2020-09-08T15:50:24.750441+0000 mon.a (mon.0) 1448 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:50:26.164 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:25 smithi104 bash[9203]: audit 2020-09-08T15:50:24.761133+0000 mon.a (mon.0) 1449 : audit [INF] from='client.? v1:172.21.15.104:0/1514255881' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:50:26.164 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:25 smithi104 bash[9203]: cluster 2020-09-08T15:50:24.761167+0000 mon.a (mon.0) 1450 : cluster [DBG] osdmap e375: 12 total, 12 up, 12 in 2020-09-08T15:50:26.202 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:50:26.334 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:50:26.408 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:50:26 smithi055 bash[11956]: debug 2020-09-08T15:50:26.091+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:26.408 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:50:26 smithi055 bash[16646]: debug 2020-09-08T15:50:26.327+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:26.436 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:26.575 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:50:26.609 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:26.612 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:50:26.644 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:50:26.675 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:50:26.706 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:50:26.714 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:26.723 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:50:26 smithi104 bash[15273]: debug 2020-09-08T15:50:26.611+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:26.741 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:50:26.774 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:50:26.805 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:50:26.830 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:26.838 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:50:26.874 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:50:26.913 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:50:26.946 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:26.947 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:50:26.963 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:50:26.963 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:50:26.963 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:50:26.963 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:50:26.964 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:50:26.964 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:50:26.964 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:50:26.964 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:50:26.964 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:50:26.965 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:50:26.965 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:50:26.965 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:50:27.041 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:26 smithi104 bash[9203]: cluster 2020-09-08T15:50:25.698314+0000 osd.3 (osd.3) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:27.041 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:26 smithi104 bash[9203]: cluster 2020-09-08T15:50:25.698380+0000 osd.3 (osd.3) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:27.041 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:26 smithi104 bash[9203]: cluster 2020-09-08T15:50:25.761588+0000 mon.a (mon.0) 1451 : cluster [DBG] osdmap e376: 12 total, 12 up, 12 in 2020-09-08T15:50:27.042 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:26 smithi104 bash[9203]: cluster 2020-09-08T15:50:25.793365+0000 mgr.x (mgr.24118) 693 : cluster [DBG] pgmap v985: 17 pgs: 17 active+clean; 3.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 832 op/s 2020-09-08T15:50:27.042 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:26 smithi104 bash[9203]: audit 2020-09-08T15:50:26.547941+0000 mon.a (mon.0) 1452 : audit [DBG] from='client.? v1:172.21.15.104:0/2463198990' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:50:27.042 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:50:26 smithi104 bash[15273]: debug 2020-09-08T15:50:26.831+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:27.066 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:27.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:26 smithi166 bash[9446]: cluster 2020-09-08T15:50:25.698314+0000 osd.3 (osd.3) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:27.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:26 smithi166 bash[9446]: cluster 2020-09-08T15:50:25.698380+0000 osd.3 (osd.3) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:27.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:26 smithi166 bash[9446]: cluster 2020-09-08T15:50:25.761588+0000 mon.a (mon.0) 1451 : cluster [DBG] osdmap e376: 12 total, 12 up, 12 in 2020-09-08T15:50:27.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:26 smithi166 bash[9446]: cluster 2020-09-08T15:50:25.793365+0000 mgr.x (mgr.24118) 693 : cluster [DBG] pgmap v985: 17 pgs: 17 active+clean; 3.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 832 op/s 2020-09-08T15:50:27.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:26 smithi166 bash[9446]: audit 2020-09-08T15:50:26.547941+0000 mon.a (mon.0) 1452 : audit [DBG] from='client.? v1:172.21.15.104:0/2463198990' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:50:27.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:26 smithi055 bash[9296]: cluster 2020-09-08T15:50:25.698314+0000 osd.3 (osd.3) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:27.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:26 smithi055 bash[9296]: cluster 2020-09-08T15:50:25.698380+0000 osd.3 (osd.3) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:27.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:26 smithi055 bash[9296]: cluster 2020-09-08T15:50:25.761588+0000 mon.a (mon.0) 1451 : cluster [DBG] osdmap e376: 12 total, 12 up, 12 in 2020-09-08T15:50:27.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:26 smithi055 bash[9296]: cluster 2020-09-08T15:50:25.793365+0000 mgr.x (mgr.24118) 693 : cluster [DBG] pgmap v985: 17 pgs: 17 active+clean; 3.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 832 op/s 2020-09-08T15:50:27.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:26 smithi055 bash[9296]: audit 2020-09-08T15:50:26.547941+0000 mon.a (mon.0) 1452 : audit [DBG] from='client.? v1:172.21.15.104:0/2463198990' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:50:27.173 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:27.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:50:27 smithi104 bash[15273]: debug 2020-09-08T15:50:27.067+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:27.733 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:50:27.733 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T15:50:27.733 INFO:teuthology.orchestra.run.smithi166.stdout: "num_ops": 0 2020-09-08T15:50:27.734 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:50:27.963 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:28.064 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:28.165 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:50:28.333 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:50:28.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:50:27 smithi104 bash[17588]: debug 2020-09-08T15:50:27.959+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:28.642 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:28.746 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:50:28.789 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2020-09-08T15:50:28.892 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:50:28.893 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:28 smithi166 bash[9446]: cluster 2020-09-08T15:50:27.793806+0000 mgr.x (mgr.24118) 694 : cluster [DBG] pgmap v986: 17 pgs: 17 active+clean; 3.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T15:50:28.894 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:50:28 smithi166 bash[14254]: debug 2020-09-08T15:50:28.888+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:28.894 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:50:28 smithi166 bash[18960]: debug 2020-09-08T15:50:28.640+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:28.995 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:50:29.000 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:29.144 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:28 smithi055 bash[9296]: cluster 2020-09-08T15:50:27.793806+0000 mgr.x (mgr.24118) 694 : cluster [DBG] pgmap v986: 17 pgs: 17 active+clean; 3.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T15:50:29.148 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:29.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:28 smithi104 bash[9203]: cluster 2020-09-08T15:50:27.793806+0000 mgr.x (mgr.24118) 694 : cluster [DBG] pgmap v986: 17 pgs: 17 active+clean; 3.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T15:50:29.252 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:29.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:50:29 smithi055 bash[11956]: debug 2020-09-08T15:50:29.143+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:29.778 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:29.886 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:50:30.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:50:29 smithi166 bash[18960]: debug 2020-09-08T15:50:29.776+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:30.213 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:50:30.375 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:50:30.509 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:50:30.613 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:30.768 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:50:30 smithi166 bash[16593]: debug 2020-09-08T15:50:30.508+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:31.142 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:31.246 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:31.406 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:31.435 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:50:31 smithi104 bash[15273]: debug 2020-09-08T15:50:31.143+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:31.508 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:31.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:50:31 smithi055 bash[11956]: debug 2020-09-08T15:50:31.407+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:31.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:31 smithi166 bash[9446]: cluster 2020-09-08T15:50:29.794242+0000 mgr.x (mgr.24118) 695 : cluster [DBG] pgmap v987: 17 pgs: 17 active+clean; 3.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T15:50:31.901 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:32.003 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:50:32.012 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:31 smithi055 bash[9296]: cluster 2020-09-08T15:50:29.794242+0000 mgr.x (mgr.24118) 695 : cluster [DBG] pgmap v987: 17 pgs: 17 active+clean; 3.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T15:50:32.108 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:50:32.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:31 smithi104 bash[9203]: cluster 2020-09-08T15:50:29.794242+0000 mgr.x (mgr.24118) 695 : cluster [DBG] pgmap v987: 17 pgs: 17 active+clean; 3.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T15:50:32.211 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:32.333 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:50:32 smithi055 bash[19003]: debug 2020-09-08T15:50:32.107+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:32.336 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:32.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:50:31 smithi166 bash[18960]: debug 2020-09-08T15:50:31.900+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:32.443 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:32.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:50:32 smithi055 bash[11956]: debug 2020-09-08T15:50:32.335+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:32.786 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:32.891 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:50:33.017 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:32 smithi055 bash[9296]: cluster 2020-09-08T15:50:31.835017+0000 osd.8 (osd.8) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:33.018 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:32 smithi055 bash[9296]: cluster 2020-09-08T15:50:31.835112+0000 osd.8 (osd.8) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:33.020 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:50:33.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:32 smithi166 bash[9446]: cluster 2020-09-08T15:50:31.835017+0000 osd.8 (osd.8) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:33.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:32 smithi166 bash[9446]: cluster 2020-09-08T15:50:31.835112+0000 osd.8 (osd.8) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:33.123 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:50:33.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:50:32 smithi104 bash[15273]: debug 2020-09-08T15:50:32.787+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:33.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:32 smithi104 bash[9203]: cluster 2020-09-08T15:50:31.835017+0000 osd.8 (osd.8) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:33.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:32 smithi104 bash[9203]: cluster 2020-09-08T15:50:31.835112+0000 osd.8 (osd.8) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:33.299 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:50:33.403 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:50:33.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:50:33 smithi055 bash[14301]: debug 2020-09-08T15:50:33.019+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:33.420 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:50:33 smithi166 bash[16593]: debug 2020-09-08T15:50:33.300+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:33.525 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:50:33.578 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:50:33.578 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T15:50:33.578 INFO:teuthology.orchestra.run.smithi166.stdout: "complaint_time": 30, 2020-09-08T15:50:33.578 INFO:teuthology.orchestra.run.smithi166.stdout: "num_blocked_ops": 0 2020-09-08T15:50:33.579 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:50:33.630 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:50:33.749 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:50:33 smithi055 bash[14301]: debug 2020-09-08T15:50:33.527+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:33.749 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:50:33.779 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:50:33.855 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:50:34.021 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:33 smithi104 bash[9203]: cluster 2020-09-08T15:50:31.794694+0000 mgr.x (mgr.24118) 696 : cluster [DBG] pgmap v988: 17 pgs: 17 active+clean; 2.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 742 op/s 2020-09-08T15:50:34.021 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:33 smithi104 bash[9203]: cluster 2020-09-08T15:50:32.768040+0000 mon.a (mon.0) 1453 : cluster [DBG] osdmap e377: 12 total, 12 up, 12 in 2020-09-08T15:50:34.022 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:33 smithi104 bash[9203]: audit 2020-09-08T15:50:33.730938+0000 mon.b (mon.1) 94 : audit [INF] from='client.? v1:172.21.15.104:0/714095715' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:50:34.022 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:33 smithi104 bash[9203]: audit 2020-09-08T15:50:33.731277+0000 mon.a (mon.0) 1454 : audit [INF] from='client.25546 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:50:34.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:33 smithi166 bash[9446]: cluster 2020-09-08T15:50:31.794694+0000 mgr.x (mgr.24118) 696 : cluster [DBG] pgmap v988: 17 pgs: 17 active+clean; 2.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 742 op/s 2020-09-08T15:50:34.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:33 smithi166 bash[9446]: cluster 2020-09-08T15:50:32.768040+0000 mon.a (mon.0) 1453 : cluster [DBG] osdmap e377: 12 total, 12 up, 12 in 2020-09-08T15:50:34.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:33 smithi166 bash[9446]: audit 2020-09-08T15:50:33.730938+0000 mon.b (mon.1) 94 : audit [INF] from='client.? v1:172.21.15.104:0/714095715' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:50:34.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:33 smithi166 bash[9446]: audit 2020-09-08T15:50:33.731277+0000 mon.a (mon.0) 1454 : audit [INF] from='client.25546 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:50:34.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:50:33 smithi055 bash[16646]: debug 2020-09-08T15:50:33.751+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:34.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:33 smithi055 bash[9296]: cluster 2020-09-08T15:50:31.794694+0000 mgr.x (mgr.24118) 696 : cluster [DBG] pgmap v988: 17 pgs: 17 active+clean; 2.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 742 op/s 2020-09-08T15:50:34.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:33 smithi055 bash[9296]: cluster 2020-09-08T15:50:32.768040+0000 mon.a (mon.0) 1453 : cluster [DBG] osdmap e377: 12 total, 12 up, 12 in 2020-09-08T15:50:34.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:33 smithi055 bash[9296]: audit 2020-09-08T15:50:33.730938+0000 mon.b (mon.1) 94 : audit [INF] from='client.? v1:172.21.15.104:0/714095715' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:50:34.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:33 smithi055 bash[9296]: audit 2020-09-08T15:50:33.731277+0000 mon.a (mon.0) 1454 : audit [INF] from='client.25546 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:50:34.509 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:50:34.612 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:50:34.632 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2020-09-08T15:50:34.744 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:50:34.772 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:34 smithi166 bash[9446]: cluster 2020-09-08T15:50:33.765750+0000 mon.a (mon.0) 1455 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:50:34.772 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:34 smithi166 bash[9446]: audit 2020-09-08T15:50:33.780534+0000 mon.a (mon.0) 1456 : audit [INF] from='client.25546 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:50:34.773 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:34 smithi166 bash[9446]: cluster 2020-09-08T15:50:33.780574+0000 mon.a (mon.0) 1457 : cluster [DBG] osdmap e378: 12 total, 12 up, 12 in 2020-09-08T15:50:34.773 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:34 smithi166 bash[9446]: cluster 2020-09-08T15:50:33.795133+0000 mgr.x (mgr.24118) 697 : cluster [DBG] pgmap v991: 17 pgs: 17 active+clean; 2.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:50:34.774 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:50:34 smithi166 bash[11928]: debug 2020-09-08T15:50:34.508+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:34.774 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:50:34 smithi166 bash[11928]: debug 2020-09-08T15:50:34.740+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:34.839 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:50:34.852 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:50:35.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:34 smithi055 bash[9296]: cluster 2020-09-08T15:50:33.765750+0000 mon.a (mon.0) 1455 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:50:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:34 smithi055 bash[9296]: audit 2020-09-08T15:50:33.780534+0000 mon.a (mon.0) 1456 : audit [INF] from='client.25546 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:50:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:34 smithi055 bash[9296]: cluster 2020-09-08T15:50:33.780574+0000 mon.a (mon.0) 1457 : cluster [DBG] osdmap e378: 12 total, 12 up, 12 in 2020-09-08T15:50:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:34 smithi055 bash[9296]: cluster 2020-09-08T15:50:33.795133+0000 mgr.x (mgr.24118) 697 : cluster [DBG] pgmap v991: 17 pgs: 17 active+clean; 2.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:50:35.158 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:50:35.159 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:34 smithi104 bash[9203]: cluster 2020-09-08T15:50:33.765750+0000 mon.a (mon.0) 1455 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:50:35.159 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:34 smithi104 bash[9203]: audit 2020-09-08T15:50:33.780534+0000 mon.a (mon.0) 1456 : audit [INF] from='client.25546 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:50:35.159 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:34 smithi104 bash[9203]: cluster 2020-09-08T15:50:33.780574+0000 mon.a (mon.0) 1457 : cluster [DBG] osdmap e378: 12 total, 12 up, 12 in 2020-09-08T15:50:35.159 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:34 smithi104 bash[9203]: cluster 2020-09-08T15:50:33.795133+0000 mgr.x (mgr.24118) 697 : cluster [DBG] pgmap v991: 17 pgs: 17 active+clean; 2.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:50:35.160 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:50:35 smithi104 bash[22305]: debug 2020-09-08T15:50:35.158+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:35.263 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:50:35.572 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:50:35 smithi166 bash[16593]: debug 2020-09-08T15:50:35.572+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:35.572 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:50:35.675 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:50:35.806 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:50:35.908 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:36.016 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:50:35 smithi055 bash[19003]: debug 2020-09-08T15:50:35.807+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:36.017 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:50:36 smithi055 bash[11956]: debug 2020-09-08T15:50:36.015+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:36.017 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:36.123 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:50:36.300 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:50:36.368 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:50:36.405 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:50:36.414 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:50:36.461 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:50:36.491 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:50:36.510 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:50:36 smithi166 bash[16593]: debug 2020-09-08T15:50:36.300+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:36.532 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:50:36.548 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:50:36.574 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:50:36.610 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:50:36.646 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:50:36.650 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:36.683 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:50:36.715 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:50:36.749 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:50:36.781 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:50:36.784 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:50:36.784 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:50:36.784 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:50:36.784 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:50:36.785 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:50:36.785 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:50:36.785 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:50:36.785 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:50:36.785 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:50:36.786 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:50:36.786 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:50:36.786 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:50:36.802 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:36.847 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:50:36 smithi055 bash[19003]: debug 2020-09-08T15:50:36.543+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:36.909 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:36.918 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:36 smithi104 bash[9203]: cluster 2020-09-08T15:50:35.483731+0000 osd.7 (osd.7) 71 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:50:36.918 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:36 smithi104 bash[9203]: cluster 2020-09-08T15:50:35.565607+0000 osd.7 (osd.7) 72 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:50:36.919 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:36 smithi104 bash[9203]: cluster 2020-09-08T15:50:35.795538+0000 mgr.x (mgr.24118) 698 : cluster [DBG] pgmap v992: 17 pgs: 17 active+clean; 2.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 907 op/s 2020-09-08T15:50:36.919 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:36 smithi104 bash[9203]: cluster 2020-09-08T15:50:35.981154+0000 osd.0 (osd.0) 47 : cluster [DBG] 4.b scrub starts 2020-09-08T15:50:36.919 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:36 smithi104 bash[9203]: cluster 2020-09-08T15:50:36.220973+0000 osd.11 (osd.11) 58 : cluster [DBG] 4.8 scrub starts 2020-09-08T15:50:36.919 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:36 smithi104 bash[9203]: audit 2020-09-08T15:50:36.329723+0000 mon.a (mon.0) 1458 : audit [DBG] from='client.? v1:172.21.15.104:0/118508492' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:50:36.919 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:50:36 smithi104 bash[15273]: debug 2020-09-08T15:50:36.802+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:37.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:36 smithi166 bash[9446]: cluster 2020-09-08T15:50:35.483731+0000 osd.7 (osd.7) 71 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:50:37.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:36 smithi166 bash[9446]: cluster 2020-09-08T15:50:35.565607+0000 osd.7 (osd.7) 72 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:50:37.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:36 smithi166 bash[9446]: cluster 2020-09-08T15:50:35.795538+0000 mgr.x (mgr.24118) 698 : cluster [DBG] pgmap v992: 17 pgs: 17 active+clean; 2.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 907 op/s 2020-09-08T15:50:37.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:36 smithi166 bash[9446]: cluster 2020-09-08T15:50:35.981154+0000 osd.0 (osd.0) 47 : cluster [DBG] 4.b scrub starts 2020-09-08T15:50:37.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:36 smithi166 bash[9446]: cluster 2020-09-08T15:50:36.220973+0000 osd.11 (osd.11) 58 : cluster [DBG] 4.8 scrub starts 2020-09-08T15:50:37.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:36 smithi166 bash[9446]: audit 2020-09-08T15:50:36.329723+0000 mon.a (mon.0) 1458 : audit [DBG] from='client.? v1:172.21.15.104:0/118508492' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:50:37.134 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:37.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:36 smithi055 bash[9296]: cluster 2020-09-08T15:50:35.483731+0000 osd.7 (osd.7) 71 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:50:37.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:36 smithi055 bash[9296]: cluster 2020-09-08T15:50:35.565607+0000 osd.7 (osd.7) 72 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:50:37.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:36 smithi055 bash[9296]: cluster 2020-09-08T15:50:35.795538+0000 mgr.x (mgr.24118) 698 : cluster [DBG] pgmap v992: 17 pgs: 17 active+clean; 2.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 907 op/s 2020-09-08T15:50:37.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:36 smithi055 bash[9296]: cluster 2020-09-08T15:50:35.981154+0000 osd.0 (osd.0) 47 : cluster [DBG] 4.b scrub starts 2020-09-08T15:50:37.162 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:36 smithi055 bash[9296]: cluster 2020-09-08T15:50:36.220973+0000 osd.11 (osd.11) 58 : cluster [DBG] 4.8 scrub starts 2020-09-08T15:50:37.162 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:36 smithi055 bash[9296]: audit 2020-09-08T15:50:36.329723+0000 mon.a (mon.0) 1458 : audit [DBG] from='client.? v1:172.21.15.104:0/118508492' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:50:37.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:50:37 smithi104 bash[15273]: debug 2020-09-08T15:50:37.134+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:37.240 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:50:37.245 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:50:37.402 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:50:37.768 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:50:37.872 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:50:38.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:37 smithi166 bash[9446]: cluster 2020-09-08T15:50:36.979272+0000 osd.0 (osd.0) 48 : cluster [DBG] 4.b scrub ok 2020-09-08T15:50:38.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:37 smithi166 bash[9446]: cluster 2020-09-08T15:50:37.108356+0000 osd.11 (osd.11) 59 : cluster [DBG] 4.8 scrub ok 2020-09-08T15:50:38.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:50:37 smithi166 bash[16593]: debug 2020-09-08T15:50:37.768+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:38.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:37 smithi055 bash[9296]: cluster 2020-09-08T15:50:36.979272+0000 osd.0 (osd.0) 48 : cluster [DBG] 4.b scrub ok 2020-09-08T15:50:38.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:37 smithi055 bash[9296]: cluster 2020-09-08T15:50:37.108356+0000 osd.11 (osd.11) 59 : cluster [DBG] 4.8 scrub ok 2020-09-08T15:50:38.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:37 smithi104 bash[9203]: cluster 2020-09-08T15:50:36.979272+0000 osd.0 (osd.0) 48 : cluster [DBG] 4.b scrub ok 2020-09-08T15:50:38.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:37 smithi104 bash[9203]: cluster 2020-09-08T15:50:37.108356+0000 osd.11 (osd.11) 59 : cluster [DBG] 4.8 scrub ok 2020-09-08T15:50:38.257 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:50:38.361 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:50:38.635 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:50:38 smithi166 bash[16593]: debug 2020-09-08T15:50:38.636+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:38.636 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:50:38.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:50:38 smithi104 bash[19922]: debug 2020-09-08T15:50:38.258+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:38.739 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:50:39.038 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:38 smithi104 bash[9203]: cluster 2020-09-08T15:50:37.796183+0000 mgr.x (mgr.24118) 699 : cluster [DBG] pgmap v993: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 976 op/s 2020-09-08T15:50:39.039 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:50:39.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:38 smithi166 bash[9446]: cluster 2020-09-08T15:50:37.796183+0000 mgr.x (mgr.24118) 699 : cluster [DBG] pgmap v993: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 976 op/s 2020-09-08T15:50:39.151 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:39.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:38 smithi055 bash[9296]: cluster 2020-09-08T15:50:37.796183+0000 mgr.x (mgr.24118) 699 : cluster [DBG] pgmap v993: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 976 op/s 2020-09-08T15:50:39.340 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:39.341 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:50:39 smithi104 bash[22305]: debug 2020-09-08T15:50:39.038+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:39.444 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:39.517 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:50:39 smithi166 bash[18960]: debug 2020-09-08T15:50:39.332+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:39.577 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:39.661 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:50:39 smithi104 bash[17588]: debug 2020-09-08T15:50:39.574+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:39.687 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:39.775 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:50:39.776 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:50:39.816 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:39.920 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:50:40.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:50:39 smithi166 bash[18960]: debug 2020-09-08T15:50:39.816+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:40.557 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:50:40.661 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:50:40.755 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_ops_in_flight 2020-09-08T15:50:40.798 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:50:40.810 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:50:40 smithi104 bash[19922]: debug 2020-09-08T15:50:40.558+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:40.912 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:50:40.959 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:40 smithi055 bash[9296]: cluster 2020-09-08T15:50:39.558373+0000 osd.1 (osd.1) 76 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:40.959 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:40 smithi055 bash[9296]: cluster 2020-09-08T15:50:39.558454+0000 osd.1 (osd.1) 77 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:40.959 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:40 smithi055 bash[9296]: cluster 2020-09-08T15:50:39.796592+0000 mgr.x (mgr.24118) 700 : cluster [DBG] pgmap v994: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T15:50:40.959 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:40 smithi055 bash[9296]: cluster 2020-09-08T15:50:39.867818+0000 mon.a (mon.0) 1459 : cluster [DBG] osdmap e379: 12 total, 12 up, 12 in 2020-09-08T15:50:40.960 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:50:40.963 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:50:41.061 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:41.072 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:40 smithi104 bash[9203]: cluster 2020-09-08T15:50:39.558373+0000 osd.1 (osd.1) 76 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:41.073 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:40 smithi104 bash[9203]: cluster 2020-09-08T15:50:39.558454+0000 osd.1 (osd.1) 77 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:41.073 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:40 smithi104 bash[9203]: cluster 2020-09-08T15:50:39.796592+0000 mgr.x (mgr.24118) 700 : cluster [DBG] pgmap v994: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T15:50:41.073 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:40 smithi104 bash[9203]: cluster 2020-09-08T15:50:39.867818+0000 mon.a (mon.0) 1459 : cluster [DBG] osdmap e379: 12 total, 12 up, 12 in 2020-09-08T15:50:41.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:40 smithi166 bash[9446]: cluster 2020-09-08T15:50:39.558373+0000 osd.1 (osd.1) 76 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:41.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:40 smithi166 bash[9446]: cluster 2020-09-08T15:50:39.558454+0000 osd.1 (osd.1) 77 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:41.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:40 smithi166 bash[9446]: cluster 2020-09-08T15:50:39.796592+0000 mgr.x (mgr.24118) 700 : cluster [DBG] pgmap v994: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T15:50:41.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:40 smithi166 bash[9446]: cluster 2020-09-08T15:50:39.867818+0000 mon.a (mon.0) 1459 : cluster [DBG] osdmap e379: 12 total, 12 up, 12 in 2020-09-08T15:50:41.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:50:40 smithi055 bash[19003]: debug 2020-09-08T15:50:40.947+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:41.626 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:41.731 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:50:41.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:50:41 smithi104 bash[17588]: debug 2020-09-08T15:50:41.626+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:42.269 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:50:42.377 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:50:42.503 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:50:42.607 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:50:42.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:50:42 smithi104 bash[22305]: debug 2020-09-08T15:50:42.266+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:42.733 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:50:42.837 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:42.845 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:50:42 smithi166 bash[16593]: debug 2020-09-08T15:50:42.503+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:42.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:50:42 smithi055 bash[16646]: debug 2020-09-08T15:50:42.731+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:42.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:42 smithi055 bash[9296]: cluster 2020-09-08T15:50:41.797075+0000 mgr.x (mgr.24118) 701 : cluster [DBG] pgmap v996: 17 pgs: 17 active+clean; 2.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 719 op/s 2020-09-08T15:50:43.077 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:42 smithi104 bash[9203]: cluster 2020-09-08T15:50:41.797075+0000 mgr.x (mgr.24118) 701 : cluster [DBG] pgmap v996: 17 pgs: 17 active+clean; 2.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 719 op/s 2020-09-08T15:50:43.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:42 smithi166 bash[9446]: cluster 2020-09-08T15:50:41.797075+0000 mgr.x (mgr.24118) 701 : cluster [DBG] pgmap v996: 17 pgs: 17 active+clean; 2.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 719 op/s 2020-09-08T15:50:43.346 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:43.450 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:50:43.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:50:43 smithi166 bash[18960]: debug 2020-09-08T15:50:43.347+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:43.885 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:50:44.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:43 smithi166 bash[9446]: audit 2020-09-08T15:50:43.359606+0000 mon.a (mon.0) 1460 : audit [INF] from='client.? v1:172.21.15.104:0/1762716188' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:50:44.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:43 smithi055 bash[9296]: audit 2020-09-08T15:50:43.359606+0000 mon.a (mon.0) 1460 : audit [INF] from='client.? v1:172.21.15.104:0/1762716188' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:50:44.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:43 smithi104 bash[9203]: audit 2020-09-08T15:50:43.359606+0000 mon.a (mon.0) 1460 : audit [INF] from='client.? v1:172.21.15.104:0/1762716188' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:50:44.184 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:50:44.289 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:50:44.447 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:50:44.551 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:44.659 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:50:44 smithi055 bash[16646]: debug 2020-09-08T15:50:44.447+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:44.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:50:44 smithi104 bash[22305]: debug 2020-09-08T15:50:44.186+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:44.831 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:50:44.934 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:45.108 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:50:44 smithi166 bash[18960]: debug 2020-09-08T15:50:44.831+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:45.108 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:44 smithi166 bash[9446]: cluster 2020-09-08T15:50:43.797575+0000 mgr.x (mgr.24118) 702 : cluster [DBG] pgmap v997: 17 pgs: 17 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 966 op/s 2020-09-08T15:50:45.109 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:44 smithi166 bash[9446]: cluster 2020-09-08T15:50:43.869356+0000 mon.a (mon.0) 1461 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:50:45.109 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:44 smithi166 bash[9446]: cluster 2020-09-08T15:50:43.869395+0000 mon.a (mon.0) 1462 : cluster [INF] Cluster is now healthy 2020-09-08T15:50:45.109 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:44 smithi166 bash[9446]: audit 2020-09-08T15:50:43.880954+0000 mon.a (mon.0) 1463 : audit [INF] from='client.? v1:172.21.15.104:0/1762716188' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:50:45.110 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:44 smithi166 bash[9446]: cluster 2020-09-08T15:50:43.881003+0000 mon.a (mon.0) 1464 : cluster [DBG] osdmap e380: 12 total, 12 up, 12 in 2020-09-08T15:50:45.110 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:44 smithi166 bash[9446]: cluster 2020-09-08T15:50:44.202696+0000 osd.11 (osd.11) 60 : cluster [DBG] 4.9 scrub starts 2020-09-08T15:50:45.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:44 smithi055 bash[9296]: cluster 2020-09-08T15:50:43.797575+0000 mgr.x (mgr.24118) 702 : cluster [DBG] pgmap v997: 17 pgs: 17 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 966 op/s 2020-09-08T15:50:45.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:44 smithi055 bash[9296]: cluster 2020-09-08T15:50:43.869356+0000 mon.a (mon.0) 1461 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:50:45.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:44 smithi055 bash[9296]: cluster 2020-09-08T15:50:43.869395+0000 mon.a (mon.0) 1462 : cluster [INF] Cluster is now healthy 2020-09-08T15:50:45.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:44 smithi055 bash[9296]: audit 2020-09-08T15:50:43.880954+0000 mon.a (mon.0) 1463 : audit [INF] from='client.? v1:172.21.15.104:0/1762716188' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:50:45.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:44 smithi055 bash[9296]: cluster 2020-09-08T15:50:43.881003+0000 mon.a (mon.0) 1464 : cluster [DBG] osdmap e380: 12 total, 12 up, 12 in 2020-09-08T15:50:45.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:44 smithi055 bash[9296]: cluster 2020-09-08T15:50:44.202696+0000 osd.11 (osd.11) 60 : cluster [DBG] 4.9 scrub starts 2020-09-08T15:50:45.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:44 smithi104 bash[9203]: cluster 2020-09-08T15:50:43.797575+0000 mgr.x (mgr.24118) 702 : cluster [DBG] pgmap v997: 17 pgs: 17 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 966 op/s 2020-09-08T15:50:45.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:44 smithi104 bash[9203]: cluster 2020-09-08T15:50:43.869356+0000 mon.a (mon.0) 1461 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:50:45.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:44 smithi104 bash[9203]: cluster 2020-09-08T15:50:43.869395+0000 mon.a (mon.0) 1462 : cluster [INF] Cluster is now healthy 2020-09-08T15:50:45.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:44 smithi104 bash[9203]: audit 2020-09-08T15:50:43.880954+0000 mon.a (mon.0) 1463 : audit [INF] from='client.? v1:172.21.15.104:0/1762716188' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:50:45.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:44 smithi104 bash[9203]: cluster 2020-09-08T15:50:43.881003+0000 mon.a (mon.0) 1464 : cluster [DBG] osdmap e380: 12 total, 12 up, 12 in 2020-09-08T15:50:45.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:44 smithi104 bash[9203]: cluster 2020-09-08T15:50:44.202696+0000 osd.11 (osd.11) 60 : cluster [DBG] 4.9 scrub starts 2020-09-08T15:50:45.295 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:50:45.296 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:50:45.530 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:45.635 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:50:45.879 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:50:45 smithi104 bash[15273]: debug 2020-09-08T15:50:45.530+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:45.879 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:45 smithi104 bash[9203]: cluster 2020-09-08T15:50:44.537086+0000 osd.6 (osd.6) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:45.880 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:45 smithi104 bash[9203]: cluster 2020-09-08T15:50:44.537205+0000 osd.6 (osd.6) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:45.880 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:45 smithi104 bash[9203]: cluster 2020-09-08T15:50:44.879759+0000 mon.a (mon.0) 1465 : cluster [DBG] osdmap e381: 12 total, 12 up, 12 in 2020-09-08T15:50:45.880 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:45 smithi104 bash[9203]: cluster 2020-09-08T15:50:45.215859+0000 osd.11 (osd.11) 61 : cluster [DBG] 4.9 scrub ok 2020-09-08T15:50:46.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:45 smithi166 bash[9446]: cluster 2020-09-08T15:50:44.537086+0000 osd.6 (osd.6) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:46.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:45 smithi166 bash[9446]: cluster 2020-09-08T15:50:44.537205+0000 osd.6 (osd.6) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:46.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:45 smithi166 bash[9446]: cluster 2020-09-08T15:50:44.879759+0000 mon.a (mon.0) 1465 : cluster [DBG] osdmap e381: 12 total, 12 up, 12 in 2020-09-08T15:50:46.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:45 smithi166 bash[9446]: cluster 2020-09-08T15:50:45.215859+0000 osd.11 (osd.11) 61 : cluster [DBG] 4.9 scrub ok 2020-09-08T15:50:46.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:45 smithi055 bash[9296]: cluster 2020-09-08T15:50:44.537086+0000 osd.6 (osd.6) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:46.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:45 smithi055 bash[9296]: cluster 2020-09-08T15:50:44.537205+0000 osd.6 (osd.6) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:46.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:45 smithi055 bash[9296]: cluster 2020-09-08T15:50:44.879759+0000 mon.a (mon.0) 1465 : cluster [DBG] osdmap e381: 12 total, 12 up, 12 in 2020-09-08T15:50:46.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:45 smithi055 bash[9296]: cluster 2020-09-08T15:50:45.215859+0000 osd.11 (osd.11) 61 : cluster [DBG] 4.9 scrub ok 2020-09-08T15:50:46.411 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:50:46.512 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:46.524 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_blocked_ops 2020-09-08T15:50:46.679 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:46.766 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:50:46.783 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:46.792 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:50:46 smithi104 bash[15273]: debug 2020-09-08T15:50:46.674+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:46.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:50:46 smithi166 bash[11928]: debug 2020-09-08T15:50:46.403+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:46.912 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:50:46.914 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:46.943 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:50:46.980 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:50:47.018 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:47.020 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:50:47.058 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:50:47.093 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:50:47.129 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:50:47.144 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:47.146 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:50:46 smithi104 bash[17588]: debug 2020-09-08T15:50:46.914+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:47.146 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:46 smithi104 bash[9203]: cluster 2020-09-08T15:50:45.797968+0000 mgr.x (mgr.24118) 703 : cluster [DBG] pgmap v1000: 17 pgs: 17 active+clean; 2.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 801 op/s 2020-09-08T15:50:47.147 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:46 smithi104 bash[9203]: audit 2020-09-08T15:50:46.878623+0000 mon.a (mon.0) 1466 : audit [DBG] from='client.? v1:172.21.15.104:0/544869771' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:50:47.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:46 smithi055 bash[9296]: cluster 2020-09-08T15:50:45.797968+0000 mgr.x (mgr.24118) 703 : cluster [DBG] pgmap v1000: 17 pgs: 17 active+clean; 2.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 801 op/s 2020-09-08T15:50:47.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:46 smithi055 bash[9296]: audit 2020-09-08T15:50:46.878623+0000 mon.a (mon.0) 1466 : audit [DBG] from='client.? v1:172.21.15.104:0/544869771' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:50:47.168 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:50:47.204 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:50:47.237 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:50:47.249 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:50:47.278 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:50:47.313 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:50:47.320 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:50:47.321 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:50:47.331 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:50:47.331 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:50:47.332 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:50:47.332 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:50:47.332 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:50:47.332 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:50:47.332 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:50:47.333 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:50:47.333 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:50:47.333 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:50:47.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:46 smithi166 bash[9446]: cluster 2020-09-08T15:50:45.797968+0000 mgr.x (mgr.24118) 703 : cluster [DBG] pgmap v1000: 17 pgs: 17 active+clean; 2.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 801 op/s 2020-09-08T15:50:47.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:46 smithi166 bash[9446]: audit 2020-09-08T15:50:46.878623+0000 mon.a (mon.0) 1466 : audit [DBG] from='client.? v1:172.21.15.104:0/544869771' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:50:47.400 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:50:47 smithi104 bash[17588]: debug 2020-09-08T15:50:47.146+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:47.782 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:50:47.939 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:50:48.161 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:50:48.266 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:50:48.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:50:48 smithi055 bash[14301]: debug 2020-09-08T15:50:48.159+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:48.698 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:50:48.801 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:50:49.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:48 smithi166 bash[9446]: cluster 2020-09-08T15:50:47.462739+0000 osd.11 (osd.11) 62 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:49.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:48 smithi166 bash[9446]: cluster 2020-09-08T15:50:47.462838+0000 osd.11 (osd.11) 63 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:49.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:48 smithi166 bash[9446]: cluster 2020-09-08T15:50:47.798446+0000 mgr.x (mgr.24118) 704 : cluster [DBG] pgmap v1001: 17 pgs: 17 active+clean; 1.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 958 op/s 2020-09-08T15:50:49.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:48 smithi166 bash[9446]: cluster 2020-09-08T15:50:47.898774+0000 mon.a (mon.0) 1467 : cluster [DBG] osdmap e382: 12 total, 12 up, 12 in 2020-09-08T15:50:49.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:48 smithi055 bash[9296]: cluster 2020-09-08T15:50:47.462739+0000 osd.11 (osd.11) 62 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:49.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:48 smithi055 bash[9296]: cluster 2020-09-08T15:50:47.462838+0000 osd.11 (osd.11) 63 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:49.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:48 smithi055 bash[9296]: cluster 2020-09-08T15:50:47.798446+0000 mgr.x (mgr.24118) 704 : cluster [DBG] pgmap v1001: 17 pgs: 17 active+clean; 1.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 958 op/s 2020-09-08T15:50:49.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:48 smithi055 bash[9296]: cluster 2020-09-08T15:50:47.898774+0000 mon.a (mon.0) 1467 : cluster [DBG] osdmap e382: 12 total, 12 up, 12 in 2020-09-08T15:50:49.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:50:48 smithi104 bash[22305]: debug 2020-09-08T15:50:48.694+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:49.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:48 smithi104 bash[9203]: cluster 2020-09-08T15:50:47.462739+0000 osd.11 (osd.11) 62 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:49.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:48 smithi104 bash[9203]: cluster 2020-09-08T15:50:47.462838+0000 osd.11 (osd.11) 63 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:49.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:48 smithi104 bash[9203]: cluster 2020-09-08T15:50:47.798446+0000 mgr.x (mgr.24118) 704 : cluster [DBG] pgmap v1001: 17 pgs: 17 active+clean; 1.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 958 op/s 2020-09-08T15:50:49.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:48 smithi104 bash[9203]: cluster 2020-09-08T15:50:47.898774+0000 mon.a (mon.0) 1467 : cluster [DBG] osdmap e382: 12 total, 12 up, 12 in 2020-09-08T15:50:49.362 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:50:49.470 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:49.614 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:49.677 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:50:49 smithi104 bash[22305]: debug 2020-09-08T15:50:49.362+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:49.718 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:49.869 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:50:49.906 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:50:49 smithi055 bash[11956]: debug 2020-09-08T15:50:49.611+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:49.983 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:50.084 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:50:49 smithi104 bash[15273]: debug 2020-09-08T15:50:49.870+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:50.090 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:50.203 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:50.335 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:50:50 smithi055 bash[11956]: debug 2020-09-08T15:50:50.091+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:50.335 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:50:50 smithi055 bash[11956]: debug 2020-09-08T15:50:50.319+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:50.336 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:50.437 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:50.847 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:50:51.001 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:51.023 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:50:51.034 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:50 smithi166 bash[9446]: cluster 2020-09-08T15:50:49.798885+0000 mgr.x (mgr.24118) 705 : cluster [DBG] pgmap v1003: 17 pgs: 17 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 826 op/s 2020-09-08T15:50:51.124 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:51.135 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:50 smithi104 bash[9203]: cluster 2020-09-08T15:50:49.798885+0000 mgr.x (mgr.24118) 705 : cluster [DBG] pgmap v1003: 17 pgs: 17 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 826 op/s 2020-09-08T15:50:51.135 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:50:51 smithi104 bash[17588]: debug 2020-09-08T15:50:50.994+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:51.176 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:50 smithi055 bash[9296]: cluster 2020-09-08T15:50:49.798885+0000 mgr.x (mgr.24118) 705 : cluster [DBG] pgmap v1003: 17 pgs: 17 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 826 op/s 2020-09-08T15:50:51.210 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:50:51.210 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T15:50:51.211 INFO:teuthology.orchestra.run.smithi166.stdout: "complaint_time": 30, 2020-09-08T15:50:51.211 INFO:teuthology.orchestra.run.smithi166.stdout: "num_blocked_ops": 0 2020-09-08T15:50:51.211 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:50:51.743 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:51.849 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:50:52.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:51 smithi166 bash[9446]: cluster 2020-09-08T15:50:50.588092+0000 osd.4 (osd.4) 39 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:52.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:51 smithi166 bash[9446]: cluster 2020-09-08T15:50:50.588176+0000 osd.4 (osd.4) 40 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:52.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:51 smithi166 bash[9446]: cluster 2020-09-08T15:50:50.769380+0000 mon.a (mon.0) 1468 : cluster [DBG] osdmap e383: 12 total, 12 up, 12 in 2020-09-08T15:50:52.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:51 smithi055 bash[9296]: cluster 2020-09-08T15:50:50.588092+0000 osd.4 (osd.4) 39 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:52.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:51 smithi055 bash[9296]: cluster 2020-09-08T15:50:50.588176+0000 osd.4 (osd.4) 40 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:52.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:51 smithi055 bash[9296]: cluster 2020-09-08T15:50:50.769380+0000 mon.a (mon.0) 1468 : cluster [DBG] osdmap e383: 12 total, 12 up, 12 in 2020-09-08T15:50:52.178 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:50:51 smithi104 bash[17588]: debug 2020-09-08T15:50:51.742+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:52.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:51 smithi104 bash[9203]: cluster 2020-09-08T15:50:50.588092+0000 osd.4 (osd.4) 39 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:50:52.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:51 smithi104 bash[9203]: cluster 2020-09-08T15:50:50.588176+0000 osd.4 (osd.4) 40 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:50:52.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:51 smithi104 bash[9203]: cluster 2020-09-08T15:50:50.769380+0000 mon.a (mon.0) 1468 : cluster [DBG] osdmap e383: 12 total, 12 up, 12 in 2020-09-08T15:50:52.235 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_historic_ops 2020-09-08T15:50:52.393 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:50:52.442 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:50:52.543 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:52.656 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:52.770 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:50:52.771 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:52 smithi104 bash[9203]: cluster 2020-09-08T15:50:51.799281+0000 mgr.x (mgr.24118) 706 : cluster [DBG] pgmap v1005: 17 pgs: 17 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 849 op/s 2020-09-08T15:50:52.772 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:50:52 smithi104 bash[19922]: debug 2020-09-08T15:50:52.434+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:52.884 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:50:52 smithi055 bash[11956]: debug 2020-09-08T15:50:52.655+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:52.884 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:52 smithi055 bash[9296]: cluster 2020-09-08T15:50:51.799281+0000 mgr.x (mgr.24118) 706 : cluster [DBG] pgmap v1005: 17 pgs: 17 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 849 op/s 2020-09-08T15:50:52.890 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:50:52.998 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:50:53.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:52 smithi166 bash[9446]: cluster 2020-09-08T15:50:51.799281+0000 mgr.x (mgr.24118) 706 : cluster [DBG] pgmap v1005: 17 pgs: 17 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 849 op/s 2020-09-08T15:50:53.117 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:50:53.163 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:50:52 smithi055 bash[19003]: debug 2020-09-08T15:50:52.883+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:53.164 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:50:53 smithi055 bash[14301]: debug 2020-09-08T15:50:53.115+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:53.222 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:50:53.400 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:50:53.504 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:50:53.680 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:50:53 smithi104 bash[22305]: debug 2020-09-08T15:50:53.398+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:53.760 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:50:53.770 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:50:53 smithi166 bash[16593]: debug 2020-09-08T15:50:53.747+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:53.783 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:50:53.862 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:54.092 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:53 smithi166 bash[9446]: audit 2020-09-08T15:50:53.465196+0000 mon.a (mon.0) 1469 : audit [INF] from='client.? v1:172.21.15.104:0/358210845' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:50:54.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:53 smithi055 bash[9296]: audit 2020-09-08T15:50:53.465196+0000 mon.a (mon.0) 1469 : audit [INF] from='client.? v1:172.21.15.104:0/358210845' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:50:54.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:53 smithi104 bash[9203]: audit 2020-09-08T15:50:53.465196+0000 mon.a (mon.0) 1469 : audit [INF] from='client.? v1:172.21.15.104:0/358210845' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:50:54.985 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:55.087 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:50:55.110 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:54 smithi055 bash[9296]: cluster 2020-09-08T15:50:53.772346+0000 mon.a (mon.0) 1470 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:50:55.110 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:54 smithi055 bash[9296]: audit 2020-09-08T15:50:53.782631+0000 mon.a (mon.0) 1471 : audit [INF] from='client.? v1:172.21.15.104:0/358210845' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:50:55.111 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:54 smithi055 bash[9296]: cluster 2020-09-08T15:50:53.782708+0000 mon.a (mon.0) 1472 : cluster [DBG] osdmap e384: 12 total, 12 up, 12 in 2020-09-08T15:50:55.111 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:54 smithi055 bash[9296]: cluster 2020-09-08T15:50:53.799642+0000 mgr.x (mgr.24118) 707 : cluster [DBG] pgmap v1007: 17 pgs: 17 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T15:50:55.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:54 smithi166 bash[9446]: cluster 2020-09-08T15:50:53.772346+0000 mon.a (mon.0) 1470 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:50:55.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:54 smithi166 bash[9446]: audit 2020-09-08T15:50:53.782631+0000 mon.a (mon.0) 1471 : audit [INF] from='client.? v1:172.21.15.104:0/358210845' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:50:55.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:54 smithi166 bash[9446]: cluster 2020-09-08T15:50:53.782708+0000 mon.a (mon.0) 1472 : cluster [DBG] osdmap e384: 12 total, 12 up, 12 in 2020-09-08T15:50:55.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:54 smithi166 bash[9446]: cluster 2020-09-08T15:50:53.799642+0000 mgr.x (mgr.24118) 707 : cluster [DBG] pgmap v1007: 17 pgs: 17 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T15:50:55.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:54 smithi104 bash[9203]: cluster 2020-09-08T15:50:53.772346+0000 mon.a (mon.0) 1470 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:50:55.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:54 smithi104 bash[9203]: audit 2020-09-08T15:50:53.782631+0000 mon.a (mon.0) 1471 : audit [INF] from='client.? v1:172.21.15.104:0/358210845' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:50:55.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:54 smithi104 bash[9203]: cluster 2020-09-08T15:50:53.782708+0000 mon.a (mon.0) 1472 : cluster [DBG] osdmap e384: 12 total, 12 up, 12 in 2020-09-08T15:50:55.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:54 smithi104 bash[9203]: cluster 2020-09-08T15:50:53.799642+0000 mgr.x (mgr.24118) 707 : cluster [DBG] pgmap v1007: 17 pgs: 17 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T15:50:55.178 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:50:54 smithi104 bash[17588]: debug 2020-09-08T15:50:54.978+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:55.244 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:50:55.346 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:50:55.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:50:55 smithi055 bash[19003]: debug 2020-09-08T15:50:55.234+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:55.920 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:50:56.023 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:50:56.146 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:50:56.251 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:50:56.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:50:55 smithi166 bash[14254]: debug 2020-09-08T15:50:55.919+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:56.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:50:56 smithi055 bash[16646]: debug 2020-09-08T15:50:56.146+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:56.446 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:50:56.550 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:50:56.613 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:50:56 smithi104 bash[22305]: debug 2020-09-08T15:50:56.446+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:56.749 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:50:56.852 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:50:56 smithi166 bash[11928]: debug 2020-09-08T15:50:56.751+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:56.858 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:56 smithi166 bash[9446]: cluster 2020-09-08T15:50:55.800058+0000 mgr.x (mgr.24118) 708 : cluster [DBG] pgmap v1008: 17 pgs: 17 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 902 op/s 2020-09-08T15:50:56.858 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:56 smithi166 bash[9446]: audit 2020-09-08T15:50:56.848776+0000 mon.a (mon.0) 1473 : audit [DBG] from='client.? v1:172.21.15.104:0/578783204' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:50:56.863 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:50:56.881 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:50:56.919 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:50:56.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:56 smithi104 bash[9203]: cluster 2020-09-08T15:50:55.800058+0000 mgr.x (mgr.24118) 708 : cluster [DBG] pgmap v1008: 17 pgs: 17 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 902 op/s 2020-09-08T15:50:56.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:56 smithi104 bash[9203]: audit 2020-09-08T15:50:56.848776+0000 mon.a (mon.0) 1473 : audit [DBG] from='client.? v1:172.21.15.104:0/578783204' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:50:56.958 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:50:56.971 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:50:56.987 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:50:57.033 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:50:57.147 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:57.149 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:50:57.149 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:50:57.149 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:50:57.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:50:56 smithi055 bash[19003]: debug 2020-09-08T15:50:56.970+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:57.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:56 smithi055 bash[9296]: cluster 2020-09-08T15:50:55.800058+0000 mgr.x (mgr.24118) 708 : cluster [DBG] pgmap v1008: 17 pgs: 17 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 902 op/s 2020-09-08T15:50:57.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:56 smithi055 bash[9296]: audit 2020-09-08T15:50:56.848776+0000 mon.a (mon.0) 1473 : audit [DBG] from='client.? v1:172.21.15.104:0/578783204' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:50:57.157 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:50:57.174 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:50:57.175 INFO:teuthology.orchestra.run.smithi166.stdout: "size": 20, 2020-09-08T15:50:57.175 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 600, 2020-09-08T15:50:57.175 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T15:50:57.175 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.176 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:80895 4.6 4:61551036:::benchmark_data_smithi166_32054_object80894:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e332)", 2020-09-08T15:50:57.176 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:00.131140+0000", 2020-09-08T15:50:57.176 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 177.04214917300001, 2020-09-08T15:50:57.176 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19727386199999999, 2020-09-08T15:50:57.176 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:50:57.177 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:50:57.177 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:50:57.177 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:50:57.177 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:50:57.177 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 80895 2020-09-08T15:50:57.178 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.178 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:50:57.178 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.178 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:50:57.178 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.131140+0000", 2020-09-08T15:50:57.179 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:50:57.179 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.179 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.179 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:50:57.179 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.131140+0000", 2020-09-08T15:50:57.180 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999642 2020-09-08T15:50:57.180 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.181 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.181 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:50:57.181 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.131105+0000", 2020-09-08T15:50:57.182 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000120266 2020-09-08T15:50:57.182 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.182 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.183 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:50:57.183 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.131225+0000", 2020-09-08T15:50:57.183 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7156e-05 2020-09-08T15:50:57.183 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.183 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.184 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:50:57.184 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.131242+0000", 2020-09-08T15:50:57.184 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8359999999999997e-06 2020-09-08T15:50:57.185 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.185 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.185 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:50:57.185 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.131248+0000", 2020-09-08T15:50:57.185 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00218724 2020-09-08T15:50:57.185 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.186 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.186 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:50:57.186 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.133435+0000", 2020-09-08T15:50:57.186 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000121471 2020-09-08T15:50:57.186 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.187 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.187 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:50:57.187 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.133556+0000", 2020-09-08T15:50:57.187 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000168309 2020-09-08T15:50:57.187 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.188 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.188 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T15:50:57.188 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.133725+0000", 2020-09-08T15:50:57.188 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0025498769999999999 2020-09-08T15:50:57.189 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.189 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.189 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:50:57.189 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.136275+0000", 2020-09-08T15:50:57.189 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19181920299999999 2020-09-08T15:50:57.190 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.190 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.190 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:50:57.190 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.328094+0000", 2020-09-08T15:50:57.190 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000190719 2020-09-08T15:50:57.191 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.191 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.191 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:50:57.191 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.328285+0000", 2020-09-08T15:50:57.191 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012947999999999999 2020-09-08T15:50:57.191 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.192 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.192 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:50:57.192 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.328414+0000", 2020-09-08T15:50:57.192 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19727386199999999 2020-09-08T15:50:57.193 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.194 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.194 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.194 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.194 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.195 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:80905 4.6 4:6ab9f432:::benchmark_data_smithi166_32054_object80904:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e332)", 2020-09-08T15:50:57.195 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:00.137162+0000", 2020-09-08T15:50:57.195 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 177.036127443, 2020-09-08T15:50:57.195 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19165459300000001, 2020-09-08T15:50:57.195 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:50:57.196 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:50:57.196 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:50:57.196 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:50:57.196 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:50:57.196 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 80905 2020-09-08T15:50:57.197 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.197 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:50:57.197 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.197 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:50:57.197 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.137162+0000", 2020-09-08T15:50:57.198 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:50:57.198 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.198 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.198 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:50:57.198 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.137162+0000", 2020-09-08T15:50:57.199 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999638 2020-09-08T15:50:57.199 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.199 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.199 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:50:57.199 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.137126+0000", 2020-09-08T15:50:57.199 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000118009 2020-09-08T15:50:57.200 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.200 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.200 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:50:57.200 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.137244+0000", 2020-09-08T15:50:57.200 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.5429999999999999e-06 2020-09-08T15:50:57.201 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.201 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.201 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:50:57.201 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.137253+0000", 2020-09-08T15:50:57.201 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.6700000000000002e-06 2020-09-08T15:50:57.202 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.202 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.202 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:50:57.202 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.137258+0000", 2020-09-08T15:50:57.202 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.6660999999999999e-05 2020-09-08T15:50:57.203 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.203 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.203 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:50:57.203 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.137305+0000", 2020-09-08T15:50:57.204 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014171100000000001 2020-09-08T15:50:57.204 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.204 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.204 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:50:57.205 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.137446+0000", 2020-09-08T15:50:57.205 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000143654 2020-09-08T15:50:57.205 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.205 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.206 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T15:50:57.206 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.137590+0000", 2020-09-08T15:50:57.206 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.002511391 2020-09-08T15:50:57.206 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.206 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.207 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:50:57.207 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.140101+0000", 2020-09-08T15:50:57.207 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.188392064 2020-09-08T15:50:57.207 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.207 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.207 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:50:57.208 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.328494+0000", 2020-09-08T15:50:57.208 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016733899999999999 2020-09-08T15:50:57.208 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.208 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.208 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:50:57.209 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.328661+0000", 2020-09-08T15:50:57.209 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000155673 2020-09-08T15:50:57.211 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.211 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.211 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:50:57.211 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.328817+0000", 2020-09-08T15:50:57.211 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19165459300000001 2020-09-08T15:50:57.212 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.212 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.212 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.212 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.212 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.213 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:105392 4.2 4:46272ea7:::benchmark_data_smithi166_32054_object105391:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e339)", 2020-09-08T15:50:57.213 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:17.825550+0000", 2020-09-08T15:50:57.213 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 159.347739741, 2020-09-08T15:50:57.213 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22101335599999999, 2020-09-08T15:50:57.213 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:50:57.214 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:50:57.214 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:50:57.214 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:50:57.215 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:50:57.215 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 105392 2020-09-08T15:50:57.215 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.215 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:50:57.215 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.216 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:50:57.216 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.825550+0000", 2020-09-08T15:50:57.216 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:50:57.216 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.216 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.217 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:50:57.217 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.825550+0000", 2020-09-08T15:50:57.217 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999471 2020-09-08T15:50:57.217 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.217 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.218 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:50:57.218 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.825497+0000", 2020-09-08T15:50:57.218 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000195761 2020-09-08T15:50:57.218 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.219 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.219 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:50:57.219 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.825692+0000", 2020-09-08T15:50:57.220 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.5179999999999999e-05 2020-09-08T15:50:57.220 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.221 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.221 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:50:57.221 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.825717+0000", 2020-09-08T15:50:57.221 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3782e-05 2020-09-08T15:50:57.221 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.222 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.222 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:50:57.222 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.825731+0000", 2020-09-08T15:50:57.222 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8343000000000004e-05 2020-09-08T15:50:57.222 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.223 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.223 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:50:57.223 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.825820+0000", 2020-09-08T15:50:57.223 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018716499999999999 2020-09-08T15:50:57.223 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.223 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.224 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:50:57.224 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.826007+0000", 2020-09-08T15:50:57.224 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015770500000000001 2020-09-08T15:50:57.224 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.224 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.225 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T15:50:57.225 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.826164+0000", 2020-09-08T15:50:57.225 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0036108350000000002 2020-09-08T15:50:57.225 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.225 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.226 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:50:57.226 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.829775+0000", 2020-09-08T15:50:57.227 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.216637368 2020-09-08T15:50:57.227 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.227 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.227 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:50:57.227 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.046413+0000", 2020-09-08T15:50:57.227 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.3028000000000005e-05 2020-09-08T15:50:57.228 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.228 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.228 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:50:57.228 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.046496+0000", 2020-09-08T15:50:57.229 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.7298999999999994e-05 2020-09-08T15:50:57.229 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.229 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.229 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:50:57.229 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.046563+0000", 2020-09-08T15:50:57.230 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22101335599999999 2020-09-08T15:50:57.236 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.236 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.237 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.237 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.237 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.237 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:105410 4.2 4:447ae90f:::benchmark_data_smithi166_32054_object105409:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e339)", 2020-09-08T15:50:57.238 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:17.840593+0000", 2020-09-08T15:50:57.239 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 159.332696298, 2020-09-08T15:50:57.239 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21007357300000001, 2020-09-08T15:50:57.239 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:50:57.239 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:50:57.239 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:50:57.240 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:50:57.240 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:50:57.240 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 105410 2020-09-08T15:50:57.240 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.240 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:50:57.241 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.242 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:50:57.242 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.840593+0000", 2020-09-08T15:50:57.243 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:50:57.243 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.243 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.243 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:50:57.243 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.840593+0000", 2020-09-08T15:50:57.244 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999776 2020-09-08T15:50:57.244 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.244 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.244 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:50:57.244 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.840571+0000", 2020-09-08T15:50:57.245 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7286999999999994e-05 2020-09-08T15:50:57.245 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.245 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.245 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:50:57.245 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.840648+0000", 2020-09-08T15:50:57.246 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3988999999999999e-05 2020-09-08T15:50:57.246 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.246 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.246 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:50:57.247 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.840662+0000", 2020-09-08T15:50:57.247 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.8760000000000004e-06 2020-09-08T15:50:57.247 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.248 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.248 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:50:57.248 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.840667+0000", 2020-09-08T15:50:57.249 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0359999999999999e-05 2020-09-08T15:50:57.249 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.249 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.249 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:50:57.250 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.840717+0000", 2020-09-08T15:50:57.250 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000119008 2020-09-08T15:50:57.250 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.250 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.250 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:50:57.251 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.840836+0000", 2020-09-08T15:50:57.251 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018360699999999999 2020-09-08T15:50:57.251 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.251 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.252 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T15:50:57.252 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.841020+0000", 2020-09-08T15:50:57.252 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0075218459999999996 2020-09-08T15:50:57.252 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.252 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.253 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:50:57.253 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.848541+0000", 2020-09-08T15:50:57.253 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20198524300000001 2020-09-08T15:50:57.253 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.253 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.254 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:50:57.254 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.050527+0000", 2020-09-08T15:50:57.254 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.8275000000000005e-05 2020-09-08T15:50:57.254 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.254 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.255 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:50:57.255 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.050605+0000", 2020-09-08T15:50:57.255 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.1641000000000002e-05 2020-09-08T15:50:57.255 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.255 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.255 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:50:57.256 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.050667+0000", 2020-09-08T15:50:57.256 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21007357300000001 2020-09-08T15:50:57.256 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.256 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.256 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.257 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.257 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.257 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:105411 4.2 4:467d7578:::benchmark_data_smithi166_32054_object105410:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e339)", 2020-09-08T15:50:57.258 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:17.842458+0000", 2020-09-08T15:50:57.258 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 159.33083140799999, 2020-09-08T15:50:57.258 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20848314300000001, 2020-09-08T15:50:57.258 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:50:57.259 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:50:57.259 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:50:57.260 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:50:57.260 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:50:57.260 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 105411 2020-09-08T15:50:57.260 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.261 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:50:57.261 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.261 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:50:57.261 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.842458+0000", 2020-09-08T15:50:57.261 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:50:57.262 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.262 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.262 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:50:57.262 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.842458+0000", 2020-09-08T15:50:57.263 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999642 2020-09-08T15:50:57.263 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.263 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.263 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:50:57.263 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.842422+0000", 2020-09-08T15:50:57.264 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000188152 2020-09-08T15:50:57.264 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.264 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.264 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:50:57.264 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.842610+0000", 2020-09-08T15:50:57.265 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6612999999999998e-05 2020-09-08T15:50:57.265 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.265 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.265 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:50:57.265 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.842627+0000", 2020-09-08T15:50:57.266 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.952e-06 2020-09-08T15:50:57.266 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.266 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.266 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:50:57.266 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.842636+0000", 2020-09-08T15:50:57.267 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.001858113 2020-09-08T15:50:57.267 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.267 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.267 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:50:57.267 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.844494+0000", 2020-09-08T15:50:57.267 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018164099999999999 2020-09-08T15:50:57.268 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.268 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.268 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:50:57.268 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.844676+0000", 2020-09-08T15:50:57.268 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000207813 2020-09-08T15:50:57.269 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.269 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.269 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T15:50:57.270 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.844884+0000", 2020-09-08T15:50:57.270 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.005358565 2020-09-08T15:50:57.270 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.271 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.271 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:50:57.271 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.850242+0000", 2020-09-08T15:50:57.271 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20050157299999999 2020-09-08T15:50:57.271 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.272 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.272 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:50:57.272 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.050744+0000", 2020-09-08T15:50:57.272 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000109436 2020-09-08T15:50:57.273 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.273 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.273 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:50:57.273 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.050853+0000", 2020-09-08T15:50:57.274 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.7960000000000005e-05 2020-09-08T15:50:57.274 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.274 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.274 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:50:57.274 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.050941+0000", 2020-09-08T15:50:57.275 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20848314300000001 2020-09-08T15:50:57.275 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.275 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.275 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.275 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.276 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.276 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:105425 4.2 4:4ea40f39:::benchmark_data_smithi166_32054_object105424:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e339)", 2020-09-08T15:50:57.276 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:17.853184+0000", 2020-09-08T15:50:57.276 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 159.32010583100001, 2020-09-08T15:50:57.276 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20860477299999999, 2020-09-08T15:50:57.277 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:50:57.277 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:50:57.277 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:50:57.277 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:50:57.277 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:50:57.278 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 105425 2020-09-08T15:50:57.278 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.278 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:50:57.278 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.279 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:50:57.279 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.853184+0000", 2020-09-08T15:50:57.279 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:50:57.279 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.279 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.279 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:50:57.280 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.853184+0000", 2020-09-08T15:50:57.280 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999599 2020-09-08T15:50:57.282 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.282 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.282 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:50:57.282 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.853143+0000", 2020-09-08T15:50:57.283 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000154802 2020-09-08T15:50:57.283 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.283 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.283 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:50:57.283 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.853298+0000", 2020-09-08T15:50:57.284 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.8411999999999998e-05 2020-09-08T15:50:57.284 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.284 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.284 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:50:57.284 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.853327+0000", 2020-09-08T15:50:57.285 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.0049999999999998e-06 2020-09-08T15:50:57.285 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.286 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.286 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:50:57.286 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.853334+0000", 2020-09-08T15:50:57.286 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012921300000000001 2020-09-08T15:50:57.286 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.287 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.287 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:50:57.287 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.853463+0000", 2020-09-08T15:50:57.287 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00022158200000000001 2020-09-08T15:50:57.287 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.289 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.289 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:50:57.289 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.853684+0000", 2020-09-08T15:50:57.289 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018970099999999999 2020-09-08T15:50:57.290 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.290 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.290 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T15:50:57.290 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.853874+0000", 2020-09-08T15:50:57.290 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0071453180000000003 2020-09-08T15:50:57.290 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.291 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.291 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:50:57.291 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.861019+0000", 2020-09-08T15:50:57.291 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20058150399999999 2020-09-08T15:50:57.291 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.292 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.292 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:50:57.292 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.061601+0000", 2020-09-08T15:50:57.292 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000113155 2020-09-08T15:50:57.292 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.293 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.293 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:50:57.293 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.061714+0000", 2020-09-08T15:50:57.293 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.428e-05 2020-09-08T15:50:57.293 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.294 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.294 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:50:57.294 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.061788+0000", 2020-09-08T15:50:57.294 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20860477299999999 2020-09-08T15:50:57.294 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.295 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.295 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.295 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.295 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.295 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:105429 4.2 4:497b055c:::benchmark_data_smithi166_32054_object105428:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e339)", 2020-09-08T15:50:57.296 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:17.855411+0000", 2020-09-08T15:50:57.296 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 159.317878369, 2020-09-08T15:50:57.296 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20664822899999999, 2020-09-08T15:50:57.296 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:50:57.296 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:50:57.296 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:50:57.297 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:50:57.297 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:50:57.297 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 105429 2020-09-08T15:50:57.297 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.297 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:50:57.298 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.298 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:50:57.298 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.855411+0000", 2020-09-08T15:50:57.298 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:50:57.299 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.299 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.299 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:50:57.299 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.855411+0000", 2020-09-08T15:50:57.300 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999537 2020-09-08T15:50:57.300 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.300 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.300 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:50:57.300 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.855365+0000", 2020-09-08T15:50:57.301 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014474999999999999 2020-09-08T15:50:57.301 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.301 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.301 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:50:57.301 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.855509+0000", 2020-09-08T15:50:57.302 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5058000000000001e-05 2020-09-08T15:50:57.302 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.302 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.302 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:50:57.302 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.855525+0000", 2020-09-08T15:50:57.303 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.5780000000000001e-06 2020-09-08T15:50:57.303 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.303 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.303 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:50:57.303 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.855532+0000", 2020-09-08T15:50:57.304 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0022733459999999999 2020-09-08T15:50:57.304 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.304 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.304 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:50:57.304 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.857805+0000", 2020-09-08T15:50:57.304 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000155773 2020-09-08T15:50:57.305 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.305 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.305 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:50:57.305 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.857961+0000", 2020-09-08T15:50:57.305 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00020873300000000001 2020-09-08T15:50:57.306 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.306 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.306 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T15:50:57.306 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.858170+0000", 2020-09-08T15:50:57.306 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0030094950000000001 2020-09-08T15:50:57.307 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.307 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.307 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:50:57.307 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.861179+0000", 2020-09-08T15:50:57.307 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.200696597 2020-09-08T15:50:57.307 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.308 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.308 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:50:57.308 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.061876+0000", 2020-09-08T15:50:57.308 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000107132 2020-09-08T15:50:57.308 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.309 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.309 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:50:57.309 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.061983+0000", 2020-09-08T15:50:57.310 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.6056999999999994e-05 2020-09-08T15:50:57.310 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.310 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.310 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:50:57.310 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.062059+0000", 2020-09-08T15:50:57.311 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20664822899999999 2020-09-08T15:50:57.311 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.311 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.311 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.311 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.312 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.312 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:105441 4.2 4:446ca20e:::benchmark_data_smithi166_32054_object105440:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e339)", 2020-09-08T15:50:57.312 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:17.866739+0000", 2020-09-08T15:50:57.312 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 159.30655063899999, 2020-09-08T15:50:57.312 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20372923600000001, 2020-09-08T15:50:57.313 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:50:57.313 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:50:57.313 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:50:57.313 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:50:57.313 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:50:57.313 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 105441 2020-09-08T15:50:57.314 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.314 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:50:57.314 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.314 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:50:57.314 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.866739+0000", 2020-09-08T15:50:57.315 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:50:57.315 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.315 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.315 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:50:57.315 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.866739+0000", 2020-09-08T15:50:57.316 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999623 2020-09-08T15:50:57.316 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.316 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.316 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:50:57.316 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.866701+0000", 2020-09-08T15:50:57.317 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7627000000000006e-05 2020-09-08T15:50:57.317 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.317 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.317 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:50:57.317 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.866798+0000", 2020-09-08T15:50:57.317 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.0639999999999999e-05 2020-09-08T15:50:57.318 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.318 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.318 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:50:57.318 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.866819+0000", 2020-09-08T15:50:57.318 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.6879999999999996e-06 2020-09-08T15:50:57.319 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.319 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.319 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:50:57.320 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.866825+0000", 2020-09-08T15:50:57.320 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.3525000000000001e-05 2020-09-08T15:50:57.320 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.320 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.320 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:50:57.321 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.866868+0000", 2020-09-08T15:50:57.321 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000113715 2020-09-08T15:50:57.321 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.322 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.322 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:50:57.322 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.866982+0000", 2020-09-08T15:50:57.322 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000127448 2020-09-08T15:50:57.322 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.323 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.323 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T15:50:57.323 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.867109+0000", 2020-09-08T15:50:57.323 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0075386200000000002 2020-09-08T15:50:57.323 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.323 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.324 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:50:57.324 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.874648+0000", 2020-09-08T15:50:57.324 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19560765999999999 2020-09-08T15:50:57.324 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.324 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.325 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:50:57.325 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.070256+0000", 2020-09-08T15:50:57.325 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012985199999999999 2020-09-08T15:50:57.325 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.325 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.326 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:50:57.326 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.070386+0000", 2020-09-08T15:50:57.326 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.2356999999999998e-05 2020-09-08T15:50:57.326 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.326 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.327 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:50:57.327 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.070468+0000", 2020-09-08T15:50:57.327 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20372923600000001 2020-09-08T15:50:57.327 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.327 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.327 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.328 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.328 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.328 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:105477 4.2 4:45e453c7:::benchmark_data_smithi166_32054_object105476:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e339)", 2020-09-08T15:50:57.328 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:17.898359+0000", 2020-09-08T15:50:57.328 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 159.27493032800001, 2020-09-08T15:50:57.329 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17236843600000001, 2020-09-08T15:50:57.329 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:50:57.329 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:50:57.329 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:50:57.330 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:50:57.330 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:50:57.331 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 105477 2020-09-08T15:50:57.331 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.331 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:50:57.331 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.331 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:50:57.332 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.898359+0000", 2020-09-08T15:50:57.332 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:50:57.332 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.332 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.332 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:50:57.333 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.898359+0000", 2020-09-08T15:50:57.333 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999642 2020-09-08T15:50:57.333 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.333 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.333 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:50:57.334 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.898323+0000", 2020-09-08T15:50:57.334 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014085399999999999 2020-09-08T15:50:57.334 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.334 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.334 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:50:57.335 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.898464+0000", 2020-09-08T15:50:57.335 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0378e-05 2020-09-08T15:50:57.335 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.335 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.335 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:50:57.335 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.898475+0000", 2020-09-08T15:50:57.336 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.5629999999999998e-06 2020-09-08T15:50:57.336 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.336 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.336 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:50:57.336 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.898484+0000", 2020-09-08T15:50:57.337 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.2391999999999994e-05 2020-09-08T15:50:57.337 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.337 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.337 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:50:57.337 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.898556+0000", 2020-09-08T15:50:57.338 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00021068200000000001 2020-09-08T15:50:57.338 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.338 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.338 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:50:57.338 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.898767+0000", 2020-09-08T15:50:57.339 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00023829799999999999 2020-09-08T15:50:57.339 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.339 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.339 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T15:50:57.339 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.899005+0000", 2020-09-08T15:50:57.340 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00354719 2020-09-08T15:50:57.340 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.340 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.340 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:50:57.341 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.902553+0000", 2020-09-08T15:50:57.341 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16799788399999999 2020-09-08T15:50:57.341 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.341 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.342 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:50:57.342 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.070551+0000", 2020-09-08T15:50:57.342 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000104671 2020-09-08T15:50:57.342 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.342 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.343 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:50:57.343 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.070655+0000", 2020-09-08T15:50:57.343 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.2293999999999996e-05 2020-09-08T15:50:57.343 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.344 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.344 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:50:57.344 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.070727+0000", 2020-09-08T15:50:57.344 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17236843600000001 2020-09-08T15:50:57.344 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.344 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.345 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.345 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.345 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.345 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:105488 4.2 4:4e1f29c5:::benchmark_data_smithi166_32054_object105487:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e339)", 2020-09-08T15:50:57.345 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:17.906472+0000", 2020-09-08T15:50:57.346 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 159.266817145, 2020-09-08T15:50:57.346 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.164516882, 2020-09-08T15:50:57.346 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:50:57.346 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:50:57.346 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:50:57.347 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:50:57.347 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:50:57.347 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 105488 2020-09-08T15:50:57.347 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.347 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:50:57.348 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.348 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:50:57.348 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.906472+0000", 2020-09-08T15:50:57.348 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:50:57.348 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.348 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.349 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:50:57.349 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.906472+0000", 2020-09-08T15:50:57.349 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999576 2020-09-08T15:50:57.349 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.350 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.350 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:50:57.350 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.906430+0000", 2020-09-08T15:50:57.350 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013662400000000001 2020-09-08T15:50:57.351 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.351 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.351 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:50:57.351 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.906566+0000", 2020-09-08T15:50:57.352 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.959e-05 2020-09-08T15:50:57.352 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.352 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.352 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:50:57.352 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.906586+0000", 2020-09-08T15:50:57.353 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0196e-05 2020-09-08T15:50:57.353 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.353 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.353 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:50:57.353 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.906596+0000", 2020-09-08T15:50:57.354 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.2305000000000007e-05 2020-09-08T15:50:57.354 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.354 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.354 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:50:57.354 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.906668+0000", 2020-09-08T15:50:57.355 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000190891 2020-09-08T15:50:57.355 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.355 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.355 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:50:57.355 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.906859+0000", 2020-09-08T15:50:57.356 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00021537699999999999 2020-09-08T15:50:57.356 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.356 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.356 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T15:50:57.356 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.907075+0000", 2020-09-08T15:50:57.357 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0044540389999999999 2020-09-08T15:50:57.357 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.357 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.357 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:50:57.357 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.911529+0000", 2020-09-08T15:50:57.357 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.159265199 2020-09-08T15:50:57.358 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.358 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.358 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:50:57.358 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.070794+0000", 2020-09-08T15:50:57.358 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.4635000000000004e-05 2020-09-08T15:50:57.359 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.359 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.359 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:50:57.359 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.070888+0000", 2020-09-08T15:50:57.359 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010061299999999999 2020-09-08T15:50:57.360 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.360 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.360 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:50:57.360 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.070989+0000", 2020-09-08T15:50:57.360 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.164516882 2020-09-08T15:50:57.360 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.361 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.361 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.362 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.362 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.362 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:105499 4.2 4:456654c1:::benchmark_data_smithi166_32054_object105498:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e339)", 2020-09-08T15:50:57.362 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:17.919016+0000", 2020-09-08T15:50:57.362 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 159.25427288700001, 2020-09-08T15:50:57.363 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15227105099999999, 2020-09-08T15:50:57.363 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:50:57.363 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:50:57.363 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:50:57.363 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:50:57.364 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:50:57.364 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 105499 2020-09-08T15:50:57.364 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.364 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:50:57.364 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.365 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:50:57.365 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.919016+0000", 2020-09-08T15:50:57.365 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:50:57.365 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.366 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.366 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:50:57.366 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.919016+0000", 2020-09-08T15:50:57.366 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999714 2020-09-08T15:50:57.366 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.367 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.367 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:50:57.367 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.918988+0000", 2020-09-08T15:50:57.367 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011919899999999999 2020-09-08T15:50:57.367 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.368 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.368 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:50:57.368 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.919107+0000", 2020-09-08T15:50:57.369 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0414999999999999e-05 2020-09-08T15:50:57.369 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.369 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.369 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:50:57.369 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.919117+0000", 2020-09-08T15:50:57.370 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.5300000000000004e-06 2020-09-08T15:50:57.370 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.370 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.370 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:50:57.370 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.919123+0000", 2020-09-08T15:50:57.371 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.4573000000000001e-05 2020-09-08T15:50:57.371 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.371 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.372 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:50:57.372 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.919177+0000", 2020-09-08T15:50:57.372 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000180955 2020-09-08T15:50:57.372 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.372 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.373 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:50:57.373 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.919358+0000", 2020-09-08T15:50:57.373 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00024404099999999999 2020-09-08T15:50:57.373 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.373 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.374 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T15:50:57.374 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.919602+0000", 2020-09-08T15:50:57.374 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0045129819999999996 2020-09-08T15:50:57.374 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.374 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.375 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:50:57.375 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.924115+0000", 2020-09-08T15:50:57.375 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14695534399999999 2020-09-08T15:50:57.375 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.375 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.376 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:50:57.376 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.071071+0000", 2020-09-08T15:50:57.376 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000114757 2020-09-08T15:50:57.376 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.376 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.376 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:50:57.377 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.071186+0000", 2020-09-08T15:50:57.377 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000101962 2020-09-08T15:50:57.377 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.377 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.377 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:50:57.378 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.071288+0000", 2020-09-08T15:50:57.378 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15227105099999999 2020-09-08T15:50:57.378 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.378 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.378 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.379 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.379 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.379 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:113600 4.6 4:603824bc:::benchmark_data_smithi166_32054_object113599:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e341)", 2020-09-08T15:50:57.379 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:24.644594+0000", 2020-09-08T15:50:57.380 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 152.528695379, 2020-09-08T15:50:57.380 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.188084585, 2020-09-08T15:50:57.380 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:50:57.380 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:50:57.380 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:50:57.381 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:50:57.381 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:50:57.381 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 113600 2020-09-08T15:50:57.381 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.381 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:50:57.382 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.382 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:50:57.383 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.644594+0000", 2020-09-08T15:50:57.383 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:50:57.383 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.383 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.383 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:50:57.384 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.644594+0000", 2020-09-08T15:50:57.384 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999604 2020-09-08T15:50:57.384 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.384 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.384 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:50:57.385 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.644555+0000", 2020-09-08T15:50:57.385 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000131338 2020-09-08T15:50:57.385 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.385 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.385 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:50:57.386 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.644686+0000", 2020-09-08T15:50:57.386 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4857e-05 2020-09-08T15:50:57.386 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.386 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.386 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:50:57.387 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.644701+0000", 2020-09-08T15:50:57.387 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6440000000000003e-06 2020-09-08T15:50:57.387 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.387 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.388 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:50:57.388 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.644707+0000", 2020-09-08T15:50:57.388 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9605999999999999e-05 2020-09-08T15:50:57.388 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.388 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.388 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:50:57.389 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.644757+0000", 2020-09-08T15:50:57.389 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000163812 2020-09-08T15:50:57.390 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.390 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.390 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:50:57.390 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.644921+0000", 2020-09-08T15:50:57.390 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019153400000000001 2020-09-08T15:50:57.391 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.391 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.391 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T15:50:57.391 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.645112+0000", 2020-09-08T15:50:57.391 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0039535739999999996 2020-09-08T15:50:57.392 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.392 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.392 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:50:57.393 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.649066+0000", 2020-09-08T15:50:57.393 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18316455400000001 2020-09-08T15:50:57.393 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.393 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.393 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:50:57.394 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.832231+0000", 2020-09-08T15:50:57.394 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00024548799999999998 2020-09-08T15:50:57.394 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.394 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.394 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:50:57.395 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.832476+0000", 2020-09-08T15:50:57.395 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000202566 2020-09-08T15:50:57.395 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.395 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.395 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:50:57.396 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.832679+0000", 2020-09-08T15:50:57.396 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.188084585 2020-09-08T15:50:57.396 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.396 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.396 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.397 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.397 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.397 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:116122 4.6 4:60608c08:::benchmark_data_smithi166_32054_object116121:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e343)", 2020-09-08T15:50:57.397 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:27.398569+0000", 2020-09-08T15:50:57.397 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 149.774720323, 2020-09-08T15:50:57.398 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.179969832, 2020-09-08T15:50:57.398 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:50:57.398 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:50:57.398 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:50:57.398 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:50:57.398 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:50:57.399 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 116122 2020-09-08T15:50:57.399 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.399 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:50:57.399 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.399 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:50:57.400 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.398569+0000", 2020-09-08T15:50:57.400 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:50:57.400 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.400 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.400 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:50:57.401 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.398569+0000", 2020-09-08T15:50:57.401 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999776 2020-09-08T15:50:57.401 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.401 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.401 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:50:57.402 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.398546+0000", 2020-09-08T15:50:57.402 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011290699999999999 2020-09-08T15:50:57.402 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.402 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.402 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:50:57.404 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.398659+0000", 2020-09-08T15:50:57.404 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5480000000000001e-05 2020-09-08T15:50:57.404 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.404 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.404 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:50:57.405 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.398675+0000", 2020-09-08T15:50:57.405 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.8210000000000004e-06 2020-09-08T15:50:57.405 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.405 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.405 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:50:57.406 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.398682+0000", 2020-09-08T15:50:57.406 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.4580999999999997e-05 2020-09-08T15:50:57.406 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.406 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.407 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:50:57.407 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.398746+0000", 2020-09-08T15:50:57.407 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018969700000000001 2020-09-08T15:50:57.407 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.407 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.408 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:50:57.408 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.398936+0000", 2020-09-08T15:50:57.408 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00021333699999999999 2020-09-08T15:50:57.408 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.408 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.409 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T15:50:57.409 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.399149+0000", 2020-09-08T15:50:57.409 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0037601660000000001 2020-09-08T15:50:57.409 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.409 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.410 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:50:57.410 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.402909+0000", 2020-09-08T15:50:57.410 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.175362343 2020-09-08T15:50:57.410 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.410 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.411 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:50:57.411 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.578272+0000", 2020-09-08T15:50:57.411 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014603899999999999 2020-09-08T15:50:57.411 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.412 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.412 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:50:57.412 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.578418+0000", 2020-09-08T15:50:57.412 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000121057 2020-09-08T15:50:57.412 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.413 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.413 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:50:57.413 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.578539+0000", 2020-09-08T15:50:57.413 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.179969832 2020-09-08T15:50:57.413 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.414 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.414 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.414 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.414 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.414 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:142001 4.2 4:435b9d62:::benchmark_data_smithi166_32054_object142000:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e350)", 2020-09-08T15:50:57.415 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:53.091577+0000", 2020-09-08T15:50:57.415 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 124.081712369, 2020-09-08T15:50:57.415 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16663236100000001, 2020-09-08T15:50:57.415 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:50:57.416 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:50:57.416 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:50:57.416 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:50:57.416 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:50:57.416 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 142001 2020-09-08T15:50:57.417 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.417 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:50:57.417 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.417 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:50:57.417 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.091577+0000", 2020-09-08T15:50:57.418 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:50:57.418 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.418 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.418 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:50:57.419 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.091577+0000", 2020-09-08T15:50:57.419 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999666 2020-09-08T15:50:57.419 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.419 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.419 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:50:57.420 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.091543+0000", 2020-09-08T15:50:57.420 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010740599999999999 2020-09-08T15:50:57.420 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.420 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.420 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:50:57.420 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.091651+0000", 2020-09-08T15:50:57.421 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3954000000000001e-05 2020-09-08T15:50:57.421 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.421 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.421 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:50:57.421 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.091665+0000", 2020-09-08T15:50:57.422 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5270000000000004e-06 2020-09-08T15:50:57.422 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.422 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.423 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:50:57.423 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.091671+0000", 2020-09-08T15:50:57.423 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.3033000000000004e-05 2020-09-08T15:50:57.423 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.423 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.424 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:50:57.424 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.091734+0000", 2020-09-08T15:50:57.424 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000143027 2020-09-08T15:50:57.424 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.424 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.425 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:50:57.425 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.091877+0000", 2020-09-08T15:50:57.425 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15638299899999999 2020-09-08T15:50:57.425 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.425 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.426 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T15:50:57.426 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.248260+0000", 2020-09-08T15:50:57.426 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0096406410000000001 2020-09-08T15:50:57.426 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.426 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.426 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:50:57.427 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.257901+0000", 2020-09-08T15:50:57.427 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.0284000000000005e-05 2020-09-08T15:50:57.427 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.427 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.427 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:50:57.428 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.257991+0000", 2020-09-08T15:50:57.428 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000128264 2020-09-08T15:50:57.428 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.428 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.428 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:50:57.428 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.258120+0000", 2020-09-08T15:50:57.429 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.9784999999999995e-05 2020-09-08T15:50:57.429 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.429 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.429 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:50:57.430 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.258209+0000", 2020-09-08T15:50:57.430 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16663236100000001 2020-09-08T15:50:57.430 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.430 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.430 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.430 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.431 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.431 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:175094 4.6 4:678f16b9:::benchmark_data_smithi166_32054_object32952:head [delete] snapc 0=[] ondisk+write+known_if_redirected e359)", 2020-09-08T15:50:57.431 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:49:27.212949+0000", 2020-09-08T15:50:57.432 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 89.960339998999999, 2020-09-08T15:50:57.432 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15852648599999999, 2020-09-08T15:50:57.432 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:50:57.432 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:50:57.432 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:50:57.433 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:50:57.433 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:50:57.433 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 175094 2020-09-08T15:50:57.433 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.433 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:50:57.434 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.434 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:50:57.434 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:27.212949+0000", 2020-09-08T15:50:57.434 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:50:57.435 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.435 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.435 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:50:57.435 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:27.212949+0000", 2020-09-08T15:50:57.435 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.999948 2020-09-08T15:50:57.436 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.436 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.436 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:50:57.436 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:27.212898+0000", 2020-09-08T15:50:57.436 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000115402 2020-09-08T15:50:57.437 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.437 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.437 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:50:57.437 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:27.213013+0000", 2020-09-08T15:50:57.437 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.8936e-05 2020-09-08T15:50:57.438 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.438 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.438 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:50:57.438 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:27.213032+0000", 2020-09-08T15:50:57.438 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.687e-06 2020-09-08T15:50:57.438 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.439 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.439 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:50:57.439 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:27.213041+0000", 2020-09-08T15:50:57.439 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.4260999999999998e-05 2020-09-08T15:50:57.439 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.440 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.440 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:50:57.440 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:27.213135+0000", 2020-09-08T15:50:57.440 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000176398 2020-09-08T15:50:57.440 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.441 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.441 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:50:57.441 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:27.213311+0000", 2020-09-08T15:50:57.441 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000182931 2020-09-08T15:50:57.442 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.442 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.442 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T15:50:57.442 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:27.213494+0000", 2020-09-08T15:50:57.443 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15151482899999999 2020-09-08T15:50:57.443 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.443 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.443 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:50:57.443 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:27.365009+0000", 2020-09-08T15:50:57.443 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0062708299999999998 2020-09-08T15:50:57.444 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.444 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.444 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:50:57.444 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:27.371280+0000", 2020-09-08T15:50:57.444 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.9295000000000004e-05 2020-09-08T15:50:57.445 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.445 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.445 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:50:57.445 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:27.371379+0000", 2020-09-08T15:50:57.445 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.6663999999999995e-05 2020-09-08T15:50:57.446 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.446 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.446 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:50:57.446 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:27.371476+0000", 2020-09-08T15:50:57.446 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15852648599999999 2020-09-08T15:50:57.447 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.447 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.447 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.447 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.447 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.447 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:216416 4.6 4:6e40413f:::benchmark_data_smithi166_32054_object74274:head [delete] snapc 0=[] ondisk+write+known_if_redirected e369)", 2020-09-08T15:50:57.448 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:50:08.781007+0000", 2020-09-08T15:50:57.448 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 48.392282682999998, 2020-09-08T15:50:57.448 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15034505300000001, 2020-09-08T15:50:57.448 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:50:57.448 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:50:57.449 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:50:57.449 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:50:57.449 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:50:57.449 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 216416 2020-09-08T15:50:57.450 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.450 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:50:57.450 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.451 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:50:57.451 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.781007+0000", 2020-09-08T15:50:57.451 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:50:57.451 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.452 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.452 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:50:57.452 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.781007+0000", 2020-09-08T15:50:57.452 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999638 2020-09-08T15:50:57.452 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.452 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.453 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:50:57.453 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.780970+0000", 2020-09-08T15:50:57.453 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.1135000000000002e-05 2020-09-08T15:50:57.453 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.453 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.454 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:50:57.454 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.781051+0000", 2020-09-08T15:50:57.454 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3388e-05 2020-09-08T15:50:57.454 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.454 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.455 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:50:57.455 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.781065+0000", 2020-09-08T15:50:57.455 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.964e-06 2020-09-08T15:50:57.455 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.455 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.455 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:50:57.456 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.781071+0000", 2020-09-08T15:50:57.456 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0540000000000001e-05 2020-09-08T15:50:57.456 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.456 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.456 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:50:57.457 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.781121+0000", 2020-09-08T15:50:57.457 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.5316999999999996e-05 2020-09-08T15:50:57.457 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.457 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.457 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:50:57.458 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.781207+0000", 2020-09-08T15:50:57.458 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.2248000000000003e-05 2020-09-08T15:50:57.458 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.458 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.458 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T15:50:57.459 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.781289+0000", 2020-09-08T15:50:57.459 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0097080389999999999 2020-09-08T15:50:57.459 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.459 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.460 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:50:57.460 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.790997+0000", 2020-09-08T15:50:57.460 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14008590000000001 2020-09-08T15:50:57.460 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.460 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.461 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:50:57.461 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.931083+0000", 2020-09-08T15:50:57.461 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014620799999999999 2020-09-08T15:50:57.461 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.462 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.462 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:50:57.462 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.931229+0000", 2020-09-08T15:50:57.462 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000122784 2020-09-08T15:50:57.462 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.462 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.463 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:50:57.463 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.931352+0000", 2020-09-08T15:50:57.463 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15034505300000001 2020-09-08T15:50:57.463 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.463 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.464 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.464 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.464 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.464 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:216421 4.6 4:65f0eb6e:::benchmark_data_smithi166_32054_object74279:head [delete] snapc 0=[] ondisk+write+known_if_redirected e369)", 2020-09-08T15:50:57.464 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:50:08.789048+0000", 2020-09-08T15:50:57.465 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 48.384241779, 2020-09-08T15:50:57.465 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.159611907, 2020-09-08T15:50:57.465 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:50:57.465 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:50:57.465 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:50:57.466 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:50:57.466 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:50:57.466 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 216421 2020-09-08T15:50:57.466 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.466 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:50:57.467 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.467 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:50:57.467 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.789048+0000", 2020-09-08T15:50:57.467 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:50:57.467 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.468 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.468 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:50:57.468 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.789048+0000", 2020-09-08T15:50:57.468 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999552 2020-09-08T15:50:57.468 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.469 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.469 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:50:57.469 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.789003+0000", 2020-09-08T15:50:57.469 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.4965999999999998e-05 2020-09-08T15:50:57.469 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.470 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.470 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:50:57.470 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.789098+0000", 2020-09-08T15:50:57.470 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4535000000000001e-05 2020-09-08T15:50:57.470 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.470 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.471 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:50:57.471 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.789112+0000", 2020-09-08T15:50:57.471 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.2079999999999997e-06 2020-09-08T15:50:57.471 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.471 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.472 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:50:57.472 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.789119+0000", 2020-09-08T15:50:57.472 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0031527579999999999 2020-09-08T15:50:57.472 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.472 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.473 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:50:57.473 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.792271+0000", 2020-09-08T15:50:57.473 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.3747999999999998e-05 2020-09-08T15:50:57.473 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.473 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.474 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:50:57.474 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.792355+0000", 2020-09-08T15:50:57.474 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.0872999999999996e-05 2020-09-08T15:50:57.474 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.474 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.474 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T15:50:57.475 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.792436+0000", 2020-09-08T15:50:57.475 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.13926424800000001 2020-09-08T15:50:57.475 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.475 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.475 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:50:57.476 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.931700+0000", 2020-09-08T15:50:57.476 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.016819555999999999 2020-09-08T15:50:57.476 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.476 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.476 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:50:57.477 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.948520+0000", 2020-09-08T15:50:57.477 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.3306999999999994e-05 2020-09-08T15:50:57.477 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.477 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.478 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:50:57.478 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.948603+0000", 2020-09-08T15:50:57.478 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.6427e-05 2020-09-08T15:50:57.478 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.478 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.478 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:50:57.479 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.948659+0000", 2020-09-08T15:50:57.479 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.159611907 2020-09-08T15:50:57.479 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.479 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.479 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.480 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.480 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.480 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:255309 4.6 4:68c19ab3:::benchmark_data_smithi166_32054_object113167:head [delete] snapc 0=[] ondisk+write+known_if_redirected e382)", 2020-09-08T15:50:57.480 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:50:48.065335+0000", 2020-09-08T15:50:57.480 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 9.1079548280000004, 2020-09-08T15:50:57.481 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16226527399999999, 2020-09-08T15:50:57.481 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:50:57.481 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:50:57.481 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:50:57.481 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:50:57.482 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:50:57.482 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 255309 2020-09-08T15:50:57.482 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.482 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:50:57.482 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.482 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:50:57.483 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.065335+0000", 2020-09-08T15:50:57.483 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:50:57.483 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.483 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.483 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:50:57.484 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.065335+0000", 2020-09-08T15:50:57.484 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999523 2020-09-08T15:50:57.484 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.484 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.484 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:50:57.485 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.065287+0000", 2020-09-08T15:50:57.485 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000112459 2020-09-08T15:50:57.485 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.485 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.486 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:50:57.486 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.065399+0000", 2020-09-08T15:50:57.486 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.8306999999999999e-05 2020-09-08T15:50:57.487 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.487 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.487 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:50:57.487 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.065428+0000", 2020-09-08T15:50:57.487 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2820000000000002e-06 2020-09-08T15:50:57.487 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.488 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.488 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:50:57.488 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.065437+0000", 2020-09-08T15:50:57.488 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.5964999999999997e-05 2020-09-08T15:50:57.488 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.489 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.489 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:50:57.489 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.065493+0000", 2020-09-08T15:50:57.489 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014469600000000001 2020-09-08T15:50:57.489 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.490 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.490 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:50:57.490 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.065637+0000", 2020-09-08T15:50:57.490 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000149087 2020-09-08T15:50:57.490 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.491 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.491 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T15:50:57.491 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.065787+0000", 2020-09-08T15:50:57.491 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0097449249999999998 2020-09-08T15:50:57.491 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.491 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.492 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:50:57.492 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.075531+0000", 2020-09-08T15:50:57.492 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15187118599999999 2020-09-08T15:50:57.492 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.492 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.493 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:50:57.493 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.227403+0000", 2020-09-08T15:50:57.493 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000123867 2020-09-08T15:50:57.493 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.493 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.494 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:50:57.494 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.227527+0000", 2020-09-08T15:50:57.494 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.3280000000000003e-05 2020-09-08T15:50:57.494 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.494 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.494 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:50:57.495 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.227600+0000", 2020-09-08T15:50:57.495 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16226527399999999 2020-09-08T15:50:57.495 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.495 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.495 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.496 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.496 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.496 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:255310 4.6 4:692e0ed2:::benchmark_data_smithi166_32054_object113168:head [delete] snapc 0=[] ondisk+write+known_if_redirected e382)", 2020-09-08T15:50:57.496 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:50:48.066041+0000", 2020-09-08T15:50:57.496 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 9.1072486359999996, 2020-09-08T15:50:57.497 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16186495100000001, 2020-09-08T15:50:57.497 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:50:57.497 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:50:57.497 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:50:57.497 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:50:57.497 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:50:57.498 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 255310 2020-09-08T15:50:57.498 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.498 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:50:57.498 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.498 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:50:57.499 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.066041+0000", 2020-09-08T15:50:57.499 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:50:57.499 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.499 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.499 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:50:57.500 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.066041+0000", 2020-09-08T15:50:57.500 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999428 2020-09-08T15:50:57.500 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.500 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.501 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:50:57.501 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.065984+0000", 2020-09-08T15:50:57.501 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000119695 2020-09-08T15:50:57.501 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.502 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.502 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:50:57.502 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.066103+0000", 2020-09-08T15:50:57.502 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.8243000000000001e-05 2020-09-08T15:50:57.502 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.503 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.503 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:50:57.503 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.066122+0000", 2020-09-08T15:50:57.503 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.0260000000000004e-06 2020-09-08T15:50:57.504 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.504 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.504 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:50:57.504 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.066128+0000", 2020-09-08T15:50:57.504 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0051837740000000004 2020-09-08T15:50:57.504 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.505 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.505 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:50:57.505 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.071311+0000", 2020-09-08T15:50:57.505 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000113422 2020-09-08T15:50:57.505 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.506 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.506 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:50:57.506 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.071425+0000", 2020-09-08T15:50:57.506 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000100053 2020-09-08T15:50:57.506 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.507 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.507 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T15:50:57.507 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.071525+0000", 2020-09-08T15:50:57.507 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0042436289999999996 2020-09-08T15:50:57.507 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.508 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.508 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:50:57.508 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.075768+0000", 2020-09-08T15:50:57.508 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15200413700000001 2020-09-08T15:50:57.508 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.509 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.509 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:50:57.509 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.227773+0000", 2020-09-08T15:50:57.509 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.8267000000000001e-05 2020-09-08T15:50:57.509 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.510 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.510 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:50:57.510 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.227851+0000", 2020-09-08T15:50:57.510 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.4839999999999997e-05 2020-09-08T15:50:57.510 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.510 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.511 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:50:57.511 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.227906+0000", 2020-09-08T15:50:57.511 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16186495100000001 2020-09-08T15:50:57.511 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.511 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.512 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.512 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.512 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.512 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:255323 4.6 4:6aa667b4:::benchmark_data_smithi166_32054_object113181:head [delete] snapc 0=[] ondisk+write+known_if_redirected e382)", 2020-09-08T15:50:57.512 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:50:48.086739+0000", 2020-09-08T15:50:57.513 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 9.0865504520000009, 2020-09-08T15:50:57.513 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15285927099999999, 2020-09-08T15:50:57.513 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:50:57.513 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:50:57.513 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:50:57.514 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:50:57.514 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:50:57.514 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 255323 2020-09-08T15:50:57.514 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.514 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:50:57.515 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.515 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:50:57.515 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.086739+0000", 2020-09-08T15:50:57.515 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:50:57.515 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.515 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.516 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:50:57.516 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.086739+0000", 2020-09-08T15:50:57.516 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999719 2020-09-08T15:50:57.516 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.516 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.517 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:50:57.517 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.086711+0000", 2020-09-08T15:50:57.517 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.1983000000000006e-05 2020-09-08T15:50:57.517 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.517 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.518 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:50:57.518 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.086773+0000", 2020-09-08T15:50:57.518 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2534000000000001e-05 2020-09-08T15:50:57.518 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.518 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.518 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:50:57.519 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.086785+0000", 2020-09-08T15:50:57.519 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.3819999999999997e-06 2020-09-08T15:50:57.519 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.519 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.519 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:50:57.520 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.086789+0000", 2020-09-08T15:50:57.520 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.4860000000000002e-05 2020-09-08T15:50:57.520 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.520 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.520 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:50:57.521 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.086824+0000", 2020-09-08T15:50:57.521 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012942699999999999 2020-09-08T15:50:57.521 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.521 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.521 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:50:57.521 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.086954+0000", 2020-09-08T15:50:57.522 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000149307 2020-09-08T15:50:57.522 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.522 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.522 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T15:50:57.522 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.087103+0000", 2020-09-08T15:50:57.523 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0044095660000000002 2020-09-08T15:50:57.523 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.523 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.523 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:50:57.523 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.091513+0000", 2020-09-08T15:50:57.524 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14801876899999999 2020-09-08T15:50:57.524 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.524 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.524 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:50:57.524 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.239531+0000", 2020-09-08T15:50:57.524 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.6599e-05 2020-09-08T15:50:57.525 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.525 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.525 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:50:57.525 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.239568+0000", 2020-09-08T15:50:57.525 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.0162999999999999e-05 2020-09-08T15:50:57.526 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:50:57.526 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:50:57.526 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:50:57.526 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.239598+0000", 2020-09-08T15:50:57.526 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15285927099999999 2020-09-08T15:50:57.527 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.527 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.527 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.527 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:50:57.527 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:50:57.529 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:50:57.530 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:50:57.533 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:50:57.533 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:50:57.533 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:50:57.533 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:50:57.538 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:50:57.538 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:50:57.538 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:50:57.538 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:50:57.539 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:50:57.539 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:50:57.539 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:50:57.539 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:50:57.539 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:50:57.540 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:50:57.540 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:50:57.540 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:57.540 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:50:57 smithi104 bash[17588]: debug 2020-09-08T15:50:57.274+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:57.641 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:50:57.734 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:50:57.762 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:50:57.870 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:50:57.906 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:50:57 smithi055 bash[19003]: debug 2020-09-08T15:50:57.762+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:58.280 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2020-09-08T15:50:58.433 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:50:58.464 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:50:58.567 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:50:58.701 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:50:58.804 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:58.812 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:50:58 smithi104 bash[19922]: debug 2020-09-08T15:50:58.466+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:58.859 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:58 smithi055 bash[9296]: cluster 2020-09-08T15:50:57.800480+0000 mgr.x (mgr.24118) 709 : cluster [DBG] pgmap v1009: 17 pgs: 17 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 978 op/s 2020-09-08T15:50:58.860 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:50:58 smithi055 bash[14301]: debug 2020-09-08T15:50:58.690+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:59.124 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:58 smithi166 bash[9446]: cluster 2020-09-08T15:50:57.800480+0000 mgr.x (mgr.24118) 709 : cluster [DBG] pgmap v1009: 17 pgs: 17 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 978 op/s 2020-09-08T15:50:59.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:58 smithi104 bash[9203]: cluster 2020-09-08T15:50:57.800480+0000 mgr.x (mgr.24118) 709 : cluster [DBG] pgmap v1009: 17 pgs: 17 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 978 op/s 2020-09-08T15:50:59.211 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:50:59.315 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:59.428 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:59.486 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:50:59 smithi104 bash[17588]: debug 2020-09-08T15:50:59.210+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:59.532 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:50:59.657 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:50:59 smithi055 bash[11956]: debug 2020-09-08T15:50:59.426+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:59.768 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:50:59.870 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:50:59 smithi166 bash[14254]: debug 2020-09-08T15:50:59.775+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:50:59.874 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:50:59.987 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:51:00.090 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:00.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:50:59 smithi055 bash[11956]: debug 2020-09-08T15:50:59.986+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:00.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:59 smithi055 bash[9296]: cluster 2020-09-08T15:50:59.055976+0000 osd.9 (osd.9) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:00.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:50:59 smithi055 bash[9296]: cluster 2020-09-08T15:50:59.056037+0000 osd.9 (osd.9) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:00.166 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:59 smithi166 bash[9446]: cluster 2020-09-08T15:50:59.055976+0000 osd.9 (osd.9) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:00.167 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:50:59 smithi166 bash[9446]: cluster 2020-09-08T15:50:59.056037+0000 osd.9 (osd.9) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:00.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:59 smithi104 bash[9203]: cluster 2020-09-08T15:50:59.055976+0000 osd.9 (osd.9) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:00.192 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:50:59 smithi104 bash[9203]: cluster 2020-09-08T15:50:59.056037+0000 osd.9 (osd.9) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:00.262 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:00.364 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:00.587 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:51:00.616 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:51:00 smithi166 bash[14254]: debug 2020-09-08T15:51:00.255+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:00.703 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:00.748 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:51:00.808 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:01.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:00 smithi055 bash[9296]: cluster 2020-09-08T15:50:59.800870+0000 mgr.x (mgr.24118) 710 : cluster [DBG] pgmap v1010: 17 pgs: 17 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.11k op/s 2020-09-08T15:51:01.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:00 smithi055 bash[9296]: cluster 2020-09-08T15:50:59.874377+0000 mon.a (mon.0) 1474 : cluster [DBG] osdmap e385: 12 total, 12 up, 12 in 2020-09-08T15:51:01.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:00 smithi104 bash[9203]: cluster 2020-09-08T15:50:59.800870+0000 mgr.x (mgr.24118) 710 : cluster [DBG] pgmap v1010: 17 pgs: 17 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.11k op/s 2020-09-08T15:51:01.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:00 smithi104 bash[9203]: cluster 2020-09-08T15:50:59.874377+0000 mon.a (mon.0) 1474 : cluster [DBG] osdmap e385: 12 total, 12 up, 12 in 2020-09-08T15:51:01.177 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:51:00 smithi104 bash[22305]: debug 2020-09-08T15:51:00.702+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:01.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:00 smithi166 bash[9446]: cluster 2020-09-08T15:50:59.800870+0000 mgr.x (mgr.24118) 710 : cluster [DBG] pgmap v1010: 17 pgs: 17 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.11k op/s 2020-09-08T15:51:01.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:00 smithi166 bash[9446]: cluster 2020-09-08T15:50:59.874377+0000 mon.a (mon.0) 1474 : cluster [DBG] osdmap e385: 12 total, 12 up, 12 in 2020-09-08T15:51:01.551 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:01.655 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:51:01.783 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:51:01.890 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:01.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:51:01 smithi104 bash[17588]: debug 2020-09-08T15:51:01.550+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:02.133 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:02.160 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:51:01 smithi055 bash[16646]: debug 2020-09-08T15:51:01.782+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:02.243 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:02.355 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:51:02 smithi166 bash[18960]: debug 2020-09-08T15:51:02.135+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:02.356 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:02.465 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:02.602 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:51:02 smithi055 bash[19003]: debug 2020-09-08T15:51:02.350+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:02.603 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:02.616 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:51:02.616 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:51:02.705 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:02.901 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:51:02 smithi055 bash[19003]: debug 2020-09-08T15:51:02.602+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:02.918 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:03.020 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:51:03.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:02 smithi166 bash[9446]: cluster 2020-09-08T15:51:01.801304+0000 mgr.x (mgr.24118) 711 : cluster [DBG] pgmap v1012: 17 pgs: 17 active+clean; 1.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 743 op/s 2020-09-08T15:51:03.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:02 smithi166 bash[9446]: cluster 2020-09-08T15:51:02.031094+0000 osd.2 (osd.2) 50 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:03.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:02 smithi166 bash[9446]: cluster 2020-09-08T15:51:02.031168+0000 osd.2 (osd.2) 51 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:03.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:02 smithi055 bash[9296]: cluster 2020-09-08T15:51:01.801304+0000 mgr.x (mgr.24118) 711 : cluster [DBG] pgmap v1012: 17 pgs: 17 active+clean; 1.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 743 op/s 2020-09-08T15:51:03.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:02 smithi055 bash[9296]: cluster 2020-09-08T15:51:02.031094+0000 osd.2 (osd.2) 50 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:03.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:02 smithi055 bash[9296]: cluster 2020-09-08T15:51:02.031168+0000 osd.2 (osd.2) 51 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:03.174 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:51:03.203 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:02 smithi104 bash[9203]: cluster 2020-09-08T15:51:01.801304+0000 mgr.x (mgr.24118) 711 : cluster [DBG] pgmap v1012: 17 pgs: 17 active+clean; 1.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 743 op/s 2020-09-08T15:51:03.203 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:02 smithi104 bash[9203]: cluster 2020-09-08T15:51:02.031094+0000 osd.2 (osd.2) 50 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:03.203 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:02 smithi104 bash[9203]: cluster 2020-09-08T15:51:02.031168+0000 osd.2 (osd.2) 51 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:03.204 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:51:02 smithi104 bash[15273]: debug 2020-09-08T15:51:02.910+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:03.278 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:51:03.392 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:51:03.499 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:03.617 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2020-09-08T15:51:03.627 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:03.660 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:51:03 smithi055 bash[16646]: debug 2020-09-08T15:51:03.390+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:03.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:51:03 smithi104 bash[19922]: debug 2020-09-08T15:51:03.170+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:03.731 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:51:03.786 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:51:03.855 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:51:03.867 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:51:03 smithi166 bash[14254]: debug 2020-09-08T15:51:03.627+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:03.920 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:51:03.959 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:51:04.080 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:51:03 smithi055 bash[16646]: debug 2020-09-08T15:51:03.846+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:04.081 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:51:04 smithi055 bash[16646]: debug 2020-09-08T15:51:04.082+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:04.081 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:03 smithi055 bash[9296]: cluster 2020-09-08T15:51:02.904150+0000 mon.a (mon.0) 1475 : cluster [DBG] osdmap e386: 12 total, 12 up, 12 in 2020-09-08T15:51:04.081 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:03 smithi055 bash[9296]: audit 2020-09-08T15:51:03.330107+0000 mon.c (mon.2) 55 : audit [INF] from='client.? v1:172.21.15.104:0/1438041723' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:51:04.082 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:03 smithi055 bash[9296]: audit 2020-09-08T15:51:03.332552+0000 mon.a (mon.0) 1476 : audit [INF] from='client.35591 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:51:04.082 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:51:04.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:03 smithi104 bash[9203]: cluster 2020-09-08T15:51:02.904150+0000 mon.a (mon.0) 1475 : cluster [DBG] osdmap e386: 12 total, 12 up, 12 in 2020-09-08T15:51:04.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:03 smithi104 bash[9203]: audit 2020-09-08T15:51:03.330107+0000 mon.c (mon.2) 55 : audit [INF] from='client.? v1:172.21.15.104:0/1438041723' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:51:04.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:03 smithi104 bash[9203]: audit 2020-09-08T15:51:03.332552+0000 mon.a (mon.0) 1476 : audit [INF] from='client.35591 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:51:04.186 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:51:04.380 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:03 smithi166 bash[9446]: cluster 2020-09-08T15:51:02.904150+0000 mon.a (mon.0) 1475 : cluster [DBG] osdmap e386: 12 total, 12 up, 12 in 2020-09-08T15:51:04.380 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:03 smithi166 bash[9446]: audit 2020-09-08T15:51:03.330107+0000 mon.c (mon.2) 55 : audit [INF] from='client.? v1:172.21.15.104:0/1438041723' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:51:04.380 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:03 smithi166 bash[9446]: audit 2020-09-08T15:51:03.332552+0000 mon.a (mon.0) 1476 : audit [INF] from='client.35591 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:51:04.765 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:51:04.870 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:05.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:51:04 smithi104 bash[19922]: debug 2020-09-08T15:51:04.762+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:05.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:04 smithi104 bash[9203]: cluster 2020-09-08T15:51:03.801694+0000 mgr.x (mgr.24118) 712 : cluster [DBG] pgmap v1014: 17 pgs: 17 active+clean; 897 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.27k op/s 2020-09-08T15:51:05.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:04 smithi104 bash[9203]: cluster 2020-09-08T15:51:03.907811+0000 mon.a (mon.0) 1477 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:51:05.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:04 smithi104 bash[9203]: audit 2020-09-08T15:51:03.919859+0000 mon.a (mon.0) 1478 : audit [INF] from='client.35591 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:51:05.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:04 smithi104 bash[9203]: cluster 2020-09-08T15:51:03.919906+0000 mon.a (mon.0) 1479 : cluster [DBG] osdmap e387: 12 total, 12 up, 12 in 2020-09-08T15:51:05.243 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:05.288 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:04 smithi166 bash[9446]: cluster 2020-09-08T15:51:03.801694+0000 mgr.x (mgr.24118) 712 : cluster [DBG] pgmap v1014: 17 pgs: 17 active+clean; 897 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.27k op/s 2020-09-08T15:51:05.289 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:04 smithi166 bash[9446]: cluster 2020-09-08T15:51:03.907811+0000 mon.a (mon.0) 1477 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:51:05.289 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:04 smithi166 bash[9446]: audit 2020-09-08T15:51:03.919859+0000 mon.a (mon.0) 1478 : audit [INF] from='client.35591 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:51:05.289 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:04 smithi166 bash[9446]: cluster 2020-09-08T15:51:03.919906+0000 mon.a (mon.0) 1479 : cluster [DBG] osdmap e387: 12 total, 12 up, 12 in 2020-09-08T15:51:05.346 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:05.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:04 smithi055 bash[9296]: cluster 2020-09-08T15:51:03.801694+0000 mgr.x (mgr.24118) 712 : cluster [DBG] pgmap v1014: 17 pgs: 17 active+clean; 897 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.27k op/s 2020-09-08T15:51:05.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:04 smithi055 bash[9296]: cluster 2020-09-08T15:51:03.907811+0000 mon.a (mon.0) 1477 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:51:05.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:04 smithi055 bash[9296]: audit 2020-09-08T15:51:03.919859+0000 mon.a (mon.0) 1478 : audit [INF] from='client.35591 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:51:05.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:04 smithi055 bash[9296]: cluster 2020-09-08T15:51:03.919906+0000 mon.a (mon.0) 1479 : cluster [DBG] osdmap e387: 12 total, 12 up, 12 in 2020-09-08T15:51:05.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:51:05 smithi104 bash[17588]: debug 2020-09-08T15:51:05.234+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:05.811 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:05.930 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:06.072 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:06.160 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:51:05 smithi104 bash[17588]: debug 2020-09-08T15:51:05.810+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:06.173 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:06.283 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:51:06 smithi055 bash[19003]: debug 2020-09-08T15:51:06.066+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:06.284 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:06.395 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:06.444 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:51:06.475 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:51:06.506 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:51:06.545 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:51:06.597 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:06.597 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:51:06.621 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:51:06.647 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:51:06.659 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:51:06 smithi055 bash[14301]: debug 2020-09-08T15:51:06.282+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:06.678 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:51:06.699 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:06.715 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:51:06.743 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:51:06 smithi166 bash[14254]: debug 2020-09-08T15:51:06.587+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:06.769 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:51:06.796 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:51:06.829 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:51:06.838 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:51:06.839 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:51:06.839 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:51:06.839 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:51:06.840 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:51:06.840 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:51:06.840 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:51:06.840 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:51:06.840 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:51:06.840 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:51:06.841 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:51:06.841 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:51:07.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:06 smithi166 bash[9446]: cluster 2020-09-08T15:51:05.802100+0000 mgr.x (mgr.24118) 713 : cluster [DBG] pgmap v1016: 17 pgs: 17 active+clean; 897 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 842 op/s 2020-09-08T15:51:07.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:06 smithi166 bash[9446]: audit 2020-09-08T15:51:06.411859+0000 mon.a (mon.0) 1480 : audit [DBG] from='client.? v1:172.21.15.104:0/507469450' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:51:07.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:06 smithi104 bash[9203]: cluster 2020-09-08T15:51:05.802100+0000 mgr.x (mgr.24118) 713 : cluster [DBG] pgmap v1016: 17 pgs: 17 active+clean; 897 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 842 op/s 2020-09-08T15:51:07.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:06 smithi104 bash[9203]: audit 2020-09-08T15:51:06.411859+0000 mon.a (mon.0) 1480 : audit [DBG] from='client.? v1:172.21.15.104:0/507469450' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:51:07.282 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:07.387 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:07.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:06 smithi055 bash[9296]: cluster 2020-09-08T15:51:05.802100+0000 mgr.x (mgr.24118) 713 : cluster [DBG] pgmap v1016: 17 pgs: 17 active+clean; 897 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 842 op/s 2020-09-08T15:51:07.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:06 smithi055 bash[9296]: audit 2020-09-08T15:51:06.411859+0000 mon.a (mon.0) 1480 : audit [DBG] from='client.? v1:172.21.15.104:0/507469450' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:51:07.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:51:07 smithi166 bash[18960]: debug 2020-09-08T15:51:07.271+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:07.848 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:51:07.944 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:08.013 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:51:08.047 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:08.220 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:51:07 smithi166 bash[11928]: debug 2020-09-08T15:51:07.943+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:08.220 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:08.325 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:08.563 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:51:08.563 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T15:51:08.563 INFO:teuthology.orchestra.run.smithi166.stdout: "complaint_time": 30, 2020-09-08T15:51:08.564 INFO:teuthology.orchestra.run.smithi166.stdout: "num_blocked_ops": 0 2020-09-08T15:51:08.564 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:51:08.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:51:08 smithi166 bash[11928]: debug 2020-09-08T15:51:08.219+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:08.679 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:08.784 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:08.934 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:08 smithi104 bash[9203]: cluster 2020-09-08T15:51:07.802640+0000 mgr.x (mgr.24118) 714 : cluster [DBG] pgmap v1017: 17 pgs: 17 active+clean; 726 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 981 op/s 2020-09-08T15:51:08.934 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:51:08 smithi104 bash[22305]: debug 2020-09-08T15:51:08.674+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:09.325 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:08 smithi166 bash[9446]: cluster 2020-09-08T15:51:07.802640+0000 mgr.x (mgr.24118) 714 : cluster [DBG] pgmap v1017: 17 pgs: 17 active+clean; 726 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 981 op/s 2020-09-08T15:51:09.326 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:51:09 smithi166 bash[11928]: debug 2020-09-08T15:51:09.327+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:09.329 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:09.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:08 smithi055 bash[9296]: cluster 2020-09-08T15:51:07.802640+0000 mgr.x (mgr.24118) 714 : cluster [DBG] pgmap v1017: 17 pgs: 17 active+clean; 726 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 981 op/s 2020-09-08T15:51:09.434 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:09.582 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:09.611 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2020-09-08T15:51:09.687 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:09.697 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:51:09 smithi055 bash[14301]: debug 2020-09-08T15:51:09.582+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:09.778 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:51:09.805 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:09.907 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:10.006 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:51:10.107 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:10.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:51:09 smithi055 bash[19003]: debug 2020-09-08T15:51:09.802+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:10.162 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:51:10.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:51:10 smithi104 bash[17588]: debug 2020-09-08T15:51:10.110+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:10.211 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:10.324 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:10.433 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:10.665 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:51:10 smithi055 bash[14301]: debug 2020-09-08T15:51:10.322+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:10.906 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:11.008 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:11.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:10 smithi166 bash[9446]: cluster 2020-09-08T15:51:09.803139+0000 mgr.x (mgr.24118) 715 : cluster [DBG] pgmap v1018: 17 pgs: 17 active+clean; 593 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T15:51:11.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:10 smithi055 bash[9296]: cluster 2020-09-08T15:51:09.803139+0000 mgr.x (mgr.24118) 715 : cluster [DBG] pgmap v1018: 17 pgs: 17 active+clean; 593 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T15:51:11.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:10 smithi104 bash[9203]: cluster 2020-09-08T15:51:09.803139+0000 mgr.x (mgr.24118) 715 : cluster [DBG] pgmap v1018: 17 pgs: 17 active+clean; 593 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T15:51:11.181 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:51:10 smithi104 bash[22305]: debug 2020-09-08T15:51:10.902+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:11.214 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:11.315 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:11.430 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:11.532 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:11.618 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:51:11 smithi166 bash[11928]: debug 2020-09-08T15:51:11.431+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:11.618 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:51:11 smithi166 bash[18960]: debug 2020-09-08T15:51:11.211+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:11.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:11 smithi104 bash[9203]: cluster 2020-09-08T15:51:10.376286+0000 osd.7 (osd.7) 73 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:11.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:11 smithi104 bash[9203]: cluster 2020-09-08T15:51:10.376384+0000 osd.7 (osd.7) 74 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:11.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:11 smithi104 bash[9203]: cluster 2020-09-08T15:51:10.779594+0000 mon.a (mon.0) 1481 : cluster [DBG] osdmap e388: 12 total, 12 up, 12 in 2020-09-08T15:51:11.955 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:12.057 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:12.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:11 smithi166 bash[9446]: cluster 2020-09-08T15:51:10.376286+0000 osd.7 (osd.7) 73 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:12.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:11 smithi166 bash[9446]: cluster 2020-09-08T15:51:10.376384+0000 osd.7 (osd.7) 74 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:12.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:11 smithi166 bash[9446]: cluster 2020-09-08T15:51:10.779594+0000 mon.a (mon.0) 1481 : cluster [DBG] osdmap e388: 12 total, 12 up, 12 in 2020-09-08T15:51:12.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:11 smithi055 bash[9296]: cluster 2020-09-08T15:51:10.376286+0000 osd.7 (osd.7) 73 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:12.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:11 smithi055 bash[9296]: cluster 2020-09-08T15:51:10.376384+0000 osd.7 (osd.7) 74 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:12.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:11 smithi055 bash[9296]: cluster 2020-09-08T15:51:10.779594+0000 mon.a (mon.0) 1481 : cluster [DBG] osdmap e388: 12 total, 12 up, 12 in 2020-09-08T15:51:12.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:51:11 smithi104 bash[22305]: debug 2020-09-08T15:51:11.950+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:12.631 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:12.735 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:12.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:51:12 smithi104 bash[17588]: debug 2020-09-08T15:51:12.630+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:12.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:12 smithi104 bash[9203]: cluster 2020-09-08T15:51:11.503444+0000 osd.0 (osd.0) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:12.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:12 smithi104 bash[9203]: cluster 2020-09-08T15:51:11.503533+0000 osd.0 (osd.0) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:12.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:12 smithi104 bash[9203]: cluster 2020-09-08T15:51:11.784121+0000 mon.a (mon.0) 1482 : cluster [DBG] osdmap e389: 12 total, 12 up, 12 in 2020-09-08T15:51:12.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:12 smithi104 bash[9203]: cluster 2020-09-08T15:51:11.803535+0000 mgr.x (mgr.24118) 716 : cluster [DBG] pgmap v1021: 17 pgs: 17 active+clean; 532 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 739 op/s 2020-09-08T15:51:12.938 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:13.042 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:13.049 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:12 smithi166 bash[9446]: cluster 2020-09-08T15:51:11.503444+0000 osd.0 (osd.0) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:13.050 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:12 smithi166 bash[9446]: cluster 2020-09-08T15:51:11.503533+0000 osd.0 (osd.0) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:13.050 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:12 smithi166 bash[9446]: cluster 2020-09-08T15:51:11.784121+0000 mon.a (mon.0) 1482 : cluster [DBG] osdmap e389: 12 total, 12 up, 12 in 2020-09-08T15:51:13.050 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:12 smithi166 bash[9446]: cluster 2020-09-08T15:51:11.803535+0000 mgr.x (mgr.24118) 716 : cluster [DBG] pgmap v1021: 17 pgs: 17 active+clean; 532 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 739 op/s 2020-09-08T15:51:13.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:12 smithi055 bash[9296]: cluster 2020-09-08T15:51:11.503444+0000 osd.0 (osd.0) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:13.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:12 smithi055 bash[9296]: cluster 2020-09-08T15:51:11.503533+0000 osd.0 (osd.0) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:13.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:12 smithi055 bash[9296]: cluster 2020-09-08T15:51:11.784121+0000 mon.a (mon.0) 1482 : cluster [DBG] osdmap e389: 12 total, 12 up, 12 in 2020-09-08T15:51:13.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:12 smithi055 bash[9296]: cluster 2020-09-08T15:51:11.803535+0000 mgr.x (mgr.24118) 716 : cluster [DBG] pgmap v1021: 17 pgs: 17 active+clean; 532 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 739 op/s 2020-09-08T15:51:13.196 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:13.262 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:51:12 smithi104 bash[15273]: debug 2020-09-08T15:51:12.938+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:13.304 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:51:13.312 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:51:13 smithi166 bash[18960]: debug 2020-09-08T15:51:13.195+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:13.724 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:51:13.807 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:51:13.828 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:14.045 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:13 smithi104 bash[9203]: audit 2020-09-08T15:51:13.712579+0000 mon.a (mon.0) 1483 : audit [INF] from='client.? v1:172.21.15.104:0/1728397069' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:51:14.050 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:14.066 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:13 smithi166 bash[9446]: audit 2020-09-08T15:51:13.712579+0000 mon.a (mon.0) 1483 : audit [INF] from='client.? v1:172.21.15.104:0/1728397069' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:51:14.067 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:51:13 smithi166 bash[16593]: debug 2020-09-08T15:51:13.727+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:14.152 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:14.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:13 smithi055 bash[9296]: audit 2020-09-08T15:51:13.712579+0000 mon.a (mon.0) 1483 : audit [INF] from='client.? v1:172.21.15.104:0/1728397069' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:51:14.282 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:14.336 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:51:14.336 INFO:teuthology.orchestra.run.smithi166.stdout: "size": 20, 2020-09-08T15:51:14.336 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 600, 2020-09-08T15:51:14.337 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T15:51:14.337 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.337 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:177575 4.e 4:7204d027:::benchmark_data_smithi166_32054_object35433:head [delete] snapc 0=[] ondisk+write+known_if_redirected e359)", 2020-09-08T15:51:14.337 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:49:29.714692+0000", 2020-09-08T15:51:14.337 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 104.622418638, 2020-09-08T15:51:14.338 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.052818400000000001, 2020-09-08T15:51:14.338 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:14.338 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:14.339 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:14.339 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:51:14.339 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:14.339 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 177575 2020-09-08T15:51:14.340 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.340 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:14.340 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.340 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:14.340 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.714692+0000", 2020-09-08T15:51:14.341 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:14.341 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.341 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.341 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:14.341 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.714692+0000", 2020-09-08T15:51:14.342 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999599 2020-09-08T15:51:14.342 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.342 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.342 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:14.343 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.714652+0000", 2020-09-08T15:51:14.343 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.0095999999999999e-05 2020-09-08T15:51:14.343 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.343 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.343 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:14.344 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.714742+0000", 2020-09-08T15:51:14.344 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7090000000000001e-05 2020-09-08T15:51:14.344 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.344 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.344 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:14.345 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.714759+0000", 2020-09-08T15:51:14.345 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8750000000000006e-06 2020-09-08T15:51:14.345 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.346 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.346 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:14.347 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.714768+0000", 2020-09-08T15:51:14.347 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0028870329999999998 2020-09-08T15:51:14.347 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.347 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.347 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:14.348 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.717655+0000", 2020-09-08T15:51:14.348 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000147405 2020-09-08T15:51:14.348 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.348 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.348 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:14.349 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.717802+0000", 2020-09-08T15:51:14.349 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000111329 2020-09-08T15:51:14.349 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.350 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.350 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T15:51:14.350 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.717913+0000", 2020-09-08T15:51:14.350 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0087259539999999993 2020-09-08T15:51:14.350 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.351 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.351 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:51:14.351 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.726639+0000", 2020-09-08T15:51:14.351 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.040746562 2020-09-08T15:51:14.351 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.352 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.352 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:14.352 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.767386+0000", 2020-09-08T15:51:14.352 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.0301000000000003e-05 2020-09-08T15:51:14.352 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.353 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.353 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:14.353 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.767456+0000", 2020-09-08T15:51:14.353 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.384e-05 2020-09-08T15:51:14.354 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.354 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.354 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:14.354 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.767510+0000", 2020-09-08T15:51:14.354 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.052818400000000001 2020-09-08T15:51:14.355 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.355 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.355 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.355 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.356 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.356 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:177592 4.e 4:768515be:::benchmark_data_smithi166_32054_object35450:head [delete] snapc 0=[] ondisk+write+known_if_redirected e359)", 2020-09-08T15:51:14.356 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:49:29.727746+0000", 2020-09-08T15:51:14.356 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 104.609364279, 2020-09-08T15:51:14.356 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.069983675999999995, 2020-09-08T15:51:14.357 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:14.357 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:14.357 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:14.357 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:51:14.357 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:14.358 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 177592 2020-09-08T15:51:14.358 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.358 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:14.358 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.358 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:14.359 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.727746+0000", 2020-09-08T15:51:14.359 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:14.359 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.360 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.360 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:14.360 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.727746+0000", 2020-09-08T15:51:14.360 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999495 2020-09-08T15:51:14.360 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.361 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.361 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:14.361 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.727695+0000", 2020-09-08T15:51:14.361 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7874000000000005e-05 2020-09-08T15:51:14.361 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.362 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.362 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:14.362 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.727793+0000", 2020-09-08T15:51:14.362 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3733999999999999e-05 2020-09-08T15:51:14.362 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.363 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.363 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:14.363 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.727807+0000", 2020-09-08T15:51:14.363 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.28e-06 2020-09-08T15:51:14.363 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.363 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.364 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:14.364 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.727813+0000", 2020-09-08T15:51:14.364 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.2484999999999996e-05 2020-09-08T15:51:14.364 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.364 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.365 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:14.365 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.727876+0000", 2020-09-08T15:51:14.365 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013271299999999999 2020-09-08T15:51:14.365 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.365 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.366 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:14.366 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.728009+0000", 2020-09-08T15:51:14.366 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012793200000000001 2020-09-08T15:51:14.366 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.367 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.367 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T15:51:14.367 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.728136+0000", 2020-09-08T15:51:14.367 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.014195236 2020-09-08T15:51:14.367 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.368 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.368 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:51:14.368 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.742332+0000", 2020-09-08T15:51:14.368 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.055284452999999997 2020-09-08T15:51:14.368 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.368 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.369 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:14.369 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.797616+0000", 2020-09-08T15:51:14.369 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.4622000000000006e-05 2020-09-08T15:51:14.369 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.370 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.370 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:14.370 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.797681+0000", 2020-09-08T15:51:14.370 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.8927000000000001e-05 2020-09-08T15:51:14.371 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.371 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.371 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:14.371 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.797730+0000", 2020-09-08T15:51:14.371 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.069983675999999995 2020-09-08T15:51:14.372 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.372 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.372 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.372 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.372 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.373 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:177593 4.e 4:7e63d83b:::benchmark_data_smithi166_32054_object35451:head [delete] snapc 0=[] ondisk+write+known_if_redirected e359)", 2020-09-08T15:51:14.373 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:49:29.727976+0000", 2020-09-08T15:51:14.373 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 104.60913435800001, 2020-09-08T15:51:14.373 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.070128904000000006, 2020-09-08T15:51:14.373 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:14.374 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:14.374 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:14.374 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:51:14.374 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:14.374 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 177593 2020-09-08T15:51:14.375 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.376 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:14.376 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.376 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:14.376 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.727976+0000", 2020-09-08T15:51:14.376 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:14.377 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.377 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.377 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:14.377 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.727976+0000", 2020-09-08T15:51:14.377 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999628 2020-09-08T15:51:14.378 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.378 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.378 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:14.378 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.727939+0000", 2020-09-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.4002e-05 2020-09-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.728023+0000", 2020-09-08T15:51:14.380 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2907e-05 2020-09-08T15:51:14.380 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.728036+0000", 2020-09-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.6130000000000003e-06 2020-09-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.382 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.382 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:14.382 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.728041+0000", 2020-09-08T15:51:14.382 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0052103649999999998 2020-09-08T15:51:14.382 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.383 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.383 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:14.383 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.733252+0000", 2020-09-08T15:51:14.383 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000107447 2020-09-08T15:51:14.383 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.733359+0000", 2020-09-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000101464 2020-09-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.385 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.385 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T15:51:14.385 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.733461+0000", 2020-09-08T15:51:14.385 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.014235517 2020-09-08T15:51:14.385 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.386 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.386 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:51:14.386 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.747696+0000", 2020-09-08T15:51:14.386 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.050140412000000002 2020-09-08T15:51:14.386 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.387 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.387 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:14.387 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.797837+0000", 2020-09-08T15:51:14.387 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013000699999999999 2020-09-08T15:51:14.387 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.387 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.797967+0000", 2020-09-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000138357 2020-09-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.389 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:14.389 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.798105+0000", 2020-09-08T15:51:14.389 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.070128904000000006 2020-09-08T15:51:14.389 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.389 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.390 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.390 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.390 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.390 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:177594 4.e 4:713a3830:::benchmark_data_smithi166_32054_object35452:head [delete] snapc 0=[] ondisk+write+known_if_redirected e359)", 2020-09-08T15:51:14.391 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:49:29.730690+0000", 2020-09-08T15:51:14.391 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 104.606419992, 2020-09-08T15:51:14.391 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.067579048000000003, 2020-09-08T15:51:14.391 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 177594 2020-09-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.393 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:14.393 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.393 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:14.393 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.730690+0000", 2020-09-08T15:51:14.393 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:14.394 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.394 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.394 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:14.394 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.730690+0000", 2020-09-08T15:51:14.394 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999619 2020-09-08T15:51:14.395 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.395 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.395 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:14.395 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.730652+0000", 2020-09-08T15:51:14.395 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2915000000000001e-05 2020-09-08T15:51:14.396 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.396 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.396 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:14.396 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.730745+0000", 2020-09-08T15:51:14.396 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4256e-05 2020-09-08T15:51:14.397 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.397 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.397 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:14.398 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.730759+0000", 2020-09-08T15:51:14.398 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.5500000000000002e-06 2020-09-08T15:51:14.398 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.398 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.398 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:14.399 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.730765+0000", 2020-09-08T15:51:14.399 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.007198996 2020-09-08T15:51:14.399 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.399 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.399 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:14.400 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.737964+0000", 2020-09-08T15:51:14.400 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014374 2020-09-08T15:51:14.400 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.400 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.400 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:14.401 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.738107+0000", 2020-09-08T15:51:14.401 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000107499 2020-09-08T15:51:14.401 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.402 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.402 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T15:51:14.402 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.738215+0000", 2020-09-08T15:51:14.402 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0094966679999999998 2020-09-08T15:51:14.402 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.402 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.403 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:51:14.403 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.747712+0000", 2020-09-08T15:51:14.403 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.050454232000000002 2020-09-08T15:51:14.403 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.403 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.404 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:14.404 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.798166+0000", 2020-09-08T15:51:14.404 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.0061e-05 2020-09-08T15:51:14.404 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.404 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.798226+0000", 2020-09-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.3455999999999999e-05 2020-09-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.406 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:14.406 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.798269+0000", 2020-09-08T15:51:14.406 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.067579048000000003 2020-09-08T15:51:14.406 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.406 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:177598 4.e 4:7d221351:::benchmark_data_smithi166_32054_object35456:head [delete] snapc 0=[] ondisk+write+known_if_redirected e359)", 2020-09-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:49:29.737510+0000", 2020-09-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 104.599600416, 2020-09-08T15:51:14.408 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.061114636, 2020-09-08T15:51:14.408 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:14.408 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:14.408 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:14.408 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 177598 2020-09-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.410 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:14.410 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.737510+0000", 2020-09-08T15:51:14.410 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:14.410 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.410 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.411 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:14.411 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.737510+0000", 2020-09-08T15:51:14.411 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.999959 2020-09-08T15:51:14.412 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.412 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.412 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:14.412 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.737469+0000", 2020-09-08T15:51:14.413 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.5798999999999995e-05 2020-09-08T15:51:14.413 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.413 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.413 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:14.413 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.737565+0000", 2020-09-08T15:51:14.414 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.9235000000000001e-05 2020-09-08T15:51:14.414 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.414 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.414 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:14.414 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.737584+0000", 2020-09-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1199999999999999e-05 2020-09-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.737595+0000", 2020-09-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0047720180000000003 2020-09-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.742367+0000", 2020-09-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012043400000000001 2020-09-08T15:51:14.417 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.417 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.417 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:14.417 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.742487+0000", 2020-09-08T15:51:14.417 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000117583 2020-09-08T15:51:14.418 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.418 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.418 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T15:51:14.418 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.742605+0000", 2020-09-08T15:51:14.418 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0053571449999999998 2020-09-08T15:51:14.419 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.419 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.419 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:51:14.419 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.747962+0000", 2020-09-08T15:51:14.419 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.050404316999999997 2020-09-08T15:51:14.420 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.420 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.420 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:14.420 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.798367+0000", 2020-09-08T15:51:14.420 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017044900000000001 2020-09-08T15:51:14.421 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.421 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.421 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:14.421 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.798537+0000", 2020-09-08T15:51:14.421 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.7571999999999994e-05 2020-09-08T15:51:14.422 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.422 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.422 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:14.422 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:49:29.798625+0000", 2020-09-08T15:51:14.423 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.061114636 2020-09-08T15:51:14.423 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.423 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.423 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.423 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.424 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.424 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:254867 4.e 4:7eb253df:::benchmark_data_smithi166_32054_object112725:head [delete] snapc 0=[] ondisk+write+known_if_redirected e381)", 2020-09-08T15:51:14.424 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:50:47.591067+0000", 2020-09-08T15:51:14.424 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 26.746043063999998, 2020-09-08T15:51:14.424 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.058087244000000003, 2020-09-08T15:51:14.425 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:14.425 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:14.425 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:14.425 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:51:14.425 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:14.426 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 254867 2020-09-08T15:51:14.426 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.426 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:14.426 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.426 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:14.427 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.591067+0000", 2020-09-08T15:51:14.427 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:14.427 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.427 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.427 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:14.427 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.591067+0000", 2020-09-08T15:51:14.428 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999619 2020-09-08T15:51:14.428 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.428 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.428 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:14.429 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.591029+0000", 2020-09-08T15:51:14.429 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.6946000000000003e-05 2020-09-08T15:51:14.429 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.429 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.429 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:14.430 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.591126+0000", 2020-09-08T15:51:14.430 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7827000000000001e-05 2020-09-08T15:51:14.430 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.430 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.430 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:14.431 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.591144+0000", 2020-09-08T15:51:14.431 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.9200000000000004e-06 2020-09-08T15:51:14.431 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.431 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.432 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:14.432 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.591152+0000", 2020-09-08T15:51:14.432 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9817000000000001e-05 2020-09-08T15:51:14.433 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.433 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.433 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:14.434 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.591202+0000", 2020-09-08T15:51:14.434 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012998400000000001 2020-09-08T15:51:14.434 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.435 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.435 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:14.435 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.591332+0000", 2020-09-08T15:51:14.435 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013368000000000001 2020-09-08T15:51:14.436 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.436 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.436 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T15:51:14.436 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.591465+0000", 2020-09-08T15:51:14.436 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.016641090000000001 2020-09-08T15:51:14.437 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.437 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.437 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:14.437 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.608106+0000", 2020-09-08T15:51:14.437 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.040792199000000001 2020-09-08T15:51:14.438 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.438 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.438 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:51:14.438 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.648899+0000", 2020-09-08T15:51:14.438 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000154959 2020-09-08T15:51:14.439 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.439 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.439 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:14.439 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.649054+0000", 2020-09-08T15:51:14.439 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010090300000000001 2020-09-08T15:51:14.440 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.440 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.440 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:14.440 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.649155+0000", 2020-09-08T15:51:14.440 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.058087244000000003 2020-09-08T15:51:14.440 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.441 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.441 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.441 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.441 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.441 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:254873 4.e 4:7449eaf3:::benchmark_data_smithi166_32054_object112731:head [delete] snapc 0=[] ondisk+write+known_if_redirected e381)", 2020-09-08T15:51:14.442 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:50:47.594485+0000", 2020-09-08T15:51:14.442 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 26.742625818, 2020-09-08T15:51:14.448 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.054962280000000002, 2020-09-08T15:51:14.448 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:14.449 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:14.449 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:14.449 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:51:14.449 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:14.450 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 254873 2020-09-08T15:51:14.450 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.450 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:14.450 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.450 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:14.451 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.594485+0000", 2020-09-08T15:51:14.451 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:14.452 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.453 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.453 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:14.453 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.594485+0000", 2020-09-08T15:51:14.453 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999595 2020-09-08T15:51:14.454 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.454 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.454 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:14.455 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.594444+0000", 2020-09-08T15:51:14.456 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8496000000000001e-05 2020-09-08T15:51:14.456 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.457 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.457 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:14.457 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.594532+0000", 2020-09-08T15:51:14.457 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4331999999999999e-05 2020-09-08T15:51:14.458 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.458 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.458 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:14.458 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.594547+0000", 2020-09-08T15:51:14.458 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.4829999999999996e-06 2020-09-08T15:51:14.460 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.460 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.460 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:14.460 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.594553+0000", 2020-09-08T15:51:14.460 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.002210905 2020-09-08T15:51:14.461 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.461 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.461 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:14.461 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.596764+0000", 2020-09-08T15:51:14.461 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012766000000000001 2020-09-08T15:51:14.462 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.462 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.462 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:14.462 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.596892+0000", 2020-09-08T15:51:14.462 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013750899999999999 2020-09-08T15:51:14.463 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.463 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.463 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T15:51:14.463 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.597029+0000", 2020-09-08T15:51:14.463 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.052147212999999998 2020-09-08T15:51:14.464 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.464 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.464 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:51:14.464 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.649177+0000", 2020-09-08T15:51:14.464 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.4549000000000007e-05 2020-09-08T15:51:14.465 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.465 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.465 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:14.465 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.649261+0000", 2020-09-08T15:51:14.465 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010380899999999999 2020-09-08T15:51:14.466 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.466 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.466 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:14.466 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.649365+0000", 2020-09-08T15:51:14.466 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.1861999999999997e-05 2020-09-08T15:51:14.467 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.467 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.467 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:14.467 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.649447+0000", 2020-09-08T15:51:14.467 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.054962280000000002 2020-09-08T15:51:14.468 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.468 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.468 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.468 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.468 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.469 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:254875 4.e 4:7c81f47c:::benchmark_data_smithi166_32054_object112733:head [delete] snapc 0=[] ondisk+write+known_if_redirected e381)", 2020-09-08T15:51:14.469 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:50:47.596232+0000", 2020-09-08T15:51:14.469 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 26.740878005999999, 2020-09-08T15:51:14.469 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.053518268000000001, 2020-09-08T15:51:14.469 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:14.470 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:14.470 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:14.470 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:51:14.470 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:14.470 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 254875 2020-09-08T15:51:14.471 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.471 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:14.471 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.471 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:14.471 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.596232+0000", 2020-09-08T15:51:14.471 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:14.472 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.472 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.472 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:14.472 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.596232+0000", 2020-09-08T15:51:14.472 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999685 2020-09-08T15:51:14.473 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.473 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.473 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:14.473 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.596201+0000", 2020-09-08T15:51:14.473 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017487199999999999 2020-09-08T15:51:14.474 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.474 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.474 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:14.474 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.596376+0000", 2020-09-08T15:51:14.474 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4506e-05 2020-09-08T15:51:14.475 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.475 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.475 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:14.475 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.596390+0000", 2020-09-08T15:51:14.475 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5579999999999997e-06 2020-09-08T15:51:14.476 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.476 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.476 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:14.476 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.596397+0000", 2020-09-08T15:51:14.476 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0066081459999999996 2020-09-08T15:51:14.477 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.477 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.477 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:14.477 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.603005+0000", 2020-09-08T15:51:14.477 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010817199999999999 2020-09-08T15:51:14.478 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.478 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.478 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:14.478 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.603113+0000", 2020-09-08T15:51:14.478 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.0907000000000002e-05 2020-09-08T15:51:14.478 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.479 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.479 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T15:51:14.479 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.603204+0000", 2020-09-08T15:51:14.479 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.046266558999999999 2020-09-08T15:51:14.479 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.480 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.480 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:51:14.480 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.649471+0000", 2020-09-08T15:51:14.480 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7874000000000005e-05 2020-09-08T15:51:14.480 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.481 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.481 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:14.481 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.649568+0000", 2020-09-08T15:51:14.481 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.5796000000000002e-05 2020-09-08T15:51:14.481 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.481 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.482 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:14.482 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.649664+0000", 2020-09-08T15:51:14.482 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.6410999999999996e-05 2020-09-08T15:51:14.482 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.482 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.483 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:14.483 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.649751+0000", 2020-09-08T15:51:14.483 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.053518268000000001 2020-09-08T15:51:14.483 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.484 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.484 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.484 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.484 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.484 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:254880 4.e 4:7ab7a131:::benchmark_data_smithi166_32054_object112738:head [delete] snapc 0=[] ondisk+write+known_if_redirected e381)", 2020-09-08T15:51:14.485 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:50:47.600778+0000", 2020-09-08T15:51:14.485 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 26.736331883999998, 2020-09-08T15:51:14.485 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.056211019000000001, 2020-09-08T15:51:14.485 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:14.485 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:14.486 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:14.486 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:51:14.486 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:14.486 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 254880 2020-09-08T15:51:14.486 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.487 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:14.487 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.487 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:14.487 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.600778+0000", 2020-09-08T15:51:14.487 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:14.488 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.488 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.488 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:14.488 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.600778+0000", 2020-09-08T15:51:14.488 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999638 2020-09-08T15:51:14.488 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.489 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.489 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:14.489 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.600742+0000", 2020-09-08T15:51:14.489 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.0958000000000007e-05 2020-09-08T15:51:14.489 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.490 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.490 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:14.490 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.600823+0000", 2020-09-08T15:51:14.490 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3455000000000001e-05 2020-09-08T15:51:14.490 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.491 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.491 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:14.491 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.600837+0000", 2020-09-08T15:51:14.491 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7620000000000001e-06 2020-09-08T15:51:14.491 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.492 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.492 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:14.492 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.600842+0000", 2020-09-08T15:51:14.492 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.007292415 2020-09-08T15:51:14.492 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.492 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.493 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:14.493 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.608135+0000", 2020-09-08T15:51:14.493 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000154112 2020-09-08T15:51:14.494 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.494 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.495 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:14.495 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.608289+0000", 2020-09-08T15:51:14.495 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00076187 2020-09-08T15:51:14.495 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.496 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.496 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T15:51:14.496 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.609051+0000", 2020-09-08T15:51:14.496 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.040806240000000001 2020-09-08T15:51:14.496 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.497 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.497 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:14.497 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.649857+0000", 2020-09-08T15:51:14.497 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.006992491 2020-09-08T15:51:14.497 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.498 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.498 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:51:14.498 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.656850+0000", 2020-09-08T15:51:14.498 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.9030000000000006e-05 2020-09-08T15:51:14.498 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.498 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.499 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:14.499 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.656939+0000", 2020-09-08T15:51:14.499 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0864e-05 2020-09-08T15:51:14.499 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.499 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.500 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:14.500 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:47.656989+0000", 2020-09-08T15:51:14.500 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.056211019000000001 2020-09-08T15:51:14.501 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.501 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.502 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.502 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.502 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.503 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:256011 4.e 4:7eb77a9a:::benchmark_data_smithi166_32054_object113869:head [delete] snapc 0=[] ondisk+write+known_if_redirected e382)", 2020-09-08T15:51:14.504 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:50:48.877335+0000", 2020-09-08T15:51:14.504 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 25.459775723, 2020-09-08T15:51:14.504 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.057798454999999999, 2020-09-08T15:51:14.505 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:14.505 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:14.505 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:14.505 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:51:14.505 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:14.506 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 256011 2020-09-08T15:51:14.506 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.506 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:14.506 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.506 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:14.507 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.877335+0000", 2020-09-08T15:51:14.507 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:14.507 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.507 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.507 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:14.508 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.877335+0000", 2020-09-08T15:51:14.508 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999652 2020-09-08T15:51:14.508 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.508 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.508 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:14.508 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.877300+0000", 2020-09-08T15:51:14.509 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.2399999999999997e-05 2020-09-08T15:51:14.509 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.509 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.509 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:14.510 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.877382+0000", 2020-09-08T15:51:14.510 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5193e-05 2020-09-08T15:51:14.510 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.510 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.510 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:14.511 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.877397+0000", 2020-09-08T15:51:14.511 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.6790000000000005e-06 2020-09-08T15:51:14.511 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.511 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.512 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:14.512 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.877405+0000", 2020-09-08T15:51:14.512 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.040430143000000002 2020-09-08T15:51:14.512 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.513 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.513 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:14.513 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.917835+0000", 2020-09-08T15:51:14.513 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000309036 2020-09-08T15:51:14.513 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.514 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.514 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:14.514 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.918144+0000", 2020-09-08T15:51:14.514 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000378604 2020-09-08T15:51:14.515 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.515 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.515 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T15:51:14.515 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.918523+0000", 2020-09-08T15:51:14.515 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.016204658 2020-09-08T15:51:14.516 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.516 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.516 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:14.516 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.934727+0000", 2020-09-08T15:51:14.516 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00025680000000000001 2020-09-08T15:51:14.517 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.517 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.517 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:51:14.517 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.934984+0000", 2020-09-08T15:51:14.517 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.0591000000000001e-05 2020-09-08T15:51:14.518 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.518 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.518 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:14.518 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.935075+0000", 2020-09-08T15:51:14.518 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8273999999999998e-05 2020-09-08T15:51:14.519 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.519 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.519 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:14.519 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.935133+0000", 2020-09-08T15:51:14.519 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.057798454999999999 2020-09-08T15:51:14.520 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.520 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.520 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.520 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.520 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.520 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:261710 4.e 4:76084ae1:::benchmark_data_smithi166_32054_object119568:head [delete] snapc 0=[] ondisk+write+known_if_redirected e384)", 2020-09-08T15:51:14.521 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:50:54.469591+0000", 2020-09-08T15:51:14.521 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 19.867519499, 2020-09-08T15:51:14.521 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.065571832999999996, 2020-09-08T15:51:14.521 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:14.521 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:14.522 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:14.522 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:51:14.522 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:14.522 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 261710 2020-09-08T15:51:14.522 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.523 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:14.523 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.523 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:14.523 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.469591+0000", 2020-09-08T15:51:14.523 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:14.524 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.524 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.524 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:14.524 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.469591+0000", 2020-09-08T15:51:14.525 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999628 2020-09-08T15:51:14.525 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.525 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.525 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:14.525 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.469554+0000", 2020-09-08T15:51:14.526 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.4753000000000007e-05 2020-09-08T15:51:14.526 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.526 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.526 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:14.526 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.469639+0000", 2020-09-08T15:51:14.527 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4392e-05 2020-09-08T15:51:14.527 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.527 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.527 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:14.527 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.469653+0000", 2020-09-08T15:51:14.528 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.4670000000000001e-06 2020-09-08T15:51:14.528 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.528 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.528 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:14.528 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.469659+0000", 2020-09-08T15:51:14.528 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0058607449999999997 2020-09-08T15:51:14.529 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.529 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.529 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:14.529 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.475520+0000", 2020-09-08T15:51:14.529 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000137508 2020-09-08T15:51:14.530 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.530 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.530 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:14.530 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.475658+0000", 2020-09-08T15:51:14.530 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000141123 2020-09-08T15:51:14.531 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.531 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.531 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T15:51:14.531 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.475799+0000", 2020-09-08T15:51:14.531 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0043077289999999997 2020-09-08T15:51:14.532 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.532 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.532 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:51:14.532 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.480107+0000", 2020-09-08T15:51:14.532 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.054841617000000002 2020-09-08T15:51:14.532 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.533 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.533 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:14.533 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.534948+0000", 2020-09-08T15:51:14.533 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7256999999999996e-05 2020-09-08T15:51:14.533 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.534 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.534 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:14.534 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.535025+0000", 2020-09-08T15:51:14.534 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013720799999999999 2020-09-08T15:51:14.534 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.535 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.535 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:14.535 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.535163+0000", 2020-09-08T15:51:14.535 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.065571832999999996 2020-09-08T15:51:14.535 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.535 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.536 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.536 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.536 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.536 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:261721 4.e 4:7a240aca:::benchmark_data_smithi166_32054_object119579:head [delete] snapc 0=[] ondisk+write+known_if_redirected e384)", 2020-09-08T15:51:14.536 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:50:54.480292+0000", 2020-09-08T15:51:14.537 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 19.856818014000002, 2020-09-08T15:51:14.537 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.055770416000000003, 2020-09-08T15:51:14.537 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:14.537 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:14.537 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:14.538 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:51:14.538 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:14.538 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 261721 2020-09-08T15:51:14.538 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.538 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:14.539 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.539 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:14.539 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.480292+0000", 2020-09-08T15:51:14.539 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:14.540 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.540 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.540 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:14.540 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.480292+0000", 2020-09-08T15:51:14.540 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999666 2020-09-08T15:51:14.541 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.541 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.541 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:14.541 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.480259+0000", 2020-09-08T15:51:14.541 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2738000000000006e-05 2020-09-08T15:51:14.542 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.542 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.542 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:14.542 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.480352+0000", 2020-09-08T15:51:14.542 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2064e-05 2020-09-08T15:51:14.542 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.543 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.543 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:14.543 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.480364+0000", 2020-09-08T15:51:14.543 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.9313e-05 2020-09-08T15:51:14.544 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.544 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.544 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:14.544 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.480383+0000", 2020-09-08T15:51:14.544 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0630000000000001e-05 2020-09-08T15:51:14.544 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.545 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.545 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:14.545 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.480434+0000", 2020-09-08T15:51:14.545 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012849500000000001 2020-09-08T15:51:14.545 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.546 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.546 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:14.546 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.480562+0000", 2020-09-08T15:51:14.546 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000128775 2020-09-08T15:51:14.546 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.547 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.547 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T15:51:14.547 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.480691+0000", 2020-09-08T15:51:14.547 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.005750855 2020-09-08T15:51:14.547 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.547 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.548 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:51:14.548 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.486442+0000", 2020-09-08T15:51:14.548 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.048855316000000003 2020-09-08T15:51:14.548 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.548 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.549 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:14.549 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.535297+0000", 2020-09-08T15:51:14.549 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00071210100000000003 2020-09-08T15:51:14.549 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.549 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.550 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:14.550 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.536009+0000", 2020-09-08T15:51:14.550 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.3340000000000001e-05 2020-09-08T15:51:14.550 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.550 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.551 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:14.551 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.536063+0000", 2020-09-08T15:51:14.551 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.055770416000000003 2020-09-08T15:51:14.551 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.551 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.552 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.552 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.552 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.552 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.35381.0:261731 4.5 e384/314 4:af0c0529:::benchmark_data_smithi166_32054_object119589:head v 384'16401, mlcod=384'16398)", 2020-09-08T15:51:14.552 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:50:54.488311+0000", 2020-09-08T15:51:14.553 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 19.848799655000001, 2020-09-08T15:51:14.553 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.053147530999999998, 2020-09-08T15:51:14.553 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:14.553 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:14.554 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:14.554 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.554 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:14.554 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.488311+0000", 2020-09-08T15:51:14.555 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:14.555 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.555 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.555 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:14.555 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.488311+0000", 2020-09-08T15:51:14.555 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.8461000000000003e-05 2020-09-08T15:51:14.556 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.556 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.556 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:14.556 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.488349+0000", 2020-09-08T15:51:14.556 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.2840000000000003e-06 2020-09-08T15:51:14.557 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.557 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.557 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:14.557 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.488355+0000", 2020-09-08T15:51:14.558 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.0710000000000001e-06 2020-09-08T15:51:14.558 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.564 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.564 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:14.565 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.488364+0000", 2020-09-08T15:51:14.565 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.5770000000000004e-06 2020-09-08T15:51:14.565 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.565 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.565 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:14.566 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.488369+0000", 2020-09-08T15:51:14.566 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.7280000000000001e-05 2020-09-08T15:51:14.566 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.566 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.566 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:14.567 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.488416+0000", 2020-09-08T15:51:14.567 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.6461999999999998e-05 2020-09-08T15:51:14.567 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.567 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.567 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:14.567 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.488443+0000", 2020-09-08T15:51:14.568 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.052969190999999999 2020-09-08T15:51:14.568 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.568 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.568 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:14.568 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.541412+0000", 2020-09-08T15:51:14.569 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.6205000000000002e-05 2020-09-08T15:51:14.569 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.569 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.569 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:14.569 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:54.541458+0000", 2020-09-08T15:51:14.570 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.053147530999999998 2020-09-08T15:51:14.570 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.570 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.570 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.570 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.571 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.571 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.35381.0:277897 4.7 e387/314 4:e7c1d7e0:::benchmark_data_smithi166_32054_object135755:head v 387'17208, mlcod=387'17204)", 2020-09-08T15:51:14.571 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:51:10.603378+0000", 2020-09-08T15:51:14.571 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 3.7337320790000001, 2020-09-08T15:51:14.571 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.062631204999999995, 2020-09-08T15:51:14.571 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:14.572 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:14.572 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:14.572 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.572 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:14.572 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.603378+0000", 2020-09-08T15:51:14.573 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:14.573 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.573 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.573 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:14.573 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.603378+0000", 2020-09-08T15:51:14.574 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9665e-05 2020-09-08T15:51:14.574 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.574 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.574 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:14.574 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.603428+0000", 2020-09-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7039999999999999e-06 2020-09-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.603434+0000", 2020-09-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.0340000000000002e-06 2020-09-08T15:51:14.576 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.576 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.576 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:14.576 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.603443+0000", 2020-09-08T15:51:14.576 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.5379999999999998e-06 2020-09-08T15:51:14.577 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.577 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.577 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:14.577 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.603446+0000", 2020-09-08T15:51:14.577 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0040690509999999997 2020-09-08T15:51:14.578 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.578 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.578 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:14.578 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.607515+0000", 2020-09-08T15:51:14.578 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.3352000000000002e-05 2020-09-08T15:51:14.579 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.579 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.579 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:14.579 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.607549+0000", 2020-09-08T15:51:14.579 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.058383123000000002 2020-09-08T15:51:14.579 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.580 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.580 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:14.580 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.665932+0000", 2020-09-08T15:51:14.580 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7737999999999993e-05 2020-09-08T15:51:14.580 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.581 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.581 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:14.581 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.666009+0000", 2020-09-08T15:51:14.581 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.062631204999999995 2020-09-08T15:51:14.581 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.582 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.582 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.582 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.582 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.582 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.35381.0:277900 4.7 e387/314 4:e8cea5c3:::benchmark_data_smithi166_32054_object135758:head v 387'17209, mlcod=387'17204)", 2020-09-08T15:51:14.583 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:51:10.610236+0000", 2020-09-08T15:51:14.583 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 3.7268744489999999, 2020-09-08T15:51:14.583 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.055824480000000003, 2020-09-08T15:51:14.583 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:14.584 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:14.584 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:14.584 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.584 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:14.584 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.610236+0000", 2020-09-08T15:51:14.585 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:14.585 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.585 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.585 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:14.585 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.610236+0000", 2020-09-08T15:51:14.586 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.3209999999999997e-05 2020-09-08T15:51:14.586 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.586 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.586 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:14.586 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.610329+0000", 2020-09-08T15:51:14.586 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1554e-05 2020-09-08T15:51:14.587 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.587 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.587 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:14.587 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.610341+0000", 2020-09-08T15:51:14.587 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.9310999999999999e-05 2020-09-08T15:51:14.588 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.588 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.588 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:14.588 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.610360+0000", 2020-09-08T15:51:14.589 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7300000000000005e-06 2020-09-08T15:51:14.589 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.589 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.590 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:14.590 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.610368+0000", 2020-09-08T15:51:14.590 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0021997679999999999 2020-09-08T15:51:14.590 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.590 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.591 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:14.591 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.612567+0000", 2020-09-08T15:51:14.591 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.0647000000000001e-05 2020-09-08T15:51:14.591 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.591 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.591 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:14.592 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.612588+0000", 2020-09-08T15:51:14.592 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.053428178999999999 2020-09-08T15:51:14.592 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.592 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.593 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:14.593 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.666016+0000", 2020-09-08T15:51:14.593 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.4081e-05 2020-09-08T15:51:14.593 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.594 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.594 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:14.594 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.666060+0000", 2020-09-08T15:51:14.594 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.055824480000000003 2020-09-08T15:51:14.594 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.595 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.595 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.595 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.595 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.595 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:277914 4.e 4:7afe995e:::benchmark_data_smithi166_32054_object135772:head [delete] snapc 0=[] ondisk+write+known_if_redirected e387)", 2020-09-08T15:51:14.596 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:51:10.614520+0000", 2020-09-08T15:51:14.596 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 3.7225898399999999, 2020-09-08T15:51:14.596 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.066007811, 2020-09-08T15:51:14.597 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:14.597 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:14.597 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:14.597 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:51:14.598 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:14.598 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 277914 2020-09-08T15:51:14.598 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.598 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:14.598 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.598 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:14.599 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.614520+0000", 2020-09-08T15:51:14.599 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:14.599 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.599 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.599 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:14.600 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.614520+0000", 2020-09-08T15:51:14.600 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999709 2020-09-08T15:51:14.600 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.600 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.600 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:14.601 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.614491+0000", 2020-09-08T15:51:14.601 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.4987000000000006e-05 2020-09-08T15:51:14.601 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.601 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.601 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:14.602 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.614556+0000", 2020-09-08T15:51:14.602 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1280999999999999e-05 2020-09-08T15:51:14.602 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.602 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.602 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:14.602 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.614568+0000", 2020-09-08T15:51:14.603 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.2410000000000003e-06 2020-09-08T15:51:14.603 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.603 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.603 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:14.604 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.614573+0000", 2020-09-08T15:51:14.604 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.1174000000000007e-05 2020-09-08T15:51:14.604 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.604 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.604 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:14.605 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.614644+0000", 2020-09-08T15:51:14.605 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00021111099999999999 2020-09-08T15:51:14.605 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.605 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.605 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:14.606 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.614855+0000", 2020-09-08T15:51:14.606 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018367700000000001 2020-09-08T15:51:14.606 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.606 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.606 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T15:51:14.607 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.615039+0000", 2020-09-08T15:51:14.607 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.065366698000000001 2020-09-08T15:51:14.607 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.607 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.607 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:14.608 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.680406+0000", 2020-09-08T15:51:14.608 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1378e-05 2020-09-08T15:51:14.608 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.608 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.608 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:51:14.608 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.680417+0000", 2020-09-08T15:51:14.609 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.2811000000000006e-05 2020-09-08T15:51:14.609 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.609 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.609 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:14.609 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.680480+0000", 2020-09-08T15:51:14.610 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.8607999999999999e-05 2020-09-08T15:51:14.610 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.610 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.610 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:14.610 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.680528+0000", 2020-09-08T15:51:14.611 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.066007811 2020-09-08T15:51:14.611 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.611 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.611 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.611 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.612 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.612 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:277915 4.e 4:7b0959a1:::benchmark_data_smithi166_32054_object135773:head [delete] snapc 0=[] ondisk+write+known_if_redirected e387)", 2020-09-08T15:51:14.612 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:51:10.615485+0000", 2020-09-08T15:51:14.612 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 3.7216256990000001, 2020-09-08T15:51:14.613 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.070905975999999996, 2020-09-08T15:51:14.613 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:14.613 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:14.613 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:14.613 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:51:14.614 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:14.614 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 277915 2020-09-08T15:51:14.614 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.614 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:14.614 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.615 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:14.615 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.615485+0000", 2020-09-08T15:51:14.615 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:14.616 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.616 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.616 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:14.616 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.615485+0000", 2020-09-08T15:51:14.616 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999514 2020-09-08T15:51:14.616 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.617 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.617 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:14.617 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.615436+0000", 2020-09-08T15:51:14.617 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011194 2020-09-08T15:51:14.617 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.618 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.618 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:14.618 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.615548+0000", 2020-09-08T15:51:14.618 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.8802999999999999e-05 2020-09-08T15:51:14.618 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.619 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.619 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:14.619 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.615567+0000", 2020-09-08T15:51:14.619 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.0620000000000007e-06 2020-09-08T15:51:14.619 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.620 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.620 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:14.620 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.615576+0000", 2020-09-08T15:51:14.620 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.051144519999999999 2020-09-08T15:51:14.620 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.620 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.621 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:14.621 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.666720+0000", 2020-09-08T15:51:14.621 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014878299999999999 2020-09-08T15:51:14.621 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.621 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.622 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:14.622 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.666869+0000", 2020-09-08T15:51:14.622 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013994300000000001 2020-09-08T15:51:14.622 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.622 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.623 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T15:51:14.623 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.667009+0000", 2020-09-08T15:51:14.623 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.013594059 2020-09-08T15:51:14.623 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.623 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.624 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:14.624 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.680603+0000", 2020-09-08T15:51:14.624 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0056700049999999997 2020-09-08T15:51:14.624 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.625 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.625 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:51:14.625 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.686273+0000", 2020-09-08T15:51:14.625 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.7667000000000001e-05 2020-09-08T15:51:14.625 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.626 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.626 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:14.626 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.686341+0000", 2020-09-08T15:51:14.626 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9920000000000003e-05 2020-09-08T15:51:14.626 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.626 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.627 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:14.627 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.686391+0000", 2020-09-08T15:51:14.627 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.070905975999999996 2020-09-08T15:51:14.627 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.627 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.628 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.628 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.628 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.628 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.35381.0:277917 4.5 e387/314 4:a0d7c64f:::benchmark_data_smithi166_32054_object135775:head v 387'17463, mlcod=387'17461)", 2020-09-08T15:51:14.628 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:51:10.616410+0000", 2020-09-08T15:51:14.629 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 3.7206998599999999, 2020-09-08T15:51:14.629 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.068131989000000004, 2020-09-08T15:51:14.629 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:14.629 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:14.629 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:14.630 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.630 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:14.630 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.616410+0000", 2020-09-08T15:51:14.630 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:14.630 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.630 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.631 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:14.631 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.616410+0000", 2020-09-08T15:51:14.631 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0102000000000002e-05 2020-09-08T15:51:14.631 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.631 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.632 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:14.632 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.616461+0000", 2020-09-08T15:51:14.633 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.4250000000000001e-06 2020-09-08T15:51:14.633 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.633 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.633 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:14.635 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.616468+0000", 2020-09-08T15:51:14.635 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1389999999999999e-05 2020-09-08T15:51:14.635 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.635 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.636 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:14.636 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.616479+0000", 2020-09-08T15:51:14.636 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.8709999999999996e-06 2020-09-08T15:51:14.636 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.636 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.637 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:14.637 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.616484+0000", 2020-09-08T15:51:14.637 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6179000000000005e-05 2020-09-08T15:51:14.637 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.637 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.638 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:14.638 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.616550+0000", 2020-09-08T15:51:14.638 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.9845000000000002e-05 2020-09-08T15:51:14.638 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.638 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.639 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:14.639 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.616590+0000", 2020-09-08T15:51:14.639 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.067875411999999996 2020-09-08T15:51:14.639 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.639 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.640 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:14.640 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.684466+0000", 2020-09-08T15:51:14.640 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.6765000000000001e-05 2020-09-08T15:51:14.640 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.640 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.641 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:14.641 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:10.684542+0000", 2020-09-08T15:51:14.641 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.068131989000000004 2020-09-08T15:51:14.641 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.641 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.642 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.642 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.642 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.642 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:280382 4.e 4:744c8c31:::benchmark_data_smithi166_32054_object138240:head [delete] snapc 0=[] ondisk+write+known_if_redirected e389)", 2020-09-08T15:51:14.642 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:51:13.169829+0000", 2020-09-08T15:51:14.642 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 1.1672811649999999, 2020-09-08T15:51:14.643 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.10546090599999999, 2020-09-08T15:51:14.643 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:14.643 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:14.643 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:14.643 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:51:14.644 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:14.644 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 280382 2020-09-08T15:51:14.644 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.644 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:14.644 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.645 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:14.645 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.169829+0000", 2020-09-08T15:51:14.645 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:14.645 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.645 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.646 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:14.646 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.169829+0000", 2020-09-08T15:51:14.646 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999614 2020-09-08T15:51:14.646 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.646 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.647 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:14.647 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.169791+0000", 2020-09-08T15:51:14.647 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.5712999999999997e-05 2020-09-08T15:51:14.647 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.647 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.647 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:14.648 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.169886+0000", 2020-09-08T15:51:14.648 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4147e-05 2020-09-08T15:51:14.648 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.648 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.649 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:14.649 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.169900+0000", 2020-09-08T15:51:14.649 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.3770000000000001e-06 2020-09-08T15:51:14.649 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.649 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.650 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:14.650 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.169907+0000", 2020-09-08T15:51:14.650 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.2961000000000003e-05 2020-09-08T15:51:14.650 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.650 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.651 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:14.651 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.169960+0000", 2020-09-08T15:51:14.651 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014278399999999999 2020-09-08T15:51:14.651 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.651 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.652 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:14.652 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.170103+0000", 2020-09-08T15:51:14.652 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000137701 2020-09-08T15:51:14.652 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.652 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.653 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T15:51:14.653 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.170240+0000", 2020-09-08T15:51:14.653 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.085407616000000006 2020-09-08T15:51:14.653 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.653 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.653 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:51:14.654 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.255648+0000", 2020-09-08T15:51:14.654 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.019525300999999998 2020-09-08T15:51:14.654 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.654 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.654 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:14.655 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.275173+0000", 2020-09-08T15:51:14.655 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.6288e-05 2020-09-08T15:51:14.655 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.655 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.655 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:14.656 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.275229+0000", 2020-09-08T15:51:14.656 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.0628999999999998e-05 2020-09-08T15:51:14.656 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.656 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.656 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:14.657 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.275290+0000", 2020-09-08T15:51:14.657 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.10546090599999999 2020-09-08T15:51:14.657 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.657 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.657 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.658 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.658 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.658 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.35381.0:280414 4.5 e389/314 4:a640fd0b:::benchmark_data_smithi166_32054_object138272:head v 389'17631, mlcod=389'17626)", 2020-09-08T15:51:14.658 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:51:13.255716+0000", 2020-09-08T15:51:14.658 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 1.081394757, 2020-09-08T15:51:14.658 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.049234739, 2020-09-08T15:51:14.659 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:14.659 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:14.659 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:14.659 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.659 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:14.660 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.255716+0000", 2020-09-08T15:51:14.660 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:14.660 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.660 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.660 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:14.661 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.255716+0000", 2020-09-08T15:51:14.661 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.2199e-05 2020-09-08T15:51:14.661 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.661 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.661 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:14.662 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.255758+0000", 2020-09-08T15:51:14.662 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.707e-06 2020-09-08T15:51:14.662 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.662 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.662 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:14.662 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.255762+0000", 2020-09-08T15:51:14.663 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.605e-06 2020-09-08T15:51:14.663 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.663 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.663 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:14.664 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.255770+0000", 2020-09-08T15:51:14.664 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.4530000000000001e-06 2020-09-08T15:51:14.664 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.664 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.664 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:14.665 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.255773+0000", 2020-09-08T15:51:14.665 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.019607195000000001 2020-09-08T15:51:14.665 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.665 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.665 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:14.666 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.275380+0000", 2020-09-08T15:51:14.666 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.4298e-05 2020-09-08T15:51:14.666 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.666 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.666 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:14.667 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.275404+0000", 2020-09-08T15:51:14.667 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.029474631000000001 2020-09-08T15:51:14.667 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.667 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.667 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:14.668 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.304879+0000", 2020-09-08T15:51:14.668 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.1650999999999996e-05 2020-09-08T15:51:14.668 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:14.668 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:14.668 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:14.668 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:13.304950+0000", 2020-09-08T15:51:14.669 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.049234739 2020-09-08T15:51:14.669 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.669 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.669 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.669 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:14.670 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:14.670 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:51:14.670 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:14.672 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:51:14 smithi055 bash[14301]: debug 2020-09-08T15:51:14.270+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:14.672 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:51:14 smithi104 bash[15273]: debug 2020-09-08T15:51:14.046+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:14.962 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:14 smithi104 bash[9203]: cluster 2020-09-08T15:51:13.789667+0000 mon.a (mon.0) 1484 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:51:14.963 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:14 smithi104 bash[9203]: audit 2020-09-08T15:51:13.799372+0000 mon.a (mon.0) 1485 : audit [INF] from='client.? v1:172.21.15.104:0/1728397069' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:51:14.963 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:14 smithi104 bash[9203]: cluster 2020-09-08T15:51:13.799416+0000 mon.a (mon.0) 1486 : cluster [DBG] osdmap e390: 12 total, 12 up, 12 in 2020-09-08T15:51:14.963 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:14 smithi104 bash[9203]: cluster 2020-09-08T15:51:13.803957+0000 mgr.x (mgr.24118) 717 : cluster [DBG] pgmap v1023: 17 pgs: 17 active+clean; 281 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T15:51:14.964 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:14 smithi104 bash[9203]: cluster 2020-09-08T15:51:14.185720+0000 osd.1 (osd.1) 78 : cluster [DBG] 4.4 scrub starts 2020-09-08T15:51:14.964 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:51:14 smithi104 bash[15273]: debug 2020-09-08T15:51:14.962+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:14.964 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:15.068 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:15.077 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:14 smithi166 bash[9446]: cluster 2020-09-08T15:51:13.789667+0000 mon.a (mon.0) 1484 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:51:15.077 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:14 smithi166 bash[9446]: audit 2020-09-08T15:51:13.799372+0000 mon.a (mon.0) 1485 : audit [INF] from='client.? v1:172.21.15.104:0/1728397069' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:51:15.077 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:14 smithi166 bash[9446]: cluster 2020-09-08T15:51:13.799416+0000 mon.a (mon.0) 1486 : cluster [DBG] osdmap e390: 12 total, 12 up, 12 in 2020-09-08T15:51:15.078 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:14 smithi166 bash[9446]: cluster 2020-09-08T15:51:13.803957+0000 mgr.x (mgr.24118) 717 : cluster [DBG] pgmap v1023: 17 pgs: 17 active+clean; 281 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T15:51:15.078 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:14 smithi166 bash[9446]: cluster 2020-09-08T15:51:14.185720+0000 osd.1 (osd.1) 78 : cluster [DBG] 4.4 scrub starts 2020-09-08T15:51:15.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:14 smithi055 bash[9296]: cluster 2020-09-08T15:51:13.789667+0000 mon.a (mon.0) 1484 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:51:15.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:14 smithi055 bash[9296]: audit 2020-09-08T15:51:13.799372+0000 mon.a (mon.0) 1485 : audit [INF] from='client.? v1:172.21.15.104:0/1728397069' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:51:15.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:14 smithi055 bash[9296]: cluster 2020-09-08T15:51:13.799416+0000 mon.a (mon.0) 1486 : cluster [DBG] osdmap e390: 12 total, 12 up, 12 in 2020-09-08T15:51:15.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:14 smithi055 bash[9296]: cluster 2020-09-08T15:51:13.803957+0000 mgr.x (mgr.24118) 717 : cluster [DBG] pgmap v1023: 17 pgs: 17 active+clean; 281 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T15:51:15.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:14 smithi055 bash[9296]: cluster 2020-09-08T15:51:14.185720+0000 osd.1 (osd.1) 78 : cluster [DBG] 4.4 scrub starts 2020-09-08T15:51:15.317 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:15.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:51:15 smithi166 bash[11928]: debug 2020-09-08T15:51:15.319+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:15.422 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:15.472 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2020-09-08T15:51:15.555 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:15.659 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:51:15.692 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:51:15.829 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:15 smithi166 bash[9446]: cluster 2020-09-08T15:51:14.280603+0000 osd.1 (osd.1) 79 : cluster [DBG] 4.4 scrub ok 2020-09-08T15:51:15.830 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:15 smithi166 bash[9446]: cluster 2020-09-08T15:51:14.707383+0000 osd.3 (osd.3) 51 : cluster [DBG] 4.c scrub starts 2020-09-08T15:51:15.830 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:15 smithi166 bash[9446]: cluster 2020-09-08T15:51:14.758135+0000 osd.6 (osd.6) 59 : cluster [DBG] 4.5 scrub starts 2020-09-08T15:51:15.830 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:15 smithi166 bash[9446]: cluster 2020-09-08T15:51:14.759898+0000 osd.3 (osd.3) 52 : cluster [DBG] 4.c scrub ok 2020-09-08T15:51:15.830 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:51:15 smithi166 bash[14254]: debug 2020-09-08T15:51:15.555+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:15.966 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:15 smithi104 bash[9203]: cluster 2020-09-08T15:51:14.280603+0000 osd.1 (osd.1) 79 : cluster [DBG] 4.4 scrub ok 2020-09-08T15:51:15.966 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:15 smithi104 bash[9203]: cluster 2020-09-08T15:51:14.707383+0000 osd.3 (osd.3) 51 : cluster [DBG] 4.c scrub starts 2020-09-08T15:51:15.966 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:15 smithi104 bash[9203]: cluster 2020-09-08T15:51:14.758135+0000 osd.6 (osd.6) 59 : cluster [DBG] 4.5 scrub starts 2020-09-08T15:51:15.967 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:15 smithi104 bash[9203]: cluster 2020-09-08T15:51:14.759898+0000 osd.3 (osd.3) 52 : cluster [DBG] 4.c scrub ok 2020-09-08T15:51:15.967 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:15 smithi104 bash[9203]: cluster 2020-09-08T15:51:14.790822+0000 osd.6 (osd.6) 60 : cluster [DBG] 4.5 scrub ok 2020-09-08T15:51:15.967 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:15 smithi104 bash[9203]: cluster 2020-09-08T15:51:15.218987+0000 osd.10 (osd.10) 39 : cluster [DBG] 4.e scrub starts 2020-09-08T15:51:15.967 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:15 smithi104 bash[9203]: cluster 2020-09-08T15:51:15.242366+0000 osd.10 (osd.10) 40 : cluster [DBG] 4.e scrub ok 2020-09-08T15:51:16.071 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:51:16.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:15 smithi166 bash[9446]: cluster 2020-09-08T15:51:14.790822+0000 osd.6 (osd.6) 60 : cluster [DBG] 4.5 scrub ok 2020-09-08T15:51:16.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:15 smithi166 bash[9446]: cluster 2020-09-08T15:51:15.218987+0000 osd.10 (osd.10) 39 : cluster [DBG] 4.e scrub starts 2020-09-08T15:51:16.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:15 smithi166 bash[9446]: cluster 2020-09-08T15:51:15.242366+0000 osd.10 (osd.10) 40 : cluster [DBG] 4.e scrub ok 2020-09-08T15:51:16.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:15 smithi055 bash[9296]: cluster 2020-09-08T15:51:14.280603+0000 osd.1 (osd.1) 79 : cluster [DBG] 4.4 scrub ok 2020-09-08T15:51:16.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:15 smithi055 bash[9296]: cluster 2020-09-08T15:51:14.707383+0000 osd.3 (osd.3) 51 : cluster [DBG] 4.c scrub starts 2020-09-08T15:51:16.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:15 smithi055 bash[9296]: cluster 2020-09-08T15:51:14.758135+0000 osd.6 (osd.6) 59 : cluster [DBG] 4.5 scrub starts 2020-09-08T15:51:16.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:15 smithi055 bash[9296]: cluster 2020-09-08T15:51:14.759898+0000 osd.3 (osd.3) 52 : cluster [DBG] 4.c scrub ok 2020-09-08T15:51:16.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:15 smithi055 bash[9296]: cluster 2020-09-08T15:51:14.790822+0000 osd.6 (osd.6) 60 : cluster [DBG] 4.5 scrub ok 2020-09-08T15:51:16.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:15 smithi055 bash[9296]: cluster 2020-09-08T15:51:15.218987+0000 osd.10 (osd.10) 39 : cluster [DBG] 4.e scrub starts 2020-09-08T15:51:16.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:15 smithi055 bash[9296]: cluster 2020-09-08T15:51:15.242366+0000 osd.10 (osd.10) 40 : cluster [DBG] 4.e scrub ok 2020-09-08T15:51:16.176 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:16.288 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:51:16 smithi104 bash[19922]: debug 2020-09-08T15:51:16.074+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:16.477 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:16.585 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:16.587 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:51:16.626 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:51:16.662 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:51:16.697 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:51:16.734 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:51:16.763 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:51:16.793 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:51:16.828 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:51:16 smithi166 bash[14254]: debug 2020-09-08T15:51:16.479+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:16.832 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:51:16.866 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:51:16.902 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:51:16.933 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:51:16.970 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:51:16.987 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:51:16.987 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:51:16.988 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:51:16.988 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:51:16.988 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:51:16.988 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:51:16.988 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:51:16.988 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:51:16.989 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:51:16.989 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:51:16.989 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:51:16.989 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:51:17.084 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:51:17.093 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:16 smithi104 bash[9203]: cluster 2020-09-08T15:51:15.804338+0000 mgr.x (mgr.24118) 718 : cluster [DBG] pgmap v1024: 17 pgs: 17 active+clean; 281 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 830 op/s 2020-09-08T15:51:17.093 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:16 smithi104 bash[9203]: audit 2020-09-08T15:51:16.555673+0000 mon.b (mon.1) 95 : audit [DBG] from='client.? v1:172.21.15.104:0/303891467' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:51:17.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:16 smithi166 bash[9446]: cluster 2020-09-08T15:51:15.804338+0000 mgr.x (mgr.24118) 718 : cluster [DBG] pgmap v1024: 17 pgs: 17 active+clean; 281 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 830 op/s 2020-09-08T15:51:17.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:16 smithi166 bash[9446]: audit 2020-09-08T15:51:16.555673+0000 mon.b (mon.1) 95 : audit [DBG] from='client.? v1:172.21.15.104:0/303891467' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:51:17.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:16 smithi055 bash[9296]: cluster 2020-09-08T15:51:15.804338+0000 mgr.x (mgr.24118) 718 : cluster [DBG] pgmap v1024: 17 pgs: 17 active+clean; 281 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 830 op/s 2020-09-08T15:51:17.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:16 smithi055 bash[9296]: audit 2020-09-08T15:51:16.555673+0000 mon.b (mon.1) 95 : audit [DBG] from='client.? v1:172.21.15.104:0/303891467' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:51:17.204 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:17.220 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Removed 142139 objects 2020-09-08T15:51:17.220 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Clean up completed and total clean up time :142.818 2020-09-08T15:51:17.237 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:51:17.308 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:17.460 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:51:17 smithi166 bash[11928]: debug 2020-09-08T15:51:17.203+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:17.461 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:17.566 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:17.689 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:17.789 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:51:17 smithi166 bash[11928]: debug 2020-09-08T15:51:17.691+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:17.789 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:51:17 smithi166 bash[18960]: debug 2020-09-08T15:51:17.463+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:17.794 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:18.291 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:18.396 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:18.573 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:51:18 smithi104 bash[22305]: debug 2020-09-08T15:51:18.294+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:19.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:18 smithi166 bash[9446]: cluster 2020-09-08T15:51:17.805003+0000 mgr.x (mgr.24118) 719 : cluster [DBG] pgmap v1025: 17 pgs: 17 active+clean; 118 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T15:51:19.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:18 smithi055 bash[9296]: cluster 2020-09-08T15:51:17.805003+0000 mgr.x (mgr.24118) 719 : cluster [DBG] pgmap v1025: 17 pgs: 17 active+clean; 118 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T15:51:19.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:18 smithi104 bash[9203]: cluster 2020-09-08T15:51:17.805003+0000 mgr.x (mgr.24118) 719 : cluster [DBG] pgmap v1025: 17 pgs: 17 active+clean; 118 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T15:51:19.865 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:51:19.866 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:51:19.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:19 smithi166 bash[9446]: cluster 2020-09-08T15:51:18.739236+0000 osd.6 (osd.6) 61 : cluster [DBG] 4.a scrub starts 2020-09-08T15:51:19.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:19 smithi166 bash[9446]: cluster 2020-09-08T15:51:18.757648+0000 osd.6 (osd.6) 62 : cluster [DBG] 4.a scrub ok 2020-09-08T15:51:19.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:19 smithi166 bash[9446]: cluster 2020-09-08T15:51:19.045197+0000 osd.5 (osd.5) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:19.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:19 smithi166 bash[9446]: cluster 2020-09-08T15:51:19.045338+0000 osd.5 (osd.5) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:19.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:19 smithi166 bash[9446]: cluster 2020-09-08T15:51:19.083025+0000 osd.9 (osd.9) 53 : cluster [DBG] 4.2 scrub starts 2020-09-08T15:51:19.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:19 smithi166 bash[9446]: cluster 2020-09-08T15:51:19.104302+0000 osd.9 (osd.9) 54 : cluster [DBG] 4.2 scrub ok 2020-09-08T15:51:20.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:19 smithi055 bash[9296]: cluster 2020-09-08T15:51:18.739236+0000 osd.6 (osd.6) 61 : cluster [DBG] 4.a scrub starts 2020-09-08T15:51:20.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:19 smithi055 bash[9296]: cluster 2020-09-08T15:51:18.757648+0000 osd.6 (osd.6) 62 : cluster [DBG] 4.a scrub ok 2020-09-08T15:51:20.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:19 smithi055 bash[9296]: cluster 2020-09-08T15:51:19.045197+0000 osd.5 (osd.5) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:20.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:19 smithi055 bash[9296]: cluster 2020-09-08T15:51:19.045338+0000 osd.5 (osd.5) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:20.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:19 smithi055 bash[9296]: cluster 2020-09-08T15:51:19.083025+0000 osd.9 (osd.9) 53 : cluster [DBG] 4.2 scrub starts 2020-09-08T15:51:20.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:19 smithi055 bash[9296]: cluster 2020-09-08T15:51:19.104302+0000 osd.9 (osd.9) 54 : cluster [DBG] 4.2 scrub ok 2020-09-08T15:51:20.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:19 smithi104 bash[9203]: cluster 2020-09-08T15:51:18.739236+0000 osd.6 (osd.6) 61 : cluster [DBG] 4.a scrub starts 2020-09-08T15:51:20.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:19 smithi104 bash[9203]: cluster 2020-09-08T15:51:18.757648+0000 osd.6 (osd.6) 62 : cluster [DBG] 4.a scrub ok 2020-09-08T15:51:20.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:19 smithi104 bash[9203]: cluster 2020-09-08T15:51:19.045197+0000 osd.5 (osd.5) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:20.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:19 smithi104 bash[9203]: cluster 2020-09-08T15:51:19.045338+0000 osd.5 (osd.5) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:20.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:19 smithi104 bash[9203]: cluster 2020-09-08T15:51:19.083025+0000 osd.9 (osd.9) 53 : cluster [DBG] 4.2 scrub starts 2020-09-08T15:51:20.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:19 smithi104 bash[9203]: cluster 2020-09-08T15:51:19.104302+0000 osd.9 (osd.9) 54 : cluster [DBG] 4.2 scrub ok 2020-09-08T15:51:20.555 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2020-09-08T15:51:20.575 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:51:20.675 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:20.685 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:51:20.706 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:51:20.780 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:20.899 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:21.004 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:51:21.017 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:20 smithi055 bash[9296]: cluster 2020-09-08T15:51:19.805750+0000 mgr.x (mgr.24118) 720 : cluster [DBG] pgmap v1026: 17 pgs: 17 active+clean; 12 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T15:51:21.018 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:20 smithi055 bash[9296]: cluster 2020-09-08T15:51:19.848517+0000 mon.a (mon.0) 1487 : cluster [DBG] osdmap e391: 12 total, 12 up, 12 in 2020-09-08T15:51:21.018 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:51:20 smithi055 bash[14301]: debug 2020-09-08T15:51:20.674+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:21.018 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:51:20 smithi055 bash[14301]: debug 2020-09-08T15:51:20.898+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:21.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:20 smithi166 bash[9446]: cluster 2020-09-08T15:51:19.805750+0000 mgr.x (mgr.24118) 720 : cluster [DBG] pgmap v1026: 17 pgs: 17 active+clean; 12 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T15:51:21.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:20 smithi166 bash[9446]: cluster 2020-09-08T15:51:19.848517+0000 mon.a (mon.0) 1487 : cluster [DBG] osdmap e391: 12 total, 12 up, 12 in 2020-09-08T15:51:21.130 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:51:21.158 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:20 smithi104 bash[9203]: cluster 2020-09-08T15:51:19.805750+0000 mgr.x (mgr.24118) 720 : cluster [DBG] pgmap v1026: 17 pgs: 17 active+clean; 12 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T15:51:21.158 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:20 smithi104 bash[9203]: cluster 2020-09-08T15:51:19.848517+0000 mon.a (mon.0) 1487 : cluster [DBG] osdmap e391: 12 total, 12 up, 12 in 2020-09-08T15:51:21.235 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:21.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:51:21 smithi055 bash[16646]: debug 2020-09-08T15:51:21.130+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:21.455 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_2 2020-09-08T15:51:21.456 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool rm unique_pool_2 unique_pool_2 --yes-i-really-really-mean-it 2020-09-08T15:51:21.582 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:51:21.593 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:21.695 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:51:21.768 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:51:21.851 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:51:21 smithi104 bash[17588]: debug 2020-09-08T15:51:21.594+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:22.066 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:21 smithi166 bash[9446]: cluster 2020-09-08T15:51:20.810489+0000 osd.10 (osd.10) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:22.066 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:21 smithi166 bash[9446]: cluster 2020-09-08T15:51:20.810595+0000 osd.10 (osd.10) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:22.066 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:21 smithi166 bash[9446]: audit 2020-09-08T15:51:21.585887+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? v1:172.21.15.104:0/556142346' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:51:22.067 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:21 smithi166 bash[9446]: cluster 2020-09-08T15:51:21.763406+0000 mon.a (mon.0) 1489 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:51:22.067 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:21 smithi166 bash[9446]: cluster 2020-09-08T15:51:21.763450+0000 mon.a (mon.0) 1490 : cluster [INF] Cluster is now healthy 2020-09-08T15:51:22.067 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:21 smithi166 bash[9446]: audit 2020-09-08T15:51:21.770524+0000 mon.a (mon.0) 1491 : audit [INF] from='client.? v1:172.21.15.104:0/556142346' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:51:22.068 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:21 smithi166 bash[9446]: cluster 2020-09-08T15:51:21.770588+0000 mon.a (mon.0) 1492 : cluster [DBG] osdmap e392: 12 total, 12 up, 12 in 2020-09-08T15:51:22.096 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:51:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:21 smithi055 bash[9296]: cluster 2020-09-08T15:51:20.810489+0000 osd.10 (osd.10) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:21 smithi055 bash[9296]: cluster 2020-09-08T15:51:20.810595+0000 osd.10 (osd.10) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:22.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:21 smithi055 bash[9296]: audit 2020-09-08T15:51:21.585887+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? v1:172.21.15.104:0/556142346' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:51:22.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:21 smithi055 bash[9296]: cluster 2020-09-08T15:51:21.763406+0000 mon.a (mon.0) 1489 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:51:22.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:21 smithi055 bash[9296]: cluster 2020-09-08T15:51:21.763450+0000 mon.a (mon.0) 1490 : cluster [INF] Cluster is now healthy 2020-09-08T15:51:22.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:21 smithi055 bash[9296]: audit 2020-09-08T15:51:21.770524+0000 mon.a (mon.0) 1491 : audit [INF] from='client.? v1:172.21.15.104:0/556142346' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:51:22.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:21 smithi055 bash[9296]: cluster 2020-09-08T15:51:21.770588+0000 mon.a (mon.0) 1492 : cluster [DBG] osdmap e392: 12 total, 12 up, 12 in 2020-09-08T15:51:22.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:51:22 smithi104 bash[19922]: debug 2020-09-08T15:51:22.098+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:22.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:21 smithi104 bash[9203]: cluster 2020-09-08T15:51:20.810489+0000 osd.10 (osd.10) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:22.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:21 smithi104 bash[9203]: cluster 2020-09-08T15:51:20.810595+0000 osd.10 (osd.10) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:22.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:21 smithi104 bash[9203]: audit 2020-09-08T15:51:21.585887+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? v1:172.21.15.104:0/556142346' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:51:22.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:21 smithi104 bash[9203]: cluster 2020-09-08T15:51:21.763406+0000 mon.a (mon.0) 1489 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:51:22.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:21 smithi104 bash[9203]: cluster 2020-09-08T15:51:21.763450+0000 mon.a (mon.0) 1490 : cluster [INF] Cluster is now healthy 2020-09-08T15:51:22.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:21 smithi104 bash[9203]: audit 2020-09-08T15:51:21.770524+0000 mon.a (mon.0) 1491 : audit [INF] from='client.? v1:172.21.15.104:0/556142346' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:51:22.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:21 smithi104 bash[9203]: cluster 2020-09-08T15:51:21.770588+0000 mon.a (mon.0) 1492 : cluster [DBG] osdmap e392: 12 total, 12 up, 12 in 2020-09-08T15:51:22.199 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:22.731 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:22.837 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:23.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:22 smithi166 bash[9446]: cluster 2020-09-08T15:51:21.730089+0000 osd.3 (osd.3) 53 : cluster [DBG] 4.0 scrub starts 2020-09-08T15:51:23.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:22 smithi166 bash[9446]: cluster 2020-09-08T15:51:21.753021+0000 osd.3 (osd.3) 54 : cluster [DBG] 4.0 scrub ok 2020-09-08T15:51:23.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:22 smithi166 bash[9446]: cluster 2020-09-08T15:51:21.806405+0000 mgr.x (mgr.24118) 721 : cluster [DBG] pgmap v1029: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.4 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 553 op/s 2020-09-08T15:51:23.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:22 smithi166 bash[9446]: cluster 2020-09-08T15:51:22.023773+0000 osd.11 (osd.11) 64 : cluster [DBG] 4.3 scrub starts 2020-09-08T15:51:23.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:22 smithi166 bash[9446]: cluster 2020-09-08T15:51:22.035618+0000 osd.11 (osd.11) 65 : cluster [DBG] 4.3 scrub ok 2020-09-08T15:51:23.122 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:22 smithi055 bash[9296]: cluster 2020-09-08T15:51:21.730089+0000 osd.3 (osd.3) 53 : cluster [DBG] 4.0 scrub starts 2020-09-08T15:51:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:22 smithi055 bash[9296]: cluster 2020-09-08T15:51:21.753021+0000 osd.3 (osd.3) 54 : cluster [DBG] 4.0 scrub ok 2020-09-08T15:51:23.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:22 smithi055 bash[9296]: cluster 2020-09-08T15:51:21.806405+0000 mgr.x (mgr.24118) 721 : cluster [DBG] pgmap v1029: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.4 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 553 op/s 2020-09-08T15:51:23.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:22 smithi055 bash[9296]: cluster 2020-09-08T15:51:22.023773+0000 osd.11 (osd.11) 64 : cluster [DBG] 4.3 scrub starts 2020-09-08T15:51:23.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:22 smithi055 bash[9296]: cluster 2020-09-08T15:51:22.035618+0000 osd.11 (osd.11) 65 : cluster [DBG] 4.3 scrub ok 2020-09-08T15:51:23.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:51:22 smithi104 bash[17588]: debug 2020-09-08T15:51:22.734+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:23.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:22 smithi104 bash[9203]: cluster 2020-09-08T15:51:21.730089+0000 osd.3 (osd.3) 53 : cluster [DBG] 4.0 scrub starts 2020-09-08T15:51:23.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:22 smithi104 bash[9203]: cluster 2020-09-08T15:51:21.753021+0000 osd.3 (osd.3) 54 : cluster [DBG] 4.0 scrub ok 2020-09-08T15:51:23.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:22 smithi104 bash[9203]: cluster 2020-09-08T15:51:21.806405+0000 mgr.x (mgr.24118) 721 : cluster [DBG] pgmap v1029: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.4 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 553 op/s 2020-09-08T15:51:23.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:22 smithi104 bash[9203]: cluster 2020-09-08T15:51:22.023773+0000 osd.11 (osd.11) 64 : cluster [DBG] 4.3 scrub starts 2020-09-08T15:51:23.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:22 smithi104 bash[9203]: cluster 2020-09-08T15:51:22.035618+0000 osd.11 (osd.11) 65 : cluster [DBG] 4.3 scrub ok 2020-09-08T15:51:23.232 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:23.587 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:51:23 smithi166 bash[14254]: debug 2020-09-08T15:51:23.123+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:23.588 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:23.693 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:51:23.872 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:51:23 smithi166 bash[11928]: debug 2020-09-08T15:51:23.587+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:23.878 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:51:23.981 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:23.993 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:51:23 smithi055 bash[11956]: debug 2020-09-08T15:51:23.878+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:24.116 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:24.171 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:51:24.171 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:51:24.220 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:24.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:51:24 smithi055 bash[14301]: debug 2020-09-08T15:51:24.114+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:24.428 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:24.535 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:24.546 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:51:24 smithi104 bash[15273]: debug 2020-09-08T15:51:24.430+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:24.903 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2020-09-08T15:51:24.926 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:25.031 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:25.041 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:51:25.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:24 smithi166 bash[9446]: cluster 2020-09-08T15:51:23.807266+0000 mgr.x (mgr.24118) 722 : cluster [DBG] pgmap v1030: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 562 op/s 2020-09-08T15:51:25.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:24 smithi055 bash[9296]: cluster 2020-09-08T15:51:23.807266+0000 mgr.x (mgr.24118) 722 : cluster [DBG] pgmap v1030: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 562 op/s 2020-09-08T15:51:25.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:51:24 smithi104 bash[15273]: debug 2020-09-08T15:51:24.926+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:25.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:24 smithi104 bash[9203]: cluster 2020-09-08T15:51:23.807266+0000 mgr.x (mgr.24118) 722 : cluster [DBG] pgmap v1030: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 562 op/s 2020-09-08T15:51:25.359 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:25.464 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:51:25.608 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:51:25.631 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:51:25 smithi166 bash[11928]: debug 2020-09-08T15:51:25.359+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:25.690 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:51:25.712 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:51:25.823 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:51:25.840 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:51:25 smithi055 bash[16646]: debug 2020-09-08T15:51:25.606+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:25.841 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:51:25.946 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:26.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:51:25 smithi055 bash[11956]: debug 2020-09-08T15:51:25.842+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:26.172 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:26.276 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:26.320 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:51:26 smithi166 bash[11928]: debug 2020-09-08T15:51:26.171+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:26.387 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:26.493 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:26.620 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:51:26 smithi055 bash[14301]: debug 2020-09-08T15:51:26.386+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:26.621 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:26.727 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:26.808 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:51:26.836 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:51:26.859 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:51:26.875 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:51:26 smithi055 bash[19003]: debug 2020-09-08T15:51:26.618+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:26.892 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:51:26.919 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:51:26.949 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:51:26.976 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:51:27.003 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:51:27.030 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:51:27.055 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:51:27.089 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:51:27.121 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:51:27.122 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:51:27.122 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:51:27.122 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:51:27.122 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:51:27.122 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:51:27.123 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:51:27.123 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:51:27.123 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:51:27.123 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:51:27.124 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:51:27.124 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:51:27.124 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:51:27.124 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:26 smithi166 bash[9446]: cluster 2020-09-08T15:51:25.808087+0000 mgr.x (mgr.24118) 723 : cluster [DBG] pgmap v1031: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 236 op/s 2020-09-08T15:51:27.124 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:26 smithi166 bash[9446]: cluster 2020-09-08T15:51:26.020064+0000 osd.3 (osd.3) 55 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:27.125 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:26 smithi166 bash[9446]: cluster 2020-09-08T15:51:26.020191+0000 osd.3 (osd.3) 56 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:27.125 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:26 smithi166 bash[9446]: cluster 2020-09-08T15:51:26.202140+0000 osd.9 (osd.9) 55 : cluster [DBG] 4.d scrub starts 2020-09-08T15:51:27.125 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:26 smithi166 bash[9446]: cluster 2020-09-08T15:51:26.212663+0000 osd.9 (osd.9) 56 : cluster [DBG] 4.d scrub ok 2020-09-08T15:51:27.125 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:26 smithi166 bash[9446]: audit 2020-09-08T15:51:26.785669+0000 mon.a (mon.0) 1493 : audit [DBG] from='client.? v1:172.21.15.104:0/2699810859' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:51:27.146 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:27.150 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:26 smithi104 bash[9203]: cluster 2020-09-08T15:51:25.808087+0000 mgr.x (mgr.24118) 723 : cluster [DBG] pgmap v1031: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 236 op/s 2020-09-08T15:51:27.151 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:26 smithi104 bash[9203]: cluster 2020-09-08T15:51:26.020064+0000 osd.3 (osd.3) 55 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:27.151 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:26 smithi104 bash[9203]: cluster 2020-09-08T15:51:26.020191+0000 osd.3 (osd.3) 56 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:27.152 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:26 smithi104 bash[9203]: cluster 2020-09-08T15:51:26.202140+0000 osd.9 (osd.9) 55 : cluster [DBG] 4.d scrub starts 2020-09-08T15:51:27.152 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:26 smithi104 bash[9203]: cluster 2020-09-08T15:51:26.212663+0000 osd.9 (osd.9) 56 : cluster [DBG] 4.d scrub ok 2020-09-08T15:51:27.152 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:26 smithi104 bash[9203]: audit 2020-09-08T15:51:26.785669+0000 mon.a (mon.0) 1493 : audit [DBG] from='client.? v1:172.21.15.104:0/2699810859' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:51:27.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:26 smithi055 bash[9296]: cluster 2020-09-08T15:51:25.808087+0000 mgr.x (mgr.24118) 723 : cluster [DBG] pgmap v1031: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 236 op/s 2020-09-08T15:51:27.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:26 smithi055 bash[9296]: cluster 2020-09-08T15:51:26.020064+0000 osd.3 (osd.3) 55 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:27.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:26 smithi055 bash[9296]: cluster 2020-09-08T15:51:26.020191+0000 osd.3 (osd.3) 56 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:27.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:26 smithi055 bash[9296]: cluster 2020-09-08T15:51:26.202140+0000 osd.9 (osd.9) 55 : cluster [DBG] 4.d scrub starts 2020-09-08T15:51:27.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:26 smithi055 bash[9296]: cluster 2020-09-08T15:51:26.212663+0000 osd.9 (osd.9) 56 : cluster [DBG] 4.d scrub ok 2020-09-08T15:51:27.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:26 smithi055 bash[9296]: audit 2020-09-08T15:51:26.785669+0000 mon.a (mon.0) 1493 : audit [DBG] from='client.? v1:172.21.15.104:0/2699810859' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:51:27.250 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:27.398 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:27.500 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:27.617 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:51:27 smithi166 bash[18960]: debug 2020-09-08T15:51:27.147+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:27.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:51:27 smithi055 bash[19003]: debug 2020-09-08T15:51:27.390+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:27.789 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:27.892 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'unique_pool_2' removed 2020-09-08T15:51:27.894 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:28.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:27 smithi055 bash[9296]: cluster 2020-09-08T15:51:26.873475+0000 mon.a (mon.0) 1494 : cluster [DBG] osdmap e393: 12 total, 12 up, 12 in 2020-09-08T15:51:28.524 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:27 smithi055 bash[9296]: audit 2020-09-08T15:51:27.470167+0000 mon.a (mon.0) 1495 : audit [INF] from='client.? v1:172.21.15.104:0/1950612474' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T15:51:28.526 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:27 smithi166 bash[9446]: cluster 2020-09-08T15:51:26.873475+0000 mon.a (mon.0) 1494 : cluster [DBG] osdmap e393: 12 total, 12 up, 12 in 2020-09-08T15:51:28.527 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:27 smithi166 bash[9446]: audit 2020-09-08T15:51:27.470167+0000 mon.a (mon.0) 1495 : audit [INF] from='client.? v1:172.21.15.104:0/1950612474' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T15:51:28.527 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:51:27 smithi104 bash[15273]: debug 2020-09-08T15:51:27.790+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:28.527 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:27 smithi104 bash[9203]: cluster 2020-09-08T15:51:26.873475+0000 mon.a (mon.0) 1494 : cluster [DBG] osdmap e393: 12 total, 12 up, 12 in 2020-09-08T15:51:28.528 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:27 smithi104 bash[9203]: audit 2020-09-08T15:51:27.470167+0000 mon.a (mon.0) 1495 : audit [INF] from='client.? v1:172.21.15.104:0/1950612474' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T15:51:28.528 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:28.629 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:28.688 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:51:28 smithi104 bash[17588]: debug 2020-09-08T15:51:28.274+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:28.784 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:28.791 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:51:28.791 INFO:teuthology.orchestra.run.smithi166.stdout: "size": 20, 2020-09-08T15:51:28.791 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 600, 2020-09-08T15:51:28.792 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T15:51:28.792 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.792 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88121 3.c 3.4d8f2e4c (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:51:28.792 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.700024+0000", 2020-09-08T15:51:28.792 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 506.091116683, 2020-09-08T15:51:28.793 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.97689150199999997, 2020-09-08T15:51:28.793 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:28.794 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:51:28.794 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:28.794 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:51:28.794 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:28.794 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88121 2020-09-08T15:51:28.795 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.795 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:28.795 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.795 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:28.795 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.700024+0000", 2020-09-08T15:51:28.796 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:28.796 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.796 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.796 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:28.796 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.700024+0000", 2020-09-08T15:51:28.797 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999032 2020-09-08T15:51:28.797 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.797 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.797 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:28.797 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.699927+0000", 2020-09-08T15:51:28.798 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018412100000000001 2020-09-08T15:51:28.798 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.798 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.798 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:28.799 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.700111+0000", 2020-09-08T15:51:28.799 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4973999999999999e-05 2020-09-08T15:51:28.799 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.799 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.800 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:28.800 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.700126+0000", 2020-09-08T15:51:28.800 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.4910000000000002e-06 2020-09-08T15:51:28.800 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.800 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.801 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:28.801 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.700133+0000", 2020-09-08T15:51:28.801 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6443000000000001e-05 2020-09-08T15:51:28.801 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.801 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.802 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.802 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.700199+0000", 2020-09-08T15:51:28.802 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.6340000000000007e-06 2020-09-08T15:51:28.802 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.802 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.803 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:51:28.803 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.700208+0000", 2020-09-08T15:51:28.803 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.97667827799999996 2020-09-08T15:51:28.803 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.804 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.804 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.804 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.676886+0000", 2020-09-08T15:51:28.804 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.9227999999999999e-05 2020-09-08T15:51:28.804 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.805 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.805 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:28.805 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.676915+0000", 2020-09-08T15:51:28.806 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.97689150199999997 2020-09-08T15:51:28.806 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.806 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:28.806 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.807 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.807 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.807 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88124 3.c 3.62cfb55c (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:51:28.807 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.701342+0000", 2020-09-08T15:51:28.807 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 506.08979826400002, 2020-09-08T15:51:28.808 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.97560956499999996, 2020-09-08T15:51:28.808 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:28.808 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:51:28.808 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:28.808 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:51:28.809 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:28.809 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88124 2020-09-08T15:51:28.809 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.809 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:28.809 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.810 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:28.810 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.701342+0000", 2020-09-08T15:51:28.810 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:28.810 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.810 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.811 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:28.811 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.701342+0000", 2020-09-08T15:51:28.811 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999804 2020-09-08T15:51:28.811 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.811 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.812 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:28.812 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.701322+0000", 2020-09-08T15:51:28.812 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.8744000000000005e-05 2020-09-08T15:51:28.812 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.812 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.813 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:28.813 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.701391+0000", 2020-09-08T15:51:28.813 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.3339999999999996e-06 2020-09-08T15:51:28.813 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.813 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.814 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:28.814 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.701398+0000", 2020-09-08T15:51:28.814 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.0599999999999999e-06 2020-09-08T15:51:28.814 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.814 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.815 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:28.815 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.701401+0000", 2020-09-08T15:51:28.815 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.8257999999999998e-05 2020-09-08T15:51:28.815 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.815 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.816 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.816 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.701450+0000", 2020-09-08T15:51:28.816 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0329000000000001e-05 2020-09-08T15:51:28.816 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.816 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.817 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:51:28.817 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.701460+0000", 2020-09-08T15:51:28.817 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.97547700800000003 2020-09-08T15:51:28.817 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.817 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.817 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.818 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.676937+0000", 2020-09-08T15:51:28.818 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4528999999999999e-05 2020-09-08T15:51:28.818 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.818 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.818 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:28.819 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.676952+0000", 2020-09-08T15:51:28.819 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.97560956499999996 2020-09-08T15:51:28.819 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.819 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:28.819 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.820 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.820 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.820 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88161 3.c 3.f471261c (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:51:28.820 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.730886+0000", 2020-09-08T15:51:28.820 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 506.06025405000003, 2020-09-08T15:51:28.821 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.94609881299999998, 2020-09-08T15:51:28.821 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:28.821 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:51:28.821 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:28.821 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:51:28.822 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:28.822 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88161 2020-09-08T15:51:28.822 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.822 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:28.822 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.823 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:28.823 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.730886+0000", 2020-09-08T15:51:28.823 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:28.824 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.824 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.824 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:28.824 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.730886+0000", 2020-09-08T15:51:28.824 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999766 2020-09-08T15:51:28.825 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.825 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.825 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:28.825 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.730863+0000", 2020-09-08T15:51:28.825 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.6293999999999997e-05 2020-09-08T15:51:28.826 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.826 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.826 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:28.826 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.730959+0000", 2020-09-08T15:51:28.826 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.1086000000000001e-05 2020-09-08T15:51:28.827 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.827 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.827 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:28.827 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.730980+0000", 2020-09-08T15:51:28.827 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.0809999999999996e-06 2020-09-08T15:51:28.827 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.828 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.828 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:28.828 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.730987+0000", 2020-09-08T15:51:28.828 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6370000000000003e-05 2020-09-08T15:51:28.829 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.829 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.829 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.829 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.731054+0000", 2020-09-08T15:51:28.829 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.7700000000000007e-06 2020-09-08T15:51:28.830 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.830 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.830 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:51:28.830 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.731062+0000", 2020-09-08T15:51:28.830 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.94591566199999999 2020-09-08T15:51:28.831 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.831 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.831 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.831 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.676978+0000", 2020-09-08T15:51:28.831 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.9609999999999999e-06 2020-09-08T15:51:28.832 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.832 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.832 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:28.832 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.676985+0000", 2020-09-08T15:51:28.833 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.94609881299999998 2020-09-08T15:51:28.833 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.833 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:28.834 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.834 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.834 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.834 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88165 3.c 3.9f2f3bbc (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:51:28.835 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.732971+0000", 2020-09-08T15:51:28.835 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 506.05816922899999, 2020-09-08T15:51:28.835 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.94403996000000001, 2020-09-08T15:51:28.835 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:28.835 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:51:28.836 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:28.836 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:51:28.836 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:28.836 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88165 2020-09-08T15:51:28.837 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.837 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:28.837 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.837 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:28.837 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.732971+0000", 2020-09-08T15:51:28.838 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:28.838 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.838 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.838 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:28.839 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.732971+0000", 2020-09-08T15:51:28.839 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999523 2020-09-08T15:51:28.839 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.839 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.839 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:28.840 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.732923+0000", 2020-09-08T15:51:28.840 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015177800000000001 2020-09-08T15:51:28.840 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.840 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.840 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:28.841 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.733075+0000", 2020-09-08T15:51:28.841 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6350000000000001e-05 2020-09-08T15:51:28.841 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.841 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.842 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:28.842 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.733092+0000", 2020-09-08T15:51:28.842 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.1280000000000002e-06 2020-09-08T15:51:28.842 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.842 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.843 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:28.843 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.733099+0000", 2020-09-08T15:51:28.843 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5586000000000005e-05 2020-09-08T15:51:28.843 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.843 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.844 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.844 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.733164+0000", 2020-09-08T15:51:28.844 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.013e-05 2020-09-08T15:51:28.844 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.845 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.845 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:51:28.845 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.733174+0000", 2020-09-08T15:51:28.845 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.943830006 2020-09-08T15:51:28.846 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.846 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.846 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.846 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677004+0000", 2020-09-08T15:51:28.847 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.596e-06 2020-09-08T15:51:28.847 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.847 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.847 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:28.848 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677011+0000", 2020-09-08T15:51:28.848 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.94403996000000001 2020-09-08T15:51:28.848 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.848 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:28.848 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.849 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.849 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.849 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88180 3.c 3.a7f3fb9c (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:51:28.849 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.746178+0000", 2020-09-08T15:51:28.849 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 506.04496235800002, 2020-09-08T15:51:28.850 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.93086361399999995, 2020-09-08T15:51:28.850 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:28.850 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:51:28.850 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:28.851 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:51:28.851 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:28.851 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88180 2020-09-08T15:51:28.851 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.851 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:28.852 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.852 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:28.852 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.746178+0000", 2020-09-08T15:51:28.852 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:28.853 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.853 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.853 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:28.853 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.746178+0000", 2020-09-08T15:51:28.854 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999104 2020-09-08T15:51:28.854 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.854 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.855 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:28.855 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.746088+0000", 2020-09-08T15:51:28.855 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00020530400000000001 2020-09-08T15:51:28.855 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.856 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.856 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:28.856 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.746294+0000", 2020-09-08T15:51:28.856 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.1090999999999998e-05 2020-09-08T15:51:28.857 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.857 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.857 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:28.858 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.746315+0000", 2020-09-08T15:51:28.858 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.5599999999999994e-06 2020-09-08T15:51:28.858 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.858 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.859 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:28.859 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.746323+0000", 2020-09-08T15:51:28.859 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.5589000000000001e-05 2020-09-08T15:51:28.859 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.860 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.860 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.860 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.746359+0000", 2020-09-08T15:51:28.862 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.0600000000000008e-06 2020-09-08T15:51:28.862 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.862 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.863 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:51:28.863 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.746367+0000", 2020-09-08T15:51:28.863 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.93066255799999997 2020-09-08T15:51:28.864 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.864 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.864 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.865 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677030+0000", 2020-09-08T15:51:28.865 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.203e-05 2020-09-08T15:51:28.865 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.866 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.866 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:28.866 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677042+0000", 2020-09-08T15:51:28.867 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.93086361399999995 2020-09-08T15:51:28.867 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.867 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:28.868 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.868 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.868 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.869 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88185 3.c 3.a38a334c (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:51:28.869 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.747647+0000", 2020-09-08T15:51:28.869 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 506.04349330299999, 2020-09-08T15:51:28.870 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.92947780899999999, 2020-09-08T15:51:28.870 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:28.870 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:51:28.871 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:28.871 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:51:28.871 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:28.871 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88185 2020-09-08T15:51:28.872 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.872 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:28.872 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.872 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:28.872 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.747647+0000", 2020-09-08T15:51:28.873 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:28.873 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.873 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.873 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:28.873 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.747647+0000", 2020-09-08T15:51:28.874 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999719 2020-09-08T15:51:28.874 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.874 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.874 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:28.874 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.747619+0000", 2020-09-08T15:51:28.875 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010487199999999999 2020-09-08T15:51:28.875 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.875 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.875 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:28.875 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.747724+0000", 2020-09-08T15:51:28.875 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0954e-05 2020-09-08T15:51:28.876 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.876 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.876 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:28.876 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.747735+0000", 2020-09-08T15:51:28.876 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.2039999999999996e-06 2020-09-08T15:51:28.877 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.877 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.877 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:28.877 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.747740+0000", 2020-09-08T15:51:28.877 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.1072000000000002e-05 2020-09-08T15:51:28.878 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.878 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.878 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.878 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.747781+0000", 2020-09-08T15:51:28.878 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.2149999999999998e-06 2020-09-08T15:51:28.879 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.879 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.879 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:51:28.879 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.747787+0000", 2020-09-08T15:51:28.879 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.92931792700000004 2020-09-08T15:51:28.880 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.880 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.880 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.880 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677105+0000", 2020-09-08T15:51:28.881 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.9596999999999999e-05 2020-09-08T15:51:28.881 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.881 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.881 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:28.882 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677125+0000", 2020-09-08T15:51:28.882 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.92947780899999999 2020-09-08T15:51:28.882 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.882 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:28.882 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.883 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.883 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.883 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88206 3.c 3.396499ec (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:51:28.883 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.766105+0000", 2020-09-08T15:51:28.883 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 506.02503551900003, 2020-09-08T15:51:28.884 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.911052847, 2020-09-08T15:51:28.884 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:28.884 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:51:28.884 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:28.884 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:51:28.885 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:28.885 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88206 2020-09-08T15:51:28.885 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.885 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:28.885 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.886 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:28.886 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.766105+0000", 2020-09-08T15:51:28.886 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:28.886 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.886 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.887 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:28.887 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.766105+0000", 2020-09-08T15:51:28.887 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999504 2020-09-08T15:51:28.887 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.887 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.888 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:28.888 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.766055+0000", 2020-09-08T15:51:28.888 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000161633 2020-09-08T15:51:28.888 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.888 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.889 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:28.889 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.766217+0000", 2020-09-08T15:51:28.889 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.5154e-05 2020-09-08T15:51:28.889 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.889 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.890 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:28.890 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.766242+0000", 2020-09-08T15:51:28.890 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.9680000000000001e-06 2020-09-08T15:51:28.890 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.890 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.891 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:28.891 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.766252+0000", 2020-09-08T15:51:28.891 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.5038999999999997e-05 2020-09-08T15:51:28.891 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.892 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.892 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.892 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.766337+0000", 2020-09-08T15:51:28.892 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.1067999999999999e-05 2020-09-08T15:51:28.893 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.893 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.893 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:51:28.893 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.766358+0000", 2020-09-08T15:51:28.893 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.91078725599999999 2020-09-08T15:51:28.894 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.894 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.894 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.895 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677145+0000", 2020-09-08T15:51:28.895 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2483999999999999e-05 2020-09-08T15:51:28.895 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.896 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.896 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:28.896 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677158+0000", 2020-09-08T15:51:28.896 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.911052847 2020-09-08T15:51:28.897 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.897 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:28.897 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.897 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.897 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.898 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88213 3.c 3.1b7c82bc (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:51:28.898 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.770524+0000", 2020-09-08T15:51:28.898 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 506.02061602200001, 2020-09-08T15:51:28.898 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.90669372299999995, 2020-09-08T15:51:28.898 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:28.899 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:51:28.899 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:28.899 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:51:28.899 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:28.899 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88213 2020-09-08T15:51:28.900 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.900 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:28.900 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.900 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:28.900 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.770524+0000", 2020-09-08T15:51:28.901 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:28.901 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.901 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.901 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:28.901 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.770524+0000", 2020-09-08T15:51:28.902 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999757 2020-09-08T15:51:28.902 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.902 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.902 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:28.903 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.770500+0000", 2020-09-08T15:51:28.903 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.1895000000000001e-05 2020-09-08T15:51:28.903 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.903 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.903 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:28.904 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.770582+0000", 2020-09-08T15:51:28.904 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.9410000000000002e-06 2020-09-08T15:51:28.904 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.904 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.904 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:28.905 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.770592+0000", 2020-09-08T15:51:28.905 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.3240000000000002e-06 2020-09-08T15:51:28.905 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.905 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.905 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:28.906 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.770597+0000", 2020-09-08T15:51:28.906 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.5640000000000003e-05 2020-09-08T15:51:28.906 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.906 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.906 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.907 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.770653+0000", 2020-09-08T15:51:28.907 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.8029999999999999e-06 2020-09-08T15:51:28.907 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.907 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.908 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:51:28.908 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.770659+0000", 2020-09-08T15:51:28.908 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.90654226699999996 2020-09-08T15:51:28.908 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.908 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.908 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.909 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677202+0000", 2020-09-08T15:51:28.909 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6291e-05 2020-09-08T15:51:28.909 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.909 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.909 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:28.910 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677218+0000", 2020-09-08T15:51:28.910 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.90669372299999995 2020-09-08T15:51:28.910 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.910 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:28.910 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.911 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.911 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.911 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88217 3.c 3.c99c990c (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:51:28.911 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.773207+0000", 2020-09-08T15:51:28.912 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 506.01793322499998, 2020-09-08T15:51:28.912 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.90404103099999999, 2020-09-08T15:51:28.912 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:28.913 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:51:28.913 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:28.913 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:51:28.913 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:28.914 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88217 2020-09-08T15:51:28.914 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.914 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:28.915 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.915 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:28.915 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.773207+0000", 2020-09-08T15:51:28.916 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:28.916 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.916 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.917 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:28.917 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.773207+0000", 2020-09-08T15:51:28.917 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999504 2020-09-08T15:51:28.917 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.918 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.918 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:28.918 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.773157+0000", 2020-09-08T15:51:28.919 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015484 2020-09-08T15:51:28.919 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.919 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.919 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:28.919 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.773312+0000", 2020-09-08T15:51:28.920 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5645e-05 2020-09-08T15:51:28.920 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.920 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.920 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:28.921 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.773328+0000", 2020-09-08T15:51:28.921 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.3879999999999997e-06 2020-09-08T15:51:28.921 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.921 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.921 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:28.922 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.773334+0000", 2020-09-08T15:51:28.922 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0011813629999999999 2020-09-08T15:51:28.922 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.922 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.922 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.923 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.774515+0000", 2020-09-08T15:51:28.923 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1753000000000001e-05 2020-09-08T15:51:28.923 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.923 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.924 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:51:28.924 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.774527+0000", 2020-09-08T15:51:28.924 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.90270882500000005 2020-09-08T15:51:28.924 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.924 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.925 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.925 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677236+0000", 2020-09-08T15:51:28.925 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2037e-05 2020-09-08T15:51:28.925 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.925 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.926 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:28.926 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677248+0000", 2020-09-08T15:51:28.926 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.90404103099999999 2020-09-08T15:51:28.926 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.926 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:28.927 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.927 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.927 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.927 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88229 3.c 3.1e4c95fc (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:51:28.927 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.783938+0000", 2020-09-08T15:51:28.928 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 506.00720235300003, 2020-09-08T15:51:28.929 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.89337602900000002, 2020-09-08T15:51:28.929 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:28.929 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:51:28.929 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:28.931 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:51:28.931 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:28.931 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88229 2020-09-08T15:51:28.931 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.931 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:28.932 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.932 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:28.932 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.783938+0000", 2020-09-08T15:51:28.932 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:28.932 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.933 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.933 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:28.933 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.783938+0000", 2020-09-08T15:51:28.933 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999452 2020-09-08T15:51:28.933 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.934 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.934 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:28.934 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.783883+0000", 2020-09-08T15:51:28.934 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016782400000000001 2020-09-08T15:51:28.934 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.935 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.935 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:28.935 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.784051+0000", 2020-09-08T15:51:28.935 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7544e-05 2020-09-08T15:51:28.936 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.936 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.936 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:28.936 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.784068+0000", 2020-09-08T15:51:28.936 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.3669999999999999e-06 2020-09-08T15:51:28.937 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.937 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.937 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:28.937 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.784076+0000", 2020-09-08T15:51:28.937 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9112e-05 2020-09-08T15:51:28.937 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.938 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.938 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.938 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.784125+0000", 2020-09-08T15:51:28.938 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.611e-06 2020-09-08T15:51:28.938 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.939 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.939 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:51:28.939 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.784134+0000", 2020-09-08T15:51:28.939 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.89315457099999995 2020-09-08T15:51:28.939 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.940 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.940 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.940 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677289+0000", 2020-09-08T15:51:28.940 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.4997000000000001e-05 2020-09-08T15:51:28.940 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.941 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.941 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:28.941 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677314+0000", 2020-09-08T15:51:28.941 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.89337602900000002 2020-09-08T15:51:28.941 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.942 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:28.942 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.942 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.942 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.942 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88244 3.c 3.ae32ae4c (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:51:28.943 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.801246+0000", 2020-09-08T15:51:28.943 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 505.989894349, 2020-09-08T15:51:28.943 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.87618958400000002, 2020-09-08T15:51:28.943 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:28.943 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:51:28.943 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:28.944 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:51:28.944 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:28.944 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88244 2020-09-08T15:51:28.944 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.944 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:28.945 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.945 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:28.945 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.801246+0000", 2020-09-08T15:51:28.945 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:28.945 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.946 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.946 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:28.946 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.801246+0000", 2020-09-08T15:51:28.946 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9998989 2020-09-08T15:51:28.946 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.947 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.947 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:28.947 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.801145+0000", 2020-09-08T15:51:28.947 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000188982 2020-09-08T15:51:28.947 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.948 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.948 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:28.948 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.801334+0000", 2020-09-08T15:51:28.948 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7173999999999999e-05 2020-09-08T15:51:28.948 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.949 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.949 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:28.949 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.801351+0000", 2020-09-08T15:51:28.949 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2329999999999995e-06 2020-09-08T15:51:28.949 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.950 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.950 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:28.950 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.801360+0000", 2020-09-08T15:51:28.950 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.1766000000000004e-05 2020-09-08T15:51:28.950 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.951 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.951 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.951 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.801432+0000", 2020-09-08T15:51:28.951 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5519000000000002e-05 2020-09-08T15:51:28.951 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.952 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.952 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:51:28.952 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.801447+0000", 2020-09-08T15:51:28.952 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.87588484700000002 2020-09-08T15:51:28.952 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.952 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.953 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.953 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677332+0000", 2020-09-08T15:51:28.953 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000103272 2020-09-08T15:51:28.953 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.953 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.954 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:28.954 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677436+0000", 2020-09-08T15:51:28.954 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.87618958400000002 2020-09-08T15:51:28.961 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.961 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:28.961 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.961 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.961 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.962 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88252 3.c 3.7108c53c (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:51:28.962 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.811533+0000", 2020-09-08T15:51:28.962 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 505.97960691700001, 2020-09-08T15:51:28.962 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.86589799099999998, 2020-09-08T15:51:28.962 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:28.963 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:51:28.963 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:28.963 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:51:28.963 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:28.963 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88252 2020-09-08T15:51:28.964 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.964 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:28.964 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.964 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:28.964 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.811533+0000", 2020-09-08T15:51:28.964 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:28.965 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.965 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.965 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:28.965 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.811533+0000", 2020-09-08T15:51:28.965 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999599 2020-09-08T15:51:28.966 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.966 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.966 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:28.966 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.811493+0000", 2020-09-08T15:51:28.966 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013593500000000001 2020-09-08T15:51:28.967 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.967 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.967 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:28.967 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.811629+0000", 2020-09-08T15:51:28.967 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7910999999999998e-05 2020-09-08T15:51:28.968 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.968 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.968 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:28.968 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.811647+0000", 2020-09-08T15:51:28.968 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5679999999999996e-06 2020-09-08T15:51:28.968 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.969 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.969 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:28.969 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.811654+0000", 2020-09-08T15:51:28.969 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6841999999999995e-05 2020-09-08T15:51:28.969 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.970 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.970 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.970 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.811720+0000", 2020-09-08T15:51:28.970 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.037e-06 2020-09-08T15:51:28.970 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.971 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.971 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:51:28.971 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.811729+0000", 2020-09-08T15:51:28.971 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.86563225099999996 2020-09-08T15:51:28.971 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.972 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.972 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.972 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677362+0000", 2020-09-08T15:51:28.972 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.9690000000000005e-05 2020-09-08T15:51:28.972 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.973 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.973 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:28.973 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677431+0000", 2020-09-08T15:51:28.973 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.86589799099999998 2020-09-08T15:51:28.973 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.974 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:28.974 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.974 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.974 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.974 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88264 3.c 3.ab9c6e5c (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:51:28.975 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.826418+0000", 2020-09-08T15:51:28.975 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 505.96472202799998, 2020-09-08T15:51:28.975 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.85108039700000004, 2020-09-08T15:51:28.975 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:28.975 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:51:28.976 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:28.976 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:51:28.976 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:28.976 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88264 2020-09-08T15:51:28.976 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.977 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:28.977 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.977 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:28.977 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.826418+0000", 2020-09-08T15:51:28.977 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:28.978 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.978 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.978 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:28.978 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.826418+0000", 2020-09-08T15:51:28.978 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999714 2020-09-08T15:51:28.979 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.979 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.979 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:28.979 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.826390+0000", 2020-09-08T15:51:28.979 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.0500000000000004e-05 2020-09-08T15:51:28.979 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.980 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.980 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:28.980 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.826480+0000", 2020-09-08T15:51:28.980 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.5810000000000003e-06 2020-09-08T15:51:28.980 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.981 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.981 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:28.981 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.826490+0000", 2020-09-08T15:51:28.981 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.7269999999999998e-06 2020-09-08T15:51:28.981 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.982 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.982 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:28.982 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.826495+0000", 2020-09-08T15:51:28.982 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.4483e-05 2020-09-08T15:51:28.982 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.982 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.983 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.983 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.826549+0000", 2020-09-08T15:51:28.983 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.3500000000000003e-06 2020-09-08T15:51:28.983 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.983 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.984 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:51:28.984 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.826558+0000", 2020-09-08T15:51:28.984 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.85092367300000005 2020-09-08T15:51:28.984 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.984 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.985 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.985 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677482+0000", 2020-09-08T15:51:28.985 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6571000000000001e-05 2020-09-08T15:51:28.985 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.985 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.986 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:28.986 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677499+0000", 2020-09-08T15:51:28.986 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.85108039700000004 2020-09-08T15:51:28.986 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.986 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:28.987 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.987 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.987 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.987 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88277 3.c 3.88233cdc (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:51:28.987 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.850783+0000", 2020-09-08T15:51:28.988 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 505.94035751400003, 2020-09-08T15:51:28.988 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.82674549799999997, 2020-09-08T15:51:28.988 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:28.988 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:51:28.988 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:28.988 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:51:28.989 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:28.989 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88277 2020-09-08T15:51:28.989 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.989 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:28.989 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.990 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:28.990 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.850783+0000", 2020-09-08T15:51:28.990 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:28.990 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.990 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.991 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:28.991 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.850783+0000", 2020-09-08T15:51:28.991 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999638 2020-09-08T15:51:28.991 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.992 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.992 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:28.992 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.850747+0000", 2020-09-08T15:51:28.992 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012006399999999999 2020-09-08T15:51:28.992 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.993 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.993 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:28.993 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.850867+0000", 2020-09-08T15:51:28.993 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7493e-05 2020-09-08T15:51:28.993 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.994 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.994 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:28.994 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.850884+0000", 2020-09-08T15:51:28.994 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7729999999999994e-06 2020-09-08T15:51:28.994 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.994 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.995 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:28.995 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.850894+0000", 2020-09-08T15:51:28.995 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.1898999999999997e-05 2020-09-08T15:51:28.995 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.995 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.996 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.996 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.850966+0000", 2020-09-08T15:51:28.996 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5427e-05 2020-09-08T15:51:28.996 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.996 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.997 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:51:28.997 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.850981+0000", 2020-09-08T15:51:28.997 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.82653671299999998 2020-09-08T15:51:28.997 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.997 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.998 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:28.998 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677518+0000", 2020-09-08T15:51:28.998 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0154e-05 2020-09-08T15:51:28.998 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:28.998 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:28.998 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:28.999 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677528+0000", 2020-09-08T15:51:28.999 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.82674549799999997 2020-09-08T15:51:28.999 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:28.999 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:28.999 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:29.000 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.000 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.000 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88294 3.c 3.b1cb3efc (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:51:29.000 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.909023+0000", 2020-09-08T15:51:29.000 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 505.88211743699998, 2020-09-08T15:51:29.001 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.76856889299999998, 2020-09-08T15:51:29.001 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:29.001 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:51:29.001 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:29.001 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:51:29.001 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:29.002 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88294 2020-09-08T15:51:29.003 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.004 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:29.004 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.004 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:29.004 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.909023+0000", 2020-09-08T15:51:29.005 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:29.005 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.005 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.005 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:29.005 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.909023+0000", 2020-09-08T15:51:29.006 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999738 2020-09-08T15:51:29.006 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.006 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.006 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:29.006 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.908997+0000", 2020-09-08T15:51:29.007 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2019000000000001e-05 2020-09-08T15:51:29.007 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.007 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.007 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:29.007 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.909089+0000", 2020-09-08T15:51:29.008 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4188e-05 2020-09-08T15:51:29.008 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.008 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.008 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:29.009 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.909103+0000", 2020-09-08T15:51:29.009 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.6540000000000004e-06 2020-09-08T15:51:29.009 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.009 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.009 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:29.010 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.909109+0000", 2020-09-08T15:51:29.010 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7284000000000002e-05 2020-09-08T15:51:29.010 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.010 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.010 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:29.011 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.909166+0000", 2020-09-08T15:51:29.011 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8189999999999998e-06 2020-09-08T15:51:29.011 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.011 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.011 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:51:29.012 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.909175+0000", 2020-09-08T15:51:29.012 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.76839657900000002 2020-09-08T15:51:29.012 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.012 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.012 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:29.013 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677571+0000", 2020-09-08T15:51:29.013 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.0534e-05 2020-09-08T15:51:29.013 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.013 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.014 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:29.014 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677592+0000", 2020-09-08T15:51:29.014 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.76856889299999998 2020-09-08T15:51:29.014 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:29.015 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:29.015 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:29.015 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.015 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.015 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35087.0:88298 3.c 3.86504cfc (undecoded) ondisk+write+known_if_redirected e235)", 2020-09-08T15:51:29.016 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:43:02.934302+0000", 2020-09-08T15:51:29.016 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 505.85683806600002, 2020-09-08T15:51:29.016 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.74331687800000001, 2020-09-08T15:51:29.016 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:29.016 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T15:51:29.017 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:29.017 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35087", 2020-09-08T15:51:29.017 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:29.017 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 88298 2020-09-08T15:51:29.017 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.018 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:29.018 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.018 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:29.018 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.934302+0000", 2020-09-08T15:51:29.018 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:29.019 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.019 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.019 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:29.019 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.934302+0000", 2020-09-08T15:51:29.019 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999776 2020-09-08T15:51:29.020 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.020 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.020 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:29.020 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.934280+0000", 2020-09-08T15:51:29.021 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.1067000000000002e-05 2020-09-08T15:51:29.021 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.021 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.021 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:29.021 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.934361+0000", 2020-09-08T15:51:29.022 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.031e-05 2020-09-08T15:51:29.022 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.022 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.022 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:29.022 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.934371+0000", 2020-09-08T15:51:29.023 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.7739999999999999e-06 2020-09-08T15:51:29.023 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.023 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.023 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:29.023 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.934378+0000", 2020-09-08T15:51:29.024 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.2311000000000005e-05 2020-09-08T15:51:29.024 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.024 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.024 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:29.024 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.934460+0000", 2020-09-08T15:51:29.025 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1685000000000001e-05 2020-09-08T15:51:29.025 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.025 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.025 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for peered", 2020-09-08T15:51:29.025 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:02.934472+0000", 2020-09-08T15:51:29.026 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.743139683 2020-09-08T15:51:29.026 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.026 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.026 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:29.026 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677611+0000", 2020-09-08T15:51:29.027 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.6290000000000001e-06 2020-09-08T15:51:29.027 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.027 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.027 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:29.027 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:43:03.677619+0000", 2020-09-08T15:51:29.028 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.74331687800000001 2020-09-08T15:51:29.028 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:29.028 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:29.028 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:29.028 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.029 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.029 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:115255 4.8 4:16ba0e35:::benchmark_data_smithi166_32054_object115254:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e343)", 2020-09-08T15:51:29.029 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:26.164401+0000", 2020-09-08T15:51:29.029 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 182.62673887899999, 2020-09-08T15:51:29.030 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.434945048, 2020-09-08T15:51:29.030 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:29.030 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:29.030 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:29.030 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:51:29.031 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:29.031 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 115255 2020-09-08T15:51:29.031 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.031 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:29.031 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.032 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:29.032 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.164401+0000", 2020-09-08T15:51:29.032 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:29.032 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.032 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.033 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:29.033 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.164401+0000", 2020-09-08T15:51:29.033 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999599 2020-09-08T15:51:29.033 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.033 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.034 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:29.034 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.164361+0000", 2020-09-08T15:51:29.034 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013619700000000001 2020-09-08T15:51:29.034 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.034 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.035 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:29.035 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.164498+0000", 2020-09-08T15:51:29.035 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6906999999999999e-05 2020-09-08T15:51:29.035 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.035 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.036 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:29.036 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.164514+0000", 2020-09-08T15:51:29.036 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.1309999999999995e-06 2020-09-08T15:51:29.036 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.036 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.037 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:29.037 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.164524+0000", 2020-09-08T15:51:29.037 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.003450748 2020-09-08T15:51:29.037 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.037 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.038 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:29.038 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.167974+0000", 2020-09-08T15:51:29.038 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000113124 2020-09-08T15:51:29.038 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.038 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.039 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:29.039 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.168087+0000", 2020-09-08T15:51:29.039 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013468099999999999 2020-09-08T15:51:29.039 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.039 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.040 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 7", 2020-09-08T15:51:29.040 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.168222+0000", 2020-09-08T15:51:29.040 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0082220190000000005 2020-09-08T15:51:29.040 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.040 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.041 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:51:29.041 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.176444+0000", 2020-09-08T15:51:29.041 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.42266372200000002 2020-09-08T15:51:29.041 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.041 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.042 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:29.042 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.599108+0000", 2020-09-08T15:51:29.042 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013225499999999999 2020-09-08T15:51:29.042 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.042 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.043 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:29.043 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.599240+0000", 2020-09-08T15:51:29.043 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010634299999999999 2020-09-08T15:51:29.043 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.043 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.044 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:29.044 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.599346+0000", 2020-09-08T15:51:29.044 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.434945048 2020-09-08T15:51:29.044 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:29.044 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:29.045 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:29.045 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.045 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.045 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:115257 4.8 4:1c013f66:::benchmark_data_smithi166_32054_object115256:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e343)", 2020-09-08T15:51:29.045 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:26.168496+0000", 2020-09-08T15:51:29.046 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 182.622643877, 2020-09-08T15:51:29.046 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.437428763, 2020-09-08T15:51:29.046 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:29.046 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:29.046 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:29.047 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:51:29.047 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:29.047 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 115257 2020-09-08T15:51:29.047 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.047 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:29.048 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.048 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:29.048 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.168496+0000", 2020-09-08T15:51:29.048 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:29.048 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.049 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.049 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:29.049 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.168496+0000", 2020-09-08T15:51:29.050 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999628 2020-09-08T15:51:29.050 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.050 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.050 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:29.050 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.168459+0000", 2020-09-08T15:51:29.051 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000119265 2020-09-08T15:51:29.051 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.051 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.051 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:29.052 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.168579+0000", 2020-09-08T15:51:29.052 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3009e-05 2020-09-08T15:51:29.052 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.052 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.052 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:29.053 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.168592+0000", 2020-09-08T15:51:29.053 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.5180000000000004e-06 2020-09-08T15:51:29.053 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.053 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.053 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:29.054 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.168597+0000", 2020-09-08T15:51:29.054 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0040636530000000004 2020-09-08T15:51:29.054 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.054 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.054 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:29.055 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.172661+0000", 2020-09-08T15:51:29.055 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015378700000000001 2020-09-08T15:51:29.055 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.055 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.055 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:29.056 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.172814+0000", 2020-09-08T15:51:29.056 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013778400000000001 2020-09-08T15:51:29.056 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.056 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.056 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 7", 2020-09-08T15:51:29.058 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.172952+0000", 2020-09-08T15:51:29.058 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0036939690000000001 2020-09-08T15:51:29.058 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.059 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.059 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:51:29.059 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.176646+0000", 2020-09-08T15:51:29.059 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.42906232399999999 2020-09-08T15:51:29.059 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.060 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.060 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:29.060 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.605709+0000", 2020-09-08T15:51:29.060 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014526900000000001 2020-09-08T15:51:29.060 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.060 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.061 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:29.061 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.605854+0000", 2020-09-08T15:51:29.061 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.1335999999999996e-05 2020-09-08T15:51:29.061 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.062 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.062 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:29.062 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.605925+0000", 2020-09-08T15:51:29.062 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.437428763 2020-09-08T15:51:29.062 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:29.063 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:29.063 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:29.063 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.063 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.064 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:115277 4.8 4:1e221952:::benchmark_data_smithi166_32054_object115276:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e343)", 2020-09-08T15:51:29.064 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:26.180167+0000", 2020-09-08T15:51:29.064 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 182.61097325899999, 2020-09-08T15:51:29.064 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.44099396200000002, 2020-09-08T15:51:29.064 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:29.065 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:29.065 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:29.065 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:51:29.065 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:29.065 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 115277 2020-09-08T15:51:29.066 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.066 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:29.066 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.066 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:29.066 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.180167+0000", 2020-09-08T15:51:29.067 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:29.067 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.067 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.067 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:29.067 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.180167+0000", 2020-09-08T15:51:29.068 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999642 2020-09-08T15:51:29.068 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.068 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.068 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:29.068 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.180131+0000", 2020-09-08T15:51:29.069 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012684299999999999 2020-09-08T15:51:29.069 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.069 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.069 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:29.069 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.180258+0000", 2020-09-08T15:51:29.070 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6249999999999999e-05 2020-09-08T15:51:29.070 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.070 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.070 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:29.070 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.180274+0000", 2020-09-08T15:51:29.071 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.3359999999999995e-06 2020-09-08T15:51:29.071 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.071 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.071 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:29.071 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.180283+0000", 2020-09-08T15:51:29.072 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.001395709 2020-09-08T15:51:29.072 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.072 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.072 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:29.072 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.181678+0000", 2020-09-08T15:51:29.072 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018008100000000001 2020-09-08T15:51:29.073 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.073 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.073 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:29.073 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.181859+0000", 2020-09-08T15:51:29.074 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00020721199999999999 2020-09-08T15:51:29.074 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.074 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.074 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 7", 2020-09-08T15:51:29.074 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.182066+0000", 2020-09-08T15:51:29.075 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0084666389999999998 2020-09-08T15:51:29.075 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.075 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.075 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:51:29.076 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.190532+0000", 2020-09-08T15:51:29.077 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.430511229 2020-09-08T15:51:29.077 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.077 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.077 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:29.078 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.621044+0000", 2020-09-08T15:51:29.078 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.3795999999999997e-05 2020-09-08T15:51:29.078 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.078 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.078 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:29.079 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.621107+0000", 2020-09-08T15:51:29.079 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.3563999999999997e-05 2020-09-08T15:51:29.079 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.079 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.079 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:29.080 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.621161+0000", 2020-09-08T15:51:29.080 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.44099396200000002 2020-09-08T15:51:29.080 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:29.080 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:29.080 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:29.080 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.081 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.081 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:115279 4.8 4:13a40eca:::benchmark_data_smithi166_32054_object115278:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e343)", 2020-09-08T15:51:29.081 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:26.183900+0000", 2020-09-08T15:51:29.081 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 182.60724038000001, 2020-09-08T15:51:29.081 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.437427554, 2020-09-08T15:51:29.082 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:51:29.082 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:29.082 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:51:29.082 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:51:29.082 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:29.083 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 115279 2020-09-08T15:51:29.083 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.083 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:51:29.083 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.084 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:51:29.084 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.183900+0000", 2020-09-08T15:51:29.084 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:51:29.084 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.084 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.085 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:51:29.085 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.183900+0000", 2020-09-08T15:51:29.085 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999571 2020-09-08T15:51:29.085 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.085 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.086 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:51:29.086 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.183857+0000", 2020-09-08T15:51:29.086 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014292800000000001 2020-09-08T15:51:29.086 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.087 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.087 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:51:29.087 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.184000+0000", 2020-09-08T15:51:29.087 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6827999999999999e-05 2020-09-08T15:51:29.087 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.088 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.088 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:51:29.088 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.184017+0000", 2020-09-08T15:51:29.088 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.2300000000000008e-06 2020-09-08T15:51:29.088 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.089 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.089 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:51:29.089 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.184025+0000", 2020-09-08T15:51:29.090 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0020036609999999999 2020-09-08T15:51:29.090 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.090 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.090 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:51:29.091 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.186029+0000", 2020-09-08T15:51:29.091 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017262700000000001 2020-09-08T15:51:29.091 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.091 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.091 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:51:29.092 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.186201+0000", 2020-09-08T15:51:29.092 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00021605999999999999 2020-09-08T15:51:29.092 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.092 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.092 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 7", 2020-09-08T15:51:29.093 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.186417+0000", 2020-09-08T15:51:29.093 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0043667350000000001 2020-09-08T15:51:29.093 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.093 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.094 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:51:29.094 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.190784+0000", 2020-09-08T15:51:29.094 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.43043087099999999 2020-09-08T15:51:29.094 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.094 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.095 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:29.095 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.621215+0000", 2020-09-08T15:51:29.095 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.6477999999999997e-05 2020-09-08T15:51:29.095 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.095 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.096 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:51:29.096 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.621271+0000", 2020-09-08T15:51:29.096 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.6014000000000003e-05 2020-09-08T15:51:29.096 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:51:29.096 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:51:29.097 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:51:29.097 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:26.621327+0000", 2020-09-08T15:51:29.097 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.437427554 2020-09-08T15:51:29.097 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:29.098 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:29.098 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:29.098 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:51:29.098 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:51:29.098 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:51:29.100 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2020-09-08T15:51:29.102 INFO:tasks.radosbench:Beginning radosbench... 2020-09-08T15:51:29.102 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_3 2020-09-08T15:51:29.103 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool create unique_pool_3 16 2020-09-08T15:51:29.104 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:51:28 smithi055 bash[14301]: debug 2020-09-08T15:51:28.782+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:29.104 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:28 smithi055 bash[9296]: cluster 2020-09-08T15:51:27.808860+0000 mgr.x (mgr.24118) 724 : cluster [DBG] pgmap v1033: 17 pgs: 17 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 24 op/s 2020-09-08T15:51:29.104 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:28 smithi055 bash[9296]: audit 2020-09-08T15:51:27.893711+0000 mon.a (mon.0) 1496 : audit [INF] from='client.? v1:172.21.15.104:0/1950612474' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T15:51:29.105 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:28 smithi055 bash[9296]: cluster 2020-09-08T15:51:27.893791+0000 mon.a (mon.0) 1497 : cluster [DBG] osdmap e394: 12 total, 12 up, 12 in 2020-09-08T15:51:29.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:28 smithi166 bash[9446]: cluster 2020-09-08T15:51:27.808860+0000 mgr.x (mgr.24118) 724 : cluster [DBG] pgmap v1033: 17 pgs: 17 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 24 op/s 2020-09-08T15:51:29.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:28 smithi166 bash[9446]: audit 2020-09-08T15:51:27.893711+0000 mon.a (mon.0) 1496 : audit [INF] from='client.? v1:172.21.15.104:0/1950612474' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T15:51:29.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:28 smithi166 bash[9446]: cluster 2020-09-08T15:51:27.893791+0000 mon.a (mon.0) 1497 : cluster [DBG] osdmap e394: 12 total, 12 up, 12 in 2020-09-08T15:51:29.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:28 smithi104 bash[9203]: cluster 2020-09-08T15:51:27.808860+0000 mgr.x (mgr.24118) 724 : cluster [DBG] pgmap v1033: 17 pgs: 17 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 24 op/s 2020-09-08T15:51:29.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:28 smithi104 bash[9203]: audit 2020-09-08T15:51:27.893711+0000 mon.a (mon.0) 1496 : audit [INF] from='client.? v1:172.21.15.104:0/1950612474' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T15:51:29.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:28 smithi104 bash[9203]: cluster 2020-09-08T15:51:27.893791+0000 mon.a (mon.0) 1497 : cluster [DBG] osdmap e394: 12 total, 12 up, 12 in 2020-09-08T15:51:29.208 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:51:29.232 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:51:29.463 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:51:29.568 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:29.583 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:51:29 smithi166 bash[16593]: debug 2020-09-08T15:51:29.463+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:29.584 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2020-09-08T15:51:29.690 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:29.724 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:51:29.802 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:29.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:51:29 smithi166 bash[18960]: debug 2020-09-08T15:51:29.690+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:29.921 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:30.022 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:51:30.147 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:51:30.159 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:51:30 smithi055 bash[11956]: debug 2020-09-08T15:51:30.146+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:30.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:29 smithi055 bash[9296]: audit 2020-09-08T15:51:29.270394+0000 mon.a (mon.0) 1498 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:51:29.268537\"}"}]: dispatch 2020-09-08T15:51:30.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:29 smithi055 bash[9296]: audit 2020-09-08T15:51:29.276204+0000 mon.a (mon.0) 1499 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:51:29.268537\"}"}]': finished 2020-09-08T15:51:30.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:29 smithi055 bash[9296]: audit 2020-09-08T15:51:29.277575+0000 mon.a (mon.0) 1500 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:51:30.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:29 smithi055 bash[9296]: audit 2020-09-08T15:51:29.362163+0000 mon.a (mon.0) 1501 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:51:29.360460\"}"}]: dispatch 2020-09-08T15:51:30.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:29 smithi055 bash[9296]: audit 2020-09-08T15:51:29.365513+0000 mon.a (mon.0) 1502 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:51:29.360460\"}"}]': finished 2020-09-08T15:51:30.165 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:29 smithi055 bash[9296]: audit 2020-09-08T15:51:29.367081+0000 mon.a (mon.0) 1503 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:51:30.170 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:29 smithi055 bash[9296]: audit 2020-09-08T15:51:29.635686+0000 mon.a (mon.0) 1504 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868402\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868897\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869135\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869362\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:41:28.869649\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:51:29.633993\"}"}]: dispatch 2020-09-08T15:51:30.170 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:29 smithi055 bash[9296]: audit 2020-09-08T15:51:29.640982+0000 mon.a (mon.0) 1505 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868402\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868897\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869135\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869362\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:41:28.869649\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:51:29.633993\"}"}]': finished 2020-09-08T15:51:30.170 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:29 smithi055 bash[9296]: audit 2020-09-08T15:51:29.642224+0000 mon.a (mon.0) 1506 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:51:30.183 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:29 smithi104 bash[9203]: audit 2020-09-08T15:51:29.270394+0000 mon.a (mon.0) 1498 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:51:29.268537\"}"}]: dispatch 2020-09-08T15:51:30.184 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:29 smithi104 bash[9203]: audit 2020-09-08T15:51:29.276204+0000 mon.a (mon.0) 1499 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:51:29.268537\"}"}]': finished 2020-09-08T15:51:30.184 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:29 smithi104 bash[9203]: audit 2020-09-08T15:51:29.277575+0000 mon.a (mon.0) 1500 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:51:30.185 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:29 smithi104 bash[9203]: audit 2020-09-08T15:51:29.362163+0000 mon.a (mon.0) 1501 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:51:29.360460\"}"}]: dispatch 2020-09-08T15:51:30.185 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:29 smithi104 bash[9203]: audit 2020-09-08T15:51:29.365513+0000 mon.a (mon.0) 1502 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:51:29.360460\"}"}]': finished 2020-09-08T15:51:30.185 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:29 smithi104 bash[9203]: audit 2020-09-08T15:51:29.367081+0000 mon.a (mon.0) 1503 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:51:30.193 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:29 smithi104 bash[9203]: audit 2020-09-08T15:51:29.635686+0000 mon.a (mon.0) 1504 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868402\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868897\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869135\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869362\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:41:28.869649\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:51:29.633993\"}"}]: dispatch 2020-09-08T15:51:30.194 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:29 smithi104 bash[9203]: audit 2020-09-08T15:51:29.640982+0000 mon.a (mon.0) 1505 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868402\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868897\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869135\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869362\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:41:28.869649\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:51:29.633993\"}"}]': finished 2020-09-08T15:51:30.194 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:29 smithi104 bash[9203]: audit 2020-09-08T15:51:29.642224+0000 mon.a (mon.0) 1506 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:51:30.259 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:51:30.270 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:51:29 smithi166 bash[11928]: debug 2020-09-08T15:51:29.914+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:30.272 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:29 smithi166 bash[9446]: audit 2020-09-08T15:51:29.270394+0000 mon.a (mon.0) 1498 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:51:29.268537\"}"}]: dispatch 2020-09-08T15:51:30.273 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:29 smithi166 bash[9446]: audit 2020-09-08T15:51:29.276204+0000 mon.a (mon.0) 1499 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267549\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:32:54.267710\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.267803\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268170\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268443\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:32:54.268707\", \"created\": \"2020-09-08T15:32:53.363152\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:32:54.269039\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:51:29.268537\"}"}]': finished 2020-09-08T15:51:30.273 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:29 smithi166 bash[9446]: audit 2020-09-08T15:51:29.277575+0000 mon.a (mon.0) 1500 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:51:30.273 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:29 smithi166 bash[9446]: audit 2020-09-08T15:51:29.362163+0000 mon.a (mon.0) 1501 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:51:29.360460\"}"}]: dispatch 2020-09-08T15:51:30.282 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:29 smithi166 bash[9446]: audit 2020-09-08T15:51:29.365513+0000 mon.a (mon.0) 1502 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884034\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884240\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.884918\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885445\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:34:33.885959\", \"created\": \"2020-09-08T15:34:32.858122\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:34:33.886588\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:51:29.360460\"}"}]': finished 2020-09-08T15:51:30.283 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:29 smithi166 bash[9446]: audit 2020-09-08T15:51:29.367081+0000 mon.a (mon.0) 1503 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:51:30.283 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:29 smithi166 bash[9446]: audit 2020-09-08T15:51:29.635686+0000 mon.a (mon.0) 1504 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868402\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868897\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869135\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869362\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:41:28.869649\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:51:29.633993\"}"}]: dispatch 2020-09-08T15:51:30.283 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:29 smithi166 bash[9446]: audit 2020-09-08T15:51:29.640982+0000 mon.a (mon.0) 1505 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868402\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868500\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.868897\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869135\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:41:28.869362\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:41:28.869649\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:51:29.633993\"}"}]': finished 2020-09-08T15:51:30.284 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:29 smithi166 bash[9446]: audit 2020-09-08T15:51:29.642224+0000 mon.a (mon.0) 1506 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T15:51:30.397 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:51:30.518 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:30.530 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:51:30.618 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:51:30 smithi166 bash[16593]: debug 2020-09-08T15:51:30.394+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:30.695 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:51:30.977 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:31.082 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:31.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:30 smithi055 bash[9296]: cluster 2020-09-08T15:51:29.809334+0000 mgr.x (mgr.24118) 725 : cluster [DBG] pgmap v1035: 1 pgs: 1 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:31.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:30 smithi055 bash[9296]: audit 2020-09-08T15:51:30.138814+0000 mon.c (mon.2) 56 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136766\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136942\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137052\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137657\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138117\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138591\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.139157\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:51:29.268537\"}"}]: dispatch 2020-09-08T15:51:31.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:30 smithi055 bash[9296]: audit 2020-09-08T15:51:30.141630+0000 mon.a (mon.0) 1507 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136766\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136942\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137052\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137657\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138117\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138591\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.139157\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:51:29.268537\"}"}]: dispatch 2020-09-08T15:51:31.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:30 smithi055 bash[9296]: audit 2020-09-08T15:51:30.147267+0000 mon.a (mon.0) 1508 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136766\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136942\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137052\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137657\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138117\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138591\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.139157\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:51:29.268537\"}"}]': finished 2020-09-08T15:51:31.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:30 smithi055 bash[9296]: audit 2020-09-08T15:51:30.219822+0000 mon.c (mon.2) 57 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217339\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217539\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218203\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218766\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.219295\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.219948\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:51:29.360460\"}"}]: dispatch 2020-09-08T15:51:31.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:30 smithi055 bash[9296]: audit 2020-09-08T15:51:30.222832+0000 mon.a (mon.0) 1509 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217339\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217539\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218203\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218766\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.219295\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.219948\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:51:29.360460\"}"}]: dispatch 2020-09-08T15:51:31.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:30 smithi055 bash[9296]: audit 2020-09-08T15:51:30.226533+0000 mon.a (mon.0) 1510 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217339\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217539\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218203\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218766\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.219295\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.219948\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:51:29.360460\"}"}]': finished 2020-09-08T15:51:31.177 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:51:30 smithi104 bash[22305]: debug 2020-09-08T15:51:30.978+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:31.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:30 smithi104 bash[9203]: cluster 2020-09-08T15:51:29.809334+0000 mgr.x (mgr.24118) 725 : cluster [DBG] pgmap v1035: 1 pgs: 1 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:31.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:30 smithi104 bash[9203]: audit 2020-09-08T15:51:30.138814+0000 mon.c (mon.2) 56 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136766\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136942\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137052\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137657\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138117\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138591\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.139157\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:51:29.268537\"}"}]: dispatch 2020-09-08T15:51:31.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:30 smithi104 bash[9203]: audit 2020-09-08T15:51:30.141630+0000 mon.a (mon.0) 1507 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136766\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136942\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137052\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137657\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138117\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138591\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.139157\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:51:29.268537\"}"}]: dispatch 2020-09-08T15:51:31.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:30 smithi104 bash[9203]: audit 2020-09-08T15:51:30.147267+0000 mon.a (mon.0) 1508 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136766\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136942\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137052\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137657\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138117\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138591\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.139157\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:51:29.268537\"}"}]': finished 2020-09-08T15:51:31.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:30 smithi104 bash[9203]: audit 2020-09-08T15:51:30.219822+0000 mon.c (mon.2) 57 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217339\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217539\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218203\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218766\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.219295\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.219948\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:51:29.360460\"}"}]: dispatch 2020-09-08T15:51:31.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:30 smithi104 bash[9203]: audit 2020-09-08T15:51:30.222832+0000 mon.a (mon.0) 1509 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217339\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217539\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218203\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218766\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.219295\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.219948\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:51:29.360460\"}"}]: dispatch 2020-09-08T15:51:31.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:30 smithi104 bash[9203]: audit 2020-09-08T15:51:30.226533+0000 mon.a (mon.0) 1510 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217339\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217539\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218203\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218766\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.219295\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.219948\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:51:29.360460\"}"}]': finished 2020-09-08T15:51:31.203 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:30 smithi166 bash[9446]: cluster 2020-09-08T15:51:29.809334+0000 mgr.x (mgr.24118) 725 : cluster [DBG] pgmap v1035: 1 pgs: 1 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:31.204 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:30 smithi166 bash[9446]: audit 2020-09-08T15:51:30.138814+0000 mon.c (mon.2) 56 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136766\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136942\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137052\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137657\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138117\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138591\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.139157\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:51:29.268537\"}"}]: dispatch 2020-09-08T15:51:31.204 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:30 smithi166 bash[9446]: audit 2020-09-08T15:51:30.141630+0000 mon.a (mon.0) 1507 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136766\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136942\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137052\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137657\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138117\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138591\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.139157\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:51:29.268537\"}"}]: dispatch 2020-09-08T15:51:31.204 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:30 smithi166 bash[9446]: audit 2020-09-08T15:51:30.147267+0000 mon.a (mon.0) 1508 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136766\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136942\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137052\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137657\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138117\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138591\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.139157\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T15:51:29.268537\"}"}]': finished 2020-09-08T15:51:31.205 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:31.217 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:30 smithi166 bash[9446]: audit 2020-09-08T15:51:30.219822+0000 mon.c (mon.2) 57 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217339\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217539\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218203\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218766\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.219295\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.219948\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:51:29.360460\"}"}]: dispatch 2020-09-08T15:51:31.217 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:30 smithi166 bash[9446]: audit 2020-09-08T15:51:30.222832+0000 mon.a (mon.0) 1509 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217339\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217539\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218203\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218766\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.219295\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.219948\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:51:29.360460\"}"}]: dispatch 2020-09-08T15:51:31.218 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:30 smithi166 bash[9446]: audit 2020-09-08T15:51:30.226533+0000 mon.a (mon.0) 1510 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217339\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217539\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218203\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218766\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.219295\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.219948\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T15:51:29.360460\"}"}]': finished 2020-09-08T15:51:31.320 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:51:31.463 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:51:31.568 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:51:31.584 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:51:31 smithi166 bash[14254]: debug 2020-09-08T15:51:31.202+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:31.657 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:51:31 smithi055 bash[11956]: debug 2020-09-08T15:51:31.462+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:31.697 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:51:31.804 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:31.867 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:51:31 smithi166 bash[16593]: debug 2020-09-08T15:51:31.698+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:31.922 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:32.033 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:32.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:51:31 smithi055 bash[14301]: debug 2020-09-08T15:51:31.922+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:32.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:31 smithi055 bash[9296]: audit 2020-09-08T15:51:31.270367+0000 mon.c (mon.2) 58 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267773\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267979\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.268707\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269253\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269785\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:51:31.270481\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:51:29.633993\"}"}]: dispatch 2020-09-08T15:51:32.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:31 smithi055 bash[9296]: audit 2020-09-08T15:51:31.273632+0000 mon.a (mon.0) 1511 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267773\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267979\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.268707\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269253\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269785\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:51:31.270481\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:51:29.633993\"}"}]: dispatch 2020-09-08T15:51:32.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:31 smithi055 bash[9296]: audit 2020-09-08T15:51:31.278169+0000 mon.a (mon.0) 1512 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267773\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267979\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.268707\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269253\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269785\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:51:31.270481\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:51:29.633993\"}"}]': finished 2020-09-08T15:51:32.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:31 smithi055 bash[9296]: audit 2020-09-08T15:51:31.280855+0000 mon.c (mon.2) 59 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:51:32.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:31 smithi055 bash[9296]: audit 2020-09-08T15:51:31.283411+0000 mon.a (mon.0) 1513 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:51:32.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:31 smithi055 bash[9296]: audit 2020-09-08T15:51:31.286358+0000 mon.a (mon.0) 1514 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:51:32.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:31 smithi055 bash[9296]: audit 2020-09-08T15:51:31.291163+0000 mon.c (mon.2) 60 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:51:32.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:31 smithi055 bash[9296]: audit 2020-09-08T15:51:31.294029+0000 mon.a (mon.0) 1515 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:51:32.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:31 smithi055 bash[9296]: audit 2020-09-08T15:51:31.297319+0000 mon.a (mon.0) 1516 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:51:32.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:31 smithi104 bash[9203]: audit 2020-09-08T15:51:31.270367+0000 mon.c (mon.2) 58 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267773\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267979\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.268707\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269253\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269785\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:51:31.270481\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:51:29.633993\"}"}]: dispatch 2020-09-08T15:51:32.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:31 smithi104 bash[9203]: audit 2020-09-08T15:51:31.273632+0000 mon.a (mon.0) 1511 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267773\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267979\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.268707\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269253\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269785\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:51:31.270481\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:51:29.633993\"}"}]: dispatch 2020-09-08T15:51:32.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:31 smithi104 bash[9203]: audit 2020-09-08T15:51:31.278169+0000 mon.a (mon.0) 1512 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267773\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267979\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.268707\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269253\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269785\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:51:31.270481\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:51:29.633993\"}"}]': finished 2020-09-08T15:51:32.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:31 smithi104 bash[9203]: audit 2020-09-08T15:51:31.280855+0000 mon.c (mon.2) 59 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:51:32.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:31 smithi104 bash[9203]: audit 2020-09-08T15:51:31.283411+0000 mon.a (mon.0) 1513 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:51:32.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:31 smithi104 bash[9203]: audit 2020-09-08T15:51:31.286358+0000 mon.a (mon.0) 1514 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:51:32.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:31 smithi104 bash[9203]: audit 2020-09-08T15:51:31.291163+0000 mon.c (mon.2) 60 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:51:32.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:31 smithi104 bash[9203]: audit 2020-09-08T15:51:31.294029+0000 mon.a (mon.0) 1515 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:51:32.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:31 smithi104 bash[9203]: audit 2020-09-08T15:51:31.297319+0000 mon.a (mon.0) 1516 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:51:32.225 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:32.330 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:32.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:31 smithi166 bash[9446]: audit 2020-09-08T15:51:31.270367+0000 mon.c (mon.2) 58 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267773\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267979\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.268707\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269253\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269785\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:51:31.270481\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:51:29.633993\"}"}]: dispatch 2020-09-08T15:51:32.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:31 smithi166 bash[9446]: audit 2020-09-08T15:51:31.273632+0000 mon.a (mon.0) 1511 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267773\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267979\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.268707\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269253\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269785\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:51:31.270481\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:51:29.633993\"}"}]: dispatch 2020-09-08T15:51:32.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:31 smithi166 bash[9446]: audit 2020-09-08T15:51:31.278169+0000 mon.a (mon.0) 1512 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267773\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267979\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.268707\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269253\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269785\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:51:31.270481\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T15:51:29.633993\"}"}]': finished 2020-09-08T15:51:32.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:31 smithi166 bash[9446]: audit 2020-09-08T15:51:31.280855+0000 mon.c (mon.2) 59 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:51:32.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:31 smithi166 bash[9446]: audit 2020-09-08T15:51:31.283411+0000 mon.a (mon.0) 1513 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:51:32.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:31 smithi166 bash[9446]: audit 2020-09-08T15:51:31.286358+0000 mon.a (mon.0) 1514 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:51:32.373 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:31 smithi166 bash[9446]: audit 2020-09-08T15:51:31.291163+0000 mon.c (mon.2) 60 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:51:32.373 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:31 smithi166 bash[9446]: audit 2020-09-08T15:51:31.294029+0000 mon.a (mon.0) 1515 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:51:32.373 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:31 smithi166 bash[9446]: audit 2020-09-08T15:51:31.297319+0000 mon.a (mon.0) 1516 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:51:32.454 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:32.559 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:32.570 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:51:32 smithi104 bash[17588]: debug 2020-09-08T15:51:32.226+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:32.660 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:51:32 smithi055 bash[14301]: debug 2020-09-08T15:51:32.454+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:32.681 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:32.787 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:32.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:51:32 smithi166 bash[11928]: debug 2020-09-08T15:51:32.682+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:32.927 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:51:33.034 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:33.138 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:51:33.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:51:33 smithi104 bash[22305]: debug 2020-09-08T15:51:33.034+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:33.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:32 smithi104 bash[9203]: cluster 2020-09-08T15:51:31.809907+0000 mgr.x (mgr.24118) 726 : cluster [DBG] pgmap v1036: 1 pgs: 1 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:33.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:32 smithi104 bash[9203]: cluster 2020-09-08T15:51:32.139574+0000 osd.8 (osd.8) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:33.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:32 smithi104 bash[9203]: cluster 2020-09-08T15:51:32.139704+0000 osd.8 (osd.8) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:33.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:32 smithi104 bash[9203]: audit 2020-09-08T15:51:32.227035+0000 mon.a (mon.0) 1517 : audit [INF] from='client.? v1:172.21.15.104:0/3147108074' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:51:33.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:32 smithi166 bash[9446]: cluster 2020-09-08T15:51:31.809907+0000 mgr.x (mgr.24118) 726 : cluster [DBG] pgmap v1036: 1 pgs: 1 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:33.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:32 smithi166 bash[9446]: cluster 2020-09-08T15:51:32.139574+0000 osd.8 (osd.8) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:33.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:32 smithi166 bash[9446]: cluster 2020-09-08T15:51:32.139704+0000 osd.8 (osd.8) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:33.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:32 smithi166 bash[9446]: audit 2020-09-08T15:51:32.227035+0000 mon.a (mon.0) 1517 : audit [INF] from='client.? v1:172.21.15.104:0/3147108074' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:51:33.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:32 smithi055 bash[9296]: cluster 2020-09-08T15:51:31.809907+0000 mgr.x (mgr.24118) 726 : cluster [DBG] pgmap v1036: 1 pgs: 1 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:33.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:32 smithi055 bash[9296]: cluster 2020-09-08T15:51:32.139574+0000 osd.8 (osd.8) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:33.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:32 smithi055 bash[9296]: cluster 2020-09-08T15:51:32.139704+0000 osd.8 (osd.8) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:33.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:32 smithi055 bash[9296]: audit 2020-09-08T15:51:32.227035+0000 mon.a (mon.0) 1517 : audit [INF] from='client.? v1:172.21.15.104:0/3147108074' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:51:33.613 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:51:33.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:51:33 smithi104 bash[19922]: debug 2020-09-08T15:51:33.614+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:33.722 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:33.857 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:33.963 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:34.117 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:51:33 smithi166 bash[11928]: debug 2020-09-08T15:51:33.858+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:34.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:33 smithi166 bash[9446]: cluster 2020-09-08T15:51:32.914212+0000 mon.a (mon.0) 1518 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:51:34.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:33 smithi166 bash[9446]: audit 2020-09-08T15:51:32.929053+0000 mon.a (mon.0) 1519 : audit [INF] from='client.? v1:172.21.15.104:0/3147108074' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:51:34.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:33 smithi166 bash[9446]: cluster 2020-09-08T15:51:32.929101+0000 mon.a (mon.0) 1520 : cluster [DBG] osdmap e395: 12 total, 12 up, 12 in 2020-09-08T15:51:34.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:33 smithi104 bash[9203]: cluster 2020-09-08T15:51:32.914212+0000 mon.a (mon.0) 1518 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:51:34.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:33 smithi104 bash[9203]: audit 2020-09-08T15:51:32.929053+0000 mon.a (mon.0) 1519 : audit [INF] from='client.? v1:172.21.15.104:0/3147108074' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:51:34.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:33 smithi104 bash[9203]: cluster 2020-09-08T15:51:32.929101+0000 mon.a (mon.0) 1520 : cluster [DBG] osdmap e395: 12 total, 12 up, 12 in 2020-09-08T15:51:34.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:33 smithi055 bash[9296]: cluster 2020-09-08T15:51:32.914212+0000 mon.a (mon.0) 1518 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:51:34.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:33 smithi055 bash[9296]: audit 2020-09-08T15:51:32.929053+0000 mon.a (mon.0) 1519 : audit [INF] from='client.? v1:172.21.15.104:0/3147108074' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:51:34.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:33 smithi055 bash[9296]: cluster 2020-09-08T15:51:32.929101+0000 mon.a (mon.0) 1520 : cluster [DBG] osdmap e395: 12 total, 12 up, 12 in 2020-09-08T15:51:34.501 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:34.612 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:34.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:51:34 smithi104 bash[15273]: debug 2020-09-08T15:51:34.502+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:34.958 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:35.061 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:35.225 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:34 smithi166 bash[9446]: cluster 2020-09-08T15:51:33.810548+0000 mgr.x (mgr.24118) 727 : cluster [DBG] pgmap v1038: 1 pgs: 1 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:35.226 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:35.327 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:35.340 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:34 smithi055 bash[9296]: cluster 2020-09-08T15:51:33.810548+0000 mgr.x (mgr.24118) 727 : cluster [DBG] pgmap v1038: 1 pgs: 1 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:35.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:51:34 smithi104 bash[22305]: debug 2020-09-08T15:51:34.958+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:35.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:34 smithi104 bash[9203]: cluster 2020-09-08T15:51:33.810548+0000 mgr.x (mgr.24118) 727 : cluster [DBG] pgmap v1038: 1 pgs: 1 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:35.479 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:35.581 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:35.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:51:35 smithi166 bash[14254]: debug 2020-09-08T15:51:35.218+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:35.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:51:35 smithi055 bash[19003]: debug 2020-09-08T15:51:35.474+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:35.802 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:35.908 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:51:35.932 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:51:35 smithi104 bash[15273]: debug 2020-09-08T15:51:35.802+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:36.310 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:51:36.412 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:51:36.533 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:51:36.640 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:36.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:51:36 smithi104 bash[19922]: debug 2020-09-08T15:51:36.310+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:36.757 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:51:36 smithi055 bash[16646]: debug 2020-09-08T15:51:36.534+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:36.758 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:36.864 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:36.933 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:51:36.934 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T15:51:36.934 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 0 2020-09-08T15:51:36.934 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:51:37.009 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:36 smithi104 bash[9203]: cluster 2020-09-08T15:51:35.811179+0000 mgr.x (mgr.24118) 728 : cluster [DBG] pgmap v1039: 1 pgs: 1 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:37.022 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:37.128 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:37.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:36 smithi055 bash[9296]: cluster 2020-09-08T15:51:35.811179+0000 mgr.x (mgr.24118) 728 : cluster [DBG] pgmap v1039: 1 pgs: 1 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:37.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:51:36 smithi055 bash[19003]: debug 2020-09-08T15:51:36.762+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:37.345 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:37.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:36 smithi166 bash[9446]: cluster 2020-09-08T15:51:35.811179+0000 mgr.x (mgr.24118) 728 : cluster [DBG] pgmap v1039: 1 pgs: 1 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:37.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:51:37 smithi166 bash[14254]: debug 2020-09-08T15:51:37.022+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:37.450 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:51:37.670 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:51:37.683 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:51:37 smithi104 bash[17588]: debug 2020-09-08T15:51:37.346+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:37.804 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:51:37.924 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:51:37.979 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:51:38.007 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:51:38.025 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:51:38.028 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:51:38.050 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:51:38.076 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:51:38.101 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:51:38.138 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:51:38.170 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:51:38.178 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:51:37 smithi104 bash[19922]: debug 2020-09-08T15:51:37.926+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:38.182 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:51:38.191 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:51:38.223 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:51:38.248 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:51:38.274 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:51:38.276 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:51:38.276 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:51:38.276 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:51:38.276 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:51:38.276 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:51:38.277 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:51:38.277 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:51:38.277 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:51:38.277 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:51:38.277 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:51:38.278 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:51:38.278 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:51:38.287 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:51:38.327 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'unique_pool_3' created 2020-09-08T15:51:38.335 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:51:38 smithi055 bash[11956]: debug 2020-09-08T15:51:38.182+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:38.336 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:38 smithi055 bash[9296]: audit 2020-09-08T15:51:37.808884+0000 mon.c (mon.2) 61 : audit [INF] from='client.? v1:172.21.15.104:0/2920916878' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2020-09-08T15:51:38.336 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:38 smithi055 bash[9296]: audit 2020-09-08T15:51:37.811539+0000 mon.a (mon.0) 1521 : audit [INF] from='client.35633 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2020-09-08T15:51:38.336 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:38 smithi055 bash[9296]: audit 2020-09-08T15:51:37.933016+0000 mon.b (mon.1) 96 : audit [DBG] from='client.? v1:172.21.15.104:0/2826203063' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:51:38.528 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2020-09-08T15:51:38.539 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:51:38.612 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:38 smithi104 bash[9203]: audit 2020-09-08T15:51:37.808884+0000 mon.c (mon.2) 61 : audit [INF] from='client.? v1:172.21.15.104:0/2920916878' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2020-09-08T15:51:38.612 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:38 smithi104 bash[9203]: audit 2020-09-08T15:51:37.811539+0000 mon.a (mon.0) 1521 : audit [INF] from='client.35633 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2020-09-08T15:51:38.613 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:38 smithi104 bash[9203]: audit 2020-09-08T15:51:37.933016+0000 mon.b (mon.1) 96 : audit [DBG] from='client.? v1:172.21.15.104:0/2826203063' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:51:38.613 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:51:38 smithi104 bash[19922]: debug 2020-09-08T15:51:38.530+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:38.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:38 smithi166 bash[9446]: audit 2020-09-08T15:51:37.808884+0000 mon.c (mon.2) 61 : audit [INF] from='client.? v1:172.21.15.104:0/2920916878' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2020-09-08T15:51:38.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:38 smithi166 bash[9446]: audit 2020-09-08T15:51:37.811539+0000 mon.a (mon.0) 1521 : audit [INF] from='client.35633 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2020-09-08T15:51:38.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:38 smithi166 bash[9446]: audit 2020-09-08T15:51:37.933016+0000 mon.b (mon.1) 96 : audit [DBG] from='client.? v1:172.21.15.104:0/2826203063' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:51:38.644 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:38.715 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:51:39.100 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:39.205 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:51:39.361 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:51:39.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:51:39 smithi104 bash[22305]: debug 2020-09-08T15:51:39.102+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:39.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:39 smithi104 bash[9203]: cluster 2020-09-08T15:51:37.811892+0000 mgr.x (mgr.24118) 729 : cluster [DBG] pgmap v1040: 1 pgs: 1 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:39.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:39 smithi104 bash[9203]: audit 2020-09-08T15:51:38.325494+0000 mon.a (mon.0) 1522 : audit [INF] from='client.35633 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2020-09-08T15:51:39.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:39 smithi104 bash[9203]: cluster 2020-09-08T15:51:38.325546+0000 mon.a (mon.0) 1523 : cluster [DBG] osdmap e396: 12 total, 12 up, 12 in 2020-09-08T15:51:39.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:39 smithi104 bash[9203]: cluster 2020-09-08T15:51:39.328733+0000 mon.a (mon.0) 1524 : cluster [DBG] osdmap e397: 12 total, 12 up, 12 in 2020-09-08T15:51:39.466 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:39.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:39 smithi166 bash[9446]: cluster 2020-09-08T15:51:37.811892+0000 mgr.x (mgr.24118) 729 : cluster [DBG] pgmap v1040: 1 pgs: 1 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:39.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:39 smithi166 bash[9446]: audit 2020-09-08T15:51:38.325494+0000 mon.a (mon.0) 1522 : audit [INF] from='client.35633 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2020-09-08T15:51:39.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:39 smithi166 bash[9446]: cluster 2020-09-08T15:51:38.325546+0000 mon.a (mon.0) 1523 : cluster [DBG] osdmap e396: 12 total, 12 up, 12 in 2020-09-08T15:51:39.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:39 smithi166 bash[9446]: cluster 2020-09-08T15:51:39.328733+0000 mon.a (mon.0) 1524 : cluster [DBG] osdmap e397: 12 total, 12 up, 12 in 2020-09-08T15:51:39.657 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:51:39 smithi055 bash[11956]: debug 2020-09-08T15:51:39.366+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:39.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:39 smithi055 bash[9296]: cluster 2020-09-08T15:51:37.811892+0000 mgr.x (mgr.24118) 729 : cluster [DBG] pgmap v1040: 1 pgs: 1 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:39.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:39 smithi055 bash[9296]: audit 2020-09-08T15:51:38.325494+0000 mon.a (mon.0) 1522 : audit [INF] from='client.35633 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2020-09-08T15:51:39.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:39 smithi055 bash[9296]: cluster 2020-09-08T15:51:38.325546+0000 mon.a (mon.0) 1523 : cluster [DBG] osdmap e396: 12 total, 12 up, 12 in 2020-09-08T15:51:39.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:39 smithi055 bash[9296]: cluster 2020-09-08T15:51:39.328733+0000 mon.a (mon.0) 1524 : cluster [DBG] osdmap e397: 12 total, 12 up, 12 in 2020-09-08T15:51:39.786 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:39.896 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:40.179 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:51:39 smithi104 bash[15273]: debug 2020-09-08T15:51:39.782+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:40.260 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:40.365 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:40.444 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool application enable unique_pool_3 rados --yes-i-really-mean-it || true 2020-09-08T15:51:40.491 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:40.550 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:51:40 smithi104 bash[17588]: debug 2020-09-08T15:51:40.262+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:40.581 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:51:40.604 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:40.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:51:40 smithi055 bash[19003]: debug 2020-09-08T15:51:40.494+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:40.751 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:40.856 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:51:40.990 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:51:40.998 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:40 smithi104 bash[9203]: cluster 2020-09-08T15:51:39.670328+0000 osd.1 (osd.1) 80 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:40.998 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:40 smithi104 bash[9203]: cluster 2020-09-08T15:51:39.670447+0000 osd.1 (osd.1) 81 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:40.999 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:40 smithi104 bash[9203]: cluster 2020-09-08T15:51:39.812577+0000 mgr.x (mgr.24118) 730 : cluster [DBG] pgmap v1043: 17 pgs: 3 creating+peering, 1 active+clean, 13 unknown; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:40.999 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:40 smithi104 bash[9203]: cluster 2020-09-08T15:51:40.329133+0000 mon.a (mon.0) 1525 : cluster [DBG] osdmap e398: 12 total, 12 up, 12 in 2020-09-08T15:51:41.092 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:51:41.105 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:51:40 smithi166 bash[11928]: debug 2020-09-08T15:51:40.750+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:41.105 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:40 smithi166 bash[9446]: cluster 2020-09-08T15:51:39.670328+0000 osd.1 (osd.1) 80 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:41.106 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:40 smithi166 bash[9446]: cluster 2020-09-08T15:51:39.670447+0000 osd.1 (osd.1) 81 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:41.106 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:40 smithi166 bash[9446]: cluster 2020-09-08T15:51:39.812577+0000 mgr.x (mgr.24118) 730 : cluster [DBG] pgmap v1043: 17 pgs: 3 creating+peering, 1 active+clean, 13 unknown; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:41.107 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:40 smithi166 bash[9446]: cluster 2020-09-08T15:51:40.329133+0000 mon.a (mon.0) 1525 : cluster [DBG] osdmap e398: 12 total, 12 up, 12 in 2020-09-08T15:51:41.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:51:40 smithi055 bash[16646]: debug 2020-09-08T15:51:40.990+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:41.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:40 smithi055 bash[9296]: cluster 2020-09-08T15:51:39.670328+0000 osd.1 (osd.1) 80 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:41.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:40 smithi055 bash[9296]: cluster 2020-09-08T15:51:39.670447+0000 osd.1 (osd.1) 81 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:41.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:40 smithi055 bash[9296]: cluster 2020-09-08T15:51:39.812577+0000 mgr.x (mgr.24118) 730 : cluster [DBG] pgmap v1043: 17 pgs: 3 creating+peering, 1 active+clean, 13 unknown; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:41.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:40 smithi055 bash[9296]: cluster 2020-09-08T15:51:40.329133+0000 mon.a (mon.0) 1525 : cluster [DBG] osdmap e398: 12 total, 12 up, 12 in 2020-09-08T15:51:41.212 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:51:41.319 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:41.366 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:51:41 smithi166 bash[16593]: debug 2020-09-08T15:51:41.214+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:41.454 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:41.559 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:41.809 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:51:41.867 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:51:41 smithi166 bash[14254]: debug 2020-09-08T15:51:41.454+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:41.944 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:51:41.959 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:42.063 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:42.434 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:51:41 smithi104 bash[15273]: debug 2020-09-08T15:51:41.962+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:42.615 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:42.722 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:51:42.854 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:51:42.866 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:51:42 smithi104 bash[15273]: debug 2020-09-08T15:51:42.614+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:42.960 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:43.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:42 smithi055 bash[9296]: cluster 2020-09-08T15:51:41.813243+0000 mgr.x (mgr.24118) 731 : cluster [DBG] pgmap v1045: 17 pgs: 3 creating+peering, 4 active+clean, 10 unknown; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:43.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:51:42 smithi055 bash[16646]: debug 2020-09-08T15:51:42.858+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:43.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:42 smithi104 bash[9203]: cluster 2020-09-08T15:51:41.813243+0000 mgr.x (mgr.24118) 731 : cluster [DBG] pgmap v1045: 17 pgs: 3 creating+peering, 4 active+clean, 10 unknown; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:43.280 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:43.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:42 smithi166 bash[9446]: cluster 2020-09-08T15:51:41.813243+0000 mgr.x (mgr.24118) 731 : cluster [DBG] pgmap v1045: 17 pgs: 3 creating+peering, 4 active+clean, 10 unknown; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:43.383 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:43.532 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:43.638 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:43.650 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:51:43 smithi104 bash[15273]: debug 2020-09-08T15:51:43.282+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:43.869 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:51:43 smithi166 bash[18960]: debug 2020-09-08T15:51:43.530+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:44.080 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:44.186 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:44.340 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:44.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:51:44 smithi104 bash[15273]: debug 2020-09-08T15:51:44.082+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:44.446 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:44.579 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:44.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:51:44 smithi055 bash[19003]: debug 2020-09-08T15:51:44.342+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:44.683 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:44.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:51:44 smithi166 bash[18960]: debug 2020-09-08T15:51:44.578+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:45.048 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:45.152 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:45.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:44 smithi055 bash[9296]: cluster 2020-09-08T15:51:43.813988+0000 mgr.x (mgr.24118) 732 : cluster [DBG] pgmap v1046: 17 pgs: 1 creating+peering, 16 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:45.165 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:44 smithi104 bash[9203]: cluster 2020-09-08T15:51:43.813988+0000 mgr.x (mgr.24118) 732 : cluster [DBG] pgmap v1046: 17 pgs: 1 creating+peering, 16 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:45.166 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:51:45 smithi104 bash[17588]: debug 2020-09-08T15:51:45.050+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:45.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:44 smithi166 bash[9446]: cluster 2020-09-08T15:51:43.813988+0000 mgr.x (mgr.24118) 732 : cluster [DBG] pgmap v1046: 17 pgs: 1 creating+peering, 16 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:45.377 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:45.427 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:51:45 smithi104 bash[17588]: debug 2020-09-08T15:51:45.378+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:45.481 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:45.729 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:45.836 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:45.850 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:51:45 smithi104 bash[17588]: debug 2020-09-08T15:51:45.730+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:46.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:45 smithi055 bash[9296]: cluster 2020-09-08T15:51:44.645622+0000 osd.6 (osd.6) 63 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:46.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:45 smithi055 bash[9296]: cluster 2020-09-08T15:51:44.645755+0000 osd.6 (osd.6) 64 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:46.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:45 smithi055 bash[9296]: cluster 2020-09-08T15:51:44.887220+0000 mon.a (mon.0) 1526 : cluster [DBG] osdmap e399: 12 total, 12 up, 12 in 2020-09-08T15:51:46.190 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:45 smithi104 bash[9203]: cluster 2020-09-08T15:51:44.645622+0000 osd.6 (osd.6) 63 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:46.190 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:45 smithi104 bash[9203]: cluster 2020-09-08T15:51:44.645755+0000 osd.6 (osd.6) 64 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:46.190 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:45 smithi104 bash[9203]: cluster 2020-09-08T15:51:44.887220+0000 mon.a (mon.0) 1526 : cluster [DBG] osdmap e399: 12 total, 12 up, 12 in 2020-09-08T15:51:46.325 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:46.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:45 smithi166 bash[9446]: cluster 2020-09-08T15:51:44.645622+0000 osd.6 (osd.6) 63 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:46.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:45 smithi166 bash[9446]: cluster 2020-09-08T15:51:44.645755+0000 osd.6 (osd.6) 64 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:46.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:45 smithi166 bash[9446]: cluster 2020-09-08T15:51:44.887220+0000 mon.a (mon.0) 1526 : cluster [DBG] osdmap e399: 12 total, 12 up, 12 in 2020-09-08T15:51:46.431 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:46.586 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:46.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:51:46 smithi104 bash[22305]: debug 2020-09-08T15:51:46.326+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:46.692 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:46.832 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:46.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:51:46 smithi166 bash[14254]: debug 2020-09-08T15:51:46.586+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:46.909 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:51:46.953 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:47.058 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:51:47.059 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:51:47.135 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:46 smithi104 bash[9203]: cluster 2020-09-08T15:51:45.814727+0000 mgr.x (mgr.24118) 733 : cluster [DBG] pgmap v1048: 17 pgs: 17 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:47.136 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:46 smithi104 bash[9203]: audit 2020-09-08T15:51:46.542217+0000 mon.a (mon.0) 1527 : audit [INF] from='client.? v1:172.21.15.104:0/2564625329' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:51:47.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:46 smithi055 bash[9296]: cluster 2020-09-08T15:51:45.814727+0000 mgr.x (mgr.24118) 733 : cluster [DBG] pgmap v1048: 17 pgs: 17 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:47.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:46 smithi055 bash[9296]: audit 2020-09-08T15:51:46.542217+0000 mon.a (mon.0) 1527 : audit [INF] from='client.? v1:172.21.15.104:0/2564625329' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:51:47.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:51:46 smithi055 bash[19003]: debug 2020-09-08T15:51:46.834+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:47.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:46 smithi166 bash[9446]: cluster 2020-09-08T15:51:45.814727+0000 mgr.x (mgr.24118) 733 : cluster [DBG] pgmap v1048: 17 pgs: 17 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:47.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:46 smithi166 bash[9446]: audit 2020-09-08T15:51:46.542217+0000 mon.a (mon.0) 1527 : audit [INF] from='client.? v1:172.21.15.104:0/2564625329' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:51:47.419 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:47.526 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:47.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:51:47 smithi104 bash[15273]: debug 2020-09-08T15:51:47.422+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:47.996 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:48.103 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:48.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:47 smithi055 bash[9296]: cluster 2020-09-08T15:51:46.896464+0000 mon.a (mon.0) 1528 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:51:48.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:47 smithi055 bash[9296]: audit 2020-09-08T15:51:46.910452+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? v1:172.21.15.104:0/2564625329' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:51:48.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:47 smithi055 bash[9296]: cluster 2020-09-08T15:51:46.910555+0000 mon.a (mon.0) 1530 : cluster [DBG] osdmap e400: 12 total, 12 up, 12 in 2020-09-08T15:51:48.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:51:48 smithi104 bash[15273]: debug 2020-09-08T15:51:47.998+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:48.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:47 smithi104 bash[9203]: cluster 2020-09-08T15:51:46.896464+0000 mon.a (mon.0) 1528 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:51:48.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:47 smithi104 bash[9203]: audit 2020-09-08T15:51:46.910452+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? v1:172.21.15.104:0/2564625329' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:51:48.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:47 smithi104 bash[9203]: cluster 2020-09-08T15:51:46.910555+0000 mon.a (mon.0) 1530 : cluster [DBG] osdmap e400: 12 total, 12 up, 12 in 2020-09-08T15:51:48.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:47 smithi166 bash[9446]: cluster 2020-09-08T15:51:46.896464+0000 mon.a (mon.0) 1528 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:51:48.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:47 smithi166 bash[9446]: audit 2020-09-08T15:51:46.910452+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? v1:172.21.15.104:0/2564625329' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:51:48.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:47 smithi166 bash[9446]: cluster 2020-09-08T15:51:46.910555+0000 mon.a (mon.0) 1530 : cluster [DBG] osdmap e400: 12 total, 12 up, 12 in 2020-09-08T15:51:48.608 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:48.713 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:48.839 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:48.908 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:51:48 smithi104 bash[15273]: debug 2020-09-08T15:51:48.610+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:48.945 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:49.073 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:48 smithi055 bash[9296]: cluster 2020-09-08T15:51:47.815444+0000 mgr.x (mgr.24118) 734 : cluster [DBG] pgmap v1050: 17 pgs: 17 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:49.073 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:48 smithi055 bash[9296]: cluster 2020-09-08T15:51:48.014187+0000 osd.11 (osd.11) 66 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:49.073 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:48 smithi055 bash[9296]: cluster 2020-09-08T15:51:48.014326+0000 osd.11 (osd.11) 67 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:49.075 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:51:48 smithi055 bash[14301]: debug 2020-09-08T15:51:48.841+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:49.075 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:49.165 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:48 smithi104 bash[9203]: cluster 2020-09-08T15:51:47.815444+0000 mgr.x (mgr.24118) 734 : cluster [DBG] pgmap v1050: 17 pgs: 17 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:49.165 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:48 smithi104 bash[9203]: cluster 2020-09-08T15:51:48.014187+0000 osd.11 (osd.11) 66 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:49.166 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:48 smithi104 bash[9203]: cluster 2020-09-08T15:51:48.014326+0000 osd.11 (osd.11) 67 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:49.167 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2020-09-08T15:51:49.177 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:49.293 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:49.295 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:51:49.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:48 smithi166 bash[9446]: cluster 2020-09-08T15:51:47.815444+0000 mgr.x (mgr.24118) 734 : cluster [DBG] pgmap v1050: 17 pgs: 17 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:49.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:48 smithi166 bash[9446]: cluster 2020-09-08T15:51:48.014187+0000 osd.11 (osd.11) 66 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:49.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:48 smithi166 bash[9446]: cluster 2020-09-08T15:51:48.014326+0000 osd.11 (osd.11) 67 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:49.398 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:49.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:51:49 smithi055 bash[19003]: debug 2020-09-08T15:51:49.073+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:49.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:51:49 smithi055 bash[19003]: debug 2020-09-08T15:51:49.297+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:49.501 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:51:49.543 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:51:49.560 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:49.563 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:51:49.603 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:51:49.633 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:51:49.659 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:51:49.666 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:49.685 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:51:49.719 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:51:49.746 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:51:49.780 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:51:49.817 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:51:49.836 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:51:49.844 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:51:49.844 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:51:49.844 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:51:49.845 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:51:49.845 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:51:49.845 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:51:49.845 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:51:49.845 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:51:49.846 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:51:49.846 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:51:49.846 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:51:49.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:51:49 smithi166 bash[14254]: debug 2020-09-08T15:51:49.558+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:49.891 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:49.933 INFO:teuthology.orchestra.run.smithi104.stderr:enabled application 'rados' on pool 'unique_pool_3' 2020-09-08T15:51:49.998 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:51:50.004 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:51:49 smithi104 bash[22305]: debug 2020-09-08T15:51:49.890+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:50.004 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:49 smithi104 bash[9203]: cluster 2020-09-08T15:51:48.926422+0000 mon.a (mon.0) 1531 : cluster [DBG] osdmap e401: 12 total, 12 up, 12 in 2020-09-08T15:51:50.004 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:49 smithi104 bash[9203]: audit 2020-09-08T15:51:49.478691+0000 mon.a (mon.0) 1532 : audit [DBG] from='client.? v1:172.21.15.104:0/1193884844' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:51:50.005 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:49 smithi104 bash[9203]: audit 2020-09-08T15:51:49.608410+0000 mon.a (mon.0) 1533 : audit [INF] from='client.? v1:172.21.15.104:0/3628942464' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T15:51:50.117 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:51:50.223 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:50.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:49 smithi166 bash[9446]: cluster 2020-09-08T15:51:48.926422+0000 mon.a (mon.0) 1531 : cluster [DBG] osdmap e401: 12 total, 12 up, 12 in 2020-09-08T15:51:50.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:49 smithi166 bash[9446]: audit 2020-09-08T15:51:49.478691+0000 mon.a (mon.0) 1532 : audit [DBG] from='client.? v1:172.21.15.104:0/1193884844' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:51:50.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:49 smithi166 bash[9446]: audit 2020-09-08T15:51:49.608410+0000 mon.a (mon.0) 1533 : audit [INF] from='client.? v1:172.21.15.104:0/3628942464' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T15:51:50.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:51:50 smithi055 bash[11956]: debug 2020-09-08T15:51:50.121+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:50.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:49 smithi055 bash[9296]: cluster 2020-09-08T15:51:48.926422+0000 mon.a (mon.0) 1531 : cluster [DBG] osdmap e401: 12 total, 12 up, 12 in 2020-09-08T15:51:50.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:49 smithi055 bash[9296]: audit 2020-09-08T15:51:49.478691+0000 mon.a (mon.0) 1532 : audit [DBG] from='client.? v1:172.21.15.104:0/1193884844' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:51:50.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:49 smithi055 bash[9296]: audit 2020-09-08T15:51:49.608410+0000 mon.a (mon.0) 1533 : audit [INF] from='client.? v1:172.21.15.104:0/3628942464' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T15:51:50.544 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:50.639 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:51:50.647 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:50.780 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:51:50.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:51:50 smithi104 bash[22305]: debug 2020-09-08T15:51:50.546+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:51.063 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:51:51.167 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:51.181 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:51:51 smithi104 bash[15273]: debug 2020-09-08T15:51:51.066+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:51.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:50 smithi104 bash[9203]: cluster 2020-09-08T15:51:49.816107+0000 mgr.x (mgr.24118) 735 : cluster [DBG] pgmap v1052: 17 pgs: 17 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:51.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:50 smithi104 bash[9203]: audit 2020-09-08T15:51:49.936252+0000 mon.a (mon.0) 1534 : audit [INF] from='client.? v1:172.21.15.104:0/3628942464' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T15:51:51.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:50 smithi104 bash[9203]: cluster 2020-09-08T15:51:49.936353+0000 mon.a (mon.0) 1535 : cluster [DBG] osdmap e402: 12 total, 12 up, 12 in 2020-09-08T15:51:51.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:50 smithi166 bash[9446]: cluster 2020-09-08T15:51:49.816107+0000 mgr.x (mgr.24118) 735 : cluster [DBG] pgmap v1052: 17 pgs: 17 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:51.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:50 smithi166 bash[9446]: audit 2020-09-08T15:51:49.936252+0000 mon.a (mon.0) 1534 : audit [INF] from='client.? v1:172.21.15.104:0/3628942464' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T15:51:51.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:50 smithi166 bash[9446]: cluster 2020-09-08T15:51:49.936353+0000 mon.a (mon.0) 1535 : cluster [DBG] osdmap e402: 12 total, 12 up, 12 in 2020-09-08T15:51:51.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:50 smithi055 bash[9296]: cluster 2020-09-08T15:51:49.816107+0000 mgr.x (mgr.24118) 735 : cluster [DBG] pgmap v1052: 17 pgs: 17 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:51.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:50 smithi055 bash[9296]: audit 2020-09-08T15:51:49.936252+0000 mon.a (mon.0) 1534 : audit [INF] from='client.? v1:172.21.15.104:0/3628942464' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T15:51:51.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:50 smithi055 bash[9296]: cluster 2020-09-08T15:51:49.936353+0000 mon.a (mon.0) 1535 : cluster [DBG] osdmap e402: 12 total, 12 up, 12 in 2020-09-08T15:51:51.564 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:51.669 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:51.827 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:51.880 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:51:51 smithi104 bash[22305]: debug 2020-09-08T15:51:51.566+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:51.932 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:51.945 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:51:51 smithi166 bash[11928]: debug 2020-09-08T15:51:51.826+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:52.097 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:52.208 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:51:52.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:51:52 smithi166 bash[11928]: debug 2020-09-08T15:51:52.090+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:52.379 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:51:52.474 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:51:52 smithi104 bash[19922]: debug 2020-09-08T15:51:52.382+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:52.486 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:52.618 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:52.724 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:52.861 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:52.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:51:52 smithi166 bash[18960]: debug 2020-09-08T15:51:52.618+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:52.967 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:53.087 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:51:52 smithi055 bash[19003]: debug 2020-09-08T15:51:52.865+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:53.087 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:52 smithi055 bash[9296]: cluster 2020-09-08T15:51:51.451211+0000 osd.4 (osd.4) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:53.088 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:52 smithi055 bash[9296]: cluster 2020-09-08T15:51:51.451334+0000 osd.4 (osd.4) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:53.088 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:52 smithi055 bash[9296]: cluster 2020-09-08T15:51:51.816873+0000 mgr.x (mgr.24118) 736 : cluster [DBG] pgmap v1054: 17 pgs: 17 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:53.088 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:52 smithi055 bash[9296]: cluster 2020-09-08T15:51:51.949023+0000 mon.a (mon.0) 1536 : cluster [DBG] osdmap e403: 12 total, 12 up, 12 in 2020-09-08T15:51:53.089 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:53.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:52 smithi104 bash[9203]: cluster 2020-09-08T15:51:51.451211+0000 osd.4 (osd.4) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:53.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:52 smithi104 bash[9203]: cluster 2020-09-08T15:51:51.451334+0000 osd.4 (osd.4) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:53.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:52 smithi104 bash[9203]: cluster 2020-09-08T15:51:51.816873+0000 mgr.x (mgr.24118) 736 : cluster [DBG] pgmap v1054: 17 pgs: 17 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:53.186 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:52 smithi104 bash[9203]: cluster 2020-09-08T15:51:51.949023+0000 mon.a (mon.0) 1536 : cluster [DBG] osdmap e403: 12 total, 12 up, 12 in 2020-09-08T15:51:53.194 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:53.311 INFO:teuthology.orchestra.run.smithi166:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.2 -b 65536 --object-size 65536 -p unique_pool_3 bench 90 write' 2020-09-08T15:51:53.314 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2020-09-08T15:51:53.325 INFO:teuthology.orchestra.run:waiting for 3000 2020-09-08T15:51:53.326 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:53.357 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:hints = 1 2020-09-08T15:51:53.357 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2020-09-08T15:51:53.357 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Object prefix: benchmark_data_smithi166_37820 2020-09-08T15:51:53.358 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:51:53.358 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 0 0 0 0 0 0 - 0 2020-09-08T15:51:53.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:52 smithi166 bash[9446]: cluster 2020-09-08T15:51:51.451211+0000 osd.4 (osd.4) 41 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:51:53.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:52 smithi166 bash[9446]: cluster 2020-09-08T15:51:51.451334+0000 osd.4 (osd.4) 42 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:51:53.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:52 smithi166 bash[9446]: cluster 2020-09-08T15:51:51.816873+0000 mgr.x (mgr.24118) 736 : cluster [DBG] pgmap v1054: 17 pgs: 17 active+clean; 0 B data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:51:53.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:52 smithi166 bash[9446]: cluster 2020-09-08T15:51:51.949023+0000 mon.a (mon.0) 1536 : cluster [DBG] osdmap e403: 12 total, 12 up, 12 in 2020-09-08T15:51:53.412 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:51:53 smithi055 bash[14301]: debug 2020-09-08T15:51:53.089+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:53.412 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:51:53 smithi055 bash[14301]: debug 2020-09-08T15:51:53.325+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:53.428 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:51:53.647 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:51:53 smithi104 bash[19922]: debug 2020-09-08T15:51:53.646+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:53.647 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:51:53.751 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:51:53.874 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:51:53.883 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:51:53.978 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:54.049 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:51:54.126 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:54.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:51:53 smithi055 bash[16646]: debug 2020-09-08T15:51:53.877+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:54.236 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:54.245 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:51:54 smithi166 bash[18960]: debug 2020-09-08T15:51:54.126+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:54.358 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 1 16 3802 3786 236.614 236.625 0.00208062 0.00420558 2020-09-08T15:51:54.366 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:51:54.470 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:54.599 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:51:54 smithi166 bash[14254]: debug 2020-09-08T15:51:54.366+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:54.601 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:54.704 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:54.854 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:54.876 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:51:54 smithi166 bash[18960]: debug 2020-09-08T15:51:54.598+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:54.956 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:55.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:51:54 smithi055 bash[19003]: debug 2020-09-08T15:51:54.853+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:55.165 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:54 smithi055 bash[9296]: cluster 2020-09-08T15:51:53.817492+0000 mgr.x (mgr.24118) 737 : cluster [DBG] pgmap v1056: 17 pgs: 17 active+clean; 6.2 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s wr, 16 op/s 2020-09-08T15:51:55.165 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:54 smithi055 bash[9296]: cluster 2020-09-08T15:51:54.772983+0000 mon.a (mon.0) 1537 : cluster [DBG] osdmap e404: 12 total, 12 up, 12 in 2020-09-08T15:51:55.337 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:54 smithi166 bash[9446]: cluster 2020-09-08T15:51:53.817492+0000 mgr.x (mgr.24118) 737 : cluster [DBG] pgmap v1056: 17 pgs: 17 active+clean; 6.2 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s wr, 16 op/s 2020-09-08T15:51:55.338 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:54 smithi166 bash[9446]: cluster 2020-09-08T15:51:54.772983+0000 mon.a (mon.0) 1537 : cluster [DBG] osdmap e404: 12 total, 12 up, 12 in 2020-09-08T15:51:55.338 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:55.358 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 2 16 7731 7715 241.074 245.562 0.00310085 0.00413645 2020-09-08T15:51:55.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:54 smithi104 bash[9203]: cluster 2020-09-08T15:51:53.817492+0000 mgr.x (mgr.24118) 737 : cluster [DBG] pgmap v1056: 17 pgs: 17 active+clean; 6.2 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s wr, 16 op/s 2020-09-08T15:51:55.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:54 smithi104 bash[9203]: cluster 2020-09-08T15:51:54.772983+0000 mon.a (mon.0) 1537 : cluster [DBG] osdmap e404: 12 total, 12 up, 12 in 2020-09-08T15:51:55.442 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:55.617 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:51:55 smithi166 bash[18960]: debug 2020-09-08T15:51:55.338+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:56.358 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 3 16 9890 9874 205.691 134.938 0.0036662 0.00485585 2020-09-08T15:51:56.381 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:51:56.484 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:56.604 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:51:56.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:51:56 smithi104 bash[22305]: debug 2020-09-08T15:51:56.382+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:56.706 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:56.854 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:56.906 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:51:56 smithi055 bash[19003]: debug 2020-09-08T15:51:56.605+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:56.958 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:56.967 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:56 smithi166 bash[9446]: cluster 2020-09-08T15:51:55.817967+0000 mgr.x (mgr.24118) 738 : cluster [DBG] pgmap v1058: 17 pgs: 17 active+clean; 15 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s wr, 38 op/s 2020-09-08T15:51:56.967 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:51:56 smithi166 bash[11928]: debug 2020-09-08T15:51:56.854+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:57.076 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:51:57.179 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:57.294 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:56 smithi055 bash[9296]: cluster 2020-09-08T15:51:55.817967+0000 mgr.x (mgr.24118) 738 : cluster [DBG] pgmap v1058: 17 pgs: 17 active+clean; 15 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s wr, 38 op/s 2020-09-08T15:51:57.294 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:51:57.358 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 4 15 13417 13402 209.388 220.5 0.00480566 0.00477132 2020-09-08T15:51:57.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:51:57 smithi166 bash[18960]: debug 2020-09-08T15:51:57.074+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:57.399 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:57.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:56 smithi104 bash[9203]: cluster 2020-09-08T15:51:55.817967+0000 mgr.x (mgr.24118) 738 : cluster [DBG] pgmap v1058: 17 pgs: 17 active+clean; 15 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s wr, 38 op/s 2020-09-08T15:51:57.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:51:57 smithi055 bash[14301]: debug 2020-09-08T15:51:57.297+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:58.364 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 5 16 16901 16885 211.044 217.688 0.0057018 0.00473381 2020-09-08T15:51:58.952 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:51:59.060 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:59.232 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:51:58 smithi104 bash[17588]: debug 2020-09-08T15:51:58.954+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:51:59.432 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:51:58 smithi104 bash[9203]: cluster 2020-09-08T15:51:57.818549+0000 mgr.x (mgr.24118) 739 : cluster [DBG] pgmap v1059: 17 pgs: 17 active+clean; 538 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s wr, 1.09k op/s 2020-09-08T15:51:59.433 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:51:58 smithi055 bash[9296]: cluster 2020-09-08T15:51:57.818549+0000 mgr.x (mgr.24118) 739 : cluster [DBG] pgmap v1059: 17 pgs: 17 active+clean; 538 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s wr, 1.09k op/s 2020-09-08T15:51:59.434 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 6 16 19953 19937 207.659 190.75 0.0029725 0.00480654 2020-09-08T15:51:59.434 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:51:58 smithi166 bash[9446]: cluster 2020-09-08T15:51:57.818549+0000 mgr.x (mgr.24118) 739 : cluster [DBG] pgmap v1059: 17 pgs: 17 active+clean; 538 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s wr, 1.09k op/s 2020-09-08T15:51:59.511 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:51:59.512 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2020-09-08T15:51:59.512 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2020-09-08T15:51:59.512 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T15:51:59.512 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.513 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:154541 3.6 3:6d1688dd:::benchmark_data_smithi166_26097_object30380:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e263)", 2020-09-08T15:51:59.513 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:36.806581+0000", 2020-09-08T15:51:59.513 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 442.70434654500002, 2020-09-08T15:51:59.513 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.25021634799999998, 2020-09-08T15:51:59.514 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:51:59.514 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:59.514 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:51:59.515 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:51:59.515 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:59.515 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 154541 2020-09-08T15:51:59.515 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.516 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:51:59.516 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.516 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:51:59.516 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:36.806581+0000", 2020-09-08T15:51:59.516 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:51:59.517 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.517 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.517 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:51:59.517 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:36.806581+0000", 2020-09-08T15:51:59.517 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999781 2020-09-08T15:51:59.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.518 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:51:59.518 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:36.806559+0000", 2020-09-08T15:51:59.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9808999999999997e-05 2020-09-08T15:51:59.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:51:59.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:36.806609+0000", 2020-09-08T15:51:59.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9230000000000008e-06 2020-09-08T15:51:59.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:51:59.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:36.806618+0000", 2020-09-08T15:51:59.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.216e-06 2020-09-08T15:51:59.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:51:59.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:36.806622+0000", 2020-09-08T15:51:59.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18533844299999999 2020-09-08T15:51:59.521 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.522 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.522 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:51:59.522 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:36.991961+0000", 2020-09-08T15:51:59.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012709700000000001 2020-09-08T15:51:59.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:51:59.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:36.992088+0000", 2020-09-08T15:51:59.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018045100000000001 2020-09-08T15:51:59.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.524 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:51:59.524 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:36.992268+0000", 2020-09-08T15:51:59.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013989474999999999 2020-09-08T15:51:59.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:51:59.527 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:37.006258+0000", 2020-09-08T15:51:59.527 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.047477640000000002 2020-09-08T15:51:59.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.528 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:59.529 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:37.053735+0000", 2020-09-08T15:51:59.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6804999999999994e-05 2020-09-08T15:51:59.529 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.529 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.529 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:51:59.530 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:37.053812+0000", 2020-09-08T15:51:59.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0029855609999999999 2020-09-08T15:51:59.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:51:59.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:37.056798+0000", 2020-09-08T15:51:59.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.25021634799999998 2020-09-08T15:51:59.531 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.531 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.531 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.532 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:159518 3.6 3:630d1e06:::benchmark_data_smithi166_26097_object35357:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e266)", 2020-09-08T15:51:59.532 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:45.532156+0000", 2020-09-08T15:51:59.532 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 433.978771607, 2020-09-08T15:51:59.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.27662112599999999, 2020-09-08T15:51:59.533 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:51:59.533 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:59.533 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:51:59.533 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:51:59.534 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:59.534 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 159518 2020-09-08T15:51:59.534 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.534 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:51:59.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:51:59.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.532156+0000", 2020-09-08T15:51:59.536 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:51:59.536 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.536 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.536 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:51:59.537 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.532156+0000", 2020-09-08T15:51:59.537 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999557 2020-09-08T15:51:59.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.537 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:51:59.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.532112+0000", 2020-09-08T15:51:59.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0600999999999995e-05 2020-09-08T15:51:59.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.538 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.538 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:51:59.539 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.532203+0000", 2020-09-08T15:51:59.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4331000000000001e-05 2020-09-08T15:51:59.539 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.539 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:51:59.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.532217+0000", 2020-09-08T15:51:59.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3280000000000002e-06 2020-09-08T15:51:59.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:51:59.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.532223+0000", 2020-09-08T15:51:59.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21526457099999999 2020-09-08T15:51:59.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:51:59.541 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.747488+0000", 2020-09-08T15:51:59.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012507100000000001 2020-09-08T15:51:59.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:51:59.543 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.747613+0000", 2020-09-08T15:51:59.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019435500000000001 2020-09-08T15:51:59.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.543 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:51:59.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.747807+0000", 2020-09-08T15:51:59.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.029396507999999998 2020-09-08T15:51:59.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:51:59.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.777204+0000", 2020-09-08T15:51:59.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.028517411999999999 2020-09-08T15:51:59.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:59.546 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.805721+0000", 2020-09-08T15:51:59.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9368000000000004e-05 2020-09-08T15:51:59.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.547 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:51:59.547 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.805790+0000", 2020-09-08T15:51:59.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0029869940000000002 2020-09-08T15:51:59.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:51:59.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.808777+0000", 2020-09-08T15:51:59.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.27662112599999999 2020-09-08T15:51:59.548 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.549 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.549 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.550 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:159584 3.6 3:6026c71b:::benchmark_data_smithi166_26097_object35423:head [delete] snapc 0=[] ondisk+write+known_if_redirected e266)", 2020-09-08T15:51:59.550 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:45.537206+0000", 2020-09-08T15:51:59.550 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 433.973722076, 2020-09-08T15:51:59.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.27456241100000001, 2020-09-08T15:51:59.550 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:51:59.551 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:59.551 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:51:59.551 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:51:59.551 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:59.551 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 159584 2020-09-08T15:51:59.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.552 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:51:59.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:51:59.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.537206+0000", 2020-09-08T15:51:59.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:51:59.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.554 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:51:59.555 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.537206+0000", 2020-09-08T15:51:59.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999633 2020-09-08T15:51:59.555 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.555 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.555 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:51:59.556 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.537169+0000", 2020-09-08T15:51:59.556 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2824000000000005e-05 2020-09-08T15:51:59.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.557 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.557 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:51:59.557 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.537252+0000", 2020-09-08T15:51:59.557 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3548e-05 2020-09-08T15:51:59.557 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.558 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:51:59.558 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.537266+0000", 2020-09-08T15:51:59.558 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4180000000000002e-06 2020-09-08T15:51:59.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.559 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:51:59.559 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.537272+0000", 2020-09-08T15:51:59.559 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21571037300000001 2020-09-08T15:51:59.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.560 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:51:59.560 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.752982+0000", 2020-09-08T15:51:59.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010925900000000001 2020-09-08T15:51:59.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.561 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.561 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:51:59.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.753092+0000", 2020-09-08T15:51:59.561 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012448599999999999 2020-09-08T15:51:59.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.562 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.562 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:51:59.562 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.753216+0000", 2020-09-08T15:51:59.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.023998478 2020-09-08T15:51:59.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:51:59.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.777215+0000", 2020-09-08T15:51:59.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.031656517000000002 2020-09-08T15:51:59.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.564 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:59.564 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.808871+0000", 2020-09-08T15:51:59.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1524000000000001e-05 2020-09-08T15:51:59.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.565 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.565 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:51:59.565 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.808953+0000", 2020-09-08T15:51:59.565 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0028157099999999999 2020-09-08T15:51:59.565 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.566 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.566 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:51:59.566 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.811768+0000", 2020-09-08T15:51:59.566 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.27456241100000001 2020-09-08T15:51:59.566 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.567 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.567 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.567 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.568 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.568 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:159544 3.6 3:6e79caa0:::benchmark_data_smithi166_26097_object35383:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e266)", 2020-09-08T15:51:59.568 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:45.547487+0000", 2020-09-08T15:51:59.568 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 433.96344100599998, 2020-09-08T15:51:59.569 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26711421800000001, 2020-09-08T15:51:59.569 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:51:59.569 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:59.569 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:51:59.570 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:51:59.570 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:59.571 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 159544 2020-09-08T15:51:59.571 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.571 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:51:59.571 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.572 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:51:59.573 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.547487+0000", 2020-09-08T15:51:59.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:51:59.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.574 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:51:59.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.547487+0000", 2020-09-08T15:51:59.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999781 2020-09-08T15:51:59.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:51:59.576 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.547465+0000", 2020-09-08T15:51:59.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9867000000000002e-05 2020-09-08T15:51:59.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.576 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:51:59.576 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.547515+0000", 2020-09-08T15:51:59.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8439999999999993e-06 2020-09-08T15:51:59.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.578 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.578 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:51:59.578 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.547525+0000", 2020-09-08T15:51:59.578 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3600000000000004e-06 2020-09-08T15:51:59.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.579 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.579 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:51:59.579 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.547530+0000", 2020-09-08T15:51:59.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21145528899999999 2020-09-08T15:51:59.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:51:59.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.758985+0000", 2020-09-08T15:51:59.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000184446 2020-09-08T15:51:59.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.581 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.581 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:51:59.581 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.759170+0000", 2020-09-08T15:51:59.581 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00024394799999999999 2020-09-08T15:51:59.581 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.581 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:51:59.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.759414+0000", 2020-09-08T15:51:59.582 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.023618573 2020-09-08T15:51:59.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:51:59.583 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.783032+0000", 2020-09-08T15:51:59.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.028794650000000001 2020-09-08T15:51:59.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.584 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:59.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.811827+0000", 2020-09-08T15:51:59.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0613999999999999e-05 2020-09-08T15:51:59.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.585 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:51:59.585 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.811888+0000", 2020-09-08T15:51:59.585 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0027136370000000001 2020-09-08T15:51:59.585 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.585 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:51:59.586 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.814601+0000", 2020-09-08T15:51:59.586 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26711421800000001 2020-09-08T15:51:59.586 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.587 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.587 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.587 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.587 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.587 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:159589 3.6 3:6e217afc:::benchmark_data_smithi166_26097_object35428:head [delete] snapc 0=[] ondisk+write+known_if_redirected e266)", 2020-09-08T15:51:59.588 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:45.550419+0000", 2020-09-08T15:51:59.588 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 433.96050900900002, 2020-09-08T15:51:59.588 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26680337999999998, 2020-09-08T15:51:59.589 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:51:59.589 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:59.589 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:51:59.589 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:51:59.589 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:59.590 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 159589 2020-09-08T15:51:59.590 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.590 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:51:59.590 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.590 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:51:59.591 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.550419+0000", 2020-09-08T15:51:59.591 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:51:59.591 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.591 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.591 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:51:59.592 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.550419+0000", 2020-09-08T15:51:59.592 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999762 2020-09-08T15:51:59.592 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.592 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:51:59.593 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.550395+0000", 2020-09-08T15:51:59.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2611999999999997e-05 2020-09-08T15:51:59.593 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.593 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.594 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:51:59.594 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.550448+0000", 2020-09-08T15:51:59.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0429999999999996e-06 2020-09-08T15:51:59.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:51:59.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.550457+0000", 2020-09-08T15:51:59.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5759999999999999e-06 2020-09-08T15:51:59.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.597 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:51:59.597 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.550461+0000", 2020-09-08T15:51:59.597 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.215614627 2020-09-08T15:51:59.597 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.598 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.598 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:51:59.598 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.766076+0000", 2020-09-08T15:51:59.599 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013089300000000001 2020-09-08T15:51:59.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.599 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.599 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:51:59.599 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.766207+0000", 2020-09-08T15:51:59.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000177202 2020-09-08T15:51:59.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:51:59.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.766384+0000", 2020-09-08T15:51:59.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.016659257 2020-09-08T15:51:59.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:51:59.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.783043+0000", 2020-09-08T15:51:59.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.031611454999999997 2020-09-08T15:51:59.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:59.604 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.814655+0000", 2020-09-08T15:51:59.604 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5247999999999998e-05 2020-09-08T15:51:59.604 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:51:59.605 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.814700+0000", 2020-09-08T15:51:59.605 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0025223730000000001 2020-09-08T15:51:59.605 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.605 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.605 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:51:59.606 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.817222+0000", 2020-09-08T15:51:59.606 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26680337999999998 2020-09-08T15:51:59.606 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.607 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.607 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.607 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.608 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:159591 3.6 3:6f2d1afc:::benchmark_data_smithi166_26097_object35430:head [delete] snapc 0=[] ondisk+write+known_if_redirected e266)", 2020-09-08T15:51:59.608 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:45.556028+0000", 2020-09-08T15:51:59.608 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 433.95490014000001, 2020-09-08T15:51:59.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26371128599999999, 2020-09-08T15:51:59.611 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:51:59.611 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:59.612 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:51:59.612 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:51:59.612 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:59.612 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 159591 2020-09-08T15:51:59.613 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.613 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:51:59.613 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.613 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:51:59.614 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.556028+0000", 2020-09-08T15:51:59.614 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:51:59.614 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.614 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:51:59.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.556028+0000", 2020-09-08T15:51:59.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999762 2020-09-08T15:51:59.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.615 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:51:59.616 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.556004+0000", 2020-09-08T15:51:59.616 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2824000000000001e-05 2020-09-08T15:51:59.616 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.616 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.617 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:51:59.617 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.556057+0000", 2020-09-08T15:51:59.617 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0103e-05 2020-09-08T15:51:59.617 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.617 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.618 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:51:59.618 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.556067+0000", 2020-09-08T15:51:59.618 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6940000000000003e-06 2020-09-08T15:51:59.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:51:59.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.556072+0000", 2020-09-08T15:51:59.619 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21557869800000001 2020-09-08T15:51:59.619 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.620 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.620 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:51:59.620 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.771650+0000", 2020-09-08T15:51:59.620 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000133487 2020-09-08T15:51:59.620 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.621 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.621 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:51:59.621 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.771784+0000", 2020-09-08T15:51:59.621 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018799800000000001 2020-09-08T15:51:59.621 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:51:59.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.771972+0000", 2020-09-08T15:51:59.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.011247484 2020-09-08T15:51:59.623 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.623 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.623 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:51:59.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.783219+0000", 2020-09-08T15:51:59.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.034083690999999999 2020-09-08T15:51:59.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.624 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:59.624 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.817303+0000", 2020-09-08T15:51:59.624 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3438999999999999e-05 2020-09-08T15:51:59.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.625 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:51:59.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.817377+0000", 2020-09-08T15:51:59.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0023625270000000001 2020-09-08T15:51:59.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:51:59.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.819739+0000", 2020-09-08T15:51:59.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26371128599999999 2020-09-08T15:51:59.627 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.627 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.627 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.628 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:159557 3.6 3:6ef60a98:::benchmark_data_smithi166_26097_object35396:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e266)", 2020-09-08T15:51:59.628 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:44:45.566679+0000", 2020-09-08T15:51:59.628 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 433.94424886799999, 2020-09-08T15:51:59.628 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26237816400000002, 2020-09-08T15:51:59.629 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:51:59.629 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:59.629 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:51:59.629 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:51:59.629 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:59.630 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 159557 2020-09-08T15:51:59.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.630 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:51:59.630 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.630 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:51:59.631 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.566679+0000", 2020-09-08T15:51:59.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:51:59.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.631 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.631 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:51:59.632 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.566679+0000", 2020-09-08T15:51:59.632 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999938 2020-09-08T15:51:59.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.632 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.632 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:51:59.633 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.566617+0000", 2020-09-08T15:51:59.633 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011448 2020-09-08T15:51:59.633 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.634 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:51:59.634 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.566731+0000", 2020-09-08T15:51:59.634 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4863e-05 2020-09-08T15:51:59.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.634 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.635 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:51:59.635 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.566746+0000", 2020-09-08T15:51:59.635 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4710000000000004e-06 2020-09-08T15:51:59.635 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.635 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.636 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:51:59.636 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.566753+0000", 2020-09-08T15:51:59.636 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.210512002 2020-09-08T15:51:59.636 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.636 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.637 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:51:59.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.777265+0000", 2020-09-08T15:51:59.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019517300000000001 2020-09-08T15:51:59.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.638 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:51:59.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.777460+0000", 2020-09-08T15:51:59.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012716999999999999 2020-09-08T15:51:59.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:51:59.639 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.777587+0000", 2020-09-08T15:51:59.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0056399199999999997 2020-09-08T15:51:59.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.640 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:51:59.640 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.783227+0000", 2020-09-08T15:51:59.640 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.043802909000000001 2020-09-08T15:51:59.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.641 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.641 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:59.641 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.827030+0000", 2020-09-08T15:51:59.641 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7509000000000001e-05 2020-09-08T15:51:59.641 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:51:59.642 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.827087+0000", 2020-09-08T15:51:59.642 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0019697650000000001 2020-09-08T15:51:59.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.643 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.643 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:51:59.643 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:44:45.829057+0000", 2020-09-08T15:51:59.643 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26237816400000002 2020-09-08T15:51:59.643 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.644 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.644 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.644 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.644 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.645 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:210004 3.1 3:8d06d744:::benchmark_data_smithi166_26097_object85843:head [delete] snapc 0=[] ondisk+write+known_if_redirected e286)", 2020-09-08T15:51:59.645 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:45:57.366855+0000", 2020-09-08T15:51:59.645 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 362.144073295, 2020-09-08T15:51:59.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.23795509100000001, 2020-09-08T15:51:59.645 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:51:59.646 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:59.646 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:51:59.646 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:51:59.646 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:59.646 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 210004 2020-09-08T15:51:59.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.647 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:51:59.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:51:59.647 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.366855+0000", 2020-09-08T15:51:59.648 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:51:59.648 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.648 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.649 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:51:59.649 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.366855+0000", 2020-09-08T15:51:59.649 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999585 2020-09-08T15:51:59.650 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.650 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.650 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:51:59.650 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.366813+0000", 2020-09-08T15:51:59.651 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3678000000000007e-05 2020-09-08T15:51:59.651 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.651 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.651 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:51:59.651 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.366907+0000", 2020-09-08T15:51:59.652 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5118e-05 2020-09-08T15:51:59.652 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.652 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.652 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:51:59.652 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.366922+0000", 2020-09-08T15:51:59.653 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6880000000000002e-06 2020-09-08T15:51:59.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.653 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.653 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:51:59.653 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.366929+0000", 2020-09-08T15:51:59.654 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0086118610000000002 2020-09-08T15:51:59.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:51:59.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.375541+0000", 2020-09-08T15:51:59.655 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010226999999999999 2020-09-08T15:51:59.655 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.655 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.655 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:51:59.655 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.375643+0000", 2020-09-08T15:51:59.656 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000106678 2020-09-08T15:51:59.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.656 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.656 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T15:51:59.657 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.375750+0000", 2020-09-08T15:51:59.657 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.224626567 2020-09-08T15:51:59.657 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.657 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.657 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:51:59.658 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.600376+0000", 2020-09-08T15:51:59.658 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2773000000000004e-05 2020-09-08T15:51:59.658 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.658 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.658 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:59.659 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.600439+0000", 2020-09-08T15:51:59.659 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8282000000000006e-05 2020-09-08T15:51:59.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.659 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.659 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:51:59.660 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.600527+0000", 2020-09-08T15:51:59.660 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0042825010000000002 2020-09-08T15:51:59.660 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.660 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.660 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:51:59.661 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.604810+0000", 2020-09-08T15:51:59.661 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.23795509100000001 2020-09-08T15:51:59.661 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.661 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.662 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.662 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.662 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.662 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35087.0:210010 3.1 3:87751333:::benchmark_data_smithi166_26097_object85849:head [delete] snapc 0=[] ondisk+write+known_if_redirected e286)", 2020-09-08T15:51:59.662 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:45:57.370481+0000", 2020-09-08T15:51:59.663 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 362.14044668000003, 2020-09-08T15:51:59.663 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.24020914400000001, 2020-09-08T15:51:59.663 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:51:59.663 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:59.664 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:51:59.664 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35087", 2020-09-08T15:51:59.664 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2207709665", 2020-09-08T15:51:59.665 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 210010 2020-09-08T15:51:59.665 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.666 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:51:59.671 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.673 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:51:59.674 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.370481+0000", 2020-09-08T15:51:59.675 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:51:59.675 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.675 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.675 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:51:59.675 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.370481+0000", 2020-09-08T15:51:59.676 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999628 2020-09-08T15:51:59.676 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.676 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.676 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:51:59.676 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.370444+0000", 2020-09-08T15:51:59.677 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6192000000000004e-05 2020-09-08T15:51:59.677 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.677 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.677 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:51:59.677 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.370530+0000", 2020-09-08T15:51:59.678 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.312e-05 2020-09-08T15:51:59.678 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.678 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.678 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:51:59.679 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.370544+0000", 2020-09-08T15:51:59.679 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.195e-06 2020-09-08T15:51:59.679 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.679 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.679 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:51:59.679 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.370550+0000", 2020-09-08T15:51:59.680 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010056516999999999 2020-09-08T15:51:59.680 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.680 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.680 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:51:59.680 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.380606+0000", 2020-09-08T15:51:59.681 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013948899999999999 2020-09-08T15:51:59.681 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.681 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.681 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:51:59.681 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.380746+0000", 2020-09-08T15:51:59.682 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014580600000000001 2020-09-08T15:51:59.682 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.682 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.682 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T15:51:59.682 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.380892+0000", 2020-09-08T15:51:59.683 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.22394421 2020-09-08T15:51:59.683 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.683 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.683 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:51:59.683 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.604836+0000", 2020-09-08T15:51:59.684 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0057373220000000004 2020-09-08T15:51:59.684 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.684 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.684 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:59.684 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.610573+0000", 2020-09-08T15:51:59.684 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5308000000000006e-05 2020-09-08T15:51:59.685 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.685 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.685 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:51:59.685 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.610638+0000", 2020-09-08T15:51:59.685 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1950999999999998e-05 2020-09-08T15:51:59.686 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.686 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.686 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:51:59.686 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:45:57.610690+0000", 2020-09-08T15:51:59.686 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.24020914400000001 2020-09-08T15:51:59.687 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.687 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.687 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.687 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.687 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.688 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35381.0:199655 4.b 4:d51d39ba:::benchmark_data_smithi166_32054_object57513:head [delete] snapc 0=[] ondisk+write+known_if_redirected e365)", 2020-09-08T15:51:59.688 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:49:51.467870+0000", 2020-09-08T15:51:59.688 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 128.043057834, 2020-09-08T15:51:59.688 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.11106899000000001, 2020-09-08T15:51:59.688 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:51:59.689 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:59.689 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:51:59.689 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35381", 2020-09-08T15:51:59.689 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:59.689 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 199655 2020-09-08T15:51:59.690 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.690 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:51:59.690 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.690 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:51:59.690 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.467870+0000", 2020-09-08T15:51:59.690 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:51:59.691 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.691 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.691 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:51:59.691 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.467870+0000", 2020-09-08T15:51:59.691 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999714 2020-09-08T15:51:59.692 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.692 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.692 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:51:59.692 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.467842+0000", 2020-09-08T15:51:59.692 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4417000000000003e-05 2020-09-08T15:51:59.693 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.693 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.693 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:51:59.693 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.467906+0000", 2020-09-08T15:51:59.693 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1732999999999999e-05 2020-09-08T15:51:59.694 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.694 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.694 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:51:59.694 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.467918+0000", 2020-09-08T15:51:59.694 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6669999999999999e-06 2020-09-08T15:51:59.695 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.695 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.696 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:51:59.696 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.467925+0000", 2020-09-08T15:51:59.696 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5896000000000002e-05 2020-09-08T15:51:59.696 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.696 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.697 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:51:59.697 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.467970+0000", 2020-09-08T15:51:59.697 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011143 2020-09-08T15:51:59.697 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.697 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.698 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:51:59.698 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.468082+0000", 2020-09-08T15:51:59.698 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7579999999999997e-05 2020-09-08T15:51:59.698 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.698 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.699 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:51:59.699 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.468179+0000", 2020-09-08T15:51:59.699 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0048561890000000003 2020-09-08T15:51:59.699 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.699 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.700 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:51:59.700 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.473036+0000", 2020-09-08T15:51:59.700 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.10577731 2020-09-08T15:51:59.700 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.700 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.701 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:59.701 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.578813+0000", 2020-09-08T15:51:59.701 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0259999999999995e-05 2020-09-08T15:51:59.701 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.701 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.701 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:51:59.702 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.578883+0000", 2020-09-08T15:51:59.702 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5881999999999998e-05 2020-09-08T15:51:59.702 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.702 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.702 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:51:59.703 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.578939+0000", 2020-09-08T15:51:59.703 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.11106899000000001 2020-09-08T15:51:59.703 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.703 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.703 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.704 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.704 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.704 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35381.0:199656 4.b 4:d89f3293:::benchmark_data_smithi166_32054_object57514:head [delete] snapc 0=[] ondisk+write+known_if_redirected e365)", 2020-09-08T15:51:59.704 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:49:51.478088+0000", 2020-09-08T15:51:59.704 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 128.03283971299999, 2020-09-08T15:51:59.705 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.101171689, 2020-09-08T15:51:59.705 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:51:59.705 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:59.705 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:51:59.705 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35381", 2020-09-08T15:51:59.706 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:59.706 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 199656 2020-09-08T15:51:59.706 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.706 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:51:59.706 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.707 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:51:59.707 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.478088+0000", 2020-09-08T15:51:59.707 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:51:59.707 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.707 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.708 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:51:59.708 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.478088+0000", 2020-09-08T15:51:59.708 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999723 2020-09-08T15:51:59.708 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.708 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.709 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:51:59.709 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.478061+0000", 2020-09-08T15:51:59.709 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9226000000000003e-05 2020-09-08T15:51:59.709 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.709 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.710 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:51:59.710 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.478140+0000", 2020-09-08T15:51:59.710 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1224e-05 2020-09-08T15:51:59.710 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.710 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.711 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:51:59.711 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.478151+0000", 2020-09-08T15:51:59.711 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0370000000000002e-06 2020-09-08T15:51:59.711 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.711 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.712 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:51:59.712 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.478156+0000", 2020-09-08T15:51:59.712 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3414000000000001e-05 2020-09-08T15:51:59.712 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.712 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.713 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:51:59.713 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.478209+0000", 2020-09-08T15:51:59.713 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014257199999999999 2020-09-08T15:51:59.713 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.713 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.714 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:51:59.714 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.478352+0000", 2020-09-08T15:51:59.714 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000139615 2020-09-08T15:51:59.714 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.714 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.715 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:51:59.715 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.478492+0000", 2020-09-08T15:51:59.715 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0036868140000000001 2020-09-08T15:51:59.715 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.715 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.716 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:51:59.716 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.482178+0000", 2020-09-08T15:51:59.716 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.096874507999999998 2020-09-08T15:51:59.716 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.716 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.716 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:59.717 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.579053+0000", 2020-09-08T15:51:59.717 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014948699999999999 2020-09-08T15:51:59.717 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.717 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.717 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:51:59.718 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.579202+0000", 2020-09-08T15:51:59.718 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7488999999999998e-05 2020-09-08T15:51:59.718 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.718 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.718 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:51:59.719 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:49:51.579260+0000", 2020-09-08T15:51:59.719 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.101171689 2020-09-08T15:51:59.719 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.719 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.719 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.720 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.720 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.720 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35381.0:279116 4.b 4:de9b6bad:::benchmark_data_smithi166_32054_object136974:head [delete] snapc 0=[] ondisk+write+known_if_redirected e388)", 2020-09-08T15:51:59.720 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:51:11.772796+0000", 2020-09-08T15:51:59.721 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 47.738131537000001, 2020-09-08T15:51:59.721 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.081676759000000002, 2020-09-08T15:51:59.721 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:51:59.721 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:59.722 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:51:59.722 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35381", 2020-09-08T15:51:59.722 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:59.723 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 279116 2020-09-08T15:51:59.723 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.723 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:51:59.724 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.724 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:51:59.724 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:11.772796+0000", 2020-09-08T15:51:59.724 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:51:59.724 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.725 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.725 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:51:59.725 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:11.772796+0000", 2020-09-08T15:51:59.725 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999766 2020-09-08T15:51:59.725 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.726 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.726 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:51:59.726 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:11.772773+0000", 2020-09-08T15:51:59.726 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2385000000000001e-05 2020-09-08T15:51:59.726 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.727 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.727 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:51:59.727 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:11.772825+0000", 2020-09-08T15:51:59.727 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9420000000000007e-06 2020-09-08T15:51:59.727 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.728 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.728 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:51:59.729 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:11.772835+0000", 2020-09-08T15:51:59.729 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.755e-06 2020-09-08T15:51:59.729 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.729 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.730 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:51:59.730 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:11.772841+0000", 2020-09-08T15:51:59.730 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0992999999999998e-05 2020-09-08T15:51:59.731 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.731 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.731 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:51:59.731 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:11.772892+0000", 2020-09-08T15:51:59.732 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7763999999999994e-05 2020-09-08T15:51:59.732 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.732 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.732 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:51:59.732 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:11.772990+0000", 2020-09-08T15:51:59.733 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6626999999999996e-05 2020-09-08T15:51:59.734 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.735 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.735 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:51:59.735 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:11.773076+0000", 2020-09-08T15:51:59.735 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0044705070000000003 2020-09-08T15:51:59.735 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.736 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.736 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:51:59.736 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:11.777547+0000", 2020-09-08T15:51:59.736 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.076762338999999999 2020-09-08T15:51:59.736 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.737 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.737 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:59.737 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:11.854309+0000", 2020-09-08T15:51:59.737 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2665999999999997e-05 2020-09-08T15:51:59.737 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.738 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.738 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:51:59.738 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:11.854402+0000", 2020-09-08T15:51:59.738 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1282e-05 2020-09-08T15:51:59.739 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.739 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.739 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:51:59.739 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:11.854473+0000", 2020-09-08T15:51:59.739 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.081676759000000002 2020-09-08T15:51:59.740 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.740 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.740 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.740 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.740 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.740 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35381.0:279792 4.b 4:d2ef46a1:::benchmark_data_smithi166_32054_object137650:head [delete] snapc 0=[] ondisk+write+known_if_redirected e389)", 2020-09-08T15:51:59.741 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:51:12.517922+0000", 2020-09-08T15:51:59.741 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 46.993005594000003, 2020-09-08T15:51:59.741 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.100388088, 2020-09-08T15:51:59.741 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:51:59.741 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:59.742 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:51:59.742 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35381", 2020-09-08T15:51:59.742 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:59.742 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 279792 2020-09-08T15:51:59.742 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.743 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:51:59.743 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.743 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:51:59.743 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.517922+0000", 2020-09-08T15:51:59.744 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:51:59.744 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.744 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.745 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:51:59.745 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.517922+0000", 2020-09-08T15:51:59.745 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.99997 2020-09-08T15:51:59.745 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.746 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.746 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:51:59.746 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.517892+0000", 2020-09-08T15:51:59.746 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6872000000000006e-05 2020-09-08T15:51:59.747 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.747 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.747 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:51:59.747 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.517959+0000", 2020-09-08T15:51:59.747 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1664999999999999e-05 2020-09-08T15:51:59.748 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.748 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.748 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:51:59.748 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.517971+0000", 2020-09-08T15:51:59.748 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7239999999999997e-06 2020-09-08T15:51:59.748 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.749 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.749 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:51:59.749 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.517976+0000", 2020-09-08T15:51:59.749 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2222999999999998e-05 2020-09-08T15:51:59.749 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.750 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.750 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:51:59.750 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.518029+0000", 2020-09-08T15:51:59.750 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013107099999999999 2020-09-08T15:51:59.750 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.751 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.751 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:51:59.751 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.518160+0000", 2020-09-08T15:51:59.752 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000129756 2020-09-08T15:51:59.752 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.752 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.753 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:51:59.753 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.518289+0000", 2020-09-08T15:51:59.753 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0079857299999999999 2020-09-08T15:51:59.753 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.753 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.754 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:51:59.754 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.526275+0000", 2020-09-08T15:51:59.754 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.091911903000000003 2020-09-08T15:51:59.754 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.755 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.755 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:59.755 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.618187+0000", 2020-09-08T15:51:59.755 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4869000000000004e-05 2020-09-08T15:51:59.755 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.756 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.756 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:51:59.756 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.618262+0000", 2020-09-08T15:51:59.756 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8491e-05 2020-09-08T15:51:59.757 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.757 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.757 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:51:59.757 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.618310+0000", 2020-09-08T15:51:59.758 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.100388088 2020-09-08T15:51:59.758 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.758 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.758 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.758 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.759 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.759 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35381.0:279796 4.b 4:d517b66d:::benchmark_data_smithi166_32054_object137654:head [delete] snapc 0=[] ondisk+write+known_if_redirected e389)", 2020-09-08T15:51:59.759 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:51:12.521241+0000", 2020-09-08T15:51:59.759 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 46.989686796000001, 2020-09-08T15:51:59.759 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.10000893800000001, 2020-09-08T15:51:59.760 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:51:59.760 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:59.760 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:51:59.761 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35381", 2020-09-08T15:51:59.761 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:59.761 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 279796 2020-09-08T15:51:59.761 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.761 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:51:59.762 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.762 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:51:59.762 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.521241+0000", 2020-09-08T15:51:59.762 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:51:59.762 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.763 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.763 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:51:59.763 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.521241+0000", 2020-09-08T15:51:59.763 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999766 2020-09-08T15:51:59.763 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.764 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.764 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:51:59.764 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.521218+0000", 2020-09-08T15:51:59.764 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2357e-05 2020-09-08T15:51:59.764 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.765 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.765 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:51:59.765 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.521270+0000", 2020-09-08T15:51:59.765 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7620000000000007e-06 2020-09-08T15:51:59.765 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.766 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.766 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:51:59.766 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.521280+0000", 2020-09-08T15:51:59.766 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0320000000000003e-06 2020-09-08T15:51:59.767 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.767 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.767 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:51:59.768 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.521285+0000", 2020-09-08T15:51:59.768 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001630739 2020-09-08T15:51:59.768 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.768 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.768 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:51:59.769 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.522915+0000", 2020-09-08T15:51:59.769 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0242999999999996e-05 2020-09-08T15:51:59.769 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.769 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.769 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:51:59.770 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.523006+0000", 2020-09-08T15:51:59.770 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5518000000000003e-05 2020-09-08T15:51:59.770 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.770 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.771 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:51:59.771 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.523101+0000", 2020-09-08T15:51:59.771 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0033159999999999999 2020-09-08T15:51:59.771 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.771 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.772 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:51:59.772 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.526417+0000", 2020-09-08T15:51:59.772 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.094717115000000004 2020-09-08T15:51:59.772 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.772 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.773 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:59.773 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.621134+0000", 2020-09-08T15:51:59.773 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7706000000000005e-05 2020-09-08T15:51:59.773 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.773 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.774 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:51:59.774 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.621202+0000", 2020-09-08T15:51:59.774 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8019999999999998e-05 2020-09-08T15:51:59.774 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.774 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.775 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:51:59.775 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.621250+0000", 2020-09-08T15:51:59.775 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.10000893800000001 2020-09-08T15:51:59.775 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.775 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.775 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.776 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.776 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.776 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35381.0:279812 4.b 4:daf1a9ec:::benchmark_data_smithi166_32054_object137670:head [delete] snapc 0=[] ondisk+write+known_if_redirected e389)", 2020-09-08T15:51:59.776 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:51:12.545389+0000", 2020-09-08T15:51:59.777 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 46.965539303, 2020-09-08T15:51:59.777 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.091713095999999994, 2020-09-08T15:51:59.777 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:51:59.777 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:59.778 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:51:59.778 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35381", 2020-09-08T15:51:59.778 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:59.779 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 279812 2020-09-08T15:51:59.779 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.779 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:51:59.779 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.779 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:51:59.779 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.545389+0000", 2020-09-08T15:51:59.780 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:51:59.780 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.780 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.780 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:51:59.780 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.545389+0000", 2020-09-08T15:51:59.781 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999404 2020-09-08T15:51:59.781 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.781 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.781 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:51:59.781 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.545329+0000", 2020-09-08T15:51:59.782 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000133132 2020-09-08T15:51:59.782 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.782 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.782 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:51:59.782 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.545462+0000", 2020-09-08T15:51:59.783 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3842999999999999e-05 2020-09-08T15:51:59.783 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.783 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.783 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:51:59.784 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.545486+0000", 2020-09-08T15:51:59.784 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1883e-05 2020-09-08T15:51:59.784 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.784 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.784 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:51:59.785 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.545498+0000", 2020-09-08T15:51:59.785 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000110553 2020-09-08T15:51:59.785 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.785 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.785 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:51:59.786 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.545608+0000", 2020-09-08T15:51:59.786 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000218474 2020-09-08T15:51:59.786 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.786 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.786 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:51:59.787 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.545827+0000", 2020-09-08T15:51:59.787 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000219411 2020-09-08T15:51:59.788 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.788 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.788 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:51:59.789 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.546046+0000", 2020-09-08T15:51:59.789 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0064051070000000002 2020-09-08T15:51:59.789 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.789 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.789 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:51:59.790 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.552451+0000", 2020-09-08T15:51:59.790 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.084437404999999993 2020-09-08T15:51:59.790 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.791 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.791 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:59.791 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.636889+0000", 2020-09-08T15:51:59.791 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000162907 2020-09-08T15:51:59.791 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.791 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.792 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:51:59.792 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.637052+0000", 2020-09-08T15:51:59.792 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9997000000000003e-05 2020-09-08T15:51:59.793 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.794 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.794 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:51:59.795 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:12.637102+0000", 2020-09-08T15:51:59.795 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.091713095999999994 2020-09-08T15:51:59.795 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.795 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.795 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.796 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.796 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.796 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.35381.0:283035 4.b 4:dfd35794:::benchmark_data_smithi166_32054_object140893:head [delete] snapc 0=[] ondisk+write+known_if_redirected e390)", 2020-09-08T15:51:59.796 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:51:15.889326+0000", 2020-09-08T15:51:59.796 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 43.621602404999997, 2020-09-08T15:51:59.797 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.090443296000000006, 2020-09-08T15:51:59.797 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:51:59.797 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:59.797 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:51:59.797 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.35381", 2020-09-08T15:51:59.798 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:51:59.798 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 283035 2020-09-08T15:51:59.798 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.798 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:51:59.798 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.799 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:51:59.799 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:15.889326+0000", 2020-09-08T15:51:59.799 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:51:59.799 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.799 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.800 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:51:59.800 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:15.889326+0000", 2020-09-08T15:51:59.800 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999771 2020-09-08T15:51:59.800 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.800 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.801 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:51:59.801 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:15.889302+0000", 2020-09-08T15:51:59.801 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2407000000000002e-05 2020-09-08T15:51:59.801 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.801 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.802 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:51:59.802 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:15.889355+0000", 2020-09-08T15:51:59.802 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0626e-05 2020-09-08T15:51:59.802 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.802 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.802 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:51:59.803 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:15.889365+0000", 2020-09-08T15:51:59.803 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5980000000000002e-06 2020-09-08T15:51:59.803 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.803 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.803 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:51:59.804 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:15.889372+0000", 2020-09-08T15:51:59.804 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1274999999999999e-05 2020-09-08T15:51:59.804 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.804 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.805 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:51:59.805 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:15.889433+0000", 2020-09-08T15:51:59.806 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4503e-05 2020-09-08T15:51:59.806 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.806 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.807 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:51:59.807 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:15.889528+0000", 2020-09-08T15:51:59.807 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7174000000000003e-05 2020-09-08T15:51:59.808 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.808 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.808 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:51:59.808 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:15.889615+0000", 2020-09-08T15:51:59.808 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0042291630000000002 2020-09-08T15:51:59.809 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.809 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.809 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:51:59.809 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:15.893844+0000", 2020-09-08T15:51:59.809 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.085795544000000001 2020-09-08T15:51:59.810 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.810 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.810 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:59.810 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:15.979640+0000", 2020-09-08T15:51:59.810 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1469000000000003e-05 2020-09-08T15:51:59.811 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.811 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.811 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:51:59.811 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:15.979711+0000", 2020-09-08T15:51:59.811 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7620000000000001e-05 2020-09-08T15:51:59.812 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.812 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.812 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:51:59.812 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:15.979769+0000", 2020-09-08T15:51:59.812 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.090443296000000006 2020-09-08T15:51:59.812 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.813 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.813 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.813 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.813 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.814 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:7701 5.5 5:a08655ba:::benchmark_data_smithi166_37820_object7700:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e404)", 2020-09-08T15:51:59.814 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:51:55.356696+0000", 2020-09-08T15:51:59.814 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 4.1542319220000001, 2020-09-08T15:51:59.814 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.431624061, 2020-09-08T15:51:59.814 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:51:59.815 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:59.815 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:51:59.815 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T15:51:59.815 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:51:59.815 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 7701 2020-09-08T15:51:59.815 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.816 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:51:59.816 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.816 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:51:59.816 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.356696+0000", 2020-09-08T15:51:59.816 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:51:59.817 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.817 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.817 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:51:59.817 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.356696+0000", 2020-09-08T15:51:59.817 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999542 2020-09-08T15:51:59.818 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.818 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.818 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:51:59.818 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.356650+0000", 2020-09-08T15:51:59.818 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014592500000000001 2020-09-08T15:51:59.819 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.819 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.819 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:51:59.819 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.356796+0000", 2020-09-08T15:51:59.819 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1799e-05 2020-09-08T15:51:59.820 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.820 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.820 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:51:59.820 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.356808+0000", 2020-09-08T15:51:59.820 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.172e-06 2020-09-08T15:51:59.820 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.821 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.821 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:51:59.821 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.356812+0000", 2020-09-08T15:51:59.821 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8897000000000002e-05 2020-09-08T15:51:59.821 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.822 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.822 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:51:59.822 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.356871+0000", 2020-09-08T15:51:59.822 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00027441800000000001 2020-09-08T15:51:59.822 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.823 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.823 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:51:59.823 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.357146+0000", 2020-09-08T15:51:59.823 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000244367 2020-09-08T15:51:59.823 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.824 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.824 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:51:59.824 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.357390+0000", 2020-09-08T15:51:59.824 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001263651 2020-09-08T15:51:59.824 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.825 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.825 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:51:59.825 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.358654+0000", 2020-09-08T15:51:59.825 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42947748200000002 2020-09-08T15:51:59.825 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.826 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.826 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:59.826 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.788131+0000", 2020-09-08T15:51:59.826 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011090999999999999 2020-09-08T15:51:59.826 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.826 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.827 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:51:59.827 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.788242+0000", 2020-09-08T15:51:59.827 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.8153999999999997e-05 2020-09-08T15:51:59.827 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.827 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.828 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:51:59.828 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.788320+0000", 2020-09-08T15:51:59.828 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.431624061 2020-09-08T15:51:59.828 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.828 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.829 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.829 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.829 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.829 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:7727 5.5 5:aec88afa:::benchmark_data_smithi166_37820_object7726:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e404)", 2020-09-08T15:51:59.829 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:51:55.363484+0000", 2020-09-08T15:51:59.830 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 4.1474437340000003, 2020-09-08T15:51:59.830 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42549411500000001, 2020-09-08T15:51:59.830 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:51:59.830 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:59.830 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:51:59.831 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T15:51:59.831 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:51:59.831 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 7727 2020-09-08T15:51:59.831 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.831 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:51:59.831 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.832 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:51:59.832 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.363484+0000", 2020-09-08T15:51:59.832 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:51:59.832 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.832 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.833 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:51:59.833 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.363484+0000", 2020-09-08T15:51:59.833 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999595 2020-09-08T15:51:59.833 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.833 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.834 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:51:59.834 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.363444+0000", 2020-09-08T15:51:59.834 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000103169 2020-09-08T15:51:59.834 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.834 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.835 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:51:59.835 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.363547+0000", 2020-09-08T15:51:59.835 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0091e-05 2020-09-08T15:51:59.835 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.835 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.836 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:51:59.836 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.363557+0000", 2020-09-08T15:51:59.836 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8130000000000003e-06 2020-09-08T15:51:59.836 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.836 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.836 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:51:59.837 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.363562+0000", 2020-09-08T15:51:59.837 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2843999999999998e-05 2020-09-08T15:51:59.837 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.837 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.837 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:51:59.838 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.363615+0000", 2020-09-08T15:51:59.838 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000217252 2020-09-08T15:51:59.838 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.838 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.838 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:51:59.839 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.363832+0000", 2020-09-08T15:51:59.839 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016350199999999999 2020-09-08T15:51:59.839 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.839 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.839 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:51:59.840 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.363995+0000", 2020-09-08T15:51:59.840 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0012898149999999999 2020-09-08T15:51:59.840 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.840 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.840 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:51:59.841 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.365285+0000", 2020-09-08T15:51:59.841 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42340125499999998 2020-09-08T15:51:59.841 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.841 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.841 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:59.841 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.788686+0000", 2020-09-08T15:51:59.842 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018330199999999999 2020-09-08T15:51:59.842 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.842 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.842 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:51:59.842 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.788870+0000", 2020-09-08T15:51:59.843 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010857900000000001 2020-09-08T15:51:59.843 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.843 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.843 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:51:59.843 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.788978+0000", 2020-09-08T15:51:59.844 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42549411500000001 2020-09-08T15:51:59.844 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.844 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.844 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.844 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.845 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.845 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:7729 5.5 5:abebecba:::benchmark_data_smithi166_37820_object7728:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e404)", 2020-09-08T15:51:59.845 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:51:55.364104+0000", 2020-09-08T15:51:59.845 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 4.1468242350000004, 2020-09-08T15:51:59.845 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42546982999999999, 2020-09-08T15:51:59.846 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:51:59.846 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:59.846 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:51:59.846 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T15:51:59.846 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:51:59.847 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 7729 2020-09-08T15:51:59.847 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.847 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:51:59.847 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.847 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:51:59.847 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.364104+0000", 2020-09-08T15:51:59.848 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:51:59.848 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.848 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.848 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:51:59.848 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.364104+0000", 2020-09-08T15:51:59.849 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999328 2020-09-08T15:51:59.849 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.849 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.849 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:51:59.849 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.364036+0000", 2020-09-08T15:51:59.850 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000169409 2020-09-08T15:51:59.850 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.850 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.850 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:51:59.850 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.364206+0000", 2020-09-08T15:51:59.851 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4986000000000001e-05 2020-09-08T15:51:59.851 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.851 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.851 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:51:59.851 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.364231+0000", 2020-09-08T15:51:59.851 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1440000000000003e-06 2020-09-08T15:51:59.852 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.852 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.852 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:51:59.852 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.364239+0000", 2020-09-08T15:51:59.852 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019167500000000001 2020-09-08T15:51:59.853 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.853 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.853 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:51:59.853 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.364430+0000", 2020-09-08T15:51:59.853 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000203508 2020-09-08T15:51:59.854 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.854 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.854 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:51:59.854 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.364634+0000", 2020-09-08T15:51:59.854 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00023597099999999999 2020-09-08T15:51:59.855 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.855 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.855 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:51:59.855 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.364870+0000", 2020-09-08T15:51:59.855 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00056251999999999995 2020-09-08T15:51:59.856 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.856 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.856 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:51:59.856 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.365432+0000", 2020-09-08T15:51:59.856 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42375369400000001 2020-09-08T15:51:59.857 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.857 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.857 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:59.857 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.789186+0000", 2020-09-08T15:51:59.857 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019180799999999999 2020-09-08T15:51:59.857 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.858 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.858 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:51:59.858 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.789378+0000", 2020-09-08T15:51:59.858 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019555199999999999 2020-09-08T15:51:59.858 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.859 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.859 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:51:59.859 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.789574+0000", 2020-09-08T15:51:59.859 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42546982999999999 2020-09-08T15:51:59.859 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.860 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.860 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.860 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.860 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.860 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:7742 5.5 5:a56e23fa:::benchmark_data_smithi166_37820_object7741:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e404)", 2020-09-08T15:51:59.861 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:51:55.373940+0000", 2020-09-08T15:51:59.861 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 4.1369881719999997, 2020-09-08T15:51:59.861 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.416078489, 2020-09-08T15:51:59.861 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:51:59.861 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:51:59.862 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:51:59.862 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T15:51:59.862 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:51:59.862 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 7742 2020-09-08T15:51:59.862 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.863 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:51:59.863 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.863 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:51:59.863 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.373940+0000", 2020-09-08T15:51:59.863 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:51:59.863 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.864 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.864 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:51:59.864 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.373940+0000", 2020-09-08T15:51:59.864 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999695 2020-09-08T15:51:59.864 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.865 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.865 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:51:59.865 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.373909+0000", 2020-09-08T15:51:59.865 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000103714 2020-09-08T15:51:59.865 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.866 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.866 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:51:59.866 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.374013+0000", 2020-09-08T15:51:59.866 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3936e-05 2020-09-08T15:51:59.866 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.867 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.867 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:51:59.867 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.374027+0000", 2020-09-08T15:51:59.867 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7360000000000001e-06 2020-09-08T15:51:59.867 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.868 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.868 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:51:59.868 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.374035+0000", 2020-09-08T15:51:59.868 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6749999999999997e-05 2020-09-08T15:51:59.868 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.868 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.869 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:51:59.869 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.374091+0000", 2020-09-08T15:51:59.869 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019101599999999999 2020-09-08T15:51:59.869 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.869 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.870 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:51:59.875 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.374282+0000", 2020-09-08T15:51:59.876 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00023996299999999999 2020-09-08T15:51:59.876 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.876 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.876 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T15:51:59.876 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.374522+0000", 2020-09-08T15:51:59.877 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00058567999999999999 2020-09-08T15:51:59.877 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.877 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.877 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:51:59.877 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.375108+0000", 2020-09-08T15:51:59.878 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.41459593300000003 2020-09-08T15:51:59.878 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.878 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.878 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:51:59.878 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.789704+0000", 2020-09-08T15:51:59.879 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000181428 2020-09-08T15:51:59.879 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.879 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.879 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:51:59.879 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.789885+0000", 2020-09-08T15:51:59.880 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013284200000000001 2020-09-08T15:51:59.880 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:51:59.880 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:51:59.880 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:51:59.880 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.790018+0000", 2020-09-08T15:51:59.881 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.416078489 2020-09-08T15:51:59.881 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.881 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.881 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.881 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:51:59.882 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:51:59.882 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:51:59.883 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:51:59.985 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:52:00.117 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:51:59 smithi166 bash[11928]: debug 2020-09-08T15:51:59.798+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:00.359 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 7 16 22206 22190 198.108 140.812 0.00283056 0.00504473 2020-09-08T15:52:00.791 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:52:01.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:00 smithi166 bash[9446]: cluster 2020-09-08T15:51:59.775631+0000 mon.a (mon.0) 1538 : cluster [DBG] osdmap e405: 12 total, 12 up, 12 in 2020-09-08T15:52:01.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:00 smithi166 bash[9446]: cluster 2020-09-08T15:51:59.819054+0000 mgr.x (mgr.24118) 740 : cluster [DBG] pgmap v1061: 17 pgs: 17 active+clean; 858 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 109 MiB/s wr, 1.74k op/s 2020-09-08T15:52:01.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:00 smithi166 bash[9446]: cluster 2020-09-08T15:52:00.039178+0000 osd.9 (osd.9) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:01.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:00 smithi166 bash[9446]: cluster 2020-09-08T15:52:00.039277+0000 osd.9 (osd.9) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:01.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:00 smithi166 bash[9446]: audit 2020-09-08T15:52:00.128863+0000 mon.b (mon.1) 97 : audit [INF] from='client.? v1:172.21.15.104:0/2469407841' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:52:01.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:00 smithi166 bash[9446]: audit 2020-09-08T15:52:00.129542+0000 mon.a (mon.0) 1539 : audit [INF] from='client.25660 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:52:01.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:00 smithi055 bash[9296]: cluster 2020-09-08T15:51:59.775631+0000 mon.a (mon.0) 1538 : cluster [DBG] osdmap e405: 12 total, 12 up, 12 in 2020-09-08T15:52:01.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:00 smithi055 bash[9296]: cluster 2020-09-08T15:51:59.819054+0000 mgr.x (mgr.24118) 740 : cluster [DBG] pgmap v1061: 17 pgs: 17 active+clean; 858 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 109 MiB/s wr, 1.74k op/s 2020-09-08T15:52:01.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:00 smithi055 bash[9296]: cluster 2020-09-08T15:52:00.039178+0000 osd.9 (osd.9) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:01.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:00 smithi055 bash[9296]: cluster 2020-09-08T15:52:00.039277+0000 osd.9 (osd.9) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:01.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:00 smithi055 bash[9296]: audit 2020-09-08T15:52:00.128863+0000 mon.b (mon.1) 97 : audit [INF] from='client.? v1:172.21.15.104:0/2469407841' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:52:01.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:00 smithi055 bash[9296]: audit 2020-09-08T15:52:00.129542+0000 mon.a (mon.0) 1539 : audit [INF] from='client.25660 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:52:01.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:00 smithi104 bash[9203]: cluster 2020-09-08T15:51:59.775631+0000 mon.a (mon.0) 1538 : cluster [DBG] osdmap e405: 12 total, 12 up, 12 in 2020-09-08T15:52:01.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:00 smithi104 bash[9203]: cluster 2020-09-08T15:51:59.819054+0000 mgr.x (mgr.24118) 740 : cluster [DBG] pgmap v1061: 17 pgs: 17 active+clean; 858 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 109 MiB/s wr, 1.74k op/s 2020-09-08T15:52:01.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:00 smithi104 bash[9203]: cluster 2020-09-08T15:52:00.039178+0000 osd.9 (osd.9) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:01.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:00 smithi104 bash[9203]: cluster 2020-09-08T15:52:00.039277+0000 osd.9 (osd.9) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:01.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:00 smithi104 bash[9203]: audit 2020-09-08T15:52:00.128863+0000 mon.b (mon.1) 97 : audit [INF] from='client.? v1:172.21.15.104:0/2469407841' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:52:01.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:00 smithi104 bash[9203]: audit 2020-09-08T15:52:00.129542+0000 mon.a (mon.0) 1539 : audit [INF] from='client.25660 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:52:01.359 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 8 16 25263 25247 197.225 191.062 0.00913096 0.00506612 2020-09-08T15:52:02.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:01 smithi166 bash[9446]: cluster 2020-09-08T15:52:00.779780+0000 mon.a (mon.0) 1540 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:52:02.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:01 smithi166 bash[9446]: audit 2020-09-08T15:52:00.791346+0000 mon.a (mon.0) 1541 : audit [INF] from='client.25660 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:52:02.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:01 smithi166 bash[9446]: cluster 2020-09-08T15:52:00.791379+0000 mon.a (mon.0) 1542 : cluster [DBG] osdmap e406: 12 total, 12 up, 12 in 2020-09-08T15:52:02.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:01 smithi055 bash[9296]: cluster 2020-09-08T15:52:00.779780+0000 mon.a (mon.0) 1540 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:52:02.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:01 smithi055 bash[9296]: audit 2020-09-08T15:52:00.791346+0000 mon.a (mon.0) 1541 : audit [INF] from='client.25660 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:52:02.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:01 smithi055 bash[9296]: cluster 2020-09-08T15:52:00.791379+0000 mon.a (mon.0) 1542 : cluster [DBG] osdmap e406: 12 total, 12 up, 12 in 2020-09-08T15:52:02.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:01 smithi104 bash[9203]: cluster 2020-09-08T15:52:00.779780+0000 mon.a (mon.0) 1540 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:52:02.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:01 smithi104 bash[9203]: audit 2020-09-08T15:52:00.791346+0000 mon.a (mon.0) 1541 : audit [INF] from='client.25660 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:52:02.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:01 smithi104 bash[9203]: cluster 2020-09-08T15:52:00.791379+0000 mon.a (mon.0) 1542 : cluster [DBG] osdmap e406: 12 total, 12 up, 12 in 2020-09-08T15:52:02.290 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:52:02.362 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 9 16 27289 27273 189.379 126.625 0.00854102 0.00527667 2020-09-08T15:52:02.394 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:52:02.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:52:02 smithi104 bash[15273]: debug 2020-09-08T15:52:02.290+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:02.787 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:02 smithi166 bash[9446]: cluster 2020-09-08T15:52:01.819491+0000 mgr.x (mgr.24118) 741 : cluster [DBG] pgmap v1063: 17 pgs: 17 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 145 MiB/s wr, 2.32k op/s 2020-09-08T15:52:02.788 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:02 smithi166 bash[9446]: cluster 2020-09-08T15:52:02.135261+0000 osd.2 (osd.2) 52 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:02.788 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:02 smithi166 bash[9446]: cluster 2020-09-08T15:52:02.135345+0000 osd.2 (osd.2) 53 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:02.833 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:52:02.936 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:52:03.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:52:02 smithi166 bash[11928]: debug 2020-09-08T15:52:02.830+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:03.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:02 smithi055 bash[9296]: cluster 2020-09-08T15:52:01.819491+0000 mgr.x (mgr.24118) 741 : cluster [DBG] pgmap v1063: 17 pgs: 17 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 145 MiB/s wr, 2.32k op/s 2020-09-08T15:52:03.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:02 smithi055 bash[9296]: cluster 2020-09-08T15:52:02.135261+0000 osd.2 (osd.2) 52 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:03.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:02 smithi055 bash[9296]: cluster 2020-09-08T15:52:02.135345+0000 osd.2 (osd.2) 53 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:03.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:02 smithi104 bash[9203]: cluster 2020-09-08T15:52:01.819491+0000 mgr.x (mgr.24118) 741 : cluster [DBG] pgmap v1063: 17 pgs: 17 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 145 MiB/s wr, 2.32k op/s 2020-09-08T15:52:03.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:02 smithi104 bash[9203]: cluster 2020-09-08T15:52:02.135261+0000 osd.2 (osd.2) 52 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:03.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:02 smithi104 bash[9203]: cluster 2020-09-08T15:52:02.135345+0000 osd.2 (osd.2) 53 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:03.362 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 10 16 30164 30148 188.409 179.688 0.00415599 0.0053041 2020-09-08T15:52:03.743 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2020-09-08T15:52:03.865 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:52:03.970 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:04.060 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:52:04.116 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:03 smithi055 bash[9296]: cluster 2020-09-08T15:52:02.793931+0000 mon.a (mon.0) 1543 : cluster [DBG] osdmap e407: 12 total, 12 up, 12 in 2020-09-08T15:52:04.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:03 smithi166 bash[9446]: cluster 2020-09-08T15:52:02.793931+0000 mon.a (mon.0) 1543 : cluster [DBG] osdmap e407: 12 total, 12 up, 12 in 2020-09-08T15:52:04.117 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:04.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:03 smithi104 bash[9203]: cluster 2020-09-08T15:52:02.793931+0000 mon.a (mon.0) 1543 : cluster [DBG] osdmap e407: 12 total, 12 up, 12 in 2020-09-08T15:52:04.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:52:03 smithi104 bash[15273]: debug 2020-09-08T15:52:03.865+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:04.232 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:52:04.362 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 11 16 32287 32271 183.342 132.688 0.00361348 0.0054508 2020-09-08T15:52:04.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:52:04 smithi055 bash[11956]: debug 2020-09-08T15:52:04.117+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:05.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:04 smithi166 bash[9446]: cluster 2020-09-08T15:52:03.819905+0000 mgr.x (mgr.24118) 742 : cluster [DBG] pgmap v1065: 17 pgs: 17 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 191 MiB/s wr, 3.06k op/s 2020-09-08T15:52:05.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:04 smithi166 bash[9446]: cluster 2020-09-08T15:52:04.777423+0000 mon.a (mon.0) 1544 : cluster [DBG] osdmap e408: 12 total, 12 up, 12 in 2020-09-08T15:52:05.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:04 smithi055 bash[9296]: cluster 2020-09-08T15:52:03.819905+0000 mgr.x (mgr.24118) 742 : cluster [DBG] pgmap v1065: 17 pgs: 17 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 191 MiB/s wr, 3.06k op/s 2020-09-08T15:52:05.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:04 smithi055 bash[9296]: cluster 2020-09-08T15:52:04.777423+0000 mon.a (mon.0) 1544 : cluster [DBG] osdmap e408: 12 total, 12 up, 12 in 2020-09-08T15:52:05.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:04 smithi104 bash[9203]: cluster 2020-09-08T15:52:03.819905+0000 mgr.x (mgr.24118) 742 : cluster [DBG] pgmap v1065: 17 pgs: 17 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 191 MiB/s wr, 3.06k op/s 2020-09-08T15:52:05.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:04 smithi104 bash[9203]: cluster 2020-09-08T15:52:04.777423+0000 mon.a (mon.0) 1544 : cluster [DBG] osdmap e408: 12 total, 12 up, 12 in 2020-09-08T15:52:05.323 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:52:05.375 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 12 16 34653 34637 180.386 147.875 0.00358622 0.00554027 2020-09-08T15:52:05.428 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:52:05.583 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:52:05 smithi104 bash[22305]: debug 2020-09-08T15:52:05.321+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:05.584 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:52:05.689 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:05.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:52:05 smithi104 bash[15273]: debug 2020-09-08T15:52:05.585+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:06.360 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 13 16 37233 37217 178.913 161.25 0.00449228 0.00558657 2020-09-08T15:52:07.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:06 smithi055 bash[9296]: cluster 2020-09-08T15:52:05.820373+0000 mgr.x (mgr.24118) 743 : cluster [DBG] pgmap v1067: 17 pgs: 17 active+clean; 1.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 146 MiB/s wr, 2.33k op/s 2020-09-08T15:52:07.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:06 smithi104 bash[9203]: cluster 2020-09-08T15:52:05.820373+0000 mgr.x (mgr.24118) 743 : cluster [DBG] pgmap v1067: 17 pgs: 17 active+clean; 1.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 146 MiB/s wr, 2.33k op/s 2020-09-08T15:52:07.359 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 14 16 39713 39697 177.204 155 0.00759751 0.0056408 2020-09-08T15:52:07.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:06 smithi166 bash[9446]: cluster 2020-09-08T15:52:05.820373+0000 mgr.x (mgr.24118) 743 : cluster [DBG] pgmap v1067: 17 pgs: 17 active+clean; 1.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 146 MiB/s wr, 2.33k op/s 2020-09-08T15:52:07.489 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:52:07.528 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:52:07.564 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:52:07.566 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:52:07.611 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:52:07.653 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:52:07.704 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:52:07.745 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:52:07.757 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:52:07.783 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:52:07.819 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:52:07.857 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:52:07.894 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:52:07.927 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:52:07.929 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:52:07.930 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:52:07.930 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:52:07.930 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:52:07.930 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:52:07.930 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:52:07.931 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:52:07.931 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:52:07.931 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:52:07.931 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:52:07.931 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:52:07.932 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:52:08.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:07 smithi104 bash[9203]: audit 2020-09-08T15:52:07.449583+0000 mon.b (mon.1) 98 : audit [DBG] from='client.? v1:172.21.15.104:0/2308020056' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:52:08.220 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:07 smithi055 bash[9296]: audit 2020-09-08T15:52:07.449583+0000 mon.b (mon.1) 98 : audit [DBG] from='client.? v1:172.21.15.104:0/2308020056' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:52:08.360 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 15 16 41616 41600 173.318 118.938 0.00243212 0.00575853 2020-09-08T15:52:08.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:07 smithi166 bash[9446]: audit 2020-09-08T15:52:07.449583+0000 mon.b (mon.1) 98 : audit [DBG] from='client.? v1:172.21.15.104:0/2308020056' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:52:09.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:08 smithi055 bash[9296]: cluster 2020-09-08T15:52:07.384645+0000 osd.7 (osd.7) 75 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:52:09.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:08 smithi055 bash[9296]: cluster 2020-09-08T15:52:07.470731+0000 osd.7 (osd.7) 76 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:52:09.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:08 smithi055 bash[9296]: cluster 2020-09-08T15:52:07.820893+0000 mgr.x (mgr.24118) 744 : cluster [DBG] pgmap v1068: 17 pgs: 17 active+clean; 2.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 199 MiB/s wr, 3.19k op/s 2020-09-08T15:52:09.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:08 smithi104 bash[9203]: cluster 2020-09-08T15:52:07.384645+0000 osd.7 (osd.7) 75 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:52:09.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:08 smithi104 bash[9203]: cluster 2020-09-08T15:52:07.470731+0000 osd.7 (osd.7) 76 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:52:09.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:08 smithi104 bash[9203]: cluster 2020-09-08T15:52:07.820893+0000 mgr.x (mgr.24118) 744 : cluster [DBG] pgmap v1068: 17 pgs: 17 active+clean; 2.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 199 MiB/s wr, 3.19k op/s 2020-09-08T15:52:09.359 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 16 16 43105 43089 168.302 93.0625 0.00479487 0.00593869 2020-09-08T15:52:09.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:08 smithi166 bash[9446]: cluster 2020-09-08T15:52:07.384645+0000 osd.7 (osd.7) 75 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:52:09.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:08 smithi166 bash[9446]: cluster 2020-09-08T15:52:07.470731+0000 osd.7 (osd.7) 76 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:52:09.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:08 smithi166 bash[9446]: cluster 2020-09-08T15:52:07.820893+0000 mgr.x (mgr.24118) 744 : cluster [DBG] pgmap v1068: 17 pgs: 17 active+clean; 2.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 199 MiB/s wr, 3.19k op/s 2020-09-08T15:52:09.673 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:09.776 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:10.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:52:09 smithi055 bash[16646]: debug 2020-09-08T15:52:09.673+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:10.361 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 17 15 45126 45111 165.835 126.375 0.00742 0.00602753 2020-09-08T15:52:10.411 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:10.520 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:10.700 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:10.787 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:52:10 smithi055 bash[11956]: debug 2020-09-08T15:52:10.413+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:10.788 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:10 smithi055 bash[9296]: cluster 2020-09-08T15:52:09.821745+0000 mgr.x (mgr.24118) 745 : cluster [DBG] pgmap v1069: 17 pgs: 17 active+clean; 2.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 180 MiB/s wr, 2.89k op/s 2020-09-08T15:52:10.807 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:10.876 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:10 smithi166 bash[9446]: cluster 2020-09-08T15:52:09.821745+0000 mgr.x (mgr.24118) 745 : cluster [DBG] pgmap v1069: 17 pgs: 17 active+clean; 2.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 180 MiB/s wr, 2.89k op/s 2020-09-08T15:52:10.877 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:52:10 smithi166 bash[16593]: debug 2020-09-08T15:52:10.698+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:10.906 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:11.009 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:11.017 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:10 smithi104 bash[9203]: cluster 2020-09-08T15:52:09.821745+0000 mgr.x (mgr.24118) 745 : cluster [DBG] pgmap v1069: 17 pgs: 17 active+clean; 2.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 180 MiB/s wr, 2.89k op/s 2020-09-08T15:52:11.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:52:10 smithi055 bash[16646]: debug 2020-09-08T15:52:10.909+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:11.363 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 18 16 47315 47299 164.219 136.75 0.0058971 0.00608697 2020-09-08T15:52:11.710 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:11.815 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:52:11.970 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:52:12.075 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:52:12.083 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:11 smithi055 bash[9296]: cluster 2020-09-08T15:52:10.392968+0000 osd.7 (osd.7) 77 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:12.083 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:11 smithi055 bash[9296]: cluster 2020-09-08T15:52:10.393074+0000 osd.7 (osd.7) 78 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:12.084 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:11 smithi055 bash[9296]: cluster 2020-09-08T15:52:10.793906+0000 mon.a (mon.0) 1545 : cluster [DBG] osdmap e409: 12 total, 12 up, 12 in 2020-09-08T15:52:12.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:11 smithi166 bash[9446]: cluster 2020-09-08T15:52:10.392968+0000 osd.7 (osd.7) 77 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:12.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:11 smithi166 bash[9446]: cluster 2020-09-08T15:52:10.393074+0000 osd.7 (osd.7) 78 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:12.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:11 smithi166 bash[9446]: cluster 2020-09-08T15:52:10.793906+0000 mon.a (mon.0) 1545 : cluster [DBG] osdmap e409: 12 total, 12 up, 12 in 2020-09-08T15:52:12.117 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:52:11 smithi166 bash[14254]: debug 2020-09-08T15:52:11.970+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:12.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:11 smithi104 bash[9203]: cluster 2020-09-08T15:52:10.392968+0000 osd.7 (osd.7) 77 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:12.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:11 smithi104 bash[9203]: cluster 2020-09-08T15:52:10.393074+0000 osd.7 (osd.7) 78 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:12.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:11 smithi104 bash[9203]: cluster 2020-09-08T15:52:10.793906+0000 mon.a (mon.0) 1545 : cluster [DBG] osdmap e409: 12 total, 12 up, 12 in 2020-09-08T15:52:12.177 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:52:11 smithi104 bash[19922]: debug 2020-09-08T15:52:11.709+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:12.204 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:52:12.308 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:52:12.360 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 19 16 49553 49537 162.937 139.875 0.00378096 0.0061345 2020-09-08T15:52:12.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:52:12 smithi055 bash[19003]: debug 2020-09-08T15:52:12.209+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:12.430 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:52:12.534 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:52:12.718 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:52:12.797 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:12 smithi055 bash[9296]: cluster 2020-09-08T15:52:11.822748+0000 mgr.x (mgr.24118) 746 : cluster [DBG] pgmap v1071: 17 pgs: 17 active+clean; 2.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 MiB/s wr, 1.82k op/s 2020-09-08T15:52:12.798 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:52:12 smithi055 bash[19003]: debug 2020-09-08T15:52:12.433+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:12.837 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:52:12.917 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:52:12.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:52:12 smithi104 bash[22305]: debug 2020-09-08T15:52:12.837+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:12.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:12 smithi104 bash[9203]: cluster 2020-09-08T15:52:11.822748+0000 mgr.x (mgr.24118) 746 : cluster [DBG] pgmap v1071: 17 pgs: 17 active+clean; 2.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 MiB/s wr, 1.82k op/s 2020-09-08T15:52:12.940 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:13.089 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:12 smithi166 bash[9446]: cluster 2020-09-08T15:52:11.822748+0000 mgr.x (mgr.24118) 746 : cluster [DBG] pgmap v1071: 17 pgs: 17 active+clean; 2.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 MiB/s wr, 1.82k op/s 2020-09-08T15:52:13.089 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:13.198 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:52:13.360 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 15:52:13.364523 min lat: 0.00169895 max lat: 0.433439 avg lat: 0.00619462 2020-09-08T15:52:13.360 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:52:13.361 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 20 15 51657 51642 161.368 131.562 0.00864545 0.00619462 2020-09-08T15:52:13.366 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:52:13 smithi166 bash[16593]: debug 2020-09-08T15:52:13.090+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:14.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:13 smithi166 bash[9446]: cluster 2020-09-08T15:52:12.485797+0000 osd.0 (osd.0) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:14.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:13 smithi166 bash[9446]: cluster 2020-09-08T15:52:12.485907+0000 osd.0 (osd.0) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:14.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:13 smithi166 bash[9446]: cluster 2020-09-08T15:52:12.806322+0000 mon.a (mon.0) 1546 : cluster [DBG] osdmap e410: 12 total, 12 up, 12 in 2020-09-08T15:52:14.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:13 smithi055 bash[9296]: cluster 2020-09-08T15:52:12.485797+0000 osd.0 (osd.0) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:14.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:13 smithi055 bash[9296]: cluster 2020-09-08T15:52:12.485907+0000 osd.0 (osd.0) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:14.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:13 smithi055 bash[9296]: cluster 2020-09-08T15:52:12.806322+0000 mon.a (mon.0) 1546 : cluster [DBG] osdmap e410: 12 total, 12 up, 12 in 2020-09-08T15:52:14.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:13 smithi104 bash[9203]: cluster 2020-09-08T15:52:12.485797+0000 osd.0 (osd.0) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:14.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:13 smithi104 bash[9203]: cluster 2020-09-08T15:52:12.485907+0000 osd.0 (osd.0) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:14.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:13 smithi104 bash[9203]: cluster 2020-09-08T15:52:12.806322+0000 mon.a (mon.0) 1546 : cluster [DBG] osdmap e410: 12 total, 12 up, 12 in 2020-09-08T15:52:14.306 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:52:14.360 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 21 16 53868 53852 160.26 138.125 0.00394303 0.00623676 2020-09-08T15:52:14.411 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:14.564 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:14.599 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:52:14.599 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:52:14.669 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:14.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:52:14 smithi104 bash[22305]: debug 2020-09-08T15:52:14.305+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:14.788 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:14.841 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:52:14 smithi166 bash[16593]: debug 2020-09-08T15:52:14.566+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:14.893 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:15.015 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:14 smithi055 bash[9296]: cluster 2020-09-08T15:52:13.823210+0000 mgr.x (mgr.24118) 747 : cluster [DBG] pgmap v1073: 17 pgs: 17 active+clean; 3.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 166 MiB/s wr, 2.65k op/s 2020-09-08T15:52:15.015 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:52:14 smithi055 bash[16646]: debug 2020-09-08T15:52:14.785+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:15.017 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:15.090 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:14 smithi104 bash[9203]: cluster 2020-09-08T15:52:13.823210+0000 mgr.x (mgr.24118) 747 : cluster [DBG] pgmap v1073: 17 pgs: 17 active+clean; 3.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 166 MiB/s wr, 2.65k op/s 2020-09-08T15:52:15.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:14 smithi166 bash[9446]: cluster 2020-09-08T15:52:13.823210+0000 mgr.x (mgr.24118) 747 : cluster [DBG] pgmap v1073: 17 pgs: 17 active+clean; 3.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 166 MiB/s wr, 2.65k op/s 2020-09-08T15:52:15.120 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:15.245 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:15.348 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:15.360 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 22 16 55840 55824 158.577 123.25 0.0116069 0.00630322 2020-09-08T15:52:15.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:52:15 smithi055 bash[11956]: debug 2020-09-08T15:52:15.017+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:15.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:52:15 smithi055 bash[16646]: debug 2020-09-08T15:52:15.249+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:15.461 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:15.568 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:15.618 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:52:15 smithi166 bash[16593]: debug 2020-09-08T15:52:15.458+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:15.711 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:15.819 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:52:15.938 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:52:16.045 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:52:16.130 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:52:15 smithi166 bash[14254]: debug 2020-09-08T15:52:15.938+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:16.131 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:52:15 smithi166 bash[16593]: debug 2020-09-08T15:52:15.710+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:16.360 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 23 16 57728 57712 156.813 118 0.00612202 0.00634928 2020-09-08T15:52:16.942 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:52:17.046 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:52:17.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:16 smithi055 bash[9296]: cluster 2020-09-08T15:52:15.823664+0000 mgr.x (mgr.24118) 748 : cluster [DBG] pgmap v1074: 17 pgs: 17 active+clean; 3.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 MiB/s wr, 1.68k op/s 2020-09-08T15:52:17.170 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:52:17.179 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:52:16 smithi104 bash[22305]: debug 2020-09-08T15:52:16.937+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:17.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:16 smithi104 bash[9203]: cluster 2020-09-08T15:52:15.823664+0000 mgr.x (mgr.24118) 748 : cluster [DBG] pgmap v1074: 17 pgs: 17 active+clean; 3.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 MiB/s wr, 1.68k op/s 2020-09-08T15:52:17.286 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:52:17.360 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 24 16 59634 59618 155.242 119.125 0.01135 0.00643975 2020-09-08T15:52:17.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:16 smithi166 bash[9446]: cluster 2020-09-08T15:52:15.823664+0000 mgr.x (mgr.24118) 748 : cluster [DBG] pgmap v1074: 17 pgs: 17 active+clean; 3.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 MiB/s wr, 1.68k op/s 2020-09-08T15:52:17.396 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:52:17.500 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:52:17.631 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:52:17.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:52:17 smithi055 bash[19003]: debug 2020-09-08T15:52:17.173+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:17.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:52:17 smithi055 bash[19003]: debug 2020-09-08T15:52:17.397+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:17.735 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:52:17.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:52:17 smithi166 bash[14254]: debug 2020-09-08T15:52:17.629+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:18.084 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2020-09-08T15:52:18.206 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:52:18.255 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:52:18.315 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:52:18.360 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 25 16 61493 61477 153.679 116.188 0.0248939 0.00650069 2020-09-08T15:52:18.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:52:18 smithi104 bash[22305]: debug 2020-09-08T15:52:18.205+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:18.441 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:52:18.545 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:18.656 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:52:18 smithi055 bash[14301]: debug 2020-09-08T15:52:18.445+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:18.663 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:18.770 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:52:18.879 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:18 smithi104 bash[9203]: cluster 2020-09-08T15:52:17.824142+0000 mgr.x (mgr.24118) 749 : cluster [DBG] pgmap v1075: 17 pgs: 17 active+clean; 3.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 142 MiB/s wr, 2.28k op/s 2020-09-08T15:52:18.915 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:52:19.019 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:52:19.027 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:52:18 smithi055 bash[16646]: debug 2020-09-08T15:52:18.665+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:19.028 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:18 smithi055 bash[9296]: cluster 2020-09-08T15:52:17.824142+0000 mgr.x (mgr.24118) 749 : cluster [DBG] pgmap v1075: 17 pgs: 17 active+clean; 3.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 142 MiB/s wr, 2.28k op/s 2020-09-08T15:52:19.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:18 smithi166 bash[9446]: cluster 2020-09-08T15:52:17.824142+0000 mgr.x (mgr.24118) 749 : cluster [DBG] pgmap v1075: 17 pgs: 17 active+clean; 3.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 142 MiB/s wr, 2.28k op/s 2020-09-08T15:52:19.117 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:52:18 smithi166 bash[11928]: debug 2020-09-08T15:52:18.913+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:19.132 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:52:19.237 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:52:19.360 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 26 16 63433 63417 152.432 121.25 0.00380621 0.00655699 2020-09-08T15:52:19.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:52:19 smithi055 bash[14301]: debug 2020-09-08T15:52:19.133+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:19.626 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:52:19.729 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:52:19.852 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:52:19.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:52:19 smithi104 bash[15273]: debug 2020-09-08T15:52:19.625+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:19.956 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:52:20.133 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:52:20.156 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:52:19 smithi055 bash[14301]: debug 2020-09-08T15:52:19.853+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:20.237 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:52:20.254 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:52:20 smithi166 bash[11928]: debug 2020-09-08T15:52:20.133+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:20.359 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:52:20.360 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 27 16 65207 65191 150.892 110.875 0.00617152 0.00662513 2020-09-08T15:52:20.461 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:52:20.608 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:52:20.616 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:52:20 smithi166 bash[14254]: debug 2020-09-08T15:52:20.353+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:20.712 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:52:20.887 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:20 smithi055 bash[9296]: cluster 2020-09-08T15:52:19.593542+0000 osd.5 (osd.5) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:20.888 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:20 smithi055 bash[9296]: cluster 2020-09-08T15:52:19.593636+0000 osd.5 (osd.5) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:20.888 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:52:20 smithi055 bash[14301]: debug 2020-09-08T15:52:20.609+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:20.914 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:52:21.018 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:21.138 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:20 smithi055 bash[9296]: cluster 2020-09-08T15:52:19.824618+0000 mgr.x (mgr.24118) 750 : cluster [DBG] pgmap v1076: 17 pgs: 17 active+clean; 3.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 138 MiB/s wr, 2.21k op/s 2020-09-08T15:52:21.139 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:20 smithi055 bash[9296]: cluster 2020-09-08T15:52:19.893259+0000 mon.a (mon.0) 1547 : cluster [DBG] osdmap e411: 12 total, 12 up, 12 in 2020-09-08T15:52:21.139 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:21.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:52:20 smithi104 bash[22305]: debug 2020-09-08T15:52:20.913+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:21.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:20 smithi104 bash[9203]: cluster 2020-09-08T15:52:19.593542+0000 osd.5 (osd.5) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:21.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:20 smithi104 bash[9203]: cluster 2020-09-08T15:52:19.593636+0000 osd.5 (osd.5) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:21.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:20 smithi104 bash[9203]: cluster 2020-09-08T15:52:19.824618+0000 mgr.x (mgr.24118) 750 : cluster [DBG] pgmap v1076: 17 pgs: 17 active+clean; 3.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 138 MiB/s wr, 2.21k op/s 2020-09-08T15:52:21.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:20 smithi104 bash[9203]: cluster 2020-09-08T15:52:19.893259+0000 mon.a (mon.0) 1547 : cluster [DBG] osdmap e411: 12 total, 12 up, 12 in 2020-09-08T15:52:21.243 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:52:21.255 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:20 smithi166 bash[9446]: cluster 2020-09-08T15:52:19.593542+0000 osd.5 (osd.5) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:21.255 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:20 smithi166 bash[9446]: cluster 2020-09-08T15:52:19.593636+0000 osd.5 (osd.5) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:21.255 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:20 smithi166 bash[9446]: cluster 2020-09-08T15:52:19.824618+0000 mgr.x (mgr.24118) 750 : cluster [DBG] pgmap v1076: 17 pgs: 17 active+clean; 3.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 138 MiB/s wr, 2.21k op/s 2020-09-08T15:52:21.255 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:20 smithi166 bash[9446]: cluster 2020-09-08T15:52:19.893259+0000 mon.a (mon.0) 1547 : cluster [DBG] osdmap e411: 12 total, 12 up, 12 in 2020-09-08T15:52:21.356 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:52:21.360 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 28 16 66932 66916 149.353 107.812 0.00433031 0.00669338 2020-09-08T15:52:21.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:52:21 smithi055 bash[16646]: debug 2020-09-08T15:52:21.141+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:21.459 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:21.582 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:21.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:52:21 smithi166 bash[11928]: debug 2020-09-08T15:52:21.353+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:21.692 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:52:21.801 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:52:21.891 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:21 smithi055 bash[9296]: audit 2020-09-08T15:52:21.000609+0000 mon.a (mon.0) 1548 : audit [INF] from='client.? v1:172.21.15.104:0/879062851' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:52:21.892 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:52:21 smithi055 bash[16646]: debug 2020-09-08T15:52:21.585+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:21.908 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:52:21.911 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:52:22.024 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:52:22.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:21 smithi166 bash[9446]: audit 2020-09-08T15:52:21.000609+0000 mon.a (mon.0) 1548 : audit [INF] from='client.? v1:172.21.15.104:0/879062851' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:52:22.117 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:52:21 smithi166 bash[11928]: debug 2020-09-08T15:52:21.801+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:22.129 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:22.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:52:22 smithi055 bash[19003]: debug 2020-09-08T15:52:22.025+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:22.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:21 smithi104 bash[9203]: audit 2020-09-08T15:52:21.000609+0000 mon.a (mon.0) 1548 : audit [INF] from='client.? v1:172.21.15.104:0/879062851' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:52:22.252 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:22.358 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:22.361 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 29 15 68477 68462 147.535 96.625 0.010841 0.00677597 2020-09-08T15:52:22.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:52:22 smithi055 bash[16646]: debug 2020-09-08T15:52:22.253+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:23.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:22 smithi055 bash[9296]: cluster 2020-09-08T15:52:21.715063+0000 osd.10 (osd.10) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:22 smithi055 bash[9296]: cluster 2020-09-08T15:52:21.715173+0000 osd.10 (osd.10) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:22 smithi055 bash[9296]: cluster 2020-09-08T15:52:21.825300+0000 mgr.x (mgr.24118) 751 : cluster [DBG] pgmap v1078: 17 pgs: 17 active+clean; 3.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s wr, 1.67k op/s 2020-09-08T15:52:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:22 smithi055 bash[9296]: cluster 2020-09-08T15:52:21.892872+0000 mon.a (mon.0) 1549 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:52:23.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:22 smithi055 bash[9296]: cluster 2020-09-08T15:52:21.892919+0000 mon.a (mon.0) 1550 : cluster [INF] Cluster is now healthy 2020-09-08T15:52:23.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:22 smithi055 bash[9296]: audit 2020-09-08T15:52:21.905616+0000 mon.a (mon.0) 1551 : audit [INF] from='client.? v1:172.21.15.104:0/879062851' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:52:23.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:22 smithi055 bash[9296]: cluster 2020-09-08T15:52:21.905676+0000 mon.a (mon.0) 1552 : cluster [DBG] osdmap e412: 12 total, 12 up, 12 in 2020-09-08T15:52:23.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:22 smithi104 bash[9203]: cluster 2020-09-08T15:52:21.715063+0000 osd.10 (osd.10) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:23.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:22 smithi104 bash[9203]: cluster 2020-09-08T15:52:21.715173+0000 osd.10 (osd.10) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:23.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:22 smithi104 bash[9203]: cluster 2020-09-08T15:52:21.825300+0000 mgr.x (mgr.24118) 751 : cluster [DBG] pgmap v1078: 17 pgs: 17 active+clean; 3.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s wr, 1.67k op/s 2020-09-08T15:52:23.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:22 smithi104 bash[9203]: cluster 2020-09-08T15:52:21.892872+0000 mon.a (mon.0) 1549 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:52:23.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:22 smithi104 bash[9203]: cluster 2020-09-08T15:52:21.892919+0000 mon.a (mon.0) 1550 : cluster [INF] Cluster is now healthy 2020-09-08T15:52:23.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:22 smithi104 bash[9203]: audit 2020-09-08T15:52:21.905616+0000 mon.a (mon.0) 1551 : audit [INF] from='client.? v1:172.21.15.104:0/879062851' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:52:23.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:22 smithi104 bash[9203]: cluster 2020-09-08T15:52:21.905676+0000 mon.a (mon.0) 1552 : cluster [DBG] osdmap e412: 12 total, 12 up, 12 in 2020-09-08T15:52:23.361 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 30 15 70085 70070 145.967 100.5 0.0647413 0.00684553 2020-09-08T15:52:23.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:22 smithi166 bash[9446]: cluster 2020-09-08T15:52:21.715063+0000 osd.10 (osd.10) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:23.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:22 smithi166 bash[9446]: cluster 2020-09-08T15:52:21.715173+0000 osd.10 (osd.10) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:23.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:22 smithi166 bash[9446]: cluster 2020-09-08T15:52:21.825300+0000 mgr.x (mgr.24118) 751 : cluster [DBG] pgmap v1078: 17 pgs: 17 active+clean; 3.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s wr, 1.67k op/s 2020-09-08T15:52:23.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:22 smithi166 bash[9446]: cluster 2020-09-08T15:52:21.892872+0000 mon.a (mon.0) 1549 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:52:23.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:22 smithi166 bash[9446]: cluster 2020-09-08T15:52:21.892919+0000 mon.a (mon.0) 1550 : cluster [INF] Cluster is now healthy 2020-09-08T15:52:23.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:22 smithi166 bash[9446]: audit 2020-09-08T15:52:21.905616+0000 mon.a (mon.0) 1551 : audit [INF] from='client.? v1:172.21.15.104:0/879062851' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:52:23.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:22 smithi166 bash[9446]: cluster 2020-09-08T15:52:21.905676+0000 mon.a (mon.0) 1552 : cluster [DBG] osdmap e412: 12 total, 12 up, 12 in 2020-09-08T15:52:24.361 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 31 16 71811 71795 144.736 107.812 0.00452099 0.00690641 2020-09-08T15:52:24.688 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:52:24.721 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:52:24.761 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:52:24.798 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:52:24.837 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:52:24.879 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:52:24.916 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:52:24.954 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:52:24.991 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:52:25.026 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:52:25.062 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:52:25.097 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:52:25.101 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:52:25.101 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:52:25.102 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:52:25.102 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:52:25.102 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:52:25.102 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:52:25.102 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:52:25.103 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:52:25.103 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:52:25.103 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:52:25.103 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:52:25.103 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:52:25.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:24 smithi055 bash[9296]: cluster 2020-09-08T15:52:23.825922+0000 mgr.x (mgr.24118) 752 : cluster [DBG] pgmap v1080: 17 pgs: 17 active+clean; 4.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 144 MiB/s wr, 2.30k op/s 2020-09-08T15:52:25.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:24 smithi055 bash[9296]: audit 2020-09-08T15:52:24.655723+0000 mon.a (mon.0) 1553 : audit [DBG] from='client.? v1:172.21.15.104:0/3617341422' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:52:25.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:24 smithi104 bash[9203]: cluster 2020-09-08T15:52:23.825922+0000 mgr.x (mgr.24118) 752 : cluster [DBG] pgmap v1080: 17 pgs: 17 active+clean; 4.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 144 MiB/s wr, 2.30k op/s 2020-09-08T15:52:25.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:24 smithi104 bash[9203]: audit 2020-09-08T15:52:24.655723+0000 mon.a (mon.0) 1553 : audit [DBG] from='client.? v1:172.21.15.104:0/3617341422' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:52:25.362 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 32 16 73424 73408 143.363 100.812 0.0131878 0.00697288 2020-09-08T15:52:25.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:24 smithi166 bash[9446]: cluster 2020-09-08T15:52:23.825922+0000 mgr.x (mgr.24118) 752 : cluster [DBG] pgmap v1080: 17 pgs: 17 active+clean; 4.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 144 MiB/s wr, 2.30k op/s 2020-09-08T15:52:25.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:24 smithi166 bash[9446]: audit 2020-09-08T15:52:24.655723+0000 mon.a (mon.0) 1553 : audit [DBG] from='client.? v1:172.21.15.104:0/3617341422' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:52:26.133 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:26.177 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:52:26.238 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:52:26.361 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 33 16 75184 75168 142.351 110 0.00972513 0.00702255 2020-09-08T15:52:26.411 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:52:26.454 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:52:26.455 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:52:26 smithi104 bash[15273]: debug 2020-09-08T15:52:26.453+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:26.559 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:52:26.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:52:26 smithi166 bash[16593]: debug 2020-09-08T15:52:26.133+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:27.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:26 smithi166 bash[9446]: cluster 2020-09-08T15:52:25.826446+0000 mgr.x (mgr.24118) 753 : cluster [DBG] pgmap v1081: 17 pgs: 17 active+clean; 4.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 83 MiB/s wr, 1.33k op/s 2020-09-08T15:52:27.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:26 smithi166 bash[9446]: cluster 2020-09-08T15:52:26.244468+0000 osd.3 (osd.3) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:27.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:26 smithi166 bash[9446]: cluster 2020-09-08T15:52:26.244549+0000 osd.3 (osd.3) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:27.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:26 smithi104 bash[9203]: cluster 2020-09-08T15:52:25.826446+0000 mgr.x (mgr.24118) 753 : cluster [DBG] pgmap v1081: 17 pgs: 17 active+clean; 4.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 83 MiB/s wr, 1.33k op/s 2020-09-08T15:52:27.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:26 smithi104 bash[9203]: cluster 2020-09-08T15:52:26.244468+0000 osd.3 (osd.3) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:27.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:26 smithi104 bash[9203]: cluster 2020-09-08T15:52:26.244549+0000 osd.3 (osd.3) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:27.361 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 34 16 76621 76605 140.806 89.8125 0.00447062 0.00709938 2020-09-08T15:52:27.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:26 smithi055 bash[9296]: cluster 2020-09-08T15:52:25.826446+0000 mgr.x (mgr.24118) 753 : cluster [DBG] pgmap v1081: 17 pgs: 17 active+clean; 4.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 83 MiB/s wr, 1.33k op/s 2020-09-08T15:52:27.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:26 smithi055 bash[9296]: cluster 2020-09-08T15:52:26.244468+0000 osd.3 (osd.3) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:27.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:26 smithi055 bash[9296]: cluster 2020-09-08T15:52:26.244549+0000 osd.3 (osd.3) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:27.631 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:52:27.632 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T15:52:27.632 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2020-09-08T15:52:27.632 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2020-09-08T15:52:27.633 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:52:28.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:27 smithi104 bash[9203]: cluster 2020-09-08T15:52:26.913486+0000 mon.a (mon.0) 1554 : cluster [DBG] osdmap e413: 12 total, 12 up, 12 in 2020-09-08T15:52:28.361 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 35 15 78038 78023 139.315 88.625 0.00465324 0.0071755 2020-09-08T15:52:28.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:27 smithi166 bash[9446]: cluster 2020-09-08T15:52:26.913486+0000 mon.a (mon.0) 1554 : cluster [DBG] osdmap e413: 12 total, 12 up, 12 in 2020-09-08T15:52:28.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:27 smithi055 bash[9296]: cluster 2020-09-08T15:52:26.913486+0000 mon.a (mon.0) 1554 : cluster [DBG] osdmap e413: 12 total, 12 up, 12 in 2020-09-08T15:52:28.640 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:52:28.810 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:52:28.960 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:52:29.064 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:52:29.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:28 smithi104 bash[9203]: cluster 2020-09-08T15:52:27.827057+0000 mgr.x (mgr.24118) 754 : cluster [DBG] pgmap v1083: 17 pgs: 17 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 122 MiB/s wr, 1.95k op/s 2020-09-08T15:52:29.361 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 36 16 79638 79622 138.221 99.9375 0.0102467 0.00723275 2020-09-08T15:52:29.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:28 smithi166 bash[9446]: cluster 2020-09-08T15:52:27.827057+0000 mgr.x (mgr.24118) 754 : cluster [DBG] pgmap v1083: 17 pgs: 17 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 122 MiB/s wr, 1.95k op/s 2020-09-08T15:52:29.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:52:28 smithi166 bash[11928]: debug 2020-09-08T15:52:28.961+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:29.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:28 smithi055 bash[9296]: cluster 2020-09-08T15:52:27.827057+0000 mgr.x (mgr.24118) 754 : cluster [DBG] pgmap v1083: 17 pgs: 17 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 122 MiB/s wr, 1.95k op/s 2020-09-08T15:52:29.662 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:52:29.767 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:52:29.905 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:52:29.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:52:29 smithi104 bash[17588]: debug 2020-09-08T15:52:29.661+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:30.034 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:52:30.156 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:52:29 smithi055 bash[14301]: debug 2020-09-08T15:52:29.909+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:30.237 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2020-09-08T15:52:30.361 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 37 16 81175 81159 137.081 96.0625 0.00824194 0.0072927 2020-09-08T15:52:30.412 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:52:31.107 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:30 smithi104 bash[9203]: cluster 2020-09-08T15:52:29.781991+0000 mon.a (mon.0) 1555 : cluster [DBG] osdmap e414: 12 total, 12 up, 12 in 2020-09-08T15:52:31.107 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:30 smithi104 bash[9203]: cluster 2020-09-08T15:52:29.827568+0000 mgr.x (mgr.24118) 755 : cluster [DBG] pgmap v1085: 17 pgs: 17 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 115 MiB/s wr, 1.85k op/s 2020-09-08T15:52:31.108 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:52:31.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:30 smithi166 bash[9446]: cluster 2020-09-08T15:52:29.781991+0000 mon.a (mon.0) 1555 : cluster [DBG] osdmap e414: 12 total, 12 up, 12 in 2020-09-08T15:52:31.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:30 smithi166 bash[9446]: cluster 2020-09-08T15:52:29.827568+0000 mgr.x (mgr.24118) 755 : cluster [DBG] pgmap v1085: 17 pgs: 17 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 115 MiB/s wr, 1.85k op/s 2020-09-08T15:52:31.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:30 smithi055 bash[9296]: cluster 2020-09-08T15:52:29.781991+0000 mon.a (mon.0) 1555 : cluster [DBG] osdmap e414: 12 total, 12 up, 12 in 2020-09-08T15:52:31.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:30 smithi055 bash[9296]: cluster 2020-09-08T15:52:29.827568+0000 mgr.x (mgr.24118) 755 : cluster [DBG] pgmap v1085: 17 pgs: 17 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 115 MiB/s wr, 1.85k op/s 2020-09-08T15:52:31.212 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:52:31.359 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:52:31.361 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 38 16 82519 82503 135.684 84 0.0179574 0.00736708 2020-09-08T15:52:31.427 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:52:31 smithi104 bash[22305]: debug 2020-09-08T15:52:31.109+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:31.463 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:52:31.616 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:52:31 smithi166 bash[14254]: debug 2020-09-08T15:52:31.357+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:32.287 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:52:32.361 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 39 16 84157 84141 134.83 102.375 0.00718315 0.0074144 2020-09-08T15:52:32.392 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:32.677 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:52:32 smithi104 bash[15273]: debug 2020-09-08T15:52:32.289+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:33.118 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:33.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:32 smithi055 bash[9296]: cluster 2020-09-08T15:52:31.827964+0000 mgr.x (mgr.24118) 756 : cluster [DBG] pgmap v1086: 17 pgs: 17 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T15:52:33.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:52:33 smithi104 bash[19922]: debug 2020-09-08T15:52:33.117+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:33.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:32 smithi104 bash[9203]: cluster 2020-09-08T15:52:31.827964+0000 mgr.x (mgr.24118) 756 : cluster [DBG] pgmap v1086: 17 pgs: 17 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T15:52:33.223 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:52:33.362 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 15:52:33.366271 min lat: 0.00169895 max lat: 0.433439 avg lat: 0.00747095 2020-09-08T15:52:33.362 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:52:33.362 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 40 16 85664 85648 133.814 94.1875 0.00552245 0.00747095 2020-09-08T15:52:33.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:32 smithi166 bash[9446]: cluster 2020-09-08T15:52:31.827964+0000 mgr.x (mgr.24118) 756 : cluster [DBG] pgmap v1086: 17 pgs: 17 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T15:52:33.950 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:52:34.054 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:52:34.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:33 smithi055 bash[9296]: cluster 2020-09-08T15:52:32.584143+0000 osd.8 (osd.8) 53 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:34.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:33 smithi055 bash[9296]: cluster 2020-09-08T15:52:32.584261+0000 osd.8 (osd.8) 54 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:34.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:33 smithi055 bash[9296]: cluster 2020-09-08T15:52:32.884632+0000 mon.a (mon.0) 1556 : cluster [DBG] osdmap e415: 12 total, 12 up, 12 in 2020-09-08T15:52:34.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:52:33 smithi104 bash[17588]: debug 2020-09-08T15:52:33.945+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:34.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:33 smithi104 bash[9203]: cluster 2020-09-08T15:52:32.584143+0000 osd.8 (osd.8) 53 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:34.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:33 smithi104 bash[9203]: cluster 2020-09-08T15:52:32.584261+0000 osd.8 (osd.8) 54 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:34.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:33 smithi104 bash[9203]: cluster 2020-09-08T15:52:32.884632+0000 mon.a (mon.0) 1556 : cluster [DBG] osdmap e415: 12 total, 12 up, 12 in 2020-09-08T15:52:34.204 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:52:34.308 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:34.362 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 41 16 87208 87192 132.903 96.5 0.00492837 0.00752175 2020-09-08T15:52:34.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:33 smithi166 bash[9446]: cluster 2020-09-08T15:52:32.584143+0000 osd.8 (osd.8) 53 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:34.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:33 smithi166 bash[9446]: cluster 2020-09-08T15:52:32.584261+0000 osd.8 (osd.8) 54 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:34.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:33 smithi166 bash[9446]: cluster 2020-09-08T15:52:32.884632+0000 mon.a (mon.0) 1556 : cluster [DBG] osdmap e415: 12 total, 12 up, 12 in 2020-09-08T15:52:34.434 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:34.538 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:52:34.547 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:52:34 smithi055 bash[14301]: debug 2020-09-08T15:52:34.205+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:34.548 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:52:34 smithi055 bash[11956]: debug 2020-09-08T15:52:34.433+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:34.672 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:52:34.780 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:34.897 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:52:34 smithi055 bash[19003]: debug 2020-09-08T15:52:34.673+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:35.051 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:34 smithi104 bash[9203]: cluster 2020-09-08T15:52:33.828397+0000 mgr.x (mgr.24118) 757 : cluster [DBG] pgmap v1088: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s wr, 1.89k op/s 2020-09-08T15:52:35.051 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:34 smithi104 bash[9203]: cluster 2020-09-08T15:52:34.783233+0000 mon.a (mon.0) 1557 : cluster [DBG] osdmap e416: 12 total, 12 up, 12 in 2020-09-08T15:52:35.123 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:34 smithi055 bash[9296]: cluster 2020-09-08T15:52:33.828397+0000 mgr.x (mgr.24118) 757 : cluster [DBG] pgmap v1088: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s wr, 1.89k op/s 2020-09-08T15:52:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:34 smithi055 bash[9296]: cluster 2020-09-08T15:52:34.783233+0000 mon.a (mon.0) 1557 : cluster [DBG] osdmap e416: 12 total, 12 up, 12 in 2020-09-08T15:52:35.228 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:52:35.342 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:52:35.366 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 42 15 88585 88570 131.789 86.125 0.0108409 0.00758574 2020-09-08T15:52:35.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:34 smithi166 bash[9446]: cluster 2020-09-08T15:52:33.828397+0000 mgr.x (mgr.24118) 757 : cluster [DBG] pgmap v1088: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s wr, 1.89k op/s 2020-09-08T15:52:35.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:34 smithi166 bash[9446]: cluster 2020-09-08T15:52:34.783233+0000 mon.a (mon.0) 1557 : cluster [DBG] osdmap e416: 12 total, 12 up, 12 in 2020-09-08T15:52:35.432 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:52:35 smithi104 bash[19922]: debug 2020-09-08T15:52:35.121+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:35.464 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:52:35.616 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:52:35 smithi055 bash[14301]: debug 2020-09-08T15:52:35.344+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:35.617 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:52:35.720 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:35.906 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:52:35 smithi055 bash[14301]: debug 2020-09-08T15:52:35.620+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:36.362 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 43 15 90129 90114 130.968 96.5 0.0124412 0.00763341 2020-09-08T15:52:36.427 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:36.531 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:36.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:52:36 smithi104 bash[19922]: debug 2020-09-08T15:52:36.425+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:36.910 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:52:37.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:36 smithi104 bash[9203]: cluster 2020-09-08T15:52:35.828855+0000 mgr.x (mgr.24118) 758 : cluster [DBG] pgmap v1090: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T15:52:37.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:36 smithi104 bash[9203]: audit 2020-09-08T15:52:36.793313+0000 mon.b (mon.1) 99 : audit [INF] from='client.? v1:172.21.15.104:0/4250640398' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:52:37.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:36 smithi104 bash[9203]: audit 2020-09-08T15:52:36.793720+0000 mon.a (mon.0) 1558 : audit [INF] from='client.25681 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:52:37.362 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 44 16 91393 91377 129.786 78.9375 0.00858409 0.00770273 2020-09-08T15:52:37.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:36 smithi166 bash[9446]: cluster 2020-09-08T15:52:35.828855+0000 mgr.x (mgr.24118) 758 : cluster [DBG] pgmap v1090: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T15:52:37.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:36 smithi166 bash[9446]: audit 2020-09-08T15:52:36.793313+0000 mon.b (mon.1) 99 : audit [INF] from='client.? v1:172.21.15.104:0/4250640398' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:52:37.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:36 smithi166 bash[9446]: audit 2020-09-08T15:52:36.793720+0000 mon.a (mon.0) 1558 : audit [INF] from='client.25681 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:52:37.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:36 smithi055 bash[9296]: cluster 2020-09-08T15:52:35.828855+0000 mgr.x (mgr.24118) 758 : cluster [DBG] pgmap v1090: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T15:52:37.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:36 smithi055 bash[9296]: audit 2020-09-08T15:52:36.793313+0000 mon.b (mon.1) 99 : audit [INF] from='client.? v1:172.21.15.104:0/4250640398' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:52:37.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:36 smithi055 bash[9296]: audit 2020-09-08T15:52:36.793720+0000 mon.a (mon.0) 1558 : audit [INF] from='client.25681 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:52:38.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:37 smithi104 bash[9203]: cluster 2020-09-08T15:52:36.904211+0000 mon.a (mon.0) 1559 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:52:38.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:37 smithi104 bash[9203]: audit 2020-09-08T15:52:36.910100+0000 mon.a (mon.0) 1560 : audit [INF] from='client.25681 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:52:38.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:37 smithi104 bash[9203]: cluster 2020-09-08T15:52:36.910221+0000 mon.a (mon.0) 1561 : cluster [DBG] osdmap e417: 12 total, 12 up, 12 in 2020-09-08T15:52:38.362 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 45 16 92789 92773 128.84 87.25 0.00757755 0.00775096 2020-09-08T15:52:38.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:37 smithi166 bash[9446]: cluster 2020-09-08T15:52:36.904211+0000 mon.a (mon.0) 1559 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:52:38.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:37 smithi166 bash[9446]: audit 2020-09-08T15:52:36.910100+0000 mon.a (mon.0) 1560 : audit [INF] from='client.25681 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:52:38.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:37 smithi166 bash[9446]: cluster 2020-09-08T15:52:36.910221+0000 mon.a (mon.0) 1561 : cluster [DBG] osdmap e417: 12 total, 12 up, 12 in 2020-09-08T15:52:38.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:37 smithi055 bash[9296]: cluster 2020-09-08T15:52:36.904211+0000 mon.a (mon.0) 1559 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:52:38.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:37 smithi055 bash[9296]: audit 2020-09-08T15:52:36.910100+0000 mon.a (mon.0) 1560 : audit [INF] from='client.25681 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:52:38.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:37 smithi055 bash[9296]: cluster 2020-09-08T15:52:36.910221+0000 mon.a (mon.0) 1561 : cluster [DBG] osdmap e417: 12 total, 12 up, 12 in 2020-09-08T15:52:39.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:38 smithi104 bash[9203]: cluster 2020-09-08T15:52:37.829384+0000 mgr.x (mgr.24118) 759 : cluster [DBG] pgmap v1092: 17 pgs: 17 active+clean; 5.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 126 MiB/s wr, 2.01k op/s 2020-09-08T15:52:39.362 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 46 16 94131 94115 127.863 83.875 0.00851692 0.00781877 2020-09-08T15:52:39.363 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:52:39.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:38 smithi166 bash[9446]: cluster 2020-09-08T15:52:37.829384+0000 mgr.x (mgr.24118) 759 : cluster [DBG] pgmap v1092: 17 pgs: 17 active+clean; 5.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 126 MiB/s wr, 2.01k op/s 2020-09-08T15:52:39.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:38 smithi055 bash[9296]: cluster 2020-09-08T15:52:37.829384+0000 mgr.x (mgr.24118) 759 : cluster [DBG] pgmap v1092: 17 pgs: 17 active+clean; 5.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 126 MiB/s wr, 2.01k op/s 2020-09-08T15:52:39.408 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:52:39.440 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:52:39.483 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:52:39.518 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:52:39.568 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:52:39.604 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:52:39.632 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:52:39.633 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:52:39.645 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:52:39.677 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:52:39.701 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:52:39.726 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:52:39.757 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:52:39.760 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:52:39.760 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:52:39.761 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:52:39.761 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:52:39.761 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:52:39.762 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:52:39.762 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:52:39.762 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:52:39.763 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:52:39.763 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:52:39.763 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:52:39.764 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:52:40.302 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:39 smithi104 bash[9203]: audit 2020-09-08T15:52:39.314996+0000 mon.a (mon.0) 1562 : audit [DBG] from='client.? v1:172.21.15.104:0/3876435654' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:52:40.302 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:39 smithi104 bash[9203]: cluster 2020-09-08T15:52:39.788105+0000 mon.a (mon.0) 1563 : cluster [DBG] osdmap e418: 12 total, 12 up, 12 in 2020-09-08T15:52:40.362 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 47 16 95119 95103 126.456 61.75 0.0218601 0.0079057 2020-09-08T15:52:40.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:39 smithi166 bash[9446]: audit 2020-09-08T15:52:39.314996+0000 mon.a (mon.0) 1562 : audit [DBG] from='client.? v1:172.21.15.104:0/3876435654' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:52:40.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:39 smithi166 bash[9446]: cluster 2020-09-08T15:52:39.788105+0000 mon.a (mon.0) 1563 : cluster [DBG] osdmap e418: 12 total, 12 up, 12 in 2020-09-08T15:52:40.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:39 smithi055 bash[9296]: audit 2020-09-08T15:52:39.314996+0000 mon.a (mon.0) 1562 : audit [DBG] from='client.? v1:172.21.15.104:0/3876435654' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:52:40.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:39 smithi055 bash[9296]: cluster 2020-09-08T15:52:39.788105+0000 mon.a (mon.0) 1563 : cluster [DBG] osdmap e418: 12 total, 12 up, 12 in 2020-09-08T15:52:41.028 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:52:41.201 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:52:41.362 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 48 16 96563 96547 125.701 90.25 0.00787788 0.00795307 2020-09-08T15:52:41.375 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:40 smithi166 bash[9446]: cluster 2020-09-08T15:52:39.829849+0000 mgr.x (mgr.24118) 760 : cluster [DBG] pgmap v1094: 17 pgs: 17 active+clean; 5.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s wr, 1.29k op/s 2020-09-08T15:52:41.382 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:40 smithi166 bash[9446]: cluster 2020-09-08T15:52:39.894349+0000 osd.1 (osd.1) 82 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:41.383 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:40 smithi166 bash[9446]: cluster 2020-09-08T15:52:39.894425+0000 osd.1 (osd.1) 83 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:41.383 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:40 smithi166 bash[9446]: cluster 2020-09-08T15:52:40.799933+0000 mon.a (mon.0) 1564 : cluster [DBG] osdmap e419: 12 total, 12 up, 12 in 2020-09-08T15:52:41.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:40 smithi055 bash[9296]: cluster 2020-09-08T15:52:39.829849+0000 mgr.x (mgr.24118) 760 : cluster [DBG] pgmap v1094: 17 pgs: 17 active+clean; 5.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s wr, 1.29k op/s 2020-09-08T15:52:41.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:40 smithi055 bash[9296]: cluster 2020-09-08T15:52:39.894349+0000 osd.1 (osd.1) 82 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:41.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:40 smithi055 bash[9296]: cluster 2020-09-08T15:52:39.894425+0000 osd.1 (osd.1) 83 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:41.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:40 smithi055 bash[9296]: cluster 2020-09-08T15:52:40.799933+0000 mon.a (mon.0) 1564 : cluster [DBG] osdmap e419: 12 total, 12 up, 12 in 2020-09-08T15:52:41.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:40 smithi104 bash[9203]: cluster 2020-09-08T15:52:39.829849+0000 mgr.x (mgr.24118) 760 : cluster [DBG] pgmap v1094: 17 pgs: 17 active+clean; 5.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s wr, 1.29k op/s 2020-09-08T15:52:41.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:40 smithi104 bash[9203]: cluster 2020-09-08T15:52:39.894349+0000 osd.1 (osd.1) 82 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:41.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:40 smithi104 bash[9203]: cluster 2020-09-08T15:52:39.894425+0000 osd.1 (osd.1) 83 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:41.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:40 smithi104 bash[9203]: cluster 2020-09-08T15:52:40.799933+0000 mon.a (mon.0) 1564 : cluster [DBG] osdmap e419: 12 total, 12 up, 12 in 2020-09-08T15:52:41.792 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:41.896 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:52:41.970 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2020-09-08T15:52:42.066 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:52:42.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:52:41 smithi055 bash[11956]: debug 2020-09-08T15:52:41.792+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:42.169 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:42.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:52:42 smithi104 bash[17588]: debug 2020-09-08T15:52:42.065+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:42.176 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:52:42.363 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 49 16 97935 97919 124.886 85.75 0.00489087 0.00800505 2020-09-08T15:52:42.794 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:42.897 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:52:43.037 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:52:43.141 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:52:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:42 smithi104 bash[9203]: cluster 2020-09-08T15:52:41.830348+0000 mgr.x (mgr.24118) 761 : cluster [DBG] pgmap v1096: 17 pgs: 17 active+clean; 5.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 84 MiB/s wr, 1.34k op/s 2020-09-08T15:52:43.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:52:42 smithi104 bash[19922]: debug 2020-09-08T15:52:42.789+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:43.266 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:42 smithi055 bash[9296]: cluster 2020-09-08T15:52:41.830348+0000 mgr.x (mgr.24118) 761 : cluster [DBG] pgmap v1096: 17 pgs: 17 active+clean; 5.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 84 MiB/s wr, 1.34k op/s 2020-09-08T15:52:43.267 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:52:43.362 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 50 16 99223 99207 123.998 80.5 0.00630414 0.00806246 2020-09-08T15:52:43.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:52:43 smithi166 bash[11928]: debug 2020-09-08T15:52:43.037+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:43.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:42 smithi166 bash[9446]: cluster 2020-09-08T15:52:41.830348+0000 mgr.x (mgr.24118) 761 : cluster [DBG] pgmap v1096: 17 pgs: 17 active+clean; 5.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 84 MiB/s wr, 1.34k op/s 2020-09-08T15:52:43.369 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:43.494 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:43.597 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:52:43.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:52:43 smithi055 bash[16646]: debug 2020-09-08T15:52:43.492+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:43.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:52:43 smithi055 bash[19003]: debug 2020-09-08T15:52:43.268+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:43.719 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:52:43.822 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:52:43.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:52:43 smithi166 bash[18960]: debug 2020-09-08T15:52:43.717+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:43.957 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:52:44.054 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:52:44.086 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:52:44 smithi104 bash[17588]: debug 2020-09-08T15:52:44.049+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:44.131 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:52:44.158 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:44.269 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:44.364 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 51 16 100562 100546 123.208 83.6875 0.00550683 0.00811092 2020-09-08T15:52:44.376 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:44.523 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:44.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:52:44 smithi166 bash[16593]: debug 2020-09-08T15:52:44.269+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:44.625 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:44.659 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:52:44 smithi055 bash[16646]: debug 2020-09-08T15:52:44.520+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:44.767 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:44.870 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:52:44.988 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:52:45.091 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:45.100 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:52:44 smithi055 bash[11956]: debug 2020-09-08T15:52:44.764+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:45.100 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:44 smithi055 bash[9296]: cluster 2020-09-08T15:52:43.830906+0000 mgr.x (mgr.24118) 762 : cluster [DBG] pgmap v1097: 17 pgs: 17 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 108 MiB/s wr, 1.73k op/s 2020-09-08T15:52:45.101 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:44 smithi055 bash[9296]: cluster 2020-09-08T15:52:44.787676+0000 mon.a (mon.0) 1565 : cluster [DBG] osdmap e420: 12 total, 12 up, 12 in 2020-09-08T15:52:45.173 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:44 smithi104 bash[9203]: cluster 2020-09-08T15:52:43.830906+0000 mgr.x (mgr.24118) 762 : cluster [DBG] pgmap v1097: 17 pgs: 17 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 108 MiB/s wr, 1.73k op/s 2020-09-08T15:52:45.173 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:44 smithi104 bash[9203]: cluster 2020-09-08T15:52:44.787676+0000 mon.a (mon.0) 1565 : cluster [DBG] osdmap e420: 12 total, 12 up, 12 in 2020-09-08T15:52:45.203 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:45.316 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:45.363 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 52 16 101800 101784 122.326 77.375 0.0114962 0.00817288 2020-09-08T15:52:45.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:44 smithi166 bash[9446]: cluster 2020-09-08T15:52:43.830906+0000 mgr.x (mgr.24118) 762 : cluster [DBG] pgmap v1097: 17 pgs: 17 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 108 MiB/s wr, 1.73k op/s 2020-09-08T15:52:45.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:44 smithi166 bash[9446]: cluster 2020-09-08T15:52:44.787676+0000 mon.a (mon.0) 1565 : cluster [DBG] osdmap e420: 12 total, 12 up, 12 in 2020-09-08T15:52:45.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:52:44 smithi166 bash[18960]: debug 2020-09-08T15:52:44.985+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:45.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:52:45 smithi055 bash[16646]: debug 2020-09-08T15:52:45.204+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:45.426 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:45.531 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:52:45.906 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:52:45 smithi055 bash[11956]: debug 2020-09-08T15:52:45.428+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:46.363 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 53 16 103137 103121 121.594 83.5625 0.0221815 0.00822139 2020-09-08T15:52:46.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:45 smithi166 bash[9446]: cluster 2020-09-08T15:52:44.662832+0000 osd.6 (osd.6) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:46.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:45 smithi166 bash[9446]: cluster 2020-09-08T15:52:44.662915+0000 osd.6 (osd.6) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:46.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:45 smithi055 bash[9296]: cluster 2020-09-08T15:52:44.662832+0000 osd.6 (osd.6) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:46.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:45 smithi055 bash[9296]: cluster 2020-09-08T15:52:44.662915+0000 osd.6 (osd.6) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:46.442 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:45 smithi104 bash[9203]: cluster 2020-09-08T15:52:44.662832+0000 osd.6 (osd.6) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:46.442 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:45 smithi104 bash[9203]: cluster 2020-09-08T15:52:44.662915+0000 osd.6 (osd.6) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:46.526 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:52:46.630 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:46.744 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:46.850 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:52:46.861 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:52:46 smithi104 bash[15273]: debug 2020-09-08T15:52:46.525+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:46.960 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:46 smithi166 bash[9446]: cluster 2020-09-08T15:52:45.831392+0000 mgr.x (mgr.24118) 763 : cluster [DBG] pgmap v1099: 17 pgs: 17 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s wr, 1.08k op/s 2020-09-08T15:52:46.961 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:52:46 smithi166 bash[16593]: debug 2020-09-08T15:52:46.741+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:47.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:46 smithi104 bash[9203]: cluster 2020-09-08T15:52:45.831392+0000 mgr.x (mgr.24118) 763 : cluster [DBG] pgmap v1099: 17 pgs: 17 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s wr, 1.08k op/s 2020-09-08T15:52:47.363 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 54 16 104473 104457 120.889 83.5 0.00973051 0.00826993 2020-09-08T15:52:47.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:46 smithi055 bash[9296]: cluster 2020-09-08T15:52:45.831392+0000 mgr.x (mgr.24118) 763 : cluster [DBG] pgmap v1099: 17 pgs: 17 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s wr, 1.08k op/s 2020-09-08T15:52:47.721 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:52:47.824 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:47.966 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:48.055 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:52:47 smithi104 bash[17588]: debug 2020-09-08T15:52:47.717+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:48.072 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:48.118 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:52:47 smithi166 bash[16593]: debug 2020-09-08T15:52:47.965+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:48.363 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 55 16 105759 105743 120.152 80.375 0.010892 0.00832047 2020-09-08T15:52:49.010 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:49.114 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:52:49.268 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:48 smithi166 bash[9446]: cluster 2020-09-08T15:52:47.831872+0000 mgr.x (mgr.24118) 764 : cluster [DBG] pgmap v1100: 17 pgs: 17 active+clean; 6.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.48k op/s 2020-09-08T15:52:49.269 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:48 smithi166 bash[9446]: cluster 2020-09-08T15:52:48.281596+0000 osd.11 (osd.11) 68 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:49.269 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:48 smithi166 bash[9446]: cluster 2020-09-08T15:52:48.281678+0000 osd.11 (osd.11) 69 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:49.269 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:52:49 smithi166 bash[14254]: debug 2020-09-08T15:52:49.261+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:49.270 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:52:49.363 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 56 16 107137 107121 119.544 86.125 0.0132714 0.00836341 2020-09-08T15:52:49.371 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:49.381 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:48 smithi055 bash[9296]: cluster 2020-09-08T15:52:47.831872+0000 mgr.x (mgr.24118) 764 : cluster [DBG] pgmap v1100: 17 pgs: 17 active+clean; 6.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.48k op/s 2020-09-08T15:52:49.382 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:48 smithi055 bash[9296]: cluster 2020-09-08T15:52:48.281596+0000 osd.11 (osd.11) 68 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:49.382 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:48 smithi055 bash[9296]: cluster 2020-09-08T15:52:48.281678+0000 osd.11 (osd.11) 69 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:49.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:52:49 smithi104 bash[19922]: debug 2020-09-08T15:52:49.009+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:49.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:48 smithi104 bash[9203]: cluster 2020-09-08T15:52:47.831872+0000 mgr.x (mgr.24118) 764 : cluster [DBG] pgmap v1100: 17 pgs: 17 active+clean; 6.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.48k op/s 2020-09-08T15:52:49.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:48 smithi104 bash[9203]: cluster 2020-09-08T15:52:48.281596+0000 osd.11 (osd.11) 68 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:49.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:48 smithi104 bash[9203]: cluster 2020-09-08T15:52:48.281678+0000 osd.11 (osd.11) 69 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:49.490 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:49.594 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:49.662 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:52:49 smithi055 bash[16646]: debug 2020-09-08T15:52:49.488+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:49.871 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:49.981 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:50.094 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:50.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:49 smithi104 bash[9203]: cluster 2020-09-08T15:52:48.975127+0000 mon.a (mon.0) 1566 : cluster [DBG] osdmap e421: 12 total, 12 up, 12 in 2020-09-08T15:52:50.183 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:49 smithi104 bash[9203]: cluster 2020-09-08T15:52:49.788404+0000 mon.a (mon.0) 1567 : cluster [DBG] osdmap e422: 12 total, 12 up, 12 in 2020-09-08T15:52:50.183 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:49 smithi104 bash[9203]: audit 2020-09-08T15:52:49.798908+0000 mon.a (mon.0) 1568 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:52:50.183 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:49 smithi104 bash[9203]: audit 2020-09-08T15:52:49.802866+0000 mon.a (mon.0) 1569 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:52:50.183 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:52:49 smithi104 bash[19922]: debug 2020-09-08T15:52:49.865+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:50.197 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:50.307 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:49 smithi055 bash[9296]: cluster 2020-09-08T15:52:48.975127+0000 mon.a (mon.0) 1566 : cluster [DBG] osdmap e421: 12 total, 12 up, 12 in 2020-09-08T15:52:50.308 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:49 smithi055 bash[9296]: cluster 2020-09-08T15:52:49.788404+0000 mon.a (mon.0) 1567 : cluster [DBG] osdmap e422: 12 total, 12 up, 12 in 2020-09-08T15:52:50.308 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:49 smithi055 bash[9296]: audit 2020-09-08T15:52:49.798908+0000 mon.a (mon.0) 1568 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:52:50.308 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:49 smithi055 bash[9296]: audit 2020-09-08T15:52:49.802866+0000 mon.a (mon.0) 1569 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:52:50.308 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:52:50 smithi055 bash[11956]: debug 2020-09-08T15:52:50.092+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:50.309 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:52:50 smithi055 bash[11956]: debug 2020-09-08T15:52:50.308+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:50.309 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:50.367 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 57 16 108416 108400 118.849 79.9375 0.0141288 0.00841169 2020-09-08T15:52:50.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:49 smithi166 bash[9446]: cluster 2020-09-08T15:52:48.975127+0000 mon.a (mon.0) 1566 : cluster [DBG] osdmap e421: 12 total, 12 up, 12 in 2020-09-08T15:52:50.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:49 smithi166 bash[9446]: cluster 2020-09-08T15:52:49.788404+0000 mon.a (mon.0) 1567 : cluster [DBG] osdmap e422: 12 total, 12 up, 12 in 2020-09-08T15:52:50.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:49 smithi166 bash[9446]: audit 2020-09-08T15:52:49.798908+0000 mon.a (mon.0) 1568 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:52:50.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:49 smithi166 bash[9446]: audit 2020-09-08T15:52:49.802866+0000 mon.a (mon.0) 1569 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:52:50.411 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:50.535 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:50.639 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:50.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:52:50 smithi055 bash[11956]: debug 2020-09-08T15:52:50.536+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:50.959 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:51.063 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:52:51.193 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:52:51.299 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:51.308 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:52:50 smithi104 bash[19922]: debug 2020-09-08T15:52:50.957+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:51.308 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:50 smithi104 bash[9203]: cluster 2020-09-08T15:52:49.832402+0000 mgr.x (mgr.24118) 765 : cluster [DBG] pgmap v1103: 17 pgs: 17 active+clean; 6.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s wr, 1.16k op/s 2020-09-08T15:52:51.369 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 58 16 109689 109673 118.172 79.5625 0.0207856 0.00846056 2020-09-08T15:52:51.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:50 smithi166 bash[9446]: cluster 2020-09-08T15:52:49.832402+0000 mgr.x (mgr.24118) 765 : cluster [DBG] pgmap v1103: 17 pgs: 17 active+clean; 6.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s wr, 1.16k op/s 2020-09-08T15:52:51.370 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:52:51 smithi166 bash[11928]: debug 2020-09-08T15:52:51.189+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:51.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:50 smithi055 bash[9296]: cluster 2020-09-08T15:52:49.832402+0000 mgr.x (mgr.24118) 765 : cluster [DBG] pgmap v1103: 17 pgs: 17 active+clean; 6.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s wr, 1.16k op/s 2020-09-08T15:52:51.668 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:51.772 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:51.777 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:52:51.777 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:52:51.915 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:51.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:52:51 smithi104 bash[19922]: debug 2020-09-08T15:52:51.665+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:52.017 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:52.129 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:52:51 smithi055 bash[11956]: debug 2020-09-08T15:52:51.908+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:52.130 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:52.241 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:52:52.363 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 59 16 110846 110830 117.395 72.3125 0.00942312 0.00851621 2020-09-08T15:52:52.376 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:52:52.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:52:52 smithi055 bash[11956]: debug 2020-09-08T15:52:52.132+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:52.480 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:52.603 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:52.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:52:52 smithi166 bash[18960]: debug 2020-09-08T15:52:52.373+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:52.707 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:52:52.851 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:52:52 smithi055 bash[16646]: debug 2020-09-08T15:52:52.600+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:52.853 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:52:52.957 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:52.997 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:52:53.075 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:53.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:52:52 smithi055 bash[19003]: debug 2020-09-08T15:52:52.852+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:53.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:52 smithi055 bash[9296]: cluster 2020-09-08T15:52:51.571613+0000 osd.4 (osd.4) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:53.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:52 smithi055 bash[9296]: cluster 2020-09-08T15:52:51.571686+0000 osd.4 (osd.4) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:53.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:52 smithi055 bash[9296]: cluster 2020-09-08T15:52:51.832906+0000 mgr.x (mgr.24118) 766 : cluster [DBG] pgmap v1104: 17 pgs: 17 active+clean; 6.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 MiB/s wr, 1.05k op/s 2020-09-08T15:52:53.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:52 smithi055 bash[9296]: cluster 2020-09-08T15:52:51.987701+0000 mon.a (mon.0) 1570 : cluster [DBG] osdmap e423: 12 total, 12 up, 12 in 2020-09-08T15:52:53.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:52 smithi055 bash[9296]: audit 2020-09-08T15:52:52.052700+0000 mon.c (mon.2) 62 : audit [INF] from='client.? v1:172.21.15.104:0/775442757' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:52:53.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:52 smithi055 bash[9296]: audit 2020-09-08T15:52:52.055374+0000 mon.a (mon.0) 1571 : audit [INF] from='client.35696 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:52:53.178 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:52:53.179 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:53.218 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:52:53.248 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:52 smithi104 bash[9203]: cluster 2020-09-08T15:52:51.571613+0000 osd.4 (osd.4) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:53.248 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:52 smithi104 bash[9203]: cluster 2020-09-08T15:52:51.571686+0000 osd.4 (osd.4) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:53.249 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:52 smithi104 bash[9203]: cluster 2020-09-08T15:52:51.832906+0000 mgr.x (mgr.24118) 766 : cluster [DBG] pgmap v1104: 17 pgs: 17 active+clean; 6.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 MiB/s wr, 1.05k op/s 2020-09-08T15:52:53.249 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:52 smithi104 bash[9203]: cluster 2020-09-08T15:52:51.987701+0000 mon.a (mon.0) 1570 : cluster [DBG] osdmap e423: 12 total, 12 up, 12 in 2020-09-08T15:52:53.249 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:52 smithi104 bash[9203]: audit 2020-09-08T15:52:52.052700+0000 mon.c (mon.2) 62 : audit [INF] from='client.? v1:172.21.15.104:0/775442757' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:52:53.249 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:52 smithi104 bash[9203]: audit 2020-09-08T15:52:52.055374+0000 mon.a (mon.0) 1571 : audit [INF] from='client.35696 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:52:53.251 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2020-09-08T15:52:53.254 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:52:53.288 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:53.295 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:52:53.336 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:52:53.363 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 15:52:53.367991 min lat: 0.00169895 max lat: 0.433439 avg lat: 0.00856422 2020-09-08T15:52:53.364 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:52:53.364 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 60 16 112087 112071 116.731 77.5625 0.0170009 0.00856422 2020-09-08T15:52:53.366 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:52:53 smithi166 bash[16593]: debug 2020-09-08T15:52:53.073+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:53.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:52 smithi166 bash[9446]: cluster 2020-09-08T15:52:51.571613+0000 osd.4 (osd.4) 43 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:52:53.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:52 smithi166 bash[9446]: cluster 2020-09-08T15:52:51.571686+0000 osd.4 (osd.4) 44 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:52:53.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:52 smithi166 bash[9446]: cluster 2020-09-08T15:52:51.832906+0000 mgr.x (mgr.24118) 766 : cluster [DBG] pgmap v1104: 17 pgs: 17 active+clean; 6.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 MiB/s wr, 1.05k op/s 2020-09-08T15:52:53.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:52 smithi166 bash[9446]: cluster 2020-09-08T15:52:51.987701+0000 mon.a (mon.0) 1570 : cluster [DBG] osdmap e423: 12 total, 12 up, 12 in 2020-09-08T15:52:53.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:52 smithi166 bash[9446]: audit 2020-09-08T15:52:52.052700+0000 mon.c (mon.2) 62 : audit [INF] from='client.? v1:172.21.15.104:0/775442757' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:52:53.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:52 smithi166 bash[9446]: audit 2020-09-08T15:52:52.055374+0000 mon.a (mon.0) 1571 : audit [INF] from='client.35696 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:52:53.373 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:52:53.392 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:53.414 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:52:53.420 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:52:53.451 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:52:53.485 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:52:53.514 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:52:53 smithi055 bash[16646]: debug 2020-09-08T15:52:53.288+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:53.515 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:53.531 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:52:53.552 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:52:53.582 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:52:53.585 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:52:53.586 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:52:53.586 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:52:53.586 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:52:53.586 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:52:53.587 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:52:53.587 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:52:53.587 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:52:53.587 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:52:53.587 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:52:53.587 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:52:53.588 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:52:53.632 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:53.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:52:53 smithi055 bash[11956]: debug 2020-09-08T15:52:53.516+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:54.196 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:54.299 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:54.308 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:53 smithi166 bash[9446]: cluster 2020-09-08T15:52:52.983238+0000 mon.a (mon.0) 1572 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:52:54.308 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:53 smithi166 bash[9446]: audit 2020-09-08T15:52:52.994284+0000 mon.a (mon.0) 1573 : audit [INF] from='client.35696 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:52:54.308 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:53 smithi166 bash[9446]: cluster 2020-09-08T15:52:52.994335+0000 mon.a (mon.0) 1574 : cluster [DBG] osdmap e424: 12 total, 12 up, 12 in 2020-09-08T15:52:54.309 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:53 smithi166 bash[9446]: audit 2020-09-08T15:52:53.124643+0000 mon.a (mon.0) 1575 : audit [DBG] from='client.? v1:172.21.15.104:0/2975877934' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:52:54.363 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 61 16 113263 113247 116.022 73.5 0.0164787 0.00861705 2020-09-08T15:52:54.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:53 smithi055 bash[9296]: cluster 2020-09-08T15:52:52.983238+0000 mon.a (mon.0) 1572 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:52:54.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:53 smithi055 bash[9296]: audit 2020-09-08T15:52:52.994284+0000 mon.a (mon.0) 1573 : audit [INF] from='client.35696 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:52:54.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:53 smithi055 bash[9296]: cluster 2020-09-08T15:52:52.994335+0000 mon.a (mon.0) 1574 : cluster [DBG] osdmap e424: 12 total, 12 up, 12 in 2020-09-08T15:52:54.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:53 smithi055 bash[9296]: audit 2020-09-08T15:52:53.124643+0000 mon.a (mon.0) 1575 : audit [DBG] from='client.? v1:172.21.15.104:0/2975877934' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:52:54.481 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:53 smithi104 bash[9203]: cluster 2020-09-08T15:52:52.983238+0000 mon.a (mon.0) 1572 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:52:54.482 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:53 smithi104 bash[9203]: audit 2020-09-08T15:52:52.994284+0000 mon.a (mon.0) 1573 : audit [INF] from='client.35696 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:52:54.482 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:53 smithi104 bash[9203]: cluster 2020-09-08T15:52:52.994335+0000 mon.a (mon.0) 1574 : cluster [DBG] osdmap e424: 12 total, 12 up, 12 in 2020-09-08T15:52:54.482 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:53 smithi104 bash[9203]: audit 2020-09-08T15:52:53.124643+0000 mon.a (mon.0) 1575 : audit [DBG] from='client.? v1:172.21.15.104:0/2975877934' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:52:54.483 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:52:54 smithi104 bash[19922]: debug 2020-09-08T15:52:54.197+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:54.760 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:52:54.864 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:52:54.997 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:52:55.101 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:52:55.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:54 smithi166 bash[9446]: cluster 2020-09-08T15:52:53.833455+0000 mgr.x (mgr.24118) 767 : cluster [DBG] pgmap v1107: 17 pgs: 17 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T15:52:55.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:52:54 smithi166 bash[16593]: debug 2020-09-08T15:52:54.761+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:55.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:54 smithi055 bash[9296]: cluster 2020-09-08T15:52:53.833455+0000 mgr.x (mgr.24118) 767 : cluster [DBG] pgmap v1107: 17 pgs: 17 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T15:52:55.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:52:55 smithi055 bash[14301]: debug 2020-09-08T15:52:55.000+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:55.174 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:54 smithi104 bash[9203]: cluster 2020-09-08T15:52:53.833455+0000 mgr.x (mgr.24118) 767 : cluster [DBG] pgmap v1107: 17 pgs: 17 active+clean; 6.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T15:52:55.219 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:52:55.323 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:52:55.364 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 62 16 114519 114503 115.416 78.5 0.00677752 0.00866235 2020-09-08T15:52:55.440 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:52:55.545 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:55.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:52:55 smithi055 bash[19003]: debug 2020-09-08T15:52:55.220+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:55.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:52:55 smithi055 bash[19003]: debug 2020-09-08T15:52:55.440+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:55.694 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:55.796 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:55.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:52:55 smithi104 bash[19922]: debug 2020-09-08T15:52:55.693+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:55.930 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:56.034 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:52:56.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:52:55 smithi055 bash[16646]: debug 2020-09-08T15:52:55.932+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:56.364 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 63 16 115800 115784 114.855 80.0625 0.0102033 0.00870493 2020-09-08T15:52:56.681 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:52:56.786 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:56.914 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:52:56.996 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:57 smithi104 bash[9203]: cluster 2020-09-08T15:52:55.837746+0000 mgr.x (mgr.24118) 768 : cluster [DBG] pgmap v1108: 17 pgs: 17 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s wr, 1.05k op/s 2020-09-08T15:52:56.996 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:52:56 smithi104 bash[15273]: debug 2020-09-08T15:52:56.677+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:57.081 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:52:57.177 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:52:57.346 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:52:57.364 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 64 16 117067 117051 114.298 79.1875 0.0114641 0.00874703 2020-09-08T15:52:57.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:57 smithi166 bash[9446]: cluster 2020-09-08T15:52:55.837746+0000 mgr.x (mgr.24118) 768 : cluster [DBG] pgmap v1108: 17 pgs: 17 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s wr, 1.05k op/s 2020-09-08T15:52:57.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:57 smithi055 bash[9296]: cluster 2020-09-08T15:52:55.837746+0000 mgr.x (mgr.24118) 768 : cluster [DBG] pgmap v1108: 17 pgs: 17 active+clean; 6.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s wr, 1.05k op/s 2020-09-08T15:52:57.486 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:52:57.595 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:52:57.744 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:52:57.847 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:52:57.863 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:52:57 smithi104 bash[19922]: debug 2020-09-08T15:52:57.485+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:57.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:52:57 smithi166 bash[14254]: debug 2020-09-08T15:52:57.741+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:58.236 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:52:58.340 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:52:58.364 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 65 16 118291 118275 113.716 76.5 0.0171713 0.00879173 2020-09-08T15:52:58.502 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:52:58.603 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:52:58.656 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:52:58 smithi055 bash[14301]: debug 2020-09-08T15:52:58.492+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:58.680 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:52:58 smithi104 bash[17588]: debug 2020-09-08T15:52:58.237+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:58.745 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:52:58.850 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:52:58.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:52:58 smithi166 bash[11928]: debug 2020-09-08T15:52:58.745+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:59.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:52:59 smithi104 bash[9203]: cluster 2020-09-08T15:52:57.838288+0000 mgr.x (mgr.24118) 769 : cluster [DBG] pgmap v1109: 17 pgs: 17 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 89 MiB/s wr, 1.43k op/s 2020-09-08T15:52:59.184 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:52:59.288 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:59.297 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:52:59 smithi055 bash[9296]: cluster 2020-09-08T15:52:57.838288+0000 mgr.x (mgr.24118) 769 : cluster [DBG] pgmap v1109: 17 pgs: 17 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 89 MiB/s wr, 1.43k op/s 2020-09-08T15:52:59.364 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 66 16 119260 119244 112.911 60.5625 0.00889726 0.00885464 2020-09-08T15:52:59.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:52:59 smithi166 bash[9446]: cluster 2020-09-08T15:52:57.838288+0000 mgr.x (mgr.24118) 769 : cluster [DBG] pgmap v1109: 17 pgs: 17 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 89 MiB/s wr, 1.43k op/s 2020-09-08T15:52:59.400 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:52:59.504 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:52:59.609 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:52:59.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:52:59 smithi055 bash[11956]: debug 2020-09-08T15:52:59.400+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:59.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:52:59 smithi104 bash[15273]: debug 2020-09-08T15:52:59.185+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:59.714 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:59.835 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:52:59.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:52:59 smithi166 bash[18960]: debug 2020-09-08T15:52:59.605+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:52:59.940 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:53:00.093 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:53:00.093 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T15:53:00.094 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2020-09-08T15:53:00.094 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2020-09-08T15:53:00.094 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:53:00.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:52:59 smithi055 bash[16646]: debug 2020-09-08T15:52:59.836+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:00.364 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 67 16 120081 120065 111.991 51.3125 0.00644538 0.00888803 2020-09-08T15:53:01.108 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:00 smithi104 bash[9203]: cluster 2020-09-08T15:52:59.838770+0000 mgr.x (mgr.24118) 770 : cluster [DBG] pgmap v1110: 17 pgs: 17 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.47k op/s 2020-09-08T15:53:01.108 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:00 smithi104 bash[9203]: cluster 2020-09-08T15:53:00.626067+0000 osd.9 (osd.9) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:01.109 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:00 smithi104 bash[9203]: cluster 2020-09-08T15:53:00.626241+0000 osd.9 (osd.9) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:01.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:00 smithi166 bash[9446]: cluster 2020-09-08T15:52:59.838770+0000 mgr.x (mgr.24118) 770 : cluster [DBG] pgmap v1110: 17 pgs: 17 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.47k op/s 2020-09-08T15:53:01.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:00 smithi166 bash[9446]: cluster 2020-09-08T15:53:00.626067+0000 osd.9 (osd.9) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:01.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:00 smithi166 bash[9446]: cluster 2020-09-08T15:53:00.626241+0000 osd.9 (osd.9) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:01.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:00 smithi055 bash[9296]: cluster 2020-09-08T15:52:59.838770+0000 mgr.x (mgr.24118) 770 : cluster [DBG] pgmap v1110: 17 pgs: 17 active+clean; 7.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.47k op/s 2020-09-08T15:53:01.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:00 smithi055 bash[9296]: cluster 2020-09-08T15:53:00.626067+0000 osd.9 (osd.9) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:01.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:00 smithi055 bash[9296]: cluster 2020-09-08T15:53:00.626241+0000 osd.9 (osd.9) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:01.361 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2020-09-08T15:53:01.364 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 68 16 120976 120960 111.167 55.9375 0.00782223 0.00899305 2020-09-08T15:53:01.520 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:53:02.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:01 smithi166 bash[9446]: cluster 2020-09-08T15:53:00.809374+0000 mon.a (mon.0) 1576 : cluster [DBG] osdmap e425: 12 total, 12 up, 12 in 2020-09-08T15:53:02.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:01 smithi055 bash[9296]: cluster 2020-09-08T15:53:00.809374+0000 mon.a (mon.0) 1576 : cluster [DBG] osdmap e425: 12 total, 12 up, 12 in 2020-09-08T15:53:02.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:01 smithi104 bash[9203]: cluster 2020-09-08T15:53:00.809374+0000 mon.a (mon.0) 1576 : cluster [DBG] osdmap e425: 12 total, 12 up, 12 in 2020-09-08T15:53:02.364 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 69 16 122178 122162 110.644 75.125 0.0121145 0.00903599 2020-09-08T15:53:02.660 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:53:02.764 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:03.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:02 smithi166 bash[9446]: cluster 2020-09-08T15:53:01.839159+0000 mgr.x (mgr.24118) 771 : cluster [DBG] pgmap v1112: 17 pgs: 17 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s wr, 908 op/s 2020-09-08T15:53:03.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:02 smithi166 bash[9446]: cluster 2020-09-08T15:53:02.317938+0000 osd.2 (osd.2) 54 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:03.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:02 smithi166 bash[9446]: cluster 2020-09-08T15:53:02.318028+0000 osd.2 (osd.2) 55 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:03.118 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:53:02 smithi166 bash[18960]: debug 2020-09-08T15:53:02.657+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:03.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:02 smithi055 bash[9296]: cluster 2020-09-08T15:53:01.839159+0000 mgr.x (mgr.24118) 771 : cluster [DBG] pgmap v1112: 17 pgs: 17 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s wr, 908 op/s 2020-09-08T15:53:03.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:02 smithi055 bash[9296]: cluster 2020-09-08T15:53:02.317938+0000 osd.2 (osd.2) 54 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:03.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:02 smithi055 bash[9296]: cluster 2020-09-08T15:53:02.318028+0000 osd.2 (osd.2) 55 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:03.172 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:03.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:02 smithi104 bash[9203]: cluster 2020-09-08T15:53:01.839159+0000 mgr.x (mgr.24118) 771 : cluster [DBG] pgmap v1112: 17 pgs: 17 active+clean; 7.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s wr, 908 op/s 2020-09-08T15:53:03.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:02 smithi104 bash[9203]: cluster 2020-09-08T15:53:02.317938+0000 osd.2 (osd.2) 54 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:03.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:02 smithi104 bash[9203]: cluster 2020-09-08T15:53:02.318028+0000 osd.2 (osd.2) 55 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:03.276 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:03.364 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 70 16 123424 123408 110.176 77.875 0.00647351 0.00907439 2020-09-08T15:53:03.657 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:53:03 smithi055 bash[11956]: debug 2020-09-08T15:53:03.168+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:03.743 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:03.846 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:03.966 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:04.077 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:53:04.089 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:53:03 smithi055 bash[19003]: debug 2020-09-08T15:53:03.968+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:04.089 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:03 smithi055 bash[9296]: cluster 2020-09-08T15:53:02.819431+0000 mon.a (mon.0) 1577 : cluster [DBG] osdmap e426: 12 total, 12 up, 12 in 2020-09-08T15:53:04.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:03 smithi166 bash[9446]: cluster 2020-09-08T15:53:02.819431+0000 mon.a (mon.0) 1577 : cluster [DBG] osdmap e426: 12 total, 12 up, 12 in 2020-09-08T15:53:04.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:53:03 smithi104 bash[19922]: debug 2020-09-08T15:53:03.745+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:04.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:03 smithi104 bash[9203]: cluster 2020-09-08T15:53:02.819431+0000 mon.a (mon.0) 1577 : cluster [DBG] osdmap e426: 12 total, 12 up, 12 in 2020-09-08T15:53:04.193 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:53:04.302 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:04.370 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 71 16 124510 124494 109.58 67.875 0.0161239 0.00912385 2020-09-08T15:53:04.412 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:53:04 smithi055 bash[14301]: debug 2020-09-08T15:53:04.196+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:04.436 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:04.539 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:53:04.666 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:53:04.770 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:04.781 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:53:04 smithi055 bash[11956]: debug 2020-09-08T15:53:04.436+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:04.818 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:53:04 smithi166 bash[11928]: debug 2020-09-08T15:53:04.660+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:04.888 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:04.995 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:53:05.109 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:04 smithi166 bash[9446]: cluster 2020-09-08T15:53:03.839647+0000 mgr.x (mgr.24118) 772 : cluster [DBG] pgmap v1114: 17 pgs: 17 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s wr, 1.37k op/s 2020-09-08T15:53:05.112 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:53:05.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:04 smithi055 bash[9296]: cluster 2020-09-08T15:53:03.839647+0000 mgr.x (mgr.24118) 772 : cluster [DBG] pgmap v1114: 17 pgs: 17 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s wr, 1.37k op/s 2020-09-08T15:53:05.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:53:04 smithi055 bash[11956]: debug 2020-09-08T15:53:04.888+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:05.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:04 smithi104 bash[9203]: cluster 2020-09-08T15:53:03.839647+0000 mgr.x (mgr.24118) 772 : cluster [DBG] pgmap v1114: 17 pgs: 17 active+clean; 7.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s wr, 1.37k op/s 2020-09-08T15:53:05.216 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:53:05.343 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:53:05.364 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 72 16 125694 125678 109.086 74 0.00663699 0.00916471 2020-09-08T15:53:05.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:53:05 smithi166 bash[11928]: debug 2020-09-08T15:53:05.104+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:05.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:53:05 smithi166 bash[11928]: debug 2020-09-08T15:53:05.340+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:05.447 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:05.564 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:05.653 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:53:05.666 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:53:05.691 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:53:05.734 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:53:05.770 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:53:05.775 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:53:05.805 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:53:05.820 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:53:05 smithi055 bash[19003]: debug 2020-09-08T15:53:05.564+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:05.821 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:05 smithi055 bash[9296]: audit 2020-09-08T15:53:05.620774+0000 mon.a (mon.0) 1578 : audit [DBG] from='client.? v1:172.21.15.104:0/2004014751' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:53:05.848 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:53:05.873 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:53:05.889 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:53:05.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:05 smithi104 bash[9203]: audit 2020-09-08T15:53:05.620774+0000 mon.a (mon.0) 1578 : audit [DBG] from='client.? v1:172.21.15.104:0/2004014751' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:53:05.931 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:53:05.967 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:53:05.990 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:53:06.006 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:53:06.054 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:53:06.093 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:53:06.098 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:06.107 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:53:06.108 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:53:06.108 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:53:06.108 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:53:06.109 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:53:06.109 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:53:06.109 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:53:06.109 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:53:06.109 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:53:06.113 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:53:06.114 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:53:06.114 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:53:06.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:05 smithi166 bash[9446]: audit 2020-09-08T15:53:05.620774+0000 mon.a (mon.0) 1578 : audit [DBG] from='client.? v1:172.21.15.104:0/2004014751' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:53:06.117 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:53:05 smithi166 bash[14254]: debug 2020-09-08T15:53:05.768+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:06.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:53:05 smithi055 bash[16646]: debug 2020-09-08T15:53:05.992+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:06.365 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 73 16 126771 126755 108.514 67.3125 0.0150149 0.00921298 2020-09-08T15:53:06.688 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:06.792 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:53:06.835 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:53:06.916 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:53:06.942 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:06 smithi104 bash[9203]: cluster 2020-09-08T15:53:05.840144+0000 mgr.x (mgr.24118) 773 : cluster [DBG] pgmap v1115: 17 pgs: 17 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s wr, 783 op/s 2020-09-08T15:53:06.942 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:06 smithi104 bash[9203]: audit 2020-09-08T15:53:06.409711+0000 mon.c (mon.2) 63 : audit [INF] from='client.? v1:172.21.15.104:0/2840815127' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:53:06.942 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:06 smithi104 bash[9203]: audit 2020-09-08T15:53:06.412397+0000 mon.a (mon.0) 1579 : audit [INF] from='client.35714 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:53:06.942 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:53:06 smithi104 bash[17588]: debug 2020-09-08T15:53:06.685+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:07.023 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:07.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:06 smithi166 bash[9446]: cluster 2020-09-08T15:53:05.840144+0000 mgr.x (mgr.24118) 773 : cluster [DBG] pgmap v1115: 17 pgs: 17 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s wr, 783 op/s 2020-09-08T15:53:07.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:06 smithi166 bash[9446]: audit 2020-09-08T15:53:06.409711+0000 mon.c (mon.2) 63 : audit [INF] from='client.? v1:172.21.15.104:0/2840815127' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:53:07.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:06 smithi166 bash[9446]: audit 2020-09-08T15:53:06.412397+0000 mon.a (mon.0) 1579 : audit [INF] from='client.35714 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:53:07.155 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:53:07 smithi055 bash[11956]: debug 2020-09-08T15:53:07.148+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:07.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:06 smithi055 bash[9296]: cluster 2020-09-08T15:53:05.840144+0000 mgr.x (mgr.24118) 773 : cluster [DBG] pgmap v1115: 17 pgs: 17 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s wr, 783 op/s 2020-09-08T15:53:07.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:06 smithi055 bash[9296]: audit 2020-09-08T15:53:06.409711+0000 mon.c (mon.2) 63 : audit [INF] from='client.? v1:172.21.15.104:0/2840815127' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:53:07.162 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:06 smithi055 bash[9296]: audit 2020-09-08T15:53:06.412397+0000 mon.a (mon.0) 1579 : audit [INF] from='client.35714 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:53:07.162 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:07.162 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:53:06 smithi055 bash[14301]: debug 2020-09-08T15:53:06.916+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:07.263 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:07.366 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 74 16 127938 127922 108.033 72.9375 0.00886675 0.00925398 2020-09-08T15:53:07.376 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:07.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:53:07 smithi055 bash[19003]: debug 2020-09-08T15:53:07.376+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:07.480 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:07.830 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:07 smithi055 bash[9296]: cluster 2020-09-08T15:53:06.826357+0000 mon.a (mon.0) 1580 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:53:07.831 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:07 smithi055 bash[9296]: audit 2020-09-08T15:53:06.838008+0000 mon.a (mon.0) 1581 : audit [INF] from='client.35714 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:53:07.831 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:07 smithi055 bash[9296]: cluster 2020-09-08T15:53:06.838036+0000 mon.a (mon.0) 1582 : cluster [DBG] osdmap e427: 12 total, 12 up, 12 in 2020-09-08T15:53:07.832 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:07 smithi055 bash[9296]: cluster 2020-09-08T15:53:06.944124+0000 osd.6 (osd.6) 67 : cluster [DBG] 5.2 scrub starts 2020-09-08T15:53:07.832 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:07 smithi055 bash[9296]: cluster 2020-09-08T15:53:07.238125+0000 osd.5 (osd.5) 53 : cluster [DBG] 5.f scrub starts 2020-09-08T15:53:08.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:07 smithi166 bash[9446]: cluster 2020-09-08T15:53:06.826357+0000 mon.a (mon.0) 1580 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:53:08.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:07 smithi166 bash[9446]: audit 2020-09-08T15:53:06.838008+0000 mon.a (mon.0) 1581 : audit [INF] from='client.35714 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:53:08.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:07 smithi166 bash[9446]: cluster 2020-09-08T15:53:06.838036+0000 mon.a (mon.0) 1582 : cluster [DBG] osdmap e427: 12 total, 12 up, 12 in 2020-09-08T15:53:08.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:07 smithi166 bash[9446]: cluster 2020-09-08T15:53:06.944124+0000 osd.6 (osd.6) 67 : cluster [DBG] 5.2 scrub starts 2020-09-08T15:53:08.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:07 smithi166 bash[9446]: cluster 2020-09-08T15:53:07.238125+0000 osd.5 (osd.5) 53 : cluster [DBG] 5.f scrub starts 2020-09-08T15:53:08.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:07 smithi104 bash[9203]: cluster 2020-09-08T15:53:06.826357+0000 mon.a (mon.0) 1580 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:53:08.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:07 smithi104 bash[9203]: audit 2020-09-08T15:53:06.838008+0000 mon.a (mon.0) 1581 : audit [INF] from='client.35714 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:53:08.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:07 smithi104 bash[9203]: cluster 2020-09-08T15:53:06.838036+0000 mon.a (mon.0) 1582 : cluster [DBG] osdmap e427: 12 total, 12 up, 12 in 2020-09-08T15:53:08.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:07 smithi104 bash[9203]: cluster 2020-09-08T15:53:06.944124+0000 osd.6 (osd.6) 67 : cluster [DBG] 5.2 scrub starts 2020-09-08T15:53:08.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:07 smithi104 bash[9203]: cluster 2020-09-08T15:53:07.238125+0000 osd.5 (osd.5) 53 : cluster [DBG] 5.f scrub starts 2020-09-08T15:53:08.365 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 75 16 128993 128977 107.471 65.9375 0.00628476 0.00930134 2020-09-08T15:53:09.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:08 smithi166 bash[9446]: cluster 2020-09-08T15:53:07.701974+0000 osd.10 (osd.10) 45 : cluster [DBG] 5.0 scrub starts 2020-09-08T15:53:09.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:08 smithi166 bash[9446]: cluster 2020-09-08T15:53:07.840658+0000 mgr.x (mgr.24118) 774 : cluster [DBG] pgmap v1117: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s wr, 1.39k op/s 2020-09-08T15:53:09.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:08 smithi055 bash[9296]: cluster 2020-09-08T15:53:07.701974+0000 osd.10 (osd.10) 45 : cluster [DBG] 5.0 scrub starts 2020-09-08T15:53:09.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:08 smithi055 bash[9296]: cluster 2020-09-08T15:53:07.840658+0000 mgr.x (mgr.24118) 774 : cluster [DBG] pgmap v1117: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s wr, 1.39k op/s 2020-09-08T15:53:09.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:08 smithi104 bash[9203]: cluster 2020-09-08T15:53:07.701974+0000 osd.10 (osd.10) 45 : cluster [DBG] 5.0 scrub starts 2020-09-08T15:53:09.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:08 smithi104 bash[9203]: cluster 2020-09-08T15:53:07.840658+0000 mgr.x (mgr.24118) 774 : cluster [DBG] pgmap v1117: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s wr, 1.39k op/s 2020-09-08T15:53:09.365 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 76 16 130098 130082 106.966 69.0625 0.0381111 0.0093462 2020-09-08T15:53:09.515 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:53:09.515 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:53:10.365 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 77 16 131116 131100 106.403 63.625 0.0197373 0.00939627 2020-09-08T15:53:10.692 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:53:10.831 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2020-09-08T15:53:10.862 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:53:10.900 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:53:11.014 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:53:11.085 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:10 smithi055 bash[9296]: cluster 2020-09-08T15:53:09.690027+0000 osd.6 (osd.6) 68 : cluster [DBG] 5.2 scrub ok 2020-09-08T15:53:11.086 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:10 smithi055 bash[9296]: cluster 2020-09-08T15:53:09.841194+0000 mgr.x (mgr.24118) 775 : cluster [DBG] pgmap v1118: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s wr, 1.28k op/s 2020-09-08T15:53:11.086 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:10 smithi055 bash[9296]: cluster 2020-09-08T15:53:09.847500+0000 osd.5 (osd.5) 54 : cluster [DBG] 5.f scrub ok 2020-09-08T15:53:11.086 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:10 smithi055 bash[9296]: cluster 2020-09-08T15:53:10.111754+0000 osd.7 (osd.7) 79 : cluster [DBG] 5.9 scrub starts 2020-09-08T15:53:11.086 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:10 smithi055 bash[9296]: cluster 2020-09-08T15:53:10.272271+0000 osd.5 (osd.5) 55 : cluster [DBG] 5.7 scrub starts 2020-09-08T15:53:11.087 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:11.094 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:53:11.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:10 smithi166 bash[9446]: cluster 2020-09-08T15:53:09.690027+0000 osd.6 (osd.6) 68 : cluster [DBG] 5.2 scrub ok 2020-09-08T15:53:11.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:10 smithi166 bash[9446]: cluster 2020-09-08T15:53:09.841194+0000 mgr.x (mgr.24118) 775 : cluster [DBG] pgmap v1118: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s wr, 1.28k op/s 2020-09-08T15:53:11.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:10 smithi166 bash[9446]: cluster 2020-09-08T15:53:09.847500+0000 osd.5 (osd.5) 54 : cluster [DBG] 5.f scrub ok 2020-09-08T15:53:11.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:10 smithi166 bash[9446]: cluster 2020-09-08T15:53:10.111754+0000 osd.7 (osd.7) 79 : cluster [DBG] 5.9 scrub starts 2020-09-08T15:53:11.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:10 smithi166 bash[9446]: cluster 2020-09-08T15:53:10.272271+0000 osd.5 (osd.5) 55 : cluster [DBG] 5.7 scrub starts 2020-09-08T15:53:11.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:10 smithi104 bash[9203]: cluster 2020-09-08T15:53:09.690027+0000 osd.6 (osd.6) 68 : cluster [DBG] 5.2 scrub ok 2020-09-08T15:53:11.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:10 smithi104 bash[9203]: cluster 2020-09-08T15:53:09.841194+0000 mgr.x (mgr.24118) 775 : cluster [DBG] pgmap v1118: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s wr, 1.28k op/s 2020-09-08T15:53:11.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:10 smithi104 bash[9203]: cluster 2020-09-08T15:53:09.847500+0000 osd.5 (osd.5) 54 : cluster [DBG] 5.f scrub ok 2020-09-08T15:53:11.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:10 smithi104 bash[9203]: cluster 2020-09-08T15:53:10.111754+0000 osd.7 (osd.7) 79 : cluster [DBG] 5.9 scrub starts 2020-09-08T15:53:11.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:10 smithi104 bash[9203]: cluster 2020-09-08T15:53:10.272271+0000 osd.5 (osd.5) 55 : cluster [DBG] 5.7 scrub starts 2020-09-08T15:53:11.188 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:11.365 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 78 16 132176 132160 105.888 66.25 0.0117198 0.00944171 2020-09-08T15:53:11.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:53:11 smithi055 bash[19003]: debug 2020-09-08T15:53:11.088+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:11.845 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:11.948 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:12.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:11 smithi166 bash[9446]: cluster 2020-09-08T15:53:10.668606+0000 osd.9 (osd.9) 61 : cluster [DBG] 5.b scrub starts 2020-09-08T15:53:12.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:11 smithi055 bash[9296]: cluster 2020-09-08T15:53:10.668606+0000 osd.9 (osd.9) 61 : cluster [DBG] 5.b scrub starts 2020-09-08T15:53:12.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:11 smithi104 bash[9203]: cluster 2020-09-08T15:53:10.668606+0000 osd.9 (osd.9) 61 : cluster [DBG] 5.b scrub starts 2020-09-08T15:53:12.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:53:11 smithi104 bash[15273]: debug 2020-09-08T15:53:11.841+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:12.365 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 79 16 133293 133277 105.431 69.8125 0.0082246 0.00948255 2020-09-08T15:53:12.710 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:12.814 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:13.050 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:53:13 smithi104 bash[19922]: debug 2020-09-08T15:53:13.049+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:13.051 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:12 smithi104 bash[9203]: cluster 2020-09-08T15:53:11.158776+0000 osd.7 (osd.7) 80 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:13.051 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:12 smithi104 bash[9203]: cluster 2020-09-08T15:53:11.158853+0000 osd.7 (osd.7) 81 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:13.051 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:12 smithi104 bash[9203]: cluster 2020-09-08T15:53:11.811148+0000 mon.a (mon.0) 1583 : cluster [DBG] osdmap e428: 12 total, 12 up, 12 in 2020-09-08T15:53:13.052 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:12 smithi104 bash[9203]: cluster 2020-09-08T15:53:11.841584+0000 mgr.x (mgr.24118) 776 : cluster [DBG] pgmap v1120: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 53 MiB/s wr, 847 op/s 2020-09-08T15:53:13.052 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:53:12 smithi104 bash[15273]: debug 2020-09-08T15:53:12.709+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:13.053 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:13.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:12 smithi166 bash[9446]: cluster 2020-09-08T15:53:11.158776+0000 osd.7 (osd.7) 80 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:13.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:12 smithi166 bash[9446]: cluster 2020-09-08T15:53:11.158853+0000 osd.7 (osd.7) 81 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:13.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:12 smithi166 bash[9446]: cluster 2020-09-08T15:53:11.811148+0000 mon.a (mon.0) 1583 : cluster [DBG] osdmap e428: 12 total, 12 up, 12 in 2020-09-08T15:53:13.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:12 smithi166 bash[9446]: cluster 2020-09-08T15:53:11.841584+0000 mgr.x (mgr.24118) 776 : cluster [DBG] pgmap v1120: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 53 MiB/s wr, 847 op/s 2020-09-08T15:53:13.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:12 smithi055 bash[9296]: cluster 2020-09-08T15:53:11.158776+0000 osd.7 (osd.7) 80 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:13.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:12 smithi055 bash[9296]: cluster 2020-09-08T15:53:11.158853+0000 osd.7 (osd.7) 81 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:13.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:12 smithi055 bash[9296]: cluster 2020-09-08T15:53:11.811148+0000 mon.a (mon.0) 1583 : cluster [DBG] osdmap e428: 12 total, 12 up, 12 in 2020-09-08T15:53:13.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:12 smithi055 bash[9296]: cluster 2020-09-08T15:53:11.841584+0000 mgr.x (mgr.24118) 776 : cluster [DBG] pgmap v1120: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 53 MiB/s wr, 847 op/s 2020-09-08T15:53:13.159 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:13.276 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:13.365 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 15:53:13.369792 min lat: 0.00169895 max lat: 0.606193 avg lat: 0.00952383 2020-09-08T15:53:13.366 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:53:13.366 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 80 16 134400 134384 104.978 69.1875 0.00683729 0.00952383 2020-09-08T15:53:13.380 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:13.566 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:13.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:53:13 smithi055 bash[11956]: debug 2020-09-08T15:53:13.276+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:13.667 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:53:13 smithi104 bash[17588]: debug 2020-09-08T15:53:13.565+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:13.669 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:13.796 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:13.899 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:13.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:53:13 smithi104 bash[22305]: debug 2020-09-08T15:53:13.789+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:13.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:13 smithi104 bash[9203]: cluster 2020-09-08T15:53:12.227747+0000 osd.7 (osd.7) 82 : cluster [DBG] 5.9 scrub ok 2020-09-08T15:53:14.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:13 smithi166 bash[9446]: cluster 2020-09-08T15:53:12.227747+0000 osd.7 (osd.7) 82 : cluster [DBG] 5.9 scrub ok 2020-09-08T15:53:14.155 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:14.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:13 smithi055 bash[9296]: cluster 2020-09-08T15:53:12.227747+0000 osd.7 (osd.7) 82 : cluster [DBG] 5.9 scrub ok 2020-09-08T15:53:14.259 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:14.376 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 81 16 135456 135440 104.497 66 0.0105371 0.00956759 2020-09-08T15:53:14.393 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:14.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:53:14 smithi104 bash[19922]: debug 2020-09-08T15:53:14.153+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:14.495 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:14.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:53:14 smithi055 bash[19003]: debug 2020-09-08T15:53:14.396+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:14.833 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:14 smithi104 bash[9203]: cluster 2020-09-08T15:53:13.346318+0000 osd.0 (osd.0) 53 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:14.834 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:14 smithi104 bash[9203]: cluster 2020-09-08T15:53:13.346389+0000 osd.0 (osd.0) 54 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:14.834 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:14 smithi104 bash[9203]: cluster 2020-09-08T15:53:13.642752+0000 osd.5 (osd.5) 56 : cluster [DBG] 5.7 scrub ok 2020-09-08T15:53:14.834 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:14 smithi104 bash[9203]: cluster 2020-09-08T15:53:13.823837+0000 mon.a (mon.0) 1584 : cluster [DBG] osdmap e429: 12 total, 12 up, 12 in 2020-09-08T15:53:14.834 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:14 smithi104 bash[9203]: cluster 2020-09-08T15:53:13.842059+0000 mgr.x (mgr.24118) 777 : cluster [DBG] pgmap v1122: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 89 MiB/s wr, 1.43k op/s 2020-09-08T15:53:14.835 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:14 smithi104 bash[9203]: cluster 2020-09-08T15:53:13.852362+0000 osd.3 (osd.3) 59 : cluster [DBG] 5.1 scrub starts 2020-09-08T15:53:14.835 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:14 smithi104 bash[9203]: cluster 2020-09-08T15:53:13.895630+0000 osd.9 (osd.9) 62 : cluster [DBG] 5.b scrub ok 2020-09-08T15:53:15.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:14 smithi166 bash[9446]: cluster 2020-09-08T15:53:13.346318+0000 osd.0 (osd.0) 53 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:15.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:14 smithi166 bash[9446]: cluster 2020-09-08T15:53:13.346389+0000 osd.0 (osd.0) 54 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:15.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:14 smithi166 bash[9446]: cluster 2020-09-08T15:53:13.642752+0000 osd.5 (osd.5) 56 : cluster [DBG] 5.7 scrub ok 2020-09-08T15:53:15.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:14 smithi166 bash[9446]: cluster 2020-09-08T15:53:13.823837+0000 mon.a (mon.0) 1584 : cluster [DBG] osdmap e429: 12 total, 12 up, 12 in 2020-09-08T15:53:15.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:14 smithi166 bash[9446]: cluster 2020-09-08T15:53:13.842059+0000 mgr.x (mgr.24118) 777 : cluster [DBG] pgmap v1122: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 89 MiB/s wr, 1.43k op/s 2020-09-08T15:53:15.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:14 smithi166 bash[9446]: cluster 2020-09-08T15:53:13.852362+0000 osd.3 (osd.3) 59 : cluster [DBG] 5.1 scrub starts 2020-09-08T15:53:15.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:14 smithi166 bash[9446]: cluster 2020-09-08T15:53:13.895630+0000 osd.9 (osd.9) 62 : cluster [DBG] 5.b scrub ok 2020-09-08T15:53:15.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:14 smithi055 bash[9296]: cluster 2020-09-08T15:53:13.346318+0000 osd.0 (osd.0) 53 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:15.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:14 smithi055 bash[9296]: cluster 2020-09-08T15:53:13.346389+0000 osd.0 (osd.0) 54 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:15.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:14 smithi055 bash[9296]: cluster 2020-09-08T15:53:13.642752+0000 osd.5 (osd.5) 56 : cluster [DBG] 5.7 scrub ok 2020-09-08T15:53:15.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:14 smithi055 bash[9296]: cluster 2020-09-08T15:53:13.823837+0000 mon.a (mon.0) 1584 : cluster [DBG] osdmap e429: 12 total, 12 up, 12 in 2020-09-08T15:53:15.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:14 smithi055 bash[9296]: cluster 2020-09-08T15:53:13.842059+0000 mgr.x (mgr.24118) 777 : cluster [DBG] pgmap v1122: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 89 MiB/s wr, 1.43k op/s 2020-09-08T15:53:15.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:14 smithi055 bash[9296]: cluster 2020-09-08T15:53:13.852362+0000 osd.3 (osd.3) 59 : cluster [DBG] 5.1 scrub starts 2020-09-08T15:53:15.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:14 smithi055 bash[9296]: cluster 2020-09-08T15:53:13.895630+0000 osd.9 (osd.9) 62 : cluster [DBG] 5.b scrub ok 2020-09-08T15:53:15.365 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 82 16 136314 136298 103.877 53.625 0.0124213 0.00962473 2020-09-08T15:53:15.494 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:15.597 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:15.847 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:15 smithi104 bash[9203]: cluster 2020-09-08T15:53:14.674740+0000 osd.9 (osd.9) 63 : cluster [DBG] 5.e scrub starts 2020-09-08T15:53:15.847 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:53:15 smithi104 bash[15273]: debug 2020-09-08T15:53:15.493+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:15.856 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:15.960 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:53:16.097 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:15 smithi166 bash[9446]: cluster 2020-09-08T15:53:14.674740+0000 osd.9 (osd.9) 63 : cluster [DBG] 5.e scrub starts 2020-09-08T15:53:16.098 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:53:16 smithi166 bash[18960]: debug 2020-09-08T15:53:16.096+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:16.098 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:53:16.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:15 smithi055 bash[9296]: cluster 2020-09-08T15:53:14.674740+0000 osd.9 (osd.9) 63 : cluster [DBG] 5.e scrub starts 2020-09-08T15:53:16.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:53:15 smithi104 bash[19922]: debug 2020-09-08T15:53:15.857+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:16.202 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:53:16.315 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:53:16.366 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 83 15 137378 137363 103.427 66.5625 0.0131815 0.0096666 2020-09-08T15:53:16.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:53:16 smithi166 bash[11928]: debug 2020-09-08T15:53:16.312+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:16.430 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:53:16.580 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:53:16.687 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:16.696 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:53:16 smithi055 bash[14301]: debug 2020-09-08T15:53:16.584+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:16.794 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:16.898 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:17.008 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:53:16 smithi055 bash[19003]: debug 2020-09-08T15:53:16.796+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:17.009 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:16 smithi055 bash[9296]: cluster 2020-09-08T15:53:15.175107+0000 osd.7 (osd.7) 83 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:53:17.009 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:16 smithi055 bash[9296]: cluster 2020-09-08T15:53:15.263821+0000 osd.7 (osd.7) 84 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:53:17.009 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:16 smithi055 bash[9296]: cluster 2020-09-08T15:53:15.667188+0000 osd.11 (osd.11) 70 : cluster [DBG] 5.d scrub starts 2020-09-08T15:53:17.009 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:16 smithi055 bash[9296]: cluster 2020-09-08T15:53:15.842571+0000 mgr.x (mgr.24118) 778 : cluster [DBG] pgmap v1123: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s wr, 805 op/s 2020-09-08T15:53:17.010 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:16 smithi055 bash[9296]: cluster 2020-09-08T15:53:16.180661+0000 osd.0 (osd.0) 55 : cluster [DBG] 5.5 scrub starts 2020-09-08T15:53:17.010 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:17.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:16 smithi166 bash[9446]: cluster 2020-09-08T15:53:15.175107+0000 osd.7 (osd.7) 83 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:53:17.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:16 smithi166 bash[9446]: cluster 2020-09-08T15:53:15.263821+0000 osd.7 (osd.7) 84 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:53:17.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:16 smithi166 bash[9446]: cluster 2020-09-08T15:53:15.667188+0000 osd.11 (osd.11) 70 : cluster [DBG] 5.d scrub starts 2020-09-08T15:53:17.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:16 smithi166 bash[9446]: cluster 2020-09-08T15:53:15.842571+0000 mgr.x (mgr.24118) 778 : cluster [DBG] pgmap v1123: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s wr, 805 op/s 2020-09-08T15:53:17.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:16 smithi166 bash[9446]: cluster 2020-09-08T15:53:16.180661+0000 osd.0 (osd.0) 55 : cluster [DBG] 5.5 scrub starts 2020-09-08T15:53:17.119 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:17.136 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:16 smithi104 bash[9203]: cluster 2020-09-08T15:53:15.175107+0000 osd.7 (osd.7) 83 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:53:17.137 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:16 smithi104 bash[9203]: cluster 2020-09-08T15:53:15.263821+0000 osd.7 (osd.7) 84 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:53:17.137 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:16 smithi104 bash[9203]: cluster 2020-09-08T15:53:15.667188+0000 osd.11 (osd.11) 70 : cluster [DBG] 5.d scrub starts 2020-09-08T15:53:17.137 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:16 smithi104 bash[9203]: cluster 2020-09-08T15:53:15.842571+0000 mgr.x (mgr.24118) 778 : cluster [DBG] pgmap v1123: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s wr, 805 op/s 2020-09-08T15:53:17.137 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:16 smithi104 bash[9203]: cluster 2020-09-08T15:53:16.180661+0000 osd.0 (osd.0) 55 : cluster [DBG] 5.5 scrub starts 2020-09-08T15:53:17.344 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:17.365 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 84 16 138429 138413 102.977 65.625 0.0163066 0.00970858 2020-09-08T15:53:17.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:53:17 smithi055 bash[19003]: debug 2020-09-08T15:53:17.012+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:17.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:53:17 smithi104 bash[15273]: debug 2020-09-08T15:53:17.341+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:17.449 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:18.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:17 smithi166 bash[9446]: cluster 2020-09-08T15:53:16.678895+0000 osd.3 (osd.3) 60 : cluster [DBG] 5.1 scrub ok 2020-09-08T15:53:18.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:17 smithi166 bash[9446]: cluster 2020-09-08T15:53:16.698256+0000 osd.9 (osd.9) 64 : cluster [DBG] 5.e scrub ok 2020-09-08T15:53:18.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:17 smithi055 bash[9296]: cluster 2020-09-08T15:53:16.678895+0000 osd.3 (osd.3) 60 : cluster [DBG] 5.1 scrub ok 2020-09-08T15:53:18.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:17 smithi055 bash[9296]: cluster 2020-09-08T15:53:16.698256+0000 osd.9 (osd.9) 64 : cluster [DBG] 5.e scrub ok 2020-09-08T15:53:18.211 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:17 smithi104 bash[9203]: cluster 2020-09-08T15:53:16.678895+0000 osd.3 (osd.3) 60 : cluster [DBG] 5.1 scrub ok 2020-09-08T15:53:18.212 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:17 smithi104 bash[9203]: cluster 2020-09-08T15:53:16.698256+0000 osd.9 (osd.9) 64 : cluster [DBG] 5.e scrub ok 2020-09-08T15:53:18.226 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:18.330 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:53:18.366 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 85 16 139441 139425 102.509 63.25 0.0170964 0.00975287 2020-09-08T15:53:18.472 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:53:18.578 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:53:18.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:53:18 smithi166 bash[16593]: debug 2020-09-08T15:53:18.468+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:18.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:53:18 smithi104 bash[19922]: debug 2020-09-08T15:53:18.225+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:18.688 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:53:18.792 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:19.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:53:18 smithi166 bash[16593]: debug 2020-09-08T15:53:18.688+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:19.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:18 smithi166 bash[9446]: cluster 2020-09-08T15:53:17.843080+0000 mgr.x (mgr.24118) 779 : cluster [DBG] pgmap v1124: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s wr, 1.20k op/s 2020-09-08T15:53:19.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:18 smithi166 bash[9446]: cluster 2020-09-08T15:53:18.133898+0000 osd.11 (osd.11) 71 : cluster [DBG] 5.d scrub ok 2020-09-08T15:53:19.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:18 smithi055 bash[9296]: cluster 2020-09-08T15:53:17.843080+0000 mgr.x (mgr.24118) 779 : cluster [DBG] pgmap v1124: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s wr, 1.20k op/s 2020-09-08T15:53:19.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:18 smithi055 bash[9296]: cluster 2020-09-08T15:53:18.133898+0000 osd.11 (osd.11) 71 : cluster [DBG] 5.d scrub ok 2020-09-08T15:53:19.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:18 smithi104 bash[9203]: cluster 2020-09-08T15:53:17.843080+0000 mgr.x (mgr.24118) 779 : cluster [DBG] pgmap v1124: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s wr, 1.20k op/s 2020-09-08T15:53:19.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:18 smithi104 bash[9203]: cluster 2020-09-08T15:53:18.133898+0000 osd.11 (osd.11) 71 : cluster [DBG] 5.d scrub ok 2020-09-08T15:53:19.366 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 86 16 140468 140452 102.064 64.1875 0.0873158 0.00979094 2020-09-08T15:53:19.391 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:19.495 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:19.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:53:19 smithi104 bash[15273]: debug 2020-09-08T15:53:19.393+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:19.870 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:19.974 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:53:20.111 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:19 smithi166 bash[9446]: cluster 2020-09-08T15:53:19.179239+0000 osd.0 (osd.0) 56 : cluster [DBG] 5.5 scrub ok 2020-09-08T15:53:20.112 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:53:20.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:19 smithi055 bash[9296]: cluster 2020-09-08T15:53:19.179239+0000 osd.0 (osd.0) 56 : cluster [DBG] 5.5 scrub ok 2020-09-08T15:53:20.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:19 smithi104 bash[9203]: cluster 2020-09-08T15:53:19.179239+0000 osd.0 (osd.0) 56 : cluster [DBG] 5.5 scrub ok 2020-09-08T15:53:20.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:53:19 smithi104 bash[19922]: debug 2020-09-08T15:53:19.869+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:20.215 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:53:20.342 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:53:20.366 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 87 16 141454 141438 101.599 61.625 0.0050687 0.00984071 2020-09-08T15:53:20.366 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:53:20 smithi166 bash[16593]: debug 2020-09-08T15:53:20.108+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:20.367 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:53:20 smithi166 bash[16593]: debug 2020-09-08T15:53:20.340+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:20.445 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:20.665 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:20.769 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:20.874 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:20 smithi104 bash[9203]: cluster 2020-09-08T15:53:19.843469+0000 mgr.x (mgr.24118) 780 : cluster [DBG] pgmap v1125: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 8.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T15:53:20.875 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:20 smithi104 bash[9203]: cluster 2020-09-08T15:53:19.936527+0000 osd.5 (osd.5) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:20.875 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:20 smithi104 bash[9203]: cluster 2020-09-08T15:53:19.936653+0000 osd.5 (osd.5) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:20.875 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:53:20 smithi104 bash[17588]: debug 2020-09-08T15:53:20.661+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:20.896 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:20.969 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:53:20.970 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:53:20.999 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:21.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:20 smithi166 bash[9446]: cluster 2020-09-08T15:53:19.843469+0000 mgr.x (mgr.24118) 780 : cluster [DBG] pgmap v1125: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 8.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T15:53:21.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:20 smithi166 bash[9446]: cluster 2020-09-08T15:53:19.936527+0000 osd.5 (osd.5) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:21.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:20 smithi166 bash[9446]: cluster 2020-09-08T15:53:19.936653+0000 osd.5 (osd.5) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:21.118 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:53:21.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:53:20 smithi055 bash[19003]: debug 2020-09-08T15:53:20.895+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:21.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:20 smithi055 bash[9296]: cluster 2020-09-08T15:53:19.843469+0000 mgr.x (mgr.24118) 780 : cluster [DBG] pgmap v1125: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 8.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T15:53:21.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:20 smithi055 bash[9296]: cluster 2020-09-08T15:53:19.936527+0000 osd.5 (osd.5) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:21.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:20 smithi055 bash[9296]: cluster 2020-09-08T15:53:19.936653+0000 osd.5 (osd.5) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:21.158 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:53:21.192 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:53:21.219 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:21.230 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:53:21.265 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:53:21.302 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:53:21.322 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:53:21.344 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:53:21.366 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 88 16 142421 142405 101.131 60.4375 0.0112876 0.00988567 2020-09-08T15:53:21.383 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:53:21.419 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:53:21.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:53:21 smithi104 bash[22305]: debug 2020-09-08T15:53:21.213+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:21.448 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:53:21.460 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:53:21.497 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:53:21.538 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:53:21.558 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:21.559 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:53:21.559 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:53:21.560 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:53:21.560 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:53:21.560 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:53:21.560 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:53:21.560 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:53:21.561 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:53:21.561 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:53:21.561 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:53:21.561 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:53:21.561 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:53:21.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:53:21 smithi055 bash[14301]: debug 2020-09-08T15:53:21.447+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:21.679 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:21.784 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:21.877 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:53:22.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:53:21 smithi055 bash[11956]: debug 2020-09-08T15:53:21.679+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:21 smithi055 bash[9296]: cluster 2020-09-08T15:53:20.763577+0000 osd.11 (osd.11) 72 : cluster [DBG] 5.8 scrub starts 2020-09-08T15:53:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:21 smithi055 bash[9296]: cluster 2020-09-08T15:53:20.872463+0000 mon.a (mon.0) 1585 : cluster [DBG] osdmap e430: 12 total, 12 up, 12 in 2020-09-08T15:53:22.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:21 smithi055 bash[9296]: audit 2020-09-08T15:53:20.947927+0000 mon.c (mon.2) 64 : audit [INF] from='client.? v1:172.21.15.104:0/2404967796' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:53:22.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:21 smithi055 bash[9296]: audit 2020-09-08T15:53:20.950660+0000 mon.a (mon.0) 1586 : audit [INF] from='client.35720 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:53:22.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:21 smithi055 bash[9296]: audit 2020-09-08T15:53:21.076029+0000 mon.b (mon.1) 100 : audit [DBG] from='client.? v1:172.21.15.104:0/1204756155' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:53:22.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:21 smithi055 bash[9296]: cluster 2020-09-08T15:53:21.368907+0000 osd.2 (osd.2) 56 : cluster [DBG] 5.6 scrub starts 2020-09-08T15:53:22.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:21 smithi104 bash[9203]: cluster 2020-09-08T15:53:20.763577+0000 osd.11 (osd.11) 72 : cluster [DBG] 5.8 scrub starts 2020-09-08T15:53:22.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:21 smithi104 bash[9203]: cluster 2020-09-08T15:53:20.872463+0000 mon.a (mon.0) 1585 : cluster [DBG] osdmap e430: 12 total, 12 up, 12 in 2020-09-08T15:53:22.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:21 smithi104 bash[9203]: audit 2020-09-08T15:53:20.947927+0000 mon.c (mon.2) 64 : audit [INF] from='client.? v1:172.21.15.104:0/2404967796' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:53:22.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:21 smithi104 bash[9203]: audit 2020-09-08T15:53:20.950660+0000 mon.a (mon.0) 1586 : audit [INF] from='client.35720 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:53:22.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:21 smithi104 bash[9203]: audit 2020-09-08T15:53:21.076029+0000 mon.b (mon.1) 100 : audit [DBG] from='client.? v1:172.21.15.104:0/1204756155' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:53:22.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:21 smithi104 bash[9203]: cluster 2020-09-08T15:53:21.368907+0000 osd.2 (osd.2) 56 : cluster [DBG] 5.6 scrub starts 2020-09-08T15:53:22.366 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 89 16 143438 143422 100.709 63.5625 0.00884833 0.00992729 2020-09-08T15:53:22.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:21 smithi166 bash[9446]: cluster 2020-09-08T15:53:20.763577+0000 osd.11 (osd.11) 72 : cluster [DBG] 5.8 scrub starts 2020-09-08T15:53:22.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:21 smithi166 bash[9446]: cluster 2020-09-08T15:53:20.872463+0000 mon.a (mon.0) 1585 : cluster [DBG] osdmap e430: 12 total, 12 up, 12 in 2020-09-08T15:53:22.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:21 smithi166 bash[9446]: audit 2020-09-08T15:53:20.947927+0000 mon.c (mon.2) 64 : audit [INF] from='client.? v1:172.21.15.104:0/2404967796' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:53:22.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:21 smithi166 bash[9446]: audit 2020-09-08T15:53:20.950660+0000 mon.a (mon.0) 1586 : audit [INF] from='client.35720 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:53:22.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:21 smithi166 bash[9446]: audit 2020-09-08T15:53:21.076029+0000 mon.b (mon.1) 100 : audit [DBG] from='client.? v1:172.21.15.104:0/1204756155' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:53:22.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:21 smithi166 bash[9446]: cluster 2020-09-08T15:53:21.368907+0000 osd.2 (osd.2) 56 : cluster [DBG] 5.6 scrub starts 2020-09-08T15:53:22.448 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:22.551 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:22.870 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:53:22 smithi104 bash[19922]: debug 2020-09-08T15:53:22.445+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:22 smithi055 bash[9296]: cluster 2020-09-08T15:53:21.758117+0000 osd.10 (osd.10) 47 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:22 smithi055 bash[9296]: cluster 2020-09-08T15:53:21.758190+0000 osd.10 (osd.10) 48 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:22 smithi055 bash[9296]: cluster 2020-09-08T15:53:21.843852+0000 mgr.x (mgr.24118) 781 : cluster [DBG] pgmap v1127: 17 pgs: 17 active+clean; 8.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 MiB/s wr, 762 op/s 2020-09-08T15:53:23.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:22 smithi055 bash[9296]: cluster 2020-09-08T15:53:21.867811+0000 mon.a (mon.0) 1587 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:53:23.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:22 smithi055 bash[9296]: cluster 2020-09-08T15:53:21.867840+0000 mon.a (mon.0) 1588 : cluster [INF] Cluster is now healthy 2020-09-08T15:53:23.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:22 smithi055 bash[9296]: audit 2020-09-08T15:53:21.878109+0000 mon.a (mon.0) 1589 : audit [INF] from='client.35720 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:53:23.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:22 smithi055 bash[9296]: cluster 2020-09-08T15:53:21.878202+0000 mon.a (mon.0) 1590 : cluster [DBG] osdmap e431: 12 total, 12 up, 12 in 2020-09-08T15:53:23.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:22 smithi104 bash[9203]: cluster 2020-09-08T15:53:21.758117+0000 osd.10 (osd.10) 47 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:23.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:22 smithi104 bash[9203]: cluster 2020-09-08T15:53:21.758190+0000 osd.10 (osd.10) 48 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:23.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:22 smithi104 bash[9203]: cluster 2020-09-08T15:53:21.843852+0000 mgr.x (mgr.24118) 781 : cluster [DBG] pgmap v1127: 17 pgs: 17 active+clean; 8.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 MiB/s wr, 762 op/s 2020-09-08T15:53:23.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:22 smithi104 bash[9203]: cluster 2020-09-08T15:53:21.867811+0000 mon.a (mon.0) 1587 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:53:23.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:22 smithi104 bash[9203]: cluster 2020-09-08T15:53:21.867840+0000 mon.a (mon.0) 1588 : cluster [INF] Cluster is now healthy 2020-09-08T15:53:23.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:22 smithi104 bash[9203]: audit 2020-09-08T15:53:21.878109+0000 mon.a (mon.0) 1589 : audit [INF] from='client.35720 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:53:23.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:22 smithi104 bash[9203]: cluster 2020-09-08T15:53:21.878202+0000 mon.a (mon.0) 1590 : cluster [DBG] osdmap e431: 12 total, 12 up, 12 in 2020-09-08T15:53:23.366 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 90 10 144419 144409 100.275 61.6875 0.0138164 0.00996995 2020-09-08T15:53:23.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:22 smithi166 bash[9446]: cluster 2020-09-08T15:53:21.758117+0000 osd.10 (osd.10) 47 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:23.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:22 smithi166 bash[9446]: cluster 2020-09-08T15:53:21.758190+0000 osd.10 (osd.10) 48 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:23.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:22 smithi166 bash[9446]: cluster 2020-09-08T15:53:21.843852+0000 mgr.x (mgr.24118) 781 : cluster [DBG] pgmap v1127: 17 pgs: 17 active+clean; 8.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 MiB/s wr, 762 op/s 2020-09-08T15:53:23.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:22 smithi166 bash[9446]: cluster 2020-09-08T15:53:21.867811+0000 mon.a (mon.0) 1587 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:53:23.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:22 smithi166 bash[9446]: cluster 2020-09-08T15:53:21.867840+0000 mon.a (mon.0) 1588 : cluster [INF] Cluster is now healthy 2020-09-08T15:53:23.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:22 smithi166 bash[9446]: audit 2020-09-08T15:53:21.878109+0000 mon.a (mon.0) 1589 : audit [INF] from='client.35720 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:53:23.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:22 smithi166 bash[9446]: cluster 2020-09-08T15:53:21.878202+0000 mon.a (mon.0) 1590 : cluster [DBG] osdmap e431: 12 total, 12 up, 12 in 2020-09-08T15:53:23.962 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2020-09-08T15:53:23.975 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:24.079 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:24.090 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:53:24.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:23 smithi055 bash[9296]: cluster 2020-09-08T15:53:22.362629+0000 osd.1 (osd.1) 84 : cluster [DBG] 5.c scrub starts 2020-09-08T15:53:24.191 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:53:23 smithi104 bash[19922]: debug 2020-09-08T15:53:23.977+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:24.192 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:23 smithi104 bash[9203]: cluster 2020-09-08T15:53:22.362629+0000 osd.1 (osd.1) 84 : cluster [DBG] 5.c scrub starts 2020-09-08T15:53:24.367 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Total time run: 90.0112 2020-09-08T15:53:24.367 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Total writes made: 144419 2020-09-08T15:53:24.367 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Write size: 65536 2020-09-08T15:53:24.367 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Object size: 65536 2020-09-08T15:53:24.368 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Bandwidth (MB/sec): 100.278 2020-09-08T15:53:24.368 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev Bandwidth: 41.7214 2020-09-08T15:53:24.368 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max bandwidth (MB/sec): 245.562 2020-09-08T15:53:24.368 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min bandwidth (MB/sec): 51.3125 2020-09-08T15:53:24.368 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Average IOPS: 1604 2020-09-08T15:53:24.369 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev IOPS: 667 2020-09-08T15:53:24.369 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max IOPS: 3929 2020-09-08T15:53:24.369 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min IOPS: 821 2020-09-08T15:53:24.369 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Average Latency(s): 0.009971 2020-09-08T15:53:24.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:23 smithi166 bash[9446]: cluster 2020-09-08T15:53:22.362629+0000 osd.1 (osd.1) 84 : cluster [DBG] 5.c scrub starts 2020-09-08T15:53:24.370 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev Latency(s): 0.013344 2020-09-08T15:53:24.370 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max latency(s): 0.606193 2020-09-08T15:53:24.370 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min latency(s): 0.00169895 2020-09-08T15:53:24.380 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Cleaning up (deleting benchmark objects) 2020-09-08T15:53:24.639 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:24.740 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:53:24.878 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:53:24.984 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:53:24.989 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:24 smithi104 bash[9203]: cluster 2020-09-08T15:53:23.742641+0000 osd.11 (osd.11) 73 : cluster [DBG] 5.8 scrub ok 2020-09-08T15:53:24.989 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:24 smithi104 bash[9203]: cluster 2020-09-08T15:53:23.844456+0000 mgr.x (mgr.24118) 782 : cluster [DBG] pgmap v1129: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s wr, 1.22k op/s 2020-09-08T15:53:24.989 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:24 smithi104 bash[9203]: cluster 2020-09-08T15:53:23.957549+0000 osd.2 (osd.2) 57 : cluster [DBG] 5.6 scrub ok 2020-09-08T15:53:24.990 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:24 smithi104 bash[9203]: cluster 2020-09-08T15:53:24.253309+0000 osd.0 (osd.0) 57 : cluster [DBG] 5.3 scrub starts 2020-09-08T15:53:24.990 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:53:24 smithi104 bash[17588]: debug 2020-09-08T15:53:24.629+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:25.100 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:53:24 smithi166 bash[16593]: debug 2020-09-08T15:53:24.876+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:25.101 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:24 smithi166 bash[9446]: cluster 2020-09-08T15:53:23.742641+0000 osd.11 (osd.11) 73 : cluster [DBG] 5.8 scrub ok 2020-09-08T15:53:25.101 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:24 smithi166 bash[9446]: cluster 2020-09-08T15:53:23.844456+0000 mgr.x (mgr.24118) 782 : cluster [DBG] pgmap v1129: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s wr, 1.22k op/s 2020-09-08T15:53:25.101 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:24 smithi166 bash[9446]: cluster 2020-09-08T15:53:23.957549+0000 osd.2 (osd.2) 57 : cluster [DBG] 5.6 scrub ok 2020-09-08T15:53:25.101 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:24 smithi166 bash[9446]: cluster 2020-09-08T15:53:24.253309+0000 osd.0 (osd.0) 57 : cluster [DBG] 5.3 scrub starts 2020-09-08T15:53:25.102 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:53:25 smithi166 bash[18960]: debug 2020-09-08T15:53:25.100+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:25.102 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:53:25.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:24 smithi055 bash[9296]: cluster 2020-09-08T15:53:23.742641+0000 osd.11 (osd.11) 73 : cluster [DBG] 5.8 scrub ok 2020-09-08T15:53:25.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:24 smithi055 bash[9296]: cluster 2020-09-08T15:53:23.844456+0000 mgr.x (mgr.24118) 782 : cluster [DBG] pgmap v1129: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s wr, 1.22k op/s 2020-09-08T15:53:25.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:24 smithi055 bash[9296]: cluster 2020-09-08T15:53:23.957549+0000 osd.2 (osd.2) 57 : cluster [DBG] 5.6 scrub ok 2020-09-08T15:53:25.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:24 smithi055 bash[9296]: cluster 2020-09-08T15:53:24.253309+0000 osd.0 (osd.0) 57 : cluster [DBG] 5.3 scrub starts 2020-09-08T15:53:25.206 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:25.328 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:25.435 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:53:25.537 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:53:25 smithi055 bash[14301]: debug 2020-09-08T15:53:25.539+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:25.538 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:53:25 smithi055 bash[19003]: debug 2020-09-08T15:53:25.327+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:25.538 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:53:25.643 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:25.888 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:25.894 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:25 smithi055 bash[9296]: cluster 2020-09-08T15:53:24.622928+0000 osd.1 (osd.1) 85 : cluster [DBG] 5.c scrub ok 2020-09-08T15:53:25.894 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:25 smithi055 bash[9296]: cluster 2020-09-08T15:53:25.442264+0000 osd.2 (osd.2) 58 : cluster [DBG] 5.a scrub starts 2020-09-08T15:53:25.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:53:25 smithi104 bash[19922]: debug 2020-09-08T15:53:25.881+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:25.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:25 smithi104 bash[9203]: cluster 2020-09-08T15:53:24.622928+0000 osd.1 (osd.1) 85 : cluster [DBG] 5.c scrub ok 2020-09-08T15:53:25.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:25 smithi104 bash[9203]: cluster 2020-09-08T15:53:25.442264+0000 osd.2 (osd.2) 58 : cluster [DBG] 5.a scrub starts 2020-09-08T15:53:25.998 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:53:26.144 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:25 smithi166 bash[9446]: cluster 2020-09-08T15:53:24.622928+0000 osd.1 (osd.1) 85 : cluster [DBG] 5.c scrub ok 2020-09-08T15:53:26.145 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:25 smithi166 bash[9446]: cluster 2020-09-08T15:53:25.442264+0000 osd.2 (osd.2) 58 : cluster [DBG] 5.a scrub starts 2020-09-08T15:53:26.145 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:53:26.248 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:26.422 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:26.525 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:26.534 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:53:26 smithi104 bash[22305]: debug 2020-09-08T15:53:26.413+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:26.617 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:53:26 smithi166 bash[16593]: debug 2020-09-08T15:53:26.140+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:26.644 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:53:26.810 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:53:26.860 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:53:27.030 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:53:27.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:26 smithi055 bash[9296]: cluster 2020-09-08T15:53:25.844870+0000 mgr.x (mgr.24118) 783 : cluster [DBG] pgmap v1130: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s wr, 727 op/s 2020-09-08T15:53:27.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:26 smithi104 bash[9203]: cluster 2020-09-08T15:53:25.844870+0000 mgr.x (mgr.24118) 783 : cluster [DBG] pgmap v1130: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s wr, 727 op/s 2020-09-08T15:53:27.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:26 smithi166 bash[9446]: cluster 2020-09-08T15:53:25.844870+0000 mgr.x (mgr.24118) 783 : cluster [DBG] pgmap v1130: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s wr, 727 op/s 2020-09-08T15:53:28.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:27 smithi055 bash[9296]: cluster 2020-09-08T15:53:26.413470+0000 osd.3 (osd.3) 61 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:28.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:27 smithi055 bash[9296]: cluster 2020-09-08T15:53:26.413536+0000 osd.3 (osd.3) 62 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:28.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:27 smithi055 bash[9296]: cluster 2020-09-08T15:53:26.904153+0000 mon.a (mon.0) 1591 : cluster [DBG] osdmap e432: 12 total, 12 up, 12 in 2020-09-08T15:53:28.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:27 smithi104 bash[9203]: cluster 2020-09-08T15:53:26.413470+0000 osd.3 (osd.3) 61 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:28.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:27 smithi104 bash[9203]: cluster 2020-09-08T15:53:26.413536+0000 osd.3 (osd.3) 62 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:28.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:27 smithi104 bash[9203]: cluster 2020-09-08T15:53:26.904153+0000 mon.a (mon.0) 1591 : cluster [DBG] osdmap e432: 12 total, 12 up, 12 in 2020-09-08T15:53:28.295 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:28.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:27 smithi166 bash[9446]: cluster 2020-09-08T15:53:26.413470+0000 osd.3 (osd.3) 61 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:28.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:27 smithi166 bash[9446]: cluster 2020-09-08T15:53:26.413536+0000 osd.3 (osd.3) 62 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:28.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:27 smithi166 bash[9446]: cluster 2020-09-08T15:53:26.904153+0000 mon.a (mon.0) 1591 : cluster [DBG] osdmap e432: 12 total, 12 up, 12 in 2020-09-08T15:53:28.398 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:28.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:53:28 smithi104 bash[15273]: debug 2020-09-08T15:53:28.289+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:29.072 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:29.175 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:29.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:53:29 smithi104 bash[19922]: debug 2020-09-08T15:53:29.073+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:29.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:28 smithi104 bash[9203]: cluster 2020-09-08T15:53:27.601436+0000 osd.0 (osd.0) 58 : cluster [DBG] 5.3 scrub ok 2020-09-08T15:53:29.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:28 smithi104 bash[9203]: cluster 2020-09-08T15:53:27.845301+0000 mgr.x (mgr.24118) 784 : cluster [DBG] pgmap v1132: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 8.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s wr, 1.08k op/s 2020-09-08T15:53:29.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:28 smithi104 bash[9203]: cluster 2020-09-08T15:53:28.381678+0000 osd.2 (osd.2) 59 : cluster [DBG] 5.a scrub ok 2020-09-08T15:53:29.186 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:28 smithi055 bash[9296]: cluster 2020-09-08T15:53:27.601436+0000 osd.0 (osd.0) 58 : cluster [DBG] 5.3 scrub ok 2020-09-08T15:53:29.186 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:28 smithi055 bash[9296]: cluster 2020-09-08T15:53:27.845301+0000 mgr.x (mgr.24118) 784 : cluster [DBG] pgmap v1132: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 8.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s wr, 1.08k op/s 2020-09-08T15:53:29.186 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:28 smithi055 bash[9296]: cluster 2020-09-08T15:53:28.381678+0000 osd.2 (osd.2) 59 : cluster [DBG] 5.a scrub ok 2020-09-08T15:53:29.299 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:29.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:28 smithi166 bash[9446]: cluster 2020-09-08T15:53:27.601436+0000 osd.0 (osd.0) 58 : cluster [DBG] 5.3 scrub ok 2020-09-08T15:53:29.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:28 smithi166 bash[9446]: cluster 2020-09-08T15:53:27.845301+0000 mgr.x (mgr.24118) 784 : cluster [DBG] pgmap v1132: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 8.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s wr, 1.08k op/s 2020-09-08T15:53:29.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:28 smithi166 bash[9446]: cluster 2020-09-08T15:53:28.381678+0000 osd.2 (osd.2) 59 : cluster [DBG] 5.a scrub ok 2020-09-08T15:53:29.401 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:53:29.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:53:29 smithi055 bash[11956]: debug 2020-09-08T15:53:29.295+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:30.085 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:53:30.190 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:53:30.297 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:53:30.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:53:30 smithi166 bash[18960]: debug 2020-09-08T15:53:30.084+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:30.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:53:30 smithi166 bash[14254]: debug 2020-09-08T15:53:30.292+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:30.401 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:30.714 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:30.820 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:53:30.934 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:53:31.013 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:53:30 smithi104 bash[22305]: debug 2020-09-08T15:53:30.713+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:31.014 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:30 smithi104 bash[9203]: cluster 2020-09-08T15:53:29.845760+0000 mgr.x (mgr.24118) 785 : cluster [DBG] pgmap v1133: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 41 MiB/s wr, 1.04k op/s 2020-09-08T15:53:31.038 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:31.124 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:30 smithi166 bash[9446]: cluster 2020-09-08T15:53:29.845760+0000 mgr.x (mgr.24118) 785 : cluster [DBG] pgmap v1133: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 41 MiB/s wr, 1.04k op/s 2020-09-08T15:53:31.125 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:53:30 smithi166 bash[16593]: debug 2020-09-08T15:53:30.932+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:31.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:30 smithi055 bash[9296]: cluster 2020-09-08T15:53:29.845760+0000 mgr.x (mgr.24118) 785 : cluster [DBG] pgmap v1133: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 41 MiB/s wr, 1.04k op/s 2020-09-08T15:53:31.266 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:31.309 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:53:31 smithi104 bash[17588]: debug 2020-09-08T15:53:31.253+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:31.370 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:53:31.620 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:53:31.670 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:53:31.670 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T15:53:31.670 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2020-09-08T15:53:31.671 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2020-09-08T15:53:31.671 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:53:31.725 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:31.867 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:53:31 smithi166 bash[14254]: debug 2020-09-08T15:53:31.616+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:32.711 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:32.815 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:53:32.992 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:53:33.096 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:53:33.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:32 smithi166 bash[9446]: cluster 2020-09-08T15:53:31.702268+0000 osd.10 (osd.10) 49 : cluster [DBG] 5.4 deep-scrub starts 2020-09-08T15:53:33.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:32 smithi166 bash[9446]: cluster 2020-09-08T15:53:31.846247+0000 mgr.x (mgr.24118) 786 : cluster [DBG] pgmap v1134: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s rd, 33 MiB/s wr, 899 op/s 2020-09-08T15:53:33.118 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:53:32 smithi166 bash[18960]: debug 2020-09-08T15:53:32.988+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:33.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:32 smithi055 bash[9296]: cluster 2020-09-08T15:53:31.702268+0000 osd.10 (osd.10) 49 : cluster [DBG] 5.4 deep-scrub starts 2020-09-08T15:53:33.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:32 smithi055 bash[9296]: cluster 2020-09-08T15:53:31.846247+0000 mgr.x (mgr.24118) 786 : cluster [DBG] pgmap v1134: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s rd, 33 MiB/s wr, 899 op/s 2020-09-08T15:53:33.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:53:32 smithi104 bash[17588]: debug 2020-09-08T15:53:32.709+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:33.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:32 smithi104 bash[9203]: cluster 2020-09-08T15:53:31.702268+0000 osd.10 (osd.10) 49 : cluster [DBG] 5.4 deep-scrub starts 2020-09-08T15:53:33.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:32 smithi104 bash[9203]: cluster 2020-09-08T15:53:31.846247+0000 mgr.x (mgr.24118) 786 : cluster [DBG] pgmap v1134: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 8.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s rd, 33 MiB/s wr, 899 op/s 2020-09-08T15:53:33.216 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:53:33.320 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:33.431 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:33.536 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:33.657 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:53:33 smithi055 bash[11956]: debug 2020-09-08T15:53:33.431+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:33.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:53:33 smithi055 bash[14301]: debug 2020-09-08T15:53:33.215+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:33.779 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:33.882 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:33.903 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:53:33 smithi104 bash[15273]: debug 2020-09-08T15:53:33.781+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:34.002 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:34.107 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:53:34.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:53:34 smithi055 bash[19003]: debug 2020-09-08T15:53:33.999+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:34.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:33 smithi055 bash[9296]: cluster 2020-09-08T15:53:32.659153+0000 osd.8 (osd.8) 55 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:34.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:33 smithi055 bash[9296]: cluster 2020-09-08T15:53:32.659235+0000 osd.8 (osd.8) 56 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:34.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:33 smithi055 bash[9296]: cluster 2020-09-08T15:53:32.905089+0000 mon.a (mon.0) 1592 : cluster [DBG] osdmap e433: 12 total, 12 up, 12 in 2020-09-08T15:53:34.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:33 smithi104 bash[9203]: cluster 2020-09-08T15:53:32.659153+0000 osd.8 (osd.8) 55 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:34.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:33 smithi104 bash[9203]: cluster 2020-09-08T15:53:32.659235+0000 osd.8 (osd.8) 56 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:34.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:33 smithi104 bash[9203]: cluster 2020-09-08T15:53:32.905089+0000 mon.a (mon.0) 1592 : cluster [DBG] osdmap e433: 12 total, 12 up, 12 in 2020-09-08T15:53:34.236 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:33 smithi166 bash[9446]: cluster 2020-09-08T15:53:32.659153+0000 osd.8 (osd.8) 55 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:34.236 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:33 smithi166 bash[9446]: cluster 2020-09-08T15:53:32.659235+0000 osd.8 (osd.8) 56 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:34.237 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:33 smithi166 bash[9446]: cluster 2020-09-08T15:53:32.905089+0000 mon.a (mon.0) 1592 : cluster [DBG] osdmap e433: 12 total, 12 up, 12 in 2020-09-08T15:53:34.237 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:53:34.340 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2020-09-08T15:53:34.341 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:34.517 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:53:34.616 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:53:34 smithi166 bash[14254]: debug 2020-09-08T15:53:34.232+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:35.084 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:35.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:34 smithi104 bash[9203]: cluster 2020-09-08T15:53:33.846739+0000 mgr.x (mgr.24118) 787 : cluster [DBG] pgmap v1136: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 7.6 MiB/s wr, 1.09k op/s 2020-09-08T15:53:35.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:53:35 smithi104 bash[19922]: debug 2020-09-08T15:53:35.077+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:35.185 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:53:35.194 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:34 smithi055 bash[9296]: cluster 2020-09-08T15:53:33.846739+0000 mgr.x (mgr.24118) 787 : cluster [DBG] pgmap v1136: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 7.6 MiB/s wr, 1.09k op/s 2020-09-08T15:53:35.304 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:53:35.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:34 smithi166 bash[9446]: cluster 2020-09-08T15:53:33.846739+0000 mgr.x (mgr.24118) 787 : cluster [DBG] pgmap v1136: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 7.6 MiB/s wr, 1.09k op/s 2020-09-08T15:53:35.411 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:35.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:53:35 smithi055 bash[16646]: debug 2020-09-08T15:53:35.299+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:36.198 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:36.302 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:36.466 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:53:36 smithi104 bash[22305]: debug 2020-09-08T15:53:36.193+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:36.483 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:36.586 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:36.756 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:53:36.783 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:53:36.798 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:36.798 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:53:36 smithi104 bash[22305]: debug 2020-09-08T15:53:36.481+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:36.798 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:53:36 smithi104 bash[22305]: debug 2020-09-08T15:53:36.797+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:36.814 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:53:36.850 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:53:36.890 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:53:36.911 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:36.927 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:53:36.962 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:53:36.998 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:53:37.040 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:53:37.073 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:53:37.088 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:36 smithi104 bash[9203]: cluster 2020-09-08T15:53:35.847127+0000 mgr.x (mgr.24118) 788 : cluster [DBG] pgmap v1137: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 114 B/s rd, 4.4 MiB/s wr, 923 op/s 2020-09-08T15:53:37.088 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:36 smithi104 bash[9203]: audit 2020-09-08T15:53:36.719725+0000 mon.a (mon.0) 1593 : audit [DBG] from='client.? v1:172.21.15.104:0/672670590' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:53:37.105 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:53:37.113 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:37.139 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:53:37.161 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:53:37.161 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:53:37.162 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:53:37.162 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:53:37.162 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:53:37.162 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:53:37.163 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:53:37.163 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:53:37.163 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:53:37.163 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:53:37.163 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:53:37.163 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:53:37.217 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:53:37.226 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:36 smithi166 bash[9446]: cluster 2020-09-08T15:53:35.847127+0000 mgr.x (mgr.24118) 788 : cluster [DBG] pgmap v1137: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 114 B/s rd, 4.4 MiB/s wr, 923 op/s 2020-09-08T15:53:37.227 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:36 smithi166 bash[9446]: audit 2020-09-08T15:53:36.719725+0000 mon.a (mon.0) 1593 : audit [DBG] from='client.? v1:172.21.15.104:0/672670590' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:53:37.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:36 smithi055 bash[9296]: cluster 2020-09-08T15:53:35.847127+0000 mgr.x (mgr.24118) 788 : cluster [DBG] pgmap v1137: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 114 B/s rd, 4.4 MiB/s wr, 923 op/s 2020-09-08T15:53:37.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:36 smithi055 bash[9296]: audit 2020-09-08T15:53:36.719725+0000 mon.a (mon.0) 1593 : audit [DBG] from='client.? v1:172.21.15.104:0/672670590' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:53:37.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:53:37 smithi104 bash[17588]: debug 2020-09-08T15:53:37.113+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:37.936 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:53:38.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:37 smithi104 bash[9203]: audit 2020-09-08T15:53:37.664179+0000 mon.a (mon.0) 1594 : audit [INF] from='client.? v1:172.21.15.104:0/3982203931' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:53:38.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:37 smithi166 bash[9446]: audit 2020-09-08T15:53:37.664179+0000 mon.a (mon.0) 1594 : audit [INF] from='client.? v1:172.21.15.104:0/3982203931' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:53:38.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:37 smithi055 bash[9296]: audit 2020-09-08T15:53:37.664179+0000 mon.a (mon.0) 1594 : audit [INF] from='client.? v1:172.21.15.104:0/3982203931' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:53:39.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:38 smithi166 bash[9446]: cluster 2020-09-08T15:53:37.847636+0000 mgr.x (mgr.24118) 789 : cluster [DBG] pgmap v1138: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s rd, 384 KiB/s wr, 930 op/s 2020-09-08T15:53:39.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:38 smithi166 bash[9446]: cluster 2020-09-08T15:53:37.927311+0000 mon.a (mon.0) 1595 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:53:39.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:38 smithi166 bash[9446]: audit 2020-09-08T15:53:37.938890+0000 mon.a (mon.0) 1596 : audit [INF] from='client.? v1:172.21.15.104:0/3982203931' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:53:39.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:38 smithi166 bash[9446]: cluster 2020-09-08T15:53:37.938950+0000 mon.a (mon.0) 1597 : cluster [DBG] osdmap e434: 12 total, 12 up, 12 in 2020-09-08T15:53:39.393 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:38 smithi104 bash[9203]: cluster 2020-09-08T15:53:37.847636+0000 mgr.x (mgr.24118) 789 : cluster [DBG] pgmap v1138: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s rd, 384 KiB/s wr, 930 op/s 2020-09-08T15:53:39.393 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:38 smithi104 bash[9203]: cluster 2020-09-08T15:53:37.927311+0000 mon.a (mon.0) 1595 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:53:39.394 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:38 smithi104 bash[9203]: audit 2020-09-08T15:53:37.938890+0000 mon.a (mon.0) 1596 : audit [INF] from='client.? v1:172.21.15.104:0/3982203931' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:53:39.394 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:38 smithi104 bash[9203]: cluster 2020-09-08T15:53:37.938950+0000 mon.a (mon.0) 1597 : cluster [DBG] osdmap e434: 12 total, 12 up, 12 in 2020-09-08T15:53:39.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:38 smithi055 bash[9296]: cluster 2020-09-08T15:53:37.847636+0000 mgr.x (mgr.24118) 789 : cluster [DBG] pgmap v1138: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s rd, 384 KiB/s wr, 930 op/s 2020-09-08T15:53:39.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:38 smithi055 bash[9296]: cluster 2020-09-08T15:53:37.927311+0000 mon.a (mon.0) 1595 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:53:39.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:38 smithi055 bash[9296]: audit 2020-09-08T15:53:37.938890+0000 mon.a (mon.0) 1596 : audit [INF] from='client.? v1:172.21.15.104:0/3982203931' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:53:39.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:38 smithi055 bash[9296]: cluster 2020-09-08T15:53:37.938950+0000 mon.a (mon.0) 1597 : cluster [DBG] osdmap e434: 12 total, 12 up, 12 in 2020-09-08T15:53:40.881 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:53:40.986 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:41.110 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:40 smithi104 bash[9203]: cluster 2020-09-08T15:53:39.848130+0000 mgr.x (mgr.24118) 790 : cluster [DBG] pgmap v1140: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T15:53:41.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:40 smithi166 bash[9446]: cluster 2020-09-08T15:53:39.848130+0000 mgr.x (mgr.24118) 790 : cluster [DBG] pgmap v1140: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T15:53:41.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:53:40 smithi166 bash[16593]: debug 2020-09-08T15:53:40.884+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:41.118 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:41.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:40 smithi055 bash[9296]: cluster 2020-09-08T15:53:39.848130+0000 mgr.x (mgr.24118) 790 : cluster [DBG] pgmap v1140: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T15:53:41.226 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:53:41.346 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:53:41.395 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:53:41.403 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:53:41 smithi104 bash[22305]: debug 2020-09-08T15:53:41.108+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:41.410 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:53:41.411 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:53:41.454 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:41.571 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:53:41.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:53:41 smithi166 bash[11928]: debug 2020-09-08T15:53:41.348+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:41.642 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:41.663 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:53:41.673 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:53:41 smithi104 bash[22305]: debug 2020-09-08T15:53:41.640+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:41.759 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:53:41.835 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:53:41.914 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:53:41.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:41 smithi104 bash[9203]: cluster 2020-09-08T15:53:40.827604+0000 osd.1 (osd.1) 86 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:41.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:41 smithi104 bash[9203]: cluster 2020-09-08T15:53:40.827674+0000 osd.1 (osd.1) 87 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:42.017 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:53:42.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:53:41 smithi166 bash[11928]: debug 2020-09-08T15:53:41.912+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:42.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:41 smithi166 bash[9446]: cluster 2020-09-08T15:53:40.827604+0000 osd.1 (osd.1) 86 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:42.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:41 smithi166 bash[9446]: cluster 2020-09-08T15:53:40.827674+0000 osd.1 (osd.1) 87 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:42.154 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:53:42 smithi055 bash[16646]: debug 2020-09-08T15:53:42.155+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:42.155 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:41 smithi055 bash[9296]: cluster 2020-09-08T15:53:40.827604+0000 osd.1 (osd.1) 86 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:42.155 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:41 smithi055 bash[9296]: cluster 2020-09-08T15:53:40.827674+0000 osd.1 (osd.1) 87 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:42.156 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:53:42.269 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:53:42.381 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:53:42.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:53:42 smithi055 bash[16646]: debug 2020-09-08T15:53:42.371+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:42.482 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:53:42.604 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:53:42.707 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:42.821 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:53:42 smithi055 bash[16646]: debug 2020-09-08T15:53:42.603+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:42.822 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:42 smithi055 bash[9296]: cluster 2020-09-08T15:53:41.816727+0000 mon.a (mon.0) 1598 : cluster [DBG] osdmap e435: 12 total, 12 up, 12 in 2020-09-08T15:53:42.822 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:42 smithi055 bash[9296]: cluster 2020-09-08T15:53:41.848630+0000 mgr.x (mgr.24118) 791 : cluster [DBG] pgmap v1142: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 650 op/s 2020-09-08T15:53:43.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:42 smithi166 bash[9446]: cluster 2020-09-08T15:53:41.816727+0000 mon.a (mon.0) 1598 : cluster [DBG] osdmap e435: 12 total, 12 up, 12 in 2020-09-08T15:53:43.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:42 smithi166 bash[9446]: cluster 2020-09-08T15:53:41.848630+0000 mgr.x (mgr.24118) 791 : cluster [DBG] pgmap v1142: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 650 op/s 2020-09-08T15:53:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:42 smithi104 bash[9203]: cluster 2020-09-08T15:53:41.816727+0000 mon.a (mon.0) 1598 : cluster [DBG] osdmap e435: 12 total, 12 up, 12 in 2020-09-08T15:53:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:42 smithi104 bash[9203]: cluster 2020-09-08T15:53:41.848630+0000 mgr.x (mgr.24118) 791 : cluster [DBG] pgmap v1142: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 8.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 650 op/s 2020-09-08T15:53:43.281 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2020-09-08T15:53:43.338 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:43.441 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:53:43.558 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:53:43.601 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:53:43.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:53:43 smithi104 bash[17588]: debug 2020-09-08T15:53:43.336+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:43.705 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:53:43.712 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:53:43 smithi166 bash[14254]: debug 2020-09-08T15:53:43.600+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:43.816 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:53:43.921 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:44.106 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:44.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:53:43 smithi166 bash[18960]: debug 2020-09-08T15:53:43.816+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:44.186 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:53:44 smithi104 bash[15273]: debug 2020-09-08T15:53:44.104+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:44.214 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:44.344 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:44.450 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:44.458 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:53:44 smithi104 bash[19922]: debug 2020-09-08T15:53:44.340+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:44.686 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:44.790 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:53:44.896 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:53:44 smithi104 bash[22305]: debug 2020-09-08T15:53:44.680+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:44.897 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:44 smithi104 bash[9203]: cluster 2020-09-08T15:53:43.849101+0000 mgr.x (mgr.24118) 792 : cluster [DBG] pgmap v1143: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T15:53:44.940 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:53:45.045 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:53:45.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:44 smithi166 bash[9446]: cluster 2020-09-08T15:53:43.849101+0000 mgr.x (mgr.24118) 792 : cluster [DBG] pgmap v1143: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T15:53:45.117 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:53:44 smithi166 bash[11928]: debug 2020-09-08T15:53:44.940+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:45.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:44 smithi055 bash[9296]: cluster 2020-09-08T15:53:43.849101+0000 mgr.x (mgr.24118) 792 : cluster [DBG] pgmap v1143: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T15:53:45.277 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:53:45.381 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:53:45.550 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:53:45 smithi055 bash[14301]: debug 2020-09-08T15:53:45.279+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:45.550 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:53:45 smithi055 bash[14301]: debug 2020-09-08T15:53:45.551+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:45.554 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:53:45.657 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:45.848 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:53:45 smithi055 bash[11956]: debug 2020-09-08T15:53:45.843+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:45.848 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:45.951 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:46.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:45 smithi055 bash[9296]: cluster 2020-09-08T15:53:44.810863+0000 osd.6 (osd.6) 69 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:46.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:45 smithi055 bash[9296]: cluster 2020-09-08T15:53:44.810934+0000 osd.6 (osd.6) 70 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:46.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:45 smithi055 bash[9296]: cluster 2020-09-08T15:53:45.805571+0000 mon.a (mon.0) 1599 : cluster [DBG] osdmap e436: 12 total, 12 up, 12 in 2020-09-08T15:53:46.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:45 smithi104 bash[9203]: cluster 2020-09-08T15:53:44.810863+0000 osd.6 (osd.6) 69 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:46.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:45 smithi104 bash[9203]: cluster 2020-09-08T15:53:44.810934+0000 osd.6 (osd.6) 70 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:46.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:45 smithi104 bash[9203]: cluster 2020-09-08T15:53:45.805571+0000 mon.a (mon.0) 1599 : cluster [DBG] osdmap e436: 12 total, 12 up, 12 in 2020-09-08T15:53:46.306 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:46.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:45 smithi166 bash[9446]: cluster 2020-09-08T15:53:44.810863+0000 osd.6 (osd.6) 69 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:46.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:45 smithi166 bash[9446]: cluster 2020-09-08T15:53:44.810934+0000 osd.6 (osd.6) 70 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:46.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:45 smithi166 bash[9446]: cluster 2020-09-08T15:53:45.805571+0000 mon.a (mon.0) 1599 : cluster [DBG] osdmap e436: 12 total, 12 up, 12 in 2020-09-08T15:53:46.410 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:53:46.553 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:53:46.657 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:46.665 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:53:46 smithi104 bash[15273]: debug 2020-09-08T15:53:46.304+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:46.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:53:46 smithi166 bash[18960]: debug 2020-09-08T15:53:46.556+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:46.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:46 smithi104 bash[9203]: cluster 2020-09-08T15:53:45.849564+0000 mgr.x (mgr.24118) 793 : cluster [DBG] pgmap v1145: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 641 op/s 2020-09-08T15:53:47.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:46 smithi055 bash[9296]: cluster 2020-09-08T15:53:45.849564+0000 mgr.x (mgr.24118) 793 : cluster [DBG] pgmap v1145: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 641 op/s 2020-09-08T15:53:47.348 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:47.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:46 smithi166 bash[9446]: cluster 2020-09-08T15:53:45.849564+0000 mgr.x (mgr.24118) 793 : cluster [DBG] pgmap v1145: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 641 op/s 2020-09-08T15:53:47.452 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:53:47.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:53:47 smithi104 bash[19922]: debug 2020-09-08T15:53:47.348+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:47.891 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:53:47.995 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:53:48.133 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:53:48.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:53:47 smithi055 bash[16646]: debug 2020-09-08T15:53:47.891+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:48.237 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:48.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:53:48 smithi166 bash[18960]: debug 2020-09-08T15:53:48.136+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:48.401 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:48.438 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:53:48.438 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:53:48.504 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:48.523 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:53:48 smithi055 bash[11956]: debug 2020-09-08T15:53:48.399+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:48.698 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:48.774 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:53:48 smithi104 bash[19922]: debug 2020-09-08T15:53:48.692+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:48.803 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:53:49.055 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:53:49.093 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:53:49.133 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:53:49.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:48 smithi055 bash[9296]: cluster 2020-09-08T15:53:47.850052+0000 mgr.x (mgr.24118) 794 : cluster [DBG] pgmap v1146: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 972 op/s 2020-09-08T15:53:49.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:48 smithi104 bash[9203]: cluster 2020-09-08T15:53:47.850052+0000 mgr.x (mgr.24118) 794 : cluster [DBG] pgmap v1146: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 972 op/s 2020-09-08T15:53:49.184 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:53:49.223 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:53:49.251 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:53:49.284 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:53:49.313 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:53:49.328 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:53:49.349 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:53:49.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:48 smithi166 bash[9446]: cluster 2020-09-08T15:53:47.850052+0000 mgr.x (mgr.24118) 794 : cluster [DBG] pgmap v1146: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 972 op/s 2020-09-08T15:53:49.386 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:53:49.417 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:53:49.432 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:49.445 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2020-09-08T15:53:49.449 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:53:49.451 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:53:49.452 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:53:49.452 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:53:49.452 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:53:49.452 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:53:49.452 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:53:49.453 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:53:49.453 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:53:49.453 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:53:49.453 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:53:49.453 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:53:49.454 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:53:49.565 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:49.601 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:53:49.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:53:49 smithi055 bash[16646]: debug 2020-09-08T15:53:49.327+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:49.669 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:49.968 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:53:49 smithi104 bash[15273]: debug 2020-09-08T15:53:49.564+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:49.983 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:50.086 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:50.134 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:53:50.231 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:53:49 smithi104 bash[17588]: debug 2020-09-08T15:53:49.976+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:50.231 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:50 smithi104 bash[9203]: cluster 2020-09-08T15:53:48.768084+0000 osd.11 (osd.11) 74 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:50.231 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:50 smithi104 bash[9203]: cluster 2020-09-08T15:53:48.768179+0000 osd.11 (osd.11) 75 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:50.231 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:50 smithi104 bash[9203]: cluster 2020-09-08T15:53:48.919147+0000 mon.a (mon.0) 1600 : cluster [DBG] osdmap e437: 12 total, 12 up, 12 in 2020-09-08T15:53:50.232 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:50 smithi104 bash[9203]: audit 2020-09-08T15:53:49.023840+0000 mon.a (mon.0) 1601 : audit [DBG] from='client.? v1:172.21.15.104:0/2941742252' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:53:50.232 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:50 smithi104 bash[9203]: audit 2020-09-08T15:53:49.303336+0000 mon.b (mon.1) 101 : audit [INF] from='client.? v1:172.21.15.104:0/3236241070' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:53:50.232 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:50 smithi104 bash[9203]: audit 2020-09-08T15:53:49.304076+0000 mon.a (mon.0) 1602 : audit [INF] from='client.25735 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:53:50.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:50 smithi055 bash[9296]: cluster 2020-09-08T15:53:48.768084+0000 osd.11 (osd.11) 74 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:50.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:50 smithi055 bash[9296]: cluster 2020-09-08T15:53:48.768179+0000 osd.11 (osd.11) 75 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:50.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:50 smithi055 bash[9296]: cluster 2020-09-08T15:53:48.919147+0000 mon.a (mon.0) 1600 : cluster [DBG] osdmap e437: 12 total, 12 up, 12 in 2020-09-08T15:53:50.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:50 smithi055 bash[9296]: audit 2020-09-08T15:53:49.023840+0000 mon.a (mon.0) 1601 : audit [DBG] from='client.? v1:172.21.15.104:0/2941742252' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:53:50.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:50 smithi055 bash[9296]: audit 2020-09-08T15:53:49.303336+0000 mon.b (mon.1) 101 : audit [INF] from='client.? v1:172.21.15.104:0/3236241070' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:53:50.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:50 smithi055 bash[9296]: audit 2020-09-08T15:53:49.304076+0000 mon.a (mon.0) 1602 : audit [INF] from='client.25735 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:53:50.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:50 smithi166 bash[9446]: cluster 2020-09-08T15:53:48.768084+0000 osd.11 (osd.11) 74 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:50.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:50 smithi166 bash[9446]: cluster 2020-09-08T15:53:48.768179+0000 osd.11 (osd.11) 75 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:50.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:50 smithi166 bash[9446]: cluster 2020-09-08T15:53:48.919147+0000 mon.a (mon.0) 1600 : cluster [DBG] osdmap e437: 12 total, 12 up, 12 in 2020-09-08T15:53:50.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:50 smithi166 bash[9446]: audit 2020-09-08T15:53:49.023840+0000 mon.a (mon.0) 1601 : audit [DBG] from='client.? v1:172.21.15.104:0/2941742252' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:53:50.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:50 smithi166 bash[9446]: audit 2020-09-08T15:53:49.303336+0000 mon.b (mon.1) 101 : audit [INF] from='client.? v1:172.21.15.104:0/3236241070' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:53:50.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:50 smithi166 bash[9446]: audit 2020-09-08T15:53:49.304076+0000 mon.a (mon.0) 1602 : audit [INF] from='client.25735 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:53:50.878 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:50.984 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:51.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:53:50 smithi104 bash[17588]: debug 2020-09-08T15:53:50.872+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:51.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:51 smithi104 bash[9203]: cluster 2020-09-08T15:53:49.850699+0000 mgr.x (mgr.24118) 795 : cluster [DBG] pgmap v1148: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T15:53:51.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:51 smithi104 bash[9203]: cluster 2020-09-08T15:53:49.917144+0000 mon.a (mon.0) 1603 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:53:51.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:51 smithi104 bash[9203]: audit 2020-09-08T15:53:50.131949+0000 mon.a (mon.0) 1604 : audit [INF] from='client.25735 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:53:51.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:51 smithi104 bash[9203]: cluster 2020-09-08T15:53:50.132003+0000 mon.a (mon.0) 1605 : cluster [DBG] osdmap e438: 12 total, 12 up, 12 in 2020-09-08T15:53:51.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:51 smithi055 bash[9296]: cluster 2020-09-08T15:53:49.850699+0000 mgr.x (mgr.24118) 795 : cluster [DBG] pgmap v1148: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T15:53:51.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:51 smithi055 bash[9296]: cluster 2020-09-08T15:53:49.917144+0000 mon.a (mon.0) 1603 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:53:51.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:51 smithi055 bash[9296]: audit 2020-09-08T15:53:50.131949+0000 mon.a (mon.0) 1604 : audit [INF] from='client.25735 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:53:51.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:51 smithi055 bash[9296]: cluster 2020-09-08T15:53:50.132003+0000 mon.a (mon.0) 1605 : cluster [DBG] osdmap e438: 12 total, 12 up, 12 in 2020-09-08T15:53:51.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:51 smithi166 bash[9446]: cluster 2020-09-08T15:53:49.850699+0000 mgr.x (mgr.24118) 795 : cluster [DBG] pgmap v1148: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T15:53:51.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:51 smithi166 bash[9446]: cluster 2020-09-08T15:53:49.917144+0000 mon.a (mon.0) 1603 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:53:51.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:51 smithi166 bash[9446]: audit 2020-09-08T15:53:50.131949+0000 mon.a (mon.0) 1604 : audit [INF] from='client.25735 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:53:51.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:51 smithi166 bash[9446]: cluster 2020-09-08T15:53:50.132003+0000 mon.a (mon.0) 1605 : cluster [DBG] osdmap e438: 12 total, 12 up, 12 in 2020-09-08T15:53:51.637 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:51.742 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:53:51.889 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:53:51.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:53:51 smithi104 bash[22305]: debug 2020-09-08T15:53:51.636+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:51.996 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:53:52.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:53:51 smithi166 bash[11928]: debug 2020-09-08T15:53:51.887+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:52.236 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:53:52.339 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:52.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:53:52 smithi055 bash[16646]: debug 2020-09-08T15:53:52.235+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:52.473 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:52.578 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:53:52.677 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:53:52 smithi104 bash[22305]: debug 2020-09-08T15:53:52.472+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:52.997 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:53:53.101 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:53.226 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:53.331 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:53.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:53:53 smithi055 bash[14301]: debug 2020-09-08T15:53:52.995+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:53.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:53 smithi055 bash[9296]: cluster 2020-09-08T15:53:51.851205+0000 mgr.x (mgr.24118) 796 : cluster [DBG] pgmap v1150: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 801 op/s 2020-09-08T15:53:53.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:53:53 smithi104 bash[19922]: debug 2020-09-08T15:53:53.224+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:53.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:53 smithi104 bash[9203]: cluster 2020-09-08T15:53:51.851205+0000 mgr.x (mgr.24118) 796 : cluster [DBG] pgmap v1150: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 801 op/s 2020-09-08T15:53:53.442 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:53.548 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:53.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:53 smithi166 bash[9446]: cluster 2020-09-08T15:53:51.851205+0000 mgr.x (mgr.24118) 796 : cluster [DBG] pgmap v1150: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 801 op/s 2020-09-08T15:53:53.640 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:53:53.666 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:53.766 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:53:53.771 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:53.774 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:53:53 smithi104 bash[17588]: debug 2020-09-08T15:53:53.440+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:53.775 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:53:53 smithi104 bash[15273]: debug 2020-09-08T15:53:53.664+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:53.801 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:53:53.934 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:53.945 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:53:54.046 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:54.073 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:53:53 smithi104 bash[19922]: debug 2020-09-08T15:53:53.928+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:54.240 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:53:54.241 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T15:53:54.241 INFO:teuthology.orchestra.run.smithi055.stdout: "complaint_time": 30, 2020-09-08T15:53:54.241 INFO:teuthology.orchestra.run.smithi055.stdout: "num_blocked_ops": 0 2020-09-08T15:53:54.241 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:53:54.340 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:54 smithi055 bash[9296]: cluster 2020-09-08T15:53:52.385626+0000 osd.4 (osd.4) 45 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:54.341 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:54 smithi055 bash[9296]: cluster 2020-09-08T15:53:52.385697+0000 osd.4 (osd.4) 46 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:54.341 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:54 smithi055 bash[9296]: cluster 2020-09-08T15:53:53.130490+0000 mon.a (mon.0) 1606 : cluster [DBG] osdmap e439: 12 total, 12 up, 12 in 2020-09-08T15:53:54.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:54 smithi104 bash[9203]: cluster 2020-09-08T15:53:52.385626+0000 osd.4 (osd.4) 45 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:54.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:54 smithi104 bash[9203]: cluster 2020-09-08T15:53:52.385697+0000 osd.4 (osd.4) 46 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:54.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:54 smithi104 bash[9203]: cluster 2020-09-08T15:53:53.130490+0000 mon.a (mon.0) 1606 : cluster [DBG] osdmap e439: 12 total, 12 up, 12 in 2020-09-08T15:53:54.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:54 smithi166 bash[9446]: cluster 2020-09-08T15:53:52.385626+0000 osd.4 (osd.4) 45 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:53:54.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:54 smithi166 bash[9446]: cluster 2020-09-08T15:53:52.385697+0000 osd.4 (osd.4) 46 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:53:54.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:54 smithi166 bash[9446]: cluster 2020-09-08T15:53:53.130490+0000 mon.a (mon.0) 1606 : cluster [DBG] osdmap e439: 12 total, 12 up, 12 in 2020-09-08T15:53:55.110 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:55.181 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2020-09-08T15:53:55.214 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:55.344 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:53:55.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:55 smithi055 bash[9296]: cluster 2020-09-08T15:53:53.851679+0000 mgr.x (mgr.24118) 797 : cluster [DBG] pgmap v1152: 17 pgs: 17 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 858 op/s 2020-09-08T15:53:55.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:55 smithi104 bash[9203]: cluster 2020-09-08T15:53:53.851679+0000 mgr.x (mgr.24118) 797 : cluster [DBG] pgmap v1152: 17 pgs: 17 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 858 op/s 2020-09-08T15:53:55.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:53:55 smithi104 bash[19922]: debug 2020-09-08T15:53:55.108+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:55.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:55 smithi166 bash[9446]: cluster 2020-09-08T15:53:53.851679+0000 mgr.x (mgr.24118) 797 : cluster [DBG] pgmap v1152: 17 pgs: 17 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 858 op/s 2020-09-08T15:53:55.867 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:55.971 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:53:56.124 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:53:56.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:53:55 smithi104 bash[15273]: debug 2020-09-08T15:53:55.868+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:56.228 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:56.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:53:56 smithi055 bash[16646]: debug 2020-09-08T15:53:56.123+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:56.432 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:56.537 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:53:56.549 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:53:56 smithi104 bash[22305]: debug 2020-09-08T15:53:56.432+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:56.693 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:53:56.797 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:56.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:53:56 smithi166 bash[14254]: debug 2020-09-08T15:53:56.691+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:57.000 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:53:57.105 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:57.146 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:53:57 smithi104 bash[22305]: debug 2020-09-08T15:53:57.000+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:57.259 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:57 smithi055 bash[9296]: cluster 2020-09-08T15:53:55.852077+0000 mgr.x (mgr.24118) 798 : cluster [DBG] pgmap v1153: 17 pgs: 17 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 711 op/s 2020-09-08T15:53:57.260 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:57.364 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:57.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:57 smithi104 bash[9203]: cluster 2020-09-08T15:53:55.852077+0000 mgr.x (mgr.24118) 798 : cluster [DBG] pgmap v1153: 17 pgs: 17 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 711 op/s 2020-09-08T15:53:57.526 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:53:57 smithi055 bash[19003]: debug 2020-09-08T15:53:57.259+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:57.526 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:53:57.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:57 smithi166 bash[9446]: cluster 2020-09-08T15:53:55.852077+0000 mgr.x (mgr.24118) 798 : cluster [DBG] pgmap v1153: 17 pgs: 17 active+clean; 7.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 711 op/s 2020-09-08T15:53:57.630 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:53:57.784 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:53:57.886 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:53:57.895 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:53:57 smithi166 bash[18960]: debug 2020-09-08T15:53:57.779+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:57.906 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:53:57 smithi055 bash[11956]: debug 2020-09-08T15:53:57.527+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:58.003 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:53:58.112 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:58.268 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:53:58.367 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:53:58 smithi166 bash[16593]: debug 2020-09-08T15:53:58.003+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:58.371 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:53:58.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:53:58 smithi055 bash[19003]: debug 2020-09-08T15:53:58.271+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:58.489 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:53:58.592 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:53:58.707 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:53:58 smithi166 bash[18960]: debug 2020-09-08T15:53:58.487+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:58.708 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:53:58.811 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:59.006 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:53:59.110 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:59.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:53:58 smithi166 bash[16593]: debug 2020-09-08T15:53:58.707+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:59.118 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:53:59 smithi104 bash[19922]: debug 2020-09-08T15:53:59.008+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:59.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:53:59 smithi055 bash[9296]: cluster 2020-09-08T15:53:57.852563+0000 mgr.x (mgr.24118) 799 : cluster [DBG] pgmap v1154: 17 pgs: 17 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T15:53:59.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:53:59 smithi104 bash[9203]: cluster 2020-09-08T15:53:57.852563+0000 mgr.x (mgr.24118) 799 : cluster [DBG] pgmap v1154: 17 pgs: 17 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T15:53:59.493 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:53:59.596 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:59.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:53:59 smithi166 bash[9446]: cluster 2020-09-08T15:53:57.852563+0000 mgr.x (mgr.24118) 799 : cluster [DBG] pgmap v1154: 17 pgs: 17 active+clean; 7.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T15:53:59.812 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:53:59 smithi104 bash[15273]: debug 2020-09-08T15:53:59.492+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:53:59.814 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:53:59.918 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:00.056 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:00.059 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:54:00 smithi055 bash[19003]: debug 2020-09-08T15:54:00.051+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:00.113 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:54:00.113 INFO:teuthology.orchestra.run.smithi055.stdout: "size": 20, 2020-09-08T15:54:00.114 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 600, 2020-09-08T15:54:00.114 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T15:54:00.114 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.115 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:156269 3.3 e264/214 3:c5007835:::benchmark_data_smithi166_26097_object32108:head v 264'9757, mlcod=264'9755)", 2020-09-08T15:54:00.115 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:44:39.699256+0000", 2020-09-08T15:54:00.115 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 560.41262012100003, 2020-09-08T15:54:00.115 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044242320000000002, 2020-09-08T15:54:00.116 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:00.116 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:00.116 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:00.116 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.116 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:00.117 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699256+0000", 2020-09-08T15:54:00.117 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:00.117 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.117 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.117 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:00.118 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699256+0000", 2020-09-08T15:54:00.118 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.9709000000000005e-05 2020-09-08T15:54:00.118 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.118 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.118 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:00.119 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699346+0000", 2020-09-08T15:54:00.119 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.9023999999999999e-05 2020-09-08T15:54:00.119 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.119 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.120 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:00.120 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699365+0000", 2020-09-08T15:54:00.120 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.1735e-05 2020-09-08T15:54:00.120 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.121 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.121 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:00.121 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699397+0000", 2020-09-08T15:54:00.121 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5481e-05 2020-09-08T15:54:00.121 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.122 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.122 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:00.122 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699412+0000", 2020-09-08T15:54:00.122 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.9234000000000007e-05 2020-09-08T15:54:00.122 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.123 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.123 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:00.123 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699502+0000", 2020-09-08T15:54:00.123 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2763000000000003e-05 2020-09-08T15:54:00.123 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.124 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.124 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:00.124 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.699554+0000", 2020-09-08T15:54:00.124 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043752211999999999 2020-09-08T15:54:00.124 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.125 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.126 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:00.126 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.743307+0000", 2020-09-08T15:54:00.126 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019216200000000001 2020-09-08T15:54:00.126 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.126 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.127 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:00.127 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:44:39.743499+0000", 2020-09-08T15:54:00.127 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044242320000000002 2020-09-08T15:54:00.127 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.127 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.128 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.128 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.128 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.128 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:174393 3.a e271/214 3:51162694:::benchmark_data_smithi166_26097_object50232:head v 271'10901, mlcod=271'10899)", 2020-09-08T15:54:00.128 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:45:08.114501+0000", 2020-09-08T15:54:00.129 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 531.99737595700003, 2020-09-08T15:54:00.129 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.084128171000000002, 2020-09-08T15:54:00.129 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:00.129 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:00.129 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:00.129 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.130 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:00.130 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.114501+0000", 2020-09-08T15:54:00.130 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:00.130 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.130 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.131 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:00.131 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.114501+0000", 2020-09-08T15:54:00.131 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.0494999999999993e-05 2020-09-08T15:54:00.131 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.131 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.132 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:00.132 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.114591+0000", 2020-09-08T15:54:00.132 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2646000000000001e-05 2020-09-08T15:54:00.132 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.132 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.133 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:00.133 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.114604+0000", 2020-09-08T15:54:00.133 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.0993999999999999e-05 2020-09-08T15:54:00.133 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.133 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.134 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:00.134 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.114625+0000", 2020-09-08T15:54:00.134 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4939999999999992e-06 2020-09-08T15:54:00.134 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.134 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.136 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:00.136 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.114634+0000", 2020-09-08T15:54:00.136 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.9614e-05 2020-09-08T15:54:00.137 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.137 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.137 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:00.137 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.114724+0000", 2020-09-08T15:54:00.137 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.2058999999999998e-05 2020-09-08T15:54:00.138 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.138 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.138 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:00.138 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.114766+0000", 2020-09-08T15:54:00.138 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0091200340000000008 2020-09-08T15:54:00.139 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.139 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.139 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:00.139 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.123886+0000", 2020-09-08T15:54:00.139 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.074742834999999994 2020-09-08T15:54:00.140 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.140 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.140 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:00.140 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:08.198629+0000", 2020-09-08T15:54:00.140 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.084128171000000002 2020-09-08T15:54:00.140 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.141 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.141 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.141 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.141 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.141 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:188875 3.e e277/214 3:78e0085b:::benchmark_data_smithi166_26097_object64714:head v 277'11767, mlcod=277'11764)", 2020-09-08T15:54:00.142 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:45:29.700758+0000", 2020-09-08T15:54:00.142 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 510.41111849100002, 2020-09-08T15:54:00.142 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.051517127000000003, 2020-09-08T15:54:00.142 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:00.143 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:00.143 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:00.143 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.143 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:00.143 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.700758+0000", 2020-09-08T15:54:00.144 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:00.144 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.144 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.144 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:00.144 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.700758+0000", 2020-09-08T15:54:00.145 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000105216 2020-09-08T15:54:00.145 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.145 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.146 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:00.146 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.700863+0000", 2020-09-08T15:54:00.146 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6963999999999999e-05 2020-09-08T15:54:00.146 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.146 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.147 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:00.147 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.700880+0000", 2020-09-08T15:54:00.148 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.6429000000000001e-05 2020-09-08T15:54:00.148 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.148 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.148 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:00.149 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.700907+0000", 2020-09-08T15:54:00.149 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1688000000000001e-05 2020-09-08T15:54:00.149 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.149 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.149 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:00.150 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.700918+0000", 2020-09-08T15:54:00.150 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.9370999999999997e-05 2020-09-08T15:54:00.150 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.150 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.150 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:00.151 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.700988+0000", 2020-09-08T15:54:00.151 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.7089000000000001e-05 2020-09-08T15:54:00.151 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.151 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.151 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:00.152 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.701015+0000", 2020-09-08T15:54:00.152 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.051182631999999999 2020-09-08T15:54:00.152 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.152 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.152 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:00.153 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.752197+0000", 2020-09-08T15:54:00.153 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.7737999999999993e-05 2020-09-08T15:54:00.153 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.153 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.153 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:00.154 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:29.752275+0000", 2020-09-08T15:54:00.155 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.051517127000000003 2020-09-08T15:54:00.155 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.155 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.156 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.156 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.156 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.156 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35087.0:209805 3.a e286/214 3:5f9a82ac:::benchmark_data_smithi166_26097_object85644:head v 286'13119, mlcod=286'13117)", 2020-09-08T15:54:00.157 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:45:57.115910+0000", 2020-09-08T15:54:00.157 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 482.99596674100002, 2020-09-08T15:54:00.157 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.053311198999999997, 2020-09-08T15:54:00.157 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:00.157 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:00.158 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:00.158 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.158 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:00.158 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:57.115910+0000", 2020-09-08T15:54:00.158 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:00.159 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.159 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.159 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:00.159 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:57.115910+0000", 2020-09-08T15:54:00.159 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.9079999999999998e-05 2020-09-08T15:54:00.160 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.160 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.160 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:00.160 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:57.115959+0000", 2020-09-08T15:54:00.161 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.3579999999999997e-06 2020-09-08T15:54:00.161 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.161 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.161 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:00.161 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:57.115966+0000", 2020-09-08T15:54:00.162 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1231e-05 2020-09-08T15:54:00.162 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.162 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.162 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:00.162 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:57.115978+0000", 2020-09-08T15:54:00.162 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.463e-06 2020-09-08T15:54:00.163 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.163 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.163 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:00.163 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:57.115983+0000", 2020-09-08T15:54:00.163 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4694999999999997e-05 2020-09-08T15:54:00.164 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.164 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.164 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:00.164 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:57.116038+0000", 2020-09-08T15:54:00.164 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.6264000000000001e-05 2020-09-08T15:54:00.165 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.165 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.165 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:00.165 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:57.116064+0000", 2020-09-08T15:54:00.165 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.053111716000000003 2020-09-08T15:54:00.166 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.166 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.166 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:00.166 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:57.169176+0000", 2020-09-08T15:54:00.167 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.5392000000000002e-05 2020-09-08T15:54:00.167 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.167 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.167 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:00.167 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:45:57.169221+0000", 2020-09-08T15:54:00.168 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.053311198999999997 2020-09-08T15:54:00.168 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.168 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.168 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.168 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.169 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.169 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35381.0:158549 4.d e354/314 4:bce4dc56:::benchmark_data_smithi166_32054_object16407:head v 354'9897, mlcod=354'9895)", 2020-09-08T15:54:00.169 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:49:10.974090+0000", 2020-09-08T15:54:00.169 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 289.13778699800002, 2020-09-08T15:54:00.169 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.112964493, 2020-09-08T15:54:00.170 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:00.170 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:00.170 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:00.170 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.170 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:00.171 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974090+0000", 2020-09-08T15:54:00.171 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:00.171 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.171 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.171 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:00.171 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974090+0000", 2020-09-08T15:54:00.172 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.7972000000000002e-05 2020-09-08T15:54:00.172 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.172 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.172 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:00.172 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974128+0000", 2020-09-08T15:54:00.173 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0639999999999999e-06 2020-09-08T15:54:00.173 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.173 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.173 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:00.173 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974134+0000", 2020-09-08T15:54:00.174 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.1800000000000002e-06 2020-09-08T15:54:00.174 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.174 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.174 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:00.175 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974143+0000", 2020-09-08T15:54:00.175 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.8539999999999996e-06 2020-09-08T15:54:00.175 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.175 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.175 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:00.176 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974148+0000", 2020-09-08T15:54:00.176 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9451999999999999e-05 2020-09-08T15:54:00.176 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.177 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.177 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:00.177 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974207+0000", 2020-09-08T15:54:00.177 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.6696e-05 2020-09-08T15:54:00.178 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.178 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.178 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:00.178 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974234+0000", 2020-09-08T15:54:00.178 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0053667009999999998 2020-09-08T15:54:00.179 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.179 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.179 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:00.179 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.979601+0000", 2020-09-08T15:54:00.179 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.107453574 2020-09-08T15:54:00.180 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.180 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.180 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:00.180 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:11.087054+0000", 2020-09-08T15:54:00.181 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.112964493 2020-09-08T15:54:00.181 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.181 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.181 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.181 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.181 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.182 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35381.0:173566 4.d e358/314 4:b1bda86a:::benchmark_data_smithi166_32054_object31424:head v 358'10850, mlcod=358'10848)", 2020-09-08T15:54:00.182 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:49:25.745617+0000", 2020-09-08T15:54:00.182 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 274.36625970799997, 2020-09-08T15:54:00.182 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.047025483999999999, 2020-09-08T15:54:00.182 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:00.183 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:00.183 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:00.183 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.183 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:00.183 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:25.745617+0000", 2020-09-08T15:54:00.184 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:00.184 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.184 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.184 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:00.184 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:25.745617+0000", 2020-09-08T15:54:00.185 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.7506999999999999e-05 2020-09-08T15:54:00.185 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.185 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.185 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:00.185 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:25.745654+0000", 2020-09-08T15:54:00.186 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0630000000000002e-06 2020-09-08T15:54:00.186 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.186 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.187 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:00.187 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:25.745660+0000", 2020-09-08T15:54:00.188 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.0829999999999993e-06 2020-09-08T15:54:00.188 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.188 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.188 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:00.188 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:25.745670+0000", 2020-09-08T15:54:00.190 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.6099999999999999e-06 2020-09-08T15:54:00.190 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.190 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.190 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:00.191 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:25.745674+0000", 2020-09-08T15:54:00.191 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.033268842 2020-09-08T15:54:00.191 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.191 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.192 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:00.192 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:25.778943+0000", 2020-09-08T15:54:00.192 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.1858000000000003e-05 2020-09-08T15:54:00.192 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.192 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.193 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:00.193 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:25.779005+0000", 2020-09-08T15:54:00.193 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.013550797999999999 2020-09-08T15:54:00.193 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.193 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.194 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:00.194 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:25.792556+0000", 2020-09-08T15:54:00.194 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.6723000000000003e-05 2020-09-08T15:54:00.194 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.194 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.195 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:00.195 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:25.792642+0000", 2020-09-08T15:54:00.195 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.047025483999999999 2020-09-08T15:54:00.195 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.195 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.196 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.196 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.196 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.196 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35381.0:177575 4.e e359/314 4:7204d027:::benchmark_data_smithi166_32054_object35433:head v 359'11155, mlcod=359'11152)", 2020-09-08T15:54:00.197 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:49:29.720082+0000", 2020-09-08T15:54:00.197 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 270.39179428, 2020-09-08T15:54:00.197 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.048367474000000001, 2020-09-08T15:54:00.197 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:00.197 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:00.198 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:00.198 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.198 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:00.198 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.720082+0000", 2020-09-08T15:54:00.198 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:00.199 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.199 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.199 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:00.199 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.720082+0000", 2020-09-08T15:54:00.199 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000108624 2020-09-08T15:54:00.200 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.200 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.200 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:00.200 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.720191+0000", 2020-09-08T15:54:00.201 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3309000000000001e-05 2020-09-08T15:54:00.201 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.201 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.201 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:00.201 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.720204+0000", 2020-09-08T15:54:00.202 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.2960999999999999e-05 2020-09-08T15:54:00.202 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.202 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.202 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:00.202 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.720227+0000", 2020-09-08T15:54:00.203 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.6050000000000004e-06 2020-09-08T15:54:00.203 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.203 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.203 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:00.203 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.720237+0000", 2020-09-08T15:54:00.204 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00088012300000000002 2020-09-08T15:54:00.204 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.204 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.204 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:00.204 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.721117+0000", 2020-09-08T15:54:00.205 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.8958e-05 2020-09-08T15:54:00.205 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.205 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.205 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:00.206 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.721136+0000", 2020-09-08T15:54:00.206 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.047219070000000002 2020-09-08T15:54:00.206 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.206 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.206 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:00.207 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.768355+0000", 2020-09-08T15:54:00.207 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4823999999999999e-05 2020-09-08T15:54:00.207 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.207 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.207 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:00.208 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.768450+0000", 2020-09-08T15:54:00.208 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.048367474000000001 2020-09-08T15:54:00.208 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.208 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.208 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.209 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.209 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.209 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35381.0:177580 4.e e359/314 4:7a3b25e6:::benchmark_data_smithi166_32054_object35438:head v 359'11156, mlcod=359'11152)", 2020-09-08T15:54:00.209 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:49:29.724518+0000", 2020-09-08T15:54:00.209 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 270.38735861800001, 2020-09-08T15:54:00.210 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043990353000000003, 2020-09-08T15:54:00.210 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:00.210 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:00.210 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:00.210 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.211 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:00.211 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.724518+0000", 2020-09-08T15:54:00.211 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:00.211 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.211 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.212 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:00.212 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.724518+0000", 2020-09-08T15:54:00.212 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012638999999999999 2020-09-08T15:54:00.212 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.213 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.213 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:00.213 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.724644+0000", 2020-09-08T15:54:00.213 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4939999999999992e-06 2020-09-08T15:54:00.213 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.214 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.214 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:00.214 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.724654+0000", 2020-09-08T15:54:00.215 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5787e-05 2020-09-08T15:54:00.216 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.216 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.216 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:00.216 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.724670+0000", 2020-09-08T15:54:00.217 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.1750000000000002e-06 2020-09-08T15:54:00.217 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.217 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.217 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:00.217 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.724676+0000", 2020-09-08T15:54:00.218 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.001112519 2020-09-08T15:54:00.218 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.218 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.218 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:00.218 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.725788+0000", 2020-09-08T15:54:00.219 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.6767e-05 2020-09-08T15:54:00.219 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.219 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.219 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:00.219 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.725815+0000", 2020-09-08T15:54:00.220 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.042641643999999999 2020-09-08T15:54:00.220 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.220 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.220 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:00.220 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.768457+0000", 2020-09-08T15:54:00.221 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.1576999999999998e-05 2020-09-08T15:54:00.221 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.221 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.221 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:00.221 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.768508+0000", 2020-09-08T15:54:00.222 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043990353000000003 2020-09-08T15:54:00.222 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.222 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.222 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.222 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.223 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.223 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35381.0:203081 4.c e366/314 4:32e42a35:::benchmark_data_smithi166_32054_object60939:head v 366'12482, mlcod=366'12468)", 2020-09-08T15:54:00.223 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:49:54.956220+0000", 2020-09-08T15:54:00.223 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 245.155656308, 2020-09-08T15:54:00.223 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.046051688, 2020-09-08T15:54:00.223 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:00.224 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:00.224 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:00.224 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.224 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:00.224 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:54.956220+0000", 2020-09-08T15:54:00.225 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:00.225 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.225 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.225 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:00.225 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:54.956220+0000", 2020-09-08T15:54:00.226 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.7287999999999996e-05 2020-09-08T15:54:00.226 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.226 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.226 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:00.226 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:54.956298+0000", 2020-09-08T15:54:00.227 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.9040000000000003e-06 2020-09-08T15:54:00.227 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.227 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.227 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:00.227 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:54.956304+0000", 2020-09-08T15:54:00.227 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4090000000000001e-05 2020-09-08T15:54:00.228 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.228 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.228 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:00.228 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:54.956319+0000", 2020-09-08T15:54:00.228 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.9820000000000002e-06 2020-09-08T15:54:00.229 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.229 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.229 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:00.229 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:54.956323+0000", 2020-09-08T15:54:00.229 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.037550218000000003 2020-09-08T15:54:00.230 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.230 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.230 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:00.230 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:54.993873+0000", 2020-09-08T15:54:00.230 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.1723000000000001e-05 2020-09-08T15:54:00.231 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.231 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.231 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:00.231 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:54.993904+0000", 2020-09-08T15:54:00.231 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0083099460000000003 2020-09-08T15:54:00.231 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.232 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.232 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:00.232 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:55.002214+0000", 2020-09-08T15:54:00.232 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.7537000000000002e-05 2020-09-08T15:54:00.232 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.233 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.233 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:00.233 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:55.002272+0000", 2020-09-08T15:54:00.233 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.046051688 2020-09-08T15:54:00.233 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.234 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.234 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.234 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.234 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.234 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.15771.0:92706 5.c e417/396 5:3368ea65:::benchmark_data_smithi166_37820_object92705:head v 417'5783, mlcod=417'5781)", 2020-09-08T15:54:00.235 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:52:38.285778+0000", 2020-09-08T15:54:00.235 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 81.826098885999997, 2020-09-08T15:54:00.235 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.148506577, 2020-09-08T15:54:00.235 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:00.235 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:00.236 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:00.236 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.236 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:00.236 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285778+0000", 2020-09-08T15:54:00.236 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:00.236 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.237 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.237 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:00.237 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285778+0000", 2020-09-08T15:54:00.237 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.1017e-05 2020-09-08T15:54:00.237 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.238 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.238 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:00.238 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285859+0000", 2020-09-08T15:54:00.238 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.1399999999999997e-06 2020-09-08T15:54:00.238 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.239 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.239 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:00.239 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285865+0000", 2020-09-08T15:54:00.239 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4960000000000002e-06 2020-09-08T15:54:00.239 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.240 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.240 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:00.240 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285874+0000", 2020-09-08T15:54:00.240 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.5709999999999999e-06 2020-09-08T15:54:00.240 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.241 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.241 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:00.241 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285879+0000", 2020-09-08T15:54:00.241 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.6329999999999997e-05 2020-09-08T15:54:00.241 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.242 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.242 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:00.242 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285945+0000", 2020-09-08T15:54:00.242 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.5469e-05 2020-09-08T15:54:00.242 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.243 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.243 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:00.243 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285971+0000", 2020-09-08T15:54:00.243 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.14824900699999999 2020-09-08T15:54:00.243 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.243 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.244 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:00.244 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.434220+0000", 2020-09-08T15:54:00.244 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.4547000000000004e-05 2020-09-08T15:54:00.244 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.244 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.245 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:00.245 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.434284+0000", 2020-09-08T15:54:00.245 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.148506577 2020-09-08T15:54:00.245 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.245 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.246 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.251 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.251 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.251 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.15771.0:92708 5.c e417/396 5:39784271:::benchmark_data_smithi166_37820_object92707:head v 417'5784, mlcod=417'5781)", 2020-09-08T15:54:00.251 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:52:38.286728+0000", 2020-09-08T15:54:00.252 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 81.825148118000001, 2020-09-08T15:54:00.252 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.160917904, 2020-09-08T15:54:00.252 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:00.252 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:00.252 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:00.253 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.253 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:00.253 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.286728+0000", 2020-09-08T15:54:00.253 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:00.253 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.254 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.254 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:00.254 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.286728+0000", 2020-09-08T15:54:00.254 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000106149 2020-09-08T15:54:00.254 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.255 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.255 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:00.255 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.286835+0000", 2020-09-08T15:54:00.255 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.952e-06 2020-09-08T15:54:00.255 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.255 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.256 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:00.256 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.286844+0000", 2020-09-08T15:54:00.256 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4589999999999999e-05 2020-09-08T15:54:00.256 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.256 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.257 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:00.257 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.286858+0000", 2020-09-08T15:54:00.257 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.7769999999999999e-06 2020-09-08T15:54:00.257 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.257 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.258 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:00.258 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.286864+0000", 2020-09-08T15:54:00.258 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.14290330000000001 2020-09-08T15:54:00.258 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.258 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.259 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:00.259 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.429767+0000", 2020-09-08T15:54:00.259 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.6913999999999998e-05 2020-09-08T15:54:00.259 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.259 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.260 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:00.260 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.429814+0000", 2020-09-08T15:54:00.260 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.017743755 2020-09-08T15:54:00.260 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.260 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.261 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:00.261 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.447558+0000", 2020-09-08T15:54:00.261 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.8467000000000005e-05 2020-09-08T15:54:00.261 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.261 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.262 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:00.262 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.447646+0000", 2020-09-08T15:54:00.262 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.160917904 2020-09-08T15:54:00.262 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.262 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.262 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.263 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.263 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.263 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.15771.0:92717 5.f e417/396 5:f8e6b0a5:::benchmark_data_smithi166_37820_object92716:head v 417'5849, mlcod=417'5846)", 2020-09-08T15:54:00.263 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:52:38.288221+0000", 2020-09-08T15:54:00.263 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 81.823656091999993, 2020-09-08T15:54:00.264 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.14981187600000001, 2020-09-08T15:54:00.264 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:00.264 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:00.264 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:00.264 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.265 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:00.265 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.288221+0000", 2020-09-08T15:54:00.265 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:00.265 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.265 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.266 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:00.266 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.288221+0000", 2020-09-08T15:54:00.266 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013050599999999999 2020-09-08T15:54:00.266 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.266 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.267 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:00.267 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.288351+0000", 2020-09-08T15:54:00.267 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1793999999999999e-05 2020-09-08T15:54:00.267 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.267 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.267 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:00.268 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.288363+0000", 2020-09-08T15:54:00.268 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.14131466000000001 2020-09-08T15:54:00.268 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.268 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.268 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:00.269 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.429677+0000", 2020-09-08T15:54:00.269 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.6616000000000001e-05 2020-09-08T15:54:00.269 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.269 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.269 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:00.270 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.429704+0000", 2020-09-08T15:54:00.270 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011453199999999999 2020-09-08T15:54:00.270 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.270 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.270 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:00.271 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.429819+0000", 2020-09-08T15:54:00.271 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0995999999999997e-05 2020-09-08T15:54:00.271 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.271 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.271 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:00.271 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.429880+0000", 2020-09-08T15:54:00.272 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.008092106 2020-09-08T15:54:00.272 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.272 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.272 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:00.272 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.437972+0000", 2020-09-08T15:54:00.273 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0665999999999998e-05 2020-09-08T15:54:00.273 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.273 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.273 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:00.273 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.438032+0000", 2020-09-08T15:54:00.274 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.14981187600000001 2020-09-08T15:54:00.274 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.274 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.274 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.274 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.274 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.275 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.15771.0:119825 5.f e424/396 5:f52c6d29:::benchmark_data_smithi166_37820_object119824:head v 424'7512, mlcod=424'7509)", 2020-09-08T15:54:00.275 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:52:59.827902+0000", 2020-09-08T15:54:00.275 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 60.283974663999999, 2020-09-08T15:54:00.275 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043842725999999999, 2020-09-08T15:54:00.275 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:00.276 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:00.276 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:00.276 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.276 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:00.276 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.827902+0000", 2020-09-08T15:54:00.277 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:00.277 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.277 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.277 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:00.277 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.827902+0000", 2020-09-08T15:54:00.278 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010767700000000001 2020-09-08T15:54:00.278 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.278 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.278 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:00.278 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.828010+0000", 2020-09-08T15:54:00.278 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0261999999999999e-05 2020-09-08T15:54:00.279 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.279 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.279 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:00.279 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.828020+0000", 2020-09-08T15:54:00.279 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.8065000000000001e-05 2020-09-08T15:54:00.280 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.280 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.280 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:00.280 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.828038+0000", 2020-09-08T15:54:00.280 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3729999999999999e-06 2020-09-08T15:54:00.281 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.281 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.281 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:00.281 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.828047+0000", 2020-09-08T15:54:00.281 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.3152000000000002e-05 2020-09-08T15:54:00.282 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.282 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.282 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:00.282 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.828100+0000", 2020-09-08T15:54:00.282 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.7844e-05 2020-09-08T15:54:00.283 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.283 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.283 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:00.283 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.828128+0000", 2020-09-08T15:54:00.283 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043478430999999998 2020-09-08T15:54:00.283 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.284 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.284 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:00.284 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.871607+0000", 2020-09-08T15:54:00.284 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013792200000000001 2020-09-08T15:54:00.284 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.285 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.285 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:00.285 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.871745+0000", 2020-09-08T15:54:00.285 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043842725999999999 2020-09-08T15:54:00.285 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.286 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.286 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.286 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.286 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.286 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.15771.0:119832 5.1 e424/396 5:8ac9092b:::benchmark_data_smithi166_37820_object119831:head v 424'7592, mlcod=424'7590)", 2020-09-08T15:54:00.287 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:52:59.828603+0000", 2020-09-08T15:54:00.287 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 60.283273715999997, 2020-09-08T15:54:00.287 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043985637000000001, 2020-09-08T15:54:00.287 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:00.287 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:00.287 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:00.288 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.288 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:00.288 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.828603+0000", 2020-09-08T15:54:00.288 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:00.288 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.289 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.289 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:00.289 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.828603+0000", 2020-09-08T15:54:00.289 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010961199999999999 2020-09-08T15:54:00.289 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.290 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.290 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:00.290 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.828712+0000", 2020-09-08T15:54:00.290 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0917999999999999e-05 2020-09-08T15:54:00.290 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.291 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.291 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:00.291 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.828723+0000", 2020-09-08T15:54:00.291 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.8935000000000001e-05 2020-09-08T15:54:00.291 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.291 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.292 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:00.292 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.828742+0000", 2020-09-08T15:54:00.292 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2949999999999994e-06 2020-09-08T15:54:00.292 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.292 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.293 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:00.293 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.828751+0000", 2020-09-08T15:54:00.293 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.6656999999999997e-05 2020-09-08T15:54:00.293 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.293 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.294 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:00.294 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.828797+0000", 2020-09-08T15:54:00.294 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.8586e-05 2020-09-08T15:54:00.294 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.294 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.294 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:00.295 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.828826+0000", 2020-09-08T15:54:00.295 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043680135000000002 2020-09-08T15:54:00.295 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.295 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.296 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:00.296 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.872506+0000", 2020-09-08T15:54:00.296 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2498999999999997e-05 2020-09-08T15:54:00.296 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.296 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.296 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:00.297 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.872589+0000", 2020-09-08T15:54:00.297 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.043985637000000001 2020-09-08T15:54:00.297 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.297 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.297 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.298 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.298 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.298 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.15771.0:119830 5.f e424/396 5:f1ac1ada:::benchmark_data_smithi166_37820_object119829:head v 424'7513, mlcod=424'7510)", 2020-09-08T15:54:00.298 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:52:59.835106+0000", 2020-09-08T15:54:00.298 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 60.276770345999999, 2020-09-08T15:54:00.299 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.036974278999999999, 2020-09-08T15:54:00.299 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:00.299 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:00.299 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:00.299 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.300 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:00.300 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.835106+0000", 2020-09-08T15:54:00.300 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:00.300 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.300 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.300 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:00.301 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.835106+0000", 2020-09-08T15:54:00.301 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.8718999999999998e-05 2020-09-08T15:54:00.301 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.301 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.301 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:00.302 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.835165+0000", 2020-09-08T15:54:00.302 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.8709999999999999e-06 2020-09-08T15:54:00.302 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.302 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.302 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:00.303 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.835172+0000", 2020-09-08T15:54:00.303 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.9040000000000004e-06 2020-09-08T15:54:00.303 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.303 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.303 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:00.304 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.835182+0000", 2020-09-08T15:54:00.304 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2702e-05 2020-09-08T15:54:00.304 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.304 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.304 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:00.304 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.835194+0000", 2020-09-08T15:54:00.305 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.0906999999999999e-05 2020-09-08T15:54:00.305 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.305 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.305 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:00.305 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.835245+0000", 2020-09-08T15:54:00.306 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.5670000000000002e-05 2020-09-08T15:54:00.306 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.306 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.306 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:00.306 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.835281+0000", 2020-09-08T15:54:00.307 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.036735757000000001 2020-09-08T15:54:00.307 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.307 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.307 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:00.307 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.872017+0000", 2020-09-08T15:54:00.307 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3749000000000002e-05 2020-09-08T15:54:00.308 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.308 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.308 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:00.308 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:59.872081+0000", 2020-09-08T15:54:00.308 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.036974278999999999 2020-09-08T15:54:00.309 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.309 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.309 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.309 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.309 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.310 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.15771.0:149904 5.f e432/396 5:fc90b18b:::benchmark_data_smithi166_37820_object5482:head v 432'9378, mlcod=432'9375)", 2020-09-08T15:54:00.310 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:53:29.848873+0000", 2020-09-08T15:54:00.310 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 30.263003433000002, 2020-09-08T15:54:00.310 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.037569518000000003, 2020-09-08T15:54:00.310 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:00.310 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:00.311 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:00.311 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.311 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:00.311 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:29.848873+0000", 2020-09-08T15:54:00.311 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:00.312 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.312 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.312 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:00.312 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:29.848873+0000", 2020-09-08T15:54:00.312 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.3856000000000005e-05 2020-09-08T15:54:00.313 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.313 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.313 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:00.313 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:29.848957+0000", 2020-09-08T15:54:00.313 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0251000000000001e-05 2020-09-08T15:54:00.314 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.314 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.314 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:00.314 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:29.848967+0000", 2020-09-08T15:54:00.314 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5325999999999999e-05 2020-09-08T15:54:00.314 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.315 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.315 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:00.315 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:29.848983+0000", 2020-09-08T15:54:00.315 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4929999999999998e-06 2020-09-08T15:54:00.315 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.316 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.316 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:00.316 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:29.848991+0000", 2020-09-08T15:54:00.316 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00040406399999999999 2020-09-08T15:54:00.316 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.317 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.317 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:00.317 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:29.849395+0000", 2020-09-08T15:54:00.317 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.1767000000000003e-05 2020-09-08T15:54:00.317 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.318 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.318 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:00.318 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:29.849427+0000", 2020-09-08T15:54:00.318 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.036908709999999997 2020-09-08T15:54:00.318 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.318 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.319 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:00.319 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:29.886336+0000", 2020-09-08T15:54:00.319 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000107051 2020-09-08T15:54:00.319 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.319 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.320 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:00.320 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:29.886443+0000", 2020-09-08T15:54:00.320 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.037569518000000003 2020-09-08T15:54:00.320 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.320 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.321 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.321 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.321 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.321 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.15771.0:149911 5.c e432/396 5:3160f8aa:::benchmark_data_smithi166_37820_object5489:head v 432'9269, mlcod=432'9266)", 2020-09-08T15:54:00.321 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:53:29.852501+0000", 2020-09-08T15:54:00.322 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 30.259375708, 2020-09-08T15:54:00.322 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.039331098000000002, 2020-09-08T15:54:00.322 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:00.322 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:00.322 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:00.322 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.323 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:00.323 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:29.852501+0000", 2020-09-08T15:54:00.323 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:00.323 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.323 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.324 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:00.324 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:29.852501+0000", 2020-09-08T15:54:00.324 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.6828000000000005e-05 2020-09-08T15:54:00.324 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.324 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.325 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:00.325 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:29.852568+0000", 2020-09-08T15:54:00.325 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1389999999999999e-05 2020-09-08T15:54:00.325 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.325 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.326 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:00.326 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:29.852579+0000", 2020-09-08T15:54:00.326 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6475000000000001e-05 2020-09-08T15:54:00.326 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.326 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.326 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:00.327 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:29.852596+0000", 2020-09-08T15:54:00.327 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.8179999999999998e-06 2020-09-08T15:54:00.327 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.327 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.327 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:00.328 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:29.852602+0000", 2020-09-08T15:54:00.328 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.0673000000000001e-05 2020-09-08T15:54:00.328 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.328 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.328 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:00.329 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:29.852653+0000", 2020-09-08T15:54:00.329 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.4977000000000001e-05 2020-09-08T15:54:00.329 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.329 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.329 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:00.329 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:29.852688+0000", 2020-09-08T15:54:00.330 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.038975106000000002 2020-09-08T15:54:00.330 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.330 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.330 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:00.330 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:29.891663+0000", 2020-09-08T15:54:00.331 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016883100000000001 2020-09-08T15:54:00.331 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.331 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.331 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:00.331 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:29.891832+0000", 2020-09-08T15:54:00.332 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.039331098000000002 2020-09-08T15:54:00.332 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.332 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.332 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.332 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.332 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.333 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.15771.0:167294 5.c e437/396 5:352b230d:::benchmark_data_smithi166_37820_object22872:head v 437'10377, mlcod=437'10375)", 2020-09-08T15:54:00.333 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:53:49.741939+0000", 2020-09-08T15:54:00.333 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 10.369937199000001, 2020-09-08T15:54:00.333 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.047426477000000002, 2020-09-08T15:54:00.333 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:00.334 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:00.334 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:00.334 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.334 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:00.334 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:49.741939+0000", 2020-09-08T15:54:00.335 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:00.335 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.335 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.335 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:00.335 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:49.741939+0000", 2020-09-08T15:54:00.336 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.0832000000000002e-05 2020-09-08T15:54:00.336 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.336 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.336 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:00.336 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:49.742010+0000", 2020-09-08T15:54:00.337 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.9199999999999999e-06 2020-09-08T15:54:00.337 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.337 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.337 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:00.337 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:49.742020+0000", 2020-09-08T15:54:00.337 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6008e-05 2020-09-08T15:54:00.338 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.338 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.338 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:00.338 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:49.742036+0000", 2020-09-08T15:54:00.338 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.61e-06 2020-09-08T15:54:00.339 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.339 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.339 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:00.339 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:49.742044+0000", 2020-09-08T15:54:00.339 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.021576015 2020-09-08T15:54:00.340 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.340 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.340 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:00.340 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:49.763620+0000", 2020-09-08T15:54:00.340 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.1784000000000002e-05 2020-09-08T15:54:00.340 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.341 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.341 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:00.341 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:49.763702+0000", 2020-09-08T15:54:00.341 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.025588234000000001 2020-09-08T15:54:00.341 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.342 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.342 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:00.342 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:49.789290+0000", 2020-09-08T15:54:00.342 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.6074000000000004e-05 2020-09-08T15:54:00.342 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.343 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.343 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:00.343 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:49.789366+0000", 2020-09-08T15:54:00.343 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.047426477000000002 2020-09-08T15:54:00.343 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.344 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.344 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.344 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.344 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.344 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.15771.0:172452 5.c e439/396 5:38111611:::benchmark_data_smithi166_37820_object28030:head v 439'10711, mlcod=439'10707)", 2020-09-08T15:54:00.344 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:53:54.845966+0000", 2020-09-08T15:54:00.345 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 5.2659108039999998, 2020-09-08T15:54:00.345 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.050158055999999999, 2020-09-08T15:54:00.345 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:00.345 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:00.345 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:00.346 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.346 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:00.346 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.845966+0000", 2020-09-08T15:54:00.346 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:00.346 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.347 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.347 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:00.347 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.845966+0000", 2020-09-08T15:54:00.347 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000101652 2020-09-08T15:54:00.347 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.347 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.348 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:00.348 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.846067+0000", 2020-09-08T15:54:00.348 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2852e-05 2020-09-08T15:54:00.348 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.348 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.349 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:00.349 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.846080+0000", 2020-09-08T15:54:00.349 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.0873000000000002e-05 2020-09-08T15:54:00.349 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.349 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.350 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:00.350 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.846101+0000", 2020-09-08T15:54:00.350 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4885999999999999e-05 2020-09-08T15:54:00.350 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.350 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.350 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:00.351 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.846116+0000", 2020-09-08T15:54:00.351 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.001596126 2020-09-08T15:54:00.351 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.351 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.351 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:00.352 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.847712+0000", 2020-09-08T15:54:00.352 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.7428e-05 2020-09-08T15:54:00.352 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.352 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.352 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:00.353 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.847730+0000", 2020-09-08T15:54:00.353 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.048316977999999997 2020-09-08T15:54:00.353 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.353 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.353 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:00.354 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.896047+0000", 2020-09-08T15:54:00.354 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.7261000000000004e-05 2020-09-08T15:54:00.354 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.354 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.354 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:00.355 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.896124+0000", 2020-09-08T15:54:00.355 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.050158055999999999 2020-09-08T15:54:00.355 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.355 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.355 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.355 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.356 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.356 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.15771.0:172456 5.f e439/396 5:fb8bedc2:::benchmark_data_smithi166_37820_object28034:head v 439'10799, mlcod=439'10797)", 2020-09-08T15:54:00.356 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:53:54.850350+0000", 2020-09-08T15:54:00.356 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 5.2615265170000001, 2020-09-08T15:54:00.356 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.046998437999999997, 2020-09-08T15:54:00.357 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:00.357 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:00.357 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:00.357 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.357 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:00.358 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.850350+0000", 2020-09-08T15:54:00.358 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:00.358 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.358 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.358 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:00.359 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.850350+0000", 2020-09-08T15:54:00.359 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.0679999999999997e-05 2020-09-08T15:54:00.359 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.359 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.359 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:00.359 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.850391+0000", 2020-09-08T15:54:00.360 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.353e-06 2020-09-08T15:54:00.360 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.360 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.360 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:00.360 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.850397+0000", 2020-09-08T15:54:00.361 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0163000000000001e-05 2020-09-08T15:54:00.361 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.361 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.361 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:00.361 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.850407+0000", 2020-09-08T15:54:00.362 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.6879999999999996e-06 2020-09-08T15:54:00.362 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.362 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.362 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:00.362 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.850413+0000", 2020-09-08T15:54:00.362 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.9942999999999999e-05 2020-09-08T15:54:00.363 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.363 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.363 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:00.363 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.850463+0000", 2020-09-08T15:54:00.363 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.5471000000000001e-05 2020-09-08T15:54:00.364 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.364 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.364 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:00.364 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.850488+0000", 2020-09-08T15:54:00.364 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.046762748999999999 2020-09-08T15:54:00.365 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.365 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.365 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:00.365 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.897251+0000", 2020-09-08T15:54:00.365 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.7391000000000003e-05 2020-09-08T15:54:00.366 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:00.366 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:00.366 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:00.366 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.897349+0000", 2020-09-08T15:54:00.366 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.046998437999999997 2020-09-08T15:54:00.366 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.367 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.367 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.367 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:00.367 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:00.367 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:54:00.368 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:00.369 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:53:59 smithi104 bash[17588]: debug 2020-09-08T15:53:59.812+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:00.389 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:54:00.417 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:54:00.445 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:54:00.478 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:54:00.485 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:54:00 smithi104 bash[15273]: debug 2020-09-08T15:54:00.484+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:00.487 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:00.532 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:54:00.566 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:54:00.593 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:00.605 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:54:00.646 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:54:00.692 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:54:00.736 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:54:00.777 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:54:00.811 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:54:00.813 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:54:00.813 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:54:00.814 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:54:00.814 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:54:00.814 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:54:00.814 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:54:00.814 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:54:00.815 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:54:00.815 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:54:00.815 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:54:00.815 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:54:00.815 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:54:00.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:00 smithi055 bash[9296]: cluster 2020-09-08T15:53:59.853011+0000 mgr.x (mgr.24118) 800 : cluster [DBG] pgmap v1155: 17 pgs: 17 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 993 op/s 2020-09-08T15:54:00.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:00 smithi055 bash[9296]: audit 2020-09-08T15:54:00.341978+0000 mon.a (mon.0) 1607 : audit [DBG] from='client.? v1:172.21.15.104:0/1055213684' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:54:01.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:00 smithi166 bash[9446]: cluster 2020-09-08T15:53:59.853011+0000 mgr.x (mgr.24118) 800 : cluster [DBG] pgmap v1155: 17 pgs: 17 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 993 op/s 2020-09-08T15:54:01.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:00 smithi166 bash[9446]: audit 2020-09-08T15:54:00.341978+0000 mon.a (mon.0) 1607 : audit [DBG] from='client.? v1:172.21.15.104:0/1055213684' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:54:01.132 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:01.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:00 smithi104 bash[9203]: cluster 2020-09-08T15:53:59.853011+0000 mgr.x (mgr.24118) 800 : cluster [DBG] pgmap v1155: 17 pgs: 17 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 993 op/s 2020-09-08T15:54:01.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:00 smithi104 bash[9203]: audit 2020-09-08T15:54:00.341978+0000 mon.a (mon.0) 1607 : audit [DBG] from='client.? v1:172.21.15.104:0/1055213684' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:54:01.236 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:01.261 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2020-09-08T15:54:01.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:54:01 smithi055 bash[11956]: debug 2020-09-08T15:54:01.131+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:01.424 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:54:01.823 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:54:02.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:01 smithi166 bash[9446]: audit 2020-09-08T15:54:00.838053+0000 mon.a (mon.0) 1608 : audit [INF] from='client.? v1:172.21.15.104:0/4057106111' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:54:02.192 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:01 smithi166 bash[9446]: cluster 2020-09-08T15:54:00.913310+0000 osd.9 (osd.9) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:02.193 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:01 smithi166 bash[9446]: cluster 2020-09-08T15:54:00.913383+0000 osd.9 (osd.9) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:02.194 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:01 smithi055 bash[9296]: audit 2020-09-08T15:54:00.838053+0000 mon.a (mon.0) 1608 : audit [INF] from='client.? v1:172.21.15.104:0/4057106111' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:54:02.195 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:01 smithi055 bash[9296]: cluster 2020-09-08T15:54:00.913310+0000 osd.9 (osd.9) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:02.195 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:01 smithi055 bash[9296]: cluster 2020-09-08T15:54:00.913383+0000 osd.9 (osd.9) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:02.196 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:01 smithi104 bash[9203]: audit 2020-09-08T15:54:00.838053+0000 mon.a (mon.0) 1608 : audit [INF] from='client.? v1:172.21.15.104:0/4057106111' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:54:02.196 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:01 smithi104 bash[9203]: cluster 2020-09-08T15:54:00.913310+0000 osd.9 (osd.9) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:02.196 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:01 smithi104 bash[9203]: cluster 2020-09-08T15:54:00.913383+0000 osd.9 (osd.9) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:02.197 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:02.300 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:54:02.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:54:02 smithi104 bash[22305]: debug 2020-09-08T15:54:02.136+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:02.429 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:54:02.533 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:02.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:54:02 smithi166 bash[11928]: debug 2020-09-08T15:54:02.431+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:02.654 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:02.757 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:02.970 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:03.050 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:54:02 smithi055 bash[11956]: debug 2020-09-08T15:54:02.971+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:03.050 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:02 smithi055 bash[9296]: cluster 2020-09-08T15:54:01.810422+0000 mon.a (mon.0) 1609 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:54:03.050 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:02 smithi055 bash[9296]: audit 2020-09-08T15:54:01.827135+0000 mon.a (mon.0) 1610 : audit [INF] from='client.? v1:172.21.15.104:0/4057106111' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:54:03.051 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:02 smithi055 bash[9296]: cluster 2020-09-08T15:54:01.827180+0000 mon.a (mon.0) 1611 : cluster [DBG] osdmap e440: 12 total, 12 up, 12 in 2020-09-08T15:54:03.051 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:02 smithi055 bash[9296]: cluster 2020-09-08T15:54:01.853367+0000 mgr.x (mgr.24118) 801 : cluster [DBG] pgmap v1157: 17 pgs: 17 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 749 op/s 2020-09-08T15:54:03.051 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:02 smithi055 bash[9296]: cluster 2020-09-08T15:54:02.056415+0000 osd.10 (osd.10) 50 : cluster [DBG] 5.4 scrub starts 2020-09-08T15:54:03.073 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:03.082 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:02 smithi104 bash[9203]: cluster 2020-09-08T15:54:01.810422+0000 mon.a (mon.0) 1609 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:54:03.082 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:02 smithi104 bash[9203]: audit 2020-09-08T15:54:01.827135+0000 mon.a (mon.0) 1610 : audit [INF] from='client.? v1:172.21.15.104:0/4057106111' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:54:03.082 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:02 smithi104 bash[9203]: cluster 2020-09-08T15:54:01.827180+0000 mon.a (mon.0) 1611 : cluster [DBG] osdmap e440: 12 total, 12 up, 12 in 2020-09-08T15:54:03.083 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:02 smithi104 bash[9203]: cluster 2020-09-08T15:54:01.853367+0000 mgr.x (mgr.24118) 801 : cluster [DBG] pgmap v1157: 17 pgs: 17 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 749 op/s 2020-09-08T15:54:03.083 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:02 smithi104 bash[9203]: cluster 2020-09-08T15:54:02.056415+0000 osd.10 (osd.10) 50 : cluster [DBG] 5.4 scrub starts 2020-09-08T15:54:03.119 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:54:02 smithi166 bash[16593]: debug 2020-09-08T15:54:02.655+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:03.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:02 smithi166 bash[9446]: cluster 2020-09-08T15:54:01.810422+0000 mon.a (mon.0) 1609 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:54:03.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:02 smithi166 bash[9446]: audit 2020-09-08T15:54:01.827135+0000 mon.a (mon.0) 1610 : audit [INF] from='client.? v1:172.21.15.104:0/4057106111' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:54:03.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:02 smithi166 bash[9446]: cluster 2020-09-08T15:54:01.827180+0000 mon.a (mon.0) 1611 : cluster [DBG] osdmap e440: 12 total, 12 up, 12 in 2020-09-08T15:54:03.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:02 smithi166 bash[9446]: cluster 2020-09-08T15:54:01.853367+0000 mgr.x (mgr.24118) 801 : cluster [DBG] pgmap v1157: 17 pgs: 17 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 749 op/s 2020-09-08T15:54:03.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:02 smithi166 bash[9446]: cluster 2020-09-08T15:54:02.056415+0000 osd.10 (osd.10) 50 : cluster [DBG] 5.4 scrub starts 2020-09-08T15:54:03.231 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:03.335 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:54:03.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:54:03 smithi104 bash[22305]: debug 2020-09-08T15:54:03.228+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:03.446 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:54:03.553 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:03.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:54:03 smithi166 bash[14254]: debug 2020-09-08T15:54:03.447+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:03.670 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:03.773 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:54:03.832 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:03 smithi104 bash[9203]: cluster 2020-09-08T15:54:02.708286+0000 osd.2 (osd.2) 60 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:03.833 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:03 smithi104 bash[9203]: cluster 2020-09-08T15:54:02.708369+0000 osd.2 (osd.2) 61 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:03.833 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:03 smithi104 bash[9203]: cluster 2020-09-08T15:54:02.839082+0000 mon.a (mon.0) 1612 : cluster [DBG] osdmap e441: 12 total, 12 up, 12 in 2020-09-08T15:54:03.833 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:54:03 smithi104 bash[19922]: debug 2020-09-08T15:54:03.664+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:03.913 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:54:04.020 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:04.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:03 smithi166 bash[9446]: cluster 2020-09-08T15:54:02.708286+0000 osd.2 (osd.2) 60 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:04.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:03 smithi166 bash[9446]: cluster 2020-09-08T15:54:02.708369+0000 osd.2 (osd.2) 61 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:04.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:03 smithi166 bash[9446]: cluster 2020-09-08T15:54:02.839082+0000 mon.a (mon.0) 1612 : cluster [DBG] osdmap e441: 12 total, 12 up, 12 in 2020-09-08T15:54:04.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:03 smithi055 bash[9296]: cluster 2020-09-08T15:54:02.708286+0000 osd.2 (osd.2) 60 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:04.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:03 smithi055 bash[9296]: cluster 2020-09-08T15:54:02.708369+0000 osd.2 (osd.2) 61 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:04.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:03 smithi055 bash[9296]: cluster 2020-09-08T15:54:02.839082+0000 mon.a (mon.0) 1612 : cluster [DBG] osdmap e441: 12 total, 12 up, 12 in 2020-09-08T15:54:04.175 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:54:03 smithi104 bash[17588]: debug 2020-09-08T15:54:03.912+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:04.247 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:04.350 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:04.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:54:04 smithi055 bash[16646]: debug 2020-09-08T15:54:04.247+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:04.842 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:04.948 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:04.969 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:54:05.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:04 smithi166 bash[9446]: cluster 2020-09-08T15:54:03.853829+0000 mgr.x (mgr.24118) 802 : cluster [DBG] pgmap v1159: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.21k op/s 2020-09-08T15:54:05.127 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:04 smithi055 bash[9296]: cluster 2020-09-08T15:54:03.853829+0000 mgr.x (mgr.24118) 802 : cluster [DBG] pgmap v1159: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.21k op/s 2020-09-08T15:54:05.127 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:54:04 smithi055 bash[16646]: debug 2020-09-08T15:54:04.839+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:05.128 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:05.136 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:54:05.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:04 smithi104 bash[9203]: cluster 2020-09-08T15:54:03.853829+0000 mgr.x (mgr.24118) 802 : cluster [DBG] pgmap v1159: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.21k op/s 2020-09-08T15:54:05.201 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:54:05.231 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:05.370 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:54:05.410 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:54:05 smithi055 bash[19003]: debug 2020-09-08T15:54:05.127+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:05.751 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:05.854 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:06.066 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:54:05 smithi055 bash[16646]: debug 2020-09-08T15:54:05.750+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:06.348 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:54:06.349 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:54:06.430 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:06.533 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:06.660 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:06.765 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:54:06.902 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:54:06 smithi104 bash[15273]: debug 2020-09-08T15:54:06.428+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:06.903 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:54:06 smithi055 bash[16646]: debug 2020-09-08T15:54:06.658+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:06.910 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:54:07.017 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:07.026 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:06 smithi166 bash[9446]: cluster 2020-09-08T15:54:05.854281+0000 mgr.x (mgr.24118) 803 : cluster [DBG] pgmap v1160: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 730 op/s 2020-09-08T15:54:07.027 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:54:06 smithi166 bash[18960]: debug 2020-09-08T15:54:06.911+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:07.141 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:07.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:06 smithi055 bash[9296]: cluster 2020-09-08T15:54:05.854281+0000 mgr.x (mgr.24118) 803 : cluster [DBG] pgmap v1160: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 730 op/s 2020-09-08T15:54:07.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:06 smithi104 bash[9203]: cluster 2020-09-08T15:54:05.854281+0000 mgr.x (mgr.24118) 803 : cluster [DBG] pgmap v1160: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 730 op/s 2020-09-08T15:54:07.255 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:54:07.366 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:54:07 smithi166 bash[16593]: debug 2020-09-08T15:54:07.143+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:07.421 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2020-09-08T15:54:07.481 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:54:07.545 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:54:07.588 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:54:07.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:54:07 smithi104 bash[17588]: debug 2020-09-08T15:54:07.480+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:07.714 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:54:07.817 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:07.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:54:07 smithi166 bash[11928]: debug 2020-09-08T15:54:07.715+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:07.934 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:08.038 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:08.045 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:54:07 smithi104 bash[15273]: debug 2020-09-08T15:54:07.932+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:08.238 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:08.342 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:54:08.352 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:54:08 smithi104 bash[22305]: debug 2020-09-08T15:54:08.236+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:09.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:08 smithi055 bash[9296]: cluster 2020-09-08T15:54:07.854727+0000 mgr.x (mgr.24118) 804 : cluster [DBG] pgmap v1161: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T15:54:09.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:08 smithi104 bash[9203]: cluster 2020-09-08T15:54:07.854727+0000 mgr.x (mgr.24118) 804 : cluster [DBG] pgmap v1161: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T15:54:09.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:08 smithi166 bash[9446]: cluster 2020-09-08T15:54:07.854727+0000 mgr.x (mgr.24118) 804 : cluster [DBG] pgmap v1161: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T15:54:09.474 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:54:09.576 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:54:09.697 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:54:09.800 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:09.867 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:54:09 smithi166 bash[14254]: debug 2020-09-08T15:54:09.695+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:09.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:54:09 smithi104 bash[17588]: debug 2020-09-08T15:54:09.464+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:09.952 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:10.055 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:54:10.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:54:09 smithi055 bash[19003]: debug 2020-09-08T15:54:09.950+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:10.166 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:54:10.270 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:10.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:54:10 smithi166 bash[11928]: debug 2020-09-08T15:54:10.167+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:10.672 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:10.774 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:11.054 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:11.056 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:10 smithi104 bash[9203]: cluster 2020-09-08T15:54:09.855201+0000 mgr.x (mgr.24118) 805 : cluster [DBG] pgmap v1162: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.16k op/s 2020-09-08T15:54:11.056 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:54:11 smithi104 bash[15273]: debug 2020-09-08T15:54:11.052+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:11.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:10 smithi166 bash[9446]: cluster 2020-09-08T15:54:09.855201+0000 mgr.x (mgr.24118) 805 : cluster [DBG] pgmap v1162: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.16k op/s 2020-09-08T15:54:11.161 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:54:11.163 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:54:10 smithi055 bash[11956]: debug 2020-09-08T15:54:10.670+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:11.163 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:10 smithi055 bash[9296]: cluster 2020-09-08T15:54:09.855201+0000 mgr.x (mgr.24118) 805 : cluster [DBG] pgmap v1162: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.16k op/s 2020-09-08T15:54:11.830 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:11 smithi104 bash[9203]: audit 2020-09-08T15:54:11.793021+0000 mon.a (mon.0) 1613 : audit [DBG] from='client.? v1:172.21.15.104:0/980933926' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:54:11.854 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:54:11.892 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:54:11.892 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:54:11.893 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:54:11.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:11 smithi055 bash[9296]: audit 2020-09-08T15:54:11.793021+0000 mon.a (mon.0) 1613 : audit [DBG] from='client.? v1:172.21.15.104:0/980933926' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:54:11.946 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:54:12.004 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:54:12.050 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:54:12.093 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:54:12.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:11 smithi166 bash[9446]: audit 2020-09-08T15:54:11.793021+0000 mon.a (mon.0) 1613 : audit [DBG] from='client.? v1:172.21.15.104:0/980933926' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:54:12.145 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:54:12.198 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:54:12.239 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:54:12.267 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:54:12.308 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:54:12.351 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:54:12.353 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:54:12.353 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:54:12.354 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:54:12.354 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:54:12.354 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:54:12.354 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:54:12.354 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:54:12.355 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:54:12.355 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:54:12.355 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:54:12.355 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:54:12.355 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:54:12.839 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:12 smithi104 bash[9203]: cluster 2020-09-08T15:54:11.345410+0000 osd.7 (osd.7) 85 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:12.839 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:12 smithi104 bash[9203]: cluster 2020-09-08T15:54:11.345496+0000 osd.7 (osd.7) 86 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:12.839 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:12 smithi104 bash[9203]: cluster 2020-09-08T15:54:11.831945+0000 mon.a (mon.0) 1614 : cluster [DBG] osdmap e442: 12 total, 12 up, 12 in 2020-09-08T15:54:12.840 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:12 smithi104 bash[9203]: cluster 2020-09-08T15:54:11.855713+0000 mgr.x (mgr.24118) 806 : cluster [DBG] pgmap v1164: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 788 op/s 2020-09-08T15:54:12.840 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:12 smithi104 bash[9203]: audit 2020-09-08T15:54:12.236368+0000 mon.c (mon.2) 65 : audit [INF] from='client.? v1:172.21.15.104:0/2080867200' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:54:12.840 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:12 smithi104 bash[9203]: audit 2020-09-08T15:54:12.239197+0000 mon.a (mon.0) 1615 : audit [INF] from='client.35750 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:54:12.847 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:54:12.870 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2020-09-08T15:54:13.034 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:54:13.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:12 smithi166 bash[9446]: cluster 2020-09-08T15:54:11.345410+0000 osd.7 (osd.7) 85 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:13.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:12 smithi166 bash[9446]: cluster 2020-09-08T15:54:11.345496+0000 osd.7 (osd.7) 86 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:13.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:12 smithi166 bash[9446]: cluster 2020-09-08T15:54:11.831945+0000 mon.a (mon.0) 1614 : cluster [DBG] osdmap e442: 12 total, 12 up, 12 in 2020-09-08T15:54:13.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:12 smithi166 bash[9446]: cluster 2020-09-08T15:54:11.855713+0000 mgr.x (mgr.24118) 806 : cluster [DBG] pgmap v1164: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 788 op/s 2020-09-08T15:54:13.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:12 smithi166 bash[9446]: audit 2020-09-08T15:54:12.236368+0000 mon.c (mon.2) 65 : audit [INF] from='client.? v1:172.21.15.104:0/2080867200' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:54:13.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:12 smithi166 bash[9446]: audit 2020-09-08T15:54:12.239197+0000 mon.a (mon.0) 1615 : audit [INF] from='client.35750 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:54:13.182 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:12 smithi055 bash[9296]: cluster 2020-09-08T15:54:11.345410+0000 osd.7 (osd.7) 85 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:13.182 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:12 smithi055 bash[9296]: cluster 2020-09-08T15:54:11.345496+0000 osd.7 (osd.7) 86 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:13.182 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:12 smithi055 bash[9296]: cluster 2020-09-08T15:54:11.831945+0000 mon.a (mon.0) 1614 : cluster [DBG] osdmap e442: 12 total, 12 up, 12 in 2020-09-08T15:54:13.183 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:12 smithi055 bash[9296]: cluster 2020-09-08T15:54:11.855713+0000 mgr.x (mgr.24118) 806 : cluster [DBG] pgmap v1164: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 788 op/s 2020-09-08T15:54:13.183 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:12 smithi055 bash[9296]: audit 2020-09-08T15:54:12.236368+0000 mon.c (mon.2) 65 : audit [INF] from='client.? v1:172.21.15.104:0/2080867200' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:54:13.183 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:12 smithi055 bash[9296]: audit 2020-09-08T15:54:12.239197+0000 mon.a (mon.0) 1615 : audit [INF] from='client.35750 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:54:14.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:13 smithi166 bash[9446]: cluster 2020-09-08T15:54:12.836195+0000 mon.a (mon.0) 1616 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:54:14.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:13 smithi166 bash[9446]: cluster 2020-09-08T15:54:12.836230+0000 mon.a (mon.0) 1617 : cluster [INF] Cluster is now healthy 2020-09-08T15:54:14.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:13 smithi166 bash[9446]: audit 2020-09-08T15:54:12.847058+0000 mon.a (mon.0) 1618 : audit [INF] from='client.35750 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:54:14.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:13 smithi166 bash[9446]: cluster 2020-09-08T15:54:12.847112+0000 mon.a (mon.0) 1619 : cluster [DBG] osdmap e443: 12 total, 12 up, 12 in 2020-09-08T15:54:14.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:13 smithi055 bash[9296]: cluster 2020-09-08T15:54:12.836195+0000 mon.a (mon.0) 1616 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:54:14.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:13 smithi055 bash[9296]: cluster 2020-09-08T15:54:12.836230+0000 mon.a (mon.0) 1617 : cluster [INF] Cluster is now healthy 2020-09-08T15:54:14.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:13 smithi055 bash[9296]: audit 2020-09-08T15:54:12.847058+0000 mon.a (mon.0) 1618 : audit [INF] from='client.35750 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:54:14.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:13 smithi055 bash[9296]: cluster 2020-09-08T15:54:12.847112+0000 mon.a (mon.0) 1619 : cluster [DBG] osdmap e443: 12 total, 12 up, 12 in 2020-09-08T15:54:14.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:13 smithi104 bash[9203]: cluster 2020-09-08T15:54:12.836195+0000 mon.a (mon.0) 1616 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:54:14.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:13 smithi104 bash[9203]: cluster 2020-09-08T15:54:12.836230+0000 mon.a (mon.0) 1617 : cluster [INF] Cluster is now healthy 2020-09-08T15:54:14.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:13 smithi104 bash[9203]: audit 2020-09-08T15:54:12.847058+0000 mon.a (mon.0) 1618 : audit [INF] from='client.35750 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:54:14.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:13 smithi104 bash[9203]: cluster 2020-09-08T15:54:12.847112+0000 mon.a (mon.0) 1619 : cluster [DBG] osdmap e443: 12 total, 12 up, 12 in 2020-09-08T15:54:14.344 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:54:14.446 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:14.568 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:14.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:54:14 smithi166 bash[11928]: debug 2020-09-08T15:54:14.339+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:14.672 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:14.822 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:14.849 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:54:14 smithi055 bash[11956]: debug 2020-09-08T15:54:14.566+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:14.849 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:14 smithi055 bash[9296]: cluster 2020-09-08T15:54:13.379876+0000 osd.0 (osd.0) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:14.850 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:14 smithi055 bash[9296]: cluster 2020-09-08T15:54:13.379944+0000 osd.0 (osd.0) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:14.850 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:14 smithi055 bash[9296]: cluster 2020-09-08T15:54:13.851104+0000 mon.a (mon.0) 1620 : cluster [DBG] osdmap e444: 12 total, 12 up, 12 in 2020-09-08T15:54:14.850 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:14 smithi055 bash[9296]: cluster 2020-09-08T15:54:13.856168+0000 mgr.x (mgr.24118) 807 : cluster [DBG] pgmap v1167: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 903 op/s 2020-09-08T15:54:14.930 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:14.941 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:14 smithi104 bash[9203]: cluster 2020-09-08T15:54:13.379876+0000 osd.0 (osd.0) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:14.941 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:14 smithi104 bash[9203]: cluster 2020-09-08T15:54:13.379944+0000 osd.0 (osd.0) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:14.941 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:14 smithi104 bash[9203]: cluster 2020-09-08T15:54:13.851104+0000 mon.a (mon.0) 1620 : cluster [DBG] osdmap e444: 12 total, 12 up, 12 in 2020-09-08T15:54:14.941 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:14 smithi104 bash[9203]: cluster 2020-09-08T15:54:13.856168+0000 mgr.x (mgr.24118) 807 : cluster [DBG] pgmap v1167: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 903 op/s 2020-09-08T15:54:14.942 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:54:14 smithi104 bash[15273]: debug 2020-09-08T15:54:14.816+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:15.095 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:15.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:14 smithi166 bash[9446]: cluster 2020-09-08T15:54:13.379876+0000 osd.0 (osd.0) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:15.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:14 smithi166 bash[9446]: cluster 2020-09-08T15:54:13.379944+0000 osd.0 (osd.0) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:15.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:14 smithi166 bash[9446]: cluster 2020-09-08T15:54:13.851104+0000 mon.a (mon.0) 1620 : cluster [DBG] osdmap e444: 12 total, 12 up, 12 in 2020-09-08T15:54:15.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:14 smithi166 bash[9446]: cluster 2020-09-08T15:54:13.856168+0000 mgr.x (mgr.24118) 807 : cluster [DBG] pgmap v1167: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 903 op/s 2020-09-08T15:54:15.199 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:15.206 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:54:15 smithi104 bash[19922]: debug 2020-09-08T15:54:15.096+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:15.315 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:15.419 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:15.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:54:15 smithi104 bash[22305]: debug 2020-09-08T15:54:15.316+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:15.702 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:15.806 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:15.969 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:16.072 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:16.080 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:54:15 smithi104 bash[15273]: debug 2020-09-08T15:54:15.964+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:16.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:54:15 smithi055 bash[11956]: debug 2020-09-08T15:54:15.702+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:16.199 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:16.211 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:54:16.303 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:16.329 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:54:16.337 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:54:16 smithi104 bash[22305]: debug 2020-09-08T15:54:16.196+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:16.417 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:54:16.484 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:16.508 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:54:16.588 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:16.597 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:54:16 smithi055 bash[16646]: debug 2020-09-08T15:54:16.482+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:16.898 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:16 smithi055 bash[9296]: cluster 2020-09-08T15:54:15.856594+0000 mgr.x (mgr.24118) 808 : cluster [DBG] pgmap v1168: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 797 op/s 2020-09-08T15:54:17.102 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:17.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:54:17 smithi055 bash[11956]: debug 2020-09-08T15:54:17.102+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:17.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:16 smithi104 bash[9203]: cluster 2020-09-08T15:54:15.856594+0000 mgr.x (mgr.24118) 808 : cluster [DBG] pgmap v1168: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 797 op/s 2020-09-08T15:54:17.206 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:54:17.231 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:16 smithi166 bash[9446]: cluster 2020-09-08T15:54:15.856594+0000 mgr.x (mgr.24118) 808 : cluster [DBG] pgmap v1168: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 797 op/s 2020-09-08T15:54:17.354 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:54:17.459 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:17.571 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:54:17 smithi166 bash[18960]: debug 2020-09-08T15:54:17.355+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:17.572 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:17.676 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:54:17.716 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:54:17.717 INFO:teuthology.orchestra.run.smithi055.stdout: "size": 20, 2020-09-08T15:54:17.717 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 600, 2020-09-08T15:54:17.717 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T15:54:17.718 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.718 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:7721 5.7 5:e5efaeae:::benchmark_data_smithi166_37820_object7720:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e404)", 2020-09-08T15:54:17.718 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:51:55.361827+0000", 2020-09-08T15:54:17.718 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 142.357317631, 2020-09-08T15:54:17.718 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.42692171600000001, 2020-09-08T15:54:17.719 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:17.719 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:17.719 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:17.719 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:17.719 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:17.720 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 7721 2020-09-08T15:54:17.720 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.720 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:17.720 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.720 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:17.721 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.361827+0000", 2020-09-08T15:54:17.721 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:17.721 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.721 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.721 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:17.722 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.361827+0000", 2020-09-08T15:54:17.722 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999576 2020-09-08T15:54:17.722 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.722 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.722 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:17.723 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.361785+0000", 2020-09-08T15:54:17.723 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000135136 2020-09-08T15:54:17.723 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.723 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.723 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:17.724 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.361920+0000", 2020-09-08T15:54:17.724 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5922000000000001e-05 2020-09-08T15:54:17.724 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.724 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.724 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:17.725 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.361936+0000", 2020-09-08T15:54:17.725 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.08e-06 2020-09-08T15:54:17.725 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.725 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.725 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:17.726 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.361943+0000", 2020-09-08T15:54:17.726 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.7031000000000003e-05 2020-09-08T15:54:17.726 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.726 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.726 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:17.726 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.362000+0000", 2020-09-08T15:54:17.727 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019992 2020-09-08T15:54:17.727 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.727 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.727 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:17.727 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.362200+0000", 2020-09-08T15:54:17.728 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00034805399999999998 2020-09-08T15:54:17.728 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.728 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.728 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:17.728 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.362548+0000", 2020-09-08T15:54:17.729 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0022300879999999999 2020-09-08T15:54:17.729 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.729 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.729 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:17.730 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.364778+0000", 2020-09-08T15:54:17.730 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.42372765099999998 2020-09-08T15:54:17.730 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.731 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.731 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:17.731 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.788506+0000", 2020-09-08T15:54:17.731 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011177500000000001 2020-09-08T15:54:17.731 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.732 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.732 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:17.732 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.788617+0000", 2020-09-08T15:54:17.732 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000131364 2020-09-08T15:54:17.732 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.732 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.733 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:17.733 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.788749+0000", 2020-09-08T15:54:17.733 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.42692171600000001 2020-09-08T15:54:17.733 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.733 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:17.734 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.734 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.734 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.734 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:7728 5.7 5:e70c88e0:::benchmark_data_smithi166_37820_object7727:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e404)", 2020-09-08T15:54:17.734 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:51:55.364016+0000", 2020-09-08T15:54:17.735 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 142.35512913700001, 2020-09-08T15:54:17.735 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.42501676399999999, 2020-09-08T15:54:17.735 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:17.735 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:17.735 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:17.736 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:17.736 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:17.736 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 7728 2020-09-08T15:54:17.736 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.736 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:17.737 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.737 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:17.737 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.364016+0000", 2020-09-08T15:54:17.737 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:17.737 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.738 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.738 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:17.738 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.364016+0000", 2020-09-08T15:54:17.738 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999599 2020-09-08T15:54:17.738 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.738 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.739 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:17.739 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.363976+0000", 2020-09-08T15:54:17.739 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013130999999999999 2020-09-08T15:54:17.739 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.740 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.740 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:17.740 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.364107+0000", 2020-09-08T15:54:17.740 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3939e-05 2020-09-08T15:54:17.740 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.741 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.741 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:17.742 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.364121+0000", 2020-09-08T15:54:17.742 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.2530000000000001e-06 2020-09-08T15:54:17.742 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.742 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.742 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:17.743 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.364127+0000", 2020-09-08T15:54:17.743 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.4125999999999997e-05 2020-09-08T15:54:17.743 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.743 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.744 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:17.744 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.364171+0000", 2020-09-08T15:54:17.744 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.9084000000000002e-05 2020-09-08T15:54:17.744 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.744 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.745 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:17.745 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.364270+0000", 2020-09-08T15:54:17.745 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000102379 2020-09-08T15:54:17.745 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.745 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.746 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:17.746 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.364373+0000", 2020-09-08T15:54:17.746 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00094922300000000002 2020-09-08T15:54:17.746 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.746 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.747 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:17.747 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.365322+0000", 2020-09-08T15:54:17.747 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.42349800799999998 2020-09-08T15:54:17.747 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.747 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.748 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:17.748 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.788820+0000", 2020-09-08T15:54:17.748 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.8298e-05 2020-09-08T15:54:17.748 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.748 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.748 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:17.749 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.788918+0000", 2020-09-08T15:54:17.749 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000114201 2020-09-08T15:54:17.749 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.749 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.749 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:17.750 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.789032+0000", 2020-09-08T15:54:17.750 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.42501676399999999 2020-09-08T15:54:17.750 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.751 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:17.751 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.751 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.751 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.751 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:7734 5.7 5:e2d28a7c:::benchmark_data_smithi166_37820_object7733:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e404)", 2020-09-08T15:54:17.752 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:51:55.367315+0000", 2020-09-08T15:54:17.752 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 142.35182955400001, 2020-09-08T15:54:17.752 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.42196542399999998, 2020-09-08T15:54:17.752 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:17.753 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:17.753 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:17.753 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:17.753 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:17.753 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 7734 2020-09-08T15:54:17.754 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.754 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:17.754 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.754 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:17.754 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.367315+0000", 2020-09-08T15:54:17.755 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:17.755 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.755 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.755 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:17.755 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.367315+0000", 2020-09-08T15:54:17.756 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999752 2020-09-08T15:54:17.756 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.756 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.756 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:17.756 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.367291+0000", 2020-09-08T15:54:17.757 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.0097000000000002e-05 2020-09-08T15:54:17.757 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.757 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.757 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:17.757 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.367371+0000", 2020-09-08T15:54:17.758 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.38e-06 2020-09-08T15:54:17.758 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.758 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.758 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:17.758 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.367380+0000", 2020-09-08T15:54:17.759 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.6750000000000001e-06 2020-09-08T15:54:17.759 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.759 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.759 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:17.759 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.367385+0000", 2020-09-08T15:54:17.759 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.2551e-05 2020-09-08T15:54:17.760 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.760 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.760 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:17.760 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.367427+0000", 2020-09-08T15:54:17.760 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000116212 2020-09-08T15:54:17.761 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.761 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.761 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:17.761 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.367543+0000", 2020-09-08T15:54:17.761 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011844400000000001 2020-09-08T15:54:17.762 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.762 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.762 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:17.762 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.367662+0000", 2020-09-08T15:54:17.762 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00056058900000000001 2020-09-08T15:54:17.763 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.763 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.763 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:17.763 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.368222+0000", 2020-09-08T15:54:17.763 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.42087834000000002 2020-09-08T15:54:17.764 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.764 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.764 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:17.764 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.789101+0000", 2020-09-08T15:54:17.764 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.1833000000000001e-05 2020-09-08T15:54:17.764 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.765 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.765 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:17.765 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.789183+0000", 2020-09-08T15:54:17.765 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.7937999999999996e-05 2020-09-08T15:54:17.765 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.766 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.766 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:17.766 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.789281+0000", 2020-09-08T15:54:17.766 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.42196542399999998 2020-09-08T15:54:17.767 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.767 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:17.767 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.767 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.767 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.768 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:7721 5.7 5:e5efaeae:::benchmark_data_smithi166_37820_object7720:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e404)", 2020-09-08T15:54:17.768 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:51:55.384963+0000", 2020-09-08T15:54:17.768 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 142.334181427, 2020-09-08T15:54:17.768 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40369577400000001, 2020-09-08T15:54:17.768 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:17.769 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "delayed", 2020-09-08T15:54:17.769 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:17.769 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:17.769 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:17.769 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 7721 2020-09-08T15:54:17.770 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.770 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:17.770 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.770 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:17.771 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.384963+0000", 2020-09-08T15:54:17.771 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:17.771 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.771 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.771 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:17.772 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.384963+0000", 2020-09-08T15:54:17.772 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999685 2020-09-08T15:54:17.772 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.772 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.772 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:17.773 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.384932+0000", 2020-09-08T15:54:17.773 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019371300000000001 2020-09-08T15:54:17.773 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.773 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.773 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:17.773 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385126+0000", 2020-09-08T15:54:17.774 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5367999999999999e-05 2020-09-08T15:54:17.774 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.774 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.774 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:17.774 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385141+0000", 2020-09-08T15:54:17.775 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2190000000000004e-06 2020-09-08T15:54:17.775 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.775 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.775 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:17.775 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385149+0000", 2020-09-08T15:54:17.776 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.5701999999999997e-05 2020-09-08T15:54:17.776 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.776 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.776 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:17.776 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385195+0000", 2020-09-08T15:54:17.777 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.4374000000000004e-05 2020-09-08T15:54:17.777 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.777 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.777 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for ondisk", 2020-09-08T15:54:17.777 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385259+0000", 2020-09-08T15:54:17.777 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40339982400000002 2020-09-08T15:54:17.778 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.778 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.778 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:17.778 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.788659+0000", 2020-09-08T15:54:17.778 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40369577400000001 2020-09-08T15:54:17.779 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.779 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:17.779 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.779 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.779 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.780 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:7728 5.7 5:e70c88e0:::benchmark_data_smithi166_37820_object7727:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e404)", 2020-09-08T15:54:17.780 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:51:55.385203+0000", 2020-09-08T15:54:17.780 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 142.333941353, 2020-09-08T15:54:17.780 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40374453199999999, 2020-09-08T15:54:17.780 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:17.780 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "delayed", 2020-09-08T15:54:17.781 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:17.781 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:17.781 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:17.781 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 7728 2020-09-08T15:54:17.781 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.782 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:17.782 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.782 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:17.782 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385203+0000", 2020-09-08T15:54:17.782 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:17.783 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.783 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.783 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:17.783 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385203+0000", 2020-09-08T15:54:17.783 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999709 2020-09-08T15:54:17.784 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.784 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.784 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:17.784 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385174+0000", 2020-09-08T15:54:17.784 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000109769 2020-09-08T15:54:17.784 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.785 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.785 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:17.785 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385284+0000", 2020-09-08T15:54:17.785 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4056000000000001e-05 2020-09-08T15:54:17.785 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.786 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.786 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:17.786 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385298+0000", 2020-09-08T15:54:17.786 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.5719999999999996e-06 2020-09-08T15:54:17.786 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.787 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.787 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:17.787 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385303+0000", 2020-09-08T15:54:17.787 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.0525999999999999e-05 2020-09-08T15:54:17.787 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.787 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.788 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:17.788 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385333+0000", 2020-09-08T15:54:17.788 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2222999999999998e-05 2020-09-08T15:54:17.788 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.788 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.789 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for ondisk", 2020-09-08T15:54:17.789 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385385+0000", 2020-09-08T15:54:17.789 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.403562435 2020-09-08T15:54:17.789 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.789 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.790 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:17.790 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.788948+0000", 2020-09-08T15:54:17.790 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40374453199999999 2020-09-08T15:54:17.790 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.790 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:17.791 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.791 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.791 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.791 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:7734 5.7 5:e2d28a7c:::benchmark_data_smithi166_37820_object7733:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e404)", 2020-09-08T15:54:17.791 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:51:55.385358+0000", 2020-09-08T15:54:17.791 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 142.333786594, 2020-09-08T15:54:17.792 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40385063799999998, 2020-09-08T15:54:17.792 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:17.792 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "delayed", 2020-09-08T15:54:17.792 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:17.792 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:17.793 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:17.793 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 7734 2020-09-08T15:54:17.793 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.793 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:17.793 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.794 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:17.794 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385358+0000", 2020-09-08T15:54:17.794 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:17.794 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.794 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.795 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:17.795 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385358+0000", 2020-09-08T15:54:17.795 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999685 2020-09-08T15:54:17.795 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.795 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.795 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:17.796 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385327+0000", 2020-09-08T15:54:17.796 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000210998 2020-09-08T15:54:17.796 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.796 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.796 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:17.797 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385538+0000", 2020-09-08T15:54:17.797 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3524e-05 2020-09-08T15:54:17.797 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.797 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.797 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:17.798 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385551+0000", 2020-09-08T15:54:17.798 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.5260000000000004e-06 2020-09-08T15:54:17.798 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.798 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.798 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:17.799 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385556+0000", 2020-09-08T15:54:17.799 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.0417999999999999e-05 2020-09-08T15:54:17.799 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.799 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.799 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:17.800 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385586+0000", 2020-09-08T15:54:17.800 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.0593000000000002e-05 2020-09-08T15:54:17.800 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.800 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.800 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for ondisk", 2020-09-08T15:54:17.800 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.385637+0000", 2020-09-08T15:54:17.801 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40357198300000002 2020-09-08T15:54:17.801 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.801 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.801 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:17.801 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.789209+0000", 2020-09-08T15:54:17.802 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40385063799999998 2020-09-08T15:54:17.802 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.802 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:17.802 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.802 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.803 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.803 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:7750 5.7 5:e41d35a6:::benchmark_data_smithi166_37820_object7749:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e404)", 2020-09-08T15:54:17.803 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:51:55.393605+0000", 2020-09-08T15:54:17.803 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 142.32553958, 2020-09-08T15:54:17.803 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.39622833699999999, 2020-09-08T15:54:17.804 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:17.804 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:17.804 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:17.804 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:17.804 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:17.804 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 7750 2020-09-08T15:54:17.805 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.805 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:17.805 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.805 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:17.805 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.393605+0000", 2020-09-08T15:54:17.806 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:17.806 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.806 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.806 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:17.806 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.393605+0000", 2020-09-08T15:54:17.807 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999695 2020-09-08T15:54:17.807 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.807 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.807 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:17.807 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.393574+0000", 2020-09-08T15:54:17.808 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000104987 2020-09-08T15:54:17.808 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.808 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.808 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:17.808 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.393679+0000", 2020-09-08T15:54:17.809 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4914e-05 2020-09-08T15:54:17.809 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.809 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.809 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:17.809 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.393694+0000", 2020-09-08T15:54:17.810 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0630000000000002e-06 2020-09-08T15:54:17.810 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.810 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.810 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:17.811 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.393700+0000", 2020-09-08T15:54:17.811 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.5776000000000003e-05 2020-09-08T15:54:17.811 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.811 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.811 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:17.812 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.393756+0000", 2020-09-08T15:54:17.812 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015693200000000001 2020-09-08T15:54:17.812 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.812 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.812 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:17.812 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.393913+0000", 2020-09-08T15:54:17.813 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017153399999999999 2020-09-08T15:54:17.813 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.813 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.813 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:17.813 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.394085+0000", 2020-09-08T15:54:17.814 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00080678900000000003 2020-09-08T15:54:17.814 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.814 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.814 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:17.814 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.394891+0000", 2020-09-08T15:54:17.815 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.39456645899999998 2020-09-08T15:54:17.815 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.815 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.815 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:17.815 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.789458+0000", 2020-09-08T15:54:17.816 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00024560599999999999 2020-09-08T15:54:17.816 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.816 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.816 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:17.816 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.789703+0000", 2020-09-08T15:54:17.816 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012998099999999999 2020-09-08T15:54:17.817 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.817 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.817 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:17.817 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:51:55.789833+0000", 2020-09-08T15:54:17.817 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.39622833699999999 2020-09-08T15:54:17.818 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.818 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:17.818 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.818 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.818 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.819 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:92717 5.f 5:f8e6b0a5:::benchmark_data_smithi166_37820_object92716:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e417)", 2020-09-08T15:54:17.819 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:52:38.285172+0000", 2020-09-08T15:54:17.819 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 99.433972748000002, 2020-09-08T15:54:17.819 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.15352211499999999, 2020-09-08T15:54:17.819 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:17.820 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:17.820 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:17.820 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:17.820 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:17.820 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 92717 2020-09-08T15:54:17.820 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.821 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:17.821 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.821 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:17.821 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285172+0000", 2020-09-08T15:54:17.821 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:17.822 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.822 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.822 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:17.822 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285172+0000", 2020-09-08T15:54:17.822 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999499 2020-09-08T15:54:17.823 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.823 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.823 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:17.823 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285122+0000", 2020-09-08T15:54:17.823 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019855799999999999 2020-09-08T15:54:17.823 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.824 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.824 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:17.824 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285320+0000", 2020-09-08T15:54:17.824 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.9026999999999999e-05 2020-09-08T15:54:17.824 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.825 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.825 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:17.825 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285339+0000", 2020-09-08T15:54:17.825 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.2590000000000003e-06 2020-09-08T15:54:17.825 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.826 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.826 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:17.826 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285347+0000", 2020-09-08T15:54:17.826 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0023039499999999999 2020-09-08T15:54:17.826 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.827 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.827 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:17.827 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.287651+0000", 2020-09-08T15:54:17.827 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00018092999999999999 2020-09-08T15:54:17.827 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.827 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.828 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:17.828 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.287832+0000", 2020-09-08T15:54:17.828 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013279000000000001 2020-09-08T15:54:17.828 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.828 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.829 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 4", 2020-09-08T15:54:17.829 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.287964+0000", 2020-09-08T15:54:17.829 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0037143990000000002 2020-09-08T15:54:17.829 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.829 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.830 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:17.830 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.291679+0000", 2020-09-08T15:54:17.830 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.14684634699999999 2020-09-08T15:54:17.830 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.830 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.831 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:17.831 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.438525+0000", 2020-09-08T15:54:17.831 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.6830999999999997e-05 2020-09-08T15:54:17.831 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.832 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.832 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:17.832 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.438602+0000", 2020-09-08T15:54:17.832 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.2103999999999998e-05 2020-09-08T15:54:17.832 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.832 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.833 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:17.833 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.438694+0000", 2020-09-08T15:54:17.833 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.15352211499999999 2020-09-08T15:54:17.833 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.833 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:17.834 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.834 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.834 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.834 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:92727 5.f 5:fac0a0d6:::benchmark_data_smithi166_37820_object92726:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e417)", 2020-09-08T15:54:17.834 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:52:38.293273+0000", 2020-09-08T15:54:17.835 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 99.425871443000005, 2020-09-08T15:54:17.835 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.15058411699999999, 2020-09-08T15:54:17.835 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:17.835 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:17.835 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:17.836 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:17.836 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:17.836 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 92727 2020-09-08T15:54:17.836 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.836 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:17.836 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.837 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:17.837 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.293273+0000", 2020-09-08T15:54:17.837 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:17.837 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.837 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.838 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:17.838 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.293273+0000", 2020-09-08T15:54:17.838 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999475 2020-09-08T15:54:17.838 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.838 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.839 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:17.839 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.293221+0000", 2020-09-08T15:54:17.839 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000174678 2020-09-08T15:54:17.839 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.839 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.840 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:17.840 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.293395+0000", 2020-09-08T15:54:17.840 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.7972e-05 2020-09-08T15:54:17.840 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.840 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.841 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:17.841 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.293413+0000", 2020-09-08T15:54:17.841 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.9720000000000004e-06 2020-09-08T15:54:17.841 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.842 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.842 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:17.842 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.293423+0000", 2020-09-08T15:54:17.843 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.0769000000000001e-05 2020-09-08T15:54:17.843 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.843 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.843 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:17.844 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.293474+0000", 2020-09-08T15:54:17.844 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015940799999999999 2020-09-08T15:54:17.844 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.844 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.844 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:17.845 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.293633+0000", 2020-09-08T15:54:17.845 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000175601 2020-09-08T15:54:17.845 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.845 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.845 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 4", 2020-09-08T15:54:17.846 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.293809+0000", 2020-09-08T15:54:17.846 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.003667188 2020-09-08T15:54:17.846 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.846 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.846 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:17.847 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.297476+0000", 2020-09-08T15:54:17.847 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.146222765 2020-09-08T15:54:17.847 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.847 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.847 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:17.847 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.443699+0000", 2020-09-08T15:54:17.848 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000113917 2020-09-08T15:54:17.848 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.848 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.848 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:17.848 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.443813+0000", 2020-09-08T15:54:17.849 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.4421e-05 2020-09-08T15:54:17.849 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.849 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.849 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:17.849 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.443857+0000", 2020-09-08T15:54:17.850 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.15058411699999999 2020-09-08T15:54:17.850 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.850 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:17.850 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.850 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.851 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.851 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:92747 5.f 5:fa8f7b5b:::benchmark_data_smithi166_37820_object92746:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e417)", 2020-09-08T15:54:17.851 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:52:38.309760+0000", 2020-09-08T15:54:17.851 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 99.409384954000004, 2020-09-08T15:54:17.852 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13434099999999999, 2020-09-08T15:54:17.852 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:17.852 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:17.852 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:17.853 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:17.853 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:17.853 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 92747 2020-09-08T15:54:17.853 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.853 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:17.854 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.854 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:17.854 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.309760+0000", 2020-09-08T15:54:17.854 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:17.854 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.855 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.855 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:17.855 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.309760+0000", 2020-09-08T15:54:17.855 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999969 2020-09-08T15:54:17.855 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.856 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.856 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:17.856 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.309729+0000", 2020-09-08T15:54:17.856 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012339 2020-09-08T15:54:17.856 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.857 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.857 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:17.857 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.309852+0000", 2020-09-08T15:54:17.857 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5109999999999999e-05 2020-09-08T15:54:17.857 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.857 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.858 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:17.858 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.309867+0000", 2020-09-08T15:54:17.858 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.6370000000000005e-06 2020-09-08T15:54:17.858 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.858 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.859 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:17.859 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.309876+0000", 2020-09-08T15:54:17.859 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2418999999999998e-05 2020-09-08T15:54:17.859 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.859 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.860 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:17.860 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.309958+0000", 2020-09-08T15:54:17.860 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00024121099999999999 2020-09-08T15:54:17.860 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.860 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.860 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:17.861 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.310199+0000", 2020-09-08T15:54:17.861 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00030235499999999997 2020-09-08T15:54:17.861 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.861 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.861 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 4", 2020-09-08T15:54:17.862 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.310502+0000", 2020-09-08T15:54:17.862 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0028761139999999999 2020-09-08T15:54:17.862 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.862 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.862 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:17.863 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.313378+0000", 2020-09-08T15:54:17.863 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13055862900000001 2020-09-08T15:54:17.863 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.863 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.863 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:17.864 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.443937+0000", 2020-09-08T15:54:17.864 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.1349999999999998e-05 2020-09-08T15:54:17.864 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.864 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.864 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:17.864 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.444028+0000", 2020-09-08T15:54:17.865 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.2843999999999996e-05 2020-09-08T15:54:17.865 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.865 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.865 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:17.865 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.444101+0000", 2020-09-08T15:54:17.866 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13434099999999999 2020-09-08T15:54:17.866 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.866 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:17.866 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.867 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.867 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.867 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:118563 5.7 5:efd5b601:::benchmark_data_smithi166_37820_object118562:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T15:54:17.867 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:52:58.581231+0000", 2020-09-08T15:54:17.867 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 79.137914086999999, 2020-09-08T15:54:17.868 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.297967064, 2020-09-08T15:54:17.868 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:17.868 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:17.868 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:17.868 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:17.869 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:17.869 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 118563 2020-09-08T15:54:17.869 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.869 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:17.869 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.870 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:17.870 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.581231+0000", 2020-09-08T15:54:17.870 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:17.870 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.870 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.871 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:17.871 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.581231+0000", 2020-09-08T15:54:17.871 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999557 2020-09-08T15:54:17.871 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.871 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.872 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:17.872 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.581186+0000", 2020-09-08T15:54:17.872 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000127033 2020-09-08T15:54:17.872 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.872 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.873 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:17.873 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.581313+0000", 2020-09-08T15:54:17.873 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2547e-05 2020-09-08T15:54:17.873 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.873 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.874 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:17.874 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.581326+0000", 2020-09-08T15:54:17.874 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0804e-05 2020-09-08T15:54:17.874 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.874 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.875 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:17.875 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.581337+0000", 2020-09-08T15:54:17.875 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.1172999999999999e-05 2020-09-08T15:54:17.875 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.875 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.875 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:17.876 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.581388+0000", 2020-09-08T15:54:17.876 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000172671 2020-09-08T15:54:17.876 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.876 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.876 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:17.877 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.581561+0000", 2020-09-08T15:54:17.877 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000207166 2020-09-08T15:54:17.877 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.877 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.877 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:17.878 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.581768+0000", 2020-09-08T15:54:17.878 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0050238440000000004 2020-09-08T15:54:17.878 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.878 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.878 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:17.879 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.586792+0000", 2020-09-08T15:54:17.879 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.292262682 2020-09-08T15:54:17.879 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.879 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.879 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:17.879 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.879054+0000", 2020-09-08T15:54:17.880 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.9379e-05 2020-09-08T15:54:17.880 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.880 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.880 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:17.880 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.879134+0000", 2020-09-08T15:54:17.881 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3966000000000004e-05 2020-09-08T15:54:17.881 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.881 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.881 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:17.881 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.879198+0000", 2020-09-08T15:54:17.882 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.297967064 2020-09-08T15:54:17.882 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.882 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:17.883 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.883 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.884 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.884 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:118580 5.7 5:e8d09e1e:::benchmark_data_smithi166_37820_object118579:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T15:54:17.884 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:52:58.594587+0000", 2020-09-08T15:54:17.884 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 79.124557441999997, 2020-09-08T15:54:17.884 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.29332519000000001, 2020-09-08T15:54:17.884 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:17.885 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:17.885 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:17.885 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:17.885 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:17.885 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 118580 2020-09-08T15:54:17.886 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.886 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:17.886 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.886 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:17.886 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.594587+0000", 2020-09-08T15:54:17.887 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:17.887 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.887 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.887 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:17.887 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.594587+0000", 2020-09-08T15:54:17.888 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999499 2020-09-08T15:54:17.888 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.888 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.888 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:17.888 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.594537+0000", 2020-09-08T15:54:17.888 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017796000000000001 2020-09-08T15:54:17.889 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.889 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.889 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:17.889 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.594715+0000", 2020-09-08T15:54:17.889 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.8093000000000001e-05 2020-09-08T15:54:17.890 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.890 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.890 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:17.890 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.594733+0000", 2020-09-08T15:54:17.890 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1066e-05 2020-09-08T15:54:17.891 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.891 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.891 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:17.891 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.594745+0000", 2020-09-08T15:54:17.891 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9453000000000002e-05 2020-09-08T15:54:17.892 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.892 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.892 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:17.892 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.594804+0000", 2020-09-08T15:54:17.892 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00020589299999999999 2020-09-08T15:54:17.892 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.893 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.893 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:17.893 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.595010+0000", 2020-09-08T15:54:17.893 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00025979699999999998 2020-09-08T15:54:17.893 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.894 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.894 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:17.894 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.595270+0000", 2020-09-08T15:54:17.894 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0051502780000000003 2020-09-08T15:54:17.894 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.895 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.895 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:17.895 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.600420+0000", 2020-09-08T15:54:17.895 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.28735190599999999 2020-09-08T15:54:17.895 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.895 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.896 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:17.896 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.887772+0000", 2020-09-08T15:54:17.896 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.8019e-05 2020-09-08T15:54:17.896 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.896 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.897 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:17.897 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.887850+0000", 2020-09-08T15:54:17.897 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.2570999999999994e-05 2020-09-08T15:54:17.897 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.897 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.898 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:17.898 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.887912+0000", 2020-09-08T15:54:17.898 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.29332519000000001 2020-09-08T15:54:17.898 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.898 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:17.898 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.899 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.899 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.899 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:118607 5.7 5:ed862969:::benchmark_data_smithi166_37820_object118606:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T15:54:17.899 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:52:58.629997+0000", 2020-09-08T15:54:17.899 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 79.089147448999995, 2020-09-08T15:54:17.900 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.258051635, 2020-09-08T15:54:17.900 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:17.900 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:17.900 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:17.900 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:17.901 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:17.901 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 118607 2020-09-08T15:54:17.901 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.901 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:17.901 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.902 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:17.902 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.629997+0000", 2020-09-08T15:54:17.902 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:17.902 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.902 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.902 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:17.903 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.629997+0000", 2020-09-08T15:54:17.903 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999528 2020-09-08T15:54:17.903 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.903 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.903 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:17.904 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.629950+0000", 2020-09-08T15:54:17.904 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014771000000000001 2020-09-08T15:54:17.904 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.904 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.904 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:17.905 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.630098+0000", 2020-09-08T15:54:17.905 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.2722e-05 2020-09-08T15:54:17.905 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.905 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.905 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:17.906 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.630121+0000", 2020-09-08T15:54:17.906 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.4240000000000004e-06 2020-09-08T15:54:17.906 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.906 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.906 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:17.906 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.630128+0000", 2020-09-08T15:54:17.907 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.6507e-05 2020-09-08T15:54:17.907 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.907 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.907 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:17.907 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.630175+0000", 2020-09-08T15:54:17.908 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013917199999999999 2020-09-08T15:54:17.908 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.908 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.908 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:17.908 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.630314+0000", 2020-09-08T15:54:17.909 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000116754 2020-09-08T15:54:17.909 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.909 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.909 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:17.909 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.630430+0000", 2020-09-08T15:54:17.910 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0041225749999999999 2020-09-08T15:54:17.910 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.910 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.910 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:17.910 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.634553+0000", 2020-09-08T15:54:17.910 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.25341049399999999 2020-09-08T15:54:17.911 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.911 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.911 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:17.911 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.887964+0000", 2020-09-08T15:54:17.912 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.8741e-05 2020-09-08T15:54:17.912 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.912 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.912 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:17.912 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.888012+0000", 2020-09-08T15:54:17.913 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.6581000000000001e-05 2020-09-08T15:54:17.913 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.913 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.913 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:17.913 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.888049+0000", 2020-09-08T15:54:17.914 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.258051635 2020-09-08T15:54:17.914 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.914 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:17.914 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.914 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.915 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.915 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:118627 5.7 5:e8e8242f:::benchmark_data_smithi166_37820_object118626:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T15:54:17.915 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:52:58.679349+0000", 2020-09-08T15:54:17.915 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 79.039795213999994, 2020-09-08T15:54:17.915 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.20879808599999999, 2020-09-08T15:54:17.915 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:17.916 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:17.916 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:17.916 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:17.916 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:17.916 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 118627 2020-09-08T15:54:17.917 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.917 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:17.917 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.917 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:17.917 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.679349+0000", 2020-09-08T15:54:17.918 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:17.918 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.918 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.918 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:17.918 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.679349+0000", 2020-09-08T15:54:17.918 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999585 2020-09-08T15:54:17.919 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.919 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.919 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:17.919 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.679308+0000", 2020-09-08T15:54:17.919 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013570699999999999 2020-09-08T15:54:17.920 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.920 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.920 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:17.920 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.679444+0000", 2020-09-08T15:54:17.920 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.7288000000000001e-05 2020-09-08T15:54:17.921 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.921 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.921 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:17.921 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.679461+0000", 2020-09-08T15:54:17.921 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.1159999999999999e-06 2020-09-08T15:54:17.922 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.922 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.922 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:17.922 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.679469+0000", 2020-09-08T15:54:17.922 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.8280000000000003e-05 2020-09-08T15:54:17.923 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.923 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.923 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:17.923 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.679548+0000", 2020-09-08T15:54:17.923 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00022315599999999999 2020-09-08T15:54:17.923 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.924 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.924 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:17.924 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.679771+0000", 2020-09-08T15:54:17.924 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000200144 2020-09-08T15:54:17.924 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.925 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.925 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:17.925 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.679971+0000", 2020-09-08T15:54:17.925 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0054856480000000001 2020-09-08T15:54:17.925 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.926 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.926 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:17.926 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.685457+0000", 2020-09-08T15:54:17.926 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.20262358899999999 2020-09-08T15:54:17.926 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.926 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.927 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:17.927 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.888080+0000", 2020-09-08T15:54:17.927 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.7607000000000001e-05 2020-09-08T15:54:17.927 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.927 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.928 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:17.928 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.888118+0000", 2020-09-08T15:54:17.928 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.9818000000000001e-05 2020-09-08T15:54:17.928 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.928 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.929 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:17.929 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:58.888148+0000", 2020-09-08T15:54:17.929 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.20879808599999999 2020-09-08T15:54:17.929 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.929 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:17.929 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.930 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.930 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.930 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:148762 5.7 5:eb267060:::benchmark_data_smithi166_37820_object4340:head [delete] snapc 0=[] ondisk+write+known_if_redirected e432)", 2020-09-08T15:54:17.930 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:53:28.609626+0000", 2020-09-08T15:54:17.930 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 49.109518340999998, 2020-09-08T15:54:17.931 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.162976863, 2020-09-08T15:54:17.931 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:17.931 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:17.931 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:17.931 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:17.932 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:17.932 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 148762 2020-09-08T15:54:17.932 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.932 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:17.933 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.933 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:17.933 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.609626+0000", 2020-09-08T15:54:17.933 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:17.933 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.934 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.934 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:17.934 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.609626+0000", 2020-09-08T15:54:17.934 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999442 2020-09-08T15:54:17.934 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.934 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.935 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:17.935 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.609570+0000", 2020-09-08T15:54:17.935 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012692500000000001 2020-09-08T15:54:17.935 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.935 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.936 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:17.936 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.609697+0000", 2020-09-08T15:54:17.936 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.7744000000000001e-05 2020-09-08T15:54:17.936 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.936 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.937 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:17.937 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.609725+0000", 2020-09-08T15:54:17.937 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.1499999999999999e-06 2020-09-08T15:54:17.937 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.937 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.937 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:17.938 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.609733+0000", 2020-09-08T15:54:17.938 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.7277000000000001e-05 2020-09-08T15:54:17.938 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.938 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.938 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:17.939 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.609791+0000", 2020-09-08T15:54:17.939 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00020830100000000001 2020-09-08T15:54:17.939 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.939 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.939 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:17.940 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.609999+0000", 2020-09-08T15:54:17.940 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00020896000000000001 2020-09-08T15:54:17.940 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.940 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.940 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:17.941 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.610208+0000", 2020-09-08T15:54:17.941 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0050321309999999996 2020-09-08T15:54:17.941 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.941 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.941 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:17.942 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.615240+0000", 2020-09-08T15:54:17.942 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.157114427 2020-09-08T15:54:17.942 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.942 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.942 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:17.943 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.772354+0000", 2020-09-08T15:54:17.943 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000141939 2020-09-08T15:54:17.943 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.944 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.944 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:17.944 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.772496+0000", 2020-09-08T15:54:17.944 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000106932 2020-09-08T15:54:17.945 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.945 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.945 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:17.945 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.772603+0000", 2020-09-08T15:54:17.945 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.162976863 2020-09-08T15:54:17.946 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.946 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:17.946 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.946 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.946 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.946 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:148771 5.7 5:e49e29af:::benchmark_data_smithi166_37820_object4349:head [delete] snapc 0=[] ondisk+write+known_if_redirected e432)", 2020-09-08T15:54:17.947 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:53:28.619026+0000", 2020-09-08T15:54:17.947 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 49.100118887000001, 2020-09-08T15:54:17.947 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.15382300800000001, 2020-09-08T15:54:17.947 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:17.947 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:17.948 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:17.948 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:17.948 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:17.948 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 148771 2020-09-08T15:54:17.948 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.949 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:17.949 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.949 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:17.949 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.619026+0000", 2020-09-08T15:54:17.949 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:17.950 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.950 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.950 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:17.950 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.619026+0000", 2020-09-08T15:54:17.950 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999619 2020-09-08T15:54:17.951 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.951 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.951 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:17.951 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.618988+0000", 2020-09-08T15:54:17.952 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000106918 2020-09-08T15:54:17.952 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.952 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.952 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:17.953 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.619095+0000", 2020-09-08T15:54:17.953 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.6689000000000001e-05 2020-09-08T15:54:17.953 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.953 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.953 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:17.954 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.619121+0000", 2020-09-08T15:54:17.954 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1892000000000001e-05 2020-09-08T15:54:17.954 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.954 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.954 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:17.955 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.619133+0000", 2020-09-08T15:54:17.955 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2491000000000003e-05 2020-09-08T15:54:17.955 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.955 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.955 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:17.956 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.619186+0000", 2020-09-08T15:54:17.956 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000177737 2020-09-08T15:54:17.956 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.956 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.956 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:17.957 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.619363+0000", 2020-09-08T15:54:17.957 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00020401999999999999 2020-09-08T15:54:17.957 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.957 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.957 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:17.957 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.619567+0000", 2020-09-08T15:54:17.958 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0056505119999999999 2020-09-08T15:54:17.958 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.958 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.958 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:17.958 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.625218+0000", 2020-09-08T15:54:17.959 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.147468075 2020-09-08T15:54:17.959 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.959 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.959 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:17.959 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.772686+0000", 2020-09-08T15:54:17.960 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.7661e-05 2020-09-08T15:54:17.960 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.960 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.960 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:17.960 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.772774+0000", 2020-09-08T15:54:17.960 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.5080999999999995e-05 2020-09-08T15:54:17.961 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.961 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.961 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:17.961 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.772849+0000", 2020-09-08T15:54:17.961 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.15382300800000001 2020-09-08T15:54:17.962 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.962 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:17.962 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.962 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.962 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.963 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:148777 5.7 5:e76d3ca4:::benchmark_data_smithi166_37820_object4355:head [delete] snapc 0=[] ondisk+write+known_if_redirected e432)", 2020-09-08T15:54:17.963 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:53:28.628517+0000", 2020-09-08T15:54:17.963 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 49.090627423999997, 2020-09-08T15:54:17.963 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.144666129, 2020-09-08T15:54:17.963 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:17.964 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:17.964 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:17.964 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:17.964 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:17.964 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 148777 2020-09-08T15:54:17.964 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.965 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:17.965 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.965 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:17.965 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.628517+0000", 2020-09-08T15:54:17.965 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:17.966 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.966 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.966 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:17.966 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.628517+0000", 2020-09-08T15:54:17.966 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999166 2020-09-08T15:54:17.967 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.967 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.967 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:17.967 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.628434+0000", 2020-09-08T15:54:17.967 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015749500000000001 2020-09-08T15:54:17.968 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.968 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.968 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:17.968 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.628591+0000", 2020-09-08T15:54:17.968 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.7773000000000001e-05 2020-09-08T15:54:17.968 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.969 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.969 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:17.969 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.628609+0000", 2020-09-08T15:54:17.969 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.8340000000000005e-06 2020-09-08T15:54:17.969 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.970 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.970 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:17.970 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.628618+0000", 2020-09-08T15:54:17.970 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.9162999999999997e-05 2020-09-08T15:54:17.970 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.971 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.971 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:17.971 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.628667+0000", 2020-09-08T15:54:17.971 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000114953 2020-09-08T15:54:17.971 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.972 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.972 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:17.972 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.628782+0000", 2020-09-08T15:54:17.972 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000100321 2020-09-08T15:54:17.973 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.973 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.973 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:17.973 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.628882+0000", 2020-09-08T15:54:17.973 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0042727629999999997 2020-09-08T15:54:17.973 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.974 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.974 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:17.974 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.633155+0000", 2020-09-08T15:54:17.974 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13980462199999999 2020-09-08T15:54:17.974 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.975 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.975 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:17.975 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.772960+0000", 2020-09-08T15:54:17.975 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011994600000000001 2020-09-08T15:54:17.975 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.976 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.976 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:17.976 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.773080+0000", 2020-09-08T15:54:17.976 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010381400000000001 2020-09-08T15:54:17.976 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.976 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.977 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:17.977 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.773183+0000", 2020-09-08T15:54:17.977 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.144666129 2020-09-08T15:54:17.977 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.977 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:17.978 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.978 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.978 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.978 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:148788 5.7 5:ec7211c9:::benchmark_data_smithi166_37820_object4366:head [delete] snapc 0=[] ondisk+write+known_if_redirected e432)", 2020-09-08T15:54:17.978 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:53:28.648431+0000", 2020-09-08T15:54:17.979 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 49.070713781999999, 2020-09-08T15:54:17.979 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13209558499999999, 2020-09-08T15:54:17.979 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:17.979 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:17.979 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:17.980 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:17.980 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:17.980 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 148788 2020-09-08T15:54:17.980 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.980 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:17.981 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.981 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:17.981 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.648431+0000", 2020-09-08T15:54:17.981 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:17.981 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.981 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.982 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:17.982 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.648431+0000", 2020-09-08T15:54:17.982 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999557 2020-09-08T15:54:17.982 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.982 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.983 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:17.983 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.648386+0000", 2020-09-08T15:54:17.983 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.2847000000000001e-05 2020-09-08T15:54:17.983 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.983 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.984 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:17.984 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.648479+0000", 2020-09-08T15:54:17.984 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.0715000000000001e-05 2020-09-08T15:54:17.984 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.984 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.985 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:17.985 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.648500+0000", 2020-09-08T15:54:17.985 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0125999999999999e-05 2020-09-08T15:54:17.985 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.985 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.986 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:17.986 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.648510+0000", 2020-09-08T15:54:17.986 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.7396000000000001e-05 2020-09-08T15:54:17.987 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.987 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.987 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:17.987 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.648597+0000", 2020-09-08T15:54:17.987 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017104500000000001 2020-09-08T15:54:17.988 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.988 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.988 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:17.988 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.648769+0000", 2020-09-08T15:54:17.988 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016116700000000001 2020-09-08T15:54:17.988 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.989 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.989 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:17.989 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.648930+0000", 2020-09-08T15:54:17.989 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0051187539999999997 2020-09-08T15:54:17.989 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.990 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.990 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:17.990 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.654048+0000", 2020-09-08T15:54:17.990 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.12636256300000001 2020-09-08T15:54:17.990 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.991 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.991 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:17.991 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.780411+0000", 2020-09-08T15:54:17.991 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.2092000000000001e-05 2020-09-08T15:54:17.991 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.992 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.992 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:17.992 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.780473+0000", 2020-09-08T15:54:17.992 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.342e-05 2020-09-08T15:54:17.992 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.992 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.993 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:17.993 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.780527+0000", 2020-09-08T15:54:17.993 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13209558499999999 2020-09-08T15:54:17.993 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.993 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:17.994 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:17.994 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.994 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.994 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:148799 5.7 5:ef06098c:::benchmark_data_smithi166_37820_object4377:head [delete] snapc 0=[] ondisk+write+known_if_redirected e432)", 2020-09-08T15:54:17.994 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:53:28.687775+0000", 2020-09-08T15:54:17.995 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 49.031370045000003, 2020-09-08T15:54:17.995 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.095343312, 2020-09-08T15:54:17.995 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:17.995 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:17.995 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:17.996 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:17.996 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:17.996 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 148799 2020-09-08T15:54:17.996 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.996 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:17.996 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.997 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:17.997 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.687775+0000", 2020-09-08T15:54:17.997 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:17.997 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.997 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.998 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:17.998 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.687775+0000", 2020-09-08T15:54:17.998 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999352 2020-09-08T15:54:17.998 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.998 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.999 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:17.999 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.687710+0000", 2020-09-08T15:54:17.999 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016217599999999999 2020-09-08T15:54:17.999 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:17.999 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:17.999 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:18.000 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.687872+0000", 2020-09-08T15:54:18.000 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.7308000000000002e-05 2020-09-08T15:54:18.000 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.000 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.000 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:18.001 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.687919+0000", 2020-09-08T15:54:18.001 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.9865e-05 2020-09-08T15:54:18.001 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.001 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.001 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:18.002 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.687939+0000", 2020-09-08T15:54:18.002 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015666700000000001 2020-09-08T15:54:18.002 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.002 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.002 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:18.002 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.688096+0000", 2020-09-08T15:54:18.003 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00023476700000000001 2020-09-08T15:54:18.003 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.003 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.003 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:18.003 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.688330+0000", 2020-09-08T15:54:18.004 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00020127599999999999 2020-09-08T15:54:18.004 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.005 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.005 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:18.005 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.688532+0000", 2020-09-08T15:54:18.006 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0064424640000000002 2020-09-08T15:54:18.006 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.006 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.006 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:18.006 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.694974+0000", 2020-09-08T15:54:18.007 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.088059160999999997 2020-09-08T15:54:18.007 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.007 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.007 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:18.007 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.783033+0000", 2020-09-08T15:54:18.008 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.9598999999999997e-05 2020-09-08T15:54:18.008 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.008 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.008 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:18.008 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.783083+0000", 2020-09-08T15:54:18.009 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.5107000000000001e-05 2020-09-08T15:54:18.009 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.009 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.009 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:18.009 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:28.783118+0000", 2020-09-08T15:54:18.010 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.095343312 2020-09-08T15:54:18.010 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:18.010 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:18.010 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:18.010 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.010 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.011 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:153303 5.7 5:e911faf7:::benchmark_data_smithi166_37820_object8881:head [delete] snapc 0=[] ondisk+write+known_if_redirected e433)", 2020-09-08T15:54:18.011 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:53:33.428518+0000", 2020-09-08T15:54:18.011 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 44.290626250000003, 2020-09-08T15:54:18.011 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.077090734999999994, 2020-09-08T15:54:18.011 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:18.012 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:18.012 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:18.012 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:18.012 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:18.012 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 153303 2020-09-08T15:54:18.013 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.013 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:18.013 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.013 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:18.014 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:33.428518+0000", 2020-09-08T15:54:18.014 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:18.014 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.014 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.015 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:18.015 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:33.428518+0000", 2020-09-08T15:54:18.015 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999228 2020-09-08T15:54:18.015 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.015 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.016 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:18.016 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:33.428441+0000", 2020-09-08T15:54:18.016 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000166572 2020-09-08T15:54:18.016 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.016 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.017 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:18.017 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:33.428608+0000", 2020-09-08T15:54:18.017 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.8107999999999999e-05 2020-09-08T15:54:18.017 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.017 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.017 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:18.018 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:33.428636+0000", 2020-09-08T15:54:18.018 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2480000000000001e-05 2020-09-08T15:54:18.018 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.018 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.018 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:18.019 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:33.428648+0000", 2020-09-08T15:54:18.019 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.7350999999999999e-05 2020-09-08T15:54:18.019 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.019 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.019 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:18.020 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:33.428715+0000", 2020-09-08T15:54:18.020 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019896099999999999 2020-09-08T15:54:18.020 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.020 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.020 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:18.021 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:33.428914+0000", 2020-09-08T15:54:18.021 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019864399999999999 2020-09-08T15:54:18.021 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.021 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.021 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:18.021 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:33.429113+0000", 2020-09-08T15:54:18.022 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0060621879999999996 2020-09-08T15:54:18.022 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.022 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.022 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:18.022 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:33.435175+0000", 2020-09-08T15:54:18.023 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.070315068999999994 2020-09-08T15:54:18.024 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.024 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.024 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:18.024 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:33.505490+0000", 2020-09-08T15:54:18.024 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4792999999999999e-05 2020-09-08T15:54:18.025 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.025 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.025 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:18.025 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:33.505575+0000", 2020-09-08T15:54:18.026 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.4045999999999999e-05 2020-09-08T15:54:18.026 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:18.026 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:18.026 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:18.026 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:33.505609+0000", 2020-09-08T15:54:18.027 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.077090734999999994 2020-09-08T15:54:18.027 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:18.027 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:18.027 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:18.028 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:18.028 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:18.028 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:54:18.030 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:54:18.030 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:54:17 smithi166 bash[16593]: debug 2020-09-08T15:54:17.571+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:18.031 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:54:17 smithi166 bash[18960]: debug 2020-09-08T15:54:17.795+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:18.131 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:18.696 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:18.800 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:18.851 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2020-09-08T15:54:18.947 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:18.949 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:54:18 smithi055 bash[19003]: debug 2020-09-08T15:54:18.694+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:18.950 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:54:18 smithi055 bash[11956]: debug 2020-09-08T15:54:18.938+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:18.950 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:18 smithi055 bash[9296]: cluster 2020-09-08T15:54:17.857055+0000 mgr.x (mgr.24118) 809 : cluster [DBG] pgmap v1169: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:54:19.023 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:54:19.055 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:54:19.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:18 smithi104 bash[9203]: cluster 2020-09-08T15:54:17.857055+0000 mgr.x (mgr.24118) 809 : cluster [DBG] pgmap v1169: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:54:19.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:18 smithi166 bash[9446]: cluster 2020-09-08T15:54:17.857055+0000 mgr.x (mgr.24118) 809 : cluster [DBG] pgmap v1169: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T15:54:19.757 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:54:19.860 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:19.988 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:20.091 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:20.101 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:54:19 smithi055 bash[14301]: debug 2020-09-08T15:54:19.758+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:20.366 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:54:19 smithi166 bash[16593]: debug 2020-09-08T15:54:19.987+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:20.520 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:20.626 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:20.813 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:54:20 smithi055 bash[11956]: debug 2020-09-08T15:54:20.514+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:20.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:20 smithi104 bash[9203]: cluster 2020-09-08T15:54:19.857448+0000 mgr.x (mgr.24118) 810 : cluster [DBG] pgmap v1170: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.15k op/s 2020-09-08T15:54:21.053 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:21.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:20 smithi166 bash[9446]: cluster 2020-09-08T15:54:19.857448+0000 mgr.x (mgr.24118) 810 : cluster [DBG] pgmap v1170: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.15k op/s 2020-09-08T15:54:21.149 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:20 smithi055 bash[9296]: cluster 2020-09-08T15:54:19.857448+0000 mgr.x (mgr.24118) 810 : cluster [DBG] pgmap v1170: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.15k op/s 2020-09-08T15:54:21.158 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:21.322 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:21.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:54:21 smithi055 bash[19003]: debug 2020-09-08T15:54:21.322+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:21.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:54:21 smithi104 bash[15273]: debug 2020-09-08T15:54:21.048+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:21.427 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:54:21.814 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:21 smithi104 bash[9203]: cluster 2020-09-08T15:54:20.734218+0000 osd.5 (osd.5) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:21.814 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:21 smithi104 bash[9203]: cluster 2020-09-08T15:54:20.734315+0000 osd.5 (osd.5) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:21.814 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:21 smithi104 bash[9203]: cluster 2020-09-08T15:54:20.824308+0000 mon.a (mon.0) 1621 : cluster [DBG] osdmap e445: 12 total, 12 up, 12 in 2020-09-08T15:54:21.956 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:54:22.064 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:22.072 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:54:21 smithi104 bash[17588]: debug 2020-09-08T15:54:21.956+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:22.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:21 smithi166 bash[9446]: cluster 2020-09-08T15:54:20.734218+0000 osd.5 (osd.5) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:22.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:21 smithi166 bash[9446]: cluster 2020-09-08T15:54:20.734315+0000 osd.5 (osd.5) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:22.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:21 smithi166 bash[9446]: cluster 2020-09-08T15:54:20.824308+0000 mon.a (mon.0) 1621 : cluster [DBG] osdmap e445: 12 total, 12 up, 12 in 2020-09-08T15:54:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:21 smithi055 bash[9296]: cluster 2020-09-08T15:54:20.734218+0000 osd.5 (osd.5) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:21 smithi055 bash[9296]: cluster 2020-09-08T15:54:20.734315+0000 osd.5 (osd.5) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:21 smithi055 bash[9296]: cluster 2020-09-08T15:54:20.824308+0000 mon.a (mon.0) 1621 : cluster [DBG] osdmap e445: 12 total, 12 up, 12 in 2020-09-08T15:54:22.287 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:22.391 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:22.427 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:54:22 smithi104 bash[19922]: debug 2020-09-08T15:54:22.288+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:22.793 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:22.838 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:54:22.873 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:54:22.897 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:54:22.911 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:54:22.951 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:54:22.990 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:54:23.034 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:54:23.056 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:54:23.068 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:54:23.105 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:54:23.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:22 smithi166 bash[9446]: cluster 2020-09-08T15:54:21.601224+0000 osd.6 (osd.6) 71 : cluster [DBG] 5.2 scrub starts 2020-09-08T15:54:23.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:22 smithi166 bash[9446]: cluster 2020-09-08T15:54:21.857820+0000 mgr.x (mgr.24118) 811 : cluster [DBG] pgmap v1172: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 744 op/s 2020-09-08T15:54:23.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:22 smithi166 bash[9446]: cluster 2020-09-08T15:54:22.655648+0000 osd.10 (osd.10) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:23.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:22 smithi166 bash[9446]: cluster 2020-09-08T15:54:22.655733+0000 osd.10 (osd.10) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:23.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:22 smithi166 bash[9446]: audit 2020-09-08T15:54:22.804973+0000 mon.a (mon.0) 1622 : audit [DBG] from='client.? v1:172.21.15.104:0/2456722161' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:54:23.145 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:54:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:22 smithi055 bash[9296]: cluster 2020-09-08T15:54:21.601224+0000 osd.6 (osd.6) 71 : cluster [DBG] 5.2 scrub starts 2020-09-08T15:54:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:22 smithi055 bash[9296]: cluster 2020-09-08T15:54:21.857820+0000 mgr.x (mgr.24118) 811 : cluster [DBG] pgmap v1172: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 744 op/s 2020-09-08T15:54:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:22 smithi055 bash[9296]: cluster 2020-09-08T15:54:22.655648+0000 osd.10 (osd.10) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:23.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:22 smithi055 bash[9296]: cluster 2020-09-08T15:54:22.655733+0000 osd.10 (osd.10) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:23.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:22 smithi055 bash[9296]: audit 2020-09-08T15:54:22.804973+0000 mon.a (mon.0) 1622 : audit [DBG] from='client.? v1:172.21.15.104:0/2456722161' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:54:23.159 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:54:22 smithi055 bash[11956]: debug 2020-09-08T15:54:22.794+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:23.161 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:23.170 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:22 smithi104 bash[9203]: cluster 2020-09-08T15:54:21.601224+0000 osd.6 (osd.6) 71 : cluster [DBG] 5.2 scrub starts 2020-09-08T15:54:23.170 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:22 smithi104 bash[9203]: cluster 2020-09-08T15:54:21.857820+0000 mgr.x (mgr.24118) 811 : cluster [DBG] pgmap v1172: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 744 op/s 2020-09-08T15:54:23.170 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:22 smithi104 bash[9203]: cluster 2020-09-08T15:54:22.655648+0000 osd.10 (osd.10) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:23.171 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:22 smithi104 bash[9203]: cluster 2020-09-08T15:54:22.655733+0000 osd.10 (osd.10) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:23.171 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:22 smithi104 bash[9203]: audit 2020-09-08T15:54:22.804973+0000 mon.a (mon.0) 1622 : audit [DBG] from='client.? v1:172.21.15.104:0/2456722161' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:54:23.171 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:54:23 smithi104 bash[17588]: debug 2020-09-08T15:54:23.056+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:23.180 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:54:23.221 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:54:23.262 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:54:23.268 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:54:23.268 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:54:23.269 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:54:23.269 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:54:23.269 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:54:23.269 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:54:23.270 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:54:23.270 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:54:23.270 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:54:23.270 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:54:23.270 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:54:23.271 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:54:23.282 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:23.386 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:23.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:54:23 smithi104 bash[19922]: debug 2020-09-08T15:54:23.280+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:23.593 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:23.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:54:23 smithi055 bash[16646]: debug 2020-09-08T15:54:23.594+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:23.697 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:23.838 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:54:23.846 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:23.949 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:24.085 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:54:24 smithi166 bash[16593]: debug 2020-09-08T15:54:24.087+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:24.086 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:24.087 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:23 smithi166 bash[9446]: cluster 2020-09-08T15:54:22.828795+0000 mon.a (mon.0) 1623 : cluster [DBG] osdmap e446: 12 total, 12 up, 12 in 2020-09-08T15:54:24.088 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:23 smithi166 bash[9446]: audit 2020-09-08T15:54:23.166105+0000 mon.a (mon.0) 1624 : audit [INF] from='client.? v1:172.21.15.104:0/4107551312' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:54:24.109 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:54:24.109 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:54:24.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:54:23 smithi055 bash[16646]: debug 2020-09-08T15:54:23.842+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:24.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:23 smithi055 bash[9296]: cluster 2020-09-08T15:54:22.828795+0000 mon.a (mon.0) 1623 : cluster [DBG] osdmap e446: 12 total, 12 up, 12 in 2020-09-08T15:54:24.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:23 smithi055 bash[9296]: audit 2020-09-08T15:54:23.166105+0000 mon.a (mon.0) 1624 : audit [INF] from='client.? v1:172.21.15.104:0/4107551312' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:54:24.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:23 smithi104 bash[9203]: cluster 2020-09-08T15:54:22.828795+0000 mon.a (mon.0) 1623 : cluster [DBG] osdmap e446: 12 total, 12 up, 12 in 2020-09-08T15:54:24.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:23 smithi104 bash[9203]: audit 2020-09-08T15:54:23.166105+0000 mon.a (mon.0) 1624 : audit [INF] from='client.? v1:172.21.15.104:0/4107551312' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:54:24.189 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:25.106 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:54:25 smithi104 bash[15273]: debug 2020-09-08T15:54:25.104+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:25.107 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:24 smithi104 bash[9203]: cluster 2020-09-08T15:54:23.649247+0000 osd.6 (osd.6) 72 : cluster [DBG] 5.2 scrub ok 2020-09-08T15:54:25.107 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:24 smithi104 bash[9203]: cluster 2020-09-08T15:54:23.825946+0000 mon.a (mon.0) 1625 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:54:25.107 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:24 smithi104 bash[9203]: audit 2020-09-08T15:54:23.841521+0000 mon.a (mon.0) 1626 : audit [INF] from='client.? v1:172.21.15.104:0/4107551312' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:54:25.108 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:24 smithi104 bash[9203]: cluster 2020-09-08T15:54:23.841567+0000 mon.a (mon.0) 1627 : cluster [DBG] osdmap e447: 12 total, 12 up, 12 in 2020-09-08T15:54:25.108 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:24 smithi104 bash[9203]: cluster 2020-09-08T15:54:23.858399+0000 mgr.x (mgr.24118) 812 : cluster [DBG] pgmap v1175: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 5.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T15:54:25.108 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:25.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:24 smithi166 bash[9446]: cluster 2020-09-08T15:54:23.649247+0000 osd.6 (osd.6) 72 : cluster [DBG] 5.2 scrub ok 2020-09-08T15:54:25.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:24 smithi166 bash[9446]: cluster 2020-09-08T15:54:23.825946+0000 mon.a (mon.0) 1625 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:54:25.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:24 smithi166 bash[9446]: audit 2020-09-08T15:54:23.841521+0000 mon.a (mon.0) 1626 : audit [INF] from='client.? v1:172.21.15.104:0/4107551312' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:54:25.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:24 smithi166 bash[9446]: cluster 2020-09-08T15:54:23.841567+0000 mon.a (mon.0) 1627 : cluster [DBG] osdmap e447: 12 total, 12 up, 12 in 2020-09-08T15:54:25.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:24 smithi166 bash[9446]: cluster 2020-09-08T15:54:23.858399+0000 mgr.x (mgr.24118) 812 : cluster [DBG] pgmap v1175: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 5.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T15:54:25.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:24 smithi055 bash[9296]: cluster 2020-09-08T15:54:23.649247+0000 osd.6 (osd.6) 72 : cluster [DBG] 5.2 scrub ok 2020-09-08T15:54:25.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:24 smithi055 bash[9296]: cluster 2020-09-08T15:54:23.825946+0000 mon.a (mon.0) 1625 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:54:25.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:24 smithi055 bash[9296]: audit 2020-09-08T15:54:23.841521+0000 mon.a (mon.0) 1626 : audit [INF] from='client.? v1:172.21.15.104:0/4107551312' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:54:25.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:24 smithi055 bash[9296]: cluster 2020-09-08T15:54:23.841567+0000 mon.a (mon.0) 1627 : cluster [DBG] osdmap e447: 12 total, 12 up, 12 in 2020-09-08T15:54:25.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:24 smithi055 bash[9296]: cluster 2020-09-08T15:54:23.858399+0000 mgr.x (mgr.24118) 812 : cluster [DBG] pgmap v1175: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 5.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T15:54:25.211 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:25.235 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2020-09-08T15:54:25.331 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:25.431 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:54:25.433 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:25.661 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:54:25 smithi055 bash[19003]: debug 2020-09-08T15:54:25.318+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:26.036 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:26.140 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:26.264 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:26.371 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:54:26.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:54:26 smithi055 bash[16646]: debug 2020-09-08T15:54:26.262+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:26.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:54:26 smithi055 bash[19003]: debug 2020-09-08T15:54:26.034+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:26.520 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:54:26.623 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:54:26.764 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:54:26 smithi166 bash[18960]: debug 2020-09-08T15:54:26.519+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:26.764 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:54:26.867 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:54:27.089 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:54:27.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:26 smithi166 bash[9446]: cluster 2020-09-08T15:54:25.858924+0000 mgr.x (mgr.24118) 813 : cluster [DBG] pgmap v1176: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 5.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 820 op/s 2020-09-08T15:54:27.118 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:54:26 smithi166 bash[14254]: debug 2020-09-08T15:54:26.763+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:27.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:26 smithi055 bash[9296]: cluster 2020-09-08T15:54:25.858924+0000 mgr.x (mgr.24118) 813 : cluster [DBG] pgmap v1176: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 5.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 820 op/s 2020-09-08T15:54:27.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:54:27 smithi055 bash[14301]: debug 2020-09-08T15:54:27.090+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:27.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:26 smithi104 bash[9203]: cluster 2020-09-08T15:54:25.858924+0000 mgr.x (mgr.24118) 813 : cluster [DBG] pgmap v1176: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 5.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 820 op/s 2020-09-08T15:54:27.198 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:27.310 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:27.373 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:54:27.390 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:54:27.413 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:27.481 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:54:27 smithi055 bash[16646]: debug 2020-09-08T15:54:27.310+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:27.552 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:54:27.572 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:27.581 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:54:27.675 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:27.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:54:27 smithi055 bash[11956]: debug 2020-09-08T15:54:27.570+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:28.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:27 smithi104 bash[9203]: cluster 2020-09-08T15:54:27.188539+0000 osd.3 (osd.3) 63 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:28.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:27 smithi104 bash[9203]: cluster 2020-09-08T15:54:27.188631+0000 osd.3 (osd.3) 64 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:28.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:27 smithi166 bash[9446]: cluster 2020-09-08T15:54:27.188539+0000 osd.3 (osd.3) 63 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:28.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:27 smithi166 bash[9446]: cluster 2020-09-08T15:54:27.188631+0000 osd.3 (osd.3) 64 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:28.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:27 smithi055 bash[9296]: cluster 2020-09-08T15:54:27.188539+0000 osd.3 (osd.3) 63 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:28.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:27 smithi055 bash[9296]: cluster 2020-09-08T15:54:27.188631+0000 osd.3 (osd.3) 64 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:28.802 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:28.907 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:54:29.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:54:28 smithi104 bash[22305]: debug 2020-09-08T15:54:28.804+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:29.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:28 smithi104 bash[9203]: cluster 2020-09-08T15:54:27.859340+0000 mgr.x (mgr.24118) 814 : cluster [DBG] pgmap v1177: 17 pgs: 17 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.27k op/s 2020-09-08T15:54:29.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:28 smithi104 bash[9203]: cluster 2020-09-08T15:54:27.936694+0000 mon.a (mon.0) 1628 : cluster [DBG] osdmap e448: 12 total, 12 up, 12 in 2020-09-08T15:54:29.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:28 smithi166 bash[9446]: cluster 2020-09-08T15:54:27.859340+0000 mgr.x (mgr.24118) 814 : cluster [DBG] pgmap v1177: 17 pgs: 17 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.27k op/s 2020-09-08T15:54:29.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:28 smithi166 bash[9446]: cluster 2020-09-08T15:54:27.936694+0000 mon.a (mon.0) 1628 : cluster [DBG] osdmap e448: 12 total, 12 up, 12 in 2020-09-08T15:54:29.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:28 smithi055 bash[9296]: cluster 2020-09-08T15:54:27.859340+0000 mgr.x (mgr.24118) 814 : cluster [DBG] pgmap v1177: 17 pgs: 17 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.27k op/s 2020-09-08T15:54:29.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:28 smithi055 bash[9296]: cluster 2020-09-08T15:54:27.936694+0000 mon.a (mon.0) 1628 : cluster [DBG] osdmap e448: 12 total, 12 up, 12 in 2020-09-08T15:54:29.538 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:54:29.642 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:29.857 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:54:29 smithi104 bash[17588]: debug 2020-09-08T15:54:29.532+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:29.858 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:29.961 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:54:30.076 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:54:30.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:54:29 smithi104 bash[19922]: debug 2020-09-08T15:54:29.856+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:30.179 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:54:30.235 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:54:30.235 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:54:30.310 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:54:30 smithi055 bash[14301]: debug 2020-09-08T15:54:30.074+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:30.320 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:54:30.424 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:54:30.542 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:54:30 smithi166 bash[14254]: debug 2020-09-08T15:54:30.319+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:30.544 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:54:30.650 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:54:30.780 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:54:30.818 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:54:30 smithi166 bash[14254]: debug 2020-09-08T15:54:30.543+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:30.819 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:54:30 smithi166 bash[14254]: debug 2020-09-08T15:54:30.779+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:30.888 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:54:30.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:30 smithi104 bash[9203]: cluster 2020-09-08T15:54:29.859736+0000 mgr.x (mgr.24118) 815 : cluster [DBG] pgmap v1179: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 913 op/s 2020-09-08T15:54:31.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:30 smithi166 bash[9446]: cluster 2020-09-08T15:54:29.859736+0000 mgr.x (mgr.24118) 815 : cluster [DBG] pgmap v1179: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 913 op/s 2020-09-08T15:54:31.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:30 smithi055 bash[9296]: cluster 2020-09-08T15:54:29.859736+0000 mgr.x (mgr.24118) 815 : cluster [DBG] pgmap v1179: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 913 op/s 2020-09-08T15:54:31.157 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:54:31.262 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:54:31.275 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2020-09-08T15:54:31.377 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:54:31.427 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:54:31.483 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:54:31.594 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:54:31 smithi166 bash[18960]: debug 2020-09-08T15:54:31.379+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:31.594 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:54:31.656 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:54:31 smithi055 bash[14301]: debug 2020-09-08T15:54:31.158+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:31.699 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:31.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:54:31 smithi166 bash[11928]: debug 2020-09-08T15:54:31.595+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:32.110 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:32.215 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:32.340 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:32.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:54:32 smithi104 bash[15273]: debug 2020-09-08T15:54:32.112+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:32.443 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:32.617 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:54:32 smithi166 bash[16593]: debug 2020-09-08T15:54:32.335+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:32.979 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:32 smithi104 bash[9203]: cluster 2020-09-08T15:54:31.860186+0000 mgr.x (mgr.24118) 816 : cluster [DBG] pgmap v1180: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 727 op/s 2020-09-08T15:54:32.986 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:33.089 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:33.197 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:32 smithi055 bash[9296]: cluster 2020-09-08T15:54:31.860186+0000 mgr.x (mgr.24118) 816 : cluster [DBG] pgmap v1180: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 727 op/s 2020-09-08T15:54:33.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:32 smithi166 bash[9446]: cluster 2020-09-08T15:54:31.860186+0000 mgr.x (mgr.24118) 816 : cluster [DBG] pgmap v1180: 17 pgs: 17 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 727 op/s 2020-09-08T15:54:33.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:54:32 smithi104 bash[15273]: debug 2020-09-08T15:54:32.988+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:33.750 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:33.854 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:54:33.994 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:54:34.097 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:34.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:54:33 smithi166 bash[14254]: debug 2020-09-08T15:54:33.995+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:34.180 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:54:33 smithi104 bash[15273]: debug 2020-09-08T15:54:33.752+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:34.668 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:34.772 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:34.856 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:54:34.906 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:54:34.970 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:54:35.008 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:54:35.044 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:54:35.084 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:54:35.118 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:54:35.155 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:54:35.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:54:34 smithi055 bash[11956]: debug 2020-09-08T15:54:34.666+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:34 smithi055 bash[9296]: cluster 2020-09-08T15:54:33.381024+0000 osd.8 (osd.8) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:34 smithi055 bash[9296]: cluster 2020-09-08T15:54:33.381106+0000 osd.8 (osd.8) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:34 smithi055 bash[9296]: cluster 2020-09-08T15:54:33.860693+0000 mgr.x (mgr.24118) 817 : cluster [DBG] pgmap v1181: 17 pgs: 17 active+clean; 4.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 988 op/s 2020-09-08T15:54:35.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:34 smithi055 bash[9296]: cluster 2020-09-08T15:54:33.922231+0000 mon.a (mon.0) 1629 : cluster [DBG] osdmap e449: 12 total, 12 up, 12 in 2020-09-08T15:54:35.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:34 smithi055 bash[9296]: audit 2020-09-08T15:54:34.793759+0000 mon.b (mon.1) 102 : audit [DBG] from='client.? v1:172.21.15.104:0/381669927' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:54:35.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:34 smithi104 bash[9203]: cluster 2020-09-08T15:54:33.381024+0000 osd.8 (osd.8) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:35.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:34 smithi104 bash[9203]: cluster 2020-09-08T15:54:33.381106+0000 osd.8 (osd.8) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:35.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:34 smithi104 bash[9203]: cluster 2020-09-08T15:54:33.860693+0000 mgr.x (mgr.24118) 817 : cluster [DBG] pgmap v1181: 17 pgs: 17 active+clean; 4.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 988 op/s 2020-09-08T15:54:35.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:34 smithi104 bash[9203]: cluster 2020-09-08T15:54:33.922231+0000 mon.a (mon.0) 1629 : cluster [DBG] osdmap e449: 12 total, 12 up, 12 in 2020-09-08T15:54:35.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:34 smithi104 bash[9203]: audit 2020-09-08T15:54:34.793759+0000 mon.b (mon.1) 102 : audit [DBG] from='client.? v1:172.21.15.104:0/381669927' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:54:35.195 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:54:35.226 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:54:35.266 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:54:35.297 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:54:35.303 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:54:35.303 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:54:35.303 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:54:35.303 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:54:35.304 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:54:35.304 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:54:35.304 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:54:35.304 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:54:35.304 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:54:35.304 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:54:35.305 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:54:35.305 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:54:35.328 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:35.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:34 smithi166 bash[9446]: cluster 2020-09-08T15:54:33.381024+0000 osd.8 (osd.8) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:35.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:34 smithi166 bash[9446]: cluster 2020-09-08T15:54:33.381106+0000 osd.8 (osd.8) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:35.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:34 smithi166 bash[9446]: cluster 2020-09-08T15:54:33.860693+0000 mgr.x (mgr.24118) 817 : cluster [DBG] pgmap v1181: 17 pgs: 17 active+clean; 4.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 988 op/s 2020-09-08T15:54:35.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:34 smithi166 bash[9446]: cluster 2020-09-08T15:54:33.922231+0000 mon.a (mon.0) 1629 : cluster [DBG] osdmap e449: 12 total, 12 up, 12 in 2020-09-08T15:54:35.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:34 smithi166 bash[9446]: audit 2020-09-08T15:54:34.793759+0000 mon.b (mon.1) 102 : audit [DBG] from='client.? v1:172.21.15.104:0/381669927' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:54:35.432 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:35.582 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:54:35 smithi055 bash[16646]: debug 2020-09-08T15:54:35.322+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:35.582 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:54:35 smithi055 bash[11956]: debug 2020-09-08T15:54:35.582+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:35.583 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:35.689 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:54:35.827 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:54:35.931 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:35.933 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:54:35.933 INFO:teuthology.orchestra.run.smithi055.stdout: "size": 20, 2020-09-08T15:54:35.933 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 600, 2020-09-08T15:54:35.934 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T15:54:35.934 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.934 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35381.0:110320 4.a 4:564a75ba:::benchmark_data_smithi166_32054_object110319:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e341)", 2020-09-08T15:54:35.934 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:48:21.859786+0000", 2020-09-08T15:54:35.934 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 374.07496247300003, 2020-09-08T15:54:35.935 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13075439999999999, 2020-09-08T15:54:35.935 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:35.935 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:35.935 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:35.935 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35381", 2020-09-08T15:54:35.936 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:54:35.936 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 110320 2020-09-08T15:54:35.936 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.936 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:35.936 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.937 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:35.937 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:21.859786+0000", 2020-09-08T15:54:35.937 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:35.937 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.937 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.938 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:35.938 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:21.859786+0000", 2020-09-08T15:54:35.938 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999552 2020-09-08T15:54:35.938 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.938 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.939 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:35.939 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:21.859741+0000", 2020-09-08T15:54:35.939 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010720100000000001 2020-09-08T15:54:35.939 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.939 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.939 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:35.940 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:21.859849+0000", 2020-09-08T15:54:35.940 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1447999999999999e-05 2020-09-08T15:54:35.940 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.940 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.940 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:35.941 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:21.859860+0000", 2020-09-08T15:54:35.941 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.7350000000000004e-06 2020-09-08T15:54:35.941 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.941 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.941 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:35.942 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:21.859865+0000", 2020-09-08T15:54:35.942 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.6563999999999997e-05 2020-09-08T15:54:35.942 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.942 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.942 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:35.943 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:21.859911+0000", 2020-09-08T15:54:35.943 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012758800000000001 2020-09-08T15:54:35.943 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.943 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.943 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:35.943 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:21.860039+0000", 2020-09-08T15:54:35.944 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012362500000000001 2020-09-08T15:54:35.944 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.944 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.944 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:35.944 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:21.860162+0000", 2020-09-08T15:54:35.945 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0031305489999999998 2020-09-08T15:54:35.945 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.945 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.945 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:35.945 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:21.863293+0000", 2020-09-08T15:54:35.946 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.12698815299999999 2020-09-08T15:54:35.946 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.946 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.946 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:35.946 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:21.990281+0000", 2020-09-08T15:54:35.947 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000149073 2020-09-08T15:54:35.947 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.947 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.947 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:35.947 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:21.990430+0000", 2020-09-08T15:54:35.947 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000110287 2020-09-08T15:54:35.948 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.948 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.948 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:35.948 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:21.990541+0000", 2020-09-08T15:54:35.948 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13075439999999999 2020-09-08T15:54:35.949 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:35.949 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:35.949 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:35.949 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.949 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.950 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35381.0:113594 4.a 4:57ff12af:::benchmark_data_smithi166_32054_object113593:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e341)", 2020-09-08T15:54:35.950 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:48:24.641851+0000", 2020-09-08T15:54:35.950 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 371.29289712100001, 2020-09-08T15:54:35.950 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.18400145200000001, 2020-09-08T15:54:35.951 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:35.951 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:35.951 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:35.951 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35381", 2020-09-08T15:54:35.951 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:54:35.952 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 113594 2020-09-08T15:54:35.952 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.952 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:35.952 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.952 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:35.953 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.641851+0000", 2020-09-08T15:54:35.953 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:35.953 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.954 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.954 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:35.954 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.641851+0000", 2020-09-08T15:54:35.954 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999704 2020-09-08T15:54:35.954 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.954 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.955 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:35.955 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.641822+0000", 2020-09-08T15:54:35.955 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4364000000000007e-05 2020-09-08T15:54:35.955 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.955 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.956 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:35.956 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.641916+0000", 2020-09-08T15:54:35.956 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.9680000000000001e-06 2020-09-08T15:54:35.956 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.956 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.957 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:35.957 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.641926+0000", 2020-09-08T15:54:35.957 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.5204000000000001e-05 2020-09-08T15:54:35.957 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.957 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.958 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:35.958 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.641951+0000", 2020-09-08T15:54:35.958 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00185614 2020-09-08T15:54:35.958 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.958 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.959 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:35.959 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.643807+0000", 2020-09-08T15:54:35.959 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00027461 2020-09-08T15:54:35.959 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.959 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.959 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:35.960 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.644082+0000", 2020-09-08T15:54:35.960 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00021400299999999999 2020-09-08T15:54:35.960 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.960 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.960 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:35.961 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.644296+0000", 2020-09-08T15:54:35.961 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.01069876 2020-09-08T15:54:35.961 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.961 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.961 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:35.962 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.654995+0000", 2020-09-08T15:54:35.962 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.17071048899999999 2020-09-08T15:54:35.962 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.962 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.962 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:35.963 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.825705+0000", 2020-09-08T15:54:35.963 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.8972000000000001e-05 2020-09-08T15:54:35.963 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.963 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.963 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:35.964 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.825794+0000", 2020-09-08T15:54:35.964 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.8644000000000003e-05 2020-09-08T15:54:35.964 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.964 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.964 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:35.964 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.825853+0000", 2020-09-08T15:54:35.965 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.18400145200000001 2020-09-08T15:54:35.965 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:35.965 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:35.965 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:35.965 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.966 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.966 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35381.0:113597 4.a 4:551d5bad:::benchmark_data_smithi166_32054_object113596:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e341)", 2020-09-08T15:54:35.966 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:48:24.642604+0000", 2020-09-08T15:54:35.966 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 371.292145027, 2020-09-08T15:54:35.967 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.206369099, 2020-09-08T15:54:35.967 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:35.967 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:35.967 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:35.967 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35381", 2020-09-08T15:54:35.968 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:54:35.968 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 113597 2020-09-08T15:54:35.968 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.968 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:35.968 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.968 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:35.969 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.642604+0000", 2020-09-08T15:54:35.969 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:35.969 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.969 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.969 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:35.970 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.642604+0000", 2020-09-08T15:54:35.970 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999576 2020-09-08T15:54:35.971 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.971 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.971 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:35.971 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.642561+0000", 2020-09-08T15:54:35.971 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00028575200000000001 2020-09-08T15:54:35.972 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.972 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.972 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:35.972 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.642847+0000", 2020-09-08T15:54:35.972 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5515999999999998e-05 2020-09-08T15:54:35.973 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.973 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.973 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:35.973 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.642862+0000", 2020-09-08T15:54:35.973 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0190000000000003e-06 2020-09-08T15:54:35.974 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.974 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.974 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:35.974 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.642868+0000", 2020-09-08T15:54:35.974 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.006725506 2020-09-08T15:54:35.974 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.975 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.975 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:35.975 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.649594+0000", 2020-09-08T15:54:35.975 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013936600000000001 2020-09-08T15:54:35.975 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.976 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.976 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:35.976 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.649733+0000", 2020-09-08T15:54:35.976 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016271999999999999 2020-09-08T15:54:35.976 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.977 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.977 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:35.977 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.649896+0000", 2020-09-08T15:54:35.977 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.01274861 2020-09-08T15:54:35.977 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.977 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.978 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:35.978 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.662644+0000", 2020-09-08T15:54:35.978 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.186217193 2020-09-08T15:54:35.978 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.978 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.979 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:35.979 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.848862+0000", 2020-09-08T15:54:35.979 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.7253000000000001e-05 2020-09-08T15:54:35.979 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.979 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.980 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:35.980 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.848929+0000", 2020-09-08T15:54:35.980 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.3744e-05 2020-09-08T15:54:35.980 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.980 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.981 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:35.981 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.848973+0000", 2020-09-08T15:54:35.981 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.206369099 2020-09-08T15:54:35.981 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:35.981 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:35.981 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:35.982 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.982 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.982 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35381.0:113601 4.a 4:513d4ad2:::benchmark_data_smithi166_32054_object113600:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e341)", 2020-09-08T15:54:35.982 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:48:24.648250+0000", 2020-09-08T15:54:35.982 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 371.28649861399998, 2020-09-08T15:54:35.983 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.207495392, 2020-09-08T15:54:35.983 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:35.983 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:35.983 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:35.983 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35381", 2020-09-08T15:54:35.984 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:54:35.984 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 113601 2020-09-08T15:54:35.984 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.984 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:35.984 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.984 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:35.985 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.648250+0000", 2020-09-08T15:54:35.985 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:35.985 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.985 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.985 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:35.986 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.648250+0000", 2020-09-08T15:54:35.986 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999619 2020-09-08T15:54:35.986 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.986 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.986 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:35.987 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.648212+0000", 2020-09-08T15:54:35.987 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000141252 2020-09-08T15:54:35.987 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.987 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.987 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:35.987 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.648353+0000", 2020-09-08T15:54:35.988 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6433999999999999e-05 2020-09-08T15:54:35.988 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.988 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.988 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:35.988 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.648369+0000", 2020-09-08T15:54:35.989 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.4640000000000003e-06 2020-09-08T15:54:35.989 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.989 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.989 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:35.989 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.648376+0000", 2020-09-08T15:54:35.990 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0066580609999999998 2020-09-08T15:54:35.990 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.990 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.990 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:35.990 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.655034+0000", 2020-09-08T15:54:35.991 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00020932200000000001 2020-09-08T15:54:35.991 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.991 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.991 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:35.991 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.655243+0000", 2020-09-08T15:54:35.991 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00023825699999999999 2020-09-08T15:54:35.992 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.992 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.992 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:35.992 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.655481+0000", 2020-09-08T15:54:35.992 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0071682899999999999 2020-09-08T15:54:35.993 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.993 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.993 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:35.993 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.662650+0000", 2020-09-08T15:54:35.993 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.19293550400000001 2020-09-08T15:54:35.994 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.994 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.994 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:35.994 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.855585+0000", 2020-09-08T15:54:35.994 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.9516000000000001e-05 2020-09-08T15:54:35.994 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.995 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.995 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:35.995 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.855685+0000", 2020-09-08T15:54:35.995 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0597999999999998e-05 2020-09-08T15:54:35.995 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.996 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.996 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:35.996 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.855745+0000", 2020-09-08T15:54:35.996 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.207495392 2020-09-08T15:54:35.996 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:35.997 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:35.997 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:35.997 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.997 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:35.997 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35381.0:113607 4.a 4:50f0523e:::benchmark_data_smithi166_32054_object113606:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e341)", 2020-09-08T15:54:35.998 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:48:24.656126+0000", 2020-09-08T15:54:35.998 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 371.27862212899998, 2020-09-08T15:54:35.998 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.19978485700000001, 2020-09-08T15:54:35.998 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:35.998 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:35.998 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:35.999 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35381", 2020-09-08T15:54:35.999 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:54:35.999 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 113607 2020-09-08T15:54:35.999 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:35.999 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:36.000 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.000 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:36.000 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.656126+0000", 2020-09-08T15:54:36.000 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:36.000 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.001 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.001 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:36.001 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.656126+0000", 2020-09-08T15:54:36.001 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999743 2020-09-08T15:54:36.001 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.001 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.002 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:36.002 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.656101+0000", 2020-09-08T15:54:36.002 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000105142 2020-09-08T15:54:36.002 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.002 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.003 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:36.003 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.656206+0000", 2020-09-08T15:54:36.003 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5288e-05 2020-09-08T15:54:36.003 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.003 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.004 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:36.004 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.656221+0000", 2020-09-08T15:54:36.004 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.1720000000000001e-06 2020-09-08T15:54:36.004 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.004 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.004 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:36.005 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.656228+0000", 2020-09-08T15:54:36.005 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0030769220000000002 2020-09-08T15:54:36.005 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.005 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.005 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:36.006 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.659305+0000", 2020-09-08T15:54:36.006 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000148264 2020-09-08T15:54:36.006 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.006 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.006 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:36.007 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.659453+0000", 2020-09-08T15:54:36.007 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00022748399999999999 2020-09-08T15:54:36.007 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.007 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.007 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:36.008 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.659681+0000", 2020-09-08T15:54:36.008 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0031909389999999998 2020-09-08T15:54:36.008 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.008 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.008 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:36.008 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.662872+0000", 2020-09-08T15:54:36.009 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.19293276700000001 2020-09-08T15:54:36.009 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.009 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.009 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:36.009 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.855804+0000", 2020-09-08T15:54:36.010 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9264e-05 2020-09-08T15:54:36.010 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.010 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.010 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:36.010 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.855864+0000", 2020-09-08T15:54:36.011 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.7590999999999999e-05 2020-09-08T15:54:36.011 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.011 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.011 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:36.011 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:24.855911+0000", 2020-09-08T15:54:36.012 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.19978485700000001 2020-09-08T15:54:36.012 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.012 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:36.012 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.012 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.012 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.013 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35381.0:116137 4.a 4:57fa5297:::benchmark_data_smithi166_32054_object116136:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e343)", 2020-09-08T15:54:36.013 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:48:27.422395+0000", 2020-09-08T15:54:36.013 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 368.51235404699997, 2020-09-08T15:54:36.013 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.17148010999999999, 2020-09-08T15:54:36.013 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:36.014 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:36.014 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:36.014 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35381", 2020-09-08T15:54:36.014 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:54:36.014 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 116137 2020-09-08T15:54:36.015 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.015 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:36.015 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.015 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:36.015 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:27.422395+0000", 2020-09-08T15:54:36.016 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:36.016 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.016 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.016 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:36.016 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:27.422395+0000", 2020-09-08T15:54:36.016 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999695 2020-09-08T15:54:36.017 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.017 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.017 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:36.017 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:27.422364+0000", 2020-09-08T15:54:36.017 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000108078 2020-09-08T15:54:36.018 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.018 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.018 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:36.018 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:27.422472+0000", 2020-09-08T15:54:36.018 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6733999999999999e-05 2020-09-08T15:54:36.019 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.019 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.019 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:36.019 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:27.422489+0000", 2020-09-08T15:54:36.019 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.5839999999999995e-06 2020-09-08T15:54:36.019 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.020 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.020 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:36.020 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:27.422497+0000", 2020-09-08T15:54:36.020 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.5569999999999997e-05 2020-09-08T15:54:36.020 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.021 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.021 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:36.021 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:27.422563+0000", 2020-09-08T15:54:36.021 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016406099999999999 2020-09-08T15:54:36.021 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.022 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.022 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:36.022 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:27.422727+0000", 2020-09-08T15:54:36.022 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014496999999999999 2020-09-08T15:54:36.022 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.023 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.023 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:36.023 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:27.422872+0000", 2020-09-08T15:54:36.023 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.003669046 2020-09-08T15:54:36.023 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.024 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.024 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:36.024 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:27.426541+0000", 2020-09-08T15:54:36.024 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.16712445400000001 2020-09-08T15:54:36.024 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.025 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.025 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:36.025 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:27.593665+0000", 2020-09-08T15:54:36.025 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000115553 2020-09-08T15:54:36.025 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.026 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.026 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:36.026 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:27.593781+0000", 2020-09-08T15:54:36.026 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3777999999999996e-05 2020-09-08T15:54:36.026 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.027 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.027 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:36.027 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:48:27.593875+0000", 2020-09-08T15:54:36.027 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.17148010999999999 2020-09-08T15:54:36.027 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.028 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:36.028 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.028 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.028 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.028 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35381.0:158500 4.a 4:5787aec7:::benchmark_data_smithi166_32054_object16358:head [delete] snapc 0=[] ondisk+write+known_if_redirected e354)", 2020-09-08T15:54:36.028 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:49:10.923701+0000", 2020-09-08T15:54:36.029 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 325.01104775499999, 2020-09-08T15:54:36.029 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.18672731100000001, 2020-09-08T15:54:36.029 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:36.029 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:36.029 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:36.030 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35381", 2020-09-08T15:54:36.030 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:54:36.030 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 158500 2020-09-08T15:54:36.030 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.030 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:36.031 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.031 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:36.031 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.923701+0000", 2020-09-08T15:54:36.031 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:36.031 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.032 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.032 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:36.032 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.923701+0000", 2020-09-08T15:54:36.032 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999719 2020-09-08T15:54:36.032 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.032 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.033 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:36.033 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.923672+0000", 2020-09-08T15:54:36.033 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.6552999999999998e-05 2020-09-08T15:54:36.033 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.033 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.034 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:36.034 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.923739+0000", 2020-09-08T15:54:36.034 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4090000000000001e-05 2020-09-08T15:54:36.034 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.034 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.035 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:36.035 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.923753+0000", 2020-09-08T15:54:36.035 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.083e-06 2020-09-08T15:54:36.035 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.035 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.036 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:36.036 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.923759+0000", 2020-09-08T15:54:36.036 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.0016000000000002e-05 2020-09-08T15:54:36.036 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.036 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.036 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:36.037 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.923829+0000", 2020-09-08T15:54:36.037 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000164685 2020-09-08T15:54:36.037 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.037 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.037 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:36.038 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.923994+0000", 2020-09-08T15:54:36.038 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012002699999999999 2020-09-08T15:54:36.038 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.038 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.038 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:36.039 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.924114+0000", 2020-09-08T15:54:36.039 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.17661674499999999 2020-09-08T15:54:36.039 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.039 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.039 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:36.039 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:11.100731+0000", 2020-09-08T15:54:36.040 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0095449190000000007 2020-09-08T15:54:36.040 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.040 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.040 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:36.040 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:11.110276+0000", 2020-09-08T15:54:36.041 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.1063000000000005e-05 2020-09-08T15:54:36.041 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.041 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.041 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:36.041 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:11.110367+0000", 2020-09-08T15:54:36.042 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.1496999999999998e-05 2020-09-08T15:54:36.042 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.042 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.042 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:36.042 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:11.110428+0000", 2020-09-08T15:54:36.043 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.18672731100000001 2020-09-08T15:54:36.043 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.043 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:36.043 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.043 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.043 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.044 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35381.0:215836 4.a 4:5bf7e2cb:::benchmark_data_smithi166_32054_object73694:head [delete] snapc 0=[] ondisk+write+known_if_redirected e369)", 2020-09-08T15:54:36.044 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:50:08.137886+0000", 2020-09-08T15:54:36.044 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 267.79686269699999, 2020-09-08T15:54:36.044 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.12267346899999999, 2020-09-08T15:54:36.044 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:36.045 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:36.045 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:36.045 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35381", 2020-09-08T15:54:36.045 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:54:36.045 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 215836 2020-09-08T15:54:36.046 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.046 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:36.046 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.046 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:36.046 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.137886+0000", 2020-09-08T15:54:36.047 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:36.047 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.047 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.047 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:36.047 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.137886+0000", 2020-09-08T15:54:36.047 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999461 2020-09-08T15:54:36.048 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.048 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.048 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:36.048 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.137832+0000", 2020-09-08T15:54:36.048 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012018 2020-09-08T15:54:36.049 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.049 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.049 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:36.049 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.137952+0000", 2020-09-08T15:54:36.049 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.387e-05 2020-09-08T15:54:36.050 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.050 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.050 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:36.050 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.137976+0000", 2020-09-08T15:54:36.050 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3336000000000001e-05 2020-09-08T15:54:36.051 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.051 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.051 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:36.051 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.137990+0000", 2020-09-08T15:54:36.051 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4629999999999994e-05 2020-09-08T15:54:36.051 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.052 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.052 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:36.052 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.138074+0000", 2020-09-08T15:54:36.052 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000205892 2020-09-08T15:54:36.052 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.053 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.053 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:36.053 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.138280+0000", 2020-09-08T15:54:36.053 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000243663 2020-09-08T15:54:36.053 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.054 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.054 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:36.054 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.138524+0000", 2020-09-08T15:54:36.055 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0045534010000000003 2020-09-08T15:54:36.055 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.055 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.055 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:36.055 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.143077+0000", 2020-09-08T15:54:36.056 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.117332111 2020-09-08T15:54:36.056 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.056 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.056 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:36.056 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.260409+0000", 2020-09-08T15:54:36.056 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4503000000000001e-05 2020-09-08T15:54:36.057 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.057 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.057 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:36.057 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.260494+0000", 2020-09-08T15:54:36.057 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.5622000000000003e-05 2020-09-08T15:54:36.058 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.058 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.058 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:36.058 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:08.260559+0000", 2020-09-08T15:54:36.058 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.12267346899999999 2020-09-08T15:54:36.059 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.059 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:36.059 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.059 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.059 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.060 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35381.0:255314 4.a 4:5b1fcaf5:::benchmark_data_smithi166_32054_object113172:head [delete] snapc 0=[] ondisk+write+known_if_redirected e382)", 2020-09-08T15:54:36.060 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:50:48.073459+0000", 2020-09-08T15:54:36.060 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 227.86128911899999, 2020-09-08T15:54:36.060 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.163795407, 2020-09-08T15:54:36.060 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:36.060 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:36.061 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:36.061 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35381", 2020-09-08T15:54:36.061 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:54:36.061 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 255314 2020-09-08T15:54:36.061 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.062 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:36.062 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.062 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:36.062 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.073459+0000", 2020-09-08T15:54:36.062 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:36.063 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.063 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.063 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:36.063 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.073459+0000", 2020-09-08T15:54:36.063 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999752 2020-09-08T15:54:36.064 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.064 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.064 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:36.064 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.073435+0000", 2020-09-08T15:54:36.064 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.6647999999999997e-05 2020-09-08T15:54:36.064 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.065 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.065 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:36.065 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.073491+0000", 2020-09-08T15:54:36.065 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2641e-05 2020-09-08T15:54:36.065 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.066 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.066 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:36.066 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.073504+0000", 2020-09-08T15:54:36.066 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.6399999999999997e-06 2020-09-08T15:54:36.066 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.067 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.067 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:36.067 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.073511+0000", 2020-09-08T15:54:36.067 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4011000000000002e-05 2020-09-08T15:54:36.067 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.067 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.068 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:36.068 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.073565+0000", 2020-09-08T15:54:36.068 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013249799999999999 2020-09-08T15:54:36.068 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.068 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.069 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:36.069 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.073698+0000", 2020-09-08T15:54:36.069 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000113942 2020-09-08T15:54:36.069 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.069 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.070 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:36.070 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.073812+0000", 2020-09-08T15:54:36.070 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0046375710000000001 2020-09-08T15:54:36.070 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.070 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.071 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:36.071 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.078450+0000", 2020-09-08T15:54:36.071 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.158603468 2020-09-08T15:54:36.071 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.071 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.071 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:36.072 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.237053+0000", 2020-09-08T15:54:36.072 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010614400000000001 2020-09-08T15:54:36.072 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.072 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.072 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:36.073 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.237159+0000", 2020-09-08T15:54:36.073 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.5774999999999997e-05 2020-09-08T15:54:36.073 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.073 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.073 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:36.074 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.237255+0000", 2020-09-08T15:54:36.074 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.163795407 2020-09-08T15:54:36.074 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.074 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:36.074 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.074 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.075 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.075 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35381.0:255321 4.a 4:5a5f1d71:::benchmark_data_smithi166_32054_object113179:head [delete] snapc 0=[] ondisk+write+known_if_redirected e382)", 2020-09-08T15:54:36.075 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:50:48.082738+0000", 2020-09-08T15:54:36.075 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 227.852010912, 2020-09-08T15:54:36.075 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.154678915, 2020-09-08T15:54:36.076 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:36.076 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:36.076 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:36.076 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35381", 2020-09-08T15:54:36.076 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:54:36.077 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 255321 2020-09-08T15:54:36.077 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.077 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:36.077 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.077 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:36.078 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.082738+0000", 2020-09-08T15:54:36.078 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:36.078 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.078 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.078 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:36.078 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.082738+0000", 2020-09-08T15:54:36.079 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999375 2020-09-08T15:54:36.079 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.079 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.079 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:36.079 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.082675+0000", 2020-09-08T15:54:36.080 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000145625 2020-09-08T15:54:36.080 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.080 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.080 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:36.080 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.082821+0000", 2020-09-08T15:54:36.081 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.0216999999999999e-05 2020-09-08T15:54:36.081 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.081 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.081 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:36.081 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.082851+0000", 2020-09-08T15:54:36.082 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0654e-05 2020-09-08T15:54:36.082 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.082 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.082 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:36.082 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.082862+0000", 2020-09-08T15:54:36.082 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.8295999999999998e-05 2020-09-08T15:54:36.083 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.083 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.083 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:36.083 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.082930+0000", 2020-09-08T15:54:36.083 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00018518399999999999 2020-09-08T15:54:36.084 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.084 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.084 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:36.084 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.083115+0000", 2020-09-08T15:54:36.084 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016329199999999999 2020-09-08T15:54:36.085 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.085 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.085 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:36.085 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.083278+0000", 2020-09-08T15:54:36.085 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0056371290000000003 2020-09-08T15:54:36.085 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.086 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.086 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:36.086 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.088915+0000", 2020-09-08T15:54:36.086 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.14840083100000001 2020-09-08T15:54:36.086 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.087 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.087 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:36.087 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.237316+0000", 2020-09-08T15:54:36.087 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.2864e-05 2020-09-08T15:54:36.087 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.088 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.088 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:36.088 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.237379+0000", 2020-09-08T15:54:36.088 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.7487999999999998e-05 2020-09-08T15:54:36.088 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.088 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.089 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:36.089 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.237417+0000", 2020-09-08T15:54:36.089 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.154678915 2020-09-08T15:54:36.089 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.089 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:36.090 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.090 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.090 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.090 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.35381.0:255331 4.a 4:55e5582d:::benchmark_data_smithi166_32054_object113189:head [delete] snapc 0=[] ondisk+write+known_if_redirected e382)", 2020-09-08T15:54:36.090 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:50:48.110021+0000", 2020-09-08T15:54:36.091 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 227.824727728, 2020-09-08T15:54:36.091 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.134370869, 2020-09-08T15:54:36.091 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:36.091 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:36.091 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:36.092 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.35381", 2020-09-08T15:54:36.092 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:54:36.092 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 255331 2020-09-08T15:54:36.092 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.092 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:36.092 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.093 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:36.093 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.110021+0000", 2020-09-08T15:54:36.093 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:36.093 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.093 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.094 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:36.094 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.110021+0000", 2020-09-08T15:54:36.094 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999366 2020-09-08T15:54:36.094 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.094 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.095 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:36.095 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.109958+0000", 2020-09-08T15:54:36.095 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000140235 2020-09-08T15:54:36.095 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.095 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.095 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:36.096 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.110098+0000", 2020-09-08T15:54:36.096 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.6415e-05 2020-09-08T15:54:36.096 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.096 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.096 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:36.097 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.110124+0000", 2020-09-08T15:54:36.097 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3664e-05 2020-09-08T15:54:36.097 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.097 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.098 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:36.098 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.110138+0000", 2020-09-08T15:54:36.098 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.3978000000000002e-05 2020-09-08T15:54:36.098 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.098 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.099 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:36.099 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.110212+0000", 2020-09-08T15:54:36.099 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014602300000000001 2020-09-08T15:54:36.099 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.099 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.099 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:36.100 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.110358+0000", 2020-09-08T15:54:36.100 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000120368 2020-09-08T15:54:36.100 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.100 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.100 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:54:36.101 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.110478+0000", 2020-09-08T15:54:36.101 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0047162549999999999 2020-09-08T15:54:36.101 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.101 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.101 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:36.102 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.115194+0000", 2020-09-08T15:54:36.102 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.129092125 2020-09-08T15:54:36.102 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.102 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.102 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:36.103 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.244287+0000", 2020-09-08T15:54:36.103 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.9495e-05 2020-09-08T15:54:36.103 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.103 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.103 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:36.103 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.244336+0000", 2020-09-08T15:54:36.104 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.5670000000000001e-05 2020-09-08T15:54:36.104 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.104 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.104 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:36.104 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:50:48.244392+0000", 2020-09-08T15:54:36.105 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.134370869 2020-09-08T15:54:36.105 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.105 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:36.105 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.105 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.106 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.106 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:33283 5.2 5:4be405b6:::benchmark_data_smithi166_37820_object33282:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e407)", 2020-09-08T15:54:36.106 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:52:04.727638+0000", 2020-09-08T15:54:36.106 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 151.20711104599999, 2020-09-08T15:54:36.106 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13968840199999999, 2020-09-08T15:54:36.107 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:36.107 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:36.107 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:36.107 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:36.107 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:36.107 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 33283 2020-09-08T15:54:36.108 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.108 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:36.108 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.108 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:36.108 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.727638+0000", 2020-09-08T15:54:36.109 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:36.109 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.109 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.109 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:36.109 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.727638+0000", 2020-09-08T15:54:36.110 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999399 2020-09-08T15:54:36.110 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.110 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.110 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:36.110 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.727578+0000", 2020-09-08T15:54:36.111 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000153055 2020-09-08T15:54:36.111 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.111 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.111 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:36.111 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.727731+0000", 2020-09-08T15:54:36.111 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2764e-05 2020-09-08T15:54:36.112 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.112 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.112 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:36.112 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.727744+0000", 2020-09-08T15:54:36.112 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.3930000000000002e-06 2020-09-08T15:54:36.113 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.113 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.113 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:36.113 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.727751+0000", 2020-09-08T15:54:36.113 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.8557000000000002e-05 2020-09-08T15:54:36.114 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.114 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.114 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:36.114 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.727809+0000", 2020-09-08T15:54:36.114 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000184103 2020-09-08T15:54:36.114 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.115 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.115 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:36.115 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.727994+0000", 2020-09-08T15:54:36.115 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00022058900000000001 2020-09-08T15:54:36.115 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.116 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.116 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 0", 2020-09-08T15:54:36.116 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.728214+0000", 2020-09-08T15:54:36.116 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0025377059999999998 2020-09-08T15:54:36.116 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.117 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.117 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:36.117 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.730752+0000", 2020-09-08T15:54:36.117 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.136360919 2020-09-08T15:54:36.117 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.118 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.118 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:36.118 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.867113+0000", 2020-09-08T15:54:36.118 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012882400000000001 2020-09-08T15:54:36.118 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.118 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.119 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:36.119 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.867242+0000", 2020-09-08T15:54:36.119 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4345999999999995e-05 2020-09-08T15:54:36.119 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.119 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.120 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:36.120 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.867326+0000", 2020-09-08T15:54:36.120 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13968840199999999 2020-09-08T15:54:36.120 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.120 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:36.121 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.121 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.121 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.121 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:33285 5.2 5:41a79adb:::benchmark_data_smithi166_37820_object33284:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e407)", 2020-09-08T15:54:36.121 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:52:04.727817+0000", 2020-09-08T15:54:36.121 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 151.20693151500001, 2020-09-08T15:54:36.122 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.144333238, 2020-09-08T15:54:36.122 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:36.122 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:36.122 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:36.122 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:36.123 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:36.123 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 33285 2020-09-08T15:54:36.123 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.123 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:36.123 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.124 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:36.124 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.727817+0000", 2020-09-08T15:54:36.124 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:36.124 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.124 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.125 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:36.125 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.727817+0000", 2020-09-08T15:54:36.125 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999771 2020-09-08T15:54:36.125 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.126 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.126 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:36.126 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.727794+0000", 2020-09-08T15:54:36.126 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3117999999999999e-05 2020-09-08T15:54:36.126 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.127 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.127 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:36.127 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.727887+0000", 2020-09-08T15:54:36.127 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.6470000000000004e-06 2020-09-08T15:54:36.127 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.127 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.128 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:36.128 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.727896+0000", 2020-09-08T15:54:36.128 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.1250000000000003e-06 2020-09-08T15:54:36.128 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.128 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.129 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:36.129 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.727900+0000", 2020-09-08T15:54:36.129 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00136994 2020-09-08T15:54:36.129 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.129 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.130 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:36.130 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.729270+0000", 2020-09-08T15:54:36.130 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00028820700000000001 2020-09-08T15:54:36.130 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.130 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.130 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:36.131 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.729558+0000", 2020-09-08T15:54:36.131 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000252155 2020-09-08T15:54:36.132 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.132 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.132 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 0", 2020-09-08T15:54:36.133 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.729810+0000", 2020-09-08T15:54:36.133 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.001145793 2020-09-08T15:54:36.133 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.133 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.133 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:36.134 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.730956+0000", 2020-09-08T15:54:36.134 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.14103624200000001 2020-09-08T15:54:36.134 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.134 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.135 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:36.135 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.871993+0000", 2020-09-08T15:54:36.135 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.8496e-05 2020-09-08T15:54:36.135 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.135 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.135 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:36.136 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.872091+0000", 2020-09-08T15:54:36.136 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9311999999999997e-05 2020-09-08T15:54:36.136 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.136 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.136 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:36.137 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:04.872150+0000", 2020-09-08T15:54:36.137 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.144333238 2020-09-08T15:54:36.137 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.137 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:36.137 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.138 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.138 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.138 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:152518 5.2 5:49762bf2:::benchmark_data_smithi166_37820_object8096:head [delete] snapc 0=[] ondisk+write+known_if_redirected e432)", 2020-09-08T15:54:36.138 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:53:32.462415+0000", 2020-09-08T15:54:36.138 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 63.472333685999999, 2020-09-08T15:54:36.139 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13736862699999999, 2020-09-08T15:54:36.139 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:36.139 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:36.139 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:36.139 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:36.139 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:36.140 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 152518 2020-09-08T15:54:36.140 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.140 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:36.140 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.141 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:36.141 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.462415+0000", 2020-09-08T15:54:36.141 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:36.141 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.142 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.142 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:36.142 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.462415+0000", 2020-09-08T15:54:36.142 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999456 2020-09-08T15:54:36.142 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.143 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.143 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:36.143 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.462361+0000", 2020-09-08T15:54:36.143 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011404 2020-09-08T15:54:36.143 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.144 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.144 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:36.144 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.462475+0000", 2020-09-08T15:54:36.144 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.1614999999999999e-05 2020-09-08T15:54:36.144 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.144 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.145 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:36.145 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.462496+0000", 2020-09-08T15:54:36.145 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.7270000000000007e-06 2020-09-08T15:54:36.145 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.145 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.146 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:36.146 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.462504+0000", 2020-09-08T15:54:36.146 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.8653999999999995e-05 2020-09-08T15:54:36.146 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.146 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.147 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:36.147 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.462593+0000", 2020-09-08T15:54:36.147 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00018763800000000001 2020-09-08T15:54:36.147 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.147 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.147 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:36.148 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.462780+0000", 2020-09-08T15:54:36.148 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016012200000000001 2020-09-08T15:54:36.148 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.148 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.148 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 0", 2020-09-08T15:54:36.149 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.462940+0000", 2020-09-08T15:54:36.149 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0059158709999999996 2020-09-08T15:54:36.149 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.150 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.150 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:36.150 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.468856+0000", 2020-09-08T15:54:36.150 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.130743516 2020-09-08T15:54:36.150 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.150 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.151 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:36.151 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.599600+0000", 2020-09-08T15:54:36.151 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.6200999999999996e-05 2020-09-08T15:54:36.152 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.152 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.152 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:36.152 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.599696+0000", 2020-09-08T15:54:36.152 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.7616999999999998e-05 2020-09-08T15:54:36.153 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.153 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.153 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:36.153 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.599784+0000", 2020-09-08T15:54:36.153 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13736862699999999 2020-09-08T15:54:36.154 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.154 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:36.154 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.154 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.154 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.155 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:152523 5.2 5:486e849c:::benchmark_data_smithi166_37820_object8101:head [delete] snapc 0=[] ondisk+write+known_if_redirected e432)", 2020-09-08T15:54:36.155 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:53:32.470935+0000", 2020-09-08T15:54:36.155 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 63.463813731000002, 2020-09-08T15:54:36.155 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.12935644900000001, 2020-09-08T15:54:36.155 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:36.156 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:36.156 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:36.156 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:36.156 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:36.156 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 152523 2020-09-08T15:54:36.156 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.157 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:36.157 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.157 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:36.157 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.470935+0000", 2020-09-08T15:54:36.157 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:36.158 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.158 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.158 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:36.158 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.470935+0000", 2020-09-08T15:54:36.158 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999523 2020-09-08T15:54:36.159 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.159 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.159 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:36.159 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.470887+0000", 2020-09-08T15:54:36.159 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011238099999999999 2020-09-08T15:54:36.160 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.160 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.160 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:36.160 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.471000+0000", 2020-09-08T15:54:36.160 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.3387000000000002e-05 2020-09-08T15:54:36.160 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.161 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.161 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:36.161 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.471023+0000", 2020-09-08T15:54:36.162 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0242e-05 2020-09-08T15:54:36.162 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.162 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.162 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:36.162 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.471033+0000", 2020-09-08T15:54:36.163 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.6751e-05 2020-09-08T15:54:36.163 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.163 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.163 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:36.163 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.471080+0000", 2020-09-08T15:54:36.164 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013037699999999999 2020-09-08T15:54:36.164 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.164 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.164 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:36.164 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.471210+0000", 2020-09-08T15:54:36.165 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011612600000000001 2020-09-08T15:54:36.165 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.165 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.165 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 0", 2020-09-08T15:54:36.165 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.471327+0000", 2020-09-08T15:54:36.165 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0045841010000000001 2020-09-08T15:54:36.166 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.166 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.166 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:36.166 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.475911+0000", 2020-09-08T15:54:36.166 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.12403791 2020-09-08T15:54:36.167 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.167 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.167 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:36.167 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.599949+0000", 2020-09-08T15:54:36.167 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019667999999999999 2020-09-08T15:54:36.168 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.168 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.168 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:36.168 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.600145+0000", 2020-09-08T15:54:36.168 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014612699999999999 2020-09-08T15:54:36.168 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.169 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.169 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:36.169 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:32.600291+0000", 2020-09-08T15:54:36.169 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.12935644900000001 2020-09-08T15:54:36.169 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.170 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:36.170 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.170 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.170 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.170 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:175149 5.2 5:497a21f7:::benchmark_data_smithi166_37820_object30727:head [delete] snapc 0=[] ondisk+write+known_if_redirected e439)", 2020-09-08T15:54:36.171 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:53:57.506791+0000", 2020-09-08T15:54:36.171 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 38.427957984999999, 2020-09-08T15:54:36.171 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.14407172300000001, 2020-09-08T15:54:36.171 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:36.172 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:36.172 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:36.172 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:36.172 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:36.172 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 175149 2020-09-08T15:54:36.173 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.173 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:36.173 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.173 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:36.173 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.506791+0000", 2020-09-08T15:54:36.174 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:36.174 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.174 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.174 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:36.174 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.506791+0000", 2020-09-08T15:54:36.175 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999958 2020-09-08T15:54:36.175 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.175 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.175 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:36.175 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.506749+0000", 2020-09-08T15:54:36.176 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.187e-05 2020-09-08T15:54:36.176 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.176 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.176 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:36.176 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.506831+0000", 2020-09-08T15:54:36.176 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.5000000000000005e-06 2020-09-08T15:54:36.177 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.177 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.177 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:36.177 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.506840+0000", 2020-09-08T15:54:36.177 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.8949999999999997e-06 2020-09-08T15:54:36.178 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.178 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.178 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:36.178 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.506845+0000", 2020-09-08T15:54:36.178 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.8699000000000002e-05 2020-09-08T15:54:36.179 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.179 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.179 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:36.179 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.506894+0000", 2020-09-08T15:54:36.179 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000101851 2020-09-08T15:54:36.180 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.180 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.180 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:36.180 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.506996+0000", 2020-09-08T15:54:36.180 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.7424999999999997e-05 2020-09-08T15:54:36.180 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.181 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.181 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 0", 2020-09-08T15:54:36.181 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.507083+0000", 2020-09-08T15:54:36.181 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0047443759999999998 2020-09-08T15:54:36.181 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.182 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.182 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:36.182 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.511827+0000", 2020-09-08T15:54:36.183 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13878532800000001 2020-09-08T15:54:36.183 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.183 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.183 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:36.183 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.650613+0000", 2020-09-08T15:54:36.183 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015213 2020-09-08T15:54:36.184 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.184 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.184 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:36.184 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.650765+0000", 2020-09-08T15:54:36.184 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.7458000000000001e-05 2020-09-08T15:54:36.185 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.185 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.185 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:36.185 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.650862+0000", 2020-09-08T15:54:36.185 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.14407172300000001 2020-09-08T15:54:36.186 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.186 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:36.186 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.186 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.186 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.187 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:175163 5.2 5:45e95140:::benchmark_data_smithi166_37820_object30741:head [delete] snapc 0=[] ondisk+write+known_if_redirected e439)", 2020-09-08T15:54:36.187 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:53:57.517082+0000", 2020-09-08T15:54:36.187 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 38.417666562000001, 2020-09-08T15:54:36.187 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.14671690500000001, 2020-09-08T15:54:36.187 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:36.187 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:36.188 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:36.188 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:36.188 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:36.188 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 175163 2020-09-08T15:54:36.188 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.189 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:36.189 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.189 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:36.189 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.517082+0000", 2020-09-08T15:54:36.189 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:36.190 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.190 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.190 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:36.190 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.517082+0000", 2020-09-08T15:54:36.190 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999542 2020-09-08T15:54:36.196 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.196 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.196 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:36.196 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.517036+0000", 2020-09-08T15:54:36.197 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000101275 2020-09-08T15:54:36.197 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.197 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.197 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:36.198 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.517137+0000", 2020-09-08T15:54:36.198 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.1710000000000001e-05 2020-09-08T15:54:36.198 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.198 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.198 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:36.199 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.517159+0000", 2020-09-08T15:54:36.199 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0404000000000001e-05 2020-09-08T15:54:36.199 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.199 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.200 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:36.200 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.517170+0000", 2020-09-08T15:54:36.200 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.2905999999999997e-05 2020-09-08T15:54:36.200 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.201 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.201 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:36.201 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.517232+0000", 2020-09-08T15:54:36.201 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016885399999999999 2020-09-08T15:54:36.202 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.202 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.202 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:36.202 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.517401+0000", 2020-09-08T15:54:36.203 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000186145 2020-09-08T15:54:36.203 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.203 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.203 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 0", 2020-09-08T15:54:36.203 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.517587+0000", 2020-09-08T15:54:36.204 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0056378330000000001 2020-09-08T15:54:36.204 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.204 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.204 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:36.205 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.523225+0000", 2020-09-08T15:54:36.205 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.14047873499999999 2020-09-08T15:54:36.205 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.205 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.205 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:36.206 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.663704+0000", 2020-09-08T15:54:36.206 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4574000000000003e-05 2020-09-08T15:54:36.206 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.206 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.206 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:36.207 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.663759+0000", 2020-09-08T15:54:36.207 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.0368999999999999e-05 2020-09-08T15:54:36.207 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.207 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.207 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:36.208 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:57.663799+0000", 2020-09-08T15:54:36.208 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.14671690500000001 2020-09-08T15:54:36.208 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.208 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:36.208 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.209 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.209 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.209 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:197345 5.2 5:4115f134:::benchmark_data_smithi166_37820_object52923:head [delete] snapc 0=[] ondisk+write+known_if_redirected e444)", 2020-09-08T15:54:36.210 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:54:19.954851+0000", 2020-09-08T15:54:36.210 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 15.979897251000001, 2020-09-08T15:54:36.210 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13993740900000001, 2020-09-08T15:54:36.210 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:36.211 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:36.211 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:36.211 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:36.211 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:36.211 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 197345 2020-09-08T15:54:36.212 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.212 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:36.212 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.212 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:36.212 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.954851+0000", 2020-09-08T15:54:36.213 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:36.213 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.213 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.213 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:36.213 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.954851+0000", 2020-09-08T15:54:36.213 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999958 2020-09-08T15:54:36.214 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.214 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.214 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:36.214 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.954810+0000", 2020-09-08T15:54:36.214 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.7002000000000004e-05 2020-09-08T15:54:36.215 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.215 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.215 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:36.215 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.954907+0000", 2020-09-08T15:54:36.215 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.7982000000000002e-05 2020-09-08T15:54:36.216 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.216 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.216 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:36.216 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.954925+0000", 2020-09-08T15:54:36.216 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.8640000000000008e-06 2020-09-08T15:54:36.217 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.217 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.217 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:36.217 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.954934+0000", 2020-09-08T15:54:36.217 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.4951999999999997e-05 2020-09-08T15:54:36.218 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.218 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.218 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:36.218 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.954999+0000", 2020-09-08T15:54:36.219 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010153300000000001 2020-09-08T15:54:36.219 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.219 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.219 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:36.219 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.955101+0000", 2020-09-08T15:54:36.220 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.5025000000000006e-05 2020-09-08T15:54:36.220 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.220 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.220 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 0", 2020-09-08T15:54:36.220 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.955196+0000", 2020-09-08T15:54:36.221 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.016659825 2020-09-08T15:54:36.221 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.221 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.221 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:36.221 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.971856+0000", 2020-09-08T15:54:36.222 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.122600004 2020-09-08T15:54:36.222 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.222 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.222 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:36.222 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:20.094456+0000", 2020-09-08T15:54:36.222 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000182313 2020-09-08T15:54:36.223 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.223 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.223 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:36.223 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:20.094638+0000", 2020-09-08T15:54:36.223 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000150727 2020-09-08T15:54:36.224 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.224 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.224 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:36.224 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:20.094789+0000", 2020-09-08T15:54:36.224 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13993740900000001 2020-09-08T15:54:36.225 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.225 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:36.225 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.225 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.225 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.226 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:197349 5.2 5:444cc2dc:::benchmark_data_smithi166_37820_object52927:head [delete] snapc 0=[] ondisk+write+known_if_redirected e444)", 2020-09-08T15:54:36.226 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:54:19.957732+0000", 2020-09-08T15:54:36.226 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 15.977016167, 2020-09-08T15:54:36.227 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13751001700000001, 2020-09-08T15:54:36.227 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:36.227 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:36.227 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:36.227 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:36.228 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:36.228 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 197349 2020-09-08T15:54:36.228 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.228 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:36.228 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.229 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:36.229 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.957732+0000", 2020-09-08T15:54:36.229 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:36.229 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.229 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.230 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:36.230 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.957732+0000", 2020-09-08T15:54:36.230 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999766 2020-09-08T15:54:36.230 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.230 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.231 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:36.231 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.957709+0000", 2020-09-08T15:54:36.231 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2518000000000001e-05 2020-09-08T15:54:36.231 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.231 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.231 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:36.232 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.957762+0000", 2020-09-08T15:54:36.232 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.6949999999999995e-06 2020-09-08T15:54:36.232 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.232 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.232 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:36.233 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.957771+0000", 2020-09-08T15:54:36.233 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.3000000000000004e-06 2020-09-08T15:54:36.233 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.233 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.233 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:36.234 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.957779+0000", 2020-09-08T15:54:36.234 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0034518729999999998 2020-09-08T15:54:36.234 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.234 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.235 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:36.235 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.961230+0000", 2020-09-08T15:54:36.235 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016051800000000001 2020-09-08T15:54:36.235 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.236 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.236 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:36.236 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.961391+0000", 2020-09-08T15:54:36.236 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019024299999999999 2020-09-08T15:54:36.236 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.237 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.237 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 0", 2020-09-08T15:54:36.237 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.961581+0000", 2020-09-08T15:54:36.237 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.010284875000000001 2020-09-08T15:54:36.237 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.238 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.238 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:36.238 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.971866+0000", 2020-09-08T15:54:36.238 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.123091667 2020-09-08T15:54:36.238 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.239 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.239 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:36.239 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:20.094958+0000", 2020-09-08T15:54:36.239 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019512899999999999 2020-09-08T15:54:36.239 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.240 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.240 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:36.240 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:20.095153+0000", 2020-09-08T15:54:36.240 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.9616000000000004e-05 2020-09-08T15:54:36.240 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.241 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.241 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:36.241 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:20.095242+0000", 2020-09-08T15:54:36.241 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13751001700000001 2020-09-08T15:54:36.241 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.242 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:36.242 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.242 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.242 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.242 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:197351 5.2 5:4c483614:::benchmark_data_smithi166_37820_object52929:head [delete] snapc 0=[] ondisk+write+known_if_redirected e444)", 2020-09-08T15:54:36.243 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:54:19.958696+0000", 2020-09-08T15:54:36.243 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 15.976052395, 2020-09-08T15:54:36.243 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13700955200000001, 2020-09-08T15:54:36.243 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:36.244 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:54:36.244 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:36.244 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:36.244 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:36.245 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 197351 2020-09-08T15:54:36.245 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.245 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:36.245 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.245 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:36.246 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.958696+0000", 2020-09-08T15:54:36.246 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:36.246 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.246 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.246 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:36.247 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.958696+0000", 2020-09-08T15:54:36.247 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999939 2020-09-08T15:54:36.247 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.247 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.247 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:36.247 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.958635+0000", 2020-09-08T15:54:36.248 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014008799999999999 2020-09-08T15:54:36.248 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.248 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.248 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:36.248 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.958775+0000", 2020-09-08T15:54:36.249 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.5627999999999999e-05 2020-09-08T15:54:36.249 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.249 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.249 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:36.249 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.958801+0000", 2020-09-08T15:54:36.250 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2041e-05 2020-09-08T15:54:36.250 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.250 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.250 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:36.250 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.958813+0000", 2020-09-08T15:54:36.251 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0074529510000000002 2020-09-08T15:54:36.251 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.251 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.251 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:36.251 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.966266+0000", 2020-09-08T15:54:36.252 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012519300000000001 2020-09-08T15:54:36.252 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.252 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.252 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:36.253 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.966391+0000", 2020-09-08T15:54:36.253 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013751900000000001 2020-09-08T15:54:36.253 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.253 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.254 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 0", 2020-09-08T15:54:36.254 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.966528+0000", 2020-09-08T15:54:36.254 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0055274959999999998 2020-09-08T15:54:36.254 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.254 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.255 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:36.255 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:19.972056+0000", 2020-09-08T15:54:36.255 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.12330885 2020-09-08T15:54:36.255 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.255 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.256 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:36.256 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:20.095365+0000", 2020-09-08T15:54:36.256 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017044 2020-09-08T15:54:36.256 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.256 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.257 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:54:36.257 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:20.095535+0000", 2020-09-08T15:54:36.257 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000170567 2020-09-08T15:54:36.257 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:36.258 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:36.258 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:54:36.259 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:20.095706+0000", 2020-09-08T15:54:36.259 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13700955200000001 2020-09-08T15:54:36.259 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.260 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:36.260 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.260 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:36.260 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:36.260 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:54:36.262 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:35 smithi166 bash[9446]: audit 2020-09-08T15:54:35.115796+0000 mon.a (mon.0) 1630 : audit [INF] from='client.? v1:172.21.15.104:0/2625060576' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:54:36.262 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:54:35 smithi166 bash[18960]: debug 2020-09-08T15:54:35.827+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:36.263 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:54:36.263 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:35 smithi104 bash[9203]: audit 2020-09-08T15:54:35.115796+0000 mon.a (mon.0) 1630 : audit [INF] from='client.? v1:172.21.15.104:0/2625060576' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:54:36.268 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:35 smithi055 bash[9296]: audit 2020-09-08T15:54:35.115796+0000 mon.a (mon.0) 1630 : audit [INF] from='client.? v1:172.21.15.104:0/2625060576' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:54:36.844 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:36.946 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:36.968 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2020-09-08T15:54:37.069 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:37.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:36 smithi055 bash[9296]: cluster 2020-09-08T15:54:35.861168+0000 mgr.x (mgr.24118) 818 : cluster [DBG] pgmap v1183: 17 pgs: 17 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 730 op/s 2020-09-08T15:54:37.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:36 smithi055 bash[9296]: cluster 2020-09-08T15:54:35.921378+0000 mon.a (mon.0) 1631 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:54:37.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:36 smithi055 bash[9296]: audit 2020-09-08T15:54:35.939984+0000 mon.a (mon.0) 1632 : audit [INF] from='client.? v1:172.21.15.104:0/2625060576' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:54:37.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:36 smithi055 bash[9296]: cluster 2020-09-08T15:54:35.940062+0000 mon.a (mon.0) 1633 : cluster [DBG] osdmap e450: 12 total, 12 up, 12 in 2020-09-08T15:54:37.158 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:54:36 smithi055 bash[19003]: debug 2020-09-08T15:54:36.838+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:37.158 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:54:37 smithi055 bash[19003]: debug 2020-09-08T15:54:37.066+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:37.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:36 smithi104 bash[9203]: cluster 2020-09-08T15:54:35.861168+0000 mgr.x (mgr.24118) 818 : cluster [DBG] pgmap v1183: 17 pgs: 17 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 730 op/s 2020-09-08T15:54:37.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:36 smithi104 bash[9203]: cluster 2020-09-08T15:54:35.921378+0000 mon.a (mon.0) 1631 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:54:37.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:36 smithi104 bash[9203]: audit 2020-09-08T15:54:35.939984+0000 mon.a (mon.0) 1632 : audit [INF] from='client.? v1:172.21.15.104:0/2625060576' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:54:37.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:36 smithi104 bash[9203]: cluster 2020-09-08T15:54:35.940062+0000 mon.a (mon.0) 1633 : cluster [DBG] osdmap e450: 12 total, 12 up, 12 in 2020-09-08T15:54:37.179 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:54:37.191 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:37.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:36 smithi166 bash[9446]: cluster 2020-09-08T15:54:35.861168+0000 mgr.x (mgr.24118) 818 : cluster [DBG] pgmap v1183: 17 pgs: 17 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 730 op/s 2020-09-08T15:54:37.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:36 smithi166 bash[9446]: cluster 2020-09-08T15:54:35.921378+0000 mon.a (mon.0) 1631 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:54:37.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:36 smithi166 bash[9446]: audit 2020-09-08T15:54:35.939984+0000 mon.a (mon.0) 1632 : audit [INF] from='client.? v1:172.21.15.104:0/2625060576' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:54:37.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:36 smithi166 bash[9446]: cluster 2020-09-08T15:54:35.940062+0000 mon.a (mon.0) 1633 : cluster [DBG] osdmap e450: 12 total, 12 up, 12 in 2020-09-08T15:54:37.415 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:37.519 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:37.527 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:54:37 smithi104 bash[15273]: debug 2020-09-08T15:54:37.420+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:37.646 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:37.750 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:54:37.881 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:54:37.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:54:37 smithi104 bash[15273]: debug 2020-09-08T15:54:37.644+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:37.985 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:54:38.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:54:37 smithi166 bash[18960]: debug 2020-09-08T15:54:37.883+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:38.128 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:54:38.232 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:38.360 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:38.464 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:54:38.617 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:54:38 smithi166 bash[18960]: debug 2020-09-08T15:54:38.127+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:38.617 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:54:38.621 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:54:38 smithi104 bash[17588]: debug 2020-09-08T15:54:38.620+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:38.621 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:54:38 smithi104 bash[15273]: debug 2020-09-08T15:54:38.364+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:38.722 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:38.849 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:38.957 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:39.029 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:38 smithi055 bash[9296]: cluster 2020-09-08T15:54:37.861672+0000 mgr.x (mgr.24118) 819 : cluster [DBG] pgmap v1185: 17 pgs: 17 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T15:54:39.029 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:54:38 smithi055 bash[19003]: debug 2020-09-08T15:54:38.850+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:39.094 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:39.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:38 smithi104 bash[9203]: cluster 2020-09-08T15:54:37.861672+0000 mgr.x (mgr.24118) 819 : cluster [DBG] pgmap v1185: 17 pgs: 17 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T15:54:39.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:54:39 smithi104 bash[19922]: debug 2020-09-08T15:54:39.096+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:39.198 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:54:39.206 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:38 smithi166 bash[9446]: cluster 2020-09-08T15:54:37.861672+0000 mgr.x (mgr.24118) 819 : cluster [DBG] pgmap v1185: 17 pgs: 17 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T15:54:39.332 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:54:39.349 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:54:39.436 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:54:39.444 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:54:39.550 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:54:39.566 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:54:39.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:54:39 smithi166 bash[11928]: debug 2020-09-08T15:54:39.330+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:39.655 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:54:39.672 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:39.675 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:54:39 smithi104 bash[17588]: debug 2020-09-08T15:54:39.564+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:40.594 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:40.697 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:40.862 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:54:40 smithi104 bash[22305]: debug 2020-09-08T15:54:40.864+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:40.863 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:40.863 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:40 smithi104 bash[9203]: cluster 2020-09-08T15:54:39.862080+0000 mgr.x (mgr.24118) 820 : cluster [DBG] pgmap v1186: 17 pgs: 17 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.18k op/s 2020-09-08T15:54:40.864 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:54:40 smithi104 bash[19922]: debug 2020-09-08T15:54:40.592+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:40.967 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:54:41.073 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:40 smithi055 bash[9296]: cluster 2020-09-08T15:54:39.862080+0000 mgr.x (mgr.24118) 820 : cluster [DBG] pgmap v1186: 17 pgs: 17 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.18k op/s 2020-09-08T15:54:41.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:40 smithi166 bash[9446]: cluster 2020-09-08T15:54:39.862080+0000 mgr.x (mgr.24118) 820 : cluster [DBG] pgmap v1186: 17 pgs: 17 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.18k op/s 2020-09-08T15:54:41.336 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:54:41.361 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:54:41.361 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T15:54:41.362 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.362 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:218654 5.9 5:998954d5:::benchmark_data_smithi166_37820_object74232:head [delete] snapc 0=[] ondisk+write+known_if_redirected e450)", 2020-09-08T15:54:41.362 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:54:41.351173+0000", 2020-09-08T15:54:41.362 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.015695002, 2020-09-08T15:54:41.363 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.015730759, 2020-09-08T15:54:41.363 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:41.363 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:54:41.363 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:41.363 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:41.364 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:41.364 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 218654 2020-09-08T15:54:41.364 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.364 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:41.364 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.365 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:41.365 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.351173+0000", 2020-09-08T15:54:41.365 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:41.365 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.365 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.366 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:41.366 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.351173+0000", 2020-09-08T15:54:41.366 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999959 2020-09-08T15:54:41.366 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.366 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.367 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:41.367 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.351132+0000", 2020-09-08T15:54:41.367 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.9331000000000002e-05 2020-09-08T15:54:41.367 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.367 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.368 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:41.368 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.351222+0000", 2020-09-08T15:54:41.368 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5480000000000001e-05 2020-09-08T15:54:41.368 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.369 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.369 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:41.369 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.351237+0000", 2020-09-08T15:54:41.369 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.0650000000000001e-06 2020-09-08T15:54:41.369 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.370 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.370 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:41.370 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.351244+0000", 2020-09-08T15:54:41.370 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.2834000000000001e-05 2020-09-08T15:54:41.370 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.371 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.371 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:41.371 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.351317+0000", 2020-09-08T15:54:41.371 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019289200000000001 2020-09-08T15:54:41.371 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.372 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.372 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:41.372 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.351510+0000", 2020-09-08T15:54:41.372 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017468900000000001 2020-09-08T15:54:41.372 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.373 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.373 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 6", 2020-09-08T15:54:41.373 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.351685+0000", 2020-09-08T15:54:41.373 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.013380484999999999 2020-09-08T15:54:41.373 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.374 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.374 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:41.374 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.365065+0000", 2020-09-08T15:54:41.374 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.001806644 2020-09-08T15:54:41.374 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.375 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.375 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:54:41.375 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.366872+0000", 2020-09-08T15:54:41.375 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.015698502999999999 2020-09-08T15:54:41.375 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:41.376 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:41.376 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:41.376 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.376 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.376 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.15771.0:218655 5.9 5:96fe6ec0:::benchmark_data_smithi166_37820_object74233:head [delete] snapc 0=[] ondisk+write+known_if_redirected e450)", 2020-09-08T15:54:41.377 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:54:41.351786+0000", 2020-09-08T15:54:41.377 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.015082465, 2020-09-08T15:54:41.377 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.015227894, 2020-09-08T15:54:41.377 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:54:41.378 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:54:41.378 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:54:41.378 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.15771", 2020-09-08T15:54:41.378 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:54:41.378 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 218655 2020-09-08T15:54:41.379 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.379 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:54:41.379 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.379 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:54:41.379 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.351786+0000", 2020-09-08T15:54:41.380 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:54:41.380 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.380 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.380 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:54:41.380 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.351786+0000", 2020-09-08T15:54:41.381 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999785 2020-09-08T15:54:41.381 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.381 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.381 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:54:41.381 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.351764+0000", 2020-09-08T15:54:41.382 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9673999999999998e-05 2020-09-08T15:54:41.382 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.382 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.382 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:54:41.382 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.351824+0000", 2020-09-08T15:54:41.383 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6124e-05 2020-09-08T15:54:41.383 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.383 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.383 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:54:41.383 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.351840+0000", 2020-09-08T15:54:41.384 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.161e-06 2020-09-08T15:54:41.384 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.384 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.384 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:54:41.384 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.351846+0000", 2020-09-08T15:54:41.385 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0057351360000000001 2020-09-08T15:54:41.385 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.385 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.385 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:54:41.385 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.357581+0000", 2020-09-08T15:54:41.386 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00021499900000000001 2020-09-08T15:54:41.386 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.386 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.386 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:54:41.386 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.357796+0000", 2020-09-08T15:54:41.387 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00022326600000000001 2020-09-08T15:54:41.387 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.387 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.387 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 6", 2020-09-08T15:54:41.387 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.358020+0000", 2020-09-08T15:54:41.388 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.007236475 2020-09-08T15:54:41.388 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:54:41.388 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:54:41.388 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:54:41.388 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:54:41.365256+0000", 2020-09-08T15:54:41.389 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.013470204 2020-09-08T15:54:41.389 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:41.389 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:54:41.389 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:41.389 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:54:41.390 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2020-09-08T15:54:41.390 INFO:teuthology.orchestra.run.smithi055.stdout: "num_ops": 2 2020-09-08T15:54:41.390 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:54:41.439 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:54:41.447 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:54:41 smithi166 bash[14254]: debug 2020-09-08T15:54:41.334+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:41.554 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:54:41.659 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:54:41.782 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:54:41 smithi166 bash[18960]: debug 2020-09-08T15:54:41.554+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:41.783 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:54:41 smithi166 bash[18960]: debug 2020-09-08T15:54:41.782+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:41.783 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:54:41.887 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:54:42.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:41 smithi166 bash[9446]: cluster 2020-09-08T15:54:40.896179+0000 osd.1 (osd.1) 88 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:42.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:41 smithi166 bash[9446]: cluster 2020-09-08T15:54:40.896259+0000 osd.1 (osd.1) 89 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:42.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:41 smithi055 bash[9296]: cluster 2020-09-08T15:54:40.896179+0000 osd.1 (osd.1) 88 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:42.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:41 smithi055 bash[9296]: cluster 2020-09-08T15:54:40.896259+0000 osd.1 (osd.1) 89 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:42.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:41 smithi104 bash[9203]: cluster 2020-09-08T15:54:40.896179+0000 osd.1 (osd.1) 88 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:42.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:41 smithi104 bash[9203]: cluster 2020-09-08T15:54:40.896259+0000 osd.1 (osd.1) 89 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:42.278 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:54:42.381 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:42.422 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2020-09-08T15:54:42.528 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:42.528 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:54:42 smithi104 bash[22305]: debug 2020-09-08T15:54:42.532+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:42.589 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:54:42.632 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:42.660 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:54:42 smithi055 bash[14301]: debug 2020-09-08T15:54:42.282+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:42.791 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:54:42 smithi104 bash[19922]: debug 2020-09-08T15:54:42.796+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:42.792 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:42.896 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:54:43.039 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:54:43.117 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:54:43 smithi166 bash[11928]: debug 2020-09-08T15:54:43.038+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:43.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:42 smithi166 bash[9446]: cluster 2020-09-08T15:54:41.830452+0000 mon.a (mon.0) 1634 : cluster [DBG] osdmap e451: 12 total, 12 up, 12 in 2020-09-08T15:54:43.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:42 smithi166 bash[9446]: cluster 2020-09-08T15:54:41.862545+0000 mgr.x (mgr.24118) 821 : cluster [DBG] pgmap v1188: 17 pgs: 17 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 755 op/s 2020-09-08T15:54:43.142 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:43.150 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:42 smithi104 bash[9203]: cluster 2020-09-08T15:54:41.830452+0000 mon.a (mon.0) 1634 : cluster [DBG] osdmap e451: 12 total, 12 up, 12 in 2020-09-08T15:54:43.150 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:42 smithi104 bash[9203]: cluster 2020-09-08T15:54:41.862545+0000 mgr.x (mgr.24118) 821 : cluster [DBG] pgmap v1188: 17 pgs: 17 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 755 op/s 2020-09-08T15:54:43.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:42 smithi055 bash[9296]: cluster 2020-09-08T15:54:41.830452+0000 mon.a (mon.0) 1634 : cluster [DBG] osdmap e451: 12 total, 12 up, 12 in 2020-09-08T15:54:43.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:42 smithi055 bash[9296]: cluster 2020-09-08T15:54:41.862545+0000 mgr.x (mgr.24118) 821 : cluster [DBG] pgmap v1188: 17 pgs: 17 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 755 op/s 2020-09-08T15:54:44.123 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:44.227 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:44.340 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:44.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:54:44 smithi104 bash[15273]: debug 2020-09-08T15:54:44.124+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:44.444 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:54:44.483 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:54:44 smithi055 bash[16646]: debug 2020-09-08T15:54:44.346+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:45.166 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:54:45.168 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:44 smithi104 bash[9203]: cluster 2020-09-08T15:54:43.863051+0000 mgr.x (mgr.24118) 822 : cluster [DBG] pgmap v1189: 17 pgs: 17 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.23k op/s 2020-09-08T15:54:45.169 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:54:45 smithi104 bash[17588]: debug 2020-09-08T15:54:45.172+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:45.270 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:45.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:44 smithi166 bash[9446]: cluster 2020-09-08T15:54:43.863051+0000 mgr.x (mgr.24118) 822 : cluster [DBG] pgmap v1189: 17 pgs: 17 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.23k op/s 2020-09-08T15:54:45.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:44 smithi055 bash[9296]: cluster 2020-09-08T15:54:43.863051+0000 mgr.x (mgr.24118) 822 : cluster [DBG] pgmap v1189: 17 pgs: 17 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.23k op/s 2020-09-08T15:54:45.971 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:46.075 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:54:46.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:45 smithi104 bash[9203]: cluster 2020-09-08T15:54:44.964170+0000 osd.6 (osd.6) 73 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:46.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:45 smithi104 bash[9203]: cluster 2020-09-08T15:54:44.964258+0000 osd.6 (osd.6) 74 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:46.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:54:45 smithi104 bash[19922]: debug 2020-09-08T15:54:45.976+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:46.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:45 smithi166 bash[9446]: cluster 2020-09-08T15:54:44.964170+0000 osd.6 (osd.6) 73 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:46.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:45 smithi166 bash[9446]: cluster 2020-09-08T15:54:44.964258+0000 osd.6 (osd.6) 74 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:46.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:45 smithi055 bash[9296]: cluster 2020-09-08T15:54:44.964170+0000 osd.6 (osd.6) 73 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:46.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:45 smithi055 bash[9296]: cluster 2020-09-08T15:54:44.964258+0000 osd.6 (osd.6) 74 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:46.974 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:54:47.010 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:54:47.067 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:54:47.098 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:54:47.099 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T15:54:47.099 INFO:teuthology.orchestra.run.smithi055.stdout: "complaint_time": 30, 2020-09-08T15:54:47.099 INFO:teuthology.orchestra.run.smithi055.stdout: "num_blocked_ops": 0 2020-09-08T15:54:47.099 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:54:47.110 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:54:47.120 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:54:47.142 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:54:47.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:46 smithi055 bash[9296]: cluster 2020-09-08T15:54:45.863616+0000 mgr.x (mgr.24118) 823 : cluster [DBG] pgmap v1190: 17 pgs: 17 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.03k op/s 2020-09-08T15:54:47.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:46 smithi055 bash[9296]: cluster 2020-09-08T15:54:45.933862+0000 mon.a (mon.0) 1635 : cluster [DBG] osdmap e452: 12 total, 12 up, 12 in 2020-09-08T15:54:47.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:46 smithi055 bash[9296]: audit 2020-09-08T15:54:46.916873+0000 mon.a (mon.0) 1636 : audit [DBG] from='client.? v1:172.21.15.104:0/877871142' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:54:47.190 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:54:47.217 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:54:47.225 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:47.258 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:54:47.294 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:54:47.331 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:54:47.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:54:47 smithi166 bash[14254]: debug 2020-09-08T15:54:47.118+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:47.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:46 smithi166 bash[9446]: cluster 2020-09-08T15:54:45.863616+0000 mgr.x (mgr.24118) 823 : cluster [DBG] pgmap v1190: 17 pgs: 17 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.03k op/s 2020-09-08T15:54:47.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:46 smithi166 bash[9446]: cluster 2020-09-08T15:54:45.933862+0000 mon.a (mon.0) 1635 : cluster [DBG] osdmap e452: 12 total, 12 up, 12 in 2020-09-08T15:54:47.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:46 smithi166 bash[9446]: audit 2020-09-08T15:54:46.916873+0000 mon.a (mon.0) 1636 : audit [DBG] from='client.? v1:172.21.15.104:0/877871142' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:54:47.369 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:54:47.392 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:54:47.406 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:54:47.419 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:54:47.419 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:54:47.419 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:54:47.419 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:54:47.420 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:54:47.420 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:54:47.420 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:54:47.420 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:54:47.420 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:54:47.421 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:54:47.421 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:54:47.421 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:54:47.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:46 smithi104 bash[9203]: cluster 2020-09-08T15:54:45.863616+0000 mgr.x (mgr.24118) 823 : cluster [DBG] pgmap v1190: 17 pgs: 17 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.03k op/s 2020-09-08T15:54:47.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:46 smithi104 bash[9203]: cluster 2020-09-08T15:54:45.933862+0000 mon.a (mon.0) 1635 : cluster [DBG] osdmap e452: 12 total, 12 up, 12 in 2020-09-08T15:54:47.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:46 smithi104 bash[9203]: audit 2020-09-08T15:54:46.916873+0000 mon.a (mon.0) 1636 : audit [DBG] from='client.? v1:172.21.15.104:0/877871142' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:54:47.496 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:54:47.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:54:47 smithi055 bash[19003]: debug 2020-09-08T15:54:47.390+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:47.950 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:54:48.132 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2020-09-08T15:54:48.296 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:54:48.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:47 smithi166 bash[9446]: audit 2020-09-08T15:54:47.266719+0000 mon.a (mon.0) 1637 : audit [INF] from='client.? v1:172.21.15.104:0/1983987903' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:54:48.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:47 smithi055 bash[9296]: audit 2020-09-08T15:54:47.266719+0000 mon.a (mon.0) 1637 : audit [INF] from='client.? v1:172.21.15.104:0/1983987903' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:54:48.431 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:47 smithi104 bash[9203]: audit 2020-09-08T15:54:47.266719+0000 mon.a (mon.0) 1637 : audit [INF] from='client.? v1:172.21.15.104:0/1983987903' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:54:49.339 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:48 smithi104 bash[9203]: cluster 2020-09-08T15:54:47.864213+0000 mgr.x (mgr.24118) 824 : cluster [DBG] pgmap v1192: 17 pgs: 17 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T15:54:49.339 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:48 smithi104 bash[9203]: cluster 2020-09-08T15:54:47.939561+0000 mon.a (mon.0) 1638 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:54:49.339 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:48 smithi104 bash[9203]: audit 2020-09-08T15:54:47.950424+0000 mon.a (mon.0) 1639 : audit [INF] from='client.? v1:172.21.15.104:0/1983987903' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:54:49.339 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:48 smithi104 bash[9203]: cluster 2020-09-08T15:54:47.950506+0000 mon.a (mon.0) 1640 : cluster [DBG] osdmap e453: 12 total, 12 up, 12 in 2020-09-08T15:54:49.340 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:48 smithi104 bash[9203]: cluster 2020-09-08T15:54:47.988626+0000 osd.3 (osd.3) 65 : cluster [DBG] 5.1 scrub starts 2020-09-08T15:54:49.340 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:48 smithi104 bash[9203]: cluster 2020-09-08T15:54:48.006121+0000 osd.11 (osd.11) 76 : cluster [DBG] 5.d scrub starts 2020-09-08T15:54:49.340 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:48 smithi104 bash[9203]: cluster 2020-09-08T15:54:48.308682+0000 osd.9 (osd.9) 67 : cluster [DBG] 5.b scrub starts 2020-09-08T15:54:49.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:48 smithi166 bash[9446]: cluster 2020-09-08T15:54:47.864213+0000 mgr.x (mgr.24118) 824 : cluster [DBG] pgmap v1192: 17 pgs: 17 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T15:54:49.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:48 smithi166 bash[9446]: cluster 2020-09-08T15:54:47.939561+0000 mon.a (mon.0) 1638 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:54:49.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:48 smithi166 bash[9446]: audit 2020-09-08T15:54:47.950424+0000 mon.a (mon.0) 1639 : audit [INF] from='client.? v1:172.21.15.104:0/1983987903' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:54:49.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:48 smithi166 bash[9446]: cluster 2020-09-08T15:54:47.950506+0000 mon.a (mon.0) 1640 : cluster [DBG] osdmap e453: 12 total, 12 up, 12 in 2020-09-08T15:54:49.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:48 smithi166 bash[9446]: cluster 2020-09-08T15:54:47.988626+0000 osd.3 (osd.3) 65 : cluster [DBG] 5.1 scrub starts 2020-09-08T15:54:49.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:48 smithi166 bash[9446]: cluster 2020-09-08T15:54:48.006121+0000 osd.11 (osd.11) 76 : cluster [DBG] 5.d scrub starts 2020-09-08T15:54:49.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:48 smithi166 bash[9446]: cluster 2020-09-08T15:54:48.308682+0000 osd.9 (osd.9) 67 : cluster [DBG] 5.b scrub starts 2020-09-08T15:54:49.422 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:48 smithi055 bash[9296]: cluster 2020-09-08T15:54:47.864213+0000 mgr.x (mgr.24118) 824 : cluster [DBG] pgmap v1192: 17 pgs: 17 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T15:54:49.422 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:48 smithi055 bash[9296]: cluster 2020-09-08T15:54:47.939561+0000 mon.a (mon.0) 1638 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:54:49.422 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:48 smithi055 bash[9296]: audit 2020-09-08T15:54:47.950424+0000 mon.a (mon.0) 1639 : audit [INF] from='client.? v1:172.21.15.104:0/1983987903' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:54:49.423 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:48 smithi055 bash[9296]: cluster 2020-09-08T15:54:47.950506+0000 mon.a (mon.0) 1640 : cluster [DBG] osdmap e453: 12 total, 12 up, 12 in 2020-09-08T15:54:49.423 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:48 smithi055 bash[9296]: cluster 2020-09-08T15:54:47.988626+0000 osd.3 (osd.3) 65 : cluster [DBG] 5.1 scrub starts 2020-09-08T15:54:49.423 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:48 smithi055 bash[9296]: cluster 2020-09-08T15:54:48.006121+0000 osd.11 (osd.11) 76 : cluster [DBG] 5.d scrub starts 2020-09-08T15:54:49.423 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:48 smithi055 bash[9296]: cluster 2020-09-08T15:54:48.308682+0000 osd.9 (osd.9) 67 : cluster [DBG] 5.b scrub starts 2020-09-08T15:54:49.568 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:54:49.674 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:49.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:54:49 smithi166 bash[11928]: debug 2020-09-08T15:54:49.566+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:49.942 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:50.045 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:50.172 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:50.237 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:49 smithi055 bash[9296]: cluster 2020-09-08T15:54:48.871197+0000 osd.11 (osd.11) 77 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:50.237 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:49 smithi055 bash[9296]: cluster 2020-09-08T15:54:48.871265+0000 osd.11 (osd.11) 78 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:50.237 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:49 smithi055 bash[9296]: cluster 2020-09-08T15:54:48.893220+0000 osd.2 (osd.2) 62 : cluster [DBG] 5.a scrub starts 2020-09-08T15:54:50.238 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:49 smithi055 bash[9296]: cluster 2020-09-08T15:54:48.950767+0000 mon.a (mon.0) 1641 : cluster [DBG] osdmap e454: 12 total, 12 up, 12 in 2020-09-08T15:54:50.238 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:49 smithi055 bash[9296]: cluster 2020-09-08T15:54:49.175593+0000 osd.3 (osd.3) 66 : cluster [DBG] 5.1 scrub ok 2020-09-08T15:54:50.238 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:49 smithi055 bash[9296]: cluster 2020-09-08T15:54:49.299922+0000 osd.11 (osd.11) 79 : cluster [DBG] 5.d scrub ok 2020-09-08T15:54:50.279 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:50.288 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:49 smithi104 bash[9203]: cluster 2020-09-08T15:54:48.871197+0000 osd.11 (osd.11) 77 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:50.289 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:49 smithi104 bash[9203]: cluster 2020-09-08T15:54:48.871265+0000 osd.11 (osd.11) 78 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:50.289 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:49 smithi104 bash[9203]: cluster 2020-09-08T15:54:48.893220+0000 osd.2 (osd.2) 62 : cluster [DBG] 5.a scrub starts 2020-09-08T15:54:50.289 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:49 smithi104 bash[9203]: cluster 2020-09-08T15:54:48.950767+0000 mon.a (mon.0) 1641 : cluster [DBG] osdmap e454: 12 total, 12 up, 12 in 2020-09-08T15:54:50.289 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:49 smithi104 bash[9203]: cluster 2020-09-08T15:54:49.175593+0000 osd.3 (osd.3) 66 : cluster [DBG] 5.1 scrub ok 2020-09-08T15:54:50.289 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:49 smithi104 bash[9203]: cluster 2020-09-08T15:54:49.299922+0000 osd.11 (osd.11) 79 : cluster [DBG] 5.d scrub ok 2020-09-08T15:54:50.290 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:54:49 smithi104 bash[15273]: debug 2020-09-08T15:54:49.944+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:50.367 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:54:50 smithi166 bash[16593]: debug 2020-09-08T15:54:50.170+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:50.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:49 smithi166 bash[9446]: cluster 2020-09-08T15:54:48.871197+0000 osd.11 (osd.11) 77 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:50.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:49 smithi166 bash[9446]: cluster 2020-09-08T15:54:48.871265+0000 osd.11 (osd.11) 78 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:50.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:49 smithi166 bash[9446]: cluster 2020-09-08T15:54:48.893220+0000 osd.2 (osd.2) 62 : cluster [DBG] 5.a scrub starts 2020-09-08T15:54:50.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:49 smithi166 bash[9446]: cluster 2020-09-08T15:54:48.950767+0000 mon.a (mon.0) 1641 : cluster [DBG] osdmap e454: 12 total, 12 up, 12 in 2020-09-08T15:54:50.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:49 smithi166 bash[9446]: cluster 2020-09-08T15:54:49.175593+0000 osd.3 (osd.3) 66 : cluster [DBG] 5.1 scrub ok 2020-09-08T15:54:50.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:49 smithi166 bash[9446]: cluster 2020-09-08T15:54:49.299922+0000 osd.11 (osd.11) 79 : cluster [DBG] 5.d scrub ok 2020-09-08T15:54:50.434 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:50.543 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:50.551 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:54:50 smithi104 bash[19922]: debug 2020-09-08T15:54:50.436+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:50.662 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:50.766 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:50.906 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:50.907 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:54:50 smithi104 bash[22305]: debug 2020-09-08T15:54:50.664+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:51.010 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:51.142 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:51.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:54:51 smithi104 bash[22305]: debug 2020-09-08T15:54:51.144+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:51.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:54:50 smithi104 bash[19922]: debug 2020-09-08T15:54:50.912+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:51.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:50 smithi104 bash[9203]: cluster 2020-09-08T15:54:49.820893+0000 osd.9 (osd.9) 68 : cluster [DBG] 5.b scrub ok 2020-09-08T15:54:51.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:50 smithi104 bash[9203]: cluster 2020-09-08T15:54:49.864648+0000 mgr.x (mgr.24118) 825 : cluster [DBG] pgmap v1195: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 900 op/s 2020-09-08T15:54:51.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:50 smithi104 bash[9203]: cluster 2020-09-08T15:54:50.037353+0000 osd.2 (osd.2) 63 : cluster [DBG] 5.a scrub ok 2020-09-08T15:54:51.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:50 smithi104 bash[9203]: cluster 2020-09-08T15:54:50.361956+0000 osd.1 (osd.1) 90 : cluster [DBG] 5.c scrub starts 2020-09-08T15:54:51.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:50 smithi104 bash[9203]: cluster 2020-09-08T15:54:50.660024+0000 osd.10 (osd.10) 53 : cluster [DBG] 5.4 scrub starts 2020-09-08T15:54:51.247 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:51.341 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:54:51.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:50 smithi166 bash[9446]: cluster 2020-09-08T15:54:49.820893+0000 osd.9 (osd.9) 68 : cluster [DBG] 5.b scrub ok 2020-09-08T15:54:51.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:50 smithi166 bash[9446]: cluster 2020-09-08T15:54:49.864648+0000 mgr.x (mgr.24118) 825 : cluster [DBG] pgmap v1195: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 900 op/s 2020-09-08T15:54:51.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:50 smithi166 bash[9446]: cluster 2020-09-08T15:54:50.037353+0000 osd.2 (osd.2) 63 : cluster [DBG] 5.a scrub ok 2020-09-08T15:54:51.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:50 smithi166 bash[9446]: cluster 2020-09-08T15:54:50.361956+0000 osd.1 (osd.1) 90 : cluster [DBG] 5.c scrub starts 2020-09-08T15:54:51.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:50 smithi166 bash[9446]: cluster 2020-09-08T15:54:50.660024+0000 osd.10 (osd.10) 53 : cluster [DBG] 5.4 scrub starts 2020-09-08T15:54:51.373 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:50 smithi055 bash[9296]: cluster 2020-09-08T15:54:49.820893+0000 osd.9 (osd.9) 68 : cluster [DBG] 5.b scrub ok 2020-09-08T15:54:51.374 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:50 smithi055 bash[9296]: cluster 2020-09-08T15:54:49.864648+0000 mgr.x (mgr.24118) 825 : cluster [DBG] pgmap v1195: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 900 op/s 2020-09-08T15:54:51.374 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:50 smithi055 bash[9296]: cluster 2020-09-08T15:54:50.037353+0000 osd.2 (osd.2) 63 : cluster [DBG] 5.a scrub ok 2020-09-08T15:54:51.374 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:50 smithi055 bash[9296]: cluster 2020-09-08T15:54:50.361956+0000 osd.1 (osd.1) 90 : cluster [DBG] 5.c scrub starts 2020-09-08T15:54:51.374 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:50 smithi055 bash[9296]: cluster 2020-09-08T15:54:50.660024+0000 osd.10 (osd.10) 53 : cluster [DBG] 5.4 scrub starts 2020-09-08T15:54:51.522 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:54:51.820 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:51.924 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:52.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:54:51 smithi055 bash[16646]: debug 2020-09-08T15:54:51.822+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:52.314 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:54:52.480 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:54:52.486 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:52.617 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:52.837 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:54:52 smithi055 bash[11956]: debug 2020-09-08T15:54:52.490+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:53.133 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:53.142 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:54:53.143 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:54:53.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:52 smithi055 bash[9296]: cluster 2020-09-08T15:54:51.681810+0000 osd.1 (osd.1) 91 : cluster [DBG] 5.c scrub ok 2020-09-08T15:54:53.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:52 smithi055 bash[9296]: cluster 2020-09-08T15:54:51.737453+0000 osd.10 (osd.10) 54 : cluster [DBG] 5.4 scrub ok 2020-09-08T15:54:53.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:52 smithi055 bash[9296]: cluster 2020-09-08T15:54:51.865106+0000 mgr.x (mgr.24118) 826 : cluster [DBG] pgmap v1196: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.9 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 942 op/s 2020-09-08T15:54:53.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:52 smithi055 bash[9296]: cluster 2020-09-08T15:54:52.035037+0000 osd.11 (osd.11) 80 : cluster [DBG] 5.8 scrub starts 2020-09-08T15:54:53.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:52 smithi055 bash[9296]: cluster 2020-09-08T15:54:52.305053+0000 osd.9 (osd.9) 69 : cluster [DBG] 5.e scrub starts 2020-09-08T15:54:53.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:52 smithi055 bash[9296]: cluster 2020-09-08T15:54:52.597191+0000 osd.10 (osd.10) 55 : cluster [DBG] 5.0 scrub starts 2020-09-08T15:54:53.238 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:53.246 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:52 smithi166 bash[9446]: cluster 2020-09-08T15:54:51.681810+0000 osd.1 (osd.1) 91 : cluster [DBG] 5.c scrub ok 2020-09-08T15:54:53.247 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:52 smithi166 bash[9446]: cluster 2020-09-08T15:54:51.737453+0000 osd.10 (osd.10) 54 : cluster [DBG] 5.4 scrub ok 2020-09-08T15:54:53.247 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:52 smithi166 bash[9446]: cluster 2020-09-08T15:54:51.865106+0000 mgr.x (mgr.24118) 826 : cluster [DBG] pgmap v1196: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.9 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 942 op/s 2020-09-08T15:54:53.247 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:52 smithi166 bash[9446]: cluster 2020-09-08T15:54:52.035037+0000 osd.11 (osd.11) 80 : cluster [DBG] 5.8 scrub starts 2020-09-08T15:54:53.247 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:52 smithi166 bash[9446]: cluster 2020-09-08T15:54:52.305053+0000 osd.9 (osd.9) 69 : cluster [DBG] 5.e scrub starts 2020-09-08T15:54:53.247 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:52 smithi166 bash[9446]: cluster 2020-09-08T15:54:52.597191+0000 osd.10 (osd.10) 55 : cluster [DBG] 5.0 scrub starts 2020-09-08T15:54:53.375 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:53.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:54:53 smithi104 bash[22305]: debug 2020-09-08T15:54:53.136+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:53.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:52 smithi104 bash[9203]: cluster 2020-09-08T15:54:51.681810+0000 osd.1 (osd.1) 91 : cluster [DBG] 5.c scrub ok 2020-09-08T15:54:53.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:52 smithi104 bash[9203]: cluster 2020-09-08T15:54:51.737453+0000 osd.10 (osd.10) 54 : cluster [DBG] 5.4 scrub ok 2020-09-08T15:54:53.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:52 smithi104 bash[9203]: cluster 2020-09-08T15:54:51.865106+0000 mgr.x (mgr.24118) 826 : cluster [DBG] pgmap v1196: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.9 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 942 op/s 2020-09-08T15:54:53.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:52 smithi104 bash[9203]: cluster 2020-09-08T15:54:52.035037+0000 osd.11 (osd.11) 80 : cluster [DBG] 5.8 scrub starts 2020-09-08T15:54:53.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:52 smithi104 bash[9203]: cluster 2020-09-08T15:54:52.305053+0000 osd.9 (osd.9) 69 : cluster [DBG] 5.e scrub starts 2020-09-08T15:54:53.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:52 smithi104 bash[9203]: cluster 2020-09-08T15:54:52.597191+0000 osd.10 (osd.10) 55 : cluster [DBG] 5.0 scrub starts 2020-09-08T15:54:53.478 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:53.617 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:54:53 smithi166 bash[16593]: debug 2020-09-08T15:54:53.374+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:54.353 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:54 smithi055 bash[9296]: cluster 2020-09-08T15:54:52.806448+0000 osd.7 (osd.7) 87 : cluster [DBG] 5.9 scrub starts 2020-09-08T15:54:54.354 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:54 smithi055 bash[9296]: cluster 2020-09-08T15:54:52.991027+0000 osd.9 (osd.9) 70 : cluster [DBG] 5.e scrub ok 2020-09-08T15:54:54.354 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:54 smithi055 bash[9296]: cluster 2020-09-08T15:54:53.241177+0000 osd.11 (osd.11) 81 : cluster [DBG] 5.8 scrub ok 2020-09-08T15:54:54.354 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:54 smithi055 bash[9296]: cluster 2020-09-08T15:54:53.255343+0000 osd.4 (osd.4) 47 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:54.354 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:54 smithi055 bash[9296]: cluster 2020-09-08T15:54:53.255412+0000 osd.4 (osd.4) 48 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:54.355 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:54.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:54 smithi166 bash[9446]: cluster 2020-09-08T15:54:52.806448+0000 osd.7 (osd.7) 87 : cluster [DBG] 5.9 scrub starts 2020-09-08T15:54:54.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:54 smithi166 bash[9446]: cluster 2020-09-08T15:54:52.991027+0000 osd.9 (osd.9) 70 : cluster [DBG] 5.e scrub ok 2020-09-08T15:54:54.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:54 smithi166 bash[9446]: cluster 2020-09-08T15:54:53.241177+0000 osd.11 (osd.11) 81 : cluster [DBG] 5.8 scrub ok 2020-09-08T15:54:54.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:54 smithi166 bash[9446]: cluster 2020-09-08T15:54:53.255343+0000 osd.4 (osd.4) 47 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:54.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:54 smithi166 bash[9446]: cluster 2020-09-08T15:54:53.255412+0000 osd.4 (osd.4) 48 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:54.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:54 smithi104 bash[9203]: cluster 2020-09-08T15:54:52.806448+0000 osd.7 (osd.7) 87 : cluster [DBG] 5.9 scrub starts 2020-09-08T15:54:54.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:54 smithi104 bash[9203]: cluster 2020-09-08T15:54:52.991027+0000 osd.9 (osd.9) 70 : cluster [DBG] 5.e scrub ok 2020-09-08T15:54:54.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:54 smithi104 bash[9203]: cluster 2020-09-08T15:54:53.241177+0000 osd.11 (osd.11) 81 : cluster [DBG] 5.8 scrub ok 2020-09-08T15:54:54.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:54 smithi104 bash[9203]: cluster 2020-09-08T15:54:53.255343+0000 osd.4 (osd.4) 47 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:54:54.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:54 smithi104 bash[9203]: cluster 2020-09-08T15:54:53.255412+0000 osd.4 (osd.4) 48 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:54:54.463 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:54.471 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2020-09-08T15:54:54.647 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:54.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:54:54 smithi055 bash[16646]: debug 2020-09-08T15:54:54.357+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:54.701 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:54:54.749 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:54.756 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:54:54 smithi104 bash[22305]: debug 2020-09-08T15:54:54.640+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:54.955 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:55.007 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:55 smithi104 bash[9203]: cluster 2020-09-08T15:54:53.691763+0000 osd.10 (osd.10) 56 : cluster [DBG] 5.0 scrub ok 2020-09-08T15:54:55.008 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:55 smithi104 bash[9203]: cluster 2020-09-08T15:54:53.865588+0000 mgr.x (mgr.24118) 827 : cluster [DBG] pgmap v1197: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 3.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T15:54:55.008 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:55 smithi104 bash[9203]: cluster 2020-09-08T15:54:53.932541+0000 osd.7 (osd.7) 88 : cluster [DBG] 5.9 scrub ok 2020-09-08T15:54:55.008 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:55 smithi104 bash[9203]: cluster 2020-09-08T15:54:54.009548+0000 mon.a (mon.0) 1642 : cluster [DBG] osdmap e455: 12 total, 12 up, 12 in 2020-09-08T15:54:55.008 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:55 smithi104 bash[9203]: cluster 2020-09-08T15:54:54.098591+0000 osd.0 (osd.0) 61 : cluster [DBG] 5.5 scrub starts 2020-09-08T15:54:55.009 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:54:54 smithi104 bash[15273]: debug 2020-09-08T15:54:54.960+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:55.060 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:55.266 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:54:55.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:55 smithi166 bash[9446]: cluster 2020-09-08T15:54:53.691763+0000 osd.10 (osd.10) 56 : cluster [DBG] 5.0 scrub ok 2020-09-08T15:54:55.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:55 smithi166 bash[9446]: cluster 2020-09-08T15:54:53.865588+0000 mgr.x (mgr.24118) 827 : cluster [DBG] pgmap v1197: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 3.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T15:54:55.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:55 smithi166 bash[9446]: cluster 2020-09-08T15:54:53.932541+0000 osd.7 (osd.7) 88 : cluster [DBG] 5.9 scrub ok 2020-09-08T15:54:55.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:55 smithi166 bash[9446]: cluster 2020-09-08T15:54:54.009548+0000 mon.a (mon.0) 1642 : cluster [DBG] osdmap e455: 12 total, 12 up, 12 in 2020-09-08T15:54:55.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:55 smithi166 bash[9446]: cluster 2020-09-08T15:54:54.098591+0000 osd.0 (osd.0) 61 : cluster [DBG] 5.5 scrub starts 2020-09-08T15:54:55.370 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:55.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:55 smithi055 bash[9296]: cluster 2020-09-08T15:54:53.691763+0000 osd.10 (osd.10) 56 : cluster [DBG] 5.0 scrub ok 2020-09-08T15:54:55.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:55 smithi055 bash[9296]: cluster 2020-09-08T15:54:53.865588+0000 mgr.x (mgr.24118) 827 : cluster [DBG] pgmap v1197: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 3.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T15:54:55.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:55 smithi055 bash[9296]: cluster 2020-09-08T15:54:53.932541+0000 osd.7 (osd.7) 88 : cluster [DBG] 5.9 scrub ok 2020-09-08T15:54:55.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:55 smithi055 bash[9296]: cluster 2020-09-08T15:54:54.009548+0000 mon.a (mon.0) 1642 : cluster [DBG] osdmap e455: 12 total, 12 up, 12 in 2020-09-08T15:54:55.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:55 smithi055 bash[9296]: cluster 2020-09-08T15:54:54.098591+0000 osd.0 (osd.0) 61 : cluster [DBG] 5.5 scrub starts 2020-09-08T15:54:55.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:54:55 smithi104 bash[22305]: debug 2020-09-08T15:54:55.268+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:56.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:56 smithi166 bash[9446]: cluster 2020-09-08T15:54:55.421703+0000 osd.0 (osd.0) 62 : cluster [DBG] 5.5 scrub ok 2020-09-08T15:54:56.384 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:54:56.385 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:56 smithi104 bash[9203]: cluster 2020-09-08T15:54:55.421703+0000 osd.0 (osd.0) 62 : cluster [DBG] 5.5 scrub ok 2020-09-08T15:54:56.385 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:54:56 smithi104 bash[15273]: debug 2020-09-08T15:54:56.388+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:56.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:56 smithi055 bash[9296]: cluster 2020-09-08T15:54:55.421703+0000 osd.0 (osd.0) 62 : cluster [DBG] 5.5 scrub ok 2020-09-08T15:54:56.489 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:54:56.629 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:54:56.731 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:56.856 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:54:56 smithi055 bash[14301]: debug 2020-09-08T15:54:56.625+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:56.857 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:54:56.962 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:57.086 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:54:57.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:54:56 smithi055 bash[11956]: debug 2020-09-08T15:54:56.861+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:57.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:54:57 smithi055 bash[16646]: debug 2020-09-08T15:54:57.089+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:57.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:57 smithi055 bash[9296]: cluster 2020-09-08T15:54:55.829243+0000 osd.7 (osd.7) 89 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:54:57.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:57 smithi055 bash[9296]: cluster 2020-09-08T15:54:55.865987+0000 mgr.x (mgr.24118) 828 : cluster [DBG] pgmap v1199: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 3.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 785 op/s 2020-09-08T15:54:57.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:57 smithi055 bash[9296]: cluster 2020-09-08T15:54:55.909157+0000 osd.7 (osd.7) 90 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:54:57.193 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:57.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:57 smithi166 bash[9446]: cluster 2020-09-08T15:54:55.829243+0000 osd.7 (osd.7) 89 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:54:57.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:57 smithi166 bash[9446]: cluster 2020-09-08T15:54:55.865987+0000 mgr.x (mgr.24118) 828 : cluster [DBG] pgmap v1199: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 3.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 785 op/s 2020-09-08T15:54:57.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:57 smithi166 bash[9446]: cluster 2020-09-08T15:54:55.909157+0000 osd.7 (osd.7) 90 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:54:57.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:57 smithi104 bash[9203]: cluster 2020-09-08T15:54:55.829243+0000 osd.7 (osd.7) 89 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:54:57.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:57 smithi104 bash[9203]: cluster 2020-09-08T15:54:55.865987+0000 mgr.x (mgr.24118) 828 : cluster [DBG] pgmap v1199: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 3.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 785 op/s 2020-09-08T15:54:57.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:57 smithi104 bash[9203]: cluster 2020-09-08T15:54:55.909157+0000 osd.7 (osd.7) 90 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:54:57.757 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:57.860 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:58.021 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:54:57 smithi166 bash[16593]: debug 2020-09-08T15:54:57.754+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:58.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:58 smithi104 bash[9203]: cluster 2020-09-08T15:54:56.977194+0000 osd.2 (osd.2) 64 : cluster [DBG] 5.6 scrub starts 2020-09-08T15:54:58.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:58 smithi104 bash[9203]: cluster 2020-09-08T15:54:57.032129+0000 osd.0 (osd.0) 63 : cluster [DBG] 5.3 scrub starts 2020-09-08T15:54:58.320 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:58 smithi166 bash[9446]: cluster 2020-09-08T15:54:56.977194+0000 osd.2 (osd.2) 64 : cluster [DBG] 5.6 scrub starts 2020-09-08T15:54:58.321 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:58 smithi166 bash[9446]: cluster 2020-09-08T15:54:57.032129+0000 osd.0 (osd.0) 63 : cluster [DBG] 5.3 scrub starts 2020-09-08T15:54:58.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:58 smithi055 bash[9296]: cluster 2020-09-08T15:54:56.977194+0000 osd.2 (osd.2) 64 : cluster [DBG] 5.6 scrub starts 2020-09-08T15:54:58.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:58 smithi055 bash[9296]: cluster 2020-09-08T15:54:57.032129+0000 osd.0 (osd.0) 63 : cluster [DBG] 5.3 scrub starts 2020-09-08T15:54:58.409 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:54:58.513 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:58.677 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:54:58 smithi104 bash[19922]: debug 2020-09-08T15:54:58.412+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:59.176 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:59.279 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:59.303 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:54:59 smithi166 bash[16593]: debug 2020-09-08T15:54:59.170+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:54:59.303 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:59 smithi166 bash[9446]: audit 2020-09-08T15:54:58.033094+0000 mon.a (mon.0) 1643 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:54:59.304 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:59 smithi166 bash[9446]: audit 2020-09-08T15:54:58.038534+0000 mon.a (mon.0) 1644 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:54:59.304 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:54:59 smithi166 bash[9446]: cluster 2020-09-08T15:54:58.138398+0000 osd.2 (osd.2) 65 : cluster [DBG] 5.6 scrub ok 2020-09-08T15:54:59.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:59 smithi055 bash[9296]: audit 2020-09-08T15:54:58.033094+0000 mon.a (mon.0) 1643 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:54:59.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:59 smithi055 bash[9296]: audit 2020-09-08T15:54:58.038534+0000 mon.a (mon.0) 1644 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:54:59.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:54:59 smithi055 bash[9296]: cluster 2020-09-08T15:54:58.138398+0000 osd.2 (osd.2) 65 : cluster [DBG] 5.6 scrub ok 2020-09-08T15:54:59.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:59 smithi104 bash[9203]: audit 2020-09-08T15:54:58.033094+0000 mon.a (mon.0) 1643 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:54:59.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:59 smithi104 bash[9203]: audit 2020-09-08T15:54:58.038534+0000 mon.a (mon.0) 1644 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:54:59.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:54:59 smithi104 bash[9203]: cluster 2020-09-08T15:54:58.138398+0000 osd.2 (osd.2) 65 : cluster [DBG] 5.6 scrub ok 2020-09-08T15:54:59.751 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:54:59.855 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:00.008 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:00.021 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:54:59 smithi166 bash[16593]: debug 2020-09-08T15:54:59.750+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:00.027 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:55:00.075 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:55:00.113 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:00.122 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:55:00.172 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:55:00.211 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:55:00.246 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:55:00 smithi104 bash[22305]: debug 2020-09-08T15:55:00.012+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:00.246 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:00 smithi104 bash[9203]: cluster 2020-09-08T15:54:58.600561+0000 osd.0 (osd.0) 64 : cluster [DBG] 5.3 scrub ok 2020-09-08T15:55:00.247 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:00 smithi104 bash[9203]: audit 2020-09-08T15:54:59.990528+0000 mon.a (mon.0) 1645 : audit [DBG] from='client.? v1:172.21.15.104:0/2073366598' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:55:00.247 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:00.247 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:55:00.289 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:55:00.337 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:55:00.353 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:55:00.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:00 smithi166 bash[9446]: cluster 2020-09-08T15:54:58.600561+0000 osd.0 (osd.0) 64 : cluster [DBG] 5.3 scrub ok 2020-09-08T15:55:00.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:00 smithi166 bash[9446]: audit 2020-09-08T15:54:59.990528+0000 mon.a (mon.0) 1645 : audit [DBG] from='client.? v1:172.21.15.104:0/2073366598' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:55:00.371 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:00 smithi055 bash[9296]: cluster 2020-09-08T15:54:58.600561+0000 osd.0 (osd.0) 64 : cluster [DBG] 5.3 scrub ok 2020-09-08T15:55:00.372 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:00 smithi055 bash[9296]: audit 2020-09-08T15:54:59.990528+0000 mon.a (mon.0) 1645 : audit [DBG] from='client.? v1:172.21.15.104:0/2073366598' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:55:00.392 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:55:00.423 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:55:00.473 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:55:00.509 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:55:00.510 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:55:00.511 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:55:00.511 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:55:00.511 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:55:00.511 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:55:00.512 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:55:00.512 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:55:00.512 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:55:00.512 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:55:00.512 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:55:00.512 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:55:00.513 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:55:00.520 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:55:00.602 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:55:00 smithi104 bash[22305]: debug 2020-09-08T15:55:00.248+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:00.624 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:00.631 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:55:00 smithi055 bash[14301]: debug 2020-09-08T15:55:00.521+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:00.740 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:00.847 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:55:00.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:55:00 smithi055 bash[16646]: debug 2020-09-08T15:55:00.741+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:00.964 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:55:01.023 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:01 smithi166 bash[9446]: cluster 2020-09-08T15:54:59.866961+0000 mgr.x (mgr.24118) 830 : cluster [DBG] pgmap v1201: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 947 op/s 2020-09-08T15:55:01.023 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:01 smithi166 bash[9446]: audit 2020-09-08T15:55:00.568993+0000 mon.c (mon.2) 66 : audit [INF] from='client.? v1:172.21.15.104:0/2629094802' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:55:01.024 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:01 smithi166 bash[9446]: audit 2020-09-08T15:55:00.572009+0000 mon.a (mon.0) 1646 : audit [INF] from='client.35783 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:55:01.042 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:55:01.063 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:55:01.064 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T15:55:01.064 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:01.064 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:237886 5.e e456/396 5:777cfce9:::benchmark_data_smithi166_37820_object93464:head v 456'14797, mlcod=455'14794)", 2020-09-08T15:55:01.065 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:55:01.065745+0000", 2020-09-08T15:55:01.065 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 0.0016332810000000001, 2020-09-08T15:55:01.065 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0016566930000000001, 2020-09-08T15:55:01.065 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:01.066 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "started", 2020-09-08T15:55:01.066 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:01.066 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:01.066 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:01.067 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:01.065745+0000", 2020-09-08T15:55:01.067 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:01.068 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:01.068 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:01.068 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:01.068 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:01.065745+0000", 2020-09-08T15:55:01.068 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2248000000000002e-05 2020-09-08T15:55:01.068 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:01.069 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:01.069 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:01.069 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:01.065838+0000", 2020-09-08T15:55:01.069 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3006999999999999e-05 2020-09-08T15:55:01.069 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:01.070 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:01.070 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:01.070 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:01.065851+0000", 2020-09-08T15:55:01.070 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.6855e-05 2020-09-08T15:55:01.070 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:01.071 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:01.071 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:01.071 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:01.065877+0000", 2020-09-08T15:55:01.071 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.8260000000000005e-06 2020-09-08T15:55:01.071 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:01.072 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:01.072 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:01.072 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:01.065887+0000", 2020-09-08T15:55:01.072 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.5247999999999997e-05 2020-09-08T15:55:01.072 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:01.073 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:01.073 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:01.073 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:01.065943+0000", 2020-09-08T15:55:01.074 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.7898999999999997e-05 2020-09-08T15:55:01.074 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:01.074 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:01.074 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:01.074 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:01.065980+0000", 2020-09-08T15:55:01.075 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000235083 2020-09-08T15:55:01.075 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:01.075 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:01.075 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:01.076 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:01.076 INFO:teuthology.orchestra.run.smithi166.stdout: ], 2020-09-08T15:55:01.076 INFO:teuthology.orchestra.run.smithi166.stdout: "num_ops": 1 2020-09-08T15:55:01.076 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:55:01.080 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:01.188 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:01.296 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:01.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:55:00 smithi055 bash[11956]: debug 2020-09-08T15:55:00.961+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:01.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:55:01 smithi055 bash[16646]: debug 2020-09-08T15:55:01.189+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:01.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:01 smithi055 bash[9296]: cluster 2020-09-08T15:54:59.866961+0000 mgr.x (mgr.24118) 830 : cluster [DBG] pgmap v1201: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 947 op/s 2020-09-08T15:55:01.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:01 smithi055 bash[9296]: audit 2020-09-08T15:55:00.568993+0000 mon.c (mon.2) 66 : audit [INF] from='client.? v1:172.21.15.104:0/2629094802' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:55:01.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:01 smithi055 bash[9296]: audit 2020-09-08T15:55:00.572009+0000 mon.a (mon.0) 1646 : audit [INF] from='client.35783 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:55:01.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:01 smithi104 bash[9203]: cluster 2020-09-08T15:54:59.866961+0000 mgr.x (mgr.24118) 830 : cluster [DBG] pgmap v1201: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 947 op/s 2020-09-08T15:55:01.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:01 smithi104 bash[9203]: audit 2020-09-08T15:55:00.568993+0000 mon.c (mon.2) 66 : audit [INF] from='client.? v1:172.21.15.104:0/2629094802' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:55:01.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:01 smithi104 bash[9203]: audit 2020-09-08T15:55:00.572009+0000 mon.a (mon.0) 1646 : audit [INF] from='client.35783 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:55:02.031 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:02.117 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:55:02 smithi166 bash[11928]: debug 2020-09-08T15:55:02.030+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:02.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:02 smithi166 bash[9446]: cluster 2020-09-08T15:55:01.027376+0000 mon.a (mon.0) 1647 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:55:02.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:02 smithi166 bash[9446]: cluster 2020-09-08T15:55:01.027409+0000 mon.a (mon.0) 1648 : cluster [INF] Cluster is now healthy 2020-09-08T15:55:02.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:02 smithi166 bash[9446]: audit 2020-09-08T15:55:01.038841+0000 mon.a (mon.0) 1649 : audit [INF] from='client.35783 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:55:02.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:02 smithi166 bash[9446]: cluster 2020-09-08T15:55:01.038885+0000 mon.a (mon.0) 1650 : cluster [DBG] osdmap e456: 12 total, 12 up, 12 in 2020-09-08T15:55:02.134 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:02.181 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2020-09-08T15:55:02.280 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:02.326 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:02 smithi104 bash[9203]: cluster 2020-09-08T15:55:01.027376+0000 mon.a (mon.0) 1647 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:55:02.327 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:02 smithi104 bash[9203]: cluster 2020-09-08T15:55:01.027409+0000 mon.a (mon.0) 1648 : cluster [INF] Cluster is now healthy 2020-09-08T15:55:02.327 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:02 smithi104 bash[9203]: audit 2020-09-08T15:55:01.038841+0000 mon.a (mon.0) 1649 : audit [INF] from='client.35783 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:55:02.327 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:02 smithi104 bash[9203]: cluster 2020-09-08T15:55:01.038885+0000 mon.a (mon.0) 1650 : cluster [DBG] osdmap e456: 12 total, 12 up, 12 in 2020-09-08T15:55:02.335 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:55:02.387 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:02.396 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:55:02 smithi166 bash[11928]: debug 2020-09-08T15:55:02.278+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:02.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:02 smithi055 bash[9296]: cluster 2020-09-08T15:55:01.027376+0000 mon.a (mon.0) 1647 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:55:02.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:02 smithi055 bash[9296]: cluster 2020-09-08T15:55:01.027409+0000 mon.a (mon.0) 1648 : cluster [INF] Cluster is now healthy 2020-09-08T15:55:02.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:02 smithi055 bash[9296]: audit 2020-09-08T15:55:01.038841+0000 mon.a (mon.0) 1649 : audit [INF] from='client.35783 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:55:02.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:02 smithi055 bash[9296]: cluster 2020-09-08T15:55:01.038885+0000 mon.a (mon.0) 1650 : cluster [DBG] osdmap e456: 12 total, 12 up, 12 in 2020-09-08T15:55:03.081 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:03.190 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:03.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:03 smithi166 bash[9446]: cluster 2020-09-08T15:55:01.666219+0000 osd.9 (osd.9) 71 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:03.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:03 smithi166 bash[9446]: cluster 2020-09-08T15:55:01.666284+0000 osd.9 (osd.9) 72 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:03.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:03 smithi166 bash[9446]: cluster 2020-09-08T15:55:01.867401+0000 mgr.x (mgr.24118) 831 : cluster [DBG] pgmap v1203: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 717 op/s 2020-09-08T15:55:03.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:03 smithi166 bash[9446]: cluster 2020-09-08T15:55:02.043715+0000 mon.a (mon.0) 1651 : cluster [DBG] osdmap e457: 12 total, 12 up, 12 in 2020-09-08T15:55:03.370 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:55:03 smithi166 bash[16593]: debug 2020-09-08T15:55:03.078+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:03.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:03 smithi055 bash[9296]: cluster 2020-09-08T15:55:01.666219+0000 osd.9 (osd.9) 71 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:03.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:03 smithi055 bash[9296]: cluster 2020-09-08T15:55:01.666284+0000 osd.9 (osd.9) 72 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:03.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:03 smithi055 bash[9296]: cluster 2020-09-08T15:55:01.867401+0000 mgr.x (mgr.24118) 831 : cluster [DBG] pgmap v1203: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 717 op/s 2020-09-08T15:55:03.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:03 smithi055 bash[9296]: cluster 2020-09-08T15:55:02.043715+0000 mon.a (mon.0) 1651 : cluster [DBG] osdmap e457: 12 total, 12 up, 12 in 2020-09-08T15:55:03.436 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:03 smithi104 bash[9203]: cluster 2020-09-08T15:55:01.666219+0000 osd.9 (osd.9) 71 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:03.437 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:03 smithi104 bash[9203]: cluster 2020-09-08T15:55:01.666284+0000 osd.9 (osd.9) 72 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:03.437 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:03 smithi104 bash[9203]: cluster 2020-09-08T15:55:01.867401+0000 mgr.x (mgr.24118) 831 : cluster [DBG] pgmap v1203: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 717 op/s 2020-09-08T15:55:03.437 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:03 smithi104 bash[9203]: cluster 2020-09-08T15:55:02.043715+0000 mon.a (mon.0) 1651 : cluster [DBG] osdmap e457: 12 total, 12 up, 12 in 2020-09-08T15:55:03.698 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:03.848 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:55:03.965 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:55:04.013 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:55:03 smithi166 bash[18960]: debug 2020-09-08T15:55:03.698+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:04.069 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:55:04.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:55:03 smithi055 bash[11956]: debug 2020-09-08T15:55:03.965+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:04.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:04 smithi055 bash[9296]: cluster 2020-09-08T15:55:03.664126+0000 osd.2 (osd.2) 66 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:04.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:04 smithi055 bash[9296]: cluster 2020-09-08T15:55:03.664217+0000 osd.2 (osd.2) 67 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:04.189 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:55:04.294 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:04.327 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:55:04 smithi166 bash[14254]: debug 2020-09-08T15:55:04.190+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:04.328 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:04 smithi166 bash[9446]: cluster 2020-09-08T15:55:03.664126+0000 osd.2 (osd.2) 66 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:04.328 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:04 smithi166 bash[9446]: cluster 2020-09-08T15:55:03.664217+0000 osd.2 (osd.2) 67 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:04.330 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:55:04.338 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:04 smithi104 bash[9203]: cluster 2020-09-08T15:55:03.664126+0000 osd.2 (osd.2) 66 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:04.339 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:04 smithi104 bash[9203]: cluster 2020-09-08T15:55:03.664217+0000 osd.2 (osd.2) 67 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:04.411 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:04.430 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:55:04.503 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:55:04.514 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:04.597 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:55:04.660 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:55:04 smithi055 bash[19003]: debug 2020-09-08T15:55:04.409+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:05.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:05 smithi166 bash[9446]: cluster 2020-09-08T15:55:03.867905+0000 mgr.x (mgr.24118) 832 : cluster [DBG] pgmap v1205: 17 pgs: 17 active+clean; 3.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T15:55:05.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:05 smithi166 bash[9446]: cluster 2020-09-08T15:55:04.055589+0000 mon.a (mon.0) 1652 : cluster [DBG] osdmap e458: 12 total, 12 up, 12 in 2020-09-08T15:55:05.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:05 smithi166 bash[9446]: cluster 2020-09-08T15:55:04.236293+0000 osd.5 (osd.5) 61 : cluster [DBG] 5.7 deep-scrub starts 2020-09-08T15:55:05.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:05 smithi055 bash[9296]: cluster 2020-09-08T15:55:03.867905+0000 mgr.x (mgr.24118) 832 : cluster [DBG] pgmap v1205: 17 pgs: 17 active+clean; 3.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T15:55:05.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:05 smithi055 bash[9296]: cluster 2020-09-08T15:55:04.055589+0000 mon.a (mon.0) 1652 : cluster [DBG] osdmap e458: 12 total, 12 up, 12 in 2020-09-08T15:55:05.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:05 smithi055 bash[9296]: cluster 2020-09-08T15:55:04.236293+0000 osd.5 (osd.5) 61 : cluster [DBG] 5.7 deep-scrub starts 2020-09-08T15:55:05.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:05 smithi104 bash[9203]: cluster 2020-09-08T15:55:03.867905+0000 mgr.x (mgr.24118) 832 : cluster [DBG] pgmap v1205: 17 pgs: 17 active+clean; 3.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T15:55:05.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:05 smithi104 bash[9203]: cluster 2020-09-08T15:55:04.055589+0000 mon.a (mon.0) 1652 : cluster [DBG] osdmap e458: 12 total, 12 up, 12 in 2020-09-08T15:55:05.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:05 smithi104 bash[9203]: cluster 2020-09-08T15:55:04.236293+0000 osd.5 (osd.5) 61 : cluster [DBG] 5.7 deep-scrub starts 2020-09-08T15:55:05.589 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:05.693 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:05.925 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:55:05 smithi104 bash[22305]: debug 2020-09-08T15:55:05.592+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:06.172 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:06.277 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:06.422 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:06.425 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:55:06 smithi104 bash[22305]: debug 2020-09-08T15:55:06.176+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:06.528 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:06.621 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:55:06.622 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T15:55:06.622 INFO:teuthology.orchestra.run.smithi166.stdout: "complaint_time": 30, 2020-09-08T15:55:06.622 INFO:teuthology.orchestra.run.smithi166.stdout: "num_blocked_ops": 0 2020-09-08T15:55:06.622 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:55:06.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:55:06 smithi055 bash[19003]: debug 2020-09-08T15:55:06.425+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:06.795 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:06.899 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:07.054 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:55:06 smithi104 bash[22305]: debug 2020-09-08T15:55:06.800+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:07.055 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:07 smithi104 bash[9203]: cluster 2020-09-08T15:55:05.868248+0000 mgr.x (mgr.24118) 833 : cluster [DBG] pgmap v1207: 17 pgs: 17 active+clean; 3.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 833 op/s 2020-09-08T15:55:07.130 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:07.233 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:07.353 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:55:07 smithi055 bash[19003]: debug 2020-09-08T15:55:07.357+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:07.354 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:07 smithi055 bash[9296]: cluster 2020-09-08T15:55:05.868248+0000 mgr.x (mgr.24118) 833 : cluster [DBG] pgmap v1207: 17 pgs: 17 active+clean; 3.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 833 op/s 2020-09-08T15:55:07.359 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:07 smithi166 bash[9446]: cluster 2020-09-08T15:55:05.868248+0000 mgr.x (mgr.24118) 833 : cluster [DBG] pgmap v1207: 17 pgs: 17 active+clean; 3.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 833 op/s 2020-09-08T15:55:07.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:55:07 smithi104 bash[15273]: debug 2020-09-08T15:55:07.132+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:07.468 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:07.477 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2020-09-08T15:55:07.592 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:07.611 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:55:07.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:55:07 smithi055 bash[16646]: debug 2020-09-08T15:55:07.593+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:07.705 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:08.218 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:08.322 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:08.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:55:08 smithi166 bash[11928]: debug 2020-09-08T15:55:08.214+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:08.766 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:08.870 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:09.008 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:09.046 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:55:08 smithi166 bash[11928]: debug 2020-09-08T15:55:08.766+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:09.113 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:55:09.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:09 smithi055 bash[9296]: cluster 2020-09-08T15:55:07.868815+0000 mgr.x (mgr.24118) 834 : cluster [DBG] pgmap v1208: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.17k op/s 2020-09-08T15:55:09.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:55:09 smithi055 bash[19003]: debug 2020-09-08T15:55:09.009+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:09.218 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:55:09.324 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:09.331 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:09 smithi166 bash[9446]: cluster 2020-09-08T15:55:07.868815+0000 mgr.x (mgr.24118) 834 : cluster [DBG] pgmap v1208: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.17k op/s 2020-09-08T15:55:09.331 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:55:09 smithi166 bash[14254]: debug 2020-09-08T15:55:09.218+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:09.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:09 smithi104 bash[9203]: cluster 2020-09-08T15:55:07.868815+0000 mgr.x (mgr.24118) 834 : cluster [DBG] pgmap v1208: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.17k op/s 2020-09-08T15:55:09.500 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:09.604 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:09.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:55:09 smithi166 bash[16593]: debug 2020-09-08T15:55:09.498+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:11.085 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:10 smithi104 bash[9203]: cluster 2020-09-08T15:55:09.869344+0000 mgr.x (mgr.24118) 835 : cluster [DBG] pgmap v1209: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.07k op/s 2020-09-08T15:55:11.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:10 smithi166 bash[9446]: cluster 2020-09-08T15:55:09.869344+0000 mgr.x (mgr.24118) 835 : cluster [DBG] pgmap v1209: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.07k op/s 2020-09-08T15:55:11.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:10 smithi055 bash[9296]: cluster 2020-09-08T15:55:09.869344+0000 mgr.x (mgr.24118) 835 : cluster [DBG] pgmap v1209: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.07k op/s 2020-09-08T15:55:11.388 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:55:11.436 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:55:11.478 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:55:11.531 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:55:11.547 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:55:11.547 INFO:teuthology.orchestra.run.smithi166.stdout: "size": 20, 2020-09-08T15:55:11.547 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 600, 2020-09-08T15:55:11.548 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T15:55:11.548 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.548 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:112250 5.a e424/396 5:55313165:::benchmark_data_smithi166_37820_object112249:head v 424'7056, mlcod=424'7046)", 2020-09-08T15:55:11.548 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:52:53.624181+0000", 2020-09-08T15:55:11.548 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 137.92416893699999, 2020-09-08T15:55:11.549 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.028468752, 2020-09-08T15:55:11.549 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:11.549 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:11.549 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:11.549 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.550 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:11.550 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.624181+0000", 2020-09-08T15:55:11.550 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:11.550 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.550 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.551 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:11.551 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.624181+0000", 2020-09-08T15:55:11.551 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00024662300000000001 2020-09-08T15:55:11.551 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.551 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.552 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:11.552 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.624427+0000", 2020-09-08T15:55:11.552 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.3489999999999998e-06 2020-09-08T15:55:11.552 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.552 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.553 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:11.553 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.624437+0000", 2020-09-08T15:55:11.553 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5302e-05 2020-09-08T15:55:11.553 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.553 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.554 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:11.554 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.624452+0000", 2020-09-08T15:55:11.554 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.0079999999999999e-06 2020-09-08T15:55:11.554 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.554 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.555 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:11.555 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.624458+0000", 2020-09-08T15:55:11.555 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.017147094000000002 2020-09-08T15:55:11.555 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.555 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.556 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:11.556 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.641605+0000", 2020-09-08T15:55:11.556 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0583e-05 2020-09-08T15:55:11.556 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.557 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.557 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:11.557 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.641656+0000", 2020-09-08T15:55:11.557 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.010915609999999999 2020-09-08T15:55:11.557 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.558 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.558 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:11.558 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.652571+0000", 2020-09-08T15:55:11.558 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.8182999999999993e-05 2020-09-08T15:55:11.558 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.559 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.559 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:11.559 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.652649+0000", 2020-09-08T15:55:11.559 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.028468752 2020-09-08T15:55:11.559 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.560 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.560 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.560 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.560 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.560 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:112260 5.a e424/396 5:5e806c69:::benchmark_data_smithi166_37820_object112259:head v 424'7057, mlcod=424'7046)", 2020-09-08T15:55:11.560 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:52:53.624507+0000", 2020-09-08T15:55:11.561 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 137.92384236199999, 2020-09-08T15:55:11.561 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.033472633000000002, 2020-09-08T15:55:11.561 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:11.561 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:11.561 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:11.562 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.562 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:11.562 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.624507+0000", 2020-09-08T15:55:11.562 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:11.562 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.563 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.563 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:11.563 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.624507+0000", 2020-09-08T15:55:11.563 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017668 2020-09-08T15:55:11.563 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.564 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.564 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:11.564 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.624684+0000", 2020-09-08T15:55:11.564 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2727e-05 2020-09-08T15:55:11.564 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.564 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.565 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:11.565 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.624697+0000", 2020-09-08T15:55:11.565 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.8615e-05 2020-09-08T15:55:11.565 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.565 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.566 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:11.566 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.624715+0000", 2020-09-08T15:55:11.566 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.48e-06 2020-09-08T15:55:11.566 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.566 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.567 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:11.567 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.624721+0000", 2020-09-08T15:55:11.567 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.020639784000000001 2020-09-08T15:55:11.567 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.567 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.568 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:11.568 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.645361+0000", 2020-09-08T15:55:11.568 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.5987e-05 2020-09-08T15:55:11.568 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.568 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.569 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:11.569 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.645396+0000", 2020-09-08T15:55:11.569 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.012528815 2020-09-08T15:55:11.569 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.569 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.570 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:11.570 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.657925+0000", 2020-09-08T15:55:11.570 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.4545e-05 2020-09-08T15:55:11.570 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.570 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.570 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:11.571 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.657980+0000", 2020-09-08T15:55:11.571 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.033472633000000002 2020-09-08T15:55:11.571 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.571 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.571 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.572 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.572 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.572 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:112275 5.a e424/396 5:5ca89ea2:::benchmark_data_smithi166_37820_object112274:head v 424'7058, mlcod=424'7046)", 2020-09-08T15:55:11.572 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:52:53.624787+0000", 2020-09-08T15:55:11.572 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 137.923562312, 2020-09-08T15:55:11.573 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.033227281999999997, 2020-09-08T15:55:11.573 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:11.573 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:11.573 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:11.573 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.573 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:11.574 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.624787+0000", 2020-09-08T15:55:11.574 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:11.574 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.574 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.574 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:11.575 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.624787+0000", 2020-09-08T15:55:11.575 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017353900000000001 2020-09-08T15:55:11.575 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.575 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.575 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:11.576 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.624961+0000", 2020-09-08T15:55:11.576 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2098e-05 2020-09-08T15:55:11.576 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.576 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.576 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:11.576 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.624973+0000", 2020-09-08T15:55:11.577 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.8439000000000001e-05 2020-09-08T15:55:11.577 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.577 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.577 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:11.577 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.624991+0000", 2020-09-08T15:55:11.578 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.3179999999999997e-06 2020-09-08T15:55:11.578 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.578 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.578 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:11.578 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.624997+0000", 2020-09-08T15:55:11.579 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.024426541 2020-09-08T15:55:11.579 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.579 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.579 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:11.579 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.649423+0000", 2020-09-08T15:55:11.579 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.9538000000000001e-05 2020-09-08T15:55:11.580 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.580 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.580 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:11.580 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.649453+0000", 2020-09-08T15:55:11.580 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0085319620000000006 2020-09-08T15:55:11.581 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.581 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.581 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:11.581 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.657985+0000", 2020-09-08T15:55:11.581 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.9847000000000001e-05 2020-09-08T15:55:11.582 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.582 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.582 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:11.582 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.658015+0000", 2020-09-08T15:55:11.582 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.033227281999999997 2020-09-08T15:55:11.583 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.583 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.583 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.583 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.583 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.584 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:112280 5.a e424/396 5:505b3afe:::benchmark_data_smithi166_37820_object112279:head v 424'7059, mlcod=424'7046)", 2020-09-08T15:55:11.584 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:52:53.625069+0000", 2020-09-08T15:55:11.584 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 137.923280205, 2020-09-08T15:55:11.584 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.032973020999999998, 2020-09-08T15:55:11.585 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:11.585 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:11.585 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:11.585 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.586 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:11.586 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.625069+0000", 2020-09-08T15:55:11.586 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:11.586 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.587 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.587 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:11.588 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.625069+0000", 2020-09-08T15:55:11.588 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019127599999999999 2020-09-08T15:55:11.588 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.588 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.589 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:11.589 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.625261+0000", 2020-09-08T15:55:11.589 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.8034999999999999e-05 2020-09-08T15:55:11.589 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.590 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.590 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:11.590 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.625279+0000", 2020-09-08T15:55:11.590 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.6414000000000002e-05 2020-09-08T15:55:11.590 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.591 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.591 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:11.591 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.625305+0000", 2020-09-08T15:55:11.591 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.5960000000000003e-06 2020-09-08T15:55:11.591 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.592 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.592 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:11.592 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.625311+0000", 2020-09-08T15:55:11.592 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.027413793999999998 2020-09-08T15:55:11.592 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.593 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.593 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:11.593 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.652724+0000", 2020-09-08T15:55:11.593 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.0374000000000001e-05 2020-09-08T15:55:11.593 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.593 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.594 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:11.594 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.652755+0000", 2020-09-08T15:55:11.594 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0052610319999999997 2020-09-08T15:55:11.594 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.594 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.595 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:11.595 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.658016+0000", 2020-09-08T15:55:11.595 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.65e-05 2020-09-08T15:55:11.595 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.595 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.596 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:11.596 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.658042+0000", 2020-09-08T15:55:11.596 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.032973020999999998 2020-09-08T15:55:11.596 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.596 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.597 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.597 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.597 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.597 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:112319 5.a e424/396 5:5a725112:::benchmark_data_smithi166_37820_object112318:head v 424'7060, mlcod=424'7046)", 2020-09-08T15:55:11.597 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:52:53.627009+0000", 2020-09-08T15:55:11.597 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 137.92134095, 2020-09-08T15:55:11.598 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.031225335999999999, 2020-09-08T15:55:11.598 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:11.598 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:11.598 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:11.598 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.599 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:11.599 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.627009+0000", 2020-09-08T15:55:11.599 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:11.599 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.599 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.600 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:11.600 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.627009+0000", 2020-09-08T15:55:11.600 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000118838 2020-09-08T15:55:11.601 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.601 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.601 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:11.601 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.627127+0000", 2020-09-08T15:55:11.602 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.3059999999999998e-06 2020-09-08T15:55:11.602 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.602 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.602 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:11.602 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.627136+0000", 2020-09-08T15:55:11.603 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2751999999999999e-05 2020-09-08T15:55:11.603 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.603 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.603 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:11.603 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.627149+0000", 2020-09-08T15:55:11.603 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.3890000000000004e-06 2020-09-08T15:55:11.604 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.604 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.604 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:11.604 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.627154+0000", 2020-09-08T15:55:11.604 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.028244308999999999 2020-09-08T15:55:11.605 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.605 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.605 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:11.605 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.655398+0000", 2020-09-08T15:55:11.605 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.6120999999999999e-05 2020-09-08T15:55:11.606 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.606 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.606 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:11.606 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.655424+0000", 2020-09-08T15:55:11.606 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0027483550000000001 2020-09-08T15:55:11.607 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.607 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.607 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:11.607 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.658173+0000", 2020-09-08T15:55:11.607 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.1266000000000006e-05 2020-09-08T15:55:11.607 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.608 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.608 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:11.608 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:53.658234+0000", 2020-09-08T15:55:11.608 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.031225335999999999 2020-09-08T15:55:11.608 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.609 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.609 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.609 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.610 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.610 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:113847 5.a e424/396 5:5b3582c7:::benchmark_data_smithi166_37820_object113846:head v 424'7178, mlcod=424'7176)", 2020-09-08T15:55:11.610 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:52:54.816896+0000", 2020-09-08T15:55:11.610 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 136.73145335199999, 2020-09-08T15:55:11.610 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.031068442000000002, 2020-09-08T15:55:11.611 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:11.611 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:11.611 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:11.611 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.611 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:11.612 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.816896+0000", 2020-09-08T15:55:11.612 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:11.612 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.612 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.612 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:11.612 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.816896+0000", 2020-09-08T15:55:11.613 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000119598 2020-09-08T15:55:11.613 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.613 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.613 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:11.613 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.817016+0000", 2020-09-08T15:55:11.614 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2865e-05 2020-09-08T15:55:11.614 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.614 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.614 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:11.614 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.817029+0000", 2020-09-08T15:55:11.615 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.9426999999999999e-05 2020-09-08T15:55:11.615 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.615 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.615 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:11.615 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.817048+0000", 2020-09-08T15:55:11.615 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.6429999999999995e-06 2020-09-08T15:55:11.616 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.616 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.616 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:11.616 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.817056+0000", 2020-09-08T15:55:11.616 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0010951999999999999 2020-09-08T15:55:11.617 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.617 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.617 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:11.617 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.818151+0000", 2020-09-08T15:55:11.617 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.7282999999999999e-05 2020-09-08T15:55:11.618 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.618 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.618 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:11.618 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.818178+0000", 2020-09-08T15:55:11.618 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.029725870000000001 2020-09-08T15:55:11.618 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.619 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.619 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:11.619 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.847904+0000", 2020-09-08T15:55:11.619 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.0556000000000001e-05 2020-09-08T15:55:11.619 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.620 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.620 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:11.620 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.847965+0000", 2020-09-08T15:55:11.620 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.031068442000000002 2020-09-08T15:55:11.620 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.621 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.621 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.621 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.621 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.621 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:113854 5.a e424/396 5:52b02aeb:::benchmark_data_smithi166_37820_object113853:head v 424'7179, mlcod=424'7176)", 2020-09-08T15:55:11.622 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:52:54.827028+0000", 2020-09-08T15:55:11.622 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 136.721321227, 2020-09-08T15:55:11.622 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.034545094999999998, 2020-09-08T15:55:11.622 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:11.622 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:11.622 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:11.623 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.623 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:11.623 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.827028+0000", 2020-09-08T15:55:11.623 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:11.623 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.624 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.624 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:11.624 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.827028+0000", 2020-09-08T15:55:11.624 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011736899999999999 2020-09-08T15:55:11.624 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.625 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.625 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:11.625 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.827146+0000", 2020-09-08T15:55:11.625 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0101999999999999e-05 2020-09-08T15:55:11.625 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.626 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.626 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:11.626 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.827156+0000", 2020-09-08T15:55:11.626 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7825999999999998e-05 2020-09-08T15:55:11.626 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.626 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.627 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:11.627 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.827174+0000", 2020-09-08T15:55:11.627 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0360000000000001e-05 2020-09-08T15:55:11.627 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.627 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.628 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:11.628 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.827184+0000", 2020-09-08T15:55:11.628 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.4870999999999996e-05 2020-09-08T15:55:11.628 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.628 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.629 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:11.629 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.827249+0000", 2020-09-08T15:55:11.629 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.2716999999999998e-05 2020-09-08T15:55:11.629 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.629 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.629 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:11.630 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.827292+0000", 2020-09-08T15:55:11.630 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.03415344 2020-09-08T15:55:11.630 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.630 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.630 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:11.631 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.861445+0000", 2020-09-08T15:55:11.631 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012841 2020-09-08T15:55:11.631 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.631 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.631 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:11.632 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.861573+0000", 2020-09-08T15:55:11.632 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.034545094999999998 2020-09-08T15:55:11.632 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.632 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.632 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.632 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.633 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.633 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:113856 5.a e424/396 5:561a1929:::benchmark_data_smithi166_37820_object113855:head v 424'7180, mlcod=424'7176)", 2020-09-08T15:55:11.633 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:52:54.830081+0000", 2020-09-08T15:55:11.633 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 136.718268107, 2020-09-08T15:55:11.633 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.032244986000000003, 2020-09-08T15:55:11.634 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:11.634 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:11.634 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:11.634 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.634 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:11.635 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.830081+0000", 2020-09-08T15:55:11.635 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:11.635 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.635 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.635 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:11.636 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.830081+0000", 2020-09-08T15:55:11.636 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.1238999999999999e-05 2020-09-08T15:55:11.636 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.636 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.636 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:11.636 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.830163+0000", 2020-09-08T15:55:11.637 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.9179999999999997e-06 2020-09-08T15:55:11.637 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.637 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.637 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:11.637 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.830170+0000", 2020-09-08T15:55:11.638 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4878999999999999e-05 2020-09-08T15:55:11.638 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.638 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.638 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:11.638 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.830185+0000", 2020-09-08T15:55:11.639 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.6790000000000005e-06 2020-09-08T15:55:11.639 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.639 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.639 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:11.639 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.830192+0000", 2020-09-08T15:55:11.639 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.9235999999999997e-05 2020-09-08T15:55:11.640 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.640 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.640 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:11.640 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.830281+0000", 2020-09-08T15:55:11.640 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.2700999999999998e-05 2020-09-08T15:55:11.641 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.641 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.641 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:11.641 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.830304+0000", 2020-09-08T15:55:11.641 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.03189811 2020-09-08T15:55:11.642 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.642 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.642 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:11.642 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.862202+0000", 2020-09-08T15:55:11.642 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012422400000000001 2020-09-08T15:55:11.642 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.643 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.643 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:11.643 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:54.862326+0000", 2020-09-08T15:55:11.643 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.032244986000000003 2020-09-08T15:55:11.643 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.644 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.644 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.644 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.644 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.644 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:144909 5.e e431/396 5:73616b48:::benchmark_data_smithi166_37820_object487:head v 431'8977, mlcod=431'8975)", 2020-09-08T15:55:11.645 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:53:24.840355+0000", 2020-09-08T15:55:11.645 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 106.707994984, 2020-09-08T15:55:11.645 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.044145459999999997, 2020-09-08T15:55:11.645 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:11.645 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:11.646 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:11.646 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.646 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:11.646 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:24.840355+0000", 2020-09-08T15:55:11.646 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:11.646 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.647 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.647 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:11.647 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:24.840355+0000", 2020-09-08T15:55:11.647 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.8347000000000001e-05 2020-09-08T15:55:11.647 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.648 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.648 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:11.648 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:24.840433+0000", 2020-09-08T15:55:11.648 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0678e-05 2020-09-08T15:55:11.648 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.648 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.649 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:11.649 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:24.840444+0000", 2020-09-08T15:55:11.649 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6724000000000001e-05 2020-09-08T15:55:11.649 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.649 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.650 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:11.650 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:24.840460+0000", 2020-09-08T15:55:11.650 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1464999999999999e-05 2020-09-08T15:55:11.650 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.650 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.651 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:11.651 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:24.840472+0000", 2020-09-08T15:55:11.651 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8884000000000001e-05 2020-09-08T15:55:11.651 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.651 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.651 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:11.652 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:24.840531+0000", 2020-09-08T15:55:11.652 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.1061e-05 2020-09-08T15:55:11.652 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.652 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.652 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:11.653 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:24.840562+0000", 2020-09-08T15:55:11.653 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.043844375999999997 2020-09-08T15:55:11.653 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.653 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.654 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:11.654 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:24.884406+0000", 2020-09-08T15:55:11.654 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.3925000000000006e-05 2020-09-08T15:55:11.654 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.654 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.654 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:11.655 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:24.884500+0000", 2020-09-08T15:55:11.655 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.044145459999999997 2020-09-08T15:55:11.655 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.655 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.655 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.656 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.656 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.656 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:149910 5.e e432/396 5:72be4a4f:::benchmark_data_smithi166_37820_object5488:head v 432'9299, mlcod=432'9296)", 2020-09-08T15:55:11.656 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:53:29.850287+0000", 2020-09-08T15:55:11.656 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 101.69806235599999, 2020-09-08T15:55:11.657 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.029316617999999999, 2020-09-08T15:55:11.657 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:11.657 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:11.657 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:11.658 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.658 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:11.658 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:29.850287+0000", 2020-09-08T15:55:11.658 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:11.658 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.659 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.659 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:11.659 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:29.850287+0000", 2020-09-08T15:55:11.659 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.3300000000000002e-05 2020-09-08T15:55:11.659 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.660 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.660 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:11.660 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:29.850331+0000", 2020-09-08T15:55:11.660 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.9829999999999996e-06 2020-09-08T15:55:11.660 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.660 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.661 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:11.661 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:29.850339+0000", 2020-09-08T15:55:11.661 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3278000000000001e-05 2020-09-08T15:55:11.661 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.662 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.662 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:11.662 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:29.850352+0000", 2020-09-08T15:55:11.662 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.5380000000000002e-06 2020-09-08T15:55:11.662 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.662 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.663 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:11.663 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:29.850360+0000", 2020-09-08T15:55:11.663 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.4879999999999997e-05 2020-09-08T15:55:11.664 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.664 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.664 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:11.664 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:29.850405+0000", 2020-09-08T15:55:11.664 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.2342000000000001e-05 2020-09-08T15:55:11.664 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.665 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.665 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:11.665 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:29.850428+0000", 2020-09-08T15:55:11.665 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.029090365999999999 2020-09-08T15:55:11.665 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.666 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.666 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:11.666 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:29.879518+0000", 2020-09-08T15:55:11.666 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.5931000000000001e-05 2020-09-08T15:55:11.666 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.667 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.667 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:11.667 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:29.879604+0000", 2020-09-08T15:55:11.667 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.029316617999999999 2020-09-08T15:55:11.667 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.668 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.668 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.668 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.668 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.668 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:172461 5.a e439/396 5:5fae033e:::benchmark_data_smithi166_37820_object28039:head v 439'10920, mlcod=439'10918)", 2020-09-08T15:55:11.668 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:53:54.851510+0000", 2020-09-08T15:55:11.669 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 76.696839257999997, 2020-09-08T15:55:11.669 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.035652412000000001, 2020-09-08T15:55:11.669 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:11.669 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:11.669 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:11.670 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.670 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:11.670 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:54.851510+0000", 2020-09-08T15:55:11.670 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:11.670 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.671 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.671 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:11.671 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:54.851510+0000", 2020-09-08T15:55:11.671 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.9094000000000002e-05 2020-09-08T15:55:11.671 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.672 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.672 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:11.672 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:54.851549+0000", 2020-09-08T15:55:11.672 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2520000000000005e-06 2020-09-08T15:55:11.672 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.672 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.673 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:11.673 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:54.851559+0000", 2020-09-08T15:55:11.673 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.365e-05 2020-09-08T15:55:11.673 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.673 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.674 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:11.674 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:54.851572+0000", 2020-09-08T15:55:11.675 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.1070000000000005e-06 2020-09-08T15:55:11.675 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.675 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.675 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:11.676 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:54.851580+0000", 2020-09-08T15:55:11.676 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7587000000000003e-05 2020-09-08T15:55:11.676 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.677 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.677 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:11.677 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:54.851638+0000", 2020-09-08T15:55:11.677 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.7699999999999999e-05 2020-09-08T15:55:11.677 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.678 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.678 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:11.678 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:54.851666+0000", 2020-09-08T15:55:11.678 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.035284976000000003 2020-09-08T15:55:11.678 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.678 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.679 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:11.679 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:54.886951+0000", 2020-09-08T15:55:11.679 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000212046 2020-09-08T15:55:11.679 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.679 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.680 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:11.680 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:54.887163+0000", 2020-09-08T15:55:11.680 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.035652412000000001 2020-09-08T15:55:11.680 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.680 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.681 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.681 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.681 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.681 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:192184 5.a e444/396 5:5b2d320e:::benchmark_data_smithi166_37820_object47762:head v 444'12152, mlcod=444'12150)", 2020-09-08T15:55:11.681 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:54:14.854365+0000", 2020-09-08T15:55:11.681 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 56.693984647999997, 2020-09-08T15:55:11.682 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.039477225999999997, 2020-09-08T15:55:11.682 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:11.682 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:11.682 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:11.682 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.683 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:11.683 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:14.854365+0000", 2020-09-08T15:55:11.683 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:11.683 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.683 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.684 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:11.684 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:14.854365+0000", 2020-09-08T15:55:11.684 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6266000000000006e-05 2020-09-08T15:55:11.684 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.685 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.685 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:11.685 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:14.854431+0000", 2020-09-08T15:55:11.685 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3954000000000001e-05 2020-09-08T15:55:11.685 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.686 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.686 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:11.686 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:14.854445+0000", 2020-09-08T15:55:11.686 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.8275e-05 2020-09-08T15:55:11.686 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.687 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.687 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:11.687 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:14.854463+0000", 2020-09-08T15:55:11.687 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0835999999999999e-05 2020-09-08T15:55:11.687 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.687 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.688 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:11.688 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:14.854474+0000", 2020-09-08T15:55:11.688 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013577000000000001 2020-09-08T15:55:11.688 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.688 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.689 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:11.689 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:14.854610+0000", 2020-09-08T15:55:11.689 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.0874000000000003e-05 2020-09-08T15:55:11.689 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.689 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.690 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:11.690 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:14.854641+0000", 2020-09-08T15:55:11.690 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.039097172999999999 2020-09-08T15:55:11.690 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.690 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.690 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:11.691 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:14.893738+0000", 2020-09-08T15:55:11.691 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000104078 2020-09-08T15:55:11.691 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.691 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.691 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:11.692 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:14.893842+0000", 2020-09-08T15:55:11.692 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.039477225999999997 2020-09-08T15:55:11.692 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.692 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.692 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.693 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.693 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.693 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:192188 5.a e444/396 5:56f4add0:::benchmark_data_smithi166_37820_object47766:head v 444'12153, mlcod=444'12150)", 2020-09-08T15:55:11.693 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:54:14.862242+0000", 2020-09-08T15:55:11.693 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 56.686107139999997, 2020-09-08T15:55:11.693 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.031680099000000003, 2020-09-08T15:55:11.694 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:11.694 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:11.694 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:11.694 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.694 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:11.695 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:14.862242+0000", 2020-09-08T15:55:11.695 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:11.695 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.695 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.695 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:11.696 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:14.862242+0000", 2020-09-08T15:55:11.696 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.8092000000000001e-05 2020-09-08T15:55:11.696 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.696 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.696 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:11.696 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:14.862281+0000", 2020-09-08T15:55:11.697 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.7120000000000006e-06 2020-09-08T15:55:11.697 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.697 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.697 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:11.697 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:14.862289+0000", 2020-09-08T15:55:11.698 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1864e-05 2020-09-08T15:55:11.698 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.698 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.699 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:11.699 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:14.862301+0000", 2020-09-08T15:55:11.699 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.989e-06 2020-09-08T15:55:11.699 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.699 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.700 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:11.700 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:14.862306+0000", 2020-09-08T15:55:11.700 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.6977000000000001e-05 2020-09-08T15:55:11.700 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.701 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.701 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:11.701 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:14.862353+0000", 2020-09-08T15:55:11.701 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.8576999999999998e-05 2020-09-08T15:55:11.702 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.702 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.702 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:11.702 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:14.862372+0000", 2020-09-08T15:55:11.702 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.031475233999999998 2020-09-08T15:55:11.703 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.703 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.703 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:11.703 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:14.893847+0000", 2020-09-08T15:55:11.703 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.5654000000000004e-05 2020-09-08T15:55:11.703 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.704 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.704 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:11.704 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:14.893923+0000", 2020-09-08T15:55:11.704 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.031680099000000003 2020-09-08T15:55:11.704 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.705 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.705 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.705 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.705 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.705 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:217086 5.a e450/396 5:545f366f:::benchmark_data_smithi166_37820_object72664:head v 450'13658, mlcod=450'13656)", 2020-09-08T15:55:11.706 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:54:39.855391+0000", 2020-09-08T15:55:11.706 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 31.692959022, 2020-09-08T15:55:11.706 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.034913851000000003, 2020-09-08T15:55:11.706 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:11.706 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:11.707 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:11.707 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.707 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:11.707 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:39.855391+0000", 2020-09-08T15:55:11.707 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:11.707 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.708 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.708 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:11.708 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:39.855391+0000", 2020-09-08T15:55:11.708 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.9455000000000001e-05 2020-09-08T15:55:11.708 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.709 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.709 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:11.709 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:39.855430+0000", 2020-09-08T15:55:11.709 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.4810000000000003e-06 2020-09-08T15:55:11.710 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.710 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.710 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:11.710 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:39.855436+0000", 2020-09-08T15:55:11.711 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0029e-05 2020-09-08T15:55:11.711 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.711 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.711 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:11.711 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:39.855447+0000", 2020-09-08T15:55:11.711 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1229999999999999e-06 2020-09-08T15:55:11.712 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.712 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.712 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:11.712 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:39.855452+0000", 2020-09-08T15:55:11.712 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.3304999999999999e-05 2020-09-08T15:55:11.713 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.713 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.713 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:11.713 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:39.855505+0000", 2020-09-08T15:55:11.713 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.0562999999999999e-05 2020-09-08T15:55:11.714 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.714 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.714 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:11.714 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:39.855536+0000", 2020-09-08T15:55:11.714 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.034652602999999997 2020-09-08T15:55:11.715 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.715 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.715 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:11.715 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:39.890188+0000", 2020-09-08T15:55:11.715 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000116292 2020-09-08T15:55:11.715 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.716 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.716 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:11.716 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:39.890304+0000", 2020-09-08T15:55:11.716 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.034913851000000003 2020-09-08T15:55:11.716 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.717 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.717 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.717 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.717 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.717 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:217088 5.e e450/396 5:7a315c2f:::benchmark_data_smithi166_37820_object72666:head v 450'13540, mlcod=450'13538)", 2020-09-08T15:55:11.718 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:54:39.862385+0000", 2020-09-08T15:55:11.718 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 31.68596462, 2020-09-08T15:55:11.718 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.028088795999999999, 2020-09-08T15:55:11.718 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:11.718 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:11.718 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:11.719 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.719 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:11.719 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:39.862385+0000", 2020-09-08T15:55:11.719 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:11.719 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.720 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.720 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:11.720 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:39.862385+0000", 2020-09-08T15:55:11.720 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.3647999999999995e-05 2020-09-08T15:55:11.720 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.721 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.721 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:11.721 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:39.862479+0000", 2020-09-08T15:55:11.721 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1934e-05 2020-09-08T15:55:11.721 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.722 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.722 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:11.722 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:39.862491+0000", 2020-09-08T15:55:11.722 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.9524000000000001e-05 2020-09-08T15:55:11.722 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.722 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.723 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:11.723 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:39.862510+0000", 2020-09-08T15:55:11.723 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.1139999999999995e-06 2020-09-08T15:55:11.723 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.723 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.724 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:11.724 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:39.862519+0000", 2020-09-08T15:55:11.724 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9551e-05 2020-09-08T15:55:11.724 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.724 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.724 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:11.725 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:39.862569+0000", 2020-09-08T15:55:11.725 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.1339999999999999e-05 2020-09-08T15:55:11.725 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.725 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.726 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:11.727 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:39.862590+0000", 2020-09-08T15:55:11.728 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.027802753999999999 2020-09-08T15:55:11.728 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.728 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.729 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:11.729 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:39.890393+0000", 2020-09-08T15:55:11.729 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.0931000000000002e-05 2020-09-08T15:55:11.729 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.729 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.730 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:11.730 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:54:39.890474+0000", 2020-09-08T15:55:11.730 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.028088795999999999 2020-09-08T15:55:11.730 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.731 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.731 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.731 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.731 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.732 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:240954 5.a e457/396 5:5ab9ee63:::benchmark_data_smithi166_37820_object96532:head v 457'15125, mlcod=457'15121)", 2020-09-08T15:55:11.733 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:55:04.049803+0000", 2020-09-08T15:55:11.733 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 7.498546889, 2020-09-08T15:55:11.734 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.034782629000000002, 2020-09-08T15:55:11.734 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:11.734 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:11.734 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:11.734 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.735 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:11.735 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.049803+0000", 2020-09-08T15:55:11.735 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:11.735 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.735 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.736 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:11.736 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.049803+0000", 2020-09-08T15:55:11.736 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.7740999999999999e-05 2020-09-08T15:55:11.736 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.736 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.737 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:11.737 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.049890+0000", 2020-09-08T15:55:11.737 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7850000000000003e-06 2020-09-08T15:55:11.737 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.737 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.738 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:11.738 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.049900+0000", 2020-09-08T15:55:11.738 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1460999999999999e-05 2020-09-08T15:55:11.738 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.738 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.738 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:11.739 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.049912+0000", 2020-09-08T15:55:11.739 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.8379999999999996e-06 2020-09-08T15:55:11.739 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.739 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.739 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:11.740 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.049919+0000", 2020-09-08T15:55:11.740 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.5884999999999998e-05 2020-09-08T15:55:11.740 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.740 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.740 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:11.741 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.049974+0000", 2020-09-08T15:55:11.741 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.7855999999999999e-05 2020-09-08T15:55:11.741 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.741 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.741 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:11.742 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.050002+0000", 2020-09-08T15:55:11.742 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.034497321999999997 2020-09-08T15:55:11.742 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.742 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.742 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:11.743 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.084500+0000", 2020-09-08T15:55:11.743 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.5741000000000005e-05 2020-09-08T15:55:11.743 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.743 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.743 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:11.743 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.084585+0000", 2020-09-08T15:55:11.744 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.034782629000000002 2020-09-08T15:55:11.744 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.744 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.744 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.744 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.745 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.745 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:240955 5.a e457/396 5:51e219cd:::benchmark_data_smithi166_37820_object96533:head v 457'15126, mlcod=457'15121)", 2020-09-08T15:55:11.745 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:55:04.052109+0000", 2020-09-08T15:55:11.745 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 7.4962406489999998, 2020-09-08T15:55:11.745 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.032538749999999998, 2020-09-08T15:55:11.746 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:11.746 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:11.746 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:11.746 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.746 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:11.747 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.052109+0000", 2020-09-08T15:55:11.747 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:11.747 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.747 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.747 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:11.748 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.052109+0000", 2020-09-08T15:55:11.748 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2497000000000006e-05 2020-09-08T15:55:11.748 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.748 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.748 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:11.748 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.052201+0000", 2020-09-08T15:55:11.749 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4158999999999999e-05 2020-09-08T15:55:11.749 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.749 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.749 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:11.749 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.052216+0000", 2020-09-08T15:55:11.750 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7797999999999998e-05 2020-09-08T15:55:11.750 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.750 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.750 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:11.750 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.052273+0000", 2020-09-08T15:55:11.751 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.5750000000000001e-06 2020-09-08T15:55:11.751 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.751 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.751 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:11.751 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.052282+0000", 2020-09-08T15:55:11.752 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0037864679999999999 2020-09-08T15:55:11.752 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.752 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.752 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:11.752 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.056068+0000", 2020-09-08T15:55:11.752 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011475 2020-09-08T15:55:11.753 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.753 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.753 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:11.753 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.056183+0000", 2020-09-08T15:55:11.753 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.028410006000000002 2020-09-08T15:55:11.754 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.754 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.754 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:11.754 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.084593+0000", 2020-09-08T15:55:11.754 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.4496999999999997e-05 2020-09-08T15:55:11.755 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.755 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.755 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:11.756 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.084648+0000", 2020-09-08T15:55:11.756 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.032538749999999998 2020-09-08T15:55:11.756 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.756 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.756 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.756 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.757 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.757 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:240967 5.a e457/396 5:5900cb4a:::benchmark_data_smithi166_37820_object96545:head v 457'15127, mlcod=457'15123)", 2020-09-08T15:55:11.757 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:55:04.057069+0000", 2020-09-08T15:55:11.757 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 7.4912802420000002, 2020-09-08T15:55:11.757 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.027640609999999999, 2020-09-08T15:55:11.758 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:11.758 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:11.758 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:11.758 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.758 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:11.759 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.057069+0000", 2020-09-08T15:55:11.759 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:11.759 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.759 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.759 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:11.760 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.057069+0000", 2020-09-08T15:55:11.760 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9786000000000001e-05 2020-09-08T15:55:11.760 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.760 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.760 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:11.760 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.057119+0000", 2020-09-08T15:55:11.761 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.4219999999999997e-06 2020-09-08T15:55:11.761 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.761 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.761 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:11.761 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.057126+0000", 2020-09-08T15:55:11.762 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.8689999999999999e-06 2020-09-08T15:55:11.762 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.762 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.762 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:11.763 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.057135+0000", 2020-09-08T15:55:11.763 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9030000000000003e-06 2020-09-08T15:55:11.763 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.763 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.763 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:11.764 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.057140+0000", 2020-09-08T15:55:11.764 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0052460650000000003 2020-09-08T15:55:11.764 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.764 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.764 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:11.764 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.062386+0000", 2020-09-08T15:55:11.765 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.6732000000000001e-05 2020-09-08T15:55:11.765 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.765 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.766 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:11.766 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.062413+0000", 2020-09-08T15:55:11.766 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.022241489999999999 2020-09-08T15:55:11.766 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.766 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.767 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:11.767 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.084655+0000", 2020-09-08T15:55:11.767 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.5343000000000002e-05 2020-09-08T15:55:11.767 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.768 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.768 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:11.768 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.084710+0000", 2020-09-08T15:55:11.768 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.027640609999999999 2020-09-08T15:55:11.768 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.769 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.769 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.769 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.769 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.769 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:241569 5.a e458/396 5:5a9d2620:::benchmark_data_smithi166_37820_object97147:head v 458'15174, mlcod=458'15172)", 2020-09-08T15:55:11.770 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:55:04.858805+0000", 2020-09-08T15:55:11.770 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 6.6895449749999996, 2020-09-08T15:55:11.770 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.035892838000000003, 2020-09-08T15:55:11.770 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:11.770 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:11.771 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:11.771 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.771 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:11.772 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.858805+0000", 2020-09-08T15:55:11.772 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:11.772 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.772 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.772 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:11.773 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.858805+0000", 2020-09-08T15:55:11.773 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.4635999999999995e-05 2020-09-08T15:55:11.773 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.773 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.773 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:11.774 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.858879+0000", 2020-09-08T15:55:11.774 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.151e-05 2020-09-08T15:55:11.774 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.775 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.775 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:11.775 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.858891+0000", 2020-09-08T15:55:11.776 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.9879000000000001e-05 2020-09-08T15:55:11.776 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.776 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.776 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:11.776 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.858911+0000", 2020-09-08T15:55:11.776 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1117e-05 2020-09-08T15:55:11.777 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.777 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.777 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:11.777 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.858922+0000", 2020-09-08T15:55:11.777 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2942000000000006e-05 2020-09-08T15:55:11.778 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.778 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.778 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:11.778 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.859015+0000", 2020-09-08T15:55:11.778 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.7808999999999997e-05 2020-09-08T15:55:11.779 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.779 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.779 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:11.779 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.859053+0000", 2020-09-08T15:55:11.779 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.035474211999999998 2020-09-08T15:55:11.780 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.780 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.780 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:11.780 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.894527+0000", 2020-09-08T15:55:11.780 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017073300000000001 2020-09-08T15:55:11.781 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.781 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.781 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:11.781 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:04.894697+0000", 2020-09-08T15:55:11.781 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.035892838000000003 2020-09-08T15:55:11.781 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.782 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.782 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.782 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.782 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.782 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.15771.0:245344 5.a e458/396 5:5c3337ce:::benchmark_data_smithi166_37820_object100922:head v 458'15443, mlcod=458'15441)", 2020-09-08T15:55:11.783 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:55:09.862988+0000", 2020-09-08T15:55:11.783 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 1.6853616810000001, 2020-09-08T15:55:11.783 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.034594005999999997, 2020-09-08T15:55:11.783 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:11.783 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:11.784 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:11.784 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.784 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:11.784 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:09.862988+0000", 2020-09-08T15:55:11.784 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:11.785 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.785 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.785 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:11.785 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:09.862988+0000", 2020-09-08T15:55:11.786 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9848000000000001e-05 2020-09-08T15:55:11.786 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.786 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.786 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:11.786 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:09.863038+0000", 2020-09-08T15:55:11.787 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.1189999999999997e-06 2020-09-08T15:55:11.787 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.787 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.787 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:11.787 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:09.863046+0000", 2020-09-08T15:55:11.788 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.184e-05 2020-09-08T15:55:11.788 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.788 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.788 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:11.788 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:09.863058+0000", 2020-09-08T15:55:11.788 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8289999999999996e-06 2020-09-08T15:55:11.789 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.789 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.789 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:11.789 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:09.863064+0000", 2020-09-08T15:55:11.789 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.2558000000000001e-05 2020-09-08T15:55:11.790 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.790 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.790 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:11.790 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:09.863116+0000", 2020-09-08T15:55:11.790 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.6713999999999999e-05 2020-09-08T15:55:11.791 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.791 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.791 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:11.791 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:09.863143+0000", 2020-09-08T15:55:11.791 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.034253081999999997 2020-09-08T15:55:11.792 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.792 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.792 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:11.792 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:09.897396+0000", 2020-09-08T15:55:11.792 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000186016 2020-09-08T15:55:11.793 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:11.793 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:11.793 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:11.793 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:09.897582+0000", 2020-09-08T15:55:11.793 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.034594005999999997 2020-09-08T15:55:11.794 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.794 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.794 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.794 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:11.794 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:11.794 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:55:11.796 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:55:11.796 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:55:11.796 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:55:11.796 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:55:11.797 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:55:11.797 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:55:11.805 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:55:11.836 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:55:11.839 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:55:11.840 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:55:11.840 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:55:11.840 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:55:11.840 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:55:11.840 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:55:11.841 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:55:11.841 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:55:11.841 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:55:11.841 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:55:11.842 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:55:11.842 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:55:11.843 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:55:12.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:11 smithi166 bash[9446]: audit 2020-09-08T15:55:11.351180+0000 mon.a (mon.0) 1653 : audit [DBG] from='client.? v1:172.21.15.104:0/3263161561' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:55:12.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:11 smithi166 bash[9446]: audit 2020-09-08T15:55:11.684168+0000 mon.a (mon.0) 1654 : audit [INF] from='client.? v1:172.21.15.104:0/484479669' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:55:12.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:11 smithi055 bash[9296]: audit 2020-09-08T15:55:11.351180+0000 mon.a (mon.0) 1653 : audit [DBG] from='client.? v1:172.21.15.104:0/3263161561' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:55:12.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:11 smithi055 bash[9296]: audit 2020-09-08T15:55:11.684168+0000 mon.a (mon.0) 1654 : audit [INF] from='client.? v1:172.21.15.104:0/484479669' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:55:12.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:11 smithi104 bash[9203]: audit 2020-09-08T15:55:11.351180+0000 mon.a (mon.0) 1653 : audit [DBG] from='client.? v1:172.21.15.104:0/3263161561' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:55:12.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:11 smithi104 bash[9203]: audit 2020-09-08T15:55:11.684168+0000 mon.a (mon.0) 1654 : audit [INF] from='client.? v1:172.21.15.104:0/484479669' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:55:12.577 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:12.592 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_ops_in_flight 2020-09-08T15:55:12.681 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:12.741 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:55:12.827 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:12.833 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:55:12 smithi055 bash[16646]: debug 2020-09-08T15:55:12.577+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:12.834 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:12 smithi055 bash[9296]: cluster 2020-09-08T15:55:11.833903+0000 mon.a (mon.0) 1655 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:55:12.834 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:12 smithi055 bash[9296]: audit 2020-09-08T15:55:11.846713+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? v1:172.21.15.104:0/484479669' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:55:12.835 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:12 smithi055 bash[9296]: cluster 2020-09-08T15:55:11.846773+0000 mon.a (mon.0) 1657 : cluster [DBG] osdmap e459: 12 total, 12 up, 12 in 2020-09-08T15:55:12.835 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:12 smithi055 bash[9296]: cluster 2020-09-08T15:55:11.869841+0000 mgr.x (mgr.24118) 836 : cluster [DBG] pgmap v1211: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 621 op/s 2020-09-08T15:55:12.835 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:12 smithi055 bash[9296]: cluster 2020-09-08T15:55:12.100368+0000 osd.7 (osd.7) 91 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:12.835 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:12 smithi055 bash[9296]: cluster 2020-09-08T15:55:12.100457+0000 osd.7 (osd.7) 92 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:12.848 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:12 smithi166 bash[9446]: cluster 2020-09-08T15:55:11.833903+0000 mon.a (mon.0) 1655 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:55:12.848 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:12 smithi166 bash[9446]: audit 2020-09-08T15:55:11.846713+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? v1:172.21.15.104:0/484479669' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:55:12.849 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:12 smithi166 bash[9446]: cluster 2020-09-08T15:55:11.846773+0000 mon.a (mon.0) 1657 : cluster [DBG] osdmap e459: 12 total, 12 up, 12 in 2020-09-08T15:55:12.849 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:12 smithi166 bash[9446]: cluster 2020-09-08T15:55:11.869841+0000 mgr.x (mgr.24118) 836 : cluster [DBG] pgmap v1211: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 621 op/s 2020-09-08T15:55:12.849 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:12 smithi166 bash[9446]: cluster 2020-09-08T15:55:12.100368+0000 osd.7 (osd.7) 91 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:12.849 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:12 smithi166 bash[9446]: cluster 2020-09-08T15:55:12.100457+0000 osd.7 (osd.7) 92 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:12.929 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:13.043 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:13.117 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:55:12 smithi166 bash[18960]: debug 2020-09-08T15:55:12.826+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:13.147 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:13.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:55:13 smithi055 bash[16646]: debug 2020-09-08T15:55:13.045+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:13.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:12 smithi104 bash[9203]: cluster 2020-09-08T15:55:11.833903+0000 mon.a (mon.0) 1655 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:55:13.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:12 smithi104 bash[9203]: audit 2020-09-08T15:55:11.846713+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? v1:172.21.15.104:0/484479669' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:55:13.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:12 smithi104 bash[9203]: cluster 2020-09-08T15:55:11.846773+0000 mon.a (mon.0) 1657 : cluster [DBG] osdmap e459: 12 total, 12 up, 12 in 2020-09-08T15:55:13.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:12 smithi104 bash[9203]: cluster 2020-09-08T15:55:11.869841+0000 mgr.x (mgr.24118) 836 : cluster [DBG] pgmap v1211: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 621 op/s 2020-09-08T15:55:13.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:12 smithi104 bash[9203]: cluster 2020-09-08T15:55:12.100368+0000 osd.7 (osd.7) 91 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:13.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:12 smithi104 bash[9203]: cluster 2020-09-08T15:55:12.100457+0000 osd.7 (osd.7) 92 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:13.260 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:13.370 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:13.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:55:13 smithi055 bash[16646]: debug 2020-09-08T15:55:13.257+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:13.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:13 smithi104 bash[9203]: cluster 2020-09-08T15:55:12.851185+0000 mon.a (mon.0) 1658 : cluster [DBG] osdmap e460: 12 total, 12 up, 12 in 2020-09-08T15:55:13.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:13 smithi104 bash[9203]: cluster 2020-09-08T15:55:13.544822+0000 osd.0 (osd.0) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:13.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:13 smithi104 bash[9203]: cluster 2020-09-08T15:55:13.544927+0000 osd.0 (osd.0) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:14.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:13 smithi166 bash[9446]: cluster 2020-09-08T15:55:12.851185+0000 mon.a (mon.0) 1658 : cluster [DBG] osdmap e460: 12 total, 12 up, 12 in 2020-09-08T15:55:14.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:13 smithi166 bash[9446]: cluster 2020-09-08T15:55:13.544822+0000 osd.0 (osd.0) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:14.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:13 smithi166 bash[9446]: cluster 2020-09-08T15:55:13.544927+0000 osd.0 (osd.0) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:14.122 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:13 smithi055 bash[9296]: cluster 2020-09-08T15:55:12.851185+0000 mon.a (mon.0) 1658 : cluster [DBG] osdmap e460: 12 total, 12 up, 12 in 2020-09-08T15:55:14.122 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:13 smithi055 bash[9296]: cluster 2020-09-08T15:55:13.544822+0000 osd.0 (osd.0) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:14.123 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:13 smithi055 bash[9296]: cluster 2020-09-08T15:55:13.544927+0000 osd.0 (osd.0) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:14.124 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:14.229 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:14.378 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:14.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:55:14 smithi055 bash[19003]: debug 2020-09-08T15:55:14.125+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:14.486 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:14.498 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:55:14 smithi104 bash[19922]: debug 2020-09-08T15:55:14.380+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:14.618 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:14.725 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:14.860 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:14 smithi104 bash[9203]: cluster 2020-09-08T15:55:13.860876+0000 mon.a (mon.0) 1659 : cluster [DBG] osdmap e461: 12 total, 12 up, 12 in 2020-09-08T15:55:14.861 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:14 smithi104 bash[9203]: cluster 2020-09-08T15:55:13.870452+0000 mgr.x (mgr.24118) 837 : cluster [DBG] pgmap v1214: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 777 op/s 2020-09-08T15:55:14.861 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:55:14 smithi104 bash[15273]: debug 2020-09-08T15:55:14.620+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:15.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:14 smithi166 bash[9446]: cluster 2020-09-08T15:55:13.860876+0000 mon.a (mon.0) 1659 : cluster [DBG] osdmap e461: 12 total, 12 up, 12 in 2020-09-08T15:55:15.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:14 smithi166 bash[9446]: cluster 2020-09-08T15:55:13.870452+0000 mgr.x (mgr.24118) 837 : cluster [DBG] pgmap v1214: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 777 op/s 2020-09-08T15:55:15.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:14 smithi055 bash[9296]: cluster 2020-09-08T15:55:13.860876+0000 mon.a (mon.0) 1659 : cluster [DBG] osdmap e461: 12 total, 12 up, 12 in 2020-09-08T15:55:15.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:14 smithi055 bash[9296]: cluster 2020-09-08T15:55:13.870452+0000 mgr.x (mgr.24118) 837 : cluster [DBG] pgmap v1214: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 777 op/s 2020-09-08T15:55:15.160 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:15.264 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:55:15.407 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:55:15.507 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:55:15.617 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:55:15 smithi166 bash[18960]: debug 2020-09-08T15:55:15.158+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:15.632 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:55:15.636 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:55:15 smithi055 bash[11956]: debug 2020-09-08T15:55:15.401+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:15.636 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:55:15 smithi055 bash[11956]: debug 2020-09-08T15:55:15.641+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:15.637 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:55:15.715 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:55:15.740 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:55:15.798 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:55:15.874 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:55:16.044 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:55:16.149 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:55:16.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:55:16 smithi104 bash[17588]: debug 2020-09-08T15:55:16.040+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:16.725 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:55:16.726 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:55:16.790 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:55:16.895 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:55:17.069 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:16 smithi104 bash[9203]: cluster 2020-09-08T15:55:15.871066+0000 mgr.x (mgr.24118) 838 : cluster [DBG] pgmap v1215: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 685 op/s 2020-09-08T15:55:17.070 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:55:16 smithi104 bash[17588]: debug 2020-09-08T15:55:16.788+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:17.070 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:55:17.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:16 smithi166 bash[9446]: cluster 2020-09-08T15:55:15.871066+0000 mgr.x (mgr.24118) 838 : cluster [DBG] pgmap v1215: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 685 op/s 2020-09-08T15:55:17.175 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:17.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:16 smithi055 bash[9296]: cluster 2020-09-08T15:55:15.871066+0000 mgr.x (mgr.24118) 838 : cluster [DBG] pgmap v1215: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 685 op/s 2020-09-08T15:55:17.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:55:17 smithi104 bash[17588]: debug 2020-09-08T15:55:17.072+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:17.512 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:17.615 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:17.638 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_blocked_ops 2020-09-08T15:55:17.723 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:17.828 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:17.831 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:55:17.867 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:55:17 smithi166 bash[16593]: debug 2020-09-08T15:55:17.722+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:17.868 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:55:17 smithi166 bash[18960]: debug 2020-09-08T15:55:17.514+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:17.964 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:18.070 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:18.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:55:17 smithi055 bash[16646]: debug 2020-09-08T15:55:17.965+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:18.459 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:18.563 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:18.868 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:55:18 smithi166 bash[18960]: debug 2020-09-08T15:55:18.458+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:18.961 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:19.064 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:19.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:18 smithi104 bash[9203]: cluster 2020-09-08T15:55:17.871536+0000 mgr.x (mgr.24118) 839 : cluster [DBG] pgmap v1216: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 959 op/s 2020-09-08T15:55:19.253 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:18 smithi166 bash[9446]: cluster 2020-09-08T15:55:17.871536+0000 mgr.x (mgr.24118) 839 : cluster [DBG] pgmap v1216: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 959 op/s 2020-09-08T15:55:19.254 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:55:18 smithi166 bash[11928]: debug 2020-09-08T15:55:18.958+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:19.262 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:19.372 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:19.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:18 smithi055 bash[9296]: cluster 2020-09-08T15:55:17.871536+0000 mgr.x (mgr.24118) 839 : cluster [DBG] pgmap v1216: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 959 op/s 2020-09-08T15:55:19.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:55:19 smithi166 bash[16593]: debug 2020-09-08T15:55:19.254+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:19.950 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:20.055 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:20.187 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:20.293 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:20.425 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:55:19 smithi104 bash[19922]: debug 2020-09-08T15:55:19.952+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:20.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:55:20 smithi104 bash[22305]: debug 2020-09-08T15:55:20.192+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:20.549 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:20.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:55:20 smithi166 bash[18960]: debug 2020-09-08T15:55:20.550+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:20.654 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:20.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:20 smithi104 bash[9203]: cluster 2020-09-08T15:55:19.871974+0000 mgr.x (mgr.24118) 840 : cluster [DBG] pgmap v1217: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 872 op/s 2020-09-08T15:55:20.956 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:21.060 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:21.078 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:20 smithi055 bash[9296]: cluster 2020-09-08T15:55:19.871974+0000 mgr.x (mgr.24118) 840 : cluster [DBG] pgmap v1217: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 872 op/s 2020-09-08T15:55:21.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:20 smithi166 bash[9446]: cluster 2020-09-08T15:55:19.871974+0000 mgr.x (mgr.24118) 840 : cluster [DBG] pgmap v1217: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 872 op/s 2020-09-08T15:55:21.212 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:21.316 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:21.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:55:21 smithi055 bash[19003]: debug 2020-09-08T15:55:21.213+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:21.425 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:55:20 smithi104 bash[15273]: debug 2020-09-08T15:55:20.956+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:21.685 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:21.789 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:21.923 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:21.978 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:21 smithi166 bash[9446]: cluster 2020-09-08T15:55:21.386507+0000 osd.5 (osd.5) 63 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:21.979 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:55:21 smithi166 bash[11928]: debug 2020-09-08T15:55:21.686+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:21.979 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:55:21 smithi166 bash[16593]: debug 2020-09-08T15:55:21.922+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:22.030 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:55:22.132 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:21 smithi104 bash[9203]: cluster 2020-09-08T15:55:21.386507+0000 osd.5 (osd.5) 63 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:22.149 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:21 smithi055 bash[9296]: cluster 2020-09-08T15:55:21.386507+0000 osd.5 (osd.5) 63 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:22.150 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:55:22.217 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:55:22.218 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:55:22.254 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:22.395 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:55:22.404 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:22.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:55:22 smithi055 bash[14301]: debug 2020-09-08T15:55:22.153+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:22.438 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:55:22.469 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:55:22.499 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:55:22.508 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:22.546 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:55:22.586 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:55:22.622 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:55:22.670 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:55:22.674 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:22.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:55:22 smithi104 bash[19922]: debug 2020-09-08T15:55:22.407+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:22.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:55:22 smithi104 bash[19922]: debug 2020-09-08T15:55:22.679+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:22.719 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:55:22.756 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:55:22.779 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:55:22.798 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:55:22.828 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:55:22.834 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:55:22.834 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:55:22.835 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:55:22.835 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:55:22.835 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:55:22.835 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:55:22.835 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:55:22.836 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:55:22.836 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:55:22.836 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:55:22.836 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:55:22.836 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:55:22.858 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:55:22.906 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:55:22.942 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:22 smithi104 bash[9203]: cluster 2020-09-08T15:55:21.837711+0000 mon.a (mon.0) 1660 : cluster [DBG] osdmap e462: 12 total, 12 up, 12 in 2020-09-08T15:55:22.943 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:22 smithi104 bash[9203]: cluster 2020-09-08T15:55:21.872458+0000 mgr.x (mgr.24118) 841 : cluster [DBG] pgmap v1219: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 552 op/s 2020-09-08T15:55:22.943 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:22 smithi104 bash[9203]: audit 2020-09-08T15:55:22.359202+0000 mon.a (mon.0) 1661 : audit [DBG] from='client.? v1:172.21.15.104:0/1645898386' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:55:22.943 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:22 smithi104 bash[9203]: audit 2020-09-08T15:55:22.726305+0000 mon.c (mon.2) 67 : audit [INF] from='client.? v1:172.21.15.104:0/3770995889' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:55:22.944 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:22 smithi104 bash[9203]: audit 2020-09-08T15:55:22.729376+0000 mon.a (mon.0) 1662 : audit [INF] from='client.35804 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:55:23.010 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:23.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:22 smithi166 bash[9446]: cluster 2020-09-08T15:55:21.837711+0000 mon.a (mon.0) 1660 : cluster [DBG] osdmap e462: 12 total, 12 up, 12 in 2020-09-08T15:55:23.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:22 smithi166 bash[9446]: cluster 2020-09-08T15:55:21.872458+0000 mgr.x (mgr.24118) 841 : cluster [DBG] pgmap v1219: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 552 op/s 2020-09-08T15:55:23.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:22 smithi166 bash[9446]: audit 2020-09-08T15:55:22.359202+0000 mon.a (mon.0) 1661 : audit [DBG] from='client.? v1:172.21.15.104:0/1645898386' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:55:23.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:22 smithi166 bash[9446]: audit 2020-09-08T15:55:22.726305+0000 mon.c (mon.2) 67 : audit [INF] from='client.? v1:172.21.15.104:0/3770995889' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:55:23.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:22 smithi166 bash[9446]: audit 2020-09-08T15:55:22.729376+0000 mon.a (mon.0) 1662 : audit [INF] from='client.35804 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:55:23.156 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:55:22 smithi055 bash[14301]: debug 2020-09-08T15:55:22.909+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:22 smithi055 bash[9296]: cluster 2020-09-08T15:55:21.837711+0000 mon.a (mon.0) 1660 : cluster [DBG] osdmap e462: 12 total, 12 up, 12 in 2020-09-08T15:55:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:22 smithi055 bash[9296]: cluster 2020-09-08T15:55:21.872458+0000 mgr.x (mgr.24118) 841 : cluster [DBG] pgmap v1219: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 552 op/s 2020-09-08T15:55:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:22 smithi055 bash[9296]: audit 2020-09-08T15:55:22.359202+0000 mon.a (mon.0) 1661 : audit [DBG] from='client.? v1:172.21.15.104:0/1645898386' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:55:23.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:22 smithi055 bash[9296]: audit 2020-09-08T15:55:22.726305+0000 mon.c (mon.2) 67 : audit [INF] from='client.? v1:172.21.15.104:0/3770995889' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:55:23.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:22 smithi055 bash[9296]: audit 2020-09-08T15:55:22.729376+0000 mon.a (mon.0) 1662 : audit [INF] from='client.35804 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:55:23.172 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:23.229 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_historic_ops 2020-09-08T15:55:23.277 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:55:23.377 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:55:23.398 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:55:23.504 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:23.617 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:55:23 smithi166 bash[18960]: debug 2020-09-08T15:55:23.170+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:23.683 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:55:23 smithi055 bash[11956]: debug 2020-09-08T15:55:23.401+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:23.849 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:23 smithi104 bash[9203]: cluster 2020-09-08T15:55:22.841554+0000 mon.a (mon.0) 1663 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:55:23.849 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:23 smithi104 bash[9203]: audit 2020-09-08T15:55:22.856612+0000 mon.a (mon.0) 1664 : audit [INF] from='client.35804 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:55:23.849 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:23 smithi104 bash[9203]: cluster 2020-09-08T15:55:22.856659+0000 mon.a (mon.0) 1665 : cluster [DBG] osdmap e463: 12 total, 12 up, 12 in 2020-09-08T15:55:23.850 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:23 smithi104 bash[9203]: cluster 2020-09-08T15:55:23.020617+0000 osd.10 (osd.10) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:23.850 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:23 smithi104 bash[9203]: cluster 2020-09-08T15:55:23.020695+0000 osd.10 (osd.10) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:24.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:23 smithi166 bash[9446]: cluster 2020-09-08T15:55:22.841554+0000 mon.a (mon.0) 1663 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:55:24.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:23 smithi166 bash[9446]: audit 2020-09-08T15:55:22.856612+0000 mon.a (mon.0) 1664 : audit [INF] from='client.35804 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:55:24.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:23 smithi166 bash[9446]: cluster 2020-09-08T15:55:22.856659+0000 mon.a (mon.0) 1665 : cluster [DBG] osdmap e463: 12 total, 12 up, 12 in 2020-09-08T15:55:24.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:23 smithi166 bash[9446]: cluster 2020-09-08T15:55:23.020617+0000 osd.10 (osd.10) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:24.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:23 smithi166 bash[9446]: cluster 2020-09-08T15:55:23.020695+0000 osd.10 (osd.10) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:24.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:23 smithi055 bash[9296]: cluster 2020-09-08T15:55:22.841554+0000 mon.a (mon.0) 1663 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:55:24.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:23 smithi055 bash[9296]: audit 2020-09-08T15:55:22.856612+0000 mon.a (mon.0) 1664 : audit [INF] from='client.35804 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:55:24.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:23 smithi055 bash[9296]: cluster 2020-09-08T15:55:22.856659+0000 mon.a (mon.0) 1665 : cluster [DBG] osdmap e463: 12 total, 12 up, 12 in 2020-09-08T15:55:24.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:23 smithi055 bash[9296]: cluster 2020-09-08T15:55:23.020617+0000 osd.10 (osd.10) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:24.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:23 smithi055 bash[9296]: cluster 2020-09-08T15:55:23.020695+0000 osd.10 (osd.10) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:24.496 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:24.601 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:24.751 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:55:24 smithi104 bash[15273]: debug 2020-09-08T15:55:24.499+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:24.798 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:24.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:55:24 smithi166 bash[18960]: debug 2020-09-08T15:55:24.798+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:24.901 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:55:25.044 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:55:25.149 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:55:25.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:55:25 smithi055 bash[11956]: debug 2020-09-08T15:55:25.045+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:25.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:24 smithi055 bash[9296]: cluster 2020-09-08T15:55:23.863921+0000 mon.a (mon.0) 1666 : cluster [DBG] osdmap e464: 12 total, 12 up, 12 in 2020-09-08T15:55:25.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:24 smithi055 bash[9296]: cluster 2020-09-08T15:55:23.872866+0000 mgr.x (mgr.24118) 842 : cluster [DBG] pgmap v1222: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 797 op/s 2020-09-08T15:55:25.171 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:24 smithi166 bash[9446]: cluster 2020-09-08T15:55:23.863921+0000 mon.a (mon.0) 1666 : cluster [DBG] osdmap e464: 12 total, 12 up, 12 in 2020-09-08T15:55:25.172 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:24 smithi166 bash[9446]: cluster 2020-09-08T15:55:23.872866+0000 mgr.x (mgr.24118) 842 : cluster [DBG] pgmap v1222: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 797 op/s 2020-09-08T15:55:25.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:24 smithi104 bash[9203]: cluster 2020-09-08T15:55:23.863921+0000 mon.a (mon.0) 1666 : cluster [DBG] osdmap e464: 12 total, 12 up, 12 in 2020-09-08T15:55:25.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:24 smithi104 bash[9203]: cluster 2020-09-08T15:55:23.872866+0000 mgr.x (mgr.24118) 842 : cluster [DBG] pgmap v1222: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 797 op/s 2020-09-08T15:55:25.380 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:55:25.484 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:25.591 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:25.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:55:25 smithi166 bash[14254]: debug 2020-09-08T15:55:25.377+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:25.702 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:25.832 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:55:25 smithi055 bash[19003]: debug 2020-09-08T15:55:25.593+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:25.833 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:25.936 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:55:26.052 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:55:26.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:55:25 smithi055 bash[19003]: debug 2020-09-08T15:55:25.837+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:26.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:55:26 smithi055 bash[14301]: debug 2020-09-08T15:55:26.053+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:26.159 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:55:26.288 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:55:26.394 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:26.595 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:26.618 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:55:26 smithi166 bash[14254]: debug 2020-09-08T15:55:26.285+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:26.675 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:55:26 smithi104 bash[15273]: debug 2020-09-08T15:55:26.599+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:26.699 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:26.709 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:55:26.753 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:55:26.888 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:55:26.925 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:26.939 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:55:27.028 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:27.069 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:26 smithi104 bash[9203]: cluster 2020-09-08T15:55:25.873301+0000 mgr.x (mgr.24118) 843 : cluster [DBG] pgmap v1223: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 727 op/s 2020-09-08T15:55:27.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:26 smithi166 bash[9446]: cluster 2020-09-08T15:55:25.873301+0000 mgr.x (mgr.24118) 843 : cluster [DBG] pgmap v1223: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 727 op/s 2020-09-08T15:55:27.118 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:55:26 smithi166 bash[16593]: debug 2020-09-08T15:55:26.921+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:27.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:26 smithi055 bash[9296]: cluster 2020-09-08T15:55:25.873301+0000 mgr.x (mgr.24118) 843 : cluster [DBG] pgmap v1223: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 727 op/s 2020-09-08T15:55:28.028 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:55:28.028 INFO:teuthology.orchestra.run.smithi166.stdout: "size": 20, 2020-09-08T15:55:28.029 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 600, 2020-09-08T15:55:28.029 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T15:55:28.029 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.029 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:80895 4.6 4:61551036:::benchmark_data_smithi166_32054_object80894:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e332)", 2020-09-08T15:55:28.029 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:00.131140+0000", 2020-09-08T15:55:28.030 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 447.89740674199999, 2020-09-08T15:55:28.030 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19727386199999999, 2020-09-08T15:55:28.030 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:28.030 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:28.031 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:28.031 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:55:28.031 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:55:28.031 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 80895 2020-09-08T15:55:28.031 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.032 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:28.032 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.032 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:28.032 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.131140+0000", 2020-09-08T15:55:28.032 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:28.032 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.033 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.033 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:28.033 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.131140+0000", 2020-09-08T15:55:28.033 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999642 2020-09-08T15:55:28.033 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.034 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.034 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:28.034 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.131105+0000", 2020-09-08T15:55:28.034 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000120266 2020-09-08T15:55:28.034 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.035 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.035 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:28.035 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.131225+0000", 2020-09-08T15:55:28.035 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7156e-05 2020-09-08T15:55:28.035 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.036 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.036 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:28.036 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.131242+0000", 2020-09-08T15:55:28.036 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8359999999999997e-06 2020-09-08T15:55:28.036 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.037 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.037 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:28.037 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.131248+0000", 2020-09-08T15:55:28.037 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00218724 2020-09-08T15:55:28.037 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.038 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.038 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:28.038 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.133435+0000", 2020-09-08T15:55:28.038 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000121471 2020-09-08T15:55:28.038 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.038 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.039 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:28.039 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.133556+0000", 2020-09-08T15:55:28.039 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000168309 2020-09-08T15:55:28.039 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.039 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.040 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T15:55:28.040 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.133725+0000", 2020-09-08T15:55:28.040 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0025498769999999999 2020-09-08T15:55:28.040 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.040 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.041 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:55:28.041 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.136275+0000", 2020-09-08T15:55:28.041 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19181920299999999 2020-09-08T15:55:28.041 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.041 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.043 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:55:28.043 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.328094+0000", 2020-09-08T15:55:28.043 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000190719 2020-09-08T15:55:28.043 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.043 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.044 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:28.044 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.328285+0000", 2020-09-08T15:55:28.044 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012947999999999999 2020-09-08T15:55:28.044 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.044 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.045 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:28.046 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.328414+0000", 2020-09-08T15:55:28.047 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19727386199999999 2020-09-08T15:55:28.047 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.047 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.047 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.047 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.048 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.048 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:80905 4.6 4:6ab9f432:::benchmark_data_smithi166_32054_object80904:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e332)", 2020-09-08T15:55:28.048 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:00.137162+0000", 2020-09-08T15:55:28.048 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 447.891385012, 2020-09-08T15:55:28.048 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19165459300000001, 2020-09-08T15:55:28.049 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:28.049 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:28.049 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:28.049 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:55:28.049 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:55:28.050 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 80905 2020-09-08T15:55:28.050 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.050 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:28.050 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.050 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:28.051 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.137162+0000", 2020-09-08T15:55:28.051 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:28.051 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.051 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.051 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:28.052 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.137162+0000", 2020-09-08T15:55:28.052 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999638 2020-09-08T15:55:28.052 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.052 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.052 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:28.052 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.137126+0000", 2020-09-08T15:55:28.053 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000118009 2020-09-08T15:55:28.053 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.053 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.053 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:28.054 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.137244+0000", 2020-09-08T15:55:28.054 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.5429999999999999e-06 2020-09-08T15:55:28.055 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.055 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.055 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:28.055 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.137253+0000", 2020-09-08T15:55:28.055 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.6700000000000002e-06 2020-09-08T15:55:28.056 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.056 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.056 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:28.056 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.137258+0000", 2020-09-08T15:55:28.056 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.6660999999999999e-05 2020-09-08T15:55:28.057 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.057 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.057 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:28.057 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.137305+0000", 2020-09-08T15:55:28.057 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014171100000000001 2020-09-08T15:55:28.058 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.058 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.058 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:28.058 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.137446+0000", 2020-09-08T15:55:28.058 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000143654 2020-09-08T15:55:28.059 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.059 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.059 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T15:55:28.059 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.137590+0000", 2020-09-08T15:55:28.059 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.002511391 2020-09-08T15:55:28.060 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.060 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.060 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:55:28.060 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.140101+0000", 2020-09-08T15:55:28.060 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.188392064 2020-09-08T15:55:28.060 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.061 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.061 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:55:28.061 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.328494+0000", 2020-09-08T15:55:28.061 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016733899999999999 2020-09-08T15:55:28.064 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.064 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.064 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:28.065 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.328661+0000", 2020-09-08T15:55:28.065 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000155673 2020-09-08T15:55:28.065 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.065 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.065 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:28.066 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:00.328817+0000", 2020-09-08T15:55:28.066 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19165459300000001 2020-09-08T15:55:28.066 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.066 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.066 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.067 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.067 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.067 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:105392 4.2 4:46272ea7:::benchmark_data_smithi166_32054_object105391:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e339)", 2020-09-08T15:55:28.067 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:17.825550+0000", 2020-09-08T15:55:28.067 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 430.20299731, 2020-09-08T15:55:28.068 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22101335599999999, 2020-09-08T15:55:28.068 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:28.068 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:28.068 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:28.068 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:55:28.069 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:55:28.069 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 105392 2020-09-08T15:55:28.069 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.069 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:28.069 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.070 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:28.070 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.825550+0000", 2020-09-08T15:55:28.070 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:28.070 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.070 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.070 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:28.071 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.825550+0000", 2020-09-08T15:55:28.071 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999471 2020-09-08T15:55:28.071 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.071 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.071 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:28.072 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.825497+0000", 2020-09-08T15:55:28.072 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000195761 2020-09-08T15:55:28.072 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.072 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.072 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:28.073 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.825692+0000", 2020-09-08T15:55:28.073 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.5179999999999999e-05 2020-09-08T15:55:28.073 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.073 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.073 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:28.074 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.825717+0000", 2020-09-08T15:55:28.074 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3782e-05 2020-09-08T15:55:28.075 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.075 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.075 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:28.075 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.825731+0000", 2020-09-08T15:55:28.075 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8343000000000004e-05 2020-09-08T15:55:28.076 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.076 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.076 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:28.076 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.825820+0000", 2020-09-08T15:55:28.076 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018716499999999999 2020-09-08T15:55:28.076 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.077 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.077 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:28.077 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.826007+0000", 2020-09-08T15:55:28.077 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015770500000000001 2020-09-08T15:55:28.077 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.078 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.078 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T15:55:28.078 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.826164+0000", 2020-09-08T15:55:28.078 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0036108350000000002 2020-09-08T15:55:28.078 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.079 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.079 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:55:28.079 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.829775+0000", 2020-09-08T15:55:28.079 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.216637368 2020-09-08T15:55:28.079 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.080 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.080 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:55:28.080 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.046413+0000", 2020-09-08T15:55:28.080 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.3028000000000005e-05 2020-09-08T15:55:28.080 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.081 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.081 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:28.081 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.046496+0000", 2020-09-08T15:55:28.081 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.7298999999999994e-05 2020-09-08T15:55:28.081 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.081 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.082 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:28.082 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.046563+0000", 2020-09-08T15:55:28.082 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22101335599999999 2020-09-08T15:55:28.082 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.083 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.083 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.083 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.083 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.083 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:105410 4.2 4:447ae90f:::benchmark_data_smithi166_32054_object105409:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e339)", 2020-09-08T15:55:28.083 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:17.840593+0000", 2020-09-08T15:55:28.084 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 430.18795386699998, 2020-09-08T15:55:28.084 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21007357300000001, 2020-09-08T15:55:28.084 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:28.084 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:28.085 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:28.085 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:55:28.085 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:55:28.085 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 105410 2020-09-08T15:55:28.086 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.086 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:28.086 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.086 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:28.086 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.840593+0000", 2020-09-08T15:55:28.087 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:28.087 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.087 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.087 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:28.088 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.840593+0000", 2020-09-08T15:55:28.088 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999776 2020-09-08T15:55:28.088 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.088 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.088 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:28.089 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.840571+0000", 2020-09-08T15:55:28.089 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7286999999999994e-05 2020-09-08T15:55:28.089 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.089 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.089 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:28.090 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.840648+0000", 2020-09-08T15:55:28.090 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3988999999999999e-05 2020-09-08T15:55:28.090 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.090 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.090 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:28.091 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.840662+0000", 2020-09-08T15:55:28.091 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.8760000000000004e-06 2020-09-08T15:55:28.091 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.091 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.091 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:28.092 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.840667+0000", 2020-09-08T15:55:28.092 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0359999999999999e-05 2020-09-08T15:55:28.092 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.092 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.092 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:28.092 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.840717+0000", 2020-09-08T15:55:28.093 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000119008 2020-09-08T15:55:28.093 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.093 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.093 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:28.093 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.840836+0000", 2020-09-08T15:55:28.094 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018360699999999999 2020-09-08T15:55:28.094 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.094 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.095 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T15:55:28.095 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.841020+0000", 2020-09-08T15:55:28.095 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0075218459999999996 2020-09-08T15:55:28.095 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.095 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.096 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:55:28.096 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.848541+0000", 2020-09-08T15:55:28.096 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20198524300000001 2020-09-08T15:55:28.096 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.096 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.096 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:55:28.097 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.050527+0000", 2020-09-08T15:55:28.097 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.8275000000000005e-05 2020-09-08T15:55:28.097 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.097 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.097 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:28.098 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.050605+0000", 2020-09-08T15:55:28.098 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.1641000000000002e-05 2020-09-08T15:55:28.098 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.098 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.098 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:28.099 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.050667+0000", 2020-09-08T15:55:28.099 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21007357300000001 2020-09-08T15:55:28.099 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.099 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.099 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.100 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.100 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.100 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:105411 4.2 4:467d7578:::benchmark_data_smithi166_32054_object105410:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e339)", 2020-09-08T15:55:28.100 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:17.842458+0000", 2020-09-08T15:55:28.100 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 430.186088977, 2020-09-08T15:55:28.101 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20848314300000001, 2020-09-08T15:55:28.101 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:28.101 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:28.101 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:28.101 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:55:28.101 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:55:28.102 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 105411 2020-09-08T15:55:28.102 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.102 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:28.102 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.102 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:28.103 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.842458+0000", 2020-09-08T15:55:28.103 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:28.103 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.103 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.103 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:28.104 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.842458+0000", 2020-09-08T15:55:28.104 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999642 2020-09-08T15:55:28.104 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.104 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.104 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:28.105 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.842422+0000", 2020-09-08T15:55:28.105 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000188152 2020-09-08T15:55:28.105 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.105 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.105 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:28.106 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.842610+0000", 2020-09-08T15:55:28.106 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6612999999999998e-05 2020-09-08T15:55:28.106 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.106 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.106 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:28.106 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.842627+0000", 2020-09-08T15:55:28.107 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.952e-06 2020-09-08T15:55:28.107 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.107 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.107 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:28.107 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.842636+0000", 2020-09-08T15:55:28.108 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.001858113 2020-09-08T15:55:28.108 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.108 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.108 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:28.108 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.844494+0000", 2020-09-08T15:55:28.109 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018164099999999999 2020-09-08T15:55:28.109 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.109 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.109 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:28.109 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.844676+0000", 2020-09-08T15:55:28.110 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000207813 2020-09-08T15:55:28.110 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.111 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.111 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T15:55:28.111 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.844884+0000", 2020-09-08T15:55:28.111 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.005358565 2020-09-08T15:55:28.111 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.112 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.112 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:55:28.112 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.850242+0000", 2020-09-08T15:55:28.112 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20050157299999999 2020-09-08T15:55:28.113 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.113 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.113 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:55:28.113 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.050744+0000", 2020-09-08T15:55:28.113 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000109436 2020-09-08T15:55:28.114 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.114 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.114 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:28.114 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.050853+0000", 2020-09-08T15:55:28.114 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.7960000000000005e-05 2020-09-08T15:55:28.115 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.115 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.115 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:28.115 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.050941+0000", 2020-09-08T15:55:28.115 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20848314300000001 2020-09-08T15:55:28.116 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.116 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.116 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.117 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.117 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.117 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:105425 4.2 4:4ea40f39:::benchmark_data_smithi166_32054_object105424:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e339)", 2020-09-08T15:55:28.117 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:17.853184+0000", 2020-09-08T15:55:28.118 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 430.17536339999998, 2020-09-08T15:55:28.118 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20860477299999999, 2020-09-08T15:55:28.118 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:28.118 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:28.119 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:28.119 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:55:28.119 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:55:28.119 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 105425 2020-09-08T15:55:28.119 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.120 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:28.120 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.121 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:28.121 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.853184+0000", 2020-09-08T15:55:28.121 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:28.121 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.122 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.122 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:28.122 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.853184+0000", 2020-09-08T15:55:28.122 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999599 2020-09-08T15:55:28.122 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.123 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.123 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:28.123 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.853143+0000", 2020-09-08T15:55:28.123 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000154802 2020-09-08T15:55:28.123 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.124 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.124 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:28.124 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.853298+0000", 2020-09-08T15:55:28.124 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.8411999999999998e-05 2020-09-08T15:55:28.124 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.125 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.125 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:28.125 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.853327+0000", 2020-09-08T15:55:28.125 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.0049999999999998e-06 2020-09-08T15:55:28.126 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.126 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.126 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:28.126 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.853334+0000", 2020-09-08T15:55:28.126 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012921300000000001 2020-09-08T15:55:28.128 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.128 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.128 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:28.128 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.853463+0000", 2020-09-08T15:55:28.129 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00022158200000000001 2020-09-08T15:55:28.129 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.129 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.129 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:28.129 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.853684+0000", 2020-09-08T15:55:28.130 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018970099999999999 2020-09-08T15:55:28.130 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.130 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.130 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T15:55:28.130 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.853874+0000", 2020-09-08T15:55:28.131 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0071453180000000003 2020-09-08T15:55:28.131 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.131 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.131 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:55:28.131 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.861019+0000", 2020-09-08T15:55:28.132 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20058150399999999 2020-09-08T15:55:28.132 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.132 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.132 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:55:28.132 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.061601+0000", 2020-09-08T15:55:28.132 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000113155 2020-09-08T15:55:28.133 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.133 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.133 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:28.133 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.061714+0000", 2020-09-08T15:55:28.134 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.428e-05 2020-09-08T15:55:28.134 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.134 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.134 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:28.134 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.061788+0000", 2020-09-08T15:55:28.135 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20860477299999999 2020-09-08T15:55:28.135 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.135 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.135 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.135 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.136 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.136 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:105429 4.2 4:497b055c:::benchmark_data_smithi166_32054_object105428:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e339)", 2020-09-08T15:55:28.136 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:17.855411+0000", 2020-09-08T15:55:28.136 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 430.17313593799997, 2020-09-08T15:55:28.136 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20664822899999999, 2020-09-08T15:55:28.137 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:28.137 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:28.137 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:28.137 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:55:28.137 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:55:28.138 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 105429 2020-09-08T15:55:28.138 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.139 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:28.139 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.139 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:28.139 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.855411+0000", 2020-09-08T15:55:28.139 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:28.140 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.140 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.140 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:28.140 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.855411+0000", 2020-09-08T15:55:28.140 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999537 2020-09-08T15:55:28.141 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.141 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.141 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:28.141 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.855365+0000", 2020-09-08T15:55:28.141 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014474999999999999 2020-09-08T15:55:28.142 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.142 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.142 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:28.143 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.855509+0000", 2020-09-08T15:55:28.144 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5058000000000001e-05 2020-09-08T15:55:28.144 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.144 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.144 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:28.144 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.855525+0000", 2020-09-08T15:55:28.145 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.5780000000000001e-06 2020-09-08T15:55:28.145 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.145 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.145 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:28.145 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.855532+0000", 2020-09-08T15:55:28.146 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0022733459999999999 2020-09-08T15:55:28.146 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.146 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.146 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:28.146 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.857805+0000", 2020-09-08T15:55:28.147 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000155773 2020-09-08T15:55:28.147 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.147 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.148 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:28.148 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.857961+0000", 2020-09-08T15:55:28.149 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00020873300000000001 2020-09-08T15:55:28.149 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.149 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.149 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T15:55:28.150 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.858170+0000", 2020-09-08T15:55:28.150 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0030094950000000001 2020-09-08T15:55:28.150 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.150 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.151 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:55:28.151 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.861179+0000", 2020-09-08T15:55:28.151 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.200696597 2020-09-08T15:55:28.151 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.151 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.152 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:55:28.152 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.061876+0000", 2020-09-08T15:55:28.153 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000107132 2020-09-08T15:55:28.154 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.154 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.154 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:28.154 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.061983+0000", 2020-09-08T15:55:28.154 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.6056999999999994e-05 2020-09-08T15:55:28.155 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.155 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.155 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:28.155 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.062059+0000", 2020-09-08T15:55:28.155 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20664822899999999 2020-09-08T15:55:28.156 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.156 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.156 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.156 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.156 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.157 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:105441 4.2 4:446ca20e:::benchmark_data_smithi166_32054_object105440:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e339)", 2020-09-08T15:55:28.157 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:17.866739+0000", 2020-09-08T15:55:28.157 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 430.16180820800002, 2020-09-08T15:55:28.157 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20372923600000001, 2020-09-08T15:55:28.157 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:28.158 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:28.158 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:28.158 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:55:28.158 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:55:28.158 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 105441 2020-09-08T15:55:28.159 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.159 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:28.159 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.159 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:28.159 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.866739+0000", 2020-09-08T15:55:28.164 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:28.164 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.164 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.164 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:28.165 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.866739+0000", 2020-09-08T15:55:28.165 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999623 2020-09-08T15:55:28.165 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.165 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.165 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:28.166 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.866701+0000", 2020-09-08T15:55:28.166 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7627000000000006e-05 2020-09-08T15:55:28.166 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.166 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.166 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:28.167 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.866798+0000", 2020-09-08T15:55:28.167 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.0639999999999999e-05 2020-09-08T15:55:28.167 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.167 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.167 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:28.167 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.866819+0000", 2020-09-08T15:55:28.168 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.6879999999999996e-06 2020-09-08T15:55:28.168 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.168 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.168 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:28.168 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.866825+0000", 2020-09-08T15:55:28.169 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.3525000000000001e-05 2020-09-08T15:55:28.169 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.169 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.169 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:28.169 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.866868+0000", 2020-09-08T15:55:28.170 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000113715 2020-09-08T15:55:28.170 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.170 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.170 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:28.170 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.866982+0000", 2020-09-08T15:55:28.171 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000127448 2020-09-08T15:55:28.171 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.171 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.171 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T15:55:28.171 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.867109+0000", 2020-09-08T15:55:28.172 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0075386200000000002 2020-09-08T15:55:28.172 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.172 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.172 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:55:28.173 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.874648+0000", 2020-09-08T15:55:28.173 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19560765999999999 2020-09-08T15:55:28.173 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.173 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.174 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:55:28.174 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.070256+0000", 2020-09-08T15:55:28.174 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012985199999999999 2020-09-08T15:55:28.174 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.175 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.175 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:28.175 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.070386+0000", 2020-09-08T15:55:28.176 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.2356999999999998e-05 2020-09-08T15:55:28.176 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.176 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.176 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:28.176 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.070468+0000", 2020-09-08T15:55:28.177 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20372923600000001 2020-09-08T15:55:28.177 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.177 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.177 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.177 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.178 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.178 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:105477 4.2 4:45e453c7:::benchmark_data_smithi166_32054_object105476:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e339)", 2020-09-08T15:55:28.178 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:17.898359+0000", 2020-09-08T15:55:28.178 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 430.13018789699998, 2020-09-08T15:55:28.178 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17236843600000001, 2020-09-08T15:55:28.179 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:28.179 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:28.179 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:28.179 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:55:28.179 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:55:28.180 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 105477 2020-09-08T15:55:28.180 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.180 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:28.180 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.180 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:28.181 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.898359+0000", 2020-09-08T15:55:28.181 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:28.181 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.181 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.181 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:28.181 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.898359+0000", 2020-09-08T15:55:28.182 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999642 2020-09-08T15:55:28.182 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.182 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.182 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:28.182 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.898323+0000", 2020-09-08T15:55:28.183 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014085399999999999 2020-09-08T15:55:28.183 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.183 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.183 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:28.183 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.898464+0000", 2020-09-08T15:55:28.184 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0378e-05 2020-09-08T15:55:28.184 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.184 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.184 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:28.184 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.898475+0000", 2020-09-08T15:55:28.185 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.5629999999999998e-06 2020-09-08T15:55:28.185 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.185 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.185 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:28.185 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.898484+0000", 2020-09-08T15:55:28.186 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.2391999999999994e-05 2020-09-08T15:55:28.186 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.186 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.186 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:28.187 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.898556+0000", 2020-09-08T15:55:28.187 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00021068200000000001 2020-09-08T15:55:28.187 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.187 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.188 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:28.188 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.898767+0000", 2020-09-08T15:55:28.188 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00023829799999999999 2020-09-08T15:55:28.188 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.189 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.189 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T15:55:28.189 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.899005+0000", 2020-09-08T15:55:28.189 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00354719 2020-09-08T15:55:28.189 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.190 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.190 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:55:28.190 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.902553+0000", 2020-09-08T15:55:28.190 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16799788399999999 2020-09-08T15:55:28.190 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.191 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.191 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:55:28.191 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.070551+0000", 2020-09-08T15:55:28.191 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000104671 2020-09-08T15:55:28.191 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.191 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.192 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:28.192 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.070655+0000", 2020-09-08T15:55:28.192 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.2293999999999996e-05 2020-09-08T15:55:28.192 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.192 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.193 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:28.193 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.070727+0000", 2020-09-08T15:55:28.193 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17236843600000001 2020-09-08T15:55:28.193 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.193 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.194 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.194 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.194 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.194 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:105488 4.2 4:4e1f29c5:::benchmark_data_smithi166_32054_object105487:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e339)", 2020-09-08T15:55:28.194 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:17.906472+0000", 2020-09-08T15:55:28.195 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 430.12207471400001, 2020-09-08T15:55:28.195 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.164516882, 2020-09-08T15:55:28.195 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:28.195 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:28.195 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:28.196 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:55:28.196 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:55:28.196 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 105488 2020-09-08T15:55:28.196 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.196 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:28.196 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.197 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:28.197 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.906472+0000", 2020-09-08T15:55:28.197 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:28.197 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.197 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.198 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:28.198 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.906472+0000", 2020-09-08T15:55:28.198 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999576 2020-09-08T15:55:28.198 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.198 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.199 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:28.199 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.906430+0000", 2020-09-08T15:55:28.199 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013662400000000001 2020-09-08T15:55:28.199 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.199 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.200 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:28.200 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.906566+0000", 2020-09-08T15:55:28.200 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.959e-05 2020-09-08T15:55:28.200 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.200 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.201 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:28.201 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.906586+0000", 2020-09-08T15:55:28.201 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0196e-05 2020-09-08T15:55:28.201 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.201 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.202 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:28.202 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.906596+0000", 2020-09-08T15:55:28.202 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.2305000000000007e-05 2020-09-08T15:55:28.202 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.202 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.202 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:28.203 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.906668+0000", 2020-09-08T15:55:28.203 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000190891 2020-09-08T15:55:28.203 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.203 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.203 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:28.204 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.906859+0000", 2020-09-08T15:55:28.204 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00021537699999999999 2020-09-08T15:55:28.204 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.204 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.204 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T15:55:28.205 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.907075+0000", 2020-09-08T15:55:28.205 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0044540389999999999 2020-09-08T15:55:28.205 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.205 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.205 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:55:28.206 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:17.911529+0000", 2020-09-08T15:55:28.206 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.159265199 2020-09-08T15:55:28.206 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.206 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.206 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:55:28.207 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.070794+0000", 2020-09-08T15:55:28.207 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.4635000000000004e-05 2020-09-08T15:55:28.207 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.207 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.207 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:28.207 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.070888+0000", 2020-09-08T15:55:28.208 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010061299999999999 2020-09-08T15:55:28.208 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.208 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.208 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:28.208 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:18.070989+0000", 2020-09-08T15:55:28.209 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.164516882 2020-09-08T15:55:28.209 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.209 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.209 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.209 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.210 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.210 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:113600 4.6 4:603824bc:::benchmark_data_smithi166_32054_object113599:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e341)", 2020-09-08T15:55:28.210 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:24.644594+0000", 2020-09-08T15:55:28.210 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 423.383952948, 2020-09-08T15:55:28.210 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.188084585, 2020-09-08T15:55:28.211 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:28.211 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:28.211 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:28.211 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:55:28.211 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:55:28.212 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 113600 2020-09-08T15:55:28.212 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.212 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:28.212 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.212 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:28.213 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.644594+0000", 2020-09-08T15:55:28.213 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:28.213 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.213 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.213 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:28.214 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.644594+0000", 2020-09-08T15:55:28.214 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999604 2020-09-08T15:55:28.214 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.214 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.215 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:28.215 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.644555+0000", 2020-09-08T15:55:28.215 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000131338 2020-09-08T15:55:28.215 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.215 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.215 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:28.216 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.644686+0000", 2020-09-08T15:55:28.216 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4857e-05 2020-09-08T15:55:28.216 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.216 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.216 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:28.217 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.644701+0000", 2020-09-08T15:55:28.217 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6440000000000003e-06 2020-09-08T15:55:28.217 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.217 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.217 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:28.218 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.644707+0000", 2020-09-08T15:55:28.218 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9605999999999999e-05 2020-09-08T15:55:28.218 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.218 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.218 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:28.219 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.644757+0000", 2020-09-08T15:55:28.219 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000163812 2020-09-08T15:55:28.219 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.219 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.219 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:28.219 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.644921+0000", 2020-09-08T15:55:28.220 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019153400000000001 2020-09-08T15:55:28.220 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.220 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.220 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T15:55:28.220 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.645112+0000", 2020-09-08T15:55:28.221 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0039535739999999996 2020-09-08T15:55:28.221 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.221 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.221 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:55:28.221 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.649066+0000", 2020-09-08T15:55:28.222 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18316455400000001 2020-09-08T15:55:28.222 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.222 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.222 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:55:28.222 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.832231+0000", 2020-09-08T15:55:28.223 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00024548799999999998 2020-09-08T15:55:28.223 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.223 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.223 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:28.224 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.832476+0000", 2020-09-08T15:55:28.224 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000202566 2020-09-08T15:55:28.224 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.224 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.224 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:28.225 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:24.832679+0000", 2020-09-08T15:55:28.225 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.188084585 2020-09-08T15:55:28.225 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.225 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.225 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.226 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.226 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.226 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:116122 4.6 4:60608c08:::benchmark_data_smithi166_32054_object116121:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e343)", 2020-09-08T15:55:28.226 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:27.398569+0000", 2020-09-08T15:55:28.226 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 420.629977892, 2020-09-08T15:55:28.226 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.179969832, 2020-09-08T15:55:28.227 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:28.227 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:28.227 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:28.227 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:55:28.227 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:55:28.228 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 116122 2020-09-08T15:55:28.228 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.228 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:28.228 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.228 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:28.229 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.398569+0000", 2020-09-08T15:55:28.229 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:28.229 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.229 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.229 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:28.230 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.398569+0000", 2020-09-08T15:55:28.230 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999776 2020-09-08T15:55:28.230 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.230 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.230 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:28.231 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.398546+0000", 2020-09-08T15:55:28.231 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011290699999999999 2020-09-08T15:55:28.231 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.231 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.231 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:28.231 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.398659+0000", 2020-09-08T15:55:28.232 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5480000000000001e-05 2020-09-08T15:55:28.232 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.232 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.232 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:28.232 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.398675+0000", 2020-09-08T15:55:28.233 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.8210000000000004e-06 2020-09-08T15:55:28.233 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.233 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.233 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:28.233 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.398682+0000", 2020-09-08T15:55:28.234 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.4580999999999997e-05 2020-09-08T15:55:28.234 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.234 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.234 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:28.234 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.398746+0000", 2020-09-08T15:55:28.235 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018969700000000001 2020-09-08T15:55:28.235 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.235 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.235 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:28.235 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.398936+0000", 2020-09-08T15:55:28.235 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00021333699999999999 2020-09-08T15:55:28.236 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.236 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.236 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T15:55:28.236 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.399149+0000", 2020-09-08T15:55:28.236 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0037601660000000001 2020-09-08T15:55:28.237 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.237 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.237 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:55:28.237 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.402909+0000", 2020-09-08T15:55:28.237 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.175362343 2020-09-08T15:55:28.238 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.238 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.238 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:55:28.238 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.578272+0000", 2020-09-08T15:55:28.238 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014603899999999999 2020-09-08T15:55:28.239 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.239 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.239 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:28.239 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.578418+0000", 2020-09-08T15:55:28.239 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000121057 2020-09-08T15:55:28.240 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.240 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.240 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:28.240 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:27.578539+0000", 2020-09-08T15:55:28.240 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.179969832 2020-09-08T15:55:28.240 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.241 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.241 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.241 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.241 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.242 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:142001 4.2 4:435b9d62:::benchmark_data_smithi166_32054_object142000:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e350)", 2020-09-08T15:55:28.242 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:48:53.091577+0000", 2020-09-08T15:55:28.242 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 394.936969938, 2020-09-08T15:55:28.242 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16663236100000001, 2020-09-08T15:55:28.242 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:28.242 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:28.243 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:28.243 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:55:28.243 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:55:28.243 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 142001 2020-09-08T15:55:28.243 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.244 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:28.244 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.244 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:28.244 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.091577+0000", 2020-09-08T15:55:28.244 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:28.245 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.245 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.245 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:28.245 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.091577+0000", 2020-09-08T15:55:28.245 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999666 2020-09-08T15:55:28.246 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.246 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.246 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:28.246 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.091543+0000", 2020-09-08T15:55:28.246 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010740599999999999 2020-09-08T15:55:28.247 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.247 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.247 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:28.247 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.091651+0000", 2020-09-08T15:55:28.247 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3954000000000001e-05 2020-09-08T15:55:28.247 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.248 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.248 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:28.248 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.091665+0000", 2020-09-08T15:55:28.248 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5270000000000004e-06 2020-09-08T15:55:28.248 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.249 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.249 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:28.249 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.091671+0000", 2020-09-08T15:55:28.249 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.3033000000000004e-05 2020-09-08T15:55:28.249 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.250 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.250 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:28.250 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.091734+0000", 2020-09-08T15:55:28.250 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000143027 2020-09-08T15:55:28.250 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.251 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.251 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:28.251 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.091877+0000", 2020-09-08T15:55:28.251 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15638299899999999 2020-09-08T15:55:28.251 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.251 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.252 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T15:55:28.252 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.248260+0000", 2020-09-08T15:55:28.252 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0096406410000000001 2020-09-08T15:55:28.252 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.252 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.253 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:55:28.253 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.257901+0000", 2020-09-08T15:55:28.253 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.0284000000000005e-05 2020-09-08T15:55:28.253 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.253 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.254 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:55:28.254 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.257991+0000", 2020-09-08T15:55:28.254 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000128264 2020-09-08T15:55:28.254 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.254 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.255 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:28.255 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.258120+0000", 2020-09-08T15:55:28.255 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.9784999999999995e-05 2020-09-08T15:55:28.255 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.255 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.255 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:28.256 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:48:53.258209+0000", 2020-09-08T15:55:28.256 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16663236100000001 2020-09-08T15:55:28.256 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.256 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.256 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.257 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.257 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.257 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:216421 4.6 4:65f0eb6e:::benchmark_data_smithi166_32054_object74279:head [delete] snapc 0=[] ondisk+write+known_if_redirected e369)", 2020-09-08T15:55:28.257 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:50:08.789048+0000", 2020-09-08T15:55:28.257 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 319.23949934799998, 2020-09-08T15:55:28.258 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.159611907, 2020-09-08T15:55:28.258 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:28.258 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:28.258 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:28.258 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:55:28.259 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:55:28.259 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 216421 2020-09-08T15:55:28.259 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.259 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:28.259 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.259 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:28.260 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.789048+0000", 2020-09-08T15:55:28.260 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:28.260 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.260 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.260 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:28.261 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.789048+0000", 2020-09-08T15:55:28.261 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999552 2020-09-08T15:55:28.261 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.261 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.261 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:28.262 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.789003+0000", 2020-09-08T15:55:28.262 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.4965999999999998e-05 2020-09-08T15:55:28.262 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.262 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.262 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:28.263 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.789098+0000", 2020-09-08T15:55:28.263 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4535000000000001e-05 2020-09-08T15:55:28.263 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.263 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.263 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:28.263 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.789112+0000", 2020-09-08T15:55:28.264 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.2079999999999997e-06 2020-09-08T15:55:28.264 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.264 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.264 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:28.264 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.789119+0000", 2020-09-08T15:55:28.265 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0031527579999999999 2020-09-08T15:55:28.265 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.265 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.265 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:28.266 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.792271+0000", 2020-09-08T15:55:28.266 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.3747999999999998e-05 2020-09-08T15:55:28.266 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.266 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.266 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:28.267 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.792355+0000", 2020-09-08T15:55:28.267 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.0872999999999996e-05 2020-09-08T15:55:28.267 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.267 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.267 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T15:55:28.267 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.792436+0000", 2020-09-08T15:55:28.268 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.13926424800000001 2020-09-08T15:55:28.268 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.268 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.268 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:55:28.268 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.931700+0000", 2020-09-08T15:55:28.269 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.016819555999999999 2020-09-08T15:55:28.269 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.269 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.269 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:55:28.269 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.948520+0000", 2020-09-08T15:55:28.270 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.3306999999999994e-05 2020-09-08T15:55:28.270 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.270 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.270 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:28.270 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.948603+0000", 2020-09-08T15:55:28.271 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.6427e-05 2020-09-08T15:55:28.271 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.271 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.271 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:28.271 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:08.948659+0000", 2020-09-08T15:55:28.272 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.159611907 2020-09-08T15:55:28.272 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.272 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.272 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.272 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.272 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.273 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:255309 4.6 4:68c19ab3:::benchmark_data_smithi166_32054_object113167:head [delete] snapc 0=[] ondisk+write+known_if_redirected e382)", 2020-09-08T15:55:28.273 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:50:48.065335+0000", 2020-09-08T15:55:28.273 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 279.96321239700001, 2020-09-08T15:55:28.273 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16226527399999999, 2020-09-08T15:55:28.274 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:28.274 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:28.274 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:28.274 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:55:28.274 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:55:28.275 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 255309 2020-09-08T15:55:28.275 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.275 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:28.276 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.276 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:28.276 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.065335+0000", 2020-09-08T15:55:28.276 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:28.276 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.277 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.277 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:28.277 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.065335+0000", 2020-09-08T15:55:28.277 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999523 2020-09-08T15:55:28.277 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.278 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.278 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:28.278 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.065287+0000", 2020-09-08T15:55:28.278 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000112459 2020-09-08T15:55:28.279 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.279 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.279 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:28.279 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.065399+0000", 2020-09-08T15:55:28.279 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.8306999999999999e-05 2020-09-08T15:55:28.280 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.280 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.280 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:28.280 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.065428+0000", 2020-09-08T15:55:28.280 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2820000000000002e-06 2020-09-08T15:55:28.281 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.281 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.281 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:28.281 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.065437+0000", 2020-09-08T15:55:28.281 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.5964999999999997e-05 2020-09-08T15:55:28.282 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.282 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.282 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:28.282 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.065493+0000", 2020-09-08T15:55:28.283 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014469600000000001 2020-09-08T15:55:28.283 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.283 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.283 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:28.283 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.065637+0000", 2020-09-08T15:55:28.283 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000149087 2020-09-08T15:55:28.284 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.284 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.284 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T15:55:28.284 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.065787+0000", 2020-09-08T15:55:28.284 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0097449249999999998 2020-09-08T15:55:28.285 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.285 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.285 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:55:28.285 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.075531+0000", 2020-09-08T15:55:28.285 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15187118599999999 2020-09-08T15:55:28.286 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.286 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.286 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:55:28.286 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.227403+0000", 2020-09-08T15:55:28.287 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000123867 2020-09-08T15:55:28.287 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.287 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.287 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:28.287 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.227527+0000", 2020-09-08T15:55:28.288 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.3280000000000003e-05 2020-09-08T15:55:28.288 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.288 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.288 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:28.288 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.227600+0000", 2020-09-08T15:55:28.289 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16226527399999999 2020-09-08T15:55:28.289 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.289 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.289 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.290 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.290 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.290 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.35381.0:255310 4.6 4:692e0ed2:::benchmark_data_smithi166_32054_object113168:head [delete] snapc 0=[] ondisk+write+known_if_redirected e382)", 2020-09-08T15:55:28.290 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:50:48.066041+0000", 2020-09-08T15:55:28.290 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 279.96250620500001, 2020-09-08T15:55:28.291 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16186495100000001, 2020-09-08T15:55:28.291 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:28.291 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:28.291 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:28.291 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.35381", 2020-09-08T15:55:28.292 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/613600250", 2020-09-08T15:55:28.292 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 255310 2020-09-08T15:55:28.292 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.292 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:28.292 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.293 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:28.293 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.066041+0000", 2020-09-08T15:55:28.293 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:28.293 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.293 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.294 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:28.294 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.066041+0000", 2020-09-08T15:55:28.294 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999428 2020-09-08T15:55:28.294 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.294 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.294 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:28.295 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.065984+0000", 2020-09-08T15:55:28.295 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000119695 2020-09-08T15:55:28.295 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.295 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.295 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:28.296 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.066103+0000", 2020-09-08T15:55:28.296 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.8243000000000001e-05 2020-09-08T15:55:28.296 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.296 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.296 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:28.297 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.066122+0000", 2020-09-08T15:55:28.297 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.0260000000000004e-06 2020-09-08T15:55:28.297 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.297 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.297 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:28.298 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.066128+0000", 2020-09-08T15:55:28.298 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0051837740000000004 2020-09-08T15:55:28.298 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.298 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.298 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:28.298 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.071311+0000", 2020-09-08T15:55:28.299 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000113422 2020-09-08T15:55:28.299 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.299 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.299 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:28.299 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.071425+0000", 2020-09-08T15:55:28.300 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000100053 2020-09-08T15:55:28.300 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.300 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.300 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T15:55:28.300 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.071525+0000", 2020-09-08T15:55:28.301 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0042436289999999996 2020-09-08T15:55:28.301 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.301 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.301 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:55:28.301 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.075768+0000", 2020-09-08T15:55:28.301 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15200413700000001 2020-09-08T15:55:28.302 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.302 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.302 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:55:28.302 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.227773+0000", 2020-09-08T15:55:28.302 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.8267000000000001e-05 2020-09-08T15:55:28.303 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.303 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.303 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:28.303 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.227851+0000", 2020-09-08T15:55:28.303 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.4839999999999997e-05 2020-09-08T15:55:28.304 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.304 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.304 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:28.304 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:50:48.227906+0000", 2020-09-08T15:55:28.304 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16186495100000001 2020-09-08T15:55:28.304 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.305 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.305 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.305 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.305 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.305 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:120062 5.b 5:d04433ad:::benchmark_data_smithi166_37820_object120061:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T15:55:28.306 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:53:00.064325+0000", 2020-09-08T15:55:28.306 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 147.96422181400001, 2020-09-08T15:55:28.306 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.56732750200000004, 2020-09-08T15:55:28.306 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:28.306 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:28.307 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:28.307 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T15:55:28.307 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:55:28.307 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 120062 2020-09-08T15:55:28.307 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.308 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:28.308 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.308 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:28.308 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.064325+0000", 2020-09-08T15:55:28.308 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:28.308 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.309 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.309 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:28.309 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.064325+0000", 2020-09-08T15:55:28.309 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999733 2020-09-08T15:55:28.309 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.310 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.310 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:28.310 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.064299+0000", 2020-09-08T15:55:28.310 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010194100000000001 2020-09-08T15:55:28.311 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.311 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.311 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:28.311 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.064401+0000", 2020-09-08T15:55:28.311 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1205e-05 2020-09-08T15:55:28.311 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.312 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.312 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:28.312 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.064412+0000", 2020-09-08T15:55:28.312 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.305e-06 2020-09-08T15:55:28.312 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.313 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.313 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:28.313 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.064417+0000", 2020-09-08T15:55:28.313 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8560000000000002e-05 2020-09-08T15:55:28.313 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.314 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.314 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:28.314 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.064476+0000", 2020-09-08T15:55:28.314 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015337 2020-09-08T15:55:28.315 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.315 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.315 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:28.315 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.064629+0000", 2020-09-08T15:55:28.315 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017367399999999999 2020-09-08T15:55:28.316 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.316 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.316 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:55:28.316 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.064803+0000", 2020-09-08T15:55:28.316 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.56645555299999995 2020-09-08T15:55:28.316 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.317 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.317 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:55:28.317 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.631258+0000", 2020-09-08T15:55:28.317 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.5216e-05 2020-09-08T15:55:28.317 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.318 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.318 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:55:28.318 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.631283+0000", 2020-09-08T15:55:28.318 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00023212600000000001 2020-09-08T15:55:28.318 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.319 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.319 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:28.319 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.631516+0000", 2020-09-08T15:55:28.319 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013703000000000001 2020-09-08T15:55:28.319 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.320 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.320 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:28.320 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.631653+0000", 2020-09-08T15:55:28.320 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.56732750200000004 2020-09-08T15:55:28.320 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.321 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.321 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.321 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.321 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.321 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:120065 5.e 5:7cd67fbc:::benchmark_data_smithi166_37820_object120064:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T15:55:28.322 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:53:00.065103+0000", 2020-09-08T15:55:28.322 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 147.96344394900001, 2020-09-08T15:55:28.322 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.58498226900000005, 2020-09-08T15:55:28.322 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:28.322 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:28.323 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:28.323 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T15:55:28.323 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:55:28.323 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 120065 2020-09-08T15:55:28.324 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.324 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:28.324 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.324 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:28.325 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.065103+0000", 2020-09-08T15:55:28.325 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:28.325 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.325 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.325 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:28.326 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.065103+0000", 2020-09-08T15:55:28.326 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999604 2020-09-08T15:55:28.326 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.326 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.327 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:28.327 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.065064+0000", 2020-09-08T15:55:28.327 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012207599999999999 2020-09-08T15:55:28.327 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.327 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.328 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:28.328 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.065186+0000", 2020-09-08T15:55:28.328 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2500000000000001e-05 2020-09-08T15:55:28.328 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.328 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.329 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:28.329 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.065198+0000", 2020-09-08T15:55:28.329 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5230000000000001e-06 2020-09-08T15:55:28.329 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.329 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.330 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:28.330 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.065205+0000", 2020-09-08T15:55:28.330 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8696999999999997e-05 2020-09-08T15:55:28.330 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.330 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.331 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:28.331 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.065263+0000", 2020-09-08T15:55:28.331 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019886 2020-09-08T15:55:28.331 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.331 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.331 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:28.332 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.065462+0000", 2020-09-08T15:55:28.332 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000181492 2020-09-08T15:55:28.332 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.332 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.332 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 8", 2020-09-08T15:55:28.333 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.065644+0000", 2020-09-08T15:55:28.333 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.57882055300000002 2020-09-08T15:55:28.333 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.333 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.333 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:55:28.334 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.644464+0000", 2020-09-08T15:55:28.334 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0054758790000000003 2020-09-08T15:55:28.334 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.334 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.334 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:55:28.335 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.649940+0000", 2020-09-08T15:55:28.335 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.6984000000000006e-05 2020-09-08T15:55:28.335 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.335 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.335 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:28.335 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.650027+0000", 2020-09-08T15:55:28.336 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8093000000000001e-05 2020-09-08T15:55:28.336 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.336 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.336 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:28.336 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.650085+0000", 2020-09-08T15:55:28.337 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.58498226900000005 2020-09-08T15:55:28.337 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.337 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.337 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.337 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.338 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.338 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:120068 5.e 5:7f64bdfc:::benchmark_data_smithi166_37820_object120067:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T15:55:28.338 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:53:00.065700+0000", 2020-09-08T15:55:28.338 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 147.962847431, 2020-09-08T15:55:28.338 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.58449586399999998, 2020-09-08T15:55:28.339 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:28.339 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:28.339 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:28.339 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T15:55:28.339 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:55:28.339 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 120068 2020-09-08T15:55:28.340 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.340 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:28.340 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.340 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:28.340 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.065700+0000", 2020-09-08T15:55:28.341 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:28.341 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.341 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.341 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:28.341 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.065700+0000", 2020-09-08T15:55:28.342 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999428 2020-09-08T15:55:28.342 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.342 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.342 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:28.342 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.065642+0000", 2020-09-08T15:55:28.342 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000177388 2020-09-08T15:55:28.343 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.343 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.343 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:28.343 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.065820+0000", 2020-09-08T15:55:28.343 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.9854999999999999e-05 2020-09-08T15:55:28.344 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.344 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.344 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:28.344 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.065840+0000", 2020-09-08T15:55:28.344 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.7700000000000007e-06 2020-09-08T15:55:28.345 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.345 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.345 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:28.345 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.065848+0000", 2020-09-08T15:55:28.345 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.56199569800000004 2020-09-08T15:55:28.345 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.346 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.346 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:28.346 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.627844+0000", 2020-09-08T15:55:28.346 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00045768399999999998 2020-09-08T15:55:28.346 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.347 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.347 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:28.347 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.628302+0000", 2020-09-08T15:55:28.347 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00047090700000000001 2020-09-08T15:55:28.347 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.348 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.348 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 8", 2020-09-08T15:55:28.348 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.628773+0000", 2020-09-08T15:55:28.348 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.015731328999999999 2020-09-08T15:55:28.348 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.349 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.349 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:55:28.349 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.644504+0000", 2020-09-08T15:55:28.349 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0055955149999999997 2020-09-08T15:55:28.349 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.350 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.350 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:55:28.350 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.650099+0000", 2020-09-08T15:55:28.350 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.5145000000000001e-05 2020-09-08T15:55:28.350 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.350 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.351 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:28.351 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.650155+0000", 2020-09-08T15:55:28.351 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.0775000000000002e-05 2020-09-08T15:55:28.351 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.351 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.352 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:28.352 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.650195+0000", 2020-09-08T15:55:28.352 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.58449586399999998 2020-09-08T15:55:28.352 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.352 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.353 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.353 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.353 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.353 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:120069 5.b 5:de89853e:::benchmark_data_smithi166_37820_object120068:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T15:55:28.353 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:53:00.068466+0000", 2020-09-08T15:55:28.354 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 147.96008057500001, 2020-09-08T15:55:28.354 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.583942972, 2020-09-08T15:55:28.354 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:28.354 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:55:28.354 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:28.354 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T15:55:28.355 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:55:28.355 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 120069 2020-09-08T15:55:28.355 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.355 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:28.355 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.356 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:28.356 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.068466+0000", 2020-09-08T15:55:28.356 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:28.356 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.356 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.357 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:28.357 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.068466+0000", 2020-09-08T15:55:28.357 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999614 2020-09-08T15:55:28.357 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.357 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.357 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:28.358 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.068428+0000", 2020-09-08T15:55:28.358 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.55851846599999999 2020-09-08T15:55:28.358 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.358 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.358 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:28.359 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.626946+0000", 2020-09-08T15:55:28.359 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.9942999999999999e-05 2020-09-08T15:55:28.359 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.359 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.359 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:28.360 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.626966+0000", 2020-09-08T15:55:28.360 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.8290000000000008e-06 2020-09-08T15:55:28.360 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.360 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.360 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:28.361 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.626974+0000", 2020-09-08T15:55:28.361 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0047687110000000001 2020-09-08T15:55:28.361 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.361 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.361 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:28.361 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.631743+0000", 2020-09-08T15:55:28.362 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00035113599999999999 2020-09-08T15:55:28.362 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.362 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.362 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:28.362 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.632094+0000", 2020-09-08T15:55:28.363 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00048625699999999998 2020-09-08T15:55:28.363 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.363 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.363 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:55:28.363 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.632580+0000", 2020-09-08T15:55:28.364 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.019634951000000001 2020-09-08T15:55:28.364 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.364 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.364 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:55:28.364 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.652215+0000", 2020-09-08T15:55:28.364 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.1386999999999999e-05 2020-09-08T15:55:28.365 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.365 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.365 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:55:28.365 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.652296+0000", 2020-09-08T15:55:28.365 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.7564999999999994e-05 2020-09-08T15:55:28.366 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.366 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.366 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:55:28.366 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.652364+0000", 2020-09-08T15:55:28.366 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.5521e-05 2020-09-08T15:55:28.367 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:28.367 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:28.367 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:55:28.367 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.652409+0000", 2020-09-08T15:55:28.367 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.583942972 2020-09-08T15:55:28.368 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.368 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.368 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.368 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:28.368 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:28.368 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:55:28.370 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:28.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:55:28 smithi104 bash[19922]: debug 2020-09-08T15:55:28.043+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:28.471 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:55:28.608 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:55:28.711 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:55:28.906 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:55:28 smithi055 bash[14301]: debug 2020-09-08T15:55:28.609+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:28.962 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2020-09-08T15:55:29.061 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:28 smithi104 bash[9203]: cluster 2020-09-08T15:55:27.771524+0000 osd.3 (osd.3) 67 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:29.062 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:28 smithi104 bash[9203]: cluster 2020-09-08T15:55:27.771620+0000 osd.3 (osd.3) 68 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:29.062 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:28 smithi104 bash[9203]: cluster 2020-09-08T15:55:27.873813+0000 mgr.x (mgr.24118) 844 : cluster [DBG] pgmap v1224: 17 pgs: 17 active+clean; 1.9 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T15:55:29.062 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:28 smithi104 bash[9203]: cluster 2020-09-08T15:55:27.942210+0000 mon.a (mon.0) 1667 : cluster [DBG] osdmap e465: 12 total, 12 up, 12 in 2020-09-08T15:55:29.062 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:55:29 smithi104 bash[17588]: debug 2020-09-08T15:55:29.063+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:29.064 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:55:29.119 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:55:29.168 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:55:29.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:28 smithi166 bash[9446]: cluster 2020-09-08T15:55:27.771524+0000 osd.3 (osd.3) 67 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:29.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:28 smithi166 bash[9446]: cluster 2020-09-08T15:55:27.771620+0000 osd.3 (osd.3) 68 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:29.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:28 smithi166 bash[9446]: cluster 2020-09-08T15:55:27.873813+0000 mgr.x (mgr.24118) 844 : cluster [DBG] pgmap v1224: 17 pgs: 17 active+clean; 1.9 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T15:55:29.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:28 smithi166 bash[9446]: cluster 2020-09-08T15:55:27.942210+0000 mon.a (mon.0) 1667 : cluster [DBG] osdmap e465: 12 total, 12 up, 12 in 2020-09-08T15:55:29.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:28 smithi055 bash[9296]: cluster 2020-09-08T15:55:27.771524+0000 osd.3 (osd.3) 67 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:29.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:28 smithi055 bash[9296]: cluster 2020-09-08T15:55:27.771620+0000 osd.3 (osd.3) 68 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:29.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:28 smithi055 bash[9296]: cluster 2020-09-08T15:55:27.873813+0000 mgr.x (mgr.24118) 844 : cluster [DBG] pgmap v1224: 17 pgs: 17 active+clean; 1.9 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T15:55:29.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:28 smithi055 bash[9296]: cluster 2020-09-08T15:55:27.942210+0000 mon.a (mon.0) 1667 : cluster [DBG] osdmap e465: 12 total, 12 up, 12 in 2020-09-08T15:55:29.753 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:55:29.856 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:30.102 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:30.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:55:29 smithi166 bash[14254]: debug 2020-09-08T15:55:29.749+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:30.144 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:55:30 smithi104 bash[19922]: debug 2020-09-08T15:55:30.099+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:30.206 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:55:30.348 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:55:30.454 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:30.661 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:55:30 smithi055 bash[11956]: debug 2020-09-08T15:55:30.349+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:31.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:30 smithi166 bash[9446]: cluster 2020-09-08T15:55:29.874313+0000 mgr.x (mgr.24118) 845 : cluster [DBG] pgmap v1226: 17 pgs: 17 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 849 op/s 2020-09-08T15:55:31.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:30 smithi055 bash[9296]: cluster 2020-09-08T15:55:29.874313+0000 mgr.x (mgr.24118) 845 : cluster [DBG] pgmap v1226: 17 pgs: 17 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 849 op/s 2020-09-08T15:55:31.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:30 smithi104 bash[9203]: cluster 2020-09-08T15:55:29.874313+0000 mgr.x (mgr.24118) 845 : cluster [DBG] pgmap v1226: 17 pgs: 17 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 849 op/s 2020-09-08T15:55:31.350 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:31.452 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:31.587 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:31.675 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:55:31 smithi104 bash[19922]: debug 2020-09-08T15:55:31.347+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:31.691 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:55:31.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:55:31 smithi055 bash[16646]: debug 2020-09-08T15:55:31.585+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:31.925 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:55:32.034 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:55:32.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:55:31 smithi166 bash[14254]: debug 2020-09-08T15:55:31.925+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:32.152 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:55:32.258 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:32.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:55:32 smithi055 bash[14301]: debug 2020-09-08T15:55:32.149+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:32.467 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:32.569 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:32.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:55:32 smithi166 bash[18960]: debug 2020-09-08T15:55:32.457+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:33.012 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:33.115 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:55:33.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:32 smithi104 bash[9203]: cluster 2020-09-08T15:55:31.874914+0000 mgr.x (mgr.24118) 846 : cluster [DBG] pgmap v1227: 17 pgs: 17 active+clean; 1.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 706 op/s 2020-09-08T15:55:33.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:55:33 smithi104 bash[15273]: debug 2020-09-08T15:55:33.015+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:33.235 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:32 smithi166 bash[9446]: cluster 2020-09-08T15:55:31.874914+0000 mgr.x (mgr.24118) 846 : cluster [DBG] pgmap v1227: 17 pgs: 17 active+clean; 1.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 706 op/s 2020-09-08T15:55:33.250 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:32 smithi055 bash[9296]: cluster 2020-09-08T15:55:31.874914+0000 mgr.x (mgr.24118) 846 : cluster [DBG] pgmap v1227: 17 pgs: 17 active+clean; 1.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 706 op/s 2020-09-08T15:55:33.254 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:55:33.359 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:33.497 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:55:33.498 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T15:55:33.498 INFO:teuthology.orchestra.run.smithi166.stdout: "num_ops": 0 2020-09-08T15:55:33.498 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:55:33.614 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:33.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:55:33 smithi055 bash[14301]: debug 2020-09-08T15:55:33.253+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:33.667 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:55:33 smithi104 bash[19922]: debug 2020-09-08T15:55:33.615+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:33.718 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:55:33.836 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:55:33.921 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:55:33.940 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:33.966 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:55:33.995 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:55:34.038 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:55:34.076 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:55:34.114 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:55:34.154 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:55:34.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:33 smithi055 bash[9296]: audit 2020-09-08T15:55:33.881349+0000 mon.b (mon.1) 103 : audit [DBG] from='client.? v1:172.21.15.104:0/1668767775' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:55:34.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:55:33 smithi055 bash[14301]: debug 2020-09-08T15:55:33.837+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:34.188 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:55:34.218 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:55:34.260 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:55:34.298 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:55:34.327 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:55:34.329 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:55:34.330 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:55:34.330 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:55:34.330 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:55:34.331 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:55:34.331 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:55:34.331 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:55:34.332 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:55:34.332 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:55:34.332 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:55:34.332 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:55:34.333 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:55:34.352 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:33 smithi166 bash[9446]: audit 2020-09-08T15:55:33.881349+0000 mon.b (mon.1) 103 : audit [DBG] from='client.? v1:172.21.15.104:0/1668767775' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:55:34.353 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:34.353 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:55:34 smithi166 bash[16593]: debug 2020-09-08T15:55:34.349+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:34.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:33 smithi104 bash[9203]: audit 2020-09-08T15:55:33.881349+0000 mon.b (mon.1) 103 : audit [DBG] from='client.? v1:172.21.15.104:0/1668767775' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:55:34.457 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:55:34.502 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2020-09-08T15:55:34.654 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:55:34.674 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:55:34.777 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:34.925 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:55:34 smithi104 bash[17588]: debug 2020-09-08T15:55:34.671+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:34.952 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:55:35.153 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:34 smithi166 bash[9446]: cluster 2020-09-08T15:55:33.875407+0000 mgr.x (mgr.24118) 847 : cluster [DBG] pgmap v1228: 17 pgs: 17 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 978 op/s 2020-09-08T15:55:35.153 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:34 smithi166 bash[9446]: audit 2020-09-08T15:55:34.269164+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? v1:172.21.15.104:0/235097051' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:55:35.336 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:35.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:34 smithi055 bash[9296]: cluster 2020-09-08T15:55:33.875407+0000 mgr.x (mgr.24118) 847 : cluster [DBG] pgmap v1228: 17 pgs: 17 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 978 op/s 2020-09-08T15:55:35.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:34 smithi055 bash[9296]: audit 2020-09-08T15:55:34.269164+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? v1:172.21.15.104:0/235097051' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:55:35.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:34 smithi104 bash[9203]: cluster 2020-09-08T15:55:33.875407+0000 mgr.x (mgr.24118) 847 : cluster [DBG] pgmap v1228: 17 pgs: 17 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 978 op/s 2020-09-08T15:55:35.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:34 smithi104 bash[9203]: audit 2020-09-08T15:55:34.269164+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? v1:172.21.15.104:0/235097051' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:55:35.510 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:35.617 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:55:35 smithi166 bash[16593]: debug 2020-09-08T15:55:35.333+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:35.852 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:35.956 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:36.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:55:35 smithi104 bash[22305]: debug 2020-09-08T15:55:35.855+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:36.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:35 smithi104 bash[9203]: cluster 2020-09-08T15:55:34.424473+0000 osd.8 (osd.8) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:36.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:35 smithi104 bash[9203]: cluster 2020-09-08T15:55:34.424567+0000 osd.8 (osd.8) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:36.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:35 smithi104 bash[9203]: cluster 2020-09-08T15:55:34.937657+0000 mon.a (mon.0) 1669 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:55:36.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:35 smithi104 bash[9203]: audit 2020-09-08T15:55:34.955835+0000 mon.a (mon.0) 1670 : audit [INF] from='client.? v1:172.21.15.104:0/235097051' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:55:36.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:35 smithi104 bash[9203]: cluster 2020-09-08T15:55:34.955889+0000 mon.a (mon.0) 1671 : cluster [DBG] osdmap e466: 12 total, 12 up, 12 in 2020-09-08T15:55:36.276 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:35 smithi166 bash[9446]: cluster 2020-09-08T15:55:34.424473+0000 osd.8 (osd.8) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:36.276 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:35 smithi166 bash[9446]: cluster 2020-09-08T15:55:34.424567+0000 osd.8 (osd.8) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:36.277 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:35 smithi166 bash[9446]: cluster 2020-09-08T15:55:34.937657+0000 mon.a (mon.0) 1669 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:55:36.277 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:35 smithi166 bash[9446]: audit 2020-09-08T15:55:34.955835+0000 mon.a (mon.0) 1670 : audit [INF] from='client.? v1:172.21.15.104:0/235097051' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:55:36.277 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:35 smithi166 bash[9446]: cluster 2020-09-08T15:55:34.955889+0000 mon.a (mon.0) 1671 : cluster [DBG] osdmap e466: 12 total, 12 up, 12 in 2020-09-08T15:55:36.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:35 smithi055 bash[9296]: cluster 2020-09-08T15:55:34.424473+0000 osd.8 (osd.8) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:36.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:35 smithi055 bash[9296]: cluster 2020-09-08T15:55:34.424567+0000 osd.8 (osd.8) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:36.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:35 smithi055 bash[9296]: cluster 2020-09-08T15:55:34.937657+0000 mon.a (mon.0) 1669 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:55:36.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:35 smithi055 bash[9296]: audit 2020-09-08T15:55:34.955835+0000 mon.a (mon.0) 1670 : audit [INF] from='client.? v1:172.21.15.104:0/235097051' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:55:36.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:35 smithi055 bash[9296]: cluster 2020-09-08T15:55:34.955889+0000 mon.a (mon.0) 1671 : cluster [DBG] osdmap e466: 12 total, 12 up, 12 in 2020-09-08T15:55:36.462 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:36.566 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:36.675 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:55:36 smithi104 bash[22305]: debug 2020-09-08T15:55:36.463+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:36.749 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:36.853 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:55:36.910 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:55:36 smithi166 bash[18960]: debug 2020-09-08T15:55:36.749+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:36.988 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:55:37.092 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:37.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:55:36 smithi055 bash[11956]: debug 2020-09-08T15:55:36.989+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:37.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:36 smithi055 bash[9296]: cluster 2020-09-08T15:55:35.875857+0000 mgr.x (mgr.24118) 848 : cluster [DBG] pgmap v1230: 17 pgs: 17 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 913 op/s 2020-09-08T15:55:37.220 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:55:37 smithi104 bash[22305]: debug 2020-09-08T15:55:37.223+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:37.220 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:36 smithi104 bash[9203]: cluster 2020-09-08T15:55:35.875857+0000 mgr.x (mgr.24118) 848 : cluster [DBG] pgmap v1230: 17 pgs: 17 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 913 op/s 2020-09-08T15:55:37.226 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:37.330 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:55:37.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:36 smithi166 bash[9446]: cluster 2020-09-08T15:55:35.875857+0000 mgr.x (mgr.24118) 848 : cluster [DBG] pgmap v1230: 17 pgs: 17 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 913 op/s 2020-09-08T15:55:37.449 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:55:37.552 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:37.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:55:37 smithi055 bash[11956]: debug 2020-09-08T15:55:37.453+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:37.688 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:37.791 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:37.912 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:37.952 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:55:37 smithi055 bash[16646]: debug 2020-09-08T15:55:37.685+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:38.015 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:38.139 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:55:37 smithi104 bash[15273]: debug 2020-09-08T15:55:37.915+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:38.140 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:37 smithi104 bash[9203]: cluster 2020-09-08T15:55:36.962260+0000 osd.6 (osd.6) 75 : cluster [DBG] 5.2 scrub starts 2020-09-08T15:55:38.140 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:37 smithi104 bash[9203]: cluster 2020-09-08T15:55:37.344318+0000 osd.6 (osd.6) 76 : cluster [DBG] 5.2 scrub ok 2020-09-08T15:55:38.140 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:37 smithi104 bash[9203]: cluster 2020-09-08T15:55:37.443468+0000 osd.5 (osd.5) 64 : cluster [DBG] 5.7 scrub starts 2020-09-08T15:55:38.141 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:38.244 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:38.253 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:37 smithi166 bash[9446]: cluster 2020-09-08T15:55:36.962260+0000 osd.6 (osd.6) 75 : cluster [DBG] 5.2 scrub starts 2020-09-08T15:55:38.253 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:37 smithi166 bash[9446]: cluster 2020-09-08T15:55:37.344318+0000 osd.6 (osd.6) 76 : cluster [DBG] 5.2 scrub ok 2020-09-08T15:55:38.254 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:37 smithi166 bash[9446]: cluster 2020-09-08T15:55:37.443468+0000 osd.5 (osd.5) 64 : cluster [DBG] 5.7 scrub starts 2020-09-08T15:55:38.352 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:55:38.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:37 smithi055 bash[9296]: cluster 2020-09-08T15:55:36.962260+0000 osd.6 (osd.6) 75 : cluster [DBG] 5.2 scrub starts 2020-09-08T15:55:38.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:37 smithi055 bash[9296]: cluster 2020-09-08T15:55:37.344318+0000 osd.6 (osd.6) 76 : cluster [DBG] 5.2 scrub ok 2020-09-08T15:55:38.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:37 smithi055 bash[9296]: cluster 2020-09-08T15:55:37.443468+0000 osd.5 (osd.5) 64 : cluster [DBG] 5.7 scrub starts 2020-09-08T15:55:38.425 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:55:38 smithi104 bash[22305]: debug 2020-09-08T15:55:38.143+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:38.524 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:55:38.596 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:55:38.712 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:38.778 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:55:38.818 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:39.042 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:38 smithi166 bash[9446]: cluster 2020-09-08T15:55:37.876272+0000 mgr.x (mgr.24118) 849 : cluster [DBG] pgmap v1231: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:55:39.043 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:55:38 smithi166 bash[11928]: debug 2020-09-08T15:55:38.709+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:39.314 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:55:39.315 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:55:39.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:38 smithi055 bash[9296]: cluster 2020-09-08T15:55:37.876272+0000 mgr.x (mgr.24118) 849 : cluster [DBG] pgmap v1231: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:55:39.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:38 smithi104 bash[9203]: cluster 2020-09-08T15:55:37.876272+0000 mgr.x (mgr.24118) 849 : cluster [DBG] pgmap v1231: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:55:39.826 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:55:39.930 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:55:40.175 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:55:39 smithi104 bash[22305]: debug 2020-09-08T15:55:39.827+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:40.343 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2020-09-08T15:55:40.497 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:55:41.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:40 smithi166 bash[9446]: cluster 2020-09-08T15:55:39.876750+0000 mgr.x (mgr.24118) 850 : cluster [DBG] pgmap v1232: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:55:41.127 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:40 smithi104 bash[9203]: cluster 2020-09-08T15:55:39.876750+0000 mgr.x (mgr.24118) 850 : cluster [DBG] pgmap v1232: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:55:41.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:40 smithi055 bash[9296]: cluster 2020-09-08T15:55:39.876750+0000 mgr.x (mgr.24118) 850 : cluster [DBG] pgmap v1232: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:55:43.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:42 smithi166 bash[9446]: cluster 2020-09-08T15:55:41.816521+0000 osd.1 (osd.1) 92 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:43.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:42 smithi166 bash[9446]: cluster 2020-09-08T15:55:41.816632+0000 osd.1 (osd.1) 93 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:43.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:42 smithi166 bash[9446]: cluster 2020-09-08T15:55:41.842337+0000 mon.a (mon.0) 1672 : cluster [DBG] osdmap e467: 12 total, 12 up, 12 in 2020-09-08T15:55:43.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:42 smithi166 bash[9446]: cluster 2020-09-08T15:55:41.877466+0000 mgr.x (mgr.24118) 851 : cluster [DBG] pgmap v1234: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 761 op/s 2020-09-08T15:55:43.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:42 smithi055 bash[9296]: cluster 2020-09-08T15:55:41.816521+0000 osd.1 (osd.1) 92 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:43.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:42 smithi055 bash[9296]: cluster 2020-09-08T15:55:41.816632+0000 osd.1 (osd.1) 93 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:43.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:42 smithi055 bash[9296]: cluster 2020-09-08T15:55:41.842337+0000 mon.a (mon.0) 1672 : cluster [DBG] osdmap e467: 12 total, 12 up, 12 in 2020-09-08T15:55:43.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:42 smithi055 bash[9296]: cluster 2020-09-08T15:55:41.877466+0000 mgr.x (mgr.24118) 851 : cluster [DBG] pgmap v1234: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 761 op/s 2020-09-08T15:55:43.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:42 smithi104 bash[9203]: cluster 2020-09-08T15:55:41.816521+0000 osd.1 (osd.1) 92 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:42 smithi104 bash[9203]: cluster 2020-09-08T15:55:41.816632+0000 osd.1 (osd.1) 93 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:42 smithi104 bash[9203]: cluster 2020-09-08T15:55:41.842337+0000 mon.a (mon.0) 1672 : cluster [DBG] osdmap e467: 12 total, 12 up, 12 in 2020-09-08T15:55:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:42 smithi104 bash[9203]: cluster 2020-09-08T15:55:41.877466+0000 mgr.x (mgr.24118) 851 : cluster [DBG] pgmap v1234: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 761 op/s 2020-09-08T15:55:43.453 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:55:43.555 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:43.672 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:43.778 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:43.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:55:43 smithi055 bash[19003]: debug 2020-09-08T15:55:43.672+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:43.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:55:43 smithi055 bash[14301]: debug 2020-09-08T15:55:43.448+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:43.961 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:44.050 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:55:43 smithi166 bash[18960]: debug 2020-09-08T15:55:43.957+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:44.066 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:55:44.289 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:55:44.289 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:55:44.308 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:55:44.412 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:55:44.420 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:55:44 smithi104 bash[17588]: debug 2020-09-08T15:55:44.311+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:44.642 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:55:44.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:55:44 smithi104 bash[17588]: debug 2020-09-08T15:55:44.639+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:44.746 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:44.928 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:45.030 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:45.032 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:55:45.070 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:55:45.110 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:55:45.161 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:55:45.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:44 smithi104 bash[9203]: cluster 2020-09-08T15:55:43.878066+0000 mgr.x (mgr.24118) 852 : cluster [DBG] pgmap v1235: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 899 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T15:55:45.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:55:44 smithi104 bash[15273]: debug 2020-09-08T15:55:44.927+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:45.195 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:44 smithi166 bash[9446]: cluster 2020-09-08T15:55:43.878066+0000 mgr.x (mgr.24118) 852 : cluster [DBG] pgmap v1235: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 899 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T15:55:45.196 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:45.204 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:55:45.250 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:55:45.282 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:55:45.299 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:55:45.306 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2020-09-08T15:55:45.311 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:55:45.344 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:55:45.401 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:55:45.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:44 smithi055 bash[9296]: cluster 2020-09-08T15:55:43.878066+0000 mgr.x (mgr.24118) 852 : cluster [DBG] pgmap v1235: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 899 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T15:55:45.416 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:55:45.436 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:55:45.474 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:55:45.481 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:55:45.481 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:55:45.481 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:55:45.482 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:55:45.482 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:55:45.482 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:55:45.482 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:55:45.483 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:55:45.483 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:55:45.483 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:55:45.483 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:55:45.484 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:55:45.497 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:55:45.537 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:45.570 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:55:45 smithi166 bash[18960]: debug 2020-09-08T15:55:45.193+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:45.571 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:55:45 smithi166 bash[14254]: debug 2020-09-08T15:55:45.413+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:45.939 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:55:46.177 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:46.281 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:46.291 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:45 smithi055 bash[9296]: audit 2020-09-08T15:55:44.985909+0000 mon.a (mon.0) 1673 : audit [DBG] from='client.? v1:172.21.15.104:0/2901333835' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:55:46.291 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:45 smithi055 bash[9296]: audit 2020-09-08T15:55:45.302534+0000 mon.a (mon.0) 1674 : audit [INF] from='client.25843 v1:172.21.15.104:0/2850337250' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:55:46.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:45 smithi166 bash[9446]: audit 2020-09-08T15:55:44.985909+0000 mon.a (mon.0) 1673 : audit [DBG] from='client.? v1:172.21.15.104:0/2901333835' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:55:46.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:45 smithi166 bash[9446]: audit 2020-09-08T15:55:45.302534+0000 mon.a (mon.0) 1674 : audit [INF] from='client.25843 v1:172.21.15.104:0/2850337250' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:55:46.368 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:55:46 smithi166 bash[16593]: debug 2020-09-08T15:55:46.173+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:46.409 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:46.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:45 smithi104 bash[9203]: audit 2020-09-08T15:55:44.985909+0000 mon.a (mon.0) 1673 : audit [DBG] from='client.? v1:172.21.15.104:0/2901333835' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:55:46.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:45 smithi104 bash[9203]: audit 2020-09-08T15:55:45.302534+0000 mon.a (mon.0) 1674 : audit [INF] from='client.25843 v1:172.21.15.104:0/2850337250' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:55:46.513 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:46.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:55:46 smithi055 bash[16646]: debug 2020-09-08T15:55:46.408+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:47.174 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:47.281 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:47.291 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:46 smithi166 bash[9446]: cluster 2020-09-08T15:55:45.770750+0000 osd.6 (osd.6) 77 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:47.291 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:46 smithi166 bash[9446]: cluster 2020-09-08T15:55:45.770847+0000 osd.6 (osd.6) 78 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:47.291 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:46 smithi166 bash[9446]: cluster 2020-09-08T15:55:45.878560+0000 mgr.x (mgr.24118) 853 : cluster [DBG] pgmap v1236: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 843 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 991 op/s 2020-09-08T15:55:47.292 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:46 smithi166 bash[9446]: cluster 2020-09-08T15:55:45.932742+0000 mon.a (mon.0) 1675 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:55:47.292 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:46 smithi166 bash[9446]: cluster 2020-09-08T15:55:45.932770+0000 mon.a (mon.0) 1676 : cluster [INF] Cluster is now healthy 2020-09-08T15:55:47.292 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:46 smithi166 bash[9446]: audit 2020-09-08T15:55:45.942046+0000 mon.a (mon.0) 1677 : audit [INF] from='client.25843 v1:172.21.15.104:0/2850337250' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:55:47.292 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:46 smithi166 bash[9446]: cluster 2020-09-08T15:55:45.942100+0000 mon.a (mon.0) 1678 : cluster [DBG] osdmap e468: 12 total, 12 up, 12 in 2020-09-08T15:55:47.405 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:47.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:46 smithi055 bash[9296]: cluster 2020-09-08T15:55:45.770750+0000 osd.6 (osd.6) 77 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:47.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:46 smithi055 bash[9296]: cluster 2020-09-08T15:55:45.770847+0000 osd.6 (osd.6) 78 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:47.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:46 smithi055 bash[9296]: cluster 2020-09-08T15:55:45.878560+0000 mgr.x (mgr.24118) 853 : cluster [DBG] pgmap v1236: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 843 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 991 op/s 2020-09-08T15:55:47.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:46 smithi055 bash[9296]: cluster 2020-09-08T15:55:45.932742+0000 mon.a (mon.0) 1675 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:55:47.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:46 smithi055 bash[9296]: cluster 2020-09-08T15:55:45.932770+0000 mon.a (mon.0) 1676 : cluster [INF] Cluster is now healthy 2020-09-08T15:55:47.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:46 smithi055 bash[9296]: audit 2020-09-08T15:55:45.942046+0000 mon.a (mon.0) 1677 : audit [INF] from='client.25843 v1:172.21.15.104:0/2850337250' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:55:47.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:46 smithi055 bash[9296]: cluster 2020-09-08T15:55:45.942100+0000 mon.a (mon.0) 1678 : cluster [DBG] osdmap e468: 12 total, 12 up, 12 in 2020-09-08T15:55:47.425 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:55:47 smithi104 bash[15273]: debug 2020-09-08T15:55:47.175+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:47.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:46 smithi104 bash[9203]: cluster 2020-09-08T15:55:45.770750+0000 osd.6 (osd.6) 77 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:47.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:46 smithi104 bash[9203]: cluster 2020-09-08T15:55:45.770847+0000 osd.6 (osd.6) 78 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:47.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:46 smithi104 bash[9203]: cluster 2020-09-08T15:55:45.878560+0000 mgr.x (mgr.24118) 853 : cluster [DBG] pgmap v1236: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 843 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 991 op/s 2020-09-08T15:55:47.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:46 smithi104 bash[9203]: cluster 2020-09-08T15:55:45.932742+0000 mon.a (mon.0) 1675 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:55:47.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:46 smithi104 bash[9203]: cluster 2020-09-08T15:55:45.932770+0000 mon.a (mon.0) 1676 : cluster [INF] Cluster is now healthy 2020-09-08T15:55:47.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:46 smithi104 bash[9203]: audit 2020-09-08T15:55:45.942046+0000 mon.a (mon.0) 1677 : audit [INF] from='client.25843 v1:172.21.15.104:0/2850337250' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:55:47.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:46 smithi104 bash[9203]: cluster 2020-09-08T15:55:45.942100+0000 mon.a (mon.0) 1678 : cluster [DBG] osdmap e468: 12 total, 12 up, 12 in 2020-09-08T15:55:47.509 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:47.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:55:47 smithi166 bash[11928]: debug 2020-09-08T15:55:47.401+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:48.761 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:48.864 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:48.994 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:49.097 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:55:49.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:48 smithi055 bash[9296]: cluster 2020-09-08T15:55:47.879046+0000 mgr.x (mgr.24118) 854 : cluster [DBG] pgmap v1238: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 672 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.03k op/s 2020-09-08T15:55:49.158 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:55:48 smithi055 bash[16646]: debug 2020-09-08T15:55:48.760+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:49.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:55:49 smithi104 bash[19922]: debug 2020-09-08T15:55:48.999+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:49.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:48 smithi104 bash[9203]: cluster 2020-09-08T15:55:47.879046+0000 mgr.x (mgr.24118) 854 : cluster [DBG] pgmap v1238: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 672 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.03k op/s 2020-09-08T15:55:49.214 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:55:49.322 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:49.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:48 smithi166 bash[9446]: cluster 2020-09-08T15:55:47.879046+0000 mgr.x (mgr.24118) 854 : cluster [DBG] pgmap v1238: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 672 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.03k op/s 2020-09-08T15:55:49.449 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:49.552 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:55:49.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:55:49 smithi055 bash[16646]: debug 2020-09-08T15:55:49.448+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:49.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:55:49 smithi104 bash[17588]: debug 2020-09-08T15:55:49.215+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:49.724 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:55:49.762 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:55:49.827 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:49.844 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:55:49.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:55:49 smithi166 bash[14254]: debug 2020-09-08T15:55:49.717+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:49.935 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:55:49.979 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:55:49.980 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T15:55:49.980 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:49.980 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:282036 5.8 5:1bfe5fcc:::benchmark_data_smithi166_37820_object137614:head [delete] snapc 0=[] ondisk+write+known_if_redirected e468)", 2020-09-08T15:55:49.980 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:55:49.966465+0000", 2020-09-08T15:55:49.980 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 0.015966456, 2020-09-08T15:55:49.981 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.015992724999999999, 2020-09-08T15:55:49.981 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:49.981 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:55:49.982 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:49.982 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T15:55:49.982 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:55:49.983 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 282036 2020-09-08T15:55:49.983 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:49.983 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:49.983 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:49.984 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:49.984 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:49.966465+0000", 2020-09-08T15:55:49.984 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:49.985 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:49.985 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:49.985 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:49.985 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:49.966465+0000", 2020-09-08T15:55:49.985 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999514 2020-09-08T15:55:49.986 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:49.986 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:49.986 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:49.986 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:49.966416+0000", 2020-09-08T15:55:49.986 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016567900000000001 2020-09-08T15:55:49.987 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:49.987 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:49.987 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:49.987 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:49.966582+0000", 2020-09-08T15:55:49.988 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.9981000000000001e-05 2020-09-08T15:55:49.988 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:49.988 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:49.988 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:49.988 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:49.966602+0000", 2020-09-08T15:55:49.988 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1718e-05 2020-09-08T15:55:49.989 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:49.989 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:49.989 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:49.989 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:49.966614+0000", 2020-09-08T15:55:49.989 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0025953439999999999 2020-09-08T15:55:49.990 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:49.990 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:49.990 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:49.990 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:49.969209+0000", 2020-09-08T15:55:49.991 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000198569 2020-09-08T15:55:49.991 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:49.991 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:49.991 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:55:49.991 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:49.969408+0000", 2020-09-08T15:55:49.991 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000216517 2020-09-08T15:55:49.992 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:49.992 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:49.992 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T15:55:49.992 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:49.969624+0000", 2020-09-08T15:55:49.993 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0031590749999999999 2020-09-08T15:55:49.993 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:49.993 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:49.994 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:49.994 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:49.994 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:49.994 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:282042 5.d 5.24d16ccd (undecoded) ondisk+write+known_if_redirected e469)", 2020-09-08T15:55:49.994 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:55:49.974221+0000", 2020-09-08T15:55:49.995 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 0.0082107359999999997, 2020-09-08T15:55:49.995 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0083011479999999995, 2020-09-08T15:55:49.995 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:55:49.995 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "delayed", 2020-09-08T15:55:49.995 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:55:49.996 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T15:55:49.996 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:55:49.996 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 282042 2020-09-08T15:55:49.996 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:49.996 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:55:49.996 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:49.997 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:55:49.997 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:49.974221+0000", 2020-09-08T15:55:49.997 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:55:49.997 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:49.998 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:49.998 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:55:49.998 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:49.974221+0000", 2020-09-08T15:55:49.999 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999566 2020-09-08T15:55:50.000 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:50.000 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:50.000 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:55:50.000 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:49.974178+0000", 2020-09-08T15:55:50.000 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.4158000000000002e-05 2020-09-08T15:55:50.001 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:50.001 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:50.001 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:55:50.001 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:49.974272+0000", 2020-09-08T15:55:50.001 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.0035e-05 2020-09-08T15:55:50.002 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:50.002 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:50.002 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:55:50.002 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:49.974292+0000", 2020-09-08T15:55:50.002 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7940000000000003e-06 2020-09-08T15:55:50.003 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:50.003 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:50.003 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:55:50.004 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:49.974300+0000", 2020-09-08T15:55:50.004 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.1185000000000003e-05 2020-09-08T15:55:50.004 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:50.004 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:50.004 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:55:50.005 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:49.974381+0000", 2020-09-08T15:55:50.005 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.4200000000000007e-06 2020-09-08T15:55:50.005 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:55:50.005 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:55:50.005 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op must wait for map", 2020-09-08T15:55:50.006 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:49.974389+0000", 2020-09-08T15:55:50.006 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016822199999999999 2020-09-08T15:55:50.006 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:50.006 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:55:50.007 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:50.007 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:55:50.007 INFO:teuthology.orchestra.run.smithi166.stdout: ], 2020-09-08T15:55:50.007 INFO:teuthology.orchestra.run.smithi166.stdout: "num_ops": 2 2020-09-08T15:55:50.007 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:55:50.008 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:50.039 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:55:50.110 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:55:50.147 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:55:49 smithi104 bash[19922]: debug 2020-09-08T15:55:49.983+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:50.148 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:49 smithi104 bash[9203]: cluster 2020-09-08T15:55:49.184230+0000 osd.11 (osd.11) 82 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:50.148 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:49 smithi104 bash[9203]: cluster 2020-09-08T15:55:49.184331+0000 osd.11 (osd.11) 83 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:50.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:49 smithi166 bash[9446]: cluster 2020-09-08T15:55:49.184230+0000 osd.11 (osd.11) 82 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:50.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:49 smithi166 bash[9446]: cluster 2020-09-08T15:55:49.184331+0000 osd.11 (osd.11) 83 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:50.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:49 smithi055 bash[9296]: cluster 2020-09-08T15:55:49.184230+0000 osd.11 (osd.11) 82 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:50.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:49 smithi055 bash[9296]: cluster 2020-09-08T15:55:49.184331+0000 osd.11 (osd.11) 83 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:51.035 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2020-09-08T15:55:51.179 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:55:51.326 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:55:51.327 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:50 smithi104 bash[9203]: cluster 2020-09-08T15:55:49.879589+0000 mgr.x (mgr.24118) 855 : cluster [DBG] pgmap v1239: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 578 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T15:55:51.327 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:50 smithi104 bash[9203]: cluster 2020-09-08T15:55:49.975560+0000 mon.a (mon.0) 1679 : cluster [DBG] osdmap e469: 12 total, 12 up, 12 in 2020-09-08T15:55:51.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:50 smithi166 bash[9446]: cluster 2020-09-08T15:55:49.879589+0000 mgr.x (mgr.24118) 855 : cluster [DBG] pgmap v1239: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 578 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T15:55:51.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:50 smithi166 bash[9446]: cluster 2020-09-08T15:55:49.975560+0000 mon.a (mon.0) 1679 : cluster [DBG] osdmap e469: 12 total, 12 up, 12 in 2020-09-08T15:55:51.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:50 smithi055 bash[9296]: cluster 2020-09-08T15:55:49.879589+0000 mgr.x (mgr.24118) 855 : cluster [DBG] pgmap v1239: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 578 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T15:55:51.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:50 smithi055 bash[9296]: cluster 2020-09-08T15:55:49.975560+0000 mon.a (mon.0) 1679 : cluster [DBG] osdmap e469: 12 total, 12 up, 12 in 2020-09-08T15:55:51.430 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:51.675 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:55:51 smithi104 bash[17588]: debug 2020-09-08T15:55:51.327+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:51.917 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:52.020 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:52.366 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:55:51 smithi166 bash[16593]: debug 2020-09-08T15:55:51.917+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:52.548 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:52.654 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:52.799 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:55:52 smithi166 bash[11928]: debug 2020-09-08T15:55:52.537+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:52.799 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:52.903 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:53.037 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:53.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:52 smithi166 bash[9446]: cluster 2020-09-08T15:55:51.880063+0000 mgr.x (mgr.24118) 856 : cluster [DBG] pgmap v1241: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 541 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 716 op/s 2020-09-08T15:55:53.117 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:55:52 smithi166 bash[11928]: debug 2020-09-08T15:55:52.797+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:53.142 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:53.361 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:53.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:55:53 smithi055 bash[19003]: debug 2020-09-08T15:55:53.040+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:53.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:52 smithi055 bash[9296]: cluster 2020-09-08T15:55:51.880063+0000 mgr.x (mgr.24118) 856 : cluster [DBG] pgmap v1241: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 541 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 716 op/s 2020-09-08T15:55:53.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:52 smithi104 bash[9203]: cluster 2020-09-08T15:55:51.880063+0000 mgr.x (mgr.24118) 856 : cluster [DBG] pgmap v1241: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 541 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 716 op/s 2020-09-08T15:55:53.462 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:53.471 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:55:53 smithi166 bash[16593]: debug 2020-09-08T15:55:53.353+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:54.121 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:54.226 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:54.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:55:54 smithi166 bash[11928]: debug 2020-09-08T15:55:54.121+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:54.710 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:54.814 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:54.973 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:55.002 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:55 smithi166 bash[9446]: cluster 2020-09-08T15:55:53.439942+0000 osd.4 (osd.4) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:55.002 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:55 smithi166 bash[9446]: cluster 2020-09-08T15:55:53.440029+0000 osd.4 (osd.4) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:55.002 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:55 smithi166 bash[9446]: cluster 2020-09-08T15:55:53.985835+0000 mon.a (mon.0) 1680 : cluster [DBG] osdmap e470: 12 total, 12 up, 12 in 2020-09-08T15:55:55.003 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:55 smithi166 bash[9446]: audit 2020-09-08T15:55:53.993105+0000 mon.a (mon.0) 1681 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:55:55.003 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:55 smithi166 bash[9446]: audit 2020-09-08T15:55:53.998275+0000 mon.a (mon.0) 1682 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:55:55.004 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:55:55 smithi166 bash[16593]: debug 2020-09-08T15:55:54.969+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:55.005 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:55:54 smithi166 bash[18960]: debug 2020-09-08T15:55:54.705+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:55.081 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:55.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:54 smithi055 bash[9296]: cluster 2020-09-08T15:55:53.439942+0000 osd.4 (osd.4) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:55.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:54 smithi055 bash[9296]: cluster 2020-09-08T15:55:53.440029+0000 osd.4 (osd.4) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:55.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:54 smithi055 bash[9296]: cluster 2020-09-08T15:55:53.985835+0000 mon.a (mon.0) 1680 : cluster [DBG] osdmap e470: 12 total, 12 up, 12 in 2020-09-08T15:55:55.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:54 smithi055 bash[9296]: audit 2020-09-08T15:55:53.993105+0000 mon.a (mon.0) 1681 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:55:55.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:54 smithi055 bash[9296]: audit 2020-09-08T15:55:53.998275+0000 mon.a (mon.0) 1682 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:55:55.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:54 smithi104 bash[9203]: cluster 2020-09-08T15:55:53.439942+0000 osd.4 (osd.4) 49 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:55:55.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:54 smithi104 bash[9203]: cluster 2020-09-08T15:55:53.440029+0000 osd.4 (osd.4) 50 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:55:55.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:54 smithi104 bash[9203]: cluster 2020-09-08T15:55:53.985835+0000 mon.a (mon.0) 1680 : cluster [DBG] osdmap e470: 12 total, 12 up, 12 in 2020-09-08T15:55:55.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:54 smithi104 bash[9203]: audit 2020-09-08T15:55:53.993105+0000 mon.a (mon.0) 1681 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:55:55.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:54 smithi104 bash[9203]: audit 2020-09-08T15:55:53.998275+0000 mon.a (mon.0) 1682 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:55:55.455 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:55.559 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:55.794 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:55.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:55:55 smithi166 bash[11928]: debug 2020-09-08T15:55:55.453+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:55.897 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:55.905 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:55:55 smithi104 bash[19922]: debug 2020-09-08T15:55:55.791+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:56.120 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:55:56.121 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T15:55:56.121 INFO:teuthology.orchestra.run.smithi166.stdout: "complaint_time": 30, 2020-09-08T15:55:56.121 INFO:teuthology.orchestra.run.smithi166.stdout: "num_blocked_ops": 0 2020-09-08T15:55:56.121 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:55:56.138 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:56.175 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:55:56 smithi104 bash[15273]: debug 2020-09-08T15:55:56.139+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:56.244 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:56.371 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:55:56.476 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:56.588 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:55:56 smithi055 bash[19003]: debug 2020-09-08T15:55:56.372+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:56.592 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:56.663 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:55:56.698 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:56.722 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:55:56.759 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:55:56.792 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:55:56.830 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:55:56.831 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:56.853 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:55:56.886 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:55:56.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:55:56 smithi055 bash[16646]: debug 2020-09-08T15:55:56.592+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:56.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:55:56 smithi055 bash[16646]: debug 2020-09-08T15:55:56.824+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:56.926 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:55:56.932 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:56.988 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:55:57.021 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:55:57.033 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:55:57.042 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Removed 144419 objects 2020-09-08T15:55:57.042 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Clean up completed and total clean up time :152.662 2020-09-08T15:55:57.048 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:57.068 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:55:57.095 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:55:57.096 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:55:57.097 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:55:57.097 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:55:57.097 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:55:57.097 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:55:57.097 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:55:57.098 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:55:57.098 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:55:57.098 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:55:57.098 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:55:57.098 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:55:57.099 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:55:57.153 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:57.215 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2020-09-08T15:55:57.304 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:56 smithi166 bash[9446]: cluster 2020-09-08T15:55:55.881095+0000 mgr.x (mgr.24118) 858 : cluster [DBG] pgmap v1244: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 230 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 884 op/s 2020-09-08T15:55:57.305 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:56 smithi166 bash[9446]: audit 2020-09-08T15:55:56.622579+0000 mon.a (mon.0) 1683 : audit [DBG] from='client.? v1:172.21.15.104:0/2112575399' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:55:57.305 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:56 smithi166 bash[9446]: audit 2020-09-08T15:55:56.949266+0000 mon.a (mon.0) 1684 : audit [INF] from='client.? v1:172.21.15.104:0/2053844357' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:55:57.305 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:55:57.344 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:55:57.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:56 smithi055 bash[9296]: cluster 2020-09-08T15:55:55.881095+0000 mgr.x (mgr.24118) 858 : cluster [DBG] pgmap v1244: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 230 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 884 op/s 2020-09-08T15:55:57.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:56 smithi055 bash[9296]: audit 2020-09-08T15:55:56.622579+0000 mon.a (mon.0) 1683 : audit [DBG] from='client.? v1:172.21.15.104:0/2112575399' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:55:57.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:56 smithi055 bash[9296]: audit 2020-09-08T15:55:56.949266+0000 mon.a (mon.0) 1684 : audit [INF] from='client.? v1:172.21.15.104:0/2053844357' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:55:57.409 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:57.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:55:57 smithi104 bash[19922]: debug 2020-09-08T15:55:57.051+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:57.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:56 smithi104 bash[9203]: cluster 2020-09-08T15:55:55.881095+0000 mgr.x (mgr.24118) 858 : cluster [DBG] pgmap v1244: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 230 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 884 op/s 2020-09-08T15:55:57.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:56 smithi104 bash[9203]: audit 2020-09-08T15:55:56.622579+0000 mon.a (mon.0) 1683 : audit [DBG] from='client.? v1:172.21.15.104:0/2112575399' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:55:57.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:56 smithi104 bash[9203]: audit 2020-09-08T15:55:56.949266+0000 mon.a (mon.0) 1684 : audit [INF] from='client.? v1:172.21.15.104:0/2053844357' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:55:57.534 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:55:57.618 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:55:57 smithi166 bash[11928]: debug 2020-09-08T15:55:57.301+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:57.638 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:57.834 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_3 2020-09-08T15:55:57.836 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool rm unique_pool_3 unique_pool_3 --yes-i-really-really-mean-it 2020-09-08T15:55:57.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:55:57 smithi055 bash[16646]: debug 2020-09-08T15:55:57.536+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:57.981 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:55:58.029 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:55:58.134 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:58.147 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:57 smithi104 bash[9203]: cluster 2020-09-08T15:55:56.987101+0000 mon.a (mon.0) 1685 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:55:58.147 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:57 smithi104 bash[9203]: audit 2020-09-08T15:55:56.999934+0000 mon.a (mon.0) 1686 : audit [INF] from='client.? v1:172.21.15.104:0/2053844357' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:55:58.147 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:57 smithi104 bash[9203]: cluster 2020-09-08T15:55:56.999977+0000 mon.a (mon.0) 1687 : cluster [DBG] osdmap e471: 12 total, 12 up, 12 in 2020-09-08T15:55:58.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:55:58 smithi166 bash[18960]: debug 2020-09-08T15:55:58.029+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:58.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:57 smithi166 bash[9446]: cluster 2020-09-08T15:55:56.987101+0000 mon.a (mon.0) 1685 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:55:58.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:57 smithi166 bash[9446]: audit 2020-09-08T15:55:56.999934+0000 mon.a (mon.0) 1686 : audit [INF] from='client.? v1:172.21.15.104:0/2053844357' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:55:58.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:57 smithi166 bash[9446]: cluster 2020-09-08T15:55:56.999977+0000 mon.a (mon.0) 1687 : cluster [DBG] osdmap e471: 12 total, 12 up, 12 in 2020-09-08T15:55:58.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:57 smithi055 bash[9296]: cluster 2020-09-08T15:55:56.987101+0000 mon.a (mon.0) 1685 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:55:58.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:57 smithi055 bash[9296]: audit 2020-09-08T15:55:56.999934+0000 mon.a (mon.0) 1686 : audit [INF] from='client.? v1:172.21.15.104:0/2053844357' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:55:58.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:57 smithi055 bash[9296]: cluster 2020-09-08T15:55:56.999977+0000 mon.a (mon.0) 1687 : cluster [DBG] osdmap e471: 12 total, 12 up, 12 in 2020-09-08T15:55:58.641 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:55:58.745 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:55:58.863 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:55:58.925 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:55:58 smithi104 bash[19922]: debug 2020-09-08T15:55:58.643+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:58.964 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:59.080 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:55:58 smithi166 bash[9446]: cluster 2020-09-08T15:55:57.881873+0000 mgr.x (mgr.24118) 859 : cluster [DBG] pgmap v1246: 17 pgs: 17 active+clean; 76 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:55:59.080 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:55:58 smithi166 bash[14254]: debug 2020-09-08T15:55:58.857+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:59.234 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:55:59 smithi104 bash[9203]: cluster 2020-09-08T15:55:57.881873+0000 mgr.x (mgr.24118) 859 : cluster [DBG] pgmap v1246: 17 pgs: 17 active+clean; 76 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:55:59.235 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:55:59.339 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:59.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:55:59 smithi055 bash[9296]: cluster 2020-09-08T15:55:57.881873+0000 mgr.x (mgr.24118) 859 : cluster [DBG] pgmap v1246: 17 pgs: 17 active+clean; 76 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:55:59.573 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:55:59 smithi104 bash[15273]: debug 2020-09-08T15:55:59.235+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:55:59.801 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:55:59.906 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:00.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:55:59 smithi166 bash[16593]: debug 2020-09-08T15:55:59.801+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:00.239 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:00.345 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:00.542 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:00.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:56:00 smithi166 bash[14254]: debug 2020-09-08T15:56:00.237+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:00.648 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:00.675 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:56:00 smithi104 bash[17588]: debug 2020-09-08T15:56:00.543+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:00.884 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:00.990 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:01.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:56:00 smithi166 bash[16593]: debug 2020-09-08T15:56:00.881+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:01.138 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:00 smithi166 bash[9446]: cluster 2020-09-08T15:55:59.882637+0000 mgr.x (mgr.24118) 860 : cluster [DBG] pgmap v1247: 17 pgs: 17 active+clean; 9.7 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.06k op/s 2020-09-08T15:56:01.143 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:56:01.143 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:56:01.145 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:00 smithi055 bash[9296]: cluster 2020-09-08T15:55:59.882637+0000 mgr.x (mgr.24118) 860 : cluster [DBG] pgmap v1247: 17 pgs: 17 active+clean; 9.7 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.06k op/s 2020-09-08T15:56:01.145 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:01.161 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:00 smithi104 bash[9203]: cluster 2020-09-08T15:55:59.882637+0000 mgr.x (mgr.24118) 860 : cluster [DBG] pgmap v1247: 17 pgs: 17 active+clean; 9.7 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.06k op/s 2020-09-08T15:56:01.250 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:01.284 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:56:01.286 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:56:01.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:56:01 smithi055 bash[14301]: debug 2020-09-08T15:56:01.132+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:01.434 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:56:01.434 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:56:01.581 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:01.686 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:56:01.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:56:01 smithi104 bash[22305]: debug 2020-09-08T15:56:01.583+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:01.987 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:56:02.092 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:02.188 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2020-09-08T15:56:02.229 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:02.322 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:56:02.333 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:02.349 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:56:01 smithi104 bash[19922]: debug 2020-09-08T15:56:01.991+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:02.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:56:02 smithi055 bash[11956]: debug 2020-09-08T15:56:02.220+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:02.942 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:03.054 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:03.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:56:02 smithi104 bash[15273]: debug 2020-09-08T15:56:02.943+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:03.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:02 smithi104 bash[9203]: cluster 2020-09-08T15:56:01.883248+0000 mgr.x (mgr.24118) 861 : cluster [DBG] pgmap v1248: 17 pgs: 17 active+clean; 448 KiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 571 op/s 2020-09-08T15:56:03.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:02 smithi104 bash[9203]: cluster 2020-09-08T15:56:02.615879+0000 osd.9 (osd.9) 73 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:03.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:02 smithi104 bash[9203]: cluster 2020-09-08T15:56:02.616007+0000 osd.9 (osd.9) 74 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:03.183 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:03.192 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:56:03 smithi055 bash[16646]: debug 2020-09-08T15:56:03.184+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:03.193 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:02 smithi055 bash[9296]: cluster 2020-09-08T15:56:01.883248+0000 mgr.x (mgr.24118) 861 : cluster [DBG] pgmap v1248: 17 pgs: 17 active+clean; 448 KiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 571 op/s 2020-09-08T15:56:03.193 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:02 smithi055 bash[9296]: cluster 2020-09-08T15:56:02.615879+0000 osd.9 (osd.9) 73 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:03.193 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:02 smithi055 bash[9296]: cluster 2020-09-08T15:56:02.616007+0000 osd.9 (osd.9) 74 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:03.294 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:03.308 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:02 smithi166 bash[9446]: cluster 2020-09-08T15:56:01.883248+0000 mgr.x (mgr.24118) 861 : cluster [DBG] pgmap v1248: 17 pgs: 17 active+clean; 448 KiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 571 op/s 2020-09-08T15:56:03.308 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:02 smithi166 bash[9446]: cluster 2020-09-08T15:56:02.615879+0000 osd.9 (osd.9) 73 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:03.309 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:02 smithi166 bash[9446]: cluster 2020-09-08T15:56:02.616007+0000 osd.9 (osd.9) 74 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:03.431 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:03.538 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:03.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:56:03 smithi166 bash[14254]: debug 2020-09-08T15:56:03.429+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:03.651 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:03.757 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:03.885 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:56:03 smithi055 bash[11956]: debug 2020-09-08T15:56:03.652+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:03.885 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:03.959 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'unique_pool_3' removed 2020-09-08T15:56:03.992 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:56:04.121 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:56:04.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:03 smithi055 bash[9296]: cluster 2020-09-08T15:56:02.952861+0000 mon.a (mon.0) 1688 : cluster [DBG] osdmap e472: 12 total, 12 up, 12 in 2020-09-08T15:56:04.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:03 smithi055 bash[9296]: audit 2020-09-08T15:56:03.291325+0000 mon.a (mon.0) 1689 : audit [INF] from='client.? v1:172.21.15.104:0/2201553238' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T15:56:04.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:56:03 smithi055 bash[19003]: debug 2020-09-08T15:56:03.888+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:04.222 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:03 smithi104 bash[9203]: cluster 2020-09-08T15:56:02.952861+0000 mon.a (mon.0) 1688 : cluster [DBG] osdmap e472: 12 total, 12 up, 12 in 2020-09-08T15:56:04.223 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:03 smithi104 bash[9203]: audit 2020-09-08T15:56:03.291325+0000 mon.a (mon.0) 1689 : audit [INF] from='client.? v1:172.21.15.104:0/2201553238' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T15:56:04.228 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:04.240 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:56:04 smithi166 bash[11928]: debug 2020-09-08T15:56:04.117+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:04.241 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:03 smithi166 bash[9446]: cluster 2020-09-08T15:56:02.952861+0000 mon.a (mon.0) 1688 : cluster [DBG] osdmap e472: 12 total, 12 up, 12 in 2020-09-08T15:56:04.241 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:03 smithi166 bash[9446]: audit 2020-09-08T15:56:03.291325+0000 mon.a (mon.0) 1689 : audit [INF] from='client.? v1:172.21.15.104:0/2201553238' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T15:56:04.348 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:04.453 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:04.580 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:56:04 smithi166 bash[18960]: debug 2020-09-08T15:56:04.345+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:04.581 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:04.685 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:04.816 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:04.870 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:56:04 smithi166 bash[18960]: debug 2020-09-08T15:56:04.577+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:04.870 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:56:04 smithi166 bash[18960]: debug 2020-09-08T15:56:04.813+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:04.921 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:05.199 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:04 smithi104 bash[9203]: cluster 2020-09-08T15:56:03.884123+0000 mgr.x (mgr.24118) 862 : cluster [DBG] pgmap v1250: 17 pgs: 17 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 460 op/s 2020-09-08T15:56:05.199 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:04 smithi104 bash[9203]: audit 2020-09-08T15:56:03.962141+0000 mon.a (mon.0) 1690 : audit [INF] from='client.? v1:172.21.15.104:0/2201553238' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T15:56:05.199 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:04 smithi104 bash[9203]: cluster 2020-09-08T15:56:03.962252+0000 mon.a (mon.0) 1691 : cluster [DBG] osdmap e473: 12 total, 12 up, 12 in 2020-09-08T15:56:05.200 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:04 smithi104 bash[9203]: cluster 2020-09-08T15:56:04.476459+0000 osd.2 (osd.2) 68 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:05.200 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:04 smithi104 bash[9203]: cluster 2020-09-08T15:56:04.476615+0000 osd.2 (osd.2) 69 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:05.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:04 smithi166 bash[9446]: cluster 2020-09-08T15:56:03.884123+0000 mgr.x (mgr.24118) 862 : cluster [DBG] pgmap v1250: 17 pgs: 17 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 460 op/s 2020-09-08T15:56:05.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:04 smithi166 bash[9446]: audit 2020-09-08T15:56:03.962141+0000 mon.a (mon.0) 1690 : audit [INF] from='client.? v1:172.21.15.104:0/2201553238' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T15:56:05.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:04 smithi166 bash[9446]: cluster 2020-09-08T15:56:03.962252+0000 mon.a (mon.0) 1691 : cluster [DBG] osdmap e473: 12 total, 12 up, 12 in 2020-09-08T15:56:05.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:04 smithi166 bash[9446]: cluster 2020-09-08T15:56:04.476459+0000 osd.2 (osd.2) 68 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:05.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:04 smithi166 bash[9446]: cluster 2020-09-08T15:56:04.476615+0000 osd.2 (osd.2) 69 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:05.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:04 smithi055 bash[9296]: cluster 2020-09-08T15:56:03.884123+0000 mgr.x (mgr.24118) 862 : cluster [DBG] pgmap v1250: 17 pgs: 17 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 460 op/s 2020-09-08T15:56:05.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:04 smithi055 bash[9296]: audit 2020-09-08T15:56:03.962141+0000 mon.a (mon.0) 1690 : audit [INF] from='client.? v1:172.21.15.104:0/2201553238' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T15:56:05.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:04 smithi055 bash[9296]: cluster 2020-09-08T15:56:03.962252+0000 mon.a (mon.0) 1691 : cluster [DBG] osdmap e473: 12 total, 12 up, 12 in 2020-09-08T15:56:05.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:04 smithi055 bash[9296]: cluster 2020-09-08T15:56:04.476459+0000 osd.2 (osd.2) 68 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:05.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:04 smithi055 bash[9296]: cluster 2020-09-08T15:56:04.476615+0000 osd.2 (osd.2) 69 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:05.502 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:05.609 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:05.751 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:05.844 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:56:05 smithi104 bash[22305]: debug 2020-09-08T15:56:05.507+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:05.852 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:05.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:56:05 smithi055 bash[14301]: debug 2020-09-08T15:56:05.744+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:06.146 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:05 smithi104 bash[9203]: cluster 2020-09-08T15:56:04.972357+0000 mon.a (mon.0) 1692 : cluster [DBG] osdmap e474: 12 total, 12 up, 12 in 2020-09-08T15:56:06.146 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:06.250 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:06.309 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:05 smithi166 bash[9446]: cluster 2020-09-08T15:56:04.972357+0000 mon.a (mon.0) 1692 : cluster [DBG] osdmap e474: 12 total, 12 up, 12 in 2020-09-08T15:56:06.310 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2020-09-08T15:56:06.312 INFO:tasks.radosbench:Beginning radosbench... 2020-09-08T15:56:06.312 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_4 2020-09-08T15:56:06.313 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool create unique_pool_4 16 2020-09-08T15:56:06.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:05 smithi055 bash[9296]: cluster 2020-09-08T15:56:04.972357+0000 mon.a (mon.0) 1692 : cluster [DBG] osdmap e474: 12 total, 12 up, 12 in 2020-09-08T15:56:06.427 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:56:06 smithi104 bash[17588]: debug 2020-09-08T15:56:06.147+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:06.452 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:06.557 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:56:06.642 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:56:06 smithi166 bash[14254]: debug 2020-09-08T15:56:06.449+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:06.655 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:56:06.936 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:56:07.042 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:07.169 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:07.275 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:07.289 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:06 smithi055 bash[9296]: cluster 2020-09-08T15:56:05.884909+0000 mgr.x (mgr.24118) 863 : cluster [DBG] pgmap v1253: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:07.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:06 smithi166 bash[9446]: cluster 2020-09-08T15:56:05.884909+0000 mgr.x (mgr.24118) 863 : cluster [DBG] pgmap v1253: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:07.368 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:56:07 smithi166 bash[14254]: debug 2020-09-08T15:56:07.169+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:07.420 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:07.431 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:06 smithi104 bash[9203]: cluster 2020-09-08T15:56:05.884909+0000 mgr.x (mgr.24118) 863 : cluster [DBG] pgmap v1253: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:07.431 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:56:06 smithi104 bash[19922]: debug 2020-09-08T15:56:06.939+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:07.526 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:07.657 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:56:07 smithi055 bash[11956]: debug 2020-09-08T15:56:07.420+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:07.919 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:08.020 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:08.152 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:08.175 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:56:07 smithi104 bash[15273]: debug 2020-09-08T15:56:07.919+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:08.257 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:08.382 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:56:08 smithi055 bash[11956]: debug 2020-09-08T15:56:08.152+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:08.383 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:08.489 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:08.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:56:08 smithi055 bash[16646]: debug 2020-09-08T15:56:08.384+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:08.735 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:08.843 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:08.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:56:08 smithi104 bash[22305]: debug 2020-09-08T15:56:08.739+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:08.960 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:09.067 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:09.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:08 smithi055 bash[9296]: cluster 2020-09-08T15:56:07.885677+0000 mgr.x (mgr.24118) 864 : cluster [DBG] pgmap v1254: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:09.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:56:08 smithi055 bash[19003]: debug 2020-09-08T15:56:08.960+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:09.210 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:09.316 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:09.345 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:08 smithi104 bash[9203]: cluster 2020-09-08T15:56:07.885677+0000 mgr.x (mgr.24118) 864 : cluster [DBG] pgmap v1254: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:09.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:08 smithi166 bash[9446]: cluster 2020-09-08T15:56:07.885677+0000 mgr.x (mgr.24118) 864 : cluster [DBG] pgmap v1254: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:09.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:56:09 smithi166 bash[18960]: debug 2020-09-08T15:56:09.209+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:09.576 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:09.675 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:56:09 smithi104 bash[15273]: debug 2020-09-08T15:56:09.579+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:09.681 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:09.810 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:09.916 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:10.026 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:56:09 smithi055 bash[16646]: debug 2020-09-08T15:56:09.812+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:10.034 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:10.038 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:56:10.041 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:56:10.063 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:56:10.089 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:56:10.113 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:56:10.141 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:10.142 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:56:10.169 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:56:10.196 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:56:10.222 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:56:10.260 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:56:10.263 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:56:10 smithi166 bash[16593]: debug 2020-09-08T15:56:10.033+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:10.263 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:10 smithi166 bash[9446]: audit 2020-09-08T15:56:09.602195+0000 mon.a (mon.0) 1693 : audit [INF] from='client.? v1:172.21.15.104:0/282076038' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:56:10.264 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:10.288 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:56:10.314 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:56:10.346 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:56:10.349 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:56:10.350 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:56:10.350 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:56:10.350 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:56:10.351 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:56:10.351 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:56:10.351 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:56:10.351 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:56:10.351 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:56:10.352 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:56:10.352 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:56:10.352 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:56:10.371 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:56:10.413 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:56:10.413 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T15:56:10.414 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 0 2020-09-08T15:56:10.414 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:56:10.414 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:10 smithi055 bash[9296]: audit 2020-09-08T15:56:09.602195+0000 mon.a (mon.0) 1693 : audit [INF] from='client.? v1:172.21.15.104:0/282076038' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:56:10.421 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:10 smithi104 bash[9203]: audit 2020-09-08T15:56:09.602195+0000 mon.a (mon.0) 1693 : audit [INF] from='client.? v1:172.21.15.104:0/282076038' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:56:10.493 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:56:10.599 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:10.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:56:10 smithi166 bash[11928]: debug 2020-09-08T15:56:10.493+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:10.617 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:56:10 smithi166 bash[16593]: debug 2020-09-08T15:56:10.261+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:10.752 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:10.857 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:10.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:56:10 smithi055 bash[11956]: debug 2020-09-08T15:56:10.756+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:11 smithi166 bash[9446]: cluster 2020-09-08T15:56:09.886475+0000 mgr.x (mgr.24118) 865 : cluster [DBG] pgmap v1255: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:11.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:11 smithi166 bash[9446]: audit 2020-09-08T15:56:10.012923+0000 mon.c (mon.2) 68 : audit [DBG] from='client.? v1:172.21.15.104:0/3389589096' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:56:11.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:11 smithi166 bash[9446]: cluster 2020-09-08T15:56:10.026360+0000 mon.a (mon.0) 1694 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:56:11.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:11 smithi166 bash[9446]: audit 2020-09-08T15:56:10.043908+0000 mon.a (mon.0) 1695 : audit [INF] from='client.? v1:172.21.15.104:0/282076038' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:56:11.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:11 smithi166 bash[9446]: cluster 2020-09-08T15:56:10.043966+0000 mon.a (mon.0) 1696 : cluster [DBG] osdmap e475: 12 total, 12 up, 12 in 2020-09-08T15:56:11.396 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:11 smithi104 bash[9203]: cluster 2020-09-08T15:56:09.886475+0000 mgr.x (mgr.24118) 865 : cluster [DBG] pgmap v1255: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:11.396 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:11 smithi104 bash[9203]: audit 2020-09-08T15:56:10.012923+0000 mon.c (mon.2) 68 : audit [DBG] from='client.? v1:172.21.15.104:0/3389589096' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:56:11.396 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:11 smithi104 bash[9203]: cluster 2020-09-08T15:56:10.026360+0000 mon.a (mon.0) 1694 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:56:11.396 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:11 smithi104 bash[9203]: audit 2020-09-08T15:56:10.043908+0000 mon.a (mon.0) 1695 : audit [INF] from='client.? v1:172.21.15.104:0/282076038' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:56:11.397 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:11 smithi104 bash[9203]: cluster 2020-09-08T15:56:10.043966+0000 mon.a (mon.0) 1696 : cluster [DBG] osdmap e475: 12 total, 12 up, 12 in 2020-09-08T15:56:11.397 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:11.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:11 smithi055 bash[9296]: cluster 2020-09-08T15:56:09.886475+0000 mgr.x (mgr.24118) 865 : cluster [DBG] pgmap v1255: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:11.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:11 smithi055 bash[9296]: audit 2020-09-08T15:56:10.012923+0000 mon.c (mon.2) 68 : audit [DBG] from='client.? v1:172.21.15.104:0/3389589096' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:56:11.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:11 smithi055 bash[9296]: cluster 2020-09-08T15:56:10.026360+0000 mon.a (mon.0) 1694 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:56:11.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:11 smithi055 bash[9296]: audit 2020-09-08T15:56:10.043908+0000 mon.a (mon.0) 1695 : audit [INF] from='client.? v1:172.21.15.104:0/282076038' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:56:11.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:11 smithi055 bash[9296]: cluster 2020-09-08T15:56:10.043966+0000 mon.a (mon.0) 1696 : cluster [DBG] osdmap e475: 12 total, 12 up, 12 in 2020-09-08T15:56:11.503 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:11.675 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:56:11 smithi104 bash[22305]: debug 2020-09-08T15:56:11.399+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:11.904 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:12.009 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:12.122 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:12.175 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:56:11 smithi104 bash[17588]: debug 2020-09-08T15:56:11.907+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:12.229 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:12.378 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:56:12 smithi055 bash[14301]: debug 2020-09-08T15:56:12.124+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:12.379 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:12.485 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:12.632 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:12.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:56:12 smithi055 bash[11956]: debug 2020-09-08T15:56:12.380+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:12.745 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:12.777 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2020-09-08T15:56:12.867 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:56:12 smithi166 bash[14254]: debug 2020-09-08T15:56:12.628+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:12.872 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:12.959 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:56:12.977 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:13.044 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:56:12 smithi055 bash[14301]: debug 2020-09-08T15:56:12.872+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:13.113 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:13.219 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:56:13.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:13 smithi166 bash[9446]: cluster 2020-09-08T15:56:11.887134+0000 mgr.x (mgr.24118) 866 : cluster [DBG] pgmap v1257: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:13.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:13 smithi055 bash[9296]: cluster 2020-09-08T15:56:11.887134+0000 mgr.x (mgr.24118) 866 : cluster [DBG] pgmap v1257: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:13.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:56:13 smithi055 bash[16646]: debug 2020-09-08T15:56:13.116+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:13.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:13 smithi104 bash[9203]: cluster 2020-09-08T15:56:11.887134+0000 mgr.x (mgr.24118) 866 : cluster [DBG] pgmap v1257: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:13.549 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:56:13.656 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:13.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:56:13 smithi104 bash[19922]: debug 2020-09-08T15:56:13.551+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:14.089 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:14.158 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:56:14.195 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:14.289 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:56:14.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:14 smithi166 bash[9446]: cluster 2020-09-08T15:56:13.015511+0000 osd.7 (osd.7) 93 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:14.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:14 smithi166 bash[9446]: cluster 2020-09-08T15:56:13.015645+0000 osd.7 (osd.7) 94 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:14.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:14 smithi166 bash[9446]: cluster 2020-09-08T15:56:13.046886+0000 mon.a (mon.0) 1697 : cluster [DBG] osdmap e476: 12 total, 12 up, 12 in 2020-09-08T15:56:14.404 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:14 smithi055 bash[9296]: cluster 2020-09-08T15:56:13.015511+0000 osd.7 (osd.7) 93 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:14.405 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:14 smithi055 bash[9296]: cluster 2020-09-08T15:56:13.015645+0000 osd.7 (osd.7) 94 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:14.405 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:14 smithi055 bash[9296]: cluster 2020-09-08T15:56:13.046886+0000 mon.a (mon.0) 1697 : cluster [DBG] osdmap e476: 12 total, 12 up, 12 in 2020-09-08T15:56:14.405 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:14.425 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:56:14 smithi104 bash[17588]: debug 2020-09-08T15:56:14.091+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:14.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:14 smithi104 bash[9203]: cluster 2020-09-08T15:56:13.015511+0000 osd.7 (osd.7) 93 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:14.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:14 smithi104 bash[9203]: cluster 2020-09-08T15:56:13.015645+0000 osd.7 (osd.7) 94 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:14.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:14 smithi104 bash[9203]: cluster 2020-09-08T15:56:13.046886+0000 mon.a (mon.0) 1697 : cluster [DBG] osdmap e476: 12 total, 12 up, 12 in 2020-09-08T15:56:14.511 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:14.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:56:14 smithi055 bash[14301]: debug 2020-09-08T15:56:14.408+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:14.756 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:56:14.831 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:14.890 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:56:14.935 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:15.061 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:15.075 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'unique_pool_4' created 2020-09-08T15:56:15.130 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:56:14 smithi104 bash[15273]: debug 2020-09-08T15:56:14.835+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:15.131 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:15 smithi104 bash[9203]: cluster 2020-09-08T15:56:13.697341+0000 osd.0 (osd.0) 67 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:15.131 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:15 smithi104 bash[9203]: cluster 2020-09-08T15:56:13.697443+0000 osd.0 (osd.0) 68 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:15.132 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:15 smithi104 bash[9203]: cluster 2020-09-08T15:56:13.887765+0000 mgr.x (mgr.24118) 867 : cluster [DBG] pgmap v1259: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:15.132 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:15 smithi104 bash[9203]: cluster 2020-09-08T15:56:14.064332+0000 mon.a (mon.0) 1698 : cluster [DBG] osdmap e477: 12 total, 12 up, 12 in 2020-09-08T15:56:15.133 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:15 smithi104 bash[9203]: audit 2020-09-08T15:56:14.266522+0000 mon.a (mon.0) 1699 : audit [INF] from='client.? v1:172.21.15.104:0/1050354892' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2020-09-08T15:56:15.168 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:56:15.328 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:15 smithi166 bash[9446]: cluster 2020-09-08T15:56:13.697341+0000 osd.0 (osd.0) 67 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:15.329 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:15 smithi166 bash[9446]: cluster 2020-09-08T15:56:13.697443+0000 osd.0 (osd.0) 68 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:15.329 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:15 smithi166 bash[9446]: cluster 2020-09-08T15:56:13.887765+0000 mgr.x (mgr.24118) 867 : cluster [DBG] pgmap v1259: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:15.329 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:15 smithi166 bash[9446]: cluster 2020-09-08T15:56:14.064332+0000 mon.a (mon.0) 1698 : cluster [DBG] osdmap e477: 12 total, 12 up, 12 in 2020-09-08T15:56:15.329 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:15 smithi166 bash[9446]: audit 2020-09-08T15:56:14.266522+0000 mon.a (mon.0) 1699 : audit [INF] from='client.? v1:172.21.15.104:0/1050354892' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2020-09-08T15:56:15.330 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:56:15.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:56:15 smithi055 bash[11956]: debug 2020-09-08T15:56:15.060+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:15.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:15 smithi055 bash[9296]: cluster 2020-09-08T15:56:13.697341+0000 osd.0 (osd.0) 67 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:15.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:15 smithi055 bash[9296]: cluster 2020-09-08T15:56:13.697443+0000 osd.0 (osd.0) 68 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:15.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:15 smithi055 bash[9296]: cluster 2020-09-08T15:56:13.887765+0000 mgr.x (mgr.24118) 867 : cluster [DBG] pgmap v1259: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:15.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:15 smithi055 bash[9296]: cluster 2020-09-08T15:56:14.064332+0000 mon.a (mon.0) 1698 : cluster [DBG] osdmap e477: 12 total, 12 up, 12 in 2020-09-08T15:56:15.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:15 smithi055 bash[9296]: audit 2020-09-08T15:56:14.266522+0000 mon.a (mon.0) 1699 : audit [INF] from='client.? v1:172.21.15.104:0/1050354892' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2020-09-08T15:56:15.435 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:15.556 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:15.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:56:15 smithi166 bash[11928]: debug 2020-09-08T15:56:15.324+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:15.663 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:15.778 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:15.887 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:15.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:56:15 smithi055 bash[16646]: debug 2020-09-08T15:56:15.556+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:16.088 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:56:15 smithi166 bash[18960]: debug 2020-09-08T15:56:15.776+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:16.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:16 smithi104 bash[9203]: audit 2020-09-08T15:56:15.072286+0000 mon.a (mon.0) 1700 : audit [INF] from='client.? v1:172.21.15.104:0/1050354892' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2020-09-08T15:56:16.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:16 smithi104 bash[9203]: cluster 2020-09-08T15:56:15.072534+0000 mon.a (mon.0) 1701 : cluster [DBG] osdmap e478: 12 total, 12 up, 12 in 2020-09-08T15:56:16.295 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:16.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:16 smithi166 bash[9446]: audit 2020-09-08T15:56:15.072286+0000 mon.a (mon.0) 1700 : audit [INF] from='client.? v1:172.21.15.104:0/1050354892' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2020-09-08T15:56:16.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:16 smithi166 bash[9446]: cluster 2020-09-08T15:56:15.072534+0000 mon.a (mon.0) 1701 : cluster [DBG] osdmap e478: 12 total, 12 up, 12 in 2020-09-08T15:56:16.399 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:16.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:16 smithi055 bash[9296]: audit 2020-09-08T15:56:15.072286+0000 mon.a (mon.0) 1700 : audit [INF] from='client.? v1:172.21.15.104:0/1050354892' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2020-09-08T15:56:16.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:16 smithi055 bash[9296]: cluster 2020-09-08T15:56:15.072534+0000 mon.a (mon.0) 1701 : cluster [DBG] osdmap e478: 12 total, 12 up, 12 in 2020-09-08T15:56:16.515 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:16.632 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:16.675 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:56:16 smithi104 bash[22305]: debug 2020-09-08T15:56:16.295+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:16.744 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:56:16 smithi055 bash[19003]: debug 2020-09-08T15:56:16.516+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:16.744 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:16.850 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:16.969 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:17.076 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:17.087 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:56:16 smithi055 bash[14301]: debug 2020-09-08T15:56:16.744+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:17.088 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:56:16 smithi055 bash[19003]: debug 2020-09-08T15:56:16.972+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:17.185 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool application enable unique_pool_4 rados --yes-i-really-mean-it || true 2020-09-08T15:56:17.247 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:17.324 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:56:17.353 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:17.361 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:17 smithi104 bash[9203]: cluster 2020-09-08T15:56:15.888482+0000 mgr.x (mgr.24118) 868 : cluster [DBG] pgmap v1262: 17 pgs: 2 creating+peering, 14 unknown, 1 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:17.361 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:17 smithi104 bash[9203]: cluster 2020-09-08T15:56:16.083952+0000 mon.a (mon.0) 1702 : cluster [DBG] osdmap e479: 12 total, 12 up, 12 in 2020-09-08T15:56:17.374 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:56:17 smithi166 bash[16593]: debug 2020-09-08T15:56:17.244+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:17.374 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:17 smithi166 bash[9446]: cluster 2020-09-08T15:56:15.888482+0000 mgr.x (mgr.24118) 868 : cluster [DBG] pgmap v1262: 17 pgs: 2 creating+peering, 14 unknown, 1 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:17.374 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:17 smithi166 bash[9446]: cluster 2020-09-08T15:56:16.083952+0000 mon.a (mon.0) 1702 : cluster [DBG] osdmap e479: 12 total, 12 up, 12 in 2020-09-08T15:56:17.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:17 smithi055 bash[9296]: cluster 2020-09-08T15:56:15.888482+0000 mgr.x (mgr.24118) 868 : cluster [DBG] pgmap v1262: 17 pgs: 2 creating+peering, 14 unknown, 1 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:17.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:17 smithi055 bash[9296]: cluster 2020-09-08T15:56:16.083952+0000 mon.a (mon.0) 1702 : cluster [DBG] osdmap e479: 12 total, 12 up, 12 in 2020-09-08T15:56:17.825 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:17.929 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:17.974 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:56:17 smithi104 bash[22305]: debug 2020-09-08T15:56:17.827+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:18.216 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:18.227 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:56:18 smithi104 bash[15273]: debug 2020-09-08T15:56:18.219+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:18.228 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:18 smithi104 bash[9203]: cluster 2020-09-08T15:56:17.085427+0000 mon.a (mon.0) 1703 : cluster [DBG] osdmap e480: 12 total, 12 up, 12 in 2020-09-08T15:56:18.321 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:18.373 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:18 smithi166 bash[9446]: cluster 2020-09-08T15:56:17.085427+0000 mon.a (mon.0) 1703 : cluster [DBG] osdmap e480: 12 total, 12 up, 12 in 2020-09-08T15:56:18.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:18 smithi055 bash[9296]: cluster 2020-09-08T15:56:17.085427+0000 mon.a (mon.0) 1703 : cluster [DBG] osdmap e480: 12 total, 12 up, 12 in 2020-09-08T15:56:18.445 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:18.550 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:18.714 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:18.819 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:18.834 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:56:18 smithi055 bash[16646]: debug 2020-09-08T15:56:18.448+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:18.867 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:56:18 smithi166 bash[14254]: debug 2020-09-08T15:56:18.712+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:19.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:19 smithi166 bash[9446]: cluster 2020-09-08T15:56:17.889133+0000 mgr.x (mgr.24118) 869 : cluster [DBG] pgmap v1265: 17 pgs: 2 creating+peering, 3 unknown, 12 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:19.379 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:19 smithi055 bash[9296]: cluster 2020-09-08T15:56:17.889133+0000 mgr.x (mgr.24118) 869 : cluster [DBG] pgmap v1265: 17 pgs: 2 creating+peering, 3 unknown, 12 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:19.380 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:19.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:19 smithi104 bash[9203]: cluster 2020-09-08T15:56:17.889133+0000 mgr.x (mgr.24118) 869 : cluster [DBG] pgmap v1265: 17 pgs: 2 creating+peering, 3 unknown, 12 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:19.486 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:19.606 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:19.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:56:19 smithi055 bash[14301]: debug 2020-09-08T15:56:19.380+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:19.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:56:19 smithi055 bash[19003]: debug 2020-09-08T15:56:19.608+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:19.715 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:20.102 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:20.207 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:20.316 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:20.330 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:56:20.330 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T15:56:20.330 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2020-09-08T15:56:20.330 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2020-09-08T15:56:20.331 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:56:20.422 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:20.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:56:20 smithi104 bash[15273]: debug 2020-09-08T15:56:20.103+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:20.576 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:20.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:56:20 smithi055 bash[14301]: debug 2020-09-08T15:56:20.320+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:20.682 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:56:20.695 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:56:20 smithi166 bash[16593]: debug 2020-09-08T15:56:20.572+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:20.804 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:56:20.909 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:21.033 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:20 smithi166 bash[9446]: cluster 2020-09-08T15:56:19.889850+0000 mgr.x (mgr.24118) 870 : cluster [DBG] pgmap v1266: 17 pgs: 2 creating+peering, 15 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:21.033 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:56:20 smithi166 bash[11928]: debug 2020-09-08T15:56:20.800+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:21.033 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:21.140 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:21.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:20 smithi055 bash[9296]: cluster 2020-09-08T15:56:19.889850+0000 mgr.x (mgr.24118) 870 : cluster [DBG] pgmap v1266: 17 pgs: 2 creating+peering, 15 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:21.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:56:21 smithi166 bash[14254]: debug 2020-09-08T15:56:21.028+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:21.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:20 smithi104 bash[9203]: cluster 2020-09-08T15:56:19.889850+0000 mgr.x (mgr.24118) 870 : cluster [DBG] pgmap v1266: 17 pgs: 2 creating+peering, 15 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:21.629 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:21.732 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:21.930 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:56:21 smithi104 bash[17588]: debug 2020-09-08T15:56:21.627+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:22.191 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:22.296 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:22.426 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:22.516 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:56:22 smithi104 bash[17588]: debug 2020-09-08T15:56:22.195+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:22.517 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2020-09-08T15:56:22.528 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:22.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:56:22 smithi055 bash[16646]: debug 2020-09-08T15:56:22.424+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:22.660 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:56:23.295 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:22 smithi104 bash[9203]: cluster 2020-09-08T15:56:21.890549+0000 mgr.x (mgr.24118) 871 : cluster [DBG] pgmap v1267: 17 pgs: 17 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:23.296 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:22 smithi104 bash[9203]: cluster 2020-09-08T15:56:22.364135+0000 osd.5 (osd.5) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:23.296 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:22 smithi104 bash[9203]: cluster 2020-09-08T15:56:22.364244+0000 osd.5 (osd.5) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:23.296 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:23.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:22 smithi166 bash[9446]: cluster 2020-09-08T15:56:21.890549+0000 mgr.x (mgr.24118) 871 : cluster [DBG] pgmap v1267: 17 pgs: 17 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:23.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:22 smithi166 bash[9446]: cluster 2020-09-08T15:56:22.364135+0000 osd.5 (osd.5) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:23.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:22 smithi166 bash[9446]: cluster 2020-09-08T15:56:22.364244+0000 osd.5 (osd.5) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:23.404 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:56:23.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:22 smithi055 bash[9296]: cluster 2020-09-08T15:56:21.890549+0000 mgr.x (mgr.24118) 871 : cluster [DBG] pgmap v1267: 17 pgs: 17 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:23.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:22 smithi055 bash[9296]: cluster 2020-09-08T15:56:22.364135+0000 osd.5 (osd.5) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:23.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:22 smithi055 bash[9296]: cluster 2020-09-08T15:56:22.364244+0000 osd.5 (osd.5) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:23.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:56:23 smithi104 bash[15273]: debug 2020-09-08T15:56:23.299+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:23.879 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:56:23.983 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:56:24.105 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:56:24.126 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:56:24.149 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:56:24.158 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:56:24.175 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:56:23 smithi104 bash[19922]: debug 2020-09-08T15:56:23.883+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:24.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:23 smithi104 bash[9203]: cluster 2020-09-08T15:56:22.958468+0000 mon.a (mon.0) 1704 : cluster [DBG] osdmap e481: 12 total, 12 up, 12 in 2020-09-08T15:56:24.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:23 smithi104 bash[9203]: cluster 2020-09-08T15:56:23.091641+0000 osd.10 (osd.10) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:24.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:23 smithi104 bash[9203]: cluster 2020-09-08T15:56:23.091760+0000 osd.10 (osd.10) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:24.179 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:56:24.203 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:56:24.246 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:56:24.255 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:56:24.268 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:56:24 smithi166 bash[11928]: debug 2020-09-08T15:56:24.148+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:24.268 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:23 smithi166 bash[9446]: cluster 2020-09-08T15:56:22.958468+0000 mon.a (mon.0) 1704 : cluster [DBG] osdmap e481: 12 total, 12 up, 12 in 2020-09-08T15:56:24.268 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:23 smithi166 bash[9446]: cluster 2020-09-08T15:56:23.091641+0000 osd.10 (osd.10) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:24.268 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:23 smithi166 bash[9446]: cluster 2020-09-08T15:56:23.091760+0000 osd.10 (osd.10) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:24.278 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:56:24.298 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:56:24.324 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:56:24.355 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:56:24.370 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:56:24.379 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:56:24.403 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:56:24.406 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:56:24.406 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:56:24.406 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:56:24.407 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:56:24.407 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:56:24.408 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:56:24.408 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:56:24.408 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:56:24.408 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:56:24.409 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:56:24.409 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:56:24.410 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:56:24.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:23 smithi055 bash[9296]: cluster 2020-09-08T15:56:22.958468+0000 mon.a (mon.0) 1704 : cluster [DBG] osdmap e481: 12 total, 12 up, 12 in 2020-09-08T15:56:24.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:23 smithi055 bash[9296]: cluster 2020-09-08T15:56:23.091641+0000 osd.10 (osd.10) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:24.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:23 smithi055 bash[9296]: cluster 2020-09-08T15:56:23.091760+0000 osd.10 (osd.10) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:24.476 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:24.619 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:56:24 smithi166 bash[11928]: debug 2020-09-08T15:56:24.368+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:24.651 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:24.726 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:56:24 smithi104 bash[22305]: debug 2020-09-08T15:56:24.655+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:24.756 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:24.902 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:24.972 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:56:25.011 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:25.024 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:56:24 smithi055 bash[16646]: debug 2020-09-08T15:56:24.900+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:25.026 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:24 smithi055 bash[9296]: cluster 2020-09-08T15:56:23.891193+0000 mgr.x (mgr.24118) 872 : cluster [DBG] pgmap v1269: 17 pgs: 17 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:25.026 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:24 smithi055 bash[9296]: cluster 2020-09-08T15:56:23.968752+0000 mon.a (mon.0) 1705 : cluster [DBG] osdmap e482: 12 total, 12 up, 12 in 2020-09-08T15:56:25.027 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:24 smithi055 bash[9296]: audit 2020-09-08T15:56:24.082170+0000 mon.a (mon.0) 1706 : audit [DBG] from='client.? v1:172.21.15.104:0/1582238297' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:56:25.027 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:24 smithi055 bash[9296]: audit 2020-09-08T15:56:24.320822+0000 mon.a (mon.0) 1707 : audit [INF] from='client.? v1:172.21.15.104:0/1222260875' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:56:25.125 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:25.142 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:24 smithi104 bash[9203]: cluster 2020-09-08T15:56:23.891193+0000 mgr.x (mgr.24118) 872 : cluster [DBG] pgmap v1269: 17 pgs: 17 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:25.142 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:24 smithi104 bash[9203]: cluster 2020-09-08T15:56:23.968752+0000 mon.a (mon.0) 1705 : cluster [DBG] osdmap e482: 12 total, 12 up, 12 in 2020-09-08T15:56:25.142 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:24 smithi104 bash[9203]: audit 2020-09-08T15:56:24.082170+0000 mon.a (mon.0) 1706 : audit [DBG] from='client.? v1:172.21.15.104:0/1582238297' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:56:25.143 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:24 smithi104 bash[9203]: audit 2020-09-08T15:56:24.320822+0000 mon.a (mon.0) 1707 : audit [INF] from='client.? v1:172.21.15.104:0/1222260875' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:56:25.226 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:25.239 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:24 smithi166 bash[9446]: cluster 2020-09-08T15:56:23.891193+0000 mgr.x (mgr.24118) 872 : cluster [DBG] pgmap v1269: 17 pgs: 17 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:25.239 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:24 smithi166 bash[9446]: cluster 2020-09-08T15:56:23.968752+0000 mon.a (mon.0) 1705 : cluster [DBG] osdmap e482: 12 total, 12 up, 12 in 2020-09-08T15:56:25.240 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:24 smithi166 bash[9446]: audit 2020-09-08T15:56:24.082170+0000 mon.a (mon.0) 1706 : audit [DBG] from='client.? v1:172.21.15.104:0/1582238297' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:56:25.240 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:24 smithi166 bash[9446]: audit 2020-09-08T15:56:24.320822+0000 mon.a (mon.0) 1707 : audit [INF] from='client.? v1:172.21.15.104:0/1222260875' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:56:25.362 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:25.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:56:25 smithi055 bash[11956]: debug 2020-09-08T15:56:25.120+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:25.467 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:25.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:56:25 smithi166 bash[14254]: debug 2020-09-08T15:56:25.360+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:25.625 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:25.730 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:25.743 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:56:25 smithi055 bash[16646]: debug 2020-09-08T15:56:25.628+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:25.862 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:25.977 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:25.991 INFO:teuthology.orchestra.run.smithi104.stderr:enabled application 'rados' on pool 'unique_pool_4' 2020-09-08T15:56:26.101 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:56:25 smithi055 bash[14301]: debug 2020-09-08T15:56:25.860+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:26.102 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:25 smithi055 bash[9296]: cluster 2020-09-08T15:56:24.962708+0000 mon.a (mon.0) 1708 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:56:26.102 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:25 smithi055 bash[9296]: audit 2020-09-08T15:56:24.973050+0000 mon.a (mon.0) 1709 : audit [INF] from='client.? v1:172.21.15.104:0/1222260875' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:56:26.102 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:25 smithi055 bash[9296]: cluster 2020-09-08T15:56:24.973130+0000 mon.a (mon.0) 1710 : cluster [DBG] osdmap e483: 12 total, 12 up, 12 in 2020-09-08T15:56:26.102 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:25 smithi055 bash[9296]: audit 2020-09-08T15:56:25.462441+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? v1:172.21.15.104:0/885950905' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T15:56:26.103 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:26.210 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:26.354 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:26.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:25 smithi166 bash[9446]: cluster 2020-09-08T15:56:24.962708+0000 mon.a (mon.0) 1708 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:56:26.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:25 smithi166 bash[9446]: audit 2020-09-08T15:56:24.973050+0000 mon.a (mon.0) 1709 : audit [INF] from='client.? v1:172.21.15.104:0/1222260875' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:56:26.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:25 smithi166 bash[9446]: cluster 2020-09-08T15:56:24.973130+0000 mon.a (mon.0) 1710 : cluster [DBG] osdmap e483: 12 total, 12 up, 12 in 2020-09-08T15:56:26.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:25 smithi166 bash[9446]: audit 2020-09-08T15:56:25.462441+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? v1:172.21.15.104:0/885950905' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T15:56:26.408 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:56:26 smithi055 bash[11956]: debug 2020-09-08T15:56:26.104+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:26.408 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:56:26 smithi055 bash[19003]: debug 2020-09-08T15:56:26.340+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:26.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:25 smithi104 bash[9203]: cluster 2020-09-08T15:56:24.962708+0000 mon.a (mon.0) 1708 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:56:26.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:25 smithi104 bash[9203]: audit 2020-09-08T15:56:24.973050+0000 mon.a (mon.0) 1709 : audit [INF] from='client.? v1:172.21.15.104:0/1222260875' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:56:26.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:25 smithi104 bash[9203]: cluster 2020-09-08T15:56:24.973130+0000 mon.a (mon.0) 1710 : cluster [DBG] osdmap e483: 12 total, 12 up, 12 in 2020-09-08T15:56:26.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:25 smithi104 bash[9203]: audit 2020-09-08T15:56:25.462441+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? v1:172.21.15.104:0/885950905' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T15:56:26.456 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:26.778 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:26.882 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:27.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:56:26 smithi104 bash[17588]: debug 2020-09-08T15:56:26.779+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:27.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:26 smithi104 bash[9203]: cluster 2020-09-08T15:56:25.891849+0000 mgr.x (mgr.24118) 873 : cluster [DBG] pgmap v1272: 17 pgs: 17 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:27.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:26 smithi104 bash[9203]: audit 2020-09-08T15:56:25.992213+0000 mon.a (mon.0) 1712 : audit [INF] from='client.? v1:172.21.15.104:0/885950905' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T15:56:27.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:26 smithi104 bash[9203]: cluster 2020-09-08T15:56:25.992259+0000 mon.a (mon.0) 1713 : cluster [DBG] osdmap e484: 12 total, 12 up, 12 in 2020-09-08T15:56:27.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:26 smithi104 bash[9203]: cluster 2020-09-08T15:56:26.072580+0000 osd.7 (osd.7) 95 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:56:27.247 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:27.354 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:27.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:26 smithi166 bash[9446]: cluster 2020-09-08T15:56:25.891849+0000 mgr.x (mgr.24118) 873 : cluster [DBG] pgmap v1272: 17 pgs: 17 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:27.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:26 smithi166 bash[9446]: audit 2020-09-08T15:56:25.992213+0000 mon.a (mon.0) 1712 : audit [INF] from='client.? v1:172.21.15.104:0/885950905' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T15:56:27.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:26 smithi166 bash[9446]: cluster 2020-09-08T15:56:25.992259+0000 mon.a (mon.0) 1713 : cluster [DBG] osdmap e484: 12 total, 12 up, 12 in 2020-09-08T15:56:27.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:26 smithi166 bash[9446]: cluster 2020-09-08T15:56:26.072580+0000 osd.7 (osd.7) 95 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:56:27.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:26 smithi055 bash[9296]: cluster 2020-09-08T15:56:25.891849+0000 mgr.x (mgr.24118) 873 : cluster [DBG] pgmap v1272: 17 pgs: 17 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:27.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:26 smithi055 bash[9296]: audit 2020-09-08T15:56:25.992213+0000 mon.a (mon.0) 1712 : audit [INF] from='client.? v1:172.21.15.104:0/885950905' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T15:56:27.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:26 smithi055 bash[9296]: cluster 2020-09-08T15:56:25.992259+0000 mon.a (mon.0) 1713 : cluster [DBG] osdmap e484: 12 total, 12 up, 12 in 2020-09-08T15:56:27.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:26 smithi055 bash[9296]: cluster 2020-09-08T15:56:26.072580+0000 osd.7 (osd.7) 95 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:56:27.617 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:56:27.639 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:56:27 smithi104 bash[17588]: debug 2020-09-08T15:56:27.251+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:27.759 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:27.759 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:56:27.864 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:27.980 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:28.086 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:28.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:28 smithi166 bash[9446]: cluster 2020-09-08T15:56:26.157806+0000 osd.7 (osd.7) 96 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:56:28.117 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:56:27 smithi166 bash[18960]: debug 2020-09-08T15:56:27.756+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:28.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:56:27 smithi055 bash[11956]: debug 2020-09-08T15:56:27.980+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:28.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:28 smithi055 bash[9296]: cluster 2020-09-08T15:56:26.157806+0000 osd.7 (osd.7) 96 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:56:28.284 INFO:teuthology.orchestra.run.smithi166:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.2 -b 65536 --object-size 65536 -p unique_pool_4 bench 90 write' 2020-09-08T15:56:28.302 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2020-09-08T15:56:28.303 INFO:teuthology.orchestra.run:waiting for 3000 2020-09-08T15:56:28.334 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:28 smithi104 bash[9203]: cluster 2020-09-08T15:56:26.157806+0000 osd.7 (osd.7) 96 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:56:28.335 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:28.439 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:28.462 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:56:28.488 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:hints = 1 2020-09-08T15:56:28.488 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2020-09-08T15:56:28.488 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Object prefix: benchmark_data_smithi166_43187 2020-09-08T15:56:28.489 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:56:28.489 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 0 0 0 0 0 0 - 0 2020-09-08T15:56:28.660 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:56:28 smithi104 bash[17588]: debug 2020-09-08T15:56:28.335+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:28.675 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:56:28.798 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:28.903 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:28.925 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:56:28 smithi104 bash[15273]: debug 2020-09-08T15:56:28.799+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:28.975 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:56:28.975 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:56:29.050 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:29.167 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:29.180 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:56:29 smithi166 bash[18960]: debug 2020-09-08T15:56:29.048+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:29.181 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:29 smithi166 bash[9446]: cluster 2020-09-08T15:56:27.834330+0000 osd.3 (osd.3) 69 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:29.181 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:29 smithi166 bash[9446]: cluster 2020-09-08T15:56:27.834440+0000 osd.3 (osd.3) 70 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:29.181 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:29 smithi166 bash[9446]: cluster 2020-09-08T15:56:27.892760+0000 mgr.x (mgr.24118) 874 : cluster [DBG] pgmap v1274: 17 pgs: 17 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:29.182 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:29 smithi166 bash[9446]: cluster 2020-09-08T15:56:28.004638+0000 mon.a (mon.0) 1714 : cluster [DBG] osdmap e485: 12 total, 12 up, 12 in 2020-09-08T15:56:29.315 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:29.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:29 smithi055 bash[9296]: cluster 2020-09-08T15:56:27.834330+0000 osd.3 (osd.3) 69 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:29.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:29 smithi055 bash[9296]: cluster 2020-09-08T15:56:27.834440+0000 osd.3 (osd.3) 70 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:29.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:29 smithi055 bash[9296]: cluster 2020-09-08T15:56:27.892760+0000 mgr.x (mgr.24118) 874 : cluster [DBG] pgmap v1274: 17 pgs: 17 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:29.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:29 smithi055 bash[9296]: cluster 2020-09-08T15:56:28.004638+0000 mon.a (mon.0) 1714 : cluster [DBG] osdmap e485: 12 total, 12 up, 12 in 2020-09-08T15:56:29.421 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:56:29.434 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:29 smithi104 bash[9203]: cluster 2020-09-08T15:56:27.834330+0000 osd.3 (osd.3) 69 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:29.435 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:29 smithi104 bash[9203]: cluster 2020-09-08T15:56:27.834440+0000 osd.3 (osd.3) 70 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:29.435 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:29 smithi104 bash[9203]: cluster 2020-09-08T15:56:27.892760+0000 mgr.x (mgr.24118) 874 : cluster [DBG] pgmap v1274: 17 pgs: 17 active+clean; 0 B data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:29.435 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:29 smithi104 bash[9203]: cluster 2020-09-08T15:56:28.004638+0000 mon.a (mon.0) 1714 : cluster [DBG] osdmap e485: 12 total, 12 up, 12 in 2020-09-08T15:56:29.490 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 1 16 2335 2319 144.93 144.938 0.00482199 0.00686553 2020-09-08T15:56:29.617 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:56:29 smithi166 bash[16593]: debug 2020-09-08T15:56:29.312+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:29.675 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:56:29.777 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:29.907 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:29.925 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:56:29 smithi104 bash[19922]: debug 2020-09-08T15:56:29.675+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:30.012 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:30.143 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:56:30 smithi055 bash[14301]: debug 2020-09-08T15:56:30.144+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:30.143 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:30.143 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:56:29 smithi055 bash[11956]: debug 2020-09-08T15:56:29.908+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:30.247 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:30.368 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:30.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:56:30 smithi055 bash[19003]: debug 2020-09-08T15:56:30.368+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:30.472 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:30.489 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 2 16 5713 5697 178.015 211.125 0.0036129 0.00560382 2020-09-08T15:56:30.848 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:30 smithi104 bash[9203]: cluster 2020-09-08T15:56:29.893362+0000 mgr.x (mgr.24118) 875 : cluster [DBG] pgmap v1276: 17 pgs: 17 active+clean; 20 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s wr, 52 op/s 2020-09-08T15:56:31.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:30 smithi166 bash[9446]: cluster 2020-09-08T15:56:29.893362+0000 mgr.x (mgr.24118) 875 : cluster [DBG] pgmap v1276: 17 pgs: 17 active+clean; 20 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s wr, 52 op/s 2020-09-08T15:56:31.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:30 smithi055 bash[9296]: cluster 2020-09-08T15:56:29.893362+0000 mgr.x (mgr.24118) 875 : cluster [DBG] pgmap v1276: 17 pgs: 17 active+clean; 20 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s wr, 52 op/s 2020-09-08T15:56:31.489 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 3 16 9197 9181 191.254 217.75 0.00423274 0.00522201 2020-09-08T15:56:32.489 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 4 15 11948 11933 186.436 172 0.00454483 0.00535798 2020-09-08T15:56:32.518 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:32.623 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:32.770 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:32.874 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:32.943 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:56:32 smithi166 bash[16593]: debug 2020-09-08T15:56:32.768+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:33.160 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:56:32 smithi104 bash[17588]: debug 2020-09-08T15:56:32.519+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:33.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:32 smithi166 bash[9446]: cluster 2020-09-08T15:56:31.893860+0000 mgr.x (mgr.24118) 876 : cluster [DBG] pgmap v1277: 17 pgs: 17 active+clean; 184 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s wr, 425 op/s 2020-09-08T15:56:33.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:32 smithi055 bash[9296]: cluster 2020-09-08T15:56:31.893860+0000 mgr.x (mgr.24118) 876 : cluster [DBG] pgmap v1277: 17 pgs: 17 active+clean; 184 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s wr, 425 op/s 2020-09-08T15:56:33.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:33 smithi104 bash[9203]: cluster 2020-09-08T15:56:31.893860+0000 mgr.x (mgr.24118) 876 : cluster [DBG] pgmap v1277: 17 pgs: 17 active+clean; 184 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s wr, 425 op/s 2020-09-08T15:56:33.489 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 5 16 14536 14520 181.483 161.688 0.0056981 0.00550569 2020-09-08T15:56:34.080 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2020-09-08T15:56:34.132 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:34.235 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:34.277 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:56:34.341 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:56:34 smithi104 bash[17588]: debug 2020-09-08T15:56:34.135+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:34.371 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:34.475 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:34.491 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 6 16 17549 17533 182.619 188.312 0.00393067 0.00547224 2020-09-08T15:56:34.593 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:56:34 smithi166 bash[18960]: debug 2020-09-08T15:56:34.368+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:34.594 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:34.697 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:34.868 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:56:34 smithi166 bash[18960]: debug 2020-09-08T15:56:34.592+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:35.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:34 smithi104 bash[9203]: cluster 2020-09-08T15:56:33.894342+0000 mgr.x (mgr.24118) 877 : cluster [DBG] pgmap v1278: 17 pgs: 17 active+clean; 530 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s wr, 1.06k op/s 2020-09-08T15:56:35.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:34 smithi104 bash[9203]: audit 2020-09-08T15:56:34.865744+0000 mon.c (mon.2) 69 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:56:35.191 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:34 smithi104 bash[9203]: audit 2020-09-08T15:56:34.869184+0000 mon.a (mon.0) 1715 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:56:35.192 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:34 smithi104 bash[9203]: audit 2020-09-08T15:56:34.875817+0000 mon.a (mon.0) 1716 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:56:35.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:34 smithi166 bash[9446]: cluster 2020-09-08T15:56:33.894342+0000 mgr.x (mgr.24118) 877 : cluster [DBG] pgmap v1278: 17 pgs: 17 active+clean; 530 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s wr, 1.06k op/s 2020-09-08T15:56:35.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:34 smithi166 bash[9446]: audit 2020-09-08T15:56:34.865744+0000 mon.c (mon.2) 69 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:56:35.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:34 smithi166 bash[9446]: audit 2020-09-08T15:56:34.869184+0000 mon.a (mon.0) 1715 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:56:35.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:34 smithi166 bash[9446]: audit 2020-09-08T15:56:34.875817+0000 mon.a (mon.0) 1716 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:56:35.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:34 smithi055 bash[9296]: cluster 2020-09-08T15:56:33.894342+0000 mgr.x (mgr.24118) 877 : cluster [DBG] pgmap v1278: 17 pgs: 17 active+clean; 530 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s wr, 1.06k op/s 2020-09-08T15:56:35.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:34 smithi055 bash[9296]: audit 2020-09-08T15:56:34.865744+0000 mon.c (mon.2) 69 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:56:35.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:34 smithi055 bash[9296]: audit 2020-09-08T15:56:34.869184+0000 mon.a (mon.0) 1715 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:56:35.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:34 smithi055 bash[9296]: audit 2020-09-08T15:56:34.875817+0000 mon.a (mon.0) 1716 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:56:35.491 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 7 15 19876 19861 177.314 145.5 0.0040661 0.00563559 2020-09-08T15:56:36.227 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:36.230 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:35 smithi104 bash[9203]: cluster 2020-09-08T15:56:35.111934+0000 osd.8 (osd.8) 61 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:36.231 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:35 smithi104 bash[9203]: cluster 2020-09-08T15:56:35.112009+0000 osd.8 (osd.8) 62 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:36.334 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:36.354 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:35 smithi166 bash[9446]: cluster 2020-09-08T15:56:35.111934+0000 osd.8 (osd.8) 61 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:36.354 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:35 smithi166 bash[9446]: cluster 2020-09-08T15:56:35.112009+0000 osd.8 (osd.8) 62 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:36.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:35 smithi055 bash[9296]: cluster 2020-09-08T15:56:35.111934+0000 osd.8 (osd.8) 61 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:36.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:35 smithi055 bash[9296]: cluster 2020-09-08T15:56:35.112009+0000 osd.8 (osd.8) 62 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:36.486 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:36.489 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 8 15 22646 22631 176.789 173.125 0.00524688 0.00565319 2020-09-08T15:56:36.590 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:36.617 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:56:36 smithi166 bash[18960]: debug 2020-09-08T15:56:36.484+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:36.675 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:56:36 smithi104 bash[22305]: debug 2020-09-08T15:56:36.231+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:36.730 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:36.832 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:37.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:36 smithi166 bash[9446]: cluster 2020-09-08T15:56:35.894846+0000 mgr.x (mgr.24118) 878 : cluster [DBG] pgmap v1279: 17 pgs: 17 active+clean; 817 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 MiB/s wr, 1.32k op/s 2020-09-08T15:56:37.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:36 smithi166 bash[9446]: cluster 2020-09-08T15:56:35.969315+0000 mon.a (mon.0) 1717 : cluster [DBG] osdmap e486: 12 total, 12 up, 12 in 2020-09-08T15:56:37.118 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:56:36 smithi166 bash[16593]: debug 2020-09-08T15:56:36.724+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:37.252 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:36 smithi055 bash[9296]: cluster 2020-09-08T15:56:35.894846+0000 mgr.x (mgr.24118) 878 : cluster [DBG] pgmap v1279: 17 pgs: 17 active+clean; 817 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 MiB/s wr, 1.32k op/s 2020-09-08T15:56:37.252 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:36 smithi055 bash[9296]: cluster 2020-09-08T15:56:35.969315+0000 mon.a (mon.0) 1717 : cluster [DBG] osdmap e486: 12 total, 12 up, 12 in 2020-09-08T15:56:37.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:36 smithi104 bash[9203]: cluster 2020-09-08T15:56:35.894846+0000 mgr.x (mgr.24118) 878 : cluster [DBG] pgmap v1279: 17 pgs: 17 active+clean; 817 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 MiB/s wr, 1.32k op/s 2020-09-08T15:56:37.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:36 smithi104 bash[9203]: cluster 2020-09-08T15:56:35.969315+0000 mon.a (mon.0) 1717 : cluster [DBG] osdmap e486: 12 total, 12 up, 12 in 2020-09-08T15:56:37.490 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 9 16 24648 24632 171.04 125.062 0.00491048 0.0058389 2020-09-08T15:56:38.490 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 10 16 24805 24789 154.917 9.8125 0.00533976 0.00582879 2020-09-08T15:56:39.108 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:39.214 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:56:39.243 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:38 smithi166 bash[9446]: cluster 2020-09-08T15:56:37.895546+0000 mgr.x (mgr.24118) 879 : cluster [DBG] pgmap v1281: 17 pgs: 17 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 138 MiB/s wr, 2.21k op/s 2020-09-08T15:56:39.373 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:56:39.408 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:56:39 smithi055 bash[16646]: debug 2020-09-08T15:56:39.107+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:39.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:38 smithi055 bash[9296]: cluster 2020-09-08T15:56:37.895546+0000 mgr.x (mgr.24118) 879 : cluster [DBG] pgmap v1281: 17 pgs: 17 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 138 MiB/s wr, 2.21k op/s 2020-09-08T15:56:39.452 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:38 smithi104 bash[9203]: cluster 2020-09-08T15:56:37.895546+0000 mgr.x (mgr.24118) 879 : cluster [DBG] pgmap v1281: 17 pgs: 17 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 138 MiB/s wr, 2.21k op/s 2020-09-08T15:56:39.476 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:39.489 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:56:39 smithi166 bash[11928]: debug 2020-09-08T15:56:39.364+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:39.490 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 11 16 24805 24789 140.833 0 - 0.00582879 2020-09-08T15:56:39.620 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:39.722 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:39.839 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:39.869 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:56:39 smithi166 bash[14254]: debug 2020-09-08T15:56:39.612+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:39.961 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:40.091 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:40.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:56:39 smithi055 bash[11956]: debug 2020-09-08T15:56:39.839+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:40.197 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:40.341 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:40.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:56:40 smithi166 bash[14254]: debug 2020-09-08T15:56:40.088+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:40.398 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:56:40.423 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:56:40.443 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:56:40.447 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:40.474 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:56:40.490 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 12 16 24805 24789 129.096 0 - 0.00582879 2020-09-08T15:56:40.500 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:56:40.528 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:56:40.561 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:56:40.575 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:40.588 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:56:40.616 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:56:40.643 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:56:40.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:56:40 smithi055 bash[19003]: debug 2020-09-08T15:56:40.343+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:40.669 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:56:40.680 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:40.703 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:56:40.705 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:56:40.705 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:56:40.706 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:56:40.706 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:56:40.706 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:56:40.706 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:56:40.706 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:56:40.707 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:56:40.707 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:56:40.707 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:56:40.707 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:56:40.707 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:56:40.810 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:56:40 smithi166 bash[18960]: debug 2020-09-08T15:56:40.572+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:40.810 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:40.913 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:41.074 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:40 smithi166 bash[9446]: cluster 2020-09-08T15:56:39.896225+0000 mgr.x (mgr.24118) 880 : cluster [DBG] pgmap v1282: 17 pgs: 17 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 145 MiB/s wr, 2.31k op/s 2020-09-08T15:56:41.075 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:40 smithi166 bash[9446]: audit 2020-09-08T15:56:40.380650+0000 mon.a (mon.0) 1718 : audit [DBG] from='client.? v1:172.21.15.104:0/2679505917' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:56:41.075 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:56:40 smithi166 bash[18960]: debug 2020-09-08T15:56:40.796+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:41.075 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:41.094 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:40 smithi104 bash[9203]: cluster 2020-09-08T15:56:39.896225+0000 mgr.x (mgr.24118) 880 : cluster [DBG] pgmap v1282: 17 pgs: 17 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 145 MiB/s wr, 2.31k op/s 2020-09-08T15:56:41.094 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:40 smithi104 bash[9203]: audit 2020-09-08T15:56:40.380650+0000 mon.a (mon.0) 1718 : audit [DBG] from='client.? v1:172.21.15.104:0/2679505917' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:56:41.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:40 smithi055 bash[9296]: cluster 2020-09-08T15:56:39.896225+0000 mgr.x (mgr.24118) 880 : cluster [DBG] pgmap v1282: 17 pgs: 17 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 145 MiB/s wr, 2.31k op/s 2020-09-08T15:56:41.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:40 smithi055 bash[9296]: audit 2020-09-08T15:56:40.380650+0000 mon.a (mon.0) 1718 : audit [DBG] from='client.? v1:172.21.15.104:0/2679505917' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:56:41.180 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:41.367 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:56:41 smithi166 bash[16593]: debug 2020-09-08T15:56:41.072+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:41.490 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 13 16 24805 24789 119.166 0 - 0.00582879 2020-09-08T15:56:41.678 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:41.783 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:41.878 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:56:41.959 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:41.963 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:41 smithi104 bash[9203]: audit 2020-09-08T15:56:41.443095+0000 mon.a (mon.0) 1719 : audit [INF] from='client.? v1:172.21.15.104:0/840601738' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:56:41.963 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:56:41 smithi104 bash[15273]: debug 2020-09-08T15:56:41.683+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:42.066 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:42.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:41 smithi166 bash[9446]: audit 2020-09-08T15:56:41.443095+0000 mon.a (mon.0) 1719 : audit [INF] from='client.? v1:172.21.15.104:0/840601738' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:56:42.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:41 smithi055 bash[9296]: audit 2020-09-08T15:56:41.443095+0000 mon.a (mon.0) 1719 : audit [INF] from='client.? v1:172.21.15.104:0/840601738' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:56:42.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:56:41 smithi104 bash[17588]: debug 2020-09-08T15:56:41.963+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:42.459 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:42.491 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 14 16 24805 24789 110.653 0 - 0.00582879 2020-09-08T15:56:42.564 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:42.685 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:42.791 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:42.865 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:56:42 smithi055 bash[14301]: debug 2020-09-08T15:56:42.687+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:42.866 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:42 smithi055 bash[9296]: cluster 2020-09-08T15:56:41.860277+0000 mon.a (mon.0) 1720 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:56:42.866 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:42 smithi055 bash[9296]: cluster 2020-09-08T15:56:41.860334+0000 mon.a (mon.0) 1721 : cluster [INF] Cluster is now healthy 2020-09-08T15:56:42.867 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:42 smithi055 bash[9296]: audit 2020-09-08T15:56:41.874190+0000 mon.a (mon.0) 1722 : audit [INF] from='client.? v1:172.21.15.104:0/840601738' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:56:42.867 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:42 smithi055 bash[9296]: cluster 2020-09-08T15:56:41.874274+0000 mon.a (mon.0) 1723 : cluster [DBG] osdmap e487: 12 total, 12 up, 12 in 2020-09-08T15:56:42.868 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:42 smithi055 bash[9296]: cluster 2020-09-08T15:56:41.896827+0000 mgr.x (mgr.24118) 881 : cluster [DBG] pgmap v1284: 17 pgs: 17 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 124 MiB/s wr, 1.98k op/s 2020-09-08T15:56:42.868 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:42 smithi055 bash[9296]: cluster 2020-09-08T15:56:41.925092+0000 osd.1 (osd.1) 94 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:42.868 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:56:42 smithi104 bash[17588]: debug 2020-09-08T15:56:42.463+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:43.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:42 smithi166 bash[9446]: cluster 2020-09-08T15:56:41.860277+0000 mon.a (mon.0) 1720 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:56:43.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:42 smithi166 bash[9446]: cluster 2020-09-08T15:56:41.860334+0000 mon.a (mon.0) 1721 : cluster [INF] Cluster is now healthy 2020-09-08T15:56:43.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:42 smithi166 bash[9446]: audit 2020-09-08T15:56:41.874190+0000 mon.a (mon.0) 1722 : audit [INF] from='client.? v1:172.21.15.104:0/840601738' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:56:43.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:42 smithi166 bash[9446]: cluster 2020-09-08T15:56:41.874274+0000 mon.a (mon.0) 1723 : cluster [DBG] osdmap e487: 12 total, 12 up, 12 in 2020-09-08T15:56:43.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:42 smithi166 bash[9446]: cluster 2020-09-08T15:56:41.896827+0000 mgr.x (mgr.24118) 881 : cluster [DBG] pgmap v1284: 17 pgs: 17 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 124 MiB/s wr, 1.98k op/s 2020-09-08T15:56:43.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:42 smithi166 bash[9446]: cluster 2020-09-08T15:56:41.925092+0000 osd.1 (osd.1) 94 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:43.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:42 smithi166 bash[9446]: cluster 2020-09-08T15:56:41.925252+0000 osd.1 (osd.1) 95 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:43.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:42 smithi055 bash[9296]: cluster 2020-09-08T15:56:41.925252+0000 osd.1 (osd.1) 95 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:42 smithi104 bash[9203]: cluster 2020-09-08T15:56:41.860277+0000 mon.a (mon.0) 1720 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:56:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:42 smithi104 bash[9203]: cluster 2020-09-08T15:56:41.860334+0000 mon.a (mon.0) 1721 : cluster [INF] Cluster is now healthy 2020-09-08T15:56:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:42 smithi104 bash[9203]: audit 2020-09-08T15:56:41.874190+0000 mon.a (mon.0) 1722 : audit [INF] from='client.? v1:172.21.15.104:0/840601738' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:56:43.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:42 smithi104 bash[9203]: cluster 2020-09-08T15:56:41.874274+0000 mon.a (mon.0) 1723 : cluster [DBG] osdmap e487: 12 total, 12 up, 12 in 2020-09-08T15:56:43.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:42 smithi104 bash[9203]: cluster 2020-09-08T15:56:41.896827+0000 mgr.x (mgr.24118) 881 : cluster [DBG] pgmap v1284: 17 pgs: 17 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 124 MiB/s wr, 1.98k op/s 2020-09-08T15:56:43.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:42 smithi104 bash[9203]: cluster 2020-09-08T15:56:41.925092+0000 osd.1 (osd.1) 94 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:43.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:42 smithi104 bash[9203]: cluster 2020-09-08T15:56:41.925252+0000 osd.1 (osd.1) 95 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:43.491 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 15 16 24805 24789 103.276 0 - 0.00582879 2020-09-08T15:56:43.656 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:56:43.656 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T15:56:43.656 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 0 2020-09-08T15:56:43.657 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:56:43.995 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:56:44.013 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:43 smithi104 bash[9203]: cluster 2020-09-08T15:56:42.881496+0000 mon.a (mon.0) 1724 : cluster [DBG] osdmap e488: 12 total, 12 up, 12 in 2020-09-08T15:56:44.063 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:44.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:43 smithi055 bash[9296]: cluster 2020-09-08T15:56:42.881496+0000 mon.a (mon.0) 1724 : cluster [DBG] osdmap e488: 12 total, 12 up, 12 in 2020-09-08T15:56:44.168 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:44.221 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:56:44.299 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:44.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:43 smithi166 bash[9446]: cluster 2020-09-08T15:56:42.881496+0000 mon.a (mon.0) 1724 : cluster [DBG] osdmap e488: 12 total, 12 up, 12 in 2020-09-08T15:56:44.367 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:56:44 smithi166 bash[16593]: debug 2020-09-08T15:56:44.060+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:44.405 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:44.491 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 16 16 24805 24789 96.8214 0 - 0.00582879 2020-09-08T15:56:44.658 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:56:44 smithi055 bash[19003]: debug 2020-09-08T15:56:44.299+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:44.721 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:44.807 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2020-09-08T15:56:44.830 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:56:44.947 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:56:45.141 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:44 smithi104 bash[9203]: cluster 2020-09-08T15:56:43.897521+0000 mgr.x (mgr.24118) 882 : cluster [DBG] pgmap v1286: 17 pgs: 17 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.48k op/s 2020-09-08T15:56:45.142 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:56:44 smithi104 bash[22305]: debug 2020-09-08T15:56:44.723+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:45.142 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:56:45.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:44 smithi055 bash[9296]: cluster 2020-09-08T15:56:43.897521+0000 mgr.x (mgr.24118) 882 : cluster [DBG] pgmap v1286: 17 pgs: 17 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.48k op/s 2020-09-08T15:56:45.246 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:45.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:44 smithi166 bash[9446]: cluster 2020-09-08T15:56:43.897521+0000 mgr.x (mgr.24118) 882 : cluster [DBG] pgmap v1286: 17 pgs: 17 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.48k op/s 2020-09-08T15:56:45.387 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:45.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:56:45 smithi104 bash[19922]: debug 2020-09-08T15:56:45.143+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:45.439 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:56:45.495 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:45.497 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 17 16 24805 24789 91.1258 0 - 0.00582879 2020-09-08T15:56:45.575 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:56:45.636 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:45.656 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:56:45 smithi055 bash[14301]: debug 2020-09-08T15:56:45.387+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:45.741 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:45.867 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:56:45 smithi166 bash[16593]: debug 2020-09-08T15:56:45.632+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:45.868 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:45.973 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:46.101 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:56:45 smithi055 bash[16646]: debug 2020-09-08T15:56:45.867+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:46.101 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:46.206 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:46.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:56:46 smithi055 bash[19003]: debug 2020-09-08T15:56:46.103+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:46.491 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 18 16 24805 24789 86.0632 0 - 0.00582879 2020-09-08T15:56:46.663 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:46.768 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:56:46.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:56:46 smithi104 bash[15273]: debug 2020-09-08T15:56:46.663+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:46.927 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:56:47.033 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:47.046 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:46 smithi166 bash[9446]: cluster 2020-09-08T15:56:45.898418+0000 mgr.x (mgr.24118) 883 : cluster [DBG] pgmap v1287: 17 pgs: 17 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s wr, 370 op/s 2020-09-08T15:56:47.047 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:56:46 smithi166 bash[11928]: debug 2020-09-08T15:56:46.924+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:47.185 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:47.300 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:47.321 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:56:47 smithi166 bash[16593]: debug 2020-09-08T15:56:47.188+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:47.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:46 smithi055 bash[9296]: cluster 2020-09-08T15:56:45.898418+0000 mgr.x (mgr.24118) 883 : cluster [DBG] pgmap v1287: 17 pgs: 17 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s wr, 370 op/s 2020-09-08T15:56:47.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:46 smithi104 bash[9203]: cluster 2020-09-08T15:56:45.898418+0000 mgr.x (mgr.24118) 883 : cluster [DBG] pgmap v1287: 17 pgs: 17 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s wr, 370 op/s 2020-09-08T15:56:47.440 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:47.491 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 19 16 24805 24789 81.5334 0 - 0.00582879 2020-09-08T15:56:47.546 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:47.618 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:56:47 smithi166 bash[14254]: debug 2020-09-08T15:56:47.440+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:47.695 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:47.801 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:47.906 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:56:47 smithi055 bash[11956]: debug 2020-09-08T15:56:47.695+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:48.144 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:47 smithi104 bash[9203]: cluster 2020-09-08T15:56:46.564750+0000 osd.6 (osd.6) 79 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:48.145 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:47 smithi104 bash[9203]: cluster 2020-09-08T15:56:46.564897+0000 osd.6 (osd.6) 80 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:48.145 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:47 smithi104 bash[9203]: cluster 2020-09-08T15:56:46.962082+0000 mon.a (mon.0) 1725 : cluster [DBG] osdmap e489: 12 total, 12 up, 12 in 2020-09-08T15:56:48.145 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:48.250 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:48.262 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:47 smithi166 bash[9446]: cluster 2020-09-08T15:56:46.564750+0000 osd.6 (osd.6) 79 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:48.262 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:47 smithi166 bash[9446]: cluster 2020-09-08T15:56:46.564897+0000 osd.6 (osd.6) 80 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:48.262 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:47 smithi166 bash[9446]: cluster 2020-09-08T15:56:46.962082+0000 mon.a (mon.0) 1725 : cluster [DBG] osdmap e489: 12 total, 12 up, 12 in 2020-09-08T15:56:48.374 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:48.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:47 smithi055 bash[9296]: cluster 2020-09-08T15:56:46.564750+0000 osd.6 (osd.6) 79 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:48.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:47 smithi055 bash[9296]: cluster 2020-09-08T15:56:46.564897+0000 osd.6 (osd.6) 80 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:48.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:47 smithi055 bash[9296]: cluster 2020-09-08T15:56:46.962082+0000 mon.a (mon.0) 1725 : cluster [DBG] osdmap e489: 12 total, 12 up, 12 in 2020-09-08T15:56:48.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:56:48 smithi104 bash[22305]: debug 2020-09-08T15:56:48.147+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:48.479 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:48.491 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 15:56:48.495757 min lat: 0.00166475 max lat: 0.217607 avg lat: 0.00582879 2020-09-08T15:56:48.492 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:56:48.492 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 20 16 24805 24789 77.4565 0 - 0.00582879 2020-09-08T15:56:48.604 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:48.618 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:56:48 smithi166 bash[14254]: debug 2020-09-08T15:56:48.372+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:48.710 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:48.836 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:56:48 smithi055 bash[14301]: debug 2020-09-08T15:56:48.603+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:48.837 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:48.943 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:49.073 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:49.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:48 smithi055 bash[9296]: cluster 2020-09-08T15:56:47.899204+0000 mgr.x (mgr.24118) 884 : cluster [DBG] pgmap v1289: 17 pgs: 17 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s wr, 73 op/s 2020-09-08T15:56:49.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:56:48 smithi055 bash[19003]: debug 2020-09-08T15:56:48.839+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:49.179 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:49.299 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:56:49 smithi166 bash[14254]: debug 2020-09-08T15:56:49.076+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:49.300 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:48 smithi166 bash[9446]: cluster 2020-09-08T15:56:47.899204+0000 mgr.x (mgr.24118) 884 : cluster [DBG] pgmap v1289: 17 pgs: 17 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s wr, 73 op/s 2020-09-08T15:56:49.300 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:49.405 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:56:49.418 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:48 smithi104 bash[9203]: cluster 2020-09-08T15:56:47.899204+0000 mgr.x (mgr.24118) 884 : cluster [DBG] pgmap v1289: 17 pgs: 17 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s wr, 73 op/s 2020-09-08T15:56:49.492 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 21 16 24805 24789 73.7679 0 - 0.00582879 2020-09-08T15:56:49.595 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:56:49.619 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:56:49 smithi166 bash[18960]: debug 2020-09-08T15:56:49.300+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:49.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:56:49 smithi104 bash[19922]: debug 2020-09-08T15:56:49.598+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:49.700 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:49.817 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:49.922 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:50.035 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:50.118 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:56:49 smithi166 bash[14254]: debug 2020-09-08T15:56:49.816+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:50.140 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:50.281 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:56:50.399 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:56:50.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:56:50 smithi055 bash[19003]: debug 2020-09-08T15:56:50.035+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:50.492 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 22 16 24805 24789 70.4147 0 - 0.00582879 2020-09-08T15:56:50.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:56:50 smithi166 bash[14254]: debug 2020-09-08T15:56:50.276+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:50.672 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:56:50.677 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:56:50.700 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:56:50.723 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:56:50.747 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:56:50.772 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:56:50.777 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:50.803 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:56:50.826 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:56:50.858 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:56:50.891 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:56:50.911 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:50.919 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:56:50.927 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:56:50 smithi104 bash[19922]: debug 2020-09-08T15:56:50.674+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:50.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:50 smithi104 bash[9203]: cluster 2020-09-08T15:56:49.855148+0000 mon.a (mon.0) 1726 : cluster [DBG] osdmap e490: 12 total, 12 up, 12 in 2020-09-08T15:56:50.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:50 smithi104 bash[9203]: cluster 2020-09-08T15:56:49.899880+0000 mgr.x (mgr.24118) 885 : cluster [DBG] pgmap v1291: 17 pgs: 17 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:50.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:50 smithi104 bash[9203]: cluster 2020-09-08T15:56:50.120951+0000 osd.11 (osd.11) 84 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:50.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:50 smithi104 bash[9203]: cluster 2020-09-08T15:56:50.121087+0000 osd.11 (osd.11) 85 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:50.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:50 smithi104 bash[9203]: audit 2020-09-08T15:56:50.652418+0000 mon.a (mon.0) 1727 : audit [DBG] from='client.? v1:172.21.15.104:0/4290399563' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:56:50.956 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:56:50.985 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:56:50.987 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:56:50.987 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:56:50.987 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:56:50.988 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:56:50.988 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:56:50.988 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:56:50.988 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:56:50.988 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:56:50.988 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:56:50.989 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:56:50.989 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:56:50.989 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:56:51.017 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:51.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:50 smithi166 bash[9446]: cluster 2020-09-08T15:56:49.855148+0000 mon.a (mon.0) 1726 : cluster [DBG] osdmap e490: 12 total, 12 up, 12 in 2020-09-08T15:56:51.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:50 smithi166 bash[9446]: cluster 2020-09-08T15:56:49.899880+0000 mgr.x (mgr.24118) 885 : cluster [DBG] pgmap v1291: 17 pgs: 17 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:51.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:50 smithi166 bash[9446]: cluster 2020-09-08T15:56:50.120951+0000 osd.11 (osd.11) 84 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:51.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:50 smithi166 bash[9446]: cluster 2020-09-08T15:56:50.121087+0000 osd.11 (osd.11) 85 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:51.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:50 smithi166 bash[9446]: audit 2020-09-08T15:56:50.652418+0000 mon.a (mon.0) 1727 : audit [DBG] from='client.? v1:172.21.15.104:0/4290399563' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:56:51.142 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:56:50 smithi055 bash[19003]: debug 2020-09-08T15:56:50.911+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:51.142 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:50 smithi055 bash[9296]: cluster 2020-09-08T15:56:49.855148+0000 mon.a (mon.0) 1726 : cluster [DBG] osdmap e490: 12 total, 12 up, 12 in 2020-09-08T15:56:51.142 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:50 smithi055 bash[9296]: cluster 2020-09-08T15:56:49.899880+0000 mgr.x (mgr.24118) 885 : cluster [DBG] pgmap v1291: 17 pgs: 17 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:51.143 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:50 smithi055 bash[9296]: cluster 2020-09-08T15:56:50.120951+0000 osd.11 (osd.11) 84 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:51.143 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:50 smithi055 bash[9296]: cluster 2020-09-08T15:56:50.121087+0000 osd.11 (osd.11) 85 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:51.143 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:50 smithi055 bash[9296]: audit 2020-09-08T15:56:50.652418+0000 mon.a (mon.0) 1727 : audit [DBG] from='client.? v1:172.21.15.104:0/4290399563' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:56:51.144 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:56:51 smithi055 bash[14301]: debug 2020-09-08T15:56:51.143+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:51.145 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:51.152 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:56:51.152 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:56:51.249 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:56:51.492 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 23 16 24805 24789 67.3531 0 - 0.00582879 2020-09-08T15:56:51.881 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:56:51.909 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:56:52.027 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:52.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:51 smithi055 bash[9296]: cluster 2020-09-08T15:56:50.875659+0000 mon.a (mon.0) 1728 : cluster [DBG] osdmap e491: 12 total, 12 up, 12 in 2020-09-08T15:56:52.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:51 smithi055 bash[9296]: audit 2020-09-08T15:56:51.662701+0000 mon.a (mon.0) 1729 : audit [INF] from='client.? v1:172.21.15.104:0/3089943896' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:56:52.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:56:51 smithi104 bash[19922]: debug 2020-09-08T15:56:51.910+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:52.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:51 smithi104 bash[9203]: cluster 2020-09-08T15:56:50.875659+0000 mon.a (mon.0) 1728 : cluster [DBG] osdmap e491: 12 total, 12 up, 12 in 2020-09-08T15:56:52.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:51 smithi104 bash[9203]: audit 2020-09-08T15:56:51.662701+0000 mon.a (mon.0) 1729 : audit [INF] from='client.? v1:172.21.15.104:0/3089943896' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:56:52.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:51 smithi166 bash[9446]: cluster 2020-09-08T15:56:50.875659+0000 mon.a (mon.0) 1728 : cluster [DBG] osdmap e491: 12 total, 12 up, 12 in 2020-09-08T15:56:52.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:51 smithi166 bash[9446]: audit 2020-09-08T15:56:51.662701+0000 mon.a (mon.0) 1729 : audit [INF] from='client.? v1:172.21.15.104:0/3089943896' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:56:52.449 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:52.492 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 24 16 24857 24841 64.6821 0.232143 0.00879198 0.0154565 2020-09-08T15:56:52.554 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:52.869 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:52 smithi104 bash[9203]: cluster 2020-09-08T15:56:51.871243+0000 mon.a (mon.0) 1730 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:56:52.870 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:52 smithi104 bash[9203]: audit 2020-09-08T15:56:51.882321+0000 mon.a (mon.0) 1731 : audit [INF] from='client.? v1:172.21.15.104:0/3089943896' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:56:52.870 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:52 smithi104 bash[9203]: cluster 2020-09-08T15:56:51.882369+0000 mon.a (mon.0) 1732 : cluster [DBG] osdmap e492: 12 total, 12 up, 12 in 2020-09-08T15:56:52.870 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:52 smithi104 bash[9203]: cluster 2020-09-08T15:56:51.900462+0000 mgr.x (mgr.24118) 886 : cluster [DBG] pgmap v1294: 17 pgs: 17 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:52.871 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:56:52 smithi104 bash[15273]: debug 2020-09-08T15:56:52.450+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:52.934 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:56:52.951 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2020-09-08T15:56:53.046 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:53.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:52 smithi055 bash[9296]: cluster 2020-09-08T15:56:51.871243+0000 mon.a (mon.0) 1730 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:56:53.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:52 smithi055 bash[9296]: audit 2020-09-08T15:56:51.882321+0000 mon.a (mon.0) 1731 : audit [INF] from='client.? v1:172.21.15.104:0/3089943896' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:56:53.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:52 smithi055 bash[9296]: cluster 2020-09-08T15:56:51.882369+0000 mon.a (mon.0) 1732 : cluster [DBG] osdmap e492: 12 total, 12 up, 12 in 2020-09-08T15:56:53.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:52 smithi055 bash[9296]: cluster 2020-09-08T15:56:51.900462+0000 mgr.x (mgr.24118) 886 : cluster [DBG] pgmap v1294: 17 pgs: 17 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:53.172 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:56:53.175 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:56:52 smithi104 bash[15273]: debug 2020-09-08T15:56:52.930+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:53.178 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:53.282 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:53.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:52 smithi166 bash[9446]: cluster 2020-09-08T15:56:51.871243+0000 mon.a (mon.0) 1730 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:56:53.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:52 smithi166 bash[9446]: audit 2020-09-08T15:56:51.882321+0000 mon.a (mon.0) 1731 : audit [INF] from='client.? v1:172.21.15.104:0/3089943896' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:56:53.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:52 smithi166 bash[9446]: cluster 2020-09-08T15:56:51.882369+0000 mon.a (mon.0) 1732 : cluster [DBG] osdmap e492: 12 total, 12 up, 12 in 2020-09-08T15:56:53.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:52 smithi166 bash[9446]: cluster 2020-09-08T15:56:51.900462+0000 mgr.x (mgr.24118) 886 : cluster [DBG] pgmap v1294: 17 pgs: 17 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:53.439 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:56:53 smithi055 bash[19003]: debug 2020-09-08T15:56:53.179+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:53.440 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:53.492 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 25 16 27182 27166 67.9066 145.312 0.112803 0.0146765 2020-09-08T15:56:53.545 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:53.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:56:53 smithi055 bash[16646]: debug 2020-09-08T15:56:53.439+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:54.181 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:54.286 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:54.381 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:54.474 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:56:54.482 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:56:54 smithi104 bash[22305]: debug 2020-09-08T15:56:54.182+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:54.486 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:54.492 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 26 16 29395 29379 70.614 138.312 0.0074289 0.014158 2020-09-08T15:56:54.594 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:56:54 smithi055 bash[14301]: debug 2020-09-08T15:56:54.383+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:54.595 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:54.650 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:56:54.698 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:54.834 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:54.914 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:56:54 smithi055 bash[19003]: debug 2020-09-08T15:56:54.595+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:54.940 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:55.062 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:56:54 smithi166 bash[18960]: debug 2020-09-08T15:56:54.836+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:55.063 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:56:55 smithi166 bash[18960]: debug 2020-09-08T15:56:55.064+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:55.063 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:54 smithi166 bash[9446]: cluster 2020-09-08T15:56:53.901835+0000 mgr.x (mgr.24118) 887 : cluster [DBG] pgmap v1295: 17 pgs: 17 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:55.063 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:54 smithi166 bash[9446]: cluster 2020-09-08T15:56:54.855326+0000 mon.a (mon.0) 1733 : cluster [DBG] osdmap e493: 12 total, 12 up, 12 in 2020-09-08T15:56:55.064 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:55.093 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:56:55.165 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:55.281 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:56:55.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:54 smithi055 bash[9296]: cluster 2020-09-08T15:56:53.901835+0000 mgr.x (mgr.24118) 887 : cluster [DBG] pgmap v1295: 17 pgs: 17 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:55.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:54 smithi055 bash[9296]: cluster 2020-09-08T15:56:54.855326+0000 mon.a (mon.0) 1733 : cluster [DBG] osdmap e493: 12 total, 12 up, 12 in 2020-09-08T15:56:55.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:54 smithi104 bash[9203]: cluster 2020-09-08T15:56:53.901835+0000 mgr.x (mgr.24118) 887 : cluster [DBG] pgmap v1295: 17 pgs: 17 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T15:56:55.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:54 smithi104 bash[9203]: cluster 2020-09-08T15:56:54.855326+0000 mon.a (mon.0) 1733 : cluster [DBG] osdmap e493: 12 total, 12 up, 12 in 2020-09-08T15:56:55.492 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 27 16 31798 31782 73.5605 150.188 0.0063906 0.0135908 2020-09-08T15:56:56.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:55 smithi166 bash[9446]: cluster 2020-09-08T15:56:54.438172+0000 osd.4 (osd.4) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:56.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:55 smithi166 bash[9446]: cluster 2020-09-08T15:56:54.438268+0000 osd.4 (osd.4) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:56.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:55 smithi055 bash[9296]: cluster 2020-09-08T15:56:54.438172+0000 osd.4 (osd.4) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:56.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:55 smithi055 bash[9296]: cluster 2020-09-08T15:56:54.438268+0000 osd.4 (osd.4) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:56.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:55 smithi104 bash[9203]: cluster 2020-09-08T15:56:54.438172+0000 osd.4 (osd.4) 51 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:56:56.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:55 smithi104 bash[9203]: cluster 2020-09-08T15:56:54.438268+0000 osd.4 (osd.4) 52 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:56:56.496 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 28 16 33585 33569 74.9218 111.688 0.00438418 0.0133435 2020-09-08T15:56:56.502 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:56:56.605 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:56.786 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:56.867 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:56:56 smithi166 bash[16593]: debug 2020-09-08T15:56:56.788+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:56.891 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:56.996 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:56:56 smithi104 bash[22305]: debug 2020-09-08T15:56:56.502+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:57.041 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:56:57.149 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:57.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:56:57 smithi055 bash[11956]: debug 2020-09-08T15:56:57.043+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:57.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:56 smithi055 bash[9296]: cluster 2020-09-08T15:56:55.902277+0000 mgr.x (mgr.24118) 888 : cluster [DBG] pgmap v1297: 17 pgs: 17 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s wr, 271 op/s 2020-09-08T15:56:57.260 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:57.365 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:57.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:56 smithi166 bash[9446]: cluster 2020-09-08T15:56:55.902277+0000 mgr.x (mgr.24118) 888 : cluster [DBG] pgmap v1297: 17 pgs: 17 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s wr, 271 op/s 2020-09-08T15:56:57.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:57 smithi104 bash[9203]: cluster 2020-09-08T15:56:55.902277+0000 mgr.x (mgr.24118) 888 : cluster [DBG] pgmap v1297: 17 pgs: 17 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s wr, 271 op/s 2020-09-08T15:56:57.493 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 29 16 35809 35793 77.1309 139 0.00738164 0.012962 2020-09-08T15:56:57.526 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:57.631 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:57.643 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:56:57 smithi166 bash[18960]: debug 2020-09-08T15:56:57.528+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:57.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:56:57 smithi055 bash[19003]: debug 2020-09-08T15:56:57.259+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:57.750 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:56:57.855 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:56:58.117 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:56:57 smithi166 bash[18960]: debug 2020-09-08T15:56:57.752+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:58.272 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:56:58.376 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:58.491 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:56:58.492 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 30 16 38135 38119 79.4053 145.375 0.00691508 0.0125901 2020-09-08T15:56:58.595 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:58.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:56:58 smithi104 bash[19922]: debug 2020-09-08T15:56:58.274+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:58.719 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:58.825 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:58.867 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:56:58 smithi166 bash[16593]: debug 2020-09-08T15:56:58.491+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:58.908 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:56:58 smithi055 bash[19003]: debug 2020-09-08T15:56:58.719+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:58.948 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:56:59.052 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:59.165 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:56:59.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:56:58 smithi104 bash[9203]: cluster 2020-09-08T15:56:57.903700+0000 mgr.x (mgr.24118) 889 : cluster [DBG] pgmap v1298: 17 pgs: 17 active+clean; 2.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s wr, 1.20k op/s 2020-09-08T15:56:59.270 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:59.277 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:56:58 smithi055 bash[19003]: debug 2020-09-08T15:56:58.947+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:59.278 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:56:58 smithi055 bash[9296]: cluster 2020-09-08T15:56:57.903700+0000 mgr.x (mgr.24118) 889 : cluster [DBG] pgmap v1298: 17 pgs: 17 active+clean; 2.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s wr, 1.20k op/s 2020-09-08T15:56:59.278 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:56:59 smithi055 bash[14301]: debug 2020-09-08T15:56:59.167+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:56:59.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:56:58 smithi166 bash[9446]: cluster 2020-09-08T15:56:57.903700+0000 mgr.x (mgr.24118) 889 : cluster [DBG] pgmap v1298: 17 pgs: 17 active+clean; 2.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s wr, 1.20k op/s 2020-09-08T15:56:59.397 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:56:59.496 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 31 16 40290 40274 81.1881 134.688 0.00450987 0.0123135 2020-09-08T15:56:59.502 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:56:59.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:56:59 smithi055 bash[16646]: debug 2020-09-08T15:56:59.391+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:00.212 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:57:00.317 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:57:00.470 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:57:00 smithi104 bash[17588]: debug 2020-09-08T15:57:00.210+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:00.493 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 32 15 42353 42338 82.6818 129 0.00630939 0.0120914 2020-09-08T15:57:00.542 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:57:00.647 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:57:00.849 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:00 smithi104 bash[9203]: cluster 2020-09-08T15:56:59.857407+0000 mon.a (mon.0) 1734 : cluster [DBG] osdmap e494: 12 total, 12 up, 12 in 2020-09-08T15:57:00.849 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:00 smithi104 bash[9203]: cluster 2020-09-08T15:56:59.904146+0000 mgr.x (mgr.24118) 890 : cluster [DBG] pgmap v1300: 17 pgs: 17 active+clean; 2.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s wr, 1.29k op/s 2020-09-08T15:57:00.850 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:57:00 smithi104 bash[15273]: debug 2020-09-08T15:57:00.542+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:01.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:00 smithi166 bash[9446]: cluster 2020-09-08T15:56:59.857407+0000 mon.a (mon.0) 1734 : cluster [DBG] osdmap e494: 12 total, 12 up, 12 in 2020-09-08T15:57:01.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:00 smithi166 bash[9446]: cluster 2020-09-08T15:56:59.904146+0000 mgr.x (mgr.24118) 890 : cluster [DBG] pgmap v1300: 17 pgs: 17 active+clean; 2.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s wr, 1.29k op/s 2020-09-08T15:57:01.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:00 smithi055 bash[9296]: cluster 2020-09-08T15:56:59.857407+0000 mon.a (mon.0) 1734 : cluster [DBG] osdmap e494: 12 total, 12 up, 12 in 2020-09-08T15:57:01.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:00 smithi055 bash[9296]: cluster 2020-09-08T15:56:59.904146+0000 mgr.x (mgr.24118) 890 : cluster [DBG] pgmap v1300: 17 pgs: 17 active+clean; 2.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s wr, 1.29k op/s 2020-09-08T15:57:01.493 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 33 16 44432 44416 84.1115 129.875 0.0101662 0.0118859 2020-09-08T15:57:01.905 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:57:02.010 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:57:02.175 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:57:01 smithi104 bash[17588]: debug 2020-09-08T15:57:01.906+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:02.301 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:57:02.403 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:57:02.493 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 34 16 46559 46543 85.5472 132.938 0.0105459 0.0116871 2020-09-08T15:57:02.690 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:57:02 smithi104 bash[19922]: debug 2020-09-08T15:57:02.298+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:02.958 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:02 smithi104 bash[9203]: cluster 2020-09-08T15:57:01.904680+0000 mgr.x (mgr.24118) 891 : cluster [DBG] pgmap v1301: 17 pgs: 17 active+clean; 2.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 107 MiB/s wr, 1.72k op/s 2020-09-08T15:57:03.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:02 smithi166 bash[9446]: cluster 2020-09-08T15:57:01.904680+0000 mgr.x (mgr.24118) 891 : cluster [DBG] pgmap v1301: 17 pgs: 17 active+clean; 2.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 107 MiB/s wr, 1.72k op/s 2020-09-08T15:57:03.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:02 smithi055 bash[9296]: cluster 2020-09-08T15:57:01.904680+0000 mgr.x (mgr.24118) 891 : cluster [DBG] pgmap v1301: 17 pgs: 17 active+clean; 2.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 107 MiB/s wr, 1.72k op/s 2020-09-08T15:57:03.471 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:57:03.493 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 35 16 48316 48300 86.2402 109.812 0.00615148 0.0115919 2020-09-08T15:57:03.575 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:57:03.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:57:03 smithi166 bash[11928]: debug 2020-09-08T15:57:03.471+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:04.360 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:03 smithi166 bash[9446]: cluster 2020-09-08T15:57:03.560732+0000 osd.9 (osd.9) 75 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:04.361 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:03 smithi166 bash[9446]: cluster 2020-09-08T15:57:03.560851+0000 osd.9 (osd.9) 76 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:04.361 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:57:04.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:03 smithi055 bash[9296]: cluster 2020-09-08T15:57:03.560732+0000 osd.9 (osd.9) 75 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:04.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:03 smithi055 bash[9296]: cluster 2020-09-08T15:57:03.560851+0000 osd.9 (osd.9) 76 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:04.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:03 smithi104 bash[9203]: cluster 2020-09-08T15:57:03.560732+0000 osd.9 (osd.9) 75 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:04.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:03 smithi104 bash[9203]: cluster 2020-09-08T15:57:03.560851+0000 osd.9 (osd.9) 76 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:04.464 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:57:04.493 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 36 16 50146 50130 87.0214 114.375 0.00558314 0.0114888 2020-09-08T15:57:04.617 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:57:04 smithi166 bash[16593]: debug 2020-09-08T15:57:04.359+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:04.722 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:57:04.724 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:57:04 smithi104 bash[17588]: debug 2020-09-08T15:57:04.722+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:04.827 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:57:04.943 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:57:05.047 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:57:05.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:04 smithi166 bash[9446]: cluster 2020-09-08T15:57:03.905162+0000 mgr.x (mgr.24118) 892 : cluster [DBG] pgmap v1302: 17 pgs: 17 active+clean; 2.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 128 MiB/s wr, 2.05k op/s 2020-09-08T15:57:05.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:04 smithi166 bash[9446]: cluster 2020-09-08T15:57:03.975241+0000 mon.a (mon.0) 1735 : cluster [DBG] osdmap e495: 12 total, 12 up, 12 in 2020-09-08T15:57:05.118 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:57:04 smithi166 bash[18960]: debug 2020-09-08T15:57:04.943+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:05.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:04 smithi104 bash[9203]: cluster 2020-09-08T15:57:03.905162+0000 mgr.x (mgr.24118) 892 : cluster [DBG] pgmap v1302: 17 pgs: 17 active+clean; 2.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 128 MiB/s wr, 2.05k op/s 2020-09-08T15:57:05.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:04 smithi104 bash[9203]: cluster 2020-09-08T15:57:03.975241+0000 mon.a (mon.0) 1735 : cluster [DBG] osdmap e495: 12 total, 12 up, 12 in 2020-09-08T15:57:05.195 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:57:05.300 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:57:05.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:04 smithi055 bash[9296]: cluster 2020-09-08T15:57:03.905162+0000 mgr.x (mgr.24118) 892 : cluster [DBG] pgmap v1302: 17 pgs: 17 active+clean; 2.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 128 MiB/s wr, 2.05k op/s 2020-09-08T15:57:05.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:04 smithi055 bash[9296]: cluster 2020-09-08T15:57:03.975241+0000 mon.a (mon.0) 1735 : cluster [DBG] osdmap e495: 12 total, 12 up, 12 in 2020-09-08T15:57:05.409 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:57:05 smithi055 bash[14301]: debug 2020-09-08T15:57:05.195+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:05.493 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 37 16 51983 51967 87.7722 114.812 0.00402431 0.0113904 2020-09-08T15:57:05.965 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:57:06.069 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:57:06.162 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:57:06.163 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:57:06.215 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:57:06.283 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:57:05 smithi104 bash[15273]: debug 2020-09-08T15:57:05.966+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:06.284 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:05 smithi104 bash[9203]: cluster 2020-09-08T15:57:05.377797+0000 osd.2 (osd.2) 70 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:06.284 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:05 smithi104 bash[9203]: cluster 2020-09-08T15:57:05.377882+0000 osd.2 (osd.2) 71 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:06.319 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:57:06.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:05 smithi166 bash[9446]: cluster 2020-09-08T15:57:05.377797+0000 osd.2 (osd.2) 70 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:06.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:05 smithi166 bash[9446]: cluster 2020-09-08T15:57:05.377882+0000 osd.2 (osd.2) 71 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:06.368 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:57:06 smithi166 bash[14254]: debug 2020-09-08T15:57:06.215+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:06.424 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:05 smithi055 bash[9296]: cluster 2020-09-08T15:57:05.377797+0000 osd.2 (osd.2) 70 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:06.424 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:05 smithi055 bash[9296]: cluster 2020-09-08T15:57:05.377882+0000 osd.2 (osd.2) 71 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:06.493 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 38 16 53795 53779 88.4424 113.25 0.00561363 0.0113041 2020-09-08T15:57:07.334 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:57:07.335 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:06 smithi104 bash[9203]: cluster 2020-09-08T15:57:05.905636+0000 mgr.x (mgr.24118) 893 : cluster [DBG] pgmap v1304: 17 pgs: 17 active+clean; 2.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 108 MiB/s wr, 1.72k op/s 2020-09-08T15:57:07.335 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:06 smithi104 bash[9203]: cluster 2020-09-08T15:57:05.987832+0000 mon.a (mon.0) 1736 : cluster [DBG] osdmap e496: 12 total, 12 up, 12 in 2020-09-08T15:57:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:06 smithi166 bash[9446]: cluster 2020-09-08T15:57:05.905636+0000 mgr.x (mgr.24118) 893 : cluster [DBG] pgmap v1304: 17 pgs: 17 active+clean; 2.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 108 MiB/s wr, 1.72k op/s 2020-09-08T15:57:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:06 smithi166 bash[9446]: cluster 2020-09-08T15:57:05.987832+0000 mon.a (mon.0) 1736 : cluster [DBG] osdmap e496: 12 total, 12 up, 12 in 2020-09-08T15:57:07.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:06 smithi055 bash[9296]: cluster 2020-09-08T15:57:05.905636+0000 mgr.x (mgr.24118) 893 : cluster [DBG] pgmap v1304: 17 pgs: 17 active+clean; 2.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 108 MiB/s wr, 1.72k op/s 2020-09-08T15:57:07.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:06 smithi055 bash[9296]: cluster 2020-09-08T15:57:05.987832+0000 mon.a (mon.0) 1736 : cluster [DBG] osdmap e496: 12 total, 12 up, 12 in 2020-09-08T15:57:07.438 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:07.494 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 39 16 55684 55668 89.2015 118.062 0.00414213 0.0112006 2020-09-08T15:57:07.574 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:07.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:57:07 smithi104 bash[17588]: debug 2020-09-08T15:57:07.334+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:07.681 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:57:07.802 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:57:07 smithi055 bash[11956]: debug 2020-09-08T15:57:07.571+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:07.803 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:57:07 smithi055 bash[14301]: debug 2020-09-08T15:57:07.803+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:07.803 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:57:07.907 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:57:08.494 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 15:57:08.497907 min lat: 0.00166475 max lat: 15.0142 avg lat: 0.011159 2020-09-08T15:57:08.495 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:57:08.495 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 40 16 57362 57346 89.593 104.875 0.0067325 0.011159 2020-09-08T15:57:08.762 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:57:08.865 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:08.928 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:57:08.965 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:57:09.000 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:57:09.034 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:57:09.058 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:08 smithi104 bash[9203]: cluster 2020-09-08T15:57:07.906082+0000 mgr.x (mgr.24118) 894 : cluster [DBG] pgmap v1306: 17 pgs: 17 active+clean; 3.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 142 MiB/s wr, 2.28k op/s 2020-09-08T15:57:09.058 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:08 smithi104 bash[9203]: audit 2020-09-08T15:57:08.888140+0000 mon.a (mon.0) 1737 : audit [DBG] from='client.? v1:172.21.15.104:0/1691119943' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:57:09.059 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:57:08 smithi104 bash[17588]: debug 2020-09-08T15:57:08.762+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:09.067 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:57:09.082 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2020-09-08T15:57:09.099 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:57:09.141 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:57:09.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:08 smithi055 bash[9296]: cluster 2020-09-08T15:57:07.906082+0000 mgr.x (mgr.24118) 894 : cluster [DBG] pgmap v1306: 17 pgs: 17 active+clean; 3.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 142 MiB/s wr, 2.28k op/s 2020-09-08T15:57:09.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:08 smithi055 bash[9296]: audit 2020-09-08T15:57:08.888140+0000 mon.a (mon.0) 1737 : audit [DBG] from='client.? v1:172.21.15.104:0/1691119943' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:57:09.171 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:57:09.203 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:57:09.214 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:57:09.226 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:57:09.252 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:57:09.292 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:57:09.295 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:57:09.295 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:57:09.295 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:57:09.295 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:57:09.295 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:57:09.296 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:57:09.296 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:57:09.296 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:57:09.296 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:57:09.296 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:57:09.297 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:57:09.297 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:57:09.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:08 smithi166 bash[9446]: cluster 2020-09-08T15:57:07.906082+0000 mgr.x (mgr.24118) 894 : cluster [DBG] pgmap v1306: 17 pgs: 17 active+clean; 3.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 142 MiB/s wr, 2.28k op/s 2020-09-08T15:57:09.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:08 smithi166 bash[9446]: audit 2020-09-08T15:57:08.888140+0000 mon.a (mon.0) 1737 : audit [DBG] from='client.? v1:172.21.15.104:0/1691119943' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:57:09.494 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 41 16 58608 58592 89.307 77.875 0.00513791 0.011173 2020-09-08T15:57:09.859 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:57:10.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:09 smithi166 bash[9446]: audit 2020-09-08T15:57:09.485001+0000 mon.a (mon.0) 1738 : audit [INF] from='client.? v1:172.21.15.104:0/552166936' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:57:10.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:09 smithi166 bash[9446]: cluster 2020-09-08T15:57:09.853961+0000 mon.a (mon.0) 1739 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:57:10.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:09 smithi166 bash[9446]: audit 2020-09-08T15:57:09.860394+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? v1:172.21.15.104:0/552166936' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:57:10.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:09 smithi166 bash[9446]: cluster 2020-09-08T15:57:09.860441+0000 mon.a (mon.0) 1741 : cluster [DBG] osdmap e497: 12 total, 12 up, 12 in 2020-09-08T15:57:10.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:09 smithi055 bash[9296]: audit 2020-09-08T15:57:09.485001+0000 mon.a (mon.0) 1738 : audit [INF] from='client.? v1:172.21.15.104:0/552166936' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:57:10.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:09 smithi055 bash[9296]: cluster 2020-09-08T15:57:09.853961+0000 mon.a (mon.0) 1739 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:57:10.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:09 smithi055 bash[9296]: audit 2020-09-08T15:57:09.860394+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? v1:172.21.15.104:0/552166936' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:57:10.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:09 smithi055 bash[9296]: cluster 2020-09-08T15:57:09.860441+0000 mon.a (mon.0) 1741 : cluster [DBG] osdmap e497: 12 total, 12 up, 12 in 2020-09-08T15:57:10.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:09 smithi104 bash[9203]: audit 2020-09-08T15:57:09.485001+0000 mon.a (mon.0) 1738 : audit [INF] from='client.? v1:172.21.15.104:0/552166936' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:57:10.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:09 smithi104 bash[9203]: cluster 2020-09-08T15:57:09.853961+0000 mon.a (mon.0) 1739 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:57:10.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:09 smithi104 bash[9203]: audit 2020-09-08T15:57:09.860394+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? v1:172.21.15.104:0/552166936' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:57:10.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:09 smithi104 bash[9203]: cluster 2020-09-08T15:57:09.860441+0000 mon.a (mon.0) 1741 : cluster [DBG] osdmap e497: 12 total, 12 up, 12 in 2020-09-08T15:57:10.494 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 42 16 60266 60250 89.6476 103.625 0.00937734 0.0111521 2020-09-08T15:57:11.311 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:10 smithi055 bash[9296]: cluster 2020-09-08T15:57:09.906519+0000 mgr.x (mgr.24118) 895 : cluster [DBG] pgmap v1308: 17 pgs: 17 active+clean; 3.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 116 MiB/s wr, 1.85k op/s 2020-09-08T15:57:11.312 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:10 smithi166 bash[9446]: cluster 2020-09-08T15:57:09.906519+0000 mgr.x (mgr.24118) 895 : cluster [DBG] pgmap v1308: 17 pgs: 17 active+clean; 3.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 116 MiB/s wr, 1.85k op/s 2020-09-08T15:57:11.416 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:57:11.424 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:10 smithi104 bash[9203]: cluster 2020-09-08T15:57:09.906519+0000 mgr.x (mgr.24118) 895 : cluster [DBG] pgmap v1308: 17 pgs: 17 active+clean; 3.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 116 MiB/s wr, 1.85k op/s 2020-09-08T15:57:11.494 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 43 16 62005 61989 90.0902 108.688 0.00497046 0.0110871 2020-09-08T15:57:11.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:57:11 smithi055 bash[16646]: debug 2020-09-08T15:57:11.307+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:12.086 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:57:12.188 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:57:12.330 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:57:12.425 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:57:12 smithi104 bash[22305]: debug 2020-09-08T15:57:12.074+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:12.436 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:57:12.494 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 44 16 63605 63589 90.3151 100 0.00493565 0.0110692 2020-09-08T15:57:12.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:57:12 smithi166 bash[11928]: debug 2020-09-08T15:57:12.331+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:13.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:12 smithi166 bash[9446]: cluster 2020-09-08T15:57:11.906998+0000 mgr.x (mgr.24118) 896 : cluster [DBG] pgmap v1309: 17 pgs: 17 active+clean; 3.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 MiB/s wr, 1.72k op/s 2020-09-08T15:57:13.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:12 smithi055 bash[9296]: cluster 2020-09-08T15:57:11.906998+0000 mgr.x (mgr.24118) 896 : cluster [DBG] pgmap v1309: 17 pgs: 17 active+clean; 3.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 MiB/s wr, 1.72k op/s 2020-09-08T15:57:13.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:12 smithi104 bash[9203]: cluster 2020-09-08T15:57:11.906998+0000 mgr.x (mgr.24118) 896 : cluster [DBG] pgmap v1309: 17 pgs: 17 active+clean; 3.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 MiB/s wr, 1.72k op/s 2020-09-08T15:57:13.495 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 45 16 65165 65149 90.4746 97.5 0.0113588 0.0110502 2020-09-08T15:57:14.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:14 smithi104 bash[9203]: cluster 2020-09-08T15:57:13.179624+0000 osd.7 (osd.7) 97 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:14.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:14 smithi104 bash[9203]: cluster 2020-09-08T15:57:13.179706+0000 osd.7 (osd.7) 98 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:14.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:14 smithi166 bash[9446]: cluster 2020-09-08T15:57:13.179624+0000 osd.7 (osd.7) 97 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:14.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:14 smithi166 bash[9446]: cluster 2020-09-08T15:57:13.179706+0000 osd.7 (osd.7) 98 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:14.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:14 smithi055 bash[9296]: cluster 2020-09-08T15:57:13.179624+0000 osd.7 (osd.7) 97 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:14.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:14 smithi055 bash[9296]: cluster 2020-09-08T15:57:13.179706+0000 osd.7 (osd.7) 98 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:14.494 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 46 16 66590 66574 90.4437 89.0625 0.00476967 0.0110539 2020-09-08T15:57:15.172 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:57:15.173 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:57:15.359 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:57:15.368 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:57:15.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:15 smithi166 bash[9446]: cluster 2020-09-08T15:57:13.907595+0000 mgr.x (mgr.24118) 897 : cluster [DBG] pgmap v1310: 17 pgs: 17 active+clean; 3.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 114 MiB/s wr, 1.82k op/s 2020-09-08T15:57:15.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:15 smithi166 bash[9446]: cluster 2020-09-08T15:57:14.007609+0000 mon.a (mon.0) 1742 : cluster [DBG] osdmap e498: 12 total, 12 up, 12 in 2020-09-08T15:57:15.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:15 smithi166 bash[9446]: cluster 2020-09-08T15:57:14.571997+0000 osd.0 (osd.0) 69 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:15.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:15 smithi166 bash[9446]: cluster 2020-09-08T15:57:14.572078+0000 osd.0 (osd.0) 70 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:15.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:15 smithi166 bash[9446]: cluster 2020-09-08T15:57:14.858886+0000 mon.a (mon.0) 1743 : cluster [DBG] osdmap e499: 12 total, 12 up, 12 in 2020-09-08T15:57:15.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:15 smithi055 bash[9296]: cluster 2020-09-08T15:57:13.907595+0000 mgr.x (mgr.24118) 897 : cluster [DBG] pgmap v1310: 17 pgs: 17 active+clean; 3.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 114 MiB/s wr, 1.82k op/s 2020-09-08T15:57:15.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:15 smithi055 bash[9296]: cluster 2020-09-08T15:57:14.007609+0000 mon.a (mon.0) 1742 : cluster [DBG] osdmap e498: 12 total, 12 up, 12 in 2020-09-08T15:57:15.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:15 smithi055 bash[9296]: cluster 2020-09-08T15:57:14.571997+0000 osd.0 (osd.0) 69 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:15.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:15 smithi055 bash[9296]: cluster 2020-09-08T15:57:14.572078+0000 osd.0 (osd.0) 70 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:15.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:15 smithi055 bash[9296]: cluster 2020-09-08T15:57:14.858886+0000 mon.a (mon.0) 1743 : cluster [DBG] osdmap e499: 12 total, 12 up, 12 in 2020-09-08T15:57:15.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:15 smithi104 bash[9203]: cluster 2020-09-08T15:57:13.907595+0000 mgr.x (mgr.24118) 897 : cluster [DBG] pgmap v1310: 17 pgs: 17 active+clean; 3.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 114 MiB/s wr, 1.82k op/s 2020-09-08T15:57:15.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:15 smithi104 bash[9203]: cluster 2020-09-08T15:57:14.007609+0000 mon.a (mon.0) 1742 : cluster [DBG] osdmap e498: 12 total, 12 up, 12 in 2020-09-08T15:57:15.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:15 smithi104 bash[9203]: cluster 2020-09-08T15:57:14.571997+0000 osd.0 (osd.0) 69 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:15.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:15 smithi104 bash[9203]: cluster 2020-09-08T15:57:14.572078+0000 osd.0 (osd.0) 70 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:15.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:15 smithi104 bash[9203]: cluster 2020-09-08T15:57:14.858886+0000 mon.a (mon.0) 1743 : cluster [DBG] osdmap e499: 12 total, 12 up, 12 in 2020-09-08T15:57:15.494 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 47 16 68196 68180 90.6548 100.375 0.0168953 0.0110284 2020-09-08T15:57:16.494 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 48 16 69882 69866 90.9613 105.375 0.0123018 0.0109914 2020-09-08T15:57:16.793 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:57:16.897 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:17.035 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:17.139 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:57:17.148 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:17 smithi055 bash[9296]: cluster 2020-09-08T15:57:15.908308+0000 mgr.x (mgr.24118) 898 : cluster [DBG] pgmap v1313: 17 pgs: 17 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 93 MiB/s wr, 1.48k op/s 2020-09-08T15:57:17.148 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:57:16 smithi055 bash[14301]: debug 2020-09-08T15:57:16.791+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:17.149 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:57:17 smithi055 bash[16646]: debug 2020-09-08T15:57:17.027+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:17.265 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:57:17.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:17 smithi166 bash[9446]: cluster 2020-09-08T15:57:15.908308+0000 mgr.x (mgr.24118) 898 : cluster [DBG] pgmap v1313: 17 pgs: 17 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 93 MiB/s wr, 1.48k op/s 2020-09-08T15:57:17.368 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:57:17.378 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:17 smithi104 bash[9203]: cluster 2020-09-08T15:57:15.908308+0000 mgr.x (mgr.24118) 898 : cluster [DBG] pgmap v1313: 17 pgs: 17 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 93 MiB/s wr, 1.48k op/s 2020-09-08T15:57:17.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:57:17 smithi055 bash[14301]: debug 2020-09-08T15:57:17.263+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:17.502 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 49 16 71373 71357 91.0066 93.1875 0.00426626 0.0109857 2020-09-08T15:57:17.561 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:57:17.664 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:57:17.678 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:57:17 smithi104 bash[15273]: debug 2020-09-08T15:57:17.562+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:17.886 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:57:17.990 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:18.123 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:57:17 smithi104 bash[22305]: debug 2020-09-08T15:57:17.886+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:18.123 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:18.224 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:57:18.364 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:57:18.365 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:57:18 smithi055 bash[19003]: debug 2020-09-08T15:57:18.363+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:18.366 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:57:18 smithi055 bash[11956]: debug 2020-09-08T15:57:18.111+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:18.469 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:57:18.495 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 50 16 72702 72686 90.8475 83.0625 0.0115924 0.0110043 2020-09-08T15:57:18.602 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:57:18.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:57:18 smithi055 bash[14301]: debug 2020-09-08T15:57:18.595+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:18.704 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:57:19.010 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:19 smithi104 bash[9203]: cluster 2020-09-08T15:57:17.908780+0000 mgr.x (mgr.24118) 899 : cluster [DBG] pgmap v1314: 17 pgs: 17 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 112 MiB/s wr, 1.79k op/s 2020-09-08T15:57:19.368 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:57:19.368 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T15:57:19.368 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 0 2020-09-08T15:57:19.368 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:57:19.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:19 smithi166 bash[9446]: cluster 2020-09-08T15:57:17.908780+0000 mgr.x (mgr.24118) 899 : cluster [DBG] pgmap v1314: 17 pgs: 17 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 112 MiB/s wr, 1.79k op/s 2020-09-08T15:57:19.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:19 smithi055 bash[9296]: cluster 2020-09-08T15:57:17.908780+0000 mgr.x (mgr.24118) 899 : cluster [DBG] pgmap v1314: 17 pgs: 17 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 112 MiB/s wr, 1.79k op/s 2020-09-08T15:57:19.495 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 51 16 74265 74249 90.9814 97.6875 0.00525814 0.0109886 2020-09-08T15:57:20.275 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:57:20.384 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:57:20.495 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 52 16 75251 75235 90.4168 61.625 0.0117434 0.0110578 2020-09-08T15:57:20.536 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:57:20.640 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:57:20.648 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:57:20 smithi104 bash[17588]: debug 2020-09-08T15:57:20.274+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:20.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:57:20 smithi166 bash[18960]: debug 2020-09-08T15:57:20.535+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:20.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:20 smithi104 bash[9203]: cluster 2020-09-08T15:57:19.909276+0000 mgr.x (mgr.24118) 900 : cluster [DBG] pgmap v1315: 17 pgs: 17 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 103 MiB/s wr, 1.64k op/s 2020-09-08T15:57:21.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:20 smithi055 bash[9296]: cluster 2020-09-08T15:57:19.909276+0000 mgr.x (mgr.24118) 900 : cluster [DBG] pgmap v1315: 17 pgs: 17 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 103 MiB/s wr, 1.64k op/s 2020-09-08T15:57:21.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:20 smithi166 bash[9446]: cluster 2020-09-08T15:57:19.909276+0000 mgr.x (mgr.24118) 900 : cluster [DBG] pgmap v1315: 17 pgs: 17 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 103 MiB/s wr, 1.64k op/s 2020-09-08T15:57:21.497 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 53 16 76454 76438 90.1293 75.1875 0.00876574 0.0110923 2020-09-08T15:57:21.709 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:57:21.818 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:57:21.959 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:57:22.064 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:57:22.117 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:57:21 smithi166 bash[14254]: debug 2020-09-08T15:57:21.959+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:22.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:57:21 smithi104 bash[15273]: debug 2020-09-08T15:57:21.710+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:22.180 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:57:22.284 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:22.455 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2020-09-08T15:57:22.495 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 54 16 77941 77925 90.1812 92.9375 0.0195641 0.0110861 2020-09-08T15:57:22.617 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:57:22 smithi166 bash[16593]: debug 2020-09-08T15:57:22.179+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:22.634 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:57:23.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:22 smithi166 bash[9446]: cluster 2020-09-08T15:57:21.909816+0000 mgr.x (mgr.24118) 901 : cluster [DBG] pgmap v1316: 17 pgs: 17 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 89 MiB/s wr, 1.42k op/s 2020-09-08T15:57:23.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:22 smithi166 bash[9446]: cluster 2020-09-08T15:57:22.431944+0000 osd.5 (osd.5) 67 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:23.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:22 smithi166 bash[9446]: cluster 2020-09-08T15:57:22.432022+0000 osd.5 (osd.5) 68 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:23.414 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:22 smithi055 bash[9296]: cluster 2020-09-08T15:57:21.909816+0000 mgr.x (mgr.24118) 901 : cluster [DBG] pgmap v1316: 17 pgs: 17 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 89 MiB/s wr, 1.42k op/s 2020-09-08T15:57:23.415 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:22 smithi055 bash[9296]: cluster 2020-09-08T15:57:22.431944+0000 osd.5 (osd.5) 67 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:23.415 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:22 smithi055 bash[9296]: cluster 2020-09-08T15:57:22.432022+0000 osd.5 (osd.5) 68 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:23.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:22 smithi104 bash[9203]: cluster 2020-09-08T15:57:21.909816+0000 mgr.x (mgr.24118) 901 : cluster [DBG] pgmap v1316: 17 pgs: 17 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 89 MiB/s wr, 1.42k op/s 2020-09-08T15:57:23.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:22 smithi104 bash[9203]: cluster 2020-09-08T15:57:22.431944+0000 osd.5 (osd.5) 67 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:23.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:22 smithi104 bash[9203]: cluster 2020-09-08T15:57:22.432022+0000 osd.5 (osd.5) 68 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:23.495 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 55 16 78639 78623 89.3346 43.625 0.00542271 0.0111795 2020-09-08T15:57:23.992 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:24.095 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:57:24.220 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:57:24.234 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:57:24 smithi055 bash[14301]: debug 2020-09-08T15:57:24.214+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:24.235 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:57:23 smithi055 bash[16646]: debug 2020-09-08T15:57:23.990+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:24.235 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:23 smithi055 bash[9296]: cluster 2020-09-08T15:57:22.978277+0000 mon.a (mon.0) 1744 : cluster [DBG] osdmap e500: 12 total, 12 up, 12 in 2020-09-08T15:57:24.335 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:57:24.354 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:23 smithi104 bash[9203]: cluster 2020-09-08T15:57:22.978277+0000 mon.a (mon.0) 1744 : cluster [DBG] osdmap e500: 12 total, 12 up, 12 in 2020-09-08T15:57:24.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:23 smithi166 bash[9446]: cluster 2020-09-08T15:57:22.978277+0000 mon.a (mon.0) 1744 : cluster [DBG] osdmap e500: 12 total, 12 up, 12 in 2020-09-08T15:57:24.495 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 56 16 79956 79940 89.2091 82.3125 0.006801 0.0112061 2020-09-08T15:57:24.547 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:57:24.642 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:57:24 smithi104 bash[22305]: debug 2020-09-08T15:57:24.546+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:24.660 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:24.757 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:24.758 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:57:24.790 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:57:24.838 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:57:24.867 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:57:24.903 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:57:24.940 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:57:24.977 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:57:24.985 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:57:24 smithi055 bash[16646]: debug 2020-09-08T15:57:24.758+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:25.029 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:57:25.050 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:57:25.076 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:57:25.119 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:57:25.152 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:57:25.155 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:57:25.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:57:25 smithi104 bash[17588]: debug 2020-09-08T15:57:25.042+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:25.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:24 smithi104 bash[9203]: cluster 2020-09-08T15:57:23.910274+0000 mgr.x (mgr.24118) 902 : cluster [DBG] pgmap v1318: 17 pgs: 17 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 91 MiB/s wr, 1.46k op/s 2020-09-08T15:57:25.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:24 smithi104 bash[9203]: cluster 2020-09-08T15:57:24.087879+0000 osd.10 (osd.10) 61 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:25.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:24 smithi104 bash[9203]: cluster 2020-09-08T15:57:24.087964+0000 osd.10 (osd.10) 62 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:25.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:24 smithi104 bash[9203]: audit 2020-09-08T15:57:24.720055+0000 mon.a (mon.0) 1745 : audit [DBG] from='client.? v1:172.21.15.104:0/231338066' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:57:25.200 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:57:25.231 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:57:25.232 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:57:25.234 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:57:25.234 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:57:25.235 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:57:25.236 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:57:25.236 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:57:25.236 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:57:25.236 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:57:25.237 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:57:25.237 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:57:25.238 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:57:25.238 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:57:25.298 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:24 smithi166 bash[9446]: cluster 2020-09-08T15:57:23.910274+0000 mgr.x (mgr.24118) 902 : cluster [DBG] pgmap v1318: 17 pgs: 17 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 91 MiB/s wr, 1.46k op/s 2020-09-08T15:57:25.298 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:24 smithi166 bash[9446]: cluster 2020-09-08T15:57:24.087879+0000 osd.10 (osd.10) 61 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:25.298 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:24 smithi166 bash[9446]: cluster 2020-09-08T15:57:24.087964+0000 osd.10 (osd.10) 62 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:25.299 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:24 smithi166 bash[9446]: audit 2020-09-08T15:57:24.720055+0000 mon.a (mon.0) 1745 : audit [DBG] from='client.? v1:172.21.15.104:0/231338066' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:57:25.299 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:57:25 smithi166 bash[16593]: debug 2020-09-08T15:57:25.299+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:25.301 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:57:25.406 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:25.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:24 smithi055 bash[9296]: cluster 2020-09-08T15:57:23.910274+0000 mgr.x (mgr.24118) 902 : cluster [DBG] pgmap v1318: 17 pgs: 17 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 91 MiB/s wr, 1.46k op/s 2020-09-08T15:57:25.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:24 smithi055 bash[9296]: cluster 2020-09-08T15:57:24.087879+0000 osd.10 (osd.10) 61 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:25.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:24 smithi055 bash[9296]: cluster 2020-09-08T15:57:24.087964+0000 osd.10 (osd.10) 62 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:25.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:24 smithi055 bash[9296]: audit 2020-09-08T15:57:24.720055+0000 mon.a (mon.0) 1745 : audit [DBG] from='client.? v1:172.21.15.104:0/231338066' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:57:25.495 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 57 16 81290 81274 89.1066 83.375 0.0174805 0.01122 2020-09-08T15:57:25.526 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:25.631 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:25.763 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:57:25 smithi055 bash[16646]: debug 2020-09-08T15:57:25.526+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:25.764 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:25.866 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:57:25.997 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:57:25.998 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:57:26.101 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:57:26.110 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:25 smithi055 bash[9296]: cluster 2020-09-08T15:57:24.986392+0000 mon.a (mon.0) 1746 : cluster [DBG] osdmap e501: 12 total, 12 up, 12 in 2020-09-08T15:57:26.110 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:25 smithi055 bash[9296]: audit 2020-09-08T15:57:25.206450+0000 mon.b (mon.1) 104 : audit [INF] from='client.? v1:172.21.15.104:0/2486256937' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:57:26.110 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:25 smithi055 bash[9296]: audit 2020-09-08T15:57:25.207119+0000 mon.a (mon.0) 1747 : audit [INF] from='client.25957 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:57:26.111 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:57:25 smithi055 bash[11956]: debug 2020-09-08T15:57:25.758+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:26.220 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:57:26.323 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:57:26.332 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:25 smithi104 bash[9203]: cluster 2020-09-08T15:57:24.986392+0000 mon.a (mon.0) 1746 : cluster [DBG] osdmap e501: 12 total, 12 up, 12 in 2020-09-08T15:57:26.333 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:25 smithi104 bash[9203]: audit 2020-09-08T15:57:25.206450+0000 mon.b (mon.1) 104 : audit [INF] from='client.? v1:172.21.15.104:0/2486256937' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:57:26.333 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:25 smithi104 bash[9203]: audit 2020-09-08T15:57:25.207119+0000 mon.a (mon.0) 1747 : audit [INF] from='client.25957 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:57:26.368 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:57:26 smithi166 bash[16593]: debug 2020-09-08T15:57:25.999+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:26.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:25 smithi166 bash[9446]: cluster 2020-09-08T15:57:24.986392+0000 mon.a (mon.0) 1746 : cluster [DBG] osdmap e501: 12 total, 12 up, 12 in 2020-09-08T15:57:26.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:25 smithi166 bash[9446]: audit 2020-09-08T15:57:25.206450+0000 mon.b (mon.1) 104 : audit [INF] from='client.? v1:172.21.15.104:0/2486256937' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:57:26.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:25 smithi166 bash[9446]: audit 2020-09-08T15:57:25.207119+0000 mon.a (mon.0) 1747 : audit [INF] from='client.25957 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:57:26.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:57:26 smithi055 bash[14301]: debug 2020-09-08T15:57:26.218+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:26.495 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 58 16 82755 82739 89.1487 91.5625 0.00868327 0.0112148 2020-09-08T15:57:27.234 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:57:27.338 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:57:27.349 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:26 smithi166 bash[9446]: cluster 2020-09-08T15:57:25.912556+0000 mgr.x (mgr.24118) 903 : cluster [DBG] pgmap v1320: 17 pgs: 17 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s wr, 1.09k op/s 2020-09-08T15:57:27.349 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:26 smithi166 bash[9446]: cluster 2020-09-08T15:57:25.990281+0000 mon.a (mon.0) 1748 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:57:27.349 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:26 smithi166 bash[9446]: audit 2020-09-08T15:57:26.000221+0000 mon.a (mon.0) 1749 : audit [INF] from='client.25957 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:57:27.350 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:26 smithi166 bash[9446]: cluster 2020-09-08T15:57:26.000290+0000 mon.a (mon.0) 1750 : cluster [DBG] osdmap e502: 12 total, 12 up, 12 in 2020-09-08T15:57:27.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:26 smithi055 bash[9296]: cluster 2020-09-08T15:57:25.912556+0000 mgr.x (mgr.24118) 903 : cluster [DBG] pgmap v1320: 17 pgs: 17 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s wr, 1.09k op/s 2020-09-08T15:57:27.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:26 smithi055 bash[9296]: cluster 2020-09-08T15:57:25.990281+0000 mon.a (mon.0) 1748 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:57:27.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:26 smithi055 bash[9296]: audit 2020-09-08T15:57:26.000221+0000 mon.a (mon.0) 1749 : audit [INF] from='client.25957 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:57:27.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:26 smithi055 bash[9296]: cluster 2020-09-08T15:57:26.000290+0000 mon.a (mon.0) 1750 : cluster [DBG] osdmap e502: 12 total, 12 up, 12 in 2020-09-08T15:57:27.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:57:27 smithi104 bash[19922]: debug 2020-09-08T15:57:27.234+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:27.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:26 smithi104 bash[9203]: cluster 2020-09-08T15:57:25.912556+0000 mgr.x (mgr.24118) 903 : cluster [DBG] pgmap v1320: 17 pgs: 17 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s wr, 1.09k op/s 2020-09-08T15:57:27.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:26 smithi104 bash[9203]: cluster 2020-09-08T15:57:25.990281+0000 mon.a (mon.0) 1748 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:57:27.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:26 smithi104 bash[9203]: audit 2020-09-08T15:57:26.000221+0000 mon.a (mon.0) 1749 : audit [INF] from='client.25957 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:57:27.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:26 smithi104 bash[9203]: cluster 2020-09-08T15:57:26.000290+0000 mon.a (mon.0) 1750 : cluster [DBG] osdmap e502: 12 total, 12 up, 12 in 2020-09-08T15:57:27.487 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:57:27.497 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 59 16 84223 84207 89.1927 91.75 0.0068794 0.0112088 2020-09-08T15:57:27.591 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:57:27.602 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:57:27 smithi166 bash[16593]: debug 2020-09-08T15:57:27.487+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:27.717 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:57:27.821 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:57:27.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:57:27 smithi166 bash[18960]: debug 2020-09-08T15:57:27.715+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:27.928 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:57:28.032 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:57:28.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:28 smithi166 bash[9446]: cluster 2020-09-08T15:57:27.418991+0000 osd.2 (osd.2) 72 : cluster [DBG] 6.f scrub starts 2020-09-08T15:57:28.368 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:57:27 smithi166 bash[11928]: debug 2020-09-08T15:57:27.927+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:28.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:28 smithi055 bash[9296]: cluster 2020-09-08T15:57:27.418991+0000 osd.2 (osd.2) 72 : cluster [DBG] 6.f scrub starts 2020-09-08T15:57:28.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:28 smithi104 bash[9203]: cluster 2020-09-08T15:57:27.418991+0000 osd.2 (osd.2) 72 : cluster [DBG] 6.f scrub starts 2020-09-08T15:57:28.496 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 15:57:28.499872 min lat: 0.00166475 max lat: 15.0142 avg lat: 0.0112098 2020-09-08T15:57:28.496 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:57:28.496 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 60 16 85647 85631 89.1893 89 0.0112063 0.0112098 2020-09-08T15:57:28.948 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:57:29.052 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:57:29.257 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:57:28 smithi104 bash[22305]: debug 2020-09-08T15:57:28.946+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:29.258 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:29 smithi104 bash[9203]: cluster 2020-09-08T15:57:27.913075+0000 mgr.x (mgr.24118) 904 : cluster [DBG] pgmap v1322: 17 pgs: 17 active+clean; 5.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s wr, 1.58k op/s 2020-09-08T15:57:29.258 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:29 smithi104 bash[9203]: cluster 2020-09-08T15:57:28.702342+0000 osd.5 (osd.5) 69 : cluster [DBG] 6.7 scrub starts 2020-09-08T15:57:29.258 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:29 smithi104 bash[9203]: cluster 2020-09-08T15:57:28.706476+0000 osd.2 (osd.2) 73 : cluster [DBG] 6.f scrub ok 2020-09-08T15:57:29.258 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:57:29 smithi104 bash[15273]: debug 2020-09-08T15:57:29.258+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:29.261 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:57:29.366 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:57:29.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:29 smithi166 bash[9446]: cluster 2020-09-08T15:57:27.913075+0000 mgr.x (mgr.24118) 904 : cluster [DBG] pgmap v1322: 17 pgs: 17 active+clean; 5.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s wr, 1.58k op/s 2020-09-08T15:57:29.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:29 smithi166 bash[9446]: cluster 2020-09-08T15:57:28.702342+0000 osd.5 (osd.5) 69 : cluster [DBG] 6.7 scrub starts 2020-09-08T15:57:29.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:29 smithi166 bash[9446]: cluster 2020-09-08T15:57:28.706476+0000 osd.2 (osd.2) 73 : cluster [DBG] 6.f scrub ok 2020-09-08T15:57:29.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:29 smithi055 bash[9296]: cluster 2020-09-08T15:57:27.913075+0000 mgr.x (mgr.24118) 904 : cluster [DBG] pgmap v1322: 17 pgs: 17 active+clean; 5.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s wr, 1.58k op/s 2020-09-08T15:57:29.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:29 smithi055 bash[9296]: cluster 2020-09-08T15:57:28.702342+0000 osd.5 (osd.5) 69 : cluster [DBG] 6.7 scrub starts 2020-09-08T15:57:29.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:29 smithi055 bash[9296]: cluster 2020-09-08T15:57:28.706476+0000 osd.2 (osd.2) 73 : cluster [DBG] 6.f scrub ok 2020-09-08T15:57:29.496 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 61 16 87028 87012 89.142 86.3125 0.00656465 0.0112158 2020-09-08T15:57:29.930 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:57:30.026 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:57:30.036 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:30.123 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:57:30.177 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:30.195 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:57:30.282 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:30.285 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:57:29 smithi104 bash[19922]: debug 2020-09-08T15:57:29.930+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:30.286 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:30 smithi104 bash[9203]: cluster 2020-09-08T15:57:28.810852+0000 osd.3 (osd.3) 71 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:30.286 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:30 smithi104 bash[9203]: cluster 2020-09-08T15:57:28.810945+0000 osd.3 (osd.3) 72 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:30.286 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:30 smithi104 bash[9203]: cluster 2020-09-08T15:57:29.018832+0000 mon.a (mon.0) 1751 : cluster [DBG] osdmap e503: 12 total, 12 up, 12 in 2020-09-08T15:57:30.290 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:57:30 smithi055 bash[16646]: debug 2020-09-08T15:57:30.174+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:30.291 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:30 smithi055 bash[9296]: cluster 2020-09-08T15:57:28.810852+0000 osd.3 (osd.3) 71 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:30.291 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:30 smithi055 bash[9296]: cluster 2020-09-08T15:57:28.810945+0000 osd.3 (osd.3) 72 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:30.291 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:30 smithi055 bash[9296]: cluster 2020-09-08T15:57:29.018832+0000 mon.a (mon.0) 1751 : cluster [DBG] osdmap e503: 12 total, 12 up, 12 in 2020-09-08T15:57:30.294 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:57:30.374 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:30 smithi166 bash[9446]: cluster 2020-09-08T15:57:28.810852+0000 osd.3 (osd.3) 71 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:30.374 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:30 smithi166 bash[9446]: cluster 2020-09-08T15:57:28.810945+0000 osd.3 (osd.3) 72 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:30.374 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:30 smithi166 bash[9446]: cluster 2020-09-08T15:57:29.018832+0000 mon.a (mon.0) 1751 : cluster [DBG] osdmap e503: 12 total, 12 up, 12 in 2020-09-08T15:57:30.400 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:30.496 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 62 16 88272 88256 88.9582 77.75 0.0126849 0.0112386 2020-09-08T15:57:30.505 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:30.618 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:57:30 smithi055 bash[16646]: debug 2020-09-08T15:57:30.398+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:30.621 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:30.725 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:30.840 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:30.910 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:57:30 smithi055 bash[11956]: debug 2020-09-08T15:57:30.618+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:30.910 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:57:30 smithi055 bash[11956]: debug 2020-09-08T15:57:30.838+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:30.955 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:57:31.087 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:57:31.191 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:57:31.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:31 smithi166 bash[9446]: cluster 2020-09-08T15:57:29.913453+0000 mgr.x (mgr.24118) 905 : cluster [DBG] pgmap v1324: 17 pgs: 17 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T15:57:31.411 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:57:31 smithi055 bash[19003]: debug 2020-09-08T15:57:31.082+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:31.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:31 smithi055 bash[9296]: cluster 2020-09-08T15:57:29.913453+0000 mgr.x (mgr.24118) 905 : cluster [DBG] pgmap v1324: 17 pgs: 17 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T15:57:31.450 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:31 smithi104 bash[9203]: cluster 2020-09-08T15:57:29.913453+0000 mgr.x (mgr.24118) 905 : cluster [DBG] pgmap v1324: 17 pgs: 17 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T15:57:31.496 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 63 16 89641 89625 88.9042 85.5625 0.0121722 0.0112458 2020-09-08T15:57:31.569 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:57:31.569 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:57:31.590 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:57:31.702 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:57:31.834 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:57:31.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:57:31 smithi104 bash[15273]: debug 2020-09-08T15:57:31.590+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:31.946 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:57:32.022 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:32 smithi166 bash[9446]: cluster 2020-09-08T15:57:31.152254+0000 osd.5 (osd.5) 70 : cluster [DBG] 6.7 scrub ok 2020-09-08T15:57:32.023 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:57:31 smithi166 bash[16593]: debug 2020-09-08T15:57:31.835+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:32.418 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:32 smithi055 bash[9296]: cluster 2020-09-08T15:57:31.152254+0000 osd.5 (osd.5) 70 : cluster [DBG] 6.7 scrub ok 2020-09-08T15:57:32.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:32 smithi104 bash[9203]: cluster 2020-09-08T15:57:31.152254+0000 osd.5 (osd.5) 70 : cluster [DBG] 6.7 scrub ok 2020-09-08T15:57:32.496 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 64 16 91003 90987 88.845 85.125 0.0111307 0.0112536 2020-09-08T15:57:33.080 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:57:33.184 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:33.335 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:33 smithi055 bash[9296]: cluster 2020-09-08T15:57:31.913899+0000 mgr.x (mgr.24118) 906 : cluster [DBG] pgmap v1325: 17 pgs: 17 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T15:57:33.335 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:33 smithi055 bash[9296]: cluster 2020-09-08T15:57:32.017722+0000 osd.0 (osd.0) 71 : cluster [DBG] 6.0 scrub starts 2020-09-08T15:57:33.336 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:33 smithi055 bash[9296]: cluster 2020-09-08T15:57:32.082065+0000 osd.1 (osd.1) 96 : cluster [DBG] 6.4 scrub starts 2020-09-08T15:57:33.336 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:33 smithi055 bash[9296]: cluster 2020-09-08T15:57:32.132908+0000 osd.7 (osd.7) 99 : cluster [DBG] 6.3 scrub starts 2020-09-08T15:57:33.336 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:33.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:33 smithi166 bash[9446]: cluster 2020-09-08T15:57:31.913899+0000 mgr.x (mgr.24118) 906 : cluster [DBG] pgmap v1325: 17 pgs: 17 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T15:57:33.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:33 smithi166 bash[9446]: cluster 2020-09-08T15:57:32.017722+0000 osd.0 (osd.0) 71 : cluster [DBG] 6.0 scrub starts 2020-09-08T15:57:33.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:33 smithi166 bash[9446]: cluster 2020-09-08T15:57:32.082065+0000 osd.1 (osd.1) 96 : cluster [DBG] 6.4 scrub starts 2020-09-08T15:57:33.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:33 smithi166 bash[9446]: cluster 2020-09-08T15:57:32.132908+0000 osd.7 (osd.7) 99 : cluster [DBG] 6.3 scrub starts 2020-09-08T15:57:33.433 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:33 smithi104 bash[9203]: cluster 2020-09-08T15:57:31.913899+0000 mgr.x (mgr.24118) 906 : cluster [DBG] pgmap v1325: 17 pgs: 17 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T15:57:33.433 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:33 smithi104 bash[9203]: cluster 2020-09-08T15:57:32.017722+0000 osd.0 (osd.0) 71 : cluster [DBG] 6.0 scrub starts 2020-09-08T15:57:33.433 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:33 smithi104 bash[9203]: cluster 2020-09-08T15:57:32.082065+0000 osd.1 (osd.1) 96 : cluster [DBG] 6.4 scrub starts 2020-09-08T15:57:33.433 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:33 smithi104 bash[9203]: cluster 2020-09-08T15:57:32.132908+0000 osd.7 (osd.7) 99 : cluster [DBG] 6.3 scrub starts 2020-09-08T15:57:33.434 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:57:33 smithi104 bash[19922]: debug 2020-09-08T15:57:33.082+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:33.454 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:57:33.498 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 65 16 92239 92223 88.6665 77.25 0.00912589 0.0112752 2020-09-08T15:57:33.499 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2020-09-08T15:57:33.597 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:57:33.658 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:57:33 smithi055 bash[11956]: debug 2020-09-08T15:57:33.334+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:33.662 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:57:33.702 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:57:33.836 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:57:33.837 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:57:33 smithi166 bash[16593]: debug 2020-09-08T15:57:33.835+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:33.838 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:57:33 smithi166 bash[18960]: debug 2020-09-08T15:57:33.599+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:33.941 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:34.074 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:34.178 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:57:34.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:57:34 smithi055 bash[11956]: debug 2020-09-08T15:57:34.074+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:34.497 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 66 16 93439 93423 88.4593 75 0.0053781 0.0113026 2020-09-08T15:57:34.899 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:57:35.003 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:57:35.129 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:57:35.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:57:34 smithi104 bash[19922]: debug 2020-09-08T15:57:34.902+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:35.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:35 smithi104 bash[9203]: cluster 2020-09-08T15:57:33.400571+0000 osd.8 (osd.8) 63 : cluster [DBG] 6.1 scrub starts 2020-09-08T15:57:35.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:35 smithi104 bash[9203]: cluster 2020-09-08T15:57:33.645979+0000 osd.0 (osd.0) 72 : cluster [DBG] 6.0 scrub ok 2020-09-08T15:57:35.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:35 smithi104 bash[9203]: cluster 2020-09-08T15:57:33.914400+0000 mgr.x (mgr.24118) 907 : cluster [DBG] pgmap v1326: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s wr, 1.40k op/s 2020-09-08T15:57:35.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:35 smithi104 bash[9203]: cluster 2020-09-08T15:57:34.049661+0000 osd.7 (osd.7) 100 : cluster [DBG] 6.3 scrub ok 2020-09-08T15:57:35.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:35 smithi104 bash[9203]: cluster 2020-09-08T15:57:34.109499+0000 osd.1 (osd.1) 97 : cluster [DBG] 6.4 scrub ok 2020-09-08T15:57:35.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:35 smithi104 bash[9203]: cluster 2020-09-08T15:57:34.155068+0000 osd.1 (osd.1) 98 : cluster [DBG] 6.e scrub starts 2020-09-08T15:57:35.240 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:57:35.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:57:35 smithi166 bash[11928]: debug 2020-09-08T15:57:35.131+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:35.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:35 smithi166 bash[9446]: cluster 2020-09-08T15:57:33.400571+0000 osd.8 (osd.8) 63 : cluster [DBG] 6.1 scrub starts 2020-09-08T15:57:35.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:35 smithi166 bash[9446]: cluster 2020-09-08T15:57:33.645979+0000 osd.0 (osd.0) 72 : cluster [DBG] 6.0 scrub ok 2020-09-08T15:57:35.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:35 smithi166 bash[9446]: cluster 2020-09-08T15:57:33.914400+0000 mgr.x (mgr.24118) 907 : cluster [DBG] pgmap v1326: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s wr, 1.40k op/s 2020-09-08T15:57:35.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:35 smithi166 bash[9446]: cluster 2020-09-08T15:57:34.049661+0000 osd.7 (osd.7) 100 : cluster [DBG] 6.3 scrub ok 2020-09-08T15:57:35.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:35 smithi166 bash[9446]: cluster 2020-09-08T15:57:34.109499+0000 osd.1 (osd.1) 97 : cluster [DBG] 6.4 scrub ok 2020-09-08T15:57:35.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:35 smithi166 bash[9446]: cluster 2020-09-08T15:57:34.155068+0000 osd.1 (osd.1) 98 : cluster [DBG] 6.e scrub starts 2020-09-08T15:57:35.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:35 smithi055 bash[9296]: cluster 2020-09-08T15:57:33.400571+0000 osd.8 (osd.8) 63 : cluster [DBG] 6.1 scrub starts 2020-09-08T15:57:35.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:35 smithi055 bash[9296]: cluster 2020-09-08T15:57:33.645979+0000 osd.0 (osd.0) 72 : cluster [DBG] 6.0 scrub ok 2020-09-08T15:57:35.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:35 smithi055 bash[9296]: cluster 2020-09-08T15:57:33.914400+0000 mgr.x (mgr.24118) 907 : cluster [DBG] pgmap v1326: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s wr, 1.40k op/s 2020-09-08T15:57:35.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:35 smithi055 bash[9296]: cluster 2020-09-08T15:57:34.049661+0000 osd.7 (osd.7) 100 : cluster [DBG] 6.3 scrub ok 2020-09-08T15:57:35.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:35 smithi055 bash[9296]: cluster 2020-09-08T15:57:34.109499+0000 osd.1 (osd.1) 97 : cluster [DBG] 6.4 scrub ok 2020-09-08T15:57:35.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:35 smithi055 bash[9296]: cluster 2020-09-08T15:57:34.155068+0000 osd.1 (osd.1) 98 : cluster [DBG] 6.e scrub starts 2020-09-08T15:57:35.496 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 67 16 94526 94510 88.1529 67.9375 0.013809 0.0113416 2020-09-08T15:57:35.903 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:57:36.007 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:57:36.173 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:57:36.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:36 smithi104 bash[9203]: cluster 2020-09-08T15:57:34.586765+0000 osd.8 (osd.8) 64 : cluster [DBG] 6.1 scrub ok 2020-09-08T15:57:36.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:36 smithi104 bash[9203]: cluster 2020-09-08T15:57:34.754037+0000 osd.5 (osd.5) 71 : cluster [DBG] 6.a scrub starts 2020-09-08T15:57:36.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:36 smithi104 bash[9203]: audit 2020-09-08T15:57:35.042824+0000 mon.b (mon.1) 105 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:57:36.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:36 smithi104 bash[9203]: audit 2020-09-08T15:57:35.043400+0000 mon.a (mon.0) 1752 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:57:36.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:36 smithi104 bash[9203]: audit 2020-09-08T15:57:35.046648+0000 mon.a (mon.0) 1753 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:57:36.177 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:57:35 smithi104 bash[19922]: debug 2020-09-08T15:57:35.906+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:36.278 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:36.301 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:36 smithi055 bash[9296]: cluster 2020-09-08T15:57:34.586765+0000 osd.8 (osd.8) 64 : cluster [DBG] 6.1 scrub ok 2020-09-08T15:57:36.301 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:36 smithi055 bash[9296]: cluster 2020-09-08T15:57:34.754037+0000 osd.5 (osd.5) 71 : cluster [DBG] 6.a scrub starts 2020-09-08T15:57:36.302 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:36 smithi055 bash[9296]: audit 2020-09-08T15:57:35.042824+0000 mon.b (mon.1) 105 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:57:36.302 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:36 smithi055 bash[9296]: audit 2020-09-08T15:57:35.043400+0000 mon.a (mon.0) 1752 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:57:36.302 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:36 smithi055 bash[9296]: audit 2020-09-08T15:57:35.046648+0000 mon.a (mon.0) 1753 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:57:36.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:57:36 smithi166 bash[11928]: debug 2020-09-08T15:57:36.171+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:36.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:36 smithi166 bash[9446]: cluster 2020-09-08T15:57:34.586765+0000 osd.8 (osd.8) 64 : cluster [DBG] 6.1 scrub ok 2020-09-08T15:57:36.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:36 smithi166 bash[9446]: cluster 2020-09-08T15:57:34.754037+0000 osd.5 (osd.5) 71 : cluster [DBG] 6.a scrub starts 2020-09-08T15:57:36.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:36 smithi166 bash[9446]: audit 2020-09-08T15:57:35.042824+0000 mon.b (mon.1) 105 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:57:36.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:36 smithi166 bash[9446]: audit 2020-09-08T15:57:35.043400+0000 mon.a (mon.0) 1752 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:57:36.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:36 smithi166 bash[9446]: audit 2020-09-08T15:57:35.046648+0000 mon.a (mon.0) 1753 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:57:36.432 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:36.496 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 68 16 95751 95735 87.9824 76.5625 0.00554812 0.0113631 2020-09-08T15:57:36.540 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:57:36.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:57:36 smithi055 bash[16646]: debug 2020-09-08T15:57:36.430+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:36.671 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:57:36.776 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:36.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:57:36 smithi166 bash[11928]: debug 2020-09-08T15:57:36.671+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:36.913 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:37.014 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:57:37.048 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:57:36 smithi055 bash[11956]: debug 2020-09-08T15:57:36.910+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:37.049 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:37 smithi055 bash[9296]: cluster 2020-09-08T15:57:35.300425+0000 osd.8 (osd.8) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:37.050 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:37 smithi055 bash[9296]: cluster 2020-09-08T15:57:35.300523+0000 osd.8 (osd.8) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:37.050 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:37 smithi055 bash[9296]: cluster 2020-09-08T15:57:35.914869+0000 mgr.x (mgr.24118) 908 : cluster [DBG] pgmap v1327: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 5.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 83 MiB/s wr, 1.33k op/s 2020-09-08T15:57:37.050 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:37 smithi055 bash[9296]: cluster 2020-09-08T15:57:36.051896+0000 mon.a (mon.0) 1754 : cluster [DBG] osdmap e504: 12 total, 12 up, 12 in 2020-09-08T15:57:37.050 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:37 smithi055 bash[9296]: cluster 2020-09-08T15:57:36.195577+0000 osd.1 (osd.1) 99 : cluster [DBG] 6.e scrub ok 2020-09-08T15:57:37.051 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:37 smithi055 bash[9296]: cluster 2020-09-08T15:57:36.679049+0000 osd.5 (osd.5) 72 : cluster [DBG] 6.a scrub ok 2020-09-08T15:57:37.130 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:57:37.234 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:57:37.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:57:37 smithi166 bash[14254]: debug 2020-09-08T15:57:37.131+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:37.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:37 smithi166 bash[9446]: cluster 2020-09-08T15:57:35.300425+0000 osd.8 (osd.8) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:37.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:37 smithi166 bash[9446]: cluster 2020-09-08T15:57:35.300523+0000 osd.8 (osd.8) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:37.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:37 smithi166 bash[9446]: cluster 2020-09-08T15:57:35.914869+0000 mgr.x (mgr.24118) 908 : cluster [DBG] pgmap v1327: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 5.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 83 MiB/s wr, 1.33k op/s 2020-09-08T15:57:37.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:37 smithi166 bash[9446]: cluster 2020-09-08T15:57:36.051896+0000 mon.a (mon.0) 1754 : cluster [DBG] osdmap e504: 12 total, 12 up, 12 in 2020-09-08T15:57:37.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:37 smithi166 bash[9446]: cluster 2020-09-08T15:57:36.195577+0000 osd.1 (osd.1) 99 : cluster [DBG] 6.e scrub ok 2020-09-08T15:57:37.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:37 smithi166 bash[9446]: cluster 2020-09-08T15:57:36.679049+0000 osd.5 (osd.5) 72 : cluster [DBG] 6.a scrub ok 2020-09-08T15:57:37.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:37 smithi104 bash[9203]: cluster 2020-09-08T15:57:35.300425+0000 osd.8 (osd.8) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:37.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:37 smithi104 bash[9203]: cluster 2020-09-08T15:57:35.300523+0000 osd.8 (osd.8) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:37.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:37 smithi104 bash[9203]: cluster 2020-09-08T15:57:35.914869+0000 mgr.x (mgr.24118) 908 : cluster [DBG] pgmap v1327: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 5.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 83 MiB/s wr, 1.33k op/s 2020-09-08T15:57:37.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:37 smithi104 bash[9203]: cluster 2020-09-08T15:57:36.051896+0000 mon.a (mon.0) 1754 : cluster [DBG] osdmap e504: 12 total, 12 up, 12 in 2020-09-08T15:57:37.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:37 smithi104 bash[9203]: cluster 2020-09-08T15:57:36.195577+0000 osd.1 (osd.1) 99 : cluster [DBG] 6.e scrub ok 2020-09-08T15:57:37.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:37 smithi104 bash[9203]: cluster 2020-09-08T15:57:36.679049+0000 osd.5 (osd.5) 72 : cluster [DBG] 6.a scrub ok 2020-09-08T15:57:37.497 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 69 16 97059 97043 87.8919 81.75 0.00926914 0.0113749 2020-09-08T15:57:38.215 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:57:38.317 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:57:38.326 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:57:38 smithi104 bash[17588]: debug 2020-09-08T15:57:38.214+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:38.326 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:38 smithi104 bash[9203]: cluster 2020-09-08T15:57:37.154677+0000 osd.0 (osd.0) 73 : cluster [DBG] 6.9 scrub starts 2020-09-08T15:57:38.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:38 smithi166 bash[9446]: cluster 2020-09-08T15:57:37.154677+0000 osd.0 (osd.0) 73 : cluster [DBG] 6.9 scrub starts 2020-09-08T15:57:38.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:38 smithi055 bash[9296]: cluster 2020-09-08T15:57:37.154677+0000 osd.0 (osd.0) 73 : cluster [DBG] 6.9 scrub starts 2020-09-08T15:57:38.496 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 70 16 98169 98153 87.6273 69.375 0.0098661 0.0114093 2020-09-08T15:57:38.558 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:57:38.638 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:57:38 smithi104 bash[15273]: debug 2020-09-08T15:57:38.558+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:38.662 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:57:39.225 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:39 smithi055 bash[9296]: cluster 2020-09-08T15:57:37.807432+0000 osd.5 (osd.5) 73 : cluster [DBG] 6.d scrub starts 2020-09-08T15:57:39.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:39 smithi166 bash[9446]: cluster 2020-09-08T15:57:37.807432+0000 osd.5 (osd.5) 73 : cluster [DBG] 6.d scrub starts 2020-09-08T15:57:39.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:39 smithi166 bash[9446]: cluster 2020-09-08T15:57:37.915286+0000 mgr.x (mgr.24118) 909 : cluster [DBG] pgmap v1329: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s wr, 1.32k op/s 2020-09-08T15:57:39.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:39 smithi166 bash[9446]: cluster 2020-09-08T15:57:37.974453+0000 osd.3 (osd.3) 73 : cluster [DBG] 6.6 scrub starts 2020-09-08T15:57:39.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:39 smithi166 bash[9446]: cluster 2020-09-08T15:57:38.038260+0000 osd.7 (osd.7) 101 : cluster [DBG] 6.5 scrub starts 2020-09-08T15:57:39.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:39 smithi104 bash[9203]: cluster 2020-09-08T15:57:37.807432+0000 osd.5 (osd.5) 73 : cluster [DBG] 6.d scrub starts 2020-09-08T15:57:39.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:39 smithi104 bash[9203]: cluster 2020-09-08T15:57:37.915286+0000 mgr.x (mgr.24118) 909 : cluster [DBG] pgmap v1329: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s wr, 1.32k op/s 2020-09-08T15:57:39.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:39 smithi104 bash[9203]: cluster 2020-09-08T15:57:37.974453+0000 osd.3 (osd.3) 73 : cluster [DBG] 6.6 scrub starts 2020-09-08T15:57:39.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:39 smithi104 bash[9203]: cluster 2020-09-08T15:57:38.038260+0000 osd.7 (osd.7) 101 : cluster [DBG] 6.5 scrub starts 2020-09-08T15:57:39.497 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 71 16 98867 98851 87.0075 43.625 0.011951 0.0114903 2020-09-08T15:57:39.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:39 smithi055 bash[9296]: cluster 2020-09-08T15:57:37.915286+0000 mgr.x (mgr.24118) 909 : cluster [DBG] pgmap v1329: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s wr, 1.32k op/s 2020-09-08T15:57:39.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:39 smithi055 bash[9296]: cluster 2020-09-08T15:57:37.974453+0000 osd.3 (osd.3) 73 : cluster [DBG] 6.6 scrub starts 2020-09-08T15:57:39.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:39 smithi055 bash[9296]: cluster 2020-09-08T15:57:38.038260+0000 osd.7 (osd.7) 101 : cluster [DBG] 6.5 scrub starts 2020-09-08T15:57:40.058 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:40 smithi104 bash[9203]: cluster 2020-09-08T15:57:38.830030+0000 osd.0 (osd.0) 74 : cluster [DBG] 6.9 scrub ok 2020-09-08T15:57:40.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:40 smithi166 bash[9446]: cluster 2020-09-08T15:57:38.830030+0000 osd.0 (osd.0) 74 : cluster [DBG] 6.9 scrub ok 2020-09-08T15:57:40.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:40 smithi055 bash[9296]: cluster 2020-09-08T15:57:38.830030+0000 osd.0 (osd.0) 74 : cluster [DBG] 6.9 scrub ok 2020-09-08T15:57:40.497 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 72 15 99947 99932 86.7374 67.5625 0.036809 0.0115269 2020-09-08T15:57:41.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:41 smithi166 bash[9446]: cluster 2020-09-08T15:57:39.915769+0000 mgr.x (mgr.24118) 910 : cluster [DBG] pgmap v1330: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s wr, 1.28k op/s 2020-09-08T15:57:41.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:41 smithi166 bash[9446]: cluster 2020-09-08T15:57:40.457826+0000 osd.3 (osd.3) 74 : cluster [DBG] 6.6 scrub ok 2020-09-08T15:57:41.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:41 smithi166 bash[9446]: cluster 2020-09-08T15:57:40.547153+0000 osd.5 (osd.5) 74 : cluster [DBG] 6.d scrub ok 2020-09-08T15:57:41.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:41 smithi055 bash[9296]: cluster 2020-09-08T15:57:39.915769+0000 mgr.x (mgr.24118) 910 : cluster [DBG] pgmap v1330: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s wr, 1.28k op/s 2020-09-08T15:57:41.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:41 smithi055 bash[9296]: cluster 2020-09-08T15:57:40.457826+0000 osd.3 (osd.3) 74 : cluster [DBG] 6.6 scrub ok 2020-09-08T15:57:41.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:41 smithi055 bash[9296]: cluster 2020-09-08T15:57:40.547153+0000 osd.5 (osd.5) 74 : cluster [DBG] 6.d scrub ok 2020-09-08T15:57:41.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:41 smithi104 bash[9203]: cluster 2020-09-08T15:57:39.915769+0000 mgr.x (mgr.24118) 910 : cluster [DBG] pgmap v1330: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s wr, 1.28k op/s 2020-09-08T15:57:41.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:41 smithi104 bash[9203]: cluster 2020-09-08T15:57:40.457826+0000 osd.3 (osd.3) 74 : cluster [DBG] 6.6 scrub ok 2020-09-08T15:57:41.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:41 smithi104 bash[9203]: cluster 2020-09-08T15:57:40.547153+0000 osd.5 (osd.5) 74 : cluster [DBG] 6.d scrub ok 2020-09-08T15:57:41.498 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 73 15 101248 101233 86.6629 81.3125 0.0112778 0.0115366 2020-09-08T15:57:41.598 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:57:41.701 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:41.910 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:57:41 smithi055 bash[19003]: debug 2020-09-08T15:57:41.594+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:42.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:42 smithi166 bash[9446]: cluster 2020-09-08T15:57:40.312968+0000 osd.7 (osd.7) 102 : cluster [DBG] 6.5 scrub ok 2020-09-08T15:57:42.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:42 smithi055 bash[9296]: cluster 2020-09-08T15:57:40.312968+0000 osd.7 (osd.7) 102 : cluster [DBG] 6.5 scrub ok 2020-09-08T15:57:42.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:42 smithi104 bash[9203]: cluster 2020-09-08T15:57:40.312968+0000 osd.7 (osd.7) 102 : cluster [DBG] 6.5 scrub ok 2020-09-08T15:57:42.497 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 74 16 102128 102112 86.2341 54.9375 0.0053076 0.011593 2020-09-08T15:57:43.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:43 smithi104 bash[9203]: cluster 2020-09-08T15:57:41.916246+0000 mgr.x (mgr.24118) 911 : cluster [DBG] pgmap v1331: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T15:57:43.183 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:43 smithi104 bash[9203]: cluster 2020-09-08T15:57:42.301320+0000 osd.1 (osd.1) 100 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:43.183 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:43 smithi104 bash[9203]: cluster 2020-09-08T15:57:42.301410+0000 osd.1 (osd.1) 101 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:43.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:43 smithi166 bash[9446]: cluster 2020-09-08T15:57:41.916246+0000 mgr.x (mgr.24118) 911 : cluster [DBG] pgmap v1331: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T15:57:43.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:43 smithi166 bash[9446]: cluster 2020-09-08T15:57:42.301320+0000 osd.1 (osd.1) 100 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:43.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:43 smithi166 bash[9446]: cluster 2020-09-08T15:57:42.301410+0000 osd.1 (osd.1) 101 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:43.406 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:57:43.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:43 smithi055 bash[9296]: cluster 2020-09-08T15:57:41.916246+0000 mgr.x (mgr.24118) 911 : cluster [DBG] pgmap v1331: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T15:57:43.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:43 smithi055 bash[9296]: cluster 2020-09-08T15:57:42.301320+0000 osd.1 (osd.1) 100 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:43.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:43 smithi055 bash[9296]: cluster 2020-09-08T15:57:42.301410+0000 osd.1 (osd.1) 101 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:43.462 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:57:43.497 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 75 16 103271 103255 86.0368 71.4375 0.0113609 0.0116211 2020-09-08T15:57:43.518 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:57:43.563 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:57:43.616 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:57:43.656 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:57:43.702 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:57:43.749 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:57:43.783 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:57:43.816 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:57:43.847 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:57:43.890 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:57:43.894 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:57:43.894 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:57:43.895 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:57:43.895 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:57:43.895 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:57:43.895 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:57:43.895 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:57:43.896 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:57:43.896 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:57:43.896 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:57:43.896 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:57:43.896 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:57:43.904 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:57:43.905 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2020-09-08T15:57:43.905 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2020-09-08T15:57:43.905 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T15:57:43.905 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.906 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:112134 5.a 5:5325df92:::benchmark_data_smithi166_37820_object112133:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T15:57:43.906 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:52:53.414546+0000", 2020-09-08T15:57:43.906 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 290.49309072599999, 2020-09-08T15:57:43.906 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.224163854, 2020-09-08T15:57:43.906 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:43.907 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:57:43.907 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:43.907 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T15:57:43.907 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:57:43.907 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 112134 2020-09-08T15:57:43.908 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.908 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:43.908 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.908 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:43.908 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.414546+0000", 2020-09-08T15:57:43.909 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:43.909 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.909 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.909 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:43.909 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.414546+0000", 2020-09-08T15:57:43.910 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999642 2020-09-08T15:57:43.910 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.910 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.910 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:43.911 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.414511+0000", 2020-09-08T15:57:43.911 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000124683 2020-09-08T15:57:43.911 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.911 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.911 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:43.912 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.414635+0000", 2020-09-08T15:57:43.912 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3521e-05 2020-09-08T15:57:43.912 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.912 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.912 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:43.913 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.414649+0000", 2020-09-08T15:57:43.913 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0499999999999997e-06 2020-09-08T15:57:43.913 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.913 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.913 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:43.913 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.414655+0000", 2020-09-08T15:57:43.914 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0036279329999999999 2020-09-08T15:57:43.914 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.914 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.914 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:43.914 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.418283+0000", 2020-09-08T15:57:43.915 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000167138 2020-09-08T15:57:43.915 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.915 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.915 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:43.916 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.418450+0000", 2020-09-08T15:57:43.916 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000182041 2020-09-08T15:57:43.916 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.916 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.916 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8", 2020-09-08T15:57:43.916 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.418632+0000", 2020-09-08T15:57:43.917 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0095663669999999992 2020-09-08T15:57:43.917 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.917 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.917 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:57:43.918 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.428198+0000", 2020-09-08T15:57:43.918 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21040160399999999 2020-09-08T15:57:43.918 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.919 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.919 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:57:43.919 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.638600+0000", 2020-09-08T15:57:43.919 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5520999999999999e-05 2020-09-08T15:57:43.919 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.920 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.920 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:57:43.920 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.638655+0000", 2020-09-08T15:57:43.920 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4565999999999999e-05 2020-09-08T15:57:43.920 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.920 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.921 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:57:43.921 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.638710+0000", 2020-09-08T15:57:43.921 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.224163854 2020-09-08T15:57:43.922 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:43.922 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:43.922 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:43.922 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.922 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.922 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:112144 5.a 5:5a2d4869:::benchmark_data_smithi166_37820_object112143:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T15:57:43.923 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:52:53.424479+0000", 2020-09-08T15:57:43.923 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 290.48315827599998, 2020-09-08T15:57:43.923 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21966026, 2020-09-08T15:57:43.923 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:43.923 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:57:43.924 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:43.924 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T15:57:43.924 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:57:43.924 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 112144 2020-09-08T15:57:43.924 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.925 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:43.925 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.925 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:43.925 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.424479+0000", 2020-09-08T15:57:43.925 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:43.926 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.926 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.926 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:43.926 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.424479+0000", 2020-09-08T15:57:43.926 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999633 2020-09-08T15:57:43.927 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.927 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.927 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:43.927 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.424442+0000", 2020-09-08T15:57:43.927 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000139436 2020-09-08T15:57:43.928 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.928 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.928 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:43.928 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.424582+0000", 2020-09-08T15:57:43.928 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6667999999999999e-05 2020-09-08T15:57:43.928 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.929 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.929 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:43.929 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.424608+0000", 2020-09-08T15:57:43.929 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6779999999999997e-06 2020-09-08T15:57:43.929 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.930 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.930 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:43.930 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.424614+0000", 2020-09-08T15:57:43.930 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4540999999999998e-05 2020-09-08T15:57:43.930 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.931 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.931 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:43.931 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.424668+0000", 2020-09-08T15:57:43.932 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018118400000000001 2020-09-08T15:57:43.932 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.932 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.932 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:43.933 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.424850+0000", 2020-09-08T15:57:43.933 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019531299999999999 2020-09-08T15:57:43.933 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.933 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.933 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8", 2020-09-08T15:57:43.934 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.425045+0000", 2020-09-08T15:57:43.934 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0035487880000000002 2020-09-08T15:57:43.934 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.934 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.934 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:57:43.935 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.428594+0000", 2020-09-08T15:57:43.935 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21538117700000001 2020-09-08T15:57:43.935 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.935 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.935 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:57:43.936 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.643975+0000", 2020-09-08T15:57:43.936 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2937999999999997e-05 2020-09-08T15:57:43.936 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.936 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.936 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:57:43.936 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.644068+0000", 2020-09-08T15:57:43.937 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1098000000000003e-05 2020-09-08T15:57:43.937 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.937 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.937 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:57:43.937 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.644139+0000", 2020-09-08T15:57:43.938 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21966026 2020-09-08T15:57:43.938 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:43.938 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:43.938 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:43.938 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.939 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.939 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:112176 5.a 5:5572b007:::benchmark_data_smithi166_37820_object112175:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T15:57:43.939 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:52:53.450267+0000", 2020-09-08T15:57:43.939 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 290.457370416, 2020-09-08T15:57:43.939 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.19406781100000001, 2020-09-08T15:57:43.940 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:43.940 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:57:43.940 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:43.940 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T15:57:43.940 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:57:43.940 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 112176 2020-09-08T15:57:43.941 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.941 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:43.941 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.941 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:43.941 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.450267+0000", 2020-09-08T15:57:43.942 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:43.942 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.942 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.942 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:43.942 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.450267+0000", 2020-09-08T15:57:43.943 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999561 2020-09-08T15:57:43.943 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.943 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.943 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:43.943 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.450223+0000", 2020-09-08T15:57:43.944 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015672900000000001 2020-09-08T15:57:43.944 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.944 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.944 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:43.944 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.450379+0000", 2020-09-08T15:57:43.945 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4669e-05 2020-09-08T15:57:43.945 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.945 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.945 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:43.945 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.450404+0000", 2020-09-08T15:57:43.946 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9399999999999996e-06 2020-09-08T15:57:43.946 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.946 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.946 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:43.946 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.450411+0000", 2020-09-08T15:57:43.946 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5079e-05 2020-09-08T15:57:43.947 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.947 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.947 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:43.947 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.450456+0000", 2020-09-08T15:57:43.947 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000185568 2020-09-08T15:57:43.948 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.948 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.948 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:43.948 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.450642+0000", 2020-09-08T15:57:43.948 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013574099999999999 2020-09-08T15:57:43.949 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.949 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.949 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8", 2020-09-08T15:57:43.949 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.450777+0000", 2020-09-08T15:57:43.949 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0069434730000000004 2020-09-08T15:57:43.949 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.950 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.950 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:57:43.950 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.457721+0000", 2020-09-08T15:57:43.950 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.186493935 2020-09-08T15:57:43.950 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.951 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.951 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:57:43.951 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.644215+0000", 2020-09-08T15:57:43.951 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5506000000000006e-05 2020-09-08T15:57:43.951 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.952 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.952 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:57:43.952 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.644280+0000", 2020-09-08T15:57:43.952 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3943000000000002e-05 2020-09-08T15:57:43.952 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.952 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.953 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:57:43.953 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.644334+0000", 2020-09-08T15:57:43.953 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.19406781100000001 2020-09-08T15:57:43.953 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:43.953 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:43.954 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:43.954 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.954 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.954 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:112204 5.a 5:5803a901:::benchmark_data_smithi166_37820_object112203:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T15:57:43.954 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:52:53.471393+0000", 2020-09-08T15:57:43.955 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 290.43624342599998, 2020-09-08T15:57:43.955 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.173087886, 2020-09-08T15:57:43.955 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:43.956 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:57:43.957 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:43.957 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T15:57:43.957 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:57:43.957 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 112204 2020-09-08T15:57:43.957 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.958 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:43.958 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.958 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:43.958 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.471393+0000", 2020-09-08T15:57:43.958 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:43.959 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.959 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.959 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:43.959 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.471393+0000", 2020-09-08T15:57:43.959 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999638 2020-09-08T15:57:43.960 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.960 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.960 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:43.960 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.471357+0000", 2020-09-08T15:57:43.960 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000140624 2020-09-08T15:57:43.961 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.961 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.961 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:43.961 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.471498+0000", 2020-09-08T15:57:43.961 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8723e-05 2020-09-08T15:57:43.961 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.962 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.962 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:43.962 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.471516+0000", 2020-09-08T15:57:43.962 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8170000000000004e-06 2020-09-08T15:57:43.962 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.963 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.963 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:43.963 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.471522+0000", 2020-09-08T15:57:43.963 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2429999999999998e-05 2020-09-08T15:57:43.963 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.964 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.964 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:43.964 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.471575+0000", 2020-09-08T15:57:43.964 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000191879 2020-09-08T15:57:43.964 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.965 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.965 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:43.965 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.471767+0000", 2020-09-08T15:57:43.965 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020280100000000001 2020-09-08T15:57:43.966 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.966 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.966 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8", 2020-09-08T15:57:43.966 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.471969+0000", 2020-09-08T15:57:43.966 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0035255759999999999 2020-09-08T15:57:43.967 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.967 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.967 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:57:43.967 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.475495+0000", 2020-09-08T15:57:43.967 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16888451300000001 2020-09-08T15:57:43.968 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.968 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.968 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:57:43.968 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.644380+0000", 2020-09-08T15:57:43.968 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6224000000000002e-05 2020-09-08T15:57:43.969 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.969 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.969 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:57:43.969 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.644436+0000", 2020-09-08T15:57:43.969 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5657e-05 2020-09-08T15:57:43.970 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.970 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.970 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:57:43.970 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.644481+0000", 2020-09-08T15:57:43.970 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.173087886 2020-09-08T15:57:43.970 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:43.971 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:43.971 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:43.971 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.971 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.971 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:112211 5.a 5:5c187f72:::benchmark_data_smithi166_37820_object112210:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T15:57:43.972 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:52:53.478782+0000", 2020-09-08T15:57:43.972 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 290.42885495799999, 2020-09-08T15:57:43.972 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16599981, 2020-09-08T15:57:43.972 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:43.972 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:57:43.973 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:43.973 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T15:57:43.973 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:57:43.973 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 112211 2020-09-08T15:57:43.973 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.974 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:43.974 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.974 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:43.974 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.478782+0000", 2020-09-08T15:57:43.974 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:43.974 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.975 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.975 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:43.975 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.478782+0000", 2020-09-08T15:57:43.975 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999766 2020-09-08T15:57:43.975 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.976 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.976 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:43.976 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.478759+0000", 2020-09-08T15:57:43.976 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3548999999999996e-05 2020-09-08T15:57:43.976 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.977 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.977 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:43.977 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.478842+0000", 2020-09-08T15:57:43.977 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3512999999999999e-05 2020-09-08T15:57:43.977 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.978 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.978 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:43.978 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.478856+0000", 2020-09-08T15:57:43.978 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9779999999999999e-06 2020-09-08T15:57:43.978 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.978 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.979 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:43.979 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.478863+0000", 2020-09-08T15:57:43.979 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7024000000000002e-05 2020-09-08T15:57:43.979 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.979 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.980 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:43.980 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.478910+0000", 2020-09-08T15:57:43.980 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012508800000000001 2020-09-08T15:57:43.980 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.981 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.981 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:43.981 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.479035+0000", 2020-09-08T15:57:43.981 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00023760900000000001 2020-09-08T15:57:43.982 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.982 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.982 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8", 2020-09-08T15:57:43.982 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.479272+0000", 2020-09-08T15:57:43.982 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0041793940000000003 2020-09-08T15:57:43.982 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.983 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.983 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:57:43.983 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.483452+0000", 2020-09-08T15:57:43.983 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.161163367 2020-09-08T15:57:43.983 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.984 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.984 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:57:43.984 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.644615+0000", 2020-09-08T15:57:43.984 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3368999999999993e-05 2020-09-08T15:57:43.984 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.985 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.985 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:57:43.985 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.644708+0000", 2020-09-08T15:57:43.985 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3357999999999998e-05 2020-09-08T15:57:43.985 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.986 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.986 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:57:43.986 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.644782+0000", 2020-09-08T15:57:43.986 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16599981 2020-09-08T15:57:43.986 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:43.987 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:43.987 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:43.987 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.987 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.987 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:112240 5.a 5:5ee2d7d1:::benchmark_data_smithi166_37820_object112239:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T15:57:43.987 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:52:53.506412+0000", 2020-09-08T15:57:43.988 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 290.40122444999997, 2020-09-08T15:57:43.988 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14664049500000001, 2020-09-08T15:57:43.988 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:43.988 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:57:43.989 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:43.989 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T15:57:43.989 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:57:43.989 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 112240 2020-09-08T15:57:43.990 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.990 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:43.990 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.990 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:43.990 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.506412+0000", 2020-09-08T15:57:43.991 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:43.991 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.991 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.991 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:43.991 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.506412+0000", 2020-09-08T15:57:43.991 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999738 2020-09-08T15:57:43.992 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.992 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.992 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:43.992 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.506386+0000", 2020-09-08T15:57:43.992 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010602 2020-09-08T15:57:43.993 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.993 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.993 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:43.993 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.506492+0000", 2020-09-08T15:57:43.993 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3308e-05 2020-09-08T15:57:43.994 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.994 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.994 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:43.994 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.506505+0000", 2020-09-08T15:57:43.994 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.8359999999999992e-06 2020-09-08T15:57:43.995 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.995 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.995 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:43.995 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.506513+0000", 2020-09-08T15:57:43.996 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0081000000000003e-05 2020-09-08T15:57:43.997 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.997 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.997 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:43.997 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.506573+0000", 2020-09-08T15:57:43.997 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00023053200000000001 2020-09-08T15:57:43.998 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.998 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.998 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:43.998 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.506804+0000", 2020-09-08T15:57:43.998 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00021310200000000001 2020-09-08T15:57:43.999 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:43.999 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:43.999 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8", 2020-09-08T15:57:43.999 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.507017+0000", 2020-09-08T15:57:43.999 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.016118067 2020-09-08T15:57:44.000 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.000 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.000 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:57:44.000 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.523135+0000", 2020-09-08T15:57:44.000 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.12974059299999999 2020-09-08T15:57:44.000 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.001 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.001 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:57:44.001 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.652876+0000", 2020-09-08T15:57:44.001 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010265899999999999 2020-09-08T15:57:44.001 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.002 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.002 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:57:44.002 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.652978+0000", 2020-09-08T15:57:44.002 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4740999999999995e-05 2020-09-08T15:57:44.002 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.003 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.003 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:57:44.003 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.653053+0000", 2020-09-08T15:57:44.003 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14664049500000001 2020-09-08T15:57:44.003 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.004 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:44.004 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.004 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.004 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.004 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:112241 5.a 5:5e06626f:::benchmark_data_smithi166_37820_object112240:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T15:57:44.004 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:52:53.506596+0000", 2020-09-08T15:57:44.005 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 290.40104120900003, 2020-09-08T15:57:44.005 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14842851700000001, 2020-09-08T15:57:44.005 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:44.005 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:57:44.005 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:44.006 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T15:57:44.006 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:57:44.006 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 112241 2020-09-08T15:57:44.006 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.006 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:44.007 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.007 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:44.007 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.506596+0000", 2020-09-08T15:57:44.007 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:44.007 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.008 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.008 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:44.008 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.506596+0000", 2020-09-08T15:57:44.008 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999958 2020-09-08T15:57:44.008 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.009 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.009 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:44.009 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.506554+0000", 2020-09-08T15:57:44.009 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019701899999999999 2020-09-08T15:57:44.009 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.010 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.010 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:44.010 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.506751+0000", 2020-09-08T15:57:44.010 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1248e-05 2020-09-08T15:57:44.010 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.011 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.011 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:44.011 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.506762+0000", 2020-09-08T15:57:44.011 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9439999999999996e-06 2020-09-08T15:57:44.012 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.012 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.012 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:44.012 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.506767+0000", 2020-09-08T15:57:44.013 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0044323219999999998 2020-09-08T15:57:44.013 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.013 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.013 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:44.013 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.511199+0000", 2020-09-08T15:57:44.013 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00022384600000000001 2020-09-08T15:57:44.014 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.014 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.014 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:44.014 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.511423+0000", 2020-09-08T15:57:44.014 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000219874 2020-09-08T15:57:44.015 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.015 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.015 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8", 2020-09-08T15:57:44.015 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.511643+0000", 2020-09-08T15:57:44.015 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.011505259 2020-09-08T15:57:44.016 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.016 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.016 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:57:44.016 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.523148+0000", 2020-09-08T15:57:44.016 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13165874399999999 2020-09-08T15:57:44.016 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.017 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.017 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:57:44.017 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.654807+0000", 2020-09-08T15:57:44.017 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011819699999999999 2020-09-08T15:57:44.018 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.018 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.018 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:57:44.018 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.654925+0000", 2020-09-08T15:57:44.018 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8973000000000003e-05 2020-09-08T15:57:44.019 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.019 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.019 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:57:44.019 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.655024+0000", 2020-09-08T15:57:44.020 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14842851700000001 2020-09-08T15:57:44.020 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.020 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:44.020 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.020 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.021 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.021 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:112242 5.a 5:5e3627e1:::benchmark_data_smithi166_37820_object112241:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T15:57:44.021 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:52:53.506822+0000", 2020-09-08T15:57:44.021 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 290.40081506199999, 2020-09-08T15:57:44.021 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.148460025, 2020-09-08T15:57:44.022 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:44.022 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:57:44.022 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:44.022 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T15:57:44.022 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:57:44.023 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 112242 2020-09-08T15:57:44.023 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.023 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:44.023 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.023 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:44.024 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.506822+0000", 2020-09-08T15:57:44.024 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:44.024 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.024 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.024 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:44.024 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.506822+0000", 2020-09-08T15:57:44.025 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999738 2020-09-08T15:57:44.025 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.025 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.025 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:44.025 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.506796+0000", 2020-09-08T15:57:44.026 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012886000000000001 2020-09-08T15:57:44.026 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.026 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.026 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:44.026 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.506925+0000", 2020-09-08T15:57:44.027 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5279999999999999e-05 2020-09-08T15:57:44.027 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.027 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.027 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:44.027 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.506940+0000", 2020-09-08T15:57:44.028 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.037e-06 2020-09-08T15:57:44.028 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.028 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.028 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:44.028 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.506946+0000", 2020-09-08T15:57:44.028 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0085777230000000006 2020-09-08T15:57:44.029 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.029 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.029 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:44.029 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.515524+0000", 2020-09-08T15:57:44.029 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000190946 2020-09-08T15:57:44.030 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.030 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.030 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:44.030 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.515714+0000", 2020-09-08T15:57:44.030 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020444300000000001 2020-09-08T15:57:44.031 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.031 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.031 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8", 2020-09-08T15:57:44.031 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.515919+0000", 2020-09-08T15:57:44.031 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0072392580000000002 2020-09-08T15:57:44.032 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.032 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.032 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:57:44.032 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.523158+0000", 2020-09-08T15:57:44.033 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13193813600000001 2020-09-08T15:57:44.034 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.034 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.035 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:57:44.035 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.655096+0000", 2020-09-08T15:57:44.035 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010037 2020-09-08T15:57:44.036 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.036 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.036 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:57:44.037 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.655197+0000", 2020-09-08T15:57:44.037 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5190000000000003e-05 2020-09-08T15:57:44.037 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.038 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.038 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:57:44.038 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.655282+0000", 2020-09-08T15:57:44.039 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.148460025 2020-09-08T15:57:44.039 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.039 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:44.040 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.040 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.040 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.041 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:112250 5.a 5:55313165:::benchmark_data_smithi166_37820_object112249:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T15:57:44.041 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:52:53.516673+0000", 2020-09-08T15:57:44.042 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 290.390964134, 2020-09-08T15:57:44.042 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14021298500000001, 2020-09-08T15:57:44.042 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:44.043 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:57:44.043 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:44.043 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T15:57:44.044 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:57:44.044 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 112250 2020-09-08T15:57:44.044 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.045 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:44.045 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.045 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:44.046 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.516673+0000", 2020-09-08T15:57:44.046 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:44.046 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.047 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.047 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:44.047 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.516673+0000", 2020-09-08T15:57:44.048 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999723 2020-09-08T15:57:44.048 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.048 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.049 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:44.049 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.516645+0000", 2020-09-08T15:57:44.049 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000162652 2020-09-08T15:57:44.050 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.050 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.050 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:44.051 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.516808+0000", 2020-09-08T15:57:44.051 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4417999999999999e-05 2020-09-08T15:57:44.051 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.052 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.052 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:44.052 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.516832+0000", 2020-09-08T15:57:44.053 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9079999999999998e-06 2020-09-08T15:57:44.053 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.053 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.054 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:44.054 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.516839+0000", 2020-09-08T15:57:44.054 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0028430959999999998 2020-09-08T15:57:44.055 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.055 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.055 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:44.056 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.519682+0000", 2020-09-08T15:57:44.056 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018684800000000001 2020-09-08T15:57:44.056 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.057 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.057 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:44.058 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.519869+0000", 2020-09-08T15:57:44.058 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000228731 2020-09-08T15:57:44.058 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.059 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.059 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8", 2020-09-08T15:57:44.059 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.520098+0000", 2020-09-08T15:57:44.060 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0032144579999999999 2020-09-08T15:57:44.060 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.060 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.061 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:57:44.068 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.523312+0000", 2020-09-08T15:57:44.068 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13345491600000001 2020-09-08T15:57:44.068 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.069 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.069 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:57:44.069 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.656767+0000", 2020-09-08T15:57:44.069 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0819000000000001e-05 2020-09-08T15:57:44.069 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.070 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.070 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:57:44.070 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.656838+0000", 2020-09-08T15:57:44.070 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8004999999999999e-05 2020-09-08T15:57:44.070 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.071 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.071 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:57:44.071 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.656886+0000", 2020-09-08T15:57:44.071 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14021298500000001 2020-09-08T15:57:44.071 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.072 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:44.072 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.072 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.072 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.072 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:112260 5.a 5:5e806c69:::benchmark_data_smithi166_37820_object112259:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T15:57:44.073 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:52:53.526193+0000", 2020-09-08T15:57:44.073 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 290.38144343300002, 2020-09-08T15:57:44.073 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13452776, 2020-09-08T15:57:44.073 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:44.073 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:57:44.074 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:44.074 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T15:57:44.074 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:57:44.075 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 112260 2020-09-08T15:57:44.075 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.075 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:44.075 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.075 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:44.076 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.526193+0000", 2020-09-08T15:57:44.076 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:44.076 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.076 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.076 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:44.077 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.526193+0000", 2020-09-08T15:57:44.077 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999657 2020-09-08T15:57:44.077 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.077 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.077 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:44.078 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.526159+0000", 2020-09-08T15:57:44.078 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012470499999999999 2020-09-08T15:57:44.078 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.078 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.078 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:44.079 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.526284+0000", 2020-09-08T15:57:44.079 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5968999999999999e-05 2020-09-08T15:57:44.079 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.079 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.079 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:44.080 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.526300+0000", 2020-09-08T15:57:44.080 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0799999999999996e-06 2020-09-08T15:57:44.080 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.080 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.080 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:44.081 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.526305+0000", 2020-09-08T15:57:44.081 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9008000000000001e-05 2020-09-08T15:57:44.081 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.081 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.082 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:44.082 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.526374+0000", 2020-09-08T15:57:44.082 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017605999999999999 2020-09-08T15:57:44.082 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.082 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.083 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:44.083 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.526550+0000", 2020-09-08T15:57:44.083 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012493500000000001 2020-09-08T15:57:44.083 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.083 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.083 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8", 2020-09-08T15:57:44.084 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.526675+0000", 2020-09-08T15:57:44.084 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0035878529999999998 2020-09-08T15:57:44.085 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.085 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.085 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:57:44.085 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.530263+0000", 2020-09-08T15:57:44.085 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13030620200000001 2020-09-08T15:57:44.086 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.086 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.086 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:57:44.086 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.660569+0000", 2020-09-08T15:57:44.086 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9058000000000001e-05 2020-09-08T15:57:44.087 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.087 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.087 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:57:44.087 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.660648+0000", 2020-09-08T15:57:44.087 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3269000000000006e-05 2020-09-08T15:57:44.088 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.088 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.088 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:57:44.088 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:53.660721+0000", 2020-09-08T15:57:44.088 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13452776 2020-09-08T15:57:44.089 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.089 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:44.089 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.089 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.089 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.090 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(osd.10.0:41059 5.4 e437/396 5:24dba43e:::benchmark_data_smithi166_37820_object27152:head v 437'12672, mlcod=437'12656)", 2020-09-08T15:57:44.090 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:53:49.794636+0000", 2020-09-08T15:57:44.090 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 234.11300047899999, 2020-09-08T15:57:44.090 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.12526306400000001, 2020-09-08T15:57:44.090 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:44.091 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:57:44.091 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:44.091 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.091 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:44.091 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:53:49.794636+0000", 2020-09-08T15:57:44.091 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:44.092 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.092 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.092 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:44.092 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:53:49.794636+0000", 2020-09-08T15:57:44.092 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9455000000000003e-05 2020-09-08T15:57:44.093 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.093 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.093 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:44.093 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:53:49.794726+0000", 2020-09-08T15:57:44.093 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0747999999999999e-05 2020-09-08T15:57:44.094 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.094 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.094 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:44.094 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:53:49.794737+0000", 2020-09-08T15:57:44.094 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7560000000000001e-05 2020-09-08T15:57:44.095 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.095 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.095 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:44.096 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:53:49.794754+0000", 2020-09-08T15:57:44.096 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.25e-06 2020-09-08T15:57:44.096 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.096 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.096 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:44.097 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:53:49.794761+0000", 2020-09-08T15:57:44.097 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.062712748999999998 2020-09-08T15:57:44.097 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.097 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.097 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:44.098 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:53:49.857474+0000", 2020-09-08T15:57:44.098 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0854e-05 2020-09-08T15:57:44.098 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.098 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.098 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:44.099 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:53:49.857505+0000", 2020-09-08T15:57:44.099 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.062304981000000002 2020-09-08T15:57:44.099 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.099 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.099 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:57:44.100 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:53:49.919810+0000", 2020-09-08T15:57:44.100 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9467000000000002e-05 2020-09-08T15:57:44.100 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.100 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.100 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:57:44.100 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:53:49.919900+0000", 2020-09-08T15:57:44.101 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.12526306400000001 2020-09-08T15:57:44.101 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.101 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:44.101 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.102 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.102 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.102 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:255637 5.a 5:54b2a826:::benchmark_data_smithi166_37820_object111215:head [delete] snapc 0=[] ondisk+write+known_if_redirected e463)", 2020-09-08T15:57:44.102 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:55:23.364954+0000", 2020-09-08T15:57:44.102 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 140.54268340300001, 2020-09-08T15:57:44.103 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.160508875, 2020-09-08T15:57:44.103 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:44.103 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:57:44.103 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:44.103 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T15:57:44.104 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:57:44.104 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 255637 2020-09-08T15:57:44.104 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.104 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:44.104 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.104 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:44.105 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.364954+0000", 2020-09-08T15:57:44.105 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:44.105 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.105 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.105 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:44.106 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.364954+0000", 2020-09-08T15:57:44.106 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999576 2020-09-08T15:57:44.106 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.106 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.106 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:44.107 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.364911+0000", 2020-09-08T15:57:44.107 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000101686 2020-09-08T15:57:44.107 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.107 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.107 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:44.108 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.365013+0000", 2020-09-08T15:57:44.108 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3123e-05 2020-09-08T15:57:44.108 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.108 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.108 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:44.108 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.365026+0000", 2020-09-08T15:57:44.109 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4699999999999999e-06 2020-09-08T15:57:44.109 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.109 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.109 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:44.109 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.365032+0000", 2020-09-08T15:57:44.110 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0043377240000000003 2020-09-08T15:57:44.110 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.110 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.110 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:44.110 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.369370+0000", 2020-09-08T15:57:44.111 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015232399999999999 2020-09-08T15:57:44.111 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.111 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.111 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:44.111 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.369522+0000", 2020-09-08T15:57:44.112 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2441000000000006e-05 2020-09-08T15:57:44.112 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.112 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.112 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8", 2020-09-08T15:57:44.112 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.369605+0000", 2020-09-08T15:57:44.112 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0048127459999999997 2020-09-08T15:57:44.113 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.113 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.113 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:57:44.113 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.374418+0000", 2020-09-08T15:57:44.113 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0046989629999999996 2020-09-08T15:57:44.114 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.114 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.114 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:57:44.114 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.379117+0000", 2020-09-08T15:57:44.114 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3522000000000002e-05 2020-09-08T15:57:44.115 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.115 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.115 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:57:44.115 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.379150+0000", 2020-09-08T15:57:44.115 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14631229100000001 2020-09-08T15:57:44.116 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.116 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.116 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:57:44.116 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.525462+0000", 2020-09-08T15:57:44.116 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.160508875 2020-09-08T15:57:44.117 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.117 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:44.117 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.117 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.117 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.118 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:255643 5.a 5:5b8f269e:::benchmark_data_smithi166_37820_object111221:head [delete] snapc 0=[] ondisk+write+known_if_redirected e463)", 2020-09-08T15:57:44.118 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:55:23.370555+0000", 2020-09-08T15:57:44.118 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 140.537081769, 2020-09-08T15:57:44.118 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18176403399999999, 2020-09-08T15:57:44.118 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:44.119 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:57:44.119 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:44.119 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T15:57:44.119 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:57:44.119 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 255643 2020-09-08T15:57:44.119 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.120 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:44.120 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.120 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:44.120 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.370555+0000", 2020-09-08T15:57:44.120 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:44.121 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.121 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.121 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:44.121 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.370555+0000", 2020-09-08T15:57:44.121 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999638 2020-09-08T15:57:44.122 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.122 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.122 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:44.122 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.370519+0000", 2020-09-08T15:57:44.122 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1823000000000006e-05 2020-09-08T15:57:44.123 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.123 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.123 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:44.123 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.370601+0000", 2020-09-08T15:57:44.123 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3467e-05 2020-09-08T15:57:44.123 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.124 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.124 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:44.124 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.370614+0000", 2020-09-08T15:57:44.124 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6960000000000002e-06 2020-09-08T15:57:44.124 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.125 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.125 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:44.125 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.370620+0000", 2020-09-08T15:57:44.125 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0038412799999999999 2020-09-08T15:57:44.125 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.126 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.126 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:44.126 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.374461+0000", 2020-09-08T15:57:44.126 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7646999999999996e-05 2020-09-08T15:57:44.126 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.126 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.127 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:44.127 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.374549+0000", 2020-09-08T15:57:44.127 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0659000000000001e-05 2020-09-08T15:57:44.127 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.127 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.128 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8", 2020-09-08T15:57:44.128 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.374640+0000", 2020-09-08T15:57:44.128 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16388561500000001 2020-09-08T15:57:44.128 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.128 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.129 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:57:44.129 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.538525+0000", 2020-09-08T15:57:44.129 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013680952 2020-09-08T15:57:44.129 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.129 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.130 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:57:44.130 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.552206+0000", 2020-09-08T15:57:44.130 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3245999999999997e-05 2020-09-08T15:57:44.130 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.130 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.130 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:57:44.131 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.552269+0000", 2020-09-08T15:57:44.131 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9752000000000001e-05 2020-09-08T15:57:44.131 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.131 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.131 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:57:44.132 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.552319+0000", 2020-09-08T15:57:44.132 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18176403399999999 2020-09-08T15:57:44.132 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.132 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:44.132 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.133 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.133 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.133 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:255649 5.a 5:5ada3016:::benchmark_data_smithi166_37820_object111227:head [delete] snapc 0=[] ondisk+write+known_if_redirected e463)", 2020-09-08T15:57:44.133 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:55:23.373740+0000", 2020-09-08T15:57:44.133 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 140.53389734500001, 2020-09-08T15:57:44.134 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17867983300000001, 2020-09-08T15:57:44.134 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:44.134 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:57:44.134 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:44.134 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T15:57:44.134 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:57:44.135 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 255649 2020-09-08T15:57:44.135 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.135 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:44.135 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.135 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:44.136 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.373740+0000", 2020-09-08T15:57:44.136 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:44.136 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.136 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.136 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:44.137 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.373740+0000", 2020-09-08T15:57:44.137 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999642 2020-09-08T15:57:44.137 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.137 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.137 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:44.138 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.373704+0000", 2020-09-08T15:57:44.138 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.8973000000000004e-05 2020-09-08T15:57:44.138 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.138 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.138 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:44.138 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.373783+0000", 2020-09-08T15:57:44.139 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2984e-05 2020-09-08T15:57:44.139 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.139 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.139 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:44.139 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.373796+0000", 2020-09-08T15:57:44.140 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.631e-06 2020-09-08T15:57:44.140 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.140 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.140 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:44.140 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.373801+0000", 2020-09-08T15:57:44.141 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.151780051 2020-09-08T15:57:44.141 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.141 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.141 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:44.141 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.525581+0000", 2020-09-08T15:57:44.141 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00034587099999999998 2020-09-08T15:57:44.142 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.142 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.142 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:44.142 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.525927+0000", 2020-09-08T15:57:44.142 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00040827300000000001 2020-09-08T15:57:44.143 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.143 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.143 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8", 2020-09-08T15:57:44.143 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.526335+0000", 2020-09-08T15:57:44.143 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.025853594000000001 2020-09-08T15:57:44.144 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.144 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.144 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:57:44.144 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.552189+0000", 2020-09-08T15:57:44.144 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014506 2020-09-08T15:57:44.144 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.145 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.145 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:57:44.145 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.552334+0000", 2020-09-08T15:57:44.145 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5318999999999998e-05 2020-09-08T15:57:44.145 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.146 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.146 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:57:44.146 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.552379+0000", 2020-09-08T15:57:44.146 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9956999999999997e-05 2020-09-08T15:57:44.146 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.147 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.147 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:57:44.147 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.552419+0000", 2020-09-08T15:57:44.147 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17867983300000001 2020-09-08T15:57:44.147 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.148 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:44.148 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.148 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.148 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.148 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:255650 5.6 5:6d9f14ed:::benchmark_data_smithi166_37820_object111228:head [delete] snapc 0=[] ondisk+write+known_if_redirected e463)", 2020-09-08T15:57:44.148 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:55:23.374623+0000", 2020-09-08T15:57:44.149 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 140.533013858, 2020-09-08T15:57:44.149 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16450193699999999, 2020-09-08T15:57:44.149 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:44.149 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:57:44.149 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:44.151 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T15:57:44.151 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:57:44.151 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 255650 2020-09-08T15:57:44.151 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.152 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:44.152 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.152 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:44.152 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.374623+0000", 2020-09-08T15:57:44.153 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:44.153 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.153 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.153 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:44.153 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.374623+0000", 2020-09-08T15:57:44.154 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999633 2020-09-08T15:57:44.154 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.154 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.154 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:44.155 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.374587+0000", 2020-09-08T15:57:44.155 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9536999999999994e-05 2020-09-08T15:57:44.155 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.155 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.155 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:44.156 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.374666+0000", 2020-09-08T15:57:44.156 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2656999999999999e-05 2020-09-08T15:57:44.156 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.156 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.156 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:44.156 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.374679+0000", 2020-09-08T15:57:44.157 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.643e-06 2020-09-08T15:57:44.157 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.157 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.157 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:44.158 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.374684+0000", 2020-09-08T15:57:44.158 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0475999999999997e-05 2020-09-08T15:57:44.158 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.158 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.158 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:44.158 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.374725+0000", 2020-09-08T15:57:44.159 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000136964 2020-09-08T15:57:44.159 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.159 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.159 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:44.159 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.374862+0000", 2020-09-08T15:57:44.160 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000143115 2020-09-08T15:57:44.160 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.160 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.160 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 5", 2020-09-08T15:57:44.160 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.375005+0000", 2020-09-08T15:57:44.161 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.163981342 2020-09-08T15:57:44.161 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.161 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.161 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:57:44.161 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.538986+0000", 2020-09-08T15:57:44.162 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9451999999999999e-05 2020-09-08T15:57:44.162 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.162 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.162 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:57:44.162 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.539006+0000", 2020-09-08T15:57:44.163 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6985999999999999e-05 2020-09-08T15:57:44.163 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.163 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.163 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:57:44.163 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.539073+0000", 2020-09-08T15:57:44.164 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2327999999999998e-05 2020-09-08T15:57:44.164 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.164 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.164 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:57:44.164 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.539125+0000", 2020-09-08T15:57:44.165 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16450193699999999 2020-09-08T15:57:44.165 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.165 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:44.165 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.165 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.166 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.166 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop_reply(client.15771.0:255643 5.a e463/396 ondisk, result = 0)", 2020-09-08T15:57:44.166 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:55:23.378809+0000", 2020-09-08T15:57:44.166 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 140.528828229, 2020-09-08T15:57:44.166 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15972038899999999, 2020-09-08T15:57:44.167 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:44.167 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2020-09-08T15:57:44.167 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:44.167 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.167 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:44.168 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.378809+0000", 2020-09-08T15:57:44.168 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:44.168 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.168 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.168 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:44.169 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.378809+0000", 2020-09-08T15:57:44.169 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9314999999999998e-05 2020-09-08T15:57:44.169 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.170 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.170 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:44.170 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.378858+0000", 2020-09-08T15:57:44.170 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4160000000000004e-06 2020-09-08T15:57:44.171 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.171 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.171 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:44.171 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.378866+0000", 2020-09-08T15:57:44.171 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4977000000000001e-05 2020-09-08T15:57:44.172 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.172 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.172 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:44.172 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.378881+0000", 2020-09-08T15:57:44.172 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4559999999999997e-06 2020-09-08T15:57:44.173 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.173 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.173 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:44.173 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.378888+0000", 2020-09-08T15:57:44.173 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15961131200000001 2020-09-08T15:57:44.174 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.174 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.174 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:44.174 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.538499+0000", 2020-09-08T15:57:44.174 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2415999999999999e-05 2020-09-08T15:57:44.175 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.175 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.175 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:44.175 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.538512+0000", 2020-09-08T15:57:44.175 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7496999999999999e-05 2020-09-08T15:57:44.176 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.176 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.176 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:57:44.176 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.538529+0000", 2020-09-08T15:57:44.176 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15972038899999999 2020-09-08T15:57:44.176 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.177 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:44.177 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.177 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.177 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.178 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:255652 5.a 5:5f066362:::benchmark_data_smithi166_37820_object111230:head [delete] snapc 0=[] ondisk+write+known_if_redirected e463)", 2020-09-08T15:57:44.178 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:55:23.378999+0000", 2020-09-08T15:57:44.178 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 140.52863804399999, 2020-09-08T15:57:44.178 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.187265021, 2020-09-08T15:57:44.178 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:44.178 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:57:44.179 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:44.179 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T15:57:44.179 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:57:44.179 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 255652 2020-09-08T15:57:44.179 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.180 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:44.180 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.180 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:44.180 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.378999+0000", 2020-09-08T15:57:44.180 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:44.181 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.181 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.181 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:44.181 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.378999+0000", 2020-09-08T15:57:44.181 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999752 2020-09-08T15:57:44.182 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.182 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.182 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:44.182 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.378974+0000", 2020-09-08T15:57:44.182 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5821e-05 2020-09-08T15:57:44.183 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.183 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.183 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:44.183 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.379030+0000", 2020-09-08T15:57:44.183 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7030000000000001e-06 2020-09-08T15:57:44.184 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.184 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.184 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:44.184 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.379040+0000", 2020-09-08T15:57:44.184 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0180000000000001e-06 2020-09-08T15:57:44.185 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.185 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.185 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:44.185 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.379045+0000", 2020-09-08T15:57:44.185 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.159639854 2020-09-08T15:57:44.185 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.186 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.186 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:44.186 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.538684+0000", 2020-09-08T15:57:44.186 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020697900000000001 2020-09-08T15:57:44.186 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.187 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.187 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:44.187 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.538891+0000", 2020-09-08T15:57:44.187 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00048839800000000002 2020-09-08T15:57:44.187 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.188 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.188 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8", 2020-09-08T15:57:44.188 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.539380+0000", 2020-09-08T15:57:44.188 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013046952000000001 2020-09-08T15:57:44.188 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.189 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.189 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:57:44.189 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.552427+0000", 2020-09-08T15:57:44.189 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013643251 2020-09-08T15:57:44.189 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.190 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.190 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:57:44.190 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.566070+0000", 2020-09-08T15:57:44.190 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000146417 2020-09-08T15:57:44.190 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.191 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.191 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:57:44.191 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.566216+0000", 2020-09-08T15:57:44.191 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7457999999999999e-05 2020-09-08T15:57:44.191 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.192 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.192 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:57:44.192 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:55:23.566264+0000", 2020-09-08T15:57:44.192 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.187265021 2020-09-08T15:57:44.192 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.193 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:44.193 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.193 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.193 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.193 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:58447 6.f 6:fb23ca51:::benchmark_data_smithi166_43187_object58446:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e496)", 2020-09-08T15:57:44.194 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:09.350794+0000", 2020-09-08T15:57:44.194 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 34.556842840000002, 2020-09-08T15:57:44.194 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.192354574, 2020-09-08T15:57:44.194 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:44.194 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:57:44.195 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:44.195 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:57:44.195 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:57:44.195 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 58447 2020-09-08T15:57:44.196 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.196 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:44.196 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.196 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:44.196 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.350794+0000", 2020-09-08T15:57:44.197 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:44.197 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.199 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.199 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:44.199 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.350794+0000", 2020-09-08T15:57:44.199 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999633 2020-09-08T15:57:44.200 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.200 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.200 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:44.200 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.350757+0000", 2020-09-08T15:57:44.200 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014111500000000001 2020-09-08T15:57:44.201 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.201 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.201 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:44.201 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.350899+0000", 2020-09-08T15:57:44.201 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4800000000000001e-05 2020-09-08T15:57:44.202 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.202 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.202 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:44.203 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.350913+0000", 2020-09-08T15:57:44.203 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6100000000000002e-06 2020-09-08T15:57:44.203 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.203 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.203 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:44.204 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.350920+0000", 2020-09-08T15:57:44.204 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7420000000000003e-05 2020-09-08T15:57:44.204 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.204 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.204 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:44.205 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.350967+0000", 2020-09-08T15:57:44.205 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010871599999999999 2020-09-08T15:57:44.205 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.205 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.205 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:44.206 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.351076+0000", 2020-09-08T15:57:44.206 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012915800000000001 2020-09-08T15:57:44.206 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.206 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.206 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 10", 2020-09-08T15:57:44.207 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.351205+0000", 2020-09-08T15:57:44.207 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002054302 2020-09-08T15:57:44.207 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.207 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.207 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:57:44.207 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.353260+0000", 2020-09-08T15:57:44.208 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18975368100000001 2020-09-08T15:57:44.208 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.208 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.208 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:57:44.208 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.543013+0000", 2020-09-08T15:57:44.209 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.8219000000000004e-05 2020-09-08T15:57:44.209 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.209 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.209 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:57:44.209 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.543091+0000", 2020-09-08T15:57:44.210 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7197000000000001e-05 2020-09-08T15:57:44.210 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.210 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.210 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:57:44.211 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.543149+0000", 2020-09-08T15:57:44.211 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.192354574 2020-09-08T15:57:44.211 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.211 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:44.211 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.212 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.212 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.212 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:58451 6.f 6:ffbae624:::benchmark_data_smithi166_43187_object58450:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e496)", 2020-09-08T15:57:44.212 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:09.353636+0000", 2020-09-08T15:57:44.213 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 34.554000688999999, 2020-09-08T15:57:44.213 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18974203100000001, 2020-09-08T15:57:44.213 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:44.213 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:57:44.213 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:44.214 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:57:44.214 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:57:44.214 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 58451 2020-09-08T15:57:44.214 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.215 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:44.215 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.216 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:44.216 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.353636+0000", 2020-09-08T15:57:44.216 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:44.216 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.216 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.217 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:44.217 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.353636+0000", 2020-09-08T15:57:44.217 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999557 2020-09-08T15:57:44.217 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.217 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.218 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:44.218 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.353592+0000", 2020-09-08T15:57:44.218 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015739500000000001 2020-09-08T15:57:44.218 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.219 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.219 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:44.219 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.353749+0000", 2020-09-08T15:57:44.219 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5225000000000001e-05 2020-09-08T15:57:44.219 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.220 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.220 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:44.220 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.353765+0000", 2020-09-08T15:57:44.220 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4230000000000002e-06 2020-09-08T15:57:44.220 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.221 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.221 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:44.221 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.353771+0000", 2020-09-08T15:57:44.221 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5271000000000002e-05 2020-09-08T15:57:44.222 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.222 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.222 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:44.222 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.353806+0000", 2020-09-08T15:57:44.223 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012520000000000001 2020-09-08T15:57:44.223 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.223 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.223 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:44.223 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.353932+0000", 2020-09-08T15:57:44.224 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016672099999999999 2020-09-08T15:57:44.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.224 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.224 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 10", 2020-09-08T15:57:44.224 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.354098+0000", 2020-09-08T15:57:44.225 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0019882530000000002 2020-09-08T15:57:44.225 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.225 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.225 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:57:44.227 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.356087+0000", 2020-09-08T15:57:44.227 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18716497800000001 2020-09-08T15:57:44.227 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.227 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.228 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:57:44.228 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.543252+0000", 2020-09-08T15:57:44.228 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6455e-05 2020-09-08T15:57:44.228 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.228 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.229 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:57:44.229 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.543318+0000", 2020-09-08T15:57:44.229 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0304999999999999e-05 2020-09-08T15:57:44.229 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.229 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.230 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:57:44.230 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.543378+0000", 2020-09-08T15:57:44.230 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18974203100000001 2020-09-08T15:57:44.230 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.231 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:44.231 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.231 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.231 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.231 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:58456 6.f 6:fab2ebd3:::benchmark_data_smithi166_43187_object58455:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e496)", 2020-09-08T15:57:44.232 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:09.356598+0000", 2020-09-08T15:57:44.232 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 34.551038433000002, 2020-09-08T15:57:44.232 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.19181124199999999, 2020-09-08T15:57:44.232 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:44.232 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:57:44.233 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:44.233 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:57:44.233 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:57:44.233 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 58456 2020-09-08T15:57:44.233 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.234 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:44.234 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.234 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:44.234 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.356598+0000", 2020-09-08T15:57:44.235 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:44.235 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.235 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.235 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:44.235 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.356598+0000", 2020-09-08T15:57:44.236 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999595 2020-09-08T15:57:44.236 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.236 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.236 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:44.236 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.356558+0000", 2020-09-08T15:57:44.237 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000134084 2020-09-08T15:57:44.237 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.237 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.237 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:44.237 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.356692+0000", 2020-09-08T15:57:44.238 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3723999999999999e-05 2020-09-08T15:57:44.238 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.238 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.238 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:44.238 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.356706+0000", 2020-09-08T15:57:44.239 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5830000000000003e-06 2020-09-08T15:57:44.239 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.239 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.239 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:44.239 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.356712+0000", 2020-09-08T15:57:44.240 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9480000000000001e-05 2020-09-08T15:57:44.240 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.240 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.240 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:44.240 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.356762+0000", 2020-09-08T15:57:44.241 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000127494 2020-09-08T15:57:44.241 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.241 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.241 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:44.242 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.356889+0000", 2020-09-08T15:57:44.242 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000153502 2020-09-08T15:57:44.242 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.242 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.243 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 10", 2020-09-08T15:57:44.243 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.357043+0000", 2020-09-08T15:57:44.243 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.19117718 2020-09-08T15:57:44.243 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.243 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.244 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:57:44.244 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.548220+0000", 2020-09-08T15:57:44.244 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8300000000000001e-05 2020-09-08T15:57:44.244 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.244 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.245 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:57:44.245 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.548238+0000", 2020-09-08T15:57:44.245 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000100648 2020-09-08T15:57:44.245 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.246 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.246 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:57:44.246 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.548339+0000", 2020-09-08T15:57:44.246 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0882000000000003e-05 2020-09-08T15:57:44.246 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:44.247 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:44.247 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:57:44.247 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:09.548410+0000", 2020-09-08T15:57:44.247 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.19181124199999999 2020-09-08T15:57:44.247 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.248 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:44.248 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.248 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:44.248 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:44.248 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:57:44.249 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:57:44.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:44 smithi166 bash[9446]: cluster 2020-09-08T15:57:42.296298+0000 osd.7 (osd.7) 103 : cluster [DBG] 6.8 scrub starts 2020-09-08T15:57:44.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:44 smithi166 bash[9446]: cluster 2020-09-08T15:57:43.079357+0000 mon.a (mon.0) 1755 : cluster [DBG] osdmap e505: 12 total, 12 up, 12 in 2020-09-08T15:57:44.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:44 smithi166 bash[9446]: audit 2020-09-08T15:57:43.364652+0000 mon.a (mon.0) 1756 : audit [DBG] from='client.? v1:172.21.15.104:0/4028989048' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:57:44.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:44 smithi166 bash[9446]: audit 2020-09-08T15:57:43.748748+0000 mon.a (mon.0) 1757 : audit [INF] from='client.? v1:172.21.15.104:0/1054945871' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:57:44.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:44 smithi055 bash[9296]: cluster 2020-09-08T15:57:42.296298+0000 osd.7 (osd.7) 103 : cluster [DBG] 6.8 scrub starts 2020-09-08T15:57:44.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:44 smithi055 bash[9296]: cluster 2020-09-08T15:57:43.079357+0000 mon.a (mon.0) 1755 : cluster [DBG] osdmap e505: 12 total, 12 up, 12 in 2020-09-08T15:57:44.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:44 smithi055 bash[9296]: audit 2020-09-08T15:57:43.364652+0000 mon.a (mon.0) 1756 : audit [DBG] from='client.? v1:172.21.15.104:0/4028989048' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:57:44.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:44 smithi055 bash[9296]: audit 2020-09-08T15:57:43.748748+0000 mon.a (mon.0) 1757 : audit [INF] from='client.? v1:172.21.15.104:0/1054945871' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:57:44.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:44 smithi104 bash[9203]: cluster 2020-09-08T15:57:42.296298+0000 osd.7 (osd.7) 103 : cluster [DBG] 6.8 scrub starts 2020-09-08T15:57:44.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:44 smithi104 bash[9203]: cluster 2020-09-08T15:57:43.079357+0000 mon.a (mon.0) 1755 : cluster [DBG] osdmap e505: 12 total, 12 up, 12 in 2020-09-08T15:57:44.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:44 smithi104 bash[9203]: audit 2020-09-08T15:57:43.364652+0000 mon.a (mon.0) 1756 : audit [DBG] from='client.? v1:172.21.15.104:0/4028989048' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:57:44.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:44 smithi104 bash[9203]: audit 2020-09-08T15:57:43.748748+0000 mon.a (mon.0) 1757 : audit [INF] from='client.? v1:172.21.15.104:0/1054945871' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:57:44.500 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 76 16 104233 104217 85.6957 60.125 0.0130853 0.0116669 2020-09-08T15:57:45.362 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:45 smithi055 bash[9296]: cluster 2020-09-08T15:57:43.916687+0000 mgr.x (mgr.24118) 912 : cluster [DBG] pgmap v1333: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 74 MiB/s wr, 1.19k op/s 2020-09-08T15:57:45.363 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:45 smithi055 bash[9296]: cluster 2020-09-08T15:57:44.036014+0000 osd.3 (osd.3) 75 : cluster [DBG] 6.c scrub starts 2020-09-08T15:57:45.363 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:45 smithi055 bash[9296]: cluster 2020-09-08T15:57:44.085467+0000 mon.a (mon.0) 1758 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:57:45.363 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:45 smithi055 bash[9296]: cluster 2020-09-08T15:57:44.085491+0000 mon.a (mon.0) 1759 : cluster [INF] Cluster is now healthy 2020-09-08T15:57:45.363 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:45 smithi055 bash[9296]: audit 2020-09-08T15:57:44.095120+0000 mon.a (mon.0) 1760 : audit [INF] from='client.? v1:172.21.15.104:0/1054945871' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:57:45.364 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:45 smithi055 bash[9296]: cluster 2020-09-08T15:57:44.095146+0000 mon.a (mon.0) 1761 : cluster [DBG] osdmap e506: 12 total, 12 up, 12 in 2020-09-08T15:57:45.364 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:45.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:45 smithi166 bash[9446]: cluster 2020-09-08T15:57:43.916687+0000 mgr.x (mgr.24118) 912 : cluster [DBG] pgmap v1333: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 74 MiB/s wr, 1.19k op/s 2020-09-08T15:57:45.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:45 smithi166 bash[9446]: cluster 2020-09-08T15:57:44.036014+0000 osd.3 (osd.3) 75 : cluster [DBG] 6.c scrub starts 2020-09-08T15:57:45.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:45 smithi166 bash[9446]: cluster 2020-09-08T15:57:44.085467+0000 mon.a (mon.0) 1758 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:57:45.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:45 smithi166 bash[9446]: cluster 2020-09-08T15:57:44.085491+0000 mon.a (mon.0) 1759 : cluster [INF] Cluster is now healthy 2020-09-08T15:57:45.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:45 smithi166 bash[9446]: audit 2020-09-08T15:57:44.095120+0000 mon.a (mon.0) 1760 : audit [INF] from='client.? v1:172.21.15.104:0/1054945871' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:57:45.373 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:45 smithi166 bash[9446]: cluster 2020-09-08T15:57:44.095146+0000 mon.a (mon.0) 1761 : cluster [DBG] osdmap e506: 12 total, 12 up, 12 in 2020-09-08T15:57:45.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:45 smithi104 bash[9203]: cluster 2020-09-08T15:57:43.916687+0000 mgr.x (mgr.24118) 912 : cluster [DBG] pgmap v1333: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 74 MiB/s wr, 1.19k op/s 2020-09-08T15:57:45.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:45 smithi104 bash[9203]: cluster 2020-09-08T15:57:44.036014+0000 osd.3 (osd.3) 75 : cluster [DBG] 6.c scrub starts 2020-09-08T15:57:45.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:45 smithi104 bash[9203]: cluster 2020-09-08T15:57:44.085467+0000 mon.a (mon.0) 1758 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:57:45.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:45 smithi104 bash[9203]: cluster 2020-09-08T15:57:44.085491+0000 mon.a (mon.0) 1759 : cluster [INF] Cluster is now healthy 2020-09-08T15:57:45.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:45 smithi104 bash[9203]: audit 2020-09-08T15:57:44.095120+0000 mon.a (mon.0) 1760 : audit [INF] from='client.? v1:172.21.15.104:0/1054945871' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:57:45.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:45 smithi104 bash[9203]: cluster 2020-09-08T15:57:44.095146+0000 mon.a (mon.0) 1761 : cluster [DBG] osdmap e506: 12 total, 12 up, 12 in 2020-09-08T15:57:45.474 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:57:45.503 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 77 16 105426 105410 85.5511 74.5625 0.00614488 0.0116859 2020-09-08T15:57:45.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:57:45 smithi055 bash[16646]: debug 2020-09-08T15:57:45.358+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:45.962 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:57:46.065 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:46.199 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:46.300 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:46.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:46 smithi166 bash[9446]: cluster 2020-09-08T15:57:44.305212+0000 osd.7 (osd.7) 104 : cluster [DBG] 6.8 scrub ok 2020-09-08T15:57:46.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:46 smithi055 bash[9296]: cluster 2020-09-08T15:57:44.305212+0000 osd.7 (osd.7) 104 : cluster [DBG] 6.8 scrub ok 2020-09-08T15:57:46.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:57:46 smithi055 bash[16646]: debug 2020-09-08T15:57:46.198+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:46.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:57:45 smithi104 bash[22305]: debug 2020-09-08T15:57:45.962+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:46.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:46 smithi104 bash[9203]: cluster 2020-09-08T15:57:44.305212+0000 osd.7 (osd.7) 104 : cluster [DBG] 6.8 scrub ok 2020-09-08T15:57:46.444 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:46.497 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 78 16 106618 106602 85.4093 74.5 0.0171854 0.011706 2020-09-08T15:57:46.549 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:57:46.777 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:57:46.881 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:57:46.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:57:46 smithi055 bash[11956]: debug 2020-09-08T15:57:46.442+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:46.914 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:57:46 smithi104 bash[22305]: debug 2020-09-08T15:57:46.778+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:46.915 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2020-09-08T15:57:47.024 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:57:47.076 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:57:47.127 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:47.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:47 smithi104 bash[9203]: cluster 2020-09-08T15:57:45.917174+0000 mgr.x (mgr.24118) 913 : cluster [DBG] pgmap v1335: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s wr, 882 op/s 2020-09-08T15:57:47.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:47 smithi104 bash[9203]: cluster 2020-09-08T15:57:46.449038+0000 osd.3 (osd.3) 76 : cluster [DBG] 6.c scrub ok 2020-09-08T15:57:47.244 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:47.351 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:57:47.360 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:47 smithi166 bash[9446]: cluster 2020-09-08T15:57:45.917174+0000 mgr.x (mgr.24118) 913 : cluster [DBG] pgmap v1335: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s wr, 882 op/s 2020-09-08T15:57:47.360 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:47 smithi166 bash[9446]: cluster 2020-09-08T15:57:46.449038+0000 osd.3 (osd.3) 76 : cluster [DBG] 6.c scrub ok 2020-09-08T15:57:47.361 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:57:47 smithi166 bash[18960]: debug 2020-09-08T15:57:47.022+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:47.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:57:47 smithi055 bash[16646]: debug 2020-09-08T15:57:47.242+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:47.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:47 smithi055 bash[9296]: cluster 2020-09-08T15:57:45.917174+0000 mgr.x (mgr.24118) 913 : cluster [DBG] pgmap v1335: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s wr, 882 op/s 2020-09-08T15:57:47.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:47 smithi055 bash[9296]: cluster 2020-09-08T15:57:46.449038+0000 osd.3 (osd.3) 76 : cluster [DBG] 6.c scrub ok 2020-09-08T15:57:47.463 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:57:47.503 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 79 16 107797 107781 85.2608 73.6875 0.00828606 0.0117262 2020-09-08T15:57:47.567 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:57:47.618 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:57:47 smithi166 bash[16593]: debug 2020-09-08T15:57:47.462+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:47.681 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:57:47.789 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:47.936 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:48.038 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:57:48.049 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:57:47 smithi166 bash[11928]: debug 2020-09-08T15:57:47.678+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:48.114 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:57:47 smithi055 bash[16646]: debug 2020-09-08T15:57:47.934+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:48.115 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:48 smithi055 bash[9296]: cluster 2020-09-08T15:57:46.826323+0000 osd.6 (osd.6) 81 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:48.115 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:48 smithi055 bash[9296]: cluster 2020-09-08T15:57:46.826399+0000 osd.6 (osd.6) 82 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:48.115 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:48 smithi055 bash[9296]: cluster 2020-09-08T15:57:47.117299+0000 mon.a (mon.0) 1762 : cluster [DBG] osdmap e507: 12 total, 12 up, 12 in 2020-09-08T15:57:48.156 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:57:48.267 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:57:48.370 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:57:48 smithi166 bash[14254]: debug 2020-09-08T15:57:48.154+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:48.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:48 smithi166 bash[9446]: cluster 2020-09-08T15:57:46.826323+0000 osd.6 (osd.6) 81 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:48.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:48 smithi166 bash[9446]: cluster 2020-09-08T15:57:46.826399+0000 osd.6 (osd.6) 82 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:48.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:48 smithi166 bash[9446]: cluster 2020-09-08T15:57:47.117299+0000 mon.a (mon.0) 1762 : cluster [DBG] osdmap e507: 12 total, 12 up, 12 in 2020-09-08T15:57:48.421 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:57:48.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:48 smithi104 bash[9203]: cluster 2020-09-08T15:57:46.826323+0000 osd.6 (osd.6) 81 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:48.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:48 smithi104 bash[9203]: cluster 2020-09-08T15:57:46.826399+0000 osd.6 (osd.6) 82 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:48.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:48 smithi104 bash[9203]: cluster 2020-09-08T15:57:47.117299+0000 mon.a (mon.0) 1762 : cluster [DBG] osdmap e507: 12 total, 12 up, 12 in 2020-09-08T15:57:48.497 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 15:57:48.501780 min lat: 0.00166475 max lat: 15.0142 avg lat: 0.0117413 2020-09-08T15:57:48.498 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T15:57:48.498 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 80 16 109026 109010 85.1551 76.8125 0.00880363 0.0117413 2020-09-08T15:57:48.524 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:57:48.766 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:57:48.870 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:57:48.882 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:57:48 smithi055 bash[19003]: debug 2020-09-08T15:57:48.418+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:48.917 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:57:48.924 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:57:48.930 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:57:48 smithi104 bash[17588]: debug 2020-09-08T15:57:48.766+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:48.996 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:57:49.083 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:57:49.095 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:57:49.103 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:57:49.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:49 smithi055 bash[9296]: cluster 2020-09-08T15:57:47.917604+0000 mgr.x (mgr.24118) 914 : cluster [DBG] pgmap v1337: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 84 MiB/s wr, 1.34k op/s 2020-09-08T15:57:49.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:57:49 smithi055 bash[14301]: debug 2020-09-08T15:57:48.994+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:49.200 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:49 smithi104 bash[9203]: cluster 2020-09-08T15:57:47.917604+0000 mgr.x (mgr.24118) 914 : cluster [DBG] pgmap v1337: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 84 MiB/s wr, 1.34k op/s 2020-09-08T15:57:49.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:49 smithi166 bash[9446]: cluster 2020-09-08T15:57:47.917604+0000 mgr.x (mgr.24118) 914 : cluster [DBG] pgmap v1337: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 84 MiB/s wr, 1.34k op/s 2020-09-08T15:57:49.498 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 81 16 110166 110150 84.9834 71.25 0.0092818 0.0117645 2020-09-08T15:57:50.318 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:57:50.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:50 smithi055 bash[9296]: cluster 2020-09-08T15:57:49.178808+0000 osd.3 (osd.3) 77 : cluster [DBG] 6.2 scrub starts 2020-09-08T15:57:50.422 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:57:50.429 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:57:50 smithi104 bash[15273]: debug 2020-09-08T15:57:50.314+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:50.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:50 smithi104 bash[9203]: cluster 2020-09-08T15:57:49.178808+0000 osd.3 (osd.3) 77 : cluster [DBG] 6.2 scrub starts 2020-09-08T15:57:50.498 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 82 16 111084 111068 84.6466 57.375 0.0077145 0.0118114 2020-09-08T15:57:50.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:50 smithi166 bash[9446]: cluster 2020-09-08T15:57:49.178808+0000 osd.3 (osd.3) 77 : cluster [DBG] 6.2 scrub starts 2020-09-08T15:57:50.750 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:57:50.852 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:57:51.001 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:57:51.107 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:57:51.117 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:57:51 smithi166 bash[14254]: debug 2020-09-08T15:57:51.002+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:51.120 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:57:50 smithi104 bash[22305]: debug 2020-09-08T15:57:50.750+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:51.120 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:51 smithi104 bash[9203]: cluster 2020-09-08T15:57:49.918037+0000 mgr.x (mgr.24118) 915 : cluster [DBG] pgmap v1338: 17 pgs: 17 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s wr, 967 op/s 2020-09-08T15:57:51.252 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:57:51.356 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:57:51.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:51 smithi055 bash[9296]: cluster 2020-09-08T15:57:49.918037+0000 mgr.x (mgr.24118) 915 : cluster [DBG] pgmap v1338: 17 pgs: 17 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s wr, 967 op/s 2020-09-08T15:57:51.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:57:51 smithi055 bash[14301]: debug 2020-09-08T15:57:51.250+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:51.484 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:57:51.498 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 83 16 112177 112161 84.4497 68.3125 0.00861026 0.011839 2020-09-08T15:57:51.592 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:51.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:51 smithi166 bash[9446]: cluster 2020-09-08T15:57:49.918037+0000 mgr.x (mgr.24118) 915 : cluster [DBG] pgmap v1338: 17 pgs: 17 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s wr, 967 op/s 2020-09-08T15:57:51.716 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:51.820 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:57:51.912 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:57:51 smithi055 bash[11956]: debug 2020-09-08T15:57:51.714+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:51.912 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:57:51 smithi055 bash[19003]: debug 2020-09-08T15:57:51.478+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:52.126 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:52 smithi104 bash[9203]: cluster 2020-09-08T15:57:50.746053+0000 osd.11 (osd.11) 86 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:52.127 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:52 smithi104 bash[9203]: cluster 2020-09-08T15:57:50.746174+0000 osd.11 (osd.11) 87 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:52.127 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:52 smithi104 bash[9203]: cluster 2020-09-08T15:57:51.134377+0000 mon.a (mon.0) 1763 : cluster [DBG] osdmap e508: 12 total, 12 up, 12 in 2020-09-08T15:57:52.238 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:57:52.341 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:57:52.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:52 smithi055 bash[9296]: cluster 2020-09-08T15:57:50.746053+0000 osd.11 (osd.11) 86 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:52.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:52 smithi055 bash[9296]: cluster 2020-09-08T15:57:50.746174+0000 osd.11 (osd.11) 87 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:52.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:52 smithi055 bash[9296]: cluster 2020-09-08T15:57:51.134377+0000 mon.a (mon.0) 1763 : cluster [DBG] osdmap e508: 12 total, 12 up, 12 in 2020-09-08T15:57:52.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:52 smithi055 bash[9296]: cluster 2020-09-08T15:57:51.584665+0000 osd.3 (osd.3) 78 : cluster [DBG] 6.2 scrub ok 2020-09-08T15:57:52.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:57:52 smithi104 bash[22305]: debug 2020-09-08T15:57:52.238+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:52.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:52 smithi104 bash[9203]: cluster 2020-09-08T15:57:51.584665+0000 osd.3 (osd.3) 78 : cluster [DBG] 6.2 scrub ok 2020-09-08T15:57:52.499 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 84 16 113368 113352 84.3305 74.4375 0.00565655 0.0118557 2020-09-08T15:57:52.578 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:57:52.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:52 smithi166 bash[9446]: cluster 2020-09-08T15:57:50.746053+0000 osd.11 (osd.11) 86 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:52.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:52 smithi166 bash[9446]: cluster 2020-09-08T15:57:50.746174+0000 osd.11 (osd.11) 87 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:52.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:52 smithi166 bash[9446]: cluster 2020-09-08T15:57:51.134377+0000 mon.a (mon.0) 1763 : cluster [DBG] osdmap e508: 12 total, 12 up, 12 in 2020-09-08T15:57:52.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:52 smithi166 bash[9446]: cluster 2020-09-08T15:57:51.584665+0000 osd.3 (osd.3) 78 : cluster [DBG] 6.2 scrub ok 2020-09-08T15:57:52.684 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:57:52.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:57:52 smithi104 bash[15273]: debug 2020-09-08T15:57:52.570+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:53.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:53 smithi055 bash[9296]: cluster 2020-09-08T15:57:51.918505+0000 mgr.x (mgr.24118) 916 : cluster [DBG] pgmap v1340: 17 pgs: 17 active+clean; 6.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 MiB/s wr, 1.07k op/s 2020-09-08T15:57:53.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:53 smithi055 bash[9296]: cluster 2020-09-08T15:57:52.116323+0000 osd.3 (osd.3) 79 : cluster [DBG] 6.b scrub starts 2020-09-08T15:57:53.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:53 smithi104 bash[9203]: cluster 2020-09-08T15:57:51.918505+0000 mgr.x (mgr.24118) 916 : cluster [DBG] pgmap v1340: 17 pgs: 17 active+clean; 6.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 MiB/s wr, 1.07k op/s 2020-09-08T15:57:53.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:53 smithi104 bash[9203]: cluster 2020-09-08T15:57:52.116323+0000 osd.3 (osd.3) 79 : cluster [DBG] 6.b scrub starts 2020-09-08T15:57:53.498 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 85 16 114483 114467 84.1581 69.6875 0.0283281 0.0118798 2020-09-08T15:57:53.622 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:53 smithi166 bash[9446]: cluster 2020-09-08T15:57:51.918505+0000 mgr.x (mgr.24118) 916 : cluster [DBG] pgmap v1340: 17 pgs: 17 active+clean; 6.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 MiB/s wr, 1.07k op/s 2020-09-08T15:57:53.622 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:53 smithi166 bash[9446]: cluster 2020-09-08T15:57:52.116323+0000 osd.3 (osd.3) 79 : cluster [DBG] 6.b scrub starts 2020-09-08T15:57:53.776 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:57:53.881 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:57:54.057 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:57:53 smithi104 bash[22305]: debug 2020-09-08T15:57:53.774+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:54.498 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 86 16 115563 115547 83.9644 67.5 0.0218635 0.0119077 2020-09-08T15:57:54.688 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:57:54.791 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:54.941 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:57:55.045 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:57:55.053 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:57:54 smithi055 bash[16646]: debug 2020-09-08T15:57:54.942+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:55.141 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:57:54 smithi104 bash[15273]: debug 2020-09-08T15:57:54.686+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:55.160 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:57:55.261 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:55.382 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:57:55 smithi055 bash[19003]: debug 2020-09-08T15:57:55.146+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:55.383 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:55 smithi055 bash[9296]: cluster 2020-09-08T15:57:53.918943+0000 mgr.x (mgr.24118) 917 : cluster [DBG] pgmap v1341: 17 pgs: 17 active+clean; 6.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 74 MiB/s wr, 1.19k op/s 2020-09-08T15:57:55.384 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:57:55.406 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:55 smithi104 bash[9203]: cluster 2020-09-08T15:57:53.918943+0000 mgr.x (mgr.24118) 917 : cluster [DBG] pgmap v1341: 17 pgs: 17 active+clean; 6.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 74 MiB/s wr, 1.19k op/s 2020-09-08T15:57:55.491 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:57:55.499 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 87 16 116630 116614 83.7657 66.6875 0.00420807 0.0119316 2020-09-08T15:57:55.504 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:55 smithi166 bash[9446]: cluster 2020-09-08T15:57:53.918943+0000 mgr.x (mgr.24118) 917 : cluster [DBG] pgmap v1341: 17 pgs: 17 active+clean; 6.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 74 MiB/s wr, 1.19k op/s 2020-09-08T15:57:55.628 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:57:55.658 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:57:55 smithi055 bash[11956]: debug 2020-09-08T15:57:55.382+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:55.735 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:57:55.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:57:55 smithi166 bash[11928]: debug 2020-09-08T15:57:55.626+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:56.150 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:56 smithi104 bash[9203]: cluster 2020-09-08T15:57:55.267663+0000 osd.4 (osd.4) 53 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:56.157 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:56 smithi104 bash[9203]: cluster 2020-09-08T15:57:55.267753+0000 osd.4 (osd.4) 54 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:56.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:56 smithi055 bash[9296]: cluster 2020-09-08T15:57:55.267663+0000 osd.4 (osd.4) 53 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:56.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:56 smithi055 bash[9296]: cluster 2020-09-08T15:57:55.267753+0000 osd.4 (osd.4) 54 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:56.498 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 88 16 117586 117570 83.4927 59.75 0.0116994 0.011975 2020-09-08T15:57:56.580 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:57:56.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:56 smithi166 bash[9446]: cluster 2020-09-08T15:57:55.267663+0000 osd.4 (osd.4) 53 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:57:56.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:56 smithi166 bash[9446]: cluster 2020-09-08T15:57:55.267753+0000 osd.4 (osd.4) 54 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:57:56.684 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:57:56.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:57:56 smithi104 bash[19922]: debug 2020-09-08T15:57:56.581+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:57:56.980 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:57:56.981 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T15:57:56.981 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:56.981 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:118102 6.b 6:ddf6976d:::benchmark_data_smithi166_43187_object118101:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e509)", 2020-09-08T15:57:56.981 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:56.961497+0000", 2020-09-08T15:57:56.981 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.024554731, 2020-09-08T15:57:56.982 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.024581461999999998, 2020-09-08T15:57:56.982 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:56.982 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:57:56.982 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:56.983 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:57:56.983 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:57:56.983 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 118102 2020-09-08T15:57:56.983 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:56.983 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:56.983 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:56.984 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:56.984 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.961497+0000", 2020-09-08T15:57:56.984 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:56.984 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:56.984 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:56.985 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:56.985 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.961497+0000", 2020-09-08T15:57:56.985 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999609 2020-09-08T15:57:56.985 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:56.987 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:56.988 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:56.988 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.961458+0000", 2020-09-08T15:57:56.988 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015443100000000001 2020-09-08T15:57:56.988 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:56.989 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:56.989 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:56.989 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.961612+0000", 2020-09-08T15:57:56.989 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3784000000000001e-05 2020-09-08T15:57:56.989 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:56.990 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:56.990 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:56.990 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.961626+0000", 2020-09-08T15:57:56.990 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9560000000000002e-06 2020-09-08T15:57:56.991 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:56.991 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:56.991 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:56.991 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.961632+0000", 2020-09-08T15:57:56.991 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0038216919999999998 2020-09-08T15:57:56.992 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:56.992 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:56.992 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:56.992 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.965454+0000", 2020-09-08T15:57:56.992 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014280200000000001 2020-09-08T15:57:56.993 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:56.993 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:56.993 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:56.993 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.965596+0000", 2020-09-08T15:57:56.993 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016789999999999999 2020-09-08T15:57:56.994 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:56.994 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:56.994 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 7", 2020-09-08T15:57:56.994 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.965764+0000", 2020-09-08T15:57:56.994 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0144432 2020-09-08T15:57:56.995 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:56.995 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:56.995 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:57:56.995 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.980208+0000", 2020-09-08T15:57:56.995 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.018710649999999999 2020-09-08T15:57:56.995 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:56.996 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:56.996 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:56.996 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:56.996 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:56.996 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:118103 6.b 6:d8d6a6c4:::benchmark_data_smithi166_43187_object118102:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e509)", 2020-09-08T15:57:56.997 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:56.961695+0000", 2020-09-08T15:57:57.000 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.024356250999999999, 2020-09-08T15:57:57.000 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.024449378000000001, 2020-09-08T15:57:57.000 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:57.000 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:57:57.001 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:57.001 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:57:57.001 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:57:57.003 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 118103 2020-09-08T15:57:57.003 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.004 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:57.004 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.004 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:57.004 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.961695+0000", 2020-09-08T15:57:57.004 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:57.005 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.005 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.005 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:57.006 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.961695+0000", 2020-09-08T15:57:57.006 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999628 2020-09-08T15:57:57.006 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.006 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.006 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:57.007 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.961658+0000", 2020-09-08T15:57:57.007 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013374900000000001 2020-09-08T15:57:57.007 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.007 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.007 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:57.008 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.961792+0000", 2020-09-08T15:57:57.008 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2081e-05 2020-09-08T15:57:57.008 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.008 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.008 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:57.008 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.961804+0000", 2020-09-08T15:57:57.009 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.045e-06 2020-09-08T15:57:57.009 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.009 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.009 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:57.010 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.961809+0000", 2020-09-08T15:57:57.010 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0080700640000000001 2020-09-08T15:57:57.010 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.011 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.011 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:57.011 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.969879+0000", 2020-09-08T15:57:57.011 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013326500000000001 2020-09-08T15:57:57.011 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.012 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.012 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:57.012 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.970012+0000", 2020-09-08T15:57:57.012 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015601100000000001 2020-09-08T15:57:57.012 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.013 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.013 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 7", 2020-09-08T15:57:57.013 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.970168+0000", 2020-09-08T15:57:57.013 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010051302 2020-09-08T15:57:57.013 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.014 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.014 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:57:57.014 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.980220+0000", 2020-09-08T15:57:57.015 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.018524232000000002 2020-09-08T15:57:57.015 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:57.015 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:57.015 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:57.015 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.016 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.016 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:118106 6.b 6:d7966846:::benchmark_data_smithi166_43187_object118105:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e509)", 2020-09-08T15:57:57.016 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:56.961869+0000", 2020-09-08T15:57:57.016 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.024182493999999999, 2020-09-08T15:57:57.016 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.024332965000000002, 2020-09-08T15:57:57.017 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:57.017 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:57:57.017 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:57.017 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:57:57.017 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:57:57.017 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 118106 2020-09-08T15:57:57.018 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.018 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:57.018 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.018 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:57.018 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.961869+0000", 2020-09-08T15:57:57.019 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:57.019 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.019 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.019 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:57.019 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.961869+0000", 2020-09-08T15:57:57.020 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999633 2020-09-08T15:57:57.020 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.020 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.021 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:57.021 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.961832+0000", 2020-09-08T15:57:57.021 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012592899999999999 2020-09-08T15:57:57.021 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.021 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.022 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:57.022 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.961958+0000", 2020-09-08T15:57:57.022 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1922e-05 2020-09-08T15:57:57.022 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.022 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.023 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:57.023 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.961970+0000", 2020-09-08T15:57:57.023 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.549e-06 2020-09-08T15:57:57.023 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.023 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.024 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:57.024 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.961975+0000", 2020-09-08T15:57:57.024 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.012742497 2020-09-08T15:57:57.024 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.024 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.025 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:57.025 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.974717+0000", 2020-09-08T15:57:57.025 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013573 2020-09-08T15:57:57.025 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.025 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.026 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:57.026 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.974853+0000", 2020-09-08T15:57:57.026 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000167622 2020-09-08T15:57:57.026 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.026 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.027 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 7", 2020-09-08T15:57:57.027 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.975021+0000", 2020-09-08T15:57:57.027 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013151319999999999 2020-09-08T15:57:57.027 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:57.027 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:57.028 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:57.028 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.028 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.028 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:118109 6.b 6:de099669:::benchmark_data_smithi166_43187_object118108:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e509)", 2020-09-08T15:57:57.028 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:56.963559+0000", 2020-09-08T15:57:57.029 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.022492941999999998, 2020-09-08T15:57:57.029 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.022695967000000001, 2020-09-08T15:57:57.029 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:57.029 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:57:57.029 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:57.030 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:57:57.030 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:57:57.030 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 118109 2020-09-08T15:57:57.030 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.031 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:57.031 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.031 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:57.031 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.963559+0000", 2020-09-08T15:57:57.032 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:57.032 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.032 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.032 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:57.032 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.963559+0000", 2020-09-08T15:57:57.033 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999547 2020-09-08T15:57:57.033 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.033 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.033 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:57.033 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.963513+0000", 2020-09-08T15:57:57.034 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016115899999999999 2020-09-08T15:57:57.034 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.034 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.034 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:57.034 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.963675+0000", 2020-09-08T15:57:57.035 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5622000000000001e-05 2020-09-08T15:57:57.035 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.035 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.035 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:57.035 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.963690+0000", 2020-09-08T15:57:57.036 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6969999999999996e-06 2020-09-08T15:57:57.036 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.036 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.036 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:57.036 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.963697+0000", 2020-09-08T15:57:57.037 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.016845335999999999 2020-09-08T15:57:57.037 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.037 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.037 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:57.037 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.980542+0000", 2020-09-08T15:57:57.037 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000192738 2020-09-08T15:57:57.038 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.038 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.038 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:57.038 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.980735+0000", 2020-09-08T15:57:57.038 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000147906 2020-09-08T15:57:57.039 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.039 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.039 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 7", 2020-09-08T15:57:57.039 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.980883+0000", 2020-09-08T15:57:57.039 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.017324188000000001 2020-09-08T15:57:57.040 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:57.040 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:57.040 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:57.040 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.040 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.041 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:118124 6.b 6:d87f1f51:::benchmark_data_smithi166_43187_object118123:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e509)", 2020-09-08T15:57:57.041 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:56.980617+0000", 2020-09-08T15:57:57.041 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0054344509999999999, 2020-09-08T15:57:57.042 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0057093969999999997, 2020-09-08T15:57:57.042 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:57:57.042 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:57:57.042 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:57:57.043 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:57:57.043 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:57:57.043 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 118124 2020-09-08T15:57:57.043 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.044 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:57:57.044 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.044 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:57:57.044 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.980617+0000", 2020-09-08T15:57:57.044 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:57:57.045 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.045 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.045 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:57:57.045 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.980617+0000", 2020-09-08T15:57:57.045 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999576 2020-09-08T15:57:57.046 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.046 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.046 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:57:57.046 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.980575+0000", 2020-09-08T15:57:57.046 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000146377 2020-09-08T15:57:57.047 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.047 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.047 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:57:57.047 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.980721+0000", 2020-09-08T15:57:57.047 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6003000000000002e-05 2020-09-08T15:57:57.047 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.048 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.048 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:57:57.048 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.980747+0000", 2020-09-08T15:57:57.048 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5110000000000003e-06 2020-09-08T15:57:57.048 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.049 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.049 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:57:57.049 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.980756+0000", 2020-09-08T15:57:57.049 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0032148469999999998 2020-09-08T15:57:57.049 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.050 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.050 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:57:57.050 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.983971+0000", 2020-09-08T15:57:57.050 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010571 2020-09-08T15:57:57.050 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.051 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.051 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:57:57.051 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.984076+0000", 2020-09-08T15:57:57.051 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000126771 2020-09-08T15:57:57.051 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:57:57.052 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:57:57.052 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 7", 2020-09-08T15:57:57.052 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:56.984203+0000", 2020-09-08T15:57:57.053 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0035857710000000002 2020-09-08T15:57:57.053 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:57.053 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:57:57.053 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:57.053 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:57:57.054 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T15:57:57.054 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 5 2020-09-08T15:57:57.054 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:57:57.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:57 smithi055 bash[9296]: cluster 2020-09-08T15:57:55.758803+0000 osd.3 (osd.3) 80 : cluster [DBG] 6.b scrub ok 2020-09-08T15:57:57.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:57 smithi055 bash[9296]: cluster 2020-09-08T15:57:55.919413+0000 mgr.x (mgr.24118) 918 : cluster [DBG] pgmap v1342: 17 pgs: 17 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s wr, 992 op/s 2020-09-08T15:57:57.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:57 smithi055 bash[9296]: cluster 2020-09-08T15:57:56.158347+0000 mon.a (mon.0) 1764 : cluster [DBG] osdmap e509: 12 total, 12 up, 12 in 2020-09-08T15:57:57.478 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:57 smithi104 bash[9203]: cluster 2020-09-08T15:57:55.758803+0000 osd.3 (osd.3) 80 : cluster [DBG] 6.b scrub ok 2020-09-08T15:57:57.479 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:57 smithi104 bash[9203]: cluster 2020-09-08T15:57:55.919413+0000 mgr.x (mgr.24118) 918 : cluster [DBG] pgmap v1342: 17 pgs: 17 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s wr, 992 op/s 2020-09-08T15:57:57.479 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:57 smithi104 bash[9203]: cluster 2020-09-08T15:57:56.158347+0000 mon.a (mon.0) 1764 : cluster [DBG] osdmap e509: 12 total, 12 up, 12 in 2020-09-08T15:57:57.499 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 89 16 118579 118563 83.2519 62.0625 0.0734298 0.0120046 2020-09-08T15:57:57.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:57 smithi166 bash[9446]: cluster 2020-09-08T15:57:55.758803+0000 osd.3 (osd.3) 80 : cluster [DBG] 6.b scrub ok 2020-09-08T15:57:57.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:57 smithi166 bash[9446]: cluster 2020-09-08T15:57:55.919413+0000 mgr.x (mgr.24118) 918 : cluster [DBG] pgmap v1342: 17 pgs: 17 active+clean; 6.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s wr, 992 op/s 2020-09-08T15:57:57.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:57 smithi166 bash[9446]: cluster 2020-09-08T15:57:56.158347+0000 mon.a (mon.0) 1764 : cluster [DBG] osdmap e509: 12 total, 12 up, 12 in 2020-09-08T15:57:58.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:58 smithi055 bash[9296]: cluster 2020-09-08T15:57:56.327752+0000 osd.7 (osd.7) 105 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:57:58.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:58 smithi055 bash[9296]: cluster 2020-09-08T15:57:56.423051+0000 osd.7 (osd.7) 106 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:57:58.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:58 smithi104 bash[9203]: cluster 2020-09-08T15:57:56.327752+0000 osd.7 (osd.7) 105 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:57:58.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:58 smithi104 bash[9203]: cluster 2020-09-08T15:57:56.423051+0000 osd.7 (osd.7) 106 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:57:58.499 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 90 16 119766 119750 83.1511 74.1875 0.007463 0.0120231 2020-09-08T15:57:58.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:58 smithi166 bash[9446]: cluster 2020-09-08T15:57:56.327752+0000 osd.7 (osd.7) 105 : cluster [DBG] 1.0 scrub starts 2020-09-08T15:57:58.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:58 smithi166 bash[9446]: cluster 2020-09-08T15:57:56.423051+0000 osd.7 (osd.7) 106 : cluster [DBG] 1.0 scrub ok 2020-09-08T15:57:59.173 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2020-09-08T15:57:59.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:57:59 smithi104 bash[9203]: cluster 2020-09-08T15:57:57.919940+0000 mgr.x (mgr.24118) 919 : cluster [DBG] pgmap v1344: 17 pgs: 17 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T15:57:59.427 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:57:59.499 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Total time run: 90.0107 2020-09-08T15:57:59.499 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Total writes made: 119766 2020-09-08T15:57:59.499 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Write size: 65536 2020-09-08T15:57:59.500 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Object size: 65536 2020-09-08T15:57:59.500 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Bandwidth (MB/sec): 83.1609 2020-09-08T15:57:59.500 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev Bandwidth: 50.3193 2020-09-08T15:57:59.500 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max bandwidth (MB/sec): 217.75 2020-09-08T15:57:59.500 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min bandwidth (MB/sec): 0 2020-09-08T15:57:59.501 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Average IOPS: 1330 2020-09-08T15:57:59.501 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev IOPS: 805 2020-09-08T15:57:59.501 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max IOPS: 3484 2020-09-08T15:57:59.501 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min IOPS: 0 2020-09-08T15:57:59.501 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Average Latency(s): 0.0120236 2020-09-08T15:57:59.502 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev Latency(s): 0.173193 2020-09-08T15:57:59.504 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max latency(s): 15.0142 2020-09-08T15:57:59.504 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min latency(s): 0.00166475 2020-09-08T15:57:59.511 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Cleaning up (deleting benchmark objects) 2020-09-08T15:57:59.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:57:59 smithi166 bash[9446]: cluster 2020-09-08T15:57:57.919940+0000 mgr.x (mgr.24118) 919 : cluster [DBG] pgmap v1344: 17 pgs: 17 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T15:57:59.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:57:59 smithi055 bash[9296]: cluster 2020-09-08T15:57:57.919940+0000 mgr.x (mgr.24118) 919 : cluster [DBG] pgmap v1344: 17 pgs: 17 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T15:58:00.587 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:00.694 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:00.822 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:00.877 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:58:00 smithi055 bash[14301]: debug 2020-09-08T15:58:00.586+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:00.877 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:58:00 smithi055 bash[14301]: debug 2020-09-08T15:58:00.826+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:00.928 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:00.932 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:00 smithi104 bash[9203]: cluster 2020-09-08T15:57:59.920364+0000 mgr.x (mgr.24118) 920 : cluster [DBG] pgmap v1345: 17 pgs: 17 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s wr, 1.14k op/s 2020-09-08T15:58:01.044 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:01.123 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:58:01.150 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:01.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:00 smithi055 bash[9296]: cluster 2020-09-08T15:57:59.920364+0000 mgr.x (mgr.24118) 920 : cluster [DBG] pgmap v1345: 17 pgs: 17 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s wr, 1.14k op/s 2020-09-08T15:58:01.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:58:01 smithi055 bash[14301]: debug 2020-09-08T15:58:01.046+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:01.159 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:58:01.196 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:58:01.231 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:58:01.268 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:58:01.274 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:01.304 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:58:01.339 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:58:01.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:00 smithi166 bash[9446]: cluster 2020-09-08T15:57:59.920364+0000 mgr.x (mgr.24118) 920 : cluster [DBG] pgmap v1345: 17 pgs: 17 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s wr, 1.14k op/s 2020-09-08T15:58:01.376 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:58:01.380 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:58:01.418 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:58:01.457 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:58:01.500 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:58:01.518 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:58:01.536 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:58:01.537 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:58:01.538 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:58:01.538 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:58:01.538 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:58:01.538 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:58:01.539 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:58:01.539 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:58:01.539 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:58:01.539 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:58:01.540 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:58:01.540 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:58:01.540 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:58:01.621 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:58:01.659 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:58:01 smithi055 bash[14301]: debug 2020-09-08T15:58:01.278+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:01.736 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:58:01 smithi166 bash[18960]: debug 2020-09-08T15:58:01.518+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:01.736 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:58:01.839 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:01.887 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:58:02.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:58:01 smithi166 bash[16593]: debug 2020-09-08T15:58:01.734+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:02.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:01 smithi166 bash[9446]: audit 2020-09-08T15:58:01.095832+0000 mon.a (mon.0) 1765 : audit [DBG] from='client.? v1:172.21.15.104:0/2133860680' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:58:02.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:01 smithi166 bash[9446]: audit 2020-09-08T15:58:01.720897+0000 mon.b (mon.1) 106 : audit [INF] from='client.? v1:172.21.15.104:0/3778931643' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:58:02.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:01 smithi166 bash[9446]: audit 2020-09-08T15:58:01.721378+0000 mon.a (mon.0) 1766 : audit [INF] from='client.25978 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:58:02.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:01 smithi055 bash[9296]: audit 2020-09-08T15:58:01.095832+0000 mon.a (mon.0) 1765 : audit [DBG] from='client.? v1:172.21.15.104:0/2133860680' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:58:02.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:01 smithi055 bash[9296]: audit 2020-09-08T15:58:01.720897+0000 mon.b (mon.1) 106 : audit [INF] from='client.? v1:172.21.15.104:0/3778931643' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:58:02.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:01 smithi055 bash[9296]: audit 2020-09-08T15:58:01.721378+0000 mon.a (mon.0) 1766 : audit [INF] from='client.25978 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:58:02.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:01 smithi104 bash[9203]: audit 2020-09-08T15:58:01.095832+0000 mon.a (mon.0) 1765 : audit [DBG] from='client.? v1:172.21.15.104:0/2133860680' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:58:02.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:01 smithi104 bash[9203]: audit 2020-09-08T15:58:01.720897+0000 mon.b (mon.1) 106 : audit [INF] from='client.? v1:172.21.15.104:0/3778931643' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:58:02.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:01 smithi104 bash[9203]: audit 2020-09-08T15:58:01.721378+0000 mon.a (mon.0) 1766 : audit [INF] from='client.25978 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:58:03.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:02 smithi055 bash[9296]: cluster 2020-09-08T15:58:01.882718+0000 mon.a (mon.0) 1767 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:58:03.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:02 smithi055 bash[9296]: audit 2020-09-08T15:58:01.887734+0000 mon.a (mon.0) 1768 : audit [INF] from='client.25978 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:58:03.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:02 smithi055 bash[9296]: cluster 2020-09-08T15:58:01.887786+0000 mon.a (mon.0) 1769 : cluster [DBG] osdmap e510: 12 total, 12 up, 12 in 2020-09-08T15:58:03.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:02 smithi055 bash[9296]: cluster 2020-09-08T15:58:01.920750+0000 mgr.x (mgr.24118) 921 : cluster [DBG] pgmap v1347: 17 pgs: 17 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 54 MiB/s wr, 932 op/s 2020-09-08T15:58:03.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:02 smithi104 bash[9203]: cluster 2020-09-08T15:58:01.882718+0000 mon.a (mon.0) 1767 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:58:03.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:02 smithi104 bash[9203]: audit 2020-09-08T15:58:01.887734+0000 mon.a (mon.0) 1768 : audit [INF] from='client.25978 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:58:03.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:02 smithi104 bash[9203]: cluster 2020-09-08T15:58:01.887786+0000 mon.a (mon.0) 1769 : cluster [DBG] osdmap e510: 12 total, 12 up, 12 in 2020-09-08T15:58:03.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:02 smithi104 bash[9203]: cluster 2020-09-08T15:58:01.920750+0000 mgr.x (mgr.24118) 921 : cluster [DBG] pgmap v1347: 17 pgs: 17 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 54 MiB/s wr, 932 op/s 2020-09-08T15:58:03.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:02 smithi166 bash[9446]: cluster 2020-09-08T15:58:01.882718+0000 mon.a (mon.0) 1767 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:58:03.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:02 smithi166 bash[9446]: audit 2020-09-08T15:58:01.887734+0000 mon.a (mon.0) 1768 : audit [INF] from='client.25978 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:58:03.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:02 smithi166 bash[9446]: cluster 2020-09-08T15:58:01.887786+0000 mon.a (mon.0) 1769 : cluster [DBG] osdmap e510: 12 total, 12 up, 12 in 2020-09-08T15:58:03.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:02 smithi166 bash[9446]: cluster 2020-09-08T15:58:01.920750+0000 mgr.x (mgr.24118) 921 : cluster [DBG] pgmap v1347: 17 pgs: 17 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 54 MiB/s wr, 932 op/s 2020-09-08T15:58:03.745 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:03.849 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:58:04.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:03 smithi055 bash[9296]: cluster 2020-09-08T15:58:03.720795+0000 osd.9 (osd.9) 77 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:04.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:03 smithi055 bash[9296]: cluster 2020-09-08T15:58:03.720920+0000 osd.9 (osd.9) 78 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:04.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:58:03 smithi104 bash[22305]: debug 2020-09-08T15:58:03.745+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:04.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:03 smithi104 bash[9203]: cluster 2020-09-08T15:58:03.720795+0000 osd.9 (osd.9) 77 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:04.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:03 smithi104 bash[9203]: cluster 2020-09-08T15:58:03.720920+0000 osd.9 (osd.9) 78 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:04.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:03 smithi166 bash[9446]: cluster 2020-09-08T15:58:03.720795+0000 osd.9 (osd.9) 77 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:04.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:03 smithi166 bash[9446]: cluster 2020-09-08T15:58:03.720920+0000 osd.9 (osd.9) 78 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:04.446 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:58:04.549 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:58:04.695 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:58:04.700 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:58:04 smithi104 bash[19922]: debug 2020-09-08T15:58:04.445+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:04.799 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:04.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:58:04 smithi166 bash[18960]: debug 2020-09-08T15:58:04.694+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:04.938 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:05.042 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:58:05.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:58:04 smithi055 bash[14301]: debug 2020-09-08T15:58:04.942+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:05.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:04 smithi055 bash[9296]: cluster 2020-09-08T15:58:03.911517+0000 mon.a (mon.0) 1770 : cluster [DBG] osdmap e511: 12 total, 12 up, 12 in 2020-09-08T15:58:05.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:04 smithi055 bash[9296]: cluster 2020-09-08T15:58:03.921166+0000 mgr.x (mgr.24118) 922 : cluster [DBG] pgmap v1349: 17 pgs: 17 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 131 B/s rd, 48 MiB/s wr, 1.03k op/s 2020-09-08T15:58:05.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:04 smithi104 bash[9203]: cluster 2020-09-08T15:58:03.911517+0000 mon.a (mon.0) 1770 : cluster [DBG] osdmap e511: 12 total, 12 up, 12 in 2020-09-08T15:58:05.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:04 smithi104 bash[9203]: cluster 2020-09-08T15:58:03.921166+0000 mgr.x (mgr.24118) 922 : cluster [DBG] pgmap v1349: 17 pgs: 17 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 131 B/s rd, 48 MiB/s wr, 1.03k op/s 2020-09-08T15:58:05.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:04 smithi166 bash[9446]: cluster 2020-09-08T15:58:03.911517+0000 mon.a (mon.0) 1770 : cluster [DBG] osdmap e511: 12 total, 12 up, 12 in 2020-09-08T15:58:05.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:04 smithi166 bash[9446]: cluster 2020-09-08T15:58:03.921166+0000 mgr.x (mgr.24118) 922 : cluster [DBG] pgmap v1349: 17 pgs: 17 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 131 B/s rd, 48 MiB/s wr, 1.03k op/s 2020-09-08T15:58:05.529 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:58:05.633 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:05.778 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:05.882 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:58:05.897 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:58:05 smithi166 bash[14254]: debug 2020-09-08T15:58:05.778+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:05.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:58:05 smithi104 bash[15273]: debug 2020-09-08T15:58:05.529+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:06.013 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:58:06.118 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:58:06.241 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:58:06.345 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:58:06.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:58:06 smithi166 bash[18960]: debug 2020-09-08T15:58:06.010+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:06.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:58:06 smithi055 bash[16646]: debug 2020-09-08T15:58:06.242+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:06.492 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:58:06.577 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:58:06.597 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:58:06.701 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:58:06.704 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:58:06.743 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:58:06.812 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:06.865 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:58:06.906 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:58:06 smithi055 bash[11956]: debug 2020-09-08T15:58:06.494+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:06.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:58:06 smithi055 bash[11956]: debug 2020-09-08T15:58:06.706+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:07.033 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T15:58:07.033 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:58:07.156 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:06 smithi104 bash[9203]: cluster 2020-09-08T15:58:05.921645+0000 mgr.x (mgr.24118) 923 : cluster [DBG] pgmap v1350: 17 pgs: 17 active+clean; 7.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 19 MiB/s wr, 769 op/s 2020-09-08T15:58:07.157 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:06 smithi104 bash[9203]: cluster 2020-09-08T15:58:06.236732+0000 osd.2 (osd.2) 74 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:07.157 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:06 smithi104 bash[9203]: cluster 2020-09-08T15:58:06.236824+0000 osd.2 (osd.2) 75 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:06 smithi166 bash[9446]: cluster 2020-09-08T15:58:05.921645+0000 mgr.x (mgr.24118) 923 : cluster [DBG] pgmap v1350: 17 pgs: 17 active+clean; 7.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 19 MiB/s wr, 769 op/s 2020-09-08T15:58:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:06 smithi166 bash[9446]: cluster 2020-09-08T15:58:06.236732+0000 osd.2 (osd.2) 74 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:07.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:06 smithi166 bash[9446]: cluster 2020-09-08T15:58:06.236824+0000 osd.2 (osd.2) 75 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:07.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:06 smithi055 bash[9296]: cluster 2020-09-08T15:58:05.921645+0000 mgr.x (mgr.24118) 923 : cluster [DBG] pgmap v1350: 17 pgs: 17 active+clean; 7.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 19 MiB/s wr, 769 op/s 2020-09-08T15:58:07.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:06 smithi055 bash[9296]: cluster 2020-09-08T15:58:06.236732+0000 osd.2 (osd.2) 74 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:07.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:06 smithi055 bash[9296]: cluster 2020-09-08T15:58:06.236824+0000 osd.2 (osd.2) 75 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:08.046 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:08.150 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:58:08.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:07 smithi166 bash[9446]: cluster 2020-09-08T15:58:06.986708+0000 mon.a (mon.0) 1771 : cluster [DBG] osdmap e512: 12 total, 12 up, 12 in 2020-09-08T15:58:08.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:08 smithi055 bash[9296]: cluster 2020-09-08T15:58:06.986708+0000 mon.a (mon.0) 1771 : cluster [DBG] osdmap e512: 12 total, 12 up, 12 in 2020-09-08T15:58:08.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:58:08 smithi104 bash[22305]: debug 2020-09-08T15:58:08.045+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:08.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:08 smithi104 bash[9203]: cluster 2020-09-08T15:58:06.986708+0000 mon.a (mon.0) 1771 : cluster [DBG] osdmap e512: 12 total, 12 up, 12 in 2020-09-08T15:58:08.952 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:58:09.056 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:09.191 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:09.295 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:09.303 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:09 smithi166 bash[9446]: cluster 2020-09-08T15:58:07.922132+0000 mgr.x (mgr.24118) 924 : cluster [DBG] pgmap v1352: 17 pgs: 17 active+clean; 6.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s wr, 1.31k op/s 2020-09-08T15:58:09.304 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:58:09 smithi166 bash[14254]: debug 2020-09-08T15:58:09.190+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:09.358 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:58:08 smithi104 bash[19922]: debug 2020-09-08T15:58:08.953+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:09.359 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2020-09-08T15:58:09.360 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:09 smithi104 bash[9203]: cluster 2020-09-08T15:58:07.922132+0000 mgr.x (mgr.24118) 924 : cluster [DBG] pgmap v1352: 17 pgs: 17 active+clean; 6.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s wr, 1.31k op/s 2020-09-08T15:58:09.406 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:09.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:09 smithi055 bash[9296]: cluster 2020-09-08T15:58:07.922132+0000 mgr.x (mgr.24118) 924 : cluster [DBG] pgmap v1352: 17 pgs: 17 active+clean; 6.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s wr, 1.31k op/s 2020-09-08T15:58:09.518 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:58:09.520 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:58:09.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:58:09 smithi166 bash[11928]: debug 2020-09-08T15:58:09.406+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:09.673 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:58:09.778 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:10.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:58:09 smithi166 bash[16593]: debug 2020-09-08T15:58:09.670+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:10.625 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:10.728 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:10.873 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:10.900 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:10 smithi104 bash[9203]: cluster 2020-09-08T15:58:09.922552+0000 mgr.x (mgr.24118) 925 : cluster [DBG] pgmap v1353: 17 pgs: 17 active+clean; 6.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s wr, 1.12k op/s 2020-09-08T15:58:10.900 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:58:10 smithi104 bash[22305]: debug 2020-09-08T15:58:10.625+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:10.974 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:58:11.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:10 smithi166 bash[9446]: cluster 2020-09-08T15:58:09.922552+0000 mgr.x (mgr.24118) 925 : cluster [DBG] pgmap v1353: 17 pgs: 17 active+clean; 6.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s wr, 1.12k op/s 2020-09-08T15:58:11.462 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:58:10 smithi166 bash[14254]: debug 2020-09-08T15:58:10.866+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:11.463 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:10 smithi055 bash[9296]: cluster 2020-09-08T15:58:09.922552+0000 mgr.x (mgr.24118) 925 : cluster [DBG] pgmap v1353: 17 pgs: 17 active+clean; 6.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s wr, 1.12k op/s 2020-09-08T15:58:11.885 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:58:11.990 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:58:12.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:58:11 smithi104 bash[19922]: debug 2020-09-08T15:58:11.873+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:12.263 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:58:12.364 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:58:12.510 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:58:12.614 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:12.622 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:58:12 smithi104 bash[19922]: debug 2020-09-08T15:58:12.249+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:12.867 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:58:12 smithi166 bash[16593]: debug 2020-09-08T15:58:12.510+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:13.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:12 smithi166 bash[9446]: cluster 2020-09-08T15:58:11.923005+0000 mgr.x (mgr.24118) 926 : cluster [DBG] pgmap v1354: 17 pgs: 17 active+clean; 6.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T15:58:13.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:12 smithi055 bash[9296]: cluster 2020-09-08T15:58:11.923005+0000 mgr.x (mgr.24118) 926 : cluster [DBG] pgmap v1354: 17 pgs: 17 active+clean; 6.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T15:58:13.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:12 smithi104 bash[9203]: cluster 2020-09-08T15:58:11.923005+0000 mgr.x (mgr.24118) 926 : cluster [DBG] pgmap v1354: 17 pgs: 17 active+clean; 6.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T15:58:13.446 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:13.550 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:58:13.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:58:13 smithi104 bash[22305]: debug 2020-09-08T15:58:13.445+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:14.250 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:58:14.356 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:58:14.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:58:14 smithi104 bash[15273]: debug 2020-09-08T15:58:14.245+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:15.030 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:58:15.134 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:15.237 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:58:15 smithi104 bash[17588]: debug 2020-09-08T15:58:15.029+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:15.238 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:14 smithi104 bash[9203]: cluster 2020-09-08T15:58:13.476770+0000 osd.7 (osd.7) 107 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:15.238 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:14 smithi104 bash[9203]: cluster 2020-09-08T15:58:13.476838+0000 osd.7 (osd.7) 108 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:15.238 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:14 smithi104 bash[9203]: cluster 2020-09-08T15:58:13.923459+0000 mgr.x (mgr.24118) 927 : cluster [DBG] pgmap v1355: 17 pgs: 17 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T15:58:15.238 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:14 smithi104 bash[9203]: cluster 2020-09-08T15:58:13.988191+0000 mon.a (mon.0) 1772 : cluster [DBG] osdmap e513: 12 total, 12 up, 12 in 2020-09-08T15:58:15.362 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:15.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:14 smithi166 bash[9446]: cluster 2020-09-08T15:58:13.476770+0000 osd.7 (osd.7) 107 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:15.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:14 smithi166 bash[9446]: cluster 2020-09-08T15:58:13.476838+0000 osd.7 (osd.7) 108 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:15.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:14 smithi166 bash[9446]: cluster 2020-09-08T15:58:13.923459+0000 mgr.x (mgr.24118) 927 : cluster [DBG] pgmap v1355: 17 pgs: 17 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T15:58:15.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:14 smithi166 bash[9446]: cluster 2020-09-08T15:58:13.988191+0000 mon.a (mon.0) 1772 : cluster [DBG] osdmap e513: 12 total, 12 up, 12 in 2020-09-08T15:58:15.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:14 smithi055 bash[9296]: cluster 2020-09-08T15:58:13.476770+0000 osd.7 (osd.7) 107 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:15.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:14 smithi055 bash[9296]: cluster 2020-09-08T15:58:13.476838+0000 osd.7 (osd.7) 108 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:15.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:14 smithi055 bash[9296]: cluster 2020-09-08T15:58:13.923459+0000 mgr.x (mgr.24118) 927 : cluster [DBG] pgmap v1355: 17 pgs: 17 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T15:58:15.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:14 smithi055 bash[9296]: cluster 2020-09-08T15:58:13.988191+0000 mon.a (mon.0) 1772 : cluster [DBG] osdmap e513: 12 total, 12 up, 12 in 2020-09-08T15:58:15.467 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:58:15.628 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:58:15.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:58:15 smithi104 bash[22305]: debug 2020-09-08T15:58:15.361+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:15.732 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:58:15.744 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:58:15 smithi055 bash[11956]: debug 2020-09-08T15:58:15.629+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:15.848 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:58:15.955 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:58:15.991 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:58:15 smithi055 bash[16646]: debug 2020-09-08T15:58:15.849+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:15.992 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:15 smithi055 bash[9296]: cluster 2020-09-08T15:58:14.730505+0000 osd.0 (osd.0) 75 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:15.992 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:15 smithi055 bash[9296]: cluster 2020-09-08T15:58:14.730577+0000 osd.0 (osd.0) 76 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:15.992 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:15 smithi055 bash[9296]: cluster 2020-09-08T15:58:14.995825+0000 mon.a (mon.0) 1773 : cluster [DBG] osdmap e514: 12 total, 12 up, 12 in 2020-09-08T15:58:16.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:15 smithi104 bash[9203]: cluster 2020-09-08T15:58:14.730505+0000 osd.0 (osd.0) 75 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:16.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:15 smithi104 bash[9203]: cluster 2020-09-08T15:58:14.730577+0000 osd.0 (osd.0) 76 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:16.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:15 smithi104 bash[9203]: cluster 2020-09-08T15:58:14.995825+0000 mon.a (mon.0) 1773 : cluster [DBG] osdmap e514: 12 total, 12 up, 12 in 2020-09-08T15:58:16.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:15 smithi166 bash[9446]: cluster 2020-09-08T15:58:14.730505+0000 osd.0 (osd.0) 75 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:16.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:15 smithi166 bash[9446]: cluster 2020-09-08T15:58:14.730577+0000 osd.0 (osd.0) 76 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:16.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:15 smithi166 bash[9446]: cluster 2020-09-08T15:58:14.995825+0000 mon.a (mon.0) 1773 : cluster [DBG] osdmap e514: 12 total, 12 up, 12 in 2020-09-08T15:58:16.409 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:58:16.512 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:16.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:58:16 smithi104 bash[17588]: debug 2020-09-08T15:58:16.409+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:16.756 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:16.863 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:58:17.007 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:58:17.012 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:58:17.116 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:17.124 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:17 smithi104 bash[9203]: cluster 2020-09-08T15:58:15.923915+0000 mgr.x (mgr.24118) 928 : cluster [DBG] pgmap v1358: 17 pgs: 17 active+clean; 6.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 880 op/s 2020-09-08T15:58:17.125 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:17 smithi104 bash[9203]: audit 2020-09-08T15:58:16.812409+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? v1:172.21.15.104:0/617734284' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:58:17.125 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:58:16 smithi104 bash[22305]: debug 2020-09-08T15:58:16.749+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:17.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:17 smithi055 bash[9296]: cluster 2020-09-08T15:58:15.923915+0000 mgr.x (mgr.24118) 928 : cluster [DBG] pgmap v1358: 17 pgs: 17 active+clean; 6.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 880 op/s 2020-09-08T15:58:17.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:17 smithi055 bash[9296]: audit 2020-09-08T15:58:16.812409+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? v1:172.21.15.104:0/617734284' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:58:17.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:58:17 smithi055 bash[19003]: debug 2020-09-08T15:58:17.009+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:17.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:17 smithi166 bash[9446]: cluster 2020-09-08T15:58:15.923915+0000 mgr.x (mgr.24118) 928 : cluster [DBG] pgmap v1358: 17 pgs: 17 active+clean; 6.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 880 op/s 2020-09-08T15:58:17.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:17 smithi166 bash[9446]: audit 2020-09-08T15:58:16.812409+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? v1:172.21.15.104:0/617734284' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:58:17.925 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:18.028 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:58:18.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:58:17 smithi104 bash[22305]: debug 2020-09-08T15:58:17.925+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:18.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:18 smithi104 bash[9203]: cluster 2020-09-08T15:58:17.002093+0000 mon.a (mon.0) 1775 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:58:18.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:18 smithi104 bash[9203]: audit 2020-09-08T15:58:17.010414+0000 mon.a (mon.0) 1776 : audit [INF] from='client.? v1:172.21.15.104:0/617734284' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:58:18.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:18 smithi104 bash[9203]: cluster 2020-09-08T15:58:17.010462+0000 mon.a (mon.0) 1777 : cluster [DBG] osdmap e515: 12 total, 12 up, 12 in 2020-09-08T15:58:18.177 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:58:18.280 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:58:18.378 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:18 smithi166 bash[9446]: cluster 2020-09-08T15:58:17.002093+0000 mon.a (mon.0) 1775 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:58:18.378 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:18 smithi166 bash[9446]: audit 2020-09-08T15:58:17.010414+0000 mon.a (mon.0) 1776 : audit [INF] from='client.? v1:172.21.15.104:0/617734284' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:58:18.379 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:18 smithi166 bash[9446]: cluster 2020-09-08T15:58:17.010462+0000 mon.a (mon.0) 1777 : cluster [DBG] osdmap e515: 12 total, 12 up, 12 in 2020-09-08T15:58:18.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:18 smithi055 bash[9296]: cluster 2020-09-08T15:58:17.002093+0000 mon.a (mon.0) 1775 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:58:18.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:18 smithi055 bash[9296]: audit 2020-09-08T15:58:17.010414+0000 mon.a (mon.0) 1776 : audit [INF] from='client.? v1:172.21.15.104:0/617734284' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:58:18.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:18 smithi055 bash[9296]: cluster 2020-09-08T15:58:17.010462+0000 mon.a (mon.0) 1777 : cluster [DBG] osdmap e515: 12 total, 12 up, 12 in 2020-09-08T15:58:18.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:58:18 smithi055 bash[16646]: debug 2020-09-08T15:58:18.177+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:18.630 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:58:18.731 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:58:18.865 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:58:18.866 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:58:18.905 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:58:18.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:58:18 smithi104 bash[17588]: debug 2020-09-08T15:58:18.617+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:18.931 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:58:18.957 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:58:18.969 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:18.995 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:58:19.010 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:58:18 smithi166 bash[18960]: debug 2020-09-08T15:58:18.866+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:19.048 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:58:19.076 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:58:19.084 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:19.117 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:58:19.150 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:58:19.199 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:19.206 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:58:19.232 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:58:19.266 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:58:19.272 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:58:19.272 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:58:19.273 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:58:19.273 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:58:19.273 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:58:19.273 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:58:19.273 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:58:19.274 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:58:19.274 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:58:19.274 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:58:19.274 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:58:19.274 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:58:19.324 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:19 smithi166 bash[9446]: cluster 2020-09-08T15:58:17.924369+0000 mgr.x (mgr.24118) 929 : cluster [DBG] pgmap v1360: 17 pgs: 17 active+clean; 6.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.32k op/s 2020-09-08T15:58:19.324 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:19 smithi166 bash[9446]: audit 2020-09-08T15:58:18.839536+0000 mon.a (mon.0) 1778 : audit [DBG] from='client.? v1:172.21.15.104:0/23921061' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:58:19.325 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:58:19 smithi166 bash[11928]: debug 2020-09-08T15:58:19.082+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:19.325 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:19.363 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:19 smithi104 bash[9203]: cluster 2020-09-08T15:58:17.924369+0000 mgr.x (mgr.24118) 929 : cluster [DBG] pgmap v1360: 17 pgs: 17 active+clean; 6.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.32k op/s 2020-09-08T15:58:19.364 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:19 smithi104 bash[9203]: audit 2020-09-08T15:58:18.839536+0000 mon.a (mon.0) 1778 : audit [DBG] from='client.? v1:172.21.15.104:0/23921061' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:58:19.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:19 smithi055 bash[9296]: cluster 2020-09-08T15:58:17.924369+0000 mgr.x (mgr.24118) 929 : cluster [DBG] pgmap v1360: 17 pgs: 17 active+clean; 6.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.32k op/s 2020-09-08T15:58:19.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:19 smithi055 bash[9296]: audit 2020-09-08T15:58:18.839536+0000 mon.a (mon.0) 1778 : audit [DBG] from='client.? v1:172.21.15.104:0/23921061' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:58:19.429 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:19.564 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:19.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:58:19 smithi166 bash[14254]: debug 2020-09-08T15:58:19.322+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:19.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:58:19 smithi166 bash[14254]: debug 2020-09-08T15:58:19.562+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:19.667 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:19.810 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:19.913 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:20.037 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:20.138 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:58:20.146 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:58:19 smithi055 bash[14301]: debug 2020-09-08T15:58:19.813+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:20.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:58:20 smithi166 bash[14254]: debug 2020-09-08T15:58:20.034+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:20.404 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T15:58:20.405 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2020-09-08T15:58:20.405 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2020-09-08T15:58:20.405 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T15:58:20.405 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.406 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:90535 5.1 5:8fa10cc1:::benchmark_data_smithi166_37820_object90534:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e416)", 2020-09-08T15:58:20.406 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:52:36.655177+0000", 2020-09-08T15:58:20.406 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 343.75116222899999, 2020-09-08T15:58:20.406 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28572327400000003, 2020-09-08T15:58:20.406 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:58:20.407 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:20.407 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:58:20.407 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T15:58:20.407 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T15:58:20.408 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 90535 2020-09-08T15:58:20.408 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.408 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:58:20.408 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.409 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:58:20.409 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:36.655177+0000", 2020-09-08T15:58:20.409 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:58:20.409 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.409 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.410 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:58:20.410 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:36.655177+0000", 2020-09-08T15:58:20.410 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999614 2020-09-08T15:58:20.410 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.410 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.411 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:58:20.411 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:36.655138+0000", 2020-09-08T15:58:20.411 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.27316333999999998 2020-09-08T15:58:20.411 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.411 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.412 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:58:20.412 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:36.928301+0000", 2020-09-08T15:58:20.412 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8221999999999999e-05 2020-09-08T15:58:20.412 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.412 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.413 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:58:20.413 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:36.928319+0000", 2020-09-08T15:58:20.413 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9729999999999997e-06 2020-09-08T15:58:20.413 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.414 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.414 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:58:20.414 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:36.928327+0000", 2020-09-08T15:58:20.415 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016297699999999999 2020-09-08T15:58:20.415 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.415 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.415 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:58:20.416 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:36.928490+0000", 2020-09-08T15:58:20.416 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00031209899999999998 2020-09-08T15:58:20.416 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.417 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.418 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:58:20.418 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:36.928803+0000", 2020-09-08T15:58:20.419 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00033797299999999998 2020-09-08T15:58:20.419 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.419 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.420 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T15:58:20.420 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:36.929140+0000", 2020-09-08T15:58:20.420 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0085641139999999994 2020-09-08T15:58:20.421 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.421 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.421 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:58:20.422 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:36.937705+0000", 2020-09-08T15:58:20.422 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0030520629999999998 2020-09-08T15:58:20.422 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.423 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.423 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:58:20.423 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:36.940757+0000", 2020-09-08T15:58:20.424 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5643000000000007e-05 2020-09-08T15:58:20.424 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.424 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.425 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:58:20.425 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:36.940842+0000", 2020-09-08T15:58:20.425 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7587999999999998e-05 2020-09-08T15:58:20.426 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.426 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.426 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:58:20.427 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:36.940900+0000", 2020-09-08T15:58:20.427 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28572327400000003 2020-09-08T15:58:20.427 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.428 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.428 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.428 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.429 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.429 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:72177 6.c 6:32f17896:::benchmark_data_smithi166_43187_object72176:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T15:58:20.429 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:18.012410+0000", 2020-09-08T15:58:20.430 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 62.393928901999999, 2020-09-08T15:58:20.430 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21640279600000001, 2020-09-08T15:58:20.431 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:58:20.432 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:20.432 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:58:20.432 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:58:20.433 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:20.433 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 72177 2020-09-08T15:58:20.433 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.434 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:58:20.434 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.434 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:58:20.435 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.012410+0000", 2020-09-08T15:58:20.435 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:58:20.435 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.436 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.436 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:58:20.436 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.012410+0000", 2020-09-08T15:58:20.437 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999523 2020-09-08T15:58:20.437 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.438 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.438 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:58:20.438 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.012362+0000", 2020-09-08T15:58:20.439 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000178543 2020-09-08T15:58:20.439 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.439 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.440 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:58:20.440 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.012541+0000", 2020-09-08T15:58:20.440 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7224999999999999e-05 2020-09-08T15:58:20.441 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.441 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.441 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:58:20.442 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.012558+0000", 2020-09-08T15:58:20.442 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.888e-06 2020-09-08T15:58:20.443 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.443 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.443 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:58:20.443 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.012565+0000", 2020-09-08T15:58:20.444 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00023556200000000001 2020-09-08T15:58:20.444 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.445 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.445 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:58:20.445 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.012801+0000", 2020-09-08T15:58:20.446 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012632100000000001 2020-09-08T15:58:20.446 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.446 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.447 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:58:20.447 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.012927+0000", 2020-09-08T15:58:20.447 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012829599999999999 2020-09-08T15:58:20.448 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.448 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.448 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 6", 2020-09-08T15:58:20.449 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.013055+0000", 2020-09-08T15:58:20.449 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0050656 2020-09-08T15:58:20.449 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.450 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.450 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:58:20.451 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.018121+0000", 2020-09-08T15:58:20.451 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21047006400000001 2020-09-08T15:58:20.452 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.452 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.452 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:58:20.453 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.228591+0000", 2020-09-08T15:58:20.453 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000134473 2020-09-08T15:58:20.453 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.454 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.454 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:58:20.454 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.228725+0000", 2020-09-08T15:58:20.454 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7460999999999995e-05 2020-09-08T15:58:20.455 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.455 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.455 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:58:20.455 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.228813+0000", 2020-09-08T15:58:20.455 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21640279600000001 2020-09-08T15:58:20.455 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.456 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.456 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.456 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.456 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.456 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:72179 6.c 6:3d9d0e03:::benchmark_data_smithi166_43187_object72178:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T15:58:20.457 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:18.012644+0000", 2020-09-08T15:58:20.457 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 62.393694623999998, 2020-09-08T15:58:20.457 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21651737600000001, 2020-09-08T15:58:20.457 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:58:20.457 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:20.458 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:58:20.458 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:58:20.458 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:20.458 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 72179 2020-09-08T15:58:20.458 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.459 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:58:20.459 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.459 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:58:20.459 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.012644+0000", 2020-09-08T15:58:20.459 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:58:20.460 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.460 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.460 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:58:20.460 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.012644+0000", 2020-09-08T15:58:20.460 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999609 2020-09-08T15:58:20.461 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.461 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.461 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:58:20.461 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.012605+0000", 2020-09-08T15:58:20.461 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000115215 2020-09-08T15:58:20.462 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.462 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.462 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:58:20.462 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.012720+0000", 2020-09-08T15:58:20.462 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1569000000000001e-05 2020-09-08T15:58:20.463 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.463 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.463 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:58:20.463 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.012732+0000", 2020-09-08T15:58:20.463 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8040000000000001e-06 2020-09-08T15:58:20.463 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.464 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.464 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:58:20.464 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.012737+0000", 2020-09-08T15:58:20.464 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0028404950000000002 2020-09-08T15:58:20.464 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.465 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.465 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:58:20.465 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.015577+0000", 2020-09-08T15:58:20.465 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014480499999999999 2020-09-08T15:58:20.465 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.466 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.466 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:58:20.466 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.015722+0000", 2020-09-08T15:58:20.466 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000163382 2020-09-08T15:58:20.466 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.467 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.467 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 6", 2020-09-08T15:58:20.467 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.015886+0000", 2020-09-08T15:58:20.467 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0029570730000000002 2020-09-08T15:58:20.467 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.468 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.468 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:58:20.468 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.018843+0000", 2020-09-08T15:58:20.468 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21006517 2020-09-08T15:58:20.468 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.468 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.469 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:58:20.469 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.228908+0000", 2020-09-08T15:58:20.469 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000151762 2020-09-08T15:58:20.469 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.469 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.470 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:58:20.470 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.229060+0000", 2020-09-08T15:58:20.470 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010206500000000001 2020-09-08T15:58:20.470 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.470 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.471 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:58:20.471 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.229162+0000", 2020-09-08T15:58:20.471 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21651737600000001 2020-09-08T15:58:20.471 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.471 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.472 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.472 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.472 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.472 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:72218 6.c 6:3df7783a:::benchmark_data_smithi166_43187_object72217:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T15:58:20.472 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:18.044139+0000", 2020-09-08T15:58:20.473 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 62.362199926999999, 2020-09-08T15:58:20.473 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.187659363, 2020-09-08T15:58:20.473 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:58:20.473 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:20.473 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:58:20.474 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:58:20.475 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:20.475 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 72218 2020-09-08T15:58:20.475 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.475 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:58:20.476 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.476 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:58:20.476 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.044139+0000", 2020-09-08T15:58:20.476 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:58:20.476 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.476 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.477 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:58:20.477 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.044139+0000", 2020-09-08T15:58:20.477 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999671 2020-09-08T15:58:20.477 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.477 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.478 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:58:20.478 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.044106+0000", 2020-09-08T15:58:20.478 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013476800000000001 2020-09-08T15:58:20.478 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.478 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.479 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:58:20.479 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.044241+0000", 2020-09-08T15:58:20.479 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4687e-05 2020-09-08T15:58:20.479 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.479 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.480 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:58:20.480 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.044255+0000", 2020-09-08T15:58:20.480 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6380000000000001e-06 2020-09-08T15:58:20.480 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.481 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.481 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:58:20.481 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.044261+0000", 2020-09-08T15:58:20.481 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3940000000000003e-05 2020-09-08T15:58:20.481 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.482 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.482 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:58:20.482 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.044305+0000", 2020-09-08T15:58:20.482 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000110208 2020-09-08T15:58:20.482 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.483 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.483 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:58:20.483 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.044415+0000", 2020-09-08T15:58:20.483 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017219699999999999 2020-09-08T15:58:20.483 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.484 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.484 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 6", 2020-09-08T15:58:20.484 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.044587+0000", 2020-09-08T15:58:20.484 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0088313599999999999 2020-09-08T15:58:20.484 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.485 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.485 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:58:20.485 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.053419+0000", 2020-09-08T15:58:20.485 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17816673299999999 2020-09-08T15:58:20.486 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.486 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.486 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:58:20.486 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.231586+0000", 2020-09-08T15:58:20.486 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000117106 2020-09-08T15:58:20.487 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.487 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.487 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:58:20.487 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.231703+0000", 2020-09-08T15:58:20.487 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5631000000000006e-05 2020-09-08T15:58:20.488 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.488 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.488 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:58:20.488 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.231798+0000", 2020-09-08T15:58:20.488 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.187659363 2020-09-08T15:58:20.488 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.489 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.489 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.489 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.489 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.489 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:72231 6.c 6:39a73660:::benchmark_data_smithi166_43187_object72230:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T15:58:20.495 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:18.050269+0000", 2020-09-08T15:58:20.495 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 62.356069482999999, 2020-09-08T15:58:20.496 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18188580800000001, 2020-09-08T15:58:20.496 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:58:20.496 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:20.496 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:58:20.496 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:58:20.497 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:20.497 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 72231 2020-09-08T15:58:20.497 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.497 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:58:20.497 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.498 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:58:20.498 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.050269+0000", 2020-09-08T15:58:20.498 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:58:20.498 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.499 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.499 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:58:20.499 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.050269+0000", 2020-09-08T15:58:20.499 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999585 2020-09-08T15:58:20.499 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.500 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.500 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:58:20.500 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.050228+0000", 2020-09-08T15:58:20.500 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014753999999999999 2020-09-08T15:58:20.500 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.501 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.501 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:58:20.501 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.050375+0000", 2020-09-08T15:58:20.501 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6684000000000002e-05 2020-09-08T15:58:20.502 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.502 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.502 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:58:20.502 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.050392+0000", 2020-09-08T15:58:20.502 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0870000000000001e-06 2020-09-08T15:58:20.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.503 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.503 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:58:20.503 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.050399+0000", 2020-09-08T15:58:20.503 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9597000000000001e-05 2020-09-08T15:58:20.504 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.504 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.504 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:58:20.504 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.050439+0000", 2020-09-08T15:58:20.504 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015532 2020-09-08T15:58:20.505 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.505 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.505 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:58:20.505 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.050594+0000", 2020-09-08T15:58:20.505 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015173300000000001 2020-09-08T15:58:20.506 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.506 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.506 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 6", 2020-09-08T15:58:20.506 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.050746+0000", 2020-09-08T15:58:20.506 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0029695300000000002 2020-09-08T15:58:20.507 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.507 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.507 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:58:20.507 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.053715+0000", 2020-09-08T15:58:20.507 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.178184283 2020-09-08T15:58:20.508 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.508 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.508 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:58:20.508 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.231900+0000", 2020-09-08T15:58:20.508 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0001516 2020-09-08T15:58:20.509 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.509 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.509 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:58:20.509 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.232051+0000", 2020-09-08T15:58:20.509 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000104078 2020-09-08T15:58:20.510 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.510 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.510 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:58:20.510 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.232155+0000", 2020-09-08T15:58:20.510 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18188580800000001 2020-09-08T15:58:20.510 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.511 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.511 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.511 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.511 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.511 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:72236 6.c 6:36eb20c6:::benchmark_data_smithi166_43187_object72235:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T15:58:20.512 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:18.055129+0000", 2020-09-08T15:58:20.512 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 62.351209638, 2020-09-08T15:58:20.512 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18000761900000001, 2020-09-08T15:58:20.512 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:58:20.512 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:20.513 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:58:20.513 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:58:20.513 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:20.513 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 72236 2020-09-08T15:58:20.513 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.514 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:58:20.514 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.514 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:58:20.514 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.055129+0000", 2020-09-08T15:58:20.514 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:58:20.514 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.515 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.515 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:58:20.515 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.055129+0000", 2020-09-08T15:58:20.515 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999714 2020-09-08T15:58:20.515 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.516 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.516 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:58:20.516 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.055100+0000", 2020-09-08T15:58:20.516 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7096999999999995e-05 2020-09-08T15:58:20.516 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.517 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.517 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:58:20.517 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.055198+0000", 2020-09-08T15:58:20.517 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2496e-05 2020-09-08T15:58:20.517 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.518 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:58:20.518 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.055210+0000", 2020-09-08T15:58:20.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6290000000000004e-06 2020-09-08T15:58:20.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:58:20.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.055217+0000", 2020-09-08T15:58:20.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6152999999999998e-05 2020-09-08T15:58:20.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:58:20.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.055253+0000", 2020-09-08T15:58:20.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000119354 2020-09-08T15:58:20.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:58:20.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.055372+0000", 2020-09-08T15:58:20.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000128909 2020-09-08T15:58:20.521 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.522 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 6", 2020-09-08T15:58:20.522 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.055501+0000", 2020-09-08T15:58:20.522 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.005196862 2020-09-08T15:58:20.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:58:20.524 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.060698+0000", 2020-09-08T15:58:20.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.174176725 2020-09-08T15:58:20.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:58:20.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.234875+0000", 2020-09-08T15:58:20.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000147353 2020-09-08T15:58:20.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.527 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:58:20.527 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.235022+0000", 2020-09-08T15:58:20.527 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000114839 2020-09-08T15:58:20.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.528 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:58:20.529 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.235137+0000", 2020-09-08T15:58:20.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18000761900000001 2020-09-08T15:58:20.529 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.530 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.530 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.531 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:72238 6.c 6:353d834a:::benchmark_data_smithi166_43187_object72237:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T15:58:20.531 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:18.055516+0000", 2020-09-08T15:58:20.532 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 62.350822415000003, 2020-09-08T15:58:20.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.179816858, 2020-09-08T15:58:20.533 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:58:20.533 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:20.533 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:58:20.534 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:58:20.534 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:20.534 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 72238 2020-09-08T15:58:20.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.535 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:58:20.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.536 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:58:20.536 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.055516+0000", 2020-09-08T15:58:20.536 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:58:20.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.537 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:58:20.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.055516+0000", 2020-09-08T15:58:20.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999676 2020-09-08T15:58:20.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.539 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:58:20.539 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.055484+0000", 2020-09-08T15:58:20.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000111263 2020-09-08T15:58:20.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:58:20.541 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.055595+0000", 2020-09-08T15:58:20.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3971999999999999e-05 2020-09-08T15:58:20.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:58:20.543 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.055609+0000", 2020-09-08T15:58:20.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.755e-06 2020-09-08T15:58:20.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:58:20.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.055615+0000", 2020-09-08T15:58:20.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002256231 2020-09-08T15:58:20.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.546 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:58:20.546 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.057871+0000", 2020-09-08T15:58:20.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016386800000000001 2020-09-08T15:58:20.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:58:20.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.058035+0000", 2020-09-08T15:58:20.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000177837 2020-09-08T15:58:20.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 6", 2020-09-08T15:58:20.550 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.058213+0000", 2020-09-08T15:58:20.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0047093370000000001 2020-09-08T15:58:20.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.551 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.551 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:58:20.551 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.062922+0000", 2020-09-08T15:58:20.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17227669900000001 2020-09-08T15:58:20.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.553 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:58:20.553 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.235199+0000", 2020-09-08T15:58:20.554 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7558999999999994e-05 2020-09-08T15:58:20.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.555 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:58:20.555 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.235277+0000", 2020-09-08T15:58:20.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6637e-05 2020-09-08T15:58:20.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.556 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.556 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:58:20.557 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.235333+0000", 2020-09-08T15:58:20.557 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.179816858 2020-09-08T15:58:20.557 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.558 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.558 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.559 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:72242 6.c 6:3c645502:::benchmark_data_smithi166_43187_object72241:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T15:58:20.559 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:18.057203+0000", 2020-09-08T15:58:20.560 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 62.349135848000003, 2020-09-08T15:58:20.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17942830600000001, 2020-09-08T15:58:20.560 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:58:20.561 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:20.561 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:58:20.561 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:58:20.562 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:20.562 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 72242 2020-09-08T15:58:20.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.563 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:58:20.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:58:20.564 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.057203+0000", 2020-09-08T15:58:20.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:58:20.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.565 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.565 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:58:20.565 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.057203+0000", 2020-09-08T15:58:20.566 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999561 2020-09-08T15:58:20.566 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.566 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.567 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:58:20.567 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.057159+0000", 2020-09-08T15:58:20.568 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000108664 2020-09-08T15:58:20.568 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.568 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.569 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:58:20.569 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.057268+0000", 2020-09-08T15:58:20.571 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1946999999999999e-05 2020-09-08T15:58:20.571 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.571 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.572 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:58:20.572 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.057280+0000", 2020-09-08T15:58:20.572 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5240000000000002e-06 2020-09-08T15:58:20.572 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.572 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.573 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:58:20.573 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.057284+0000", 2020-09-08T15:58:20.573 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0034408939999999999 2020-09-08T15:58:20.573 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.574 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:58:20.574 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.060725+0000", 2020-09-08T15:58:20.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012345800000000001 2020-09-08T15:58:20.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:58:20.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.060849+0000", 2020-09-08T15:58:20.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013632499999999999 2020-09-08T15:58:20.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.576 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 6", 2020-09-08T15:58:20.576 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.060985+0000", 2020-09-08T15:58:20.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0021832290000000001 2020-09-08T15:58:20.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:58:20.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.063168+0000", 2020-09-08T15:58:20.578 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17333683899999999 2020-09-08T15:58:20.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.578 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.578 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:58:20.578 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.236505+0000", 2020-09-08T15:58:20.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7646999999999998e-05 2020-09-08T15:58:20.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.579 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.579 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:58:20.579 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.236573+0000", 2020-09-08T15:58:20.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8615e-05 2020-09-08T15:58:20.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:58:20.581 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.236631+0000", 2020-09-08T15:58:20.581 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17942830600000001 2020-09-08T15:58:20.581 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.581 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.581 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.582 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:72251 6.c 6:3a6eada6:::benchmark_data_smithi166_43187_object72250:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T15:58:20.582 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:18.063491+0000", 2020-09-08T15:58:20.582 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 62.342848314999998, 2020-09-08T15:58:20.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17353454400000001, 2020-09-08T15:58:20.583 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:58:20.583 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:20.583 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:58:20.584 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:58:20.584 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:20.584 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 72251 2020-09-08T15:58:20.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.584 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:58:20.585 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.585 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:58:20.585 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.063491+0000", 2020-09-08T15:58:20.585 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:58:20.585 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.586 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:58:20.586 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.063491+0000", 2020-09-08T15:58:20.586 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999504 2020-09-08T15:58:20.587 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.587 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.587 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:58:20.587 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.063441+0000", 2020-09-08T15:58:20.587 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014486100000000001 2020-09-08T15:58:20.588 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.588 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.588 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:58:20.588 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.063586+0000", 2020-09-08T15:58:20.588 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9366999999999999e-05 2020-09-08T15:58:20.589 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.589 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.589 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:58:20.589 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.063605+0000", 2020-09-08T15:58:20.590 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4969999999999998e-06 2020-09-08T15:58:20.590 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.590 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.590 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:58:20.590 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.063612+0000", 2020-09-08T15:58:20.591 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3646999999999997e-05 2020-09-08T15:58:20.591 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.591 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.591 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:58:20.591 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.063655+0000", 2020-09-08T15:58:20.592 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015382300000000001 2020-09-08T15:58:20.592 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.592 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.592 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:58:20.592 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.063809+0000", 2020-09-08T15:58:20.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000187968 2020-09-08T15:58:20.593 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.593 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 6", 2020-09-08T15:58:20.593 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.063997+0000", 2020-09-08T15:58:20.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0025670929999999999 2020-09-08T15:58:20.594 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.594 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:58:20.595 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.066564+0000", 2020-09-08T15:58:20.595 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17032741400000001 2020-09-08T15:58:20.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.595 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.595 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:58:20.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.236892+0000", 2020-09-08T15:58:20.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8938999999999998e-05 2020-09-08T15:58:20.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.597 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:58:20.597 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.236961+0000", 2020-09-08T15:58:20.597 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4369000000000007e-05 2020-09-08T15:58:20.597 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.598 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:58:20.598 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.237025+0000", 2020-09-08T15:58:20.598 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17353454400000001 2020-09-08T15:58:20.598 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.598 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.599 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.599 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.599 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:72266 6.c 6:3dfffac6:::benchmark_data_smithi166_43187_object72265:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T15:58:20.599 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:18.074838+0000", 2020-09-08T15:58:20.600 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 62.331500454999997, 2020-09-08T15:58:20.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.170837563, 2020-09-08T15:58:20.600 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:58:20.600 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:20.600 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:58:20.601 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:58:20.601 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:20.601 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 72266 2020-09-08T15:58:20.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.602 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:58:20.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:58:20.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.074838+0000", 2020-09-08T15:58:20.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:58:20.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:58:20.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.074838+0000", 2020-09-08T15:58:20.604 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999948 2020-09-08T15:58:20.604 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:58:20.604 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.074786+0000", 2020-09-08T15:58:20.605 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014118799999999999 2020-09-08T15:58:20.605 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.605 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.605 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:58:20.605 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.074927+0000", 2020-09-08T15:58:20.606 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3662000000000001e-05 2020-09-08T15:58:20.606 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.606 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:58:20.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.074941+0000", 2020-09-08T15:58:20.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.271e-06 2020-09-08T15:58:20.607 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:58:20.608 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.074946+0000", 2020-09-08T15:58:20.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2519999999999999e-05 2020-09-08T15:58:20.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.609 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.609 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:58:20.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.074999+0000", 2020-09-08T15:58:20.610 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015573200000000001 2020-09-08T15:58:20.610 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.610 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.610 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:58:20.611 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.075155+0000", 2020-09-08T15:58:20.611 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016914600000000001 2020-09-08T15:58:20.611 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.611 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.612 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 6", 2020-09-08T15:58:20.612 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.075324+0000", 2020-09-08T15:58:20.612 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0050578389999999997 2020-09-08T15:58:20.612 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.613 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.613 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:58:20.613 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.080382+0000", 2020-09-08T15:58:20.613 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.164917701 2020-09-08T15:58:20.613 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.614 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:58:20.614 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.245299+0000", 2020-09-08T15:58:20.614 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000209767 2020-09-08T15:58:20.614 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.615 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:58:20.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.245509+0000", 2020-09-08T15:58:20.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000166935 2020-09-08T15:58:20.616 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.616 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.616 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:58:20.616 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.245676+0000", 2020-09-08T15:58:20.616 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.170837563 2020-09-08T15:58:20.617 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.617 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.617 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.617 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.617 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.618 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:72267 6.c 6:32bd220e:::benchmark_data_smithi166_43187_object72266:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T15:58:20.618 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:18.077079+0000", 2020-09-08T15:58:20.618 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 62.329259575000002, 2020-09-08T15:58:20.618 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17094072099999999, 2020-09-08T15:58:20.618 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:58:20.619 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:20.619 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:58:20.619 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:58:20.619 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:20.620 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 72267 2020-09-08T15:58:20.620 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.620 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:58:20.620 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.620 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:58:20.621 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.077079+0000", 2020-09-08T15:58:20.621 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:58:20.621 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.621 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.621 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:58:20.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.077079+0000", 2020-09-08T15:58:20.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999657 2020-09-08T15:58:20.622 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:58:20.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.077045+0000", 2020-09-08T15:58:20.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000135675 2020-09-08T15:58:20.623 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.623 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.623 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:58:20.624 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.077181+0000", 2020-09-08T15:58:20.624 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3234999999999999e-05 2020-09-08T15:58:20.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.624 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:58:20.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.077194+0000", 2020-09-08T15:58:20.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9689999999999999e-06 2020-09-08T15:58:20.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:58:20.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.077200+0000", 2020-09-08T15:58:20.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00058339899999999996 2020-09-08T15:58:20.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.627 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:58:20.627 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.077783+0000", 2020-09-08T15:58:20.627 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00021581200000000001 2020-09-08T15:58:20.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.628 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:58:20.628 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.077999+0000", 2020-09-08T15:58:20.628 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000212544 2020-09-08T15:58:20.628 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.629 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 6", 2020-09-08T15:58:20.629 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.078212+0000", 2020-09-08T15:58:20.629 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0024595590000000001 2020-09-08T15:58:20.629 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.630 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.630 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:58:20.630 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.080671+0000", 2020-09-08T15:58:20.630 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16718254199999999 2020-09-08T15:58:20.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.631 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.631 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:58:20.631 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.247854+0000", 2020-09-08T15:58:20.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000121328 2020-09-08T15:58:20.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.632 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.632 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:58:20.632 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.247975+0000", 2020-09-08T15:58:20.632 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4815999999999999e-05 2020-09-08T15:58:20.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.633 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:58:20.633 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:18.248020+0000", 2020-09-08T15:58:20.633 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17094072099999999 2020-09-08T15:58:20.634 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.634 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.636 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.636 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.637 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:78115 6.c 6:3ead95db:::benchmark_data_smithi166_43187_object78114:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T15:58:20.638 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:22.626264+0000", 2020-09-08T15:58:20.639 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 57.780075287000003, 2020-09-08T15:58:20.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.38050951999999999, 2020-09-08T15:58:20.639 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:58:20.640 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:20.640 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:58:20.641 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:58:20.641 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:20.641 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 78115 2020-09-08T15:58:20.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.642 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:58:20.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:58:20.643 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.626264+0000", 2020-09-08T15:58:20.643 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:58:20.643 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.643 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.643 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:58:20.644 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.626264+0000", 2020-09-08T15:58:20.644 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999619 2020-09-08T15:58:20.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.645 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:58:20.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.626225+0000", 2020-09-08T15:58:20.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013275799999999999 2020-09-08T15:58:20.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.646 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:58:20.646 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.626358+0000", 2020-09-08T15:58:20.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4022000000000001e-05 2020-09-08T15:58:20.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:58:20.647 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.626372+0000", 2020-09-08T15:58:20.648 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8830000000000003e-06 2020-09-08T15:58:20.648 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.648 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.648 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:58:20.649 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.626378+0000", 2020-09-08T15:58:20.649 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1301000000000001e-05 2020-09-08T15:58:20.649 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.649 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.649 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:58:20.650 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.626439+0000", 2020-09-08T15:58:20.650 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000136525 2020-09-08T15:58:20.650 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.650 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.650 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:58:20.651 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.626576+0000", 2020-09-08T15:58:20.651 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000149753 2020-09-08T15:58:20.651 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.651 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.651 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 6", 2020-09-08T15:58:20.652 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.626725+0000", 2020-09-08T15:58:20.652 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003455624 2020-09-08T15:58:20.652 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.652 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.652 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:58:20.653 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.630181+0000", 2020-09-08T15:58:20.653 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.37615404499999999 2020-09-08T15:58:20.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.653 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.653 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:58:20.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.006335+0000", 2020-09-08T15:58:20.654 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000107513 2020-09-08T15:58:20.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:58:20.655 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.006443+0000", 2020-09-08T15:58:20.655 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000330441 2020-09-08T15:58:20.655 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.655 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.655 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:58:20.655 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.006773+0000", 2020-09-08T15:58:20.656 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.38050951999999999 2020-09-08T15:58:20.656 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.656 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.656 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.657 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.657 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:78132 6.c 6:3637c683:::benchmark_data_smithi166_43187_object78131:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T15:58:20.657 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:22.637468+0000", 2020-09-08T15:58:20.657 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 57.768870792000001, 2020-09-08T15:58:20.657 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.369461439, 2020-09-08T15:58:20.658 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:58:20.658 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:20.658 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:58:20.658 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:58:20.658 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:20.659 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 78132 2020-09-08T15:58:20.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.659 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:58:20.659 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.659 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:58:20.659 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.637468+0000", 2020-09-08T15:58:20.660 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:58:20.660 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.660 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.660 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:58:20.660 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.637468+0000", 2020-09-08T15:58:20.661 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999747 2020-09-08T15:58:20.661 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.661 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.661 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:58:20.661 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.637443+0000", 2020-09-08T15:58:20.662 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9765000000000005e-05 2020-09-08T15:58:20.662 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.662 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.662 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:58:20.662 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.637542+0000", 2020-09-08T15:58:20.663 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7122999999999999e-05 2020-09-08T15:58:20.663 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.663 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.663 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:58:20.663 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.637560+0000", 2020-09-08T15:58:20.663 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7080000000000002e-06 2020-09-08T15:58:20.664 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.664 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.664 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:58:20.664 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.637565+0000", 2020-09-08T15:58:20.664 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2513000000000003e-05 2020-09-08T15:58:20.665 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.665 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.665 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:58:20.665 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.637628+0000", 2020-09-08T15:58:20.665 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016613000000000001 2020-09-08T15:58:20.666 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.666 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.666 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:58:20.666 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.637794+0000", 2020-09-08T15:58:20.666 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000171368 2020-09-08T15:58:20.667 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.667 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.667 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 6", 2020-09-08T15:58:20.667 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.637965+0000", 2020-09-08T15:58:20.667 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0031642749999999998 2020-09-08T15:58:20.668 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.668 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.668 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:58:20.668 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.641130+0000", 2020-09-08T15:58:20.668 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.36571284199999998 2020-09-08T15:58:20.668 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.669 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.669 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:58:20.669 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.006842+0000", 2020-09-08T15:58:20.669 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9119000000000002e-05 2020-09-08T15:58:20.669 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.670 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.670 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:58:20.670 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.006891+0000", 2020-09-08T15:58:20.670 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8006999999999997e-05 2020-09-08T15:58:20.670 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.671 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.671 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:58:20.671 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.006930+0000", 2020-09-08T15:58:20.671 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.369461439 2020-09-08T15:58:20.671 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.672 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.672 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.672 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.672 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.672 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:78147 6.c 6:36bcf7ce:::benchmark_data_smithi166_43187_object78146:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T15:58:20.673 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:22.657886+0000", 2020-09-08T15:58:20.673 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 57.748452823999997, 2020-09-08T15:58:20.673 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.34918189100000002, 2020-09-08T15:58:20.673 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:58:20.673 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:20.674 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:58:20.674 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:58:20.674 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:20.674 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 78147 2020-09-08T15:58:20.674 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.674 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:58:20.675 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.675 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:58:20.675 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.657886+0000", 2020-09-08T15:58:20.675 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:58:20.675 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.676 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.676 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:58:20.676 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.657886+0000", 2020-09-08T15:58:20.676 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999385 2020-09-08T15:58:20.676 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.677 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.677 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:58:20.677 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.657825+0000", 2020-09-08T15:58:20.677 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020615799999999999 2020-09-08T15:58:20.677 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.678 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.678 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:58:20.678 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.658031+0000", 2020-09-08T15:58:20.678 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.5831999999999999e-05 2020-09-08T15:58:20.679 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.679 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.679 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:58:20.679 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.658057+0000", 2020-09-08T15:58:20.679 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.272e-05 2020-09-08T15:58:20.680 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.680 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.680 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:58:20.680 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.658069+0000", 2020-09-08T15:58:20.680 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3972000000000003e-05 2020-09-08T15:58:20.680 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.681 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.681 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:58:20.681 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.658133+0000", 2020-09-08T15:58:20.681 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00021289799999999999 2020-09-08T15:58:20.681 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.682 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.682 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:58:20.682 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.658346+0000", 2020-09-08T15:58:20.682 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00026123999999999999 2020-09-08T15:58:20.682 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.683 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.683 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 6", 2020-09-08T15:58:20.683 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.658608+0000", 2020-09-08T15:58:20.683 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0030369569999999999 2020-09-08T15:58:20.683 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.684 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.684 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:58:20.684 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.661645+0000", 2020-09-08T15:58:20.684 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.34534172800000001 2020-09-08T15:58:20.684 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.684 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.685 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:58:20.685 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.006986+0000", 2020-09-08T15:58:20.685 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5206000000000001e-05 2020-09-08T15:58:20.685 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.685 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.686 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:58:20.686 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.007031+0000", 2020-09-08T15:58:20.686 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6473000000000001e-05 2020-09-08T15:58:20.686 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.686 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.687 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:58:20.687 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.007068+0000", 2020-09-08T15:58:20.687 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.34918189100000002 2020-09-08T15:58:20.687 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.687 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.688 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.688 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.688 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.688 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:78150 6.c 6:39955be7:::benchmark_data_smithi166_43187_object78149:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T15:58:20.688 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:22.662795+0000", 2020-09-08T15:58:20.689 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 57.743543613999996, 2020-09-08T15:58:20.689 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.34442853400000001, 2020-09-08T15:58:20.689 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:58:20.689 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:20.689 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:58:20.690 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:58:20.690 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:20.690 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 78150 2020-09-08T15:58:20.690 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.690 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:58:20.690 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.691 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:58:20.691 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.662795+0000", 2020-09-08T15:58:20.691 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:58:20.691 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.691 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.692 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:58:20.692 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.662795+0000", 2020-09-08T15:58:20.692 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999352 2020-09-08T15:58:20.692 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.692 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.693 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:58:20.693 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.662731+0000", 2020-09-08T15:58:20.693 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00022093699999999999 2020-09-08T15:58:20.693 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.693 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.694 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:58:20.694 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.662951+0000", 2020-09-08T15:58:20.694 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9011000000000001e-05 2020-09-08T15:58:20.700 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.700 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.700 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:58:20.700 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.662980+0000", 2020-09-08T15:58:20.700 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2734999999999999e-05 2020-09-08T15:58:20.701 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.701 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.701 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:58:20.701 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.662993+0000", 2020-09-08T15:58:20.701 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3434000000000003e-05 2020-09-08T15:58:20.702 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.702 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.702 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:58:20.702 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.663057+0000", 2020-09-08T15:58:20.702 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018343499999999999 2020-09-08T15:58:20.703 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.703 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.703 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:58:20.703 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.663240+0000", 2020-09-08T15:58:20.703 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000212444 2020-09-08T15:58:20.704 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.704 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.704 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 6", 2020-09-08T15:58:20.705 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.663453+0000", 2020-09-08T15:58:20.705 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0034880219999999999 2020-09-08T15:58:20.705 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.705 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.705 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:58:20.705 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:22.666941+0000", 2020-09-08T15:58:20.706 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.340205123 2020-09-08T15:58:20.706 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.706 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.706 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:58:20.706 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.007146+0000", 2020-09-08T15:58:20.707 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3065000000000002e-05 2020-09-08T15:58:20.707 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.707 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.707 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:58:20.707 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.007189+0000", 2020-09-08T15:58:20.708 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5021000000000002e-05 2020-09-08T15:58:20.708 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.708 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.708 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:58:20.708 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.007224+0000", 2020-09-08T15:58:20.709 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.34442853400000001 2020-09-08T15:58:20.709 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.709 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.709 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.709 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.710 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.710 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:104049 6.2 6:44345000:::benchmark_data_smithi166_43187_object104048:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e506)", 2020-09-08T15:58:20.710 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:44.209472+0000", 2020-09-08T15:58:20.710 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 36.196866647, 2020-09-08T15:58:20.710 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18045304400000001, 2020-09-08T15:58:20.711 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:58:20.711 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:20.711 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:58:20.711 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:58:20.711 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:20.711 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 104049 2020-09-08T15:58:20.712 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.712 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:58:20.712 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.712 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:58:20.712 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:44.209472+0000", 2020-09-08T15:58:20.713 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:58:20.713 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.713 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.713 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:58:20.713 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:44.209472+0000", 2020-09-08T15:58:20.714 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999733 2020-09-08T15:58:20.714 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.714 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.714 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:58:20.714 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:44.209446+0000", 2020-09-08T15:58:20.715 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000110222 2020-09-08T15:58:20.715 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.715 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.715 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:58:20.715 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:44.209556+0000", 2020-09-08T15:58:20.716 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3857e-05 2020-09-08T15:58:20.716 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.716 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.716 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:58:20.716 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:44.209570+0000", 2020-09-08T15:58:20.716 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1449999999999999e-06 2020-09-08T15:58:20.717 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.717 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.717 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:58:20.717 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:44.209575+0000", 2020-09-08T15:58:20.717 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003706277 2020-09-08T15:58:20.718 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.718 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.718 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:58:20.718 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:44.213281+0000", 2020-09-08T15:58:20.718 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000168025 2020-09-08T15:58:20.719 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.719 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.719 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:58:20.719 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:44.213449+0000", 2020-09-08T15:58:20.719 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015134799999999999 2020-09-08T15:58:20.719 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.720 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.720 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T15:58:20.720 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:44.213601+0000", 2020-09-08T15:58:20.720 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.012632661 2020-09-08T15:58:20.721 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.722 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.722 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:58:20.722 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:44.226233+0000", 2020-09-08T15:58:20.723 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.163323469 2020-09-08T15:58:20.723 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.723 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.723 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:58:20.723 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:44.389557+0000", 2020-09-08T15:58:20.724 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00022580699999999999 2020-09-08T15:58:20.724 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.724 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.724 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:58:20.724 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:44.389782+0000", 2020-09-08T15:58:20.725 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014281100000000001 2020-09-08T15:58:20.725 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.725 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.725 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:58:20.725 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:44.389925+0000", 2020-09-08T15:58:20.726 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18045304400000001 2020-09-08T15:58:20.726 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.726 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.726 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.726 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.727 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.727 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:116602 6.b 6:db98ed45:::benchmark_data_smithi166_43187_object116601:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e508)", 2020-09-08T15:58:20.727 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:55.451397+0000", 2020-09-08T15:58:20.727 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 24.954941857000001, 2020-09-08T15:58:20.727 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.19276940200000001, 2020-09-08T15:58:20.727 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:58:20.728 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:20.728 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:58:20.728 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:58:20.728 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:20.728 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 116602 2020-09-08T15:58:20.729 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.729 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:58:20.729 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.729 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:58:20.729 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.451397+0000", 2020-09-08T15:58:20.730 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:58:20.730 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.730 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.730 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:58:20.731 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.451397+0000", 2020-09-08T15:58:20.731 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999576 2020-09-08T15:58:20.731 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.732 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.732 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:58:20.732 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.451355+0000", 2020-09-08T15:58:20.732 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011963700000000001 2020-09-08T15:58:20.733 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.733 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.733 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:58:20.733 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.451474+0000", 2020-09-08T15:58:20.733 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2483000000000001e-05 2020-09-08T15:58:20.734 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.734 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.734 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:58:20.734 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.451487+0000", 2020-09-08T15:58:20.734 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9370000000000003e-06 2020-09-08T15:58:20.735 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.735 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.735 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:58:20.735 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.451492+0000", 2020-09-08T15:58:20.735 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1952999999999997e-05 2020-09-08T15:58:20.736 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.736 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.736 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:58:20.736 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.451524+0000", 2020-09-08T15:58:20.736 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3351999999999997e-05 2020-09-08T15:58:20.737 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.737 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.737 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:58:20.737 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.451617+0000", 2020-09-08T15:58:20.737 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000113417 2020-09-08T15:58:20.738 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.738 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.738 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 7", 2020-09-08T15:58:20.738 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.451731+0000", 2020-09-08T15:58:20.738 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0062509660000000002 2020-09-08T15:58:20.739 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.739 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.739 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:58:20.739 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.457981+0000", 2020-09-08T15:58:20.739 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18591661100000001 2020-09-08T15:58:20.739 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.740 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.740 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:58:20.740 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.643898+0000", 2020-09-08T15:58:20.740 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014634399999999999 2020-09-08T15:58:20.740 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.741 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.741 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:58:20.741 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.644044+0000", 2020-09-08T15:58:20.741 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000121975 2020-09-08T15:58:20.741 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.742 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.742 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:58:20.742 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.644166+0000", 2020-09-08T15:58:20.742 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.19276940200000001 2020-09-08T15:58:20.742 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.743 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.743 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.743 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.744 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.744 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:116603 6.c 6:324e1a20:::benchmark_data_smithi166_43187_object116602:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e508)", 2020-09-08T15:58:20.744 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:55.452164+0000", 2020-09-08T15:58:20.744 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 24.954174436999999, 2020-09-08T15:58:20.744 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18818670500000001, 2020-09-08T15:58:20.745 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:58:20.745 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:20.745 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:58:20.745 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:58:20.746 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:20.746 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 116603 2020-09-08T15:58:20.746 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.746 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:58:20.746 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.747 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:58:20.747 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.452164+0000", 2020-09-08T15:58:20.747 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:58:20.747 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.748 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.748 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:58:20.748 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.452164+0000", 2020-09-08T15:58:20.748 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999676 2020-09-08T15:58:20.749 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.749 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.749 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:58:20.749 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.452132+0000", 2020-09-08T15:58:20.749 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011053900000000001 2020-09-08T15:58:20.750 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.750 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.750 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:58:20.750 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.452243+0000", 2020-09-08T15:58:20.750 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.094e-05 2020-09-08T15:58:20.751 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.751 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.751 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:58:20.751 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.452254+0000", 2020-09-08T15:58:20.751 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7500000000000003e-06 2020-09-08T15:58:20.751 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.752 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.752 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:58:20.752 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.452258+0000", 2020-09-08T15:58:20.752 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8846e-05 2020-09-08T15:58:20.752 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.753 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.753 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:58:20.753 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.452307+0000", 2020-09-08T15:58:20.753 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000134692 2020-09-08T15:58:20.753 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.754 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.754 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:58:20.754 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.452442+0000", 2020-09-08T15:58:20.754 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000224699 2020-09-08T15:58:20.755 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.755 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.755 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 6", 2020-09-08T15:58:20.755 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.452667+0000", 2020-09-08T15:58:20.755 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0035563090000000001 2020-09-08T15:58:20.756 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.756 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.756 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:58:20.756 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.456223+0000", 2020-09-08T15:58:20.756 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18388112200000001 2020-09-08T15:58:20.757 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.757 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.757 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:58:20.757 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.640104+0000", 2020-09-08T15:58:20.757 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000170397 2020-09-08T15:58:20.757 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.758 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.758 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:58:20.758 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.640274+0000", 2020-09-08T15:58:20.758 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6759000000000001e-05 2020-09-08T15:58:20.758 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.759 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.759 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:58:20.759 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.640351+0000", 2020-09-08T15:58:20.759 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18818670500000001 2020-09-08T15:58:20.759 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.760 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.760 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.760 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.760 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.761 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:116604 6.b 6:d5067192:::benchmark_data_smithi166_43187_object116603:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e508)", 2020-09-08T15:58:20.761 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:55.454503+0000", 2020-09-08T15:58:20.761 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 24.951836105000002, 2020-09-08T15:58:20.761 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18985633499999999, 2020-09-08T15:58:20.761 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:58:20.762 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:20.762 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T15:58:20.762 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T15:58:20.762 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:20.762 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 116604 2020-09-08T15:58:20.763 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.763 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:58:20.763 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.763 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:58:20.763 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.454503+0000", 2020-09-08T15:58:20.764 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:58:20.764 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.764 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.764 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:58:20.765 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.454503+0000", 2020-09-08T15:58:20.765 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999604 2020-09-08T15:58:20.765 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.765 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.765 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:58:20.766 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.454463+0000", 2020-09-08T15:58:20.766 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000120937 2020-09-08T15:58:20.766 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.766 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.766 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:58:20.767 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.454584+0000", 2020-09-08T15:58:20.767 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1729000000000001e-05 2020-09-08T15:58:20.767 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.767 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.767 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:58:20.768 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.454596+0000", 2020-09-08T15:58:20.768 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5300000000000004e-06 2020-09-08T15:58:20.768 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.768 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.768 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:58:20.768 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.454601+0000", 2020-09-08T15:58:20.769 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00041096600000000002 2020-09-08T15:58:20.769 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.769 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.769 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:58:20.769 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.455012+0000", 2020-09-08T15:58:20.770 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1823000000000005e-05 2020-09-08T15:58:20.770 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.770 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.770 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:58:20.771 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.455104+0000", 2020-09-08T15:58:20.771 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011297499999999999 2020-09-08T15:58:20.771 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.771 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.771 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 7", 2020-09-08T15:58:20.772 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.455217+0000", 2020-09-08T15:58:20.772 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18855746000000001 2020-09-08T15:58:20.772 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.772 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.772 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T15:58:20.773 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.643774+0000", 2020-09-08T15:58:20.773 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00048771399999999999 2020-09-08T15:58:20.773 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.773 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.774 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:58:20.774 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.644262+0000", 2020-09-08T15:58:20.774 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3813999999999997e-05 2020-09-08T15:58:20.774 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.774 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.775 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T15:58:20.775 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.644316+0000", 2020-09-08T15:58:20.775 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3161000000000002e-05 2020-09-08T15:58:20.775 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.776 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.776 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:58:20.776 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.644359+0000", 2020-09-08T15:58:20.776 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18985633499999999 2020-09-08T15:58:20.776 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.776 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.777 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.777 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.777 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.777 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop_reply(client.16014.0:116603 6.c e508/478 ondisk, result = 0)", 2020-09-08T15:58:20.777 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:55.458068+0000", 2020-09-08T15:58:20.778 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 24.948270457, 2020-09-08T15:58:20.778 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18229253300000001, 2020-09-08T15:58:20.778 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T15:58:20.778 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2020-09-08T15:58:20.778 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T15:58:20.779 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.779 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T15:58:20.779 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.458068+0000", 2020-09-08T15:58:20.779 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T15:58:20.780 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.780 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.780 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T15:58:20.780 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.458068+0000", 2020-09-08T15:58:20.781 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.205e-05 2020-09-08T15:58:20.781 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.781 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.781 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T15:58:20.782 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.458100+0000", 2020-09-08T15:58:20.782 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3070000000000002e-06 2020-09-08T15:58:20.782 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.783 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.783 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T15:58:20.783 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.458106+0000", 2020-09-08T15:58:20.783 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0006e-05 2020-09-08T15:58:20.783 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.784 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.784 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T15:58:20.784 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.458116+0000", 2020-09-08T15:58:20.785 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.442e-06 2020-09-08T15:58:20.785 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.785 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.785 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T15:58:20.786 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.458120+0000", 2020-09-08T15:58:20.786 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18192597499999999 2020-09-08T15:58:20.786 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.787 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.787 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T15:58:20.787 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.640046+0000", 2020-09-08T15:58:20.787 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1062999999999997e-05 2020-09-08T15:58:20.788 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.788 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.788 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T15:58:20.788 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.640077+0000", 2020-09-08T15:58:20.789 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00028369000000000002 2020-09-08T15:58:20.789 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T15:58:20.789 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T15:58:20.789 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T15:58:20.790 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:55.640361+0000", 2020-09-08T15:58:20.790 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18229253300000001 2020-09-08T15:58:20.790 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.790 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.790 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.791 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T15:58:20.791 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T15:58:20.791 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T15:58:21.017 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:58:21.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:20 smithi055 bash[9296]: cluster 2020-09-08T15:58:19.924844+0000 mgr.x (mgr.24118) 930 : cluster [DBG] pgmap v1361: 17 pgs: 17 active+clean; 6.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T15:58:21.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:20 smithi104 bash[9203]: cluster 2020-09-08T15:58:19.924844+0000 mgr.x (mgr.24118) 930 : cluster [DBG] pgmap v1361: 17 pgs: 17 active+clean; 6.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T15:58:21.181 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:58:21.226 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:58:21.331 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:21.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:20 smithi166 bash[9446]: cluster 2020-09-08T15:58:19.924844+0000 mgr.x (mgr.24118) 930 : cluster [DBG] pgmap v1361: 17 pgs: 17 active+clean; 6.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T15:58:21.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:58:21 smithi055 bash[16646]: debug 2020-09-08T15:58:21.229+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:21.908 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2020-09-08T15:58:21.975 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:22.079 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:22.157 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:58:22.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:58:21 smithi104 bash[22305]: debug 2020-09-08T15:58:21.977+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:22.261 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:58:22 smithi055 bash[14301]: debug 2020-09-08T15:58:22.257+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:22.261 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:22.366 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:58:22.793 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:58:22.964 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:58:23.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:22 smithi104 bash[9203]: cluster 2020-09-08T15:58:21.925336+0000 mgr.x (mgr.24118) 931 : cluster [DBG] pgmap v1362: 17 pgs: 17 active+clean; 6.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 999 op/s 2020-09-08T15:58:23.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:22 smithi104 bash[9203]: cluster 2020-09-08T15:58:22.434342+0000 osd.5 (osd.5) 75 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:23.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:22 smithi104 bash[9203]: cluster 2020-09-08T15:58:22.434441+0000 osd.5 (osd.5) 76 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:23.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:22 smithi166 bash[9446]: cluster 2020-09-08T15:58:21.925336+0000 mgr.x (mgr.24118) 931 : cluster [DBG] pgmap v1362: 17 pgs: 17 active+clean; 6.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 999 op/s 2020-09-08T15:58:23.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:22 smithi166 bash[9446]: cluster 2020-09-08T15:58:22.434342+0000 osd.5 (osd.5) 75 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:23.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:22 smithi166 bash[9446]: cluster 2020-09-08T15:58:22.434441+0000 osd.5 (osd.5) 76 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:23.416 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:22 smithi055 bash[9296]: cluster 2020-09-08T15:58:21.925336+0000 mgr.x (mgr.24118) 931 : cluster [DBG] pgmap v1362: 17 pgs: 17 active+clean; 6.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 999 op/s 2020-09-08T15:58:23.416 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:22 smithi055 bash[9296]: cluster 2020-09-08T15:58:22.434342+0000 osd.5 (osd.5) 75 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:23.426 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:22 smithi055 bash[9296]: cluster 2020-09-08T15:58:22.434441+0000 osd.5 (osd.5) 76 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:24.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:23 smithi104 bash[9203]: cluster 2020-09-08T15:58:22.987193+0000 mon.a (mon.0) 1779 : cluster [DBG] osdmap e516: 12 total, 12 up, 12 in 2020-09-08T15:58:24.377 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:23 smithi166 bash[9446]: cluster 2020-09-08T15:58:22.987193+0000 mon.a (mon.0) 1779 : cluster [DBG] osdmap e516: 12 total, 12 up, 12 in 2020-09-08T15:58:24.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:23 smithi055 bash[9296]: cluster 2020-09-08T15:58:22.987193+0000 mon.a (mon.0) 1779 : cluster [DBG] osdmap e516: 12 total, 12 up, 12 in 2020-09-08T15:58:25.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:24 smithi166 bash[9446]: cluster 2020-09-08T15:58:23.925789+0000 mgr.x (mgr.24118) 932 : cluster [DBG] pgmap v1364: 17 pgs: 17 active+clean; 5.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T15:58:25.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:24 smithi166 bash[9446]: cluster 2020-09-08T15:58:24.339958+0000 osd.10 (osd.10) 63 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:25.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:24 smithi166 bash[9446]: cluster 2020-09-08T15:58:24.340064+0000 osd.10 (osd.10) 64 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:25.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:24 smithi055 bash[9296]: cluster 2020-09-08T15:58:23.925789+0000 mgr.x (mgr.24118) 932 : cluster [DBG] pgmap v1364: 17 pgs: 17 active+clean; 5.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T15:58:25.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:24 smithi055 bash[9296]: cluster 2020-09-08T15:58:24.339958+0000 osd.10 (osd.10) 63 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:25.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:24 smithi055 bash[9296]: cluster 2020-09-08T15:58:24.340064+0000 osd.10 (osd.10) 64 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:25.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:24 smithi104 bash[9203]: cluster 2020-09-08T15:58:23.925789+0000 mgr.x (mgr.24118) 932 : cluster [DBG] pgmap v1364: 17 pgs: 17 active+clean; 5.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T15:58:25.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:24 smithi104 bash[9203]: cluster 2020-09-08T15:58:24.339958+0000 osd.10 (osd.10) 63 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:25.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:24 smithi104 bash[9203]: cluster 2020-09-08T15:58:24.340064+0000 osd.10 (osd.10) 64 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:25.785 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:58:25.897 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:26.047 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:26.148 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:26.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:26 smithi055 bash[9296]: cluster 2020-09-08T15:58:25.007612+0000 mon.a (mon.0) 1780 : cluster [DBG] osdmap e517: 12 total, 12 up, 12 in 2020-09-08T15:58:26.158 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:58:25 smithi055 bash[16646]: debug 2020-09-08T15:58:25.785+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:26.318 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:26 smithi104 bash[9203]: cluster 2020-09-08T15:58:25.007612+0000 mon.a (mon.0) 1780 : cluster [DBG] osdmap e517: 12 total, 12 up, 12 in 2020-09-08T15:58:26.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:58:26 smithi166 bash[11928]: debug 2020-09-08T15:58:26.030+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:26.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:26 smithi166 bash[9446]: cluster 2020-09-08T15:58:25.007612+0000 mon.a (mon.0) 1780 : cluster [DBG] osdmap e517: 12 total, 12 up, 12 in 2020-09-08T15:58:27.184 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:27.288 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:58:27.314 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:27 smithi104 bash[9203]: cluster 2020-09-08T15:58:25.926279+0000 mgr.x (mgr.24118) 933 : cluster [DBG] pgmap v1366: 17 pgs: 17 active+clean; 5.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 824 op/s 2020-09-08T15:58:27.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:27 smithi166 bash[9446]: cluster 2020-09-08T15:58:25.926279+0000 mgr.x (mgr.24118) 933 : cluster [DBG] pgmap v1366: 17 pgs: 17 active+clean; 5.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 824 op/s 2020-09-08T15:58:27.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:27 smithi055 bash[9296]: cluster 2020-09-08T15:58:25.926279+0000 mgr.x (mgr.24118) 933 : cluster [DBG] pgmap v1366: 17 pgs: 17 active+clean; 5.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 824 op/s 2020-09-08T15:58:27.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:58:27 smithi055 bash[14301]: debug 2020-09-08T15:58:27.185+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:27.549 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:58:27.565 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:58:27 smithi104 bash[17588]: debug 2020-09-08T15:58:27.545+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:27.654 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:27.791 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:27.896 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:28.008 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:58:27 smithi166 bash[14254]: debug 2020-09-08T15:58:27.790+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:28.009 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:28.115 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:58:28.296 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:58:28.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:58:28 smithi166 bash[11928]: debug 2020-09-08T15:58:28.006+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:28.399 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:58:28.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:58:28 smithi055 bash[16646]: debug 2020-09-08T15:58:28.297+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:28.527 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:58:28.636 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:58:28.719 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:58:28 smithi055 bash[11956]: debug 2020-09-08T15:58:28.529+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:28.820 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:58:28.924 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:58:29.015 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:58:28 smithi055 bash[19003]: debug 2020-09-08T15:58:28.821+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:29.026 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:58:29.282 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:58:29.328 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:58:29.357 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:58:29.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:29 smithi166 bash[9446]: cluster 2020-09-08T15:58:27.926784+0000 mgr.x (mgr.24118) 934 : cluster [DBG] pgmap v1367: 17 pgs: 17 active+clean; 5.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.11k op/s 2020-09-08T15:58:29.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:29 smithi166 bash[9446]: audit 2020-09-08T15:58:28.580288+0000 mon.a (mon.0) 1781 : audit [INF] from='client.? v1:172.21.15.104:0/41289302' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:58:29.394 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:58:29.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:29 smithi055 bash[9296]: cluster 2020-09-08T15:58:27.926784+0000 mgr.x (mgr.24118) 934 : cluster [DBG] pgmap v1367: 17 pgs: 17 active+clean; 5.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.11k op/s 2020-09-08T15:58:29.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:29 smithi055 bash[9296]: audit 2020-09-08T15:58:28.580288+0000 mon.a (mon.0) 1781 : audit [INF] from='client.? v1:172.21.15.104:0/41289302' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:58:29.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:29 smithi104 bash[9203]: cluster 2020-09-08T15:58:27.926784+0000 mgr.x (mgr.24118) 934 : cluster [DBG] pgmap v1367: 17 pgs: 17 active+clean; 5.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.11k op/s 2020-09-08T15:58:29.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:29 smithi104 bash[9203]: audit 2020-09-08T15:58:28.580288+0000 mon.a (mon.0) 1781 : audit [INF] from='client.? v1:172.21.15.104:0/41289302' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:58:29.436 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:58:29.476 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:58:29.513 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:58:29.545 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:58:29.578 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:58:29.609 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:58:29.645 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:58:29.674 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:58:29.676 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:58:29.676 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:58:29.676 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:58:29.676 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:58:29.676 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:58:29.677 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:58:29.677 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:58:29.677 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:58:29.677 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:58:29.677 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:58:29.678 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:58:29.678 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:58:29.791 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:58:29.901 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:30.021 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:30.125 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:30.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:30 smithi055 bash[9296]: cluster 2020-09-08T15:58:29.012998+0000 mon.a (mon.0) 1782 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:58:30.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:30 smithi055 bash[9296]: audit 2020-09-08T15:58:29.024426+0000 mon.a (mon.0) 1783 : audit [INF] from='client.? v1:172.21.15.104:0/41289302' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:58:30.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:30 smithi055 bash[9296]: cluster 2020-09-08T15:58:29.024461+0000 mon.a (mon.0) 1784 : cluster [DBG] osdmap e518: 12 total, 12 up, 12 in 2020-09-08T15:58:30.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:30 smithi055 bash[9296]: cluster 2020-09-08T15:58:29.060933+0000 osd.3 (osd.3) 81 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:30.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:30 smithi055 bash[9296]: cluster 2020-09-08T15:58:29.061015+0000 osd.3 (osd.3) 82 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:30.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:30 smithi055 bash[9296]: audit 2020-09-08T15:58:29.248547+0000 mon.a (mon.0) 1785 : audit [DBG] from='client.? v1:172.21.15.104:0/3410780888' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:58:30.159 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:58:29 smithi055 bash[11956]: debug 2020-09-08T15:58:29.789+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:30.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:58:30 smithi166 bash[11928]: debug 2020-09-08T15:58:30.022+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:30.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:30 smithi166 bash[9446]: cluster 2020-09-08T15:58:29.012998+0000 mon.a (mon.0) 1782 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:58:30.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:30 smithi166 bash[9446]: audit 2020-09-08T15:58:29.024426+0000 mon.a (mon.0) 1783 : audit [INF] from='client.? v1:172.21.15.104:0/41289302' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:58:30.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:30 smithi166 bash[9446]: cluster 2020-09-08T15:58:29.024461+0000 mon.a (mon.0) 1784 : cluster [DBG] osdmap e518: 12 total, 12 up, 12 in 2020-09-08T15:58:30.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:30 smithi166 bash[9446]: cluster 2020-09-08T15:58:29.060933+0000 osd.3 (osd.3) 81 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:30.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:30 smithi166 bash[9446]: cluster 2020-09-08T15:58:29.061015+0000 osd.3 (osd.3) 82 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:30.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:30 smithi166 bash[9446]: audit 2020-09-08T15:58:29.248547+0000 mon.a (mon.0) 1785 : audit [DBG] from='client.? v1:172.21.15.104:0/3410780888' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:58:30.388 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:30 smithi104 bash[9203]: cluster 2020-09-08T15:58:29.012998+0000 mon.a (mon.0) 1782 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:58:30.388 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:30 smithi104 bash[9203]: audit 2020-09-08T15:58:29.024426+0000 mon.a (mon.0) 1783 : audit [INF] from='client.? v1:172.21.15.104:0/41289302' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:58:30.388 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:30 smithi104 bash[9203]: cluster 2020-09-08T15:58:29.024461+0000 mon.a (mon.0) 1784 : cluster [DBG] osdmap e518: 12 total, 12 up, 12 in 2020-09-08T15:58:30.389 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:30 smithi104 bash[9203]: cluster 2020-09-08T15:58:29.060933+0000 osd.3 (osd.3) 81 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:30.389 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:30 smithi104 bash[9203]: cluster 2020-09-08T15:58:29.061015+0000 osd.3 (osd.3) 82 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:30.389 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:30 smithi104 bash[9203]: audit 2020-09-08T15:58:29.248547+0000 mon.a (mon.0) 1785 : audit [DBG] from='client.? v1:172.21.15.104:0/3410780888' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:58:30.389 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:30.492 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:58:30.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:58:30 smithi104 bash[22305]: debug 2020-09-08T15:58:30.389+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:31.068 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:58:31.171 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:31.180 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:58:31 smithi104 bash[17588]: debug 2020-09-08T15:58:31.065+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:31.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:31 smithi104 bash[9203]: cluster 2020-09-08T15:58:29.927227+0000 mgr.x (mgr.24118) 935 : cluster [DBG] pgmap v1369: 17 pgs: 17 active+clean; 5.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 816 op/s 2020-09-08T15:58:31.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:31 smithi104 bash[9203]: cluster 2020-09-08T15:58:30.065020+0000 mon.a (mon.0) 1786 : cluster [DBG] osdmap e519: 12 total, 12 up, 12 in 2020-09-08T15:58:31.301 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:31.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:31 smithi166 bash[9446]: cluster 2020-09-08T15:58:29.927227+0000 mgr.x (mgr.24118) 935 : cluster [DBG] pgmap v1369: 17 pgs: 17 active+clean; 5.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 816 op/s 2020-09-08T15:58:31.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:31 smithi166 bash[9446]: cluster 2020-09-08T15:58:30.065020+0000 mon.a (mon.0) 1786 : cluster [DBG] osdmap e519: 12 total, 12 up, 12 in 2020-09-08T15:58:31.369 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:58:31 smithi166 bash[14254]: debug 2020-09-08T15:58:31.301+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:31.406 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:58:31.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:31 smithi055 bash[9296]: cluster 2020-09-08T15:58:29.927227+0000 mgr.x (mgr.24118) 935 : cluster [DBG] pgmap v1369: 17 pgs: 17 active+clean; 5.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 816 op/s 2020-09-08T15:58:31.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:31 smithi055 bash[9296]: cluster 2020-09-08T15:58:30.065020+0000 mon.a (mon.0) 1786 : cluster [DBG] osdmap e519: 12 total, 12 up, 12 in 2020-09-08T15:58:31.563 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:58:31.666 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:31.789 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:31.869 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:58:31.869 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:58:31.896 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:58:31.905 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:58:31 smithi055 bash[16646]: debug 2020-09-08T15:58:31.565+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:32.100 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:58:32.117 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:58:31 smithi166 bash[11928]: debug 2020-09-08T15:58:31.789+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:32.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:58:32 smithi055 bash[11956]: debug 2020-09-08T15:58:32.101+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:32.203 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:32.334 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:32.437 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:32.591 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:32.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:58:32 smithi166 bash[11928]: debug 2020-09-08T15:58:32.329+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:32.696 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:32.805 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:32.879 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:58:32.887 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:58:32 smithi104 bash[22305]: debug 2020-09-08T15:58:32.589+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:32.908 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:58:33.038 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:58:33.062 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:58:32 smithi166 bash[11928]: debug 2020-09-08T15:58:32.805+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:33.074 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:58:33.084 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2020-09-08T15:58:33.178 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:58:33.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:33 smithi104 bash[9203]: cluster 2020-09-08T15:58:31.927749+0000 mgr.x (mgr.24118) 936 : cluster [DBG] pgmap v1371: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 905 op/s 2020-09-08T15:58:33.180 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:58:33 smithi104 bash[19922]: debug 2020-09-08T15:58:33.073+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:33.199 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:58:33.277 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:58:33.309 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:58:33.315 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:33 smithi055 bash[9296]: cluster 2020-09-08T15:58:31.927749+0000 mgr.x (mgr.24118) 936 : cluster [DBG] pgmap v1371: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 905 op/s 2020-09-08T15:58:33.316 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:58:33 smithi055 bash[19003]: debug 2020-09-08T15:58:33.313+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:33.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:33 smithi166 bash[9446]: cluster 2020-09-08T15:58:31.927749+0000 mgr.x (mgr.24118) 936 : cluster [DBG] pgmap v1371: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 905 op/s 2020-09-08T15:58:33.417 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:33.451 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:58:34.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:34 smithi166 bash[9446]: cluster 2020-09-08T15:58:33.618868+0000 osd.2 (osd.2) 76 : cluster [DBG] 6.f scrub starts 2020-09-08T15:58:34.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:34 smithi055 bash[9296]: cluster 2020-09-08T15:58:33.618868+0000 osd.2 (osd.2) 76 : cluster [DBG] 6.f scrub starts 2020-09-08T15:58:34.411 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:34 smithi104 bash[9203]: cluster 2020-09-08T15:58:33.618868+0000 osd.2 (osd.2) 76 : cluster [DBG] 6.f scrub starts 2020-09-08T15:58:34.411 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:34.514 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:58:34.680 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:58:34 smithi104 bash[22305]: debug 2020-09-08T15:58:34.413+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:34.725 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:58:34.834 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:35.078 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:58:34 smithi104 bash[17588]: debug 2020-09-08T15:58:34.725+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:35.317 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:35 smithi055 bash[9296]: cluster 2020-09-08T15:58:33.928322+0000 mgr.x (mgr.24118) 937 : cluster [DBG] pgmap v1372: 17 pgs: 17 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:58:35.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:35 smithi166 bash[9446]: cluster 2020-09-08T15:58:33.928322+0000 mgr.x (mgr.24118) 937 : cluster [DBG] pgmap v1372: 17 pgs: 17 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:58:35.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:35 smithi104 bash[9203]: cluster 2020-09-08T15:58:33.928322+0000 mgr.x (mgr.24118) 937 : cluster [DBG] pgmap v1372: 17 pgs: 17 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:58:35.516 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:35.621 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:58:35.802 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:58:35 smithi104 bash[22305]: debug 2020-09-08T15:58:35.517+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:35.803 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:58:35.906 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:36.043 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:36.081 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:58:35 smithi104 bash[15273]: debug 2020-09-08T15:58:35.801+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:36.150 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:36.298 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:36 smithi166 bash[9446]: cluster 2020-09-08T15:58:34.862912+0000 osd.2 (osd.2) 77 : cluster [DBG] 6.f scrub ok 2020-09-08T15:58:36.299 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:58:36 smithi166 bash[14254]: debug 2020-09-08T15:58:36.041+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:36.299 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:36.350 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:36 smithi104 bash[9203]: cluster 2020-09-08T15:58:34.862912+0000 osd.2 (osd.2) 77 : cluster [DBG] 6.f scrub ok 2020-09-08T15:58:36.403 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:36.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:36 smithi055 bash[9296]: cluster 2020-09-08T15:58:34.862912+0000 osd.2 (osd.2) 77 : cluster [DBG] 6.f scrub ok 2020-09-08T15:58:36.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:58:36 smithi166 bash[14254]: debug 2020-09-08T15:58:36.297+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:36.945 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:37.048 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:37.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:37 smithi166 bash[9446]: cluster 2020-09-08T15:58:35.928725+0000 mgr.x (mgr.24118) 938 : cluster [DBG] pgmap v1373: 17 pgs: 17 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 790 op/s 2020-09-08T15:58:37.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:37 smithi166 bash[9446]: cluster 2020-09-08T15:58:36.162414+0000 osd.8 (osd.8) 67 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:37.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:37 smithi166 bash[9446]: cluster 2020-09-08T15:58:36.162497+0000 osd.8 (osd.8) 68 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:37.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:37 smithi055 bash[9296]: cluster 2020-09-08T15:58:35.928725+0000 mgr.x (mgr.24118) 938 : cluster [DBG] pgmap v1373: 17 pgs: 17 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 790 op/s 2020-09-08T15:58:37.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:37 smithi055 bash[9296]: cluster 2020-09-08T15:58:36.162414+0000 osd.8 (osd.8) 67 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:37.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:37 smithi055 bash[9296]: cluster 2020-09-08T15:58:36.162497+0000 osd.8 (osd.8) 68 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:37.408 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:58:36 smithi055 bash[14301]: debug 2020-09-08T15:58:36.949+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:37.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:37 smithi104 bash[9203]: cluster 2020-09-08T15:58:35.928725+0000 mgr.x (mgr.24118) 938 : cluster [DBG] pgmap v1373: 17 pgs: 17 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 790 op/s 2020-09-08T15:58:37.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:37 smithi104 bash[9203]: cluster 2020-09-08T15:58:36.162414+0000 osd.8 (osd.8) 67 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:37.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:37 smithi104 bash[9203]: cluster 2020-09-08T15:58:36.162497+0000 osd.8 (osd.8) 68 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:38.269 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:58:38.269 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:58:38.278 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:38.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:38 smithi166 bash[9446]: cluster 2020-09-08T15:58:36.882290+0000 osd.0 (osd.0) 77 : cluster [DBG] 6.0 scrub starts 2020-09-08T15:58:38.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:38 smithi166 bash[9446]: cluster 2020-09-08T15:58:37.097738+0000 mon.a (mon.0) 1787 : cluster [DBG] osdmap e520: 12 total, 12 up, 12 in 2020-09-08T15:58:38.374 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:38 smithi055 bash[9296]: cluster 2020-09-08T15:58:36.882290+0000 osd.0 (osd.0) 77 : cluster [DBG] 6.0 scrub starts 2020-09-08T15:58:38.375 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:38 smithi055 bash[9296]: cluster 2020-09-08T15:58:37.097738+0000 mon.a (mon.0) 1787 : cluster [DBG] osdmap e520: 12 total, 12 up, 12 in 2020-09-08T15:58:38.382 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:58:38.391 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:58:38 smithi104 bash[22305]: debug 2020-09-08T15:58:38.277+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:38.392 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:38 smithi104 bash[9203]: cluster 2020-09-08T15:58:36.882290+0000 osd.0 (osd.0) 77 : cluster [DBG] 6.0 scrub starts 2020-09-08T15:58:38.392 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:38 smithi104 bash[9203]: cluster 2020-09-08T15:58:37.097738+0000 mon.a (mon.0) 1787 : cluster [DBG] osdmap e520: 12 total, 12 up, 12 in 2020-09-08T15:58:38.574 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:58:38.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:58:38 smithi104 bash[19922]: debug 2020-09-08T15:58:38.573+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:38.686 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:58:39.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:39 smithi166 bash[9446]: cluster 2020-09-08T15:58:37.929165+0000 mgr.x (mgr.24118) 939 : cluster [DBG] pgmap v1375: 17 pgs: 17 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T15:58:39.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:39 smithi166 bash[9446]: cluster 2020-09-08T15:58:38.368665+0000 osd.0 (osd.0) 78 : cluster [DBG] 6.0 scrub ok 2020-09-08T15:58:39.395 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2020-09-08T15:58:39.398 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:39 smithi055 bash[9296]: cluster 2020-09-08T15:58:37.929165+0000 mgr.x (mgr.24118) 939 : cluster [DBG] pgmap v1375: 17 pgs: 17 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T15:58:39.398 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:39 smithi055 bash[9296]: cluster 2020-09-08T15:58:38.368665+0000 osd.0 (osd.0) 78 : cluster [DBG] 6.0 scrub ok 2020-09-08T15:58:39.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:39 smithi104 bash[9203]: cluster 2020-09-08T15:58:37.929165+0000 mgr.x (mgr.24118) 939 : cluster [DBG] pgmap v1375: 17 pgs: 17 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T15:58:39.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:39 smithi104 bash[9203]: cluster 2020-09-08T15:58:38.368665+0000 osd.0 (osd.0) 78 : cluster [DBG] 6.0 scrub ok 2020-09-08T15:58:39.470 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:58:39.568 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:58:39.573 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:39.719 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:39.824 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:39.834 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:58:39 smithi166 bash[11928]: debug 2020-09-08T15:58:39.717+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:39.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:58:39 smithi104 bash[15273]: debug 2020-09-08T15:58:39.469+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:39.960 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:40.066 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:40.117 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:58:39 smithi166 bash[11928]: debug 2020-09-08T15:58:39.957+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:40.413 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:40.517 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:40.881 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:58:40 smithi055 bash[14301]: debug 2020-09-08T15:58:40.413+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:41.025 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:41.130 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:58:41.140 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:40 smithi166 bash[9446]: cluster 2020-09-08T15:58:39.225450+0000 osd.1 (osd.1) 102 : cluster [DBG] 6.e scrub starts 2020-09-08T15:58:41.141 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:40 smithi166 bash[9446]: cluster 2020-09-08T15:58:39.929665+0000 mgr.x (mgr.24118) 940 : cluster [DBG] pgmap v1376: 17 pgs: 17 active+clean; 4.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 967 op/s 2020-09-08T15:58:41.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:40 smithi055 bash[9296]: cluster 2020-09-08T15:58:39.225450+0000 osd.1 (osd.1) 102 : cluster [DBG] 6.e scrub starts 2020-09-08T15:58:41.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:40 smithi055 bash[9296]: cluster 2020-09-08T15:58:39.929665+0000 mgr.x (mgr.24118) 940 : cluster [DBG] pgmap v1376: 17 pgs: 17 active+clean; 4.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 967 op/s 2020-09-08T15:58:41.158 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:58:41 smithi055 bash[14301]: debug 2020-09-08T15:58:41.029+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:41.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:40 smithi104 bash[9203]: cluster 2020-09-08T15:58:39.225450+0000 osd.1 (osd.1) 102 : cluster [DBG] 6.e scrub starts 2020-09-08T15:58:41.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:40 smithi104 bash[9203]: cluster 2020-09-08T15:58:39.929665+0000 mgr.x (mgr.24118) 940 : cluster [DBG] pgmap v1376: 17 pgs: 17 active+clean; 4.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 967 op/s 2020-09-08T15:58:41.261 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:58:41.365 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:58:41.376 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:58:41.421 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:58:41.466 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:58:41.498 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:58:41.514 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:58:41.560 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:58:41.592 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:58:41.603 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:58:41.618 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:58:41 smithi166 bash[18960]: debug 2020-09-08T15:58:41.261+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:41.623 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:58:41.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:58:41 smithi055 bash[19003]: debug 2020-09-08T15:58:41.493+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:41.667 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:58:41.706 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:58:41.744 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:58:41.779 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:58:41.796 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:58:41.811 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:58:41.818 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:58:41.818 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:58:41.832 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:58:41.833 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:58:41.833 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:58:41.833 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:58:41.833 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:58:41.833 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:58:41.834 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:58:41.834 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:58:41.834 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:58:41.834 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:58:41.898 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:58:41.900 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:58:42.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:58:41 smithi055 bash[11956]: debug 2020-09-08T15:58:41.797+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:42.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:41 smithi055 bash[9296]: cluster 2020-09-08T15:58:40.838342+0000 osd.1 (osd.1) 103 : cluster [DBG] 6.e scrub ok 2020-09-08T15:58:42.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:41 smithi055 bash[9296]: audit 2020-09-08T15:58:41.335475+0000 mon.c (mon.2) 70 : audit [DBG] from='client.? v1:172.21.15.104:0/966171785' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:58:42.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:41 smithi055 bash[9296]: audit 2020-09-08T15:58:41.651921+0000 mon.a (mon.0) 1788 : audit [INF] from='client.? v1:172.21.15.104:0/3371261375' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:58:42.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:41 smithi104 bash[9203]: cluster 2020-09-08T15:58:40.838342+0000 osd.1 (osd.1) 103 : cluster [DBG] 6.e scrub ok 2020-09-08T15:58:42.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:41 smithi104 bash[9203]: audit 2020-09-08T15:58:41.335475+0000 mon.c (mon.2) 70 : audit [DBG] from='client.? v1:172.21.15.104:0/966171785' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:58:42.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:41 smithi104 bash[9203]: audit 2020-09-08T15:58:41.651921+0000 mon.a (mon.0) 1788 : audit [INF] from='client.? v1:172.21.15.104:0/3371261375' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:58:42.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:41 smithi166 bash[9446]: cluster 2020-09-08T15:58:40.838342+0000 osd.1 (osd.1) 103 : cluster [DBG] 6.e scrub ok 2020-09-08T15:58:42.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:41 smithi166 bash[9446]: audit 2020-09-08T15:58:41.335475+0000 mon.c (mon.2) 70 : audit [DBG] from='client.? v1:172.21.15.104:0/966171785' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:58:42.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:41 smithi166 bash[9446]: audit 2020-09-08T15:58:41.651921+0000 mon.a (mon.0) 1788 : audit [INF] from='client.? v1:172.21.15.104:0/3371261375' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:58:42.569 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:58:42.675 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:42.876 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:42.892 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:42 smithi055 bash[9296]: cluster 2020-09-08T15:58:41.887260+0000 mon.a (mon.0) 1789 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:58:42.892 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:42 smithi055 bash[9296]: cluster 2020-09-08T15:58:41.887301+0000 mon.a (mon.0) 1790 : cluster [INF] Cluster is now healthy 2020-09-08T15:58:42.892 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:42 smithi055 bash[9296]: audit 2020-09-08T15:58:41.893546+0000 mon.a (mon.0) 1791 : audit [INF] from='client.? v1:172.21.15.104:0/3371261375' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:58:42.893 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:42 smithi055 bash[9296]: cluster 2020-09-08T15:58:41.893733+0000 mon.a (mon.0) 1792 : cluster [DBG] osdmap e521: 12 total, 12 up, 12 in 2020-09-08T15:58:42.893 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:42 smithi055 bash[9296]: audit 2020-09-08T15:58:41.938858+0000 mon.a (mon.0) 1793 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:58:42.893 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:42 smithi055 bash[9296]: audit 2020-09-08T15:58:41.942665+0000 mon.a (mon.0) 1794 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:58:42.894 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:58:42 smithi055 bash[19003]: debug 2020-09-08T15:58:42.573+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:42.981 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:58:43.117 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:58:42 smithi166 bash[11928]: debug 2020-09-08T15:58:42.873+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:43.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:42 smithi166 bash[9446]: cluster 2020-09-08T15:58:41.887260+0000 mon.a (mon.0) 1789 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:58:43.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:42 smithi166 bash[9446]: cluster 2020-09-08T15:58:41.887301+0000 mon.a (mon.0) 1790 : cluster [INF] Cluster is now healthy 2020-09-08T15:58:43.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:42 smithi166 bash[9446]: audit 2020-09-08T15:58:41.893546+0000 mon.a (mon.0) 1791 : audit [INF] from='client.? v1:172.21.15.104:0/3371261375' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:58:43.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:42 smithi166 bash[9446]: cluster 2020-09-08T15:58:41.893733+0000 mon.a (mon.0) 1792 : cluster [DBG] osdmap e521: 12 total, 12 up, 12 in 2020-09-08T15:58:43.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:42 smithi166 bash[9446]: audit 2020-09-08T15:58:41.938858+0000 mon.a (mon.0) 1793 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:58:43.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:42 smithi166 bash[9446]: audit 2020-09-08T15:58:41.942665+0000 mon.a (mon.0) 1794 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:58:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:42 smithi104 bash[9203]: cluster 2020-09-08T15:58:41.887260+0000 mon.a (mon.0) 1789 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:58:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:42 smithi104 bash[9203]: cluster 2020-09-08T15:58:41.887301+0000 mon.a (mon.0) 1790 : cluster [INF] Cluster is now healthy 2020-09-08T15:58:43.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:42 smithi104 bash[9203]: audit 2020-09-08T15:58:41.893546+0000 mon.a (mon.0) 1791 : audit [INF] from='client.? v1:172.21.15.104:0/3371261375' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:58:43.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:42 smithi104 bash[9203]: cluster 2020-09-08T15:58:41.893733+0000 mon.a (mon.0) 1792 : cluster [DBG] osdmap e521: 12 total, 12 up, 12 in 2020-09-08T15:58:43.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:42 smithi104 bash[9203]: audit 2020-09-08T15:58:41.938858+0000 mon.a (mon.0) 1793 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:58:43.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:42 smithi104 bash[9203]: audit 2020-09-08T15:58:41.942665+0000 mon.a (mon.0) 1794 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:58:43.663 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:58:43.779 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:43.921 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:58:43 smithi104 bash[19922]: debug 2020-09-08T15:58:43.665+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:44.146 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:58:44 smithi055 bash[14301]: debug 2020-09-08T15:58:44.149+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:44.148 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:44.253 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:58:44.431 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:58:44.536 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:44.670 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:44.773 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:58:44.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:58:44 smithi166 bash[11928]: debug 2020-09-08T15:58:44.669+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:44.871 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:58:44.871 INFO:teuthology.orchestra.run.smithi055.stdout: "size": 20, 2020-09-08T15:58:44.871 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 600, 2020-09-08T15:58:44.872 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T15:58:44.872 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.872 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35381.0:158549 4.d e354/314 4:bce4dc56:::benchmark_data_smithi166_32054_object16407:head v 354'9897, mlcod=354'9895)", 2020-09-08T15:58:44.872 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:49:10.974090+0000", 2020-09-08T15:58:44.872 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 573.90109099400001, 2020-09-08T15:58:44.873 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.112964493, 2020-09-08T15:58:44.873 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:44.873 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:44.873 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:44.873 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.874 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:44.874 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974090+0000", 2020-09-08T15:58:44.874 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:44.874 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.875 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.875 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:44.875 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974090+0000", 2020-09-08T15:58:44.875 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.7972000000000002e-05 2020-09-08T15:58:44.875 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.875 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.876 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:44.876 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974128+0000", 2020-09-08T15:58:44.876 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0639999999999999e-06 2020-09-08T15:58:44.876 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.876 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.877 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:44.877 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974134+0000", 2020-09-08T15:58:44.877 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.1800000000000002e-06 2020-09-08T15:58:44.877 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.877 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.878 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:44.878 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974143+0000", 2020-09-08T15:58:44.878 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.8539999999999996e-06 2020-09-08T15:58:44.878 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.878 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.879 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:44.879 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974148+0000", 2020-09-08T15:58:44.879 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9451999999999999e-05 2020-09-08T15:58:44.879 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.879 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.880 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:44.880 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974207+0000", 2020-09-08T15:58:44.880 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.6696e-05 2020-09-08T15:58:44.880 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.880 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.880 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:44.881 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.974234+0000", 2020-09-08T15:58:44.881 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0053667009999999998 2020-09-08T15:58:44.881 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.881 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.881 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:58:44.882 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:10.979601+0000", 2020-09-08T15:58:44.882 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.107453574 2020-09-08T15:58:44.882 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.882 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.882 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:58:44.883 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:11.087054+0000", 2020-09-08T15:58:44.883 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.112964493 2020-09-08T15:58:44.883 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:44.883 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:44.883 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:44.884 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.884 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.884 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.35381.0:177575 4.e e359/314 4:7204d027:::benchmark_data_smithi166_32054_object35433:head v 359'11155, mlcod=359'11152)", 2020-09-08T15:58:44.884 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:49:29.720082+0000", 2020-09-08T15:58:44.884 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 555.15509827599999, 2020-09-08T15:58:44.885 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.048367474000000001, 2020-09-08T15:58:44.885 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:44.885 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:44.885 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:44.885 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.885 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:44.886 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.720082+0000", 2020-09-08T15:58:44.886 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:44.886 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.886 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.886 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:44.887 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.720082+0000", 2020-09-08T15:58:44.887 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000108624 2020-09-08T15:58:44.887 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.887 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.887 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:44.888 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.720191+0000", 2020-09-08T15:58:44.888 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3309000000000001e-05 2020-09-08T15:58:44.888 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.888 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.888 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:44.889 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.720204+0000", 2020-09-08T15:58:44.889 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.2960999999999999e-05 2020-09-08T15:58:44.889 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.889 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.889 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:44.890 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.720227+0000", 2020-09-08T15:58:44.890 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.6050000000000004e-06 2020-09-08T15:58:44.890 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.890 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.890 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:44.890 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.720237+0000", 2020-09-08T15:58:44.891 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00088012300000000002 2020-09-08T15:58:44.891 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.892 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.893 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:44.893 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.721117+0000", 2020-09-08T15:58:44.893 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.8958e-05 2020-09-08T15:58:44.893 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.893 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.894 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:44.894 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.721136+0000", 2020-09-08T15:58:44.894 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.047219070000000002 2020-09-08T15:58:44.894 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.894 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.895 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:58:44.895 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.768355+0000", 2020-09-08T15:58:44.895 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4823999999999999e-05 2020-09-08T15:58:44.895 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.895 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.896 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:58:44.896 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:49:29.768450+0000", 2020-09-08T15:58:44.896 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.048367474000000001 2020-09-08T15:58:44.896 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:44.896 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:44.897 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:44.897 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.897 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.897 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.15771.0:92706 5.c e417/396 5:3368ea65:::benchmark_data_smithi166_37820_object92705:head v 417'5783, mlcod=417'5781)", 2020-09-08T15:58:44.897 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:52:38.285778+0000", 2020-09-08T15:58:44.898 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 366.589402882, 2020-09-08T15:58:44.898 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.148506577, 2020-09-08T15:58:44.898 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:44.898 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:44.898 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:44.899 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.899 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:44.899 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285778+0000", 2020-09-08T15:58:44.899 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:44.899 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.899 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.900 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:44.900 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285778+0000", 2020-09-08T15:58:44.900 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.1017e-05 2020-09-08T15:58:44.900 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.900 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.901 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:44.901 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285859+0000", 2020-09-08T15:58:44.901 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.1399999999999997e-06 2020-09-08T15:58:44.901 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.901 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.902 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:44.902 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285865+0000", 2020-09-08T15:58:44.902 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4960000000000002e-06 2020-09-08T15:58:44.902 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.903 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.903 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:44.903 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285874+0000", 2020-09-08T15:58:44.903 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.5709999999999999e-06 2020-09-08T15:58:44.903 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.903 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.904 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:44.904 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285879+0000", 2020-09-08T15:58:44.904 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.6329999999999997e-05 2020-09-08T15:58:44.904 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.904 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.905 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:44.905 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285945+0000", 2020-09-08T15:58:44.905 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.5469e-05 2020-09-08T15:58:44.905 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.905 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.906 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:44.906 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.285971+0000", 2020-09-08T15:58:44.906 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.14824900699999999 2020-09-08T15:58:44.906 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.906 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.907 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:58:44.907 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.434220+0000", 2020-09-08T15:58:44.907 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.4547000000000004e-05 2020-09-08T15:58:44.907 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.907 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.907 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:58:44.908 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.434284+0000", 2020-09-08T15:58:44.908 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.148506577 2020-09-08T15:58:44.908 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:44.908 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:44.908 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:44.909 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.909 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.909 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.15771.0:92708 5.c e417/396 5:39784271:::benchmark_data_smithi166_37820_object92707:head v 417'5784, mlcod=417'5781)", 2020-09-08T15:58:44.909 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:52:38.286728+0000", 2020-09-08T15:58:44.909 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 366.58845211400001, 2020-09-08T15:58:44.910 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.160917904, 2020-09-08T15:58:44.910 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:44.910 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:44.910 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:44.910 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.911 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:44.911 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.286728+0000", 2020-09-08T15:58:44.911 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:44.911 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.911 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.912 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:44.912 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.286728+0000", 2020-09-08T15:58:44.912 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000106149 2020-09-08T15:58:44.912 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.912 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.913 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:44.913 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.286835+0000", 2020-09-08T15:58:44.913 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.952e-06 2020-09-08T15:58:44.913 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.913 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.913 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:44.914 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.286844+0000", 2020-09-08T15:58:44.914 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4589999999999999e-05 2020-09-08T15:58:44.914 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.914 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.914 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:44.915 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.286858+0000", 2020-09-08T15:58:44.915 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.7769999999999999e-06 2020-09-08T15:58:44.917 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.918 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.918 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:44.925 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.286864+0000", 2020-09-08T15:58:44.925 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.14290330000000001 2020-09-08T15:58:44.926 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.926 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.926 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:44.926 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.429767+0000", 2020-09-08T15:58:44.926 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.6913999999999998e-05 2020-09-08T15:58:44.927 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.927 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.927 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:44.927 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.429814+0000", 2020-09-08T15:58:44.927 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.017743755 2020-09-08T15:58:44.928 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.928 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.928 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:58:44.928 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.447558+0000", 2020-09-08T15:58:44.928 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.8467000000000005e-05 2020-09-08T15:58:44.928 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.929 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.929 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:58:44.929 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.447646+0000", 2020-09-08T15:58:44.929 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.160917904 2020-09-08T15:58:44.929 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:44.930 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:44.930 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:44.930 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.930 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.930 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.15771.0:92717 5.f e417/396 5:f8e6b0a5:::benchmark_data_smithi166_37820_object92716:head v 417'5849, mlcod=417'5846)", 2020-09-08T15:58:44.931 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:52:38.288221+0000", 2020-09-08T15:58:44.931 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 366.58696008800001, 2020-09-08T15:58:44.931 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.14981187600000001, 2020-09-08T15:58:44.931 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:44.931 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:44.932 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:44.932 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.932 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:44.932 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.288221+0000", 2020-09-08T15:58:44.932 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:44.933 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.933 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.933 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:44.933 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.288221+0000", 2020-09-08T15:58:44.933 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013050599999999999 2020-09-08T15:58:44.933 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.934 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.934 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:44.934 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.288351+0000", 2020-09-08T15:58:44.934 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1793999999999999e-05 2020-09-08T15:58:44.934 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.935 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.935 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:44.935 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.288363+0000", 2020-09-08T15:58:44.935 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.14131466000000001 2020-09-08T15:58:44.935 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.936 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.936 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:44.936 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.429677+0000", 2020-09-08T15:58:44.936 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.6616000000000001e-05 2020-09-08T15:58:44.936 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.937 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.937 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:44.937 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.429704+0000", 2020-09-08T15:58:44.937 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011453199999999999 2020-09-08T15:58:44.937 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.938 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.938 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:44.938 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.429819+0000", 2020-09-08T15:58:44.938 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0995999999999997e-05 2020-09-08T15:58:44.938 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.938 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.939 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:44.939 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.429880+0000", 2020-09-08T15:58:44.939 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.008092106 2020-09-08T15:58:44.939 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.939 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.940 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:58:44.940 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.437972+0000", 2020-09-08T15:58:44.940 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0665999999999998e-05 2020-09-08T15:58:44.940 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.940 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.941 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:58:44.941 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:52:38.438032+0000", 2020-09-08T15:58:44.941 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.14981187600000001 2020-09-08T15:58:44.941 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:44.942 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:44.942 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:44.942 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.942 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.942 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.15771.0:167294 5.c e437/396 5:352b230d:::benchmark_data_smithi166_37820_object22872:head v 437'10377, mlcod=437'10375)", 2020-09-08T15:58:44.942 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:53:49.741939+0000", 2020-09-08T15:58:44.943 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 295.13324119499998, 2020-09-08T15:58:44.943 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.047426477000000002, 2020-09-08T15:58:44.943 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:44.943 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:44.943 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:44.944 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.944 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:44.944 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:49.741939+0000", 2020-09-08T15:58:44.944 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:44.944 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.945 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.945 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:44.945 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:49.741939+0000", 2020-09-08T15:58:44.945 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.0832000000000002e-05 2020-09-08T15:58:44.945 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.946 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.946 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:44.946 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:49.742010+0000", 2020-09-08T15:58:44.946 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.9199999999999999e-06 2020-09-08T15:58:44.946 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.946 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.947 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:44.947 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:49.742020+0000", 2020-09-08T15:58:44.947 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6008e-05 2020-09-08T15:58:44.947 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.947 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.948 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:44.948 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:49.742036+0000", 2020-09-08T15:58:44.948 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.61e-06 2020-09-08T15:58:44.948 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.948 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.949 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:44.949 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:49.742044+0000", 2020-09-08T15:58:44.949 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.021576015 2020-09-08T15:58:44.949 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.949 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.950 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:44.950 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:49.763620+0000", 2020-09-08T15:58:44.950 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.1784000000000002e-05 2020-09-08T15:58:44.950 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.950 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.951 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:44.951 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:49.763702+0000", 2020-09-08T15:58:44.951 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.025588234000000001 2020-09-08T15:58:44.951 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.952 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.952 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:58:44.952 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:49.789290+0000", 2020-09-08T15:58:44.952 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.6074000000000004e-05 2020-09-08T15:58:44.952 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.952 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.953 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:58:44.953 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:49.789366+0000", 2020-09-08T15:58:44.953 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.047426477000000002 2020-09-08T15:58:44.953 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:44.953 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:44.954 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:44.954 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.954 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.954 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.15771.0:172452 5.c e439/396 5:38111611:::benchmark_data_smithi166_37820_object28030:head v 439'10711, mlcod=439'10707)", 2020-09-08T15:58:44.954 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:53:54.845966+0000", 2020-09-08T15:58:44.955 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 290.02921479999998, 2020-09-08T15:58:44.955 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.050158055999999999, 2020-09-08T15:58:44.955 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:44.955 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:44.955 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:44.956 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.956 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:44.956 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.845966+0000", 2020-09-08T15:58:44.956 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:44.956 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.957 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.957 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:44.957 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.845966+0000", 2020-09-08T15:58:44.957 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000101652 2020-09-08T15:58:44.957 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.958 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.958 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:44.958 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.846067+0000", 2020-09-08T15:58:44.958 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2852e-05 2020-09-08T15:58:44.958 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.959 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.959 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:44.959 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.846080+0000", 2020-09-08T15:58:44.959 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.0873000000000002e-05 2020-09-08T15:58:44.959 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.959 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.960 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:44.960 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.846101+0000", 2020-09-08T15:58:44.960 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4885999999999999e-05 2020-09-08T15:58:44.960 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.960 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.961 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:44.961 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.846116+0000", 2020-09-08T15:58:44.961 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.001596126 2020-09-08T15:58:44.961 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.961 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.962 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:44.962 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.847712+0000", 2020-09-08T15:58:44.962 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.7428e-05 2020-09-08T15:58:44.962 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.962 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.963 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:44.963 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.847730+0000", 2020-09-08T15:58:44.963 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.048316977999999997 2020-09-08T15:58:44.963 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.963 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.963 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:58:44.964 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.896047+0000", 2020-09-08T15:58:44.964 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.7261000000000004e-05 2020-09-08T15:58:44.964 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.964 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.964 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:58:44.965 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:53:54.896124+0000", 2020-09-08T15:58:44.965 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.050158055999999999 2020-09-08T15:58:44.965 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:44.965 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:44.965 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:44.966 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.966 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.966 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.15771.0:280226 5.f e468/396 5:fb8116d7:::benchmark_data_smithi166_37820_object135804:head v 468'17551, mlcod=468'17539)", 2020-09-08T15:58:44.966 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:55:48.167862+0000", 2020-09-08T15:58:44.966 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 176.70731904499999, 2020-09-08T15:58:44.967 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.048041084999999997, 2020-09-08T15:58:44.967 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:44.967 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:44.967 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:44.967 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.967 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:44.968 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.167862+0000", 2020-09-08T15:58:44.968 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:44.968 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.968 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.968 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:44.969 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.167862+0000", 2020-09-08T15:58:44.969 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.0130000000000006e-05 2020-09-08T15:58:44.969 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.969 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.969 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:44.970 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.167952+0000", 2020-09-08T15:58:44.970 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4739999999999994e-06 2020-09-08T15:58:44.970 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.970 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.970 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:44.971 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.167961+0000", 2020-09-08T15:58:44.971 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5891000000000001e-05 2020-09-08T15:58:44.971 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.971 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.971 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:44.972 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.167977+0000", 2020-09-08T15:58:44.972 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.0119999999999996e-06 2020-09-08T15:58:44.972 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.972 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.972 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:44.972 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.167982+0000", 2020-09-08T15:58:44.973 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.028972255999999998 2020-09-08T15:58:44.973 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.973 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.973 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:44.973 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.196954+0000", 2020-09-08T15:58:44.974 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.1304000000000001e-05 2020-09-08T15:58:44.974 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.974 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.974 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:44.974 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.196976+0000", 2020-09-08T15:58:44.975 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.018846521000000001 2020-09-08T15:58:44.975 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.975 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.975 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:58:44.975 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.215822+0000", 2020-09-08T15:58:44.976 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.0496999999999999e-05 2020-09-08T15:58:44.976 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.976 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.976 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:58:44.976 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.215903+0000", 2020-09-08T15:58:44.976 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.048041084999999997 2020-09-08T15:58:44.977 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:44.977 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:44.977 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:44.977 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.977 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.978 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.15771.0:280229 5.f e468/396 5:f5909bf4:::benchmark_data_smithi166_37820_object135807:head v 468'17552, mlcod=468'17539)", 2020-09-08T15:58:44.978 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:55:48.168039+0000", 2020-09-08T15:58:44.978 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 176.707141828, 2020-09-08T15:58:44.978 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.047914249999999999, 2020-09-08T15:58:44.978 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:44.979 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:44.979 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:44.979 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.979 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:44.979 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.168039+0000", 2020-09-08T15:58:44.980 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:44.980 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.980 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.980 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:44.980 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.168039+0000", 2020-09-08T15:58:44.980 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4940999999999998e-05 2020-09-08T15:58:44.981 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.981 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.981 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:44.981 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.168134+0000", 2020-09-08T15:58:44.981 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.7319999999999993e-06 2020-09-08T15:58:44.982 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.982 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.982 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:44.982 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.168143+0000", 2020-09-08T15:58:44.982 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6574000000000001e-05 2020-09-08T15:58:44.983 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.983 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.983 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:44.983 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.168160+0000", 2020-09-08T15:58:44.983 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.0649999999999998e-06 2020-09-08T15:58:44.984 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.984 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.984 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:44.984 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.168165+0000", 2020-09-08T15:58:44.984 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.032101829999999998 2020-09-08T15:58:44.984 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.985 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.985 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:44.985 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.200267+0000", 2020-09-08T15:58:44.985 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.1605000000000004e-05 2020-09-08T15:58:44.985 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.986 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.986 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:44.986 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.200329+0000", 2020-09-08T15:58:44.986 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.015578609 2020-09-08T15:58:44.986 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.987 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.987 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:58:44.987 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.215907+0000", 2020-09-08T15:58:44.987 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.5893999999999998e-05 2020-09-08T15:58:44.987 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.987 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.988 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:58:44.988 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.215953+0000", 2020-09-08T15:58:44.988 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.047914249999999999 2020-09-08T15:58:44.988 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:44.988 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:44.989 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:44.989 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.989 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.989 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.15771.0:280232 5.f e468/396 5:f3a861a2:::benchmark_data_smithi166_37820_object135810:head v 468'17553, mlcod=468'17539)", 2020-09-08T15:58:44.989 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:55:48.168222+0000", 2020-09-08T15:58:44.990 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 176.70695841099999, 2020-09-08T15:58:44.990 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.047787893999999997, 2020-09-08T15:58:44.990 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:44.990 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:44.990 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:44.991 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.991 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:44.991 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.168222+0000", 2020-09-08T15:58:44.991 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:44.991 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.992 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.992 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:44.992 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.168222+0000", 2020-09-08T15:58:44.992 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.0364000000000004e-05 2020-09-08T15:58:44.992 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.992 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.993 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:44.993 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.168313+0000", 2020-09-08T15:58:44.993 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.5200000000000003e-06 2020-09-08T15:58:44.993 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.994 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.994 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:44.994 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.168322+0000", 2020-09-08T15:58:44.994 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6345e-05 2020-09-08T15:58:44.995 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.995 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.995 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:44.995 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.168338+0000", 2020-09-08T15:58:44.995 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.023e-06 2020-09-08T15:58:44.995 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.996 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.996 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:44.996 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.168343+0000", 2020-09-08T15:58:44.996 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.037991434999999997 2020-09-08T15:58:44.996 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.997 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.997 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:44.997 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.206335+0000", 2020-09-08T15:58:44.997 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.6260000000000001e-05 2020-09-08T15:58:44.997 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.998 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.998 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:44.998 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.206381+0000", 2020-09-08T15:58:44.998 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0095738579999999993 2020-09-08T15:58:44.998 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.999 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:44.999 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:58:44.999 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.215955+0000", 2020-09-08T15:58:44.999 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.5089000000000001e-05 2020-09-08T15:58:44.999 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:44.999 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.000 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:58:45.000 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:55:48.216010+0000", 2020-09-08T15:58:45.000 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.047787893999999997 2020-09-08T15:58:45.000 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.000 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:45.001 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.001 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.001 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.001 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.16014.0:61940 6.e e497/478 6:7334a779:::benchmark_data_smithi166_43187_object61939:head v 497'3916, mlcod=497'3914)", 2020-09-08T15:58:45.001 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:57:11.434620+0000", 2020-09-08T15:58:45.002 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 93.440560226000002, 2020-09-08T15:58:45.002 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.133169975, 2020-09-08T15:58:45.002 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:45.002 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:45.002 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:45.003 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.003 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:45.003 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:11.434620+0000", 2020-09-08T15:58:45.003 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:45.003 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.003 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.004 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:45.004 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:11.434620+0000", 2020-09-08T15:58:45.004 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.0835999999999997e-05 2020-09-08T15:58:45.004 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.004 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.005 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:45.005 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:11.434701+0000", 2020-09-08T15:58:45.005 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.5900000000000002e-06 2020-09-08T15:58:45.005 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.005 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.006 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:45.006 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:11.434709+0000", 2020-09-08T15:58:45.006 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1885000000000001e-05 2020-09-08T15:58:45.006 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.006 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.007 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:45.007 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:11.434721+0000", 2020-09-08T15:58:45.007 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3060000000000003e-06 2020-09-08T15:58:45.007 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.007 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.007 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:45.008 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:11.434727+0000", 2020-09-08T15:58:45.008 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.116080618 2020-09-08T15:58:45.008 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.008 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.008 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:45.009 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:11.550808+0000", 2020-09-08T15:58:45.009 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.5503000000000001e-05 2020-09-08T15:58:45.009 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.009 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.009 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:45.010 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:11.550853+0000", 2020-09-08T15:58:45.010 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.01684397 2020-09-08T15:58:45.010 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.010 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.010 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:58:45.011 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:11.567697+0000", 2020-09-08T15:58:45.011 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3267e-05 2020-09-08T15:58:45.011 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.011 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.011 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:58:45.011 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:11.567790+0000", 2020-09-08T15:58:45.012 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.133169975 2020-09-08T15:58:45.012 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.012 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:45.012 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.012 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.013 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.013 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.16014.0:110635 6.2 e507/478 6:4251807a:::benchmark_data_smithi166_43187_object110634:head v 507'6813, mlcod=507'6811)", 2020-09-08T15:58:45.013 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:57:49.905978+0000", 2020-09-08T15:58:45.013 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 54.969203043, 2020-09-08T15:58:45.013 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.049534450000000001, 2020-09-08T15:58:45.014 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:45.014 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:45.014 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:45.014 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.014 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:45.014 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:49.905978+0000", 2020-09-08T15:58:45.015 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:45.015 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.015 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.015 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:45.015 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:49.905978+0000", 2020-09-08T15:58:45.016 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000117927 2020-09-08T15:58:45.016 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.016 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.016 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:45.016 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:49.906095+0000", 2020-09-08T15:58:45.017 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3309999999999993e-06 2020-09-08T15:58:45.017 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.017 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.017 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:45.017 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:49.906105+0000", 2020-09-08T15:58:45.018 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4072e-05 2020-09-08T15:58:45.018 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.018 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.018 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:45.018 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:49.906119+0000", 2020-09-08T15:58:45.018 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.3379999999999999e-06 2020-09-08T15:58:45.019 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.019 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.019 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:45.019 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:49.906126+0000", 2020-09-08T15:58:45.019 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.7015000000000002e-05 2020-09-08T15:58:45.020 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.020 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.020 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:45.020 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:49.906183+0000", 2020-09-08T15:58:45.020 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.1052e-05 2020-09-08T15:58:45.021 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.021 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.021 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:45.021 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:49.906214+0000", 2020-09-08T15:58:45.021 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.049218779999999997 2020-09-08T15:58:45.022 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.022 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.022 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:58:45.022 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:49.955433+0000", 2020-09-08T15:58:45.022 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.8935000000000002e-05 2020-09-08T15:58:45.022 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.023 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.023 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:58:45.023 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:49.955512+0000", 2020-09-08T15:58:45.023 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.049534450000000001 2020-09-08T15:58:45.023 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.024 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:45.024 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.024 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.024 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.024 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.16014.0:120208 6.2 e509/478 6:42ee3933:::benchmark_data_smithi166_43187_object439:head v 509'7398, mlcod=509'7395)", 2020-09-08T15:58:45.025 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:57:59.909353+0000", 2020-09-08T15:58:45.025 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 44.965827826000002, 2020-09-08T15:58:45.025 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.053755075999999999, 2020-09-08T15:58:45.025 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:45.026 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:45.026 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:45.026 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.027 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:45.027 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.909353+0000", 2020-09-08T15:58:45.027 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:45.027 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.028 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.028 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:45.028 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.909353+0000", 2020-09-08T15:58:45.028 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.4193e-05 2020-09-08T15:58:45.028 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.029 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.029 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:45.029 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.909417+0000", 2020-09-08T15:58:45.029 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1823e-05 2020-09-08T15:58:45.029 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.030 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.030 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:45.030 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.909429+0000", 2020-09-08T15:58:45.030 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6433999999999999e-05 2020-09-08T15:58:45.030 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.031 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.031 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:45.031 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.909445+0000", 2020-09-08T15:58:45.031 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.9499999999999996e-06 2020-09-08T15:58:45.031 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.032 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.032 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:45.032 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.909455+0000", 2020-09-08T15:58:45.032 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014883499999999999 2020-09-08T15:58:45.032 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.033 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.033 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:45.033 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.909604+0000", 2020-09-08T15:58:45.033 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000142106 2020-09-08T15:58:45.034 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.034 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.034 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:45.034 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.909746+0000", 2020-09-08T15:58:45.035 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.053252165999999997 2020-09-08T15:58:45.035 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.035 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.035 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:58:45.036 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.962998+0000", 2020-09-08T15:58:45.036 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010956899999999999 2020-09-08T15:58:45.036 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.037 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.037 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:58:45.037 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.963108+0000", 2020-09-08T15:58:45.037 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.053755075999999999 2020-09-08T15:58:45.037 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.038 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:45.038 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.038 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.038 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.039 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.16014.0:120222 6.e e509/478 6:70e89a10:::benchmark_data_smithi166_43187_object453:head v 509'7532, mlcod=509'7529)", 2020-09-08T15:58:45.039 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:57:59.911195+0000", 2020-09-08T15:58:45.039 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 44.963985831000002, 2020-09-08T15:58:45.039 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.051332317000000002, 2020-09-08T15:58:45.039 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:45.040 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:45.040 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:45.040 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.040 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:45.041 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.911195+0000", 2020-09-08T15:58:45.041 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:45.041 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.041 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.041 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:45.042 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.911195+0000", 2020-09-08T15:58:45.042 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4614e-05 2020-09-08T15:58:45.042 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.042 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.042 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:45.043 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.911279+0000", 2020-09-08T15:58:45.043 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2568000000000001e-05 2020-09-08T15:58:45.043 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.043 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.044 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:45.045 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.911292+0000", 2020-09-08T15:58:45.045 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.2663999999999998e-05 2020-09-08T15:58:45.045 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.045 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.046 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:45.046 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.911315+0000", 2020-09-08T15:58:45.047 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1301000000000001e-05 2020-09-08T15:58:45.047 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.047 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.047 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:45.047 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.911326+0000", 2020-09-08T15:58:45.048 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014761900000000001 2020-09-08T15:58:45.048 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.048 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.048 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:45.048 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.911474+0000", 2020-09-08T15:58:45.049 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.3145000000000002e-05 2020-09-08T15:58:45.049 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.049 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.049 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:45.049 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.911507+0000", 2020-09-08T15:58:45.050 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.050859711000000002 2020-09-08T15:58:45.050 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.050 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.050 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:58:45.050 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.962366+0000", 2020-09-08T15:58:45.051 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016069500000000001 2020-09-08T15:58:45.051 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.051 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.051 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:58:45.051 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.962527+0000", 2020-09-08T15:58:45.052 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.051332317000000002 2020-09-08T15:58:45.052 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.052 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:45.052 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.052 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.053 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.053 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.16014.0:120224 6.2 e509/478 6:43a1dbf5:::benchmark_data_smithi166_43187_object455:head v 509'7399, mlcod=509'7396)", 2020-09-08T15:58:45.053 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:57:59.911822+0000", 2020-09-08T15:58:45.053 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 44.963358182999997, 2020-09-08T15:58:45.054 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.051353095000000001, 2020-09-08T15:58:45.054 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:45.054 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:45.054 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:45.055 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.055 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:45.055 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.911822+0000", 2020-09-08T15:58:45.056 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:45.056 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.056 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.057 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:45.057 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.911822+0000", 2020-09-08T15:58:45.057 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.5167999999999997e-05 2020-09-08T15:58:45.057 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.057 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.058 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:45.058 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.911888+0000", 2020-09-08T15:58:45.058 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2400000000000007e-06 2020-09-08T15:58:45.058 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.058 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.059 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:45.059 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.911896+0000", 2020-09-08T15:58:45.059 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3953e-05 2020-09-08T15:58:45.059 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.063 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.064 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:45.064 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.911910+0000", 2020-09-08T15:58:45.064 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.8770000000000004e-06 2020-09-08T15:58:45.064 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.065 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.065 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:45.065 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.911917+0000", 2020-09-08T15:58:45.065 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0016999960000000001 2020-09-08T15:58:45.065 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.066 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.066 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:45.066 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.913617+0000", 2020-09-08T15:58:45.066 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00084890400000000004 2020-09-08T15:58:45.066 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.067 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.067 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:45.067 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.914466+0000", 2020-09-08T15:58:45.068 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.048646271999999997 2020-09-08T15:58:45.068 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.068 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.068 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:58:45.068 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.963112+0000", 2020-09-08T15:58:45.069 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3684999999999998e-05 2020-09-08T15:58:45.069 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.069 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.069 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:58:45.069 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:59.963176+0000", 2020-09-08T15:58:45.070 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.051353095000000001 2020-09-08T15:58:45.070 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.070 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:45.070 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.070 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.071 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.071 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.16014.0:149025 6.2 e517/478 6:47a46973:::benchmark_data_smithi166_43187_object29256:head v 517'9152, mlcod=517'9150)", 2020-09-08T15:58:45.071 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:58:27.346188+0000", 2020-09-08T15:58:45.071 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 17.528993021000002, 2020-09-08T15:58:45.071 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13718873100000001, 2020-09-08T15:58:45.072 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:45.072 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:45.072 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:45.072 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.072 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:45.073 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.346188+0000", 2020-09-08T15:58:45.073 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:45.073 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.073 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.073 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:45.073 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.346188+0000", 2020-09-08T15:58:45.074 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.8723000000000002e-05 2020-09-08T15:58:45.074 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.074 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.074 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:45.074 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.346226+0000", 2020-09-08T15:58:45.075 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.7969999999999997e-06 2020-09-08T15:58:45.075 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.075 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.075 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:45.075 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.346232+0000", 2020-09-08T15:58:45.076 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.0759999999999992e-06 2020-09-08T15:58:45.076 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.076 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.076 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:45.077 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.346241+0000", 2020-09-08T15:58:45.077 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.9470000000000002e-06 2020-09-08T15:58:45.077 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.078 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.078 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:45.078 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.346246+0000", 2020-09-08T15:58:45.078 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.9805000000000003e-05 2020-09-08T15:58:45.078 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.079 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.079 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:45.079 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.346286+0000", 2020-09-08T15:58:45.079 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.9362000000000001e-05 2020-09-08T15:58:45.080 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.080 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.080 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:45.080 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.346315+0000", 2020-09-08T15:58:45.081 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0048466179999999996 2020-09-08T15:58:45.081 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.081 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.081 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:58:45.081 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.351162+0000", 2020-09-08T15:58:45.082 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13221440300000001 2020-09-08T15:58:45.082 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.082 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.082 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:58:45.082 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.483376+0000", 2020-09-08T15:58:45.083 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13718873100000001 2020-09-08T15:58:45.083 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.083 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:45.083 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.083 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.084 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.084 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.16014.0:149028 6.6 e517/478 6:61f6bd6a:::benchmark_data_smithi166_43187_object29259:head v 517'9098, mlcod=517'9096)", 2020-09-08T15:58:45.084 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:58:27.347122+0000", 2020-09-08T15:58:45.084 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 17.528058689000002, 2020-09-08T15:58:45.084 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13626903700000001, 2020-09-08T15:58:45.085 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:45.085 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:45.085 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:45.085 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.085 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:45.085 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.347122+0000", 2020-09-08T15:58:45.086 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:45.086 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.086 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.086 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:45.086 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.347122+0000", 2020-09-08T15:58:45.087 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.1093000000000005e-05 2020-09-08T15:58:45.087 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.087 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.087 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:45.088 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.347193+0000", 2020-09-08T15:58:45.088 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.1260000000000004e-06 2020-09-08T15:58:45.088 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.088 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.089 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:45.089 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.347202+0000", 2020-09-08T15:58:45.089 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4791e-05 2020-09-08T15:58:45.089 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.090 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.090 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:45.090 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.347217+0000", 2020-09-08T15:58:45.091 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.2179999999999996e-06 2020-09-08T15:58:45.091 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.091 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.091 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:45.091 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.347223+0000", 2020-09-08T15:58:45.092 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.0142999999999994e-05 2020-09-08T15:58:45.092 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.092 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.092 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:45.093 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.347313+0000", 2020-09-08T15:58:45.093 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.1313000000000001e-05 2020-09-08T15:58:45.093 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.093 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.093 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:45.094 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.347335+0000", 2020-09-08T15:58:45.094 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13597110000000001 2020-09-08T15:58:45.094 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.094 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.094 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:58:45.095 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.483306+0000", 2020-09-08T15:58:45.095 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.5253000000000005e-05 2020-09-08T15:58:45.095 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.096 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.096 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:58:45.096 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.483391+0000", 2020-09-08T15:58:45.096 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13626903700000001 2020-09-08T15:58:45.096 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.096 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:45.097 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.097 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.097 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.097 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.16014.0:149029 6.2 e517/478 6:44ad2275:::benchmark_data_smithi166_43187_object29260:head v 517'9153, mlcod=517'9150)", 2020-09-08T15:58:45.098 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:58:27.351032+0000", 2020-09-08T15:58:45.098 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 17.524148874000002, 2020-09-08T15:58:45.098 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.149478582, 2020-09-08T15:58:45.099 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:45.099 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:45.099 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:45.099 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.099 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:45.100 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.351032+0000", 2020-09-08T15:58:45.100 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:45.100 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.100 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.101 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:45.101 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.351032+0000", 2020-09-08T15:58:45.101 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13237591900000001 2020-09-08T15:58:45.101 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.101 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.102 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:45.102 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.483408+0000", 2020-09-08T15:58:45.102 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.9361999999999999e-05 2020-09-08T15:58:45.102 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.102 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.103 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:45.103 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.483427+0000", 2020-09-08T15:58:45.103 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.0403e-05 2020-09-08T15:58:45.103 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.103 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.104 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:45.104 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.483457+0000", 2020-09-08T15:58:45.104 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3135999999999999e-05 2020-09-08T15:58:45.104 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.104 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.105 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:45.105 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.483471+0000", 2020-09-08T15:58:45.105 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9599000000000003e-05 2020-09-08T15:58:45.105 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.105 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.106 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:45.106 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.483530+0000", 2020-09-08T15:58:45.106 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.4850000000000001e-05 2020-09-08T15:58:45.106 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.106 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.107 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:45.107 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.483555+0000", 2020-09-08T15:58:45.107 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.016824848999999999 2020-09-08T15:58:45.107 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.107 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.108 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:58:45.108 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.500380+0000", 2020-09-08T15:58:45.108 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000130464 2020-09-08T15:58:45.108 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.108 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.108 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:58:45.109 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:27.500510+0000", 2020-09-08T15:58:45.109 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.149478582 2020-09-08T15:58:45.109 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.109 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:45.109 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.110 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.110 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.110 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.16014.0:162857 6.6 e521/478 6:6ed66810:::benchmark_data_smithi166_43187_object43088:head v 521'9917, mlcod=521'9912)", 2020-09-08T15:58:45.110 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:58:42.029543+0000", 2020-09-08T15:58:45.110 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 2.8456376780000001, 2020-09-08T15:58:45.111 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.052195449999999997, 2020-09-08T15:58:45.111 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:45.111 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:45.111 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:45.111 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.112 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:45.112 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:42.029543+0000", 2020-09-08T15:58:45.112 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:45.112 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.112 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.112 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:45.113 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:42.029543+0000", 2020-09-08T15:58:45.113 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.9166999999999995e-05 2020-09-08T15:58:45.113 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.113 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.113 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:45.114 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:42.029632+0000", 2020-09-08T15:58:45.114 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1557e-05 2020-09-08T15:58:45.114 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.114 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.114 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:45.115 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:42.029644+0000", 2020-09-08T15:58:45.115 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.8238000000000001e-05 2020-09-08T15:58:45.115 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.115 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.115 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:45.116 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:42.029662+0000", 2020-09-08T15:58:45.116 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.1519999999999992e-06 2020-09-08T15:58:45.116 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.116 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.116 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:45.117 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:42.029670+0000", 2020-09-08T15:58:45.117 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.015748274999999999 2020-09-08T15:58:45.117 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.117 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.117 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:45.118 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:42.045418+0000", 2020-09-08T15:58:45.118 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.6774000000000001e-05 2020-09-08T15:58:45.118 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.118 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.118 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:45.119 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:42.045445+0000", 2020-09-08T15:58:45.119 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.036204801000000002 2020-09-08T15:58:45.119 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.119 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.119 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:58:45.119 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:42.081650+0000", 2020-09-08T15:58:45.120 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.8486000000000006e-05 2020-09-08T15:58:45.120 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.120 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.120 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:58:45.120 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:42.081738+0000", 2020-09-08T15:58:45.121 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.052195449999999997 2020-09-08T15:58:45.121 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.121 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:45.121 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.121 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.122 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.122 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.16014.0:162853 6.e e521/478 6:71a4c7b1:::benchmark_data_smithi166_43187_object43084:head v 521'10233, mlcod=521'10228)", 2020-09-08T15:58:45.122 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:58:42.034030+0000", 2020-09-08T15:58:45.122 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 2.8411510369999999, 2020-09-08T15:58:45.122 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.049845483000000003, 2020-09-08T15:58:45.123 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:45.123 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:58:45.123 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:45.123 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.123 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:45.124 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:42.034030+0000", 2020-09-08T15:58:45.124 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:45.124 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.124 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.124 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:45.124 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:42.034030+0000", 2020-09-08T15:58:45.125 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.1276e-05 2020-09-08T15:58:45.125 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.125 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.125 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:45.126 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:42.034101+0000", 2020-09-08T15:58:45.126 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.1109999999999997e-06 2020-09-08T15:58:45.126 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.126 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.127 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:45.127 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:42.034110+0000", 2020-09-08T15:58:45.127 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4654999999999999e-05 2020-09-08T15:58:45.127 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.127 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.128 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:45.128 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:42.034125+0000", 2020-09-08T15:58:45.128 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.4110000000000001e-06 2020-09-08T15:58:45.128 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.129 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.129 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:45.129 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:42.034131+0000", 2020-09-08T15:58:45.129 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.012444584999999999 2020-09-08T15:58:45.129 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.130 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.130 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:45.130 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:42.046576+0000", 2020-09-08T15:58:45.130 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.1918000000000002e-05 2020-09-08T15:58:45.130 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.131 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.131 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:45.131 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:42.046608+0000", 2020-09-08T15:58:45.131 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.037196053999999999 2020-09-08T15:58:45.131 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.132 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.132 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T15:58:45.132 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:42.083804+0000", 2020-09-08T15:58:45.132 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.1472999999999998e-05 2020-09-08T15:58:45.132 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:45.132 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:45.133 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T15:58:45.133 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:42.083875+0000", 2020-09-08T15:58:45.133 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.049845483000000003 2020-09-08T15:58:45.133 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.133 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:45.134 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.134 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:45.134 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:45.134 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:58:45.136 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:58:45.136 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:58:44 smithi055 bash[19003]: debug 2020-09-08T15:58:44.433+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:45.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:44 smithi104 bash[9203]: cluster 2020-09-08T15:58:43.332192+0000 osd.1 (osd.1) 104 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:45.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:44 smithi104 bash[9203]: cluster 2020-09-08T15:58:43.332298+0000 osd.1 (osd.1) 105 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:45.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:44 smithi104 bash[9203]: cluster 2020-09-08T15:58:43.903565+0000 mon.a (mon.0) 1795 : cluster [DBG] osdmap e522: 12 total, 12 up, 12 in 2020-09-08T15:58:45.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:44 smithi104 bash[9203]: cluster 2020-09-08T15:58:43.930454+0000 mgr.x (mgr.24118) 942 : cluster [DBG] pgmap v1380: 17 pgs: 17 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:58:45.238 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:58:45.291 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:44 smithi055 bash[9296]: cluster 2020-09-08T15:58:43.332192+0000 osd.1 (osd.1) 104 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:45.291 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:44 smithi055 bash[9296]: cluster 2020-09-08T15:58:43.332298+0000 osd.1 (osd.1) 105 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:45.291 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:44 smithi055 bash[9296]: cluster 2020-09-08T15:58:43.903565+0000 mon.a (mon.0) 1795 : cluster [DBG] osdmap e522: 12 total, 12 up, 12 in 2020-09-08T15:58:45.291 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:44 smithi055 bash[9296]: cluster 2020-09-08T15:58:43.930454+0000 mgr.x (mgr.24118) 942 : cluster [DBG] pgmap v1380: 17 pgs: 17 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:58:45.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:44 smithi166 bash[9446]: cluster 2020-09-08T15:58:43.332192+0000 osd.1 (osd.1) 104 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:45.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:44 smithi166 bash[9446]: cluster 2020-09-08T15:58:43.332298+0000 osd.1 (osd.1) 105 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:45.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:44 smithi166 bash[9446]: cluster 2020-09-08T15:58:43.903565+0000 mon.a (mon.0) 1795 : cluster [DBG] osdmap e522: 12 total, 12 up, 12 in 2020-09-08T15:58:45.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:44 smithi166 bash[9446]: cluster 2020-09-08T15:58:43.930454+0000 mgr.x (mgr.24118) 942 : cluster [DBG] pgmap v1380: 17 pgs: 17 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:58:45.368 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:58:44 smithi166 bash[16593]: debug 2020-09-08T15:58:44.889+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:45.924 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:58:45.985 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:58:46.056 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:58:46.097 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:58:46.147 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:58:46.159 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:58:46.162 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2020-09-08T15:58:46.167 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:45 smithi055 bash[9296]: cluster 2020-09-08T15:58:44.752365+0000 osd.5 (osd.5) 77 : cluster [DBG] 6.7 scrub starts 2020-09-08T15:58:46.168 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:58:46 smithi055 bash[11956]: debug 2020-09-08T15:58:46.057+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:46.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:45 smithi104 bash[9203]: cluster 2020-09-08T15:58:44.752365+0000 osd.5 (osd.5) 77 : cluster [DBG] 6.7 scrub starts 2020-09-08T15:58:46.303 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:58:46.346 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:58:46.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:45 smithi166 bash[9446]: cluster 2020-09-08T15:58:44.752365+0000 osd.5 (osd.5) 77 : cluster [DBG] 6.7 scrub starts 2020-09-08T15:58:46.413 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:46.658 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:58:46 smithi055 bash[19003]: debug 2020-09-08T15:58:46.297+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:47.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:46 smithi104 bash[9203]: cluster 2020-09-08T15:58:45.930861+0000 mgr.x (mgr.24118) 943 : cluster [DBG] pgmap v1381: 17 pgs: 17 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 751 op/s 2020-09-08T15:58:47.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:46 smithi166 bash[9446]: cluster 2020-09-08T15:58:45.930861+0000 mgr.x (mgr.24118) 943 : cluster [DBG] pgmap v1381: 17 pgs: 17 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 751 op/s 2020-09-08T15:58:47.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:46 smithi055 bash[9296]: cluster 2020-09-08T15:58:45.930861+0000 mgr.x (mgr.24118) 943 : cluster [DBG] pgmap v1381: 17 pgs: 17 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 751 op/s 2020-09-08T15:58:47.407 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:47.513 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:47.644 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:47.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:58:47 smithi104 bash[22305]: debug 2020-09-08T15:58:47.405+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:47.753 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:58:47.867 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:58:47 smithi166 bash[14254]: debug 2020-09-08T15:58:47.641+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:48.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:47 smithi104 bash[9203]: cluster 2020-09-08T15:58:46.359774+0000 osd.8 (osd.8) 69 : cluster [DBG] 6.1 deep-scrub starts 2020-09-08T15:58:48.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:47 smithi104 bash[9203]: cluster 2020-09-08T15:58:46.927076+0000 osd.6 (osd.6) 83 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:48.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:47 smithi104 bash[9203]: cluster 2020-09-08T15:58:46.927251+0000 osd.6 (osd.6) 84 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:48.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:47 smithi104 bash[9203]: audit 2020-09-08T15:58:46.933260+0000 mon.c (mon.2) 71 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:58:48.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:47 smithi104 bash[9203]: audit 2020-09-08T15:58:46.936557+0000 mon.a (mon.0) 1796 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:58:48.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:47 smithi104 bash[9203]: audit 2020-09-08T15:58:46.943787+0000 mon.a (mon.0) 1797 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:58:48.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:47 smithi104 bash[9203]: cluster 2020-09-08T15:58:47.198541+0000 osd.5 (osd.5) 78 : cluster [DBG] 6.7 scrub ok 2020-09-08T15:58:48.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:47 smithi166 bash[9446]: cluster 2020-09-08T15:58:46.359774+0000 osd.8 (osd.8) 69 : cluster [DBG] 6.1 deep-scrub starts 2020-09-08T15:58:48.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:47 smithi166 bash[9446]: cluster 2020-09-08T15:58:46.927076+0000 osd.6 (osd.6) 83 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:48.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:47 smithi166 bash[9446]: cluster 2020-09-08T15:58:46.927251+0000 osd.6 (osd.6) 84 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:48.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:47 smithi166 bash[9446]: audit 2020-09-08T15:58:46.933260+0000 mon.c (mon.2) 71 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:58:48.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:47 smithi166 bash[9446]: audit 2020-09-08T15:58:46.936557+0000 mon.a (mon.0) 1796 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:58:48.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:47 smithi166 bash[9446]: audit 2020-09-08T15:58:46.943787+0000 mon.a (mon.0) 1797 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:58:48.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:47 smithi166 bash[9446]: cluster 2020-09-08T15:58:47.198541+0000 osd.5 (osd.5) 78 : cluster [DBG] 6.7 scrub ok 2020-09-08T15:58:48.393 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:47 smithi055 bash[9296]: cluster 2020-09-08T15:58:46.359774+0000 osd.8 (osd.8) 69 : cluster [DBG] 6.1 deep-scrub starts 2020-09-08T15:58:48.393 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:47 smithi055 bash[9296]: cluster 2020-09-08T15:58:46.927076+0000 osd.6 (osd.6) 83 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:48.393 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:47 smithi055 bash[9296]: cluster 2020-09-08T15:58:46.927251+0000 osd.6 (osd.6) 84 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:48.393 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:47 smithi055 bash[9296]: audit 2020-09-08T15:58:46.933260+0000 mon.c (mon.2) 71 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:58:48.394 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:47 smithi055 bash[9296]: audit 2020-09-08T15:58:46.936557+0000 mon.a (mon.0) 1796 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T15:58:48.394 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:47 smithi055 bash[9296]: audit 2020-09-08T15:58:46.943787+0000 mon.a (mon.0) 1797 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T15:58:48.394 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:47 smithi055 bash[9296]: cluster 2020-09-08T15:58:47.198541+0000 osd.5 (osd.5) 78 : cluster [DBG] 6.7 scrub ok 2020-09-08T15:58:48.822 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:58:48.926 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:58:49.048 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:58:49.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:48 smithi166 bash[9446]: cluster 2020-09-08T15:58:47.931245+0000 mgr.x (mgr.24118) 944 : cluster [DBG] pgmap v1383: 17 pgs: 1 active+clean+scrubbing, 1 active+clean+scrubbing+deep, 15 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T15:58:49.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:48 smithi166 bash[9446]: cluster 2020-09-08T15:58:47.932254+0000 mon.a (mon.0) 1798 : cluster [DBG] osdmap e523: 12 total, 12 up, 12 in 2020-09-08T15:58:49.118 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:58:48 smithi166 bash[18960]: debug 2020-09-08T15:58:48.821+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:49.154 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:49.161 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:58:49 smithi104 bash[19922]: debug 2020-09-08T15:58:49.049+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:49.169 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:48 smithi104 bash[9203]: cluster 2020-09-08T15:58:47.931245+0000 mgr.x (mgr.24118) 944 : cluster [DBG] pgmap v1383: 17 pgs: 1 active+clean+scrubbing, 1 active+clean+scrubbing+deep, 15 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T15:58:49.169 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:48 smithi104 bash[9203]: cluster 2020-09-08T15:58:47.932254+0000 mon.a (mon.0) 1798 : cluster [DBG] osdmap e523: 12 total, 12 up, 12 in 2020-09-08T15:58:49.354 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:49.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:48 smithi055 bash[9296]: cluster 2020-09-08T15:58:47.931245+0000 mgr.x (mgr.24118) 944 : cluster [DBG] pgmap v1383: 17 pgs: 1 active+clean+scrubbing, 1 active+clean+scrubbing+deep, 15 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T15:58:49.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:48 smithi055 bash[9296]: cluster 2020-09-08T15:58:47.932254+0000 mon.a (mon.0) 1798 : cluster [DBG] osdmap e523: 12 total, 12 up, 12 in 2020-09-08T15:58:49.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:58:49 smithi104 bash[22305]: debug 2020-09-08T15:58:49.353+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:49.459 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:58:49.998 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:58:50.102 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:58:50.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:58:50 smithi055 bash[16646]: debug 2020-09-08T15:58:50.001+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:50.962 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:58:51.066 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:51.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:50 smithi055 bash[9296]: cluster 2020-09-08T15:58:49.794191+0000 osd.5 (osd.5) 79 : cluster [DBG] 6.d scrub starts 2020-09-08T15:58:51.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:50 smithi055 bash[9296]: cluster 2020-09-08T15:58:49.931708+0000 mgr.x (mgr.24118) 945 : cluster [DBG] pgmap v1384: 17 pgs: 1 active+clean+scrubbing, 1 active+clean+scrubbing+deep, 15 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 927 op/s 2020-09-08T15:58:51.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:50 smithi104 bash[9203]: cluster 2020-09-08T15:58:49.794191+0000 osd.5 (osd.5) 79 : cluster [DBG] 6.d scrub starts 2020-09-08T15:58:51.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:50 smithi104 bash[9203]: cluster 2020-09-08T15:58:49.931708+0000 mgr.x (mgr.24118) 945 : cluster [DBG] pgmap v1384: 17 pgs: 1 active+clean+scrubbing, 1 active+clean+scrubbing+deep, 15 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 927 op/s 2020-09-08T15:58:51.177 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:58:50 smithi104 bash[19922]: debug 2020-09-08T15:58:50.961+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:51.212 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:50 smithi166 bash[9446]: cluster 2020-09-08T15:58:49.794191+0000 osd.5 (osd.5) 79 : cluster [DBG] 6.d scrub starts 2020-09-08T15:58:51.212 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:50 smithi166 bash[9446]: cluster 2020-09-08T15:58:49.931708+0000 mgr.x (mgr.24118) 945 : cluster [DBG] pgmap v1384: 17 pgs: 1 active+clean+scrubbing, 1 active+clean+scrubbing+deep, 15 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 927 op/s 2020-09-08T15:58:51.212 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:58:51 smithi166 bash[11928]: debug 2020-09-08T15:58:51.209+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:51.216 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:51.248 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:58:51.248 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T15:58:51.248 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.248 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.16014.0:170515 6.a 6:5bb7b3bd:::benchmark_data_smithi166_43187_object50746:head [delete] snapc 0=[] ondisk+write+known_if_redirected e524)", 2020-09-08T15:58:51.249 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:58:51.240340+0000", 2020-09-08T15:58:51.249 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.012585692000000001, 2020-09-08T15:58:51.249 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.012622292, 2020-09-08T15:58:51.249 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:51.250 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:58:51.250 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:58:51.250 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.16014", 2020-09-08T15:58:51.250 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:51.250 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 170515 2020-09-08T15:58:51.251 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.251 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:51.251 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.251 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:51.251 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.240340+0000", 2020-09-08T15:58:51.252 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:51.252 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.252 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.253 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:51.253 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.240340+0000", 2020-09-08T15:58:51.253 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999623 2020-09-08T15:58:51.254 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.254 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.254 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:51.254 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.240303+0000", 2020-09-08T15:58:51.255 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4513999999999998e-05 2020-09-08T15:58:51.255 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.255 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.255 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:51.255 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.240387+0000", 2020-09-08T15:58:51.256 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3369e-05 2020-09-08T15:58:51.256 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.256 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.256 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:51.257 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.240400+0000", 2020-09-08T15:58:51.257 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.2790000000000004e-06 2020-09-08T15:58:51.257 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.257 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.257 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:51.258 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.240407+0000", 2020-09-08T15:58:51.258 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.003168854 2020-09-08T15:58:51.258 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.258 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.258 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:51.259 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.243576+0000", 2020-09-08T15:58:51.259 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000172974 2020-09-08T15:58:51.259 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.259 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.260 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:51.260 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.243749+0000", 2020-09-08T15:58:51.260 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000159381 2020-09-08T15:58:51.260 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.261 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.261 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 6", 2020-09-08T15:58:51.261 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.243908+0000", 2020-09-08T15:58:51.261 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0046253960000000004 2020-09-08T15:58:51.261 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.262 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.262 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:58:51.262 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.248533+0000", 2020-09-08T15:58:51.262 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0081930130000000007 2020-09-08T15:58:51.266 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:51.266 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:51.267 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:51.267 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.267 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.267 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.16014.0:170516 6.d 6:b44557e8:::benchmark_data_smithi166_43187_object50747:head [delete] snapc 0=[] ondisk+write+known_if_redirected e524)", 2020-09-08T15:58:51.268 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:58:51.244352+0000", 2020-09-08T15:58:51.268 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.0085743099999999999, 2020-09-08T15:58:51.268 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0087084329999999998, 2020-09-08T15:58:51.268 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:51.269 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:58:51.269 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:58:51.269 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.16014", 2020-09-08T15:58:51.270 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:51.270 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 170516 2020-09-08T15:58:51.270 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.271 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:51.271 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.272 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:51.273 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.244352+0000", 2020-09-08T15:58:51.273 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:51.274 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.274 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.274 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:51.274 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.244352+0000", 2020-09-08T15:58:51.275 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999958 2020-09-08T15:58:51.275 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.275 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.275 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:51.276 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.244310+0000", 2020-09-08T15:58:51.276 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.9779999999999998e-05 2020-09-08T15:58:51.276 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.276 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.277 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:51.277 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.244399+0000", 2020-09-08T15:58:51.277 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4867e-05 2020-09-08T15:58:51.277 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.277 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.278 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:51.278 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.244414+0000", 2020-09-08T15:58:51.278 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.4320000000000004e-06 2020-09-08T15:58:51.278 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.279 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.279 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:51.279 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.244421+0000", 2020-09-08T15:58:51.279 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0048194429999999996 2020-09-08T15:58:51.279 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.280 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.280 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:51.280 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.249240+0000", 2020-09-08T15:58:51.280 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000166797 2020-09-08T15:58:51.281 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.281 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.281 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:51.281 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.249407+0000", 2020-09-08T15:58:51.281 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017008299999999999 2020-09-08T15:58:51.282 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.282 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.282 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T15:58:51.282 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.249577+0000", 2020-09-08T15:58:51.283 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0052253289999999999 2020-09-08T15:58:51.283 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:51.283 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:51.285 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:51.285 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.285 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.286 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.16014.0:170521 6.a 6:568a5ebe:::benchmark_data_smithi166_43187_object50752:head [delete] snapc 0=[] ondisk+write+known_if_redirected e524)", 2020-09-08T15:58:51.287 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:58:51.249993+0000", 2020-09-08T15:58:51.287 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.0029333279999999998, 2020-09-08T15:58:51.287 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.003149579, 2020-09-08T15:58:51.287 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:51.288 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:58:51.288 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:58:51.288 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.16014", 2020-09-08T15:58:51.288 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:58:51.288 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 170521 2020-09-08T15:58:51.289 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.289 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:51.289 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.289 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:51.290 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.249993+0000", 2020-09-08T15:58:51.290 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:51.290 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.290 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.290 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:51.291 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.249993+0000", 2020-09-08T15:58:51.291 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999781 2020-09-08T15:58:51.291 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.291 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.292 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:51.292 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.249971+0000", 2020-09-08T15:58:51.292 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.905e-05 2020-09-08T15:58:51.292 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.292 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.293 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:51.293 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.250020+0000", 2020-09-08T15:58:51.293 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.5990000000000002e-06 2020-09-08T15:58:51.293 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.294 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.294 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:51.295 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.250028+0000", 2020-09-08T15:58:51.295 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.4750000000000004e-06 2020-09-08T15:58:51.295 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.295 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.296 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:51.296 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.250033+0000", 2020-09-08T15:58:51.296 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.4301999999999997e-05 2020-09-08T15:58:51.296 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.296 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.297 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:58:51.297 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.250077+0000", 2020-09-08T15:58:51.297 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000116229 2020-09-08T15:58:51.297 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.298 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.298 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:58:51.298 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.250193+0000", 2020-09-08T15:58:51.298 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4253999999999997e-05 2020-09-08T15:58:51.299 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.299 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.299 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 6", 2020-09-08T15:58:51.299 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.250277+0000", 2020-09-08T15:58:51.299 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00028477599999999999 2020-09-08T15:58:51.300 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:51.300 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:51.300 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:51.300 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.300 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.301 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop_reply(client.16014.0:170515 6.a e524/478)", 2020-09-08T15:58:51.301 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:58:51.250501+0000", 2020-09-08T15:58:51.301 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.002425483, 2020-09-08T15:58:51.301 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0027197200000000001, 2020-09-08T15:58:51.302 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:58:51.302 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "queued for pg", 2020-09-08T15:58:51.302 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:58:51.302 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.303 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:58:51.303 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.250501+0000", 2020-09-08T15:58:51.303 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:58:51.303 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.303 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.304 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:58:51.304 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.250501+0000", 2020-09-08T15:58:51.305 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.3029999999999999e-05 2020-09-08T15:58:51.305 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.305 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.305 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:58:51.306 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.250554+0000", 2020-09-08T15:58:51.306 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.174e-06 2020-09-08T15:58:51.306 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.306 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.306 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:58:51.307 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.250562+0000", 2020-09-08T15:58:51.307 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4270000000000001e-05 2020-09-08T15:58:51.307 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.307 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.308 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:58:51.308 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.250576+0000", 2020-09-08T15:58:51.308 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.716e-06 2020-09-08T15:58:51.308 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:58:51.309 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:58:51.309 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:58:51.309 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:51.250582+0000", 2020-09-08T15:58:51.309 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.119e-05 2020-09-08T15:58:51.310 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:51.310 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:58:51.310 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:51.311 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:58:51.311 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2020-09-08T15:58:51.311 INFO:teuthology.orchestra.run.smithi055.stdout: "num_ops": 4 2020-09-08T15:58:51.311 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:58:51.320 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:51.447 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:51.552 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:51.561 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:58:51 smithi166 bash[11928]: debug 2020-09-08T15:58:51.445+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:51.665 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:51.772 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:51.867 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:58:51 smithi166 bash[14254]: debug 2020-09-08T15:58:51.661+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:52.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:51 smithi055 bash[9296]: cluster 2020-09-08T15:58:50.796889+0000 osd.11 (osd.11) 88 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:52.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:51 smithi055 bash[9296]: cluster 2020-09-08T15:58:50.796985+0000 osd.11 (osd.11) 89 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:52.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:51 smithi055 bash[9296]: cluster 2020-09-08T15:58:50.894545+0000 mon.a (mon.0) 1799 : cluster [DBG] osdmap e524: 12 total, 12 up, 12 in 2020-09-08T15:58:52.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:51 smithi055 bash[9296]: cluster 2020-09-08T15:58:51.142259+0000 osd.5 (osd.5) 80 : cluster [DBG] 6.d scrub ok 2020-09-08T15:58:52.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:51 smithi104 bash[9203]: cluster 2020-09-08T15:58:50.796889+0000 osd.11 (osd.11) 88 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:52.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:51 smithi104 bash[9203]: cluster 2020-09-08T15:58:50.796985+0000 osd.11 (osd.11) 89 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:52.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:51 smithi104 bash[9203]: cluster 2020-09-08T15:58:50.894545+0000 mon.a (mon.0) 1799 : cluster [DBG] osdmap e524: 12 total, 12 up, 12 in 2020-09-08T15:58:52.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:51 smithi104 bash[9203]: cluster 2020-09-08T15:58:51.142259+0000 osd.5 (osd.5) 80 : cluster [DBG] 6.d scrub ok 2020-09-08T15:58:52.288 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:52.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:51 smithi166 bash[9446]: cluster 2020-09-08T15:58:50.796889+0000 osd.11 (osd.11) 88 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:52.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:51 smithi166 bash[9446]: cluster 2020-09-08T15:58:50.796985+0000 osd.11 (osd.11) 89 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:52.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:51 smithi166 bash[9446]: cluster 2020-09-08T15:58:50.894545+0000 mon.a (mon.0) 1799 : cluster [DBG] osdmap e524: 12 total, 12 up, 12 in 2020-09-08T15:58:52.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:51 smithi166 bash[9446]: cluster 2020-09-08T15:58:51.142259+0000 osd.5 (osd.5) 80 : cluster [DBG] 6.d scrub ok 2020-09-08T15:58:52.404 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:58:52.411 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2020-09-08T15:58:52.532 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:58:52.539 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:58:52 smithi055 bash[11956]: debug 2020-09-08T15:58:52.533+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:52.540 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:58:52 smithi055 bash[14301]: debug 2020-09-08T15:58:52.285+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:52.614 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:58:52.640 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:58:53.150 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:52 smithi104 bash[9203]: cluster 2020-09-08T15:58:51.932083+0000 mgr.x (mgr.24118) 946 : cluster [DBG] pgmap v1386: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 787 op/s 2020-09-08T15:58:53.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:52 smithi055 bash[9296]: cluster 2020-09-08T15:58:51.932083+0000 mgr.x (mgr.24118) 946 : cluster [DBG] pgmap v1386: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 787 op/s 2020-09-08T15:58:53.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:52 smithi166 bash[9446]: cluster 2020-09-08T15:58:51.932083+0000 mgr.x (mgr.24118) 946 : cluster [DBG] pgmap v1386: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 787 op/s 2020-09-08T15:58:53.403 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:58:53.428 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:58:53.458 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:58:53.501 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:58:53.507 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:53.540 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:58:53.575 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:58:53.606 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:58:53.646 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:58:53.652 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:58:53.662 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:58:53 smithi055 bash[16646]: debug 2020-09-08T15:58:53.405+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:53.685 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:58:53.713 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:58:53.743 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:58:53.750 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:58:53.781 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:58:53.824 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:58:53.826 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:58:53.826 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:58:53.827 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:58:53.827 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:58:53.827 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:58:53.827 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:58:53.827 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:58:53.828 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:58:53.829 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:58:53.829 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:58:53.829 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:58:53.830 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:58:53.871 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:58:53.872 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:58:53 smithi104 bash[22305]: debug 2020-09-08T15:58:53.641+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:53.914 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:58:53.981 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:58:54.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:53 smithi055 bash[9296]: cluster 2020-09-08T15:58:52.691872+0000 osd.7 (osd.7) 109 : cluster [DBG] 6.8 deep-scrub starts 2020-09-08T15:58:54.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:53 smithi055 bash[9296]: audit 2020-09-08T15:58:53.402829+0000 mon.b (mon.1) 107 : audit [DBG] from='client.? v1:172.21.15.104:0/1191393008' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:58:54.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:53 smithi055 bash[9296]: audit 2020-09-08T15:58:53.701442+0000 mon.c (mon.2) 72 : audit [INF] from='client.? v1:172.21.15.104:0/144181256' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:58:54.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:53 smithi055 bash[9296]: audit 2020-09-08T15:58:53.704397+0000 mon.a (mon.0) 1800 : audit [INF] from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:58:54.162 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:53 smithi104 bash[9203]: cluster 2020-09-08T15:58:52.691872+0000 osd.7 (osd.7) 109 : cluster [DBG] 6.8 deep-scrub starts 2020-09-08T15:58:54.162 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:53 smithi104 bash[9203]: audit 2020-09-08T15:58:53.402829+0000 mon.b (mon.1) 107 : audit [DBG] from='client.? v1:172.21.15.104:0/1191393008' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:58:54.162 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:53 smithi104 bash[9203]: audit 2020-09-08T15:58:53.701442+0000 mon.c (mon.2) 72 : audit [INF] from='client.? v1:172.21.15.104:0/144181256' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:58:54.163 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:53 smithi104 bash[9203]: audit 2020-09-08T15:58:53.704397+0000 mon.a (mon.0) 1800 : audit [INF] from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:58:54.163 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:58:54.163 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:58:53 smithi104 bash[19922]: debug 2020-09-08T15:58:53.873+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:54.265 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:58:54.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:53 smithi166 bash[9446]: cluster 2020-09-08T15:58:52.691872+0000 osd.7 (osd.7) 109 : cluster [DBG] 6.8 deep-scrub starts 2020-09-08T15:58:54.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:53 smithi166 bash[9446]: audit 2020-09-08T15:58:53.402829+0000 mon.b (mon.1) 107 : audit [DBG] from='client.? v1:172.21.15.104:0/1191393008' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:58:54.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:53 smithi166 bash[9446]: audit 2020-09-08T15:58:53.701442+0000 mon.c (mon.2) 72 : audit [INF] from='client.? v1:172.21.15.104:0/144181256' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:58:54.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:53 smithi166 bash[9446]: audit 2020-09-08T15:58:53.704397+0000 mon.a (mon.0) 1800 : audit [INF] from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:58:54.416 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:58:54.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:58:54 smithi104 bash[17588]: debug 2020-09-08T15:58:54.161+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:54.520 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:58:54.532 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:58:54 smithi055 bash[19003]: debug 2020-09-08T15:58:54.417+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:55.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:54 smithi166 bash[9446]: cluster 2020-09-08T15:58:53.899051+0000 mon.a (mon.0) 1801 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:58:55.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:54 smithi166 bash[9446]: audit 2020-09-08T15:58:53.908202+0000 mon.a (mon.0) 1802 : audit [INF] from='client.35993 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:58:55.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:54 smithi166 bash[9446]: cluster 2020-09-08T15:58:53.908244+0000 mon.a (mon.0) 1803 : cluster [DBG] osdmap e525: 12 total, 12 up, 12 in 2020-09-08T15:58:55.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:54 smithi166 bash[9446]: cluster 2020-09-08T15:58:53.932513+0000 mgr.x (mgr.24118) 947 : cluster [DBG] pgmap v1388: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 672 op/s 2020-09-08T15:58:55.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:54 smithi055 bash[9296]: cluster 2020-09-08T15:58:53.899051+0000 mon.a (mon.0) 1801 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:58:55.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:54 smithi055 bash[9296]: audit 2020-09-08T15:58:53.908202+0000 mon.a (mon.0) 1802 : audit [INF] from='client.35993 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:58:55.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:54 smithi055 bash[9296]: cluster 2020-09-08T15:58:53.908244+0000 mon.a (mon.0) 1803 : cluster [DBG] osdmap e525: 12 total, 12 up, 12 in 2020-09-08T15:58:55.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:54 smithi055 bash[9296]: cluster 2020-09-08T15:58:53.932513+0000 mgr.x (mgr.24118) 947 : cluster [DBG] pgmap v1388: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 672 op/s 2020-09-08T15:58:55.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:54 smithi104 bash[9203]: cluster 2020-09-08T15:58:53.899051+0000 mon.a (mon.0) 1801 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:58:55.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:54 smithi104 bash[9203]: audit 2020-09-08T15:58:53.908202+0000 mon.a (mon.0) 1802 : audit [INF] from='client.35993 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:58:55.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:54 smithi104 bash[9203]: cluster 2020-09-08T15:58:53.908244+0000 mon.a (mon.0) 1803 : cluster [DBG] osdmap e525: 12 total, 12 up, 12 in 2020-09-08T15:58:55.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:54 smithi104 bash[9203]: cluster 2020-09-08T15:58:53.932513+0000 mgr.x (mgr.24118) 947 : cluster [DBG] pgmap v1388: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 672 op/s 2020-09-08T15:58:55.380 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:58:55.485 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:55.744 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:55.747 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:58:55 smithi055 bash[14301]: debug 2020-09-08T15:58:55.745+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:55.848 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:55.867 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:58:55 smithi166 bash[16593]: debug 2020-09-08T15:58:55.377+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:56.382 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:56.487 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:56.636 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:58:56.656 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:58:56 smithi055 bash[14301]: debug 2020-09-08T15:58:56.385+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:56.741 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:58:56.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:58:56 smithi166 bash[11928]: debug 2020-09-08T15:58:56.633+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:57.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:57 smithi055 bash[9296]: cluster 2020-09-08T15:58:55.933011+0000 mgr.x (mgr.24118) 948 : cluster [DBG] pgmap v1389: 17 pgs: 2 active+clean+scrubbing+deep, 15 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 653 op/s 2020-09-08T15:58:57.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:57 smithi055 bash[9296]: cluster 2020-09-08T15:58:56.235985+0000 osd.4 (osd.4) 55 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:57.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:57 smithi055 bash[9296]: cluster 2020-09-08T15:58:56.236069+0000 osd.4 (osd.4) 56 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:57.261 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:58:57.364 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:57.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:56 smithi166 bash[9446]: cluster 2020-09-08T15:58:55.933011+0000 mgr.x (mgr.24118) 948 : cluster [DBG] pgmap v1389: 17 pgs: 2 active+clean+scrubbing+deep, 15 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 653 op/s 2020-09-08T15:58:57.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:56 smithi166 bash[9446]: cluster 2020-09-08T15:58:56.235985+0000 osd.4 (osd.4) 55 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:57.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:56 smithi166 bash[9446]: cluster 2020-09-08T15:58:56.236069+0000 osd.4 (osd.4) 56 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:57.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:57 smithi104 bash[9203]: cluster 2020-09-08T15:58:55.933011+0000 mgr.x (mgr.24118) 948 : cluster [DBG] pgmap v1389: 17 pgs: 2 active+clean+scrubbing+deep, 15 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 653 op/s 2020-09-08T15:58:57.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:57 smithi104 bash[9203]: cluster 2020-09-08T15:58:56.235985+0000 osd.4 (osd.4) 55 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:58:57.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:57 smithi104 bash[9203]: cluster 2020-09-08T15:58:56.236069+0000 osd.4 (osd.4) 56 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:58:57.474 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:57.578 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:58:57.594 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:58:57 smithi055 bash[19003]: debug 2020-09-08T15:58:57.261+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:57.684 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:58:57 smithi166 bash[14254]: debug 2020-09-08T15:58:57.473+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:57.685 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:58:57.741 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:58:57.789 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:57.821 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:58:57.878 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:58:57.879 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:58:57.906 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:58:57.927 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:58:57.980 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:58:57.998 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:58 smithi166 bash[9446]: cluster 2020-09-08T15:58:56.990999+0000 mon.a (mon.0) 1804 : cluster [DBG] osdmap e526: 12 total, 12 up, 12 in 2020-09-08T15:58:57.999 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:58:57 smithi166 bash[14254]: debug 2020-09-08T15:58:57.925+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:57.999 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:58:57 smithi166 bash[16593]: debug 2020-09-08T15:58:57.681+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:58.031 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:58:58.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:58 smithi104 bash[9203]: cluster 2020-09-08T15:58:56.990999+0000 mon.a (mon.0) 1804 : cluster [DBG] osdmap e526: 12 total, 12 up, 12 in 2020-09-08T15:58:58.184 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:58:58.287 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:58:58.414 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:58:58 smithi055 bash[16646]: debug 2020-09-08T15:58:58.185+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:58.415 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:58 smithi055 bash[9296]: cluster 2020-09-08T15:58:56.990999+0000 mon.a (mon.0) 1804 : cluster [DBG] osdmap e526: 12 total, 12 up, 12 in 2020-09-08T15:58:58.415 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:58:58.516 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:58:58.617 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:58:58 smithi166 bash[18960]: debug 2020-09-08T15:58:58.401+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:59.002 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:58:59.025 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2020-09-08T15:58:59.105 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:58:59.188 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:58:59.245 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:58:59.350 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:58:59.358 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:58:59 smithi055 bash[19003]: debug 2020-09-08T15:58:59.245+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:58:59.358 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:58:59 smithi055 bash[9296]: cluster 2020-09-08T15:58:57.933498+0000 mgr.x (mgr.24118) 949 : cluster [DBG] pgmap v1391: 17 pgs: 2 active+clean+scrubbing+deep, 15 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 904 op/s 2020-09-08T15:58:59.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:58:59 smithi166 bash[9446]: cluster 2020-09-08T15:58:57.933498+0000 mgr.x (mgr.24118) 949 : cluster [DBG] pgmap v1391: 17 pgs: 2 active+clean+scrubbing+deep, 15 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 904 op/s 2020-09-08T15:58:59.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:58:59 smithi104 bash[9203]: cluster 2020-09-08T15:58:57.933498+0000 mgr.x (mgr.24118) 949 : cluster [DBG] pgmap v1391: 17 pgs: 2 active+clean+scrubbing+deep, 15 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 904 op/s 2020-09-08T15:58:59.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:58:59 smithi104 bash[15273]: debug 2020-09-08T15:58:59.001+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:00.032 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:00.137 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:59:00.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:59:00 smithi055 bash[14301]: debug 2020-09-08T15:59:00.041+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:00.717 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:59:00.822 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:00.890 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:00 smithi104 bash[9203]: cluster 2020-09-08T15:58:59.933985+0000 mgr.x (mgr.24118) 950 : cluster [DBG] pgmap v1392: 17 pgs: 2 active+clean+scrubbing+deep, 15 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 777 op/s 2020-09-08T15:59:00.953 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:01.019 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:59:00 smithi055 bash[11956]: debug 2020-09-08T15:59:00.717+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:01.019 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:00 smithi055 bash[9296]: cluster 2020-09-08T15:58:59.933985+0000 mgr.x (mgr.24118) 950 : cluster [DBG] pgmap v1392: 17 pgs: 2 active+clean+scrubbing+deep, 15 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 777 op/s 2020-09-08T15:59:01.057 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:59:01.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:00 smithi166 bash[9446]: cluster 2020-09-08T15:58:59.933985+0000 mgr.x (mgr.24118) 950 : cluster [DBG] pgmap v1392: 17 pgs: 2 active+clean+scrubbing+deep, 15 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 777 op/s 2020-09-08T15:59:01.117 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:59:00 smithi166 bash[14254]: debug 2020-09-08T15:59:00.953+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:01.187 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:59:01.292 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:01.422 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:01.530 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:01.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:59:01 smithi166 bash[11928]: debug 2020-09-08T15:59:01.185+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:01.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:59:01 smithi166 bash[14254]: debug 2020-09-08T15:59:01.421+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:01.979 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:02.083 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:02.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:59:01 smithi104 bash[19922]: debug 2020-09-08T15:59:01.977+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:02.627 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:02.732 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:02.953 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:59:02 smithi104 bash[15273]: debug 2020-09-08T15:59:02.629+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:03.004 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:02 smithi055 bash[9296]: cluster 2020-09-08T15:59:01.934461+0000 mgr.x (mgr.24118) 951 : cluster [DBG] pgmap v1393: 17 pgs: 2 active+clean+scrubbing+deep, 15 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 651 op/s 2020-09-08T15:59:03.005 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:03.109 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:03.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:02 smithi166 bash[9446]: cluster 2020-09-08T15:59:01.934461+0000 mgr.x (mgr.24118) 951 : cluster [DBG] pgmap v1393: 17 pgs: 2 active+clean+scrubbing+deep, 15 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 651 op/s 2020-09-08T15:59:03.388 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:03.388 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:02 smithi104 bash[9203]: cluster 2020-09-08T15:59:01.934461+0000 mgr.x (mgr.24118) 951 : cluster [DBG] pgmap v1393: 17 pgs: 2 active+clean+scrubbing+deep, 15 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 651 op/s 2020-09-08T15:59:03.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:59:03 smithi055 bash[16646]: debug 2020-09-08T15:59:03.009+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:03.499 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:59:03.633 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:59:03.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:59:03 smithi104 bash[22305]: debug 2020-09-08T15:59:03.389+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:03.736 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:03.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:59:03 smithi166 bash[11928]: debug 2020-09-08T15:59:03.633+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:03.896 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:03.998 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:04.010 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:59:03 smithi104 bash[19922]: debug 2020-09-08T15:59:03.897+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:04.169 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:04.274 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:04.286 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:59:04.286 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:59:04.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:59:04 smithi104 bash[15273]: debug 2020-09-08T15:59:04.169+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:04.442 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:04.484 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:59:04.524 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:59:04.546 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:59:04.553 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:59:04 smithi055 bash[16646]: debug 2020-09-08T15:59:04.437+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:04.556 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:59:04.594 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:59:04.627 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:59:04.664 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:59:04.729 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:59:04.759 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:59:04.785 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:59:04.819 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:59:04.849 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:59:04.881 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:59:04.882 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:59:04.883 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:59:04.883 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:59:04.883 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:59:04.883 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:59:04.883 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:59:04.884 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:59:04.884 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:59:04.884 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:59:04.884 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:59:04.884 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:59:04.885 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:59:05.004 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:59:05.278 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:04 smithi055 bash[9296]: cluster 2020-09-08T15:59:03.859096+0000 osd.9 (osd.9) 79 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:05.278 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:04 smithi055 bash[9296]: cluster 2020-09-08T15:59:03.859169+0000 osd.9 (osd.9) 80 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:05.278 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:04 smithi055 bash[9296]: cluster 2020-09-08T15:59:03.934877+0000 mgr.x (mgr.24118) 952 : cluster [DBG] pgmap v1394: 17 pgs: 2 active+clean+scrubbing+deep, 15 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 687 op/s 2020-09-08T15:59:05.279 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:04 smithi055 bash[9296]: cluster 2020-09-08T15:59:03.999711+0000 mon.a (mon.0) 1805 : cluster [DBG] osdmap e527: 12 total, 12 up, 12 in 2020-09-08T15:59:05.279 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:04 smithi055 bash[9296]: audit 2020-09-08T15:59:04.451354+0000 mon.a (mon.0) 1806 : audit [DBG] from='client.? v1:172.21.15.104:0/2293722030' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:59:05.279 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:04 smithi055 bash[9296]: audit 2020-09-08T15:59:04.751912+0000 mon.a (mon.0) 1807 : audit [INF] from='client.? v1:172.21.15.104:0/2190712178' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:59:05.280 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:59:05.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:04 smithi166 bash[9446]: cluster 2020-09-08T15:59:03.859096+0000 osd.9 (osd.9) 79 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:05.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:04 smithi166 bash[9446]: cluster 2020-09-08T15:59:03.859169+0000 osd.9 (osd.9) 80 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:05.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:04 smithi166 bash[9446]: cluster 2020-09-08T15:59:03.934877+0000 mgr.x (mgr.24118) 952 : cluster [DBG] pgmap v1394: 17 pgs: 2 active+clean+scrubbing+deep, 15 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 687 op/s 2020-09-08T15:59:05.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:04 smithi166 bash[9446]: cluster 2020-09-08T15:59:03.999711+0000 mon.a (mon.0) 1805 : cluster [DBG] osdmap e527: 12 total, 12 up, 12 in 2020-09-08T15:59:05.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:04 smithi166 bash[9446]: audit 2020-09-08T15:59:04.451354+0000 mon.a (mon.0) 1806 : audit [DBG] from='client.? v1:172.21.15.104:0/2293722030' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:59:05.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:04 smithi166 bash[9446]: audit 2020-09-08T15:59:04.751912+0000 mon.a (mon.0) 1807 : audit [INF] from='client.? v1:172.21.15.104:0/2190712178' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:59:05.384 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:05.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:04 smithi104 bash[9203]: cluster 2020-09-08T15:59:03.859096+0000 osd.9 (osd.9) 79 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:05.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:04 smithi104 bash[9203]: cluster 2020-09-08T15:59:03.859169+0000 osd.9 (osd.9) 80 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:05.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:04 smithi104 bash[9203]: cluster 2020-09-08T15:59:03.934877+0000 mgr.x (mgr.24118) 952 : cluster [DBG] pgmap v1394: 17 pgs: 2 active+clean+scrubbing+deep, 15 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 687 op/s 2020-09-08T15:59:05.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:04 smithi104 bash[9203]: cluster 2020-09-08T15:59:03.999711+0000 mon.a (mon.0) 1805 : cluster [DBG] osdmap e527: 12 total, 12 up, 12 in 2020-09-08T15:59:05.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:04 smithi104 bash[9203]: audit 2020-09-08T15:59:04.451354+0000 mon.a (mon.0) 1806 : audit [DBG] from='client.? v1:172.21.15.104:0/2293722030' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:59:05.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:04 smithi104 bash[9203]: audit 2020-09-08T15:59:04.751912+0000 mon.a (mon.0) 1807 : audit [INF] from='client.? v1:172.21.15.104:0/2190712178' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:59:05.428 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2020-09-08T15:59:05.505 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:05.611 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:05.638 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:59:05.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:59:05 smithi055 bash[19003]: debug 2020-09-08T15:59:05.281+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:05.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:59:05 smithi055 bash[14301]: debug 2020-09-08T15:59:05.501+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:05.756 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:05.862 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:05.999 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:59:05 smithi166 bash[14254]: debug 2020-09-08T15:59:05.753+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:06.000 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:06 smithi166 bash[9446]: cluster 2020-09-08T15:59:04.994525+0000 mon.a (mon.0) 1808 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:59:06.000 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:06 smithi166 bash[9446]: audit 2020-09-08T15:59:05.003463+0000 mon.a (mon.0) 1809 : audit [INF] from='client.? v1:172.21.15.104:0/2190712178' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:59:06.000 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:06 smithi166 bash[9446]: cluster 2020-09-08T15:59:05.003501+0000 mon.a (mon.0) 1810 : cluster [DBG] osdmap e528: 12 total, 12 up, 12 in 2020-09-08T15:59:06.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:06 smithi055 bash[9296]: cluster 2020-09-08T15:59:04.994525+0000 mon.a (mon.0) 1808 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:59:06.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:06 smithi055 bash[9296]: audit 2020-09-08T15:59:05.003463+0000 mon.a (mon.0) 1809 : audit [INF] from='client.? v1:172.21.15.104:0/2190712178' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:59:06.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:06 smithi055 bash[9296]: cluster 2020-09-08T15:59:05.003501+0000 mon.a (mon.0) 1810 : cluster [DBG] osdmap e528: 12 total, 12 up, 12 in 2020-09-08T15:59:06.418 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:06.451 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:06 smithi104 bash[9203]: cluster 2020-09-08T15:59:04.994525+0000 mon.a (mon.0) 1808 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:59:06.451 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:06 smithi104 bash[9203]: audit 2020-09-08T15:59:05.003463+0000 mon.a (mon.0) 1809 : audit [INF] from='client.? v1:172.21.15.104:0/2190712178' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:59:06.453 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:06 smithi104 bash[9203]: cluster 2020-09-08T15:59:05.003501+0000 mon.a (mon.0) 1810 : cluster [DBG] osdmap e528: 12 total, 12 up, 12 in 2020-09-08T15:59:06.522 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:06.722 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:06.826 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:06.834 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:59:06 smithi055 bash[14301]: debug 2020-09-08T15:59:06.421+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:06.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:59:06 smithi104 bash[15273]: debug 2020-09-08T15:59:06.716+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:07.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:07 smithi055 bash[9296]: cluster 2020-09-08T15:59:05.935308+0000 mgr.x (mgr.24118) 953 : cluster [DBG] pgmap v1397: 17 pgs: 2 active+clean+scrubbing+deep, 15 active+clean; 3.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 558 op/s 2020-09-08T15:59:07.352 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:07 smithi166 bash[9446]: cluster 2020-09-08T15:59:05.935308+0000 mgr.x (mgr.24118) 953 : cluster [DBG] pgmap v1397: 17 pgs: 2 active+clean+scrubbing+deep, 15 active+clean; 3.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 558 op/s 2020-09-08T15:59:07.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:07 smithi104 bash[9203]: cluster 2020-09-08T15:59:05.935308+0000 mgr.x (mgr.24118) 953 : cluster [DBG] pgmap v1397: 17 pgs: 2 active+clean+scrubbing+deep, 15 active+clean; 3.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 558 op/s 2020-09-08T15:59:07.456 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:07.591 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:07.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:59:07 smithi055 bash[16646]: debug 2020-09-08T15:59:07.345+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:07.696 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:07.849 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:07.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:59:07 smithi104 bash[19922]: debug 2020-09-08T15:59:07.592+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:07.955 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:59:07.962 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:59:07 smithi166 bash[16593]: debug 2020-09-08T15:59:07.849+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:08.063 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:59:08.167 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:08.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:59:08 smithi166 bash[18960]: debug 2020-09-08T15:59:08.061+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:08.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:08 smithi166 bash[9446]: cluster 2020-09-08T15:59:07.247961+0000 osd.2 (osd.2) 78 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:08.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:08 smithi166 bash[9446]: cluster 2020-09-08T15:59:07.248037+0000 osd.2 (osd.2) 79 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:08.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:08 smithi055 bash[9296]: cluster 2020-09-08T15:59:07.247961+0000 osd.2 (osd.2) 78 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:08.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:08 smithi055 bash[9296]: cluster 2020-09-08T15:59:07.248037+0000 osd.2 (osd.2) 79 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:08.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:08 smithi104 bash[9203]: cluster 2020-09-08T15:59:07.247961+0000 osd.2 (osd.2) 78 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:08.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:08 smithi104 bash[9203]: cluster 2020-09-08T15:59:07.248037+0000 osd.2 (osd.2) 79 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:08.671 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:59:08.733 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:59:08.831 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:59:08.903 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:59:08.994 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:09.098 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:09.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:09 smithi166 bash[9446]: cluster 2020-09-08T15:59:07.935716+0000 mgr.x (mgr.24118) 954 : cluster [DBG] pgmap v1398: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 3.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 809 op/s 2020-09-08T15:59:09.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:09 smithi166 bash[9446]: cluster 2020-09-08T15:59:08.020212+0000 mon.a (mon.0) 1811 : cluster [DBG] osdmap e529: 12 total, 12 up, 12 in 2020-09-08T15:59:09.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:09 smithi055 bash[9296]: cluster 2020-09-08T15:59:07.935716+0000 mgr.x (mgr.24118) 954 : cluster [DBG] pgmap v1398: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 3.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 809 op/s 2020-09-08T15:59:09.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:09 smithi055 bash[9296]: cluster 2020-09-08T15:59:08.020212+0000 mon.a (mon.0) 1811 : cluster [DBG] osdmap e529: 12 total, 12 up, 12 in 2020-09-08T15:59:09.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:59:09 smithi055 bash[14301]: debug 2020-09-08T15:59:08.996+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:09.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:09 smithi104 bash[9203]: cluster 2020-09-08T15:59:07.935716+0000 mgr.x (mgr.24118) 954 : cluster [DBG] pgmap v1398: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 3.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 809 op/s 2020-09-08T15:59:09.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:09 smithi104 bash[9203]: cluster 2020-09-08T15:59:08.020212+0000 mon.a (mon.0) 1811 : cluster [DBG] osdmap e529: 12 total, 12 up, 12 in 2020-09-08T15:59:10.194 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:10.298 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:10.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:59:10 smithi104 bash[17588]: debug 2020-09-08T15:59:10.192+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:10.966 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:11.048 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:59:11.048 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T15:59:11.048 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.048 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.16014.0:185486 6.c e529/478 6:3839fc43:::benchmark_data_smithi166_43187_object65717:head v 529'11847, mlcod=529'11845)", 2020-09-08T15:59:11.049 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:59:11.037950+0000", 2020-09-08T15:59:11.049 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.0048269200000000002, 2020-09-08T15:59:11.049 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0048614849999999996, 2020-09-08T15:59:11.049 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:59:11.050 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "started", 2020-09-08T15:59:11.051 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:59:11.051 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.051 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:59:11.051 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.037950+0000", 2020-09-08T15:59:11.051 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:59:11.052 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.052 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.052 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:59:11.052 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.037950+0000", 2020-09-08T15:59:11.052 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2977000000000002e-05 2020-09-08T15:59:11.053 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.053 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.053 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:59:11.053 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.038033+0000", 2020-09-08T15:59:11.053 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2653000000000001e-05 2020-09-08T15:59:11.054 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.054 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.054 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:59:11.054 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.038046+0000", 2020-09-08T15:59:11.054 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.8029999999999998e-05 2020-09-08T15:59:11.055 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.055 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.055 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:59:11.055 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.038064+0000", 2020-09-08T15:59:11.055 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0032e-05 2020-09-08T15:59:11.056 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.056 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.056 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:59:11.056 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.038074+0000", 2020-09-08T15:59:11.056 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.0574000000000006e-05 2020-09-08T15:59:11.056 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.057 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.057 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:59:11.057 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.038144+0000", 2020-09-08T15:59:11.057 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.2428000000000002e-05 2020-09-08T15:59:11.057 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.074 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.075 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:59:11.075 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.038177+0000", 2020-09-08T15:59:11.078 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00022669399999999999 2020-09-08T15:59:11.078 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:59:11.078 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:59:11.078 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:59:11.079 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.079 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.079 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.16014.0:185492 6.a e529/478 6:5eb28f14:::benchmark_data_smithi166_43187_object65723:head v 529'11683, mlcod=529'11681)", 2020-09-08T15:59:11.079 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:59:11.039919+0000", 2020-09-08T15:59:11.079 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.0028584399999999999, 2020-09-08T15:59:11.080 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0029676080000000001, 2020-09-08T15:59:11.080 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:59:11.080 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "started", 2020-09-08T15:59:11.080 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:59:11.080 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.081 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:59:11.081 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.039919+0000", 2020-09-08T15:59:11.081 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:59:11.081 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.081 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.082 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:59:11.082 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.039919+0000", 2020-09-08T15:59:11.082 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.0954000000000001e-05 2020-09-08T15:59:11.082 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.082 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.083 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:59:11.083 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.039960+0000", 2020-09-08T15:59:11.083 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.6890000000000001e-06 2020-09-08T15:59:11.083 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.083 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.083 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:59:11.084 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.039965+0000", 2020-09-08T15:59:11.084 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.8740000000000001e-06 2020-09-08T15:59:11.084 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.084 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.084 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:59:11.085 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.039974+0000", 2020-09-08T15:59:11.085 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.5289999999999997e-06 2020-09-08T15:59:11.085 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.085 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.085 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:59:11.086 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.039984+0000", 2020-09-08T15:59:11.086 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.3435e-05 2020-09-08T15:59:11.086 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.086 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.086 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:59:11.087 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.040027+0000", 2020-09-08T15:59:11.087 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.7194000000000001e-05 2020-09-08T15:59:11.087 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.087 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.087 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:59:11.088 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.040054+0000", 2020-09-08T15:59:11.088 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000135675 2020-09-08T15:59:11.089 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:59:11.089 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:59:11.089 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:59:11.089 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.089 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.090 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.16014.0:185488 6.c e529/478)", 2020-09-08T15:59:11.090 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:59:11.040282+0000", 2020-09-08T15:59:11.090 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.0024954, 2020-09-08T15:59:11.090 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0026665640000000002, 2020-09-08T15:59:11.091 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:59:11.091 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "queued for pg", 2020-09-08T15:59:11.091 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:59:11.091 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.091 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:59:11.092 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.040282+0000", 2020-09-08T15:59:11.092 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:59:11.092 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.092 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.092 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:59:11.093 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.040282+0000", 2020-09-08T15:59:11.093 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.0352000000000007e-05 2020-09-08T15:59:11.093 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.093 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.093 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:59:11.094 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.040352+0000", 2020-09-08T15:59:11.094 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0774999999999999e-05 2020-09-08T15:59:11.094 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.094 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.094 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:59:11.095 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.040363+0000", 2020-09-08T15:59:11.095 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6388e-05 2020-09-08T15:59:11.095 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.095 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.095 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:59:11.095 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.040379+0000", 2020-09-08T15:59:11.096 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0050000000000001e-06 2020-09-08T15:59:11.096 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:11.096 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:11.096 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:59:11.096 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:11.040385+0000", 2020-09-08T15:59:11.097 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010352 2020-09-08T15:59:11.097 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:59:11.097 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:59:11.097 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:59:11.097 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:59:11.098 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2020-09-08T15:59:11.098 INFO:teuthology.orchestra.run.smithi055.stdout: "num_ops": 3 2020-09-08T15:59:11.098 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:59:11.100 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:11.145 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:10 smithi055 bash[9296]: cluster 2020-09-08T15:59:09.936144+0000 mgr.x (mgr.24118) 955 : cluster [DBG] pgmap v1400: 17 pgs: 17 active+clean; 3.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 766 op/s 2020-09-08T15:59:11.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:59:10 smithi104 bash[22305]: debug 2020-09-08T15:59:10.964+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:11.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:10 smithi104 bash[9203]: cluster 2020-09-08T15:59:09.936144+0000 mgr.x (mgr.24118) 955 : cluster [DBG] pgmap v1400: 17 pgs: 17 active+clean; 3.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 766 op/s 2020-09-08T15:59:11.308 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:10 smithi166 bash[9446]: cluster 2020-09-08T15:59:09.936144+0000 mgr.x (mgr.24118) 955 : cluster [DBG] pgmap v1400: 17 pgs: 17 active+clean; 3.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 766 op/s 2020-09-08T15:59:11.420 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:11.557 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:11.646 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:59:11 smithi104 bash[22305]: debug 2020-09-08T15:59:11.304+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:11.661 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:11.783 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:59:11 smithi166 bash[16593]: debug 2020-09-08T15:59:11.557+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:11.783 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:11.886 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:12.090 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2020-09-08T15:59:12.117 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:59:11 smithi166 bash[14254]: debug 2020-09-08T15:59:11.781+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:12.148 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:12.151 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:59:12 smithi104 bash[17588]: debug 2020-09-08T15:59:12.144+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:12.253 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:12.264 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:59:13.218 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:13.322 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:13.333 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:12 smithi166 bash[9446]: cluster 2020-09-08T15:59:11.936783+0000 mgr.x (mgr.24118) 956 : cluster [DBG] pgmap v1401: 17 pgs: 17 active+clean; 3.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 762 op/s 2020-09-08T15:59:13.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:12 smithi055 bash[9296]: cluster 2020-09-08T15:59:11.936783+0000 mgr.x (mgr.24118) 956 : cluster [DBG] pgmap v1401: 17 pgs: 17 active+clean; 3.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 762 op/s 2020-09-08T15:59:13.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:59:13 smithi104 bash[19922]: debug 2020-09-08T15:59:13.216+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:13.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:12 smithi104 bash[9203]: cluster 2020-09-08T15:59:11.936783+0000 mgr.x (mgr.24118) 956 : cluster [DBG] pgmap v1401: 17 pgs: 17 active+clean; 3.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 762 op/s 2020-09-08T15:59:13.469 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:13.573 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:13.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:59:13 smithi166 bash[14254]: debug 2020-09-08T15:59:13.469+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:13.714 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:13.927 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:59:13 smithi104 bash[17588]: debug 2020-09-08T15:59:13.712+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:13.979 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:59:14.128 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:59:14.232 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:14.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:59:14 smithi166 bash[18960]: debug 2020-09-08T15:59:14.125+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:14.911 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:15.014 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:15.177 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:59:14 smithi055 bash[14301]: debug 2020-09-08T15:59:14.912+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:15.177 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:15 smithi055 bash[9296]: cluster 2020-09-08T15:59:13.798730+0000 osd.7 (osd.7) 110 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:15.178 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:15 smithi055 bash[9296]: cluster 2020-09-08T15:59:13.798816+0000 osd.7 (osd.7) 111 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:15.178 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:15 smithi055 bash[9296]: cluster 2020-09-08T15:59:13.942425+0000 mgr.x (mgr.24118) 957 : cluster [DBG] pgmap v1402: 17 pgs: 17 active+clean; 3.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 846 op/s 2020-09-08T15:59:15.178 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:15 smithi055 bash[9296]: cluster 2020-09-08T15:59:14.003862+0000 mon.a (mon.0) 1812 : cluster [DBG] osdmap e530: 12 total, 12 up, 12 in 2020-09-08T15:59:15.178 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:15.280 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:59:15.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:15 smithi166 bash[9446]: cluster 2020-09-08T15:59:13.798730+0000 osd.7 (osd.7) 110 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:15.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:15 smithi166 bash[9446]: cluster 2020-09-08T15:59:13.798816+0000 osd.7 (osd.7) 111 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:15.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:15 smithi166 bash[9446]: cluster 2020-09-08T15:59:13.942425+0000 mgr.x (mgr.24118) 957 : cluster [DBG] pgmap v1402: 17 pgs: 17 active+clean; 3.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 846 op/s 2020-09-08T15:59:15.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:15 smithi166 bash[9446]: cluster 2020-09-08T15:59:14.003862+0000 mon.a (mon.0) 1812 : cluster [DBG] osdmap e530: 12 total, 12 up, 12 in 2020-09-08T15:59:15.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:15 smithi104 bash[9203]: cluster 2020-09-08T15:59:13.798730+0000 osd.7 (osd.7) 110 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:15.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:15 smithi104 bash[9203]: cluster 2020-09-08T15:59:13.798816+0000 osd.7 (osd.7) 111 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:15.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:15 smithi104 bash[9203]: cluster 2020-09-08T15:59:13.942425+0000 mgr.x (mgr.24118) 957 : cluster [DBG] pgmap v1402: 17 pgs: 17 active+clean; 3.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 846 op/s 2020-09-08T15:59:15.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:15 smithi104 bash[9203]: cluster 2020-09-08T15:59:14.003862+0000 mon.a (mon.0) 1812 : cluster [DBG] osdmap e530: 12 total, 12 up, 12 in 2020-09-08T15:59:15.656 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:59:15 smithi055 bash[14301]: debug 2020-09-08T15:59:15.180+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:15.943 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:59:16.006 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:16 smithi104 bash[9203]: cluster 2020-09-08T15:59:15.050194+0000 osd.0 (osd.0) 79 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:16.006 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:16 smithi104 bash[9203]: cluster 2020-09-08T15:59:15.050283+0000 osd.0 (osd.0) 80 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:16.047 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:16.132 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:59:16.164 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:59:16.193 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:16.199 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:59:16.248 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:59:16.298 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:16.299 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:59:16.345 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:59:16.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:16 smithi166 bash[9446]: cluster 2020-09-08T15:59:15.050194+0000 osd.0 (osd.0) 79 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:16.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:16 smithi166 bash[9446]: cluster 2020-09-08T15:59:15.050283+0000 osd.0 (osd.0) 80 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:16.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:59:16 smithi166 bash[14254]: debug 2020-09-08T15:59:16.193+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:16.393 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:59:16.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:59:15 smithi055 bash[11956]: debug 2020-09-08T15:59:15.944+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:16.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:16 smithi055 bash[9296]: cluster 2020-09-08T15:59:15.050194+0000 osd.0 (osd.0) 79 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:16.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:16 smithi055 bash[9296]: cluster 2020-09-08T15:59:15.050283+0000 osd.0 (osd.0) 80 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:16.432 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:59:16.462 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:16.470 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:59:16.501 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:59:16.535 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:59:16.562 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:59:16.564 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:59:16.564 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:59:16.564 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:59:16.565 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:59:16.565 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:59:16.565 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:59:16.565 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:59:16.565 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:59:16.566 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:59:16.566 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:59:16.566 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:59:16.567 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:16.664 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:59:16 smithi104 bash[17588]: debug 2020-09-08T15:59:16.456+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:16.812 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:16.906 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:59:16 smithi055 bash[14301]: debug 2020-09-08T15:59:16.808+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:16.915 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:17.009 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T15:59:17.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:17 smithi104 bash[9203]: cluster 2020-09-08T15:59:15.942901+0000 mgr.x (mgr.24118) 958 : cluster [DBG] pgmap v1404: 17 pgs: 17 active+clean; 3.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 731 op/s 2020-09-08T15:59:17.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:17 smithi104 bash[9203]: cluster 2020-09-08T15:59:16.014794+0000 mon.a (mon.0) 1813 : cluster [DBG] osdmap e531: 12 total, 12 up, 12 in 2020-09-08T15:59:17.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:17 smithi104 bash[9203]: audit 2020-09-08T15:59:16.100477+0000 mon.a (mon.0) 1814 : audit [DBG] from='client.? v1:172.21.15.104:0/3397024285' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:59:17.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:17 smithi104 bash[9203]: audit 2020-09-08T15:59:16.442070+0000 mon.a (mon.0) 1815 : audit [INF] from='client.? v1:172.21.15.104:0/1813376945' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:59:17.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:17 smithi166 bash[9446]: cluster 2020-09-08T15:59:15.942901+0000 mgr.x (mgr.24118) 958 : cluster [DBG] pgmap v1404: 17 pgs: 17 active+clean; 3.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 731 op/s 2020-09-08T15:59:17.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:17 smithi166 bash[9446]: cluster 2020-09-08T15:59:16.014794+0000 mon.a (mon.0) 1813 : cluster [DBG] osdmap e531: 12 total, 12 up, 12 in 2020-09-08T15:59:17.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:17 smithi166 bash[9446]: audit 2020-09-08T15:59:16.100477+0000 mon.a (mon.0) 1814 : audit [DBG] from='client.? v1:172.21.15.104:0/3397024285' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:59:17.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:17 smithi166 bash[9446]: audit 2020-09-08T15:59:16.442070+0000 mon.a (mon.0) 1815 : audit [INF] from='client.? v1:172.21.15.104:0/1813376945' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:59:17.379 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:59:17.380 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:59:17.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:17 smithi055 bash[9296]: cluster 2020-09-08T15:59:15.942901+0000 mgr.x (mgr.24118) 958 : cluster [DBG] pgmap v1404: 17 pgs: 17 active+clean; 3.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 731 op/s 2020-09-08T15:59:17.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:17 smithi055 bash[9296]: cluster 2020-09-08T15:59:16.014794+0000 mon.a (mon.0) 1813 : cluster [DBG] osdmap e531: 12 total, 12 up, 12 in 2020-09-08T15:59:17.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:17 smithi055 bash[9296]: audit 2020-09-08T15:59:16.100477+0000 mon.a (mon.0) 1814 : audit [DBG] from='client.? v1:172.21.15.104:0/3397024285' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:59:17.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:17 smithi055 bash[9296]: audit 2020-09-08T15:59:16.442070+0000 mon.a (mon.0) 1815 : audit [INF] from='client.? v1:172.21.15.104:0/1813376945' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T15:59:18.309 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:18 smithi104 bash[9203]: cluster 2020-09-08T15:59:17.009252+0000 mon.a (mon.0) 1816 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:59:18.309 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:18 smithi104 bash[9203]: audit 2020-09-08T15:59:17.012706+0000 mon.a (mon.0) 1817 : audit [INF] from='client.? v1:172.21.15.104:0/1813376945' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:59:18.310 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:18 smithi104 bash[9203]: cluster 2020-09-08T15:59:17.012747+0000 mon.a (mon.0) 1818 : cluster [DBG] osdmap e532: 12 total, 12 up, 12 in 2020-09-08T15:59:18.310 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:18 smithi104 bash[9203]: cluster 2020-09-08T15:59:17.072207+0000 osd.7 (osd.7) 112 : cluster [DBG] 6.8 scrub starts 2020-09-08T15:59:18.310 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:18 smithi104 bash[9203]: cluster 2020-09-08T15:59:17.278803+0000 osd.0 (osd.0) 81 : cluster [DBG] 6.9 scrub starts 2020-09-08T15:59:18.311 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:18.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:18 smithi166 bash[9446]: cluster 2020-09-08T15:59:17.009252+0000 mon.a (mon.0) 1816 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:59:18.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:18 smithi166 bash[9446]: audit 2020-09-08T15:59:17.012706+0000 mon.a (mon.0) 1817 : audit [INF] from='client.? v1:172.21.15.104:0/1813376945' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:59:18.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:18 smithi166 bash[9446]: cluster 2020-09-08T15:59:17.012747+0000 mon.a (mon.0) 1818 : cluster [DBG] osdmap e532: 12 total, 12 up, 12 in 2020-09-08T15:59:18.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:18 smithi166 bash[9446]: cluster 2020-09-08T15:59:17.072207+0000 osd.7 (osd.7) 112 : cluster [DBG] 6.8 scrub starts 2020-09-08T15:59:18.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:18 smithi166 bash[9446]: cluster 2020-09-08T15:59:17.278803+0000 osd.0 (osd.0) 81 : cluster [DBG] 6.9 scrub starts 2020-09-08T15:59:18.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:18 smithi055 bash[9296]: cluster 2020-09-08T15:59:17.009252+0000 mon.a (mon.0) 1816 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:59:18.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:18 smithi055 bash[9296]: audit 2020-09-08T15:59:17.012706+0000 mon.a (mon.0) 1817 : audit [INF] from='client.? v1:172.21.15.104:0/1813376945' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T15:59:18.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:18 smithi055 bash[9296]: cluster 2020-09-08T15:59:17.012747+0000 mon.a (mon.0) 1818 : cluster [DBG] osdmap e532: 12 total, 12 up, 12 in 2020-09-08T15:59:18.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:18 smithi055 bash[9296]: cluster 2020-09-08T15:59:17.072207+0000 osd.7 (osd.7) 112 : cluster [DBG] 6.8 scrub starts 2020-09-08T15:59:18.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:18 smithi055 bash[9296]: cluster 2020-09-08T15:59:17.278803+0000 osd.0 (osd.0) 81 : cluster [DBG] 6.9 scrub starts 2020-09-08T15:59:18.418 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:18.535 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2020-09-08T15:59:18.650 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:59:18 smithi104 bash[17588]: debug 2020-09-08T15:59:18.308+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:18.650 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:59:18 smithi104 bash[17588]: debug 2020-09-08T15:59:18.648+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:18.653 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:18.705 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:59:18.757 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:18.889 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:18.993 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:59:19.035 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:19 smithi166 bash[9446]: cluster 2020-09-08T15:59:17.943398+0000 mgr.x (mgr.24118) 959 : cluster [DBG] pgmap v1407: 17 pgs: 17 active+clean; 3.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T15:59:19.036 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:19 smithi166 bash[9446]: cluster 2020-09-08T15:59:18.093696+0000 osd.0 (osd.0) 82 : cluster [DBG] 6.9 scrub ok 2020-09-08T15:59:19.036 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:59:18 smithi166 bash[14254]: debug 2020-09-08T15:59:18.889+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:19.112 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:59:19.221 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:59:19.332 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:59:19 smithi166 bash[18960]: debug 2020-09-08T15:59:19.109+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:19.333 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:59:19.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:19 smithi055 bash[9296]: cluster 2020-09-08T15:59:17.943398+0000 mgr.x (mgr.24118) 959 : cluster [DBG] pgmap v1407: 17 pgs: 17 active+clean; 3.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T15:59:19.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:19 smithi055 bash[9296]: cluster 2020-09-08T15:59:18.093696+0000 osd.0 (osd.0) 82 : cluster [DBG] 6.9 scrub ok 2020-09-08T15:59:19.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:19 smithi104 bash[9203]: cluster 2020-09-08T15:59:17.943398+0000 mgr.x (mgr.24118) 959 : cluster [DBG] pgmap v1407: 17 pgs: 17 active+clean; 3.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T15:59:19.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:19 smithi104 bash[9203]: cluster 2020-09-08T15:59:18.093696+0000 osd.0 (osd.0) 82 : cluster [DBG] 6.9 scrub ok 2020-09-08T15:59:19.443 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:59:19.569 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:59:19.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:59:19 smithi166 bash[11928]: debug 2020-09-08T15:59:19.329+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:19.617 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:59:19 smithi166 bash[18960]: debug 2020-09-08T15:59:19.561+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:19.674 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:19.829 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:19.932 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:59:20.039 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:59:19 smithi104 bash[22305]: debug 2020-09-08T15:59:19.828+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:20.278 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:20 smithi055 bash[9296]: cluster 2020-09-08T15:59:18.311183+0000 osd.8 (osd.8) 70 : cluster [DBG] 6.1 scrub starts 2020-09-08T15:59:20.279 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:20 smithi055 bash[9296]: cluster 2020-09-08T15:59:19.146471+0000 osd.1 (osd.1) 106 : cluster [DBG] 6.4 scrub starts 2020-09-08T15:59:20.279 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:59:20.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:20 smithi166 bash[9446]: cluster 2020-09-08T15:59:18.311183+0000 osd.8 (osd.8) 70 : cluster [DBG] 6.1 scrub starts 2020-09-08T15:59:20.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:20 smithi166 bash[9446]: cluster 2020-09-08T15:59:19.146471+0000 osd.1 (osd.1) 106 : cluster [DBG] 6.4 scrub starts 2020-09-08T15:59:20.383 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:20.398 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:20 smithi104 bash[9203]: cluster 2020-09-08T15:59:18.311183+0000 osd.8 (osd.8) 70 : cluster [DBG] 6.1 scrub starts 2020-09-08T15:59:20.399 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:20 smithi104 bash[9203]: cluster 2020-09-08T15:59:19.146471+0000 osd.1 (osd.1) 106 : cluster [DBG] 6.4 scrub starts 2020-09-08T15:59:20.506 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:20.609 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:20.639 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:59:20 smithi055 bash[11956]: debug 2020-09-08T15:59:20.280+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:20.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:59:20 smithi104 bash[15273]: debug 2020-09-08T15:59:20.496+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:20.728 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:20.741 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:59:20.762 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T15:59:20.832 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:59:20.909 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:59:20.967 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:59:21.008 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:59:21.042 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:59:20 smithi104 bash[19922]: debug 2020-09-08T15:59:20.720+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:21.122 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:21.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:21 smithi055 bash[9296]: cluster 2020-09-08T15:59:19.943793+0000 mgr.x (mgr.24118) 960 : cluster [DBG] pgmap v1408: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 858 op/s 2020-09-08T15:59:21.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:21 smithi055 bash[9296]: cluster 2020-09-08T15:59:20.268699+0000 osd.1 (osd.1) 107 : cluster [DBG] 6.4 scrub ok 2020-09-08T15:59:21.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:59:21 smithi055 bash[19003]: debug 2020-09-08T15:59:21.012+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:21.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:21 smithi166 bash[9446]: cluster 2020-09-08T15:59:19.943793+0000 mgr.x (mgr.24118) 960 : cluster [DBG] pgmap v1408: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 858 op/s 2020-09-08T15:59:21.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:21 smithi166 bash[9446]: cluster 2020-09-08T15:59:20.268699+0000 osd.1 (osd.1) 107 : cluster [DBG] 6.4 scrub ok 2020-09-08T15:59:21.372 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:21 smithi104 bash[9203]: cluster 2020-09-08T15:59:19.943793+0000 mgr.x (mgr.24118) 960 : cluster [DBG] pgmap v1408: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 858 op/s 2020-09-08T15:59:21.373 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:21 smithi104 bash[9203]: cluster 2020-09-08T15:59:20.268699+0000 osd.1 (osd.1) 107 : cluster [DBG] 6.4 scrub ok 2020-09-08T15:59:22.346 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:22 smithi104 bash[9203]: cluster 2020-09-08T15:59:20.822759+0000 osd.5 (osd.5) 81 : cluster [DBG] 6.a scrub starts 2020-09-08T15:59:22.350 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:22.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:22 smithi166 bash[9446]: cluster 2020-09-08T15:59:20.822759+0000 osd.5 (osd.5) 81 : cluster [DBG] 6.a scrub starts 2020-09-08T15:59:22.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:22 smithi055 bash[9296]: cluster 2020-09-08T15:59:20.822759+0000 osd.5 (osd.5) 81 : cluster [DBG] 6.a scrub starts 2020-09-08T15:59:22.454 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:22.591 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:22.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:59:22 smithi104 bash[22305]: debug 2020-09-08T15:59:22.348+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:22.697 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:22.867 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:59:22 smithi166 bash[14254]: debug 2020-09-08T15:59:22.589+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:23.049 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:23 smithi104 bash[9203]: cluster 2020-09-08T15:59:21.803918+0000 osd.5 (osd.5) 82 : cluster [DBG] 6.a scrub ok 2020-09-08T15:59:23.049 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:23 smithi104 bash[9203]: cluster 2020-09-08T15:59:21.944249+0000 mgr.x (mgr.24118) 961 : cluster [DBG] pgmap v1409: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 821 op/s 2020-09-08T15:59:23.050 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:23 smithi104 bash[9203]: cluster 2020-09-08T15:59:22.031676+0000 osd.3 (osd.3) 83 : cluster [DBG] 6.6 scrub starts 2020-09-08T15:59:23.186 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:23.289 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:23.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:23 smithi166 bash[9446]: cluster 2020-09-08T15:59:21.803918+0000 osd.5 (osd.5) 82 : cluster [DBG] 6.a scrub ok 2020-09-08T15:59:23.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:23 smithi166 bash[9446]: cluster 2020-09-08T15:59:21.944249+0000 mgr.x (mgr.24118) 961 : cluster [DBG] pgmap v1409: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 821 op/s 2020-09-08T15:59:23.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:23 smithi166 bash[9446]: cluster 2020-09-08T15:59:22.031676+0000 osd.3 (osd.3) 83 : cluster [DBG] 6.6 scrub starts 2020-09-08T15:59:23.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:23 smithi055 bash[9296]: cluster 2020-09-08T15:59:21.803918+0000 osd.5 (osd.5) 82 : cluster [DBG] 6.a scrub ok 2020-09-08T15:59:23.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:23 smithi055 bash[9296]: cluster 2020-09-08T15:59:21.944249+0000 mgr.x (mgr.24118) 961 : cluster [DBG] pgmap v1409: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 821 op/s 2020-09-08T15:59:23.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:23 smithi055 bash[9296]: cluster 2020-09-08T15:59:22.031676+0000 osd.3 (osd.3) 83 : cluster [DBG] 6.6 scrub starts 2020-09-08T15:59:23.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:59:23 smithi104 bash[15273]: debug 2020-09-08T15:59:23.184+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:23.503 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:23.606 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:23.774 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:59:23 smithi055 bash[16646]: debug 2020-09-08T15:59:23.772+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:23.776 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:23.792 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:59:23 smithi104 bash[17588]: debug 2020-09-08T15:59:23.504+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:23.800 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:59:23.800 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:59:23.880 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:23.998 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:24.058 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:59:24 smithi104 bash[22305]: debug 2020-09-08T15:59:23.996+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:24.058 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:24 smithi104 bash[9203]: cluster 2020-09-08T15:59:23.281344+0000 osd.5 (osd.5) 83 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:24.059 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:24 smithi104 bash[9203]: cluster 2020-09-08T15:59:23.281438+0000 osd.5 (osd.5) 84 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:24.059 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:24 smithi104 bash[9203]: cluster 2020-09-08T15:59:23.290859+0000 osd.3 (osd.3) 84 : cluster [DBG] 6.6 scrub ok 2020-09-08T15:59:24.103 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:24.242 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:24.351 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:24.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:59:24 smithi166 bash[14254]: debug 2020-09-08T15:59:24.240+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:24.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:24 smithi166 bash[9446]: cluster 2020-09-08T15:59:23.281344+0000 osd.5 (osd.5) 83 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:24.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:24 smithi166 bash[9446]: cluster 2020-09-08T15:59:23.281438+0000 osd.5 (osd.5) 84 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:24.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:24 smithi166 bash[9446]: cluster 2020-09-08T15:59:23.290859+0000 osd.3 (osd.3) 84 : cluster [DBG] 6.6 scrub ok 2020-09-08T15:59:24.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:24 smithi055 bash[9296]: cluster 2020-09-08T15:59:23.281344+0000 osd.5 (osd.5) 83 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:24.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:24 smithi055 bash[9296]: cluster 2020-09-08T15:59:23.281438+0000 osd.5 (osd.5) 84 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:24.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:24 smithi055 bash[9296]: cluster 2020-09-08T15:59:23.290859+0000 osd.3 (osd.3) 84 : cluster [DBG] 6.6 scrub ok 2020-09-08T15:59:24.948 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2020-09-08T15:59:25.162 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:59:25.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:25 smithi166 bash[9446]: cluster 2020-09-08T15:59:23.944678+0000 mgr.x (mgr.24118) 962 : cluster [DBG] pgmap v1410: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 2.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 932 op/s 2020-09-08T15:59:25.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:25 smithi166 bash[9446]: cluster 2020-09-08T15:59:24.065573+0000 mon.a (mon.0) 1819 : cluster [DBG] osdmap e533: 12 total, 12 up, 12 in 2020-09-08T15:59:25.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:25 smithi055 bash[9296]: cluster 2020-09-08T15:59:23.944678+0000 mgr.x (mgr.24118) 962 : cluster [DBG] pgmap v1410: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 2.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 932 op/s 2020-09-08T15:59:25.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:25 smithi055 bash[9296]: cluster 2020-09-08T15:59:24.065573+0000 mon.a (mon.0) 1819 : cluster [DBG] osdmap e533: 12 total, 12 up, 12 in 2020-09-08T15:59:25.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:25 smithi104 bash[9203]: cluster 2020-09-08T15:59:23.944678+0000 mgr.x (mgr.24118) 962 : cluster [DBG] pgmap v1410: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 2.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 932 op/s 2020-09-08T15:59:25.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:25 smithi104 bash[9203]: cluster 2020-09-08T15:59:24.065573+0000 mon.a (mon.0) 1819 : cluster [DBG] osdmap e533: 12 total, 12 up, 12 in 2020-09-08T15:59:25.534 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:25.638 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:59:25.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:59:25 smithi104 bash[17588]: debug 2020-09-08T15:59:25.532+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:26.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:26 smithi166 bash[9446]: cluster 2020-09-08T15:59:25.062421+0000 osd.10 (osd.10) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:26.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:26 smithi166 bash[9446]: cluster 2020-09-08T15:59:25.062510+0000 osd.10 (osd.10) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:26.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:26 smithi055 bash[9296]: cluster 2020-09-08T15:59:25.062421+0000 osd.10 (osd.10) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:26.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:26 smithi055 bash[9296]: cluster 2020-09-08T15:59:25.062510+0000 osd.10 (osd.10) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:26.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:26 smithi104 bash[9203]: cluster 2020-09-08T15:59:25.062421+0000 osd.10 (osd.10) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:26.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:26 smithi104 bash[9203]: cluster 2020-09-08T15:59:25.062510+0000 osd.10 (osd.10) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:27.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:27 smithi166 bash[9446]: cluster 2020-09-08T15:59:25.945098+0000 mgr.x (mgr.24118) 963 : cluster [DBG] pgmap v1412: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 2.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 903 op/s 2020-09-08T15:59:27.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:27 smithi166 bash[9446]: cluster 2020-09-08T15:59:26.075669+0000 mon.a (mon.0) 1820 : cluster [DBG] osdmap e534: 12 total, 12 up, 12 in 2020-09-08T15:59:27.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:27 smithi055 bash[9296]: cluster 2020-09-08T15:59:25.945098+0000 mgr.x (mgr.24118) 963 : cluster [DBG] pgmap v1412: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 2.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 903 op/s 2020-09-08T15:59:27.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:27 smithi055 bash[9296]: cluster 2020-09-08T15:59:26.075669+0000 mon.a (mon.0) 1820 : cluster [DBG] osdmap e534: 12 total, 12 up, 12 in 2020-09-08T15:59:27.440 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:27 smithi104 bash[9203]: cluster 2020-09-08T15:59:25.945098+0000 mgr.x (mgr.24118) 963 : cluster [DBG] pgmap v1412: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 2.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 903 op/s 2020-09-08T15:59:27.440 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:27 smithi104 bash[9203]: cluster 2020-09-08T15:59:26.075669+0000 mon.a (mon.0) 1820 : cluster [DBG] osdmap e534: 12 total, 12 up, 12 in 2020-09-08T15:59:27.478 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:59:27.582 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:27.706 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:27.810 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:27.871 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:59:27 smithi166 bash[16593]: debug 2020-09-08T15:59:27.704+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:27.872 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:59:27 smithi166 bash[18960]: debug 2020-09-08T15:59:27.476+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:28.053 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:28.154 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:59:28.269 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:59:28.313 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:59:28.347 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:59:28.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:28 smithi166 bash[9446]: cluster 2020-09-08T15:59:27.010910+0000 osd.3 (osd.3) 85 : cluster [DBG] 6.c scrub starts 2020-09-08T15:59:28.394 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:59:28.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:28 smithi055 bash[9296]: cluster 2020-09-08T15:59:27.010910+0000 osd.3 (osd.3) 85 : cluster [DBG] 6.c scrub starts 2020-09-08T15:59:28.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:28 smithi104 bash[9203]: cluster 2020-09-08T15:59:27.010910+0000 osd.3 (osd.3) 85 : cluster [DBG] 6.c scrub starts 2020-09-08T15:59:28.427 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:59:28 smithi104 bash[19922]: debug 2020-09-08T15:59:28.044+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:28.427 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:59:28.469 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:59:28.496 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:59:28.529 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:59:28.562 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:59:28.599 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:59:28.633 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:59:28.674 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:59:28.683 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:59:28.683 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:59:28.684 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:59:28.684 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:59:28.684 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:59:28.684 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:59:28.684 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:59:28.685 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:59:28.694 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:59:28.694 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:59:28.694 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:59:28.695 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:59:28.872 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:59:28.977 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:29.086 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T15:59:29.088 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:29 smithi104 bash[9203]: cluster 2020-09-08T15:59:27.945573+0000 mgr.x (mgr.24118) 964 : cluster [DBG] pgmap v1414: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 2.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:59:29.089 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:29 smithi104 bash[9203]: cluster 2020-09-08T15:59:27.990771+0000 osd.3 (osd.3) 86 : cluster [DBG] 6.c scrub ok 2020-09-08T15:59:29.089 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:29 smithi104 bash[9203]: audit 2020-09-08T15:59:28.220827+0000 mon.b (mon.1) 108 : audit [DBG] from='client.? v1:172.21.15.104:0/1056372450' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:59:29.089 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:29 smithi104 bash[9203]: audit 2020-09-08T15:59:28.547939+0000 mon.a (mon.0) 1821 : audit [INF] from='client.? v1:172.21.15.104:0/3382219831' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:59:29.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:59:28 smithi055 bash[19003]: debug 2020-09-08T15:59:28.872+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:29.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:29 smithi055 bash[9296]: cluster 2020-09-08T15:59:27.945573+0000 mgr.x (mgr.24118) 964 : cluster [DBG] pgmap v1414: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 2.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:59:29.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:29 smithi055 bash[9296]: cluster 2020-09-08T15:59:27.990771+0000 osd.3 (osd.3) 86 : cluster [DBG] 6.c scrub ok 2020-09-08T15:59:29.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:29 smithi055 bash[9296]: audit 2020-09-08T15:59:28.220827+0000 mon.b (mon.1) 108 : audit [DBG] from='client.? v1:172.21.15.104:0/1056372450' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:59:29.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:29 smithi055 bash[9296]: audit 2020-09-08T15:59:28.547939+0000 mon.a (mon.0) 1821 : audit [INF] from='client.? v1:172.21.15.104:0/3382219831' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:59:29.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:29 smithi166 bash[9446]: cluster 2020-09-08T15:59:27.945573+0000 mgr.x (mgr.24118) 964 : cluster [DBG] pgmap v1414: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 2.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T15:59:29.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:29 smithi166 bash[9446]: cluster 2020-09-08T15:59:27.990771+0000 osd.3 (osd.3) 86 : cluster [DBG] 6.c scrub ok 2020-09-08T15:59:29.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:29 smithi166 bash[9446]: audit 2020-09-08T15:59:28.220827+0000 mon.b (mon.1) 108 : audit [DBG] from='client.? v1:172.21.15.104:0/1056372450' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:59:29.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:29 smithi166 bash[9446]: audit 2020-09-08T15:59:28.547939+0000 mon.a (mon.0) 1821 : audit [INF] from='client.? v1:172.21.15.104:0/3382219831' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:59:29.568 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:29.671 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:59:29.835 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:59:29 smithi055 bash[16646]: debug 2020-09-08T15:59:29.564+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:29.835 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:59:29.836 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:59:29 smithi055 bash[11956]: debug 2020-09-08T15:59:29.836+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:29.939 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:30.100 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:30.205 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:59:30.213 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:59:30 smithi166 bash[16593]: debug 2020-09-08T15:59:30.096+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:30.213 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:30 smithi166 bash[9446]: cluster 2020-09-08T15:59:29.085093+0000 mon.a (mon.0) 1822 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:59:30.214 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:30 smithi166 bash[9446]: cluster 2020-09-08T15:59:29.085129+0000 mon.a (mon.0) 1823 : cluster [INF] Cluster is now healthy 2020-09-08T15:59:30.214 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:30 smithi166 bash[9446]: audit 2020-09-08T15:59:29.088848+0000 mon.a (mon.0) 1824 : audit [INF] from='client.? v1:172.21.15.104:0/3382219831' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:59:30.214 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:30 smithi166 bash[9446]: cluster 2020-09-08T15:59:29.088885+0000 mon.a (mon.0) 1825 : cluster [DBG] osdmap e535: 12 total, 12 up, 12 in 2020-09-08T15:59:30.215 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:30 smithi166 bash[9446]: cluster 2020-09-08T15:59:29.612549+0000 osd.3 (osd.3) 87 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:30.215 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:30 smithi166 bash[9446]: cluster 2020-09-08T15:59:29.612631+0000 osd.3 (osd.3) 88 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:30.233 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:59:30.234 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T15:59:30.234 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.234 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.16014.0:202506 6.5 6:a27c230c:::benchmark_data_smithi166_43187_object82737:head [delete] snapc 0=[] ondisk+write+known_if_redirected e536)", 2020-09-08T15:59:30.234 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:59:30.222798+0000", 2020-09-08T15:59:30.234 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.015855261999999998, 2020-09-08T15:59:30.235 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.015889001, 2020-09-08T15:59:30.235 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:59:30.235 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:59:30.235 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:59:30.236 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.16014", 2020-09-08T15:59:30.236 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:30.236 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 202506 2020-09-08T15:59:30.237 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.237 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:59:30.237 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.237 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:59:30.237 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.222798+0000", 2020-09-08T15:59:30.238 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:59:30.238 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.238 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.238 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:59:30.239 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.222798+0000", 2020-09-08T15:59:30.239 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999619 2020-09-08T15:59:30.239 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.240 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.240 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:59:30.240 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.222760+0000", 2020-09-08T15:59:30.240 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.6050000000000005e-05 2020-09-08T15:59:30.240 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.241 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.241 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:59:30.241 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.222846+0000", 2020-09-08T15:59:30.241 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2811e-05 2020-09-08T15:59:30.242 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.242 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.242 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:59:30.242 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.222859+0000", 2020-09-08T15:59:30.242 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0800000000000002e-06 2020-09-08T15:59:30.243 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.243 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.243 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:59:30.244 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.222865+0000", 2020-09-08T15:59:30.244 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00068233599999999997 2020-09-08T15:59:30.244 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.244 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.244 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:59:30.245 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.223547+0000", 2020-09-08T15:59:30.245 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013452900000000001 2020-09-08T15:59:30.245 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.245 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.245 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:59:30.246 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.223682+0000", 2020-09-08T15:59:30.246 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013829000000000001 2020-09-08T15:59:30.246 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.246 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.247 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 6", 2020-09-08T15:59:30.247 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.223820+0000", 2020-09-08T15:59:30.247 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.013313774 2020-09-08T15:59:30.247 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.248 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.248 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:59:30.248 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.237134+0000", 2020-09-08T15:59:30.248 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.014335522 2020-09-08T15:59:30.249 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:59:30.249 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:59:30.249 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:59:30.250 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.250 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.250 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.16014.0:202512 6.8 6:13ff709e:::benchmark_data_smithi166_43187_object82743:head [delete] snapc 0=[] ondisk+write+known_if_redirected e536)", 2020-09-08T15:59:30.250 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:59:30.231141+0000", 2020-09-08T15:59:30.250 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.0075128469999999996, 2020-09-08T15:59:30.251 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0076661639999999996, 2020-09-08T15:59:30.251 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:59:30.251 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "waiting for sub ops", 2020-09-08T15:59:30.251 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:59:30.252 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.16014", 2020-09-08T15:59:30.252 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:30.252 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 202512 2020-09-08T15:59:30.253 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.253 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:59:30.253 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.253 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:59:30.253 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.231141+0000", 2020-09-08T15:59:30.254 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:59:30.254 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.254 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.254 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:59:30.255 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.231141+0000", 2020-09-08T15:59:30.255 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999557 2020-09-08T15:59:30.255 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.255 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.256 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:59:30.256 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.231096+0000", 2020-09-08T15:59:30.256 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011425999999999999 2020-09-08T15:59:30.256 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.256 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.257 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:59:30.257 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.231211+0000", 2020-09-08T15:59:30.257 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5846999999999999e-05 2020-09-08T15:59:30.257 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.258 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.258 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:59:30.258 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.231226+0000", 2020-09-08T15:59:30.258 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.0670000000000003e-06 2020-09-08T15:59:30.258 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.259 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.259 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:59:30.259 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.231233+0000", 2020-09-08T15:59:30.259 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.4171999999999994e-05 2020-09-08T15:59:30.260 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.260 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.260 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:59:30.260 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.231308+0000", 2020-09-08T15:59:30.261 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00024343600000000001 2020-09-08T15:59:30.261 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.261 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.261 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:59:30.262 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.231551+0000", 2020-09-08T15:59:30.262 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000184675 2020-09-08T15:59:30.262 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.262 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.262 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 11", 2020-09-08T15:59:30.263 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.231736+0000", 2020-09-08T15:59:30.263 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0059722029999999997 2020-09-08T15:59:30.263 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.263 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.263 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T15:59:30.264 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.237708+0000", 2020-09-08T15:59:30.264 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0065674110000000004 2020-09-08T15:59:30.264 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:59:30.264 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:59:30.264 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:59:30.265 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.265 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.265 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop_reply(client.16014.0:202499 6.5 e536/478 ondisk, result = 0)", 2020-09-08T15:59:30.265 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:59:30.238486+0000", 2020-09-08T15:59:30.265 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.00016737300000000001, 2020-09-08T15:59:30.266 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00042103799999999999, 2020-09-08T15:59:30.266 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:59:30.266 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "started", 2020-09-08T15:59:30.266 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:59:30.266 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.267 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:59:30.267 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238486+0000", 2020-09-08T15:59:30.267 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:59:30.268 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.268 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.268 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:59:30.268 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238486+0000", 2020-09-08T15:59:30.268 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.1171999999999998e-05 2020-09-08T15:59:30.269 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.269 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.269 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:59:30.269 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238527+0000", 2020-09-08T15:59:30.269 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2159999999999997e-06 2020-09-08T15:59:30.270 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.270 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.270 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:59:30.271 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238532+0000", 2020-09-08T15:59:30.271 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6864999999999999e-05 2020-09-08T15:59:30.271 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.271 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.272 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:59:30.272 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238549+0000", 2020-09-08T15:59:30.272 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.6099999999999997e-06 2020-09-08T15:59:30.272 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.273 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.273 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:59:30.273 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238555+0000", 2020-09-08T15:59:30.273 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.7713000000000003e-05 2020-09-08T15:59:30.274 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.274 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.274 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:59:30.274 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238593+0000", 2020-09-08T15:59:30.274 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.8870000000000004e-06 2020-09-08T15:59:30.275 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.275 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.275 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:59:30.275 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238603+0000", 2020-09-08T15:59:30.275 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000116463 2020-09-08T15:59:30.276 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:59:30.276 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:59:30.276 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:59:30.276 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.276 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.277 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop_reply(client.16014.0:202506 6.5 e536/478 ondisk, result = 0)", 2020-09-08T15:59:30.277 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:59:30.238620+0000", 2020-09-08T15:59:30.277 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 3.3076e-05, 2020-09-08T15:59:30.277 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00036168099999999998, 2020-09-08T15:59:30.277 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:59:30.278 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "started", 2020-09-08T15:59:30.278 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:59:30.278 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.278 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:59:30.279 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238620+0000", 2020-09-08T15:59:30.279 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:59:30.279 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.279 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.279 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:59:30.280 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238620+0000", 2020-09-08T15:59:30.280 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.189e-05 2020-09-08T15:59:30.280 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.280 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.281 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:59:30.281 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238672+0000", 2020-09-08T15:59:30.281 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.5719999999999997e-06 2020-09-08T15:59:30.281 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.282 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.282 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:59:30.282 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238680+0000", 2020-09-08T15:59:30.282 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2665e-05 2020-09-08T15:59:30.282 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.283 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.283 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:59:30.283 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238692+0000", 2020-09-08T15:59:30.283 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.1619999999999999e-06 2020-09-08T15:59:30.283 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.284 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.284 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:59:30.284 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238698+0000", 2020-09-08T15:59:30.284 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000238364 2020-09-08T15:59:30.285 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.285 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.285 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:59:30.285 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238936+0000", 2020-09-08T15:59:30.285 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.2519999999999996e-06 2020-09-08T15:59:30.286 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.286 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.286 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:59:30.286 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238942+0000", 2020-09-08T15:59:30.287 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00032190499999999999 2020-09-08T15:59:30.287 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:59:30.287 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:59:30.287 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:59:30.287 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.288 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.288 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.16014.0:202519 6.3 6:cba2a8ae:::benchmark_data_smithi166_43187_object82750:head [delete] snapc 0=[] ondisk+write+known_if_redirected e536)", 2020-09-08T15:59:30.288 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:59:30.238680+0000", 2020-09-08T15:59:30.288 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 4294967295.9999728, 2020-09-08T15:59:30.289 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00038558199999999998, 2020-09-08T15:59:30.289 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T15:59:30.289 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "started", 2020-09-08T15:59:30.289 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T15:59:30.289 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.16014", 2020-09-08T15:59:30.290 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:30.296 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 202519 2020-09-08T15:59:30.297 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.297 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T15:59:30.297 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.297 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T15:59:30.298 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238680+0000", 2020-09-08T15:59:30.298 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T15:59:30.298 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.298 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.298 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T15:59:30.299 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238680+0000", 2020-09-08T15:59:30.299 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999958 2020-09-08T15:59:30.299 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.299 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.299 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T15:59:30.300 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238639+0000", 2020-09-08T15:59:30.300 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000108811 2020-09-08T15:59:30.300 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.300 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.300 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T15:59:30.301 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238748+0000", 2020-09-08T15:59:30.301 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4952e-05 2020-09-08T15:59:30.301 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.301 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.302 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T15:59:30.302 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238762+0000", 2020-09-08T15:59:30.302 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.1600000000000003e-06 2020-09-08T15:59:30.302 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.303 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.303 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T15:59:30.303 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238769+0000", 2020-09-08T15:59:30.303 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.4436000000000004e-05 2020-09-08T15:59:30.303 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.304 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.304 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T15:59:30.304 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.238843+0000", 2020-09-08T15:59:30.304 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015850900000000001 2020-09-08T15:59:30.304 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T15:59:30.305 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T15:59:30.305 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T15:59:30.305 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:59:30.239002+0000", 2020-09-08T15:59:30.305 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0003211 2020-09-08T15:59:30.305 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:59:30.306 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T15:59:30.306 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:59:30.306 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T15:59:30.307 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2020-09-08T15:59:30.307 INFO:teuthology.orchestra.run.smithi055.stdout: "num_ops": 5 2020-09-08T15:59:30.307 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:59:30.327 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:59:30.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:30 smithi055 bash[9296]: cluster 2020-09-08T15:59:29.085093+0000 mon.a (mon.0) 1822 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:59:30.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:30 smithi055 bash[9296]: cluster 2020-09-08T15:59:29.085129+0000 mon.a (mon.0) 1823 : cluster [INF] Cluster is now healthy 2020-09-08T15:59:30.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:30 smithi055 bash[9296]: audit 2020-09-08T15:59:29.088848+0000 mon.a (mon.0) 1824 : audit [INF] from='client.? v1:172.21.15.104:0/3382219831' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:59:30.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:30 smithi055 bash[9296]: cluster 2020-09-08T15:59:29.088885+0000 mon.a (mon.0) 1825 : cluster [DBG] osdmap e535: 12 total, 12 up, 12 in 2020-09-08T15:59:30.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:30 smithi055 bash[9296]: cluster 2020-09-08T15:59:29.612549+0000 osd.3 (osd.3) 87 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:30.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:30 smithi055 bash[9296]: cluster 2020-09-08T15:59:29.612631+0000 osd.3 (osd.3) 88 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:30.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:30 smithi104 bash[9203]: cluster 2020-09-08T15:59:29.085093+0000 mon.a (mon.0) 1822 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T15:59:30.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:30 smithi104 bash[9203]: cluster 2020-09-08T15:59:29.085129+0000 mon.a (mon.0) 1823 : cluster [INF] Cluster is now healthy 2020-09-08T15:59:30.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:30 smithi104 bash[9203]: audit 2020-09-08T15:59:29.088848+0000 mon.a (mon.0) 1824 : audit [INF] from='client.? v1:172.21.15.104:0/3382219831' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T15:59:30.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:30 smithi104 bash[9203]: cluster 2020-09-08T15:59:29.088885+0000 mon.a (mon.0) 1825 : cluster [DBG] osdmap e535: 12 total, 12 up, 12 in 2020-09-08T15:59:30.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:30 smithi104 bash[9203]: cluster 2020-09-08T15:59:29.612549+0000 osd.3 (osd.3) 87 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:30.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:30 smithi104 bash[9203]: cluster 2020-09-08T15:59:29.612631+0000 osd.3 (osd.3) 88 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:30.432 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:30.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:59:30 smithi166 bash[11928]: debug 2020-09-08T15:59:30.324+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:31.104 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:31 smithi104 bash[9203]: cluster 2020-09-08T15:59:29.946007+0000 mgr.x (mgr.24118) 965 : cluster [DBG] pgmap v1416: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 2.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 892 op/s 2020-09-08T15:59:31.105 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:31 smithi104 bash[9203]: cluster 2020-09-08T15:59:30.111021+0000 mon.a (mon.0) 1826 : cluster [DBG] osdmap e536: 12 total, 12 up, 12 in 2020-09-08T15:59:31.160 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:31.263 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:31.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:31 smithi166 bash[9446]: cluster 2020-09-08T15:59:29.946007+0000 mgr.x (mgr.24118) 965 : cluster [DBG] pgmap v1416: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 2.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 892 op/s 2020-09-08T15:59:31.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:31 smithi166 bash[9446]: cluster 2020-09-08T15:59:30.111021+0000 mon.a (mon.0) 1826 : cluster [DBG] osdmap e536: 12 total, 12 up, 12 in 2020-09-08T15:59:31.393 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:31.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:31 smithi055 bash[9296]: cluster 2020-09-08T15:59:29.946007+0000 mgr.x (mgr.24118) 965 : cluster [DBG] pgmap v1416: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 2.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 892 op/s 2020-09-08T15:59:31.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:31 smithi055 bash[9296]: cluster 2020-09-08T15:59:30.111021+0000 mon.a (mon.0) 1826 : cluster [DBG] osdmap e536: 12 total, 12 up, 12 in 2020-09-08T15:59:31.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:59:31 smithi055 bash[16646]: debug 2020-09-08T15:59:31.160+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:31.498 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:59:31.524 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2020-09-08T15:59:31.611 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:59:31.704 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:59:31.714 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:31.724 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:59:31 smithi166 bash[11928]: debug 2020-09-08T15:59:31.608+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:31.725 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:59:31 smithi166 bash[14254]: debug 2020-09-08T15:59:31.392+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:31.863 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:31.968 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:59:32.117 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:59:31 smithi166 bash[14254]: debug 2020-09-08T15:59:31.860+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:32.461 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:59:32.568 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:59:32.704 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:59:32.808 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:32.843 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T15:59:32.857 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:59:32.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:59:32 smithi166 bash[18960]: debug 2020-09-08T15:59:32.700+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:32.906 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:59:32 smithi055 bash[11956]: debug 2020-09-08T15:59:32.460+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:32.974 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:33.056 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:59:33.057 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:59:33.077 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:33.114 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:33 smithi104 bash[9203]: cluster 2020-09-08T15:59:31.946857+0000 mgr.x (mgr.24118) 966 : cluster [DBG] pgmap v1418: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 2.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 897 op/s 2020-09-08T15:59:33.114 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:59:32 smithi104 bash[15273]: debug 2020-09-08T15:59:32.972+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:33.190 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:33.291 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:59:33.373 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:33 smithi166 bash[9446]: cluster 2020-09-08T15:59:31.946857+0000 mgr.x (mgr.24118) 966 : cluster [DBG] pgmap v1418: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 2.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 897 op/s 2020-09-08T15:59:33.374 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:59:33 smithi166 bash[16593]: debug 2020-09-08T15:59:33.184+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:33.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:33 smithi055 bash[9296]: cluster 2020-09-08T15:59:31.946857+0000 mgr.x (mgr.24118) 966 : cluster [DBG] pgmap v1418: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 2.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 897 op/s 2020-09-08T15:59:33.456 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:59:33.560 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:33.681 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:33.788 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:33.867 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:59:33 smithi166 bash[16593]: debug 2020-09-08T15:59:33.676+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:33.880 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:59:33 smithi055 bash[19003]: debug 2020-09-08T15:59:33.452+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:34.347 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:34.453 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:59:34.564 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:59:34.668 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:34.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:59:34 smithi104 bash[17588]: debug 2020-09-08T15:59:34.344+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:34.821 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:59:34 smithi166 bash[11928]: debug 2020-09-08T15:59:34.560+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:34.822 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:34.926 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:35.064 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:35.110 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:59:35 smithi166 bash[14254]: debug 2020-09-08T15:59:35.056+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:35.111 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:59:34 smithi166 bash[16593]: debug 2020-09-08T15:59:34.820+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:35.168 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:35.351 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:35.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:35 smithi166 bash[9446]: cluster 2020-09-08T15:59:33.947354+0000 mgr.x (mgr.24118) 967 : cluster [DBG] pgmap v1419: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 2.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 970 op/s 2020-09-08T15:59:35.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:35 smithi055 bash[9296]: cluster 2020-09-08T15:59:33.947354+0000 mgr.x (mgr.24118) 967 : cluster [DBG] pgmap v1419: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 2.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 970 op/s 2020-09-08T15:59:35.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:35 smithi104 bash[9203]: cluster 2020-09-08T15:59:33.947354+0000 mgr.x (mgr.24118) 967 : cluster [DBG] pgmap v1419: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 2.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 970 op/s 2020-09-08T15:59:35.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:59:35 smithi104 bash[22305]: debug 2020-09-08T15:59:35.352+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:35.455 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:35.976 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:36.080 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:59:36.250 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:59:36 smithi055 bash[19003]: debug 2020-09-08T15:59:36.252+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:36.251 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:59:35 smithi055 bash[14301]: debug 2020-09-08T15:59:35.976+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:36.252 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:59:36.356 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:36.491 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:36.599 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:59:36.622 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:59:36 smithi166 bash[14254]: debug 2020-09-08T15:59:36.488+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:36.720 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:59:36.795 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T15:59:36.795 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T15:59:36.796 INFO:teuthology.orchestra.run.smithi055.stdout: "complaint_time": 30, 2020-09-08T15:59:36.796 INFO:teuthology.orchestra.run.smithi055.stdout: "num_blocked_ops": 0 2020-09-08T15:59:36.796 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T15:59:36.825 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:36.895 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:59:36 smithi055 bash[19003]: debug 2020-09-08T15:59:36.720+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:36.947 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:37.051 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:37.113 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:59:36 smithi166 bash[16593]: debug 2020-09-08T15:59:36.944+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:37.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:37 smithi166 bash[9446]: cluster 2020-09-08T15:59:35.948065+0000 mgr.x (mgr.24118) 968 : cluster [DBG] pgmap v1420: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 2.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 745 op/s 2020-09-08T15:59:37.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:37 smithi055 bash[9296]: cluster 2020-09-08T15:59:35.948065+0000 mgr.x (mgr.24118) 968 : cluster [DBG] pgmap v1420: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 2.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 745 op/s 2020-09-08T15:59:37.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:37 smithi104 bash[9203]: cluster 2020-09-08T15:59:35.948065+0000 mgr.x (mgr.24118) 968 : cluster [DBG] pgmap v1420: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 2.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 745 op/s 2020-09-08T15:59:37.935 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2020-09-08T15:59:38.042 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:38.105 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T15:59:38.146 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:38.339 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:38 smithi104 bash[9203]: cluster 2020-09-08T15:59:36.555151+0000 osd.8 (osd.8) 71 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:38.340 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:38 smithi104 bash[9203]: cluster 2020-09-08T15:59:36.555263+0000 osd.8 (osd.8) 72 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:38.340 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:38 smithi104 bash[9203]: cluster 2020-09-08T15:59:37.127139+0000 mon.a (mon.0) 1827 : cluster [DBG] osdmap e537: 12 total, 12 up, 12 in 2020-09-08T15:59:38.340 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:59:38 smithi104 bash[15273]: debug 2020-09-08T15:59:38.040+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:38.341 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:59:38 smithi104 bash[22305]: debug 2020-09-08T15:59:38.336+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:38.341 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:38.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:38 smithi055 bash[9296]: cluster 2020-09-08T15:59:36.555151+0000 osd.8 (osd.8) 71 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:38.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:38 smithi055 bash[9296]: cluster 2020-09-08T15:59:36.555263+0000 osd.8 (osd.8) 72 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:38.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:38 smithi055 bash[9296]: cluster 2020-09-08T15:59:37.127139+0000 mon.a (mon.0) 1827 : cluster [DBG] osdmap e537: 12 total, 12 up, 12 in 2020-09-08T15:59:38.445 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:59:38.454 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:38 smithi166 bash[9446]: cluster 2020-09-08T15:59:36.555151+0000 osd.8 (osd.8) 71 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:38.454 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:38 smithi166 bash[9446]: cluster 2020-09-08T15:59:36.555263+0000 osd.8 (osd.8) 72 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:38.455 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:38 smithi166 bash[9446]: cluster 2020-09-08T15:59:37.127139+0000 mon.a (mon.0) 1827 : cluster [DBG] osdmap e537: 12 total, 12 up, 12 in 2020-09-08T15:59:38.586 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:59:38.690 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:38.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:59:38 smithi166 bash[18960]: debug 2020-09-08T15:59:38.584+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:38.980 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:38.981 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:59:38 smithi055 bash[16646]: debug 2020-09-08T15:59:38.980+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:39.084 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:39.236 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:39.339 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:39.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:39 smithi055 bash[9296]: cluster 2020-09-08T15:59:37.948551+0000 mgr.x (mgr.24118) 969 : cluster [DBG] pgmap v1422: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.03k op/s 2020-09-08T15:59:39.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:39 smithi104 bash[9203]: cluster 2020-09-08T15:59:37.948551+0000 mgr.x (mgr.24118) 969 : cluster [DBG] pgmap v1422: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.03k op/s 2020-09-08T15:59:39.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:59:39 smithi104 bash[15273]: debug 2020-09-08T15:59:39.236+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:39.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:39 smithi166 bash[9446]: cluster 2020-09-08T15:59:37.948551+0000 mgr.x (mgr.24118) 969 : cluster [DBG] pgmap v1422: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.03k op/s 2020-09-08T15:59:39.963 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:40.066 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:59:40.194 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:59:40.297 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:40.305 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:59:39 smithi104 bash[19922]: debug 2020-09-08T15:59:39.960+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:40.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:59:40 smithi166 bash[11928]: debug 2020-09-08T15:59:40.192+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:40.569 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:40.672 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:59:40.822 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:59:40 smithi104 bash[22305]: debug 2020-09-08T15:59:40.568+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:41.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:40 smithi055 bash[9296]: cluster 2020-09-08T15:59:39.948969+0000 mgr.x (mgr.24118) 970 : cluster [DBG] pgmap v1423: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 929 op/s 2020-09-08T15:59:41.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:40 smithi104 bash[9203]: cluster 2020-09-08T15:59:39.948969+0000 mgr.x (mgr.24118) 970 : cluster [DBG] pgmap v1423: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 929 op/s 2020-09-08T15:59:41.201 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:59:41.220 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:59:41.266 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:59:41.302 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:59:41.304 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:41.343 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:59:41.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:40 smithi166 bash[9446]: cluster 2020-09-08T15:59:39.948969+0000 mgr.x (mgr.24118) 970 : cluster [DBG] pgmap v1423: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 929 op/s 2020-09-08T15:59:41.390 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:59:41.439 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:59:41.493 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:59:41.513 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:59:41.546 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:59:41.579 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:59:41.612 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:59:41.650 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:59:41.655 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T15:59:41.655 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T15:59:41.655 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T15:59:41.655 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T15:59:41.656 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T15:59:41.656 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T15:59:41.656 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T15:59:41.656 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T15:59:41.656 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T15:59:41.657 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T15:59:41.657 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T15:59:41.657 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T15:59:41.658 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:59:41 smithi055 bash[19003]: debug 2020-09-08T15:59:41.212+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:41.856 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:41.911 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T15:59:41.969 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:42.009 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:41 smithi104 bash[9203]: audit 2020-09-08T15:59:41.195469+0000 mon.a (mon.0) 1828 : audit [DBG] from='client.? v1:172.21.15.104:0/959779196' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:59:42.010 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:41 smithi104 bash[9203]: audit 2020-09-08T15:59:41.511152+0000 mon.c (mon.2) 73 : audit [INF] from='client.? v1:172.21.15.104:0/2748205112' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:59:42.010 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:41 smithi104 bash[9203]: audit 2020-09-08T15:59:41.513972+0000 mon.a (mon.0) 1829 : audit [INF] from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:59:42.110 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:42.199 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:41 smithi055 bash[9296]: audit 2020-09-08T15:59:41.195469+0000 mon.a (mon.0) 1828 : audit [DBG] from='client.? v1:172.21.15.104:0/959779196' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:59:42.199 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:41 smithi055 bash[9296]: audit 2020-09-08T15:59:41.511152+0000 mon.c (mon.2) 73 : audit [INF] from='client.? v1:172.21.15.104:0/2748205112' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:59:42.200 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:41 smithi055 bash[9296]: audit 2020-09-08T15:59:41.513972+0000 mon.a (mon.0) 1829 : audit [INF] from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:59:42.200 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:59:41 smithi055 bash[16646]: debug 2020-09-08T15:59:41.852+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:42.228 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:59:42.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:41 smithi166 bash[9446]: audit 2020-09-08T15:59:41.195469+0000 mon.a (mon.0) 1828 : audit [DBG] from='client.? v1:172.21.15.104:0/959779196' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:59:42.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:41 smithi166 bash[9446]: audit 2020-09-08T15:59:41.511152+0000 mon.c (mon.2) 73 : audit [INF] from='client.? v1:172.21.15.104:0/2748205112' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:59:42.376 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:41 smithi166 bash[9446]: audit 2020-09-08T15:59:41.513972+0000 mon.a (mon.0) 1829 : audit [INF] from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T15:59:42.377 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:59:42 smithi166 bash[16593]: debug 2020-09-08T15:59:42.108+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:42.833 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:59:42.936 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:43.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:59:42 smithi055 bash[11956]: debug 2020-09-08T15:59:42.832+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:43.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:42 smithi055 bash[9296]: cluster 2020-09-08T15:59:41.903155+0000 mon.a (mon.0) 1830 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:59:43.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:42 smithi055 bash[9296]: audit 2020-09-08T15:59:41.914038+0000 mon.a (mon.0) 1831 : audit [INF] from='client.36035 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:59:43.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:42 smithi055 bash[9296]: cluster 2020-09-08T15:59:41.914076+0000 mon.a (mon.0) 1832 : cluster [DBG] osdmap e538: 12 total, 12 up, 12 in 2020-09-08T15:59:43.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:42 smithi055 bash[9296]: cluster 2020-09-08T15:59:41.949404+0000 mgr.x (mgr.24118) 971 : cluster [DBG] pgmap v1425: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 1.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 857 op/s 2020-09-08T15:59:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:42 smithi104 bash[9203]: cluster 2020-09-08T15:59:41.903155+0000 mon.a (mon.0) 1830 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:59:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:42 smithi104 bash[9203]: audit 2020-09-08T15:59:41.914038+0000 mon.a (mon.0) 1831 : audit [INF] from='client.36035 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:59:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:42 smithi104 bash[9203]: cluster 2020-09-08T15:59:41.914076+0000 mon.a (mon.0) 1832 : cluster [DBG] osdmap e538: 12 total, 12 up, 12 in 2020-09-08T15:59:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:42 smithi104 bash[9203]: cluster 2020-09-08T15:59:41.949404+0000 mgr.x (mgr.24118) 971 : cluster [DBG] pgmap v1425: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 1.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 857 op/s 2020-09-08T15:59:43.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:42 smithi166 bash[9446]: cluster 2020-09-08T15:59:41.903155+0000 mon.a (mon.0) 1830 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:59:43.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:42 smithi166 bash[9446]: audit 2020-09-08T15:59:41.914038+0000 mon.a (mon.0) 1831 : audit [INF] from='client.36035 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T15:59:43.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:42 smithi166 bash[9446]: cluster 2020-09-08T15:59:41.914076+0000 mon.a (mon.0) 1832 : cluster [DBG] osdmap e538: 12 total, 12 up, 12 in 2020-09-08T15:59:43.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:42 smithi166 bash[9446]: cluster 2020-09-08T15:59:41.949404+0000 mgr.x (mgr.24118) 971 : cluster [DBG] pgmap v1425: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 1.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 857 op/s 2020-09-08T15:59:43.390 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T15:59:43.390 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:59:43.715 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:43.820 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:43.955 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:43.972 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:59:43 smithi104 bash[15273]: debug 2020-09-08T15:59:43.720+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:44.058 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:44.125 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:59:43 smithi166 bash[14254]: debug 2020-09-08T15:59:43.952+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:44.195 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:44.300 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:44.312 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:59:44 smithi104 bash[22305]: debug 2020-09-08T15:59:44.200+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:44.455 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:44.559 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:44.566 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:59:44 smithi104 bash[19922]: debug 2020-09-08T15:59:44.456+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:44.627 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2020-09-08T15:59:44.668 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:44.778 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:44.840 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:59:44.892 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:44.916 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:44 smithi104 bash[9203]: cluster 2020-09-08T15:59:43.410815+0000 osd.1 (osd.1) 108 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:44.916 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:44 smithi104 bash[9203]: cluster 2020-09-08T15:59:43.410913+0000 osd.1 (osd.1) 109 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:44.916 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:44 smithi104 bash[9203]: cluster 2020-09-08T15:59:43.918548+0000 mon.a (mon.0) 1833 : cluster [DBG] osdmap e539: 12 total, 12 up, 12 in 2020-09-08T15:59:44.918 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:44 smithi104 bash[9203]: cluster 2020-09-08T15:59:43.949854+0000 mgr.x (mgr.24118) 972 : cluster [DBG] pgmap v1427: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:59:44.925 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:59:44 smithi104 bash[15273]: debug 2020-09-08T15:59:44.672+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:45.001 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:59:45.011 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:44 smithi166 bash[9446]: cluster 2020-09-08T15:59:43.410815+0000 osd.1 (osd.1) 108 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:45.011 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:44 smithi166 bash[9446]: cluster 2020-09-08T15:59:43.410913+0000 osd.1 (osd.1) 109 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:45.011 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:44 smithi166 bash[9446]: cluster 2020-09-08T15:59:43.918548+0000 mon.a (mon.0) 1833 : cluster [DBG] osdmap e539: 12 total, 12 up, 12 in 2020-09-08T15:59:45.011 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:44 smithi166 bash[9446]: cluster 2020-09-08T15:59:43.949854+0000 mgr.x (mgr.24118) 972 : cluster [DBG] pgmap v1427: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:59:45.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:59:44 smithi055 bash[16646]: debug 2020-09-08T15:59:44.892+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:45.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:44 smithi055 bash[9296]: cluster 2020-09-08T15:59:43.410815+0000 osd.1 (osd.1) 108 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:45.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:44 smithi055 bash[9296]: cluster 2020-09-08T15:59:43.410913+0000 osd.1 (osd.1) 109 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:45.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:44 smithi055 bash[9296]: cluster 2020-09-08T15:59:43.918548+0000 mon.a (mon.0) 1833 : cluster [DBG] osdmap e539: 12 total, 12 up, 12 in 2020-09-08T15:59:45.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:44 smithi055 bash[9296]: cluster 2020-09-08T15:59:43.949854+0000 mgr.x (mgr.24118) 972 : cluster [DBG] pgmap v1427: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T15:59:45.874 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T15:59:45.974 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T15:59:46.042 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:59:46.148 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:59:46.480 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:59:46.586 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:59:46.745 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:59:46 smithi166 bash[11928]: debug 2020-09-08T15:59:46.476+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:46.746 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:59:46.854 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:46.998 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:59:46 smithi166 bash[11928]: debug 2020-09-08T15:59:46.744+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:47.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:47 smithi104 bash[9203]: cluster 2020-09-08T15:59:45.950332+0000 mgr.x (mgr.24118) 973 : cluster [DBG] pgmap v1428: 17 pgs: 17 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 756 op/s 2020-09-08T15:59:47.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:47 smithi166 bash[9446]: cluster 2020-09-08T15:59:45.950332+0000 mgr.x (mgr.24118) 973 : cluster [DBG] pgmap v1428: 17 pgs: 17 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 756 op/s 2020-09-08T15:59:47.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:47 smithi055 bash[9296]: cluster 2020-09-08T15:59:45.950332+0000 mgr.x (mgr.24118) 973 : cluster [DBG] pgmap v1428: 17 pgs: 17 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 756 op/s 2020-09-08T15:59:47.598 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:47.704 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:59:47.865 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:59:47.927 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:59:47 smithi104 bash[19922]: debug 2020-09-08T15:59:47.596+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:47.968 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:48.004 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:59:47 smithi055 bash[19003]: debug 2020-09-08T15:59:47.864+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:48.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:48 smithi166 bash[9446]: cluster 2020-09-08T15:59:47.300186+0000 osd.6 (osd.6) 85 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:48.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:48 smithi166 bash[9446]: cluster 2020-09-08T15:59:47.300278+0000 osd.6 (osd.6) 86 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:48.268 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:48 smithi104 bash[9203]: cluster 2020-09-08T15:59:47.300186+0000 osd.6 (osd.6) 85 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:48.269 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:48 smithi104 bash[9203]: cluster 2020-09-08T15:59:47.300278+0000 osd.6 (osd.6) 86 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:48.275 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:48.378 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:48.387 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:48 smithi055 bash[9296]: cluster 2020-09-08T15:59:47.300186+0000 osd.6 (osd.6) 85 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:48.387 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:48 smithi055 bash[9296]: cluster 2020-09-08T15:59:47.300278+0000 osd.6 (osd.6) 86 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:48.542 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:48.649 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:48.663 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:59:48 smithi055 bash[14301]: debug 2020-09-08T15:59:48.544+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:48.664 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:59:48 smithi104 bash[17588]: debug 2020-09-08T15:59:48.272+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:48.847 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:48.928 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:59:48 smithi104 bash[19922]: debug 2020-09-08T15:59:48.848+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:48.952 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:49.083 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:49.186 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:49.230 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:49 smithi104 bash[9203]: cluster 2020-09-08T15:59:47.950778+0000 mgr.x (mgr.24118) 974 : cluster [DBG] pgmap v1429: 17 pgs: 17 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T15:59:49.230 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:49 smithi104 bash[9203]: cluster 2020-09-08T15:59:48.013459+0000 mon.a (mon.0) 1834 : cluster [DBG] osdmap e540: 12 total, 12 up, 12 in 2020-09-08T15:59:49.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:49 smithi166 bash[9446]: cluster 2020-09-08T15:59:47.950778+0000 mgr.x (mgr.24118) 974 : cluster [DBG] pgmap v1429: 17 pgs: 17 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T15:59:49.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:49 smithi166 bash[9446]: cluster 2020-09-08T15:59:48.013459+0000 mon.a (mon.0) 1834 : cluster [DBG] osdmap e540: 12 total, 12 up, 12 in 2020-09-08T15:59:49.391 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:49.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:49 smithi055 bash[9296]: cluster 2020-09-08T15:59:47.950778+0000 mgr.x (mgr.24118) 974 : cluster [DBG] pgmap v1429: 17 pgs: 17 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T15:59:49.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:49 smithi055 bash[9296]: cluster 2020-09-08T15:59:48.013459+0000 mon.a (mon.0) 1834 : cluster [DBG] osdmap e540: 12 total, 12 up, 12 in 2020-09-08T15:59:49.408 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:59:49 smithi055 bash[14301]: debug 2020-09-08T15:59:49.084+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:49.495 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:59:49.628 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:59:49.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 15:59:49 smithi104 bash[22305]: debug 2020-09-08T15:59:49.396+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:49.731 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:59:49.899 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T15:59:49.909 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:59:49 smithi055 bash[11956]: debug 2020-09-08T15:59:49.628+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:49.918 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T15:59:49.919 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T15:59:49.999 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:50.369 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 15:59:49 smithi166 bash[11928]: debug 2020-09-08T15:59:49.892+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:50.692 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:50.717 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2020-09-08T15:59:50.794 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:50.900 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:59:50.989 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:59:50 smithi104 bash[19922]: debug 2020-09-08T15:59:50.692+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:50.989 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:59:50 smithi104 bash[19922]: debug 2020-09-08T15:59:50.992+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:50.990 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:50 smithi104 bash[9203]: cluster 2020-09-08T15:59:49.951257+0000 mgr.x (mgr.24118) 975 : cluster [DBG] pgmap v1431: 17 pgs: 17 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 975 op/s 2020-09-08T15:59:50.990 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:51.097 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:51.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:50 smithi166 bash[9446]: cluster 2020-09-08T15:59:49.951257+0000 mgr.x (mgr.24118) 975 : cluster [DBG] pgmap v1431: 17 pgs: 17 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 975 op/s 2020-09-08T15:59:51.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:50 smithi055 bash[9296]: cluster 2020-09-08T15:59:49.951257+0000 mgr.x (mgr.24118) 975 : cluster [DBG] pgmap v1431: 17 pgs: 17 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 975 op/s 2020-09-08T15:59:51.867 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:51.970 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:52.084 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:52.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:51 smithi055 bash[9296]: cluster 2020-09-08T15:59:51.513124+0000 osd.11 (osd.11) 90 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:52.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:51 smithi055 bash[9296]: cluster 2020-09-08T15:59:51.513231+0000 osd.11 (osd.11) 91 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:52.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:51 smithi104 bash[9203]: cluster 2020-09-08T15:59:51.513124+0000 osd.11 (osd.11) 90 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:52.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:51 smithi104 bash[9203]: cluster 2020-09-08T15:59:51.513231+0000 osd.11 (osd.11) 91 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:52.177 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:59:51 smithi104 bash[15273]: debug 2020-09-08T15:59:51.868+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:52.188 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:59:52.236 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:59:52 smithi166 bash[16593]: debug 2020-09-08T15:59:52.080+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:52.236 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:51 smithi166 bash[9446]: cluster 2020-09-08T15:59:51.513124+0000 osd.11 (osd.11) 90 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:52.237 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:51 smithi166 bash[9446]: cluster 2020-09-08T15:59:51.513231+0000 osd.11 (osd.11) 91 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:52.332 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:59:52.437 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:52.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:59:52 smithi055 bash[11956]: debug 2020-09-08T15:59:52.332+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:52.740 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:52.846 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:52.974 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T15:59:53.077 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:53.109 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:52 smithi055 bash[9296]: cluster 2020-09-08T15:59:51.912190+0000 mon.a (mon.0) 1835 : cluster [DBG] osdmap e541: 12 total, 12 up, 12 in 2020-09-08T15:59:53.109 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:52 smithi055 bash[9296]: cluster 2020-09-08T15:59:51.951683+0000 mgr.x (mgr.24118) 976 : cluster [DBG] pgmap v1433: 17 pgs: 17 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 876 op/s 2020-09-08T15:59:53.109 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 15:59:52 smithi055 bash[14301]: debug 2020-09-08T15:59:52.972+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:53.178 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:59:52 smithi104 bash[17588]: debug 2020-09-08T15:59:52.744+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:53.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:52 smithi104 bash[9203]: cluster 2020-09-08T15:59:51.912190+0000 mon.a (mon.0) 1835 : cluster [DBG] osdmap e541: 12 total, 12 up, 12 in 2020-09-08T15:59:53.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:52 smithi104 bash[9203]: cluster 2020-09-08T15:59:51.951683+0000 mgr.x (mgr.24118) 976 : cluster [DBG] pgmap v1433: 17 pgs: 17 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 876 op/s 2020-09-08T15:59:53.235 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:53.339 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:53.351 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:52 smithi166 bash[9446]: cluster 2020-09-08T15:59:51.912190+0000 mon.a (mon.0) 1835 : cluster [DBG] osdmap e541: 12 total, 12 up, 12 in 2020-09-08T15:59:53.351 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:52 smithi166 bash[9446]: cluster 2020-09-08T15:59:51.951683+0000 mgr.x (mgr.24118) 976 : cluster [DBG] pgmap v1433: 17 pgs: 17 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 876 op/s 2020-09-08T15:59:53.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:59:53 smithi055 bash[16646]: debug 2020-09-08T15:59:53.232+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:53.496 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:53.601 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:59:53.617 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:59:53 smithi166 bash[16593]: debug 2020-09-08T15:59:53.492+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:53.630 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T15:59:53.670 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T15:59:53.705 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T15:59:53.729 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:59:53.759 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T15:59:53.802 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T15:59:53.832 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:53.836 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T15:59:53.884 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T15:59:53.906 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:59:53 smithi055 bash[11956]: debug 2020-09-08T15:59:53.732+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:53.922 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T15:59:53.954 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T15:59:53.978 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:53.987 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T15:59:54.026 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T15:59:54.060 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T15:59:54.070 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T15:59:54.071 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T15:59:54.071 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T15:59:54.071 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T15:59:54.071 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T15:59:54.072 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T15:59:54.072 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T15:59:54.072 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T15:59:54.072 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T15:59:54.072 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T15:59:54.073 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T15:59:54.073 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T15:59:54.083 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:54.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:53 smithi104 bash[9203]: audit 2020-09-08T15:59:53.596951+0000 mon.c (mon.2) 74 : audit [DBG] from='client.? v1:172.21.15.104:0/834134620' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:59:54.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:59:53 smithi104 bash[15273]: debug 2020-09-08T15:59:53.972+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:54.213 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:53 smithi055 bash[9296]: audit 2020-09-08T15:59:53.596951+0000 mon.c (mon.2) 74 : audit [DBG] from='client.? v1:172.21.15.104:0/834134620' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:59:54.213 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:59:54 smithi055 bash[16646]: debug 2020-09-08T15:59:54.212+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:54.215 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:54.299 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:59:54.300 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T15:59:54.300 INFO:teuthology.orchestra.run.smithi166.stdout: "complaint_time": 30, 2020-09-08T15:59:54.300 INFO:teuthology.orchestra.run.smithi166.stdout: "num_blocked_ops": 0 2020-09-08T15:59:54.300 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:59:54.322 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:54.332 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:53 smithi166 bash[9446]: audit 2020-09-08T15:59:53.596951+0000 mon.c (mon.2) 74 : audit [DBG] from='client.? v1:172.21.15.104:0/834134620' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T15:59:54.524 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T15:59:54.628 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:54.636 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 15:59:54 smithi166 bash[16593]: debug 2020-09-08T15:59:54.520+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:54.913 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:54 smithi104 bash[9203]: cluster 2020-09-08T15:59:53.952103+0000 mgr.x (mgr.24118) 977 : cluster [DBG] pgmap v1434: 17 pgs: 17 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 971 op/s 2020-09-08T15:59:54.914 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:54 smithi104 bash[9203]: audit 2020-09-08T15:59:54.030418+0000 mon.a (mon.0) 1836 : audit [INF] from='client.? v1:172.21.15.104:0/1429653974' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:59:54.926 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T15:59:55.247 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:54 smithi166 bash[9446]: cluster 2020-09-08T15:59:53.952103+0000 mgr.x (mgr.24118) 977 : cluster [DBG] pgmap v1434: 17 pgs: 17 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 971 op/s 2020-09-08T15:59:55.247 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:54 smithi166 bash[9446]: audit 2020-09-08T15:59:54.030418+0000 mon.a (mon.0) 1836 : audit [INF] from='client.? v1:172.21.15.104:0/1429653974' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:59:55.248 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2020-09-08T15:59:55.321 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:55.398 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T15:59:55.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:54 smithi055 bash[9296]: cluster 2020-09-08T15:59:53.952103+0000 mgr.x (mgr.24118) 977 : cluster [DBG] pgmap v1434: 17 pgs: 17 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 971 op/s 2020-09-08T15:59:55.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:54 smithi055 bash[9296]: audit 2020-09-08T15:59:54.030418+0000 mon.a (mon.0) 1836 : audit [INF] from='client.? v1:172.21.15.104:0/1429653974' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T15:59:55.426 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:55.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:59:55 smithi104 bash[17588]: debug 2020-09-08T15:59:55.324+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:56.172 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:56.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:55 smithi104 bash[9203]: cluster 2020-09-08T15:59:54.918396+0000 mon.a (mon.0) 1837 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:59:56.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:55 smithi104 bash[9203]: audit 2020-09-08T15:59:54.929509+0000 mon.a (mon.0) 1838 : audit [INF] from='client.? v1:172.21.15.104:0/1429653974' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:59:56.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:55 smithi104 bash[9203]: cluster 2020-09-08T15:59:54.929560+0000 mon.a (mon.0) 1839 : cluster [DBG] osdmap e542: 12 total, 12 up, 12 in 2020-09-08T15:59:56.177 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:59:56 smithi104 bash[15273]: debug 2020-09-08T15:59:56.168+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:56.277 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:59:56.304 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:55 smithi166 bash[9446]: cluster 2020-09-08T15:59:54.918396+0000 mon.a (mon.0) 1837 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:59:56.304 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:55 smithi166 bash[9446]: audit 2020-09-08T15:59:54.929509+0000 mon.a (mon.0) 1838 : audit [INF] from='client.? v1:172.21.15.104:0/1429653974' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:59:56.305 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:55 smithi166 bash[9446]: cluster 2020-09-08T15:59:54.929560+0000 mon.a (mon.0) 1839 : cluster [DBG] osdmap e542: 12 total, 12 up, 12 in 2020-09-08T15:59:56.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:55 smithi055 bash[9296]: cluster 2020-09-08T15:59:54.918396+0000 mon.a (mon.0) 1837 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T15:59:56.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:55 smithi055 bash[9296]: audit 2020-09-08T15:59:54.929509+0000 mon.a (mon.0) 1838 : audit [INF] from='client.? v1:172.21.15.104:0/1429653974' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T15:59:56.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:55 smithi055 bash[9296]: cluster 2020-09-08T15:59:54.929560+0000 mon.a (mon.0) 1839 : cluster [DBG] osdmap e542: 12 total, 12 up, 12 in 2020-09-08T15:59:56.461 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:59:56.565 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:59:56.631 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:59:56 smithi166 bash[18960]: debug 2020-09-08T15:59:56.456+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:56.703 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T15:59:56.805 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:56.910 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 15:59:56 smithi055 bash[11956]: debug 2020-09-08T15:59:56.696+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:56.930 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T15:59:57.031 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:59:57.157 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:59:57.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:56 smithi104 bash[9203]: cluster 2020-09-08T15:59:55.952541+0000 mgr.x (mgr.24118) 978 : cluster [DBG] pgmap v1436: 17 pgs: 17 active+clean; 1006 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 754 op/s 2020-09-08T15:59:57.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:56 smithi104 bash[9203]: cluster 2020-09-08T15:59:56.301316+0000 osd.4 (osd.4) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:57.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:56 smithi104 bash[9203]: cluster 2020-09-08T15:59:56.301395+0000 osd.4 (osd.4) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:57.178 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 15:59:56 smithi104 bash[17588]: debug 2020-09-08T15:59:56.928+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:57.262 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:57.271 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:56 smithi166 bash[9446]: cluster 2020-09-08T15:59:55.952541+0000 mgr.x (mgr.24118) 978 : cluster [DBG] pgmap v1436: 17 pgs: 17 active+clean; 1006 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 754 op/s 2020-09-08T15:59:57.271 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:56 smithi166 bash[9446]: cluster 2020-09-08T15:59:56.301316+0000 osd.4 (osd.4) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:57.271 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:56 smithi166 bash[9446]: cluster 2020-09-08T15:59:56.301395+0000 osd.4 (osd.4) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:57.272 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:59:57 smithi166 bash[18960]: debug 2020-09-08T15:59:57.156+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:57.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:56 smithi055 bash[9296]: cluster 2020-09-08T15:59:55.952541+0000 mgr.x (mgr.24118) 978 : cluster [DBG] pgmap v1436: 17 pgs: 17 active+clean; 1006 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 754 op/s 2020-09-08T15:59:57.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:56 smithi055 bash[9296]: cluster 2020-09-08T15:59:56.301316+0000 osd.4 (osd.4) 57 : cluster [DBG] purged_snaps scrub starts 2020-09-08T15:59:57.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:56 smithi055 bash[9296]: cluster 2020-09-08T15:59:56.301395+0000 osd.4 (osd.4) 58 : cluster [DBG] purged_snaps scrub ok 2020-09-08T15:59:57.626 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T15:59:57.734 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:59:57.883 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:59:57.935 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 15:59:57 smithi166 bash[14254]: debug 2020-09-08T15:59:57.624+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:57.987 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:58.025 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T15:59:58.150 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T15:59:58.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:57 smithi055 bash[9296]: cluster 2020-09-08T15:59:56.937974+0000 mon.a (mon.0) 1840 : cluster [DBG] osdmap e543: 12 total, 12 up, 12 in 2020-09-08T15:59:58.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:59:57 smithi055 bash[19003]: debug 2020-09-08T15:59:57.884+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:58.197 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:59:58.255 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:59:58.266 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:57 smithi166 bash[9446]: cluster 2020-09-08T15:59:56.937974+0000 mon.a (mon.0) 1840 : cluster [DBG] osdmap e543: 12 total, 12 up, 12 in 2020-09-08T15:59:58.296 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T15:59:58.316 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 15:59:58 smithi104 bash[19922]: debug 2020-09-08T15:59:58.148+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:58.323 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:57 smithi104 bash[9203]: cluster 2020-09-08T15:59:56.937974+0000 mon.a (mon.0) 1840 : cluster [DBG] osdmap e543: 12 total, 12 up, 12 in 2020-09-08T15:59:58.384 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T15:59:58.451 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T15:59:58.503 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:59:58.619 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 15:59:58 smithi166 bash[18960]: debug 2020-09-08T15:59:58.380+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:58.620 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T15:59:58.728 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:58.855 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 15:59:58 smithi055 bash[19003]: debug 2020-09-08T15:59:58.620+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:58.856 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T15:59:58.959 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:59.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 15:59:58 smithi055 bash[9296]: cluster 2020-09-08T15:59:57.952952+0000 mgr.x (mgr.24118) 979 : cluster [DBG] pgmap v1438: 17 pgs: 17 active+clean; 809 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.11k op/s 2020-09-08T15:59:59.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 15:59:58 smithi055 bash[16646]: debug 2020-09-08T15:59:58.840+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T15:59:59.204 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 15:59:58 smithi166 bash[9446]: cluster 2020-09-08T15:59:57.952952+0000 mgr.x (mgr.24118) 979 : cluster [DBG] pgmap v1438: 17 pgs: 17 active+clean; 809 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.11k op/s 2020-09-08T15:59:59.372 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T15:59:59.373 INFO:teuthology.orchestra.run.smithi166.stdout: "size": 20, 2020-09-08T15:59:59.373 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 600, 2020-09-08T15:59:59.373 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T15:59:59.373 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.373 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.16014.0:167699 6.1 6:8469a7ba:::benchmark_data_smithi166_43187_object47930:head [delete] snapc 0=[] ondisk+write+known_if_redirected e522)", 2020-09-08T15:59:59.374 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:58:47.786529+0000", 2020-09-08T15:59:59.374 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 71.581655171999998, 2020-09-08T15:59:59.374 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.31170601100000001, 2020-09-08T15:59:59.375 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:59:59.375 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:59:59.375 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:59:59.375 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.16014", 2020-09-08T15:59:59.375 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:59.376 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 167699 2020-09-08T15:59:59.376 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.376 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:59:59.376 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.376 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:59:59.377 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.786529+0000", 2020-09-08T15:59:59.377 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:59:59.377 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.377 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.377 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:59:59.378 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.786529+0000", 2020-09-08T15:59:59.378 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999762 2020-09-08T15:59:59.378 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.378 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.379 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:59:59.379 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.786506+0000", 2020-09-08T15:59:59.379 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.4542000000000001e-05 2020-09-08T15:59:59.379 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.379 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.380 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:59:59.380 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.786560+0000", 2020-09-08T15:59:59.380 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0723999999999999e-05 2020-09-08T15:59:59.380 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.380 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.381 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:59:59.381 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.786571+0000", 2020-09-08T15:59:59.381 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0139999999999998e-06 2020-09-08T15:59:59.381 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.381 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.381 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:59:59.382 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.786576+0000", 2020-09-08T15:59:59.382 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.073737443 2020-09-08T15:59:59.382 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.382 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.382 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.383 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.860313+0000", 2020-09-08T15:59:59.383 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2213999999999995e-05 2020-09-08T15:59:59.383 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.383 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.383 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:59:59.384 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.860406+0000", 2020-09-08T15:59:59.384 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.1240000000000001e-05 2020-09-08T15:59:59.384 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.385 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.385 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 9", 2020-09-08T15:59:59.385 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.860497+0000", 2020-09-08T15:59:59.385 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.006192782 2020-09-08T15:59:59.385 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.386 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.386 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:59:59.386 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.866690+0000", 2020-09-08T15:59:59.386 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.231331592 2020-09-08T15:59:59.386 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.387 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.387 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:59:59.387 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.098021+0000", 2020-09-08T15:59:59.387 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013225800000000001 2020-09-08T15:59:59.387 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.387 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.388 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:59:59.388 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.098153+0000", 2020-09-08T15:59:59.388 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.1910999999999996e-05 2020-09-08T15:59:59.388 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.388 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.389 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:59:59.389 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.098235+0000", 2020-09-08T15:59:59.389 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.31170601100000001 2020-09-08T15:59:59.389 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.389 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.390 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.390 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.390 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.390 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.16014.0:167734 6.1 6:866f219c:::benchmark_data_smithi166_43187_object47965:head [delete] snapc 0=[] ondisk+write+known_if_redirected e522)", 2020-09-08T15:59:59.390 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:58:47.825215+0000", 2020-09-08T15:59:59.391 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 71.542969260000007, 2020-09-08T15:59:59.391 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.28568626899999999, 2020-09-08T15:59:59.391 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:59:59.391 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:59:59.391 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:59:59.392 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.16014", 2020-09-08T15:59:59.392 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:59.392 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 167734 2020-09-08T15:59:59.392 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.392 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:59:59.392 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.393 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:59:59.393 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.825215+0000", 2020-09-08T15:59:59.393 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:59:59.393 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.393 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.394 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:59:59.394 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.825215+0000", 2020-09-08T15:59:59.394 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999614 2020-09-08T15:59:59.394 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.395 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.395 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:59:59.395 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.825177+0000", 2020-09-08T15:59:59.395 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8144000000000002e-05 2020-09-08T15:59:59.396 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.396 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.396 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:59:59.396 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.825265+0000", 2020-09-08T15:59:59.396 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6441000000000001e-05 2020-09-08T15:59:59.396 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.397 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.397 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:59:59.397 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.825281+0000", 2020-09-08T15:59:59.397 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.6939999999999995e-06 2020-09-08T15:59:59.397 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.398 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.398 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:59:59.398 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.825289+0000", 2020-09-08T15:59:59.398 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.03755054 2020-09-08T15:59:59.398 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.399 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.399 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.399 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.862839+0000", 2020-09-08T15:59:59.399 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000138567 2020-09-08T15:59:59.399 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.400 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.400 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:59:59.400 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.862978+0000", 2020-09-08T15:59:59.400 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000128007 2020-09-08T15:59:59.400 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.401 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.401 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 9", 2020-09-08T15:59:59.401 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.863106+0000", 2020-09-08T15:59:59.401 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0035912940000000001 2020-09-08T15:59:59.401 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.401 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.402 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:59:59.402 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.866697+0000", 2020-09-08T15:59:59.402 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.24405167899999999 2020-09-08T15:59:59.402 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.402 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.403 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:59:59.403 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.110749+0000", 2020-09-08T15:59:59.403 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.3163000000000002e-05 2020-09-08T15:59:59.403 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.403 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.404 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:59:59.404 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.110842+0000", 2020-09-08T15:59:59.404 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.9437e-05 2020-09-08T15:59:59.404 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.404 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.404 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:59:59.405 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.110902+0000", 2020-09-08T15:59:59.405 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.28568626899999999 2020-09-08T15:59:59.405 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.406 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.406 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.406 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.406 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.406 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.16014.0:167811 6.1 6:8077eea5:::benchmark_data_smithi166_43187_object48042:head [delete] snapc 0=[] ondisk+write+known_if_redirected e522)", 2020-09-08T15:59:59.407 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:58:47.915643+0000", 2020-09-08T15:59:59.407 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 71.452541490000002, 2020-09-08T15:59:59.407 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.195607489, 2020-09-08T15:59:59.407 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:59:59.407 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:59:59.408 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:59:59.408 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.16014", 2020-09-08T15:59:59.408 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:59.408 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 167811 2020-09-08T15:59:59.408 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.408 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:59:59.409 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.409 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:59:59.409 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.915643+0000", 2020-09-08T15:59:59.409 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:59:59.409 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.410 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.410 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:59:59.410 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.915643+0000", 2020-09-08T15:59:59.410 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999361 2020-09-08T15:59:59.410 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.411 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.411 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:59:59.411 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.915579+0000", 2020-09-08T15:59:59.411 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013407299999999999 2020-09-08T15:59:59.412 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.412 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.412 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:59:59.412 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.915713+0000", 2020-09-08T15:59:59.412 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.7396e-05 2020-09-08T15:59:59.412 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.413 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.413 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:59:59.413 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.915740+0000", 2020-09-08T15:59:59.413 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3738e-05 2020-09-08T15:59:59.413 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.414 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.414 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:59:59.414 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.915754+0000", 2020-09-08T15:59:59.414 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000106136 2020-09-08T15:59:59.414 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.415 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.415 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.415 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.915860+0000", 2020-09-08T15:59:59.416 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00026118199999999999 2020-09-08T15:59:59.416 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.416 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.416 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:59:59.416 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.916122+0000", 2020-09-08T15:59:59.417 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00031303400000000001 2020-09-08T15:59:59.417 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.417 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.417 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 9", 2020-09-08T15:59:59.417 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.916435+0000", 2020-09-08T15:59:59.418 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0070675529999999999 2020-09-08T15:59:59.418 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.418 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.418 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:59:59.418 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.923502+0000", 2020-09-08T15:59:59.418 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.187521154 2020-09-08T15:59:59.419 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.419 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.419 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:59:59.419 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.111023+0000", 2020-09-08T15:59:59.419 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000120971 2020-09-08T15:59:59.420 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.420 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.420 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:59:59.420 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.111144+0000", 2020-09-08T15:59:59.420 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000106308 2020-09-08T15:59:59.421 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.421 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.421 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:59:59.421 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.111251+0000", 2020-09-08T15:59:59.421 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.195607489 2020-09-08T15:59:59.422 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.422 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.422 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.422 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.422 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.423 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.16014.0:167847 6.1 6:89bb6c2d:::benchmark_data_smithi166_43187_object48078:head [delete] snapc 0=[] ondisk+write+known_if_redirected e523)", 2020-09-08T15:59:59.423 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:58:47.964545+0000", 2020-09-08T15:59:59.423 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 71.403639815000005, 2020-09-08T15:59:59.423 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14697974999999999, 2020-09-08T15:59:59.423 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:59:59.424 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:59:59.424 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:59:59.424 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.16014", 2020-09-08T15:59:59.424 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:59.425 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 167847 2020-09-08T15:59:59.425 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.425 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:59:59.425 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.426 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:59:59.426 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.964545+0000", 2020-09-08T15:59:59.426 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:59:59.426 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.426 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.427 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:59:59.427 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.964545+0000", 2020-09-08T15:59:59.427 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999151 2020-09-08T15:59:59.427 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.427 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.428 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:59:59.428 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.964460+0000", 2020-09-08T15:59:59.428 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000176317 2020-09-08T15:59:59.428 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.428 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.429 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:59:59.429 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.964636+0000", 2020-09-08T15:59:59.429 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.8070000000000001e-05 2020-09-08T15:59:59.429 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.429 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.430 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:59:59.430 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.964664+0000", 2020-09-08T15:59:59.430 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.394e-05 2020-09-08T15:59:59.430 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.430 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.431 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:59:59.431 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.964678+0000", 2020-09-08T15:59:59.431 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000104103 2020-09-08T15:59:59.431 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.431 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.431 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.432 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.964782+0000", 2020-09-08T15:59:59.432 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3708e-05 2020-09-08T15:59:59.432 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.432 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.432 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op must wait for map", 2020-09-08T15:59:59.433 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.964796+0000", 2020-09-08T15:59:59.433 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.002340871 2020-09-08T15:59:59.433 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.433 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.433 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.434 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.967137+0000", 2020-09-08T15:59:59.434 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00021765 2020-09-08T15:59:59.434 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.434 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.434 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:59:59.434 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.967354+0000", 2020-09-08T15:59:59.435 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00022529599999999999 2020-09-08T15:59:59.435 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.435 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.435 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 9", 2020-09-08T15:59:59.435 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.967580+0000", 2020-09-08T15:59:59.436 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0069870610000000001 2020-09-08T15:59:59.436 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.436 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.437 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:59:59.437 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:47.974567+0000", 2020-09-08T15:59:59.437 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.136779347 2020-09-08T15:59:59.437 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.437 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.438 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:59:59.438 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.111346+0000", 2020-09-08T15:59:59.438 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011578099999999999 2020-09-08T15:59:59.438 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.438 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.439 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:59:59.439 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.111462+0000", 2020-09-08T15:59:59.439 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.2705000000000004e-05 2020-09-08T15:59:59.439 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.439 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.439 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:59:59.440 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.111524+0000", 2020-09-08T15:59:59.440 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14697974999999999 2020-09-08T15:59:59.440 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.440 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.440 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.441 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.441 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.441 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.16014.0:167983 6.1 6:813cbaf9:::benchmark_data_smithi166_43187_object48214:head [delete] snapc 0=[] ondisk+write+known_if_redirected e523)", 2020-09-08T15:59:59.441 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:58:48.153079+0000", 2020-09-08T15:59:59.441 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 71.215105707999996, 2020-09-08T15:59:59.442 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.149020078, 2020-09-08T15:59:59.443 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:59:59.443 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:59:59.443 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:59:59.443 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.16014", 2020-09-08T15:59:59.443 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:59.444 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 167983 2020-09-08T15:59:59.444 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.444 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:59:59.444 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.444 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:59:59.445 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.153079+0000", 2020-09-08T15:59:59.445 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:59:59.445 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.445 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.445 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:59:59.446 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.153079+0000", 2020-09-08T15:59:59.446 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999075 2020-09-08T15:59:59.446 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.446 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.446 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:59:59.447 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.152986+0000", 2020-09-08T15:59:59.447 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00020004699999999999 2020-09-08T15:59:59.447 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.447 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.447 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:59:59.448 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.153187+0000", 2020-09-08T15:59:59.448 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.4440999999999998e-05 2020-09-08T15:59:59.448 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.449 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.449 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:59:59.449 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.153221+0000", 2020-09-08T15:59:59.449 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4905999999999999e-05 2020-09-08T15:59:59.449 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.449 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.450 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:59:59.450 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.153236+0000", 2020-09-08T15:59:59.450 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.10364907299999999 2020-09-08T15:59:59.450 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.450 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.451 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.451 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.256885+0000", 2020-09-08T15:59:59.452 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013056400000000001 2020-09-08T15:59:59.452 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.452 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.452 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:59:59.452 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.257016+0000", 2020-09-08T15:59:59.453 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012909099999999999 2020-09-08T15:59:59.453 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.453 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.453 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 9", 2020-09-08T15:59:59.453 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.257145+0000", 2020-09-08T15:59:59.454 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0094468810000000007 2020-09-08T15:59:59.454 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.454 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.454 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:59:59.454 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.266591+0000", 2020-09-08T15:59:59.455 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.035376167 2020-09-08T15:59:59.455 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.455 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.455 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:59:59.455 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.301968+0000", 2020-09-08T15:59:59.456 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1551999999999997e-05 2020-09-08T15:59:59.456 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.456 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.456 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:59:59.456 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.302019+0000", 2020-09-08T15:59:59.457 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.9723999999999998e-05 2020-09-08T15:59:59.457 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.457 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.457 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:59:59.457 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:48.302099+0000", 2020-09-08T15:59:59.458 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.149020078 2020-09-08T15:59:59.458 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.458 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.458 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.458 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.458 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.459 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.16014.0:170033 6.1 6:8dd23b17:::benchmark_data_smithi166_43187_object50264:head [delete] snapc 0=[] ondisk+write+known_if_redirected e523)", 2020-09-08T15:59:59.459 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:58:50.638389+0000", 2020-09-08T15:59:59.459 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 68.729795451000001, 2020-09-08T15:59:59.459 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17652477499999999, 2020-09-08T15:59:59.459 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:59:59.460 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:59:59.460 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:59:59.460 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.16014", 2020-09-08T15:59:59.460 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:59.460 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 170033 2020-09-08T15:59:59.461 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.461 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:59:59.461 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.461 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:59:59.461 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.638389+0000", 2020-09-08T15:59:59.462 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:59:59.462 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.462 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.462 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:59:59.462 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.638389+0000", 2020-09-08T15:59:59.463 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999375 2020-09-08T15:59:59.463 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.463 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.463 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:59:59.463 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.638326+0000", 2020-09-08T15:59:59.463 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000140834 2020-09-08T15:59:59.464 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.464 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.464 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:59:59.464 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.638467+0000", 2020-09-08T15:59:59.464 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.5863999999999998e-05 2020-09-08T15:59:59.465 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.465 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.465 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:59:59.465 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.638493+0000", 2020-09-08T15:59:59.465 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1187e-05 2020-09-08T15:59:59.466 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.466 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.466 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:59:59.466 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.638504+0000", 2020-09-08T15:59:59.466 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.039672622999999997 2020-09-08T15:59:59.467 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.467 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.468 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.468 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.678177+0000", 2020-09-08T15:59:59.468 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000168364 2020-09-08T15:59:59.468 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.468 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.469 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:59:59.469 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.678345+0000", 2020-09-08T15:59:59.469 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019783999999999999 2020-09-08T15:59:59.469 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.469 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.470 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 9", 2020-09-08T15:59:59.470 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.678543+0000", 2020-09-08T15:59:59.470 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.014642591999999999 2020-09-08T15:59:59.470 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.470 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.471 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:59:59.471 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.693186+0000", 2020-09-08T15:59:59.471 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.121610337 2020-09-08T15:59:59.471 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.471 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.471 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:59:59.472 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.814796+0000", 2020-09-08T15:59:59.472 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.8003999999999995e-05 2020-09-08T15:59:59.472 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.472 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.472 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:59:59.473 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.814864+0000", 2020-09-08T15:59:59.473 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9780000000000001e-05 2020-09-08T15:59:59.473 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.473 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.473 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:59:59.474 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.814914+0000", 2020-09-08T15:59:59.474 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17652477499999999 2020-09-08T15:59:59.474 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.474 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.474 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.475 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.475 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.475 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.16014.0:170036 6.1 6:87d334f2:::benchmark_data_smithi166_43187_object50267:head [delete] snapc 0=[] ondisk+write+known_if_redirected e523)", 2020-09-08T15:59:59.475 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:58:50.643696+0000", 2020-09-08T15:59:59.475 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 68.724488789999995, 2020-09-08T15:59:59.475 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17138906500000001, 2020-09-08T15:59:59.476 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:59:59.476 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:59:59.476 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:59:59.476 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.16014", 2020-09-08T15:59:59.476 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:59.477 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 170036 2020-09-08T15:59:59.477 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.477 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:59:59.477 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.477 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:59:59.478 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.643696+0000", 2020-09-08T15:59:59.478 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:59:59.478 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.478 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.478 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:59:59.479 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.643696+0000", 2020-09-08T15:59:59.479 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999619 2020-09-08T15:59:59.479 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.479 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.479 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:59:59.480 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.643657+0000", 2020-09-08T15:59:59.480 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.5391999999999998e-05 2020-09-08T15:59:59.480 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.480 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.480 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:59:59.480 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.643743+0000", 2020-09-08T15:59:59.481 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4276e-05 2020-09-08T15:59:59.481 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.481 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.481 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:59:59.481 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.643757+0000", 2020-09-08T15:59:59.482 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.472e-06 2020-09-08T15:59:59.482 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.482 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.482 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:59:59.482 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.643764+0000", 2020-09-08T15:59:59.483 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.038595472999999998 2020-09-08T15:59:59.483 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.483 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.483 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.483 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.682359+0000", 2020-09-08T15:59:59.483 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000159834 2020-09-08T15:59:59.484 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.484 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.484 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:59:59.484 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.682519+0000", 2020-09-08T15:59:59.484 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000187624 2020-09-08T15:59:59.485 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.485 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.485 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 9", 2020-09-08T15:59:59.485 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.682707+0000", 2020-09-08T15:59:59.486 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.01049247 2020-09-08T15:59:59.486 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.486 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.487 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:59:59.487 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.693199+0000", 2020-09-08T15:59:59.487 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.121776738 2020-09-08T15:59:59.487 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.487 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.488 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:59:59.488 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.814976+0000", 2020-09-08T15:59:59.488 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.0336e-05 2020-09-08T15:59:59.488 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.488 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.489 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:59:59.489 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.815036+0000", 2020-09-08T15:59:59.489 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.8712999999999999e-05 2020-09-08T15:59:59.489 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.490 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.490 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:59:59.490 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.815085+0000", 2020-09-08T15:59:59.490 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17138906500000001 2020-09-08T15:59:59.490 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.491 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.491 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.491 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.491 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.491 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.16014.0:170240 6.1 6:88f2a257:::benchmark_data_smithi166_43187_object50471:head [delete] snapc 0=[] ondisk+write+known_if_redirected e523)", 2020-09-08T15:59:59.492 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:58:50.886174+0000", 2020-09-08T15:59:59.492 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 68.482010153000004, 2020-09-08T15:59:59.492 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14417057999999999, 2020-09-08T15:59:59.492 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:59:59.492 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:59:59.493 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:59:59.493 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.16014", 2020-09-08T15:59:59.493 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:59.493 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 170240 2020-09-08T15:59:59.493 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.494 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:59:59.494 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.494 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:59:59.494 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.886174+0000", 2020-09-08T15:59:59.494 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:59:59.494 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.495 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.495 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:59:59.495 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.886174+0000", 2020-09-08T15:59:59.495 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999504 2020-09-08T15:59:59.495 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.496 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.496 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:59:59.496 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.886125+0000", 2020-09-08T15:59:59.496 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000106239 2020-09-08T15:59:59.496 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.497 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.497 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:59:59.497 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.886231+0000", 2020-09-08T15:59:59.497 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.9230999999999999e-05 2020-09-08T15:59:59.497 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.498 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.498 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:59:59.498 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.886250+0000", 2020-09-08T15:59:59.498 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7819999999999994e-06 2020-09-08T15:59:59.498 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.498 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.499 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:59:59.499 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.886258+0000", 2020-09-08T15:59:59.499 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.105623305 2020-09-08T15:59:59.499 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.499 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.500 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.500 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.991881+0000", 2020-09-08T15:59:59.500 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017717199999999999 2020-09-08T15:59:59.500 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.500 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.501 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:59:59.501 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.992059+0000", 2020-09-08T15:59:59.501 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00020498099999999999 2020-09-08T15:59:59.501 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.501 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.502 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 9", 2020-09-08T15:59:59.502 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.992264+0000", 2020-09-08T15:59:59.502 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0056499999999999996 2020-09-08T15:59:59.502 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.502 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.502 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:59:59.503 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:50.997914+0000", 2020-09-08T15:59:59.503 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.032347320999999998 2020-09-08T15:59:59.503 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.503 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.503 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:59:59.504 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.030261+0000", 2020-09-08T15:59:59.504 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.8358e-05 2020-09-08T15:59:59.504 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.504 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.504 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:59:59.505 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.030309+0000", 2020-09-08T15:59:59.505 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.5667999999999998e-05 2020-09-08T15:59:59.505 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.505 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.505 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:59:59.505 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.030345+0000", 2020-09-08T15:59:59.506 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14417057999999999 2020-09-08T15:59:59.506 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.506 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.506 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.506 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.507 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.507 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.16014.0:170355 6.1 6:8ba23321:::benchmark_data_smithi166_43187_object50586:head [delete] snapc 0=[] ondisk+write+known_if_redirected e524)", 2020-09-08T15:59:59.507 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:58:51.034262+0000", 2020-09-08T15:59:59.507 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 68.333922361999996, 2020-09-08T15:59:59.507 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15020129900000001, 2020-09-08T15:59:59.508 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:59:59.508 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:59:59.508 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:59:59.508 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.16014", 2020-09-08T15:59:59.509 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:59.509 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 170355 2020-09-08T15:59:59.509 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.509 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:59:59.509 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.510 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:59:59.510 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.034262+0000", 2020-09-08T15:59:59.510 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:59:59.510 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.510 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.510 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:59:59.511 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.034262+0000", 2020-09-08T15:59:59.511 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999547 2020-09-08T15:59:59.511 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.511 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.511 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:59:59.512 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.034217+0000", 2020-09-08T15:59:59.512 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.5970000000000004e-05 2020-09-08T15:59:59.512 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.512 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.512 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:59:59.513 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.034313+0000", 2020-09-08T15:59:59.513 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.8284e-05 2020-09-08T15:59:59.513 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.513 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.514 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:59:59.515 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.034331+0000", 2020-09-08T15:59:59.515 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7880000000000001e-06 2020-09-08T15:59:59.515 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.515 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.515 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:59:59.516 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.034341+0000", 2020-09-08T15:59:59.516 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00046063799999999998 2020-09-08T15:59:59.516 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.516 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.517 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.517 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.034802+0000", 2020-09-08T15:59:59.517 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014405900000000001 2020-09-08T15:59:59.517 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.518 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.518 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:59:59.518 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.034946+0000", 2020-09-08T15:59:59.518 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000117024 2020-09-08T15:59:59.518 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.519 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.519 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 9", 2020-09-08T15:59:59.519 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.035063+0000", 2020-09-08T15:59:59.519 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0092556410000000002 2020-09-08T15:59:59.519 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.520 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.520 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:59:59.520 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.044318+0000", 2020-09-08T15:59:59.520 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.140004033 2020-09-08T15:59:59.520 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.520 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.521 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:59:59.521 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.184322+0000", 2020-09-08T15:59:59.521 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.9175999999999997e-05 2020-09-08T15:59:59.521 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.521 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.522 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:59:59.522 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.184382+0000", 2020-09-08T15:59:59.522 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.1839e-05 2020-09-08T15:59:59.522 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.522 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.523 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:59:59.523 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.184463+0000", 2020-09-08T15:59:59.523 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15020129900000001 2020-09-08T15:59:59.523 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.523 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.524 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.524 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.524 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.526 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.16014.0:170360 6.1 6:844f14c3:::benchmark_data_smithi166_43187_object50591:head [delete] snapc 0=[] ondisk+write+known_if_redirected e524)", 2020-09-08T15:59:59.526 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:58:51.039585+0000", 2020-09-08T15:59:59.526 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 68.328599725000004, 2020-09-08T15:59:59.526 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14508420699999999, 2020-09-08T15:59:59.527 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:59:59.527 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:59:59.527 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:59:59.527 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.16014", 2020-09-08T15:59:59.527 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:59.528 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 170360 2020-09-08T15:59:59.528 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.528 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:59:59.528 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.528 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:59:59.529 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.039585+0000", 2020-09-08T15:59:59.529 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:59:59.529 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.529 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.529 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:59:59.530 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.039585+0000", 2020-09-08T15:59:59.530 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999547 2020-09-08T15:59:59.530 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.530 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.530 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:59:59.531 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.039539+0000", 2020-09-08T15:59:59.531 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000101241 2020-09-08T15:59:59.531 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.531 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.531 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:59:59.532 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.039641+0000", 2020-09-08T15:59:59.532 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.0043999999999999e-05 2020-09-08T15:59:59.532 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.532 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.532 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:59:59.533 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.039671+0000", 2020-09-08T15:59:59.533 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.7279999999999998e-06 2020-09-08T15:59:59.533 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.533 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.533 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:59:59.534 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.039677+0000", 2020-09-08T15:59:59.534 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016614399999999999 2020-09-08T15:59:59.534 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.534 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.534 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.534 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.039843+0000", 2020-09-08T15:59:59.535 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000180886 2020-09-08T15:59:59.535 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.535 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.535 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:59:59.535 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.040024+0000", 2020-09-08T15:59:59.536 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000136534 2020-09-08T15:59:59.536 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.536 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.536 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 9", 2020-09-08T15:59:59.536 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.040161+0000", 2020-09-08T15:59:59.537 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0085736860000000005 2020-09-08T15:59:59.537 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.537 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.537 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:59:59.537 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.048735+0000", 2020-09-08T15:59:59.538 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.13582656100000001 2020-09-08T15:59:59.538 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.538 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.538 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:59:59.538 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.184561+0000", 2020-09-08T15:59:59.539 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6835999999999995e-05 2020-09-08T15:59:59.539 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.539 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.539 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:59:59.539 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.184628+0000", 2020-09-08T15:59:59.539 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.1078000000000002e-05 2020-09-08T15:59:59.540 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.540 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.540 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:59:59.540 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.184669+0000", 2020-09-08T15:59:59.540 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14508420699999999 2020-09-08T15:59:59.541 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.541 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.541 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.541 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.541 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.542 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.16014.0:170362 6.1 6:8c58f5ba:::benchmark_data_smithi166_43187_object50593:head [delete] snapc 0=[] ondisk+write+known_if_redirected e524)", 2020-09-08T15:59:59.542 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:58:51.039981+0000", 2020-09-08T15:59:59.542 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 68.328203670999997, 2020-09-08T15:59:59.542 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14481702299999999, 2020-09-08T15:59:59.542 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:59:59.543 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:59:59.543 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:59:59.543 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.16014", 2020-09-08T15:59:59.543 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:59.543 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 170362 2020-09-08T15:59:59.544 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.544 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:59:59.544 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.544 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:59:59.544 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.039981+0000", 2020-09-08T15:59:59.545 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:59:59.545 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.545 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.545 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:59:59.545 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.039981+0000", 2020-09-08T15:59:59.545 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999547 2020-09-08T15:59:59.546 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.546 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.546 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:59:59.546 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.039936+0000", 2020-09-08T15:59:59.546 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.8768e-05 2020-09-08T15:59:59.547 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.547 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.547 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:59:59.547 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.040034+0000", 2020-09-08T15:59:59.547 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5173e-05 2020-09-08T15:59:59.548 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.548 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.548 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:59:59.548 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.040050+0000", 2020-09-08T15:59:59.548 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.7170000000000002e-06 2020-09-08T15:59:59.549 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.549 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.549 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:59:59.550 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.040056+0000", 2020-09-08T15:59:59.550 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0044843460000000002 2020-09-08T15:59:59.550 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.550 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.550 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.551 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.044541+0000", 2020-09-08T15:59:59.551 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014248599999999999 2020-09-08T15:59:59.551 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.551 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.551 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:59:59.551 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.044683+0000", 2020-09-08T15:59:59.552 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000108819 2020-09-08T15:59:59.552 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.552 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.552 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 9", 2020-09-08T15:59:59.552 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.044792+0000", 2020-09-08T15:59:59.553 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0040374989999999999 2020-09-08T15:59:59.553 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.553 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.553 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:59:59.553 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.048829+0000", 2020-09-08T15:59:59.554 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.13588389200000001 2020-09-08T15:59:59.554 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.554 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.554 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:59:59.554 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.184713+0000", 2020-09-08T15:59:59.555 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.4245000000000001e-05 2020-09-08T15:59:59.555 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.555 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.555 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:59:59.555 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.184758+0000", 2020-09-08T15:59:59.555 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.0376000000000001e-05 2020-09-08T15:59:59.556 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.556 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.556 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:59:59.556 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.184798+0000", 2020-09-08T15:59:59.556 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14481702299999999 2020-09-08T15:59:59.557 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.557 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.557 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.557 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.557 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.558 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.16014.0:170501 6.1 6:801b30c5:::benchmark_data_smithi166_43187_object50732:head [delete] snapc 0=[] ondisk+write+known_if_redirected e524)", 2020-09-08T15:59:59.558 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:58:51.227069+0000", 2020-09-08T15:59:59.558 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 68.141115302000003, 2020-09-08T15:59:59.558 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15208179399999999, 2020-09-08T15:59:59.558 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:59:59.559 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:59:59.559 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:59:59.559 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.16014", 2020-09-08T15:59:59.559 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:59.559 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 170501 2020-09-08T15:59:59.560 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.560 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:59:59.560 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.560 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:59:59.560 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.227069+0000", 2020-09-08T15:59:59.560 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:59:59.561 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.561 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.561 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:59:59.561 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.227069+0000", 2020-09-08T15:59:59.561 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999757 2020-09-08T15:59:59.562 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.562 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.562 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:59:59.562 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.227045+0000", 2020-09-08T15:59:59.562 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.4033000000000003e-05 2020-09-08T15:59:59.563 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.563 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.563 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:59:59.563 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.227099+0000", 2020-09-08T15:59:59.563 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.0469999999999999e-06 2020-09-08T15:59:59.564 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.564 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.564 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:59:59.564 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.227108+0000", 2020-09-08T15:59:59.564 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.921e-06 2020-09-08T15:59:59.564 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.565 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.565 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:59:59.565 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.227113+0000", 2020-09-08T15:59:59.565 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.022446059000000001 2020-09-08T15:59:59.565 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.566 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.566 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.566 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.249559+0000", 2020-09-08T15:59:59.566 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013539100000000001 2020-09-08T15:59:59.566 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.570 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.570 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:59:59.570 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.249694+0000", 2020-09-08T15:59:59.571 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000139333 2020-09-08T15:59:59.571 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.571 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.571 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 9", 2020-09-08T15:59:59.571 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.249834+0000", 2020-09-08T15:59:59.572 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0042530850000000002 2020-09-08T15:59:59.572 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.572 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.572 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:59:59.572 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.254087+0000", 2020-09-08T15:59:59.573 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.124899759 2020-09-08T15:59:59.573 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.573 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.573 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:59:59.573 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.378986+0000", 2020-09-08T15:59:59.574 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.8912000000000006e-05 2020-09-08T15:59:59.574 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.574 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.574 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:59:59.574 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.379065+0000", 2020-09-08T15:59:59.575 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.5691000000000003e-05 2020-09-08T15:59:59.575 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.575 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.575 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:59:59.575 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.379151+0000", 2020-09-08T15:59:59.575 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15208179399999999 2020-09-08T15:59:59.576 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.576 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.576 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.576 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.576 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.577 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.16014.0:171001 6.1 6:8ea75d51:::benchmark_data_smithi166_43187_object51232:head [delete] snapc 0=[] ondisk+write+known_if_redirected e524)", 2020-09-08T15:59:59.577 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:58:51.810016+0000", 2020-09-08T15:59:59.577 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 67.558168291000001, 2020-09-08T15:59:59.577 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14430812600000001, 2020-09-08T15:59:59.577 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:59:59.578 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:59:59.578 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:59:59.578 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.16014", 2020-09-08T15:59:59.578 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:59.578 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 171001 2020-09-08T15:59:59.579 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.579 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:59:59.579 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.579 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:59:59.579 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.810016+0000", 2020-09-08T15:59:59.580 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:59:59.580 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.580 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.580 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:59:59.580 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.810016+0000", 2020-09-08T15:59:59.581 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999595 2020-09-08T15:59:59.581 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.581 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.581 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:59:59.581 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.809976+0000", 2020-09-08T15:59:59.581 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.0988000000000004e-05 2020-09-08T15:59:59.582 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.582 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.582 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:59:59.582 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.810067+0000", 2020-09-08T15:59:59.582 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5118e-05 2020-09-08T15:59:59.583 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.583 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.583 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:59:59.583 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.810082+0000", 2020-09-08T15:59:59.583 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.8210000000000004e-06 2020-09-08T15:59:59.584 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.584 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.584 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:59:59.584 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.810089+0000", 2020-09-08T15:59:59.584 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.10558941299999999 2020-09-08T15:59:59.585 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.585 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.585 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.585 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.915678+0000", 2020-09-08T15:59:59.585 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000158682 2020-09-08T15:59:59.585 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.586 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.586 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:59:59.586 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.915837+0000", 2020-09-08T15:59:59.586 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.8325000000000005e-05 2020-09-08T15:59:59.586 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.587 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.587 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 9", 2020-09-08T15:59:59.587 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.915935+0000", 2020-09-08T15:59:59.587 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0049043979999999999 2020-09-08T15:59:59.587 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.588 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.588 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:59:59.588 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.920839+0000", 2020-09-08T15:59:59.588 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.033400665000000003 2020-09-08T15:59:59.588 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.589 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.589 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:59:59.589 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.954240+0000", 2020-09-08T15:59:59.589 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.8971999999999998e-05 2020-09-08T15:59:59.589 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.589 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.590 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:59:59.590 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.954289+0000", 2020-09-08T15:59:59.590 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.5386999999999998e-05 2020-09-08T15:59:59.590 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.590 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.591 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:59:59.591 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.954324+0000", 2020-09-08T15:59:59.591 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14430812600000001 2020-09-08T15:59:59.591 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.591 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.592 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.592 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.592 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.592 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.16014.0:171002 6.1 6:859d2f82:::benchmark_data_smithi166_43187_object51233:head [delete] snapc 0=[] ondisk+write+known_if_redirected e524)", 2020-09-08T15:59:59.592 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:58:51.810290+0000", 2020-09-08T15:59:59.593 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 67.557894069, 2020-09-08T15:59:59.593 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.144159906, 2020-09-08T15:59:59.593 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:59:59.593 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:59:59.593 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:59:59.594 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.16014", 2020-09-08T15:59:59.594 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:59.594 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 171002 2020-09-08T15:59:59.594 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.594 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:59:59.594 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.595 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:59:59.595 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.810290+0000", 2020-09-08T15:59:59.595 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:59:59.595 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.595 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.596 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:59:59.596 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.810290+0000", 2020-09-08T15:59:59.596 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999614 2020-09-08T15:59:59.596 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.596 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.597 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:59:59.597 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.810252+0000", 2020-09-08T15:59:59.597 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8684000000000007e-05 2020-09-08T15:59:59.597 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.597 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.598 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:59:59.598 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.810341+0000", 2020-09-08T15:59:59.598 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3329999999999999e-05 2020-09-08T15:59:59.598 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.598 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.598 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:59:59.599 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.810354+0000", 2020-09-08T15:59:59.599 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.9920000000000004e-06 2020-09-08T15:59:59.599 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.599 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.599 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:59:59.600 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.810360+0000", 2020-09-08T15:59:59.600 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.111860743 2020-09-08T15:59:59.600 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.600 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.600 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.601 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.922221+0000", 2020-09-08T15:59:59.601 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000166565 2020-09-08T15:59:59.601 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.601 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.601 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:59:59.601 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.922387+0000", 2020-09-08T15:59:59.602 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000189371 2020-09-08T15:59:59.602 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.602 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.602 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 9", 2020-09-08T15:59:59.602 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.922577+0000", 2020-09-08T15:59:59.603 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.013372162 2020-09-08T15:59:59.603 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.603 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.603 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:59:59.603 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.935949+0000", 2020-09-08T15:59:59.604 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.018428723000000001 2020-09-08T15:59:59.604 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.604 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.604 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:59:59.604 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.954377+0000", 2020-09-08T15:59:59.605 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.8219000000000001e-05 2020-09-08T15:59:59.605 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.605 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.605 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:59:59.605 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.954416+0000", 2020-09-08T15:59:59.605 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.4708e-05 2020-09-08T15:59:59.606 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.606 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.606 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:59:59.606 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.954450+0000", 2020-09-08T15:59:59.606 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.144159906 2020-09-08T15:59:59.607 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.607 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.607 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.607 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.607 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.608 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.16014.0:171006 6.1 6:86b1cc02:::benchmark_data_smithi166_43187_object51237:head [delete] snapc 0=[] ondisk+write+known_if_redirected e524)", 2020-09-08T15:59:59.608 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:58:51.818055+0000", 2020-09-08T15:59:59.608 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 67.550129815000005, 2020-09-08T15:59:59.608 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15114249099999999, 2020-09-08T15:59:59.608 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:59:59.609 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:59:59.609 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:59:59.609 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.16014", 2020-09-08T15:59:59.609 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:59.609 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 171006 2020-09-08T15:59:59.610 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.610 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:59:59.610 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.610 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:59:59.610 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.818055+0000", 2020-09-08T15:59:59.611 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:59:59.611 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.611 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.611 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:59:59.611 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.818055+0000", 2020-09-08T15:59:59.611 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999504 2020-09-08T15:59:59.612 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.612 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.612 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:59:59.612 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.818005+0000", 2020-09-08T15:59:59.612 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000110624 2020-09-08T15:59:59.613 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.613 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.613 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:59:59.613 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.818116+0000", 2020-09-08T15:59:59.613 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.2795e-05 2020-09-08T15:59:59.614 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.614 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.614 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:59:59.614 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.818148+0000", 2020-09-08T15:59:59.614 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.072e-06 2020-09-08T15:59:59.615 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.615 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.615 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:59:59.615 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.818156+0000", 2020-09-08T15:59:59.615 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.108977088 2020-09-08T15:59:59.615 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.616 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.616 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.616 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.927133+0000", 2020-09-08T15:59:59.616 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011758 2020-09-08T15:59:59.616 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.617 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.617 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:59:59.617 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.927251+0000", 2020-09-08T15:59:59.617 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012680999999999999 2020-09-08T15:59:59.617 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.618 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.618 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 9", 2020-09-08T15:59:59.618 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.927378+0000", 2020-09-08T15:59:59.618 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.012715529 2020-09-08T15:59:59.618 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.619 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.619 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:59:59.619 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.940093+0000", 2020-09-08T15:59:59.619 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.029021011999999999 2020-09-08T15:59:59.619 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.619 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.620 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:59:59.620 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.969114+0000", 2020-09-08T15:59:59.620 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.7752e-05 2020-09-08T15:59:59.620 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.620 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.621 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:59:59.621 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.969162+0000", 2020-09-08T15:59:59.621 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.5043999999999999e-05 2020-09-08T15:59:59.621 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.621 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.622 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:59:59.622 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.969197+0000", 2020-09-08T15:59:59.622 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15114249099999999 2020-09-08T15:59:59.622 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.622 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.623 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.623 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.623 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.623 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.16014.0:171009 6.1 6:8c90c54f:::benchmark_data_smithi166_43187_object51240:head [delete] snapc 0=[] ondisk+write+known_if_redirected e524)", 2020-09-08T15:59:59.623 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:58:51.824629+0000", 2020-09-08T15:59:59.623 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 67.543555239, 2020-09-08T15:59:59.624 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.144709906, 2020-09-08T15:59:59.624 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:59:59.624 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:59:59.624 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:59:59.624 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.16014", 2020-09-08T15:59:59.625 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:59.625 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 171009 2020-09-08T15:59:59.625 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.625 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:59:59.625 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.626 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:59:59.626 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.824629+0000", 2020-09-08T15:59:59.626 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:59:59.626 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.626 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.626 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:59:59.627 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.824629+0000", 2020-09-08T15:59:59.627 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.999948 2020-09-08T15:59:59.627 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.627 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.627 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:59:59.628 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.824577+0000", 2020-09-08T15:59:59.628 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015492199999999999 2020-09-08T15:59:59.628 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.628 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.628 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:59:59.629 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.824732+0000", 2020-09-08T15:59:59.629 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.4034999999999999e-05 2020-09-08T15:59:59.629 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.629 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.629 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:59:59.630 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.824756+0000", 2020-09-08T15:59:59.630 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0705e-05 2020-09-08T15:59:59.630 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.630 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.630 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:59:59.631 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.824767+0000", 2020-09-08T15:59:59.631 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.106820022 2020-09-08T15:59:59.631 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.631 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.631 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.631 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.931587+0000", 2020-09-08T15:59:59.632 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017361899999999999 2020-09-08T15:59:59.632 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.632 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.632 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:59:59.632 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.931761+0000", 2020-09-08T15:59:59.633 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011704700000000001 2020-09-08T15:59:59.633 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.633 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.633 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 9", 2020-09-08T15:59:59.633 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.931878+0000", 2020-09-08T15:59:59.634 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0082263530000000005 2020-09-08T15:59:59.634 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.634 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.634 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:59:59.634 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.940104+0000", 2020-09-08T15:59:59.634 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.029134151 2020-09-08T15:59:59.635 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.635 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.635 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:59:59.635 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.969238+0000", 2020-09-08T15:59:59.635 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.9954999999999998e-05 2020-09-08T15:59:59.636 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.636 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.636 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:59:59.636 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.969298+0000", 2020-09-08T15:59:59.636 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.1096000000000001e-05 2020-09-08T15:59:59.637 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.637 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.637 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:59:59.637 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:51.969339+0000", 2020-09-08T15:59:59.637 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.144709906 2020-09-08T15:59:59.638 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.638 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.638 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.638 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.638 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.639 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.16014.0:171989 6.1 6:8b650236:::benchmark_data_smithi166_43187_object52220:head [delete] snapc 0=[] ondisk+write+known_if_redirected e524)", 2020-09-08T15:59:59.639 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:58:53.064152+0000", 2020-09-08T15:59:59.639 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 66.304032636000002, 2020-09-08T15:59:59.639 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.147460546, 2020-09-08T15:59:59.639 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:59:59.639 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:59:59.640 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:59:59.640 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.16014", 2020-09-08T15:59:59.640 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:59.640 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 171989 2020-09-08T15:59:59.640 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.641 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:59:59.641 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.641 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:59:59.641 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:53.064152+0000", 2020-09-08T15:59:59.641 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:59:59.642 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.642 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.642 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:59:59.642 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:53.064152+0000", 2020-09-08T15:59:59.642 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999204 2020-09-08T15:59:59.643 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.643 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.643 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:59:59.643 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:53.064072+0000", 2020-09-08T15:59:59.643 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000199323 2020-09-08T15:59:59.643 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.644 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.644 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:59:59.644 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:53.064271+0000", 2020-09-08T15:59:59.644 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.6829e-05 2020-09-08T15:59:59.644 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.645 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.645 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:59:59.645 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:53.064298+0000", 2020-09-08T15:59:59.645 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2181000000000001e-05 2020-09-08T15:59:59.645 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.646 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.646 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:59:59.646 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:53.064310+0000", 2020-09-08T15:59:59.646 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.010458109 2020-09-08T15:59:59.646 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.647 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.647 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.647 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:53.074769+0000", 2020-09-08T15:59:59.647 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.9141000000000005e-05 2020-09-08T15:59:59.647 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.648 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.648 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:59:59.648 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:53.074868+0000", 2020-09-08T15:59:59.648 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000107007 2020-09-08T15:59:59.648 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.649 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.649 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 9", 2020-09-08T15:59:59.649 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:53.074975+0000", 2020-09-08T15:59:59.649 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0040825649999999998 2020-09-08T15:59:59.650 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.650 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.650 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:59:59.650 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:53.079057+0000", 2020-09-08T15:59:59.650 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.13243517599999999 2020-09-08T15:59:59.651 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.651 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.651 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:59:59.651 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:53.211492+0000", 2020-09-08T15:59:59.651 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1944000000000003e-05 2020-09-08T15:59:59.651 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.652 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.652 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:59:59.652 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:53.211544+0000", 2020-09-08T15:59:59.652 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.8076000000000003e-05 2020-09-08T15:59:59.652 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.653 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.653 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:59:59.653 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:58:53.211612+0000", 2020-09-08T15:59:59.653 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.147460546 2020-09-08T15:59:59.653 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.654 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.654 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.654 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.654 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.654 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.16014.0:179938 6.1 6:852536ac:::benchmark_data_smithi166_43187_object60169:head [delete] snapc 0=[] ondisk+write+known_if_redirected e527)", 2020-09-08T15:59:59.655 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:59:04.973067+0000", 2020-09-08T15:59:59.655 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 54.395117272, 2020-09-08T15:59:59.655 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15057331299999999, 2020-09-08T15:59:59.655 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:59:59.655 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:59:59.655 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:59:59.656 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.16014", 2020-09-08T15:59:59.656 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:59.656 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 179938 2020-09-08T15:59:59.656 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.656 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:59:59.657 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.657 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:59:59.657 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.973067+0000", 2020-09-08T15:59:59.657 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:59:59.657 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.658 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.658 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:59:59.658 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.973067+0000", 2020-09-08T15:59:59.658 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999766 2020-09-08T15:59:59.658 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.659 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.659 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:59:59.659 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.973044+0000", 2020-09-08T15:59:59.659 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.2677999999999999e-05 2020-09-08T15:59:59.659 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.660 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.660 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:59:59.660 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.973096+0000", 2020-09-08T15:59:59.660 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.5410000000000006e-06 2020-09-08T15:59:59.660 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.660 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.661 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:59:59.661 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.973106+0000", 2020-09-08T15:59:59.661 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.3469999999999998e-06 2020-09-08T15:59:59.661 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.661 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.662 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:59:59.662 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.973111+0000", 2020-09-08T15:59:59.662 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.011326604000000001 2020-09-08T15:59:59.663 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.663 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.663 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.664 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.984438+0000", 2020-09-08T15:59:59.664 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0040307399999999997 2020-09-08T15:59:59.664 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.664 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.664 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:59:59.665 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.988469+0000", 2020-09-08T15:59:59.665 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016869399999999999 2020-09-08T15:59:59.665 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.665 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.665 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 9", 2020-09-08T15:59:59.666 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.988637+0000", 2020-09-08T15:59:59.666 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0050932210000000002 2020-09-08T15:59:59.666 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.666 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.666 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:59:59.667 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.993731+0000", 2020-09-08T15:59:59.667 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.12981859300000001 2020-09-08T15:59:59.667 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.667 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.667 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:59:59.667 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:05.123549+0000", 2020-09-08T15:59:59.668 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.4889000000000003e-05 2020-09-08T15:59:59.668 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.668 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.668 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:59:59.668 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:05.123604+0000", 2020-09-08T15:59:59.669 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.6531e-05 2020-09-08T15:59:59.669 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.669 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.669 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:59:59.669 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:05.123641+0000", 2020-09-08T15:59:59.670 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15057331299999999 2020-09-08T15:59:59.670 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.670 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.670 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.670 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.671 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.671 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.16014.0:179941 6.1 6:899778ca:::benchmark_data_smithi166_43187_object60172:head [delete] snapc 0=[] ondisk+write+known_if_redirected e527)", 2020-09-08T15:59:59.671 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:59:04.978978+0000", 2020-09-08T15:59:59.671 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 54.389206110000003, 2020-09-08T15:59:59.671 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14515994400000001, 2020-09-08T15:59:59.672 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:59:59.672 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:59:59.672 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:59:59.672 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.16014", 2020-09-08T15:59:59.672 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:59.672 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 179941 2020-09-08T15:59:59.673 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.673 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:59:59.673 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.673 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:59:59.673 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.978978+0000", 2020-09-08T15:59:59.674 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:59:59.674 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.674 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.675 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:59:59.675 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.978978+0000", 2020-09-08T15:59:59.675 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999466 2020-09-08T15:59:59.675 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.676 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.676 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:59:59.676 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.978925+0000", 2020-09-08T15:59:59.676 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000119462 2020-09-08T15:59:59.676 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.676 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.677 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:59:59.677 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.979044+0000", 2020-09-08T15:59:59.677 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.9415e-05 2020-09-08T15:59:59.677 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.677 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.678 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:59:59.678 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.979064+0000", 2020-09-08T15:59:59.678 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4583000000000001e-05 2020-09-08T15:59:59.678 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.678 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.679 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:59:59.679 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.979078+0000", 2020-09-08T15:59:59.679 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.014730264999999999 2020-09-08T15:59:59.679 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.679 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.680 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.680 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.993809+0000", 2020-09-08T15:59:59.680 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000221482 2020-09-08T15:59:59.680 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.680 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.680 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:59:59.681 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.994030+0000", 2020-09-08T15:59:59.681 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00025096500000000002 2020-09-08T15:59:59.681 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.681 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.681 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 9", 2020-09-08T15:59:59.682 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.994281+0000", 2020-09-08T15:59:59.682 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.12920153400000001 2020-09-08T15:59:59.682 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.682 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.682 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:59:59.683 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:05.123483+0000", 2020-09-08T15:59:59.683 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00052880699999999995 2020-09-08T15:59:59.683 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.683 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.683 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:59:59.684 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:05.124012+0000", 2020-09-08T15:59:59.684 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.4927999999999998e-05 2020-09-08T15:59:59.684 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.684 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.684 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:59:59.685 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:05.124086+0000", 2020-09-08T15:59:59.685 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1913000000000003e-05 2020-09-08T15:59:59.685 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.685 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.685 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:59:59.685 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:05.124138+0000", 2020-09-08T15:59:59.686 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14515994400000001 2020-09-08T15:59:59.686 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.686 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.686 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.686 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.687 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.687 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.16014.0:179944 6.1 6:8c1aa665:::benchmark_data_smithi166_43187_object60175:head [delete] snapc 0=[] ondisk+write+known_if_redirected e527)", 2020-09-08T15:59:59.687 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:59:04.982058+0000", 2020-09-08T15:59:59.687 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 54.386126410999999, 2020-09-08T15:59:59.687 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.142229984, 2020-09-08T15:59:59.688 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T15:59:59.688 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T15:59:59.688 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T15:59:59.688 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.16014", 2020-09-08T15:59:59.688 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T15:59:59.689 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 179944 2020-09-08T15:59:59.689 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.689 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T15:59:59.689 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.689 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T15:59:59.690 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.982058+0000", 2020-09-08T15:59:59.690 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T15:59:59.690 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.690 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.690 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T15:59:59.690 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.982058+0000", 2020-09-08T15:59:59.691 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999661 2020-09-08T15:59:59.691 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.691 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.691 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T15:59:59.691 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.982025+0000", 2020-09-08T15:59:59.692 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.5236999999999998e-05 2020-09-08T15:59:59.692 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.692 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.692 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T15:59:59.692 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.982100+0000", 2020-09-08T15:59:59.693 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3884e-05 2020-09-08T15:59:59.693 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.693 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.693 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T15:59:59.693 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.982114+0000", 2020-09-08T15:59:59.694 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5289999999999997e-06 2020-09-08T15:59:59.694 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.694 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.694 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T15:59:59.694 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.982120+0000", 2020-09-08T15:59:59.695 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.017481527 2020-09-08T15:59:59.695 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.695 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.695 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T15:59:59.696 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.999602+0000", 2020-09-08T15:59:59.696 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018017400000000001 2020-09-08T15:59:59.696 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.696 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.696 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T15:59:59.697 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.999782+0000", 2020-09-08T15:59:59.697 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019513500000000001 2020-09-08T15:59:59.697 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.697 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.697 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 9", 2020-09-08T15:59:59.698 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:04.999977+0000", 2020-09-08T15:59:59.698 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.123845915 2020-09-08T15:59:59.698 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.698 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.698 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T15:59:59.698 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:05.123823+0000", 2020-09-08T15:59:59.699 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00037136299999999999 2020-09-08T15:59:59.699 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.699 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.699 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T15:59:59.699 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:05.124194+0000", 2020-09-08T15:59:59.700 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.4066000000000006e-05 2020-09-08T15:59:59.700 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.700 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.700 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T15:59:59.700 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:05.124258+0000", 2020-09-08T15:59:59.701 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.9782999999999999e-05 2020-09-08T15:59:59.701 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T15:59:59.701 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T15:59:59.701 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T15:59:59.701 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:59:05.124288+0000", 2020-09-08T15:59:59.701 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.142229984 2020-09-08T15:59:59.702 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.702 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.702 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.702 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T15:59:59.702 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T15:59:59.703 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T15:59:59.704 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 15:59:58 smithi104 bash[9203]: cluster 2020-09-08T15:59:57.952952+0000 mgr.x (mgr.24118) 979 : cluster [DBG] pgmap v1438: 17 pgs: 17 active+clean; 809 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.11k op/s 2020-09-08T15:59:59.704 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T15:59:59.808 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T15:59:59.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 15:59:59 smithi104 bash[15273]: debug 2020-09-08T15:59:59.588+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:00.212 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_ops_in_flight 2020-09-08T16:00:00.363 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:00.363 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:00:00.465 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:00.650 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:00:00 smithi104 bash[22305]: debug 2020-09-08T16:00:00.356+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:00.651 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:00.755 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:00:00.884 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:00:00.905 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:00:00 smithi104 bash[17588]: debug 2020-09-08T16:00:00.652+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:00.993 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:01.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:00 smithi055 bash[9296]: cluster 2020-09-08T15:59:59.953368+0000 mgr.x (mgr.24118) 980 : cluster [DBG] pgmap v1439: 17 pgs: 17 active+clean; 756 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 947 op/s 2020-09-08T16:00:01.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:00 smithi055 bash[9296]: cluster 2020-09-08T16:00:00.000098+0000 mon.a (mon.0) 1841 : cluster [WRN] overall HEALTH_WARN noscrub,nodeep-scrub flag(s) set 2020-09-08T16:00:01.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:00:00 smithi055 bash[14301]: debug 2020-09-08T16:00:00.884+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:01.184 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:00 smithi104 bash[9203]: cluster 2020-09-08T15:59:59.953368+0000 mgr.x (mgr.24118) 980 : cluster [DBG] pgmap v1439: 17 pgs: 17 active+clean; 756 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 947 op/s 2020-09-08T16:00:01.185 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:00 smithi104 bash[9203]: cluster 2020-09-08T16:00:00.000098+0000 mon.a (mon.0) 1841 : cluster [WRN] overall HEALTH_WARN noscrub,nodeep-scrub flag(s) set 2020-09-08T16:00:01.237 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:01.340 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:00:01.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:00 smithi166 bash[9446]: cluster 2020-09-08T15:59:59.953368+0000 mgr.x (mgr.24118) 980 : cluster [DBG] pgmap v1439: 17 pgs: 17 active+clean; 756 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 947 op/s 2020-09-08T16:00:01.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:00 smithi166 bash[9446]: cluster 2020-09-08T16:00:00.000098+0000 mon.a (mon.0) 1841 : cluster [WRN] overall HEALTH_WARN noscrub,nodeep-scrub flag(s) set 2020-09-08T16:00:01.468 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:00:01.572 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:01.580 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:00:01 smithi104 bash[15273]: debug 2020-09-08T16:00:01.240+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:01.658 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:00:01 smithi055 bash[19003]: debug 2020-09-08T16:00:01.472+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:02.823 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:02.926 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:03.114 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:03 smithi104 bash[9203]: cluster 2020-09-08T16:00:01.953831+0000 mgr.x (mgr.24118) 981 : cluster [DBG] pgmap v1440: 17 pgs: 17 active+clean; 667 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 850 op/s 2020-09-08T16:00:03.115 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:00:03 smithi104 bash[17588]: debug 2020-09-08T16:00:03.112+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:03.115 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:00:02 smithi104 bash[22305]: debug 2020-09-08T16:00:02.824+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:03.115 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:03.218 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:00:03.347 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:03 smithi055 bash[9296]: cluster 2020-09-08T16:00:01.953831+0000 mgr.x (mgr.24118) 981 : cluster [DBG] pgmap v1440: 17 pgs: 17 active+clean; 667 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 850 op/s 2020-09-08T16:00:03.348 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:00:03.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:03 smithi166 bash[9446]: cluster 2020-09-08T16:00:01.953831+0000 mgr.x (mgr.24118) 981 : cluster [DBG] pgmap v1440: 17 pgs: 17 active+clean; 667 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 850 op/s 2020-09-08T16:00:03.453 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:03.592 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:03.657 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:00:03 smithi055 bash[11956]: debug 2020-09-08T16:00:03.591+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:03.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:00:03 smithi055 bash[19003]: debug 2020-09-08T16:00:03.347+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:03.680 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:00:03.680 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T16:00:03.680 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:03.681 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.16014.0:233180 6.9 e543/478 6:9aa26a43:::benchmark_data_smithi166_43187_object113411:head v 543'14594, mlcod=543'14592)", 2020-09-08T16:00:03.681 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:00:03.682497+0000", 2020-09-08T16:00:03.681 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 0.00080938400000000001, 2020-09-08T16:00:03.681 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00083495800000000001, 2020-09-08T16:00:03.682 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:03.682 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "started", 2020-09-08T16:00:03.682 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:03.682 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:03.682 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:03.683 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:00:03.682497+0000", 2020-09-08T16:00:03.683 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:03.683 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:03.683 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:03.683 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:03.684 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:00:03.682497+0000", 2020-09-08T16:00:03.684 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.5373000000000001e-05 2020-09-08T16:00:03.684 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:03.684 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:03.684 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:03.685 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:00:03.682543+0000", 2020-09-08T16:00:03.685 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.413e-06 2020-09-08T16:00:03.685 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:03.685 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:03.686 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:03.686 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:00:03.682550+0000", 2020-09-08T16:00:03.686 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0491e-05 2020-09-08T16:00:03.686 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:03.686 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:03.687 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:03.687 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:00:03.682561+0000", 2020-09-08T16:00:03.687 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.4079999999999997e-06 2020-09-08T16:00:03.687 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:03.687 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:03.688 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:03.688 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:00:03.682566+0000", 2020-09-08T16:00:03.688 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.2258e-05 2020-09-08T16:00:03.688 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:03.688 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:03.689 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:03.690 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:00:03.682618+0000", 2020-09-08T16:00:03.690 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.1797000000000002e-05 2020-09-08T16:00:03.690 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:03.690 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:03.691 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:00:03.691 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:00:03.682640+0000", 2020-09-08T16:00:03.691 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014274000000000001 2020-09-08T16:00:03.691 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:03.691 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:03.692 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:03.692 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:03.692 INFO:teuthology.orchestra.run.smithi166.stdout: ], 2020-09-08T16:00:03.692 INFO:teuthology.orchestra.run.smithi166.stdout: "num_ops": 1 2020-09-08T16:00:03.692 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:00:03.696 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:03.958 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:03.967 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:00:03 smithi104 bash[15273]: debug 2020-09-08T16:00:03.952+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:04.068 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:04.478 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_blocked_ops 2020-09-08T16:00:04.620 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:00:04.694 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:04.799 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:05.011 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:00:04 smithi104 bash[17588]: debug 2020-09-08T16:00:04.696+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:05.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:05 smithi166 bash[9446]: cluster 2020-09-08T16:00:03.954315+0000 mgr.x (mgr.24118) 982 : cluster [DBG] pgmap v1441: 17 pgs: 17 active+clean; 521 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 917 op/s 2020-09-08T16:00:05.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:05 smithi166 bash[9446]: cluster 2020-09-08T16:00:04.576059+0000 osd.9 (osd.9) 81 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:05.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:05 smithi166 bash[9446]: cluster 2020-09-08T16:00:04.576129+0000 osd.9 (osd.9) 82 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:05.166 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:05.269 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:00:05.272 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:05 smithi104 bash[9203]: cluster 2020-09-08T16:00:03.954315+0000 mgr.x (mgr.24118) 982 : cluster [DBG] pgmap v1441: 17 pgs: 17 active+clean; 521 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 917 op/s 2020-09-08T16:00:05.272 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:05 smithi104 bash[9203]: cluster 2020-09-08T16:00:04.576059+0000 osd.9 (osd.9) 81 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:05.272 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:05 smithi104 bash[9203]: cluster 2020-09-08T16:00:04.576129+0000 osd.9 (osd.9) 82 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:05.288 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:05 smithi055 bash[9296]: cluster 2020-09-08T16:00:03.954315+0000 mgr.x (mgr.24118) 982 : cluster [DBG] pgmap v1441: 17 pgs: 17 active+clean; 521 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 917 op/s 2020-09-08T16:00:05.289 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:05 smithi055 bash[9296]: cluster 2020-09-08T16:00:04.576059+0000 osd.9 (osd.9) 81 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:05.290 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:05 smithi055 bash[9296]: cluster 2020-09-08T16:00:04.576129+0000 osd.9 (osd.9) 82 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:05.408 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:00:05.512 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:05.617 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:00:05 smithi166 bash[18960]: debug 2020-09-08T16:00:05.156+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:05.640 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:00:05 smithi055 bash[19003]: debug 2020-09-08T16:00:05.411+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:05.640 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:05.745 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:05.767 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:00:05.814 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:00:05.851 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:00:05.877 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:05.891 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:00:05.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:00:05 smithi055 bash[16646]: debug 2020-09-08T16:00:05.639+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:05.936 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:00:05.971 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:00:05.991 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:00:06.008 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:00:06.013 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:00:05 smithi104 bash[22305]: debug 2020-09-08T16:00:05.880+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:06.013 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:06 smithi104 bash[9203]: cluster 2020-09-08T16:00:05.014353+0000 mon.a (mon.0) 1842 : cluster [DBG] osdmap e544: 12 total, 12 up, 12 in 2020-09-08T16:00:06.013 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:06 smithi104 bash[9203]: audit 2020-09-08T16:00:05.720587+0000 mon.b (mon.1) 109 : audit [DBG] from='client.? v1:172.21.15.104:0/523214642' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:00:06.050 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:00:06.074 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:00:06.131 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:00:06.132 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:00:06.142 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:00:06.165 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:00:06.166 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:00:06.166 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:00:06.167 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:00:06.167 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:00:06.167 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:00:06.167 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:00:06.167 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:00:06.168 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:00:06.168 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:00:06.168 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:00:06.168 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:00:06.168 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:00:06.234 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:00:06.239 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:00:06 smithi166 bash[16593]: debug 2020-09-08T16:00:06.104+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:06.239 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:06 smithi166 bash[9446]: cluster 2020-09-08T16:00:05.014353+0000 mon.a (mon.0) 1842 : cluster [DBG] osdmap e544: 12 total, 12 up, 12 in 2020-09-08T16:00:06.239 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:06 smithi166 bash[9446]: audit 2020-09-08T16:00:05.720587+0000 mon.b (mon.1) 109 : audit [DBG] from='client.? v1:172.21.15.104:0/523214642' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:00:06.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:06 smithi055 bash[9296]: cluster 2020-09-08T16:00:05.014353+0000 mon.a (mon.0) 1842 : cluster [DBG] osdmap e544: 12 total, 12 up, 12 in 2020-09-08T16:00:06.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:06 smithi055 bash[9296]: audit 2020-09-08T16:00:05.720587+0000 mon.b (mon.1) 109 : audit [DBG] from='client.? v1:172.21.15.104:0/523214642' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:00:06.433 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:00:06.537 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:06.546 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:00:06 smithi166 bash[16593]: debug 2020-09-08T16:00:06.432+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:06.978 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:07.035 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T16:00:07.081 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:07 smithi166 bash[9446]: cluster 2020-09-08T16:00:05.954755+0000 mgr.x (mgr.24118) 983 : cluster [DBG] pgmap v1443: 17 pgs: 17 active+clean; 431 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 925 op/s 2020-09-08T16:00:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:07 smithi166 bash[9446]: audit 2020-09-08T16:00:06.046699+0000 mon.a (mon.0) 1843 : audit [INF] from='client.? v1:172.21.15.104:0/3628669361' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:00:07.368 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:00:06 smithi166 bash[18960]: debug 2020-09-08T16:00:06.980+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:07.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:07 smithi055 bash[9296]: cluster 2020-09-08T16:00:05.954755+0000 mgr.x (mgr.24118) 983 : cluster [DBG] pgmap v1443: 17 pgs: 17 active+clean; 431 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 925 op/s 2020-09-08T16:00:07.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:07 smithi055 bash[9296]: audit 2020-09-08T16:00:06.046699+0000 mon.a (mon.0) 1843 : audit [INF] from='client.? v1:172.21.15.104:0/3628669361' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:00:07.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:07 smithi104 bash[9203]: cluster 2020-09-08T16:00:05.954755+0000 mgr.x (mgr.24118) 983 : cluster [DBG] pgmap v1443: 17 pgs: 17 active+clean; 431 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 925 op/s 2020-09-08T16:00:07.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:07 smithi104 bash[9203]: audit 2020-09-08T16:00:06.046699+0000 mon.a (mon.0) 1843 : audit [INF] from='client.? v1:172.21.15.104:0/3628669361' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:00:07.467 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:07.569 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:07.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:00:07 smithi166 bash[11928]: debug 2020-09-08T16:00:07.460+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:08.171 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:08.275 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:08.283 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:08 smithi166 bash[9446]: cluster 2020-09-08T16:00:07.024818+0000 mon.a (mon.0) 1844 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:00:08.283 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:08 smithi166 bash[9446]: audit 2020-09-08T16:00:07.032975+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? v1:172.21.15.104:0/3628669361' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:00:08.284 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:08 smithi166 bash[9446]: cluster 2020-09-08T16:00:07.033065+0000 mon.a (mon.0) 1846 : cluster [DBG] osdmap e545: 12 total, 12 up, 12 in 2020-09-08T16:00:08.284 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:08 smithi166 bash[9446]: cluster 2020-09-08T16:00:07.335599+0000 osd.2 (osd.2) 80 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:08.284 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:08 smithi166 bash[9446]: cluster 2020-09-08T16:00:07.335664+0000 osd.2 (osd.2) 81 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:08.284 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:08 smithi166 bash[9446]: cluster 2020-09-08T16:00:07.515714+0000 osd.0 (osd.0) 83 : cluster [DBG] 6.0 scrub starts 2020-09-08T16:00:08.285 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:08 smithi166 bash[9446]: cluster 2020-09-08T16:00:07.582252+0000 osd.0 (osd.0) 84 : cluster [DBG] 6.0 scrub ok 2020-09-08T16:00:08.312 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:00:08 smithi104 bash[15273]: debug 2020-09-08T16:00:08.172+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:08.313 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:08 smithi104 bash[9203]: cluster 2020-09-08T16:00:07.024818+0000 mon.a (mon.0) 1844 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:00:08.313 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:08 smithi104 bash[9203]: audit 2020-09-08T16:00:07.032975+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? v1:172.21.15.104:0/3628669361' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:00:08.313 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:08 smithi104 bash[9203]: cluster 2020-09-08T16:00:07.033065+0000 mon.a (mon.0) 1846 : cluster [DBG] osdmap e545: 12 total, 12 up, 12 in 2020-09-08T16:00:08.313 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:08 smithi104 bash[9203]: cluster 2020-09-08T16:00:07.335599+0000 osd.2 (osd.2) 80 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:08.313 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:08 smithi104 bash[9203]: cluster 2020-09-08T16:00:07.335664+0000 osd.2 (osd.2) 81 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:08.314 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:08 smithi104 bash[9203]: cluster 2020-09-08T16:00:07.515714+0000 osd.0 (osd.0) 83 : cluster [DBG] 6.0 scrub starts 2020-09-08T16:00:08.314 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:08 smithi104 bash[9203]: cluster 2020-09-08T16:00:07.582252+0000 osd.0 (osd.0) 84 : cluster [DBG] 6.0 scrub ok 2020-09-08T16:00:08.391 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:08.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:08 smithi055 bash[9296]: cluster 2020-09-08T16:00:07.024818+0000 mon.a (mon.0) 1844 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:00:08.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:08 smithi055 bash[9296]: audit 2020-09-08T16:00:07.032975+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? v1:172.21.15.104:0/3628669361' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:00:08.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:08 smithi055 bash[9296]: cluster 2020-09-08T16:00:07.033065+0000 mon.a (mon.0) 1846 : cluster [DBG] osdmap e545: 12 total, 12 up, 12 in 2020-09-08T16:00:08.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:08 smithi055 bash[9296]: cluster 2020-09-08T16:00:07.335599+0000 osd.2 (osd.2) 80 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:08.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:08 smithi055 bash[9296]: cluster 2020-09-08T16:00:07.335664+0000 osd.2 (osd.2) 81 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:08.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:08 smithi055 bash[9296]: cluster 2020-09-08T16:00:07.515714+0000 osd.0 (osd.0) 83 : cluster [DBG] 6.0 scrub starts 2020-09-08T16:00:08.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:08 smithi055 bash[9296]: cluster 2020-09-08T16:00:07.582252+0000 osd.0 (osd.0) 84 : cluster [DBG] 6.0 scrub ok 2020-09-08T16:00:08.497 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:00:08.568 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T16:00:08.568 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:00:08.617 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:00:08 smithi166 bash[18960]: debug 2020-09-08T16:00:08.392+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:08.654 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:00:08.763 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:00:08.771 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:00:08 smithi055 bash[19003]: debug 2020-09-08T16:00:08.655+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:08.892 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:00:09.003 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:09.035 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:09 smithi055 bash[9296]: cluster 2020-09-08T16:00:07.819463+0000 osd.5 (osd.5) 85 : cluster [DBG] 6.d scrub starts 2020-09-08T16:00:09.036 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:09 smithi055 bash[9296]: cluster 2020-09-08T16:00:07.955241+0000 mgr.x (mgr.24118) 984 : cluster [DBG] pgmap v1445: 17 pgs: 17 active+clean; 237 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T16:00:09.036 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:00:08 smithi055 bash[19003]: debug 2020-09-08T16:00:08.891+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:09.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:09 smithi166 bash[9446]: cluster 2020-09-08T16:00:07.819463+0000 osd.5 (osd.5) 85 : cluster [DBG] 6.d scrub starts 2020-09-08T16:00:09.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:09 smithi166 bash[9446]: cluster 2020-09-08T16:00:07.955241+0000 mgr.x (mgr.24118) 984 : cluster [DBG] pgmap v1445: 17 pgs: 17 active+clean; 237 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T16:00:09.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:09 smithi166 bash[9446]: cluster 2020-09-08T16:00:07.988361+0000 osd.5 (osd.5) 86 : cluster [DBG] 6.d scrub ok 2020-09-08T16:00:09.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:09 smithi166 bash[9446]: cluster 2020-09-08T16:00:08.041058+0000 mon.a (mon.0) 1847 : cluster [DBG] osdmap e546: 12 total, 12 up, 12 in 2020-09-08T16:00:09.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:09 smithi166 bash[9446]: cluster 2020-09-08T16:00:08.794563+0000 osd.5 (osd.5) 87 : cluster [DBG] 6.7 scrub starts 2020-09-08T16:00:09.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:09 smithi166 bash[9446]: cluster 2020-09-08T16:00:08.920743+0000 osd.3 (osd.3) 89 : cluster [DBG] 6.b scrub starts 2020-09-08T16:00:09.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:09 smithi055 bash[9296]: cluster 2020-09-08T16:00:07.988361+0000 osd.5 (osd.5) 86 : cluster [DBG] 6.d scrub ok 2020-09-08T16:00:09.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:09 smithi055 bash[9296]: cluster 2020-09-08T16:00:08.041058+0000 mon.a (mon.0) 1847 : cluster [DBG] osdmap e546: 12 total, 12 up, 12 in 2020-09-08T16:00:09.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:09 smithi055 bash[9296]: cluster 2020-09-08T16:00:08.794563+0000 osd.5 (osd.5) 87 : cluster [DBG] 6.7 scrub starts 2020-09-08T16:00:09.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:09 smithi055 bash[9296]: cluster 2020-09-08T16:00:08.920743+0000 osd.3 (osd.3) 89 : cluster [DBG] 6.b scrub starts 2020-09-08T16:00:09.420 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:09.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:09 smithi104 bash[9203]: cluster 2020-09-08T16:00:07.819463+0000 osd.5 (osd.5) 85 : cluster [DBG] 6.d scrub starts 2020-09-08T16:00:09.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:09 smithi104 bash[9203]: cluster 2020-09-08T16:00:07.955241+0000 mgr.x (mgr.24118) 984 : cluster [DBG] pgmap v1445: 17 pgs: 17 active+clean; 237 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T16:00:09.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:09 smithi104 bash[9203]: cluster 2020-09-08T16:00:07.988361+0000 osd.5 (osd.5) 86 : cluster [DBG] 6.d scrub ok 2020-09-08T16:00:09.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:09 smithi104 bash[9203]: cluster 2020-09-08T16:00:08.041058+0000 mon.a (mon.0) 1847 : cluster [DBG] osdmap e546: 12 total, 12 up, 12 in 2020-09-08T16:00:09.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:09 smithi104 bash[9203]: cluster 2020-09-08T16:00:08.794563+0000 osd.5 (osd.5) 87 : cluster [DBG] 6.7 scrub starts 2020-09-08T16:00:09.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:09 smithi104 bash[9203]: cluster 2020-09-08T16:00:08.920743+0000 osd.3 (osd.3) 89 : cluster [DBG] 6.b scrub starts 2020-09-08T16:00:09.523 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:09.548 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_historic_ops 2020-09-08T16:00:09.679 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:09.686 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:00:09.780 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:09.790 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:00:09 smithi104 bash[15273]: debug 2020-09-08T16:00:09.676+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:09.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:00:09 smithi166 bash[18960]: debug 2020-09-08T16:00:09.420+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:09.905 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:10.005 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:00:10.009 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:10.157 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:00:10.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:10 smithi104 bash[9203]: cluster 2020-09-08T16:00:08.880907+0000 osd.5 (osd.5) 88 : cluster [DBG] 6.7 scrub ok 2020-09-08T16:00:10.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:10 smithi104 bash[9203]: cluster 2020-09-08T16:00:09.028076+0000 osd.3 (osd.3) 90 : cluster [DBG] 6.b scrub ok 2020-09-08T16:00:10.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:10 smithi104 bash[9203]: cluster 2020-09-08T16:00:09.092311+0000 osd.2 (osd.2) 82 : cluster [DBG] 6.f scrub starts 2020-09-08T16:00:10.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:10 smithi104 bash[9203]: cluster 2020-09-08T16:00:09.161500+0000 osd.2 (osd.2) 83 : cluster [DBG] 6.f scrub ok 2020-09-08T16:00:10.177 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:00:09 smithi104 bash[15273]: debug 2020-09-08T16:00:09.904+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:10.251 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:10.321 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T16:00:10.357 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:10.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:00:10 smithi166 bash[11928]: debug 2020-09-08T16:00:10.252+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:10.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:10 smithi166 bash[9446]: cluster 2020-09-08T16:00:08.880907+0000 osd.5 (osd.5) 88 : cluster [DBG] 6.7 scrub ok 2020-09-08T16:00:10.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:10 smithi166 bash[9446]: cluster 2020-09-08T16:00:09.028076+0000 osd.3 (osd.3) 90 : cluster [DBG] 6.b scrub ok 2020-09-08T16:00:10.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:10 smithi166 bash[9446]: cluster 2020-09-08T16:00:09.092311+0000 osd.2 (osd.2) 82 : cluster [DBG] 6.f scrub starts 2020-09-08T16:00:10.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:10 smithi166 bash[9446]: cluster 2020-09-08T16:00:09.161500+0000 osd.2 (osd.2) 83 : cluster [DBG] 6.f scrub ok 2020-09-08T16:00:10.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:10 smithi055 bash[9296]: cluster 2020-09-08T16:00:08.880907+0000 osd.5 (osd.5) 88 : cluster [DBG] 6.7 scrub ok 2020-09-08T16:00:10.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:10 smithi055 bash[9296]: cluster 2020-09-08T16:00:09.028076+0000 osd.3 (osd.3) 90 : cluster [DBG] 6.b scrub ok 2020-09-08T16:00:10.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:10 smithi055 bash[9296]: cluster 2020-09-08T16:00:09.092311+0000 osd.2 (osd.2) 82 : cluster [DBG] 6.f scrub starts 2020-09-08T16:00:10.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:10 smithi055 bash[9296]: cluster 2020-09-08T16:00:09.161500+0000 osd.2 (osd.2) 83 : cluster [DBG] 6.f scrub ok 2020-09-08T16:00:10.483 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:00:10.639 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Removed 119766 objects 2020-09-08T16:00:10.640 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Clean up completed and total clean up time :131.128 2020-09-08T16:00:10.817 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:10.922 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:11.080 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:11 smithi166 bash[9446]: cluster 2020-09-08T16:00:09.447948+0000 osd.8 (osd.8) 73 : cluster [DBG] 6.1 scrub starts 2020-09-08T16:00:11.080 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:11 smithi166 bash[9446]: cluster 2020-09-08T16:00:09.512900+0000 osd.8 (osd.8) 74 : cluster [DBG] 6.1 scrub ok 2020-09-08T16:00:11.080 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:11 smithi166 bash[9446]: cluster 2020-09-08T16:00:09.955719+0000 mgr.x (mgr.24118) 985 : cluster [DBG] pgmap v1447: 17 pgs: 17 active+clean; 185 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 896 op/s 2020-09-08T16:00:11.081 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:00:10 smithi166 bash[18960]: debug 2020-09-08T16:00:10.816+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:11.422 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:11 smithi055 bash[9296]: cluster 2020-09-08T16:00:09.447948+0000 osd.8 (osd.8) 73 : cluster [DBG] 6.1 scrub starts 2020-09-08T16:00:11.423 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:11 smithi055 bash[9296]: cluster 2020-09-08T16:00:09.512900+0000 osd.8 (osd.8) 74 : cluster [DBG] 6.1 scrub ok 2020-09-08T16:00:11.423 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:11 smithi055 bash[9296]: cluster 2020-09-08T16:00:09.955719+0000 mgr.x (mgr.24118) 985 : cluster [DBG] pgmap v1447: 17 pgs: 17 active+clean; 185 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 896 op/s 2020-09-08T16:00:11.434 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:11 smithi104 bash[9203]: cluster 2020-09-08T16:00:09.447948+0000 osd.8 (osd.8) 73 : cluster [DBG] 6.1 scrub starts 2020-09-08T16:00:11.434 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:11 smithi104 bash[9203]: cluster 2020-09-08T16:00:09.512900+0000 osd.8 (osd.8) 74 : cluster [DBG] 6.1 scrub ok 2020-09-08T16:00:11.435 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:11 smithi104 bash[9203]: cluster 2020-09-08T16:00:09.955719+0000 mgr.x (mgr.24118) 985 : cluster [DBG] pgmap v1447: 17 pgs: 17 active+clean; 185 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 896 op/s 2020-09-08T16:00:11.522 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:11.624 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:11.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:00:11 smithi104 bash[15273]: debug 2020-09-08T16:00:11.520+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:12.019 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:12.125 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:00:12.352 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:12 smithi104 bash[9203]: cluster 2020-09-08T16:00:11.928127+0000 osd.3 (osd.3) 91 : cluster [DBG] 6.2 scrub starts 2020-09-08T16:00:12.353 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:12 smithi104 bash[9203]: cluster 2020-09-08T16:00:11.949870+0000 osd.3 (osd.3) 92 : cluster [DBG] 6.2 scrub ok 2020-09-08T16:00:12.353 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:00:12.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:12 smithi166 bash[9446]: cluster 2020-09-08T16:00:11.928127+0000 osd.3 (osd.3) 91 : cluster [DBG] 6.2 scrub starts 2020-09-08T16:00:12.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:12 smithi166 bash[9446]: cluster 2020-09-08T16:00:11.949870+0000 osd.3 (osd.3) 92 : cluster [DBG] 6.2 scrub ok 2020-09-08T16:00:12.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:00:12 smithi166 bash[11928]: debug 2020-09-08T16:00:12.020+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:12.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:12 smithi055 bash[9296]: cluster 2020-09-08T16:00:11.928127+0000 osd.3 (osd.3) 91 : cluster [DBG] 6.2 scrub starts 2020-09-08T16:00:12.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:12 smithi055 bash[9296]: cluster 2020-09-08T16:00:11.949870+0000 osd.3 (osd.3) 92 : cluster [DBG] 6.2 scrub ok 2020-09-08T16:00:12.458 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:12.643 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:00:12 smithi104 bash[19922]: debug 2020-09-08T16:00:12.356+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:12.646 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:12.753 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:12.932 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:00:12 smithi104 bash[22305]: debug 2020-09-08T16:00:12.648+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:12.970 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:13.043 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:00:12 smithi166 bash[14254]: debug 2020-09-08T16:00:12.968+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:13.077 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:13.216 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:13.312 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:00:13 smithi166 bash[14254]: debug 2020-09-08T16:00:13.216+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:13.313 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:13 smithi166 bash[9446]: cluster 2020-09-08T16:00:11.375577+0000 osd.7 (osd.7) 113 : cluster [DBG] 6.8 scrub starts 2020-09-08T16:00:13.313 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:13 smithi166 bash[9446]: cluster 2020-09-08T16:00:11.390351+0000 osd.7 (osd.7) 114 : cluster [DBG] 6.8 scrub ok 2020-09-08T16:00:13.313 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:13 smithi166 bash[9446]: cluster 2020-09-08T16:00:11.555984+0000 osd.1 (osd.1) 110 : cluster [DBG] 6.e scrub starts 2020-09-08T16:00:13.314 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:13 smithi166 bash[9446]: cluster 2020-09-08T16:00:11.568934+0000 osd.1 (osd.1) 111 : cluster [DBG] 6.e scrub ok 2020-09-08T16:00:13.314 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:13 smithi166 bash[9446]: cluster 2020-09-08T16:00:11.956410+0000 mgr.x (mgr.24118) 986 : cluster [DBG] pgmap v1448: 17 pgs: 17 active+clean; 106 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 836 op/s 2020-09-08T16:00:13.322 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:13.335 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:13 smithi104 bash[9203]: cluster 2020-09-08T16:00:11.375577+0000 osd.7 (osd.7) 113 : cluster [DBG] 6.8 scrub starts 2020-09-08T16:00:13.335 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:13 smithi104 bash[9203]: cluster 2020-09-08T16:00:11.390351+0000 osd.7 (osd.7) 114 : cluster [DBG] 6.8 scrub ok 2020-09-08T16:00:13.336 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:13 smithi104 bash[9203]: cluster 2020-09-08T16:00:11.555984+0000 osd.1 (osd.1) 110 : cluster [DBG] 6.e scrub starts 2020-09-08T16:00:13.336 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:13 smithi104 bash[9203]: cluster 2020-09-08T16:00:11.568934+0000 osd.1 (osd.1) 111 : cluster [DBG] 6.e scrub ok 2020-09-08T16:00:13.336 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:13 smithi104 bash[9203]: cluster 2020-09-08T16:00:11.956410+0000 mgr.x (mgr.24118) 986 : cluster [DBG] pgmap v1448: 17 pgs: 17 active+clean; 106 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 836 op/s 2020-09-08T16:00:13.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:13 smithi055 bash[9296]: cluster 2020-09-08T16:00:11.375577+0000 osd.7 (osd.7) 113 : cluster [DBG] 6.8 scrub starts 2020-09-08T16:00:13.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:13 smithi055 bash[9296]: cluster 2020-09-08T16:00:11.390351+0000 osd.7 (osd.7) 114 : cluster [DBG] 6.8 scrub ok 2020-09-08T16:00:13.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:13 smithi055 bash[9296]: cluster 2020-09-08T16:00:11.555984+0000 osd.1 (osd.1) 110 : cluster [DBG] 6.e scrub starts 2020-09-08T16:00:13.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:13 smithi055 bash[9296]: cluster 2020-09-08T16:00:11.568934+0000 osd.1 (osd.1) 111 : cluster [DBG] 6.e scrub ok 2020-09-08T16:00:13.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:13 smithi055 bash[9296]: cluster 2020-09-08T16:00:11.956410+0000 mgr.x (mgr.24118) 986 : cluster [DBG] pgmap v1448: 17 pgs: 17 active+clean; 106 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 836 op/s 2020-09-08T16:00:13.470 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T16:00:13.470 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:00:13.918 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:14.026 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:00:14.087 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_4 2020-09-08T16:00:14.090 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool rm unique_pool_4 unique_pool_4 --yes-i-really-really-mean-it 2020-09-08T16:00:14.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:00:13 smithi104 bash[17588]: debug 2020-09-08T16:00:13.920+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:14.218 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:00:14.255 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:00:14.313 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2020-09-08T16:00:14.358 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:14.458 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:00:14.677 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:00:14 smithi104 bash[19922]: debug 2020-09-08T16:00:14.256+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:14.828 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:14.937 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:15.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:00:14 smithi104 bash[22305]: debug 2020-09-08T16:00:14.832+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:15.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:15 smithi104 bash[9203]: cluster 2020-09-08T16:00:13.390427+0000 osd.7 (osd.7) 115 : cluster [DBG] 6.5 scrub starts 2020-09-08T16:00:15.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:15 smithi104 bash[9203]: cluster 2020-09-08T16:00:13.407038+0000 osd.7 (osd.7) 116 : cluster [DBG] 6.5 scrub ok 2020-09-08T16:00:15.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:15 smithi104 bash[9203]: cluster 2020-09-08T16:00:13.957103+0000 mgr.x (mgr.24118) 987 : cluster [DBG] pgmap v1449: 17 pgs: 17 active+clean; 10 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 841 op/s 2020-09-08T16:00:15.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:15 smithi104 bash[9203]: cluster 2020-09-08T16:00:13.987270+0000 osd.7 (osd.7) 117 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:15.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:15 smithi104 bash[9203]: cluster 2020-09-08T16:00:13.987390+0000 osd.7 (osd.7) 118 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:15.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:15 smithi104 bash[9203]: cluster 2020-09-08T16:00:14.058601+0000 mon.a (mon.0) 1848 : cluster [DBG] osdmap e547: 12 total, 12 up, 12 in 2020-09-08T16:00:15.248 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:15 smithi166 bash[9446]: cluster 2020-09-08T16:00:13.390427+0000 osd.7 (osd.7) 115 : cluster [DBG] 6.5 scrub starts 2020-09-08T16:00:15.248 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:15 smithi166 bash[9446]: cluster 2020-09-08T16:00:13.407038+0000 osd.7 (osd.7) 116 : cluster [DBG] 6.5 scrub ok 2020-09-08T16:00:15.248 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:15 smithi166 bash[9446]: cluster 2020-09-08T16:00:13.957103+0000 mgr.x (mgr.24118) 987 : cluster [DBG] pgmap v1449: 17 pgs: 17 active+clean; 10 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 841 op/s 2020-09-08T16:00:15.249 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:15 smithi166 bash[9446]: cluster 2020-09-08T16:00:13.987270+0000 osd.7 (osd.7) 117 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:15.249 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:15 smithi166 bash[9446]: cluster 2020-09-08T16:00:13.987390+0000 osd.7 (osd.7) 118 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:15.249 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:15 smithi166 bash[9446]: cluster 2020-09-08T16:00:14.058601+0000 mon.a (mon.0) 1848 : cluster [DBG] osdmap e547: 12 total, 12 up, 12 in 2020-09-08T16:00:15.250 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:15.356 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:15.390 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:15 smithi055 bash[9296]: cluster 2020-09-08T16:00:13.390427+0000 osd.7 (osd.7) 115 : cluster [DBG] 6.5 scrub starts 2020-09-08T16:00:15.391 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:15 smithi055 bash[9296]: cluster 2020-09-08T16:00:13.407038+0000 osd.7 (osd.7) 116 : cluster [DBG] 6.5 scrub ok 2020-09-08T16:00:15.391 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:15 smithi055 bash[9296]: cluster 2020-09-08T16:00:13.957103+0000 mgr.x (mgr.24118) 987 : cluster [DBG] pgmap v1449: 17 pgs: 17 active+clean; 10 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 841 op/s 2020-09-08T16:00:15.391 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:15 smithi055 bash[9296]: cluster 2020-09-08T16:00:13.987270+0000 osd.7 (osd.7) 117 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:15.391 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:15 smithi055 bash[9296]: cluster 2020-09-08T16:00:13.987390+0000 osd.7 (osd.7) 118 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:15.391 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:15 smithi055 bash[9296]: cluster 2020-09-08T16:00:14.058601+0000 mon.a (mon.0) 1848 : cluster [DBG] osdmap e547: 12 total, 12 up, 12 in 2020-09-08T16:00:15.495 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:15.580 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:00:15 smithi166 bash[11928]: debug 2020-09-08T16:00:15.248+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:15.601 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:15.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:00:15 smithi055 bash[11956]: debug 2020-09-08T16:00:15.495+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:15.770 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:15.868 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:00:15 smithi166 bash[18960]: debug 2020-09-08T16:00:15.768+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:15.873 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:16.118 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:16.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:00:16 smithi104 bash[22305]: debug 2020-09-08T16:00:16.120+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:16.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:16 smithi104 bash[9203]: cluster 2020-09-08T16:00:14.404211+0000 osd.7 (osd.7) 119 : cluster [DBG] 6.3 scrub starts 2020-09-08T16:00:16.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:16 smithi104 bash[9203]: cluster 2020-09-08T16:00:14.417468+0000 osd.7 (osd.7) 120 : cluster [DBG] 6.3 scrub ok 2020-09-08T16:00:16.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:16 smithi104 bash[9203]: cluster 2020-09-08T16:00:15.093236+0000 osd.0 (osd.0) 85 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:16.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:16 smithi104 bash[9203]: cluster 2020-09-08T16:00:15.093370+0000 osd.0 (osd.0) 86 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:16.234 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:16.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:16 smithi166 bash[9446]: cluster 2020-09-08T16:00:14.404211+0000 osd.7 (osd.7) 119 : cluster [DBG] 6.3 scrub starts 2020-09-08T16:00:16.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:16 smithi166 bash[9446]: cluster 2020-09-08T16:00:14.417468+0000 osd.7 (osd.7) 120 : cluster [DBG] 6.3 scrub ok 2020-09-08T16:00:16.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:16 smithi166 bash[9446]: cluster 2020-09-08T16:00:15.093236+0000 osd.0 (osd.0) 85 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:16.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:16 smithi166 bash[9446]: cluster 2020-09-08T16:00:15.093370+0000 osd.0 (osd.0) 86 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:16.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:16 smithi055 bash[9296]: cluster 2020-09-08T16:00:14.404211+0000 osd.7 (osd.7) 119 : cluster [DBG] 6.3 scrub starts 2020-09-08T16:00:16.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:16 smithi055 bash[9296]: cluster 2020-09-08T16:00:14.417468+0000 osd.7 (osd.7) 120 : cluster [DBG] 6.3 scrub ok 2020-09-08T16:00:16.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:16 smithi055 bash[9296]: cluster 2020-09-08T16:00:15.093236+0000 osd.0 (osd.0) 85 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:16.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:16 smithi055 bash[9296]: cluster 2020-09-08T16:00:15.093370+0000 osd.0 (osd.0) 86 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:16.442 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:16.494 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:00:16 smithi104 bash[22305]: debug 2020-09-08T16:00:16.444+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:16.546 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:16.721 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:16.826 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:16.828 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:00:16 smithi104 bash[17588]: debug 2020-09-08T16:00:16.724+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:17.000 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:17.090 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:17 smithi104 bash[9203]: cluster 2020-09-08T16:00:15.957769+0000 mgr.x (mgr.24118) 988 : cluster [DBG] pgmap v1451: 17 pgs: 17 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 473 op/s 2020-09-08T16:00:17.091 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:17 smithi104 bash[9203]: cluster 2020-09-08T16:00:16.072197+0000 mon.a (mon.0) 1849 : cluster [DBG] osdmap e548: 12 total, 12 up, 12 in 2020-09-08T16:00:17.106 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:00:17.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:17 smithi166 bash[9446]: cluster 2020-09-08T16:00:15.957769+0000 mgr.x (mgr.24118) 988 : cluster [DBG] pgmap v1451: 17 pgs: 17 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 473 op/s 2020-09-08T16:00:17.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:17 smithi166 bash[9446]: cluster 2020-09-08T16:00:16.072197+0000 mon.a (mon.0) 1849 : cluster [DBG] osdmap e548: 12 total, 12 up, 12 in 2020-09-08T16:00:17.117 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:00:17 smithi166 bash[18960]: debug 2020-09-08T16:00:17.000+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:17.228 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:00:17.335 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:17.349 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:00:17 smithi055 bash[19003]: debug 2020-09-08T16:00:17.231+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:17.350 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:17 smithi055 bash[9296]: cluster 2020-09-08T16:00:15.957769+0000 mgr.x (mgr.24118) 988 : cluster [DBG] pgmap v1451: 17 pgs: 17 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 473 op/s 2020-09-08T16:00:17.350 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:17 smithi055 bash[9296]: cluster 2020-09-08T16:00:16.072197+0000 mon.a (mon.0) 1849 : cluster [DBG] osdmap e548: 12 total, 12 up, 12 in 2020-09-08T16:00:17.465 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:17.574 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:17.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:00:17 smithi055 bash[11956]: debug 2020-09-08T16:00:17.467+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:17.730 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:17.817 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T16:00:17.817 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:00:17.834 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:17.867 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:00:17 smithi166 bash[14254]: debug 2020-09-08T16:00:17.732+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:17.870 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:00:17.898 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:00:17.927 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:00:17.950 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:00:17.977 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:00:17.978 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:18.003 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:00:18.031 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:00:18.055 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:00:18.083 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:00:18.084 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:00:18.097 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:00:17 smithi055 bash[11956]: debug 2020-09-08T16:00:17.979+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:18.098 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:18 smithi055 bash[9296]: cluster 2020-09-08T16:00:16.370362+0000 osd.7 (osd.7) 121 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:00:18.098 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:18 smithi055 bash[9296]: cluster 2020-09-08T16:00:16.462012+0000 osd.7 (osd.7) 122 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:00:18.098 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:18 smithi055 bash[9296]: audit 2020-09-08T16:00:17.845369+0000 mon.a (mon.0) 1850 : audit [DBG] from='client.? v1:172.21.15.104:0/86812965' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:00:18.111 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:00:18.137 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:00:18.164 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:00:18.166 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:00:18.166 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:00:18.167 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:00:18.167 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:00:18.168 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:00:18.168 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:00:18.168 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:00:18.168 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:00:18.169 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:00:18.169 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:00:18.169 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:00:18.169 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:00:18.204 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:00:18.312 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:18.332 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:18 smithi104 bash[9203]: cluster 2020-09-08T16:00:16.370362+0000 osd.7 (osd.7) 121 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:00:18.332 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:18 smithi104 bash[9203]: cluster 2020-09-08T16:00:16.462012+0000 osd.7 (osd.7) 122 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:00:18.332 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:18 smithi104 bash[9203]: audit 2020-09-08T16:00:17.845369+0000 mon.a (mon.0) 1850 : audit [DBG] from='client.? v1:172.21.15.104:0/86812965' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:00:18.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:18 smithi166 bash[9446]: cluster 2020-09-08T16:00:16.370362+0000 osd.7 (osd.7) 121 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:00:18.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:18 smithi166 bash[9446]: cluster 2020-09-08T16:00:16.462012+0000 osd.7 (osd.7) 122 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:00:18.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:18 smithi166 bash[9446]: audit 2020-09-08T16:00:17.845369+0000 mon.a (mon.0) 1850 : audit [DBG] from='client.? v1:172.21.15.104:0/86812965' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:00:18.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:00:18 smithi055 bash[19003]: debug 2020-09-08T16:00:18.203+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:18.603 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2020-09-08T16:00:18.738 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:00:18.779 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:18.884 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:19.077 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:00:18 smithi104 bash[15273]: debug 2020-09-08T16:00:18.784+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:19.078 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:19 smithi104 bash[9203]: cluster 2020-09-08T16:00:17.958375+0000 mgr.x (mgr.24118) 989 : cluster [DBG] pgmap v1453: 17 pgs: 17 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 369 op/s 2020-09-08T16:00:19.079 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:19 smithi104 bash[9203]: audit 2020-09-08T16:00:18.076645+0000 mon.a (mon.0) 1851 : audit [INF] from='client.? v1:172.21.15.104:0/100991229' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:00:19.079 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:19 smithi104 bash[9203]: cluster 2020-09-08T16:00:19.071582+0000 mon.a (mon.0) 1852 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:00:19.079 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:19 smithi104 bash[9203]: cluster 2020-09-08T16:00:19.071655+0000 mon.a (mon.0) 1853 : cluster [INF] Cluster is now healthy 2020-09-08T16:00:19.080 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T16:00:19.359 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:19.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:19 smithi166 bash[9446]: cluster 2020-09-08T16:00:17.958375+0000 mgr.x (mgr.24118) 989 : cluster [DBG] pgmap v1453: 17 pgs: 17 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 369 op/s 2020-09-08T16:00:19.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:19 smithi166 bash[9446]: audit 2020-09-08T16:00:18.076645+0000 mon.a (mon.0) 1851 : audit [INF] from='client.? v1:172.21.15.104:0/100991229' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:00:19.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:19 smithi166 bash[9446]: cluster 2020-09-08T16:00:19.071582+0000 mon.a (mon.0) 1852 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:00:19.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:19 smithi166 bash[9446]: cluster 2020-09-08T16:00:19.071655+0000 mon.a (mon.0) 1853 : cluster [INF] Cluster is now healthy 2020-09-08T16:00:19.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:19 smithi055 bash[9296]: cluster 2020-09-08T16:00:17.958375+0000 mgr.x (mgr.24118) 989 : cluster [DBG] pgmap v1453: 17 pgs: 17 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 369 op/s 2020-09-08T16:00:19.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:19 smithi055 bash[9296]: audit 2020-09-08T16:00:18.076645+0000 mon.a (mon.0) 1851 : audit [INF] from='client.? v1:172.21.15.104:0/100991229' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:00:19.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:19 smithi055 bash[9296]: cluster 2020-09-08T16:00:19.071582+0000 mon.a (mon.0) 1852 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:00:19.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:19 smithi055 bash[9296]: cluster 2020-09-08T16:00:19.071655+0000 mon.a (mon.0) 1853 : cluster [INF] Cluster is now healthy 2020-09-08T16:00:19.467 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:00:19.658 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:00:19 smithi104 bash[15273]: debug 2020-09-08T16:00:19.360+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:19.743 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:00:19.848 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:19.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:00:19 smithi104 bash[19922]: debug 2020-09-08T16:00:19.748+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:19.993 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:20.094 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:20.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:00:19 smithi055 bash[16646]: debug 2020-09-08T16:00:19.991+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:20.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:20 smithi055 bash[9296]: audit 2020-09-08T16:00:19.079500+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? v1:172.21.15.104:0/100991229' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:00:20.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:20 smithi055 bash[9296]: cluster 2020-09-08T16:00:19.079576+0000 mon.a (mon.0) 1855 : cluster [DBG] osdmap e549: 12 total, 12 up, 12 in 2020-09-08T16:00:20.166 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:20 smithi166 bash[9446]: audit 2020-09-08T16:00:19.079500+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? v1:172.21.15.104:0/100991229' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:00:20.166 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:20 smithi166 bash[9446]: cluster 2020-09-08T16:00:19.079576+0000 mon.a (mon.0) 1855 : cluster [DBG] osdmap e549: 12 total, 12 up, 12 in 2020-09-08T16:00:20.311 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:20.417 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:20.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:20 smithi104 bash[9203]: audit 2020-09-08T16:00:19.079500+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? v1:172.21.15.104:0/100991229' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:00:20.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:20 smithi104 bash[9203]: cluster 2020-09-08T16:00:19.079576+0000 mon.a (mon.0) 1855 : cluster [DBG] osdmap e549: 12 total, 12 up, 12 in 2020-09-08T16:00:20.427 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:00:20 smithi104 bash[17588]: debug 2020-09-08T16:00:20.315+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:20.741 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:20.846 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:00:21.083 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:00:20 smithi166 bash[11928]: debug 2020-09-08T16:00:20.739+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:21.105 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'unique_pool_4' removed 2020-09-08T16:00:21.187 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:00:21.301 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:21.367 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:00:21 smithi166 bash[16593]: debug 2020-09-08T16:00:21.187+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:21.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:21 smithi166 bash[9446]: cluster 2020-09-08T16:00:19.958969+0000 mgr.x (mgr.24118) 990 : cluster [DBG] pgmap v1455: 17 pgs: 17 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 27 op/s 2020-09-08T16:00:21.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:21 smithi166 bash[9446]: audit 2020-09-08T16:00:20.718829+0000 mon.a (mon.0) 1856 : audit [INF] from='client.? v1:172.21.15.104:0/3186853091' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T16:00:21.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:21 smithi055 bash[9296]: cluster 2020-09-08T16:00:19.958969+0000 mgr.x (mgr.24118) 990 : cluster [DBG] pgmap v1455: 17 pgs: 17 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 27 op/s 2020-09-08T16:00:21.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:21 smithi055 bash[9296]: audit 2020-09-08T16:00:20.718829+0000 mon.a (mon.0) 1856 : audit [INF] from='client.? v1:172.21.15.104:0/3186853091' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T16:00:21.431 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:21 smithi104 bash[9203]: cluster 2020-09-08T16:00:19.958969+0000 mgr.x (mgr.24118) 990 : cluster [DBG] pgmap v1455: 17 pgs: 17 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 27 op/s 2020-09-08T16:00:21.432 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:21 smithi104 bash[9203]: audit 2020-09-08T16:00:20.718829+0000 mon.a (mon.0) 1856 : audit [INF] from='client.? v1:172.21.15.104:0/3186853091' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T16:00:21.432 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:21.543 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:21.675 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:00:21.758 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:21.814 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:00:21.863 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:21.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:00:21 smithi166 bash[11928]: debug 2020-09-08T16:00:21.759+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:21.913 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:00:21 smithi055 bash[16646]: debug 2020-09-08T16:00:21.435+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:22.035 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:22.141 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:22.153 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:22 smithi166 bash[9446]: audit 2020-09-08T16:00:21.100312+0000 mon.a (mon.0) 1857 : audit [INF] from='client.? v1:172.21.15.104:0/3186853091' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T16:00:22.153 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:22 smithi166 bash[9446]: cluster 2020-09-08T16:00:21.100489+0000 mon.a (mon.0) 1858 : cluster [DBG] osdmap e550: 12 total, 12 up, 12 in 2020-09-08T16:00:22.154 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:00:22 smithi166 bash[14254]: debug 2020-09-08T16:00:22.035+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:22.309 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2020-09-08T16:00:22.311 INFO:tasks.radosbench:Beginning radosbench... 2020-09-08T16:00:22.312 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_5 2020-09-08T16:00:22.312 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool create unique_pool_5 16 2020-09-08T16:00:22.314 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:22.342 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:00:22.343 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T16:00:22.343 INFO:teuthology.orchestra.run.smithi166.stdout: "complaint_time": 30, 2020-09-08T16:00:22.344 INFO:teuthology.orchestra.run.smithi166.stdout: "num_blocked_ops": 0 2020-09-08T16:00:22.344 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:00:22.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:22 smithi055 bash[9296]: audit 2020-09-08T16:00:21.100312+0000 mon.a (mon.0) 1857 : audit [INF] from='client.? v1:172.21.15.104:0/3186853091' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T16:00:22.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:22 smithi055 bash[9296]: cluster 2020-09-08T16:00:21.100489+0000 mon.a (mon.0) 1858 : cluster [DBG] osdmap e550: 12 total, 12 up, 12 in 2020-09-08T16:00:22.416 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:00:22 smithi166 bash[18960]: debug 2020-09-08T16:00:22.311+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:22.418 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:22.443 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:22 smithi104 bash[9203]: audit 2020-09-08T16:00:21.100312+0000 mon.a (mon.0) 1857 : audit [INF] from='client.? v1:172.21.15.104:0/3186853091' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T16:00:22.444 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:22 smithi104 bash[9203]: cluster 2020-09-08T16:00:21.100489+0000 mon.a (mon.0) 1858 : cluster [DBG] osdmap e550: 12 total, 12 up, 12 in 2020-09-08T16:00:22.446 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:00:22.558 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:22.588 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T16:00:22.661 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:22.743 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:00:22.784 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:00:22 smithi055 bash[16646]: debug 2020-09-08T16:00:22.555+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:22.784 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:22.890 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:23.059 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:23.101 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:00:22 smithi055 bash[11956]: debug 2020-09-08T16:00:22.787+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:23.164 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:00:23 smithi166 bash[11928]: debug 2020-09-08T16:00:23.059+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:23.164 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:23 smithi166 bash[9446]: cluster 2020-09-08T16:00:21.959568+0000 mgr.x (mgr.24118) 991 : cluster [DBG] pgmap v1457: 1 pgs: 1 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:23.165 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:23.167 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2020-09-08T16:00:23.294 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:23.298 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:00:23.400 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:23.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:00:23 smithi055 bash[11956]: debug 2020-09-08T16:00:23.295+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:23.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:23 smithi055 bash[9296]: cluster 2020-09-08T16:00:21.959568+0000 mgr.x (mgr.24118) 991 : cluster [DBG] pgmap v1457: 1 pgs: 1 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:23.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:23 smithi104 bash[9203]: cluster 2020-09-08T16:00:21.959568+0000 mgr.x (mgr.24118) 991 : cluster [DBG] pgmap v1457: 1 pgs: 1 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:23.889 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:23.994 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:24.293 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:00:23 smithi166 bash[14254]: debug 2020-09-08T16:00:23.887+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:24.294 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:24.402 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:24.547 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:24.594 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:00:24 smithi166 bash[18960]: debug 2020-09-08T16:00:24.295+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:24.653 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:24.793 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:24.861 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:00:24 smithi166 bash[18960]: debug 2020-09-08T16:00:24.787+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:24.894 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:00:24.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:00:24 smithi055 bash[16646]: debug 2020-09-08T16:00:24.547+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:25.073 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:00:25.118 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:00:25 smithi166 bash[16593]: debug 2020-09-08T16:00:25.071+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:25.179 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:25.353 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:25 smithi104 bash[9203]: cluster 2020-09-08T16:00:23.960250+0000 mgr.x (mgr.24118) 992 : cluster [DBG] pgmap v1458: 1 pgs: 1 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:25.354 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:25 smithi104 bash[9203]: cluster 2020-09-08T16:00:24.074856+0000 osd.5 (osd.5) 89 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:25.354 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:25 smithi104 bash[9203]: cluster 2020-09-08T16:00:24.074985+0000 osd.5 (osd.5) 90 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:25.355 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:25 smithi104 bash[9203]: cluster 2020-09-08T16:00:24.120029+0000 mon.a (mon.0) 1859 : cluster [DBG] osdmap e551: 12 total, 12 up, 12 in 2020-09-08T16:00:25.355 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:25.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:25 smithi166 bash[9446]: cluster 2020-09-08T16:00:23.960250+0000 mgr.x (mgr.24118) 992 : cluster [DBG] pgmap v1458: 1 pgs: 1 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:25.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:25 smithi166 bash[9446]: cluster 2020-09-08T16:00:24.074856+0000 osd.5 (osd.5) 89 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:25.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:25 smithi166 bash[9446]: cluster 2020-09-08T16:00:24.074985+0000 osd.5 (osd.5) 90 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:25.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:25 smithi166 bash[9446]: cluster 2020-09-08T16:00:24.120029+0000 mon.a (mon.0) 1859 : cluster [DBG] osdmap e551: 12 total, 12 up, 12 in 2020-09-08T16:00:25.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:25 smithi055 bash[9296]: cluster 2020-09-08T16:00:23.960250+0000 mgr.x (mgr.24118) 992 : cluster [DBG] pgmap v1458: 1 pgs: 1 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:25.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:25 smithi055 bash[9296]: cluster 2020-09-08T16:00:24.074856+0000 osd.5 (osd.5) 89 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:25.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:25 smithi055 bash[9296]: cluster 2020-09-08T16:00:24.074985+0000 osd.5 (osd.5) 90 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:25.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:25 smithi055 bash[9296]: cluster 2020-09-08T16:00:24.120029+0000 mon.a (mon.0) 1859 : cluster [DBG] osdmap e551: 12 total, 12 up, 12 in 2020-09-08T16:00:25.457 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:25.674 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:00:25 smithi104 bash[22305]: debug 2020-09-08T16:00:25.355+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:25.891 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:25.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:00:25 smithi104 bash[22305]: debug 2020-09-08T16:00:25.891+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:25.996 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:26.373 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:26.477 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:26.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:00:26 smithi104 bash[17588]: debug 2020-09-08T16:00:26.375+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:26.884 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:00:26.885 INFO:teuthology.orchestra.run.smithi166.stdout: "size": 20, 2020-09-08T16:00:26.885 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 600, 2020-09-08T16:00:26.885 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T16:00:26.885 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.887 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:7704 5.0 5:0bbaded2:::benchmark_data_smithi166_37820_object7703:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e404)", 2020-09-08T16:00:26.887 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:51:55.354677+0000", 2020-09-08T16:00:26.887 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 511.527225423, 2020-09-08T16:00:26.887 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.43157515899999999, 2020-09-08T16:00:26.888 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:26.888 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:00:26.888 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:00:26.888 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T16:00:26.888 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:00:26.889 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 7704 2020-09-08T16:00:26.889 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.889 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:26.890 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.890 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:26.890 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.354677+0000", 2020-09-08T16:00:26.891 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:26.891 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.891 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.891 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:26.892 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.354677+0000", 2020-09-08T16:00:26.892 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999785 2020-09-08T16:00:26.892 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.892 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.892 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:26.893 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.354655+0000", 2020-09-08T16:00:26.893 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.9197999999999995e-05 2020-09-08T16:00:26.893 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.893 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.894 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:26.894 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.354754+0000", 2020-09-08T16:00:26.894 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0932e-05 2020-09-08T16:00:26.894 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.895 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.895 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:26.895 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.354765+0000", 2020-09-08T16:00:26.896 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5749999999999997e-06 2020-09-08T16:00:26.896 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.896 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.897 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:26.898 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.354772+0000", 2020-09-08T16:00:26.898 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.8350999999999999e-05 2020-09-08T16:00:26.898 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.899 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.899 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:26.899 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.354810+0000", 2020-09-08T16:00:26.900 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000123998 2020-09-08T16:00:26.900 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.900 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.901 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:00:26.901 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.354934+0000", 2020-09-08T16:00:26.901 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00080971999999999999 2020-09-08T16:00:26.902 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.902 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.902 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T16:00:26.902 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.355744+0000", 2020-09-08T16:00:26.903 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00089288200000000001 2020-09-08T16:00:26.903 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.903 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.904 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:00:26.904 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.356637+0000", 2020-09-08T16:00:26.904 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.429248611 2020-09-08T16:00:26.905 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.905 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.905 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:00:26.906 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.785885+0000", 2020-09-08T16:00:26.906 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000170753 2020-09-08T16:00:26.906 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.907 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.908 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:00:26.908 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.786056+0000", 2020-09-08T16:00:26.908 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019570000000000001 2020-09-08T16:00:26.909 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.909 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.909 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:00:26.910 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.786252+0000", 2020-09-08T16:00:26.910 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.43157515899999999 2020-09-08T16:00:26.910 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:26.911 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:26.911 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:26.911 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.912 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.912 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:7704 5.0 5:0bbaded2:::benchmark_data_smithi166_37820_object7703:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e404)", 2020-09-08T16:00:26.912 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:51:55.357082+0000", 2020-09-08T16:00:26.912 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 511.52481988099998, 2020-09-08T16:00:26.913 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.42902537899999998, 2020-09-08T16:00:26.913 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:26.913 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "delayed", 2020-09-08T16:00:26.914 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:00:26.914 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T16:00:26.915 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:00:26.915 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 7704 2020-09-08T16:00:26.915 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.915 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:26.916 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.916 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:26.916 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357082+0000", 2020-09-08T16:00:26.917 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:26.917 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.917 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.918 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:26.918 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357082+0000", 2020-09-08T16:00:26.919 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.999958 2020-09-08T16:00:26.919 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.919 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.920 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:26.920 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357040+0000", 2020-09-08T16:00:26.920 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015511400000000001 2020-09-08T16:00:26.921 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.921 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.921 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:26.922 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357195+0000", 2020-09-08T16:00:26.922 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.9808000000000001e-05 2020-09-08T16:00:26.922 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.922 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.923 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:26.923 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357215+0000", 2020-09-08T16:00:26.923 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.147e-06 2020-09-08T16:00:26.924 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.924 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.924 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:26.925 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357220+0000", 2020-09-08T16:00:26.925 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.1962999999999998e-05 2020-09-08T16:00:26.925 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.925 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.926 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:26.926 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357262+0000", 2020-09-08T16:00:26.926 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8329000000000001e-05 2020-09-08T16:00:26.927 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.927 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.927 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for ondisk", 2020-09-08T16:00:26.928 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357350+0000", 2020-09-08T16:00:26.928 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.428757055 2020-09-08T16:00:26.929 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.929 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.929 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:00:26.930 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.786108+0000", 2020-09-08T16:00:26.930 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.42902537899999998 2020-09-08T16:00:26.930 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:26.931 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:26.931 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:26.931 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.931 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.931 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:7706 5.0 5:097a8012:::benchmark_data_smithi166_37820_object7705:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e404)", 2020-09-08T16:00:26.932 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:51:55.357287+0000", 2020-09-08T16:00:26.932 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 511.52461506399999, 2020-09-08T16:00:26.932 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.42959861999999999, 2020-09-08T16:00:26.932 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:26.932 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:00:26.933 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:00:26.933 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T16:00:26.933 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:00:26.933 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 7706 2020-09-08T16:00:26.934 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.934 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:26.934 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.934 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:26.934 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357287+0000", 2020-09-08T16:00:26.934 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:26.935 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.935 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.935 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:26.935 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357287+0000", 2020-09-08T16:00:26.935 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999628 2020-09-08T16:00:26.936 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.936 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.936 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:26.936 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357250+0000", 2020-09-08T16:00:26.936 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010648499999999999 2020-09-08T16:00:26.937 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.937 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.937 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:26.937 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357356+0000", 2020-09-08T16:00:26.937 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3390000000000001e-05 2020-09-08T16:00:26.938 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.938 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.938 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:26.938 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357370+0000", 2020-09-08T16:00:26.938 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.146e-06 2020-09-08T16:00:26.938 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.939 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.939 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:26.939 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357373+0000", 2020-09-08T16:00:26.940 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9023999999999997e-05 2020-09-08T16:00:26.940 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.940 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.940 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:26.940 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357422+0000", 2020-09-08T16:00:26.941 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000154573 2020-09-08T16:00:26.941 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.941 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.941 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:00:26.941 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357577+0000", 2020-09-08T16:00:26.942 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017475699999999999 2020-09-08T16:00:26.942 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.942 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.942 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T16:00:26.942 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357751+0000", 2020-09-08T16:00:26.943 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.001286073 2020-09-08T16:00:26.943 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.943 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.943 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:00:26.943 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.359037+0000", 2020-09-08T16:00:26.944 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.427345425 2020-09-08T16:00:26.944 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.944 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.944 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:00:26.944 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.786383+0000", 2020-09-08T16:00:26.945 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00035875300000000001 2020-09-08T16:00:26.945 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.945 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.945 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:00:26.945 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.786742+0000", 2020-09-08T16:00:26.945 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000144017 2020-09-08T16:00:26.946 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.946 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.946 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:00:26.946 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.786886+0000", 2020-09-08T16:00:26.946 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.42959861999999999 2020-09-08T16:00:26.947 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:26.947 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:26.947 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:26.947 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.947 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.948 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:7715 5.0 5:0f2edb07:::benchmark_data_smithi166_37820_object7714:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e404)", 2020-09-08T16:00:26.948 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:51:55.357601+0000", 2020-09-08T16:00:26.948 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 511.52430062500002, 2020-09-08T16:00:26.948 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.429648998, 2020-09-08T16:00:26.948 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:26.949 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:00:26.949 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:00:26.949 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T16:00:26.949 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:00:26.950 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 7715 2020-09-08T16:00:26.950 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.950 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:26.950 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.951 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:26.951 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357601+0000", 2020-09-08T16:00:26.951 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:26.951 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.951 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.952 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:26.952 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357601+0000", 2020-09-08T16:00:26.952 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999628 2020-09-08T16:00:26.952 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.953 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.953 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:26.953 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357564+0000", 2020-09-08T16:00:26.953 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012373700000000001 2020-09-08T16:00:26.953 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.954 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.954 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:26.954 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357688+0000", 2020-09-08T16:00:26.955 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2170999999999999e-05 2020-09-08T16:00:26.955 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.956 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.956 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:26.956 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357700+0000", 2020-09-08T16:00:26.956 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.3579999999999999e-06 2020-09-08T16:00:26.957 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.957 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.957 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:26.957 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.357706+0000", 2020-09-08T16:00:26.957 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00064858600000000004 2020-09-08T16:00:26.958 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.958 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.958 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:26.959 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.358355+0000", 2020-09-08T16:00:26.959 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015122099999999999 2020-09-08T16:00:26.959 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.959 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.959 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:00:26.960 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.358506+0000", 2020-09-08T16:00:26.960 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016547900000000001 2020-09-08T16:00:26.960 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.960 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.960 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T16:00:26.961 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.358672+0000", 2020-09-08T16:00:26.961 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.001504989 2020-09-08T16:00:26.961 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.961 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.962 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:00:26.962 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.360177+0000", 2020-09-08T16:00:26.962 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.42683196499999998 2020-09-08T16:00:26.962 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.963 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.963 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:00:26.963 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.787009+0000", 2020-09-08T16:00:26.963 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011330799999999999 2020-09-08T16:00:26.964 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.964 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.964 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:00:26.964 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.787122+0000", 2020-09-08T16:00:26.964 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000128381 2020-09-08T16:00:26.965 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.965 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.965 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:00:26.965 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.787250+0000", 2020-09-08T16:00:26.965 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.429648998 2020-09-08T16:00:26.966 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:26.966 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:26.966 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:26.966 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.966 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.967 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:7720 5.0 5:0b59d8a1:::benchmark_data_smithi166_37820_object7719:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e404)", 2020-09-08T16:00:26.967 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:51:55.359387+0000", 2020-09-08T16:00:26.967 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 511.52251475100002, 2020-09-08T16:00:26.967 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.428431909, 2020-09-08T16:00:26.968 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:26.968 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:00:26.968 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:00:26.968 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T16:00:26.969 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:00:26.969 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 7720 2020-09-08T16:00:26.969 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.969 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:26.969 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.970 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:26.970 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.359387+0000", 2020-09-08T16:00:26.970 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:26.970 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.970 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.970 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:26.971 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.359387+0000", 2020-09-08T16:00:26.971 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999661 2020-09-08T16:00:26.971 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.971 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.971 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:26.972 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.359353+0000", 2020-09-08T16:00:26.972 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000111826 2020-09-08T16:00:26.972 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.972 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.973 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:26.973 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.359465+0000", 2020-09-08T16:00:26.973 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3842000000000001e-05 2020-09-08T16:00:26.974 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.974 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.974 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:26.974 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.359479+0000", 2020-09-08T16:00:26.974 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.1249999999999998e-06 2020-09-08T16:00:26.975 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.975 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.975 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:26.975 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.359485+0000", 2020-09-08T16:00:26.975 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8758000000000002e-05 2020-09-08T16:00:26.976 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.976 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.976 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:26.976 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.359544+0000", 2020-09-08T16:00:26.976 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000140316 2020-09-08T16:00:26.977 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.977 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.977 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:00:26.977 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.359684+0000", 2020-09-08T16:00:26.977 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000171428 2020-09-08T16:00:26.978 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.978 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.978 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T16:00:26.978 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.359856+0000", 2020-09-08T16:00:26.978 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00064573500000000004 2020-09-08T16:00:26.979 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.979 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.979 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:00:26.979 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.360501+0000", 2020-09-08T16:00:26.979 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.426978362 2020-09-08T16:00:26.980 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.980 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.980 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:00:26.980 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.787480+0000", 2020-09-08T16:00:26.980 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017051600000000001 2020-09-08T16:00:26.981 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.981 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.981 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:00:26.981 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.787650+0000", 2020-09-08T16:00:26.982 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000168877 2020-09-08T16:00:26.982 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.982 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.982 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:00:26.983 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:51:55.787819+0000", 2020-09-08T16:00:26.983 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.428431909 2020-09-08T16:00:26.983 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:26.983 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:26.983 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:26.984 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.984 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.984 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:118561 5.0 5:078509c7:::benchmark_data_smithi166_37820_object118560:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T16:00:26.984 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:52:58.577384+0000", 2020-09-08T16:00:26.984 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 448.30451836999998, 2020-09-08T16:00:26.985 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.29188393699999998, 2020-09-08T16:00:26.985 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:26.985 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:00:26.985 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:00:26.986 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T16:00:26.986 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:00:26.986 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 118561 2020-09-08T16:00:26.987 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.987 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:26.987 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.987 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:26.987 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.577384+0000", 2020-09-08T16:00:26.988 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:26.988 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.988 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.988 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:26.988 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.577384+0000", 2020-09-08T16:00:26.989 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999614 2020-09-08T16:00:26.989 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.989 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.990 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:26.990 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.577345+0000", 2020-09-08T16:00:26.990 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000121805 2020-09-08T16:00:26.990 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.991 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.991 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:26.991 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.577467+0000", 2020-09-08T16:00:26.991 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6928000000000001e-05 2020-09-08T16:00:26.991 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.992 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.992 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:26.992 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.577484+0000", 2020-09-08T16:00:26.992 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.5660000000000007e-06 2020-09-08T16:00:26.992 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.993 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.993 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:26.993 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.577492+0000", 2020-09-08T16:00:26.993 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.7833000000000001e-05 2020-09-08T16:00:26.993 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.994 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.994 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:26.994 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.577540+0000", 2020-09-08T16:00:26.994 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000188111 2020-09-08T16:00:26.994 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.995 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.995 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:00:26.995 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.577728+0000", 2020-09-08T16:00:26.995 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016928000000000001 2020-09-08T16:00:26.995 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.996 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.996 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T16:00:26.996 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.577898+0000", 2020-09-08T16:00:26.996 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0038669419999999999 2020-09-08T16:00:26.996 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.996 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.997 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:00:26.997 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.581765+0000", 2020-09-08T16:00:26.997 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.28721324799999998 2020-09-08T16:00:26.997 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.997 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.998 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:00:26.998 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.868978+0000", 2020-09-08T16:00:26.998 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000203122 2020-09-08T16:00:26.998 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:26.999 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:26.999 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:00:26.999 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.869181+0000", 2020-09-08T16:00:26.999 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.6612000000000004e-05 2020-09-08T16:00:26.999 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.000 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.000 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:00:27.000 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.869268+0000", 2020-09-08T16:00:27.000 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.29188393699999998 2020-09-08T16:00:27.000 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.001 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:27.001 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.001 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.001 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.001 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:118591 5.0 5:0fba6394:::benchmark_data_smithi166_37820_object118590:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T16:00:27.002 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:52:58.605137+0000", 2020-09-08T16:00:27.002 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 448.27676484, 2020-09-08T16:00:27.002 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.26457295800000002, 2020-09-08T16:00:27.002 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:27.002 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:00:27.003 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:00:27.003 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T16:00:27.003 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:00:27.003 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 118591 2020-09-08T16:00:27.003 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.004 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:27.004 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.004 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:27.004 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.605137+0000", 2020-09-08T16:00:27.004 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:27.005 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.005 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.005 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:27.005 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.605137+0000", 2020-09-08T16:00:27.005 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999762 2020-09-08T16:00:27.005 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.006 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.006 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:27.006 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.605113+0000", 2020-09-08T16:00:27.006 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.1184000000000001e-05 2020-09-08T16:00:27.006 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.007 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.007 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:27.007 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.605194+0000", 2020-09-08T16:00:27.007 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.182e-05 2020-09-08T16:00:27.007 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.008 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.008 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:27.008 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.605206+0000", 2020-09-08T16:00:27.009 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.169e-06 2020-09-08T16:00:27.009 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.009 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.009 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:27.009 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.605211+0000", 2020-09-08T16:00:27.010 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.736e-05 2020-09-08T16:00:27.010 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.010 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.010 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:27.011 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.605269+0000", 2020-09-08T16:00:27.011 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000219244 2020-09-08T16:00:27.011 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.011 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.011 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:00:27.011 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.605488+0000", 2020-09-08T16:00:27.012 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00021581000000000001 2020-09-08T16:00:27.012 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.012 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.012 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T16:00:27.012 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.605704+0000", 2020-09-08T16:00:27.013 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0043706270000000002 2020-09-08T16:00:27.013 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.013 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.013 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:00:27.013 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.610074+0000", 2020-09-08T16:00:27.014 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.25934383700000002 2020-09-08T16:00:27.014 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.014 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.014 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:00:27.015 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.869418+0000", 2020-09-08T16:00:27.015 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013434300000000001 2020-09-08T16:00:27.015 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.015 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.015 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:00:27.016 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.869553+0000", 2020-09-08T16:00:27.016 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015758699999999999 2020-09-08T16:00:27.016 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.016 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.017 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:00:27.017 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:52:58.869710+0000", 2020-09-08T16:00:27.017 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.26457295800000002 2020-09-08T16:00:27.017 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.017 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:27.018 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.018 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.018 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.018 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:120072 5.0 5:0ac44702:::benchmark_data_smithi166_37820_object120071:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T16:00:27.019 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:53:00.071721+0000", 2020-09-08T16:00:27.019 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 446.81018149300002, 2020-09-08T16:00:27.019 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.57124581799999996, 2020-09-08T16:00:27.020 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:27.020 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:00:27.020 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:00:27.020 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T16:00:27.020 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:00:27.021 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 120072 2020-09-08T16:00:27.021 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.021 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:27.021 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.021 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:27.022 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.071721+0000", 2020-09-08T16:00:27.022 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:27.023 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.023 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.023 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:27.023 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.071721+0000", 2020-09-08T16:00:27.023 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999485 2020-09-08T16:00:27.024 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.024 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.024 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:27.024 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.071669+0000", 2020-09-08T16:00:27.025 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000150407 2020-09-08T16:00:27.025 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.025 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.025 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:27.026 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.071820+0000", 2020-09-08T16:00:27.026 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.555011488 2020-09-08T16:00:27.026 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.026 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.026 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:27.027 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.626831+0000", 2020-09-08T16:00:27.027 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.9632999999999999e-05 2020-09-08T16:00:27.027 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.027 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.027 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:27.028 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.626861+0000", 2020-09-08T16:00:27.028 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00045113999999999999 2020-09-08T16:00:27.028 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.028 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.028 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:27.029 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.627312+0000", 2020-09-08T16:00:27.029 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019331400000000001 2020-09-08T16:00:27.029 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.029 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.029 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:00:27.030 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.627505+0000", 2020-09-08T16:00:27.030 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00021965299999999999 2020-09-08T16:00:27.030 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.030 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.030 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T16:00:27.031 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.627725+0000", 2020-09-08T16:00:27.031 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.015100173 2020-09-08T16:00:27.031 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.031 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.031 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:00:27.032 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.642825+0000", 2020-09-08T16:00:27.032 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.747e-06 2020-09-08T16:00:27.032 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.032 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.032 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:00:27.032 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.642835+0000", 2020-09-08T16:00:27.033 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.3275999999999994e-05 2020-09-08T16:00:27.033 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.033 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.033 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:00:27.034 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.642908+0000", 2020-09-08T16:00:27.034 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8337000000000001e-05 2020-09-08T16:00:27.034 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.034 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.034 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:00:27.035 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:00.642966+0000", 2020-09-08T16:00:27.035 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.57124581799999996 2020-09-08T16:00:27.035 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.036 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:27.036 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.036 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.036 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.036 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:148759 5.0 5:03ec4fdd:::benchmark_data_smithi166_37820_object4337:head [delete] snapc 0=[] ondisk+write+known_if_redirected e432)", 2020-09-08T16:00:27.037 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:53:28.604376+0000", 2020-09-08T16:00:27.037 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 418.277525787, 2020-09-08T16:00:27.037 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.159459764, 2020-09-08T16:00:27.037 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:27.037 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:00:27.038 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:00:27.051 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T16:00:27.051 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:00:27.051 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 148759 2020-09-08T16:00:27.051 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.051 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:27.052 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.052 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:27.052 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:28.604376+0000", 2020-09-08T16:00:27.052 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:27.053 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.053 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.053 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:27.053 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:28.604376+0000", 2020-09-08T16:00:27.053 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999576 2020-09-08T16:00:27.054 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.054 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.054 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:27.054 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:28.604334+0000", 2020-09-08T16:00:27.054 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.4959999999999999e-05 2020-09-08T16:00:27.055 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.055 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.055 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:27.055 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:28.604429+0000", 2020-09-08T16:00:27.055 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.8697000000000001e-05 2020-09-08T16:00:27.055 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.056 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.057 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:27.057 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:28.604447+0000", 2020-09-08T16:00:27.057 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7599999999999997e-06 2020-09-08T16:00:27.057 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.058 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.058 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:27.058 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:28.604457+0000", 2020-09-08T16:00:27.058 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.9703999999999998e-05 2020-09-08T16:00:27.059 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.059 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.059 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:27.059 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:28.604497+0000", 2020-09-08T16:00:27.059 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010247 2020-09-08T16:00:27.060 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.060 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.060 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:00:27.060 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:28.604599+0000", 2020-09-08T16:00:27.060 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7341000000000004e-05 2020-09-08T16:00:27.061 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.061 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.061 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T16:00:27.061 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:28.604677+0000", 2020-09-08T16:00:27.061 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0041124239999999999 2020-09-08T16:00:27.062 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.062 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.062 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:00:27.062 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:28.608789+0000", 2020-09-08T16:00:27.062 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15494918999999999 2020-09-08T16:00:27.063 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.063 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.063 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:00:27.063 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:28.763738+0000", 2020-09-08T16:00:27.064 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.5909000000000003e-05 2020-09-08T16:00:27.064 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.064 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.064 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:00:27.064 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:28.763794+0000", 2020-09-08T16:00:27.065 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.1807000000000001e-05 2020-09-08T16:00:27.065 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.065 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.065 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:00:27.065 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:28.763836+0000", 2020-09-08T16:00:27.066 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.159459764 2020-09-08T16:00:27.066 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.066 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:27.067 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.067 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.067 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.068 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:152618 5.4 5:29440e3a:::benchmark_data_smithi166_37820_object8196:head [delete] snapc 0=[] ondisk+write+known_if_redirected e432)", 2020-09-08T16:00:27.068 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:53:32.656212+0000", 2020-09-08T16:00:27.068 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 414.22568975399997, 2020-09-08T16:00:27.068 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22814868099999999, 2020-09-08T16:00:27.069 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:27.069 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:00:27.069 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:00:27.069 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T16:00:27.069 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:00:27.070 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 152618 2020-09-08T16:00:27.070 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.070 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:27.070 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.070 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:27.071 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:32.656212+0000", 2020-09-08T16:00:27.071 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:27.071 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.071 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.071 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:27.072 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:32.656212+0000", 2020-09-08T16:00:27.072 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999499 2020-09-08T16:00:27.072 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.072 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.072 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:27.073 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:32.656162+0000", 2020-09-08T16:00:27.073 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011575500000000001 2020-09-08T16:00:27.073 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.073 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.074 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:27.074 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:32.656278+0000", 2020-09-08T16:00:27.074 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1717e-05 2020-09-08T16:00:27.074 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.074 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.075 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:27.075 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:32.656290+0000", 2020-09-08T16:00:27.075 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.348e-06 2020-09-08T16:00:27.075 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.075 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.076 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:27.076 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:32.656296+0000", 2020-09-08T16:00:27.076 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.077970992000000003 2020-09-08T16:00:27.076 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.076 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.077 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:27.077 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:32.734267+0000", 2020-09-08T16:00:27.078 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00024681900000000001 2020-09-08T16:00:27.078 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.078 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.078 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:00:27.079 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:32.734514+0000", 2020-09-08T16:00:27.079 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00025069799999999999 2020-09-08T16:00:27.079 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.079 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.079 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:00:27.080 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:32.734765+0000", 2020-09-08T16:00:27.080 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0044107089999999996 2020-09-08T16:00:27.080 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.080 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.080 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:00:27.081 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:32.739175+0000", 2020-09-08T16:00:27.081 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14503124100000001 2020-09-08T16:00:27.081 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.081 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.082 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:00:27.082 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:32.884207+0000", 2020-09-08T16:00:27.082 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7489000000000001e-05 2020-09-08T16:00:27.082 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.082 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.083 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:00:27.083 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:32.884304+0000", 2020-09-08T16:00:27.083 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.6919999999999997e-05 2020-09-08T16:00:27.083 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.083 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.084 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:00:27.084 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:32.884361+0000", 2020-09-08T16:00:27.084 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22814868099999999 2020-09-08T16:00:27.084 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.084 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:27.085 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.085 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.085 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.085 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:153287 5.4 5:2871df86:::benchmark_data_smithi166_37820_object8865:head [delete] snapc 0=[] ondisk+write+known_if_redirected e433)", 2020-09-08T16:00:27.085 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:53:33.408162+0000", 2020-09-08T16:00:27.086 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 413.47374005500001, 2020-09-08T16:00:27.086 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19575261899999999, 2020-09-08T16:00:27.086 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:27.086 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:00:27.086 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:00:27.087 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T16:00:27.087 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:00:27.087 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 153287 2020-09-08T16:00:27.087 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.088 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:27.088 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.089 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:27.089 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.408162+0000", 2020-09-08T16:00:27.089 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:27.089 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.089 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.090 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:27.090 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.408162+0000", 2020-09-08T16:00:27.090 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999585 2020-09-08T16:00:27.090 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.090 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.091 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:27.091 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.408120+0000", 2020-09-08T16:00:27.091 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.7521000000000005e-05 2020-09-08T16:00:27.091 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.091 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.092 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:27.092 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.408208+0000", 2020-09-08T16:00:27.092 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0706999999999999e-05 2020-09-08T16:00:27.092 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.092 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.093 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:27.093 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.408219+0000", 2020-09-08T16:00:27.093 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.3900000000000001e-06 2020-09-08T16:00:27.093 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.094 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.094 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:27.094 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.408224+0000", 2020-09-08T16:00:27.094 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0095456580000000003 2020-09-08T16:00:27.094 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.095 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.095 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:27.095 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.417770+0000", 2020-09-08T16:00:27.095 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00042037200000000002 2020-09-08T16:00:27.095 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.096 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.096 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:00:27.096 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.418190+0000", 2020-09-08T16:00:27.096 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00026876800000000001 2020-09-08T16:00:27.096 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.097 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.097 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:00:27.097 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.418459+0000", 2020-09-08T16:00:27.097 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.011956525000000001 2020-09-08T16:00:27.097 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.098 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.098 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:00:27.099 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.430415+0000", 2020-09-08T16:00:27.099 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17341330599999999 2020-09-08T16:00:27.099 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.099 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.099 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:00:27.100 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.603829+0000", 2020-09-08T16:00:27.100 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9336000000000003e-05 2020-09-08T16:00:27.100 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.100 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.101 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:00:27.101 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.603878+0000", 2020-09-08T16:00:27.101 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.6677999999999997e-05 2020-09-08T16:00:27.101 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.101 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.102 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:00:27.102 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.603915+0000", 2020-09-08T16:00:27.102 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19575261899999999 2020-09-08T16:00:27.102 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.102 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:27.103 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.103 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.103 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.103 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:153289 5.4 5:28a65d76:::benchmark_data_smithi166_37820_object8867:head [delete] snapc 0=[] ondisk+write+known_if_redirected e433)", 2020-09-08T16:00:27.103 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:53:33.408537+0000", 2020-09-08T16:00:27.104 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 413.47336481299999, 2020-09-08T16:00:27.104 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19553702000000001, 2020-09-08T16:00:27.104 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:27.104 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:00:27.104 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:00:27.105 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T16:00:27.105 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:00:27.105 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 153289 2020-09-08T16:00:27.105 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.105 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:27.106 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.106 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:27.106 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.408537+0000", 2020-09-08T16:00:27.106 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:27.106 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.107 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.107 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:27.107 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.408537+0000", 2020-09-08T16:00:27.107 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999709 2020-09-08T16:00:27.108 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.108 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.108 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:27.108 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.408508+0000", 2020-09-08T16:00:27.108 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.1779999999999995e-05 2020-09-08T16:00:27.110 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.111 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.111 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:27.111 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.408570+0000", 2020-09-08T16:00:27.111 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.611e-06 2020-09-08T16:00:27.112 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.112 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.112 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:27.112 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.408580+0000", 2020-09-08T16:00:27.112 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.553e-06 2020-09-08T16:00:27.113 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.113 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.113 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:27.113 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.408585+0000", 2020-09-08T16:00:27.113 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.01753242 2020-09-08T16:00:27.114 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.114 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.114 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:27.114 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.426118+0000", 2020-09-08T16:00:27.114 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014281 2020-09-08T16:00:27.115 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.115 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.115 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:00:27.115 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.426260+0000", 2020-09-08T16:00:27.115 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011341900000000001 2020-09-08T16:00:27.116 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.116 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.116 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:00:27.117 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.426374+0000", 2020-09-08T16:00:27.117 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0042433840000000002 2020-09-08T16:00:27.118 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.118 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.118 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:00:27.118 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.430617+0000", 2020-09-08T16:00:27.118 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.173358333 2020-09-08T16:00:27.119 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.119 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.120 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:00:27.120 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.603976+0000", 2020-09-08T16:00:27.120 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.6937000000000001e-05 2020-09-08T16:00:27.120 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.120 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.121 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:00:27.121 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.604023+0000", 2020-09-08T16:00:27.121 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1724999999999997e-05 2020-09-08T16:00:27.122 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.122 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.122 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:00:27.122 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:33.604074+0000", 2020-09-08T16:00:27.122 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19553702000000001 2020-09-08T16:00:27.123 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.123 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:27.123 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.123 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.123 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.124 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:154245 5.4 5:2b628ad5:::benchmark_data_smithi166_37820_object9823:head [delete] snapc 0=[] ondisk+write+known_if_redirected e433)", 2020-09-08T16:00:27.124 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:53:34.576625+0000", 2020-09-08T16:00:27.124 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 412.305277198, 2020-09-08T16:00:27.125 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.162009718, 2020-09-08T16:00:27.125 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:27.125 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:00:27.125 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:00:27.126 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T16:00:27.126 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:00:27.126 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 154245 2020-09-08T16:00:27.126 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.126 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:27.127 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.127 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:27.127 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:34.576625+0000", 2020-09-08T16:00:27.127 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:27.127 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.128 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.128 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:27.128 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:34.576625+0000", 2020-09-08T16:00:27.128 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999604 2020-09-08T16:00:27.128 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.129 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.129 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:27.129 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:34.576585+0000", 2020-09-08T16:00:27.129 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.7276999999999998e-05 2020-09-08T16:00:27.130 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.130 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.130 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:27.131 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:34.576672+0000", 2020-09-08T16:00:27.131 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.8896e-05 2020-09-08T16:00:27.131 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.131 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.131 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:27.132 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:34.576691+0000", 2020-09-08T16:00:27.132 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9300000000000002e-06 2020-09-08T16:00:27.132 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.132 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.132 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:27.133 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:34.576696+0000", 2020-09-08T16:00:27.133 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0046893610000000004 2020-09-08T16:00:27.133 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.133 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.134 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:27.134 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:34.581386+0000", 2020-09-08T16:00:27.134 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.9020999999999999e-05 2020-09-08T16:00:27.134 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.134 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.135 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:00:27.135 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:34.581485+0000", 2020-09-08T16:00:27.135 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000106706 2020-09-08T16:00:27.135 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.135 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.136 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:00:27.136 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:34.581591+0000", 2020-09-08T16:00:27.136 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.003156233 2020-09-08T16:00:27.136 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.136 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.136 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:00:27.137 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:34.584748+0000", 2020-09-08T16:00:27.137 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15382163700000001 2020-09-08T16:00:27.137 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.137 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.137 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:00:27.138 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:34.738569+0000", 2020-09-08T16:00:27.138 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.7013e-05 2020-09-08T16:00:27.138 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.138 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.138 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:00:27.139 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:34.738606+0000", 2020-09-08T16:00:27.139 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.8306999999999999e-05 2020-09-08T16:00:27.139 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.139 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.139 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:00:27.140 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:34.738635+0000", 2020-09-08T16:00:27.140 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.162009718 2020-09-08T16:00:27.140 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.141 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:27.141 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.141 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.141 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.141 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:155833 5.4 5:2a35e4b5:::benchmark_data_smithi166_37820_object11411:head [delete] snapc 0=[] ondisk+write+known_if_redirected e433)", 2020-09-08T16:00:27.142 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:53:36.347859+0000", 2020-09-08T16:00:27.142 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 410.534042795, 2020-09-08T16:00:27.142 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16742858499999999, 2020-09-08T16:00:27.142 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:27.143 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:00:27.143 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:00:27.143 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T16:00:27.143 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:00:27.143 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 155833 2020-09-08T16:00:27.144 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.144 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:27.144 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.144 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:27.144 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.347859+0000", 2020-09-08T16:00:27.145 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:27.145 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.145 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.145 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:27.145 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.347859+0000", 2020-09-08T16:00:27.146 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999423 2020-09-08T16:00:27.146 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.146 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.146 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:27.146 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.347802+0000", 2020-09-08T16:00:27.147 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012487899999999999 2020-09-08T16:00:27.147 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.147 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.147 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:27.147 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.347927+0000", 2020-09-08T16:00:27.147 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.9905999999999999e-05 2020-09-08T16:00:27.148 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.148 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.148 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:27.148 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.347946+0000", 2020-09-08T16:00:27.148 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4416999999999999e-05 2020-09-08T16:00:27.149 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.149 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.149 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:27.149 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.347961+0000", 2020-09-08T16:00:27.149 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.6391000000000003e-05 2020-09-08T16:00:27.150 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.150 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.150 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:27.150 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.348017+0000", 2020-09-08T16:00:27.151 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000137451 2020-09-08T16:00:27.151 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.151 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.152 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:00:27.152 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.348155+0000", 2020-09-08T16:00:27.152 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017077100000000001 2020-09-08T16:00:27.152 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.152 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.152 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:00:27.153 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.348326+0000", 2020-09-08T16:00:27.153 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0066158889999999998 2020-09-08T16:00:27.153 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.153 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.154 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:00:27.154 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.354941+0000", 2020-09-08T16:00:27.155 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.160246271 2020-09-08T16:00:27.155 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.155 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.155 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:00:27.155 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.515188+0000", 2020-09-08T16:00:27.156 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.2052000000000003e-05 2020-09-08T16:00:27.156 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.156 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.156 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:00:27.157 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.515230+0000", 2020-09-08T16:00:27.157 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8105e-05 2020-09-08T16:00:27.157 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.157 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.157 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:00:27.158 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.515288+0000", 2020-09-08T16:00:27.158 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16742858499999999 2020-09-08T16:00:27.158 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.158 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:27.158 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.159 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.159 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.159 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:155839 5.4 5:2d740a50:::benchmark_data_smithi166_37820_object11417:head [delete] snapc 0=[] ondisk+write+known_if_redirected e433)", 2020-09-08T16:00:27.159 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:53:36.355618+0000", 2020-09-08T16:00:27.159 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 410.52628356600002, 2020-09-08T16:00:27.160 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15988096500000001, 2020-09-08T16:00:27.160 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:27.160 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:00:27.160 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:00:27.161 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T16:00:27.161 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:00:27.161 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 155839 2020-09-08T16:00:27.161 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.161 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:27.162 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.162 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:27.163 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.355618+0000", 2020-09-08T16:00:27.163 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:27.163 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.163 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.163 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:27.164 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.355618+0000", 2020-09-08T16:00:27.164 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999762 2020-09-08T16:00:27.164 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.164 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.164 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:27.165 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.355595+0000", 2020-09-08T16:00:27.165 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.1563000000000007e-05 2020-09-08T16:00:27.165 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.165 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.165 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:27.166 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.355656+0000", 2020-09-08T16:00:27.166 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.5759999999999995e-06 2020-09-08T16:00:27.166 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.166 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.167 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:27.167 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.355666+0000", 2020-09-08T16:00:27.167 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.5279999999999997e-06 2020-09-08T16:00:27.167 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.167 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.168 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:27.168 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.355670+0000", 2020-09-08T16:00:27.169 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.4131999999999997e-05 2020-09-08T16:00:27.170 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.170 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.170 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:27.170 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.355714+0000", 2020-09-08T16:00:27.171 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7517999999999997e-05 2020-09-08T16:00:27.171 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.171 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.171 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:00:27.171 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.355812+0000", 2020-09-08T16:00:27.172 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7700000000000003e-05 2020-09-08T16:00:27.172 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.173 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.173 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:00:27.173 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.355910+0000", 2020-09-08T16:00:27.173 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0045226019999999997 2020-09-08T16:00:27.174 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.174 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.174 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:00:27.174 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.360432+0000", 2020-09-08T16:00:27.174 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15492819199999999 2020-09-08T16:00:27.175 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.175 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.175 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:00:27.175 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.515360+0000", 2020-09-08T16:00:27.175 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.7874999999999995e-05 2020-09-08T16:00:27.176 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.176 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.176 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:00:27.176 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.515448+0000", 2020-09-08T16:00:27.176 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1165000000000002e-05 2020-09-08T16:00:27.177 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.177 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.178 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:00:27.178 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:53:36.515499+0000", 2020-09-08T16:00:27.179 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15988096500000001 2020-09-08T16:00:27.179 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.179 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:27.180 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.180 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.180 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.180 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:255658 5.4 5:23bbcfe3:::benchmark_data_smithi166_37820_object111236:head [delete] snapc 0=[] ondisk+write+known_if_redirected e463)", 2020-09-08T16:00:27.180 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:55:23.381681+0000", 2020-09-08T16:00:27.181 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 303.50022151000002, 2020-09-08T16:00:27.181 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16221497500000001, 2020-09-08T16:00:27.181 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:27.181 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:00:27.182 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:00:27.182 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T16:00:27.182 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:00:27.182 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 255658 2020-09-08T16:00:27.182 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.183 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:27.183 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.183 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:27.183 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.381681+0000", 2020-09-08T16:00:27.183 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:27.183 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.184 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.184 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:27.184 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.381681+0000", 2020-09-08T16:00:27.184 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999704 2020-09-08T16:00:27.185 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.185 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.185 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:27.185 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.381651+0000", 2020-09-08T16:00:27.185 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.9956e-05 2020-09-08T16:00:27.186 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.186 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.186 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:27.186 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.381711+0000", 2020-09-08T16:00:27.186 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.1560000000000001e-06 2020-09-08T16:00:27.186 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.187 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.187 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:27.187 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.381720+0000", 2020-09-08T16:00:27.187 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0189999999999997e-06 2020-09-08T16:00:27.187 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.188 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.188 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:27.188 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.381725+0000", 2020-09-08T16:00:27.188 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1517999999999997e-05 2020-09-08T16:00:27.188 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.189 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.189 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:27.189 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.381777+0000", 2020-09-08T16:00:27.189 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013760899999999999 2020-09-08T16:00:27.189 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.189 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.190 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:00:27.190 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.381914+0000", 2020-09-08T16:00:27.190 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.5273999999999997e-05 2020-09-08T16:00:27.190 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.191 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.191 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:00:27.191 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.381999+0000", 2020-09-08T16:00:27.191 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.009473436 2020-09-08T16:00:27.191 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.192 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.192 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:00:27.192 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.391473+0000", 2020-09-08T16:00:27.192 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15208806699999999 2020-09-08T16:00:27.192 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.193 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.193 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:00:27.193 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.543561+0000", 2020-09-08T16:00:27.193 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00020859900000000001 2020-09-08T16:00:27.193 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.194 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.194 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:00:27.194 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.543769+0000", 2020-09-08T16:00:27.194 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012600099999999999 2020-09-08T16:00:27.194 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.195 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.195 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:00:27.195 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.543895+0000", 2020-09-08T16:00:27.195 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16221497500000001 2020-09-08T16:00:27.196 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.196 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:27.196 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.196 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.196 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.197 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:255659 5.4 5:2cb8ac1e:::benchmark_data_smithi166_37820_object111237:head [delete] snapc 0=[] ondisk+write+known_if_redirected e463)", 2020-09-08T16:00:27.197 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:55:23.381792+0000", 2020-09-08T16:00:27.197 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 303.50010993299998, 2020-09-08T16:00:27.197 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.162588015, 2020-09-08T16:00:27.197 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:27.198 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:00:27.198 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:00:27.198 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T16:00:27.198 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:00:27.198 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 255659 2020-09-08T16:00:27.199 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.199 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:27.199 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.199 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:27.199 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.381792+0000", 2020-09-08T16:00:27.200 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:27.200 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.200 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.200 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:27.200 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.381792+0000", 2020-09-08T16:00:27.200 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999638 2020-09-08T16:00:27.201 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.201 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.201 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:27.201 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.381756+0000", 2020-09-08T16:00:27.201 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.0875000000000001e-05 2020-09-08T16:00:27.202 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.202 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.202 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:27.202 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.381837+0000", 2020-09-08T16:00:27.202 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0896e-05 2020-09-08T16:00:27.203 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.203 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.203 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:27.203 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.381848+0000", 2020-09-08T16:00:27.203 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.4379999999999997e-06 2020-09-08T16:00:27.204 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.204 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.204 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:27.204 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.381852+0000", 2020-09-08T16:00:27.204 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0044942130000000004 2020-09-08T16:00:27.205 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.205 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.205 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:27.206 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.386346+0000", 2020-09-08T16:00:27.206 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014527799999999999 2020-09-08T16:00:27.206 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.206 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.207 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:00:27.207 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.386492+0000", 2020-09-08T16:00:27.207 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014385299999999999 2020-09-08T16:00:27.207 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.207 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.208 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:00:27.208 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.386636+0000", 2020-09-08T16:00:27.208 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.005128161 2020-09-08T16:00:27.208 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.208 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.209 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:00:27.209 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.391764+0000", 2020-09-08T16:00:27.209 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15227894 2020-09-08T16:00:27.209 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.209 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.210 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:00:27.210 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.544043+0000", 2020-09-08T16:00:27.211 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017755599999999999 2020-09-08T16:00:27.212 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.212 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.212 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:00:27.212 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.544220+0000", 2020-09-08T16:00:27.212 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015986400000000001 2020-09-08T16:00:27.213 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.213 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.213 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:00:27.213 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:23.544380+0000", 2020-09-08T16:00:27.214 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.162588015 2020-09-08T16:00:27.214 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.214 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:27.214 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.214 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.215 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.215 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:286005 5.0 5:0156df2f:::benchmark_data_smithi166_37820_object141583:head [delete] snapc 0=[] ondisk+write+known_if_redirected e469)", 2020-09-08T16:00:27.215 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:55:53.949604+0000", 2020-09-08T16:00:27.215 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 272.932297699, 2020-09-08T16:00:27.216 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.2355044, 2020-09-08T16:00:27.216 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:27.216 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:00:27.216 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:00:27.216 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T16:00:27.217 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:00:27.217 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 286005 2020-09-08T16:00:27.217 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.217 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:27.217 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.218 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:27.218 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.949604+0000", 2020-09-08T16:00:27.218 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:27.218 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.218 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.219 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:27.220 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.949604+0000", 2020-09-08T16:00:27.220 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999466 2020-09-08T16:00:27.220 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.220 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.221 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:27.221 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.949551+0000", 2020-09-08T16:00:27.221 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000120872 2020-09-08T16:00:27.221 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.221 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.222 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:27.222 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.949672+0000", 2020-09-08T16:00:27.222 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.4372999999999999e-05 2020-09-08T16:00:27.222 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.222 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.223 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:27.223 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.949696+0000", 2020-09-08T16:00:27.223 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0710000000000001e-05 2020-09-08T16:00:27.223 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.224 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.224 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:27.224 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.949707+0000", 2020-09-08T16:00:27.224 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.1512000000000003e-05 2020-09-08T16:00:27.224 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.225 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.225 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:27.225 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.949768+0000", 2020-09-08T16:00:27.225 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016761900000000001 2020-09-08T16:00:27.225 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.226 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.226 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:00:27.226 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.949936+0000", 2020-09-08T16:00:27.226 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015923399999999999 2020-09-08T16:00:27.226 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.227 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.227 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T16:00:27.227 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.950095+0000", 2020-09-08T16:00:27.227 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.011716992000000001 2020-09-08T16:00:27.228 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.228 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.230 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:00:27.231 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.961812+0000", 2020-09-08T16:00:27.231 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22315553599999999 2020-09-08T16:00:27.231 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.232 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.232 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:00:27.232 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:54.184968+0000", 2020-09-08T16:00:27.232 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.0550000000000006e-05 2020-09-08T16:00:27.232 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.233 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.233 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:00:27.233 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:54.185048+0000", 2020-09-08T16:00:27.233 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.0631000000000003e-05 2020-09-08T16:00:27.234 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.234 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.234 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:00:27.234 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:54.185109+0000", 2020-09-08T16:00:27.234 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.2355044 2020-09-08T16:00:27.235 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.235 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:27.235 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.235 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.235 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.236 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:286007 5.0 5:037628b5:::benchmark_data_smithi166_37820_object141585:head [delete] snapc 0=[] ondisk+write+known_if_redirected e469)", 2020-09-08T16:00:27.236 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:55:53.950334+0000", 2020-09-08T16:00:27.236 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 272.93156810300002, 2020-09-08T16:00:27.236 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.23495454099999999, 2020-09-08T16:00:27.236 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:27.237 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:00:27.237 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:00:27.237 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T16:00:27.237 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:00:27.238 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 286007 2020-09-08T16:00:27.238 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.238 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:27.238 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.239 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:27.239 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.950334+0000", 2020-09-08T16:00:27.239 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:27.239 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.239 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.240 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:27.240 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.950334+0000", 2020-09-08T16:00:27.240 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999509 2020-09-08T16:00:27.240 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.240 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.241 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:27.241 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.950285+0000", 2020-09-08T16:00:27.241 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000102309 2020-09-08T16:00:27.241 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.242 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.242 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:27.242 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.950387+0000", 2020-09-08T16:00:27.242 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6439e-05 2020-09-08T16:00:27.242 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.243 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.243 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:27.243 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.950404+0000", 2020-09-08T16:00:27.243 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.1749999999999999e-06 2020-09-08T16:00:27.244 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.244 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.244 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:27.244 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.950411+0000", 2020-09-08T16:00:27.244 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0051273660000000004 2020-09-08T16:00:27.245 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.245 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.245 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:27.245 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.955538+0000", 2020-09-08T16:00:27.245 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00020504400000000001 2020-09-08T16:00:27.246 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.246 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.246 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:00:27.246 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.955743+0000", 2020-09-08T16:00:27.246 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015186 2020-09-08T16:00:27.247 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.247 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.247 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T16:00:27.247 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.955895+0000", 2020-09-08T16:00:27.247 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0061448020000000004 2020-09-08T16:00:27.247 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.248 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.248 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:00:27.248 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.962040+0000", 2020-09-08T16:00:27.248 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22311962399999999 2020-09-08T16:00:27.248 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.249 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.249 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:00:27.249 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:54.185160+0000", 2020-09-08T16:00:27.249 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.5420999999999994e-05 2020-09-08T16:00:27.249 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.250 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.250 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:00:27.250 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:54.185245+0000", 2020-09-08T16:00:27.250 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.3472e-05 2020-09-08T16:00:27.250 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.251 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.251 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:00:27.251 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:54.185288+0000", 2020-09-08T16:00:27.251 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.23495454099999999 2020-09-08T16:00:27.251 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.252 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:27.252 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.252 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.253 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.254 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.15771.0:286023 5.0 5:03ada256:::benchmark_data_smithi166_37820_object141601:head [delete] snapc 0=[] ondisk+write+known_if_redirected e469)", 2020-09-08T16:00:27.254 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T15:55:53.969112+0000", 2020-09-08T16:00:27.254 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 272.91278992500003, 2020-09-08T16:00:27.254 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22368803400000001, 2020-09-08T16:00:27.255 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:00:27.255 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:00:27.255 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:00:27.255 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.15771", 2020-09-08T16:00:27.255 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:00:27.256 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 286023 2020-09-08T16:00:27.256 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.256 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:00:27.257 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.257 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:00:27.257 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.969112+0000", 2020-09-08T16:00:27.258 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:00:27.258 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.258 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.258 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:00:27.258 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.969112+0000", 2020-09-08T16:00:27.259 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999747 2020-09-08T16:00:27.259 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.259 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.259 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:00:27.259 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.969087+0000", 2020-09-08T16:00:27.260 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.6360000000000002e-05 2020-09-08T16:00:27.260 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.260 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.260 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:00:27.260 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.969143+0000", 2020-09-08T16:00:27.261 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1854e-05 2020-09-08T16:00:27.261 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.261 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.261 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:00:27.261 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.969155+0000", 2020-09-08T16:00:27.262 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.8800000000000002e-06 2020-09-08T16:00:27.262 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.262 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.262 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:00:27.262 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.969162+0000", 2020-09-08T16:00:27.263 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.5924999999999998e-05 2020-09-08T16:00:27.263 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.263 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.263 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:00:27.263 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.969208+0000", 2020-09-08T16:00:27.263 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012300500000000001 2020-09-08T16:00:27.264 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.264 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.264 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:00:27.264 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.969331+0000", 2020-09-08T16:00:27.264 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000110226 2020-09-08T16:00:27.265 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.265 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.265 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T16:00:27.265 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.969441+0000", 2020-09-08T16:00:27.265 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0050503789999999998 2020-09-08T16:00:27.266 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.266 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.266 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:00:27.266 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:53.974492+0000", 2020-09-08T16:00:27.267 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21824163999999999 2020-09-08T16:00:27.267 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.268 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.268 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:00:27.269 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:54.192733+0000", 2020-09-08T16:00:27.269 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.9861999999999999e-05 2020-09-08T16:00:27.270 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.270 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.270 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:00:27.270 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:54.192773+0000", 2020-09-08T16:00:27.270 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.6976000000000001e-05 2020-09-08T16:00:27.271 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:00:27.271 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:00:27.271 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:00:27.271 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T15:55:54.192800+0000", 2020-09-08T16:00:27.271 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22368803400000001 2020-09-08T16:00:27.272 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.272 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:27.272 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.272 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:00:27.272 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:00:27.273 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:00:27.274 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:27.275 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:00:26 smithi104 bash[22305]: debug 2020-09-08T16:00:26.915+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:27.275 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:27 smithi104 bash[9203]: cluster 2020-09-08T16:00:25.953691+0000 osd.10 (osd.10) 67 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:27.275 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:27 smithi104 bash[9203]: cluster 2020-09-08T16:00:25.953813+0000 osd.10 (osd.10) 68 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:27.275 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:27 smithi104 bash[9203]: cluster 2020-09-08T16:00:25.960895+0000 mgr.x (mgr.24118) 993 : cluster [DBG] pgmap v1460: 1 pgs: 1 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:27.275 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:27 smithi104 bash[9203]: cluster 2020-09-08T16:00:26.133258+0000 mon.a (mon.0) 1860 : cluster [DBG] osdmap e552: 12 total, 12 up, 12 in 2020-09-08T16:00:27.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:27 smithi166 bash[9446]: cluster 2020-09-08T16:00:25.953691+0000 osd.10 (osd.10) 67 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:27.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:27 smithi166 bash[9446]: cluster 2020-09-08T16:00:25.953813+0000 osd.10 (osd.10) 68 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:27.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:27 smithi166 bash[9446]: cluster 2020-09-08T16:00:25.960895+0000 mgr.x (mgr.24118) 993 : cluster [DBG] pgmap v1460: 1 pgs: 1 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:27.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:27 smithi166 bash[9446]: cluster 2020-09-08T16:00:26.133258+0000 mon.a (mon.0) 1860 : cluster [DBG] osdmap e552: 12 total, 12 up, 12 in 2020-09-08T16:00:27.375 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:27.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:27 smithi055 bash[9296]: cluster 2020-09-08T16:00:25.953691+0000 osd.10 (osd.10) 67 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:27.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:27 smithi055 bash[9296]: cluster 2020-09-08T16:00:25.953813+0000 osd.10 (osd.10) 68 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:27.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:27 smithi055 bash[9296]: cluster 2020-09-08T16:00:25.960895+0000 mgr.x (mgr.24118) 993 : cluster [DBG] pgmap v1460: 1 pgs: 1 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:27.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:27 smithi055 bash[9296]: cluster 2020-09-08T16:00:26.133258+0000 mon.a (mon.0) 1860 : cluster [DBG] osdmap e552: 12 total, 12 up, 12 in 2020-09-08T16:00:27.531 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2020-09-08T16:00:27.534 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:27.636 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:27.663 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:00:27.836 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:00:27.865 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:00:27.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:00:27 smithi166 bash[14254]: debug 2020-09-08T16:00:27.531+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:27.888 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:00:27.914 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:00:27.941 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:00:27.969 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:00:27.998 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:00:28.023 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:00:28.054 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:00:28.076 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:28.082 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:00:28.112 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:00:28.137 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:00:28.140 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:00:28.140 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:00:28.140 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:00:28.141 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:00:28.141 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:00:28.141 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:00:28.141 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:00:28.141 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:00:28.142 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:00:28.142 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:00:28.142 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:00:28.142 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:00:28.181 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:28.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:28 smithi055 bash[9296]: audit 2020-09-08T16:00:27.812093+0000 mon.a (mon.0) 1861 : audit [DBG] from='client.? v1:172.21.15.104:0/2555696841' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:00:28.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:28 smithi104 bash[9203]: audit 2020-09-08T16:00:27.812093+0000 mon.a (mon.0) 1861 : audit [DBG] from='client.? v1:172.21.15.104:0/2555696841' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:00:28.427 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:00:28 smithi104 bash[15273]: debug 2020-09-08T16:00:28.079+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:28.552 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:28 smithi166 bash[9446]: audit 2020-09-08T16:00:27.812093+0000 mon.a (mon.0) 1861 : audit [DBG] from='client.? v1:172.21.15.104:0/2555696841' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:00:28.552 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:28.655 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:28.807 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:28.852 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:00:28 smithi166 bash[14254]: debug 2020-09-08T16:00:28.551+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:28.913 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:29.067 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:29.146 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:00:28 smithi055 bash[16646]: debug 2020-09-08T16:00:28.807+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:29.175 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:29.191 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:29 smithi166 bash[9446]: cluster 2020-09-08T16:00:27.961522+0000 mgr.x (mgr.24118) 994 : cluster [DBG] pgmap v1462: 1 pgs: 1 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:29.192 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:00:29 smithi166 bash[18960]: debug 2020-09-08T16:00:29.067+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:29.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:29 smithi055 bash[9296]: cluster 2020-09-08T16:00:27.961522+0000 mgr.x (mgr.24118) 994 : cluster [DBG] pgmap v1462: 1 pgs: 1 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:29.408 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:29 smithi104 bash[9203]: cluster 2020-09-08T16:00:27.961522+0000 mgr.x (mgr.24118) 994 : cluster [DBG] pgmap v1462: 1 pgs: 1 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:29.542 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:29.650 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:00:29.829 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:00:29 smithi166 bash[14254]: debug 2020-09-08T16:00:29.539+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:29.830 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:00:29.935 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:30.087 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:30.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:00:29 smithi166 bash[16593]: debug 2020-09-08T16:00:29.831+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:30.171 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'unique_pool_5' created 2020-09-08T16:00:30.172 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T16:00:30.189 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:30.345 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:30.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:00:30 smithi055 bash[16646]: debug 2020-09-08T16:00:30.079+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:30.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:30 smithi055 bash[9296]: audit 2020-09-08T16:00:29.442633+0000 mon.c (mon.2) 75 : audit [INF] from='client.? v1:172.21.15.104:0/2970742806' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_5", "pg_num": 16}]: dispatch 2020-09-08T16:00:30.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:30 smithi055 bash[9296]: audit 2020-09-08T16:00:29.445483+0000 mon.a (mon.0) 1862 : audit [INF] from='client.36086 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_5", "pg_num": 16}]: dispatch 2020-09-08T16:00:30.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:30 smithi055 bash[9296]: audit 2020-09-08T16:00:29.808065+0000 mon.c (mon.2) 76 : audit [INF] from='client.? v1:172.21.15.104:0/106193101' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:00:30.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:30 smithi055 bash[9296]: audit 2020-09-08T16:00:29.891000+0000 mon.a (mon.0) 1863 : audit [INF] from='client.36092 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:00:30.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:00:30 smithi104 bash[17588]: debug 2020-09-08T16:00:30.343+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:30.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:30 smithi104 bash[9203]: audit 2020-09-08T16:00:29.442633+0000 mon.c (mon.2) 75 : audit [INF] from='client.? v1:172.21.15.104:0/2970742806' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_5", "pg_num": 16}]: dispatch 2020-09-08T16:00:30.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:30 smithi104 bash[9203]: audit 2020-09-08T16:00:29.445483+0000 mon.a (mon.0) 1862 : audit [INF] from='client.36086 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_5", "pg_num": 16}]: dispatch 2020-09-08T16:00:30.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:30 smithi104 bash[9203]: audit 2020-09-08T16:00:29.808065+0000 mon.c (mon.2) 76 : audit [INF] from='client.? v1:172.21.15.104:0/106193101' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:00:30.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:30 smithi104 bash[9203]: audit 2020-09-08T16:00:29.891000+0000 mon.a (mon.0) 1863 : audit [INF] from='client.36092 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:00:30.455 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:30.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:30 smithi166 bash[9446]: audit 2020-09-08T16:00:29.442633+0000 mon.c (mon.2) 75 : audit [INF] from='client.? v1:172.21.15.104:0/2970742806' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_5", "pg_num": 16}]: dispatch 2020-09-08T16:00:30.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:30 smithi166 bash[9446]: audit 2020-09-08T16:00:29.445483+0000 mon.a (mon.0) 1862 : audit [INF] from='client.36086 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_5", "pg_num": 16}]: dispatch 2020-09-08T16:00:30.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:30 smithi166 bash[9446]: audit 2020-09-08T16:00:29.808065+0000 mon.c (mon.2) 76 : audit [INF] from='client.? v1:172.21.15.104:0/106193101' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:00:30.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:30 smithi166 bash[9446]: audit 2020-09-08T16:00:29.891000+0000 mon.a (mon.0) 1863 : audit [INF] from='client.36092 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:00:31.046 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:31.153 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:31.219 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T16:00:31.220 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:00:31.283 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:31 smithi166 bash[9446]: cluster 2020-09-08T16:00:29.961971+0000 mgr.x (mgr.24118) 995 : cluster [DBG] pgmap v1463: 1 pgs: 1 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:31.283 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:31 smithi166 bash[9446]: cluster 2020-09-08T16:00:30.149692+0000 mon.a (mon.0) 1864 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:00:31.283 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:31 smithi166 bash[9446]: audit 2020-09-08T16:00:30.166552+0000 mon.a (mon.0) 1865 : audit [INF] from='client.36086 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_5", "pg_num": 16}]': finished 2020-09-08T16:00:31.283 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:31 smithi166 bash[9446]: audit 2020-09-08T16:00:30.166634+0000 mon.a (mon.0) 1866 : audit [INF] from='client.36092 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:00:31.284 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:31 smithi166 bash[9446]: cluster 2020-09-08T16:00:30.166689+0000 mon.a (mon.0) 1867 : cluster [DBG] osdmap e553: 12 total, 12 up, 12 in 2020-09-08T16:00:31.284 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:31 smithi166 bash[9446]: cluster 2020-09-08T16:00:30.518619+0000 osd.3 (osd.3) 93 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:31.284 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:31 smithi166 bash[9446]: cluster 2020-09-08T16:00:30.518693+0000 osd.3 (osd.3) 94 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:31.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:31 smithi055 bash[9296]: cluster 2020-09-08T16:00:29.961971+0000 mgr.x (mgr.24118) 995 : cluster [DBG] pgmap v1463: 1 pgs: 1 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:31.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:31 smithi055 bash[9296]: cluster 2020-09-08T16:00:30.149692+0000 mon.a (mon.0) 1864 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:00:31.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:31 smithi055 bash[9296]: audit 2020-09-08T16:00:30.166552+0000 mon.a (mon.0) 1865 : audit [INF] from='client.36086 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_5", "pg_num": 16}]': finished 2020-09-08T16:00:31.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:31 smithi055 bash[9296]: audit 2020-09-08T16:00:30.166634+0000 mon.a (mon.0) 1866 : audit [INF] from='client.36092 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:00:31.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:31 smithi055 bash[9296]: cluster 2020-09-08T16:00:30.166689+0000 mon.a (mon.0) 1867 : cluster [DBG] osdmap e553: 12 total, 12 up, 12 in 2020-09-08T16:00:31.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:31 smithi055 bash[9296]: cluster 2020-09-08T16:00:30.518619+0000 osd.3 (osd.3) 93 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:31.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:31 smithi055 bash[9296]: cluster 2020-09-08T16:00:30.518693+0000 osd.3 (osd.3) 94 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:31.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:00:31 smithi104 bash[15273]: debug 2020-09-08T16:00:31.047+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:31.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:31 smithi104 bash[9203]: cluster 2020-09-08T16:00:29.961971+0000 mgr.x (mgr.24118) 995 : cluster [DBG] pgmap v1463: 1 pgs: 1 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:31.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:31 smithi104 bash[9203]: cluster 2020-09-08T16:00:30.149692+0000 mon.a (mon.0) 1864 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:00:31.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:31 smithi104 bash[9203]: audit 2020-09-08T16:00:30.166552+0000 mon.a (mon.0) 1865 : audit [INF] from='client.36086 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_5", "pg_num": 16}]': finished 2020-09-08T16:00:31.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:31 smithi104 bash[9203]: audit 2020-09-08T16:00:30.166634+0000 mon.a (mon.0) 1866 : audit [INF] from='client.36092 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:00:31.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:31 smithi104 bash[9203]: cluster 2020-09-08T16:00:30.166689+0000 mon.a (mon.0) 1867 : cluster [DBG] osdmap e553: 12 total, 12 up, 12 in 2020-09-08T16:00:31.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:31 smithi104 bash[9203]: cluster 2020-09-08T16:00:30.518619+0000 osd.3 (osd.3) 93 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:31.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:31 smithi104 bash[9203]: cluster 2020-09-08T16:00:30.518693+0000 osd.3 (osd.3) 94 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:31.455 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:00:31.544 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:31.559 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool application enable unique_pool_5 rados --yes-i-really-mean-it || true 2020-09-08T16:00:31.592 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:00:31.651 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:00:31.726 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:00:31.900 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2020-09-08T16:00:31.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:00:31 smithi104 bash[15273]: debug 2020-09-08T16:00:31.547+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:32.029 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:00:32.241 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:00:32.346 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:32.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:00:32 smithi104 bash[19922]: debug 2020-09-08T16:00:32.243+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:32.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:32 smithi104 bash[9203]: cluster 2020-09-08T16:00:31.169855+0000 mon.a (mon.0) 1868 : cluster [DBG] osdmap e554: 12 total, 12 up, 12 in 2020-09-08T16:00:32.506 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:32 smithi166 bash[9446]: cluster 2020-09-08T16:00:31.169855+0000 mon.a (mon.0) 1868 : cluster [DBG] osdmap e554: 12 total, 12 up, 12 in 2020-09-08T16:00:32.507 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:32.613 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:00:32.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:32 smithi055 bash[9296]: cluster 2020-09-08T16:00:31.169855+0000 mon.a (mon.0) 1868 : cluster [DBG] osdmap e554: 12 total, 12 up, 12 in 2020-09-08T16:00:32.865 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:00:32 smithi166 bash[14254]: debug 2020-09-08T16:00:32.507+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:32.865 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:00:32.978 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:33.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:00:32 smithi166 bash[16593]: debug 2020-09-08T16:00:32.867+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:33.154 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:33.262 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:00:33.386 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:00:33.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:33 smithi104 bash[9203]: cluster 2020-09-08T16:00:31.962712+0000 mgr.x (mgr.24118) 996 : cluster [DBG] pgmap v1466: 17 pgs: 4 creating+peering, 11 unknown, 2 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:33.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:33 smithi104 bash[9203]: cluster 2020-09-08T16:00:32.178926+0000 mon.a (mon.0) 1869 : cluster [DBG] osdmap e555: 12 total, 12 up, 12 in 2020-09-08T16:00:33.441 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:33 smithi166 bash[9446]: cluster 2020-09-08T16:00:31.962712+0000 mgr.x (mgr.24118) 996 : cluster [DBG] pgmap v1466: 17 pgs: 4 creating+peering, 11 unknown, 2 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:33.442 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:33 smithi166 bash[9446]: cluster 2020-09-08T16:00:32.178926+0000 mon.a (mon.0) 1869 : cluster [DBG] osdmap e555: 12 total, 12 up, 12 in 2020-09-08T16:00:33.442 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:00:33 smithi166 bash[11928]: debug 2020-09-08T16:00:33.155+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:33.489 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:33.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:00:33 smithi055 bash[19003]: debug 2020-09-08T16:00:33.383+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:33.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:33 smithi055 bash[9296]: cluster 2020-09-08T16:00:31.962712+0000 mgr.x (mgr.24118) 996 : cluster [DBG] pgmap v1466: 17 pgs: 4 creating+peering, 11 unknown, 2 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:33.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:33 smithi055 bash[9296]: cluster 2020-09-08T16:00:32.178926+0000 mon.a (mon.0) 1869 : cluster [DBG] osdmap e555: 12 total, 12 up, 12 in 2020-09-08T16:00:33.668 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:33.686 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T16:00:33.774 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:33.802 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:00:33 smithi104 bash[15273]: debug 2020-09-08T16:00:33.671+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:33.830 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:00:34.032 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:34.117 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:00:34 smithi166 bash[18960]: debug 2020-09-08T16:00:34.031+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:34.138 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:00:34.488 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:00:34.601 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:34.751 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:34.856 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:34.869 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:00:34 smithi055 bash[16646]: debug 2020-09-08T16:00:34.751+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:34.942 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:00:34 smithi104 bash[19922]: debug 2020-09-08T16:00:34.491+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:34.980 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:35.092 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:35.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:00:34 smithi055 bash[16646]: debug 2020-09-08T16:00:34.983+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:35.251 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:35.318 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T16:00:35.318 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:00:35.356 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:35.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:35 smithi166 bash[9446]: cluster 2020-09-08T16:00:33.963463+0000 mgr.x (mgr.24118) 997 : cluster [DBG] pgmap v1468: 17 pgs: 4 creating+peering, 13 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:35.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:00:35 smithi166 bash[18960]: debug 2020-09-08T16:00:35.251+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:35.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:35 smithi104 bash[9203]: cluster 2020-09-08T16:00:33.963463+0000 mgr.x (mgr.24118) 997 : cluster [DBG] pgmap v1468: 17 pgs: 4 creating+peering, 13 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:35.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:35 smithi055 bash[9296]: cluster 2020-09-08T16:00:33.963463+0000 mgr.x (mgr.24118) 997 : cluster [DBG] pgmap v1468: 17 pgs: 4 creating+peering, 13 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:35.717 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:35.823 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:00:35.943 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:00:36.049 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:36.067 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2020-09-08T16:00:36.156 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:00:35 smithi055 bash[14301]: debug 2020-09-08T16:00:35.943+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:36.177 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:00:35 smithi104 bash[17588]: debug 2020-09-08T16:00:35.719+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:36.182 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:36.196 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:00:36.289 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:00:36.612 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:00:36.662 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:00:36 smithi055 bash[11956]: debug 2020-09-08T16:00:36.183+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:36.719 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:36.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:00:36 smithi104 bash[19922]: debug 2020-09-08T16:00:36.615+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:37.321 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:37 smithi104 bash[9203]: cluster 2020-09-08T16:00:35.964099+0000 mgr.x (mgr.24118) 998 : cluster [DBG] pgmap v1469: 17 pgs: 17 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:37.322 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:37.376 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:37 smithi166 bash[9446]: cluster 2020-09-08T16:00:35.964099+0000 mgr.x (mgr.24118) 998 : cluster [DBG] pgmap v1469: 17 pgs: 17 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:37.427 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:37.588 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:37.649 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:00:37 smithi104 bash[22305]: debug 2020-09-08T16:00:37.323+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:37.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:37 smithi055 bash[9296]: cluster 2020-09-08T16:00:35.964099+0000 mgr.x (mgr.24118) 998 : cluster [DBG] pgmap v1469: 17 pgs: 17 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:37.693 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:00:37.869 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:00:37 smithi166 bash[11928]: debug 2020-09-08T16:00:37.587+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:38.207 INFO:teuthology.orchestra.run.smithi104.stderr:enabled application 'rados' on pool 'unique_pool_5' 2020-09-08T16:00:38.308 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:00:38.413 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:38.546 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:38 smithi055 bash[9296]: cluster 2020-09-08T16:00:36.738694+0000 osd.8 (osd.8) 75 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:38.546 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:38 smithi055 bash[9296]: cluster 2020-09-08T16:00:36.738839+0000 osd.8 (osd.8) 76 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:38.546 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:38 smithi055 bash[9296]: cluster 2020-09-08T16:00:37.198676+0000 mon.a (mon.0) 1870 : cluster [DBG] osdmap e556: 12 total, 12 up, 12 in 2020-09-08T16:00:38.547 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:38 smithi055 bash[9296]: audit 2020-09-08T16:00:38.056448+0000 mon.a (mon.0) 1871 : audit [INF] from='client.? v1:172.21.15.104:0/1996522718' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_5", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T16:00:38.547 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:38.592 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:00:38 smithi104 bash[19922]: debug 2020-09-08T16:00:38.311+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:38.593 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:38 smithi104 bash[9203]: cluster 2020-09-08T16:00:36.738694+0000 osd.8 (osd.8) 75 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:38.593 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:38 smithi104 bash[9203]: cluster 2020-09-08T16:00:36.738839+0000 osd.8 (osd.8) 76 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:38.593 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:38 smithi104 bash[9203]: cluster 2020-09-08T16:00:37.198676+0000 mon.a (mon.0) 1870 : cluster [DBG] osdmap e556: 12 total, 12 up, 12 in 2020-09-08T16:00:38.594 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:38 smithi104 bash[9203]: audit 2020-09-08T16:00:38.056448+0000 mon.a (mon.0) 1871 : audit [INF] from='client.? v1:172.21.15.104:0/1996522718' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_5", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T16:00:38.624 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:38 smithi166 bash[9446]: cluster 2020-09-08T16:00:36.738694+0000 osd.8 (osd.8) 75 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:38.624 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:38 smithi166 bash[9446]: cluster 2020-09-08T16:00:36.738839+0000 osd.8 (osd.8) 76 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:38.625 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:38 smithi166 bash[9446]: cluster 2020-09-08T16:00:37.198676+0000 mon.a (mon.0) 1870 : cluster [DBG] osdmap e556: 12 total, 12 up, 12 in 2020-09-08T16:00:38.625 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:38 smithi166 bash[9446]: audit 2020-09-08T16:00:38.056448+0000 mon.a (mon.0) 1871 : audit [INF] from='client.? v1:172.21.15.104:0/1996522718' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_5", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T16:00:38.658 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:00:38.782 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:00:38.888 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:00:38.902 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:00:38 smithi055 bash[14301]: debug 2020-09-08T16:00:38.783+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:38.902 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:00:38 smithi055 bash[16646]: debug 2020-09-08T16:00:38.547+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:39.150 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:00:39.174 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:00:39.195 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:00:39.228 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:00:39.256 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:00:39.258 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:00:39.285 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:00:39.313 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:00:39.340 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:00:39.356 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T16:00:39.356 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:00:39.364 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:00:39.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:39 smithi166 bash[9446]: cluster 2020-09-08T16:00:37.964677+0000 mgr.x (mgr.24118) 999 : cluster [DBG] pgmap v1471: 17 pgs: 17 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:39.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:39 smithi166 bash[9446]: audit 2020-09-08T16:00:38.208189+0000 mon.a (mon.0) 1872 : audit [INF] from='client.? v1:172.21.15.104:0/1996522718' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_5", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T16:00:39.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:39 smithi166 bash[9446]: cluster 2020-09-08T16:00:38.208268+0000 mon.a (mon.0) 1873 : cluster [DBG] osdmap e557: 12 total, 12 up, 12 in 2020-09-08T16:00:39.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:39 smithi166 bash[9446]: audit 2020-09-08T16:00:39.126986+0000 mon.a (mon.0) 1874 : audit [DBG] from='client.? v1:172.21.15.104:0/1572965007' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:00:39.375 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:00:39.401 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:00:39.429 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:00:39.458 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:00:39.460 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:00:39.461 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:00:39.461 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:00:39.461 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:00:39.461 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:00:39.461 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:00:39.462 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:00:39.462 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:00:39.462 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:00:39.462 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:00:39.463 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:00:39.463 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:00:39.537 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:39 smithi104 bash[9203]: cluster 2020-09-08T16:00:37.964677+0000 mgr.x (mgr.24118) 999 : cluster [DBG] pgmap v1471: 17 pgs: 17 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:39.537 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:39 smithi104 bash[9203]: audit 2020-09-08T16:00:38.208189+0000 mon.a (mon.0) 1872 : audit [INF] from='client.? v1:172.21.15.104:0/1996522718' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_5", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T16:00:39.538 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:39 smithi104 bash[9203]: cluster 2020-09-08T16:00:38.208268+0000 mon.a (mon.0) 1873 : cluster [DBG] osdmap e557: 12 total, 12 up, 12 in 2020-09-08T16:00:39.538 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:39 smithi104 bash[9203]: audit 2020-09-08T16:00:39.126986+0000 mon.a (mon.0) 1874 : audit [DBG] from='client.? v1:172.21.15.104:0/1572965007' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:00:39.538 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:00:39.643 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:39.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:00:39 smithi055 bash[19003]: debug 2020-09-08T16:00:39.259+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:39.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:39 smithi055 bash[9296]: cluster 2020-09-08T16:00:37.964677+0000 mgr.x (mgr.24118) 999 : cluster [DBG] pgmap v1471: 17 pgs: 17 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:39.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:39 smithi055 bash[9296]: audit 2020-09-08T16:00:38.208189+0000 mon.a (mon.0) 1872 : audit [INF] from='client.? v1:172.21.15.104:0/1996522718' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_5", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T16:00:39.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:39 smithi055 bash[9296]: cluster 2020-09-08T16:00:38.208268+0000 mon.a (mon.0) 1873 : cluster [DBG] osdmap e557: 12 total, 12 up, 12 in 2020-09-08T16:00:39.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:39 smithi055 bash[9296]: audit 2020-09-08T16:00:39.126986+0000 mon.a (mon.0) 1874 : audit [DBG] from='client.? v1:172.21.15.104:0/1572965007' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:00:39.867 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:00:39 smithi166 bash[16593]: debug 2020-09-08T16:00:39.539+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:39.887 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:39.991 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:40.156 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:40.177 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:00:39 smithi104 bash[17588]: debug 2020-09-08T16:00:39.891+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:40.188 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2020-09-08T16:00:40.261 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:40.324 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:00:40.359 INFO:teuthology.orchestra.run.smithi166:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.2 -b 65536 --object-size 65536 -p unique_pool_5 bench 90 write' 2020-09-08T16:00:40.363 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2020-09-08T16:00:40.364 INFO:teuthology.orchestra.run:waiting for 3000 2020-09-08T16:00:40.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:00:40 smithi166 bash[18960]: debug 2020-09-08T16:00:40.155+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:40.405 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:hints = 1 2020-09-08T16:00:40.405 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2020-09-08T16:00:40.405 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Object prefix: benchmark_data_smithi166_48533 2020-09-08T16:00:40.406 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T16:00:40.406 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 0 0 0 0 0 0 - 0 2020-09-08T16:00:40.415 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:40.520 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:40.529 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:00:40 smithi055 bash[11956]: debug 2020-09-08T16:00:40.411+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:40.638 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:40.743 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:40.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:00:40 smithi055 bash[11956]: debug 2020-09-08T16:00:40.639+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:40.948 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:41.051 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:41.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:00:40 smithi104 bash[17588]: debug 2020-09-08T16:00:40.951+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:41.180 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:40 smithi104 bash[9203]: cluster 2020-09-08T16:00:39.965338+0000 mgr.x (mgr.24118) 1000 : cluster [DBG] pgmap v1473: 17 pgs: 17 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:41.198 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:40 smithi166 bash[9446]: cluster 2020-09-08T16:00:39.965338+0000 mgr.x (mgr.24118) 1000 : cluster [DBG] pgmap v1473: 17 pgs: 17 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:41.199 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:00:41 smithi166 bash[14254]: debug 2020-09-08T16:00:41.199+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:41.199 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:41.313 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:41.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:40 smithi055 bash[9296]: cluster 2020-09-08T16:00:39.965338+0000 mgr.x (mgr.24118) 1000 : cluster [DBG] pgmap v1473: 17 pgs: 17 active+clean; 0 B data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:00:41.407 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 1 16 3806 3790 236.862 236.875 0.00261249 0.00421212 2020-09-08T16:00:41.449 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:41.450 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:00:41 smithi166 bash[14254]: debug 2020-09-08T16:00:41.447+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:41.552 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:41.958 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:42.063 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:42.202 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:42.307 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:00:42.409 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 2 16 7542 7526 235.16 233.5 0.00711793 0.00423967 2020-09-08T16:00:42.409 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:00:42 smithi055 bash[16646]: debug 2020-09-08T16:00:42.203+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:42.430 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:00:42.536 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:42.617 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:00:42 smithi166 bash[16593]: debug 2020-09-08T16:00:42.431+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:42.657 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:42.734 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:00:41 smithi104 bash[17588]: debug 2020-09-08T16:00:41.959+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:42.761 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:42.866 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:42.890 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:00:42.970 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:00:43.024 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:00:42 smithi166 bash[11928]: debug 2020-09-08T16:00:42.655+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:43.025 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T16:00:43.091 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:43 smithi055 bash[9296]: cluster 2020-09-08T16:00:41.965900+0000 mgr.x (mgr.24118) 1001 : cluster [DBG] pgmap v1474: 17 pgs: 17 active+clean; 47 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s wr, 93 op/s 2020-09-08T16:00:43.092 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:43 smithi055 bash[9296]: audit 2020-09-08T16:00:42.990771+0000 mon.a (mon.0) 1875 : audit [INF] from='client.26134 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:00:43.092 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:43 smithi055 bash[9296]: audit 2020-09-08T16:00:42.990831+0000 mon.b (mon.1) 110 : audit [INF] from='client.? v1:172.21.15.104:0/2712167756' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:00:43.092 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:00:42 smithi055 bash[11956]: debug 2020-09-08T16:00:42.867+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:43.093 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:00:43.118 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:00:43.198 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:43.207 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:43 smithi104 bash[9203]: cluster 2020-09-08T16:00:41.965900+0000 mgr.x (mgr.24118) 1001 : cluster [DBG] pgmap v1474: 17 pgs: 17 active+clean; 47 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s wr, 93 op/s 2020-09-08T16:00:43.208 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:43 smithi104 bash[9203]: audit 2020-09-08T16:00:42.990771+0000 mon.a (mon.0) 1875 : audit [INF] from='client.26134 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:00:43.208 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:43 smithi104 bash[9203]: audit 2020-09-08T16:00:42.990831+0000 mon.b (mon.1) 110 : audit [INF] from='client.? v1:172.21.15.104:0/2712167756' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:00:43.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:43 smithi166 bash[9446]: cluster 2020-09-08T16:00:41.965900+0000 mgr.x (mgr.24118) 1001 : cluster [DBG] pgmap v1474: 17 pgs: 17 active+clean; 47 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s wr, 93 op/s 2020-09-08T16:00:43.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:43 smithi166 bash[9446]: audit 2020-09-08T16:00:42.990771+0000 mon.a (mon.0) 1875 : audit [INF] from='client.26134 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:00:43.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:43 smithi166 bash[9446]: audit 2020-09-08T16:00:42.990831+0000 mon.b (mon.1) 110 : audit [INF] from='client.? v1:172.21.15.104:0/2712167756' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:00:43.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:00:43 smithi055 bash[19003]: debug 2020-09-08T16:00:43.091+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:43.407 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 3 16 10139 10123 210.87 162.312 0.00179575 0.00443262 2020-09-08T16:00:44.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:44 smithi166 bash[9446]: cluster 2020-09-08T16:00:43.018522+0000 mon.a (mon.0) 1876 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:00:44.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:44 smithi166 bash[9446]: audit 2020-09-08T16:00:43.023270+0000 mon.a (mon.0) 1877 : audit [INF] from='client.26134 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:00:44.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:44 smithi166 bash[9446]: cluster 2020-09-08T16:00:43.023323+0000 mon.a (mon.0) 1878 : cluster [DBG] osdmap e558: 12 total, 12 up, 12 in 2020-09-08T16:00:44.417 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 4 15 12770 12755 199.273 164.5 0.00533809 0.00501323 2020-09-08T16:00:44.418 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:44 smithi055 bash[9296]: cluster 2020-09-08T16:00:43.018522+0000 mon.a (mon.0) 1876 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:00:44.418 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:44 smithi055 bash[9296]: audit 2020-09-08T16:00:43.023270+0000 mon.a (mon.0) 1877 : audit [INF] from='client.26134 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:00:44.419 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:44 smithi055 bash[9296]: cluster 2020-09-08T16:00:43.023323+0000 mon.a (mon.0) 1878 : cluster [DBG] osdmap e558: 12 total, 12 up, 12 in 2020-09-08T16:00:44.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:44 smithi104 bash[9203]: cluster 2020-09-08T16:00:43.018522+0000 mon.a (mon.0) 1876 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:00:44.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:44 smithi104 bash[9203]: audit 2020-09-08T16:00:43.023270+0000 mon.a (mon.0) 1877 : audit [INF] from='client.26134 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:00:44.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:44 smithi104 bash[9203]: cluster 2020-09-08T16:00:43.023323+0000 mon.a (mon.0) 1878 : cluster [DBG] osdmap e558: 12 total, 12 up, 12 in 2020-09-08T16:00:45.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:45 smithi166 bash[9446]: cluster 2020-09-08T16:00:43.966440+0000 mgr.x (mgr.24118) 1002 : cluster [DBG] pgmap v1476: 17 pgs: 17 active+clean; 374 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s wr, 884 op/s 2020-09-08T16:00:45.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:45 smithi055 bash[9296]: cluster 2020-09-08T16:00:43.966440+0000 mgr.x (mgr.24118) 1002 : cluster [DBG] pgmap v1476: 17 pgs: 17 active+clean; 374 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s wr, 884 op/s 2020-09-08T16:00:45.407 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 5 16 15735 15719 196.466 185.25 0.00725388 0.00508547 2020-09-08T16:00:45.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:45 smithi104 bash[9203]: cluster 2020-09-08T16:00:43.966440+0000 mgr.x (mgr.24118) 1002 : cluster [DBG] pgmap v1476: 17 pgs: 17 active+clean; 374 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s wr, 884 op/s 2020-09-08T16:00:46.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:46 smithi166 bash[9446]: cluster 2020-09-08T16:00:44.324559+0000 osd.1 (osd.1) 112 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:46.503 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:46 smithi166 bash[9446]: cluster 2020-09-08T16:00:44.324659+0000 osd.1 (osd.1) 113 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:46.503 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:46 smithi166 bash[9446]: cluster 2020-09-08T16:00:45.037976+0000 mon.a (mon.0) 1879 : cluster [DBG] osdmap e559: 12 total, 12 up, 12 in 2020-09-08T16:00:46.505 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:46 smithi104 bash[9203]: cluster 2020-09-08T16:00:44.324559+0000 osd.1 (osd.1) 112 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:46.506 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:46 smithi104 bash[9203]: cluster 2020-09-08T16:00:44.324659+0000 osd.1 (osd.1) 113 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:46.506 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:46 smithi104 bash[9203]: cluster 2020-09-08T16:00:45.037976+0000 mon.a (mon.0) 1879 : cluster [DBG] osdmap e559: 12 total, 12 up, 12 in 2020-09-08T16:00:46.514 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:46 smithi055 bash[9296]: cluster 2020-09-08T16:00:44.324559+0000 osd.1 (osd.1) 112 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:46.515 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:46 smithi055 bash[9296]: cluster 2020-09-08T16:00:44.324659+0000 osd.1 (osd.1) 113 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:46.515 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:46 smithi055 bash[9296]: cluster 2020-09-08T16:00:45.037976+0000 mon.a (mon.0) 1879 : cluster [DBG] osdmap e559: 12 total, 12 up, 12 in 2020-09-08T16:00:46.515 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 6 15 19091 19076 198.687 209.812 0.00519741 0.00502933 2020-09-08T16:00:46.794 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:46.900 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:47.047 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:00:46 smithi104 bash[15273]: debug 2020-09-08T16:00:46.795+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:47.199 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:00:47.314 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:47.328 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:47 smithi166 bash[9446]: cluster 2020-09-08T16:00:45.966930+0000 mgr.x (mgr.24118) 1003 : cluster [DBG] pgmap v1478: 17 pgs: 17 active+clean; 616 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 79 MiB/s wr, 1.27k op/s 2020-09-08T16:00:47.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:47 smithi055 bash[9296]: cluster 2020-09-08T16:00:45.966930+0000 mgr.x (mgr.24118) 1003 : cluster [DBG] pgmap v1478: 17 pgs: 17 active+clean; 616 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 79 MiB/s wr, 1.27k op/s 2020-09-08T16:00:47.407 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 7 16 21894 21878 195.318 175.125 0.0023842 0.00506227 2020-09-08T16:00:47.464 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:47 smithi104 bash[9203]: cluster 2020-09-08T16:00:45.966930+0000 mgr.x (mgr.24118) 1003 : cluster [DBG] pgmap v1478: 17 pgs: 17 active+clean; 616 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 79 MiB/s wr, 1.27k op/s 2020-09-08T16:00:47.465 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:00:47 smithi104 bash[17588]: debug 2020-09-08T16:00:47.203+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:48.290 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:48.375 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:00:48 smithi166 bash[18960]: debug 2020-09-08T16:00:48.291+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:48.375 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:48 smithi166 bash[9446]: cluster 2020-09-08T16:00:47.379734+0000 osd.6 (osd.6) 87 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:48.376 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:48 smithi166 bash[9446]: cluster 2020-09-08T16:00:47.379851+0000 osd.6 (osd.6) 88 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:48.395 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:48.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:48 smithi055 bash[9296]: cluster 2020-09-08T16:00:47.379734+0000 osd.6 (osd.6) 87 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:48.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:48 smithi055 bash[9296]: cluster 2020-09-08T16:00:47.379851+0000 osd.6 (osd.6) 88 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:48.407 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 8 16 24213 24197 189.018 144.938 0.00577979 0.00518568 2020-09-08T16:00:48.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:48 smithi104 bash[9203]: cluster 2020-09-08T16:00:47.379734+0000 osd.6 (osd.6) 87 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:48.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:48 smithi104 bash[9203]: cluster 2020-09-08T16:00:47.379851+0000 osd.6 (osd.6) 88 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:48.536 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:48.641 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:00:48.758 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:00:48.873 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:48.882 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:00:48 smithi055 bash[11956]: debug 2020-09-08T16:00:48.535+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:48.997 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:49.056 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:49 smithi166 bash[9446]: cluster 2020-09-08T16:00:47.967378+0000 mgr.x (mgr.24118) 1004 : cluster [DBG] pgmap v1479: 17 pgs: 17 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 148 MiB/s wr, 2.36k op/s 2020-09-08T16:00:49.057 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:49 smithi166 bash[9446]: cluster 2020-09-08T16:00:48.062749+0000 mon.a (mon.0) 1880 : cluster [DBG] osdmap e560: 12 total, 12 up, 12 in 2020-09-08T16:00:49.057 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:00:48 smithi166 bash[16593]: debug 2020-09-08T16:00:48.755+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:49.097 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:49.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:00:48 smithi055 bash[16646]: debug 2020-09-08T16:00:48.991+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:49.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:49 smithi055 bash[9296]: cluster 2020-09-08T16:00:47.967378+0000 mgr.x (mgr.24118) 1004 : cluster [DBG] pgmap v1479: 17 pgs: 17 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 148 MiB/s wr, 2.36k op/s 2020-09-08T16:00:49.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:49 smithi055 bash[9296]: cluster 2020-09-08T16:00:48.062749+0000 mon.a (mon.0) 1880 : cluster [DBG] osdmap e560: 12 total, 12 up, 12 in 2020-09-08T16:00:49.207 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:00:49.310 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:49.369 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:00:49 smithi166 bash[18960]: debug 2020-09-08T16:00:49.207+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:49.390 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:49 smithi104 bash[9203]: cluster 2020-09-08T16:00:47.967378+0000 mgr.x (mgr.24118) 1004 : cluster [DBG] pgmap v1479: 17 pgs: 17 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 148 MiB/s wr, 2.36k op/s 2020-09-08T16:00:49.391 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:49 smithi104 bash[9203]: cluster 2020-09-08T16:00:48.062749+0000 mon.a (mon.0) 1880 : cluster [DBG] osdmap e560: 12 total, 12 up, 12 in 2020-09-08T16:00:49.410 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 9 16 26908 26892 186.729 168.438 0.00514975 0.00535205 2020-09-08T16:00:50.271 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:50.375 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:50.407 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 10 16 29194 29178 182.343 142.875 0.0052963 0.00548094 2020-09-08T16:00:50.503 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:50.613 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:50.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:00:50 smithi055 bash[16646]: debug 2020-09-08T16:00:50.503+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:50.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:00:50 smithi104 bash[22305]: debug 2020-09-08T16:00:50.275+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:50.742 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:50.848 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:00:50.910 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:50 smithi166 bash[9446]: cluster 2020-09-08T16:00:49.967880+0000 mgr.x (mgr.24118) 1005 : cluster [DBG] pgmap v1481: 17 pgs: 17 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 143 MiB/s wr, 2.29k op/s 2020-09-08T16:00:50.910 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:00:50 smithi166 bash[14254]: debug 2020-09-08T16:00:50.743+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:50.962 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:00:51.065 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:51.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:50 smithi055 bash[9296]: cluster 2020-09-08T16:00:49.967880+0000 mgr.x (mgr.24118) 1005 : cluster [DBG] pgmap v1481: 17 pgs: 17 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 143 MiB/s wr, 2.29k op/s 2020-09-08T16:00:51.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:00:50 smithi055 bash[14301]: debug 2020-09-08T16:00:50.963+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:51.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:50 smithi104 bash[9203]: cluster 2020-09-08T16:00:49.967880+0000 mgr.x (mgr.24118) 1005 : cluster [DBG] pgmap v1481: 17 pgs: 17 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 143 MiB/s wr, 2.29k op/s 2020-09-08T16:00:51.407 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 11 16 31423 31407 178.43 139.312 0.00460225 0.00560185 2020-09-08T16:00:51.684 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T16:00:52.047 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:00:52.408 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 12 16 33672 33656 175.274 140.562 0.00672928 0.00570292 2020-09-08T16:00:52.617 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:52.884 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:52.960 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:00:52 smithi104 bash[15273]: debug 2020-09-08T16:00:52.615+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:53.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:53 smithi166 bash[9446]: cluster 2020-09-08T16:00:51.968401+0000 mgr.x (mgr.24118) 1006 : cluster [DBG] pgmap v1482: 17 pgs: 17 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 147 MiB/s wr, 2.35k op/s 2020-09-08T16:00:53.375 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:53 smithi166 bash[9446]: cluster 2020-09-08T16:00:52.035749+0000 osd.11 (osd.11) 92 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:53.375 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:53 smithi166 bash[9446]: cluster 2020-09-08T16:00:52.035873+0000 osd.11 (osd.11) 93 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:53.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:53 smithi055 bash[9296]: cluster 2020-09-08T16:00:51.968401+0000 mgr.x (mgr.24118) 1006 : cluster [DBG] pgmap v1482: 17 pgs: 17 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 147 MiB/s wr, 2.35k op/s 2020-09-08T16:00:53.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:53 smithi055 bash[9296]: cluster 2020-09-08T16:00:52.035749+0000 osd.11 (osd.11) 92 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:53.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:53 smithi055 bash[9296]: cluster 2020-09-08T16:00:52.035873+0000 osd.11 (osd.11) 93 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:53.407 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 13 16 36206 36190 173.973 158.375 0.00288809 0.00574575 2020-09-08T16:00:53.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:53 smithi104 bash[9203]: cluster 2020-09-08T16:00:51.968401+0000 mgr.x (mgr.24118) 1006 : cluster [DBG] pgmap v1482: 17 pgs: 17 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 147 MiB/s wr, 2.35k op/s 2020-09-08T16:00:53.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:53 smithi104 bash[9203]: cluster 2020-09-08T16:00:52.035749+0000 osd.11 (osd.11) 92 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:53.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:53 smithi104 bash[9203]: cluster 2020-09-08T16:00:52.035873+0000 osd.11 (osd.11) 93 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:54.093 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:54.202 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:54.340 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:54 smithi166 bash[9446]: cluster 2020-09-08T16:00:53.033639+0000 mon.a (mon.0) 1881 : cluster [DBG] osdmap e561: 12 total, 12 up, 12 in 2020-09-08T16:00:54.340 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:54.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:54 smithi055 bash[9296]: cluster 2020-09-08T16:00:53.033639+0000 mon.a (mon.0) 1881 : cluster [DBG] osdmap e561: 12 total, 12 up, 12 in 2020-09-08T16:00:54.407 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 14 16 38714 38698 172.742 156.75 0.0130545 0.00578602 2020-09-08T16:00:54.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:00:54 smithi104 bash[22305]: debug 2020-09-08T16:00:54.095+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:54.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:54 smithi104 bash[9203]: cluster 2020-09-08T16:00:53.033639+0000 mon.a (mon.0) 1881 : cluster [DBG] osdmap e561: 12 total, 12 up, 12 in 2020-09-08T16:00:54.446 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:54.567 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:54.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:00:54 smithi166 bash[11928]: debug 2020-09-08T16:00:54.567+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:54.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:00:54 smithi166 bash[14254]: debug 2020-09-08T16:00:54.339+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:54.671 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:54.806 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:54.910 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:55.032 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:00:54 smithi055 bash[11956]: debug 2020-09-08T16:00:54.802+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:55.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:55 smithi166 bash[9446]: cluster 2020-09-08T16:00:53.968828+0000 mgr.x (mgr.24118) 1007 : cluster [DBG] pgmap v1484: 17 pgs: 17 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 179 MiB/s wr, 2.86k op/s 2020-09-08T16:00:55.411 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 15 16 41236 41220 171.733 157.625 0.00323966 0.00582042 2020-09-08T16:00:55.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:55 smithi055 bash[9296]: cluster 2020-09-08T16:00:53.968828+0000 mgr.x (mgr.24118) 1007 : cluster [DBG] pgmap v1484: 17 pgs: 17 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 179 MiB/s wr, 2.86k op/s 2020-09-08T16:00:55.417 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:55 smithi104 bash[9203]: cluster 2020-09-08T16:00:53.968828+0000 mgr.x (mgr.24118) 1007 : cluster [DBG] pgmap v1484: 17 pgs: 17 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 179 MiB/s wr, 2.86k op/s 2020-09-08T16:00:56.408 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 16 16 43085 43069 168.222 115.562 0.00227859 0.00584497 2020-09-08T16:00:57.274 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:00:57.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:57 smithi166 bash[9446]: cluster 2020-09-08T16:00:55.969319+0000 mgr.x (mgr.24118) 1008 : cluster [DBG] pgmap v1485: 17 pgs: 17 active+clean; 2.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 131 MiB/s wr, 2.10k op/s 2020-09-08T16:00:57.377 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:00:57.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:57 smithi055 bash[9296]: cluster 2020-09-08T16:00:55.969319+0000 mgr.x (mgr.24118) 1008 : cluster [DBG] pgmap v1485: 17 pgs: 17 active+clean; 2.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 131 MiB/s wr, 2.10k op/s 2020-09-08T16:00:57.408 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 17 16 45414 45398 166.888 145.562 0.00572315 0.0059892 2020-09-08T16:00:57.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:00:57 smithi104 bash[15273]: debug 2020-09-08T16:00:57.275+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:57.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:57 smithi104 bash[9203]: cluster 2020-09-08T16:00:55.969319+0000 mgr.x (mgr.24118) 1008 : cluster [DBG] pgmap v1485: 17 pgs: 17 active+clean; 2.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 131 MiB/s wr, 2.10k op/s 2020-09-08T16:00:57.518 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:00:57.622 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:57.733 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:00:57 smithi166 bash[16593]: debug 2020-09-08T16:00:57.519+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:57.736 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:00:57.841 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:58.000 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:00:58.051 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:00:57 smithi166 bash[14254]: debug 2020-09-08T16:00:57.735+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:58.109 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:00:58.125 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:00:58 smithi055 bash[16646]: debug 2020-09-08T16:00:58.002+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:58.126 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:58 smithi055 bash[9296]: cluster 2020-09-08T16:00:56.383009+0000 osd.4 (osd.4) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:58.126 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:58 smithi055 bash[9296]: cluster 2020-09-08T16:00:56.383134+0000 osd.4 (osd.4) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:58.126 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:58 smithi055 bash[9296]: cluster 2020-09-08T16:00:57.054412+0000 mon.a (mon.0) 1882 : cluster [DBG] osdmap e562: 12 total, 12 up, 12 in 2020-09-08T16:00:58.219 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:00:58.324 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:58.332 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:58 smithi166 bash[9446]: cluster 2020-09-08T16:00:56.383009+0000 osd.4 (osd.4) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:58.333 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:58 smithi166 bash[9446]: cluster 2020-09-08T16:00:56.383134+0000 osd.4 (osd.4) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:58.333 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:58 smithi166 bash[9446]: cluster 2020-09-08T16:00:57.054412+0000 mon.a (mon.0) 1882 : cluster [DBG] osdmap e562: 12 total, 12 up, 12 in 2020-09-08T16:00:58.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:00:58 smithi055 bash[14301]: debug 2020-09-08T16:00:58.218+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:58.408 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 18 15 47446 47431 164.675 127.062 0.0139103 0.00606854 2020-09-08T16:00:58.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:58 smithi104 bash[9203]: cluster 2020-09-08T16:00:56.383009+0000 osd.4 (osd.4) 59 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:00:58.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:58 smithi104 bash[9203]: cluster 2020-09-08T16:00:56.383134+0000 osd.4 (osd.4) 60 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:00:58.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:58 smithi104 bash[9203]: cluster 2020-09-08T16:00:57.054412+0000 mon.a (mon.0) 1882 : cluster [DBG] osdmap e562: 12 total, 12 up, 12 in 2020-09-08T16:00:58.458 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:00:58.573 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:58.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:00:58 smithi166 bash[11928]: debug 2020-09-08T16:00:58.451+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:58.695 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:00:58.797 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:58.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:00:58 smithi055 bash[11956]: debug 2020-09-08T16:00:58.686+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:00:59.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:00:59 smithi104 bash[9203]: cluster 2020-09-08T16:00:57.969761+0000 mgr.x (mgr.24118) 1009 : cluster [DBG] pgmap v1487: 17 pgs: 17 active+clean; 2.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 175 MiB/s wr, 2.80k op/s 2020-09-08T16:00:59.291 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:59.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:00:59 smithi166 bash[9446]: cluster 2020-09-08T16:00:57.969761+0000 mgr.x (mgr.24118) 1009 : cluster [DBG] pgmap v1487: 17 pgs: 17 active+clean; 2.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 175 MiB/s wr, 2.80k op/s 2020-09-08T16:00:59.395 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:00:59.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:00:59 smithi055 bash[9296]: cluster 2020-09-08T16:00:57.969761+0000 mgr.x (mgr.24118) 1009 : cluster [DBG] pgmap v1487: 17 pgs: 17 active+clean; 2.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 175 MiB/s wr, 2.80k op/s 2020-09-08T16:00:59.408 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 19 16 49700 49684 163.418 140.812 0.0101414 0.00611588 2020-09-08T16:00:59.630 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:00:59 smithi104 bash[22305]: debug 2020-09-08T16:00:59.295+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:00.410 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 16:01:00.412575 min lat: 0.00178141 max lat: 0.479134 avg lat: 0.00618381 2020-09-08T16:01:00.411 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T16:01:00.411 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 20 16 51750 51734 161.653 128.125 0.0067895 0.00618381 2020-09-08T16:01:00.901 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:01.009 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:01:01.129 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:01:01.189 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:00 smithi104 bash[9203]: cluster 2020-09-08T16:00:59.970198+0000 mgr.x (mgr.24118) 1010 : cluster [DBG] pgmap v1488: 17 pgs: 17 active+clean; 2.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 154 MiB/s wr, 2.47k op/s 2020-09-08T16:01:01.189 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:01:00 smithi104 bash[22305]: debug 2020-09-08T16:01:00.903+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:01.234 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:01:01.249 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:00 smithi055 bash[9296]: cluster 2020-09-08T16:00:59.970198+0000 mgr.x (mgr.24118) 1010 : cluster [DBG] pgmap v1488: 17 pgs: 17 active+clean; 2.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 154 MiB/s wr, 2.47k op/s 2020-09-08T16:01:01.249 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:01:01 smithi055 bash[14301]: debug 2020-09-08T16:01:01.130+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:01.353 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:01:01.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:00 smithi166 bash[9446]: cluster 2020-09-08T16:00:59.970198+0000 mgr.x (mgr.24118) 1010 : cluster [DBG] pgmap v1488: 17 pgs: 17 active+clean; 2.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 154 MiB/s wr, 2.47k op/s 2020-09-08T16:01:01.408 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 21 16 53765 53749 159.951 125.938 0.00857739 0.00624961 2020-09-08T16:01:01.454 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:01:01.615 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T16:01:01.615 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T16:01:01.616 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.616 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:54122 7.f 7:fa2f7acf:::benchmark_data_smithi166_48533_object54121:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e562)", 2020-09-08T16:01:01.616 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:01:01.618091+0000", 2020-09-08T16:01:01.617 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0016560349999999999, 2020-09-08T16:01:01.617 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0016790290000000001, 2020-09-08T16:01:01.617 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:01.617 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:01:01.617 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:01.618 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:01:01.618 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:01:01.618 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 54122 2020-09-08T16:01:01.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.619 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:01.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:01.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618091+0000", 2020-09-08T16:01:01.620 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:01.620 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.620 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.620 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:01.621 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618091+0000", 2020-09-08T16:01:01.621 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999595 2020-09-08T16:01:01.621 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.621 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:01.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618050+0000", 2020-09-08T16:01:01.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014577399999999999 2020-09-08T16:01:01.623 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.624 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:01.624 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618196+0000", 2020-09-08T16:01:01.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2178e-05 2020-09-08T16:01:01.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:01.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618218+0000", 2020-09-08T16:01:01.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3890000000000004e-06 2020-09-08T16:01:01.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.627 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:01.627 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618224+0000", 2020-09-08T16:01:01.628 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7286000000000002e-05 2020-09-08T16:01:01.628 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.628 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.628 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:01.628 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618261+0000", 2020-09-08T16:01:01.629 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000145749 2020-09-08T16:01:01.629 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.629 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:01.629 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618407+0000", 2020-09-08T16:01:01.630 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014580999999999999 2020-09-08T16:01:01.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.630 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.630 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8", 2020-09-08T16:01:01.631 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618552+0000", 2020-09-08T16:01:01.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00046176399999999998 2020-09-08T16:01:01.631 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:01.631 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:01.631 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:01.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.632 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.632 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:54123 7.9 7:99d508e3:::benchmark_data_smithi166_48533_object54122:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e562)", 2020-09-08T16:01:01.632 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:01:01.618268+0000", 2020-09-08T16:01:01.633 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.001478458, 2020-09-08T16:01:01.633 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0015565220000000001, 2020-09-08T16:01:01.633 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:01.633 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:01:01.633 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:01.634 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:01:01.634 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:01:01.634 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 54123 2020-09-08T16:01:01.635 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.635 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:01.635 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.635 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:01.636 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618268+0000", 2020-09-08T16:01:01.636 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:01.636 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.636 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.637 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:01.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618268+0000", 2020-09-08T16:01:01.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999795 2020-09-08T16:01:01.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:01.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618248+0000", 2020-09-08T16:01:01.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9850000000000003e-05 2020-09-08T16:01:01.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:01.639 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618328+0000", 2020-09-08T16:01:01.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7109999999999995e-06 2020-09-08T16:01:01.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.640 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:01.640 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618335+0000", 2020-09-08T16:01:01.640 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.422e-06 2020-09-08T16:01:01.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.641 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.641 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:01.641 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618339+0000", 2020-09-08T16:01:01.641 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000433881 2020-09-08T16:01:01.641 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:01.642 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618773+0000", 2020-09-08T16:01:01.642 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014871400000000001 2020-09-08T16:01:01.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.643 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.643 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:01.643 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618921+0000", 2020-09-08T16:01:01.643 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017684800000000001 2020-09-08T16:01:01.643 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.644 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.644 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T16:01:01.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.619098+0000", 2020-09-08T16:01:01.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00082996600000000002 2020-09-08T16:01:01.646 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:01.646 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:01.646 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:01.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.647 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:54124 7.4 7:2cb09799:::benchmark_data_smithi166_48533_object54123:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e562)", 2020-09-08T16:01:01.647 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:01:01.618373+0000", 2020-09-08T16:01:01.647 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.001374103, 2020-09-08T16:01:01.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001500574, 2020-09-08T16:01:01.648 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:01.648 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:01:01.648 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:01.648 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:01:01.648 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:01:01.649 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 54124 2020-09-08T16:01:01.649 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.649 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:01.649 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.649 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:01.650 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618373+0000", 2020-09-08T16:01:01.650 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:01.650 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.650 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.651 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:01.651 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618373+0000", 2020-09-08T16:01:01.651 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999804 2020-09-08T16:01:01.651 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.652 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.652 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:01.653 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618353+0000", 2020-09-08T16:01:01.653 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4874000000000002e-05 2020-09-08T16:01:01.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.653 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.653 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:01.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618428+0000", 2020-09-08T16:01:01.654 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7839999999999998e-06 2020-09-08T16:01:01.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.655 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:01.655 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618435+0000", 2020-09-08T16:01:01.655 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8320000000000002e-06 2020-09-08T16:01:01.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.656 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.656 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:01.656 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618437+0000", 2020-09-08T16:01:01.656 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2349999999999999e-05 2020-09-08T16:01:01.657 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.657 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.657 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:01.657 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618480+0000", 2020-09-08T16:01:01.657 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000118559 2020-09-08T16:01:01.658 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.659 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.659 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:01.659 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618598+0000", 2020-09-08T16:01:01.659 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000401975 2020-09-08T16:01:01.660 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.660 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.661 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:01:01.661 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.619000+0000", 2020-09-08T16:01:01.661 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00062764299999999999 2020-09-08T16:01:01.662 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:01.662 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:01.663 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:01.664 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.665 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.666 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop_reply(client.36113.0:54118 7.4 e562/553)", 2020-09-08T16:01:01.667 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:01:01.618643+0000", 2020-09-08T16:01:01.674 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.001103967, 2020-09-08T16:01:01.674 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0012736539999999999, 2020-09-08T16:01:01.675 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:01.675 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "queued for pg", 2020-09-08T16:01:01.675 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:01.675 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.675 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:01.676 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618643+0000", 2020-09-08T16:01:01.676 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:01.677 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.677 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.677 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:01.677 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618643+0000", 2020-09-08T16:01:01.678 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4923000000000003e-05 2020-09-08T16:01:01.678 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.679 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.679 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:01.679 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618698+0000", 2020-09-08T16:01:01.679 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.277e-06 2020-09-08T16:01:01.680 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.680 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.680 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:01.680 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618705+0000", 2020-09-08T16:01:01.680 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3796e-05 2020-09-08T16:01:01.681 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.681 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.681 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:01.681 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618719+0000", 2020-09-08T16:01:01.682 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9880000000000004e-06 2020-09-08T16:01:01.683 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.683 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.683 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:01.683 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.618724+0000", 2020-09-08T16:01:01.683 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0983999999999996e-05 2020-09-08T16:01:01.684 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:01.684 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:01.684 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:01.684 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.684 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.685 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:54112 7.9 7:9816ef72:::benchmark_data_smithi166_48533_object54111:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e562)", 2020-09-08T16:01:01.685 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:01:01.613847+0000", 2020-09-08T16:01:01.685 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0058994499999999997, 2020-09-08T16:01:01.685 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0061023359999999999, 2020-09-08T16:01:01.685 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:01.686 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:01:01.686 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:01.686 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:01:01.686 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:01:01.687 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 54112 2020-09-08T16:01:01.687 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.687 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:01.687 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.688 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:01.688 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.613847+0000", 2020-09-08T16:01:01.688 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:01.688 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.689 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.689 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:01.689 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.613847+0000", 2020-09-08T16:01:01.689 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999599 2020-09-08T16:01:01.689 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.690 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.690 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:01.691 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.613807+0000", 2020-09-08T16:01:01.691 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000122578 2020-09-08T16:01:01.691 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.692 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.692 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:01.692 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.613930+0000", 2020-09-08T16:01:01.692 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6208000000000001e-05 2020-09-08T16:01:01.693 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.693 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.693 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:01.693 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.613946+0000", 2020-09-08T16:01:01.693 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4040000000000001e-06 2020-09-08T16:01:01.694 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.694 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.694 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:01.694 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.613955+0000", 2020-09-08T16:01:01.695 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3585000000000003e-05 2020-09-08T16:01:01.695 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.695 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.695 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:01.695 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.614009+0000", 2020-09-08T16:01:01.696 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000168214 2020-09-08T16:01:01.696 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.696 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.696 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:01.696 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.614177+0000", 2020-09-08T16:01:01.697 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014933399999999999 2020-09-08T16:01:01.697 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.697 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.697 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T16:01:01.697 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.614326+0000", 2020-09-08T16:01:01.698 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000479073 2020-09-08T16:01:01.698 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:01.698 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:01.698 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:01.699 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.699 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.699 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:54118 7.4 7:2bb9970d:::benchmark_data_smithi166_48533_object54117:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e562)", 2020-09-08T16:01:01.699 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:01:01.615273+0000", 2020-09-08T16:01:01.700 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0044739630000000001, 2020-09-08T16:01:01.700 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0047229849999999999, 2020-09-08T16:01:01.700 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:01.700 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:01:01.700 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:01.701 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:01:01.701 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:01:01.701 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 54118 2020-09-08T16:01:01.701 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.702 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:01.702 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.702 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:01.702 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.615273+0000", 2020-09-08T16:01:01.702 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:01.703 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.703 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.703 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:01.703 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.615273+0000", 2020-09-08T16:01:01.703 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999628 2020-09-08T16:01:01.704 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.704 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.704 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:01.704 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.615236+0000", 2020-09-08T16:01:01.704 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011873 2020-09-08T16:01:01.705 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.705 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.705 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:01.705 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.615354+0000", 2020-09-08T16:01:01.705 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3026e-05 2020-09-08T16:01:01.706 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.706 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.706 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:01.706 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.615367+0000", 2020-09-08T16:01:01.707 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8379999999999998e-06 2020-09-08T16:01:01.707 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.707 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.707 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:01.707 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.615373+0000", 2020-09-08T16:01:01.708 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9883000000000003e-05 2020-09-08T16:01:01.708 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.708 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.708 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:01.709 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.615423+0000", 2020-09-08T16:01:01.709 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013750899999999999 2020-09-08T16:01:01.709 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.710 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.710 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:01.710 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.615561+0000", 2020-09-08T16:01:01.710 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015303499999999999 2020-09-08T16:01:01.710 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.711 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.711 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:01:01.711 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.615714+0000", 2020-09-08T16:01:01.711 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002006294 2020-09-08T16:01:01.711 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.712 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.713 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:01:01.713 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.617720+0000", 2020-09-08T16:01:01.713 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002447117 2020-09-08T16:01:01.714 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:01.714 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:01.714 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:01.714 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.714 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.715 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:54119 7.f 7:fa5e050d:::benchmark_data_smithi166_48533_object54118:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e562)", 2020-09-08T16:01:01.715 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:01:01.616024+0000", 2020-09-08T16:01:01.715 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0037221849999999998, 2020-09-08T16:01:01.715 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0040207070000000001, 2020-09-08T16:01:01.715 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:01.716 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:01:01.716 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:01.717 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:01:01.717 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:01:01.717 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 54119 2020-09-08T16:01:01.718 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.718 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:01.718 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.718 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:01.718 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.616024+0000", 2020-09-08T16:01:01.719 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:01.719 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.719 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.719 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:01.719 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.616024+0000", 2020-09-08T16:01:01.719 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999628 2020-09-08T16:01:01.720 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.720 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.720 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:01.720 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.615987+0000", 2020-09-08T16:01:01.720 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012583399999999999 2020-09-08T16:01:01.721 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.721 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.721 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:01.721 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.616113+0000", 2020-09-08T16:01:01.721 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.7200999999999999e-05 2020-09-08T16:01:01.722 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.722 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.722 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:01.722 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.616140+0000", 2020-09-08T16:01:01.722 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1179999999999997e-06 2020-09-08T16:01:01.723 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.723 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.723 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:01.723 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.616146+0000", 2020-09-08T16:01:01.723 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4529000000000005e-05 2020-09-08T16:01:01.724 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.724 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.724 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:01.724 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.616211+0000", 2020-09-08T16:01:01.724 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014486100000000001 2020-09-08T16:01:01.724 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.726 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.726 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:01.726 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.616356+0000", 2020-09-08T16:01:01.727 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018594799999999999 2020-09-08T16:01:01.727 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.727 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.727 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 8", 2020-09-08T16:01:01.727 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.616542+0000", 2020-09-08T16:01:01.728 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000517262 2020-09-08T16:01:01.728 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:01.728 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:01.728 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:01.728 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.728 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.729 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:54120 7.9 7:959dcff2:::benchmark_data_smithi166_48533_object54119:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e562)", 2020-09-08T16:01:01.729 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:01:01.616757+0000", 2020-09-08T16:01:01.729 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0029897280000000001, 2020-09-08T16:01:01.729 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0033358770000000001, 2020-09-08T16:01:01.729 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:01.730 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:01:01.730 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:01.730 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:01:01.730 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:01:01.730 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 54120 2020-09-08T16:01:01.731 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.731 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:01.731 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.731 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:01.731 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.616757+0000", 2020-09-08T16:01:01.732 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:01.732 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.732 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.732 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:01.732 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.616757+0000", 2020-09-08T16:01:01.732 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999614 2020-09-08T16:01:01.733 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.733 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.733 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:01.733 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.616718+0000", 2020-09-08T16:01:01.733 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011560700000000001 2020-09-08T16:01:01.734 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.734 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.734 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:01.734 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.616834+0000", 2020-09-08T16:01:01.734 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2308e-05 2020-09-08T16:01:01.735 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.735 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.735 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:01.735 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.616846+0000", 2020-09-08T16:01:01.736 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4609999999999999e-06 2020-09-08T16:01:01.736 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.736 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.736 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:01.737 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.616852+0000", 2020-09-08T16:01:01.737 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000118656 2020-09-08T16:01:01.737 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.737 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.737 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:01.738 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.616970+0000", 2020-09-08T16:01:01.738 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7488e-05 2020-09-08T16:01:01.738 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.738 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.739 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:01.739 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.617058+0000", 2020-09-08T16:01:01.739 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000129014 2020-09-08T16:01:01.739 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:01.739 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:01.739 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T16:01:01.740 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:01:01.617187+0000", 2020-09-08T16:01:01.740 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00042977999999999999 2020-09-08T16:01:01.740 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:01.741 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:01.741 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:01.742 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:01.742 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T16:01:01.742 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 8 2020-09-08T16:01:01.742 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T16:01:01.744 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:01:01 smithi055 bash[19003]: debug 2020-09-08T16:01:01.346+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:01.744 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:01:01.846 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:01:02.177 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:01:01 smithi104 bash[15273]: debug 2020-09-08T16:01:01.679+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:02.409 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 22 16 55793 55777 158.442 126.75 0.00604658 0.00630967 2020-09-08T16:01:02.474 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:01:02.579 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:02.709 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:02.817 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:01:02.830 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:01:02 smithi055 bash[19003]: debug 2020-09-08T16:01:02.474+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:02.831 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:02 smithi055 bash[11956]: debug 2020-09-08T16:01:02.710+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:02.929 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:01:03.034 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:01:03.155 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:01:02 smithi055 bash[14301]: debug 2020-09-08T16:01:02.930+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:03.155 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:03 smithi055 bash[9296]: cluster 2020-09-08T16:01:01.970617+0000 mgr.x (mgr.24118) 1011 : cluster [DBG] pgmap v1489: 17 pgs: 17 active+clean; 2.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 115 MiB/s wr, 1.84k op/s 2020-09-08T16:01:03.156 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:01:03.263 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:03.273 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:03 smithi104 bash[9203]: cluster 2020-09-08T16:01:01.970617+0000 mgr.x (mgr.24118) 1011 : cluster [DBG] pgmap v1489: 17 pgs: 17 active+clean; 2.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 115 MiB/s wr, 1.84k op/s 2020-09-08T16:01:03.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:03 smithi166 bash[9446]: cluster 2020-09-08T16:01:01.970617+0000 mgr.x (mgr.24118) 1011 : cluster [DBG] pgmap v1489: 17 pgs: 17 active+clean; 2.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 115 MiB/s wr, 1.84k op/s 2020-09-08T16:01:03.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:01:03 smithi055 bash[16646]: debug 2020-09-08T16:01:03.154+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:03.408 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 23 15 57729 57714 156.816 121.062 0.00818367 0.00637458 2020-09-08T16:01:04.408 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 24 16 59782 59766 155.625 128.25 0.00386368 0.0064234 2020-09-08T16:01:04.695 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:04.799 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:01:04.935 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:01:05.010 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:01:04 smithi104 bash[22305]: debug 2020-09-08T16:01:04.699+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:05.041 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:05.117 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:01:04 smithi166 bash[14254]: debug 2020-09-08T16:01:04.935+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:05.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:05 smithi166 bash[9446]: cluster 2020-09-08T16:01:03.971053+0000 mgr.x (mgr.24118) 1012 : cluster [DBG] pgmap v1490: 17 pgs: 17 active+clean; 3.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 139 MiB/s wr, 2.22k op/s 2020-09-08T16:01:05.149 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:05.228 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2020-09-08T16:01:05.256 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:01:05.364 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:01:05.393 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:01:05.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:05 smithi055 bash[11956]: debug 2020-09-08T16:01:05.150+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:05.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:05 smithi055 bash[9296]: cluster 2020-09-08T16:01:03.971053+0000 mgr.x (mgr.24118) 1012 : cluster [DBG] pgmap v1490: 17 pgs: 17 active+clean; 3.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 139 MiB/s wr, 2.22k op/s 2020-09-08T16:01:05.408 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 25 16 61650 61634 154.07 116.75 0.00820745 0.00648799 2020-09-08T16:01:05.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:05 smithi104 bash[9203]: cluster 2020-09-08T16:01:03.971053+0000 mgr.x (mgr.24118) 1012 : cluster [DBG] pgmap v1490: 17 pgs: 17 active+clean; 3.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 139 MiB/s wr, 2.22k op/s 2020-09-08T16:01:05.468 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:05.585 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:05.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:01:05 smithi166 bash[14254]: debug 2020-09-08T16:01:05.363+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:05.691 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:01:05.815 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:01:05.906 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:05 smithi055 bash[11956]: debug 2020-09-08T16:01:05.582+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:05.917 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:01:05.940 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:01:05.979 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:01:06.025 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:01:06.029 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:01:05 smithi166 bash[16593]: debug 2020-09-08T16:01:05.811+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:06.045 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:01:06.058 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:01:06.087 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:01:06.125 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:01:06.150 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:01:06.166 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:01:06.205 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:01:06.237 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:01:06.270 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:01:06.306 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:01:06.307 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:01:06 smithi166 bash[11928]: debug 2020-09-08T16:01:06.043+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:06.307 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:06 smithi166 bash[9446]: cluster 2020-09-08T16:01:05.521491+0000 osd.9 (osd.9) 83 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:06.308 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:06 smithi166 bash[9446]: cluster 2020-09-08T16:01:05.521582+0000 osd.9 (osd.9) 84 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:06.308 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:06 smithi166 bash[9446]: audit 2020-09-08T16:01:05.907581+0000 mon.a (mon.0) 1883 : audit [DBG] from='client.? v1:172.21.15.104:0/2701490186' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:01:06.308 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:01:06.342 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:01:06.346 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:01:06.346 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:01:06.347 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:01:06.347 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:01:06.347 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:01:06.347 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:01:06.347 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:01:06.348 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:01:06.348 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:01:06.348 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:01:06.348 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:01:06.348 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:01:06.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:06 smithi055 bash[9296]: cluster 2020-09-08T16:01:05.521491+0000 osd.9 (osd.9) 83 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:06.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:06 smithi055 bash[9296]: cluster 2020-09-08T16:01:05.521582+0000 osd.9 (osd.9) 84 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:06.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:06 smithi055 bash[9296]: audit 2020-09-08T16:01:05.907581+0000 mon.a (mon.0) 1883 : audit [DBG] from='client.? v1:172.21.15.104:0/2701490186' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:01:06.408 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 26 16 63513 63497 152.622 116.438 0.00394384 0.00654981 2020-09-08T16:01:06.411 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:01:06.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:06 smithi104 bash[9203]: cluster 2020-09-08T16:01:05.521491+0000 osd.9 (osd.9) 83 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:06.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:06 smithi104 bash[9203]: cluster 2020-09-08T16:01:05.521582+0000 osd.9 (osd.9) 84 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:06.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:06 smithi104 bash[9203]: audit 2020-09-08T16:01:05.907581+0000 mon.a (mon.0) 1883 : audit [DBG] from='client.? v1:172.21.15.104:0/2701490186' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:01:06.523 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:01:06.616 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:01:06 smithi166 bash[14254]: debug 2020-09-08T16:01:06.307+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:06.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:01:06 smithi166 bash[14254]: debug 2020-09-08T16:01:06.523+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:06.627 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:01:07.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:07 smithi166 bash[9446]: cluster 2020-09-08T16:01:05.971472+0000 mgr.x (mgr.24118) 1013 : cluster [DBG] pgmap v1491: 17 pgs: 17 active+clean; 3.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 135 MiB/s wr, 2.17k op/s 2020-09-08T16:01:07.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:07 smithi166 bash[9446]: cluster 2020-09-08T16:01:06.035911+0000 mon.a (mon.0) 1884 : cluster [DBG] osdmap e563: 12 total, 12 up, 12 in 2020-09-08T16:01:07.406 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:01:07.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:07 smithi055 bash[9296]: cluster 2020-09-08T16:01:05.971472+0000 mgr.x (mgr.24118) 1013 : cluster [DBG] pgmap v1491: 17 pgs: 17 active+clean; 3.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 135 MiB/s wr, 2.17k op/s 2020-09-08T16:01:07.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:07 smithi055 bash[9296]: cluster 2020-09-08T16:01:06.035911+0000 mon.a (mon.0) 1884 : cluster [DBG] osdmap e563: 12 total, 12 up, 12 in 2020-09-08T16:01:07.408 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 27 16 65124 65108 150.698 100.688 0.00621672 0.00663334 2020-09-08T16:01:07.433 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:07 smithi104 bash[9203]: cluster 2020-09-08T16:01:05.971472+0000 mgr.x (mgr.24118) 1013 : cluster [DBG] pgmap v1491: 17 pgs: 17 active+clean; 3.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 135 MiB/s wr, 2.17k op/s 2020-09-08T16:01:07.434 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:07 smithi104 bash[9203]: cluster 2020-09-08T16:01:06.035911+0000 mon.a (mon.0) 1884 : cluster [DBG] osdmap e563: 12 total, 12 up, 12 in 2020-09-08T16:01:07.511 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:07.928 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:01:07 smithi104 bash[15273]: debug 2020-09-08T16:01:07.407+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:08.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:08 smithi166 bash[9446]: cluster 2020-09-08T16:01:07.592946+0000 osd.2 (osd.2) 84 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:08.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:08 smithi166 bash[9446]: cluster 2020-09-08T16:01:07.593023+0000 osd.2 (osd.2) 85 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:08.414 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:08 smithi055 bash[9296]: cluster 2020-09-08T16:01:07.592946+0000 osd.2 (osd.2) 84 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:08.415 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:08 smithi055 bash[9296]: cluster 2020-09-08T16:01:07.593023+0000 osd.2 (osd.2) 85 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:08.415 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 28 16 66913 66897 149.309 111.812 0.00482336 0.00669517 2020-09-08T16:01:08.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:08 smithi104 bash[9203]: cluster 2020-09-08T16:01:07.592946+0000 osd.2 (osd.2) 84 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:08.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:08 smithi104 bash[9203]: cluster 2020-09-08T16:01:07.593023+0000 osd.2 (osd.2) 85 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:08.719 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:08.823 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:01:09.036 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:01:08 smithi104 bash[22305]: debug 2020-09-08T16:01:08.723+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:09.037 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:09 smithi104 bash[9203]: cluster 2020-09-08T16:01:07.972030+0000 mgr.x (mgr.24118) 1014 : cluster [DBG] pgmap v1493: 17 pgs: 17 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 127 MiB/s wr, 2.03k op/s 2020-09-08T16:01:09.037 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:09 smithi104 bash[9203]: cluster 2020-09-08T16:01:08.043130+0000 mon.a (mon.0) 1885 : cluster [DBG] osdmap e564: 12 total, 12 up, 12 in 2020-09-08T16:01:09.118 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:01:09.222 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:01:09.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:09 smithi166 bash[9446]: cluster 2020-09-08T16:01:07.972030+0000 mgr.x (mgr.24118) 1014 : cluster [DBG] pgmap v1493: 17 pgs: 17 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 127 MiB/s wr, 2.03k op/s 2020-09-08T16:01:09.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:09 smithi166 bash[9446]: cluster 2020-09-08T16:01:08.043130+0000 mon.a (mon.0) 1885 : cluster [DBG] osdmap e564: 12 total, 12 up, 12 in 2020-09-08T16:01:09.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:09 smithi055 bash[9296]: cluster 2020-09-08T16:01:07.972030+0000 mgr.x (mgr.24118) 1014 : cluster [DBG] pgmap v1493: 17 pgs: 17 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 127 MiB/s wr, 2.03k op/s 2020-09-08T16:01:09.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:09 smithi055 bash[9296]: cluster 2020-09-08T16:01:08.043130+0000 mon.a (mon.0) 1885 : cluster [DBG] osdmap e564: 12 total, 12 up, 12 in 2020-09-08T16:01:09.410 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 29 16 68704 68688 148.02 111.938 0.00558807 0.00675415 2020-09-08T16:01:09.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:01:09 smithi104 bash[17588]: debug 2020-09-08T16:01:09.115+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:09.456 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:01:09.561 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:01:09.754 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:01:09 smithi104 bash[15273]: debug 2020-09-08T16:01:09.459+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:10.409 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 30 15 70002 69987 145.792 81.1875 0.00975343 0.00685701 2020-09-08T16:01:10.932 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T16:01:11.219 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:10 smithi104 bash[9203]: cluster 2020-09-08T16:01:09.915069+0000 mon.a (mon.0) 1886 : cluster [DBG] osdmap e565: 12 total, 12 up, 12 in 2020-09-08T16:01:11.220 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:10 smithi104 bash[9203]: cluster 2020-09-08T16:01:09.972594+0000 mgr.x (mgr.24118) 1015 : cluster [DBG] pgmap v1496: 17 pgs: 17 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 100 MiB/s wr, 1.60k op/s 2020-09-08T16:01:11.220 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:10 smithi104 bash[9203]: audit 2020-09-08T16:01:10.713071+0000 mon.a (mon.0) 1887 : audit [INF] from='client.26146 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:01:11.220 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:10 smithi104 bash[9203]: audit 2020-09-08T16:01:10.713135+0000 mon.b (mon.1) 111 : audit [INF] from='client.? v1:172.21.15.104:0/4152782239' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:01:11.278 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:01:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:10 smithi166 bash[9446]: cluster 2020-09-08T16:01:09.915069+0000 mon.a (mon.0) 1886 : cluster [DBG] osdmap e565: 12 total, 12 up, 12 in 2020-09-08T16:01:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:10 smithi166 bash[9446]: cluster 2020-09-08T16:01:09.972594+0000 mgr.x (mgr.24118) 1015 : cluster [DBG] pgmap v1496: 17 pgs: 17 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 100 MiB/s wr, 1.60k op/s 2020-09-08T16:01:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:10 smithi166 bash[9446]: audit 2020-09-08T16:01:10.713071+0000 mon.a (mon.0) 1887 : audit [INF] from='client.26146 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:01:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:10 smithi166 bash[9446]: audit 2020-09-08T16:01:10.713135+0000 mon.b (mon.1) 111 : audit [INF] from='client.? v1:172.21.15.104:0/4152782239' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:01:11.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:10 smithi055 bash[9296]: cluster 2020-09-08T16:01:09.915069+0000 mon.a (mon.0) 1886 : cluster [DBG] osdmap e565: 12 total, 12 up, 12 in 2020-09-08T16:01:11.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:10 smithi055 bash[9296]: cluster 2020-09-08T16:01:09.972594+0000 mgr.x (mgr.24118) 1015 : cluster [DBG] pgmap v1496: 17 pgs: 17 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 100 MiB/s wr, 1.60k op/s 2020-09-08T16:01:11.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:10 smithi055 bash[9296]: audit 2020-09-08T16:01:10.713071+0000 mon.a (mon.0) 1887 : audit [INF] from='client.26146 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:01:11.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:10 smithi055 bash[9296]: audit 2020-09-08T16:01:10.713135+0000 mon.b (mon.1) 111 : audit [INF] from='client.? v1:172.21.15.104:0/4152782239' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:01:11.409 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 31 16 71423 71407 143.952 88.75 0.0130883 0.00694482 2020-09-08T16:01:11.442 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:01:12.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:11 smithi104 bash[9203]: cluster 2020-09-08T16:01:10.921365+0000 mon.a (mon.0) 1888 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:01:12.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:11 smithi104 bash[9203]: audit 2020-09-08T16:01:10.932179+0000 mon.a (mon.0) 1889 : audit [INF] from='client.26146 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:01:12.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:11 smithi104 bash[9203]: cluster 2020-09-08T16:01:10.932230+0000 mon.a (mon.0) 1890 : cluster [DBG] osdmap e566: 12 total, 12 up, 12 in 2020-09-08T16:01:12.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:11 smithi166 bash[9446]: cluster 2020-09-08T16:01:10.921365+0000 mon.a (mon.0) 1888 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:01:12.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:11 smithi166 bash[9446]: audit 2020-09-08T16:01:10.932179+0000 mon.a (mon.0) 1889 : audit [INF] from='client.26146 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:01:12.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:11 smithi166 bash[9446]: cluster 2020-09-08T16:01:10.932230+0000 mon.a (mon.0) 1890 : cluster [DBG] osdmap e566: 12 total, 12 up, 12 in 2020-09-08T16:01:12.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:11 smithi055 bash[9296]: cluster 2020-09-08T16:01:10.921365+0000 mon.a (mon.0) 1888 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:01:12.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:11 smithi055 bash[9296]: audit 2020-09-08T16:01:10.932179+0000 mon.a (mon.0) 1889 : audit [INF] from='client.26146 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:01:12.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:11 smithi055 bash[9296]: cluster 2020-09-08T16:01:10.932230+0000 mon.a (mon.0) 1890 : cluster [DBG] osdmap e566: 12 total, 12 up, 12 in 2020-09-08T16:01:12.412 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 32 16 73174 73158 142.873 109.438 0.00512333 0.00699671 2020-09-08T16:01:12.585 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:01:12.688 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:01:12.844 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:01:12.906 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:01:12 smithi055 bash[19003]: debug 2020-09-08T16:01:12.582+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:12.948 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:01:12.958 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:12 smithi166 bash[9446]: cluster 2020-09-08T16:01:11.973096+0000 mgr.x (mgr.24118) 1016 : cluster [DBG] pgmap v1498: 17 pgs: 17 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 102 MiB/s wr, 1.63k op/s 2020-09-08T16:01:12.959 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:01:12 smithi166 bash[11928]: debug 2020-09-08T16:01:12.842+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:13.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:12 smithi104 bash[9203]: cluster 2020-09-08T16:01:11.973096+0000 mgr.x (mgr.24118) 1016 : cluster [DBG] pgmap v1498: 17 pgs: 17 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 102 MiB/s wr, 1.63k op/s 2020-09-08T16:01:13.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:12 smithi055 bash[9296]: cluster 2020-09-08T16:01:11.973096+0000 mgr.x (mgr.24118) 1016 : cluster [DBG] pgmap v1498: 17 pgs: 17 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 102 MiB/s wr, 1.63k op/s 2020-09-08T16:01:13.409 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 33 16 74780 74764 141.585 100.375 0.011718 0.00706086 2020-09-08T16:01:14.409 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 34 16 76393 76377 140.386 100.812 0.00984953 0.00712114 2020-09-08T16:01:15.314 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:15 smithi104 bash[9203]: cluster 2020-09-08T16:01:13.973654+0000 mgr.x (mgr.24118) 1017 : cluster [DBG] pgmap v1499: 17 pgs: 17 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 95 MiB/s wr, 1.53k op/s 2020-09-08T16:01:15.314 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:15 smithi104 bash[9203]: cluster 2020-09-08T16:01:13.996070+0000 osd.7 (osd.7) 123 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:15.315 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:15 smithi104 bash[9203]: cluster 2020-09-08T16:01:13.996178+0000 osd.7 (osd.7) 124 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:15.315 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:15 smithi104 bash[9203]: cluster 2020-09-08T16:01:14.050977+0000 mon.a (mon.0) 1891 : cluster [DBG] osdmap e567: 12 total, 12 up, 12 in 2020-09-08T16:01:15.315 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:15 smithi104 bash[9203]: cluster 2020-09-08T16:01:14.918859+0000 mon.a (mon.0) 1892 : cluster [DBG] osdmap e568: 12 total, 12 up, 12 in 2020-09-08T16:01:15.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:15 smithi166 bash[9446]: cluster 2020-09-08T16:01:13.973654+0000 mgr.x (mgr.24118) 1017 : cluster [DBG] pgmap v1499: 17 pgs: 17 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 95 MiB/s wr, 1.53k op/s 2020-09-08T16:01:15.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:15 smithi166 bash[9446]: cluster 2020-09-08T16:01:13.996070+0000 osd.7 (osd.7) 123 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:15.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:15 smithi166 bash[9446]: cluster 2020-09-08T16:01:13.996178+0000 osd.7 (osd.7) 124 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:15.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:15 smithi166 bash[9446]: cluster 2020-09-08T16:01:14.050977+0000 mon.a (mon.0) 1891 : cluster [DBG] osdmap e567: 12 total, 12 up, 12 in 2020-09-08T16:01:15.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:15 smithi166 bash[9446]: cluster 2020-09-08T16:01:14.918859+0000 mon.a (mon.0) 1892 : cluster [DBG] osdmap e568: 12 total, 12 up, 12 in 2020-09-08T16:01:15.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:15 smithi055 bash[9296]: cluster 2020-09-08T16:01:13.973654+0000 mgr.x (mgr.24118) 1017 : cluster [DBG] pgmap v1499: 17 pgs: 17 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 95 MiB/s wr, 1.53k op/s 2020-09-08T16:01:15.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:15 smithi055 bash[9296]: cluster 2020-09-08T16:01:13.996070+0000 osd.7 (osd.7) 123 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:15.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:15 smithi055 bash[9296]: cluster 2020-09-08T16:01:13.996178+0000 osd.7 (osd.7) 124 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:15.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:15 smithi055 bash[9296]: cluster 2020-09-08T16:01:14.050977+0000 mon.a (mon.0) 1891 : cluster [DBG] osdmap e567: 12 total, 12 up, 12 in 2020-09-08T16:01:15.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:15 smithi055 bash[9296]: cluster 2020-09-08T16:01:14.918859+0000 mon.a (mon.0) 1892 : cluster [DBG] osdmap e568: 12 total, 12 up, 12 in 2020-09-08T16:01:15.416 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 35 16 77793 77777 138.874 87.5 0.00867185 0.00719851 2020-09-08T16:01:16.018 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T16:01:16.197 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:01:16.410 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 36 15 79341 79326 137.706 96.8125 0.00499349 0.00725974 2020-09-08T16:01:16.566 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T16:01:16.566 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:01:17.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:17 smithi166 bash[9446]: cluster 2020-09-08T16:01:15.739880+0000 osd.0 (osd.0) 87 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:17.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:17 smithi166 bash[9446]: cluster 2020-09-08T16:01:15.739965+0000 osd.0 (osd.0) 88 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:17.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:17 smithi166 bash[9446]: cluster 2020-09-08T16:01:15.974124+0000 mgr.x (mgr.24118) 1018 : cluster [DBG] pgmap v1502: 17 pgs: 17 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s wr, 1.66k op/s 2020-09-08T16:01:17.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:17 smithi166 bash[9446]: cluster 2020-09-08T16:01:16.063066+0000 mon.a (mon.0) 1893 : cluster [DBG] osdmap e569: 12 total, 12 up, 12 in 2020-09-08T16:01:17.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:17 smithi055 bash[9296]: cluster 2020-09-08T16:01:15.739880+0000 osd.0 (osd.0) 87 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:17.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:17 smithi055 bash[9296]: cluster 2020-09-08T16:01:15.739965+0000 osd.0 (osd.0) 88 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:17.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:17 smithi055 bash[9296]: cluster 2020-09-08T16:01:15.974124+0000 mgr.x (mgr.24118) 1018 : cluster [DBG] pgmap v1502: 17 pgs: 17 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s wr, 1.66k op/s 2020-09-08T16:01:17.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:17 smithi055 bash[9296]: cluster 2020-09-08T16:01:16.063066+0000 mon.a (mon.0) 1893 : cluster [DBG] osdmap e569: 12 total, 12 up, 12 in 2020-09-08T16:01:17.411 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 37 16 80636 80620 136.17 80.875 0.00828793 0.00734174 2020-09-08T16:01:17.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:17 smithi104 bash[9203]: cluster 2020-09-08T16:01:15.739880+0000 osd.0 (osd.0) 87 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:17.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:17 smithi104 bash[9203]: cluster 2020-09-08T16:01:15.739965+0000 osd.0 (osd.0) 88 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:17.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:17 smithi104 bash[9203]: cluster 2020-09-08T16:01:15.974124+0000 mgr.x (mgr.24118) 1018 : cluster [DBG] pgmap v1502: 17 pgs: 17 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s wr, 1.66k op/s 2020-09-08T16:01:17.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:17 smithi104 bash[9203]: cluster 2020-09-08T16:01:16.063066+0000 mon.a (mon.0) 1893 : cluster [DBG] osdmap e569: 12 total, 12 up, 12 in 2020-09-08T16:01:18.410 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 38 16 81989 81973 134.811 84.5625 0.0118708 0.00741578 2020-09-08T16:01:19.237 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2020-09-08T16:01:19.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:19 smithi166 bash[9446]: cluster 2020-09-08T16:01:17.974639+0000 mgr.x (mgr.24118) 1019 : cluster [DBG] pgmap v1504: 17 pgs: 17 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 126 MiB/s wr, 2.02k op/s 2020-09-08T16:01:19.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:19 smithi055 bash[9296]: cluster 2020-09-08T16:01:17.974639+0000 mgr.x (mgr.24118) 1019 : cluster [DBG] pgmap v1504: 17 pgs: 17 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 126 MiB/s wr, 2.02k op/s 2020-09-08T16:01:19.410 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 39 16 83558 83542 133.869 98.0625 0.00848837 0.00746768 2020-09-08T16:01:19.413 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:01:19.433 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:19 smithi104 bash[9203]: cluster 2020-09-08T16:01:17.974639+0000 mgr.x (mgr.24118) 1019 : cluster [DBG] pgmap v1504: 17 pgs: 17 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 126 MiB/s wr, 2.02k op/s 2020-09-08T16:01:19.769 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:01:19.871 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:01:19.991 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:01:20.103 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:01:20.114 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:01:19 smithi166 bash[11928]: debug 2020-09-08T16:01:19.766+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:20.156 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:01:19 smithi055 bash[14301]: debug 2020-09-08T16:01:19.990+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:20.224 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:01:20.328 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:20.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:01:20 smithi166 bash[11928]: debug 2020-09-08T16:01:20.222+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:20.414 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 16:01:20.414303 min lat: 0.00178141 max lat: 0.479134 avg lat: 0.00752571 2020-09-08T16:01:20.414 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T16:01:20.415 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 40 15 85040 85025 132.839 92.6875 0.0115726 0.00752571 2020-09-08T16:01:20.683 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:20.785 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:01:20.909 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:01:21.012 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:21.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:20 smithi166 bash[9446]: cluster 2020-09-08T16:01:19.918101+0000 mon.a (mon.0) 1894 : cluster [DBG] osdmap e570: 12 total, 12 up, 12 in 2020-09-08T16:01:21.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:20 smithi166 bash[9446]: cluster 2020-09-08T16:01:19.975213+0000 mgr.x (mgr.24118) 1020 : cluster [DBG] pgmap v1506: 17 pgs: 17 active+clean; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s wr, 1.28k op/s 2020-09-08T16:01:21.117 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:01:20 smithi166 bash[18960]: debug 2020-09-08T16:01:20.906+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:21.132 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:21.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:01:20 smithi104 bash[22305]: debug 2020-09-08T16:01:20.683+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:21.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:20 smithi104 bash[9203]: cluster 2020-09-08T16:01:19.918101+0000 mon.a (mon.0) 1894 : cluster [DBG] osdmap e570: 12 total, 12 up, 12 in 2020-09-08T16:01:21.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:20 smithi104 bash[9203]: cluster 2020-09-08T16:01:19.975213+0000 mgr.x (mgr.24118) 1020 : cluster [DBG] pgmap v1506: 17 pgs: 17 active+clean; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s wr, 1.28k op/s 2020-09-08T16:01:21.236 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:01:21.249 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:21 smithi055 bash[11956]: debug 2020-09-08T16:01:21.130+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:21.249 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:20 smithi055 bash[9296]: cluster 2020-09-08T16:01:19.918101+0000 mon.a (mon.0) 1894 : cluster [DBG] osdmap e570: 12 total, 12 up, 12 in 2020-09-08T16:01:21.249 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:20 smithi055 bash[9296]: cluster 2020-09-08T16:01:19.975213+0000 mgr.x (mgr.24118) 1020 : cluster [DBG] pgmap v1506: 17 pgs: 17 active+clean; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s wr, 1.28k op/s 2020-09-08T16:01:21.351 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:01:21.410 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 41 15 86600 86585 131.977 97.5 0.00675696 0.00757506 2020-09-08T16:01:21.456 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:21.567 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:01:21 smithi055 bash[14301]: debug 2020-09-08T16:01:21.350+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:21.568 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:21.671 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:01:21.792 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:01:21.897 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:01:21.903 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:21 smithi055 bash[11956]: debug 2020-09-08T16:01:21.566+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:22.010 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:01:22.114 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:01:22.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:21 smithi166 bash[9446]: cluster 2020-09-08T16:01:20.418209+0000 osd.7 (osd.7) 125 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:01:22.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:21 smithi166 bash[9446]: cluster 2020-09-08T16:01:20.528297+0000 osd.7 (osd.7) 126 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:01:22.119 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:01:21 smithi166 bash[18960]: debug 2020-09-08T16:01:21.790+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:22.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:01:22 smithi055 bash[16646]: debug 2020-09-08T16:01:22.010+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:22.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:21 smithi055 bash[9296]: cluster 2020-09-08T16:01:20.418209+0000 osd.7 (osd.7) 125 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:01:22.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:21 smithi055 bash[9296]: cluster 2020-09-08T16:01:20.528297+0000 osd.7 (osd.7) 126 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:01:22.233 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:01:22.341 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:01:22.350 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:21 smithi104 bash[9203]: cluster 2020-09-08T16:01:20.418209+0000 osd.7 (osd.7) 125 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:01:22.351 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:21 smithi104 bash[9203]: cluster 2020-09-08T16:01:20.528297+0000 osd.7 (osd.7) 126 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:01:22.410 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 42 16 87885 87869 130.745 80.25 0.00427912 0.00764054 2020-09-08T16:01:22.658 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:01:22 smithi055 bash[14301]: debug 2020-09-08T16:01:22.234+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:23.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:22 smithi166 bash[9446]: cluster 2020-09-08T16:01:21.975667+0000 mgr.x (mgr.24118) 1021 : cluster [DBG] pgmap v1507: 17 pgs: 17 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s wr, 1.39k op/s 2020-09-08T16:01:23.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:22 smithi055 bash[9296]: cluster 2020-09-08T16:01:21.975667+0000 mgr.x (mgr.24118) 1021 : cluster [DBG] pgmap v1507: 17 pgs: 17 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s wr, 1.39k op/s 2020-09-08T16:01:23.410 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 43 16 89409 89393 129.919 95.25 0.0110858 0.00769519 2020-09-08T16:01:23.414 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:22 smithi104 bash[9203]: cluster 2020-09-08T16:01:21.975667+0000 mgr.x (mgr.24118) 1021 : cluster [DBG] pgmap v1507: 17 pgs: 17 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s wr, 1.39k op/s 2020-09-08T16:01:23.695 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:01:23.800 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:01:24.229 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:01:23 smithi104 bash[15273]: debug 2020-09-08T16:01:23.695+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:24.271 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:01:24.375 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:01:24.377 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:01:24.411 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 44 16 90797 90781 128.938 86.75 0.0168968 0.007753 2020-09-08T16:01:24.437 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:01:24.487 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:01:24.510 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:01:24.515 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:01:24.550 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:01:24.582 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:01:24.614 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:01:24.623 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:01:24 smithi104 bash[19922]: debug 2020-09-08T16:01:24.267+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:24.632 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:01:24.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:01:24 smithi055 bash[16646]: debug 2020-09-08T16:01:24.514+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:24.667 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:01:24.703 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:01:24.749 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:01:24.786 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:01:24.825 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:01:24.830 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:01:24.830 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:01:24.831 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:01:24.831 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:01:24.831 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:01:24.831 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:01:24.831 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:01:24.832 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:01:24.832 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:01:24.832 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:01:24.832 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:01:24.832 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:01:25.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:25 smithi104 bash[9203]: cluster 2020-09-08T16:01:23.976278+0000 mgr.x (mgr.24118) 1022 : cluster [DBG] pgmap v1508: 17 pgs: 17 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 101 MiB/s wr, 1.61k op/s 2020-09-08T16:01:25.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:25 smithi104 bash[9203]: audit 2020-09-08T16:01:24.323924+0000 mon.a (mon.0) 1895 : audit [DBG] from='client.? v1:172.21.15.104:0/2751536874' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:01:25.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:25 smithi166 bash[9446]: cluster 2020-09-08T16:01:23.976278+0000 mgr.x (mgr.24118) 1022 : cluster [DBG] pgmap v1508: 17 pgs: 17 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 101 MiB/s wr, 1.61k op/s 2020-09-08T16:01:25.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:25 smithi166 bash[9446]: audit 2020-09-08T16:01:24.323924+0000 mon.a (mon.0) 1895 : audit [DBG] from='client.? v1:172.21.15.104:0/2751536874' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:01:25.418 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 45 15 92225 92210 128.057 89.3125 0.0102035 0.00780673 2020-09-08T16:01:25.419 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:25 smithi055 bash[9296]: cluster 2020-09-08T16:01:23.976278+0000 mgr.x (mgr.24118) 1022 : cluster [DBG] pgmap v1508: 17 pgs: 17 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 101 MiB/s wr, 1.61k op/s 2020-09-08T16:01:25.419 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:25 smithi055 bash[9296]: audit 2020-09-08T16:01:24.323924+0000 mon.a (mon.0) 1895 : audit [DBG] from='client.? v1:172.21.15.104:0/2751536874' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:01:25.738 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:01:25.853 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:01:25.970 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:01:26.074 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:01:26.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:01:25 smithi055 bash[19003]: debug 2020-09-08T16:01:25.974+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:26.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:26 smithi055 bash[9296]: cluster 2020-09-08T16:01:24.965908+0000 osd.5 (osd.5) 91 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:26.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:26 smithi055 bash[9296]: cluster 2020-09-08T16:01:24.966004+0000 osd.5 (osd.5) 92 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:26.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:26 smithi055 bash[9296]: cluster 2020-09-08T16:01:25.920683+0000 mon.a (mon.0) 1896 : cluster [DBG] osdmap e571: 12 total, 12 up, 12 in 2020-09-08T16:01:26.174 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:01:25 smithi104 bash[19922]: debug 2020-09-08T16:01:25.739+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:26.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:26 smithi104 bash[9203]: cluster 2020-09-08T16:01:24.965908+0000 osd.5 (osd.5) 91 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:26.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:26 smithi104 bash[9203]: cluster 2020-09-08T16:01:24.966004+0000 osd.5 (osd.5) 92 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:26.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:26 smithi104 bash[9203]: cluster 2020-09-08T16:01:25.920683+0000 mon.a (mon.0) 1896 : cluster [DBG] osdmap e571: 12 total, 12 up, 12 in 2020-09-08T16:01:26.229 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:01:26.333 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:26.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:01:26 smithi166 bash[18960]: debug 2020-09-08T16:01:26.226+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:26.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:26 smithi166 bash[9446]: cluster 2020-09-08T16:01:24.965908+0000 osd.5 (osd.5) 91 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:26.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:26 smithi166 bash[9446]: cluster 2020-09-08T16:01:24.966004+0000 osd.5 (osd.5) 92 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:26.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:26 smithi166 bash[9446]: cluster 2020-09-08T16:01:25.920683+0000 mon.a (mon.0) 1896 : cluster [DBG] osdmap e571: 12 total, 12 up, 12 in 2020-09-08T16:01:26.410 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 46 16 93378 93362 126.839 72 0.0108777 0.00788204 2020-09-08T16:01:27.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:27 smithi166 bash[9446]: cluster 2020-09-08T16:01:25.976676+0000 mgr.x (mgr.24118) 1023 : cluster [DBG] pgmap v1510: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T16:01:27.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:27 smithi055 bash[9296]: cluster 2020-09-08T16:01:25.976676+0000 mgr.x (mgr.24118) 1023 : cluster [DBG] pgmap v1510: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T16:01:27.410 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 47 16 94854 94838 126.102 92.25 0.0146718 0.00792837 2020-09-08T16:01:27.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:27 smithi104 bash[9203]: cluster 2020-09-08T16:01:25.976676+0000 mgr.x (mgr.24118) 1023 : cluster [DBG] pgmap v1510: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T16:01:27.443 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:27.546 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:01:27.706 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:01:27.807 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:27.818 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:01:27 smithi104 bash[22305]: debug 2020-09-08T16:01:27.443+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:27.869 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:01:27 smithi166 bash[14254]: debug 2020-09-08T16:01:27.702+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:28.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:28 smithi104 bash[9203]: cluster 2020-09-08T16:01:26.971528+0000 osd.10 (osd.10) 69 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:28.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:28 smithi104 bash[9203]: cluster 2020-09-08T16:01:26.971617+0000 osd.10 (osd.10) 70 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:28.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:28 smithi104 bash[9203]: cluster 2020-09-08T16:01:27.931442+0000 mon.a (mon.0) 1897 : cluster [DBG] osdmap e572: 12 total, 12 up, 12 in 2020-09-08T16:01:28.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:28 smithi166 bash[9446]: cluster 2020-09-08T16:01:26.971528+0000 osd.10 (osd.10) 69 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:28.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:28 smithi166 bash[9446]: cluster 2020-09-08T16:01:26.971617+0000 osd.10 (osd.10) 70 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:28.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:28 smithi166 bash[9446]: cluster 2020-09-08T16:01:27.931442+0000 mon.a (mon.0) 1897 : cluster [DBG] osdmap e572: 12 total, 12 up, 12 in 2020-09-08T16:01:28.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:28 smithi055 bash[9296]: cluster 2020-09-08T16:01:26.971528+0000 osd.10 (osd.10) 69 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:28.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:28 smithi055 bash[9296]: cluster 2020-09-08T16:01:26.971617+0000 osd.10 (osd.10) 70 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:28.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:28 smithi055 bash[9296]: cluster 2020-09-08T16:01:27.931442+0000 mon.a (mon.0) 1897 : cluster [DBG] osdmap e572: 12 total, 12 up, 12 in 2020-09-08T16:01:28.411 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 48 15 96244 96229 125.286 86.9375 0.0145662 0.00797997 2020-09-08T16:01:28.641 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:28.747 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:01:28.920 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:01:28.929 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:01:28 smithi104 bash[22305]: debug 2020-09-08T16:01:28.643+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:29.022 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:01:29.030 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:01:28 smithi166 bash[18960]: debug 2020-09-08T16:01:28.906+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:29.129 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:01:29.236 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:01:29.347 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:29 smithi055 bash[9296]: cluster 2020-09-08T16:01:27.977094+0000 mgr.x (mgr.24118) 1024 : cluster [DBG] pgmap v1512: 17 pgs: 17 active+clean; 5.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s wr, 1.66k op/s 2020-09-08T16:01:29.347 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:01:29.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:29 smithi166 bash[9446]: cluster 2020-09-08T16:01:27.977094+0000 mgr.x (mgr.24118) 1024 : cluster [DBG] pgmap v1512: 17 pgs: 17 active+clean; 5.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s wr, 1.66k op/s 2020-09-08T16:01:29.368 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:01:29 smithi166 bash[11928]: debug 2020-09-08T16:01:29.130+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:29.411 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 49 16 97651 97635 124.523 87.875 0.00900806 0.00802867 2020-09-08T16:01:29.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:29 smithi104 bash[9203]: cluster 2020-09-08T16:01:27.977094+0000 mgr.x (mgr.24118) 1024 : cluster [DBG] pgmap v1512: 17 pgs: 17 active+clean; 5.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s wr, 1.66k op/s 2020-09-08T16:01:29.450 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:29.582 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:29.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:29 smithi055 bash[11956]: debug 2020-09-08T16:01:29.586+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:29.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:01:29 smithi055 bash[14301]: debug 2020-09-08T16:01:29.350+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:29.686 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:29.919 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:01:30.095 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:01:30.188 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:30.303 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:01:30.411 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 50 16 98849 98833 123.53 74.875 0.00865858 0.00809237 2020-09-08T16:01:30.444 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:01:30.549 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:01:30.621 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:01:30 smithi166 bash[16593]: debug 2020-09-08T16:01:30.442+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:30.666 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:01:30.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:01:30 smithi104 bash[22305]: debug 2020-09-08T16:01:30.187+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:30.774 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:01:30.927 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:30 smithi166 bash[9446]: cluster 2020-09-08T16:01:29.977525+0000 mgr.x (mgr.24118) 1025 : cluster [DBG] pgmap v1513: 17 pgs: 17 active+clean; 5.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 93 MiB/s wr, 1.49k op/s 2020-09-08T16:01:30.927 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:30 smithi166 bash[9446]: audit 2020-09-08T16:01:30.260641+0000 mon.a (mon.0) 1898 : audit [INF] from='client.? v1:172.21.15.104:0/3483157768' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:01:30.928 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:01:30 smithi166 bash[14254]: debug 2020-09-08T16:01:30.666+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:30.942 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T16:01:31.055 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:30 smithi104 bash[9203]: cluster 2020-09-08T16:01:29.977525+0000 mgr.x (mgr.24118) 1025 : cluster [DBG] pgmap v1513: 17 pgs: 17 active+clean; 5.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 93 MiB/s wr, 1.49k op/s 2020-09-08T16:01:31.056 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:30 smithi104 bash[9203]: audit 2020-09-08T16:01:30.260641+0000 mon.a (mon.0) 1898 : audit [INF] from='client.? v1:172.21.15.104:0/3483157768' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:01:31.219 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:01:31.324 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:31.333 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:01:31 smithi104 bash[19922]: debug 2020-09-08T16:01:31.223+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:31.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:30 smithi055 bash[9296]: cluster 2020-09-08T16:01:29.977525+0000 mgr.x (mgr.24118) 1025 : cluster [DBG] pgmap v1513: 17 pgs: 17 active+clean; 5.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 93 MiB/s wr, 1.49k op/s 2020-09-08T16:01:31.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:30 smithi055 bash[9296]: audit 2020-09-08T16:01:30.260641+0000 mon.a (mon.0) 1898 : audit [INF] from='client.? v1:172.21.15.104:0/3483157768' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:01:31.412 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 51 16 100221 100205 122.789 85.75 0.0051226 0.00814222 2020-09-08T16:01:32.074 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:32.177 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:01:32.186 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:31 smithi166 bash[9446]: cluster 2020-09-08T16:01:30.930506+0000 mon.a (mon.0) 1899 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:01:32.186 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:31 smithi166 bash[9446]: cluster 2020-09-08T16:01:30.930540+0000 mon.a (mon.0) 1900 : cluster [INF] Cluster is now healthy 2020-09-08T16:01:32.186 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:31 smithi166 bash[9446]: cluster 2020-09-08T16:01:30.935067+0000 osd.3 (osd.3) 95 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:32.186 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:31 smithi166 bash[9446]: cluster 2020-09-08T16:01:30.935137+0000 osd.3 (osd.3) 96 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:32.187 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:31 smithi166 bash[9446]: audit 2020-09-08T16:01:30.941635+0000 mon.a (mon.0) 1901 : audit [INF] from='client.? v1:172.21.15.104:0/3483157768' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:01:32.187 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:31 smithi166 bash[9446]: cluster 2020-09-08T16:01:30.941776+0000 mon.a (mon.0) 1902 : cluster [DBG] osdmap e573: 12 total, 12 up, 12 in 2020-09-08T16:01:32.187 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:31 smithi166 bash[9446]: cluster 2020-09-08T16:01:31.426251+0000 osd.2 (osd.2) 86 : cluster [DBG] 7.8 scrub starts 2020-09-08T16:01:32.302 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:01:32 smithi104 bash[22305]: debug 2020-09-08T16:01:32.075+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:32.303 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:31 smithi104 bash[9203]: cluster 2020-09-08T16:01:30.930506+0000 mon.a (mon.0) 1899 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:01:32.303 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:31 smithi104 bash[9203]: cluster 2020-09-08T16:01:30.930540+0000 mon.a (mon.0) 1900 : cluster [INF] Cluster is now healthy 2020-09-08T16:01:32.304 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:31 smithi104 bash[9203]: cluster 2020-09-08T16:01:30.935067+0000 osd.3 (osd.3) 95 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:32.304 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:31 smithi104 bash[9203]: cluster 2020-09-08T16:01:30.935137+0000 osd.3 (osd.3) 96 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:32.304 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:31 smithi104 bash[9203]: audit 2020-09-08T16:01:30.941635+0000 mon.a (mon.0) 1901 : audit [INF] from='client.? v1:172.21.15.104:0/3483157768' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:01:32.305 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:31 smithi104 bash[9203]: cluster 2020-09-08T16:01:30.941776+0000 mon.a (mon.0) 1902 : cluster [DBG] osdmap e573: 12 total, 12 up, 12 in 2020-09-08T16:01:32.316 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:31 smithi104 bash[9203]: cluster 2020-09-08T16:01:31.426251+0000 osd.2 (osd.2) 86 : cluster [DBG] 7.8 scrub starts 2020-09-08T16:01:32.320 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:01:32.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:31 smithi055 bash[9296]: cluster 2020-09-08T16:01:30.930506+0000 mon.a (mon.0) 1899 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:01:32.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:31 smithi055 bash[9296]: cluster 2020-09-08T16:01:30.930540+0000 mon.a (mon.0) 1900 : cluster [INF] Cluster is now healthy 2020-09-08T16:01:32.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:31 smithi055 bash[9296]: cluster 2020-09-08T16:01:30.935067+0000 osd.3 (osd.3) 95 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:32.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:31 smithi055 bash[9296]: cluster 2020-09-08T16:01:30.935137+0000 osd.3 (osd.3) 96 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:32.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:31 smithi055 bash[9296]: audit 2020-09-08T16:01:30.941635+0000 mon.a (mon.0) 1901 : audit [INF] from='client.? v1:172.21.15.104:0/3483157768' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:01:32.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:31 smithi055 bash[9296]: cluster 2020-09-08T16:01:30.941776+0000 mon.a (mon.0) 1902 : cluster [DBG] osdmap e573: 12 total, 12 up, 12 in 2020-09-08T16:01:32.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:31 smithi055 bash[9296]: cluster 2020-09-08T16:01:31.426251+0000 osd.2 (osd.2) 86 : cluster [DBG] 7.8 scrub starts 2020-09-08T16:01:32.411 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 52 16 101488 101472 121.95 79.1875 0.00646797 0.00819795 2020-09-08T16:01:32.422 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:01:32.555 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:01:32 smithi166 bash[16593]: debug 2020-09-08T16:01:32.318+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:32.557 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:01:32.660 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:01:32.867 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:01:32 smithi166 bash[14254]: debug 2020-09-08T16:01:32.554+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:33.018 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T16:01:33.018 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2020-09-08T16:01:33.018 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2020-09-08T16:01:33.019 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T16:01:33.019 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.019 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:7701 5.5 5:a08655ba:::benchmark_data_smithi166_37820_object7700:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e404)", 2020-09-08T16:01:33.019 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:51:55.356696+0000", 2020-09-08T16:01:33.019 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 577.66271742399999, 2020-09-08T16:01:33.020 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.431624061, 2020-09-08T16:01:33.020 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:33.020 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:01:33.020 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:33.021 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T16:01:33.021 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:01:33.021 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 7701 2020-09-08T16:01:33.021 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.021 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:33.022 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.022 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:33.022 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.356696+0000", 2020-09-08T16:01:33.022 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:33.022 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.023 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.023 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:33.023 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.356696+0000", 2020-09-08T16:01:33.023 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999542 2020-09-08T16:01:33.023 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.023 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.024 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:33.024 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.356650+0000", 2020-09-08T16:01:33.024 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014592500000000001 2020-09-08T16:01:33.024 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.024 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.025 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:33.026 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.356796+0000", 2020-09-08T16:01:33.026 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1799e-05 2020-09-08T16:01:33.026 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.026 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.026 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:33.027 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.356808+0000", 2020-09-08T16:01:33.027 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.172e-06 2020-09-08T16:01:33.027 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.027 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.027 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:33.028 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.356812+0000", 2020-09-08T16:01:33.028 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8897000000000002e-05 2020-09-08T16:01:33.028 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.028 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.028 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:33.029 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.356871+0000", 2020-09-08T16:01:33.029 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00027441800000000001 2020-09-08T16:01:33.029 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.029 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.029 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:33.030 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.357146+0000", 2020-09-08T16:01:33.030 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000244367 2020-09-08T16:01:33.036 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.036 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.036 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T16:01:33.036 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.357390+0000", 2020-09-08T16:01:33.037 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001263651 2020-09-08T16:01:33.037 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.037 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.037 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:01:33.037 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.358654+0000", 2020-09-08T16:01:33.038 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42947748200000002 2020-09-08T16:01:33.038 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.038 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.038 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:01:33.038 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.788131+0000", 2020-09-08T16:01:33.039 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011090999999999999 2020-09-08T16:01:33.039 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.039 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.039 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:01:33.039 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.788242+0000", 2020-09-08T16:01:33.040 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.8153999999999997e-05 2020-09-08T16:01:33.040 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.040 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.040 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:01:33.040 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.788320+0000", 2020-09-08T16:01:33.041 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.431624061 2020-09-08T16:01:33.041 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.041 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.041 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.041 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.042 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.042 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:7727 5.5 5:aec88afa:::benchmark_data_smithi166_37820_object7726:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e404)", 2020-09-08T16:01:33.042 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:51:55.363484+0000", 2020-09-08T16:01:33.042 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 577.65592923600002, 2020-09-08T16:01:33.042 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42549411500000001, 2020-09-08T16:01:33.043 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:33.043 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:01:33.044 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:33.044 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T16:01:33.044 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:01:33.044 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 7727 2020-09-08T16:01:33.044 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.045 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:33.045 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.045 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:33.045 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.363484+0000", 2020-09-08T16:01:33.045 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:33.046 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.046 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.046 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:33.046 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.363484+0000", 2020-09-08T16:01:33.046 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999595 2020-09-08T16:01:33.047 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.047 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.047 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:33.047 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.363444+0000", 2020-09-08T16:01:33.047 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000103169 2020-09-08T16:01:33.048 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.048 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.048 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:33.048 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.363547+0000", 2020-09-08T16:01:33.048 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0091e-05 2020-09-08T16:01:33.049 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.049 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.049 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:33.049 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.363557+0000", 2020-09-08T16:01:33.050 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8130000000000003e-06 2020-09-08T16:01:33.050 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.050 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.050 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:33.051 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.363562+0000", 2020-09-08T16:01:33.051 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2843999999999998e-05 2020-09-08T16:01:33.051 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.051 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.051 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:33.052 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.363615+0000", 2020-09-08T16:01:33.052 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000217252 2020-09-08T16:01:33.052 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.052 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.052 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:33.052 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.363832+0000", 2020-09-08T16:01:33.053 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016350199999999999 2020-09-08T16:01:33.053 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.053 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.053 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T16:01:33.054 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.363995+0000", 2020-09-08T16:01:33.055 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0012898149999999999 2020-09-08T16:01:33.055 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.055 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.055 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:01:33.055 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.365285+0000", 2020-09-08T16:01:33.056 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42340125499999998 2020-09-08T16:01:33.056 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.056 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.056 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:01:33.056 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.788686+0000", 2020-09-08T16:01:33.057 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018330199999999999 2020-09-08T16:01:33.057 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.057 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.057 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:01:33.057 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.788870+0000", 2020-09-08T16:01:33.058 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010857900000000001 2020-09-08T16:01:33.058 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.058 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.058 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:01:33.058 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.788978+0000", 2020-09-08T16:01:33.059 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42549411500000001 2020-09-08T16:01:33.059 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.059 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.059 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.059 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.060 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.060 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:7729 5.5 5:abebecba:::benchmark_data_smithi166_37820_object7728:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e404)", 2020-09-08T16:01:33.061 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:51:55.364104+0000", 2020-09-08T16:01:33.061 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 577.65530973700004, 2020-09-08T16:01:33.061 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42546982999999999, 2020-09-08T16:01:33.061 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:33.062 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:01:33.062 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:33.062 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T16:01:33.062 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:01:33.062 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 7729 2020-09-08T16:01:33.063 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.063 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:33.063 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.063 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:33.063 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.364104+0000", 2020-09-08T16:01:33.065 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:33.065 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.065 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.065 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:33.065 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.364104+0000", 2020-09-08T16:01:33.066 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999328 2020-09-08T16:01:33.066 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.066 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.066 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:33.067 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.364036+0000", 2020-09-08T16:01:33.067 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000169409 2020-09-08T16:01:33.067 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.067 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.068 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:33.068 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.364206+0000", 2020-09-08T16:01:33.068 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4986000000000001e-05 2020-09-08T16:01:33.068 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.068 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.068 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:33.069 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.364231+0000", 2020-09-08T16:01:33.069 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1440000000000003e-06 2020-09-08T16:01:33.069 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.069 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.070 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:33.070 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.364239+0000", 2020-09-08T16:01:33.070 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019167500000000001 2020-09-08T16:01:33.070 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.071 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.071 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:33.072 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.364430+0000", 2020-09-08T16:01:33.073 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000203508 2020-09-08T16:01:33.073 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.073 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.074 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:33.074 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.364634+0000", 2020-09-08T16:01:33.074 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00023597099999999999 2020-09-08T16:01:33.074 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.075 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.075 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T16:01:33.075 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.364870+0000", 2020-09-08T16:01:33.075 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00056251999999999995 2020-09-08T16:01:33.075 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.076 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.076 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:01:33.076 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.365432+0000", 2020-09-08T16:01:33.076 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42375369400000001 2020-09-08T16:01:33.076 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.077 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.077 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:01:33.077 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.789186+0000", 2020-09-08T16:01:33.077 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019180799999999999 2020-09-08T16:01:33.077 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.078 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.078 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:01:33.078 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.789378+0000", 2020-09-08T16:01:33.078 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019555199999999999 2020-09-08T16:01:33.078 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.079 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.079 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:01:33.079 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.789574+0000", 2020-09-08T16:01:33.079 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42546982999999999 2020-09-08T16:01:33.079 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.080 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.080 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.080 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.080 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.081 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:7742 5.5 5:a56e23fa:::benchmark_data_smithi166_37820_object7741:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e404)", 2020-09-08T16:01:33.082 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:51:55.373940+0000", 2020-09-08T16:01:33.082 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 577.64547367399996, 2020-09-08T16:01:33.082 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.416078489, 2020-09-08T16:01:33.082 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:33.083 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:01:33.083 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:33.083 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T16:01:33.083 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:01:33.083 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 7742 2020-09-08T16:01:33.084 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.084 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:33.084 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.084 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:33.084 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.373940+0000", 2020-09-08T16:01:33.085 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:33.085 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.085 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.085 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:33.085 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.373940+0000", 2020-09-08T16:01:33.086 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999695 2020-09-08T16:01:33.086 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.086 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.086 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:33.086 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.373909+0000", 2020-09-08T16:01:33.087 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000103714 2020-09-08T16:01:33.087 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.088 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.088 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:33.088 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.374013+0000", 2020-09-08T16:01:33.088 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3936e-05 2020-09-08T16:01:33.088 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.089 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.089 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:33.089 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.374027+0000", 2020-09-08T16:01:33.089 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7360000000000001e-06 2020-09-08T16:01:33.089 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.090 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.090 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:33.090 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.374035+0000", 2020-09-08T16:01:33.090 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6749999999999997e-05 2020-09-08T16:01:33.091 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.091 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.092 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:33.093 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.374091+0000", 2020-09-08T16:01:33.093 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019101599999999999 2020-09-08T16:01:33.093 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.093 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.094 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:33.094 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.374282+0000", 2020-09-08T16:01:33.094 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00023996299999999999 2020-09-08T16:01:33.094 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.095 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.095 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T16:01:33.095 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.374522+0000", 2020-09-08T16:01:33.095 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00058567999999999999 2020-09-08T16:01:33.095 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.096 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.096 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:01:33.096 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.375108+0000", 2020-09-08T16:01:33.096 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.41459593300000003 2020-09-08T16:01:33.096 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.097 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.097 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:01:33.097 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.789704+0000", 2020-09-08T16:01:33.097 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000181428 2020-09-08T16:01:33.097 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.098 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.098 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:01:33.098 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.789885+0000", 2020-09-08T16:01:33.098 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013284200000000001 2020-09-08T16:01:33.098 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.099 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.099 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:01:33.099 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:51:55.790018+0000", 2020-09-08T16:01:33.099 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.416078489 2020-09-08T16:01:33.099 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.100 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.100 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.100 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.100 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.100 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:118549 5.5 5:a7a44c3a:::benchmark_data_smithi166_37820_object118548:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T16:01:33.101 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:52:58.570994+0000", 2020-09-08T16:01:33.101 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 514.44841904800001, 2020-09-08T16:01:33.101 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28777210600000003, 2020-09-08T16:01:33.101 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:33.101 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:01:33.102 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:33.102 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T16:01:33.102 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:01:33.102 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 118549 2020-09-08T16:01:33.103 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.103 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:33.103 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.104 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:33.104 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.570994+0000", 2020-09-08T16:01:33.104 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:33.104 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.104 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.105 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:33.105 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.570994+0000", 2020-09-08T16:01:33.105 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999619 2020-09-08T16:01:33.105 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.105 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.106 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:33.106 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.570956+0000", 2020-09-08T16:01:33.106 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013166799999999999 2020-09-08T16:01:33.106 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.106 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.107 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:33.107 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.571088+0000", 2020-09-08T16:01:33.107 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6183e-05 2020-09-08T16:01:33.107 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.107 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.108 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:33.108 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.571114+0000", 2020-09-08T16:01:33.108 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9039999999999997e-06 2020-09-08T16:01:33.108 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.108 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.109 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:33.109 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.571120+0000", 2020-09-08T16:01:33.109 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1997999999999999e-05 2020-09-08T16:01:33.109 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.109 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.110 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:33.110 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.571172+0000", 2020-09-08T16:01:33.110 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015709200000000001 2020-09-08T16:01:33.110 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.110 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.111 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:33.111 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.571329+0000", 2020-09-08T16:01:33.111 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017345199999999999 2020-09-08T16:01:33.111 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.112 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.112 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T16:01:33.112 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.571502+0000", 2020-09-08T16:01:33.112 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0038532499999999999 2020-09-08T16:01:33.112 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.112 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.113 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:01:33.114 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.575356+0000", 2020-09-08T16:01:33.114 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28304232099999999 2020-09-08T16:01:33.114 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.114 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.114 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:01:33.115 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.858398+0000", 2020-09-08T16:01:33.115 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000197579 2020-09-08T16:01:33.115 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.115 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.115 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:01:33.116 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.858596+0000", 2020-09-08T16:01:33.116 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000170958 2020-09-08T16:01:33.116 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.116 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.116 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:01:33.116 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.858766+0000", 2020-09-08T16:01:33.117 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28777210600000003 2020-09-08T16:01:33.117 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.117 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.117 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.117 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.118 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.118 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:118558 5.5 5:a7834221:::benchmark_data_smithi166_37820_object118557:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T16:01:33.118 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:52:58.576984+0000", 2020-09-08T16:01:33.118 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 514.442429866, 2020-09-08T16:01:33.118 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28229906900000001, 2020-09-08T16:01:33.119 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:33.119 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:01:33.119 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:33.119 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T16:01:33.119 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:01:33.120 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 118558 2020-09-08T16:01:33.120 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.120 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:33.120 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.120 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:33.121 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.576984+0000", 2020-09-08T16:01:33.121 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:33.121 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.121 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.121 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:33.122 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.576984+0000", 2020-09-08T16:01:33.122 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999542 2020-09-08T16:01:33.122 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.122 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.122 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:33.123 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.576938+0000", 2020-09-08T16:01:33.123 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015623500000000001 2020-09-08T16:01:33.123 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.123 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.123 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:33.125 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.577094+0000", 2020-09-08T16:01:33.125 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7130999999999999e-05 2020-09-08T16:01:33.125 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.126 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.126 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:33.127 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.577111+0000", 2020-09-08T16:01:33.127 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3099999999999997e-06 2020-09-08T16:01:33.127 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.127 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.127 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:33.128 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.577118+0000", 2020-09-08T16:01:33.128 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5639e-05 2020-09-08T16:01:33.129 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.129 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.129 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:33.129 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.577173+0000", 2020-09-08T16:01:33.130 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000165847 2020-09-08T16:01:33.130 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.130 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.130 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:33.130 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.577339+0000", 2020-09-08T16:01:33.131 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016937000000000001 2020-09-08T16:01:33.131 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.131 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.131 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T16:01:33.131 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.577508+0000", 2020-09-08T16:01:33.132 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0042521470000000004 2020-09-08T16:01:33.132 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.132 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.132 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:01:33.132 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.581761+0000", 2020-09-08T16:01:33.133 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.27713099800000002 2020-09-08T16:01:33.133 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.133 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.133 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:01:33.133 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.858892+0000", 2020-09-08T16:01:33.134 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018323100000000001 2020-09-08T16:01:33.134 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.134 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.134 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:01:33.134 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.859075+0000", 2020-09-08T16:01:33.135 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020785500000000001 2020-09-08T16:01:33.135 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.135 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.136 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:01:33.136 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.859283+0000", 2020-09-08T16:01:33.136 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28229906900000001 2020-09-08T16:01:33.136 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.136 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.137 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.137 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.137 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.137 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:118595 5.5 5:a6e7060b:::benchmark_data_smithi166_37820_object118594:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T16:01:33.138 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:52:58.613681+0000", 2020-09-08T16:01:33.138 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 514.40573283100002, 2020-09-08T16:01:33.138 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.25784727600000001, 2020-09-08T16:01:33.138 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:33.139 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:01:33.139 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:33.139 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T16:01:33.139 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:01:33.139 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 118595 2020-09-08T16:01:33.140 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.140 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:33.140 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.140 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:33.140 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.613681+0000", 2020-09-08T16:01:33.141 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:33.141 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.141 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.141 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:33.141 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.613681+0000", 2020-09-08T16:01:33.142 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999685 2020-09-08T16:01:33.142 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.142 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.142 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:33.142 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.613649+0000", 2020-09-08T16:01:33.143 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011227599999999999 2020-09-08T16:01:33.143 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.143 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.143 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:33.143 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.613761+0000", 2020-09-08T16:01:33.144 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6949999999999999e-05 2020-09-08T16:01:33.144 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.144 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.144 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:33.144 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.613778+0000", 2020-09-08T16:01:33.145 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2900000000000002e-06 2020-09-08T16:01:33.145 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.145 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.145 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:33.145 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.613787+0000", 2020-09-08T16:01:33.146 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7966000000000006e-05 2020-09-08T16:01:33.146 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.146 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.146 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:33.147 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.613855+0000", 2020-09-08T16:01:33.147 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020327400000000001 2020-09-08T16:01:33.147 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.147 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.147 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:33.148 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.614058+0000", 2020-09-08T16:01:33.148 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000247996 2020-09-08T16:01:33.148 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.148 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.149 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T16:01:33.149 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.614306+0000", 2020-09-08T16:01:33.149 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0038292399999999998 2020-09-08T16:01:33.149 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.149 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.150 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:01:33.150 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.618135+0000", 2020-09-08T16:01:33.150 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.253265711 2020-09-08T16:01:33.150 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.150 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.151 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:01:33.151 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.871401+0000", 2020-09-08T16:01:33.151 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7243999999999994e-05 2020-09-08T16:01:33.151 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.151 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.152 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:01:33.152 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.871478+0000", 2020-09-08T16:01:33.152 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9861000000000003e-05 2020-09-08T16:01:33.152 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.152 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.153 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:01:33.153 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.871528+0000", 2020-09-08T16:01:33.153 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.25784727600000001 2020-09-08T16:01:33.153 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.154 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.154 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.154 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.154 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.155 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15771.0:118621 5.5 5:ab5ee8c7:::benchmark_data_smithi166_37820_object118620:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e424)", 2020-09-08T16:01:33.155 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:52:58.660523+0000", 2020-09-08T16:01:33.155 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 514.35889001600003, 2020-09-08T16:01:33.155 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.211471715, 2020-09-08T16:01:33.155 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:33.156 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:01:33.156 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:33.156 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15771", 2020-09-08T16:01:33.156 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3421460998", 2020-09-08T16:01:33.156 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 118621 2020-09-08T16:01:33.157 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.157 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:33.157 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.157 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:33.157 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.660523+0000", 2020-09-08T16:01:33.158 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:33.158 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.158 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.158 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:33.159 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.660523+0000", 2020-09-08T16:01:33.159 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999461 2020-09-08T16:01:33.159 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.159 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.159 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:33.160 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.660470+0000", 2020-09-08T16:01:33.160 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000170502 2020-09-08T16:01:33.160 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.160 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.160 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:33.161 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.660640+0000", 2020-09-08T16:01:33.161 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7776000000000001e-05 2020-09-08T16:01:33.161 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.161 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.161 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:33.162 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.660658+0000", 2020-09-08T16:01:33.162 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9489999999999999e-06 2020-09-08T16:01:33.162 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.162 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.163 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:33.163 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.660665+0000", 2020-09-08T16:01:33.163 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000103868 2020-09-08T16:01:33.163 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.164 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.164 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:33.164 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.660769+0000", 2020-09-08T16:01:33.164 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00026018299999999999 2020-09-08T16:01:33.164 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.164 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.165 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:33.165 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.661029+0000", 2020-09-08T16:01:33.165 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00037966699999999998 2020-09-08T16:01:33.165 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.166 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.166 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1", 2020-09-08T16:01:33.166 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.661409+0000", 2020-09-08T16:01:33.166 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0049121529999999998 2020-09-08T16:01:33.166 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.167 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.167 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:01:33.167 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.666321+0000", 2020-09-08T16:01:33.167 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.20557313499999999 2020-09-08T16:01:33.167 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.168 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.168 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:01:33.168 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.871894+0000", 2020-09-08T16:01:33.168 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3078000000000002e-05 2020-09-08T16:01:33.169 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.169 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.169 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:01:33.169 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.871947+0000", 2020-09-08T16:01:33.169 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8220999999999998e-05 2020-09-08T16:01:33.170 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.170 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.170 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:01:33.170 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.871995+0000", 2020-09-08T16:01:33.170 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.211471715 2020-09-08T16:01:33.171 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.171 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.172 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.172 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.172 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.173 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:166989 6.0 6:01663469:::benchmark_data_smithi166_43187_object47220:head [delete] snapc 0=[] ondisk+write+known_if_redirected e522)", 2020-09-08T16:01:33.173 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:58:46.609176+0000", 2020-09-08T16:01:33.173 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 166.410237524, 2020-09-08T16:01:33.173 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.327917442, 2020-09-08T16:01:33.174 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:33.174 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:01:33.174 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:33.174 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:01:33.174 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:01:33.175 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 166989 2020-09-08T16:01:33.175 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.175 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:33.175 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.175 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:33.176 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:46.609176+0000", 2020-09-08T16:01:33.176 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:33.176 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.176 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.176 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:33.177 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:46.609176+0000", 2020-09-08T16:01:33.177 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9998469 2020-09-08T16:01:33.177 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.177 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.177 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:33.178 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:46.609023+0000", 2020-09-08T16:01:33.178 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000249721 2020-09-08T16:01:33.178 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.178 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.179 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:33.179 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:46.609272+0000", 2020-09-08T16:01:33.179 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8003000000000002e-05 2020-09-08T16:01:33.179 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.179 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.180 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:33.180 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:46.609310+0000", 2020-09-08T16:01:33.180 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7706e-05 2020-09-08T16:01:33.180 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.181 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.181 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:33.181 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:46.609328+0000", 2020-09-08T16:01:33.181 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1785000000000005e-05 2020-09-08T16:01:33.181 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.182 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.182 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:33.182 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:46.609400+0000", 2020-09-08T16:01:33.183 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012793700000000001 2020-09-08T16:01:33.183 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.183 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.183 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:33.183 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:46.609528+0000", 2020-09-08T16:01:33.184 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000132683 2020-09-08T16:01:33.184 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.184 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.184 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3", 2020-09-08T16:01:33.184 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:46.609661+0000", 2020-09-08T16:01:33.185 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0058274779999999997 2020-09-08T16:01:33.185 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.185 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.185 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:01:33.185 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:46.615488+0000", 2020-09-08T16:01:33.186 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.32131408900000002 2020-09-08T16:01:33.186 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.186 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.186 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:01:33.186 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:46.936802+0000", 2020-09-08T16:01:33.187 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020422100000000001 2020-09-08T16:01:33.187 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.187 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.187 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:01:33.188 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:46.937006+0000", 2020-09-08T16:01:33.188 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7058999999999994e-05 2020-09-08T16:01:33.188 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.189 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.189 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:01:33.189 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:46.937093+0000", 2020-09-08T16:01:33.190 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.327917442 2020-09-08T16:01:33.190 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.191 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.191 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.191 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.191 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.192 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:186924 6.0 6:015a1ac1:::benchmark_data_smithi166_43187_object67155:head [delete] snapc 0=[] ondisk+write+known_if_redirected e529)", 2020-09-08T16:01:33.192 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:59:12.775035+0000", 2020-09-08T16:01:33.192 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 140.24437823299999, 2020-09-08T16:01:33.192 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29535584399999998, 2020-09-08T16:01:33.192 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:33.193 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:01:33.193 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:33.193 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:01:33.193 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:01:33.193 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 186924 2020-09-08T16:01:33.194 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.194 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:33.194 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.194 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:33.194 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:12.775035+0000", 2020-09-08T16:01:33.194 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:33.195 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.195 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.195 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:33.195 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:12.775035+0000", 2020-09-08T16:01:33.196 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999633 2020-09-08T16:01:33.196 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.196 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.196 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:33.196 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:12.774999+0000", 2020-09-08T16:01:33.196 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2043e-05 2020-09-08T16:01:33.197 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.197 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.197 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:33.197 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:12.775081+0000", 2020-09-08T16:01:33.198 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4272e-05 2020-09-08T16:01:33.198 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.198 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.198 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:33.198 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:12.775095+0000", 2020-09-08T16:01:33.198 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7989999999999996e-06 2020-09-08T16:01:33.199 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.199 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.199 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:33.199 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:12.775102+0000", 2020-09-08T16:01:33.199 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2313999999999998e-05 2020-09-08T16:01:33.200 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.200 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.200 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:33.200 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:12.775184+0000", 2020-09-08T16:01:33.200 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017624900000000001 2020-09-08T16:01:33.201 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.201 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.201 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:33.201 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:12.775360+0000", 2020-09-08T16:01:33.201 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000119579 2020-09-08T16:01:33.202 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.202 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.202 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3", 2020-09-08T16:01:33.202 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:12.775480+0000", 2020-09-08T16:01:33.202 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0099714650000000005 2020-09-08T16:01:33.203 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.203 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.203 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:01:33.203 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:12.785451+0000", 2020-09-08T16:01:33.203 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28448573900000002 2020-09-08T16:01:33.204 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.204 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.204 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:01:33.204 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:13.069937+0000", 2020-09-08T16:01:33.204 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00029276299999999998 2020-09-08T16:01:33.204 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.205 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.205 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:01:33.205 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:13.070230+0000", 2020-09-08T16:01:33.205 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016123400000000001 2020-09-08T16:01:33.205 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.206 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.206 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:01:33.206 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:13.070391+0000", 2020-09-08T16:01:33.206 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29535584399999998 2020-09-08T16:01:33.207 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.207 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.207 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.207 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.207 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.207 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:186928 6.0 6:0da10f3d:::benchmark_data_smithi166_43187_object67159:head [delete] snapc 0=[] ondisk+write+known_if_redirected e529)", 2020-09-08T16:01:33.208 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:59:12.778541+0000", 2020-09-08T16:01:33.208 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 140.24087199499999, 2020-09-08T16:01:33.208 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29232361400000001, 2020-09-08T16:01:33.208 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:33.208 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:01:33.209 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:33.209 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:01:33.209 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:01:33.209 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 186928 2020-09-08T16:01:33.209 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.210 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:33.210 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.210 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:33.210 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:12.778541+0000", 2020-09-08T16:01:33.210 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:33.211 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.211 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.211 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:33.211 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:12.778541+0000", 2020-09-08T16:01:33.211 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999647 2020-09-08T16:01:33.212 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.212 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.212 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:33.212 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:12.778506+0000", 2020-09-08T16:01:33.212 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9271999999999996e-05 2020-09-08T16:01:33.213 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.213 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.213 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:33.213 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:12.778585+0000", 2020-09-08T16:01:33.213 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3203e-05 2020-09-08T16:01:33.213 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.214 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.214 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:33.214 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:12.778598+0000", 2020-09-08T16:01:33.214 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3729999999999998e-06 2020-09-08T16:01:33.214 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.215 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.215 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:33.215 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:12.778605+0000", 2020-09-08T16:01:33.215 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001844191 2020-09-08T16:01:33.215 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.216 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.216 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:33.216 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:12.780449+0000", 2020-09-08T16:01:33.216 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016894300000000001 2020-09-08T16:01:33.216 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.217 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.217 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:33.217 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:12.780618+0000", 2020-09-08T16:01:33.217 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015134799999999999 2020-09-08T16:01:33.217 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.218 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.218 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3", 2020-09-08T16:01:33.218 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:12.780769+0000", 2020-09-08T16:01:33.218 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00485397 2020-09-08T16:01:33.218 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.218 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.219 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:01:33.219 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:12.785623+0000", 2020-09-08T16:01:33.219 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28492188400000001 2020-09-08T16:01:33.219 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.219 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.220 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:01:33.220 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:13.070545+0000", 2020-09-08T16:01:33.220 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017445500000000001 2020-09-08T16:01:33.220 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.220 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.221 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:01:33.221 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:13.070720+0000", 2020-09-08T16:01:33.221 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000145458 2020-09-08T16:01:33.221 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.221 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.222 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:01:33.222 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:13.070865+0000", 2020-09-08T16:01:33.222 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29232361400000001 2020-09-08T16:01:33.222 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.222 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.222 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.223 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.223 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.223 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:10100 7.6 7:66d23fe4:::benchmark_data_smithi166_48533_object10099:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e558)", 2020-09-08T16:01:33.223 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:00:43.211431+0000", 2020-09-08T16:01:33.223 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 49.807982392, 2020-09-08T16:01:33.224 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.472728117, 2020-09-08T16:01:33.224 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:33.224 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:01:33.224 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:33.224 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:01:33.225 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:01:33.225 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 10100 2020-09-08T16:01:33.225 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.225 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:33.225 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.226 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:33.226 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.211431+0000", 2020-09-08T16:01:33.226 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:33.226 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.226 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.226 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:33.227 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.211431+0000", 2020-09-08T16:01:33.227 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999695 2020-09-08T16:01:33.227 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.227 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.227 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:33.228 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.211401+0000", 2020-09-08T16:01:33.228 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3598999999999997e-05 2020-09-08T16:01:33.228 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.228 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.228 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:33.229 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.211484+0000", 2020-09-08T16:01:33.229 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5660000000000007e-06 2020-09-08T16:01:33.229 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.229 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.229 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:33.230 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.211493+0000", 2020-09-08T16:01:33.230 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3449999999999999e-06 2020-09-08T16:01:33.230 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.230 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.230 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:33.231 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.211497+0000", 2020-09-08T16:01:33.231 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3207999999999997e-05 2020-09-08T16:01:33.231 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.231 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.231 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:33.231 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.211540+0000", 2020-09-08T16:01:33.232 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014018 2020-09-08T16:01:33.232 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.232 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.232 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:33.232 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.211680+0000", 2020-09-08T16:01:33.233 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000147446 2020-09-08T16:01:33.233 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.233 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.233 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 7", 2020-09-08T16:01:33.233 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.211828+0000", 2020-09-08T16:01:33.234 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.46460727699999999 2020-09-08T16:01:33.234 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.234 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.234 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:01:33.235 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.676435+0000", 2020-09-08T16:01:33.235 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0074174330000000002 2020-09-08T16:01:33.235 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.235 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.241 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:01:33.241 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.683853+0000", 2020-09-08T16:01:33.241 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016973100000000001 2020-09-08T16:01:33.241 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.242 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.242 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:01:33.242 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.684022+0000", 2020-09-08T16:01:33.242 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013680300000000001 2020-09-08T16:01:33.242 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.243 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.243 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:01:33.243 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.684159+0000", 2020-09-08T16:01:33.243 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.472728117 2020-09-08T16:01:33.243 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.244 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.244 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.244 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.244 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.244 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:10102 7.6 7:6f5564de:::benchmark_data_smithi166_48533_object10101:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e558)", 2020-09-08T16:01:33.245 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:00:43.212435+0000", 2020-09-08T16:01:33.245 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 49.806978772999997, 2020-09-08T16:01:33.245 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.47217306599999997, 2020-09-08T16:01:33.245 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:33.246 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:01:33.246 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:33.246 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:01:33.247 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:01:33.248 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 10102 2020-09-08T16:01:33.248 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.248 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:33.248 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.249 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:33.249 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.212435+0000", 2020-09-08T16:01:33.249 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:33.249 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.249 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.250 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:33.250 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.212435+0000", 2020-09-08T16:01:33.250 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999604 2020-09-08T16:01:33.250 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.251 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.251 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:33.251 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.212395+0000", 2020-09-08T16:01:33.251 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014177800000000001 2020-09-08T16:01:33.251 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.252 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.252 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:33.252 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.212537+0000", 2020-09-08T16:01:33.252 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4619e-05 2020-09-08T16:01:33.252 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.253 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.253 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:33.253 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.212551+0000", 2020-09-08T16:01:33.253 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4169999999999997e-06 2020-09-08T16:01:33.254 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.254 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.254 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:33.254 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.212558+0000", 2020-09-08T16:01:33.255 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00024726099999999998 2020-09-08T16:01:33.255 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.255 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.255 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:33.255 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.212805+0000", 2020-09-08T16:01:33.256 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014621999999999999 2020-09-08T16:01:33.256 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.256 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.256 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:33.256 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.212951+0000", 2020-09-08T16:01:33.257 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5155999999999995e-05 2020-09-08T16:01:33.259 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.259 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.260 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 7", 2020-09-08T16:01:33.260 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.213047+0000", 2020-09-08T16:01:33.260 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.46340467299999999 2020-09-08T16:01:33.260 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.260 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.261 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:01:33.261 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.676451+0000", 2020-09-08T16:01:33.261 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0078203790000000006 2020-09-08T16:01:33.261 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.261 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.262 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:01:33.262 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.684272+0000", 2020-09-08T16:01:33.262 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000149915 2020-09-08T16:01:33.262 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.262 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.263 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:01:33.263 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.684421+0000", 2020-09-08T16:01:33.263 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018625799999999999 2020-09-08T16:01:33.263 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.263 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.264 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:01:33.264 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:43.684608+0000", 2020-09-08T16:01:33.264 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.47217306599999997 2020-09-08T16:01:33.264 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.264 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.265 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.265 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.266 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.266 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:27634 7.9 7:9dd3d8a6:::benchmark_data_smithi166_48533_object27633:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e560)", 2020-09-08T16:01:33.266 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:00:49.663077+0000", 2020-09-08T16:01:33.266 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 43.356335962999999, 2020-09-08T16:01:33.266 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.220807636, 2020-09-08T16:01:33.267 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:33.267 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:01:33.267 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:33.267 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:01:33.267 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:01:33.268 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 27634 2020-09-08T16:01:33.268 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.268 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:33.268 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.268 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:33.269 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.663077+0000", 2020-09-08T16:01:33.269 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:33.269 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.269 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.269 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:33.270 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.663077+0000", 2020-09-08T16:01:33.270 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999619 2020-09-08T16:01:33.270 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.271 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.271 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:33.271 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.663039+0000", 2020-09-08T16:01:33.271 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000120145 2020-09-08T16:01:33.271 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.271 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.272 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:33.272 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.663160+0000", 2020-09-08T16:01:33.272 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2449999999999999e-05 2020-09-08T16:01:33.272 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.272 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.273 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:33.273 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.663172+0000", 2020-09-08T16:01:33.273 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1719999999999998e-06 2020-09-08T16:01:33.273 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.273 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.274 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:33.274 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.663177+0000", 2020-09-08T16:01:33.274 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00062971900000000005 2020-09-08T16:01:33.274 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.275 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.275 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:33.275 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.663807+0000", 2020-09-08T16:01:33.275 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012773400000000001 2020-09-08T16:01:33.275 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.278 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.278 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:33.278 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.663935+0000", 2020-09-08T16:01:33.278 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016284399999999999 2020-09-08T16:01:33.279 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.279 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.279 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T16:01:33.279 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.664097+0000", 2020-09-08T16:01:33.279 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0027369130000000001 2020-09-08T16:01:33.280 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.280 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.280 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:01:33.280 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.666834+0000", 2020-09-08T16:01:33.280 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21684383800000001 2020-09-08T16:01:33.281 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.281 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.281 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:01:33.281 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.883678+0000", 2020-09-08T16:01:33.281 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010666399999999999 2020-09-08T16:01:33.282 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.282 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.282 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:01:33.282 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.883785+0000", 2020-09-08T16:01:33.282 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000100222 2020-09-08T16:01:33.283 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.283 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.283 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:01:33.283 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.883885+0000", 2020-09-08T16:01:33.283 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.220807636 2020-09-08T16:01:33.284 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.284 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.284 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.284 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.284 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.285 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:27670 7.9 7:93000046:::benchmark_data_smithi166_48533_object27669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e560)", 2020-09-08T16:01:33.285 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:00:49.675490+0000", 2020-09-08T16:01:33.285 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 43.343923232999998, 2020-09-08T16:01:33.285 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.20874988899999999, 2020-09-08T16:01:33.285 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:33.286 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:01:33.286 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:33.286 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:01:33.286 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:01:33.286 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 27670 2020-09-08T16:01:33.287 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.287 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:33.287 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.287 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:33.287 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.675490+0000", 2020-09-08T16:01:33.288 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:33.288 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.288 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.288 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:33.288 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.675490+0000", 2020-09-08T16:01:33.288 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999628 2020-09-08T16:01:33.289 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.289 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.289 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:33.289 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.675453+0000", 2020-09-08T16:01:33.289 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000132003 2020-09-08T16:01:33.290 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.290 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.290 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:33.290 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.675585+0000", 2020-09-08T16:01:33.290 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5184999999999999e-05 2020-09-08T16:01:33.291 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.291 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.291 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:33.291 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.675600+0000", 2020-09-08T16:01:33.291 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2320000000000001e-06 2020-09-08T16:01:33.292 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.292 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.292 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:33.293 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.675605+0000", 2020-09-08T16:01:33.293 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4159999999999996e-05 2020-09-08T16:01:33.293 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.293 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.294 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:33.294 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.675669+0000", 2020-09-08T16:01:33.294 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000156188 2020-09-08T16:01:33.294 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.294 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.295 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:33.295 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.675826+0000", 2020-09-08T16:01:33.295 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000161674 2020-09-08T16:01:33.295 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.295 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.296 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T16:01:33.296 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.675987+0000", 2020-09-08T16:01:33.296 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020683030000000001 2020-09-08T16:01:33.296 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.296 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.297 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:01:33.297 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.678056+0000", 2020-09-08T16:01:33.297 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.20596515800000001 2020-09-08T16:01:33.297 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.297 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.298 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:01:33.298 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.884021+0000", 2020-09-08T16:01:33.299 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010063199999999999 2020-09-08T16:01:33.300 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.300 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.300 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:01:33.300 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.884121+0000", 2020-09-08T16:01:33.301 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000118742 2020-09-08T16:01:33.301 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.301 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.301 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:01:33.301 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.884240+0000", 2020-09-08T16:01:33.302 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.20874988899999999 2020-09-08T16:01:33.302 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.302 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.302 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.302 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.303 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.303 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:27678 7.9 7:9d8601b5:::benchmark_data_smithi166_48533_object27677:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e560)", 2020-09-08T16:01:33.303 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:00:49.678458+0000", 2020-09-08T16:01:33.303 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 43.340955059999999, 2020-09-08T16:01:33.303 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.20811733700000001, 2020-09-08T16:01:33.304 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:33.304 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:01:33.304 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:33.304 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:01:33.304 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:01:33.304 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 27678 2020-09-08T16:01:33.305 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.305 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:33.305 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.305 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:33.305 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.678458+0000", 2020-09-08T16:01:33.306 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:33.306 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.306 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.306 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:33.306 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.678458+0000", 2020-09-08T16:01:33.307 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999714 2020-09-08T16:01:33.307 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.307 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.307 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:33.307 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.678430+0000", 2020-09-08T16:01:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000118144 2020-09-08T16:01:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.678548+0000", 2020-09-08T16:01:33.308 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.5437000000000001e-05 2020-09-08T16:01:33.309 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.309 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.309 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:33.309 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.678573+0000", 2020-09-08T16:01:33.309 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2119999999999997e-06 2020-09-08T16:01:33.310 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.310 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.310 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:33.310 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.678578+0000", 2020-09-08T16:01:33.310 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4184999999999999e-05 2020-09-08T16:01:33.311 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.311 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.311 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:33.311 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.678612+0000", 2020-09-08T16:01:33.311 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9103000000000003e-05 2020-09-08T16:01:33.312 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.312 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.312 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:33.312 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.678711+0000", 2020-09-08T16:01:33.312 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010486500000000001 2020-09-08T16:01:33.312 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.313 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.313 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T16:01:33.313 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.678816+0000", 2020-09-08T16:01:33.313 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00065247899999999997 2020-09-08T16:01:33.313 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.314 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.314 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:01:33.314 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.679468+0000", 2020-09-08T16:01:33.314 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.20688709699999999 2020-09-08T16:01:33.314 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.315 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.315 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:01:33.315 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.886355+0000", 2020-09-08T16:01:33.315 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8884999999999999e-05 2020-09-08T16:01:33.315 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.316 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.316 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:01:33.316 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.886454+0000", 2020-09-08T16:01:33.316 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012139899999999999 2020-09-08T16:01:33.316 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.316 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.317 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:01:33.317 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:49.886576+0000", 2020-09-08T16:01:33.317 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.20811733700000001 2020-09-08T16:01:33.317 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.317 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.318 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.318 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.318 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.318 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:43053 7.6 7:601ad6e1:::benchmark_data_smithi166_48533_object43052:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e561)", 2020-09-08T16:01:33.318 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:00:56.140518+0000", 2020-09-08T16:01:33.319 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 36.878895018000001, 2020-09-08T16:01:33.319 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26722710900000002, 2020-09-08T16:01:33.319 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:33.319 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:01:33.319 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:33.320 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:01:33.320 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:01:33.320 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 43053 2020-09-08T16:01:33.320 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.320 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:33.320 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.321 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:33.321 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.140518+0000", 2020-09-08T16:01:33.321 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:33.321 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.321 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.322 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:33.322 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.140518+0000", 2020-09-08T16:01:33.322 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999523 2020-09-08T16:01:33.322 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.322 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.323 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:33.323 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.140471+0000", 2020-09-08T16:01:33.323 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000148548 2020-09-08T16:01:33.323 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.323 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.324 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:33.324 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.140619+0000", 2020-09-08T16:01:33.324 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2441e-05 2020-09-08T16:01:33.324 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.324 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.325 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:33.325 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.140632+0000", 2020-09-08T16:01:33.325 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3959999999999997e-06 2020-09-08T16:01:33.325 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.325 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.326 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:33.326 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.140637+0000", 2020-09-08T16:01:33.326 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0036730600000000001 2020-09-08T16:01:33.326 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.326 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.326 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:33.327 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.144310+0000", 2020-09-08T16:01:33.327 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013028000000000001 2020-09-08T16:01:33.327 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.327 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.327 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:33.328 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.144440+0000", 2020-09-08T16:01:33.328 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000108341 2020-09-08T16:01:33.328 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.328 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.328 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 7", 2020-09-08T16:01:33.329 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.144549+0000", 2020-09-08T16:01:33.329 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0011319940000000001 2020-09-08T16:01:33.329 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.329 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.329 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:01:33.330 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.145681+0000", 2020-09-08T16:01:33.330 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001113532 2020-09-08T16:01:33.330 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.330 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.330 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:01:33.331 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.146794+0000", 2020-09-08T16:01:33.331 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8038999999999999e-05 2020-09-08T16:01:33.331 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.331 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.332 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:01:33.332 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.146842+0000", 2020-09-08T16:01:33.332 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26090318899999998 2020-09-08T16:01:33.332 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.332 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.333 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:01:33.333 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.407746+0000", 2020-09-08T16:01:33.333 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26722710900000002 2020-09-08T16:01:33.333 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.333 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.334 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.334 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.334 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.334 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:43058 7.9 7:9140aa04:::benchmark_data_smithi166_48533_object43057:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e561)", 2020-09-08T16:01:33.334 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:00:56.144068+0000", 2020-09-08T16:01:33.335 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 36.875345633999999, 2020-09-08T16:01:33.335 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26465545800000001, 2020-09-08T16:01:33.335 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:33.335 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:01:33.335 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:33.336 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:01:33.337 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:01:33.337 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 43058 2020-09-08T16:01:33.337 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.337 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:33.338 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.338 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:33.338 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.144068+0000", 2020-09-08T16:01:33.338 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:33.339 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.339 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.340 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:33.340 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.144068+0000", 2020-09-08T16:01:33.341 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999743 2020-09-08T16:01:33.341 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.341 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.341 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:33.342 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.144042+0000", 2020-09-08T16:01:33.342 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012505200000000001 2020-09-08T16:01:33.342 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.342 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.342 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:33.343 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.144167+0000", 2020-09-08T16:01:33.343 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6710000000000005e-06 2020-09-08T16:01:33.343 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.343 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.343 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:33.344 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.144176+0000", 2020-09-08T16:01:33.344 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3889999999999998e-06 2020-09-08T16:01:33.344 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.344 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.344 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:33.345 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.144180+0000", 2020-09-08T16:01:33.345 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3930000000000001e-05 2020-09-08T16:01:33.345 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.345 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.345 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:33.346 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.144224+0000", 2020-09-08T16:01:33.346 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011063799999999999 2020-09-08T16:01:33.346 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.346 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.346 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:33.347 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.144335+0000", 2020-09-08T16:01:33.347 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013499700000000001 2020-09-08T16:01:33.347 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.347 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.347 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T16:01:33.347 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.144470+0000", 2020-09-08T16:01:33.348 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0016456400000000001 2020-09-08T16:01:33.348 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.349 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.349 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:01:33.350 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.146115+0000", 2020-09-08T16:01:33.350 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26226903400000001 2020-09-08T16:01:33.350 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.350 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.350 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:01:33.351 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.408385+0000", 2020-09-08T16:01:33.351 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000199824 2020-09-08T16:01:33.351 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.351 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.351 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:01:33.351 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.408584+0000", 2020-09-08T16:01:33.352 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000138906 2020-09-08T16:01:33.352 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.352 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.352 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:01:33.352 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.408723+0000", 2020-09-08T16:01:33.353 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26465545800000001 2020-09-08T16:01:33.353 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.353 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.353 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.353 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.354 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.354 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:43063 7.4 7:266a4a30:::benchmark_data_smithi166_48533_object43062:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e561)", 2020-09-08T16:01:33.354 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:00:56.146327+0000", 2020-09-08T16:01:33.354 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 36.873086919999999, 2020-09-08T16:01:33.355 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.27361758899999999, 2020-09-08T16:01:33.355 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:33.355 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:01:33.355 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:01:33.355 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:01:33.356 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:01:33.356 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 43063 2020-09-08T16:01:33.356 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.356 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:33.356 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.357 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:33.357 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.146327+0000", 2020-09-08T16:01:33.357 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:33.357 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.357 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.358 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:33.358 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.146327+0000", 2020-09-08T16:01:33.358 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999552 2020-09-08T16:01:33.358 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.358 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.358 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:33.359 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.146282+0000", 2020-09-08T16:01:33.359 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000137224 2020-09-08T16:01:33.360 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.360 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.360 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:33.360 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.146419+0000", 2020-09-08T16:01:33.361 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3315e-05 2020-09-08T16:01:33.361 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.361 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.361 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:33.361 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.146432+0000", 2020-09-08T16:01:33.362 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0789999999999997e-06 2020-09-08T16:01:33.362 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.362 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.362 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:33.362 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.146438+0000", 2020-09-08T16:01:33.363 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2975e-05 2020-09-08T16:01:33.363 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.363 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.363 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:33.363 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.146491+0000", 2020-09-08T16:01:33.364 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000132958 2020-09-08T16:01:33.364 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.364 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.364 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:33.364 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.146624+0000", 2020-09-08T16:01:33.365 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000155334 2020-09-08T16:01:33.365 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.365 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.365 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:01:33.365 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.146780+0000", 2020-09-08T16:01:33.366 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26760098300000001 2020-09-08T16:01:33.366 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.366 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.366 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:01:33.366 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.414381+0000", 2020-09-08T16:01:33.367 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0054347850000000001 2020-09-08T16:01:33.367 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.367 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.367 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:01:33.367 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.419815+0000", 2020-09-08T16:01:33.368 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.2205000000000004e-05 2020-09-08T16:01:33.368 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.368 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.368 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:01:33.368 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.419888+0000", 2020-09-08T16:01:33.369 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6456999999999998e-05 2020-09-08T16:01:33.369 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.369 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.369 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:01:33.369 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.419944+0000", 2020-09-08T16:01:33.370 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.27361758899999999 2020-09-08T16:01:33.370 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.370 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.370 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.370 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.371 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.371 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop_reply(client.36113.0:43053 7.6 e561/553 ondisk, result = 0)", 2020-09-08T16:01:33.371 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:00:56.146642+0000", 2020-09-08T16:01:33.371 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 36.872771106000002, 2020-09-08T16:01:33.371 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26112757399999997, 2020-09-08T16:01:33.372 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:01:33.372 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2020-09-08T16:01:33.372 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:01:33.372 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.372 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:01:33.373 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.146642+0000", 2020-09-08T16:01:33.373 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:01:33.373 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.373 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.373 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:01:33.374 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.146642+0000", 2020-09-08T16:01:33.374 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.8349999999999994e-05 2020-09-08T16:01:33.374 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.374 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.374 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:01:33.374 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.146721+0000", 2020-09-08T16:01:33.375 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4289999999999998e-06 2020-09-08T16:01:33.375 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.375 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.375 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:01:33.375 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.146727+0000", 2020-09-08T16:01:33.376 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1420000000000001e-05 2020-09-08T16:01:33.376 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.376 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.376 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:01:33.376 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.146739+0000", 2020-09-08T16:01:33.377 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0550000000000001e-06 2020-09-08T16:01:33.377 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.377 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.377 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:01:33.377 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.146743+0000", 2020-09-08T16:01:33.378 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6974999999999998e-05 2020-09-08T16:01:33.378 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.378 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.378 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:01:33.378 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.146780+0000", 2020-09-08T16:01:33.379 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1790000000000002e-06 2020-09-08T16:01:33.379 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.379 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.379 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:01:33.379 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.146787+0000", 2020-09-08T16:01:33.380 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26098316599999999 2020-09-08T16:01:33.380 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:01:33.380 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:01:33.380 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:01:33.380 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:00:56.407770+0000", 2020-09-08T16:01:33.380 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.26112757399999997 2020-09-08T16:01:33.381 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.381 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.381 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.381 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:01:33.381 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:01:33.382 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T16:01:33.383 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:32 smithi166 bash[9446]: cluster 2020-09-08T16:01:31.941587+0000 mon.a (mon.0) 1903 : cluster [DBG] osdmap e574: 12 total, 12 up, 12 in 2020-09-08T16:01:33.383 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:32 smithi166 bash[9446]: cluster 2020-09-08T16:01:31.977935+0000 mgr.x (mgr.24118) 1026 : cluster [DBG] pgmap v1516: 17 pgs: 17 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T16:01:33.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:32 smithi055 bash[9296]: cluster 2020-09-08T16:01:31.941587+0000 mon.a (mon.0) 1903 : cluster [DBG] osdmap e574: 12 total, 12 up, 12 in 2020-09-08T16:01:33.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:32 smithi055 bash[9296]: cluster 2020-09-08T16:01:31.977935+0000 mgr.x (mgr.24118) 1026 : cluster [DBG] pgmap v1516: 17 pgs: 17 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T16:01:33.412 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 53 16 102608 102592 120.97 70 0.00643442 0.00826433 2020-09-08T16:01:33.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:32 smithi104 bash[9203]: cluster 2020-09-08T16:01:31.941587+0000 mon.a (mon.0) 1903 : cluster [DBG] osdmap e574: 12 total, 12 up, 12 in 2020-09-08T16:01:33.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:32 smithi104 bash[9203]: cluster 2020-09-08T16:01:31.977935+0000 mgr.x (mgr.24118) 1026 : cluster [DBG] pgmap v1516: 17 pgs: 17 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T16:01:33.708 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:01:33.817 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:33.946 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:34.048 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:01:34.167 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:33 smithi055 bash[11956]: debug 2020-09-08T16:01:33.942+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:34.168 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:01:33 smithi055 bash[16646]: debug 2020-09-08T16:01:33.710+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:34.169 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:33 smithi055 bash[9296]: cluster 2020-09-08T16:01:33.647870+0000 osd.2 (osd.2) 87 : cluster [DBG] 7.8 scrub ok 2020-09-08T16:01:34.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:33 smithi166 bash[9446]: cluster 2020-09-08T16:01:33.647870+0000 osd.2 (osd.2) 87 : cluster [DBG] 7.8 scrub ok 2020-09-08T16:01:34.411 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 54 16 103910 103894 120.236 81.375 0.00724557 0.00830946 2020-09-08T16:01:34.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:33 smithi104 bash[9203]: cluster 2020-09-08T16:01:33.647870+0000 osd.2 (osd.2) 87 : cluster [DBG] 7.8 scrub ok 2020-09-08T16:01:35.135 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:01:35.237 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:01:35.259 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:34 smithi104 bash[9203]: cluster 2020-09-08T16:01:33.773749+0000 osd.10 (osd.10) 71 : cluster [DBG] 7.1 scrub starts 2020-09-08T16:01:35.260 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:34 smithi104 bash[9203]: cluster 2020-09-08T16:01:33.978418+0000 mgr.x (mgr.24118) 1027 : cluster [DBG] pgmap v1517: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T16:01:35.260 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:01:35 smithi104 bash[15273]: debug 2020-09-08T16:01:35.131+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:35.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:34 smithi166 bash[9446]: cluster 2020-09-08T16:01:33.773749+0000 osd.10 (osd.10) 71 : cluster [DBG] 7.1 scrub starts 2020-09-08T16:01:35.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:34 smithi166 bash[9446]: cluster 2020-09-08T16:01:33.978418+0000 mgr.x (mgr.24118) 1027 : cluster [DBG] pgmap v1517: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T16:01:35.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:34 smithi055 bash[9296]: cluster 2020-09-08T16:01:33.773749+0000 osd.10 (osd.10) 71 : cluster [DBG] 7.1 scrub starts 2020-09-08T16:01:35.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:34 smithi055 bash[9296]: cluster 2020-09-08T16:01:33.978418+0000 mgr.x (mgr.24118) 1027 : cluster [DBG] pgmap v1517: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T16:01:35.411 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 55 16 105132 105116 119.439 76.375 0.00674481 0.00837022 2020-09-08T16:01:35.501 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:01:35 smithi104 bash[17588]: debug 2020-09-08T16:01:35.503+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:35.505 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:01:35.531 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T16:01:35.610 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:01:35.645 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2020-09-08T16:01:35.724 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:01:35.733 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:01:35.831 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:01:35.834 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:01:35.913 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:01:35 smithi055 bash[19003]: debug 2020-09-08T16:01:35.730+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:36.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:35 smithi104 bash[9203]: cluster 2020-09-08T16:01:35.348466+0000 osd.10 (osd.10) 72 : cluster [DBG] 7.1 scrub ok 2020-09-08T16:01:36.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:35 smithi166 bash[9446]: cluster 2020-09-08T16:01:35.348466+0000 osd.10 (osd.10) 72 : cluster [DBG] 7.1 scrub ok 2020-09-08T16:01:36.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:36 smithi055 bash[9296]: cluster 2020-09-08T16:01:35.348466+0000 osd.10 (osd.10) 72 : cluster [DBG] 7.1 scrub ok 2020-09-08T16:01:36.411 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 56 16 106199 106183 118.497 66.6875 0.0179263 0.00843688 2020-09-08T16:01:36.539 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:01:36.643 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:01:36.787 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:01:36.888 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:01:36.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:01:36 smithi104 bash[17588]: debug 2020-09-08T16:01:36.539+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:36.994 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:01:36 smithi166 bash[18960]: debug 2020-09-08T16:01:36.782+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:36.995 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:36 smithi166 bash[9446]: cluster 2020-09-08T16:01:35.978965+0000 mgr.x (mgr.24118) 1028 : cluster [DBG] pgmap v1518: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s wr, 1.14k op/s 2020-09-08T16:01:37.024 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:01:37.132 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:01:37.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:01:37 smithi055 bash[19003]: debug 2020-09-08T16:01:37.029+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:37.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:37 smithi055 bash[9296]: cluster 2020-09-08T16:01:35.978965+0000 mgr.x (mgr.24118) 1028 : cluster [DBG] pgmap v1518: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s wr, 1.14k op/s 2020-09-08T16:01:37.256 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:01:37 smithi166 bash[18960]: debug 2020-09-08T16:01:37.250+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:37.257 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:01:37.360 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:37.412 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 57 16 107506 107490 117.851 81.6875 0.0214397 0.00847999 2020-09-08T16:01:37.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:36 smithi104 bash[9203]: cluster 2020-09-08T16:01:35.978965+0000 mgr.x (mgr.24118) 1028 : cluster [DBG] pgmap v1518: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s wr, 1.14k op/s 2020-09-08T16:01:37.488 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:37.598 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:01:37.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:37 smithi055 bash[11956]: debug 2020-09-08T16:01:37.489+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:38.278 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:01:38.282 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:38 smithi104 bash[9203]: cluster 2020-09-08T16:01:37.450910+0000 osd.6 (osd.6) 89 : cluster [DBG] 7.d scrub starts 2020-09-08T16:01:38.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:38 smithi166 bash[9446]: cluster 2020-09-08T16:01:37.450910+0000 osd.6 (osd.6) 89 : cluster [DBG] 7.d scrub starts 2020-09-08T16:01:38.385 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:01:38.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:38 smithi055 bash[9296]: cluster 2020-09-08T16:01:37.450910+0000 osd.6 (osd.6) 89 : cluster [DBG] 7.d scrub starts 2020-09-08T16:01:38.411 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 58 16 108778 108762 117.189 79.5 0.0221793 0.00853135 2020-09-08T16:01:38.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:01:38 smithi104 bash[19922]: debug 2020-09-08T16:01:38.279+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:39.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:39 smithi166 bash[9446]: cluster 2020-09-08T16:01:37.504389+0000 osd.8 (osd.8) 77 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:39.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:39 smithi166 bash[9446]: cluster 2020-09-08T16:01:37.504464+0000 osd.8 (osd.8) 78 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:39.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:39 smithi166 bash[9446]: cluster 2020-09-08T16:01:37.979486+0000 mgr.x (mgr.24118) 1029 : cluster [DBG] pgmap v1519: 17 pgs: 17 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 94 MiB/s wr, 1.50k op/s 2020-09-08T16:01:39.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:39 smithi166 bash[9446]: cluster 2020-09-08T16:01:38.011663+0000 mon.a (mon.0) 1904 : cluster [DBG] osdmap e575: 12 total, 12 up, 12 in 2020-09-08T16:01:39.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:39 smithi055 bash[9296]: cluster 2020-09-08T16:01:37.504389+0000 osd.8 (osd.8) 77 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:39.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:39 smithi055 bash[9296]: cluster 2020-09-08T16:01:37.504464+0000 osd.8 (osd.8) 78 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:39.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:39 smithi055 bash[9296]: cluster 2020-09-08T16:01:37.979486+0000 mgr.x (mgr.24118) 1029 : cluster [DBG] pgmap v1519: 17 pgs: 17 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 94 MiB/s wr, 1.50k op/s 2020-09-08T16:01:39.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:39 smithi055 bash[9296]: cluster 2020-09-08T16:01:38.011663+0000 mon.a (mon.0) 1904 : cluster [DBG] osdmap e575: 12 total, 12 up, 12 in 2020-09-08T16:01:39.411 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 59 16 109906 109890 116.398 70.5 0.00569831 0.00858293 2020-09-08T16:01:39.454 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:39 smithi104 bash[9203]: cluster 2020-09-08T16:01:37.504389+0000 osd.8 (osd.8) 77 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:39.454 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:39 smithi104 bash[9203]: cluster 2020-09-08T16:01:37.504464+0000 osd.8 (osd.8) 78 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:39.454 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:39 smithi104 bash[9203]: cluster 2020-09-08T16:01:37.979486+0000 mgr.x (mgr.24118) 1029 : cluster [DBG] pgmap v1519: 17 pgs: 17 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 94 MiB/s wr, 1.50k op/s 2020-09-08T16:01:39.455 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:39 smithi104 bash[9203]: cluster 2020-09-08T16:01:38.011663+0000 mon.a (mon.0) 1904 : cluster [DBG] osdmap e575: 12 total, 12 up, 12 in 2020-09-08T16:01:39.526 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:01:39.629 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:01:39.918 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:01:39 smithi104 bash[15273]: debug 2020-09-08T16:01:39.527+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:39.919 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:01:40.022 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:01:40.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:01:39 smithi104 bash[19922]: debug 2020-09-08T16:01:39.919+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:40.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:40 smithi104 bash[9203]: cluster 2020-09-08T16:01:39.028620+0000 osd.0 (osd.0) 89 : cluster [DBG] 7.4 scrub starts 2020-09-08T16:01:40.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:40 smithi104 bash[9203]: cluster 2020-09-08T16:01:39.479004+0000 osd.6 (osd.6) 90 : cluster [DBG] 7.d scrub ok 2020-09-08T16:01:40.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:40 smithi104 bash[9203]: cluster 2020-09-08T16:01:39.792971+0000 osd.3 (osd.3) 97 : cluster [DBG] 7.2 scrub starts 2020-09-08T16:01:40.269 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:01:40.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:40 smithi166 bash[9446]: cluster 2020-09-08T16:01:39.028620+0000 osd.0 (osd.0) 89 : cluster [DBG] 7.4 scrub starts 2020-09-08T16:01:40.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:40 smithi166 bash[9446]: cluster 2020-09-08T16:01:39.479004+0000 osd.6 (osd.6) 90 : cluster [DBG] 7.d scrub ok 2020-09-08T16:01:40.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:40 smithi166 bash[9446]: cluster 2020-09-08T16:01:39.792971+0000 osd.3 (osd.3) 97 : cluster [DBG] 7.2 scrub starts 2020-09-08T16:01:40.373 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:40.388 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:40 smithi055 bash[9296]: cluster 2020-09-08T16:01:39.028620+0000 osd.0 (osd.0) 89 : cluster [DBG] 7.4 scrub starts 2020-09-08T16:01:40.389 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:40 smithi055 bash[9296]: cluster 2020-09-08T16:01:39.479004+0000 osd.6 (osd.6) 90 : cluster [DBG] 7.d scrub ok 2020-09-08T16:01:40.389 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:40 smithi055 bash[9296]: cluster 2020-09-08T16:01:39.792971+0000 osd.3 (osd.3) 97 : cluster [DBG] 7.2 scrub starts 2020-09-08T16:01:40.412 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 16:01:40.416189 min lat: 0.00178141 max lat: 0.479134 avg lat: 0.0086499 2020-09-08T16:01:40.412 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T16:01:40.412 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 60 16 110981 110965 115.578 67.1875 0.00540751 0.0086499 2020-09-08T16:01:40.530 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:01:40 smithi104 bash[17588]: debug 2020-09-08T16:01:40.271+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:41.007 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:41 smithi104 bash[9203]: cluster 2020-09-08T16:01:39.979953+0000 mgr.x (mgr.24118) 1030 : cluster [DBG] pgmap v1521: 17 pgs: 17 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 MiB/s wr, 1.32k op/s 2020-09-08T16:01:41.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:41 smithi166 bash[9446]: cluster 2020-09-08T16:01:39.979953+0000 mgr.x (mgr.24118) 1030 : cluster [DBG] pgmap v1521: 17 pgs: 17 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 MiB/s wr, 1.32k op/s 2020-09-08T16:01:41.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:41 smithi055 bash[9296]: cluster 2020-09-08T16:01:39.979953+0000 mgr.x (mgr.24118) 1030 : cluster [DBG] pgmap v1521: 17 pgs: 17 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 MiB/s wr, 1.32k op/s 2020-09-08T16:01:41.411 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 61 16 111885 111869 114.609 56.5 0.00971753 0.00872328 2020-09-08T16:01:42.205 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:01:42.236 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:42.243 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:01:42.284 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:01:42.313 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:01:42.347 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:42.348 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:01:42.376 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:01:42.411 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:01:42.412 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 62 16 112941 112925 113.825 66 0.0105087 0.0087751 2020-09-08T16:01:42.448 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:01:42.476 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:42.486 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:01:42.520 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:01:42.555 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:01:42.582 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:42.595 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:01:42.595 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:01:42.596 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:01:42.596 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:01:42.596 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:01:42.596 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:01:42.597 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:01:42.597 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:01:42.597 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:01:42.597 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:01:42.597 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:01:42.598 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:01:42.598 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:01:42.598 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:42 smithi055 bash[11956]: debug 2020-09-08T16:01:42.241+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:42.598 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:42 smithi055 bash[11956]: debug 2020-09-08T16:01:42.477+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:42.689 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:42.795 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:42.906 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:42 smithi055 bash[11956]: debug 2020-09-08T16:01:42.693+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:42.915 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:43.021 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:01:43.128 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:01:43.233 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:43.310 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:43 smithi104 bash[9203]: cluster 2020-09-08T16:01:41.923716+0000 osd.0 (osd.0) 90 : cluster [DBG] 7.4 scrub ok 2020-09-08T16:01:43.311 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:43 smithi104 bash[9203]: cluster 2020-09-08T16:01:41.980444+0000 mgr.x (mgr.24118) 1031 : cluster [DBG] pgmap v1522: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T16:01:43.311 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:43 smithi104 bash[9203]: cluster 2020-09-08T16:01:42.031161+0000 osd.0 (osd.0) 91 : cluster [DBG] 7.6 deep-scrub starts 2020-09-08T16:01:43.311 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:43 smithi104 bash[9203]: audit 2020-09-08T16:01:42.175343+0000 mon.a (mon.0) 1905 : audit [DBG] from='client.? v1:172.21.15.104:0/3357028040' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:01:43.311 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:43 smithi104 bash[9203]: cluster 2020-09-08T16:01:42.403798+0000 osd.3 (osd.3) 98 : cluster [DBG] 7.2 scrub ok 2020-09-08T16:01:43.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:43 smithi166 bash[9446]: cluster 2020-09-08T16:01:41.923716+0000 osd.0 (osd.0) 90 : cluster [DBG] 7.4 scrub ok 2020-09-08T16:01:43.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:43 smithi166 bash[9446]: cluster 2020-09-08T16:01:41.980444+0000 mgr.x (mgr.24118) 1031 : cluster [DBG] pgmap v1522: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T16:01:43.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:43 smithi166 bash[9446]: cluster 2020-09-08T16:01:42.031161+0000 osd.0 (osd.0) 91 : cluster [DBG] 7.6 deep-scrub starts 2020-09-08T16:01:43.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:43 smithi166 bash[9446]: audit 2020-09-08T16:01:42.175343+0000 mon.a (mon.0) 1905 : audit [DBG] from='client.? v1:172.21.15.104:0/3357028040' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:01:43.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:43 smithi166 bash[9446]: cluster 2020-09-08T16:01:42.403798+0000 osd.3 (osd.3) 98 : cluster [DBG] 7.2 scrub ok 2020-09-08T16:01:43.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:42 smithi055 bash[11956]: debug 2020-09-08T16:01:42.917+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:43.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:01:43 smithi055 bash[16646]: debug 2020-09-08T16:01:43.129+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:43.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:43 smithi055 bash[9296]: cluster 2020-09-08T16:01:41.923716+0000 osd.0 (osd.0) 90 : cluster [DBG] 7.4 scrub ok 2020-09-08T16:01:43.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:43 smithi055 bash[9296]: cluster 2020-09-08T16:01:41.980444+0000 mgr.x (mgr.24118) 1031 : cluster [DBG] pgmap v1522: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T16:01:43.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:43 smithi055 bash[9296]: cluster 2020-09-08T16:01:42.031161+0000 osd.0 (osd.0) 91 : cluster [DBG] 7.6 deep-scrub starts 2020-09-08T16:01:43.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:43 smithi055 bash[9296]: audit 2020-09-08T16:01:42.175343+0000 mon.a (mon.0) 1905 : audit [DBG] from='client.? v1:172.21.15.104:0/3357028040' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:01:43.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:43 smithi055 bash[9296]: cluster 2020-09-08T16:01:42.403798+0000 osd.3 (osd.3) 98 : cluster [DBG] 7.2 scrub ok 2020-09-08T16:01:43.414 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 63 16 113946 113930 113.015 62.8125 0.00603801 0.0088394 2020-09-08T16:01:43.619 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:43.722 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:01:43.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:01:43 smithi104 bash[22305]: debug 2020-09-08T16:01:43.622+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:44.412 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 64 16 114668 114652 111.954 45.125 0.00619837 0.00892885 2020-09-08T16:01:44.430 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:01:44.535 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:01:44.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:01:44 smithi104 bash[15273]: debug 2020-09-08T16:01:44.430+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:45.312 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:45 smithi104 bash[9203]: cluster 2020-09-08T16:01:43.980972+0000 mgr.x (mgr.24118) 1032 : cluster [DBG] pgmap v1523: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s wr, 1.16k op/s 2020-09-08T16:01:45.313 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:01:45.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:45 smithi166 bash[9446]: cluster 2020-09-08T16:01:43.980972+0000 mgr.x (mgr.24118) 1032 : cluster [DBG] pgmap v1523: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s wr, 1.16k op/s 2020-09-08T16:01:45.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:45 smithi055 bash[9296]: cluster 2020-09-08T16:01:43.980972+0000 mgr.x (mgr.24118) 1032 : cluster [DBG] pgmap v1523: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s wr, 1.16k op/s 2020-09-08T16:01:45.414 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 65 16 115536 115520 111.067 54.25 0.00765595 0.00899387 2020-09-08T16:01:45.415 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:01:45.552 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:01:45.656 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:01:45.667 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:01:45 smithi104 bash[17588]: debug 2020-09-08T16:01:45.314+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:45.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:01:45 smithi166 bash[11928]: debug 2020-09-08T16:01:45.550+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:46.255 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:01:46.359 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:01:46.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:46 smithi166 bash[9446]: cluster 2020-09-08T16:01:45.419099+0000 osd.6 (osd.6) 91 : cluster [DBG] 7.c scrub starts 2020-09-08T16:01:46.370 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:46 smithi055 bash[9296]: cluster 2020-09-08T16:01:45.419099+0000 osd.6 (osd.6) 91 : cluster [DBG] 7.c scrub starts 2020-09-08T16:01:46.412 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 66 16 116237 116221 110.048 43.8125 0.00667254 0.00906828 2020-09-08T16:01:46.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:01:46 smithi104 bash[19922]: debug 2020-09-08T16:01:46.254+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:46.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:46 smithi104 bash[9203]: cluster 2020-09-08T16:01:45.419099+0000 osd.6 (osd.6) 91 : cluster [DBG] 7.c scrub starts 2020-09-08T16:01:46.496 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:01:46.600 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:01:46.656 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:01:46 smithi055 bash[14301]: debug 2020-09-08T16:01:46.497+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:46.708 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:01:46.812 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:46.926 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:47.023 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:46 smithi055 bash[11956]: debug 2020-09-08T16:01:46.929+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:47.023 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:47 smithi055 bash[9296]: cluster 2020-09-08T16:01:45.237583+0000 osd.1 (osd.1) 114 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:47.024 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:47 smithi055 bash[9296]: cluster 2020-09-08T16:01:45.237683+0000 osd.1 (osd.1) 115 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:47.024 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:47 smithi055 bash[9296]: cluster 2020-09-08T16:01:45.981494+0000 mgr.x (mgr.24118) 1033 : cluster [DBG] pgmap v1524: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 63 MiB/s wr, 1.01k op/s 2020-09-08T16:01:47.024 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:47 smithi055 bash[9296]: cluster 2020-09-08T16:01:46.028104+0000 mon.a (mon.0) 1906 : cluster [DBG] osdmap e576: 12 total, 12 up, 12 in 2020-09-08T16:01:47.025 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:01:46 smithi055 bash[14301]: debug 2020-09-08T16:01:46.709+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:47.029 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:01:47.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:47 smithi104 bash[9203]: cluster 2020-09-08T16:01:45.237583+0000 osd.1 (osd.1) 114 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:47.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:47 smithi104 bash[9203]: cluster 2020-09-08T16:01:45.237683+0000 osd.1 (osd.1) 115 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:47.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:47 smithi104 bash[9203]: cluster 2020-09-08T16:01:45.981494+0000 mgr.x (mgr.24118) 1033 : cluster [DBG] pgmap v1524: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 63 MiB/s wr, 1.01k op/s 2020-09-08T16:01:47.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:47 smithi104 bash[9203]: cluster 2020-09-08T16:01:46.028104+0000 mon.a (mon.0) 1906 : cluster [DBG] osdmap e576: 12 total, 12 up, 12 in 2020-09-08T16:01:47.214 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:01:47.319 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:47.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:47 smithi166 bash[9446]: cluster 2020-09-08T16:01:45.237583+0000 osd.1 (osd.1) 114 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:47.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:47 smithi166 bash[9446]: cluster 2020-09-08T16:01:45.237683+0000 osd.1 (osd.1) 115 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:47.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:47 smithi166 bash[9446]: cluster 2020-09-08T16:01:45.981494+0000 mgr.x (mgr.24118) 1033 : cluster [DBG] pgmap v1524: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 63 MiB/s wr, 1.01k op/s 2020-09-08T16:01:47.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:47 smithi166 bash[9446]: cluster 2020-09-08T16:01:46.028104+0000 mon.a (mon.0) 1906 : cluster [DBG] osdmap e576: 12 total, 12 up, 12 in 2020-09-08T16:01:47.419 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 67 16 116978 116962 109.096 46.3125 0.00951035 0.00915831 2020-09-08T16:01:47.453 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:01:47.463 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:01:47 smithi104 bash[19922]: debug 2020-09-08T16:01:47.210+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:47.499 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:47.500 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T16:01:47.500 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:01:47.604 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:47.626 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:01:48.047 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T16:01:48.268 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:01:47 smithi104 bash[22305]: debug 2020-09-08T16:01:47.490+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:48.268 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:48 smithi104 bash[9203]: cluster 2020-09-08T16:01:46.860505+0000 osd.6 (osd.6) 92 : cluster [DBG] 7.c scrub ok 2020-09-08T16:01:48.268 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:48 smithi104 bash[9203]: audit 2020-09-08T16:01:47.780187+0000 mon.c (mon.2) 77 : audit [INF] from='client.? v1:172.21.15.104:0/3893145848' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:01:48.269 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:48 smithi104 bash[9203]: audit 2020-09-08T16:01:47.782918+0000 mon.a (mon.0) 1907 : audit [INF] from='client.36155 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:01:48.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:48 smithi166 bash[9446]: cluster 2020-09-08T16:01:46.860505+0000 osd.6 (osd.6) 92 : cluster [DBG] 7.c scrub ok 2020-09-08T16:01:48.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:48 smithi166 bash[9446]: audit 2020-09-08T16:01:47.780187+0000 mon.c (mon.2) 77 : audit [INF] from='client.? v1:172.21.15.104:0/3893145848' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:01:48.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:48 smithi166 bash[9446]: audit 2020-09-08T16:01:47.782918+0000 mon.a (mon.0) 1907 : audit [INF] from='client.36155 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:01:48.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:48 smithi055 bash[9296]: cluster 2020-09-08T16:01:46.860505+0000 osd.6 (osd.6) 92 : cluster [DBG] 7.c scrub ok 2020-09-08T16:01:48.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:48 smithi055 bash[9296]: audit 2020-09-08T16:01:47.780187+0000 mon.c (mon.2) 77 : audit [INF] from='client.? v1:172.21.15.104:0/3893145848' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:01:48.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:48 smithi055 bash[9296]: audit 2020-09-08T16:01:47.782918+0000 mon.a (mon.0) 1907 : audit [INF] from='client.36155 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:01:48.418 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 68 16 117696 117680 108.152 44.875 0.00761847 0.00924015 2020-09-08T16:01:49.363 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:49 smithi104 bash[9203]: cluster 2020-09-08T16:01:47.981998+0000 mgr.x (mgr.24118) 1034 : cluster [DBG] pgmap v1526: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s wr, 994 op/s 2020-09-08T16:01:49.364 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:49 smithi104 bash[9203]: cluster 2020-09-08T16:01:48.028675+0000 mon.a (mon.0) 1908 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:01:49.364 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:49 smithi104 bash[9203]: audit 2020-09-08T16:01:48.038994+0000 mon.a (mon.0) 1909 : audit [INF] from='client.36155 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:01:49.364 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:49 smithi104 bash[9203]: cluster 2020-09-08T16:01:48.039070+0000 mon.a (mon.0) 1910 : cluster [DBG] osdmap e577: 12 total, 12 up, 12 in 2020-09-08T16:01:49.364 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:49 smithi104 bash[9203]: cluster 2020-09-08T16:01:48.118314+0000 osd.6 (osd.6) 93 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:49.364 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:49 smithi104 bash[9203]: cluster 2020-09-08T16:01:48.118402+0000 osd.6 (osd.6) 94 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:49.365 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:49.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:49 smithi166 bash[9446]: cluster 2020-09-08T16:01:47.981998+0000 mgr.x (mgr.24118) 1034 : cluster [DBG] pgmap v1526: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s wr, 994 op/s 2020-09-08T16:01:49.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:49 smithi166 bash[9446]: cluster 2020-09-08T16:01:48.028675+0000 mon.a (mon.0) 1908 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:01:49.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:49 smithi166 bash[9446]: audit 2020-09-08T16:01:48.038994+0000 mon.a (mon.0) 1909 : audit [INF] from='client.36155 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:01:49.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:49 smithi166 bash[9446]: cluster 2020-09-08T16:01:48.039070+0000 mon.a (mon.0) 1910 : cluster [DBG] osdmap e577: 12 total, 12 up, 12 in 2020-09-08T16:01:49.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:49 smithi166 bash[9446]: cluster 2020-09-08T16:01:48.118314+0000 osd.6 (osd.6) 93 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:49.373 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:49 smithi166 bash[9446]: cluster 2020-09-08T16:01:48.118402+0000 osd.6 (osd.6) 94 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:49.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:49 smithi055 bash[9296]: cluster 2020-09-08T16:01:47.981998+0000 mgr.x (mgr.24118) 1034 : cluster [DBG] pgmap v1526: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s wr, 994 op/s 2020-09-08T16:01:49.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:49 smithi055 bash[9296]: cluster 2020-09-08T16:01:48.028675+0000 mon.a (mon.0) 1908 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:01:49.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:49 smithi055 bash[9296]: audit 2020-09-08T16:01:48.038994+0000 mon.a (mon.0) 1909 : audit [INF] from='client.36155 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:01:49.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:49 smithi055 bash[9296]: cluster 2020-09-08T16:01:48.039070+0000 mon.a (mon.0) 1910 : cluster [DBG] osdmap e577: 12 total, 12 up, 12 in 2020-09-08T16:01:49.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:49 smithi055 bash[9296]: cluster 2020-09-08T16:01:48.118314+0000 osd.6 (osd.6) 93 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:49.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:49 smithi055 bash[9296]: cluster 2020-09-08T16:01:48.118402+0000 osd.6 (osd.6) 94 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:49.413 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 69 16 118461 118445 107.277 47.8125 0.0159767 0.00931474 2020-09-08T16:01:49.473 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:49.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:01:49 smithi104 bash[22305]: debug 2020-09-08T16:01:49.366+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:50.175 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:50.278 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:01:50.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:50 smithi166 bash[9446]: cluster 2020-09-08T16:01:49.046989+0000 mon.a (mon.0) 1911 : cluster [DBG] osdmap e578: 12 total, 12 up, 12 in 2020-09-08T16:01:50.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:50 smithi055 bash[9296]: cluster 2020-09-08T16:01:49.046989+0000 mon.a (mon.0) 1911 : cluster [DBG] osdmap e578: 12 total, 12 up, 12 in 2020-09-08T16:01:50.413 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 70 16 119135 119119 106.346 42.125 0.00689881 0.00940033 2020-09-08T16:01:50.426 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:01:50.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:50 smithi104 bash[9203]: cluster 2020-09-08T16:01:49.046989+0000 mon.a (mon.0) 1911 : cluster [DBG] osdmap e578: 12 total, 12 up, 12 in 2020-09-08T16:01:50.427 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:01:50 smithi104 bash[22305]: debug 2020-09-08T16:01:50.178+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:50.527 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:50.637 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:50.743 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:50.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:50 smithi055 bash[11956]: debug 2020-09-08T16:01:50.641+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:50.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:01:50 smithi055 bash[16646]: debug 2020-09-08T16:01:50.421+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:50.985 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2020-09-08T16:01:50.995 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:01:51.005 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:01:51 smithi104 bash[22305]: debug 2020-09-08T16:01:50.994+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:51.106 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:51.157 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:01:51.218 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:51.324 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:01:51.337 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:51 smithi055 bash[9296]: cluster 2020-09-08T16:01:49.982648+0000 mgr.x (mgr.24118) 1035 : cluster [DBG] pgmap v1529: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s wr, 685 op/s 2020-09-08T16:01:51.337 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:51 smithi055 bash[11956]: debug 2020-09-08T16:01:51.221+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:51.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:51 smithi166 bash[9446]: cluster 2020-09-08T16:01:49.982648+0000 mgr.x (mgr.24118) 1035 : cluster [DBG] pgmap v1529: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s wr, 685 op/s 2020-09-08T16:01:51.413 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 71 16 119894 119878 105.517 47.4375 0.00612156 0.00944789 2020-09-08T16:01:51.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:51 smithi104 bash[9203]: cluster 2020-09-08T16:01:49.982648+0000 mgr.x (mgr.24118) 1035 : cluster [DBG] pgmap v1529: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s wr, 685 op/s 2020-09-08T16:01:51.433 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:01:51.538 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:51.658 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:01:51 smithi055 bash[19003]: debug 2020-09-08T16:01:51.437+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:51.686 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:51.791 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:01:52.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:51 smithi055 bash[11956]: debug 2020-09-08T16:01:51.689+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:52.250 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:01:52.358 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:52.413 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 72 16 120316 120300 104.417 26.375 0.00674194 0.00956953 2020-09-08T16:01:52.471 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:52.585 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:01:52.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:52 smithi055 bash[11956]: debug 2020-09-08T16:01:52.473+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:52.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:01:52 smithi104 bash[19922]: debug 2020-09-08T16:01:52.250+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:52.698 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:01:52.804 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:01:53.009 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T16:01:53.040 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:01:53.047 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:01:52 smithi055 bash[14301]: debug 2020-09-08T16:01:52.701+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:53.145 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:53.175 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:01:53.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:01:53 smithi104 bash[17588]: debug 2020-09-08T16:01:53.034+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:53.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:53 smithi104 bash[9203]: cluster 2020-09-08T16:01:51.983234+0000 mgr.x (mgr.24118) 1036 : cluster [DBG] pgmap v1530: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s wr, 882 op/s 2020-09-08T16:01:53.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:53 smithi104 bash[9203]: cluster 2020-09-08T16:01:52.118566+0000 osd.11 (osd.11) 94 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:53.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:53 smithi104 bash[9203]: cluster 2020-09-08T16:01:52.118664+0000 osd.11 (osd.11) 95 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:53.279 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:53.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:53 smithi166 bash[9446]: cluster 2020-09-08T16:01:51.983234+0000 mgr.x (mgr.24118) 1036 : cluster [DBG] pgmap v1530: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s wr, 882 op/s 2020-09-08T16:01:53.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:53 smithi166 bash[9446]: cluster 2020-09-08T16:01:52.118566+0000 osd.11 (osd.11) 94 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:53.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:53 smithi166 bash[9446]: cluster 2020-09-08T16:01:52.118664+0000 osd.11 (osd.11) 95 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:53.382 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:01:53.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:53 smithi055 bash[11956]: debug 2020-09-08T16:01:53.281+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:53.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:53 smithi055 bash[9296]: cluster 2020-09-08T16:01:51.983234+0000 mgr.x (mgr.24118) 1036 : cluster [DBG] pgmap v1530: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s wr, 882 op/s 2020-09-08T16:01:53.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:53 smithi055 bash[9296]: cluster 2020-09-08T16:01:52.118566+0000 osd.11 (osd.11) 94 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:53.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:53 smithi055 bash[9296]: cluster 2020-09-08T16:01:52.118664+0000 osd.11 (osd.11) 95 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:53.413 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 73 16 121029 121013 103.597 44.5625 0.00525462 0.00964426 2020-09-08T16:01:53.921 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:01:54.030 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:01:54.167 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:01:54.268 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:01:54.273 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:54 smithi104 bash[9203]: cluster 2020-09-08T16:01:53.060379+0000 mon.a (mon.0) 1912 : cluster [DBG] osdmap e579: 12 total, 12 up, 12 in 2020-09-08T16:01:54.273 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:01:53 smithi104 bash[15273]: debug 2020-09-08T16:01:53.922+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:54.374 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:54 smithi166 bash[9446]: cluster 2020-09-08T16:01:53.060379+0000 mon.a (mon.0) 1912 : cluster [DBG] osdmap e579: 12 total, 12 up, 12 in 2020-09-08T16:01:54.375 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:01:54 smithi166 bash[14254]: debug 2020-09-08T16:01:54.162+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:54.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:54 smithi055 bash[9296]: cluster 2020-09-08T16:01:53.060379+0000 mon.a (mon.0) 1912 : cluster [DBG] osdmap e579: 12 total, 12 up, 12 in 2020-09-08T16:01:54.413 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 74 16 121659 121643 102.729 39.375 0.0124089 0.00972942 2020-09-08T16:01:54.414 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:01:54.521 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:01:54.656 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:01:54.763 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:01:54.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:01:54 smithi166 bash[11928]: debug 2020-09-08T16:01:54.414+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:54.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:01:54 smithi055 bash[16646]: debug 2020-09-08T16:01:54.657+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:55.059 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:55 smithi104 bash[9203]: cluster 2020-09-08T16:01:53.983691+0000 mgr.x (mgr.24118) 1037 : cluster [DBG] pgmap v1532: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s wr, 653 op/s 2020-09-08T16:01:55.236 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:01:55.340 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:55.348 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:55 smithi055 bash[9296]: cluster 2020-09-08T16:01:53.983691+0000 mgr.x (mgr.24118) 1037 : cluster [DBG] pgmap v1532: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s wr, 653 op/s 2020-09-08T16:01:55.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:55 smithi166 bash[9446]: cluster 2020-09-08T16:01:53.983691+0000 mgr.x (mgr.24118) 1037 : cluster [DBG] pgmap v1532: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s wr, 653 op/s 2020-09-08T16:01:55.413 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 75 16 122381 122365 101.961 45.125 0.00713823 0.00979058 2020-09-08T16:01:55.459 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:55.565 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:01:55.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:55 smithi055 bash[11956]: debug 2020-09-08T16:01:55.461+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:55.667 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:01:55 smithi104 bash[19922]: debug 2020-09-08T16:01:55.238+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:56.165 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:01:56.270 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:56.382 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:56.424 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 76 16 123012 122996 101.138 39.4375 0.00772765 0.00987734 2020-09-08T16:01:56.427 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:01:56 smithi104 bash[15273]: debug 2020-09-08T16:01:56.162+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:56.487 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:01:56.592 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:56 smithi055 bash[11956]: debug 2020-09-08T16:01:56.385+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:56.595 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:01:56 smithi055 bash[14301]: debug 2020-09-08T16:01:56.597+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:56.596 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:01:56.704 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:01:57.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:57 smithi104 bash[9203]: cluster 2020-09-08T16:01:55.984193+0000 mgr.x (mgr.24118) 1038 : cluster [DBG] pgmap v1533: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s wr, 493 op/s 2020-09-08T16:01:57.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:57 smithi166 bash[9446]: cluster 2020-09-08T16:01:55.984193+0000 mgr.x (mgr.24118) 1038 : cluster [DBG] pgmap v1533: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s wr, 493 op/s 2020-09-08T16:01:57.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:57 smithi055 bash[9296]: cluster 2020-09-08T16:01:55.984193+0000 mgr.x (mgr.24118) 1038 : cluster [DBG] pgmap v1533: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s wr, 493 op/s 2020-09-08T16:01:57.414 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 77 16 123617 123601 100.316 37.8125 0.00544637 0.0099456 2020-09-08T16:01:58.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:58 smithi166 bash[9446]: cluster 2020-09-08T16:01:57.123976+0000 osd.4 (osd.4) 61 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:58.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:58 smithi166 bash[9446]: cluster 2020-09-08T16:01:57.124074+0000 osd.4 (osd.4) 62 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:58.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:58 smithi055 bash[9296]: cluster 2020-09-08T16:01:57.123976+0000 osd.4 (osd.4) 61 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:58.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:58 smithi055 bash[9296]: cluster 2020-09-08T16:01:57.124074+0000 osd.4 (osd.4) 62 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:58.413 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 78 16 124366 124350 99.6299 46.8125 0.0129464 0.0100311 2020-09-08T16:01:58.457 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:58 smithi104 bash[9203]: cluster 2020-09-08T16:01:57.123976+0000 osd.4 (osd.4) 61 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:01:58.457 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:58 smithi104 bash[9203]: cluster 2020-09-08T16:01:57.124074+0000 osd.4 (osd.4) 62 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:01:58.661 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:01:58.766 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:01:58.902 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:01:59.005 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:01:59.073 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:01:58 smithi166 bash[11928]: debug 2020-09-08T16:01:58.662+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:59.073 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:59 smithi166 bash[9446]: cluster 2020-09-08T16:01:57.984721+0000 mgr.x (mgr.24118) 1039 : cluster [DBG] pgmap v1534: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 MiB/s wr, 705 op/s 2020-09-08T16:01:59.074 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:01:59 smithi166 bash[9446]: cluster 2020-09-08T16:01:58.079392+0000 mon.a (mon.0) 1913 : cluster [DBG] osdmap e580: 12 total, 12 up, 12 in 2020-09-08T16:01:59.074 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:01:58 smithi166 bash[14254]: debug 2020-09-08T16:01:58.902+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:59.132 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:01:59.240 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:01:59.358 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:01:59.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:01:59 smithi055 bash[19003]: debug 2020-09-08T16:01:59.133+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:59.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:59 smithi055 bash[9296]: cluster 2020-09-08T16:01:57.984721+0000 mgr.x (mgr.24118) 1039 : cluster [DBG] pgmap v1534: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 MiB/s wr, 705 op/s 2020-09-08T16:01:59.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:01:59 smithi055 bash[9296]: cluster 2020-09-08T16:01:58.079392+0000 mon.a (mon.0) 1913 : cluster [DBG] osdmap e580: 12 total, 12 up, 12 in 2020-09-08T16:01:59.415 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 79 16 124994 124978 98.8656 39.25 0.0059321 0.0101023 2020-09-08T16:01:59.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:59 smithi104 bash[9203]: cluster 2020-09-08T16:01:57.984721+0000 mgr.x (mgr.24118) 1039 : cluster [DBG] pgmap v1534: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 MiB/s wr, 705 op/s 2020-09-08T16:01:59.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:01:59 smithi104 bash[9203]: cluster 2020-09-08T16:01:58.079392+0000 mon.a (mon.0) 1913 : cluster [DBG] osdmap e580: 12 total, 12 up, 12 in 2020-09-08T16:01:59.460 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:59.567 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:01:59.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:01:59 smithi166 bash[11928]: debug 2020-09-08T16:01:59.350+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:59.671 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:01:59.817 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:01:59 smithi055 bash[11956]: debug 2020-09-08T16:01:59.569+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:01:59.819 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:01:59.926 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:02:00.036 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:02:00.142 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:02:00.148 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:02:00.150 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:01:59 smithi055 bash[19003]: debug 2020-09-08T16:01:59.821+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:00.179 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:02:00.203 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:02:00.231 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:02:00.254 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:02:00.262 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:02:00.292 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:02:00.322 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:02:00.352 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:02:00.357 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:02:00.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:02:00 smithi166 bash[11928]: debug 2020-09-08T16:02:00.034+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:00.386 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:02:00.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:02:00 smithi055 bash[14301]: debug 2020-09-08T16:02:00.253+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:00.413 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 16:02:00.418196 min lat: 0.00178141 max lat: 0.813986 avg lat: 0.0101924 2020-09-08T16:02:00.414 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T16:02:00.414 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 80 16 125483 125467 98.0118 30.5625 0.00856295 0.0101924 2020-09-08T16:02:00.426 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:02:00.469 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:02:00.498 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:02:00.499 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:02:00.500 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:02:00.500 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:02:00.500 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:02:00.500 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:02:00.501 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:02:00.501 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:02:00.501 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:02:00.501 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:02:00.501 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:02:00.502 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:02:00.502 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:02:00.599 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T16:02:00.599 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T16:02:00.599 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2020-09-08T16:02:00.599 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2020-09-08T16:02:00.600 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T16:02:00.601 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:02:00.630 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:02:00 smithi104 bash[19922]: debug 2020-09-08T16:02:00.602+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:00.706 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:00.827 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:00.931 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:01.103 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:01.161 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:02:00 smithi055 bash[11956]: debug 2020-09-08T16:02:00.829+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:01.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:00 smithi055 bash[9296]: cluster 2020-09-08T16:01:59.985212+0000 mgr.x (mgr.24118) 1040 : cluster [DBG] pgmap v1536: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s wr, 687 op/s 2020-09-08T16:02:01.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:00 smithi055 bash[9296]: audit 2020-09-08T16:02:00.118347+0000 mon.b (mon.1) 112 : audit [DBG] from='client.? v1:172.21.15.104:0/1720466305' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:02:01.205 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:01.326 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:01.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:02:01 smithi166 bash[18960]: debug 2020-09-08T16:02:01.102+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:01.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:00 smithi166 bash[9446]: cluster 2020-09-08T16:01:59.985212+0000 mgr.x (mgr.24118) 1040 : cluster [DBG] pgmap v1536: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s wr, 687 op/s 2020-09-08T16:02:01.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:00 smithi166 bash[9446]: audit 2020-09-08T16:02:00.118347+0000 mon.b (mon.1) 112 : audit [DBG] from='client.? v1:172.21.15.104:0/1720466305' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:02:01.414 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 81 16 126111 126095 97.2862 39.25 0.00712186 0.0102683 2020-09-08T16:02:01.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:00 smithi104 bash[9203]: cluster 2020-09-08T16:01:59.985212+0000 mgr.x (mgr.24118) 1040 : cluster [DBG] pgmap v1536: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s wr, 687 op/s 2020-09-08T16:02:01.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:00 smithi104 bash[9203]: audit 2020-09-08T16:02:00.118347+0000 mon.b (mon.1) 112 : audit [DBG] from='client.? v1:172.21.15.104:0/1720466305' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:02:01.430 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:02:01.562 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:02:01.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:02:01 smithi055 bash[11956]: debug 2020-09-08T16:02:01.329+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:01.667 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:01.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:02:01 smithi166 bash[11928]: debug 2020-09-08T16:02:01.562+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:02.414 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 82 16 126966 126950 96.7514 53.4375 0.00718618 0.010326 2020-09-08T16:02:02.909 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:02.916 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2020-09-08T16:02:03.013 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:03.076 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:02:03.135 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:03 smithi104 bash[9203]: cluster 2020-09-08T16:02:01.985695+0000 mgr.x (mgr.24118) 1041 : cluster [DBG] pgmap v1537: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s wr, 530 op/s 2020-09-08T16:02:03.138 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:03.248 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:02:03.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:03 smithi166 bash[9446]: cluster 2020-09-08T16:02:01.985695+0000 mgr.x (mgr.24118) 1041 : cluster [DBG] pgmap v1537: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s wr, 530 op/s 2020-09-08T16:02:03.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:03 smithi055 bash[9296]: cluster 2020-09-08T16:02:01.985695+0000 mgr.x (mgr.24118) 1041 : cluster [DBG] pgmap v1537: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s wr, 530 op/s 2020-09-08T16:02:03.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:02:02 smithi055 bash[16646]: debug 2020-09-08T16:02:02.913+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:03.414 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 83 16 127699 127683 96.1376 45.8125 0.0155953 0.0103935 2020-09-08T16:02:03.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:02:03 smithi104 bash[17588]: debug 2020-09-08T16:02:03.134+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:03.960 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:02:04.051 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T16:02:04.064 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:02:04.218 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:02:04.322 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:04.330 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:02:04 smithi166 bash[14254]: debug 2020-09-08T16:02:04.205+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:04.331 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:04 smithi166 bash[9446]: audit 2020-09-08T16:02:03.728279+0000 mon.b (mon.1) 113 : audit [INF] from='client.? v1:172.21.15.104:0/1590455487' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:02:04.331 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:04 smithi166 bash[9446]: audit 2020-09-08T16:02:03.728415+0000 mon.a (mon.0) 1914 : audit [INF] from='client.26173 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:02:04.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:04 smithi055 bash[9296]: audit 2020-09-08T16:02:03.728279+0000 mon.b (mon.1) 113 : audit [INF] from='client.? v1:172.21.15.104:0/1590455487' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:02:04.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:04 smithi055 bash[9296]: audit 2020-09-08T16:02:03.728415+0000 mon.a (mon.0) 1914 : audit [INF] from='client.26173 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:02:04.422 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 84 16 128369 128353 95.4916 41.875 0.00770863 0.0104614 2020-09-08T16:02:04.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:02:03 smithi104 bash[22305]: debug 2020-09-08T16:02:03.958+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:04.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:04 smithi104 bash[9203]: audit 2020-09-08T16:02:03.728279+0000 mon.b (mon.1) 113 : audit [INF] from='client.? v1:172.21.15.104:0/1590455487' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:02:04.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:04 smithi104 bash[9203]: audit 2020-09-08T16:02:03.728415+0000 mon.a (mon.0) 1914 : audit [INF] from='client.26173 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:02:04.433 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:04.538 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:04.626 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:02:04 smithi166 bash[18960]: debug 2020-09-08T16:02:04.433+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:04.839 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:02:04.864 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:04.974 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:04.998 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:02:05.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:05 smithi104 bash[9203]: cluster 2020-09-08T16:02:03.986345+0000 mgr.x (mgr.24118) 1042 : cluster [DBG] pgmap v1538: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s wr, 655 op/s 2020-09-08T16:02:05.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:05 smithi104 bash[9203]: cluster 2020-09-08T16:02:04.042367+0000 mon.a (mon.0) 1915 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:02:05.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:05 smithi104 bash[9203]: audit 2020-09-08T16:02:04.050712+0000 mon.a (mon.0) 1916 : audit [INF] from='client.26173 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:02:05.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:05 smithi104 bash[9203]: cluster 2020-09-08T16:02:04.050767+0000 mon.a (mon.0) 1917 : cluster [DBG] osdmap e581: 12 total, 12 up, 12 in 2020-09-08T16:02:05.177 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:02:04 smithi104 bash[15273]: debug 2020-09-08T16:02:04.862+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:05.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:05 smithi166 bash[9446]: cluster 2020-09-08T16:02:03.986345+0000 mgr.x (mgr.24118) 1042 : cluster [DBG] pgmap v1538: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s wr, 655 op/s 2020-09-08T16:02:05.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:05 smithi166 bash[9446]: cluster 2020-09-08T16:02:04.042367+0000 mon.a (mon.0) 1915 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:02:05.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:05 smithi166 bash[9446]: audit 2020-09-08T16:02:04.050712+0000 mon.a (mon.0) 1916 : audit [INF] from='client.26173 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:02:05.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:05 smithi166 bash[9446]: cluster 2020-09-08T16:02:04.050767+0000 mon.a (mon.0) 1917 : cluster [DBG] osdmap e581: 12 total, 12 up, 12 in 2020-09-08T16:02:05.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:05 smithi055 bash[9296]: cluster 2020-09-08T16:02:03.986345+0000 mgr.x (mgr.24118) 1042 : cluster [DBG] pgmap v1538: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s wr, 655 op/s 2020-09-08T16:02:05.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:05 smithi055 bash[9296]: cluster 2020-09-08T16:02:04.042367+0000 mon.a (mon.0) 1915 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:02:05.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:05 smithi055 bash[9296]: audit 2020-09-08T16:02:04.050712+0000 mon.a (mon.0) 1916 : audit [INF] from='client.26173 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:02:05.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:05 smithi055 bash[9296]: cluster 2020-09-08T16:02:04.050767+0000 mon.a (mon.0) 1917 : cluster [DBG] osdmap e581: 12 total, 12 up, 12 in 2020-09-08T16:02:05.414 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 85 16 129273 129257 95.0328 56.5 0.0265244 0.0105209 2020-09-08T16:02:05.923 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:06.027 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:06.136 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:06.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:02:05 smithi104 bash[15273]: debug 2020-09-08T16:02:05.922+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:06.241 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:06.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:02:06 smithi055 bash[11956]: debug 2020-09-08T16:02:06.137+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:06.414 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 86 16 130444 130428 94.7787 73.1875 0.0107648 0.0105488 2020-09-08T16:02:07.047 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:07.151 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:02:07.280 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:02:07.298 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:07 smithi104 bash[9203]: cluster 2020-09-08T16:02:05.986774+0000 mgr.x (mgr.24118) 1043 : cluster [DBG] pgmap v1540: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s wr, 463 op/s 2020-09-08T16:02:07.298 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:07 smithi104 bash[9203]: cluster 2020-09-08T16:02:06.206044+0000 osd.9 (osd.9) 85 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:07.299 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:07 smithi104 bash[9203]: cluster 2020-09-08T16:02:06.206138+0000 osd.9 (osd.9) 86 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:07.299 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:02:07 smithi104 bash[17588]: debug 2020-09-08T16:02:07.046+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:07.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:02:07 smithi166 bash[11928]: debug 2020-09-08T16:02:07.277+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:07 smithi166 bash[9446]: cluster 2020-09-08T16:02:05.986774+0000 mgr.x (mgr.24118) 1043 : cluster [DBG] pgmap v1540: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s wr, 463 op/s 2020-09-08T16:02:07.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:07 smithi166 bash[9446]: cluster 2020-09-08T16:02:06.206044+0000 osd.9 (osd.9) 85 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:07.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:07 smithi166 bash[9446]: cluster 2020-09-08T16:02:06.206138+0000 osd.9 (osd.9) 86 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:07.389 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:07.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:07 smithi055 bash[9296]: cluster 2020-09-08T16:02:05.986774+0000 mgr.x (mgr.24118) 1043 : cluster [DBG] pgmap v1540: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s wr, 463 op/s 2020-09-08T16:02:07.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:07 smithi055 bash[9296]: cluster 2020-09-08T16:02:06.206044+0000 osd.9 (osd.9) 85 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:07.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:07 smithi055 bash[9296]: cluster 2020-09-08T16:02:06.206138+0000 osd.9 (osd.9) 86 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:07.414 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 87 16 131577 131561 94.5032 70.8125 0.0143631 0.0105798 2020-09-08T16:02:07.501 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:07.606 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:07.745 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:02:07 smithi166 bash[18960]: debug 2020-09-08T16:02:07.497+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:07.748 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:07.854 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:08.051 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:02:07 smithi166 bash[18960]: debug 2020-09-08T16:02:07.745+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:08.342 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:08 smithi104 bash[9203]: cluster 2020-09-08T16:02:07.060361+0000 mon.a (mon.0) 1918 : cluster [DBG] osdmap e582: 12 total, 12 up, 12 in 2020-09-08T16:02:08.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:08 smithi166 bash[9446]: cluster 2020-09-08T16:02:07.060361+0000 mon.a (mon.0) 1918 : cluster [DBG] osdmap e582: 12 total, 12 up, 12 in 2020-09-08T16:02:08.414 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 88 16 132353 132337 93.9803 48.5 0.0057651 0.0106058 2020-09-08T16:02:08.421 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:08 smithi055 bash[9296]: cluster 2020-09-08T16:02:07.060361+0000 mon.a (mon.0) 1918 : cluster [DBG] osdmap e582: 12 total, 12 up, 12 in 2020-09-08T16:02:08.860 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T16:02:09.030 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:02:09.182 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:09 smithi104 bash[9203]: cluster 2020-09-08T16:02:07.987195+0000 mgr.x (mgr.24118) 1044 : cluster [DBG] pgmap v1542: 17 pgs: 17 active+clean; 8.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s wr, 903 op/s 2020-09-08T16:02:09.183 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:09 smithi104 bash[9203]: cluster 2020-09-08T16:02:08.498448+0000 osd.2 (osd.2) 88 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:09.183 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:09 smithi104 bash[9203]: cluster 2020-09-08T16:02:08.498523+0000 osd.2 (osd.2) 89 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:09.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:09 smithi166 bash[9446]: cluster 2020-09-08T16:02:07.987195+0000 mgr.x (mgr.24118) 1044 : cluster [DBG] pgmap v1542: 17 pgs: 17 active+clean; 8.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s wr, 903 op/s 2020-09-08T16:02:09.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:09 smithi166 bash[9446]: cluster 2020-09-08T16:02:08.498448+0000 osd.2 (osd.2) 88 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:09.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:09 smithi166 bash[9446]: cluster 2020-09-08T16:02:08.498523+0000 osd.2 (osd.2) 89 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:09.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:09 smithi055 bash[9296]: cluster 2020-09-08T16:02:07.987195+0000 mgr.x (mgr.24118) 1044 : cluster [DBG] pgmap v1542: 17 pgs: 17 active+clean; 8.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s wr, 903 op/s 2020-09-08T16:02:09.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:09 smithi055 bash[9296]: cluster 2020-09-08T16:02:08.498448+0000 osd.2 (osd.2) 88 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:09.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:09 smithi055 bash[9296]: cluster 2020-09-08T16:02:08.498523+0000 osd.2 (osd.2) 89 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:09.418 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 89 16 133400 133384 93.6596 65.4375 0.00767814 0.0106745 2020-09-08T16:02:09.754 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:09.856 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:10.031 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:10.062 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:02:09 smithi055 bash[16646]: debug 2020-09-08T16:02:09.749+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:10.135 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:02:10.143 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:10 smithi166 bash[9446]: cluster 2020-09-08T16:02:09.070318+0000 mon.a (mon.0) 1919 : cluster [DBG] osdmap e583: 12 total, 12 up, 12 in 2020-09-08T16:02:10.143 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:02:10 smithi166 bash[18960]: debug 2020-09-08T16:02:10.025+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:10.259 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:02:10.371 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:02:10.414 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:10 smithi055 bash[9296]: cluster 2020-09-08T16:02:09.070318+0000 mon.a (mon.0) 1919 : cluster [DBG] osdmap e583: 12 total, 12 up, 12 in 2020-09-08T16:02:10.414 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:10 smithi104 bash[9203]: cluster 2020-09-08T16:02:09.070318+0000 mon.a (mon.0) 1919 : cluster [DBG] osdmap e583: 12 total, 12 up, 12 in 2020-09-08T16:02:10.415 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 90 15 134514 134499 93.3931 69.6875 0.0223203 0.0107044 2020-09-08T16:02:10.617 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:02:10 smithi166 bash[16593]: debug 2020-09-08T16:02:10.257+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:10.955 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:02:11.059 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:02:11.181 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:02:11.291 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:11.322 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:02:10 smithi104 bash[19922]: debug 2020-09-08T16:02:10.954+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:11.323 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:11 smithi104 bash[9203]: cluster 2020-09-08T16:02:09.987684+0000 mgr.x (mgr.24118) 1045 : cluster [DBG] pgmap v1544: 17 pgs: 17 active+clean; 8.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s wr, 751 op/s 2020-09-08T16:02:11.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:02:11 smithi166 bash[14254]: debug 2020-09-08T16:02:11.177+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:11 smithi166 bash[9446]: cluster 2020-09-08T16:02:09.987684+0000 mgr.x (mgr.24118) 1045 : cluster [DBG] pgmap v1544: 17 pgs: 17 active+clean; 8.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s wr, 751 op/s 2020-09-08T16:02:11.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:11 smithi055 bash[9296]: cluster 2020-09-08T16:02:09.987684+0000 mgr.x (mgr.24118) 1045 : cluster [DBG] pgmap v1544: 17 pgs: 17 active+clean; 8.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s wr, 751 op/s 2020-09-08T16:02:11.415 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Total time run: 90.0161 2020-09-08T16:02:11.416 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Total writes made: 134514 2020-09-08T16:02:11.416 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Write size: 65536 2020-09-08T16:02:11.416 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Object size: 65536 2020-09-08T16:02:11.416 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Bandwidth (MB/sec): 93.3958 2020-09-08T16:02:11.417 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev Bandwidth: 46.005 2020-09-08T16:02:11.417 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max bandwidth (MB/sec): 236.875 2020-09-08T16:02:11.417 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min bandwidth (MB/sec): 26.375 2020-09-08T16:02:11.417 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Average IOPS: 1494 2020-09-08T16:02:11.418 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev IOPS: 736 2020-09-08T16:02:11.418 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max IOPS: 3790 2020-09-08T16:02:11.418 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min IOPS: 422 2020-09-08T16:02:11.418 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Average Latency(s): 0.0107058 2020-09-08T16:02:11.418 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev Latency(s): 0.022592 2020-09-08T16:02:11.419 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max latency(s): 0.813986 2020-09-08T16:02:11.419 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min latency(s): 0.00178141 2020-09-08T16:02:11.428 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Cleaning up (deleting benchmark objects) 2020-09-08T16:02:11.852 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:11.955 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:02:12.074 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:02:12.176 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:12.177 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:02:11 smithi104 bash[15273]: debug 2020-09-08T16:02:11.850+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:12.283 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:02:12 smithi055 bash[19003]: debug 2020-09-08T16:02:12.069+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:12.284 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:12.391 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:12.541 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T16:02:12.542 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2020-09-08T16:02:12.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2020-09-08T16:02:12.546 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T16:02:12.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.546 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.15771.0:118561 5.0 e424/396 5:078509c7:::benchmark_data_smithi166_37820_object118560:head v 424'7447, mlcod=424'7444)", 2020-09-08T16:02:12.547 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:52:58.580414+0000", 2020-09-08T16:02:12.547 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 553.95766489100004, 2020-09-08T16:02:12.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28559745600000003, 2020-09-08T16:02:12.548 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:02:12.548 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:02:12.549 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:02:12.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.550 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:02:12.550 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.580414+0000", 2020-09-08T16:02:12.551 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:02:12.551 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.551 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.561 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:02:12.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.580414+0000", 2020-09-08T16:02:12.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013476199999999999 2020-09-08T16:02:12.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.562 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.562 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:02:12.562 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.580549+0000", 2020-09-08T16:02:12.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9930000000000001e-06 2020-09-08T16:02:12.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:02:12.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.580558+0000", 2020-09-08T16:02:12.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.499e-05 2020-09-08T16:02:12.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.571 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.571 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:02:12.572 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.580573+0000", 2020-09-08T16:02:12.572 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1299999999999998e-06 2020-09-08T16:02:12.572 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.572 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.572 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:02:12.573 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.580579+0000", 2020-09-08T16:02:12.573 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4800000000000003e-05 2020-09-08T16:02:12.573 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.573 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.573 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:02:12.574 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.580644+0000", 2020-09-08T16:02:12.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2364000000000001e-05 2020-09-08T16:02:12.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.574 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:02:12.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.580666+0000", 2020-09-08T16:02:12.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28526225599999999 2020-09-08T16:02:12.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.576 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:02:12.576 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.865929+0000", 2020-09-08T16:02:12.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3160999999999999e-05 2020-09-08T16:02:12.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:02:12.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:52:58.866012+0000", 2020-09-08T16:02:12.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28559745600000003 2020-09-08T16:02:12.578 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.578 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.578 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.578 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.579 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:66327 6.e 6:712098af:::benchmark_data_smithi166_43187_object66326:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e498)", 2020-09-08T16:02:12.579 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:14.211067+0000", 2020-09-08T16:02:12.579 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 298.32701190900002, 2020-09-08T16:02:12.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.166895186, 2020-09-08T16:02:12.579 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:02:12.580 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:02:12.580 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:02:12.580 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:02:12.580 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:02:12.580 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 66327 2020-09-08T16:02:12.581 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.581 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:02:12.581 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.581 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:02:12.581 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.211067+0000", 2020-09-08T16:02:12.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:02:12.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:02:12.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.211067+0000", 2020-09-08T16:02:12.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999747 2020-09-08T16:02:12.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:02:12.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.211042+0000", 2020-09-08T16:02:12.604 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9693999999999999e-05 2020-09-08T16:02:12.604 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:02:12.605 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.211131+0000", 2020-09-08T16:02:12.605 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0919e-05 2020-09-08T16:02:12.605 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.605 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.606 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:02:12.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.211142+0000", 2020-09-08T16:02:12.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3809999999999998e-06 2020-09-08T16:02:12.607 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.607 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:02:12.608 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.211148+0000", 2020-09-08T16:02:12.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8335999999999999e-05 2020-09-08T16:02:12.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:02:12.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.211196+0000", 2020-09-08T16:02:12.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000104621 2020-09-08T16:02:12.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.609 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.609 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:02:12.610 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.211301+0000", 2020-09-08T16:02:12.610 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017376499999999999 2020-09-08T16:02:12.610 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.610 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.610 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:02:12.611 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.211474+0000", 2020-09-08T16:02:12.611 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.157118745 2020-09-08T16:02:12.611 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.611 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.611 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:02:12.612 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.368593+0000", 2020-09-08T16:02:12.612 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0092767390000000009 2020-09-08T16:02:12.612 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.612 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.612 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:02:12.613 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.377870+0000", 2020-09-08T16:02:12.613 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9525999999999999e-05 2020-09-08T16:02:12.613 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.613 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.613 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:02:12.614 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.377929+0000", 2020-09-08T16:02:12.614 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2969999999999998e-05 2020-09-08T16:02:12.614 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:02:12.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.377962+0000", 2020-09-08T16:02:12.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.166895186 2020-09-08T16:02:12.616 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.616 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.616 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.616 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.617 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.617 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.16014.0:66330 6.d e498/478 6:b1242892:::benchmark_data_smithi166_43187_object66329:head v 498'4157, mlcod=498'4155)", 2020-09-08T16:02:12.617 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:14.213521+0000", 2020-09-08T16:02:12.617 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 298.32455786999998, 2020-09-08T16:02:12.617 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17103138600000001, 2020-09-08T16:02:12.618 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:02:12.618 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:02:12.618 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:02:12.618 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.618 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:02:12.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.213521+0000", 2020-09-08T16:02:12.619 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:02:12.619 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:02:12.620 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.213521+0000", 2020-09-08T16:02:12.620 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15418963699999999 2020-09-08T16:02:12.620 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.620 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.620 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:02:12.621 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.367711+0000", 2020-09-08T16:02:12.621 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3496999999999999e-05 2020-09-08T16:02:12.621 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.621 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.621 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:02:12.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.367734+0000", 2020-09-08T16:02:12.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0791999999999998e-05 2020-09-08T16:02:12.622 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:02:12.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.367775+0000", 2020-09-08T16:02:12.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3633e-05 2020-09-08T16:02:12.623 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.623 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.623 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:02:12.624 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.367789+0000", 2020-09-08T16:02:12.624 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000125344 2020-09-08T16:02:12.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.624 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:02:12.624 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.367914+0000", 2020-09-08T16:02:12.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3584999999999995e-05 2020-09-08T16:02:12.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:02:12.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.367978+0000", 2020-09-08T16:02:12.627 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.016471189000000001 2020-09-08T16:02:12.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.627 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:02:12.627 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.384449+0000", 2020-09-08T16:02:12.629 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010370900000000001 2020-09-08T16:02:12.629 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.630 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:02:12.630 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:14.384553+0000", 2020-09-08T16:02:12.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17103138600000001 2020-09-08T16:02:12.631 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.632 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.632 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.633 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:74329 6.e 6:7b884ce3:::benchmark_data_smithi166_43187_object74328:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T16:02:12.634 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:19.544001+0000", 2020-09-08T16:02:12.634 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 292.99407810000002, 2020-09-08T16:02:12.634 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42241560700000003, 2020-09-08T16:02:12.635 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:02:12.635 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:02:12.635 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:02:12.636 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:02:12.636 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:02:12.636 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 74329 2020-09-08T16:02:12.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.637 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:02:12.638 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:02:12.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.544001+0000", 2020-09-08T16:02:12.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:02:12.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.640 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:02:12.640 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.544001+0000", 2020-09-08T16:02:12.640 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999485 2020-09-08T16:02:12.641 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.641 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:02:12.642 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.543950+0000", 2020-09-08T16:02:12.642 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015840599999999999 2020-09-08T16:02:12.643 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.643 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.644 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:02:12.644 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.544108+0000", 2020-09-08T16:02:12.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5276000000000001e-05 2020-09-08T16:02:12.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.646 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:02:12.646 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.544123+0000", 2020-09-08T16:02:12.646 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8460000000000002e-06 2020-09-08T16:02:12.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:02:12.648 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.544130+0000", 2020-09-08T16:02:12.648 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6257e-05 2020-09-08T16:02:12.648 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.649 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.649 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:02:12.650 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.544186+0000", 2020-09-08T16:02:12.650 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000177193 2020-09-08T16:02:12.650 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.651 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.651 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:02:12.652 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.544364+0000", 2020-09-08T16:02:12.652 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000220039 2020-09-08T16:02:12.652 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.653 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.653 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:02:12.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.544584+0000", 2020-09-08T16:02:12.654 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0054646479999999999 2020-09-08T16:02:12.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.655 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.655 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:02:12.655 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.550048+0000", 2020-09-08T16:02:12.656 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.41618292499999998 2020-09-08T16:02:12.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.657 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.657 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:02:12.657 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.966231+0000", 2020-09-08T16:02:12.658 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000100868 2020-09-08T16:02:12.658 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.658 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.658 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:02:12.659 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.966332+0000", 2020-09-08T16:02:12.659 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4656999999999999e-05 2020-09-08T16:02:12.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.659 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.659 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:02:12.660 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.966417+0000", 2020-09-08T16:02:12.660 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42241560700000003 2020-09-08T16:02:12.660 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.660 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.661 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.661 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.661 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.661 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:74331 6.e 6:7febd58a:::benchmark_data_smithi166_43187_object74330:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T16:02:12.661 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:19.544424+0000", 2020-09-08T16:02:12.662 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 292.99365510000001, 2020-09-08T16:02:12.662 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42874506000000001, 2020-09-08T16:02:12.662 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:02:12.662 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:02:12.662 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:02:12.663 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:02:12.663 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:02:12.663 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 74331 2020-09-08T16:02:12.663 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.663 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:02:12.664 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.664 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:02:12.664 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.544424+0000", 2020-09-08T16:02:12.664 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:02:12.664 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.665 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.665 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:02:12.665 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.544424+0000", 2020-09-08T16:02:12.665 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999647 2020-09-08T16:02:12.666 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.666 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.666 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:02:12.667 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.544389+0000", 2020-09-08T16:02:12.667 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000156532 2020-09-08T16:02:12.667 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.667 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.667 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:02:12.668 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.544545+0000", 2020-09-08T16:02:12.668 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3373999999999999e-05 2020-09-08T16:02:12.668 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.668 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.669 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:02:12.669 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.544559+0000", 2020-09-08T16:02:12.669 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0809999999999999e-06 2020-09-08T16:02:12.669 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.670 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.670 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:02:12.670 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.544565+0000", 2020-09-08T16:02:12.670 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0024245759999999999 2020-09-08T16:02:12.670 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.671 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.671 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:02:12.671 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.546989+0000", 2020-09-08T16:02:12.671 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000140717 2020-09-08T16:02:12.671 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.672 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.672 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:02:12.672 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.547130+0000", 2020-09-08T16:02:12.672 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000197973 2020-09-08T16:02:12.672 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.673 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.673 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:02:12.673 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.547328+0000", 2020-09-08T16:02:12.673 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0057302200000000003 2020-09-08T16:02:12.673 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.674 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.674 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:02:12.674 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.553058+0000", 2020-09-08T16:02:12.674 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.41984641499999997 2020-09-08T16:02:12.675 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.675 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.675 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:02:12.675 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.972905+0000", 2020-09-08T16:02:12.676 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015845 2020-09-08T16:02:12.676 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.676 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.676 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:02:12.676 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.973063+0000", 2020-09-08T16:02:12.677 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000105831 2020-09-08T16:02:12.677 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.677 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.677 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:02:12.677 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.973169+0000", 2020-09-08T16:02:12.678 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42874506000000001 2020-09-08T16:02:12.678 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.678 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.678 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.678 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.679 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.679 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:74332 6.e 6:7400d942:::benchmark_data_smithi166_43187_object74331:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T16:02:12.679 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:19.544629+0000", 2020-09-08T16:02:12.679 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 292.99345058099999, 2020-09-08T16:02:12.680 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42892215500000003, 2020-09-08T16:02:12.680 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:02:12.680 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:02:12.680 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:02:12.680 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:02:12.681 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:02:12.681 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 74332 2020-09-08T16:02:12.681 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.681 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:02:12.682 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.682 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:02:12.682 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.544629+0000", 2020-09-08T16:02:12.682 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:02:12.682 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.683 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.683 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:02:12.683 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.544629+0000", 2020-09-08T16:02:12.683 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999647 2020-09-08T16:02:12.683 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.684 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.684 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:02:12.684 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.544593+0000", 2020-09-08T16:02:12.684 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013083499999999999 2020-09-08T16:02:12.684 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.685 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.685 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:02:12.685 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.544724+0000", 2020-09-08T16:02:12.685 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1357e-05 2020-09-08T16:02:12.685 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.686 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.686 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:02:12.686 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.544735+0000", 2020-09-08T16:02:12.686 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7090000000000001e-06 2020-09-08T16:02:12.687 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.687 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.688 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:02:12.688 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.544740+0000", 2020-09-08T16:02:12.688 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0054806509999999996 2020-09-08T16:02:12.688 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.688 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.689 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:02:12.689 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.550221+0000", 2020-09-08T16:02:12.689 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018145600000000001 2020-09-08T16:02:12.689 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.689 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.690 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:02:12.690 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.550402+0000", 2020-09-08T16:02:12.690 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019754899999999999 2020-09-08T16:02:12.690 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.690 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.691 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:02:12.691 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.550600+0000", 2020-09-08T16:02:12.691 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0026018780000000002 2020-09-08T16:02:12.691 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.691 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.692 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:02:12.692 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.553202+0000", 2020-09-08T16:02:12.692 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42007989899999998 2020-09-08T16:02:12.692 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.692 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.693 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:02:12.693 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.973281+0000", 2020-09-08T16:02:12.693 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020160800000000001 2020-09-08T16:02:12.693 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.693 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.694 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:02:12.694 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.973483+0000", 2020-09-08T16:02:12.694 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7654e-05 2020-09-08T16:02:12.694 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.694 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.695 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:02:12.695 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:19.973551+0000", 2020-09-08T16:02:12.695 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.42892215500000003 2020-09-08T16:02:12.695 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.695 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.695 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.696 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.696 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.696 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:78574 6.4 6:2cbaf32c:::benchmark_data_smithi166_43187_object78573:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e500)", 2020-09-08T16:02:12.696 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:57:23.410765+0000", 2020-09-08T16:02:12.697 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 289.127313668, 2020-09-08T16:02:12.697 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.20804850999999999, 2020-09-08T16:02:12.697 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:02:12.697 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:02:12.697 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:02:12.698 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:02:12.698 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:02:12.698 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 78574 2020-09-08T16:02:12.698 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.698 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:02:12.698 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.699 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:02:12.699 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.410765+0000", 2020-09-08T16:02:12.699 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:02:12.699 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.699 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.700 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:02:12.700 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.410765+0000", 2020-09-08T16:02:12.700 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999747 2020-09-08T16:02:12.700 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.700 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.701 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:02:12.701 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.410740+0000", 2020-09-08T16:02:12.701 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.19368825100000001 2020-09-08T16:02:12.702 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.702 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.702 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:02:12.702 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.604428+0000", 2020-09-08T16:02:12.703 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1353999999999997e-05 2020-09-08T16:02:12.703 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.703 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.703 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:02:12.703 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.604470+0000", 2020-09-08T16:02:12.704 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3465999999999999e-05 2020-09-08T16:02:12.704 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.704 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.704 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:02:12.704 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.604483+0000", 2020-09-08T16:02:12.705 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000156465 2020-09-08T16:02:12.705 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.705 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.705 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:02:12.705 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.604640+0000", 2020-09-08T16:02:12.706 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00036429000000000002 2020-09-08T16:02:12.706 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.706 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.706 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:02:12.706 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.605004+0000", 2020-09-08T16:02:12.707 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00043279599999999999 2020-09-08T16:02:12.707 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.707 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.707 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 5", 2020-09-08T16:02:12.707 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.605437+0000", 2020-09-08T16:02:12.708 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010255666 2020-09-08T16:02:12.708 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.708 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.708 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:02:12.709 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.615693+0000", 2020-09-08T16:02:12.709 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002981044 2020-09-08T16:02:12.709 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.709 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.709 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:02:12.709 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.618674+0000", 2020-09-08T16:02:12.710 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1697000000000001e-05 2020-09-08T16:02:12.710 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.710 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.710 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:02:12.710 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.618755+0000", 2020-09-08T16:02:12.711 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8724000000000002e-05 2020-09-08T16:02:12.711 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.711 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.711 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:02:12.711 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:57:23.618814+0000", 2020-09-08T16:02:12.712 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.20804850999999999 2020-09-08T16:02:12.712 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.712 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.712 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.712 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.713 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.713 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:149042 6.e 6:7c32f567:::benchmark_data_smithi166_43187_object29273:head [delete] snapc 0=[] ondisk+write+known_if_redirected e517)", 2020-09-08T16:02:12.713 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:58:27.359002+0000", 2020-09-08T16:02:12.713 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 225.179076853, 2020-09-08T16:02:12.714 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15643912400000001, 2020-09-08T16:02:12.714 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:02:12.714 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:02:12.714 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:02:12.714 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:02:12.715 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:02:12.715 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 149042 2020-09-08T16:02:12.715 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.715 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:02:12.715 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.716 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:02:12.716 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:27.359002+0000", 2020-09-08T16:02:12.716 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:02:12.716 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.716 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.717 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:02:12.717 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:27.359002+0000", 2020-09-08T16:02:12.717 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.99998 2020-09-08T16:02:12.717 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.717 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.718 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:02:12.718 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:27.358982+0000", 2020-09-08T16:02:12.718 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4854000000000001e-05 2020-09-08T16:02:12.718 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.718 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.718 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:02:12.719 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:27.359027+0000", 2020-09-08T16:02:12.719 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.2529999999999998e-06 2020-09-08T16:02:12.719 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.719 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.720 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:02:12.720 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:27.359034+0000", 2020-09-08T16:02:12.720 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6160000000000002e-06 2020-09-08T16:02:12.720 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.721 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.721 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:02:12.721 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:27.359038+0000", 2020-09-08T16:02:12.721 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0041854969999999998 2020-09-08T16:02:12.721 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.722 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.722 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:02:12.722 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:27.363224+0000", 2020-09-08T16:02:12.722 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000100326 2020-09-08T16:02:12.722 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.723 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.723 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:02:12.723 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:27.363324+0000", 2020-09-08T16:02:12.723 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2153999999999999e-05 2020-09-08T16:02:12.723 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.724 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.724 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:02:12.724 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:27.363416+0000", 2020-09-08T16:02:12.724 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.012536886000000001 2020-09-08T16:02:12.724 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.725 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.725 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:02:12.725 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:27.375953+0000", 2020-09-08T16:02:12.725 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13941604799999999 2020-09-08T16:02:12.725 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.726 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.726 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:02:12.726 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:27.515369+0000", 2020-09-08T16:02:12.726 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0546000000000001e-05 2020-09-08T16:02:12.726 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.727 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.727 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:02:12.727 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:27.515410+0000", 2020-09-08T16:02:12.727 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1850000000000002e-05 2020-09-08T16:02:12.727 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.728 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.728 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:02:12.728 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:58:27.515441+0000", 2020-09-08T16:02:12.728 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15643912400000001 2020-09-08T16:02:12.728 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.729 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.729 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.729 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.729 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.729 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:202698 6.e 6:782a9619:::benchmark_data_smithi166_43187_object82929:head [delete] snapc 0=[] ondisk+write+known_if_redirected e536)", 2020-09-08T16:02:12.730 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:59:30.430177+0000", 2020-09-08T16:02:12.730 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 162.10790230000001, 2020-09-08T16:02:12.730 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.22294625900000001, 2020-09-08T16:02:12.730 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:02:12.730 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:02:12.731 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:02:12.731 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:02:12.731 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:02:12.731 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202698 2020-09-08T16:02:12.731 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.732 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:02:12.732 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.732 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:02:12.732 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.430177+0000", 2020-09-08T16:02:12.732 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:02:12.733 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.733 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.733 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:02:12.733 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.430177+0000", 2020-09-08T16:02:12.734 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999585 2020-09-08T16:02:12.734 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.734 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.734 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:02:12.734 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.430136+0000", 2020-09-08T16:02:12.735 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8554000000000006e-05 2020-09-08T16:02:12.735 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.735 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.735 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:02:12.735 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.430224+0000", 2020-09-08T16:02:12.736 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3008e-05 2020-09-08T16:02:12.736 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.736 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.736 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:02:12.736 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.430237+0000", 2020-09-08T16:02:12.737 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0540000000000002e-06 2020-09-08T16:02:12.737 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.737 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.737 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:02:12.738 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.430242+0000", 2020-09-08T16:02:12.738 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4307e-05 2020-09-08T16:02:12.738 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.738 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.738 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:02:12.738 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.430296+0000", 2020-09-08T16:02:12.739 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8831000000000004e-05 2020-09-08T16:02:12.739 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.739 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.739 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:02:12.739 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.430385+0000", 2020-09-08T16:02:12.740 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5544000000000006e-05 2020-09-08T16:02:12.740 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.740 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.740 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:02:12.740 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.430471+0000", 2020-09-08T16:02:12.741 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0053805620000000002 2020-09-08T16:02:12.741 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.741 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.741 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:02:12.742 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.435851+0000", 2020-09-08T16:02:12.742 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21708075399999999 2020-09-08T16:02:12.742 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.742 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.742 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:02:12.743 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.652932+0000", 2020-09-08T16:02:12.743 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000106551 2020-09-08T16:02:12.743 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.743 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.743 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:02:12.744 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.653039+0000", 2020-09-08T16:02:12.744 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4392000000000001e-05 2020-09-08T16:02:12.744 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.744 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.744 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:02:12.745 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.653123+0000", 2020-09-08T16:02:12.745 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.22294625900000001 2020-09-08T16:02:12.745 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.745 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.745 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.746 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.746 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.746 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:202714 6.e 6:7adb235e:::benchmark_data_smithi166_43187_object82945:head [delete] snapc 0=[] ondisk+write+known_if_redirected e536)", 2020-09-08T16:02:12.746 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:59:30.446298+0000", 2020-09-08T16:02:12.746 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 162.09178107599999, 2020-09-08T16:02:12.747 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21457531099999999, 2020-09-08T16:02:12.747 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:02:12.747 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:02:12.747 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:02:12.747 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:02:12.748 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:02:12.748 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202714 2020-09-08T16:02:12.748 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.748 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:02:12.748 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.749 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:02:12.749 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.446298+0000", 2020-09-08T16:02:12.749 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:02:12.749 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.749 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.750 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:02:12.750 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.446298+0000", 2020-09-08T16:02:12.750 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999958 2020-09-08T16:02:12.750 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.751 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.751 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:02:12.751 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.446256+0000", 2020-09-08T16:02:12.751 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3294999999999994e-05 2020-09-08T16:02:12.751 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.752 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.752 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:02:12.752 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.446339+0000", 2020-09-08T16:02:12.752 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3199e-05 2020-09-08T16:02:12.752 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.753 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.753 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:02:12.753 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.446353+0000", 2020-09-08T16:02:12.753 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5919999999999997e-06 2020-09-08T16:02:12.753 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.754 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.754 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:02:12.754 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.446359+0000", 2020-09-08T16:02:12.754 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9181000000000002e-05 2020-09-08T16:02:12.754 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.754 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.755 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:02:12.755 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.446408+0000", 2020-09-08T16:02:12.755 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017045999999999999 2020-09-08T16:02:12.755 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.755 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.756 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:02:12.756 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.446579+0000", 2020-09-08T16:02:12.756 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017048700000000001 2020-09-08T16:02:12.756 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.756 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.757 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:02:12.757 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.446749+0000", 2020-09-08T16:02:12.757 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.005460688 2020-09-08T16:02:12.757 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.757 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.758 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:02:12.758 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.452210+0000", 2020-09-08T16:02:12.758 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.208480316 2020-09-08T16:02:12.758 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.758 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.759 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:02:12.759 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.660690+0000", 2020-09-08T16:02:12.759 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000109472 2020-09-08T16:02:12.759 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.759 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.760 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:02:12.760 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.660800+0000", 2020-09-08T16:02:12.760 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3552000000000003e-05 2020-09-08T16:02:12.760 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.760 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.761 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:02:12.761 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.660873+0000", 2020-09-08T16:02:12.761 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21457531099999999 2020-09-08T16:02:12.761 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.761 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.762 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.762 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.762 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.762 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:202719 6.e 6:77f77f1f:::benchmark_data_smithi166_43187_object82950:head [delete] snapc 0=[] ondisk+write+known_if_redirected e536)", 2020-09-08T16:02:12.762 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:59:30.456871+0000", 2020-09-08T16:02:12.763 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 162.08120777900001, 2020-09-08T16:02:12.763 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21736169999999999, 2020-09-08T16:02:12.763 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:02:12.763 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:02:12.764 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:02:12.764 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:02:12.764 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:02:12.764 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202719 2020-09-08T16:02:12.764 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.764 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:02:12.765 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.765 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:02:12.765 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.456871+0000", 2020-09-08T16:02:12.765 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:02:12.765 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.766 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.766 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:02:12.766 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.456871+0000", 2020-09-08T16:02:12.766 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999762 2020-09-08T16:02:12.767 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.767 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.767 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:02:12.767 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.456848+0000", 2020-09-08T16:02:12.768 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2540000000000002e-05 2020-09-08T16:02:12.768 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.768 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.768 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:02:12.768 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.456900+0000", 2020-09-08T16:02:12.769 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9653999999999999e-05 2020-09-08T16:02:12.769 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.769 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.769 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:02:12.769 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.456920+0000", 2020-09-08T16:02:12.770 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1890000000000001e-06 2020-09-08T16:02:12.770 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.770 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.770 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:02:12.770 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.456927+0000", 2020-09-08T16:02:12.770 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0541999999999998e-05 2020-09-08T16:02:12.771 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.771 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.771 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:02:12.771 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.456978+0000", 2020-09-08T16:02:12.771 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000162318 2020-09-08T16:02:12.772 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.772 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.772 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:02:12.772 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.457140+0000", 2020-09-08T16:02:12.772 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000103735 2020-09-08T16:02:12.773 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.773 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.773 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:02:12.773 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.457244+0000", 2020-09-08T16:02:12.774 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0046529809999999996 2020-09-08T16:02:12.774 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.774 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.774 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:02:12.775 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.461897+0000", 2020-09-08T16:02:12.775 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21201972299999999 2020-09-08T16:02:12.775 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.776 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.776 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:02:12.776 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.673916+0000", 2020-09-08T16:02:12.776 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018826699999999999 2020-09-08T16:02:12.777 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.777 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.777 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:02:12.777 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.674105+0000", 2020-09-08T16:02:12.777 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000128472 2020-09-08T16:02:12.778 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.778 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.778 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:02:12.778 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.674233+0000", 2020-09-08T16:02:12.778 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21736169999999999 2020-09-08T16:02:12.779 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.780 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.780 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.780 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.780 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.781 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:202729 6.e 6:73babc97:::benchmark_data_smithi166_43187_object82960:head [delete] snapc 0=[] ondisk+write+known_if_redirected e536)", 2020-09-08T16:02:12.781 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:59:30.464418+0000", 2020-09-08T16:02:12.781 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 162.07366131800001, 2020-09-08T16:02:12.781 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21018515800000001, 2020-09-08T16:02:12.781 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:02:12.782 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:02:12.782 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:02:12.782 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:02:12.782 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:02:12.782 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202729 2020-09-08T16:02:12.783 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.783 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:02:12.783 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.783 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:02:12.783 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.464418+0000", 2020-09-08T16:02:12.784 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:02:12.784 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.784 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.784 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:02:12.784 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.464418+0000", 2020-09-08T16:02:12.785 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999766 2020-09-08T16:02:12.785 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.785 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.785 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:02:12.785 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.464394+0000", 2020-09-08T16:02:12.786 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1405e-05 2020-09-08T16:02:12.786 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.786 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.786 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:02:12.786 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.464446+0000", 2020-09-08T16:02:12.786 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4700000000000008e-06 2020-09-08T16:02:12.787 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.787 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.787 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:02:12.787 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.464455+0000", 2020-09-08T16:02:12.787 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8520000000000003e-06 2020-09-08T16:02:12.788 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.788 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.788 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:02:12.788 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.464460+0000", 2020-09-08T16:02:12.788 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1064999999999998e-05 2020-09-08T16:02:12.789 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.789 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.789 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:02:12.789 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.464531+0000", 2020-09-08T16:02:12.789 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000117853 2020-09-08T16:02:12.790 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.790 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.790 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:02:12.790 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.464649+0000", 2020-09-08T16:02:12.790 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000111267 2020-09-08T16:02:12.791 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.791 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.791 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:02:12.791 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.464760+0000", 2020-09-08T16:02:12.791 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0048323619999999998 2020-09-08T16:02:12.792 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.792 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.792 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:02:12.792 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.469593+0000", 2020-09-08T16:02:12.792 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.20474008399999999 2020-09-08T16:02:12.793 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.793 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.793 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:02:12.793 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.674333+0000", 2020-09-08T16:02:12.793 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015335500000000001 2020-09-08T16:02:12.794 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.794 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.794 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:02:12.794 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.674486+0000", 2020-09-08T16:02:12.794 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000116796 2020-09-08T16:02:12.794 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.795 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.795 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:02:12.795 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.674603+0000", 2020-09-08T16:02:12.795 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21018515800000001 2020-09-08T16:02:12.795 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.796 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.796 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.796 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.796 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.796 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:202752 6.e 6:7b6c0d15:::benchmark_data_smithi166_43187_object82983:head [delete] snapc 0=[] ondisk+write+known_if_redirected e536)", 2020-09-08T16:02:12.797 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:59:30.487656+0000", 2020-09-08T16:02:12.797 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 162.05042318400001, 2020-09-08T16:02:12.797 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.19728971300000001, 2020-09-08T16:02:12.797 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:02:12.797 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:02:12.797 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:02:12.798 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:02:12.798 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:02:12.798 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202752 2020-09-08T16:02:12.798 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.798 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:02:12.799 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.799 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:02:12.799 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.487656+0000", 2020-09-08T16:02:12.799 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:02:12.800 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.800 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.800 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:02:12.800 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.487656+0000", 2020-09-08T16:02:12.800 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999557 2020-09-08T16:02:12.800 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.801 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.801 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:02:12.801 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.487611+0000", 2020-09-08T16:02:12.801 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5141000000000003e-05 2020-09-08T16:02:12.801 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.802 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.802 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:02:12.802 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.487707+0000", 2020-09-08T16:02:12.802 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7898999999999999e-05 2020-09-08T16:02:12.802 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.803 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.803 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:02:12.803 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.487725+0000", 2020-09-08T16:02:12.803 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0089999999999996e-06 2020-09-08T16:02:12.803 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.804 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.804 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:02:12.804 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.487734+0000", 2020-09-08T16:02:12.804 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5980999999999998e-05 2020-09-08T16:02:12.804 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.804 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.805 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:02:12.805 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.487790+0000", 2020-09-08T16:02:12.805 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016943500000000001 2020-09-08T16:02:12.805 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.806 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.806 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:02:12.806 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.487959+0000", 2020-09-08T16:02:12.806 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017985700000000001 2020-09-08T16:02:12.806 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.806 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.807 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:02:12.807 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.488139+0000", 2020-09-08T16:02:12.807 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0058527539999999999 2020-09-08T16:02:12.807 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.807 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.808 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:02:12.808 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.493992+0000", 2020-09-08T16:02:12.808 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.19080227399999999 2020-09-08T16:02:12.808 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.808 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.809 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:02:12.809 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.684794+0000", 2020-09-08T16:02:12.809 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7337999999999995e-05 2020-09-08T16:02:12.809 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.809 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.810 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:02:12.810 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.684891+0000", 2020-09-08T16:02:12.810 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4496000000000001e-05 2020-09-08T16:02:12.810 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.810 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.811 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:02:12.811 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.684946+0000", 2020-09-08T16:02:12.811 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.19728971300000001 2020-09-08T16:02:12.811 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.811 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.812 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.812 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.812 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.812 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:202764 6.e 6:796d5322:::benchmark_data_smithi166_43187_object82995:head [delete] snapc 0=[] ondisk+write+known_if_redirected e536)", 2020-09-08T16:02:12.812 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:59:30.507673+0000", 2020-09-08T16:02:12.813 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 162.03040633800001, 2020-09-08T16:02:12.813 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.177583359, 2020-09-08T16:02:12.813 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:02:12.813 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:02:12.813 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:02:12.814 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:02:12.814 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:02:12.814 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202764 2020-09-08T16:02:12.814 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.814 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:02:12.815 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.815 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:02:12.815 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.507673+0000", 2020-09-08T16:02:12.815 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:02:12.815 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.816 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.816 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:02:12.816 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.507673+0000", 2020-09-08T16:02:12.816 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999132 2020-09-08T16:02:12.816 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.817 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.817 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:02:12.817 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.507586+0000", 2020-09-08T16:02:12.817 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018901000000000001 2020-09-08T16:02:12.817 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.818 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.818 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:02:12.818 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.507775+0000", 2020-09-08T16:02:12.818 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9556e-05 2020-09-08T16:02:12.818 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.818 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.819 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:02:12.819 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.507804+0000", 2020-09-08T16:02:12.819 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1733e-05 2020-09-08T16:02:12.819 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.819 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.820 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:02:12.820 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.507826+0000", 2020-09-08T16:02:12.820 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1586000000000001e-05 2020-09-08T16:02:12.820 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.820 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.821 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:02:12.821 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.507908+0000", 2020-09-08T16:02:12.821 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00021437999999999999 2020-09-08T16:02:12.821 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.821 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.822 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:02:12.833 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.508122+0000", 2020-09-08T16:02:12.833 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00023220599999999999 2020-09-08T16:02:12.833 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.833 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.834 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:02:12.834 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.508354+0000", 2020-09-08T16:02:12.834 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0063759740000000004 2020-09-08T16:02:12.834 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.835 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.835 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:02:12.835 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.514730+0000", 2020-09-08T16:02:12.835 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17041372299999999 2020-09-08T16:02:12.835 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.836 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.836 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:02:12.836 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.685144+0000", 2020-09-08T16:02:12.836 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2648000000000001e-05 2020-09-08T16:02:12.836 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.836 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.837 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:02:12.837 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.685207+0000", 2020-09-08T16:02:12.837 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9456000000000002e-05 2020-09-08T16:02:12.838 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.838 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.838 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:02:12.838 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.685256+0000", 2020-09-08T16:02:12.838 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.177583359 2020-09-08T16:02:12.839 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.839 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.839 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.839 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.839 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.840 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:202780 6.e 6:75f96cf7:::benchmark_data_smithi166_43187_object83011:head [delete] snapc 0=[] ondisk+write+known_if_redirected e536)", 2020-09-08T16:02:12.841 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:59:30.525922+0000", 2020-09-08T16:02:12.841 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 162.01215680300001, 2020-09-08T16:02:12.842 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.159758544, 2020-09-08T16:02:12.842 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:02:12.842 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:02:12.842 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:02:12.842 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:02:12.843 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:02:12.843 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202780 2020-09-08T16:02:12.843 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.843 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:02:12.843 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.844 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:02:12.844 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.525922+0000", 2020-09-08T16:02:12.844 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:02:12.844 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.844 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.845 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:02:12.845 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.525922+0000", 2020-09-08T16:02:12.845 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999604 2020-09-08T16:02:12.845 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.846 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.846 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:02:12.846 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.525883+0000", 2020-09-08T16:02:12.846 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2145999999999995e-05 2020-09-08T16:02:12.847 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.847 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.847 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:02:12.847 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.525975+0000", 2020-09-08T16:02:12.847 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7166000000000001e-05 2020-09-08T16:02:12.847 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.848 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.848 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:02:12.848 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.525992+0000", 2020-09-08T16:02:12.848 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9959999999999999e-06 2020-09-08T16:02:12.848 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.849 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.849 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:02:12.849 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.526001+0000", 2020-09-08T16:02:12.850 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8238e-05 2020-09-08T16:02:12.850 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.850 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.850 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:02:12.850 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.526059+0000", 2020-09-08T16:02:12.851 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015962699999999999 2020-09-08T16:02:12.851 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.851 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.851 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:02:12.852 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.526219+0000", 2020-09-08T16:02:12.852 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017547999999999999 2020-09-08T16:02:12.852 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.852 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.852 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:02:12.852 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.526394+0000", 2020-09-08T16:02:12.853 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010704174 2020-09-08T16:02:12.853 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.853 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.853 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:02:12.853 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.537098+0000", 2020-09-08T16:02:12.854 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14845812 2020-09-08T16:02:12.854 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.854 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.854 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:02:12.854 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.685556+0000", 2020-09-08T16:02:12.855 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4846999999999997e-05 2020-09-08T16:02:12.855 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.855 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.855 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:02:12.856 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.685631+0000", 2020-09-08T16:02:12.856 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9558999999999998e-05 2020-09-08T16:02:12.856 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.856 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.856 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:02:12.857 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.685681+0000", 2020-09-08T16:02:12.857 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.159758544 2020-09-08T16:02:12.857 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.857 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.857 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.858 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.858 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.858 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:202781 6.e 6:792632b1:::benchmark_data_smithi166_43187_object83012:head [delete] snapc 0=[] ondisk+write+known_if_redirected e536)", 2020-09-08T16:02:12.858 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:59:30.527485+0000", 2020-09-08T16:02:12.859 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 162.01059457299999, 2020-09-08T16:02:12.859 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.158781687, 2020-09-08T16:02:12.859 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:02:12.859 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:02:12.859 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:02:12.859 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:02:12.860 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:02:12.860 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202781 2020-09-08T16:02:12.860 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.860 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:02:12.860 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.861 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:02:12.861 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.527485+0000", 2020-09-08T16:02:12.861 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:02:12.861 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.861 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.862 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:02:12.862 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.527485+0000", 2020-09-08T16:02:12.862 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999609 2020-09-08T16:02:12.862 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.862 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.863 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:02:12.863 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.527445+0000", 2020-09-08T16:02:12.863 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5767999999999996e-05 2020-09-08T16:02:12.863 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.864 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.864 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:02:12.864 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.527531+0000", 2020-09-08T16:02:12.865 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6487999999999999e-05 2020-09-08T16:02:12.865 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.865 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.865 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:02:12.865 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.527548+0000", 2020-09-08T16:02:12.866 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7090000000000008e-06 2020-09-08T16:02:12.866 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.866 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.866 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:02:12.867 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.527556+0000", 2020-09-08T16:02:12.867 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0035586020000000001 2020-09-08T16:02:12.867 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.867 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.867 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:02:12.868 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.531115+0000", 2020-09-08T16:02:12.868 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015404399999999999 2020-09-08T16:02:12.868 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.868 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.868 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:02:12.869 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.531269+0000", 2020-09-08T16:02:12.869 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000164427 2020-09-08T16:02:12.869 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.869 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.869 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:02:12.872 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.531434+0000", 2020-09-08T16:02:12.873 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0060429339999999998 2020-09-08T16:02:12.873 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.873 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.873 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:02:12.873 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.537476+0000", 2020-09-08T16:02:12.874 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.148598647 2020-09-08T16:02:12.874 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.874 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.875 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:02:12.875 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.686075+0000", 2020-09-08T16:02:12.876 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000102934 2020-09-08T16:02:12.876 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.876 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.876 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:02:12.877 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.686178+0000", 2020-09-08T16:02:12.877 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8232999999999994e-05 2020-09-08T16:02:12.878 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.879 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.879 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:02:12.879 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.686266+0000", 2020-09-08T16:02:12.880 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.158781687 2020-09-08T16:02:12.880 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.880 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.880 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.880 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.881 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.881 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:132315 7.b 7:ddb8020b:::benchmark_data_smithi166_48533_object132314:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e582)", 2020-09-08T16:02:12.881 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:02:08.120129+0000", 2020-09-08T16:02:12.881 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 4.4179502780000002, 2020-09-08T16:02:12.882 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.33257541400000001, 2020-09-08T16:02:12.882 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:02:12.882 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:02:12.882 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:02:12.882 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:02:12.883 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:02:12.883 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 132315 2020-09-08T16:02:12.883 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.883 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:02:12.884 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.884 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:02:12.884 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.120129+0000", 2020-09-08T16:02:12.884 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:02:12.885 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.885 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.885 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:02:12.885 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.120129+0000", 2020-09-08T16:02:12.885 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999533 2020-09-08T16:02:12.886 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.886 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.886 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:02:12.886 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.120082+0000", 2020-09-08T16:02:12.887 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013359799999999999 2020-09-08T16:02:12.887 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.887 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.887 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:02:12.887 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.120216+0000", 2020-09-08T16:02:12.888 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3717999999999999e-05 2020-09-08T16:02:12.888 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.888 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.888 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:02:12.888 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.120239+0000", 2020-09-08T16:02:12.889 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1121000000000001e-05 2020-09-08T16:02:12.889 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.889 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.889 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:02:12.889 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.120250+0000", 2020-09-08T16:02:12.890 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0232000000000001e-05 2020-09-08T16:02:12.890 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.890 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.890 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:02:12.891 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.120321+0000", 2020-09-08T16:02:12.891 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020806700000000001 2020-09-08T16:02:12.891 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.891 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.891 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:02:12.892 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.120529+0000", 2020-09-08T16:02:12.892 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000203279 2020-09-08T16:02:12.892 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.892 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.893 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 5", 2020-09-08T16:02:12.893 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.120732+0000", 2020-09-08T16:02:12.893 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0051322019999999998 2020-09-08T16:02:12.893 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.893 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.894 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:02:12.894 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.125864+0000", 2020-09-08T16:02:12.894 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.32670120800000002 2020-09-08T16:02:12.895 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.895 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.895 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:02:12.895 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.452565+0000", 2020-09-08T16:02:12.895 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3814999999999997e-05 2020-09-08T16:02:12.896 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.896 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.896 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:02:12.896 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.452639+0000", 2020-09-08T16:02:12.897 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5105999999999996e-05 2020-09-08T16:02:12.897 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.897 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.897 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:02:12.897 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.452704+0000", 2020-09-08T16:02:12.898 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.33257541400000001 2020-09-08T16:02:12.898 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.898 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.898 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.898 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.899 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.899 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:132337 7.b 7:dbdb4841:::benchmark_data_smithi166_48533_object132336:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e582)", 2020-09-08T16:02:12.899 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:02:08.141196+0000", 2020-09-08T16:02:12.901 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 4.3968835009999996, 2020-09-08T16:02:12.901 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.31203498000000002, 2020-09-08T16:02:12.901 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:02:12.901 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:02:12.901 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:02:12.902 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:02:12.902 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:02:12.902 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 132337 2020-09-08T16:02:12.902 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.903 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:02:12.903 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.903 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:02:12.903 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.141196+0000", 2020-09-08T16:02:12.903 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:02:12.904 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.904 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.904 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:02:12.904 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.141196+0000", 2020-09-08T16:02:12.904 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999566 2020-09-08T16:02:12.905 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.905 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.905 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:02:12.905 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.141152+0000", 2020-09-08T16:02:12.905 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013531700000000001 2020-09-08T16:02:12.906 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.906 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.906 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:02:12.906 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.141287+0000", 2020-09-08T16:02:12.906 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1562e-05 2020-09-08T16:02:12.907 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.907 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.907 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:02:12.907 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.141299+0000", 2020-09-08T16:02:12.907 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1059999999999996e-06 2020-09-08T16:02:12.908 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.908 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.908 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:02:12.908 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.141305+0000", 2020-09-08T16:02:12.908 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9994999999999996e-05 2020-09-08T16:02:12.909 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.909 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.909 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:02:12.909 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.141375+0000", 2020-09-08T16:02:12.909 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000211397 2020-09-08T16:02:12.909 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.910 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.910 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:02:12.910 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.141586+0000", 2020-09-08T16:02:12.910 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018713399999999999 2020-09-08T16:02:12.911 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.911 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.911 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 5", 2020-09-08T16:02:12.911 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.141774+0000", 2020-09-08T16:02:12.911 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0049451019999999998 2020-09-08T16:02:12.911 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.912 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.912 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:02:12.912 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.146719+0000", 2020-09-08T16:02:12.912 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.30620335100000001 2020-09-08T16:02:12.912 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.913 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.913 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:02:12.913 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.452922+0000", 2020-09-08T16:02:12.913 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00024726799999999998 2020-09-08T16:02:12.913 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.914 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.914 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:02:12.914 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.453169+0000", 2020-09-08T16:02:12.914 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1344000000000001e-05 2020-09-08T16:02:12.914 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.915 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.915 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:02:12.915 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.453231+0000", 2020-09-08T16:02:12.915 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.31203498000000002 2020-09-08T16:02:12.915 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.916 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.916 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.916 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.916 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.916 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:132345 7.b 7:dbb244e9:::benchmark_data_smithi166_48533_object132344:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e582)", 2020-09-08T16:02:12.917 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:02:08.150586+0000", 2020-09-08T16:02:12.917 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 4.3874933350000003, 2020-09-08T16:02:12.917 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.303833462, 2020-09-08T16:02:12.917 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:02:12.917 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:02:12.917 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:02:12.918 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:02:12.918 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:02:12.918 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 132345 2020-09-08T16:02:12.918 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.918 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:02:12.919 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.919 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:02:12.919 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.150586+0000", 2020-09-08T16:02:12.919 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:02:12.919 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.920 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.920 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:02:12.920 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.150586+0000", 2020-09-08T16:02:12.920 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999623 2020-09-08T16:02:12.920 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.920 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.921 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:02:12.921 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.150548+0000", 2020-09-08T16:02:12.921 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013342200000000001 2020-09-08T16:02:12.921 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.921 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.922 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:02:12.922 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.150682+0000", 2020-09-08T16:02:12.922 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9137e-05 2020-09-08T16:02:12.922 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.922 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.923 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:02:12.923 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.150701+0000", 2020-09-08T16:02:12.923 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6990000000000003e-06 2020-09-08T16:02:12.923 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.924 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.924 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:02:12.924 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.150709+0000", 2020-09-08T16:02:12.925 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9232000000000003e-05 2020-09-08T16:02:12.925 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.925 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.925 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:02:12.925 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.150788+0000", 2020-09-08T16:02:12.926 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019832300000000001 2020-09-08T16:02:12.926 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.926 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.926 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:02:12.926 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.150986+0000", 2020-09-08T16:02:12.927 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019861400000000001 2020-09-08T16:02:12.927 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.927 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.927 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 5", 2020-09-08T16:02:12.927 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.151185+0000", 2020-09-08T16:02:12.928 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0074976080000000002 2020-09-08T16:02:12.928 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.928 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.928 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:02:12.928 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.158682+0000", 2020-09-08T16:02:12.929 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29559272199999997 2020-09-08T16:02:12.929 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.929 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.929 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:02:12.929 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.454275+0000", 2020-09-08T16:02:12.930 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2961000000000007e-05 2020-09-08T16:02:12.930 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.930 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.930 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:02:12.930 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.454368+0000", 2020-09-08T16:02:12.930 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1221999999999998e-05 2020-09-08T16:02:12.931 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.931 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.931 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:02:12.931 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.454419+0000", 2020-09-08T16:02:12.931 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.303833462 2020-09-08T16:02:12.932 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.932 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.932 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.932 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.932 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.933 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:132347 7.b 7:dff06c55:::benchmark_data_smithi166_48533_object132346:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e582)", 2020-09-08T16:02:12.934 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:02:08.151199+0000", 2020-09-08T16:02:12.934 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 4.3868800280000002, 2020-09-08T16:02:12.934 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.30559632799999997, 2020-09-08T16:02:12.934 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:02:12.934 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:02:12.935 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:02:12.935 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:02:12.935 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:02:12.936 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 132347 2020-09-08T16:02:12.936 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.936 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:02:12.936 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.936 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:02:12.937 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.151199+0000", 2020-09-08T16:02:12.937 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:02:12.937 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.937 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.937 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:02:12.938 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.151199+0000", 2020-09-08T16:02:12.938 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999037 2020-09-08T16:02:12.938 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.938 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.938 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:02:12.939 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.151103+0000", 2020-09-08T16:02:12.939 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019263799999999999 2020-09-08T16:02:12.939 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.939 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.939 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:02:12.939 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.151295+0000", 2020-09-08T16:02:12.940 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5247e-05 2020-09-08T16:02:12.940 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.940 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.940 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:02:12.940 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.151311+0000", 2020-09-08T16:02:12.941 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9199999999999998e-06 2020-09-08T16:02:12.941 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.941 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.941 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:02:12.941 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.151317+0000", 2020-09-08T16:02:12.942 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003701466 2020-09-08T16:02:12.943 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.943 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.943 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:02:12.943 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.155019+0000", 2020-09-08T16:02:12.944 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014563600000000001 2020-09-08T16:02:12.944 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.944 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.944 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:02:12.944 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.155165+0000", 2020-09-08T16:02:12.944 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012374200000000001 2020-09-08T16:02:12.945 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.945 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.945 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 5", 2020-09-08T16:02:12.945 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.155288+0000", 2020-09-08T16:02:12.946 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0035475559999999999 2020-09-08T16:02:12.946 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.946 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.947 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:02:12.947 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.158836+0000", 2020-09-08T16:02:12.947 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29784127199999999 2020-09-08T16:02:12.947 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.947 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.948 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:02:12.948 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.456677+0000", 2020-09-08T16:02:12.948 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7261000000000005e-05 2020-09-08T16:02:12.948 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.948 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.949 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:02:12.949 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.456744+0000", 2020-09-08T16:02:12.949 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1042999999999999e-05 2020-09-08T16:02:12.949 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:02:12.949 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:02:12.950 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:02:12.950 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.456795+0000", 2020-09-08T16:02:12.950 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.30559632799999997 2020-09-08T16:02:12.950 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.950 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.951 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.951 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:02:12.951 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:02:12.951 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T16:02:12.953 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:02:12 smithi055 bash[11956]: debug 2020-09-08T16:02:12.285+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:12.953 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:02:12 smithi104 bash[17588]: debug 2020-09-08T16:02:12.618+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:12.953 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:13.055 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:13.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:13 smithi166 bash[9446]: cluster 2020-09-08T16:02:11.988189+0000 mgr.x (mgr.24118) 1046 : cluster [DBG] pgmap v1545: 17 pgs: 17 active+clean; 8.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s wr, 743 op/s 2020-09-08T16:02:13.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:13 smithi055 bash[9296]: cluster 2020-09-08T16:02:11.988189+0000 mgr.x (mgr.24118) 1046 : cluster [DBG] pgmap v1545: 17 pgs: 17 active+clean; 8.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s wr, 743 op/s 2020-09-08T16:02:13.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:13 smithi104 bash[9203]: cluster 2020-09-08T16:02:11.988189+0000 mgr.x (mgr.24118) 1046 : cluster [DBG] pgmap v1545: 17 pgs: 17 active+clean; 8.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s wr, 743 op/s 2020-09-08T16:02:14.314 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:14.398 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:02:14.420 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:02:14.445 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:02:14.475 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:02:14.510 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:02:14.546 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:02:14.574 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:02:14.587 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:02:14.617 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:02:14.666 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:02:14.678 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:02:14 smithi104 bash[17588]: debug 2020-09-08T16:02:14.310+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:14.687 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:02:14.698 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:02:14.733 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:02:14.770 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:02:14.818 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:02:14.819 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:02:14.819 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:02:14.819 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:02:14.819 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:02:14.819 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:02:14.820 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:02:14.820 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:02:14.820 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:02:14.820 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:02:14.820 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:02:14.821 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:02:14.821 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:02:14.871 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:02:14 smithi166 bash[14254]: debug 2020-09-08T16:02:14.573+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:14.915 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:02:14.966 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:02:14 smithi104 bash[19922]: debug 2020-09-08T16:02:14.914+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:14.967 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2020-09-08T16:02:15.021 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:02:15.150 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:02:15.165 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:02:15.268 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:15.283 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:15 smithi104 bash[9203]: cluster 2020-09-08T16:02:13.988695+0000 mgr.x (mgr.24118) 1047 : cluster [DBG] pgmap v1546: 17 pgs: 17 active+clean; 8.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 61 MiB/s wr, 1.08k op/s 2020-09-08T16:02:15.284 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:15 smithi104 bash[9203]: audit 2020-09-08T16:02:14.364599+0000 mon.a (mon.0) 1920 : audit [DBG] from='client.? v1:172.21.15.104:0/3598440570' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:02:15.371 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:02:15 smithi166 bash[16593]: debug 2020-09-08T16:02:15.161+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:15.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:15 smithi166 bash[9446]: cluster 2020-09-08T16:02:13.988695+0000 mgr.x (mgr.24118) 1047 : cluster [DBG] pgmap v1546: 17 pgs: 17 active+clean; 8.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 61 MiB/s wr, 1.08k op/s 2020-09-08T16:02:15.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:15 smithi166 bash[9446]: audit 2020-09-08T16:02:14.364599+0000 mon.a (mon.0) 1920 : audit [DBG] from='client.? v1:172.21.15.104:0/3598440570' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:02:15.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:15 smithi055 bash[9296]: cluster 2020-09-08T16:02:13.988695+0000 mgr.x (mgr.24118) 1047 : cluster [DBG] pgmap v1546: 17 pgs: 17 active+clean; 8.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 61 MiB/s wr, 1.08k op/s 2020-09-08T16:02:15.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:15 smithi055 bash[9296]: audit 2020-09-08T16:02:14.364599+0000 mon.a (mon.0) 1920 : audit [DBG] from='client.? v1:172.21.15.104:0/3598440570' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:02:15.996 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:16.102 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:02:16.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:16 smithi166 bash[9446]: cluster 2020-09-08T16:02:14.784067+0000 osd.7 (osd.7) 127 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:16.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:16 smithi166 bash[9446]: cluster 2020-09-08T16:02:14.784148+0000 osd.7 (osd.7) 128 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:16.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:16 smithi166 bash[9446]: cluster 2020-09-08T16:02:15.080607+0000 mon.a (mon.0) 1921 : cluster [DBG] osdmap e584: 12 total, 12 up, 12 in 2020-09-08T16:02:16.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:16 smithi055 bash[9296]: cluster 2020-09-08T16:02:14.784067+0000 osd.7 (osd.7) 127 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:16.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:16 smithi055 bash[9296]: cluster 2020-09-08T16:02:14.784148+0000 osd.7 (osd.7) 128 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:16.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:16 smithi055 bash[9296]: cluster 2020-09-08T16:02:15.080607+0000 mon.a (mon.0) 1921 : cluster [DBG] osdmap e584: 12 total, 12 up, 12 in 2020-09-08T16:02:16.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:16 smithi104 bash[9203]: cluster 2020-09-08T16:02:14.784067+0000 osd.7 (osd.7) 127 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:16.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:16 smithi104 bash[9203]: cluster 2020-09-08T16:02:14.784148+0000 osd.7 (osd.7) 128 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:16.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:16 smithi104 bash[9203]: cluster 2020-09-08T16:02:15.080607+0000 mon.a (mon.0) 1921 : cluster [DBG] osdmap e584: 12 total, 12 up, 12 in 2020-09-08T16:02:16.428 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:02:16 smithi104 bash[15273]: debug 2020-09-08T16:02:15.994+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:16.783 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:02:16.893 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:02:17.088 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:02:16 smithi104 bash[22305]: debug 2020-09-08T16:02:16.782+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:17.103 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:02:17.214 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:17.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:17 smithi166 bash[9446]: cluster 2020-09-08T16:02:15.989192+0000 mgr.x (mgr.24118) 1048 : cluster [DBG] pgmap v1548: 17 pgs: 17 active+clean; 8.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 30 MiB/s wr, 592 op/s 2020-09-08T16:02:17.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:17 smithi166 bash[9446]: cluster 2020-09-08T16:02:16.038783+0000 osd.0 (osd.0) 92 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:17.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:17 smithi166 bash[9446]: cluster 2020-09-08T16:02:16.038862+0000 osd.0 (osd.0) 93 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:17.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:17 smithi166 bash[9446]: cluster 2020-09-08T16:02:16.091405+0000 mon.a (mon.0) 1922 : cluster [DBG] osdmap e585: 12 total, 12 up, 12 in 2020-09-08T16:02:17.372 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:02:17 smithi166 bash[18960]: debug 2020-09-08T16:02:17.361+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:17.372 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:17.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:17 smithi055 bash[9296]: cluster 2020-09-08T16:02:15.989192+0000 mgr.x (mgr.24118) 1048 : cluster [DBG] pgmap v1548: 17 pgs: 17 active+clean; 8.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 30 MiB/s wr, 592 op/s 2020-09-08T16:02:17.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:17 smithi055 bash[9296]: cluster 2020-09-08T16:02:16.038783+0000 osd.0 (osd.0) 92 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:17.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:17 smithi055 bash[9296]: cluster 2020-09-08T16:02:16.038862+0000 osd.0 (osd.0) 93 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:17.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:17 smithi055 bash[9296]: cluster 2020-09-08T16:02:16.091405+0000 mon.a (mon.0) 1922 : cluster [DBG] osdmap e585: 12 total, 12 up, 12 in 2020-09-08T16:02:17.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:17 smithi104 bash[9203]: cluster 2020-09-08T16:02:15.989192+0000 mgr.x (mgr.24118) 1048 : cluster [DBG] pgmap v1548: 17 pgs: 17 active+clean; 8.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 30 MiB/s wr, 592 op/s 2020-09-08T16:02:17.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:17 smithi104 bash[9203]: cluster 2020-09-08T16:02:16.038783+0000 osd.0 (osd.0) 92 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:17.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:17 smithi104 bash[9203]: cluster 2020-09-08T16:02:16.038862+0000 osd.0 (osd.0) 93 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:17.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:17 smithi104 bash[9203]: cluster 2020-09-08T16:02:16.091405+0000 mon.a (mon.0) 1922 : cluster [DBG] osdmap e585: 12 total, 12 up, 12 in 2020-09-08T16:02:17.427 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:02:17 smithi104 bash[19922]: debug 2020-09-08T16:02:17.102+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:17.474 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:02:17.589 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:02:17.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:02:17 smithi166 bash[11928]: debug 2020-09-08T16:02:17.585+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:17.694 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:17.806 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:17.912 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:18.117 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:02:17 smithi166 bash[18960]: debug 2020-09-08T16:02:17.801+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:18.131 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:18.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:02:18 smithi104 bash[17588]: debug 2020-09-08T16:02:18.130+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:18.235 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:18.352 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:18.456 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:02:18.564 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:02:18 smithi055 bash[11956]: debug 2020-09-08T16:02:18.349+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:18.566 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:02:18.669 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:18.783 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:18.786 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:02:18.888 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:18.897 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:02:18 smithi055 bash[14301]: debug 2020-09-08T16:02:18.569+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:18.897 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:02:18 smithi055 bash[16646]: debug 2020-09-08T16:02:18.785+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:18.946 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:02:19.001 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:19.105 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:02:19.111 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T16:02:19.166 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:02:19 smithi055 bash[16646]: debug 2020-09-08T16:02:19.001+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:19.167 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:19 smithi055 bash[9296]: cluster 2020-09-08T16:02:17.989756+0000 mgr.x (mgr.24118) 1049 : cluster [DBG] pgmap v1550: 17 pgs: 17 active+clean; 7.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 25 MiB/s wr, 1.13k op/s 2020-09-08T16:02:19.167 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:19 smithi055 bash[9296]: audit 2020-09-08T16:02:18.436327+0000 mon.c (mon.2) 78 : audit [INF] from='client.? v1:172.21.15.104:0/2615510444' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:02:19.167 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:19 smithi055 bash[9296]: audit 2020-09-08T16:02:18.438778+0000 mon.a (mon.0) 1923 : audit [INF] from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:02:19.346 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:19 smithi104 bash[9203]: cluster 2020-09-08T16:02:17.989756+0000 mgr.x (mgr.24118) 1049 : cluster [DBG] pgmap v1550: 17 pgs: 17 active+clean; 7.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 25 MiB/s wr, 1.13k op/s 2020-09-08T16:02:19.347 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:19 smithi104 bash[9203]: audit 2020-09-08T16:02:18.436327+0000 mon.c (mon.2) 78 : audit [INF] from='client.? v1:172.21.15.104:0/2615510444' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:02:19.347 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:19 smithi104 bash[9203]: audit 2020-09-08T16:02:18.438778+0000 mon.a (mon.0) 1923 : audit [INF] from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:02:19.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:19 smithi166 bash[9446]: cluster 2020-09-08T16:02:17.989756+0000 mgr.x (mgr.24118) 1049 : cluster [DBG] pgmap v1550: 17 pgs: 17 active+clean; 7.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 25 MiB/s wr, 1.13k op/s 2020-09-08T16:02:19.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:19 smithi166 bash[9446]: audit 2020-09-08T16:02:18.436327+0000 mon.c (mon.2) 78 : audit [INF] from='client.? v1:172.21.15.104:0/2615510444' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:02:19.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:19 smithi166 bash[9446]: audit 2020-09-08T16:02:18.438778+0000 mon.a (mon.0) 1923 : audit [INF] from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:02:19.912 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:02:20.016 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:02:20.150 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:02:20.177 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:02:19 smithi104 bash[22305]: debug 2020-09-08T16:02:19.914+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:20.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:20 smithi104 bash[9203]: cluster 2020-09-08T16:02:19.094702+0000 mon.a (mon.0) 1924 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:02:20.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:20 smithi104 bash[9203]: audit 2020-09-08T16:02:19.109894+0000 mon.a (mon.0) 1925 : audit [INF] from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:02:20.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:20 smithi104 bash[9203]: cluster 2020-09-08T16:02:19.109950+0000 mon.a (mon.0) 1926 : cluster [DBG] osdmap e586: 12 total, 12 up, 12 in 2020-09-08T16:02:20.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:20 smithi104 bash[9203]: cluster 2020-09-08T16:02:19.488307+0000 osd.5 (osd.5) 93 : cluster [DBG] 7.0 scrub starts 2020-09-08T16:02:20.257 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:20.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:20 smithi166 bash[9446]: cluster 2020-09-08T16:02:19.094702+0000 mon.a (mon.0) 1924 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:02:20.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:20 smithi166 bash[9446]: audit 2020-09-08T16:02:19.109894+0000 mon.a (mon.0) 1925 : audit [INF] from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:02:20.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:20 smithi166 bash[9446]: cluster 2020-09-08T16:02:19.109950+0000 mon.a (mon.0) 1926 : cluster [DBG] osdmap e586: 12 total, 12 up, 12 in 2020-09-08T16:02:20.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:20 smithi166 bash[9446]: cluster 2020-09-08T16:02:19.488307+0000 osd.5 (osd.5) 93 : cluster [DBG] 7.0 scrub starts 2020-09-08T16:02:20.368 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:02:20 smithi166 bash[16593]: debug 2020-09-08T16:02:20.145+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:20.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:20 smithi055 bash[9296]: cluster 2020-09-08T16:02:19.094702+0000 mon.a (mon.0) 1924 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:02:20.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:20 smithi055 bash[9296]: audit 2020-09-08T16:02:19.109894+0000 mon.a (mon.0) 1925 : audit [INF] from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:02:20.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:20 smithi055 bash[9296]: cluster 2020-09-08T16:02:19.109950+0000 mon.a (mon.0) 1926 : cluster [DBG] osdmap e586: 12 total, 12 up, 12 in 2020-09-08T16:02:20.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:20 smithi055 bash[9296]: cluster 2020-09-08T16:02:19.488307+0000 osd.5 (osd.5) 93 : cluster [DBG] 7.0 scrub starts 2020-09-08T16:02:20.851 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:20.953 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:21.125 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:21.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:21 smithi104 bash[9203]: cluster 2020-09-08T16:02:19.990259+0000 mgr.x (mgr.24118) 1050 : cluster [DBG] pgmap v1552: 17 pgs: 17 active+clean; 7.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 938 op/s 2020-09-08T16:02:21.177 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:02:20 smithi104 bash[15273]: debug 2020-09-08T16:02:20.838+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:21.229 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:02:21.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:21 smithi166 bash[9446]: cluster 2020-09-08T16:02:19.990259+0000 mgr.x (mgr.24118) 1050 : cluster [DBG] pgmap v1552: 17 pgs: 17 active+clean; 7.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 938 op/s 2020-09-08T16:02:21.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:02:21 smithi166 bash[18960]: debug 2020-09-08T16:02:21.121+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:21.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:21 smithi055 bash[9296]: cluster 2020-09-08T16:02:19.990259+0000 mgr.x (mgr.24118) 1050 : cluster [DBG] pgmap v1552: 17 pgs: 17 active+clean; 7.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 938 op/s 2020-09-08T16:02:21.645 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:02:21 smithi104 bash[19922]: debug 2020-09-08T16:02:21.646+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:21.648 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:02:21.751 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:02:22.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:22 smithi166 bash[9446]: cluster 2020-09-08T16:02:20.628274+0000 osd.6 (osd.6) 95 : cluster [DBG] 7.7 scrub starts 2020-09-08T16:02:22.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:22 smithi166 bash[9446]: cluster 2020-09-08T16:02:21.212314+0000 osd.0 (osd.0) 94 : cluster [DBG] 7.6 scrub starts 2020-09-08T16:02:22.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:22 smithi166 bash[9446]: cluster 2020-09-08T16:02:21.662139+0000 osd.5 (osd.5) 94 : cluster [DBG] 7.0 scrub ok 2020-09-08T16:02:22.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:22 smithi055 bash[9296]: cluster 2020-09-08T16:02:20.628274+0000 osd.6 (osd.6) 95 : cluster [DBG] 7.7 scrub starts 2020-09-08T16:02:22.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:22 smithi055 bash[9296]: cluster 2020-09-08T16:02:21.212314+0000 osd.0 (osd.0) 94 : cluster [DBG] 7.6 scrub starts 2020-09-08T16:02:22.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:22 smithi055 bash[9296]: cluster 2020-09-08T16:02:21.662139+0000 osd.5 (osd.5) 94 : cluster [DBG] 7.0 scrub ok 2020-09-08T16:02:22.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:22 smithi104 bash[9203]: cluster 2020-09-08T16:02:20.628274+0000 osd.6 (osd.6) 95 : cluster [DBG] 7.7 scrub starts 2020-09-08T16:02:22.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:22 smithi104 bash[9203]: cluster 2020-09-08T16:02:21.212314+0000 osd.0 (osd.0) 94 : cluster [DBG] 7.6 scrub starts 2020-09-08T16:02:22.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:22 smithi104 bash[9203]: cluster 2020-09-08T16:02:21.662139+0000 osd.5 (osd.5) 94 : cluster [DBG] 7.0 scrub ok 2020-09-08T16:02:22.670 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:02:22.774 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:22.941 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:02:22 smithi104 bash[19922]: debug 2020-09-08T16:02:22.670+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:23.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:23 smithi166 bash[9446]: cluster 2020-09-08T16:02:21.990703+0000 mgr.x (mgr.24118) 1051 : cluster [DBG] pgmap v1553: 17 pgs: 17 active+clean; 7.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T16:02:23.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:23 smithi166 bash[9446]: cluster 2020-09-08T16:02:22.314093+0000 osd.3 (osd.3) 99 : cluster [DBG] 7.3 scrub starts 2020-09-08T16:02:23.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:23 smithi055 bash[9296]: cluster 2020-09-08T16:02:21.990703+0000 mgr.x (mgr.24118) 1051 : cluster [DBG] pgmap v1553: 17 pgs: 17 active+clean; 7.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T16:02:23.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:23 smithi055 bash[9296]: cluster 2020-09-08T16:02:22.314093+0000 osd.3 (osd.3) 99 : cluster [DBG] 7.3 scrub starts 2020-09-08T16:02:23.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:23 smithi104 bash[9203]: cluster 2020-09-08T16:02:21.990703+0000 mgr.x (mgr.24118) 1051 : cluster [DBG] pgmap v1553: 17 pgs: 17 active+clean; 7.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T16:02:23.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:23 smithi104 bash[9203]: cluster 2020-09-08T16:02:22.314093+0000 osd.3 (osd.3) 99 : cluster [DBG] 7.3 scrub starts 2020-09-08T16:02:23.463 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:23.565 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:23.674 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:23.724 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T16:02:23.736 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:02:23 smithi104 bash[15273]: debug 2020-09-08T16:02:23.442+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:23.777 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:23.882 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:02:23 smithi055 bash[16646]: debug 2020-09-08T16:02:23.676+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:23.882 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:23.897 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:02:23.988 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:24.108 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:24 smithi104 bash[9203]: cluster 2020-09-08T16:02:22.732852+0000 osd.6 (osd.6) 96 : cluster [DBG] 7.7 scrub ok 2020-09-08T16:02:24.108 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:24 smithi104 bash[9203]: cluster 2020-09-08T16:02:22.907738+0000 osd.11 (osd.11) 96 : cluster [DBG] 7.5 scrub starts 2020-09-08T16:02:24.112 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:24.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:24 smithi055 bash[9296]: cluster 2020-09-08T16:02:22.732852+0000 osd.6 (osd.6) 96 : cluster [DBG] 7.7 scrub ok 2020-09-08T16:02:24.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:24 smithi055 bash[9296]: cluster 2020-09-08T16:02:22.907738+0000 osd.11 (osd.11) 96 : cluster [DBG] 7.5 scrub starts 2020-09-08T16:02:24.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:02:23 smithi055 bash[11956]: debug 2020-09-08T16:02:23.884+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:24.158 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:02:24 smithi055 bash[11956]: debug 2020-09-08T16:02:24.112+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:24.217 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:02:24.336 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:02:24.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:24 smithi166 bash[9446]: cluster 2020-09-08T16:02:22.732852+0000 osd.6 (osd.6) 96 : cluster [DBG] 7.7 scrub ok 2020-09-08T16:02:24.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:24 smithi166 bash[9446]: cluster 2020-09-08T16:02:22.907738+0000 osd.11 (osd.11) 96 : cluster [DBG] 7.5 scrub starts 2020-09-08T16:02:24.444 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:02:24.656 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:02:24 smithi055 bash[14301]: debug 2020-09-08T16:02:24.336+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:25.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:25 smithi166 bash[9446]: cluster 2020-09-08T16:02:23.969306+0000 osd.0 (osd.0) 95 : cluster [DBG] 7.6 scrub ok 2020-09-08T16:02:25.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:25 smithi166 bash[9446]: cluster 2020-09-08T16:02:23.991272+0000 mgr.x (mgr.24118) 1052 : cluster [DBG] pgmap v1554: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 7.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.36k op/s 2020-09-08T16:02:25.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:25 smithi166 bash[9446]: cluster 2020-09-08T16:02:24.659960+0000 osd.11 (osd.11) 97 : cluster [DBG] 7.5 scrub ok 2020-09-08T16:02:25.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:25 smithi055 bash[9296]: cluster 2020-09-08T16:02:23.969306+0000 osd.0 (osd.0) 95 : cluster [DBG] 7.6 scrub ok 2020-09-08T16:02:25.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:25 smithi055 bash[9296]: cluster 2020-09-08T16:02:23.991272+0000 mgr.x (mgr.24118) 1052 : cluster [DBG] pgmap v1554: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 7.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.36k op/s 2020-09-08T16:02:25.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:25 smithi055 bash[9296]: cluster 2020-09-08T16:02:24.659960+0000 osd.11 (osd.11) 97 : cluster [DBG] 7.5 scrub ok 2020-09-08T16:02:25.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:25 smithi104 bash[9203]: cluster 2020-09-08T16:02:23.969306+0000 osd.0 (osd.0) 95 : cluster [DBG] 7.6 scrub ok 2020-09-08T16:02:25.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:25 smithi104 bash[9203]: cluster 2020-09-08T16:02:23.991272+0000 mgr.x (mgr.24118) 1052 : cluster [DBG] pgmap v1554: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 7.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.36k op/s 2020-09-08T16:02:25.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:25 smithi104 bash[9203]: cluster 2020-09-08T16:02:24.659960+0000 osd.11 (osd.11) 97 : cluster [DBG] 7.5 scrub ok 2020-09-08T16:02:25.642 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:02:25.750 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:25.871 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:25.912 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T16:02:25.913 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:02:25.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:02:25 smithi104 bash[22305]: debug 2020-09-08T16:02:25.642+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:25.978 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:26.097 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:02:25 smithi055 bash[11956]: debug 2020-09-08T16:02:25.872+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:26.098 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:26.199 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:02:26.343 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:02:26.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:26 smithi166 bash[9446]: cluster 2020-09-08T16:02:25.158012+0000 osd.3 (osd.3) 100 : cluster [DBG] 7.3 scrub ok 2020-09-08T16:02:26.369 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:02:26 smithi166 bash[16593]: debug 2020-09-08T16:02:26.337+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:26.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:02:26 smithi055 bash[11956]: debug 2020-09-08T16:02:26.088+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:26.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:26 smithi055 bash[9296]: cluster 2020-09-08T16:02:25.158012+0000 osd.3 (osd.3) 100 : cluster [DBG] 7.3 scrub ok 2020-09-08T16:02:26.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:26 smithi104 bash[9203]: cluster 2020-09-08T16:02:25.158012+0000 osd.3 (osd.3) 100 : cluster [DBG] 7.3 scrub ok 2020-09-08T16:02:26.447 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:26.571 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:26.673 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:26.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:02:26 smithi166 bash[18960]: debug 2020-09-08T16:02:26.569+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:27.121 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:27 smithi104 bash[9203]: cluster 2020-09-08T16:02:25.991952+0000 mgr.x (mgr.24118) 1053 : cluster [DBG] pgmap v1555: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 7.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T16:02:27.121 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:27 smithi104 bash[9203]: cluster 2020-09-08T16:02:25.993412+0000 osd.5 (osd.5) 95 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:27.121 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:27 smithi104 bash[9203]: cluster 2020-09-08T16:02:25.993521+0000 osd.5 (osd.5) 96 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:27.122 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:27 smithi104 bash[9203]: cluster 2020-09-08T16:02:26.130603+0000 mon.a (mon.0) 1927 : cluster [DBG] osdmap e587: 12 total, 12 up, 12 in 2020-09-08T16:02:27.122 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:27 smithi104 bash[9203]: cluster 2020-09-08T16:02:26.483578+0000 osd.2 (osd.2) 90 : cluster [DBG] 7.e scrub starts 2020-09-08T16:02:27.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:27 smithi055 bash[9296]: cluster 2020-09-08T16:02:25.991952+0000 mgr.x (mgr.24118) 1053 : cluster [DBG] pgmap v1555: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 7.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T16:02:27.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:27 smithi055 bash[9296]: cluster 2020-09-08T16:02:25.993412+0000 osd.5 (osd.5) 95 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:27.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:27 smithi055 bash[9296]: cluster 2020-09-08T16:02:25.993521+0000 osd.5 (osd.5) 96 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:27.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:27 smithi055 bash[9296]: cluster 2020-09-08T16:02:26.130603+0000 mon.a (mon.0) 1927 : cluster [DBG] osdmap e587: 12 total, 12 up, 12 in 2020-09-08T16:02:27.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:27 smithi055 bash[9296]: cluster 2020-09-08T16:02:26.483578+0000 osd.2 (osd.2) 90 : cluster [DBG] 7.e scrub starts 2020-09-08T16:02:27.507 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:27.611 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:27.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:27 smithi166 bash[9446]: cluster 2020-09-08T16:02:25.991952+0000 mgr.x (mgr.24118) 1053 : cluster [DBG] pgmap v1555: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 7.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T16:02:27.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:27 smithi166 bash[9446]: cluster 2020-09-08T16:02:25.993412+0000 osd.5 (osd.5) 95 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:27.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:27 smithi166 bash[9446]: cluster 2020-09-08T16:02:25.993521+0000 osd.5 (osd.5) 96 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:27.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:27 smithi166 bash[9446]: cluster 2020-09-08T16:02:26.130603+0000 mon.a (mon.0) 1927 : cluster [DBG] osdmap e587: 12 total, 12 up, 12 in 2020-09-08T16:02:27.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:27 smithi166 bash[9446]: cluster 2020-09-08T16:02:26.483578+0000 osd.2 (osd.2) 90 : cluster [DBG] 7.e scrub starts 2020-09-08T16:02:27.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:02:27 smithi104 bash[15273]: debug 2020-09-08T16:02:27.502+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:28.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:28 smithi055 bash[9296]: cluster 2020-09-08T16:02:27.100129+0000 osd.10 (osd.10) 73 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:28.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:28 smithi055 bash[9296]: cluster 2020-09-08T16:02:27.100196+0000 osd.10 (osd.10) 74 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:28.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:28 smithi055 bash[9296]: cluster 2020-09-08T16:02:27.138094+0000 mon.a (mon.0) 1928 : cluster [DBG] osdmap e588: 12 total, 12 up, 12 in 2020-09-08T16:02:28.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:28 smithi055 bash[9296]: cluster 2020-09-08T16:02:27.638466+0000 osd.1 (osd.1) 116 : cluster [DBG] 7.b scrub starts 2020-09-08T16:02:28.439 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:28 smithi104 bash[9203]: cluster 2020-09-08T16:02:27.100129+0000 osd.10 (osd.10) 73 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:28.439 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:28 smithi104 bash[9203]: cluster 2020-09-08T16:02:27.100196+0000 osd.10 (osd.10) 74 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:28.439 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:28 smithi104 bash[9203]: cluster 2020-09-08T16:02:27.138094+0000 mon.a (mon.0) 1928 : cluster [DBG] osdmap e588: 12 total, 12 up, 12 in 2020-09-08T16:02:28.439 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:28 smithi104 bash[9203]: cluster 2020-09-08T16:02:27.638466+0000 osd.1 (osd.1) 116 : cluster [DBG] 7.b scrub starts 2020-09-08T16:02:28.440 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2020-09-08T16:02:28.613 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:02:28.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:28 smithi166 bash[9446]: cluster 2020-09-08T16:02:27.100129+0000 osd.10 (osd.10) 73 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:28.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:28 smithi166 bash[9446]: cluster 2020-09-08T16:02:27.100196+0000 osd.10 (osd.10) 74 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:28.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:28 smithi166 bash[9446]: cluster 2020-09-08T16:02:27.138094+0000 mon.a (mon.0) 1928 : cluster [DBG] osdmap e588: 12 total, 12 up, 12 in 2020-09-08T16:02:28.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:28 smithi166 bash[9446]: cluster 2020-09-08T16:02:27.638466+0000 osd.1 (osd.1) 116 : cluster [DBG] 7.b scrub starts 2020-09-08T16:02:28.660 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:28.762 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:02:28.798 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:02:28 smithi104 bash[15273]: debug 2020-09-08T16:02:28.654+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:28.912 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:02:29.016 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:02:29.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:02:28 smithi166 bash[16593]: debug 2020-09-08T16:02:28.909+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:29.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:29 smithi055 bash[9296]: cluster 2020-09-08T16:02:27.992509+0000 mgr.x (mgr.24118) 1054 : cluster [DBG] pgmap v1558: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.22k op/s 2020-09-08T16:02:29.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:29 smithi055 bash[9296]: cluster 2020-09-08T16:02:28.169761+0000 osd.0 (osd.0) 96 : cluster [DBG] 7.f scrub starts 2020-09-08T16:02:29.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:29 smithi104 bash[9203]: cluster 2020-09-08T16:02:27.992509+0000 mgr.x (mgr.24118) 1054 : cluster [DBG] pgmap v1558: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.22k op/s 2020-09-08T16:02:29.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:29 smithi104 bash[9203]: cluster 2020-09-08T16:02:28.169761+0000 osd.0 (osd.0) 96 : cluster [DBG] 7.f scrub starts 2020-09-08T16:02:29.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:29 smithi166 bash[9446]: cluster 2020-09-08T16:02:27.992509+0000 mgr.x (mgr.24118) 1054 : cluster [DBG] pgmap v1558: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.22k op/s 2020-09-08T16:02:29.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:29 smithi166 bash[9446]: cluster 2020-09-08T16:02:28.169761+0000 osd.0 (osd.0) 96 : cluster [DBG] 7.f scrub starts 2020-09-08T16:02:30.023 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:02:30.127 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:30.243 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:30.283 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:02:30 smithi104 bash[19922]: debug 2020-09-08T16:02:30.018+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:30.283 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:30 smithi104 bash[9203]: cluster 2020-09-08T16:02:29.053812+0000 osd.2 (osd.2) 91 : cluster [DBG] 7.e scrub ok 2020-09-08T16:02:30.348 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:30.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:30 smithi055 bash[9296]: cluster 2020-09-08T16:02:29.053812+0000 osd.2 (osd.2) 91 : cluster [DBG] 7.e scrub ok 2020-09-08T16:02:30.408 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:02:30 smithi055 bash[11956]: debug 2020-09-08T16:02:30.244+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:30.640 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:30 smithi166 bash[9446]: cluster 2020-09-08T16:02:29.053812+0000 osd.2 (osd.2) 91 : cluster [DBG] 7.e scrub ok 2020-09-08T16:02:31.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:31 smithi055 bash[9296]: cluster 2020-09-08T16:02:29.993001+0000 mgr.x (mgr.24118) 1055 : cluster [DBG] pgmap v1559: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T16:02:31.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:31 smithi055 bash[9296]: cluster 2020-09-08T16:02:30.229035+0000 osd.1 (osd.1) 117 : cluster [DBG] 7.b scrub ok 2020-09-08T16:02:31.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:31 smithi055 bash[9296]: cluster 2020-09-08T16:02:30.533281+0000 osd.9 (osd.9) 87 : cluster [DBG] 7.a scrub starts 2020-09-08T16:02:31.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:31 smithi104 bash[9203]: cluster 2020-09-08T16:02:29.993001+0000 mgr.x (mgr.24118) 1055 : cluster [DBG] pgmap v1559: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T16:02:31.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:31 smithi104 bash[9203]: cluster 2020-09-08T16:02:30.229035+0000 osd.1 (osd.1) 117 : cluster [DBG] 7.b scrub ok 2020-09-08T16:02:31.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:31 smithi104 bash[9203]: cluster 2020-09-08T16:02:30.533281+0000 osd.9 (osd.9) 87 : cluster [DBG] 7.a scrub starts 2020-09-08T16:02:31.607 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:31 smithi166 bash[9446]: cluster 2020-09-08T16:02:29.993001+0000 mgr.x (mgr.24118) 1055 : cluster [DBG] pgmap v1559: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T16:02:31.607 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:31 smithi166 bash[9446]: cluster 2020-09-08T16:02:30.229035+0000 osd.1 (osd.1) 117 : cluster [DBG] 7.b scrub ok 2020-09-08T16:02:31.608 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:31 smithi166 bash[9446]: cluster 2020-09-08T16:02:30.533281+0000 osd.9 (osd.9) 87 : cluster [DBG] 7.a scrub starts 2020-09-08T16:02:31.608 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:31.712 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:31.820 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:31.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:02:31 smithi166 bash[18960]: debug 2020-09-08T16:02:31.605+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:31.923 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:02:32.153 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:02:31 smithi055 bash[16646]: debug 2020-09-08T16:02:31.820+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:32.208 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:02:32.247 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:02:32.284 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:02:32.323 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:02:32.361 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:02:32.392 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:02:32.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:32 smithi055 bash[9296]: cluster 2020-09-08T16:02:31.227634+0000 osd.0 (osd.0) 97 : cluster [DBG] 7.f scrub ok 2020-09-08T16:02:32.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:32 smithi055 bash[9296]: cluster 2020-09-08T16:02:31.551056+0000 osd.3 (osd.3) 101 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:32.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:32 smithi055 bash[9296]: cluster 2020-09-08T16:02:31.551135+0000 osd.3 (osd.3) 102 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:32.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:32 smithi104 bash[9203]: cluster 2020-09-08T16:02:31.227634+0000 osd.0 (osd.0) 97 : cluster [DBG] 7.f scrub ok 2020-09-08T16:02:32.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:32 smithi104 bash[9203]: cluster 2020-09-08T16:02:31.551056+0000 osd.3 (osd.3) 101 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:32.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:32 smithi104 bash[9203]: cluster 2020-09-08T16:02:31.551135+0000 osd.3 (osd.3) 102 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:32.427 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:02:32.473 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:02:32.512 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:02:32.547 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:02:32.580 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:02:32.614 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:02:32.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:32 smithi166 bash[9446]: cluster 2020-09-08T16:02:31.227634+0000 osd.0 (osd.0) 97 : cluster [DBG] 7.f scrub ok 2020-09-08T16:02:32.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:32 smithi166 bash[9446]: cluster 2020-09-08T16:02:31.551056+0000 osd.3 (osd.3) 101 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:32.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:32 smithi166 bash[9446]: cluster 2020-09-08T16:02:31.551135+0000 osd.3 (osd.3) 102 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:32.618 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:02:32.618 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:02:32.618 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:02:32.618 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:02:32.619 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:02:32.619 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:02:32.619 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:02:32.619 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:02:32.619 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:02:32.620 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:02:32.620 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:02:32.620 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:02:32.884 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:02:32.987 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:33.126 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:33.219 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:02:32 smithi104 bash[22305]: debug 2020-09-08T16:02:32.874+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:33.228 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:02:33.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:02:33 smithi055 bash[16646]: debug 2020-09-08T16:02:33.124+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:33.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:33 smithi055 bash[9296]: cluster 2020-09-08T16:02:31.993426+0000 mgr.x (mgr.24118) 1056 : cluster [DBG] pgmap v1560: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 841 op/s 2020-09-08T16:02:33.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:33 smithi055 bash[9296]: cluster 2020-09-08T16:02:32.161041+0000 mon.a (mon.0) 1929 : cluster [DBG] osdmap e589: 12 total, 12 up, 12 in 2020-09-08T16:02:33.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:33 smithi055 bash[9296]: audit 2020-09-08T16:02:32.164137+0000 mon.c (mon.2) 79 : audit [DBG] from='client.? v1:172.21.15.104:0/1521195260' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:02:33.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:33 smithi166 bash[9446]: cluster 2020-09-08T16:02:31.993426+0000 mgr.x (mgr.24118) 1056 : cluster [DBG] pgmap v1560: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 841 op/s 2020-09-08T16:02:33.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:33 smithi166 bash[9446]: cluster 2020-09-08T16:02:32.161041+0000 mon.a (mon.0) 1929 : cluster [DBG] osdmap e589: 12 total, 12 up, 12 in 2020-09-08T16:02:33.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:33 smithi166 bash[9446]: audit 2020-09-08T16:02:32.164137+0000 mon.c (mon.2) 79 : audit [DBG] from='client.? v1:172.21.15.104:0/1521195260' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:02:33.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:33 smithi104 bash[9203]: cluster 2020-09-08T16:02:31.993426+0000 mgr.x (mgr.24118) 1056 : cluster [DBG] pgmap v1560: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 841 op/s 2020-09-08T16:02:33.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:33 smithi104 bash[9203]: cluster 2020-09-08T16:02:32.161041+0000 mon.a (mon.0) 1929 : cluster [DBG] osdmap e589: 12 total, 12 up, 12 in 2020-09-08T16:02:33.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:33 smithi104 bash[9203]: audit 2020-09-08T16:02:32.164137+0000 mon.c (mon.2) 79 : audit [DBG] from='client.? v1:172.21.15.104:0/1521195260' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:02:34.259 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:02:34.364 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:34.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:02:34 smithi104 bash[22305]: debug 2020-09-08T16:02:34.262+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:34.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:34 smithi104 bash[9203]: cluster 2020-09-08T16:02:32.912433+0000 osd.9 (osd.9) 88 : cluster [DBG] 7.a scrub ok 2020-09-08T16:02:34.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:34 smithi166 bash[9446]: cluster 2020-09-08T16:02:32.912433+0000 osd.9 (osd.9) 88 : cluster [DBG] 7.a scrub ok 2020-09-08T16:02:34.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:34 smithi055 bash[9296]: cluster 2020-09-08T16:02:32.912433+0000 osd.9 (osd.9) 88 : cluster [DBG] 7.a scrub ok 2020-09-08T16:02:35.085 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:35.151 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:02:35 smithi104 bash[17588]: debug 2020-09-08T16:02:35.086+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:35.190 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:35.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:35 smithi104 bash[9203]: cluster 2020-09-08T16:02:33.993933+0000 mgr.x (mgr.24118) 1057 : cluster [DBG] pgmap v1562: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.28k op/s 2020-09-08T16:02:35.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:35 smithi104 bash[9203]: cluster 2020-09-08T16:02:34.124885+0000 osd.0 (osd.0) 98 : cluster [DBG] 7.9 scrub starts 2020-09-08T16:02:35.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:35 smithi166 bash[9446]: cluster 2020-09-08T16:02:33.993933+0000 mgr.x (mgr.24118) 1057 : cluster [DBG] pgmap v1562: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.28k op/s 2020-09-08T16:02:35.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:35 smithi166 bash[9446]: cluster 2020-09-08T16:02:34.124885+0000 osd.0 (osd.0) 98 : cluster [DBG] 7.9 scrub starts 2020-09-08T16:02:35.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:35 smithi055 bash[9296]: cluster 2020-09-08T16:02:33.993933+0000 mgr.x (mgr.24118) 1057 : cluster [DBG] pgmap v1562: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.28k op/s 2020-09-08T16:02:35.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:35 smithi055 bash[9296]: cluster 2020-09-08T16:02:34.124885+0000 osd.0 (osd.0) 98 : cluster [DBG] 7.9 scrub starts 2020-09-08T16:02:35.995 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:36.102 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:36.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:02:35 smithi104 bash[17588]: debug 2020-09-08T16:02:35.994+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:36.220 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:36.334 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:02:36.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:02:36 smithi055 bash[11956]: debug 2020-09-08T16:02:36.224+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:37.157 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:02:37.263 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:02:37.315 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:02:37.366 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:02:37.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:37 smithi104 bash[9203]: cluster 2020-09-08T16:02:35.994389+0000 mgr.x (mgr.24118) 1058 : cluster [DBG] pgmap v1563: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 954 op/s 2020-09-08T16:02:37.431 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:37 smithi104 bash[9203]: cluster 2020-09-08T16:02:36.680014+0000 osd.0 (osd.0) 99 : cluster [DBG] 7.9 scrub ok 2020-09-08T16:02:37.432 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:02:37 smithi104 bash[22305]: debug 2020-09-08T16:02:37.258+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:37.626 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:37 smithi166 bash[9446]: cluster 2020-09-08T16:02:35.994389+0000 mgr.x (mgr.24118) 1058 : cluster [DBG] pgmap v1563: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 954 op/s 2020-09-08T16:02:37.638 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:37 smithi166 bash[9446]: cluster 2020-09-08T16:02:36.680014+0000 osd.0 (osd.0) 99 : cluster [DBG] 7.9 scrub ok 2020-09-08T16:02:37.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:37 smithi055 bash[9296]: cluster 2020-09-08T16:02:35.994389+0000 mgr.x (mgr.24118) 1058 : cluster [DBG] pgmap v1563: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 954 op/s 2020-09-08T16:02:37.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:37 smithi055 bash[9296]: cluster 2020-09-08T16:02:36.680014+0000 osd.0 (osd.0) 99 : cluster [DBG] 7.9 scrub ok 2020-09-08T16:02:38.555 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:02:38.658 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:38.796 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:38.899 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:02:38.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:02:38 smithi104 bash[19922]: debug 2020-09-08T16:02:38.558+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:39.010 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:02:38 smithi055 bash[16646]: debug 2020-09-08T16:02:38.796+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:39.010 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:02:39.114 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:39.194 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T16:02:39.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:39 smithi055 bash[9296]: cluster 2020-09-08T16:02:37.509496+0000 osd.8 (osd.8) 79 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:39.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:39 smithi055 bash[9296]: cluster 2020-09-08T16:02:37.509585+0000 osd.8 (osd.8) 80 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:39.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:39 smithi055 bash[9296]: cluster 2020-09-08T16:02:37.994944+0000 mgr.x (mgr.24118) 1059 : cluster [DBG] pgmap v1564: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T16:02:39.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:39 smithi055 bash[9296]: cluster 2020-09-08T16:02:38.187463+0000 mon.a (mon.0) 1930 : cluster [DBG] osdmap e590: 12 total, 12 up, 12 in 2020-09-08T16:02:39.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:39 smithi055 bash[9296]: audit 2020-09-08T16:02:38.783776+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? v1:172.21.15.104:0/4216469661' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:02:39.408 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:02:39 smithi055 bash[14301]: debug 2020-09-08T16:02:39.008+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:39.433 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:39 smithi104 bash[9203]: cluster 2020-09-08T16:02:37.509496+0000 osd.8 (osd.8) 79 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:39.434 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:39 smithi104 bash[9203]: cluster 2020-09-08T16:02:37.509585+0000 osd.8 (osd.8) 80 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:39.434 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:39 smithi104 bash[9203]: cluster 2020-09-08T16:02:37.994944+0000 mgr.x (mgr.24118) 1059 : cluster [DBG] pgmap v1564: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T16:02:39.434 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:39 smithi104 bash[9203]: cluster 2020-09-08T16:02:38.187463+0000 mon.a (mon.0) 1930 : cluster [DBG] osdmap e590: 12 total, 12 up, 12 in 2020-09-08T16:02:39.434 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:39 smithi104 bash[9203]: audit 2020-09-08T16:02:38.783776+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? v1:172.21.15.104:0/4216469661' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:02:39.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:39 smithi166 bash[9446]: cluster 2020-09-08T16:02:37.509496+0000 osd.8 (osd.8) 79 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:39.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:39 smithi166 bash[9446]: cluster 2020-09-08T16:02:37.509585+0000 osd.8 (osd.8) 80 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:39.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:39 smithi166 bash[9446]: cluster 2020-09-08T16:02:37.994944+0000 mgr.x (mgr.24118) 1059 : cluster [DBG] pgmap v1564: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T16:02:39.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:39 smithi166 bash[9446]: cluster 2020-09-08T16:02:38.187463+0000 mon.a (mon.0) 1930 : cluster [DBG] osdmap e590: 12 total, 12 up, 12 in 2020-09-08T16:02:39.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:39 smithi166 bash[9446]: audit 2020-09-08T16:02:38.783776+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? v1:172.21.15.104:0/4216469661' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:02:39.644 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:39.748 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:39.855 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:39.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:02:39 smithi104 bash[15273]: debug 2020-09-08T16:02:39.634+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:39.959 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:02:40.132 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:02:40.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:02:39 smithi055 bash[16646]: debug 2020-09-08T16:02:39.856+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:40.236 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:02:40.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:40 smithi166 bash[9446]: cluster 2020-09-08T16:02:39.185440+0000 mon.a (mon.0) 1932 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:02:40.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:40 smithi166 bash[9446]: cluster 2020-09-08T16:02:39.185473+0000 mon.a (mon.0) 1933 : cluster [INF] Cluster is now healthy 2020-09-08T16:02:40.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:40 smithi166 bash[9446]: audit 2020-09-08T16:02:39.192967+0000 mon.a (mon.0) 1934 : audit [INF] from='client.? v1:172.21.15.104:0/4216469661' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:02:40.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:40 smithi166 bash[9446]: cluster 2020-09-08T16:02:39.193016+0000 mon.a (mon.0) 1935 : cluster [DBG] osdmap e591: 12 total, 12 up, 12 in 2020-09-08T16:02:40.370 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:02:40 smithi166 bash[14254]: debug 2020-09-08T16:02:40.133+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:40.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:40 smithi055 bash[9296]: cluster 2020-09-08T16:02:39.185440+0000 mon.a (mon.0) 1932 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:02:40.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:40 smithi055 bash[9296]: cluster 2020-09-08T16:02:39.185473+0000 mon.a (mon.0) 1933 : cluster [INF] Cluster is now healthy 2020-09-08T16:02:40.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:40 smithi055 bash[9296]: audit 2020-09-08T16:02:39.192967+0000 mon.a (mon.0) 1934 : audit [INF] from='client.? v1:172.21.15.104:0/4216469661' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:02:40.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:40 smithi055 bash[9296]: cluster 2020-09-08T16:02:39.193016+0000 mon.a (mon.0) 1935 : cluster [DBG] osdmap e591: 12 total, 12 up, 12 in 2020-09-08T16:02:40.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:40 smithi104 bash[9203]: cluster 2020-09-08T16:02:39.185440+0000 mon.a (mon.0) 1932 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:02:40.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:40 smithi104 bash[9203]: cluster 2020-09-08T16:02:39.185473+0000 mon.a (mon.0) 1933 : cluster [INF] Cluster is now healthy 2020-09-08T16:02:40.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:40 smithi104 bash[9203]: audit 2020-09-08T16:02:39.192967+0000 mon.a (mon.0) 1934 : audit [INF] from='client.? v1:172.21.15.104:0/4216469661' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:02:40.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:40 smithi104 bash[9203]: cluster 2020-09-08T16:02:39.193016+0000 mon.a (mon.0) 1935 : cluster [DBG] osdmap e591: 12 total, 12 up, 12 in 2020-09-08T16:02:41.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:41 smithi104 bash[9203]: cluster 2020-09-08T16:02:39.995359+0000 mgr.x (mgr.24118) 1060 : cluster [DBG] pgmap v1567: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.07k op/s 2020-09-08T16:02:41.517 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T16:02:41.518 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T16:02:41.518 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2020-09-08T16:02:41.518 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2020-09-08T16:02:41.518 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T16:02:41.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:41 smithi166 bash[9446]: cluster 2020-09-08T16:02:39.995359+0000 mgr.x (mgr.24118) 1060 : cluster [DBG] pgmap v1567: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.07k op/s 2020-09-08T16:02:41.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:41 smithi055 bash[9296]: cluster 2020-09-08T16:02:39.995359+0000 mgr.x (mgr.24118) 1060 : cluster [DBG] pgmap v1567: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.07k op/s 2020-09-08T16:02:42.988 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T16:02:43.125 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:02:43.431 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:43 smithi104 bash[9203]: cluster 2020-09-08T16:02:41.995745+0000 mgr.x (mgr.24118) 1061 : cluster [DBG] pgmap v1568: 17 pgs: 17 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 842 op/s 2020-09-08T16:02:43.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:43 smithi166 bash[9446]: cluster 2020-09-08T16:02:41.995745+0000 mgr.x (mgr.24118) 1061 : cluster [DBG] pgmap v1568: 17 pgs: 17 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 842 op/s 2020-09-08T16:02:43.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:43 smithi055 bash[9296]: cluster 2020-09-08T16:02:41.995745+0000 mgr.x (mgr.24118) 1061 : cluster [DBG] pgmap v1568: 17 pgs: 17 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 842 op/s 2020-09-08T16:02:43.773 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2020-09-08T16:02:43.784 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:02:43.887 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:43.945 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:02:44.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:02:43 smithi055 bash[19003]: debug 2020-09-08T16:02:43.784+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:44.777 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:44.883 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:45.174 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:02:44 smithi104 bash[15273]: debug 2020-09-08T16:02:44.778+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:45.175 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:45.285 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:02:45.417 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:02:45.435 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:45 smithi104 bash[9203]: cluster 2020-09-08T16:02:43.996214+0000 mgr.x (mgr.24118) 1062 : cluster [DBG] pgmap v1569: 17 pgs: 17 active+clean; 6.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.28k op/s 2020-09-08T16:02:45.435 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:02:45 smithi104 bash[17588]: debug 2020-09-08T16:02:45.174+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:45.537 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:45.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:45 smithi166 bash[9446]: cluster 2020-09-08T16:02:43.996214+0000 mgr.x (mgr.24118) 1062 : cluster [DBG] pgmap v1569: 17 pgs: 17 active+clean; 6.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.28k op/s 2020-09-08T16:02:45.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:45 smithi055 bash[9296]: cluster 2020-09-08T16:02:43.996214+0000 mgr.x (mgr.24118) 1062 : cluster [DBG] pgmap v1569: 17 pgs: 17 active+clean; 6.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.28k op/s 2020-09-08T16:02:45.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:02:45 smithi055 bash[19003]: debug 2020-09-08T16:02:45.420+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:46.212 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:46.316 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:46.451 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:46.454 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:46 smithi166 bash[9446]: cluster 2020-09-08T16:02:45.340760+0000 osd.1 (osd.1) 118 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:46.455 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:46 smithi166 bash[9446]: cluster 2020-09-08T16:02:45.340860+0000 osd.1 (osd.1) 119 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:46.455 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:02:46 smithi166 bash[18960]: debug 2020-09-08T16:02:46.449+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:46.511 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:46 smithi104 bash[9203]: cluster 2020-09-08T16:02:45.340760+0000 osd.1 (osd.1) 118 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:46.511 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:46 smithi104 bash[9203]: cluster 2020-09-08T16:02:45.340860+0000 osd.1 (osd.1) 119 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:46.512 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:02:46 smithi104 bash[17588]: debug 2020-09-08T16:02:46.214+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:46.556 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:02:46.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:46 smithi055 bash[9296]: cluster 2020-09-08T16:02:45.340760+0000 osd.1 (osd.1) 118 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:46.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:46 smithi055 bash[9296]: cluster 2020-09-08T16:02:45.340860+0000 osd.1 (osd.1) 119 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:46.689 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:02:46.793 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:46.868 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:02:46 smithi166 bash[16593]: debug 2020-09-08T16:02:46.685+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:46.926 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:47.030 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:02:47.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:02:46 smithi055 bash[11956]: debug 2020-09-08T16:02:46.928+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:47.353 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:02:47.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:02:47 smithi104 bash[22305]: debug 2020-09-08T16:02:47.346+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:47.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:47 smithi104 bash[9203]: cluster 2020-09-08T16:02:45.996600+0000 mgr.x (mgr.24118) 1063 : cluster [DBG] pgmap v1570: 17 pgs: 17 active+clean; 6.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 726 op/s 2020-09-08T16:02:47.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:47 smithi104 bash[9203]: cluster 2020-09-08T16:02:46.211575+0000 mon.a (mon.0) 1936 : cluster [DBG] osdmap e592: 12 total, 12 up, 12 in 2020-09-08T16:02:47.455 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:47.465 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:47 smithi055 bash[9296]: cluster 2020-09-08T16:02:45.996600+0000 mgr.x (mgr.24118) 1063 : cluster [DBG] pgmap v1570: 17 pgs: 17 active+clean; 6.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 726 op/s 2020-09-08T16:02:47.465 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:47 smithi055 bash[9296]: cluster 2020-09-08T16:02:46.211575+0000 mon.a (mon.0) 1936 : cluster [DBG] osdmap e592: 12 total, 12 up, 12 in 2020-09-08T16:02:47.564 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:47.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:47 smithi166 bash[9446]: cluster 2020-09-08T16:02:45.996600+0000 mgr.x (mgr.24118) 1063 : cluster [DBG] pgmap v1570: 17 pgs: 17 active+clean; 6.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 726 op/s 2020-09-08T16:02:47.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:47 smithi166 bash[9446]: cluster 2020-09-08T16:02:46.211575+0000 mon.a (mon.0) 1936 : cluster [DBG] osdmap e592: 12 total, 12 up, 12 in 2020-09-08T16:02:47.669 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:02:47.791 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:02:47 smithi055 bash[16646]: debug 2020-09-08T16:02:47.568+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:47.792 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:02:47 smithi055 bash[19003]: debug 2020-09-08T16:02:47.792+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:47.792 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:02:47.896 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:48.419 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:48.523 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:48.635 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:48.678 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:02:48 smithi104 bash[15273]: debug 2020-09-08T16:02:48.418+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:48.739 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:48.859 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:48.860 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:02:48 smithi055 bash[16646]: debug 2020-09-08T16:02:48.852+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:48.860 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:02:48 smithi055 bash[11956]: debug 2020-09-08T16:02:48.636+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:48.961 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:49.068 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:49.158 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:02:49 smithi055 bash[11956]: debug 2020-09-08T16:02:49.068+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:49.172 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:49.314 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:49.326 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:02:49.370 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:02:49.408 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:02:49.416 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:49.443 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:02:49.489 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:02:49.528 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:02:49.567 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:49.567 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:49 smithi055 bash[9296]: cluster 2020-09-08T16:02:47.997016+0000 mgr.x (mgr.24118) 1064 : cluster [DBG] pgmap v1572: 17 pgs: 17 active+clean; 6.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T16:02:49.567 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:49 smithi055 bash[9296]: cluster 2020-09-08T16:02:48.226995+0000 osd.6 (osd.6) 97 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:49.568 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:49 smithi055 bash[9296]: cluster 2020-09-08T16:02:48.227098+0000 osd.6 (osd.6) 98 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:49.568 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:02:49 smithi055 bash[11956]: debug 2020-09-08T16:02:49.564+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:49.572 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:02:49.616 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:02:49.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:49 smithi166 bash[9446]: cluster 2020-09-08T16:02:47.997016+0000 mgr.x (mgr.24118) 1064 : cluster [DBG] pgmap v1572: 17 pgs: 17 active+clean; 6.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T16:02:49.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:49 smithi166 bash[9446]: cluster 2020-09-08T16:02:48.226995+0000 osd.6 (osd.6) 97 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:49.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:49 smithi166 bash[9446]: cluster 2020-09-08T16:02:48.227098+0000 osd.6 (osd.6) 98 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:49.617 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:02:49 smithi166 bash[18960]: debug 2020-09-08T16:02:49.309+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:49.659 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:02:49.669 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:49.682 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:49 smithi104 bash[9203]: cluster 2020-09-08T16:02:47.997016+0000 mgr.x (mgr.24118) 1064 : cluster [DBG] pgmap v1572: 17 pgs: 17 active+clean; 6.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T16:02:49.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:49 smithi104 bash[9203]: cluster 2020-09-08T16:02:48.226995+0000 osd.6 (osd.6) 97 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:49.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:49 smithi104 bash[9203]: cluster 2020-09-08T16:02:48.227098+0000 osd.6 (osd.6) 98 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:49.702 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:02:49.741 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:02:49.775 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:02:49.779 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:02:49.780 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:02:49.780 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:02:49.780 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:02:49.781 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:02:49.781 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:02:49.781 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:02:49.782 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:02:49.782 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:02:49.782 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:02:49.783 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:02:49.783 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:02:49.794 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:49.899 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:02:49.906 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:02:49 smithi055 bash[11956]: debug 2020-09-08T16:02:49.796+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:50.258 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:02:50.360 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:02:50.512 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:50 smithi166 bash[9446]: cluster 2020-09-08T16:02:49.216278+0000 mon.a (mon.0) 1937 : cluster [DBG] osdmap e593: 12 total, 12 up, 12 in 2020-09-08T16:02:50.512 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:50 smithi166 bash[9446]: audit 2020-09-08T16:02:49.276681+0000 mon.a (mon.0) 1938 : audit [DBG] from='client.? v1:172.21.15.104:0/3786000032' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:02:50.513 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:50 smithi166 bash[9446]: cluster 2020-09-08T16:02:49.297013+0000 osd.0 (osd.0) 100 : cluster [DBG] 7.4 scrub starts 2020-09-08T16:02:50.513 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:50 smithi166 bash[9446]: cluster 2020-09-08T16:02:49.547506+0000 osd.2 (osd.2) 92 : cluster [DBG] 7.8 scrub starts 2020-09-08T16:02:50.513 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:02:50 smithi166 bash[16593]: debug 2020-09-08T16:02:50.509+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:50.514 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:02:50.616 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:50.625 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:50 smithi055 bash[9296]: cluster 2020-09-08T16:02:49.216278+0000 mon.a (mon.0) 1937 : cluster [DBG] osdmap e593: 12 total, 12 up, 12 in 2020-09-08T16:02:50.626 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:50 smithi055 bash[9296]: audit 2020-09-08T16:02:49.276681+0000 mon.a (mon.0) 1938 : audit [DBG] from='client.? v1:172.21.15.104:0/3786000032' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:02:50.626 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:50 smithi055 bash[9296]: cluster 2020-09-08T16:02:49.297013+0000 osd.0 (osd.0) 100 : cluster [DBG] 7.4 scrub starts 2020-09-08T16:02:50.626 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:50 smithi055 bash[9296]: cluster 2020-09-08T16:02:49.547506+0000 osd.2 (osd.2) 92 : cluster [DBG] 7.8 scrub starts 2020-09-08T16:02:50.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:50 smithi104 bash[9203]: cluster 2020-09-08T16:02:49.216278+0000 mon.a (mon.0) 1937 : cluster [DBG] osdmap e593: 12 total, 12 up, 12 in 2020-09-08T16:02:50.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:50 smithi104 bash[9203]: audit 2020-09-08T16:02:49.276681+0000 mon.a (mon.0) 1938 : audit [DBG] from='client.? v1:172.21.15.104:0/3786000032' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:02:50.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:50 smithi104 bash[9203]: cluster 2020-09-08T16:02:49.297013+0000 osd.0 (osd.0) 100 : cluster [DBG] 7.4 scrub starts 2020-09-08T16:02:50.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:50 smithi104 bash[9203]: cluster 2020-09-08T16:02:49.547506+0000 osd.2 (osd.2) 92 : cluster [DBG] 7.8 scrub starts 2020-09-08T16:02:50.678 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:02:50 smithi104 bash[19922]: debug 2020-09-08T16:02:50.254+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:50.727 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:50.836 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:50.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:02:50 smithi055 bash[16646]: debug 2020-09-08T16:02:50.728+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:50.985 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:02:51.089 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:51.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:02:50 smithi166 bash[18960]: debug 2020-09-08T16:02:50.981+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:51.415 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:51 smithi104 bash[9203]: cluster 2020-09-08T16:02:49.997439+0000 mgr.x (mgr.24118) 1065 : cluster [DBG] pgmap v1574: 17 pgs: 17 active+clean; 6.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.06k op/s 2020-09-08T16:02:51.418 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:51.523 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:02:51.531 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:51 smithi055 bash[9296]: cluster 2020-09-08T16:02:49.997439+0000 mgr.x (mgr.24118) 1065 : cluster [DBG] pgmap v1574: 17 pgs: 17 active+clean; 6.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.06k op/s 2020-09-08T16:02:51.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:51 smithi166 bash[9446]: cluster 2020-09-08T16:02:49.997439+0000 mgr.x (mgr.24118) 1065 : cluster [DBG] pgmap v1574: 17 pgs: 17 active+clean; 6.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.06k op/s 2020-09-08T16:02:51.632 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:02:51.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:02:51 smithi104 bash[17588]: debug 2020-09-08T16:02:51.414+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:51.739 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:51.848 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:02:51 smithi055 bash[14301]: debug 2020-09-08T16:02:51.632+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:51.849 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:51.956 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:02:52.058 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:02:52.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:02:51 smithi055 bash[16646]: debug 2020-09-08T16:02:51.848+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:52.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:02:52 smithi055 bash[14301]: debug 2020-09-08T16:02:52.060+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:52.162 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:52.276 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:52.380 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:02:52.547 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:52 smithi166 bash[9446]: cluster 2020-09-08T16:02:51.533798+0000 osd.2 (osd.2) 93 : cluster [DBG] 7.8 scrub ok 2020-09-08T16:02:52.547 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:52 smithi166 bash[9446]: cluster 2020-09-08T16:02:51.695237+0000 osd.0 (osd.0) 101 : cluster [DBG] 7.4 scrub ok 2020-09-08T16:02:52.552 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:02:52.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:52 smithi055 bash[9296]: cluster 2020-09-08T16:02:51.533798+0000 osd.2 (osd.2) 93 : cluster [DBG] 7.8 scrub ok 2020-09-08T16:02:52.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:52 smithi055 bash[9296]: cluster 2020-09-08T16:02:51.695237+0000 osd.0 (osd.0) 101 : cluster [DBG] 7.4 scrub ok 2020-09-08T16:02:52.658 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:02:52 smithi055 bash[16646]: debug 2020-09-08T16:02:52.276+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:52.660 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:02:52.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:52 smithi104 bash[9203]: cluster 2020-09-08T16:02:51.533798+0000 osd.2 (osd.2) 93 : cluster [DBG] 7.8 scrub ok 2020-09-08T16:02:52.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:52 smithi104 bash[9203]: cluster 2020-09-08T16:02:51.695237+0000 osd.0 (osd.0) 101 : cluster [DBG] 7.4 scrub ok 2020-09-08T16:02:52.781 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:02:52.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:02:52 smithi166 bash[11928]: debug 2020-09-08T16:02:52.776+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:52.867 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:02:52 smithi166 bash[16593]: debug 2020-09-08T16:02:52.545+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:52.888 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:53.127 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:53.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:02:53 smithi104 bash[15273]: debug 2020-09-08T16:02:53.126+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:53.231 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:02:53.458 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:02:53.562 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:02:53.571 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:53 smithi055 bash[9296]: cluster 2020-09-08T16:02:51.997874+0000 mgr.x (mgr.24118) 1066 : cluster [DBG] pgmap v1575: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 795 op/s 2020-09-08T16:02:53.572 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:53 smithi055 bash[9296]: cluster 2020-09-08T16:02:52.104389+0000 osd.7 (osd.7) 129 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:02:53.572 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:53 smithi055 bash[9296]: cluster 2020-09-08T16:02:52.209516+0000 osd.7 (osd.7) 130 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:02:53.572 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:53 smithi055 bash[9296]: cluster 2020-09-08T16:02:52.224342+0000 osd.11 (osd.11) 98 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:53.572 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:53 smithi055 bash[9296]: cluster 2020-09-08T16:02:52.224423+0000 osd.11 (osd.11) 99 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:53.573 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:02:53 smithi104 bash[22305]: debug 2020-09-08T16:02:53.458+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:53.573 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:53 smithi104 bash[9203]: cluster 2020-09-08T16:02:51.997874+0000 mgr.x (mgr.24118) 1066 : cluster [DBG] pgmap v1575: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 795 op/s 2020-09-08T16:02:53.573 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:53 smithi104 bash[9203]: cluster 2020-09-08T16:02:52.104389+0000 osd.7 (osd.7) 129 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:02:53.574 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:53 smithi104 bash[9203]: cluster 2020-09-08T16:02:52.209516+0000 osd.7 (osd.7) 130 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:02:53.574 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:53 smithi104 bash[9203]: cluster 2020-09-08T16:02:52.224342+0000 osd.11 (osd.11) 98 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:53.574 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:53 smithi104 bash[9203]: cluster 2020-09-08T16:02:52.224423+0000 osd.11 (osd.11) 99 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:53.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:53 smithi166 bash[9446]: cluster 2020-09-08T16:02:51.997874+0000 mgr.x (mgr.24118) 1066 : cluster [DBG] pgmap v1575: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 795 op/s 2020-09-08T16:02:53.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:53 smithi166 bash[9446]: cluster 2020-09-08T16:02:52.104389+0000 osd.7 (osd.7) 129 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:02:53.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:53 smithi166 bash[9446]: cluster 2020-09-08T16:02:52.209516+0000 osd.7 (osd.7) 130 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:02:53.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:53 smithi166 bash[9446]: cluster 2020-09-08T16:02:52.224342+0000 osd.11 (osd.11) 98 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:53.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:53 smithi166 bash[9446]: cluster 2020-09-08T16:02:52.224423+0000 osd.11 (osd.11) 99 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:53.676 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:02:53.783 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:53.906 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:02:53 smithi055 bash[14301]: debug 2020-09-08T16:02:53.676+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:53.913 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T16:02:53.913 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:02:53.991 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:54.097 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:02:54.248 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:02:54.250 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T16:02:54.302 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:02:54.311 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:02:53 smithi104 bash[15273]: debug 2020-09-08T16:02:53.986+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:54.311 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:54 smithi104 bash[9203]: cluster 2020-09-08T16:02:53.234354+0000 mon.a (mon.0) 1939 : cluster [DBG] osdmap e594: 12 total, 12 up, 12 in 2020-09-08T16:02:54.311 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:54 smithi104 bash[9203]: audit 2020-09-08T16:02:53.804636+0000 mon.c (mon.2) 80 : audit [INF] from='client.? v1:172.21.15.104:0/2015052371' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:02:54.312 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:54 smithi104 bash[9203]: audit 2020-09-08T16:02:53.807144+0000 mon.a (mon.0) 1940 : audit [INF] from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:02:54.352 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:02:54.360 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:54 smithi166 bash[9446]: cluster 2020-09-08T16:02:53.234354+0000 mon.a (mon.0) 1939 : cluster [DBG] osdmap e594: 12 total, 12 up, 12 in 2020-09-08T16:02:54.360 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:54 smithi166 bash[9446]: audit 2020-09-08T16:02:53.804636+0000 mon.c (mon.2) 80 : audit [INF] from='client.? v1:172.21.15.104:0/2015052371' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:02:54.361 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:54 smithi166 bash[9446]: audit 2020-09-08T16:02:53.807144+0000 mon.a (mon.0) 1940 : audit [INF] from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:02:54.361 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:02:54 smithi166 bash[14254]: debug 2020-09-08T16:02:54.244+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:54.463 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:02:54.467 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:02:54.570 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:02:54.616 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:02:54 smithi166 bash[14254]: debug 2020-09-08T16:02:54.460+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:54.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:54 smithi055 bash[9296]: cluster 2020-09-08T16:02:53.234354+0000 mon.a (mon.0) 1939 : cluster [DBG] osdmap e594: 12 total, 12 up, 12 in 2020-09-08T16:02:54.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:54 smithi055 bash[9296]: audit 2020-09-08T16:02:53.804636+0000 mon.c (mon.2) 80 : audit [INF] from='client.? v1:172.21.15.104:0/2015052371' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:02:54.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:54 smithi055 bash[9296]: audit 2020-09-08T16:02:53.807144+0000 mon.a (mon.0) 1940 : audit [INF] from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:02:54.686 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:02:54.790 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:54.932 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:55.036 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:55.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:02:54 smithi166 bash[11928]: debug 2020-09-08T16:02:54.684+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:55.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:02:54 smithi055 bash[16646]: debug 2020-09-08T16:02:54.932+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:55.158 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:02:55.262 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:55.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:55 smithi166 bash[9446]: cluster 2020-09-08T16:02:53.832340+0000 osd.10 (osd.10) 75 : cluster [DBG] 7.1 scrub starts 2020-09-08T16:02:55.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:55 smithi166 bash[9446]: cluster 2020-09-08T16:02:53.998268+0000 mgr.x (mgr.24118) 1067 : cluster [DBG] pgmap v1577: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T16:02:55.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:55 smithi166 bash[9446]: cluster 2020-09-08T16:02:54.239875+0000 mon.a (mon.0) 1941 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:02:55.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:55 smithi166 bash[9446]: audit 2020-09-08T16:02:54.247881+0000 mon.a (mon.0) 1942 : audit [INF] from='client.36194 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:02:55.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:55 smithi166 bash[9446]: cluster 2020-09-08T16:02:54.247922+0000 mon.a (mon.0) 1943 : cluster [DBG] osdmap e595: 12 total, 12 up, 12 in 2020-09-08T16:02:55.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:02:55 smithi055 bash[16646]: debug 2020-09-08T16:02:55.160+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:55.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:55 smithi055 bash[9296]: cluster 2020-09-08T16:02:53.832340+0000 osd.10 (osd.10) 75 : cluster [DBG] 7.1 scrub starts 2020-09-08T16:02:55.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:55 smithi055 bash[9296]: cluster 2020-09-08T16:02:53.998268+0000 mgr.x (mgr.24118) 1067 : cluster [DBG] pgmap v1577: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T16:02:55.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:55 smithi055 bash[9296]: cluster 2020-09-08T16:02:54.239875+0000 mon.a (mon.0) 1941 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:02:55.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:55 smithi055 bash[9296]: audit 2020-09-08T16:02:54.247881+0000 mon.a (mon.0) 1942 : audit [INF] from='client.36194 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:02:55.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:55 smithi055 bash[9296]: cluster 2020-09-08T16:02:54.247922+0000 mon.a (mon.0) 1943 : cluster [DBG] osdmap e595: 12 total, 12 up, 12 in 2020-09-08T16:02:55.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:55 smithi104 bash[9203]: cluster 2020-09-08T16:02:53.832340+0000 osd.10 (osd.10) 75 : cluster [DBG] 7.1 scrub starts 2020-09-08T16:02:55.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:55 smithi104 bash[9203]: cluster 2020-09-08T16:02:53.998268+0000 mgr.x (mgr.24118) 1067 : cluster [DBG] pgmap v1577: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T16:02:55.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:55 smithi104 bash[9203]: cluster 2020-09-08T16:02:54.239875+0000 mon.a (mon.0) 1941 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:02:55.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:55 smithi104 bash[9203]: audit 2020-09-08T16:02:54.247881+0000 mon.a (mon.0) 1942 : audit [INF] from='client.36194 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:02:55.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:55 smithi104 bash[9203]: cluster 2020-09-08T16:02:54.247922+0000 mon.a (mon.0) 1943 : cluster [DBG] osdmap e595: 12 total, 12 up, 12 in 2020-09-08T16:02:56.052 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:56.156 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:02:56.427 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:02:56 smithi104 bash[15273]: debug 2020-09-08T16:02:56.054+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:56.803 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:02:56.907 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:57.016 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2020-09-08T16:02:57.065 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:57.066 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:02:56 smithi104 bash[19922]: debug 2020-09-08T16:02:56.806+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:57.066 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:02:57 smithi104 bash[15273]: debug 2020-09-08T16:02:57.066+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:57.172 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:57.192 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:02:57.309 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:57.412 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:57.515 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:57 smithi055 bash[9296]: cluster 2020-09-08T16:02:55.998715+0000 mgr.x (mgr.24118) 1068 : cluster [DBG] pgmap v1579: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 698 op/s 2020-09-08T16:02:57.516 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:02:57 smithi055 bash[11956]: debug 2020-09-08T16:02:57.312+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:57.516 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:02:57.521 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:57 smithi104 bash[9203]: cluster 2020-09-08T16:02:55.998715+0000 mgr.x (mgr.24118) 1068 : cluster [DBG] pgmap v1579: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 698 op/s 2020-09-08T16:02:57.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:57 smithi166 bash[9446]: cluster 2020-09-08T16:02:55.998715+0000 mgr.x (mgr.24118) 1068 : cluster [DBG] pgmap v1579: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 698 op/s 2020-09-08T16:02:57.620 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:57.906 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:02:57 smithi055 bash[11956]: debug 2020-09-08T16:02:57.516+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:58.243 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:02:58.345 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:02:58.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:58 smithi104 bash[9203]: cluster 2020-09-08T16:02:57.165571+0000 osd.4 (osd.4) 63 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:58.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:58 smithi104 bash[9203]: cluster 2020-09-08T16:02:57.165655+0000 osd.4 (osd.4) 64 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:58.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:58 smithi104 bash[9203]: cluster 2020-09-08T16:02:57.254497+0000 mon.a (mon.0) 1944 : cluster [DBG] osdmap e596: 12 total, 12 up, 12 in 2020-09-08T16:02:58.427 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:02:58 smithi104 bash[15273]: debug 2020-09-08T16:02:58.238+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:58.480 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:02:58.585 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:58.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:02:58 smithi166 bash[14254]: debug 2020-09-08T16:02:58.476+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:58.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:58 smithi166 bash[9446]: cluster 2020-09-08T16:02:57.165571+0000 osd.4 (osd.4) 63 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:58.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:58 smithi166 bash[9446]: cluster 2020-09-08T16:02:57.165655+0000 osd.4 (osd.4) 64 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:58.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:58 smithi166 bash[9446]: cluster 2020-09-08T16:02:57.254497+0000 mon.a (mon.0) 1944 : cluster [DBG] osdmap e596: 12 total, 12 up, 12 in 2020-09-08T16:02:58.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:58 smithi055 bash[9296]: cluster 2020-09-08T16:02:57.165571+0000 osd.4 (osd.4) 63 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:02:58.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:58 smithi055 bash[9296]: cluster 2020-09-08T16:02:57.165655+0000 osd.4 (osd.4) 64 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:02:58.663 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:58 smithi055 bash[9296]: cluster 2020-09-08T16:02:57.254497+0000 mon.a (mon.0) 1944 : cluster [DBG] osdmap e596: 12 total, 12 up, 12 in 2020-09-08T16:02:58.800 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T16:02:58.979 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:02:59.028 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:02:59.137 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:02:59.146 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:02:59 smithi104 bash[17588]: debug 2020-09-08T16:02:59.022+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:02:59.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:02:59 smithi104 bash[9203]: cluster 2020-09-08T16:02:57.999124+0000 mgr.x (mgr.24118) 1069 : cluster [DBG] pgmap v1581: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T16:02:59.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:02:59 smithi166 bash[9446]: cluster 2020-09-08T16:02:57.999124+0000 mgr.x (mgr.24118) 1069 : cluster [DBG] pgmap v1581: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T16:02:59.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:02:59 smithi055 bash[9296]: cluster 2020-09-08T16:02:57.999124+0000 mgr.x (mgr.24118) 1069 : cluster [DBG] pgmap v1581: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T16:03:00.674 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:03:00.778 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:00.961 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:01.022 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:03:00 smithi104 bash[22305]: debug 2020-09-08T16:03:00.674+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:01.065 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:01.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:00 smithi055 bash[9296]: cluster 2020-09-08T16:02:59.999563+0000 mgr.x (mgr.24118) 1070 : cluster [DBG] pgmap v1582: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 715 op/s 2020-09-08T16:03:01.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:03:00 smithi055 bash[16646]: debug 2020-09-08T16:03:00.964+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:01.221 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:01.325 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:01.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:00 smithi166 bash[9446]: cluster 2020-09-08T16:02:59.999563+0000 mgr.x (mgr.24118) 1070 : cluster [DBG] pgmap v1582: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 715 op/s 2020-09-08T16:03:01.367 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:03:01 smithi166 bash[16593]: debug 2020-09-08T16:03:01.220+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:01.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:01 smithi104 bash[9203]: cluster 2020-09-08T16:02:59.999563+0000 mgr.x (mgr.24118) 1070 : cluster [DBG] pgmap v1582: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 715 op/s 2020-09-08T16:03:01.436 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:01.542 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:01.656 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:03:01 smithi055 bash[14301]: debug 2020-09-08T16:03:01.436+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:01.660 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:01.764 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:01.872 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:01.976 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:01.986 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:03:01 smithi055 bash[14301]: debug 2020-09-08T16:03:01.664+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:01.988 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:03:01 smithi055 bash[16646]: debug 2020-09-08T16:03:01.872+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:02.088 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:02.195 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:02.315 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:03:02 smithi055 bash[16646]: debug 2020-09-08T16:03:02.088+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:02.315 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:03:02 smithi055 bash[11956]: debug 2020-09-08T16:03:02.312+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:02.316 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:02.424 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:02.537 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:02.643 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:02.653 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:03:02 smithi055 bash[14301]: debug 2020-09-08T16:03:02.540+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:02.752 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:02.856 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:02.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:03:02 smithi055 bash[16646]: debug 2020-09-08T16:03:02.748+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:02.982 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:03.086 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:03.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:03 smithi166 bash[9446]: cluster 2020-09-08T16:03:01.999955+0000 mgr.x (mgr.24118) 1071 : cluster [DBG] pgmap v1583: 17 pgs: 17 active+clean; 5.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 787 op/s 2020-09-08T16:03:03.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:03:02 smithi055 bash[11956]: debug 2020-09-08T16:03:02.984+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:03.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:03 smithi055 bash[9296]: cluster 2020-09-08T16:03:01.999955+0000 mgr.x (mgr.24118) 1071 : cluster [DBG] pgmap v1583: 17 pgs: 17 active+clean; 5.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 787 op/s 2020-09-08T16:03:03.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:03 smithi104 bash[9203]: cluster 2020-09-08T16:03:01.999955+0000 mgr.x (mgr.24118) 1071 : cluster [DBG] pgmap v1583: 17 pgs: 17 active+clean; 5.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 787 op/s 2020-09-08T16:03:03.938 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:04.042 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:04.306 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:03:03 smithi104 bash[19922]: debug 2020-09-08T16:03:03.938+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:04.307 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:04.416 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:04.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:03:04 smithi104 bash[17588]: debug 2020-09-08T16:03:04.306+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:05.184 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:05.287 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:05.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:05 smithi166 bash[9446]: cluster 2020-09-08T16:03:04.000372+0000 mgr.x (mgr.24118) 1072 : cluster [DBG] pgmap v1584: 17 pgs: 17 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 988 op/s 2020-09-08T16:03:05.368 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:03:05 smithi166 bash[16593]: debug 2020-09-08T16:03:05.180+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:05.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:05 smithi055 bash[9296]: cluster 2020-09-08T16:03:04.000372+0000 mgr.x (mgr.24118) 1072 : cluster [DBG] pgmap v1584: 17 pgs: 17 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 988 op/s 2020-09-08T16:03:05.408 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:05.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:05 smithi104 bash[9203]: cluster 2020-09-08T16:03:04.000372+0000 mgr.x (mgr.24118) 1072 : cluster [DBG] pgmap v1584: 17 pgs: 17 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 988 op/s 2020-09-08T16:03:05.516 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:05.644 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:05.750 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:05.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:03:05 smithi166 bash[16593]: debug 2020-09-08T16:03:05.404+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:05.882 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:03:05 smithi055 bash[19003]: debug 2020-09-08T16:03:05.643+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:05.888 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:05.902 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:03:05.930 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:03:05.966 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:03:05.991 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:06.005 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:03:06.081 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:03:06.097 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:06.107 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:03:06.146 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:03:06.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:03:05 smithi055 bash[11956]: debug 2020-09-08T16:03:05.883+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:06.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:06 smithi055 bash[9296]: audit 2020-09-08T16:03:05.872289+0000 mon.a (mon.0) 1945 : audit [DBG] from='client.? v1:172.21.15.104:0/387051816' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:03:06.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:03:06 smithi055 bash[19003]: debug 2020-09-08T16:03:06.099+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:06.197 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:03:06.201 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:03:06.231 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:03:06.261 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:03:06.297 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:03:06.322 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:06 smithi166 bash[9446]: audit 2020-09-08T16:03:05.872289+0000 mon.a (mon.0) 1945 : audit [DBG] from='client.? v1:172.21.15.104:0/387051816' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:03:06.327 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:03:06.332 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:03:06.333 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:03:06.334 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:03:06.334 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:03:06.334 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:03:06.334 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:03:06.335 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:03:06.335 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:03:06.335 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:03:06.335 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:03:06.335 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:03:06.336 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:03:06.336 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:03:06.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:06 smithi104 bash[9203]: audit 2020-09-08T16:03:05.872289+0000 mon.a (mon.0) 1945 : audit [DBG] from='client.? v1:172.21.15.104:0/387051816' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:03:06.434 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:06.567 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:06.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:03:06 smithi166 bash[11928]: debug 2020-09-08T16:03:06.320+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:06.617 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:03:06 smithi166 bash[16593]: debug 2020-09-08T16:03:06.564+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:06.674 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:06.792 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:06.896 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:03:07.011 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:03:06 smithi166 bash[16593]: debug 2020-09-08T16:03:06.788+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:07.012 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:03:07.126 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:07.256 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:07.360 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:07.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:07 smithi166 bash[9446]: cluster 2020-09-08T16:03:06.000834+0000 mgr.x (mgr.24118) 1073 : cluster [DBG] pgmap v1585: 17 pgs: 17 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 963 op/s 2020-09-08T16:03:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:07 smithi166 bash[9446]: cluster 2020-09-08T16:03:06.772264+0000 osd.9 (osd.9) 89 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:07 smithi166 bash[9446]: cluster 2020-09-08T16:03:06.772336+0000 osd.9 (osd.9) 90 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:07.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:03:07 smithi166 bash[18960]: debug 2020-09-08T16:03:07.008+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:07.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:03:07 smithi055 bash[19003]: debug 2020-09-08T16:03:07.255+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:07.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:07 smithi055 bash[9296]: cluster 2020-09-08T16:03:06.000834+0000 mgr.x (mgr.24118) 1073 : cluster [DBG] pgmap v1585: 17 pgs: 17 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 963 op/s 2020-09-08T16:03:07.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:07 smithi055 bash[9296]: cluster 2020-09-08T16:03:06.772264+0000 osd.9 (osd.9) 89 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:07.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:07 smithi055 bash[9296]: cluster 2020-09-08T16:03:06.772336+0000 osd.9 (osd.9) 90 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:07.573 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:07 smithi104 bash[9203]: cluster 2020-09-08T16:03:06.000834+0000 mgr.x (mgr.24118) 1073 : cluster [DBG] pgmap v1585: 17 pgs: 17 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 963 op/s 2020-09-08T16:03:07.573 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:07 smithi104 bash[9203]: cluster 2020-09-08T16:03:06.772264+0000 osd.9 (osd.9) 89 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:07.574 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:07 smithi104 bash[9203]: cluster 2020-09-08T16:03:06.772336+0000 osd.9 (osd.9) 90 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:08.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:08 smithi166 bash[9446]: cluster 2020-09-08T16:03:07.081109+0000 mon.a (mon.0) 1946 : cluster [DBG] osdmap e597: 12 total, 12 up, 12 in 2020-09-08T16:03:08.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:08 smithi055 bash[9296]: cluster 2020-09-08T16:03:07.081109+0000 mon.a (mon.0) 1946 : cluster [DBG] osdmap e597: 12 total, 12 up, 12 in 2020-09-08T16:03:08.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:08 smithi104 bash[9203]: cluster 2020-09-08T16:03:07.081109+0000 mon.a (mon.0) 1946 : cluster [DBG] osdmap e597: 12 total, 12 up, 12 in 2020-09-08T16:03:08.431 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:08.536 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:08.679 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:08.782 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:08.790 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:03:08 smithi104 bash[15273]: debug 2020-09-08T16:03:08.429+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:08.867 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:03:08 smithi166 bash[16593]: debug 2020-09-08T16:03:08.676+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:08.994 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:09.096 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:09.116 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:03:08 smithi104 bash[17588]: debug 2020-09-08T16:03:08.985+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:09.226 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:09.300 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T16:03:09.301 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T16:03:09.301 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.301 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:192899 7.3 7:c945a75f:::benchmark_data_smithi166_48533_object58382:head [delete] snapc 0=[] ondisk+write+known_if_redirected e598)", 2020-09-08T16:03:09.302 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:03:09.289744+0000", 2020-09-08T16:03:09.302 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.016513297, 2020-09-08T16:03:09.302 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.016551038000000001, 2020-09-08T16:03:09.304 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:03:09.304 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:03:09.304 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:03:09.304 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:03:09.305 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:03:09.305 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 192899 2020-09-08T16:03:09.305 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.305 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:03:09.305 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.306 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:03:09.306 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.289744+0000", 2020-09-08T16:03:09.306 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:03:09.306 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.306 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.307 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:03:09.307 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.289744+0000", 2020-09-08T16:03:09.307 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999466 2020-09-08T16:03:09.307 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.307 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.308 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:03:09.308 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.289691+0000", 2020-09-08T16:03:09.308 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011721 2020-09-08T16:03:09.308 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.308 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.309 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:03:09.309 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.289808+0000", 2020-09-08T16:03:09.309 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2067999999999999e-05 2020-09-08T16:03:09.309 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.309 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.310 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:03:09.310 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.289830+0000", 2020-09-08T16:03:09.310 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9790000000000005e-06 2020-09-08T16:03:09.310 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.310 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.310 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:03:09.311 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.289840+0000", 2020-09-08T16:03:09.311 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0044157040000000003 2020-09-08T16:03:09.311 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.311 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.311 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:03:09.312 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.294256+0000", 2020-09-08T16:03:09.312 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000189247 2020-09-08T16:03:09.312 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.312 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.312 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:03:09.313 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.294445+0000", 2020-09-08T16:03:09.313 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000199173 2020-09-08T16:03:09.313 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.320 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.320 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:03:09.321 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.294645+0000", 2020-09-08T16:03:09.321 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0061400309999999998 2020-09-08T16:03:09.321 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.321 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.321 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:03:09.322 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.300785+0000", 2020-09-08T16:03:09.322 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.011040148 2020-09-08T16:03:09.322 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:03:09.322 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:03:09.322 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:03:09.323 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.323 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.323 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:192912 7.2 7:42968855:::benchmark_data_smithi166_48533_object58395:head [delete] snapc 0=[] ondisk+write+known_if_redirected e598)", 2020-09-08T16:03:09.323 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:03:09.302885+0000", 2020-09-08T16:03:09.323 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0033724219999999999, 2020-09-08T16:03:09.324 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003505167, 2020-09-08T16:03:09.325 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:03:09.325 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:03:09.326 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:03:09.326 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:03:09.326 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:03:09.326 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 192912 2020-09-08T16:03:09.327 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.327 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:03:09.327 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.327 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:03:09.327 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.302885+0000", 2020-09-08T16:03:09.327 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:03:09.328 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.328 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.328 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:03:09.328 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.302885+0000", 2020-09-08T16:03:09.329 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999518 2020-09-08T16:03:09.329 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.329 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.329 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:03:09.330 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.302837+0000", 2020-09-08T16:03:09.330 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6748000000000003e-05 2020-09-08T16:03:09.330 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.331 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.331 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:03:09.331 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.302934+0000", 2020-09-08T16:03:09.332 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1562000000000001e-05 2020-09-08T16:03:09.332 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.332 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.333 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:03:09.333 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.302955+0000", 2020-09-08T16:03:09.334 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3570000000000003e-06 2020-09-08T16:03:09.334 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.335 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.336 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:03:09.336 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.302962+0000", 2020-09-08T16:03:09.336 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5843e-05 2020-09-08T16:03:09.337 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.337 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.337 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:03:09.337 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.303028+0000", 2020-09-08T16:03:09.338 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015265499999999999 2020-09-08T16:03:09.338 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.338 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.338 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:03:09.338 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.303180+0000", 2020-09-08T16:03:09.339 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00030947700000000001 2020-09-08T16:03:09.339 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.339 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.339 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 10", 2020-09-08T16:03:09.339 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.303490+0000", 2020-09-08T16:03:09.340 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00060444199999999996 2020-09-08T16:03:09.340 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:03:09.340 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:03:09.340 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:03:09.340 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.340 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.341 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop_reply(client.36113.0:192899 7.3 e598/553)", 2020-09-08T16:03:09.341 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:03:09.305039+0000", 2020-09-08T16:03:09.341 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0012184380000000001, 2020-09-08T16:03:09.341 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0014280510000000001, 2020-09-08T16:03:09.341 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:03:09.342 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "queued for pg", 2020-09-08T16:03:09.342 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:03:09.342 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.342 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:03:09.342 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.305039+0000", 2020-09-08T16:03:09.343 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:03:09.343 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.343 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.343 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:03:09.343 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.305039+0000", 2020-09-08T16:03:09.344 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1616999999999997e-05 2020-09-08T16:03:09.344 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.344 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.344 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:03:09.355 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.305091+0000", 2020-09-08T16:03:09.355 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3529999999999995e-06 2020-09-08T16:03:09.355 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.356 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.356 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:03:09.356 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.305099+0000", 2020-09-08T16:03:09.356 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4148000000000001e-05 2020-09-08T16:03:09.356 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.357 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.357 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:03:09.357 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.305113+0000", 2020-09-08T16:03:09.357 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8499999999999999e-06 2020-09-08T16:03:09.357 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:03:09.359 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:03:09.359 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:03:09.359 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:09.305119+0000", 2020-09-08T16:03:09.360 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9968000000000004e-05 2020-09-08T16:03:09.360 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:03:09.360 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:03:09.360 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:03:09.360 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:03:09.361 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T16:03:09.361 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 3 2020-09-08T16:03:09.361 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T16:03:09.363 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:09.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:09 smithi166 bash[9446]: cluster 2020-09-08T16:03:08.001341+0000 mgr.x (mgr.24118) 1074 : cluster [DBG] pgmap v1587: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 971 op/s 2020-09-08T16:03:09.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:09 smithi166 bash[9446]: cluster 2020-09-08T16:03:08.783669+0000 osd.2 (osd.2) 94 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:09.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:09 smithi166 bash[9446]: cluster 2020-09-08T16:03:08.783750+0000 osd.2 (osd.2) 95 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:09.371 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:09 smithi104 bash[9203]: cluster 2020-09-08T16:03:08.001341+0000 mgr.x (mgr.24118) 1074 : cluster [DBG] pgmap v1587: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 971 op/s 2020-09-08T16:03:09.371 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:09 smithi104 bash[9203]: cluster 2020-09-08T16:03:08.783669+0000 osd.2 (osd.2) 94 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:09.371 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:09 smithi104 bash[9203]: cluster 2020-09-08T16:03:08.783750+0000 osd.2 (osd.2) 95 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:09.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:03:09 smithi055 bash[19003]: debug 2020-09-08T16:03:09.227+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:09.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:09 smithi055 bash[9296]: cluster 2020-09-08T16:03:08.001341+0000 mgr.x (mgr.24118) 1074 : cluster [DBG] pgmap v1587: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 971 op/s 2020-09-08T16:03:09.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:09 smithi055 bash[9296]: cluster 2020-09-08T16:03:08.783669+0000 osd.2 (osd.2) 94 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:09.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:09 smithi055 bash[9296]: cluster 2020-09-08T16:03:08.783750+0000 osd.2 (osd.2) 95 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:09.588 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:09.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:03:09 smithi104 bash[17588]: debug 2020-09-08T16:03:09.589+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:09.692 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:09.823 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:09.929 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:10.093 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:03:09 smithi166 bash[16593]: debug 2020-09-08T16:03:09.820+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:10.108 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T16:03:10.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:10 smithi166 bash[9446]: cluster 2020-09-08T16:03:09.098427+0000 mon.a (mon.0) 1947 : cluster [DBG] osdmap e598: 12 total, 12 up, 12 in 2020-09-08T16:03:10.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:10 smithi166 bash[9446]: audit 2020-09-08T16:03:09.825218+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? v1:172.21.15.104:0/1881459555' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:03:10.444 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:10 smithi104 bash[9203]: cluster 2020-09-08T16:03:09.098427+0000 mon.a (mon.0) 1947 : cluster [DBG] osdmap e598: 12 total, 12 up, 12 in 2020-09-08T16:03:10.444 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:10 smithi104 bash[9203]: audit 2020-09-08T16:03:09.825218+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? v1:172.21.15.104:0/1881459555' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:03:10.534 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:10 smithi055 bash[9296]: cluster 2020-09-08T16:03:09.098427+0000 mon.a (mon.0) 1947 : cluster [DBG] osdmap e598: 12 total, 12 up, 12 in 2020-09-08T16:03:10.534 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:10 smithi055 bash[9296]: audit 2020-09-08T16:03:09.825218+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? v1:172.21.15.104:0/1881459555' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:03:10.959 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:03:11.127 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:03:11.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:11 smithi166 bash[9446]: cluster 2020-09-08T16:03:10.002076+0000 mgr.x (mgr.24118) 1075 : cluster [DBG] pgmap v1589: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T16:03:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:11 smithi166 bash[9446]: cluster 2020-09-08T16:03:10.097002+0000 mon.a (mon.0) 1949 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:03:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:11 smithi166 bash[9446]: audit 2020-09-08T16:03:10.108335+0000 mon.a (mon.0) 1950 : audit [INF] from='client.? v1:172.21.15.104:0/1881459555' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:03:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:11 smithi166 bash[9446]: cluster 2020-09-08T16:03:10.108415+0000 mon.a (mon.0) 1951 : cluster [DBG] osdmap e599: 12 total, 12 up, 12 in 2020-09-08T16:03:11.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:11 smithi055 bash[9296]: cluster 2020-09-08T16:03:10.002076+0000 mgr.x (mgr.24118) 1075 : cluster [DBG] pgmap v1589: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T16:03:11.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:11 smithi055 bash[9296]: cluster 2020-09-08T16:03:10.097002+0000 mon.a (mon.0) 1949 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:03:11.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:11 smithi055 bash[9296]: audit 2020-09-08T16:03:10.108335+0000 mon.a (mon.0) 1950 : audit [INF] from='client.? v1:172.21.15.104:0/1881459555' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:03:11.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:11 smithi055 bash[9296]: cluster 2020-09-08T16:03:10.108415+0000 mon.a (mon.0) 1951 : cluster [DBG] osdmap e599: 12 total, 12 up, 12 in 2020-09-08T16:03:11.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:11 smithi104 bash[9203]: cluster 2020-09-08T16:03:10.002076+0000 mgr.x (mgr.24118) 1075 : cluster [DBG] pgmap v1589: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T16:03:11.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:11 smithi104 bash[9203]: cluster 2020-09-08T16:03:10.097002+0000 mon.a (mon.0) 1949 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:03:11.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:11 smithi104 bash[9203]: audit 2020-09-08T16:03:10.108335+0000 mon.a (mon.0) 1950 : audit [INF] from='client.? v1:172.21.15.104:0/1881459555' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:03:11.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:11 smithi104 bash[9203]: cluster 2020-09-08T16:03:10.108415+0000 mon.a (mon.0) 1951 : cluster [DBG] osdmap e599: 12 total, 12 up, 12 in 2020-09-08T16:03:11.727 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2020-09-08T16:03:11.888 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:03:12.403 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:12.506 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:12.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:03:12 smithi055 bash[19003]: debug 2020-09-08T16:03:12.403+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:12.729 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:12.835 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:12.843 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:03:12 smithi104 bash[19922]: debug 2020-09-08T16:03:12.729+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:13.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:13 smithi166 bash[9446]: cluster 2020-09-08T16:03:12.002674+0000 mgr.x (mgr.24118) 1076 : cluster [DBG] pgmap v1591: 17 pgs: 17 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T16:03:13.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:13 smithi055 bash[9296]: cluster 2020-09-08T16:03:12.002674+0000 mgr.x (mgr.24118) 1076 : cluster [DBG] pgmap v1591: 17 pgs: 17 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T16:03:13.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:13 smithi104 bash[9203]: cluster 2020-09-08T16:03:12.002674+0000 mgr.x (mgr.24118) 1076 : cluster [DBG] pgmap v1591: 17 pgs: 17 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T16:03:13.582 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:13.686 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:13.801 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:13.870 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:03:13 smithi104 bash[17588]: debug 2020-09-08T16:03:13.581+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:13.904 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:03:13.986 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T16:03:14.075 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:03:14.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:03:13 smithi055 bash[11956]: debug 2020-09-08T16:03:13.803+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:14.163 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:03:14.179 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:14.189 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:03:14 smithi104 bash[22305]: debug 2020-09-08T16:03:14.073+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:14.352 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:14.455 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:14.617 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:03:14 smithi166 bash[16593]: debug 2020-09-08T16:03:14.348+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:15.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:15 smithi166 bash[9446]: cluster 2020-09-08T16:03:14.003131+0000 mgr.x (mgr.24118) 1077 : cluster [DBG] pgmap v1592: 17 pgs: 17 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.11k op/s 2020-09-08T16:03:15.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:15 smithi055 bash[9296]: cluster 2020-09-08T16:03:14.003131+0000 mgr.x (mgr.24118) 1077 : cluster [DBG] pgmap v1592: 17 pgs: 17 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.11k op/s 2020-09-08T16:03:15.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:15 smithi104 bash[9203]: cluster 2020-09-08T16:03:14.003131+0000 mgr.x (mgr.24118) 1077 : cluster [DBG] pgmap v1592: 17 pgs: 17 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.11k op/s 2020-09-08T16:03:15.537 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:15.642 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:15.788 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:15.892 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:15.901 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:03:15 smithi055 bash[11956]: debug 2020-09-08T16:03:15.783+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:15.939 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:03:15 smithi104 bash[19922]: debug 2020-09-08T16:03:15.537+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:15.996 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:16.103 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:16.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:03:16 smithi055 bash[16646]: debug 2020-09-08T16:03:15.995+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:16.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:16 smithi055 bash[9296]: cluster 2020-09-08T16:03:14.821060+0000 osd.7 (osd.7) 131 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:16.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:16 smithi055 bash[9296]: cluster 2020-09-08T16:03:14.821144+0000 osd.7 (osd.7) 132 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:16.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:16 smithi055 bash[9296]: cluster 2020-09-08T16:03:15.107888+0000 mon.a (mon.0) 1952 : cluster [DBG] osdmap e600: 12 total, 12 up, 12 in 2020-09-08T16:03:16.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:16 smithi166 bash[9446]: cluster 2020-09-08T16:03:14.821060+0000 osd.7 (osd.7) 131 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:16.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:16 smithi166 bash[9446]: cluster 2020-09-08T16:03:14.821144+0000 osd.7 (osd.7) 132 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:16.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:16 smithi166 bash[9446]: cluster 2020-09-08T16:03:15.107888+0000 mon.a (mon.0) 1952 : cluster [DBG] osdmap e600: 12 total, 12 up, 12 in 2020-09-08T16:03:16.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:16 smithi104 bash[9203]: cluster 2020-09-08T16:03:14.821060+0000 osd.7 (osd.7) 131 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:16.431 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:16 smithi104 bash[9203]: cluster 2020-09-08T16:03:14.821144+0000 osd.7 (osd.7) 132 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:16.431 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:16 smithi104 bash[9203]: cluster 2020-09-08T16:03:15.107888+0000 mon.a (mon.0) 1952 : cluster [DBG] osdmap e600: 12 total, 12 up, 12 in 2020-09-08T16:03:16.698 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:16.802 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:16.912 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:17.016 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:17.123 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:03:16 smithi104 bash[15273]: debug 2020-09-08T16:03:16.693+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:17.125 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:03:16 smithi055 bash[16646]: debug 2020-09-08T16:03:16.911+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:17.127 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:17 smithi055 bash[9296]: cluster 2020-09-08T16:03:16.003535+0000 mgr.x (mgr.24118) 1078 : cluster [DBG] pgmap v1594: 17 pgs: 17 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 618 op/s 2020-09-08T16:03:17.127 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:17 smithi055 bash[9296]: cluster 2020-09-08T16:03:16.175884+0000 osd.0 (osd.0) 102 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:17.128 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:17 smithi055 bash[9296]: cluster 2020-09-08T16:03:16.175969+0000 osd.0 (osd.0) 103 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:17.137 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:17.244 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:17.404 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:03:17 smithi055 bash[14301]: debug 2020-09-08T16:03:17.135+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:17.405 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:17.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:17 smithi104 bash[9203]: cluster 2020-09-08T16:03:16.003535+0000 mgr.x (mgr.24118) 1078 : cluster [DBG] pgmap v1594: 17 pgs: 17 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 618 op/s 2020-09-08T16:03:17.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:17 smithi104 bash[9203]: cluster 2020-09-08T16:03:16.175884+0000 osd.0 (osd.0) 102 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:17.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:17 smithi104 bash[9203]: cluster 2020-09-08T16:03:16.175969+0000 osd.0 (osd.0) 103 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:17.519 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:17.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:17 smithi166 bash[9446]: cluster 2020-09-08T16:03:16.003535+0000 mgr.x (mgr.24118) 1078 : cluster [DBG] pgmap v1594: 17 pgs: 17 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 618 op/s 2020-09-08T16:03:17.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:17 smithi166 bash[9446]: cluster 2020-09-08T16:03:16.175884+0000 osd.0 (osd.0) 102 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:17.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:17 smithi166 bash[9446]: cluster 2020-09-08T16:03:16.175969+0000 osd.0 (osd.0) 103 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:17.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:03:17 smithi055 bash[11956]: debug 2020-09-08T16:03:17.403+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:17.748 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:17.856 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:17.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:03:17 smithi104 bash[17588]: debug 2020-09-08T16:03:17.745+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:18.020 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:18.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:03:18 smithi166 bash[16593]: debug 2020-09-08T16:03:18.016+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:18.124 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:03:18.244 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:03:18.348 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:18.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:18 smithi055 bash[9296]: cluster 2020-09-08T16:03:17.122606+0000 mon.a (mon.0) 1953 : cluster [DBG] osdmap e601: 12 total, 12 up, 12 in 2020-09-08T16:03:18.421 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:18 smithi104 bash[9203]: cluster 2020-09-08T16:03:17.122606+0000 mon.a (mon.0) 1953 : cluster [DBG] osdmap e601: 12 total, 12 up, 12 in 2020-09-08T16:03:18.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:18 smithi166 bash[9446]: cluster 2020-09-08T16:03:17.122606+0000 mon.a (mon.0) 1953 : cluster [DBG] osdmap e601: 12 total, 12 up, 12 in 2020-09-08T16:03:18.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:03:18 smithi166 bash[14254]: debug 2020-09-08T16:03:18.240+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:18.673 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:03:18 smithi104 bash[17588]: debug 2020-09-08T16:03:18.669+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:18.678 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:18.779 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:18.909 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:19.013 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:19.140 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:03:18 smithi055 bash[19003]: debug 2020-09-08T16:03:18.911+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:19.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:19 smithi055 bash[9296]: cluster 2020-09-08T16:03:18.003999+0000 mgr.x (mgr.24118) 1079 : cluster [DBG] pgmap v1596: 17 pgs: 17 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T16:03:19.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:19 smithi104 bash[9203]: cluster 2020-09-08T16:03:18.003999+0000 mgr.x (mgr.24118) 1079 : cluster [DBG] pgmap v1596: 17 pgs: 17 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T16:03:19.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:19 smithi166 bash[9446]: cluster 2020-09-08T16:03:18.003999+0000 mgr.x (mgr.24118) 1079 : cluster [DBG] pgmap v1596: 17 pgs: 17 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T16:03:19.935 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:20.042 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:03:20.370 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:03:19 smithi104 bash[19922]: debug 2020-09-08T16:03:19.933+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:21.147 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:03:21.252 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:21.259 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:03:21 smithi104 bash[22305]: debug 2020-09-08T16:03:21.145+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:21.260 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:20 smithi104 bash[9203]: cluster 2020-09-08T16:03:20.004415+0000 mgr.x (mgr.24118) 1080 : cluster [DBG] pgmap v1597: 17 pgs: 17 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T16:03:21.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:20 smithi166 bash[9446]: cluster 2020-09-08T16:03:20.004415+0000 mgr.x (mgr.24118) 1080 : cluster [DBG] pgmap v1597: 17 pgs: 17 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T16:03:21.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:20 smithi055 bash[9296]: cluster 2020-09-08T16:03:20.004415+0000 mgr.x (mgr.24118) 1080 : cluster [DBG] pgmap v1597: 17 pgs: 17 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T16:03:21.589 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:03:21 smithi104 bash[19922]: debug 2020-09-08T16:03:21.585+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:21.589 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:21.691 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:03:21.722 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:03:21.758 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:03:21.790 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:03:21.821 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:03:21.832 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:03:21.858 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:03:21.905 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:03:21.936 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:21.941 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:03:21.958 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:03:21 smithi166 bash[11928]: debug 2020-09-08T16:03:21.828+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:21.972 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:03:22.025 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:03:22.037 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T16:03:22.037 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:03:22.066 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:03:22.097 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:03:22.133 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:03:22.134 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:03:22.135 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:03:22.136 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:03:22.136 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:03:22.136 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:03:22.136 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:03:22.137 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:03:22.137 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:03:22.137 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:03:22.137 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:03:22.137 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:03:22.138 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:03:22.255 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:21 smithi104 bash[9203]: audit 2020-09-08T16:03:21.669367+0000 mon.a (mon.0) 1954 : audit [DBG] from='client.? v1:172.21.15.104:0/4262201182' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:03:22.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:21 smithi166 bash[9446]: audit 2020-09-08T16:03:21.669367+0000 mon.a (mon.0) 1954 : audit [DBG] from='client.? v1:172.21.15.104:0/4262201182' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:03:22.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:21 smithi055 bash[9296]: audit 2020-09-08T16:03:21.669367+0000 mon.a (mon.0) 1954 : audit [DBG] from='client.? v1:172.21.15.104:0/4262201182' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:03:22.767 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:22.875 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:23.013 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:23.117 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:23.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:22 smithi055 bash[9296]: cluster 2020-09-08T16:03:22.004843+0000 mgr.x (mgr.24118) 1081 : cluster [DBG] pgmap v1598: 17 pgs: 17 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 822 op/s 2020-09-08T16:03:23.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:03:23 smithi055 bash[11956]: debug 2020-09-08T16:03:23.015+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:23.187 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:03:22 smithi104 bash[15273]: debug 2020-09-08T16:03:22.761+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:23.261 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:22 smithi166 bash[9446]: cluster 2020-09-08T16:03:22.004843+0000 mgr.x (mgr.24118) 1081 : cluster [DBG] pgmap v1598: 17 pgs: 17 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 822 op/s 2020-09-08T16:03:23.261 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:03:23 smithi166 bash[16593]: debug 2020-09-08T16:03:23.260+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:23.266 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:23.371 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:23.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:23 smithi104 bash[9203]: cluster 2020-09-08T16:03:22.004843+0000 mgr.x (mgr.24118) 1081 : cluster [DBG] pgmap v1598: 17 pgs: 17 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 822 op/s 2020-09-08T16:03:24.010 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:24.114 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:24.339 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:03:24 smithi104 bash[15273]: debug 2020-09-08T16:03:24.009+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:24.339 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:03:24 smithi104 bash[15273]: debug 2020-09-08T16:03:24.337+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:24.339 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:24.443 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:24.618 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2020-09-08T16:03:24.659 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:24.768 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:24.785 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:03:24.892 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:24.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:03:24 smithi104 bash[15273]: debug 2020-09-08T16:03:24.653+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:24.998 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:03:25.054 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:03:24 smithi055 bash[19003]: debug 2020-09-08T16:03:24.891+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:25.128 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:03:25.231 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:25.340 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:25.340 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:25 smithi055 bash[9296]: cluster 2020-09-08T16:03:24.005257+0000 mgr.x (mgr.24118) 1082 : cluster [DBG] pgmap v1599: 17 pgs: 17 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T16:03:25.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:25 smithi166 bash[9446]: cluster 2020-09-08T16:03:24.005257+0000 mgr.x (mgr.24118) 1082 : cluster [DBG] pgmap v1599: 17 pgs: 17 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T16:03:25.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:03:25 smithi166 bash[18960]: debug 2020-09-08T16:03:25.124+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:25.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:25 smithi104 bash[9203]: cluster 2020-09-08T16:03:24.005257+0000 mgr.x (mgr.24118) 1082 : cluster [DBG] pgmap v1599: 17 pgs: 17 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T16:03:25.448 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:25.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:03:25 smithi055 bash[11956]: debug 2020-09-08T16:03:25.339+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:25.913 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:25.917 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:03:25 smithi104 bash[17588]: debug 2020-09-08T16:03:25.913+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:26.017 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:03:26.085 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T16:03:26.176 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:03:26.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:26 smithi104 bash[9203]: audit 2020-09-08T16:03:25.344103+0000 mon.a (mon.0) 1955 : audit [INF] from='client.? v1:172.21.15.104:0/717091886' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:03:26.280 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:26.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:26 smithi166 bash[9446]: audit 2020-09-08T16:03:25.344103+0000 mon.a (mon.0) 1955 : audit [INF] from='client.? v1:172.21.15.104:0/717091886' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:03:26.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:03:26 smithi166 bash[18960]: debug 2020-09-08T16:03:26.172+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:26.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:26 smithi055 bash[9296]: audit 2020-09-08T16:03:25.344103+0000 mon.a (mon.0) 1955 : audit [INF] from='client.? v1:172.21.15.104:0/717091886' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:03:26.437 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:03:26.604 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:03:26.948 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:27.058 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:27.212 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:27.317 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:27.325 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:27 smithi104 bash[9203]: cluster 2020-09-08T16:03:26.005660+0000 mgr.x (mgr.24118) 1083 : cluster [DBG] pgmap v1600: 17 pgs: 17 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 999 op/s 2020-09-08T16:03:27.326 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:27 smithi104 bash[9203]: cluster 2020-09-08T16:03:26.060926+0000 mon.a (mon.0) 1956 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:03:27.326 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:27 smithi104 bash[9203]: audit 2020-09-08T16:03:26.074156+0000 mon.a (mon.0) 1957 : audit [INF] from='client.? v1:172.21.15.104:0/717091886' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:03:27.326 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:27 smithi104 bash[9203]: cluster 2020-09-08T16:03:26.074203+0000 mon.a (mon.0) 1958 : cluster [DBG] osdmap e602: 12 total, 12 up, 12 in 2020-09-08T16:03:27.326 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:03:26 smithi104 bash[19922]: debug 2020-09-08T16:03:26.949+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:27.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:27 smithi166 bash[9446]: cluster 2020-09-08T16:03:26.005660+0000 mgr.x (mgr.24118) 1083 : cluster [DBG] pgmap v1600: 17 pgs: 17 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 999 op/s 2020-09-08T16:03:27.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:27 smithi166 bash[9446]: cluster 2020-09-08T16:03:26.060926+0000 mon.a (mon.0) 1956 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:03:27.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:27 smithi166 bash[9446]: audit 2020-09-08T16:03:26.074156+0000 mon.a (mon.0) 1957 : audit [INF] from='client.? v1:172.21.15.104:0/717091886' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:03:27.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:27 smithi166 bash[9446]: cluster 2020-09-08T16:03:26.074203+0000 mon.a (mon.0) 1958 : cluster [DBG] osdmap e602: 12 total, 12 up, 12 in 2020-09-08T16:03:27.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:27 smithi055 bash[9296]: cluster 2020-09-08T16:03:26.005660+0000 mgr.x (mgr.24118) 1083 : cluster [DBG] pgmap v1600: 17 pgs: 17 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 999 op/s 2020-09-08T16:03:27.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:27 smithi055 bash[9296]: cluster 2020-09-08T16:03:26.060926+0000 mon.a (mon.0) 1956 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:03:27.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:27 smithi055 bash[9296]: audit 2020-09-08T16:03:26.074156+0000 mon.a (mon.0) 1957 : audit [INF] from='client.? v1:172.21.15.104:0/717091886' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:03:27.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:27 smithi055 bash[9296]: cluster 2020-09-08T16:03:26.074203+0000 mon.a (mon.0) 1958 : cluster [DBG] osdmap e602: 12 total, 12 up, 12 in 2020-09-08T16:03:27.408 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:03:27 smithi055 bash[14301]: debug 2020-09-08T16:03:27.207+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:27.930 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:28.035 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:28.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:28 smithi166 bash[9446]: cluster 2020-09-08T16:03:26.959527+0000 osd.5 (osd.5) 97 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:28.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:28 smithi166 bash[9446]: cluster 2020-09-08T16:03:26.959640+0000 osd.5 (osd.5) 98 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:28.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:28 smithi166 bash[9446]: cluster 2020-09-08T16:03:27.075251+0000 mon.a (mon.0) 1959 : cluster [DBG] osdmap e603: 12 total, 12 up, 12 in 2020-09-08T16:03:28.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:28 smithi166 bash[9446]: cluster 2020-09-08T16:03:27.465623+0000 osd.10 (osd.10) 76 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:28.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:28 smithi166 bash[9446]: cluster 2020-09-08T16:03:27.465688+0000 osd.10 (osd.10) 77 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:28.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:28 smithi166 bash[9446]: cluster 2020-09-08T16:03:27.714023+0000 osd.10 (osd.10) 78 : cluster [DBG] 7.1 scrub starts 2020-09-08T16:03:28.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:28 smithi055 bash[9296]: cluster 2020-09-08T16:03:26.959527+0000 osd.5 (osd.5) 97 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:28.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:28 smithi055 bash[9296]: cluster 2020-09-08T16:03:26.959640+0000 osd.5 (osd.5) 98 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:28.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:28 smithi055 bash[9296]: cluster 2020-09-08T16:03:27.075251+0000 mon.a (mon.0) 1959 : cluster [DBG] osdmap e603: 12 total, 12 up, 12 in 2020-09-08T16:03:28.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:28 smithi055 bash[9296]: cluster 2020-09-08T16:03:27.465623+0000 osd.10 (osd.10) 76 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:28.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:28 smithi055 bash[9296]: cluster 2020-09-08T16:03:27.465688+0000 osd.10 (osd.10) 77 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:28.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:28 smithi055 bash[9296]: cluster 2020-09-08T16:03:27.714023+0000 osd.10 (osd.10) 78 : cluster [DBG] 7.1 scrub starts 2020-09-08T16:03:28.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:03:27 smithi104 bash[19922]: debug 2020-09-08T16:03:27.929+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:28.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:28 smithi104 bash[9203]: cluster 2020-09-08T16:03:26.959527+0000 osd.5 (osd.5) 97 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:28.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:28 smithi104 bash[9203]: cluster 2020-09-08T16:03:26.959640+0000 osd.5 (osd.5) 98 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:28.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:28 smithi104 bash[9203]: cluster 2020-09-08T16:03:27.075251+0000 mon.a (mon.0) 1959 : cluster [DBG] osdmap e603: 12 total, 12 up, 12 in 2020-09-08T16:03:28.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:28 smithi104 bash[9203]: cluster 2020-09-08T16:03:27.465623+0000 osd.10 (osd.10) 76 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:28.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:28 smithi104 bash[9203]: cluster 2020-09-08T16:03:27.465688+0000 osd.10 (osd.10) 77 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:28.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:28 smithi104 bash[9203]: cluster 2020-09-08T16:03:27.714023+0000 osd.10 (osd.10) 78 : cluster [DBG] 7.1 scrub starts 2020-09-08T16:03:28.909 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:28.980 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:03:28 smithi104 bash[19922]: debug 2020-09-08T16:03:28.905+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:29.014 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:29.148 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:29.263 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:29.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:29 smithi166 bash[9446]: cluster 2020-09-08T16:03:28.006393+0000 mgr.x (mgr.24118) 1084 : cluster [DBG] pgmap v1603: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.17k op/s 2020-09-08T16:03:29.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:29 smithi166 bash[9446]: cluster 2020-09-08T16:03:28.084262+0000 mon.a (mon.0) 1960 : cluster [DBG] osdmap e604: 12 total, 12 up, 12 in 2020-09-08T16:03:29.367 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:03:29 smithi166 bash[16593]: debug 2020-09-08T16:03:29.144+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:29.376 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:29 smithi104 bash[9203]: cluster 2020-09-08T16:03:28.006393+0000 mgr.x (mgr.24118) 1084 : cluster [DBG] pgmap v1603: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.17k op/s 2020-09-08T16:03:29.377 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:29 smithi104 bash[9203]: cluster 2020-09-08T16:03:28.084262+0000 mon.a (mon.0) 1960 : cluster [DBG] osdmap e604: 12 total, 12 up, 12 in 2020-09-08T16:03:29.404 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:29 smithi055 bash[9296]: cluster 2020-09-08T16:03:28.006393+0000 mgr.x (mgr.24118) 1084 : cluster [DBG] pgmap v1603: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.17k op/s 2020-09-08T16:03:29.404 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:29 smithi055 bash[9296]: cluster 2020-09-08T16:03:28.084262+0000 mon.a (mon.0) 1960 : cluster [DBG] osdmap e604: 12 total, 12 up, 12 in 2020-09-08T16:03:29.405 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:03:29 smithi055 bash[19003]: debug 2020-09-08T16:03:29.403+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:29.408 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:29.513 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:29.640 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:29.745 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:29.859 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:29.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:03:29 smithi166 bash[16593]: debug 2020-09-08T16:03:29.636+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:29.963 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:30.087 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:03:29 smithi055 bash[14301]: debug 2020-09-08T16:03:29.855+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:30.339 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:30 smithi104 bash[9203]: cluster 2020-09-08T16:03:28.780736+0000 osd.10 (osd.10) 79 : cluster [DBG] 7.1 scrub ok 2020-09-08T16:03:30.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:30 smithi166 bash[9446]: cluster 2020-09-08T16:03:28.780736+0000 osd.10 (osd.10) 79 : cluster [DBG] 7.1 scrub ok 2020-09-08T16:03:30.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:30 smithi055 bash[9296]: cluster 2020-09-08T16:03:28.780736+0000 osd.10 (osd.10) 79 : cluster [DBG] 7.1 scrub ok 2020-09-08T16:03:30.682 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T16:03:30.864 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:03:31.095 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:31 smithi104 bash[9203]: cluster 2020-09-08T16:03:30.006970+0000 mgr.x (mgr.24118) 1085 : cluster [DBG] pgmap v1605: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 849 op/s 2020-09-08T16:03:31.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:31 smithi166 bash[9446]: cluster 2020-09-08T16:03:30.006970+0000 mgr.x (mgr.24118) 1085 : cluster [DBG] pgmap v1605: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 849 op/s 2020-09-08T16:03:31.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:31 smithi055 bash[9296]: cluster 2020-09-08T16:03:30.006970+0000 mgr.x (mgr.24118) 1085 : cluster [DBG] pgmap v1605: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 849 op/s 2020-09-08T16:03:31.458 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:31.560 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:31.697 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:31.809 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:31.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:03:31 smithi166 bash[16593]: debug 2020-09-08T16:03:31.452+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:31.906 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:03:31 smithi055 bash[19003]: debug 2020-09-08T16:03:31.699+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:31.917 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:32.024 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:32.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:03:31 smithi055 bash[16646]: debug 2020-09-08T16:03:31.915+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:32.507 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:32.612 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:03:32.740 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:03:32.843 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:03:32.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:03:32 smithi104 bash[17588]: debug 2020-09-08T16:03:32.505+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:32.955 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:03:32 smithi166 bash[11928]: debug 2020-09-08T16:03:32.736+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:32.960 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:03:33.065 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:33.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:33 smithi166 bash[9446]: cluster 2020-09-08T16:03:31.739172+0000 osd.3 (osd.3) 103 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:33.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:33 smithi166 bash[9446]: cluster 2020-09-08T16:03:31.739255+0000 osd.3 (osd.3) 104 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:33.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:33 smithi166 bash[9446]: cluster 2020-09-08T16:03:31.757571+0000 osd.3 (osd.3) 105 : cluster [DBG] 7.2 scrub starts 2020-09-08T16:03:33.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:33 smithi166 bash[9446]: cluster 2020-09-08T16:03:32.007400+0000 mgr.x (mgr.24118) 1086 : cluster [DBG] pgmap v1606: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T16:03:33.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:33 smithi166 bash[9446]: cluster 2020-09-08T16:03:32.104743+0000 mon.a (mon.0) 1961 : cluster [DBG] osdmap e605: 12 total, 12 up, 12 in 2020-09-08T16:03:33.369 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:03:32 smithi166 bash[14254]: debug 2020-09-08T16:03:32.952+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:33.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:33 smithi055 bash[9296]: cluster 2020-09-08T16:03:31.739172+0000 osd.3 (osd.3) 103 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:33.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:33 smithi055 bash[9296]: cluster 2020-09-08T16:03:31.739255+0000 osd.3 (osd.3) 104 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:33.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:33 smithi055 bash[9296]: cluster 2020-09-08T16:03:31.757571+0000 osd.3 (osd.3) 105 : cluster [DBG] 7.2 scrub starts 2020-09-08T16:03:33.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:33 smithi055 bash[9296]: cluster 2020-09-08T16:03:32.007400+0000 mgr.x (mgr.24118) 1086 : cluster [DBG] pgmap v1606: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T16:03:33.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:33 smithi055 bash[9296]: cluster 2020-09-08T16:03:32.104743+0000 mon.a (mon.0) 1961 : cluster [DBG] osdmap e605: 12 total, 12 up, 12 in 2020-09-08T16:03:33.475 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:33.582 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:33.639 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:03:33 smithi104 bash[19922]: debug 2020-09-08T16:03:33.477+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:33.639 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:33 smithi104 bash[9203]: cluster 2020-09-08T16:03:31.739172+0000 osd.3 (osd.3) 103 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:33.639 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:33 smithi104 bash[9203]: cluster 2020-09-08T16:03:31.739255+0000 osd.3 (osd.3) 104 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:33.639 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:33 smithi104 bash[9203]: cluster 2020-09-08T16:03:31.757571+0000 osd.3 (osd.3) 105 : cluster [DBG] 7.2 scrub starts 2020-09-08T16:03:33.640 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:33 smithi104 bash[9203]: cluster 2020-09-08T16:03:32.007400+0000 mgr.x (mgr.24118) 1086 : cluster [DBG] pgmap v1606: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.09k op/s 2020-09-08T16:03:33.640 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:33 smithi104 bash[9203]: cluster 2020-09-08T16:03:32.104743+0000 mon.a (mon.0) 1961 : cluster [DBG] osdmap e605: 12 total, 12 up, 12 in 2020-09-08T16:03:33.739 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:33.843 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:33.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:03:33 smithi055 bash[16646]: debug 2020-09-08T16:03:33.739+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:33.948 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:34.059 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:34.112 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:03:33 smithi166 bash[16593]: debug 2020-09-08T16:03:33.944+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:34.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:34 smithi166 bash[9446]: cluster 2020-09-08T16:03:32.876947+0000 osd.3 (osd.3) 106 : cluster [DBG] 7.2 scrub ok 2020-09-08T16:03:34.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:34 smithi055 bash[9296]: cluster 2020-09-08T16:03:32.876947+0000 osd.3 (osd.3) 106 : cluster [DBG] 7.2 scrub ok 2020-09-08T16:03:34.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:34 smithi104 bash[9203]: cluster 2020-09-08T16:03:32.876947+0000 osd.3 (osd.3) 106 : cluster [DBG] 7.2 scrub ok 2020-09-08T16:03:34.459 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:34.562 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:34.683 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:34.757 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:03:34 smithi104 bash[19922]: debug 2020-09-08T16:03:34.457+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:34.784 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:34.906 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:03:34 smithi055 bash[11956]: debug 2020-09-08T16:03:34.679+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:35.234 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:35.340 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:03:35.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:35 smithi166 bash[9446]: cluster 2020-09-08T16:03:34.007869+0000 mgr.x (mgr.24118) 1087 : cluster [DBG] pgmap v1608: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.20k op/s 2020-09-08T16:03:35.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:35 smithi055 bash[9296]: cluster 2020-09-08T16:03:34.007869+0000 mgr.x (mgr.24118) 1087 : cluster [DBG] pgmap v1608: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.20k op/s 2020-09-08T16:03:35.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:35 smithi104 bash[9203]: cluster 2020-09-08T16:03:34.007869+0000 mgr.x (mgr.24118) 1087 : cluster [DBG] pgmap v1608: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.20k op/s 2020-09-08T16:03:35.427 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:03:35 smithi104 bash[17588]: debug 2020-09-08T16:03:35.233+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:35.477 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T16:03:35.478 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:03:35.639 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:03:35.744 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:35.844 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:03:35.877 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:03:35.926 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:03:35.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:03:35 smithi104 bash[22305]: debug 2020-09-08T16:03:35.641+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:35.972 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:03:36.011 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:03:36.047 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:03:36.082 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:03:36.112 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:03:36.151 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:03:36.190 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:03:36.222 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:03:36.258 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:03:36.259 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:03:36.259 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:03:36.259 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:03:36.259 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:03:36.260 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:03:36.260 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:03:36.260 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:03:36.260 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:03:36.260 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:03:36.261 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:03:36.261 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:03:36.261 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:03:36.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:36 smithi166 bash[9446]: audit 2020-09-08T16:03:35.794099+0000 mon.a (mon.0) 1962 : audit [DBG] from='client.? v1:172.21.15.104:0/706301829' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:03:36.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:36 smithi055 bash[9296]: audit 2020-09-08T16:03:35.794099+0000 mon.a (mon.0) 1962 : audit [DBG] from='client.? v1:172.21.15.104:0/706301829' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:03:36.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:36 smithi104 bash[9203]: audit 2020-09-08T16:03:35.794099+0000 mon.a (mon.0) 1962 : audit [DBG] from='client.? v1:172.21.15.104:0/706301829' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:03:37.124 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:37 smithi104 bash[9203]: cluster 2020-09-08T16:03:36.008331+0000 mgr.x (mgr.24118) 1088 : cluster [DBG] pgmap v1609: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 684 op/s 2020-09-08T16:03:37.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:37 smithi055 bash[9296]: cluster 2020-09-08T16:03:36.008331+0000 mgr.x (mgr.24118) 1088 : cluster [DBG] pgmap v1609: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 684 op/s 2020-09-08T16:03:37.626 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:37 smithi166 bash[9446]: cluster 2020-09-08T16:03:36.008331+0000 mgr.x (mgr.24118) 1088 : cluster [DBG] pgmap v1609: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 684 op/s 2020-09-08T16:03:38.380 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:38.383 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:38 smithi166 bash[9446]: cluster 2020-09-08T16:03:37.230088+0000 osd.6 (osd.6) 99 : cluster [DBG] 7.d scrub starts 2020-09-08T16:03:38.383 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:03:38 smithi166 bash[16593]: debug 2020-09-08T16:03:38.380+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:38.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:38 smithi055 bash[9296]: cluster 2020-09-08T16:03:37.230088+0000 osd.6 (osd.6) 99 : cluster [DBG] 7.d scrub starts 2020-09-08T16:03:38.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:38 smithi104 bash[9203]: cluster 2020-09-08T16:03:37.230088+0000 osd.6 (osd.6) 99 : cluster [DBG] 7.d scrub starts 2020-09-08T16:03:38.484 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:03:38.606 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:03:38.621 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2020-09-08T16:03:38.708 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:38.867 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:03:38 smithi166 bash[14254]: debug 2020-09-08T16:03:38.600+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:39.068 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:03:39.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:39 smithi055 bash[9296]: cluster 2020-09-08T16:03:37.583883+0000 osd.8 (osd.8) 81 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:39.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:39 smithi055 bash[9296]: cluster 2020-09-08T16:03:37.583958+0000 osd.8 (osd.8) 82 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:39.413 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:39 smithi055 bash[9296]: cluster 2020-09-08T16:03:38.008769+0000 mgr.x (mgr.24118) 1089 : cluster [DBG] pgmap v1610: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 988 op/s 2020-09-08T16:03:39.413 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:39 smithi055 bash[9296]: cluster 2020-09-08T16:03:38.076140+0000 osd.6 (osd.6) 100 : cluster [DBG] 7.d scrub ok 2020-09-08T16:03:39.413 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:39 smithi055 bash[9296]: cluster 2020-09-08T16:03:38.138009+0000 mon.a (mon.0) 1963 : cluster [DBG] osdmap e606: 12 total, 12 up, 12 in 2020-09-08T16:03:39.413 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:39 smithi055 bash[9296]: cluster 2020-09-08T16:03:38.174966+0000 osd.6 (osd.6) 101 : cluster [DBG] 7.c scrub starts 2020-09-08T16:03:39.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:39 smithi104 bash[9203]: cluster 2020-09-08T16:03:37.583883+0000 osd.8 (osd.8) 81 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:39.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:39 smithi104 bash[9203]: cluster 2020-09-08T16:03:37.583958+0000 osd.8 (osd.8) 82 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:39.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:39 smithi104 bash[9203]: cluster 2020-09-08T16:03:38.008769+0000 mgr.x (mgr.24118) 1089 : cluster [DBG] pgmap v1610: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 988 op/s 2020-09-08T16:03:39.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:39 smithi104 bash[9203]: cluster 2020-09-08T16:03:38.076140+0000 osd.6 (osd.6) 100 : cluster [DBG] 7.d scrub ok 2020-09-08T16:03:39.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:39 smithi104 bash[9203]: cluster 2020-09-08T16:03:38.138009+0000 mon.a (mon.0) 1963 : cluster [DBG] osdmap e606: 12 total, 12 up, 12 in 2020-09-08T16:03:39.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:39 smithi104 bash[9203]: cluster 2020-09-08T16:03:38.174966+0000 osd.6 (osd.6) 101 : cluster [DBG] 7.c scrub starts 2020-09-08T16:03:39.470 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:39.578 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:39.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:39 smithi166 bash[9446]: cluster 2020-09-08T16:03:37.583883+0000 osd.8 (osd.8) 81 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:39.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:39 smithi166 bash[9446]: cluster 2020-09-08T16:03:37.583958+0000 osd.8 (osd.8) 82 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:39.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:39 smithi166 bash[9446]: cluster 2020-09-08T16:03:38.008769+0000 mgr.x (mgr.24118) 1089 : cluster [DBG] pgmap v1610: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 988 op/s 2020-09-08T16:03:39.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:39 smithi166 bash[9446]: cluster 2020-09-08T16:03:38.076140+0000 osd.6 (osd.6) 100 : cluster [DBG] 7.d scrub ok 2020-09-08T16:03:39.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:39 smithi166 bash[9446]: cluster 2020-09-08T16:03:38.138009+0000 mon.a (mon.0) 1963 : cluster [DBG] osdmap e606: 12 total, 12 up, 12 in 2020-09-08T16:03:39.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:39 smithi166 bash[9446]: cluster 2020-09-08T16:03:38.174966+0000 osd.6 (osd.6) 101 : cluster [DBG] 7.c scrub starts 2020-09-08T16:03:39.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:03:39 smithi104 bash[15273]: debug 2020-09-08T16:03:39.465+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:40.266 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:40.371 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:03:40.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:40 smithi055 bash[9296]: cluster 2020-09-08T16:03:39.171489+0000 osd.6 (osd.6) 102 : cluster [DBG] 7.c scrub ok 2020-09-08T16:03:40.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:40 smithi055 bash[9296]: cluster 2020-09-08T16:03:39.213557+0000 osd.6 (osd.6) 103 : cluster [DBG] 7.7 scrub starts 2020-09-08T16:03:40.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:40 smithi104 bash[9203]: cluster 2020-09-08T16:03:39.171489+0000 osd.6 (osd.6) 102 : cluster [DBG] 7.c scrub ok 2020-09-08T16:03:40.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:40 smithi104 bash[9203]: cluster 2020-09-08T16:03:39.213557+0000 osd.6 (osd.6) 103 : cluster [DBG] 7.7 scrub starts 2020-09-08T16:03:40.427 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:03:40 smithi104 bash[15273]: debug 2020-09-08T16:03:40.265+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:40.504 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:40 smithi166 bash[9446]: cluster 2020-09-08T16:03:39.171489+0000 osd.6 (osd.6) 102 : cluster [DBG] 7.c scrub ok 2020-09-08T16:03:40.504 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:40 smithi166 bash[9446]: cluster 2020-09-08T16:03:39.213557+0000 osd.6 (osd.6) 103 : cluster [DBG] 7.7 scrub starts 2020-09-08T16:03:40.505 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:03:40.610 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:40.752 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:40.796 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:03:40.857 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:40.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:03:40 smithi166 bash[18960]: debug 2020-09-08T16:03:40.508+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:40.927 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:03:40 smithi104 bash[15273]: debug 2020-09-08T16:03:40.749+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:40.961 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:40.964 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:03:41.067 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:41.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:03:40 smithi166 bash[16593]: debug 2020-09-08T16:03:40.960+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:41.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:41 smithi104 bash[9203]: cluster 2020-09-08T16:03:39.958704+0000 osd.6 (osd.6) 104 : cluster [DBG] 7.7 scrub ok 2020-09-08T16:03:41.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:41 smithi104 bash[9203]: cluster 2020-09-08T16:03:40.009156+0000 mgr.x (mgr.24118) 1090 : cluster [DBG] pgmap v1612: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T16:03:41.429 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:41 smithi055 bash[9296]: cluster 2020-09-08T16:03:39.958704+0000 osd.6 (osd.6) 104 : cluster [DBG] 7.7 scrub ok 2020-09-08T16:03:41.429 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:41 smithi055 bash[9296]: cluster 2020-09-08T16:03:40.009156+0000 mgr.x (mgr.24118) 1090 : cluster [DBG] pgmap v1612: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T16:03:41.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:41 smithi166 bash[9446]: cluster 2020-09-08T16:03:39.958704+0000 osd.6 (osd.6) 104 : cluster [DBG] 7.7 scrub ok 2020-09-08T16:03:41.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:41 smithi166 bash[9446]: cluster 2020-09-08T16:03:40.009156+0000 mgr.x (mgr.24118) 1090 : cluster [DBG] pgmap v1612: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T16:03:41.974 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:42.076 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:42.158 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T16:03:42.237 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:42.282 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:03:41 smithi104 bash[19922]: debug 2020-09-08T16:03:41.969+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:42.283 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:42 smithi104 bash[9203]: audit 2020-09-08T16:03:41.298852+0000 mon.c (mon.2) 81 : audit [INF] from='client.? v1:172.21.15.104:0/2975784045' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:03:42.283 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:42 smithi104 bash[9203]: audit 2020-09-08T16:03:41.301186+0000 mon.a (mon.0) 1964 : audit [INF] from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:03:42.347 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:42.354 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:03:42 smithi055 bash[11956]: debug 2020-09-08T16:03:42.239+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:42.355 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:42 smithi055 bash[9296]: audit 2020-09-08T16:03:41.298852+0000 mon.c (mon.2) 81 : audit [INF] from='client.? v1:172.21.15.104:0/2975784045' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:03:42.355 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:42 smithi055 bash[9296]: audit 2020-09-08T16:03:41.301186+0000 mon.a (mon.0) 1964 : audit [INF] from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:03:42.552 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:42.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:42 smithi166 bash[9446]: audit 2020-09-08T16:03:41.298852+0000 mon.c (mon.2) 81 : audit [INF] from='client.? v1:172.21.15.104:0/2975784045' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:03:42.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:42 smithi166 bash[9446]: audit 2020-09-08T16:03:41.301186+0000 mon.a (mon.0) 1964 : audit [INF] from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:03:42.656 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:42.658 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:03:42 smithi055 bash[14301]: debug 2020-09-08T16:03:42.551+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:42.786 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:42.888 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:43.147 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:03:42 smithi055 bash[11956]: debug 2020-09-08T16:03:42.783+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:43.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:43 smithi055 bash[9296]: cluster 2020-09-08T16:03:42.009841+0000 mgr.x (mgr.24118) 1091 : cluster [DBG] pgmap v1613: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.9 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 994 op/s 2020-09-08T16:03:43.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:43 smithi055 bash[9296]: cluster 2020-09-08T16:03:42.141159+0000 mon.a (mon.0) 1965 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:03:43.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:43 smithi055 bash[9296]: cluster 2020-09-08T16:03:42.141192+0000 mon.a (mon.0) 1966 : cluster [INF] Cluster is now healthy 2020-09-08T16:03:43.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:43 smithi055 bash[9296]: audit 2020-09-08T16:03:42.153336+0000 mon.a (mon.0) 1967 : audit [INF] from='client.36230 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:03:43.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:43 smithi055 bash[9296]: cluster 2020-09-08T16:03:42.153368+0000 mon.a (mon.0) 1968 : cluster [DBG] osdmap e607: 12 total, 12 up, 12 in 2020-09-08T16:03:43.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:43 smithi104 bash[9203]: cluster 2020-09-08T16:03:42.009841+0000 mgr.x (mgr.24118) 1091 : cluster [DBG] pgmap v1613: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.9 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 994 op/s 2020-09-08T16:03:43.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:43 smithi104 bash[9203]: cluster 2020-09-08T16:03:42.141159+0000 mon.a (mon.0) 1965 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:03:43.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:43 smithi104 bash[9203]: cluster 2020-09-08T16:03:42.141192+0000 mon.a (mon.0) 1966 : cluster [INF] Cluster is now healthy 2020-09-08T16:03:43.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:43 smithi104 bash[9203]: audit 2020-09-08T16:03:42.153336+0000 mon.a (mon.0) 1967 : audit [INF] from='client.36230 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:03:43.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:43 smithi104 bash[9203]: cluster 2020-09-08T16:03:42.153368+0000 mon.a (mon.0) 1968 : cluster [DBG] osdmap e607: 12 total, 12 up, 12 in 2020-09-08T16:03:43.462 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:43.564 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:43.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:43 smithi166 bash[9446]: cluster 2020-09-08T16:03:42.009841+0000 mgr.x (mgr.24118) 1091 : cluster [DBG] pgmap v1613: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.9 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 994 op/s 2020-09-08T16:03:43.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:43 smithi166 bash[9446]: cluster 2020-09-08T16:03:42.141159+0000 mon.a (mon.0) 1965 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:03:43.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:43 smithi166 bash[9446]: cluster 2020-09-08T16:03:42.141192+0000 mon.a (mon.0) 1966 : cluster [INF] Cluster is now healthy 2020-09-08T16:03:43.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:43 smithi166 bash[9446]: audit 2020-09-08T16:03:42.153336+0000 mon.a (mon.0) 1967 : audit [INF] from='client.36230 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:03:43.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:43 smithi166 bash[9446]: cluster 2020-09-08T16:03:42.153368+0000 mon.a (mon.0) 1968 : cluster [DBG] osdmap e607: 12 total, 12 up, 12 in 2020-09-08T16:03:43.894 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:43.906 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:03:43 smithi055 bash[19003]: debug 2020-09-08T16:03:43.459+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:43.998 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:44.133 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:44.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:44 smithi104 bash[9203]: cluster 2020-09-08T16:03:43.452750+0000 osd.0 (osd.0) 104 : cluster [DBG] 7.9 deep-scrub starts 2020-09-08T16:03:44.177 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:03:43 smithi104 bash[17588]: debug 2020-09-08T16:03:43.889+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:44.177 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:03:44 smithi104 bash[15273]: debug 2020-09-08T16:03:44.133+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:44.244 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:03:44.393 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:03:44.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:44 smithi055 bash[9296]: cluster 2020-09-08T16:03:43.452750+0000 osd.0 (osd.0) 104 : cluster [DBG] 7.9 deep-scrub starts 2020-09-08T16:03:44.499 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:44.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:44 smithi166 bash[9446]: cluster 2020-09-08T16:03:43.452750+0000 osd.0 (osd.0) 104 : cluster [DBG] 7.9 deep-scrub starts 2020-09-08T16:03:44.618 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:03:44 smithi166 bash[11928]: debug 2020-09-08T16:03:44.392+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:44.781 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:44.884 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:45.045 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:03:44 smithi055 bash[11956]: debug 2020-09-08T16:03:44.779+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:45.045 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:03:45 smithi055 bash[14301]: debug 2020-09-08T16:03:45.043+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:45.045 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:45.149 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:03:45.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:45 smithi055 bash[9296]: cluster 2020-09-08T16:03:44.010385+0000 mgr.x (mgr.24118) 1092 : cluster [DBG] pgmap v1615: 17 pgs: 17 active+clean; 2.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.25k op/s 2020-09-08T16:03:45.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:45 smithi055 bash[9296]: cluster 2020-09-08T16:03:44.880545+0000 osd.3 (osd.3) 107 : cluster [DBG] 7.3 scrub starts 2020-09-08T16:03:45.411 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:45 smithi104 bash[9203]: cluster 2020-09-08T16:03:44.010385+0000 mgr.x (mgr.24118) 1092 : cluster [DBG] pgmap v1615: 17 pgs: 17 active+clean; 2.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.25k op/s 2020-09-08T16:03:45.411 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:45 smithi104 bash[9203]: cluster 2020-09-08T16:03:44.880545+0000 osd.3 (osd.3) 107 : cluster [DBG] 7.3 scrub starts 2020-09-08T16:03:45.427 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:03:45 smithi104 bash[22305]: debug 2020-09-08T16:03:45.409+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:45.427 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:03:45.474 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:03:45.475 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T16:03:45.475 INFO:teuthology.orchestra.run.smithi055.stdout: "num_ops": 0 2020-09-08T16:03:45.475 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:03:45.531 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:45.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:45 smithi166 bash[9446]: cluster 2020-09-08T16:03:44.010385+0000 mgr.x (mgr.24118) 1092 : cluster [DBG] pgmap v1615: 17 pgs: 17 active+clean; 2.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.25k op/s 2020-09-08T16:03:45.620 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:45 smithi166 bash[9446]: cluster 2020-09-08T16:03:44.880545+0000 osd.3 (osd.3) 107 : cluster [DBG] 7.3 scrub starts 2020-09-08T16:03:45.998 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T16:03:46.174 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:03:46.272 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:46 smithi104 bash[9203]: cluster 2020-09-08T16:03:45.485478+0000 osd.3 (osd.3) 108 : cluster [DBG] 7.3 scrub ok 2020-09-08T16:03:46.272 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:46.376 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:03:46.389 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2020-09-08T16:03:46.522 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:46 smithi166 bash[9446]: cluster 2020-09-08T16:03:45.485478+0000 osd.3 (osd.3) 108 : cluster [DBG] 7.3 scrub ok 2020-09-08T16:03:46.522 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:03:46.557 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:03:46.625 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:03:46.662 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:46 smithi055 bash[9296]: cluster 2020-09-08T16:03:45.485478+0000 osd.3 (osd.3) 108 : cluster [DBG] 7.3 scrub ok 2020-09-08T16:03:46.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:03:46 smithi104 bash[15273]: debug 2020-09-08T16:03:46.273+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:46.736 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:03:46.847 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:46.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:03:46 smithi166 bash[11928]: debug 2020-09-08T16:03:46.524+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:46.867 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:03:46 smithi166 bash[14254]: debug 2020-09-08T16:03:46.736+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:47.209 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:03:47.313 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:03:47.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:47 smithi104 bash[9203]: cluster 2020-09-08T16:03:46.010873+0000 mgr.x (mgr.24118) 1093 : cluster [DBG] pgmap v1616: 17 pgs: 17 active+clean; 2.7 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 708 op/s 2020-09-08T16:03:47.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:47 smithi104 bash[9203]: cluster 2020-09-08T16:03:46.174889+0000 osd.1 (osd.1) 120 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:47.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:47 smithi104 bash[9203]: cluster 2020-09-08T16:03:46.174963+0000 osd.1 (osd.1) 121 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:47.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:47 smithi104 bash[9203]: cluster 2020-09-08T16:03:46.330762+0000 osd.1 (osd.1) 122 : cluster [DBG] 7.b scrub starts 2020-09-08T16:03:47.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:47 smithi104 bash[9203]: cluster 2020-09-08T16:03:46.989377+0000 osd.1 (osd.1) 123 : cluster [DBG] 7.b scrub ok 2020-09-08T16:03:47.434 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:47 smithi166 bash[9446]: cluster 2020-09-08T16:03:46.010873+0000 mgr.x (mgr.24118) 1093 : cluster [DBG] pgmap v1616: 17 pgs: 17 active+clean; 2.7 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 708 op/s 2020-09-08T16:03:47.434 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:47 smithi166 bash[9446]: cluster 2020-09-08T16:03:46.174889+0000 osd.1 (osd.1) 120 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:47.434 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:47 smithi166 bash[9446]: cluster 2020-09-08T16:03:46.174963+0000 osd.1 (osd.1) 121 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:47.435 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:47 smithi166 bash[9446]: cluster 2020-09-08T16:03:46.330762+0000 osd.1 (osd.1) 122 : cluster [DBG] 7.b scrub starts 2020-09-08T16:03:47.435 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:47 smithi166 bash[9446]: cluster 2020-09-08T16:03:46.989377+0000 osd.1 (osd.1) 123 : cluster [DBG] 7.b scrub ok 2020-09-08T16:03:47.435 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:03:47 smithi166 bash[18960]: debug 2020-09-08T16:03:47.436+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:47.436 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:03:47.539 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:47.660 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:03:47 smithi055 bash[11956]: debug 2020-09-08T16:03:47.211+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:47.661 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:47 smithi055 bash[9296]: cluster 2020-09-08T16:03:46.010873+0000 mgr.x (mgr.24118) 1093 : cluster [DBG] pgmap v1616: 17 pgs: 17 active+clean; 2.7 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 708 op/s 2020-09-08T16:03:47.661 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:47 smithi055 bash[9296]: cluster 2020-09-08T16:03:46.174889+0000 osd.1 (osd.1) 120 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:47.661 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:47 smithi055 bash[9296]: cluster 2020-09-08T16:03:46.174963+0000 osd.1 (osd.1) 121 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:47.662 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:47 smithi055 bash[9296]: cluster 2020-09-08T16:03:46.330762+0000 osd.1 (osd.1) 122 : cluster [DBG] 7.b scrub starts 2020-09-08T16:03:47.662 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:47 smithi055 bash[9296]: cluster 2020-09-08T16:03:46.989377+0000 osd.1 (osd.1) 123 : cluster [DBG] 7.b scrub ok 2020-09-08T16:03:47.975 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:48.078 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:03:48.206 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:03:48.230 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:48 smithi104 bash[9203]: cluster 2020-09-08T16:03:47.171330+0000 mon.a (mon.0) 1969 : cluster [DBG] osdmap e608: 12 total, 12 up, 12 in 2020-09-08T16:03:48.231 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:03:47 smithi104 bash[15273]: debug 2020-09-08T16:03:47.973+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:48.310 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:48.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:48 smithi166 bash[9446]: cluster 2020-09-08T16:03:47.171330+0000 mon.a (mon.0) 1969 : cluster [DBG] osdmap e608: 12 total, 12 up, 12 in 2020-09-08T16:03:48.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:03:48 smithi166 bash[14254]: debug 2020-09-08T16:03:48.196+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:48.393 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:48 smithi055 bash[9296]: cluster 2020-09-08T16:03:47.171330+0000 mon.a (mon.0) 1969 : cluster [DBG] osdmap e608: 12 total, 12 up, 12 in 2020-09-08T16:03:48.444 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:48.549 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:48.656 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:03:48 smithi055 bash[14301]: debug 2020-09-08T16:03:48.443+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:48.665 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:48.769 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:03:48.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:03:48 smithi166 bash[16593]: debug 2020-09-08T16:03:48.668+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:49.147 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:03:49.169 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:03:49.176 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:03:49.220 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:03:49.233 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:03:49.262 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:03:49.275 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:49.290 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:03:49.319 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:03:49.348 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:03:49.376 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:03:49.407 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:03:49.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:03:49 smithi104 bash[22305]: debug 2020-09-08T16:03:49.169+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:49.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:49 smithi104 bash[9203]: cluster 2020-09-08T16:03:48.011338+0000 mgr.x (mgr.24118) 1094 : cluster [DBG] pgmap v1618: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T16:03:49.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:49 smithi104 bash[9203]: audit 2020-09-08T16:03:49.110977+0000 mon.a (mon.0) 1970 : audit [DBG] from='client.? v1:172.21.15.104:0/1557380199' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:03:49.439 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:03:49.467 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:03:49.480 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:03:49.480 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:03:49.480 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:03:49.481 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:03:49.481 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:03:49.481 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:03:49.481 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:03:49.481 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:03:49.482 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:03:49.482 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:03:49.482 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:03:49.482 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:03:49.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:49 smithi166 bash[9446]: cluster 2020-09-08T16:03:48.011338+0000 mgr.x (mgr.24118) 1094 : cluster [DBG] pgmap v1618: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T16:03:49.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:49 smithi166 bash[9446]: audit 2020-09-08T16:03:49.110977+0000 mon.a (mon.0) 1970 : audit [DBG] from='client.? v1:172.21.15.104:0/1557380199' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:03:49.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:49 smithi055 bash[9296]: cluster 2020-09-08T16:03:48.011338+0000 mgr.x (mgr.24118) 1094 : cluster [DBG] pgmap v1618: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T16:03:49.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:49 smithi055 bash[9296]: audit 2020-09-08T16:03:49.110977+0000 mon.a (mon.0) 1970 : audit [DBG] from='client.? v1:172.21.15.104:0/1557380199' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:03:49.700 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:49.804 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:50.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:03:49 smithi055 bash[19003]: debug 2020-09-08T16:03:49.699+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:50.311 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:50.415 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:50.616 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:03:50 smithi055 bash[14301]: debug 2020-09-08T16:03:50.311+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:50.616 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:50 smithi055 bash[9296]: cluster 2020-09-08T16:03:48.808143+0000 osd.9 (osd.9) 91 : cluster [DBG] 7.a scrub starts 2020-09-08T16:03:50.617 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:50 smithi055 bash[9296]: cluster 2020-09-08T16:03:48.958385+0000 osd.6 (osd.6) 105 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:50.617 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:50 smithi055 bash[9296]: cluster 2020-09-08T16:03:48.958476+0000 osd.6 (osd.6) 106 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:50.617 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:50 smithi055 bash[9296]: cluster 2020-09-08T16:03:49.200726+0000 mon.a (mon.0) 1971 : cluster [DBG] osdmap e609: 12 total, 12 up, 12 in 2020-09-08T16:03:50.617 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:50 smithi055 bash[9296]: cluster 2020-09-08T16:03:49.301145+0000 osd.9 (osd.9) 92 : cluster [DBG] 7.a scrub ok 2020-09-08T16:03:50.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:50 smithi166 bash[9446]: cluster 2020-09-08T16:03:48.808143+0000 osd.9 (osd.9) 91 : cluster [DBG] 7.a scrub starts 2020-09-08T16:03:50.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:50 smithi166 bash[9446]: cluster 2020-09-08T16:03:48.958385+0000 osd.6 (osd.6) 105 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:50.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:50 smithi166 bash[9446]: cluster 2020-09-08T16:03:48.958476+0000 osd.6 (osd.6) 106 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:50.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:50 smithi166 bash[9446]: cluster 2020-09-08T16:03:49.200726+0000 mon.a (mon.0) 1971 : cluster [DBG] osdmap e609: 12 total, 12 up, 12 in 2020-09-08T16:03:50.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:50 smithi166 bash[9446]: cluster 2020-09-08T16:03:49.301145+0000 osd.9 (osd.9) 92 : cluster [DBG] 7.a scrub ok 2020-09-08T16:03:50.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:50 smithi104 bash[9203]: cluster 2020-09-08T16:03:48.808143+0000 osd.9 (osd.9) 91 : cluster [DBG] 7.a scrub starts 2020-09-08T16:03:50.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:50 smithi104 bash[9203]: cluster 2020-09-08T16:03:48.958385+0000 osd.6 (osd.6) 105 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:50.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:50 smithi104 bash[9203]: cluster 2020-09-08T16:03:48.958476+0000 osd.6 (osd.6) 106 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:50.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:50 smithi104 bash[9203]: cluster 2020-09-08T16:03:49.200726+0000 mon.a (mon.0) 1971 : cluster [DBG] osdmap e609: 12 total, 12 up, 12 in 2020-09-08T16:03:50.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:50 smithi104 bash[9203]: cluster 2020-09-08T16:03:49.301145+0000 osd.9 (osd.9) 92 : cluster [DBG] 7.a scrub ok 2020-09-08T16:03:50.917 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T16:03:50.917 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:03:50.936 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:51.041 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:03:51.186 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:03:51.195 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:03:50 smithi104 bash[19922]: debug 2020-09-08T16:03:50.937+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:51.195 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:51 smithi104 bash[9203]: cluster 2020-09-08T16:03:50.011862+0000 mgr.x (mgr.24118) 1095 : cluster [DBG] pgmap v1620: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 879 op/s 2020-09-08T16:03:51.290 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:51.302 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:51 smithi055 bash[9296]: cluster 2020-09-08T16:03:50.011862+0000 mgr.x (mgr.24118) 1095 : cluster [DBG] pgmap v1620: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 879 op/s 2020-09-08T16:03:51.373 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:51 smithi166 bash[9446]: cluster 2020-09-08T16:03:50.011862+0000 mgr.x (mgr.24118) 1095 : cluster [DBG] pgmap v1620: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 879 op/s 2020-09-08T16:03:51.373 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:03:51 smithi166 bash[14254]: debug 2020-09-08T16:03:51.187+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:51.769 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:51.871 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:51.887 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2020-09-08T16:03:51.986 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:03:52.070 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:03:52.091 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:52.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:03:51 smithi055 bash[16646]: debug 2020-09-08T16:03:51.762+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:52.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:03:51 smithi055 bash[19003]: debug 2020-09-08T16:03:51.982+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:52.235 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:52.339 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:52.366 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:03:52 smithi166 bash[16593]: debug 2020-09-08T16:03:52.235+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:52.736 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:52.841 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:03:53.132 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:03:53.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:03:52 smithi055 bash[16646]: debug 2020-09-08T16:03:52.734+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:53.215 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T16:03:53.240 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:03:53.276 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:03:53.345 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:53.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:53 smithi104 bash[9203]: cluster 2020-09-08T16:03:52.012450+0000 mgr.x (mgr.24118) 1096 : cluster [DBG] pgmap v1621: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 540 op/s 2020-09-08T16:03:53.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:53 smithi104 bash[9203]: cluster 2020-09-08T16:03:52.295141+0000 osd.5 (osd.5) 99 : cluster [DBG] 7.0 scrub starts 2020-09-08T16:03:53.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:53 smithi104 bash[9203]: cluster 2020-09-08T16:03:52.482016+0000 osd.11 (osd.11) 100 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:53.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:53 smithi104 bash[9203]: cluster 2020-09-08T16:03:52.482088+0000 osd.11 (osd.11) 101 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:53.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:53 smithi104 bash[9203]: cluster 2020-09-08T16:03:52.718663+0000 osd.5 (osd.5) 100 : cluster [DBG] 7.0 scrub ok 2020-09-08T16:03:53.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:53 smithi104 bash[9203]: audit 2020-09-08T16:03:53.083010+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? v1:172.21.15.104:0/748306435' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:03:53.535 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:53 smithi055 bash[9296]: cluster 2020-09-08T16:03:52.012450+0000 mgr.x (mgr.24118) 1096 : cluster [DBG] pgmap v1621: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 540 op/s 2020-09-08T16:03:53.535 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:53 smithi055 bash[9296]: cluster 2020-09-08T16:03:52.295141+0000 osd.5 (osd.5) 99 : cluster [DBG] 7.0 scrub starts 2020-09-08T16:03:53.536 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:53 smithi055 bash[9296]: cluster 2020-09-08T16:03:52.482016+0000 osd.11 (osd.11) 100 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:53.536 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:53 smithi055 bash[9296]: cluster 2020-09-08T16:03:52.482088+0000 osd.11 (osd.11) 101 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:53.536 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:53 smithi055 bash[9296]: cluster 2020-09-08T16:03:52.718663+0000 osd.5 (osd.5) 100 : cluster [DBG] 7.0 scrub ok 2020-09-08T16:03:53.537 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:53 smithi055 bash[9296]: audit 2020-09-08T16:03:53.083010+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? v1:172.21.15.104:0/748306435' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:03:53.537 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:03:53.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:03:53 smithi166 bash[11928]: debug 2020-09-08T16:03:53.239+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:53.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:53 smithi166 bash[9446]: cluster 2020-09-08T16:03:52.012450+0000 mgr.x (mgr.24118) 1096 : cluster [DBG] pgmap v1621: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 540 op/s 2020-09-08T16:03:53.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:53 smithi166 bash[9446]: cluster 2020-09-08T16:03:52.295141+0000 osd.5 (osd.5) 99 : cluster [DBG] 7.0 scrub starts 2020-09-08T16:03:53.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:53 smithi166 bash[9446]: cluster 2020-09-08T16:03:52.482016+0000 osd.11 (osd.11) 100 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:53.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:53 smithi166 bash[9446]: cluster 2020-09-08T16:03:52.482088+0000 osd.11 (osd.11) 101 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:53.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:53 smithi166 bash[9446]: cluster 2020-09-08T16:03:52.718663+0000 osd.5 (osd.5) 100 : cluster [DBG] 7.0 scrub ok 2020-09-08T16:03:53.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:53 smithi166 bash[9446]: audit 2020-09-08T16:03:53.083010+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? v1:172.21.15.104:0/748306435' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:03:53.641 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:53.860 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:03:53 smithi055 bash[14301]: debug 2020-09-08T16:03:53.534+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:54.054 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:54.159 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:54.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:54 smithi055 bash[9296]: cluster 2020-09-08T16:03:53.202701+0000 mon.a (mon.0) 1973 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:03:54.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:54 smithi055 bash[9296]: audit 2020-09-08T16:03:53.215438+0000 mon.a (mon.0) 1974 : audit [INF] from='client.? v1:172.21.15.104:0/748306435' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:03:54.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:54 smithi055 bash[9296]: cluster 2020-09-08T16:03:53.215493+0000 mon.a (mon.0) 1975 : cluster [DBG] osdmap e610: 12 total, 12 up, 12 in 2020-09-08T16:03:54.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:54 smithi104 bash[9203]: cluster 2020-09-08T16:03:53.202701+0000 mon.a (mon.0) 1973 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:03:54.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:54 smithi104 bash[9203]: audit 2020-09-08T16:03:53.215438+0000 mon.a (mon.0) 1974 : audit [INF] from='client.? v1:172.21.15.104:0/748306435' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:03:54.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:54 smithi104 bash[9203]: cluster 2020-09-08T16:03:53.215493+0000 mon.a (mon.0) 1975 : cluster [DBG] osdmap e610: 12 total, 12 up, 12 in 2020-09-08T16:03:54.427 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:03:54 smithi104 bash[15273]: debug 2020-09-08T16:03:54.053+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:54.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:54 smithi166 bash[9446]: cluster 2020-09-08T16:03:53.202701+0000 mon.a (mon.0) 1973 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:03:54.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:54 smithi166 bash[9446]: audit 2020-09-08T16:03:53.215438+0000 mon.a (mon.0) 1974 : audit [INF] from='client.? v1:172.21.15.104:0/748306435' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:03:54.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:54 smithi166 bash[9446]: cluster 2020-09-08T16:03:53.215493+0000 mon.a (mon.0) 1975 : cluster [DBG] osdmap e610: 12 total, 12 up, 12 in 2020-09-08T16:03:54.898 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:55.003 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:55.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:03:54 smithi104 bash[17588]: debug 2020-09-08T16:03:54.897+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:55.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:55 smithi166 bash[9446]: cluster 2020-09-08T16:03:54.012899+0000 mgr.x (mgr.24118) 1097 : cluster [DBG] pgmap v1623: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 746 op/s 2020-09-08T16:03:55.624 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:55.627 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:55 smithi104 bash[9203]: cluster 2020-09-08T16:03:54.012899+0000 mgr.x (mgr.24118) 1097 : cluster [DBG] pgmap v1623: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 746 op/s 2020-09-08T16:03:55.628 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:03:55 smithi104 bash[15273]: debug 2020-09-08T16:03:55.625+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:55.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:55 smithi055 bash[9296]: cluster 2020-09-08T16:03:54.012899+0000 mgr.x (mgr.24118) 1097 : cluster [DBG] pgmap v1623: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 746 op/s 2020-09-08T16:03:55.729 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:03:55.908 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:03:56.014 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:03:56.160 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:03:56.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:03:55 smithi104 bash[22305]: debug 2020-09-08T16:03:55.909+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:56.270 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:56.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:03:56 smithi166 bash[11928]: debug 2020-09-08T16:03:56.155+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:56.427 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:03:56.474 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T16:03:56.475 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:03:56.530 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:56.539 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:03:56 smithi104 bash[19922]: debug 2020-09-08T16:03:56.429+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:56.657 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:56.761 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:56.927 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:03:56 smithi104 bash[15273]: debug 2020-09-08T16:03:56.657+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:57.288 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2020-09-08T16:03:57.461 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:03:57.561 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:57 smithi104 bash[9203]: cluster 2020-09-08T16:03:56.013337+0000 mgr.x (mgr.24118) 1098 : cluster [DBG] pgmap v1624: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 411 op/s 2020-09-08T16:03:57.562 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:57.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:57 smithi166 bash[9446]: cluster 2020-09-08T16:03:56.013337+0000 mgr.x (mgr.24118) 1098 : cluster [DBG] pgmap v1624: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 411 op/s 2020-09-08T16:03:57.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:57 smithi055 bash[9296]: cluster 2020-09-08T16:03:56.013337+0000 mgr.x (mgr.24118) 1098 : cluster [DBG] pgmap v1624: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 411 op/s 2020-09-08T16:03:57.667 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:03:57.704 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T16:03:57.809 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:03:57.852 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:03:57.913 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:57.924 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:03:57 smithi104 bash[15273]: debug 2020-09-08T16:03:57.561+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:58.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:03:57 smithi166 bash[11928]: debug 2020-09-08T16:03:57.811+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:58.462 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:03:58.566 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:03:58.692 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:03:58.795 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:03:58.868 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:03:58 smithi166 bash[18960]: debug 2020-09-08T16:03:58.691+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:58.920 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:03:58.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:03:58 smithi104 bash[15273]: debug 2020-09-08T16:03:58.457+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:59.025 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:59.190 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:03:59.223 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:03:58 smithi166 bash[18960]: debug 2020-09-08T16:03:58.919+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:59.295 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:59.401 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:59.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:03:59 smithi055 bash[16646]: debug 2020-09-08T16:03:59.190+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:59.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:59 smithi055 bash[9296]: cluster 2020-09-08T16:03:58.014010+0000 mgr.x (mgr.24118) 1099 : cluster [DBG] pgmap v1625: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 624 op/s 2020-09-08T16:03:59.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:59 smithi055 bash[9296]: cluster 2020-09-08T16:03:58.019441+0000 osd.4 (osd.4) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:59.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:59 smithi055 bash[9296]: cluster 2020-09-08T16:03:58.019545+0000 osd.4 (osd.4) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:59.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:03:59 smithi055 bash[9296]: cluster 2020-09-08T16:03:58.229884+0000 mon.a (mon.0) 1976 : cluster [DBG] osdmap e611: 12 total, 12 up, 12 in 2020-09-08T16:03:59.505 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:59.520 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:59 smithi166 bash[9446]: cluster 2020-09-08T16:03:58.014010+0000 mgr.x (mgr.24118) 1099 : cluster [DBG] pgmap v1625: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 624 op/s 2020-09-08T16:03:59.521 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:59 smithi166 bash[9446]: cluster 2020-09-08T16:03:58.019441+0000 osd.4 (osd.4) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:59.521 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:59 smithi166 bash[9446]: cluster 2020-09-08T16:03:58.019545+0000 osd.4 (osd.4) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:59.521 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:03:59 smithi166 bash[9446]: cluster 2020-09-08T16:03:58.229884+0000 mon.a (mon.0) 1976 : cluster [DBG] osdmap e611: 12 total, 12 up, 12 in 2020-09-08T16:03:59.522 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:03:59 smithi166 bash[16593]: debug 2020-09-08T16:03:59.403+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:59.617 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:03:59.636 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:59 smithi104 bash[9203]: cluster 2020-09-08T16:03:58.014010+0000 mgr.x (mgr.24118) 1099 : cluster [DBG] pgmap v1625: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 624 op/s 2020-09-08T16:03:59.637 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:59 smithi104 bash[9203]: cluster 2020-09-08T16:03:58.019441+0000 osd.4 (osd.4) 65 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:03:59.637 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:59 smithi104 bash[9203]: cluster 2020-09-08T16:03:58.019545+0000 osd.4 (osd.4) 66 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:03:59.637 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:03:59 smithi104 bash[9203]: cluster 2020-09-08T16:03:58.229884+0000 mon.a (mon.0) 1976 : cluster [DBG] osdmap e611: 12 total, 12 up, 12 in 2020-09-08T16:03:59.724 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:59.854 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:03:59.872 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:03:59 smithi166 bash[16593]: debug 2020-09-08T16:03:59.619+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:59.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:03:59 smithi104 bash[17588]: debug 2020-09-08T16:03:59.853+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:03:59.962 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:04:00.240 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:04:00 smithi104 bash[19922]: debug 2020-09-08T16:04:00.241+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:00.248 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:04:00.349 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:00.535 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:04:00.555 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:00.573 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:04:00.607 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:04:00.638 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:04:00.660 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:00.676 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:04:00.708 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:04:00.747 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:04:00.783 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:04:00.806 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:00.830 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:04:00.868 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:04:00.905 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:04:00.909 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:00.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:04:00 smithi104 bash[22305]: debug 2020-09-08T16:04:00.553+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:00.937 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:04:00.937 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:04:00.943 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:04:00.944 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:04:00.944 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:04:00.944 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:04:00.944 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:04:00.945 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:04:00.945 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:04:00.945 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:04:00.945 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:04:00.946 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:04:00.946 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:04:00.966 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:04:00 smithi055 bash[11956]: debug 2020-09-08T16:04:00.806+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:00.966 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:00 smithi055 bash[9296]: cluster 2020-09-08T16:04:00.014565+0000 mgr.x (mgr.24118) 1100 : cluster [DBG] pgmap v1627: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 619 op/s 2020-09-08T16:04:00.966 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:00 smithi055 bash[9296]: cluster 2020-09-08T16:04:00.111538+0000 osd.0 (osd.0) 105 : cluster [DBG] 7.9 deep-scrub ok 2020-09-08T16:04:00.966 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:00 smithi055 bash[9296]: audit 2020-09-08T16:04:00.498254+0000 mon.a (mon.0) 1977 : audit [DBG] from='client.? v1:172.21.15.104:0/68823291' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:04:01.045 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:01.150 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:01.334 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:01.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:04:01 smithi166 bash[14254]: debug 2020-09-08T16:04:01.047+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:01.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:00 smithi166 bash[9446]: cluster 2020-09-08T16:04:00.014565+0000 mgr.x (mgr.24118) 1100 : cluster [DBG] pgmap v1627: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 619 op/s 2020-09-08T16:04:01.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:00 smithi166 bash[9446]: cluster 2020-09-08T16:04:00.111538+0000 osd.0 (osd.0) 105 : cluster [DBG] 7.9 deep-scrub ok 2020-09-08T16:04:01.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:00 smithi166 bash[9446]: audit 2020-09-08T16:04:00.498254+0000 mon.a (mon.0) 1977 : audit [DBG] from='client.? v1:172.21.15.104:0/68823291' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:04:01.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:00 smithi104 bash[9203]: cluster 2020-09-08T16:04:00.014565+0000 mgr.x (mgr.24118) 1100 : cluster [DBG] pgmap v1627: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 619 op/s 2020-09-08T16:04:01.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:00 smithi104 bash[9203]: cluster 2020-09-08T16:04:00.111538+0000 osd.0 (osd.0) 105 : cluster [DBG] 7.9 deep-scrub ok 2020-09-08T16:04:01.431 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:00 smithi104 bash[9203]: audit 2020-09-08T16:04:00.498254+0000 mon.a (mon.0) 1977 : audit [DBG] from='client.? v1:172.21.15.104:0/68823291' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:04:01.432 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:04:01.433 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T16:04:01.433 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:01.433 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.36113.0:237209 7.b e611/553 7:dde091c5:::benchmark_data_smithi166_48533_object102692:head v 611'14852, mlcod=611'14849)", 2020-09-08T16:04:01.433 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:04:01.424066+0000", 2020-09-08T16:04:01.434 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.014214927, 2020-09-08T16:04:01.434 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.014261922, 2020-09-08T16:04:01.434 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:01.434 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "started", 2020-09-08T16:04:01.434 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:01.435 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:01.435 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:01.435 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:01.424066+0000", 2020-09-08T16:04:01.435 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:01.435 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:01.436 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:01.436 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:01.436 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:01.424066+0000", 2020-09-08T16:04:01.436 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000102161 2020-09-08T16:04:01.436 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:01.436 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:01.437 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:01.437 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:01.424168+0000", 2020-09-08T16:04:01.437 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2859e-05 2020-09-08T16:04:01.437 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:01.437 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:01.438 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:01.438 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:01.424181+0000", 2020-09-08T16:04:01.438 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.2226e-05 2020-09-08T16:04:01.438 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:01.439 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:01.439 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:01.439 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:01.424204+0000", 2020-09-08T16:04:01.439 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.5939999999999996e-06 2020-09-08T16:04:01.439 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:01.440 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:01.440 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:01.440 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:01.424211+0000", 2020-09-08T16:04:01.440 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.004613448 2020-09-08T16:04:01.440 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:01.440 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:01.441 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:01.441 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:01.428825+0000", 2020-09-08T16:04:01.441 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.2561999999999998e-05 2020-09-08T16:04:01.441 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:01.441 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:01.442 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:01.442 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:01.428857+0000", 2020-09-08T16:04:01.442 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0047908500000000001 2020-09-08T16:04:01.442 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:01.442 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:01.443 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:01.443 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:01.443 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:01.443 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.36113.0:237215 7.b e611/553 7:d3f417b3:::benchmark_data_smithi166_48533_object102698:head v 611'14853, mlcod=611'14849)", 2020-09-08T16:04:01.443 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:04:01.433428+0000", 2020-09-08T16:04:01.444 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.0048532740000000003, 2020-09-08T16:04:01.444 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0049772310000000004, 2020-09-08T16:04:01.444 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:01.444 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "started", 2020-09-08T16:04:01.444 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:01.444 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:01.445 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:01.445 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:01.433428+0000", 2020-09-08T16:04:01.445 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:01.445 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:01.445 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:01.462 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:01.462 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:01.433428+0000", 2020-09-08T16:04:01.462 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000126954 2020-09-08T16:04:01.463 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:01.463 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:01.463 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:01.463 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:01.433555+0000", 2020-09-08T16:04:01.463 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.7249999999999999e-05 2020-09-08T16:04:01.463 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:01.464 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:01.464 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:01.464 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:01.433572+0000", 2020-09-08T16:04:01.464 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.1590000000000001e-05 2020-09-08T16:04:01.464 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:01.465 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:01.465 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:01.465 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:01.433604+0000", 2020-09-08T16:04:01.465 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3655000000000001e-05 2020-09-08T16:04:01.465 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:01.466 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:01.466 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:01.467 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:01.433617+0000", 2020-09-08T16:04:01.467 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.001060535 2020-09-08T16:04:01.467 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:01.467 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:01.467 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:01.468 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:01.434678+0000", 2020-09-08T16:04:01.468 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.5041999999999998e-05 2020-09-08T16:04:01.468 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:01.468 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:01.468 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:01.468 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:01.434703+0000", 2020-09-08T16:04:01.469 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0012750260000000001 2020-09-08T16:04:01.469 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:01.469 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:01.469 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:01.469 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:01.470 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2020-09-08T16:04:01.470 INFO:teuthology.orchestra.run.smithi055.stdout: "num_ops": 2 2020-09-08T16:04:01.470 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:04:01.482 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:04:01.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:04:01 smithi055 bash[16646]: debug 2020-09-08T16:04:01.334+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:01.701 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:04:01.805 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:01.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:04:01 smithi104 bash[19922]: debug 2020-09-08T16:04:01.701+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:02.355 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:02.459 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:02.499 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2020-09-08T16:04:02.601 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:02.649 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:04:02.656 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:04:02 smithi055 bash[14301]: debug 2020-09-08T16:04:02.354+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:02.705 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:02.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:04:02 smithi166 bash[16593]: debug 2020-09-08T16:04:02.599+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:03.283 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:03.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:03 smithi166 bash[9446]: cluster 2020-09-08T16:04:02.015130+0000 mgr.x (mgr.24118) 1101 : cluster [DBG] pgmap v1628: 17 pgs: 17 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 484 op/s 2020-09-08T16:04:03.384 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:03.392 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:03 smithi104 bash[9203]: cluster 2020-09-08T16:04:02.015130+0000 mgr.x (mgr.24118) 1101 : cluster [DBG] pgmap v1628: 17 pgs: 17 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 484 op/s 2020-09-08T16:04:03.393 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:04:03 smithi104 bash[22305]: debug 2020-09-08T16:04:03.277+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:03.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:03 smithi055 bash[9296]: cluster 2020-09-08T16:04:02.015130+0000 mgr.x (mgr.24118) 1101 : cluster [DBG] pgmap v1628: 17 pgs: 17 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 484 op/s 2020-09-08T16:04:03.999 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:04.106 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:04.238 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:04.343 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:04.358 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:04:04 smithi104 bash[15273]: debug 2020-09-08T16:04:03.997+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:04.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:04:04 smithi055 bash[14301]: debug 2020-09-08T16:04:04.238+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:05.129 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:04:05.297 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:04:05.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:05 smithi166 bash[9446]: cluster 2020-09-08T16:04:04.015670+0000 mgr.x (mgr.24118) 1102 : cluster [DBG] pgmap v1629: 17 pgs: 17 active+clean; 1.9 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 679 op/s 2020-09-08T16:04:05.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:05 smithi166 bash[9446]: audit 2020-09-08T16:04:05.009800+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? v1:172.21.15.104:0/1040410984' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:04:05.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:05 smithi055 bash[9296]: cluster 2020-09-08T16:04:04.015670+0000 mgr.x (mgr.24118) 1102 : cluster [DBG] pgmap v1629: 17 pgs: 17 active+clean; 1.9 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 679 op/s 2020-09-08T16:04:05.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:05 smithi055 bash[9296]: audit 2020-09-08T16:04:05.009800+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? v1:172.21.15.104:0/1040410984' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:04:05.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:05 smithi104 bash[9203]: cluster 2020-09-08T16:04:04.015670+0000 mgr.x (mgr.24118) 1102 : cluster [DBG] pgmap v1629: 17 pgs: 17 active+clean; 1.9 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 679 op/s 2020-09-08T16:04:05.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:05 smithi104 bash[9203]: audit 2020-09-08T16:04:05.009800+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? v1:172.21.15.104:0/1040410984' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:04:05.892 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:05.967 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T16:04:05.996 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:06.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:06 smithi166 bash[9446]: cluster 2020-09-08T16:04:05.957580+0000 mon.a (mon.0) 1979 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:04:06.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:06 smithi166 bash[9446]: audit 2020-09-08T16:04:05.965980+0000 mon.a (mon.0) 1980 : audit [INF] from='client.? v1:172.21.15.104:0/1040410984' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:04:06.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:06 smithi166 bash[9446]: cluster 2020-09-08T16:04:05.966024+0000 mon.a (mon.0) 1981 : cluster [DBG] osdmap e612: 12 total, 12 up, 12 in 2020-09-08T16:04:06.368 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:04:05 smithi166 bash[16593]: debug 2020-09-08T16:04:05.891+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:06.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:06 smithi055 bash[9296]: cluster 2020-09-08T16:04:05.957580+0000 mon.a (mon.0) 1979 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:04:06.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:06 smithi055 bash[9296]: audit 2020-09-08T16:04:05.965980+0000 mon.a (mon.0) 1980 : audit [INF] from='client.? v1:172.21.15.104:0/1040410984' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:04:06.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:06 smithi055 bash[9296]: cluster 2020-09-08T16:04:05.966024+0000 mon.a (mon.0) 1981 : cluster [DBG] osdmap e612: 12 total, 12 up, 12 in 2020-09-08T16:04:06.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:06 smithi104 bash[9203]: cluster 2020-09-08T16:04:05.957580+0000 mon.a (mon.0) 1979 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:04:06.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:06 smithi104 bash[9203]: audit 2020-09-08T16:04:05.965980+0000 mon.a (mon.0) 1980 : audit [INF] from='client.? v1:172.21.15.104:0/1040410984' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:04:06.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:06 smithi104 bash[9203]: cluster 2020-09-08T16:04:05.966024+0000 mon.a (mon.0) 1981 : cluster [DBG] osdmap e612: 12 total, 12 up, 12 in 2020-09-08T16:04:06.624 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:06.728 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:06.906 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:04:06 smithi055 bash[11956]: debug 2020-09-08T16:04:06.622+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:07.080 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:07 smithi166 bash[9446]: audit 2020-09-08T16:04:06.702195+0000 mon.a (mon.0) 1982 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136766\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:51:30.136942\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137052\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137657\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138117\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138591\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.139157\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:04:06.700991\"}"}]: dispatch 2020-09-08T16:04:07.080 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:07 smithi166 bash[9446]: audit 2020-09-08T16:04:06.708251+0000 mon.a (mon.0) 1983 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136766\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:51:30.136942\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137052\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137657\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138117\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138591\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.139157\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:04:06.700991\"}"}]': finished 2020-09-08T16:04:07.081 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:07 smithi166 bash[9446]: audit 2020-09-08T16:04:06.709346+0000 mon.a (mon.0) 1984 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:04:07.123 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:07.227 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:04:07.340 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:07 smithi166 bash[9446]: audit 2020-09-08T16:04:06.806966+0000 mon.a (mon.0) 1985 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217339\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217539\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218203\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218766\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.219295\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.219948\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:04:06.805863\"}"}]: dispatch 2020-09-08T16:04:07.340 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:07 smithi166 bash[9446]: audit 2020-09-08T16:04:06.815642+0000 mon.a (mon.0) 1986 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217339\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217539\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218203\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218766\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.219295\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.219948\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:04:06.805863\"}"}]': finished 2020-09-08T16:04:07.341 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:07 smithi166 bash[9446]: audit 2020-09-08T16:04:06.816819+0000 mon.a (mon.0) 1987 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:04:07.341 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:07 smithi166 bash[9446]: audit 2020-09-08T16:04:06.829707+0000 mon.a (mon.0) 1988 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267773\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267979\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.268707\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269253\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269785\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:51:31.270481\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:04:06.828755\"}"}]: dispatch 2020-09-08T16:04:07.341 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:07 smithi166 bash[9446]: audit 2020-09-08T16:04:06.833159+0000 mon.a (mon.0) 1989 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267773\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267979\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.268707\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269253\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269785\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:51:31.270481\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:04:06.828755\"}"}]': finished 2020-09-08T16:04:07.342 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:07 smithi166 bash[9446]: audit 2020-09-08T16:04:06.834220+0000 mon.a (mon.0) 1990 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:04:07.342 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:04:07 smithi166 bash[16593]: debug 2020-09-08T16:04:07.123+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:07.342 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:04:07.344 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:04:07.345 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T16:04:07.345 INFO:teuthology.orchestra.run.smithi055.stdout: "complaint_time": 30, 2020-09-08T16:04:07.345 INFO:teuthology.orchestra.run.smithi055.stdout: "num_blocked_ops": 0 2020-09-08T16:04:07.345 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:04:07.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:07 smithi055 bash[9296]: audit 2020-09-08T16:04:06.702195+0000 mon.a (mon.0) 1982 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136766\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:51:30.136942\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137052\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137657\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138117\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138591\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.139157\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:04:06.700991\"}"}]: dispatch 2020-09-08T16:04:07.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:07 smithi055 bash[9296]: audit 2020-09-08T16:04:06.708251+0000 mon.a (mon.0) 1983 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136766\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:51:30.136942\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137052\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137657\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138117\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138591\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.139157\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:04:06.700991\"}"}]': finished 2020-09-08T16:04:07.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:07 smithi055 bash[9296]: audit 2020-09-08T16:04:06.709346+0000 mon.a (mon.0) 1984 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:04:07.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:07 smithi055 bash[9296]: audit 2020-09-08T16:04:06.806966+0000 mon.a (mon.0) 1985 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217339\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217539\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218203\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218766\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.219295\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.219948\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:04:06.805863\"}"}]: dispatch 2020-09-08T16:04:07.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:07 smithi055 bash[9296]: audit 2020-09-08T16:04:06.815642+0000 mon.a (mon.0) 1986 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217339\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217539\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218203\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218766\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.219295\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.219948\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:04:06.805863\"}"}]': finished 2020-09-08T16:04:07.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:07 smithi055 bash[9296]: audit 2020-09-08T16:04:06.816819+0000 mon.a (mon.0) 1987 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:04:07.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:07 smithi055 bash[9296]: audit 2020-09-08T16:04:06.829707+0000 mon.a (mon.0) 1988 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267773\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267979\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.268707\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269253\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269785\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:51:31.270481\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:04:06.828755\"}"}]: dispatch 2020-09-08T16:04:07.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:07 smithi055 bash[9296]: audit 2020-09-08T16:04:06.833159+0000 mon.a (mon.0) 1989 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267773\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267979\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.268707\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269253\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269785\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:51:31.270481\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:04:06.828755\"}"}]': finished 2020-09-08T16:04:07.418 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:07 smithi055 bash[9296]: audit 2020-09-08T16:04:06.834220+0000 mon.a (mon.0) 1990 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:04:07.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:07 smithi104 bash[9203]: audit 2020-09-08T16:04:06.702195+0000 mon.a (mon.0) 1982 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136766\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:51:30.136942\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137052\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137657\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138117\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138591\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.139157\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:04:06.700991\"}"}]: dispatch 2020-09-08T16:04:07.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:07 smithi104 bash[9203]: audit 2020-09-08T16:04:06.708251+0000 mon.a (mon.0) 1983 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.136766\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T15:51:30.136942\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137052\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.137657\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138117\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.138591\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.139157\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:04:06.700991\"}"}]': finished 2020-09-08T16:04:07.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:07 smithi104 bash[9203]: audit 2020-09-08T16:04:06.709346+0000 mon.a (mon.0) 1984 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:04:07.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:07 smithi104 bash[9203]: audit 2020-09-08T16:04:06.806966+0000 mon.a (mon.0) 1985 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217339\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217539\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218203\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218766\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.219295\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.219948\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:04:06.805863\"}"}]: dispatch 2020-09-08T16:04:07.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:07 smithi104 bash[9203]: audit 2020-09-08T16:04:06.815642+0000 mon.a (mon.0) 1986 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217339\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.217539\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218203\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.218766\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:30.219295\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T15:51:30.219948\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:04:06.805863\"}"}]': finished 2020-09-08T16:04:07.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:07 smithi104 bash[9203]: audit 2020-09-08T16:04:06.816819+0000 mon.a (mon.0) 1987 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:04:07.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:07 smithi104 bash[9203]: audit 2020-09-08T16:04:06.829707+0000 mon.a (mon.0) 1988 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267773\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267979\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.268707\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269253\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269785\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:51:31.270481\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:04:06.828755\"}"}]: dispatch 2020-09-08T16:04:07.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:07 smithi104 bash[9203]: audit 2020-09-08T16:04:06.833159+0000 mon.a (mon.0) 1989 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267773\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.267979\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.268707\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269253\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T15:51:31.269785\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T15:51:31.270481\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:04:06.828755\"}"}]': finished 2020-09-08T16:04:07.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:07 smithi104 bash[9203]: audit 2020-09-08T16:04:06.834220+0000 mon.a (mon.0) 1990 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:04:07.447 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:07.630 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:04:07 smithi166 bash[18960]: debug 2020-09-08T16:04:07.339+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:07.650 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:07.757 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:07.908 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:04:07 smithi055 bash[19003]: debug 2020-09-08T16:04:07.650+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:08.074 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:08.179 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:08.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:08 smithi166 bash[9446]: cluster 2020-09-08T16:04:07.716716+0000 osd.9 (osd.9) 93 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:08.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:08 smithi166 bash[9446]: cluster 2020-09-08T16:04:07.716794+0000 osd.9 (osd.9) 94 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:08.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:08 smithi166 bash[9446]: audit 2020-09-08T16:04:07.788693+0000 mon.a (mon.0) 1991 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:04:06.805863\"}"}]: dispatch 2020-09-08T16:04:08.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:08 smithi166 bash[9446]: audit 2020-09-08T16:04:07.795904+0000 mon.a (mon.0) 1992 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:04:06.805863\"}"}]': finished 2020-09-08T16:04:08.394 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:08 smithi055 bash[9296]: cluster 2020-09-08T16:04:07.716716+0000 osd.9 (osd.9) 93 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:08.394 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:08 smithi055 bash[9296]: cluster 2020-09-08T16:04:07.716794+0000 osd.9 (osd.9) 94 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:08.394 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:08 smithi055 bash[9296]: audit 2020-09-08T16:04:07.788693+0000 mon.a (mon.0) 1991 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:04:06.805863\"}"}]: dispatch 2020-09-08T16:04:08.395 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:08 smithi055 bash[9296]: audit 2020-09-08T16:04:07.795904+0000 mon.a (mon.0) 1992 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:04:06.805863\"}"}]': finished 2020-09-08T16:04:08.395 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:08.428 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:04:08 smithi104 bash[22305]: debug 2020-09-08T16:04:08.073+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:08.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:08 smithi104 bash[9203]: cluster 2020-09-08T16:04:07.716716+0000 osd.9 (osd.9) 93 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:08.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:08 smithi104 bash[9203]: cluster 2020-09-08T16:04:07.716794+0000 osd.9 (osd.9) 94 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:08.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:08 smithi104 bash[9203]: audit 2020-09-08T16:04:07.788693+0000 mon.a (mon.0) 1991 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:04:06.805863\"}"}]: dispatch 2020-09-08T16:04:08.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:08 smithi104 bash[9203]: audit 2020-09-08T16:04:07.795904+0000 mon.a (mon.0) 1992 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"Insufficient space (<5GB) on vgs\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\", \"SCSI VPD page 0x83 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T15:34:40.135389\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:04:06.805863\"}"}]': finished 2020-09-08T16:04:08.497 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:04:08.524 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2020-09-08T16:04:08.646 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:04:08.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:04:08 smithi055 bash[11956]: debug 2020-09-08T16:04:08.394+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:08.698 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:04:08.754 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:08.763 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:04:08 smithi104 bash[19922]: debug 2020-09-08T16:04:08.645+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:09.090 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:09 smithi104 bash[9203]: cluster 2020-09-08T16:04:08.016610+0000 mgr.x (mgr.24118) 1104 : cluster [DBG] pgmap v1632: 17 pgs: 17 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 833 op/s 2020-09-08T16:04:09.091 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:09 smithi104 bash[9203]: cluster 2020-09-08T16:04:08.087039+0000 mon.a (mon.0) 1993 : cluster [DBG] osdmap e613: 12 total, 12 up, 12 in 2020-09-08T16:04:09.091 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:09 smithi104 bash[9203]: audit 2020-09-08T16:04:08.216462+0000 mon.a (mon.0) 1994 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:04:06.700991\"}"}]: dispatch 2020-09-08T16:04:09.091 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:09 smithi104 bash[9203]: audit 2020-09-08T16:04:08.220341+0000 mon.a (mon.0) 1995 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:04:06.700991\"}"}]': finished 2020-09-08T16:04:09.092 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:09 smithi104 bash[9203]: audit 2020-09-08T16:04:08.221615+0000 mon.a (mon.0) 1996 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T16:04:09.092 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:09 smithi104 bash[9203]: audit 2020-09-08T16:04:08.552925+0000 mon.a (mon.0) 1997 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:04:06.828755\"}"}]: dispatch 2020-09-08T16:04:09.092 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:09 smithi104 bash[9203]: audit 2020-09-08T16:04:08.556860+0000 mon.a (mon.0) 1998 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:04:06.828755\"}"}]': finished 2020-09-08T16:04:09.092 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:09 smithi104 bash[9203]: audit 2020-09-08T16:04:08.557662+0000 mon.a (mon.0) 1999 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T16:04:09.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:09 smithi166 bash[9446]: cluster 2020-09-08T16:04:08.016610+0000 mgr.x (mgr.24118) 1104 : cluster [DBG] pgmap v1632: 17 pgs: 17 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 833 op/s 2020-09-08T16:04:09.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:09 smithi166 bash[9446]: cluster 2020-09-08T16:04:08.087039+0000 mon.a (mon.0) 1993 : cluster [DBG] osdmap e613: 12 total, 12 up, 12 in 2020-09-08T16:04:09.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:09 smithi166 bash[9446]: audit 2020-09-08T16:04:08.216462+0000 mon.a (mon.0) 1994 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:04:06.700991\"}"}]: dispatch 2020-09-08T16:04:09.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:09 smithi166 bash[9446]: audit 2020-09-08T16:04:08.220341+0000 mon.a (mon.0) 1995 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:04:06.700991\"}"}]': finished 2020-09-08T16:04:09.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:09 smithi166 bash[9446]: audit 2020-09-08T16:04:08.221615+0000 mon.a (mon.0) 1996 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T16:04:09.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:09 smithi166 bash[9446]: audit 2020-09-08T16:04:08.552925+0000 mon.a (mon.0) 1997 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:04:06.828755\"}"}]: dispatch 2020-09-08T16:04:09.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:09 smithi166 bash[9446]: audit 2020-09-08T16:04:08.556860+0000 mon.a (mon.0) 1998 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:04:06.828755\"}"}]': finished 2020-09-08T16:04:09.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:09 smithi166 bash[9446]: audit 2020-09-08T16:04:08.557662+0000 mon.a (mon.0) 1999 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T16:04:09.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:09 smithi055 bash[9296]: cluster 2020-09-08T16:04:08.016610+0000 mgr.x (mgr.24118) 1104 : cluster [DBG] pgmap v1632: 17 pgs: 17 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 833 op/s 2020-09-08T16:04:09.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:09 smithi055 bash[9296]: cluster 2020-09-08T16:04:08.087039+0000 mon.a (mon.0) 1993 : cluster [DBG] osdmap e613: 12 total, 12 up, 12 in 2020-09-08T16:04:09.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:09 smithi055 bash[9296]: audit 2020-09-08T16:04:08.216462+0000 mon.a (mon.0) 1994 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:04:06.700991\"}"}]: dispatch 2020-09-08T16:04:09.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:09 smithi055 bash[9296]: audit 2020-09-08T16:04:08.220341+0000 mon.a (mon.0) 1995 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0xb1 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T15:33:00.860023\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:04:06.700991\"}"}]': finished 2020-09-08T16:04:09.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:09 smithi055 bash[9296]: audit 2020-09-08T16:04:08.221615+0000 mon.a (mon.0) 1996 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T16:04:09.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:09 smithi055 bash[9296]: audit 2020-09-08T16:04:08.552925+0000 mon.a (mon.0) 1997 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:04:06.828755\"}"}]: dispatch 2020-09-08T16:04:09.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:09 smithi055 bash[9296]: audit 2020-09-08T16:04:08.556860+0000 mon.a (mon.0) 1998 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T15:31:18.755775\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:04:06.828755\"}"}]': finished 2020-09-08T16:04:09.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:09 smithi055 bash[9296]: audit 2020-09-08T16:04:08.557662+0000 mon.a (mon.0) 1999 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T16:04:09.830 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T16:04:09.905 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:09.996 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:04:10.009 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:10.094 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:04:09 smithi104 bash[22305]: debug 2020-09-08T16:04:09.905+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:10.216 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:10.319 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:10.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:10 smithi166 bash[9446]: cluster 2020-09-08T16:04:09.251926+0000 osd.2 (osd.2) 96 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:10.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:10 smithi166 bash[9446]: cluster 2020-09-08T16:04:09.252006+0000 osd.2 (osd.2) 97 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:10.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:04:10 smithi055 bash[11956]: debug 2020-09-08T16:04:10.214+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:10.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:10 smithi055 bash[9296]: cluster 2020-09-08T16:04:09.251926+0000 osd.2 (osd.2) 96 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:10.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:10 smithi055 bash[9296]: cluster 2020-09-08T16:04:09.252006+0000 osd.2 (osd.2) 97 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:10.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:10 smithi104 bash[9203]: cluster 2020-09-08T16:04:09.251926+0000 osd.2 (osd.2) 96 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:10.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:10 smithi104 bash[9203]: cluster 2020-09-08T16:04:09.252006+0000 osd.2 (osd.2) 97 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:10.994 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:11.099 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:11.100 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:11 smithi055 bash[9296]: cluster 2020-09-08T16:04:10.017209+0000 mgr.x (mgr.24118) 1105 : cluster [DBG] pgmap v1634: 17 pgs: 17 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 957 op/s 2020-09-08T16:04:11.100 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:11 smithi055 bash[9296]: cluster 2020-09-08T16:04:10.104244+0000 mon.a (mon.0) 2000 : cluster [DBG] osdmap e614: 12 total, 12 up, 12 in 2020-09-08T16:04:11.248 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:11.352 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:11.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:04:11 smithi166 bash[14254]: debug 2020-09-08T16:04:11.247+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:11 smithi166 bash[9446]: cluster 2020-09-08T16:04:10.017209+0000 mgr.x (mgr.24118) 1105 : cluster [DBG] pgmap v1634: 17 pgs: 17 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 957 op/s 2020-09-08T16:04:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:11 smithi166 bash[9446]: cluster 2020-09-08T16:04:10.104244+0000 mon.a (mon.0) 2000 : cluster [DBG] osdmap e614: 12 total, 12 up, 12 in 2020-09-08T16:04:11.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:04:10 smithi104 bash[15273]: debug 2020-09-08T16:04:10.993+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:11.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:11 smithi104 bash[9203]: cluster 2020-09-08T16:04:10.017209+0000 mgr.x (mgr.24118) 1105 : cluster [DBG] pgmap v1634: 17 pgs: 17 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 957 op/s 2020-09-08T16:04:11.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:11 smithi104 bash[9203]: cluster 2020-09-08T16:04:10.104244+0000 mon.a (mon.0) 2000 : cluster [DBG] osdmap e614: 12 total, 12 up, 12 in 2020-09-08T16:04:11.880 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:11.984 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:12.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:04:11 smithi055 bash[11956]: debug 2020-09-08T16:04:11.878+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:12.344 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:12.447 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:12.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:04:12 smithi055 bash[11956]: debug 2020-09-08T16:04:12.342+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:13.101 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:13 smithi104 bash[9203]: cluster 2020-09-08T16:04:12.017702+0000 mgr.x (mgr.24118) 1106 : cluster [DBG] pgmap v1636: 17 pgs: 17 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.07k op/s 2020-09-08T16:04:13.128 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:13.232 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:13.363 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:04:13 smithi104 bash[17588]: debug 2020-09-08T16:04:13.125+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:13.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:13 smithi166 bash[9446]: cluster 2020-09-08T16:04:12.017702+0000 mgr.x (mgr.24118) 1106 : cluster [DBG] pgmap v1636: 17 pgs: 17 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.07k op/s 2020-09-08T16:04:13.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:13 smithi055 bash[9296]: cluster 2020-09-08T16:04:12.017702+0000 mgr.x (mgr.24118) 1106 : cluster [DBG] pgmap v1636: 17 pgs: 17 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.07k op/s 2020-09-08T16:04:13.840 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:13.946 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:14.112 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:14.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:04:13 smithi055 bash[19003]: debug 2020-09-08T16:04:13.834+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:14.218 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:04:14.226 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:04:14 smithi166 bash[14254]: debug 2020-09-08T16:04:14.111+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:14.357 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:04:14.461 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:14.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:04:14 smithi166 bash[18960]: debug 2020-09-08T16:04:14.359+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:14.692 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:14.795 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:04:14.829 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:04:14.829 INFO:teuthology.orchestra.run.smithi055.stdout: "size": 20, 2020-09-08T16:04:14.829 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 600, 2020-09-08T16:04:14.830 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T16:04:14.830 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.830 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.16014.0:74375 6.d 6:b7fa7f46:::benchmark_data_smithi166_43187_object74374:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T16:04:14.830 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:57:19.570763+0000", 2020-09-08T16:04:14.830 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 415.261328483, 2020-09-08T16:04:14.831 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.41064162799999998, 2020-09-08T16:04:14.831 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:14.831 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:14.831 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:14.832 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.16014", 2020-09-08T16:04:14.832 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:04:14.832 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 74375 2020-09-08T16:04:14.832 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.832 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:14.833 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.833 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:14.833 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.570763+0000", 2020-09-08T16:04:14.833 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:14.833 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.834 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.834 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:14.834 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.570763+0000", 2020-09-08T16:04:14.834 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999514 2020-09-08T16:04:14.834 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.834 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.835 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:14.835 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.570715+0000", 2020-09-08T16:04:14.835 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015374799999999999 2020-09-08T16:04:14.835 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.835 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.836 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:14.836 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.570869+0000", 2020-09-08T16:04:14.836 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.0256000000000001e-05 2020-09-08T16:04:14.836 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.836 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.837 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:14.837 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.570889+0000", 2020-09-08T16:04:14.837 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0379e-05 2020-09-08T16:04:14.837 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.837 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.838 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:14.838 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.570899+0000", 2020-09-08T16:04:14.838 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.8862e-05 2020-09-08T16:04:14.838 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.838 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.839 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:14.839 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.570958+0000", 2020-09-08T16:04:14.839 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00018679699999999999 2020-09-08T16:04:14.839 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.839 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.840 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:14.840 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.571145+0000", 2020-09-08T16:04:14.840 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019703599999999999 2020-09-08T16:04:14.840 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.840 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.841 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T16:04:14.841 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.571342+0000", 2020-09-08T16:04:14.841 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00280131 2020-09-08T16:04:14.841 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.842 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.842 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:14.842 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.574143+0000", 2020-09-08T16:04:14.842 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40708195800000002 2020-09-08T16:04:14.842 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.843 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.843 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:14.843 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.981225+0000", 2020-09-08T16:04:14.844 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.6917999999999996e-05 2020-09-08T16:04:14.844 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.844 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.844 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:14.844 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.981322+0000", 2020-09-08T16:04:14.845 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2764999999999998e-05 2020-09-08T16:04:14.845 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.845 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.845 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:14.845 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.981405+0000", 2020-09-08T16:04:14.846 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.41064162799999998 2020-09-08T16:04:14.846 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:14.846 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:14.846 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:14.846 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.847 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.847 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.16014.0:74388 6.d 6:b4e705c2:::benchmark_data_smithi166_43187_object74387:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T16:04:14.847 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:57:19.581710+0000", 2020-09-08T16:04:14.847 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 415.25038132100002, 2020-09-08T16:04:14.847 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40267957799999998, 2020-09-08T16:04:14.848 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:14.848 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:14.848 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:14.848 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.16014", 2020-09-08T16:04:14.848 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:04:14.849 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 74388 2020-09-08T16:04:14.849 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.849 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:14.850 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.850 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:14.850 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.581710+0000", 2020-09-08T16:04:14.850 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:14.850 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.851 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.851 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:14.851 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.581710+0000", 2020-09-08T16:04:14.851 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999958 2020-09-08T16:04:14.852 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.852 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.852 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:14.852 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.581668+0000", 2020-09-08T16:04:14.852 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000176214 2020-09-08T16:04:14.853 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.853 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.853 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:14.853 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.581845+0000", 2020-09-08T16:04:14.853 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.4443999999999998e-05 2020-09-08T16:04:14.854 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.854 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.854 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:14.854 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.581869+0000", 2020-09-08T16:04:14.854 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0013999999999999e-05 2020-09-08T16:04:14.855 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.855 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.855 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:14.855 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.581879+0000", 2020-09-08T16:04:14.855 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.5612999999999998e-05 2020-09-08T16:04:14.856 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.856 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.856 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:14.857 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.581935+0000", 2020-09-08T16:04:14.857 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000281231 2020-09-08T16:04:14.857 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.857 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.858 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:14.858 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.582216+0000", 2020-09-08T16:04:14.858 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00026418699999999998 2020-09-08T16:04:14.858 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T16:04:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.582480+0000", 2020-09-08T16:04:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0026592090000000001 2020-09-08T16:04:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.585139+0000", 2020-09-08T16:04:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.39909159599999999 2020-09-08T16:04:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.984231+0000", 2020-09-08T16:04:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.0936e-05 2020-09-08T16:04:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.984302+0000", 2020-09-08T16:04:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.8082000000000001e-05 2020-09-08T16:04:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.863 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:14.863 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.984390+0000", 2020-09-08T16:04:14.863 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40267957799999998 2020-09-08T16:04:14.863 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:14.863 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:14.864 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:14.864 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.864 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.864 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.16014.0:74392 6.d 6:b77d6839:::benchmark_data_smithi166_43187_object74391:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T16:04:14.864 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:57:19.585469+0000", 2020-09-08T16:04:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 415.24662313699997, 2020-09-08T16:04:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.39911287899999998, 2020-09-08T16:04:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:14.866 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.16014", 2020-09-08T16:04:14.866 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:04:14.866 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 74392 2020-09-08T16:04:14.866 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.866 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:14.866 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.867 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:14.867 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.585469+0000", 2020-09-08T16:04:14.867 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:14.867 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.867 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.868 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:14.868 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.585469+0000", 2020-09-08T16:04:14.868 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999619 2020-09-08T16:04:14.868 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.868 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.869 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:14.869 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.585430+0000", 2020-09-08T16:04:14.869 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000138529 2020-09-08T16:04:14.869 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.869 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.870 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:14.870 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.585569+0000", 2020-09-08T16:04:14.870 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4814000000000001e-05 2020-09-08T16:04:14.870 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.870 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.871 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:14.871 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.585584+0000", 2020-09-08T16:04:14.871 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.1739999999999997e-06 2020-09-08T16:04:14.871 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.871 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.872 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:14.872 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.585590+0000", 2020-09-08T16:04:14.872 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.8352000000000001e-05 2020-09-08T16:04:14.872 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.872 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.873 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:14.873 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.585638+0000", 2020-09-08T16:04:14.873 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014737999999999999 2020-09-08T16:04:14.873 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.874 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.874 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:14.874 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.585785+0000", 2020-09-08T16:04:14.874 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000119009 2020-09-08T16:04:14.874 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.875 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.875 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T16:04:14.875 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.585904+0000", 2020-09-08T16:04:14.875 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0057142169999999997 2020-09-08T16:04:14.875 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.876 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.876 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:14.876 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.591619+0000", 2020-09-08T16:04:14.876 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.392820108 2020-09-08T16:04:14.876 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.877 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.877 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:14.877 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.984439+0000", 2020-09-08T16:04:14.877 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.3498999999999995e-05 2020-09-08T16:04:14.877 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.877 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.878 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:14.878 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.984512+0000", 2020-09-08T16:04:14.878 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.9177000000000005e-05 2020-09-08T16:04:14.878 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.878 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.879 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:14.879 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.984581+0000", 2020-09-08T16:04:14.879 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.39911287899999998 2020-09-08T16:04:14.879 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:14.879 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.16014.0:74396 6.d 6:b9c82e1f:::benchmark_data_smithi166_43187_object74395:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T16:04:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:57:19.588419+0000", 2020-09-08T16:04:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 415.24367240100003, 2020-09-08T16:04:14.881 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40133078700000002, 2020-09-08T16:04:14.881 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:14.881 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:14.881 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:14.882 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.16014", 2020-09-08T16:04:14.882 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:04:14.883 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 74396 2020-09-08T16:04:14.883 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.884 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:14.884 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.884 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:14.884 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.588419+0000", 2020-09-08T16:04:14.884 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:14.885 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.886 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.886 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:14.886 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.588419+0000", 2020-09-08T16:04:14.887 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999571 2020-09-08T16:04:14.887 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.887 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.887 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:14.887 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.588377+0000", 2020-09-08T16:04:14.888 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00020347099999999999 2020-09-08T16:04:14.888 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.888 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.888 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:14.888 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.588580+0000", 2020-09-08T16:04:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5017999999999999e-05 2020-09-08T16:04:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.588595+0000", 2020-09-08T16:04:14.890 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.1190000000000002e-06 2020-09-08T16:04:14.890 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.890 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.890 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:14.890 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.588601+0000", 2020-09-08T16:04:14.891 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000108695 2020-09-08T16:04:14.891 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.891 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.891 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:14.891 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.588710+0000", 2020-09-08T16:04:14.891 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000151175 2020-09-08T16:04:14.892 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.892 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.892 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:14.892 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.588861+0000", 2020-09-08T16:04:14.892 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000202401 2020-09-08T16:04:14.893 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.893 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.893 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T16:04:14.893 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.589063+0000", 2020-09-08T16:04:14.893 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0031794589999999999 2020-09-08T16:04:14.894 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.894 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.894 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:14.894 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.592243+0000", 2020-09-08T16:04:14.894 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.39736044799999998 2020-09-08T16:04:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.989603+0000", 2020-09-08T16:04:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.9627000000000003e-05 2020-09-08T16:04:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.989673+0000", 2020-09-08T16:04:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.7101999999999995e-05 2020-09-08T16:04:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.897 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.897 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:14.897 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.989750+0000", 2020-09-08T16:04:14.897 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40133078700000002 2020-09-08T16:04:14.897 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:14.898 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:14.898 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:14.898 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.898 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.898 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.16014.0:74419 6.d 6:be2a3839:::benchmark_data_smithi166_43187_object74418:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T16:04:14.898 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:57:19.613167+0000", 2020-09-08T16:04:14.899 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 415.21892506500001, 2020-09-08T16:04:14.899 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.37687683100000002, 2020-09-08T16:04:14.899 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:14.899 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:14.899 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:14.900 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.16014", 2020-09-08T16:04:14.900 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:04:14.900 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 74419 2020-09-08T16:04:14.900 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.900 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.613167+0000", 2020-09-08T16:04:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.902 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:14.902 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.613167+0000", 2020-09-08T16:04:14.902 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999633 2020-09-08T16:04:14.902 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.902 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.903 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:14.903 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.613130+0000", 2020-09-08T16:04:14.903 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000150149 2020-09-08T16:04:14.903 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.903 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.904 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:14.904 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.613280+0000", 2020-09-08T16:04:14.904 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.1988e-05 2020-09-08T16:04:14.904 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.904 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.904 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:14.905 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.613302+0000", 2020-09-08T16:04:14.905 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1919999999999999e-05 2020-09-08T16:04:14.905 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.905 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.905 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:14.906 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.613314+0000", 2020-09-08T16:04:14.906 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.6912e-05 2020-09-08T16:04:14.906 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.906 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.906 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.613371+0000", 2020-09-08T16:04:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013859500000000001 2020-09-08T16:04:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.613510+0000", 2020-09-08T16:04:14.908 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015276299999999999 2020-09-08T16:04:14.908 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.908 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.908 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T16:04:14.908 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.613662+0000", 2020-09-08T16:04:14.909 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.002381457 2020-09-08T16:04:14.909 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.909 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.909 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:14.909 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.616044+0000", 2020-09-08T16:04:14.910 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.37387160899999999 2020-09-08T16:04:14.910 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.910 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.910 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:14.910 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.989915+0000", 2020-09-08T16:04:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.1564999999999998e-05 2020-09-08T16:04:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.989977+0000", 2020-09-08T16:04:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.6550000000000005e-05 2020-09-08T16:04:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.990043+0000", 2020-09-08T16:04:14.913 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.37687683100000002 2020-09-08T16:04:14.913 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:14.913 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:14.913 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:14.913 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.914 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.914 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.16014.0:74425 6.d 6:babce355:::benchmark_data_smithi166_43187_object74424:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T16:04:14.914 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:57:19.625040+0000", 2020-09-08T16:04:14.914 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 415.20705133000001, 2020-09-08T16:04:14.914 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.36523786400000002, 2020-09-08T16:04:14.915 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:14.915 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:14.915 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:14.915 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.16014", 2020-09-08T16:04:14.915 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:04:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 74425 2020-09-08T16:04:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.625040+0000", 2020-09-08T16:04:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.625040+0000", 2020-09-08T16:04:14.918 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999471 2020-09-08T16:04:14.918 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.918 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.918 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:14.918 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.624987+0000", 2020-09-08T16:04:14.919 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017899600000000001 2020-09-08T16:04:14.919 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.919 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.919 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:14.919 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.625166+0000", 2020-09-08T16:04:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.8389999999999998e-05 2020-09-08T16:04:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.625185+0000", 2020-09-08T16:04:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4754e-05 2020-09-08T16:04:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.625199+0000", 2020-09-08T16:04:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.6064999999999999e-05 2020-09-08T16:04:14.922 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.922 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.922 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:14.922 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.625255+0000", 2020-09-08T16:04:14.922 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017153399999999999 2020-09-08T16:04:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.625427+0000", 2020-09-08T16:04:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000202337 2020-09-08T16:04:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T16:04:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.625629+0000", 2020-09-08T16:04:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.002124286 2020-09-08T16:04:14.925 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.925 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.925 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:14.925 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.627754+0000", 2020-09-08T16:04:14.925 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.36235015599999998 2020-09-08T16:04:14.926 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.926 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.926 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:14.926 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.990104+0000", 2020-09-08T16:04:14.926 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000100166 2020-09-08T16:04:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.990204+0000", 2020-09-08T16:04:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.4320000000000007e-05 2020-09-08T16:04:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.928 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.928 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:14.928 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:19.990278+0000", 2020-09-08T16:04:14.928 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.36523786400000002 2020-09-08T16:04:14.928 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:14.929 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:14.929 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:14.929 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.929 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.929 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.16014.0:78097 6.a 6:50adb852:::benchmark_data_smithi166_43187_object78096:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T16:04:14.930 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:57:22.616782+0000", 2020-09-08T16:04:14.930 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 412.21531017199999, 2020-09-08T16:04:14.930 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.38073221000000002, 2020-09-08T16:04:14.930 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:14.930 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:14.930 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:14.931 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.16014", 2020-09-08T16:04:14.931 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:04:14.931 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 78097 2020-09-08T16:04:14.931 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.931 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.616782+0000", 2020-09-08T16:04:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.933 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.933 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:14.933 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.616782+0000", 2020-09-08T16:04:14.933 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999958 2020-09-08T16:04:14.933 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.933 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.934 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:14.934 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.616740+0000", 2020-09-08T16:04:14.934 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013124999999999999 2020-09-08T16:04:14.934 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.934 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.935 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:14.935 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.616871+0000", 2020-09-08T16:04:14.935 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.2229999999999996e-06 2020-09-08T16:04:14.935 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.935 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.936 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:14.936 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.616880+0000", 2020-09-08T16:04:14.936 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.5630000000000002e-06 2020-09-08T16:04:14.936 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.936 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.936 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.616885+0000", 2020-09-08T16:04:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0029027179999999999 2020-09-08T16:04:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:14.938 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.619787+0000", 2020-09-08T16:04:14.938 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014096000000000001 2020-09-08T16:04:14.938 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.938 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.938 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.619928+0000", 2020-09-08T16:04:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000126735 2020-09-08T16:04:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 6", 2020-09-08T16:04:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.620055+0000", 2020-09-08T16:04:14.940 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0090583869999999993 2020-09-08T16:04:14.940 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.940 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.940 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:14.940 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.629113+0000", 2020-09-08T16:04:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.36787397300000002 2020-09-08T16:04:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.996987+0000", 2020-09-08T16:04:14.942 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.8280000000000002e-05 2020-09-08T16:04:14.942 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.942 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.942 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:14.942 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.997076+0000", 2020-09-08T16:04:14.943 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00043804800000000002 2020-09-08T16:04:14.943 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.943 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.943 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:14.943 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.997514+0000", 2020-09-08T16:04:14.943 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.38073221000000002 2020-09-08T16:04:14.944 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:14.944 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:14.944 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:14.944 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.944 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.945 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.16014.0:78105 6.a 6:577ceeae:::benchmark_data_smithi166_43187_object78104:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T16:04:14.945 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:57:22.620960+0000", 2020-09-08T16:04:14.945 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 412.21113158100002, 2020-09-08T16:04:14.945 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.37729238399999998, 2020-09-08T16:04:14.945 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:14.946 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:14.946 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:14.946 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.16014", 2020-09-08T16:04:14.946 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:04:14.946 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 78105 2020-09-08T16:04:14.947 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.947 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:14.947 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.947 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:14.947 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.620960+0000", 2020-09-08T16:04:14.947 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.620960+0000", 2020-09-08T16:04:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999604 2020-09-08T16:04:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.620921+0000", 2020-09-08T16:04:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013446899999999999 2020-09-08T16:04:14.950 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.950 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.950 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:14.950 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.621055+0000", 2020-09-08T16:04:14.950 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.9240000000000002e-06 2020-09-08T16:04:14.950 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.951 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.951 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:14.951 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.621065+0000", 2020-09-08T16:04:14.951 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.1579999999999999e-06 2020-09-08T16:04:14.951 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.952 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.952 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:14.952 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.621072+0000", 2020-09-08T16:04:14.952 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0018232109999999999 2020-09-08T16:04:14.952 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.622896+0000", 2020-09-08T16:04:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000174795 2020-09-08T16:04:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.954 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:14.954 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.623070+0000", 2020-09-08T16:04:14.954 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000192055 2020-09-08T16:04:14.954 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.954 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.955 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 6", 2020-09-08T16:04:14.955 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.623262+0000", 2020-09-08T16:04:14.955 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0058605089999999999 2020-09-08T16:04:14.955 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.955 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.956 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:14.956 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.629123+0000", 2020-09-08T16:04:14.956 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.36898113700000001 2020-09-08T16:04:14.956 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.956 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.956 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:14.957 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.998104+0000", 2020-09-08T16:04:14.957 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4414999999999996e-05 2020-09-08T16:04:14.957 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.957 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.957 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:14.958 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.998198+0000", 2020-09-08T16:04:14.958 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4058000000000003e-05 2020-09-08T16:04:14.958 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.958 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.958 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:14.959 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.998252+0000", 2020-09-08T16:04:14.959 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.37729238399999998 2020-09-08T16:04:14.959 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:14.959 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:14.959 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:14.959 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.960 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.960 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.16014.0:78107 6.a 6:5e7bb68a:::benchmark_data_smithi166_43187_object78106:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T16:04:14.960 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:57:22.621118+0000", 2020-09-08T16:04:14.960 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 412.210973438, 2020-09-08T16:04:14.960 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.37945717699999998, 2020-09-08T16:04:14.961 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:14.961 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:14.961 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:14.961 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.16014", 2020-09-08T16:04:14.961 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:04:14.962 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 78107 2020-09-08T16:04:14.962 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.962 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:14.962 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.962 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:14.963 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.621118+0000", 2020-09-08T16:04:14.963 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:14.963 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.966 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.966 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:14.966 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.621118+0000", 2020-09-08T16:04:14.967 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999776 2020-09-08T16:04:14.967 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.967 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.967 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:14.967 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.621096+0000", 2020-09-08T16:04:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000108638 2020-09-08T16:04:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.621204+0000", 2020-09-08T16:04:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3273e-05 2020-09-08T16:04:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.621218+0000", 2020-09-08T16:04:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.6329999999999996e-06 2020-09-08T16:04:14.970 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.970 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.970 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:14.970 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.621225+0000", 2020-09-08T16:04:14.970 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.004539347 2020-09-08T16:04:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.625765+0000", 2020-09-08T16:04:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014013200000000001 2020-09-08T16:04:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.625905+0000", 2020-09-08T16:04:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015754500000000001 2020-09-08T16:04:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.973 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.973 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 6", 2020-09-08T16:04:14.973 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.626062+0000", 2020-09-08T16:04:14.973 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.003068159 2020-09-08T16:04:14.973 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.974 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.974 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:14.974 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.629130+0000", 2020-09-08T16:04:14.974 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.37130887899999998 2020-09-08T16:04:14.974 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.975 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.975 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:14.975 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:23.000439+0000", 2020-09-08T16:04:14.975 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.6630000000000003e-05 2020-09-08T16:04:14.975 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.975 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.976 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:14.976 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:23.000516+0000", 2020-09-08T16:04:14.976 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9579e-05 2020-09-08T16:04:14.976 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.976 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.977 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:14.977 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:23.000575+0000", 2020-09-08T16:04:14.977 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.37945717699999998 2020-09-08T16:04:14.977 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:14.977 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:14.978 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:14.978 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.978 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.978 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.16014.0:78120 6.a 6:5f4460a9:::benchmark_data_smithi166_43187_object78119:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e499)", 2020-09-08T16:04:14.978 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:57:22.629512+0000", 2020-09-08T16:04:14.978 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 412.20257921299998, 2020-09-08T16:04:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.371256166, 2020-09-08T16:04:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.16014", 2020-09-08T16:04:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:04:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 78120 2020-09-08T16:04:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.629512+0000", 2020-09-08T16:04:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.982 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:14.982 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.629512+0000", 2020-09-08T16:04:14.982 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999738 2020-09-08T16:04:14.982 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.982 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.982 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:14.983 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.629486+0000", 2020-09-08T16:04:14.983 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3262000000000003e-05 2020-09-08T16:04:14.983 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.983 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.983 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.629580+0000", 2020-09-08T16:04:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1466e-05 2020-09-08T16:04:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:14.985 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.629591+0000", 2020-09-08T16:04:14.985 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.8369999999999996e-06 2020-09-08T16:04:14.985 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.629596+0000", 2020-09-08T16:04:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.8455999999999998e-05 2020-09-08T16:04:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.987 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:14.987 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.629644+0000", 2020-09-08T16:04:14.987 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013207900000000001 2020-09-08T16:04:14.987 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.987 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.629777+0000", 2020-09-08T16:04:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000118013 2020-09-08T16:04:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 6", 2020-09-08T16:04:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.629895+0000", 2020-09-08T16:04:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0037697920000000001 2020-09-08T16:04:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:14.990 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.633664+0000", 2020-09-08T16:04:14.990 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.366981848 2020-09-08T16:04:14.990 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.990 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.990 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:14.991 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:23.000646+0000", 2020-09-08T16:04:14.991 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.1748999999999994e-05 2020-09-08T16:04:14.991 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.991 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.991 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:23.000718+0000", 2020-09-08T16:04:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.0714999999999998e-05 2020-09-08T16:04:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:23.000769+0000", 2020-09-08T16:04:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.371256166 2020-09-08T16:04:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.994 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.994 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.16014.0:166975 6.7 6:e454c3cd:::benchmark_data_smithi166_43187_object47206:head [delete] snapc 0=[] ondisk+write+known_if_redirected e522)", 2020-09-08T16:04:14.994 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:58:46.534605+0000", 2020-09-08T16:04:14.994 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 328.297487042, 2020-09-08T16:04:14.994 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.415648028, 2020-09-08T16:04:14.995 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:14.995 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:14.995 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:14.995 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.16014", 2020-09-08T16:04:14.995 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:04:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 166975 2020-09-08T16:04:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.534605+0000", 2020-09-08T16:04:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.534605+0000", 2020-09-08T16:04:14.998 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999609 2020-09-08T16:04:14.998 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.998 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.998 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:14.998 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.534566+0000", 2020-09-08T16:04:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4493999999999994e-05 2020-09-08T16:04:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.534650+0000", 2020-09-08T16:04:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2792999999999999e-05 2020-09-08T16:04:15.000 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.000 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.000 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:15.000 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.534663+0000", 2020-09-08T16:04:15.000 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.9280000000000001e-06 2020-09-08T16:04:15.001 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.001 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.001 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:15.001 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.534668+0000", 2020-09-08T16:04:15.001 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000104843 2020-09-08T16:04:15.002 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.002 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.002 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:15.002 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.534773+0000", 2020-09-08T16:04:15.002 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000202838 2020-09-08T16:04:15.002 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.003 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.003 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:15.003 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.534975+0000", 2020-09-08T16:04:15.003 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00030267500000000003 2020-09-08T16:04:15.003 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.004 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.004 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:04:15.004 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.535278+0000", 2020-09-08T16:04:15.004 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0045081499999999998 2020-09-08T16:04:15.004 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.005 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.005 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:15.005 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.539786+0000", 2020-09-08T16:04:15.005 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.410263567 2020-09-08T16:04:15.005 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.006 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.006 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:15.006 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.950050+0000", 2020-09-08T16:04:15.006 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000120615 2020-09-08T16:04:15.006 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.006 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.007 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:15.007 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.950170+0000", 2020-09-08T16:04:15.007 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2198000000000001e-05 2020-09-08T16:04:15.007 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.007 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.008 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:15.008 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.950253+0000", 2020-09-08T16:04:15.008 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.415648028 2020-09-08T16:04:15.008 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.008 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:15.008 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.009 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.009 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.009 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.16014.0:166986 6.7 6:ef8e0dcf:::benchmark_data_smithi166_43187_object47217:head [delete] snapc 0=[] ondisk+write+known_if_redirected e522)", 2020-09-08T16:04:15.009 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:58:46.546720+0000", 2020-09-08T16:04:15.009 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 328.28537206999999, 2020-09-08T16:04:15.010 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40948933999999998, 2020-09-08T16:04:15.010 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:15.010 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:15.010 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:15.010 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.16014", 2020-09-08T16:04:15.011 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:04:15.011 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 166986 2020-09-08T16:04:15.011 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.011 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:15.011 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.012 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:15.012 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.546720+0000", 2020-09-08T16:04:15.012 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:15.012 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.013 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.013 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:15.013 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.546720+0000", 2020-09-08T16:04:15.013 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999743 2020-09-08T16:04:15.013 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.014 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.014 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:15.014 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.546694+0000", 2020-09-08T16:04:15.014 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.8578000000000001e-05 2020-09-08T16:04:15.014 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.014 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.015 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:15.015 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.546753+0000", 2020-09-08T16:04:15.015 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2565000000000001e-05 2020-09-08T16:04:15.015 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.015 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.016 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:15.016 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.546765+0000", 2020-09-08T16:04:15.016 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.968e-06 2020-09-08T16:04:15.016 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.016 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.546772+0000", 2020-09-08T16:04:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.7849000000000006e-05 2020-09-08T16:04:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:15.018 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.546850+0000", 2020-09-08T16:04:15.018 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013104300000000001 2020-09-08T16:04:15.018 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.018 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.018 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:15.019 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.546981+0000", 2020-09-08T16:04:15.019 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012648499999999999 2020-09-08T16:04:15.019 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.019 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.019 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:04:15.020 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.547108+0000", 2020-09-08T16:04:15.020 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0078733060000000001 2020-09-08T16:04:15.020 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.020 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.020 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:15.021 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.554981+0000", 2020-09-08T16:04:15.021 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40108070800000001 2020-09-08T16:04:15.021 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.021 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.021 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:15.022 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.956062+0000", 2020-09-08T16:04:15.022 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.9721000000000006e-05 2020-09-08T16:04:15.022 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.022 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.022 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:15.023 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.956131+0000", 2020-09-08T16:04:15.023 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.7667e-05 2020-09-08T16:04:15.023 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.023 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.023 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:15.024 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.956209+0000", 2020-09-08T16:04:15.024 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40948933999999998 2020-09-08T16:04:15.024 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.024 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:15.025 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.025 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.025 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.025 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.16014.0:166988 6.7 6:e80314fc:::benchmark_data_smithi166_43187_object47219:head [delete] snapc 0=[] ondisk+write+known_if_redirected e522)", 2020-09-08T16:04:15.025 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:58:46.551253+0000", 2020-09-08T16:04:15.026 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 328.28083878199999, 2020-09-08T16:04:15.026 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40513985400000002, 2020-09-08T16:04:15.026 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:15.026 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:15.026 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:15.027 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.16014", 2020-09-08T16:04:15.027 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:04:15.027 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 166988 2020-09-08T16:04:15.027 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.027 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:15.027 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.028 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:15.028 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.551253+0000", 2020-09-08T16:04:15.028 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:15.028 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.028 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.029 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:15.029 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.551253+0000", 2020-09-08T16:04:15.029 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999757 2020-09-08T16:04:15.029 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.029 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.030 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:15.030 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.551229+0000", 2020-09-08T16:04:15.030 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.3801999999999998e-05 2020-09-08T16:04:15.030 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.030 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.030 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:15.031 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.551283+0000", 2020-09-08T16:04:15.031 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1593999999999999e-05 2020-09-08T16:04:15.031 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.031 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.031 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:15.032 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.551294+0000", 2020-09-08T16:04:15.032 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.8080000000000004e-06 2020-09-08T16:04:15.032 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.032 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.032 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:15.033 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.551299+0000", 2020-09-08T16:04:15.033 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019438499999999999 2020-09-08T16:04:15.034 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.034 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.034 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:15.034 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.551493+0000", 2020-09-08T16:04:15.035 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000107388 2020-09-08T16:04:15.035 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.035 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.035 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:15.035 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.551601+0000", 2020-09-08T16:04:15.036 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010147 2020-09-08T16:04:15.036 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.036 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.036 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:04:15.036 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.551702+0000", 2020-09-08T16:04:15.036 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0035337390000000002 2020-09-08T16:04:15.037 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.037 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.037 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:15.037 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.555236+0000", 2020-09-08T16:04:15.037 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40103633300000002 2020-09-08T16:04:15.038 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.038 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.038 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:15.038 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.956272+0000", 2020-09-08T16:04:15.039 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0631000000000003e-05 2020-09-08T16:04:15.039 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.040 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.040 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:15.040 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.956333+0000", 2020-09-08T16:04:15.040 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9793999999999997e-05 2020-09-08T16:04:15.040 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:58:46.956393+0000", 2020-09-08T16:04:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.40513985400000002 2020-09-08T16:04:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.042 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:15.042 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.042 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.042 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.043 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:132310 7.0 7:08a563c3:::benchmark_data_smithi166_48533_object132309:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e582)", 2020-09-08T16:04:15.043 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:02:08.118745+0000", 2020-09-08T16:04:15.043 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 126.71334671300001, 2020-09-08T16:04:15.043 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.32255850899999999, 2020-09-08T16:04:15.044 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:15.044 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:15.044 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:15.045 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:04:15.045 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:04:15.045 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 132310 2020-09-08T16:04:15.045 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.045 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:15.046 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.046 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:15.046 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.118745+0000", 2020-09-08T16:04:15.047 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:15.047 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.048 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.048 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:15.049 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.118745+0000", 2020-09-08T16:04:15.049 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999309 2020-09-08T16:04:15.049 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.049 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.050 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:15.050 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.118676+0000", 2020-09-08T16:04:15.050 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00021206 2020-09-08T16:04:15.050 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.050 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.050 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:15.051 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.118888+0000", 2020-09-08T16:04:15.051 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.5772999999999999e-05 2020-09-08T16:04:15.051 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.051 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.051 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:15.052 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.118914+0000", 2020-09-08T16:04:15.052 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1042e-05 2020-09-08T16:04:15.052 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.052 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.052 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:15.053 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.118925+0000", 2020-09-08T16:04:15.053 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.6842000000000006e-05 2020-09-08T16:04:15.053 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.053 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.053 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:15.054 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.119011+0000", 2020-09-08T16:04:15.054 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000188061 2020-09-08T16:04:15.054 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.054 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.054 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.119199+0000", 2020-09-08T16:04:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00024967400000000002 2020-09-08T16:04:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 9", 2020-09-08T16:04:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.119449+0000", 2020-09-08T16:04:15.056 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.30866220500000002 2020-09-08T16:04:15.056 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.056 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.056 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:15.057 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.428111+0000", 2020-09-08T16:04:15.057 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.013081615 2020-09-08T16:04:15.057 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.057 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.057 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:15.058 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.441193+0000", 2020-09-08T16:04:15.058 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.4659999999999993e-05 2020-09-08T16:04:15.058 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.058 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.058 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:15.059 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.441268+0000", 2020-09-08T16:04:15.059 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.5871999999999998e-05 2020-09-08T16:04:15.059 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.441303+0000", 2020-09-08T16:04:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.32255850899999999 2020-09-08T16:04:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:132314 7.0 7:01256802:::benchmark_data_smithi166_48533_object132313:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e582)", 2020-09-08T16:04:15.062 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:02:08.119704+0000", 2020-09-08T16:04:15.062 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 126.71238757899999, 2020-09-08T16:04:15.062 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.326628003, 2020-09-08T16:04:15.062 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:15.062 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:04:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:04:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 132314 2020-09-08T16:04:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.119704+0000", 2020-09-08T16:04:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.119704+0000", 2020-09-08T16:04:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999328 2020-09-08T16:04:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.066 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.066 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:15.066 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.119637+0000", 2020-09-08T16:04:15.066 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00020777299999999999 2020-09-08T16:04:15.066 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.067 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.067 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:15.067 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.119845+0000", 2020-09-08T16:04:15.067 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.3206000000000001e-05 2020-09-08T16:04:15.067 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.119868+0000", 2020-09-08T16:04:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.696e-06 2020-09-08T16:04:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.069 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:15.069 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.119878+0000", 2020-09-08T16:04:15.070 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.30908945100000002 2020-09-08T16:04:15.070 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.070 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.070 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:15.070 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.428967+0000", 2020-09-08T16:04:15.071 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00044533700000000002 2020-09-08T16:04:15.071 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.071 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.071 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:15.071 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.429413+0000", 2020-09-08T16:04:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000532026 2020-09-08T16:04:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 9", 2020-09-08T16:04:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.429945+0000", 2020-09-08T16:04:15.073 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.011417996999999999 2020-09-08T16:04:15.073 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.073 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.073 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:15.073 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.441363+0000", 2020-09-08T16:04:15.074 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0047973770000000002 2020-09-08T16:04:15.074 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.074 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.074 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:15.075 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.446160+0000", 2020-09-08T16:04:15.075 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010149000000000001 2020-09-08T16:04:15.075 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.075 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.075 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.446261+0000", 2020-09-08T16:04:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.0711999999999997e-05 2020-09-08T16:04:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.446332+0000", 2020-09-08T16:04:15.077 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.326628003 2020-09-08T16:04:15.077 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.077 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:15.077 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.077 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.078 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.078 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:220228 7.0 7:07439957:::benchmark_data_smithi166_48533_object85711:head [delete] snapc 0=[] ondisk+write+known_if_redirected e605)", 2020-09-08T16:04:15.078 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:37.508509+0000", 2020-09-08T16:04:15.078 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 37.323582760999997, 2020-09-08T16:04:15.078 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.050859544, 2020-09-08T16:04:15.079 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:15.079 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:15.079 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:15.079 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:04:15.080 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:04:15.080 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 220228 2020-09-08T16:04:15.080 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.080 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:15.080 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:37.508509+0000", 2020-09-08T16:04:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.082 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:15.082 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:37.508509+0000", 2020-09-08T16:04:15.082 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999747 2020-09-08T16:04:15.082 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.083 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.083 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:15.083 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:37.508484+0000", 2020-09-08T16:04:15.083 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.5319999999999999e-05 2020-09-08T16:04:15.083 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:37.508539+0000", 2020-09-08T16:04:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0749e-05 2020-09-08T16:04:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.085 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:15.085 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:37.508550+0000", 2020-09-08T16:04:15.085 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.6200000000000001e-06 2020-09-08T16:04:15.085 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.085 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.086 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:15.086 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:37.508557+0000", 2020-09-08T16:04:15.086 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.263e-05 2020-09-08T16:04:15.086 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.086 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.087 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:15.087 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:37.508639+0000", 2020-09-08T16:04:15.087 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00022186300000000001 2020-09-08T16:04:15.087 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.087 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.088 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:15.088 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:37.508861+0000", 2020-09-08T16:04:15.088 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00024228800000000001 2020-09-08T16:04:15.088 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.089 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.089 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 9", 2020-09-08T16:04:15.089 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:37.509103+0000", 2020-09-08T16:04:15.089 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0052099650000000004 2020-09-08T16:04:15.089 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.090 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.090 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:15.090 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:37.514313+0000", 2020-09-08T16:04:15.091 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.044855216000000003 2020-09-08T16:04:15.091 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.091 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.091 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:15.091 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:37.559168+0000", 2020-09-08T16:04:15.091 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000116025 2020-09-08T16:04:15.092 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.092 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.092 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:15.092 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:37.559285+0000", 2020-09-08T16:04:15.092 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.3945999999999999e-05 2020-09-08T16:04:15.093 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.093 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.093 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:15.093 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:37.559368+0000", 2020-09-08T16:04:15.093 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.050859544 2020-09-08T16:04:15.094 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.094 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:15.094 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.094 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.095 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.095 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:220752 7.0 7:0f8d1be1:::benchmark_data_smithi166_48533_object86235:head [delete] snapc 0=[] ondisk+write+known_if_redirected e605)", 2020-09-08T16:04:15.095 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:38.031589+0000", 2020-09-08T16:04:15.095 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 36.800503178, 2020-09-08T16:04:15.095 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.118256688, 2020-09-08T16:04:15.095 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:15.096 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:15.096 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:15.096 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:04:15.096 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:04:15.096 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 220752 2020-09-08T16:04:15.097 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.097 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:15.097 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.097 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:15.097 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031589+0000", 2020-09-08T16:04:15.098 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:15.098 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.098 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.098 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:15.098 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031589+0000", 2020-09-08T16:04:15.099 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999728 2020-09-08T16:04:15.099 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.099 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.099 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:15.099 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031561+0000", 2020-09-08T16:04:15.100 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0479000000000002e-05 2020-09-08T16:04:15.100 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.100 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.100 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:15.101 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031622+0000", 2020-09-08T16:04:15.101 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0919e-05 2020-09-08T16:04:15.101 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.101 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.101 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:15.102 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031633+0000", 2020-09-08T16:04:15.102 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4739999999999996e-06 2020-09-08T16:04:15.102 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.102 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.102 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031638+0000", 2020-09-08T16:04:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2747999999999997e-05 2020-09-08T16:04:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:15.104 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031691+0000", 2020-09-08T16:04:15.104 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011123799999999999 2020-09-08T16:04:15.104 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.104 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.104 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031802+0000", 2020-09-08T16:04:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.8571000000000002e-05 2020-09-08T16:04:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 9", 2020-09-08T16:04:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031901+0000", 2020-09-08T16:04:15.106 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.10433806499999999 2020-09-08T16:04:15.106 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.106 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.106 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:15.106 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.136239+0000", 2020-09-08T16:04:15.107 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.013515574000000001 2020-09-08T16:04:15.107 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.107 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.107 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:15.107 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.149754+0000", 2020-09-08T16:04:15.108 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.0717999999999998e-05 2020-09-08T16:04:15.108 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.108 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.108 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:15.108 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.149805+0000", 2020-09-08T16:04:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.0042e-05 2020-09-08T16:04:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.149845+0000", 2020-09-08T16:04:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.118256688 2020-09-08T16:04:15.110 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.110 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:15.110 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.111 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.111 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.111 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:220754 7.0 7:02b4744d:::benchmark_data_smithi166_48533_object86237:head [delete] snapc 0=[] ondisk+write+known_if_redirected e605)", 2020-09-08T16:04:15.111 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:38.031703+0000", 2020-09-08T16:04:15.111 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 36.800388218999998, 2020-09-08T16:04:15.112 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.122364417, 2020-09-08T16:04:15.112 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:15.112 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:15.112 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:15.112 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:04:15.113 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:04:15.113 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 220754 2020-09-08T16:04:15.113 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.113 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:15.113 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.114 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:15.114 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031703+0000", 2020-09-08T16:04:15.115 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:15.115 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.115 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.116 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:15.116 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031703+0000", 2020-09-08T16:04:15.116 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999738 2020-09-08T16:04:15.116 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.117 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.117 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:15.117 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031677+0000", 2020-09-08T16:04:15.118 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.5869000000000003e-05 2020-09-08T16:04:15.118 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.118 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.118 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:15.118 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031733+0000", 2020-09-08T16:04:15.119 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2830000000000001e-06 2020-09-08T16:04:15.119 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.119 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.119 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:15.119 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031741+0000", 2020-09-08T16:04:15.120 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.7479999999999999e-06 2020-09-08T16:04:15.120 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.120 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.120 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:15.120 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031745+0000", 2020-09-08T16:04:15.121 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0043302230000000002 2020-09-08T16:04:15.121 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.121 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.122 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:15.122 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.036075+0000", 2020-09-08T16:04:15.122 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3675e-05 2020-09-08T16:04:15.122 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.123 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.123 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:15.123 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.036169+0000", 2020-09-08T16:04:15.123 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000138276 2020-09-08T16:04:15.123 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.124 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.124 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 9", 2020-09-08T16:04:15.124 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.036307+0000", 2020-09-08T16:04:15.124 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.099943807999999995 2020-09-08T16:04:15.124 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.125 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.125 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:15.125 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.136251+0000", 2020-09-08T16:04:15.125 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.017666213 2020-09-08T16:04:15.126 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.126 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.126 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:15.126 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.153917+0000", 2020-09-08T16:04:15.126 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4060000000000005e-05 2020-09-08T16:04:15.127 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.127 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.127 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:15.127 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.154001+0000", 2020-09-08T16:04:15.128 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.6455e-05 2020-09-08T16:04:15.129 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.129 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.129 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:15.129 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.154068+0000", 2020-09-08T16:04:15.129 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.122364417 2020-09-08T16:04:15.130 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.130 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:15.130 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.130 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.130 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.131 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.36113.0:220758 7.b e605/553 7:dec6751e:::benchmark_data_smithi166_48533_object86241:head v 605'13841, mlcod=605'13838)", 2020-09-08T16:04:15.131 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:38.035434+0000", 2020-09-08T16:04:15.131 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 36.796657795999998, 2020-09-08T16:04:15.131 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.10085925499999999, 2020-09-08T16:04:15.131 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:15.132 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:15.132 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:15.132 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.132 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:15.132 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.035434+0000", 2020-09-08T16:04:15.133 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:15.133 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.133 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.133 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:15.133 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.035434+0000", 2020-09-08T16:04:15.134 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.4913999999999995e-05 2020-09-08T16:04:15.134 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.134 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.134 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:15.134 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.035499+0000", 2020-09-08T16:04:15.135 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2039999999999997e-06 2020-09-08T16:04:15.135 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.135 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.135 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:15.135 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.035507+0000", 2020-09-08T16:04:15.135 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3402e-05 2020-09-08T16:04:15.136 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.136 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.136 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:15.136 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.035520+0000", 2020-09-08T16:04:15.136 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0020000000000003e-06 2020-09-08T16:04:15.137 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.137 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.137 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:15.137 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.035526+0000", 2020-09-08T16:04:15.137 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.6520000000000001e-05 2020-09-08T16:04:15.138 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.138 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.138 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:15.138 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.035583+0000", 2020-09-08T16:04:15.138 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.9415e-05 2020-09-08T16:04:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.035602+0000", 2020-09-08T16:04:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.100610595 2020-09-08T16:04:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.140 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.140 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:15.140 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.136213+0000", 2020-09-08T16:04:15.140 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.0203000000000004e-05 2020-09-08T16:04:15.140 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.141 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.141 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:15.141 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.136293+0000", 2020-09-08T16:04:15.141 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.10085925499999999 2020-09-08T16:04:15.141 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.142 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:15.142 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.142 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.142 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.142 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:240848 7.0 7:0e51330a:::benchmark_data_smithi166_48533_object106331:head [delete] snapc 0=[] ondisk+write+known_if_redirected e611)", 2020-09-08T16:04:15.143 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:04:04.969927+0000", 2020-09-08T16:04:15.143 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 9.8621651450000005, 2020-09-08T16:04:15.143 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.045941708999999997, 2020-09-08T16:04:15.143 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:15.143 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:15.144 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:15.144 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:04:15.144 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:04:15.144 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 240848 2020-09-08T16:04:15.144 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.144 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:15.145 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.145 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:15.145 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:04.969927+0000", 2020-09-08T16:04:15.145 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:15.145 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.146 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.146 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:15.146 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:04.969927+0000", 2020-09-08T16:04:15.146 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999619 2020-09-08T16:04:15.146 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:04.969888+0000", 2020-09-08T16:04:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.3675000000000001e-05 2020-09-08T16:04:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.148 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:15.148 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:04.969972+0000", 2020-09-08T16:04:15.148 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.8842999999999999e-05 2020-09-08T16:04:15.148 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.148 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.149 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:15.149 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:04.969991+0000", 2020-09-08T16:04:15.149 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.8849999999999999e-06 2020-09-08T16:04:15.149 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.149 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.150 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:15.150 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:04.969999+0000", 2020-09-08T16:04:15.150 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.7990000000000005e-05 2020-09-08T16:04:15.150 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.150 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:04.970067+0000", 2020-09-08T16:04:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015292100000000001 2020-09-08T16:04:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:04.970220+0000", 2020-09-08T16:04:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000163938 2020-09-08T16:04:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 9", 2020-09-08T16:04:15.153 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:04.970383+0000", 2020-09-08T16:04:15.153 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0055627130000000004 2020-09-08T16:04:15.153 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.153 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.153 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:04.975946+0000", 2020-09-08T16:04:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.039808122000000001 2020-09-08T16:04:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:05.015754+0000", 2020-09-08T16:04:15.155 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.4641999999999995e-05 2020-09-08T16:04:15.155 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.155 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.155 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:15.155 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:05.015819+0000", 2020-09-08T16:04:15.156 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.9276000000000001e-05 2020-09-08T16:04:15.156 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:15.156 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:15.156 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:15.156 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:04:05.015868+0000", 2020-09-08T16:04:15.157 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.045941708999999997 2020-09-08T16:04:15.157 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.157 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:15.157 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.157 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:15.157 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:15.158 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:04:15.159 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:04:15.160 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:04:14 smithi166 bash[18960]: debug 2020-09-08T16:04:14.903+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:15.160 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:15 smithi166 bash[9446]: cluster 2020-09-08T16:04:14.018139+0000 mgr.x (mgr.24118) 1107 : cluster [DBG] pgmap v1637: 17 pgs: 17 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T16:04:15.160 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:04:14 smithi055 bash[14301]: debug 2020-09-08T16:04:14.686+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:15.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:15 smithi055 bash[9296]: cluster 2020-09-08T16:04:14.018139+0000 mgr.x (mgr.24118) 1107 : cluster [DBG] pgmap v1637: 17 pgs: 17 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T16:04:15.261 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:15.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:15 smithi104 bash[9203]: cluster 2020-09-08T16:04:14.018139+0000 mgr.x (mgr.24118) 1107 : cluster [DBG] pgmap v1637: 17 pgs: 17 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T16:04:15.643 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:04:15.692 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:04:15.726 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:04:15.770 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:04:15.811 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:04:15.848 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:04:15.882 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:04:15.922 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:04:15.954 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:04:15.993 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:04:16.029 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:04:16.061 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:04:16.063 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:04:16.064 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:04:16.064 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:04:16.064 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:04:16.064 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:04:16.065 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:04:16.065 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:04:16.065 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:04:16.065 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:04:16.065 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:04:16.066 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:04:16.066 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:04:16.280 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:16.375 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:16 smithi104 bash[9203]: audit 2020-09-08T16:04:15.595624+0000 mon.a (mon.0) 2001 : audit [DBG] from='client.? v1:172.21.15.104:0/1184424539' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:04:16.384 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:16.395 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:16 smithi055 bash[9296]: audit 2020-09-08T16:04:15.595624+0000 mon.a (mon.0) 2001 : audit [DBG] from='client.? v1:172.21.15.104:0/1184424539' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:04:16.395 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:04:16 smithi055 bash[19003]: debug 2020-09-08T16:04:16.278+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:16.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:16 smithi166 bash[9446]: audit 2020-09-08T16:04:15.595624+0000 mon.a (mon.0) 2001 : audit [DBG] from='client.? v1:172.21.15.104:0/1184424539' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:04:16.650 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:16.713 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2020-09-08T16:04:16.752 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:16.872 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:04:16.887 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:16.906 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:04:16 smithi055 bash[11956]: debug 2020-09-08T16:04:16.882+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:16.906 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:04:16 smithi055 bash[19003]: debug 2020-09-08T16:04:16.646+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:16.990 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:17.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:17 smithi055 bash[9296]: cluster 2020-09-08T16:04:15.567642+0000 osd.7 (osd.7) 133 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:17.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:17 smithi055 bash[9296]: cluster 2020-09-08T16:04:15.567715+0000 osd.7 (osd.7) 134 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:17.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:17 smithi055 bash[9296]: cluster 2020-09-08T16:04:16.018557+0000 mgr.x (mgr.24118) 1108 : cluster [DBG] pgmap v1638: 17 pgs: 17 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 692 op/s 2020-09-08T16:04:17.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:17 smithi055 bash[9296]: cluster 2020-09-08T16:04:16.131337+0000 mon.a (mon.0) 2002 : cluster [DBG] osdmap e615: 12 total, 12 up, 12 in 2020-09-08T16:04:17.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:17 smithi055 bash[9296]: cluster 2020-09-08T16:04:16.636534+0000 osd.0 (osd.0) 106 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:17.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:17 smithi055 bash[9296]: cluster 2020-09-08T16:04:16.636618+0000 osd.0 (osd.0) 107 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:17.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:17 smithi104 bash[9203]: cluster 2020-09-08T16:04:15.567642+0000 osd.7 (osd.7) 133 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:17.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:17 smithi104 bash[9203]: cluster 2020-09-08T16:04:15.567715+0000 osd.7 (osd.7) 134 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:17.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:17 smithi104 bash[9203]: cluster 2020-09-08T16:04:16.018557+0000 mgr.x (mgr.24118) 1108 : cluster [DBG] pgmap v1638: 17 pgs: 17 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 692 op/s 2020-09-08T16:04:17.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:17 smithi104 bash[9203]: cluster 2020-09-08T16:04:16.131337+0000 mon.a (mon.0) 2002 : cluster [DBG] osdmap e615: 12 total, 12 up, 12 in 2020-09-08T16:04:17.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:17 smithi104 bash[9203]: cluster 2020-09-08T16:04:16.636534+0000 osd.0 (osd.0) 106 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:17.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:17 smithi104 bash[9203]: cluster 2020-09-08T16:04:16.636618+0000 osd.0 (osd.0) 107 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:17.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:17 smithi166 bash[9446]: cluster 2020-09-08T16:04:15.567642+0000 osd.7 (osd.7) 133 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:17.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:17 smithi166 bash[9446]: cluster 2020-09-08T16:04:15.567715+0000 osd.7 (osd.7) 134 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:17.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:17 smithi166 bash[9446]: cluster 2020-09-08T16:04:16.018557+0000 mgr.x (mgr.24118) 1108 : cluster [DBG] pgmap v1638: 17 pgs: 17 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 692 op/s 2020-09-08T16:04:17.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:17 smithi166 bash[9446]: cluster 2020-09-08T16:04:16.131337+0000 mon.a (mon.0) 2002 : cluster [DBG] osdmap e615: 12 total, 12 up, 12 in 2020-09-08T16:04:17.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:17 smithi166 bash[9446]: cluster 2020-09-08T16:04:16.636534+0000 osd.0 (osd.0) 106 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:17.620 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:17 smithi166 bash[9446]: cluster 2020-09-08T16:04:16.636618+0000 osd.0 (osd.0) 107 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:17.872 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:17.976 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:04:18.140 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:04:17 smithi055 bash[16646]: debug 2020-09-08T16:04:17.870+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:18.171 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:04:18.276 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:18.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:18 smithi055 bash[9296]: cluster 2020-09-08T16:04:17.138448+0000 mon.a (mon.0) 2003 : cluster [DBG] osdmap e616: 12 total, 12 up, 12 in 2020-09-08T16:04:18.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:04:18 smithi104 bash[19922]: debug 2020-09-08T16:04:18.165+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:18.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:18 smithi104 bash[9203]: cluster 2020-09-08T16:04:17.138448+0000 mon.a (mon.0) 2003 : cluster [DBG] osdmap e616: 12 total, 12 up, 12 in 2020-09-08T16:04:18.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:18 smithi166 bash[9446]: cluster 2020-09-08T16:04:17.138448+0000 mon.a (mon.0) 2003 : cluster [DBG] osdmap e616: 12 total, 12 up, 12 in 2020-09-08T16:04:18.808 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:18.921 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:19.143 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:19 smithi055 bash[9296]: cluster 2020-09-08T16:04:18.019118+0000 mgr.x (mgr.24118) 1109 : cluster [DBG] pgmap v1641: 17 pgs: 17 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.17k op/s 2020-09-08T16:04:19.144 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:19 smithi055 bash[9296]: audit 2020-09-08T16:04:18.819288+0000 mon.b (mon.1) 114 : audit [INF] from='client.? v1:172.21.15.104:0/4080416893' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:04:19.144 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:19 smithi055 bash[9296]: audit 2020-09-08T16:04:18.819612+0000 mon.a (mon.0) 2004 : audit [INF] from='client.26266 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:04:19.144 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:04:18 smithi055 bash[14301]: debug 2020-09-08T16:04:18.806+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:19.161 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T16:04:19.327 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:19 smithi104 bash[9203]: cluster 2020-09-08T16:04:18.019118+0000 mgr.x (mgr.24118) 1109 : cluster [DBG] pgmap v1641: 17 pgs: 17 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.17k op/s 2020-09-08T16:04:19.328 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:19 smithi104 bash[9203]: audit 2020-09-08T16:04:18.819288+0000 mon.b (mon.1) 114 : audit [INF] from='client.? v1:172.21.15.104:0/4080416893' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:04:19.328 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:19 smithi104 bash[9203]: audit 2020-09-08T16:04:18.819612+0000 mon.a (mon.0) 2004 : audit [INF] from='client.26266 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:04:19.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:19 smithi166 bash[9446]: cluster 2020-09-08T16:04:18.019118+0000 mgr.x (mgr.24118) 1109 : cluster [DBG] pgmap v1641: 17 pgs: 17 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.17k op/s 2020-09-08T16:04:19.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:19 smithi166 bash[9446]: audit 2020-09-08T16:04:18.819288+0000 mon.b (mon.1) 114 : audit [INF] from='client.? v1:172.21.15.104:0/4080416893' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:04:19.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:19 smithi166 bash[9446]: audit 2020-09-08T16:04:18.819612+0000 mon.a (mon.0) 2004 : audit [INF] from='client.26266 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:04:19.644 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:04:19.812 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:04:20.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:20 smithi055 bash[9296]: cluster 2020-09-08T16:04:19.146445+0000 mon.a (mon.0) 2005 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:04:20.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:20 smithi055 bash[9296]: audit 2020-09-08T16:04:19.157701+0000 mon.a (mon.0) 2006 : audit [INF] from='client.26266 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:04:20.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:20 smithi055 bash[9296]: cluster 2020-09-08T16:04:19.157803+0000 mon.a (mon.0) 2007 : cluster [DBG] osdmap e617: 12 total, 12 up, 12 in 2020-09-08T16:04:20.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:20 smithi055 bash[9296]: cluster 2020-09-08T16:04:19.419501+0000 osd.0 (osd.0) 108 : cluster [DBG] 7.6 scrub starts 2020-09-08T16:04:20.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:20 smithi055 bash[9296]: cluster 2020-09-08T16:04:19.725971+0000 osd.0 (osd.0) 109 : cluster [DBG] 7.6 scrub ok 2020-09-08T16:04:20.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:20 smithi055 bash[9296]: audit 2020-09-08T16:04:19.977716+0000 mon.a (mon.0) 2008 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:04:20.433 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:20 smithi104 bash[9203]: cluster 2020-09-08T16:04:19.146445+0000 mon.a (mon.0) 2005 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:04:20.433 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:20 smithi104 bash[9203]: audit 2020-09-08T16:04:19.157701+0000 mon.a (mon.0) 2006 : audit [INF] from='client.26266 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:04:20.434 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:20 smithi104 bash[9203]: cluster 2020-09-08T16:04:19.157803+0000 mon.a (mon.0) 2007 : cluster [DBG] osdmap e617: 12 total, 12 up, 12 in 2020-09-08T16:04:20.434 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:20 smithi104 bash[9203]: cluster 2020-09-08T16:04:19.419501+0000 osd.0 (osd.0) 108 : cluster [DBG] 7.6 scrub starts 2020-09-08T16:04:20.434 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:20 smithi104 bash[9203]: cluster 2020-09-08T16:04:19.725971+0000 osd.0 (osd.0) 109 : cluster [DBG] 7.6 scrub ok 2020-09-08T16:04:20.435 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:20 smithi104 bash[9203]: audit 2020-09-08T16:04:19.977716+0000 mon.a (mon.0) 2008 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:04:20.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:20 smithi166 bash[9446]: cluster 2020-09-08T16:04:19.146445+0000 mon.a (mon.0) 2005 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:04:20.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:20 smithi166 bash[9446]: audit 2020-09-08T16:04:19.157701+0000 mon.a (mon.0) 2006 : audit [INF] from='client.26266 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:04:20.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:20 smithi166 bash[9446]: cluster 2020-09-08T16:04:19.157803+0000 mon.a (mon.0) 2007 : cluster [DBG] osdmap e617: 12 total, 12 up, 12 in 2020-09-08T16:04:20.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:20 smithi166 bash[9446]: cluster 2020-09-08T16:04:19.419501+0000 osd.0 (osd.0) 108 : cluster [DBG] 7.6 scrub starts 2020-09-08T16:04:20.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:20 smithi166 bash[9446]: cluster 2020-09-08T16:04:19.725971+0000 osd.0 (osd.0) 109 : cluster [DBG] 7.6 scrub ok 2020-09-08T16:04:20.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:20 smithi166 bash[9446]: audit 2020-09-08T16:04:19.977716+0000 mon.a (mon.0) 2008 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:04:21.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:21 smithi055 bash[9296]: cluster 2020-09-08T16:04:20.019598+0000 mgr.x (mgr.24118) 1110 : cluster [DBG] pgmap v1643: 17 pgs: 17 active+clean; 1007 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 814 op/s 2020-09-08T16:04:21.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:21 smithi055 bash[9296]: audit 2020-09-08T16:04:20.380605+0000 mon.a (mon.0) 2009 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:04:06.700991\"}"}]: dispatch 2020-09-08T16:04:21.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:21 smithi055 bash[9296]: audit 2020-09-08T16:04:20.383978+0000 mon.a (mon.0) 2010 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:04:06.700991\"}"}]': finished 2020-09-08T16:04:21.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:21 smithi104 bash[9203]: cluster 2020-09-08T16:04:20.019598+0000 mgr.x (mgr.24118) 1110 : cluster [DBG] pgmap v1643: 17 pgs: 17 active+clean; 1007 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 814 op/s 2020-09-08T16:04:21.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:21 smithi104 bash[9203]: audit 2020-09-08T16:04:20.380605+0000 mon.a (mon.0) 2009 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:04:06.700991\"}"}]: dispatch 2020-09-08T16:04:21.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:21 smithi104 bash[9203]: audit 2020-09-08T16:04:20.383978+0000 mon.a (mon.0) 2010 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:04:06.700991\"}"}]': finished 2020-09-08T16:04:21.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:21 smithi166 bash[9446]: cluster 2020-09-08T16:04:20.019598+0000 mgr.x (mgr.24118) 1110 : cluster [DBG] pgmap v1643: 17 pgs: 17 active+clean; 1007 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 814 op/s 2020-09-08T16:04:21.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:21 smithi166 bash[9446]: audit 2020-09-08T16:04:20.380605+0000 mon.a (mon.0) 2009 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:04:06.700991\"}"}]: dispatch 2020-09-08T16:04:21.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:21 smithi166 bash[9446]: audit 2020-09-08T16:04:20.383978+0000 mon.a (mon.0) 2010 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:04:06.700991\"}"}]': finished 2020-09-08T16:04:22.161 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:22.281 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:22.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:22 smithi055 bash[9296]: cluster 2020-09-08T16:04:20.473208+0000 osd.7 (osd.7) 135 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:04:22.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:22 smithi055 bash[9296]: cluster 2020-09-08T16:04:20.572887+0000 osd.7 (osd.7) 136 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:04:22.427 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:22 smithi166 bash[9446]: cluster 2020-09-08T16:04:20.473208+0000 osd.7 (osd.7) 135 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:04:22.427 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:22 smithi166 bash[9446]: cluster 2020-09-08T16:04:20.572887+0000 osd.7 (osd.7) 136 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:04:22.428 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:22.497 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:22 smithi104 bash[9203]: cluster 2020-09-08T16:04:20.473208+0000 osd.7 (osd.7) 135 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:04:22.498 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:22 smithi104 bash[9203]: cluster 2020-09-08T16:04:20.572887+0000 osd.7 (osd.7) 136 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:04:22.498 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:04:22 smithi104 bash[15273]: debug 2020-09-08T16:04:22.161+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:22.533 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:22.656 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:22.762 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:22.788 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T16:04:22.789 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:04:22.861 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:22.869 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:04:22 smithi166 bash[14254]: debug 2020-09-08T16:04:22.655+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:22.869 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:04:22 smithi166 bash[16593]: debug 2020-09-08T16:04:22.427+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:22.969 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:04:23.164 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:04:22 smithi055 bash[16646]: debug 2020-09-08T16:04:22.858+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:23.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:23 smithi104 bash[9203]: cluster 2020-09-08T16:04:21.993030+0000 osd.11 (osd.11) 102 : cluster [DBG] 7.5 scrub starts 2020-09-08T16:04:23.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:23 smithi104 bash[9203]: cluster 2020-09-08T16:04:22.020005+0000 mgr.x (mgr.24118) 1111 : cluster [DBG] pgmap v1644: 17 pgs: 17 active+clean; 906 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T16:04:23.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:23 smithi104 bash[9203]: cluster 2020-09-08T16:04:22.158118+0000 osd.11 (osd.11) 103 : cluster [DBG] 7.5 scrub ok 2020-09-08T16:04:23.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:23 smithi104 bash[9203]: cluster 2020-09-08T16:04:22.647404+0000 osd.2 (osd.2) 98 : cluster [DBG] 7.e scrub starts 2020-09-08T16:04:23.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:23 smithi166 bash[9446]: cluster 2020-09-08T16:04:21.993030+0000 osd.11 (osd.11) 102 : cluster [DBG] 7.5 scrub starts 2020-09-08T16:04:23.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:23 smithi166 bash[9446]: cluster 2020-09-08T16:04:22.020005+0000 mgr.x (mgr.24118) 1111 : cluster [DBG] pgmap v1644: 17 pgs: 17 active+clean; 906 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T16:04:23.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:23 smithi166 bash[9446]: cluster 2020-09-08T16:04:22.158118+0000 osd.11 (osd.11) 103 : cluster [DBG] 7.5 scrub ok 2020-09-08T16:04:23.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:23 smithi166 bash[9446]: cluster 2020-09-08T16:04:22.647404+0000 osd.2 (osd.2) 98 : cluster [DBG] 7.e scrub starts 2020-09-08T16:04:23.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:23 smithi055 bash[9296]: cluster 2020-09-08T16:04:21.993030+0000 osd.11 (osd.11) 102 : cluster [DBG] 7.5 scrub starts 2020-09-08T16:04:23.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:23 smithi055 bash[9296]: cluster 2020-09-08T16:04:22.020005+0000 mgr.x (mgr.24118) 1111 : cluster [DBG] pgmap v1644: 17 pgs: 17 active+clean; 906 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T16:04:23.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:23 smithi055 bash[9296]: cluster 2020-09-08T16:04:22.158118+0000 osd.11 (osd.11) 103 : cluster [DBG] 7.5 scrub ok 2020-09-08T16:04:23.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:23 smithi055 bash[9296]: cluster 2020-09-08T16:04:22.647404+0000 osd.2 (osd.2) 98 : cluster [DBG] 7.e scrub starts 2020-09-08T16:04:23.776 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2020-09-08T16:04:23.786 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:04:23.890 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:23.942 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:04:24.025 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:24.131 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:24.168 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:04:23 smithi104 bash[19922]: debug 2020-09-08T16:04:23.785+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:24.169 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:04:24 smithi166 bash[14254]: debug 2020-09-08T16:04:24.027+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:24.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:24 smithi055 bash[9296]: cluster 2020-09-08T16:04:23.008153+0000 osd.2 (osd.2) 99 : cluster [DBG] 7.e scrub ok 2020-09-08T16:04:24.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:24 smithi055 bash[9296]: cluster 2020-09-08T16:04:23.475410+0000 osd.0 (osd.0) 110 : cluster [DBG] 7.f scrub starts 2020-09-08T16:04:24.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:24 smithi055 bash[9296]: cluster 2020-09-08T16:04:23.615526+0000 osd.2 (osd.2) 100 : cluster [DBG] 7.8 scrub starts 2020-09-08T16:04:24.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:24 smithi055 bash[9296]: cluster 2020-09-08T16:04:23.701984+0000 osd.0 (osd.0) 111 : cluster [DBG] 7.f scrub ok 2020-09-08T16:04:24.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:24 smithi055 bash[9296]: cluster 2020-09-08T16:04:23.863620+0000 osd.2 (osd.2) 101 : cluster [DBG] 7.8 scrub ok 2020-09-08T16:04:24.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:24 smithi104 bash[9203]: cluster 2020-09-08T16:04:23.008153+0000 osd.2 (osd.2) 99 : cluster [DBG] 7.e scrub ok 2020-09-08T16:04:24.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:24 smithi104 bash[9203]: cluster 2020-09-08T16:04:23.475410+0000 osd.0 (osd.0) 110 : cluster [DBG] 7.f scrub starts 2020-09-08T16:04:24.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:24 smithi104 bash[9203]: cluster 2020-09-08T16:04:23.615526+0000 osd.2 (osd.2) 100 : cluster [DBG] 7.8 scrub starts 2020-09-08T16:04:24.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:24 smithi104 bash[9203]: cluster 2020-09-08T16:04:23.701984+0000 osd.0 (osd.0) 111 : cluster [DBG] 7.f scrub ok 2020-09-08T16:04:24.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:24 smithi104 bash[9203]: cluster 2020-09-08T16:04:23.863620+0000 osd.2 (osd.2) 101 : cluster [DBG] 7.8 scrub ok 2020-09-08T16:04:24.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:24 smithi166 bash[9446]: cluster 2020-09-08T16:04:23.008153+0000 osd.2 (osd.2) 99 : cluster [DBG] 7.e scrub ok 2020-09-08T16:04:24.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:24 smithi166 bash[9446]: cluster 2020-09-08T16:04:23.475410+0000 osd.0 (osd.0) 110 : cluster [DBG] 7.f scrub starts 2020-09-08T16:04:24.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:24 smithi166 bash[9446]: cluster 2020-09-08T16:04:23.615526+0000 osd.2 (osd.2) 100 : cluster [DBG] 7.8 scrub starts 2020-09-08T16:04:24.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:24 smithi166 bash[9446]: cluster 2020-09-08T16:04:23.701984+0000 osd.0 (osd.0) 111 : cluster [DBG] 7.f scrub ok 2020-09-08T16:04:24.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:24 smithi166 bash[9446]: cluster 2020-09-08T16:04:23.863620+0000 osd.2 (osd.2) 101 : cluster [DBG] 7.8 scrub ok 2020-09-08T16:04:24.618 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:24.724 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:24.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:04:24 smithi055 bash[16646]: debug 2020-09-08T16:04:24.622+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:24.935 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:25.041 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:04:25.176 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:04:25.281 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:25.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:25 smithi166 bash[9446]: cluster 2020-09-08T16:04:24.020472+0000 mgr.x (mgr.24118) 1112 : cluster [DBG] pgmap v1645: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 702 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T16:04:25.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:04:25 smithi166 bash[11928]: debug 2020-09-08T16:04:25.175+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:25.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:04:24 smithi104 bash[22305]: debug 2020-09-08T16:04:24.933+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:25.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:25 smithi104 bash[9203]: cluster 2020-09-08T16:04:24.020472+0000 mgr.x (mgr.24118) 1112 : cluster [DBG] pgmap v1645: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 702 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T16:04:25.460 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:25 smithi055 bash[9296]: cluster 2020-09-08T16:04:24.020472+0000 mgr.x (mgr.24118) 1112 : cluster [DBG] pgmap v1645: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 702 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T16:04:25.460 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:25.564 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:25.686 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:25.742 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T16:04:25.792 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:25.800 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:04:25 smithi055 bash[19003]: debug 2020-09-08T16:04:25.690+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:25.801 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:04:25 smithi055 bash[16646]: debug 2020-09-08T16:04:25.462+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:25.901 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:04:25.963 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:26.066 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:26.170 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:04:25 smithi055 bash[14301]: debug 2020-09-08T16:04:25.954+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:26.199 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:26.303 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:26.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:04:26 smithi166 bash[14254]: debug 2020-09-08T16:04:26.199+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:26.800 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:26.903 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:04:27.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:04:26 smithi055 bash[19003]: debug 2020-09-08T16:04:26.802+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:27.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:27 smithi104 bash[9203]: cluster 2020-09-08T16:04:26.020976+0000 mgr.x (mgr.24118) 1113 : cluster [DBG] pgmap v1646: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 702 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 932 op/s 2020-09-08T16:04:27.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:27 smithi166 bash[9446]: cluster 2020-09-08T16:04:26.020976+0000 mgr.x (mgr.24118) 1113 : cluster [DBG] pgmap v1646: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 702 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 932 op/s 2020-09-08T16:04:27.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:27 smithi055 bash[9296]: cluster 2020-09-08T16:04:26.020976+0000 mgr.x (mgr.24118) 1113 : cluster [DBG] pgmap v1646: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 702 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 932 op/s 2020-09-08T16:04:27.860 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:04:27.964 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:28.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:04:27 smithi166 bash[11928]: debug 2020-09-08T16:04:27.859+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:28.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:28 smithi104 bash[9203]: audit 2020-09-08T16:04:27.745690+0000 mon.a (mon.0) 2011 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:04:28.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:28 smithi104 bash[9203]: cluster 2020-09-08T16:04:27.752536+0000 osd.5 (osd.5) 101 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:28.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:28 smithi104 bash[9203]: cluster 2020-09-08T16:04:27.752623+0000 osd.5 (osd.5) 102 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:28.476 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:28 smithi055 bash[9296]: audit 2020-09-08T16:04:27.745690+0000 mon.a (mon.0) 2011 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:04:28.476 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:28 smithi055 bash[9296]: cluster 2020-09-08T16:04:27.752536+0000 osd.5 (osd.5) 101 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:28.476 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:28 smithi055 bash[9296]: cluster 2020-09-08T16:04:27.752623+0000 osd.5 (osd.5) 102 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:28.590 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:28.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:28 smithi166 bash[9446]: audit 2020-09-08T16:04:27.745690+0000 mon.a (mon.0) 2011 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:04:28.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:28 smithi166 bash[9446]: cluster 2020-09-08T16:04:27.752536+0000 osd.5 (osd.5) 101 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:28.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:28 smithi166 bash[9446]: cluster 2020-09-08T16:04:27.752623+0000 osd.5 (osd.5) 102 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:28.697 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:28.723 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T16:04:28.723 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:04:28.815 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:28.919 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:04:28.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:04:28 smithi104 bash[17588]: debug 2020-09-08T16:04:28.589+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:29.044 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:04:29.150 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:29.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:04:28 smithi055 bash[16646]: debug 2020-09-08T16:04:28.814+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:29.188 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:04:29 smithi166 bash[18960]: debug 2020-09-08T16:04:29.043+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:29.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:29 smithi166 bash[9446]: cluster 2020-09-08T16:04:28.021443+0000 mgr.x (mgr.24118) 1114 : cluster [DBG] pgmap v1647: 17 pgs: 17 active+clean; 444 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 962 op/s 2020-09-08T16:04:29.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:29 smithi166 bash[9446]: cluster 2020-09-08T16:04:28.193447+0000 mon.a (mon.0) 2012 : cluster [DBG] osdmap e618: 12 total, 12 up, 12 in 2020-09-08T16:04:29.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:29 smithi166 bash[9446]: cluster 2020-09-08T16:04:28.381931+0000 osd.10 (osd.10) 80 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:29.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:29 smithi166 bash[9446]: cluster 2020-09-08T16:04:28.382021+0000 osd.10 (osd.10) 81 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:29.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:29 smithi166 bash[9446]: audit 2020-09-08T16:04:28.421885+0000 mon.a (mon.0) 2013 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:04:06.828755\"}"}]: dispatch 2020-09-08T16:04:29.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:29 smithi166 bash[9446]: audit 2020-09-08T16:04:28.425212+0000 mon.a (mon.0) 2014 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:04:06.828755\"}"}]': finished 2020-09-08T16:04:29.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:29 smithi166 bash[9446]: audit 2020-09-08T16:04:28.427053+0000 mon.a (mon.0) 2015 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:04:29.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:29 smithi166 bash[9446]: audit 2020-09-08T16:04:28.440982+0000 mon.a (mon.0) 2016 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:04:29.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:29 smithi055 bash[9296]: cluster 2020-09-08T16:04:28.021443+0000 mgr.x (mgr.24118) 1114 : cluster [DBG] pgmap v1647: 17 pgs: 17 active+clean; 444 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 962 op/s 2020-09-08T16:04:29.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:29 smithi055 bash[9296]: cluster 2020-09-08T16:04:28.193447+0000 mon.a (mon.0) 2012 : cluster [DBG] osdmap e618: 12 total, 12 up, 12 in 2020-09-08T16:04:29.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:29 smithi055 bash[9296]: cluster 2020-09-08T16:04:28.381931+0000 osd.10 (osd.10) 80 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:29.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:29 smithi055 bash[9296]: cluster 2020-09-08T16:04:28.382021+0000 osd.10 (osd.10) 81 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:29.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:29 smithi055 bash[9296]: audit 2020-09-08T16:04:28.421885+0000 mon.a (mon.0) 2013 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:04:06.828755\"}"}]: dispatch 2020-09-08T16:04:29.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:29 smithi055 bash[9296]: audit 2020-09-08T16:04:28.425212+0000 mon.a (mon.0) 2014 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:04:06.828755\"}"}]': finished 2020-09-08T16:04:29.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:29 smithi055 bash[9296]: audit 2020-09-08T16:04:28.427053+0000 mon.a (mon.0) 2015 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:04:29.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:29 smithi055 bash[9296]: audit 2020-09-08T16:04:28.440982+0000 mon.a (mon.0) 2016 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:04:29.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:29 smithi104 bash[9203]: cluster 2020-09-08T16:04:28.021443+0000 mgr.x (mgr.24118) 1114 : cluster [DBG] pgmap v1647: 17 pgs: 17 active+clean; 444 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 962 op/s 2020-09-08T16:04:29.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:29 smithi104 bash[9203]: cluster 2020-09-08T16:04:28.193447+0000 mon.a (mon.0) 2012 : cluster [DBG] osdmap e618: 12 total, 12 up, 12 in 2020-09-08T16:04:29.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:29 smithi104 bash[9203]: cluster 2020-09-08T16:04:28.381931+0000 osd.10 (osd.10) 80 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:29.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:29 smithi104 bash[9203]: cluster 2020-09-08T16:04:28.382021+0000 osd.10 (osd.10) 81 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:29.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:29 smithi104 bash[9203]: audit 2020-09-08T16:04:28.421885+0000 mon.a (mon.0) 2013 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:04:06.828755\"}"}]: dispatch 2020-09-08T16:04:29.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:29 smithi104 bash[9203]: audit 2020-09-08T16:04:28.425212+0000 mon.a (mon.0) 2014 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:04:06.828755\"}"}]': finished 2020-09-08T16:04:29.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:29 smithi104 bash[9203]: audit 2020-09-08T16:04:28.427053+0000 mon.a (mon.0) 2015 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:04:29.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:29 smithi104 bash[9203]: audit 2020-09-08T16:04:28.440982+0000 mon.a (mon.0) 2016 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:04:29.680 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:29.791 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:29.813 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2020-09-08T16:04:29.915 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:30.013 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:04:30.022 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:30.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:04:29 smithi055 bash[11956]: debug 2020-09-08T16:04:29.682+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:30.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:04:29 smithi055 bash[14301]: debug 2020-09-08T16:04:29.914+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:30.158 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:30.260 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:30.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:30 smithi166 bash[9446]: cluster 2020-09-08T16:04:29.203953+0000 mon.a (mon.0) 2017 : cluster [DBG] osdmap e619: 12 total, 12 up, 12 in 2020-09-08T16:04:30.367 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:04:30 smithi166 bash[16593]: debug 2020-09-08T16:04:30.159+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:30.382 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:30.485 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:04:30.550 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:30 smithi104 bash[9203]: cluster 2020-09-08T16:04:29.203953+0000 mon.a (mon.0) 2017 : cluster [DBG] osdmap e619: 12 total, 12 up, 12 in 2020-09-08T16:04:30.623 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:04:30.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:30 smithi055 bash[9296]: cluster 2020-09-08T16:04:29.203953+0000 mon.a (mon.0) 2017 : cluster [DBG] osdmap e619: 12 total, 12 up, 12 in 2020-09-08T16:04:30.727 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:04:30.736 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:04:30 smithi166 bash[11928]: debug 2020-09-08T16:04:30.619+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:30.736 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:04:30 smithi166 bash[16593]: debug 2020-09-08T16:04:30.379+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:30.836 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:04:30.942 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:31.102 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:04:31.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:04:30 smithi166 bash[11928]: debug 2020-09-08T16:04:30.839+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:31.142 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:04:31.190 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:04:31.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:31 smithi055 bash[9296]: cluster 2020-09-08T16:04:30.021859+0000 mgr.x (mgr.24118) 1115 : cluster [DBG] pgmap v1650: 17 pgs: 17 active+clean; 406 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 999 op/s 2020-09-08T16:04:31.202 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:31 smithi055 bash[9296]: audit 2020-09-08T16:04:31.067764+0000 mon.a (mon.0) 2018 : audit [DBG] from='client.? v1:172.21.15.104:0/3425767496' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:04:31.232 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:04:31.262 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:04:31.304 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:04:31.329 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:31.342 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:04:31.378 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:04:31.414 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:04:31.442 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:31.452 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:04:31.491 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:04:31.530 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:04:31.542 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:04:31.542 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:04:31.542 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:04:31.543 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:04:31.543 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:04:31.543 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:04:31.543 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:04:31.543 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:04:31.544 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:04:31.544 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:04:31.544 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:04:31.544 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:04:31.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:31 smithi166 bash[9446]: cluster 2020-09-08T16:04:30.021859+0000 mgr.x (mgr.24118) 1115 : cluster [DBG] pgmap v1650: 17 pgs: 17 active+clean; 406 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 999 op/s 2020-09-08T16:04:31.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:31 smithi166 bash[9446]: audit 2020-09-08T16:04:31.067764+0000 mon.a (mon.0) 2018 : audit [DBG] from='client.? v1:172.21.15.104:0/3425767496' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:04:31.628 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:04:31 smithi055 bash[16646]: debug 2020-09-08T16:04:31.334+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:31.629 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:31.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:31 smithi104 bash[9203]: cluster 2020-09-08T16:04:30.021859+0000 mgr.x (mgr.24118) 1115 : cluster [DBG] pgmap v1650: 17 pgs: 17 active+clean; 406 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 999 op/s 2020-09-08T16:04:31.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:31 smithi104 bash[9203]: audit 2020-09-08T16:04:31.067764+0000 mon.a (mon.0) 2018 : audit [DBG] from='client.? v1:172.21.15.104:0/3425767496' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:04:31.734 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:31.867 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:31.906 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:04:31 smithi055 bash[19003]: debug 2020-09-08T16:04:31.634+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:31.971 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:32.084 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:32.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:04:31 smithi166 bash[16593]: debug 2020-09-08T16:04:31.867+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:32.188 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:04:32.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:32 smithi055 bash[9296]: cluster 2020-09-08T16:04:31.816090+0000 osd.3 (osd.3) 109 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:32.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:32 smithi055 bash[9296]: cluster 2020-09-08T16:04:31.816160+0000 osd.3 (osd.3) 110 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:32.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:04:32 smithi055 bash[16646]: debug 2020-09-08T16:04:32.086+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:32.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:32 smithi166 bash[9446]: cluster 2020-09-08T16:04:31.816090+0000 osd.3 (osd.3) 109 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:32.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:32 smithi166 bash[9446]: cluster 2020-09-08T16:04:31.816160+0000 osd.3 (osd.3) 110 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:32.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:32 smithi104 bash[9203]: cluster 2020-09-08T16:04:31.816090+0000 osd.3 (osd.3) 109 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:32.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:32 smithi104 bash[9203]: cluster 2020-09-08T16:04:31.816160+0000 osd.3 (osd.3) 110 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:33.623 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:33 smithi166 bash[9446]: cluster 2020-09-08T16:04:32.022245+0000 mgr.x (mgr.24118) 1116 : cluster [DBG] pgmap v1651: 17 pgs: 17 active+clean; 315 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 772 op/s 2020-09-08T16:04:33.623 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:33 smithi166 bash[9446]: cluster 2020-09-08T16:04:32.217766+0000 mon.a (mon.0) 2019 : cluster [DBG] osdmap e620: 12 total, 12 up, 12 in 2020-09-08T16:04:33.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:33 smithi055 bash[9296]: cluster 2020-09-08T16:04:32.022245+0000 mgr.x (mgr.24118) 1116 : cluster [DBG] pgmap v1651: 17 pgs: 17 active+clean; 315 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 772 op/s 2020-09-08T16:04:33.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:33 smithi055 bash[9296]: cluster 2020-09-08T16:04:32.217766+0000 mon.a (mon.0) 2019 : cluster [DBG] osdmap e620: 12 total, 12 up, 12 in 2020-09-08T16:04:33.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:33 smithi104 bash[9203]: cluster 2020-09-08T16:04:32.022245+0000 mgr.x (mgr.24118) 1116 : cluster [DBG] pgmap v1651: 17 pgs: 17 active+clean; 315 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 772 op/s 2020-09-08T16:04:33.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:33 smithi104 bash[9203]: cluster 2020-09-08T16:04:32.217766+0000 mon.a (mon.0) 2019 : cluster [DBG] osdmap e620: 12 total, 12 up, 12 in 2020-09-08T16:04:34.135 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:04:34.172 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Removed 134514 objects 2020-09-08T16:04:34.172 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Clean up completed and total clean up time :142.744 2020-09-08T16:04:34.228 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T16:04:34.240 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:34.415 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:34.520 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:34.532 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:34 smithi055 bash[9296]: cluster 2020-09-08T16:04:33.536465+0000 osd.0 (osd.0) 112 : cluster [DBG] 7.4 scrub starts 2020-09-08T16:04:34.533 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:34 smithi055 bash[9296]: cluster 2020-09-08T16:04:33.701763+0000 osd.0 (osd.0) 113 : cluster [DBG] 7.4 scrub ok 2020-09-08T16:04:34.533 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:34 smithi055 bash[9296]: audit 2020-09-08T16:04:33.798523+0000 mon.a (mon.0) 2020 : audit [INF] from='client.? v1:172.21.15.104:0/1602183609' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:04:34.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:04:34 smithi166 bash[11928]: debug 2020-09-08T16:04:34.135+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:34.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:34 smithi166 bash[9446]: cluster 2020-09-08T16:04:33.536465+0000 osd.0 (osd.0) 112 : cluster [DBG] 7.4 scrub starts 2020-09-08T16:04:34.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:34 smithi166 bash[9446]: cluster 2020-09-08T16:04:33.701763+0000 osd.0 (osd.0) 113 : cluster [DBG] 7.4 scrub ok 2020-09-08T16:04:34.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:34 smithi166 bash[9446]: audit 2020-09-08T16:04:33.798523+0000 mon.a (mon.0) 2020 : audit [INF] from='client.? v1:172.21.15.104:0/1602183609' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:04:34.633 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:04:34.634 INFO:teuthology.orchestra.run.smithi055.stdout: "size": 20, 2020-09-08T16:04:34.634 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 600, 2020-09-08T16:04:34.634 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T16:04:34.635 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.635 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.16014.0:78097 6.a e499/478 6:50adb852:::benchmark_data_smithi166_43187_object78096:head v 499'4954, mlcod=499'4950)", 2020-09-08T16:04:34.635 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:57:22.621506+0000", 2020-09-08T16:04:34.635 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 432.01362984100001, 2020-09-08T16:04:34.635 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.37386508499999999, 2020-09-08T16:04:34.636 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:34.636 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:34.636 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:34.637 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.637 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:34.637 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.621506+0000", 2020-09-08T16:04:34.637 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:34.637 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.638 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.638 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:34.638 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.621506+0000", 2020-09-08T16:04:34.638 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.7595000000000006e-05 2020-09-08T16:04:34.639 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.639 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.640 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:34.640 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.621574+0000", 2020-09-08T16:04:34.641 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.1009999999999997e-06 2020-09-08T16:04:34.641 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.641 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.642 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:34.642 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.621580+0000", 2020-09-08T16:04:34.642 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.1500000000000005e-06 2020-09-08T16:04:34.642 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.643 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.643 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:34.643 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.621589+0000", 2020-09-08T16:04:34.643 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.8459999999999999e-06 2020-09-08T16:04:34.643 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.643 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.644 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:34.644 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.621594+0000", 2020-09-08T16:04:34.644 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.6891999999999998e-05 2020-09-08T16:04:34.644 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.645 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.645 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:34.645 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.621641+0000", 2020-09-08T16:04:34.646 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.5259999999999999e-05 2020-09-08T16:04:34.646 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.646 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.647 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:34.647 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.621666+0000", 2020-09-08T16:04:34.647 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.37362874299999999 2020-09-08T16:04:34.647 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.648 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.648 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:34.648 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.995295+0000", 2020-09-08T16:04:34.648 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.6497999999999998e-05 2020-09-08T16:04:34.648 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.649 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.649 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:34.649 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.995371+0000", 2020-09-08T16:04:34.649 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.37386508499999999 2020-09-08T16:04:34.649 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.650 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.650 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.650 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.650 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.650 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.16014.0:78105 6.a e499/478 6:577ceeae:::benchmark_data_smithi166_43187_object78104:head v 499'4955, mlcod=499'4952)", 2020-09-08T16:04:34.651 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:57:22.623459+0000", 2020-09-08T16:04:34.651 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 432.01167630999998, 2020-09-08T16:04:34.651 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.37197326200000003, 2020-09-08T16:04:34.651 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:34.651 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:34.652 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:34.652 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.652 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:34.652 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.623459+0000", 2020-09-08T16:04:34.652 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:34.653 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.653 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.653 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:34.653 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.623459+0000", 2020-09-08T16:04:34.653 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3177000000000006e-05 2020-09-08T16:04:34.654 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.654 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.654 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:34.654 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.623553+0000", 2020-09-08T16:04:34.654 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2609999999999993e-06 2020-09-08T16:04:34.655 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.655 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.655 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:34.655 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.623561+0000", 2020-09-08T16:04:34.656 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4061e-05 2020-09-08T16:04:34.656 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.656 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.656 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:34.657 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.623575+0000", 2020-09-08T16:04:34.657 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.1009999999999997e-06 2020-09-08T16:04:34.657 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.657 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.658 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:34.658 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.623581+0000", 2020-09-08T16:04:34.658 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.36193794000000001 2020-09-08T16:04:34.658 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.658 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.659 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:34.659 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.985519+0000", 2020-09-08T16:04:34.659 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.0653000000000003e-05 2020-09-08T16:04:34.659 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.660 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.660 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:34.660 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.985590+0000", 2020-09-08T16:04:34.660 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0097909669999999994 2020-09-08T16:04:34.660 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.661 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.661 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:34.661 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.995381+0000", 2020-09-08T16:04:34.662 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2101999999999997e-05 2020-09-08T16:04:34.662 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.662 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.662 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:34.662 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.995433+0000", 2020-09-08T16:04:34.663 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.37197326200000003 2020-09-08T16:04:34.663 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.663 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.663 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.663 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.664 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.664 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.16014.0:78112 6.3 e499/478 6:c8197f62:::benchmark_data_smithi166_43187_object78111:head v 499'4923, mlcod=499'4921)", 2020-09-08T16:04:34.664 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:57:22.623947+0000", 2020-09-08T16:04:34.664 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 432.011188737, 2020-09-08T16:04:34.664 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.36161531400000002, 2020-09-08T16:04:34.665 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:34.665 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:34.665 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:34.666 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.666 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:34.666 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.623947+0000", 2020-09-08T16:04:34.666 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:34.666 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.667 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.667 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:34.667 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.623947+0000", 2020-09-08T16:04:34.668 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.5593000000000005e-05 2020-09-08T16:04:34.668 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.668 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.668 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:34.668 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.624033+0000", 2020-09-08T16:04:34.669 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.1769999999999998e-06 2020-09-08T16:04:34.669 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.669 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.669 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:34.669 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.624041+0000", 2020-09-08T16:04:34.670 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3356999999999999e-05 2020-09-08T16:04:34.670 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.670 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.670 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:34.670 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.624054+0000", 2020-09-08T16:04:34.671 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9630000000000003e-06 2020-09-08T16:04:34.672 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.672 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.673 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:34.673 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.624060+0000", 2020-09-08T16:04:34.673 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4258999999999997e-05 2020-09-08T16:04:34.673 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.674 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.674 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:34.674 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.624114+0000", 2020-09-08T16:04:34.674 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.2595e-05 2020-09-08T16:04:34.674 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.675 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.675 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:34.675 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.624137+0000", 2020-09-08T16:04:34.675 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0030363 2020-09-08T16:04:34.675 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.676 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.676 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:34.676 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.627173+0000", 2020-09-08T16:04:34.677 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.35838906999999998 2020-09-08T16:04:34.677 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.677 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.678 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:34.678 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.985562+0000", 2020-09-08T16:04:34.678 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.36161531400000002 2020-09-08T16:04:34.678 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.678 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.678 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.679 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.679 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.679 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.16014.0:78107 6.a e499/478 6:5e7bb68a:::benchmark_data_smithi166_43187_object78106:head v 499'4956, mlcod=499'4952)", 2020-09-08T16:04:34.679 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T15:57:22.626831+0000", 2020-09-08T16:04:34.679 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 432.00830444500002, 2020-09-08T16:04:34.680 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.372686605, 2020-09-08T16:04:34.680 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:34.680 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:34.680 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:34.680 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.681 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:34.681 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.626831+0000", 2020-09-08T16:04:34.681 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:34.681 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.681 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.682 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:34.682 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.626831+0000", 2020-09-08T16:04:34.682 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.35882424000000002 2020-09-08T16:04:34.682 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.683 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.683 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:34.683 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.985656+0000", 2020-09-08T16:04:34.683 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.5836000000000001e-05 2020-09-08T16:04:34.684 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.684 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.684 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:34.684 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.985681+0000", 2020-09-08T16:04:34.684 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.349e-05 2020-09-08T16:04:34.685 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.685 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.685 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:34.686 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.985715+0000", 2020-09-08T16:04:34.686 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2362e-05 2020-09-08T16:04:34.686 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.686 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.687 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:34.687 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.985727+0000", 2020-09-08T16:04:34.687 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0068859919999999996 2020-09-08T16:04:34.688 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.688 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.688 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:34.688 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.992613+0000", 2020-09-08T16:04:34.688 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.9986999999999999e-05 2020-09-08T16:04:34.689 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.689 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.689 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:34.690 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.992643+0000", 2020-09-08T16:04:34.690 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0067812150000000002 2020-09-08T16:04:34.691 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.691 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.691 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:34.691 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.999424+0000", 2020-09-08T16:04:34.691 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3482999999999999e-05 2020-09-08T16:04:34.692 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.692 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.692 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:34.692 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T15:57:22.999518+0000", 2020-09-08T16:04:34.692 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.372686605 2020-09-08T16:04:34.693 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.693 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.693 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.693 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.693 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.694 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:43069 7.c 7:3d43a1a7:::benchmark_data_smithi166_48533_object43068:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e561)", 2020-09-08T16:04:34.694 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:00:56.148019+0000", 2020-09-08T16:04:34.694 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 218.48711682000001, 2020-09-08T16:04:34.694 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.26833472600000002, 2020-09-08T16:04:34.694 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:34.695 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:34.695 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:34.695 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:04:34.695 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:04:34.695 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 43069 2020-09-08T16:04:34.696 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.696 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:34.696 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.696 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:34.697 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.148019+0000", 2020-09-08T16:04:34.697 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:34.699 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.699 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.700 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:34.700 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.148019+0000", 2020-09-08T16:04:34.700 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999576 2020-09-08T16:04:34.700 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.700 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.701 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:34.701 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.147977+0000", 2020-09-08T16:04:34.701 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016601499999999999 2020-09-08T16:04:34.701 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.701 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.702 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:34.702 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.148143+0000", 2020-09-08T16:04:34.702 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4425e-05 2020-09-08T16:04:34.702 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.702 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.703 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:34.703 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.148157+0000", 2020-09-08T16:04:34.703 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3500000000000002e-06 2020-09-08T16:04:34.703 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.703 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.704 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:34.704 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.148163+0000", 2020-09-08T16:04:34.704 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.2063999999999994e-05 2020-09-08T16:04:34.704 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.704 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.705 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:34.705 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.148225+0000", 2020-09-08T16:04:34.705 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000158763 2020-09-08T16:04:34.705 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.705 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.706 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:34.706 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.148384+0000", 2020-09-08T16:04:34.706 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00018265100000000001 2020-09-08T16:04:34.706 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.707 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.707 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:04:34.707 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.148567+0000", 2020-09-08T16:04:34.708 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.001758176 2020-09-08T16:04:34.708 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.708 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.709 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:34.709 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.150325+0000", 2020-09-08T16:04:34.709 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.26582709399999999 2020-09-08T16:04:34.709 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.710 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.710 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:34.710 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.416152+0000", 2020-09-08T16:04:34.710 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000104687 2020-09-08T16:04:34.711 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.711 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.711 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:34.711 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.416257+0000", 2020-09-08T16:04:34.712 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.6906999999999999e-05 2020-09-08T16:04:34.712 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.712 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.713 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:34.713 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.416354+0000", 2020-09-08T16:04:34.713 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.26833472600000002 2020-09-08T16:04:34.713 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.713 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.714 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.714 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.714 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.714 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:111353 7.c 7:399953f5:::benchmark_data_smithi166_48533_object111352:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e575)", 2020-09-08T16:04:34.715 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:01:40.772336+0000", 2020-09-08T16:04:34.715 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 173.86280013699999, 2020-09-08T16:04:34.715 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.24925328599999999, 2020-09-08T16:04:34.715 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:34.715 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:34.716 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:34.716 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:04:34.716 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:04:34.716 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 111353 2020-09-08T16:04:34.716 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.717 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:34.717 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.717 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:34.718 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.772336+0000", 2020-09-08T16:04:34.718 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:34.718 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.718 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.719 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:34.719 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.772336+0000", 2020-09-08T16:04:34.719 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999294 2020-09-08T16:04:34.719 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.719 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.720 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:34.720 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.772265+0000", 2020-09-08T16:04:34.720 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000232969 2020-09-08T16:04:34.720 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.720 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.721 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:34.721 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.772498+0000", 2020-09-08T16:04:34.721 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.0729999999999999e-05 2020-09-08T16:04:34.721 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.721 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.722 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:34.722 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.772529+0000", 2020-09-08T16:04:34.722 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3115999999999999e-05 2020-09-08T16:04:34.722 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.722 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.723 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:34.723 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.772542+0000", 2020-09-08T16:04:34.723 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000455926 2020-09-08T16:04:34.723 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.723 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.724 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:34.724 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.772998+0000", 2020-09-08T16:04:34.724 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00067665900000000003 2020-09-08T16:04:34.724 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.725 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.725 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:34.725 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.773675+0000", 2020-09-08T16:04:34.725 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016333799999999999 2020-09-08T16:04:34.728 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:04:34 smithi104 bash[22305]: debug 2020-09-08T16:04:34.417+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:34.728 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:34 smithi104 bash[9203]: cluster 2020-09-08T16:04:33.536465+0000 osd.0 (osd.0) 112 : cluster [DBG] 7.4 scrub starts 2020-09-08T16:04:34.728 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:34 smithi104 bash[9203]: cluster 2020-09-08T16:04:33.701763+0000 osd.0 (osd.0) 113 : cluster [DBG] 7.4 scrub ok 2020-09-08T16:04:34.729 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:34 smithi104 bash[9203]: audit 2020-09-08T16:04:33.798523+0000 mon.a (mon.0) 2020 : audit [INF] from='client.? v1:172.21.15.104:0/1602183609' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:04:34.729 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.729 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.729 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:04:34.730 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.773838+0000", 2020-09-08T16:04:34.730 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0042308370000000003 2020-09-08T16:04:34.730 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.730 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.730 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:34.731 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.778069+0000", 2020-09-08T16:04:34.731 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.24343125800000001 2020-09-08T16:04:34.731 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.731 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.731 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:34.732 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:41.021500+0000", 2020-09-08T16:04:34.732 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2710000000000002e-05 2020-09-08T16:04:34.732 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.732 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.732 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:34.733 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:41.021553+0000", 2020-09-08T16:04:34.733 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.6216e-05 2020-09-08T16:04:34.733 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.733 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.733 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:34.734 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:41.021589+0000", 2020-09-08T16:04:34.734 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.24925328599999999 2020-09-08T16:04:34.734 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.734 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.734 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.735 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.735 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.735 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:111366 7.c 7:3b1f0ab3:::benchmark_data_smithi166_48533_object111365:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e575)", 2020-09-08T16:04:34.735 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:01:40.784524+0000", 2020-09-08T16:04:34.735 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 173.85061203699999, 2020-09-08T16:04:34.735 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.237212543, 2020-09-08T16:04:34.736 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:34.736 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:34.736 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:34.736 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:04:34.736 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:04:34.737 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 111366 2020-09-08T16:04:34.737 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.737 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:34.737 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.737 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:34.738 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.784524+0000", 2020-09-08T16:04:34.738 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:34.738 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.740 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.740 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:34.741 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.784524+0000", 2020-09-08T16:04:34.741 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999747 2020-09-08T16:04:34.742 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.742 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.742 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:34.742 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.784498+0000", 2020-09-08T16:04:34.743 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.2670000000000006e-05 2020-09-08T16:04:34.743 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.743 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.743 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:34.744 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.784591+0000", 2020-09-08T16:04:34.744 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1248e-05 2020-09-08T16:04:34.744 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.744 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.744 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:34.745 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.784602+0000", 2020-09-08T16:04:34.745 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.4279999999999998e-06 2020-09-08T16:04:34.745 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.745 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.745 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:34.746 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.784607+0000", 2020-09-08T16:04:34.746 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.7509000000000006e-05 2020-09-08T16:04:34.746 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.746 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.746 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:34.747 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.784684+0000", 2020-09-08T16:04:34.747 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000172492 2020-09-08T16:04:34.747 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.747 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.748 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:34.748 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.784857+0000", 2020-09-08T16:04:34.748 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000193253 2020-09-08T16:04:34.749 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.749 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.749 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:04:34.750 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.785050+0000", 2020-09-08T16:04:34.750 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.003769166 2020-09-08T16:04:34.750 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.750 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.750 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:34.751 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.788819+0000", 2020-09-08T16:04:34.751 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.23284532299999999 2020-09-08T16:04:34.751 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.751 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.751 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:34.752 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:41.021664+0000", 2020-09-08T16:04:34.752 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.189e-05 2020-09-08T16:04:34.752 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.752 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.752 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:34.753 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:41.021706+0000", 2020-09-08T16:04:34.753 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.9927e-05 2020-09-08T16:04:34.753 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.753 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.753 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:34.754 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:41.021736+0000", 2020-09-08T16:04:34.754 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.237212543 2020-09-08T16:04:34.754 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.754 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.754 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.755 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.755 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.755 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:132325 7.d 7:bd65781f:::benchmark_data_smithi166_48533_object132324:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e582)", 2020-09-08T16:04:34.755 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:02:08.129672+0000", 2020-09-08T16:04:34.755 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 146.50546426299999, 2020-09-08T16:04:34.756 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.30912979699999998, 2020-09-08T16:04:34.756 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:34.756 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:34.756 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:34.756 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:04:34.757 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:04:34.757 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 132325 2020-09-08T16:04:34.757 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.757 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:34.757 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.758 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:34.758 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.129672+0000", 2020-09-08T16:04:34.758 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:34.758 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.759 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.759 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:34.759 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.129672+0000", 2020-09-08T16:04:34.760 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999266 2020-09-08T16:04:34.760 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.760 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.760 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:34.760 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.129598+0000", 2020-09-08T16:04:34.761 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00023092699999999999 2020-09-08T16:04:34.761 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.761 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.762 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:34.762 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.129829+0000", 2020-09-08T16:04:34.762 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.8436999999999999e-05 2020-09-08T16:04:34.762 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.762 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.763 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:34.763 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.129857+0000", 2020-09-08T16:04:34.763 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1005999999999999e-05 2020-09-08T16:04:34.763 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.763 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.764 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:34.764 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.129868+0000", 2020-09-08T16:04:34.764 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.5742000000000003e-05 2020-09-08T16:04:34.764 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.764 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.764 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:34.765 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.129924+0000", 2020-09-08T16:04:34.765 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000117793 2020-09-08T16:04:34.765 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.765 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.766 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:34.766 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.130042+0000", 2020-09-08T16:04:34.766 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000125203 2020-09-08T16:04:34.766 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.766 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.766 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T16:04:34.767 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.130167+0000", 2020-09-08T16:04:34.767 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.308075339 2020-09-08T16:04:34.767 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.767 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.767 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:34.768 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.438242+0000", 2020-09-08T16:04:34.768 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000157093 2020-09-08T16:04:34.768 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.768 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.769 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:34.769 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.438399+0000", 2020-09-08T16:04:34.769 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00024729500000000002 2020-09-08T16:04:34.770 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.770 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.770 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:34.770 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.438647+0000", 2020-09-08T16:04:34.771 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015457000000000001 2020-09-08T16:04:34.771 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.771 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.772 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:34.772 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.438801+0000", 2020-09-08T16:04:34.772 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.30912979699999998 2020-09-08T16:04:34.772 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.773 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.773 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.773 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.773 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.774 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:132328 7.d 7:be1d4a9c:::benchmark_data_smithi166_48533_object132327:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e582)", 2020-09-08T16:04:34.774 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:02:08.131595+0000", 2020-09-08T16:04:34.774 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 146.50354043900001, 2020-09-08T16:04:34.774 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.31398954499999998, 2020-09-08T16:04:34.774 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:34.775 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:34.775 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:34.775 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:04:34.775 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:04:34.775 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 132328 2020-09-08T16:04:34.776 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.776 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:34.776 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.776 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:34.776 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.131595+0000", 2020-09-08T16:04:34.776 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:34.777 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.777 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.777 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:34.777 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.131595+0000", 2020-09-08T16:04:34.778 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999657 2020-09-08T16:04:34.778 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.778 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.778 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:34.778 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.131561+0000", 2020-09-08T16:04:34.779 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011364100000000001 2020-09-08T16:04:34.779 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.780 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.780 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:34.780 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.131675+0000", 2020-09-08T16:04:34.781 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4521e-05 2020-09-08T16:04:34.781 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.781 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.781 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:34.783 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.131689+0000", 2020-09-08T16:04:34.783 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.7000000000000002e-06 2020-09-08T16:04:34.783 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.783 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.784 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:34.784 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.131696+0000", 2020-09-08T16:04:34.784 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0024308530000000002 2020-09-08T16:04:34.784 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.785 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.785 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:34.785 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.134127+0000", 2020-09-08T16:04:34.786 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000221628 2020-09-08T16:04:34.786 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.786 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.786 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:34.786 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.134348+0000", 2020-09-08T16:04:34.787 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.29382143500000002 2020-09-08T16:04:34.787 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.787 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.787 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T16:04:34.787 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.428170+0000", 2020-09-08T16:04:34.788 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.010774832 2020-09-08T16:04:34.788 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.788 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.788 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:34.788 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.438944+0000", 2020-09-08T16:04:34.789 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0065263630000000003 2020-09-08T16:04:34.789 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.789 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.789 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:34.790 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.445471+0000", 2020-09-08T16:04:34.791 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.8069999999999998e-05 2020-09-08T16:04:34.791 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.791 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.791 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:34.791 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.445529+0000", 2020-09-08T16:04:34.792 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.5976000000000001e-05 2020-09-08T16:04:34.792 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.792 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.792 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:34.792 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.445585+0000", 2020-09-08T16:04:34.793 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.31398954499999998 2020-09-08T16:04:34.793 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.793 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.793 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.794 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.794 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.794 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:167413 7.d 7:b9730c85:::benchmark_data_smithi166_48533_object32896:head [delete] snapc 0=[] ondisk+write+known_if_redirected e591)", 2020-09-08T16:04:34.794 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:02:42.979998+0000", 2020-09-08T16:04:34.794 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 111.655137497, 2020-09-08T16:04:34.795 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.20686253700000001, 2020-09-08T16:04:34.795 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:34.795 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:34.795 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:34.795 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:04:34.795 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:04:34.796 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 167413 2020-09-08T16:04:34.796 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.796 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:34.797 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.797 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:34.798 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.979998+0000", 2020-09-08T16:04:34.798 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:34.798 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.798 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.798 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:34.799 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.979998+0000", 2020-09-08T16:04:34.799 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999918 2020-09-08T16:04:34.799 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.800 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.800 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:34.800 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.979916+0000", 2020-09-08T16:04:34.800 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000184785 2020-09-08T16:04:34.800 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.801 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.801 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:34.801 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.980101+0000", 2020-09-08T16:04:34.801 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.0874999999999998e-05 2020-09-08T16:04:34.801 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.802 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.802 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:34.802 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.980132+0000", 2020-09-08T16:04:34.802 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.43e-05 2020-09-08T16:04:34.802 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.802 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.803 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:34.803 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.980146+0000", 2020-09-08T16:04:34.803 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0064882969999999996 2020-09-08T16:04:34.803 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.803 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.804 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:34.804 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.986635+0000", 2020-09-08T16:04:34.804 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00023545400000000001 2020-09-08T16:04:34.804 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.804 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.805 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:34.805 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.986870+0000", 2020-09-08T16:04:34.805 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00023533699999999999 2020-09-08T16:04:34.805 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.806 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.806 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T16:04:34.806 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.987105+0000", 2020-09-08T16:04:34.806 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.199244117 2020-09-08T16:04:34.806 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.807 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.807 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:34.807 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:43.186349+0000", 2020-09-08T16:04:34.807 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019089399999999999 2020-09-08T16:04:34.807 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.808 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.808 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:34.808 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:43.186540+0000", 2020-09-08T16:04:34.808 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017134499999999999 2020-09-08T16:04:34.808 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.809 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.809 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:34.809 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:43.186712+0000", 2020-09-08T16:04:34.809 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014912000000000001 2020-09-08T16:04:34.809 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.810 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.810 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:34.810 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:43.186861+0000", 2020-09-08T16:04:34.810 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.20686253700000001 2020-09-08T16:04:34.810 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.811 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.811 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.811 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.811 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.811 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:167424 7.d 7:bed25e6c:::benchmark_data_smithi166_48533_object32907:head [delete] snapc 0=[] ondisk+write+known_if_redirected e591)", 2020-09-08T16:04:34.812 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:02:42.981480+0000", 2020-09-08T16:04:34.812 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 111.653655421, 2020-09-08T16:04:34.812 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.206409863, 2020-09-08T16:04:34.812 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:34.812 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:34.813 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:34.813 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:04:34.813 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:04:34.813 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 167424 2020-09-08T16:04:34.813 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.814 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:34.814 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.814 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:34.814 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.981480+0000", 2020-09-08T16:04:34.814 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:34.815 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.815 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.815 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:34.815 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.981480+0000", 2020-09-08T16:04:34.815 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999251 2020-09-08T16:04:34.816 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.816 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.816 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:34.816 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.981405+0000", 2020-09-08T16:04:34.817 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000164981 2020-09-08T16:04:34.817 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.817 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.817 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:34.817 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.981570+0000", 2020-09-08T16:04:34.818 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.3201e-05 2020-09-08T16:04:34.818 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.818 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.818 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:34.818 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.981593+0000", 2020-09-08T16:04:34.819 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.6940000000000007e-06 2020-09-08T16:04:34.819 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.819 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.819 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:34.819 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.981603+0000", 2020-09-08T16:04:34.820 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.017031066000000001 2020-09-08T16:04:34.820 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.820 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.820 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:34.820 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.998634+0000", 2020-09-08T16:04:34.821 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015101099999999999 2020-09-08T16:04:34.821 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.821 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.821 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:34.821 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.998785+0000", 2020-09-08T16:04:34.822 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015327900000000001 2020-09-08T16:04:34.822 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.822 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.822 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T16:04:34.823 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.998939+0000", 2020-09-08T16:04:34.823 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.18845735599999999 2020-09-08T16:04:34.823 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.823 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.823 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:34.823 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:43.187396+0000", 2020-09-08T16:04:34.824 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00022437600000000001 2020-09-08T16:04:34.824 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.824 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.824 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:34.824 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:43.187620+0000", 2020-09-08T16:04:34.825 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014537400000000001 2020-09-08T16:04:34.825 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.825 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.825 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:34.825 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:43.187766+0000", 2020-09-08T16:04:34.826 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012461800000000001 2020-09-08T16:04:34.826 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.826 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.826 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:34.826 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:43.187890+0000", 2020-09-08T16:04:34.827 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.206409863 2020-09-08T16:04:34.827 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.827 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.827 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.827 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.828 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.828 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:193178 7.7 7:e8a92310:::benchmark_data_smithi166_48533_object58661:head [delete] snapc 0=[] ondisk+write+known_if_redirected e598)", 2020-09-08T16:04:34.828 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:09.592514+0000", 2020-09-08T16:04:34.828 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 85.042621878000006, 2020-09-08T16:04:34.828 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.59101522799999995, 2020-09-08T16:04:34.829 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:34.829 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:34.829 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:34.829 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:04:34.829 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:04:34.830 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 193178 2020-09-08T16:04:34.830 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.830 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:34.830 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.830 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:34.831 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.592514+0000", 2020-09-08T16:04:34.831 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:34.831 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.831 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.831 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:34.832 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.592514+0000", 2020-09-08T16:04:34.832 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999399 2020-09-08T16:04:34.832 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.832 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.832 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:34.833 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.592454+0000", 2020-09-08T16:04:34.833 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013446700000000001 2020-09-08T16:04:34.833 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.833 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.833 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:34.834 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.592588+0000", 2020-09-08T16:04:34.834 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.1747999999999999e-05 2020-09-08T16:04:34.834 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.834 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.834 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:34.835 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.592610+0000", 2020-09-08T16:04:34.835 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.2560000000000008e-06 2020-09-08T16:04:34.835 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.835 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.835 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:34.836 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.592619+0000", 2020-09-08T16:04:34.836 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0037480899999999999 2020-09-08T16:04:34.836 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.836 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.836 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:34.837 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.596367+0000", 2020-09-08T16:04:34.837 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000146856 2020-09-08T16:04:34.837 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.837 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.837 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:34.838 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.596514+0000", 2020-09-08T16:04:34.838 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.6421000000000004e-05 2020-09-08T16:04:34.838 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.838 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.838 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 8", 2020-09-08T16:04:34.839 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.596611+0000", 2020-09-08T16:04:34.839 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.57584053899999998 2020-09-08T16:04:34.839 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.839 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.840 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:34.840 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172451+0000", 2020-09-08T16:04:34.840 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.010788818 2020-09-08T16:04:34.840 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.840 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.841 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:34.841 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183240+0000", 2020-09-08T16:04:34.841 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016227700000000001 2020-09-08T16:04:34.842 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.842 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.842 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:34.842 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183402+0000", 2020-09-08T16:04:34.843 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012677200000000001 2020-09-08T16:04:34.843 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.843 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.843 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:34.844 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183529+0000", 2020-09-08T16:04:34.844 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.59101522799999995 2020-09-08T16:04:34.844 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.844 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.844 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.845 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.845 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.845 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:193184 7.7 7:e3f87857:::benchmark_data_smithi166_48533_object58667:head [delete] snapc 0=[] ondisk+write+known_if_redirected e598)", 2020-09-08T16:04:34.845 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:09.594130+0000", 2020-09-08T16:04:34.845 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 85.041005542999997, 2020-09-08T16:04:34.846 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.59013966100000004, 2020-09-08T16:04:34.846 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:34.846 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:34.846 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:34.846 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:04:34.847 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:04:34.847 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 193184 2020-09-08T16:04:34.847 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.847 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:34.847 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.848 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:34.848 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.594130+0000", 2020-09-08T16:04:34.848 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:34.848 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.848 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.848 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:34.849 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.594130+0000", 2020-09-08T16:04:34.849 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999776 2020-09-08T16:04:34.849 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.849 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.849 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:34.850 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.594108+0000", 2020-09-08T16:04:34.850 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.1103000000000002e-05 2020-09-08T16:04:34.850 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.850 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.851 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:34.851 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.594159+0000", 2020-09-08T16:04:34.851 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.2590000000000006e-06 2020-09-08T16:04:34.851 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.851 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.852 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:34.852 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.594168+0000", 2020-09-08T16:04:34.852 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.7550000000000002e-06 2020-09-08T16:04:34.852 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.852 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.853 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:34.853 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.594173+0000", 2020-09-08T16:04:34.853 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0075788399999999999 2020-09-08T16:04:34.853 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.853 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.854 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:34.854 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.601752+0000", 2020-09-08T16:04:34.854 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000142695 2020-09-08T16:04:34.854 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.854 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.855 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:34.855 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.601894+0000", 2020-09-08T16:04:34.855 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016692000000000001 2020-09-08T16:04:34.855 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.856 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.856 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 8", 2020-09-08T16:04:34.856 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602061+0000", 2020-09-08T16:04:34.856 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.58155135999999996 2020-09-08T16:04:34.857 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.857 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.857 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:34.857 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183613+0000", 2020-09-08T16:04:34.857 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00041249499999999997 2020-09-08T16:04:34.858 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.858 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.858 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:34.858 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.184025+0000", 2020-09-08T16:04:34.859 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013872900000000001 2020-09-08T16:04:34.859 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.859 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.859 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:34.859 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.184164+0000", 2020-09-08T16:04:34.860 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000105926 2020-09-08T16:04:34.860 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.860 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.860 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:34.860 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.184270+0000", 2020-09-08T16:04:34.861 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.59013966100000004 2020-09-08T16:04:34.861 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.861 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.861 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.861 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.862 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.862 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:193186 7.c 7:356146cd:::benchmark_data_smithi166_48533_object58669:head [delete] snapc 0=[] ondisk+write+known_if_redirected e598)", 2020-09-08T16:04:34.862 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:09.598767+0000", 2020-09-08T16:04:34.862 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 85.036369085999993, 2020-09-08T16:04:34.862 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.58441285899999995, 2020-09-08T16:04:34.863 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:34.863 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:34.863 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:34.863 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:04:34.863 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:04:34.864 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 193186 2020-09-08T16:04:34.864 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.864 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:34.864 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.864 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:34.865 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.598767+0000", 2020-09-08T16:04:34.865 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:34.865 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.865 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.865 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:34.866 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.598767+0000", 2020-09-08T16:04:34.866 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999766 2020-09-08T16:04:34.866 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.866 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.866 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:34.867 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.598743+0000", 2020-09-08T16:04:34.867 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2475999999999997e-05 2020-09-08T16:04:34.867 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.867 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.868 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:34.868 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.598796+0000", 2020-09-08T16:04:34.868 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.8800000000000003e-06 2020-09-08T16:04:34.868 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.868 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.869 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:34.869 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.598806+0000", 2020-09-08T16:04:34.869 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2179999999999998e-06 2020-09-08T16:04:34.869 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.870 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.870 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:34.870 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.598811+0000", 2020-09-08T16:04:34.870 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.4370999999999997e-05 2020-09-08T16:04:34.871 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.871 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.871 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:34.871 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.598885+0000", 2020-09-08T16:04:34.871 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014553299999999999 2020-09-08T16:04:34.872 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.872 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.872 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:34.872 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.599031+0000", 2020-09-08T16:04:34.872 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016689 2020-09-08T16:04:34.873 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.873 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.873 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:04:34.873 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.599198+0000", 2020-09-08T16:04:34.874 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.57324991800000002 2020-09-08T16:04:34.874 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.874 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.874 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:34.874 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172448+0000", 2020-09-08T16:04:34.875 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.010612709 2020-09-08T16:04:34.875 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.875 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.876 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:34.876 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183060+0000", 2020-09-08T16:04:34.877 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0356000000000003e-05 2020-09-08T16:04:34.877 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.877 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.877 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:34.877 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183121+0000", 2020-09-08T16:04:34.878 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.8757e-05 2020-09-08T16:04:34.878 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.878 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.878 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:34.878 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183180+0000", 2020-09-08T16:04:34.879 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.58441285899999995 2020-09-08T16:04:34.879 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.879 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.879 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.879 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.880 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.880 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop_reply(client.36113.0:193175 7.7 e598/553 ondisk, result = 0)", 2020-09-08T16:04:34.880 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:09.602167+0000", 2020-09-08T16:04:34.880 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 85.032968709000002, 2020-09-08T16:04:34.880 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.570143972, 2020-09-08T16:04:34.881 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:34.881 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "started", 2020-09-08T16:04:34.881 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:34.881 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.882 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:34.882 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602167+0000", 2020-09-08T16:04:34.882 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:34.882 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.882 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.883 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:34.883 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602167+0000", 2020-09-08T16:04:34.883 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4880999999999999e-05 2020-09-08T16:04:34.883 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.883 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.884 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:34.884 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602222+0000", 2020-09-08T16:04:34.884 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4469999999999995e-06 2020-09-08T16:04:34.884 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.884 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.884 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:34.885 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602231+0000", 2020-09-08T16:04:34.885 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5197999999999999e-05 2020-09-08T16:04:34.885 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.885 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.886 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:34.886 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602247+0000", 2020-09-08T16:04:34.886 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.7388e-05 2020-09-08T16:04:34.886 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.886 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.886 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:34.887 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602264+0000", 2020-09-08T16:04:34.887 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0053359840000000002 2020-09-08T16:04:34.887 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.887 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.887 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:34.888 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607600+0000", 2020-09-08T16:04:34.888 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1474000000000001e-05 2020-09-08T16:04:34.888 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.888 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.888 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:34.889 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607611+0000", 2020-09-08T16:04:34.889 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.56469959999999997 2020-09-08T16:04:34.889 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.889 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.889 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:34.890 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172311+0000", 2020-09-08T16:04:34.890 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.570143972 2020-09-08T16:04:34.890 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.890 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.890 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.891 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.891 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.891 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop_reply(client.36113.0:193178 7.7 e598/553 ondisk, result = 0)", 2020-09-08T16:04:34.891 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:09.602323+0000", 2020-09-08T16:04:34.891 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 85.032812500000006, 2020-09-08T16:04:34.892 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.57013256700000003, 2020-09-08T16:04:34.892 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:34.892 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "started", 2020-09-08T16:04:34.892 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:34.892 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.893 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:34.893 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602323+0000", 2020-09-08T16:04:34.893 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:34.893 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.893 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.894 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:34.894 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602323+0000", 2020-09-08T16:04:34.894 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.0728999999999997e-05 2020-09-08T16:04:34.894 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.894 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.895 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:34.895 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602354+0000", 2020-09-08T16:04:34.895 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.826e-06 2020-09-08T16:04:34.895 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.895 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.896 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:34.896 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602359+0000", 2020-09-08T16:04:34.896 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.8469999999999996e-06 2020-09-08T16:04:34.896 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.896 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.897 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:34.897 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602367+0000", 2020-09-08T16:04:34.897 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.4320000000000001e-06 2020-09-08T16:04:34.897 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.897 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.898 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:34.898 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602371+0000", 2020-09-08T16:04:34.898 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.57003088400000002 2020-09-08T16:04:34.898 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.898 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.899 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:34.899 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172402+0000", 2020-09-08T16:04:34.899 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.7949e-05 2020-09-08T16:04:34.899 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.900 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.900 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:34.900 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172430+0000", 2020-09-08T16:04:34.900 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.5899999999999999e-05 2020-09-08T16:04:34.901 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.901 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.901 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:34.901 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172456+0000", 2020-09-08T16:04:34.901 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.57013256700000003 2020-09-08T16:04:34.902 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.902 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.902 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.902 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.902 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.903 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:193191 7.c 7:3a26bee4:::benchmark_data_smithi166_48533_object58674:head [delete] snapc 0=[] ondisk+write+known_if_redirected e598)", 2020-09-08T16:04:34.903 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:09.605163+0000", 2020-09-08T16:04:34.903 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 85.029973149, 2020-09-08T16:04:34.903 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.57837770200000005, 2020-09-08T16:04:34.903 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:34.904 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:34.904 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:34.904 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:04:34.904 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:04:34.904 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 193191 2020-09-08T16:04:34.905 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.905 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:34.905 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.906 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:34.906 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605163+0000", 2020-09-08T16:04:34.906 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:34.906 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.906 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.907 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:34.907 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605163+0000", 2020-09-08T16:04:34.907 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999628 2020-09-08T16:04:34.907 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.907 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.908 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:34.908 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605126+0000", 2020-09-08T16:04:34.908 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.3867000000000002e-05 2020-09-08T16:04:34.908 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.908 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.909 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:34.909 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605210+0000", 2020-09-08T16:04:34.909 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4372000000000001e-05 2020-09-08T16:04:34.909 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.909 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.910 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:34.910 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605224+0000", 2020-09-08T16:04:34.910 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.6499999999999999e-06 2020-09-08T16:04:34.910 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.910 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.911 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:34.911 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605231+0000", 2020-09-08T16:04:34.911 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00034729100000000001 2020-09-08T16:04:34.911 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.911 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.912 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:34.912 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605578+0000", 2020-09-08T16:04:34.912 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015052500000000001 2020-09-08T16:04:34.912 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.912 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.913 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:34.913 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605728+0000", 2020-09-08T16:04:34.913 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015432799999999999 2020-09-08T16:04:34.913 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.913 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.914 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:04:34.914 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605883+0000", 2020-09-08T16:04:34.914 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.56658740699999999 2020-09-08T16:04:34.914 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.914 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.915 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:34.915 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172470+0000", 2020-09-08T16:04:34.915 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.010821318999999999 2020-09-08T16:04:34.915 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.915 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.916 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:34.916 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183291+0000", 2020-09-08T16:04:34.916 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000148677 2020-09-08T16:04:34.916 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.916 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.917 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:34.917 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183440+0000", 2020-09-08T16:04:34.917 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010026000000000001 2020-09-08T16:04:34.917 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.918 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.918 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:34.918 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183540+0000", 2020-09-08T16:04:34.918 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.57837770200000005 2020-09-08T16:04:34.918 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.919 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.919 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.919 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.919 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.919 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:193193 7.c 7:300ec92f:::benchmark_data_smithi166_48533_object58676:head [delete] snapc 0=[] ondisk+write+known_if_redirected e598)", 2020-09-08T16:04:34.920 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:09.605299+0000", 2020-09-08T16:04:34.920 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 85.029837243000003, 2020-09-08T16:04:34.920 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.59181674100000004, 2020-09-08T16:04:34.920 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:34.920 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:34.921 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:34.921 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:04:34.921 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:04:34.921 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 193193 2020-09-08T16:04:34.921 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.922 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:34.922 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.922 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:34.922 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605299+0000", 2020-09-08T16:04:34.922 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:34.923 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.923 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.923 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:34.923 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605299+0000", 2020-09-08T16:04:34.923 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999661 2020-09-08T16:04:34.924 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.924 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.924 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:34.924 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605265+0000", 2020-09-08T16:04:34.924 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.5384999999999997e-05 2020-09-08T16:04:34.925 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.925 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.925 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:34.925 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605340+0000", 2020-09-08T16:04:34.925 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1066999999999999e-05 2020-09-08T16:04:34.926 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.926 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.926 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:34.926 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605351+0000", 2020-09-08T16:04:34.927 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.6249999999999998e-06 2020-09-08T16:04:34.927 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.927 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.927 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:34.928 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605356+0000", 2020-09-08T16:04:34.928 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.56720643599999998 2020-09-08T16:04:34.928 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.928 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.928 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:34.929 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172562+0000", 2020-09-08T16:04:34.929 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00041356200000000002 2020-09-08T16:04:34.929 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.929 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.930 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:34.930 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172976+0000", 2020-09-08T16:04:34.930 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00047280800000000002 2020-09-08T16:04:34.930 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.930 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.930 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:04:34.931 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.173449+0000", 2020-09-08T16:04:34.931 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.010238517000000001 2020-09-08T16:04:34.931 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.931 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.932 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:34.932 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183687+0000", 2020-09-08T16:04:34.932 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.013308952000000001 2020-09-08T16:04:34.932 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.932 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.933 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:34.933 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.196996+0000", 2020-09-08T16:04:34.933 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.583e-05 2020-09-08T16:04:34.933 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.933 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.934 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:34.934 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.197042+0000", 2020-09-08T16:04:34.934 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.3201000000000006e-05 2020-09-08T16:04:34.934 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.934 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.935 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:34.935 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.197115+0000", 2020-09-08T16:04:34.935 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.59181674100000004 2020-09-08T16:04:34.935 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.935 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.936 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.936 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.936 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.936 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:193194 7.d 7:bf0541f8:::benchmark_data_smithi166_48533_object58677:head [delete] snapc 0=[] ondisk+write+known_if_redirected e598)", 2020-09-08T16:04:34.937 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:09.607175+0000", 2020-09-08T16:04:34.937 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 85.027960849999999, 2020-09-08T16:04:34.937 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.56647141700000003, 2020-09-08T16:04:34.937 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:34.937 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:34.938 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:34.938 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:04:34.938 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:04:34.938 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 193194 2020-09-08T16:04:34.938 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.939 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:34.939 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.939 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:34.939 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607175+0000", 2020-09-08T16:04:34.940 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:34.940 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.940 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.940 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:34.940 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607175+0000", 2020-09-08T16:04:34.941 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999585 2020-09-08T16:04:34.941 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.941 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.941 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:34.942 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607133+0000", 2020-09-08T16:04:34.942 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4347999999999999e-05 2020-09-08T16:04:34.942 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.942 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.942 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:34.943 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607228+0000", 2020-09-08T16:04:34.943 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5886999999999999e-05 2020-09-08T16:04:34.943 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.943 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.943 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:34.944 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607243+0000", 2020-09-08T16:04:34.944 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.1749999999999999e-06 2020-09-08T16:04:34.944 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.944 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.944 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:34.945 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607251+0000", 2020-09-08T16:04:34.945 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3350999999999997e-05 2020-09-08T16:04:34.945 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.945 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.945 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:34.946 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607314+0000", 2020-09-08T16:04:34.946 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013483200000000001 2020-09-08T16:04:34.946 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.946 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.947 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:34.947 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607449+0000", 2020-09-08T16:04:34.947 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000140638 2020-09-08T16:04:34.947 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.947 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.948 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T16:04:34.948 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607589+0000", 2020-09-08T16:04:34.948 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.56484538299999998 2020-09-08T16:04:34.948 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.948 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.949 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:34.949 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172435+0000", 2020-09-08T16:04:34.949 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00094178999999999997 2020-09-08T16:04:34.949 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.950 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.950 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:34.950 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.173377+0000", 2020-09-08T16:04:34.950 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015073699999999999 2020-09-08T16:04:34.950 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.951 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.951 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:34.951 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.173527+0000", 2020-09-08T16:04:34.951 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011898 2020-09-08T16:04:34.951 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.952 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.952 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:34.952 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.173646+0000", 2020-09-08T16:04:34.952 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.56647141700000003 2020-09-08T16:04:34.952 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.953 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.953 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.953 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.953 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.953 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:193195 7.7 7:e6302a85:::benchmark_data_smithi166_48533_object58678:head [delete] snapc 0=[] ondisk+write+known_if_redirected e598)", 2020-09-08T16:04:34.954 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:09.607347+0000", 2020-09-08T16:04:34.954 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 85.027788627000007, 2020-09-08T16:04:34.954 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.58303484500000002, 2020-09-08T16:04:34.954 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:04:34.954 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:04:34.955 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:04:34.955 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:04:34.955 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:04:34.955 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 193195 2020-09-08T16:04:34.955 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.956 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:04:34.956 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.956 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:04:34.957 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607347+0000", 2020-09-08T16:04:34.957 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:04:34.957 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.957 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.957 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:04:34.958 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607347+0000", 2020-09-08T16:04:34.958 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999571 2020-09-08T16:04:34.958 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.958 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.958 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:04:34.959 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607304+0000", 2020-09-08T16:04:34.959 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011198500000000001 2020-09-08T16:04:34.959 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.959 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.959 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:04:34.960 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607416+0000", 2020-09-08T16:04:34.960 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.7640000000000001e-05 2020-09-08T16:04:34.960 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.960 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.960 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:04:34.960 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607434+0000", 2020-09-08T16:04:34.961 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9519999999999999e-06 2020-09-08T16:04:34.961 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.961 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.961 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:04:34.961 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607440+0000", 2020-09-08T16:04:34.962 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.56508348200000003 2020-09-08T16:04:34.962 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.962 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.962 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:04:34.962 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172523+0000", 2020-09-08T16:04:34.963 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00032738999999999999 2020-09-08T16:04:34.963 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.963 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.963 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:04:34.963 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172851+0000", 2020-09-08T16:04:34.964 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000334035 2020-09-08T16:04:34.964 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.964 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.964 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 8", 2020-09-08T16:04:34.964 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.173185+0000", 2020-09-08T16:04:34.965 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.011111555 2020-09-08T16:04:34.965 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.965 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.965 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:04:34.965 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.184296+0000", 2020-09-08T16:04:34.966 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0059360080000000004 2020-09-08T16:04:34.966 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.966 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.966 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:04:34.966 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.190232+0000", 2020-09-08T16:04:34.967 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.6401000000000002e-05 2020-09-08T16:04:34.967 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.967 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.967 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:04:34.967 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.190319+0000", 2020-09-08T16:04:34.968 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3299999999999994e-05 2020-09-08T16:04:34.968 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:04:34.968 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:04:34.968 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:04:34.968 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.190382+0000", 2020-09-08T16:04:34.969 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.58303484500000002 2020-09-08T16:04:34.969 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.969 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.969 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.969 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:04:34.970 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:04:34.970 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:04:34.970 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:34.971 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:04:34 smithi055 bash[16646]: debug 2020-09-08T16:04:34.650+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:35.072 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:35.191 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:04:35.215 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:35.320 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:35.326 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:04:35.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:04:35 smithi166 bash[14254]: debug 2020-09-08T16:04:35.215+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:35.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:35 smithi166 bash[9446]: cluster 2020-09-08T16:04:34.022802+0000 mgr.x (mgr.24118) 1117 : cluster [DBG] pgmap v1653: 17 pgs: 17 active+clean; 111 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 886 op/s 2020-09-08T16:04:35.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:35 smithi166 bash[9446]: cluster 2020-09-08T16:04:34.217877+0000 mon.a (mon.0) 2021 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:04:35.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:35 smithi166 bash[9446]: cluster 2020-09-08T16:04:34.217911+0000 mon.a (mon.0) 2022 : cluster [INF] Cluster is now healthy 2020-09-08T16:04:35.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:35 smithi166 bash[9446]: audit 2020-09-08T16:04:34.229842+0000 mon.a (mon.0) 2023 : audit [INF] from='client.? v1:172.21.15.104:0/1602183609' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:04:35.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:35 smithi166 bash[9446]: cluster 2020-09-08T16:04:34.229937+0000 mon.a (mon.0) 2024 : cluster [DBG] osdmap e621: 12 total, 12 up, 12 in 2020-09-08T16:04:35.530 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:35 smithi055 bash[9296]: cluster 2020-09-08T16:04:34.022802+0000 mgr.x (mgr.24118) 1117 : cluster [DBG] pgmap v1653: 17 pgs: 17 active+clean; 111 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 886 op/s 2020-09-08T16:04:35.530 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:35 smithi055 bash[9296]: cluster 2020-09-08T16:04:34.217877+0000 mon.a (mon.0) 2021 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:04:35.530 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:35 smithi055 bash[9296]: cluster 2020-09-08T16:04:34.217911+0000 mon.a (mon.0) 2022 : cluster [INF] Cluster is now healthy 2020-09-08T16:04:35.531 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:35 smithi055 bash[9296]: audit 2020-09-08T16:04:34.229842+0000 mon.a (mon.0) 2023 : audit [INF] from='client.? v1:172.21.15.104:0/1602183609' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:04:35.531 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:35 smithi055 bash[9296]: cluster 2020-09-08T16:04:34.229937+0000 mon.a (mon.0) 2024 : cluster [DBG] osdmap e621: 12 total, 12 up, 12 in 2020-09-08T16:04:35.531 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:35.591 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2020-09-08T16:04:35.636 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:35.651 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:35 smithi104 bash[9203]: cluster 2020-09-08T16:04:34.022802+0000 mgr.x (mgr.24118) 1117 : cluster [DBG] pgmap v1653: 17 pgs: 17 active+clean; 111 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 886 op/s 2020-09-08T16:04:35.652 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:35 smithi104 bash[9203]: cluster 2020-09-08T16:04:34.217877+0000 mon.a (mon.0) 2021 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:04:35.652 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:35 smithi104 bash[9203]: cluster 2020-09-08T16:04:34.217911+0000 mon.a (mon.0) 2022 : cluster [INF] Cluster is now healthy 2020-09-08T16:04:35.652 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:35 smithi104 bash[9203]: audit 2020-09-08T16:04:34.229842+0000 mon.a (mon.0) 2023 : audit [INF] from='client.? v1:172.21.15.104:0/1602183609' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:04:35.652 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:35 smithi104 bash[9203]: cluster 2020-09-08T16:04:34.229937+0000 mon.a (mon.0) 2024 : cluster [DBG] osdmap e621: 12 total, 12 up, 12 in 2020-09-08T16:04:35.727 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:04:35.870 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:35.906 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:04:35 smithi055 bash[19003]: debug 2020-09-08T16:04:35.534+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:35.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:04:35 smithi104 bash[15273]: debug 2020-09-08T16:04:35.869+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:35.976 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:36.349 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:36.454 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:36.631 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:36.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:04:36 smithi055 bash[16646]: debug 2020-09-08T16:04:36.350+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:36.737 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:36.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:04:36 smithi166 bash[14254]: debug 2020-09-08T16:04:36.631+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:36.985 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:37.092 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:37.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:04:36 smithi055 bash[14301]: debug 2020-09-08T16:04:36.990+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:37.264 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:37 smithi104 bash[9203]: cluster 2020-09-08T16:04:36.023474+0000 mgr.x (mgr.24118) 1118 : cluster [DBG] pgmap v1655: 17 pgs: 17 active+clean; 111 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 691 op/s 2020-09-08T16:04:37.265 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:37.368 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:37.525 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:37 smithi055 bash[9296]: cluster 2020-09-08T16:04:36.023474+0000 mgr.x (mgr.24118) 1118 : cluster [DBG] pgmap v1655: 17 pgs: 17 active+clean; 111 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 691 op/s 2020-09-08T16:04:37.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:37 smithi166 bash[9446]: cluster 2020-09-08T16:04:36.023474+0000 mgr.x (mgr.24118) 1118 : cluster [DBG] pgmap v1655: 17 pgs: 17 active+clean; 111 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 691 op/s 2020-09-08T16:04:37.665 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:04:37 smithi104 bash[15273]: debug 2020-09-08T16:04:37.265+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:37.666 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:37.778 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:37.809 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T16:04:37.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:04:37 smithi104 bash[15273]: debug 2020-09-08T16:04:37.665+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:37.938 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:04:38.097 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:38.202 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:38.364 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:38.410 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:04:38 smithi055 bash[11956]: debug 2020-09-08T16:04:38.098+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:38.416 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_5 2020-09-08T16:04:38.418 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool rm unique_pool_5 unique_pool_5 --yes-i-really-really-mean-it 2020-09-08T16:04:38.470 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:38.557 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:04:38.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:04:38 smithi166 bash[16593]: debug 2020-09-08T16:04:38.363+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:38.910 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:39.016 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:04:39.162 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:04:39.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:04:38 smithi104 bash[17588]: debug 2020-09-08T16:04:38.909+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:39.267 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:39.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:39 smithi166 bash[9446]: cluster 2020-09-08T16:04:38.024225+0000 mgr.x (mgr.24118) 1119 : cluster [DBG] pgmap v1656: 17 pgs: 17 active+clean; 6.1 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 800 op/s 2020-09-08T16:04:39.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:39 smithi166 bash[9446]: cluster 2020-09-08T16:04:38.326612+0000 osd.8 (osd.8) 83 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:39.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:39 smithi166 bash[9446]: cluster 2020-09-08T16:04:38.326747+0000 osd.8 (osd.8) 84 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:39.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:04:39 smithi166 bash[18960]: debug 2020-09-08T16:04:39.163+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:39.383 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:04:39.487 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:39.534 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:04:39.534 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T16:04:39.535 INFO:teuthology.orchestra.run.smithi055.stdout: "num_ops": 0 2020-09-08T16:04:39.535 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:04:39.606 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:04:39 smithi055 bash[16646]: debug 2020-09-08T16:04:39.385+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:39.607 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:39 smithi055 bash[9296]: cluster 2020-09-08T16:04:38.024225+0000 mgr.x (mgr.24118) 1119 : cluster [DBG] pgmap v1656: 17 pgs: 17 active+clean; 6.1 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 800 op/s 2020-09-08T16:04:39.607 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:39 smithi055 bash[9296]: cluster 2020-09-08T16:04:38.326612+0000 osd.8 (osd.8) 83 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:39.607 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:39 smithi055 bash[9296]: cluster 2020-09-08T16:04:38.326747+0000 osd.8 (osd.8) 84 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:39.612 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:39.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:39 smithi104 bash[9203]: cluster 2020-09-08T16:04:38.024225+0000 mgr.x (mgr.24118) 1119 : cluster [DBG] pgmap v1656: 17 pgs: 17 active+clean; 6.1 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 800 op/s 2020-09-08T16:04:39.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:39 smithi104 bash[9203]: cluster 2020-09-08T16:04:38.326612+0000 osd.8 (osd.8) 83 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:39.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:39 smithi104 bash[9203]: cluster 2020-09-08T16:04:38.326747+0000 osd.8 (osd.8) 84 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:39.717 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:04:39.834 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:04:39 smithi166 bash[16593]: debug 2020-09-08T16:04:39.615+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:39.834 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:04:39.939 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:40.088 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:04:39 smithi166 bash[18960]: debug 2020-09-08T16:04:39.835+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:40.089 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:40.194 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:40.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:40 smithi166 bash[9446]: cluster 2020-09-08T16:04:39.245954+0000 mon.a (mon.0) 2025 : cluster [DBG] osdmap e622: 12 total, 12 up, 12 in 2020-09-08T16:04:40.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:04:40 smithi166 bash[14254]: debug 2020-09-08T16:04:40.091+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:40.374 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:40.389 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2020-09-08T16:04:40.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:40 smithi104 bash[9203]: cluster 2020-09-08T16:04:39.245954+0000 mon.a (mon.0) 2025 : cluster [DBG] osdmap e622: 12 total, 12 up, 12 in 2020-09-08T16:04:40.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:04:40 smithi104 bash[17588]: debug 2020-09-08T16:04:40.376+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:40.479 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:40.519 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:04:40.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:40 smithi055 bash[9296]: cluster 2020-09-08T16:04:39.245954+0000 mon.a (mon.0) 2025 : cluster [DBG] osdmap e622: 12 total, 12 up, 12 in 2020-09-08T16:04:40.741 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:04:40.767 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:04:40.790 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:04:40.797 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:40.814 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:04:40.839 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:04:40.876 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:04:40.903 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:40.912 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:04:40.942 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:04:40.966 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:04:41.001 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:04:41.028 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:04:41.061 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:04:41.064 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:04:41.066 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:04:41.066 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:04:41.066 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:04:41.067 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:04:41.067 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:04:41.067 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:04:41.067 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:04:41.067 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:04:41.068 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:04:41.068 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:04:41.068 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:04:41.082 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:41.140 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:04:40 smithi104 bash[22305]: debug 2020-09-08T16:04:40.796+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:41.192 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:41.262 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:04:41 smithi055 bash[14301]: debug 2020-09-08T16:04:41.085+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:41.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:41 smithi104 bash[9203]: cluster 2020-09-08T16:04:40.024862+0000 mgr.x (mgr.24118) 1120 : cluster [DBG] pgmap v1658: 17 pgs: 17 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 646 op/s 2020-09-08T16:04:41.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:41 smithi104 bash[9203]: audit 2020-09-08T16:04:40.717071+0000 mon.a (mon.0) 2026 : audit [DBG] from='client.? v1:172.21.15.104:0/3509191345' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:04:41.578 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:41.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:41 smithi166 bash[9446]: cluster 2020-09-08T16:04:40.024862+0000 mgr.x (mgr.24118) 1120 : cluster [DBG] pgmap v1658: 17 pgs: 17 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 646 op/s 2020-09-08T16:04:41.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:41 smithi166 bash[9446]: audit 2020-09-08T16:04:40.717071+0000 mon.a (mon.0) 2026 : audit [DBG] from='client.? v1:172.21.15.104:0/3509191345' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:04:41.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:41 smithi055 bash[9296]: cluster 2020-09-08T16:04:40.024862+0000 mgr.x (mgr.24118) 1120 : cluster [DBG] pgmap v1658: 17 pgs: 17 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 646 op/s 2020-09-08T16:04:41.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:41 smithi055 bash[9296]: audit 2020-09-08T16:04:40.717071+0000 mon.a (mon.0) 2026 : audit [DBG] from='client.? v1:172.21.15.104:0/3509191345' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:04:41.684 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:04:41.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:04:41 smithi104 bash[17588]: debug 2020-09-08T16:04:41.576+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:42.288 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:04:42.393 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:04:42.530 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:04:42.635 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:42.678 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:04:42 smithi104 bash[19922]: debug 2020-09-08T16:04:42.288+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:42.759 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:42.863 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:42.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:04:42 smithi166 bash[11928]: debug 2020-09-08T16:04:42.531+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:42.906 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:04:42 smithi055 bash[14301]: debug 2020-09-08T16:04:42.761+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:43.249 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:43.354 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:43.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:43 smithi104 bash[9203]: cluster 2020-09-08T16:04:42.025364+0000 mgr.x (mgr.24118) 1121 : cluster [DBG] pgmap v1659: 17 pgs: 17 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 222 op/s 2020-09-08T16:04:43.575 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:43.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:43 smithi166 bash[9446]: cluster 2020-09-08T16:04:42.025364+0000 mgr.x (mgr.24118) 1121 : cluster [DBG] pgmap v1659: 17 pgs: 17 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 222 op/s 2020-09-08T16:04:43.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:43 smithi055 bash[9296]: cluster 2020-09-08T16:04:42.025364+0000 mgr.x (mgr.24118) 1121 : cluster [DBG] pgmap v1659: 17 pgs: 17 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 222 op/s 2020-09-08T16:04:43.658 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:04:43 smithi055 bash[14301]: debug 2020-09-08T16:04:43.253+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:43.683 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:43.987 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:04:43 smithi104 bash[15273]: debug 2020-09-08T16:04:43.572+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:44.049 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:44.155 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:44.297 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:44.337 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T16:04:44.338 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:04:44.400 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:44.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:04:44 smithi055 bash[19003]: debug 2020-09-08T16:04:44.301+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:44.427 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:04:44 smithi104 bash[15273]: debug 2020-09-08T16:04:44.048+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:44.575 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:44.682 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:44.836 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:44.906 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:04:44 smithi055 bash[14301]: debug 2020-09-08T16:04:44.577+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:44.940 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:45.054 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:45.117 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:04:44 smithi166 bash[14254]: debug 2020-09-08T16:04:44.835+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:45.125 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:04:45.172 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:04:45.180 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2020-09-08T16:04:45.261 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:04:45.270 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:04:45 smithi104 bash[17588]: debug 2020-09-08T16:04:45.052+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:45.285 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T16:04:45.290 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'unique_pool_5' removed 2020-09-08T16:04:45.296 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:04:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:04:45.401 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:45.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:45 smithi166 bash[9446]: cluster 2020-09-08T16:04:44.026075+0000 mgr.x (mgr.24118) 1122 : cluster [DBG] pgmap v1660: 17 pgs: 17 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 182 op/s 2020-09-08T16:04:45.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:45 smithi166 bash[9446]: audit 2020-09-08T16:04:44.836900+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? v1:172.21.15.104:0/2416473847' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:04:45.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:45 smithi166 bash[9446]: audit 2020-09-08T16:04:44.952173+0000 mon.a (mon.0) 2028 : audit [INF] from='client.? v1:172.21.15.104:0/3434024946' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_5", "pool2": "unique_pool_5", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T16:04:45.617 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:04:45 smithi166 bash[18960]: debug 2020-09-08T16:04:45.294+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:45.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:45 smithi055 bash[9296]: cluster 2020-09-08T16:04:44.026075+0000 mgr.x (mgr.24118) 1122 : cluster [DBG] pgmap v1660: 17 pgs: 17 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 182 op/s 2020-09-08T16:04:45.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:45 smithi055 bash[9296]: audit 2020-09-08T16:04:44.836900+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? v1:172.21.15.104:0/2416473847' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:04:45.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:45 smithi055 bash[9296]: audit 2020-09-08T16:04:44.952173+0000 mon.a (mon.0) 2028 : audit [INF] from='client.? v1:172.21.15.104:0/3434024946' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_5", "pool2": "unique_pool_5", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T16:04:45.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:45 smithi104 bash[9203]: cluster 2020-09-08T16:04:44.026075+0000 mgr.x (mgr.24118) 1122 : cluster [DBG] pgmap v1660: 17 pgs: 17 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 182 op/s 2020-09-08T16:04:45.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:45 smithi104 bash[9203]: audit 2020-09-08T16:04:44.836900+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? v1:172.21.15.104:0/2416473847' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:04:45.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:45 smithi104 bash[9203]: audit 2020-09-08T16:04:44.952173+0000 mon.a (mon.0) 2028 : audit [INF] from='client.? v1:172.21.15.104:0/3434024946' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_5", "pool2": "unique_pool_5", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T16:04:45.915 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:46.020 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:46.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:04:45 smithi104 bash[22305]: debug 2020-09-08T16:04:45.916+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:46.389 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:46.493 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:46.536 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:46 smithi055 bash[9296]: cluster 2020-09-08T16:04:45.273438+0000 mon.a (mon.0) 2029 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:04:46.537 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:46 smithi055 bash[9296]: audit 2020-09-08T16:04:45.285684+0000 mon.a (mon.0) 2030 : audit [INF] from='client.? v1:172.21.15.104:0/2416473847' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:04:46.537 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:46 smithi055 bash[9296]: audit 2020-09-08T16:04:45.285726+0000 mon.a (mon.0) 2031 : audit [INF] from='client.? v1:172.21.15.104:0/3434024946' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_5", "pool2": "unique_pool_5", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T16:04:46.537 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:46 smithi055 bash[9296]: cluster 2020-09-08T16:04:45.285757+0000 mon.a (mon.0) 2032 : cluster [DBG] osdmap e623: 12 total, 12 up, 12 in 2020-09-08T16:04:46.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:46 smithi166 bash[9446]: cluster 2020-09-08T16:04:45.273438+0000 mon.a (mon.0) 2029 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:04:46.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:46 smithi166 bash[9446]: audit 2020-09-08T16:04:45.285684+0000 mon.a (mon.0) 2030 : audit [INF] from='client.? v1:172.21.15.104:0/2416473847' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:04:46.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:46 smithi166 bash[9446]: audit 2020-09-08T16:04:45.285726+0000 mon.a (mon.0) 2031 : audit [INF] from='client.? v1:172.21.15.104:0/3434024946' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_5", "pool2": "unique_pool_5", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T16:04:46.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:46 smithi166 bash[9446]: cluster 2020-09-08T16:04:45.285757+0000 mon.a (mon.0) 2032 : cluster [DBG] osdmap e623: 12 total, 12 up, 12 in 2020-09-08T16:04:46.668 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:46.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:04:46 smithi104 bash[22305]: debug 2020-09-08T16:04:46.388+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:46.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:46 smithi104 bash[9203]: cluster 2020-09-08T16:04:45.273438+0000 mon.a (mon.0) 2029 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:04:46.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:46 smithi104 bash[9203]: audit 2020-09-08T16:04:45.285684+0000 mon.a (mon.0) 2030 : audit [INF] from='client.? v1:172.21.15.104:0/2416473847' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:04:46.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:46 smithi104 bash[9203]: audit 2020-09-08T16:04:45.285726+0000 mon.a (mon.0) 2031 : audit [INF] from='client.? v1:172.21.15.104:0/3434024946' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_5", "pool2": "unique_pool_5", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T16:04:46.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:46 smithi104 bash[9203]: cluster 2020-09-08T16:04:45.285757+0000 mon.a (mon.0) 2032 : cluster [DBG] osdmap e623: 12 total, 12 up, 12 in 2020-09-08T16:04:46.774 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:46.892 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:46.998 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:47.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:04:46 smithi166 bash[14254]: debug 2020-09-08T16:04:46.666+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:47.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:04:46 smithi166 bash[16593]: debug 2020-09-08T16:04:46.894+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:47.201 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2020-09-08T16:04:47.204 INFO:tasks.radosbench:Beginning radosbench... 2020-09-08T16:04:47.204 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_6 2020-09-08T16:04:47.205 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool create unique_pool_6 16 2020-09-08T16:04:47.300 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:47 smithi055 bash[9296]: cluster 2020-09-08T16:04:46.026654+0000 mgr.x (mgr.24118) 1123 : cluster [DBG] pgmap v1662: 1 pgs: 1 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:47.300 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:47 smithi055 bash[9296]: cluster 2020-09-08T16:04:46.364791+0000 osd.1 (osd.1) 124 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:47.301 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:47 smithi055 bash[9296]: cluster 2020-09-08T16:04:46.364922+0000 osd.1 (osd.1) 125 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:47.323 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:47.339 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:04:47.428 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:47.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:47 smithi166 bash[9446]: cluster 2020-09-08T16:04:46.026654+0000 mgr.x (mgr.24118) 1123 : cluster [DBG] pgmap v1662: 1 pgs: 1 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:47.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:47 smithi166 bash[9446]: cluster 2020-09-08T16:04:46.364791+0000 osd.1 (osd.1) 124 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:47.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:47 smithi166 bash[9446]: cluster 2020-09-08T16:04:46.364922+0000 osd.1 (osd.1) 125 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:47.656 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:04:47 smithi055 bash[14301]: debug 2020-09-08T16:04:47.325+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:47.657 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:47 smithi104 bash[9203]: cluster 2020-09-08T16:04:46.026654+0000 mgr.x (mgr.24118) 1123 : cluster [DBG] pgmap v1662: 1 pgs: 1 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:47.658 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:47 smithi104 bash[9203]: cluster 2020-09-08T16:04:46.364791+0000 osd.1 (osd.1) 124 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:47.658 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:47 smithi104 bash[9203]: cluster 2020-09-08T16:04:46.364922+0000 osd.1 (osd.1) 125 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:47.878 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:47.932 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:04:47 smithi104 bash[15273]: debug 2020-09-08T16:04:47.876+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:47.988 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:48.416 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:48.522 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:48.612 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:48 smithi055 bash[9296]: cluster 2020-09-08T16:04:47.294272+0000 mon.a (mon.0) 2033 : cluster [DBG] osdmap e624: 12 total, 12 up, 12 in 2020-09-08T16:04:48.613 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:48 smithi055 bash[9296]: audit 2020-09-08T16:04:47.306136+0000 mon.a (mon.0) 2034 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T16:04:48.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:48 smithi166 bash[9446]: cluster 2020-09-08T16:04:47.294272+0000 mon.a (mon.0) 2033 : cluster [DBG] osdmap e624: 12 total, 12 up, 12 in 2020-09-08T16:04:48.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:48 smithi166 bash[9446]: audit 2020-09-08T16:04:47.306136+0000 mon.a (mon.0) 2034 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T16:04:48.680 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:04:48 smithi104 bash[22305]: debug 2020-09-08T16:04:48.416+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:48.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:48 smithi104 bash[9203]: cluster 2020-09-08T16:04:47.294272+0000 mon.a (mon.0) 2033 : cluster [DBG] osdmap e624: 12 total, 12 up, 12 in 2020-09-08T16:04:48.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:48 smithi104 bash[9203]: audit 2020-09-08T16:04:47.306136+0000 mon.a (mon.0) 2034 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "osd", "key": "container_image"}]: dispatch 2020-09-08T16:04:48.767 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T16:04:48.767 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:04:48.977 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:49.082 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:04:49.254 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:04:49.310 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:04:48 smithi104 bash[22305]: debug 2020-09-08T16:04:48.976+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:49.348 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:04:49 smithi166 bash[11928]: debug 2020-09-08T16:04:49.254+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:49.349 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:49 smithi166 bash[9446]: cluster 2020-09-08T16:04:48.027245+0000 mgr.x (mgr.24118) 1124 : cluster [DBG] pgmap v1664: 1 pgs: 1 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:49.364 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:49.373 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T16:04:49.451 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2020-09-08T16:04:49.531 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:04:49.587 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:04:49.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:49 smithi055 bash[9296]: cluster 2020-09-08T16:04:48.027245+0000 mgr.x (mgr.24118) 1124 : cluster [DBG] pgmap v1664: 1 pgs: 1 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:49.688 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:49 smithi104 bash[9203]: cluster 2020-09-08T16:04:48.027245+0000 mgr.x (mgr.24118) 1124 : cluster [DBG] pgmap v1664: 1 pgs: 1 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:49.701 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:49.807 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:49.955 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:04:49 smithi104 bash[17588]: debug 2020-09-08T16:04:49.700+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:49.966 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:50.072 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:50.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:04:49 smithi055 bash[19003]: debug 2020-09-08T16:04:49.969+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:50.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:50 smithi166 bash[9446]: cluster 2020-09-08T16:04:49.132214+0000 osd.6 (osd.6) 107 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:50.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:50 smithi166 bash[9446]: cluster 2020-09-08T16:04:49.132344+0000 osd.6 (osd.6) 108 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:50.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:50 smithi166 bash[9446]: cluster 2020-09-08T16:04:49.327499+0000 mon.a (mon.0) 2035 : cluster [DBG] osdmap e625: 12 total, 12 up, 12 in 2020-09-08T16:04:50.452 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:50.556 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:04:50.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:50 smithi055 bash[9296]: cluster 2020-09-08T16:04:49.132214+0000 osd.6 (osd.6) 107 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:50.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:50 smithi055 bash[9296]: cluster 2020-09-08T16:04:49.132344+0000 osd.6 (osd.6) 108 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:50.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:50 smithi055 bash[9296]: cluster 2020-09-08T16:04:49.327499+0000 mon.a (mon.0) 2035 : cluster [DBG] osdmap e625: 12 total, 12 up, 12 in 2020-09-08T16:04:50.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:50 smithi104 bash[9203]: cluster 2020-09-08T16:04:49.132214+0000 osd.6 (osd.6) 107 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:50.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:50 smithi104 bash[9203]: cluster 2020-09-08T16:04:49.132344+0000 osd.6 (osd.6) 108 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:50.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:50 smithi104 bash[9203]: cluster 2020-09-08T16:04:49.327499+0000 mon.a (mon.0) 2035 : cluster [DBG] osdmap e625: 12 total, 12 up, 12 in 2020-09-08T16:04:50.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:04:50 smithi166 bash[16593]: debug 2020-09-08T16:04:50.450+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:50.928 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:04:51.034 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:51.326 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:04:50 smithi166 bash[11928]: debug 2020-09-08T16:04:50.930+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:51.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:51 smithi166 bash[9446]: cluster 2020-09-08T16:04:50.027893+0000 mgr.x (mgr.24118) 1125 : cluster [DBG] pgmap v1666: 1 pgs: 1 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:51.662 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:51 smithi055 bash[9296]: cluster 2020-09-08T16:04:50.027893+0000 mgr.x (mgr.24118) 1125 : cluster [DBG] pgmap v1666: 1 pgs: 1 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:51.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:51 smithi104 bash[9203]: cluster 2020-09-08T16:04:50.027893+0000 mgr.x (mgr.24118) 1125 : cluster [DBG] pgmap v1666: 1 pgs: 1 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:51.734 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:51.837 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:51.979 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:52.086 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:52.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:04:51 smithi055 bash[11956]: debug 2020-09-08T16:04:51.981+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:52.177 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:04:51 smithi104 bash[17588]: debug 2020-09-08T16:04:51.732+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:52.284 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:52.367 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:04:52 smithi166 bash[16593]: debug 2020-09-08T16:04:52.282+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:52.391 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:52.476 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:04:52.500 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:04:52.523 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:04:52.546 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:04:52.575 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:52.579 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:04:52.606 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:04:52.633 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:04:52.661 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:04:52.679 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:52.685 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:04:52.711 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:04:52.735 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:04:52.764 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:04:52.766 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:04:52.766 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:04:52.767 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:04:52.767 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:04:52.767 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:04:52.767 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:04:52.767 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:04:52.768 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:04:52.768 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:04:52.768 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:04:52.768 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:04:52.768 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:04:52.845 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:04:52 smithi104 bash[15273]: debug 2020-09-08T16:04:52.572+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:53.233 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:53.337 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:04:53.491 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:04:53 smithi104 bash[17588]: debug 2020-09-08T16:04:53.232+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:53.491 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:53 smithi104 bash[9203]: cluster 2020-09-08T16:04:52.028553+0000 mgr.x (mgr.24118) 1126 : cluster [DBG] pgmap v1667: 1 pgs: 1 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:53.491 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:53 smithi104 bash[9203]: audit 2020-09-08T16:04:52.456335+0000 mon.a (mon.0) 2036 : audit [DBG] from='client.? v1:172.21.15.104:0/2238835356' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:04:53.492 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:53 smithi104 bash[9203]: cluster 2020-09-08T16:04:52.678436+0000 osd.11 (osd.11) 104 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:53.492 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:53 smithi104 bash[9203]: cluster 2020-09-08T16:04:52.678572+0000 osd.11 (osd.11) 105 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:53.493 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:04:53.598 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:53.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:53 smithi166 bash[9446]: cluster 2020-09-08T16:04:52.028553+0000 mgr.x (mgr.24118) 1126 : cluster [DBG] pgmap v1667: 1 pgs: 1 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:53.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:53 smithi166 bash[9446]: audit 2020-09-08T16:04:52.456335+0000 mon.a (mon.0) 2036 : audit [DBG] from='client.? v1:172.21.15.104:0/2238835356' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:04:53.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:53 smithi166 bash[9446]: cluster 2020-09-08T16:04:52.678436+0000 osd.11 (osd.11) 104 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:53.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:53 smithi166 bash[9446]: cluster 2020-09-08T16:04:52.678572+0000 osd.11 (osd.11) 105 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:53.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:53 smithi055 bash[9296]: cluster 2020-09-08T16:04:52.028553+0000 mgr.x (mgr.24118) 1126 : cluster [DBG] pgmap v1667: 1 pgs: 1 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:53.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:53 smithi055 bash[9296]: audit 2020-09-08T16:04:52.456335+0000 mon.a (mon.0) 2036 : audit [DBG] from='client.? v1:172.21.15.104:0/2238835356' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:04:53.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:53 smithi055 bash[9296]: cluster 2020-09-08T16:04:52.678436+0000 osd.11 (osd.11) 104 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:53.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:53 smithi055 bash[9296]: cluster 2020-09-08T16:04:52.678572+0000 osd.11 (osd.11) 105 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:53.989 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:04:53 smithi104 bash[19922]: debug 2020-09-08T16:04:53.492+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:54.106 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:04:54.213 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:54.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:54 smithi104 bash[9203]: cluster 2020-09-08T16:04:53.340440+0000 mon.a (mon.0) 2037 : cluster [DBG] osdmap e626: 12 total, 12 up, 12 in 2020-09-08T16:04:54.427 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:04:54 smithi104 bash[22305]: debug 2020-09-08T16:04:54.104+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:54.526 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:54.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:54 smithi166 bash[9446]: cluster 2020-09-08T16:04:53.340440+0000 mon.a (mon.0) 2037 : cluster [DBG] osdmap e626: 12 total, 12 up, 12 in 2020-09-08T16:04:54.632 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:54.645 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:54 smithi055 bash[9296]: cluster 2020-09-08T16:04:53.340440+0000 mon.a (mon.0) 2037 : cluster [DBG] osdmap e626: 12 total, 12 up, 12 in 2020-09-08T16:04:54.761 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:54.867 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:54.874 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:04:54 smithi104 bash[15273]: debug 2020-09-08T16:04:54.524+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:54.906 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:04:54 smithi055 bash[14301]: debug 2020-09-08T16:04:54.765+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:54.996 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:55.103 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:04:55.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:04:55 smithi055 bash[11956]: debug 2020-09-08T16:04:54.997+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:55.452 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:04:55.556 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:55.672 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:04:55.717 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:55 smithi166 bash[9446]: cluster 2020-09-08T16:04:54.029196+0000 mgr.x (mgr.24118) 1127 : cluster [DBG] pgmap v1669: 1 pgs: 1 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:55.718 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:04:55 smithi166 bash[11928]: debug 2020-09-08T16:04:55.450+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:55.778 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:55.896 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:55.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:55 smithi104 bash[9203]: cluster 2020-09-08T16:04:54.029196+0000 mgr.x (mgr.24118) 1127 : cluster [DBG] pgmap v1669: 1 pgs: 1 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:55.927 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:04:55 smithi104 bash[15273]: debug 2020-09-08T16:04:55.672+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:55.968 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:04:55.969 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T16:04:55.969 INFO:teuthology.orchestra.run.smithi166.stdout: "num_ops": 0 2020-09-08T16:04:55.969 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:04:55.988 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'unique_pool_6' created 2020-09-08T16:04:55.990 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T16:04:56.001 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:56.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:55 smithi055 bash[9296]: cluster 2020-09-08T16:04:54.029196+0000 mgr.x (mgr.24118) 1127 : cluster [DBG] pgmap v1669: 1 pgs: 1 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:56.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:04:55 smithi055 bash[19003]: debug 2020-09-08T16:04:55.897+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:56.277 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:56.382 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:56.619 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:04:56 smithi166 bash[16593]: debug 2020-09-08T16:04:56.278+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:56.882 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:56.929 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:04:56 smithi104 bash[17588]: debug 2020-09-08T16:04:56.880+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:56.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:56 smithi104 bash[9203]: audit 2020-09-08T16:04:55.192587+0000 mon.a (mon.0) 2038 : audit [INF] from='client.? v1:172.21.15.104:0/2671760652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_6", "pg_num": 16}]: dispatch 2020-09-08T16:04:56.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:56 smithi104 bash[9203]: audit 2020-09-08T16:04:55.783639+0000 mon.c (mon.2) 82 : audit [INF] from='client.? v1:172.21.15.104:0/2255157445' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:04:56.936 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:56 smithi104 bash[9203]: audit 2020-09-08T16:04:55.786043+0000 mon.a (mon.0) 2039 : audit [INF] from='client.36311 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:04:56.936 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:56 smithi104 bash[9203]: cluster 2020-09-08T16:04:55.978281+0000 mon.a (mon.0) 2040 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:04:56.936 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:56 smithi104 bash[9203]: audit 2020-09-08T16:04:55.985772+0000 mon.a (mon.0) 2041 : audit [INF] from='client.? v1:172.21.15.104:0/2671760652' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_6", "pg_num": 16}]': finished 2020-09-08T16:04:56.937 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:56 smithi104 bash[9203]: audit 2020-09-08T16:04:55.985859+0000 mon.a (mon.0) 2042 : audit [INF] from='client.36311 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:04:56.937 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:56 smithi104 bash[9203]: cluster 2020-09-08T16:04:55.985984+0000 mon.a (mon.0) 2043 : cluster [DBG] osdmap e627: 12 total, 12 up, 12 in 2020-09-08T16:04:56.985 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:57.000 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:56 smithi055 bash[9296]: audit 2020-09-08T16:04:55.192587+0000 mon.a (mon.0) 2038 : audit [INF] from='client.? v1:172.21.15.104:0/2671760652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_6", "pg_num": 16}]: dispatch 2020-09-08T16:04:57.001 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:56 smithi055 bash[9296]: audit 2020-09-08T16:04:55.783639+0000 mon.c (mon.2) 82 : audit [INF] from='client.? v1:172.21.15.104:0/2255157445' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:04:57.001 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:56 smithi055 bash[9296]: audit 2020-09-08T16:04:55.786043+0000 mon.a (mon.0) 2039 : audit [INF] from='client.36311 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:04:57.001 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:56 smithi055 bash[9296]: cluster 2020-09-08T16:04:55.978281+0000 mon.a (mon.0) 2040 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:04:57.001 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:56 smithi055 bash[9296]: audit 2020-09-08T16:04:55.985772+0000 mon.a (mon.0) 2041 : audit [INF] from='client.? v1:172.21.15.104:0/2671760652' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_6", "pg_num": 16}]': finished 2020-09-08T16:04:57.002 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:56 smithi055 bash[9296]: audit 2020-09-08T16:04:55.985859+0000 mon.a (mon.0) 2042 : audit [INF] from='client.36311 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:04:57.002 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:56 smithi055 bash[9296]: cluster 2020-09-08T16:04:55.985984+0000 mon.a (mon.0) 2043 : cluster [DBG] osdmap e627: 12 total, 12 up, 12 in 2020-09-08T16:04:57.114 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:04:57.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:56 smithi166 bash[9446]: audit 2020-09-08T16:04:55.192587+0000 mon.a (mon.0) 2038 : audit [INF] from='client.? v1:172.21.15.104:0/2671760652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_6", "pg_num": 16}]: dispatch 2020-09-08T16:04:57.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:56 smithi166 bash[9446]: audit 2020-09-08T16:04:55.783639+0000 mon.c (mon.2) 82 : audit [INF] from='client.? v1:172.21.15.104:0/2255157445' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:04:57.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:56 smithi166 bash[9446]: audit 2020-09-08T16:04:55.786043+0000 mon.a (mon.0) 2039 : audit [INF] from='client.36311 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:04:57.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:56 smithi166 bash[9446]: cluster 2020-09-08T16:04:55.978281+0000 mon.a (mon.0) 2040 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:04:57.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:56 smithi166 bash[9446]: audit 2020-09-08T16:04:55.985772+0000 mon.a (mon.0) 2041 : audit [INF] from='client.? v1:172.21.15.104:0/2671760652' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_6", "pg_num": 16}]': finished 2020-09-08T16:04:57.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:56 smithi166 bash[9446]: audit 2020-09-08T16:04:55.985859+0000 mon.a (mon.0) 2042 : audit [INF] from='client.36311 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:04:57.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:56 smithi166 bash[9446]: cluster 2020-09-08T16:04:55.985984+0000 mon.a (mon.0) 2043 : cluster [DBG] osdmap e627: 12 total, 12 up, 12 in 2020-09-08T16:04:57.121 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:57.226 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:57.249 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:04:57.319 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2020-09-08T16:04:57.355 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:04:57 smithi055 bash[11956]: debug 2020-09-08T16:04:57.125+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:57.356 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:57.365 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool application enable unique_pool_6 rados --yes-i-really-mean-it || true 2020-09-08T16:04:57.448 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:04:57.466 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:57.496 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:04:57.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:04:57 smithi055 bash[19003]: debug 2020-09-08T16:04:57.357+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:57.810 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:04:57.920 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:57.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:57 smithi104 bash[9203]: cluster 2020-09-08T16:04:56.029787+0000 mgr.x (mgr.24118) 1128 : cluster [DBG] pgmap v1671: 17 pgs: 16 unknown, 1 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:57.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:57 smithi104 bash[9203]: cluster 2020-09-08T16:04:56.994927+0000 mon.a (mon.0) 2044 : cluster [DBG] osdmap e628: 12 total, 12 up, 12 in 2020-09-08T16:04:57.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:04:57 smithi104 bash[17588]: debug 2020-09-08T16:04:57.808+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:57.934 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:57 smithi055 bash[9296]: cluster 2020-09-08T16:04:56.029787+0000 mgr.x (mgr.24118) 1128 : cluster [DBG] pgmap v1671: 17 pgs: 16 unknown, 1 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:57.934 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:57 smithi055 bash[9296]: cluster 2020-09-08T16:04:56.994927+0000 mon.a (mon.0) 2044 : cluster [DBG] osdmap e628: 12 total, 12 up, 12 in 2020-09-08T16:04:58.055 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:58.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:57 smithi166 bash[9446]: cluster 2020-09-08T16:04:56.029787+0000 mgr.x (mgr.24118) 1128 : cluster [DBG] pgmap v1671: 17 pgs: 16 unknown, 1 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:58.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:57 smithi166 bash[9446]: cluster 2020-09-08T16:04:56.994927+0000 mon.a (mon.0) 2044 : cluster [DBG] osdmap e628: 12 total, 12 up, 12 in 2020-09-08T16:04:58.162 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:58.291 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:04:58 smithi055 bash[14301]: debug 2020-09-08T16:04:58.057+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:58.292 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:04:58.397 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:04:58.588 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:04:58.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:04:58 smithi055 bash[19003]: debug 2020-09-08T16:04:58.293+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:58.693 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:58.715 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:04:58 smithi166 bash[11928]: debug 2020-09-08T16:04:58.586+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:58.809 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:04:58.914 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:59.001 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:04:58 smithi055 bash[11956]: debug 2020-09-08T16:04:58.813+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:59.026 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:04:59.132 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:59.319 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:59 smithi104 bash[9203]: cluster 2020-09-08T16:04:57.997392+0000 mon.a (mon.0) 2045 : cluster [DBG] osdmap e629: 12 total, 12 up, 12 in 2020-09-08T16:04:59.320 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:59 smithi104 bash[9203]: cluster 2020-09-08T16:04:58.030393+0000 mgr.x (mgr.24118) 1129 : cluster [DBG] pgmap v1674: 17 pgs: 3 creating+peering, 5 unknown, 9 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:59.321 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:59 smithi104 bash[9203]: audit 2020-09-08T16:04:58.269254+0000 mon.a (mon.0) 2046 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:04:59.321 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:59 smithi104 bash[9203]: cluster 2020-09-08T16:04:58.292628+0000 osd.4 (osd.4) 67 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:59.321 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:59 smithi104 bash[9203]: cluster 2020-09-08T16:04:58.292736+0000 osd.4 (osd.4) 68 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:59.321 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:59 smithi104 bash[9203]: audit 2020-09-08T16:04:58.623237+0000 mon.a (mon.0) 2047 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:04:06.805863\"}"}]: dispatch 2020-09-08T16:04:59.322 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:59 smithi104 bash[9203]: audit 2020-09-08T16:04:58.629344+0000 mon.a (mon.0) 2048 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:04:06.805863\"}"}]': finished 2020-09-08T16:04:59.322 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:59 smithi104 bash[9203]: audit 2020-09-08T16:04:58.631892+0000 mon.a (mon.0) 2049 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:04:59.322 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:04:59 smithi104 bash[9203]: audit 2020-09-08T16:04:58.636609+0000 mon.a (mon.0) 2050 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:04:59.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:59 smithi166 bash[9446]: cluster 2020-09-08T16:04:57.997392+0000 mon.a (mon.0) 2045 : cluster [DBG] osdmap e629: 12 total, 12 up, 12 in 2020-09-08T16:04:59.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:59 smithi166 bash[9446]: cluster 2020-09-08T16:04:58.030393+0000 mgr.x (mgr.24118) 1129 : cluster [DBG] pgmap v1674: 17 pgs: 3 creating+peering, 5 unknown, 9 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:59.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:59 smithi166 bash[9446]: audit 2020-09-08T16:04:58.269254+0000 mon.a (mon.0) 2046 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:04:59.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:59 smithi166 bash[9446]: cluster 2020-09-08T16:04:58.292628+0000 osd.4 (osd.4) 67 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:59.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:59 smithi166 bash[9446]: cluster 2020-09-08T16:04:58.292736+0000 osd.4 (osd.4) 68 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:59.372 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:59 smithi166 bash[9446]: audit 2020-09-08T16:04:58.623237+0000 mon.a (mon.0) 2047 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:04:06.805863\"}"}]: dispatch 2020-09-08T16:04:59.373 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:59 smithi166 bash[9446]: audit 2020-09-08T16:04:58.629344+0000 mon.a (mon.0) 2048 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:04:06.805863\"}"}]': finished 2020-09-08T16:04:59.373 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:59 smithi166 bash[9446]: audit 2020-09-08T16:04:58.631892+0000 mon.a (mon.0) 2049 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:04:59.373 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:04:59 smithi166 bash[9446]: audit 2020-09-08T16:04:58.636609+0000 mon.a (mon.0) 2050 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:04:59.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:04:59 smithi055 bash[14301]: debug 2020-09-08T16:04:59.029+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:59.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:59 smithi055 bash[9296]: cluster 2020-09-08T16:04:57.997392+0000 mon.a (mon.0) 2045 : cluster [DBG] osdmap e629: 12 total, 12 up, 12 in 2020-09-08T16:04:59.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:59 smithi055 bash[9296]: cluster 2020-09-08T16:04:58.030393+0000 mgr.x (mgr.24118) 1129 : cluster [DBG] pgmap v1674: 17 pgs: 3 creating+peering, 5 unknown, 9 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:04:59.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:59 smithi055 bash[9296]: audit 2020-09-08T16:04:58.269254+0000 mon.a (mon.0) 2046 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:04:59.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:59 smithi055 bash[9296]: cluster 2020-09-08T16:04:58.292628+0000 osd.4 (osd.4) 67 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:04:59.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:59 smithi055 bash[9296]: cluster 2020-09-08T16:04:58.292736+0000 osd.4 (osd.4) 68 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:04:59.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:59 smithi055 bash[9296]: audit 2020-09-08T16:04:58.623237+0000 mon.a (mon.0) 2047 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:04:06.805863\"}"}]: dispatch 2020-09-08T16:04:59.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:59 smithi055 bash[9296]: audit 2020-09-08T16:04:58.629344+0000 mon.a (mon.0) 2048 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:04:06.805863\"}"}]': finished 2020-09-08T16:04:59.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:59 smithi055 bash[9296]: audit 2020-09-08T16:04:58.631892+0000 mon.a (mon.0) 2049 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:04:59.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:04:59 smithi055 bash[9296]: audit 2020-09-08T16:04:58.636609+0000 mon.a (mon.0) 2050 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:04:59.697 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:04:59.806 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:04:59.964 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:04:59 smithi166 bash[14254]: debug 2020-09-08T16:04:59.698+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:04:59.965 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:00.022 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T16:05:00.072 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:05:00.153 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:05:00.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:00 smithi166 bash[9446]: cluster 2020-09-08T16:04:59.008811+0000 mon.a (mon.0) 2051 : cluster [DBG] osdmap e630: 12 total, 12 up, 12 in 2020-09-08T16:05:00.367 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:04:59 smithi166 bash[16593]: debug 2020-09-08T16:04:59.966+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:00.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:00 smithi055 bash[9296]: cluster 2020-09-08T16:04:59.008811+0000 mon.a (mon.0) 2051 : cluster [DBG] osdmap e630: 12 total, 12 up, 12 in 2020-09-08T16:05:00.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:00 smithi104 bash[9203]: cluster 2020-09-08T16:04:59.008811+0000 mon.a (mon.0) 2051 : cluster [DBG] osdmap e630: 12 total, 12 up, 12 in 2020-09-08T16:05:00.430 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:05:00.532 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:05:00.871 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:05:00 smithi166 bash[14254]: debug 2020-09-08T16:05:00.414+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:00.965 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:05:01.071 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:05:01.270 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:01 smithi166 bash[9446]: cluster 2020-09-08T16:05:00.030915+0000 mgr.x (mgr.24118) 1130 : cluster [DBG] pgmap v1676: 17 pgs: 3 creating+peering, 14 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:05:01.270 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:05:00 smithi166 bash[14254]: debug 2020-09-08T16:05:00.962+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:01.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:01 smithi055 bash[9296]: cluster 2020-09-08T16:05:00.030915+0000 mgr.x (mgr.24118) 1130 : cluster [DBG] pgmap v1676: 17 pgs: 3 creating+peering, 14 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:05:01.422 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:05:01.475 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:01 smithi104 bash[9203]: cluster 2020-09-08T16:05:00.030915+0000 mgr.x (mgr.24118) 1130 : cluster [DBG] pgmap v1676: 17 pgs: 3 creating+peering, 14 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:05:01.496 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:05:01.497 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T16:05:01.497 INFO:teuthology.orchestra.run.smithi166.stdout: "complaint_time": 30, 2020-09-08T16:05:01.497 INFO:teuthology.orchestra.run.smithi166.stdout: "num_blocked_ops": 0 2020-09-08T16:05:01.497 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:05:01.533 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:05:01.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:05:01 smithi104 bash[19922]: debug 2020-09-08T16:05:01.420+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:02.171 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:05:02.277 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:02.309 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2020-09-08T16:05:02.406 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:02.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:05:02 smithi104 bash[22305]: debug 2020-09-08T16:05:02.172+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:02.488 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:05:02.513 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:05:02.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:05:02 smithi166 bash[11928]: debug 2020-09-08T16:05:02.406+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:02.762 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:05:02.800 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:05:02 smithi104 bash[19922]: debug 2020-09-08T16:05:02.760+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:02.866 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:02.992 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:03.098 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:03.156 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:03 smithi055 bash[9296]: cluster 2020-09-08T16:05:02.031588+0000 mgr.x (mgr.24118) 1131 : cluster [DBG] pgmap v1677: 17 pgs: 17 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:05:03.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:05:02 smithi055 bash[11956]: debug 2020-09-08T16:05:02.993+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:03.349 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:03 smithi166 bash[9446]: cluster 2020-09-08T16:05:02.031588+0000 mgr.x (mgr.24118) 1131 : cluster [DBG] pgmap v1677: 17 pgs: 17 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:05:03.350 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:03.432 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:03 smithi104 bash[9203]: cluster 2020-09-08T16:05:02.031588+0000 mgr.x (mgr.24118) 1131 : cluster [DBG] pgmap v1677: 17 pgs: 17 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:05:03.454 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:03.615 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:05:03 smithi166 bash[16593]: debug 2020-09-08T16:05:03.350+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:03.616 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:03.721 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:03.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:05:03 smithi166 bash[11928]: debug 2020-09-08T16:05:03.614+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:03.951 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:03.955 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:05:03.977 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:05:03.998 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:05:04.028 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:05:04.056 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:05:04.057 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:05:04.086 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:05:04.101 INFO:teuthology.orchestra.run.smithi104.stderr:enabled application 'rados' on pool 'unique_pool_6' 2020-09-08T16:05:04.113 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:05:04.146 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:05:04.171 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:05:04.197 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:05:04.225 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:05:04.259 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:05:04.261 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:05:04.262 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:05:04.262 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:05:04.262 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:05:04.262 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:05:04.263 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:05:04.263 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:05:04.263 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:05:04.263 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:05:04.263 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:05:04.264 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:05:04.264 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:05:04.339 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:05:03 smithi104 bash[15273]: debug 2020-09-08T16:05:03.952+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:04.339 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:04 smithi104 bash[9203]: audit 2020-09-08T16:05:03.931914+0000 mon.a (mon.0) 2052 : audit [DBG] from='client.? v1:172.21.15.104:0/1581717671' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:05:04.340 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:04 smithi104 bash[9203]: audit 2020-09-08T16:05:04.044934+0000 mon.a (mon.0) 2053 : audit [INF] from='client.? v1:172.21.15.104:0/4085270777' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_6", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T16:05:04.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:04 smithi166 bash[9446]: audit 2020-09-08T16:05:03.931914+0000 mon.a (mon.0) 2052 : audit [DBG] from='client.? v1:172.21.15.104:0/1581717671' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:05:04.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:04 smithi166 bash[9446]: audit 2020-09-08T16:05:04.044934+0000 mon.a (mon.0) 2053 : audit [INF] from='client.? v1:172.21.15.104:0/4085270777' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_6", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T16:05:04.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:04 smithi055 bash[9296]: audit 2020-09-08T16:05:03.931914+0000 mon.a (mon.0) 2052 : audit [DBG] from='client.? v1:172.21.15.104:0/1581717671' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:05:04.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:04 smithi055 bash[9296]: audit 2020-09-08T16:05:04.044934+0000 mon.a (mon.0) 2053 : audit [INF] from='client.? v1:172.21.15.104:0/4085270777' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_6", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T16:05:04.572 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:05:04.676 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:05:04.797 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:05:04.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:05:04 smithi166 bash[18960]: debug 2020-09-08T16:05:04.570+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:04.903 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:05.101 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:05:04 smithi055 bash[16646]: debug 2020-09-08T16:05:04.801+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:05.188 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:05 smithi166 bash[9446]: cluster 2020-09-08T16:05:04.032339+0000 mgr.x (mgr.24118) 1132 : cluster [DBG] pgmap v1678: 17 pgs: 17 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:05:05.188 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:05 smithi166 bash[9446]: audit 2020-09-08T16:05:04.104755+0000 mon.a (mon.0) 2054 : audit [INF] from='client.? v1:172.21.15.104:0/4085270777' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_6", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T16:05:05.188 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:05 smithi166 bash[9446]: cluster 2020-09-08T16:05:04.104812+0000 mon.a (mon.0) 2055 : cluster [DBG] osdmap e631: 12 total, 12 up, 12 in 2020-09-08T16:05:05.189 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:05.292 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:05.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:05 smithi055 bash[9296]: cluster 2020-09-08T16:05:04.032339+0000 mgr.x (mgr.24118) 1132 : cluster [DBG] pgmap v1678: 17 pgs: 17 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:05:05.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:05 smithi055 bash[9296]: audit 2020-09-08T16:05:04.104755+0000 mon.a (mon.0) 2054 : audit [INF] from='client.? v1:172.21.15.104:0/4085270777' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_6", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T16:05:05.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:05 smithi055 bash[9296]: cluster 2020-09-08T16:05:04.104812+0000 mon.a (mon.0) 2055 : cluster [DBG] osdmap e631: 12 total, 12 up, 12 in 2020-09-08T16:05:05.407 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:05.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:05 smithi104 bash[9203]: cluster 2020-09-08T16:05:04.032339+0000 mgr.x (mgr.24118) 1132 : cluster [DBG] pgmap v1678: 17 pgs: 17 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:05:05.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:05 smithi104 bash[9203]: audit 2020-09-08T16:05:04.104755+0000 mon.a (mon.0) 2054 : audit [INF] from='client.? v1:172.21.15.104:0/4085270777' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_6", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T16:05:05.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:05 smithi104 bash[9203]: cluster 2020-09-08T16:05:04.104812+0000 mon.a (mon.0) 2055 : cluster [DBG] osdmap e631: 12 total, 12 up, 12 in 2020-09-08T16:05:05.512 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:05.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:05:05 smithi166 bash[16593]: debug 2020-09-08T16:05:05.186+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:05.631 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:05.663 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:05:05 smithi055 bash[11956]: debug 2020-09-08T16:05:05.409+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:05.663 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:05:05 smithi055 bash[11956]: debug 2020-09-08T16:05:05.633+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:05.736 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:05:05.924 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:05:06.035 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:06.036 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:05:05 smithi166 bash[18960]: debug 2020-09-08T16:05:05.922+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:06.160 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:06.267 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:06.271 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T16:05:06.271 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:05:06.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:05:06 smithi055 bash[11956]: debug 2020-09-08T16:05:06.161+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:06.427 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:06.529 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:05:06.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:05:06 smithi166 bash[16593]: debug 2020-09-08T16:05:06.422+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:06.679 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:05:06.784 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:05:06.807 INFO:teuthology.orchestra.run.smithi166:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.2 -b 65536 --object-size 65536 -p unique_pool_6 bench 90 write' 2020-09-08T16:05:06.833 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2020-09-08T16:05:06.834 INFO:teuthology.orchestra.run:waiting for 3000 2020-09-08T16:05:06.866 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:hints = 1 2020-09-08T16:05:06.866 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2020-09-08T16:05:06.867 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Object prefix: benchmark_data_smithi166_52302 2020-09-08T16:05:06.867 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T16:05:06.867 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 0 0 0 0 0 0 - 0 2020-09-08T16:05:06.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:05:06 smithi104 bash[17588]: debug 2020-09-08T16:05:06.680+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:07.022 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:05:07.105 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:05:07 smithi166 bash[14254]: debug 2020-09-08T16:05:07.022+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:07.116 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T16:05:07.133 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:05:07.134 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_ops_in_flight 2020-09-08T16:05:07.271 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:05:07.327 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:05:07.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:07 smithi166 bash[9446]: cluster 2020-09-08T16:05:06.032986+0000 mgr.x (mgr.24118) 1133 : cluster [DBG] pgmap v1680: 17 pgs: 17 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:05:07.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:07 smithi166 bash[9446]: audit 2020-09-08T16:05:06.510437+0000 mon.a (mon.0) 2056 : audit [INF] from='client.? v1:172.21.15.104:0/813694677' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:05:07.376 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:05:07.388 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:07 smithi055 bash[9296]: cluster 2020-09-08T16:05:06.032986+0000 mgr.x (mgr.24118) 1133 : cluster [DBG] pgmap v1680: 17 pgs: 17 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:05:07.389 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:07 smithi055 bash[9296]: audit 2020-09-08T16:05:06.510437+0000 mon.a (mon.0) 2056 : audit [INF] from='client.? v1:172.21.15.104:0/813694677' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:05:07.389 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:05:07 smithi055 bash[14301]: debug 2020-09-08T16:05:07.273+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:07.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:07 smithi104 bash[9203]: cluster 2020-09-08T16:05:06.032986+0000 mgr.x (mgr.24118) 1133 : cluster [DBG] pgmap v1680: 17 pgs: 17 active+clean; 0 B data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:05:07.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:07 smithi104 bash[9203]: audit 2020-09-08T16:05:06.510437+0000 mon.a (mon.0) 2056 : audit [INF] from='client.? v1:172.21.15.104:0/813694677' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:05:07.507 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:05:07.612 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:07.659 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:05:07 smithi055 bash[14301]: debug 2020-09-08T16:05:07.509+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:07.866 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 1 16 3371 3355 209.676 209.688 0.00576857 0.00475035 2020-09-08T16:05:08.061 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:05:08.235 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:05:08.358 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:08 smithi104 bash[9203]: cluster 2020-09-08T16:05:07.108181+0000 mon.a (mon.0) 2057 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:05:08.359 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:08 smithi104 bash[9203]: audit 2020-09-08T16:05:07.118059+0000 mon.a (mon.0) 2058 : audit [INF] from='client.? v1:172.21.15.104:0/813694677' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:05:08.359 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:08 smithi104 bash[9203]: cluster 2020-09-08T16:05:07.118122+0000 mon.a (mon.0) 2059 : cluster [DBG] osdmap e632: 12 total, 12 up, 12 in 2020-09-08T16:05:08.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:08 smithi166 bash[9446]: cluster 2020-09-08T16:05:07.108181+0000 mon.a (mon.0) 2057 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:05:08.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:08 smithi166 bash[9446]: audit 2020-09-08T16:05:07.118059+0000 mon.a (mon.0) 2058 : audit [INF] from='client.? v1:172.21.15.104:0/813694677' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:05:08.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:08 smithi166 bash[9446]: cluster 2020-09-08T16:05:07.118122+0000 mon.a (mon.0) 2059 : cluster [DBG] osdmap e632: 12 total, 12 up, 12 in 2020-09-08T16:05:08.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:08 smithi055 bash[9296]: cluster 2020-09-08T16:05:07.108181+0000 mon.a (mon.0) 2057 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:05:08.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:08 smithi055 bash[9296]: audit 2020-09-08T16:05:07.118059+0000 mon.a (mon.0) 2058 : audit [INF] from='client.? v1:172.21.15.104:0/813694677' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:05:08.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:08 smithi055 bash[9296]: cluster 2020-09-08T16:05:07.118122+0000 mon.a (mon.0) 2059 : cluster [DBG] osdmap e632: 12 total, 12 up, 12 in 2020-09-08T16:05:08.868 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 2 16 6384 6368 198.984 188.312 0.00901196 0.00501881 2020-09-08T16:05:09.265 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:09.369 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:05:09.379 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:09 smithi055 bash[9296]: cluster 2020-09-08T16:05:08.033692+0000 mgr.x (mgr.24118) 1134 : cluster [DBG] pgmap v1682: 17 pgs: 17 active+clean; 92 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s wr, 183 op/s 2020-09-08T16:05:09.380 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:09 smithi055 bash[9296]: cluster 2020-09-08T16:05:08.456925+0000 osd.9 (osd.9) 95 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:09.380 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:09 smithi055 bash[9296]: cluster 2020-09-08T16:05:08.457003+0000 osd.9 (osd.9) 96 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:09.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:09 smithi104 bash[9203]: cluster 2020-09-08T16:05:08.033692+0000 mgr.x (mgr.24118) 1134 : cluster [DBG] pgmap v1682: 17 pgs: 17 active+clean; 92 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s wr, 183 op/s 2020-09-08T16:05:09.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:09 smithi104 bash[9203]: cluster 2020-09-08T16:05:08.456925+0000 osd.9 (osd.9) 95 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:09.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:09 smithi104 bash[9203]: cluster 2020-09-08T16:05:08.457003+0000 osd.9 (osd.9) 96 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:09.512 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:05:09.616 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:09.618 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:05:09 smithi166 bash[16593]: debug 2020-09-08T16:05:09.262+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:09.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:09 smithi166 bash[9446]: cluster 2020-09-08T16:05:08.033692+0000 mgr.x (mgr.24118) 1134 : cluster [DBG] pgmap v1682: 17 pgs: 17 active+clean; 92 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s wr, 183 op/s 2020-09-08T16:05:09.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:09 smithi166 bash[9446]: cluster 2020-09-08T16:05:08.456925+0000 osd.9 (osd.9) 95 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:09.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:09 smithi166 bash[9446]: cluster 2020-09-08T16:05:08.457003+0000 osd.9 (osd.9) 96 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:09.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:05:09 smithi055 bash[16646]: debug 2020-09-08T16:05:09.513+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:09.866 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 3 16 9756 9740 202.899 210.75 0.00757307 0.00492375 2020-09-08T16:05:10.361 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T16:05:10.415 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:10 smithi055 bash[9296]: cluster 2020-09-08T16:05:09.133405+0000 mon.a (mon.0) 2060 : cluster [DBG] osdmap e633: 12 total, 12 up, 12 in 2020-09-08T16:05:10.415 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:10 smithi055 bash[9296]: cluster 2020-09-08T16:05:09.346739+0000 osd.2 (osd.2) 102 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:10.416 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:10 smithi055 bash[9296]: cluster 2020-09-08T16:05:09.346850+0000 osd.2 (osd.2) 103 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:10.416 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:10 smithi055 bash[9296]: cluster 2020-09-08T16:05:09.978791+0000 mon.a (mon.0) 2061 : cluster [DBG] osdmap e634: 12 total, 12 up, 12 in 2020-09-08T16:05:10.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:10 smithi104 bash[9203]: cluster 2020-09-08T16:05:09.133405+0000 mon.a (mon.0) 2060 : cluster [DBG] osdmap e633: 12 total, 12 up, 12 in 2020-09-08T16:05:10.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:10 smithi104 bash[9203]: cluster 2020-09-08T16:05:09.346739+0000 osd.2 (osd.2) 102 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:10.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:10 smithi104 bash[9203]: cluster 2020-09-08T16:05:09.346850+0000 osd.2 (osd.2) 103 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:10.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:10 smithi104 bash[9203]: cluster 2020-09-08T16:05:09.978791+0000 mon.a (mon.0) 2061 : cluster [DBG] osdmap e634: 12 total, 12 up, 12 in 2020-09-08T16:05:10.532 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:05:10.626 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:10 smithi166 bash[9446]: cluster 2020-09-08T16:05:09.133405+0000 mon.a (mon.0) 2060 : cluster [DBG] osdmap e633: 12 total, 12 up, 12 in 2020-09-08T16:05:10.627 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:10 smithi166 bash[9446]: cluster 2020-09-08T16:05:09.346739+0000 osd.2 (osd.2) 102 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:10.627 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:10 smithi166 bash[9446]: cluster 2020-09-08T16:05:09.346850+0000 osd.2 (osd.2) 103 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:10.627 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:10 smithi166 bash[9446]: cluster 2020-09-08T16:05:09.978791+0000 mon.a (mon.0) 2061 : cluster [DBG] osdmap e634: 12 total, 12 up, 12 in 2020-09-08T16:05:10.867 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 4 16 11886 11870 185.452 133.125 0.00561546 0.00529575 2020-09-08T16:05:11.077 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:11.181 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:05:11.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:11 smithi055 bash[9296]: cluster 2020-09-08T16:05:10.034148+0000 mgr.x (mgr.24118) 1135 : cluster [DBG] pgmap v1685: 17 pgs: 17 active+clean; 171 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s wr, 461 op/s 2020-09-08T16:05:11.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:05:11 smithi055 bash[11956]: debug 2020-09-08T16:05:11.077+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:11.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:11 smithi104 bash[9203]: cluster 2020-09-08T16:05:10.034148+0000 mgr.x (mgr.24118) 1135 : cluster [DBG] pgmap v1685: 17 pgs: 17 active+clean; 171 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s wr, 461 op/s 2020-09-08T16:05:11.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:11 smithi166 bash[9446]: cluster 2020-09-08T16:05:10.034148+0000 mgr.x (mgr.24118) 1135 : cluster [DBG] pgmap v1685: 17 pgs: 17 active+clean; 171 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s wr, 461 op/s 2020-09-08T16:05:11.730 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:05:11.833 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:05:11.868 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 5 15 14989 14974 187.158 194 0.00467826 0.00533869 2020-09-08T16:05:11.967 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:05:11.995 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:05:11 smithi166 bash[18960]: debug 2020-09-08T16:05:11.730+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:12.070 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:05:12.156 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:05:11 smithi055 bash[14301]: debug 2020-09-08T16:05:11.969+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:12.185 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:05:12.288 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:05:12.333 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:05:12 smithi166 bash[18960]: debug 2020-09-08T16:05:12.186+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:12.410 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:05:12.515 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:05:12.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:05:12 smithi055 bash[16646]: debug 2020-09-08T16:05:12.413+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:12.867 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 6 16 17334 17318 180.379 146.5 0.00615321 0.00554025 2020-09-08T16:05:13.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:13 smithi055 bash[9296]: cluster 2020-09-08T16:05:12.034743+0000 mgr.x (mgr.24118) 1136 : cluster [DBG] pgmap v1686: 17 pgs: 17 active+clean; 346 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s wr, 921 op/s 2020-09-08T16:05:13.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:13 smithi104 bash[9203]: cluster 2020-09-08T16:05:12.034743+0000 mgr.x (mgr.24118) 1136 : cluster [DBG] pgmap v1686: 17 pgs: 17 active+clean; 346 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s wr, 921 op/s 2020-09-08T16:05:13.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:13 smithi166 bash[9446]: cluster 2020-09-08T16:05:12.034743+0000 mgr.x (mgr.24118) 1136 : cluster [DBG] pgmap v1686: 17 pgs: 17 active+clean; 346 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s wr, 921 op/s 2020-09-08T16:05:13.867 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 7 15 20166 20151 179.903 177.062 0.00394838 0.00555512 2020-09-08T16:05:14.433 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:05:14.538 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:14.774 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:14.775 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:05:14 smithi166 bash[14254]: debug 2020-09-08T16:05:14.434+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:14.881 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:05:14.882 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 8 16 22799 22783 177.881 164.5 0.00447898 0.00561458 2020-09-08T16:05:14.993 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:05:15.097 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:05:15.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:05:14 smithi166 bash[16593]: debug 2020-09-08T16:05:14.774+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:15.145 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:05:14 smithi055 bash[16646]: debug 2020-09-08T16:05:14.993+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:15.317 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:05:15.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:15 smithi055 bash[9296]: cluster 2020-09-08T16:05:14.035372+0000 mgr.x (mgr.24118) 1137 : cluster [DBG] pgmap v1687: 17 pgs: 17 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 153 MiB/s wr, 2.45k op/s 2020-09-08T16:05:15.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:15 smithi055 bash[9296]: cluster 2020-09-08T16:05:14.981009+0000 mon.a (mon.0) 2062 : cluster [DBG] osdmap e635: 12 total, 12 up, 12 in 2020-09-08T16:05:15.423 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:05:15.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:05:15 smithi104 bash[17588]: debug 2020-09-08T16:05:15.316+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:15.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:15 smithi104 bash[9203]: cluster 2020-09-08T16:05:14.035372+0000 mgr.x (mgr.24118) 1137 : cluster [DBG] pgmap v1687: 17 pgs: 17 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 153 MiB/s wr, 2.45k op/s 2020-09-08T16:05:15.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:15 smithi104 bash[9203]: cluster 2020-09-08T16:05:14.981009+0000 mon.a (mon.0) 2062 : cluster [DBG] osdmap e635: 12 total, 12 up, 12 in 2020-09-08T16:05:15.667 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:15 smithi166 bash[9446]: cluster 2020-09-08T16:05:14.035372+0000 mgr.x (mgr.24118) 1137 : cluster [DBG] pgmap v1687: 17 pgs: 17 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 153 MiB/s wr, 2.45k op/s 2020-09-08T16:05:15.667 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:15 smithi166 bash[9446]: cluster 2020-09-08T16:05:14.981009+0000 mon.a (mon.0) 2062 : cluster [DBG] osdmap e635: 12 total, 12 up, 12 in 2020-09-08T16:05:15.871 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 9 16 25167 25151 174.562 148 0.00568111 0.00572528 2020-09-08T16:05:16.703 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:05:16.807 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:05:16.871 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 10 16 27856 27840 173.911 168.062 0.00449588 0.00574726 2020-09-08T16:05:16.926 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:05:16.984 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:05:16 smithi104 bash[19922]: debug 2020-09-08T16:05:16.700+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:17.035 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:17.145 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:05:16 smithi055 bash[19003]: debug 2020-09-08T16:05:16.929+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:17.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:17 smithi166 bash[9446]: cluster 2020-09-08T16:05:16.035858+0000 mgr.x (mgr.24118) 1138 : cluster [DBG] pgmap v1689: 17 pgs: 17 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s wr, 2.06k op/s 2020-09-08T16:05:17.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:17 smithi166 bash[9446]: cluster 2020-09-08T16:05:16.438883+0000 osd.7 (osd.7) 137 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:17.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:17 smithi166 bash[9446]: cluster 2020-09-08T16:05:16.438980+0000 osd.7 (osd.7) 138 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:17.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:17 smithi055 bash[9296]: cluster 2020-09-08T16:05:16.035858+0000 mgr.x (mgr.24118) 1138 : cluster [DBG] pgmap v1689: 17 pgs: 17 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s wr, 2.06k op/s 2020-09-08T16:05:17.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:17 smithi055 bash[9296]: cluster 2020-09-08T16:05:16.438883+0000 osd.7 (osd.7) 137 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:17.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:17 smithi055 bash[9296]: cluster 2020-09-08T16:05:16.438980+0000 osd.7 (osd.7) 138 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:17.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:17 smithi104 bash[9203]: cluster 2020-09-08T16:05:16.035858+0000 mgr.x (mgr.24118) 1138 : cluster [DBG] pgmap v1689: 17 pgs: 17 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s wr, 2.06k op/s 2020-09-08T16:05:17.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:17 smithi104 bash[9203]: cluster 2020-09-08T16:05:16.438883+0000 osd.7 (osd.7) 137 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:17.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:17 smithi104 bash[9203]: cluster 2020-09-08T16:05:16.438980+0000 osd.7 (osd.7) 138 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:17.871 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 11 15 29750 29735 168.869 118.438 0.00869939 0.00591891 2020-09-08T16:05:18.123 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:18.166 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:05:18.201 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:05:18.225 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:18.234 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:05:18.260 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:05:18.289 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:05:18.325 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:05:18.360 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:05:18.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:18 smithi166 bash[9446]: cluster 2020-09-08T16:05:17.162127+0000 mon.a (mon.0) 2063 : cluster [DBG] osdmap e636: 12 total, 12 up, 12 in 2020-09-08T16:05:18.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:18 smithi166 bash[9446]: cluster 2020-09-08T16:05:17.359131+0000 osd.0 (osd.0) 114 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:18.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:18 smithi166 bash[9446]: cluster 2020-09-08T16:05:17.359225+0000 osd.0 (osd.0) 115 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:18.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:18 smithi166 bash[9446]: audit 2020-09-08T16:05:18.134015+0000 mon.c (mon.2) 83 : audit [DBG] from='client.? v1:172.21.15.104:0/1857873093' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:05:18.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:05:18 smithi166 bash[11928]: debug 2020-09-08T16:05:18.114+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:18.382 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:18.400 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:05:18.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:18 smithi104 bash[9203]: cluster 2020-09-08T16:05:17.162127+0000 mon.a (mon.0) 2063 : cluster [DBG] osdmap e636: 12 total, 12 up, 12 in 2020-09-08T16:05:18.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:18 smithi104 bash[9203]: cluster 2020-09-08T16:05:17.359131+0000 osd.0 (osd.0) 114 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:18.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:18 smithi104 bash[9203]: cluster 2020-09-08T16:05:17.359225+0000 osd.0 (osd.0) 115 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:18.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:18 smithi104 bash[9203]: audit 2020-09-08T16:05:18.134015+0000 mon.c (mon.2) 83 : audit [DBG] from='client.? v1:172.21.15.104:0/1857873093' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:05:18.429 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:05:18.460 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:05:18.487 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:05:18.494 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:05:18.536 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:05:18.538 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:05:18.538 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:05:18.538 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:05:18.539 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:05:18.539 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:05:18.539 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:05:18.539 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:05:18.539 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:05:18.539 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:05:18.540 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:05:18.540 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:05:18.540 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:05:18.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:18 smithi055 bash[9296]: cluster 2020-09-08T16:05:17.162127+0000 mon.a (mon.0) 2063 : cluster [DBG] osdmap e636: 12 total, 12 up, 12 in 2020-09-08T16:05:18.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:18 smithi055 bash[9296]: cluster 2020-09-08T16:05:17.359131+0000 osd.0 (osd.0) 114 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:18.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:18 smithi055 bash[9296]: cluster 2020-09-08T16:05:17.359225+0000 osd.0 (osd.0) 115 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:18.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:18 smithi055 bash[9296]: audit 2020-09-08T16:05:18.134015+0000 mon.c (mon.2) 83 : audit [DBG] from='client.? v1:172.21.15.104:0/1857873093' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:05:18.698 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:05:18.698 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T16:05:18.698 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:18.699 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:31516 8.c 8:30ba4d66:::benchmark_data_smithi166_52302_object31515:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:05:18.699 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:05:18.695871+0000", 2020-09-08T16:05:18.699 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 0.0053968540000000004, 2020-09-08T16:05:18.699 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0054265379999999998, 2020-09-08T16:05:18.700 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:05:18.700 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:05:18.700 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:05:18.700 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:05:18.701 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:05:18.701 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 31516 2020-09-08T16:05:18.701 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:18.701 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:05:18.702 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:18.702 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:05:18.702 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:18.695871+0000", 2020-09-08T16:05:18.702 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:05:18.702 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:18.703 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:18.703 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:05:18.703 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:18.695871+0000", 2020-09-08T16:05:18.703 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999633 2020-09-08T16:05:18.703 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:18.704 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:18.704 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:05:18.704 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:18.695834+0000", 2020-09-08T16:05:18.704 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013415700000000001 2020-09-08T16:05:18.705 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:18.705 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:18.705 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:05:18.705 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:18.695968+0000", 2020-09-08T16:05:18.705 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4555e-05 2020-09-08T16:05:18.706 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:18.706 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:18.706 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:05:18.706 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:18.695983+0000", 2020-09-08T16:05:18.707 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.9440000000000002e-06 2020-09-08T16:05:18.707 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:18.707 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:18.707 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:05:18.707 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:18.695989+0000", 2020-09-08T16:05:18.708 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0011173120000000001 2020-09-08T16:05:18.708 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:18.708 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:18.708 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:05:18.708 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:18.697106+0000", 2020-09-08T16:05:18.709 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012948199999999999 2020-09-08T16:05:18.709 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:18.709 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:18.709 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:05:18.710 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:18.697236+0000", 2020-09-08T16:05:18.710 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015965900000000001 2020-09-08T16:05:18.710 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:18.710 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:18.710 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 8", 2020-09-08T16:05:18.711 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:18.697395+0000", 2020-09-08T16:05:18.711 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0034950250000000001 2020-09-08T16:05:18.711 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:18.712 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:18.712 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:05:18.712 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:18.700890+0000", 2020-09-08T16:05:18.712 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0050192359999999998 2020-09-08T16:05:18.712 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:05:18.712 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:05:18.713 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:05:18.713 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:18.713 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:18.713 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:31527 8.c 8.d5f26c1c (undecoded) ondisk+write+known_if_redirected e637)", 2020-09-08T16:05:18.713 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:05:18.701112+0000", 2020-09-08T16:05:18.714 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 0.000156377, 2020-09-08T16:05:18.714 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00028654599999999999, 2020-09-08T16:05:18.714 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:05:18.714 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "queued for pg", 2020-09-08T16:05:18.714 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:05:18.715 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:05:18.715 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:05:18.715 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 31527 2020-09-08T16:05:18.715 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:18.715 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:05:18.716 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:18.716 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:05:18.716 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:18.701112+0000", 2020-09-08T16:05:18.716 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:05:18.716 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:18.717 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:18.717 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:05:18.717 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:18.701112+0000", 2020-09-08T16:05:18.717 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.999979 2020-09-08T16:05:18.717 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:18.717 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:18.718 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:05:18.718 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:18.701091+0000", 2020-09-08T16:05:18.718 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.1230000000000007e-05 2020-09-08T16:05:18.718 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:18.718 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:18.719 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:05:18.719 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:18.701172+0000", 2020-09-08T16:05:18.719 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.738e-05 2020-09-08T16:05:18.719 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:18.719 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:18.720 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:05:18.720 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:18.701189+0000", 2020-09-08T16:05:18.720 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.9030000000000001e-06 2020-09-08T16:05:18.721 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:18.722 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:18.722 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:05:18.722 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:18.701195+0000", 2020-09-08T16:05:18.722 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00020690899999999999 2020-09-08T16:05:18.723 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:18.723 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:18.723 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:05:18.723 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:18.701402+0000", 2020-09-08T16:05:18.723 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00029063 2020-09-08T16:05:18.724 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:05:18.724 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:05:18.724 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:05:18.724 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:05:18.724 INFO:teuthology.orchestra.run.smithi166.stdout: ], 2020-09-08T16:05:18.724 INFO:teuthology.orchestra.run.smithi166.stdout: "num_ops": 2 2020-09-08T16:05:18.725 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:05:18.785 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:05:18 smithi166 bash[16593]: debug 2020-09-08T16:05:18.382+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:18.872 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 12 16 31874 31858 165.854 132.688 0.00431766 0.0060041 2020-09-08T16:05:19.273 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:05:19.379 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:19.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:05:19 smithi104 bash[17588]: debug 2020-09-08T16:05:19.272+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:19.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:19 smithi104 bash[9203]: cluster 2020-09-08T16:05:18.036443+0000 mgr.x (mgr.24118) 1139 : cluster [DBG] pgmap v1691: 17 pgs: 17 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 180 MiB/s wr, 2.88k op/s 2020-09-08T16:05:19.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:19 smithi104 bash[9203]: cluster 2020-09-08T16:05:18.169511+0000 mon.a (mon.0) 2064 : cluster [DBG] osdmap e637: 12 total, 12 up, 12 in 2020-09-08T16:05:19.522 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:19 smithi166 bash[9446]: cluster 2020-09-08T16:05:18.036443+0000 mgr.x (mgr.24118) 1139 : cluster [DBG] pgmap v1691: 17 pgs: 17 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 180 MiB/s wr, 2.88k op/s 2020-09-08T16:05:19.523 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:19 smithi166 bash[9446]: cluster 2020-09-08T16:05:18.169511+0000 mon.a (mon.0) 2064 : cluster [DBG] osdmap e637: 12 total, 12 up, 12 in 2020-09-08T16:05:19.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:19 smithi055 bash[9296]: cluster 2020-09-08T16:05:18.036443+0000 mgr.x (mgr.24118) 1139 : cluster [DBG] pgmap v1691: 17 pgs: 17 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 180 MiB/s wr, 2.88k op/s 2020-09-08T16:05:19.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:19 smithi055 bash[9296]: cluster 2020-09-08T16:05:18.169511+0000 mon.a (mon.0) 2064 : cluster [DBG] osdmap e637: 12 total, 12 up, 12 in 2020-09-08T16:05:19.872 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 13 16 32800 32784 157.55 57.875 0.00369526 0.00634395 2020-09-08T16:05:20.554 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:20.667 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:05:20.873 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:05:20 smithi166 bash[11928]: debug 2020-09-08T16:05:20.554+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:20.874 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 14 16 35043 35027 156.309 140.188 0.00719036 0.00639441 2020-09-08T16:05:20.907 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:05:20.910 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_blocked_ops 2020-09-08T16:05:20.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:05:20 smithi104 bash[19922]: debug 2020-09-08T16:05:20.904+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:21.012 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:05:21.082 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:05:21.159 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:05:21.263 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:05:21.280 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:20 smithi055 bash[9296]: cluster 2020-09-08T16:05:19.981300+0000 mon.a (mon.0) 2065 : cluster [DBG] osdmap e638: 12 total, 12 up, 12 in 2020-09-08T16:05:21.280 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:20 smithi055 bash[9296]: cluster 2020-09-08T16:05:20.036958+0000 mgr.x (mgr.24118) 1140 : cluster [DBG] pgmap v1694: 17 pgs: 17 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 155 MiB/s wr, 2.48k op/s 2020-09-08T16:05:21.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:20 smithi166 bash[9446]: cluster 2020-09-08T16:05:19.981300+0000 mon.a (mon.0) 2065 : cluster [DBG] osdmap e638: 12 total, 12 up, 12 in 2020-09-08T16:05:21.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:20 smithi166 bash[9446]: cluster 2020-09-08T16:05:20.036958+0000 mgr.x (mgr.24118) 1140 : cluster [DBG] pgmap v1694: 17 pgs: 17 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 155 MiB/s wr, 2.48k op/s 2020-09-08T16:05:21.399 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:05:21.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:05:21 smithi104 bash[22305]: debug 2020-09-08T16:05:21.160+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:21.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:20 smithi104 bash[9203]: cluster 2020-09-08T16:05:19.981300+0000 mon.a (mon.0) 2065 : cluster [DBG] osdmap e638: 12 total, 12 up, 12 in 2020-09-08T16:05:21.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:20 smithi104 bash[9203]: cluster 2020-09-08T16:05:20.036958+0000 mgr.x (mgr.24118) 1140 : cluster [DBG] pgmap v1694: 17 pgs: 17 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 155 MiB/s wr, 2.48k op/s 2020-09-08T16:05:21.503 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:05:21.612 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:05:21 smithi055 bash[14301]: debug 2020-09-08T16:05:21.401+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:21.612 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:05:21 smithi055 bash[14301]: debug 2020-09-08T16:05:21.613+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:21.612 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:05:21.716 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:05:21.842 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:05:21.872 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 15 16 37243 37227 155.055 137.5 0.00417374 0.00644663 2020-09-08T16:05:21.945 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:21.984 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:05:21 smithi104 bash[22305]: debug 2020-09-08T16:05:21.840+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:21.984 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:21 smithi104 bash[9203]: audit 2020-09-08T16:05:21.095345+0000 mon.a (mon.0) 2066 : audit [INF] from='client.? v1:172.21.15.104:0/3439195826' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:05:22.002 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T16:05:22.081 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:22.185 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:05:22.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:21 smithi166 bash[9446]: audit 2020-09-08T16:05:21.095345+0000 mon.a (mon.0) 2066 : audit [INF] from='client.? v1:172.21.15.104:0/3439195826' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:05:22.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:21 smithi055 bash[9296]: audit 2020-09-08T16:05:21.095345+0000 mon.a (mon.0) 2066 : audit [INF] from='client.? v1:172.21.15.104:0/3439195826' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:05:22.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:05:22 smithi104 bash[15273]: debug 2020-09-08T16:05:22.080+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:22.456 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:05:22.598 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:05:22.612 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:05:22.715 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:05:22.830 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:05:22.872 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 16 16 39042 39026 152.391 112.438 0.00257821 0.00654462 2020-09-08T16:05:22.876 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:05:22 smithi166 bash[14254]: debug 2020-09-08T16:05:22.610+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:22.935 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:05:22.988 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:05:22 smithi055 bash[19003]: debug 2020-09-08T16:05:22.833+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:22.989 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:22 smithi055 bash[9296]: cluster 2020-09-08T16:05:21.986681+0000 mon.a (mon.0) 2067 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:05:22.989 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:22 smithi055 bash[9296]: cluster 2020-09-08T16:05:21.986721+0000 mon.a (mon.0) 2068 : cluster [INF] Cluster is now healthy 2020-09-08T16:05:22.990 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:22 smithi055 bash[9296]: audit 2020-09-08T16:05:21.998697+0000 mon.a (mon.0) 2069 : audit [INF] from='client.? v1:172.21.15.104:0/3439195826' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:05:22.990 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:22 smithi055 bash[9296]: cluster 2020-09-08T16:05:21.998749+0000 mon.a (mon.0) 2070 : cluster [DBG] osdmap e639: 12 total, 12 up, 12 in 2020-09-08T16:05:22.990 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:22 smithi055 bash[9296]: cluster 2020-09-08T16:05:22.037406+0000 mgr.x (mgr.24118) 1141 : cluster [DBG] pgmap v1696: 17 pgs: 17 active+clean; 1.9 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 152 MiB/s wr, 2.43k op/s 2020-09-08T16:05:23.057 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:05:23.162 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:23.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:22 smithi166 bash[9446]: cluster 2020-09-08T16:05:21.986681+0000 mon.a (mon.0) 2067 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:05:23.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:22 smithi166 bash[9446]: cluster 2020-09-08T16:05:21.986721+0000 mon.a (mon.0) 2068 : cluster [INF] Cluster is now healthy 2020-09-08T16:05:23.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:22 smithi166 bash[9446]: audit 2020-09-08T16:05:21.998697+0000 mon.a (mon.0) 2069 : audit [INF] from='client.? v1:172.21.15.104:0/3439195826' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:05:23.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:22 smithi166 bash[9446]: cluster 2020-09-08T16:05:21.998749+0000 mon.a (mon.0) 2070 : cluster [DBG] osdmap e639: 12 total, 12 up, 12 in 2020-09-08T16:05:23.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:22 smithi166 bash[9446]: cluster 2020-09-08T16:05:22.037406+0000 mgr.x (mgr.24118) 1141 : cluster [DBG] pgmap v1696: 17 pgs: 17 active+clean; 1.9 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 152 MiB/s wr, 2.43k op/s 2020-09-08T16:05:23.408 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:05:23 smithi055 bash[19003]: debug 2020-09-08T16:05:23.061+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:23.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:22 smithi104 bash[9203]: cluster 2020-09-08T16:05:21.986681+0000 mon.a (mon.0) 2067 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:05:23.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:22 smithi104 bash[9203]: cluster 2020-09-08T16:05:21.986721+0000 mon.a (mon.0) 2068 : cluster [INF] Cluster is now healthy 2020-09-08T16:05:23.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:22 smithi104 bash[9203]: audit 2020-09-08T16:05:21.998697+0000 mon.a (mon.0) 2069 : audit [INF] from='client.? v1:172.21.15.104:0/3439195826' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:05:23.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:22 smithi104 bash[9203]: cluster 2020-09-08T16:05:21.998749+0000 mon.a (mon.0) 2070 : cluster [DBG] osdmap e639: 12 total, 12 up, 12 in 2020-09-08T16:05:23.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:22 smithi104 bash[9203]: cluster 2020-09-08T16:05:22.037406+0000 mgr.x (mgr.24118) 1141 : cluster [DBG] pgmap v1696: 17 pgs: 17 active+clean; 1.9 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 152 MiB/s wr, 2.43k op/s 2020-09-08T16:05:23.872 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 17 15 41276 41261 151.644 139.688 0.00653237 0.00659176 2020-09-08T16:05:24.076 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:24.177 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:24.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:05:24 smithi104 bash[15273]: debug 2020-09-08T16:05:24.032+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:24.873 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 18 15 43426 43411 150.684 134.375 0.00461051 0.00663418 2020-09-08T16:05:24.926 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:25.030 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:05:25.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:05:24 smithi104 bash[15273]: debug 2020-09-08T16:05:24.924+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:25.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:25 smithi104 bash[9203]: cluster 2020-09-08T16:05:24.038188+0000 mgr.x (mgr.24118) 1142 : cluster [DBG] pgmap v1697: 17 pgs: 17 active+clean; 2.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 135 MiB/s wr, 2.15k op/s 2020-09-08T16:05:25.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:25 smithi104 bash[9203]: cluster 2020-09-08T16:05:24.982105+0000 mon.a (mon.0) 2071 : cluster [DBG] osdmap e640: 12 total, 12 up, 12 in 2020-09-08T16:05:25.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:25 smithi166 bash[9446]: cluster 2020-09-08T16:05:24.038188+0000 mgr.x (mgr.24118) 1142 : cluster [DBG] pgmap v1697: 17 pgs: 17 active+clean; 2.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 135 MiB/s wr, 2.15k op/s 2020-09-08T16:05:25.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:25 smithi166 bash[9446]: cluster 2020-09-08T16:05:24.982105+0000 mon.a (mon.0) 2071 : cluster [DBG] osdmap e640: 12 total, 12 up, 12 in 2020-09-08T16:05:25.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:25 smithi055 bash[9296]: cluster 2020-09-08T16:05:24.038188+0000 mgr.x (mgr.24118) 1142 : cluster [DBG] pgmap v1697: 17 pgs: 17 active+clean; 2.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 135 MiB/s wr, 2.15k op/s 2020-09-08T16:05:25.413 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:25 smithi055 bash[9296]: cluster 2020-09-08T16:05:24.982105+0000 mon.a (mon.0) 2071 : cluster [DBG] osdmap e640: 12 total, 12 up, 12 in 2020-09-08T16:05:25.872 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 19 16 45109 45093 148.286 105.125 0.00829443 0.00674167 2020-09-08T16:05:26.405 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:05:26.508 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:05:26.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:05:26 smithi166 bash[14254]: debug 2020-09-08T16:05:26.402+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:26.872 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 16:05:26.877270 min lat: 0.00181942 max lat: 0.615684 avg lat: 0.00681258 2020-09-08T16:05:26.872 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T16:05:26.873 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 20 16 46986 46970 146.737 117.312 0.00803924 0.00681258 2020-09-08T16:05:27.070 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:05:27.139 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T16:05:27.173 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:27.297 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:05:27.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:27 smithi166 bash[9446]: cluster 2020-09-08T16:05:26.038785+0000 mgr.x (mgr.24118) 1143 : cluster [DBG] pgmap v1699: 17 pgs: 17 active+clean; 2.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 95 MiB/s wr, 1.52k op/s 2020-09-08T16:05:27.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:27 smithi055 bash[9296]: cluster 2020-09-08T16:05:26.038785+0000 mgr.x (mgr.24118) 1143 : cluster [DBG] pgmap v1699: 17 pgs: 17 active+clean; 2.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 95 MiB/s wr, 1.52k op/s 2020-09-08T16:05:27.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:27 smithi104 bash[9203]: cluster 2020-09-08T16:05:26.038785+0000 mgr.x (mgr.24118) 1143 : cluster [DBG] pgmap v1699: 17 pgs: 17 active+clean; 2.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 95 MiB/s wr, 1.52k op/s 2020-09-08T16:05:27.429 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:05:27 smithi104 bash[17588]: debug 2020-09-08T16:05:27.072+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:27.840 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:27.872 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 21 16 48973 48957 145.663 124.188 0.00721558 0.0068635 2020-09-08T16:05:27.944 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:28.122 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:05:27 smithi166 bash[11928]: debug 2020-09-08T16:05:27.838+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:28.873 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 22 16 50402 50386 143.102 89.3125 0.00563701 0.0069861 2020-09-08T16:05:29.021 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:29.125 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:05:29.315 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:05:29 smithi166 bash[11928]: debug 2020-09-08T16:05:29.022+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:29.315 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:29 smithi166 bash[9446]: cluster 2020-09-08T16:05:27.875519+0000 osd.5 (osd.5) 103 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:29.315 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:29 smithi166 bash[9446]: cluster 2020-09-08T16:05:27.875621+0000 osd.5 (osd.5) 104 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:29.316 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:29 smithi166 bash[9446]: cluster 2020-09-08T16:05:28.039403+0000 mgr.x (mgr.24118) 1144 : cluster [DBG] pgmap v1700: 17 pgs: 17 active+clean; 2.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 125 MiB/s wr, 2.00k op/s 2020-09-08T16:05:29.316 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:29 smithi166 bash[9446]: cluster 2020-09-08T16:05:28.110424+0000 mon.a (mon.0) 2072 : cluster [DBG] osdmap e641: 12 total, 12 up, 12 in 2020-09-08T16:05:29.317 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:05:29.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:29 smithi055 bash[9296]: cluster 2020-09-08T16:05:27.875519+0000 osd.5 (osd.5) 103 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:29.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:29 smithi055 bash[9296]: cluster 2020-09-08T16:05:27.875621+0000 osd.5 (osd.5) 104 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:29.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:29 smithi055 bash[9296]: cluster 2020-09-08T16:05:28.039403+0000 mgr.x (mgr.24118) 1144 : cluster [DBG] pgmap v1700: 17 pgs: 17 active+clean; 2.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 125 MiB/s wr, 2.00k op/s 2020-09-08T16:05:29.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:29 smithi055 bash[9296]: cluster 2020-09-08T16:05:28.110424+0000 mon.a (mon.0) 2072 : cluster [DBG] osdmap e641: 12 total, 12 up, 12 in 2020-09-08T16:05:29.421 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:29.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:29 smithi104 bash[9203]: cluster 2020-09-08T16:05:27.875519+0000 osd.5 (osd.5) 103 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:29.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:29 smithi104 bash[9203]: cluster 2020-09-08T16:05:27.875621+0000 osd.5 (osd.5) 104 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:29.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:29 smithi104 bash[9203]: cluster 2020-09-08T16:05:28.039403+0000 mgr.x (mgr.24118) 1144 : cluster [DBG] pgmap v1700: 17 pgs: 17 active+clean; 2.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 125 MiB/s wr, 2.00k op/s 2020-09-08T16:05:29.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:29 smithi104 bash[9203]: cluster 2020-09-08T16:05:28.110424+0000 mon.a (mon.0) 2072 : cluster [DBG] osdmap e641: 12 total, 12 up, 12 in 2020-09-08T16:05:29.576 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:05:29 smithi166 bash[18960]: debug 2020-09-08T16:05:29.314+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:29.721 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:05:29.756 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:05:29.777 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:29.790 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:05:29.806 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:05:29.806 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T16:05:29.806 INFO:teuthology.orchestra.run.smithi166.stdout: "complaint_time": 30, 2020-09-08T16:05:29.806 INFO:teuthology.orchestra.run.smithi166.stdout: "num_blocked_ops": 0 2020-09-08T16:05:29.806 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:05:29.824 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:05:29.862 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:05:29.886 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:05:29.887 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 23 16 52235 52219 141.861 114.562 0.00524632 0.00704695 2020-09-08T16:05:29.902 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:05:29.926 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:05:29.957 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:05:29.986 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:05:30.018 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:05:30.032 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:05:30.053 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:05:30.088 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:05:30.090 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:05:30.090 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:05:30.090 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:05:30.090 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:05:30.091 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:05:30.091 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:05:30.091 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:05:30.091 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:05:30.091 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:05:30.092 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:05:30.092 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:05:30.092 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:05:30.105 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:05:29 smithi104 bash[15273]: debug 2020-09-08T16:05:29.776+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:30.106 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:30 smithi104 bash[9203]: cluster 2020-09-08T16:05:29.299307+0000 osd.10 (osd.10) 82 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:30.106 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:30 smithi104 bash[9203]: cluster 2020-09-08T16:05:29.299364+0000 osd.10 (osd.10) 83 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:30.106 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:30 smithi104 bash[9203]: audit 2020-09-08T16:05:29.693150+0000 mon.a (mon.0) 2073 : audit [DBG] from='client.? v1:172.21.15.104:0/2002669092' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:05:30.106 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:30 smithi104 bash[9203]: cluster 2020-09-08T16:05:29.983560+0000 mon.a (mon.0) 2074 : cluster [DBG] osdmap e642: 12 total, 12 up, 12 in 2020-09-08T16:05:30.140 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:05:30.156 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:05:30 smithi055 bash[14301]: debug 2020-09-08T16:05:30.032+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:30.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:30 smithi055 bash[9296]: cluster 2020-09-08T16:05:29.299307+0000 osd.10 (osd.10) 82 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:30.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:30 smithi055 bash[9296]: cluster 2020-09-08T16:05:29.299364+0000 osd.10 (osd.10) 83 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:30.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:30 smithi055 bash[9296]: audit 2020-09-08T16:05:29.693150+0000 mon.a (mon.0) 2073 : audit [DBG] from='client.? v1:172.21.15.104:0/2002669092' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:05:30.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:30 smithi055 bash[9296]: cluster 2020-09-08T16:05:29.983560+0000 mon.a (mon.0) 2074 : cluster [DBG] osdmap e642: 12 total, 12 up, 12 in 2020-09-08T16:05:30.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:30 smithi166 bash[9446]: cluster 2020-09-08T16:05:29.299307+0000 osd.10 (osd.10) 82 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:30.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:30 smithi166 bash[9446]: cluster 2020-09-08T16:05:29.299364+0000 osd.10 (osd.10) 83 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:30.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:30 smithi166 bash[9446]: audit 2020-09-08T16:05:29.693150+0000 mon.a (mon.0) 2073 : audit [DBG] from='client.? v1:172.21.15.104:0/2002669092' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:05:30.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:30 smithi166 bash[9446]: cluster 2020-09-08T16:05:29.983560+0000 mon.a (mon.0) 2074 : cluster [DBG] osdmap e642: 12 total, 12 up, 12 in 2020-09-08T16:05:30.872 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 24 16 53857 53841 140.174 101.375 0.0133693 0.00713182 2020-09-08T16:05:31.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:31 smithi166 bash[9446]: cluster 2020-09-08T16:05:30.039854+0000 mgr.x (mgr.24118) 1145 : cluster [DBG] pgmap v1703: 17 pgs: 17 active+clean; 3.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 MiB/s wr, 1.64k op/s 2020-09-08T16:05:31.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:31 smithi055 bash[9296]: cluster 2020-09-08T16:05:30.039854+0000 mgr.x (mgr.24118) 1145 : cluster [DBG] pgmap v1703: 17 pgs: 17 active+clean; 3.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 MiB/s wr, 1.64k op/s 2020-09-08T16:05:31.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:31 smithi104 bash[9203]: cluster 2020-09-08T16:05:30.039854+0000 mgr.x (mgr.24118) 1145 : cluster [DBG] pgmap v1703: 17 pgs: 17 active+clean; 3.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 MiB/s wr, 1.64k op/s 2020-09-08T16:05:31.575 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_historic_ops 2020-09-08T16:05:31.743 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:05:31.835 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:05:31.874 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 25 16 55592 55576 138.904 108.438 0.00833976 0.00719747 2020-09-08T16:05:31.939 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:32.150 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:05:31 smithi104 bash[17588]: debug 2020-09-08T16:05:31.836+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:32.151 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:05:32 smithi104 bash[15273]: debug 2020-09-08T16:05:32.156+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:32.151 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:32.257 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:32.873 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 26 16 57399 57383 137.906 112.938 0.00854343 0.0072493 2020-09-08T16:05:32.968 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:33.073 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:05:33.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:33 smithi166 bash[9446]: cluster 2020-09-08T16:05:32.040548+0000 mgr.x (mgr.24118) 1146 : cluster [DBG] pgmap v1704: 17 pgs: 17 active+clean; 3.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 101 MiB/s wr, 1.62k op/s 2020-09-08T16:05:33.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:33 smithi055 bash[9296]: cluster 2020-09-08T16:05:32.040548+0000 mgr.x (mgr.24118) 1146 : cluster [DBG] pgmap v1704: 17 pgs: 17 active+clean; 3.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 101 MiB/s wr, 1.62k op/s 2020-09-08T16:05:33.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:05:32 smithi104 bash[15273]: debug 2020-09-08T16:05:32.972+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:33.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:33 smithi104 bash[9203]: cluster 2020-09-08T16:05:32.040548+0000 mgr.x (mgr.24118) 1146 : cluster [DBG] pgmap v1704: 17 pgs: 17 active+clean; 3.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 101 MiB/s wr, 1.62k op/s 2020-09-08T16:05:33.873 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 27 16 59306 59290 137.212 119.188 0.0189962 0.00728583 2020-09-08T16:05:34.197 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:05:34.333 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:05:34.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:34 smithi166 bash[9446]: cluster 2020-09-08T16:05:32.619465+0000 osd.3 (osd.3) 111 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:34.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:34 smithi166 bash[9446]: cluster 2020-09-08T16:05:32.619586+0000 osd.3 (osd.3) 112 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:34.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:34 smithi166 bash[9446]: cluster 2020-09-08T16:05:33.124735+0000 mon.a (mon.0) 2075 : cluster [DBG] osdmap e643: 12 total, 12 up, 12 in 2020-09-08T16:05:34.381 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:05:34.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:05:34 smithi055 bash[14301]: debug 2020-09-08T16:05:34.336+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:34.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:34 smithi055 bash[9296]: cluster 2020-09-08T16:05:32.619465+0000 osd.3 (osd.3) 111 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:34.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:34 smithi055 bash[9296]: cluster 2020-09-08T16:05:32.619586+0000 osd.3 (osd.3) 112 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:34.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:34 smithi055 bash[9296]: cluster 2020-09-08T16:05:33.124735+0000 mon.a (mon.0) 2075 : cluster [DBG] osdmap e643: 12 total, 12 up, 12 in 2020-09-08T16:05:34.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:34 smithi104 bash[9203]: cluster 2020-09-08T16:05:32.619465+0000 osd.3 (osd.3) 111 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:34.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:34 smithi104 bash[9203]: cluster 2020-09-08T16:05:32.619586+0000 osd.3 (osd.3) 112 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:34.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:34 smithi104 bash[9203]: cluster 2020-09-08T16:05:33.124735+0000 mon.a (mon.0) 2075 : cluster [DBG] osdmap e643: 12 total, 12 up, 12 in 2020-09-08T16:05:34.437 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:34.571 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:34.672 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:05:34.873 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 28 16 60950 60934 135.981 102.75 0.00631488 0.00735186 2020-09-08T16:05:34.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:05:34 smithi055 bash[11956]: debug 2020-09-08T16:05:34.560+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:34.984 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T16:05:35.129 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:35 smithi104 bash[9203]: cluster 2020-09-08T16:05:34.041161+0000 mgr.x (mgr.24118) 1147 : cluster [DBG] pgmap v1706: 17 pgs: 17 active+clean; 3.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 120 MiB/s wr, 1.93k op/s 2020-09-08T16:05:35.130 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:35 smithi104 bash[9203]: audit 2020-09-08T16:05:34.738423+0000 mon.a (mon.0) 2076 : audit [INF] from='client.? v1:172.21.15.104:0/657838993' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:05:35.130 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:35 smithi104 bash[9203]: cluster 2020-09-08T16:05:34.980980+0000 mon.a (mon.0) 2077 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:05:35.130 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:35 smithi104 bash[9203]: audit 2020-09-08T16:05:34.985122+0000 mon.a (mon.0) 2078 : audit [INF] from='client.? v1:172.21.15.104:0/657838993' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:05:35.130 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:35 smithi104 bash[9203]: cluster 2020-09-08T16:05:34.985163+0000 mon.a (mon.0) 2079 : cluster [DBG] osdmap e644: 12 total, 12 up, 12 in 2020-09-08T16:05:35.334 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:05:35.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:35 smithi055 bash[9296]: cluster 2020-09-08T16:05:34.041161+0000 mgr.x (mgr.24118) 1147 : cluster [DBG] pgmap v1706: 17 pgs: 17 active+clean; 3.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 120 MiB/s wr, 1.93k op/s 2020-09-08T16:05:35.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:35 smithi055 bash[9296]: audit 2020-09-08T16:05:34.738423+0000 mon.a (mon.0) 2076 : audit [INF] from='client.? v1:172.21.15.104:0/657838993' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:05:35.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:35 smithi055 bash[9296]: cluster 2020-09-08T16:05:34.980980+0000 mon.a (mon.0) 2077 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:05:35.413 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:35 smithi055 bash[9296]: audit 2020-09-08T16:05:34.985122+0000 mon.a (mon.0) 2078 : audit [INF] from='client.? v1:172.21.15.104:0/657838993' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:05:35.413 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:35 smithi055 bash[9296]: cluster 2020-09-08T16:05:34.985163+0000 mon.a (mon.0) 2079 : cluster [DBG] osdmap e644: 12 total, 12 up, 12 in 2020-09-08T16:05:35.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:05:35 smithi104 bash[22305]: debug 2020-09-08T16:05:35.336+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:35.439 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:05:35.446 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:35 smithi166 bash[9446]: cluster 2020-09-08T16:05:34.041161+0000 mgr.x (mgr.24118) 1147 : cluster [DBG] pgmap v1706: 17 pgs: 17 active+clean; 3.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 120 MiB/s wr, 1.93k op/s 2020-09-08T16:05:35.446 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:35 smithi166 bash[9446]: audit 2020-09-08T16:05:34.738423+0000 mon.a (mon.0) 2076 : audit [INF] from='client.? v1:172.21.15.104:0/657838993' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:05:35.447 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:35 smithi166 bash[9446]: cluster 2020-09-08T16:05:34.980980+0000 mon.a (mon.0) 2077 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:05:35.447 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:35 smithi166 bash[9446]: audit 2020-09-08T16:05:34.985122+0000 mon.a (mon.0) 2078 : audit [INF] from='client.? v1:172.21.15.104:0/657838993' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:05:35.447 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:35 smithi166 bash[9446]: cluster 2020-09-08T16:05:34.985163+0000 mon.a (mon.0) 2079 : cluster [DBG] osdmap e644: 12 total, 12 up, 12 in 2020-09-08T16:05:35.842 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:05:35.842 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:05:35 smithi166 bash[18960]: debug 2020-09-08T16:05:35.842+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:35.873 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 29 16 62654 62638 134.964 106.5 0.00983401 0.00740715 2020-09-08T16:05:35.946 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:05:36.461 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:05:36.570 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:36.820 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:05:36 smithi104 bash[19922]: debug 2020-09-08T16:05:36.464+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:36.873 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 30 16 64368 64352 134.036 107.125 0.00513992 0.00745855 2020-09-08T16:05:37.136 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:37 smithi104 bash[9203]: cluster 2020-09-08T16:05:36.041696+0000 mgr.x (mgr.24118) 1148 : cluster [DBG] pgmap v1708: 17 pgs: 17 active+clean; 3.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 88 MiB/s wr, 1.40k op/s 2020-09-08T16:05:37.196 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:37.300 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:37.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:37 smithi055 bash[9296]: cluster 2020-09-08T16:05:36.041696+0000 mgr.x (mgr.24118) 1148 : cluster [DBG] pgmap v1708: 17 pgs: 17 active+clean; 3.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 88 MiB/s wr, 1.40k op/s 2020-09-08T16:05:37.425 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:37.534 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:05:37.622 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:37 smithi166 bash[9446]: cluster 2020-09-08T16:05:36.041696+0000 mgr.x (mgr.24118) 1148 : cluster [DBG] pgmap v1708: 17 pgs: 17 active+clean; 3.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 88 MiB/s wr, 1.40k op/s 2020-09-08T16:05:37.623 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:05:37 smithi166 bash[11928]: debug 2020-09-08T16:05:37.194+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:37.873 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 31 16 66098 66082 133.2 108.125 0.0062868 0.007505 2020-09-08T16:05:37.906 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:05:37 smithi055 bash[11956]: debug 2020-09-08T16:05:37.428+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:38.367 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:05:38.471 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:38.635 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:38.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:05:38 smithi104 bash[22305]: debug 2020-09-08T16:05:38.368+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:38.689 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T16:05:38.692 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T16:05:38.693 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:05:38.739 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:05:38.854 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:05:38.873 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 32 16 67736 67720 132.237 102.375 0.00514802 0.00755998 2020-09-08T16:05:39.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:05:38 smithi166 bash[11928]: debug 2020-09-08T16:05:38.634+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:39.147 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:39 smithi104 bash[9203]: cluster 2020-09-08T16:05:38.042295+0000 mgr.x (mgr.24118) 1149 : cluster [DBG] pgmap v1709: 17 pgs: 17 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 114 MiB/s wr, 1.82k op/s 2020-09-08T16:05:39.257 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:05:39.367 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:39.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:39 smithi055 bash[9296]: cluster 2020-09-08T16:05:38.042295+0000 mgr.x (mgr.24118) 1149 : cluster [DBG] pgmap v1709: 17 pgs: 17 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 114 MiB/s wr, 1.82k op/s 2020-09-08T16:05:39.433 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:05:39 smithi104 bash[17588]: debug 2020-09-08T16:05:39.244+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:39.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:39 smithi166 bash[9446]: cluster 2020-09-08T16:05:38.042295+0000 mgr.x (mgr.24118) 1149 : cluster [DBG] pgmap v1709: 17 pgs: 17 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 114 MiB/s wr, 1.82k op/s 2020-09-08T16:05:39.875 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 33 16 69211 69195 131.023 92.1875 0.0100435 0.00762984 2020-09-08T16:05:40.365 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:40.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:40 smithi055 bash[9296]: cluster 2020-09-08T16:05:38.429943+0000 osd.8 (osd.8) 85 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:40.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:40 smithi055 bash[9296]: cluster 2020-09-08T16:05:38.430019+0000 osd.8 (osd.8) 86 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:40.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:40 smithi055 bash[9296]: cluster 2020-09-08T16:05:39.154883+0000 mon.a (mon.0) 2080 : cluster [DBG] osdmap e645: 12 total, 12 up, 12 in 2020-09-08T16:05:40.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:40 smithi055 bash[9296]: cluster 2020-09-08T16:05:39.986851+0000 mon.a (mon.0) 2081 : cluster [DBG] osdmap e646: 12 total, 12 up, 12 in 2020-09-08T16:05:40.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:40 smithi104 bash[9203]: cluster 2020-09-08T16:05:38.429943+0000 osd.8 (osd.8) 85 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:40.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:40 smithi104 bash[9203]: cluster 2020-09-08T16:05:38.430019+0000 osd.8 (osd.8) 86 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:40.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:40 smithi104 bash[9203]: cluster 2020-09-08T16:05:39.154883+0000 mon.a (mon.0) 2080 : cluster [DBG] osdmap e645: 12 total, 12 up, 12 in 2020-09-08T16:05:40.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:40 smithi104 bash[9203]: cluster 2020-09-08T16:05:39.986851+0000 mon.a (mon.0) 2081 : cluster [DBG] osdmap e646: 12 total, 12 up, 12 in 2020-09-08T16:05:40.468 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:40.476 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:40 smithi166 bash[9446]: cluster 2020-09-08T16:05:38.429943+0000 osd.8 (osd.8) 85 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:40.476 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:40 smithi166 bash[9446]: cluster 2020-09-08T16:05:38.430019+0000 osd.8 (osd.8) 86 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:40.477 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:40 smithi166 bash[9446]: cluster 2020-09-08T16:05:39.154883+0000 mon.a (mon.0) 2080 : cluster [DBG] osdmap e645: 12 total, 12 up, 12 in 2020-09-08T16:05:40.477 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:40 smithi166 bash[9446]: cluster 2020-09-08T16:05:39.986851+0000 mon.a (mon.0) 2081 : cluster [DBG] osdmap e646: 12 total, 12 up, 12 in 2020-09-08T16:05:40.477 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:05:40 smithi166 bash[16593]: debug 2020-09-08T16:05:40.361+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:40.504 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2020-09-08T16:05:40.590 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:40.698 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:40.717 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:05:40.825 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:40.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:05:40 smithi166 bash[16593]: debug 2020-09-08T16:05:40.589+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:40.873 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 34 16 70727 70711 129.956 94.75 0.00707097 0.00769233 2020-09-08T16:05:40.932 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:05:41.046 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:05:41.047 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:05:40 smithi055 bash[11956]: debug 2020-09-08T16:05:40.828+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:41.148 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:05:41.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:41 smithi055 bash[9296]: cluster 2020-09-08T16:05:40.042746+0000 mgr.x (mgr.24118) 1150 : cluster [DBG] pgmap v1712: 17 pgs: 17 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 91 MiB/s wr, 1.46k op/s 2020-09-08T16:05:41.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:05:41 smithi055 bash[19003]: debug 2020-09-08T16:05:41.044+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:41.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:41 smithi104 bash[9203]: cluster 2020-09-08T16:05:40.042746+0000 mgr.x (mgr.24118) 1150 : cluster [DBG] pgmap v1712: 17 pgs: 17 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 91 MiB/s wr, 1.46k op/s 2020-09-08T16:05:41.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:41 smithi166 bash[9446]: cluster 2020-09-08T16:05:40.042746+0000 mgr.x (mgr.24118) 1150 : cluster [DBG] pgmap v1712: 17 pgs: 17 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 91 MiB/s wr, 1.46k op/s 2020-09-08T16:05:41.623 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:05:41.657 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:05:41.691 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:05:41.721 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:05:41.754 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:05:41.790 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:05:41.821 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:05:41.849 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:05:41.873 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:05:41.874 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 35 16 72186 72170 128.848 91.1875 0.00419114 0.00775273 2020-09-08T16:05:41.900 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:05:41.939 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:05:41.981 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:05:41.983 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:05:41.983 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:05:41.983 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:05:41.984 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:05:41.984 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:05:41.984 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:05:41.984 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:05:41.984 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:05:41.985 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:05:41.985 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:05:41.985 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:05:41.985 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:05:42.098 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:05:42.202 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:42.390 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:42 smithi166 bash[9446]: audit 2020-09-08T16:05:41.587820+0000 mon.a (mon.0) 2082 : audit [DBG] from='client.? v1:172.21.15.104:0/3809483046' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:05:42.391 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:05:42 smithi166 bash[14254]: debug 2020-09-08T16:05:42.093+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:42.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:42 smithi104 bash[9203]: audit 2020-09-08T16:05:41.587820+0000 mon.a (mon.0) 2082 : audit [DBG] from='client.? v1:172.21.15.104:0/3809483046' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:05:42.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:42 smithi055 bash[9296]: audit 2020-09-08T16:05:41.587820+0000 mon.a (mon.0) 2082 : audit [DBG] from='client.? v1:172.21.15.104:0/3809483046' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:05:42.783 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:42.874 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 36 16 73747 73731 127.979 97.5625 0.00951205 0.00781167 2020-09-08T16:05:42.886 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:05:43.160 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:05:42 smithi104 bash[15273]: debug 2020-09-08T16:05:42.784+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:43.160 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:43 smithi166 bash[9446]: cluster 2020-09-08T16:05:42.043286+0000 mgr.x (mgr.24118) 1151 : cluster [DBG] pgmap v1713: 17 pgs: 17 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 MiB/s wr, 1.44k op/s 2020-09-08T16:05:43.332 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:05:43.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:43 smithi104 bash[9203]: cluster 2020-09-08T16:05:42.043286+0000 mgr.x (mgr.24118) 1151 : cluster [DBG] pgmap v1713: 17 pgs: 17 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 MiB/s wr, 1.44k op/s 2020-09-08T16:05:43.436 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:05:43.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:43 smithi055 bash[9296]: cluster 2020-09-08T16:05:42.043286+0000 mgr.x (mgr.24118) 1151 : cluster [DBG] pgmap v1713: 17 pgs: 17 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 MiB/s wr, 1.44k op/s 2020-09-08T16:05:43.498 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:05:43 smithi166 bash[18960]: debug 2020-09-08T16:05:43.329+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:43.565 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:05:43.669 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:43.818 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:43.874 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 37 16 75281 75265 127.111 95.875 0.00749656 0.00786458 2020-09-08T16:05:43.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:05:43 smithi055 bash[14301]: debug 2020-09-08T16:05:43.568+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:43.922 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:44.076 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:44.079 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:05:44 smithi166 bash[11928]: debug 2020-09-08T16:05:44.065+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:44.079 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:05:43 smithi166 bash[16593]: debug 2020-09-08T16:05:43.813+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:44.180 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:44.876 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 38 15 76843 76828 126.337 97.6875 0.0159355 0.00791332 2020-09-08T16:05:45.356 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:45.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:45 smithi104 bash[9203]: cluster 2020-09-08T16:05:44.043805+0000 mgr.x (mgr.24118) 1152 : cluster [DBG] pgmap v1714: 17 pgs: 17 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 128 MiB/s wr, 2.05k op/s 2020-09-08T16:05:45.458 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:05:45.578 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:05:45.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:45 smithi166 bash[9446]: cluster 2020-09-08T16:05:44.043805+0000 mgr.x (mgr.24118) 1152 : cluster [DBG] pgmap v1714: 17 pgs: 17 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 128 MiB/s wr, 2.05k op/s 2020-09-08T16:05:45.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:05:45 smithi166 bash[11928]: debug 2020-09-08T16:05:45.353+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:45.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:45 smithi055 bash[9296]: cluster 2020-09-08T16:05:44.043805+0000 mgr.x (mgr.24118) 1152 : cluster [DBG] pgmap v1714: 17 pgs: 17 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 128 MiB/s wr, 2.05k op/s 2020-09-08T16:05:45.683 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:45.825 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:45.874 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 39 16 78369 78353 125.541 95.3125 0.00872406 0.0079636 2020-09-08T16:05:45.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:05:45 smithi104 bash[19922]: debug 2020-09-08T16:05:45.580+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:45.930 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:46.130 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:05:46.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:05:45 smithi055 bash[11956]: debug 2020-09-08T16:05:45.828+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:46.186 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T16:05:46.298 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:05:46.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:46 smithi166 bash[9446]: audit 2020-09-08T16:05:45.697547+0000 mon.a (mon.0) 2083 : audit [INF] from='client.? v1:172.21.15.104:0/3868351938' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:05:46.491 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:46 smithi104 bash[9203]: audit 2020-09-08T16:05:45.697547+0000 mon.a (mon.0) 2083 : audit [INF] from='client.? v1:172.21.15.104:0/3868351938' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:05:46.625 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:46.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:46 smithi055 bash[9296]: audit 2020-09-08T16:05:45.697547+0000 mon.a (mon.0) 2083 : audit [INF] from='client.? v1:172.21.15.104:0/3868351938' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:05:46.730 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:46.879 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 16:05:46.878929 min lat: 0.00181942 max lat: 0.615684 avg lat: 0.00802151 2020-09-08T16:05:46.880 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T16:05:46.880 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 40 16 79792 79776 124.626 88.9375 0.0102794 0.00802151 2020-09-08T16:05:47.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:05:46 smithi166 bash[16593]: debug 2020-09-08T16:05:46.625+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:47.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:47 smithi104 bash[9203]: cluster 2020-09-08T16:05:46.044370+0000 mgr.x (mgr.24118) 1153 : cluster [DBG] pgmap v1715: 17 pgs: 17 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s wr, 1.29k op/s 2020-09-08T16:05:47.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:47 smithi104 bash[9203]: cluster 2020-09-08T16:05:46.171291+0000 mon.a (mon.0) 2084 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:05:47.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:47 smithi104 bash[9203]: audit 2020-09-08T16:05:46.183057+0000 mon.a (mon.0) 2085 : audit [INF] from='client.? v1:172.21.15.104:0/3868351938' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:05:47.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:47 smithi104 bash[9203]: cluster 2020-09-08T16:05:46.183108+0000 mon.a (mon.0) 2086 : cluster [DBG] osdmap e647: 12 total, 12 up, 12 in 2020-09-08T16:05:47.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:47 smithi104 bash[9203]: cluster 2020-09-08T16:05:46.677274+0000 osd.1 (osd.1) 126 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:47.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:47 smithi104 bash[9203]: cluster 2020-09-08T16:05:46.677367+0000 osd.1 (osd.1) 127 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:47.587 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:47 smithi166 bash[9446]: cluster 2020-09-08T16:05:46.044370+0000 mgr.x (mgr.24118) 1153 : cluster [DBG] pgmap v1715: 17 pgs: 17 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s wr, 1.29k op/s 2020-09-08T16:05:47.588 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:47 smithi166 bash[9446]: cluster 2020-09-08T16:05:46.171291+0000 mon.a (mon.0) 2084 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:05:47.588 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:47 smithi166 bash[9446]: audit 2020-09-08T16:05:46.183057+0000 mon.a (mon.0) 2085 : audit [INF] from='client.? v1:172.21.15.104:0/3868351938' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:05:47.588 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:47 smithi166 bash[9446]: cluster 2020-09-08T16:05:46.183108+0000 mon.a (mon.0) 2086 : cluster [DBG] osdmap e647: 12 total, 12 up, 12 in 2020-09-08T16:05:47.589 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:47 smithi166 bash[9446]: cluster 2020-09-08T16:05:46.677274+0000 osd.1 (osd.1) 126 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:47.589 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:47 smithi166 bash[9446]: cluster 2020-09-08T16:05:46.677367+0000 osd.1 (osd.1) 127 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:47.589 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:05:47 smithi166 bash[11928]: debug 2020-09-08T16:05:47.585+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:47.590 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:05:47.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:47 smithi055 bash[9296]: cluster 2020-09-08T16:05:46.044370+0000 mgr.x (mgr.24118) 1153 : cluster [DBG] pgmap v1715: 17 pgs: 17 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s wr, 1.29k op/s 2020-09-08T16:05:47.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:47 smithi055 bash[9296]: cluster 2020-09-08T16:05:46.171291+0000 mon.a (mon.0) 2084 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:05:47.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:47 smithi055 bash[9296]: audit 2020-09-08T16:05:46.183057+0000 mon.a (mon.0) 2085 : audit [INF] from='client.? v1:172.21.15.104:0/3868351938' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:05:47.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:47 smithi055 bash[9296]: cluster 2020-09-08T16:05:46.183108+0000 mon.a (mon.0) 2086 : cluster [DBG] osdmap e647: 12 total, 12 up, 12 in 2020-09-08T16:05:47.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:47 smithi055 bash[9296]: cluster 2020-09-08T16:05:46.677274+0000 osd.1 (osd.1) 126 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:47.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:47 smithi055 bash[9296]: cluster 2020-09-08T16:05:46.677367+0000 osd.1 (osd.1) 127 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:47.696 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:47.874 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 41 15 81339 81324 123.946 96.75 0.0135733 0.00806577 2020-09-08T16:05:48.253 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:05:48.254 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T16:05:48.254 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:48.254 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:81865 8.2 8:4dfa5d7b:::benchmark_data_smithi166_52302_object81864:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e648)", 2020-09-08T16:05:48.254 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:05:48.254900+0000", 2020-09-08T16:05:48.255 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 0.0022515539999999998, 2020-09-08T16:05:48.255 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0022947990000000001, 2020-09-08T16:05:48.255 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:05:48.255 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:05:48.256 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:05:48.257 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:05:48.257 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:05:48.257 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 81865 2020-09-08T16:05:48.257 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:48.258 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:05:48.258 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:48.258 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:05:48.258 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:48.254900+0000", 2020-09-08T16:05:48.258 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:05:48.259 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:48.259 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:48.259 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:05:48.259 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:48.254900+0000", 2020-09-08T16:05:48.259 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999561 2020-09-08T16:05:48.260 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:48.260 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:48.260 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:05:48.260 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:48.254857+0000", 2020-09-08T16:05:48.260 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013393699999999999 2020-09-08T16:05:48.261 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:48.261 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:48.261 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:05:48.261 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:48.254991+0000", 2020-09-08T16:05:48.261 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.961e-05 2020-09-08T16:05:48.262 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:48.262 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:48.262 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:05:48.262 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:48.255010+0000", 2020-09-08T16:05:48.262 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.1949999999999992e-06 2020-09-08T16:05:48.263 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:48.263 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:48.263 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:05:48.263 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:48.255019+0000", 2020-09-08T16:05:48.263 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1245999999999997e-05 2020-09-08T16:05:48.264 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:48.264 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:48.264 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:05:48.264 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:48.255071+0000", 2020-09-08T16:05:48.264 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012174 2020-09-08T16:05:48.265 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:48.265 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:48.265 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:05:48.265 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:48.255192+0000", 2020-09-08T16:05:48.265 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010480300000000001 2020-09-08T16:05:48.265 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:05:48.266 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:05:48.266 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 0", 2020-09-08T16:05:48.266 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:48.255297+0000", 2020-09-08T16:05:48.266 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000396883 2020-09-08T16:05:48.266 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:05:48.267 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:05:48.267 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:05:48.267 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:05:48.267 INFO:teuthology.orchestra.run.smithi166.stdout: ], 2020-09-08T16:05:48.267 INFO:teuthology.orchestra.run.smithi166.stdout: "num_ops": 1 2020-09-08T16:05:48.268 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:05:48.274 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:48.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:48 smithi166 bash[9446]: cluster 2020-09-08T16:05:47.185854+0000 mon.a (mon.0) 2087 : cluster [DBG] osdmap e648: 12 total, 12 up, 12 in 2020-09-08T16:05:48.388 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:05:48.517 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:48 smithi055 bash[9296]: cluster 2020-09-08T16:05:47.185854+0000 mon.a (mon.0) 2087 : cluster [DBG] osdmap e648: 12 total, 12 up, 12 in 2020-09-08T16:05:48.518 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:05:48 smithi055 bash[14301]: debug 2020-09-08T16:05:48.516+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:48.518 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:05:48.620 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:05:48.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:48 smithi104 bash[9203]: cluster 2020-09-08T16:05:47.185854+0000 mon.a (mon.0) 2087 : cluster [DBG] osdmap e648: 12 total, 12 up, 12 in 2020-09-08T16:05:48.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:05:48 smithi104 bash[15273]: debug 2020-09-08T16:05:48.276+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:48.732 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:05:48.835 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:05:48.846 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:05:48 smithi055 bash[19003]: debug 2020-09-08T16:05:48.732+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:48.874 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 42 15 82782 82767 123.142 90.1875 0.0221991 0.00811875 2020-09-08T16:05:48.940 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:05:49.044 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:05:49.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:05:48 smithi055 bash[16646]: debug 2020-09-08T16:05:48.940+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:49.213 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:05:49.316 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:05:49.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:05:49 smithi104 bash[17588]: debug 2020-09-08T16:05:49.216+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:49.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:49 smithi104 bash[9203]: cluster 2020-09-08T16:05:48.044858+0000 mgr.x (mgr.24118) 1154 : cluster [DBG] pgmap v1718: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 MiB/s wr, 1.60k op/s 2020-09-08T16:05:49.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:49 smithi166 bash[9446]: cluster 2020-09-08T16:05:48.044858+0000 mgr.x (mgr.24118) 1154 : cluster [DBG] pgmap v1718: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 MiB/s wr, 1.60k op/s 2020-09-08T16:05:49.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:49 smithi055 bash[9296]: cluster 2020-09-08T16:05:48.044858+0000 mgr.x (mgr.24118) 1154 : cluster [DBG] pgmap v1718: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 MiB/s wr, 1.60k op/s 2020-09-08T16:05:49.879 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 43 16 84269 84253 122.438 92.875 0.0131699 0.00816488 2020-09-08T16:05:49.932 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:05:50.036 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:50.038 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2020-09-08T16:05:50.213 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:05:50.287 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:50.324 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T16:05:50.332 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:05:50 smithi104 bash[15273]: debug 2020-09-08T16:05:50.292+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:50.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:05:49 smithi166 bash[14254]: debug 2020-09-08T16:05:49.933+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:50.392 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:05:50.491 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:05:50.874 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 44 16 85678 85662 121.657 88.0625 0.00551424 0.00821777 2020-09-08T16:05:51.117 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:05:51.226 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:05:51.332 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:05:51.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:05:51 smithi104 bash[17588]: debug 2020-09-08T16:05:51.120+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:51.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:51 smithi104 bash[9203]: cluster 2020-09-08T16:05:49.830113+0000 osd.6 (osd.6) 109 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:51.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:51 smithi104 bash[9203]: cluster 2020-09-08T16:05:49.830224+0000 osd.6 (osd.6) 110 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:51.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:51 smithi104 bash[9203]: cluster 2020-09-08T16:05:50.045466+0000 mgr.x (mgr.24118) 1155 : cluster [DBG] pgmap v1719: 17 pgs: 17 active+clean; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 108 MiB/s wr, 1.72k op/s 2020-09-08T16:05:51.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:51 smithi104 bash[9203]: cluster 2020-09-08T16:05:50.199742+0000 mon.a (mon.0) 2088 : cluster [DBG] osdmap e649: 12 total, 12 up, 12 in 2020-09-08T16:05:51.437 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:51.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:51 smithi166 bash[9446]: cluster 2020-09-08T16:05:49.830113+0000 osd.6 (osd.6) 109 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:51.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:51 smithi166 bash[9446]: cluster 2020-09-08T16:05:49.830224+0000 osd.6 (osd.6) 110 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:51.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:51 smithi166 bash[9446]: cluster 2020-09-08T16:05:50.045466+0000 mgr.x (mgr.24118) 1155 : cluster [DBG] pgmap v1719: 17 pgs: 17 active+clean; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 108 MiB/s wr, 1.72k op/s 2020-09-08T16:05:51.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:51 smithi166 bash[9446]: cluster 2020-09-08T16:05:50.199742+0000 mon.a (mon.0) 2088 : cluster [DBG] osdmap e649: 12 total, 12 up, 12 in 2020-09-08T16:05:51.656 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:05:51 smithi055 bash[19003]: debug 2020-09-08T16:05:51.332+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:51.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:51 smithi055 bash[9296]: cluster 2020-09-08T16:05:49.830113+0000 osd.6 (osd.6) 109 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:51.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:51 smithi055 bash[9296]: cluster 2020-09-08T16:05:49.830224+0000 osd.6 (osd.6) 110 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:51.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:51 smithi055 bash[9296]: cluster 2020-09-08T16:05:50.045466+0000 mgr.x (mgr.24118) 1155 : cluster [DBG] pgmap v1719: 17 pgs: 17 active+clean; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 108 MiB/s wr, 1.72k op/s 2020-09-08T16:05:51.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:51 smithi055 bash[9296]: cluster 2020-09-08T16:05:50.199742+0000 mon.a (mon.0) 2088 : cluster [DBG] osdmap e649: 12 total, 12 up, 12 in 2020-09-08T16:05:51.874 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 45 15 87103 87088 120.934 89.125 0.0162193 0.00826662 2020-09-08T16:05:51.915 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:52.021 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:05:52.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:05:51 smithi104 bash[15273]: debug 2020-09-08T16:05:51.920+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:52.230 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:05:52.333 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:05:52.451 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:05:52.491 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:05:52 smithi104 bash[22305]: debug 2020-09-08T16:05:52.228+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:52.556 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:52.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:05:52 smithi055 bash[16646]: debug 2020-09-08T16:05:52.452+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:52.673 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:52.779 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:05:52.875 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 46 16 88544 88528 120.261 90 0.0057774 0.00831276 2020-09-08T16:05:53.000 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:05:53.034 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:05:53.074 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:05:53.090 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:05:53.111 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:05:53.144 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:05:53.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:05:52 smithi055 bash[11956]: debug 2020-09-08T16:05:52.672+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:53.184 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:05:53.194 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:53.220 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:05:53.256 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:05:53.293 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:05:53.334 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:05:53.370 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:05:53.410 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:05:53.412 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:05:53.413 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:05:53.413 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:05:53.413 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:05:53.413 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:05:53.414 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:05:53.414 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:05:53.414 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:05:53.414 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:05:53.414 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:05:53.415 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:05:53.415 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:05:53.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:05:53 smithi104 bash[22305]: debug 2020-09-08T16:05:53.092+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:53.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:53 smithi104 bash[9203]: cluster 2020-09-08T16:05:52.046029+0000 mgr.x (mgr.24118) 1156 : cluster [DBG] pgmap v1721: 17 pgs: 17 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 93 MiB/s wr, 1.49k op/s 2020-09-08T16:05:53.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:53 smithi104 bash[9203]: audit 2020-09-08T16:05:52.970678+0000 mon.b (mon.1) 115 : audit [DBG] from='client.? v1:172.21.15.104:0/1317317074' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:05:53.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:53 smithi166 bash[9446]: cluster 2020-09-08T16:05:52.046029+0000 mgr.x (mgr.24118) 1156 : cluster [DBG] pgmap v1721: 17 pgs: 17 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 93 MiB/s wr, 1.49k op/s 2020-09-08T16:05:53.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:53 smithi166 bash[9446]: audit 2020-09-08T16:05:52.970678+0000 mon.b (mon.1) 115 : audit [DBG] from='client.? v1:172.21.15.104:0/1317317074' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:05:53.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:53 smithi055 bash[9296]: cluster 2020-09-08T16:05:52.046029+0000 mgr.x (mgr.24118) 1156 : cluster [DBG] pgmap v1721: 17 pgs: 17 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 93 MiB/s wr, 1.49k op/s 2020-09-08T16:05:53.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:53 smithi055 bash[9296]: audit 2020-09-08T16:05:52.970678+0000 mon.b (mon.1) 115 : audit [DBG] from='client.? v1:172.21.15.104:0/1317317074' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:05:53.874 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 47 16 89976 89960 119.607 89.5 0.011364 0.00835799 2020-09-08T16:05:54.145 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:54.250 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:54.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:54 smithi104 bash[9203]: cluster 2020-09-08T16:05:53.030808+0000 osd.11 (osd.11) 106 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:54.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:54 smithi104 bash[9203]: cluster 2020-09-08T16:05:53.030883+0000 osd.11 (osd.11) 107 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:54.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:54 smithi104 bash[9203]: cluster 2020-09-08T16:05:53.215657+0000 mon.a (mon.0) 2089 : cluster [DBG] osdmap e650: 12 total, 12 up, 12 in 2020-09-08T16:05:54.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:54 smithi104 bash[9203]: audit 2020-09-08T16:05:54.053403+0000 mon.c (mon.2) 84 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:05:54.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:54 smithi104 bash[9203]: audit 2020-09-08T16:05:54.055666+0000 mon.a (mon.0) 2090 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:05:54.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:54 smithi104 bash[9203]: audit 2020-09-08T16:05:54.060116+0000 mon.a (mon.0) 2091 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:05:54.428 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:05:54 smithi104 bash[15273]: debug 2020-09-08T16:05:54.148+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:54.490 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:54.594 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:05:54.655 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:54 smithi166 bash[9446]: cluster 2020-09-08T16:05:53.030808+0000 osd.11 (osd.11) 106 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:54.655 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:54 smithi166 bash[9446]: cluster 2020-09-08T16:05:53.030883+0000 osd.11 (osd.11) 107 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:54.656 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:54 smithi166 bash[9446]: cluster 2020-09-08T16:05:53.215657+0000 mon.a (mon.0) 2089 : cluster [DBG] osdmap e650: 12 total, 12 up, 12 in 2020-09-08T16:05:54.656 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:54 smithi166 bash[9446]: audit 2020-09-08T16:05:54.053403+0000 mon.c (mon.2) 84 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:05:54.656 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:54 smithi166 bash[9446]: audit 2020-09-08T16:05:54.055666+0000 mon.a (mon.0) 2090 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:05:54.656 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:54 smithi166 bash[9446]: audit 2020-09-08T16:05:54.060116+0000 mon.a (mon.0) 2091 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:05:54.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:54 smithi055 bash[9296]: cluster 2020-09-08T16:05:53.030808+0000 osd.11 (osd.11) 106 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:05:54.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:54 smithi055 bash[9296]: cluster 2020-09-08T16:05:53.030883+0000 osd.11 (osd.11) 107 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:05:54.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:54 smithi055 bash[9296]: cluster 2020-09-08T16:05:53.215657+0000 mon.a (mon.0) 2089 : cluster [DBG] osdmap e650: 12 total, 12 up, 12 in 2020-09-08T16:05:54.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:54 smithi055 bash[9296]: audit 2020-09-08T16:05:54.053403+0000 mon.c (mon.2) 84 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:05:54.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:54 smithi055 bash[9296]: audit 2020-09-08T16:05:54.055666+0000 mon.a (mon.0) 2090 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:05:54.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:54 smithi055 bash[9296]: audit 2020-09-08T16:05:54.060116+0000 mon.a (mon.0) 2091 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:05:54.874 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 48 16 91268 91252 118.797 80.75 0.00475982 0.00841546 2020-09-08T16:05:54.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:05:54 smithi104 bash[15273]: debug 2020-09-08T16:05:54.492+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:55.369 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:05:55.474 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:55.674 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:05:55.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:05:55 smithi104 bash[19922]: debug 2020-09-08T16:05:55.368+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:55.779 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:55.797 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:05:55 smithi166 bash[16593]: debug 2020-09-08T16:05:55.669+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:55.876 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 49 16 92545 92529 118.001 79.8125 0.00747213 0.00847257 2020-09-08T16:05:56.046 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:56.094 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T16:05:56.094 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:05:56.150 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:05:56.284 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:05:56.385 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:05:56.394 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:05:56 smithi104 bash[15273]: debug 2020-09-08T16:05:56.048+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:56.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:05:56 smithi055 bash[16646]: debug 2020-09-08T16:05:56.284+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:56.875 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 50 16 93882 93866 117.313 83.5625 0.00756433 0.00852212 2020-09-08T16:05:56.933 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:05:57.038 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:57.240 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:57 smithi104 bash[9203]: cluster 2020-09-08T16:05:56.047105+0000 mgr.x (mgr.24118) 1158 : cluster [DBG] pgmap v1724: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s wr, 1.20k op/s 2020-09-08T16:05:57.240 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:05:56 smithi104 bash[22305]: debug 2020-09-08T16:05:56.936+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:57.240 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:05:57.343 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:05:57.362 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:57 smithi166 bash[9446]: cluster 2020-09-08T16:05:56.047105+0000 mgr.x (mgr.24118) 1158 : cluster [DBG] pgmap v1724: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s wr, 1.20k op/s 2020-09-08T16:05:57.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:57 smithi055 bash[9296]: cluster 2020-09-08T16:05:56.047105+0000 mgr.x (mgr.24118) 1158 : cluster [DBG] pgmap v1724: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s wr, 1.20k op/s 2020-09-08T16:05:57.556 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:05:57.597 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2020-09-08T16:05:57.616 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:05:57 smithi166 bash[14254]: debug 2020-09-08T16:05:57.553+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:57.660 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:05:57.678 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:05:57 smithi104 bash[15273]: debug 2020-09-08T16:05:57.244+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:57.776 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:05:57.799 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:05:57.875 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 51 16 94939 94923 116.308 66.0625 0.00633598 0.00859508 2020-09-08T16:05:57.903 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:05:57.911 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:05:58.013 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:05:57 smithi055 bash[14301]: debug 2020-09-08T16:05:57.800+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:58.015 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:05:58.093 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:05:58.104 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:58 smithi166 bash[9446]: audit 2020-09-08T16:05:58.036530+0000 mon.a (mon.0) 2092 : audit [INF] from='client.16476 v1:172.21.15.104:0/3472765039' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:05:58.118 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T16:05:58.122 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:58.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:58 smithi104 bash[9203]: audit 2020-09-08T16:05:58.036530+0000 mon.a (mon.0) 2092 : audit [INF] from='client.16476 v1:172.21.15.104:0/3472765039' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:05:58.238 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:58.341 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:05:58.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:05:58 smithi055 bash[19003]: debug 2020-09-08T16:05:58.012+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:58.408 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:05:58 smithi055 bash[11956]: debug 2020-09-08T16:05:58.240+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:58.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:58 smithi055 bash[9296]: audit 2020-09-08T16:05:58.036530+0000 mon.a (mon.0) 2092 : audit [INF] from='client.16476 v1:172.21.15.104:0/3472765039' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:05:58.821 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:05:58.875 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 52 16 96282 96266 115.685 83.9375 0.00671348 0.00864187 2020-09-08T16:05:58.925 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:05:59.106 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:05:58 smithi166 bash[18960]: debug 2020-09-08T16:05:58.821+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:05:59.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:59 smithi166 bash[9446]: cluster 2020-09-08T16:05:58.047625+0000 mgr.x (mgr.24118) 1159 : cluster [DBG] pgmap v1725: 17 pgs: 17 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.46k op/s 2020-09-08T16:05:59.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:59 smithi166 bash[9446]: cluster 2020-09-08T16:05:58.107745+0000 mon.a (mon.0) 2093 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:05:59.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:59 smithi166 bash[9446]: audit 2020-09-08T16:05:58.119395+0000 mon.a (mon.0) 2094 : audit [INF] from='client.16476 v1:172.21.15.104:0/3472765039' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:05:59.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:05:59 smithi166 bash[9446]: cluster 2020-09-08T16:05:58.119473+0000 mon.a (mon.0) 2095 : cluster [DBG] osdmap e651: 12 total, 12 up, 12 in 2020-09-08T16:05:59.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:59 smithi055 bash[9296]: cluster 2020-09-08T16:05:58.047625+0000 mgr.x (mgr.24118) 1159 : cluster [DBG] pgmap v1725: 17 pgs: 17 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.46k op/s 2020-09-08T16:05:59.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:59 smithi055 bash[9296]: cluster 2020-09-08T16:05:58.107745+0000 mon.a (mon.0) 2093 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:05:59.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:59 smithi055 bash[9296]: audit 2020-09-08T16:05:58.119395+0000 mon.a (mon.0) 2094 : audit [INF] from='client.16476 v1:172.21.15.104:0/3472765039' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:05:59.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:05:59 smithi055 bash[9296]: cluster 2020-09-08T16:05:58.119473+0000 mon.a (mon.0) 2095 : cluster [DBG] osdmap e651: 12 total, 12 up, 12 in 2020-09-08T16:05:59.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:59 smithi104 bash[9203]: cluster 2020-09-08T16:05:58.047625+0000 mgr.x (mgr.24118) 1159 : cluster [DBG] pgmap v1725: 17 pgs: 17 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.46k op/s 2020-09-08T16:05:59.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:59 smithi104 bash[9203]: cluster 2020-09-08T16:05:58.107745+0000 mon.a (mon.0) 2093 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:05:59.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:59 smithi104 bash[9203]: audit 2020-09-08T16:05:58.119395+0000 mon.a (mon.0) 2094 : audit [INF] from='client.16476 v1:172.21.15.104:0/3472765039' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:05:59.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:05:59 smithi104 bash[9203]: cluster 2020-09-08T16:05:58.119473+0000 mon.a (mon.0) 2095 : cluster [DBG] osdmap e651: 12 total, 12 up, 12 in 2020-09-08T16:05:59.875 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 53 16 97403 97387 114.824 70.0625 0.014879 0.00870685 2020-09-08T16:06:00.045 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:00.150 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:00.288 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:00 smithi166 bash[9446]: cluster 2020-09-08T16:05:59.297599+0000 osd.4 (osd.4) 69 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:00.288 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:00 smithi166 bash[9446]: cluster 2020-09-08T16:05:59.297746+0000 osd.4 (osd.4) 70 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:00.357 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:00.358 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:06:00 smithi104 bash[19922]: debug 2020-09-08T16:06:00.360+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:00.358 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:00 smithi104 bash[9203]: cluster 2020-09-08T16:05:59.297599+0000 osd.4 (osd.4) 69 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:00.358 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:00 smithi104 bash[9203]: cluster 2020-09-08T16:05:59.297746+0000 osd.4 (osd.4) 70 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:00.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:06:00 smithi055 bash[11956]: debug 2020-09-08T16:06:00.044+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:00.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:00 smithi055 bash[9296]: cluster 2020-09-08T16:05:59.297599+0000 osd.4 (osd.4) 69 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:00.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:00 smithi055 bash[9296]: cluster 2020-09-08T16:05:59.297746+0000 osd.4 (osd.4) 70 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:00.460 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:00.580 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:00.685 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:06:00.798 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:06:00.875 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 54 16 98585 98569 114.066 73.875 0.00621381 0.00876475 2020-09-08T16:06:00.904 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:00.906 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:06:00 smithi055 bash[14301]: debug 2020-09-08T16:06:00.580+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:00.912 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:06:00 smithi104 bash[22305]: debug 2020-09-08T16:06:00.800+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:01.076 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:01.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:06:01 smithi104 bash[15273]: debug 2020-09-08T16:06:01.080+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:01.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:01 smithi104 bash[9203]: cluster 2020-09-08T16:06:00.039743+0000 osd.11 (osd.11) 108 : cluster [DBG] 8.d scrub starts 2020-09-08T16:06:01.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:01 smithi104 bash[9203]: cluster 2020-09-08T16:06:00.048272+0000 mgr.x (mgr.24118) 1160 : cluster [DBG] pgmap v1727: 17 pgs: 17 active+clean; 5.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 MiB/s wr, 1.55k op/s 2020-09-08T16:06:01.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:01 smithi104 bash[9203]: cluster 2020-09-08T16:06:00.123213+0000 mon.a (mon.0) 2096 : cluster [DBG] osdmap e652: 12 total, 12 up, 12 in 2020-09-08T16:06:01.182 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:01.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:01 smithi055 bash[9296]: cluster 2020-09-08T16:06:00.039743+0000 osd.11 (osd.11) 108 : cluster [DBG] 8.d scrub starts 2020-09-08T16:06:01.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:01 smithi055 bash[9296]: cluster 2020-09-08T16:06:00.048272+0000 mgr.x (mgr.24118) 1160 : cluster [DBG] pgmap v1727: 17 pgs: 17 active+clean; 5.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 MiB/s wr, 1.55k op/s 2020-09-08T16:06:01.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:01 smithi055 bash[9296]: cluster 2020-09-08T16:06:00.123213+0000 mon.a (mon.0) 2096 : cluster [DBG] osdmap e652: 12 total, 12 up, 12 in 2020-09-08T16:06:01.481 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:01 smithi166 bash[9446]: cluster 2020-09-08T16:06:00.039743+0000 osd.11 (osd.11) 108 : cluster [DBG] 8.d scrub starts 2020-09-08T16:06:01.482 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:01 smithi166 bash[9446]: cluster 2020-09-08T16:06:00.048272+0000 mgr.x (mgr.24118) 1160 : cluster [DBG] pgmap v1727: 17 pgs: 17 active+clean; 5.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 MiB/s wr, 1.55k op/s 2020-09-08T16:06:01.482 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:01 smithi166 bash[9446]: cluster 2020-09-08T16:06:00.123213+0000 mon.a (mon.0) 2096 : cluster [DBG] osdmap e652: 12 total, 12 up, 12 in 2020-09-08T16:06:01.482 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:01.482 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:06:01 smithi166 bash[18960]: debug 2020-09-08T16:06:01.481+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:01.585 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:01.716 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:01.820 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:06:01.875 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 55 16 99794 99778 113.365 75.5625 0.0150044 0.00881918 2020-09-08T16:06:01.906 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:06:01 smithi055 bash[19003]: debug 2020-09-08T16:06:01.716+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:02.260 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T16:06:02.416 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:06:02.636 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:06:02.739 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:06:02.875 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 56 16 101073 101057 112.768 79.9375 0.0121681 0.00886519 2020-09-08T16:06:03.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:06:02 smithi166 bash[14254]: debug 2020-09-08T16:06:02.637+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:03.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:03 smithi055 bash[9296]: cluster 2020-09-08T16:06:02.048742+0000 mgr.x (mgr.24118) 1161 : cluster [DBG] pgmap v1729: 17 pgs: 17 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s wr, 950 op/s 2020-09-08T16:06:03.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:03 smithi055 bash[9296]: cluster 2020-09-08T16:06:02.361933+0000 osd.11 (osd.11) 109 : cluster [DBG] 8.d scrub ok 2020-09-08T16:06:03.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:03 smithi104 bash[9203]: cluster 2020-09-08T16:06:02.048742+0000 mgr.x (mgr.24118) 1161 : cluster [DBG] pgmap v1729: 17 pgs: 17 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s wr, 950 op/s 2020-09-08T16:06:03.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:03 smithi104 bash[9203]: cluster 2020-09-08T16:06:02.361933+0000 osd.11 (osd.11) 109 : cluster [DBG] 8.d scrub ok 2020-09-08T16:06:03.556 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:03 smithi166 bash[9446]: cluster 2020-09-08T16:06:02.048742+0000 mgr.x (mgr.24118) 1161 : cluster [DBG] pgmap v1729: 17 pgs: 17 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s wr, 950 op/s 2020-09-08T16:06:03.556 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:03 smithi166 bash[9446]: cluster 2020-09-08T16:06:02.361933+0000 osd.11 (osd.11) 109 : cluster [DBG] 8.d scrub ok 2020-09-08T16:06:03.557 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:06:03 smithi166 bash[14254]: debug 2020-09-08T16:06:03.553+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:03.557 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:06:03.660 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:03.876 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 57 16 102393 102377 112.237 82.5 0.0225244 0.00890807 2020-09-08T16:06:03.896 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:03.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:06:03 smithi104 bash[15273]: debug 2020-09-08T16:06:03.900+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:03.999 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:04.199 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T16:06:04.200 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:06:04.291 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:04 smithi166 bash[9446]: cluster 2020-09-08T16:06:03.350368+0000 osd.7 (osd.7) 139 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:06:04.422 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:04 smithi104 bash[9203]: cluster 2020-09-08T16:06:03.350368+0000 osd.7 (osd.7) 139 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:06:04.431 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:04 smithi055 bash[9296]: cluster 2020-09-08T16:06:03.350368+0000 osd.7 (osd.7) 139 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:06:04.527 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:06:04.554 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:06:04.582 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:06:04.616 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:06:04.648 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:06:04.683 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:06:04.718 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:06:04.746 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:06:04.781 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:06:04.812 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:06:04.849 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:06:04.876 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 58 16 103642 103626 111.648 78.0625 0.0148964 0.00895505 2020-09-08T16:06:04.882 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:06:04.884 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:06:04.884 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:06:04.884 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:06:04.885 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:06:04.885 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:06:04.885 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:06:04.885 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:06:04.886 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:06:04.886 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:06:04.886 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:06:04.887 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:06:04.887 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:06:05.349 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:05.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:06:05 smithi055 bash[19003]: debug 2020-09-08T16:06:05.352+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:05.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:05 smithi055 bash[9296]: cluster 2020-09-08T16:06:03.466114+0000 osd.7 (osd.7) 140 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:06:05.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:05 smithi055 bash[9296]: cluster 2020-09-08T16:06:04.049229+0000 mgr.x (mgr.24118) 1162 : cluster [DBG] pgmap v1730: 17 pgs: 17 active+clean; 6.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s wr, 1.58k op/s 2020-09-08T16:06:05.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:05 smithi055 bash[9296]: audit 2020-09-08T16:06:04.496560+0000 mon.c (mon.2) 85 : audit [DBG] from='client.? v1:172.21.15.104:0/2391552261' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:06:05.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:05 smithi104 bash[9203]: cluster 2020-09-08T16:06:03.466114+0000 osd.7 (osd.7) 140 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:06:05.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:05 smithi104 bash[9203]: cluster 2020-09-08T16:06:04.049229+0000 mgr.x (mgr.24118) 1162 : cluster [DBG] pgmap v1730: 17 pgs: 17 active+clean; 6.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s wr, 1.58k op/s 2020-09-08T16:06:05.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:05 smithi104 bash[9203]: audit 2020-09-08T16:06:04.496560+0000 mon.c (mon.2) 85 : audit [DBG] from='client.? v1:172.21.15.104:0/2391552261' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:06:05.453 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:05.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:05 smithi166 bash[9446]: cluster 2020-09-08T16:06:03.466114+0000 osd.7 (osd.7) 140 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:06:05.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:05 smithi166 bash[9446]: cluster 2020-09-08T16:06:04.049229+0000 mgr.x (mgr.24118) 1162 : cluster [DBG] pgmap v1730: 17 pgs: 17 active+clean; 6.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s wr, 1.58k op/s 2020-09-08T16:06:05.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:05 smithi166 bash[9446]: audit 2020-09-08T16:06:04.496560+0000 mon.c (mon.2) 85 : audit [DBG] from='client.? v1:172.21.15.104:0/2391552261' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:06:05.812 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2020-09-08T16:06:05.876 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 59 16 104849 104833 111.034 75.4375 0.00853462 0.00900371 2020-09-08T16:06:05.943 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:05.981 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:06:06.048 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:06.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:06 smithi055 bash[9296]: cluster 2020-09-08T16:06:04.944438+0000 osd.8 (osd.8) 87 : cluster [DBG] 8.a scrub starts 2020-09-08T16:06:06.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:06:05 smithi104 bash[17588]: debug 2020-09-08T16:06:05.944+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:06.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:06 smithi104 bash[9203]: cluster 2020-09-08T16:06:04.944438+0000 osd.8 (osd.8) 87 : cluster [DBG] 8.a scrub starts 2020-09-08T16:06:06.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:06 smithi166 bash[9446]: cluster 2020-09-08T16:06:04.944438+0000 osd.8 (osd.8) 87 : cluster [DBG] 8.a scrub starts 2020-09-08T16:06:06.876 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 16:06:06.881011 min lat: 0.00181942 max lat: 0.615684 avg lat: 0.00905516 2020-09-08T16:06:06.876 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T16:06:06.877 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 60 16 106024 106008 110.407 73.4375 0.00706713 0.00905516 2020-09-08T16:06:07.143 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:07 smithi104 bash[9203]: cluster 2020-09-08T16:06:06.049734+0000 mgr.x (mgr.24118) 1163 : cluster [DBG] pgmap v1731: 17 pgs: 17 active+clean; 6.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 63 MiB/s wr, 1.01k op/s 2020-09-08T16:06:07.144 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:07 smithi104 bash[9203]: cluster 2020-09-08T16:06:06.705922+0000 osd.10 (osd.10) 84 : cluster [DBG] 8.5 scrub starts 2020-09-08T16:06:07.144 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:07 smithi104 bash[9203]: cluster 2020-09-08T16:06:06.794576+0000 osd.5 (osd.5) 105 : cluster [DBG] 8.4 scrub starts 2020-09-08T16:06:07.241 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:07.345 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:07.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:06:07 smithi055 bash[19003]: debug 2020-09-08T16:06:07.244+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:07.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:07 smithi055 bash[9296]: cluster 2020-09-08T16:06:06.049734+0000 mgr.x (mgr.24118) 1163 : cluster [DBG] pgmap v1731: 17 pgs: 17 active+clean; 6.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 63 MiB/s wr, 1.01k op/s 2020-09-08T16:06:07.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:07 smithi055 bash[9296]: cluster 2020-09-08T16:06:06.705922+0000 osd.10 (osd.10) 84 : cluster [DBG] 8.5 scrub starts 2020-09-08T16:06:07.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:07 smithi055 bash[9296]: cluster 2020-09-08T16:06:06.794576+0000 osd.5 (osd.5) 105 : cluster [DBG] 8.4 scrub starts 2020-09-08T16:06:07.458 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:07.564 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:06:07.572 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:07 smithi166 bash[9446]: cluster 2020-09-08T16:06:06.049734+0000 mgr.x (mgr.24118) 1163 : cluster [DBG] pgmap v1731: 17 pgs: 17 active+clean; 6.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 63 MiB/s wr, 1.01k op/s 2020-09-08T16:06:07.573 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:07 smithi166 bash[9446]: cluster 2020-09-08T16:06:06.705922+0000 osd.10 (osd.10) 84 : cluster [DBG] 8.5 scrub starts 2020-09-08T16:06:07.573 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:07 smithi166 bash[9446]: cluster 2020-09-08T16:06:06.794576+0000 osd.5 (osd.5) 105 : cluster [DBG] 8.4 scrub starts 2020-09-08T16:06:07.876 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 61 16 107228 107212 109.831 75.25 0.00869584 0.00910275 2020-09-08T16:06:07.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:06:07 smithi055 bash[19003]: debug 2020-09-08T16:06:07.460+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:08.167 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:06:08.271 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:08.392 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:08.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:08 smithi055 bash[9296]: cluster 2020-09-08T16:06:06.884669+0000 osd.8 (osd.8) 88 : cluster [DBG] 8.a scrub ok 2020-09-08T16:06:08.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:06:08 smithi055 bash[14301]: debug 2020-09-08T16:06:08.392+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:08.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:08 smithi104 bash[9203]: cluster 2020-09-08T16:06:06.884669+0000 osd.8 (osd.8) 88 : cluster [DBG] 8.a scrub ok 2020-09-08T16:06:08.488 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:08 smithi166 bash[9446]: cluster 2020-09-08T16:06:06.884669+0000 osd.8 (osd.8) 88 : cluster [DBG] 8.a scrub ok 2020-09-08T16:06:08.488 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:06:08 smithi166 bash[16593]: debug 2020-09-08T16:06:08.165+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:08.498 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:08.622 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:08.724 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:06:08.725 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:08.843 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:06:08 smithi055 bash[11956]: debug 2020-09-08T16:06:08.620+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:08.844 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:08.876 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 62 16 108396 108380 109.237 73 0.00552446 0.00915239 2020-09-08T16:06:08.877 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:06:08.951 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:06:09.152 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:09 smithi104 bash[9203]: cluster 2020-09-08T16:06:08.050379+0000 mgr.x (mgr.24118) 1164 : cluster [DBG] pgmap v1732: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 6.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T16:06:09.153 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:09 smithi104 bash[9203]: cluster 2020-09-08T16:06:08.642141+0000 osd.9 (osd.9) 97 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:09.153 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:09 smithi104 bash[9203]: cluster 2020-09-08T16:06:08.642347+0000 osd.9 (osd.9) 98 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:09.153 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:09 smithi104 bash[9203]: audit 2020-09-08T16:06:08.791413+0000 mon.a (mon.0) 2097 : audit [INF] from='client.? v1:172.21.15.104:0/322694134' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:06:09.154 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:06:08 smithi055 bash[19003]: debug 2020-09-08T16:06:08.844+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:09.154 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:09 smithi055 bash[9296]: cluster 2020-09-08T16:06:08.050379+0000 mgr.x (mgr.24118) 1164 : cluster [DBG] pgmap v1732: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 6.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T16:06:09.155 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:09 smithi055 bash[9296]: cluster 2020-09-08T16:06:08.642141+0000 osd.9 (osd.9) 97 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:09.155 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:09 smithi055 bash[9296]: cluster 2020-09-08T16:06:08.642347+0000 osd.9 (osd.9) 98 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:09.155 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:09 smithi055 bash[9296]: audit 2020-09-08T16:06:08.791413+0000 mon.a (mon.0) 2097 : audit [INF] from='client.? v1:172.21.15.104:0/322694134' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:06:09.170 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T16:06:09.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:09 smithi166 bash[9446]: cluster 2020-09-08T16:06:08.050379+0000 mgr.x (mgr.24118) 1164 : cluster [DBG] pgmap v1732: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 6.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.25k op/s 2020-09-08T16:06:09.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:09 smithi166 bash[9446]: cluster 2020-09-08T16:06:08.642141+0000 osd.9 (osd.9) 97 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:09.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:09 smithi166 bash[9446]: cluster 2020-09-08T16:06:08.642347+0000 osd.9 (osd.9) 98 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:09.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:09 smithi166 bash[9446]: audit 2020-09-08T16:06:08.791413+0000 mon.a (mon.0) 2097 : audit [INF] from='client.? v1:172.21.15.104:0/322694134' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:06:09.876 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 63 16 109492 109476 108.59 68.5 0.00516165 0.00920701 2020-09-08T16:06:10.213 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:06:10.316 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:10.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:10 smithi055 bash[9296]: cluster 2020-09-08T16:06:08.954375+0000 osd.5 (osd.5) 106 : cluster [DBG] 8.4 scrub ok 2020-09-08T16:06:10.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:10 smithi055 bash[9296]: cluster 2020-09-08T16:06:09.019961+0000 osd.10 (osd.10) 85 : cluster [DBG] 8.5 scrub ok 2020-09-08T16:06:10.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:10 smithi055 bash[9296]: cluster 2020-09-08T16:06:09.156550+0000 mon.a (mon.0) 2098 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:06:10.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:10 smithi055 bash[9296]: cluster 2020-09-08T16:06:09.156589+0000 mon.a (mon.0) 2099 : cluster [INF] Cluster is now healthy 2020-09-08T16:06:10.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:10 smithi055 bash[9296]: audit 2020-09-08T16:06:09.167496+0000 mon.a (mon.0) 2100 : audit [INF] from='client.? v1:172.21.15.104:0/322694134' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:06:10.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:10 smithi055 bash[9296]: cluster 2020-09-08T16:06:09.167547+0000 mon.a (mon.0) 2101 : cluster [DBG] osdmap e653: 12 total, 12 up, 12 in 2020-09-08T16:06:10.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:10 smithi055 bash[9296]: cluster 2020-09-08T16:06:09.480510+0000 osd.2 (osd.2) 104 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:10.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:10 smithi055 bash[9296]: cluster 2020-09-08T16:06:09.480606+0000 osd.2 (osd.2) 105 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:10.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:10 smithi104 bash[9203]: cluster 2020-09-08T16:06:08.954375+0000 osd.5 (osd.5) 106 : cluster [DBG] 8.4 scrub ok 2020-09-08T16:06:10.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:10 smithi104 bash[9203]: cluster 2020-09-08T16:06:09.019961+0000 osd.10 (osd.10) 85 : cluster [DBG] 8.5 scrub ok 2020-09-08T16:06:10.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:10 smithi104 bash[9203]: cluster 2020-09-08T16:06:09.156550+0000 mon.a (mon.0) 2098 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:06:10.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:10 smithi104 bash[9203]: cluster 2020-09-08T16:06:09.156589+0000 mon.a (mon.0) 2099 : cluster [INF] Cluster is now healthy 2020-09-08T16:06:10.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:10 smithi104 bash[9203]: audit 2020-09-08T16:06:09.167496+0000 mon.a (mon.0) 2100 : audit [INF] from='client.? v1:172.21.15.104:0/322694134' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:06:10.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:10 smithi104 bash[9203]: cluster 2020-09-08T16:06:09.167547+0000 mon.a (mon.0) 2101 : cluster [DBG] osdmap e653: 12 total, 12 up, 12 in 2020-09-08T16:06:10.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:10 smithi104 bash[9203]: cluster 2020-09-08T16:06:09.480510+0000 osd.2 (osd.2) 104 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:10.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:10 smithi104 bash[9203]: cluster 2020-09-08T16:06:09.480606+0000 osd.2 (osd.2) 105 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:10.447 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:10.550 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:10.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:06:10 smithi166 bash[16593]: debug 2020-09-08T16:06:10.213+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:10.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:10 smithi166 bash[9446]: cluster 2020-09-08T16:06:08.954375+0000 osd.5 (osd.5) 106 : cluster [DBG] 8.4 scrub ok 2020-09-08T16:06:10.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:10 smithi166 bash[9446]: cluster 2020-09-08T16:06:09.019961+0000 osd.10 (osd.10) 85 : cluster [DBG] 8.5 scrub ok 2020-09-08T16:06:10.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:10 smithi166 bash[9446]: cluster 2020-09-08T16:06:09.156550+0000 mon.a (mon.0) 2098 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:06:10.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:10 smithi166 bash[9446]: cluster 2020-09-08T16:06:09.156589+0000 mon.a (mon.0) 2099 : cluster [INF] Cluster is now healthy 2020-09-08T16:06:10.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:10 smithi166 bash[9446]: audit 2020-09-08T16:06:09.167496+0000 mon.a (mon.0) 2100 : audit [INF] from='client.? v1:172.21.15.104:0/322694134' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:06:10.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:10 smithi166 bash[9446]: cluster 2020-09-08T16:06:09.167547+0000 mon.a (mon.0) 2101 : cluster [DBG] osdmap e653: 12 total, 12 up, 12 in 2020-09-08T16:06:10.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:10 smithi166 bash[9446]: cluster 2020-09-08T16:06:09.480510+0000 osd.2 (osd.2) 104 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:10.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:10 smithi166 bash[9446]: cluster 2020-09-08T16:06:09.480606+0000 osd.2 (osd.2) 105 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:10.683 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:10.787 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:10.876 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 64 16 110536 110520 107.913 65.25 0.00869061 0.00926177 2020-09-08T16:06:10.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:06:10 smithi055 bash[16646]: debug 2020-09-08T16:06:10.448+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:10.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:06:10 smithi055 bash[16646]: debug 2020-09-08T16:06:10.684+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:11.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:11 smithi104 bash[9203]: cluster 2020-09-08T16:06:09.943633+0000 osd.8 (osd.8) 89 : cluster [DBG] 8.b scrub starts 2020-09-08T16:06:11.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:11 smithi104 bash[9203]: cluster 2020-09-08T16:06:10.050825+0000 mgr.x (mgr.24118) 1165 : cluster [DBG] pgmap v1734: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.24k op/s 2020-09-08T16:06:11.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:11 smithi104 bash[9203]: cluster 2020-09-08T16:06:10.173180+0000 mon.a (mon.0) 2102 : cluster [DBG] osdmap e654: 12 total, 12 up, 12 in 2020-09-08T16:06:11.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:11 smithi104 bash[9203]: cluster 2020-09-08T16:06:10.352107+0000 osd.0 (osd.0) 116 : cluster [DBG] 8.e deep-scrub starts 2020-09-08T16:06:11.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:11 smithi104 bash[9203]: cluster 2020-09-08T16:06:10.705908+0000 osd.1 (osd.1) 128 : cluster [DBG] 8.1 scrub starts 2020-09-08T16:06:11.461 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:11 smithi166 bash[9446]: cluster 2020-09-08T16:06:09.943633+0000 osd.8 (osd.8) 89 : cluster [DBG] 8.b scrub starts 2020-09-08T16:06:11.462 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:11 smithi166 bash[9446]: cluster 2020-09-08T16:06:10.050825+0000 mgr.x (mgr.24118) 1165 : cluster [DBG] pgmap v1734: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.24k op/s 2020-09-08T16:06:11.462 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:11 smithi166 bash[9446]: cluster 2020-09-08T16:06:10.173180+0000 mon.a (mon.0) 2102 : cluster [DBG] osdmap e654: 12 total, 12 up, 12 in 2020-09-08T16:06:11.463 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:11 smithi166 bash[9446]: cluster 2020-09-08T16:06:10.352107+0000 osd.0 (osd.0) 116 : cluster [DBG] 8.e deep-scrub starts 2020-09-08T16:06:11.463 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:11 smithi166 bash[9446]: cluster 2020-09-08T16:06:10.705908+0000 osd.1 (osd.1) 128 : cluster [DBG] 8.1 scrub starts 2020-09-08T16:06:11.464 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:06:11 smithi166 bash[18960]: debug 2020-09-08T16:06:11.461+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:11.464 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:11.566 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:11.575 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:11 smithi055 bash[9296]: cluster 2020-09-08T16:06:09.943633+0000 osd.8 (osd.8) 89 : cluster [DBG] 8.b scrub starts 2020-09-08T16:06:11.576 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:11 smithi055 bash[9296]: cluster 2020-09-08T16:06:10.050825+0000 mgr.x (mgr.24118) 1165 : cluster [DBG] pgmap v1734: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 6.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.24k op/s 2020-09-08T16:06:11.576 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:11 smithi055 bash[9296]: cluster 2020-09-08T16:06:10.173180+0000 mon.a (mon.0) 2102 : cluster [DBG] osdmap e654: 12 total, 12 up, 12 in 2020-09-08T16:06:11.576 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:11 smithi055 bash[9296]: cluster 2020-09-08T16:06:10.352107+0000 osd.0 (osd.0) 116 : cluster [DBG] 8.e deep-scrub starts 2020-09-08T16:06:11.577 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:11 smithi055 bash[9296]: cluster 2020-09-08T16:06:10.705908+0000 osd.1 (osd.1) 128 : cluster [DBG] 8.1 scrub starts 2020-09-08T16:06:11.718 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:11.823 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:11.876 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 65 16 111331 111315 107.017 49.6875 0.00925148 0.00934067 2020-09-08T16:06:11.906 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:06:11 smithi055 bash[11956]: debug 2020-09-08T16:06:11.720+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:12.368 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:12.471 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:06:12.654 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:06:12 smithi166 bash[16593]: debug 2020-09-08T16:06:12.653+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:12.654 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:06:12 smithi166 bash[18960]: debug 2020-09-08T16:06:12.369+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:12.661 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:06:12.739 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T16:06:12.762 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:12.876 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 66 16 112230 112214 106.247 56.1875 0.0102234 0.00940987 2020-09-08T16:06:12.909 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:06:13.043 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:06:13.043 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T16:06:13.043 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.043 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:112366 8.f 8:f92bb65b:::benchmark_data_smithi166_52302_object112365:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e654)", 2020-09-08T16:06:13.044 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:06:13.027816+0000", 2020-09-08T16:06:13.044 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 0.018120134999999999, 2020-09-08T16:06:13.044 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.018156414999999999, 2020-09-08T16:06:13.044 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:06:13.044 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:06:13.045 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:06:13.045 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:06:13.045 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:06:13.045 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 112366 2020-09-08T16:06:13.046 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.046 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:06:13.046 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.046 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:06:13.046 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.027816+0000", 2020-09-08T16:06:13.046 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:06:13.047 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.047 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.048 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:06:13.048 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.027816+0000", 2020-09-08T16:06:13.048 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999623 2020-09-08T16:06:13.048 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.048 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.049 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:06:13.049 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.027778+0000", 2020-09-08T16:06:13.049 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013076400000000001 2020-09-08T16:06:13.049 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.049 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.050 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:06:13.050 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.027909+0000", 2020-09-08T16:06:13.050 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.0123999999999999e-05 2020-09-08T16:06:13.050 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.050 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.051 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:06:13.051 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.027929+0000", 2020-09-08T16:06:13.051 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.9159999999999996e-06 2020-09-08T16:06:13.051 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.051 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.051 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:06:13.052 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.027939+0000", 2020-09-08T16:06:13.052 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0080510919999999993 2020-09-08T16:06:13.052 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.052 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.052 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:06:13.054 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.035990+0000", 2020-09-08T16:06:13.054 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00038267900000000003 2020-09-08T16:06:13.054 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.054 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.055 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:06:13.055 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.036372+0000", 2020-09-08T16:06:13.055 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017360300000000001 2020-09-08T16:06:13.055 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.056 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.056 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:06:13.056 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.036546+0000", 2020-09-08T16:06:13.056 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0089464829999999999 2020-09-08T16:06:13.056 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.057 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.057 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:06:13.057 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.045492+0000", 2020-09-08T16:06:13.058 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.017676853999999999 2020-09-08T16:06:13.058 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:06:13.058 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:06:13.058 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:06:13.058 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.059 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.059 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:112368 8.6 8:66a95355:::benchmark_data_smithi166_52302_object112367:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e654)", 2020-09-08T16:06:13.059 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:06:13.028868+0000", 2020-09-08T16:06:13.059 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 0.017068169000000001, 2020-09-08T16:06:13.059 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.017202673000000002, 2020-09-08T16:06:13.060 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:06:13.060 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:06:13.060 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:06:13.060 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:06:13.060 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:06:13.061 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 112368 2020-09-08T16:06:13.061 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.061 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:06:13.061 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.061 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:06:13.062 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.028868+0000", 2020-09-08T16:06:13.062 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:06:13.062 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.062 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.062 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:06:13.063 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.028868+0000", 2020-09-08T16:06:13.063 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999328 2020-09-08T16:06:13.063 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.063 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.063 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:06:13.064 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.028800+0000", 2020-09-08T16:06:13.064 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016110500000000001 2020-09-08T16:06:13.064 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.064 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.065 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:06:13.065 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.028962+0000", 2020-09-08T16:06:13.065 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.8609999999999993e-06 2020-09-08T16:06:13.072 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.072 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.072 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:06:13.073 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.028971+0000", 2020-09-08T16:06:13.073 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.2429999999999996e-06 2020-09-08T16:06:13.073 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.073 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.073 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:06:13.074 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.028977+0000", 2020-09-08T16:06:13.074 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00034386200000000001 2020-09-08T16:06:13.075 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.075 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.075 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:06:13.075 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.029320+0000", 2020-09-08T16:06:13.075 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015879800000000001 2020-09-08T16:06:13.076 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.076 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.076 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:06:13.076 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.029479+0000", 2020-09-08T16:06:13.077 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000208044 2020-09-08T16:06:13.077 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.077 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.077 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 10", 2020-09-08T16:06:13.077 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.029687+0000", 2020-09-08T16:06:13.078 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0046785189999999999 2020-09-08T16:06:13.078 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.078 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.078 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:06:13.078 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.034366+0000", 2020-09-08T16:06:13.079 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.005498335 2020-09-08T16:06:13.079 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:06:13.079 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:06:13.079 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:06:13.079 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.080 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.080 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:112369 8.f 8:f9cd95b9:::benchmark_data_smithi166_52302_object112368:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e654)", 2020-09-08T16:06:13.080 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:06:13.029334+0000", 2020-09-08T16:06:13.080 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 0.016601319, 2020-09-08T16:06:13.080 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.016824573999999998, 2020-09-08T16:06:13.081 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:06:13.081 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:06:13.081 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:06:13.081 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:06:13.081 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:06:13.082 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 112369 2020-09-08T16:06:13.082 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.083 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:06:13.083 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.083 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:06:13.083 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.029334+0000", 2020-09-08T16:06:13.083 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:06:13.084 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.084 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.084 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:06:13.084 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.029334+0000", 2020-09-08T16:06:13.085 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999785 2020-09-08T16:06:13.085 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.085 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.085 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:06:13.086 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.029313+0000", 2020-09-08T16:06:13.086 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.4693999999999998e-05 2020-09-08T16:06:13.086 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.086 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.086 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:06:13.087 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.029408+0000", 2020-09-08T16:06:13.087 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.9100000000000005e-06 2020-09-08T16:06:13.087 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.087 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.088 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:06:13.088 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.029416+0000", 2020-09-08T16:06:13.088 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.7100000000000001e-06 2020-09-08T16:06:13.088 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.089 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.089 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:06:13.089 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.029419+0000", 2020-09-08T16:06:13.089 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.011582947999999999 2020-09-08T16:06:13.089 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.090 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.090 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:06:13.090 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.041002+0000", 2020-09-08T16:06:13.090 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000151199 2020-09-08T16:06:13.090 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.091 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.091 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:06:13.091 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.041153+0000", 2020-09-08T16:06:13.091 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015887200000000001 2020-09-08T16:06:13.091 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.092 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.092 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:06:13.092 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.041312+0000", 2020-09-08T16:06:13.093 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.01197791 2020-09-08T16:06:13.093 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:06:13.093 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:06:13.093 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:06:13.094 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.094 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.094 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.36335.0:112371 8.7 e654/627 8:ec1ca0e0:::benchmark_data_smithi166_52302_object112370:head v 654'7075, mlcod=654'7073)", 2020-09-08T16:06:13.094 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:06:13.030875+0000", 2020-09-08T16:06:13.094 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 0.015060809, 2020-09-08T16:06:13.095 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.015364622, 2020-09-08T16:06:13.095 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:06:13.095 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "started", 2020-09-08T16:06:13.096 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:06:13.096 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.096 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:06:13.097 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.030875+0000", 2020-09-08T16:06:13.097 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:06:13.097 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.097 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.097 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:06:13.098 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.030875+0000", 2020-09-08T16:06:13.098 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7752999999999997e-05 2020-09-08T16:06:13.098 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.098 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.098 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:06:13.098 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.030973+0000", 2020-09-08T16:06:13.099 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8599999999999999e-06 2020-09-08T16:06:13.099 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.099 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.099 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:06:13.099 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.030981+0000", 2020-09-08T16:06:13.100 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3213999999999999e-05 2020-09-08T16:06:13.100 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.100 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.100 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:06:13.100 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.030995+0000", 2020-09-08T16:06:13.101 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7209999999999999e-06 2020-09-08T16:06:13.101 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.101 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.102 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:06:13.102 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.031000+0000", 2020-09-08T16:06:13.102 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000739622 2020-09-08T16:06:13.102 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.102 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.103 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:06:13.103 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.031740+0000", 2020-09-08T16:06:13.104 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.3790000000000001e-05 2020-09-08T16:06:13.104 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.104 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.104 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:06:13.104 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.031764+0000", 2020-09-08T16:06:13.105 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00088896000000000005 2020-09-08T16:06:13.105 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:06:13.105 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:06:13.106 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:06:13.106 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.106 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.106 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.36335.0:112376 8.7 e654/627 8:ec61ec56:::benchmark_data_smithi166_52302_object112375:head v 654'7076, mlcod=654'7073)", 2020-09-08T16:06:13.106 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:06:13.037406+0000", 2020-09-08T16:06:13.107 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 0.0085292489999999992, 2020-09-08T16:06:13.107 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0089054379999999999, 2020-09-08T16:06:13.107 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:06:13.107 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "started", 2020-09-08T16:06:13.107 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:06:13.108 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.108 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:06:13.108 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.037406+0000", 2020-09-08T16:06:13.108 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:06:13.108 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.109 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.109 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:06:13.109 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.037406+0000", 2020-09-08T16:06:13.109 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.9554000000000003e-05 2020-09-08T16:06:13.109 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.110 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.110 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:06:13.110 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.037506+0000", 2020-09-08T16:06:13.110 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.3110000000000006e-06 2020-09-08T16:06:13.110 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.110 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.111 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:06:13.111 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.037514+0000", 2020-09-08T16:06:13.111 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.1316999999999998e-05 2020-09-08T16:06:13.111 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.111 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.112 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:06:13.112 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.037546+0000", 2020-09-08T16:06:13.112 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.0730000000000001e-06 2020-09-08T16:06:13.112 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.112 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.113 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:06:13.113 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.037552+0000", 2020-09-08T16:06:13.113 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0033758099999999999 2020-09-08T16:06:13.114 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.114 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.114 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:06:13.114 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.040928+0000", 2020-09-08T16:06:13.114 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0048881460000000003 2020-09-08T16:06:13.115 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.115 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.115 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:06:13.115 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.045816+0000", 2020-09-08T16:06:13.116 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0084092109999999998 2020-09-08T16:06:13.116 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:06:13.116 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:06:13.116 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:06:13.116 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.117 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.117 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:112378 8.6 8:60e6aaa2:::benchmark_data_smithi166_52302_object112377:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e654)", 2020-09-08T16:06:13.117 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:06:13.038342+0000", 2020-09-08T16:06:13.117 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 0.0075940979999999996, 2020-09-08T16:06:13.117 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0080368390000000005, 2020-09-08T16:06:13.118 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:06:13.118 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:06:13.118 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:06:13.118 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:06:13.119 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:06:13.119 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 112378 2020-09-08T16:06:13.119 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.119 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:06:13.119 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.120 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:06:13.120 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.038342+0000", 2020-09-08T16:06:13.120 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:06:13.120 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.120 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.121 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:06:13.121 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.038342+0000", 2020-09-08T16:06:13.121 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.999958 2020-09-08T16:06:13.121 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.121 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.121 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:06:13.122 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.038300+0000", 2020-09-08T16:06:13.122 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014255999999999999 2020-09-08T16:06:13.122 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.122 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.122 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:06:13.123 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.038442+0000", 2020-09-08T16:06:13.123 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6512e-05 2020-09-08T16:06:13.123 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.123 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.123 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:06:13.124 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.038459+0000", 2020-09-08T16:06:13.124 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.9929999999999998e-06 2020-09-08T16:06:13.124 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.125 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.125 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:06:13.125 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.038466+0000", 2020-09-08T16:06:13.125 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9178000000000002e-05 2020-09-08T16:06:13.125 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.126 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.126 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:06:13.126 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.038515+0000", 2020-09-08T16:06:13.127 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00028539 2020-09-08T16:06:13.127 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.127 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.127 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:06:13.127 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.038800+0000", 2020-09-08T16:06:13.128 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000169801 2020-09-08T16:06:13.128 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.128 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.128 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 10", 2020-09-08T16:06:13.128 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.038970+0000", 2020-09-08T16:06:13.128 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0056122979999999999 2020-09-08T16:06:13.129 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.129 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.129 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:06:13.129 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.044582+0000", 2020-09-08T16:06:13.129 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0062408259999999997 2020-09-08T16:06:13.130 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:06:13.130 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:06:13.130 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:06:13.130 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.130 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.131 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.36335.0:112381 8.7 e654/627)", 2020-09-08T16:06:13.131 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:06:13.042632+0000", 2020-09-08T16:06:13.131 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 0.0033039929999999999, 2020-09-08T16:06:13.131 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0038263939999999999, 2020-09-08T16:06:13.132 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:06:13.132 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "queued for pg", 2020-09-08T16:06:13.132 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:06:13.132 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.132 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:06:13.133 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.042632+0000", 2020-09-08T16:06:13.133 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:06:13.133 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.133 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.133 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:06:13.134 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.042632+0000", 2020-09-08T16:06:13.134 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2199000000000003e-05 2020-09-08T16:06:13.134 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.135 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.135 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:06:13.135 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.042724+0000", 2020-09-08T16:06:13.135 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.4069999999999993e-06 2020-09-08T16:06:13.135 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.136 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.136 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:06:13.136 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.042732+0000", 2020-09-08T16:06:13.136 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.8392e-05 2020-09-08T16:06:13.137 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.137 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.137 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:06:13.137 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.042781+0000", 2020-09-08T16:06:13.137 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2133999999999999e-05 2020-09-08T16:06:13.138 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.138 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.138 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:06:13.138 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.042793+0000", 2020-09-08T16:06:13.139 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000161132 2020-09-08T16:06:13.139 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:06:13.139 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:06:13.139 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:06:13.139 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.139 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.140 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop_reply(client.36335.0:112369 8.f e654/627)", 2020-09-08T16:06:13.140 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:06:13.046197+0000", 2020-09-08T16:06:13.140 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 4294967295.9997387, 2020-09-08T16:06:13.140 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00031180500000000001, 2020-09-08T16:06:13.140 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:06:13.141 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "queued for pg", 2020-09-08T16:06:13.141 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:06:13.141 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.141 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:06:13.141 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.046197+0000", 2020-09-08T16:06:13.142 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:06:13.142 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.142 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.142 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:06:13.142 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.046197+0000", 2020-09-08T16:06:13.143 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.9349999999999999e-05 2020-09-08T16:06:13.143 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.143 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.143 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:06:13.143 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.046226+0000", 2020-09-08T16:06:13.144 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.5210000000000002e-06 2020-09-08T16:06:13.144 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.144 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.144 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:06:13.145 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.046232+0000", 2020-09-08T16:06:13.145 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.4110000000000002e-06 2020-09-08T16:06:13.145 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.145 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.146 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:06:13.146 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.046241+0000", 2020-09-08T16:06:13.146 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.4240000000000003e-06 2020-09-08T16:06:13.147 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:06:13.147 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:06:13.147 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:06:13.147 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:13.046246+0000", 2020-09-08T16:06:13.147 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.8705999999999998e-05 2020-09-08T16:06:13.148 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:06:13.148 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:06:13.148 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:06:13.148 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:06:13.148 INFO:teuthology.orchestra.run.smithi166.stdout: ], 2020-09-08T16:06:13.149 INFO:teuthology.orchestra.run.smithi166.stdout: "num_ops": 8 2020-09-08T16:06:13.149 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:06:13.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:13 smithi104 bash[9203]: cluster 2020-09-08T16:06:11.628266+0000 osd.8 (osd.8) 90 : cluster [DBG] 8.b scrub ok 2020-09-08T16:06:13.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:13 smithi104 bash[9203]: cluster 2020-09-08T16:06:12.051608+0000 mgr.x (mgr.24118) 1166 : cluster [DBG] pgmap v1736: 17 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 15 active+clean; 6.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s wr, 897 op/s 2020-09-08T16:06:13.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:13 smithi104 bash[9203]: cluster 2020-09-08T16:06:12.162117+0000 osd.1 (osd.1) 129 : cluster [DBG] 8.1 scrub ok 2020-09-08T16:06:13.615 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:13.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:13 smithi166 bash[9446]: cluster 2020-09-08T16:06:11.628266+0000 osd.8 (osd.8) 90 : cluster [DBG] 8.b scrub ok 2020-09-08T16:06:13.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:13 smithi166 bash[9446]: cluster 2020-09-08T16:06:12.051608+0000 mgr.x (mgr.24118) 1166 : cluster [DBG] pgmap v1736: 17 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 15 active+clean; 6.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s wr, 897 op/s 2020-09-08T16:06:13.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:13 smithi166 bash[9446]: cluster 2020-09-08T16:06:12.162117+0000 osd.1 (osd.1) 129 : cluster [DBG] 8.1 scrub ok 2020-09-08T16:06:13.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:13 smithi055 bash[9296]: cluster 2020-09-08T16:06:11.628266+0000 osd.8 (osd.8) 90 : cluster [DBG] 8.b scrub ok 2020-09-08T16:06:13.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:13 smithi055 bash[9296]: cluster 2020-09-08T16:06:12.051608+0000 mgr.x (mgr.24118) 1166 : cluster [DBG] pgmap v1736: 17 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 15 active+clean; 6.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s wr, 897 op/s 2020-09-08T16:06:13.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:13 smithi055 bash[9296]: cluster 2020-09-08T16:06:12.162117+0000 osd.1 (osd.1) 129 : cluster [DBG] 8.1 scrub ok 2020-09-08T16:06:13.720 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:13.876 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 67 16 113132 113116 105.502 56.375 0.166891 0.00946814 2020-09-08T16:06:13.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:06:13 smithi104 bash[19922]: debug 2020-09-08T16:06:13.620+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:14.402 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:14.505 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:06:14.565 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2020-09-08T16:06:14.639 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:06:14.731 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:06:14.743 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:14.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:06:14 smithi166 bash[14254]: debug 2020-09-08T16:06:14.637+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:14.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:06:14 smithi166 bash[18960]: debug 2020-09-08T16:06:14.401+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:14.876 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 68 15 113978 113963 104.729 52.9375 0.00681632 0.00954191 2020-09-08T16:06:14.951 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:15.056 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:15.070 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:06:14 smithi104 bash[17588]: debug 2020-09-08T16:06:14.951+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:15.178 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:15.281 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:06:15.410 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:06:15 smithi055 bash[16646]: debug 2020-09-08T16:06:15.180+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:15.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:15 smithi055 bash[9296]: cluster 2020-09-08T16:06:14.052428+0000 mgr.x (mgr.24118) 1167 : cluster [DBG] pgmap v1737: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 6.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 88 MiB/s wr, 1.40k op/s 2020-09-08T16:06:15.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:15 smithi104 bash[9203]: cluster 2020-09-08T16:06:14.052428+0000 mgr.x (mgr.24118) 1167 : cluster [DBG] pgmap v1737: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 6.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 88 MiB/s wr, 1.40k op/s 2020-09-08T16:06:15.484 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:06:15.525 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:06:15.558 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:06:15.592 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:15 smithi166 bash[9446]: cluster 2020-09-08T16:06:14.052428+0000 mgr.x (mgr.24118) 1167 : cluster [DBG] pgmap v1737: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 6.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 88 MiB/s wr, 1.40k op/s 2020-09-08T16:06:15.593 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:06:15.595 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:06:15.625 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:06:15.667 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:06:15.700 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:15.701 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:06:15.734 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:06:15.764 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:06:15.807 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:06:15.858 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:06:15.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:06:15 smithi166 bash[11928]: debug 2020-09-08T16:06:15.589+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:15.879 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 69 16 114941 114925 104.083 60.125 0.0139647 0.00960405 2020-09-08T16:06:15.886 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:06:15.887 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:06:15.887 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:06:15.887 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:06:15.887 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:06:15.888 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:06:15.888 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:06:15.888 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:06:15.888 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:06:15.888 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:06:15.889 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:06:15.889 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:06:15.890 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:06:16.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:16 smithi104 bash[9203]: cluster 2020-09-08T16:06:15.382460+0000 osd.9 (osd.9) 99 : cluster [DBG] 8.c scrub starts 2020-09-08T16:06:16.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:16 smithi104 bash[9203]: audit 2020-09-08T16:06:15.454654+0000 mon.a (mon.0) 2103 : audit [DBG] from='client.? v1:172.21.15.104:0/1881191617' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:06:16.564 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:16.564 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:16 smithi166 bash[9446]: cluster 2020-09-08T16:06:15.382460+0000 osd.9 (osd.9) 99 : cluster [DBG] 8.c scrub starts 2020-09-08T16:06:16.564 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:16 smithi166 bash[9446]: audit 2020-09-08T16:06:15.454654+0000 mon.a (mon.0) 2103 : audit [DBG] from='client.? v1:172.21.15.104:0/1881191617' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:06:16.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:16 smithi055 bash[9296]: cluster 2020-09-08T16:06:15.382460+0000 osd.9 (osd.9) 99 : cluster [DBG] 8.c scrub starts 2020-09-08T16:06:16.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:16 smithi055 bash[9296]: audit 2020-09-08T16:06:15.454654+0000 mon.a (mon.0) 2103 : audit [DBG] from='client.? v1:172.21.15.104:0/1881191617' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:06:16.668 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:16.782 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:16.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:06:16 smithi166 bash[18960]: debug 2020-09-08T16:06:16.561+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:16.877 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 70 15 115685 115670 103.261 46.5625 0.00877874 0.00967163 2020-09-08T16:06:16.887 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:17.003 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:06:16 smithi055 bash[19003]: debug 2020-09-08T16:06:16.783+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:17.003 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:17.105 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:06:17.270 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:06:17.372 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:17.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:17 smithi055 bash[9296]: cluster 2020-09-08T16:06:16.053018+0000 mgr.x (mgr.24118) 1168 : cluster [DBG] pgmap v1738: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 6.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s wr, 864 op/s 2020-09-08T16:06:17.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:17 smithi055 bash[9296]: cluster 2020-09-08T16:06:16.167003+0000 osd.11 (osd.11) 110 : cluster [DBG] 8.6 scrub starts 2020-09-08T16:06:17.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:06:17 smithi055 bash[11956]: debug 2020-09-08T16:06:16.995+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:17.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:17 smithi166 bash[9446]: cluster 2020-09-08T16:06:16.053018+0000 mgr.x (mgr.24118) 1168 : cluster [DBG] pgmap v1738: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 6.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s wr, 864 op/s 2020-09-08T16:06:17.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:17 smithi166 bash[9446]: cluster 2020-09-08T16:06:16.167003+0000 osd.11 (osd.11) 110 : cluster [DBG] 8.6 scrub starts 2020-09-08T16:06:17.617 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:06:17 smithi166 bash[16593]: debug 2020-09-08T16:06:17.265+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:17.643 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:17 smithi104 bash[9203]: cluster 2020-09-08T16:06:16.053018+0000 mgr.x (mgr.24118) 1168 : cluster [DBG] pgmap v1738: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 6.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s wr, 864 op/s 2020-09-08T16:06:17.644 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:17 smithi104 bash[9203]: cluster 2020-09-08T16:06:16.167003+0000 osd.11 (osd.11) 110 : cluster [DBG] 8.6 scrub starts 2020-09-08T16:06:17.644 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:06:17 smithi104 bash[19922]: debug 2020-09-08T16:06:17.647+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:17.645 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:17.755 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:17.877 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 71 16 116421 116405 102.454 45.9375 0.00674043 0.00975657 2020-09-08T16:06:18.106 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:18.210 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:18.340 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:18.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:18 smithi104 bash[9203]: cluster 2020-09-08T16:06:17.043401+0000 osd.9 (osd.9) 100 : cluster [DBG] 8.c scrub ok 2020-09-08T16:06:18.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:18 smithi104 bash[9203]: cluster 2020-09-08T16:06:17.341225+0000 osd.7 (osd.7) 141 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:18.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:18 smithi104 bash[9203]: cluster 2020-09-08T16:06:17.341338+0000 osd.7 (osd.7) 142 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:18.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:18 smithi104 bash[9203]: cluster 2020-09-08T16:06:17.548979+0000 osd.0 (osd.0) 117 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:18.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:18 smithi104 bash[9203]: cluster 2020-09-08T16:06:17.549070+0000 osd.0 (osd.0) 118 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:18.428 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:06:18 smithi104 bash[17588]: debug 2020-09-08T16:06:18.107+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:18.444 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:06:18.453 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:18 smithi166 bash[9446]: cluster 2020-09-08T16:06:17.043401+0000 osd.9 (osd.9) 100 : cluster [DBG] 8.c scrub ok 2020-09-08T16:06:18.454 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:18 smithi166 bash[9446]: cluster 2020-09-08T16:06:17.341225+0000 osd.7 (osd.7) 141 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:18.454 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:18 smithi166 bash[9446]: cluster 2020-09-08T16:06:17.341338+0000 osd.7 (osd.7) 142 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:18.454 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:18 smithi166 bash[9446]: cluster 2020-09-08T16:06:17.548979+0000 osd.0 (osd.0) 117 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:18.454 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:18 smithi166 bash[9446]: cluster 2020-09-08T16:06:17.549070+0000 osd.0 (osd.0) 118 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:18.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:06:18 smithi055 bash[11956]: debug 2020-09-08T16:06:18.339+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:18.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:18 smithi055 bash[9296]: cluster 2020-09-08T16:06:17.043401+0000 osd.9 (osd.9) 100 : cluster [DBG] 8.c scrub ok 2020-09-08T16:06:18.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:18 smithi055 bash[9296]: cluster 2020-09-08T16:06:17.341225+0000 osd.7 (osd.7) 141 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:18.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:18 smithi055 bash[9296]: cluster 2020-09-08T16:06:17.341338+0000 osd.7 (osd.7) 142 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:18.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:18 smithi055 bash[9296]: cluster 2020-09-08T16:06:17.548979+0000 osd.0 (osd.0) 117 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:18.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:18 smithi055 bash[9296]: cluster 2020-09-08T16:06:17.549070+0000 osd.0 (osd.0) 118 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:18.746 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:06:18.747 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:06:18 smithi166 bash[16593]: debug 2020-09-08T16:06:18.737+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:18.849 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:18.877 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 72 16 117321 117305 101.812 56.25 0.0126378 0.00981365 2020-09-08T16:06:18.964 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:19.068 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:06:19.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:06:18 smithi055 bash[11956]: debug 2020-09-08T16:06:18.963+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:19.252 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:06:19.356 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:19.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:06:19 smithi104 bash[22305]: debug 2020-09-08T16:06:19.255+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:19.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:19 smithi104 bash[9203]: cluster 2020-09-08T16:06:18.053641+0000 mgr.x (mgr.24118) 1169 : cluster [DBG] pgmap v1739: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s wr, 898 op/s 2020-09-08T16:06:19.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:19 smithi104 bash[9203]: cluster 2020-09-08T16:06:18.206060+0000 mon.a (mon.0) 2104 : cluster [DBG] osdmap e655: 12 total, 12 up, 12 in 2020-09-08T16:06:19.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:19 smithi104 bash[9203]: cluster 2020-09-08T16:06:18.242616+0000 osd.11 (osd.11) 111 : cluster [DBG] 8.6 scrub ok 2020-09-08T16:06:19.473 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:19 smithi055 bash[9296]: cluster 2020-09-08T16:06:18.053641+0000 mgr.x (mgr.24118) 1169 : cluster [DBG] pgmap v1739: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s wr, 898 op/s 2020-09-08T16:06:19.474 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:19 smithi055 bash[9296]: cluster 2020-09-08T16:06:18.206060+0000 mon.a (mon.0) 2104 : cluster [DBG] osdmap e655: 12 total, 12 up, 12 in 2020-09-08T16:06:19.474 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:19 smithi055 bash[9296]: cluster 2020-09-08T16:06:18.242616+0000 osd.11 (osd.11) 111 : cluster [DBG] 8.6 scrub ok 2020-09-08T16:06:19.474 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:19.584 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:19.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:19 smithi166 bash[9446]: cluster 2020-09-08T16:06:18.053641+0000 mgr.x (mgr.24118) 1169 : cluster [DBG] pgmap v1739: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s wr, 898 op/s 2020-09-08T16:06:19.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:19 smithi166 bash[9446]: cluster 2020-09-08T16:06:18.206060+0000 mon.a (mon.0) 2104 : cluster [DBG] osdmap e655: 12 total, 12 up, 12 in 2020-09-08T16:06:19.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:19 smithi166 bash[9446]: cluster 2020-09-08T16:06:18.242616+0000 osd.11 (osd.11) 111 : cluster [DBG] 8.6 scrub ok 2020-09-08T16:06:19.710 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:19.750 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:06:19.814 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:19.877 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 73 16 118123 118107 101.104 50.125 0.0142037 0.00987837 2020-09-08T16:06:19.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:06:19 smithi055 bash[14301]: debug 2020-09-08T16:06:19.707+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:19.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:06:19 smithi055 bash[19003]: debug 2020-09-08T16:06:19.475+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:19.920 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:06:20.004 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:20.108 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:20.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:06:20 smithi104 bash[15273]: debug 2020-09-08T16:06:19.995+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:20.216 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:20.323 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:20.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:06:20 smithi055 bash[11956]: debug 2020-09-08T16:06:20.215+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:20.433 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:20.537 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:20.877 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 74 16 118967 118951 100.45 52.75 0.0205936 0.00994861 2020-09-08T16:06:20.884 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T16:06:20.884 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:06:20.890 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:20.906 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:06:20 smithi055 bash[19003]: debug 2020-09-08T16:06:20.435+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:20.995 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:21.004 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T16:06:21.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:06:20 smithi104 bash[17588]: debug 2020-09-08T16:06:20.891+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:21.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:20 smithi104 bash[9203]: cluster 2020-09-08T16:06:20.054135+0000 mgr.x (mgr.24118) 1170 : cluster [DBG] pgmap v1741: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s wr, 942 op/s 2020-09-08T16:06:21.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:20 smithi104 bash[9203]: audit 2020-09-08T16:06:20.090604+0000 mon.a (mon.0) 2105 : audit [INF] from='client.? v1:172.21.15.104:0/811326076' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:06:21.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:20 smithi166 bash[9446]: cluster 2020-09-08T16:06:20.054135+0000 mgr.x (mgr.24118) 1170 : cluster [DBG] pgmap v1741: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s wr, 942 op/s 2020-09-08T16:06:21.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:20 smithi166 bash[9446]: audit 2020-09-08T16:06:20.090604+0000 mon.a (mon.0) 2105 : audit [INF] from='client.? v1:172.21.15.104:0/811326076' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:06:21.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:20 smithi055 bash[9296]: cluster 2020-09-08T16:06:20.054135+0000 mgr.x (mgr.24118) 1170 : cluster [DBG] pgmap v1741: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s wr, 942 op/s 2020-09-08T16:06:21.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:20 smithi055 bash[9296]: audit 2020-09-08T16:06:20.090604+0000 mon.a (mon.0) 2105 : audit [INF] from='client.? v1:172.21.15.104:0/811326076' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:06:21.825 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:21.877 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 75 16 119814 119798 99.817 52.9375 0.00785128 0.0100052 2020-09-08T16:06:21.929 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:06:22.026 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2020-09-08T16:06:22.175 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:06:22.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:06:21 smithi104 bash[17588]: debug 2020-09-08T16:06:21.823+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:22.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:21 smithi104 bash[9203]: cluster 2020-09-08T16:06:20.993611+0000 mon.a (mon.0) 2106 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:06:22.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:21 smithi104 bash[9203]: audit 2020-09-08T16:06:21.004179+0000 mon.a (mon.0) 2107 : audit [INF] from='client.? v1:172.21.15.104:0/811326076' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:06:22.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:21 smithi104 bash[9203]: cluster 2020-09-08T16:06:21.004218+0000 mon.a (mon.0) 2108 : cluster [DBG] osdmap e656: 12 total, 12 up, 12 in 2020-09-08T16:06:22.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:21 smithi166 bash[9446]: cluster 2020-09-08T16:06:20.993611+0000 mon.a (mon.0) 2106 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:06:22.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:21 smithi166 bash[9446]: audit 2020-09-08T16:06:21.004179+0000 mon.a (mon.0) 2107 : audit [INF] from='client.? v1:172.21.15.104:0/811326076' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:06:22.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:21 smithi166 bash[9446]: cluster 2020-09-08T16:06:21.004218+0000 mon.a (mon.0) 2108 : cluster [DBG] osdmap e656: 12 total, 12 up, 12 in 2020-09-08T16:06:22.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:21 smithi055 bash[9296]: cluster 2020-09-08T16:06:20.993611+0000 mon.a (mon.0) 2106 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:06:22.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:21 smithi055 bash[9296]: audit 2020-09-08T16:06:21.004179+0000 mon.a (mon.0) 2107 : audit [INF] from='client.? v1:172.21.15.104:0/811326076' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:06:22.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:21 smithi055 bash[9296]: cluster 2020-09-08T16:06:21.004218+0000 mon.a (mon.0) 2108 : cluster [DBG] osdmap e656: 12 total, 12 up, 12 in 2020-09-08T16:06:22.758 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:06:22.860 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:06:22.877 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 76 16 120435 120419 99.0143 38.8125 0.0104516 0.0100897 2020-09-08T16:06:23.005 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:06:22 smithi104 bash[22305]: debug 2020-09-08T16:06:22.755+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:23.301 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:23 smithi104 bash[9203]: cluster 2020-09-08T16:06:22.054974+0000 mgr.x (mgr.24118) 1171 : cluster [DBG] pgmap v1743: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s wr, 628 op/s 2020-09-08T16:06:23.364 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:23 smithi166 bash[9446]: cluster 2020-09-08T16:06:22.054974+0000 mgr.x (mgr.24118) 1171 : cluster [DBG] pgmap v1743: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s wr, 628 op/s 2020-09-08T16:06:23.365 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:06:23.421 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:23 smithi055 bash[9296]: cluster 2020-09-08T16:06:22.054974+0000 mgr.x (mgr.24118) 1171 : cluster [DBG] pgmap v1743: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s wr, 628 op/s 2020-09-08T16:06:23.469 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:23.620 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:06:23 smithi166 bash[11928]: debug 2020-09-08T16:06:23.365+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:23.660 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:23.763 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:23.879 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 77 16 121155 121139 98.3128 45 0.0162285 0.010159 2020-09-08T16:06:23.894 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:23.911 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:06:23 smithi166 bash[18960]: debug 2020-09-08T16:06:23.653+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:23.999 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:24.108 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:06:23 smithi055 bash[16646]: debug 2020-09-08T16:06:23.895+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:24.108 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:24.215 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:24.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:06:24 smithi055 bash[19003]: debug 2020-09-08T16:06:24.107+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:24.437 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:24.541 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:24.667 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:24.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:06:24 smithi104 bash[17588]: debug 2020-09-08T16:06:24.439+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:24.773 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:24.879 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 78 16 121877 121861 97.6309 45.125 0.0130413 0.0102358 2020-09-08T16:06:24.906 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:06:24 smithi055 bash[14301]: debug 2020-09-08T16:06:24.667+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:24.986 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T16:06:25.105 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:25 smithi104 bash[9203]: cluster 2020-09-08T16:06:24.055688+0000 mgr.x (mgr.24118) 1172 : cluster [DBG] pgmap v1744: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 MiB/s wr, 1.03k op/s 2020-09-08T16:06:25.139 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:06:25.146 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:25.249 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:25.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:25 smithi166 bash[9446]: cluster 2020-09-08T16:06:24.055688+0000 mgr.x (mgr.24118) 1172 : cluster [DBG] pgmap v1744: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 MiB/s wr, 1.03k op/s 2020-09-08T16:06:25.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:25 smithi055 bash[9296]: cluster 2020-09-08T16:06:24.055688+0000 mgr.x (mgr.24118) 1172 : cluster [DBG] pgmap v1744: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 MiB/s wr, 1.03k op/s 2020-09-08T16:06:25.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:06:25 smithi104 bash[19922]: debug 2020-09-08T16:06:25.147+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:25.436 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:25.540 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:25.644 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:25.749 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:25.760 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:06:25 smithi055 bash[16646]: debug 2020-09-08T16:06:25.435+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:25.760 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:06:25 smithi055 bash[16646]: debug 2020-09-08T16:06:25.643+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:25.872 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:25.877 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 79 16 122645 122629 97.0026 48 0.00756334 0.0102995 2020-09-08T16:06:25.977 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:26.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:06:25 smithi055 bash[19003]: debug 2020-09-08T16:06:25.871+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:26.314 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:26.418 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:26.617 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:06:26 smithi166 bash[18960]: debug 2020-09-08T16:06:26.309+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:26.642 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:26.746 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:26.753 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:06:26 smithi104 bash[19922]: debug 2020-09-08T16:06:26.643+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:26.834 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T16:06:26.834 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:06:26.878 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 16:06:26.882774 min lat: 0.00181942 max lat: 0.615684 avg lat: 0.0103701 2020-09-08T16:06:26.878 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T16:06:26.878 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 80 16 123432 123416 96.4049 49.1875 0.060696 0.0103701 2020-09-08T16:06:27.130 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:27.234 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:06:27.243 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:27 smithi166 bash[9446]: cluster 2020-09-08T16:06:26.056262+0000 mgr.x (mgr.24118) 1173 : cluster [DBG] pgmap v1745: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s wr, 663 op/s 2020-09-08T16:06:27.287 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:06:27.322 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:06:27.353 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:06:27.387 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:06:27.392 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:27 smithi104 bash[9203]: cluster 2020-09-08T16:06:26.056262+0000 mgr.x (mgr.24118) 1173 : cluster [DBG] pgmap v1745: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s wr, 663 op/s 2020-09-08T16:06:27.392 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:06:27 smithi104 bash[19922]: debug 2020-09-08T16:06:27.131+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:27.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:27 smithi055 bash[9296]: cluster 2020-09-08T16:06:26.056262+0000 mgr.x (mgr.24118) 1173 : cluster [DBG] pgmap v1745: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s wr, 663 op/s 2020-09-08T16:06:27.418 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:06:27.457 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:06:27.493 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:06:27.522 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:06:27.558 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:06:27.593 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:06:27.630 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:06:27.673 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:06:27.676 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:06:27.676 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:06:27.676 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:06:27.676 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:06:27.677 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:06:27.677 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:06:27.677 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:06:27.677 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:06:27.677 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:06:27.678 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:06:27.678 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:06:27.678 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:06:27.792 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:06:27.878 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 81 16 124016 124000 95.6653 36.5 0.00710998 0.0104227 2020-09-08T16:06:27.896 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:27.931 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2020-09-08T16:06:28.073 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:06:28.115 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:06:27 smithi166 bash[11928]: debug 2020-09-08T16:06:27.789+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:28.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:28 smithi166 bash[9446]: audit 2020-09-08T16:06:27.264093+0000 mon.a (mon.0) 2109 : audit [DBG] from='client.? v1:172.21.15.104:0/3644038996' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:06:28.121 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:28.229 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:28.376 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:28.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:28 smithi055 bash[9296]: audit 2020-09-08T16:06:27.264093+0000 mon.a (mon.0) 2109 : audit [DBG] from='client.? v1:172.21.15.104:0/3644038996' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:06:28.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:06:28 smithi055 bash[14301]: debug 2020-09-08T16:06:28.123+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:28.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:28 smithi104 bash[9203]: audit 2020-09-08T16:06:27.264093+0000 mon.a (mon.0) 2109 : audit [DBG] from='client.? v1:172.21.15.104:0/3644038996' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:06:28.481 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:28.598 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:28.710 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:28.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:06:28 smithi166 bash[18960]: debug 2020-09-08T16:06:28.373+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:28.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:06:28 smithi166 bash[18960]: debug 2020-09-08T16:06:28.593+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:28.878 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 82 16 124701 124685 95.0208 42.8125 0.00704449 0.0105198 2020-09-08T16:06:29.226 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:29.328 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:29.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:29 smithi055 bash[9296]: cluster 2020-09-08T16:06:28.057000+0000 mgr.x (mgr.24118) 1174 : cluster [DBG] pgmap v1746: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s wr, 792 op/s 2020-09-08T16:06:29.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:29 smithi055 bash[9296]: cluster 2020-09-08T16:06:28.395863+0000 osd.5 (osd.5) 107 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:29.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:29 smithi055 bash[9296]: cluster 2020-09-08T16:06:28.395944+0000 osd.5 (osd.5) 108 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:29.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:06:29 smithi104 bash[19922]: debug 2020-09-08T16:06:29.227+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:29.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:29 smithi104 bash[9203]: cluster 2020-09-08T16:06:28.057000+0000 mgr.x (mgr.24118) 1174 : cluster [DBG] pgmap v1746: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s wr, 792 op/s 2020-09-08T16:06:29.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:29 smithi104 bash[9203]: cluster 2020-09-08T16:06:28.395863+0000 osd.5 (osd.5) 107 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:29.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:29 smithi104 bash[9203]: cluster 2020-09-08T16:06:28.395944+0000 osd.5 (osd.5) 108 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:29.430 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:29.535 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:06:29.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:29 smithi166 bash[9446]: cluster 2020-09-08T16:06:28.057000+0000 mgr.x (mgr.24118) 1174 : cluster [DBG] pgmap v1746: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s wr, 792 op/s 2020-09-08T16:06:29.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:29 smithi166 bash[9446]: cluster 2020-09-08T16:06:28.395863+0000 osd.5 (osd.5) 107 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:29.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:29 smithi166 bash[9446]: cluster 2020-09-08T16:06:28.395944+0000 osd.5 (osd.5) 108 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:29.878 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 83 16 125425 125409 94.4211 45.25 0.0103465 0.0105844 2020-09-08T16:06:29.906 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:06:29 smithi055 bash[14301]: debug 2020-09-08T16:06:29.431+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:30.070 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:06:30.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:06:30 smithi104 bash[22305]: debug 2020-09-08T16:06:30.051+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:30.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:30 smithi104 bash[9203]: cluster 2020-09-08T16:06:29.125084+0000 mon.a (mon.0) 2110 : cluster [DBG] osdmap e657: 12 total, 12 up, 12 in 2020-09-08T16:06:30.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:30 smithi104 bash[9203]: cluster 2020-09-08T16:06:29.524370+0000 osd.10 (osd.10) 86 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:30.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:30 smithi104 bash[9203]: cluster 2020-09-08T16:06:29.524439+0000 osd.10 (osd.10) 87 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:30.179 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:30.294 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:30.396 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:30.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:06:30 smithi055 bash[19003]: debug 2020-09-08T16:06:30.287+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:30.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:30 smithi055 bash[9296]: cluster 2020-09-08T16:06:29.125084+0000 mon.a (mon.0) 2110 : cluster [DBG] osdmap e657: 12 total, 12 up, 12 in 2020-09-08T16:06:30.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:30 smithi055 bash[9296]: cluster 2020-09-08T16:06:29.524370+0000 osd.10 (osd.10) 86 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:30.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:30 smithi055 bash[9296]: cluster 2020-09-08T16:06:29.524439+0000 osd.10 (osd.10) 87 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:30.524 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:30.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:30 smithi166 bash[9446]: cluster 2020-09-08T16:06:29.125084+0000 mon.a (mon.0) 2110 : cluster [DBG] osdmap e657: 12 total, 12 up, 12 in 2020-09-08T16:06:30.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:30 smithi166 bash[9446]: cluster 2020-09-08T16:06:29.524370+0000 osd.10 (osd.10) 86 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:30.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:30 smithi166 bash[9446]: cluster 2020-09-08T16:06:29.524439+0000 osd.10 (osd.10) 87 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:30.630 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:30.871 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:30.878 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 84 16 126182 126166 93.8603 47.3125 0.00606236 0.0106406 2020-09-08T16:06:30.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:06:30 smithi055 bash[16646]: debug 2020-09-08T16:06:30.523+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:30.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:06:30 smithi104 bash[17588]: debug 2020-09-08T16:06:30.875+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:30.975 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:31.105 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:31.209 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:31.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:06:31 smithi055 bash[19003]: debug 2020-09-08T16:06:31.107+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:31.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:31 smithi055 bash[9296]: cluster 2020-09-08T16:06:30.057613+0000 mgr.x (mgr.24118) 1175 : cluster [DBG] pgmap v1748: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s wr, 834 op/s 2020-09-08T16:06:31.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:31 smithi055 bash[9296]: cluster 2020-09-08T16:06:30.145399+0000 mon.a (mon.0) 2111 : cluster [DBG] osdmap e658: 12 total, 12 up, 12 in 2020-09-08T16:06:31.510 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:31 smithi104 bash[9203]: cluster 2020-09-08T16:06:30.057613+0000 mgr.x (mgr.24118) 1175 : cluster [DBG] pgmap v1748: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s wr, 834 op/s 2020-09-08T16:06:31.511 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:31 smithi104 bash[9203]: cluster 2020-09-08T16:06:30.145399+0000 mon.a (mon.0) 2111 : cluster [DBG] osdmap e658: 12 total, 12 up, 12 in 2020-09-08T16:06:31.546 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:31.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:31 smithi166 bash[9446]: cluster 2020-09-08T16:06:30.057613+0000 mgr.x (mgr.24118) 1175 : cluster [DBG] pgmap v1748: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s wr, 834 op/s 2020-09-08T16:06:31.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:31 smithi166 bash[9446]: cluster 2020-09-08T16:06:30.145399+0000 mon.a (mon.0) 2111 : cluster [DBG] osdmap e658: 12 total, 12 up, 12 in 2020-09-08T16:06:31.651 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:31.764 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:31.868 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:06:31.877 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:06:31 smithi104 bash[19922]: debug 2020-09-08T16:06:31.547+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:31.878 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 85 16 126781 126765 93.1965 37.4375 0.0101025 0.0107189 2020-09-08T16:06:31.921 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:06:32.080 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:06:32.115 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:06:32.132 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:06:32 smithi104 bash[22305]: debug 2020-09-08T16:06:32.115+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:32.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:06:31 smithi055 bash[19003]: debug 2020-09-08T16:06:31.763+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:32.221 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:32.878 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 86 16 127293 127277 92.4849 32 0.00702368 0.0108057 2020-09-08T16:06:33.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:33 smithi055 bash[9296]: cluster 2020-09-08T16:06:32.058496+0000 mgr.x (mgr.24118) 1176 : cluster [DBG] pgmap v1750: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s wr, 527 op/s 2020-09-08T16:06:33.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:33 smithi104 bash[9203]: cluster 2020-09-08T16:06:32.058496+0000 mgr.x (mgr.24118) 1176 : cluster [DBG] pgmap v1750: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s wr, 527 op/s 2020-09-08T16:06:33.581 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:33.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:33 smithi166 bash[9446]: cluster 2020-09-08T16:06:32.058496+0000 mgr.x (mgr.24118) 1176 : cluster [DBG] pgmap v1750: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s wr, 527 op/s 2020-09-08T16:06:33.686 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:33.878 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 87 16 127932 127916 91.8809 39.9375 0.0071186 0.0108713 2020-09-08T16:06:33.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:06:33 smithi104 bash[17588]: debug 2020-09-08T16:06:33.579+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:34.162 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T16:06:34.383 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:34.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:06:34 smithi104 bash[15273]: debug 2020-09-08T16:06:34.387+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:34.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:34 smithi104 bash[9203]: cluster 2020-09-08T16:06:32.774782+0000 osd.3 (osd.3) 113 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:34.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:34 smithi104 bash[9203]: cluster 2020-09-08T16:06:32.774899+0000 osd.3 (osd.3) 114 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:34.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:34 smithi104 bash[9203]: cluster 2020-09-08T16:06:33.159832+0000 mon.a (mon.0) 2112 : cluster [DBG] osdmap e659: 12 total, 12 up, 12 in 2020-09-08T16:06:34.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:34 smithi104 bash[9203]: audit 2020-09-08T16:06:33.428891+0000 mon.a (mon.0) 2113 : audit [INF] from='client.? v1:172.21.15.104:0/2371865811' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:06:34.488 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:34.500 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:34 smithi055 bash[9296]: cluster 2020-09-08T16:06:32.774782+0000 osd.3 (osd.3) 113 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:34.501 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:34 smithi055 bash[9296]: cluster 2020-09-08T16:06:32.774899+0000 osd.3 (osd.3) 114 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:34.501 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:34 smithi055 bash[9296]: cluster 2020-09-08T16:06:33.159832+0000 mon.a (mon.0) 2112 : cluster [DBG] osdmap e659: 12 total, 12 up, 12 in 2020-09-08T16:06:34.501 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:34 smithi055 bash[9296]: audit 2020-09-08T16:06:33.428891+0000 mon.a (mon.0) 2113 : audit [INF] from='client.? v1:172.21.15.104:0/2371865811' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:06:34.605 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:34.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:34 smithi166 bash[9446]: cluster 2020-09-08T16:06:32.774782+0000 osd.3 (osd.3) 113 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:34.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:34 smithi166 bash[9446]: cluster 2020-09-08T16:06:32.774899+0000 osd.3 (osd.3) 114 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:34.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:34 smithi166 bash[9446]: cluster 2020-09-08T16:06:33.159832+0000 mon.a (mon.0) 2112 : cluster [DBG] osdmap e659: 12 total, 12 up, 12 in 2020-09-08T16:06:34.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:34 smithi166 bash[9446]: audit 2020-09-08T16:06:33.428891+0000 mon.a (mon.0) 2113 : audit [INF] from='client.? v1:172.21.15.104:0/2371865811' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:06:34.712 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:34.823 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:06:34 smithi055 bash[16646]: debug 2020-09-08T16:06:34.607+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:34.828 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:34.878 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 88 15 128847 128832 91.4873 57.25 0.0116711 0.0109285 2020-09-08T16:06:34.931 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:35.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:06:34 smithi055 bash[16646]: debug 2020-09-08T16:06:34.823+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:35.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:35 smithi104 bash[9203]: cluster 2020-09-08T16:06:34.059160+0000 mgr.x (mgr.24118) 1177 : cluster [DBG] pgmap v1752: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s wr, 723 op/s 2020-09-08T16:06:35.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:35 smithi104 bash[9203]: cluster 2020-09-08T16:06:34.155764+0000 mon.a (mon.0) 2114 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:06:35.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:35 smithi104 bash[9203]: audit 2020-09-08T16:06:34.165365+0000 mon.a (mon.0) 2115 : audit [INF] from='client.? v1:172.21.15.104:0/2371865811' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:06:35.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:35 smithi104 bash[9203]: cluster 2020-09-08T16:06:34.165417+0000 mon.a (mon.0) 2116 : cluster [DBG] osdmap e660: 12 total, 12 up, 12 in 2020-09-08T16:06:35.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:35 smithi166 bash[9446]: cluster 2020-09-08T16:06:34.059160+0000 mgr.x (mgr.24118) 1177 : cluster [DBG] pgmap v1752: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s wr, 723 op/s 2020-09-08T16:06:35.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:35 smithi166 bash[9446]: cluster 2020-09-08T16:06:34.155764+0000 mon.a (mon.0) 2114 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:06:35.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:35 smithi166 bash[9446]: audit 2020-09-08T16:06:34.165365+0000 mon.a (mon.0) 2115 : audit [INF] from='client.? v1:172.21.15.104:0/2371865811' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:06:35.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:35 smithi166 bash[9446]: cluster 2020-09-08T16:06:34.165417+0000 mon.a (mon.0) 2116 : cluster [DBG] osdmap e660: 12 total, 12 up, 12 in 2020-09-08T16:06:35.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:35 smithi055 bash[9296]: cluster 2020-09-08T16:06:34.059160+0000 mgr.x (mgr.24118) 1177 : cluster [DBG] pgmap v1752: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s wr, 723 op/s 2020-09-08T16:06:35.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:35 smithi055 bash[9296]: cluster 2020-09-08T16:06:34.155764+0000 mon.a (mon.0) 2114 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:06:35.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:35 smithi055 bash[9296]: audit 2020-09-08T16:06:34.165365+0000 mon.a (mon.0) 2115 : audit [INF] from='client.? v1:172.21.15.104:0/2371865811' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:06:35.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:35 smithi055 bash[9296]: cluster 2020-09-08T16:06:34.165417+0000 mon.a (mon.0) 2116 : cluster [DBG] osdmap e660: 12 total, 12 up, 12 in 2020-09-08T16:06:35.731 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:35.836 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:35.878 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 89 16 129839 129823 91.1553 61.9375 0.0107801 0.0109681 2020-09-08T16:06:35.981 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:36.091 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:36.098 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:06:35 smithi055 bash[11956]: debug 2020-09-08T16:06:35.983+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:36.194 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:06:35 smithi104 bash[17588]: debug 2020-09-08T16:06:35.731+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:36.202 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:36.308 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:36.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:06:36 smithi055 bash[16646]: debug 2020-09-08T16:06:36.199+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:36.415 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:36.522 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:36.806 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:36.879 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 90 12 130918 130906 90.8945 67.6875 0.0144862 0.0109991 2020-09-08T16:06:36.906 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:06:36 smithi055 bash[11956]: debug 2020-09-08T16:06:36.415+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:36.911 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:37.099 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:06:36 smithi104 bash[19922]: debug 2020-09-08T16:06:36.803+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:37.100 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:37.205 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:06:37.360 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:06:37.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:06:37 smithi104 bash[17588]: debug 2020-09-08T16:06:37.103+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:37.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:37 smithi104 bash[9203]: cluster 2020-09-08T16:06:36.059710+0000 mgr.x (mgr.24118) 1178 : cluster [DBG] pgmap v1754: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s wr, 542 op/s 2020-09-08T16:06:37.468 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:06:37.485 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:37 smithi166 bash[9446]: cluster 2020-09-08T16:06:36.059710+0000 mgr.x (mgr.24118) 1178 : cluster [DBG] pgmap v1754: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s wr, 542 op/s 2020-09-08T16:06:37.487 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:06:37 smithi166 bash[14254]: debug 2020-09-08T16:06:37.356+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:37.620 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:06:37.660 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:37 smithi055 bash[9296]: cluster 2020-09-08T16:06:36.059710+0000 mgr.x (mgr.24118) 1178 : cluster [DBG] pgmap v1754: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s wr, 542 op/s 2020-09-08T16:06:37.722 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:06:37.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:06:37 smithi166 bash[11928]: debug 2020-09-08T16:06:37.616+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:37.879 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Total time run: 90.0139 2020-09-08T16:06:37.879 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Total writes made: 130918 2020-09-08T16:06:37.880 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Write size: 65536 2020-09-08T16:06:37.880 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Object size: 65536 2020-09-08T16:06:37.880 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Bandwidth (MB/sec): 90.9013 2020-09-08T16:06:37.880 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev Bandwidth: 40.4963 2020-09-08T16:06:37.880 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max bandwidth (MB/sec): 210.75 2020-09-08T16:06:37.881 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min bandwidth (MB/sec): 32 2020-09-08T16:06:37.881 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Average IOPS: 1454 2020-09-08T16:06:37.881 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev IOPS: 647 2020-09-08T16:06:37.882 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max IOPS: 3372 2020-09-08T16:06:37.882 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min IOPS: 512 2020-09-08T16:06:37.882 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Average Latency(s): 0.0109999 2020-09-08T16:06:37.883 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev Latency(s): 0.0206043 2020-09-08T16:06:37.883 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max latency(s): 0.615684 2020-09-08T16:06:37.883 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min latency(s): 0.00181942 2020-09-08T16:06:37.901 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Cleaning up (deleting benchmark objects) 2020-09-08T16:06:37.999 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:06:38 smithi104 bash[22305]: debug 2020-09-08T16:06:38.003+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:38.006 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:06:38.113 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:38.243 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:38.292 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T16:06:38.293 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T16:06:38.293 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 0 2020-09-08T16:06:38.293 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T16:06:38.351 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:38.482 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:38.578 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T16:06:38.587 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:38.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:06:38 smithi166 bash[18960]: debug 2020-09-08T16:06:38.236+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:38.656 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:06:38 smithi055 bash[11956]: debug 2020-09-08T16:06:38.479+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:38.748 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:06:39.028 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:39.131 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:39.299 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:39.403 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:39.414 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:39 smithi104 bash[9203]: cluster 2020-09-08T16:06:38.060349+0000 mgr.x (mgr.24118) 1179 : cluster [DBG] pgmap v1755: 17 pgs: 17 active+clean; 7.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s wr, 752 op/s 2020-09-08T16:06:39.414 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:06:39 smithi104 bash[15273]: debug 2020-09-08T16:06:39.031+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:39.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:39 smithi166 bash[9446]: cluster 2020-09-08T16:06:38.060349+0000 mgr.x (mgr.24118) 1179 : cluster [DBG] pgmap v1755: 17 pgs: 17 active+clean; 7.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s wr, 752 op/s 2020-09-08T16:06:39.617 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:06:39 smithi166 bash[18960]: debug 2020-09-08T16:06:39.296+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:39.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:39 smithi055 bash[9296]: cluster 2020-09-08T16:06:38.060349+0000 mgr.x (mgr.24118) 1179 : cluster [DBG] pgmap v1755: 17 pgs: 17 active+clean; 7.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s wr, 752 op/s 2020-09-08T16:06:40.121 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:40.225 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:40.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:06:40 smithi104 bash[17588]: debug 2020-09-08T16:06:40.123+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:40.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:40 smithi104 bash[9203]: cluster 2020-09-08T16:06:38.433974+0000 osd.8 (osd.8) 91 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:40.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:40 smithi104 bash[9203]: cluster 2020-09-08T16:06:38.434039+0000 osd.8 (osd.8) 92 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:40.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:40 smithi104 bash[9203]: cluster 2020-09-08T16:06:39.176529+0000 mon.a (mon.0) 2117 : cluster [DBG] osdmap e661: 12 total, 12 up, 12 in 2020-09-08T16:06:40.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:40 smithi166 bash[9446]: cluster 2020-09-08T16:06:38.433974+0000 osd.8 (osd.8) 91 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:40.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:40 smithi166 bash[9446]: cluster 2020-09-08T16:06:38.434039+0000 osd.8 (osd.8) 92 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:40.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:40 smithi166 bash[9446]: cluster 2020-09-08T16:06:39.176529+0000 mon.a (mon.0) 2117 : cluster [DBG] osdmap e661: 12 total, 12 up, 12 in 2020-09-08T16:06:40.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:40 smithi055 bash[9296]: cluster 2020-09-08T16:06:38.433974+0000 osd.8 (osd.8) 91 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:40.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:40 smithi055 bash[9296]: cluster 2020-09-08T16:06:38.434039+0000 osd.8 (osd.8) 92 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:40.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:40 smithi055 bash[9296]: cluster 2020-09-08T16:06:39.176529+0000 mon.a (mon.0) 2117 : cluster [DBG] osdmap e661: 12 total, 12 up, 12 in 2020-09-08T16:06:40.787 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:40.891 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:40.940 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2020-09-08T16:06:41.018 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:41.091 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:06:41.130 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:41.139 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:06:40 smithi104 bash[17588]: debug 2020-09-08T16:06:40.791+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:41.156 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:06:41 smithi055 bash[14301]: debug 2020-09-08T16:06:41.019+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:41.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:41 smithi104 bash[9203]: cluster 2020-09-08T16:06:40.060789+0000 mgr.x (mgr.24118) 1180 : cluster [DBG] pgmap v1757: 17 pgs: 17 active+clean; 8.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 148 B/s rd, 43 MiB/s wr, 709 op/s 2020-09-08T16:06:41.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:41 smithi166 bash[9446]: cluster 2020-09-08T16:06:40.060789+0000 mgr.x (mgr.24118) 1180 : cluster [DBG] pgmap v1757: 17 pgs: 17 active+clean; 8.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 148 B/s rd, 43 MiB/s wr, 709 op/s 2020-09-08T16:06:41.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:41 smithi055 bash[9296]: cluster 2020-09-08T16:06:40.060789+0000 mgr.x (mgr.24118) 1180 : cluster [DBG] pgmap v1757: 17 pgs: 17 active+clean; 8.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 148 B/s rd, 43 MiB/s wr, 709 op/s 2020-09-08T16:06:42.005 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:42.109 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:42.269 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:42.372 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:42.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:06:42 smithi055 bash[14301]: debug 2020-09-08T16:06:42.271+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:42.503 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:06:42 smithi104 bash[15273]: debug 2020-09-08T16:06:41.999+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:43.014 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:06:43.029 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:43.045 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:06:43.072 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:06:43.103 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:06:43.135 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:06:43.137 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:06:43.163 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:06:43.197 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:06:43.228 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:06:43.256 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:06:43.290 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:06:43.324 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:06:43.360 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:06:43.366 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:06:43.366 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:06:43.366 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:06:43.367 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:06:43.367 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:06:43.367 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:06:43.367 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:06:43.367 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:06:43.367 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:06:43.368 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:06:43.368 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:06:43.368 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:06:43.390 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:06:43 smithi104 bash[19922]: debug 2020-09-08T16:06:43.027+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:43.390 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:43 smithi104 bash[9203]: cluster 2020-09-08T16:06:42.061256+0000 mgr.x (mgr.24118) 1181 : cluster [DBG] pgmap v1758: 17 pgs: 17 active+clean; 7.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 29 MiB/s wr, 585 op/s 2020-09-08T16:06:43.390 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:43 smithi104 bash[9203]: audit 2020-09-08T16:06:42.978318+0000 mon.c (mon.2) 86 : audit [DBG] from='client.? v1:172.21.15.104:0/2506403699' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:06:43.391 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:06:43.499 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:06:43.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:43 smithi166 bash[9446]: cluster 2020-09-08T16:06:42.061256+0000 mgr.x (mgr.24118) 1181 : cluster [DBG] pgmap v1758: 17 pgs: 17 active+clean; 7.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 29 MiB/s wr, 585 op/s 2020-09-08T16:06:43.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:43 smithi166 bash[9446]: audit 2020-09-08T16:06:42.978318+0000 mon.c (mon.2) 86 : audit [DBG] from='client.? v1:172.21.15.104:0/2506403699' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:06:43.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:43 smithi055 bash[9296]: cluster 2020-09-08T16:06:42.061256+0000 mgr.x (mgr.24118) 1181 : cluster [DBG] pgmap v1758: 17 pgs: 17 active+clean; 7.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 29 MiB/s wr, 585 op/s 2020-09-08T16:06:43.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:43 smithi055 bash[9296]: audit 2020-09-08T16:06:42.978318+0000 mon.c (mon.2) 86 : audit [DBG] from='client.? v1:172.21.15.104:0/2506403699' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:06:43.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:06:43 smithi104 bash[22305]: debug 2020-09-08T16:06:43.391+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:43.689 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:06:43.790 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:44.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:06:43 smithi104 bash[22305]: debug 2020-09-08T16:06:43.687+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:44.722 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:44.826 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:45.117 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:06:44 smithi104 bash[17588]: debug 2020-09-08T16:06:44.723+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:45.119 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:45.220 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:06:45.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:45 smithi104 bash[9203]: cluster 2020-09-08T16:06:44.061769+0000 mgr.x (mgr.24118) 1182 : cluster [DBG] pgmap v1759: 17 pgs: 17 active+clean; 7.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s rd, 24 MiB/s wr, 870 op/s 2020-09-08T16:06:45.427 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:06:45 smithi104 bash[15273]: debug 2020-09-08T16:06:45.115+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:45.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:45 smithi166 bash[9446]: cluster 2020-09-08T16:06:44.061769+0000 mgr.x (mgr.24118) 1182 : cluster [DBG] pgmap v1759: 17 pgs: 17 active+clean; 7.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s rd, 24 MiB/s wr, 870 op/s 2020-09-08T16:06:45.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:45 smithi055 bash[9296]: cluster 2020-09-08T16:06:44.061769+0000 mgr.x (mgr.24118) 1182 : cluster [DBG] pgmap v1759: 17 pgs: 17 active+clean; 7.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s rd, 24 MiB/s wr, 870 op/s 2020-09-08T16:06:45.912 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:06:46.021 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:46.144 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:46.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:06:45 smithi104 bash[22305]: debug 2020-09-08T16:06:45.915+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:46.249 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:46.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:06:46 smithi055 bash[11956]: debug 2020-09-08T16:06:46.143+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:46.412 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:46.527 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:06:46.537 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:06:46 smithi166 bash[18960]: debug 2020-09-08T16:06:46.408+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:46.650 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:06:46.755 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:46.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:06:46 smithi166 bash[16593]: debug 2020-09-08T16:06:46.644+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:46.889 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:46.999 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:47.125 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:47.202 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:06:46 smithi166 bash[18960]: debug 2020-09-08T16:06:46.888+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:47.203 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:06:47 smithi166 bash[18960]: debug 2020-09-08T16:06:47.120+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:47.228 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:47.317 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:06:47.335 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:47.439 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:47.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:47 smithi055 bash[9296]: cluster 2020-09-08T16:06:46.062249+0000 mgr.x (mgr.24118) 1183 : cluster [DBG] pgmap v1760: 17 pgs: 17 active+clean; 7.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s rd, 23 MiB/s wr, 861 op/s 2020-09-08T16:06:47.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:47 smithi055 bash[9296]: cluster 2020-09-08T16:06:46.985390+0000 osd.1 (osd.1) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:47.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:47 smithi055 bash[9296]: cluster 2020-09-08T16:06:46.985465+0000 osd.1 (osd.1) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:47.452 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:06:47 smithi055 bash[14301]: debug 2020-09-08T16:06:47.335+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:47.467 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:06:47.552 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:47.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:47 smithi166 bash[9446]: cluster 2020-09-08T16:06:46.062249+0000 mgr.x (mgr.24118) 1183 : cluster [DBG] pgmap v1760: 17 pgs: 17 active+clean; 7.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s rd, 23 MiB/s wr, 861 op/s 2020-09-08T16:06:47.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:47 smithi166 bash[9446]: cluster 2020-09-08T16:06:46.985390+0000 osd.1 (osd.1) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:47.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:47 smithi166 bash[9446]: cluster 2020-09-08T16:06:46.985465+0000 osd.1 (osd.1) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:47.664 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:47.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:47 smithi104 bash[9203]: cluster 2020-09-08T16:06:46.062249+0000 mgr.x (mgr.24118) 1183 : cluster [DBG] pgmap v1760: 17 pgs: 17 active+clean; 7.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s rd, 23 MiB/s wr, 861 op/s 2020-09-08T16:06:47.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:47 smithi104 bash[9203]: cluster 2020-09-08T16:06:46.985390+0000 osd.1 (osd.1) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:47.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:47 smithi104 bash[9203]: cluster 2020-09-08T16:06:46.985465+0000 osd.1 (osd.1) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:47.778 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:06:47 smithi055 bash[14301]: debug 2020-09-08T16:06:47.551+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:47.778 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:06:47 smithi055 bash[14301]: debug 2020-09-08T16:06:47.779+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:47.779 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:47.882 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:48.267 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:48.372 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:48.486 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:48 smithi104 bash[9203]: cluster 2020-09-08T16:06:47.204596+0000 mon.a (mon.0) 2118 : cluster [DBG] osdmap e662: 12 total, 12 up, 12 in 2020-09-08T16:06:48.487 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:06:48 smithi104 bash[19922]: debug 2020-09-08T16:06:48.271+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:48.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:48 smithi166 bash[9446]: cluster 2020-09-08T16:06:47.204596+0000 mon.a (mon.0) 2118 : cluster [DBG] osdmap e662: 12 total, 12 up, 12 in 2020-09-08T16:06:48.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:48 smithi055 bash[9296]: cluster 2020-09-08T16:06:47.204596+0000 mon.a (mon.0) 2118 : cluster [DBG] osdmap e662: 12 total, 12 up, 12 in 2020-09-08T16:06:49.085 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:06:49.187 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:49.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:49 smithi104 bash[9203]: cluster 2020-09-08T16:06:48.062870+0000 mgr.x (mgr.24118) 1184 : cluster [DBG] pgmap v1762: 17 pgs: 17 active+clean; 7.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s wr, 936 op/s 2020-09-08T16:06:49.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:06:49 smithi104 bash[17588]: debug 2020-09-08T16:06:49.079+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:49.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:49 smithi166 bash[9446]: cluster 2020-09-08T16:06:48.062870+0000 mgr.x (mgr.24118) 1184 : cluster [DBG] pgmap v1762: 17 pgs: 17 active+clean; 7.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s wr, 936 op/s 2020-09-08T16:06:49.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:49 smithi055 bash[9296]: cluster 2020-09-08T16:06:48.062870+0000 mgr.x (mgr.24118) 1184 : cluster [DBG] pgmap v1762: 17 pgs: 17 active+clean; 7.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s wr, 936 op/s 2020-09-08T16:06:49.992 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T16:06:49.992 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:06:50.223 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T16:06:50.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:50 smithi166 bash[9446]: audit 2020-09-08T16:06:49.538248+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? v1:172.21.15.104:0/3360541938' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:06:50.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:50 smithi055 bash[9296]: audit 2020-09-08T16:06:49.538248+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? v1:172.21.15.104:0/3360541938' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:06:50.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:50 smithi104 bash[9203]: audit 2020-09-08T16:06:49.538248+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? v1:172.21.15.104:0/3360541938' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:06:51.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:51 smithi166 bash[9446]: cluster 2020-09-08T16:06:50.063355+0000 mgr.x (mgr.24118) 1185 : cluster [DBG] pgmap v1763: 17 pgs: 17 active+clean; 7.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s wr, 990 op/s 2020-09-08T16:06:51.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:51 smithi166 bash[9446]: cluster 2020-09-08T16:06:50.133225+0000 osd.6 (osd.6) 111 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:51.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:51 smithi166 bash[9446]: cluster 2020-09-08T16:06:50.133349+0000 osd.6 (osd.6) 112 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:51.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:51 smithi166 bash[9446]: cluster 2020-09-08T16:06:50.210562+0000 mon.a (mon.0) 2120 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:06:51.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:51 smithi166 bash[9446]: audit 2020-09-08T16:06:50.222237+0000 mon.a (mon.0) 2121 : audit [INF] from='client.? v1:172.21.15.104:0/3360541938' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:06:51.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:51 smithi166 bash[9446]: cluster 2020-09-08T16:06:50.222285+0000 mon.a (mon.0) 2122 : cluster [DBG] osdmap e663: 12 total, 12 up, 12 in 2020-09-08T16:06:51.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:51 smithi166 bash[9446]: cluster 2020-09-08T16:06:50.670943+0000 osd.10 (osd.10) 88 : cluster [DBG] 8.2 scrub starts 2020-09-08T16:06:51.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:51 smithi166 bash[9446]: cluster 2020-09-08T16:06:50.771362+0000 osd.5 (osd.5) 109 : cluster [DBG] 8.8 scrub starts 2020-09-08T16:06:51.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:51 smithi055 bash[9296]: cluster 2020-09-08T16:06:50.063355+0000 mgr.x (mgr.24118) 1185 : cluster [DBG] pgmap v1763: 17 pgs: 17 active+clean; 7.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s wr, 990 op/s 2020-09-08T16:06:51.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:51 smithi055 bash[9296]: cluster 2020-09-08T16:06:50.133225+0000 osd.6 (osd.6) 111 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:51.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:51 smithi055 bash[9296]: cluster 2020-09-08T16:06:50.133349+0000 osd.6 (osd.6) 112 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:51.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:51 smithi055 bash[9296]: cluster 2020-09-08T16:06:50.210562+0000 mon.a (mon.0) 2120 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:06:51.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:51 smithi055 bash[9296]: audit 2020-09-08T16:06:50.222237+0000 mon.a (mon.0) 2121 : audit [INF] from='client.? v1:172.21.15.104:0/3360541938' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:06:51.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:51 smithi055 bash[9296]: cluster 2020-09-08T16:06:50.222285+0000 mon.a (mon.0) 2122 : cluster [DBG] osdmap e663: 12 total, 12 up, 12 in 2020-09-08T16:06:51.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:51 smithi055 bash[9296]: cluster 2020-09-08T16:06:50.670943+0000 osd.10 (osd.10) 88 : cluster [DBG] 8.2 scrub starts 2020-09-08T16:06:51.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:51 smithi055 bash[9296]: cluster 2020-09-08T16:06:50.771362+0000 osd.5 (osd.5) 109 : cluster [DBG] 8.8 scrub starts 2020-09-08T16:06:51.722 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:51 smithi104 bash[9203]: cluster 2020-09-08T16:06:50.063355+0000 mgr.x (mgr.24118) 1185 : cluster [DBG] pgmap v1763: 17 pgs: 17 active+clean; 7.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s wr, 990 op/s 2020-09-08T16:06:51.722 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:51 smithi104 bash[9203]: cluster 2020-09-08T16:06:50.133225+0000 osd.6 (osd.6) 111 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:51.722 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:51 smithi104 bash[9203]: cluster 2020-09-08T16:06:50.133349+0000 osd.6 (osd.6) 112 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:51.722 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:51 smithi104 bash[9203]: cluster 2020-09-08T16:06:50.210562+0000 mon.a (mon.0) 2120 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:06:51.723 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:51 smithi104 bash[9203]: audit 2020-09-08T16:06:50.222237+0000 mon.a (mon.0) 2121 : audit [INF] from='client.? v1:172.21.15.104:0/3360541938' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:06:51.723 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:51 smithi104 bash[9203]: cluster 2020-09-08T16:06:50.222285+0000 mon.a (mon.0) 2122 : cluster [DBG] osdmap e663: 12 total, 12 up, 12 in 2020-09-08T16:06:51.723 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:51 smithi104 bash[9203]: cluster 2020-09-08T16:06:50.670943+0000 osd.10 (osd.10) 88 : cluster [DBG] 8.2 scrub starts 2020-09-08T16:06:51.723 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:51 smithi104 bash[9203]: cluster 2020-09-08T16:06:50.771362+0000 osd.5 (osd.5) 109 : cluster [DBG] 8.8 scrub starts 2020-09-08T16:06:51.856 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:51.960 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:52.100 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:52.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:06:51 smithi166 bash[18960]: debug 2020-09-08T16:06:51.852+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:52.205 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:52.325 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2020-09-08T16:06:52.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:06:52 smithi055 bash[16646]: debug 2020-09-08T16:06:52.103+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:52.484 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:06:52.523 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:06:52 smithi104 bash[19922]: debug 2020-09-08T16:06:52.527+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:52.529 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:52.633 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:52.754 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:52.863 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:52.906 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:06:52 smithi055 bash[19003]: debug 2020-09-08T16:06:52.755+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:52.981 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:53.085 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:06:53.236 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:06:53.340 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:53.354 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:06:52 smithi055 bash[11956]: debug 2020-09-08T16:06:52.983+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:53.354 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:53 smithi055 bash[9296]: cluster 2020-09-08T16:06:52.063926+0000 mgr.x (mgr.24118) 1186 : cluster [DBG] pgmap v1765: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 757 op/s 2020-09-08T16:06:53.354 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:53 smithi055 bash[9296]: cluster 2020-09-08T16:06:53.149877+0000 osd.1 (osd.1) 132 : cluster [DBG] 8.7 scrub starts 2020-09-08T16:06:53.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:06:53 smithi166 bash[14254]: debug 2020-09-08T16:06:53.236+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:53.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:53 smithi166 bash[9446]: cluster 2020-09-08T16:06:52.063926+0000 mgr.x (mgr.24118) 1186 : cluster [DBG] pgmap v1765: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 757 op/s 2020-09-08T16:06:53.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:53 smithi166 bash[9446]: cluster 2020-09-08T16:06:53.149877+0000 osd.1 (osd.1) 132 : cluster [DBG] 8.7 scrub starts 2020-09-08T16:06:53.463 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:06:53.568 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:53.577 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:53 smithi104 bash[9203]: cluster 2020-09-08T16:06:52.063926+0000 mgr.x (mgr.24118) 1186 : cluster [DBG] pgmap v1765: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 757 op/s 2020-09-08T16:06:53.577 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:53 smithi104 bash[9203]: cluster 2020-09-08T16:06:53.149877+0000 osd.1 (osd.1) 132 : cluster [DBG] 8.7 scrub starts 2020-09-08T16:06:53.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:06:53 smithi055 bash[16646]: debug 2020-09-08T16:06:53.455+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:53.932 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:54.036 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:54.171 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:54.223 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:54 smithi104 bash[9203]: cluster 2020-09-08T16:06:52.945129+0000 osd.5 (osd.5) 110 : cluster [DBG] 8.8 scrub ok 2020-09-08T16:06:54.223 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:54 smithi104 bash[9203]: cluster 2020-09-08T16:06:53.351261+0000 osd.10 (osd.10) 89 : cluster [DBG] 8.2 scrub ok 2020-09-08T16:06:54.223 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:54 smithi104 bash[9203]: cluster 2020-09-08T16:06:53.476861+0000 osd.11 (osd.11) 112 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:54.224 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:54 smithi104 bash[9203]: cluster 2020-09-08T16:06:53.476957+0000 osd.11 (osd.11) 113 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:54.224 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:54 smithi104 bash[9203]: cluster 2020-09-08T16:06:53.829188+0000 osd.0 (osd.0) 119 : cluster [DBG] 8.e scrub starts 2020-09-08T16:06:54.224 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:06:53 smithi104 bash[15273]: debug 2020-09-08T16:06:53.935+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:54.282 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:54.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:06:54 smithi166 bash[18960]: debug 2020-09-08T16:06:54.164+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:54.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:54 smithi166 bash[9446]: cluster 2020-09-08T16:06:52.945129+0000 osd.5 (osd.5) 110 : cluster [DBG] 8.8 scrub ok 2020-09-08T16:06:54.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:54 smithi166 bash[9446]: cluster 2020-09-08T16:06:53.351261+0000 osd.10 (osd.10) 89 : cluster [DBG] 8.2 scrub ok 2020-09-08T16:06:54.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:54 smithi166 bash[9446]: cluster 2020-09-08T16:06:53.476861+0000 osd.11 (osd.11) 112 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:54.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:54 smithi166 bash[9446]: cluster 2020-09-08T16:06:53.476957+0000 osd.11 (osd.11) 113 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:54.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:54 smithi166 bash[9446]: cluster 2020-09-08T16:06:53.829188+0000 osd.0 (osd.0) 119 : cluster [DBG] 8.e scrub starts 2020-09-08T16:06:54.470 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:54.473 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:06:54 smithi104 bash[19922]: debug 2020-09-08T16:06:54.467+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:54.574 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T16:06:54.575 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:06:54.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:54 smithi055 bash[9296]: cluster 2020-09-08T16:06:52.945129+0000 osd.5 (osd.5) 110 : cluster [DBG] 8.8 scrub ok 2020-09-08T16:06:54.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:54 smithi055 bash[9296]: cluster 2020-09-08T16:06:53.351261+0000 osd.10 (osd.10) 89 : cluster [DBG] 8.2 scrub ok 2020-09-08T16:06:54.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:54 smithi055 bash[9296]: cluster 2020-09-08T16:06:53.476861+0000 osd.11 (osd.11) 112 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:06:54.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:54 smithi055 bash[9296]: cluster 2020-09-08T16:06:53.476957+0000 osd.11 (osd.11) 113 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:06:54.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:54 smithi055 bash[9296]: cluster 2020-09-08T16:06:53.829188+0000 osd.0 (osd.0) 119 : cluster [DBG] 8.e scrub starts 2020-09-08T16:06:54.703 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:06:54.735 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:06:54.810 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:54.874 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:06:54 smithi166 bash[14254]: debug 2020-09-08T16:06:54.700+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:55.300 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:06:55.330 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:06:55.361 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:06:55.390 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:06:55.421 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:06:55.456 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:06:55.493 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:06:55.521 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:06:55.552 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:06:55.592 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:06:55.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:55 smithi166 bash[9446]: cluster 2020-09-08T16:06:54.064515+0000 mgr.x (mgr.24118) 1187 : cluster [DBG] pgmap v1766: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.22k op/s 2020-09-08T16:06:55.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:55 smithi166 bash[9446]: cluster 2020-09-08T16:06:54.229642+0000 mon.a (mon.0) 2123 : cluster [DBG] osdmap e664: 12 total, 12 up, 12 in 2020-09-08T16:06:55.617 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:06:55.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:55 smithi055 bash[9296]: cluster 2020-09-08T16:06:54.064515+0000 mgr.x (mgr.24118) 1187 : cluster [DBG] pgmap v1766: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.22k op/s 2020-09-08T16:06:55.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:55 smithi055 bash[9296]: cluster 2020-09-08T16:06:54.229642+0000 mon.a (mon.0) 2123 : cluster [DBG] osdmap e664: 12 total, 12 up, 12 in 2020-09-08T16:06:55.658 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:06:55.659 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:06:55.659 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:06:55.659 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:06:55.659 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:06:55.660 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:06:55.660 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:06:55.660 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:06:55.660 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:06:55.660 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:06:55.661 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:06:55.661 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:06:55.661 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:06:55.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:55 smithi104 bash[9203]: cluster 2020-09-08T16:06:54.064515+0000 mgr.x (mgr.24118) 1187 : cluster [DBG] pgmap v1766: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.22k op/s 2020-09-08T16:06:55.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:55 smithi104 bash[9203]: cluster 2020-09-08T16:06:54.229642+0000 mon.a (mon.0) 2123 : cluster [DBG] osdmap e664: 12 total, 12 up, 12 in 2020-09-08T16:06:55.760 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:06:55.867 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:06:56.008 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:06:56.111 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:56.203 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:06:55 smithi104 bash[19922]: debug 2020-09-08T16:06:55.767+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:56.229 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:06:56 smithi166 bash[16593]: debug 2020-09-08T16:06:56.008+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:56.249 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:56.354 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:56.464 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:56 smithi055 bash[9296]: audit 2020-09-08T16:06:55.268876+0000 mon.b (mon.1) 116 : audit [DBG] from='client.? v1:172.21.15.104:0/1261395315' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:06:56.464 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:06:56 smithi055 bash[14301]: debug 2020-09-08T16:06:56.251+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:56.612 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:56.612 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:56 smithi166 bash[9446]: audit 2020-09-08T16:06:55.268876+0000 mon.b (mon.1) 116 : audit [DBG] from='client.? v1:172.21.15.104:0/1261395315' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:06:56.613 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:06:56 smithi166 bash[18960]: debug 2020-09-08T16:06:56.608+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:56.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:56 smithi104 bash[9203]: audit 2020-09-08T16:06:55.268876+0000 mon.b (mon.1) 116 : audit [DBG] from='client.? v1:172.21.15.104:0/1261395315' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:06:56.715 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:56.838 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:06:56.946 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:57.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:06:56 smithi055 bash[19003]: debug 2020-09-08T16:06:56.839+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:57.236 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:57 smithi104 bash[9203]: cluster 2020-09-08T16:06:55.832369+0000 osd.3 (osd.3) 115 : cluster [DBG] 8.0 scrub starts 2020-09-08T16:06:57.237 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:57 smithi104 bash[9203]: cluster 2020-09-08T16:06:56.064992+0000 mgr.x (mgr.24118) 1188 : cluster [DBG] pgmap v1768: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 767 op/s 2020-09-08T16:06:57.237 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:57 smithi104 bash[9203]: cluster 2020-09-08T16:06:56.546880+0000 osd.1 (osd.1) 133 : cluster [DBG] 8.7 scrub ok 2020-09-08T16:06:57.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:57 smithi166 bash[9446]: cluster 2020-09-08T16:06:55.832369+0000 osd.3 (osd.3) 115 : cluster [DBG] 8.0 scrub starts 2020-09-08T16:06:57.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:57 smithi166 bash[9446]: cluster 2020-09-08T16:06:56.064992+0000 mgr.x (mgr.24118) 1188 : cluster [DBG] pgmap v1768: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 767 op/s 2020-09-08T16:06:57.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:57 smithi166 bash[9446]: cluster 2020-09-08T16:06:56.546880+0000 osd.1 (osd.1) 133 : cluster [DBG] 8.7 scrub ok 2020-09-08T16:06:57.622 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:57.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:57 smithi055 bash[9296]: cluster 2020-09-08T16:06:55.832369+0000 osd.3 (osd.3) 115 : cluster [DBG] 8.0 scrub starts 2020-09-08T16:06:57.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:57 smithi055 bash[9296]: cluster 2020-09-08T16:06:56.064992+0000 mgr.x (mgr.24118) 1188 : cluster [DBG] pgmap v1768: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 767 op/s 2020-09-08T16:06:57.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:57 smithi055 bash[9296]: cluster 2020-09-08T16:06:56.546880+0000 osd.1 (osd.1) 133 : cluster [DBG] 8.7 scrub ok 2020-09-08T16:06:57.732 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:57.888 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:06:57.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:06:57 smithi104 bash[15273]: debug 2020-09-08T16:06:57.623+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:57.992 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:58.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:06:57 smithi166 bash[18960]: debug 2020-09-08T16:06:57.880+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:58.433 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:58.537 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:58.669 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:06:58 smithi104 bash[15273]: debug 2020-09-08T16:06:58.435+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:59.056 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:06:59.161 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:59.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:06:59 smithi055 bash[11956]: debug 2020-09-08T16:06:59.059+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:59.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:59 smithi055 bash[9296]: cluster 2020-09-08T16:06:57.704113+0000 osd.3 (osd.3) 116 : cluster [DBG] 8.0 scrub ok 2020-09-08T16:06:59.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:06:59 smithi055 bash[9296]: cluster 2020-09-08T16:06:58.065697+0000 mgr.x (mgr.24118) 1189 : cluster [DBG] pgmap v1769: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 6.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 980 op/s 2020-09-08T16:06:59.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:59 smithi104 bash[9203]: cluster 2020-09-08T16:06:57.704113+0000 osd.3 (osd.3) 116 : cluster [DBG] 8.0 scrub ok 2020-09-08T16:06:59.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:06:59 smithi104 bash[9203]: cluster 2020-09-08T16:06:58.065697+0000 mgr.x (mgr.24118) 1189 : cluster [DBG] pgmap v1769: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 6.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 980 op/s 2020-09-08T16:06:59.465 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:06:59.570 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:06:59.579 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:59 smithi166 bash[9446]: cluster 2020-09-08T16:06:57.704113+0000 osd.3 (osd.3) 116 : cluster [DBG] 8.0 scrub ok 2020-09-08T16:06:59.579 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:06:59 smithi166 bash[9446]: cluster 2020-09-08T16:06:58.065697+0000 mgr.x (mgr.24118) 1189 : cluster [DBG] pgmap v1769: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 6.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 980 op/s 2020-09-08T16:06:59.720 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:06:59.824 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:06:59.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:06:59 smithi166 bash[16593]: debug 2020-09-08T16:06:59.716+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:06:59.942 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:00.047 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:07:00.060 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:06:59 smithi104 bash[15273]: debug 2020-09-08T16:06:59.463+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:00.156 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:06:59 smithi055 bash[14301]: debug 2020-09-08T16:06:59.943+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:00.163 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:07:00.267 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:00.399 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:07:00.401 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:00.504 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:07:00.513 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:00 smithi055 bash[9296]: cluster 2020-09-08T16:06:59.299494+0000 osd.4 (osd.4) 71 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:00.514 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:00 smithi055 bash[9296]: cluster 2020-09-08T16:06:59.299575+0000 osd.4 (osd.4) 72 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:00.514 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:07:00 smithi055 bash[19003]: debug 2020-09-08T16:07:00.163+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:00.547 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:07:00.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:00 smithi166 bash[9446]: cluster 2020-09-08T16:06:59.299494+0000 osd.4 (osd.4) 71 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:00.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:00 smithi166 bash[9446]: cluster 2020-09-08T16:06:59.299575+0000 osd.4 (osd.4) 72 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:00.617 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:07:00 smithi166 bash[16593]: debug 2020-09-08T16:07:00.396+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:00.617 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:07:00.722 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:00.729 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:00 smithi104 bash[9203]: cluster 2020-09-08T16:06:59.299494+0000 osd.4 (osd.4) 71 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:00.729 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:00 smithi104 bash[9203]: cluster 2020-09-08T16:06:59.299575+0000 osd.4 (osd.4) 72 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:00.853 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:07:00 smithi055 bash[19003]: debug 2020-09-08T16:07:00.619+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:00.853 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:07:00 smithi055 bash[14301]: debug 2020-09-08T16:07:00.855+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:00.854 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:00.959 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:07:01.244 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:01 smithi104 bash[9203]: cluster 2020-09-08T16:07:00.066266+0000 mgr.x (mgr.24118) 1190 : cluster [DBG] pgmap v1770: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 6.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 948 op/s 2020-09-08T16:07:01.244 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:01 smithi104 bash[9203]: cluster 2020-09-08T16:07:00.215578+0000 osd.11 (osd.11) 114 : cluster [DBG] 8.3 scrub starts 2020-09-08T16:07:01.460 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:07:01.566 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:01.579 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:07:01 smithi104 bash[17588]: debug 2020-09-08T16:07:01.463+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:01.579 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:01 smithi104 bash[9203]: cluster 2020-09-08T16:07:00.243698+0000 mon.a (mon.0) 2124 : cluster [DBG] osdmap e665: 12 total, 12 up, 12 in 2020-09-08T16:07:01.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:01 smithi166 bash[9446]: cluster 2020-09-08T16:07:00.066266+0000 mgr.x (mgr.24118) 1190 : cluster [DBG] pgmap v1770: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 6.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 948 op/s 2020-09-08T16:07:01.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:01 smithi166 bash[9446]: cluster 2020-09-08T16:07:00.215578+0000 osd.11 (osd.11) 114 : cluster [DBG] 8.3 scrub starts 2020-09-08T16:07:01.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:01 smithi166 bash[9446]: cluster 2020-09-08T16:07:00.243698+0000 mon.a (mon.0) 2124 : cluster [DBG] osdmap e665: 12 total, 12 up, 12 in 2020-09-08T16:07:01.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:01 smithi055 bash[9296]: cluster 2020-09-08T16:07:00.066266+0000 mgr.x (mgr.24118) 1190 : cluster [DBG] pgmap v1770: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 6.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 948 op/s 2020-09-08T16:07:01.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:01 smithi055 bash[9296]: cluster 2020-09-08T16:07:00.215578+0000 osd.11 (osd.11) 114 : cluster [DBG] 8.3 scrub starts 2020-09-08T16:07:01.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:01 smithi055 bash[9296]: cluster 2020-09-08T16:07:00.243698+0000 mon.a (mon.0) 2124 : cluster [DBG] osdmap e665: 12 total, 12 up, 12 in 2020-09-08T16:07:02.274 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:02.378 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:03.002 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:07:02 smithi104 bash[15273]: debug 2020-09-08T16:07:02.275+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:03.067 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T16:07:03.068 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2020-09-08T16:07:03.068 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2020-09-08T16:07:03.068 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T16:07:03.069 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.069 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:226907 7.9 7:9de792d3:::benchmark_data_smithi166_48533_object92390:head [delete] snapc 0=[] ondisk+write+known_if_redirected e607)", 2020-09-08T16:07:03.069 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:03:44.529863+0000", 2020-09-08T16:07:03.070 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 198.53627886699999, 2020-09-08T16:07:03.070 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.485049389, 2020-09-08T16:07:03.070 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:03.071 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:03.071 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:03.071 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:07:03.072 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:07:03.072 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 226907 2020-09-08T16:07:03.072 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.073 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:03.073 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.073 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:03.075 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.529863+0000", 2020-09-08T16:07:03.075 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:03.076 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.076 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.076 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:03.077 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.529863+0000", 2020-09-08T16:07:03.077 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999766 2020-09-08T16:07:03.077 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.078 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.078 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:03.078 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.529839+0000", 2020-09-08T16:07:03.079 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9314999999999998e-05 2020-09-08T16:07:03.079 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.079 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.080 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:03.080 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.529889+0000", 2020-09-08T16:07:03.080 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1980000000000001e-06 2020-09-08T16:07:03.080 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.081 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.081 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:03.081 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.529897+0000", 2020-09-08T16:07:03.082 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2440000000000004e-06 2020-09-08T16:07:03.082 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.082 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.083 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:03.083 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.529901+0000", 2020-09-08T16:07:03.083 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.34672876400000002 2020-09-08T16:07:03.083 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.084 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.084 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:03.084 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.876630+0000", 2020-09-08T16:07:03.084 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000114522 2020-09-08T16:07:03.084 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.085 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.085 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:03.085 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.876744+0000", 2020-09-08T16:07:03.085 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011068700000000001 2020-09-08T16:07:03.086 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.086 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.086 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T16:07:03.086 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.876855+0000", 2020-09-08T16:07:03.086 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13790612699999999 2020-09-08T16:07:03.087 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.087 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.087 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:03.087 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.014761+0000", 2020-09-08T16:07:03.087 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1551999999999995e-05 2020-09-08T16:07:03.088 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.088 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.088 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:03.088 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.014833+0000", 2020-09-08T16:07:03.088 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6205999999999998e-05 2020-09-08T16:07:03.089 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.089 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.089 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:03.089 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.014879+0000", 2020-09-08T16:07:03.089 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3129000000000001e-05 2020-09-08T16:07:03.090 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.090 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.090 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:03.090 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.014912+0000", 2020-09-08T16:07:03.091 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.485049389 2020-09-08T16:07:03.091 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.091 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.091 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.091 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.092 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.092 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:226980 7.9 7:9fe58f55:::benchmark_data_smithi166_48533_object92463:head [delete] snapc 0=[] ondisk+write+known_if_redirected e607)", 2020-09-08T16:07:03.092 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:03:44.721098+0000", 2020-09-08T16:07:03.092 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 198.345043035, 2020-09-08T16:07:03.093 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.493088937, 2020-09-08T16:07:03.093 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:03.093 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:03.093 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:03.093 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:07:03.094 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:07:03.094 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 226980 2020-09-08T16:07:03.094 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.094 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:03.094 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.095 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:03.095 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.721098+0000", 2020-09-08T16:07:03.095 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:03.095 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.095 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.096 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:03.096 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.721098+0000", 2020-09-08T16:07:03.096 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999747 2020-09-08T16:07:03.096 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.096 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.097 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:03.097 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.721073+0000", 2020-09-08T16:07:03.097 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5899000000000001e-05 2020-09-08T16:07:03.097 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.097 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.098 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:03.098 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.721129+0000", 2020-09-08T16:07:03.098 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6709999999999994e-06 2020-09-08T16:07:03.098 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.098 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.098 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:03.099 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.721139+0000", 2020-09-08T16:07:03.099 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5349999999999998e-06 2020-09-08T16:07:03.099 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.099 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.099 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:03.100 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.721143+0000", 2020-09-08T16:07:03.100 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.322221333 2020-09-08T16:07:03.100 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.100 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.100 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:03.101 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.043365+0000", 2020-09-08T16:07:03.101 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012756000000000001 2020-09-08T16:07:03.101 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.101 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.101 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:03.102 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.043492+0000", 2020-09-08T16:07:03.102 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000115265 2020-09-08T16:07:03.102 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.102 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.102 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T16:07:03.102 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.043608+0000", 2020-09-08T16:07:03.103 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.158479866 2020-09-08T16:07:03.103 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.103 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.103 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:03.104 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.202087+0000", 2020-09-08T16:07:03.104 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.011980032999999999 2020-09-08T16:07:03.104 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.104 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.104 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:03.105 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.214067+0000", 2020-09-08T16:07:03.105 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4586999999999997e-05 2020-09-08T16:07:03.105 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.105 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.106 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:03.106 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.214132+0000", 2020-09-08T16:07:03.106 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5337000000000002e-05 2020-09-08T16:07:03.106 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.106 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.107 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:03.107 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.214187+0000", 2020-09-08T16:07:03.107 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.493088937 2020-09-08T16:07:03.107 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.107 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.108 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.108 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.108 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.108 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:227340 7.9 7:9b3f70e5:::benchmark_data_smithi166_48533_object92823:head [delete] snapc 0=[] ondisk+write+known_if_redirected e607)", 2020-09-08T16:07:03.109 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:03:45.370870+0000", 2020-09-08T16:07:03.109 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 197.69527129599999, 2020-09-08T16:07:03.109 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.51068199400000003, 2020-09-08T16:07:03.109 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:03.109 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:03.110 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:03.110 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:07:03.110 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:07:03.110 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 227340 2020-09-08T16:07:03.110 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.111 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:03.111 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.111 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:03.111 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.370870+0000", 2020-09-08T16:07:03.111 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:03.112 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.112 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.112 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:03.112 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.370870+0000", 2020-09-08T16:07:03.112 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999771 2020-09-08T16:07:03.113 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.113 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.113 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:03.113 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.370847+0000", 2020-09-08T16:07:03.114 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2298e-05 2020-09-08T16:07:03.114 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.114 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.114 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:03.114 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.370900+0000", 2020-09-08T16:07:03.115 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1313e-05 2020-09-08T16:07:03.115 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.115 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.115 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:03.115 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.370911+0000", 2020-09-08T16:07:03.116 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6820000000000002e-06 2020-09-08T16:07:03.116 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.116 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.116 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:03.116 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.370916+0000", 2020-09-08T16:07:03.117 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.370712027 2020-09-08T16:07:03.117 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.117 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.117 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:03.117 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.741628+0000", 2020-09-08T16:07:03.118 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000194477 2020-09-08T16:07:03.118 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.118 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.118 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:03.118 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.741822+0000", 2020-09-08T16:07:03.119 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000172304 2020-09-08T16:07:03.119 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.119 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.119 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T16:07:03.119 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.741994+0000", 2020-09-08T16:07:03.120 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0056553039999999999 2020-09-08T16:07:03.120 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.120 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.121 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:03.121 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.747650+0000", 2020-09-08T16:07:03.121 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13372262600000001 2020-09-08T16:07:03.121 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.121 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.122 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:03.122 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.881372+0000", 2020-09-08T16:07:03.122 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000106035 2020-09-08T16:07:03.123 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.123 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.123 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:03.123 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.881478+0000", 2020-09-08T16:07:03.124 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3815999999999999e-05 2020-09-08T16:07:03.124 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.124 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.125 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:03.125 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.881552+0000", 2020-09-08T16:07:03.125 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.51068199400000003 2020-09-08T16:07:03.126 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.126 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.126 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.126 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.126 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.127 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31643 8.e 8:736bce26:::benchmark_data_smithi166_52302_object31642:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:07:03.127 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.750728+0000", 2020-09-08T16:07:03.127 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 104.315413073, 2020-09-08T16:07:03.127 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.61505463800000004, 2020-09-08T16:07:03.128 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:03.128 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:03.128 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:03.128 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:03.128 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:03.128 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31643 2020-09-08T16:07:03.129 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.129 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:03.129 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.129 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:03.130 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.750728+0000", 2020-09-08T16:07:03.130 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:03.130 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.130 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.130 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:03.131 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.750728+0000", 2020-09-08T16:07:03.131 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999356 2020-09-08T16:07:03.131 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.131 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.132 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:03.132 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.750664+0000", 2020-09-08T16:07:03.132 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000181678 2020-09-08T16:07:03.132 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.133 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.133 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:03.133 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.750846+0000", 2020-09-08T16:07:03.133 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6409000000000002e-05 2020-09-08T16:07:03.134 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.134 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.134 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:03.135 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.750862+0000", 2020-09-08T16:07:03.135 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5299999999999996e-06 2020-09-08T16:07:03.135 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.136 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.136 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:03.136 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.750870+0000", 2020-09-08T16:07:03.136 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6950999999999997e-05 2020-09-08T16:07:03.136 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.137 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.137 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:03.137 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.750927+0000", 2020-09-08T16:07:03.137 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000201898 2020-09-08T16:07:03.138 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.138 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.138 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:03.138 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.751129+0000", 2020-09-08T16:07:03.139 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00023285100000000001 2020-09-08T16:07:03.139 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.139 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.139 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:07:03.139 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.751362+0000", 2020-09-08T16:07:03.140 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0013584879999999999 2020-09-08T16:07:03.140 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.140 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.140 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:03.140 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.752721+0000", 2020-09-08T16:07:03.140 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.61295605099999995 2020-09-08T16:07:03.141 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.141 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.141 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:03.141 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.365677+0000", 2020-09-08T16:07:03.141 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4460000000000003e-05 2020-09-08T16:07:03.142 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.142 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.142 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:03.142 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.365741+0000", 2020-09-08T16:07:03.142 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1807000000000001e-05 2020-09-08T16:07:03.143 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.143 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.143 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:03.143 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.365783+0000", 2020-09-08T16:07:03.143 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.61505463800000004 2020-09-08T16:07:03.144 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.144 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.144 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.144 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.145 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.145 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31705 8.e 8:71681bc2:::benchmark_data_smithi166_52302_object31704:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:07:03.145 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.782456+0000", 2020-09-08T16:07:03.145 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 104.28368525800001, 2020-09-08T16:07:03.146 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.58349529499999997, 2020-09-08T16:07:03.146 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:03.146 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:03.146 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:03.146 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:03.147 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:03.147 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31705 2020-09-08T16:07:03.147 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.147 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:03.148 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.148 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:03.148 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.782456+0000", 2020-09-08T16:07:03.148 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:03.148 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.149 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.149 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:03.149 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.782456+0000", 2020-09-08T16:07:03.149 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999528 2020-09-08T16:07:03.149 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.150 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.150 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:03.150 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.782409+0000", 2020-09-08T16:07:03.150 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017631600000000001 2020-09-08T16:07:03.150 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.151 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.151 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:03.151 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.782585+0000", 2020-09-08T16:07:03.151 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4258e-05 2020-09-08T16:07:03.151 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.151 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.152 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:03.152 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.782600+0000", 2020-09-08T16:07:03.152 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0360000000000006e-06 2020-09-08T16:07:03.152 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.152 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.153 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:03.153 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.782608+0000", 2020-09-08T16:07:03.153 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4220000000000005e-05 2020-09-08T16:07:03.153 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.153 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.154 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:03.154 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.782672+0000", 2020-09-08T16:07:03.154 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020994500000000001 2020-09-08T16:07:03.154 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.155 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.155 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:03.155 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.782882+0000", 2020-09-08T16:07:03.156 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00022301600000000001 2020-09-08T16:07:03.156 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.156 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.156 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:07:03.157 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.783105+0000", 2020-09-08T16:07:03.157 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0016531040000000001 2020-09-08T16:07:03.157 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.157 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.158 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:03.158 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.784758+0000", 2020-09-08T16:07:03.158 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.58109964199999997 2020-09-08T16:07:03.158 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.158 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.158 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:03.159 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.365858+0000", 2020-09-08T16:07:03.159 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4354999999999998e-05 2020-09-08T16:07:03.159 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.159 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.160 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:03.160 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.365902+0000", 2020-09-08T16:07:03.160 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9607999999999997e-05 2020-09-08T16:07:03.160 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.161 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.161 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:03.161 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.365951+0000", 2020-09-08T16:07:03.162 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.58349529499999997 2020-09-08T16:07:03.162 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.162 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.163 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.163 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.164 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.164 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31736 8.e 8:7c50f50c:::benchmark_data_smithi166_52302_object31735:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:07:03.164 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.797191+0000", 2020-09-08T16:07:03.165 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 104.268950872, 2020-09-08T16:07:03.165 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.57092679499999999, 2020-09-08T16:07:03.165 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:03.166 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:03.166 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:03.166 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:03.167 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:03.167 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31736 2020-09-08T16:07:03.167 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.168 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:03.168 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.168 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:03.169 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.797191+0000", 2020-09-08T16:07:03.169 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:03.169 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.170 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.170 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:03.170 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.797191+0000", 2020-09-08T16:07:03.171 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999437 2020-09-08T16:07:03.171 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.171 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.172 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:03.172 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.797134+0000", 2020-09-08T16:07:03.173 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017540200000000001 2020-09-08T16:07:03.173 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.173 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.174 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:03.174 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.797310+0000", 2020-09-08T16:07:03.174 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4871999999999998e-05 2020-09-08T16:07:03.175 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.175 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.176 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:03.176 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.797334+0000", 2020-09-08T16:07:03.176 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6299999999999998e-06 2020-09-08T16:07:03.177 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.177 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.177 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:03.178 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.797342+0000", 2020-09-08T16:07:03.178 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000104518 2020-09-08T16:07:03.178 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.179 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.179 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:03.180 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.797447+0000", 2020-09-08T16:07:03.180 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000231299 2020-09-08T16:07:03.180 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.181 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.181 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:03.181 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.797678+0000", 2020-09-08T16:07:03.182 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018062800000000001 2020-09-08T16:07:03.182 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.183 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.183 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:07:03.184 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.797859+0000", 2020-09-08T16:07:03.184 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0026988749999999999 2020-09-08T16:07:03.184 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.184 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.185 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:03.185 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.800557+0000", 2020-09-08T16:07:03.186 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.56747323500000002 2020-09-08T16:07:03.186 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.186 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.187 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:03.187 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.368031+0000", 2020-09-08T16:07:03.187 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0442000000000003e-05 2020-09-08T16:07:03.188 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.188 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.188 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:03.189 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.368081+0000", 2020-09-08T16:07:03.189 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6242000000000003e-05 2020-09-08T16:07:03.189 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.190 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.190 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:03.191 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.368117+0000", 2020-09-08T16:07:03.191 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.57092679499999999 2020-09-08T16:07:03.191 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.192 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.192 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.192 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.192 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.193 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31756 8.e 8:76fbe07a:::benchmark_data_smithi166_52302_object31755:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:07:03.193 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.805769+0000", 2020-09-08T16:07:03.193 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 104.26037272000001, 2020-09-08T16:07:03.194 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.56253396600000005, 2020-09-08T16:07:03.194 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:03.194 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:03.195 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:03.195 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:03.195 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:03.196 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31756 2020-09-08T16:07:03.196 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.196 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:03.197 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.197 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:03.197 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.805769+0000", 2020-09-08T16:07:03.198 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:03.198 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.198 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.199 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:03.199 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.805769+0000", 2020-09-08T16:07:03.199 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999695 2020-09-08T16:07:03.200 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.200 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.200 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:03.201 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.805738+0000", 2020-09-08T16:07:03.201 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011208 2020-09-08T16:07:03.201 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.202 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.202 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:03.202 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.805850+0000", 2020-09-08T16:07:03.202 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5447999999999998e-05 2020-09-08T16:07:03.203 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.203 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.203 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:03.203 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.805866+0000", 2020-09-08T16:07:03.203 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.242e-06 2020-09-08T16:07:03.204 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.204 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.204 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:03.204 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.805874+0000", 2020-09-08T16:07:03.204 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3807999999999996e-05 2020-09-08T16:07:03.204 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.205 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.205 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:03.205 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.805938+0000", 2020-09-08T16:07:03.205 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017503199999999999 2020-09-08T16:07:03.211 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.211 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.212 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:03.212 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.806113+0000", 2020-09-08T16:07:03.212 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019673 2020-09-08T16:07:03.213 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.213 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.213 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:07:03.213 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.806310+0000", 2020-09-08T16:07:03.213 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0015218429999999999 2020-09-08T16:07:03.214 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.214 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.214 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:03.214 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.807832+0000", 2020-09-08T16:07:03.214 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.56039003600000004 2020-09-08T16:07:03.215 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.215 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.215 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:03.215 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.368222+0000", 2020-09-08T16:07:03.215 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1275e-05 2020-09-08T16:07:03.216 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.216 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.216 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:03.216 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.368263+0000", 2020-09-08T16:07:03.216 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9841e-05 2020-09-08T16:07:03.217 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.217 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.217 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:03.217 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.368303+0000", 2020-09-08T16:07:03.217 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.56253396600000005 2020-09-08T16:07:03.217 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.218 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.218 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.218 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.218 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.219 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31768 8.e 8:75141454:::benchmark_data_smithi166_52302_object31767:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:07:03.219 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.811615+0000", 2020-09-08T16:07:03.219 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 104.254526387, 2020-09-08T16:07:03.219 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.55852875300000004, 2020-09-08T16:07:03.219 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:03.219 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:03.220 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:03.220 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:03.220 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:03.220 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31768 2020-09-08T16:07:03.220 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.221 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:03.221 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.221 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:03.221 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.811615+0000", 2020-09-08T16:07:03.221 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:03.222 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.222 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.222 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:03.222 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.811615+0000", 2020-09-08T16:07:03.222 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999371 2020-09-08T16:07:03.223 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.223 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.223 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:03.223 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.811552+0000", 2020-09-08T16:07:03.223 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000193864 2020-09-08T16:07:03.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.224 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.224 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:03.224 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.811746+0000", 2020-09-08T16:07:03.224 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6623999999999999e-05 2020-09-08T16:07:03.225 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.225 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.225 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:03.225 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.811762+0000", 2020-09-08T16:07:03.225 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5710000000000004e-06 2020-09-08T16:07:03.226 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.226 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.226 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:03.226 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.811772+0000", 2020-09-08T16:07:03.227 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6433000000000004e-05 2020-09-08T16:07:03.227 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.227 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.227 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:03.227 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.811858+0000", 2020-09-08T16:07:03.228 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000288282 2020-09-08T16:07:03.228 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.228 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.228 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:03.228 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.812147+0000", 2020-09-08T16:07:03.229 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00026367099999999999 2020-09-08T16:07:03.229 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.229 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.229 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:07:03.229 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.812410+0000", 2020-09-08T16:07:03.230 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0018146010000000001 2020-09-08T16:07:03.230 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.230 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.230 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:03.231 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.814225+0000", 2020-09-08T16:07:03.232 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.55576947099999996 2020-09-08T16:07:03.232 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.232 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.232 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:03.233 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.369995+0000", 2020-09-08T16:07:03.233 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9792999999999999e-05 2020-09-08T16:07:03.234 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.234 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.234 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:03.234 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.370084+0000", 2020-09-08T16:07:03.234 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9508e-05 2020-09-08T16:07:03.235 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.235 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.235 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:03.235 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.370144+0000", 2020-09-08T16:07:03.235 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.55852875300000004 2020-09-08T16:07:03.236 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.236 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.236 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.236 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.237 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.237 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31783 8.e 8:70e2f01d:::benchmark_data_smithi166_52302_object31782:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:07:03.237 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.818023+0000", 2020-09-08T16:07:03.237 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 104.24811881700001, 2020-09-08T16:07:03.238 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.552480792, 2020-09-08T16:07:03.238 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:03.238 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:03.238 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:03.239 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:03.239 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:03.239 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31783 2020-09-08T16:07:03.239 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.240 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:03.240 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.240 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:03.240 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818023+0000", 2020-09-08T16:07:03.241 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:03.241 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.241 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.241 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:03.241 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818023+0000", 2020-09-08T16:07:03.242 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999285 2020-09-08T16:07:03.242 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.242 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.242 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:03.242 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.817951+0000", 2020-09-08T16:07:03.243 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016045400000000001 2020-09-08T16:07:03.243 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.243 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.244 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:03.244 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818112+0000", 2020-09-08T16:07:03.245 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8515000000000001e-05 2020-09-08T16:07:03.245 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.245 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.245 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:03.245 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818130+0000", 2020-09-08T16:07:03.246 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3018e-05 2020-09-08T16:07:03.246 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.246 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.246 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:03.247 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818143+0000", 2020-09-08T16:07:03.247 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2869999999999997e-05 2020-09-08T16:07:03.247 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.247 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.247 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:03.248 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818236+0000", 2020-09-08T16:07:03.248 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018752999999999999 2020-09-08T16:07:03.248 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.248 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.248 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:03.249 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818423+0000", 2020-09-08T16:07:03.249 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018469299999999999 2020-09-08T16:07:03.249 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.249 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.249 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:07:03.250 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818608+0000", 2020-09-08T16:07:03.250 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0034836210000000001 2020-09-08T16:07:03.250 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.251 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.251 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:03.251 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.822092+0000", 2020-09-08T16:07:03.251 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.54820981400000002 2020-09-08T16:07:03.251 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.252 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.252 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:03.252 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.370302+0000", 2020-09-08T16:07:03.252 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014577500000000001 2020-09-08T16:07:03.252 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.252 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.253 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:03.253 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.370447+0000", 2020-09-08T16:07:03.253 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6060000000000002e-05 2020-09-08T16:07:03.253 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.254 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.254 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:03.254 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.370503+0000", 2020-09-08T16:07:03.255 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.552480792 2020-09-08T16:07:03.255 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.255 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.256 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.257 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.257 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.257 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31785 8.e 8:729ea262:::benchmark_data_smithi166_52302_object31784:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:07:03.258 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.818851+0000", 2020-09-08T16:07:03.259 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 104.24729089500001, 2020-09-08T16:07:03.259 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.55661606799999996, 2020-09-08T16:07:03.259 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:03.259 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:03.259 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:03.260 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:03.260 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:03.260 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31785 2020-09-08T16:07:03.260 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.260 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:03.261 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.261 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:03.261 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818851+0000", 2020-09-08T16:07:03.261 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:03.261 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.262 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.262 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:03.262 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818851+0000", 2020-09-08T16:07:03.262 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999475 2020-09-08T16:07:03.262 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.263 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.263 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:03.263 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818798+0000", 2020-09-08T16:07:03.263 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000192981 2020-09-08T16:07:03.263 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.264 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.264 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:03.264 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818991+0000", 2020-09-08T16:07:03.264 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6706000000000003e-05 2020-09-08T16:07:03.265 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.265 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.265 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:03.265 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.819038+0000", 2020-09-08T16:07:03.265 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9558000000000001e-05 2020-09-08T16:07:03.266 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.266 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.266 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:03.266 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.819057+0000", 2020-09-08T16:07:03.266 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0012248630000000001 2020-09-08T16:07:03.267 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.267 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.267 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:03.267 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.820282+0000", 2020-09-08T16:07:03.267 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00030317400000000002 2020-09-08T16:07:03.268 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.268 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.268 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:03.268 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.820585+0000", 2020-09-08T16:07:03.268 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019230500000000001 2020-09-08T16:07:03.269 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.269 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.269 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:07:03.269 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.820778+0000", 2020-09-08T16:07:03.269 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0017226139999999999 2020-09-08T16:07:03.270 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.270 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.270 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:03.270 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.822500+0000", 2020-09-08T16:07:03.270 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.55285292500000005 2020-09-08T16:07:03.270 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.271 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.271 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:03.271 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.375353+0000", 2020-09-08T16:07:03.271 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.2335000000000005e-05 2020-09-08T16:07:03.271 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.272 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.272 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:03.272 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.375425+0000", 2020-09-08T16:07:03.272 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1214999999999998e-05 2020-09-08T16:07:03.273 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.273 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.273 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:03.273 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.375467+0000", 2020-09-08T16:07:03.273 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.55661606799999996 2020-09-08T16:07:03.274 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.274 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.274 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.275 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.275 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.275 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31796 8.e 8:7d49049a:::benchmark_data_smithi166_52302_object31795:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:07:03.275 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.823126+0000", 2020-09-08T16:07:03.275 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 104.24301543599999, 2020-09-08T16:07:03.276 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.55262376300000005, 2020-09-08T16:07:03.276 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:03.276 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:03.276 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:03.276 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:03.277 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:03.277 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31796 2020-09-08T16:07:03.277 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.277 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:03.277 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.278 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:03.278 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.823126+0000", 2020-09-08T16:07:03.278 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:03.278 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.278 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.278 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:03.279 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.823126+0000", 2020-09-08T16:07:03.279 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999585 2020-09-08T16:07:03.279 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.279 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.279 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:03.280 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.823085+0000", 2020-09-08T16:07:03.280 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000205186 2020-09-08T16:07:03.280 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.280 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.280 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:03.281 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.823290+0000", 2020-09-08T16:07:03.281 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2133999999999998e-05 2020-09-08T16:07:03.281 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.281 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.281 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:03.282 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.823312+0000", 2020-09-08T16:07:03.282 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2764e-05 2020-09-08T16:07:03.282 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.282 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.282 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:03.282 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.823325+0000", 2020-09-08T16:07:03.283 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7626000000000006e-05 2020-09-08T16:07:03.283 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.283 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.283 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:03.283 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.823392+0000", 2020-09-08T16:07:03.284 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019909499999999999 2020-09-08T16:07:03.284 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.284 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.284 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:03.285 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.823591+0000", 2020-09-08T16:07:03.285 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00026281999999999998 2020-09-08T16:07:03.285 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.285 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.285 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:07:03.286 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.823854+0000", 2020-09-08T16:07:03.287 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001309209 2020-09-08T16:07:03.287 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.287 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.287 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:03.287 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.825163+0000", 2020-09-08T16:07:03.288 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.550413869 2020-09-08T16:07:03.288 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.288 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.288 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:03.288 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.375577+0000", 2020-09-08T16:07:03.289 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4975999999999993e-05 2020-09-08T16:07:03.289 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.289 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.289 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:03.289 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.375672+0000", 2020-09-08T16:07:03.290 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7526000000000003e-05 2020-09-08T16:07:03.290 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.290 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.290 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:03.290 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.375750+0000", 2020-09-08T16:07:03.291 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.55262376300000005 2020-09-08T16:07:03.291 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.291 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.291 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.291 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.291 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.292 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31818 8.e 8:755728de:::benchmark_data_smithi166_52302_object31817:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:07:03.292 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.834010+0000", 2020-09-08T16:07:03.292 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 104.232131233, 2020-09-08T16:07:03.292 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.54188254300000005, 2020-09-08T16:07:03.292 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:03.293 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:03.293 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:03.293 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:03.293 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:03.293 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31818 2020-09-08T16:07:03.294 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.294 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:03.294 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.294 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:03.294 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.834010+0000", 2020-09-08T16:07:03.295 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:03.295 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.295 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.295 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:03.295 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.834010+0000", 2020-09-08T16:07:03.295 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999766 2020-09-08T16:07:03.296 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.297 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.297 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:03.297 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.833987+0000", 2020-09-08T16:07:03.297 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011306000000000001 2020-09-08T16:07:03.297 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.298 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.298 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:03.298 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.834100+0000", 2020-09-08T16:07:03.298 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.306e-05 2020-09-08T16:07:03.299 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.299 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.299 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:03.299 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.834113+0000", 2020-09-08T16:07:03.299 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9039999999999992e-06 2020-09-08T16:07:03.300 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.300 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.300 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:03.300 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.834121+0000", 2020-09-08T16:07:03.300 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8900000000000003e-05 2020-09-08T16:07:03.301 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.301 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.301 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:03.301 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.834170+0000", 2020-09-08T16:07:03.301 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012346699999999999 2020-09-08T16:07:03.302 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.302 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.302 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:03.302 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.834293+0000", 2020-09-08T16:07:03.303 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000121549 2020-09-08T16:07:03.303 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.303 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.303 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:07:03.303 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.834415+0000", 2020-09-08T16:07:03.304 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001383544 2020-09-08T16:07:03.304 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.304 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.304 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:03.304 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.835798+0000", 2020-09-08T16:07:03.305 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.53999969400000003 2020-09-08T16:07:03.305 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.305 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.305 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:03.305 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.375798+0000", 2020-09-08T16:07:03.306 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5198000000000002e-05 2020-09-08T16:07:03.306 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.306 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.306 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:03.306 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.375853+0000", 2020-09-08T16:07:03.306 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9635000000000003e-05 2020-09-08T16:07:03.307 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.308 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.308 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:03.308 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.375893+0000", 2020-09-08T16:07:03.308 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.54188254300000005 2020-09-08T16:07:03.308 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.309 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.310 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.310 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.310 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.310 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31823 8.e 8:75100c31:::benchmark_data_smithi166_52302_object31822:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:07:03.311 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.837476+0000", 2020-09-08T16:07:03.311 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 104.22866540299999, 2020-09-08T16:07:03.311 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.53864092699999999, 2020-09-08T16:07:03.311 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:03.311 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:03.313 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:03.313 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:03.313 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:03.313 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31823 2020-09-08T16:07:03.314 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.314 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:03.314 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.314 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:03.314 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837476+0000", 2020-09-08T16:07:03.315 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:03.315 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.315 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.315 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:03.315 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837476+0000", 2020-09-08T16:07:03.315 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999728 2020-09-08T16:07:03.316 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.316 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.316 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:03.318 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837449+0000", 2020-09-08T16:07:03.318 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020261500000000001 2020-09-08T16:07:03.318 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.318 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.318 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:03.319 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837651+0000", 2020-09-08T16:07:03.319 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2478999999999999e-05 2020-09-08T16:07:03.319 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.319 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.319 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:03.320 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837674+0000", 2020-09-08T16:07:03.320 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0239999999999997e-06 2020-09-08T16:07:03.320 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.320 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.320 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:03.321 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837682+0000", 2020-09-08T16:07:03.321 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6309000000000005e-05 2020-09-08T16:07:03.321 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.321 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.321 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:03.322 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837748+0000", 2020-09-08T16:07:03.322 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020641600000000001 2020-09-08T16:07:03.322 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.322 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.322 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:03.323 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837955+0000", 2020-09-08T16:07:03.323 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000256066 2020-09-08T16:07:03.323 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.323 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.323 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:07:03.323 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.838211+0000", 2020-09-08T16:07:03.324 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0021088489999999999 2020-09-08T16:07:03.324 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.324 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.324 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:03.324 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.840319+0000", 2020-09-08T16:07:03.325 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.53564020400000001 2020-09-08T16:07:03.325 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.325 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.325 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:03.325 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.375960+0000", 2020-09-08T16:07:03.326 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5110000000000003e-05 2020-09-08T16:07:03.326 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.326 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.326 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:03.326 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376045+0000", 2020-09-08T16:07:03.327 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.2197e-05 2020-09-08T16:07:03.327 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.327 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.327 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:03.328 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376117+0000", 2020-09-08T16:07:03.328 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.53864092699999999 2020-09-08T16:07:03.328 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.328 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.328 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.329 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.329 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.329 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31825 8.e 8:7a912d15:::benchmark_data_smithi166_52302_object31824:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:07:03.329 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.837773+0000", 2020-09-08T16:07:03.329 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 104.22836845800001, 2020-09-08T16:07:03.330 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.53848058200000004, 2020-09-08T16:07:03.330 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:03.330 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:03.330 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:03.330 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:03.331 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:03.331 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31825 2020-09-08T16:07:03.331 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.331 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:03.331 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.331 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:03.332 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837773+0000", 2020-09-08T16:07:03.332 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:03.332 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.332 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.332 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:03.333 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837773+0000", 2020-09-08T16:07:03.333 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999552 2020-09-08T16:07:03.333 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.333 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.333 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:03.334 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837728+0000", 2020-09-08T16:07:03.334 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017725999999999999 2020-09-08T16:07:03.334 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.334 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.334 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:03.335 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837905+0000", 2020-09-08T16:07:03.335 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5651e-05 2020-09-08T16:07:03.335 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.335 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.335 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:03.335 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837921+0000", 2020-09-08T16:07:03.336 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8410000000000002e-06 2020-09-08T16:07:03.336 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.336 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.336 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:03.336 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837928+0000", 2020-09-08T16:07:03.337 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0013380829999999999 2020-09-08T16:07:03.337 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.338 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.338 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:03.338 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.839266+0000", 2020-09-08T16:07:03.339 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012799299999999999 2020-09-08T16:07:03.339 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.340 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.340 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:03.341 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.839394+0000", 2020-09-08T16:07:03.341 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015803 2020-09-08T16:07:03.341 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.341 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.342 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:07:03.342 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.839552+0000", 2020-09-08T16:07:03.342 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00098459299999999997 2020-09-08T16:07:03.342 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.342 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.343 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:03.343 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.840536+0000", 2020-09-08T16:07:03.343 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.53563405900000005 2020-09-08T16:07:03.343 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.343 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.344 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:03.344 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376170+0000", 2020-09-08T16:07:03.344 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8896000000000001e-05 2020-09-08T16:07:03.344 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.344 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.344 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:03.345 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376219+0000", 2020-09-08T16:07:03.345 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4177999999999997e-05 2020-09-08T16:07:03.345 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.345 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.345 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:03.346 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376254+0000", 2020-09-08T16:07:03.346 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.53848058200000004 2020-09-08T16:07:03.346 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.346 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.346 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.347 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.347 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.347 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31864 8.e 8:7f85638c:::benchmark_data_smithi166_52302_object31863:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:07:03.347 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.866337+0000", 2020-09-08T16:07:03.347 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 104.19980489, 2020-09-08T16:07:03.348 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.51017505299999999, 2020-09-08T16:07:03.348 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:03.348 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:03.348 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:03.348 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:03.349 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:03.349 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31864 2020-09-08T16:07:03.349 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.349 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:03.349 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.350 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:03.350 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.866337+0000", 2020-09-08T16:07:03.350 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:03.350 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.350 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.351 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:03.351 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.866337+0000", 2020-09-08T16:07:03.351 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999552 2020-09-08T16:07:03.351 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.351 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.351 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:03.352 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.866292+0000", 2020-09-08T16:07:03.352 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014819099999999999 2020-09-08T16:07:03.352 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.352 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.352 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:03.353 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.866440+0000", 2020-09-08T16:07:03.353 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1081999999999999e-05 2020-09-08T16:07:03.353 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.353 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.353 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:03.354 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.866461+0000", 2020-09-08T16:07:03.354 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5469999999999997e-06 2020-09-08T16:07:03.354 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.354 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.354 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:03.355 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.866470+0000", 2020-09-08T16:07:03.355 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0965999999999997e-05 2020-09-08T16:07:03.355 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.355 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.355 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:03.355 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.866551+0000", 2020-09-08T16:07:03.356 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000222134 2020-09-08T16:07:03.356 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.356 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.356 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:03.356 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.866773+0000", 2020-09-08T16:07:03.357 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00025741099999999998 2020-09-08T16:07:03.357 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.357 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.357 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:07:03.357 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.867030+0000", 2020-09-08T16:07:03.358 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0017267059999999999 2020-09-08T16:07:03.358 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.359 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.359 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:03.360 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.868757+0000", 2020-09-08T16:07:03.361 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.507591825 2020-09-08T16:07:03.361 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.361 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.361 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:03.362 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376349+0000", 2020-09-08T16:07:03.362 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2278000000000001e-05 2020-09-08T16:07:03.362 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.363 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.363 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:03.363 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376431+0000", 2020-09-08T16:07:03.363 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0594999999999997e-05 2020-09-08T16:07:03.364 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.364 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.364 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:03.364 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376512+0000", 2020-09-08T16:07:03.364 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.51017505299999999 2020-09-08T16:07:03.365 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.365 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.365 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.365 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.365 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.366 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31885 8.e 8:7d64cde5:::benchmark_data_smithi166_52302_object31884:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:07:03.366 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.888695+0000", 2020-09-08T16:07:03.366 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 104.177445976, 2020-09-08T16:07:03.366 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.48797019899999999, 2020-09-08T16:07:03.366 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:03.367 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:03.367 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:03.367 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:03.367 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:03.367 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31885 2020-09-08T16:07:03.368 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.368 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:03.368 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.368 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:03.368 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.888695+0000", 2020-09-08T16:07:03.369 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:03.370 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.370 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.371 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:03.371 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.888695+0000", 2020-09-08T16:07:03.371 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999337 2020-09-08T16:07:03.371 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.371 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.372 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:03.372 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.888629+0000", 2020-09-08T16:07:03.372 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00021185999999999999 2020-09-08T16:07:03.372 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.372 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.372 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:03.373 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.888841+0000", 2020-09-08T16:07:03.373 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6365999999999997e-05 2020-09-08T16:07:03.373 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.373 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.373 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:03.374 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.888878+0000", 2020-09-08T16:07:03.374 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3505e-05 2020-09-08T16:07:03.374 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.374 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.374 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:03.375 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.888891+0000", 2020-09-08T16:07:03.375 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000101461 2020-09-08T16:07:03.375 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.375 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.376 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:03.376 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.888993+0000", 2020-09-08T16:07:03.376 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00028271600000000002 2020-09-08T16:07:03.376 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.376 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.377 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:03.377 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.889275+0000", 2020-09-08T16:07:03.377 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00021280800000000001 2020-09-08T16:07:03.377 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.377 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.378 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:07:03.378 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.889488+0000", 2020-09-08T16:07:03.378 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020818630000000002 2020-09-08T16:07:03.378 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.378 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.378 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:03.379 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.891570+0000", 2020-09-08T16:07:03.379 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.48500274799999998 2020-09-08T16:07:03.380 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.380 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.380 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:03.381 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376573+0000", 2020-09-08T16:07:03.381 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0531999999999997e-05 2020-09-08T16:07:03.381 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.382 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.382 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:03.382 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376623+0000", 2020-09-08T16:07:03.382 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2401000000000003e-05 2020-09-08T16:07:03.382 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.383 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.383 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:03.383 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376666+0000", 2020-09-08T16:07:03.383 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.48797019899999999 2020-09-08T16:07:03.383 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.384 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.384 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.384 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.384 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.384 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:123852 8.e 8:7fbe813d:::benchmark_data_smithi166_52302_object123851:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e656)", 2020-09-08T16:07:03.384 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:06:27.491840+0000", 2020-09-08T16:07:03.385 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 35.574301771000002, 2020-09-08T16:07:03.385 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.57404195000000002, 2020-09-08T16:07:03.385 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:03.385 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:03.385 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:03.386 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:03.386 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:03.386 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 123852 2020-09-08T16:07:03.386 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.386 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:03.387 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.387 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:03.387 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.491840+0000", 2020-09-08T16:07:03.387 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:03.387 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.388 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.388 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:03.388 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.491840+0000", 2020-09-08T16:07:03.388 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999518 2020-09-08T16:07:03.388 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.389 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.389 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:03.389 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.491791+0000", 2020-09-08T16:07:03.389 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0001906 2020-09-08T16:07:03.389 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.390 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.390 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:03.391 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.491982+0000", 2020-09-08T16:07:03.391 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3278999999999999e-05 2020-09-08T16:07:03.391 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.391 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.391 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:03.394 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.492005+0000", 2020-09-08T16:07:03.394 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8269999999999993e-06 2020-09-08T16:07:03.394 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.394 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.394 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:03.395 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.492015+0000", 2020-09-08T16:07:03.395 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.449991539 2020-09-08T16:07:03.395 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.395 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.395 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:03.396 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.942007+0000", 2020-09-08T16:07:03.396 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00025862900000000002 2020-09-08T16:07:03.396 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.396 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.396 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:03.397 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.942265+0000", 2020-09-08T16:07:03.397 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00033429400000000001 2020-09-08T16:07:03.397 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.397 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.397 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:07:03.398 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.942600+0000", 2020-09-08T16:07:03.398 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.024758130999999999 2020-09-08T16:07:03.398 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.398 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.398 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:03.399 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.967358+0000", 2020-09-08T16:07:03.399 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.098383480999999995 2020-09-08T16:07:03.399 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.399 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.399 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:03.399 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.065741+0000", 2020-09-08T16:07:03.400 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2052000000000007e-05 2020-09-08T16:07:03.400 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.400 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.400 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:03.400 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.065823+0000", 2020-09-08T16:07:03.401 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8335000000000003e-05 2020-09-08T16:07:03.401 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.401 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.401 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:03.401 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.065882+0000", 2020-09-08T16:07:03.402 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.57404195000000002 2020-09-08T16:07:03.402 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.402 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.402 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.402 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.403 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.403 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:123863 8.e 8:70bdf1a9:::benchmark_data_smithi166_52302_object123862:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e656)", 2020-09-08T16:07:03.403 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:06:27.521581+0000", 2020-09-08T16:07:03.403 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 35.544560324999999, 2020-09-08T16:07:03.403 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.54466917999999998, 2020-09-08T16:07:03.404 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:03.404 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:03.404 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:03.404 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:03.404 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:03.404 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 123863 2020-09-08T16:07:03.405 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.405 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:03.405 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.405 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:03.405 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.521581+0000", 2020-09-08T16:07:03.406 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:03.406 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.406 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.406 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:03.406 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.521581+0000", 2020-09-08T16:07:03.407 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999447 2020-09-08T16:07:03.407 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.407 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.407 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:03.407 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.521526+0000", 2020-09-08T16:07:03.408 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016246399999999999 2020-09-08T16:07:03.408 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.408 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.408 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:03.408 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.521688+0000", 2020-09-08T16:07:03.409 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3019000000000001e-05 2020-09-08T16:07:03.409 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.409 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.409 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:03.409 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.521711+0000", 2020-09-08T16:07:03.409 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8069999999999994e-06 2020-09-08T16:07:03.410 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.415 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.415 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:03.416 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.521721+0000", 2020-09-08T16:07:03.416 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.43898494999999998 2020-09-08T16:07:03.416 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.416 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.417 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:03.417 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.960706+0000", 2020-09-08T16:07:03.417 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013872799999999999 2020-09-08T16:07:03.417 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.417 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.418 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:03.418 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.960845+0000", 2020-09-08T16:07:03.418 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018871400000000001 2020-09-08T16:07:03.418 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.418 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.419 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:07:03.419 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.961034+0000", 2020-09-08T16:07:03.419 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.104627553 2020-09-08T16:07:03.419 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.419 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.420 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:03.420 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.065661+0000", 2020-09-08T16:07:03.420 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00037425099999999998 2020-09-08T16:07:03.420 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.420 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.420 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:03.421 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.066035+0000", 2020-09-08T16:07:03.421 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000125983 2020-09-08T16:07:03.421 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.421 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.421 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:03.422 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.066161+0000", 2020-09-08T16:07:03.422 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8944999999999996e-05 2020-09-08T16:07:03.422 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.422 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.422 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:03.423 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.066250+0000", 2020-09-08T16:07:03.423 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.54466917999999998 2020-09-08T16:07:03.423 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.423 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.423 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.424 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.424 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.424 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:123888 8.e 8:76457be3:::benchmark_data_smithi166_52302_object123887:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e656)", 2020-09-08T16:07:03.424 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:06:27.586295+0000", 2020-09-08T16:07:03.424 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 35.479846135000003, 2020-09-08T16:07:03.424 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.49297180800000001, 2020-09-08T16:07:03.425 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:03.425 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:03.425 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:03.425 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:03.425 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:03.426 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 123888 2020-09-08T16:07:03.426 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.426 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:03.426 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.426 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:03.427 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.586295+0000", 2020-09-08T16:07:03.427 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:03.427 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.427 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.427 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:03.428 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.586295+0000", 2020-09-08T16:07:03.428 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999766 2020-09-08T16:07:03.428 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.428 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.428 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:03.428 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.586272+0000", 2020-09-08T16:07:03.429 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6551999999999994e-05 2020-09-08T16:07:03.429 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.429 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.429 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:03.429 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.586359+0000", 2020-09-08T16:07:03.430 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2830000000000001e-05 2020-09-08T16:07:03.430 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.430 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.430 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:03.430 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.586371+0000", 2020-09-08T16:07:03.431 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5549999999999999e-06 2020-09-08T16:07:03.431 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.431 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.431 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:03.431 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.586378+0000", 2020-09-08T16:07:03.431 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.47403673400000002 2020-09-08T16:07:03.432 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.432 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.432 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:03.432 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.060415+0000", 2020-09-08T16:07:03.432 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000222986 2020-09-08T16:07:03.433 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.433 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.433 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:03.433 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.060638+0000", 2020-09-08T16:07:03.433 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00029462100000000001 2020-09-08T16:07:03.434 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.434 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.434 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:07:03.434 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.060932+0000", 2020-09-08T16:07:03.434 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0057477409999999998 2020-09-08T16:07:03.435 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.435 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.435 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:03.435 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.066680+0000", 2020-09-08T16:07:03.435 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.012502819 2020-09-08T16:07:03.435 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.436 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.436 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:03.436 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.079183+0000", 2020-09-08T16:07:03.436 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9733e-05 2020-09-08T16:07:03.436 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.437 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.437 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:03.437 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.079233+0000", 2020-09-08T16:07:03.437 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4449999999999997e-05 2020-09-08T16:07:03.437 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.438 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.438 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:03.438 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.079267+0000", 2020-09-08T16:07:03.438 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.49297180800000001 2020-09-08T16:07:03.438 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.439 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.439 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.439 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.439 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.439 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:126481 8.e 8:789e7fac:::benchmark_data_smithi166_52302_object126480:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e658)", 2020-09-08T16:07:03.439 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:06:31.331487+0000", 2020-09-08T16:07:03.440 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 31.734654752000001, 2020-09-08T16:07:03.440 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.50117400099999998, 2020-09-08T16:07:03.440 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:03.440 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:03.440 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:03.441 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:03.441 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:03.441 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 126481 2020-09-08T16:07:03.441 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.441 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:03.442 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.442 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:03.442 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.331487+0000", 2020-09-08T16:07:03.442 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:03.442 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.443 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.443 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:03.443 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.331487+0000", 2020-09-08T16:07:03.443 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999743 2020-09-08T16:07:03.443 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.444 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.444 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:03.444 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.331461+0000", 2020-09-08T16:07:03.444 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7122999999999998e-05 2020-09-08T16:07:03.444 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.445 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.445 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:03.445 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.331558+0000", 2020-09-08T16:07:03.445 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.38e-05 2020-09-08T16:07:03.445 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.446 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.446 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:03.446 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.331572+0000", 2020-09-08T16:07:03.446 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1410000000000004e-06 2020-09-08T16:07:03.446 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.447 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.447 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:03.447 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.331577+0000", 2020-09-08T16:07:03.447 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.378500324 2020-09-08T16:07:03.447 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.448 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.448 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:03.448 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.710077+0000", 2020-09-08T16:07:03.448 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000109687 2020-09-08T16:07:03.448 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.448 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.449 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:03.449 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.710187+0000", 2020-09-08T16:07:03.449 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018271100000000001 2020-09-08T16:07:03.449 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.449 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.450 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:07:03.450 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.710369+0000", 2020-09-08T16:07:03.450 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.023681298999999999 2020-09-08T16:07:03.450 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.450 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.451 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:03.451 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.734051+0000", 2020-09-08T16:07:03.451 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.098485091999999996 2020-09-08T16:07:03.451 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.451 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.452 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:03.452 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.832536+0000", 2020-09-08T16:07:03.452 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5665000000000001e-05 2020-09-08T16:07:03.452 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.452 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.452 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:03.453 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.832612+0000", 2020-09-08T16:07:03.453 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9141000000000003e-05 2020-09-08T16:07:03.453 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:03.453 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:03.453 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:03.454 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.832661+0000", 2020-09-08T16:07:03.454 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.50117400099999998 2020-09-08T16:07:03.454 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.454 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.454 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.455 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:03.455 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:03.455 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T16:07:03.456 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:03.457 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:03 smithi104 bash[9203]: cluster 2020-09-08T16:07:02.066692+0000 mgr.x (mgr.24118) 1191 : cluster [DBG] pgmap v1772: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 716 op/s 2020-09-08T16:07:03.457 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:07:03 smithi104 bash[15273]: debug 2020-09-08T16:07:03.159+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:03.558 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:03.620 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:03 smithi166 bash[9446]: cluster 2020-09-08T16:07:02.066692+0000 mgr.x (mgr.24118) 1191 : cluster [DBG] pgmap v1772: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 716 op/s 2020-09-08T16:07:03.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:03 smithi055 bash[9296]: cluster 2020-09-08T16:07:02.066692+0000 mgr.x (mgr.24118) 1191 : cluster [DBG] pgmap v1772: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 716 op/s 2020-09-08T16:07:03.874 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:07:03 smithi104 bash[15273]: debug 2020-09-08T16:07:03.875+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:03.875 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:03.979 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:04.128 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:04.231 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:07:04.258 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:04 smithi055 bash[9296]: cluster 2020-09-08T16:07:02.852978+0000 osd.11 (osd.11) 115 : cluster [DBG] 8.3 scrub ok 2020-09-08T16:07:04.259 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:07:04 smithi055 bash[16646]: debug 2020-09-08T16:07:04.127+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:04.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:04 smithi166 bash[9446]: cluster 2020-09-08T16:07:02.852978+0000 osd.11 (osd.11) 115 : cluster [DBG] 8.3 scrub ok 2020-09-08T16:07:04.673 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:07:04.686 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:04 smithi104 bash[9203]: cluster 2020-09-08T16:07:02.852978+0000 osd.11 (osd.11) 115 : cluster [DBG] 8.3 scrub ok 2020-09-08T16:07:04.778 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:04.943 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:07:04 smithi104 bash[17588]: debug 2020-09-08T16:07:04.675+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:04.960 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2020-09-08T16:07:04.982 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:05.086 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:05.111 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:07:05.263 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:07:04 smithi104 bash[15273]: debug 2020-09-08T16:07:04.983+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:05.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:05 smithi166 bash[9446]: cluster 2020-09-08T16:07:04.067162+0000 mgr.x (mgr.24118) 1192 : cluster [DBG] pgmap v1773: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 967 op/s 2020-09-08T16:07:05.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:05 smithi166 bash[9446]: cluster 2020-09-08T16:07:04.656457+0000 osd.10 (osd.10) 90 : cluster [DBG] 8.9 scrub starts 2020-09-08T16:07:05.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:05 smithi055 bash[9296]: cluster 2020-09-08T16:07:04.067162+0000 mgr.x (mgr.24118) 1192 : cluster [DBG] pgmap v1773: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 967 op/s 2020-09-08T16:07:05.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:05 smithi055 bash[9296]: cluster 2020-09-08T16:07:04.656457+0000 osd.10 (osd.10) 90 : cluster [DBG] 8.9 scrub starts 2020-09-08T16:07:05.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:05 smithi104 bash[9203]: cluster 2020-09-08T16:07:04.067162+0000 mgr.x (mgr.24118) 1192 : cluster [DBG] pgmap v1773: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 967 op/s 2020-09-08T16:07:05.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:05 smithi104 bash[9203]: cluster 2020-09-08T16:07:04.656457+0000 osd.10 (osd.10) 90 : cluster [DBG] 8.9 scrub starts 2020-09-08T16:07:05.877 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:05.980 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:07:06.177 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:07:05 smithi104 bash[15273]: debug 2020-09-08T16:07:05.875+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:06.263 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:06 smithi055 bash[9296]: audit 2020-09-08T16:07:05.829468+0000 mon.a (mon.0) 2125 : audit [INF] from='client.? v1:172.21.15.104:0/963010727' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:07:06.275 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T16:07:06.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:06 smithi166 bash[9446]: audit 2020-09-08T16:07:05.829468+0000 mon.a (mon.0) 2125 : audit [INF] from='client.? v1:172.21.15.104:0/963010727' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:07:06.655 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:06 smithi104 bash[9203]: audit 2020-09-08T16:07:05.829468+0000 mon.a (mon.0) 2125 : audit [INF] from='client.? v1:172.21.15.104:0/963010727' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:07:06.840 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:07:06.944 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:07.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:07:06 smithi055 bash[11956]: debug 2020-09-08T16:07:06.839+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:07.185 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:07.293 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:07:07.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:07:07 smithi166 bash[18960]: debug 2020-09-08T16:07:07.184+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:07.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:07 smithi166 bash[9446]: cluster 2020-09-08T16:07:06.067968+0000 mgr.x (mgr.24118) 1193 : cluster [DBG] pgmap v1774: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 951 op/s 2020-09-08T16:07:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:07 smithi166 bash[9446]: cluster 2020-09-08T16:07:06.265202+0000 mon.a (mon.0) 2126 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:07:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:07 smithi166 bash[9446]: cluster 2020-09-08T16:07:06.265232+0000 mon.a (mon.0) 2127 : cluster [INF] Cluster is now healthy 2020-09-08T16:07:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:07 smithi166 bash[9446]: audit 2020-09-08T16:07:06.276962+0000 mon.a (mon.0) 2128 : audit [INF] from='client.? v1:172.21.15.104:0/963010727' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:07:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:07 smithi166 bash[9446]: cluster 2020-09-08T16:07:06.277008+0000 mon.a (mon.0) 2129 : cluster [DBG] osdmap e666: 12 total, 12 up, 12 in 2020-09-08T16:07:07.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:07 smithi055 bash[9296]: cluster 2020-09-08T16:07:06.067968+0000 mgr.x (mgr.24118) 1193 : cluster [DBG] pgmap v1774: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 951 op/s 2020-09-08T16:07:07.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:07 smithi055 bash[9296]: cluster 2020-09-08T16:07:06.265202+0000 mon.a (mon.0) 2126 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:07:07.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:07 smithi055 bash[9296]: cluster 2020-09-08T16:07:06.265232+0000 mon.a (mon.0) 2127 : cluster [INF] Cluster is now healthy 2020-09-08T16:07:07.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:07 smithi055 bash[9296]: audit 2020-09-08T16:07:06.276962+0000 mon.a (mon.0) 2128 : audit [INF] from='client.? v1:172.21.15.104:0/963010727' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:07:07.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:07 smithi055 bash[9296]: cluster 2020-09-08T16:07:06.277008+0000 mon.a (mon.0) 2129 : cluster [DBG] osdmap e666: 12 total, 12 up, 12 in 2020-09-08T16:07:07.692 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:07 smithi104 bash[9203]: cluster 2020-09-08T16:07:06.067968+0000 mgr.x (mgr.24118) 1193 : cluster [DBG] pgmap v1774: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 951 op/s 2020-09-08T16:07:07.692 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:07 smithi104 bash[9203]: cluster 2020-09-08T16:07:06.265202+0000 mon.a (mon.0) 2126 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:07:07.692 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:07 smithi104 bash[9203]: cluster 2020-09-08T16:07:06.265232+0000 mon.a (mon.0) 2127 : cluster [INF] Cluster is now healthy 2020-09-08T16:07:07.693 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:07 smithi104 bash[9203]: audit 2020-09-08T16:07:06.276962+0000 mon.a (mon.0) 2128 : audit [INF] from='client.? v1:172.21.15.104:0/963010727' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:07:07.693 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:07 smithi104 bash[9203]: cluster 2020-09-08T16:07:06.277008+0000 mon.a (mon.0) 2129 : cluster [DBG] osdmap e666: 12 total, 12 up, 12 in 2020-09-08T16:07:08.081 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:07:08.196 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:07:08.321 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:07:08.426 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:08.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:08 smithi104 bash[9203]: cluster 2020-09-08T16:07:07.345685+0000 osd.10 (osd.10) 91 : cluster [DBG] 8.9 scrub ok 2020-09-08T16:07:08.427 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:07:08 smithi104 bash[19922]: debug 2020-09-08T16:07:08.079+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:08.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:08 smithi166 bash[9446]: cluster 2020-09-08T16:07:07.345685+0000 osd.10 (osd.10) 91 : cluster [DBG] 8.9 scrub ok 2020-09-08T16:07:08.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:08 smithi055 bash[9296]: cluster 2020-09-08T16:07:07.345685+0000 osd.10 (osd.10) 91 : cluster [DBG] 8.9 scrub ok 2020-09-08T16:07:08.657 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:07:08 smithi055 bash[11956]: debug 2020-09-08T16:07:08.322+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:08.666 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:08.750 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:07:08 smithi104 bash[15273]: debug 2020-09-08T16:07:08.663+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:08.771 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:08.884 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:08.989 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:09.094 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:09.099 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:07:08 smithi055 bash[16646]: debug 2020-09-08T16:07:08.882+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:09.099 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:07:09 smithi055 bash[16646]: debug 2020-09-08T16:07:09.094+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:09.200 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:09.204 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:07:09.237 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:07:09.279 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:07:09.311 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:07:09.343 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:07:09.377 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:07:09.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:09 smithi055 bash[9296]: cluster 2020-09-08T16:07:08.068475+0000 mgr.x (mgr.24118) 1194 : cluster [DBG] pgmap v1776: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T16:07:09.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:09 smithi055 bash[9296]: audit 2020-09-08T16:07:09.172138+0000 mon.a (mon.0) 2130 : audit [DBG] from='client.? v1:172.21.15.104:0/3319006021' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:07:09.407 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:07:09.441 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:07:09.470 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:07:09.510 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:07:09.539 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:07:09.578 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:07:09.580 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:07:09.580 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:07:09.580 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:07:09.581 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:07:09.581 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:07:09.581 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:07:09.581 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:07:09.581 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:07:09.582 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:07:09.582 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:07:09.582 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:07:09.582 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:07:09.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:09 smithi166 bash[9446]: cluster 2020-09-08T16:07:08.068475+0000 mgr.x (mgr.24118) 1194 : cluster [DBG] pgmap v1776: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T16:07:09.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:09 smithi166 bash[9446]: audit 2020-09-08T16:07:09.172138+0000 mon.a (mon.0) 2130 : audit [DBG] from='client.? v1:172.21.15.104:0/3319006021' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:07:09.665 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:09 smithi104 bash[9203]: cluster 2020-09-08T16:07:08.068475+0000 mgr.x (mgr.24118) 1194 : cluster [DBG] pgmap v1776: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T16:07:09.665 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:09 smithi104 bash[9203]: audit 2020-09-08T16:07:09.172138+0000 mon.a (mon.0) 2130 : audit [DBG] from='client.? v1:172.21.15.104:0/3319006021' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:07:09.999 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:10.103 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:10.290 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:10 smithi104 bash[9203]: cluster 2020-09-08T16:07:08.872798+0000 osd.9 (osd.9) 101 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:10.291 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:10 smithi104 bash[9203]: cluster 2020-09-08T16:07:08.872882+0000 osd.9 (osd.9) 102 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:10.291 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:10 smithi104 bash[9203]: cluster 2020-09-08T16:07:09.284392+0000 mon.a (mon.0) 2131 : cluster [DBG] osdmap e667: 12 total, 12 up, 12 in 2020-09-08T16:07:10.291 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:07:10 smithi104 bash[15273]: debug 2020-09-08T16:07:09.999+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:10.292 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:10.401 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:10.479 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T16:07:10.531 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:10.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:10 smithi166 bash[9446]: cluster 2020-09-08T16:07:08.872798+0000 osd.9 (osd.9) 101 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:10.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:10 smithi166 bash[9446]: cluster 2020-09-08T16:07:08.872882+0000 osd.9 (osd.9) 102 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:10.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:10 smithi166 bash[9446]: cluster 2020-09-08T16:07:09.284392+0000 mon.a (mon.0) 2131 : cluster [DBG] osdmap e667: 12 total, 12 up, 12 in 2020-09-08T16:07:10.617 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:07:10 smithi166 bash[18960]: debug 2020-09-08T16:07:10.288+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:10.628 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:07:10.640 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:07:10.656 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:07:10 smithi055 bash[14301]: debug 2020-09-08T16:07:10.530+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:10.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:10 smithi055 bash[9296]: cluster 2020-09-08T16:07:08.872798+0000 osd.9 (osd.9) 101 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:10.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:10 smithi055 bash[9296]: cluster 2020-09-08T16:07:08.872882+0000 osd.9 (osd.9) 102 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:10.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:10 smithi055 bash[9296]: cluster 2020-09-08T16:07:09.284392+0000 mon.a (mon.0) 2131 : cluster [DBG] osdmap e667: 12 total, 12 up, 12 in 2020-09-08T16:07:10.755 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:07:10.866 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:11.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:07:10 smithi166 bash[14254]: debug 2020-09-08T16:07:10.752+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:11.304 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:11.409 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:11.520 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:11.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:11 smithi166 bash[9446]: cluster 2020-09-08T16:07:10.068941+0000 mgr.x (mgr.24118) 1195 : cluster [DBG] pgmap v1778: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.07k op/s 2020-09-08T16:07:11.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:11 smithi166 bash[9446]: cluster 2020-09-08T16:07:10.468558+0000 osd.2 (osd.2) 106 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:11.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:11 smithi166 bash[9446]: cluster 2020-09-08T16:07:10.468671+0000 osd.2 (osd.2) 107 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:11.628 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:11.636 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:11 smithi104 bash[9203]: cluster 2020-09-08T16:07:10.068941+0000 mgr.x (mgr.24118) 1195 : cluster [DBG] pgmap v1778: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.07k op/s 2020-09-08T16:07:11.636 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:11 smithi104 bash[9203]: cluster 2020-09-08T16:07:10.468558+0000 osd.2 (osd.2) 106 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:11.636 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:11 smithi104 bash[9203]: cluster 2020-09-08T16:07:10.468671+0000 osd.2 (osd.2) 107 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:11.637 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:07:11 smithi104 bash[15273]: debug 2020-09-08T16:07:11.307+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:11.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:07:11 smithi055 bash[16646]: debug 2020-09-08T16:07:11.518+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:11.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:11 smithi055 bash[9296]: cluster 2020-09-08T16:07:10.068941+0000 mgr.x (mgr.24118) 1195 : cluster [DBG] pgmap v1778: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.07k op/s 2020-09-08T16:07:11.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:11 smithi055 bash[9296]: cluster 2020-09-08T16:07:10.468558+0000 osd.2 (osd.2) 106 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:11.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:11 smithi055 bash[9296]: cluster 2020-09-08T16:07:10.468671+0000 osd.2 (osd.2) 107 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:11.808 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:11.912 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:11.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:07:11 smithi104 bash[22305]: debug 2020-09-08T16:07:11.807+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:12.318 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:12.419 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:12.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:12 smithi166 bash[9446]: cluster 2020-09-08T16:07:11.300571+0000 mon.a (mon.0) 2132 : cluster [DBG] osdmap e668: 12 total, 12 up, 12 in 2020-09-08T16:07:12.617 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:07:12 smithi166 bash[16593]: debug 2020-09-08T16:07:12.312+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:12.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:12 smithi055 bash[9296]: cluster 2020-09-08T16:07:11.300571+0000 mon.a (mon.0) 2132 : cluster [DBG] osdmap e668: 12 total, 12 up, 12 in 2020-09-08T16:07:12.779 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:12 smithi104 bash[9203]: cluster 2020-09-08T16:07:11.300571+0000 mon.a (mon.0) 2132 : cluster [DBG] osdmap e668: 12 total, 12 up, 12 in 2020-09-08T16:07:12.887 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:12.992 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:07:13.145 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:07:13.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:07:12 smithi104 bash[15273]: debug 2020-09-08T16:07:12.891+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:13.248 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:07:13.255 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:07:13 smithi166 bash[14254]: debug 2020-09-08T16:07:13.144+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:13.401 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:07:13.504 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:13.607 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:13 smithi104 bash[9203]: cluster 2020-09-08T16:07:12.069461+0000 mgr.x (mgr.24118) 1196 : cluster [DBG] pgmap v1780: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 942 op/s 2020-09-08T16:07:13.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:13 smithi166 bash[9446]: cluster 2020-09-08T16:07:12.069461+0000 mgr.x (mgr.24118) 1196 : cluster [DBG] pgmap v1780: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 942 op/s 2020-09-08T16:07:13.616 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:07:13 smithi166 bash[14254]: debug 2020-09-08T16:07:13.396+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:13.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:13 smithi055 bash[9296]: cluster 2020-09-08T16:07:12.069461+0000 mgr.x (mgr.24118) 1196 : cluster [DBG] pgmap v1780: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 6.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 942 op/s 2020-09-08T16:07:13.746 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T16:07:13.746 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:07:13.768 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:07:13.791 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:13.896 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:13.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:07:13 smithi104 bash[22305]: debug 2020-09-08T16:07:13.795+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:13.928 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:07:14.030 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:14.138 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:14.320 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:14 smithi055 bash[9296]: cluster 2020-09-08T16:07:13.185119+0000 osd.7 (osd.7) 143 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:07:14.321 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:14 smithi055 bash[9296]: cluster 2020-09-08T16:07:13.324886+0000 osd.7 (osd.7) 144 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:07:14.321 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:07:14 smithi055 bash[14301]: debug 2020-09-08T16:07:14.026+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:14.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:14 smithi104 bash[9203]: cluster 2020-09-08T16:07:13.185119+0000 osd.7 (osd.7) 143 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:07:14.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:14 smithi104 bash[9203]: cluster 2020-09-08T16:07:13.324886+0000 osd.7 (osd.7) 144 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:07:14.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:14 smithi166 bash[9446]: cluster 2020-09-08T16:07:13.185119+0000 osd.7 (osd.7) 143 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:07:14.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:14 smithi166 bash[9446]: cluster 2020-09-08T16:07:13.324886+0000 osd.7 (osd.7) 144 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:07:15.285 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:15.390 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:15.499 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:15.605 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:07:15.613 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:15 smithi104 bash[9203]: audit 2020-09-08T16:07:14.949873+0000 mon.a (mon.0) 2133 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:07:15.613 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:15 smithi104 bash[9203]: audit 2020-09-08T16:07:14.952665+0000 mon.a (mon.0) 2134 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:07:15.614 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:07:15 smithi104 bash[15273]: debug 2020-09-08T16:07:15.287+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:15.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:15 smithi166 bash[9446]: audit 2020-09-08T16:07:14.949873+0000 mon.a (mon.0) 2133 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:07:15.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:15 smithi166 bash[9446]: audit 2020-09-08T16:07:14.952665+0000 mon.a (mon.0) 2134 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:07:15.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:07:15 smithi055 bash[16646]: debug 2020-09-08T16:07:15.498+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:15.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:15 smithi055 bash[9296]: audit 2020-09-08T16:07:14.949873+0000 mon.a (mon.0) 2133 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:07:15.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:15 smithi055 bash[9296]: audit 2020-09-08T16:07:14.952665+0000 mon.a (mon.0) 2134 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:07:15.701 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2020-09-08T16:07:15.897 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:07:16.362 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:07:16.467 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:16.684 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:07:16 smithi104 bash[17588]: debug 2020-09-08T16:07:16.363+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:17.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:17 smithi166 bash[9446]: cluster 2020-09-08T16:07:16.070486+0000 mgr.x (mgr.24118) 1198 : cluster [DBG] pgmap v1782: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 755 op/s 2020-09-08T16:07:17.633 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:17 smithi104 bash[9203]: cluster 2020-09-08T16:07:16.070486+0000 mgr.x (mgr.24118) 1198 : cluster [DBG] pgmap v1782: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 755 op/s 2020-09-08T16:07:17.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:17 smithi055 bash[9296]: cluster 2020-09-08T16:07:16.070486+0000 mgr.x (mgr.24118) 1198 : cluster [DBG] pgmap v1782: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 755 op/s 2020-09-08T16:07:18.332 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:18 smithi104 bash[9203]: cluster 2020-09-08T16:07:17.354681+0000 osd.7 (osd.7) 145 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:18.333 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:18 smithi104 bash[9203]: cluster 2020-09-08T16:07:17.354803+0000 osd.7 (osd.7) 146 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:18.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:18 smithi166 bash[9446]: cluster 2020-09-08T16:07:17.354681+0000 osd.7 (osd.7) 145 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:18.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:18 smithi166 bash[9446]: cluster 2020-09-08T16:07:17.354803+0000 osd.7 (osd.7) 146 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:18.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:18 smithi055 bash[9296]: cluster 2020-09-08T16:07:17.354681+0000 osd.7 (osd.7) 145 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:18.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:18 smithi055 bash[9296]: cluster 2020-09-08T16:07:17.354803+0000 osd.7 (osd.7) 146 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:19.352 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T16:07:19.622 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:19 smithi166 bash[9446]: cluster 2020-09-08T16:07:18.071032+0000 mgr.x (mgr.24118) 1199 : cluster [DBG] pgmap v1783: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 898 op/s 2020-09-08T16:07:19.623 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:19 smithi166 bash[9446]: cluster 2020-09-08T16:07:18.341655+0000 mon.a (mon.0) 2135 : cluster [DBG] osdmap e669: 12 total, 12 up, 12 in 2020-09-08T16:07:19.623 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:19 smithi166 bash[9446]: cluster 2020-09-08T16:07:18.525794+0000 osd.0 (osd.0) 120 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:19.623 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:19 smithi166 bash[9446]: cluster 2020-09-08T16:07:18.525873+0000 osd.0 (osd.0) 121 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:19.623 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:19 smithi166 bash[9446]: audit 2020-09-08T16:07:18.544643+0000 mon.a (mon.0) 2136 : audit [INF] from='client.? v1:172.21.15.104:0/1344358175' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:07:19.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:19 smithi055 bash[9296]: cluster 2020-09-08T16:07:18.071032+0000 mgr.x (mgr.24118) 1199 : cluster [DBG] pgmap v1783: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 898 op/s 2020-09-08T16:07:19.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:19 smithi055 bash[9296]: cluster 2020-09-08T16:07:18.341655+0000 mon.a (mon.0) 2135 : cluster [DBG] osdmap e669: 12 total, 12 up, 12 in 2020-09-08T16:07:19.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:19 smithi055 bash[9296]: cluster 2020-09-08T16:07:18.525794+0000 osd.0 (osd.0) 120 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:19.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:19 smithi055 bash[9296]: cluster 2020-09-08T16:07:18.525873+0000 osd.0 (osd.0) 121 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:19.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:19 smithi055 bash[9296]: audit 2020-09-08T16:07:18.544643+0000 mon.a (mon.0) 2136 : audit [INF] from='client.? v1:172.21.15.104:0/1344358175' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:07:19.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:19 smithi104 bash[9203]: cluster 2020-09-08T16:07:18.071032+0000 mgr.x (mgr.24118) 1199 : cluster [DBG] pgmap v1783: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 898 op/s 2020-09-08T16:07:19.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:19 smithi104 bash[9203]: cluster 2020-09-08T16:07:18.341655+0000 mon.a (mon.0) 2135 : cluster [DBG] osdmap e669: 12 total, 12 up, 12 in 2020-09-08T16:07:19.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:19 smithi104 bash[9203]: cluster 2020-09-08T16:07:18.525794+0000 osd.0 (osd.0) 120 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:19.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:19 smithi104 bash[9203]: cluster 2020-09-08T16:07:18.525873+0000 osd.0 (osd.0) 121 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:19.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:19 smithi104 bash[9203]: audit 2020-09-08T16:07:18.544643+0000 mon.a (mon.0) 2136 : audit [INF] from='client.? v1:172.21.15.104:0/1344358175' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:07:20.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:20 smithi166 bash[9446]: cluster 2020-09-08T16:07:19.337047+0000 mon.a (mon.0) 2137 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:07:20.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:20 smithi166 bash[9446]: audit 2020-09-08T16:07:19.351283+0000 mon.a (mon.0) 2138 : audit [INF] from='client.? v1:172.21.15.104:0/1344358175' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:07:20.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:20 smithi166 bash[9446]: cluster 2020-09-08T16:07:19.351334+0000 mon.a (mon.0) 2139 : cluster [DBG] osdmap e670: 12 total, 12 up, 12 in 2020-09-08T16:07:20.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:20 smithi055 bash[9296]: cluster 2020-09-08T16:07:19.337047+0000 mon.a (mon.0) 2137 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:07:20.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:20 smithi055 bash[9296]: audit 2020-09-08T16:07:19.351283+0000 mon.a (mon.0) 2138 : audit [INF] from='client.? v1:172.21.15.104:0/1344358175' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:07:20.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:20 smithi055 bash[9296]: cluster 2020-09-08T16:07:19.351334+0000 mon.a (mon.0) 2139 : cluster [DBG] osdmap e670: 12 total, 12 up, 12 in 2020-09-08T16:07:20.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:20 smithi104 bash[9203]: cluster 2020-09-08T16:07:19.337047+0000 mon.a (mon.0) 2137 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:07:20.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:20 smithi104 bash[9203]: audit 2020-09-08T16:07:19.351283+0000 mon.a (mon.0) 2138 : audit [INF] from='client.? v1:172.21.15.104:0/1344358175' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:07:20.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:20 smithi104 bash[9203]: cluster 2020-09-08T16:07:19.351334+0000 mon.a (mon.0) 2139 : cluster [DBG] osdmap e670: 12 total, 12 up, 12 in 2020-09-08T16:07:21.620 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:21 smithi166 bash[9446]: cluster 2020-09-08T16:07:20.071471+0000 mgr.x (mgr.24118) 1200 : cluster [DBG] pgmap v1786: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T16:07:21.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:21 smithi055 bash[9296]: cluster 2020-09-08T16:07:20.071471+0000 mgr.x (mgr.24118) 1200 : cluster [DBG] pgmap v1786: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T16:07:21.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:21 smithi104 bash[9203]: cluster 2020-09-08T16:07:20.071471+0000 mgr.x (mgr.24118) 1200 : cluster [DBG] pgmap v1786: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T16:07:21.853 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:22.011 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:22.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:07:21 smithi166 bash[16593]: debug 2020-09-08T16:07:21.852+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:22.270 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:22.274 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:07:22 smithi104 bash[15273]: debug 2020-09-08T16:07:22.271+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:22.376 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:07:22.444 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:07:22.469 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:07:22.498 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:07:22.524 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:07:22.555 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:07:22 smithi104 bash[19922]: debug 2020-09-08T16:07:22.559+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:22.556 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:07:22.556 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:07:22.587 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:07:22.620 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:07:22.650 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:07:22.660 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:07:22.682 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:07:22.716 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:07:22.751 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:07:22.794 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:07:22.802 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:07:22.802 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:07:22.802 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:07:22.803 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:07:22.803 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:07:22.803 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:07:22.803 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:07:22.803 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:07:22.804 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:07:22.804 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:07:22.804 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:07:22.804 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:07:22.943 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T16:07:22.943 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T16:07:22.943 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2020-09-08T16:07:22.944 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2020-09-08T16:07:22.944 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T16:07:23.238 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:07:23.358 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:23.476 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:23.520 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T16:07:23.579 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:23.587 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:23 smithi166 bash[9446]: cluster 2020-09-08T16:07:22.071975+0000 mgr.x (mgr.24118) 1201 : cluster [DBG] pgmap v1787: 17 pgs: 17 active+clean; 5.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 667 op/s 2020-09-08T16:07:23.587 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:23 smithi166 bash[9446]: audit 2020-09-08T16:07:22.414350+0000 mon.a (mon.0) 2140 : audit [DBG] from='client.? v1:172.21.15.104:0/3610566043' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:07:23.587 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:07:23 smithi166 bash[14254]: debug 2020-09-08T16:07:23.239+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:23.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:23 smithi055 bash[9296]: cluster 2020-09-08T16:07:22.071975+0000 mgr.x (mgr.24118) 1201 : cluster [DBG] pgmap v1787: 17 pgs: 17 active+clean; 5.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 667 op/s 2020-09-08T16:07:23.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:23 smithi055 bash[9296]: audit 2020-09-08T16:07:22.414350+0000 mon.a (mon.0) 2140 : audit [DBG] from='client.? v1:172.21.15.104:0/3610566043' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:07:23.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:07:23 smithi055 bash[14301]: debug 2020-09-08T16:07:23.470+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:23.668 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:07:23.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:23 smithi104 bash[9203]: cluster 2020-09-08T16:07:22.071975+0000 mgr.x (mgr.24118) 1201 : cluster [DBG] pgmap v1787: 17 pgs: 17 active+clean; 5.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 667 op/s 2020-09-08T16:07:23.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:23 smithi104 bash[9203]: audit 2020-09-08T16:07:22.414350+0000 mon.a (mon.0) 2140 : audit [DBG] from='client.? v1:172.21.15.104:0/3610566043' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:07:23.711 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:23.814 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:07:23.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:07:23 smithi166 bash[16593]: debug 2020-09-08T16:07:23.711+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:24.422 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:07:24.529 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:07:24.673 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:07:24.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:07:24 smithi104 bash[17588]: debug 2020-09-08T16:07:24.423+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:24.778 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:07:24.906 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:07:24 smithi055 bash[19003]: debug 2020-09-08T16:07:24.674+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:24.906 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:07:24.990 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2020-09-08T16:07:25.008 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:07:25.147 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:07:25.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:07:24 smithi055 bash[19003]: debug 2020-09-08T16:07:24.902+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:25.232 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:07:25.338 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:25.341 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:07:25 smithi104 bash[17588]: debug 2020-09-08T16:07:25.227+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:25.650 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:25 smithi166 bash[9446]: cluster 2020-09-08T16:07:24.072681+0000 mgr.x (mgr.24118) 1202 : cluster [DBG] pgmap v1788: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T16:07:25.663 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:25 smithi055 bash[9296]: cluster 2020-09-08T16:07:24.072681+0000 mgr.x (mgr.24118) 1202 : cluster [DBG] pgmap v1788: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T16:07:25.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:25 smithi104 bash[9203]: cluster 2020-09-08T16:07:24.072681+0000 mgr.x (mgr.24118) 1202 : cluster [DBG] pgmap v1788: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T16:07:25.775 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:25.876 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:07:26.026 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:07:26.124 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:07:25 smithi104 bash[22305]: debug 2020-09-08T16:07:25.771+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:26.130 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:07:26.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:07:26 smithi166 bash[14254]: debug 2020-09-08T16:07:26.023+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:26.838 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:07:26.940 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:27.180 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:07:26 smithi104 bash[17588]: debug 2020-09-08T16:07:26.811+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:27.228 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:07:27.376 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:07:27.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:27 smithi055 bash[9296]: cluster 2020-09-08T16:07:26.073248+0000 mgr.x (mgr.24118) 1203 : cluster [DBG] pgmap v1789: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 722 op/s 2020-09-08T16:07:27.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:27 smithi104 bash[9203]: cluster 2020-09-08T16:07:26.073248+0000 mgr.x (mgr.24118) 1203 : cluster [DBG] pgmap v1789: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 722 op/s 2020-09-08T16:07:27.749 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:27 smithi166 bash[9446]: cluster 2020-09-08T16:07:26.073248+0000 mgr.x (mgr.24118) 1203 : cluster [DBG] pgmap v1789: 17 pgs: 17 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 722 op/s 2020-09-08T16:07:27.784 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:27.888 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:28.034 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:28.118 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:07:27 smithi166 bash[18960]: debug 2020-09-08T16:07:27.783+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:28.140 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:07:28.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:07:28 smithi055 bash[16646]: debug 2020-09-08T16:07:28.030+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:28.709 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:07:28.814 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:07:29.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:07:28 smithi104 bash[19922]: debug 2020-09-08T16:07:28.711+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:29.393 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:07:29.507 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:29.620 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:29 smithi166 bash[9446]: cluster 2020-09-08T16:07:28.073803+0000 mgr.x (mgr.24118) 1204 : cluster [DBG] pgmap v1790: 17 pgs: 17 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 942 op/s 2020-09-08T16:07:29.621 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:29 smithi166 bash[9446]: cluster 2020-09-08T16:07:28.532830+0000 osd.5 (osd.5) 111 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:29.621 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:29 smithi166 bash[9446]: cluster 2020-09-08T16:07:28.532988+0000 osd.5 (osd.5) 112 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:29.638 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:29 smithi055 bash[9296]: cluster 2020-09-08T16:07:28.073803+0000 mgr.x (mgr.24118) 1204 : cluster [DBG] pgmap v1790: 17 pgs: 17 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 942 op/s 2020-09-08T16:07:29.638 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:29 smithi055 bash[9296]: cluster 2020-09-08T16:07:28.532830+0000 osd.5 (osd.5) 111 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:29.638 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:29 smithi055 bash[9296]: cluster 2020-09-08T16:07:28.532988+0000 osd.5 (osd.5) 112 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:29.640 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:29.745 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:07:29.857 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:07:29.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:07:29 smithi055 bash[11956]: debug 2020-09-08T16:07:29.850+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:29.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:07:29 smithi055 bash[14301]: debug 2020-09-08T16:07:29.638+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:29.959 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:30.032 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:07:29 smithi104 bash[17588]: debug 2020-09-08T16:07:29.395+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:30.032 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:29 smithi104 bash[9203]: cluster 2020-09-08T16:07:28.073803+0000 mgr.x (mgr.24118) 1204 : cluster [DBG] pgmap v1790: 17 pgs: 17 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 942 op/s 2020-09-08T16:07:30.032 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:29 smithi104 bash[9203]: cluster 2020-09-08T16:07:28.532830+0000 osd.5 (osd.5) 111 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:30.033 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:29 smithi104 bash[9203]: cluster 2020-09-08T16:07:28.532988+0000 osd.5 (osd.5) 112 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:30.064 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:30.172 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:30.356 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:07:30 smithi055 bash[14301]: debug 2020-09-08T16:07:30.062+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:30.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:30 smithi166 bash[9446]: cluster 2020-09-08T16:07:29.355694+0000 mon.a (mon.0) 2141 : cluster [DBG] osdmap e671: 12 total, 12 up, 12 in 2020-09-08T16:07:30.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:30 smithi055 bash[9296]: cluster 2020-09-08T16:07:29.355694+0000 mon.a (mon.0) 2141 : cluster [DBG] osdmap e671: 12 total, 12 up, 12 in 2020-09-08T16:07:30.715 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:30 smithi104 bash[9203]: cluster 2020-09-08T16:07:29.355694+0000 mon.a (mon.0) 2141 : cluster [DBG] osdmap e671: 12 total, 12 up, 12 in 2020-09-08T16:07:30.790 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:30.895 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:31.012 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:31.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:07:30 smithi166 bash[16593]: debug 2020-09-08T16:07:30.791+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:31.119 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:31.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:07:31 smithi055 bash[14301]: debug 2020-09-08T16:07:31.010+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:31.252 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:31.358 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:31.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:07:31 smithi166 bash[16593]: debug 2020-09-08T16:07:31.251+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:31.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:31 smithi166 bash[9446]: cluster 2020-09-08T16:07:30.074247+0000 mgr.x (mgr.24118) 1205 : cluster [DBG] pgmap v1792: 17 pgs: 17 active+clean; 5.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 913 op/s 2020-09-08T16:07:31.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:31 smithi166 bash[9446]: cluster 2020-09-08T16:07:30.399411+0000 osd.10 (osd.10) 92 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:31.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:31 smithi166 bash[9446]: cluster 2020-09-08T16:07:30.399508+0000 osd.10 (osd.10) 93 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:31.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:31 smithi055 bash[9296]: cluster 2020-09-08T16:07:30.074247+0000 mgr.x (mgr.24118) 1205 : cluster [DBG] pgmap v1792: 17 pgs: 17 active+clean; 5.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 913 op/s 2020-09-08T16:07:31.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:31 smithi055 bash[9296]: cluster 2020-09-08T16:07:30.399411+0000 osd.10 (osd.10) 92 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:31.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:31 smithi055 bash[9296]: cluster 2020-09-08T16:07:30.399508+0000 osd.10 (osd.10) 93 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:31.672 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:31 smithi104 bash[9203]: cluster 2020-09-08T16:07:30.074247+0000 mgr.x (mgr.24118) 1205 : cluster [DBG] pgmap v1792: 17 pgs: 17 active+clean; 5.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 913 op/s 2020-09-08T16:07:31.672 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:31 smithi104 bash[9203]: cluster 2020-09-08T16:07:30.399411+0000 osd.10 (osd.10) 92 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:31.672 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:31 smithi104 bash[9203]: cluster 2020-09-08T16:07:30.399508+0000 osd.10 (osd.10) 93 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:31.714 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:31.832 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:31.932 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:07:31 smithi104 bash[22305]: debug 2020-09-08T16:07:31.715+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:32.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:32 smithi055 bash[9296]: cluster 2020-09-08T16:07:31.372419+0000 mon.a (mon.0) 2142 : cluster [DBG] osdmap e672: 12 total, 12 up, 12 in 2020-09-08T16:07:32.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:32 smithi104 bash[9203]: cluster 2020-09-08T16:07:31.372419+0000 mon.a (mon.0) 2142 : cluster [DBG] osdmap e672: 12 total, 12 up, 12 in 2020-09-08T16:07:32.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:32 smithi166 bash[9446]: cluster 2020-09-08T16:07:31.372419+0000 mon.a (mon.0) 2142 : cluster [DBG] osdmap e672: 12 total, 12 up, 12 in 2020-09-08T16:07:33.385 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T16:07:33.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:33 smithi055 bash[9296]: cluster 2020-09-08T16:07:32.074708+0000 mgr.x (mgr.24118) 1206 : cluster [DBG] pgmap v1794: 17 pgs: 17 active+clean; 5.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 663 op/s 2020-09-08T16:07:33.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:33 smithi055 bash[9296]: audit 2020-09-08T16:07:32.549164+0000 mon.a (mon.0) 2143 : audit [INF] from='client.? v1:172.21.15.104:0/1920622974' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:07:33.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:33 smithi055 bash[9296]: cluster 2020-09-08T16:07:32.866160+0000 osd.3 (osd.3) 117 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:33.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:33 smithi055 bash[9296]: cluster 2020-09-08T16:07:32.866245+0000 osd.3 (osd.3) 118 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:33.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:33 smithi104 bash[9203]: cluster 2020-09-08T16:07:32.074708+0000 mgr.x (mgr.24118) 1206 : cluster [DBG] pgmap v1794: 17 pgs: 17 active+clean; 5.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 663 op/s 2020-09-08T16:07:33.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:33 smithi104 bash[9203]: audit 2020-09-08T16:07:32.549164+0000 mon.a (mon.0) 2143 : audit [INF] from='client.? v1:172.21.15.104:0/1920622974' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:07:33.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:33 smithi104 bash[9203]: cluster 2020-09-08T16:07:32.866160+0000 osd.3 (osd.3) 117 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:33.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:33 smithi104 bash[9203]: cluster 2020-09-08T16:07:32.866245+0000 osd.3 (osd.3) 118 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:33.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:33 smithi166 bash[9446]: cluster 2020-09-08T16:07:32.074708+0000 mgr.x (mgr.24118) 1206 : cluster [DBG] pgmap v1794: 17 pgs: 17 active+clean; 5.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 663 op/s 2020-09-08T16:07:33.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:33 smithi166 bash[9446]: audit 2020-09-08T16:07:32.549164+0000 mon.a (mon.0) 2143 : audit [INF] from='client.? v1:172.21.15.104:0/1920622974' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:07:33.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:33 smithi166 bash[9446]: cluster 2020-09-08T16:07:32.866160+0000 osd.3 (osd.3) 117 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:33.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:33 smithi166 bash[9446]: cluster 2020-09-08T16:07:32.866245+0000 osd.3 (osd.3) 118 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:33.922 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:34.027 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:34.164 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:34.271 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:34.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:07:34 smithi166 bash[18960]: debug 2020-09-08T16:07:34.163+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:34.372 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:07:33 smithi055 bash[16646]: debug 2020-09-08T16:07:33.922+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:34.373 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:34 smithi055 bash[9296]: cluster 2020-09-08T16:07:33.375030+0000 mon.a (mon.0) 2144 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:07:34.373 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:34 smithi055 bash[9296]: audit 2020-09-08T16:07:33.388096+0000 mon.a (mon.0) 2145 : audit [INF] from='client.? v1:172.21.15.104:0/1920622974' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:07:34.460 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T16:07:34.460 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2020-09-08T16:07:34.460 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2020-09-08T16:07:34.461 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T16:07:34.461 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.461 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:202698 6.e 6:782a9619:::benchmark_data_smithi166_43187_object82929:head [delete] snapc 0=[] ondisk+write+known_if_redirected e536)", 2020-09-08T16:07:34.461 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:59:30.430177+0000", 2020-09-08T16:07:34.462 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 484.02969762800001, 2020-09-08T16:07:34.462 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.22294625900000001, 2020-09-08T16:07:34.463 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:34.463 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:34.464 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:34.464 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:07:34.464 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:07:34.465 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202698 2020-09-08T16:07:34.465 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.466 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:34.467 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.467 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:34.479 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.430177+0000", 2020-09-08T16:07:34.480 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:34.480 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.480 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.480 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:34.481 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.430177+0000", 2020-09-08T16:07:34.481 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999585 2020-09-08T16:07:34.481 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.481 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.481 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:34.482 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.430136+0000", 2020-09-08T16:07:34.482 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8554000000000006e-05 2020-09-08T16:07:34.482 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.482 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.483 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:34.483 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.430224+0000", 2020-09-08T16:07:34.483 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3008e-05 2020-09-08T16:07:34.483 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.483 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.484 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:34.484 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.430237+0000", 2020-09-08T16:07:34.484 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0540000000000002e-06 2020-09-08T16:07:34.484 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.484 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.485 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:34.485 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.430242+0000", 2020-09-08T16:07:34.486 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4307e-05 2020-09-08T16:07:34.486 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.486 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.486 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:34.486 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.430296+0000", 2020-09-08T16:07:34.487 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8831000000000004e-05 2020-09-08T16:07:34.487 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.488 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.488 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:34.489 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.430385+0000", 2020-09-08T16:07:34.489 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5544000000000006e-05 2020-09-08T16:07:34.489 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.489 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.489 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:07:34.490 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.430471+0000", 2020-09-08T16:07:34.490 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0053805620000000002 2020-09-08T16:07:34.490 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.490 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.490 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:34.491 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.435851+0000", 2020-09-08T16:07:34.491 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21708075399999999 2020-09-08T16:07:34.491 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.491 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.491 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:34.492 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.652932+0000", 2020-09-08T16:07:34.492 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000106551 2020-09-08T16:07:34.492 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.492 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.492 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:34.492 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.653039+0000", 2020-09-08T16:07:34.493 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4392000000000001e-05 2020-09-08T16:07:34.493 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.493 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.493 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:34.493 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.653123+0000", 2020-09-08T16:07:34.494 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.22294625900000001 2020-09-08T16:07:34.494 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.494 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.494 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.494 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.495 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.495 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:202714 6.e 6:7adb235e:::benchmark_data_smithi166_43187_object82945:head [delete] snapc 0=[] ondisk+write+known_if_redirected e536)", 2020-09-08T16:07:34.495 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:59:30.446298+0000", 2020-09-08T16:07:34.496 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 484.01357640399999, 2020-09-08T16:07:34.496 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21457531099999999, 2020-09-08T16:07:34.496 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:34.496 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:34.496 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:34.497 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:07:34.497 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:07:34.497 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202714 2020-09-08T16:07:34.497 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.497 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:34.498 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.498 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:34.498 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.446298+0000", 2020-09-08T16:07:34.498 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:34.499 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.499 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.499 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:34.499 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.446298+0000", 2020-09-08T16:07:34.500 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999958 2020-09-08T16:07:34.500 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.500 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.501 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:34.501 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.446256+0000", 2020-09-08T16:07:34.502 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3294999999999994e-05 2020-09-08T16:07:34.502 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.502 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.502 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:34.503 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.446339+0000", 2020-09-08T16:07:34.503 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3199e-05 2020-09-08T16:07:34.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.503 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.504 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:34.504 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.446353+0000", 2020-09-08T16:07:34.505 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5919999999999997e-06 2020-09-08T16:07:34.505 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.505 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.505 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:34.506 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.446359+0000", 2020-09-08T16:07:34.506 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9181000000000002e-05 2020-09-08T16:07:34.506 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.506 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.507 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:34.507 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.446408+0000", 2020-09-08T16:07:34.507 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017045999999999999 2020-09-08T16:07:34.507 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.507 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.508 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:34.508 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.446579+0000", 2020-09-08T16:07:34.508 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017048700000000001 2020-09-08T16:07:34.508 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.509 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.509 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:07:34.509 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.446749+0000", 2020-09-08T16:07:34.509 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.005460688 2020-09-08T16:07:34.509 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.510 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.510 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:34.510 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.452210+0000", 2020-09-08T16:07:34.510 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.208480316 2020-09-08T16:07:34.510 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.511 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.511 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:34.511 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.660690+0000", 2020-09-08T16:07:34.511 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000109472 2020-09-08T16:07:34.512 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.512 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.512 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:34.512 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.660800+0000", 2020-09-08T16:07:34.512 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3552000000000003e-05 2020-09-08T16:07:34.513 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.513 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.513 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:34.513 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.660873+0000", 2020-09-08T16:07:34.513 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21457531099999999 2020-09-08T16:07:34.514 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.514 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.514 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.514 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.514 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.515 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:202719 6.e 6:77f77f1f:::benchmark_data_smithi166_43187_object82950:head [delete] snapc 0=[] ondisk+write+known_if_redirected e536)", 2020-09-08T16:07:34.515 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:59:30.456871+0000", 2020-09-08T16:07:34.515 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 484.00300310699998, 2020-09-08T16:07:34.515 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21736169999999999, 2020-09-08T16:07:34.515 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:34.516 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:34.516 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:34.516 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:07:34.516 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:07:34.516 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202719 2020-09-08T16:07:34.517 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.517 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:34.517 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.517 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:34.518 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.456871+0000", 2020-09-08T16:07:34.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:34.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:34.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.456871+0000", 2020-09-08T16:07:34.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999762 2020-09-08T16:07:34.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:34.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.456848+0000", 2020-09-08T16:07:34.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2540000000000002e-05 2020-09-08T16:07:34.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:34.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.456900+0000", 2020-09-08T16:07:34.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9653999999999999e-05 2020-09-08T16:07:34.521 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.522 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.522 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:34.522 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.456920+0000", 2020-09-08T16:07:34.522 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1890000000000001e-06 2020-09-08T16:07:34.522 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:34.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.456927+0000", 2020-09-08T16:07:34.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0541999999999998e-05 2020-09-08T16:07:34.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.524 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:34.524 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.456978+0000", 2020-09-08T16:07:34.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000162318 2020-09-08T16:07:34.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:34.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.457140+0000", 2020-09-08T16:07:34.525 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000103735 2020-09-08T16:07:34.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:07:34.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.457244+0000", 2020-09-08T16:07:34.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0046529809999999996 2020-09-08T16:07:34.527 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.527 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.527 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:34.527 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.461897+0000", 2020-09-08T16:07:34.527 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21201972299999999 2020-09-08T16:07:34.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.528 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:34.528 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.673916+0000", 2020-09-08T16:07:34.528 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018826699999999999 2020-09-08T16:07:34.529 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.529 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.529 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:34.529 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.674105+0000", 2020-09-08T16:07:34.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000128472 2020-09-08T16:07:34.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:34.530 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.674233+0000", 2020-09-08T16:07:34.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21736169999999999 2020-09-08T16:07:34.531 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.531 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.531 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.532 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:202729 6.e 6:73babc97:::benchmark_data_smithi166_43187_object82960:head [delete] snapc 0=[] ondisk+write+known_if_redirected e536)", 2020-09-08T16:07:34.532 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:59:30.464418+0000", 2020-09-08T16:07:34.532 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 483.99545664599998, 2020-09-08T16:07:34.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21018515800000001, 2020-09-08T16:07:34.533 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:34.533 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:34.533 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:34.533 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:07:34.534 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:07:34.534 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202729 2020-09-08T16:07:34.534 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.535 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:34.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:34.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.464418+0000", 2020-09-08T16:07:34.536 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:34.536 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.536 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.536 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:34.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.464418+0000", 2020-09-08T16:07:34.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999766 2020-09-08T16:07:34.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.539 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:34.539 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.464394+0000", 2020-09-08T16:07:34.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1405e-05 2020-09-08T16:07:34.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:34.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.464446+0000", 2020-09-08T16:07:34.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4700000000000008e-06 2020-09-08T16:07:34.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:34.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.464455+0000", 2020-09-08T16:07:34.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8520000000000003e-06 2020-09-08T16:07:34.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.543 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:34.543 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.464460+0000", 2020-09-08T16:07:34.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1064999999999998e-05 2020-09-08T16:07:34.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:34.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.464531+0000", 2020-09-08T16:07:34.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000117853 2020-09-08T16:07:34.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:34.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.464649+0000", 2020-09-08T16:07:34.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000111267 2020-09-08T16:07:34.546 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.546 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:07:34.547 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.464760+0000", 2020-09-08T16:07:34.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0048323619999999998 2020-09-08T16:07:34.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:34.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.469593+0000", 2020-09-08T16:07:34.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.20474008399999999 2020-09-08T16:07:34.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:34.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.674333+0000", 2020-09-08T16:07:34.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015335500000000001 2020-09-08T16:07:34.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.550 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.550 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:34.550 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.674486+0000", 2020-09-08T16:07:34.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000116796 2020-09-08T16:07:34.551 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.551 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.551 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:34.551 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.674603+0000", 2020-09-08T16:07:34.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.21018515800000001 2020-09-08T16:07:34.552 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.552 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.552 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.553 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:202752 6.e 6:7b6c0d15:::benchmark_data_smithi166_43187_object82983:head [delete] snapc 0=[] ondisk+write+known_if_redirected e536)", 2020-09-08T16:07:34.553 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:59:30.487656+0000", 2020-09-08T16:07:34.553 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 483.97221851199998, 2020-09-08T16:07:34.554 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.19728971300000001, 2020-09-08T16:07:34.554 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:34.554 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:34.554 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:34.554 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:07:34.554 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:07:34.555 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202752 2020-09-08T16:07:34.555 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.555 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:34.555 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.556 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:34.556 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.487656+0000", 2020-09-08T16:07:34.556 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:34.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.557 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.557 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:34.557 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.487656+0000", 2020-09-08T16:07:34.557 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999557 2020-09-08T16:07:34.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.558 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:34.558 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.487611+0000", 2020-09-08T16:07:34.559 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5141000000000003e-05 2020-09-08T16:07:34.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.559 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:34.559 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.487707+0000", 2020-09-08T16:07:34.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7898999999999999e-05 2020-09-08T16:07:34.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.560 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:34.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.487725+0000", 2020-09-08T16:07:34.561 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0089999999999996e-06 2020-09-08T16:07:34.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.561 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.562 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:34.562 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.487734+0000", 2020-09-08T16:07:34.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5980999999999998e-05 2020-09-08T16:07:34.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:34.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.487790+0000", 2020-09-08T16:07:34.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016943500000000001 2020-09-08T16:07:34.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.564 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:34.564 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.487959+0000", 2020-09-08T16:07:34.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017985700000000001 2020-09-08T16:07:34.565 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.565 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.565 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:07:34.566 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.488139+0000", 2020-09-08T16:07:34.566 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0058527539999999999 2020-09-08T16:07:34.566 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.566 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.566 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:34.567 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.493992+0000", 2020-09-08T16:07:34.567 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.19080227399999999 2020-09-08T16:07:34.567 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.567 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.568 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:34.568 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.684794+0000", 2020-09-08T16:07:34.568 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7337999999999995e-05 2020-09-08T16:07:34.568 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.569 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.569 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:34.569 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.684891+0000", 2020-09-08T16:07:34.569 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4496000000000001e-05 2020-09-08T16:07:34.570 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.570 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.570 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:34.570 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.684946+0000", 2020-09-08T16:07:34.571 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.19728971300000001 2020-09-08T16:07:34.571 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.571 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.571 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.571 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.572 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.572 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:202764 6.e 6:796d5322:::benchmark_data_smithi166_43187_object82995:head [delete] snapc 0=[] ondisk+write+known_if_redirected e536)", 2020-09-08T16:07:34.572 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:59:30.507673+0000", 2020-09-08T16:07:34.572 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 483.95220166600001, 2020-09-08T16:07:34.573 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.177583359, 2020-09-08T16:07:34.573 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:34.573 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:34.573 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:34.574 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:07:34.574 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:07:34.574 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202764 2020-09-08T16:07:34.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.574 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:34.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:34.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.507673+0000", 2020-09-08T16:07:34.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:34.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.576 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:34.576 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.507673+0000", 2020-09-08T16:07:34.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999132 2020-09-08T16:07:34.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:34.578 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.507586+0000", 2020-09-08T16:07:34.578 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018901000000000001 2020-09-08T16:07:34.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.578 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.579 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:34.579 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.507775+0000", 2020-09-08T16:07:34.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9556e-05 2020-09-08T16:07:34.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.579 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:34.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.507804+0000", 2020-09-08T16:07:34.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1733e-05 2020-09-08T16:07:34.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.581 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.581 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:34.581 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.507826+0000", 2020-09-08T16:07:34.581 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1586000000000001e-05 2020-09-08T16:07:34.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:34.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.507908+0000", 2020-09-08T16:07:34.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00021437999999999999 2020-09-08T16:07:34.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:34.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.508122+0000", 2020-09-08T16:07:34.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00023220599999999999 2020-09-08T16:07:34.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.585 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:07:34.585 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.508354+0000", 2020-09-08T16:07:34.585 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0063759740000000004 2020-09-08T16:07:34.585 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.586 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:34.586 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.514730+0000", 2020-09-08T16:07:34.586 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17041372299999999 2020-09-08T16:07:34.587 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.587 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.587 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:34.587 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.685144+0000", 2020-09-08T16:07:34.588 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2648000000000001e-05 2020-09-08T16:07:34.588 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.588 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.588 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:34.589 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.685207+0000", 2020-09-08T16:07:34.589 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9456000000000002e-05 2020-09-08T16:07:34.589 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.589 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.590 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:34.590 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.685256+0000", 2020-09-08T16:07:34.590 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.177583359 2020-09-08T16:07:34.591 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.591 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.591 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.591 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.592 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.592 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:202780 6.e 6:75f96cf7:::benchmark_data_smithi166_43187_object83011:head [delete] snapc 0=[] ondisk+write+known_if_redirected e536)", 2020-09-08T16:07:34.592 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:59:30.525922+0000", 2020-09-08T16:07:34.592 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 483.93395213100001, 2020-09-08T16:07:34.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.159758544, 2020-09-08T16:07:34.593 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:34.593 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:34.593 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:34.594 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:07:34.594 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:07:34.594 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202780 2020-09-08T16:07:34.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.595 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:34.595 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.595 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:34.595 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.525922+0000", 2020-09-08T16:07:34.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:34.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:34.597 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.525922+0000", 2020-09-08T16:07:34.597 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999604 2020-09-08T16:07:34.597 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.597 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:34.598 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.525883+0000", 2020-09-08T16:07:34.598 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2145999999999995e-05 2020-09-08T16:07:34.598 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.598 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.598 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:34.599 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.525975+0000", 2020-09-08T16:07:34.599 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7166000000000001e-05 2020-09-08T16:07:34.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.599 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:34.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.525992+0000", 2020-09-08T16:07:34.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9959999999999999e-06 2020-09-08T16:07:34.600 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:34.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.526001+0000", 2020-09-08T16:07:34.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8238e-05 2020-09-08T16:07:34.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:34.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.526059+0000", 2020-09-08T16:07:34.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015962699999999999 2020-09-08T16:07:34.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:34.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.526219+0000", 2020-09-08T16:07:34.604 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017547999999999999 2020-09-08T16:07:34.604 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:07:34.604 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.526394+0000", 2020-09-08T16:07:34.605 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010704174 2020-09-08T16:07:34.605 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.605 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.605 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:34.606 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.537098+0000", 2020-09-08T16:07:34.606 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14845812 2020-09-08T16:07:34.606 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.606 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.606 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:34.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.685556+0000", 2020-09-08T16:07:34.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4846999999999997e-05 2020-09-08T16:07:34.607 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.607 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:34.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.685631+0000", 2020-09-08T16:07:34.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9558999999999998e-05 2020-09-08T16:07:34.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:34.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.685681+0000", 2020-09-08T16:07:34.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.159758544 2020-09-08T16:07:34.609 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.609 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.610 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.610 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.610 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.610 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.16014.0:202781 6.e 6:792632b1:::benchmark_data_smithi166_43187_object83012:head [delete] snapc 0=[] ondisk+write+known_if_redirected e536)", 2020-09-08T16:07:34.610 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T15:59:30.527485+0000", 2020-09-08T16:07:34.611 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 483.93238990100002, 2020-09-08T16:07:34.611 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.158781687, 2020-09-08T16:07:34.611 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:34.611 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:34.612 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:34.612 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.16014", 2020-09-08T16:07:34.612 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2453820557", 2020-09-08T16:07:34.612 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202781 2020-09-08T16:07:34.612 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.613 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:34.613 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.613 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:34.613 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.527485+0000", 2020-09-08T16:07:34.614 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:34.614 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.614 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:34.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.527485+0000", 2020-09-08T16:07:34.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999609 2020-09-08T16:07:34.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.615 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:34.616 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.527445+0000", 2020-09-08T16:07:34.617 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5767999999999996e-05 2020-09-08T16:07:34.617 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.617 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.617 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:34.618 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.527531+0000", 2020-09-08T16:07:34.618 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6487999999999999e-05 2020-09-08T16:07:34.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.618 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.618 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:34.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.527548+0000", 2020-09-08T16:07:34.619 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7090000000000008e-06 2020-09-08T16:07:34.619 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.620 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:34.620 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.527556+0000", 2020-09-08T16:07:34.620 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0035586020000000001 2020-09-08T16:07:34.621 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:34.624 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.531115+0000", 2020-09-08T16:07:34.624 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015404399999999999 2020-09-08T16:07:34.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.625 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:34.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.531269+0000", 2020-09-08T16:07:34.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000164427 2020-09-08T16:07:34.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4", 2020-09-08T16:07:34.627 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.531434+0000", 2020-09-08T16:07:34.627 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0060429339999999998 2020-09-08T16:07:34.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.628 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:34.628 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.537476+0000", 2020-09-08T16:07:34.628 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.148598647 2020-09-08T16:07:34.629 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.629 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:34.629 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.686075+0000", 2020-09-08T16:07:34.629 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000102934 2020-09-08T16:07:34.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.630 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.631 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:34.631 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.686178+0000", 2020-09-08T16:07:34.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8232999999999994e-05 2020-09-08T16:07:34.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.632 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.632 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:34.632 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T15:59:30.686266+0000", 2020-09-08T16:07:34.632 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.158781687 2020-09-08T16:07:34.633 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.633 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.633 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.633 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.634 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:132315 7.b 7:ddb8020b:::benchmark_data_smithi166_48533_object132314:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e582)", 2020-09-08T16:07:34.634 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:02:08.120129+0000", 2020-09-08T16:07:34.634 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 326.33974560600001, 2020-09-08T16:07:34.634 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.33257541400000001, 2020-09-08T16:07:34.635 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:34.636 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:34.636 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:34.636 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:07:34.636 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:07:34.636 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 132315 2020-09-08T16:07:34.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.637 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:34.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.637 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:34.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.120129+0000", 2020-09-08T16:07:34.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:34.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.638 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:34.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.120129+0000", 2020-09-08T16:07:34.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999533 2020-09-08T16:07:34.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:34.639 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.120082+0000", 2020-09-08T16:07:34.640 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013359799999999999 2020-09-08T16:07:34.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.640 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:34.640 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.120216+0000", 2020-09-08T16:07:34.641 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3717999999999999e-05 2020-09-08T16:07:34.641 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.641 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.641 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:34.641 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.120239+0000", 2020-09-08T16:07:34.642 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1121000000000001e-05 2020-09-08T16:07:34.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:34.642 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.120250+0000", 2020-09-08T16:07:34.643 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0232000000000001e-05 2020-09-08T16:07:34.643 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.643 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.643 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:34.643 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.120321+0000", 2020-09-08T16:07:34.644 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020806700000000001 2020-09-08T16:07:34.644 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.644 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.644 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:34.644 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.120529+0000", 2020-09-08T16:07:34.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000203279 2020-09-08T16:07:34.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.645 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 5", 2020-09-08T16:07:34.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.120732+0000", 2020-09-08T16:07:34.646 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0051322019999999998 2020-09-08T16:07:34.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.646 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:34.646 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.125864+0000", 2020-09-08T16:07:34.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.32670120800000002 2020-09-08T16:07:34.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:34.647 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.452565+0000", 2020-09-08T16:07:34.648 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3814999999999997e-05 2020-09-08T16:07:34.648 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.648 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.648 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:34.648 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.452639+0000", 2020-09-08T16:07:34.648 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5105999999999996e-05 2020-09-08T16:07:34.649 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.649 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.649 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:34.649 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.452704+0000", 2020-09-08T16:07:34.650 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.33257541400000001 2020-09-08T16:07:34.650 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.650 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.650 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.650 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.651 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.651 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:132337 7.b 7:dbdb4841:::benchmark_data_smithi166_48533_object132336:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e582)", 2020-09-08T16:07:34.651 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:02:08.141196+0000", 2020-09-08T16:07:34.651 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 326.31867882900002, 2020-09-08T16:07:34.651 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.31203498000000002, 2020-09-08T16:07:34.651 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:34.652 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:34.652 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:34.652 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:07:34.652 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:07:34.652 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 132337 2020-09-08T16:07:34.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.653 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:34.653 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.653 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:34.653 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.141196+0000", 2020-09-08T16:07:34.654 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:34.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:34.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.141196+0000", 2020-09-08T16:07:34.655 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999566 2020-09-08T16:07:34.655 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.655 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.655 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:34.655 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.141152+0000", 2020-09-08T16:07:34.656 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013531700000000001 2020-09-08T16:07:34.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.656 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.656 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:34.656 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.141287+0000", 2020-09-08T16:07:34.657 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1562e-05 2020-09-08T16:07:34.657 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.657 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.657 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:34.657 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.141299+0000", 2020-09-08T16:07:34.658 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1059999999999996e-06 2020-09-08T16:07:34.658 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.658 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.658 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:34.658 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.141305+0000", 2020-09-08T16:07:34.659 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9994999999999996e-05 2020-09-08T16:07:34.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.659 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.659 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:34.659 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.141375+0000", 2020-09-08T16:07:34.660 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000211397 2020-09-08T16:07:34.660 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.660 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.660 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:34.660 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.141586+0000", 2020-09-08T16:07:34.661 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018713399999999999 2020-09-08T16:07:34.661 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.661 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.661 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 5", 2020-09-08T16:07:34.661 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.141774+0000", 2020-09-08T16:07:34.662 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0049451019999999998 2020-09-08T16:07:34.662 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.662 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.662 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:34.662 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.146719+0000", 2020-09-08T16:07:34.663 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.30620335100000001 2020-09-08T16:07:34.663 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.663 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.663 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:34.663 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.452922+0000", 2020-09-08T16:07:34.664 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00024726799999999998 2020-09-08T16:07:34.664 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.664 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.664 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:34.664 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.453169+0000", 2020-09-08T16:07:34.665 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1344000000000001e-05 2020-09-08T16:07:34.665 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.665 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.665 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:34.665 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.453231+0000", 2020-09-08T16:07:34.666 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.31203498000000002 2020-09-08T16:07:34.666 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.666 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.666 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.666 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.667 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.667 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:132345 7.b 7:dbb244e9:::benchmark_data_smithi166_48533_object132344:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e582)", 2020-09-08T16:07:34.667 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:02:08.150586+0000", 2020-09-08T16:07:34.667 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 326.30928866300002, 2020-09-08T16:07:34.667 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.303833462, 2020-09-08T16:07:34.668 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:34.668 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:34.668 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:34.668 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:07:34.668 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:07:34.669 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 132345 2020-09-08T16:07:34.669 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.669 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:34.669 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.669 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:34.670 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.150586+0000", 2020-09-08T16:07:34.670 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:34.670 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.670 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.670 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:34.671 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.150586+0000", 2020-09-08T16:07:34.671 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999623 2020-09-08T16:07:34.671 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.672 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.672 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:34.672 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.150548+0000", 2020-09-08T16:07:34.672 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013342200000000001 2020-09-08T16:07:34.672 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.673 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.673 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:34.673 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.150682+0000", 2020-09-08T16:07:34.673 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9137e-05 2020-09-08T16:07:34.674 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.674 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.674 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:34.674 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.150701+0000", 2020-09-08T16:07:34.674 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6990000000000003e-06 2020-09-08T16:07:34.674 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.675 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.675 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:34.675 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.150709+0000", 2020-09-08T16:07:34.675 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9232000000000003e-05 2020-09-08T16:07:34.675 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.676 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.676 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:34.676 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.150788+0000", 2020-09-08T16:07:34.676 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019832300000000001 2020-09-08T16:07:34.676 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.677 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.677 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:34.677 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.150986+0000", 2020-09-08T16:07:34.677 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019861400000000001 2020-09-08T16:07:34.677 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.678 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.678 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 5", 2020-09-08T16:07:34.678 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.151185+0000", 2020-09-08T16:07:34.678 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0074976080000000002 2020-09-08T16:07:34.678 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.679 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.679 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:34.679 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.158682+0000", 2020-09-08T16:07:34.679 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29559272199999997 2020-09-08T16:07:34.679 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.679 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.680 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:34.680 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.454275+0000", 2020-09-08T16:07:34.680 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2961000000000007e-05 2020-09-08T16:07:34.680 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.680 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.681 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:34.681 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.454368+0000", 2020-09-08T16:07:34.681 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1221999999999998e-05 2020-09-08T16:07:34.681 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.681 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.682 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:34.682 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.454419+0000", 2020-09-08T16:07:34.682 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.303833462 2020-09-08T16:07:34.682 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.682 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.683 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.683 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.683 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.683 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:132347 7.b 7:dff06c55:::benchmark_data_smithi166_48533_object132346:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e582)", 2020-09-08T16:07:34.683 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:02:08.151199+0000", 2020-09-08T16:07:34.684 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 326.30867535599998, 2020-09-08T16:07:34.684 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.30559632799999997, 2020-09-08T16:07:34.684 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:34.684 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:34.684 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:34.685 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:07:34.685 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:07:34.685 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 132347 2020-09-08T16:07:34.686 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.686 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:34.686 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.686 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:34.687 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.151199+0000", 2020-09-08T16:07:34.687 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:34.687 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.687 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.687 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:34.688 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.151199+0000", 2020-09-08T16:07:34.688 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999037 2020-09-08T16:07:34.688 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.688 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.688 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:34.689 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.151103+0000", 2020-09-08T16:07:34.689 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019263799999999999 2020-09-08T16:07:34.689 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.690 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.690 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:34.690 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.151295+0000", 2020-09-08T16:07:34.690 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5247e-05 2020-09-08T16:07:34.690 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.690 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.691 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:34.691 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.151311+0000", 2020-09-08T16:07:34.691 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9199999999999998e-06 2020-09-08T16:07:34.691 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.691 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.692 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:34.692 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.151317+0000", 2020-09-08T16:07:34.692 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003701466 2020-09-08T16:07:34.692 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.692 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.693 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:34.693 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.155019+0000", 2020-09-08T16:07:34.693 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014563600000000001 2020-09-08T16:07:34.693 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.693 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.694 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:34.694 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.155165+0000", 2020-09-08T16:07:34.694 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012374200000000001 2020-09-08T16:07:34.694 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.694 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.695 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 5", 2020-09-08T16:07:34.695 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.155288+0000", 2020-09-08T16:07:34.695 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0035475559999999999 2020-09-08T16:07:34.695 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.695 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.696 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:34.696 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.158836+0000", 2020-09-08T16:07:34.696 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29784127199999999 2020-09-08T16:07:34.696 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.696 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.697 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:34.697 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.456677+0000", 2020-09-08T16:07:34.697 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7261000000000005e-05 2020-09-08T16:07:34.697 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.697 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.698 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:34.698 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.456744+0000", 2020-09-08T16:07:34.698 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1042999999999999e-05 2020-09-08T16:07:34.701 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.701 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.701 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:34.701 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:02:08.456795+0000", 2020-09-08T16:07:34.702 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.30559632799999997 2020-09-08T16:07:34.702 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.702 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.702 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.703 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.703 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.703 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:135157 8.7 8:e24407e4:::benchmark_data_smithi166_52302_object4236:head [delete] snapc 0=[] ondisk+write+known_if_redirected e661)", 2020-09-08T16:07:34.703 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:06:41.999722+0000", 2020-09-08T16:07:34.704 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 52.460152813999997, 2020-09-08T16:07:34.704 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15919487199999999, 2020-09-08T16:07:34.704 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:34.704 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:34.704 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:34.705 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:34.705 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:34.705 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 135157 2020-09-08T16:07:34.705 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.706 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:34.706 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.706 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:34.706 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:41.999722+0000", 2020-09-08T16:07:34.706 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:34.707 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.707 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.707 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:34.707 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:41.999722+0000", 2020-09-08T16:07:34.707 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999959 2020-09-08T16:07:34.708 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.708 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.708 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:34.708 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:41.999681+0000", 2020-09-08T16:07:34.708 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2147999999999999e-05 2020-09-08T16:07:34.709 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.709 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.709 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:34.709 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:41.999773+0000", 2020-09-08T16:07:34.710 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4891e-05 2020-09-08T16:07:34.710 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.710 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.710 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:34.710 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:41.999788+0000", 2020-09-08T16:07:34.711 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0330000000000003e-06 2020-09-08T16:07:34.711 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.711 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.711 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:34.712 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:41.999795+0000", 2020-09-08T16:07:34.712 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8245000000000003e-05 2020-09-08T16:07:34.712 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.712 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.713 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:34.713 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:41.999843+0000", 2020-09-08T16:07:34.713 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000122626 2020-09-08T16:07:34.713 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.713 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.714 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:34.714 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:41.999966+0000", 2020-09-08T16:07:34.714 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000112137 2020-09-08T16:07:34.714 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.714 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.715 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T16:07:34.715 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:42.000078+0000", 2020-09-08T16:07:34.715 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.004696881 2020-09-08T16:07:34.715 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.715 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.716 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:34.716 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:42.004775+0000", 2020-09-08T16:07:34.716 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.154023195 2020-09-08T16:07:34.716 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.716 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.717 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:34.717 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:42.158798+0000", 2020-09-08T16:07:34.717 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8324000000000005e-05 2020-09-08T16:07:34.717 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.717 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.718 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:34.718 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:42.158866+0000", 2020-09-08T16:07:34.718 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.041e-05 2020-09-08T16:07:34.718 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.718 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.719 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:34.719 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:42.158917+0000", 2020-09-08T16:07:34.719 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15919487199999999 2020-09-08T16:07:34.719 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.719 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.720 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.720 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.720 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.720 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:135168 8.7 8:e522427a:::benchmark_data_smithi166_52302_object4247:head [delete] snapc 0=[] ondisk+write+known_if_redirected e661)", 2020-09-08T16:07:34.720 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:06:42.012288+0000", 2020-09-08T16:07:34.721 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 52.447586311000002, 2020-09-08T16:07:34.721 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16727240199999999, 2020-09-08T16:07:34.721 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:34.721 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:34.721 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:34.722 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:34.722 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:34.722 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 135168 2020-09-08T16:07:34.722 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.722 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:34.723 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.723 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:34.723 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:42.012288+0000", 2020-09-08T16:07:34.723 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:34.723 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.724 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.724 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:34.724 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:42.012288+0000", 2020-09-08T16:07:34.724 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999771 2020-09-08T16:07:34.724 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.725 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.725 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:34.725 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:42.012265+0000", 2020-09-08T16:07:34.725 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2030000000000002e-05 2020-09-08T16:07:34.725 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.726 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.726 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:34.726 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:42.012317+0000", 2020-09-08T16:07:34.726 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0057999999999999e-05 2020-09-08T16:07:34.726 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.727 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.727 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:34.727 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:42.012327+0000", 2020-09-08T16:07:34.727 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4650000000000002e-06 2020-09-08T16:07:34.727 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.728 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.728 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:34.728 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:42.012333+0000", 2020-09-08T16:07:34.728 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.139211628 2020-09-08T16:07:34.728 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.729 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.729 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:34.729 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:42.151544+0000", 2020-09-08T16:07:34.729 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000225835 2020-09-08T16:07:34.730 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.730 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.730 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:34.730 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:42.151770+0000", 2020-09-08T16:07:34.735 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000222372 2020-09-08T16:07:34.736 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.736 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.736 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T16:07:34.736 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:42.151993+0000", 2020-09-08T16:07:34.736 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.016358213 2020-09-08T16:07:34.737 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.737 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.737 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:34.737 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:42.168351+0000", 2020-09-08T16:07:34.737 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.011071874000000001 2020-09-08T16:07:34.738 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.738 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.738 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:34.738 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:42.179423+0000", 2020-09-08T16:07:34.738 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4827999999999995e-05 2020-09-08T16:07:34.738 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.739 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.739 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:34.739 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:42.179508+0000", 2020-09-08T16:07:34.739 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2945000000000002e-05 2020-09-08T16:07:34.739 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.740 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.740 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:34.740 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:42.179561+0000", 2020-09-08T16:07:34.740 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16727240199999999 2020-09-08T16:07:34.740 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.741 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.741 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.741 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.741 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.741 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:177558 8.7 8:e2f8d6db:::benchmark_data_smithi166_52302_object46637:head [delete] snapc 0=[] ondisk+write+known_if_redirected e670)", 2020-09-08T16:07:34.742 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:07:27.186073+0000", 2020-09-08T16:07:34.742 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 7.2738019310000004, 2020-09-08T16:07:34.742 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17110523899999999, 2020-09-08T16:07:34.742 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:34.742 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:34.743 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:34.743 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:34.743 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:34.743 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 177558 2020-09-08T16:07:34.743 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.744 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:34.744 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.744 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:34.744 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.186073+0000", 2020-09-08T16:07:34.744 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:34.744 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.745 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.745 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:34.745 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.186073+0000", 2020-09-08T16:07:34.745 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999638 2020-09-08T16:07:34.745 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.746 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.746 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:34.746 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.186036+0000", 2020-09-08T16:07:34.746 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1860999999999993e-05 2020-09-08T16:07:34.746 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.747 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.747 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:34.747 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.186128+0000", 2020-09-08T16:07:34.747 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.7452000000000001e-05 2020-09-08T16:07:34.747 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.748 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.748 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:34.748 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.186156+0000", 2020-09-08T16:07:34.748 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1012999999999999e-05 2020-09-08T16:07:34.748 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.749 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.749 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:34.749 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.186167+0000", 2020-09-08T16:07:34.749 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6610999999999998e-05 2020-09-08T16:07:34.750 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.750 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.750 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:34.751 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.186213+0000", 2020-09-08T16:07:34.751 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000110896 2020-09-08T16:07:34.751 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.751 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.752 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:34.752 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.186324+0000", 2020-09-08T16:07:34.752 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000148465 2020-09-08T16:07:34.752 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.752 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.752 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T16:07:34.753 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.186473+0000", 2020-09-08T16:07:34.753 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0093415749999999995 2020-09-08T16:07:34.753 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.753 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.753 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:34.754 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.195814+0000", 2020-09-08T16:07:34.754 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16123236899999999 2020-09-08T16:07:34.754 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.754 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.754 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:34.755 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.357046+0000", 2020-09-08T16:07:34.755 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1165000000000001e-05 2020-09-08T16:07:34.755 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.755 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.755 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:34.756 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.357118+0000", 2020-09-08T16:07:34.756 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0161000000000002e-05 2020-09-08T16:07:34.756 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.756 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.756 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:34.757 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.357178+0000", 2020-09-08T16:07:34.757 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17110523899999999 2020-09-08T16:07:34.757 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.757 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.758 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.758 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.758 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.758 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:177560 8.7 8:e606e3bd:::benchmark_data_smithi166_52302_object46639:head [delete] snapc 0=[] ondisk+write+known_if_redirected e670)", 2020-09-08T16:07:34.758 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:07:27.186236+0000", 2020-09-08T16:07:34.759 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 7.2736382339999999, 2020-09-08T16:07:34.759 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17110724899999999, 2020-09-08T16:07:34.759 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:34.759 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:34.759 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:34.760 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:34.760 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:34.760 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 177560 2020-09-08T16:07:34.760 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.760 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:34.761 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.761 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:34.761 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.186236+0000", 2020-09-08T16:07:34.761 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:34.761 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.762 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.762 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:34.762 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.186236+0000", 2020-09-08T16:07:34.762 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999647 2020-09-08T16:07:34.762 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.763 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.763 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:34.763 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.186201+0000", 2020-09-08T16:07:34.763 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.8978000000000002e-05 2020-09-08T16:07:34.763 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.764 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.764 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:34.764 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.186280+0000", 2020-09-08T16:07:34.764 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2627e-05 2020-09-08T16:07:34.764 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.765 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.765 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:34.765 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.186292+0000", 2020-09-08T16:07:34.765 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7260000000000002e-06 2020-09-08T16:07:34.765 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.766 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.766 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:34.766 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.186297+0000", 2020-09-08T16:07:34.766 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0045383300000000001 2020-09-08T16:07:34.766 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.767 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.767 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:34.767 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.190836+0000", 2020-09-08T16:07:34.767 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019442799999999999 2020-09-08T16:07:34.767 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.767 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.768 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:34.768 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.191030+0000", 2020-09-08T16:07:34.768 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014400200000000001 2020-09-08T16:07:34.768 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.768 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.769 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T16:07:34.769 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.191174+0000", 2020-09-08T16:07:34.769 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.004842697 2020-09-08T16:07:34.769 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.770 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.770 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:34.770 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.196017+0000", 2020-09-08T16:07:34.770 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16122350599999999 2020-09-08T16:07:34.770 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.771 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.771 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:34.771 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.357240+0000", 2020-09-08T16:07:34.771 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8919999999999998e-05 2020-09-08T16:07:34.771 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.772 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.772 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:34.772 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.357299+0000", 2020-09-08T16:07:34.772 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4400999999999997e-05 2020-09-08T16:07:34.772 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.773 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.773 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:34.773 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.357343+0000", 2020-09-08T16:07:34.773 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17110724899999999 2020-09-08T16:07:34.773 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.773 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.774 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.774 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.774 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.774 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:178111 8.7 8:e44fb9f9:::benchmark_data_smithi166_52302_object47190:head [delete] snapc 0=[] ondisk+write+known_if_redirected e670)", 2020-09-08T16:07:34.774 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:07:27.893393+0000", 2020-09-08T16:07:34.775 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 6.5664814009999999, 2020-09-08T16:07:34.775 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14025886600000001, 2020-09-08T16:07:34.775 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:34.775 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:34.775 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:34.776 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:34.776 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:34.776 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 178111 2020-09-08T16:07:34.776 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.776 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:34.777 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.777 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:34.777 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.893393+0000", 2020-09-08T16:07:34.777 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:34.778 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.778 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.778 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:34.778 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.893393+0000", 2020-09-08T16:07:34.779 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999619 2020-09-08T16:07:34.779 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.779 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.779 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:34.779 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.893355+0000", 2020-09-08T16:07:34.779 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4691999999999995e-05 2020-09-08T16:07:34.780 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.780 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.780 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:34.780 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.893439+0000", 2020-09-08T16:07:34.780 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6682999999999999e-05 2020-09-08T16:07:34.781 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.781 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.781 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:34.781 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.893456+0000", 2020-09-08T16:07:34.781 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9809999999999992e-06 2020-09-08T16:07:34.782 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.782 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.782 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:34.782 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.893465+0000", 2020-09-08T16:07:34.782 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9426999999999999e-05 2020-09-08T16:07:34.783 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.783 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.783 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:34.783 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.893525+0000", 2020-09-08T16:07:34.783 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016463299999999999 2020-09-08T16:07:34.784 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.784 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.784 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:34.784 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.893689+0000", 2020-09-08T16:07:34.784 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016986099999999999 2020-09-08T16:07:34.785 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.785 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.785 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T16:07:34.785 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.893859+0000", 2020-09-08T16:07:34.785 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0093005659999999997 2020-09-08T16:07:34.786 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.786 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.786 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:34.786 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.903160+0000", 2020-09-08T16:07:34.786 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.130341282 2020-09-08T16:07:34.787 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.787 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.787 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:34.787 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:28.033501+0000", 2020-09-08T16:07:34.787 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5585000000000001e-05 2020-09-08T16:07:34.788 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.788 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.788 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:34.788 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:28.033586+0000", 2020-09-08T16:07:34.788 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5492000000000002e-05 2020-09-08T16:07:34.788 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.789 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.789 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:34.789 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:28.033652+0000", 2020-09-08T16:07:34.789 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14025886600000001 2020-09-08T16:07:34.790 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.790 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.790 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.790 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.790 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.791 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:178113 8.7 8:e17362fd:::benchmark_data_smithi166_52302_object47192:head [delete] snapc 0=[] ondisk+write+known_if_redirected e670)", 2020-09-08T16:07:34.791 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:07:27.893949+0000", 2020-09-08T16:07:34.791 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 6.5659258060000001, 2020-09-08T16:07:34.791 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16223615499999999, 2020-09-08T16:07:34.791 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:34.792 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:34.792 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:34.792 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:34.792 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:34.792 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 178113 2020-09-08T16:07:34.792 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.793 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:34.793 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.793 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:34.793 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.893949+0000", 2020-09-08T16:07:34.793 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:34.794 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.794 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.794 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:34.794 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.893949+0000", 2020-09-08T16:07:34.794 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999228 2020-09-08T16:07:34.795 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.795 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.795 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:34.795 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.893871+0000", 2020-09-08T16:07:34.795 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000166458 2020-09-08T16:07:34.796 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.796 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.796 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:34.796 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.894038+0000", 2020-09-08T16:07:34.796 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.5888e-05 2020-09-08T16:07:34.797 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.797 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.797 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:34.797 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.894064+0000", 2020-09-08T16:07:34.797 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1102e-05 2020-09-08T16:07:34.798 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.798 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.798 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:34.798 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.894075+0000", 2020-09-08T16:07:34.798 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0045735430000000002 2020-09-08T16:07:34.799 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.799 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.799 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:34.799 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.898648+0000", 2020-09-08T16:07:34.799 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1261999999999995e-05 2020-09-08T16:07:34.800 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.800 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.800 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:34.800 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.898740+0000", 2020-09-08T16:07:34.800 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6179000000000002e-05 2020-09-08T16:07:34.801 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.801 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.801 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T16:07:34.801 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.898836+0000", 2020-09-08T16:07:34.801 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0098985710000000001 2020-09-08T16:07:34.802 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.802 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.802 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:34.802 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.908734+0000", 2020-09-08T16:07:34.802 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14732830699999999 2020-09-08T16:07:34.803 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.803 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.803 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:34.803 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:28.056063+0000", 2020-09-08T16:07:34.803 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8677000000000007e-05 2020-09-08T16:07:34.804 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.804 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.804 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:34.804 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:28.056131+0000", 2020-09-08T16:07:34.804 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3375000000000003e-05 2020-09-08T16:07:34.804 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.805 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.805 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:34.805 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:28.056185+0000", 2020-09-08T16:07:34.805 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16223615499999999 2020-09-08T16:07:34.806 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.806 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.806 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.806 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.806 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.807 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:178115 8.7 8:efe6a462:::benchmark_data_smithi166_52302_object47194:head [delete] snapc 0=[] ondisk+write+known_if_redirected e670)", 2020-09-08T16:07:34.807 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:07:27.894785+0000", 2020-09-08T16:07:34.807 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 6.5650899679999997, 2020-09-08T16:07:34.807 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.182127235, 2020-09-08T16:07:34.807 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:34.807 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:34.808 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:34.808 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:34.808 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:34.808 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 178115 2020-09-08T16:07:34.808 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.809 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:34.809 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.809 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:34.809 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.894785+0000", 2020-09-08T16:07:34.809 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:34.810 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.810 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.810 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:34.810 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.894785+0000", 2020-09-08T16:07:34.810 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999948 2020-09-08T16:07:34.811 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.811 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.811 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:34.811 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.894733+0000", 2020-09-08T16:07:34.811 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000105096 2020-09-08T16:07:34.811 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.812 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.812 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:34.812 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.894838+0000", 2020-09-08T16:07:34.812 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.732e-05 2020-09-08T16:07:34.813 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.813 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.813 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:34.813 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.894855+0000", 2020-09-08T16:07:34.813 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5220000000000007e-06 2020-09-08T16:07:34.814 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.814 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.814 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:34.814 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.894864+0000", 2020-09-08T16:07:34.814 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0083368000000000001 2020-09-08T16:07:34.814 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.815 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.815 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:34.815 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.903200+0000", 2020-09-08T16:07:34.815 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000156059 2020-09-08T16:07:34.815 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.816 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.816 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:34.816 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.903356+0000", 2020-09-08T16:07:34.816 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015706699999999999 2020-09-08T16:07:34.816 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.817 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.817 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T16:07:34.817 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.903513+0000", 2020-09-08T16:07:34.817 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0054212000000000002 2020-09-08T16:07:34.817 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.818 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.818 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:34.818 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:27.908935+0000", 2020-09-08T16:07:34.818 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16785159999999999 2020-09-08T16:07:34.818 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.819 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.819 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:34.819 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:28.076786+0000", 2020-09-08T16:07:34.819 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.135e-05 2020-09-08T16:07:34.819 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.820 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.820 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:34.820 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:28.076858+0000", 2020-09-08T16:07:34.820 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4107999999999998e-05 2020-09-08T16:07:34.820 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.821 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.821 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:34.821 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:28.076912+0000", 2020-09-08T16:07:34.821 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.182127235 2020-09-08T16:07:34.821 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.822 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.822 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.822 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.822 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.822 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:181040 8.1 8:8584c86e:::benchmark_data_smithi166_52302_object50119:head [delete] snapc 0=[] ondisk+write+known_if_redirected e671)", 2020-09-08T16:07:34.823 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:07:31.121999+0000", 2020-09-08T16:07:34.823 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 3.3378751599999998, 2020-09-08T16:07:34.823 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14269346299999999, 2020-09-08T16:07:34.823 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:07:34.823 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:07:34.824 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:07:34.824 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:07:34.824 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:07:34.824 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 181040 2020-09-08T16:07:34.824 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.825 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:07:34.825 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.825 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:07:34.825 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:31.121999+0000", 2020-09-08T16:07:34.825 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:07:34.826 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.826 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.826 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:07:34.826 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:31.121999+0000", 2020-09-08T16:07:34.826 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999781 2020-09-08T16:07:34.827 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.827 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.827 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:07:34.827 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:31.121977+0000", 2020-09-08T16:07:34.827 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0033e-05 2020-09-08T16:07:34.828 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.828 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.828 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:07:34.828 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:31.122027+0000", 2020-09-08T16:07:34.828 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1929999999999999e-06 2020-09-08T16:07:34.829 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.829 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.829 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:07:34.829 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:31.122036+0000", 2020-09-08T16:07:34.829 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.746e-06 2020-09-08T16:07:34.830 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.830 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.830 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:07:34.830 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:31.122041+0000", 2020-09-08T16:07:34.830 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8287000000000001e-05 2020-09-08T16:07:34.831 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.831 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.831 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:07:34.831 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:31.122079+0000", 2020-09-08T16:07:34.831 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000114252 2020-09-08T16:07:34.831 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.832 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.832 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:07:34.832 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:31.122194+0000", 2020-09-08T16:07:34.832 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011338899999999999 2020-09-08T16:07:34.832 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.833 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.833 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 9", 2020-09-08T16:07:34.833 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:31.122307+0000", 2020-09-08T16:07:34.833 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14201755999999999 2020-09-08T16:07:34.833 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.834 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.834 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:07:34.834 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:31.264325+0000", 2020-09-08T16:07:34.834 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5738999999999999e-05 2020-09-08T16:07:34.834 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.835 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.835 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:07:34.835 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:31.264420+0000", 2020-09-08T16:07:34.835 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015005500000000001 2020-09-08T16:07:34.835 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.836 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.836 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:07:34.836 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:31.264570+0000", 2020-09-08T16:07:34.836 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012238200000000001 2020-09-08T16:07:34.836 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:07:34.837 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:07:34.837 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:07:34.837 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:07:31.264693+0000", 2020-09-08T16:07:34.837 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14269346299999999 2020-09-08T16:07:34.837 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.838 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.838 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.838 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:07:34.838 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:07:34.838 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T16:07:34.840 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:34 smithi055 bash[9296]: cluster 2020-09-08T16:07:33.388148+0000 mon.a (mon.0) 2146 : cluster [DBG] osdmap e673: 12 total, 12 up, 12 in 2020-09-08T16:07:34.840 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:34 smithi104 bash[9203]: cluster 2020-09-08T16:07:33.375030+0000 mon.a (mon.0) 2144 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:07:34.841 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:34 smithi104 bash[9203]: audit 2020-09-08T16:07:33.388096+0000 mon.a (mon.0) 2145 : audit [INF] from='client.? v1:172.21.15.104:0/1920622974' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:07:34.841 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:34 smithi104 bash[9203]: cluster 2020-09-08T16:07:33.388148+0000 mon.a (mon.0) 2146 : cluster [DBG] osdmap e673: 12 total, 12 up, 12 in 2020-09-08T16:07:34.841 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:34.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:34 smithi166 bash[9446]: cluster 2020-09-08T16:07:33.375030+0000 mon.a (mon.0) 2144 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:07:34.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:34 smithi166 bash[9446]: audit 2020-09-08T16:07:33.388096+0000 mon.a (mon.0) 2145 : audit [INF] from='client.? v1:172.21.15.104:0/1920622974' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:07:34.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:34 smithi166 bash[9446]: cluster 2020-09-08T16:07:33.388148+0000 mon.a (mon.0) 2146 : cluster [DBG] osdmap e673: 12 total, 12 up, 12 in 2020-09-08T16:07:34.942 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:34.944 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:07:34 smithi104 bash[22305]: debug 2020-09-08T16:07:34.727+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:35.293 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:35.397 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:35.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:35 smithi055 bash[9296]: cluster 2020-09-08T16:07:34.075188+0000 mgr.x (mgr.24118) 1207 : cluster [DBG] pgmap v1796: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 943 op/s 2020-09-08T16:07:35.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:07:35 smithi104 bash[22305]: debug 2020-09-08T16:07:35.295+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:35.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:35 smithi104 bash[9203]: cluster 2020-09-08T16:07:34.075188+0000 mgr.x (mgr.24118) 1207 : cluster [DBG] pgmap v1796: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 943 op/s 2020-09-08T16:07:35.719 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:35.827 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:07:35.835 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:35 smithi166 bash[9446]: cluster 2020-09-08T16:07:34.075188+0000 mgr.x (mgr.24118) 1207 : cluster [DBG] pgmap v1796: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 943 op/s 2020-09-08T16:07:35.906 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2020-09-08T16:07:35.978 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:07:36.026 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:07:35 smithi104 bash[22305]: debug 2020-09-08T16:07:35.723+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:36.074 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:07:36.083 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:36.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:07:35 smithi166 bash[14254]: debug 2020-09-08T16:07:35.979+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:36.236 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:36.310 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:07:36.342 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:36.344 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:07:36.371 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:07:36.387 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:07:36 smithi055 bash[16646]: debug 2020-09-08T16:07:36.238+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:36.394 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:07:36.425 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:07:36.463 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:07:36.484 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:07:36.495 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:36.517 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:07:36.549 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:07:36.582 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:07:36.601 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:36.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:36 smithi166 bash[9446]: audit 2020-09-08T16:07:36.284569+0000 mon.a (mon.0) 2147 : audit [DBG] from='client.? v1:172.21.15.104:0/2481859004' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:07:36.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:07:36 smithi166 bash[16593]: debug 2020-09-08T16:07:36.491+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:36.618 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:07:36.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:36 smithi055 bash[9296]: audit 2020-09-08T16:07:36.284569+0000 mon.a (mon.0) 2147 : audit [DBG] from='client.? v1:172.21.15.104:0/2481859004' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:07:36.657 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:07:36.663 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:07:36.664 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:07:36.664 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:07:36.664 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:07:36.664 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:07:36.665 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:07:36.665 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:07:36.665 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:07:36.665 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:07:36.665 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:07:36.666 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:07:36.666 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:07:36.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:36 smithi104 bash[9203]: audit 2020-09-08T16:07:36.284569+0000 mon.a (mon.0) 2147 : audit [DBG] from='client.? v1:172.21.15.104:0/2481859004' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:07:36.720 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:36.825 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:07:36.911 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T16:07:37.005 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:07:37.064 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:07:37.110 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:07:37.120 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:07:36 smithi055 bash[16646]: debug 2020-09-08T16:07:36.718+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:37.223 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:07:37.328 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:07:37.389 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:07:37 smithi055 bash[11956]: debug 2020-09-08T16:07:37.222+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:37.390 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:07:37 smithi104 bash[17588]: debug 2020-09-08T16:07:37.007+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:37.472 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:07:37.578 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:37.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:37 smithi055 bash[9296]: cluster 2020-09-08T16:07:36.075770+0000 mgr.x (mgr.24118) 1208 : cluster [DBG] pgmap v1797: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 679 op/s 2020-09-08T16:07:37.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:07:37 smithi055 bash[19003]: debug 2020-09-08T16:07:37.474+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:37.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:37 smithi104 bash[9203]: cluster 2020-09-08T16:07:36.075770+0000 mgr.x (mgr.24118) 1208 : cluster [DBG] pgmap v1797: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 679 op/s 2020-09-08T16:07:37.699 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:37.805 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:37.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:37 smithi166 bash[9446]: cluster 2020-09-08T16:07:36.075770+0000 mgr.x (mgr.24118) 1208 : cluster [DBG] pgmap v1797: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 679 op/s 2020-09-08T16:07:37.934 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:38.038 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:38.046 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:07:37 smithi055 bash[16646]: debug 2020-09-08T16:07:37.698+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:38.046 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:07:37 smithi055 bash[16646]: debug 2020-09-08T16:07:37.934+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:38.163 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:38.270 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:38.393 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:07:38 smithi055 bash[14301]: debug 2020-09-08T16:07:38.162+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:38.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:38 smithi055 bash[9296]: audit 2020-09-08T16:07:37.404744+0000 mon.b (mon.1) 117 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:07:38.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:38 smithi055 bash[9296]: audit 2020-09-08T16:07:37.405125+0000 mon.a (mon.0) 2148 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:07:38.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:38 smithi055 bash[9296]: audit 2020-09-08T16:07:37.408617+0000 mon.a (mon.0) 2149 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:07:38.667 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:38 smithi104 bash[9203]: audit 2020-09-08T16:07:37.404744+0000 mon.b (mon.1) 117 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:07:38.668 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:38 smithi104 bash[9203]: audit 2020-09-08T16:07:37.405125+0000 mon.a (mon.0) 2148 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:07:38.668 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:38 smithi104 bash[9203]: audit 2020-09-08T16:07:37.408617+0000 mon.a (mon.0) 2149 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:07:38.841 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:38.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:38 smithi166 bash[9446]: audit 2020-09-08T16:07:37.404744+0000 mon.b (mon.1) 117 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:07:38.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:38 smithi166 bash[9446]: audit 2020-09-08T16:07:37.405125+0000 mon.a (mon.0) 2148 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:07:38.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:38 smithi166 bash[9446]: audit 2020-09-08T16:07:37.408617+0000 mon.a (mon.0) 2149 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:07:38.926 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:07:38 smithi104 bash[15273]: debug 2020-09-08T16:07:38.843+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:38.946 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:39.099 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:39.203 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:07:39.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:07:39 smithi055 bash[16646]: debug 2020-09-08T16:07:39.098+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:39.418 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:07:39.526 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:39.650 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:39.716 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:39 smithi104 bash[9203]: cluster 2020-09-08T16:07:38.076417+0000 mgr.x (mgr.24118) 1209 : cluster [DBG] pgmap v1798: 17 pgs: 17 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T16:07:39.717 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:39 smithi104 bash[9203]: cluster 2020-09-08T16:07:39.163416+0000 osd.8 (osd.8) 93 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:39.717 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:39 smithi104 bash[9203]: cluster 2020-09-08T16:07:39.163503+0000 osd.8 (osd.8) 94 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:39.717 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:07:39 smithi104 bash[17588]: debug 2020-09-08T16:07:39.419+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:39.759 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:39.766 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:39 smithi055 bash[9296]: cluster 2020-09-08T16:07:38.076417+0000 mgr.x (mgr.24118) 1209 : cluster [DBG] pgmap v1798: 17 pgs: 17 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T16:07:39.766 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:39 smithi055 bash[9296]: cluster 2020-09-08T16:07:39.163416+0000 osd.8 (osd.8) 93 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:39.766 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:39 smithi055 bash[9296]: cluster 2020-09-08T16:07:39.163503+0000 osd.8 (osd.8) 94 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:39.767 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:07:39 smithi055 bash[14301]: debug 2020-09-08T16:07:39.650+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:39.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:39 smithi166 bash[9446]: cluster 2020-09-08T16:07:38.076417+0000 mgr.x (mgr.24118) 1209 : cluster [DBG] pgmap v1798: 17 pgs: 17 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.00k op/s 2020-09-08T16:07:39.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:39 smithi166 bash[9446]: cluster 2020-09-08T16:07:39.163416+0000 osd.8 (osd.8) 93 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:39.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:39 smithi166 bash[9446]: cluster 2020-09-08T16:07:39.163503+0000 osd.8 (osd.8) 94 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:39.867 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:07:39.971 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:40.101 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:07:39 smithi055 bash[14301]: debug 2020-09-08T16:07:39.866+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:40.102 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:40.205 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:40.340 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:40.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:07:40 smithi055 bash[16646]: debug 2020-09-08T16:07:40.102+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:40.447 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:40.593 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:40 smithi166 bash[9446]: cluster 2020-09-08T16:07:39.415944+0000 mon.a (mon.0) 2150 : cluster [DBG] osdmap e674: 12 total, 12 up, 12 in 2020-09-08T16:07:40.594 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:07:40 smithi166 bash[16593]: debug 2020-09-08T16:07:40.591+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:40.594 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:07:40 smithi166 bash[18960]: debug 2020-09-08T16:07:40.343+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:40.595 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:40.667 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:07:40.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:40 smithi104 bash[9203]: cluster 2020-09-08T16:07:39.415944+0000 mon.a (mon.0) 2150 : cluster [DBG] osdmap e674: 12 total, 12 up, 12 in 2020-09-08T16:07:40.703 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:07:40.713 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:40 smithi055 bash[9296]: cluster 2020-09-08T16:07:39.415944+0000 mon.a (mon.0) 2150 : cluster [DBG] osdmap e674: 12 total, 12 up, 12 in 2020-09-08T16:07:40.819 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:07:40.828 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:07:40.923 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:41.026 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:07:40 smithi055 bash[11956]: debug 2020-09-08T16:07:40.818+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:41.030 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:41.137 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:41.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:07:41 smithi055 bash[16646]: debug 2020-09-08T16:07:41.026+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:41.417 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:41 smithi104 bash[9203]: cluster 2020-09-08T16:07:40.076852+0000 mgr.x (mgr.24118) 1210 : cluster [DBG] pgmap v1800: 17 pgs: 17 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T16:07:41.511 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:41.619 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:41.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:07:41 smithi104 bash[22305]: debug 2020-09-08T16:07:41.511+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:41.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:41 smithi166 bash[9446]: cluster 2020-09-08T16:07:40.076852+0000 mgr.x (mgr.24118) 1210 : cluster [DBG] pgmap v1800: 17 pgs: 17 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T16:07:41.924 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:41 smithi055 bash[9296]: cluster 2020-09-08T16:07:40.076852+0000 mgr.x (mgr.24118) 1210 : cluster [DBG] pgmap v1800: 17 pgs: 17 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T16:07:42.034 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:42.138 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:07:42.274 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:07:42.379 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:07:42.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:07:42 smithi055 bash[19003]: debug 2020-09-08T16:07:42.274+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:42.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:07:42 smithi104 bash[15273]: debug 2020-09-08T16:07:42.031+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:42.495 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:07:42.603 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:07:42.741 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:07:42.846 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:42.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:07:42 smithi166 bash[11928]: debug 2020-09-08T16:07:42.751+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:42.906 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:07:42 smithi055 bash[19003]: debug 2020-09-08T16:07:42.494+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:43.182 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:43.285 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:43.400 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:43.505 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:07:43.514 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:07:43 smithi104 bash[22305]: debug 2020-09-08T16:07:43.183+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:43.515 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:43 smithi104 bash[9203]: cluster 2020-09-08T16:07:42.077518+0000 mgr.x (mgr.24118) 1211 : cluster [DBG] pgmap v1801: 17 pgs: 17 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 772 op/s 2020-09-08T16:07:43.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:43 smithi166 bash[9446]: cluster 2020-09-08T16:07:42.077518+0000 mgr.x (mgr.24118) 1211 : cluster [DBG] pgmap v1801: 17 pgs: 17 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 772 op/s 2020-09-08T16:07:43.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:07:43 smithi166 bash[18960]: debug 2020-09-08T16:07:43.399+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:43.713 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:07:43.817 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:07:43.826 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:07:43 smithi104 bash[19922]: debug 2020-09-08T16:07:43.715+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:43.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:43 smithi055 bash[9296]: cluster 2020-09-08T16:07:42.077518+0000 mgr.x (mgr.24118) 1211 : cluster [DBG] pgmap v1801: 17 pgs: 17 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 772 op/s 2020-09-08T16:07:44.113 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:07:44.216 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:07:44.343 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:07:44.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:07:44 smithi104 bash[19922]: debug 2020-09-08T16:07:44.111+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:44.447 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:44.558 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:07:44 smithi166 bash[11928]: debug 2020-09-08T16:07:44.339+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:44.566 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:44.604 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T16:07:44.605 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:07:44.668 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:07:44.817 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:07:44 smithi166 bash[18960]: debug 2020-09-08T16:07:44.559+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:44.818 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:07:44.925 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:45.035 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:45.123 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:07:44 smithi166 bash[11928]: debug 2020-09-08T16:07:44.819+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:45.139 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:45.274 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:45.375 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:45.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:07:45 smithi055 bash[16646]: debug 2020-09-08T16:07:45.038+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:45.430 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:07:45 smithi166 bash[16593]: debug 2020-09-08T16:07:45.267+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:45.430 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:45 smithi166 bash[9446]: cluster 2020-09-08T16:07:44.078179+0000 mgr.x (mgr.24118) 1212 : cluster [DBG] pgmap v1802: 17 pgs: 17 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 922 op/s 2020-09-08T16:07:45.430 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:45 smithi166 bash[9446]: audit 2020-09-08T16:07:45.169163+0000 mon.a (mon.0) 2151 : audit [INF] from='client.? v1:172.21.15.104:0/1056890003' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:07:45.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:45 smithi104 bash[9203]: cluster 2020-09-08T16:07:44.078179+0000 mgr.x (mgr.24118) 1212 : cluster [DBG] pgmap v1802: 17 pgs: 17 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 922 op/s 2020-09-08T16:07:45.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:45 smithi104 bash[9203]: audit 2020-09-08T16:07:45.169163+0000 mon.a (mon.0) 2151 : audit [INF] from='client.? v1:172.21.15.104:0/1056890003' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:07:45.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:45 smithi055 bash[9296]: cluster 2020-09-08T16:07:44.078179+0000 mgr.x (mgr.24118) 1212 : cluster [DBG] pgmap v1802: 17 pgs: 17 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 922 op/s 2020-09-08T16:07:45.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:45 smithi055 bash[9296]: audit 2020-09-08T16:07:45.169163+0000 mon.a (mon.0) 2151 : audit [INF] from='client.? v1:172.21.15.104:0/1056890003' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:07:46.031 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T16:07:46.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:46 smithi104 bash[9203]: cluster 2020-09-08T16:07:46.029230+0000 mon.a (mon.0) 2152 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:07:46.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:46 smithi104 bash[9203]: audit 2020-09-08T16:07:46.033341+0000 mon.a (mon.0) 2153 : audit [INF] from='client.? v1:172.21.15.104:0/1056890003' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:07:46.680 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:46 smithi104 bash[9203]: cluster 2020-09-08T16:07:46.033392+0000 mon.a (mon.0) 2154 : cluster [DBG] osdmap e675: 12 total, 12 up, 12 in 2020-09-08T16:07:46.704 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:46 smithi166 bash[9446]: cluster 2020-09-08T16:07:46.029230+0000 mon.a (mon.0) 2152 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:07:46.705 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:46 smithi166 bash[9446]: audit 2020-09-08T16:07:46.033341+0000 mon.a (mon.0) 2153 : audit [INF] from='client.? v1:172.21.15.104:0/1056890003' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:07:46.705 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:46 smithi166 bash[9446]: cluster 2020-09-08T16:07:46.033392+0000 mon.a (mon.0) 2154 : cluster [DBG] osdmap e675: 12 total, 12 up, 12 in 2020-09-08T16:07:46.705 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:07:46 smithi166 bash[16593]: debug 2020-09-08T16:07:46.703+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:46.707 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:46.786 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2020-09-08T16:07:46.808 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:07:46.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:46 smithi055 bash[9296]: cluster 2020-09-08T16:07:46.029230+0000 mon.a (mon.0) 2152 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:07:46.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:46 smithi055 bash[9296]: audit 2020-09-08T16:07:46.033341+0000 mon.a (mon.0) 2153 : audit [INF] from='client.? v1:172.21.15.104:0/1056890003' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:07:46.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:46 smithi055 bash[9296]: cluster 2020-09-08T16:07:46.033392+0000 mon.a (mon.0) 2154 : cluster [DBG] osdmap e675: 12 total, 12 up, 12 in 2020-09-08T16:07:46.933 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:07:46.944 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:07:47.049 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:47.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:07:46 smithi166 bash[11928]: debug 2020-09-08T16:07:46.943+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:47.160 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:47.264 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:47.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:07:47 smithi055 bash[16646]: debug 2020-09-08T16:07:47.162+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:47.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:47 smithi104 bash[9203]: cluster 2020-09-08T16:07:46.078676+0000 mgr.x (mgr.24118) 1213 : cluster [DBG] pgmap v1804: 17 pgs: 17 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 723 op/s 2020-09-08T16:07:47.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:47 smithi104 bash[9203]: cluster 2020-09-08T16:07:46.371596+0000 osd.11 (osd.11) 116 : cluster [DBG] 8.f scrub starts 2020-09-08T16:07:47.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:47 smithi104 bash[9203]: cluster 2020-09-08T16:07:47.237106+0000 osd.1 (osd.1) 134 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:47.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:47 smithi104 bash[9203]: cluster 2020-09-08T16:07:47.237202+0000 osd.1 (osd.1) 135 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:47.692 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:47.793 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:07:47.802 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:47 smithi055 bash[9296]: cluster 2020-09-08T16:07:46.078676+0000 mgr.x (mgr.24118) 1213 : cluster [DBG] pgmap v1804: 17 pgs: 17 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 723 op/s 2020-09-08T16:07:47.802 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:47 smithi055 bash[9296]: cluster 2020-09-08T16:07:46.371596+0000 osd.11 (osd.11) 116 : cluster [DBG] 8.f scrub starts 2020-09-08T16:07:47.802 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:47 smithi055 bash[9296]: cluster 2020-09-08T16:07:47.237106+0000 osd.1 (osd.1) 134 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:47.803 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:47 smithi055 bash[9296]: cluster 2020-09-08T16:07:47.237202+0000 osd.1 (osd.1) 135 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:47.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:47 smithi166 bash[9446]: cluster 2020-09-08T16:07:46.078676+0000 mgr.x (mgr.24118) 1213 : cluster [DBG] pgmap v1804: 17 pgs: 17 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 723 op/s 2020-09-08T16:07:47.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:47 smithi166 bash[9446]: cluster 2020-09-08T16:07:46.371596+0000 osd.11 (osd.11) 116 : cluster [DBG] 8.f scrub starts 2020-09-08T16:07:47.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:47 smithi166 bash[9446]: cluster 2020-09-08T16:07:47.237106+0000 osd.1 (osd.1) 134 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:47.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:47 smithi166 bash[9446]: cluster 2020-09-08T16:07:47.237202+0000 osd.1 (osd.1) 135 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:47.906 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:07:48.010 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:48.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:07:47 smithi055 bash[19003]: debug 2020-09-08T16:07:47.906+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:48.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:07:47 smithi104 bash[22305]: debug 2020-09-08T16:07:47.687+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:48.197 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:48.301 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:48.320 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:07:48 smithi166 bash[18960]: debug 2020-09-08T16:07:48.199+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:48.444 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:48.547 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:07:48.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:48 smithi166 bash[9446]: cluster 2020-09-08T16:07:46.871343+0000 osd.5 (osd.5) 113 : cluster [DBG] 8.4 scrub starts 2020-09-08T16:07:48.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:48 smithi166 bash[9446]: cluster 2020-09-08T16:07:47.611909+0000 osd.10 (osd.10) 94 : cluster [DBG] 8.5 scrub starts 2020-09-08T16:07:48.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:48 smithi166 bash[9446]: cluster 2020-09-08T16:07:48.036818+0000 mon.a (mon.0) 2155 : cluster [DBG] osdmap e676: 12 total, 12 up, 12 in 2020-09-08T16:07:48.617 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:07:48 smithi166 bash[18960]: debug 2020-09-08T16:07:48.443+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:48.835 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:07:48.876 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:48 smithi104 bash[9203]: cluster 2020-09-08T16:07:46.871343+0000 osd.5 (osd.5) 113 : cluster [DBG] 8.4 scrub starts 2020-09-08T16:07:48.877 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:48 smithi104 bash[9203]: cluster 2020-09-08T16:07:47.611909+0000 osd.10 (osd.10) 94 : cluster [DBG] 8.5 scrub starts 2020-09-08T16:07:48.877 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:48 smithi104 bash[9203]: cluster 2020-09-08T16:07:48.036818+0000 mon.a (mon.0) 2155 : cluster [DBG] osdmap e676: 12 total, 12 up, 12 in 2020-09-08T16:07:48.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:48 smithi055 bash[9296]: cluster 2020-09-08T16:07:46.871343+0000 osd.5 (osd.5) 113 : cluster [DBG] 8.4 scrub starts 2020-09-08T16:07:48.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:48 smithi055 bash[9296]: cluster 2020-09-08T16:07:47.611909+0000 osd.10 (osd.10) 94 : cluster [DBG] 8.5 scrub starts 2020-09-08T16:07:48.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:48 smithi055 bash[9296]: cluster 2020-09-08T16:07:48.036818+0000 mon.a (mon.0) 2155 : cluster [DBG] osdmap e676: 12 total, 12 up, 12 in 2020-09-08T16:07:48.939 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:07:49.078 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:07:49.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:07:48 smithi104 bash[19922]: debug 2020-09-08T16:07:48.839+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:49.182 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:49.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:07:49 smithi166 bash[14254]: debug 2020-09-08T16:07:49.079+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:49.407 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:49.511 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:49.672 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:49.673 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:07:49 smithi104 bash[22305]: debug 2020-09-08T16:07:49.411+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:49.674 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:49 smithi104 bash[9203]: cluster 2020-09-08T16:07:48.079159+0000 mgr.x (mgr.24118) 1214 : cluster [DBG] pgmap v1806: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 973 op/s 2020-09-08T16:07:49.674 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:49 smithi104 bash[9203]: cluster 2020-09-08T16:07:48.321106+0000 osd.11 (osd.11) 117 : cluster [DBG] 8.f scrub ok 2020-09-08T16:07:49.674 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:49 smithi104 bash[9203]: cluster 2020-09-08T16:07:48.414994+0000 osd.5 (osd.5) 114 : cluster [DBG] 8.4 scrub ok 2020-09-08T16:07:49.674 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:49 smithi104 bash[9203]: cluster 2020-09-08T16:07:48.858271+0000 osd.8 (osd.8) 95 : cluster [DBG] 8.a scrub starts 2020-09-08T16:07:49.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:49 smithi104 bash[9203]: cluster 2020-09-08T16:07:48.883552+0000 osd.1 (osd.1) 136 : cluster [DBG] 8.1 scrub starts 2020-09-08T16:07:49.774 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:49.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:49 smithi166 bash[9446]: cluster 2020-09-08T16:07:48.079159+0000 mgr.x (mgr.24118) 1214 : cluster [DBG] pgmap v1806: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 973 op/s 2020-09-08T16:07:49.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:49 smithi166 bash[9446]: cluster 2020-09-08T16:07:48.321106+0000 osd.11 (osd.11) 117 : cluster [DBG] 8.f scrub ok 2020-09-08T16:07:49.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:49 smithi166 bash[9446]: cluster 2020-09-08T16:07:48.414994+0000 osd.5 (osd.5) 114 : cluster [DBG] 8.4 scrub ok 2020-09-08T16:07:49.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:49 smithi166 bash[9446]: cluster 2020-09-08T16:07:48.858271+0000 osd.8 (osd.8) 95 : cluster [DBG] 8.a scrub starts 2020-09-08T16:07:49.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:49 smithi166 bash[9446]: cluster 2020-09-08T16:07:48.883552+0000 osd.1 (osd.1) 136 : cluster [DBG] 8.1 scrub starts 2020-09-08T16:07:49.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:07:49 smithi166 bash[18960]: debug 2020-09-08T16:07:49.667+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:49.905 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:49.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:49 smithi055 bash[9296]: cluster 2020-09-08T16:07:48.079159+0000 mgr.x (mgr.24118) 1214 : cluster [DBG] pgmap v1806: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 973 op/s 2020-09-08T16:07:49.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:49 smithi055 bash[9296]: cluster 2020-09-08T16:07:48.321106+0000 osd.11 (osd.11) 117 : cluster [DBG] 8.f scrub ok 2020-09-08T16:07:49.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:49 smithi055 bash[9296]: cluster 2020-09-08T16:07:48.414994+0000 osd.5 (osd.5) 114 : cluster [DBG] 8.4 scrub ok 2020-09-08T16:07:49.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:49 smithi055 bash[9296]: cluster 2020-09-08T16:07:48.858271+0000 osd.8 (osd.8) 95 : cluster [DBG] 8.a scrub starts 2020-09-08T16:07:49.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:49 smithi055 bash[9296]: cluster 2020-09-08T16:07:48.883552+0000 osd.1 (osd.1) 136 : cluster [DBG] 8.1 scrub starts 2020-09-08T16:07:49.925 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:07:49 smithi104 bash[22305]: debug 2020-09-08T16:07:49.907+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:50.009 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:07:50.295 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:07:50.300 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:07:50 smithi104 bash[19922]: debug 2020-09-08T16:07:50.295+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:50.359 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:07:50.396 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:07:50.400 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:50.434 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:07:50.466 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:07:50.497 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:07:50.523 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:07:50.545 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T16:07:50.557 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:07:50.587 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:07:50.609 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:07:50.632 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:07:50.661 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:07:50.678 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:07:50.687 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:07:50.689 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:07:50.689 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:07:50.689 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:07:50.690 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:07:50.690 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:07:50.696 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:07:50.696 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:07:50.697 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:07:50.697 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:07:50.697 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:07:50.697 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:07:50.697 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:07:50.698 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:50 smithi104 bash[9203]: cluster 2020-09-08T16:07:48.918722+0000 osd.10 (osd.10) 95 : cluster [DBG] 8.5 scrub ok 2020-09-08T16:07:50.698 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:50 smithi104 bash[9203]: cluster 2020-09-08T16:07:50.176651+0000 osd.1 (osd.1) 137 : cluster [DBG] 8.1 scrub ok 2020-09-08T16:07:50.699 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:50 smithi104 bash[9203]: audit 2020-09-08T16:07:50.332181+0000 mon.a (mon.0) 2156 : audit [DBG] from='client.? v1:172.21.15.104:0/2158954853' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:07:50.699 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:50 smithi104 bash[9203]: cluster 2020-09-08T16:07:50.413293+0000 osd.8 (osd.8) 96 : cluster [DBG] 8.a scrub ok 2020-09-08T16:07:50.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:50 smithi166 bash[9446]: cluster 2020-09-08T16:07:48.918722+0000 osd.10 (osd.10) 95 : cluster [DBG] 8.5 scrub ok 2020-09-08T16:07:50.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:50 smithi166 bash[9446]: cluster 2020-09-08T16:07:50.176651+0000 osd.1 (osd.1) 137 : cluster [DBG] 8.1 scrub ok 2020-09-08T16:07:50.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:50 smithi166 bash[9446]: audit 2020-09-08T16:07:50.332181+0000 mon.a (mon.0) 2156 : audit [DBG] from='client.? v1:172.21.15.104:0/2158954853' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:07:50.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:50 smithi166 bash[9446]: cluster 2020-09-08T16:07:50.413293+0000 osd.8 (osd.8) 96 : cluster [DBG] 8.a scrub ok 2020-09-08T16:07:50.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:50 smithi055 bash[9296]: cluster 2020-09-08T16:07:48.918722+0000 osd.10 (osd.10) 95 : cluster [DBG] 8.5 scrub ok 2020-09-08T16:07:50.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:50 smithi055 bash[9296]: cluster 2020-09-08T16:07:50.176651+0000 osd.1 (osd.1) 137 : cluster [DBG] 8.1 scrub ok 2020-09-08T16:07:50.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:50 smithi055 bash[9296]: audit 2020-09-08T16:07:50.332181+0000 mon.a (mon.0) 2156 : audit [DBG] from='client.? v1:172.21.15.104:0/2158954853' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:07:50.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:50 smithi055 bash[9296]: cluster 2020-09-08T16:07:50.413293+0000 osd.8 (osd.8) 96 : cluster [DBG] 8.a scrub ok 2020-09-08T16:07:51.002 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:51.123 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:51.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:07:51 smithi104 bash[22305]: debug 2020-09-08T16:07:50.999+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:51.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:51 smithi166 bash[9446]: cluster 2020-09-08T16:07:50.079653+0000 mgr.x (mgr.24118) 1215 : cluster [DBG] pgmap v1807: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 4.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 998 op/s 2020-09-08T16:07:51.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:51 smithi166 bash[9446]: cluster 2020-09-08T16:07:50.495552+0000 osd.6 (osd.6) 113 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:51.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:51 smithi166 bash[9446]: cluster 2020-09-08T16:07:50.495635+0000 osd.6 (osd.6) 114 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:51.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:51 smithi055 bash[9296]: cluster 2020-09-08T16:07:50.079653+0000 mgr.x (mgr.24118) 1215 : cluster [DBG] pgmap v1807: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 4.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 998 op/s 2020-09-08T16:07:51.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:51 smithi055 bash[9296]: cluster 2020-09-08T16:07:50.495552+0000 osd.6 (osd.6) 113 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:51.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:51 smithi055 bash[9296]: cluster 2020-09-08T16:07:50.495635+0000 osd.6 (osd.6) 114 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:51.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:51 smithi104 bash[9203]: cluster 2020-09-08T16:07:50.079653+0000 mgr.x (mgr.24118) 1215 : cluster [DBG] pgmap v1807: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 4.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 998 op/s 2020-09-08T16:07:51.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:51 smithi104 bash[9203]: cluster 2020-09-08T16:07:50.495552+0000 osd.6 (osd.6) 113 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:51.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:51 smithi104 bash[9203]: cluster 2020-09-08T16:07:50.495635+0000 osd.6 (osd.6) 114 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:52.775 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:52.775 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:52 smithi166 bash[9446]: cluster 2020-09-08T16:07:51.407441+0000 osd.11 (osd.11) 118 : cluster [DBG] 8.d scrub starts 2020-09-08T16:07:52.776 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:52 smithi166 bash[9446]: cluster 2020-09-08T16:07:51.446365+0000 mon.a (mon.0) 2157 : cluster [DBG] osdmap e677: 12 total, 12 up, 12 in 2020-09-08T16:07:52.878 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:07:52.887 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:52 smithi055 bash[9296]: cluster 2020-09-08T16:07:51.407441+0000 osd.11 (osd.11) 118 : cluster [DBG] 8.d scrub starts 2020-09-08T16:07:52.888 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:52 smithi055 bash[9296]: cluster 2020-09-08T16:07:51.446365+0000 mon.a (mon.0) 2157 : cluster [DBG] osdmap e677: 12 total, 12 up, 12 in 2020-09-08T16:07:52.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:52 smithi104 bash[9203]: cluster 2020-09-08T16:07:51.407441+0000 osd.11 (osd.11) 118 : cluster [DBG] 8.d scrub starts 2020-09-08T16:07:52.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:52 smithi104 bash[9203]: cluster 2020-09-08T16:07:51.446365+0000 mon.a (mon.0) 2157 : cluster [DBG] osdmap e677: 12 total, 12 up, 12 in 2020-09-08T16:07:52.992 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:07:53.102 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:53.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:07:52 smithi166 bash[18960]: debug 2020-09-08T16:07:52.771+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:53.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:07:52 smithi055 bash[11956]: debug 2020-09-08T16:07:52.994+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:53.447 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:53 smithi166 bash[9446]: cluster 2020-09-08T16:07:52.080057+0000 mgr.x (mgr.24118) 1216 : cluster [DBG] pgmap v1809: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 4.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 750 op/s 2020-09-08T16:07:53.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:53 smithi104 bash[9203]: cluster 2020-09-08T16:07:52.080057+0000 mgr.x (mgr.24118) 1216 : cluster [DBG] pgmap v1809: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 4.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 750 op/s 2020-09-08T16:07:53.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:53 smithi055 bash[9296]: cluster 2020-09-08T16:07:52.080057+0000 mgr.x (mgr.24118) 1216 : cluster [DBG] pgmap v1809: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 4.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 750 op/s 2020-09-08T16:07:54.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:54 smithi104 bash[9203]: cluster 2020-09-08T16:07:52.916791+0000 osd.11 (osd.11) 119 : cluster [DBG] 8.d scrub ok 2020-09-08T16:07:54.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:54 smithi104 bash[9203]: cluster 2020-09-08T16:07:53.551250+0000 osd.11 (osd.11) 120 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:54.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:54 smithi104 bash[9203]: cluster 2020-09-08T16:07:53.551334+0000 osd.11 (osd.11) 121 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:54.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:54 smithi166 bash[9446]: cluster 2020-09-08T16:07:52.916791+0000 osd.11 (osd.11) 119 : cluster [DBG] 8.d scrub ok 2020-09-08T16:07:54.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:54 smithi166 bash[9446]: cluster 2020-09-08T16:07:53.551250+0000 osd.11 (osd.11) 120 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:54.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:54 smithi166 bash[9446]: cluster 2020-09-08T16:07:53.551334+0000 osd.11 (osd.11) 121 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:54.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:54 smithi055 bash[9296]: cluster 2020-09-08T16:07:52.916791+0000 osd.11 (osd.11) 119 : cluster [DBG] 8.d scrub ok 2020-09-08T16:07:54.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:54 smithi055 bash[9296]: cluster 2020-09-08T16:07:53.551250+0000 osd.11 (osd.11) 120 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:54.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:54 smithi055 bash[9296]: cluster 2020-09-08T16:07:53.551334+0000 osd.11 (osd.11) 121 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:55.247 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T16:07:55.248 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T16:07:55.248 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2020-09-08T16:07:55.248 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2020-09-08T16:07:55.248 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T16:07:55.268 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:07:55.435 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:07:55.574 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:55 smithi104 bash[9203]: cluster 2020-09-08T16:07:54.080744+0000 mgr.x (mgr.24118) 1217 : cluster [DBG] pgmap v1810: 17 pgs: 17 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.06k op/s 2020-09-08T16:07:55.575 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:55 smithi104 bash[9203]: cluster 2020-09-08T16:07:54.417477+0000 osd.11 (osd.11) 122 : cluster [DBG] 8.6 scrub starts 2020-09-08T16:07:55.575 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:55 smithi104 bash[9203]: cluster 2020-09-08T16:07:54.460401+0000 mon.a (mon.0) 2158 : cluster [DBG] osdmap e678: 12 total, 12 up, 12 in 2020-09-08T16:07:55.575 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:55 smithi104 bash[9203]: cluster 2020-09-08T16:07:54.751623+0000 osd.9 (osd.9) 103 : cluster [DBG] 8.c scrub starts 2020-09-08T16:07:55.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:55 smithi166 bash[9446]: cluster 2020-09-08T16:07:54.080744+0000 mgr.x (mgr.24118) 1217 : cluster [DBG] pgmap v1810: 17 pgs: 17 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.06k op/s 2020-09-08T16:07:55.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:55 smithi166 bash[9446]: cluster 2020-09-08T16:07:54.417477+0000 osd.11 (osd.11) 122 : cluster [DBG] 8.6 scrub starts 2020-09-08T16:07:55.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:55 smithi166 bash[9446]: cluster 2020-09-08T16:07:54.460401+0000 mon.a (mon.0) 2158 : cluster [DBG] osdmap e678: 12 total, 12 up, 12 in 2020-09-08T16:07:55.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:55 smithi166 bash[9446]: cluster 2020-09-08T16:07:54.751623+0000 osd.9 (osd.9) 103 : cluster [DBG] 8.c scrub starts 2020-09-08T16:07:55.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:55 smithi055 bash[9296]: cluster 2020-09-08T16:07:54.080744+0000 mgr.x (mgr.24118) 1217 : cluster [DBG] pgmap v1810: 17 pgs: 17 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.06k op/s 2020-09-08T16:07:55.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:55 smithi055 bash[9296]: cluster 2020-09-08T16:07:54.417477+0000 osd.11 (osd.11) 122 : cluster [DBG] 8.6 scrub starts 2020-09-08T16:07:55.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:55 smithi055 bash[9296]: cluster 2020-09-08T16:07:54.460401+0000 mon.a (mon.0) 2158 : cluster [DBG] osdmap e678: 12 total, 12 up, 12 in 2020-09-08T16:07:55.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:55 smithi055 bash[9296]: cluster 2020-09-08T16:07:54.751623+0000 osd.9 (osd.9) 103 : cluster [DBG] 8.c scrub starts 2020-09-08T16:07:55.976 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:56.080 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:07:56.226 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:07:56.231 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:07:56 smithi166 bash[11928]: debug 2020-09-08T16:07:56.219+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:56.231 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:07:55 smithi166 bash[16593]: debug 2020-09-08T16:07:55.971+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:56.334 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:56.480 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:07:56.486 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:56 smithi166 bash[9446]: cluster 2020-09-08T16:07:55.234219+0000 osd.0 (osd.0) 122 : cluster [DBG] 8.e scrub starts 2020-09-08T16:07:56.486 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:07:56 smithi166 bash[16593]: debug 2020-09-08T16:07:56.471+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:56.586 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:07:56.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:56 smithi055 bash[9296]: cluster 2020-09-08T16:07:55.234219+0000 osd.0 (osd.0) 122 : cluster [DBG] 8.e scrub starts 2020-09-08T16:07:56.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:56 smithi104 bash[9203]: cluster 2020-09-08T16:07:55.234219+0000 osd.0 (osd.0) 122 : cluster [DBG] 8.e scrub starts 2020-09-08T16:07:57.070 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:07:57.173 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:07:57.300 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:07:57.406 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:57.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:07:57 smithi104 bash[17588]: debug 2020-09-08T16:07:57.070+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:57.465 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:07:57 smithi055 bash[19003]: debug 2020-09-08T16:07:57.302+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:57.514 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:57.621 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:57.665 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2020-09-08T16:07:57.862 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:07:57.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:57 smithi166 bash[9446]: cluster 2020-09-08T16:07:55.938560+0000 osd.9 (osd.9) 104 : cluster [DBG] 8.c scrub ok 2020-09-08T16:07:57.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:57 smithi166 bash[9446]: cluster 2020-09-08T16:07:56.067206+0000 osd.11 (osd.11) 123 : cluster [DBG] 8.6 scrub ok 2020-09-08T16:07:57.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:57 smithi166 bash[9446]: cluster 2020-09-08T16:07:56.081323+0000 mgr.x (mgr.24118) 1218 : cluster [DBG] pgmap v1812: 17 pgs: 17 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 658 op/s 2020-09-08T16:07:57.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:57 smithi166 bash[9446]: cluster 2020-09-08T16:07:56.273350+0000 osd.0 (osd.0) 123 : cluster [DBG] 8.e scrub ok 2020-09-08T16:07:57.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:57 smithi166 bash[9446]: cluster 2020-09-08T16:07:56.934812+0000 osd.8 (osd.8) 97 : cluster [DBG] 8.b scrub starts 2020-09-08T16:07:57.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:07:57 smithi055 bash[16646]: debug 2020-09-08T16:07:57.518+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:57.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:57 smithi055 bash[9296]: cluster 2020-09-08T16:07:55.938560+0000 osd.9 (osd.9) 104 : cluster [DBG] 8.c scrub ok 2020-09-08T16:07:57.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:57 smithi055 bash[9296]: cluster 2020-09-08T16:07:56.067206+0000 osd.11 (osd.11) 123 : cluster [DBG] 8.6 scrub ok 2020-09-08T16:07:57.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:57 smithi055 bash[9296]: cluster 2020-09-08T16:07:56.081323+0000 mgr.x (mgr.24118) 1218 : cluster [DBG] pgmap v1812: 17 pgs: 17 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 658 op/s 2020-09-08T16:07:57.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:57 smithi055 bash[9296]: cluster 2020-09-08T16:07:56.273350+0000 osd.0 (osd.0) 123 : cluster [DBG] 8.e scrub ok 2020-09-08T16:07:57.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:57 smithi055 bash[9296]: cluster 2020-09-08T16:07:56.934812+0000 osd.8 (osd.8) 97 : cluster [DBG] 8.b scrub starts 2020-09-08T16:07:57.920 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:57 smithi104 bash[9203]: cluster 2020-09-08T16:07:55.938560+0000 osd.9 (osd.9) 104 : cluster [DBG] 8.c scrub ok 2020-09-08T16:07:57.920 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:57 smithi104 bash[9203]: cluster 2020-09-08T16:07:56.067206+0000 osd.11 (osd.11) 123 : cluster [DBG] 8.6 scrub ok 2020-09-08T16:07:57.920 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:57 smithi104 bash[9203]: cluster 2020-09-08T16:07:56.081323+0000 mgr.x (mgr.24118) 1218 : cluster [DBG] pgmap v1812: 17 pgs: 17 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 658 op/s 2020-09-08T16:07:57.921 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:57 smithi104 bash[9203]: cluster 2020-09-08T16:07:56.273350+0000 osd.0 (osd.0) 123 : cluster [DBG] 8.e scrub ok 2020-09-08T16:07:57.921 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:57 smithi104 bash[9203]: cluster 2020-09-08T16:07:56.934812+0000 osd.8 (osd.8) 97 : cluster [DBG] 8.b scrub starts 2020-09-08T16:07:57.921 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:07:57 smithi104 bash[22305]: debug 2020-09-08T16:07:57.922+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:57.923 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:58.031 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:07:58.185 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:07:58.353 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:58.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:07:58 smithi166 bash[14254]: debug 2020-09-08T16:07:58.179+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:58.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:58 smithi104 bash[9203]: cluster 2020-09-08T16:07:58.250752+0000 osd.8 (osd.8) 98 : cluster [DBG] 8.b scrub ok 2020-09-08T16:07:58.702 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:07:58.806 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:58.814 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:58 smithi166 bash[9446]: cluster 2020-09-08T16:07:58.250752+0000 osd.8 (osd.8) 98 : cluster [DBG] 8.b scrub ok 2020-09-08T16:07:58.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:58 smithi055 bash[9296]: cluster 2020-09-08T16:07:58.250752+0000 osd.8 (osd.8) 98 : cluster [DBG] 8.b scrub ok 2020-09-08T16:07:58.918 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:07:59.022 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:59.121 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:07:58 smithi166 bash[18960]: debug 2020-09-08T16:07:58.911+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:59.154 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:07:58 smithi104 bash[22305]: debug 2020-09-08T16:07:58.702+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:07:59.789 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:59 smithi104 bash[9203]: cluster 2020-09-08T16:07:58.081859+0000 mgr.x (mgr.24118) 1219 : cluster [DBG] pgmap v1813: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 938 op/s 2020-09-08T16:07:59.790 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:59 smithi104 bash[9203]: cluster 2020-09-08T16:07:59.305877+0000 osd.4 (osd.4) 73 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:59.790 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:07:59 smithi104 bash[9203]: cluster 2020-09-08T16:07:59.306005+0000 osd.4 (osd.4) 74 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:59.791 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:07:59.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:59 smithi166 bash[9446]: cluster 2020-09-08T16:07:58.081859+0000 mgr.x (mgr.24118) 1219 : cluster [DBG] pgmap v1813: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 938 op/s 2020-09-08T16:07:59.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:59 smithi166 bash[9446]: cluster 2020-09-08T16:07:59.305877+0000 osd.4 (osd.4) 73 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:59.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:07:59 smithi166 bash[9446]: cluster 2020-09-08T16:07:59.306005+0000 osd.4 (osd.4) 74 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:07:59.896 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:07:59.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:59 smithi055 bash[9296]: cluster 2020-09-08T16:07:58.081859+0000 mgr.x (mgr.24118) 1219 : cluster [DBG] pgmap v1813: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 938 op/s 2020-09-08T16:07:59.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:59 smithi055 bash[9296]: cluster 2020-09-08T16:07:59.305877+0000 osd.4 (osd.4) 73 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:07:59.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:07:59 smithi055 bash[9296]: cluster 2020-09-08T16:07:59.306005+0000 osd.4 (osd.4) 74 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:00.000 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:00.049 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:07:59 smithi104 bash[15273]: debug 2020-09-08T16:07:59.790+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:00.107 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:00.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:08:00 smithi055 bash[16646]: debug 2020-09-08T16:08:00.001+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:00.268 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:00.366 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:08:00 smithi166 bash[16593]: debug 2020-09-08T16:08:00.267+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:00.371 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:08:00.483 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:08:00.590 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:00.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:00 smithi055 bash[9296]: cluster 2020-09-08T16:07:59.485116+0000 mon.a (mon.0) 2159 : cluster [DBG] osdmap e679: 12 total, 12 up, 12 in 2020-09-08T16:08:00.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:08:00 smithi055 bash[19003]: debug 2020-09-08T16:08:00.485+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:00.706 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:00.810 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:08:00.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:00 smithi166 bash[9446]: cluster 2020-09-08T16:07:59.485116+0000 mon.a (mon.0) 2159 : cluster [DBG] osdmap e679: 12 total, 12 up, 12 in 2020-09-08T16:08:00.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:00 smithi104 bash[9203]: cluster 2020-09-08T16:07:59.485116+0000 mon.a (mon.0) 2159 : cluster [DBG] osdmap e679: 12 total, 12 up, 12 in 2020-09-08T16:08:00.929 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:08:01.033 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:01.041 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:08:00 smithi055 bash[11956]: debug 2020-09-08T16:08:00.933+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:01.041 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:08:00 smithi055 bash[14301]: debug 2020-09-08T16:08:00.709+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:01.142 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:01.246 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:01.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:08:01 smithi055 bash[16646]: debug 2020-09-08T16:08:01.145+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:01.408 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:01.500 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T16:08:01.518 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:01.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:08:01 smithi166 bash[11928]: debug 2020-09-08T16:08:01.407+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:01.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:01 smithi166 bash[9446]: cluster 2020-09-08T16:08:00.082423+0000 mgr.x (mgr.24118) 1220 : cluster [DBG] pgmap v1815: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T16:08:01.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:01 smithi166 bash[9446]: audit 2020-09-08T16:08:00.810144+0000 mon.b (mon.1) 118 : audit [INF] from='client.? v1:172.21.15.104:0/1619172823' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:08:01.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:01 smithi166 bash[9446]: audit 2020-09-08T16:08:00.810601+0000 mon.a (mon.0) 2160 : audit [INF] from='client.26479 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:08:01.618 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:01 smithi104 bash[9203]: cluster 2020-09-08T16:08:00.082423+0000 mgr.x (mgr.24118) 1220 : cluster [DBG] pgmap v1815: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T16:08:01.618 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:01 smithi104 bash[9203]: audit 2020-09-08T16:08:00.810144+0000 mon.b (mon.1) 118 : audit [INF] from='client.? v1:172.21.15.104:0/1619172823' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:08:01.619 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:01 smithi104 bash[9203]: audit 2020-09-08T16:08:00.810601+0000 mon.a (mon.0) 2160 : audit [INF] from='client.26479 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:08:01.636 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:01.738 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:01.750 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:01 smithi055 bash[9296]: cluster 2020-09-08T16:08:00.082423+0000 mgr.x (mgr.24118) 1220 : cluster [DBG] pgmap v1815: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T16:08:01.750 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:01 smithi055 bash[9296]: audit 2020-09-08T16:08:00.810144+0000 mon.b (mon.1) 118 : audit [INF] from='client.? v1:172.21.15.104:0/1619172823' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:08:01.750 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:01 smithi055 bash[9296]: audit 2020-09-08T16:08:00.810601+0000 mon.a (mon.0) 2160 : audit [INF] from='client.26479 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:08:01.751 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:08:01 smithi055 bash[14301]: debug 2020-09-08T16:08:01.637+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:01.849 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:01.955 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:02.057 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:08:01 smithi055 bash[16646]: debug 2020-09-08T16:08:01.853+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:02.058 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:02.163 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:02.267 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:02.374 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:02.383 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:08:02 smithi055 bash[16646]: debug 2020-09-08T16:08:02.269+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:02.384 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:08:02 smithi055 bash[14301]: debug 2020-09-08T16:08:02.061+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:02.514 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:02.618 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:08:02.661 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:02 smithi055 bash[9296]: cluster 2020-09-08T16:08:01.485389+0000 mon.a (mon.0) 2161 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:08:02.661 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:02 smithi055 bash[9296]: cluster 2020-09-08T16:08:01.485440+0000 mon.a (mon.0) 2162 : cluster [INF] Cluster is now healthy 2020-09-08T16:08:02.661 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:02 smithi055 bash[9296]: audit 2020-09-08T16:08:01.499510+0000 mon.a (mon.0) 2163 : audit [INF] from='client.26479 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:08:02.662 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:02 smithi055 bash[9296]: cluster 2020-09-08T16:08:01.499691+0000 mon.a (mon.0) 2164 : cluster [DBG] osdmap e680: 12 total, 12 up, 12 in 2020-09-08T16:08:02.662 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:08:02 smithi055 bash[14301]: debug 2020-09-08T16:08:02.509+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:02.799 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:02 smithi166 bash[9446]: cluster 2020-09-08T16:08:01.485389+0000 mon.a (mon.0) 2161 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:08:02.799 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:02 smithi166 bash[9446]: cluster 2020-09-08T16:08:01.485440+0000 mon.a (mon.0) 2162 : cluster [INF] Cluster is now healthy 2020-09-08T16:08:02.800 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:02 smithi166 bash[9446]: audit 2020-09-08T16:08:01.499510+0000 mon.a (mon.0) 2163 : audit [INF] from='client.26479 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:08:02.800 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:02 smithi166 bash[9446]: cluster 2020-09-08T16:08:01.499691+0000 mon.a (mon.0) 2164 : cluster [DBG] osdmap e680: 12 total, 12 up, 12 in 2020-09-08T16:08:02.800 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:08:02.905 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:02.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:02 smithi104 bash[9203]: cluster 2020-09-08T16:08:01.485389+0000 mon.a (mon.0) 2161 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:08:02.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:02 smithi104 bash[9203]: cluster 2020-09-08T16:08:01.485440+0000 mon.a (mon.0) 2162 : cluster [INF] Cluster is now healthy 2020-09-08T16:08:02.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:02 smithi104 bash[9203]: audit 2020-09-08T16:08:01.499510+0000 mon.a (mon.0) 2163 : audit [INF] from='client.26479 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:08:02.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:02 smithi104 bash[9203]: cluster 2020-09-08T16:08:01.499691+0000 mon.a (mon.0) 2164 : cluster [DBG] osdmap e680: 12 total, 12 up, 12 in 2020-09-08T16:08:03.118 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:08:02 smithi166 bash[14254]: debug 2020-09-08T16:08:02.799+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:03.270 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:03.373 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:08:03.488 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:08:03.593 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:03.601 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:08:03 smithi104 bash[22305]: debug 2020-09-08T16:08:03.270+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:03.602 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:03 smithi104 bash[9203]: cluster 2020-09-08T16:08:01.865101+0000 osd.5 (osd.5) 115 : cluster [DBG] 8.8 scrub starts 2020-09-08T16:08:03.602 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:03 smithi104 bash[9203]: cluster 2020-09-08T16:08:02.082928+0000 mgr.x (mgr.24118) 1221 : cluster [DBG] pgmap v1817: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 621 op/s 2020-09-08T16:08:03.602 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:03 smithi104 bash[9203]: cluster 2020-09-08T16:08:02.848793+0000 osd.5 (osd.5) 116 : cluster [DBG] 8.8 scrub ok 2020-09-08T16:08:03.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:08:03 smithi055 bash[19003]: debug 2020-09-08T16:08:03.489+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:03.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:03 smithi055 bash[9296]: cluster 2020-09-08T16:08:01.865101+0000 osd.5 (osd.5) 115 : cluster [DBG] 8.8 scrub starts 2020-09-08T16:08:03.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:03 smithi055 bash[9296]: cluster 2020-09-08T16:08:02.082928+0000 mgr.x (mgr.24118) 1221 : cluster [DBG] pgmap v1817: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 621 op/s 2020-09-08T16:08:03.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:03 smithi055 bash[9296]: cluster 2020-09-08T16:08:02.848793+0000 osd.5 (osd.5) 116 : cluster [DBG] 8.8 scrub ok 2020-09-08T16:08:03.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:03 smithi166 bash[9446]: cluster 2020-09-08T16:08:01.865101+0000 osd.5 (osd.5) 115 : cluster [DBG] 8.8 scrub starts 2020-09-08T16:08:03.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:03 smithi166 bash[9446]: cluster 2020-09-08T16:08:02.082928+0000 mgr.x (mgr.24118) 1221 : cluster [DBG] pgmap v1817: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 621 op/s 2020-09-08T16:08:03.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:03 smithi166 bash[9446]: cluster 2020-09-08T16:08:02.848793+0000 osd.5 (osd.5) 116 : cluster [DBG] 8.8 scrub ok 2020-09-08T16:08:03.910 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:04.015 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:08:04.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:08:03 smithi104 bash[15273]: debug 2020-09-08T16:08:03.910+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:04.737 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:08:04.840 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:08:05.176 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:08:04 smithi104 bash[19922]: debug 2020-09-08T16:08:04.738+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:05.612 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:08:05.645 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:08:05.668 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:08:05.686 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T16:08:05.703 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:08:05.736 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:08:05.749 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:05 smithi104 bash[9203]: cluster 2020-09-08T16:08:04.083493+0000 mgr.x (mgr.24118) 1222 : cluster [DBG] pgmap v1818: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T16:08:05.750 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:08:05 smithi104 bash[17588]: debug 2020-09-08T16:08:05.750+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:05.750 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:08:05.787 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:08:05.826 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:08:05.843 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:08:05.855 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:05.864 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:08:05.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:05 smithi166 bash[9446]: cluster 2020-09-08T16:08:04.083493+0000 mgr.x (mgr.24118) 1222 : cluster [DBG] pgmap v1818: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T16:08:05.898 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:08:05.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:05 smithi055 bash[9296]: cluster 2020-09-08T16:08:04.083493+0000 mgr.x (mgr.24118) 1222 : cluster [DBG] pgmap v1818: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T16:08:05.930 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:08:05.961 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:08:05.995 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:08:05.999 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:08:05.999 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:08:05.999 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:08:06.000 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:08:06.000 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:08:06.000 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:08:06.000 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:08:06.000 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:08:06.001 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:08:06.001 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:08:06.001 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:08:06.001 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:08:06.538 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:06.641 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:06.777 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:06 smithi055 bash[9296]: audit 2020-09-08T16:08:05.592346+0000 mon.c (mon.2) 87 : audit [DBG] from='client.? v1:172.21.15.104:0/4113269512' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:08:06.778 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:08:06 smithi055 bash[14301]: debug 2020-09-08T16:08:06.781+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:06.778 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:06.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:06 smithi166 bash[9446]: audit 2020-09-08T16:08:05.592346+0000 mon.c (mon.2) 87 : audit [DBG] from='client.? v1:172.21.15.104:0/4113269512' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:08:06.883 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:06.892 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:06 smithi104 bash[9203]: audit 2020-09-08T16:08:05.592346+0000 mon.c (mon.2) 87 : audit [DBG] from='client.? v1:172.21.15.104:0/4113269512' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:08:06.892 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:08:06 smithi104 bash[15273]: debug 2020-09-08T16:08:06.538+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:07.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:07 smithi104 bash[9203]: cluster 2020-09-08T16:08:06.083909+0000 mgr.x (mgr.24118) 1223 : cluster [DBG] pgmap v1819: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 724 op/s 2020-09-08T16:08:07.788 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:07.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:07 smithi166 bash[9446]: cluster 2020-09-08T16:08:06.083909+0000 mgr.x (mgr.24118) 1223 : cluster [DBG] pgmap v1819: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 724 op/s 2020-09-08T16:08:07.892 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:08:07.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:07 smithi055 bash[9296]: cluster 2020-09-08T16:08:06.083909+0000 mgr.x (mgr.24118) 1223 : cluster [DBG] pgmap v1819: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 3.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 724 op/s 2020-09-08T16:08:08.065 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T16:08:08.065 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:08:08.117 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:08:07 smithi104 bash[15273]: debug 2020-09-08T16:08:07.790+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:08.118 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:08:08.221 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:08:08.382 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:08:08.438 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:08:08 smithi104 bash[17588]: debug 2020-09-08T16:08:08.118+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:08.487 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:08:08.496 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:08:08 smithi166 bash[14254]: debug 2020-09-08T16:08:08.383+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:08.616 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:08:08.723 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:08.840 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:08:08 smithi166 bash[14254]: debug 2020-09-08T16:08:08.615+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:08.841 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:08.944 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:08:09.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:08:08 smithi166 bash[16593]: debug 2020-09-08T16:08:08.839+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:09.414 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:08:09.518 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:09.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:09 smithi104 bash[9203]: cluster 2020-09-08T16:08:08.084463+0000 mgr.x (mgr.24118) 1224 : cluster [DBG] pgmap v1820: 17 pgs: 17 active+clean; 3.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 849 op/s 2020-09-08T16:08:09.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:09 smithi104 bash[9203]: cluster 2020-09-08T16:08:09.226646+0000 osd.3 (osd.3) 119 : cluster [DBG] 8.0 scrub starts 2020-09-08T16:08:09.677 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:08:09 smithi104 bash[19922]: debug 2020-09-08T16:08:09.414+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:09.722 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:09.826 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2020-09-08T16:08:09.827 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:09.835 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:09 smithi166 bash[9446]: cluster 2020-09-08T16:08:08.084463+0000 mgr.x (mgr.24118) 1224 : cluster [DBG] pgmap v1820: 17 pgs: 17 active+clean; 3.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 849 op/s 2020-09-08T16:08:09.836 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:09 smithi166 bash[9446]: cluster 2020-09-08T16:08:09.226646+0000 osd.3 (osd.3) 119 : cluster [DBG] 8.0 scrub starts 2020-09-08T16:08:09.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:09 smithi055 bash[9296]: cluster 2020-09-08T16:08:08.084463+0000 mgr.x (mgr.24118) 1224 : cluster [DBG] pgmap v1820: 17 pgs: 17 active+clean; 3.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 849 op/s 2020-09-08T16:08:09.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:09 smithi055 bash[9296]: cluster 2020-09-08T16:08:09.226646+0000 osd.3 (osd.3) 119 : cluster [DBG] 8.0 scrub starts 2020-09-08T16:08:09.973 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:08:09.974 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:10.078 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:10.085 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:08:09 smithi104 bash[22305]: debug 2020-09-08T16:08:09.722+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:10.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:08:09 smithi166 bash[18960]: debug 2020-09-08T16:08:09.975+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:10.168 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:08:10.323 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:08:10.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:10 smithi166 bash[9446]: cluster 2020-09-08T16:08:09.603995+0000 osd.9 (osd.9) 105 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:10.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:10 smithi166 bash[9446]: cluster 2020-09-08T16:08:09.604080+0000 osd.9 (osd.9) 106 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:10.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:10 smithi166 bash[9446]: cluster 2020-09-08T16:08:10.085543+0000 osd.3 (osd.3) 120 : cluster [DBG] 8.0 scrub ok 2020-09-08T16:08:10.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:10 smithi055 bash[9296]: cluster 2020-09-08T16:08:09.603995+0000 osd.9 (osd.9) 105 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:10.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:10 smithi055 bash[9296]: cluster 2020-09-08T16:08:09.604080+0000 osd.9 (osd.9) 106 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:10.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:10 smithi055 bash[9296]: cluster 2020-09-08T16:08:10.085543+0000 osd.3 (osd.3) 120 : cluster [DBG] 8.0 scrub ok 2020-09-08T16:08:10.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:10 smithi104 bash[9203]: cluster 2020-09-08T16:08:09.603995+0000 osd.9 (osd.9) 105 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:10.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:10 smithi104 bash[9203]: cluster 2020-09-08T16:08:09.604080+0000 osd.9 (osd.9) 106 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:10.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:10 smithi104 bash[9203]: cluster 2020-09-08T16:08:10.085543+0000 osd.3 (osd.3) 120 : cluster [DBG] 8.0 scrub ok 2020-09-08T16:08:10.976 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:11.086 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:11.200 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:11.306 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:11.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:08:11 smithi055 bash[14301]: debug 2020-09-08T16:08:11.201+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:11.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:08:10 smithi104 bash[22305]: debug 2020-09-08T16:08:10.978+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:11.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:11 smithi166 bash[9446]: cluster 2020-09-08T16:08:10.084895+0000 mgr.x (mgr.24118) 1225 : cluster [DBG] pgmap v1821: 17 pgs: 17 active+clean; 2.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 926 op/s 2020-09-08T16:08:11.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:11 smithi166 bash[9446]: cluster 2020-09-08T16:08:10.528856+0000 mon.a (mon.0) 2165 : cluster [DBG] osdmap e681: 12 total, 12 up, 12 in 2020-09-08T16:08:11.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:11 smithi166 bash[9446]: cluster 2020-09-08T16:08:10.949923+0000 osd.2 (osd.2) 108 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:11.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:11 smithi166 bash[9446]: cluster 2020-09-08T16:08:10.950012+0000 osd.2 (osd.2) 109 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:11.888 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:11.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:11 smithi055 bash[9296]: cluster 2020-09-08T16:08:10.084895+0000 mgr.x (mgr.24118) 1225 : cluster [DBG] pgmap v1821: 17 pgs: 17 active+clean; 2.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 926 op/s 2020-09-08T16:08:11.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:11 smithi055 bash[9296]: cluster 2020-09-08T16:08:10.528856+0000 mon.a (mon.0) 2165 : cluster [DBG] osdmap e681: 12 total, 12 up, 12 in 2020-09-08T16:08:11.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:11 smithi055 bash[9296]: cluster 2020-09-08T16:08:10.949923+0000 osd.2 (osd.2) 108 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:11.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:11 smithi055 bash[9296]: cluster 2020-09-08T16:08:10.950012+0000 osd.2 (osd.2) 109 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:11.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:11 smithi104 bash[9203]: cluster 2020-09-08T16:08:10.084895+0000 mgr.x (mgr.24118) 1225 : cluster [DBG] pgmap v1821: 17 pgs: 17 active+clean; 2.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 926 op/s 2020-09-08T16:08:11.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:11 smithi104 bash[9203]: cluster 2020-09-08T16:08:10.528856+0000 mon.a (mon.0) 2165 : cluster [DBG] osdmap e681: 12 total, 12 up, 12 in 2020-09-08T16:08:11.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:11 smithi104 bash[9203]: cluster 2020-09-08T16:08:10.949923+0000 osd.2 (osd.2) 108 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:11.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:11 smithi104 bash[9203]: cluster 2020-09-08T16:08:10.950012+0000 osd.2 (osd.2) 109 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:11.991 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:12.108 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:12.216 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:12.224 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:08:11 smithi166 bash[11928]: debug 2020-09-08T16:08:11.887+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:12.361 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:12.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:08:12 smithi055 bash[16646]: debug 2020-09-08T16:08:12.109+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:12.463 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:12.542 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:08:12 smithi166 bash[11928]: debug 2020-09-08T16:08:12.363+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:12.615 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:12.722 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:08:12.837 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:12 smithi166 bash[9446]: cluster 2020-09-08T16:08:11.544376+0000 mon.a (mon.0) 2166 : cluster [DBG] osdmap e682: 12 total, 12 up, 12 in 2020-09-08T16:08:12.838 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:08:12 smithi166 bash[16593]: debug 2020-09-08T16:08:12.615+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:12.840 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:08:12.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:12 smithi055 bash[9296]: cluster 2020-09-08T16:08:11.544376+0000 mon.a (mon.0) 2166 : cluster [DBG] osdmap e682: 12 total, 12 up, 12 in 2020-09-08T16:08:12.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:12 smithi104 bash[9203]: cluster 2020-09-08T16:08:11.544376+0000 mon.a (mon.0) 2166 : cluster [DBG] osdmap e682: 12 total, 12 up, 12 in 2020-09-08T16:08:12.946 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:08:13.085 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:08:13.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:08:12 smithi166 bash[14254]: debug 2020-09-08T16:08:12.835+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:13.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:08:13 smithi166 bash[14254]: debug 2020-09-08T16:08:13.071+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:13.187 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:13.363 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:13.466 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:13.483 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:08:13 smithi104 bash[15273]: debug 2020-09-08T16:08:13.362+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:13.664 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:13.769 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:08:13.778 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:13 smithi104 bash[9203]: cluster 2020-09-08T16:08:12.085322+0000 mgr.x (mgr.24118) 1226 : cluster [DBG] pgmap v1824: 17 pgs: 17 active+clean; 2.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 608 op/s 2020-09-08T16:08:13.779 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:13 smithi104 bash[9203]: cluster 2020-09-08T16:08:12.564485+0000 osd.10 (osd.10) 96 : cluster [DBG] 8.9 deep-scrub starts 2020-09-08T16:08:13.779 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:08:13 smithi104 bash[15273]: debug 2020-09-08T16:08:13.666+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:13.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:13 smithi166 bash[9446]: cluster 2020-09-08T16:08:12.085322+0000 mgr.x (mgr.24118) 1226 : cluster [DBG] pgmap v1824: 17 pgs: 17 active+clean; 2.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 608 op/s 2020-09-08T16:08:13.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:13 smithi166 bash[9446]: cluster 2020-09-08T16:08:12.564485+0000 osd.10 (osd.10) 96 : cluster [DBG] 8.9 deep-scrub starts 2020-09-08T16:08:13.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:13 smithi055 bash[9296]: cluster 2020-09-08T16:08:12.085322+0000 mgr.x (mgr.24118) 1226 : cluster [DBG] pgmap v1824: 17 pgs: 17 active+clean; 2.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 608 op/s 2020-09-08T16:08:13.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:13 smithi055 bash[9296]: cluster 2020-09-08T16:08:12.564485+0000 osd.10 (osd.10) 96 : cluster [DBG] 8.9 deep-scrub starts 2020-09-08T16:08:14.486 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:08:14.595 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:14.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:08:14 smithi104 bash[19922]: debug 2020-09-08T16:08:14.486+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:15.179 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:15.280 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:15.405 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:15.428 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:08:15 smithi104 bash[15273]: debug 2020-09-08T16:08:15.170+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:15.509 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:15.561 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:08:15 smithi055 bash[14301]: debug 2020-09-08T16:08:15.409+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:15.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:15 smithi166 bash[9446]: cluster 2020-09-08T16:08:14.085766+0000 mgr.x (mgr.24118) 1227 : cluster [DBG] pgmap v1825: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T16:08:15.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:15 smithi166 bash[9446]: audit 2020-09-08T16:08:15.321358+0000 mon.a (mon.0) 2167 : audit [INF] from='client.? v1:172.21.15.104:0/3958799047' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:08:15.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:15 smithi166 bash[9446]: audit 2020-09-08T16:08:15.321539+0000 mon.c (mon.2) 88 : audit [INF] from='client.16596 v1:172.21.15.104:0/3958799047' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:08:15.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:15 smithi166 bash[9446]: audit 2020-09-08T16:08:15.323816+0000 mon.a (mon.0) 2168 : audit [INF] from='client.16596 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:08:15.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:15 smithi055 bash[9296]: cluster 2020-09-08T16:08:14.085766+0000 mgr.x (mgr.24118) 1227 : cluster [DBG] pgmap v1825: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T16:08:15.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:15 smithi055 bash[9296]: audit 2020-09-08T16:08:15.321358+0000 mon.a (mon.0) 2167 : audit [INF] from='client.? v1:172.21.15.104:0/3958799047' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:08:15.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:15 smithi055 bash[9296]: audit 2020-09-08T16:08:15.321539+0000 mon.c (mon.2) 88 : audit [INF] from='client.16596 v1:172.21.15.104:0/3958799047' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:08:15.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:15 smithi055 bash[9296]: audit 2020-09-08T16:08:15.323816+0000 mon.a (mon.0) 2168 : audit [INF] from='client.16596 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:08:15.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:15 smithi104 bash[9203]: cluster 2020-09-08T16:08:14.085766+0000 mgr.x (mgr.24118) 1227 : cluster [DBG] pgmap v1825: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T16:08:15.935 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:15 smithi104 bash[9203]: audit 2020-09-08T16:08:15.321358+0000 mon.a (mon.0) 2167 : audit [INF] from='client.? v1:172.21.15.104:0/3958799047' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:08:15.935 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:15 smithi104 bash[9203]: audit 2020-09-08T16:08:15.321539+0000 mon.c (mon.2) 88 : audit [INF] from='client.16596 v1:172.21.15.104:0/3958799047' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:08:15.935 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:15 smithi104 bash[9203]: audit 2020-09-08T16:08:15.323816+0000 mon.a (mon.0) 2168 : audit [INF] from='client.16596 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:08:15.976 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:16.025 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T16:08:16.086 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:16.429 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:08:15 smithi104 bash[22305]: debug 2020-09-08T16:08:15.978+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:16.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:16 smithi166 bash[9446]: cluster 2020-09-08T16:08:16.021086+0000 mon.a (mon.0) 2169 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:08:16.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:16 smithi166 bash[9446]: audit 2020-09-08T16:08:16.026814+0000 mon.a (mon.0) 2170 : audit [INF] from='client.16596 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:08:16.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:16 smithi166 bash[9446]: cluster 2020-09-08T16:08:16.026866+0000 mon.a (mon.0) 2171 : cluster [DBG] osdmap e683: 12 total, 12 up, 12 in 2020-09-08T16:08:16.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:16 smithi055 bash[9296]: cluster 2020-09-08T16:08:16.021086+0000 mon.a (mon.0) 2169 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:08:16.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:16 smithi055 bash[9296]: audit 2020-09-08T16:08:16.026814+0000 mon.a (mon.0) 2170 : audit [INF] from='client.16596 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:08:16.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:16 smithi055 bash[9296]: cluster 2020-09-08T16:08:16.026866+0000 mon.a (mon.0) 2171 : cluster [DBG] osdmap e683: 12 total, 12 up, 12 in 2020-09-08T16:08:16.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:16 smithi104 bash[9203]: cluster 2020-09-08T16:08:16.021086+0000 mon.a (mon.0) 2169 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:08:16.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:16 smithi104 bash[9203]: audit 2020-09-08T16:08:16.026814+0000 mon.a (mon.0) 2170 : audit [INF] from='client.16596 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:08:16.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:16 smithi104 bash[9203]: cluster 2020-09-08T16:08:16.026866+0000 mon.a (mon.0) 2171 : cluster [DBG] osdmap e683: 12 total, 12 up, 12 in 2020-09-08T16:08:17.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:17 smithi166 bash[9446]: cluster 2020-09-08T16:08:16.086387+0000 mgr.x (mgr.24118) 1228 : cluster [DBG] pgmap v1827: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 744 op/s 2020-09-08T16:08:17.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:17 smithi055 bash[9296]: cluster 2020-09-08T16:08:16.086387+0000 mgr.x (mgr.24118) 1228 : cluster [DBG] pgmap v1827: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 744 op/s 2020-09-08T16:08:17.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:17 smithi104 bash[9203]: cluster 2020-09-08T16:08:16.086387+0000 mgr.x (mgr.24118) 1228 : cluster [DBG] pgmap v1827: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 744 op/s 2020-09-08T16:08:18.850 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:18 smithi104 bash[9203]: cluster 2020-09-08T16:08:18.317628+0000 osd.7 (osd.7) 147 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:18.850 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:18 smithi104 bash[9203]: cluster 2020-09-08T16:08:18.317729+0000 osd.7 (osd.7) 148 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:18.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:18 smithi166 bash[9446]: cluster 2020-09-08T16:08:18.317628+0000 osd.7 (osd.7) 147 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:18.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:18 smithi166 bash[9446]: cluster 2020-09-08T16:08:18.317729+0000 osd.7 (osd.7) 148 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:18.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:18 smithi055 bash[9296]: cluster 2020-09-08T16:08:18.317628+0000 osd.7 (osd.7) 147 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:18.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:18 smithi055 bash[9296]: cluster 2020-09-08T16:08:18.317729+0000 osd.7 (osd.7) 148 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:18.918 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:08:18.950 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:08:18.983 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:08:19.015 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:08:19.017 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T16:08:19.018 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T16:08:19.018 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 0 2020-09-08T16:08:19.018 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T16:08:19.043 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:08:19.069 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:08:19.098 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:08:19.130 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:08:19.157 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:08:19.192 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:08:19.232 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:08:19.258 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:08:19.265 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:08:19.265 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:08:19.266 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:08:19.266 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:08:19.266 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:08:19.266 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:08:19.267 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:08:19.267 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:08:19.267 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:08:19.268 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:08:19.268 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:08:19.268 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:08:19.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:19 smithi104 bash[9203]: cluster 2020-09-08T16:08:18.087041+0000 mgr.x (mgr.24118) 1229 : cluster [DBG] pgmap v1828: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 883 op/s 2020-09-08T16:08:19.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:19 smithi104 bash[9203]: cluster 2020-09-08T16:08:18.578838+0000 mon.a (mon.0) 2172 : cluster [DBG] osdmap e684: 12 total, 12 up, 12 in 2020-09-08T16:08:19.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:19 smithi104 bash[9203]: cluster 2020-09-08T16:08:18.628644+0000 osd.0 (osd.0) 124 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:19.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:19 smithi104 bash[9203]: cluster 2020-09-08T16:08:18.628739+0000 osd.0 (osd.0) 125 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:19.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:19 smithi104 bash[9203]: audit 2020-09-08T16:08:18.894348+0000 mon.b (mon.1) 119 : audit [DBG] from='client.? v1:172.21.15.104:0/3369544812' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:08:19.800 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:19.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:08:19 smithi166 bash[11928]: debug 2020-09-08T16:08:19.802+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:19.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:19 smithi166 bash[9446]: cluster 2020-09-08T16:08:18.087041+0000 mgr.x (mgr.24118) 1229 : cluster [DBG] pgmap v1828: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 883 op/s 2020-09-08T16:08:19.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:19 smithi166 bash[9446]: cluster 2020-09-08T16:08:18.578838+0000 mon.a (mon.0) 2172 : cluster [DBG] osdmap e684: 12 total, 12 up, 12 in 2020-09-08T16:08:19.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:19 smithi166 bash[9446]: cluster 2020-09-08T16:08:18.628644+0000 osd.0 (osd.0) 124 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:19.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:19 smithi166 bash[9446]: cluster 2020-09-08T16:08:18.628739+0000 osd.0 (osd.0) 125 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:19.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:19 smithi166 bash[9446]: audit 2020-09-08T16:08:18.894348+0000 mon.b (mon.1) 119 : audit [DBG] from='client.? v1:172.21.15.104:0/3369544812' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:08:19.914 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:19.916 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:19 smithi055 bash[9296]: cluster 2020-09-08T16:08:18.087041+0000 mgr.x (mgr.24118) 1229 : cluster [DBG] pgmap v1828: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 883 op/s 2020-09-08T16:08:19.916 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:19 smithi055 bash[9296]: cluster 2020-09-08T16:08:18.578838+0000 mon.a (mon.0) 2172 : cluster [DBG] osdmap e684: 12 total, 12 up, 12 in 2020-09-08T16:08:19.916 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:19 smithi055 bash[9296]: cluster 2020-09-08T16:08:18.628644+0000 osd.0 (osd.0) 124 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:19.917 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:19 smithi055 bash[9296]: cluster 2020-09-08T16:08:18.628739+0000 osd.0 (osd.0) 125 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:19.917 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:19 smithi055 bash[9296]: audit 2020-09-08T16:08:18.894348+0000 mon.b (mon.1) 119 : audit [DBG] from='client.? v1:172.21.15.104:0/3369544812' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:08:20.028 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T16:08:20.180 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:08:20.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:20 smithi166 bash[9446]: cluster 2020-09-08T16:08:19.584237+0000 mon.a (mon.0) 2173 : cluster [DBG] osdmap e685: 12 total, 12 up, 12 in 2020-09-08T16:08:20.872 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:20.905 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:20 smithi104 bash[9203]: cluster 2020-09-08T16:08:19.584237+0000 mon.a (mon.0) 2173 : cluster [DBG] osdmap e685: 12 total, 12 up, 12 in 2020-09-08T16:08:20.906 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2020-09-08T16:08:20.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:20 smithi055 bash[9296]: cluster 2020-09-08T16:08:19.584237+0000 mon.a (mon.0) 2173 : cluster [DBG] osdmap e685: 12 total, 12 up, 12 in 2020-09-08T16:08:20.976 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:21.065 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:08:21.369 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:08:20 smithi166 bash[16593]: debug 2020-09-08T16:08:20.874+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:21.610 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:21.718 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:21.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:21 smithi166 bash[9446]: cluster 2020-09-08T16:08:20.087583+0000 mgr.x (mgr.24118) 1230 : cluster [DBG] pgmap v1831: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 647 op/s 2020-09-08T16:08:21.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:21 smithi055 bash[9296]: cluster 2020-09-08T16:08:20.087583+0000 mgr.x (mgr.24118) 1230 : cluster [DBG] pgmap v1831: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 647 op/s 2020-09-08T16:08:21.909 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:21.925 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:08:21 smithi104 bash[22305]: debug 2020-09-08T16:08:21.610+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:21.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:21 smithi104 bash[9203]: cluster 2020-09-08T16:08:20.087583+0000 mgr.x (mgr.24118) 1230 : cluster [DBG] pgmap v1831: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 647 op/s 2020-09-08T16:08:22.013 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:08:22.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:08:21 smithi166 bash[11928]: debug 2020-09-08T16:08:21.906+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:22.795 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:08:22.899 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:08:23.118 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:08:22 smithi104 bash[17588]: debug 2020-09-08T16:08:22.798+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:23.120 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:08:23.226 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:23.344 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:23.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:08:23 smithi104 bash[17588]: debug 2020-09-08T16:08:23.122+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:23.449 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:08:23.491 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:08:23.580 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:08:23 smithi055 bash[16646]: debug 2020-09-08T16:08:23.345+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:23.580 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:08:23 smithi055 bash[11956]: debug 2020-09-08T16:08:23.581+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:23.580 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:08:23.652 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:08:23.686 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:08:23.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:23 smithi166 bash[9446]: cluster 2020-09-08T16:08:22.088153+0000 mgr.x (mgr.24118) 1231 : cluster [DBG] pgmap v1832: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 676 op/s 2020-09-08T16:08:23.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:23 smithi055 bash[9296]: cluster 2020-09-08T16:08:22.088153+0000 mgr.x (mgr.24118) 1231 : cluster [DBG] pgmap v1832: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 676 op/s 2020-09-08T16:08:23.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:23 smithi104 bash[9203]: cluster 2020-09-08T16:08:22.088153+0000 mgr.x (mgr.24118) 1231 : cluster [DBG] pgmap v1832: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 676 op/s 2020-09-08T16:08:24.507 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:08:24.610 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:24.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:08:24 smithi104 bash[19922]: debug 2020-09-08T16:08:24.510+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:24.760 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:24.868 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:08:25.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:08:24 smithi166 bash[18960]: debug 2020-09-08T16:08:24.758+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:25.477 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:08:25.589 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:08:25.866 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:25 smithi104 bash[9203]: cluster 2020-09-08T16:08:24.088588+0000 mgr.x (mgr.24118) 1232 : cluster [DBG] pgmap v1833: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 969 op/s 2020-09-08T16:08:25.866 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:08:25 smithi104 bash[17588]: debug 2020-09-08T16:08:25.478+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:25.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:25 smithi166 bash[9446]: cluster 2020-09-08T16:08:24.088588+0000 mgr.x (mgr.24118) 1232 : cluster [DBG] pgmap v1833: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 969 op/s 2020-09-08T16:08:25.867 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:08:25.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:25 smithi055 bash[9296]: cluster 2020-09-08T16:08:24.088588+0000 mgr.x (mgr.24118) 1232 : cluster [DBG] pgmap v1833: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 969 op/s 2020-09-08T16:08:25.975 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:26.093 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:26.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:08:25 smithi104 bash[17588]: debug 2020-09-08T16:08:25.866+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:26.201 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:26.376 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:26.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:08:26 smithi055 bash[14301]: debug 2020-09-08T16:08:26.097+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:26.479 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:26.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:08:26 smithi166 bash[16593]: debug 2020-09-08T16:08:26.374+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:27.030 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:27.131 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:27.307 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:27.414 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:27.422 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:08:27 smithi166 bash[18960]: debug 2020-09-08T16:08:27.298+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:27.523 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:08:27 smithi104 bash[15273]: debug 2020-09-08T16:08:27.026+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:27.545 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:27.651 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:27.814 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:27.815 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:27 smithi166 bash[9446]: cluster 2020-09-08T16:08:26.089073+0000 mgr.x (mgr.24118) 1233 : cluster [DBG] pgmap v1834: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 692 op/s 2020-09-08T16:08:27.816 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:08:27 smithi166 bash[18960]: debug 2020-09-08T16:08:27.546+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:27.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:27 smithi055 bash[9296]: cluster 2020-09-08T16:08:26.089073+0000 mgr.x (mgr.24118) 1233 : cluster [DBG] pgmap v1834: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 692 op/s 2020-09-08T16:08:27.918 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:08:27.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:27 smithi104 bash[9203]: cluster 2020-09-08T16:08:26.089073+0000 mgr.x (mgr.24118) 1233 : cluster [DBG] pgmap v1834: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 692 op/s 2020-09-08T16:08:28.028 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:08:28.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:08:27 smithi166 bash[16593]: debug 2020-09-08T16:08:27.814+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:28.135 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:08:28.271 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:08:28 smithi055 bash[19003]: debug 2020-09-08T16:08:28.029+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:28.271 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:08:28 smithi055 bash[11956]: debug 2020-09-08T16:08:28.273+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:28.272 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:08:28.375 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:08:28.975 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:08:29.076 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:29.178 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:29.399 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:29.406 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:08:29 smithi055 bash[16646]: debug 2020-09-08T16:08:29.181+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:29.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:08:28 smithi104 bash[19922]: debug 2020-09-08T16:08:28.970+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:29.541 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:29.611 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T16:08:29.648 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:29.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:29 smithi166 bash[9446]: cluster 2020-09-08T16:08:28.089690+0000 mgr.x (mgr.24118) 1234 : cluster [DBG] pgmap v1835: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 635 op/s 2020-09-08T16:08:29.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:29 smithi166 bash[9446]: cluster 2020-09-08T16:08:28.646147+0000 osd.5 (osd.5) 117 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:29.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:29 smithi166 bash[9446]: cluster 2020-09-08T16:08:28.646226+0000 osd.5 (osd.5) 118 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:29.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:29 smithi166 bash[9446]: audit 2020-09-08T16:08:29.469196+0000 mon.a (mon.0) 2174 : audit [INF] from='client.? v1:172.21.15.104:0/510555956' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:08:29.868 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:08:29 smithi166 bash[18960]: debug 2020-09-08T16:08:29.542+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:29.896 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T16:08:29.896 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T16:08:29.896 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2020-09-08T16:08:29.897 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2020-09-08T16:08:29.897 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T16:08:29.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:29 smithi055 bash[9296]: cluster 2020-09-08T16:08:28.089690+0000 mgr.x (mgr.24118) 1234 : cluster [DBG] pgmap v1835: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 635 op/s 2020-09-08T16:08:29.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:29 smithi055 bash[9296]: cluster 2020-09-08T16:08:28.646147+0000 osd.5 (osd.5) 117 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:29.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:29 smithi055 bash[9296]: cluster 2020-09-08T16:08:28.646226+0000 osd.5 (osd.5) 118 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:29.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:29 smithi055 bash[9296]: audit 2020-09-08T16:08:29.469196+0000 mon.a (mon.0) 2174 : audit [INF] from='client.? v1:172.21.15.104:0/510555956' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:08:29.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:29 smithi104 bash[9203]: cluster 2020-09-08T16:08:28.089690+0000 mgr.x (mgr.24118) 1234 : cluster [DBG] pgmap v1835: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 2.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 635 op/s 2020-09-08T16:08:29.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:29 smithi104 bash[9203]: cluster 2020-09-08T16:08:28.646147+0000 osd.5 (osd.5) 117 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:29.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:29 smithi104 bash[9203]: cluster 2020-09-08T16:08:28.646226+0000 osd.5 (osd.5) 118 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:29.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:29 smithi104 bash[9203]: audit 2020-09-08T16:08:29.469196+0000 mon.a (mon.0) 2174 : audit [INF] from='client.? v1:172.21.15.104:0/510555956' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:08:30.238 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:30.342 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:08:30.500 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:08:30.598 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:08:30 smithi104 bash[15273]: debug 2020-09-08T16:08:30.238+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:30.605 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:30.657 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:08:30 smithi055 bash[11956]: debug 2020-09-08T16:08:30.501+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:30.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:30 smithi055 bash[9296]: cluster 2020-09-08T16:08:29.599470+0000 mon.a (mon.0) 2175 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:08:30.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:30 smithi055 bash[9296]: audit 2020-09-08T16:08:29.610257+0000 mon.a (mon.0) 2176 : audit [INF] from='client.? v1:172.21.15.104:0/510555956' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:08:30.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:30 smithi055 bash[9296]: cluster 2020-09-08T16:08:29.610306+0000 mon.a (mon.0) 2177 : cluster [DBG] osdmap e686: 12 total, 12 up, 12 in 2020-09-08T16:08:30.788 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:30.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:30 smithi166 bash[9446]: cluster 2020-09-08T16:08:29.599470+0000 mon.a (mon.0) 2175 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:08:30.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:30 smithi166 bash[9446]: audit 2020-09-08T16:08:29.610257+0000 mon.a (mon.0) 2176 : audit [INF] from='client.? v1:172.21.15.104:0/510555956' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:08:30.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:30 smithi166 bash[9446]: cluster 2020-09-08T16:08:29.610306+0000 mon.a (mon.0) 2177 : cluster [DBG] osdmap e686: 12 total, 12 up, 12 in 2020-09-08T16:08:30.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:08:30 smithi166 bash[11928]: debug 2020-09-08T16:08:30.782+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:30.896 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:08:30.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:30 smithi104 bash[9203]: cluster 2020-09-08T16:08:29.599470+0000 mon.a (mon.0) 2175 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:08:30.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:30 smithi104 bash[9203]: audit 2020-09-08T16:08:29.610257+0000 mon.a (mon.0) 2176 : audit [INF] from='client.? v1:172.21.15.104:0/510555956' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:08:30.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:30 smithi104 bash[9203]: cluster 2020-09-08T16:08:29.610306+0000 mon.a (mon.0) 2177 : cluster [DBG] osdmap e686: 12 total, 12 up, 12 in 2020-09-08T16:08:31.014 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:08:31.120 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:31.406 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:08:31 smithi055 bash[11956]: debug 2020-09-08T16:08:31.017+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:31.809 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:31.873 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:31 smithi166 bash[9446]: cluster 2020-09-08T16:08:30.090341+0000 mgr.x (mgr.24118) 1235 : cluster [DBG] pgmap v1837: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 1.9 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 775 op/s 2020-09-08T16:08:31.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:31 smithi055 bash[9296]: cluster 2020-09-08T16:08:30.090341+0000 mgr.x (mgr.24118) 1235 : cluster [DBG] pgmap v1837: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 1.9 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 775 op/s 2020-09-08T16:08:31.910 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:31.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:08:31 smithi104 bash[22305]: debug 2020-09-08T16:08:31.806+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:31.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:31 smithi104 bash[9203]: cluster 2020-09-08T16:08:30.090341+0000 mgr.x (mgr.24118) 1235 : cluster [DBG] pgmap v1837: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 1.9 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 775 op/s 2020-09-08T16:08:32.044 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:32.158 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:32.200 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2020-09-08T16:08:32.355 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:08:32.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:08:32 smithi166 bash[18960]: debug 2020-09-08T16:08:32.042+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:32.600 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:32 smithi055 bash[9296]: cluster 2020-09-08T16:08:31.233016+0000 osd.10 (osd.10) 97 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:32.600 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:32 smithi055 bash[9296]: cluster 2020-09-08T16:08:31.233103+0000 osd.10 (osd.10) 98 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:32.601 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:32 smithi055 bash[9296]: cluster 2020-09-08T16:08:32.030418+0000 mon.a (mon.0) 2178 : cluster [DBG] osdmap e687: 12 total, 12 up, 12 in 2020-09-08T16:08:32.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:32 smithi166 bash[9446]: cluster 2020-09-08T16:08:31.233016+0000 osd.10 (osd.10) 97 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:32.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:32 smithi166 bash[9446]: cluster 2020-09-08T16:08:31.233103+0000 osd.10 (osd.10) 98 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:32.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:32 smithi166 bash[9446]: cluster 2020-09-08T16:08:32.030418+0000 mon.a (mon.0) 2178 : cluster [DBG] osdmap e687: 12 total, 12 up, 12 in 2020-09-08T16:08:32.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:32 smithi104 bash[9203]: cluster 2020-09-08T16:08:31.233016+0000 osd.10 (osd.10) 97 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:32.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:32 smithi104 bash[9203]: cluster 2020-09-08T16:08:31.233103+0000 osd.10 (osd.10) 98 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:32.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:32 smithi104 bash[9203]: cluster 2020-09-08T16:08:32.030418+0000 mon.a (mon.0) 2178 : cluster [DBG] osdmap e687: 12 total, 12 up, 12 in 2020-09-08T16:08:32.960 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:33.065 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:33.210 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:33.278 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:08:33.313 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:33.318 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:08:33.345 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:08:33.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:08:33 smithi166 bash[11928]: debug 2020-09-08T16:08:33.206+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:33.370 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:08:33.408 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:08:32 smithi055 bash[16646]: debug 2020-09-08T16:08:32.961+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:33.408 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:08:33.436 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:33.454 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:08:33.474 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:08:33.503 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:08:33.532 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:08:33.541 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:33.559 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:08:33.588 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:08:33.628 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:08:33.635 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:08:33.635 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:08:33.635 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:08:33.636 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:08:33.636 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:08:33.636 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:08:33.636 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:08:33.636 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:08:33.637 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:08:33.637 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:08:33.637 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:08:33.649 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:08:33.709 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:33 smithi166 bash[9446]: cluster 2020-09-08T16:08:32.090870+0000 mgr.x (mgr.24118) 1236 : cluster [DBG] pgmap v1839: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 1.9 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 516 op/s 2020-09-08T16:08:33.709 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:33 smithi166 bash[9446]: audit 2020-09-08T16:08:33.256835+0000 mon.a (mon.0) 2179 : audit [DBG] from='client.? v1:172.21.15.104:0/1383785259' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:08:33.710 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:08:33 smithi166 bash[16593]: debug 2020-09-08T16:08:33.434+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:33.710 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:08:33 smithi166 bash[18960]: debug 2020-09-08T16:08:33.706+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:33.710 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:33.816 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:08:33.823 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T16:08:33.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:33 smithi055 bash[9296]: cluster 2020-09-08T16:08:32.090870+0000 mgr.x (mgr.24118) 1236 : cluster [DBG] pgmap v1839: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 1.9 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 516 op/s 2020-09-08T16:08:33.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:33 smithi055 bash[9296]: audit 2020-09-08T16:08:33.256835+0000 mon.a (mon.0) 2179 : audit [DBG] from='client.? v1:172.21.15.104:0/1383785259' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:08:33.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:33 smithi104 bash[9203]: cluster 2020-09-08T16:08:32.090870+0000 mgr.x (mgr.24118) 1236 : cluster [DBG] pgmap v1839: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 1.9 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 516 op/s 2020-09-08T16:08:33.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:33 smithi104 bash[9203]: audit 2020-09-08T16:08:33.256835+0000 mon.a (mon.0) 2179 : audit [DBG] from='client.? v1:172.21.15.104:0/1383785259' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:08:33.929 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:08:33.970 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:08:34.035 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:34.177 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:34.281 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:34.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:08:34 smithi166 bash[11928]: debug 2020-09-08T16:08:34.174+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:34.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:08:33 smithi055 bash[19003]: debug 2020-09-08T16:08:33.933+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:34.607 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:34 smithi104 bash[9203]: cluster 2020-09-08T16:08:33.669259+0000 osd.3 (osd.3) 121 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:34.608 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:34 smithi104 bash[9203]: cluster 2020-09-08T16:08:33.669322+0000 osd.3 (osd.3) 122 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:34.872 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:34 smithi166 bash[9446]: cluster 2020-09-08T16:08:33.669259+0000 osd.3 (osd.3) 121 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:34.873 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:34 smithi166 bash[9446]: cluster 2020-09-08T16:08:33.669322+0000 osd.3 (osd.3) 122 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:34.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:34 smithi055 bash[9296]: cluster 2020-09-08T16:08:33.669259+0000 osd.3 (osd.3) 121 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:34.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:34 smithi055 bash[9296]: cluster 2020-09-08T16:08:33.669322+0000 osd.3 (osd.3) 122 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:35.227 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:35.332 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:08:35.449 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:08:35.489 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:08:35 smithi104 bash[15273]: debug 2020-09-08T16:08:35.230+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:35.566 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:08:35.608 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:08:35 smithi055 bash[19003]: debug 2020-09-08T16:08:35.453+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:35.608 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:35 smithi055 bash[9296]: cluster 2020-09-08T16:08:34.091461+0000 mgr.x (mgr.24118) 1237 : cluster [DBG] pgmap v1840: 17 pgs: 17 active+clean; 1.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T16:08:35.609 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:35 smithi055 bash[9296]: cluster 2020-09-08T16:08:34.615594+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e688: 12 total, 12 up, 12 in 2020-09-08T16:08:35.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:35 smithi166 bash[9446]: cluster 2020-09-08T16:08:34.091461+0000 mgr.x (mgr.24118) 1237 : cluster [DBG] pgmap v1840: 17 pgs: 17 active+clean; 1.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T16:08:35.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:35 smithi166 bash[9446]: cluster 2020-09-08T16:08:34.615594+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e688: 12 total, 12 up, 12 in 2020-09-08T16:08:36.008 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:35 smithi104 bash[9203]: cluster 2020-09-08T16:08:34.091461+0000 mgr.x (mgr.24118) 1237 : cluster [DBG] pgmap v1840: 17 pgs: 17 active+clean; 1.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T16:08:36.008 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:35 smithi104 bash[9203]: cluster 2020-09-08T16:08:34.615594+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e688: 12 total, 12 up, 12 in 2020-09-08T16:08:36.185 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:08:36.288 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:08:36.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:08:36 smithi104 bash[17588]: debug 2020-09-08T16:08:36.186+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:36.429 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:08:36.536 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:36.616 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:08:36 smithi166 bash[14254]: debug 2020-09-08T16:08:36.430+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:36.694 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:36.801 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:08:37.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:08:36 smithi166 bash[16593]: debug 2020-09-08T16:08:36.694+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:37.404 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:08:37.509 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:37.579 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:08:37 smithi104 bash[17588]: debug 2020-09-08T16:08:37.402+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:37.638 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:37.743 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:08:37.863 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:08:37.873 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:08:37 smithi055 bash[19003]: debug 2020-09-08T16:08:37.861+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:37.873 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:37 smithi055 bash[9296]: cluster 2020-09-08T16:08:36.092024+0000 mgr.x (mgr.24118) 1238 : cluster [DBG] pgmap v1842: 17 pgs: 17 active+clean; 1.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 665 op/s 2020-09-08T16:08:37.874 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:37 smithi166 bash[9446]: cluster 2020-09-08T16:08:36.092024+0000 mgr.x (mgr.24118) 1238 : cluster [DBG] pgmap v1842: 17 pgs: 17 active+clean; 1.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 665 op/s 2020-09-08T16:08:37.874 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:08:37 smithi166 bash[18960]: debug 2020-09-08T16:08:37.638+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:37.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:37 smithi104 bash[9203]: cluster 2020-09-08T16:08:36.092024+0000 mgr.x (mgr.24118) 1238 : cluster [DBG] pgmap v1842: 17 pgs: 17 active+clean; 1.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 665 op/s 2020-09-08T16:08:37.974 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:08:38.104 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:08:38.134 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:08:38.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:08:38 smithi055 bash[19003]: debug 2020-09-08T16:08:38.105+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:38.209 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:38.288 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:08:38.805 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:38.907 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:39.015 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:39.126 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:39.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:08:38 smithi104 bash[22305]: debug 2020-09-08T16:08:38.806+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:39.252 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:08:39 smithi055 bash[16646]: debug 2020-09-08T16:08:39.017+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:39.254 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:39.361 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:39.510 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:39.614 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:08:39 smithi055 bash[16646]: debug 2020-09-08T16:08:39.257+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:39.616 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:39.730 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:39.836 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:39.844 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:39 smithi166 bash[9446]: cluster 2020-09-08T16:08:38.092630+0000 mgr.x (mgr.24118) 1239 : cluster [DBG] pgmap v1843: 17 pgs: 17 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 865 op/s 2020-09-08T16:08:39.845 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:39 smithi166 bash[9446]: cluster 2020-09-08T16:08:39.275670+0000 osd.8 (osd.8) 99 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:39.845 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:39 smithi166 bash[9446]: cluster 2020-09-08T16:08:39.275757+0000 osd.8 (osd.8) 100 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:39.845 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:08:39 smithi166 bash[16593]: debug 2020-09-08T16:08:39.510+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:39.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:08:39 smithi055 bash[14301]: debug 2020-09-08T16:08:39.729+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:39.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:39 smithi055 bash[9296]: cluster 2020-09-08T16:08:38.092630+0000 mgr.x (mgr.24118) 1239 : cluster [DBG] pgmap v1843: 17 pgs: 17 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 865 op/s 2020-09-08T16:08:39.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:39 smithi055 bash[9296]: cluster 2020-09-08T16:08:39.275670+0000 osd.8 (osd.8) 99 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:39.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:39 smithi055 bash[9296]: cluster 2020-09-08T16:08:39.275757+0000 osd.8 (osd.8) 100 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:39.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:39 smithi104 bash[9203]: cluster 2020-09-08T16:08:38.092630+0000 mgr.x (mgr.24118) 1239 : cluster [DBG] pgmap v1843: 17 pgs: 17 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 865 op/s 2020-09-08T16:08:39.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:39 smithi104 bash[9203]: cluster 2020-09-08T16:08:39.275670+0000 osd.8 (osd.8) 99 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:39.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:39 smithi104 bash[9203]: cluster 2020-09-08T16:08:39.275757+0000 osd.8 (osd.8) 100 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:39.995 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:40.096 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:08:40.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:08:39 smithi166 bash[18960]: debug 2020-09-08T16:08:39.990+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:40.782 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:08:40.886 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:08:40.894 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:40 smithi055 bash[9296]: cluster 2020-09-08T16:08:39.627192+0000 mon.a (mon.0) 2181 : cluster [DBG] osdmap e689: 12 total, 12 up, 12 in 2020-09-08T16:08:40.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:40 smithi104 bash[9203]: cluster 2020-09-08T16:08:39.627192+0000 mon.a (mon.0) 2181 : cluster [DBG] osdmap e689: 12 total, 12 up, 12 in 2020-09-08T16:08:40.927 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:08:40 smithi104 bash[17588]: debug 2020-09-08T16:08:40.782+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:41.029 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:08:41.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:40 smithi166 bash[9446]: cluster 2020-09-08T16:08:39.627192+0000 mon.a (mon.0) 2181 : cluster [DBG] osdmap e689: 12 total, 12 up, 12 in 2020-09-08T16:08:41.134 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:08:41.156 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:08:41 smithi055 bash[19003]: debug 2020-09-08T16:08:41.033+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:41.245 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:08:41.350 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:41.470 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:41.579 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:08:41.632 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:08:41 smithi055 bash[16646]: debug 2020-09-08T16:08:41.473+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:41.633 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:08:41 smithi055 bash[19003]: debug 2020-09-08T16:08:41.249+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:41.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:41 smithi055 bash[9296]: cluster 2020-09-08T16:08:40.093149+0000 mgr.x (mgr.24118) 1240 : cluster [DBG] pgmap v1845: 17 pgs: 17 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T16:08:41.922 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:08:41.923 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:41 smithi104 bash[9203]: cluster 2020-09-08T16:08:40.093149+0000 mgr.x (mgr.24118) 1240 : cluster [DBG] pgmap v1845: 17 pgs: 17 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T16:08:41.923 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:08:41 smithi104 bash[17588]: debug 2020-09-08T16:08:41.922+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:41.949 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T16:08:41.950 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:08:42.025 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:08:42.043 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:41 smithi166 bash[9446]: cluster 2020-09-08T16:08:40.093149+0000 mgr.x (mgr.24118) 1240 : cluster [DBG] pgmap v1845: 17 pgs: 17 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T16:08:42.192 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:08:42.294 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:42.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:08:42 smithi166 bash[14254]: debug 2020-09-08T16:08:42.182+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:42.420 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:42.523 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:42.627 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:08:42 smithi055 bash[16646]: debug 2020-09-08T16:08:42.417+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:42.635 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T16:08:42.703 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:42.808 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:42.817 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:42 smithi166 bash[9446]: audit 2020-09-08T16:08:42.184388+0000 mon.a (mon.0) 2182 : audit [INF] from='client.? v1:172.21.15.104:0/3115203738' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:08:42.817 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:08:42 smithi166 bash[18960]: debug 2020-09-08T16:08:42.702+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:42.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:42 smithi055 bash[9296]: audit 2020-09-08T16:08:42.184388+0000 mon.a (mon.0) 2182 : audit [INF] from='client.? v1:172.21.15.104:0/3115203738' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:08:42.919 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:43.023 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:43.034 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:42 smithi104 bash[9203]: audit 2020-09-08T16:08:42.184388+0000 mon.a (mon.0) 2182 : audit [INF] from='client.? v1:172.21.15.104:0/3115203738' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:08:43.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:08:42 smithi166 bash[16593]: debug 2020-09-08T16:08:42.918+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:43.650 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:43.753 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:08:43.868 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:08:43.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:43 smithi055 bash[9296]: cluster 2020-09-08T16:08:42.093781+0000 mgr.x (mgr.24118) 1241 : cluster [DBG] pgmap v1846: 17 pgs: 17 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 618 op/s 2020-09-08T16:08:43.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:43 smithi055 bash[9296]: cluster 2020-09-08T16:08:42.629129+0000 mon.a (mon.0) 2183 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:08:43.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:43 smithi055 bash[9296]: audit 2020-09-08T16:08:42.637910+0000 mon.a (mon.0) 2184 : audit [INF] from='client.? v1:172.21.15.104:0/3115203738' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:08:43.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:43 smithi055 bash[9296]: cluster 2020-09-08T16:08:42.637977+0000 mon.a (mon.0) 2185 : cluster [DBG] osdmap e690: 12 total, 12 up, 12 in 2020-09-08T16:08:43.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:43 smithi055 bash[9296]: cluster 2020-09-08T16:08:43.200216+0000 osd.7 (osd.7) 149 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:08:43.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:43 smithi055 bash[9296]: cluster 2020-09-08T16:08:43.319371+0000 osd.7 (osd.7) 150 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:08:43.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:08:43 smithi104 bash[22305]: debug 2020-09-08T16:08:43.650+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:43.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:43 smithi104 bash[9203]: cluster 2020-09-08T16:08:42.093781+0000 mgr.x (mgr.24118) 1241 : cluster [DBG] pgmap v1846: 17 pgs: 17 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 618 op/s 2020-09-08T16:08:43.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:43 smithi104 bash[9203]: cluster 2020-09-08T16:08:42.629129+0000 mon.a (mon.0) 2183 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:08:43.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:43 smithi104 bash[9203]: audit 2020-09-08T16:08:42.637910+0000 mon.a (mon.0) 2184 : audit [INF] from='client.? v1:172.21.15.104:0/3115203738' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:08:43.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:43 smithi104 bash[9203]: cluster 2020-09-08T16:08:42.637977+0000 mon.a (mon.0) 2185 : cluster [DBG] osdmap e690: 12 total, 12 up, 12 in 2020-09-08T16:08:43.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:43 smithi104 bash[9203]: cluster 2020-09-08T16:08:43.200216+0000 osd.7 (osd.7) 149 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:08:43.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:43 smithi104 bash[9203]: cluster 2020-09-08T16:08:43.319371+0000 osd.7 (osd.7) 150 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:08:43.978 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:43.986 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:43 smithi166 bash[9446]: cluster 2020-09-08T16:08:42.093781+0000 mgr.x (mgr.24118) 1241 : cluster [DBG] pgmap v1846: 17 pgs: 17 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 618 op/s 2020-09-08T16:08:43.987 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:43 smithi166 bash[9446]: cluster 2020-09-08T16:08:42.629129+0000 mon.a (mon.0) 2183 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:08:43.987 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:43 smithi166 bash[9446]: audit 2020-09-08T16:08:42.637910+0000 mon.a (mon.0) 2184 : audit [INF] from='client.? v1:172.21.15.104:0/3115203738' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:08:43.987 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:43 smithi166 bash[9446]: cluster 2020-09-08T16:08:42.637977+0000 mon.a (mon.0) 2185 : cluster [DBG] osdmap e690: 12 total, 12 up, 12 in 2020-09-08T16:08:43.987 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:43 smithi166 bash[9446]: cluster 2020-09-08T16:08:43.200216+0000 osd.7 (osd.7) 149 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:08:43.987 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:43 smithi166 bash[9446]: cluster 2020-09-08T16:08:43.319371+0000 osd.7 (osd.7) 150 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:08:43.992 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:08:43 smithi166 bash[14254]: debug 2020-09-08T16:08:43.866+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:44.154 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2020-09-08T16:08:44.179 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:44.285 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:44.340 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:08:44.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:08:44 smithi166 bash[11928]: debug 2020-09-08T16:08:44.174+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:44.766 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:44.870 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:44.896 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:44 smithi104 bash[9203]: cluster 2020-09-08T16:08:43.551282+0000 osd.1 (osd.1) 138 : cluster [DBG] 8.7 scrub starts 2020-09-08T16:08:44.897 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:44 smithi104 bash[9203]: cluster 2020-09-08T16:08:44.201575+0000 osd.1 (osd.1) 139 : cluster [DBG] 8.7 scrub ok 2020-09-08T16:08:44.897 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:08:44 smithi104 bash[15273]: debug 2020-09-08T16:08:44.766+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:44.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:44 smithi055 bash[9296]: cluster 2020-09-08T16:08:43.551282+0000 osd.1 (osd.1) 138 : cluster [DBG] 8.7 scrub starts 2020-09-08T16:08:44.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:44 smithi055 bash[9296]: cluster 2020-09-08T16:08:44.201575+0000 osd.1 (osd.1) 139 : cluster [DBG] 8.7 scrub ok 2020-09-08T16:08:45.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:44 smithi166 bash[9446]: cluster 2020-09-08T16:08:43.551282+0000 osd.1 (osd.1) 138 : cluster [DBG] 8.7 scrub starts 2020-09-08T16:08:45.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:44 smithi166 bash[9446]: cluster 2020-09-08T16:08:44.201575+0000 osd.1 (osd.1) 139 : cluster [DBG] 8.7 scrub ok 2020-09-08T16:08:45.505 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:45.611 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:45.768 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:45.870 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:08:45.883 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:08:45 smithi104 bash[22305]: debug 2020-09-08T16:08:45.506+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:45.883 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:45 smithi104 bash[9203]: cluster 2020-09-08T16:08:44.094357+0000 mgr.x (mgr.24118) 1242 : cluster [DBG] pgmap v1848: 17 pgs: 17 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T16:08:45.884 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:45 smithi104 bash[9203]: cluster 2020-09-08T16:08:44.757630+0000 osd.10 (osd.10) 99 : cluster [DBG] 8.9 scrub starts 2020-09-08T16:08:45.906 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:45 smithi055 bash[9296]: cluster 2020-09-08T16:08:44.094357+0000 mgr.x (mgr.24118) 1242 : cluster [DBG] pgmap v1848: 17 pgs: 17 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T16:08:45.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:45 smithi055 bash[9296]: cluster 2020-09-08T16:08:44.757630+0000 osd.10 (osd.10) 99 : cluster [DBG] 8.9 scrub starts 2020-09-08T16:08:46.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:45 smithi166 bash[9446]: cluster 2020-09-08T16:08:44.094357+0000 mgr.x (mgr.24118) 1242 : cluster [DBG] pgmap v1848: 17 pgs: 17 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T16:08:46.123 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:45 smithi166 bash[9446]: cluster 2020-09-08T16:08:44.757630+0000 osd.10 (osd.10) 99 : cluster [DBG] 8.9 scrub starts 2020-09-08T16:08:46.123 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:08:45 smithi166 bash[11928]: debug 2020-09-08T16:08:45.762+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:46.123 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:08:46.156 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:08:46.189 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:08:46.215 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:08:46.226 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:46.243 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:08:46.278 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:08:46.309 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:08:46.341 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:08:46.374 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:08:46.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:08:46 smithi055 bash[11956]: debug 2020-09-08T16:08:46.116+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:46.409 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:08:46.446 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:08:46.485 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:08:46.518 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:08:46.520 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:08:46.520 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:08:46.520 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:08:46.520 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:08:46.521 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:08:46.521 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:08:46.521 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:08:46.521 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:08:46.521 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:08:46.522 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:08:46.522 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:08:46.522 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:08:46.779 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:46.814 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T16:08:46.883 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:46.906 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:08:46 smithi055 bash[16646]: debug 2020-09-08T16:08:46.780+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:46.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:46 smithi055 bash[9296]: cluster 2020-09-08T16:08:45.224763+0000 osd.10 (osd.10) 100 : cluster [DBG] 8.9 scrub ok 2020-09-08T16:08:46.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:46 smithi055 bash[9296]: audit 2020-09-08T16:08:46.137157+0000 mon.a (mon.0) 2186 : audit [DBG] from='client.? v1:172.21.15.104:0/3504129648' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:08:46.949 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:46 smithi104 bash[9203]: cluster 2020-09-08T16:08:45.224763+0000 osd.10 (osd.10) 100 : cluster [DBG] 8.9 scrub ok 2020-09-08T16:08:46.949 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:46 smithi104 bash[9203]: audit 2020-09-08T16:08:46.137157+0000 mon.a (mon.0) 2186 : audit [DBG] from='client.? v1:172.21.15.104:0/3504129648' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:08:46.971 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:08:47.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:46 smithi166 bash[9446]: cluster 2020-09-08T16:08:45.224763+0000 osd.10 (osd.10) 100 : cluster [DBG] 8.9 scrub ok 2020-09-08T16:08:47.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:46 smithi166 bash[9446]: audit 2020-09-08T16:08:46.137157+0000 mon.a (mon.0) 2186 : audit [DBG] from='client.? v1:172.21.15.104:0/3504129648' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:08:47.574 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:47.683 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:47.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:47 smithi055 bash[9296]: cluster 2020-09-08T16:08:46.094918+0000 mgr.x (mgr.24118) 1243 : cluster [DBG] pgmap v1849: 17 pgs: 17 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 804 op/s 2020-09-08T16:08:47.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:47 smithi055 bash[9296]: cluster 2020-09-08T16:08:46.326609+0000 osd.11 (osd.11) 124 : cluster [DBG] 8.3 scrub starts 2020-09-08T16:08:47.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:47 smithi055 bash[9296]: cluster 2020-09-08T16:08:46.650834+0000 osd.11 (osd.11) 125 : cluster [DBG] 8.3 scrub ok 2020-09-08T16:08:47.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:47 smithi055 bash[9296]: cluster 2020-09-08T16:08:46.754900+0000 osd.10 (osd.10) 101 : cluster [DBG] 8.2 scrub starts 2020-09-08T16:08:47.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:47 smithi055 bash[9296]: cluster 2020-09-08T16:08:47.267107+0000 osd.1 (osd.1) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:47.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:47 smithi055 bash[9296]: cluster 2020-09-08T16:08:47.267199+0000 osd.1 (osd.1) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:47.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:08:47 smithi104 bash[22305]: debug 2020-09-08T16:08:47.574+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:47.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:47 smithi104 bash[9203]: cluster 2020-09-08T16:08:46.094918+0000 mgr.x (mgr.24118) 1243 : cluster [DBG] pgmap v1849: 17 pgs: 17 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 804 op/s 2020-09-08T16:08:47.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:47 smithi104 bash[9203]: cluster 2020-09-08T16:08:46.326609+0000 osd.11 (osd.11) 124 : cluster [DBG] 8.3 scrub starts 2020-09-08T16:08:47.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:47 smithi104 bash[9203]: cluster 2020-09-08T16:08:46.650834+0000 osd.11 (osd.11) 125 : cluster [DBG] 8.3 scrub ok 2020-09-08T16:08:47.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:47 smithi104 bash[9203]: cluster 2020-09-08T16:08:46.754900+0000 osd.10 (osd.10) 101 : cluster [DBG] 8.2 scrub starts 2020-09-08T16:08:47.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:47 smithi104 bash[9203]: cluster 2020-09-08T16:08:47.267107+0000 osd.1 (osd.1) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:47.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:47 smithi104 bash[9203]: cluster 2020-09-08T16:08:47.267199+0000 osd.1 (osd.1) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:48.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:47 smithi166 bash[9446]: cluster 2020-09-08T16:08:46.094918+0000 mgr.x (mgr.24118) 1243 : cluster [DBG] pgmap v1849: 17 pgs: 17 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 804 op/s 2020-09-08T16:08:48.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:47 smithi166 bash[9446]: cluster 2020-09-08T16:08:46.326609+0000 osd.11 (osd.11) 124 : cluster [DBG] 8.3 scrub starts 2020-09-08T16:08:48.123 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:47 smithi166 bash[9446]: cluster 2020-09-08T16:08:46.650834+0000 osd.11 (osd.11) 125 : cluster [DBG] 8.3 scrub ok 2020-09-08T16:08:48.123 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:47 smithi166 bash[9446]: cluster 2020-09-08T16:08:46.754900+0000 osd.10 (osd.10) 101 : cluster [DBG] 8.2 scrub starts 2020-09-08T16:08:48.124 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:47 smithi166 bash[9446]: cluster 2020-09-08T16:08:47.267107+0000 osd.1 (osd.1) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:48.124 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:47 smithi166 bash[9446]: cluster 2020-09-08T16:08:47.267199+0000 osd.1 (osd.1) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:48.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:48 smithi104 bash[9203]: cluster 2020-09-08T16:08:47.066037+0000 osd.10 (osd.10) 102 : cluster [DBG] 8.2 scrub ok 2020-09-08T16:08:48.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:48 smithi104 bash[9203]: cluster 2020-09-08T16:08:47.650701+0000 mon.a (mon.0) 2187 : cluster [DBG] osdmap e691: 12 total, 12 up, 12 in 2020-09-08T16:08:49.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:48 smithi166 bash[9446]: cluster 2020-09-08T16:08:47.066037+0000 osd.10 (osd.10) 102 : cluster [DBG] 8.2 scrub ok 2020-09-08T16:08:49.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:48 smithi166 bash[9446]: cluster 2020-09-08T16:08:47.650701+0000 mon.a (mon.0) 2187 : cluster [DBG] osdmap e691: 12 total, 12 up, 12 in 2020-09-08T16:08:49.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:48 smithi055 bash[9296]: cluster 2020-09-08T16:08:47.066037+0000 osd.10 (osd.10) 102 : cluster [DBG] 8.2 scrub ok 2020-09-08T16:08:49.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:48 smithi055 bash[9296]: cluster 2020-09-08T16:08:47.650701+0000 mon.a (mon.0) 2187 : cluster [DBG] osdmap e691: 12 total, 12 up, 12 in 2020-09-08T16:08:49.671 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:49 smithi104 bash[9203]: cluster 2020-09-08T16:08:48.095476+0000 mgr.x (mgr.24118) 1244 : cluster [DBG] pgmap v1851: 17 pgs: 17 active+clean; 954 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 944 op/s 2020-09-08T16:08:49.857 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:08:49.858 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T16:08:49.858 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:08:49.858 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop(client.36335.0:249544 8.b e691/627 8:de77f2f9:::benchmark_data_smithi166_52302_object118623:head v 691'15521, mlcod=691'15519)", 2020-09-08T16:08:49.858 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:08:49.853711+0000", 2020-09-08T16:08:49.859 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.0058799050000000004, 2020-09-08T16:08:49.859 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0059159010000000003, 2020-09-08T16:08:49.859 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:08:49.859 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:08:49.859 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:08:49.860 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:08:49.871 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:08:49.871 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.853711+0000", 2020-09-08T16:08:49.885 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:08:49.885 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:08:49.888 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:08:49.892 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:08:49.893 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.853711+0000", 2020-09-08T16:08:49.893 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.7102999999999997e-05 2020-09-08T16:08:49.893 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:08:49.894 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:08:49.894 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:08:49.894 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.853788+0000", 2020-09-08T16:08:49.895 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0341e-05 2020-09-08T16:08:49.895 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:08:49.895 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:08:49.896 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:08:49.896 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.853798+0000", 2020-09-08T16:08:49.896 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.0176000000000001e-05 2020-09-08T16:08:49.897 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:08:49.897 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:08:49.897 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:08:49.898 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.853818+0000", 2020-09-08T16:08:49.902 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.4290000000000004e-06 2020-09-08T16:08:49.902 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:08:49.902 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:08:49.903 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:08:49.903 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.853826+0000", 2020-09-08T16:08:49.903 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9284999999999999e-05 2020-09-08T16:08:49.903 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:08:49.904 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:08:49.904 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:08:49.905 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.853885+0000", 2020-09-08T16:08:49.905 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.3927000000000002e-05 2020-09-08T16:08:49.906 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:08:49.906 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:08:49.907 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:08:49.907 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.853919+0000", 2020-09-08T16:08:49.908 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.005654549 2020-09-08T16:08:49.908 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:08:49.908 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:08:49.909 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:08:49.909 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.859574+0000", 2020-09-08T16:08:49.909 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0058628100000000004 2020-09-08T16:08:49.909 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:08:49.910 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:08:49.910 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:08:49.910 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:08:49.911 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2020-09-08T16:08:49.911 INFO:teuthology.orchestra.run.smithi055.stdout: "num_ops": 1 2020-09-08T16:08:49.911 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:08:49.912 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:49 smithi055 bash[9296]: cluster 2020-09-08T16:08:48.095476+0000 mgr.x (mgr.24118) 1244 : cluster [DBG] pgmap v1851: 17 pgs: 17 active+clean; 954 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 944 op/s 2020-09-08T16:08:50.069 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:08:50.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:49 smithi166 bash[9446]: cluster 2020-09-08T16:08:48.095476+0000 mgr.x (mgr.24118) 1244 : cluster [DBG] pgmap v1851: 17 pgs: 17 active+clean; 954 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 944 op/s 2020-09-08T16:08:50.228 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:08:50.647 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2020-09-08T16:08:50.786 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:08:51.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:51 smithi166 bash[9446]: cluster 2020-09-08T16:08:50.095999+0000 mgr.x (mgr.24118) 1245 : cluster [DBG] pgmap v1852: 17 pgs: 17 active+clean; 882 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T16:08:51.416 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:51 smithi055 bash[9296]: cluster 2020-09-08T16:08:50.095999+0000 mgr.x (mgr.24118) 1245 : cluster [DBG] pgmap v1852: 17 pgs: 17 active+clean; 882 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T16:08:51.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:51 smithi104 bash[9203]: cluster 2020-09-08T16:08:50.095999+0000 mgr.x (mgr.24118) 1245 : cluster [DBG] pgmap v1852: 17 pgs: 17 active+clean; 882 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T16:08:51.792 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:51.899 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:08:52.036 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:08:52.049 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:08:52 smithi166 bash[14254]: debug 2020-09-08T16:08:52.030+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:52.050 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:08:51 smithi166 bash[18960]: debug 2020-09-08T16:08:51.790+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:52.140 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:52.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:52 smithi104 bash[9203]: cluster 2020-09-08T16:08:51.214710+0000 osd.6 (osd.6) 115 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:52.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:52 smithi104 bash[9203]: cluster 2020-09-08T16:08:51.214792+0000 osd.6 (osd.6) 116 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:52.276 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:52.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:52 smithi166 bash[9446]: cluster 2020-09-08T16:08:51.214710+0000 osd.6 (osd.6) 115 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:52.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:52 smithi166 bash[9446]: cluster 2020-09-08T16:08:51.214792+0000 osd.6 (osd.6) 116 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:52.368 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:52 smithi055 bash[9296]: cluster 2020-09-08T16:08:51.214710+0000 osd.6 (osd.6) 115 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:52.368 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:52 smithi055 bash[9296]: cluster 2020-09-08T16:08:51.214792+0000 osd.6 (osd.6) 116 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:52.368 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:08:52 smithi055 bash[14301]: debug 2020-09-08T16:08:52.272+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:52.380 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:52.500 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:52.606 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:52.655 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:08:52 smithi055 bash[16646]: debug 2020-09-08T16:08:52.500+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:53.065 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T16:08:53.202 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:53.311 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:53.319 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:53 smithi166 bash[9446]: cluster 2020-09-08T16:08:52.050493+0000 mon.a (mon.0) 2188 : cluster [DBG] osdmap e692: 12 total, 12 up, 12 in 2020-09-08T16:08:53.320 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:53 smithi166 bash[9446]: cluster 2020-09-08T16:08:52.096436+0000 mgr.x (mgr.24118) 1246 : cluster [DBG] pgmap v1854: 17 pgs: 17 active+clean; 867 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 564 op/s 2020-09-08T16:08:53.320 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:53 smithi166 bash[9446]: audit 2020-09-08T16:08:52.713019+0000 mon.a (mon.0) 2189 : audit [INF] from='client.? v1:172.21.15.104:0/880519447' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:08:53.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:53 smithi055 bash[9296]: cluster 2020-09-08T16:08:52.050493+0000 mon.a (mon.0) 2188 : cluster [DBG] osdmap e692: 12 total, 12 up, 12 in 2020-09-08T16:08:53.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:53 smithi055 bash[9296]: cluster 2020-09-08T16:08:52.096436+0000 mgr.x (mgr.24118) 1246 : cluster [DBG] pgmap v1854: 17 pgs: 17 active+clean; 867 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 564 op/s 2020-09-08T16:08:53.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:53 smithi055 bash[9296]: audit 2020-09-08T16:08:52.713019+0000 mon.a (mon.0) 2189 : audit [INF] from='client.? v1:172.21.15.104:0/880519447' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:08:53.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:08:53 smithi104 bash[22305]: debug 2020-09-08T16:08:53.202+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:53.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:53 smithi104 bash[9203]: cluster 2020-09-08T16:08:52.050493+0000 mon.a (mon.0) 2188 : cluster [DBG] osdmap e692: 12 total, 12 up, 12 in 2020-09-08T16:08:53.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:53 smithi104 bash[9203]: cluster 2020-09-08T16:08:52.096436+0000 mgr.x (mgr.24118) 1246 : cluster [DBG] pgmap v1854: 17 pgs: 17 active+clean; 867 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 564 op/s 2020-09-08T16:08:53.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:53 smithi104 bash[9203]: audit 2020-09-08T16:08:52.713019+0000 mon.a (mon.0) 2189 : audit [INF] from='client.? v1:172.21.15.104:0/880519447' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:08:53.461 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:53.563 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:08:53.571 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:08:53 smithi166 bash[11928]: debug 2020-09-08T16:08:53.458+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:53.687 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:08:53.820 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:08:53.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:08:53 smithi166 bash[14254]: debug 2020-09-08T16:08:53.682+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:54.073 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:08:54.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:08:54 smithi055 bash[11956]: debug 2020-09-08T16:08:54.072+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:54.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:54 smithi055 bash[9296]: cluster 2020-09-08T16:08:53.054333+0000 mon.a (mon.0) 2190 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:08:54.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:54 smithi055 bash[9296]: cluster 2020-09-08T16:08:53.054373+0000 mon.a (mon.0) 2191 : cluster [INF] Cluster is now healthy 2020-09-08T16:08:54.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:54 smithi055 bash[9296]: audit 2020-09-08T16:08:53.063505+0000 mon.a (mon.0) 2192 : audit [INF] from='client.? v1:172.21.15.104:0/880519447' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:08:54.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:54 smithi055 bash[9296]: cluster 2020-09-08T16:08:53.063540+0000 mon.a (mon.0) 2193 : cluster [DBG] osdmap e693: 12 total, 12 up, 12 in 2020-09-08T16:08:54.181 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:54.310 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:54.315 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:54 smithi166 bash[9446]: cluster 2020-09-08T16:08:53.054333+0000 mon.a (mon.0) 2190 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:08:54.315 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:54 smithi166 bash[9446]: cluster 2020-09-08T16:08:53.054373+0000 mon.a (mon.0) 2191 : cluster [INF] Cluster is now healthy 2020-09-08T16:08:54.316 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:54 smithi166 bash[9446]: audit 2020-09-08T16:08:53.063505+0000 mon.a (mon.0) 2192 : audit [INF] from='client.? v1:172.21.15.104:0/880519447' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:08:54.316 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:54 smithi166 bash[9446]: cluster 2020-09-08T16:08:53.063540+0000 mon.a (mon.0) 2193 : cluster [DBG] osdmap e693: 12 total, 12 up, 12 in 2020-09-08T16:08:54.418 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:08:54.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:54 smithi104 bash[9203]: cluster 2020-09-08T16:08:53.054333+0000 mon.a (mon.0) 2190 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:08:54.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:54 smithi104 bash[9203]: cluster 2020-09-08T16:08:53.054373+0000 mon.a (mon.0) 2191 : cluster [INF] Cluster is now healthy 2020-09-08T16:08:54.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:54 smithi104 bash[9203]: audit 2020-09-08T16:08:53.063505+0000 mon.a (mon.0) 2192 : audit [INF] from='client.? v1:172.21.15.104:0/880519447' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:08:54.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:54 smithi104 bash[9203]: cluster 2020-09-08T16:08:53.063540+0000 mon.a (mon.0) 2193 : cluster [DBG] osdmap e693: 12 total, 12 up, 12 in 2020-09-08T16:08:54.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:08:54 smithi166 bash[16593]: debug 2020-09-08T16:08:54.306+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:54.678 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:08:54.780 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:54.898 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:54.914 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:08:54 smithi055 bash[19003]: debug 2020-09-08T16:08:54.664+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:55.003 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:55.077 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:55 smithi166 bash[9446]: cluster 2020-09-08T16:08:54.096876+0000 mgr.x (mgr.24118) 1247 : cluster [DBG] pgmap v1856: 17 pgs: 17 active+clean; 606 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.11k op/s 2020-09-08T16:08:55.078 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:55 smithi166 bash[9446]: cluster 2020-09-08T16:08:54.241947+0000 osd.11 (osd.11) 126 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:55.078 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:55 smithi166 bash[9446]: cluster 2020-09-08T16:08:54.242043+0000 osd.11 (osd.11) 127 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:55.078 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:08:54 smithi166 bash[11928]: debug 2020-09-08T16:08:54.898+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:55.136 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:08:55.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:55 smithi104 bash[9203]: cluster 2020-09-08T16:08:54.096876+0000 mgr.x (mgr.24118) 1247 : cluster [DBG] pgmap v1856: 17 pgs: 17 active+clean; 606 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.11k op/s 2020-09-08T16:08:55.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:55 smithi104 bash[9203]: cluster 2020-09-08T16:08:54.241947+0000 osd.11 (osd.11) 126 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:55.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:55 smithi104 bash[9203]: cluster 2020-09-08T16:08:54.242043+0000 osd.11 (osd.11) 127 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:55.244 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:55.254 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:55 smithi055 bash[9296]: cluster 2020-09-08T16:08:54.096876+0000 mgr.x (mgr.24118) 1247 : cluster [DBG] pgmap v1856: 17 pgs: 17 active+clean; 606 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.11k op/s 2020-09-08T16:08:55.254 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:55 smithi055 bash[9296]: cluster 2020-09-08T16:08:54.241947+0000 osd.11 (osd.11) 126 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:08:55.254 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:55 smithi055 bash[9296]: cluster 2020-09-08T16:08:54.242043+0000 osd.11 (osd.11) 127 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:08:55.364 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:08:55.364 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T16:08:55.364 INFO:teuthology.orchestra.run.smithi055.stdout: "complaint_time": 30, 2020-09-08T16:08:55.364 INFO:teuthology.orchestra.run.smithi055.stdout: "num_blocked_ops": 0 2020-09-08T16:08:55.365 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:08:55.370 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:08:55 smithi166 bash[11928]: debug 2020-09-08T16:08:55.134+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:55.370 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:55.475 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:08:55.611 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:08:55 smithi104 bash[19922]: debug 2020-09-08T16:08:55.614+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:55.614 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:08:55.642 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:08:55.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:08:55 smithi055 bash[16646]: debug 2020-09-08T16:08:55.368+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:55.674 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:08:55.717 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:08:55.719 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:55.758 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:08:55.789 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:08:55.828 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:08:55.856 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:08:55.887 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:08:55.920 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:08:55.954 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:08:55.986 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:08:56.018 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:08:56.019 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:08:56.019 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:08:56.019 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:08:56.019 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:08:56.019 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:08:56.020 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:08:56.020 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:08:56.020 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:08:56.020 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:08:56.020 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:08:56.021 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:08:56.021 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:08:56.070 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:56 smithi055 bash[9296]: cluster 2020-09-08T16:08:55.075017+0000 mon.a (mon.0) 2194 : cluster [DBG] osdmap e694: 12 total, 12 up, 12 in 2020-09-08T16:08:56.071 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:56 smithi055 bash[9296]: audit 2020-09-08T16:08:55.611515+0000 mon.b (mon.1) 120 : audit [DBG] from='client.? v1:172.21.15.104:0/2935380765' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:08:56.226 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:56.331 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:56.341 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:08:56 smithi055 bash[16646]: debug 2020-09-08T16:08:56.228+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:56.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:56 smithi166 bash[9446]: cluster 2020-09-08T16:08:55.075017+0000 mon.a (mon.0) 2194 : cluster [DBG] osdmap e694: 12 total, 12 up, 12 in 2020-09-08T16:08:56.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:56 smithi166 bash[9446]: audit 2020-09-08T16:08:55.611515+0000 mon.b (mon.1) 120 : audit [DBG] from='client.? v1:172.21.15.104:0/2935380765' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:08:56.373 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2020-09-08T16:08:56.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:56 smithi104 bash[9203]: cluster 2020-09-08T16:08:55.075017+0000 mon.a (mon.0) 2194 : cluster [DBG] osdmap e694: 12 total, 12 up, 12 in 2020-09-08T16:08:56.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:56 smithi104 bash[9203]: audit 2020-09-08T16:08:55.611515+0000 mon.b (mon.1) 120 : audit [DBG] from='client.? v1:172.21.15.104:0/2935380765' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:08:56.437 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:56.541 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:56.564 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:08:56.658 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:08:56 smithi055 bash[14301]: debug 2020-09-08T16:08:56.440+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:56.875 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T16:08:57.040 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:08:57.115 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:08:57.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:08:57 smithi104 bash[22305]: debug 2020-09-08T16:08:57.114+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:57.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:57 smithi104 bash[9203]: cluster 2020-09-08T16:08:56.097352+0000 mgr.x (mgr.24118) 1248 : cluster [DBG] pgmap v1858: 17 pgs: 17 active+clean; 606 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 736 op/s 2020-09-08T16:08:57.221 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:57.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:57 smithi166 bash[9446]: cluster 2020-09-08T16:08:56.097352+0000 mgr.x (mgr.24118) 1248 : cluster [DBG] pgmap v1858: 17 pgs: 17 active+clean; 606 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 736 op/s 2020-09-08T16:08:57.389 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:08:57.389 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:57 smithi055 bash[9296]: cluster 2020-09-08T16:08:56.097352+0000 mgr.x (mgr.24118) 1248 : cluster [DBG] pgmap v1858: 17 pgs: 17 active+clean; 606 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 736 op/s 2020-09-08T16:08:57.494 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:57.634 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:57.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:08:57 smithi055 bash[16646]: debug 2020-09-08T16:08:57.392+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:57.741 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:57.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:08:57 smithi166 bash[18960]: debug 2020-09-08T16:08:57.634+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:57.880 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:08:57.984 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:58.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:08:57 smithi166 bash[18960]: debug 2020-09-08T16:08:57.874+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:58.414 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:08:58.516 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:58.655 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:08:58.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:08:58 smithi104 bash[15273]: debug 2020-09-08T16:08:58.414+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:58.758 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:58.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:08:58 smithi166 bash[16593]: debug 2020-09-08T16:08:58.650+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:59.271 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:08:59.375 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:08:59.398 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:08:59 smithi104 bash[9203]: cluster 2020-09-08T16:08:58.098073+0000 mgr.x (mgr.24118) 1249 : cluster [DBG] pgmap v1859: 17 pgs: 17 active+clean; 389 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T16:08:59.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:08:59 smithi055 bash[9296]: cluster 2020-09-08T16:08:58.098073+0000 mgr.x (mgr.24118) 1249 : cluster [DBG] pgmap v1859: 17 pgs: 17 active+clean; 389 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T16:08:59.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:08:59 smithi055 bash[14301]: debug 2020-09-08T16:08:59.272+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:08:59.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:08:59 smithi166 bash[9446]: cluster 2020-09-08T16:08:58.098073+0000 mgr.x (mgr.24118) 1249 : cluster [DBG] pgmap v1859: 17 pgs: 17 active+clean; 389 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.26k op/s 2020-09-08T16:08:59.824 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:08:59.982 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:09:00.111 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:09:00 smithi104 bash[19922]: debug 2020-09-08T16:09:00.114+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:00.114 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:09:00.224 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:00.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:00 smithi055 bash[9296]: cluster 2020-09-08T16:08:58.494124+0000 osd.1 (osd.1) 142 : cluster [DBG] 8.1 deep-scrub starts 2020-09-08T16:09:01.206 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:00 smithi166 bash[9446]: cluster 2020-09-08T16:08:58.494124+0000 osd.1 (osd.1) 142 : cluster [DBG] 8.1 deep-scrub starts 2020-09-08T16:09:01.206 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:00 smithi104 bash[9203]: cluster 2020-09-08T16:08:58.494124+0000 osd.1 (osd.1) 142 : cluster [DBG] 8.1 deep-scrub starts 2020-09-08T16:09:01.207 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:01 smithi104 bash[9203]: cluster 2020-09-08T16:09:00.098538+0000 mgr.x (mgr.24118) 1250 : cluster [DBG] pgmap v1860: 17 pgs: 17 active+clean; 314 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T16:09:01.207 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:01 smithi104 bash[9203]: cluster 2020-09-08T16:09:00.311610+0000 osd.4 (osd.4) 75 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:01.207 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:01 smithi104 bash[9203]: cluster 2020-09-08T16:09:00.311717+0000 osd.4 (osd.4) 76 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:01.207 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:01.208 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:09:00 smithi104 bash[15273]: debug 2020-09-08T16:09:00.778+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:01.208 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T16:09:01.208 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:09:01.315 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:01.328 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:01 smithi055 bash[9296]: cluster 2020-09-08T16:09:00.098538+0000 mgr.x (mgr.24118) 1250 : cluster [DBG] pgmap v1860: 17 pgs: 17 active+clean; 314 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T16:09:01.328 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:01 smithi055 bash[9296]: cluster 2020-09-08T16:09:00.311610+0000 osd.4 (osd.4) 75 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:01.329 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:01 smithi055 bash[9296]: cluster 2020-09-08T16:09:00.311717+0000 osd.4 (osd.4) 76 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:01.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:01 smithi166 bash[9446]: cluster 2020-09-08T16:09:00.098538+0000 mgr.x (mgr.24118) 1250 : cluster [DBG] pgmap v1860: 17 pgs: 17 active+clean; 314 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T16:09:01.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:01 smithi166 bash[9446]: cluster 2020-09-08T16:09:00.311610+0000 osd.4 (osd.4) 75 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:01.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:01 smithi166 bash[9446]: cluster 2020-09-08T16:09:00.311717+0000 osd.4 (osd.4) 76 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:01.677 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:01.782 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:09:01.825 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2020-09-08T16:09:01.912 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:09:02.013 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:09:02.016 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:02.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:09:01 smithi055 bash[16646]: debug 2020-09-08T16:09:01.680+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:02.158 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:09:01 smithi055 bash[19003]: debug 2020-09-08T16:09:01.912+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:02.344 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:02.344 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:02 smithi104 bash[9203]: cluster 2020-09-08T16:09:00.785869+0000 osd.1 (osd.1) 143 : cluster [DBG] 8.1 deep-scrub ok 2020-09-08T16:09:02.345 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:02 smithi104 bash[9203]: cluster 2020-09-08T16:09:01.163054+0000 mon.a (mon.0) 2195 : cluster [DBG] osdmap e695: 12 total, 12 up, 12 in 2020-09-08T16:09:02.345 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:09:02 smithi104 bash[15273]: debug 2020-09-08T16:09:02.346+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:02.447 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:02.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:02 smithi166 bash[9446]: cluster 2020-09-08T16:09:00.785869+0000 osd.1 (osd.1) 143 : cluster [DBG] 8.1 deep-scrub ok 2020-09-08T16:09:02.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:02 smithi166 bash[9446]: cluster 2020-09-08T16:09:01.163054+0000 mon.a (mon.0) 2195 : cluster [DBG] osdmap e695: 12 total, 12 up, 12 in 2020-09-08T16:09:02.632 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:02.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:02 smithi055 bash[9296]: cluster 2020-09-08T16:09:00.785869+0000 osd.1 (osd.1) 143 : cluster [DBG] 8.1 deep-scrub ok 2020-09-08T16:09:02.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:02 smithi055 bash[9296]: cluster 2020-09-08T16:09:01.163054+0000 mon.a (mon.0) 2195 : cluster [DBG] osdmap e695: 12 total, 12 up, 12 in 2020-09-08T16:09:02.736 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:09:02.911 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:09:02 smithi104 bash[17588]: debug 2020-09-08T16:09:02.634+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:03.152 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:09:03.171 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:03 smithi104 bash[9203]: cluster 2020-09-08T16:09:02.098906+0000 mgr.x (mgr.24118) 1251 : cluster [DBG] pgmap v1862: 17 pgs: 17 active+clean; 296 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 621 op/s 2020-09-08T16:09:03.257 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:03.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:03 smithi055 bash[9296]: cluster 2020-09-08T16:09:02.098906+0000 mgr.x (mgr.24118) 1251 : cluster [DBG] pgmap v1862: 17 pgs: 17 active+clean; 296 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 621 op/s 2020-09-08T16:09:03.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:09:03 smithi055 bash[11956]: debug 2020-09-08T16:09:03.156+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:03.432 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:03.487 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Removed 130918 objects 2020-09-08T16:09:03.487 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Clean up completed and total clean up time :145.595 2020-09-08T16:09:03.534 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:03.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:03 smithi166 bash[9446]: cluster 2020-09-08T16:09:02.098906+0000 mgr.x (mgr.24118) 1251 : cluster [DBG] pgmap v1862: 17 pgs: 17 active+clean; 296 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 621 op/s 2020-09-08T16:09:03.873 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:09:03 smithi055 bash[16646]: debug 2020-09-08T16:09:03.432+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:04.161 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:04.182 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T16:09:04.266 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:04.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:04 smithi104 bash[9203]: audit 2020-09-08T16:09:03.741369+0000 mon.a (mon.0) 2196 : audit [INF] from='client.? v1:172.21.15.104:0/136602906' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:09:04.427 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:09:04 smithi104 bash[15273]: debug 2020-09-08T16:09:04.162+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:04.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:04 smithi166 bash[9446]: audit 2020-09-08T16:09:03.741369+0000 mon.a (mon.0) 2196 : audit [INF] from='client.? v1:172.21.15.104:0/136602906' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:09:04.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:04 smithi055 bash[9296]: audit 2020-09-08T16:09:03.741369+0000 mon.a (mon.0) 2196 : audit [INF] from='client.? v1:172.21.15.104:0/136602906' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:09:04.688 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:04.793 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:09:04.815 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_6 2020-09-08T16:09:04.817 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool rm unique_pool_6 unique_pool_6 --yes-i-really-really-mean-it 2020-09-08T16:09:04.954 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:09:05.131 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:09:05.169 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:09:04 smithi104 bash[17588]: debug 2020-09-08T16:09:04.686+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:05.256 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:09:05.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:09:05 smithi055 bash[19003]: debug 2020-09-08T16:09:05.132+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:05.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:05 smithi055 bash[9296]: cluster 2020-09-08T16:09:04.099734+0000 mgr.x (mgr.24118) 1252 : cluster [DBG] pgmap v1863: 17 pgs: 17 active+clean; 42 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1k op/s 2020-09-08T16:09:05.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:05 smithi055 bash[9296]: cluster 2020-09-08T16:09:04.170226+0000 mon.a (mon.0) 2197 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:09:05.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:05 smithi055 bash[9296]: audit 2020-09-08T16:09:04.183688+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? v1:172.21.15.104:0/136602906' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:09:05.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:05 smithi055 bash[9296]: cluster 2020-09-08T16:09:04.183767+0000 mon.a (mon.0) 2199 : cluster [DBG] osdmap e696: 12 total, 12 up, 12 in 2020-09-08T16:09:05.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:05 smithi104 bash[9203]: cluster 2020-09-08T16:09:04.099734+0000 mgr.x (mgr.24118) 1252 : cluster [DBG] pgmap v1863: 17 pgs: 17 active+clean; 42 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1k op/s 2020-09-08T16:09:05.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:05 smithi104 bash[9203]: cluster 2020-09-08T16:09:04.170226+0000 mon.a (mon.0) 2197 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:09:05.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:05 smithi104 bash[9203]: audit 2020-09-08T16:09:04.183688+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? v1:172.21.15.104:0/136602906' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:09:05.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:05 smithi104 bash[9203]: cluster 2020-09-08T16:09:04.183767+0000 mon.a (mon.0) 2199 : cluster [DBG] osdmap e696: 12 total, 12 up, 12 in 2020-09-08T16:09:05.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:05 smithi166 bash[9446]: cluster 2020-09-08T16:09:04.099734+0000 mgr.x (mgr.24118) 1252 : cluster [DBG] pgmap v1863: 17 pgs: 17 active+clean; 42 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1k op/s 2020-09-08T16:09:05.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:05 smithi166 bash[9446]: cluster 2020-09-08T16:09:04.170226+0000 mon.a (mon.0) 2197 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:09:05.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:05 smithi166 bash[9446]: audit 2020-09-08T16:09:04.183688+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? v1:172.21.15.104:0/136602906' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:09:05.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:05 smithi166 bash[9446]: cluster 2020-09-08T16:09:04.183767+0000 mon.a (mon.0) 2199 : cluster [DBG] osdmap e696: 12 total, 12 up, 12 in 2020-09-08T16:09:05.802 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:09:05.911 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:09:06.096 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:09:05 smithi055 bash[19003]: debug 2020-09-08T16:09:05.804+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:06.214 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:09:06.321 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:06.395 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T16:09:06.395 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:09:06.463 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:06.554 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:09:06 smithi104 bash[19922]: debug 2020-09-08T16:09:06.214+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:06.569 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:06.619 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:09:06 smithi166 bash[11928]: debug 2020-09-08T16:09:06.461+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:06.798 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:09:06.825 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:09:06.850 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:09:06.879 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:09:06.903 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:09:06.931 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:09:06.957 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:09:06.984 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:09:07.017 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:09:07.045 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:09:07.055 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:07.070 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:09:07.096 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:09:07.098 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:09:07.098 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:09:07.098 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:09:07.099 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:09:07.099 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:09:07.099 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:09:07.099 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:09:07.099 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:09:07.100 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:09:07.100 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:09:07.100 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:09:07.100 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:09:07.160 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:09:07.184 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2020-09-08T16:09:07.280 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:09:07.354 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:09:07.384 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:07.398 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:07 smithi104 bash[9203]: cluster 2020-09-08T16:09:06.100548+0000 mgr.x (mgr.24118) 1253 : cluster [DBG] pgmap v1865: 17 pgs: 17 active+clean; 42 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 694 op/s 2020-09-08T16:09:07.398 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:07 smithi104 bash[9203]: audit 2020-09-08T16:09:06.772911+0000 mon.a (mon.0) 2200 : audit [DBG] from='client.? v1:172.21.15.104:0/3924433318' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:09:07.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:09:07 smithi055 bash[16646]: debug 2020-09-08T16:09:07.056+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:07.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:07 smithi055 bash[9296]: cluster 2020-09-08T16:09:06.100548+0000 mgr.x (mgr.24118) 1253 : cluster [DBG] pgmap v1865: 17 pgs: 17 active+clean; 42 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 694 op/s 2020-09-08T16:09:07.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:07 smithi055 bash[9296]: audit 2020-09-08T16:09:06.772911+0000 mon.a (mon.0) 2200 : audit [DBG] from='client.? v1:172.21.15.104:0/3924433318' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:09:07.408 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:09:07 smithi055 bash[11956]: debug 2020-09-08T16:09:07.280+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:07.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:07 smithi166 bash[9446]: cluster 2020-09-08T16:09:06.100548+0000 mgr.x (mgr.24118) 1253 : cluster [DBG] pgmap v1865: 17 pgs: 17 active+clean; 42 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 694 op/s 2020-09-08T16:09:07.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:07 smithi166 bash[9446]: audit 2020-09-08T16:09:06.772911+0000 mon.a (mon.0) 2200 : audit [DBG] from='client.? v1:172.21.15.104:0/3924433318' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:09:07.711 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:07.818 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:07.961 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:07.977 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T16:09:07.997 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:09:07 smithi104 bash[15273]: debug 2020-09-08T16:09:07.714+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:08.063 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:08.120 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:09:08.156 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:09:07 smithi055 bash[16646]: debug 2020-09-08T16:09:07.952+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:08.473 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:08.584 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:09:08.838 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:09:08 smithi104 bash[15273]: debug 2020-09-08T16:09:08.474+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:08.838 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:09:08.940 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:09.246 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:09:08 smithi104 bash[19922]: debug 2020-09-08T16:09:08.830+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:09.329 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:09.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:09 smithi055 bash[9296]: cluster 2020-09-08T16:09:08.101294+0000 mgr.x (mgr.24118) 1254 : cluster [DBG] pgmap v1866: 17 pgs: 17 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 628 op/s 2020-09-08T16:09:09.438 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:09:09.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:09 smithi166 bash[9446]: cluster 2020-09-08T16:09:08.101294+0000 mgr.x (mgr.24118) 1254 : cluster [DBG] pgmap v1866: 17 pgs: 17 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 628 op/s 2020-09-08T16:09:09.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:09 smithi104 bash[9203]: cluster 2020-09-08T16:09:08.101294+0000 mgr.x (mgr.24118) 1254 : cluster [DBG] pgmap v1866: 17 pgs: 17 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 628 op/s 2020-09-08T16:09:09.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:09:09 smithi104 bash[15273]: debug 2020-09-08T16:09:09.330+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:09.708 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:09:09.814 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:09:09.991 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:09:10.010 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:09:09 smithi104 bash[22305]: debug 2020-09-08T16:09:09.710+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:10.102 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:09:10.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:09:09 smithi166 bash[18960]: debug 2020-09-08T16:09:09.989+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:10.254 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:09:10.355 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:09:10.368 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:09:10 smithi166 bash[18960]: debug 2020-09-08T16:09:10.249+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:10.498 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:09:10.611 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:09:10.624 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:09:10 smithi166 bash[16593]: debug 2020-09-08T16:09:10.497+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:10.754 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:09:10.839 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:09:10.840 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T16:09:10.840 INFO:teuthology.orchestra.run.smithi055.stdout: "complaint_time": 30, 2020-09-08T16:09:10.840 INFO:teuthology.orchestra.run.smithi055.stdout: "num_blocked_ops": 0 2020-09-08T16:09:10.840 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:09:10.848 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:09:10.860 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:09:10.986 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:09:11.030 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:09:11.031 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:09:10 smithi166 bash[16593]: debug 2020-09-08T16:09:10.753+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:11.045 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'unique_pool_6' removed 2020-09-08T16:09:11.121 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:11 smithi104 bash[9203]: cluster 2020-09-08T16:09:10.102092+0000 mgr.x (mgr.24118) 1255 : cluster [DBG] pgmap v1867: 17 pgs: 17 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 562 op/s 2020-09-08T16:09:11.122 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:11 smithi104 bash[9203]: cluster 2020-09-08T16:09:10.342987+0000 osd.9 (osd.9) 107 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:11.122 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:11 smithi104 bash[9203]: cluster 2020-09-08T16:09:10.343119+0000 osd.9 (osd.9) 108 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:11.122 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:11 smithi104 bash[9203]: audit 2020-09-08T16:09:10.350879+0000 mon.a (mon.0) 2201 : audit [INF] from='client.? v1:172.21.15.104:0/886058081' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_6", "pool2": "unique_pool_6", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T16:09:11.147 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:11.160 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:09:11 smithi055 bash[19003]: debug 2020-09-08T16:09:11.032+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:11.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:11 smithi055 bash[9296]: cluster 2020-09-08T16:09:10.102092+0000 mgr.x (mgr.24118) 1255 : cluster [DBG] pgmap v1867: 17 pgs: 17 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 562 op/s 2020-09-08T16:09:11.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:11 smithi055 bash[9296]: cluster 2020-09-08T16:09:10.342987+0000 osd.9 (osd.9) 107 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:11.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:11 smithi055 bash[9296]: cluster 2020-09-08T16:09:10.343119+0000 osd.9 (osd.9) 108 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:11.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:11 smithi055 bash[9296]: audit 2020-09-08T16:09:10.350879+0000 mon.a (mon.0) 2201 : audit [INF] from='client.? v1:172.21.15.104:0/886058081' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_6", "pool2": "unique_pool_6", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T16:09:11.261 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:11.365 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:09:11.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:09:11 smithi166 bash[11928]: debug 2020-09-08T16:09:11.261+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:11 smithi166 bash[9446]: cluster 2020-09-08T16:09:10.102092+0000 mgr.x (mgr.24118) 1255 : cluster [DBG] pgmap v1867: 17 pgs: 17 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 562 op/s 2020-09-08T16:09:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:11 smithi166 bash[9446]: cluster 2020-09-08T16:09:10.342987+0000 osd.9 (osd.9) 107 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:11 smithi166 bash[9446]: cluster 2020-09-08T16:09:10.343119+0000 osd.9 (osd.9) 108 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:11 smithi166 bash[9446]: audit 2020-09-08T16:09:10.350879+0000 mon.a (mon.0) 2201 : audit [INF] from='client.? v1:172.21.15.104:0/886058081' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_6", "pool2": "unique_pool_6", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T16:09:11.671 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:09:11.740 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2020-09-08T16:09:11.776 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:11.885 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:09:11.901 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:11.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:09:11 smithi104 bash[22305]: debug 2020-09-08T16:09:11.670+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:12.005 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:12.017 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:09:11 smithi055 bash[14301]: debug 2020-09-08T16:09:11.900+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:12.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:12 smithi166 bash[9446]: audit 2020-09-08T16:09:11.048385+0000 mon.a (mon.0) 2202 : audit [INF] from='client.? v1:172.21.15.104:0/886058081' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_6", "pool2": "unique_pool_6", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T16:09:12.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:12 smithi166 bash[9446]: cluster 2020-09-08T16:09:11.048445+0000 mon.a (mon.0) 2203 : cluster [DBG] osdmap e697: 12 total, 12 up, 12 in 2020-09-08T16:09:12.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:12 smithi166 bash[9446]: cluster 2020-09-08T16:09:11.666181+0000 osd.2 (osd.2) 110 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:12.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:12 smithi166 bash[9446]: cluster 2020-09-08T16:09:11.666302+0000 osd.2 (osd.2) 111 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:12.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:12 smithi055 bash[9296]: audit 2020-09-08T16:09:11.048385+0000 mon.a (mon.0) 2202 : audit [INF] from='client.? v1:172.21.15.104:0/886058081' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_6", "pool2": "unique_pool_6", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T16:09:12.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:12 smithi055 bash[9296]: cluster 2020-09-08T16:09:11.048445+0000 mon.a (mon.0) 2203 : cluster [DBG] osdmap e697: 12 total, 12 up, 12 in 2020-09-08T16:09:12.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:12 smithi055 bash[9296]: cluster 2020-09-08T16:09:11.666181+0000 osd.2 (osd.2) 110 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:12.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:12 smithi055 bash[9296]: cluster 2020-09-08T16:09:11.666302+0000 osd.2 (osd.2) 111 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:12.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:12 smithi104 bash[9203]: audit 2020-09-08T16:09:11.048385+0000 mon.a (mon.0) 2202 : audit [INF] from='client.? v1:172.21.15.104:0/886058081' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_6", "pool2": "unique_pool_6", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T16:09:12.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:12 smithi104 bash[9203]: cluster 2020-09-08T16:09:11.048445+0000 mon.a (mon.0) 2203 : cluster [DBG] osdmap e697: 12 total, 12 up, 12 in 2020-09-08T16:09:12.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:12 smithi104 bash[9203]: cluster 2020-09-08T16:09:11.666181+0000 osd.2 (osd.2) 110 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:12.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:12 smithi104 bash[9203]: cluster 2020-09-08T16:09:11.666302+0000 osd.2 (osd.2) 111 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:12.433 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:12.534 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:09:12.831 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2020-09-08T16:09:12.832 INFO:tasks.radosbench:Beginning radosbench... 2020-09-08T16:09:12.833 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_7 2020-09-08T16:09:12.834 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool create unique_pool_7 16 2020-09-08T16:09:12.906 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:09:12 smithi055 bash[14301]: debug 2020-09-08T16:09:12.428+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:12.968 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:09:12.982 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:09:13.089 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:09:13.107 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:09:12 smithi104 bash[22305]: debug 2020-09-08T16:09:12.982+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:13.107 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:13 smithi104 bash[9203]: cluster 2020-09-08T16:09:12.057813+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e698: 12 total, 12 up, 12 in 2020-09-08T16:09:13.107 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:13 smithi104 bash[9203]: cluster 2020-09-08T16:09:12.102699+0000 mgr.x (mgr.24118) 1256 : cluster [DBG] pgmap v1870: 1 pgs: 1 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:13.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:13 smithi166 bash[9446]: cluster 2020-09-08T16:09:12.057813+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e698: 12 total, 12 up, 12 in 2020-09-08T16:09:13.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:13 smithi166 bash[9446]: cluster 2020-09-08T16:09:12.102699+0000 mgr.x (mgr.24118) 1256 : cluster [DBG] pgmap v1870: 1 pgs: 1 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:13.380 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:13 smithi055 bash[9296]: cluster 2020-09-08T16:09:12.057813+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e698: 12 total, 12 up, 12 in 2020-09-08T16:09:13.381 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:13 smithi055 bash[9296]: cluster 2020-09-08T16:09:12.102699+0000 mgr.x (mgr.24118) 1256 : cluster [DBG] pgmap v1870: 1 pgs: 1 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:13.734 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:09:13.836 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:14.115 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:14.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:09:13 smithi104 bash[22305]: debug 2020-09-08T16:09:13.734+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:14.222 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:14.406 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:09:14 smithi055 bash[14301]: debug 2020-09-08T16:09:14.116+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:14.553 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:14.660 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:14.810 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:14.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:09:14 smithi055 bash[16646]: debug 2020-09-08T16:09:14.556+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:14.912 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:14.926 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:09:14 smithi166 bash[11928]: debug 2020-09-08T16:09:14.805+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:15.033 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:15.138 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:15.171 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T16:09:15.353 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:15.369 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:09:15 smithi166 bash[11928]: debug 2020-09-08T16:09:15.033+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:15.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:15 smithi166 bash[9446]: cluster 2020-09-08T16:09:14.103348+0000 mgr.x (mgr.24118) 1257 : cluster [DBG] pgmap v1871: 1 pgs: 1 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:15.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:15 smithi166 bash[9446]: audit 2020-09-08T16:09:14.813367+0000 mon.a (mon.0) 2205 : audit [INF] from='client.? v1:172.21.15.104:0/274585065' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:09:15.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:15 smithi055 bash[9296]: cluster 2020-09-08T16:09:14.103348+0000 mgr.x (mgr.24118) 1257 : cluster [DBG] pgmap v1871: 1 pgs: 1 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:15.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:15 smithi055 bash[9296]: audit 2020-09-08T16:09:14.813367+0000 mon.a (mon.0) 2205 : audit [INF] from='client.? v1:172.21.15.104:0/274585065' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:09:15.410 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:15 smithi104 bash[9203]: cluster 2020-09-08T16:09:14.103348+0000 mgr.x (mgr.24118) 1257 : cluster [DBG] pgmap v1871: 1 pgs: 1 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:15.410 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:15 smithi104 bash[9203]: audit 2020-09-08T16:09:14.813367+0000 mon.a (mon.0) 2205 : audit [INF] from='client.? v1:172.21.15.104:0/274585065' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:09:15.410 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:09:15 smithi104 bash[15273]: debug 2020-09-08T16:09:15.350+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:15.456 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:15.583 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:15.691 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:09:15.803 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:09:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "size": 20, 2020-09-08T16:09:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 600, 2020-09-08T16:09:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T16:09:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:132310 7.0 7:08a563c3:::benchmark_data_smithi166_48533_object132309:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e582)", 2020-09-08T16:09:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:02:08.118745+0000", 2020-09-08T16:09:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 427.68536168600002, 2020-09-08T16:09:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.32255850899999999, 2020-09-08T16:09:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:09:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:09:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 132310 2020-09-08T16:09:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.118745+0000", 2020-09-08T16:09:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.118745+0000", 2020-09-08T16:09:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999309 2020-09-08T16:09:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.118676+0000", 2020-09-08T16:09:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00021206 2020-09-08T16:09:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.118888+0000", 2020-09-08T16:09:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.5772999999999999e-05 2020-09-08T16:09:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.118914+0000", 2020-09-08T16:09:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1042e-05 2020-09-08T16:09:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.118925+0000", 2020-09-08T16:09:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.6842000000000006e-05 2020-09-08T16:09:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.119011+0000", 2020-09-08T16:09:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000188061 2020-09-08T16:09:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.119199+0000", 2020-09-08T16:09:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00024967400000000002 2020-09-08T16:09:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.818 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 9", 2020-09-08T16:09:15.818 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.119449+0000", 2020-09-08T16:09:15.818 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.30866220500000002 2020-09-08T16:09:15.818 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.818 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.428111+0000", 2020-09-08T16:09:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.013081615 2020-09-08T16:09:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.820 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:15.820 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.441193+0000", 2020-09-08T16:09:15.820 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.4659999999999993e-05 2020-09-08T16:09:15.820 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.820 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.821 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:15.821 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.441268+0000", 2020-09-08T16:09:15.821 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.5871999999999998e-05 2020-09-08T16:09:15.821 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.821 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.822 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:15.822 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.441303+0000", 2020-09-08T16:09:15.822 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.32255850899999999 2020-09-08T16:09:15.822 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.822 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:15.823 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.823 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.823 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.823 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:132314 7.0 7:01256802:::benchmark_data_smithi166_48533_object132313:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e582)", 2020-09-08T16:09:15.823 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:02:08.119704+0000", 2020-09-08T16:09:15.823 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 427.68440255199999, 2020-09-08T16:09:15.824 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.326628003, 2020-09-08T16:09:15.824 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:15.824 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:15.824 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:15.824 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:09:15.825 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:09:15.825 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 132314 2020-09-08T16:09:15.825 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.825 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:15.825 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.826 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:15.826 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.119704+0000", 2020-09-08T16:09:15.826 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:15.826 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.826 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.119704+0000", 2020-09-08T16:09:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999328 2020-09-08T16:09:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.119637+0000", 2020-09-08T16:09:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00020777299999999999 2020-09-08T16:09:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:15.829 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.119845+0000", 2020-09-08T16:09:15.829 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.3206000000000001e-05 2020-09-08T16:09:15.829 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.829 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.829 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:15.830 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.119868+0000", 2020-09-08T16:09:15.830 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.696e-06 2020-09-08T16:09:15.830 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.830 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.830 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:15.831 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.119878+0000", 2020-09-08T16:09:15.831 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.30908945100000002 2020-09-08T16:09:15.831 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.831 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.831 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:15.831 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.428967+0000", 2020-09-08T16:09:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00044533700000000002 2020-09-08T16:09:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.429413+0000", 2020-09-08T16:09:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000532026 2020-09-08T16:09:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 9", 2020-09-08T16:09:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.429945+0000", 2020-09-08T16:09:15.834 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.011417996999999999 2020-09-08T16:09:15.834 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.834 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.834 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:15.834 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.441363+0000", 2020-09-08T16:09:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0047973770000000002 2020-09-08T16:09:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.446160+0000", 2020-09-08T16:09:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010149000000000001 2020-09-08T16:09:15.836 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.836 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.836 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:15.836 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.446261+0000", 2020-09-08T16:09:15.836 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.0711999999999997e-05 2020-09-08T16:09:15.837 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.837 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.837 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:15.837 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.446332+0000", 2020-09-08T16:09:15.837 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.326628003 2020-09-08T16:09:15.838 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.838 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:15.838 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.838 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.839 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.839 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:220752 7.0 7:0f8d1be1:::benchmark_data_smithi166_48533_object86235:head [delete] snapc 0=[] ondisk+write+known_if_redirected e605)", 2020-09-08T16:09:15.839 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:38.031589+0000", 2020-09-08T16:09:15.839 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 337.77251815099999, 2020-09-08T16:09:15.839 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.118256688, 2020-09-08T16:09:15.840 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:15.840 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:15.840 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:15.840 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:09:15.840 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:09:15.841 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 220752 2020-09-08T16:09:15.841 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.841 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:15.841 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.841 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031589+0000", 2020-09-08T16:09:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031589+0000", 2020-09-08T16:09:15.843 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999728 2020-09-08T16:09:15.843 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.843 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.843 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:15.843 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031561+0000", 2020-09-08T16:09:15.844 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0479000000000002e-05 2020-09-08T16:09:15.844 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.844 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.844 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:15.844 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031622+0000", 2020-09-08T16:09:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0919e-05 2020-09-08T16:09:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031633+0000", 2020-09-08T16:09:15.846 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4739999999999996e-06 2020-09-08T16:09:15.846 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.846 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.846 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:15.846 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031638+0000", 2020-09-08T16:09:15.847 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2747999999999997e-05 2020-09-08T16:09:15.847 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.847 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.847 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:15.847 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031691+0000", 2020-09-08T16:09:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011123799999999999 2020-09-08T16:09:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031802+0000", 2020-09-08T16:09:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.8571000000000002e-05 2020-09-08T16:09:15.849 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.849 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.849 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 9", 2020-09-08T16:09:15.849 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031901+0000", 2020-09-08T16:09:15.849 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.10433806499999999 2020-09-08T16:09:15.850 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.850 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.850 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:15.850 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.136239+0000", 2020-09-08T16:09:15.850 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.013515574000000001 2020-09-08T16:09:15.851 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.851 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.851 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:15.851 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.149754+0000", 2020-09-08T16:09:15.851 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.0717999999999998e-05 2020-09-08T16:09:15.852 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.852 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.852 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:15.852 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.149805+0000", 2020-09-08T16:09:15.852 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.0042e-05 2020-09-08T16:09:15.852 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.853 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.853 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:15.853 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.149845+0000", 2020-09-08T16:09:15.853 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.118256688 2020-09-08T16:09:15.853 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.854 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:15.854 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.854 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.854 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.854 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:220754 7.0 7:02b4744d:::benchmark_data_smithi166_48533_object86237:head [delete] snapc 0=[] ondisk+write+known_if_redirected e605)", 2020-09-08T16:09:15.855 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:38.031703+0000", 2020-09-08T16:09:15.855 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 337.77240319200001, 2020-09-08T16:09:15.855 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.122364417, 2020-09-08T16:09:15.855 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:15.855 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:09:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:09:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 220754 2020-09-08T16:09:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031703+0000", 2020-09-08T16:09:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.858 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.858 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:15.858 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031703+0000", 2020-09-08T16:09:15.858 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999738 2020-09-08T16:09:15.858 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031677+0000", 2020-09-08T16:09:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.5869000000000003e-05 2020-09-08T16:09:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.860 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:15.860 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031733+0000", 2020-09-08T16:09:15.860 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2830000000000001e-06 2020-09-08T16:09:15.860 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.860 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.861 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:15.861 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031741+0000", 2020-09-08T16:09:15.861 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.7479999999999999e-06 2020-09-08T16:09:15.861 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.861 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.862 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:15.862 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.031745+0000", 2020-09-08T16:09:15.862 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0043302230000000002 2020-09-08T16:09:15.862 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.862 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.036075+0000", 2020-09-08T16:09:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.3675e-05 2020-09-08T16:09:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:15.864 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.036169+0000", 2020-09-08T16:09:15.864 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000138276 2020-09-08T16:09:15.864 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.864 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.864 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 9", 2020-09-08T16:09:15.865 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.036307+0000", 2020-09-08T16:09:15.865 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.099943807999999995 2020-09-08T16:09:15.865 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.865 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.865 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:15.866 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.136251+0000", 2020-09-08T16:09:15.866 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.017666213 2020-09-08T16:09:15.866 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.866 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.866 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:15.866 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.153917+0000", 2020-09-08T16:09:15.867 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4060000000000005e-05 2020-09-08T16:09:15.867 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.867 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.867 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:15.867 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.154001+0000", 2020-09-08T16:09:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.6455e-05 2020-09-08T16:09:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:38.154068+0000", 2020-09-08T16:09:15.869 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.122364417 2020-09-08T16:09:15.869 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.869 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:15.869 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.869 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.870 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.870 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36335.0:131908 8.8 8:18ed8ba4:::benchmark_data_smithi166_52302_object987:head [delete] snapc 0=[] ondisk+write+known_if_redirected e660)", 2020-09-08T16:09:15.870 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:06:38.807843+0000", 2020-09-08T16:09:15.870 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 156.996263191, 2020-09-08T16:09:15.870 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.14121747500000001, 2020-09-08T16:09:15.870 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36335", 2020-09-08T16:09:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:09:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 131908 2020-09-08T16:09:15.872 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.872 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:15.872 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:06:38.807843+0000", 2020-09-08T16:09:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.874 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:15.874 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:06:38.807843+0000", 2020-09-08T16:09:15.874 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999642 2020-09-08T16:09:15.875 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.875 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:06:38.807808+0000", 2020-09-08T16:09:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.6385999999999997e-05 2020-09-08T16:09:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.878 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:15.878 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:06:38.807884+0000", 2020-09-08T16:09:15.878 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3393999999999999e-05 2020-09-08T16:09:15.878 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.878 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:06:38.807897+0000", 2020-09-08T16:09:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.7329999999999997e-06 2020-09-08T16:09:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:15.880 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:06:38.807904+0000", 2020-09-08T16:09:15.880 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.3957999999999999e-05 2020-09-08T16:09:15.880 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.880 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.880 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:15.881 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:06:38.807978+0000", 2020-09-08T16:09:15.881 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015903299999999999 2020-09-08T16:09:15.881 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.881 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.881 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:15.882 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:06:38.808137+0000", 2020-09-08T16:09:15.882 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000110005 2020-09-08T16:09:15.882 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.882 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.882 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:09:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:06:38.808247+0000", 2020-09-08T16:09:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0048694180000000004 2020-09-08T16:09:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:06:38.813116+0000", 2020-09-08T16:09:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.13573106500000001 2020-09-08T16:09:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:06:38.948848+0000", 2020-09-08T16:09:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011634200000000001 2020-09-08T16:09:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:06:38.948964+0000", 2020-09-08T16:09:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.7078999999999997e-05 2020-09-08T16:09:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:06:38.949061+0000", 2020-09-08T16:09:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.14121747500000001 2020-09-08T16:09:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36335.0:153730 8.8 8:16775da7:::benchmark_data_smithi166_52302_object22809:head [delete] snapc 0=[] ondisk+write+known_if_redirected e665)", 2020-09-08T16:09:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:07:01.329804+0000", 2020-09-08T16:09:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 134.474303123, 2020-09-08T16:09:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.104676462, 2020-09-08T16:09:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:15.889 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:15.889 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:15.889 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36335", 2020-09-08T16:09:15.889 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:09:15.889 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 153730 2020-09-08T16:09:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.329804+0000", 2020-09-08T16:09:15.891 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:15.891 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.891 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.891 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:15.891 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.329804+0000", 2020-09-08T16:09:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999585 2020-09-08T16:09:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.329762+0000", 2020-09-08T16:09:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.5644999999999997e-05 2020-09-08T16:09:15.893 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.893 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.893 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:15.893 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.329848+0000", 2020-09-08T16:09:15.893 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6858999999999999e-05 2020-09-08T16:09:15.894 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.894 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.894 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:15.894 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.329865+0000", 2020-09-08T16:09:15.894 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.0949999999999998e-06 2020-09-08T16:09:15.895 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.895 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.895 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:15.895 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.329872+0000", 2020-09-08T16:09:15.895 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0054121689999999997 2020-09-08T16:09:15.896 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.896 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.896 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:15.896 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.335284+0000", 2020-09-08T16:09:15.896 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000108293 2020-09-08T16:09:15.896 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.897 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.897 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:15.897 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.335392+0000", 2020-09-08T16:09:15.897 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.6045999999999996e-05 2020-09-08T16:09:15.897 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.898 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.898 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:09:15.898 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.335478+0000", 2020-09-08T16:09:15.898 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0094298839999999995 2020-09-08T16:09:15.898 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.344908+0000", 2020-09-08T16:09:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.089494052000000004 2020-09-08T16:09:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.900 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:15.900 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.434402+0000", 2020-09-08T16:09:15.900 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.3137999999999999e-05 2020-09-08T16:09:15.900 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.900 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.901 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:15.901 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.434445+0000", 2020-09-08T16:09:15.901 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.4709999999999998e-05 2020-09-08T16:09:15.901 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.901 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.902 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:15.902 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.434480+0000", 2020-09-08T16:09:15.902 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.104676462 2020-09-08T16:09:15.902 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.902 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:15.902 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.903 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.903 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.903 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36335.0:153735 8.8 8:13228b58:::benchmark_data_smithi166_52302_object22814:head [delete] snapc 0=[] ondisk+write+known_if_redirected e665)", 2020-09-08T16:09:15.903 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:07:01.333395+0000", 2020-09-08T16:09:15.903 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 134.470712056, 2020-09-08T16:09:15.904 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.106883512, 2020-09-08T16:09:15.904 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:15.904 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:15.904 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:15.904 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36335", 2020-09-08T16:09:15.905 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:09:15.905 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 153735 2020-09-08T16:09:15.905 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.905 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:15.905 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.333395+0000", 2020-09-08T16:09:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:15.907 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.333395+0000", 2020-09-08T16:09:15.907 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999256 2020-09-08T16:09:15.907 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.907 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.907 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:15.908 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.333320+0000", 2020-09-08T16:09:15.908 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017020300000000001 2020-09-08T16:09:15.908 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.908 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.908 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.333490+0000", 2020-09-08T16:09:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.1998000000000001e-05 2020-09-08T16:09:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.333522+0000", 2020-09-08T16:09:15.910 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0883999999999999e-05 2020-09-08T16:09:15.910 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.910 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.910 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:15.910 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.333533+0000", 2020-09-08T16:09:15.911 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0064422200000000002 2020-09-08T16:09:15.911 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.911 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.911 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:15.911 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.339975+0000", 2020-09-08T16:09:15.912 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019091 2020-09-08T16:09:15.912 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.912 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.912 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:15.912 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.340166+0000", 2020-09-08T16:09:15.912 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000152158 2020-09-08T16:09:15.913 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.913 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.913 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:09:15.913 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.340318+0000", 2020-09-08T16:09:15.913 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0046670130000000002 2020-09-08T16:09:15.914 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.914 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.914 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:15.914 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.344986+0000", 2020-09-08T16:09:15.914 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.095222768999999999 2020-09-08T16:09:15.915 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.915 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.915 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:15.915 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.440208+0000", 2020-09-08T16:09:15.915 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.9529e-05 2020-09-08T16:09:15.915 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.916 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.916 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:15.916 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.440248+0000", 2020-09-08T16:09:15.916 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.0307e-05 2020-09-08T16:09:15.916 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.917 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.917 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:15.917 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:01.440278+0000", 2020-09-08T16:09:15.917 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.106883512 2020-09-08T16:09:15.917 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.918 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:15.918 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.918 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.918 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.918 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36335.0:181074 8.8 8:1c294597:::benchmark_data_smithi166_52302_object50153:head [delete] snapc 0=[] ondisk+write+known_if_redirected e671)", 2020-09-08T16:09:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:07:31.160189+0000", 2020-09-08T16:09:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 104.643917989, 2020-09-08T16:09:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.129145329, 2020-09-08T16:09:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:15.920 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36335", 2020-09-08T16:09:15.920 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:09:15.920 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 181074 2020-09-08T16:09:15.920 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.920 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:15.921 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.921 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:15.921 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:31.160189+0000", 2020-09-08T16:09:15.921 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:15.921 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.922 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.922 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:15.922 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:31.160189+0000", 2020-09-08T16:09:15.922 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999256 2020-09-08T16:09:15.922 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.922 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.923 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:15.923 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:31.160114+0000", 2020-09-08T16:09:15.923 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012438600000000001 2020-09-08T16:09:15.923 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.923 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:31.160239+0000", 2020-09-08T16:09:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.471e-05 2020-09-08T16:09:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.925 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:15.925 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:31.160254+0000", 2020-09-08T16:09:15.925 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.6200000000000005e-06 2020-09-08T16:09:15.925 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.925 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:31.160262+0000", 2020-09-08T16:09:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.8742e-05 2020-09-08T16:09:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:31.160331+0000", 2020-09-08T16:09:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000210793 2020-09-08T16:09:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:31.160542+0000", 2020-09-08T16:09:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00020573 2020-09-08T16:09:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:09:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:31.160747+0000", 2020-09-08T16:09:15.929 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.004434131 2020-09-08T16:09:15.929 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.929 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.929 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:15.929 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:31.165182+0000", 2020-09-08T16:09:15.930 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.123974952 2020-09-08T16:09:15.930 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.930 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.930 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:15.930 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:31.289157+0000", 2020-09-08T16:09:15.931 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.8084000000000005e-05 2020-09-08T16:09:15.931 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.932 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.932 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:15.932 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:31.289255+0000", 2020-09-08T16:09:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.9377999999999998e-05 2020-09-08T16:09:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:31.289334+0000", 2020-09-08T16:09:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.129145329 2020-09-08T16:09:15.934 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.934 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:15.934 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.934 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.934 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.935 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36335.0:185746 8.8 8:180df4af:::benchmark_data_smithi166_52302_object54825:head [delete] snapc 0=[] ondisk+write+known_if_redirected e673)", 2020-09-08T16:09:15.935 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:07:36.210699+0000", 2020-09-08T16:09:15.935 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 99.593407416000005, 2020-09-08T16:09:15.935 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.093639744999999996, 2020-09-08T16:09:15.935 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:15.936 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:15.936 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:15.936 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36335", 2020-09-08T16:09:15.936 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:09:15.936 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 185746 2020-09-08T16:09:15.937 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.937 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:15.937 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.937 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:15.937 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:36.210699+0000", 2020-09-08T16:09:15.938 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:15.938 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.938 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.938 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:15.938 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:36.210699+0000", 2020-09-08T16:09:15.939 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999456 2020-09-08T16:09:15.939 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.939 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.939 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:15.939 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:36.210645+0000", 2020-09-08T16:09:15.940 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000116691 2020-09-08T16:09:15.940 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.940 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.940 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:15.940 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:36.210761+0000", 2020-09-08T16:09:15.941 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.8396000000000001e-05 2020-09-08T16:09:15.941 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.941 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.941 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:15.941 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:36.210780+0000", 2020-09-08T16:09:15.942 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.2259999999999999e-06 2020-09-08T16:09:15.942 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.942 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.942 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:15.942 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:36.210787+0000", 2020-09-08T16:09:15.943 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0046352340000000002 2020-09-08T16:09:15.943 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.943 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.943 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:15.943 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:36.215422+0000", 2020-09-08T16:09:15.943 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012376999999999999 2020-09-08T16:09:15.944 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.944 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.944 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:15.944 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:36.215546+0000", 2020-09-08T16:09:15.944 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012937800000000001 2020-09-08T16:09:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:09:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:36.215675+0000", 2020-09-08T16:09:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0097615199999999992 2020-09-08T16:09:15.946 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.946 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.946 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:15.946 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:36.225437+0000", 2020-09-08T16:09:15.946 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.078745499999999996 2020-09-08T16:09:15.946 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.947 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.947 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:15.947 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:36.304182+0000", 2020-09-08T16:09:15.947 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.7730000000000002e-05 2020-09-08T16:09:15.947 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.948 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.948 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:15.948 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:36.304270+0000", 2020-09-08T16:09:15.948 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.8867000000000003e-05 2020-09-08T16:09:15.948 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.949 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.949 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:15.949 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:07:36.304339+0000", 2020-09-08T16:09:15.949 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.093639744999999996 2020-09-08T16:09:15.949 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.949 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:15.950 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.950 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.950 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.950 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36335.0:211699 8.8 8:1f59f493:::benchmark_data_smithi166_52302_object80778:head [delete] snapc 0=[] ondisk+write+known_if_redirected e680)", 2020-09-08T16:09:15.950 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:08:05.338787+0000", 2020-09-08T16:09:15.951 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 70.465320055000007, 2020-09-08T16:09:15.951 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.089027168000000004, 2020-09-08T16:09:15.951 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:15.951 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:15.951 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:15.952 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36335", 2020-09-08T16:09:15.952 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:09:15.952 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 211699 2020-09-08T16:09:15.952 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.952 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:15.952 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.953 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:15.953 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.338787+0000", 2020-09-08T16:09:15.953 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:15.953 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.953 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.954 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:15.954 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.338787+0000", 2020-09-08T16:09:15.954 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999599 2020-09-08T16:09:15.954 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.954 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.955 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:15.955 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.338746+0000", 2020-09-08T16:09:15.955 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.6484999999999995e-05 2020-09-08T16:09:15.955 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.955 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.955 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:15.956 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.338843+0000", 2020-09-08T16:09:15.956 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.6353999999999999e-05 2020-09-08T16:09:15.956 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.956 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.956 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:15.957 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.338869+0000", 2020-09-08T16:09:15.957 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4217e-05 2020-09-08T16:09:15.957 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.957 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.957 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:15.958 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.338884+0000", 2020-09-08T16:09:15.958 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0038989879999999999 2020-09-08T16:09:15.958 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.958 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.958 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:15.959 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.342783+0000", 2020-09-08T16:09:15.959 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000195064 2020-09-08T16:09:15.959 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.959 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.959 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:15.960 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.342978+0000", 2020-09-08T16:09:15.960 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019260999999999999 2020-09-08T16:09:15.960 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.960 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.960 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:09:15.960 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.343170+0000", 2020-09-08T16:09:15.961 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0084553549999999995 2020-09-08T16:09:15.961 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.961 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.961 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:15.961 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.351626+0000", 2020-09-08T16:09:15.962 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.075944417 2020-09-08T16:09:15.962 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.962 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.962 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:15.962 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.427570+0000", 2020-09-08T16:09:15.963 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013915100000000001 2020-09-08T16:09:15.963 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.963 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.963 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:15.963 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.427709+0000", 2020-09-08T16:09:15.964 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000104651 2020-09-08T16:09:15.964 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.964 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.964 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:15.964 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.427814+0000", 2020-09-08T16:09:15.964 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.089027168000000004 2020-09-08T16:09:15.965 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.965 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:15.965 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.965 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.965 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.966 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36335.0:211709 8.8 8:16464c38:::benchmark_data_smithi166_52302_object80788:head [delete] snapc 0=[] ondisk+write+known_if_redirected e680)", 2020-09-08T16:09:15.966 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:08:05.347863+0000", 2020-09-08T16:09:15.966 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 70.456243290000003, 2020-09-08T16:09:15.966 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.106843521, 2020-09-08T16:09:15.966 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:15.967 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:15.967 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:15.967 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36335", 2020-09-08T16:09:15.967 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:09:15.967 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 211709 2020-09-08T16:09:15.968 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.968 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:15.968 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.968 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:15.968 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.347863+0000", 2020-09-08T16:09:15.968 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:15.969 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.969 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.969 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:15.969 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.347863+0000", 2020-09-08T16:09:15.969 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999509 2020-09-08T16:09:15.970 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.970 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.970 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:15.970 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.347814+0000", 2020-09-08T16:09:15.970 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000112996 2020-09-08T16:09:15.971 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.971 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.971 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:15.971 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.347927+0000", 2020-09-08T16:09:15.971 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.9177e-05 2020-09-08T16:09:15.972 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.972 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.972 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:15.972 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.347946+0000", 2020-09-08T16:09:15.972 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5651999999999999e-05 2020-09-08T16:09:15.972 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.973 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.973 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:15.973 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.347962+0000", 2020-09-08T16:09:15.973 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4641000000000001e-05 2020-09-08T16:09:15.973 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.974 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.974 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:15.974 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.348017+0000", 2020-09-08T16:09:15.974 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012306700000000001 2020-09-08T16:09:15.974 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.975 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.975 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:15.975 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.348140+0000", 2020-09-08T16:09:15.977 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011608 2020-09-08T16:09:15.977 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.977 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.978 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:09:15.978 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.348256+0000", 2020-09-08T16:09:15.978 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0037231539999999998 2020-09-08T16:09:15.978 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.978 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.979 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:15.979 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.351979+0000", 2020-09-08T16:09:15.979 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.102640577 2020-09-08T16:09:15.979 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.979 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.980 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:15.980 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.454620+0000", 2020-09-08T16:09:15.980 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.9391999999999997e-05 2020-09-08T16:09:15.980 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.980 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.981 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:15.981 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.454669+0000", 2020-09-08T16:09:15.981 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.7984000000000001e-05 2020-09-08T16:09:15.981 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.981 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.982 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:15.988 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.454707+0000", 2020-09-08T16:09:15.988 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.106843521 2020-09-08T16:09:15.988 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.989 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:15.989 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:15.989 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.989 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.989 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36335.0:211712 8.8 8:1631db75:::benchmark_data_smithi166_52302_object80791:head [delete] snapc 0=[] ondisk+write+known_if_redirected e680)", 2020-09-08T16:09:15.990 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:08:05.353224+0000", 2020-09-08T16:09:15.990 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 70.450882636000003, 2020-09-08T16:09:15.990 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.101731585, 2020-09-08T16:09:15.990 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:15.990 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:15.991 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:15.991 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36335", 2020-09-08T16:09:15.991 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:09:15.991 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 211712 2020-09-08T16:09:15.991 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.991 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:15.992 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.992 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:15.992 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.353224+0000", 2020-09-08T16:09:15.992 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:15.992 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.993 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.993 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:15.993 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.353224+0000", 2020-09-08T16:09:15.993 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999347 2020-09-08T16:09:15.993 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.994 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.994 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:15.994 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.353158+0000", 2020-09-08T16:09:15.994 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013491999999999999 2020-09-08T16:09:15.994 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.995 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.995 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:15.995 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.353293+0000", 2020-09-08T16:09:15.996 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.4759000000000001e-05 2020-09-08T16:09:15.996 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.996 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.997 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:15.997 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.353318+0000", 2020-09-08T16:09:15.997 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2007999999999999e-05 2020-09-08T16:09:15.997 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.998 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.998 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:15.998 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.353330+0000", 2020-09-08T16:09:15.998 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.1311999999999998e-05 2020-09-08T16:09:15.998 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:15.999 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:15.999 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:15.999 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.353401+0000", 2020-09-08T16:09:15.999 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000212043 2020-09-08T16:09:15.999 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.000 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.000 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:16.000 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.353614+0000", 2020-09-08T16:09:16.000 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000230542 2020-09-08T16:09:16.001 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.001 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.001 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:09:16.001 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.353844+0000", 2020-09-08T16:09:16.001 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0050914109999999997 2020-09-08T16:09:16.002 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.002 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.002 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:16.002 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.358935+0000", 2020-09-08T16:09:16.003 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.095949791000000006 2020-09-08T16:09:16.003 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.003 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.003 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:16.003 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.454885+0000", 2020-09-08T16:09:16.004 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.9972000000000003e-05 2020-09-08T16:09:16.004 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.004 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.004 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:16.004 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.454925+0000", 2020-09-08T16:09:16.005 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.0368000000000001e-05 2020-09-08T16:09:16.005 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.005 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.005 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:16.005 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.454956+0000", 2020-09-08T16:09:16.006 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.101731585 2020-09-08T16:09:16.006 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:16.006 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:16.006 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:16.007 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.007 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.008 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36335.0:211722 8.8 8:153a155c:::benchmark_data_smithi166_52302_object80801:head [delete] snapc 0=[] ondisk+write+known_if_redirected e680)", 2020-09-08T16:09:16.008 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:08:05.363296+0000", 2020-09-08T16:09:16.008 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 70.44081079, 2020-09-08T16:09:16.008 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.091906923000000001, 2020-09-08T16:09:16.009 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:16.009 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:16.009 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:16.009 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36335", 2020-09-08T16:09:16.009 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:09:16.010 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 211722 2020-09-08T16:09:16.010 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.010 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:16.010 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.010 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:16.011 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.363296+0000", 2020-09-08T16:09:16.011 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:16.011 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.011 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.011 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:16.011 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.363296+0000", 2020-09-08T16:09:16.012 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999199 2020-09-08T16:09:16.012 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.012 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.012 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:16.012 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.363216+0000", 2020-09-08T16:09:16.013 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000186548 2020-09-08T16:09:16.013 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.013 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.013 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:16.013 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.363403+0000", 2020-09-08T16:09:16.014 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.8860000000000002e-05 2020-09-08T16:09:16.014 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.014 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.014 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:16.014 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.363431+0000", 2020-09-08T16:09:16.015 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4465e-05 2020-09-08T16:09:16.015 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.015 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.015 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:16.015 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.363446+0000", 2020-09-08T16:09:16.016 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.0448e-05 2020-09-08T16:09:16.016 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.016 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.016 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:16.016 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.363516+0000", 2020-09-08T16:09:16.016 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014831800000000001 2020-09-08T16:09:16.017 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.017 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.017 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:16.017 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.363665+0000", 2020-09-08T16:09:16.018 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00086324299999999997 2020-09-08T16:09:16.018 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.018 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.018 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:09:16.018 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.364528+0000", 2020-09-08T16:09:16.019 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0049493030000000004 2020-09-08T16:09:16.019 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.019 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.019 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:16.019 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.369477+0000", 2020-09-08T16:09:16.019 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.085660869000000001 2020-09-08T16:09:16.020 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.020 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.020 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:16.020 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.455138+0000", 2020-09-08T16:09:16.020 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.6789000000000003e-05 2020-09-08T16:09:16.021 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.021 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.021 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:16.021 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.455175+0000", 2020-09-08T16:09:16.022 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.7963999999999999e-05 2020-09-08T16:09:16.022 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.022 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.022 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:16.023 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.455203+0000", 2020-09-08T16:09:16.023 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.091906923000000001 2020-09-08T16:09:16.023 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:16.023 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:16.023 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:16.024 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.024 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.024 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36335.0:211729 8.8 8:1be2b3ef:::benchmark_data_smithi166_52302_object80808:head [delete] snapc 0=[] ondisk+write+known_if_redirected e680)", 2020-09-08T16:09:16.024 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:08:05.370489+0000", 2020-09-08T16:09:16.024 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 70.433617362999996, 2020-09-08T16:09:16.025 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.091058444000000002, 2020-09-08T16:09:16.025 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:16.025 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:16.025 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:16.026 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36335", 2020-09-08T16:09:16.026 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:09:16.026 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 211729 2020-09-08T16:09:16.026 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.026 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:16.027 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.027 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:16.027 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.370489+0000", 2020-09-08T16:09:16.027 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:16.027 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.027 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.028 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:16.028 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.370489+0000", 2020-09-08T16:09:16.028 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999232 2020-09-08T16:09:16.028 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.028 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.029 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:16.029 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.370412+0000", 2020-09-08T16:09:16.029 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000168369 2020-09-08T16:09:16.029 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.029 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.030 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:16.030 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.370581+0000", 2020-09-08T16:09:16.030 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.8555999999999999e-05 2020-09-08T16:09:16.030 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.030 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.031 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:16.031 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.370609+0000", 2020-09-08T16:09:16.031 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4110999999999999e-05 2020-09-08T16:09:16.031 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.031 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.031 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:16.032 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.370623+0000", 2020-09-08T16:09:16.032 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.4084999999999994e-05 2020-09-08T16:09:16.032 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.032 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.032 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:16.033 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.370687+0000", 2020-09-08T16:09:16.033 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017481800000000001 2020-09-08T16:09:16.033 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.033 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.033 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:16.034 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.370862+0000", 2020-09-08T16:09:16.034 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019263099999999999 2020-09-08T16:09:16.034 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.035 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.035 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:09:16.035 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.371055+0000", 2020-09-08T16:09:16.035 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0044445439999999999 2020-09-08T16:09:16.036 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.036 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.036 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:16.036 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.375499+0000", 2020-09-08T16:09:16.036 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.085937066000000006 2020-09-08T16:09:16.036 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.037 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.037 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:16.037 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.461436+0000", 2020-09-08T16:09:16.037 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.8267999999999998e-05 2020-09-08T16:09:16.037 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.038 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.038 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:16.038 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.461495+0000", 2020-09-08T16:09:16.038 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.3000000000000001e-05 2020-09-08T16:09:16.039 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.039 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.039 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:16.039 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:05.461548+0000", 2020-09-08T16:09:16.039 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.091058444000000002 2020-09-08T16:09:16.040 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:16.040 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:16.040 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:16.040 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.040 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.040 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36335.0:232029 8.8 8:1215f33a:::benchmark_data_smithi166_52302_object101108:head [delete] snapc 0=[] ondisk+write+known_if_redirected e686)", 2020-09-08T16:09:16.041 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:08:30.336612+0000", 2020-09-08T16:09:16.041 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 45.467495094, 2020-09-08T16:09:16.041 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.10874228900000001, 2020-09-08T16:09:16.041 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:16.041 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:16.042 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:16.042 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36335", 2020-09-08T16:09:16.042 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:09:16.042 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 232029 2020-09-08T16:09:16.042 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.043 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:16.043 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.043 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:16.044 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.336612+0000", 2020-09-08T16:09:16.044 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:16.044 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.044 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.044 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:16.045 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.336612+0000", 2020-09-08T16:09:16.045 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9998727 2020-09-08T16:09:16.045 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.045 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.045 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:16.045 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.336484+0000", 2020-09-08T16:09:16.046 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000278139 2020-09-08T16:09:16.046 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.046 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.046 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:16.046 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.336762+0000", 2020-09-08T16:09:16.047 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.0510999999999999e-05 2020-09-08T16:09:16.047 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.047 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.047 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:16.047 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.336803+0000", 2020-09-08T16:09:16.048 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.7581999999999999e-05 2020-09-08T16:09:16.048 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.048 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.048 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:16.048 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.336820+0000", 2020-09-08T16:09:16.049 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4005999999999995e-05 2020-09-08T16:09:16.049 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.049 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.049 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:16.049 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.336904+0000", 2020-09-08T16:09:16.050 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014931999999999999 2020-09-08T16:09:16.050 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.050 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.050 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:16.050 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.337054+0000", 2020-09-08T16:09:16.050 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000145332 2020-09-08T16:09:16.051 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.051 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.051 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:09:16.051 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.337199+0000", 2020-09-08T16:09:16.051 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.010946912 2020-09-08T16:09:16.052 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.052 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.052 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:16.052 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.348146+0000", 2020-09-08T16:09:16.052 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.097109695999999995 2020-09-08T16:09:16.053 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.053 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.053 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:16.053 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.445256+0000", 2020-09-08T16:09:16.053 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.8016000000000002e-05 2020-09-08T16:09:16.054 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.054 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.054 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:16.054 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.445314+0000", 2020-09-08T16:09:16.054 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.0133999999999999e-05 2020-09-08T16:09:16.054 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.055 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.055 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:16.055 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.445354+0000", 2020-09-08T16:09:16.055 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.10874228900000001 2020-09-08T16:09:16.055 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:16.056 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:16.056 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:16.056 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.056 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.056 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36335.0:232033 8.8 8:1c332f10:::benchmark_data_smithi166_52302_object101112:head [delete] snapc 0=[] ondisk+write+known_if_redirected e686)", 2020-09-08T16:09:16.057 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:08:30.338905+0000", 2020-09-08T16:09:16.057 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 45.465202050999999, 2020-09-08T16:09:16.057 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.106893242, 2020-09-08T16:09:16.057 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:16.057 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:16.058 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:16.058 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36335", 2020-09-08T16:09:16.058 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:09:16.058 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 232033 2020-09-08T16:09:16.058 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.058 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:16.059 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.059 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:16.059 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.338905+0000", 2020-09-08T16:09:16.059 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:16.059 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.060 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.060 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:16.060 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.338905+0000", 2020-09-08T16:09:16.060 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999638 2020-09-08T16:09:16.060 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.061 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.061 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:16.061 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.338868+0000", 2020-09-08T16:09:16.061 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.4364999999999997e-05 2020-09-08T16:09:16.061 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.062 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.062 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:16.062 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.338943+0000", 2020-09-08T16:09:16.062 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.6166000000000001e-05 2020-09-08T16:09:16.062 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.062 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.063 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:16.063 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.338959+0000", 2020-09-08T16:09:16.063 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4840000000000004e-06 2020-09-08T16:09:16.063 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.063 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.064 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:16.064 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.338967+0000", 2020-09-08T16:09:16.064 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0032801449999999999 2020-09-08T16:09:16.064 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.064 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.065 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:16.065 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.342247+0000", 2020-09-08T16:09:16.065 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00020531 2020-09-08T16:09:16.065 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.065 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.065 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:16.066 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.342453+0000", 2020-09-08T16:09:16.066 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000190094 2020-09-08T16:09:16.066 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.066 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.066 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:09:16.067 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.342643+0000", 2020-09-08T16:09:16.067 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0056820259999999997 2020-09-08T16:09:16.067 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.067 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.067 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:16.068 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.348325+0000", 2020-09-08T16:09:16.068 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.097402303999999995 2020-09-08T16:09:16.068 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.068 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.068 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:16.068 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.445727+0000", 2020-09-08T16:09:16.069 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.2796999999999997e-05 2020-09-08T16:09:16.069 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.069 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.069 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:16.069 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.445770+0000", 2020-09-08T16:09:16.070 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.7926e-05 2020-09-08T16:09:16.070 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.070 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.071 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:16.071 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.445798+0000", 2020-09-08T16:09:16.071 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.106893242 2020-09-08T16:09:16.071 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:16.071 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:16.072 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:16.072 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.072 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.072 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36335.0:232049 8.8 8:18c494e2:::benchmark_data_smithi166_52302_object101128:head [delete] snapc 0=[] ondisk+write+known_if_redirected e686)", 2020-09-08T16:09:16.073 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:08:30.358926+0000", 2020-09-08T16:09:16.073 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 45.445181126000001, 2020-09-08T16:09:16.073 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.10267459600000001, 2020-09-08T16:09:16.073 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:16.073 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:16.074 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:16.074 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36335", 2020-09-08T16:09:16.074 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:09:16.074 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 232049 2020-09-08T16:09:16.074 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.075 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:16.075 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.075 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:16.075 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.358926+0000", 2020-09-08T16:09:16.075 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:16.076 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.076 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.076 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:16.076 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.358926+0000", 2020-09-08T16:09:16.076 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999917 2020-09-08T16:09:16.076 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.077 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.077 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:16.077 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.358842+0000", 2020-09-08T16:09:16.077 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000199903 2020-09-08T16:09:16.077 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.078 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.078 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:16.078 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.359042+0000", 2020-09-08T16:09:16.078 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.0397999999999999e-05 2020-09-08T16:09:16.078 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.079 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.079 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:16.079 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.359073+0000", 2020-09-08T16:09:16.079 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4701999999999999e-05 2020-09-08T16:09:16.079 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.080 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.080 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:16.080 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.359087+0000", 2020-09-08T16:09:16.080 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.5734000000000005e-05 2020-09-08T16:09:16.081 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.081 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.081 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:16.081 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.359153+0000", 2020-09-08T16:09:16.081 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012976099999999999 2020-09-08T16:09:16.082 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.082 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.082 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:16.082 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.359283+0000", 2020-09-08T16:09:16.082 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000113717 2020-09-08T16:09:16.083 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.083 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.083 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:09:16.083 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.359397+0000", 2020-09-08T16:09:16.083 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0046960919999999998 2020-09-08T16:09:16.084 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.084 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.084 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:16.084 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.364093+0000", 2020-09-08T16:09:16.084 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.097170477000000005 2020-09-08T16:09:16.085 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.085 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.085 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:16.085 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.461263+0000", 2020-09-08T16:09:16.085 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000190533 2020-09-08T16:09:16.086 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.086 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.086 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:16.086 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.461454+0000", 2020-09-08T16:09:16.086 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014643699999999999 2020-09-08T16:09:16.087 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.087 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.087 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:16.087 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:30.461600+0000", 2020-09-08T16:09:16.087 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.10267459600000001 2020-09-08T16:09:16.088 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:16.088 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:16.088 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:16.088 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.088 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.089 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36335.0:248828 8.4 8:2732e55b:::benchmark_data_smithi166_52302_object117907:head [delete] snapc 0=[] ondisk+write+known_if_redirected e691)", 2020-09-08T16:09:16.089 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:08:49.099381+0000", 2020-09-08T16:09:16.089 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 26.704726109999999, 2020-09-08T16:09:16.089 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.048781321000000002, 2020-09-08T16:09:16.089 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:16.090 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:16.090 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:16.090 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36335", 2020-09-08T16:09:16.090 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:09:16.090 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 248828 2020-09-08T16:09:16.091 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.091 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:16.091 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.091 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:16.091 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.099381+0000", 2020-09-08T16:09:16.091 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:16.092 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.092 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.092 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:16.092 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.099381+0000", 2020-09-08T16:09:16.092 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999959 2020-09-08T16:09:16.093 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.093 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.093 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:16.093 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.099340+0000", 2020-09-08T16:09:16.093 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.2801999999999997e-05 2020-09-08T16:09:16.094 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.094 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.094 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:16.094 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.099432+0000", 2020-09-08T16:09:16.094 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4612e-05 2020-09-08T16:09:16.095 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.095 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.095 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:16.095 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.099447+0000", 2020-09-08T16:09:16.095 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.6810000000000001e-06 2020-09-08T16:09:16.095 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.096 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.096 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:16.096 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.099454+0000", 2020-09-08T16:09:16.096 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0058648260000000001 2020-09-08T16:09:16.096 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.097 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.097 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:16.097 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.105319+0000", 2020-09-08T16:09:16.097 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.2583000000000005e-05 2020-09-08T16:09:16.097 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.098 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.098 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:16.098 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.105401+0000", 2020-09-08T16:09:16.098 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.5210000000000005e-05 2020-09-08T16:09:16.098 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.098 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.099 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 9", 2020-09-08T16:09:16.099 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.105496+0000", 2020-09-08T16:09:16.099 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0040927409999999996 2020-09-08T16:09:16.099 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.099 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.100 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:16.100 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.109589+0000", 2020-09-08T16:09:16.100 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.038399884000000002 2020-09-08T16:09:16.100 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.100 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.101 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:16.101 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.147989+0000", 2020-09-08T16:09:16.101 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000106538 2020-09-08T16:09:16.101 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.101 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.101 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:16.102 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.148096+0000", 2020-09-08T16:09:16.102 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.6340000000000005e-05 2020-09-08T16:09:16.102 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.102 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.102 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:16.103 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:49.148162+0000", 2020-09-08T16:09:16.103 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.048781321000000002 2020-09-08T16:09:16.103 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:16.103 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:16.103 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:16.104 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.104 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.104 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36335.0:249745 8.4 8:2d9a12fc:::benchmark_data_smithi166_52302_object118824:head [delete] snapc 0=[] ondisk+write+known_if_redirected e691)", 2020-09-08T16:09:16.104 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:08:50.071566+0000", 2020-09-08T16:09:16.104 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 25.732540289999999, 2020-09-08T16:09:16.104 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.054717254999999999, 2020-09-08T16:09:16.105 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:16.105 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:16.105 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:16.105 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36335", 2020-09-08T16:09:16.105 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:09:16.106 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 249745 2020-09-08T16:09:16.106 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.106 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:16.106 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.106 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:16.107 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.071566+0000", 2020-09-08T16:09:16.107 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:16.107 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.107 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.107 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:16.108 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.071566+0000", 2020-09-08T16:09:16.108 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999949 2020-09-08T16:09:16.108 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.108 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.108 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:16.108 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.071515+0000", 2020-09-08T16:09:16.109 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010149600000000001 2020-09-08T16:09:16.109 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.109 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.109 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:16.109 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.071617+0000", 2020-09-08T16:09:16.110 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3249e-05 2020-09-08T16:09:16.110 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.110 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.110 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:16.110 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.071630+0000", 2020-09-08T16:09:16.111 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.6359999999999999e-06 2020-09-08T16:09:16.111 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.111 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.111 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:16.111 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.071636+0000", 2020-09-08T16:09:16.111 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0064282699999999998 2020-09-08T16:09:16.112 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.112 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.112 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:16.112 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.078064+0000", 2020-09-08T16:09:16.112 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000136377 2020-09-08T16:09:16.113 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.113 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.113 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:16.113 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.078200+0000", 2020-09-08T16:09:16.113 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012705299999999999 2020-09-08T16:09:16.113 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.114 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.114 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 9", 2020-09-08T16:09:16.114 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.078327+0000", 2020-09-08T16:09:16.114 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.047570925 2020-09-08T16:09:16.114 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.115 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.115 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:16.115 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.125898+0000", 2020-09-08T16:09:16.115 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011134299999999999 2020-09-08T16:09:16.115 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.116 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.116 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:16.116 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.126010+0000", 2020-09-08T16:09:16.116 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015087400000000001 2020-09-08T16:09:16.116 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.117 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.117 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:16.117 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.126161+0000", 2020-09-08T16:09:16.117 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012311700000000001 2020-09-08T16:09:16.117 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.117 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.118 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:16.118 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.126284+0000", 2020-09-08T16:09:16.118 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.054717254999999999 2020-09-08T16:09:16.118 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:16.118 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:16.119 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:16.119 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.119 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.119 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36335.0:250533 8.8 8:1464b6d2:::benchmark_data_smithi166_52302_object119612:head [delete] snapc 0=[] ondisk+write+known_if_redirected e691)", 2020-09-08T16:09:16.119 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:08:50.952886+0000", 2020-09-08T16:09:16.120 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 24.851221038999999, 2020-09-08T16:09:16.120 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.051508974999999999, 2020-09-08T16:09:16.120 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:16.120 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:16.120 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:16.121 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36335", 2020-09-08T16:09:16.121 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:09:16.121 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 250533 2020-09-08T16:09:16.121 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.121 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:16.122 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.122 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:16.122 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.952886+0000", 2020-09-08T16:09:16.123 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:16.123 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.123 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.123 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:16.123 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.952886+0000", 2020-09-08T16:09:16.124 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999918 2020-09-08T16:09:16.124 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.124 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.124 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:16.125 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.952804+0000", 2020-09-08T16:09:16.125 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000178159 2020-09-08T16:09:16.125 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.125 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.125 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:16.126 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.952982+0000", 2020-09-08T16:09:16.126 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.6509999999999999e-05 2020-09-08T16:09:16.126 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.126 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.127 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:16.127 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.953008+0000", 2020-09-08T16:09:16.127 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.2500000000000001e-05 2020-09-08T16:09:16.127 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.127 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.127 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:16.128 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.953021+0000", 2020-09-08T16:09:16.128 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.012159265000000001 2020-09-08T16:09:16.128 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.128 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.128 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:16.129 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.965180+0000", 2020-09-08T16:09:16.129 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00018411999999999999 2020-09-08T16:09:16.129 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.129 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.129 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:16.130 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.965364+0000", 2020-09-08T16:09:16.130 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000176207 2020-09-08T16:09:16.130 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.130 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.130 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:09:16.131 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.965541+0000", 2020-09-08T16:09:16.131 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.014169467999999999 2020-09-08T16:09:16.131 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.131 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.131 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:16.131 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:50.979710+0000", 2020-09-08T16:09:16.132 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.024513852999999999 2020-09-08T16:09:16.132 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.132 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.132 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:16.132 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:51.004224+0000", 2020-09-08T16:09:16.133 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.9660999999999993e-05 2020-09-08T16:09:16.133 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.133 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.133 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:16.133 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:51.004324+0000", 2020-09-08T16:09:16.133 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.1080000000000004e-05 2020-09-08T16:09:16.134 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:16.134 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:16.134 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:16.134 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:08:51.004395+0000", 2020-09-08T16:09:16.134 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.051508974999999999 2020-09-08T16:09:16.135 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:16.135 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:16.135 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:16.135 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:16.135 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:16.136 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:09:16.137 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:09:15 smithi166 bash[11928]: debug 2020-09-08T16:09:15.581+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:16.137 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:09:16.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:09:15 smithi055 bash[11956]: debug 2020-09-08T16:09:15.812+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:16.238 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:16.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:16 smithi104 bash[9203]: cluster 2020-09-08T16:09:15.158788+0000 mon.a (mon.0) 2206 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:09:16.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:16 smithi104 bash[9203]: audit 2020-09-08T16:09:15.173296+0000 mon.a (mon.0) 2207 : audit [INF] from='client.? v1:172.21.15.104:0/274585065' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:09:16.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:16 smithi104 bash[9203]: cluster 2020-09-08T16:09:15.173366+0000 mon.a (mon.0) 2208 : cluster [DBG] osdmap e699: 12 total, 12 up, 12 in 2020-09-08T16:09:16.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:16 smithi166 bash[9446]: cluster 2020-09-08T16:09:15.158788+0000 mon.a (mon.0) 2206 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:09:16.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:16 smithi166 bash[9446]: audit 2020-09-08T16:09:15.173296+0000 mon.a (mon.0) 2207 : audit [INF] from='client.? v1:172.21.15.104:0/274585065' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:09:16.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:16 smithi166 bash[9446]: cluster 2020-09-08T16:09:15.173366+0000 mon.a (mon.0) 2208 : cluster [DBG] osdmap e699: 12 total, 12 up, 12 in 2020-09-08T16:09:16.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:16 smithi055 bash[9296]: cluster 2020-09-08T16:09:15.158788+0000 mon.a (mon.0) 2206 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:09:16.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:16 smithi055 bash[9296]: audit 2020-09-08T16:09:15.173296+0000 mon.a (mon.0) 2207 : audit [INF] from='client.? v1:172.21.15.104:0/274585065' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:09:16.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:16 smithi055 bash[9296]: cluster 2020-09-08T16:09:15.173366+0000 mon.a (mon.0) 2208 : cluster [DBG] osdmap e699: 12 total, 12 up, 12 in 2020-09-08T16:09:16.675 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2020-09-08T16:09:16.770 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:16.862 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:09:16.876 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:17.027 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:17.133 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:17.147 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:09:16 smithi104 bash[17588]: debug 2020-09-08T16:09:16.770+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:17.170 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:09:17 smithi166 bash[11928]: debug 2020-09-08T16:09:17.025+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:17.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:17 smithi104 bash[9203]: cluster 2020-09-08T16:09:16.103999+0000 mgr.x (mgr.24118) 1258 : cluster [DBG] pgmap v1873: 1 pgs: 1 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:17.456 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:17.560 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:09:17.574 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:17 smithi166 bash[9446]: cluster 2020-09-08T16:09:16.103999+0000 mgr.x (mgr.24118) 1258 : cluster [DBG] pgmap v1873: 1 pgs: 1 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:17.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:17 smithi055 bash[9296]: cluster 2020-09-08T16:09:16.103999+0000 mgr.x (mgr.24118) 1258 : cluster [DBG] pgmap v1873: 1 pgs: 1 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:17.693 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:09:17.783 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:09:17 smithi104 bash[15273]: debug 2020-09-08T16:09:17.454+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:17.798 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:17.868 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:09:17 smithi166 bash[14254]: debug 2020-09-08T16:09:17.693+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:17.915 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:17.936 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:09:17.965 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:09:17.989 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:09:18.017 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:09:18.020 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:18.047 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:09:18.074 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:09:18.097 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:09:18.126 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:09:18.152 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:18.158 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:09:18.179 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:09:17 smithi166 bash[11928]: debug 2020-09-08T16:09:17.913+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:18.191 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:09:18.218 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:09:18.250 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:09:18.252 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:09:18.252 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:09:18.252 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:09:18.253 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:09:18.253 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:09:18.253 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:09:18.253 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:09:18.254 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:09:18.254 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:09:18.254 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:09:18.254 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:09:18.254 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:09:18.256 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:18.330 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:18 smithi055 bash[9296]: audit 2020-09-08T16:09:17.914659+0000 mon.b (mon.1) 121 : audit [DBG] from='client.? v1:172.21.15.104:0/1919066628' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:09:18.331 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:09:18 smithi055 bash[14301]: debug 2020-09-08T16:09:18.152+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:18.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:18 smithi104 bash[9203]: audit 2020-09-08T16:09:17.914659+0000 mon.b (mon.1) 121 : audit [DBG] from='client.? v1:172.21.15.104:0/1919066628' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:09:18.563 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:18.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:18 smithi166 bash[9446]: audit 2020-09-08T16:09:17.914659+0000 mon.b (mon.1) 121 : audit [DBG] from='client.? v1:172.21.15.104:0/1919066628' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:09:18.669 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:09:18.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:09:18 smithi104 bash[17588]: debug 2020-09-08T16:09:18.566+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:18.957 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:09:19.063 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:19.213 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:19.227 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T16:09:19.319 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:09:19.366 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:09:19.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:19 smithi166 bash[9446]: cluster 2020-09-08T16:09:18.104547+0000 mgr.x (mgr.24118) 1259 : cluster [DBG] pgmap v1874: 1 pgs: 1 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:19.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:09:19 smithi166 bash[11928]: debug 2020-09-08T16:09:19.209+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:19.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:19 smithi055 bash[9296]: cluster 2020-09-08T16:09:18.104547+0000 mgr.x (mgr.24118) 1259 : cluster [DBG] pgmap v1874: 1 pgs: 1 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:19.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:09:18 smithi055 bash[11956]: debug 2020-09-08T16:09:18.960+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:19.674 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:19 smithi104 bash[9203]: cluster 2020-09-08T16:09:18.104547+0000 mgr.x (mgr.24118) 1259 : cluster [DBG] pgmap v1874: 1 pgs: 1 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:19.675 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:09:19.780 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:09:19.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:09:19 smithi104 bash[19922]: debug 2020-09-08T16:09:19.674+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:20.008 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:09:20.112 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:20.410 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:09:20 smithi104 bash[22305]: debug 2020-09-08T16:09:20.010+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:20.411 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:20 smithi104 bash[9203]: cluster 2020-09-08T16:09:19.092526+0000 osd.7 (osd.7) 151 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:20.411 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:20 smithi104 bash[9203]: cluster 2020-09-08T16:09:19.092715+0000 osd.7 (osd.7) 152 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:20.411 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:20 smithi104 bash[9203]: cluster 2020-09-08T16:09:19.189974+0000 mon.a (mon.0) 2209 : cluster [DBG] osdmap e700: 12 total, 12 up, 12 in 2020-09-08T16:09:20.411 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:20 smithi104 bash[9203]: cluster 2020-09-08T16:09:19.651380+0000 osd.0 (osd.0) 126 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:20.412 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:20 smithi104 bash[9203]: cluster 2020-09-08T16:09:19.651591+0000 osd.0 (osd.0) 127 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:20.462 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T16:09:20.463 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:09:20.508 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:20.523 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:20 smithi055 bash[9296]: cluster 2020-09-08T16:09:19.092526+0000 osd.7 (osd.7) 151 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:20.524 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:20 smithi055 bash[9296]: cluster 2020-09-08T16:09:19.092715+0000 osd.7 (osd.7) 152 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:20.524 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:20 smithi055 bash[9296]: cluster 2020-09-08T16:09:19.189974+0000 mon.a (mon.0) 2209 : cluster [DBG] osdmap e700: 12 total, 12 up, 12 in 2020-09-08T16:09:20.524 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:20 smithi055 bash[9296]: cluster 2020-09-08T16:09:19.651380+0000 osd.0 (osd.0) 126 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:20.524 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:20 smithi055 bash[9296]: cluster 2020-09-08T16:09:19.651591+0000 osd.0 (osd.0) 127 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:20.615 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:09:20.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:20 smithi166 bash[9446]: cluster 2020-09-08T16:09:19.092526+0000 osd.7 (osd.7) 151 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:20.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:20 smithi166 bash[9446]: cluster 2020-09-08T16:09:19.092715+0000 osd.7 (osd.7) 152 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:20.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:20 smithi166 bash[9446]: cluster 2020-09-08T16:09:19.189974+0000 mon.a (mon.0) 2209 : cluster [DBG] osdmap e700: 12 total, 12 up, 12 in 2020-09-08T16:09:20.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:20 smithi166 bash[9446]: cluster 2020-09-08T16:09:19.651380+0000 osd.0 (osd.0) 126 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:20.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:20 smithi166 bash[9446]: cluster 2020-09-08T16:09:19.651591+0000 osd.0 (osd.0) 127 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:20.764 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:09:20.848 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:09:20 smithi104 bash[17588]: debug 2020-09-08T16:09:20.510+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:20.875 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:09:21.084 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:09:21.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:09:20 smithi166 bash[14254]: debug 2020-09-08T16:09:20.761+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:21.170 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:09:21 smithi055 bash[19003]: debug 2020-09-08T16:09:21.084+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:21.189 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:09:21.196 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2020-09-08T16:09:21.258 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'unique_pool_7' created 2020-09-08T16:09:21.333 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:09:21.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:21 smithi104 bash[9203]: cluster 2020-09-08T16:09:20.105185+0000 mgr.x (mgr.24118) 1260 : cluster [DBG] pgmap v1876: 1 pgs: 1 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:21.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:21 smithi104 bash[9203]: cluster 2020-09-08T16:09:20.243792+0000 mon.a (mon.0) 2210 : cluster [DBG] osdmap e701: 12 total, 12 up, 12 in 2020-09-08T16:09:21.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:21 smithi104 bash[9203]: audit 2020-09-08T16:09:20.747315+0000 mon.a (mon.0) 2211 : audit [INF] from='client.? v1:172.21.15.104:0/2224283316' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_7", "pg_num": 16}]: dispatch 2020-09-08T16:09:21.539 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:09:21.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:21 smithi166 bash[9446]: cluster 2020-09-08T16:09:20.105185+0000 mgr.x (mgr.24118) 1260 : cluster [DBG] pgmap v1876: 1 pgs: 1 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:21.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:21 smithi166 bash[9446]: cluster 2020-09-08T16:09:20.243792+0000 mon.a (mon.0) 2210 : cluster [DBG] osdmap e701: 12 total, 12 up, 12 in 2020-09-08T16:09:21.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:21 smithi166 bash[9446]: audit 2020-09-08T16:09:20.747315+0000 mon.a (mon.0) 2211 : audit [INF] from='client.? v1:172.21.15.104:0/2224283316' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_7", "pg_num": 16}]: dispatch 2020-09-08T16:09:21.644 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:09:21.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:21 smithi055 bash[9296]: cluster 2020-09-08T16:09:20.105185+0000 mgr.x (mgr.24118) 1260 : cluster [DBG] pgmap v1876: 1 pgs: 1 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:21.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:21 smithi055 bash[9296]: cluster 2020-09-08T16:09:20.243792+0000 mon.a (mon.0) 2210 : cluster [DBG] osdmap e701: 12 total, 12 up, 12 in 2020-09-08T16:09:21.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:21 smithi055 bash[9296]: audit 2020-09-08T16:09:20.747315+0000 mon.a (mon.0) 2211 : audit [INF] from='client.? v1:172.21.15.104:0/2224283316' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_7", "pg_num": 16}]: dispatch 2020-09-08T16:09:21.858 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:09:21.870 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:09:21 smithi104 bash[22305]: debug 2020-09-08T16:09:21.542+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:21.986 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:09:22.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:22 smithi166 bash[9446]: audit 2020-09-08T16:09:21.258753+0000 mon.a (mon.0) 2212 : audit [INF] from='client.? v1:172.21.15.104:0/2224283316' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_7", "pg_num": 16}]': finished 2020-09-08T16:09:22.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:22 smithi166 bash[9446]: cluster 2020-09-08T16:09:21.258799+0000 mon.a (mon.0) 2213 : cluster [DBG] osdmap e702: 12 total, 12 up, 12 in 2020-09-08T16:09:22.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:22 smithi055 bash[9296]: audit 2020-09-08T16:09:21.258753+0000 mon.a (mon.0) 2212 : audit [INF] from='client.? v1:172.21.15.104:0/2224283316' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_7", "pg_num": 16}]': finished 2020-09-08T16:09:22.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:22 smithi055 bash[9296]: cluster 2020-09-08T16:09:21.258799+0000 mon.a (mon.0) 2213 : cluster [DBG] osdmap e702: 12 total, 12 up, 12 in 2020-09-08T16:09:22.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:22 smithi104 bash[9203]: audit 2020-09-08T16:09:21.258753+0000 mon.a (mon.0) 2212 : audit [INF] from='client.? v1:172.21.15.104:0/2224283316' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_7", "pg_num": 16}]': finished 2020-09-08T16:09:22.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:22 smithi104 bash[9203]: cluster 2020-09-08T16:09:21.258799+0000 mon.a (mon.0) 2213 : cluster [DBG] osdmap e702: 12 total, 12 up, 12 in 2020-09-08T16:09:22.831 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool application enable unique_pool_7 rados --yes-i-really-mean-it || true 2020-09-08T16:09:23.011 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:09:23.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:23 smithi166 bash[9446]: cluster 2020-09-08T16:09:22.105827+0000 mgr.x (mgr.24118) 1261 : cluster [DBG] pgmap v1879: 17 pgs: 16 unknown, 1 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:23.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:23 smithi166 bash[9446]: cluster 2020-09-08T16:09:22.273284+0000 mon.a (mon.0) 2214 : cluster [DBG] osdmap e703: 12 total, 12 up, 12 in 2020-09-08T16:09:23.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:23 smithi055 bash[9296]: cluster 2020-09-08T16:09:22.105827+0000 mgr.x (mgr.24118) 1261 : cluster [DBG] pgmap v1879: 17 pgs: 16 unknown, 1 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:23.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:23 smithi055 bash[9296]: cluster 2020-09-08T16:09:22.273284+0000 mon.a (mon.0) 2214 : cluster [DBG] osdmap e703: 12 total, 12 up, 12 in 2020-09-08T16:09:23.886 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:23 smithi104 bash[9203]: cluster 2020-09-08T16:09:22.105827+0000 mgr.x (mgr.24118) 1261 : cluster [DBG] pgmap v1879: 17 pgs: 16 unknown, 1 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:23.886 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:23 smithi104 bash[9203]: cluster 2020-09-08T16:09:22.273284+0000 mon.a (mon.0) 2214 : cluster [DBG] osdmap e703: 12 total, 12 up, 12 in 2020-09-08T16:09:24.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:24 smithi166 bash[9446]: cluster 2020-09-08T16:09:23.272653+0000 mon.a (mon.0) 2215 : cluster [DBG] osdmap e704: 12 total, 12 up, 12 in 2020-09-08T16:09:24.640 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T16:09:24.641 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:09:24.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:24 smithi055 bash[9296]: cluster 2020-09-08T16:09:23.272653+0000 mon.a (mon.0) 2215 : cluster [DBG] osdmap e704: 12 total, 12 up, 12 in 2020-09-08T16:09:24.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:24 smithi104 bash[9203]: cluster 2020-09-08T16:09:23.272653+0000 mon.a (mon.0) 2215 : cluster [DBG] osdmap e704: 12 total, 12 up, 12 in 2020-09-08T16:09:24.808 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:09:24.913 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:25.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:09:24 smithi166 bash[16593]: debug 2020-09-08T16:09:24.805+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:25.261 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:25.299 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T16:09:25.367 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:09:25.401 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2020-09-08T16:09:25.477 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:09:25.585 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:09:25.587 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:09:25.605 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:25 smithi055 bash[9296]: cluster 2020-09-08T16:09:24.106551+0000 mgr.x (mgr.24118) 1262 : cluster [DBG] pgmap v1882: 17 pgs: 3 creating+peering, 2 unknown, 12 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:25.605 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:25 smithi055 bash[9296]: audit 2020-09-08T16:09:24.856074+0000 mon.a (mon.0) 2216 : audit [INF] from='client.? v1:172.21.15.104:0/1074714832' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:09:25.606 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:09:25 smithi055 bash[11956]: debug 2020-09-08T16:09:25.480+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:25.606 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:09:25 smithi055 bash[14301]: debug 2020-09-08T16:09:25.264+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:25.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:25 smithi166 bash[9446]: cluster 2020-09-08T16:09:24.106551+0000 mgr.x (mgr.24118) 1262 : cluster [DBG] pgmap v1882: 17 pgs: 3 creating+peering, 2 unknown, 12 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:25.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:25 smithi166 bash[9446]: audit 2020-09-08T16:09:24.856074+0000 mon.a (mon.0) 2216 : audit [INF] from='client.? v1:172.21.15.104:0/1074714832' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:09:25.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:25 smithi104 bash[9203]: cluster 2020-09-08T16:09:24.106551+0000 mgr.x (mgr.24118) 1262 : cluster [DBG] pgmap v1882: 17 pgs: 3 creating+peering, 2 unknown, 12 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:25.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:25 smithi104 bash[9203]: audit 2020-09-08T16:09:24.856074+0000 mon.a (mon.0) 2216 : audit [INF] from='client.? v1:172.21.15.104:0/1074714832' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:09:25.989 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:09:26.095 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:26.283 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:09:25 smithi055 bash[19003]: debug 2020-09-08T16:09:25.992+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:26.284 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:26.390 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:26.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:26 smithi166 bash[9446]: cluster 2020-09-08T16:09:25.287295+0000 mon.a (mon.0) 2217 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:09:26.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:26 smithi166 bash[9446]: audit 2020-09-08T16:09:25.301479+0000 mon.a (mon.0) 2218 : audit [INF] from='client.? v1:172.21.15.104:0/1074714832' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:09:26.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:26 smithi166 bash[9446]: cluster 2020-09-08T16:09:25.301551+0000 mon.a (mon.0) 2219 : cluster [DBG] osdmap e705: 12 total, 12 up, 12 in 2020-09-08T16:09:26.658 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:09:26 smithi055 bash[14301]: debug 2020-09-08T16:09:26.284+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:26.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:26 smithi055 bash[9296]: cluster 2020-09-08T16:09:25.287295+0000 mon.a (mon.0) 2217 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:09:26.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:26 smithi055 bash[9296]: audit 2020-09-08T16:09:25.301479+0000 mon.a (mon.0) 2218 : audit [INF] from='client.? v1:172.21.15.104:0/1074714832' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:09:26.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:26 smithi055 bash[9296]: cluster 2020-09-08T16:09:25.301551+0000 mon.a (mon.0) 2219 : cluster [DBG] osdmap e705: 12 total, 12 up, 12 in 2020-09-08T16:09:26.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:26 smithi104 bash[9203]: cluster 2020-09-08T16:09:25.287295+0000 mon.a (mon.0) 2217 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:09:26.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:26 smithi104 bash[9203]: audit 2020-09-08T16:09:25.301479+0000 mon.a (mon.0) 2218 : audit [INF] from='client.? v1:172.21.15.104:0/1074714832' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:09:26.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:26 smithi104 bash[9203]: cluster 2020-09-08T16:09:25.301551+0000 mon.a (mon.0) 2219 : cluster [DBG] osdmap e705: 12 total, 12 up, 12 in 2020-09-08T16:09:26.838 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:26.946 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:27.126 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:09:26 smithi055 bash[16646]: debug 2020-09-08T16:09:26.840+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:27.241 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:27.346 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:27.393 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:27 smithi055 bash[9296]: cluster 2020-09-08T16:09:26.107199+0000 mgr.x (mgr.24118) 1263 : cluster [DBG] pgmap v1884: 17 pgs: 3 creating+peering, 14 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:27.501 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:27.606 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:27.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:27 smithi166 bash[9446]: cluster 2020-09-08T16:09:26.107199+0000 mgr.x (mgr.24118) 1263 : cluster [DBG] pgmap v1884: 17 pgs: 3 creating+peering, 14 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:27.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:09:27 smithi166 bash[11928]: debug 2020-09-08T16:09:27.501+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:27.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:27 smithi104 bash[9203]: cluster 2020-09-08T16:09:26.107199+0000 mgr.x (mgr.24118) 1263 : cluster [DBG] pgmap v1884: 17 pgs: 3 creating+peering, 14 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:27.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:09:27 smithi104 bash[15273]: debug 2020-09-08T16:09:27.242+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:27.996 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:09:28.020 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:09:28.039 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:28.042 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:09:28.069 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:09:28.097 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:09:28.129 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:09:28.144 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:09:28.155 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:09:28.183 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:09:28.212 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:09:28.248 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:09:28.270 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:09:28.272 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:09:28.306 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:09:28 smithi055 bash[14301]: debug 2020-09-08T16:09:28.039+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:28.307 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:09:28.309 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:09:28.309 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:09:28.310 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:09:28.311 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:09:28.311 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:09:28.312 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:09:28.312 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:09:28.313 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:09:28.313 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:09:28.313 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:09:28.313 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:09:28.313 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:09:28.377 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:28.616 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:09:28 smithi166 bash[14254]: debug 2020-09-08T16:09:28.273+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:28.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:28 smithi166 bash[9446]: audit 2020-09-08T16:09:27.972017+0000 mon.a (mon.0) 2220 : audit [DBG] from='client.? v1:172.21.15.104:0/219666660' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:09:28.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:28 smithi055 bash[9296]: audit 2020-09-08T16:09:27.972017+0000 mon.a (mon.0) 2220 : audit [DBG] from='client.? v1:172.21.15.104:0/219666660' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:09:28.686 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:28 smithi104 bash[9203]: audit 2020-09-08T16:09:27.972017+0000 mon.a (mon.0) 2220 : audit [DBG] from='client.? v1:172.21.15.104:0/219666660' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:09:28.687 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:28.788 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:09:28.914 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:09:28.992 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T16:09:29.019 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:29.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:09:28 smithi166 bash[14254]: debug 2020-09-08T16:09:28.913+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:29.142 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:09:29.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:09:28 smithi055 bash[16646]: debug 2020-09-08T16:09:28.659+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:29.165 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:29.282 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:09:29.302 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:09:29 smithi104 bash[15273]: debug 2020-09-08T16:09:29.166+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:29.325 INFO:teuthology.orchestra.run.smithi104.stderr:enabled application 'rados' on pool 'unique_pool_7' 2020-09-08T16:09:29.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:29 smithi166 bash[9446]: cluster 2020-09-08T16:09:28.107914+0000 mgr.x (mgr.24118) 1264 : cluster [DBG] pgmap v1885: 17 pgs: 17 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:29.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:29 smithi166 bash[9446]: audit 2020-09-08T16:09:28.600032+0000 mon.a (mon.0) 2221 : audit [INF] from='client.? v1:172.21.15.104:0/2407893266' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_7", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T16:09:29.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:29 smithi055 bash[9296]: cluster 2020-09-08T16:09:28.107914+0000 mgr.x (mgr.24118) 1264 : cluster [DBG] pgmap v1885: 17 pgs: 17 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:29.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:29 smithi055 bash[9296]: audit 2020-09-08T16:09:28.600032+0000 mon.a (mon.0) 2221 : audit [INF] from='client.? v1:172.21.15.104:0/2407893266' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_7", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T16:09:29.659 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:09:29.659 INFO:teuthology.orchestra.run.smithi055.stdout: "size": 20, 2020-09-08T16:09:29.660 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 600, 2020-09-08T16:09:29.660 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T16:09:29.660 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.660 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:43069 7.c 7:3d43a1a7:::benchmark_data_smithi166_48533_object43068:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e561)", 2020-09-08T16:09:29.661 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:00:56.148019+0000", 2020-09-08T16:09:29.661 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 513.51307537499997, 2020-09-08T16:09:29.661 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.26833472600000002, 2020-09-08T16:09:29.661 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:29.661 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:29.662 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:29.662 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:09:29.662 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:09:29.662 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 43069 2020-09-08T16:09:29.662 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.663 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:29.663 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.663 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:29.663 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.148019+0000", 2020-09-08T16:09:29.663 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:29.664 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.664 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.664 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:29.664 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.148019+0000", 2020-09-08T16:09:29.664 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999576 2020-09-08T16:09:29.665 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.665 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.665 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:29.665 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.147977+0000", 2020-09-08T16:09:29.665 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016601499999999999 2020-09-08T16:09:29.666 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.666 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.666 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:29.666 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.148143+0000", 2020-09-08T16:09:29.666 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4425e-05 2020-09-08T16:09:29.667 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.667 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.667 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:29.667 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.148157+0000", 2020-09-08T16:09:29.667 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3500000000000002e-06 2020-09-08T16:09:29.668 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.668 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.668 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:29.668 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.148163+0000", 2020-09-08T16:09:29.668 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.2063999999999994e-05 2020-09-08T16:09:29.668 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.669 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.669 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:29.669 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.148225+0000", 2020-09-08T16:09:29.669 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000158763 2020-09-08T16:09:29.669 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.670 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.670 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:29.670 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.148384+0000", 2020-09-08T16:09:29.670 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00018265100000000001 2020-09-08T16:09:29.670 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.671 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.671 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:09:29.671 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.148567+0000", 2020-09-08T16:09:29.671 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.001758176 2020-09-08T16:09:29.671 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.672 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.672 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:29.672 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.150325+0000", 2020-09-08T16:09:29.672 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.26582709399999999 2020-09-08T16:09:29.672 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.673 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.673 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:29.673 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.416152+0000", 2020-09-08T16:09:29.673 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000104687 2020-09-08T16:09:29.673 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.674 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.675 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:29.675 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.416257+0000", 2020-09-08T16:09:29.675 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.6906999999999999e-05 2020-09-08T16:09:29.675 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.676 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.676 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:29.676 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:00:56.416354+0000", 2020-09-08T16:09:29.676 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.26833472600000002 2020-09-08T16:09:29.676 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.677 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:29.677 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.677 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.678 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.678 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:111353 7.c 7:399953f5:::benchmark_data_smithi166_48533_object111352:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e575)", 2020-09-08T16:09:29.678 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:01:40.772336+0000", 2020-09-08T16:09:29.678 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 468.88875869200001, 2020-09-08T16:09:29.678 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.24925328599999999, 2020-09-08T16:09:29.679 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:29.679 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:29.679 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:29.679 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:09:29.679 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:09:29.680 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 111353 2020-09-08T16:09:29.680 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.680 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:29.680 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.680 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:29.681 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.772336+0000", 2020-09-08T16:09:29.681 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:29.681 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.681 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.681 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:29.682 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.772336+0000", 2020-09-08T16:09:29.690 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999294 2020-09-08T16:09:29.690 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.690 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.691 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:29.691 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.772265+0000", 2020-09-08T16:09:29.691 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000232969 2020-09-08T16:09:29.691 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.692 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.692 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:29.692 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.772498+0000", 2020-09-08T16:09:29.692 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.0729999999999999e-05 2020-09-08T16:09:29.693 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.693 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.693 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:29.693 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.772529+0000", 2020-09-08T16:09:29.694 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.3115999999999999e-05 2020-09-08T16:09:29.694 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.694 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.694 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:29.694 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.772542+0000", 2020-09-08T16:09:29.695 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000455926 2020-09-08T16:09:29.695 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.695 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.695 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:29.696 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.772998+0000", 2020-09-08T16:09:29.696 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00067665900000000003 2020-09-08T16:09:29.696 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.696 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.696 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:29.697 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.773675+0000", 2020-09-08T16:09:29.697 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016333799999999999 2020-09-08T16:09:29.697 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.697 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.697 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:09:29.698 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.773838+0000", 2020-09-08T16:09:29.698 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0042308370000000003 2020-09-08T16:09:29.698 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.698 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.699 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:29.699 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.778069+0000", 2020-09-08T16:09:29.699 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.24343125800000001 2020-09-08T16:09:29.699 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.699 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.700 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:29.700 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:41.021500+0000", 2020-09-08T16:09:29.700 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2710000000000002e-05 2020-09-08T16:09:29.700 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.700 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.701 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:29.701 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:41.021553+0000", 2020-09-08T16:09:29.701 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.6216e-05 2020-09-08T16:09:29.701 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.702 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.702 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:29.702 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:41.021589+0000", 2020-09-08T16:09:29.702 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.24925328599999999 2020-09-08T16:09:29.702 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.703 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:29.703 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.703 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.703 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.703 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:111366 7.c 7:3b1f0ab3:::benchmark_data_smithi166_48533_object111365:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e575)", 2020-09-08T16:09:29.704 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:01:40.784524+0000", 2020-09-08T16:09:29.704 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 468.87657059200001, 2020-09-08T16:09:29.704 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.237212543, 2020-09-08T16:09:29.704 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:29.704 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:29.705 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:29.705 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:09:29.705 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:09:29.714 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 111366 2020-09-08T16:09:29.714 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.715 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:29.715 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.715 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:29.715 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.784524+0000", 2020-09-08T16:09:29.716 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:29.716 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.716 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.716 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:29.717 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.784524+0000", 2020-09-08T16:09:29.717 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999747 2020-09-08T16:09:29.717 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.717 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.738 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:29.738 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.784498+0000", 2020-09-08T16:09:29.738 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.2670000000000006e-05 2020-09-08T16:09:29.738 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.739 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.739 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:29.739 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.784591+0000", 2020-09-08T16:09:29.739 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1248e-05 2020-09-08T16:09:29.739 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.740 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.740 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:29.740 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.784602+0000", 2020-09-08T16:09:29.741 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.4279999999999998e-06 2020-09-08T16:09:29.741 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.741 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.741 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:29.742 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.784607+0000", 2020-09-08T16:09:29.742 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.7509000000000006e-05 2020-09-08T16:09:29.742 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.742 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.743 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:29.743 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.784684+0000", 2020-09-08T16:09:29.743 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000172492 2020-09-08T16:09:29.743 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.743 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.744 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:29.744 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.784857+0000", 2020-09-08T16:09:29.744 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000193253 2020-09-08T16:09:29.744 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.744 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.745 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:09:29.745 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.785050+0000", 2020-09-08T16:09:29.745 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.003769166 2020-09-08T16:09:29.745 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.745 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.746 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:29.746 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:40.788819+0000", 2020-09-08T16:09:29.746 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.23284532299999999 2020-09-08T16:09:29.746 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.746 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.747 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:29.747 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:41.021664+0000", 2020-09-08T16:09:29.747 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.189e-05 2020-09-08T16:09:29.747 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.747 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.748 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:29.748 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:41.021706+0000", 2020-09-08T16:09:29.748 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.9927e-05 2020-09-08T16:09:29.749 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.749 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.749 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:29.749 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:01:41.021736+0000", 2020-09-08T16:09:29.749 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.237212543 2020-09-08T16:09:29.750 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.750 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:29.750 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.751 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.751 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.751 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:132325 7.d 7:bd65781f:::benchmark_data_smithi166_48533_object132324:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e582)", 2020-09-08T16:09:29.751 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:02:08.129672+0000", 2020-09-08T16:09:29.752 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 441.53142281800001, 2020-09-08T16:09:29.752 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.30912979699999998, 2020-09-08T16:09:29.752 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:29.752 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:29.753 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:29.753 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:09:29.753 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:09:29.753 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 132325 2020-09-08T16:09:29.753 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.754 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:29.754 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.754 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:29.754 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.129672+0000", 2020-09-08T16:09:29.754 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:29.754 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.755 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.755 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:29.755 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.129672+0000", 2020-09-08T16:09:29.755 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999266 2020-09-08T16:09:29.755 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.756 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.756 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:29.756 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.129598+0000", 2020-09-08T16:09:29.756 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00023092699999999999 2020-09-08T16:09:29.757 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.757 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.757 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:29.757 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.129829+0000", 2020-09-08T16:09:29.758 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.8436999999999999e-05 2020-09-08T16:09:29.758 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.758 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.758 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:29.758 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.129857+0000", 2020-09-08T16:09:29.759 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1005999999999999e-05 2020-09-08T16:09:29.759 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.759 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.759 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:29.760 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.129868+0000", 2020-09-08T16:09:29.760 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.5742000000000003e-05 2020-09-08T16:09:29.760 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.760 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.760 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:29.761 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.129924+0000", 2020-09-08T16:09:29.761 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000117793 2020-09-08T16:09:29.761 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.761 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.761 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:29.762 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.130042+0000", 2020-09-08T16:09:29.762 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000125203 2020-09-08T16:09:29.762 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.762 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.762 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T16:09:29.763 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.130167+0000", 2020-09-08T16:09:29.763 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.308075339 2020-09-08T16:09:29.763 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.763 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.763 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:29.764 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.438242+0000", 2020-09-08T16:09:29.764 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000157093 2020-09-08T16:09:29.764 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.764 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.765 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:29.765 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.438399+0000", 2020-09-08T16:09:29.765 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00024729500000000002 2020-09-08T16:09:29.766 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.766 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.766 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:29.766 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.438647+0000", 2020-09-08T16:09:29.767 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015457000000000001 2020-09-08T16:09:29.767 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.767 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.767 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:29.767 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.438801+0000", 2020-09-08T16:09:29.768 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.30912979699999998 2020-09-08T16:09:29.768 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.768 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:29.768 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.768 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.769 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.769 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:132328 7.d 7:be1d4a9c:::benchmark_data_smithi166_48533_object132327:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e582)", 2020-09-08T16:09:29.769 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:02:08.131595+0000", 2020-09-08T16:09:29.769 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 441.52949899399999, 2020-09-08T16:09:29.770 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.31398954499999998, 2020-09-08T16:09:29.770 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:29.770 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:29.770 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:29.771 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:09:29.771 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:09:29.771 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 132328 2020-09-08T16:09:29.771 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.771 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:29.772 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.772 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:29.772 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.131595+0000", 2020-09-08T16:09:29.772 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:29.772 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.773 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.773 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:29.773 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.131595+0000", 2020-09-08T16:09:29.773 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999657 2020-09-08T16:09:29.773 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.774 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.774 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:29.774 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.131561+0000", 2020-09-08T16:09:29.774 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011364100000000001 2020-09-08T16:09:29.775 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.775 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.775 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:29.775 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.131675+0000", 2020-09-08T16:09:29.776 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4521e-05 2020-09-08T16:09:29.776 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.776 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.776 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:29.777 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.131689+0000", 2020-09-08T16:09:29.777 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.7000000000000002e-06 2020-09-08T16:09:29.777 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.777 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.778 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:29.778 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.131696+0000", 2020-09-08T16:09:29.778 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0024308530000000002 2020-09-08T16:09:29.778 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.778 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.778 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:29.779 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.134127+0000", 2020-09-08T16:09:29.779 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000221628 2020-09-08T16:09:29.779 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.779 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.779 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:29.780 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.134348+0000", 2020-09-08T16:09:29.780 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.29382143500000002 2020-09-08T16:09:29.780 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.780 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.781 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T16:09:29.781 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.428170+0000", 2020-09-08T16:09:29.781 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.010774832 2020-09-08T16:09:29.781 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.781 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.782 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:29.782 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.438944+0000", 2020-09-08T16:09:29.782 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0065263630000000003 2020-09-08T16:09:29.782 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.783 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.783 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:29.783 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.445471+0000", 2020-09-08T16:09:29.783 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.8069999999999998e-05 2020-09-08T16:09:29.783 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.784 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.784 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:29.784 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.445529+0000", 2020-09-08T16:09:29.784 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.5976000000000001e-05 2020-09-08T16:09:29.784 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.785 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.785 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:29.785 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:08.445585+0000", 2020-09-08T16:09:29.785 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.31398954499999998 2020-09-08T16:09:29.786 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.786 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:29.786 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.786 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.787 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.787 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:167413 7.d 7:b9730c85:::benchmark_data_smithi166_48533_object32896:head [delete] snapc 0=[] ondisk+write+known_if_redirected e591)", 2020-09-08T16:09:29.787 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:02:42.979998+0000", 2020-09-08T16:09:29.787 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 406.68109605199999, 2020-09-08T16:09:29.787 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.20686253700000001, 2020-09-08T16:09:29.788 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:29.788 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:29.788 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:29.788 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:09:29.789 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:09:29.789 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 167413 2020-09-08T16:09:29.789 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.789 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:29.789 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.790 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:29.790 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.979998+0000", 2020-09-08T16:09:29.790 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:29.790 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.791 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.791 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:29.791 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.979998+0000", 2020-09-08T16:09:29.791 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999918 2020-09-08T16:09:29.792 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.792 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.792 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:29.792 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.979916+0000", 2020-09-08T16:09:29.792 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000184785 2020-09-08T16:09:29.793 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.793 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.793 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:29.793 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.980101+0000", 2020-09-08T16:09:29.794 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.0874999999999998e-05 2020-09-08T16:09:29.794 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.794 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.794 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:29.795 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.980132+0000", 2020-09-08T16:09:29.795 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.43e-05 2020-09-08T16:09:29.795 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.795 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.796 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:29.796 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.980146+0000", 2020-09-08T16:09:29.796 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0064882969999999996 2020-09-08T16:09:29.796 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.797 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.797 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:29.797 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.986635+0000", 2020-09-08T16:09:29.797 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00023545400000000001 2020-09-08T16:09:29.798 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.798 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.798 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:29.798 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.986870+0000", 2020-09-08T16:09:29.798 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00023533699999999999 2020-09-08T16:09:29.799 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.799 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.799 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T16:09:29.800 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.987105+0000", 2020-09-08T16:09:29.800 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.199244117 2020-09-08T16:09:29.800 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.800 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.801 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:29.801 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:43.186349+0000", 2020-09-08T16:09:29.801 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00019089399999999999 2020-09-08T16:09:29.801 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.801 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.802 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:29.802 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:43.186540+0000", 2020-09-08T16:09:29.802 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017134499999999999 2020-09-08T16:09:29.802 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.803 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.803 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:29.803 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:43.186712+0000", 2020-09-08T16:09:29.803 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014912000000000001 2020-09-08T16:09:29.804 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.804 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.804 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:29.804 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:43.186861+0000", 2020-09-08T16:09:29.804 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.20686253700000001 2020-09-08T16:09:29.805 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.805 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:29.805 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.805 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.806 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.806 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:167424 7.d 7:bed25e6c:::benchmark_data_smithi166_48533_object32907:head [delete] snapc 0=[] ondisk+write+known_if_redirected e591)", 2020-09-08T16:09:29.806 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:02:42.981480+0000", 2020-09-08T16:09:29.806 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 406.67961397599998, 2020-09-08T16:09:29.807 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.206409863, 2020-09-08T16:09:29.807 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:29.807 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:29.807 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:29.807 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:09:29.808 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:09:29.808 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 167424 2020-09-08T16:09:29.808 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.808 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:29.808 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.809 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:29.809 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.981480+0000", 2020-09-08T16:09:29.809 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:29.809 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.809 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.810 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:29.810 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.981480+0000", 2020-09-08T16:09:29.810 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999251 2020-09-08T16:09:29.810 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.810 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.811 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:29.811 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.981405+0000", 2020-09-08T16:09:29.811 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000164981 2020-09-08T16:09:29.811 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.812 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.812 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:29.812 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.981570+0000", 2020-09-08T16:09:29.812 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.3201e-05 2020-09-08T16:09:29.812 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.812 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.813 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:29.813 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.981593+0000", 2020-09-08T16:09:29.813 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.6940000000000007e-06 2020-09-08T16:09:29.813 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.813 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.814 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:29.814 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.981603+0000", 2020-09-08T16:09:29.814 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.017031066000000001 2020-09-08T16:09:29.814 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.814 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.815 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:29.815 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.998634+0000", 2020-09-08T16:09:29.815 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015101099999999999 2020-09-08T16:09:29.815 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.816 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.816 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:29.816 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.998785+0000", 2020-09-08T16:09:29.816 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015327900000000001 2020-09-08T16:09:29.816 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.817 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.817 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T16:09:29.817 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:42.998939+0000", 2020-09-08T16:09:29.817 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.18845735599999999 2020-09-08T16:09:29.817 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.818 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.818 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:29.818 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:43.187396+0000", 2020-09-08T16:09:29.818 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00022437600000000001 2020-09-08T16:09:29.818 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.819 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.819 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:29.819 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:43.187620+0000", 2020-09-08T16:09:29.819 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014537400000000001 2020-09-08T16:09:29.819 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.820 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.820 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:29.820 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:43.187766+0000", 2020-09-08T16:09:29.820 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012461800000000001 2020-09-08T16:09:29.820 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.821 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.821 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:29.821 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:02:43.187890+0000", 2020-09-08T16:09:29.821 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.206409863 2020-09-08T16:09:29.822 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.822 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:29.822 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.822 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.822 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.823 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:193178 7.7 7:e8a92310:::benchmark_data_smithi166_48533_object58661:head [delete] snapc 0=[] ondisk+write+known_if_redirected e598)", 2020-09-08T16:09:29.823 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:09.592514+0000", 2020-09-08T16:09:29.823 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 380.06858043300002, 2020-09-08T16:09:29.823 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.59101522799999995, 2020-09-08T16:09:29.824 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:29.824 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:29.824 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:29.824 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:09:29.824 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:09:29.825 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 193178 2020-09-08T16:09:29.825 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.825 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:29.825 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.825 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:29.826 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.592514+0000", 2020-09-08T16:09:29.826 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:29.826 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.826 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.826 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:29.827 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.592514+0000", 2020-09-08T16:09:29.827 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999399 2020-09-08T16:09:29.827 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.827 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.827 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:29.828 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.592454+0000", 2020-09-08T16:09:29.828 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013446700000000001 2020-09-08T16:09:29.828 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.828 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.828 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:29.828 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.592588+0000", 2020-09-08T16:09:29.829 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.1747999999999999e-05 2020-09-08T16:09:29.829 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.829 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.829 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:29.829 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.592610+0000", 2020-09-08T16:09:29.830 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.2560000000000008e-06 2020-09-08T16:09:29.830 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.830 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.830 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:29.830 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.592619+0000", 2020-09-08T16:09:29.831 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0037480899999999999 2020-09-08T16:09:29.831 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.831 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.831 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:29.831 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.596367+0000", 2020-09-08T16:09:29.832 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000146856 2020-09-08T16:09:29.832 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.832 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.832 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:29.832 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.596514+0000", 2020-09-08T16:09:29.833 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.6421000000000004e-05 2020-09-08T16:09:29.833 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.833 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.833 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 8", 2020-09-08T16:09:29.833 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.596611+0000", 2020-09-08T16:09:29.834 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.57584053899999998 2020-09-08T16:09:29.834 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.834 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.834 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:29.834 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172451+0000", 2020-09-08T16:09:29.834 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.010788818 2020-09-08T16:09:29.835 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.835 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.835 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:29.835 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183240+0000", 2020-09-08T16:09:29.836 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016227700000000001 2020-09-08T16:09:29.836 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.836 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.836 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:29.836 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183402+0000", 2020-09-08T16:09:29.837 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012677200000000001 2020-09-08T16:09:29.837 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.837 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.837 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:29.837 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183529+0000", 2020-09-08T16:09:29.838 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.59101522799999995 2020-09-08T16:09:29.838 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.838 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:29.838 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.838 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.839 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.839 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:193184 7.7 7:e3f87857:::benchmark_data_smithi166_48533_object58667:head [delete] snapc 0=[] ondisk+write+known_if_redirected e598)", 2020-09-08T16:09:29.839 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:09.594130+0000", 2020-09-08T16:09:29.839 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 380.06696409800003, 2020-09-08T16:09:29.840 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.59013966100000004, 2020-09-08T16:09:29.840 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:29.840 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:29.840 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:29.840 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:09:29.841 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:09:29.841 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 193184 2020-09-08T16:09:29.841 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.841 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:29.842 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.842 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:29.842 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.594130+0000", 2020-09-08T16:09:29.842 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:29.843 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.843 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.843 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:29.843 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.594130+0000", 2020-09-08T16:09:29.843 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999776 2020-09-08T16:09:29.844 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.844 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.844 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:29.844 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.594108+0000", 2020-09-08T16:09:29.844 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.1103000000000002e-05 2020-09-08T16:09:29.845 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.845 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.845 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:29.845 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.594159+0000", 2020-09-08T16:09:29.846 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.2590000000000006e-06 2020-09-08T16:09:29.846 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.846 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.846 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:29.847 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.594168+0000", 2020-09-08T16:09:29.847 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.7550000000000002e-06 2020-09-08T16:09:29.847 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.847 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.847 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:29.848 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.594173+0000", 2020-09-08T16:09:29.848 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0075788399999999999 2020-09-08T16:09:29.848 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.848 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.849 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:29.849 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.601752+0000", 2020-09-08T16:09:29.849 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000142695 2020-09-08T16:09:29.849 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.849 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.850 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:29.850 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.601894+0000", 2020-09-08T16:09:29.850 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016692000000000001 2020-09-08T16:09:29.850 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.850 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.850 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 8", 2020-09-08T16:09:29.851 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602061+0000", 2020-09-08T16:09:29.851 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.58155135999999996 2020-09-08T16:09:29.851 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.851 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.851 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:29.852 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183613+0000", 2020-09-08T16:09:29.852 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00041249499999999997 2020-09-08T16:09:29.852 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.852 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.853 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:29.853 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.184025+0000", 2020-09-08T16:09:29.853 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013872900000000001 2020-09-08T16:09:29.853 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.853 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.854 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:29.854 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.184164+0000", 2020-09-08T16:09:29.854 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000105926 2020-09-08T16:09:29.854 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.854 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.855 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:29.855 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.184270+0000", 2020-09-08T16:09:29.855 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.59013966100000004 2020-09-08T16:09:29.855 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.855 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:29.856 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.856 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.856 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.856 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:193186 7.c 7:356146cd:::benchmark_data_smithi166_48533_object58669:head [delete] snapc 0=[] ondisk+write+known_if_redirected e598)", 2020-09-08T16:09:29.856 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:09.598767+0000", 2020-09-08T16:09:29.857 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 380.06232764100002, 2020-09-08T16:09:29.857 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.58441285899999995, 2020-09-08T16:09:29.857 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:29.857 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:29.858 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:29.858 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:09:29.858 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:09:29.858 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 193186 2020-09-08T16:09:29.858 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.859 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:29.859 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.859 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:29.859 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.598767+0000", 2020-09-08T16:09:29.859 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:29.860 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.860 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.860 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:29.860 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.598767+0000", 2020-09-08T16:09:29.861 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999766 2020-09-08T16:09:29.861 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.861 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.861 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:29.862 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.598743+0000", 2020-09-08T16:09:29.862 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2475999999999997e-05 2020-09-08T16:09:29.862 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.862 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.862 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:29.863 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.598796+0000", 2020-09-08T16:09:29.863 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.8800000000000003e-06 2020-09-08T16:09:29.863 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.863 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.864 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:29.864 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.598806+0000", 2020-09-08T16:09:29.864 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.2179999999999998e-06 2020-09-08T16:09:29.864 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.864 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.865 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:29.865 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.598811+0000", 2020-09-08T16:09:29.865 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.4370999999999997e-05 2020-09-08T16:09:29.866 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.866 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.866 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:29.866 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.598885+0000", 2020-09-08T16:09:29.867 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00014553299999999999 2020-09-08T16:09:29.867 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.867 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.867 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:29.868 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.599031+0000", 2020-09-08T16:09:29.868 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016689 2020-09-08T16:09:29.868 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.868 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.868 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:09:29.869 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.599198+0000", 2020-09-08T16:09:29.869 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.57324991800000002 2020-09-08T16:09:29.869 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.870 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.870 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:29.870 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172448+0000", 2020-09-08T16:09:29.870 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.010612709 2020-09-08T16:09:29.870 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.871 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.871 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:29.871 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183060+0000", 2020-09-08T16:09:29.871 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.0356000000000003e-05 2020-09-08T16:09:29.872 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.872 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.872 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:29.872 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183121+0000", 2020-09-08T16:09:29.872 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.8757e-05 2020-09-08T16:09:29.873 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.873 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.873 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:29.873 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183180+0000", 2020-09-08T16:09:29.874 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.58441285899999995 2020-09-08T16:09:29.874 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.874 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:29.874 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.874 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.875 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.875 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop_reply(client.36113.0:193175 7.7 e598/553 ondisk, result = 0)", 2020-09-08T16:09:29.875 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:09.602167+0000", 2020-09-08T16:09:29.875 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 380.05892726399998, 2020-09-08T16:09:29.876 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.570143972, 2020-09-08T16:09:29.876 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:29.876 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "started", 2020-09-08T16:09:29.876 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:29.877 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.877 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:29.877 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602167+0000", 2020-09-08T16:09:29.878 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:29.878 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.878 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.878 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:29.878 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602167+0000", 2020-09-08T16:09:29.879 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.4880999999999999e-05 2020-09-08T16:09:29.879 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.879 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.879 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:29.880 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602222+0000", 2020-09-08T16:09:29.880 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4469999999999995e-06 2020-09-08T16:09:29.880 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.880 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.880 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:29.881 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602231+0000", 2020-09-08T16:09:29.881 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5197999999999999e-05 2020-09-08T16:09:29.881 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.881 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.881 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:29.882 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602247+0000", 2020-09-08T16:09:29.882 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.7388e-05 2020-09-08T16:09:29.882 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.882 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.882 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:29.883 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602264+0000", 2020-09-08T16:09:29.883 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0053359840000000002 2020-09-08T16:09:29.883 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.883 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.883 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:29.884 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607600+0000", 2020-09-08T16:09:29.884 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1474000000000001e-05 2020-09-08T16:09:29.884 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.884 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.884 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:29.885 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607611+0000", 2020-09-08T16:09:29.885 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.56469959999999997 2020-09-08T16:09:29.885 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.885 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.885 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:29.886 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172311+0000", 2020-09-08T16:09:29.886 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.570143972 2020-09-08T16:09:29.886 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.886 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:29.886 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.887 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.887 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.887 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_repop_reply(client.36113.0:193178 7.7 e598/553 ondisk, result = 0)", 2020-09-08T16:09:29.887 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:09.602323+0000", 2020-09-08T16:09:29.887 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 380.05877105500002, 2020-09-08T16:09:29.888 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.57013256700000003, 2020-09-08T16:09:29.888 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:29.888 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "started", 2020-09-08T16:09:29.888 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:29.888 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.889 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:29.889 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602323+0000", 2020-09-08T16:09:29.889 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:29.889 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.889 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.889 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:29.890 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602323+0000", 2020-09-08T16:09:29.890 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.0728999999999997e-05 2020-09-08T16:09:29.890 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.890 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.891 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:29.891 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602354+0000", 2020-09-08T16:09:29.891 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.826e-06 2020-09-08T16:09:29.891 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.891 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.892 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:29.892 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602359+0000", 2020-09-08T16:09:29.892 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.8469999999999996e-06 2020-09-08T16:09:29.892 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.892 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.893 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:29.893 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602367+0000", 2020-09-08T16:09:29.893 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.4320000000000001e-06 2020-09-08T16:09:29.893 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.894 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.894 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:29.894 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.602371+0000", 2020-09-08T16:09:29.894 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.57003088400000002 2020-09-08T16:09:29.895 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.895 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.895 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:29.895 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172402+0000", 2020-09-08T16:09:29.895 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.7949e-05 2020-09-08T16:09:29.896 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.896 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.896 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:29.896 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172430+0000", 2020-09-08T16:09:29.896 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.5899999999999999e-05 2020-09-08T16:09:29.897 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.897 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.897 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:29.897 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172456+0000", 2020-09-08T16:09:29.897 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.57013256700000003 2020-09-08T16:09:29.898 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.898 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:29.898 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.898 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.898 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.899 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:193191 7.c 7:3a26bee4:::benchmark_data_smithi166_48533_object58674:head [delete] snapc 0=[] ondisk+write+known_if_redirected e598)", 2020-09-08T16:09:29.899 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:09.605163+0000", 2020-09-08T16:09:29.899 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 380.05593170399999, 2020-09-08T16:09:29.899 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.57837770200000005, 2020-09-08T16:09:29.899 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:29.899 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:29.900 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:29.900 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:09:29.900 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:09:29.900 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 193191 2020-09-08T16:09:29.900 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.901 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:29.901 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.901 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:29.901 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605163+0000", 2020-09-08T16:09:29.901 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:29.902 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.902 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.902 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:29.902 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605163+0000", 2020-09-08T16:09:29.902 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999628 2020-09-08T16:09:29.903 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.903 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.903 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:29.903 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605126+0000", 2020-09-08T16:09:29.903 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.3867000000000002e-05 2020-09-08T16:09:29.904 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.904 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.904 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:29.904 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605210+0000", 2020-09-08T16:09:29.904 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4372000000000001e-05 2020-09-08T16:09:29.905 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.905 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.905 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:29.905 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605224+0000", 2020-09-08T16:09:29.905 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.6499999999999999e-06 2020-09-08T16:09:29.906 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.906 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.906 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:29.906 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605231+0000", 2020-09-08T16:09:29.906 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00034729100000000001 2020-09-08T16:09:29.907 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.907 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.907 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:29.907 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605578+0000", 2020-09-08T16:09:29.907 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015052500000000001 2020-09-08T16:09:29.908 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.908 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.908 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:29.908 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605728+0000", 2020-09-08T16:09:29.908 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015432799999999999 2020-09-08T16:09:29.909 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.909 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.909 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:09:29.909 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605883+0000", 2020-09-08T16:09:29.909 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.56658740699999999 2020-09-08T16:09:29.910 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.910 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.910 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:29.910 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172470+0000", 2020-09-08T16:09:29.910 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.010821318999999999 2020-09-08T16:09:29.911 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.911 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.911 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:29.911 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183291+0000", 2020-09-08T16:09:29.911 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000148677 2020-09-08T16:09:29.911 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.912 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.912 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:29.912 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183440+0000", 2020-09-08T16:09:29.912 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00010026000000000001 2020-09-08T16:09:29.912 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.913 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.913 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:29.913 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183540+0000", 2020-09-08T16:09:29.913 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.57837770200000005 2020-09-08T16:09:29.913 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.914 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:29.914 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.914 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.914 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.914 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:193193 7.c 7:300ec92f:::benchmark_data_smithi166_48533_object58676:head [delete] snapc 0=[] ondisk+write+known_if_redirected e598)", 2020-09-08T16:09:29.915 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:09.605299+0000", 2020-09-08T16:09:29.915 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 380.05579579800002, 2020-09-08T16:09:29.915 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.59181674100000004, 2020-09-08T16:09:29.915 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:29.915 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:29.916 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:29.916 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:09:29.916 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:09:29.916 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 193193 2020-09-08T16:09:29.916 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.917 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:29.917 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.917 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:29.917 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605299+0000", 2020-09-08T16:09:29.917 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:29.918 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.918 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.918 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:29.918 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605299+0000", 2020-09-08T16:09:29.918 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999661 2020-09-08T16:09:29.919 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.919 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.919 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:29.919 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605265+0000", 2020-09-08T16:09:29.919 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.5384999999999997e-05 2020-09-08T16:09:29.919 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.920 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.920 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:29.920 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605340+0000", 2020-09-08T16:09:29.920 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.1066999999999999e-05 2020-09-08T16:09:29.920 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.921 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.921 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:29.921 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605351+0000", 2020-09-08T16:09:29.921 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.6249999999999998e-06 2020-09-08T16:09:29.922 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.922 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.922 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:29.922 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.605356+0000", 2020-09-08T16:09:29.922 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.56720643599999998 2020-09-08T16:09:29.923 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.923 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.923 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:29.923 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172562+0000", 2020-09-08T16:09:29.924 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00041356200000000002 2020-09-08T16:09:29.924 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.924 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.924 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:29.924 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172976+0000", 2020-09-08T16:09:29.925 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00047280800000000002 2020-09-08T16:09:29.925 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.925 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.925 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 3", 2020-09-08T16:09:29.926 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.173449+0000", 2020-09-08T16:09:29.926 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.010238517000000001 2020-09-08T16:09:29.926 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.926 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.926 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:29.927 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.183687+0000", 2020-09-08T16:09:29.927 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.013308952000000001 2020-09-08T16:09:29.927 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.927 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.928 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:29.928 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.196996+0000", 2020-09-08T16:09:29.928 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.583e-05 2020-09-08T16:09:29.928 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.929 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.929 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:29.929 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.197042+0000", 2020-09-08T16:09:29.929 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.3201000000000006e-05 2020-09-08T16:09:29.930 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.930 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.930 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:29.930 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.197115+0000", 2020-09-08T16:09:29.930 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.59181674100000004 2020-09-08T16:09:29.931 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.931 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:29.931 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.931 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.932 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.932 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:193194 7.d 7:bf0541f8:::benchmark_data_smithi166_48533_object58677:head [delete] snapc 0=[] ondisk+write+known_if_redirected e598)", 2020-09-08T16:09:29.932 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:09.607175+0000", 2020-09-08T16:09:29.932 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 380.05391940499999, 2020-09-08T16:09:29.932 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.56647141700000003, 2020-09-08T16:09:29.933 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:29.933 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:29.933 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:29.933 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:09:29.934 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:09:29.934 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 193194 2020-09-08T16:09:29.934 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.934 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:29.934 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.935 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:29.935 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607175+0000", 2020-09-08T16:09:29.935 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:29.935 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.936 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.936 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:29.936 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607175+0000", 2020-09-08T16:09:29.936 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999585 2020-09-08T16:09:29.936 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.937 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.937 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:29.937 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607133+0000", 2020-09-08T16:09:29.937 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 9.4347999999999999e-05 2020-09-08T16:09:29.937 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.938 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.938 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:29.938 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607228+0000", 2020-09-08T16:09:29.938 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5886999999999999e-05 2020-09-08T16:09:29.938 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.939 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.939 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:29.939 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607243+0000", 2020-09-08T16:09:29.939 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.1749999999999999e-06 2020-09-08T16:09:29.939 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.940 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.940 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:29.940 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607251+0000", 2020-09-08T16:09:29.940 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3350999999999997e-05 2020-09-08T16:09:29.940 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.940 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.941 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:29.941 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607314+0000", 2020-09-08T16:09:29.941 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013483200000000001 2020-09-08T16:09:29.941 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.941 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.942 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:29.942 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607449+0000", 2020-09-08T16:09:29.942 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000140638 2020-09-08T16:09:29.942 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.942 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.943 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T16:09:29.943 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607589+0000", 2020-09-08T16:09:29.943 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.56484538299999998 2020-09-08T16:09:29.943 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.943 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.944 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:29.944 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172435+0000", 2020-09-08T16:09:29.944 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00094178999999999997 2020-09-08T16:09:29.944 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.944 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.945 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:29.945 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.173377+0000", 2020-09-08T16:09:29.945 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015073699999999999 2020-09-08T16:09:29.945 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.945 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.946 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:29.946 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.173527+0000", 2020-09-08T16:09:29.946 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011898 2020-09-08T16:09:29.946 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.946 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.946 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:29.947 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.173646+0000", 2020-09-08T16:09:29.947 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.56647141700000003 2020-09-08T16:09:29.947 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.947 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:29.947 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.948 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.948 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.948 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.36113.0:193195 7.7 7:e6302a85:::benchmark_data_smithi166_48533_object58678:head [delete] snapc 0=[] ondisk+write+known_if_redirected e598)", 2020-09-08T16:09:29.948 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:03:09.607347+0000", 2020-09-08T16:09:29.948 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 380.053747182, 2020-09-08T16:09:29.949 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.58303484500000002, 2020-09-08T16:09:29.949 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:29.949 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:09:29.949 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:29.949 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.36113", 2020-09-08T16:09:29.950 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:09:29.950 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 193195 2020-09-08T16:09:29.950 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.950 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:29.950 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.951 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:29.951 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607347+0000", 2020-09-08T16:09:29.951 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:29.951 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.951 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.951 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:29.952 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607347+0000", 2020-09-08T16:09:29.952 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999571 2020-09-08T16:09:29.952 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.952 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.953 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:29.953 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607304+0000", 2020-09-08T16:09:29.953 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00011198500000000001 2020-09-08T16:09:29.953 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.954 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.954 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:29.954 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607416+0000", 2020-09-08T16:09:29.954 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.7640000000000001e-05 2020-09-08T16:09:29.954 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.955 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.955 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:29.955 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607434+0000", 2020-09-08T16:09:29.955 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 5.9519999999999999e-06 2020-09-08T16:09:29.955 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.956 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.956 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:29.956 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:09.607440+0000", 2020-09-08T16:09:29.956 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.56508348200000003 2020-09-08T16:09:29.956 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.957 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.957 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:29.957 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172523+0000", 2020-09-08T16:09:29.957 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00032738999999999999 2020-09-08T16:09:29.957 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.958 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.958 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:29.958 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.172851+0000", 2020-09-08T16:09:29.958 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000334035 2020-09-08T16:09:29.958 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.958 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.959 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 8", 2020-09-08T16:09:29.959 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.173185+0000", 2020-09-08T16:09:29.959 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.011111555 2020-09-08T16:09:29.959 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.959 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.960 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "op_commit", 2020-09-08T16:09:29.960 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.184296+0000", 2020-09-08T16:09:29.960 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0059360080000000004 2020-09-08T16:09:29.960 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.960 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.961 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:29.961 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.190232+0000", 2020-09-08T16:09:29.961 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.6401000000000002e-05 2020-09-08T16:09:29.961 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.961 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.962 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "commit_sent", 2020-09-08T16:09:29.962 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.190319+0000", 2020-09-08T16:09:29.962 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3299999999999994e-05 2020-09-08T16:09:29.962 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:29.963 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:29.963 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:09:29.963 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:03:10.190382+0000", 2020-09-08T16:09:29.963 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.58303484500000002 2020-09-08T16:09:29.964 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.964 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:29.964 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.964 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:29.964 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:29.964 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:09:29.966 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:29 smithi104 bash[9203]: cluster 2020-09-08T16:09:28.107914+0000 mgr.x (mgr.24118) 1264 : cluster [DBG] pgmap v1885: 17 pgs: 17 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:29.966 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:29 smithi104 bash[9203]: audit 2020-09-08T16:09:28.600032+0000 mon.a (mon.0) 2221 : audit [INF] from='client.? v1:172.21.15.104:0/2407893266' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_7", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2020-09-08T16:09:29.966 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:09:30.068 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:09:30.079 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:09:29 smithi104 bash[22305]: debug 2020-09-08T16:09:29.786+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:30.402 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2020-09-08T16:09:30.414 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:30 smithi104 bash[9203]: cluster 2020-09-08T16:09:28.938894+0000 osd.5 (osd.5) 119 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:30.415 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:30 smithi104 bash[9203]: cluster 2020-09-08T16:09:28.939031+0000 osd.5 (osd.5) 120 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:30.415 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:30 smithi104 bash[9203]: audit 2020-09-08T16:09:29.327670+0000 mon.a (mon.0) 2222 : audit [INF] from='client.? v1:172.21.15.104:0/2407893266' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_7", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T16:09:30.415 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:30 smithi104 bash[9203]: cluster 2020-09-08T16:09:29.327737+0000 mon.a (mon.0) 2223 : cluster [DBG] osdmap e706: 12 total, 12 up, 12 in 2020-09-08T16:09:30.415 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:09:30.520 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:09:30.540 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:09:30.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:30 smithi166 bash[9446]: cluster 2020-09-08T16:09:28.938894+0000 osd.5 (osd.5) 119 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:30.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:30 smithi166 bash[9446]: cluster 2020-09-08T16:09:28.939031+0000 osd.5 (osd.5) 120 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:30.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:30 smithi166 bash[9446]: audit 2020-09-08T16:09:29.327670+0000 mon.a (mon.0) 2222 : audit [INF] from='client.? v1:172.21.15.104:0/2407893266' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_7", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T16:09:30.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:30 smithi166 bash[9446]: cluster 2020-09-08T16:09:29.327737+0000 mon.a (mon.0) 2223 : cluster [DBG] osdmap e706: 12 total, 12 up, 12 in 2020-09-08T16:09:30.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:30 smithi055 bash[9296]: cluster 2020-09-08T16:09:28.938894+0000 osd.5 (osd.5) 119 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:30.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:30 smithi055 bash[9296]: cluster 2020-09-08T16:09:28.939031+0000 osd.5 (osd.5) 120 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:30.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:30 smithi055 bash[9296]: audit 2020-09-08T16:09:29.327670+0000 mon.a (mon.0) 2222 : audit [INF] from='client.? v1:172.21.15.104:0/2407893266' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_7", "app": "rados", "yes_i_really_mean_it": true}]': finished 2020-09-08T16:09:30.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:30 smithi055 bash[9296]: cluster 2020-09-08T16:09:29.327737+0000 mon.a (mon.0) 2223 : cluster [DBG] osdmap e706: 12 total, 12 up, 12 in 2020-09-08T16:09:30.659 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:09:30.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:09:30 smithi104 bash[22305]: debug 2020-09-08T16:09:30.418+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:30.766 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:09:30.918 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:09:31.002 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:09:30 smithi104 bash[19922]: debug 2020-09-08T16:09:30.918+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:31.022 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:31.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:09:30 smithi166 bash[18960]: debug 2020-09-08T16:09:30.657+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:31.125 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:09:31.157 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:31.258 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:09:31.263 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:09:31.325 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:09:31 smithi104 bash[17588]: debug 2020-09-08T16:09:31.158+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:31.471 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:09:31.597 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:09:31.610 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:31 smithi166 bash[9446]: cluster 2020-09-08T16:09:30.108611+0000 mgr.x (mgr.24118) 1265 : cluster [DBG] pgmap v1887: 17 pgs: 17 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:31.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:31 smithi055 bash[9296]: cluster 2020-09-08T16:09:30.108611+0000 mgr.x (mgr.24118) 1265 : cluster [DBG] pgmap v1887: 17 pgs: 17 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:31.658 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:09:31 smithi055 bash[19003]: debug 2020-09-08T16:09:31.471+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:31.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:31 smithi104 bash[9203]: cluster 2020-09-08T16:09:30.108611+0000 mgr.x (mgr.24118) 1265 : cluster [DBG] pgmap v1887: 17 pgs: 17 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:31.713 INFO:teuthology.orchestra.run.smithi166:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.2 -b 65536 --object-size 65536 -p unique_pool_7 bench 90 write' 2020-09-08T16:09:31.715 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2020-09-08T16:09:31.716 INFO:teuthology.orchestra.run:waiting for 3000 2020-09-08T16:09:31.743 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:09:31.783 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:hints = 1 2020-09-08T16:09:31.784 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2020-09-08T16:09:31.784 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Object prefix: benchmark_data_smithi166_56754 2020-09-08T16:09:31.785 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T16:09:31.785 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 0 0 0 0 0 0 - 0 2020-09-08T16:09:31.849 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:09:31.861 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:09:31 smithi166 bash[18960]: debug 2020-09-08T16:09:31.741+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:31.970 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:09:32.076 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:09:32.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:09:31 smithi166 bash[14254]: debug 2020-09-08T16:09:31.969+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:32.212 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:09:32.876 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:32 smithi104 bash[9203]: cluster 2020-09-08T16:09:31.908357+0000 osd.10 (osd.10) 103 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:32.877 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:32 smithi104 bash[9203]: cluster 2020-09-08T16:09:31.908446+0000 osd.10 (osd.10) 104 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:32.878 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:32 smithi055 bash[9296]: cluster 2020-09-08T16:09:31.908357+0000 osd.10 (osd.10) 103 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:32.878 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:32 smithi055 bash[9296]: cluster 2020-09-08T16:09:31.908446+0000 osd.10 (osd.10) 104 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:32.879 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:32 smithi166 bash[9446]: cluster 2020-09-08T16:09:31.908357+0000 osd.10 (osd.10) 103 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:32.879 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:32 smithi166 bash[9446]: cluster 2020-09-08T16:09:31.908446+0000 osd.10 (osd.10) 104 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:32.879 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:09:32 smithi166 bash[16593]: debug 2020-09-08T16:09:32.209+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:32.880 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 1 16 3044 3028 189.24 189.25 0.00346065 0.00526963 2020-09-08T16:09:32.980 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:09:33.091 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:09:33.197 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:09:33.314 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:09:33 smithi166 bash[18960]: debug 2020-09-08T16:09:33.089+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:33.315 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:09:33.419 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:09:33.616 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:09:33 smithi166 bash[14254]: debug 2020-09-08T16:09:33.313+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:33.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:33 smithi166 bash[9446]: cluster 2020-09-08T16:09:32.109094+0000 mgr.x (mgr.24118) 1266 : cluster [DBG] pgmap v1888: 17 pgs: 17 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:33.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:33 smithi166 bash[9446]: cluster 2020-09-08T16:09:32.341423+0000 mon.a (mon.0) 2224 : cluster [DBG] osdmap e707: 12 total, 12 up, 12 in 2020-09-08T16:09:33.670 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:33 smithi055 bash[9296]: cluster 2020-09-08T16:09:32.109094+0000 mgr.x (mgr.24118) 1266 : cluster [DBG] pgmap v1888: 17 pgs: 17 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:33.670 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:33 smithi055 bash[9296]: cluster 2020-09-08T16:09:32.341423+0000 mon.a (mon.0) 2224 : cluster [DBG] osdmap e707: 12 total, 12 up, 12 in 2020-09-08T16:09:33.682 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:33 smithi104 bash[9203]: cluster 2020-09-08T16:09:32.109094+0000 mgr.x (mgr.24118) 1266 : cluster [DBG] pgmap v1888: 17 pgs: 17 active+clean; 0 B data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:09:33.682 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:33 smithi104 bash[9203]: cluster 2020-09-08T16:09:32.341423+0000 mon.a (mon.0) 2224 : cluster [DBG] osdmap e707: 12 total, 12 up, 12 in 2020-09-08T16:09:33.728 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:09:33.786 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 2 16 6385 6369 199.014 208.812 0.00339761 0.00501382 2020-09-08T16:09:33.838 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:34.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:09:33 smithi104 bash[22305]: debug 2020-09-08T16:09:33.729+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:34.788 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 3 16 8957 8941 186.254 160.75 0.00265968 0.00536333 2020-09-08T16:09:35.035 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:35.136 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:09:35.340 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:09:35 smithi104 bash[15273]: debug 2020-09-08T16:09:35.025+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:35.341 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:09:35.454 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:09:35.608 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:35 smithi166 bash[9446]: cluster 2020-09-08T16:09:34.109842+0000 mgr.x (mgr.24118) 1267 : cluster [DBG] pgmap v1890: 17 pgs: 17 active+clean; 145 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s wr, 289 op/s 2020-09-08T16:09:35.609 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:35 smithi166 bash[9446]: cluster 2020-09-08T16:09:34.493795+0000 osd.3 (osd.3) 123 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:35.609 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:35 smithi166 bash[9446]: cluster 2020-09-08T16:09:34.493881+0000 osd.3 (osd.3) 124 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:35.609 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:09:35.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:35 smithi055 bash[9296]: cluster 2020-09-08T16:09:34.109842+0000 mgr.x (mgr.24118) 1267 : cluster [DBG] pgmap v1890: 17 pgs: 17 active+clean; 145 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s wr, 289 op/s 2020-09-08T16:09:35.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:35 smithi055 bash[9296]: cluster 2020-09-08T16:09:34.493795+0000 osd.3 (osd.3) 123 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:35.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:35 smithi055 bash[9296]: cluster 2020-09-08T16:09:34.493881+0000 osd.3 (osd.3) 124 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:35.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:35 smithi104 bash[9203]: cluster 2020-09-08T16:09:34.109842+0000 mgr.x (mgr.24118) 1267 : cluster [DBG] pgmap v1890: 17 pgs: 17 active+clean; 145 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s wr, 289 op/s 2020-09-08T16:09:35.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:35 smithi104 bash[9203]: cluster 2020-09-08T16:09:34.493795+0000 osd.3 (osd.3) 123 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:35.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:35 smithi104 bash[9203]: cluster 2020-09-08T16:09:34.493881+0000 osd.3 (osd.3) 124 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:35.677 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:09:35 smithi104 bash[19922]: debug 2020-09-08T16:09:35.341+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:35.718 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:09:35.785 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 4 16 12554 12538 195.883 224.812 0.00506858 0.00510157 2020-09-08T16:09:35.821 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:09:35.822 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T16:09:35.822 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.822 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.26587.0:12644 9.e 9:77bb9e2c:::benchmark_data_smithi166_56754_object12643:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e708)", 2020-09-08T16:09:35.823 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:09:35.818259+0000", 2020-09-08T16:09:35.823 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.0010873320000000001, 2020-09-08T16:09:35.823 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0011230039999999999, 2020-09-08T16:09:35.823 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:35.823 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:09:35.824 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:35.824 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.26587", 2020-09-08T16:09:35.824 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:09:35.824 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 12644 2020-09-08T16:09:35.824 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.825 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:35.825 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.825 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:35.825 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.818259+0000", 2020-09-08T16:09:35.825 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:35.826 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.826 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.826 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:35.826 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.818259+0000", 2020-09-08T16:09:35.826 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999958 2020-09-08T16:09:35.827 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.827 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.827 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:35.827 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.818217+0000", 2020-09-08T16:09:35.827 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00055241399999999999 2020-09-08T16:09:35.828 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.828 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.828 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:35.828 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.818770+0000", 2020-09-08T16:09:35.828 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.8618e-05 2020-09-08T16:09:35.829 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.829 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.829 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:35.829 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.818788+0000", 2020-09-08T16:09:35.829 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.6550000000000004e-06 2020-09-08T16:09:35.830 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.830 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.830 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:35.830 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.818796+0000", 2020-09-08T16:09:35.831 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4964999999999997e-05 2020-09-08T16:09:35.832 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.832 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.832 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:35.832 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.818881+0000", 2020-09-08T16:09:35.833 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016418699999999999 2020-09-08T16:09:35.833 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.833 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.833 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:35.833 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.819045+0000", 2020-09-08T16:09:35.834 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017448400000000001 2020-09-08T16:09:35.834 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.834 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.834 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T16:09:35.834 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.819220+0000", 2020-09-08T16:09:35.834 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00096037200000000003 2020-09-08T16:09:35.835 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:35.835 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:35.835 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:35.835 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.835 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.836 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.26587.0:12631 9.e 9:7194a80f:::benchmark_data_smithi166_56754_object12630:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e708)", 2020-09-08T16:09:35.836 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:09:35.812745+0000", 2020-09-08T16:09:35.836 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.0066012930000000003, 2020-09-08T16:09:35.836 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0067469119999999999, 2020-09-08T16:09:35.836 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:35.837 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:09:35.837 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:35.837 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.26587", 2020-09-08T16:09:35.837 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:09:35.837 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 12631 2020-09-08T16:09:35.838 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.838 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:35.838 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.838 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:35.838 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.812745+0000", 2020-09-08T16:09:35.839 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:35.839 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.839 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.839 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:35.839 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.812745+0000", 2020-09-08T16:09:35.840 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999619 2020-09-08T16:09:35.840 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.840 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.840 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:35.840 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.812707+0000", 2020-09-08T16:09:35.840 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000147863 2020-09-08T16:09:35.841 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.841 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.841 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:35.841 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.812855+0000", 2020-09-08T16:09:35.842 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.5831000000000001e-05 2020-09-08T16:09:35.842 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.842 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.843 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:35.843 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.812881+0000", 2020-09-08T16:09:35.843 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.7390000000000002e-06 2020-09-08T16:09:35.843 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.843 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.844 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:35.844 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.812888+0000", 2020-09-08T16:09:35.844 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00069934999999999995 2020-09-08T16:09:35.844 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.844 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.845 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:35.845 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.813587+0000", 2020-09-08T16:09:35.845 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000184656 2020-09-08T16:09:35.845 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.845 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.845 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:35.846 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.813772+0000", 2020-09-08T16:09:35.846 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00022877800000000001 2020-09-08T16:09:35.846 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.846 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.846 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T16:09:35.847 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.814000+0000", 2020-09-08T16:09:35.847 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0037900490000000002 2020-09-08T16:09:35.847 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.847 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.847 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:09:35.848 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.817791+0000", 2020-09-08T16:09:35.848 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0050452769999999999 2020-09-08T16:09:35.848 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:35.848 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:35.848 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:35.849 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.849 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.849 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.26587.0:12635 9.e 9:7cd28bc2:::benchmark_data_smithi166_56754_object12634:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e708)", 2020-09-08T16:09:35.849 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:09:35.814222+0000", 2020-09-08T16:09:35.849 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.005124661, 2020-09-08T16:09:35.850 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0053535370000000002, 2020-09-08T16:09:35.850 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:35.850 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:09:35.850 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:35.850 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.26587", 2020-09-08T16:09:35.851 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:09:35.851 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 12635 2020-09-08T16:09:35.851 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.851 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:35.852 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.852 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:35.853 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.814222+0000", 2020-09-08T16:09:35.853 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:35.853 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.853 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.853 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:35.854 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.814222+0000", 2020-09-08T16:09:35.854 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.999958 2020-09-08T16:09:35.854 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.854 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.854 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:35.855 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.814180+0000", 2020-09-08T16:09:35.855 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016298800000000001 2020-09-08T16:09:35.855 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.855 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.855 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:35.856 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.814343+0000", 2020-09-08T16:09:35.856 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.4752999999999999e-05 2020-09-08T16:09:35.856 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.856 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.856 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:35.859 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.814358+0000", 2020-09-08T16:09:35.859 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0267e-05 2020-09-08T16:09:35.860 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.864 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.864 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:35.864 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.814368+0000", 2020-09-08T16:09:35.864 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0035219769999999999 2020-09-08T16:09:35.864 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.865 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.865 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:35.865 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.817890+0000", 2020-09-08T16:09:35.865 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017518899999999999 2020-09-08T16:09:35.865 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.866 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.866 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:35.866 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.818065+0000", 2020-09-08T16:09:35.866 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017751799999999999 2020-09-08T16:09:35.866 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.867 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.867 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 10", 2020-09-08T16:09:35.867 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.818243+0000", 2020-09-08T16:09:35.867 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0040207630000000001 2020-09-08T16:09:35.868 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:35.868 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:35.868 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:35.868 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.868 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.868 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "osd_op(client.26587.0:12640 9.d 9:b78b2940:::benchmark_data_smithi166_56754_object12639:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected e708)", 2020-09-08T16:09:35.869 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:09:35.816745+0000", 2020-09-08T16:09:35.869 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 0.002601857, 2020-09-08T16:09:35.869 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.0028877640000000001, 2020-09-08T16:09:35.869 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:09:35.870 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:09:35.870 INFO:teuthology.orchestra.run.smithi055.stdout: "client_info": { 2020-09-08T16:09:35.870 INFO:teuthology.orchestra.run.smithi055.stdout: "client": "client.26587", 2020-09-08T16:09:35.870 INFO:teuthology.orchestra.run.smithi055.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:09:35.870 INFO:teuthology.orchestra.run.smithi055.stdout: "tid": 12640 2020-09-08T16:09:35.871 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.871 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:09:35.871 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.872 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:09:35.872 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.816745+0000", 2020-09-08T16:09:35.872 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:09:35.872 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.872 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.873 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:09:35.874 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.816745+0000", 2020-09-08T16:09:35.874 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4294967295.9999614 2020-09-08T16:09:35.874 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.874 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.874 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:09:35.875 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.816706+0000", 2020-09-08T16:09:35.875 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00013015800000000001 2020-09-08T16:09:35.875 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.875 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.875 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:09:35.876 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.816836+0000", 2020-09-08T16:09:35.876 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5167000000000001e-05 2020-09-08T16:09:35.876 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.880 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.881 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:09:35.881 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.816851+0000", 2020-09-08T16:09:35.881 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 6.3439999999999997e-06 2020-09-08T16:09:35.881 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.881 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.882 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:09:35.882 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.816858+0000", 2020-09-08T16:09:35.882 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00025374300000000001 2020-09-08T16:09:35.882 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.883 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.883 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:09:35.883 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.817111+0000", 2020-09-08T16:09:35.883 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00016412600000000001 2020-09-08T16:09:35.883 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.884 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.885 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:09:35.885 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.817276+0000", 2020-09-08T16:09:35.886 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00020013799999999999 2020-09-08T16:09:35.886 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:09:35.887 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:09:35.887 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "waiting for subops from 1", 2020-09-08T16:09:35.887 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:09:35.817476+0000", 2020-09-08T16:09:35.887 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00073093400000000003 2020-09-08T16:09:35.888 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:35.888 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:09:35.888 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:35.888 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:09:35.889 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2020-09-08T16:09:35.889 INFO:teuthology.orchestra.run.smithi055.stdout: "num_ops": 4 2020-09-08T16:09:35.889 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:09:35.890 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:09:35 smithi166 bash[14254]: debug 2020-09-08T16:09:35.609+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:36.622 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:36 smithi166 bash[9446]: cluster 2020-09-08T16:09:35.349164+0000 mon.a (mon.0) 2225 : cluster [DBG] osdmap e708: 12 total, 12 up, 12 in 2020-09-08T16:09:36.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:36 smithi055 bash[9296]: cluster 2020-09-08T16:09:35.349164+0000 mon.a (mon.0) 2225 : cluster [DBG] osdmap e708: 12 total, 12 up, 12 in 2020-09-08T16:09:36.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:36 smithi104 bash[9203]: cluster 2020-09-08T16:09:35.349164+0000 mon.a (mon.0) 2225 : cluster [DBG] osdmap e708: 12 total, 12 up, 12 in 2020-09-08T16:09:36.704 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:09:36.785 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 5 16 15923 15907 198.816 210.562 0.00240864 0.00500158 2020-09-08T16:09:36.808 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:36.998 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:09:36 smithi104 bash[19922]: debug 2020-09-08T16:09:36.705+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:36.999 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:37.110 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:37.352 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:37.354 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:09:37 smithi104 bash[17588]: debug 2020-09-08T16:09:36.997+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:37.355 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:37 smithi104 bash[9203]: cluster 2020-09-08T16:09:36.110378+0000 mgr.x (mgr.24118) 1268 : cluster [DBG] pgmap v1892: 17 pgs: 17 active+clean; 214 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s wr, 505 op/s 2020-09-08T16:09:37.414 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:37 smithi055 bash[9296]: cluster 2020-09-08T16:09:36.110378+0000 mgr.x (mgr.24118) 1268 : cluster [DBG] pgmap v1892: 17 pgs: 17 active+clean; 214 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s wr, 505 op/s 2020-09-08T16:09:37.415 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:09:37 smithi055 bash[14301]: debug 2020-09-08T16:09:37.351+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:37.416 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2020-09-08T16:09:37.457 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:37.589 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:37.604 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:09:37.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:37 smithi166 bash[9446]: cluster 2020-09-08T16:09:36.110378+0000 mgr.x (mgr.24118) 1268 : cluster [DBG] pgmap v1892: 17 pgs: 17 active+clean; 214 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s wr, 505 op/s 2020-09-08T16:09:37.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:09:37 smithi055 bash[16646]: debug 2020-09-08T16:09:37.591+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:37.696 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:09:37.787 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 6 16 18783 18767 195.469 178.75 0.00597842 0.0051128 2020-09-08T16:09:38.370 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T16:09:38.526 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:09:38.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:38 smithi166 bash[9446]: audit 2020-09-08T16:09:37.853906+0000 mon.c (mon.2) 89 : audit [INF] from='client.? v1:172.21.15.104:0/3834988313' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:09:38.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:38 smithi166 bash[9446]: audit 2020-09-08T16:09:37.856326+0000 mon.a (mon.0) 2226 : audit [INF] from='client.36572 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:09:38.629 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:09:38.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:38 smithi055 bash[9296]: audit 2020-09-08T16:09:37.853906+0000 mon.c (mon.2) 89 : audit [INF] from='client.? v1:172.21.15.104:0/3834988313' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:09:38.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:38 smithi055 bash[9296]: audit 2020-09-08T16:09:37.856326+0000 mon.a (mon.0) 2226 : audit [INF] from='client.36572 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:09:38.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:38 smithi104 bash[9203]: audit 2020-09-08T16:09:37.853906+0000 mon.c (mon.2) 89 : audit [INF] from='client.? v1:172.21.15.104:0/3834988313' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:09:38.687 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:38 smithi104 bash[9203]: audit 2020-09-08T16:09:37.856326+0000 mon.a (mon.0) 2226 : audit [INF] from='client.36572 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:09:38.687 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:09:38 smithi104 bash[19922]: debug 2020-09-08T16:09:38.525+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:38.785 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 7 16 21762 21746 194.14 186.188 0.00264669 0.00507452 2020-09-08T16:09:39.016 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:09:39 smithi055 bash[11956]: debug 2020-09-08T16:09:39.015+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:39.017 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:09:39.119 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:39.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:39 smithi166 bash[9446]: cluster 2020-09-08T16:09:38.110861+0000 mgr.x (mgr.24118) 1269 : cluster [DBG] pgmap v1893: 17 pgs: 17 active+clean; 1017 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 127 MiB/s wr, 2.03k op/s 2020-09-08T16:09:39.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:39 smithi166 bash[9446]: cluster 2020-09-08T16:09:38.361946+0000 mon.a (mon.0) 2227 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:09:39.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:39 smithi166 bash[9446]: cluster 2020-09-08T16:09:38.361983+0000 mon.a (mon.0) 2228 : cluster [INF] Cluster is now healthy 2020-09-08T16:09:39.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:39 smithi166 bash[9446]: audit 2020-09-08T16:09:38.370791+0000 mon.a (mon.0) 2229 : audit [INF] from='client.36572 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:09:39.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:39 smithi166 bash[9446]: cluster 2020-09-08T16:09:38.370830+0000 mon.a (mon.0) 2230 : cluster [DBG] osdmap e709: 12 total, 12 up, 12 in 2020-09-08T16:09:39.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:39 smithi055 bash[9296]: cluster 2020-09-08T16:09:38.110861+0000 mgr.x (mgr.24118) 1269 : cluster [DBG] pgmap v1893: 17 pgs: 17 active+clean; 1017 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 127 MiB/s wr, 2.03k op/s 2020-09-08T16:09:39.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:39 smithi055 bash[9296]: cluster 2020-09-08T16:09:38.361946+0000 mon.a (mon.0) 2227 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:09:39.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:39 smithi055 bash[9296]: cluster 2020-09-08T16:09:38.361983+0000 mon.a (mon.0) 2228 : cluster [INF] Cluster is now healthy 2020-09-08T16:09:39.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:39 smithi055 bash[9296]: audit 2020-09-08T16:09:38.370791+0000 mon.a (mon.0) 2229 : audit [INF] from='client.36572 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:09:39.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:39 smithi055 bash[9296]: cluster 2020-09-08T16:09:38.370830+0000 mon.a (mon.0) 2230 : cluster [DBG] osdmap e709: 12 total, 12 up, 12 in 2020-09-08T16:09:39.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:39 smithi104 bash[9203]: cluster 2020-09-08T16:09:38.110861+0000 mgr.x (mgr.24118) 1269 : cluster [DBG] pgmap v1893: 17 pgs: 17 active+clean; 1017 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 127 MiB/s wr, 2.03k op/s 2020-09-08T16:09:39.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:39 smithi104 bash[9203]: cluster 2020-09-08T16:09:38.361946+0000 mon.a (mon.0) 2227 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:09:39.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:39 smithi104 bash[9203]: cluster 2020-09-08T16:09:38.361983+0000 mon.a (mon.0) 2228 : cluster [INF] Cluster is now healthy 2020-09-08T16:09:39.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:39 smithi104 bash[9203]: audit 2020-09-08T16:09:38.370791+0000 mon.a (mon.0) 2229 : audit [INF] from='client.36572 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:09:39.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:39 smithi104 bash[9203]: cluster 2020-09-08T16:09:38.370830+0000 mon.a (mon.0) 2230 : cluster [DBG] osdmap e709: 12 total, 12 up, 12 in 2020-09-08T16:09:39.785 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 8 16 23940 23924 186.886 136.125 0.0040769 0.00534788 2020-09-08T16:09:39.914 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:40.016 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:40.156 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:09:39 smithi055 bash[14301]: debug 2020-09-08T16:09:39.907+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:40.423 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:40.527 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:40.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:09:40 smithi104 bash[17588]: debug 2020-09-08T16:09:40.425+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:40.754 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:40.785 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 9 16 26912 26896 186.758 185.75 0.00801318 0.00535227 2020-09-08T16:09:40.862 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:09:40.975 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:09:41.080 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:41.088 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:09:40 smithi055 bash[16646]: debug 2020-09-08T16:09:40.747+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:41.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:09:40 smithi104 bash[22305]: debug 2020-09-08T16:09:40.973+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:41.248 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:09:41.286 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:09:41.312 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:09:41.349 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:09:41.366 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:41 smithi055 bash[9296]: cluster 2020-09-08T16:09:39.475946+0000 osd.8 (osd.8) 101 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:41.367 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:41 smithi055 bash[9296]: cluster 2020-09-08T16:09:39.476043+0000 osd.8 (osd.8) 102 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:41.367 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:41 smithi055 bash[9296]: cluster 2020-09-08T16:09:40.111340+0000 mgr.x (mgr.24118) 1270 : cluster [DBG] pgmap v1895: 17 pgs: 17 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 152 MiB/s wr, 2.43k op/s 2020-09-08T16:09:41.367 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:41 smithi055 bash[9296]: cluster 2020-09-08T16:09:40.369364+0000 mon.a (mon.0) 2231 : cluster [DBG] osdmap e710: 12 total, 12 up, 12 in 2020-09-08T16:09:41.368 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:41 smithi055 bash[9296]: audit 2020-09-08T16:09:41.220819+0000 mon.a (mon.0) 2232 : audit [DBG] from='client.? v1:172.21.15.104:0/3626688458' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:09:41.383 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:09:41.425 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:09:41.461 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:09:41.492 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:09:41.529 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:09:41.563 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:09:41.598 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:09:41.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:41 smithi166 bash[9446]: cluster 2020-09-08T16:09:39.475946+0000 osd.8 (osd.8) 101 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:41.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:41 smithi166 bash[9446]: cluster 2020-09-08T16:09:39.476043+0000 osd.8 (osd.8) 102 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:41.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:41 smithi166 bash[9446]: cluster 2020-09-08T16:09:40.111340+0000 mgr.x (mgr.24118) 1270 : cluster [DBG] pgmap v1895: 17 pgs: 17 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 152 MiB/s wr, 2.43k op/s 2020-09-08T16:09:41.620 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:41 smithi166 bash[9446]: cluster 2020-09-08T16:09:40.369364+0000 mon.a (mon.0) 2231 : cluster [DBG] osdmap e710: 12 total, 12 up, 12 in 2020-09-08T16:09:41.621 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:41 smithi166 bash[9446]: audit 2020-09-08T16:09:41.220819+0000 mon.a (mon.0) 2232 : audit [DBG] from='client.? v1:172.21.15.104:0/3626688458' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:09:41.628 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:09:41.631 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:09:41.632 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:09:41.632 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:09:41.633 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:09:41.633 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:09:41.633 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:09:41.634 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:09:41.634 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:09:41.634 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:09:41.635 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:09:41.635 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:09:41.635 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:09:41.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:41 smithi104 bash[9203]: cluster 2020-09-08T16:09:39.475946+0000 osd.8 (osd.8) 101 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:41.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:41 smithi104 bash[9203]: cluster 2020-09-08T16:09:39.476043+0000 osd.8 (osd.8) 102 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:41.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:41 smithi104 bash[9203]: cluster 2020-09-08T16:09:40.111340+0000 mgr.x (mgr.24118) 1270 : cluster [DBG] pgmap v1895: 17 pgs: 17 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 152 MiB/s wr, 2.43k op/s 2020-09-08T16:09:41.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:41 smithi104 bash[9203]: cluster 2020-09-08T16:09:40.369364+0000 mon.a (mon.0) 2231 : cluster [DBG] osdmap e710: 12 total, 12 up, 12 in 2020-09-08T16:09:41.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:41 smithi104 bash[9203]: audit 2020-09-08T16:09:41.220819+0000 mon.a (mon.0) 2232 : audit [DBG] from='client.? v1:172.21.15.104:0/3626688458' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:09:41.785 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 10 15 29732 29717 185.712 176.312 0.00718902 0.00538255 2020-09-08T16:09:42.367 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:42.471 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:09:42.658 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:09:42 smithi055 bash[14301]: debug 2020-09-08T16:09:42.367+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:42.678 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:09:42.781 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T16:09:42.784 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:42.785 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 11 16 31934 31918 181.334 137.562 0.0026646 0.00538835 2020-09-08T16:09:42.934 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:42.950 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:09:43.038 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:43.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:09:42 smithi166 bash[11928]: debug 2020-09-08T16:09:42.933+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:43.148 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:43.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:09:42 smithi055 bash[11956]: debug 2020-09-08T16:09:42.679+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:43.254 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:09:43.384 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:09:43.493 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:09:43.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:09:43 smithi166 bash[11928]: debug 2020-09-08T16:09:43.145+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:43.618 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:09:43 smithi166 bash[14254]: debug 2020-09-08T16:09:43.381+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:43.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:43 smithi166 bash[9446]: cluster 2020-09-08T16:09:42.111879+0000 mgr.x (mgr.24118) 1271 : cluster [DBG] pgmap v1897: 17 pgs: 17 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 145 MiB/s wr, 2.32k op/s 2020-09-08T16:09:43.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:43 smithi055 bash[9296]: cluster 2020-09-08T16:09:42.111879+0000 mgr.x (mgr.24118) 1271 : cluster [DBG] pgmap v1897: 17 pgs: 17 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 145 MiB/s wr, 2.32k op/s 2020-09-08T16:09:43.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:43 smithi104 bash[9203]: cluster 2020-09-08T16:09:42.111879+0000 mgr.x (mgr.24118) 1271 : cluster [DBG] pgmap v1897: 17 pgs: 17 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 145 MiB/s wr, 2.32k op/s 2020-09-08T16:09:43.786 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 12 16 34414 34398 179.138 155 0.00407288 0.00558004 2020-09-08T16:09:44.229 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:09:44.332 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:44.675 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:09:44 smithi104 bash[19922]: debug 2020-09-08T16:09:44.229+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:44.786 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 13 16 37115 37099 178.343 168.812 0.00422825 0.00560467 2020-09-08T16:09:45.162 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:45.266 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:09:45.363 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T16:09:45.363 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:09:45.368 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:45 smithi055 bash[9296]: cluster 2020-09-08T16:09:44.112314+0000 mgr.x (mgr.24118) 1272 : cluster [DBG] pgmap v1898: 17 pgs: 17 active+clean; 1.9 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 214 MiB/s wr, 3.43k op/s 2020-09-08T16:09:45.400 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:09:45.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:09:45 smithi104 bash[15273]: debug 2020-09-08T16:09:45.157+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:45.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:45 smithi104 bash[9203]: cluster 2020-09-08T16:09:44.112314+0000 mgr.x (mgr.24118) 1272 : cluster [DBG] pgmap v1898: 17 pgs: 17 active+clean; 1.9 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 214 MiB/s wr, 3.43k op/s 2020-09-08T16:09:45.510 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:09:45.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:45 smithi166 bash[9446]: cluster 2020-09-08T16:09:44.112314+0000 mgr.x (mgr.24118) 1272 : cluster [DBG] pgmap v1898: 17 pgs: 17 active+clean; 1.9 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 214 MiB/s wr, 3.43k op/s 2020-09-08T16:09:45.617 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:09:45 smithi166 bash[18960]: debug 2020-09-08T16:09:45.397+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:45.638 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:09:45.743 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:45.786 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 14 16 38737 38721 172.845 101.375 0.0039889 0.00578279 2020-09-08T16:09:45.947 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:46.054 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:46.065 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:09:45 smithi104 bash[17588]: debug 2020-09-08T16:09:45.949+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:46.117 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:09:45 smithi166 bash[18960]: debug 2020-09-08T16:09:45.637+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:46.692 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2020-09-08T16:09:46.786 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 15 16 40728 40712 169.617 124.438 0.00341643 0.0058924 2020-09-08T16:09:46.883 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:09:47.087 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:47.166 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:09:47.194 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:47.340 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:09:47.374 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:09:47 smithi104 bash[15273]: debug 2020-09-08T16:09:47.089+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:47.420 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:47.523 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:09:47.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:47 smithi055 bash[9296]: cluster 2020-09-08T16:09:46.112856+0000 mgr.x (mgr.24118) 1273 : cluster [DBG] pgmap v1899: 17 pgs: 17 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 127 MiB/s wr, 2.03k op/s 2020-09-08T16:09:47.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:47 smithi104 bash[9203]: cluster 2020-09-08T16:09:46.112856+0000 mgr.x (mgr.24118) 1273 : cluster [DBG] pgmap v1899: 17 pgs: 17 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 127 MiB/s wr, 2.03k op/s 2020-09-08T16:09:47.679 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:09:47 smithi104 bash[15273]: debug 2020-09-08T16:09:47.421+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:47.786 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 16 15 42808 42793 167.144 130.062 0.00405286 0.00598096 2020-09-08T16:09:47.815 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:09:47.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:47 smithi166 bash[9446]: cluster 2020-09-08T16:09:46.112856+0000 mgr.x (mgr.24118) 1273 : cluster [DBG] pgmap v1899: 17 pgs: 17 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 127 MiB/s wr, 2.03k op/s 2020-09-08T16:09:47.918 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:09:48.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:09:47 smithi055 bash[11956]: debug 2020-09-08T16:09:47.807+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:48.626 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:09:48.731 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:09:48.790 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 17 16 45001 44985 165.37 137 0.00341703 0.0060447 2020-09-08T16:09:48.909 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:09:48 smithi055 bash[11956]: debug 2020-09-08T16:09:48.623+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:48.935 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:09:49.037 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:09:49.174 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:09:49.279 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:09:49.386 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:09:48 smithi055 bash[19003]: debug 2020-09-08T16:09:48.931+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:49.387 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:09:49 smithi055 bash[19003]: debug 2020-09-08T16:09:49.175+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:49.408 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:09:49.514 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:49.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:09:49 smithi166 bash[14254]: debug 2020-09-08T16:09:49.409+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:49.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:49 smithi166 bash[9446]: cluster 2020-09-08T16:09:48.105682+0000 osd.1 (osd.1) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:49.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:49 smithi166 bash[9446]: cluster 2020-09-08T16:09:48.105774+0000 osd.1 (osd.1) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:49.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:49 smithi166 bash[9446]: cluster 2020-09-08T16:09:48.113387+0000 mgr.x (mgr.24118) 1274 : cluster [DBG] pgmap v1900: 17 pgs: 17 active+clean; 2.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 160 MiB/s wr, 2.57k op/s 2020-09-08T16:09:49.645 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:49 smithi166 bash[9446]: cluster 2020-09-08T16:09:48.387744+0000 mon.a (mon.0) 2233 : cluster [DBG] osdmap e711: 12 total, 12 up, 12 in 2020-09-08T16:09:49.654 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:49 smithi104 bash[9203]: cluster 2020-09-08T16:09:48.105682+0000 osd.1 (osd.1) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:49.654 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:49 smithi104 bash[9203]: cluster 2020-09-08T16:09:48.105774+0000 osd.1 (osd.1) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:49.654 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:49 smithi104 bash[9203]: cluster 2020-09-08T16:09:48.113387+0000 mgr.x (mgr.24118) 1274 : cluster [DBG] pgmap v1900: 17 pgs: 17 active+clean; 2.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 160 MiB/s wr, 2.57k op/s 2020-09-08T16:09:49.655 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:49 smithi104 bash[9203]: cluster 2020-09-08T16:09:48.387744+0000 mon.a (mon.0) 2233 : cluster [DBG] osdmap e711: 12 total, 12 up, 12 in 2020-09-08T16:09:49.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:49 smithi055 bash[9296]: cluster 2020-09-08T16:09:48.105682+0000 osd.1 (osd.1) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:49.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:49 smithi055 bash[9296]: cluster 2020-09-08T16:09:48.105774+0000 osd.1 (osd.1) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:49.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:49 smithi055 bash[9296]: cluster 2020-09-08T16:09:48.113387+0000 mgr.x (mgr.24118) 1274 : cluster [DBG] pgmap v1900: 17 pgs: 17 active+clean; 2.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 160 MiB/s wr, 2.57k op/s 2020-09-08T16:09:49.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:49 smithi055 bash[9296]: cluster 2020-09-08T16:09:48.387744+0000 mon.a (mon.0) 2233 : cluster [DBG] osdmap e711: 12 total, 12 up, 12 in 2020-09-08T16:09:49.786 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 18 15 47207 47192 163.845 137.938 0.00496165 0.00610098 2020-09-08T16:09:50.365 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:50.467 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:50.659 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:09:50 smithi055 bash[16646]: debug 2020-09-08T16:09:50.355+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:50.786 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 19 16 49292 49276 162.077 130.25 0.01027 0.00616773 2020-09-08T16:09:50.843 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:09:50 smithi166 bash[11928]: debug 2020-09-08T16:09:50.837+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:50.843 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:50.945 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:51.056 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T16:09:51.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:51 smithi166 bash[9446]: cluster 2020-09-08T16:09:50.113881+0000 mgr.x (mgr.24118) 1275 : cluster [DBG] pgmap v1902: 17 pgs: 17 active+clean; 2.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 155 MiB/s wr, 2.48k op/s 2020-09-08T16:09:51.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:51 smithi166 bash[9446]: audit 2020-09-08T16:09:50.164157+0000 mon.c (mon.2) 90 : audit [INF] from='client.? v1:172.21.15.104:0/355313746' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:09:51.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:51 smithi166 bash[9446]: audit 2020-09-08T16:09:50.166641+0000 mon.a (mon.0) 2234 : audit [INF] from='client.36581 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:09:51.312 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:51 smithi104 bash[9203]: cluster 2020-09-08T16:09:50.113881+0000 mgr.x (mgr.24118) 1275 : cluster [DBG] pgmap v1902: 17 pgs: 17 active+clean; 2.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 155 MiB/s wr, 2.48k op/s 2020-09-08T16:09:51.312 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:51 smithi104 bash[9203]: audit 2020-09-08T16:09:50.164157+0000 mon.c (mon.2) 90 : audit [INF] from='client.? v1:172.21.15.104:0/355313746' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:09:51.313 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:51 smithi104 bash[9203]: audit 2020-09-08T16:09:50.166641+0000 mon.a (mon.0) 2234 : audit [INF] from='client.36581 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:09:51.362 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:51.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:51 smithi055 bash[9296]: cluster 2020-09-08T16:09:50.113881+0000 mgr.x (mgr.24118) 1275 : cluster [DBG] pgmap v1902: 17 pgs: 17 active+clean; 2.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 155 MiB/s wr, 2.48k op/s 2020-09-08T16:09:51.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:51 smithi055 bash[9296]: audit 2020-09-08T16:09:50.164157+0000 mon.c (mon.2) 90 : audit [INF] from='client.? v1:172.21.15.104:0/355313746' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:09:51.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:51 smithi055 bash[9296]: audit 2020-09-08T16:09:50.166641+0000 mon.a (mon.0) 2234 : audit [INF] from='client.36581 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:09:51.466 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:51.676 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:09:51 smithi104 bash[15273]: debug 2020-09-08T16:09:51.361+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:51.786 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 16:09:51.791405 min lat: 0.00165771 max lat: 0.324694 avg lat: 0.00623626 2020-09-08T16:09:51.787 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T16:09:51.787 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 20 16 51315 51299 160.294 126.438 0.00807409 0.00623626 2020-09-08T16:09:52.108 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:52.214 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:52.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:52 smithi166 bash[9446]: cluster 2020-09-08T16:09:51.047281+0000 mon.a (mon.0) 2235 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:09:52.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:52 smithi166 bash[9446]: audit 2020-09-08T16:09:51.059175+0000 mon.a (mon.0) 2236 : audit [INF] from='client.36581 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:09:52.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:52 smithi166 bash[9446]: cluster 2020-09-08T16:09:51.059244+0000 mon.a (mon.0) 2237 : cluster [DBG] osdmap e712: 12 total, 12 up, 12 in 2020-09-08T16:09:52.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:52 smithi055 bash[9296]: cluster 2020-09-08T16:09:51.047281+0000 mon.a (mon.0) 2235 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:09:52.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:52 smithi055 bash[9296]: audit 2020-09-08T16:09:51.059175+0000 mon.a (mon.0) 2236 : audit [INF] from='client.36581 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:09:52.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:52 smithi055 bash[9296]: cluster 2020-09-08T16:09:51.059244+0000 mon.a (mon.0) 2237 : cluster [DBG] osdmap e712: 12 total, 12 up, 12 in 2020-09-08T16:09:52.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:09:52 smithi055 bash[16646]: debug 2020-09-08T16:09:52.107+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:52.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:52 smithi104 bash[9203]: cluster 2020-09-08T16:09:51.047281+0000 mon.a (mon.0) 2235 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:09:52.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:52 smithi104 bash[9203]: audit 2020-09-08T16:09:51.059175+0000 mon.a (mon.0) 2236 : audit [INF] from='client.36581 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:09:52.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:52 smithi104 bash[9203]: cluster 2020-09-08T16:09:51.059244+0000 mon.a (mon.0) 2237 : cluster [DBG] osdmap e712: 12 total, 12 up, 12 in 2020-09-08T16:09:52.787 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 21 16 53502 53486 159.169 136.688 0.00624333 0.00628063 2020-09-08T16:09:52.946 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:53.050 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:53.298 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:53 smithi055 bash[9296]: cluster 2020-09-08T16:09:52.114347+0000 osd.6 (osd.6) 117 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:53.298 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:53 smithi055 bash[9296]: cluster 2020-09-08T16:09:52.114389+0000 mgr.x (mgr.24118) 1276 : cluster [DBG] pgmap v1904: 17 pgs: 17 active+clean; 2.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 97 MiB/s wr, 1.55k op/s 2020-09-08T16:09:53.299 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:53 smithi055 bash[9296]: cluster 2020-09-08T16:09:52.114440+0000 osd.6 (osd.6) 118 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:53.299 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:09:52 smithi055 bash[16646]: debug 2020-09-08T16:09:52.943+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:53.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:53 smithi166 bash[9446]: cluster 2020-09-08T16:09:52.114347+0000 osd.6 (osd.6) 117 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:53.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:53 smithi166 bash[9446]: cluster 2020-09-08T16:09:52.114389+0000 mgr.x (mgr.24118) 1276 : cluster [DBG] pgmap v1904: 17 pgs: 17 active+clean; 2.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 97 MiB/s wr, 1.55k op/s 2020-09-08T16:09:53.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:53 smithi166 bash[9446]: cluster 2020-09-08T16:09:52.114440+0000 osd.6 (osd.6) 118 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:53.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:53 smithi104 bash[9203]: cluster 2020-09-08T16:09:52.114347+0000 osd.6 (osd.6) 117 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:53.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:53 smithi104 bash[9203]: cluster 2020-09-08T16:09:52.114389+0000 mgr.x (mgr.24118) 1276 : cluster [DBG] pgmap v1904: 17 pgs: 17 active+clean; 2.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 97 MiB/s wr, 1.55k op/s 2020-09-08T16:09:53.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:53 smithi104 bash[9203]: cluster 2020-09-08T16:09:52.114440+0000 osd.6 (osd.6) 118 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:53.524 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:53.631 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:53.718 INFO:teuthology.orchestra.run.smithi055.stdout:ERROR: (22) Invalid argument 2020-09-08T16:09:53.719 INFO:teuthology.orchestra.run.smithi055.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:09:53.755 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:53.787 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 22 16 55293 55277 157.022 111.938 0.00524133 0.00636628 2020-09-08T16:09:53.860 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:09:53.864 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:09:53 smithi104 bash[17588]: debug 2020-09-08T16:09:53.525+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:53.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:09:53 smithi055 bash[14301]: debug 2020-09-08T16:09:53.755+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:54.008 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:09:54.111 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:09:54.127 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:54 smithi166 bash[9446]: cluster 2020-09-08T16:09:53.054289+0000 mon.a (mon.0) 2238 : cluster [DBG] osdmap e713: 12 total, 12 up, 12 in 2020-09-08T16:09:54.128 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:09:54 smithi166 bash[16593]: debug 2020-09-08T16:09:54.004+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:54.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:54 smithi104 bash[9203]: cluster 2020-09-08T16:09:53.054289+0000 mon.a (mon.0) 2238 : cluster [DBG] osdmap e713: 12 total, 12 up, 12 in 2020-09-08T16:09:54.234 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:09:54.306 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:09:54.333 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:09:54.340 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:54.366 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:09:54.404 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:09:54.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:54 smithi055 bash[9296]: cluster 2020-09-08T16:09:53.054289+0000 mon.a (mon.0) 2238 : cluster [DBG] osdmap e713: 12 total, 12 up, 12 in 2020-09-08T16:09:54.437 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:09:54.454 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:54.470 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:09:54.506 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:09:54.540 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:09:54.558 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:54.576 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:09:54.607 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:09:54.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:09:54 smithi166 bash[16593]: debug 2020-09-08T16:09:54.232+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:54.649 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:09:54.677 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:09:54.677 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:09:54 smithi104 bash[15273]: debug 2020-09-08T16:09:54.453+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:54.678 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:09:54.679 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:09:54.679 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:09:54.679 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:09:54.679 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:09:54.680 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:09:54.680 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:09:54.680 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:09:54.680 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:09:54.680 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:09:54.681 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:09:54.681 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:09:54.787 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 23 16 57277 57261 155.586 124 0.00510718 0.00642498 2020-09-08T16:09:54.952 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T16:09:55.106 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:09:55.137 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:09:55.240 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:55.283 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2020-09-08T16:09:55.410 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:09:55 smithi055 bash[16646]: debug 2020-09-08T16:09:55.135+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:55.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:55 smithi055 bash[9296]: cluster 2020-09-08T16:09:54.114854+0000 mgr.x (mgr.24118) 1277 : cluster [DBG] pgmap v1906: 17 pgs: 17 active+clean; 3.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s wr, 1.88k op/s 2020-09-08T16:09:55.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:55 smithi055 bash[9296]: audit 2020-09-08T16:09:54.277296+0000 mon.a (mon.0) 2239 : audit [DBG] from='client.? v1:172.21.15.104:0/1475036779' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:09:55.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:55 smithi104 bash[9203]: cluster 2020-09-08T16:09:54.114854+0000 mgr.x (mgr.24118) 1277 : cluster [DBG] pgmap v1906: 17 pgs: 17 active+clean; 3.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s wr, 1.88k op/s 2020-09-08T16:09:55.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:55 smithi104 bash[9203]: audit 2020-09-08T16:09:54.277296+0000 mon.a (mon.0) 2239 : audit [DBG] from='client.? v1:172.21.15.104:0/1475036779' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:09:55.787 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 24 16 58831 58815 153.15 97.125 0.00597943 0.00652766 2020-09-08T16:09:55.958 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:09:56.036 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:55 smithi166 bash[9446]: cluster 2020-09-08T16:09:54.114854+0000 mgr.x (mgr.24118) 1277 : cluster [DBG] pgmap v1906: 17 pgs: 17 active+clean; 3.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s wr, 1.88k op/s 2020-09-08T16:09:56.036 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:55 smithi166 bash[9446]: audit 2020-09-08T16:09:54.277296+0000 mon.a (mon.0) 2239 : audit [DBG] from='client.? v1:172.21.15.104:0/1475036779' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:09:56.268 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:09:56.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:56 smithi166 bash[9446]: cluster 2020-09-08T16:09:55.062758+0000 osd.11 (osd.11) 128 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:56.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:56 smithi166 bash[9446]: cluster 2020-09-08T16:09:55.062866+0000 osd.11 (osd.11) 129 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:56.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:56 smithi166 bash[9446]: cluster 2020-09-08T16:09:56.069814+0000 mon.a (mon.0) 2240 : cluster [DBG] osdmap e714: 12 total, 12 up, 12 in 2020-09-08T16:09:56.372 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:56.382 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:56 smithi055 bash[9296]: cluster 2020-09-08T16:09:55.062758+0000 osd.11 (osd.11) 128 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:56.382 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:56 smithi055 bash[9296]: cluster 2020-09-08T16:09:55.062866+0000 osd.11 (osd.11) 129 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:56.382 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:56 smithi055 bash[9296]: cluster 2020-09-08T16:09:56.069814+0000 mon.a (mon.0) 2240 : cluster [DBG] osdmap e714: 12 total, 12 up, 12 in 2020-09-08T16:09:56.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:09:56 smithi104 bash[17588]: debug 2020-09-08T16:09:56.269+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:56.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:56 smithi104 bash[9203]: cluster 2020-09-08T16:09:55.062758+0000 osd.11 (osd.11) 128 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:09:56.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:56 smithi104 bash[9203]: cluster 2020-09-08T16:09:55.062866+0000 osd.11 (osd.11) 129 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:09:56.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:56 smithi104 bash[9203]: cluster 2020-09-08T16:09:56.069814+0000 mon.a (mon.0) 2240 : cluster [DBG] osdmap e714: 12 total, 12 up, 12 in 2020-09-08T16:09:56.501 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:09:56.608 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:56.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:09:56 smithi055 bash[14301]: debug 2020-09-08T16:09:56.503+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:56.790 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 25 16 60744 60728 151.806 119.562 0.0102336 0.00658544 2020-09-08T16:09:57.344 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:09:57 smithi104 bash[15273]: debug 2020-09-08T16:09:57.345+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:09:57.344 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:57 smithi104 bash[9203]: cluster 2020-09-08T16:09:56.115326+0000 mgr.x (mgr.24118) 1278 : cluster [DBG] pgmap v1908: 17 pgs: 17 active+clean; 3.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 110 MiB/s wr, 1.76k op/s 2020-09-08T16:09:57.344 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:09:57.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:57 smithi166 bash[9446]: cluster 2020-09-08T16:09:56.115326+0000 mgr.x (mgr.24118) 1278 : cluster [DBG] pgmap v1908: 17 pgs: 17 active+clean; 3.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 110 MiB/s wr, 1.76k op/s 2020-09-08T16:09:57.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:57 smithi055 bash[9296]: cluster 2020-09-08T16:09:56.115326+0000 mgr.x (mgr.24118) 1278 : cluster [DBG] pgmap v1908: 17 pgs: 17 active+clean; 3.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 110 MiB/s wr, 1.76k op/s 2020-09-08T16:09:57.454 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:57.787 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 26 16 62545 62529 150.296 112.562 0.00352796 0.00663154 2020-09-08T16:09:58.789 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 27 16 64180 64164 148.514 102.188 0.00742696 0.00673149 2020-09-08T16:09:59.182 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:09:59.365 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:09:59.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:09:59 smithi104 bash[9203]: cluster 2020-09-08T16:09:58.115839+0000 mgr.x (mgr.24118) 1279 : cluster [DBG] pgmap v1909: 17 pgs: 17 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 159 MiB/s wr, 2.55k op/s 2020-09-08T16:09:59.621 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:09:59 smithi166 bash[9446]: cluster 2020-09-08T16:09:58.115839+0000 mgr.x (mgr.24118) 1279 : cluster [DBG] pgmap v1909: 17 pgs: 17 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 159 MiB/s wr, 2.55k op/s 2020-09-08T16:09:59.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:09:59 smithi055 bash[9296]: cluster 2020-09-08T16:09:58.115839+0000 mgr.x (mgr.24118) 1279 : cluster [DBG] pgmap v1909: 17 pgs: 17 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 159 MiB/s wr, 2.55k op/s 2020-09-08T16:09:59.776 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:09:59.787 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 28 16 65852 65836 146.941 104.5 0.0227026 0.00680276 2020-09-08T16:09:59.881 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:10:00.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:09:59 smithi166 bash[11928]: debug 2020-09-08T16:09:59.772+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:00.793 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 29 16 67336 67320 145.072 92.75 0.00998595 0.00689095 2020-09-08T16:10:01.358 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:01 smithi104 bash[9203]: cluster 2020-09-08T16:10:00.116245+0000 mgr.x (mgr.24118) 1280 : cluster [DBG] pgmap v1910: 17 pgs: 17 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 153 MiB/s wr, 2.45k op/s 2020-09-08T16:10:01.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:01 smithi166 bash[9446]: cluster 2020-09-08T16:10:00.116245+0000 mgr.x (mgr.24118) 1280 : cluster [DBG] pgmap v1910: 17 pgs: 17 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 153 MiB/s wr, 2.45k op/s 2020-09-08T16:10:01.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:01 smithi055 bash[9296]: cluster 2020-09-08T16:10:00.116245+0000 mgr.x (mgr.24118) 1280 : cluster [DBG] pgmap v1910: 17 pgs: 17 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 153 MiB/s wr, 2.45k op/s 2020-09-08T16:10:01.787 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 30 16 69081 69065 143.872 109.062 0.0111138 0.00694833 2020-09-08T16:10:02.070 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T16:10:02.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:02 smithi166 bash[9446]: cluster 2020-09-08T16:10:01.176267+0000 osd.4 (osd.4) 77 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:02.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:02 smithi166 bash[9446]: cluster 2020-09-08T16:10:01.176376+0000 osd.4 (osd.4) 78 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:02.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:02 smithi166 bash[9446]: audit 2020-09-08T16:10:01.778877+0000 mon.a (mon.0) 2241 : audit [INF] from='client.? v1:172.21.15.104:0/1346986765' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:10:02.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:02 smithi055 bash[9296]: cluster 2020-09-08T16:10:01.176267+0000 osd.4 (osd.4) 77 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:02.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:02 smithi055 bash[9296]: cluster 2020-09-08T16:10:01.176376+0000 osd.4 (osd.4) 78 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:02.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:02 smithi055 bash[9296]: audit 2020-09-08T16:10:01.778877+0000 mon.a (mon.0) 2241 : audit [INF] from='client.? v1:172.21.15.104:0/1346986765' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:10:02.431 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:02 smithi104 bash[9203]: cluster 2020-09-08T16:10:01.176267+0000 osd.4 (osd.4) 77 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:02.431 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:02 smithi104 bash[9203]: cluster 2020-09-08T16:10:01.176376+0000 osd.4 (osd.4) 78 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:02.431 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:02 smithi104 bash[9203]: audit 2020-09-08T16:10:01.778877+0000 mon.a (mon.0) 2241 : audit [INF] from='client.? v1:172.21.15.104:0/1346986765' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:10:02.619 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:10:02.724 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:02.787 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 31 16 70663 70647 142.42 98.875 0.0107012 0.00701961 2020-09-08T16:10:02.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:10:02 smithi166 bash[11928]: debug 2020-09-08T16:10:02.616+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:03.059 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:03 smithi104 bash[9203]: cluster 2020-09-08T16:10:02.062006+0000 mon.a (mon.0) 2242 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:10:03.059 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:03 smithi104 bash[9203]: audit 2020-09-08T16:10:02.073233+0000 mon.a (mon.0) 2243 : audit [INF] from='client.? v1:172.21.15.104:0/1346986765' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:10:03.059 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:03 smithi104 bash[9203]: cluster 2020-09-08T16:10:02.073334+0000 mon.a (mon.0) 2244 : cluster [DBG] osdmap e715: 12 total, 12 up, 12 in 2020-09-08T16:10:03.060 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:03 smithi104 bash[9203]: cluster 2020-09-08T16:10:02.116613+0000 mgr.x (mgr.24118) 1281 : cluster [DBG] pgmap v1912: 17 pgs: 17 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s wr, 1.28k op/s 2020-09-08T16:10:03.326 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:10:03 smithi104 bash[15273]: debug 2020-09-08T16:10:03.325+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:03.326 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:03.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:03 smithi166 bash[9446]: cluster 2020-09-08T16:10:02.062006+0000 mon.a (mon.0) 2242 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:10:03.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:03 smithi166 bash[9446]: audit 2020-09-08T16:10:02.073233+0000 mon.a (mon.0) 2243 : audit [INF] from='client.? v1:172.21.15.104:0/1346986765' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:10:03.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:03 smithi166 bash[9446]: cluster 2020-09-08T16:10:02.073334+0000 mon.a (mon.0) 2244 : cluster [DBG] osdmap e715: 12 total, 12 up, 12 in 2020-09-08T16:10:03.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:03 smithi166 bash[9446]: cluster 2020-09-08T16:10:02.116613+0000 mgr.x (mgr.24118) 1281 : cluster [DBG] pgmap v1912: 17 pgs: 17 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s wr, 1.28k op/s 2020-09-08T16:10:03.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:03 smithi055 bash[9296]: cluster 2020-09-08T16:10:02.062006+0000 mon.a (mon.0) 2242 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:10:03.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:03 smithi055 bash[9296]: audit 2020-09-08T16:10:02.073233+0000 mon.a (mon.0) 2243 : audit [INF] from='client.? v1:172.21.15.104:0/1346986765' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:10:03.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:03 smithi055 bash[9296]: cluster 2020-09-08T16:10:02.073334+0000 mon.a (mon.0) 2244 : cluster [DBG] osdmap e715: 12 total, 12 up, 12 in 2020-09-08T16:10:03.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:03 smithi055 bash[9296]: cluster 2020-09-08T16:10:02.116613+0000 mgr.x (mgr.24118) 1281 : cluster [DBG] pgmap v1912: 17 pgs: 17 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s wr, 1.28k op/s 2020-09-08T16:10:03.430 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:10:03.788 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 32 16 72182 72166 140.936 94.9375 0.00598959 0.00709309 2020-09-08T16:10:04.024 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:10:04.134 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:04.368 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:10:04 smithi104 bash[17588]: debug 2020-09-08T16:10:04.025+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:04.789 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 33 16 73728 73712 139.593 96.625 0.0039099 0.00713577 2020-09-08T16:10:05.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:05 smithi104 bash[9203]: cluster 2020-09-08T16:10:04.117100+0000 mgr.x (mgr.24118) 1282 : cluster [DBG] pgmap v1913: 17 pgs: 17 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 125 MiB/s wr, 2.00k op/s 2020-09-08T16:10:05.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:05 smithi104 bash[9203]: cluster 2020-09-08T16:10:05.054956+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e716: 12 total, 12 up, 12 in 2020-09-08T16:10:05.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:05 smithi166 bash[9446]: cluster 2020-09-08T16:10:04.117100+0000 mgr.x (mgr.24118) 1282 : cluster [DBG] pgmap v1913: 17 pgs: 17 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 125 MiB/s wr, 2.00k op/s 2020-09-08T16:10:05.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:05 smithi166 bash[9446]: cluster 2020-09-08T16:10:05.054956+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e716: 12 total, 12 up, 12 in 2020-09-08T16:10:05.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:05 smithi055 bash[9296]: cluster 2020-09-08T16:10:04.117100+0000 mgr.x (mgr.24118) 1282 : cluster [DBG] pgmap v1913: 17 pgs: 17 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 125 MiB/s wr, 2.00k op/s 2020-09-08T16:10:05.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:05 smithi055 bash[9296]: cluster 2020-09-08T16:10:05.054956+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e716: 12 total, 12 up, 12 in 2020-09-08T16:10:05.795 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 34 16 75284 75268 138.347 97.25 0.00695783 0.00722569 2020-09-08T16:10:05.937 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:10:05.969 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:10:06.002 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:10:06.035 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:10:06.071 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:10:06.117 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:10:06.132 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:10:06.165 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:10:06.198 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:10:06.233 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:10:06.265 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:10:06.296 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:10:06.298 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:10:06.300 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:10:06.300 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:10:06.300 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:10:06.300 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:10:06.300 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:10:06.301 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:10:06.301 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:10:06.301 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:10:06.301 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:10:06.301 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:10:06.302 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:10:06.370 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T16:10:06.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:06 smithi104 bash[9203]: audit 2020-09-08T16:10:05.901639+0000 mon.a (mon.0) 2246 : audit [DBG] from='client.? v1:172.21.15.104:0/3354981282' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:10:06.539 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:10:06.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:06 smithi166 bash[9446]: audit 2020-09-08T16:10:05.901639+0000 mon.a (mon.0) 2246 : audit [DBG] from='client.? v1:172.21.15.104:0/3354981282' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:10:06.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:06 smithi055 bash[9296]: audit 2020-09-08T16:10:05.901639+0000 mon.a (mon.0) 2246 : audit [DBG] from='client.? v1:172.21.15.104:0/3354981282' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:10:06.788 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 35 16 76781 76765 137.067 93.5625 0.00856494 0.00729368 2020-09-08T16:10:07.330 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:07.434 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:07.443 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:07 smithi104 bash[9203]: cluster 2020-09-08T16:10:06.117649+0000 mgr.x (mgr.24118) 1283 : cluster [DBG] pgmap v1915: 17 pgs: 17 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s wr, 1.21k op/s 2020-09-08T16:10:07.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:07 smithi055 bash[9296]: cluster 2020-09-08T16:10:06.117649+0000 mgr.x (mgr.24118) 1283 : cluster [DBG] pgmap v1915: 17 pgs: 17 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s wr, 1.21k op/s 2020-09-08T16:10:07.730 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:07 smithi166 bash[9446]: cluster 2020-09-08T16:10:06.117649+0000 mgr.x (mgr.24118) 1283 : cluster [DBG] pgmap v1915: 17 pgs: 17 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s wr, 1.21k op/s 2020-09-08T16:10:07.730 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:10:07 smithi166 bash[18960]: debug 2020-09-08T16:10:07.328+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:07.769 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:07.788 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 36 16 78346 78330 135.976 97.8125 0.00433643 0.00734969 2020-09-08T16:10:07.874 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:08.010 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:08.118 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:10:08.156 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:10:08 smithi055 bash[11956]: debug 2020-09-08T16:10:08.011+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:08.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:10:07 smithi104 bash[15273]: debug 2020-09-08T16:10:07.769+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:08.549 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:10:08.665 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:08.788 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 37 16 79677 79661 134.55 83.1875 0.0203803 0.00742956 2020-09-08T16:10:08.836 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:10:08 smithi166 bash[14254]: debug 2020-09-08T16:10:08.544+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:08.838 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:08.944 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:09.056 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:09.122 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:10:08 smithi166 bash[18960]: debug 2020-09-08T16:10:08.836+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:09.163 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:09.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:10:09 smithi104 bash[15273]: debug 2020-09-08T16:10:09.057+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:09.348 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:09.454 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:09.462 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:09 smithi104 bash[9203]: cluster 2020-09-08T16:10:08.118125+0000 mgr.x (mgr.24118) 1284 : cluster [DBG] pgmap v1916: 17 pgs: 17 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 MiB/s wr, 1.79k op/s 2020-09-08T16:10:09.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:09 smithi166 bash[9446]: cluster 2020-09-08T16:10:08.118125+0000 mgr.x (mgr.24118) 1284 : cluster [DBG] pgmap v1916: 17 pgs: 17 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 MiB/s wr, 1.79k op/s 2020-09-08T16:10:09.617 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:10:09 smithi166 bash[18960]: debug 2020-09-08T16:10:09.344+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:09.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:09 smithi055 bash[9296]: cluster 2020-09-08T16:10:08.118125+0000 mgr.x (mgr.24118) 1284 : cluster [DBG] pgmap v1916: 17 pgs: 17 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 MiB/s wr, 1.79k op/s 2020-09-08T16:10:09.788 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 38 15 81272 81257 133.634 99.75 0.00495963 0.00748129 2020-09-08T16:10:10.302 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:10.315 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:10:10.407 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:10:10.516 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:10:10.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:10:10 smithi104 bash[22305]: debug 2020-09-08T16:10:10.301+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:10.788 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 39 15 82902 82887 132.819 101.875 0.00881547 0.00752692 2020-09-08T16:10:11.155 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:10:11.259 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:10:11.328 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:11 smithi166 bash[9446]: cluster 2020-09-08T16:10:10.053246+0000 mon.a (mon.0) 2247 : cluster [DBG] osdmap e717: 12 total, 12 up, 12 in 2020-09-08T16:10:11.328 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:11 smithi166 bash[9446]: cluster 2020-09-08T16:10:10.118577+0000 mgr.x (mgr.24118) 1285 : cluster [DBG] pgmap v1918: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s wr, 1.97k op/s 2020-09-08T16:10:11.329 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:11 smithi166 bash[9446]: cluster 2020-09-08T16:10:10.468916+0000 osd.9 (osd.9) 109 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:11.329 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:11 smithi166 bash[9446]: cluster 2020-09-08T16:10:10.469020+0000 osd.9 (osd.9) 110 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:11.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:11 smithi055 bash[9296]: cluster 2020-09-08T16:10:10.053246+0000 mon.a (mon.0) 2247 : cluster [DBG] osdmap e717: 12 total, 12 up, 12 in 2020-09-08T16:10:11.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:11 smithi055 bash[9296]: cluster 2020-09-08T16:10:10.118577+0000 mgr.x (mgr.24118) 1285 : cluster [DBG] pgmap v1918: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s wr, 1.97k op/s 2020-09-08T16:10:11.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:11 smithi055 bash[9296]: cluster 2020-09-08T16:10:10.468916+0000 osd.9 (osd.9) 109 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:11.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:11 smithi055 bash[9296]: cluster 2020-09-08T16:10:10.469020+0000 osd.9 (osd.9) 110 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:11.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:10:11 smithi104 bash[19922]: debug 2020-09-08T16:10:11.153+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:11.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:11 smithi104 bash[9203]: cluster 2020-09-08T16:10:10.053246+0000 mon.a (mon.0) 2247 : cluster [DBG] osdmap e717: 12 total, 12 up, 12 in 2020-09-08T16:10:11.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:11 smithi104 bash[9203]: cluster 2020-09-08T16:10:10.118577+0000 mgr.x (mgr.24118) 1285 : cluster [DBG] pgmap v1918: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s wr, 1.97k op/s 2020-09-08T16:10:11.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:11 smithi104 bash[9203]: cluster 2020-09-08T16:10:10.468916+0000 osd.9 (osd.9) 109 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:11.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:11 smithi104 bash[9203]: cluster 2020-09-08T16:10:10.469020+0000 osd.9 (osd.9) 110 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:11.788 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 16:10:11.793307 min lat: 0.00165771 max lat: 0.324694 avg lat: 0.00757551 2020-09-08T16:10:11.789 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T16:10:11.789 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 40 16 84485 84469 131.97 98.875 0.00800154 0.00757551 2020-09-08T16:10:11.834 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:10:11.940 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:12.095 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:10:11 smithi166 bash[11928]: debug 2020-09-08T16:10:11.832+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:12.095 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:12 smithi166 bash[9446]: cluster 2020-09-08T16:10:11.068760+0000 mon.a (mon.0) 2248 : cluster [DBG] osdmap e718: 12 total, 12 up, 12 in 2020-09-08T16:10:12.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:12 smithi104 bash[9203]: cluster 2020-09-08T16:10:11.068760+0000 mon.a (mon.0) 2248 : cluster [DBG] osdmap e718: 12 total, 12 up, 12 in 2020-09-08T16:10:12.469 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T16:10:12.469 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:10:12.525 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:12 smithi055 bash[9296]: cluster 2020-09-08T16:10:11.068760+0000 mon.a (mon.0) 2248 : cluster [DBG] osdmap e718: 12 total, 12 up, 12 in 2020-09-08T16:10:12.788 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 41 16 85484 85468 130.274 62.4375 0.00352116 0.00767245 2020-09-08T16:10:13.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:13 smithi166 bash[9446]: cluster 2020-09-08T16:10:12.119891+0000 mgr.x (mgr.24118) 1286 : cluster [DBG] pgmap v1920: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 67 MiB/s wr, 1.08k op/s 2020-09-08T16:10:13.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:13 smithi166 bash[9446]: cluster 2020-09-08T16:10:12.290768+0000 osd.2 (osd.2) 112 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:13.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:13 smithi166 bash[9446]: cluster 2020-09-08T16:10:12.290884+0000 osd.2 (osd.2) 113 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:13.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:13 smithi055 bash[9296]: cluster 2020-09-08T16:10:12.119891+0000 mgr.x (mgr.24118) 1286 : cluster [DBG] pgmap v1920: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 67 MiB/s wr, 1.08k op/s 2020-09-08T16:10:13.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:13 smithi055 bash[9296]: cluster 2020-09-08T16:10:12.290768+0000 osd.2 (osd.2) 112 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:13.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:13 smithi055 bash[9296]: cluster 2020-09-08T16:10:12.290884+0000 osd.2 (osd.2) 113 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:13.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:13 smithi104 bash[9203]: cluster 2020-09-08T16:10:12.119891+0000 mgr.x (mgr.24118) 1286 : cluster [DBG] pgmap v1920: 17 pgs: 17 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 67 MiB/s wr, 1.08k op/s 2020-09-08T16:10:13.431 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:13 smithi104 bash[9203]: cluster 2020-09-08T16:10:12.290768+0000 osd.2 (osd.2) 112 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:13.431 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:13 smithi104 bash[9203]: cluster 2020-09-08T16:10:12.290884+0000 osd.2 (osd.2) 113 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:13.654 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2020-09-08T16:10:13.788 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 42 16 86960 86944 129.369 92.25 0.00635311 0.00772769 2020-09-08T16:10:13.825 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:10:14.073 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T16:10:14.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:14 smithi166 bash[9446]: cluster 2020-09-08T16:10:13.074319+0000 mon.a (mon.0) 2249 : cluster [DBG] osdmap e719: 12 total, 12 up, 12 in 2020-09-08T16:10:14.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:14 smithi166 bash[9446]: audit 2020-09-08T16:10:13.458193+0000 mon.a (mon.0) 2250 : audit [INF] from='client.? v1:172.21.15.104:0/3367926943' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:10:14.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:14 smithi104 bash[9203]: cluster 2020-09-08T16:10:13.074319+0000 mon.a (mon.0) 2249 : cluster [DBG] osdmap e719: 12 total, 12 up, 12 in 2020-09-08T16:10:14.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:14 smithi104 bash[9203]: audit 2020-09-08T16:10:13.458193+0000 mon.a (mon.0) 2250 : audit [INF] from='client.? v1:172.21.15.104:0/3367926943' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:10:14.544 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:14 smithi055 bash[9296]: cluster 2020-09-08T16:10:13.074319+0000 mon.a (mon.0) 2249 : cluster [DBG] osdmap e719: 12 total, 12 up, 12 in 2020-09-08T16:10:14.544 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:14 smithi055 bash[9296]: audit 2020-09-08T16:10:13.458193+0000 mon.a (mon.0) 2250 : audit [INF] from='client.? v1:172.21.15.104:0/3367926943' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:10:14.789 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 43 16 88074 88058 127.979 69.625 0.0330503 0.00780903 2020-09-08T16:10:15.370 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:15 smithi166 bash[9446]: cluster 2020-09-08T16:10:14.070771+0000 mon.a (mon.0) 2251 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:10:15.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:15 smithi166 bash[9446]: audit 2020-09-08T16:10:14.074421+0000 mon.a (mon.0) 2252 : audit [INF] from='client.? v1:172.21.15.104:0/3367926943' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:10:15.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:15 smithi166 bash[9446]: cluster 2020-09-08T16:10:14.074455+0000 mon.a (mon.0) 2253 : cluster [DBG] osdmap e720: 12 total, 12 up, 12 in 2020-09-08T16:10:15.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:15 smithi166 bash[9446]: cluster 2020-09-08T16:10:14.120298+0000 mgr.x (mgr.24118) 1287 : cluster [DBG] pgmap v1923: 17 pgs: 17 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s wr, 1.58k op/s 2020-09-08T16:10:15.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:15 smithi166 bash[9446]: cluster 2020-09-08T16:10:15.058603+0000 mon.a (mon.0) 2254 : cluster [DBG] osdmap e721: 12 total, 12 up, 12 in 2020-09-08T16:10:15.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:15 smithi055 bash[9296]: cluster 2020-09-08T16:10:14.070771+0000 mon.a (mon.0) 2251 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:10:15.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:15 smithi055 bash[9296]: audit 2020-09-08T16:10:14.074421+0000 mon.a (mon.0) 2252 : audit [INF] from='client.? v1:172.21.15.104:0/3367926943' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:10:15.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:15 smithi055 bash[9296]: cluster 2020-09-08T16:10:14.074455+0000 mon.a (mon.0) 2253 : cluster [DBG] osdmap e720: 12 total, 12 up, 12 in 2020-09-08T16:10:15.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:15 smithi055 bash[9296]: cluster 2020-09-08T16:10:14.120298+0000 mgr.x (mgr.24118) 1287 : cluster [DBG] pgmap v1923: 17 pgs: 17 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s wr, 1.58k op/s 2020-09-08T16:10:15.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:15 smithi055 bash[9296]: cluster 2020-09-08T16:10:15.058603+0000 mon.a (mon.0) 2254 : cluster [DBG] osdmap e721: 12 total, 12 up, 12 in 2020-09-08T16:10:15.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:15 smithi104 bash[9203]: cluster 2020-09-08T16:10:14.070771+0000 mon.a (mon.0) 2251 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:10:15.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:15 smithi104 bash[9203]: audit 2020-09-08T16:10:14.074421+0000 mon.a (mon.0) 2252 : audit [INF] from='client.? v1:172.21.15.104:0/3367926943' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:10:15.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:15 smithi104 bash[9203]: cluster 2020-09-08T16:10:14.074455+0000 mon.a (mon.0) 2253 : cluster [DBG] osdmap e720: 12 total, 12 up, 12 in 2020-09-08T16:10:15.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:15 smithi104 bash[9203]: cluster 2020-09-08T16:10:14.120298+0000 mgr.x (mgr.24118) 1287 : cluster [DBG] pgmap v1923: 17 pgs: 17 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s wr, 1.58k op/s 2020-09-08T16:10:15.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:15 smithi104 bash[9203]: cluster 2020-09-08T16:10:15.058603+0000 mon.a (mon.0) 2254 : cluster [DBG] osdmap e721: 12 total, 12 up, 12 in 2020-09-08T16:10:15.577 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:15.681 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:15.788 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 44 16 89488 89472 127.079 88.375 0.0151544 0.00786692 2020-09-08T16:10:15.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:10:15 smithi055 bash[11956]: debug 2020-09-08T16:10:15.574+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:16.011 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:10:16 smithi104 bash[15273]: debug 2020-09-08T16:10:16.013+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:16.011 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:16.115 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:10:16.227 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:10:16.334 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:16.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:10:16 smithi055 bash[19003]: debug 2020-09-08T16:10:16.226+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:16.458 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:16.565 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:16.594 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:10:16.632 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:10:16.661 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:10:16.685 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:16.692 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:10:16.720 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:10:16.751 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:10:16.780 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:10:16.788 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 45 16 90899 90883 126.214 88.1875 0.00785765 0.00792112 2020-09-08T16:10:16.791 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:10:16.799 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:10:16 smithi055 bash[11956]: debug 2020-09-08T16:10:16.458+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:16.799 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:10:16 smithi055 bash[11956]: debug 2020-09-08T16:10:16.686+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:16.826 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:10:16.857 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:10:16.894 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:10:16.907 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:10:16.931 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:10:16.960 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:10:16.967 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:10:16.968 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:10:16.968 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:10:16.968 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:10:16.968 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:10:16.968 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:10:16.969 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:10:16.969 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:10:16.969 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:10:16.969 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:10:16.969 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:10:16.970 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:10:17.014 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:10:17.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:10:16 smithi055 bash[19003]: debug 2020-09-08T16:10:16.906+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:17.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:17 smithi166 bash[9446]: cluster 2020-09-08T16:10:16.120731+0000 mgr.x (mgr.24118) 1288 : cluster [DBG] pgmap v1925: 17 pgs: 17 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 90 MiB/s wr, 1.44k op/s 2020-09-08T16:10:17.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:17 smithi166 bash[9446]: cluster 2020-09-08T16:10:16.377994+0000 osd.7 (osd.7) 153 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:10:17.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:17 smithi166 bash[9446]: cluster 2020-09-08T16:10:16.527717+0000 osd.7 (osd.7) 154 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:10:17.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:17 smithi166 bash[9446]: audit 2020-09-08T16:10:16.567809+0000 mon.a (mon.0) 2255 : audit [DBG] from='client.? v1:172.21.15.104:0/656365555' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:10:17.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:17 smithi104 bash[9203]: cluster 2020-09-08T16:10:16.120731+0000 mgr.x (mgr.24118) 1288 : cluster [DBG] pgmap v1925: 17 pgs: 17 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 90 MiB/s wr, 1.44k op/s 2020-09-08T16:10:17.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:17 smithi104 bash[9203]: cluster 2020-09-08T16:10:16.377994+0000 osd.7 (osd.7) 153 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:10:17.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:17 smithi104 bash[9203]: cluster 2020-09-08T16:10:16.527717+0000 osd.7 (osd.7) 154 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:10:17.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:17 smithi104 bash[9203]: audit 2020-09-08T16:10:16.567809+0000 mon.a (mon.0) 2255 : audit [DBG] from='client.? v1:172.21.15.104:0/656365555' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:10:17.510 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:10:17.614 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:10:17.623 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:17 smithi055 bash[9296]: cluster 2020-09-08T16:10:16.120731+0000 mgr.x (mgr.24118) 1288 : cluster [DBG] pgmap v1925: 17 pgs: 17 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 90 MiB/s wr, 1.44k op/s 2020-09-08T16:10:17.624 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:17 smithi055 bash[9296]: cluster 2020-09-08T16:10:16.377994+0000 osd.7 (osd.7) 153 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:10:17.624 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:17 smithi055 bash[9296]: cluster 2020-09-08T16:10:16.527717+0000 osd.7 (osd.7) 154 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:10:17.624 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:17 smithi055 bash[9296]: audit 2020-09-08T16:10:16.567809+0000 mon.a (mon.0) 2255 : audit [DBG] from='client.? v1:172.21.15.104:0/656365555' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:10:17.720 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:10:17.747 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T16:10:17.789 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 46 16 92335 92319 125.422 89.75 0.00538015 0.00797114 2020-09-08T16:10:17.824 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:10:17.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:10:17 smithi166 bash[14254]: debug 2020-09-08T16:10:17.500+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:17.905 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:10:17.908 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:10:17 smithi055 bash[16646]: debug 2020-09-08T16:10:17.718+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:17.939 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:10:18.043 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:10:18.296 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:10:18.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:10:18 smithi166 bash[11928]: debug 2020-09-08T16:10:18.292+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:18.400 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:10:18.406 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:10:17 smithi055 bash[19003]: debug 2020-09-08T16:10:17.938+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:18.513 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:10:18.618 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:10:18.734 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:10:18 smithi055 bash[19003]: debug 2020-09-08T16:10:18.514+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:18.734 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:10:18 smithi055 bash[14301]: debug 2020-09-08T16:10:18.734+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:18.735 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:10:18.789 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 47 16 93560 93544 124.382 76.5625 0.00955916 0.00803738 2020-09-08T16:10:18.838 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:18.977 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:10:18.977 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T16:10:18.977 INFO:teuthology.orchestra.run.smithi166.stdout: "complaint_time": 30, 2020-09-08T16:10:18.978 INFO:teuthology.orchestra.run.smithi166.stdout: "num_blocked_ops": 0 2020-09-08T16:10:18.978 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:10:19.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:19 smithi104 bash[9203]: cluster 2020-09-08T16:10:18.121467+0000 mgr.x (mgr.24118) 1289 : cluster [DBG] pgmap v1926: 17 pgs: 17 active+clean; 5.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 133 MiB/s wr, 2.13k op/s 2020-09-08T16:10:19.369 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:19 smithi166 bash[9446]: cluster 2020-09-08T16:10:18.121467+0000 mgr.x (mgr.24118) 1289 : cluster [DBG] pgmap v1926: 17 pgs: 17 active+clean; 5.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 133 MiB/s wr, 2.13k op/s 2020-09-08T16:10:19.462 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:19.563 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:19.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:19 smithi055 bash[9296]: cluster 2020-09-08T16:10:18.121467+0000 mgr.x (mgr.24118) 1289 : cluster [DBG] pgmap v1926: 17 pgs: 17 active+clean; 5.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 133 MiB/s wr, 2.13k op/s 2020-09-08T16:10:19.755 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:10:19 smithi104 bash[15273]: debug 2020-09-08T16:10:19.457+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:19.755 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:19.789 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 48 16 94914 94898 123.553 84.625 0.129188 0.00808893 2020-09-08T16:10:19.857 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:10:20.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:10:19 smithi104 bash[22305]: debug 2020-09-08T16:10:19.749+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:20.361 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:10:20.464 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:10:20 smithi166 bash[16593]: debug 2020-09-08T16:10:20.360+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:20.465 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:20 smithi166 bash[9446]: cluster 2020-09-08T16:10:19.948957+0000 osd.0 (osd.0) 128 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:20.466 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:20 smithi166 bash[9446]: cluster 2020-09-08T16:10:19.949089+0000 osd.0 (osd.0) 129 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:20.467 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2020-09-08T16:10:20.467 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:10:20.588 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:10:20.630 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:10:20.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:20 smithi055 bash[9296]: cluster 2020-09-08T16:10:19.948957+0000 osd.0 (osd.0) 128 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:20.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:20 smithi055 bash[9296]: cluster 2020-09-08T16:10:19.949089+0000 osd.0 (osd.0) 129 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:20.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:20 smithi104 bash[9203]: cluster 2020-09-08T16:10:19.948957+0000 osd.0 (osd.0) 128 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:20.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:20 smithi104 bash[9203]: cluster 2020-09-08T16:10:19.949089+0000 osd.0 (osd.0) 129 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:20.694 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:10:20.783 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:10:20.789 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 49 16 96004 95988 122.422 68.125 0.0113649 0.00816638 2020-09-08T16:10:20.833 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:10:20.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:10:20 smithi166 bash[14254]: debug 2020-09-08T16:10:20.588+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:20.936 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:10:20.939 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:21.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:10:20 smithi055 bash[14301]: debug 2020-09-08T16:10:20.834+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:21.557 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:21 smithi104 bash[9203]: cluster 2020-09-08T16:10:19.674495+0000 osd.7 (osd.7) 155 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:21.558 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:21 smithi104 bash[9203]: cluster 2020-09-08T16:10:19.674583+0000 osd.7 (osd.7) 156 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:21.558 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:21 smithi104 bash[9203]: cluster 2020-09-08T16:10:20.121972+0000 mgr.x (mgr.24118) 1290 : cluster [DBG] pgmap v1927: 17 pgs: 17 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s wr, 1.15k op/s 2020-09-08T16:10:21.558 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:21 smithi104 bash[9203]: cluster 2020-09-08T16:10:20.185610+0000 mon.a (mon.0) 2256 : cluster [DBG] osdmap e722: 12 total, 12 up, 12 in 2020-09-08T16:10:21.560 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:21.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:21 smithi166 bash[9446]: cluster 2020-09-08T16:10:19.674495+0000 osd.7 (osd.7) 155 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:21.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:21 smithi166 bash[9446]: cluster 2020-09-08T16:10:19.674583+0000 osd.7 (osd.7) 156 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:21.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:21 smithi166 bash[9446]: cluster 2020-09-08T16:10:20.121972+0000 mgr.x (mgr.24118) 1290 : cluster [DBG] pgmap v1927: 17 pgs: 17 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s wr, 1.15k op/s 2020-09-08T16:10:21.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:21 smithi166 bash[9446]: cluster 2020-09-08T16:10:20.185610+0000 mon.a (mon.0) 2256 : cluster [DBG] osdmap e722: 12 total, 12 up, 12 in 2020-09-08T16:10:21.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:21 smithi055 bash[9296]: cluster 2020-09-08T16:10:19.674495+0000 osd.7 (osd.7) 155 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:21.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:21 smithi055 bash[9296]: cluster 2020-09-08T16:10:19.674583+0000 osd.7 (osd.7) 156 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:21.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:21 smithi055 bash[9296]: cluster 2020-09-08T16:10:20.121972+0000 mgr.x (mgr.24118) 1290 : cluster [DBG] pgmap v1927: 17 pgs: 17 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s wr, 1.15k op/s 2020-09-08T16:10:21.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:21 smithi055 bash[9296]: cluster 2020-09-08T16:10:20.185610+0000 mon.a (mon.0) 2256 : cluster [DBG] osdmap e722: 12 total, 12 up, 12 in 2020-09-08T16:10:21.664 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:10:21.789 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 50 15 97473 97458 121.811 91.875 0.0124254 0.00820734 2020-09-08T16:10:21.863 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:10:21 smithi104 bash[15273]: debug 2020-09-08T16:10:21.557+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:21.864 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:10:21.971 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:10:22.175 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:10:21 smithi104 bash[19922]: debug 2020-09-08T16:10:21.865+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:22.710 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:10:22.789 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 51 16 98904 98888 121.175 89.375 0.0134368 0.00825105 2020-09-08T16:10:22.819 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:10:22.958 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:10:23.063 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:10:23.159 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:10:22 smithi055 bash[19003]: debug 2020-09-08T16:10:22.958+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:23.175 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:10:22 smithi104 bash[19922]: debug 2020-09-08T16:10:22.709+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:23.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:23 smithi166 bash[9446]: cluster 2020-09-08T16:10:22.122399+0000 mgr.x (mgr.24118) 1291 : cluster [DBG] pgmap v1929: 17 pgs: 17 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s wr, 950 op/s 2020-09-08T16:10:23.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:23 smithi055 bash[9296]: cluster 2020-09-08T16:10:22.122399+0000 mgr.x (mgr.24118) 1291 : cluster [DBG] pgmap v1929: 17 pgs: 17 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s wr, 950 op/s 2020-09-08T16:10:23.699 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:23 smithi104 bash[9203]: cluster 2020-09-08T16:10:22.122399+0000 mgr.x (mgr.24118) 1291 : cluster [DBG] pgmap v1929: 17 pgs: 17 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s wr, 950 op/s 2020-09-08T16:10:23.789 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 52 16 100192 100176 120.393 80.5 0.00530727 0.00830372 2020-09-08T16:10:23.816 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:10:23.920 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:10:24.059 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:10:24.063 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:10:24 smithi166 bash[11928]: debug 2020-09-08T16:10:24.056+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:24.064 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:10:23 smithi166 bash[16593]: debug 2020-09-08T16:10:23.808+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:24.165 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:10:24.514 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:10:24.619 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:24.790 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 53 16 101522 101506 119.689 83.125 0.0174705 0.00835311 2020-09-08T16:10:24.872 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:24.927 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:10:24 smithi104 bash[19922]: debug 2020-09-08T16:10:24.513+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:24.974 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:25.108 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:25.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:10:24 smithi166 bash[18960]: debug 2020-09-08T16:10:24.860+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:25.209 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T16:10:25.212 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:10:25.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:10:25 smithi055 bash[11956]: debug 2020-09-08T16:10:25.106+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:25.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:25 smithi055 bash[9296]: cluster 2020-09-08T16:10:24.122906+0000 mgr.x (mgr.24118) 1292 : cluster [DBG] pgmap v1930: 17 pgs: 17 active+clean; 6.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s wr, 1.38k op/s 2020-09-08T16:10:25.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:25 smithi055 bash[9296]: audit 2020-09-08T16:10:24.833425+0000 mon.a (mon.0) 2257 : audit [INF] from='client.? v1:172.21.15.104:0/1140692521' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:10:25.572 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:25 smithi166 bash[9446]: cluster 2020-09-08T16:10:24.122906+0000 mgr.x (mgr.24118) 1292 : cluster [DBG] pgmap v1930: 17 pgs: 17 active+clean; 6.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s wr, 1.38k op/s 2020-09-08T16:10:25.573 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:25 smithi166 bash[9446]: audit 2020-09-08T16:10:24.833425+0000 mon.a (mon.0) 2257 : audit [INF] from='client.? v1:172.21.15.104:0/1140692521' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:10:25.573 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:10:25 smithi166 bash[16593]: debug 2020-09-08T16:10:25.572+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:25.573 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:10:25.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:25 smithi104 bash[9203]: cluster 2020-09-08T16:10:24.122906+0000 mgr.x (mgr.24118) 1292 : cluster [DBG] pgmap v1930: 17 pgs: 17 active+clean; 6.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s wr, 1.38k op/s 2020-09-08T16:10:25.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:25 smithi104 bash[9203]: audit 2020-09-08T16:10:24.833425+0000 mon.a (mon.0) 2257 : audit [INF] from='client.? v1:172.21.15.104:0/1140692521' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:10:25.695 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:10:25.789 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 54 16 102832 102816 118.989 81.875 0.021972 0.00840246 2020-09-08T16:10:25.846 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:10:25.847 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:10:25 smithi166 bash[16593]: debug 2020-09-08T16:10:25.844+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:25.950 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:26.148 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:26.252 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:10:26.317 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:10:26.318 INFO:teuthology.orchestra.run.smithi166.stdout: "size": 20, 2020-09-08T16:10:26.318 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 600, 2020-09-08T16:10:26.318 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T16:10:26.318 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.318 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:32291 8.a 8:5d0dc287:::benchmark_data_smithi166_52302_object32290:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:10:26.319 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:05:19.544842+0000", 2020-09-08T16:10:26.319 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 306.77260099799997, 2020-09-08T16:10:26.319 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.212336571, 2020-09-08T16:10:26.319 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:26.319 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:26.320 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:26.320 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:26.320 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:26.320 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 32291 2020-09-08T16:10:26.320 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.321 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:26.321 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.321 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:26.321 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.544842+0000", 2020-09-08T16:10:26.321 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:26.322 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.322 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.322 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:26.322 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.544842+0000", 2020-09-08T16:10:26.322 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999619 2020-09-08T16:10:26.323 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.323 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.323 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:26.323 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.544803+0000", 2020-09-08T16:10:26.323 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013018599999999999 2020-09-08T16:10:26.324 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.324 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.324 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:26.324 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.544934+0000", 2020-09-08T16:10:26.324 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4440000000000001e-05 2020-09-08T16:10:26.325 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.325 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.325 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:26.325 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.544948+0000", 2020-09-08T16:10:26.325 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.0240000000000003e-06 2020-09-08T16:10:26.325 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.326 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.326 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:26.326 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.544954+0000", 2020-09-08T16:10:26.326 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.1587000000000005e-05 2020-09-08T16:10:26.326 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.327 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.327 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.327 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.545016+0000", 2020-09-08T16:10:26.327 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012186800000000001 2020-09-08T16:10:26.327 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.328 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.328 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:26.328 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.545137+0000", 2020-09-08T16:10:26.328 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013769499999999999 2020-09-08T16:10:26.328 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.329 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.329 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:26.329 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.545275+0000", 2020-09-08T16:10:26.329 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0029124239999999998 2020-09-08T16:10:26.329 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.330 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.330 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:26.330 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.548188+0000", 2020-09-08T16:10:26.330 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20882023899999999 2020-09-08T16:10:26.330 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.331 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.331 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:26.331 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.757008+0000", 2020-09-08T16:10:26.331 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000103519 2020-09-08T16:10:26.331 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.332 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.332 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:26.332 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.757111+0000", 2020-09-08T16:10:26.332 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6860999999999996e-05 2020-09-08T16:10:26.332 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.333 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.333 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:26.333 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.757178+0000", 2020-09-08T16:10:26.333 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.212336571 2020-09-08T16:10:26.333 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.334 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.334 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.334 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.334 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.335 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:32293 8.a 8:51a693fb:::benchmark_data_smithi166_52302_object32292:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:10:26.335 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:05:19.545016+0000", 2020-09-08T16:10:26.335 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 306.77242712200001, 2020-09-08T16:10:26.335 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.214215392, 2020-09-08T16:10:26.335 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:26.336 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:26.336 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:26.336 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:26.336 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:26.336 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 32293 2020-09-08T16:10:26.337 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.337 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:26.337 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.337 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:26.337 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.545016+0000", 2020-09-08T16:10:26.338 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:26.338 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.338 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.338 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:26.338 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.545016+0000", 2020-09-08T16:10:26.338 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999671 2020-09-08T16:10:26.339 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.339 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.339 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:26.339 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.544983+0000", 2020-09-08T16:10:26.339 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000105738 2020-09-08T16:10:26.340 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.340 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.340 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:26.340 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.545088+0000", 2020-09-08T16:10:26.340 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1294999999999999e-05 2020-09-08T16:10:26.341 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.341 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.341 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:26.341 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.545100+0000", 2020-09-08T16:10:26.341 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.8389999999999997e-06 2020-09-08T16:10:26.342 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.342 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.342 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:26.342 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.545104+0000", 2020-09-08T16:10:26.342 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0014766340000000001 2020-09-08T16:10:26.342 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.343 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.343 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.343 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.546581+0000", 2020-09-08T16:10:26.343 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015819099999999999 2020-09-08T16:10:26.343 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.344 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.344 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:26.344 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.546739+0000", 2020-09-08T16:10:26.344 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00039706399999999998 2020-09-08T16:10:26.344 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.345 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.345 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:26.345 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.547136+0000", 2020-09-08T16:10:26.345 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0014695909999999999 2020-09-08T16:10:26.345 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.346 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.346 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:26.346 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.548606+0000", 2020-09-08T16:10:26.346 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21038283299999999 2020-09-08T16:10:26.347 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.347 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.347 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:26.347 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.758989+0000", 2020-09-08T16:10:26.347 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013624899999999999 2020-09-08T16:10:26.348 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.348 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.348 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:26.348 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.759125+0000", 2020-09-08T16:10:26.348 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000105949 2020-09-08T16:10:26.349 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.349 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.349 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:26.349 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.759231+0000", 2020-09-08T16:10:26.349 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.214215392 2020-09-08T16:10:26.349 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.350 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.350 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.350 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.350 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.350 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:32307 8.a 8:50147e3e:::benchmark_data_smithi166_52302_object32306:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:10:26.351 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:05:19.552048+0000", 2020-09-08T16:10:26.351 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 306.76539463799998, 2020-09-08T16:10:26.351 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.207646253, 2020-09-08T16:10:26.351 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:26.351 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:26.352 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:26.352 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:26.352 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:26.352 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 32307 2020-09-08T16:10:26.352 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.353 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:26.353 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.353 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:26.353 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.552048+0000", 2020-09-08T16:10:26.353 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:26.354 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.354 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.354 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:26.354 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.552048+0000", 2020-09-08T16:10:26.354 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.999958 2020-09-08T16:10:26.355 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.355 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.355 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:26.355 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.552006+0000", 2020-09-08T16:10:26.355 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000127892 2020-09-08T16:10:26.355 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.356 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.356 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:26.356 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.552134+0000", 2020-09-08T16:10:26.356 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.8558000000000001e-05 2020-09-08T16:10:26.356 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.357 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.357 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:26.357 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.552152+0000", 2020-09-08T16:10:26.357 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.3860000000000004e-06 2020-09-08T16:10:26.358 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.358 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.358 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:26.358 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.552159+0000", 2020-09-08T16:10:26.358 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.0792999999999999e-05 2020-09-08T16:10:26.359 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.359 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.359 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.359 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.552220+0000", 2020-09-08T16:10:26.359 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000194403 2020-09-08T16:10:26.360 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.360 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.360 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:26.360 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.552414+0000", 2020-09-08T16:10:26.361 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018793999999999999 2020-09-08T16:10:26.361 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.362 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.362 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:26.363 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.552602+0000", 2020-09-08T16:10:26.363 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.001328018 2020-09-08T16:10:26.363 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.363 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.363 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:26.364 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.553930+0000", 2020-09-08T16:10:26.364 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20545465800000001 2020-09-08T16:10:26.364 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.364 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.364 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:26.365 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.759385+0000", 2020-09-08T16:10:26.365 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000179865 2020-09-08T16:10:26.365 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.365 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.365 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:26.367 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.759564+0000", 2020-09-08T16:10:26.367 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000129827 2020-09-08T16:10:26.367 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.367 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.368 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:26.368 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.759694+0000", 2020-09-08T16:10:26.368 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.207646253 2020-09-08T16:10:26.368 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.368 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.369 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.369 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.369 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.369 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:32318 8.a 8:5aecebb2:::benchmark_data_smithi166_52302_object32317:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:10:26.369 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:05:19.556428+0000", 2020-09-08T16:10:26.371 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 306.76101453199999, 2020-09-08T16:10:26.371 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20520823099999999, 2020-09-08T16:10:26.371 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:26.371 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:26.371 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:26.372 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:26.372 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:26.372 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 32318 2020-09-08T16:10:26.372 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.372 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:26.373 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.373 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:26.373 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556428+0000", 2020-09-08T16:10:26.373 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:26.373 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.374 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.374 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:26.374 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556428+0000", 2020-09-08T16:10:26.374 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999666 2020-09-08T16:10:26.375 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.375 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.375 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:26.375 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556395+0000", 2020-09-08T16:10:26.375 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012139299999999999 2020-09-08T16:10:26.375 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.376 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.376 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:26.376 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556516+0000", 2020-09-08T16:10:26.376 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3336000000000001e-05 2020-09-08T16:10:26.376 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.377 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.377 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:26.377 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556530+0000", 2020-09-08T16:10:26.377 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.9359999999999996e-06 2020-09-08T16:10:26.377 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.378 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.378 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:26.378 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556536+0000", 2020-09-08T16:10:26.378 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.7509000000000002e-05 2020-09-08T16:10:26.378 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.379 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.379 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.379 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556583+0000", 2020-09-08T16:10:26.379 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000149252 2020-09-08T16:10:26.379 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.380 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.380 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:26.380 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556732+0000", 2020-09-08T16:10:26.380 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017202299999999999 2020-09-08T16:10:26.380 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.381 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.381 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:26.381 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556904+0000", 2020-09-08T16:10:26.381 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0021601089999999999 2020-09-08T16:10:26.381 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.382 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.382 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:26.382 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.559064+0000", 2020-09-08T16:10:26.382 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.202383175 2020-09-08T16:10:26.382 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.383 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.383 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:26.383 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.761448+0000", 2020-09-08T16:10:26.383 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000107454 2020-09-08T16:10:26.383 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.383 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.384 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:26.384 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.761555+0000", 2020-09-08T16:10:26.384 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.1321000000000003e-05 2020-09-08T16:10:26.384 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.384 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.385 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:26.385 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.761636+0000", 2020-09-08T16:10:26.385 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20520823099999999 2020-09-08T16:10:26.385 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.385 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.386 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.386 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.386 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.386 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:32320 8.a 8:581b9ef4:::benchmark_data_smithi166_52302_object32319:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:10:26.386 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:05:19.556597+0000", 2020-09-08T16:10:26.387 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 306.760845373, 2020-09-08T16:10:26.387 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.205298333, 2020-09-08T16:10:26.387 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:26.387 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:26.387 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:26.387 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:26.388 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:26.388 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 32320 2020-09-08T16:10:26.388 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.388 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:26.388 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.389 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:26.389 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556597+0000", 2020-09-08T16:10:26.389 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:26.389 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.390 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.390 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:26.390 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556597+0000", 2020-09-08T16:10:26.390 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999661 2020-09-08T16:10:26.391 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.391 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.391 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:26.391 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556563+0000", 2020-09-08T16:10:26.391 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000114177 2020-09-08T16:10:26.392 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.392 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.392 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:26.392 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556678+0000", 2020-09-08T16:10:26.392 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.148e-05 2020-09-08T16:10:26.393 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.393 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.393 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:26.393 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556689+0000", 2020-09-08T16:10:26.393 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.4229999999999998e-06 2020-09-08T16:10:26.394 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.394 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.394 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:26.394 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556693+0000", 2020-09-08T16:10:26.394 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.001235239 2020-09-08T16:10:26.395 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.395 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.395 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.395 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.557929+0000", 2020-09-08T16:10:26.395 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000111667 2020-09-08T16:10:26.396 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.396 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.396 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:26.396 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.558040+0000", 2020-09-08T16:10:26.396 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017242099999999999 2020-09-08T16:10:26.397 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.397 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.397 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:26.397 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.558213+0000", 2020-09-08T16:10:26.397 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0019623790000000002 2020-09-08T16:10:26.398 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.398 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.398 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:26.398 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.560175+0000", 2020-09-08T16:10:26.398 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20156875899999999 2020-09-08T16:10:26.398 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.399 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.399 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:26.399 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.761744+0000", 2020-09-08T16:10:26.399 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.8767000000000001e-05 2020-09-08T16:10:26.399 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.400 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.400 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:26.400 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.761813+0000", 2020-09-08T16:10:26.400 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.2893999999999996e-05 2020-09-08T16:10:26.400 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.401 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.401 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:26.401 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.761896+0000", 2020-09-08T16:10:26.401 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.205298333 2020-09-08T16:10:26.402 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.402 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.409 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.409 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.409 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.409 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:32321 8.a 8:5ff8881a:::benchmark_data_smithi166_52302_object32320:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:10:26.409 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:05:19.556747+0000", 2020-09-08T16:10:26.410 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 306.76069524000002, 2020-09-08T16:10:26.410 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20822859299999999, 2020-09-08T16:10:26.410 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:26.410 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:26.410 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:26.411 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:26.411 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:26.411 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 32321 2020-09-08T16:10:26.411 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.411 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:26.412 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.412 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:26.412 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556747+0000", 2020-09-08T16:10:26.412 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:26.413 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.413 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.413 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:26.413 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556747+0000", 2020-09-08T16:10:26.414 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999666 2020-09-08T16:10:26.414 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.414 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.414 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:26.414 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556714+0000", 2020-09-08T16:10:26.415 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000105517 2020-09-08T16:10:26.415 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.415 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.415 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:26.415 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556820+0000", 2020-09-08T16:10:26.416 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1925e-05 2020-09-08T16:10:26.416 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.416 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.416 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:26.416 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556832+0000", 2020-09-08T16:10:26.417 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.369e-06 2020-09-08T16:10:26.417 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.417 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.417 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:26.417 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.556836+0000", 2020-09-08T16:10:26.417 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.002253957 2020-09-08T16:10:26.418 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.418 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.418 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.418 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.559090+0000", 2020-09-08T16:10:26.418 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000158512 2020-09-08T16:10:26.419 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.419 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.419 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:26.419 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.559249+0000", 2020-09-08T16:10:26.419 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018337299999999999 2020-09-08T16:10:26.420 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.420 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.420 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:26.420 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.559432+0000", 2020-09-08T16:10:26.420 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0011040220000000001 2020-09-08T16:10:26.421 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.421 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.421 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:26.421 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.560536+0000", 2020-09-08T16:10:26.421 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20431337099999999 2020-09-08T16:10:26.422 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.422 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.422 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:26.422 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.764849+0000", 2020-09-08T16:10:26.422 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.572e-05 2020-09-08T16:10:26.422 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.423 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.423 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:26.423 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.764925+0000", 2020-09-08T16:10:26.423 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0986000000000003e-05 2020-09-08T16:10:26.423 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.424 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.424 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:26.424 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.764976+0000", 2020-09-08T16:10:26.424 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20822859299999999 2020-09-08T16:10:26.424 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.425 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.425 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.425 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.425 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.425 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:32350 8.a 8:55914443:::benchmark_data_smithi166_52302_object32349:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:10:26.426 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:05:19.572536+0000", 2020-09-08T16:10:26.426 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 306.74490678199999, 2020-09-08T16:10:26.426 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19268626999999999, 2020-09-08T16:10:26.426 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:26.427 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:26.427 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:26.427 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:26.427 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:26.427 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 32350 2020-09-08T16:10:26.428 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.428 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:26.428 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.428 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:26.428 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.572536+0000", 2020-09-08T16:10:26.429 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:26.429 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.429 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.429 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:26.429 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.572536+0000", 2020-09-08T16:10:26.430 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999528 2020-09-08T16:10:26.430 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.430 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.430 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:26.430 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.572489+0000", 2020-09-08T16:10:26.431 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014267199999999999 2020-09-08T16:10:26.431 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.431 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.431 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:26.431 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.572631+0000", 2020-09-08T16:10:26.431 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7342000000000001e-05 2020-09-08T16:10:26.432 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.432 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.432 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:26.432 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.572649+0000", 2020-09-08T16:10:26.432 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6455000000000001e-05 2020-09-08T16:10:26.433 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.433 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.433 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:26.433 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.572665+0000", 2020-09-08T16:10:26.433 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.8470999999999997e-05 2020-09-08T16:10:26.434 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.434 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.434 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.434 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.572714+0000", 2020-09-08T16:10:26.434 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012624199999999999 2020-09-08T16:10:26.435 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.435 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.435 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:26.435 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.572840+0000", 2020-09-08T16:10:26.435 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014192100000000001 2020-09-08T16:10:26.436 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.436 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.436 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:26.436 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.572982+0000", 2020-09-08T16:10:26.437 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.001536578 2020-09-08T16:10:26.437 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.437 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.437 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:26.437 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.574518+0000", 2020-09-08T16:10:26.438 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19050219900000001 2020-09-08T16:10:26.438 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.438 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.438 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:26.438 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.765021+0000", 2020-09-08T16:10:26.439 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7661000000000002e-05 2020-09-08T16:10:26.439 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.439 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.439 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:26.439 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.765078+0000", 2020-09-08T16:10:26.439 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014391299999999999 2020-09-08T16:10:26.440 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.440 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.440 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:26.440 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.765222+0000", 2020-09-08T16:10:26.440 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19268626999999999 2020-09-08T16:10:26.441 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.441 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.441 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.441 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.441 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.442 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:32375 8.a 8:5c43a6e8:::benchmark_data_smithi166_52302_object32374:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:10:26.442 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:05:19.587167+0000", 2020-09-08T16:10:26.442 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 306.73027557799998, 2020-09-08T16:10:26.442 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.179450152, 2020-09-08T16:10:26.442 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:26.443 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:26.443 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:26.443 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:26.443 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:26.443 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 32375 2020-09-08T16:10:26.444 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.444 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:26.444 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.444 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:26.444 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.587167+0000", 2020-09-08T16:10:26.445 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:26.445 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.445 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.445 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:26.445 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.587167+0000", 2020-09-08T16:10:26.446 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999771 2020-09-08T16:10:26.446 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.446 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.446 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:26.446 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.587144+0000", 2020-09-08T16:10:26.447 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000106396 2020-09-08T16:10:26.447 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.447 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.447 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:26.447 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.587251+0000", 2020-09-08T16:10:26.447 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.9853000000000001e-05 2020-09-08T16:10:26.448 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.448 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.448 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:26.448 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.587271+0000", 2020-09-08T16:10:26.448 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0108e-05 2020-09-08T16:10:26.449 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.449 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.449 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:26.449 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.587281+0000", 2020-09-08T16:10:26.449 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.2490000000000005e-05 2020-09-08T16:10:26.450 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.450 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.450 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.450 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.587363+0000", 2020-09-08T16:10:26.450 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000166553 2020-09-08T16:10:26.450 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.451 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.451 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:26.451 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.587530+0000", 2020-09-08T16:10:26.451 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019841699999999999 2020-09-08T16:10:26.451 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.452 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.452 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:26.452 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.587728+0000", 2020-09-08T16:10:26.452 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0019063350000000001 2020-09-08T16:10:26.452 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.453 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.453 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:26.453 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.589634+0000", 2020-09-08T16:10:26.453 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17683280300000001 2020-09-08T16:10:26.453 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.454 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.454 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:26.454 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.766467+0000", 2020-09-08T16:10:26.454 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.5863000000000001e-05 2020-09-08T16:10:26.454 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.454 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.455 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:26.455 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.766553+0000", 2020-09-08T16:10:26.455 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.4160999999999998e-05 2020-09-08T16:10:26.455 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.455 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.456 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:26.456 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.766617+0000", 2020-09-08T16:10:26.456 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.179450152 2020-09-08T16:10:26.456 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.456 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.457 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.457 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.457 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.457 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:32408 8.a 8:5b42b3ff:::benchmark_data_smithi166_52302_object32407:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:10:26.457 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:05:19.604347+0000", 2020-09-08T16:10:26.458 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 306.71309576300001, 2020-09-08T16:10:26.458 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16261424399999999, 2020-09-08T16:10:26.458 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:26.458 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:26.458 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:26.458 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:26.459 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:26.459 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 32408 2020-09-08T16:10:26.459 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.459 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:26.459 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.460 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:26.460 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.604347+0000", 2020-09-08T16:10:26.460 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:26.460 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.460 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.461 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:26.461 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.604347+0000", 2020-09-08T16:10:26.461 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.999959 2020-09-08T16:10:26.461 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.461 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.462 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:26.462 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.604306+0000", 2020-09-08T16:10:26.462 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000131715 2020-09-08T16:10:26.462 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.462 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.462 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:26.463 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.604438+0000", 2020-09-08T16:10:26.463 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5777000000000002e-05 2020-09-08T16:10:26.463 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.463 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.463 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:26.464 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.604454+0000", 2020-09-08T16:10:26.464 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.5399999999999996e-06 2020-09-08T16:10:26.464 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.464 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.464 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:26.465 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.604462+0000", 2020-09-08T16:10:26.465 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9379999999999998e-05 2020-09-08T16:10:26.465 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.465 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.465 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.466 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.604511+0000", 2020-09-08T16:10:26.466 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00027320600000000001 2020-09-08T16:10:26.466 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.466 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.466 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:26.467 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.604785+0000", 2020-09-08T16:10:26.467 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016966099999999999 2020-09-08T16:10:26.467 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.467 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.468 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:26.468 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.604954+0000", 2020-09-08T16:10:26.468 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.002296537 2020-09-08T16:10:26.468 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.468 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.469 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:26.469 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.607251+0000", 2020-09-08T16:10:26.469 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15941836700000001 2020-09-08T16:10:26.469 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.469 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.469 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:26.470 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.766669+0000", 2020-09-08T16:10:26.470 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.8498999999999996e-05 2020-09-08T16:10:26.470 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.470 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.470 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:26.471 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.766738+0000", 2020-09-08T16:10:26.471 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000223453 2020-09-08T16:10:26.471 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.471 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.471 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:26.472 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.766961+0000", 2020-09-08T16:10:26.472 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16261424399999999 2020-09-08T16:10:26.472 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.472 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.472 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.472 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.473 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.473 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:32409 8.a 8:556c7977:::benchmark_data_smithi166_52302_object32408:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:10:26.473 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:05:19.605258+0000", 2020-09-08T16:10:26.473 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 306.71218463399998, 2020-09-08T16:10:26.473 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.164134217, 2020-09-08T16:10:26.474 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:26.474 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:26.474 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:26.474 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:26.474 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:26.475 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 32409 2020-09-08T16:10:26.475 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.475 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:26.475 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.475 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:26.476 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.605258+0000", 2020-09-08T16:10:26.476 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:26.476 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.476 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.476 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:26.477 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.605258+0000", 2020-09-08T16:10:26.477 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999623 2020-09-08T16:10:26.477 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.477 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.477 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:26.478 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.605220+0000", 2020-09-08T16:10:26.478 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000116389 2020-09-08T16:10:26.478 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.478 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.478 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:26.478 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.605337+0000", 2020-09-08T16:10:26.479 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.9639999999999992e-06 2020-09-08T16:10:26.479 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.479 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.479 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:26.479 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.605346+0000", 2020-09-08T16:10:26.480 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.843e-06 2020-09-08T16:10:26.480 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.480 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.480 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:26.480 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.605350+0000", 2020-09-08T16:10:26.481 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00071869100000000003 2020-09-08T16:10:26.481 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.481 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.481 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.481 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.606069+0000", 2020-09-08T16:10:26.482 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016473 2020-09-08T16:10:26.482 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.482 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.482 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:26.482 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.606234+0000", 2020-09-08T16:10:26.482 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000179469 2020-09-08T16:10:26.483 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.483 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.483 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:26.483 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.606413+0000", 2020-09-08T16:10:26.483 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0010560159999999999 2020-09-08T16:10:26.484 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.484 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.484 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:26.484 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.607469+0000", 2020-09-08T16:10:26.484 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16175498999999999 2020-09-08T16:10:26.485 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.485 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.485 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:26.485 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.769224+0000", 2020-09-08T16:10:26.485 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.4753000000000006e-05 2020-09-08T16:10:26.486 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.486 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.486 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:26.486 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.769319+0000", 2020-09-08T16:10:26.486 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.3231000000000004e-05 2020-09-08T16:10:26.486 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.487 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.487 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:26.487 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:05:19.769392+0000", 2020-09-08T16:10:26.487 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.164134217 2020-09-08T16:10:26.487 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.488 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.488 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.488 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.488 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.488 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:108729 8.a 8:52517e3d:::benchmark_data_smithi166_52302_object108728:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e652)", 2020-09-08T16:10:26.489 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:06:09.161091+0000", 2020-09-08T16:10:26.489 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 257.15635122700002, 2020-09-08T16:10:26.489 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15582047399999999, 2020-09-08T16:10:26.489 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:26.489 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:26.490 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:26.490 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:26.490 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:26.490 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 108729 2020-09-08T16:10:26.490 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.491 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:26.491 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.491 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:26.492 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.161091+0000", 2020-09-08T16:10:26.492 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:26.492 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.492 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.492 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:26.493 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.161091+0000", 2020-09-08T16:10:26.493 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999652 2020-09-08T16:10:26.493 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.493 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.493 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:26.494 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.161057+0000", 2020-09-08T16:10:26.494 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013398199999999999 2020-09-08T16:10:26.494 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.494 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.494 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:26.494 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.161191+0000", 2020-09-08T16:10:26.495 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3056e-05 2020-09-08T16:10:26.495 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.495 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.495 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:26.495 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.161204+0000", 2020-09-08T16:10:26.496 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.0129999999999999e-06 2020-09-08T16:10:26.496 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.496 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.496 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:26.496 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.161210+0000", 2020-09-08T16:10:26.497 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0274e-05 2020-09-08T16:10:26.497 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.497 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.497 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.497 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.161260+0000", 2020-09-08T16:10:26.498 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000142416 2020-09-08T16:10:26.498 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.498 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.498 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:26.498 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.161402+0000", 2020-09-08T16:10:26.498 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014699799999999999 2020-09-08T16:10:26.499 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.499 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.499 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:26.499 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.161549+0000", 2020-09-08T16:10:26.499 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0074997629999999996 2020-09-08T16:10:26.500 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.500 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.500 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:26.500 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.169049+0000", 2020-09-08T16:10:26.500 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.147731165 2020-09-08T16:10:26.501 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.501 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.501 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:26.501 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.316780+0000", 2020-09-08T16:10:26.501 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7664000000000002e-05 2020-09-08T16:10:26.502 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.502 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.502 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:26.502 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.316838+0000", 2020-09-08T16:10:26.503 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.3980999999999995e-05 2020-09-08T16:10:26.503 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.503 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.503 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:26.503 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.316912+0000", 2020-09-08T16:10:26.504 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15582047399999999 2020-09-08T16:10:26.504 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.504 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.504 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.504 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.505 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.505 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:108730 8.a 8:5047d6b0:::benchmark_data_smithi166_52302_object108729:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e652)", 2020-09-08T16:10:26.505 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:06:09.161275+0000", 2020-09-08T16:10:26.505 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 257.156168132, 2020-09-08T16:10:26.505 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15595827500000001, 2020-09-08T16:10:26.506 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:26.506 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:26.506 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:26.506 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:26.506 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:26.506 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 108730 2020-09-08T16:10:26.507 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.507 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:26.507 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.507 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:26.507 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.161275+0000", 2020-09-08T16:10:26.508 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:26.508 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.508 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.508 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:26.508 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.161275+0000", 2020-09-08T16:10:26.509 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999657 2020-09-08T16:10:26.509 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.509 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.509 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:26.509 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.161240+0000", 2020-09-08T16:10:26.510 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000107995 2020-09-08T16:10:26.510 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.510 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.510 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:26.510 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.161348+0000", 2020-09-08T16:10:26.510 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1548e-05 2020-09-08T16:10:26.511 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.511 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.511 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:26.511 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.161360+0000", 2020-09-08T16:10:26.511 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.4519999999999999e-06 2020-09-08T16:10:26.512 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.512 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.512 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:26.512 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.161364+0000", 2020-09-08T16:10:26.512 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0037907850000000001 2020-09-08T16:10:26.513 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.513 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.513 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.513 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.165155+0000", 2020-09-08T16:10:26.513 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000160722 2020-09-08T16:10:26.514 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.514 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.514 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:26.514 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.165316+0000", 2020-09-08T16:10:26.515 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017494900000000001 2020-09-08T16:10:26.515 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.515 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.515 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:26.515 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.165491+0000", 2020-09-08T16:10:26.516 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15143132300000001 2020-09-08T16:10:26.516 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.516 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.516 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:26.516 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.316922+0000", 2020-09-08T16:10:26.517 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019769199999999999 2020-09-08T16:10:26.517 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.517 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.517 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:26.517 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.317120+0000", 2020-09-08T16:10:26.518 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.1363000000000002e-05 2020-09-08T16:10:26.518 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.518 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.518 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:26.518 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.317181+0000", 2020-09-08T16:10:26.519 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1672000000000003e-05 2020-09-08T16:10:26.519 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.519 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.519 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:26.519 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.317233+0000", 2020-09-08T16:10:26.519 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15595827500000001 2020-09-08T16:10:26.520 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.520 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.520 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.520 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.520 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.521 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:108740 8.b 8:dbfbced8:::benchmark_data_smithi166_52302_object108739:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e653)", 2020-09-08T16:10:26.521 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:06:09.169583+0000", 2020-09-08T16:10:26.521 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 257.14785918899997, 2020-09-08T16:10:26.521 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15706531500000001, 2020-09-08T16:10:26.521 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:26.522 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:26.522 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:26.522 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:26.522 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:26.522 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 108740 2020-09-08T16:10:26.523 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.523 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:26.523 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.523 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:26.523 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.169583+0000", 2020-09-08T16:10:26.524 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:26.524 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.524 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.524 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:26.524 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.169583+0000", 2020-09-08T16:10:26.524 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999576 2020-09-08T16:10:26.525 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.525 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.525 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:26.525 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.169541+0000", 2020-09-08T16:10:26.526 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012077400000000001 2020-09-08T16:10:26.526 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.526 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.526 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:26.527 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.169662+0000", 2020-09-08T16:10:26.527 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4994000000000001e-05 2020-09-08T16:10:26.527 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.527 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.527 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:26.527 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.169677+0000", 2020-09-08T16:10:26.528 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8919999999999997e-06 2020-09-08T16:10:26.528 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.528 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.528 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:26.528 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.169683+0000", 2020-09-08T16:10:26.529 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.0220000000000002e-05 2020-09-08T16:10:26.529 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.529 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.529 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.529 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.169753+0000", 2020-09-08T16:10:26.530 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.1079999999999999e-06 2020-09-08T16:10:26.530 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.530 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.530 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op must wait for map", 2020-09-08T16:10:26.530 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.169762+0000", 2020-09-08T16:10:26.531 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.147899951 2020-09-08T16:10:26.531 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.531 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.531 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.531 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.317662+0000", 2020-09-08T16:10:26.532 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000171266 2020-09-08T16:10:26.532 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.532 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.532 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:26.532 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.317833+0000", 2020-09-08T16:10:26.532 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000195052 2020-09-08T16:10:26.533 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.533 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.533 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T16:10:26.533 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.318028+0000", 2020-09-08T16:10:26.533 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0082736330000000007 2020-09-08T16:10:26.534 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.534 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.534 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:26.534 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.326302+0000", 2020-09-08T16:10:26.534 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00024011700000000001 2020-09-08T16:10:26.535 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.535 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.535 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:26.535 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.326542+0000", 2020-09-08T16:10:26.535 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.2830000000000007e-05 2020-09-08T16:10:26.536 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.536 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.536 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:26.536 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.326605+0000", 2020-09-08T16:10:26.536 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.4011000000000003e-05 2020-09-08T16:10:26.537 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.537 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.537 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:26.537 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.326649+0000", 2020-09-08T16:10:26.538 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15706531500000001 2020-09-08T16:10:26.538 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.538 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.538 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.538 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.539 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.539 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:108746 8.a 8:58d82738:::benchmark_data_smithi166_52302_object108745:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e653)", 2020-09-08T16:10:26.539 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:06:09.170161+0000", 2020-09-08T16:10:26.539 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 257.14728182800002, 2020-09-08T16:10:26.539 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.156975637, 2020-09-08T16:10:26.540 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:26.540 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:26.540 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:26.540 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:26.540 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:26.540 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 108746 2020-09-08T16:10:26.541 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.541 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:26.541 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.541 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:26.541 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.170161+0000", 2020-09-08T16:10:26.542 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:26.542 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.542 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.542 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:26.542 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.170161+0000", 2020-09-08T16:10:26.543 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999781 2020-09-08T16:10:26.543 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.543 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.543 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:26.543 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.170139+0000", 2020-09-08T16:10:26.544 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.9611000000000006e-05 2020-09-08T16:10:26.544 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.544 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.544 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:26.544 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.170239+0000", 2020-09-08T16:10:26.544 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.1550000000000007e-06 2020-09-08T16:10:26.545 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.545 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.545 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:26.545 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.170247+0000", 2020-09-08T16:10:26.545 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.2830000000000003e-06 2020-09-08T16:10:26.546 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.546 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.546 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:26.546 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.170251+0000", 2020-09-08T16:10:26.546 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14726167700000001 2020-09-08T16:10:26.547 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.547 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.547 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.547 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.317513+0000", 2020-09-08T16:10:26.547 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3542e-05 2020-09-08T16:10:26.548 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.548 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.548 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op must wait for map", 2020-09-08T16:10:26.548 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.317526+0000", 2020-09-08T16:10:26.548 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00032681600000000001 2020-09-08T16:10:26.548 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.549 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.549 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.549 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.317853+0000", 2020-09-08T16:10:26.549 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000237366 2020-09-08T16:10:26.549 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.550 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.550 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:26.550 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.318091+0000", 2020-09-08T16:10:26.550 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000275505 2020-09-08T16:10:26.550 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.551 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.551 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:26.551 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.318366+0000", 2020-09-08T16:10:26.551 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0081034350000000008 2020-09-08T16:10:26.551 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.552 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.552 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:26.552 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.326469+0000", 2020-09-08T16:10:26.552 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00056577799999999996 2020-09-08T16:10:26.552 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.553 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.553 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:26.553 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.327035+0000", 2020-09-08T16:10:26.553 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7908000000000002e-05 2020-09-08T16:10:26.553 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.554 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.554 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:26.554 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.327093+0000", 2020-09-08T16:10:26.554 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.3306000000000002e-05 2020-09-08T16:10:26.554 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.554 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.555 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:26.555 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.327136+0000", 2020-09-08T16:10:26.555 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.156975637 2020-09-08T16:10:26.555 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.555 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.556 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.556 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.556 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.556 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:177561 8.a 8:5d547469:::benchmark_data_smithi166_52302_object46640:head [delete] snapc 0=[] ondisk+write+known_if_redirected e670)", 2020-09-08T16:10:26.556 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:07:27.188573+0000", 2020-09-08T16:10:26.557 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 179.12886939800001, 2020-09-08T16:10:26.557 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16049644499999999, 2020-09-08T16:10:26.557 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:26.557 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:26.557 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:26.558 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:26.558 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:26.558 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 177561 2020-09-08T16:10:26.558 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.558 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:26.558 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.559 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:26.559 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:27.188573+0000", 2020-09-08T16:10:26.559 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:26.559 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.559 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.560 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:26.560 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:27.188573+0000", 2020-09-08T16:10:26.560 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999704 2020-09-08T16:10:26.560 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.560 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.561 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:26.561 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:27.188544+0000", 2020-09-08T16:10:26.561 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.127035227 2020-09-08T16:10:26.561 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.561 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.562 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:26.562 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:27.315579+0000", 2020-09-08T16:10:26.562 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7674000000000001e-05 2020-09-08T16:10:26.562 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.562 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.562 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:26.563 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:27.315596+0000", 2020-09-08T16:10:26.563 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.0579999999999998e-06 2020-09-08T16:10:26.563 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.563 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.563 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:26.564 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:27.315605+0000", 2020-09-08T16:10:26.564 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.018809277999999999 2020-09-08T16:10:26.564 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.564 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.564 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.565 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:27.334414+0000", 2020-09-08T16:10:26.565 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000145573 2020-09-08T16:10:26.565 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.565 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.565 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:26.566 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:27.334559+0000", 2020-09-08T16:10:26.566 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014760399999999999 2020-09-08T16:10:26.566 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.566 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.566 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:26.567 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:27.334707+0000", 2020-09-08T16:10:26.567 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0059943110000000004 2020-09-08T16:10:26.567 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.567 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.567 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:26.567 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:27.340701+0000", 2020-09-08T16:10:26.568 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0079771559999999991 2020-09-08T16:10:26.568 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.568 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.569 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:26.569 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:27.348678+0000", 2020-09-08T16:10:26.569 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00026738 2020-09-08T16:10:26.569 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.569 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.569 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:26.570 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:27.348946+0000", 2020-09-08T16:10:26.570 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012389599999999999 2020-09-08T16:10:26.570 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.570 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.570 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:26.571 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:27.349070+0000", 2020-09-08T16:10:26.571 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16049644499999999 2020-09-08T16:10:26.571 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.571 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.571 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.572 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.572 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.572 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:252156 8.a 8:550972db:::benchmark_data_smithi166_52302_object121235:head [delete] snapc 0=[] ondisk+write+known_if_redirected e692)", 2020-09-08T16:10:26.572 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:08:52.775520+0000", 2020-09-08T16:10:26.572 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 93.541922846000006, 2020-09-08T16:10:26.573 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16007463499999999, 2020-09-08T16:10:26.573 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:26.573 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:26.573 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:26.573 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:26.573 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:26.574 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 252156 2020-09-08T16:10:26.574 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.574 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:26.574 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.574 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:26.575 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.775520+0000", 2020-09-08T16:10:26.575 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:26.575 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.575 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.575 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:26.576 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.775520+0000", 2020-09-08T16:10:26.576 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999781 2020-09-08T16:10:26.576 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.576 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.576 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:26.577 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.775498+0000", 2020-09-08T16:10:26.577 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9842000000000002e-05 2020-09-08T16:10:26.577 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.577 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.577 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:26.577 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.775548+0000", 2020-09-08T16:10:26.578 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.3079999999999997e-06 2020-09-08T16:10:26.578 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.578 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.578 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:26.578 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.775557+0000", 2020-09-08T16:10:26.579 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.7679999999999999e-06 2020-09-08T16:10:26.579 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.579 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.579 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:26.579 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.775562+0000", 2020-09-08T16:10:26.580 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.1090000000000001e-05 2020-09-08T16:10:26.580 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.580 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.580 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.580 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.775603+0000", 2020-09-08T16:10:26.581 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.4860999999999999e-05 2020-09-08T16:10:26.581 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.581 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.581 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:26.581 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.775698+0000", 2020-09-08T16:10:26.581 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012514199999999999 2020-09-08T16:10:26.582 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.582 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.582 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:26.582 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.775823+0000", 2020-09-08T16:10:26.582 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0066517859999999998 2020-09-08T16:10:26.583 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.583 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.583 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:26.583 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.782475+0000", 2020-09-08T16:10:26.583 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.153042493 2020-09-08T16:10:26.584 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.584 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.584 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:26.584 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.935517+0000", 2020-09-08T16:10:26.584 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.5324999999999997e-05 2020-09-08T16:10:26.585 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.585 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.585 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:26.585 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.935562+0000", 2020-09-08T16:10:26.585 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.2098000000000003e-05 2020-09-08T16:10:26.585 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.586 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.586 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:26.586 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.935594+0000", 2020-09-08T16:10:26.586 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16007463499999999 2020-09-08T16:10:26.586 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.587 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.587 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.587 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.587 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.587 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:252165 8.a 8:57783bc0:::benchmark_data_smithi166_52302_object121244:head [delete] snapc 0=[] ondisk+write+known_if_redirected e692)", 2020-09-08T16:10:26.588 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:08:52.780010+0000", 2020-09-08T16:10:26.588 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 93.537432391999999, 2020-09-08T16:10:26.588 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18186845099999999, 2020-09-08T16:10:26.588 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:26.588 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:26.589 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:26.589 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:26.589 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:26.589 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 252165 2020-09-08T16:10:26.589 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.590 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:26.590 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.590 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:26.590 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.780010+0000", 2020-09-08T16:10:26.590 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:26.591 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.591 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.591 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:26.592 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.780010+0000", 2020-09-08T16:10:26.592 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999623 2020-09-08T16:10:26.592 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.592 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.592 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:26.592 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.779973+0000", 2020-09-08T16:10:26.593 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5191000000000007e-05 2020-09-08T16:10:26.593 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.593 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.593 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:26.593 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.780038+0000", 2020-09-08T16:10:26.594 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.5619999999999993e-06 2020-09-08T16:10:26.594 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.594 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.594 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:26.594 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.780047+0000", 2020-09-08T16:10:26.595 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.5480000000000003e-06 2020-09-08T16:10:26.595 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.595 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.595 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:26.595 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.780052+0000", 2020-09-08T16:10:26.596 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0036206720000000001 2020-09-08T16:10:26.596 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.596 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.596 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.596 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.783673+0000", 2020-09-08T16:10:26.597 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016280600000000001 2020-09-08T16:10:26.597 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.597 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.597 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:26.597 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.783835+0000", 2020-09-08T16:10:26.598 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000200268 2020-09-08T16:10:26.598 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.598 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.598 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:26.598 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.784036+0000", 2020-09-08T16:10:26.598 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.019263826000000001 2020-09-08T16:10:26.599 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.599 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.599 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:26.599 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.803300+0000", 2020-09-08T16:10:26.599 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15841685699999999 2020-09-08T16:10:26.600 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.600 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.600 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:26.600 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.961716+0000", 2020-09-08T16:10:26.600 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2799000000000004e-05 2020-09-08T16:10:26.601 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.601 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.601 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:26.601 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.961809+0000", 2020-09-08T16:10:26.601 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.9516000000000003e-05 2020-09-08T16:10:26.602 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.602 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.602 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:26.602 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.961879+0000", 2020-09-08T16:10:26.602 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18186845099999999 2020-09-08T16:10:26.603 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.603 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.603 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.603 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.603 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.604 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:252173 8.a 8:53282a7f:::benchmark_data_smithi166_52302_object121252:head [delete] snapc 0=[] ondisk+write+known_if_redirected e692)", 2020-09-08T16:10:26.604 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:08:52.791231+0000", 2020-09-08T16:10:26.604 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 93.526211763000006, 2020-09-08T16:10:26.604 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17097801700000001, 2020-09-08T16:10:26.605 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:26.605 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:26.605 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:26.605 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:26.605 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:26.605 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 252173 2020-09-08T16:10:26.606 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.606 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:26.606 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.606 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:26.606 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.791231+0000", 2020-09-08T16:10:26.607 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:26.607 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.607 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.607 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:26.607 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.791231+0000", 2020-09-08T16:10:26.608 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999633 2020-09-08T16:10:26.608 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.608 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.608 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:26.608 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.791194+0000", 2020-09-08T16:10:26.609 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.5025000000000007e-05 2020-09-08T16:10:26.609 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.609 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.609 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:26.609 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.791279+0000", 2020-09-08T16:10:26.610 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5620999999999998e-05 2020-09-08T16:10:26.610 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.610 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.610 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:26.610 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.791295+0000", 2020-09-08T16:10:26.610 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.9809999999999997e-06 2020-09-08T16:10:26.611 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.611 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.611 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:26.611 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.791302+0000", 2020-09-08T16:10:26.611 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0072073759999999997 2020-09-08T16:10:26.612 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.612 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.612 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.612 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.798509+0000", 2020-09-08T16:10:26.612 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000169352 2020-09-08T16:10:26.613 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.613 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.613 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:26.613 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.798678+0000", 2020-09-08T16:10:26.613 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017560000000000001 2020-09-08T16:10:26.614 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.614 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.614 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:26.614 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.798854+0000", 2020-09-08T16:10:26.614 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.009735077 2020-09-08T16:10:26.615 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.615 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.615 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:26.615 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.808589+0000", 2020-09-08T16:10:26.615 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.153476424 2020-09-08T16:10:26.616 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.616 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.616 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:26.616 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.962065+0000", 2020-09-08T16:10:26.616 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7795999999999999e-05 2020-09-08T16:10:26.617 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.617 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.617 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:26.617 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.962143+0000", 2020-09-08T16:10:26.617 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5623999999999994e-05 2020-09-08T16:10:26.618 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.618 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.618 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:26.618 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.962209+0000", 2020-09-08T16:10:26.618 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17097801700000001 2020-09-08T16:10:26.619 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.619 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.619 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.619 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.619 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.619 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:252177 8.a 8:5ffbed70:::benchmark_data_smithi166_52302_object121256:head [delete] snapc 0=[] ondisk+write+known_if_redirected e692)", 2020-09-08T16:10:26.620 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:08:52.796310+0000", 2020-09-08T16:10:26.620 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 93.521132304999995, 2020-09-08T16:10:26.620 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16954808699999999, 2020-09-08T16:10:26.620 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:26.620 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:26.621 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:26.621 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:26.621 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:26.621 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 252177 2020-09-08T16:10:26.621 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.622 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:26.622 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.622 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:26.622 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.796310+0000", 2020-09-08T16:10:26.622 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:26.623 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.623 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.623 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:26.623 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.796310+0000", 2020-09-08T16:10:26.623 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999599 2020-09-08T16:10:26.623 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.624 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.624 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:26.624 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.796270+0000", 2020-09-08T16:10:26.626 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8764000000000006e-05 2020-09-08T16:10:26.626 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.626 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.627 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:26.627 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.796359+0000", 2020-09-08T16:10:26.627 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.8121000000000001e-05 2020-09-08T16:10:26.627 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.628 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.628 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:26.628 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.796377+0000", 2020-09-08T16:10:26.628 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.4910000000000002e-06 2020-09-08T16:10:26.628 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.629 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.629 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:26.629 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.796384+0000", 2020-09-08T16:10:26.629 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0069549440000000002 2020-09-08T16:10:26.629 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.630 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.630 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.630 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.803339+0000", 2020-09-08T16:10:26.630 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00031103700000000001 2020-09-08T16:10:26.630 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.631 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.631 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:26.631 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.803650+0000", 2020-09-08T16:10:26.631 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017175900000000001 2020-09-08T16:10:26.631 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.632 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.632 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:26.632 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.803822+0000", 2020-09-08T16:10:26.632 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0049319020000000002 2020-09-08T16:10:26.632 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.633 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.633 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:26.633 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.808753+0000", 2020-09-08T16:10:26.633 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.156980331 2020-09-08T16:10:26.633 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.633 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.634 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:26.634 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.965734+0000", 2020-09-08T16:10:26.634 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.8969999999999999e-05 2020-09-08T16:10:26.635 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.635 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.635 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:26.635 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.965803+0000", 2020-09-08T16:10:26.635 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.5682e-05 2020-09-08T16:10:26.636 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.636 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.636 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:26.636 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.965858+0000", 2020-09-08T16:10:26.636 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16954808699999999 2020-09-08T16:10:26.637 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.637 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.637 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.637 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.637 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.638 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:252197 8.a 8:5af1961d:::benchmark_data_smithi166_52302_object121276:head [delete] snapc 0=[] ondisk+write+known_if_redirected e692)", 2020-09-08T16:10:26.638 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:08:52.822843+0000", 2020-09-08T16:10:26.638 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 93.494599312999995, 2020-09-08T16:10:26.638 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.149919469, 2020-09-08T16:10:26.639 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:26.639 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:26.639 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:26.639 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:26.639 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:26.640 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 252197 2020-09-08T16:10:26.640 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.640 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:26.640 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.640 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:26.641 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.822843+0000", 2020-09-08T16:10:26.641 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:26.641 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.641 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.641 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:26.641 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.822843+0000", 2020-09-08T16:10:26.642 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999585 2020-09-08T16:10:26.642 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.642 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.642 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:26.642 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.822802+0000", 2020-09-08T16:10:26.643 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2810000000000001e-05 2020-09-08T16:10:26.643 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.643 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.643 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:26.643 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.822895+0000", 2020-09-08T16:10:26.644 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4313e-05 2020-09-08T16:10:26.645 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.646 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.646 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:26.646 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.822909+0000", 2020-09-08T16:10:26.647 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.8279999999999997e-06 2020-09-08T16:10:26.647 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.647 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.647 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:26.647 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.822916+0000", 2020-09-08T16:10:26.648 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.5438000000000003e-05 2020-09-08T16:10:26.648 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.648 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.648 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:26.648 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.823011+0000", 2020-09-08T16:10:26.648 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019171800000000001 2020-09-08T16:10:26.649 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.649 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.649 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:26.649 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.823203+0000", 2020-09-08T16:10:26.649 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00022524099999999999 2020-09-08T16:10:26.650 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.650 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.650 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:26.650 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.823428+0000", 2020-09-08T16:10:26.650 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0047034140000000004 2020-09-08T16:10:26.651 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.651 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.651 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:26.651 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.828132+0000", 2020-09-08T16:10:26.651 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14449569100000001 2020-09-08T16:10:26.652 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.652 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.652 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:26.652 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.972627+0000", 2020-09-08T16:10:26.652 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.8366000000000002e-05 2020-09-08T16:10:26.653 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.653 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.653 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:26.653 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.972706+0000", 2020-09-08T16:10:26.653 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.6978000000000002e-05 2020-09-08T16:10:26.653 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:26.654 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:26.654 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:26.654 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:52.972763+0000", 2020-09-08T16:10:26.654 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.149919469 2020-09-08T16:10:26.654 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.655 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.656 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.656 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:26.657 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:26.657 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:10:26.657 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:10:26.658 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:26 smithi166 bash[9446]: cluster 2020-09-08T16:10:25.197732+0000 mon.a (mon.0) 2258 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:10:26.659 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:26 smithi166 bash[9446]: cluster 2020-09-08T16:10:25.197770+0000 mon.a (mon.0) 2259 : cluster [INF] Cluster is now healthy 2020-09-08T16:10:26.659 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:26 smithi166 bash[9446]: audit 2020-09-08T16:10:25.206918+0000 mon.a (mon.0) 2260 : audit [INF] from='client.? v1:172.21.15.104:0/1140692521' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:10:26.660 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:26 smithi166 bash[9446]: cluster 2020-09-08T16:10:25.206982+0000 mon.a (mon.0) 2261 : cluster [DBG] osdmap e723: 12 total, 12 up, 12 in 2020-09-08T16:10:26.660 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:10:26 smithi166 bash[14254]: debug 2020-09-08T16:10:26.364+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:26.660 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:10:26 smithi166 bash[18960]: debug 2020-09-08T16:10:26.144+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:26.662 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:26 smithi055 bash[9296]: cluster 2020-09-08T16:10:25.197732+0000 mon.a (mon.0) 2258 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:10:26.662 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:26 smithi055 bash[9296]: cluster 2020-09-08T16:10:25.197770+0000 mon.a (mon.0) 2259 : cluster [INF] Cluster is now healthy 2020-09-08T16:10:26.662 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:26 smithi055 bash[9296]: audit 2020-09-08T16:10:25.206918+0000 mon.a (mon.0) 2260 : audit [INF] from='client.? v1:172.21.15.104:0/1140692521' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:10:26.663 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:26 smithi055 bash[9296]: cluster 2020-09-08T16:10:25.206982+0000 mon.a (mon.0) 2261 : cluster [DBG] osdmap e723: 12 total, 12 up, 12 in 2020-09-08T16:10:26.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:26 smithi104 bash[9203]: cluster 2020-09-08T16:10:25.197732+0000 mon.a (mon.0) 2258 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:10:26.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:26 smithi104 bash[9203]: cluster 2020-09-08T16:10:25.197770+0000 mon.a (mon.0) 2259 : cluster [INF] Cluster is now healthy 2020-09-08T16:10:26.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:26 smithi104 bash[9203]: audit 2020-09-08T16:10:25.206918+0000 mon.a (mon.0) 2260 : audit [INF] from='client.? v1:172.21.15.104:0/1140692521' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:10:26.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:26 smithi104 bash[9203]: cluster 2020-09-08T16:10:25.206982+0000 mon.a (mon.0) 2261 : cluster [DBG] osdmap e723: 12 total, 12 up, 12 in 2020-09-08T16:10:26.762 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:26.791 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 55 16 103984 103968 118.131 72 0.0130926 0.00846375 2020-09-08T16:10:27.468 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:27.469 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:27 smithi166 bash[9446]: cluster 2020-09-08T16:10:26.123381+0000 mgr.x (mgr.24118) 1293 : cluster [DBG] pgmap v1932: 17 pgs: 17 active+clean; 6.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 MiB/s wr, 977 op/s 2020-09-08T16:10:27.469 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:10:27 smithi166 bash[18960]: debug 2020-09-08T16:10:27.468+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:27.574 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:27.586 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:27 smithi104 bash[9203]: cluster 2020-09-08T16:10:26.123381+0000 mgr.x (mgr.24118) 1293 : cluster [DBG] pgmap v1932: 17 pgs: 17 active+clean; 6.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 MiB/s wr, 977 op/s 2020-09-08T16:10:27.607 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_ops_in_flight 2020-09-08T16:10:27.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:27 smithi055 bash[9296]: cluster 2020-09-08T16:10:26.123381+0000 mgr.x (mgr.24118) 1293 : cluster [DBG] pgmap v1932: 17 pgs: 17 active+clean; 6.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 MiB/s wr, 977 op/s 2020-09-08T16:10:27.758 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:27.774 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:10:27.794 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 56 15 105324 105309 117.518 83.8125 0.0173586 0.008507 2020-09-08T16:10:27.861 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:10:27.882 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:10:27.913 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:10:27.925 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:10:27 smithi104 bash[22305]: debug 2020-09-08T16:10:27.757+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:27.947 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:10:27.977 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:10:28.009 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:10:28.050 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:10:28.078 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:10:28.120 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:10:28.148 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:10:28.175 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:10:28.208 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:10:28.214 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:28 smithi166 bash[9446]: audit 2020-09-08T16:10:27.861107+0000 mon.a (mon.0) 2262 : audit [DBG] from='client.? v1:172.21.15.104:0/3321664312' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:10:28.236 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:10:28.239 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:10:28.239 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:10:28.239 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:10:28.239 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:10:28.239 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:10:28.240 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:10:28.240 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:10:28.240 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:10:28.240 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:10:28.240 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:10:28.241 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:10:28.241 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:10:28.664 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:28 smithi055 bash[9296]: audit 2020-09-08T16:10:27.861107+0000 mon.a (mon.0) 2262 : audit [DBG] from='client.? v1:172.21.15.104:0/3321664312' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:10:28.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:28 smithi104 bash[9203]: audit 2020-09-08T16:10:27.861107+0000 mon.a (mon.0) 2262 : audit [DBG] from='client.? v1:172.21.15.104:0/3321664312' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:10:28.702 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:10:28.731 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T16:10:28.791 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 57 16 106581 106565 116.833 78.5 0.00525088 0.00855616 2020-09-08T16:10:28.804 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:10:28.881 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:10:29.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:10:28 smithi166 bash[16593]: debug 2020-09-08T16:10:28.684+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:29.379 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:10:29.483 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:10:29.491 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:29 smithi104 bash[9203]: cluster 2020-09-08T16:10:28.123848+0000 mgr.x (mgr.24118) 1294 : cluster [DBG] pgmap v1933: 17 pgs: 17 active+clean; 6.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.48k op/s 2020-09-08T16:10:29.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:29 smithi166 bash[9446]: cluster 2020-09-08T16:10:28.123848+0000 mgr.x (mgr.24118) 1294 : cluster [DBG] pgmap v1933: 17 pgs: 17 active+clean; 6.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.48k op/s 2020-09-08T16:10:29.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:10:29 smithi166 bash[14254]: debug 2020-09-08T16:10:29.376+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:29.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:29 smithi055 bash[9296]: cluster 2020-09-08T16:10:28.123848+0000 mgr.x (mgr.24118) 1294 : cluster [DBG] pgmap v1933: 17 pgs: 17 active+clean; 6.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s wr, 1.48k op/s 2020-09-08T16:10:29.791 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 58 16 107675 107659 115.998 68.375 0.0111134 0.00861775 2020-09-08T16:10:29.850 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:10:29.954 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:30.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:10:29 smithi104 bash[17588]: debug 2020-09-08T16:10:29.849+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:30.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:30 smithi166 bash[9446]: cluster 2020-09-08T16:10:28.739316+0000 osd.8 (osd.8) 103 : cluster [DBG] 9.b deep-scrub starts 2020-09-08T16:10:30.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:30 smithi166 bash[9446]: cluster 2020-09-08T16:10:29.667415+0000 osd.5 (osd.5) 121 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:30.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:30 smithi166 bash[9446]: cluster 2020-09-08T16:10:29.667481+0000 osd.5 (osd.5) 122 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:30.555 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:30 smithi104 bash[9203]: cluster 2020-09-08T16:10:28.739316+0000 osd.8 (osd.8) 103 : cluster [DBG] 9.b deep-scrub starts 2020-09-08T16:10:30.555 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:30 smithi104 bash[9203]: cluster 2020-09-08T16:10:29.667415+0000 osd.5 (osd.5) 121 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:30.555 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:30 smithi104 bash[9203]: cluster 2020-09-08T16:10:29.667481+0000 osd.5 (osd.5) 122 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:30.556 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:30.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:30 smithi055 bash[9296]: cluster 2020-09-08T16:10:28.739316+0000 osd.8 (osd.8) 103 : cluster [DBG] 9.b deep-scrub starts 2020-09-08T16:10:30.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:30 smithi055 bash[9296]: cluster 2020-09-08T16:10:29.667415+0000 osd.5 (osd.5) 121 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:30.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:30 smithi055 bash[9296]: cluster 2020-09-08T16:10:29.667481+0000 osd.5 (osd.5) 122 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:30.660 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:30.791 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 59 16 108651 108635 115.066 61 0.00979103 0.00868674 2020-09-08T16:10:30.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:10:30 smithi104 bash[22305]: debug 2020-09-08T16:10:30.557+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:31.502 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:31 smithi055 bash[9296]: cluster 2020-09-08T16:10:30.124338+0000 mgr.x (mgr.24118) 1295 : cluster [DBG] pgmap v1934: 17 pgs: 17 active+clean; 6.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s wr, 1.30k op/s 2020-09-08T16:10:31.503 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:31 smithi055 bash[9296]: cluster 2020-09-08T16:10:30.221589+0000 mon.a (mon.0) 2263 : cluster [DBG] osdmap e724: 12 total, 12 up, 12 in 2020-09-08T16:10:31.503 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:10:31 smithi055 bash[11956]: debug 2020-09-08T16:10:31.502+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:31.504 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:31.609 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:10:31.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:31 smithi166 bash[9446]: cluster 2020-09-08T16:10:30.124338+0000 mgr.x (mgr.24118) 1295 : cluster [DBG] pgmap v1934: 17 pgs: 17 active+clean; 6.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s wr, 1.30k op/s 2020-09-08T16:10:31.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:31 smithi166 bash[9446]: cluster 2020-09-08T16:10:30.221589+0000 mon.a (mon.0) 2263 : cluster [DBG] osdmap e724: 12 total, 12 up, 12 in 2020-09-08T16:10:31.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:31 smithi104 bash[9203]: cluster 2020-09-08T16:10:30.124338+0000 mgr.x (mgr.24118) 1295 : cluster [DBG] pgmap v1934: 17 pgs: 17 active+clean; 6.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s wr, 1.30k op/s 2020-09-08T16:10:31.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:31 smithi104 bash[9203]: cluster 2020-09-08T16:10:30.221589+0000 mon.a (mon.0) 2263 : cluster [DBG] osdmap e724: 12 total, 12 up, 12 in 2020-09-08T16:10:31.792 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 16:10:31.796674 min lat: 0.00165771 max lat: 0.375975 avg lat: 0.00875088 2020-09-08T16:10:31.792 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T16:10:31.792 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 60 15 109698 109683 114.239 65.5 0.0700995 0.00875088 2020-09-08T16:10:32.024 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:10:32.128 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:32.351 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:32.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:32 smithi166 bash[9446]: cluster 2020-09-08T16:10:31.942007+0000 osd.10 (osd.10) 105 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:32.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:32 smithi166 bash[9446]: cluster 2020-09-08T16:10:31.942072+0000 osd.10 (osd.10) 106 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:32.367 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:10:32 smithi166 bash[16593]: debug 2020-09-08T16:10:32.024+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:32.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:32 smithi104 bash[9203]: cluster 2020-09-08T16:10:31.942007+0000 osd.10 (osd.10) 105 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:32.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:32 smithi104 bash[9203]: cluster 2020-09-08T16:10:31.942072+0000 osd.10 (osd.10) 106 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:32.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:10:32 smithi104 bash[15273]: debug 2020-09-08T16:10:32.353+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:32.455 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:32.460 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:10:32.579 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:10:32 smithi055 bash[11956]: debug 2020-09-08T16:10:32.574+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:32.580 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:32 smithi055 bash[9296]: cluster 2020-09-08T16:10:31.942007+0000 osd.10 (osd.10) 105 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:32.580 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:32 smithi055 bash[9296]: cluster 2020-09-08T16:10:31.942072+0000 osd.10 (osd.10) 106 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:32.580 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:32.610 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:10:32.682 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:10:32.792 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 61 16 110710 110694 113.402 63.1875 0.00495627 0.00880771 2020-09-08T16:10:33.292 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:10:33.396 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:10:33.438 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T16:10:33.439 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:10:33.533 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:33 smithi166 bash[9446]: cluster 2020-09-08T16:10:32.124776+0000 mgr.x (mgr.24118) 1296 : cluster [DBG] pgmap v1936: 17 pgs: 17 active+clean; 6.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s wr, 900 op/s 2020-09-08T16:10:33.534 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:33 smithi166 bash[9446]: cluster 2020-09-08T16:10:32.249162+0000 mon.a (mon.0) 2264 : cluster [DBG] osdmap e725: 12 total, 12 up, 12 in 2020-09-08T16:10:33.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:33 smithi055 bash[9296]: cluster 2020-09-08T16:10:32.124776+0000 mgr.x (mgr.24118) 1296 : cluster [DBG] pgmap v1936: 17 pgs: 17 active+clean; 6.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s wr, 900 op/s 2020-09-08T16:10:33.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:33 smithi055 bash[9296]: cluster 2020-09-08T16:10:32.249162+0000 mon.a (mon.0) 2264 : cluster [DBG] osdmap e725: 12 total, 12 up, 12 in 2020-09-08T16:10:33.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:10:33 smithi104 bash[17588]: debug 2020-09-08T16:10:33.289+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:33.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:33 smithi104 bash[9203]: cluster 2020-09-08T16:10:32.124776+0000 mgr.x (mgr.24118) 1296 : cluster [DBG] pgmap v1936: 17 pgs: 17 active+clean; 6.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s wr, 900 op/s 2020-09-08T16:10:33.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:33 smithi104 bash[9203]: cluster 2020-09-08T16:10:32.249162+0000 mon.a (mon.0) 2264 : cluster [DBG] osdmap e725: 12 total, 12 up, 12 in 2020-09-08T16:10:33.792 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 62 15 111607 111592 112.479 56.125 0.0181035 0.0088873 2020-09-08T16:10:33.966 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:10:34.071 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:34.425 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:10:33 smithi104 bash[17588]: debug 2020-09-08T16:10:33.965+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:34.571 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:34.674 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:34.686 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_blocked_ops 2020-09-08T16:10:34.796 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 63 16 112646 112630 111.723 64.875 0.0159657 0.00894737 2020-09-08T16:10:34.797 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:34.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:10:34 smithi166 bash[18960]: debug 2020-09-08T16:10:34.568+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:34.868 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:10:34 smithi166 bash[18960]: debug 2020-09-08T16:10:34.796+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:34.892 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:10:34.901 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:10:35.046 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:10:35.154 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:35.248 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:10:35 smithi055 bash[16646]: debug 2020-09-08T16:10:35.046+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:35.250 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:35 smithi055 bash[9296]: cluster 2020-09-08T16:10:34.125256+0000 mgr.x (mgr.24118) 1297 : cluster [DBG] pgmap v1938: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s wr, 1.39k op/s 2020-09-08T16:10:35.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:35 smithi104 bash[9203]: cluster 2020-09-08T16:10:34.125256+0000 mgr.x (mgr.24118) 1297 : cluster [DBG] pgmap v1938: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s wr, 1.39k op/s 2020-09-08T16:10:35.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:35 smithi166 bash[9446]: cluster 2020-09-08T16:10:34.125256+0000 mgr.x (mgr.24118) 1297 : cluster [DBG] pgmap v1938: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 6.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s wr, 1.39k op/s 2020-09-08T16:10:35.738 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:35.792 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 64 16 113623 113607 110.931 61.0625 0.0193822 0.00901182 2020-09-08T16:10:35.842 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:10:36.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:10:35 smithi166 bash[18960]: debug 2020-09-08T16:10:35.740+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:36.260 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T16:10:36.518 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:36 smithi104 bash[9203]: cluster 2020-09-08T16:10:34.683986+0000 osd.3 (osd.3) 125 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:36.519 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:36 smithi104 bash[9203]: cluster 2020-09-08T16:10:34.684092+0000 osd.3 (osd.3) 126 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:36.519 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:36 smithi104 bash[9203]: cluster 2020-09-08T16:10:35.245345+0000 mon.a (mon.0) 2265 : cluster [DBG] osdmap e726: 12 total, 12 up, 12 in 2020-09-08T16:10:36.519 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:36 smithi104 bash[9203]: audit 2020-09-08T16:10:35.303353+0000 mon.c (mon.2) 91 : audit [INF] from='client.? v1:172.21.15.104:0/3127317535' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:10:36.519 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:36 smithi104 bash[9203]: audit 2020-09-08T16:10:35.305839+0000 mon.a (mon.0) 2266 : audit [INF] from='client.36623 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:10:36.519 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:36 smithi104 bash[9203]: cluster 2020-09-08T16:10:35.791732+0000 osd.5 (osd.5) 123 : cluster [DBG] 9.f scrub starts 2020-09-08T16:10:36.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:36 smithi166 bash[9446]: cluster 2020-09-08T16:10:34.683986+0000 osd.3 (osd.3) 125 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:36.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:36 smithi166 bash[9446]: cluster 2020-09-08T16:10:34.684092+0000 osd.3 (osd.3) 126 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:36.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:36 smithi166 bash[9446]: cluster 2020-09-08T16:10:35.245345+0000 mon.a (mon.0) 2265 : cluster [DBG] osdmap e726: 12 total, 12 up, 12 in 2020-09-08T16:10:36.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:36 smithi166 bash[9446]: audit 2020-09-08T16:10:35.303353+0000 mon.c (mon.2) 91 : audit [INF] from='client.? v1:172.21.15.104:0/3127317535' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:10:36.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:36 smithi166 bash[9446]: audit 2020-09-08T16:10:35.305839+0000 mon.a (mon.0) 2266 : audit [INF] from='client.36623 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:10:36.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:36 smithi166 bash[9446]: cluster 2020-09-08T16:10:35.791732+0000 osd.5 (osd.5) 123 : cluster [DBG] 9.f scrub starts 2020-09-08T16:10:36.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:36 smithi055 bash[9296]: cluster 2020-09-08T16:10:34.683986+0000 osd.3 (osd.3) 125 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:36.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:36 smithi055 bash[9296]: cluster 2020-09-08T16:10:34.684092+0000 osd.3 (osd.3) 126 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:36.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:36 smithi055 bash[9296]: cluster 2020-09-08T16:10:35.245345+0000 mon.a (mon.0) 2265 : cluster [DBG] osdmap e726: 12 total, 12 up, 12 in 2020-09-08T16:10:36.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:36 smithi055 bash[9296]: audit 2020-09-08T16:10:35.303353+0000 mon.c (mon.2) 91 : audit [INF] from='client.? v1:172.21.15.104:0/3127317535' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:10:36.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:36 smithi055 bash[9296]: audit 2020-09-08T16:10:35.305839+0000 mon.a (mon.0) 2266 : audit [INF] from='client.36623 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:10:36.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:36 smithi055 bash[9296]: cluster 2020-09-08T16:10:35.791732+0000 osd.5 (osd.5) 123 : cluster [DBG] 9.f scrub starts 2020-09-08T16:10:36.792 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 65 16 114474 114458 110.042 53.1875 0.00824362 0.00908348 2020-09-08T16:10:37.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:37 smithi166 bash[9446]: cluster 2020-09-08T16:10:36.125835+0000 mgr.x (mgr.24118) 1298 : cluster [DBG] pgmap v1940: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 6.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s wr, 883 op/s 2020-09-08T16:10:37.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:37 smithi166 bash[9446]: cluster 2020-09-08T16:10:36.252917+0000 mon.a (mon.0) 2267 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:10:37.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:37 smithi166 bash[9446]: audit 2020-09-08T16:10:36.261095+0000 mon.a (mon.0) 2268 : audit [INF] from='client.36623 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:10:37.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:37 smithi166 bash[9446]: cluster 2020-09-08T16:10:36.261126+0000 mon.a (mon.0) 2269 : cluster [DBG] osdmap e727: 12 total, 12 up, 12 in 2020-09-08T16:10:37.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:37 smithi055 bash[9296]: cluster 2020-09-08T16:10:36.125835+0000 mgr.x (mgr.24118) 1298 : cluster [DBG] pgmap v1940: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 6.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s wr, 883 op/s 2020-09-08T16:10:37.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:37 smithi055 bash[9296]: cluster 2020-09-08T16:10:36.252917+0000 mon.a (mon.0) 2267 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:10:37.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:37 smithi055 bash[9296]: audit 2020-09-08T16:10:36.261095+0000 mon.a (mon.0) 2268 : audit [INF] from='client.36623 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:10:37.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:37 smithi055 bash[9296]: cluster 2020-09-08T16:10:36.261126+0000 mon.a (mon.0) 2269 : cluster [DBG] osdmap e727: 12 total, 12 up, 12 in 2020-09-08T16:10:37.682 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:37 smithi104 bash[9203]: cluster 2020-09-08T16:10:36.125835+0000 mgr.x (mgr.24118) 1298 : cluster [DBG] pgmap v1940: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 6.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s wr, 883 op/s 2020-09-08T16:10:37.682 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:37 smithi104 bash[9203]: cluster 2020-09-08T16:10:36.252917+0000 mon.a (mon.0) 2267 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:10:37.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:37 smithi104 bash[9203]: audit 2020-09-08T16:10:36.261095+0000 mon.a (mon.0) 2268 : audit [INF] from='client.36623 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:10:37.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:37 smithi104 bash[9203]: cluster 2020-09-08T16:10:36.261126+0000 mon.a (mon.0) 2269 : cluster [DBG] osdmap e727: 12 total, 12 up, 12 in 2020-09-08T16:10:37.792 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 66 16 115359 115343 109.213 55.3125 0.00742176 0.00915311 2020-09-08T16:10:38.190 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:10:38.221 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:10:38.255 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:10:38.282 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:10:38.318 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:10:38.345 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:10:38.372 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:10:38.398 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:10:38.428 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:10:38.465 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:10:38.501 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:10:38.532 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:10:38.534 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:10:38.535 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:10:38.535 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:10:38.535 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:10:38.535 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:10:38.535 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:10:38.536 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:10:38.536 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:10:38.536 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:10:38.536 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:10:38.536 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:10:38.537 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:10:38.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:38 smithi166 bash[9446]: audit 2020-09-08T16:10:38.168085+0000 mon.c (mon.2) 92 : audit [DBG] from='client.? v1:172.21.15.104:0/1117178538' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:10:38.628 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:38 smithi104 bash[9203]: audit 2020-09-08T16:10:38.168085+0000 mon.c (mon.2) 92 : audit [DBG] from='client.? v1:172.21.15.104:0/1117178538' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:10:38.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:38 smithi055 bash[9296]: audit 2020-09-08T16:10:38.168085+0000 mon.c (mon.2) 92 : audit [DBG] from='client.? v1:172.21.15.104:0/1117178538' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:10:38.802 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 67 16 116259 116243 108.423 56.25 0.00565407 0.00921705 2020-09-08T16:10:39.080 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:10:39.190 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:39.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:39 smithi055 bash[9296]: cluster 2020-09-08T16:10:38.126640+0000 mgr.x (mgr.24118) 1299 : cluster [DBG] pgmap v1942: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 95 MiB/s wr, 1.52k op/s 2020-09-08T16:10:39.408 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:10:39 smithi055 bash[16646]: debug 2020-09-08T16:10:39.078+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:39.601 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:39 smithi104 bash[9203]: cluster 2020-09-08T16:10:38.126640+0000 mgr.x (mgr.24118) 1299 : cluster [DBG] pgmap v1942: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 95 MiB/s wr, 1.52k op/s 2020-09-08T16:10:39.602 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:39.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:39 smithi166 bash[9446]: cluster 2020-09-08T16:10:38.126640+0000 mgr.x (mgr.24118) 1299 : cluster [DBG] pgmap v1942: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 95 MiB/s wr, 1.52k op/s 2020-09-08T16:10:39.706 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:10:39.792 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 68 16 117269 117253 107.756 63.125 0.0328141 0.00927572 2020-09-08T16:10:39.802 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T16:10:39.875 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:10:39 smithi104 bash[15273]: debug 2020-09-08T16:10:39.601+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:39.876 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:10:39.961 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:10:39.981 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:10:40.109 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:10:40.167 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:10:40.168 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T16:10:40.168 INFO:teuthology.orchestra.run.smithi166.stdout: "complaint_time": 30, 2020-09-08T16:10:40.168 INFO:teuthology.orchestra.run.smithi166.stdout: "num_blocked_ops": 0 2020-09-08T16:10:40.168 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:10:40.175 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:10:39 smithi104 bash[19922]: debug 2020-09-08T16:10:39.877+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:40.211 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:40.359 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:40.408 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:10:40 smithi055 bash[16646]: debug 2020-09-08T16:10:40.102+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:40.469 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:10:40.477 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:10:40 smithi166 bash[18960]: debug 2020-09-08T16:10:40.360+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:40.792 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 69 16 118163 118147 107.005 55.875 0.00884546 0.00933907 2020-09-08T16:10:41.335 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:41 smithi166 bash[9446]: cluster 2020-09-08T16:10:40.127073+0000 mgr.x (mgr.24118) 1300 : cluster [DBG] pgmap v1943: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s wr, 1.14k op/s 2020-09-08T16:10:41.336 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:41 smithi166 bash[9446]: cluster 2020-09-08T16:10:40.473958+0000 osd.8 (osd.8) 104 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:41.336 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:41 smithi166 bash[9446]: cluster 2020-09-08T16:10:40.474043+0000 osd.8 (osd.8) 105 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:41.337 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:10:41 smithi166 bash[14254]: debug 2020-09-08T16:10:41.336+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:41.340 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:10:41.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:41 smithi055 bash[9296]: cluster 2020-09-08T16:10:40.127073+0000 mgr.x (mgr.24118) 1300 : cluster [DBG] pgmap v1943: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s wr, 1.14k op/s 2020-09-08T16:10:41.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:41 smithi055 bash[9296]: cluster 2020-09-08T16:10:40.473958+0000 osd.8 (osd.8) 104 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:41.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:41 smithi055 bash[9296]: cluster 2020-09-08T16:10:40.474043+0000 osd.8 (osd.8) 105 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:41.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:41 smithi104 bash[9203]: cluster 2020-09-08T16:10:40.127073+0000 mgr.x (mgr.24118) 1300 : cluster [DBG] pgmap v1943: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s wr, 1.14k op/s 2020-09-08T16:10:41.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:41 smithi104 bash[9203]: cluster 2020-09-08T16:10:40.473958+0000 osd.8 (osd.8) 104 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:41.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:41 smithi104 bash[9203]: cluster 2020-09-08T16:10:40.474043+0000 osd.8 (osd.8) 105 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:41.444 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:10:41.456 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_historic_ops 2020-09-08T16:10:41.571 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:10:41.623 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:10:41.675 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:41.755 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:10:41.765 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:10:41 smithi104 bash[19922]: debug 2020-09-08T16:10:41.573+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:41.793 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 70 16 119102 119086 106.314 58.6875 0.0101297 0.00940359 2020-09-08T16:10:41.912 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:10:42.064 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:42 smithi104 bash[9203]: cluster 2020-09-08T16:10:41.070997+0000 mon.a (mon.0) 2270 : cluster [DBG] osdmap e728: 12 total, 12 up, 12 in 2020-09-08T16:10:42.065 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:42 smithi166 bash[9446]: cluster 2020-09-08T16:10:41.070997+0000 mon.a (mon.0) 2270 : cluster [DBG] osdmap e728: 12 total, 12 up, 12 in 2020-09-08T16:10:42.389 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:42.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:42 smithi055 bash[9296]: cluster 2020-09-08T16:10:41.070997+0000 mon.a (mon.0) 2270 : cluster [DBG] osdmap e728: 12 total, 12 up, 12 in 2020-09-08T16:10:42.493 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:10:42.620 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:10:42.723 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:10:42.731 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:10:42 smithi166 bash[18960]: debug 2020-09-08T16:10:42.392+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:42.793 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 71 16 120123 120107 105.715 63.8125 0.00894159 0.00945649 2020-09-08T16:10:42.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:10:42 smithi055 bash[16646]: debug 2020-09-08T16:10:42.622+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:43.069 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:43 smithi166 bash[9446]: cluster 2020-09-08T16:10:42.130205+0000 mgr.x (mgr.24118) 1301 : cluster [DBG] pgmap v1945: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s wr, 682 op/s 2020-09-08T16:10:43.302 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:10:43.373 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:10:43 smithi166 bash[14254]: debug 2020-09-08T16:10:43.303+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:43.406 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:10:43.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:43 smithi055 bash[9296]: cluster 2020-09-08T16:10:42.130205+0000 mgr.x (mgr.24118) 1301 : cluster [DBG] pgmap v1945: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s wr, 682 op/s 2020-09-08T16:10:43.416 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:43 smithi104 bash[9203]: cluster 2020-09-08T16:10:42.130205+0000 mgr.x (mgr.24118) 1301 : cluster [DBG] pgmap v1945: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s wr, 682 op/s 2020-09-08T16:10:43.794 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 72 16 121112 121096 105.106 61.8125 0.012457 0.00951191 2020-09-08T16:10:44.094 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:10:44.208 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:44.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:10:44 smithi104 bash[19922]: debug 2020-09-08T16:10:44.093+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:44.439 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:44.545 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:10:44.795 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 73 16 121974 121958 104.404 53.875 0.00998283 0.00957322 2020-09-08T16:10:44.925 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:10:44 smithi104 bash[22305]: debug 2020-09-08T16:10:44.437+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:45.102 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:10:45.207 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:10:45.380 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:10:45.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:45 smithi104 bash[9203]: cluster 2020-09-08T16:10:44.130798+0000 mgr.x (mgr.24118) 1302 : cluster [DBG] pgmap v1946: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 69 MiB/s wr, 1.11k op/s 2020-09-08T16:10:45.431 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:10:45 smithi104 bash[19922]: debug 2020-09-08T16:10:45.101+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:45.482 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:45.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:45 smithi166 bash[9446]: cluster 2020-09-08T16:10:44.130798+0000 mgr.x (mgr.24118) 1302 : cluster [DBG] pgmap v1946: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 69 MiB/s wr, 1.11k op/s 2020-09-08T16:10:45.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:10:45 smithi055 bash[16646]: debug 2020-09-08T16:10:45.374+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:45.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:45 smithi055 bash[9296]: cluster 2020-09-08T16:10:44.130798+0000 mgr.x (mgr.24118) 1302 : cluster [DBG] pgmap v1946: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 69 MiB/s wr, 1.11k op/s 2020-09-08T16:10:45.692 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:45.765 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:10:45 smithi104 bash[22305]: debug 2020-09-08T16:10:45.693+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:45.793 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 74 16 122933 122917 103.803 59.9375 0.0362825 0.00963095 2020-09-08T16:10:45.796 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:46.778 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:46.793 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 75 16 123947 123931 103.264 63.375 0.0160204 0.00968185 2020-09-08T16:10:46.882 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:46.967 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:10:46.968 INFO:teuthology.orchestra.run.smithi166.stdout: "size": 20, 2020-09-08T16:10:46.968 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 600, 2020-09-08T16:10:46.968 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T16:10:46.968 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:46.968 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36113.0:103774 7.a 7:53363393:::benchmark_data_smithi166_48533_object103773:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e574)", 2020-09-08T16:10:46.969 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:01:34.296827+0000", 2020-09-08T16:10:46.969 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 552.66439970600004, 2020-09-08T16:10:46.969 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.209217813, 2020-09-08T16:10:46.971 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:46.972 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:46.972 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:46.972 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36113", 2020-09-08T16:10:46.972 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:10:46.973 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 103774 2020-09-08T16:10:46.973 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:46.973 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:46.973 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:46.974 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:46.974 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.296827+0000", 2020-09-08T16:10:46.974 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:46.974 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:46.975 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:46.975 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:46.975 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.296827+0000", 2020-09-08T16:10:46.975 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999309 2020-09-08T16:10:46.975 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:46.976 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:46.976 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:46.976 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.296758+0000", 2020-09-08T16:10:46.976 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017442299999999999 2020-09-08T16:10:46.977 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:46.977 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:46.977 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:46.977 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.296933+0000", 2020-09-08T16:10:46.978 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7337e-05 2020-09-08T16:10:46.978 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:46.978 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:46.978 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:46.979 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.296950+0000", 2020-09-08T16:10:46.979 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.7009999999999999e-06 2020-09-08T16:10:46.979 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:46.979 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:46.980 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:46.980 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.296957+0000", 2020-09-08T16:10:46.980 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7646999999999999e-05 2020-09-08T16:10:46.982 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:46.982 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:46.982 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:46.982 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.297014+0000", 2020-09-08T16:10:46.983 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00024863899999999999 2020-09-08T16:10:46.983 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:46.983 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:46.983 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:46.984 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.297263+0000", 2020-09-08T16:10:46.984 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017846600000000001 2020-09-08T16:10:46.984 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:46.984 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:46.984 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:46.985 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.297442+0000", 2020-09-08T16:10:46.985 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0074001129999999998 2020-09-08T16:10:46.985 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:46.985 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:46.986 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:46.986 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.304842+0000", 2020-09-08T16:10:46.986 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20110665799999999 2020-09-08T16:10:46.986 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:46.986 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:46.987 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:46.987 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.505948+0000", 2020-09-08T16:10:46.987 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.6162999999999997e-05 2020-09-08T16:10:46.991 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:46.993 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:46.993 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:46.993 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.506005+0000", 2020-09-08T16:10:46.994 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.0633000000000002e-05 2020-09-08T16:10:46.994 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:46.994 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:46.994 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:46.995 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.506045+0000", 2020-09-08T16:10:46.995 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.209217813 2020-09-08T16:10:46.995 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:46.995 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:46.995 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:46.996 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:46.996 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:46.996 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36113.0:103780 7.a 7:5d4a9d01:::benchmark_data_smithi166_48533_object103779:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e574)", 2020-09-08T16:10:46.997 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:01:34.298990+0000", 2020-09-08T16:10:46.997 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 552.66223685199998, 2020-09-08T16:10:46.997 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22428989799999999, 2020-09-08T16:10:46.997 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:46.997 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:46.998 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:46.999 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36113", 2020-09-08T16:10:46.999 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:10:46.999 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 103780 2020-09-08T16:10:47.000 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.000 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:47.000 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.000 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:47.000 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.298990+0000", 2020-09-08T16:10:47.001 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:47.001 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.001 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.001 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:47.001 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.298990+0000", 2020-09-08T16:10:47.003 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999547 2020-09-08T16:10:47.003 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.003 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.003 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:47.004 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.298945+0000", 2020-09-08T16:10:47.004 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000150474 2020-09-08T16:10:47.004 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.005 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.006 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:47.009 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.299096+0000", 2020-09-08T16:10:47.010 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4715000000000001e-05 2020-09-08T16:10:47.010 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.012 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.012 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:47.013 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.299110+0000", 2020-09-08T16:10:47.014 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.857e-06 2020-09-08T16:10:47.014 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.014 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.014 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:47.015 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.299116+0000", 2020-09-08T16:10:47.015 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0021804020000000001 2020-09-08T16:10:47.015 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.015 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.016 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:47.016 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.301297+0000", 2020-09-08T16:10:47.016 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017946600000000001 2020-09-08T16:10:47.016 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.016 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.017 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:47.017 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.301476+0000", 2020-09-08T16:10:47.017 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000114312 2020-09-08T16:10:47.017 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.018 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.018 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:47.018 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.301590+0000", 2020-09-08T16:10:47.018 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0032661460000000002 2020-09-08T16:10:47.019 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.019 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.020 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:47.020 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.304856+0000", 2020-09-08T16:10:47.021 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21816065300000001 2020-09-08T16:10:47.021 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.021 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.021 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:47.021 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.523017+0000", 2020-09-08T16:10:47.022 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000151438 2020-09-08T16:10:47.022 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.022 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.022 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:47.022 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.523169+0000", 2020-09-08T16:10:47.023 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011153400000000001 2020-09-08T16:10:47.023 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.023 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.023 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:47.023 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.523280+0000", 2020-09-08T16:10:47.024 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22428989799999999 2020-09-08T16:10:47.024 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.024 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:47.024 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.024 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.025 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.025 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36113.0:103782 7.a 7:590ccfd6:::benchmark_data_smithi166_48533_object103781:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e574)", 2020-09-08T16:10:47.025 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:01:34.302567+0000", 2020-09-08T16:10:47.025 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 552.65865990500004, 2020-09-08T16:10:47.025 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22737550200000001, 2020-09-08T16:10:47.026 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:47.026 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:47.026 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:47.026 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36113", 2020-09-08T16:10:47.027 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:10:47.027 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 103782 2020-09-08T16:10:47.027 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.027 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:47.027 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.028 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:47.028 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.302567+0000", 2020-09-08T16:10:47.028 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:47.028 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.028 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.028 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:47.029 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.302567+0000", 2020-09-08T16:10:47.029 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999628 2020-09-08T16:10:47.029 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.029 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.030 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:47.030 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.302530+0000", 2020-09-08T16:10:47.031 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000151144 2020-09-08T16:10:47.032 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.032 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.032 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:47.033 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.302681+0000", 2020-09-08T16:10:47.033 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.3325000000000001e-05 2020-09-08T16:10:47.033 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.033 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.033 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:47.034 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.302704+0000", 2020-09-08T16:10:47.034 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.8009999999999998e-06 2020-09-08T16:10:47.034 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.034 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.034 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:47.035 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.302711+0000", 2020-09-08T16:10:47.035 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0053525600000000001 2020-09-08T16:10:47.035 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.036 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.036 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:47.036 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.308064+0000", 2020-09-08T16:10:47.036 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016298500000000001 2020-09-08T16:10:47.036 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.036 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.037 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:47.037 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.308227+0000", 2020-09-08T16:10:47.037 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019652 2020-09-08T16:10:47.037 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.037 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.038 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:47.038 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.308423+0000", 2020-09-08T16:10:47.038 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0035200499999999998 2020-09-08T16:10:47.038 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.038 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.039 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:47.039 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.311943+0000", 2020-09-08T16:10:47.039 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.217759903 2020-09-08T16:10:47.039 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.039 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.040 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:47.040 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.529703+0000", 2020-09-08T16:10:47.040 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.9729999999999996e-05 2020-09-08T16:10:47.040 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.040 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.041 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:47.041 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.529803+0000", 2020-09-08T16:10:47.041 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013989800000000001 2020-09-08T16:10:47.041 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.041 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.042 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:47.042 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.529943+0000", 2020-09-08T16:10:47.042 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22737550200000001 2020-09-08T16:10:47.043 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.043 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:47.043 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.043 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.044 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.044 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36113.0:103799 7.a 7:5e8485e9:::benchmark_data_smithi166_48533_object103798:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e574)", 2020-09-08T16:10:47.044 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:01:34.315250+0000", 2020-09-08T16:10:47.044 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 552.64597750200005, 2020-09-08T16:10:47.044 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.214956379, 2020-09-08T16:10:47.045 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:47.045 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:47.045 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:47.045 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36113", 2020-09-08T16:10:47.045 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:10:47.046 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 103799 2020-09-08T16:10:47.046 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.046 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:47.046 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.046 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:47.047 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.315250+0000", 2020-09-08T16:10:47.047 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:47.047 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.047 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.047 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:47.047 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.315250+0000", 2020-09-08T16:10:47.048 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999619 2020-09-08T16:10:47.048 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.048 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.048 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:47.048 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.315211+0000", 2020-09-08T16:10:47.049 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012499500000000001 2020-09-08T16:10:47.049 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.049 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.049 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:47.049 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.315336+0000", 2020-09-08T16:10:47.050 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6762e-05 2020-09-08T16:10:47.050 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.050 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.050 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:47.050 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.315353+0000", 2020-09-08T16:10:47.051 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.2299999999999996e-06 2020-09-08T16:10:47.051 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.051 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.051 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:47.051 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.315359+0000", 2020-09-08T16:10:47.051 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.4727000000000006e-05 2020-09-08T16:10:47.052 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.052 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.053 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:47.053 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.315424+0000", 2020-09-08T16:10:47.053 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000160273 2020-09-08T16:10:47.054 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.054 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.054 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:47.054 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.315584+0000", 2020-09-08T16:10:47.054 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000192721 2020-09-08T16:10:47.055 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.055 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.055 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:47.055 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.315777+0000", 2020-09-08T16:10:47.055 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0041253180000000002 2020-09-08T16:10:47.056 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.056 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.056 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:47.056 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.319902+0000", 2020-09-08T16:10:47.056 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21013206400000001 2020-09-08T16:10:47.057 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.057 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.057 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:47.057 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.530034+0000", 2020-09-08T16:10:47.057 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7199000000000002e-05 2020-09-08T16:10:47.058 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.058 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.058 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:47.058 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.530132+0000", 2020-09-08T16:10:47.058 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.4225000000000002e-05 2020-09-08T16:10:47.058 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.059 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.059 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:47.059 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.530206+0000", 2020-09-08T16:10:47.059 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.214956379 2020-09-08T16:10:47.059 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.060 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:47.060 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.060 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.060 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.060 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36113.0:103831 7.a 7:5d0c4bcb:::benchmark_data_smithi166_48533_object103830:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e574)", 2020-09-08T16:10:47.061 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:01:34.344505+0000", 2020-09-08T16:10:47.061 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 552.61672252799997, 2020-09-08T16:10:47.061 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19404227900000001, 2020-09-08T16:10:47.061 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:47.061 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:47.062 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:47.062 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36113", 2020-09-08T16:10:47.062 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:10:47.062 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 103831 2020-09-08T16:10:47.062 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.063 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:47.063 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.063 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:47.063 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.344505+0000", 2020-09-08T16:10:47.063 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:47.063 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.064 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.064 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:47.064 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.344505+0000", 2020-09-08T16:10:47.064 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999661 2020-09-08T16:10:47.064 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.065 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.065 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:47.065 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.344471+0000", 2020-09-08T16:10:47.065 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000128443 2020-09-08T16:10:47.065 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.066 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.066 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:47.066 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.344599+0000", 2020-09-08T16:10:47.066 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.5741999999999998e-05 2020-09-08T16:10:47.067 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.067 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.067 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:47.067 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.344625+0000", 2020-09-08T16:10:47.068 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.4319999999999998e-06 2020-09-08T16:10:47.068 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.068 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.068 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:47.068 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.344630+0000", 2020-09-08T16:10:47.068 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5370000000000006e-05 2020-09-08T16:10:47.069 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.069 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.069 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:47.069 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.344696+0000", 2020-09-08T16:10:47.069 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018409899999999999 2020-09-08T16:10:47.070 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.070 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.070 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:47.070 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.344880+0000", 2020-09-08T16:10:47.071 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000212697 2020-09-08T16:10:47.071 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.071 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.071 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:47.072 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.345093+0000", 2020-09-08T16:10:47.072 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0048547269999999997 2020-09-08T16:10:47.072 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.072 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.072 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:47.073 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.349947+0000", 2020-09-08T16:10:47.073 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18848527500000001 2020-09-08T16:10:47.073 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.073 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.073 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:47.074 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.538433+0000", 2020-09-08T16:10:47.074 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8239999999999998e-05 2020-09-08T16:10:47.074 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.074 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.075 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:47.075 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.538491+0000", 2020-09-08T16:10:47.075 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.5887999999999998e-05 2020-09-08T16:10:47.075 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.075 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.076 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:47.076 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.538547+0000", 2020-09-08T16:10:47.076 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19404227900000001 2020-09-08T16:10:47.076 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.076 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:47.077 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.077 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.077 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.077 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36113.0:103844 7.a 7:57e8b71e:::benchmark_data_smithi166_48533_object103843:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e574)", 2020-09-08T16:10:47.078 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:01:34.353105+0000", 2020-09-08T16:10:47.078 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 552.60812221399999, 2020-09-08T16:10:47.078 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.185572714, 2020-09-08T16:10:47.078 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:47.078 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:47.079 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:47.079 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36113", 2020-09-08T16:10:47.079 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:10:47.079 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 103844 2020-09-08T16:10:47.079 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.080 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:47.080 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.080 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:47.080 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.353105+0000", 2020-09-08T16:10:47.080 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:47.081 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.081 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.081 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:47.081 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.353105+0000", 2020-09-08T16:10:47.081 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999003 2020-09-08T16:10:47.082 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.082 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.082 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:47.082 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.353005+0000", 2020-09-08T16:10:47.083 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000177615 2020-09-08T16:10:47.083 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.083 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.083 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:47.083 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.353183+0000", 2020-09-08T16:10:47.084 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.573e-05 2020-09-08T16:10:47.084 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.084 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.084 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:47.084 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.353198+0000", 2020-09-08T16:10:47.084 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0553999999999999e-05 2020-09-08T16:10:47.085 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.085 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.085 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:47.085 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.353209+0000", 2020-09-08T16:10:47.086 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.8677999999999995e-05 2020-09-08T16:10:47.086 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.086 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.086 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:47.086 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.353278+0000", 2020-09-08T16:10:47.087 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00021879500000000001 2020-09-08T16:10:47.087 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.087 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.087 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:47.087 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.353496+0000", 2020-09-08T16:10:47.088 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00024492600000000002 2020-09-08T16:10:47.088 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.088 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.088 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:47.088 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.353741+0000", 2020-09-08T16:10:47.089 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0045425609999999996 2020-09-08T16:10:47.089 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.089 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.089 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:47.089 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.358284+0000", 2020-09-08T16:10:47.090 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18029869200000001 2020-09-08T16:10:47.090 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.090 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.090 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:47.090 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.538583+0000", 2020-09-08T16:10:47.091 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.2357999999999997e-05 2020-09-08T16:10:47.091 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.091 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.091 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:47.091 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.538625+0000", 2020-09-08T16:10:47.092 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.2553000000000003e-05 2020-09-08T16:10:47.092 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.093 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.093 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:47.093 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.538678+0000", 2020-09-08T16:10:47.093 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.185572714 2020-09-08T16:10:47.094 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.094 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:47.094 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.094 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.094 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.095 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36113.0:103852 7.a 7:53ef1347:::benchmark_data_smithi166_48533_object103851:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e574)", 2020-09-08T16:10:47.095 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:01:34.359015+0000", 2020-09-08T16:10:47.095 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 552.60221161599998, 2020-09-08T16:10:47.095 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17985585900000001, 2020-09-08T16:10:47.095 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:47.096 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:47.096 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:47.096 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36113", 2020-09-08T16:10:47.096 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:10:47.096 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 103852 2020-09-08T16:10:47.097 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.097 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:47.097 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.097 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:47.097 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.359015+0000", 2020-09-08T16:10:47.098 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:47.098 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.098 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.098 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:47.098 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.359015+0000", 2020-09-08T16:10:47.099 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999771 2020-09-08T16:10:47.099 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.099 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.099 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:47.099 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.358992+0000", 2020-09-08T16:10:47.100 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.4696000000000004e-05 2020-09-08T16:10:47.100 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.100 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.100 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:47.100 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.359077+0000", 2020-09-08T16:10:47.101 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2517000000000001e-05 2020-09-08T16:10:47.101 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.102 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.102 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:47.102 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.359090+0000", 2020-09-08T16:10:47.103 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.9659999999999999e-06 2020-09-08T16:10:47.103 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.103 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.103 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:47.103 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.359097+0000", 2020-09-08T16:10:47.104 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0470000000000003e-05 2020-09-08T16:10:47.104 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.104 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.104 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:47.104 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.359147+0000", 2020-09-08T16:10:47.105 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000130853 2020-09-08T16:10:47.105 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.105 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.105 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:47.105 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.359278+0000", 2020-09-08T16:10:47.106 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000195164 2020-09-08T16:10:47.106 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.106 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.106 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:47.106 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.359473+0000", 2020-09-08T16:10:47.107 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.003335045 2020-09-08T16:10:47.107 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.107 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.107 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:47.107 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.362808+0000", 2020-09-08T16:10:47.108 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17596857399999999 2020-09-08T16:10:47.108 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.108 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.108 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:47.108 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.538777+0000", 2020-09-08T16:10:47.109 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.211e-05 2020-09-08T16:10:47.109 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.109 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.109 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:47.109 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.538839+0000", 2020-09-08T16:10:47.110 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.2388000000000002e-05 2020-09-08T16:10:47.110 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.110 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.111 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:47.111 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:01:34.538871+0000", 2020-09-08T16:10:47.111 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17985585900000001 2020-09-08T16:10:47.111 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.111 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:47.112 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.112 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.112 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.112 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36113.0:194226 7.a 7:52e29dfe:::benchmark_data_smithi166_48533_object59709:head [delete] snapc 0=[] ondisk+write+known_if_redirected e599)", 2020-09-08T16:10:47.112 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:03:11.246289+0000", 2020-09-08T16:10:47.113 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 455.71493770299998, 2020-09-08T16:10:47.113 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.180932915, 2020-09-08T16:10:47.113 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:47.113 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:47.113 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:47.114 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36113", 2020-09-08T16:10:47.114 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:10:47.114 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 194226 2020-09-08T16:10:47.114 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.114 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:47.114 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.115 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:47.115 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.246289+0000", 2020-09-08T16:10:47.115 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:47.115 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.115 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.116 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:47.116 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.246289+0000", 2020-09-08T16:10:47.116 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999661 2020-09-08T16:10:47.116 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.116 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.117 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:47.117 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.246255+0000", 2020-09-08T16:10:47.117 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.4606999999999999e-05 2020-09-08T16:10:47.117 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.117 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.118 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:47.118 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.246330+0000", 2020-09-08T16:10:47.118 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.9089e-05 2020-09-08T16:10:47.118 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.118 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.119 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:47.119 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.246349+0000", 2020-09-08T16:10:47.119 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.4120000000000004e-06 2020-09-08T16:10:47.120 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.120 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.120 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:47.120 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.246357+0000", 2020-09-08T16:10:47.120 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.5847999999999998e-05 2020-09-08T16:10:47.121 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.121 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.121 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:47.121 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.246412+0000", 2020-09-08T16:10:47.121 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013351300000000001 2020-09-08T16:10:47.121 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.122 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.122 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:47.122 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.246546+0000", 2020-09-08T16:10:47.122 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000108204 2020-09-08T16:10:47.122 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.123 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.123 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:47.123 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.246654+0000", 2020-09-08T16:10:47.123 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0049648110000000004 2020-09-08T16:10:47.123 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.124 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.124 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:47.124 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.251619+0000", 2020-09-08T16:10:47.124 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.175255257 2020-09-08T16:10:47.124 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.125 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.125 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:47.125 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.426874+0000", 2020-09-08T16:10:47.125 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019644399999999999 2020-09-08T16:10:47.125 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.126 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.126 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:47.126 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.427071+0000", 2020-09-08T16:10:47.126 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015164700000000001 2020-09-08T16:10:47.126 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.126 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.127 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:47.127 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.427222+0000", 2020-09-08T16:10:47.127 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.180932915 2020-09-08T16:10:47.127 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.127 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:47.128 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.128 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.128 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.128 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36113.0:194236 7.a 7:5c4f8b18:::benchmark_data_smithi166_48533_object59719:head [delete] snapc 0=[] ondisk+write+known_if_redirected e599)", 2020-09-08T16:10:47.128 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:03:11.258094+0000", 2020-09-08T16:10:47.129 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 455.70313307800001, 2020-09-08T16:10:47.129 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18459036100000001, 2020-09-08T16:10:47.129 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:47.129 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:47.129 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:47.130 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36113", 2020-09-08T16:10:47.130 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:10:47.130 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 194236 2020-09-08T16:10:47.130 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.130 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:47.131 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.131 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:47.131 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.258094+0000", 2020-09-08T16:10:47.131 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:47.131 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.131 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.132 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:47.132 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.258094+0000", 2020-09-08T16:10:47.132 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.999958 2020-09-08T16:10:47.132 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.132 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.133 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:47.133 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.258052+0000", 2020-09-08T16:10:47.133 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.4735999999999996e-05 2020-09-08T16:10:47.133 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.133 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.134 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:47.134 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.258147+0000", 2020-09-08T16:10:47.134 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7260000000000001e-05 2020-09-08T16:10:47.135 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.135 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.135 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:47.135 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.258164+0000", 2020-09-08T16:10:47.135 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.4510000000000003e-06 2020-09-08T16:10:47.136 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.136 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.136 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:47.136 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.258171+0000", 2020-09-08T16:10:47.136 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.4649999999999998e-05 2020-09-08T16:10:47.137 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.137 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.137 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:47.137 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.258256+0000", 2020-09-08T16:10:47.137 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000144072 2020-09-08T16:10:47.138 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.138 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.138 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:47.138 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.258400+0000", 2020-09-08T16:10:47.138 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000146969 2020-09-08T16:10:47.139 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.139 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.139 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:47.139 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.258547+0000", 2020-09-08T16:10:47.139 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0050440390000000002 2020-09-08T16:10:47.139 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.140 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.140 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:47.140 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.263591+0000", 2020-09-08T16:10:47.140 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17896443100000001 2020-09-08T16:10:47.140 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.141 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.141 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:47.141 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.442556+0000", 2020-09-08T16:10:47.141 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.3776000000000007e-05 2020-09-08T16:10:47.141 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.142 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.142 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:47.142 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.442629+0000", 2020-09-08T16:10:47.142 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.4931e-05 2020-09-08T16:10:47.142 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.143 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.143 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:47.143 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.442684+0000", 2020-09-08T16:10:47.143 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18459036100000001 2020-09-08T16:10:47.143 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.144 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:47.144 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.144 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.144 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.144 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36113.0:194242 7.a 7:57e57e89:::benchmark_data_smithi166_48533_object59725:head [delete] snapc 0=[] ondisk+write+known_if_redirected e599)", 2020-09-08T16:10:47.144 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:03:11.266332+0000", 2020-09-08T16:10:47.145 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 455.69489478899999, 2020-09-08T16:10:47.145 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17653506099999999, 2020-09-08T16:10:47.145 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:47.145 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:47.145 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:47.146 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36113", 2020-09-08T16:10:47.146 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:10:47.146 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 194242 2020-09-08T16:10:47.146 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.146 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:47.147 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.147 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:47.147 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.266332+0000", 2020-09-08T16:10:47.147 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:47.147 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.148 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.148 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:47.148 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.266332+0000", 2020-09-08T16:10:47.148 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999671 2020-09-08T16:10:47.148 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.149 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.149 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:47.149 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.266299+0000", 2020-09-08T16:10:47.149 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.551e-05 2020-09-08T16:10:47.149 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.150 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.150 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:47.150 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.266375+0000", 2020-09-08T16:10:47.150 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5954e-05 2020-09-08T16:10:47.150 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.151 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.151 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:47.151 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.266391+0000", 2020-09-08T16:10:47.151 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.5639999999999997e-06 2020-09-08T16:10:47.151 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.151 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.152 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:47.152 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.266399+0000", 2020-09-08T16:10:47.152 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.1495000000000006e-05 2020-09-08T16:10:47.152 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.152 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.153 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:47.153 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.266471+0000", 2020-09-08T16:10:47.153 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00023016599999999999 2020-09-08T16:10:47.153 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.153 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.154 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:47.154 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.266701+0000", 2020-09-08T16:10:47.154 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00025608299999999999 2020-09-08T16:10:47.154 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.154 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.155 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:10:47.155 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.266957+0000", 2020-09-08T16:10:47.155 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0049374850000000001 2020-09-08T16:10:47.155 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.155 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.155 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:47.156 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.271895+0000", 2020-09-08T16:10:47.156 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.170855539 2020-09-08T16:10:47.156 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.156 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.156 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:47.157 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.442750+0000", 2020-09-08T16:10:47.157 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5269000000000001e-05 2020-09-08T16:10:47.157 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.157 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.157 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:47.158 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.442815+0000", 2020-09-08T16:10:47.158 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1987999999999998e-05 2020-09-08T16:10:47.158 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.158 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.158 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:47.159 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:03:11.442867+0000", 2020-09-08T16:10:47.159 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17653506099999999 2020-09-08T16:10:47.159 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.159 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:47.159 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.161 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.161 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.161 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:108743 8.c 8:3860a137:::benchmark_data_smithi166_52302_object108742:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e653)", 2020-09-08T16:10:47.162 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:06:09.169346+0000", 2020-09-08T16:10:47.162 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 277.79188065800003, 2020-09-08T16:10:47.162 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.160923343, 2020-09-08T16:10:47.162 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:47.162 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:47.163 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:47.163 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:10:47.163 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:10:47.163 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 108743 2020-09-08T16:10:47.164 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.164 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:47.164 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.165 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:47.165 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.169346+0000", 2020-09-08T16:10:47.165 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:47.165 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.166 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.166 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:47.166 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.169346+0000", 2020-09-08T16:10:47.166 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.999959 2020-09-08T16:10:47.166 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.167 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.167 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:47.167 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.169306+0000", 2020-09-08T16:10:47.167 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012373899999999999 2020-09-08T16:10:47.167 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.167 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.168 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:47.168 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.169429+0000", 2020-09-08T16:10:47.168 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4090999999999999e-05 2020-09-08T16:10:47.168 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.168 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.169 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:47.169 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.169443+0000", 2020-09-08T16:10:47.169 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.5360000000000004e-06 2020-09-08T16:10:47.169 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.169 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.170 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:47.170 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.169451+0000", 2020-09-08T16:10:47.170 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.4049999999999997e-05 2020-09-08T16:10:47.170 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.170 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.171 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:47.172 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.169525+0000", 2020-09-08T16:10:47.172 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.7490000000000004e-06 2020-09-08T16:10:47.172 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.172 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.172 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op must wait for map", 2020-09-08T16:10:47.173 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.169534+0000", 2020-09-08T16:10:47.173 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0014308979999999999 2020-09-08T16:10:47.173 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.173 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.173 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:47.174 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.170965+0000", 2020-09-08T16:10:47.174 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000130164 2020-09-08T16:10:47.174 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.174 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.174 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:47.175 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.171095+0000", 2020-09-08T16:10:47.175 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014982799999999999 2020-09-08T16:10:47.175 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.175 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.175 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 8", 2020-09-08T16:10:47.176 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.171245+0000", 2020-09-08T16:10:47.176 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.003352879 2020-09-08T16:10:47.176 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.176 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.176 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:47.176 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.174598+0000", 2020-09-08T16:10:47.177 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15553709399999999 2020-09-08T16:10:47.177 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.177 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.177 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:47.177 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.330135+0000", 2020-09-08T16:10:47.178 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.9108000000000003e-05 2020-09-08T16:10:47.178 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.178 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.178 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:47.178 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.330204+0000", 2020-09-08T16:10:47.179 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5983999999999997e-05 2020-09-08T16:10:47.179 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.179 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.179 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:47.179 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:09.330270+0000", 2020-09-08T16:10:47.180 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.160923343 2020-09-08T16:10:47.180 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.180 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:47.180 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.180 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.181 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.181 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:22538 9.3 9:cb8c0fef:::benchmark_data_smithi166_56754_object22537:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e709)", 2020-09-08T16:10:47.181 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:09:39.132545+0000", 2020-09-08T16:10:47.181 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 67.828682408000006, 2020-09-08T16:10:47.181 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18162690000000001, 2020-09-08T16:10:47.182 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:47.182 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:47.182 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:47.182 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:10:47.182 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:10:47.182 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 22538 2020-09-08T16:10:47.183 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.183 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:47.183 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.183 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:47.183 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.132545+0000", 2020-09-08T16:10:47.184 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:47.184 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.184 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.184 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:47.184 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.132545+0000", 2020-09-08T16:10:47.185 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999647 2020-09-08T16:10:47.185 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.185 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.185 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:47.185 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.132510+0000", 2020-09-08T16:10:47.186 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000127465 2020-09-08T16:10:47.186 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.186 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.186 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:47.186 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.132637+0000", 2020-09-08T16:10:47.186 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3200000000000001e-05 2020-09-08T16:10:47.187 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.187 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.187 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:47.187 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.132650+0000", 2020-09-08T16:10:47.187 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.6840000000000001e-06 2020-09-08T16:10:47.188 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.188 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.188 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:47.188 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.132656+0000", 2020-09-08T16:10:47.188 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.4786e-05 2020-09-08T16:10:47.189 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.189 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.189 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:47.189 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.132711+0000", 2020-09-08T16:10:47.189 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00028845200000000002 2020-09-08T16:10:47.190 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.190 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.190 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:47.190 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.132999+0000", 2020-09-08T16:10:47.190 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000128465 2020-09-08T16:10:47.191 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.191 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.191 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 3", 2020-09-08T16:10:47.191 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.133128+0000", 2020-09-08T16:10:47.191 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0010098489999999999 2020-09-08T16:10:47.191 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.192 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.192 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:47.192 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.134137+0000", 2020-09-08T16:10:47.192 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17965806000000001 2020-09-08T16:10:47.192 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.193 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.193 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:47.193 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.313796+0000", 2020-09-08T16:10:47.193 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00022485199999999999 2020-09-08T16:10:47.193 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.194 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.194 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:47.194 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.314020+0000", 2020-09-08T16:10:47.194 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015116700000000001 2020-09-08T16:10:47.194 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.195 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.195 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:47.195 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.314172+0000", 2020-09-08T16:10:47.195 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18162690000000001 2020-09-08T16:10:47.195 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.195 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:47.196 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.196 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.196 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.196 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:22546 9.3 9:ccccbc6a:::benchmark_data_smithi166_56754_object22545:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e709)", 2020-09-08T16:10:47.196 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:09:39.134772+0000", 2020-09-08T16:10:47.197 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 67.826454673000001, 2020-09-08T16:10:47.197 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.179937135, 2020-09-08T16:10:47.197 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:47.197 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:47.198 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:47.198 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:10:47.198 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:10:47.199 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 22546 2020-09-08T16:10:47.199 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.199 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:47.199 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.199 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:47.200 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.134772+0000", 2020-09-08T16:10:47.200 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:47.200 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.200 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.200 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:47.201 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.134772+0000", 2020-09-08T16:10:47.201 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999628 2020-09-08T16:10:47.201 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.203 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.203 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:47.203 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.134735+0000", 2020-09-08T16:10:47.203 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011824000000000001 2020-09-08T16:10:47.203 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.204 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.204 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:47.204 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.134853+0000", 2020-09-08T16:10:47.204 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5123000000000001e-05 2020-09-08T16:10:47.204 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.205 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.205 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:47.205 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.134868+0000", 2020-09-08T16:10:47.205 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.4989999999999999e-06 2020-09-08T16:10:47.205 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.206 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.206 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:47.206 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.134875+0000", 2020-09-08T16:10:47.206 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000104029 2020-09-08T16:10:47.206 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.207 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.207 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:47.207 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.134979+0000", 2020-09-08T16:10:47.208 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000250413 2020-09-08T16:10:47.208 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.208 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.208 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:47.208 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.135229+0000", 2020-09-08T16:10:47.209 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00010529700000000001 2020-09-08T16:10:47.209 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.209 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.209 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 3", 2020-09-08T16:10:47.209 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.135335+0000", 2020-09-08T16:10:47.210 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0015201119999999999 2020-09-08T16:10:47.210 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.210 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.210 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:47.210 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.136855+0000", 2020-09-08T16:10:47.210 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17751244299999999 2020-09-08T16:10:47.211 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.211 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.211 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:47.211 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.314367+0000", 2020-09-08T16:10:47.211 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000231819 2020-09-08T16:10:47.212 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.212 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.212 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:47.212 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.314599+0000", 2020-09-08T16:10:47.212 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000110571 2020-09-08T16:10:47.213 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.213 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.213 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:47.213 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.314709+0000", 2020-09-08T16:10:47.213 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.179937135 2020-09-08T16:10:47.214 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.214 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:47.214 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.214 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.215 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.215 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:22560 9.3 9:c9ea7cbd:::benchmark_data_smithi166_56754_object22559:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e709)", 2020-09-08T16:10:47.215 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:09:39.141867+0000", 2020-09-08T16:10:47.215 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 67.819359860000006, 2020-09-08T16:10:47.215 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17326193100000001, 2020-09-08T16:10:47.216 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:47.216 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:47.216 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:47.216 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:10:47.216 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:10:47.216 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 22560 2020-09-08T16:10:47.217 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.217 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:47.217 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.217 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:47.217 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.141867+0000", 2020-09-08T16:10:47.218 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:47.218 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.218 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.218 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:47.218 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.141867+0000", 2020-09-08T16:10:47.219 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999595 2020-09-08T16:10:47.219 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.219 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.219 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:47.219 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.141827+0000", 2020-09-08T16:10:47.220 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012765699999999999 2020-09-08T16:10:47.220 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.220 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.220 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:47.220 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.141954+0000", 2020-09-08T16:10:47.220 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4205e-05 2020-09-08T16:10:47.221 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.221 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.221 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:47.221 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.141968+0000", 2020-09-08T16:10:47.221 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7400000000000004e-06 2020-09-08T16:10:47.222 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.222 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.222 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:47.222 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.141976+0000", 2020-09-08T16:10:47.223 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.7011000000000001e-05 2020-09-08T16:10:47.223 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.223 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.223 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:47.223 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.142023+0000", 2020-09-08T16:10:47.224 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000117261 2020-09-08T16:10:47.224 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.224 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.224 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:47.224 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.142140+0000", 2020-09-08T16:10:47.225 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000119919 2020-09-08T16:10:47.225 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.225 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.225 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 3", 2020-09-08T16:10:47.225 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.142260+0000", 2020-09-08T16:10:47.226 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00082021999999999998 2020-09-08T16:10:47.226 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.226 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.226 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:47.226 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.143081+0000", 2020-09-08T16:10:47.227 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.171765687 2020-09-08T16:10:47.227 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.227 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.227 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:47.227 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.314846+0000", 2020-09-08T16:10:47.228 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015419699999999999 2020-09-08T16:10:47.228 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.228 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.228 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:47.228 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.315000+0000", 2020-09-08T16:10:47.228 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012863800000000001 2020-09-08T16:10:47.229 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.229 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.229 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:47.229 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:39.315129+0000", 2020-09-08T16:10:47.229 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17326193100000001 2020-09-08T16:10:47.230 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.230 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:47.230 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.230 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.230 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.231 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:69093 9.3 9:cd5adc47:::benchmark_data_smithi166_56754_object69092:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e714)", 2020-09-08T16:10:47.231 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:10:01.802099+0000", 2020-09-08T16:10:47.231 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 45.159127697999999, 2020-09-08T16:10:47.231 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15761020100000001, 2020-09-08T16:10:47.231 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:47.232 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:47.232 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:47.232 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:10:47.232 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:10:47.234 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 69093 2020-09-08T16:10:47.234 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.234 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:47.235 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.235 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:47.235 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.802099+0000", 2020-09-08T16:10:47.235 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:47.237 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.237 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.237 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:47.238 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.802099+0000", 2020-09-08T16:10:47.238 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999576 2020-09-08T16:10:47.238 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.238 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.238 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:47.239 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.802057+0000", 2020-09-08T16:10:47.239 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00061531300000000004 2020-09-08T16:10:47.239 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.239 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.239 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:47.240 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.802672+0000", 2020-09-08T16:10:47.240 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4117999999999999e-05 2020-09-08T16:10:47.240 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.240 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.240 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:47.241 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.802687+0000", 2020-09-08T16:10:47.242 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.2179999999999996e-06 2020-09-08T16:10:47.242 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.242 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.242 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:47.243 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.802693+0000", 2020-09-08T16:10:47.243 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000117483 2020-09-08T16:10:47.243 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.243 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.243 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:47.244 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.802810+0000", 2020-09-08T16:10:47.244 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00024454499999999998 2020-09-08T16:10:47.244 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.244 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.244 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:47.245 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.803055+0000", 2020-09-08T16:10:47.245 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015707899999999999 2020-09-08T16:10:47.245 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.245 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.245 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 3", 2020-09-08T16:10:47.245 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.803212+0000", 2020-09-08T16:10:47.246 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0030192819999999999 2020-09-08T16:10:47.246 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.246 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.246 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:47.246 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.806231+0000", 2020-09-08T16:10:47.247 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.153184296 2020-09-08T16:10:47.247 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.247 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.247 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:47.248 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.959415+0000", 2020-09-08T16:10:47.248 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016507699999999999 2020-09-08T16:10:47.248 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.248 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.248 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:47.249 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.959581+0000", 2020-09-08T16:10:47.249 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012900800000000001 2020-09-08T16:10:47.249 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.249 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.249 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:47.250 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.959710+0000", 2020-09-08T16:10:47.250 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15761020100000001 2020-09-08T16:10:47.250 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.250 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:47.250 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.251 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.251 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.251 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:69105 9.3 9:c4e3826f:::benchmark_data_smithi166_56754_object69104:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e714)", 2020-09-08T16:10:47.251 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:10:01.808792+0000", 2020-09-08T16:10:47.251 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 45.152434790999997, 2020-09-08T16:10:47.251 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.155656303, 2020-09-08T16:10:47.252 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:47.252 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:47.252 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:47.252 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:10:47.252 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:10:47.253 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 69105 2020-09-08T16:10:47.253 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.253 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:47.253 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.253 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:47.254 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.808792+0000", 2020-09-08T16:10:47.254 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:47.254 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.254 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.254 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:47.255 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.808792+0000", 2020-09-08T16:10:47.255 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999599 2020-09-08T16:10:47.255 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.255 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.255 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:47.255 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.808752+0000", 2020-09-08T16:10:47.256 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015087600000000001 2020-09-08T16:10:47.256 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.256 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.256 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:47.256 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.808903+0000", 2020-09-08T16:10:47.257 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.348e-05 2020-09-08T16:10:47.257 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.257 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.257 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:47.257 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.808917+0000", 2020-09-08T16:10:47.258 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.3249999999999996e-06 2020-09-08T16:10:47.258 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.258 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.258 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:47.258 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.808923+0000", 2020-09-08T16:10:47.258 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8053000000000001e-05 2020-09-08T16:10:47.259 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.259 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.259 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:47.259 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.808981+0000", 2020-09-08T16:10:47.259 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013659100000000001 2020-09-08T16:10:47.260 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.260 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.260 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:47.260 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.809118+0000", 2020-09-08T16:10:47.260 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015385 2020-09-08T16:10:47.261 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.261 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.261 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 3", 2020-09-08T16:10:47.261 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.809271+0000", 2020-09-08T16:10:47.261 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.150469405 2020-09-08T16:10:47.261 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.262 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.262 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:47.262 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.959741+0000", 2020-09-08T16:10:47.262 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0043830730000000004 2020-09-08T16:10:47.262 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.263 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.263 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:47.263 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.964124+0000", 2020-09-08T16:10:47.263 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019646299999999999 2020-09-08T16:10:47.263 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.264 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.264 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:47.264 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.964320+0000", 2020-09-08T16:10:47.264 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012814099999999999 2020-09-08T16:10:47.265 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.265 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.265 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:47.265 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.964449+0000", 2020-09-08T16:10:47.265 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.155656303 2020-09-08T16:10:47.266 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.266 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:47.266 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.266 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.266 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.267 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:69112 9.3 9:c574032c:::benchmark_data_smithi166_56754_object69111:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e714)", 2020-09-08T16:10:47.267 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:10:01.811777+0000", 2020-09-08T16:10:47.267 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 45.149450414, 2020-09-08T16:10:47.267 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.161436313, 2020-09-08T16:10:47.267 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:47.267 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:47.268 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:47.268 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:10:47.268 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:10:47.268 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 69112 2020-09-08T16:10:47.268 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.269 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:47.269 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.269 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:47.269 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.811777+0000", 2020-09-08T16:10:47.269 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:47.270 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.270 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.270 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:47.270 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.811777+0000", 2020-09-08T16:10:47.270 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999781 2020-09-08T16:10:47.271 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.271 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.271 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:47.271 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.811754+0000", 2020-09-08T16:10:47.271 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.9340999999999996e-05 2020-09-08T16:10:47.272 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.272 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.272 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:47.272 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.811854+0000", 2020-09-08T16:10:47.272 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.9099999999999994e-06 2020-09-08T16:10:47.272 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.273 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.273 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:47.273 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.811863+0000", 2020-09-08T16:10:47.273 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.4989999999999996e-06 2020-09-08T16:10:47.273 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.274 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.274 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:47.274 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.811867+0000", 2020-09-08T16:10:47.274 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.7877999999999998e-05 2020-09-08T16:10:47.274 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.275 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.275 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:47.275 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.811915+0000", 2020-09-08T16:10:47.275 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000119214 2020-09-08T16:10:47.275 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.276 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.276 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:47.276 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.812034+0000", 2020-09-08T16:10:47.276 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.141770706 2020-09-08T16:10:47.276 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.276 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.277 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 3", 2020-09-08T16:10:47.277 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.953805+0000", 2020-09-08T16:10:47.277 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.010767796 2020-09-08T16:10:47.277 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.277 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.278 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:47.278 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.964573+0000", 2020-09-08T16:10:47.278 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0085415720000000007 2020-09-08T16:10:47.278 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.278 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.279 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:47.279 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.973114+0000", 2020-09-08T16:10:47.279 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.4672000000000002e-05 2020-09-08T16:10:47.279 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.279 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.279 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:47.280 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.973159+0000", 2020-09-08T16:10:47.280 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.3891000000000003e-05 2020-09-08T16:10:47.280 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.280 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.280 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:47.281 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.973213+0000", 2020-09-08T16:10:47.281 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.161436313 2020-09-08T16:10:47.281 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.281 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:47.281 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.282 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.282 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.282 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_repop(client.26587.0:69108 9.1 e714/702 9:85502845:::benchmark_data_smithi166_56754_object69107:head v 714'4410, mlcod=714'4408)", 2020-09-08T16:10:47.282 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:10:01.812000+0000", 2020-09-08T16:10:47.282 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 45.149226556000002, 2020-09-08T16:10:47.283 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16010317700000001, 2020-09-08T16:10:47.283 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:47.283 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:47.283 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:47.283 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.284 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:47.284 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.812000+0000", 2020-09-08T16:10:47.284 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:47.284 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.285 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.285 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:47.285 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.812000+0000", 2020-09-08T16:10:47.285 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5067000000000006e-05 2020-09-08T16:10:47.285 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.286 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.286 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:47.286 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.812066+0000", 2020-09-08T16:10:47.286 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0359999999999997e-06 2020-09-08T16:10:47.286 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.286 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.287 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:47.287 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.812071+0000", 2020-09-08T16:10:47.287 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.8590000000000005e-06 2020-09-08T16:10:47.287 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.287 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.288 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:47.288 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.812078+0000", 2020-09-08T16:10:47.288 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.914e-06 2020-09-08T16:10:47.288 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.288 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.289 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:47.289 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.812081+0000", 2020-09-08T16:10:47.289 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.752e-05 2020-09-08T16:10:47.289 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.289 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.289 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:47.290 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.812119+0000", 2020-09-08T16:10:47.290 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14157191 2020-09-08T16:10:47.290 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.290 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.290 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:47.291 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.953691+0000", 2020-09-08T16:10:47.291 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.018320752999999999 2020-09-08T16:10:47.291 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.291 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.291 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:47.292 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.972012+0000", 2020-09-08T16:10:47.292 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2118000000000001e-05 2020-09-08T16:10:47.292 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.292 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.292 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:47.292 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:01.972104+0000", 2020-09-08T16:10:47.293 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16010317700000001 2020-09-08T16:10:47.293 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.293 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:47.293 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.293 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.294 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.294 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:92673 9.3 9:ca0e48e1:::benchmark_data_smithi166_56754_object92672:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e721)", 2020-09-08T16:10:47.294 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:10:18.025811+0000", 2020-09-08T16:10:47.294 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 28.935416354000001, 2020-09-08T16:10:47.294 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15498917200000001, 2020-09-08T16:10:47.295 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:47.295 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:47.295 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:47.295 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:10:47.295 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:10:47.295 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 92673 2020-09-08T16:10:47.296 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.296 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:47.296 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.296 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:47.296 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.025811+0000", 2020-09-08T16:10:47.297 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:47.297 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.297 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.297 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:47.297 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.025811+0000", 2020-09-08T16:10:47.298 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999747 2020-09-08T16:10:47.298 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.298 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.298 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:47.298 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.025785+0000", 2020-09-08T16:10:47.299 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000129448 2020-09-08T16:10:47.299 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.299 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.299 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:47.299 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.025915+0000", 2020-09-08T16:10:47.299 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.8858000000000001e-05 2020-09-08T16:10:47.300 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.300 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.300 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:47.300 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.025934+0000", 2020-09-08T16:10:47.300 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.613e-06 2020-09-08T16:10:47.301 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.301 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.301 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:47.301 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.025940+0000", 2020-09-08T16:10:47.301 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8489999999999997e-05 2020-09-08T16:10:47.302 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.302 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.302 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:47.302 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.025999+0000", 2020-09-08T16:10:47.302 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000295746 2020-09-08T16:10:47.302 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.303 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.303 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:47.303 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.026295+0000", 2020-09-08T16:10:47.303 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000107837 2020-09-08T16:10:47.303 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.304 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.304 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 3", 2020-09-08T16:10:47.304 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.026402+0000", 2020-09-08T16:10:47.304 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0054442609999999997 2020-09-08T16:10:47.304 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.305 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.305 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:47.305 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.031847+0000", 2020-09-08T16:10:47.305 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14880036499999999 2020-09-08T16:10:47.306 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.306 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.306 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:47.306 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.180647+0000", 2020-09-08T16:10:47.306 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011627 2020-09-08T16:10:47.307 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.307 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.307 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:47.307 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.180763+0000", 2020-09-08T16:10:47.307 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.6473999999999997e-05 2020-09-08T16:10:47.308 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.308 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.308 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:47.308 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.180800+0000", 2020-09-08T16:10:47.308 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15498917200000001 2020-09-08T16:10:47.309 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.309 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:47.309 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.309 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.309 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.310 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:92686 9.3 9:cf56f641:::benchmark_data_smithi166_56754_object92685:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e721)", 2020-09-08T16:10:47.310 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:10:18.027123+0000", 2020-09-08T16:10:47.310 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 28.934104379000001, 2020-09-08T16:10:47.310 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15583014100000001, 2020-09-08T16:10:47.311 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:10:47.311 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:10:47.311 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:10:47.311 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:10:47.311 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:10:47.311 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 92686 2020-09-08T16:10:47.312 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.312 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:10:47.312 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.312 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:10:47.312 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.027123+0000", 2020-09-08T16:10:47.313 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:10:47.313 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.313 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.313 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:10:47.313 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.027123+0000", 2020-09-08T16:10:47.314 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999652 2020-09-08T16:10:47.314 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.314 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.314 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:10:47.314 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.027088+0000", 2020-09-08T16:10:47.314 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000112254 2020-09-08T16:10:47.315 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.315 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.315 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:10:47.316 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.027200+0000", 2020-09-08T16:10:47.317 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3337e-05 2020-09-08T16:10:47.317 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.317 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.317 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:10:47.317 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.027213+0000", 2020-09-08T16:10:47.318 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.1800000000000001e-06 2020-09-08T16:10:47.318 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.318 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.318 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:10:47.318 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.027219+0000", 2020-09-08T16:10:47.319 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.001868691 2020-09-08T16:10:47.319 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.319 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.319 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:10:47.319 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.029088+0000", 2020-09-08T16:10:47.320 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000160839 2020-09-08T16:10:47.320 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.320 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.320 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:10:47.320 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.029249+0000", 2020-09-08T16:10:47.321 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000174026 2020-09-08T16:10:47.321 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.321 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.321 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 3", 2020-09-08T16:10:47.321 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.029423+0000", 2020-09-08T16:10:47.322 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0026502869999999999 2020-09-08T16:10:47.322 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.323 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.323 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:10:47.323 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.032073+0000", 2020-09-08T16:10:47.323 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15051096799999999 2020-09-08T16:10:47.323 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.324 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.324 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:10:47.324 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.182584+0000", 2020-09-08T16:10:47.324 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000215093 2020-09-08T16:10:47.324 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.325 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.325 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:10:47.325 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.182799+0000", 2020-09-08T16:10:47.325 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015341399999999999 2020-09-08T16:10:47.326 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:10:47.326 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:10:47.326 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:10:47.326 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:18.182953+0000", 2020-09-08T16:10:47.326 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15583014100000001 2020-09-08T16:10:47.327 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.327 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:47.327 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.327 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:10:47.327 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:10:47.328 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:10:47.329 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:10:46 smithi104 bash[22305]: debug 2020-09-08T16:10:46.781+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:47.330 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T16:10:47.330 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:47.330 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:10:47 smithi055 bash[11956]: debug 2020-09-08T16:10:47.018+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:47.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:47 smithi166 bash[9446]: cluster 2020-09-08T16:10:46.131297+0000 mgr.x (mgr.24118) 1303 : cluster [DBG] pgmap v1947: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s wr, 961 op/s 2020-09-08T16:10:47.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:47 smithi166 bash[9446]: audit 2020-09-08T16:10:46.871763+0000 mon.a (mon.0) 2271 : audit [INF] from='client.? v1:172.21.15.104:0/1175831554' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:10:47.431 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:10:47.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:47 smithi055 bash[9296]: cluster 2020-09-08T16:10:46.131297+0000 mgr.x (mgr.24118) 1303 : cluster [DBG] pgmap v1947: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s wr, 961 op/s 2020-09-08T16:10:47.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:47 smithi055 bash[9296]: audit 2020-09-08T16:10:46.871763+0000 mon.a (mon.0) 2271 : audit [INF] from='client.? v1:172.21.15.104:0/1175831554' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:10:47.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:47 smithi104 bash[9203]: cluster 2020-09-08T16:10:46.131297+0000 mgr.x (mgr.24118) 1303 : cluster [DBG] pgmap v1947: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 7.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s wr, 961 op/s 2020-09-08T16:10:47.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:47 smithi104 bash[9203]: audit 2020-09-08T16:10:46.871763+0000 mon.a (mon.0) 2271 : audit [INF] from='client.? v1:172.21.15.104:0/1175831554' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:10:47.793 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 76 16 124998 124982 102.769 65.6875 0.00689826 0.00972883 2020-09-08T16:10:48.054 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2020-09-08T16:10:48.180 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:10:48.217 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:10:48.284 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:10:48.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:48 smithi166 bash[9446]: cluster 2020-09-08T16:10:47.213357+0000 mon.a (mon.0) 2272 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:10:48.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:48 smithi166 bash[9446]: audit 2020-09-08T16:10:47.223394+0000 mon.a (mon.0) 2273 : audit [INF] from='client.? v1:172.21.15.104:0/1175831554' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:10:48.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:48 smithi166 bash[9446]: cluster 2020-09-08T16:10:47.223432+0000 mon.a (mon.0) 2274 : cluster [DBG] osdmap e729: 12 total, 12 up, 12 in 2020-09-08T16:10:48.403 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:10:48.508 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:48.516 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:10:48 smithi104 bash[17588]: debug 2020-09-08T16:10:48.181+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:48.516 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:48 smithi104 bash[9203]: cluster 2020-09-08T16:10:47.213357+0000 mon.a (mon.0) 2272 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:10:48.516 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:48 smithi104 bash[9203]: audit 2020-09-08T16:10:47.223394+0000 mon.a (mon.0) 2273 : audit [INF] from='client.? v1:172.21.15.104:0/1175831554' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:10:48.517 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:48 smithi104 bash[9203]: cluster 2020-09-08T16:10:47.223432+0000 mon.a (mon.0) 2274 : cluster [DBG] osdmap e729: 12 total, 12 up, 12 in 2020-09-08T16:10:48.660 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:10:48 smithi055 bash[19003]: debug 2020-09-08T16:10:48.406+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:48.660 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:48 smithi055 bash[9296]: cluster 2020-09-08T16:10:47.213357+0000 mon.a (mon.0) 2272 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:10:48.660 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:48 smithi055 bash[9296]: audit 2020-09-08T16:10:47.223394+0000 mon.a (mon.0) 2273 : audit [INF] from='client.? v1:172.21.15.104:0/1175831554' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:10:48.661 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:48 smithi055 bash[9296]: cluster 2020-09-08T16:10:47.223432+0000 mon.a (mon.0) 2274 : cluster [DBG] osdmap e729: 12 total, 12 up, 12 in 2020-09-08T16:10:48.793 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 77 16 126155 126139 102.374 72.3125 0.0134517 0.00976663 2020-09-08T16:10:48.883 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:48.985 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:49.103 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:49.177 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:10:48 smithi104 bash[15273]: debug 2020-09-08T16:10:48.873+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:49.207 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:10:49.329 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:10:49 smithi055 bash[11956]: debug 2020-09-08T16:10:49.106+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:49.329 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:49 smithi055 bash[9296]: cluster 2020-09-08T16:10:48.131917+0000 mgr.x (mgr.24118) 1304 : cluster [DBG] pgmap v1949: 17 pgs: 17 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s wr, 1.09k op/s 2020-09-08T16:10:49.330 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:49 smithi055 bash[9296]: cluster 2020-09-08T16:10:48.311861+0000 osd.1 (osd.1) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:49.330 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:49 smithi055 bash[9296]: cluster 2020-09-08T16:10:48.311946+0000 osd.1 (osd.1) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:49.330 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:10:49.432 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:49.589 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:10:49.592 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:49 smithi104 bash[9203]: cluster 2020-09-08T16:10:48.131917+0000 mgr.x (mgr.24118) 1304 : cluster [DBG] pgmap v1949: 17 pgs: 17 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s wr, 1.09k op/s 2020-09-08T16:10:49.592 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:49 smithi104 bash[9203]: cluster 2020-09-08T16:10:48.311861+0000 osd.1 (osd.1) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:49.592 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:49 smithi104 bash[9203]: cluster 2020-09-08T16:10:48.311946+0000 osd.1 (osd.1) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:49.593 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:49.620 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:49 smithi166 bash[9446]: cluster 2020-09-08T16:10:48.131917+0000 mgr.x (mgr.24118) 1304 : cluster [DBG] pgmap v1949: 17 pgs: 17 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s wr, 1.09k op/s 2020-09-08T16:10:49.620 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:49 smithi166 bash[9446]: cluster 2020-09-08T16:10:48.311861+0000 osd.1 (osd.1) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:49.620 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:49 smithi166 bash[9446]: cluster 2020-09-08T16:10:48.311946+0000 osd.1 (osd.1) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:49.623 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:10:49.656 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:10:49 smithi055 bash[16646]: debug 2020-09-08T16:10:49.334+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:49.659 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:10:49.688 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:10:49.697 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:10:49.718 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:10:49.751 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:10:49.785 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:10:49.793 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 78 16 127318 127302 101.993 72.6875 0.010149 0.00980229 2020-09-08T16:10:49.810 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:10:49.815 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:10:49.846 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:10:49.879 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:10:49.914 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:10:49.922 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:10:49.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:10:49 smithi104 bash[19922]: debug 2020-09-08T16:10:49.813+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:49.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:10:49 smithi104 bash[22305]: debug 2020-09-08T16:10:49.593+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:49.953 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:10:49.954 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:10:49.954 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:10:49.954 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:10:49.955 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:10:49.955 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:10:49.955 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:10:49.955 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:10:49.955 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:10:49.956 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:10:49.956 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:10:49.956 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:10:49.956 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:10:50.041 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:10:50.159 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:50.235 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:50 smithi055 bash[9296]: cluster 2020-09-08T16:10:49.230158+0000 mon.a (mon.0) 2275 : cluster [DBG] osdmap e730: 12 total, 12 up, 12 in 2020-09-08T16:10:50.236 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:50 smithi055 bash[9296]: audit 2020-09-08T16:10:49.559607+0000 mon.a (mon.0) 2276 : audit [DBG] from='client.? v1:172.21.15.104:0/867766112' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:10:50.236 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:10:50 smithi055 bash[14301]: debug 2020-09-08T16:10:50.042+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:50.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:50 smithi104 bash[9203]: cluster 2020-09-08T16:10:49.230158+0000 mon.a (mon.0) 2275 : cluster [DBG] osdmap e730: 12 total, 12 up, 12 in 2020-09-08T16:10:50.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:50 smithi104 bash[9203]: audit 2020-09-08T16:10:49.559607+0000 mon.a (mon.0) 2276 : audit [DBG] from='client.? v1:172.21.15.104:0/867766112' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:10:50.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:50 smithi166 bash[9446]: cluster 2020-09-08T16:10:49.230158+0000 mon.a (mon.0) 2275 : cluster [DBG] osdmap e730: 12 total, 12 up, 12 in 2020-09-08T16:10:50.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:50 smithi166 bash[9446]: audit 2020-09-08T16:10:49.559607+0000 mon.a (mon.0) 2276 : audit [DBG] from='client.? v1:172.21.15.104:0/867766112' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:10:50.794 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 79 16 128408 128392 101.564 68.125 0.00983129 0.0098442 2020-09-08T16:10:50.958 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:50.990 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T16:10:51.062 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:51.149 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:10:51.220 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:10:50 smithi104 bash[22305]: debug 2020-09-08T16:10:50.953+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:51.222 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:51.329 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:10:51.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:51 smithi166 bash[9446]: cluster 2020-09-08T16:10:50.132479+0000 mgr.x (mgr.24118) 1305 : cluster [DBG] pgmap v1951: 17 pgs: 17 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s wr, 1.22k op/s 2020-09-08T16:10:51.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:51 smithi055 bash[9296]: cluster 2020-09-08T16:10:50.132479+0000 mgr.x (mgr.24118) 1305 : cluster [DBG] pgmap v1951: 17 pgs: 17 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s wr, 1.22k op/s 2020-09-08T16:10:51.683 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:10:51 smithi104 bash[22305]: debug 2020-09-08T16:10:51.221+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:51.684 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:51 smithi104 bash[9203]: cluster 2020-09-08T16:10:50.132479+0000 mgr.x (mgr.24118) 1305 : cluster [DBG] pgmap v1951: 17 pgs: 17 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s wr, 1.22k op/s 2020-09-08T16:10:51.692 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:10:51.794 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:2020-09-08 16:10:51.798701 min lat: 0.00165771 max lat: 0.375975 avg lat: 0.00987823 2020-09-08T16:10:51.794 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-09-08T16:10:51.794 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 80 16 129587 129571 101.216 73.6875 0.0173814 0.00987823 2020-09-08T16:10:51.797 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:10:52.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:10:51 smithi166 bash[16593]: debug 2020-09-08T16:10:51.687+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:52.308 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:10:52.412 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:10:52.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:10:52 smithi166 bash[11928]: debug 2020-09-08T16:10:52.307+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:52.640 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:10:52.675 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:10:52 smithi104 bash[17588]: debug 2020-09-08T16:10:52.637+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:52.741 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:10:52.794 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 81 16 130701 130685 100.826 69.625 0.00701418 0.00990934 2020-09-08T16:10:52.887 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:10:52.889 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:10:52 smithi166 bash[16593]: debug 2020-09-08T16:10:52.887+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:52.990 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:53.071 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T16:10:53.072 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:10:53.126 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:53.155 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:10:53 smithi166 bash[18960]: debug 2020-09-08T16:10:53.123+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:53.232 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:53.358 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:10:53.444 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:53.514 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:10:53.549 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:53.560 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:10:53 smithi104 bash[15273]: debug 2020-09-08T16:10:53.445+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:53.561 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:53 smithi104 bash[9203]: cluster 2020-09-08T16:10:52.132944+0000 mgr.x (mgr.24118) 1306 : cluster [DBG] pgmap v1952: 17 pgs: 17 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s wr, 695 op/s 2020-09-08T16:10:53.561 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:53 smithi104 bash[9203]: cluster 2020-09-08T16:10:52.180859+0000 osd.6 (osd.6) 119 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:53.561 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:53 smithi104 bash[9203]: cluster 2020-09-08T16:10:52.180996+0000 osd.6 (osd.6) 120 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:53.562 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:53 smithi104 bash[9203]: cluster 2020-09-08T16:10:52.251617+0000 mon.a (mon.0) 2277 : cluster [DBG] osdmap e731: 12 total, 12 up, 12 in 2020-09-08T16:10:53.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:53 smithi166 bash[9446]: cluster 2020-09-08T16:10:52.132944+0000 mgr.x (mgr.24118) 1306 : cluster [DBG] pgmap v1952: 17 pgs: 17 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s wr, 695 op/s 2020-09-08T16:10:53.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:53 smithi166 bash[9446]: cluster 2020-09-08T16:10:52.180859+0000 osd.6 (osd.6) 119 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:53.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:53 smithi166 bash[9446]: cluster 2020-09-08T16:10:52.180996+0000 osd.6 (osd.6) 120 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:53.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:53 smithi166 bash[9446]: cluster 2020-09-08T16:10:52.251617+0000 mon.a (mon.0) 2277 : cluster [DBG] osdmap e731: 12 total, 12 up, 12 in 2020-09-08T16:10:53.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:53 smithi055 bash[9296]: cluster 2020-09-08T16:10:52.132944+0000 mgr.x (mgr.24118) 1306 : cluster [DBG] pgmap v1952: 17 pgs: 17 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s wr, 695 op/s 2020-09-08T16:10:53.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:53 smithi055 bash[9296]: cluster 2020-09-08T16:10:52.180859+0000 osd.6 (osd.6) 119 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:53.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:53 smithi055 bash[9296]: cluster 2020-09-08T16:10:52.180996+0000 osd.6 (osd.6) 120 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:53.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:53 smithi055 bash[9296]: cluster 2020-09-08T16:10:52.251617+0000 mon.a (mon.0) 2277 : cluster [DBG] osdmap e731: 12 total, 12 up, 12 in 2020-09-08T16:10:53.794 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 82 16 131661 131645 100.328 60 0.0221838 0.00996523 2020-09-08T16:10:54.129 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:10:54.232 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:54.312 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2020-09-08T16:10:54.391 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:54.446 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:10:54 smithi104 bash[22305]: debug 2020-09-08T16:10:54.129+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:54.498 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:10:54.518 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:10:54.657 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:10:54 smithi055 bash[11956]: debug 2020-09-08T16:10:54.394+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:54.794 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 83 16 132779 132763 99.9607 69.875 0.0074358 0.010002 2020-09-08T16:10:55.018 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:10:55.122 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:10:55.340 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:10:55.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:10:55 smithi104 bash[17588]: debug 2020-09-08T16:10:55.013+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:55.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:55 smithi104 bash[9203]: cluster 2020-09-08T16:10:54.133374+0000 mgr.x (mgr.24118) 1307 : cluster [DBG] pgmap v1954: 17 pgs: 17 active+clean; 7.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s wr, 1.29k op/s 2020-09-08T16:10:55.446 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:10:55.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:55 smithi166 bash[9446]: cluster 2020-09-08T16:10:54.133374+0000 mgr.x (mgr.24118) 1307 : cluster [DBG] pgmap v1954: 17 pgs: 17 active+clean; 7.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s wr, 1.29k op/s 2020-09-08T16:10:55.617 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:10:55 smithi166 bash[16593]: debug 2020-09-08T16:10:55.343+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:55.642 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:10:55.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:55 smithi055 bash[9296]: cluster 2020-09-08T16:10:54.133374+0000 mgr.x (mgr.24118) 1307 : cluster [DBG] pgmap v1954: 17 pgs: 17 active+clean; 7.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s wr, 1.29k op/s 2020-09-08T16:10:55.746 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:55.794 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 84 16 133784 133768 99.5184 62.8125 0.017465 0.0100466 2020-09-08T16:10:55.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:10:55 smithi104 bash[19922]: debug 2020-09-08T16:10:55.641+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:55.965 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:10:56.074 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:56.117 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:10:55 smithi166 bash[18960]: debug 2020-09-08T16:10:55.963+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:56.185 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:56.289 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:10:56.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:10:56 smithi055 bash[11956]: debug 2020-09-08T16:10:56.189+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:56.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:56 smithi055 bash[9296]: cluster 2020-09-08T16:10:55.852907+0000 osd.11 (osd.11) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:56.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:56 smithi055 bash[9296]: cluster 2020-09-08T16:10:55.852993+0000 osd.11 (osd.11) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:56.487 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:56 smithi166 bash[9446]: cluster 2020-09-08T16:10:55.852907+0000 osd.11 (osd.11) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:56.488 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:56 smithi166 bash[9446]: cluster 2020-09-08T16:10:55.852993+0000 osd.11 (osd.11) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:56.488 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:10:56.518 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:56 smithi104 bash[9203]: cluster 2020-09-08T16:10:55.852907+0000 osd.11 (osd.11) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:10:56.518 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:56 smithi104 bash[9203]: cluster 2020-09-08T16:10:55.852993+0000 osd.11 (osd.11) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:10:56.591 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:10:56.794 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 85 16 134761 134745 99.066 61.0625 0.0123888 0.0100913 2020-09-08T16:10:56.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:10:56 smithi166 bash[11928]: debug 2020-09-08T16:10:56.487+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:57.223 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:10:57.328 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:10:57.592 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:57 smithi104 bash[9203]: cluster 2020-09-08T16:10:56.133831+0000 mgr.x (mgr.24118) 1308 : cluster [DBG] pgmap v1955: 17 pgs: 17 active+clean; 8.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s wr, 824 op/s 2020-09-08T16:10:57.592 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:57 smithi104 bash[9203]: cluster 2020-09-08T16:10:56.261861+0000 mon.a (mon.0) 2278 : cluster [DBG] osdmap e732: 12 total, 12 up, 12 in 2020-09-08T16:10:57.592 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:10:57 smithi104 bash[17588]: debug 2020-09-08T16:10:57.225+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:57.593 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:10:57 smithi104 bash[17588]: debug 2020-09-08T16:10:57.593+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:57.594 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:10:57.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:57 smithi166 bash[9446]: cluster 2020-09-08T16:10:56.133831+0000 mgr.x (mgr.24118) 1308 : cluster [DBG] pgmap v1955: 17 pgs: 17 active+clean; 8.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s wr, 824 op/s 2020-09-08T16:10:57.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:57 smithi166 bash[9446]: cluster 2020-09-08T16:10:56.261861+0000 mon.a (mon.0) 2278 : cluster [DBG] osdmap e732: 12 total, 12 up, 12 in 2020-09-08T16:10:57.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:57 smithi055 bash[9296]: cluster 2020-09-08T16:10:56.133831+0000 mgr.x (mgr.24118) 1308 : cluster [DBG] pgmap v1955: 17 pgs: 17 active+clean; 8.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s wr, 824 op/s 2020-09-08T16:10:57.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:57 smithi055 bash[9296]: cluster 2020-09-08T16:10:56.261861+0000 mon.a (mon.0) 2278 : cluster [DBG] osdmap e732: 12 total, 12 up, 12 in 2020-09-08T16:10:57.698 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:57.794 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 86 16 135857 135841 98.7105 68.5 0.0105382 0.0101291 2020-09-08T16:10:57.906 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:10:58.007 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:58.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:10:57 smithi055 bash[11956]: debug 2020-09-08T16:10:57.817+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:58.271 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T16:10:58.342 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:58 smithi104 bash[9203]: audit 2020-09-08T16:10:57.774930+0000 mon.a (mon.0) 2279 : audit [INF] from='client.? v1:172.21.15.104:0/1507483442' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:10:58.366 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:10:58.470 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:10:58.594 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:58 smithi055 bash[9296]: audit 2020-09-08T16:10:57.774930+0000 mon.a (mon.0) 2279 : audit [INF] from='client.? v1:172.21.15.104:0/1507483442' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:10:58.595 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:10:58.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:58 smithi166 bash[9446]: audit 2020-09-08T16:10:57.774930+0000 mon.a (mon.0) 2279 : audit [INF] from='client.? v1:172.21.15.104:0/1507483442' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:10:58.675 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:10:58 smithi104 bash[15273]: debug 2020-09-08T16:10:58.365+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:58.696 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:10:58.794 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 87 16 136960 136944 98.3682 68.9375 0.0232007 0.0101638 2020-09-08T16:10:58.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:10:58 smithi055 bash[14301]: debug 2020-09-08T16:10:58.593+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:59.234 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:10:59.338 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:10:59.350 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:59 smithi166 bash[9446]: cluster 2020-09-08T16:10:58.134375+0000 mgr.x (mgr.24118) 1309 : cluster [DBG] pgmap v1957: 17 pgs: 17 active+clean; 8.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T16:10:59.350 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:59 smithi166 bash[9446]: cluster 2020-09-08T16:10:58.260835+0000 mon.a (mon.0) 2280 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:10:59.351 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:59 smithi166 bash[9446]: audit 2020-09-08T16:10:58.272019+0000 mon.a (mon.0) 2281 : audit [INF] from='client.? v1:172.21.15.104:0/1507483442' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:10:59.351 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:59 smithi166 bash[9446]: cluster 2020-09-08T16:10:58.272077+0000 mon.a (mon.0) 2282 : cluster [DBG] osdmap e733: 12 total, 12 up, 12 in 2020-09-08T16:10:59.351 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:10:59 smithi166 bash[9446]: cluster 2020-09-08T16:10:58.681698+0000 osd.11 (osd.11) 132 : cluster [DBG] 9.5 scrub starts 2020-09-08T16:10:59.568 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T16:10:59.570 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:10:59.602 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:10:59.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:59 smithi055 bash[9296]: cluster 2020-09-08T16:10:58.134375+0000 mgr.x (mgr.24118) 1309 : cluster [DBG] pgmap v1957: 17 pgs: 17 active+clean; 8.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T16:10:59.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:59 smithi055 bash[9296]: cluster 2020-09-08T16:10:58.260835+0000 mon.a (mon.0) 2280 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:10:59.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:59 smithi055 bash[9296]: audit 2020-09-08T16:10:58.272019+0000 mon.a (mon.0) 2281 : audit [INF] from='client.? v1:172.21.15.104:0/1507483442' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:10:59.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:59 smithi055 bash[9296]: cluster 2020-09-08T16:10:58.272077+0000 mon.a (mon.0) 2282 : cluster [DBG] osdmap e733: 12 total, 12 up, 12 in 2020-09-08T16:10:59.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:10:59 smithi055 bash[9296]: cluster 2020-09-08T16:10:58.681698+0000 osd.11 (osd.11) 132 : cluster [DBG] 9.5 scrub starts 2020-09-08T16:10:59.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:59 smithi104 bash[9203]: cluster 2020-09-08T16:10:58.134375+0000 mgr.x (mgr.24118) 1309 : cluster [DBG] pgmap v1957: 17 pgs: 17 active+clean; 8.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s wr, 1.23k op/s 2020-09-08T16:10:59.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:59 smithi104 bash[9203]: cluster 2020-09-08T16:10:58.260835+0000 mon.a (mon.0) 2280 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:10:59.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:59 smithi104 bash[9203]: audit 2020-09-08T16:10:58.272019+0000 mon.a (mon.0) 2281 : audit [INF] from='client.? v1:172.21.15.104:0/1507483442' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:10:59.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:59 smithi104 bash[9203]: cluster 2020-09-08T16:10:58.272077+0000 mon.a (mon.0) 2282 : cluster [DBG] osdmap e733: 12 total, 12 up, 12 in 2020-09-08T16:10:59.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:10:59 smithi104 bash[9203]: cluster 2020-09-08T16:10:58.681698+0000 osd.11 (osd.11) 132 : cluster [DBG] 9.5 scrub starts 2020-09-08T16:10:59.677 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:10:59 smithi104 bash[17588]: debug 2020-09-08T16:10:59.233+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:59.706 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:10:59.794 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 88 16 138057 138041 98.0295 68.5625 0.0176492 0.0101992 2020-09-08T16:10:59.834 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:10:59.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:10:59 smithi166 bash[16593]: debug 2020-09-08T16:10:59.599+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:10:59.937 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:11:00.045 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:11:00 smithi055 bash[16646]: debug 2020-09-08T16:11:00.045+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:00.045 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:10:59 smithi055 bash[19003]: debug 2020-09-08T16:10:59.837+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:00.049 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:11:00.157 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:00.753 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:00.794 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 89 16 139057 139041 97.6302 62.5 0.0119865 0.0102411 2020-09-08T16:11:00.856 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:11:00.876 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2020-09-08T16:11:00.972 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:11:00.981 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:11:01.005 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:11:01.036 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:11:01.071 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:11:01.077 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:01 smithi166 bash[9446]: cluster 2020-09-08T16:11:00.073657+0000 osd.7 (osd.7) 157 : cluster [DBG] 9.d scrub starts 2020-09-08T16:11:01.078 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:01 smithi166 bash[9446]: cluster 2020-09-08T16:11:00.134841+0000 mgr.x (mgr.24118) 1310 : cluster [DBG] pgmap v1959: 17 pgs: 17 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.24k op/s 2020-09-08T16:11:01.078 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:01 smithi166 bash[9446]: audit 2020-09-08T16:11:00.948763+0000 mon.a (mon.0) 2283 : audit [DBG] from='client.? v1:172.21.15.104:0/3511351647' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:11:01.078 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:11:00 smithi166 bash[11928]: debug 2020-09-08T16:11:00.755+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:01.079 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:11:00 smithi166 bash[18960]: debug 2020-09-08T16:11:00.979+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:01.083 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:11:01.098 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:11:01.109 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:11:01.143 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:11:01.178 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:11:01.211 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:11:01.241 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:11:01.273 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:11:01.305 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:11:01.341 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:11:01.344 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:11:01.344 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:11:01.344 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:11:01.345 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:11:01.345 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:11:01.345 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:11:01.345 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:11:01.346 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:11:01.346 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:11:01.346 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:11:01.346 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:11:01.346 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:11:01.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:01 smithi055 bash[9296]: cluster 2020-09-08T16:11:00.073657+0000 osd.7 (osd.7) 157 : cluster [DBG] 9.d scrub starts 2020-09-08T16:11:01.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:01 smithi055 bash[9296]: cluster 2020-09-08T16:11:00.134841+0000 mgr.x (mgr.24118) 1310 : cluster [DBG] pgmap v1959: 17 pgs: 17 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.24k op/s 2020-09-08T16:11:01.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:01 smithi055 bash[9296]: audit 2020-09-08T16:11:00.948763+0000 mon.a (mon.0) 2283 : audit [DBG] from='client.? v1:172.21.15.104:0/3511351647' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:11:01.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:01 smithi104 bash[9203]: cluster 2020-09-08T16:11:00.073657+0000 osd.7 (osd.7) 157 : cluster [DBG] 9.d scrub starts 2020-09-08T16:11:01.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:01 smithi104 bash[9203]: cluster 2020-09-08T16:11:00.134841+0000 mgr.x (mgr.24118) 1310 : cluster [DBG] pgmap v1959: 17 pgs: 17 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s wr, 1.24k op/s 2020-09-08T16:11:01.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:01 smithi104 bash[9203]: audit 2020-09-08T16:11:00.948763+0000 mon.a (mon.0) 2283 : audit [DBG] from='client.? v1:172.21.15.104:0/3511351647' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:11:01.795 INFO:tasks.radosbench.radosbench.2.smithi166.stdout: 90 15 140030 140015 97.2218 60.875 0.0220826 0.0102827 2020-09-08T16:11:01.956 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:11:02.063 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:02.234 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:02.235 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:11:01 smithi166 bash[18960]: debug 2020-09-08T16:11:01.955+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:02.235 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:02 smithi166 bash[9446]: cluster 2020-09-08T16:11:00.910638+0000 osd.3 (osd.3) 127 : cluster [DBG] 9.c scrub starts 2020-09-08T16:11:02.235 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:02 smithi166 bash[9446]: cluster 2020-09-08T16:11:01.211208+0000 osd.4 (osd.4) 79 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:02.235 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:02 smithi166 bash[9446]: cluster 2020-09-08T16:11:01.211269+0000 osd.4 (osd.4) 80 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:02.236 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:02 smithi166 bash[9446]: cluster 2020-09-08T16:11:01.716514+0000 osd.11 (osd.11) 133 : cluster [DBG] 9.5 scrub ok 2020-09-08T16:11:02.258 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T16:11:02.337 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:02.345 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:02 smithi104 bash[9203]: cluster 2020-09-08T16:11:00.910638+0000 osd.3 (osd.3) 127 : cluster [DBG] 9.c scrub starts 2020-09-08T16:11:02.345 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:02 smithi104 bash[9203]: cluster 2020-09-08T16:11:01.211208+0000 osd.4 (osd.4) 79 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:02.346 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:02 smithi104 bash[9203]: cluster 2020-09-08T16:11:01.211269+0000 osd.4 (osd.4) 80 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:02.346 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:02 smithi104 bash[9203]: cluster 2020-09-08T16:11:01.716514+0000 osd.11 (osd.11) 133 : cluster [DBG] 9.5 scrub ok 2020-09-08T16:11:02.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:02 smithi055 bash[9296]: cluster 2020-09-08T16:11:00.910638+0000 osd.3 (osd.3) 127 : cluster [DBG] 9.c scrub starts 2020-09-08T16:11:02.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:02 smithi055 bash[9296]: cluster 2020-09-08T16:11:01.211208+0000 osd.4 (osd.4) 79 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:02.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:02 smithi055 bash[9296]: cluster 2020-09-08T16:11:01.211269+0000 osd.4 (osd.4) 80 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:02.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:02 smithi055 bash[9296]: cluster 2020-09-08T16:11:01.716514+0000 osd.11 (osd.11) 133 : cluster [DBG] 9.5 scrub ok 2020-09-08T16:11:02.413 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:11:02.616 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:11:02 smithi166 bash[14254]: debug 2020-09-08T16:11:02.231+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:02.810 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Total time run: 90.0114 2020-09-08T16:11:02.810 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Total writes made: 140030 2020-09-08T16:11:02.810 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Write size: 65536 2020-09-08T16:11:02.811 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Object size: 65536 2020-09-08T16:11:02.811 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Bandwidth (MB/sec): 97.2307 2020-09-08T16:11:02.811 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev Bandwidth: 40.9898 2020-09-08T16:11:02.811 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max bandwidth (MB/sec): 224.812 2020-09-08T16:11:02.812 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min bandwidth (MB/sec): 53.1875 2020-09-08T16:11:02.812 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Average IOPS: 1555 2020-09-08T16:11:02.812 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev IOPS: 655 2020-09-08T16:11:02.813 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max IOPS: 3597 2020-09-08T16:11:02.813 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min IOPS: 851 2020-09-08T16:11:02.813 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Average Latency(s): 0.0102838 2020-09-08T16:11:02.813 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Stddev Latency(s): 0.0142214 2020-09-08T16:11:02.814 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Max latency(s): 0.375975 2020-09-08T16:11:02.814 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Min latency(s): 0.00165771 2020-09-08T16:11:02.814 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Cleaning up (deleting benchmark objects) 2020-09-08T16:11:03.014 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:03.120 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:11:03.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:03 smithi166 bash[9446]: cluster 2020-09-08T16:11:02.090366+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e734: 12 total, 12 up, 12 in 2020-09-08T16:11:03.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:03 smithi166 bash[9446]: cluster 2020-09-08T16:11:02.135345+0000 mgr.x (mgr.24118) 1311 : cluster [DBG] pgmap v1961: 17 pgs: 17 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s wr, 866 op/s 2020-09-08T16:11:03.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:03 smithi166 bash[9446]: cluster 2020-09-08T16:11:02.222680+0000 osd.7 (osd.7) 158 : cluster [DBG] 9.d scrub ok 2020-09-08T16:11:03.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:03 smithi166 bash[9446]: cluster 2020-09-08T16:11:02.679975+0000 osd.5 (osd.5) 124 : cluster [DBG] 9.f scrub starts 2020-09-08T16:11:03.374 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:11:03 smithi104 bash[19922]: debug 2020-09-08T16:11:03.017+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:03.374 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:03 smithi104 bash[9203]: cluster 2020-09-08T16:11:02.090366+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e734: 12 total, 12 up, 12 in 2020-09-08T16:11:03.375 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:03 smithi104 bash[9203]: cluster 2020-09-08T16:11:02.135345+0000 mgr.x (mgr.24118) 1311 : cluster [DBG] pgmap v1961: 17 pgs: 17 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s wr, 866 op/s 2020-09-08T16:11:03.375 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:03 smithi104 bash[9203]: cluster 2020-09-08T16:11:02.222680+0000 osd.7 (osd.7) 158 : cluster [DBG] 9.d scrub ok 2020-09-08T16:11:03.375 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:03 smithi104 bash[9203]: cluster 2020-09-08T16:11:02.679975+0000 osd.5 (osd.5) 124 : cluster [DBG] 9.f scrub starts 2020-09-08T16:11:03.376 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:11:03.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:03 smithi055 bash[9296]: cluster 2020-09-08T16:11:02.090366+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e734: 12 total, 12 up, 12 in 2020-09-08T16:11:03.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:03 smithi055 bash[9296]: cluster 2020-09-08T16:11:02.135345+0000 mgr.x (mgr.24118) 1311 : cluster [DBG] pgmap v1961: 17 pgs: 17 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s wr, 866 op/s 2020-09-08T16:11:03.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:03 smithi055 bash[9296]: cluster 2020-09-08T16:11:02.222680+0000 osd.7 (osd.7) 158 : cluster [DBG] 9.d scrub ok 2020-09-08T16:11:03.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:03 smithi055 bash[9296]: cluster 2020-09-08T16:11:02.679975+0000 osd.5 (osd.5) 124 : cluster [DBG] 9.f scrub starts 2020-09-08T16:11:03.478 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:03.629 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:03.678 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:11:03 smithi104 bash[17588]: debug 2020-09-08T16:11:03.377+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:03.733 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:11:03.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:11:03 smithi055 bash[14301]: debug 2020-09-08T16:11:03.629+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:04.095 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:04 smithi166 bash[9446]: cluster 2020-09-08T16:11:02.862006+0000 osd.3 (osd.3) 128 : cluster [DBG] 9.c scrub ok 2020-09-08T16:11:04.095 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:04 smithi166 bash[9446]: cluster 2020-09-08T16:11:02.898849+0000 osd.3 (osd.3) 129 : cluster [DBG] 9.1 scrub starts 2020-09-08T16:11:04.096 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:04 smithi166 bash[9446]: cluster 2020-09-08T16:11:03.126434+0000 osd.10 (osd.10) 107 : cluster [DBG] 9.9 scrub starts 2020-09-08T16:11:04.266 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:11:04.366 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:11:04 smithi166 bash[18960]: debug 2020-09-08T16:11:04.267+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:04.373 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:11:04.382 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:04 smithi104 bash[9203]: cluster 2020-09-08T16:11:02.862006+0000 osd.3 (osd.3) 128 : cluster [DBG] 9.c scrub ok 2020-09-08T16:11:04.382 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:04 smithi104 bash[9203]: cluster 2020-09-08T16:11:02.898849+0000 osd.3 (osd.3) 129 : cluster [DBG] 9.1 scrub starts 2020-09-08T16:11:04.383 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:04 smithi104 bash[9203]: cluster 2020-09-08T16:11:03.126434+0000 osd.10 (osd.10) 107 : cluster [DBG] 9.9 scrub starts 2020-09-08T16:11:04.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:04 smithi055 bash[9296]: cluster 2020-09-08T16:11:02.862006+0000 osd.3 (osd.3) 128 : cluster [DBG] 9.c scrub ok 2020-09-08T16:11:04.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:04 smithi055 bash[9296]: cluster 2020-09-08T16:11:02.898849+0000 osd.3 (osd.3) 129 : cluster [DBG] 9.1 scrub starts 2020-09-08T16:11:04.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:04 smithi055 bash[9296]: cluster 2020-09-08T16:11:03.126434+0000 osd.10 (osd.10) 107 : cluster [DBG] 9.9 scrub starts 2020-09-08T16:11:04.506 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:11:04.613 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:04.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:11:04 smithi104 bash[22305]: debug 2020-09-08T16:11:04.505+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:05.103 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:05 smithi166 bash[9446]: cluster 2020-09-08T16:11:04.135944+0000 mgr.x (mgr.24118) 1312 : cluster [DBG] pgmap v1962: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 8.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 70 MiB/s wr, 1.14k op/s 2020-09-08T16:11:05.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:05 smithi104 bash[9203]: cluster 2020-09-08T16:11:04.135944+0000 mgr.x (mgr.24118) 1312 : cluster [DBG] pgmap v1962: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 8.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 70 MiB/s wr, 1.14k op/s 2020-09-08T16:11:05.243 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:05.348 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:11:05.358 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:05 smithi055 bash[9296]: cluster 2020-09-08T16:11:04.135944+0000 mgr.x (mgr.24118) 1312 : cluster [DBG] pgmap v1962: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 8.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 70 MiB/s wr, 1.14k op/s 2020-09-08T16:11:05.383 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:11:05.404 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:11:05 smithi166 bash[16593]: debug 2020-09-08T16:11:05.239+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:05.536 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:11:06.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:06 smithi166 bash[9446]: cluster 2020-09-08T16:11:05.237379+0000 osd.5 (osd.5) 125 : cluster [DBG] 9.f scrub ok 2020-09-08T16:11:06.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:06 smithi166 bash[9446]: cluster 2020-09-08T16:11:05.683670+0000 osd.5 (osd.5) 126 : cluster [DBG] 9.a scrub starts 2020-09-08T16:11:06.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:06 smithi055 bash[9296]: cluster 2020-09-08T16:11:05.237379+0000 osd.5 (osd.5) 125 : cluster [DBG] 9.f scrub ok 2020-09-08T16:11:06.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:06 smithi055 bash[9296]: cluster 2020-09-08T16:11:05.683670+0000 osd.5 (osd.5) 126 : cluster [DBG] 9.a scrub starts 2020-09-08T16:11:06.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:06 smithi104 bash[9203]: cluster 2020-09-08T16:11:05.237379+0000 osd.5 (osd.5) 125 : cluster [DBG] 9.f scrub ok 2020-09-08T16:11:06.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:06 smithi104 bash[9203]: cluster 2020-09-08T16:11:05.683670+0000 osd.5 (osd.5) 126 : cluster [DBG] 9.a scrub starts 2020-09-08T16:11:06.833 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:11:06.834 INFO:teuthology.orchestra.run.smithi166.stdout: "size": 20, 2020-09-08T16:11:06.834 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 600, 2020-09-08T16:11:06.834 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T16:11:06.834 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.835 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:132254 8.5 8:a5834ec5:::benchmark_data_smithi166_52302_object1333:head [delete] snapc 0=[] ondisk+write+known_if_redirected e661)", 2020-09-08T16:11:06.835 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:06:39.254835+0000", 2020-09-08T16:11:06.835 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 267.57848191900001, 2020-09-08T16:11:06.835 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21971317800000001, 2020-09-08T16:11:06.835 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:06.836 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:06.836 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:06.836 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:06.836 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:06.836 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 132254 2020-09-08T16:11:06.837 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.837 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:06.837 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.837 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:06.838 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.254835+0000", 2020-09-08T16:11:06.838 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:06.838 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.838 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.838 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:06.839 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.254835+0000", 2020-09-08T16:11:06.839 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999638 2020-09-08T16:11:06.839 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.839 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.839 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:06.840 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.254799+0000", 2020-09-08T16:11:06.840 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.9721000000000005e-05 2020-09-08T16:11:06.840 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.840 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.840 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:06.840 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.254878+0000", 2020-09-08T16:11:06.841 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3208e-05 2020-09-08T16:11:06.841 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.841 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.841 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:06.841 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.254892+0000", 2020-09-08T16:11:06.842 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0634e-05 2020-09-08T16:11:06.842 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.842 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.842 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:06.842 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.254902+0000", 2020-09-08T16:11:06.843 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00044658099999999999 2020-09-08T16:11:06.843 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.843 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.843 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:06.843 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.255349+0000", 2020-09-08T16:11:06.844 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013150999999999999 2020-09-08T16:11:06.844 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.844 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.844 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:06.844 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.255480+0000", 2020-09-08T16:11:06.845 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000120529 2020-09-08T16:11:06.845 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.845 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.845 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T16:11:06.845 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.255601+0000", 2020-09-08T16:11:06.846 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0041078099999999999 2020-09-08T16:11:06.846 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.846 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.846 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:06.846 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.259709+0000", 2020-09-08T16:11:06.847 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21468620499999999 2020-09-08T16:11:06.847 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.847 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.847 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:06.847 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.474395+0000", 2020-09-08T16:11:06.847 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.1370000000000001e-05 2020-09-08T16:11:06.848 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.848 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.848 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:06.848 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.474486+0000", 2020-09-08T16:11:06.848 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.1773999999999995e-05 2020-09-08T16:11:06.849 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.849 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.849 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:06.849 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.474548+0000", 2020-09-08T16:11:06.849 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21971317800000001 2020-09-08T16:11:06.850 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:06.850 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:06.850 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:06.850 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.850 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.851 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:132266 8.5 8:a28ed302:::benchmark_data_smithi166_52302_object1345:head [delete] snapc 0=[] ondisk+write+known_if_redirected e661)", 2020-09-08T16:11:06.851 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:06:39.267832+0000", 2020-09-08T16:11:06.851 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 267.5654844, 2020-09-08T16:11:06.851 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22094564899999999, 2020-09-08T16:11:06.851 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:06.852 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:06.852 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:06.852 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:06.852 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:06.852 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 132266 2020-09-08T16:11:06.853 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.853 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:06.853 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.853 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:06.853 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.267832+0000", 2020-09-08T16:11:06.853 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:06.854 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.854 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.854 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:06.854 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.267832+0000", 2020-09-08T16:11:06.854 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999595 2020-09-08T16:11:06.855 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.855 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.855 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:06.855 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.267792+0000", 2020-09-08T16:11:06.855 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8356999999999994e-05 2020-09-08T16:11:06.856 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.856 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.856 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:06.856 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.267880+0000", 2020-09-08T16:11:06.856 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3807e-05 2020-09-08T16:11:06.856 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.857 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.857 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:06.857 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.267894+0000", 2020-09-08T16:11:06.857 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.921e-06 2020-09-08T16:11:06.857 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.858 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.858 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:06.858 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.267899+0000", 2020-09-08T16:11:06.858 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.2519999999999999e-05 2020-09-08T16:11:06.858 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.859 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.859 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:06.859 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.267951+0000", 2020-09-08T16:11:06.859 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000150691 2020-09-08T16:11:06.859 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.860 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.860 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:06.860 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.268102+0000", 2020-09-08T16:11:06.860 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000129799 2020-09-08T16:11:06.860 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.860 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.861 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T16:11:06.861 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.268232+0000", 2020-09-08T16:11:06.861 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.013882774000000001 2020-09-08T16:11:06.861 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.861 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.862 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:06.862 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.282115+0000", 2020-09-08T16:11:06.862 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.206591266 2020-09-08T16:11:06.862 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.862 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.863 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:06.863 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.488706+0000", 2020-09-08T16:11:06.863 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.1044000000000002e-05 2020-09-08T16:11:06.863 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.863 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.864 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:06.864 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.488747+0000", 2020-09-08T16:11:06.864 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.1072999999999999e-05 2020-09-08T16:11:06.864 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.864 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.864 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:06.865 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.488778+0000", 2020-09-08T16:11:06.865 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22094564899999999 2020-09-08T16:11:06.865 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:06.865 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:06.866 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:06.866 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.866 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.866 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:132276 8.5 8:a37ef410:::benchmark_data_smithi166_52302_object1355:head [delete] snapc 0=[] ondisk+write+known_if_redirected e661)", 2020-09-08T16:11:06.867 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:06:39.275687+0000", 2020-09-08T16:11:06.867 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 267.55763017200002, 2020-09-08T16:11:06.867 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22171882800000001, 2020-09-08T16:11:06.867 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:06.867 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:06.868 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:06.868 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:06.868 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:06.868 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 132276 2020-09-08T16:11:06.868 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.869 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:06.869 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.869 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:06.869 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.275687+0000", 2020-09-08T16:11:06.870 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:06.870 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.870 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.870 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:06.870 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.275687+0000", 2020-09-08T16:11:06.871 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999752 2020-09-08T16:11:06.871 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.871 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.871 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:06.871 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.275662+0000", 2020-09-08T16:11:06.872 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1811999999999998e-05 2020-09-08T16:11:06.872 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.872 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.872 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:06.873 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.275714+0000", 2020-09-08T16:11:06.873 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.1269999999999992e-06 2020-09-08T16:11:06.873 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.873 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.874 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:06.874 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.275723+0000", 2020-09-08T16:11:06.874 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.8280000000000002e-06 2020-09-08T16:11:06.874 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.875 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.875 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:06.875 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.275728+0000", 2020-09-08T16:11:06.875 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00048667600000000001 2020-09-08T16:11:06.875 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.876 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.876 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:06.876 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.276215+0000", 2020-09-08T16:11:06.876 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016333600000000001 2020-09-08T16:11:06.876 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.877 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.877 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:06.877 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.276378+0000", 2020-09-08T16:11:06.877 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014281499999999999 2020-09-08T16:11:06.877 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.878 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.878 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T16:11:06.878 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.276521+0000", 2020-09-08T16:11:06.878 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0056025399999999996 2020-09-08T16:11:06.878 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.879 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.879 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:06.879 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.282123+0000", 2020-09-08T16:11:06.879 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21513143300000001 2020-09-08T16:11:06.879 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.879 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.880 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:06.880 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.497255+0000", 2020-09-08T16:11:06.880 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.1629e-05 2020-09-08T16:11:06.880 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.880 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.881 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:06.881 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.497346+0000", 2020-09-08T16:11:06.881 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.9191000000000003e-05 2020-09-08T16:11:06.881 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.881 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.882 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:06.882 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.497405+0000", 2020-09-08T16:11:06.882 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.22171882800000001 2020-09-08T16:11:06.882 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:06.882 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:06.883 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:06.883 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.883 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.883 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:132304 8.5 8:a11a7314:::benchmark_data_smithi166_52302_object1383:head [delete] snapc 0=[] ondisk+write+known_if_redirected e661)", 2020-09-08T16:11:06.883 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:06:39.305844+0000", 2020-09-08T16:11:06.884 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 267.52747316199998, 2020-09-08T16:11:06.884 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19973163299999999, 2020-09-08T16:11:06.884 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:06.884 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:06.884 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:06.885 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:06.885 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:06.885 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 132304 2020-09-08T16:11:06.885 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.885 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:06.886 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.886 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:06.886 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.305844+0000", 2020-09-08T16:11:06.886 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:06.886 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.886 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.887 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:06.887 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.305844+0000", 2020-09-08T16:11:06.887 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999495 2020-09-08T16:11:06.887 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.888 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.888 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:06.888 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.305793+0000", 2020-09-08T16:11:06.888 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000111766 2020-09-08T16:11:06.888 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.889 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.889 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:06.889 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.305905+0000", 2020-09-08T16:11:06.889 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.9860999999999998e-05 2020-09-08T16:11:06.890 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.890 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.890 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:06.890 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.305925+0000", 2020-09-08T16:11:06.890 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.5570000000000001e-06 2020-09-08T16:11:06.891 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.891 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.891 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:06.891 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.305934+0000", 2020-09-08T16:11:06.892 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.7732999999999999e-05 2020-09-08T16:11:06.892 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.892 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.892 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:06.892 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.305982+0000", 2020-09-08T16:11:06.893 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000100313 2020-09-08T16:11:06.893 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.893 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.893 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:06.893 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.306082+0000", 2020-09-08T16:11:06.894 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000132321 2020-09-08T16:11:06.894 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.894 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.894 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T16:11:06.894 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.306215+0000", 2020-09-08T16:11:06.895 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0052213219999999996 2020-09-08T16:11:06.895 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.895 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.895 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:06.895 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.311436+0000", 2020-09-08T16:11:06.896 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.193966315 2020-09-08T16:11:06.896 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.896 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.896 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:06.896 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.505402+0000", 2020-09-08T16:11:06.897 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7919999999999998e-05 2020-09-08T16:11:06.897 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.897 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.897 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:06.897 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.505500+0000", 2020-09-08T16:11:06.897 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.5172000000000005e-05 2020-09-08T16:11:06.898 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.898 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.898 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:06.898 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.505575+0000", 2020-09-08T16:11:06.898 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19973163299999999 2020-09-08T16:11:06.899 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:06.899 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:06.899 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:06.899 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.899 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.900 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:132315 8.5 8:ad32a0a6:::benchmark_data_smithi166_52302_object1394:head [delete] snapc 0=[] ondisk+write+known_if_redirected e661)", 2020-09-08T16:11:06.900 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:06:39.317728+0000", 2020-09-08T16:11:06.900 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 267.51558844800002, 2020-09-08T16:11:06.900 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.188069664, 2020-09-08T16:11:06.900 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:06.901 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:06.901 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:06.902 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:06.908 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:06.909 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 132315 2020-09-08T16:11:06.909 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.909 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:06.909 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.910 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:06.910 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.317728+0000", 2020-09-08T16:11:06.910 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:06.910 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.910 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.911 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:06.911 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.317728+0000", 2020-09-08T16:11:06.911 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999375 2020-09-08T16:11:06.911 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.911 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.912 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:06.912 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.317666+0000", 2020-09-08T16:11:06.912 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000144969 2020-09-08T16:11:06.913 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.913 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.913 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:06.913 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.317811+0000", 2020-09-08T16:11:06.913 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.5658999999999999e-05 2020-09-08T16:11:06.914 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.914 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.914 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:06.914 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.317836+0000", 2020-09-08T16:11:06.914 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.183e-05 2020-09-08T16:11:06.914 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.915 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.915 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:06.915 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.317848+0000", 2020-09-08T16:11:06.915 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5949000000000002e-05 2020-09-08T16:11:06.916 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.916 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.916 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:06.916 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.317914+0000", 2020-09-08T16:11:06.916 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000162817 2020-09-08T16:11:06.916 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.917 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.918 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:06.918 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.318077+0000", 2020-09-08T16:11:06.918 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012682500000000001 2020-09-08T16:11:06.919 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.919 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.919 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 4", 2020-09-08T16:11:06.919 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.318204+0000", 2020-09-08T16:11:06.919 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0047296049999999996 2020-09-08T16:11:06.920 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.920 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.920 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:06.920 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.322933+0000", 2020-09-08T16:11:06.921 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.182746398 2020-09-08T16:11:06.921 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.921 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.921 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:06.921 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.505680+0000", 2020-09-08T16:11:06.922 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.3717e-05 2020-09-08T16:11:06.922 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.922 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.922 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:06.923 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.505744+0000", 2020-09-08T16:11:06.923 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.4471000000000001e-05 2020-09-08T16:11:06.923 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.924 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.924 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:06.924 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:39.505798+0000", 2020-09-08T16:11:06.924 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.188069664 2020-09-08T16:11:06.925 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:06.925 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:06.925 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:06.925 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.925 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.926 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:223837 8.9 8:93faf4e1:::benchmark_data_smithi166_52302_object92916:head [delete] snapc 0=[] ondisk+write+known_if_redirected e685)", 2020-09-08T16:11:06.926 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:08:19.886254+0000", 2020-09-08T16:11:06.926 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 166.947063289, 2020-09-08T16:11:06.926 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17682824999999999, 2020-09-08T16:11:06.926 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:06.927 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:06.927 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:06.927 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:06.927 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:06.927 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223837 2020-09-08T16:11:06.928 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.928 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:06.928 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.928 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:06.928 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:19.886254+0000", 2020-09-08T16:11:06.929 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:06.929 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.929 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.929 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:06.929 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:19.886254+0000", 2020-09-08T16:11:06.930 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999628 2020-09-08T16:11:06.930 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.930 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.930 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:06.930 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:19.886216+0000", 2020-09-08T16:11:06.931 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.4808999999999994e-05 2020-09-08T16:11:06.931 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.931 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.931 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:06.931 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:19.886301+0000", 2020-09-08T16:11:06.932 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4501000000000001e-05 2020-09-08T16:11:06.932 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.932 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.932 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:06.932 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:19.886316+0000", 2020-09-08T16:11:06.933 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.3670000000000002e-06 2020-09-08T16:11:06.933 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.933 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.933 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:06.933 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:19.886322+0000", 2020-09-08T16:11:06.934 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.017319764000000001 2020-09-08T16:11:06.934 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.934 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.934 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:06.934 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:19.903642+0000", 2020-09-08T16:11:06.935 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013487899999999999 2020-09-08T16:11:06.935 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.935 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.935 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:06.935 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:19.903777+0000", 2020-09-08T16:11:06.936 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014866900000000001 2020-09-08T16:11:06.936 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.936 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.936 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:06.936 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:19.903925+0000", 2020-09-08T16:11:06.937 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.015715672999999999 2020-09-08T16:11:06.937 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.937 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.937 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:06.937 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:19.919641+0000", 2020-09-08T16:11:06.938 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14332251800000001 2020-09-08T16:11:06.938 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.938 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.938 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:06.938 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:20.062964+0000", 2020-09-08T16:11:06.938 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6296000000000004e-05 2020-09-08T16:11:06.939 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.939 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.939 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:06.939 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:20.063030+0000", 2020-09-08T16:11:06.939 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1958e-05 2020-09-08T16:11:06.940 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.940 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.940 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:06.940 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:20.063082+0000", 2020-09-08T16:11:06.940 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17682824999999999 2020-09-08T16:11:06.941 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:06.941 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:06.941 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:06.941 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.941 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.942 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:223870 8.9 8:9e1a88e4:::benchmark_data_smithi166_52302_object92949:head [delete] snapc 0=[] ondisk+write+known_if_redirected e685)", 2020-09-08T16:11:06.942 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:08:19.926536+0000", 2020-09-08T16:11:06.942 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 166.906781166, 2020-09-08T16:11:06.942 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18190946699999999, 2020-09-08T16:11:06.942 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:06.943 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:06.943 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:06.943 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:06.943 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:06.943 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 223870 2020-09-08T16:11:06.944 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.944 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:06.944 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.944 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:06.944 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:19.926536+0000", 2020-09-08T16:11:06.944 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:06.945 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.945 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.945 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:06.945 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:19.926536+0000", 2020-09-08T16:11:06.945 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999609 2020-09-08T16:11:06.946 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.946 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.946 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:06.946 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:19.926496+0000", 2020-09-08T16:11:06.946 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.0728000000000003e-05 2020-09-08T16:11:06.947 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.947 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.947 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:06.947 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:19.926587+0000", 2020-09-08T16:11:06.947 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6795000000000001e-05 2020-09-08T16:11:06.948 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.948 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.948 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:06.948 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:19.926604+0000", 2020-09-08T16:11:06.948 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.4760000000000001e-06 2020-09-08T16:11:06.949 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.949 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.949 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:06.949 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:19.926611+0000", 2020-09-08T16:11:06.949 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.139136234 2020-09-08T16:11:06.950 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.950 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.950 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:06.950 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:20.065748+0000", 2020-09-08T16:11:06.950 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.3937000000000005e-05 2020-09-08T16:11:06.950 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.951 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.951 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:06.951 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:20.065841+0000", 2020-09-08T16:11:06.951 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.4902999999999996e-05 2020-09-08T16:11:06.951 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.952 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.952 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:06.952 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:20.065936+0000", 2020-09-08T16:11:06.952 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.010701627 2020-09-08T16:11:06.952 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.953 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.953 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:06.953 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:20.076638+0000", 2020-09-08T16:11:06.953 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.031678662000000003 2020-09-08T16:11:06.954 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.954 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.954 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:06.954 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:20.108317+0000", 2020-09-08T16:11:06.954 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.2219999999999996e-05 2020-09-08T16:11:06.955 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.955 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.955 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:06.955 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:20.108389+0000", 2020-09-08T16:11:06.955 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.6217000000000001e-05 2020-09-08T16:11:06.956 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.956 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.956 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:06.956 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:20.108445+0000", 2020-09-08T16:11:06.956 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18190946699999999 2020-09-08T16:11:06.957 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:06.957 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:06.957 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:06.957 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.957 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.958 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:227573 8.9 8:9f1d2794:::benchmark_data_smithi166_52302_object96652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e685)", 2020-09-08T16:11:06.958 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:08:24.688614+0000", 2020-09-08T16:11:06.958 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 162.144702491, 2020-09-08T16:11:06.958 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.182237598, 2020-09-08T16:11:06.958 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:06.958 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:06.959 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:06.959 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:06.959 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:06.959 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 227573 2020-09-08T16:11:06.959 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.960 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:06.960 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.960 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:06.960 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.688614+0000", 2020-09-08T16:11:06.960 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:06.961 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.961 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.961 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:06.961 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.688614+0000", 2020-09-08T16:11:06.962 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.999958 2020-09-08T16:11:06.962 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.962 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.962 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:06.962 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.688572+0000", 2020-09-08T16:11:06.963 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2138000000000005e-05 2020-09-08T16:11:06.963 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.963 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.963 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:06.963 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.688664+0000", 2020-09-08T16:11:06.964 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4978e-05 2020-09-08T16:11:06.964 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.964 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.964 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:06.964 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.688679+0000", 2020-09-08T16:11:06.964 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7930000000000003e-06 2020-09-08T16:11:06.965 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.965 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.965 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:06.965 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.688685+0000", 2020-09-08T16:11:06.965 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14331985799999999 2020-09-08T16:11:06.966 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.966 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.966 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:06.966 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.832005+0000", 2020-09-08T16:11:06.966 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016234600000000001 2020-09-08T16:11:06.967 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.967 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.967 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:06.967 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.832167+0000", 2020-09-08T16:11:06.967 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00038496100000000002 2020-09-08T16:11:06.967 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.968 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.968 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:06.968 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.832552+0000", 2020-09-08T16:11:06.968 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.01099937 2020-09-08T16:11:06.969 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.969 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.969 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:06.969 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.843552+0000", 2020-09-08T16:11:06.969 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.027202681999999999 2020-09-08T16:11:06.970 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.970 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.970 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:06.970 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.870754+0000", 2020-09-08T16:11:06.970 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.8826999999999999e-05 2020-09-08T16:11:06.971 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.971 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.971 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:06.971 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.870793+0000", 2020-09-08T16:11:06.971 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8817999999999998e-05 2020-09-08T16:11:06.972 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.972 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.972 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:06.972 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.870852+0000", 2020-09-08T16:11:06.972 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.182237598 2020-09-08T16:11:06.973 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:06.973 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:06.973 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:06.973 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.973 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.974 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:227712 8.9 8:97f44f02:::benchmark_data_smithi166_52302_object96791:head [delete] snapc 0=[] ondisk+write+known_if_redirected e685)", 2020-09-08T16:11:06.974 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:08:24.871543+0000", 2020-09-08T16:11:06.974 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 161.96177396300001, 2020-09-08T16:11:06.974 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20181833099999999, 2020-09-08T16:11:06.974 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:06.975 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:06.975 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:06.975 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:06.975 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:06.975 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 227712 2020-09-08T16:11:06.976 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.976 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:06.976 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.976 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:06.976 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.871543+0000", 2020-09-08T16:11:06.977 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:06.977 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.977 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.977 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:06.977 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.871543+0000", 2020-09-08T16:11:06.977 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999585 2020-09-08T16:11:06.978 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.978 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.978 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:06.978 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.871502+0000", 2020-09-08T16:11:06.978 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015942699999999999 2020-09-08T16:11:06.979 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.979 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.979 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:06.979 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.871661+0000", 2020-09-08T16:11:06.979 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5812999999999999e-05 2020-09-08T16:11:06.980 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.980 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.980 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:06.980 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.871677+0000", 2020-09-08T16:11:06.980 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.9140000000000002e-06 2020-09-08T16:11:06.981 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.981 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.981 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:06.981 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.871684+0000", 2020-09-08T16:11:06.981 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018020499999999999 2020-09-08T16:11:06.981 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.982 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.982 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:06.982 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.871864+0000", 2020-09-08T16:11:06.982 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014890500000000001 2020-09-08T16:11:06.982 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.983 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.983 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:06.983 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.872013+0000", 2020-09-08T16:11:06.983 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016442400000000001 2020-09-08T16:11:06.983 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.984 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.984 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:06.984 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:24.872177+0000", 2020-09-08T16:11:06.984 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.201049213 2020-09-08T16:11:06.984 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.984 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.985 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:06.985 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:25.073226+0000", 2020-09-08T16:11:06.985 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1882999999999998e-05 2020-09-08T16:11:06.985 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.985 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.986 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:06.986 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:25.073278+0000", 2020-09-08T16:11:06.986 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.8674999999999997e-05 2020-09-08T16:11:06.986 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.986 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.987 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:06.987 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:25.073327+0000", 2020-09-08T16:11:06.987 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.4212999999999999e-05 2020-09-08T16:11:06.987 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.987 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.988 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:06.988 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:25.073361+0000", 2020-09-08T16:11:06.988 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.20181833099999999 2020-09-08T16:11:06.988 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:06.988 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:06.988 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:06.989 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.989 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.989 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:229325 8.9 8:9ab4a316:::benchmark_data_smithi166_52302_object98404:head [delete] snapc 0=[] ondisk+write+known_if_redirected e685)", 2020-09-08T16:11:06.989 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:08:26.942215+0000", 2020-09-08T16:11:06.989 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 159.89110149999999, 2020-09-08T16:11:06.990 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.202629849, 2020-09-08T16:11:06.990 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:06.990 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:06.990 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:06.990 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:06.991 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:06.991 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 229325 2020-09-08T16:11:06.991 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.991 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:06.991 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.992 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:06.992 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:26.942215+0000", 2020-09-08T16:11:06.992 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:06.992 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.992 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.992 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:06.993 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:26.942215+0000", 2020-09-08T16:11:06.993 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9998808 2020-09-08T16:11:06.993 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.993 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.993 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:06.994 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:26.942096+0000", 2020-09-08T16:11:06.994 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016326200000000001 2020-09-08T16:11:06.994 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.994 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.994 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:06.995 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:26.942259+0000", 2020-09-08T16:11:06.995 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3176000000000001e-05 2020-09-08T16:11:06.995 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.995 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.995 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:06.996 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:26.942273+0000", 2020-09-08T16:11:06.996 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7810000000000002e-06 2020-09-08T16:11:06.996 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.996 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.996 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:06.997 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:26.942278+0000", 2020-09-08T16:11:06.997 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.019356005999999999 2020-09-08T16:11:06.997 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.997 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.997 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:06.998 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:26.961634+0000", 2020-09-08T16:11:06.998 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011251000000000001 2020-09-08T16:11:06.998 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.998 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.998 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:06.999 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:26.961747+0000", 2020-09-08T16:11:06.999 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016686899999999999 2020-09-08T16:11:06.999 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:06.999 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:06.999 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:07.000 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:26.961914+0000", 2020-09-08T16:11:07.000 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.028812152000000001 2020-09-08T16:11:07.000 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.000 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.000 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:07.000 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:26.990726+0000", 2020-09-08T16:11:07.001 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.154016768 2020-09-08T16:11:07.001 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.001 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.001 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:07.001 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:27.144743+0000", 2020-09-08T16:11:07.002 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9833000000000002e-05 2020-09-08T16:11:07.002 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.002 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.002 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:07.002 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:27.144793+0000", 2020-09-08T16:11:07.003 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.2620000000000001e-05 2020-09-08T16:11:07.003 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.003 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.003 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:07.003 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:27.144845+0000", 2020-09-08T16:11:07.004 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.202629849 2020-09-08T16:11:07.004 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.004 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:07.004 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.004 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.005 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.005 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:229347 8.9 8:98d96b2e:::benchmark_data_smithi166_52302_object98426:head [delete] snapc 0=[] ondisk+write+known_if_redirected e685)", 2020-09-08T16:11:07.005 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:08:26.967778+0000", 2020-09-08T16:11:07.005 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 159.86553919299999, 2020-09-08T16:11:07.005 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18255249800000001, 2020-09-08T16:11:07.006 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:07.006 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:07.006 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:07.006 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:07.006 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:07.007 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 229347 2020-09-08T16:11:07.007 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.007 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:07.007 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.007 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:07.007 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:26.967778+0000", 2020-09-08T16:11:07.008 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:07.008 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.008 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.008 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:07.008 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:26.967778+0000", 2020-09-08T16:11:07.009 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999619 2020-09-08T16:11:07.009 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.009 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.009 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:07.009 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:26.967739+0000", 2020-09-08T16:11:07.010 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8887999999999993e-05 2020-09-08T16:11:07.010 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.010 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.010 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:07.010 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:26.967828+0000", 2020-09-08T16:11:07.011 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3353000000000001e-05 2020-09-08T16:11:07.011 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.011 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.011 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:07.011 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:26.967842+0000", 2020-09-08T16:11:07.012 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3266e-05 2020-09-08T16:11:07.012 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.012 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.012 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:07.012 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:26.967855+0000", 2020-09-08T16:11:07.013 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.6983000000000006e-05 2020-09-08T16:11:07.013 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.013 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.013 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:07.013 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:26.967922+0000", 2020-09-08T16:11:07.013 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015333299999999999 2020-09-08T16:11:07.014 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.014 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.014 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:07.014 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:26.968075+0000", 2020-09-08T16:11:07.014 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017716699999999999 2020-09-08T16:11:07.015 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.015 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.015 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:07.015 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:26.968252+0000", 2020-09-08T16:11:07.015 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.033649708 2020-09-08T16:11:07.016 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.016 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.016 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:07.016 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:27.001902+0000", 2020-09-08T16:11:07.016 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.148286627 2020-09-08T16:11:07.016 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.017 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.017 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:07.017 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:27.150189+0000", 2020-09-08T16:11:07.017 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7364e-05 2020-09-08T16:11:07.017 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.018 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.018 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:07.018 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:27.150266+0000", 2020-09-08T16:11:07.018 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.4092999999999998e-05 2020-09-08T16:11:07.018 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.019 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.019 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:07.019 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:27.150330+0000", 2020-09-08T16:11:07.019 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18255249800000001 2020-09-08T16:11:07.019 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.020 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:07.020 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.020 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.020 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.020 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:229528 8.9 8:9159742c:::benchmark_data_smithi166_52302_object98607:head [delete] snapc 0=[] ondisk+write+known_if_redirected e685)", 2020-09-08T16:11:07.020 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:08:27.218503+0000", 2020-09-08T16:11:07.021 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 159.614814209, 2020-09-08T16:11:07.021 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18158345400000001, 2020-09-08T16:11:07.021 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:07.021 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:07.021 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:07.022 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:07.022 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:07.022 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 229528 2020-09-08T16:11:07.022 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.022 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:07.023 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.023 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:07.023 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:27.218503+0000", 2020-09-08T16:11:07.023 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:07.023 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.024 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.024 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:07.024 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:27.218503+0000", 2020-09-08T16:11:07.024 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999633 2020-09-08T16:11:07.024 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.024 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.025 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:07.025 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:27.218466+0000", 2020-09-08T16:11:07.025 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.1228000000000002e-05 2020-09-08T16:11:07.025 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.025 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.026 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:07.026 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:27.218547+0000", 2020-09-08T16:11:07.026 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3988000000000001e-05 2020-09-08T16:11:07.026 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.026 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.027 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:07.027 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:27.218561+0000", 2020-09-08T16:11:07.027 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.5760000000000002e-06 2020-09-08T16:11:07.027 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.027 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.028 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:07.028 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:27.218568+0000", 2020-09-08T16:11:07.028 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.13340670800000001 2020-09-08T16:11:07.028 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.028 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.028 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:07.029 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:27.351975+0000", 2020-09-08T16:11:07.029 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017330899999999999 2020-09-08T16:11:07.029 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.029 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.029 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:07.030 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:27.352148+0000", 2020-09-08T16:11:07.030 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00020099399999999999 2020-09-08T16:11:07.030 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.030 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.030 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:07.031 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:27.352349+0000", 2020-09-08T16:11:07.031 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.027559391999999999 2020-09-08T16:11:07.031 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.031 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.031 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:07.032 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:27.379908+0000", 2020-09-08T16:11:07.032 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.020061829999999999 2020-09-08T16:11:07.032 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.032 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.032 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:07.032 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:27.399970+0000", 2020-09-08T16:11:07.033 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.8671999999999996e-05 2020-09-08T16:11:07.033 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.033 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.033 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:07.033 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:27.400039+0000", 2020-09-08T16:11:07.034 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.7265999999999998e-05 2020-09-08T16:11:07.034 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.034 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.034 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:07.034 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:27.400086+0000", 2020-09-08T16:11:07.035 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18158345400000001 2020-09-08T16:11:07.035 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.035 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:07.035 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.035 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.036 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.036 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:230434 8.9 8:900692c4:::benchmark_data_smithi166_52302_object99513:head [delete] snapc 0=[] ondisk+write+known_if_redirected e685)", 2020-09-08T16:11:07.036 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:08:28.357354+0000", 2020-09-08T16:11:07.036 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 158.475963044, 2020-09-08T16:11:07.036 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17850382200000001, 2020-09-08T16:11:07.036 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:07.037 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:07.037 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:07.037 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:07.037 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:07.037 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 230434 2020-09-08T16:11:07.038 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.038 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:07.038 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.038 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:07.038 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.357354+0000", 2020-09-08T16:11:07.039 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:07.039 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.039 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.039 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:07.039 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.357354+0000", 2020-09-08T16:11:07.040 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999499 2020-09-08T16:11:07.040 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.040 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.040 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:07.040 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.357304+0000", 2020-09-08T16:11:07.040 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7094000000000002e-05 2020-09-08T16:11:07.041 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.041 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.041 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:07.041 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.357401+0000", 2020-09-08T16:11:07.041 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.0990000000000005e-06 2020-09-08T16:11:07.042 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.042 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.042 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:07.042 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.357410+0000", 2020-09-08T16:11:07.042 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.3999999999999997e-06 2020-09-08T16:11:07.043 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.043 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.043 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:07.043 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.357416+0000", 2020-09-08T16:11:07.043 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0097657660000000004 2020-09-08T16:11:07.044 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.044 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.044 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:07.044 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.367182+0000", 2020-09-08T16:11:07.044 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015459 2020-09-08T16:11:07.044 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.045 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.045 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:07.045 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.367336+0000", 2020-09-08T16:11:07.045 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00085668400000000003 2020-09-08T16:11:07.045 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.046 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.046 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:07.046 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.368193+0000", 2020-09-08T16:11:07.046 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14261874599999999 2020-09-08T16:11:07.046 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.047 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.047 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:07.047 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.510812+0000", 2020-09-08T16:11:07.047 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.024931643 2020-09-08T16:11:07.047 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.048 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.048 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:07.048 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.535744+0000", 2020-09-08T16:11:07.048 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.2587000000000002e-05 2020-09-08T16:11:07.048 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.049 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.049 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:07.049 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.535806+0000", 2020-09-08T16:11:07.049 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1462999999999998e-05 2020-09-08T16:11:07.049 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.049 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.050 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:07.050 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.535858+0000", 2020-09-08T16:11:07.050 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17850382200000001 2020-09-08T16:11:07.050 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.050 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:07.051 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.051 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.051 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.051 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:230770 8.9 8:9ab56f1a:::benchmark_data_smithi166_52302_object99849:head [delete] snapc 0=[] ondisk+write+known_if_redirected e685)", 2020-09-08T16:11:07.051 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:08:28.843308+0000", 2020-09-08T16:11:07.052 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 157.99000892999999, 2020-09-08T16:11:07.052 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.202815942, 2020-09-08T16:11:07.052 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:07.052 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:07.052 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:07.053 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:07.053 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:07.053 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 230770 2020-09-08T16:11:07.053 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.053 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:07.054 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.054 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:07.054 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.843308+0000", 2020-09-08T16:11:07.054 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:07.054 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.055 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.055 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:07.055 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.843308+0000", 2020-09-08T16:11:07.055 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.999958 2020-09-08T16:11:07.055 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.056 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.056 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:07.056 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.843266+0000", 2020-09-08T16:11:07.056 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0001078 2020-09-08T16:11:07.056 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.057 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.057 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:07.057 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.843374+0000", 2020-09-08T16:11:07.057 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3267e-05 2020-09-08T16:11:07.057 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.058 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.058 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:07.058 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.843387+0000", 2020-09-08T16:11:07.058 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.9499999999999998e-06 2020-09-08T16:11:07.058 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.059 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.059 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:07.059 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.843393+0000", 2020-09-08T16:11:07.059 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.028634527999999999 2020-09-08T16:11:07.059 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.059 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.060 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:07.060 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.872028+0000", 2020-09-08T16:11:07.060 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000155859 2020-09-08T16:11:07.060 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.060 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.061 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:07.061 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.872183+0000", 2020-09-08T16:11:07.061 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000178548 2020-09-08T16:11:07.061 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.061 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.062 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:07.062 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.872362+0000", 2020-09-08T16:11:07.062 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.015015769 2020-09-08T16:11:07.062 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.062 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.063 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:07.063 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.887378+0000", 2020-09-08T16:11:07.063 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15860952 2020-09-08T16:11:07.063 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.063 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.064 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:07.064 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.045987+0000", 2020-09-08T16:11:07.064 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7397000000000005e-05 2020-09-08T16:11:07.064 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.064 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.065 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:07.065 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.046065+0000", 2020-09-08T16:11:07.065 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.9144999999999997e-05 2020-09-08T16:11:07.065 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.065 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.066 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:07.066 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.046124+0000", 2020-09-08T16:11:07.066 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.202815942 2020-09-08T16:11:07.066 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.066 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:07.066 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.067 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.067 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.067 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:230792 8.9 8:92a7b7d7:::benchmark_data_smithi166_52302_object99871:head [delete] snapc 0=[] ondisk+write+known_if_redirected e685)", 2020-09-08T16:11:07.067 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:08:28.866533+0000", 2020-09-08T16:11:07.067 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 157.96678345300001, 2020-09-08T16:11:07.068 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17976314900000001, 2020-09-08T16:11:07.068 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:07.068 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:07.068 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:07.068 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:07.069 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:07.069 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 230792 2020-09-08T16:11:07.069 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.069 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:07.070 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.070 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:07.070 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.866533+0000", 2020-09-08T16:11:07.070 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:07.070 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.071 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.071 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:07.071 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.866533+0000", 2020-09-08T16:11:07.071 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999681 2020-09-08T16:11:07.071 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.072 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.072 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:07.072 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.866501+0000", 2020-09-08T16:11:07.072 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.2687000000000003e-05 2020-09-08T16:11:07.072 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.073 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.073 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:07.073 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.866584+0000", 2020-09-08T16:11:07.073 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4011e-05 2020-09-08T16:11:07.074 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.074 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.074 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:07.074 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.866598+0000", 2020-09-08T16:11:07.074 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.691e-06 2020-09-08T16:11:07.075 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.075 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.075 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:07.075 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.866605+0000", 2020-09-08T16:11:07.075 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.011136728 2020-09-08T16:11:07.075 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.076 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.076 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:07.076 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.877742+0000", 2020-09-08T16:11:07.076 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012075599999999999 2020-09-08T16:11:07.076 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.077 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.077 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:07.077 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.877862+0000", 2020-09-08T16:11:07.077 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011708400000000001 2020-09-08T16:11:07.077 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.078 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.078 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:07.078 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.877979+0000", 2020-09-08T16:11:07.078 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0094073070000000002 2020-09-08T16:11:07.078 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.079 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.079 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:07.079 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:28.887387+0000", 2020-09-08T16:11:07.079 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15878936399999999 2020-09-08T16:11:07.079 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.080 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.080 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:07.080 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.046176+0000", 2020-09-08T16:11:07.080 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.2869999999999999e-05 2020-09-08T16:11:07.080 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.081 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.081 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:07.081 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.046239+0000", 2020-09-08T16:11:07.081 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7559000000000002e-05 2020-09-08T16:11:07.081 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.082 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.082 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:07.082 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.046297+0000", 2020-09-08T16:11:07.082 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17976314900000001 2020-09-08T16:11:07.082 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.083 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:07.083 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.083 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.083 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.083 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:231297 8.9 8:9ae03dbe:::benchmark_data_smithi166_52302_object100376:head [delete] snapc 0=[] ondisk+write+known_if_redirected e685)", 2020-09-08T16:11:07.084 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:08:29.495117+0000", 2020-09-08T16:11:07.084 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 157.33819998800001, 2020-09-08T16:11:07.084 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.191469427, 2020-09-08T16:11:07.084 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:07.084 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:07.085 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:07.085 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:07.085 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:07.085 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 231297 2020-09-08T16:11:07.085 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.086 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:07.086 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.086 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:07.086 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.495117+0000", 2020-09-08T16:11:07.086 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:07.087 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.087 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.087 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:07.087 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.495117+0000", 2020-09-08T16:11:07.087 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999652 2020-09-08T16:11:07.088 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.088 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.088 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:07.088 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.495082+0000", 2020-09-08T16:11:07.088 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.4111999999999997e-05 2020-09-08T16:11:07.088 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.089 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.089 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:07.089 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.495166+0000", 2020-09-08T16:11:07.089 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2931999999999999e-05 2020-09-08T16:11:07.089 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.090 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.090 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:07.090 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.495179+0000", 2020-09-08T16:11:07.090 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.541e-06 2020-09-08T16:11:07.090 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.091 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.091 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:07.091 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.495184+0000", 2020-09-08T16:11:07.091 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14503020999999999 2020-09-08T16:11:07.091 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.092 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.092 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:07.092 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.640215+0000", 2020-09-08T16:11:07.092 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014780800000000001 2020-09-08T16:11:07.092 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.092 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.093 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:07.093 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.640363+0000", 2020-09-08T16:11:07.093 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018291800000000001 2020-09-08T16:11:07.093 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.093 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.094 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:07.094 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.640545+0000", 2020-09-08T16:11:07.094 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.027828563000000001 2020-09-08T16:11:07.094 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.094 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.095 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:07.095 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.668374+0000", 2020-09-08T16:11:07.095 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.018067519000000001 2020-09-08T16:11:07.095 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.095 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.095 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:07.096 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.686442+0000", 2020-09-08T16:11:07.096 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.7712000000000005e-05 2020-09-08T16:11:07.096 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.096 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.096 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:07.097 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.686509+0000", 2020-09-08T16:11:07.097 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7033999999999995e-05 2020-09-08T16:11:07.097 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.097 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.097 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:07.098 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.686586+0000", 2020-09-08T16:11:07.098 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.191469427 2020-09-08T16:11:07.098 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.098 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:07.098 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.098 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.099 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.099 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:231304 8.9 8:9127df85:::benchmark_data_smithi166_52302_object100383:head [delete] snapc 0=[] ondisk+write+known_if_redirected e685)", 2020-09-08T16:11:07.099 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:08:29.504101+0000", 2020-09-08T16:11:07.099 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 157.32921604800001, 2020-09-08T16:11:07.099 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.182684332, 2020-09-08T16:11:07.100 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:07.100 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:07.100 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:07.100 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:07.100 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:07.101 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 231304 2020-09-08T16:11:07.101 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.101 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:07.101 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.101 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:07.102 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.504101+0000", 2020-09-08T16:11:07.102 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:07.102 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.102 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.102 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:07.102 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.504101+0000", 2020-09-08T16:11:07.103 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999599 2020-09-08T16:11:07.103 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.103 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.103 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:07.103 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.504061+0000", 2020-09-08T16:11:07.104 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.5557999999999996e-05 2020-09-08T16:11:07.104 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.104 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.104 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:07.104 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.504146+0000", 2020-09-08T16:11:07.104 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6435000000000001e-05 2020-09-08T16:11:07.105 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.105 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.105 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:07.105 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.504163+0000", 2020-09-08T16:11:07.105 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.9000000000000001e-06 2020-09-08T16:11:07.106 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.106 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.106 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:07.106 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.504173+0000", 2020-09-08T16:11:07.106 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.140619666 2020-09-08T16:11:07.107 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.107 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.107 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:07.107 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.644792+0000", 2020-09-08T16:11:07.107 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015704799999999999 2020-09-08T16:11:07.108 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.108 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.108 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:07.108 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.644949+0000", 2020-09-08T16:11:07.108 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018366500000000001 2020-09-08T16:11:07.109 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.109 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.109 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:07.109 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.645133+0000", 2020-09-08T16:11:07.109 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.029244789 2020-09-08T16:11:07.110 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.110 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.110 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:07.110 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.674378+0000", 2020-09-08T16:11:07.110 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.012267933 2020-09-08T16:11:07.111 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.111 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.111 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:07.111 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.686646+0000", 2020-09-08T16:11:07.111 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.7899999999999995e-05 2020-09-08T16:11:07.112 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.112 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.112 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:07.112 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.686734+0000", 2020-09-08T16:11:07.112 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.1554999999999997e-05 2020-09-08T16:11:07.112 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.113 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.113 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:07.113 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:29.686785+0000", 2020-09-08T16:11:07.113 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.182684332 2020-09-08T16:11:07.113 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.114 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:07.114 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.114 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.114 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.114 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:73684 9.7 9:e6bc3869:::benchmark_data_smithi166_56754_object73683:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e715)", 2020-09-08T16:11:07.115 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:10:04.658028+0000", 2020-09-08T16:11:07.115 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 62.175289231999997, 2020-09-08T16:11:07.115 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.180640468, 2020-09-08T16:11:07.115 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:07.115 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:07.116 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:07.116 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:11:07.116 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:11:07.116 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 73684 2020-09-08T16:11:07.116 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.117 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:07.117 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.117 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:07.117 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658028+0000", 2020-09-08T16:11:07.117 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:07.118 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.118 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.118 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:07.118 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658028+0000", 2020-09-08T16:11:07.118 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999638 2020-09-08T16:11:07.119 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.119 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.119 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:07.119 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.657992+0000", 2020-09-08T16:11:07.119 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014313800000000001 2020-09-08T16:11:07.119 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.120 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.120 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:07.120 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658135+0000", 2020-09-08T16:11:07.120 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2923e-05 2020-09-08T16:11:07.121 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.121 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.121 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:07.121 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658148+0000", 2020-09-08T16:11:07.121 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8239999999999996e-06 2020-09-08T16:11:07.122 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.122 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.122 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:07.122 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658153+0000", 2020-09-08T16:11:07.122 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.7447000000000002e-05 2020-09-08T16:11:07.123 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.123 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.123 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:07.123 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658191+0000", 2020-09-08T16:11:07.123 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017300599999999999 2020-09-08T16:11:07.124 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.124 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.124 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:07.124 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658364+0000", 2020-09-08T16:11:07.124 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019668600000000001 2020-09-08T16:11:07.125 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.125 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.125 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 7", 2020-09-08T16:11:07.125 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658561+0000", 2020-09-08T16:11:07.125 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16976551100000001 2020-09-08T16:11:07.126 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.126 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.126 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:07.126 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.828326+0000", 2020-09-08T16:11:07.126 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.010226337 2020-09-08T16:11:07.126 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.127 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.127 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:07.127 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.838552+0000", 2020-09-08T16:11:07.127 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.7131000000000005e-05 2020-09-08T16:11:07.127 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.128 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.128 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:07.128 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.838620+0000", 2020-09-08T16:11:07.128 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.8517000000000003e-05 2020-09-08T16:11:07.128 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.129 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.129 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:07.129 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.838668+0000", 2020-09-08T16:11:07.129 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.180640468 2020-09-08T16:11:07.129 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.130 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:07.130 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.130 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.130 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.130 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:73691 9.9 9:9a458dc9:::benchmark_data_smithi166_56754_object73690:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e715)", 2020-09-08T16:11:07.130 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:10:04.658238+0000", 2020-09-08T16:11:07.131 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 62.175078906000003, 2020-09-08T16:11:07.131 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17605706600000001, 2020-09-08T16:11:07.131 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:07.131 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:07.131 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:07.132 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:11:07.132 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:11:07.132 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 73691 2020-09-08T16:11:07.132 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.132 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:07.133 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.133 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:07.133 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658238+0000", 2020-09-08T16:11:07.133 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:07.133 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.134 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.134 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:07.134 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658238+0000", 2020-09-08T16:11:07.134 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999475 2020-09-08T16:11:07.134 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.134 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.135 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:07.135 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658185+0000", 2020-09-08T16:11:07.135 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000132879 2020-09-08T16:11:07.135 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.135 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.136 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:07.136 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658318+0000", 2020-09-08T16:11:07.136 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.203e-05 2020-09-08T16:11:07.136 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.136 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.137 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:07.137 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658330+0000", 2020-09-08T16:11:07.137 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0150000000000003e-06 2020-09-08T16:11:07.137 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.138 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.138 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:07.138 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658335+0000", 2020-09-08T16:11:07.138 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.7066e-05 2020-09-08T16:11:07.138 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.139 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.139 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:07.139 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658382+0000", 2020-09-08T16:11:07.139 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000118161 2020-09-08T16:11:07.139 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.140 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.140 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:07.140 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658501+0000", 2020-09-08T16:11:07.140 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013791699999999999 2020-09-08T16:11:07.140 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.141 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.141 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 11", 2020-09-08T16:11:07.141 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658639+0000", 2020-09-08T16:11:07.141 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.162409528 2020-09-08T16:11:07.141 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.142 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.142 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:07.142 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.821048+0000", 2020-09-08T16:11:07.142 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.012937026000000001 2020-09-08T16:11:07.142 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.143 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.143 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:07.143 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.833985+0000", 2020-09-08T16:11:07.143 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018198300000000001 2020-09-08T16:11:07.143 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.144 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.144 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:07.144 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.834167+0000", 2020-09-08T16:11:07.144 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012794099999999999 2020-09-08T16:11:07.144 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.145 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.145 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:07.145 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.834295+0000", 2020-09-08T16:11:07.145 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17605706600000001 2020-09-08T16:11:07.146 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.146 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:07.146 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.146 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.146 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.147 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:73694 9.7 9:ed93d263:::benchmark_data_smithi166_56754_object73693:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e715)", 2020-09-08T16:11:07.147 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:10:04.658843+0000", 2020-09-08T16:11:07.147 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 62.174473395, 2020-09-08T16:11:07.147 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.179969347, 2020-09-08T16:11:07.147 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:07.148 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:07.148 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:07.148 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:11:07.148 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:11:07.148 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 73694 2020-09-08T16:11:07.149 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.149 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:07.149 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.149 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:07.149 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658843+0000", 2020-09-08T16:11:07.150 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:07.150 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.150 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.150 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:07.150 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658843+0000", 2020-09-08T16:11:07.151 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999447 2020-09-08T16:11:07.151 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.151 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.151 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:07.151 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658788+0000", 2020-09-08T16:11:07.152 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013531799999999999 2020-09-08T16:11:07.152 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.152 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.152 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:07.153 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658924+0000", 2020-09-08T16:11:07.153 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4205e-05 2020-09-08T16:11:07.153 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.153 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.153 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:07.154 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658938+0000", 2020-09-08T16:11:07.154 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.2999999999999998e-06 2020-09-08T16:11:07.154 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.154 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.155 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:07.155 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.658944+0000", 2020-09-08T16:11:07.155 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16225551399999999 2020-09-08T16:11:07.155 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.156 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.156 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:07.156 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.821200+0000", 2020-09-08T16:11:07.156 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00045237100000000002 2020-09-08T16:11:07.156 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.157 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.157 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:07.157 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.821652+0000", 2020-09-08T16:11:07.157 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00039516900000000001 2020-09-08T16:11:07.158 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.158 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.158 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 7", 2020-09-08T16:11:07.158 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.822047+0000", 2020-09-08T16:11:07.158 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.016667606000000001 2020-09-08T16:11:07.159 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.159 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.159 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:07.159 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.838715+0000", 2020-09-08T16:11:07.160 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.2790000000000001e-06 2020-09-08T16:11:07.160 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.160 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.160 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:07.160 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.838722+0000", 2020-09-08T16:11:07.161 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9648000000000003e-05 2020-09-08T16:11:07.161 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.161 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.161 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:07.162 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.838772+0000", 2020-09-08T16:11:07.162 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.1022000000000001e-05 2020-09-08T16:11:07.162 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:07.162 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:07.162 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:07.163 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:10:04.838813+0000", 2020-09-08T16:11:07.163 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.179969347 2020-09-08T16:11:07.163 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.163 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:07.164 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.164 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:07.164 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:07.164 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:11:07.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:07 smithi166 bash[9446]: cluster 2020-09-08T16:11:05.689686+0000 osd.3 (osd.3) 130 : cluster [DBG] 9.1 scrub ok 2020-09-08T16:11:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:07 smithi166 bash[9446]: cluster 2020-09-08T16:11:06.042408+0000 osd.10 (osd.10) 108 : cluster [DBG] 9.9 scrub ok 2020-09-08T16:11:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:07 smithi166 bash[9446]: cluster 2020-09-08T16:11:06.099644+0000 osd.10 (osd.10) 109 : cluster [DBG] 9.7 scrub starts 2020-09-08T16:11:07.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:07 smithi166 bash[9446]: cluster 2020-09-08T16:11:06.136347+0000 mgr.x (mgr.24118) 1313 : cluster [DBG] pgmap v1963: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 8.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 38 MiB/s wr, 658 op/s 2020-09-08T16:11:07.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:07 smithi055 bash[9296]: cluster 2020-09-08T16:11:05.689686+0000 osd.3 (osd.3) 130 : cluster [DBG] 9.1 scrub ok 2020-09-08T16:11:07.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:07 smithi055 bash[9296]: cluster 2020-09-08T16:11:06.042408+0000 osd.10 (osd.10) 108 : cluster [DBG] 9.9 scrub ok 2020-09-08T16:11:07.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:07 smithi055 bash[9296]: cluster 2020-09-08T16:11:06.099644+0000 osd.10 (osd.10) 109 : cluster [DBG] 9.7 scrub starts 2020-09-08T16:11:07.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:07 smithi055 bash[9296]: cluster 2020-09-08T16:11:06.136347+0000 mgr.x (mgr.24118) 1313 : cluster [DBG] pgmap v1963: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 8.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 38 MiB/s wr, 658 op/s 2020-09-08T16:11:07.459 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:07 smithi104 bash[9203]: cluster 2020-09-08T16:11:05.689686+0000 osd.3 (osd.3) 130 : cluster [DBG] 9.1 scrub ok 2020-09-08T16:11:07.460 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:07 smithi104 bash[9203]: cluster 2020-09-08T16:11:06.042408+0000 osd.10 (osd.10) 108 : cluster [DBG] 9.9 scrub ok 2020-09-08T16:11:07.460 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:07 smithi104 bash[9203]: cluster 2020-09-08T16:11:06.099644+0000 osd.10 (osd.10) 109 : cluster [DBG] 9.7 scrub starts 2020-09-08T16:11:07.460 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:07 smithi104 bash[9203]: cluster 2020-09-08T16:11:06.136347+0000 mgr.x (mgr.24118) 1313 : cluster [DBG] pgmap v1963: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 8.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 38 MiB/s wr, 658 op/s 2020-09-08T16:11:08.072 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2020-09-08T16:11:08.240 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:11:09.315 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:11:09.419 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:11:09.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:09 smithi166 bash[9446]: cluster 2020-09-08T16:11:08.136866+0000 mgr.x (mgr.24118) 1314 : cluster [DBG] pgmap v1964: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s rd, 31 MiB/s wr, 917 op/s 2020-09-08T16:11:09.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:09 smithi166 bash[9446]: cluster 2020-09-08T16:11:08.490274+0000 osd.5 (osd.5) 127 : cluster [DBG] 9.a scrub ok 2020-09-08T16:11:09.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:09 smithi166 bash[9446]: cluster 2020-09-08T16:11:08.973230+0000 osd.10 (osd.10) 110 : cluster [DBG] 9.7 scrub ok 2020-09-08T16:11:09.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:11:09 smithi055 bash[19003]: debug 2020-09-08T16:11:09.313+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:09.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:09 smithi055 bash[9296]: cluster 2020-09-08T16:11:08.136866+0000 mgr.x (mgr.24118) 1314 : cluster [DBG] pgmap v1964: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s rd, 31 MiB/s wr, 917 op/s 2020-09-08T16:11:09.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:09 smithi055 bash[9296]: cluster 2020-09-08T16:11:08.490274+0000 osd.5 (osd.5) 127 : cluster [DBG] 9.a scrub ok 2020-09-08T16:11:09.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:09 smithi055 bash[9296]: cluster 2020-09-08T16:11:08.973230+0000 osd.10 (osd.10) 110 : cluster [DBG] 9.7 scrub ok 2020-09-08T16:11:09.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:09 smithi104 bash[9203]: cluster 2020-09-08T16:11:08.136866+0000 mgr.x (mgr.24118) 1314 : cluster [DBG] pgmap v1964: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s rd, 31 MiB/s wr, 917 op/s 2020-09-08T16:11:09.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:09 smithi104 bash[9203]: cluster 2020-09-08T16:11:08.490274+0000 osd.5 (osd.5) 127 : cluster [DBG] 9.a scrub ok 2020-09-08T16:11:09.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:09 smithi104 bash[9203]: cluster 2020-09-08T16:11:08.973230+0000 osd.10 (osd.10) 110 : cluster [DBG] 9.7 scrub ok 2020-09-08T16:11:09.821 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:11:09.925 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:11:10.176 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:11:09 smithi104 bash[15273]: debug 2020-09-08T16:11:09.821+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:10.194 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T16:11:10.364 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:10 smithi166 bash[9446]: cluster 2020-09-08T16:11:09.153345+0000 osd.1 (osd.1) 148 : cluster [DBG] 9.4 scrub starts 2020-09-08T16:11:10.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:10 smithi166 bash[9446]: cluster 2020-09-08T16:11:09.187188+0000 osd.7 (osd.7) 159 : cluster [DBG] 9.e scrub starts 2020-09-08T16:11:10.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:10 smithi166 bash[9446]: audit 2020-09-08T16:11:09.219627+0000 mon.c (mon.2) 93 : audit [INF] from='client.? v1:172.21.15.104:0/967747554' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:11:10.365 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:10 smithi166 bash[9446]: audit 2020-09-08T16:11:09.222356+0000 mon.a (mon.0) 2285 : audit [INF] from='client.36644 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:11:10.523 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:10 smithi104 bash[9203]: cluster 2020-09-08T16:11:09.153345+0000 osd.1 (osd.1) 148 : cluster [DBG] 9.4 scrub starts 2020-09-08T16:11:10.523 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:10 smithi104 bash[9203]: cluster 2020-09-08T16:11:09.187188+0000 osd.7 (osd.7) 159 : cluster [DBG] 9.e scrub starts 2020-09-08T16:11:10.523 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:10 smithi104 bash[9203]: audit 2020-09-08T16:11:09.219627+0000 mon.c (mon.2) 93 : audit [INF] from='client.? v1:172.21.15.104:0/967747554' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:11:10.523 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:10 smithi104 bash[9203]: audit 2020-09-08T16:11:09.222356+0000 mon.a (mon.0) 2285 : audit [INF] from='client.36644 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:11:10.526 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:11:10.630 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:10.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:10 smithi055 bash[9296]: cluster 2020-09-08T16:11:09.153345+0000 osd.1 (osd.1) 148 : cluster [DBG] 9.4 scrub starts 2020-09-08T16:11:10.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:10 smithi055 bash[9296]: cluster 2020-09-08T16:11:09.187188+0000 osd.7 (osd.7) 159 : cluster [DBG] 9.e scrub starts 2020-09-08T16:11:10.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:10 smithi055 bash[9296]: audit 2020-09-08T16:11:09.219627+0000 mon.c (mon.2) 93 : audit [INF] from='client.? v1:172.21.15.104:0/967747554' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:11:10.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:10 smithi055 bash[9296]: audit 2020-09-08T16:11:09.222356+0000 mon.a (mon.0) 2285 : audit [INF] from='client.36644 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:11:10.925 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:11:10 smithi104 bash[22305]: debug 2020-09-08T16:11:10.525+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:11.318 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:11.423 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:11.561 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:11 smithi055 bash[9296]: cluster 2020-09-08T16:11:10.137363+0000 mgr.x (mgr.24118) 1315 : cluster [DBG] pgmap v1965: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 8.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s rd, 24 MiB/s wr, 894 op/s 2020-09-08T16:11:11.561 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:11 smithi055 bash[9296]: cluster 2020-09-08T16:11:10.192107+0000 mon.a (mon.0) 2286 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:11:11.562 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:11 smithi055 bash[9296]: cluster 2020-09-08T16:11:10.192143+0000 mon.a (mon.0) 2287 : cluster [INF] Cluster is now healthy 2020-09-08T16:11:11.562 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:11 smithi055 bash[9296]: audit 2020-09-08T16:11:10.196008+0000 mon.a (mon.0) 2288 : audit [INF] from='client.36644 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:11:11.562 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:11 smithi055 bash[9296]: cluster 2020-09-08T16:11:10.196109+0000 mon.a (mon.0) 2289 : cluster [DBG] osdmap e735: 12 total, 12 up, 12 in 2020-09-08T16:11:11.562 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:11 smithi055 bash[9296]: cluster 2020-09-08T16:11:10.529289+0000 osd.9 (osd.9) 111 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:11.563 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:11 smithi055 bash[9296]: cluster 2020-09-08T16:11:10.529360+0000 osd.9 (osd.9) 112 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:11.564 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:11.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:11:11 smithi166 bash[11928]: debug 2020-09-08T16:11:11.319+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:11.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:11 smithi166 bash[9446]: cluster 2020-09-08T16:11:10.137363+0000 mgr.x (mgr.24118) 1315 : cluster [DBG] pgmap v1965: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 8.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s rd, 24 MiB/s wr, 894 op/s 2020-09-08T16:11:11.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:11 smithi166 bash[9446]: cluster 2020-09-08T16:11:10.192107+0000 mon.a (mon.0) 2286 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:11:11.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:11 smithi166 bash[9446]: cluster 2020-09-08T16:11:10.192143+0000 mon.a (mon.0) 2287 : cluster [INF] Cluster is now healthy 2020-09-08T16:11:11.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:11 smithi166 bash[9446]: audit 2020-09-08T16:11:10.196008+0000 mon.a (mon.0) 2288 : audit [INF] from='client.36644 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:11:11.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:11 smithi166 bash[9446]: cluster 2020-09-08T16:11:10.196109+0000 mon.a (mon.0) 2289 : cluster [DBG] osdmap e735: 12 total, 12 up, 12 in 2020-09-08T16:11:11.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:11 smithi166 bash[9446]: cluster 2020-09-08T16:11:10.529289+0000 osd.9 (osd.9) 111 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:11.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:11 smithi166 bash[9446]: cluster 2020-09-08T16:11:10.529360+0000 osd.9 (osd.9) 112 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:11.668 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:11:11.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:11 smithi104 bash[9203]: cluster 2020-09-08T16:11:10.137363+0000 mgr.x (mgr.24118) 1315 : cluster [DBG] pgmap v1965: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 8.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s rd, 24 MiB/s wr, 894 op/s 2020-09-08T16:11:11.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:11 smithi104 bash[9203]: cluster 2020-09-08T16:11:10.192107+0000 mon.a (mon.0) 2286 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:11:11.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:11 smithi104 bash[9203]: cluster 2020-09-08T16:11:10.192143+0000 mon.a (mon.0) 2287 : cluster [INF] Cluster is now healthy 2020-09-08T16:11:11.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:11 smithi104 bash[9203]: audit 2020-09-08T16:11:10.196008+0000 mon.a (mon.0) 2288 : audit [INF] from='client.36644 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:11:11.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:11 smithi104 bash[9203]: cluster 2020-09-08T16:11:10.196109+0000 mon.a (mon.0) 2289 : cluster [DBG] osdmap e735: 12 total, 12 up, 12 in 2020-09-08T16:11:11.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:11 smithi104 bash[9203]: cluster 2020-09-08T16:11:10.529289+0000 osd.9 (osd.9) 111 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:11.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:11 smithi104 bash[9203]: cluster 2020-09-08T16:11:10.529360+0000 osd.9 (osd.9) 112 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:11.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:11:11 smithi055 bash[14301]: debug 2020-09-08T16:11:11.565+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:12.268 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:11:12.276 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:11:12.302 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:11:12.332 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:11:12.365 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:11:12.372 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:11:12.400 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:11:12.438 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:11:12.469 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:11:12.503 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:11:12.514 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:12 smithi104 bash[9203]: cluster 2020-09-08T16:11:11.207518+0000 mon.a (mon.0) 2290 : cluster [DBG] osdmap e736: 12 total, 12 up, 12 in 2020-09-08T16:11:12.515 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:12 smithi104 bash[9203]: cluster 2020-09-08T16:11:11.428943+0000 osd.1 (osd.1) 149 : cluster [DBG] 9.4 scrub ok 2020-09-08T16:11:12.515 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:11:12 smithi104 bash[15273]: debug 2020-09-08T16:11:12.513+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:12.515 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:11:12.542 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:11:12.570 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:11:12.597 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:11:12.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:12 smithi166 bash[9446]: cluster 2020-09-08T16:11:11.207518+0000 mon.a (mon.0) 2290 : cluster [DBG] osdmap e736: 12 total, 12 up, 12 in 2020-09-08T16:11:12.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:12 smithi166 bash[9446]: cluster 2020-09-08T16:11:11.428943+0000 osd.1 (osd.1) 149 : cluster [DBG] 9.4 scrub ok 2020-09-08T16:11:12.617 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:11:12 smithi166 bash[18960]: debug 2020-09-08T16:11:12.267+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:12.619 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:12.637 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:11:12.641 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:11:12.641 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:11:12.642 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:11:12.642 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:11:12.642 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:11:12.642 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:11:12.643 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:11:12.643 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:11:12.643 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:11:12.643 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:11:12.644 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:11:12.644 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:11:12.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:12 smithi055 bash[9296]: cluster 2020-09-08T16:11:11.207518+0000 mon.a (mon.0) 2290 : cluster [DBG] osdmap e736: 12 total, 12 up, 12 in 2020-09-08T16:11:12.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:12 smithi055 bash[9296]: cluster 2020-09-08T16:11:11.428943+0000 osd.1 (osd.1) 149 : cluster [DBG] 9.4 scrub ok 2020-09-08T16:11:13.023 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:11:13 smithi166 bash[16593]: debug 2020-09-08T16:11:13.023+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:13.024 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:13.128 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:11:13.295 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:13 smithi166 bash[9446]: cluster 2020-09-08T16:11:12.137925+0000 mgr.x (mgr.24118) 1316 : cluster [DBG] pgmap v1968: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 8.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 2.0 MiB/s wr, 645 op/s 2020-09-08T16:11:13.295 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:13 smithi166 bash[9446]: audit 2020-09-08T16:11:12.256321+0000 mon.a (mon.0) 2291 : audit [DBG] from='client.? v1:172.21.15.104:0/1293058809' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:11:13.295 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:13 smithi166 bash[9446]: cluster 2020-09-08T16:11:12.378058+0000 osd.7 (osd.7) 160 : cluster [DBG] 9.e scrub ok 2020-09-08T16:11:13.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:13 smithi104 bash[9203]: cluster 2020-09-08T16:11:12.137925+0000 mgr.x (mgr.24118) 1316 : cluster [DBG] pgmap v1968: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 8.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 2.0 MiB/s wr, 645 op/s 2020-09-08T16:11:13.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:13 smithi104 bash[9203]: audit 2020-09-08T16:11:12.256321+0000 mon.a (mon.0) 2291 : audit [DBG] from='client.? v1:172.21.15.104:0/1293058809' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:11:13.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:13 smithi104 bash[9203]: cluster 2020-09-08T16:11:12.378058+0000 osd.7 (osd.7) 160 : cluster [DBG] 9.e scrub ok 2020-09-08T16:11:13.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:13 smithi055 bash[9296]: cluster 2020-09-08T16:11:12.137925+0000 mgr.x (mgr.24118) 1316 : cluster [DBG] pgmap v1968: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 8.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 2.0 MiB/s wr, 645 op/s 2020-09-08T16:11:13.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:13 smithi055 bash[9296]: audit 2020-09-08T16:11:12.256321+0000 mon.a (mon.0) 2291 : audit [DBG] from='client.? v1:172.21.15.104:0/1293058809' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:11:13.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:13 smithi055 bash[9296]: cluster 2020-09-08T16:11:12.378058+0000 osd.7 (osd.7) 160 : cluster [DBG] 9.e scrub ok 2020-09-08T16:11:13.816 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T16:11:13.862 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:11:13.908 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T16:11:13.909 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:11:13.966 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:13.980 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:11:14.175 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:11:13 smithi104 bash[15273]: debug 2020-09-08T16:11:13.860+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:14.176 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:14.280 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:14.288 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:11:14 smithi166 bash[16593]: debug 2020-09-08T16:11:14.175+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:14.289 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:14 smithi166 bash[9446]: cluster 2020-09-08T16:11:12.728951+0000 osd.8 (osd.8) 106 : cluster [DBG] 9.b scrub starts 2020-09-08T16:11:14.289 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:14 smithi166 bash[9446]: cluster 2020-09-08T16:11:13.088349+0000 osd.2 (osd.2) 114 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:14.289 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:14 smithi166 bash[9446]: cluster 2020-09-08T16:11:13.088419+0000 osd.2 (osd.2) 115 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:14.289 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:14 smithi166 bash[9446]: cluster 2020-09-08T16:11:13.218235+0000 mon.a (mon.0) 2292 : cluster [DBG] osdmap e737: 12 total, 12 up, 12 in 2020-09-08T16:11:14.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:14 smithi055 bash[9296]: cluster 2020-09-08T16:11:12.728951+0000 osd.8 (osd.8) 106 : cluster [DBG] 9.b scrub starts 2020-09-08T16:11:14.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:14 smithi055 bash[9296]: cluster 2020-09-08T16:11:13.088349+0000 osd.2 (osd.2) 114 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:14.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:14 smithi055 bash[9296]: cluster 2020-09-08T16:11:13.088419+0000 osd.2 (osd.2) 115 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:14.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:14 smithi055 bash[9296]: cluster 2020-09-08T16:11:13.218235+0000 mon.a (mon.0) 2292 : cluster [DBG] osdmap e737: 12 total, 12 up, 12 in 2020-09-08T16:11:14.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:14 smithi104 bash[9203]: cluster 2020-09-08T16:11:12.728951+0000 osd.8 (osd.8) 106 : cluster [DBG] 9.b scrub starts 2020-09-08T16:11:14.678 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:14 smithi104 bash[9203]: cluster 2020-09-08T16:11:13.088349+0000 osd.2 (osd.2) 114 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:14.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:14 smithi104 bash[9203]: cluster 2020-09-08T16:11:13.088419+0000 osd.2 (osd.2) 115 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:14.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:14 smithi104 bash[9203]: cluster 2020-09-08T16:11:13.218235+0000 mon.a (mon.0) 2292 : cluster [DBG] osdmap e737: 12 total, 12 up, 12 in 2020-09-08T16:11:15.249 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:15.353 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:15.508 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:15 smithi166 bash[9446]: cluster 2020-09-08T16:11:14.052826+0000 osd.1 (osd.1) 150 : cluster [DBG] 9.0 scrub starts 2020-09-08T16:11:15.508 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:15 smithi166 bash[9446]: cluster 2020-09-08T16:11:14.138423+0000 mgr.x (mgr.24118) 1317 : cluster [DBG] pgmap v1970: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 886 op/s 2020-09-08T16:11:15.508 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:11:15 smithi166 bash[16593]: debug 2020-09-08T16:11:15.247+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:15.509 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2020-09-08T16:11:15.521 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:11:15 smithi055 bash[14301]: debug 2020-09-08T16:11:15.525+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:15.521 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:15 smithi055 bash[9296]: cluster 2020-09-08T16:11:14.052826+0000 osd.1 (osd.1) 150 : cluster [DBG] 9.0 scrub starts 2020-09-08T16:11:15.521 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:15 smithi055 bash[9296]: cluster 2020-09-08T16:11:14.138423+0000 mgr.x (mgr.24118) 1317 : cluster [DBG] pgmap v1970: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 886 op/s 2020-09-08T16:11:15.522 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:15.626 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:11:15.672 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:11:15.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:15 smithi104 bash[9203]: cluster 2020-09-08T16:11:14.052826+0000 osd.1 (osd.1) 150 : cluster [DBG] 9.0 scrub starts 2020-09-08T16:11:15.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:15 smithi104 bash[9203]: cluster 2020-09-08T16:11:14.138423+0000 mgr.x (mgr.24118) 1317 : cluster [DBG] pgmap v1970: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 8.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 886 op/s 2020-09-08T16:11:15.801 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:11:15.904 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:11:15.994 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:11:16.016 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:11:16.119 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:11:16.128 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:11:15 smithi055 bash[11956]: debug 2020-09-08T16:11:15.805+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:16.129 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:11:16 smithi055 bash[16646]: debug 2020-09-08T16:11:16.013+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:16.158 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:11:16.248 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:11:16.353 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:16.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:16 smithi055 bash[9296]: cluster 2020-09-08T16:11:15.181314+0000 osd.8 (osd.8) 107 : cluster [DBG] 9.b scrub ok 2020-09-08T16:11:16.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:16 smithi055 bash[9296]: cluster 2020-09-08T16:11:15.897137+0000 osd.5 (osd.5) 128 : cluster [DBG] 9.6 scrub starts 2020-09-08T16:11:16.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:11:16 smithi055 bash[19003]: debug 2020-09-08T16:11:16.249+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:16.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:16 smithi104 bash[9203]: cluster 2020-09-08T16:11:15.181314+0000 osd.8 (osd.8) 107 : cluster [DBG] 9.b scrub ok 2020-09-08T16:11:16.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:16 smithi104 bash[9203]: cluster 2020-09-08T16:11:15.897137+0000 osd.5 (osd.5) 128 : cluster [DBG] 9.6 scrub starts 2020-09-08T16:11:16.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:16 smithi166 bash[9446]: cluster 2020-09-08T16:11:15.181314+0000 osd.8 (osd.8) 107 : cluster [DBG] 9.b scrub ok 2020-09-08T16:11:16.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:16 smithi166 bash[9446]: cluster 2020-09-08T16:11:15.897137+0000 osd.5 (osd.5) 128 : cluster [DBG] 9.6 scrub starts 2020-09-08T16:11:16.637 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:16.742 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:11:16.856 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:11:16.960 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:17.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:11:16 smithi166 bash[14254]: debug 2020-09-08T16:11:16.639+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:17.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:11:16 smithi055 bash[11956]: debug 2020-09-08T16:11:16.857+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:17.233 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:17 smithi104 bash[9203]: cluster 2020-09-08T16:11:16.048529+0000 osd.9 (osd.9) 113 : cluster [DBG] 9.3 scrub starts 2020-09-08T16:11:17.233 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:17 smithi104 bash[9203]: cluster 2020-09-08T16:11:16.138889+0000 mgr.x (mgr.24118) 1318 : cluster [DBG] pgmap v1971: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 844 op/s 2020-09-08T16:11:17.354 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:17.465 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:17.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:17 smithi166 bash[9446]: cluster 2020-09-08T16:11:16.048529+0000 osd.9 (osd.9) 113 : cluster [DBG] 9.3 scrub starts 2020-09-08T16:11:17.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:17 smithi166 bash[9446]: cluster 2020-09-08T16:11:16.138889+0000 mgr.x (mgr.24118) 1318 : cluster [DBG] pgmap v1971: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 844 op/s 2020-09-08T16:11:17.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:17 smithi166 bash[9446]: cluster 2020-09-08T16:11:16.225849+0000 osd.1 (osd.1) 151 : cluster [DBG] 9.0 scrub ok 2020-09-08T16:11:17.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:17 smithi055 bash[9296]: cluster 2020-09-08T16:11:16.048529+0000 osd.9 (osd.9) 113 : cluster [DBG] 9.3 scrub starts 2020-09-08T16:11:17.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:17 smithi055 bash[9296]: cluster 2020-09-08T16:11:16.138889+0000 mgr.x (mgr.24118) 1318 : cluster [DBG] pgmap v1971: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 7.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 844 op/s 2020-09-08T16:11:17.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:17 smithi055 bash[9296]: cluster 2020-09-08T16:11:16.225849+0000 osd.1 (osd.1) 151 : cluster [DBG] 9.0 scrub ok 2020-09-08T16:11:17.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:17 smithi104 bash[9203]: cluster 2020-09-08T16:11:16.225849+0000 osd.1 (osd.1) 151 : cluster [DBG] 9.0 scrub ok 2020-09-08T16:11:17.676 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:11:17 smithi104 bash[19922]: debug 2020-09-08T16:11:17.356+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:18.166 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:18.269 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:18.386 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:18.425 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:11:18 smithi104 bash[19922]: debug 2020-09-08T16:11:18.164+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:18.492 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:18.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:11:18 smithi055 bash[14301]: debug 2020-09-08T16:11:18.389+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:19.190 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:19.294 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:11:19.421 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:11:19.525 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:19.533 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:19 smithi166 bash[9446]: cluster 2020-09-08T16:11:18.139427+0000 mgr.x (mgr.24118) 1319 : cluster [DBG] pgmap v1972: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T16:11:19.533 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:19 smithi166 bash[9446]: cluster 2020-09-08T16:11:18.866199+0000 osd.5 (osd.5) 129 : cluster [DBG] 9.6 scrub ok 2020-09-08T16:11:19.636 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:19 smithi104 bash[9203]: cluster 2020-09-08T16:11:18.139427+0000 mgr.x (mgr.24118) 1319 : cluster [DBG] pgmap v1972: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T16:11:19.637 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:19 smithi104 bash[9203]: cluster 2020-09-08T16:11:18.866199+0000 osd.5 (osd.5) 129 : cluster [DBG] 9.6 scrub ok 2020-09-08T16:11:19.637 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:11:19 smithi104 bash[19922]: debug 2020-09-08T16:11:19.188+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:19.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:11:19 smithi055 bash[16646]: debug 2020-09-08T16:11:19.417+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:19.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:19 smithi055 bash[9296]: cluster 2020-09-08T16:11:18.139427+0000 mgr.x (mgr.24118) 1319 : cluster [DBG] pgmap v1972: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T16:11:19.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:19 smithi055 bash[9296]: cluster 2020-09-08T16:11:18.866199+0000 osd.5 (osd.5) 129 : cluster [DBG] 9.6 scrub ok 2020-09-08T16:11:20.005 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:20.107 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:11:20.218 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:11:20.325 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:20.335 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:20 smithi166 bash[9446]: cluster 2020-09-08T16:11:19.159000+0000 osd.9 (osd.9) 114 : cluster [DBG] 9.3 scrub ok 2020-09-08T16:11:20.336 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:11:20 smithi166 bash[16593]: debug 2020-09-08T16:11:20.003+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:20.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:11:20 smithi055 bash[11956]: debug 2020-09-08T16:11:20.217+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:20.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:20 smithi055 bash[9296]: cluster 2020-09-08T16:11:19.159000+0000 osd.9 (osd.9) 114 : cluster [DBG] 9.3 scrub ok 2020-09-08T16:11:20.690 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:20 smithi104 bash[9203]: cluster 2020-09-08T16:11:19.159000+0000 osd.9 (osd.9) 114 : cluster [DBG] 9.3 scrub ok 2020-09-08T16:11:20.713 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:20.821 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:20.992 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:11:20 smithi166 bash[11928]: debug 2020-09-08T16:11:20.711+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:21.011 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:21.115 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:11:21.124 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:11:21 smithi104 bash[19922]: debug 2020-09-08T16:11:21.012+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:21.252 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T16:11:21.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:21 smithi104 bash[9203]: cluster 2020-09-08T16:11:20.033715+0000 osd.9 (osd.9) 115 : cluster [DBG] 9.8 scrub starts 2020-09-08T16:11:21.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:21 smithi104 bash[9203]: cluster 2020-09-08T16:11:20.139982+0000 mgr.x (mgr.24118) 1320 : cluster [DBG] pgmap v1973: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T16:11:21.435 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:21 smithi104 bash[9203]: cluster 2020-09-08T16:11:20.270342+0000 osd.0 (osd.0) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:21.435 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:21 smithi104 bash[9203]: cluster 2020-09-08T16:11:20.270424+0000 osd.0 (osd.0) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:21.435 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:21 smithi104 bash[9203]: audit 2020-09-08T16:11:20.552448+0000 mon.a (mon.0) 2293 : audit [INF] from='client.? v1:172.21.15.104:0/536969532' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:11:21.477 INFO:teuthology.orchestra.run.smithi166.stdout:ERROR: (22) Invalid argument 2020-09-08T16:11:21.478 INFO:teuthology.orchestra.run.smithi166.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:11:21.572 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:21 smithi166 bash[9446]: cluster 2020-09-08T16:11:20.033715+0000 osd.9 (osd.9) 115 : cluster [DBG] 9.8 scrub starts 2020-09-08T16:11:21.572 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:21 smithi166 bash[9446]: cluster 2020-09-08T16:11:20.139982+0000 mgr.x (mgr.24118) 1320 : cluster [DBG] pgmap v1973: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T16:11:21.572 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:21 smithi166 bash[9446]: cluster 2020-09-08T16:11:20.270342+0000 osd.0 (osd.0) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:21.573 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:21 smithi166 bash[9446]: cluster 2020-09-08T16:11:20.270424+0000 osd.0 (osd.0) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:21.573 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:21 smithi166 bash[9446]: audit 2020-09-08T16:11:20.552448+0000 mon.a (mon.0) 2293 : audit [INF] from='client.? v1:172.21.15.104:0/536969532' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:11:21.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:21 smithi055 bash[9296]: cluster 2020-09-08T16:11:20.033715+0000 osd.9 (osd.9) 115 : cluster [DBG] 9.8 scrub starts 2020-09-08T16:11:21.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:21 smithi055 bash[9296]: cluster 2020-09-08T16:11:20.139982+0000 mgr.x (mgr.24118) 1320 : cluster [DBG] pgmap v1973: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T16:11:21.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:21 smithi055 bash[9296]: cluster 2020-09-08T16:11:20.270342+0000 osd.0 (osd.0) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:21.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:21 smithi055 bash[9296]: cluster 2020-09-08T16:11:20.270424+0000 osd.0 (osd.0) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:21.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:21 smithi055 bash[9296]: audit 2020-09-08T16:11:20.552448+0000 mon.a (mon.0) 2293 : audit [INF] from='client.? v1:172.21.15.104:0/536969532' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:11:21.742 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:11:21.845 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:11:21.964 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:11:22.073 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:22.085 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:11:21 smithi104 bash[17588]: debug 2020-09-08T16:11:21.740+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:22.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:11:21 smithi055 bash[11956]: debug 2020-09-08T16:11:21.965+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:22.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:22 smithi104 bash[9203]: cluster 2020-09-08T16:11:20.628030+0000 osd.7 (osd.7) 161 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:22.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:22 smithi104 bash[9203]: cluster 2020-09-08T16:11:20.628107+0000 osd.7 (osd.7) 162 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:22.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:22 smithi104 bash[9203]: cluster 2020-09-08T16:11:21.245064+0000 mon.a (mon.0) 2294 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:11:22.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:22 smithi104 bash[9203]: audit 2020-09-08T16:11:21.249539+0000 mon.a (mon.0) 2295 : audit [INF] from='client.? v1:172.21.15.104:0/536969532' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:11:22.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:22 smithi104 bash[9203]: cluster 2020-09-08T16:11:21.249611+0000 mon.a (mon.0) 2296 : cluster [DBG] osdmap e738: 12 total, 12 up, 12 in 2020-09-08T16:11:22.607 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:22.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:22 smithi166 bash[9446]: cluster 2020-09-08T16:11:20.628030+0000 osd.7 (osd.7) 161 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:22.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:22 smithi166 bash[9446]: cluster 2020-09-08T16:11:20.628107+0000 osd.7 (osd.7) 162 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:22.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:22 smithi166 bash[9446]: cluster 2020-09-08T16:11:21.245064+0000 mon.a (mon.0) 2294 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:11:22.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:22 smithi166 bash[9446]: audit 2020-09-08T16:11:21.249539+0000 mon.a (mon.0) 2295 : audit [INF] from='client.? v1:172.21.15.104:0/536969532' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:11:22.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:22 smithi166 bash[9446]: cluster 2020-09-08T16:11:21.249611+0000 mon.a (mon.0) 2296 : cluster [DBG] osdmap e738: 12 total, 12 up, 12 in 2020-09-08T16:11:22.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:22 smithi055 bash[9296]: cluster 2020-09-08T16:11:20.628030+0000 osd.7 (osd.7) 161 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:22.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:22 smithi055 bash[9296]: cluster 2020-09-08T16:11:20.628107+0000 osd.7 (osd.7) 162 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:22.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:22 smithi055 bash[9296]: cluster 2020-09-08T16:11:21.245064+0000 mon.a (mon.0) 2294 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:11:22.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:22 smithi055 bash[9296]: audit 2020-09-08T16:11:21.249539+0000 mon.a (mon.0) 2295 : audit [INF] from='client.? v1:172.21.15.104:0/536969532' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:11:22.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:22 smithi055 bash[9296]: cluster 2020-09-08T16:11:21.249611+0000 mon.a (mon.0) 2296 : cluster [DBG] osdmap e738: 12 total, 12 up, 12 in 2020-09-08T16:11:22.710 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:11:22.791 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2020-09-08T16:11:22.818 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:11:22.922 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:22.925 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:11:22 smithi104 bash[19922]: debug 2020-09-08T16:11:22.608+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:22.954 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:11:23.036 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:23.037 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:11:22 smithi055 bash[11956]: debug 2020-09-08T16:11:22.821+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:23.037 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:11:23 smithi055 bash[14301]: debug 2020-09-08T16:11:23.037+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:23.140 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:23.308 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:11:23.338 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:11:23.373 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:11:23.406 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:11:23.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:23 smithi055 bash[9296]: cluster 2020-09-08T16:11:22.140516+0000 mgr.x (mgr.24118) 1321 : cluster [DBG] pgmap v1975: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 635 op/s 2020-09-08T16:11:23.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:23 smithi104 bash[9203]: cluster 2020-09-08T16:11:22.140516+0000 mgr.x (mgr.24118) 1321 : cluster [DBG] pgmap v1975: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 635 op/s 2020-09-08T16:11:23.450 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:11:23.482 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:11:23.514 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:11:23.545 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:11:23.576 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:11:23.609 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:11:23.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:23 smithi166 bash[9446]: cluster 2020-09-08T16:11:22.140516+0000 mgr.x (mgr.24118) 1321 : cluster [DBG] pgmap v1975: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 7.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 635 op/s 2020-09-08T16:11:23.650 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:11:23.674 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:11:23.675 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:11:23.675 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:11:23.675 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:11:23.675 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:11:23.676 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:11:23.676 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:11:23.676 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:11:23.676 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:11:23.676 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:11:23.677 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:11:23.677 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:11:23.677 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:11:23.773 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:23.877 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:24.117 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:11:23 smithi166 bash[14254]: debug 2020-09-08T16:11:23.775+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:24.508 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:24.612 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:11:24.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:11:24 smithi166 bash[11928]: debug 2020-09-08T16:11:24.507+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:24.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:24 smithi166 bash[9446]: audit 2020-09-08T16:11:23.285888+0000 mon.c (mon.2) 94 : audit [DBG] from='client.? v1:172.21.15.104:0/1138294256' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:11:24.620 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:24 smithi055 bash[9296]: audit 2020-09-08T16:11:23.285888+0000 mon.c (mon.2) 94 : audit [DBG] from='client.? v1:172.21.15.104:0/1138294256' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:11:24.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:24 smithi104 bash[9203]: audit 2020-09-08T16:11:23.285888+0000 mon.c (mon.2) 94 : audit [DBG] from='client.? v1:172.21.15.104:0/1138294256' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:11:24.741 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:11:24.811 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T16:11:24.846 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:11:24.909 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:11:24 smithi055 bash[16646]: debug 2020-09-08T16:11:24.745+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:24.954 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:11:24.971 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:11:25.063 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:25.272 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:11:24 smithi055 bash[16646]: debug 2020-09-08T16:11:24.953+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:25.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:25 smithi104 bash[9203]: cluster 2020-09-08T16:11:24.141093+0000 mgr.x (mgr.24118) 1322 : cluster [DBG] pgmap v1976: 17 pgs: 17 active+clean; 7.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 963 op/s 2020-09-08T16:11:25.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:25 smithi166 bash[9446]: cluster 2020-09-08T16:11:24.141093+0000 mgr.x (mgr.24118) 1322 : cluster [DBG] pgmap v1976: 17 pgs: 17 active+clean; 7.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 963 op/s 2020-09-08T16:11:25.627 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:25.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:25 smithi055 bash[9296]: cluster 2020-09-08T16:11:24.141093+0000 mgr.x (mgr.24118) 1322 : cluster [DBG] pgmap v1976: 17 pgs: 17 active+clean; 7.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 963 op/s 2020-09-08T16:11:25.730 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:11:25.929 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:11:25 smithi104 bash[19922]: debug 2020-09-08T16:11:25.624+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:26.171 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:11:26.274 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:26.616 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:11:26 smithi166 bash[18960]: debug 2020-09-08T16:11:26.171+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:26.851 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:26.947 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:11:26.957 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:11:27.109 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:11:27.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:11:26 smithi166 bash[11928]: debug 2020-09-08T16:11:26.851+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:27.158 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:11:27.200 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:11:27 smithi104 bash[15273]: debug 2020-09-08T16:11:27.156+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:27.264 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:27.622 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:27 smithi166 bash[9446]: cluster 2020-09-08T16:11:26.141538+0000 mgr.x (mgr.24118) 1323 : cluster [DBG] pgmap v1977: 17 pgs: 17 active+clean; 7.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 963 op/s 2020-09-08T16:11:27.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:27 smithi055 bash[9296]: cluster 2020-09-08T16:11:26.141538+0000 mgr.x (mgr.24118) 1323 : cluster [DBG] pgmap v1977: 17 pgs: 17 active+clean; 7.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 963 op/s 2020-09-08T16:11:27.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:27 smithi104 bash[9203]: cluster 2020-09-08T16:11:26.141538+0000 mgr.x (mgr.24118) 1323 : cluster [DBG] pgmap v1977: 17 pgs: 17 active+clean; 7.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 963 op/s 2020-09-08T16:11:27.731 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:27.835 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:27.984 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:11:27 smithi166 bash[14254]: debug 2020-09-08T16:11:27.727+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:28.361 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:28.362 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:11:28 smithi104 bash[19922]: debug 2020-09-08T16:11:28.016+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:28.463 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:11:28.573 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:11:28.574 INFO:teuthology.orchestra.run.smithi166.stdout: "size": 20, 2020-09-08T16:11:28.574 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 600, 2020-09-08T16:11:28.574 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T16:11:28.574 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.575 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:135173 8.3 8:c13136b4:::benchmark_data_smithi166_52302_object4252:head [delete] snapc 0=[] ondisk+write+known_if_redirected e661)", 2020-09-08T16:11:28.575 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:06:42.015999+0000", 2020-09-08T16:11:28.575 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 286.55974900500001, 2020-09-08T16:11:28.575 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15268203599999999, 2020-09-08T16:11:28.576 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:28.576 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:28.576 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:28.576 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:28.577 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:28.577 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 135173 2020-09-08T16:11:28.577 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.577 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:28.577 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.578 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:28.578 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:42.015999+0000", 2020-09-08T16:11:28.578 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:28.578 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.578 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.579 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:28.579 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:42.015999+0000", 2020-09-08T16:11:28.579 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999604 2020-09-08T16:11:28.579 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.579 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.580 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:28.580 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:42.015960+0000", 2020-09-08T16:11:28.580 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.3566999999999994e-05 2020-09-08T16:11:28.580 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.580 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.581 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:28.581 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:42.016053+0000", 2020-09-08T16:11:28.581 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5954999999999999e-05 2020-09-08T16:11:28.581 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.581 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.582 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:28.582 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:42.016069+0000", 2020-09-08T16:11:28.582 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.404e-06 2020-09-08T16:11:28.583 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.583 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.583 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:28.584 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:42.016076+0000", 2020-09-08T16:11:28.584 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7167000000000002e-05 2020-09-08T16:11:28.584 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.584 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.585 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:28.585 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:42.016153+0000", 2020-09-08T16:11:28.585 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018337099999999999 2020-09-08T16:11:28.585 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.586 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.586 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:28.592 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:42.016336+0000", 2020-09-08T16:11:28.593 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000103471 2020-09-08T16:11:28.593 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.594 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.594 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T16:11:28.598 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:42.016440+0000", 2020-09-08T16:11:28.598 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0042640539999999998 2020-09-08T16:11:28.598 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.599 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.599 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:28.599 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:42.020704+0000", 2020-09-08T16:11:28.600 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14781022899999999 2020-09-08T16:11:28.600 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.600 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.601 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:28.602 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:42.168514+0000", 2020-09-08T16:11:28.602 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000103385 2020-09-08T16:11:28.602 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.603 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.603 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:28.604 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:42.168617+0000", 2020-09-08T16:11:28.604 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.4046000000000003e-05 2020-09-08T16:11:28.606 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.606 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.607 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:28.607 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:06:42.168681+0000", 2020-09-08T16:11:28.607 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15268203599999999 2020-09-08T16:11:28.608 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.608 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.608 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.609 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.609 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.609 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:181038 8.3 8:cbeed11a:::benchmark_data_smithi166_52302_object50117:head [delete] snapc 0=[] ondisk+write+known_if_redirected e671)", 2020-09-08T16:11:28.610 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:07:31.118547+0000", 2020-09-08T16:11:28.610 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 237.45720108500001, 2020-09-08T16:11:28.610 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.162554116, 2020-09-08T16:11:28.611 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:28.611 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:28.611 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:28.612 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:28.615 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:11:28.616 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:28.616 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 181038 2020-09-08T16:11:28.640 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.641 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:28.641 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.641 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:28.642 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.118547+0000", 2020-09-08T16:11:28.642 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:28.642 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.642 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.643 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:28.643 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.118547+0000", 2020-09-08T16:11:28.643 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999652 2020-09-08T16:11:28.643 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.643 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.644 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:28.644 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.118512+0000", 2020-09-08T16:11:28.644 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.8274000000000003e-05 2020-09-08T16:11:28.644 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.644 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.645 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:28.645 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.118591+0000", 2020-09-08T16:11:28.645 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6220999999999999e-05 2020-09-08T16:11:28.645 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.645 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.646 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:28.646 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.118607+0000", 2020-09-08T16:11:28.646 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.3420000000000002e-06 2020-09-08T16:11:28.646 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.646 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.647 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:28.647 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.118614+0000", 2020-09-08T16:11:28.647 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.001810341 2020-09-08T16:11:28.647 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.647 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.648 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:28.649 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.120424+0000", 2020-09-08T16:11:28.649 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012522299999999999 2020-09-08T16:11:28.649 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.649 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.650 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:28.650 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.120550+0000", 2020-09-08T16:11:28.651 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000132118 2020-09-08T16:11:28.651 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.651 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.652 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T16:11:28.652 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.120682+0000", 2020-09-08T16:11:28.652 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0075874430000000001 2020-09-08T16:11:28.652 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.652 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.654 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:28.654 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.128269+0000", 2020-09-08T16:11:28.654 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.152695529 2020-09-08T16:11:28.654 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.655 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.655 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:28.655 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.280965+0000", 2020-09-08T16:11:28.655 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.1538000000000004e-05 2020-09-08T16:11:28.655 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.655 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.656 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:28.656 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.281046+0000", 2020-09-08T16:11:28.656 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.5111000000000001e-05 2020-09-08T16:11:28.656 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.656 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.657 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:28.657 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.281101+0000", 2020-09-08T16:11:28.657 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.162554116 2020-09-08T16:11:28.657 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.657 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.659 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.659 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.659 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.659 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:181047 8.3 8:c498f7f4:::benchmark_data_smithi166_52302_object50126:head [delete] snapc 0=[] ondisk+write+known_if_redirected e671)", 2020-09-08T16:11:28.660 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:07:31.128682+0000", 2020-09-08T16:11:28.660 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 237.447066323, 2020-09-08T16:11:28.660 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15772602899999999, 2020-09-08T16:11:28.660 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:28.660 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:28.661 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:28.662 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:28.663 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:28.663 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 181047 2020-09-08T16:11:28.663 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.663 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:28.663 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.664 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:28.664 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.128682+0000", 2020-09-08T16:11:28.664 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:28.664 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.664 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.665 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:28.665 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.128682+0000", 2020-09-08T16:11:28.665 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999647 2020-09-08T16:11:28.665 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.666 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.666 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:28.668 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.128647+0000", 2020-09-08T16:11:28.669 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.9680999999999998e-05 2020-09-08T16:11:28.669 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.670 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.670 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:28.670 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.128726+0000", 2020-09-08T16:11:28.670 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3332e-05 2020-09-08T16:11:28.671 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.671 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.671 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:28.672 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.128740+0000", 2020-09-08T16:11:28.672 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.9689999999999999e-06 2020-09-08T16:11:28.672 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.672 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.672 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:28.673 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.128746+0000", 2020-09-08T16:11:28.673 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.5569999999999999e-05 2020-09-08T16:11:28.673 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.673 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.673 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:28.674 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.128791+0000", 2020-09-08T16:11:28.674 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000143986 2020-09-08T16:11:28.674 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.674 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.674 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:28.675 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.128935+0000", 2020-09-08T16:11:28.675 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018521100000000001 2020-09-08T16:11:28.675 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.675 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.675 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T16:11:28.676 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.129120+0000", 2020-09-08T16:11:28.676 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0064512069999999996 2020-09-08T16:11:28.676 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.676 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.676 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:28.677 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.135572+0000", 2020-09-08T16:11:28.677 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15070072500000001 2020-09-08T16:11:28.677 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.677 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.677 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:28.677 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.286272+0000", 2020-09-08T16:11:28.678 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.3431999999999998e-05 2020-09-08T16:11:28.678 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.678 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.678 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:28.679 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.286346+0000", 2020-09-08T16:11:28.680 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.2194999999999997e-05 2020-09-08T16:11:28.680 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.680 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.680 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:28.680 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:31.286408+0000", 2020-09-08T16:11:28.681 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15772602899999999 2020-09-08T16:11:28.681 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.681 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.681 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.681 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.682 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.682 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:198924 8.f 8:fc7d0ac1:::benchmark_data_smithi166_52302_object68003:head [delete] snapc 0=[] ondisk+write+known_if_redirected e676)", 2020-09-08T16:11:28.682 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:07:51.232693+0000", 2020-09-08T16:11:28.682 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 217.34305570199999, 2020-09-08T16:11:28.682 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15914004200000001, 2020-09-08T16:11:28.683 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:28.683 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:28.683 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:28.683 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:28.683 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:28.684 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 198924 2020-09-08T16:11:28.684 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.684 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:28.684 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.684 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:28.685 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.232693+0000", 2020-09-08T16:11:28.685 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:28.685 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.685 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.685 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:28.686 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.232693+0000", 2020-09-08T16:11:28.686 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999599 2020-09-08T16:11:28.686 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.686 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.686 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:28.687 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.232653+0000", 2020-09-08T16:11:28.687 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.1421000000000004e-05 2020-09-08T16:11:28.687 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.687 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.687 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:28.688 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.232744+0000", 2020-09-08T16:11:28.688 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5404e-05 2020-09-08T16:11:28.688 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.688 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.688 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:28.689 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.232759+0000", 2020-09-08T16:11:28.689 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.7170000000000002e-06 2020-09-08T16:11:28.689 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.689 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.690 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:28.690 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.232766+0000", 2020-09-08T16:11:28.690 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.011520588999999999 2020-09-08T16:11:28.690 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.691 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.691 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:28.691 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.244287+0000", 2020-09-08T16:11:28.692 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000114181 2020-09-08T16:11:28.692 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.692 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.692 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:28.693 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.244401+0000", 2020-09-08T16:11:28.693 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.9686999999999997e-05 2020-09-08T16:11:28.693 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.694 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.694 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:11:28.694 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.244491+0000", 2020-09-08T16:11:28.694 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.010291621000000001 2020-09-08T16:11:28.694 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.695 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.695 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:28.695 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.254782+0000", 2020-09-08T16:11:28.695 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.136922356 2020-09-08T16:11:28.695 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.696 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.696 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:28.696 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.391705+0000", 2020-09-08T16:11:28.696 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.1285999999999995e-05 2020-09-08T16:11:28.696 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.697 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.697 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:28.697 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.391776+0000", 2020-09-08T16:11:28.697 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.6895000000000003e-05 2020-09-08T16:11:28.697 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.698 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.698 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:28.698 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.391833+0000", 2020-09-08T16:11:28.698 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15914004200000001 2020-09-08T16:11:28.699 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.699 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.699 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.699 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.699 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.700 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:198928 8.f 8:fb528eab:::benchmark_data_smithi166_52302_object68007:head [delete] snapc 0=[] ondisk+write+known_if_redirected e676)", 2020-09-08T16:11:28.700 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:07:51.235600+0000", 2020-09-08T16:11:28.700 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 217.34014858800001, 2020-09-08T16:11:28.700 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.163943388, 2020-09-08T16:11:28.700 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:28.701 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:28.701 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:28.701 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:28.701 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:28.701 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 198928 2020-09-08T16:11:28.702 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.702 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:28.702 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.702 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:28.702 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.235600+0000", 2020-09-08T16:11:28.703 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:28.703 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.703 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.703 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:28.703 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.235600+0000", 2020-09-08T16:11:28.704 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999638 2020-09-08T16:11:28.704 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.704 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.704 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:28.704 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.235563+0000", 2020-09-08T16:11:28.705 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.0171000000000002e-05 2020-09-08T16:11:28.705 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.705 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.705 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:28.705 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.235644+0000", 2020-09-08T16:11:28.706 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4073999999999999e-05 2020-09-08T16:11:28.706 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.706 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.706 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:28.706 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.235658+0000", 2020-09-08T16:11:28.706 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.1739999999999997e-06 2020-09-08T16:11:28.707 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.707 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.707 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:28.707 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.235664+0000", 2020-09-08T16:11:28.707 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.019151116999999999 2020-09-08T16:11:28.708 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.708 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.708 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:28.708 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.254815+0000", 2020-09-08T16:11:28.708 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016073699999999999 2020-09-08T16:11:28.709 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.709 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.709 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:28.709 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.254976+0000", 2020-09-08T16:11:28.709 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013975599999999999 2020-09-08T16:11:28.710 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.710 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.710 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 2", 2020-09-08T16:11:28.710 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.255115+0000", 2020-09-08T16:11:28.710 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.128399187 2020-09-08T16:11:28.711 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.711 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.711 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:28.711 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.383515+0000", 2020-09-08T16:11:28.711 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.015889294000000002 2020-09-08T16:11:28.712 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.712 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.712 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:28.712 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.399404+0000", 2020-09-08T16:11:28.713 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.9902999999999997e-05 2020-09-08T16:11:28.713 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.713 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.713 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:28.713 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.399484+0000", 2020-09-08T16:11:28.714 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.9277999999999997e-05 2020-09-08T16:11:28.714 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.714 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.714 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:28.715 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.399543+0000", 2020-09-08T16:11:28.715 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.163943388 2020-09-08T16:11:28.715 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.715 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.715 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.716 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.716 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.716 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:198934 8.d 8:b6b57ac6:::benchmark_data_smithi166_52302_object68013:head [delete] snapc 0=[] ondisk+write+known_if_redirected e676)", 2020-09-08T16:11:28.716 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:07:51.248498+0000", 2020-09-08T16:11:28.716 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 217.327249999, 2020-09-08T16:11:28.717 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.158280475, 2020-09-08T16:11:28.717 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:28.717 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:28.717 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:28.717 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:28.718 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:28.718 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 198934 2020-09-08T16:11:28.718 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.718 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:28.718 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.719 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:28.719 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.248498+0000", 2020-09-08T16:11:28.719 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:28.719 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.720 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.720 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:28.720 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.248498+0000", 2020-09-08T16:11:28.720 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999576 2020-09-08T16:11:28.720 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.721 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.721 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:28.721 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.248456+0000", 2020-09-08T16:11:28.721 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.4215999999999994e-05 2020-09-08T16:11:28.721 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.722 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.722 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:28.722 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.248550+0000", 2020-09-08T16:11:28.722 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5744999999999999e-05 2020-09-08T16:11:28.722 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.723 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.723 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:28.723 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.248566+0000", 2020-09-08T16:11:28.723 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.3220000000000004e-06 2020-09-08T16:11:28.723 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.724 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.725 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:28.725 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.248573+0000", 2020-09-08T16:11:28.725 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.6353000000000002e-05 2020-09-08T16:11:28.725 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.726 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.726 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:28.727 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.248619+0000", 2020-09-08T16:11:28.727 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012769800000000001 2020-09-08T16:11:28.727 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.727 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.728 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:28.728 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.248747+0000", 2020-09-08T16:11:28.728 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013172499999999999 2020-09-08T16:11:28.728 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.728 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.729 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 7", 2020-09-08T16:11:28.729 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.248879+0000", 2020-09-08T16:11:28.729 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.011846372000000001 2020-09-08T16:11:28.729 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.730 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.730 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:28.730 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.260725+0000", 2020-09-08T16:11:28.730 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14568561099999999 2020-09-08T16:11:28.731 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.731 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.731 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:28.731 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.406411+0000", 2020-09-08T16:11:28.732 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000318525 2020-09-08T16:11:28.732 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.732 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.732 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:28.733 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.406729+0000", 2020-09-08T16:11:28.733 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.9518999999999998e-05 2020-09-08T16:11:28.733 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.733 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.733 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:28.734 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.406779+0000", 2020-09-08T16:11:28.734 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.158280475 2020-09-08T16:11:28.734 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.734 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.735 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.735 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.735 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.735 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:198939 8.d 8:bebdc7d4:::benchmark_data_smithi166_52302_object68018:head [delete] snapc 0=[] ondisk+write+known_if_redirected e676)", 2020-09-08T16:11:28.736 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:07:51.252487+0000", 2020-09-08T16:11:28.736 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 217.32326165000001, 2020-09-08T16:11:28.736 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15464972900000001, 2020-09-08T16:11:28.737 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:28.737 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:28.737 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:28.737 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:28.738 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:28.738 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 198939 2020-09-08T16:11:28.738 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.738 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:28.739 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.739 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:28.739 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.252487+0000", 2020-09-08T16:11:28.739 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:28.740 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.740 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.740 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:28.740 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.252487+0000", 2020-09-08T16:11:28.741 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999599 2020-09-08T16:11:28.741 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.741 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.741 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:28.741 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.252447+0000", 2020-09-08T16:11:28.742 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.9095e-05 2020-09-08T16:11:28.742 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.742 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.742 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:28.743 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.252536+0000", 2020-09-08T16:11:28.743 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4756999999999999e-05 2020-09-08T16:11:28.743 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.743 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.743 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:28.744 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.252551+0000", 2020-09-08T16:11:28.744 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.2989999999999999e-06 2020-09-08T16:11:28.744 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.744 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.745 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:28.745 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.252558+0000", 2020-09-08T16:11:28.745 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0019834509999999998 2020-09-08T16:11:28.745 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.746 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.746 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:28.746 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.254541+0000", 2020-09-08T16:11:28.746 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015992299999999999 2020-09-08T16:11:28.747 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.747 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.748 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:28.748 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.254701+0000", 2020-09-08T16:11:28.748 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016391100000000001 2020-09-08T16:11:28.748 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.748 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.749 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 7", 2020-09-08T16:11:28.749 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.254865+0000", 2020-09-08T16:11:28.749 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.142784516 2020-09-08T16:11:28.749 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.750 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.750 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:28.750 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.397650+0000", 2020-09-08T16:11:28.750 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0093354189999999993 2020-09-08T16:11:28.751 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.751 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.751 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:28.751 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.406985+0000", 2020-09-08T16:11:28.751 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000107494 2020-09-08T16:11:28.752 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.752 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.752 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:28.752 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.407093+0000", 2020-09-08T16:11:28.752 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.3829999999999999e-05 2020-09-08T16:11:28.753 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.753 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.753 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:28.753 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:07:51.407136+0000", 2020-09-08T16:11:28.754 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15464972900000001 2020-09-08T16:11:28.754 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.754 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.754 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.754 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.755 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.755 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.36335.0:226807 8.3 8:c27332cd:::benchmark_data_smithi166_52302_object95886:head [delete] snapc 0=[] ondisk+write+known_if_redirected e685)", 2020-09-08T16:11:28.755 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:08:23.617304+0000", 2020-09-08T16:11:28.755 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 184.95844396800001, 2020-09-08T16:11:28.756 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15469718900000001, 2020-09-08T16:11:28.756 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:28.756 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:28.756 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:28.757 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.36335", 2020-09-08T16:11:28.757 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:11:28.758 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 226807 2020-09-08T16:11:28.758 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.758 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:28.758 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.759 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:28.759 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:23.617304+0000", 2020-09-08T16:11:28.759 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:28.759 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.760 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.760 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:28.760 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:23.617304+0000", 2020-09-08T16:11:28.760 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999609 2020-09-08T16:11:28.760 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.761 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.761 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:28.761 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:23.617265+0000", 2020-09-08T16:11:28.761 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.7559000000000007e-05 2020-09-08T16:11:28.761 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.762 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.762 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:28.762 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:23.617353+0000", 2020-09-08T16:11:28.762 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3536000000000001e-05 2020-09-08T16:11:28.763 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.763 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.763 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:28.763 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:23.617366+0000", 2020-09-08T16:11:28.763 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7570000000000001e-06 2020-09-08T16:11:28.764 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.764 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.764 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:28.764 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:23.617372+0000", 2020-09-08T16:11:28.765 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0056289779999999998 2020-09-08T16:11:28.765 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.765 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.765 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:28.766 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:23.623001+0000", 2020-09-08T16:11:28.766 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013413300000000001 2020-09-08T16:11:28.766 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.766 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.767 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:28.767 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:23.623135+0000", 2020-09-08T16:11:28.767 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013754899999999999 2020-09-08T16:11:28.767 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.768 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.768 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 1", 2020-09-08T16:11:28.768 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:23.623273+0000", 2020-09-08T16:11:28.769 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0052596400000000003 2020-09-08T16:11:28.769 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.769 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.769 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:28.770 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:23.628532+0000", 2020-09-08T16:11:28.770 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.14316704499999999 2020-09-08T16:11:28.770 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.770 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.771 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:28.771 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:23.771699+0000", 2020-09-08T16:11:28.771 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00024241099999999999 2020-09-08T16:11:28.771 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.771 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.772 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:28.772 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:23.771942+0000", 2020-09-08T16:11:28.772 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.9902999999999999e-05 2020-09-08T16:11:28.772 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.773 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.773 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:28.773 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:08:23.772002+0000", 2020-09-08T16:11:28.774 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15469718900000001 2020-09-08T16:11:28.774 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.774 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.775 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.775 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.775 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.776 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:15780 9.5 9:ab493b4e:::benchmark_data_smithi166_56754_object15779:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e708)", 2020-09-08T16:11:28.776 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:09:36.720082+0000", 2020-09-08T16:11:28.776 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 111.855666169, 2020-09-08T16:11:28.776 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21316531599999999, 2020-09-08T16:11:28.777 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:28.777 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:28.777 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:28.777 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:11:28.777 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:11:28.778 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 15780 2020-09-08T16:11:28.778 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.779 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:28.779 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.779 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:28.779 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.720082+0000", 2020-09-08T16:11:28.780 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:28.780 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.780 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.780 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:28.781 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.720082+0000", 2020-09-08T16:11:28.781 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999571 2020-09-08T16:11:28.781 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.782 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.782 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:28.782 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.720039+0000", 2020-09-08T16:11:28.782 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00012099699999999999 2020-09-08T16:11:28.782 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.783 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.783 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:28.783 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.720160+0000", 2020-09-08T16:11:28.783 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4328000000000001e-05 2020-09-08T16:11:28.784 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.784 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.784 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:28.784 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.720175+0000", 2020-09-08T16:11:28.784 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0008e-05 2020-09-08T16:11:28.785 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.785 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.785 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:28.785 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.720185+0000", 2020-09-08T16:11:28.786 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.4428e-05 2020-09-08T16:11:28.786 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.786 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.786 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:28.786 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.720249+0000", 2020-09-08T16:11:28.787 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014029299999999999 2020-09-08T16:11:28.787 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.787 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.787 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:28.788 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.720389+0000", 2020-09-08T16:11:28.788 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00035429999999999999 2020-09-08T16:11:28.788 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.790 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.790 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:28.790 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.720744+0000", 2020-09-08T16:11:28.791 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0018767219999999999 2020-09-08T16:11:28.791 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.791 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.791 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:28.791 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.722620+0000", 2020-09-08T16:11:28.792 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.210377334 2020-09-08T16:11:28.792 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.792 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.792 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:28.793 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.932998+0000", 2020-09-08T16:11:28.793 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013217 2020-09-08T16:11:28.794 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.794 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.794 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:28.794 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.933130+0000", 2020-09-08T16:11:28.794 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000117601 2020-09-08T16:11:28.795 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.795 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.795 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:28.795 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.933247+0000", 2020-09-08T16:11:28.795 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21316531599999999 2020-09-08T16:11:28.796 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.796 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.796 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.796 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.797 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.797 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:15782 9.5 9:afb622a3:::benchmark_data_smithi166_56754_object15781:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e708)", 2020-09-08T16:11:28.798 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:09:36.720519+0000", 2020-09-08T16:11:28.798 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 111.855229507, 2020-09-08T16:11:28.799 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21424516499999999, 2020-09-08T16:11:28.799 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:28.799 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:28.800 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:28.800 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:11:28.800 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:11:28.800 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 15782 2020-09-08T16:11:28.801 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.801 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:28.801 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.801 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:28.802 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.720519+0000", 2020-09-08T16:11:28.802 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:28.802 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.802 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.803 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:28.803 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.720519+0000", 2020-09-08T16:11:28.803 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999628 2020-09-08T16:11:28.803 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.804 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.804 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:28.804 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.720481+0000", 2020-09-08T16:11:28.804 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000115963 2020-09-08T16:11:28.805 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.805 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.805 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:28.805 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.720597+0000", 2020-09-08T16:11:28.806 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3704e-05 2020-09-08T16:11:28.806 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.806 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.806 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:28.807 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.720611+0000", 2020-09-08T16:11:28.807 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.9359999999999996e-06 2020-09-08T16:11:28.808 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.808 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.808 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:28.808 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.720617+0000", 2020-09-08T16:11:28.809 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00092850300000000001 2020-09-08T16:11:28.809 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.809 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.809 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:28.810 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.721546+0000", 2020-09-08T16:11:28.810 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017192299999999999 2020-09-08T16:11:28.810 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.810 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.810 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:28.811 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.721717+0000", 2020-09-08T16:11:28.811 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017305099999999999 2020-09-08T16:11:28.811 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.811 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.812 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:28.812 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.721891+0000", 2020-09-08T16:11:28.812 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00088911899999999996 2020-09-08T16:11:28.812 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.813 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.813 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:28.813 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.722780+0000", 2020-09-08T16:11:28.813 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21179177099999999 2020-09-08T16:11:28.813 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.814 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.814 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:28.814 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.934571+0000", 2020-09-08T16:11:28.814 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000100865 2020-09-08T16:11:28.815 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.815 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.815 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:28.815 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.934672+0000", 2020-09-08T16:11:28.816 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.1731000000000007e-05 2020-09-08T16:11:28.816 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.816 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.816 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:28.817 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.934764+0000", 2020-09-08T16:11:28.817 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.21424516499999999 2020-09-08T16:11:28.817 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.817 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.818 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.818 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.819 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.819 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:15803 9.5 9:a8eeb4d0:::benchmark_data_smithi166_56754_object15802:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e708)", 2020-09-08T16:11:28.819 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:09:36.739461+0000", 2020-09-08T16:11:28.819 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 111.836287634, 2020-09-08T16:11:28.820 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19571539499999999, 2020-09-08T16:11:28.820 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:28.820 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:28.820 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:28.820 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:11:28.821 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:11:28.821 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 15803 2020-09-08T16:11:28.821 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.821 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:28.822 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.822 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:28.822 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.739461+0000", 2020-09-08T16:11:28.822 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:28.822 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.823 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.823 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:28.823 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.739461+0000", 2020-09-08T16:11:28.823 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999638 2020-09-08T16:11:28.823 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.824 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.824 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:28.824 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.739424+0000", 2020-09-08T16:11:28.824 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000137343 2020-09-08T16:11:28.824 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.825 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.825 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:28.825 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.739562+0000", 2020-09-08T16:11:28.825 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.5143999999999999e-05 2020-09-08T16:11:28.826 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.826 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.826 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:28.826 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.739577+0000", 2020-09-08T16:11:28.826 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.3999999999999997e-06 2020-09-08T16:11:28.827 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.827 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.827 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:28.827 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.739583+0000", 2020-09-08T16:11:28.827 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.1898999999999997e-05 2020-09-08T16:11:28.828 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.828 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.829 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:28.829 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.739655+0000", 2020-09-08T16:11:28.829 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00024058400000000001 2020-09-08T16:11:28.829 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.829 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.830 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:28.830 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.739896+0000", 2020-09-08T16:11:28.830 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00030182899999999999 2020-09-08T16:11:28.830 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.831 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.831 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:28.831 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.740198+0000", 2020-09-08T16:11:28.831 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.001643666 2020-09-08T16:11:28.831 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.832 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.832 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:28.832 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.741841+0000", 2020-09-08T16:11:28.832 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.193023259 2020-09-08T16:11:28.833 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.833 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.833 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:28.833 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.934865+0000", 2020-09-08T16:11:28.834 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017378700000000001 2020-09-08T16:11:28.834 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.834 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.834 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:28.834 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.935038+0000", 2020-09-08T16:11:28.835 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000137804 2020-09-08T16:11:28.835 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.835 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.835 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:28.835 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.935176+0000", 2020-09-08T16:11:28.836 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19571539499999999 2020-09-08T16:11:28.836 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.836 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.836 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.837 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.837 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.837 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:15805 9.5 9:a0df1439:::benchmark_data_smithi166_56754_object15804:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e708)", 2020-09-08T16:11:28.837 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:09:36.739635+0000", 2020-09-08T16:11:28.837 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 111.836112908, 2020-09-08T16:11:28.838 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19601616299999999, 2020-09-08T16:11:28.838 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:28.838 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:28.838 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:28.839 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:11:28.839 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:11:28.839 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 15805 2020-09-08T16:11:28.840 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.840 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:28.840 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.840 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:28.840 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.739635+0000", 2020-09-08T16:11:28.841 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:28.841 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.841 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.841 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:28.841 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.739635+0000", 2020-09-08T16:11:28.842 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999743 2020-09-08T16:11:28.842 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.842 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.842 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:28.842 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.739610+0000", 2020-09-08T16:11:28.842 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.4853999999999998e-05 2020-09-08T16:11:28.843 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.843 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.843 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:28.843 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.739695+0000", 2020-09-08T16:11:28.843 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.8300000000000002e-06 2020-09-08T16:11:28.844 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.844 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.844 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:28.844 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.739704+0000", 2020-09-08T16:11:28.844 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.1710000000000004e-06 2020-09-08T16:11:28.845 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.845 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.845 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:28.845 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.739708+0000", 2020-09-08T16:11:28.845 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0013076870000000001 2020-09-08T16:11:28.846 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.846 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.846 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:28.846 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.741015+0000", 2020-09-08T16:11:28.846 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000142044 2020-09-08T16:11:28.847 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.847 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.847 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:28.847 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.741157+0000", 2020-09-08T16:11:28.847 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016812899999999999 2020-09-08T16:11:28.847 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.848 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.848 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:28.848 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.741326+0000", 2020-09-08T16:11:28.848 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00071279899999999996 2020-09-08T16:11:28.848 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.849 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.849 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:28.849 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.742038+0000", 2020-09-08T16:11:28.850 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19327646400000001 2020-09-08T16:11:28.850 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.850 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.850 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:28.850 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.935315+0000", 2020-09-08T16:11:28.851 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000145642 2020-09-08T16:11:28.851 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.851 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.851 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:28.851 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.935460+0000", 2020-09-08T16:11:28.852 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019113600000000001 2020-09-08T16:11:28.852 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.852 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.852 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:28.852 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.935652+0000", 2020-09-08T16:11:28.853 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19601616299999999 2020-09-08T16:11:28.853 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.853 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.853 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.853 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.854 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.854 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:15827 9.5 9:ab0dc071:::benchmark_data_smithi166_56754_object15826:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e708)", 2020-09-08T16:11:28.854 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:09:36.749901+0000", 2020-09-08T16:11:28.854 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 111.825846907, 2020-09-08T16:11:28.854 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.188105144, 2020-09-08T16:11:28.855 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:28.855 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:28.855 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:28.855 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:11:28.855 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:11:28.855 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 15827 2020-09-08T16:11:28.856 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.856 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:28.856 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.856 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:28.856 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.749901+0000", 2020-09-08T16:11:28.857 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:28.857 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.857 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.857 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:28.857 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.749901+0000", 2020-09-08T16:11:28.858 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999223 2020-09-08T16:11:28.858 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.858 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.858 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:28.858 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.749824+0000", 2020-09-08T16:11:28.859 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00023521100000000001 2020-09-08T16:11:28.859 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.859 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.859 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:28.860 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.750059+0000", 2020-09-08T16:11:28.860 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.2490000000000002e-05 2020-09-08T16:11:28.860 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.860 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.861 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:28.861 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.750091+0000", 2020-09-08T16:11:28.861 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4661000000000001e-05 2020-09-08T16:11:28.861 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.861 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.862 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:28.862 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.750106+0000", 2020-09-08T16:11:28.862 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.1385999999999996e-05 2020-09-08T16:11:28.862 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.862 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.863 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:28.863 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.750188+0000", 2020-09-08T16:11:28.863 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000268695 2020-09-08T16:11:28.863 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.863 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.864 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:28.864 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.750456+0000", 2020-09-08T16:11:28.864 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00031533800000000002 2020-09-08T16:11:28.864 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.864 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.865 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:28.865 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.750772+0000", 2020-09-08T16:11:28.865 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0028515770000000001 2020-09-08T16:11:28.865 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.865 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.866 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:28.866 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.753623+0000", 2020-09-08T16:11:28.866 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.183834629 2020-09-08T16:11:28.866 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.866 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.866 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:28.867 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.937458+0000", 2020-09-08T16:11:28.867 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00045314500000000001 2020-09-08T16:11:28.867 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.867 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.867 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:28.868 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.937911+0000", 2020-09-08T16:11:28.868 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.5657999999999998e-05 2020-09-08T16:11:28.868 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.868 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.868 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:28.869 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.938007+0000", 2020-09-08T16:11:28.869 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.188105144 2020-09-08T16:11:28.869 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.869 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.870 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.870 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.870 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.871 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:15830 9.5 9:a1d54068:::benchmark_data_smithi166_56754_object15829:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e708)", 2020-09-08T16:11:28.871 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:09:36.751558+0000", 2020-09-08T16:11:28.871 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 111.82419023200001, 2020-09-08T16:11:28.871 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18679747299999999, 2020-09-08T16:11:28.871 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:28.872 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:28.872 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:28.872 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:11:28.872 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:11:28.872 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 15830 2020-09-08T16:11:28.873 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.873 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:28.873 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.873 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:28.873 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.751558+0000", 2020-09-08T16:11:28.874 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:28.874 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.874 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.874 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:28.874 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.751558+0000", 2020-09-08T16:11:28.875 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999328 2020-09-08T16:11:28.875 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.875 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.875 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:28.875 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.751491+0000", 2020-09-08T16:11:28.875 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018638300000000001 2020-09-08T16:11:28.876 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.876 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.876 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:28.876 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.751677+0000", 2020-09-08T16:11:28.876 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.3872999999999998e-05 2020-09-08T16:11:28.877 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.877 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.877 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:28.877 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.751721+0000", 2020-09-08T16:11:28.877 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4284000000000001e-05 2020-09-08T16:11:28.878 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.878 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.878 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:28.878 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.751735+0000", 2020-09-08T16:11:28.878 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000101576 2020-09-08T16:11:28.879 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.879 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.879 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:28.879 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.751837+0000", 2020-09-08T16:11:28.879 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000263799 2020-09-08T16:11:28.880 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.881 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.881 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:28.881 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.752101+0000", 2020-09-08T16:11:28.881 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00039629699999999999 2020-09-08T16:11:28.882 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.882 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.882 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:28.882 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.752497+0000", 2020-09-08T16:11:28.882 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0015834130000000001 2020-09-08T16:11:28.883 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.883 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.883 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:28.883 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.754080+0000", 2020-09-08T16:11:28.883 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18406562600000001 2020-09-08T16:11:28.884 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.884 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.884 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:28.884 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.938146+0000", 2020-09-08T16:11:28.884 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000116888 2020-09-08T16:11:28.885 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.885 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.885 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:28.885 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.938263+0000", 2020-09-08T16:11:28.885 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.2578999999999996e-05 2020-09-08T16:11:28.886 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.886 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.886 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:28.886 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.938356+0000", 2020-09-08T16:11:28.886 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18679747299999999 2020-09-08T16:11:28.886 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.887 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.887 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.887 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.887 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.888 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:15850 9.5 9:ac66aae1:::benchmark_data_smithi166_56754_object15849:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e708)", 2020-09-08T16:11:28.888 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:09:36.758410+0000", 2020-09-08T16:11:28.888 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 111.817338705, 2020-09-08T16:11:28.888 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18075217399999999, 2020-09-08T16:11:28.888 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:28.889 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:28.889 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:28.889 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:11:28.889 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:11:28.889 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 15850 2020-09-08T16:11:28.889 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.890 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:28.890 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.890 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:28.890 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.758410+0000", 2020-09-08T16:11:28.890 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:28.891 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.891 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.891 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:28.892 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.758410+0000", 2020-09-08T16:11:28.892 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999571 2020-09-08T16:11:28.892 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.892 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.892 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:28.893 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.758367+0000", 2020-09-08T16:11:28.893 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000135971 2020-09-08T16:11:28.893 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.893 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.893 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:28.894 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.758503+0000", 2020-09-08T16:11:28.894 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.1548000000000001e-05 2020-09-08T16:11:28.894 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.894 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.894 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:28.895 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.758524+0000", 2020-09-08T16:11:28.895 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0451e-05 2020-09-08T16:11:28.895 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.895 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.895 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:28.896 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.758535+0000", 2020-09-08T16:11:28.896 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.8894999999999998e-05 2020-09-08T16:11:28.896 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.896 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.896 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:28.896 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.758594+0000", 2020-09-08T16:11:28.897 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016027700000000001 2020-09-08T16:11:28.897 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.897 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.897 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:28.897 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.758754+0000", 2020-09-08T16:11:28.898 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019400900000000001 2020-09-08T16:11:28.898 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.898 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.898 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:28.898 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.758948+0000", 2020-09-08T16:11:28.899 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0010051459999999999 2020-09-08T16:11:28.899 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.899 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.899 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:28.899 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.759953+0000", 2020-09-08T16:11:28.900 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17868040800000001 2020-09-08T16:11:28.900 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.900 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.900 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:28.900 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.938634+0000", 2020-09-08T16:11:28.901 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00045141799999999998 2020-09-08T16:11:28.901 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.901 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.902 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:28.902 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.939085+0000", 2020-09-08T16:11:28.902 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.6841000000000005e-05 2020-09-08T16:11:28.902 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.902 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.903 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:28.903 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.939162+0000", 2020-09-08T16:11:28.903 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18075217399999999 2020-09-08T16:11:28.903 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.903 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.904 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.904 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.904 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.904 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:15855 9.5 9:a88652d1:::benchmark_data_smithi166_56754_object15854:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e708)", 2020-09-08T16:11:28.904 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:09:36.760027+0000", 2020-09-08T16:11:28.905 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 111.815721501, 2020-09-08T16:11:28.905 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17955148700000001, 2020-09-08T16:11:28.905 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:28.905 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:28.905 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:28.906 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:11:28.906 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:11:28.906 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 15855 2020-09-08T16:11:28.906 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.906 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:28.907 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.907 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:28.907 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.760027+0000", 2020-09-08T16:11:28.907 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:28.907 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.908 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.908 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:28.908 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.760027+0000", 2020-09-08T16:11:28.908 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999552 2020-09-08T16:11:28.908 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.908 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.909 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:28.909 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.759982+0000", 2020-09-08T16:11:28.909 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013693099999999999 2020-09-08T16:11:28.909 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.909 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.910 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:28.910 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.760119+0000", 2020-09-08T16:11:28.910 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6518e-05 2020-09-08T16:11:28.910 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.910 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.911 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:28.911 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.760136+0000", 2020-09-08T16:11:28.911 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.2219999999999996e-06 2020-09-08T16:11:28.911 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.911 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.912 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:28.912 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.760143+0000", 2020-09-08T16:11:28.912 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018780199999999999 2020-09-08T16:11:28.913 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.913 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.913 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:28.913 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.760331+0000", 2020-09-08T16:11:28.913 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016200000000000001 2020-09-08T16:11:28.914 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.914 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.914 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:28.914 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.760493+0000", 2020-09-08T16:11:28.914 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00019301 2020-09-08T16:11:28.914 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.915 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.915 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:28.915 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.760686+0000", 2020-09-08T16:11:28.915 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00088155899999999997 2020-09-08T16:11:28.915 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.916 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.916 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:28.916 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.761567+0000", 2020-09-08T16:11:28.916 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.177683809 2020-09-08T16:11:28.916 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.917 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.917 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:28.917 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.939251+0000", 2020-09-08T16:11:28.917 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000111518 2020-09-08T16:11:28.917 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.918 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.918 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:28.918 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.939363+0000", 2020-09-08T16:11:28.918 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000215715 2020-09-08T16:11:28.918 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.918 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.919 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:28.919 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.939578+0000", 2020-09-08T16:11:28.919 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17955148700000001 2020-09-08T16:11:28.919 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.920 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.920 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.920 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.920 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.920 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:15867 9.5 9:a9db930d:::benchmark_data_smithi166_56754_object15866:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e708)", 2020-09-08T16:11:28.921 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:09:36.764872+0000", 2020-09-08T16:11:28.921 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 111.810875981, 2020-09-08T16:11:28.921 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17742154700000001, 2020-09-08T16:11:28.921 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:28.921 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:28.922 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:28.922 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:11:28.922 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:11:28.923 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 15867 2020-09-08T16:11:28.923 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.923 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:28.923 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.924 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:28.924 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.764872+0000", 2020-09-08T16:11:28.924 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:28.924 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.925 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.925 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:28.925 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.764872+0000", 2020-09-08T16:11:28.925 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999604 2020-09-08T16:11:28.925 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.926 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.926 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:28.926 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.764833+0000", 2020-09-08T16:11:28.926 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014305700000000001 2020-09-08T16:11:28.926 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.927 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.927 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:28.927 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.764976+0000", 2020-09-08T16:11:28.927 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4739999999999999e-05 2020-09-08T16:11:28.927 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.928 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.928 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:28.928 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.764991+0000", 2020-09-08T16:11:28.928 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.3099999999999997e-06 2020-09-08T16:11:28.928 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.929 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.929 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:28.929 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.764997+0000", 2020-09-08T16:11:28.929 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.5253000000000001e-05 2020-09-08T16:11:28.929 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.930 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.930 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:28.930 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.765052+0000", 2020-09-08T16:11:28.930 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00013918200000000001 2020-09-08T16:11:28.930 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.931 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.931 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:28.931 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.765191+0000", 2020-09-08T16:11:28.931 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000172163 2020-09-08T16:11:28.931 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.932 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.932 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:28.932 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.765364+0000", 2020-09-08T16:11:28.932 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.001008524 2020-09-08T16:11:28.932 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.933 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.933 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:28.933 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.766372+0000", 2020-09-08T16:11:28.934 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17568699099999999 2020-09-08T16:11:28.934 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.934 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.934 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:28.934 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.942059+0000", 2020-09-08T16:11:28.935 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00015235199999999999 2020-09-08T16:11:28.935 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.935 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.935 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:28.935 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.942211+0000", 2020-09-08T16:11:28.936 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.2442999999999997e-05 2020-09-08T16:11:28.936 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.936 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.936 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:28.936 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.942294+0000", 2020-09-08T16:11:28.937 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.17742154700000001 2020-09-08T16:11:28.937 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.937 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.937 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.937 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.938 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.938 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:15897 9.5 9:a76e195c:::benchmark_data_smithi166_56754_object15896:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e708)", 2020-09-08T16:11:28.938 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:09:36.777340+0000", 2020-09-08T16:11:28.938 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 111.798408115, 2020-09-08T16:11:28.938 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.165211893, 2020-09-08T16:11:28.938 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:28.939 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:28.939 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:28.939 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:11:28.939 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:11:28.939 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 15897 2020-09-08T16:11:28.940 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.940 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:28.940 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.940 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:28.940 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.777340+0000", 2020-09-08T16:11:28.941 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:28.941 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.941 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.941 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:28.941 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.777340+0000", 2020-09-08T16:11:28.941 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999623 2020-09-08T16:11:28.942 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.942 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.942 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:28.942 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.777303+0000", 2020-09-08T16:11:28.942 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00016701199999999999 2020-09-08T16:11:28.943 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.943 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.943 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:28.944 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.777470+0000", 2020-09-08T16:11:28.944 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.8277000000000001e-05 2020-09-08T16:11:28.944 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.944 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.944 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:28.945 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.777488+0000", 2020-09-08T16:11:28.945 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.6590000000000004e-06 2020-09-08T16:11:28.945 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.945 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.945 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:28.946 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.777494+0000", 2020-09-08T16:11:28.946 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.4224e-05 2020-09-08T16:11:28.946 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.946 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.946 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:28.947 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.777558+0000", 2020-09-08T16:11:28.947 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017187699999999999 2020-09-08T16:11:28.947 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.947 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.947 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:28.947 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.777730+0000", 2020-09-08T16:11:28.948 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017922899999999999 2020-09-08T16:11:28.948 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.948 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.948 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:28.948 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.777909+0000", 2020-09-08T16:11:28.949 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000840535 2020-09-08T16:11:28.949 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.949 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.949 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:28.949 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.778749+0000", 2020-09-08T16:11:28.950 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16363662700000001 2020-09-08T16:11:28.950 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.950 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.950 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:28.950 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.942386+0000", 2020-09-08T16:11:28.950 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.3257000000000005e-05 2020-09-08T16:11:28.951 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.951 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.951 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:28.951 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.942479+0000", 2020-09-08T16:11:28.951 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.2818999999999995e-05 2020-09-08T16:11:28.952 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.952 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.952 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:28.952 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:09:36.942552+0000", 2020-09-08T16:11:28.952 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.165211893 2020-09-08T16:11:28.953 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.953 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.953 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.953 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.954 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.954 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:144671 9.5 9:a4b51b61:::benchmark_data_smithi166_56754_object4638:head [delete] snapc 0=[] ondisk+write+known_if_redirected e734)", 2020-09-08T16:11:28.954 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:11:07.551294+0000", 2020-09-08T16:11:28.954 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 21.024454558999999, 2020-09-08T16:11:28.955 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19339689299999999, 2020-09-08T16:11:28.955 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:28.955 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:28.955 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:28.955 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:11:28.955 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:11:28.956 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 144671 2020-09-08T16:11:28.956 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.956 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:28.956 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.956 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:28.957 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.551294+0000", 2020-09-08T16:11:28.957 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:28.957 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.957 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.957 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:28.958 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.551294+0000", 2020-09-08T16:11:28.958 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999628 2020-09-08T16:11:28.958 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.958 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.958 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:28.959 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.551257+0000", 2020-09-08T16:11:28.959 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.3072999999999996e-05 2020-09-08T16:11:28.959 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.959 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.959 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:28.959 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.551340+0000", 2020-09-08T16:11:28.960 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.4378e-05 2020-09-08T16:11:28.960 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.960 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.960 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:28.960 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.551354+0000", 2020-09-08T16:11:28.961 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.2179999999999996e-06 2020-09-08T16:11:28.961 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.961 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.961 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:28.961 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.551360+0000", 2020-09-08T16:11:28.962 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.0832999999999999e-05 2020-09-08T16:11:28.962 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.962 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.962 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:28.962 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.551411+0000", 2020-09-08T16:11:28.963 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000152218 2020-09-08T16:11:28.963 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.963 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.963 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:28.963 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.551563+0000", 2020-09-08T16:11:28.964 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014068199999999999 2020-09-08T16:11:28.964 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.964 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.964 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:28.965 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.551704+0000", 2020-09-08T16:11:28.965 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.003685079 2020-09-08T16:11:28.965 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.965 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.965 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:28.966 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.555389+0000", 2020-09-08T16:11:28.966 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.18893385200000001 2020-09-08T16:11:28.966 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.966 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.966 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:28.967 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.744323+0000", 2020-09-08T16:11:28.967 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00022050699999999999 2020-09-08T16:11:28.967 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.967 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.967 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:28.968 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.744544+0000", 2020-09-08T16:11:28.968 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00014707500000000001 2020-09-08T16:11:28.968 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.968 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.968 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:28.968 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.744691+0000", 2020-09-08T16:11:28.969 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.19339689299999999 2020-09-08T16:11:28.969 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.969 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.969 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.970 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.970 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.970 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "osd_op(client.26587.0:144685 9.5 9:a4931cc5:::benchmark_data_smithi166_56754_object4652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e734)", 2020-09-08T16:11:28.970 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:11:07.581436+0000", 2020-09-08T16:11:28.970 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 20.994312007000001, 2020-09-08T16:11:28.971 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16370211200000001, 2020-09-08T16:11:28.971 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:11:28.971 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:11:28.971 INFO:teuthology.orchestra.run.smithi166.stdout: "client_info": { 2020-09-08T16:11:28.971 INFO:teuthology.orchestra.run.smithi166.stdout: "client": "client.26587", 2020-09-08T16:11:28.971 INFO:teuthology.orchestra.run.smithi166.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:11:28.972 INFO:teuthology.orchestra.run.smithi166.stdout: "tid": 144685 2020-09-08T16:11:28.972 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.972 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:11:28.972 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.972 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:11:28.973 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.581436+0000", 2020-09-08T16:11:28.973 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:11:28.973 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.973 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.973 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:11:28.974 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.581436+0000", 2020-09-08T16:11:28.974 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4294967295.9999652 2020-09-08T16:11:28.975 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.975 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.975 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:11:28.975 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.581402+0000", 2020-09-08T16:11:28.975 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.9170000000000003e-05 2020-09-08T16:11:28.976 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.976 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.976 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:11:28.976 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.581481+0000", 2020-09-08T16:11:28.976 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7102999999999999e-05 2020-09-08T16:11:28.977 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.977 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.977 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:11:28.977 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.581498+0000", 2020-09-08T16:11:28.977 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.7680000000000003e-06 2020-09-08T16:11:28.978 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.978 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.978 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:11:28.978 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.581508+0000", 2020-09-08T16:11:28.978 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7842000000000005e-05 2020-09-08T16:11:28.979 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.979 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.979 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:11:28.979 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.581586+0000", 2020-09-08T16:11:28.979 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000168927 2020-09-08T16:11:28.980 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.980 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.980 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:11:28.980 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.581754+0000", 2020-09-08T16:11:28.980 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0001495 2020-09-08T16:11:28.981 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.981 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.981 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "waiting for subops from 5", 2020-09-08T16:11:28.981 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.581904+0000", 2020-09-08T16:11:28.981 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.0045643039999999999 2020-09-08T16:11:28.982 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.982 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.982 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "op_commit", 2020-09-08T16:11:28.982 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.586468+0000", 2020-09-08T16:11:28.982 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.15833683000000001 2020-09-08T16:11:28.983 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.983 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.983 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:11:28.983 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.744805+0000", 2020-09-08T16:11:28.983 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00011482200000000001 2020-09-08T16:11:28.984 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.984 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.984 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "commit_sent", 2020-09-08T16:11:28.984 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.744920+0000", 2020-09-08T16:11:28.984 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00021853700000000001 2020-09-08T16:11:28.985 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:11:28.985 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:11:28.985 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:11:28.985 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:11:07.745138+0000", 2020-09-08T16:11:28.985 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.16370211200000001 2020-09-08T16:11:28.985 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.986 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.986 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.986 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:11:28.986 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:11:28.986 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:11:28.987 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:11:28.988 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:11:28 smithi055 bash[16646]: debug 2020-09-08T16:11:28.585+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:29.278 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:29 smithi055 bash[9296]: cluster 2020-09-08T16:11:28.142084+0000 mgr.x (mgr.24118) 1324 : cluster [DBG] pgmap v1978: 17 pgs: 17 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T16:11:29.384 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:11:29.488 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:29.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:29 smithi166 bash[9446]: cluster 2020-09-08T16:11:28.142084+0000 mgr.x (mgr.24118) 1324 : cluster [DBG] pgmap v1978: 17 pgs: 17 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T16:11:29.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:11:29 smithi055 bash[16646]: debug 2020-09-08T16:11:29.385+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:29.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:29 smithi104 bash[9203]: cluster 2020-09-08T16:11:28.142084+0000 mgr.x (mgr.24118) 1324 : cluster [DBG] pgmap v1978: 17 pgs: 17 active+clean; 7.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.01k op/s 2020-09-08T16:11:30.124 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:30.191 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2020-09-08T16:11:30.228 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:30.351 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:11:30.618 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:11:30 smithi166 bash[11928]: debug 2020-09-08T16:11:30.123+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:31.090 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T16:11:31.354 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:31 smithi055 bash[9296]: cluster 2020-09-08T16:11:30.142590+0000 mgr.x (mgr.24118) 1325 : cluster [DBG] pgmap v1979: 17 pgs: 17 active+clean; 7.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T16:11:31.354 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:31 smithi055 bash[9296]: cluster 2020-09-08T16:11:30.411442+0000 osd.5 (osd.5) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:31.354 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:31 smithi055 bash[9296]: cluster 2020-09-08T16:11:30.411531+0000 osd.5 (osd.5) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:31.355 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:31 smithi055 bash[9296]: audit 2020-09-08T16:11:30.839412+0000 mon.b (mon.1) 122 : audit [INF] from='client.? v1:172.21.15.104:0/1788397923' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:11:31.355 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:31 smithi055 bash[9296]: audit 2020-09-08T16:11:30.840609+0000 mon.a (mon.0) 2297 : audit [INF] from='client.26701 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:11:31.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:31 smithi166 bash[9446]: cluster 2020-09-08T16:11:30.142590+0000 mgr.x (mgr.24118) 1325 : cluster [DBG] pgmap v1979: 17 pgs: 17 active+clean; 7.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T16:11:31.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:31 smithi166 bash[9446]: cluster 2020-09-08T16:11:30.411442+0000 osd.5 (osd.5) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:31.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:31 smithi166 bash[9446]: cluster 2020-09-08T16:11:30.411531+0000 osd.5 (osd.5) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:31.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:31 smithi166 bash[9446]: audit 2020-09-08T16:11:30.839412+0000 mon.b (mon.1) 122 : audit [INF] from='client.? v1:172.21.15.104:0/1788397923' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:11:31.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:31 smithi166 bash[9446]: audit 2020-09-08T16:11:30.840609+0000 mon.a (mon.0) 2297 : audit [INF] from='client.26701 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:11:31.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:31 smithi104 bash[9203]: cluster 2020-09-08T16:11:30.142590+0000 mgr.x (mgr.24118) 1325 : cluster [DBG] pgmap v1979: 17 pgs: 17 active+clean; 7.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T16:11:31.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:31 smithi104 bash[9203]: cluster 2020-09-08T16:11:30.411442+0000 osd.5 (osd.5) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:31.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:31 smithi104 bash[9203]: cluster 2020-09-08T16:11:30.411531+0000 osd.5 (osd.5) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:31.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:31 smithi104 bash[9203]: audit 2020-09-08T16:11:30.839412+0000 mon.b (mon.1) 122 : audit [INF] from='client.? v1:172.21.15.104:0/1788397923' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:11:31.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:31 smithi104 bash[9203]: audit 2020-09-08T16:11:30.840609+0000 mon.a (mon.0) 2297 : audit [INF] from='client.26701 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:11:32.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:32 smithi166 bash[9446]: cluster 2020-09-08T16:11:31.079861+0000 mon.a (mon.0) 2298 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:11:32.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:32 smithi166 bash[9446]: audit 2020-09-08T16:11:31.089048+0000 mon.a (mon.0) 2299 : audit [INF] from='client.26701 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:11:32.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:32 smithi166 bash[9446]: cluster 2020-09-08T16:11:31.089105+0000 mon.a (mon.0) 2300 : cluster [DBG] osdmap e739: 12 total, 12 up, 12 in 2020-09-08T16:11:32.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:32 smithi166 bash[9446]: audit 2020-09-08T16:11:31.089393+0000 mon.a (mon.0) 2301 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:11:32.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:32 smithi166 bash[9446]: audit 2020-09-08T16:11:31.093493+0000 mon.a (mon.0) 2302 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:11:32.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:32 smithi055 bash[9296]: cluster 2020-09-08T16:11:31.079861+0000 mon.a (mon.0) 2298 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:11:32.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:32 smithi055 bash[9296]: audit 2020-09-08T16:11:31.089048+0000 mon.a (mon.0) 2299 : audit [INF] from='client.26701 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:11:32.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:32 smithi055 bash[9296]: cluster 2020-09-08T16:11:31.089105+0000 mon.a (mon.0) 2300 : cluster [DBG] osdmap e739: 12 total, 12 up, 12 in 2020-09-08T16:11:32.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:32 smithi055 bash[9296]: audit 2020-09-08T16:11:31.089393+0000 mon.a (mon.0) 2301 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:11:32.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:32 smithi055 bash[9296]: audit 2020-09-08T16:11:31.093493+0000 mon.a (mon.0) 2302 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:11:32.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:32 smithi104 bash[9203]: cluster 2020-09-08T16:11:31.079861+0000 mon.a (mon.0) 2298 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:11:32.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:32 smithi104 bash[9203]: audit 2020-09-08T16:11:31.089048+0000 mon.a (mon.0) 2299 : audit [INF] from='client.26701 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:11:32.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:32 smithi104 bash[9203]: cluster 2020-09-08T16:11:31.089105+0000 mon.a (mon.0) 2300 : cluster [DBG] osdmap e739: 12 total, 12 up, 12 in 2020-09-08T16:11:32.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:32 smithi104 bash[9203]: audit 2020-09-08T16:11:31.089393+0000 mon.a (mon.0) 2301 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:11:32.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:32 smithi104 bash[9203]: audit 2020-09-08T16:11:31.093493+0000 mon.a (mon.0) 2302 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:11:33.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:33 smithi166 bash[9446]: cluster 2020-09-08T16:11:32.143062+0000 mgr.x (mgr.24118) 1326 : cluster [DBG] pgmap v1981: 17 pgs: 17 active+clean; 7.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T16:11:33.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:33 smithi166 bash[9446]: cluster 2020-09-08T16:11:32.833972+0000 osd.10 (osd.10) 111 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:33.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:33 smithi166 bash[9446]: cluster 2020-09-08T16:11:32.834136+0000 osd.10 (osd.10) 112 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:33.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:33 smithi055 bash[9296]: cluster 2020-09-08T16:11:32.143062+0000 mgr.x (mgr.24118) 1326 : cluster [DBG] pgmap v1981: 17 pgs: 17 active+clean; 7.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T16:11:33.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:33 smithi055 bash[9296]: cluster 2020-09-08T16:11:32.833972+0000 osd.10 (osd.10) 111 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:33.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:33 smithi055 bash[9296]: cluster 2020-09-08T16:11:32.834136+0000 osd.10 (osd.10) 112 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:33.466 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:33 smithi104 bash[9203]: cluster 2020-09-08T16:11:32.143062+0000 mgr.x (mgr.24118) 1326 : cluster [DBG] pgmap v1981: 17 pgs: 17 active+clean; 7.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.02k op/s 2020-09-08T16:11:33.467 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:33 smithi104 bash[9203]: cluster 2020-09-08T16:11:32.833972+0000 osd.10 (osd.10) 111 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:33.467 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:33 smithi104 bash[9203]: cluster 2020-09-08T16:11:32.834136+0000 osd.10 (osd.10) 112 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:33.736 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:33.858 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:11:34.014 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:11:34.095 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:11:33 smithi055 bash[14301]: debug 2020-09-08T16:11:33.737+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:34.095 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:34 smithi055 bash[9296]: cluster 2020-09-08T16:11:33.095502+0000 mon.a (mon.0) 2303 : cluster [DBG] osdmap e740: 12 total, 12 up, 12 in 2020-09-08T16:11:34.119 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:34.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:34 smithi104 bash[9203]: cluster 2020-09-08T16:11:33.095502+0000 mon.a (mon.0) 2303 : cluster [DBG] osdmap e740: 12 total, 12 up, 12 in 2020-09-08T16:11:34.176 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:11:34 smithi104 bash[22305]: debug 2020-09-08T16:11:34.004+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:34.290 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:34.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:34 smithi166 bash[9446]: cluster 2020-09-08T16:11:33.095502+0000 mon.a (mon.0) 2303 : cluster [DBG] osdmap e740: 12 total, 12 up, 12 in 2020-09-08T16:11:34.367 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:11:34 smithi166 bash[16593]: debug 2020-09-08T16:11:34.290+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:34.393 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:11:34.518 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:11:34.627 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:11:34.758 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:11:34.781 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:11:34.809 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:11:34.846 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:11:34.874 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:11:34.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:11:34 smithi055 bash[11956]: debug 2020-09-08T16:11:34.513+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:34.908 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:11:34.939 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:11:34.971 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:11:35.002 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:11:35.030 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:11:35.061 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:11:35.093 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:11:35.095 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:11:35.096 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:11:35.097 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:11:35.097 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:11:35.097 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:11:35.098 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:11:35.098 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:11:35.098 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:11:35.098 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:11:35.098 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:11:35.098 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:11:35.099 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:11:35.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:35 smithi166 bash[9446]: cluster 2020-09-08T16:11:34.143458+0000 mgr.x (mgr.24118) 1327 : cluster [DBG] pgmap v1983: 17 pgs: 17 active+clean; 6.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T16:11:35.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:35 smithi166 bash[9446]: audit 2020-09-08T16:11:34.730317+0000 mon.a (mon.0) 2304 : audit [DBG] from='client.16812 v1:172.21.15.104:0/3391313017' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:11:35.277 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T16:11:35.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:35 smithi055 bash[9296]: cluster 2020-09-08T16:11:34.143458+0000 mgr.x (mgr.24118) 1327 : cluster [DBG] pgmap v1983: 17 pgs: 17 active+clean; 6.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T16:11:35.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:35 smithi055 bash[9296]: audit 2020-09-08T16:11:34.730317+0000 mon.a (mon.0) 2304 : audit [DBG] from='client.16812 v1:172.21.15.104:0/3391313017' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:11:35.434 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:11:35.445 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:35 smithi104 bash[9203]: cluster 2020-09-08T16:11:34.143458+0000 mgr.x (mgr.24118) 1327 : cluster [DBG] pgmap v1983: 17 pgs: 17 active+clean; 6.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.19k op/s 2020-09-08T16:11:35.445 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:35 smithi104 bash[9203]: audit 2020-09-08T16:11:34.730317+0000 mon.a (mon.0) 2304 : audit [DBG] from='client.16812 v1:172.21.15.104:0/3391313017' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:11:36.014 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:11:36.119 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:11:36.255 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:11:36.360 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:11:36.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:36 smithi166 bash[9446]: cluster 2020-09-08T16:11:34.781831+0000 osd.3 (osd.3) 131 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:36.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:36 smithi166 bash[9446]: cluster 2020-09-08T16:11:34.781921+0000 osd.3 (osd.3) 132 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:36.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:36 smithi166 bash[9446]: cluster 2020-09-08T16:11:35.115735+0000 mon.a (mon.0) 2305 : cluster [DBG] osdmap e741: 12 total, 12 up, 12 in 2020-09-08T16:11:36.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:11:36 smithi166 bash[18960]: debug 2020-09-08T16:11:36.014+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:36.369 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:36 smithi104 bash[9203]: cluster 2020-09-08T16:11:34.781831+0000 osd.3 (osd.3) 131 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:36.369 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:36 smithi104 bash[9203]: cluster 2020-09-08T16:11:34.781921+0000 osd.3 (osd.3) 132 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:36.369 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:36 smithi104 bash[9203]: cluster 2020-09-08T16:11:35.115735+0000 mon.a (mon.0) 2305 : cluster [DBG] osdmap e741: 12 total, 12 up, 12 in 2020-09-08T16:11:36.410 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:11:36 smithi055 bash[11956]: debug 2020-09-08T16:11:36.253+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:36.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:36 smithi055 bash[9296]: cluster 2020-09-08T16:11:34.781831+0000 osd.3 (osd.3) 131 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:36.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:36 smithi055 bash[9296]: cluster 2020-09-08T16:11:34.781921+0000 osd.3 (osd.3) 132 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:36.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:36 smithi055 bash[9296]: cluster 2020-09-08T16:11:35.115735+0000 mon.a (mon.0) 2305 : cluster [DBG] osdmap e741: 12 total, 12 up, 12 in 2020-09-08T16:11:36.990 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:11:37.094 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:37.230 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:37.334 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:11:37.344 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:11:36 smithi104 bash[15273]: debug 2020-09-08T16:11:36.988+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:37.344 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:37 smithi104 bash[9203]: cluster 2020-09-08T16:11:36.143824+0000 mgr.x (mgr.24118) 1328 : cluster [DBG] pgmap v1985: 17 pgs: 17 active+clean; 6.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 792 op/s 2020-09-08T16:11:37.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:37 smithi166 bash[9446]: cluster 2020-09-08T16:11:36.143824+0000 mgr.x (mgr.24118) 1328 : cluster [DBG] pgmap v1985: 17 pgs: 17 active+clean; 6.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 792 op/s 2020-09-08T16:11:37.367 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:11:37 smithi166 bash[16593]: debug 2020-09-08T16:11:37.230+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:37.417 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:37 smithi055 bash[9296]: cluster 2020-09-08T16:11:36.143824+0000 mgr.x (mgr.24118) 1328 : cluster [DBG] pgmap v1985: 17 pgs: 17 active+clean; 6.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 792 op/s 2020-09-08T16:11:37.513 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:11:37.620 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:37.675 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:11:37 smithi104 bash[15273]: debug 2020-09-08T16:11:37.504+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:37.760 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:37.864 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:37.988 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:11:37 smithi166 bash[16593]: debug 2020-09-08T16:11:37.758+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:37.989 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:11:37 smithi166 bash[16593]: debug 2020-09-08T16:11:37.986+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:37.989 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:38.093 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:11:38.105 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T16:11:38.105 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:11:38.211 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:11:38.315 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:38.408 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:11:38 smithi055 bash[19003]: debug 2020-09-08T16:11:38.213+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:38.444 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:38.550 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:38.619 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:11:38 smithi166 bash[11928]: debug 2020-09-08T16:11:38.438+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:38.655 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:38.761 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:38.879 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:38.982 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:38.990 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:11:38 smithi166 bash[14254]: debug 2020-09-08T16:11:38.878+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:38.995 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:11:38 smithi166 bash[11928]: debug 2020-09-08T16:11:38.654+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:39.065 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:11:39.103 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:39.206 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:39.217 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:11:39.349 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:39.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:39 smithi166 bash[9446]: cluster 2020-09-08T16:11:38.144217+0000 mgr.x (mgr.24118) 1329 : cluster [DBG] pgmap v1986: 17 pgs: 17 active+clean; 6.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.25k op/s 2020-09-08T16:11:39.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:11:39 smithi166 bash[14254]: debug 2020-09-08T16:11:39.102+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:39.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:39 smithi104 bash[9203]: cluster 2020-09-08T16:11:38.144217+0000 mgr.x (mgr.24118) 1329 : cluster [DBG] pgmap v1986: 17 pgs: 17 active+clean; 6.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.25k op/s 2020-09-08T16:11:39.453 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:39.587 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:39.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:39 smithi055 bash[9296]: cluster 2020-09-08T16:11:38.144217+0000 mgr.x (mgr.24118) 1329 : cluster [DBG] pgmap v1986: 17 pgs: 17 active+clean; 6.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.25k op/s 2020-09-08T16:11:39.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:11:39 smithi055 bash[14301]: debug 2020-09-08T16:11:39.349+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:39.690 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:39.806 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:39.807 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:11:39 smithi166 bash[11928]: debug 2020-09-08T16:11:39.586+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:39.916 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:11:40.032 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:11:40.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:11:39 smithi166 bash[16593]: debug 2020-09-08T16:11:39.806+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:40.138 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:40.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:11:40 smithi055 bash[11956]: debug 2020-09-08T16:11:40.033+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:40.789 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2020-09-08T16:11:40.821 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:40.925 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:40.939 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:11:41.073 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:41.096 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:11:40 smithi104 bash[19922]: debug 2020-09-08T16:11:40.808+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:41.178 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:11:41.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:41 smithi166 bash[9446]: cluster 2020-09-08T16:11:40.144676+0000 mgr.x (mgr.24118) 1330 : cluster [DBG] pgmap v1987: 17 pgs: 17 active+clean; 6.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.23k op/s 2020-09-08T16:11:41.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:41 smithi055 bash[9296]: cluster 2020-09-08T16:11:40.144676+0000 mgr.x (mgr.24118) 1330 : cluster [DBG] pgmap v1987: 17 pgs: 17 active+clean; 6.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.23k op/s 2020-09-08T16:11:41.408 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:11:41 smithi055 bash[14301]: debug 2020-09-08T16:11:41.077+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:41.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:41 smithi104 bash[9203]: cluster 2020-09-08T16:11:40.144676+0000 mgr.x (mgr.24118) 1330 : cluster [DBG] pgmap v1987: 17 pgs: 17 active+clean; 6.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.23k op/s 2020-09-08T16:11:42.049 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:11:42.153 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:11:42.321 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:42 smithi104 bash[9203]: cluster 2020-09-08T16:11:41.235845+0000 osd.8 (osd.8) 108 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:42.322 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:42 smithi104 bash[9203]: cluster 2020-09-08T16:11:41.235926+0000 osd.8 (osd.8) 109 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:42.323 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:11:42 smithi104 bash[15273]: debug 2020-09-08T16:11:42.048+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:42.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:42 smithi166 bash[9446]: cluster 2020-09-08T16:11:41.235845+0000 osd.8 (osd.8) 108 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:42.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:42 smithi166 bash[9446]: cluster 2020-09-08T16:11:41.235926+0000 osd.8 (osd.8) 109 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:42.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:42 smithi055 bash[9296]: cluster 2020-09-08T16:11:41.235845+0000 osd.8 (osd.8) 108 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:42.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:42 smithi055 bash[9296]: cluster 2020-09-08T16:11:41.235926+0000 osd.8 (osd.8) 109 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:43.024 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:11:43.127 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:43.285 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:43.370 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:11:43 smithi166 bash[16593]: debug 2020-09-08T16:11:43.282+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:43.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:43 smithi166 bash[9446]: cluster 2020-09-08T16:11:42.115606+0000 mon.a (mon.0) 2306 : cluster [DBG] osdmap e742: 12 total, 12 up, 12 in 2020-09-08T16:11:43.371 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:43 smithi166 bash[9446]: cluster 2020-09-08T16:11:42.145156+0000 mgr.x (mgr.24118) 1331 : cluster [DBG] pgmap v1989: 17 pgs: 17 active+clean; 6.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 700 op/s 2020-09-08T16:11:43.392 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:43.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:43 smithi055 bash[9296]: cluster 2020-09-08T16:11:42.115606+0000 mon.a (mon.0) 2306 : cluster [DBG] osdmap e742: 12 total, 12 up, 12 in 2020-09-08T16:11:43.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:43 smithi055 bash[9296]: cluster 2020-09-08T16:11:42.145156+0000 mgr.x (mgr.24118) 1331 : cluster [DBG] pgmap v1989: 17 pgs: 17 active+clean; 6.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 700 op/s 2020-09-08T16:11:43.418 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:11:43 smithi104 bash[17588]: debug 2020-09-08T16:11:43.020+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:43.419 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:43 smithi104 bash[9203]: cluster 2020-09-08T16:11:42.115606+0000 mon.a (mon.0) 2306 : cluster [DBG] osdmap e742: 12 total, 12 up, 12 in 2020-09-08T16:11:43.419 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:43 smithi104 bash[9203]: cluster 2020-09-08T16:11:42.145156+0000 mgr.x (mgr.24118) 1331 : cluster [DBG] pgmap v1989: 17 pgs: 17 active+clean; 6.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 700 op/s 2020-09-08T16:11:43.520 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:43.624 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:43.743 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:11:43 smithi166 bash[11928]: debug 2020-09-08T16:11:43.514+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:43.744 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:43.851 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:43.992 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:43.995 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:11:43 smithi166 bash[14254]: debug 2020-09-08T16:11:43.990+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:43.995 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:11:43 smithi166 bash[16593]: debug 2020-09-08T16:11:43.742+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:44.095 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:44.188 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T16:11:44.217 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:44.278 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:44 smithi104 bash[9203]: audit 2020-09-08T16:11:44.137658+0000 mon.a (mon.0) 2307 : audit [INF] from='client.? v1:172.21.15.104:0/3282671447' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:11:44.278 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:44 smithi104 bash[9203]: cluster 2020-09-08T16:11:44.188086+0000 mon.a (mon.0) 2308 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:11:44.323 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:11:44.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:44 smithi166 bash[9446]: audit 2020-09-08T16:11:44.137658+0000 mon.a (mon.0) 2307 : audit [INF] from='client.? v1:172.21.15.104:0/3282671447' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:11:44.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:44 smithi166 bash[9446]: cluster 2020-09-08T16:11:44.188086+0000 mon.a (mon.0) 2308 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:11:44.367 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:11:44 smithi166 bash[16593]: debug 2020-09-08T16:11:44.218+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:44.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:44 smithi055 bash[9296]: audit 2020-09-08T16:11:44.137658+0000 mon.a (mon.0) 2307 : audit [INF] from='client.? v1:172.21.15.104:0/3282671447' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:11:44.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:44 smithi055 bash[9296]: cluster 2020-09-08T16:11:44.188086+0000 mon.a (mon.0) 2308 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:11:44.822 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:11:44.927 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:45.064 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:45.170 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:45.176 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:11:44 smithi104 bash[17588]: debug 2020-09-08T16:11:44.816+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:45.195 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:11:45 smithi166 bash[14254]: debug 2020-09-08T16:11:45.062+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:45.473 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:45 smithi055 bash[9296]: cluster 2020-09-08T16:11:44.145728+0000 mgr.x (mgr.24118) 1332 : cluster [DBG] pgmap v1990: 17 pgs: 17 active+clean; 6.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T16:11:45.473 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:45 smithi055 bash[9296]: audit 2020-09-08T16:11:44.191518+0000 mon.a (mon.0) 2309 : audit [INF] from='client.? v1:172.21.15.104:0/3282671447' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:11:45.473 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:45 smithi055 bash[9296]: cluster 2020-09-08T16:11:44.191576+0000 mon.a (mon.0) 2310 : cluster [DBG] osdmap e743: 12 total, 12 up, 12 in 2020-09-08T16:11:45.474 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:45 smithi055 bash[9296]: cluster 2020-09-08T16:11:44.307067+0000 osd.7 (osd.7) 163 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:11:45.474 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:45 smithi055 bash[9296]: cluster 2020-09-08T16:11:44.452398+0000 osd.7 (osd.7) 164 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:11:45.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:45 smithi166 bash[9446]: cluster 2020-09-08T16:11:44.145728+0000 mgr.x (mgr.24118) 1332 : cluster [DBG] pgmap v1990: 17 pgs: 17 active+clean; 6.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T16:11:45.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:45 smithi166 bash[9446]: audit 2020-09-08T16:11:44.191518+0000 mon.a (mon.0) 2309 : audit [INF] from='client.? v1:172.21.15.104:0/3282671447' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:11:45.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:45 smithi166 bash[9446]: cluster 2020-09-08T16:11:44.191576+0000 mon.a (mon.0) 2310 : cluster [DBG] osdmap e743: 12 total, 12 up, 12 in 2020-09-08T16:11:45.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:45 smithi166 bash[9446]: cluster 2020-09-08T16:11:44.307067+0000 osd.7 (osd.7) 163 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:11:45.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:45 smithi166 bash[9446]: cluster 2020-09-08T16:11:44.452398+0000 osd.7 (osd.7) 164 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:11:45.640 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:45 smithi104 bash[9203]: cluster 2020-09-08T16:11:44.145728+0000 mgr.x (mgr.24118) 1332 : cluster [DBG] pgmap v1990: 17 pgs: 17 active+clean; 6.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T16:11:45.640 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:45 smithi104 bash[9203]: audit 2020-09-08T16:11:44.191518+0000 mon.a (mon.0) 2309 : audit [INF] from='client.? v1:172.21.15.104:0/3282671447' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:11:45.641 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:45 smithi104 bash[9203]: cluster 2020-09-08T16:11:44.191576+0000 mon.a (mon.0) 2310 : cluster [DBG] osdmap e743: 12 total, 12 up, 12 in 2020-09-08T16:11:45.641 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:45 smithi104 bash[9203]: cluster 2020-09-08T16:11:44.307067+0000 osd.7 (osd.7) 163 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:11:45.641 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:45 smithi104 bash[9203]: cluster 2020-09-08T16:11:44.452398+0000 osd.7 (osd.7) 164 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:11:45.642 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:45.747 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:11:45.925 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:11:45 smithi104 bash[19922]: debug 2020-09-08T16:11:45.640+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:46.302 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:11:46.410 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:46.547 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:46.651 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:46.675 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:11:46 smithi104 bash[15273]: debug 2020-09-08T16:11:46.296+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:46.800 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:46.906 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:11:46.908 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:11:46 smithi055 bash[14301]: debug 2020-09-08T16:11:46.548+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:46.915 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:11:46 smithi166 bash[16593]: debug 2020-09-08T16:11:46.798+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:47.025 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:11:47.129 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:47.196 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:11:47 smithi166 bash[18960]: debug 2020-09-08T16:11:47.022+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:47.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:47 smithi104 bash[9203]: cluster 2020-09-08T16:11:46.146215+0000 mgr.x (mgr.24118) 1333 : cluster [DBG] pgmap v1992: 17 pgs: 17 active+clean; 6.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 754 op/s 2020-09-08T16:11:47.451 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:47.555 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:47.564 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:47 smithi166 bash[9446]: cluster 2020-09-08T16:11:46.146215+0000 mgr.x (mgr.24118) 1333 : cluster [DBG] pgmap v1992: 17 pgs: 17 active+clean; 6.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 754 op/s 2020-09-08T16:11:47.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:47 smithi055 bash[9296]: cluster 2020-09-08T16:11:46.146215+0000 mgr.x (mgr.24118) 1333 : cluster [DBG] pgmap v1992: 17 pgs: 17 active+clean; 6.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 754 op/s 2020-09-08T16:11:47.668 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:11:47.772 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:11:47.781 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:11:47 smithi104 bash[19922]: debug 2020-09-08T16:11:47.452+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:47.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:11:47 smithi166 bash[16593]: debug 2020-09-08T16:11:47.666+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:48.208 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:11:48.313 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:48.319 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T16:11:48.434 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:48.481 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:11:48.538 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:48.553 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:11:48 smithi104 bash[22305]: debug 2020-09-08T16:11:48.208+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:48.616 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:11:48 smithi166 bash[14254]: debug 2020-09-08T16:11:48.430+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:49.221 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:49.325 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:49.403 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:11:49.454 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:11:49.471 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:11:49.497 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:11:49.527 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:11:49.566 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:11:49.589 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:11:49.599 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:49 smithi104 bash[9203]: cluster 2020-09-08T16:11:48.146711+0000 mgr.x (mgr.24118) 1334 : cluster [DBG] pgmap v1993: 17 pgs: 17 active+clean; 5.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T16:11:49.600 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:49 smithi104 bash[9203]: cluster 2020-09-08T16:11:48.953154+0000 osd.9 (osd.9) 116 : cluster [DBG] 9.8 scrub starts 2020-09-08T16:11:49.600 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:11:49 smithi104 bash[19922]: debug 2020-09-08T16:11:49.220+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:49.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:49 smithi166 bash[9446]: cluster 2020-09-08T16:11:48.146711+0000 mgr.x (mgr.24118) 1334 : cluster [DBG] pgmap v1993: 17 pgs: 17 active+clean; 5.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T16:11:49.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:49 smithi166 bash[9446]: cluster 2020-09-08T16:11:48.953154+0000 osd.9 (osd.9) 116 : cluster [DBG] 9.8 scrub starts 2020-09-08T16:11:49.621 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:11:49.653 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:11:49.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:49 smithi055 bash[9296]: cluster 2020-09-08T16:11:48.146711+0000 mgr.x (mgr.24118) 1334 : cluster [DBG] pgmap v1993: 17 pgs: 17 active+clean; 5.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T16:11:49.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:49 smithi055 bash[9296]: cluster 2020-09-08T16:11:48.953154+0000 osd.9 (osd.9) 116 : cluster [DBG] 9.8 scrub starts 2020-09-08T16:11:49.662 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:49.698 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:11:49.727 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:11:49.764 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:11:49.766 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:49.768 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:11:49.768 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:11:49.769 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:11:49.769 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:11:49.769 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:11:49.771 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:11:49.772 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:11:49.772 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:11:49.772 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:11:49.772 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:11:49.773 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:11:49.773 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:11:49.885 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T16:11:49.886 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T16:11:49.886 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2020-09-08T16:11:49.886 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2020-09-08T16:11:49.886 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T16:11:49.890 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:11:49 smithi104 bash[19922]: debug 2020-09-08T16:11:49.660+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:49.904 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:50.010 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:11:50.158 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:11:49 smithi055 bash[14301]: debug 2020-09-08T16:11:49.904+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:50.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:50 smithi166 bash[9446]: cluster 2020-09-08T16:11:48.537940+0000 osd.1 (osd.1) 152 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:50.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:50 smithi166 bash[9446]: cluster 2020-09-08T16:11:48.538029+0000 osd.1 (osd.1) 153 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:50.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:50 smithi166 bash[9446]: cluster 2020-09-08T16:11:49.207233+0000 mon.a (mon.0) 2311 : cluster [DBG] osdmap e744: 12 total, 12 up, 12 in 2020-09-08T16:11:50.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:50 smithi166 bash[9446]: audit 2020-09-08T16:11:49.374543+0000 mon.a (mon.0) 2312 : audit [DBG] from='client.? v1:172.21.15.104:0/2568329798' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:11:50.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:50 smithi055 bash[9296]: cluster 2020-09-08T16:11:48.537940+0000 osd.1 (osd.1) 152 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:50.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:50 smithi055 bash[9296]: cluster 2020-09-08T16:11:48.538029+0000 osd.1 (osd.1) 153 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:50.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:50 smithi055 bash[9296]: cluster 2020-09-08T16:11:49.207233+0000 mon.a (mon.0) 2311 : cluster [DBG] osdmap e744: 12 total, 12 up, 12 in 2020-09-08T16:11:50.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:50 smithi055 bash[9296]: audit 2020-09-08T16:11:49.374543+0000 mon.a (mon.0) 2312 : audit [DBG] from='client.? v1:172.21.15.104:0/2568329798' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:11:50.681 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:50 smithi104 bash[9203]: cluster 2020-09-08T16:11:48.537940+0000 osd.1 (osd.1) 152 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:50.682 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:50 smithi104 bash[9203]: cluster 2020-09-08T16:11:48.538029+0000 osd.1 (osd.1) 153 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:50.682 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:50 smithi104 bash[9203]: cluster 2020-09-08T16:11:49.207233+0000 mon.a (mon.0) 2311 : cluster [DBG] osdmap e744: 12 total, 12 up, 12 in 2020-09-08T16:11:50.682 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:50 smithi104 bash[9203]: audit 2020-09-08T16:11:49.374543+0000 mon.a (mon.0) 2312 : audit [DBG] from='client.? v1:172.21.15.104:0/2568329798' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:11:50.775 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:11:50.885 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:11:51.175 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:11:50 smithi104 bash[17588]: debug 2020-09-08T16:11:50.776+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:51.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:51 smithi166 bash[9446]: cluster 2020-09-08T16:11:50.147365+0000 mgr.x (mgr.24118) 1335 : cluster [DBG] pgmap v1995: 17 pgs: 17 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.25k op/s 2020-09-08T16:11:51.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:51 smithi166 bash[9446]: cluster 2020-09-08T16:11:50.281459+0000 osd.9 (osd.9) 117 : cluster [DBG] 9.8 scrub ok 2020-09-08T16:11:51.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:51 smithi055 bash[9296]: cluster 2020-09-08T16:11:50.147365+0000 mgr.x (mgr.24118) 1335 : cluster [DBG] pgmap v1995: 17 pgs: 17 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.25k op/s 2020-09-08T16:11:51.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:51 smithi055 bash[9296]: cluster 2020-09-08T16:11:50.281459+0000 osd.9 (osd.9) 117 : cluster [DBG] 9.8 scrub ok 2020-09-08T16:11:51.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:51 smithi104 bash[9203]: cluster 2020-09-08T16:11:50.147365+0000 mgr.x (mgr.24118) 1335 : cluster [DBG] pgmap v1995: 17 pgs: 17 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.25k op/s 2020-09-08T16:11:51.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:51 smithi104 bash[9203]: cluster 2020-09-08T16:11:50.281459+0000 osd.9 (osd.9) 117 : cluster [DBG] 9.8 scrub ok 2020-09-08T16:11:52.095 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:11:52.471 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:52 smithi104 bash[9203]: cluster 2020-09-08T16:11:51.004529+0000 osd.3 (osd.3) 133 : cluster [DBG] 9.2 scrub starts 2020-09-08T16:11:52.472 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:11:52 smithi104 bash[15273]: debug 2020-09-08T16:11:52.088+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:52.518 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2020-09-08T16:11:52.572 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:11:52.582 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:52 smithi055 bash[9296]: cluster 2020-09-08T16:11:51.004529+0000 osd.3 (osd.3) 133 : cluster [DBG] 9.2 scrub starts 2020-09-08T16:11:52.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:52 smithi166 bash[9446]: cluster 2020-09-08T16:11:51.004529+0000 osd.3 (osd.3) 133 : cluster [DBG] 9.2 scrub starts 2020-09-08T16:11:52.682 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:11:52.700 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:11:52.801 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:11:52.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:11:52 smithi055 bash[16646]: debug 2020-09-08T16:11:52.692+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:53.475 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:53 smithi104 bash[9203]: cluster 2020-09-08T16:11:52.147822+0000 mgr.x (mgr.24118) 1336 : cluster [DBG] pgmap v1996: 17 pgs: 17 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 694 op/s 2020-09-08T16:11:53.611 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:11:53.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:53 smithi166 bash[9446]: cluster 2020-09-08T16:11:52.147822+0000 mgr.x (mgr.24118) 1336 : cluster [DBG] pgmap v1996: 17 pgs: 17 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 694 op/s 2020-09-08T16:11:53.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:53 smithi055 bash[9296]: cluster 2020-09-08T16:11:52.147822+0000 mgr.x (mgr.24118) 1336 : cluster [DBG] pgmap v1996: 17 pgs: 17 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 694 op/s 2020-09-08T16:11:53.714 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:53.886 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:53.925 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:11:53 smithi104 bash[15273]: debug 2020-09-08T16:11:53.608+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:53.993 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:11:54.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:11:53 smithi166 bash[14254]: debug 2020-09-08T16:11:53.886+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:54.126 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:11:54.228 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:54.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:11:54 smithi055 bash[19003]: debug 2020-09-08T16:11:54.124+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:54.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:54 smithi055 bash[9296]: cluster 2020-09-08T16:11:52.969771+0000 osd.6 (osd.6) 121 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:54.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:54 smithi055 bash[9296]: cluster 2020-09-08T16:11:52.969870+0000 osd.6 (osd.6) 122 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:54.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:54 smithi055 bash[9296]: cluster 2020-09-08T16:11:53.049056+0000 osd.3 (osd.3) 134 : cluster [DBG] 9.2 scrub ok 2020-09-08T16:11:54.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:54 smithi055 bash[9296]: cluster 2020-09-08T16:11:53.229276+0000 mon.a (mon.0) 2313 : cluster [DBG] osdmap e745: 12 total, 12 up, 12 in 2020-09-08T16:11:54.473 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:11:54.509 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:54 smithi104 bash[9203]: cluster 2020-09-08T16:11:52.969771+0000 osd.6 (osd.6) 121 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:54.510 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:54 smithi104 bash[9203]: cluster 2020-09-08T16:11:52.969870+0000 osd.6 (osd.6) 122 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:54.510 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:54 smithi104 bash[9203]: cluster 2020-09-08T16:11:53.049056+0000 osd.3 (osd.3) 134 : cluster [DBG] 9.2 scrub ok 2020-09-08T16:11:54.510 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:54 smithi104 bash[9203]: cluster 2020-09-08T16:11:53.229276+0000 mon.a (mon.0) 2313 : cluster [DBG] osdmap e745: 12 total, 12 up, 12 in 2020-09-08T16:11:54.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:54 smithi166 bash[9446]: cluster 2020-09-08T16:11:52.969771+0000 osd.6 (osd.6) 121 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:54.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:54 smithi166 bash[9446]: cluster 2020-09-08T16:11:52.969870+0000 osd.6 (osd.6) 122 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:54.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:54 smithi166 bash[9446]: cluster 2020-09-08T16:11:53.049056+0000 osd.3 (osd.3) 134 : cluster [DBG] 9.2 scrub ok 2020-09-08T16:11:54.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:54 smithi166 bash[9446]: cluster 2020-09-08T16:11:53.229276+0000 mon.a (mon.0) 2313 : cluster [DBG] osdmap e745: 12 total, 12 up, 12 in 2020-09-08T16:11:54.632 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:11:54.670 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:11:54.780 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:54.925 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:54.982 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:11:54 smithi104 bash[19922]: debug 2020-09-08T16:11:54.668+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:55.032 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:55.043 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:11:54 smithi166 bash[14254]: debug 2020-09-08T16:11:54.922+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:55.161 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:55.264 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:55.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:55 smithi166 bash[9446]: cluster 2020-09-08T16:11:54.148212+0000 mgr.x (mgr.24118) 1337 : cluster [DBG] pgmap v1998: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T16:11:55.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:11:55 smithi166 bash[14254]: debug 2020-09-08T16:11:55.162+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:55.395 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:11:55.499 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:11:55.508 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:55 smithi055 bash[9296]: cluster 2020-09-08T16:11:54.148212+0000 mgr.x (mgr.24118) 1337 : cluster [DBG] pgmap v1998: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T16:11:55.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:55 smithi104 bash[9203]: cluster 2020-09-08T16:11:54.148212+0000 mgr.x (mgr.24118) 1337 : cluster [DBG] pgmap v1998: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T16:11:55.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:11:55 smithi166 bash[14254]: debug 2020-09-08T16:11:55.394+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:57.454 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:57 smithi104 bash[9203]: cluster 2020-09-08T16:11:56.148697+0000 mgr.x (mgr.24118) 1338 : cluster [DBG] pgmap v1999: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 645 op/s 2020-09-08T16:11:57.454 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:57 smithi104 bash[9203]: cluster 2020-09-08T16:11:56.818843+0000 osd.11 (osd.11) 134 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:57.455 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:57 smithi104 bash[9203]: cluster 2020-09-08T16:11:56.818913+0000 osd.11 (osd.11) 135 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:57.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:57 smithi166 bash[9446]: cluster 2020-09-08T16:11:56.148697+0000 mgr.x (mgr.24118) 1338 : cluster [DBG] pgmap v1999: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 645 op/s 2020-09-08T16:11:57.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:57 smithi166 bash[9446]: cluster 2020-09-08T16:11:56.818843+0000 osd.11 (osd.11) 134 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:57.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:57 smithi166 bash[9446]: cluster 2020-09-08T16:11:56.818913+0000 osd.11 (osd.11) 135 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:57.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:57 smithi055 bash[9296]: cluster 2020-09-08T16:11:56.148697+0000 mgr.x (mgr.24118) 1338 : cluster [DBG] pgmap v1999: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 645 op/s 2020-09-08T16:11:57.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:57 smithi055 bash[9296]: cluster 2020-09-08T16:11:56.818843+0000 osd.11 (osd.11) 134 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:11:57.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:57 smithi055 bash[9296]: cluster 2020-09-08T16:11:56.818913+0000 osd.11 (osd.11) 135 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:11:58.080 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:11:58.186 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:58.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:58 smithi055 bash[9296]: cluster 2020-09-08T16:11:57.250830+0000 mon.a (mon.0) 2314 : cluster [DBG] osdmap e746: 12 total, 12 up, 12 in 2020-09-08T16:11:58.408 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:11:58 smithi055 bash[19003]: debug 2020-09-08T16:11:58.080+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:11:58.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:58 smithi104 bash[9203]: cluster 2020-09-08T16:11:57.250830+0000 mon.a (mon.0) 2314 : cluster [DBG] osdmap e746: 12 total, 12 up, 12 in 2020-09-08T16:11:58.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:58 smithi166 bash[9446]: cluster 2020-09-08T16:11:57.250830+0000 mon.a (mon.0) 2314 : cluster [DBG] osdmap e746: 12 total, 12 up, 12 in 2020-09-08T16:11:59.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:11:59 smithi166 bash[9446]: cluster 2020-09-08T16:11:58.149249+0000 mgr.x (mgr.24118) 1339 : cluster [DBG] pgmap v2001: 17 pgs: 17 active+clean; 5.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.03k op/s 2020-09-08T16:11:59.622 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:11:59.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:11:59 smithi055 bash[9296]: cluster 2020-09-08T16:11:58.149249+0000 mgr.x (mgr.24118) 1339 : cluster [DBG] pgmap v2001: 17 pgs: 17 active+clean; 5.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.03k op/s 2020-09-08T16:11:59.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:11:59 smithi104 bash[9203]: cluster 2020-09-08T16:11:58.149249+0000 mgr.x (mgr.24118) 1339 : cluster [DBG] pgmap v2001: 17 pgs: 17 active+clean; 5.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.03k op/s 2020-09-08T16:11:59.726 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:59.864 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:11:59.968 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:00.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:11:59 smithi166 bash[11928]: debug 2020-09-08T16:11:59.618+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:00.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:11:59 smithi055 bash[14301]: debug 2020-09-08T16:11:59.864+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:00.270 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T16:12:00.401 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:00.506 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:12:00.514 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:00 smithi166 bash[9446]: audit 2020-09-08T16:11:59.490244+0000 mon.c (mon.2) 95 : audit [INF] from='client.? v1:172.21.15.104:0/230956753' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:12:00.514 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:00 smithi166 bash[9446]: audit 2020-09-08T16:11:59.493058+0000 mon.a (mon.0) 2315 : audit [INF] from='client.36680 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:12:00.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:00 smithi055 bash[9296]: audit 2020-09-08T16:11:59.490244+0000 mon.c (mon.2) 95 : audit [INF] from='client.? v1:172.21.15.104:0/230956753' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:12:00.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:00 smithi055 bash[9296]: audit 2020-09-08T16:11:59.493058+0000 mon.a (mon.0) 2315 : audit [INF] from='client.36680 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:12:00.660 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:12:00.675 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:12:00 smithi104 bash[17588]: debug 2020-09-08T16:12:00.400+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:00.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:00 smithi104 bash[9203]: audit 2020-09-08T16:11:59.490244+0000 mon.c (mon.2) 95 : audit [INF] from='client.? v1:172.21.15.104:0/230956753' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:12:00.679 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:00 smithi104 bash[9203]: audit 2020-09-08T16:11:59.493058+0000 mon.a (mon.0) 2315 : audit [INF] from='client.36680 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:12:00.761 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:00.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:12:00 smithi166 bash[16593]: debug 2020-09-08T16:12:00.654+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:00.886 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:00.995 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:01.123 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:12:00 smithi055 bash[16646]: debug 2020-09-08T16:12:00.888+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:01.126 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:01.229 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:01.349 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:01.408 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:12:01 smithi055 bash[16646]: debug 2020-09-08T16:12:01.124+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:01.409 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:12:01 smithi055 bash[16646]: debug 2020-09-08T16:12:01.348+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:01.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:01 smithi055 bash[9296]: cluster 2020-09-08T16:12:00.149808+0000 mgr.x (mgr.24118) 1340 : cluster [DBG] pgmap v2002: 17 pgs: 17 active+clean; 5.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T16:12:01.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:01 smithi055 bash[9296]: cluster 2020-09-08T16:12:00.257262+0000 mon.a (mon.0) 2316 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:12:01.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:01 smithi055 bash[9296]: cluster 2020-09-08T16:12:00.257302+0000 mon.a (mon.0) 2317 : cluster [INF] Cluster is now healthy 2020-09-08T16:12:01.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:01 smithi055 bash[9296]: audit 2020-09-08T16:12:00.269676+0000 mon.a (mon.0) 2318 : audit [INF] from='client.36680 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:12:01.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:01 smithi055 bash[9296]: cluster 2020-09-08T16:12:00.269748+0000 mon.a (mon.0) 2319 : cluster [DBG] osdmap e747: 12 total, 12 up, 12 in 2020-09-08T16:12:01.454 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:01.567 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:01 smithi166 bash[9446]: cluster 2020-09-08T16:12:00.149808+0000 mgr.x (mgr.24118) 1340 : cluster [DBG] pgmap v2002: 17 pgs: 17 active+clean; 5.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T16:12:01.567 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:01 smithi166 bash[9446]: cluster 2020-09-08T16:12:00.257262+0000 mon.a (mon.0) 2316 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:12:01.568 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:01 smithi166 bash[9446]: cluster 2020-09-08T16:12:00.257302+0000 mon.a (mon.0) 2317 : cluster [INF] Cluster is now healthy 2020-09-08T16:12:01.568 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:01 smithi166 bash[9446]: audit 2020-09-08T16:12:00.269676+0000 mon.a (mon.0) 2318 : audit [INF] from='client.36680 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:12:01.568 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:01 smithi166 bash[9446]: cluster 2020-09-08T16:12:00.269748+0000 mon.a (mon.0) 2319 : cluster [DBG] osdmap e747: 12 total, 12 up, 12 in 2020-09-08T16:12:01.569 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:12:01 smithi166 bash[18960]: debug 2020-09-08T16:12:01.566+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:01.569 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:01.675 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:12:01.733 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:01 smithi104 bash[9203]: cluster 2020-09-08T16:12:00.149808+0000 mgr.x (mgr.24118) 1340 : cluster [DBG] pgmap v2002: 17 pgs: 17 active+clean; 5.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.14k op/s 2020-09-08T16:12:01.734 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:01 smithi104 bash[9203]: cluster 2020-09-08T16:12:00.257262+0000 mon.a (mon.0) 2316 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:12:01.734 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:01 smithi104 bash[9203]: cluster 2020-09-08T16:12:00.257302+0000 mon.a (mon.0) 2317 : cluster [INF] Cluster is now healthy 2020-09-08T16:12:01.734 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:01 smithi104 bash[9203]: audit 2020-09-08T16:12:00.269676+0000 mon.a (mon.0) 2318 : audit [INF] from='client.36680 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:12:01.735 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:01 smithi104 bash[9203]: cluster 2020-09-08T16:12:00.269748+0000 mon.a (mon.0) 2319 : cluster [DBG] osdmap e747: 12 total, 12 up, 12 in 2020-09-08T16:12:02.205 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:12:02.307 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:02.438 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:02.493 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T16:12:02.494 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2020-09-08T16:12:02.494 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2020-09-08T16:12:02.494 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T16:12:02.494 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.495 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:226907 7.9 7:9de792d3:::benchmark_data_smithi166_48533_object92390:head [delete] snapc 0=[] ondisk+write+known_if_redirected e607)", 2020-09-08T16:12:02.495 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:03:44.529863+0000", 2020-09-08T16:12:02.495 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 497.95967396499998, 2020-09-08T16:12:02.495 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.485049389, 2020-09-08T16:12:02.496 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:12:02.496 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:12:02.496 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:12:02.496 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:12:02.496 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:12:02.497 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 226907 2020-09-08T16:12:02.497 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.497 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:12:02.497 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.497 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:12:02.498 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.529863+0000", 2020-09-08T16:12:02.498 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:12:02.498 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.498 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.498 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:12:02.498 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.529863+0000", 2020-09-08T16:12:02.499 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999766 2020-09-08T16:12:02.499 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.499 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.500 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:12:02.500 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.529839+0000", 2020-09-08T16:12:02.502 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9314999999999998e-05 2020-09-08T16:12:02.502 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.503 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.503 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:12:02.503 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.529889+0000", 2020-09-08T16:12:02.503 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1980000000000001e-06 2020-09-08T16:12:02.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.504 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.504 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:12:02.504 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.529897+0000", 2020-09-08T16:12:02.505 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2440000000000004e-06 2020-09-08T16:12:02.505 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.505 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.506 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:12:02.506 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.529901+0000", 2020-09-08T16:12:02.506 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.34672876400000002 2020-09-08T16:12:02.507 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.507 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.507 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:12:02.507 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.876630+0000", 2020-09-08T16:12:02.508 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000114522 2020-09-08T16:12:02.508 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.508 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.508 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:12:02.509 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.876744+0000", 2020-09-08T16:12:02.509 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011068700000000001 2020-09-08T16:12:02.510 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.510 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.510 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T16:12:02.510 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.876855+0000", 2020-09-08T16:12:02.511 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13790612699999999 2020-09-08T16:12:02.511 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.511 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.511 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:12:02.512 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.014761+0000", 2020-09-08T16:12:02.512 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1551999999999995e-05 2020-09-08T16:12:02.512 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.513 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.513 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:12:02.513 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.014833+0000", 2020-09-08T16:12:02.514 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6205999999999998e-05 2020-09-08T16:12:02.514 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.514 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.514 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:12:02.517 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.014879+0000", 2020-09-08T16:12:02.517 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3129000000000001e-05 2020-09-08T16:12:02.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.518 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:12:02.518 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.014912+0000", 2020-09-08T16:12:02.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.485049389 2020-09-08T16:12:02.520 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.520 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.521 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.521 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.522 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:226980 7.9 7:9fe58f55:::benchmark_data_smithi166_48533_object92463:head [delete] snapc 0=[] ondisk+write+known_if_redirected e607)", 2020-09-08T16:12:02.522 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:03:44.721098+0000", 2020-09-08T16:12:02.522 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 497.76843813300002, 2020-09-08T16:12:02.522 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.493088937, 2020-09-08T16:12:02.523 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:12:02.523 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:12:02.523 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:12:02.523 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:12:02.524 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:12:02.524 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 226980 2020-09-08T16:12:02.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.524 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:12:02.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:12:02.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.721098+0000", 2020-09-08T16:12:02.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:12:02.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:12:02.527 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.721098+0000", 2020-09-08T16:12:02.527 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999747 2020-09-08T16:12:02.527 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.527 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:12:02.530 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.721073+0000", 2020-09-08T16:12:02.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5899000000000001e-05 2020-09-08T16:12:02.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:12:02.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.721129+0000", 2020-09-08T16:12:02.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6709999999999994e-06 2020-09-08T16:12:02.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.532 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:12:02.533 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.721139+0000", 2020-09-08T16:12:02.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5349999999999998e-06 2020-09-08T16:12:02.533 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.534 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:12:02.534 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:44.721143+0000", 2020-09-08T16:12:02.534 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.322221333 2020-09-08T16:12:02.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:12:02.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.043365+0000", 2020-09-08T16:12:02.536 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012756000000000001 2020-09-08T16:12:02.536 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.536 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.536 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:12:02.537 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.043492+0000", 2020-09-08T16:12:02.537 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000115265 2020-09-08T16:12:02.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.538 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.538 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T16:12:02.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.043608+0000", 2020-09-08T16:12:02.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.158479866 2020-09-08T16:12:02.539 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:12:02.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.202087+0000", 2020-09-08T16:12:02.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.011980032999999999 2020-09-08T16:12:02.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:12:02.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.214067+0000", 2020-09-08T16:12:02.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4586999999999997e-05 2020-09-08T16:12:02.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.543 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:12:02.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.214132+0000", 2020-09-08T16:12:02.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5337000000000002e-05 2020-09-08T16:12:02.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:12:02.550 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.214187+0000", 2020-09-08T16:12:02.557 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.493088937 2020-09-08T16:12:02.557 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.558 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.559 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.560 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36113.0:227340 7.9 7:9b3f70e5:::benchmark_data_smithi166_48533_object92823:head [delete] snapc 0=[] ondisk+write+known_if_redirected e607)", 2020-09-08T16:12:02.560 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:03:45.370870+0000", 2020-09-08T16:12:02.560 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 497.118666394, 2020-09-08T16:12:02.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.51068199400000003, 2020-09-08T16:12:02.561 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:12:02.561 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:12:02.561 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:12:02.562 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36113", 2020-09-08T16:12:02.562 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/3866093404", 2020-09-08T16:12:02.562 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 227340 2020-09-08T16:12:02.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.563 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:12:02.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:12:02.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.370870+0000", 2020-09-08T16:12:02.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:12:02.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.564 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:12:02.565 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.370870+0000", 2020-09-08T16:12:02.565 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999771 2020-09-08T16:12:02.565 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.566 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.566 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:12:02.567 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.370847+0000", 2020-09-08T16:12:02.567 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2298e-05 2020-09-08T16:12:02.568 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.568 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.568 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:12:02.570 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.370900+0000", 2020-09-08T16:12:02.570 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1313e-05 2020-09-08T16:12:02.570 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.570 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.571 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:12:02.571 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.370911+0000", 2020-09-08T16:12:02.571 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6820000000000002e-06 2020-09-08T16:12:02.571 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.571 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.572 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:12:02.572 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.370916+0000", 2020-09-08T16:12:02.572 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.370712027 2020-09-08T16:12:02.572 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.573 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.573 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:12:02.573 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.741628+0000", 2020-09-08T16:12:02.573 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000194477 2020-09-08T16:12:02.573 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.574 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:12:02.574 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.741822+0000", 2020-09-08T16:12:02.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000172304 2020-09-08T16:12:02.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 11", 2020-09-08T16:12:02.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.741994+0000", 2020-09-08T16:12:02.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0056553039999999999 2020-09-08T16:12:02.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.576 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:12:02.576 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.747650+0000", 2020-09-08T16:12:02.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13372262600000001 2020-09-08T16:12:02.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:12:02.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.881372+0000", 2020-09-08T16:12:02.578 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000106035 2020-09-08T16:12:02.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.578 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.578 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:12:02.579 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.881478+0000", 2020-09-08T16:12:02.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3815999999999999e-05 2020-09-08T16:12:02.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:12:02.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:03:45.881552+0000", 2020-09-08T16:12:02.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.51068199400000003 2020-09-08T16:12:02.581 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.581 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.581 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.581 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.582 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31643 8.e 8:736bce26:::benchmark_data_smithi166_52302_object31642:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:12:02.582 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.750728+0000", 2020-09-08T16:12:02.582 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 403.73880817100002, 2020-09-08T16:12:02.582 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.61505463800000004, 2020-09-08T16:12:02.583 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:12:02.583 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:12:02.583 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:12:02.583 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:12:02.584 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:12:02.584 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31643 2020-09-08T16:12:02.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.584 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:12:02.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.585 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:12:02.585 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.750728+0000", 2020-09-08T16:12:02.585 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:12:02.585 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.585 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:12:02.586 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.750728+0000", 2020-09-08T16:12:02.586 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999356 2020-09-08T16:12:02.586 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.587 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.587 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:12:02.587 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.750664+0000", 2020-09-08T16:12:02.587 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000181678 2020-09-08T16:12:02.588 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.588 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.588 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:12:02.588 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.750846+0000", 2020-09-08T16:12:02.588 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6409000000000002e-05 2020-09-08T16:12:02.589 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.589 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.589 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:12:02.589 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.750862+0000", 2020-09-08T16:12:02.590 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5299999999999996e-06 2020-09-08T16:12:02.590 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.590 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.591 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:12:02.591 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.750870+0000", 2020-09-08T16:12:02.591 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6950999999999997e-05 2020-09-08T16:12:02.591 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.591 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.592 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:12:02.592 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.750927+0000", 2020-09-08T16:12:02.592 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000201898 2020-09-08T16:12:02.592 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.593 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:12:02.593 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.751129+0000", 2020-09-08T16:12:02.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00023285100000000001 2020-09-08T16:12:02.593 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.594 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:12:02.594 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.751362+0000", 2020-09-08T16:12:02.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0013584879999999999 2020-09-08T16:12:02.594 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.595 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.595 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:12:02.595 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.752721+0000", 2020-09-08T16:12:02.595 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.61295605099999995 2020-09-08T16:12:02.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:12:02.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.365677+0000", 2020-09-08T16:12:02.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4460000000000003e-05 2020-09-08T16:12:02.597 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.597 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:12:02.597 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.365741+0000", 2020-09-08T16:12:02.597 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1807000000000001e-05 2020-09-08T16:12:02.598 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.598 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.598 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:12:02.598 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.365783+0000", 2020-09-08T16:12:02.598 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.61505463800000004 2020-09-08T16:12:02.599 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.599 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.599 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.599 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.600 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31705 8.e 8:71681bc2:::benchmark_data_smithi166_52302_object31704:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:12:02.600 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.782456+0000", 2020-09-08T16:12:02.601 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 403.70708035600001, 2020-09-08T16:12:02.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.58349529499999997, 2020-09-08T16:12:02.601 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:12:02.601 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:12:02.602 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:12:02.602 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:12:02.602 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:12:02.602 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31705 2020-09-08T16:12:02.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.603 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:12:02.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:12:02.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.782456+0000", 2020-09-08T16:12:02.604 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:12:02.604 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:12:02.604 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.782456+0000", 2020-09-08T16:12:02.605 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999528 2020-09-08T16:12:02.605 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.605 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.605 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:12:02.605 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.782409+0000", 2020-09-08T16:12:02.606 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017631600000000001 2020-09-08T16:12:02.606 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.606 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.606 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:12:02.606 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.782585+0000", 2020-09-08T16:12:02.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4258e-05 2020-09-08T16:12:02.607 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.607 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:12:02.608 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.782600+0000", 2020-09-08T16:12:02.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0360000000000006e-06 2020-09-08T16:12:02.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:12:02.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.782608+0000", 2020-09-08T16:12:02.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4220000000000005e-05 2020-09-08T16:12:02.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.609 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.609 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:12:02.610 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.782672+0000", 2020-09-08T16:12:02.610 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020994500000000001 2020-09-08T16:12:02.610 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.610 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.611 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:12:02.612 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.782882+0000", 2020-09-08T16:12:02.612 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00022301600000000001 2020-09-08T16:12:02.612 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.612 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.612 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:12:02.613 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.783105+0000", 2020-09-08T16:12:02.613 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0016531040000000001 2020-09-08T16:12:02.613 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.613 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.613 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:12:02.614 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.784758+0000", 2020-09-08T16:12:02.614 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.58109964199999997 2020-09-08T16:12:02.614 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:12:02.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.365858+0000", 2020-09-08T16:12:02.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4354999999999998e-05 2020-09-08T16:12:02.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.615 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.616 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:12:02.616 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.365902+0000", 2020-09-08T16:12:02.616 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9607999999999997e-05 2020-09-08T16:12:02.616 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.616 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.617 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:12:02.617 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.365951+0000", 2020-09-08T16:12:02.617 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.58349529499999997 2020-09-08T16:12:02.617 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.617 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.618 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.618 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.618 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31736 8.e 8:7c50f50c:::benchmark_data_smithi166_52302_object31735:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:12:02.618 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.797191+0000", 2020-09-08T16:12:02.619 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 403.69234597000002, 2020-09-08T16:12:02.619 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.57092679499999999, 2020-09-08T16:12:02.619 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:12:02.619 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:12:02.619 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:12:02.620 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:12:02.620 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:12:02.620 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31736 2020-09-08T16:12:02.621 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.621 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:12:02.621 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.621 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:12:02.621 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.797191+0000", 2020-09-08T16:12:02.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:12:02.622 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:12:02.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.797191+0000", 2020-09-08T16:12:02.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999437 2020-09-08T16:12:02.623 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.623 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.623 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:12:02.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.797134+0000", 2020-09-08T16:12:02.624 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017540200000000001 2020-09-08T16:12:02.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.624 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:12:02.624 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.797310+0000", 2020-09-08T16:12:02.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4871999999999998e-05 2020-09-08T16:12:02.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.625 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:12:02.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.797334+0000", 2020-09-08T16:12:02.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6299999999999998e-06 2020-09-08T16:12:02.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:12:02.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.797342+0000", 2020-09-08T16:12:02.627 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000104518 2020-09-08T16:12:02.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.627 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:12:02.627 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.797447+0000", 2020-09-08T16:12:02.628 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000231299 2020-09-08T16:12:02.628 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.628 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.628 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:12:02.628 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.797678+0000", 2020-09-08T16:12:02.629 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018062800000000001 2020-09-08T16:12:02.629 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.629 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:12:02.629 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.797859+0000", 2020-09-08T16:12:02.630 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0026988749999999999 2020-09-08T16:12:02.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.630 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.630 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:12:02.630 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.800557+0000", 2020-09-08T16:12:02.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.56747323500000002 2020-09-08T16:12:02.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.631 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.631 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:12:02.631 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.368031+0000", 2020-09-08T16:12:02.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0442000000000003e-05 2020-09-08T16:12:02.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.632 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.632 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:12:02.632 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.368081+0000", 2020-09-08T16:12:02.632 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6242000000000003e-05 2020-09-08T16:12:02.633 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.633 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:12:02.633 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.368117+0000", 2020-09-08T16:12:02.633 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.57092679499999999 2020-09-08T16:12:02.634 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.634 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.634 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.634 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.635 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31756 8.e 8:76fbe07a:::benchmark_data_smithi166_52302_object31755:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:12:02.635 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.805769+0000", 2020-09-08T16:12:02.635 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 403.68376781799998, 2020-09-08T16:12:02.635 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.56253396600000005, 2020-09-08T16:12:02.635 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:12:02.636 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:12:02.636 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:12:02.636 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:12:02.636 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:12:02.636 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31756 2020-09-08T16:12:02.636 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.637 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:12:02.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.637 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:12:02.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.805769+0000", 2020-09-08T16:12:02.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:12:02.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.638 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:12:02.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.805769+0000", 2020-09-08T16:12:02.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999695 2020-09-08T16:12:02.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:12:02.639 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.805738+0000", 2020-09-08T16:12:02.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011208 2020-09-08T16:12:02.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.640 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:12:02.640 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.805850+0000", 2020-09-08T16:12:02.640 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5447999999999998e-05 2020-09-08T16:12:02.641 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.641 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.641 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:12:02.641 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.805866+0000", 2020-09-08T16:12:02.641 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.242e-06 2020-09-08T16:12:02.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:12:02.642 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.805874+0000", 2020-09-08T16:12:02.642 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3807999999999996e-05 2020-09-08T16:12:02.643 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.643 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.643 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:12:02.643 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.805938+0000", 2020-09-08T16:12:02.643 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017503199999999999 2020-09-08T16:12:02.644 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.644 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.644 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:12:02.644 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.806113+0000", 2020-09-08T16:12:02.644 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019673 2020-09-08T16:12:02.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.645 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:12:02.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.806310+0000", 2020-09-08T16:12:02.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0015218429999999999 2020-09-08T16:12:02.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.646 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:12:02.646 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.807832+0000", 2020-09-08T16:12:02.646 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.56039003600000004 2020-09-08T16:12:02.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:12:02.647 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.368222+0000", 2020-09-08T16:12:02.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1275e-05 2020-09-08T16:12:02.648 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.648 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.649 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:12:02.649 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.368263+0000", 2020-09-08T16:12:02.649 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9841e-05 2020-09-08T16:12:02.649 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.649 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.650 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:12:02.650 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.368303+0000", 2020-09-08T16:12:02.650 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.56253396600000005 2020-09-08T16:12:02.650 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.650 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.651 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.651 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.651 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.651 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31768 8.e 8:75141454:::benchmark_data_smithi166_52302_object31767:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:12:02.651 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.811615+0000", 2020-09-08T16:12:02.652 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 403.67792148500001, 2020-09-08T16:12:02.652 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.55852875300000004, 2020-09-08T16:12:02.652 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:12:02.652 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:12:02.652 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:12:02.653 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:12:02.653 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:12:02.653 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31768 2020-09-08T16:12:02.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.653 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:12:02.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:12:02.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.811615+0000", 2020-09-08T16:12:02.654 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:12:02.655 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.655 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.655 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:12:02.655 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.811615+0000", 2020-09-08T16:12:02.656 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999371 2020-09-08T16:12:02.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.656 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.656 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:12:02.656 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.811552+0000", 2020-09-08T16:12:02.657 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000193864 2020-09-08T16:12:02.657 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.657 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.657 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:12:02.658 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.811746+0000", 2020-09-08T16:12:02.658 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6623999999999999e-05 2020-09-08T16:12:02.658 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.658 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.658 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:12:02.659 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.811762+0000", 2020-09-08T16:12:02.659 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5710000000000004e-06 2020-09-08T16:12:02.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.659 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.659 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:12:02.660 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.811772+0000", 2020-09-08T16:12:02.660 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6433000000000004e-05 2020-09-08T16:12:02.660 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.661 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.661 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:12:02.661 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.811858+0000", 2020-09-08T16:12:02.661 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000288282 2020-09-08T16:12:02.662 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.662 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.662 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:12:02.662 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.812147+0000", 2020-09-08T16:12:02.662 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00026367099999999999 2020-09-08T16:12:02.663 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.663 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.663 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:12:02.663 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.812410+0000", 2020-09-08T16:12:02.663 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0018146010000000001 2020-09-08T16:12:02.664 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.664 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.664 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:12:02.664 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.814225+0000", 2020-09-08T16:12:02.664 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.55576947099999996 2020-09-08T16:12:02.665 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.665 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.665 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:12:02.665 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.369995+0000", 2020-09-08T16:12:02.666 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9792999999999999e-05 2020-09-08T16:12:02.666 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.666 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.666 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:12:02.666 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.370084+0000", 2020-09-08T16:12:02.667 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9508e-05 2020-09-08T16:12:02.667 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.667 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.667 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:12:02.667 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.370144+0000", 2020-09-08T16:12:02.668 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.55852875300000004 2020-09-08T16:12:02.668 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.668 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.668 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.669 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.669 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.669 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31783 8.e 8:70e2f01d:::benchmark_data_smithi166_52302_object31782:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:12:02.669 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.818023+0000", 2020-09-08T16:12:02.670 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 403.67151391499999, 2020-09-08T16:12:02.670 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.552480792, 2020-09-08T16:12:02.670 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:12:02.670 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:12:02.671 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:12:02.671 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:12:02.671 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:12:02.671 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31783 2020-09-08T16:12:02.671 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.672 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:12:02.672 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.672 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:12:02.672 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818023+0000", 2020-09-08T16:12:02.673 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:12:02.673 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.673 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.673 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:12:02.674 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818023+0000", 2020-09-08T16:12:02.674 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999285 2020-09-08T16:12:02.674 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.674 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.674 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:12:02.675 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.817951+0000", 2020-09-08T16:12:02.675 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016045400000000001 2020-09-08T16:12:02.675 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.675 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.675 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:12:02.676 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818112+0000", 2020-09-08T16:12:02.676 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8515000000000001e-05 2020-09-08T16:12:02.676 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.676 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.676 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:12:02.677 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818130+0000", 2020-09-08T16:12:02.677 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3018e-05 2020-09-08T16:12:02.677 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.677 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.677 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:12:02.678 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818143+0000", 2020-09-08T16:12:02.678 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2869999999999997e-05 2020-09-08T16:12:02.678 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.678 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.678 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:12:02.679 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818236+0000", 2020-09-08T16:12:02.679 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018752999999999999 2020-09-08T16:12:02.679 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.679 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.679 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:12:02.680 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818423+0000", 2020-09-08T16:12:02.680 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018469299999999999 2020-09-08T16:12:02.680 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.680 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.680 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:12:02.681 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818608+0000", 2020-09-08T16:12:02.681 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0034836210000000001 2020-09-08T16:12:02.681 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.681 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.681 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:12:02.681 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.822092+0000", 2020-09-08T16:12:02.682 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.54820981400000002 2020-09-08T16:12:02.682 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.682 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.682 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:12:02.682 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.370302+0000", 2020-09-08T16:12:02.683 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014577500000000001 2020-09-08T16:12:02.683 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.683 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.683 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:12:02.684 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.370447+0000", 2020-09-08T16:12:02.684 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6060000000000002e-05 2020-09-08T16:12:02.684 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.685 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.685 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:12:02.685 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.370503+0000", 2020-09-08T16:12:02.685 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.552480792 2020-09-08T16:12:02.686 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.686 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.686 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.686 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.686 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.687 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31785 8.e 8:729ea262:::benchmark_data_smithi166_52302_object31784:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:12:02.687 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.818851+0000", 2020-09-08T16:12:02.688 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 403.67068599300001, 2020-09-08T16:12:02.688 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.55661606799999996, 2020-09-08T16:12:02.688 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:12:02.688 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:12:02.689 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:12:02.689 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:12:02.689 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:12:02.689 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31785 2020-09-08T16:12:02.690 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.690 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:12:02.690 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.690 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:12:02.690 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818851+0000", 2020-09-08T16:12:02.691 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:12:02.691 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.691 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.691 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:12:02.691 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818851+0000", 2020-09-08T16:12:02.692 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999475 2020-09-08T16:12:02.692 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.692 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.692 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:12:02.693 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818798+0000", 2020-09-08T16:12:02.693 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000192981 2020-09-08T16:12:02.693 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.693 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.693 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:12:02.694 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.818991+0000", 2020-09-08T16:12:02.694 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6706000000000003e-05 2020-09-08T16:12:02.694 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.694 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.695 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:12:02.695 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.819038+0000", 2020-09-08T16:12:02.695 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9558000000000001e-05 2020-09-08T16:12:02.695 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.695 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.696 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:12:02.696 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.819057+0000", 2020-09-08T16:12:02.696 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0012248630000000001 2020-09-08T16:12:02.696 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.697 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.697 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:12:02.697 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.820282+0000", 2020-09-08T16:12:02.697 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00030317400000000002 2020-09-08T16:12:02.697 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.698 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.698 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:12:02.698 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.820585+0000", 2020-09-08T16:12:02.698 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019230500000000001 2020-09-08T16:12:02.699 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.699 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.699 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:12:02.699 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.820778+0000", 2020-09-08T16:12:02.700 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0017226139999999999 2020-09-08T16:12:02.700 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.700 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.700 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:12:02.700 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.822500+0000", 2020-09-08T16:12:02.701 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.55285292500000005 2020-09-08T16:12:02.701 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.701 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.701 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:12:02.701 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.375353+0000", 2020-09-08T16:12:02.702 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.2335000000000005e-05 2020-09-08T16:12:02.702 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.702 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.702 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:12:02.702 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.375425+0000", 2020-09-08T16:12:02.702 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1214999999999998e-05 2020-09-08T16:12:02.703 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.703 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.703 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:12:02.703 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.375467+0000", 2020-09-08T16:12:02.703 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.55661606799999996 2020-09-08T16:12:02.704 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.704 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.704 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.704 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.704 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.705 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31796 8.e 8:7d49049a:::benchmark_data_smithi166_52302_object31795:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:12:02.705 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.823126+0000", 2020-09-08T16:12:02.705 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 403.66641053400002, 2020-09-08T16:12:02.705 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.55262376300000005, 2020-09-08T16:12:02.705 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:12:02.706 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:12:02.706 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:12:02.706 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:12:02.706 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:12:02.706 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31796 2020-09-08T16:12:02.707 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.707 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:12:02.707 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.707 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:12:02.707 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.823126+0000", 2020-09-08T16:12:02.708 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:12:02.708 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.708 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.708 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:12:02.708 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.823126+0000", 2020-09-08T16:12:02.708 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999585 2020-09-08T16:12:02.709 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.709 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.709 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:12:02.709 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.823085+0000", 2020-09-08T16:12:02.709 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000205186 2020-09-08T16:12:02.710 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.710 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.710 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:12:02.710 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.823290+0000", 2020-09-08T16:12:02.711 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2133999999999998e-05 2020-09-08T16:12:02.711 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.711 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.712 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:12:02.712 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.823312+0000", 2020-09-08T16:12:02.712 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2764e-05 2020-09-08T16:12:02.712 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.712 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.713 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:12:02.713 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.823325+0000", 2020-09-08T16:12:02.713 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7626000000000006e-05 2020-09-08T16:12:02.713 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.713 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.713 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:12:02.714 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.823392+0000", 2020-09-08T16:12:02.714 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019909499999999999 2020-09-08T16:12:02.714 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.714 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.714 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:12:02.715 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.823591+0000", 2020-09-08T16:12:02.715 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00026281999999999998 2020-09-08T16:12:02.715 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.715 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.715 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:12:02.716 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.823854+0000", 2020-09-08T16:12:02.716 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001309209 2020-09-08T16:12:02.716 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.716 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.716 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:12:02.717 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.825163+0000", 2020-09-08T16:12:02.717 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.550413869 2020-09-08T16:12:02.717 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.717 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.717 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:12:02.718 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.375577+0000", 2020-09-08T16:12:02.718 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4975999999999993e-05 2020-09-08T16:12:02.718 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.718 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.718 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:12:02.719 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.375672+0000", 2020-09-08T16:12:02.719 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7526000000000003e-05 2020-09-08T16:12:02.719 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.719 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.719 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:12:02.719 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.375750+0000", 2020-09-08T16:12:02.720 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.55262376300000005 2020-09-08T16:12:02.720 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.720 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.720 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.720 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.721 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.721 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31818 8.e 8:755728de:::benchmark_data_smithi166_52302_object31817:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:12:02.721 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.834010+0000", 2020-09-08T16:12:02.721 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 403.65552633099998, 2020-09-08T16:12:02.722 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.54188254300000005, 2020-09-08T16:12:02.722 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:12:02.722 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:12:02.722 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:12:02.723 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:12:02.723 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:12:02.723 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31818 2020-09-08T16:12:02.723 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.723 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:12:02.724 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.724 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:12:02.724 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.834010+0000", 2020-09-08T16:12:02.724 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:12:02.725 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.725 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.725 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:12:02.726 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.834010+0000", 2020-09-08T16:12:02.726 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999766 2020-09-08T16:12:02.726 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.727 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.727 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:12:02.727 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.833987+0000", 2020-09-08T16:12:02.727 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011306000000000001 2020-09-08T16:12:02.728 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.728 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.728 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:12:02.729 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.834100+0000", 2020-09-08T16:12:02.729 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.306e-05 2020-09-08T16:12:02.729 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.730 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.730 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:12:02.730 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.834113+0000", 2020-09-08T16:12:02.731 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9039999999999992e-06 2020-09-08T16:12:02.731 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.731 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.732 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:12:02.732 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.834121+0000", 2020-09-08T16:12:02.732 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8900000000000003e-05 2020-09-08T16:12:02.733 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.733 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.733 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:12:02.734 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.834170+0000", 2020-09-08T16:12:02.734 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012346699999999999 2020-09-08T16:12:02.734 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.735 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.735 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:12:02.735 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.834293+0000", 2020-09-08T16:12:02.736 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000121549 2020-09-08T16:12:02.736 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.736 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.737 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:12:02.737 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.834415+0000", 2020-09-08T16:12:02.737 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001383544 2020-09-08T16:12:02.738 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.738 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.738 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:12:02.739 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.835798+0000", 2020-09-08T16:12:02.739 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.53999969400000003 2020-09-08T16:12:02.739 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.740 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.740 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:12:02.740 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.375798+0000", 2020-09-08T16:12:02.741 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5198000000000002e-05 2020-09-08T16:12:02.741 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.741 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.742 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:12:02.742 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.375853+0000", 2020-09-08T16:12:02.742 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9635000000000003e-05 2020-09-08T16:12:02.743 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.743 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.743 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:12:02.744 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.375893+0000", 2020-09-08T16:12:02.744 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.54188254300000005 2020-09-08T16:12:02.745 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.745 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.745 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.746 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.746 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.746 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31823 8.e 8:75100c31:::benchmark_data_smithi166_52302_object31822:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:12:02.747 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.837476+0000", 2020-09-08T16:12:02.747 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 403.65206050099999, 2020-09-08T16:12:02.747 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.53864092699999999, 2020-09-08T16:12:02.748 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:12:02.748 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:12:02.748 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:12:02.749 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:12:02.749 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:12:02.749 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31823 2020-09-08T16:12:02.750 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.750 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:12:02.750 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.751 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:12:02.751 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837476+0000", 2020-09-08T16:12:02.751 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:12:02.752 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.752 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.752 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:12:02.753 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837476+0000", 2020-09-08T16:12:02.753 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999728 2020-09-08T16:12:02.753 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.761 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.761 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:12:02.762 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837449+0000", 2020-09-08T16:12:02.762 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020261500000000001 2020-09-08T16:12:02.762 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.763 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.763 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:12:02.763 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837651+0000", 2020-09-08T16:12:02.763 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2478999999999999e-05 2020-09-08T16:12:02.764 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.764 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.764 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:12:02.764 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837674+0000", 2020-09-08T16:12:02.764 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0239999999999997e-06 2020-09-08T16:12:02.765 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.765 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.765 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:12:02.765 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837682+0000", 2020-09-08T16:12:02.765 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6309000000000005e-05 2020-09-08T16:12:02.766 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.766 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.766 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:12:02.766 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837748+0000", 2020-09-08T16:12:02.767 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020641600000000001 2020-09-08T16:12:02.767 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.767 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.767 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:12:02.767 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837955+0000", 2020-09-08T16:12:02.768 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000256066 2020-09-08T16:12:02.768 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.768 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.768 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:12:02.768 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.838211+0000", 2020-09-08T16:12:02.768 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0021088489999999999 2020-09-08T16:12:02.769 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.769 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.769 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:12:02.769 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.840319+0000", 2020-09-08T16:12:02.769 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.53564020400000001 2020-09-08T16:12:02.770 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.770 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.770 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:12:02.770 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.375960+0000", 2020-09-08T16:12:02.770 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5110000000000003e-05 2020-09-08T16:12:02.771 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.771 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.771 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:12:02.771 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376045+0000", 2020-09-08T16:12:02.771 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.2197e-05 2020-09-08T16:12:02.772 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.772 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.772 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:12:02.772 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376117+0000", 2020-09-08T16:12:02.772 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.53864092699999999 2020-09-08T16:12:02.773 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.773 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.773 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.773 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.773 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.774 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31825 8.e 8:7a912d15:::benchmark_data_smithi166_52302_object31824:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:12:02.774 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.837773+0000", 2020-09-08T16:12:02.774 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 403.65176355599999, 2020-09-08T16:12:02.774 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.53848058200000004, 2020-09-08T16:12:02.774 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:12:02.775 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:12:02.775 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:12:02.775 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:12:02.776 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:12:02.776 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31825 2020-09-08T16:12:02.776 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.776 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:12:02.777 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.777 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:12:02.777 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837773+0000", 2020-09-08T16:12:02.778 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:12:02.778 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.778 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.779 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:12:02.779 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837773+0000", 2020-09-08T16:12:02.779 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999552 2020-09-08T16:12:02.780 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.780 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.780 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:12:02.780 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837728+0000", 2020-09-08T16:12:02.781 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017725999999999999 2020-09-08T16:12:02.781 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.781 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.782 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:12:02.782 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837905+0000", 2020-09-08T16:12:02.782 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5651e-05 2020-09-08T16:12:02.783 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.783 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.783 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:12:02.784 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837921+0000", 2020-09-08T16:12:02.784 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8410000000000002e-06 2020-09-08T16:12:02.784 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.784 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.785 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:12:02.785 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.837928+0000", 2020-09-08T16:12:02.785 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0013380829999999999 2020-09-08T16:12:02.785 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.785 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.786 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:12:02.786 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.839266+0000", 2020-09-08T16:12:02.786 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012799299999999999 2020-09-08T16:12:02.786 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.786 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.787 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:12:02.787 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.839394+0000", 2020-09-08T16:12:02.787 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015803 2020-09-08T16:12:02.787 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.787 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.788 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:12:02.788 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.839552+0000", 2020-09-08T16:12:02.788 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00098459299999999997 2020-09-08T16:12:02.788 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.788 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.789 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:12:02.789 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.840536+0000", 2020-09-08T16:12:02.789 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.53563405900000005 2020-09-08T16:12:02.789 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.789 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.790 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:12:02.790 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376170+0000", 2020-09-08T16:12:02.790 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8896000000000001e-05 2020-09-08T16:12:02.790 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.790 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.791 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:12:02.791 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376219+0000", 2020-09-08T16:12:02.791 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4177999999999997e-05 2020-09-08T16:12:02.791 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.792 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.792 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:12:02.792 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376254+0000", 2020-09-08T16:12:02.792 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.53848058200000004 2020-09-08T16:12:02.792 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.793 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.793 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.793 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.794 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.794 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31864 8.e 8:7f85638c:::benchmark_data_smithi166_52302_object31863:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:12:02.794 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.866337+0000", 2020-09-08T16:12:02.794 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 403.62319998800001, 2020-09-08T16:12:02.795 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.51017505299999999, 2020-09-08T16:12:02.795 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:12:02.795 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:12:02.795 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:12:02.795 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:12:02.796 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:12:02.796 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31864 2020-09-08T16:12:02.796 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.796 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:12:02.796 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.797 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:12:02.797 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.866337+0000", 2020-09-08T16:12:02.797 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:12:02.797 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.797 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.798 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:12:02.798 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.866337+0000", 2020-09-08T16:12:02.798 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999552 2020-09-08T16:12:02.798 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.798 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.799 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:12:02.799 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.866292+0000", 2020-09-08T16:12:02.799 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014819099999999999 2020-09-08T16:12:02.799 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.799 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.800 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:12:02.800 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.866440+0000", 2020-09-08T16:12:02.800 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1081999999999999e-05 2020-09-08T16:12:02.800 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.800 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.801 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:12:02.801 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.866461+0000", 2020-09-08T16:12:02.801 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5469999999999997e-06 2020-09-08T16:12:02.801 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.801 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.802 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:12:02.802 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.866470+0000", 2020-09-08T16:12:02.802 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0965999999999997e-05 2020-09-08T16:12:02.802 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.802 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.803 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:12:02.803 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.866551+0000", 2020-09-08T16:12:02.803 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000222134 2020-09-08T16:12:02.803 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.803 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.804 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:12:02.804 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.866773+0000", 2020-09-08T16:12:02.804 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00025741099999999998 2020-09-08T16:12:02.804 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.804 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.805 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:12:02.805 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.867030+0000", 2020-09-08T16:12:02.805 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0017267059999999999 2020-09-08T16:12:02.805 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.805 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.806 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:12:02.806 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.868757+0000", 2020-09-08T16:12:02.806 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.507591825 2020-09-08T16:12:02.806 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.806 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.807 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:12:02.807 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376349+0000", 2020-09-08T16:12:02.807 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2278000000000001e-05 2020-09-08T16:12:02.807 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.807 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.807 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:12:02.808 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376431+0000", 2020-09-08T16:12:02.808 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0594999999999997e-05 2020-09-08T16:12:02.808 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.808 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.808 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:12:02.809 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376512+0000", 2020-09-08T16:12:02.809 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.51017505299999999 2020-09-08T16:12:02.809 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.809 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.809 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.810 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.810 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.810 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:31885 8.e 8:7d64cde5:::benchmark_data_smithi166_52302_object31884:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e637)", 2020-09-08T16:12:02.810 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:05:18.888695+0000", 2020-09-08T16:12:02.810 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 403.60084107400002, 2020-09-08T16:12:02.811 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.48797019899999999, 2020-09-08T16:12:02.811 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:12:02.811 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:12:02.811 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:12:02.811 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:12:02.812 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:12:02.812 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31885 2020-09-08T16:12:02.812 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.812 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:12:02.812 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.813 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:12:02.813 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.888695+0000", 2020-09-08T16:12:02.813 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:12:02.813 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.813 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.814 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:12:02.814 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.888695+0000", 2020-09-08T16:12:02.814 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999337 2020-09-08T16:12:02.814 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.814 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.814 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:12:02.815 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.888629+0000", 2020-09-08T16:12:02.815 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00021185999999999999 2020-09-08T16:12:02.815 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.815 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.815 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:12:02.816 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.888841+0000", 2020-09-08T16:12:02.816 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6365999999999997e-05 2020-09-08T16:12:02.816 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.816 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.816 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:12:02.817 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.888878+0000", 2020-09-08T16:12:02.817 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3505e-05 2020-09-08T16:12:02.817 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.817 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.817 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:12:02.818 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.888891+0000", 2020-09-08T16:12:02.818 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000101461 2020-09-08T16:12:02.818 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.818 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.818 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:12:02.819 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.888993+0000", 2020-09-08T16:12:02.819 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00028271600000000002 2020-09-08T16:12:02.819 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.819 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.819 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:12:02.819 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.889275+0000", 2020-09-08T16:12:02.820 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00021280800000000001 2020-09-08T16:12:02.820 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.820 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.820 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:12:02.820 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.889488+0000", 2020-09-08T16:12:02.821 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020818630000000002 2020-09-08T16:12:02.821 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.821 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.821 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:12:02.821 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:18.891570+0000", 2020-09-08T16:12:02.822 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.48500274799999998 2020-09-08T16:12:02.822 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.822 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.822 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:12:02.822 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376573+0000", 2020-09-08T16:12:02.823 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0531999999999997e-05 2020-09-08T16:12:02.823 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.823 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.823 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:12:02.823 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376623+0000", 2020-09-08T16:12:02.824 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2401000000000003e-05 2020-09-08T16:12:02.824 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.824 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.824 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:12:02.824 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:05:19.376666+0000", 2020-09-08T16:12:02.824 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.48797019899999999 2020-09-08T16:12:02.825 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.825 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.825 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.825 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.825 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.826 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:123852 8.e 8:7fbe813d:::benchmark_data_smithi166_52302_object123851:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e656)", 2020-09-08T16:12:02.826 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:06:27.491840+0000", 2020-09-08T16:12:02.826 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 334.99769686899998, 2020-09-08T16:12:02.826 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.57404195000000002, 2020-09-08T16:12:02.826 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:12:02.827 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:12:02.827 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:12:02.827 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:12:02.827 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:12:02.827 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 123852 2020-09-08T16:12:02.828 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.828 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:12:02.828 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.828 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:12:02.828 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.491840+0000", 2020-09-08T16:12:02.829 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:12:02.829 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.829 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.829 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:12:02.829 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.491840+0000", 2020-09-08T16:12:02.830 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999518 2020-09-08T16:12:02.830 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.830 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.830 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:12:02.830 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.491791+0000", 2020-09-08T16:12:02.831 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0001906 2020-09-08T16:12:02.831 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.831 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.831 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:12:02.831 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.491982+0000", 2020-09-08T16:12:02.831 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3278999999999999e-05 2020-09-08T16:12:02.832 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.832 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.832 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:12:02.832 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.492005+0000", 2020-09-08T16:12:02.832 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8269999999999993e-06 2020-09-08T16:12:02.833 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.833 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.833 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:12:02.833 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.492015+0000", 2020-09-08T16:12:02.833 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.449991539 2020-09-08T16:12:02.834 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.834 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.834 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:12:02.834 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.942007+0000", 2020-09-08T16:12:02.834 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00025862900000000002 2020-09-08T16:12:02.835 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.835 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.835 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:12:02.835 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.942265+0000", 2020-09-08T16:12:02.835 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00033429400000000001 2020-09-08T16:12:02.836 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.836 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.836 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:12:02.836 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.942600+0000", 2020-09-08T16:12:02.836 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.024758130999999999 2020-09-08T16:12:02.836 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.837 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.837 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:12:02.837 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.967358+0000", 2020-09-08T16:12:02.837 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.098383480999999995 2020-09-08T16:12:02.837 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.838 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.838 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:12:02.838 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.065741+0000", 2020-09-08T16:12:02.838 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2052000000000007e-05 2020-09-08T16:12:02.838 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.839 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.839 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:12:02.839 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.065823+0000", 2020-09-08T16:12:02.839 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8335000000000003e-05 2020-09-08T16:12:02.839 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.840 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.840 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:12:02.840 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.065882+0000", 2020-09-08T16:12:02.840 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.57404195000000002 2020-09-08T16:12:02.840 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.841 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.841 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.841 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.841 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.841 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:123863 8.e 8:70bdf1a9:::benchmark_data_smithi166_52302_object123862:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e656)", 2020-09-08T16:12:02.842 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:06:27.521581+0000", 2020-09-08T16:12:02.842 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 334.96795542299998, 2020-09-08T16:12:02.842 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.54466917999999998, 2020-09-08T16:12:02.842 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:12:02.842 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:12:02.843 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:12:02.843 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:12:02.843 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:12:02.843 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 123863 2020-09-08T16:12:02.843 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.843 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:12:02.844 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.844 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:12:02.844 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.521581+0000", 2020-09-08T16:12:02.844 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:12:02.844 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.845 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.845 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:12:02.845 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.521581+0000", 2020-09-08T16:12:02.845 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999447 2020-09-08T16:12:02.845 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.846 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.846 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:12:02.846 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.521526+0000", 2020-09-08T16:12:02.846 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016246399999999999 2020-09-08T16:12:02.846 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.847 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.847 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:12:02.847 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.521688+0000", 2020-09-08T16:12:02.847 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3019000000000001e-05 2020-09-08T16:12:02.847 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.848 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.848 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:12:02.848 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.521711+0000", 2020-09-08T16:12:02.848 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8069999999999994e-06 2020-09-08T16:12:02.848 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.848 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.849 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:12:02.849 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.521721+0000", 2020-09-08T16:12:02.849 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.43898494999999998 2020-09-08T16:12:02.849 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.849 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.850 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:12:02.850 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.960706+0000", 2020-09-08T16:12:02.850 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013872799999999999 2020-09-08T16:12:02.850 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.850 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.851 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:12:02.851 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.960845+0000", 2020-09-08T16:12:02.851 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018871400000000001 2020-09-08T16:12:02.851 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.851 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.852 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:12:02.852 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.961034+0000", 2020-09-08T16:12:02.852 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.104627553 2020-09-08T16:12:02.852 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.852 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.853 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:12:02.853 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.065661+0000", 2020-09-08T16:12:02.853 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00037425099999999998 2020-09-08T16:12:02.853 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.853 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.853 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:12:02.854 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.066035+0000", 2020-09-08T16:12:02.854 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000125983 2020-09-08T16:12:02.854 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.854 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.854 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:12:02.855 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.066161+0000", 2020-09-08T16:12:02.855 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8944999999999996e-05 2020-09-08T16:12:02.855 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.855 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.855 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:12:02.856 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.066250+0000", 2020-09-08T16:12:02.856 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.54466917999999998 2020-09-08T16:12:02.856 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.856 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.856 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.857 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.857 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.857 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:123888 8.e 8:76457be3:::benchmark_data_smithi166_52302_object123887:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e656)", 2020-09-08T16:12:02.857 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:06:27.586295+0000", 2020-09-08T16:12:02.857 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 334.90324123300002, 2020-09-08T16:12:02.858 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.49297180800000001, 2020-09-08T16:12:02.858 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:12:02.858 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:12:02.858 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:12:02.858 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:12:02.858 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:12:02.859 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 123888 2020-09-08T16:12:02.859 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.859 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:12:02.859 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.859 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:12:02.860 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.586295+0000", 2020-09-08T16:12:02.860 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:12:02.860 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.860 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.860 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:12:02.861 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.586295+0000", 2020-09-08T16:12:02.861 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999766 2020-09-08T16:12:02.861 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.861 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.861 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:12:02.862 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.586272+0000", 2020-09-08T16:12:02.862 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6551999999999994e-05 2020-09-08T16:12:02.862 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.862 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.862 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:12:02.862 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.586359+0000", 2020-09-08T16:12:02.863 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2830000000000001e-05 2020-09-08T16:12:02.863 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.863 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.863 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:12:02.863 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.586371+0000", 2020-09-08T16:12:02.864 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5549999999999999e-06 2020-09-08T16:12:02.864 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.864 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.864 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:12:02.864 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:27.586378+0000", 2020-09-08T16:12:02.865 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.47403673400000002 2020-09-08T16:12:02.865 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.865 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.865 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:12:02.865 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.060415+0000", 2020-09-08T16:12:02.866 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000222986 2020-09-08T16:12:02.866 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.866 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.866 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:12:02.866 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.060638+0000", 2020-09-08T16:12:02.867 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00029462100000000001 2020-09-08T16:12:02.867 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.867 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.867 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:12:02.867 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.060932+0000", 2020-09-08T16:12:02.867 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0057477409999999998 2020-09-08T16:12:02.868 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.868 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.868 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:12:02.868 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.066680+0000", 2020-09-08T16:12:02.868 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.012502819 2020-09-08T16:12:02.869 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.869 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.869 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:12:02.869 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.079183+0000", 2020-09-08T16:12:02.869 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9733e-05 2020-09-08T16:12:02.870 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.870 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.870 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:12:02.870 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.079233+0000", 2020-09-08T16:12:02.870 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4449999999999997e-05 2020-09-08T16:12:02.871 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.871 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.871 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:12:02.871 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:28.079267+0000", 2020-09-08T16:12:02.871 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.49297180800000001 2020-09-08T16:12:02.872 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.872 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.872 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.872 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.872 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.872 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.36335.0:126481 8.e 8:789e7fac:::benchmark_data_smithi166_52302_object126480:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected e658)", 2020-09-08T16:12:02.873 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:06:31.331487+0000", 2020-09-08T16:12:02.873 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 331.15804985, 2020-09-08T16:12:02.873 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.50117400099999998, 2020-09-08T16:12:02.873 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:12:02.873 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2020-09-08T16:12:02.874 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:12:02.874 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.36335", 2020-09-08T16:12:02.874 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2942023479", 2020-09-08T16:12:02.874 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 126481 2020-09-08T16:12:02.874 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.875 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:12:02.875 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.875 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:12:02.875 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.331487+0000", 2020-09-08T16:12:02.875 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:12:02.876 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.876 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.876 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:12:02.876 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.331487+0000", 2020-09-08T16:12:02.876 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999743 2020-09-08T16:12:02.877 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.877 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.877 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:12:02.877 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.331461+0000", 2020-09-08T16:12:02.877 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7122999999999998e-05 2020-09-08T16:12:02.877 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.878 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.878 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:12:02.878 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.331558+0000", 2020-09-08T16:12:02.878 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.38e-05 2020-09-08T16:12:02.878 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.879 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.879 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:12:02.879 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.331572+0000", 2020-09-08T16:12:02.879 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1410000000000004e-06 2020-09-08T16:12:02.879 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.880 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.880 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:12:02.880 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.331577+0000", 2020-09-08T16:12:02.880 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.378500324 2020-09-08T16:12:02.880 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.881 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.881 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:12:02.881 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.710077+0000", 2020-09-08T16:12:02.881 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000109687 2020-09-08T16:12:02.881 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.881 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.882 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:12:02.882 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.710187+0000", 2020-09-08T16:12:02.882 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018271100000000001 2020-09-08T16:12:02.882 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.882 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.883 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2", 2020-09-08T16:12:02.883 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.710369+0000", 2020-09-08T16:12:02.883 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.023681298999999999 2020-09-08T16:12:02.883 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.883 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.884 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2020-09-08T16:12:02.884 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.734051+0000", 2020-09-08T16:12:02.884 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.098485091999999996 2020-09-08T16:12:02.884 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.884 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.885 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:12:02.885 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.832536+0000", 2020-09-08T16:12:02.885 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5665000000000001e-05 2020-09-08T16:12:02.885 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.885 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.886 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2020-09-08T16:12:02.886 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.832612+0000", 2020-09-08T16:12:02.886 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9141000000000003e-05 2020-09-08T16:12:02.886 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:12:02.886 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:12:02.886 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2020-09-08T16:12:02.887 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:06:31.832661+0000", 2020-09-08T16:12:02.887 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.50117400099999998 2020-09-08T16:12:02.887 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.887 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.887 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.888 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:12:02.888 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:12:02.888 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T16:12:02.888 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:02.890 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:12:02 smithi166 bash[18960]: debug 2020-09-08T16:12:02.434+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:02.890 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:12:02 smithi104 bash[22305]: debug 2020-09-08T16:12:02.204+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:03.410 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:03.514 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:03.523 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:03 smithi055 bash[9296]: cluster 2020-09-08T16:12:01.929412+0000 osd.4 (osd.4) 81 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:03.523 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:03 smithi055 bash[9296]: cluster 2020-09-08T16:12:01.929511+0000 osd.4 (osd.4) 82 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:03.524 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:03 smithi055 bash[9296]: cluster 2020-09-08T16:12:02.150323+0000 mgr.x (mgr.24118) 1341 : cluster [DBG] pgmap v2004: 17 pgs: 17 active+clean; 5.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 668 op/s 2020-09-08T16:12:03.524 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:03 smithi055 bash[9296]: cluster 2020-09-08T16:12:02.271757+0000 mon.a (mon.0) 2320 : cluster [DBG] osdmap e748: 12 total, 12 up, 12 in 2020-09-08T16:12:03.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:03 smithi166 bash[9446]: cluster 2020-09-08T16:12:01.929412+0000 osd.4 (osd.4) 81 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:03.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:03 smithi166 bash[9446]: cluster 2020-09-08T16:12:01.929511+0000 osd.4 (osd.4) 82 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:03.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:03 smithi166 bash[9446]: cluster 2020-09-08T16:12:02.150323+0000 mgr.x (mgr.24118) 1341 : cluster [DBG] pgmap v2004: 17 pgs: 17 active+clean; 5.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 668 op/s 2020-09-08T16:12:03.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:03 smithi166 bash[9446]: cluster 2020-09-08T16:12:02.271757+0000 mon.a (mon.0) 2320 : cluster [DBG] osdmap e748: 12 total, 12 up, 12 in 2020-09-08T16:12:03.655 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:03.667 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:12:03.675 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:12:03 smithi104 bash[19922]: debug 2020-09-08T16:12:03.408+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:03.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:03 smithi104 bash[9203]: cluster 2020-09-08T16:12:01.929412+0000 osd.4 (osd.4) 81 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:03.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:03 smithi104 bash[9203]: cluster 2020-09-08T16:12:01.929511+0000 osd.4 (osd.4) 82 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:03.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:03 smithi104 bash[9203]: cluster 2020-09-08T16:12:02.150323+0000 mgr.x (mgr.24118) 1341 : cluster [DBG] pgmap v2004: 17 pgs: 17 active+clean; 5.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 668 op/s 2020-09-08T16:12:03.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:03 smithi104 bash[9203]: cluster 2020-09-08T16:12:02.271757+0000 mon.a (mon.0) 2320 : cluster [DBG] osdmap e748: 12 total, 12 up, 12 in 2020-09-08T16:12:03.712 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:12:03.724 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2020-09-08T16:12:03.746 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:12:03.762 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:03.783 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:12:03.818 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:12:03.853 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:12:03.884 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:12:03.890 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:12:03.907 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:03.908 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:12:03 smithi055 bash[16646]: debug 2020-09-08T16:12:03.656+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:03.916 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:12:03.959 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:12:03.988 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:12:04.011 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:04.020 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:12:03 smithi166 bash[18960]: debug 2020-09-08T16:12:03.902+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:04.029 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:12:04.056 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:12:04.058 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:12:04.059 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:12:04.059 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:12:04.060 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:12:04.060 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:12:04.060 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:12:04.060 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:12:04.061 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:12:04.061 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:12:04.061 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:12:04.061 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:12:04.061 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:12:04.126 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:04.192 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T16:12:04.231 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:04.351 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:12:04.356 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:12:04 smithi166 bash[11928]: debug 2020-09-08T16:12:04.126+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:04.356 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:04.356 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:04 smithi166 bash[9446]: cluster 2020-09-08T16:12:03.425420+0000 osd.7 (osd.7) 165 : cluster [DBG] 9.d scrub starts 2020-09-08T16:12:04.357 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:04 smithi166 bash[9446]: audit 2020-09-08T16:12:03.639971+0000 mon.a (mon.0) 2321 : audit [DBG] from='client.? v1:172.21.15.104:0/3812821472' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:12:04.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:04 smithi104 bash[9203]: cluster 2020-09-08T16:12:03.425420+0000 osd.7 (osd.7) 165 : cluster [DBG] 9.d scrub starts 2020-09-08T16:12:04.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:04 smithi104 bash[9203]: audit 2020-09-08T16:12:03.639971+0000 mon.a (mon.0) 2321 : audit [DBG] from='client.? v1:172.21.15.104:0/3812821472' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:12:04.461 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:04.574 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:04 smithi055 bash[9296]: cluster 2020-09-08T16:12:03.425420+0000 osd.7 (osd.7) 165 : cluster [DBG] 9.d scrub starts 2020-09-08T16:12:04.574 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:04 smithi055 bash[9296]: audit 2020-09-08T16:12:03.639971+0000 mon.a (mon.0) 2321 : audit [DBG] from='client.? v1:172.21.15.104:0/3812821472' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:12:04.575 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:04.622 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:12:04 smithi166 bash[14254]: debug 2020-09-08T16:12:04.350+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:04.678 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:12:04.910 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:12:04 smithi055 bash[14301]: debug 2020-09-08T16:12:04.576+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:05.466 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:12:05.569 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:12:05.577 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:05 smithi166 bash[9446]: cluster 2020-09-08T16:12:04.150788+0000 mgr.x (mgr.24118) 1342 : cluster [DBG] pgmap v2006: 17 pgs: 17 active+clean; 5.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.27k op/s 2020-09-08T16:12:05.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:05 smithi055 bash[9296]: cluster 2020-09-08T16:12:04.150788+0000 mgr.x (mgr.24118) 1342 : cluster [DBG] pgmap v2006: 17 pgs: 17 active+clean; 5.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.27k op/s 2020-09-08T16:12:05.675 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:12:05 smithi104 bash[15273]: debug 2020-09-08T16:12:05.468+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:05.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:05 smithi104 bash[9203]: cluster 2020-09-08T16:12:04.150788+0000 mgr.x (mgr.24118) 1342 : cluster [DBG] pgmap v2006: 17 pgs: 17 active+clean; 5.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.27k op/s 2020-09-08T16:12:05.704 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:12:05.808 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:12:05.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:12:05 smithi166 bash[16593]: debug 2020-09-08T16:12:05.698+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:06.397 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:12:06.501 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:06.577 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:12:06 smithi104 bash[22305]: debug 2020-09-08T16:12:06.400+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:06.578 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:06 smithi104 bash[9203]: cluster 2020-09-08T16:12:04.930063+0000 osd.7 (osd.7) 166 : cluster [DBG] 9.d scrub ok 2020-09-08T16:12:06.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:06 smithi166 bash[9446]: cluster 2020-09-08T16:12:04.930063+0000 osd.7 (osd.7) 166 : cluster [DBG] 9.d scrub ok 2020-09-08T16:12:06.663 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:06 smithi055 bash[9296]: cluster 2020-09-08T16:12:04.930063+0000 osd.7 (osd.7) 166 : cluster [DBG] 9.d scrub ok 2020-09-08T16:12:06.667 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:06.772 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:12:06.975 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:12:07.078 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:07.088 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:12:06 smithi166 bash[14254]: debug 2020-09-08T16:12:06.666+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:07.202 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:07.266 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:12:06 smithi104 bash[22305]: debug 2020-09-08T16:12:06.976+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:07.311 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:07.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:07 smithi166 bash[9446]: cluster 2020-09-08T16:12:06.151258+0000 mgr.x (mgr.24118) 1343 : cluster [DBG] pgmap v2007: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 677 op/s 2020-09-08T16:12:07.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:12:07 smithi166 bash[11928]: debug 2020-09-08T16:12:07.194+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:07.443 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:07.513 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:07 smithi104 bash[9203]: cluster 2020-09-08T16:12:06.151258+0000 mgr.x (mgr.24118) 1343 : cluster [DBG] pgmap v2007: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 677 op/s 2020-09-08T16:12:07.550 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:07.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:07 smithi055 bash[9296]: cluster 2020-09-08T16:12:06.151258+0000 mgr.x (mgr.24118) 1343 : cluster [DBG] pgmap v2007: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 5.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 677 op/s 2020-09-08T16:12:07.675 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:07.777 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:07.792 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:12:07 smithi166 bash[11928]: debug 2020-09-08T16:12:07.442+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:07.792 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:12:07 smithi166 bash[18960]: debug 2020-09-08T16:12:07.666+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:07.913 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:08.022 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:08.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:12:07 smithi166 bash[11928]: debug 2020-09-08T16:12:07.914+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:08.150 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:08.258 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:08.374 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:08.485 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:08.579 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:12:08.616 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:12:08 smithi166 bash[14254]: debug 2020-09-08T16:12:08.150+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:08.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:12:08 smithi055 bash[16646]: debug 2020-09-08T16:12:08.372+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:08.738 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:12:09.002 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:09.109 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:12:09.286 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:12:09 smithi104 bash[17588]: debug 2020-09-08T16:12:09.004+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:09.287 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:09 smithi104 bash[9203]: cluster 2020-09-08T16:12:08.151742+0000 mgr.x (mgr.24118) 1344 : cluster [DBG] pgmap v2008: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T16:12:09.509 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:12:09.612 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:09.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:09 smithi166 bash[9446]: cluster 2020-09-08T16:12:08.151742+0000 mgr.x (mgr.24118) 1344 : cluster [DBG] pgmap v2008: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T16:12:09.620 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:12:09 smithi104 bash[22305]: debug 2020-09-08T16:12:09.512+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:09.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:09 smithi055 bash[9296]: cluster 2020-09-08T16:12:08.151742+0000 mgr.x (mgr.24118) 1344 : cluster [DBG] pgmap v2008: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.04k op/s 2020-09-08T16:12:10.241 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:10.344 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:10.805 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:12:10 smithi104 bash[17588]: debug 2020-09-08T16:12:10.244+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:10.823 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:10.924 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:11.085 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:12:10 smithi104 bash[17588]: debug 2020-09-08T16:12:10.816+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:11.085 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:11 smithi104 bash[9203]: cluster 2020-09-08T16:12:10.152290+0000 mgr.x (mgr.24118) 1345 : cluster [DBG] pgmap v2009: 17 pgs: 17 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 927 op/s 2020-09-08T16:12:11.086 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:11 smithi104 bash[9203]: cluster 2020-09-08T16:12:10.708821+0000 osd.9 (osd.9) 118 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:11.086 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:11 smithi104 bash[9203]: cluster 2020-09-08T16:12:10.708901+0000 osd.9 (osd.9) 119 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:11.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:11 smithi166 bash[9446]: cluster 2020-09-08T16:12:10.152290+0000 mgr.x (mgr.24118) 1345 : cluster [DBG] pgmap v2009: 17 pgs: 17 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 927 op/s 2020-09-08T16:12:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:11 smithi166 bash[9446]: cluster 2020-09-08T16:12:10.708821+0000 osd.9 (osd.9) 118 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:11.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:11 smithi166 bash[9446]: cluster 2020-09-08T16:12:10.708901+0000 osd.9 (osd.9) 119 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:11.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:11 smithi055 bash[9296]: cluster 2020-09-08T16:12:10.152290+0000 mgr.x (mgr.24118) 1345 : cluster [DBG] pgmap v2009: 17 pgs: 17 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 927 op/s 2020-09-08T16:12:11.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:11 smithi055 bash[9296]: cluster 2020-09-08T16:12:10.708821+0000 osd.9 (osd.9) 118 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:11.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:11 smithi055 bash[9296]: cluster 2020-09-08T16:12:10.708901+0000 osd.9 (osd.9) 119 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:11.552 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:11.658 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:12:11.797 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:12:11.920 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:12:11 smithi104 bash[17588]: debug 2020-09-08T16:12:11.556+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:12.016 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:12:12.085 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:12:11 smithi055 bash[19003]: debug 2020-09-08T16:12:11.796+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:12.086 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:12 smithi055 bash[9296]: cluster 2020-09-08T16:12:11.094118+0000 mon.a (mon.0) 2322 : cluster [DBG] osdmap e749: 12 total, 12 up, 12 in 2020-09-08T16:12:12.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:12 smithi104 bash[9203]: cluster 2020-09-08T16:12:11.094118+0000 mon.a (mon.0) 2322 : cluster [DBG] osdmap e749: 12 total, 12 up, 12 in 2020-09-08T16:12:12.277 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:12:12.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:12 smithi166 bash[9446]: cluster 2020-09-08T16:12:11.094118+0000 mon.a (mon.0) 2322 : cluster [DBG] osdmap e749: 12 total, 12 up, 12 in 2020-09-08T16:12:12.389 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:12.501 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:12.607 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:12.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:12:12 smithi055 bash[16646]: debug 2020-09-08T16:12:12.496+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:12.675 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:12:12 smithi104 bash[22305]: debug 2020-09-08T16:12:12.280+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:12.714 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:12.818 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:12.922 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:13.027 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:13.095 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:12:12 smithi055 bash[16646]: debug 2020-09-08T16:12:12.712+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:13.096 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:13 smithi055 bash[9296]: cluster 2020-09-08T16:12:12.152876+0000 mgr.x (mgr.24118) 1346 : cluster [DBG] pgmap v2011: 17 pgs: 17 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 843 op/s 2020-09-08T16:12:13.096 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:12:12 smithi055 bash[14301]: debug 2020-09-08T16:12:12.924+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:13.173 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:13.277 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:13.290 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:13 smithi104 bash[9203]: cluster 2020-09-08T16:12:12.152876+0000 mgr.x (mgr.24118) 1346 : cluster [DBG] pgmap v2011: 17 pgs: 17 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 843 op/s 2020-09-08T16:12:13.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:13 smithi166 bash[9446]: cluster 2020-09-08T16:12:12.152876+0000 mgr.x (mgr.24118) 1346 : cluster [DBG] pgmap v2011: 17 pgs: 17 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 843 op/s 2020-09-08T16:12:13.368 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:12:13 smithi166 bash[14254]: debug 2020-09-08T16:12:13.174+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:13.726 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:13.830 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:14.060 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:12:13 smithi104 bash[19922]: debug 2020-09-08T16:12:13.728+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:14.061 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:12:14 smithi104 bash[19922]: debug 2020-09-08T16:12:14.060+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:14.062 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:14.155 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T16:12:14.156 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:12:14.169 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:14.308 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:14.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:14 smithi166 bash[9446]: cluster 2020-09-08T16:12:13.122346+0000 osd.2 (osd.2) 116 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:14.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:14 smithi166 bash[9446]: cluster 2020-09-08T16:12:13.122425+0000 osd.2 (osd.2) 117 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:14.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:14 smithi166 bash[9446]: cluster 2020-09-08T16:12:13.257254+0000 osd.3 (osd.3) 135 : cluster [DBG] 9.c scrub starts 2020-09-08T16:12:14.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:12:14 smithi055 bash[16646]: debug 2020-09-08T16:12:14.304+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:14.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:14 smithi055 bash[9296]: cluster 2020-09-08T16:12:13.122346+0000 osd.2 (osd.2) 116 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:14.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:14 smithi055 bash[9296]: cluster 2020-09-08T16:12:13.122425+0000 osd.2 (osd.2) 117 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:14.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:14 smithi055 bash[9296]: cluster 2020-09-08T16:12:13.257254+0000 osd.3 (osd.3) 135 : cluster [DBG] 9.c scrub starts 2020-09-08T16:12:14.412 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:14.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:14 smithi104 bash[9203]: cluster 2020-09-08T16:12:13.122346+0000 osd.2 (osd.2) 116 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:14.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:14 smithi104 bash[9203]: cluster 2020-09-08T16:12:13.122425+0000 osd.2 (osd.2) 117 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:14.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:14 smithi104 bash[9203]: cluster 2020-09-08T16:12:13.257254+0000 osd.3 (osd.3) 135 : cluster [DBG] 9.c scrub starts 2020-09-08T16:12:14.544 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:14.651 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:14.777 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:12:14 smithi166 bash[18960]: debug 2020-09-08T16:12:14.542+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:14.778 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:14.888 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:15.018 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:15.108 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:12:14 smithi166 bash[14254]: debug 2020-09-08T16:12:14.778+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:15.120 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:15.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:12:15 smithi055 bash[16646]: debug 2020-09-08T16:12:15.016+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:15.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:15 smithi055 bash[9296]: cluster 2020-09-08T16:12:14.105224+0000 mon.a (mon.0) 2323 : cluster [DBG] osdmap e750: 12 total, 12 up, 12 in 2020-09-08T16:12:15.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:15 smithi055 bash[9296]: cluster 2020-09-08T16:12:14.153297+0000 mgr.x (mgr.24118) 1347 : cluster [DBG] pgmap v2013: 17 pgs: 17 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T16:12:15.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:15 smithi055 bash[9296]: cluster 2020-09-08T16:12:14.384589+0000 osd.7 (osd.7) 167 : cluster [DBG] 9.e scrub starts 2020-09-08T16:12:15.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:15 smithi055 bash[9296]: audit 2020-09-08T16:12:14.438912+0000 mon.a (mon.0) 2324 : audit [INF] from='client.? v1:172.21.15.104:0/1489112414' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:12:15.174 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T16:12:15.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:15 smithi166 bash[9446]: cluster 2020-09-08T16:12:14.105224+0000 mon.a (mon.0) 2323 : cluster [DBG] osdmap e750: 12 total, 12 up, 12 in 2020-09-08T16:12:15.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:15 smithi166 bash[9446]: cluster 2020-09-08T16:12:14.153297+0000 mgr.x (mgr.24118) 1347 : cluster [DBG] pgmap v2013: 17 pgs: 17 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T16:12:15.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:15 smithi166 bash[9446]: cluster 2020-09-08T16:12:14.384589+0000 osd.7 (osd.7) 167 : cluster [DBG] 9.e scrub starts 2020-09-08T16:12:15.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:15 smithi166 bash[9446]: audit 2020-09-08T16:12:14.438912+0000 mon.a (mon.0) 2324 : audit [INF] from='client.? v1:172.21.15.104:0/1489112414' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:12:15.386 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:15 smithi104 bash[9203]: cluster 2020-09-08T16:12:14.105224+0000 mon.a (mon.0) 2323 : cluster [DBG] osdmap e750: 12 total, 12 up, 12 in 2020-09-08T16:12:15.386 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:15 smithi104 bash[9203]: cluster 2020-09-08T16:12:14.153297+0000 mgr.x (mgr.24118) 1347 : cluster [DBG] pgmap v2013: 17 pgs: 17 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.13k op/s 2020-09-08T16:12:15.387 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:15 smithi104 bash[9203]: cluster 2020-09-08T16:12:14.384589+0000 osd.7 (osd.7) 167 : cluster [DBG] 9.e scrub starts 2020-09-08T16:12:15.387 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:15 smithi104 bash[9203]: audit 2020-09-08T16:12:14.438912+0000 mon.a (mon.0) 2324 : audit [INF] from='client.? v1:172.21.15.104:0/1489112414' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:12:15.676 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:15.785 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:15.924 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:15.925 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:12:15 smithi104 bash[17588]: debug 2020-09-08T16:12:15.680+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:16.030 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:16.112 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:12:15 smithi055 bash[16646]: debug 2020-09-08T16:12:15.924+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:16.158 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:16.276 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:12:16.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:16 smithi166 bash[9446]: cluster 2020-09-08T16:12:14.733555+0000 osd.3 (osd.3) 136 : cluster [DBG] 9.c scrub ok 2020-09-08T16:12:16.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:16 smithi166 bash[9446]: cluster 2020-09-08T16:12:15.110056+0000 mon.a (mon.0) 2325 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:12:16.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:16 smithi166 bash[9446]: audit 2020-09-08T16:12:15.171522+0000 mon.a (mon.0) 2326 : audit [INF] from='client.? v1:172.21.15.104:0/1489112414' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:12:16.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:16 smithi166 bash[9446]: cluster 2020-09-08T16:12:15.171575+0000 mon.a (mon.0) 2327 : cluster [DBG] osdmap e751: 12 total, 12 up, 12 in 2020-09-08T16:12:16.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:12:16 smithi055 bash[11956]: debug 2020-09-08T16:12:16.152+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:16.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:16 smithi055 bash[9296]: cluster 2020-09-08T16:12:14.733555+0000 osd.3 (osd.3) 136 : cluster [DBG] 9.c scrub ok 2020-09-08T16:12:16.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:16 smithi055 bash[9296]: cluster 2020-09-08T16:12:15.110056+0000 mon.a (mon.0) 2325 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:12:16.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:16 smithi055 bash[9296]: audit 2020-09-08T16:12:15.171522+0000 mon.a (mon.0) 2326 : audit [INF] from='client.? v1:172.21.15.104:0/1489112414' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:12:16.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:16 smithi055 bash[9296]: cluster 2020-09-08T16:12:15.171575+0000 mon.a (mon.0) 2327 : cluster [DBG] osdmap e751: 12 total, 12 up, 12 in 2020-09-08T16:12:16.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:16 smithi104 bash[9203]: cluster 2020-09-08T16:12:14.733555+0000 osd.3 (osd.3) 136 : cluster [DBG] 9.c scrub ok 2020-09-08T16:12:16.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:16 smithi104 bash[9203]: cluster 2020-09-08T16:12:15.110056+0000 mon.a (mon.0) 2325 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:12:16.429 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:16 smithi104 bash[9203]: audit 2020-09-08T16:12:15.171522+0000 mon.a (mon.0) 2326 : audit [INF] from='client.? v1:172.21.15.104:0/1489112414' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:12:16.430 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:16 smithi104 bash[9203]: cluster 2020-09-08T16:12:15.171575+0000 mon.a (mon.0) 2327 : cluster [DBG] osdmap e751: 12 total, 12 up, 12 in 2020-09-08T16:12:16.559 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2020-09-08T16:12:16.686 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:12:16.776 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:12:16.884 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:16.925 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:12:16 smithi104 bash[22305]: debug 2020-09-08T16:12:16.776+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:17.011 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:17.114 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:17.162 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:12:17 smithi055 bash[11956]: debug 2020-09-08T16:12:17.008+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:17.163 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:17 smithi055 bash[9296]: cluster 2020-09-08T16:12:16.153731+0000 mgr.x (mgr.24118) 1348 : cluster [DBG] pgmap v2015: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 843 op/s 2020-09-08T16:12:17.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:17 smithi166 bash[9446]: cluster 2020-09-08T16:12:16.153731+0000 mgr.x (mgr.24118) 1348 : cluster [DBG] pgmap v2015: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 843 op/s 2020-09-08T16:12:17.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:17 smithi104 bash[9203]: cluster 2020-09-08T16:12:16.153731+0000 mgr.x (mgr.24118) 1348 : cluster [DBG] pgmap v2015: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 843 op/s 2020-09-08T16:12:17.828 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:17.929 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:12:18.063 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:12:18.167 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:18.175 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:12:17 smithi104 bash[19922]: debug 2020-09-08T16:12:17.828+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:18.298 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:12:18 smithi055 bash[19003]: debug 2020-09-08T16:12:18.060+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:18.298 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:18.400 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:12:18.642 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:12:18.657 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:12:18 smithi055 bash[11956]: debug 2020-09-08T16:12:18.288+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:18.675 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:12:18 smithi104 bash[15273]: debug 2020-09-08T16:12:18.640+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:18.744 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:18.794 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:12:18.821 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:12:18.850 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:12:18.873 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:12:18.883 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:18.906 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:12:18.937 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:12:18.967 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:12:18.987 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:12:19.001 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:12:19.040 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:12:19.075 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:12:19.102 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:12:19.113 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:12:19.148 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:12:19.148 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:12:19.148 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:12:19.149 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:12:19.149 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:12:19.149 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:12:19.149 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:12:19.150 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:12:19.150 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:12:19.150 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:12:19.151 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:12:19.151 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:12:19.151 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:12:19.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:12:18 smithi055 bash[14301]: debug 2020-09-08T16:12:18.884+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:19.205 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:19.264 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:19 smithi104 bash[9203]: cluster 2020-09-08T16:12:18.154156+0000 mgr.x (mgr.24118) 1349 : cluster [DBG] pgmap v2016: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T16:12:19.264 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:19 smithi104 bash[9203]: audit 2020-09-08T16:12:18.757833+0000 mon.a (mon.0) 2328 : audit [DBG] from='client.? v1:172.21.15.104:0/455661035' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:12:19.265 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:12:19 smithi104 bash[22305]: debug 2020-09-08T16:12:19.104+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:19.475 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:19.580 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:19.588 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:19 smithi055 bash[9296]: cluster 2020-09-08T16:12:18.154156+0000 mgr.x (mgr.24118) 1349 : cluster [DBG] pgmap v2016: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T16:12:19.588 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:19 smithi055 bash[9296]: audit 2020-09-08T16:12:18.757833+0000 mon.a (mon.0) 2328 : audit [DBG] from='client.? v1:172.21.15.104:0/455661035' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:12:19.622 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:19 smithi166 bash[9446]: cluster 2020-09-08T16:12:18.154156+0000 mgr.x (mgr.24118) 1349 : cluster [DBG] pgmap v2016: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.24k op/s 2020-09-08T16:12:19.623 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:19 smithi166 bash[9446]: audit 2020-09-08T16:12:18.757833+0000 mon.a (mon.0) 2328 : audit [DBG] from='client.? v1:172.21.15.104:0/455661035' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:12:19.684 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:12:19 smithi104 bash[19922]: debug 2020-09-08T16:12:19.476+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:19.862 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T16:12:20.014 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:12:20.146 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:20.250 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:20.410 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:12:20 smithi055 bash[16646]: debug 2020-09-08T16:12:20.144+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:20.902 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:21.004 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:21.154 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:21.175 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:12:20 smithi104 bash[17588]: debug 2020-09-08T16:12:20.900+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:21.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:21 smithi104 bash[9203]: cluster 2020-09-08T16:12:20.154669+0000 mgr.x (mgr.24118) 1350 : cluster [DBG] pgmap v2017: 17 pgs: 17 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.21k op/s 2020-09-08T16:12:21.263 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:21.271 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:21 smithi166 bash[9446]: cluster 2020-09-08T16:12:20.154669+0000 mgr.x (mgr.24118) 1350 : cluster [DBG] pgmap v2017: 17 pgs: 17 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.21k op/s 2020-09-08T16:12:21.272 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:12:21 smithi166 bash[18960]: debug 2020-09-08T16:12:21.153+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:21.389 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:21.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:21 smithi055 bash[9296]: cluster 2020-09-08T16:12:20.154669+0000 mgr.x (mgr.24118) 1350 : cluster [DBG] pgmap v2017: 17 pgs: 17 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.21k op/s 2020-09-08T16:12:21.498 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:21.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:12:21 smithi166 bash[11928]: debug 2020-09-08T16:12:21.385+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:21.981 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:22.084 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:22.232 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:22.339 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:22.348 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:12:22 smithi104 bash[17588]: debug 2020-09-08T16:12:21.984+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:22.349 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:22 smithi104 bash[9203]: cluster 2020-09-08T16:12:21.217331+0000 osd.0 (osd.0) 132 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:22.350 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:22 smithi104 bash[9203]: cluster 2020-09-08T16:12:21.217414+0000 osd.0 (osd.0) 133 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:22.350 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:22 smithi104 bash[9203]: cluster 2020-09-08T16:12:21.360638+0000 osd.7 (osd.7) 168 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:22.350 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:22 smithi104 bash[9203]: cluster 2020-09-08T16:12:21.360706+0000 osd.7 (osd.7) 169 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:22.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:12:22 smithi166 bash[11928]: debug 2020-09-08T16:12:22.229+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:22.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:22 smithi166 bash[9446]: cluster 2020-09-08T16:12:21.217331+0000 osd.0 (osd.0) 132 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:22.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:22 smithi166 bash[9446]: cluster 2020-09-08T16:12:21.217414+0000 osd.0 (osd.0) 133 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:22.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:22 smithi166 bash[9446]: cluster 2020-09-08T16:12:21.360638+0000 osd.7 (osd.7) 168 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:22.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:22 smithi166 bash[9446]: cluster 2020-09-08T16:12:21.360706+0000 osd.7 (osd.7) 169 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:22.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:22 smithi055 bash[9296]: cluster 2020-09-08T16:12:21.217331+0000 osd.0 (osd.0) 132 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:22.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:22 smithi055 bash[9296]: cluster 2020-09-08T16:12:21.217414+0000 osd.0 (osd.0) 133 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:22.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:22 smithi055 bash[9296]: cluster 2020-09-08T16:12:21.360638+0000 osd.7 (osd.7) 168 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:22.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:22 smithi055 bash[9296]: cluster 2020-09-08T16:12:21.360706+0000 osd.7 (osd.7) 169 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:22.947 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:23.052 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:23.197 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:23.318 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:23.326 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:12:23 smithi055 bash[11956]: debug 2020-09-08T16:12:23.196+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:23.327 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:23 smithi055 bash[9296]: cluster 2020-09-08T16:12:22.096773+0000 mon.a (mon.0) 2329 : cluster [DBG] osdmap e752: 12 total, 12 up, 12 in 2020-09-08T16:12:23.327 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:23 smithi055 bash[9296]: cluster 2020-09-08T16:12:22.155170+0000 mgr.x (mgr.24118) 1351 : cluster [DBG] pgmap v2019: 17 pgs: 17 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 654 op/s 2020-09-08T16:12:23.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:23 smithi166 bash[9446]: cluster 2020-09-08T16:12:22.096773+0000 mon.a (mon.0) 2329 : cluster [DBG] osdmap e752: 12 total, 12 up, 12 in 2020-09-08T16:12:23.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:23 smithi166 bash[9446]: cluster 2020-09-08T16:12:22.155170+0000 mgr.x (mgr.24118) 1351 : cluster [DBG] pgmap v2019: 17 pgs: 17 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 654 op/s 2020-09-08T16:12:23.420 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:23 smithi104 bash[9203]: cluster 2020-09-08T16:12:22.096773+0000 mon.a (mon.0) 2329 : cluster [DBG] osdmap e752: 12 total, 12 up, 12 in 2020-09-08T16:12:23.421 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:23 smithi104 bash[9203]: cluster 2020-09-08T16:12:22.155170+0000 mgr.x (mgr.24118) 1351 : cluster [DBG] pgmap v2019: 17 pgs: 17 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 654 op/s 2020-09-08T16:12:23.421 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:12:22 smithi104 bash[19922]: debug 2020-09-08T16:12:22.948+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:23.435 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:23.547 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:23.657 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:12:23 smithi055 bash[14301]: debug 2020-09-08T16:12:23.436+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:23.665 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:23.774 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:23.925 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:23.960 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:12:24.017 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T16:12:24.017 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T16:12:24.018 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2020-09-08T16:12:24.018 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2020-09-08T16:12:24.020 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T16:12:24.028 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:24.037 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:12:23 smithi055 bash[16646]: debug 2020-09-08T16:12:23.664+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:24.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:12:23 smithi166 bash[14254]: debug 2020-09-08T16:12:23.921+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:24.130 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:12:24.150 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:24.262 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:24.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:12:24 smithi055 bash[11956]: debug 2020-09-08T16:12:24.152+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:25.100 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:25.204 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:12:25.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:12:25 smithi104 bash[19922]: debug 2020-09-08T16:12:25.100+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:25.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:25 smithi104 bash[9203]: cluster 2020-09-08T16:12:24.156128+0000 mgr.x (mgr.24118) 1352 : cluster [DBG] pgmap v2020: 17 pgs: 17 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 999 op/s 2020-09-08T16:12:25.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:25 smithi166 bash[9446]: cluster 2020-09-08T16:12:24.156128+0000 mgr.x (mgr.24118) 1352 : cluster [DBG] pgmap v2020: 17 pgs: 17 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 999 op/s 2020-09-08T16:12:25.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:25 smithi055 bash[9296]: cluster 2020-09-08T16:12:24.156128+0000 mgr.x (mgr.24118) 1352 : cluster [DBG] pgmap v2020: 17 pgs: 17 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 999 op/s 2020-09-08T16:12:26.462 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2020-09-08T16:12:26.607 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:12:27.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:27 smithi166 bash[9446]: cluster 2020-09-08T16:12:26.156635+0000 mgr.x (mgr.24118) 1353 : cluster [DBG] pgmap v2021: 17 pgs: 17 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 959 op/s 2020-09-08T16:12:27.734 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:27 smithi104 bash[9203]: cluster 2020-09-08T16:12:26.156635+0000 mgr.x (mgr.24118) 1353 : cluster [DBG] pgmap v2021: 17 pgs: 17 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 959 op/s 2020-09-08T16:12:27.774 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:27 smithi055 bash[9296]: cluster 2020-09-08T16:12:26.156635+0000 mgr.x (mgr.24118) 1353 : cluster [DBG] pgmap v2021: 17 pgs: 17 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 959 op/s 2020-09-08T16:12:28.377 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:12:28.481 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:28.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:12:28 smithi055 bash[19003]: debug 2020-09-08T16:12:28.372+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:29.053 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:29.157 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:12:29.300 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:12:29.403 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:12:29.411 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:12:29 smithi104 bash[19922]: debug 2020-09-08T16:12:29.056+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:29.412 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:29 smithi104 bash[9203]: cluster 2020-09-08T16:12:28.157128+0000 mgr.x (mgr.24118) 1354 : cluster [DBG] pgmap v2022: 17 pgs: 17 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 973 op/s 2020-09-08T16:12:29.598 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:12:29.626 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:29 smithi166 bash[9446]: cluster 2020-09-08T16:12:28.157128+0000 mgr.x (mgr.24118) 1354 : cluster [DBG] pgmap v2022: 17 pgs: 17 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 973 op/s 2020-09-08T16:12:29.632 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:12:29 smithi166 bash[16593]: debug 2020-09-08T16:12:29.297+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:29.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:29 smithi055 bash[9296]: cluster 2020-09-08T16:12:28.157128+0000 mgr.x (mgr.24118) 1354 : cluster [DBG] pgmap v2022: 17 pgs: 17 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 973 op/s 2020-09-08T16:12:29.683 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:12:29 smithi104 bash[15273]: debug 2020-09-08T16:12:29.600+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:29.703 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:29.853 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:29.963 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:30.085 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:30.158 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:12:29 smithi055 bash[14301]: debug 2020-09-08T16:12:29.851+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:30.198 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:30.242 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:12:30 smithi166 bash[11928]: debug 2020-09-08T16:12:30.081+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:30.254 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T16:12:30.309 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:30.422 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:30.539 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:12:30 smithi055 bash[14301]: debug 2020-09-08T16:12:30.307+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:30.540 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:30 smithi055 bash[9296]: audit 2020-09-08T16:12:29.752591+0000 mon.a (mon.0) 2330 : audit [INF] from='client.? v1:172.21.15.104:0/2831506529' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:12:30.541 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:30.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:30 smithi166 bash[9446]: audit 2020-09-08T16:12:29.752591+0000 mon.a (mon.0) 2330 : audit [INF] from='client.? v1:172.21.15.104:0/2831506529' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:12:30.644 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:30.652 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:30 smithi104 bash[9203]: audit 2020-09-08T16:12:29.752591+0000 mon.a (mon.0) 2330 : audit [INF] from='client.? v1:172.21.15.104:0/2831506529' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:12:30.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:12:30 smithi055 bash[14301]: debug 2020-09-08T16:12:30.539+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:31.410 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:31.518 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:31.526 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:31 smithi166 bash[9446]: cluster 2020-09-08T16:12:30.157679+0000 mgr.x (mgr.24118) 1355 : cluster [DBG] pgmap v2023: 17 pgs: 17 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 970 op/s 2020-09-08T16:12:31.527 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:31 smithi166 bash[9446]: cluster 2020-09-08T16:12:30.238797+0000 mon.a (mon.0) 2331 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:12:31.527 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:31 smithi166 bash[9446]: audit 2020-09-08T16:12:30.256870+0000 mon.a (mon.0) 2332 : audit [INF] from='client.? v1:172.21.15.104:0/2831506529' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:12:31.527 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:31 smithi166 bash[9446]: cluster 2020-09-08T16:12:30.256924+0000 mon.a (mon.0) 2333 : cluster [DBG] osdmap e753: 12 total, 12 up, 12 in 2020-09-08T16:12:31.527 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:31 smithi166 bash[9446]: cluster 2020-09-08T16:12:30.596300+0000 osd.5 (osd.5) 132 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:31.528 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:31 smithi166 bash[9446]: cluster 2020-09-08T16:12:30.596418+0000 osd.5 (osd.5) 133 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:31.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:31 smithi055 bash[9296]: cluster 2020-09-08T16:12:30.157679+0000 mgr.x (mgr.24118) 1355 : cluster [DBG] pgmap v2023: 17 pgs: 17 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 970 op/s 2020-09-08T16:12:31.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:31 smithi055 bash[9296]: cluster 2020-09-08T16:12:30.238797+0000 mon.a (mon.0) 2331 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:12:31.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:31 smithi055 bash[9296]: audit 2020-09-08T16:12:30.256870+0000 mon.a (mon.0) 2332 : audit [INF] from='client.? v1:172.21.15.104:0/2831506529' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:12:31.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:31 smithi055 bash[9296]: cluster 2020-09-08T16:12:30.256924+0000 mon.a (mon.0) 2333 : cluster [DBG] osdmap e753: 12 total, 12 up, 12 in 2020-09-08T16:12:31.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:31 smithi055 bash[9296]: cluster 2020-09-08T16:12:30.596300+0000 osd.5 (osd.5) 132 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:31.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:31 smithi055 bash[9296]: cluster 2020-09-08T16:12:30.596418+0000 osd.5 (osd.5) 133 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:31.675 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:12:31 smithi104 bash[17588]: debug 2020-09-08T16:12:31.412+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:31.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:31 smithi104 bash[9203]: cluster 2020-09-08T16:12:30.157679+0000 mgr.x (mgr.24118) 1355 : cluster [DBG] pgmap v2023: 17 pgs: 17 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 970 op/s 2020-09-08T16:12:31.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:31 smithi104 bash[9203]: cluster 2020-09-08T16:12:30.238797+0000 mon.a (mon.0) 2331 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:12:31.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:31 smithi104 bash[9203]: audit 2020-09-08T16:12:30.256870+0000 mon.a (mon.0) 2332 : audit [INF] from='client.? v1:172.21.15.104:0/2831506529' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:12:31.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:31 smithi104 bash[9203]: cluster 2020-09-08T16:12:30.256924+0000 mon.a (mon.0) 2333 : cluster [DBG] osdmap e753: 12 total, 12 up, 12 in 2020-09-08T16:12:31.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:31 smithi104 bash[9203]: cluster 2020-09-08T16:12:30.596300+0000 osd.5 (osd.5) 132 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:31.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:31 smithi104 bash[9203]: cluster 2020-09-08T16:12:30.596418+0000 osd.5 (osd.5) 133 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:31.680 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:31.786 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:31.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:12:31 smithi166 bash[11928]: debug 2020-09-08T16:12:31.673+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:32.145 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:32.250 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:32.385 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:32.427 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:12:32 smithi104 bash[19922]: debug 2020-09-08T16:12:32.148+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:32.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:32 smithi104 bash[9203]: cluster 2020-09-08T16:12:31.258812+0000 mon.a (mon.0) 2334 : cluster [DBG] osdmap e754: 12 total, 12 up, 12 in 2020-09-08T16:12:32.490 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:32.612 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:32 smithi166 bash[9446]: cluster 2020-09-08T16:12:31.258812+0000 mon.a (mon.0) 2334 : cluster [DBG] osdmap e754: 12 total, 12 up, 12 in 2020-09-08T16:12:32.613 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:12:32 smithi166 bash[18960]: debug 2020-09-08T16:12:32.381+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:32.613 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:12:32 smithi166 bash[11928]: debug 2020-09-08T16:12:32.613+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:32.617 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:32.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:32 smithi055 bash[9296]: cluster 2020-09-08T16:12:31.258812+0000 mon.a (mon.0) 2334 : cluster [DBG] osdmap e754: 12 total, 12 up, 12 in 2020-09-08T16:12:32.721 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:32.989 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:33.094 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:33.242 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:33.285 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:12:32 smithi104 bash[19922]: debug 2020-09-08T16:12:32.992+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:33.386 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:12:33.394 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:12:33 smithi055 bash[16646]: debug 2020-09-08T16:12:33.239+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:33.394 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:12:33.395 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:33 smithi055 bash[9296]: cluster 2020-09-08T16:12:32.158175+0000 mgr.x (mgr.24118) 1356 : cluster [DBG] pgmap v2026: 17 pgs: 17 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 666 op/s 2020-09-08T16:12:33.422 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:12:33.458 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:12:33.485 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:12:33.502 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:12:33.511 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:12:33.542 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:12:33.580 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:12:33.614 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:12:33.616 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:12:33.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:33 smithi166 bash[9446]: cluster 2020-09-08T16:12:32.158175+0000 mgr.x (mgr.24118) 1356 : cluster [DBG] pgmap v2026: 17 pgs: 17 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 666 op/s 2020-09-08T16:12:33.635 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:33 smithi104 bash[9203]: cluster 2020-09-08T16:12:32.158175+0000 mgr.x (mgr.24118) 1356 : cluster [DBG] pgmap v2026: 17 pgs: 17 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 666 op/s 2020-09-08T16:12:33.651 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:12:33.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:12:33 smithi055 bash[19003]: debug 2020-09-08T16:12:33.499+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:33.687 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:12:33.728 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:12:33.760 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:12:33.761 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:12:33.761 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:12:33.762 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:12:33.762 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:12:33.762 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:12:33.762 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:12:33.763 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:12:33.763 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:12:33.763 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:12:33.763 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:12:33.763 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:12:33.763 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:12:33.920 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:12:34.023 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:34.135 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:34.175 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:12:33 smithi104 bash[15273]: debug 2020-09-08T16:12:33.920+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:34.196 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T16:12:34.240 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:12:34.352 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:12:34.382 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:12:34.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:34 smithi055 bash[9296]: audit 2020-09-08T16:12:33.352681+0000 mon.a (mon.0) 2335 : audit [DBG] from='client.? v1:172.21.15.104:0/618568798' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:12:34.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:34 smithi055 bash[9296]: cluster 2020-09-08T16:12:33.603519+0000 osd.10 (osd.10) 113 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:34.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:34 smithi055 bash[9296]: cluster 2020-09-08T16:12:33.603613+0000 osd.10 (osd.10) 114 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:34.408 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:12:34 smithi055 bash[16646]: debug 2020-09-08T16:12:34.135+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:34.520 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:12:34.531 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:34 smithi104 bash[9203]: audit 2020-09-08T16:12:33.352681+0000 mon.a (mon.0) 2335 : audit [DBG] from='client.? v1:172.21.15.104:0/618568798' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:12:34.531 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:34 smithi104 bash[9203]: cluster 2020-09-08T16:12:33.603519+0000 osd.10 (osd.10) 113 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:34.532 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:34 smithi104 bash[9203]: cluster 2020-09-08T16:12:33.603613+0000 osd.10 (osd.10) 114 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:34.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:34 smithi166 bash[9446]: audit 2020-09-08T16:12:33.352681+0000 mon.a (mon.0) 2335 : audit [DBG] from='client.? v1:172.21.15.104:0/618568798' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:12:34.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:34 smithi166 bash[9446]: cluster 2020-09-08T16:12:33.603519+0000 osd.10 (osd.10) 113 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:34.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:34 smithi166 bash[9446]: cluster 2020-09-08T16:12:33.603613+0000 osd.10 (osd.10) 114 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:34.617 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:12:34 smithi166 bash[16593]: debug 2020-09-08T16:12:34.373+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:35.162 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:12:35.266 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:35.405 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:35.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:12:35 smithi104 bash[15273]: debug 2020-09-08T16:12:35.164+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:35.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:35 smithi104 bash[9203]: cluster 2020-09-08T16:12:34.158709+0000 mgr.x (mgr.24118) 1357 : cluster [DBG] pgmap v2027: 17 pgs: 17 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T16:12:35.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:35 smithi104 bash[9203]: cluster 2020-09-08T16:12:34.271698+0000 mon.a (mon.0) 2336 : cluster [DBG] osdmap e755: 12 total, 12 up, 12 in 2020-09-08T16:12:35.508 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:35.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:12:35 smithi166 bash[11928]: debug 2020-09-08T16:12:35.405+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:35.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:35 smithi166 bash[9446]: cluster 2020-09-08T16:12:34.158709+0000 mgr.x (mgr.24118) 1357 : cluster [DBG] pgmap v2027: 17 pgs: 17 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T16:12:35.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:35 smithi166 bash[9446]: cluster 2020-09-08T16:12:34.271698+0000 mon.a (mon.0) 2336 : cluster [DBG] osdmap e755: 12 total, 12 up, 12 in 2020-09-08T16:12:35.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:35 smithi055 bash[9296]: cluster 2020-09-08T16:12:34.158709+0000 mgr.x (mgr.24118) 1357 : cluster [DBG] pgmap v2027: 17 pgs: 17 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T16:12:35.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:35 smithi055 bash[9296]: cluster 2020-09-08T16:12:34.271698+0000 mon.a (mon.0) 2336 : cluster [DBG] osdmap e755: 12 total, 12 up, 12 in 2020-09-08T16:12:36.014 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:36.122 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:36.222 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T16:12:36.223 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:12:36.264 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:36.279 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:12:36 smithi104 bash[17588]: debug 2020-09-08T16:12:36.016+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:36.280 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:36 smithi104 bash[9203]: cluster 2020-09-08T16:12:34.782781+0000 osd.3 (osd.3) 137 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:36.280 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:36 smithi104 bash[9203]: cluster 2020-09-08T16:12:34.782876+0000 osd.3 (osd.3) 138 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:36.281 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:36 smithi104 bash[9203]: cluster 2020-09-08T16:12:35.283818+0000 mon.a (mon.0) 2337 : cluster [DBG] osdmap e756: 12 total, 12 up, 12 in 2020-09-08T16:12:36.370 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:36.484 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:12:36 smithi166 bash[11928]: debug 2020-09-08T16:12:36.257+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:36.485 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:36 smithi166 bash[9446]: cluster 2020-09-08T16:12:34.782781+0000 osd.3 (osd.3) 137 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:36.485 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:36 smithi166 bash[9446]: cluster 2020-09-08T16:12:34.782876+0000 osd.3 (osd.3) 138 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:36.485 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:36 smithi166 bash[9446]: cluster 2020-09-08T16:12:35.283818+0000 mon.a (mon.0) 2337 : cluster [DBG] osdmap e756: 12 total, 12 up, 12 in 2020-09-08T16:12:36.487 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:36.592 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:36.601 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:36 smithi055 bash[9296]: cluster 2020-09-08T16:12:34.782781+0000 osd.3 (osd.3) 137 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:36.601 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:36 smithi055 bash[9296]: cluster 2020-09-08T16:12:34.782876+0000 osd.3 (osd.3) 138 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:36.602 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:36 smithi055 bash[9296]: cluster 2020-09-08T16:12:35.283818+0000 mon.a (mon.0) 2337 : cluster [DBG] osdmap e756: 12 total, 12 up, 12 in 2020-09-08T16:12:36.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:12:36 smithi166 bash[11928]: debug 2020-09-08T16:12:36.485+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:37.543 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:37 smithi104 bash[9203]: cluster 2020-09-08T16:12:36.159186+0000 mgr.x (mgr.24118) 1358 : cluster [DBG] pgmap v2030: 17 pgs: 17 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 828 op/s 2020-09-08T16:12:37.545 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2020-09-08T16:12:37.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:37 smithi166 bash[9446]: cluster 2020-09-08T16:12:36.159186+0000 mgr.x (mgr.24118) 1358 : cluster [DBG] pgmap v2030: 17 pgs: 17 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 828 op/s 2020-09-08T16:12:37.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:37 smithi055 bash[9296]: cluster 2020-09-08T16:12:36.159186+0000 mgr.x (mgr.24118) 1358 : cluster [DBG] pgmap v2030: 17 pgs: 17 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 828 op/s 2020-09-08T16:12:37.706 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:12:37.720 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:37.825 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:37.952 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:38.023 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:12:38.060 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:38.069 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:12:37 smithi055 bash[16646]: debug 2020-09-08T16:12:37.719+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:38.069 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:12:37 smithi055 bash[16646]: debug 2020-09-08T16:12:37.951+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:38.176 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:12:38.178 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:38.290 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:38.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:12:38 smithi055 bash[11956]: debug 2020-09-08T16:12:38.179+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:38.906 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:39.010 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:39.121 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:39.175 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:12:38 smithi104 bash[19922]: debug 2020-09-08T16:12:38.912+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:39.226 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:12:39.292 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:39 smithi055 bash[9296]: cluster 2020-09-08T16:12:38.159618+0000 mgr.x (mgr.24118) 1359 : cluster [DBG] pgmap v2031: 17 pgs: 17 active+clean; 3.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.23k op/s 2020-09-08T16:12:39.293 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:12:39 smithi055 bash[14301]: debug 2020-09-08T16:12:39.119+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:39.354 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:12:39.461 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:39.605 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:39 smithi166 bash[9446]: cluster 2020-09-08T16:12:38.159618+0000 mgr.x (mgr.24118) 1359 : cluster [DBG] pgmap v2031: 17 pgs: 17 active+clean; 3.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.23k op/s 2020-09-08T16:12:39.606 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:39.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:12:39 smithi055 bash[19003]: debug 2020-09-08T16:12:39.355+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:39.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:39 smithi104 bash[9203]: cluster 2020-09-08T16:12:38.159618+0000 mgr.x (mgr.24118) 1359 : cluster [DBG] pgmap v2031: 17 pgs: 17 active+clean; 3.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.23k op/s 2020-09-08T16:12:39.712 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:39.838 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:39.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:12:39 smithi166 bash[14254]: debug 2020-09-08T16:12:39.833+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:39.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:12:39 smithi166 bash[18960]: debug 2020-09-08T16:12:39.605+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:39.939 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:40.063 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:40.169 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:40.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:12:40 smithi055 bash[14301]: debug 2020-09-08T16:12:40.063+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:40.898 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:41.004 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:41.155 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:41.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:41 smithi104 bash[9203]: cluster 2020-09-08T16:12:40.160083+0000 mgr.x (mgr.24118) 1360 : cluster [DBG] pgmap v2032: 17 pgs: 17 active+clean; 2.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.16k op/s 2020-09-08T16:12:41.178 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:12:40 smithi104 bash[19922]: debug 2020-09-08T16:12:40.896+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:41.265 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:12:41.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:41 smithi166 bash[9446]: cluster 2020-09-08T16:12:40.160083+0000 mgr.x (mgr.24118) 1360 : cluster [DBG] pgmap v2032: 17 pgs: 17 active+clean; 2.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.16k op/s 2020-09-08T16:12:41.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:12:41 smithi166 bash[18960]: debug 2020-09-08T16:12:41.149+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:41.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:41 smithi055 bash[9296]: cluster 2020-09-08T16:12:40.160083+0000 mgr.x (mgr.24118) 1360 : cluster [DBG] pgmap v2032: 17 pgs: 17 active+clean; 2.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.16k op/s 2020-09-08T16:12:41.412 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:12:41.519 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:41.648 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:41.756 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:12:41.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:12:41 smithi055 bash[19003]: debug 2020-09-08T16:12:41.407+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:41.908 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:12:41 smithi055 bash[11956]: debug 2020-09-08T16:12:41.647+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:41.914 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:12:42.020 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:42.088 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:42 smithi166 bash[9446]: cluster 2020-09-08T16:12:41.421494+0000 osd.8 (osd.8) 110 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:42.088 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:42 smithi166 bash[9446]: cluster 2020-09-08T16:12:41.421578+0000 osd.8 (osd.8) 111 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:42.089 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:12:41 smithi166 bash[16593]: debug 2020-09-08T16:12:41.909+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:42.140 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:42.244 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:12:42.351 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:42 smithi055 bash[9296]: cluster 2020-09-08T16:12:41.421494+0000 osd.8 (osd.8) 110 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:42.352 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:42 smithi055 bash[9296]: cluster 2020-09-08T16:12:41.421578+0000 osd.8 (osd.8) 111 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:42.353 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:12:42 smithi055 bash[16646]: debug 2020-09-08T16:12:42.139+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:42.353 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:12:42.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:42 smithi104 bash[9203]: cluster 2020-09-08T16:12:41.421494+0000 osd.8 (osd.8) 110 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:42.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:42 smithi104 bash[9203]: cluster 2020-09-08T16:12:41.421578+0000 osd.8 (osd.8) 111 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:42.464 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:42.589 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:42.657 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:12:42 smithi055 bash[19003]: debug 2020-09-08T16:12:42.351+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:42.701 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:42.818 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:42.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:12:42 smithi166 bash[11928]: debug 2020-09-08T16:12:42.585+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:42.926 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:43.078 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:43.114 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T16:12:43.115 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:12:42 smithi055 bash[11956]: debug 2020-09-08T16:12:42.815+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:43.183 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:12:43.192 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:12:43 smithi166 bash[11928]: debug 2020-09-08T16:12:43.069+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:43.192 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:43 smithi166 bash[9446]: cluster 2020-09-08T16:12:42.099502+0000 mon.a (mon.0) 2338 : cluster [DBG] osdmap e757: 12 total, 12 up, 12 in 2020-09-08T16:12:43.192 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:43 smithi166 bash[9446]: cluster 2020-09-08T16:12:42.160493+0000 mgr.x (mgr.24118) 1361 : cluster [DBG] pgmap v2034: 17 pgs: 17 active+clean; 2.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 632 op/s 2020-09-08T16:12:43.193 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:43 smithi166 bash[9446]: audit 2020-09-08T16:12:42.821488+0000 mon.a (mon.0) 2339 : audit [INF] from='client.? v1:172.21.15.104:0/1911984295' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:12:43.304 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:12:43.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:43 smithi055 bash[9296]: cluster 2020-09-08T16:12:42.099502+0000 mon.a (mon.0) 2338 : cluster [DBG] osdmap e757: 12 total, 12 up, 12 in 2020-09-08T16:12:43.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:43 smithi055 bash[9296]: cluster 2020-09-08T16:12:42.160493+0000 mgr.x (mgr.24118) 1361 : cluster [DBG] pgmap v2034: 17 pgs: 17 active+clean; 2.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 632 op/s 2020-09-08T16:12:43.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:43 smithi055 bash[9296]: audit 2020-09-08T16:12:42.821488+0000 mon.a (mon.0) 2339 : audit [INF] from='client.? v1:172.21.15.104:0/1911984295' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:12:43.409 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:43.416 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:43 smithi104 bash[9203]: cluster 2020-09-08T16:12:42.099502+0000 mon.a (mon.0) 2338 : cluster [DBG] osdmap e757: 12 total, 12 up, 12 in 2020-09-08T16:12:43.416 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:43 smithi104 bash[9203]: cluster 2020-09-08T16:12:42.160493+0000 mgr.x (mgr.24118) 1361 : cluster [DBG] pgmap v2034: 17 pgs: 17 active+clean; 2.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 632 op/s 2020-09-08T16:12:43.417 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:43 smithi104 bash[9203]: audit 2020-09-08T16:12:42.821488+0000 mon.a (mon.0) 2339 : audit [INF] from='client.? v1:172.21.15.104:0/1911984295' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:12:43.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:12:43 smithi166 bash[16593]: debug 2020-09-08T16:12:43.297+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:43.685 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:43.791 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:12:44.104 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:12:43 smithi104 bash[19922]: debug 2020-09-08T16:12:43.688+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:44.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:44 smithi166 bash[9446]: cluster 2020-09-08T16:12:43.098308+0000 mon.a (mon.0) 2340 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:12:44.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:44 smithi166 bash[9446]: audit 2020-09-08T16:12:43.110444+0000 mon.a (mon.0) 2341 : audit [INF] from='client.? v1:172.21.15.104:0/1911984295' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:12:44.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:44 smithi166 bash[9446]: cluster 2020-09-08T16:12:43.110482+0000 mon.a (mon.0) 2342 : cluster [DBG] osdmap e758: 12 total, 12 up, 12 in 2020-09-08T16:12:44.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:44 smithi166 bash[9446]: cluster 2020-09-08T16:12:43.899900+0000 osd.10 (osd.10) 115 : cluster [DBG] 9.9 scrub starts 2020-09-08T16:12:44.402 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:44 smithi104 bash[9203]: cluster 2020-09-08T16:12:43.098308+0000 mon.a (mon.0) 2340 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:12:44.403 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:44 smithi104 bash[9203]: audit 2020-09-08T16:12:43.110444+0000 mon.a (mon.0) 2341 : audit [INF] from='client.? v1:172.21.15.104:0/1911984295' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:12:44.403 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:44 smithi104 bash[9203]: cluster 2020-09-08T16:12:43.110482+0000 mon.a (mon.0) 2342 : cluster [DBG] osdmap e758: 12 total, 12 up, 12 in 2020-09-08T16:12:44.403 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:44 smithi104 bash[9203]: cluster 2020-09-08T16:12:43.899900+0000 osd.10 (osd.10) 115 : cluster [DBG] 9.9 scrub starts 2020-09-08T16:12:44.404 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:12:44 smithi104 bash[15273]: debug 2020-09-08T16:12:44.404+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:44.406 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:12:44.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:44 smithi055 bash[9296]: cluster 2020-09-08T16:12:43.098308+0000 mon.a (mon.0) 2340 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:12:44.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:44 smithi055 bash[9296]: audit 2020-09-08T16:12:43.110444+0000 mon.a (mon.0) 2341 : audit [INF] from='client.? v1:172.21.15.104:0/1911984295' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:12:44.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:44 smithi055 bash[9296]: cluster 2020-09-08T16:12:43.110482+0000 mon.a (mon.0) 2342 : cluster [DBG] osdmap e758: 12 total, 12 up, 12 in 2020-09-08T16:12:44.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:44 smithi055 bash[9296]: cluster 2020-09-08T16:12:43.899900+0000 osd.10 (osd.10) 115 : cluster [DBG] 9.9 scrub starts 2020-09-08T16:12:44.512 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:44.629 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:44.734 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:44.862 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:12:44 smithi055 bash[11956]: debug 2020-09-08T16:12:44.859+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:44.862 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:12:44 smithi055 bash[14301]: debug 2020-09-08T16:12:44.627+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:44.863 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:44.964 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:12:45.111 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:45 smithi055 bash[9296]: cluster 2020-09-08T16:12:44.068430+0000 osd.5 (osd.5) 134 : cluster [DBG] 9.f scrub starts 2020-09-08T16:12:45.111 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:45 smithi055 bash[9296]: cluster 2020-09-08T16:12:44.160932+0000 mgr.x (mgr.24118) 1362 : cluster [DBG] pgmap v2036: 17 pgs: 17 active+clean; 2.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T16:12:45.111 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:45 smithi055 bash[9296]: cluster 2020-09-08T16:12:44.834905+0000 osd.5 (osd.5) 135 : cluster [DBG] 9.f scrub ok 2020-09-08T16:12:45.112 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:45 smithi055 bash[9296]: cluster 2020-09-08T16:12:44.888505+0000 osd.10 (osd.10) 116 : cluster [DBG] 9.9 scrub ok 2020-09-08T16:12:45.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:45 smithi104 bash[9203]: cluster 2020-09-08T16:12:44.068430+0000 osd.5 (osd.5) 134 : cluster [DBG] 9.f scrub starts 2020-09-08T16:12:45.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:45 smithi104 bash[9203]: cluster 2020-09-08T16:12:44.160932+0000 mgr.x (mgr.24118) 1362 : cluster [DBG] pgmap v2036: 17 pgs: 17 active+clean; 2.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T16:12:45.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:45 smithi104 bash[9203]: cluster 2020-09-08T16:12:44.834905+0000 osd.5 (osd.5) 135 : cluster [DBG] 9.f scrub ok 2020-09-08T16:12:45.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:45 smithi104 bash[9203]: cluster 2020-09-08T16:12:44.888505+0000 osd.10 (osd.10) 116 : cluster [DBG] 9.9 scrub ok 2020-09-08T16:12:45.506 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:12:45.610 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:45.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:45 smithi166 bash[9446]: cluster 2020-09-08T16:12:44.068430+0000 osd.5 (osd.5) 134 : cluster [DBG] 9.f scrub starts 2020-09-08T16:12:45.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:45 smithi166 bash[9446]: cluster 2020-09-08T16:12:44.160932+0000 mgr.x (mgr.24118) 1362 : cluster [DBG] pgmap v2036: 17 pgs: 17 active+clean; 2.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.10k op/s 2020-09-08T16:12:45.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:45 smithi166 bash[9446]: cluster 2020-09-08T16:12:44.834905+0000 osd.5 (osd.5) 135 : cluster [DBG] 9.f scrub ok 2020-09-08T16:12:45.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:45 smithi166 bash[9446]: cluster 2020-09-08T16:12:44.888505+0000 osd.10 (osd.10) 116 : cluster [DBG] 9.9 scrub ok 2020-09-08T16:12:45.925 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:12:45 smithi104 bash[22305]: debug 2020-09-08T16:12:45.508+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:46.187 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:46.291 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:46.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:46 smithi166 bash[9446]: cluster 2020-09-08T16:12:45.087643+0000 osd.1 (osd.1) 154 : cluster [DBG] 9.4 scrub starts 2020-09-08T16:12:46.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:46 smithi166 bash[9446]: cluster 2020-09-08T16:12:45.226054+0000 osd.3 (osd.3) 139 : cluster [DBG] 9.1 scrub starts 2020-09-08T16:12:46.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:46 smithi166 bash[9446]: cluster 2020-09-08T16:12:45.268011+0000 osd.11 (osd.11) 136 : cluster [DBG] 9.5 scrub starts 2020-09-08T16:12:46.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:46 smithi055 bash[9296]: cluster 2020-09-08T16:12:45.087643+0000 osd.1 (osd.1) 154 : cluster [DBG] 9.4 scrub starts 2020-09-08T16:12:46.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:46 smithi055 bash[9296]: cluster 2020-09-08T16:12:45.226054+0000 osd.3 (osd.3) 139 : cluster [DBG] 9.1 scrub starts 2020-09-08T16:12:46.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:46 smithi055 bash[9296]: cluster 2020-09-08T16:12:45.268011+0000 osd.11 (osd.11) 136 : cluster [DBG] 9.5 scrub starts 2020-09-08T16:12:46.436 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:46.457 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:12:46 smithi104 bash[17588]: debug 2020-09-08T16:12:46.192+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:46.458 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:46 smithi104 bash[9203]: cluster 2020-09-08T16:12:45.087643+0000 osd.1 (osd.1) 154 : cluster [DBG] 9.4 scrub starts 2020-09-08T16:12:46.458 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:46 smithi104 bash[9203]: cluster 2020-09-08T16:12:45.226054+0000 osd.3 (osd.3) 139 : cluster [DBG] 9.1 scrub starts 2020-09-08T16:12:46.459 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:46 smithi104 bash[9203]: cluster 2020-09-08T16:12:45.268011+0000 osd.11 (osd.11) 136 : cluster [DBG] 9.5 scrub starts 2020-09-08T16:12:46.541 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:46.673 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:46.780 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:46.866 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:12:46 smithi166 bash[14254]: debug 2020-09-08T16:12:46.673+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:46.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:12:46 smithi166 bash[18960]: debug 2020-09-08T16:12:46.437+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:47.119 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:12:47 smithi104 bash[19922]: debug 2020-09-08T16:12:47.124+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:47.126 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:47.229 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:47.251 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:12:47.253 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T16:12:47.254 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:12:47.286 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:12:47.334 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:12:47.359 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:12:47.364 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:47.371 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:47 smithi104 bash[9203]: cluster 2020-09-08T16:12:45.729460+0000 osd.1 (osd.1) 155 : cluster [DBG] 9.4 scrub ok 2020-09-08T16:12:47.371 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:47 smithi104 bash[9203]: cluster 2020-09-08T16:12:46.115232+0000 osd.11 (osd.11) 137 : cluster [DBG] 9.5 scrub ok 2020-09-08T16:12:47.371 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:47 smithi104 bash[9203]: cluster 2020-09-08T16:12:46.144895+0000 osd.3 (osd.3) 140 : cluster [DBG] 9.1 scrub ok 2020-09-08T16:12:47.372 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:47 smithi104 bash[9203]: cluster 2020-09-08T16:12:46.161368+0000 mgr.x (mgr.24118) 1363 : cluster [DBG] pgmap v2037: 17 pgs: 17 active+clean; 2.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 733 op/s 2020-09-08T16:12:47.372 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:47 smithi104 bash[9203]: cluster 2020-09-08T16:12:46.549435+0000 osd.7 (osd.7) 170 : cluster [DBG] 9.e scrub starts 2020-09-08T16:12:47.390 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:12:47.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:47 smithi055 bash[9296]: cluster 2020-09-08T16:12:45.729460+0000 osd.1 (osd.1) 155 : cluster [DBG] 9.4 scrub ok 2020-09-08T16:12:47.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:47 smithi055 bash[9296]: cluster 2020-09-08T16:12:46.115232+0000 osd.11 (osd.11) 137 : cluster [DBG] 9.5 scrub ok 2020-09-08T16:12:47.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:47 smithi055 bash[9296]: cluster 2020-09-08T16:12:46.144895+0000 osd.3 (osd.3) 140 : cluster [DBG] 9.1 scrub ok 2020-09-08T16:12:47.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:47 smithi055 bash[9296]: cluster 2020-09-08T16:12:46.161368+0000 mgr.x (mgr.24118) 1363 : cluster [DBG] pgmap v2037: 17 pgs: 17 active+clean; 2.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 733 op/s 2020-09-08T16:12:47.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:47 smithi055 bash[9296]: cluster 2020-09-08T16:12:46.549435+0000 osd.7 (osd.7) 170 : cluster [DBG] 9.e scrub starts 2020-09-08T16:12:47.435 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:12:47.468 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:12:47.502 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:47.503 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T16:12:47.504 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:12:47.530 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:12:47.563 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:12:47.602 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:12:47.620 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:12:47 smithi166 bash[14254]: debug 2020-09-08T16:12:47.361+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:47.621 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:47 smithi166 bash[9446]: cluster 2020-09-08T16:12:45.729460+0000 osd.1 (osd.1) 155 : cluster [DBG] 9.4 scrub ok 2020-09-08T16:12:47.621 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:47 smithi166 bash[9446]: cluster 2020-09-08T16:12:46.115232+0000 osd.11 (osd.11) 137 : cluster [DBG] 9.5 scrub ok 2020-09-08T16:12:47.621 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:47 smithi166 bash[9446]: cluster 2020-09-08T16:12:46.144895+0000 osd.3 (osd.3) 140 : cluster [DBG] 9.1 scrub ok 2020-09-08T16:12:47.621 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:47 smithi166 bash[9446]: cluster 2020-09-08T16:12:46.161368+0000 mgr.x (mgr.24118) 1363 : cluster [DBG] pgmap v2037: 17 pgs: 17 active+clean; 2.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 733 op/s 2020-09-08T16:12:47.622 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:47 smithi166 bash[9446]: cluster 2020-09-08T16:12:46.549435+0000 osd.7 (osd.7) 170 : cluster [DBG] 9.e scrub starts 2020-09-08T16:12:47.643 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:12:47.643 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:47.644 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:12:47.644 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:12:47.644 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:12:47.644 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:12:47.645 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:12:47.645 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:12:47.645 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:12:47.645 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:12:47.645 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:12:47.646 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:12:47.646 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:12:47.646 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:12:47.656 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:12:47.744 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:12:47.859 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:12:47 smithi055 bash[11956]: debug 2020-09-08T16:12:47.627+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:47.859 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:12:47.968 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:48.124 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:48 smithi055 bash[9296]: audit 2020-09-08T16:12:47.228091+0000 mon.b (mon.1) 123 : audit [DBG] from='client.? v1:172.21.15.104:0/4215922934' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:12:48.130 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:48 smithi055 bash[9296]: cluster 2020-09-08T16:12:47.466255+0000 osd.7 (osd.7) 171 : cluster [DBG] 9.e scrub ok 2020-09-08T16:12:48.131 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:48 smithi055 bash[9296]: cluster 2020-09-08T16:12:47.822707+0000 osd.10 (osd.10) 117 : cluster [DBG] 9.7 scrub starts 2020-09-08T16:12:48.131 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:48 smithi055 bash[9296]: cluster 2020-09-08T16:12:47.961461+0000 osd.9 (osd.9) 120 : cluster [DBG] 9.3 scrub starts 2020-09-08T16:12:48.131 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:12:47 smithi055 bash[19003]: debug 2020-09-08T16:12:47.859+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:48.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:48 smithi166 bash[9446]: audit 2020-09-08T16:12:47.228091+0000 mon.b (mon.1) 123 : audit [DBG] from='client.? v1:172.21.15.104:0/4215922934' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:12:48.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:48 smithi166 bash[9446]: cluster 2020-09-08T16:12:47.466255+0000 osd.7 (osd.7) 171 : cluster [DBG] 9.e scrub ok 2020-09-08T16:12:48.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:48 smithi166 bash[9446]: cluster 2020-09-08T16:12:47.822707+0000 osd.10 (osd.10) 117 : cluster [DBG] 9.7 scrub starts 2020-09-08T16:12:48.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:48 smithi166 bash[9446]: cluster 2020-09-08T16:12:47.961461+0000 osd.9 (osd.9) 120 : cluster [DBG] 9.3 scrub starts 2020-09-08T16:12:48.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:48 smithi104 bash[9203]: audit 2020-09-08T16:12:47.228091+0000 mon.b (mon.1) 123 : audit [DBG] from='client.? v1:172.21.15.104:0/4215922934' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:12:48.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:48 smithi104 bash[9203]: cluster 2020-09-08T16:12:47.466255+0000 osd.7 (osd.7) 171 : cluster [DBG] 9.e scrub ok 2020-09-08T16:12:48.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:48 smithi104 bash[9203]: cluster 2020-09-08T16:12:47.822707+0000 osd.10 (osd.10) 117 : cluster [DBG] 9.7 scrub starts 2020-09-08T16:12:48.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:48 smithi104 bash[9203]: cluster 2020-09-08T16:12:47.961461+0000 osd.9 (osd.9) 120 : cluster [DBG] 9.3 scrub starts 2020-09-08T16:12:48.892 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:48.998 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:49.119 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:49.222 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:12:49.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:49 smithi166 bash[9446]: cluster 2020-09-08T16:12:48.161943+0000 mgr.x (mgr.24118) 1364 : cluster [DBG] pgmap v2038: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 2.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T16:12:49.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:49 smithi166 bash[9446]: cluster 2020-09-08T16:12:48.549335+0000 osd.10 (osd.10) 118 : cluster [DBG] 9.7 scrub ok 2020-09-08T16:12:49.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:49 smithi166 bash[9446]: cluster 2020-09-08T16:12:48.686023+0000 osd.9 (osd.9) 121 : cluster [DBG] 9.3 scrub ok 2020-09-08T16:12:49.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:12:48 smithi166 bash[18960]: debug 2020-09-08T16:12:48.889+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:49.407 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:12:49 smithi055 bash[11956]: debug 2020-09-08T16:12:49.119+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:49.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:49 smithi055 bash[9296]: cluster 2020-09-08T16:12:48.161943+0000 mgr.x (mgr.24118) 1364 : cluster [DBG] pgmap v2038: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 2.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T16:12:49.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:49 smithi055 bash[9296]: cluster 2020-09-08T16:12:48.549335+0000 osd.10 (osd.10) 118 : cluster [DBG] 9.7 scrub ok 2020-09-08T16:12:49.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:49 smithi055 bash[9296]: cluster 2020-09-08T16:12:48.686023+0000 osd.9 (osd.9) 121 : cluster [DBG] 9.3 scrub ok 2020-09-08T16:12:49.454 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:49 smithi104 bash[9203]: cluster 2020-09-08T16:12:48.161943+0000 mgr.x (mgr.24118) 1364 : cluster [DBG] pgmap v2038: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 2.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.05k op/s 2020-09-08T16:12:49.454 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:49 smithi104 bash[9203]: cluster 2020-09-08T16:12:48.549335+0000 osd.10 (osd.10) 118 : cluster [DBG] 9.7 scrub ok 2020-09-08T16:12:49.455 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:49 smithi104 bash[9203]: cluster 2020-09-08T16:12:48.686023+0000 osd.9 (osd.9) 121 : cluster [DBG] 9.3 scrub ok 2020-09-08T16:12:49.737 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:12:49.842 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:12:49.979 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:12:50.088 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:50.094 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:12:49 smithi104 bash[22305]: debug 2020-09-08T16:12:49.740+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:50.095 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2020-09-08T16:12:50.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:12:49 smithi166 bash[16593]: debug 2020-09-08T16:12:49.977+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:50.215 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:50.246 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:12:50.318 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:12:50.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:50 smithi104 bash[9203]: cluster 2020-09-08T16:12:49.068693+0000 osd.5 (osd.5) 136 : cluster [DBG] 9.6 scrub starts 2020-09-08T16:12:50.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:50 smithi104 bash[9203]: cluster 2020-09-08T16:12:49.429140+0000 osd.1 (osd.1) 156 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:50.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:50 smithi104 bash[9203]: cluster 2020-09-08T16:12:49.429226+0000 osd.1 (osd.1) 157 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:50.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:50 smithi104 bash[9203]: cluster 2020-09-08T16:12:49.933987+0000 osd.5 (osd.5) 137 : cluster [DBG] 9.6 scrub ok 2020-09-08T16:12:50.427 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:50 smithi055 bash[9296]: cluster 2020-09-08T16:12:49.068693+0000 osd.5 (osd.5) 136 : cluster [DBG] 9.6 scrub starts 2020-09-08T16:12:50.428 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:50 smithi055 bash[9296]: cluster 2020-09-08T16:12:49.429140+0000 osd.1 (osd.1) 156 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:50.429 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:50 smithi055 bash[9296]: cluster 2020-09-08T16:12:49.429226+0000 osd.1 (osd.1) 157 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:50.429 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:50 smithi055 bash[9296]: cluster 2020-09-08T16:12:49.933987+0000 osd.5 (osd.5) 137 : cluster [DBG] 9.6 scrub ok 2020-09-08T16:12:50.429 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:12:50 smithi055 bash[19003]: debug 2020-09-08T16:12:50.427+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:50.434 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:12:50.536 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:12:50.547 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:12:50 smithi166 bash[11928]: debug 2020-09-08T16:12:50.205+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:50.547 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:50 smithi166 bash[9446]: cluster 2020-09-08T16:12:49.068693+0000 osd.5 (osd.5) 136 : cluster [DBG] 9.6 scrub starts 2020-09-08T16:12:50.548 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:50 smithi166 bash[9446]: cluster 2020-09-08T16:12:49.429140+0000 osd.1 (osd.1) 156 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:50.548 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:50 smithi166 bash[9446]: cluster 2020-09-08T16:12:49.429226+0000 osd.1 (osd.1) 157 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:50.548 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:50 smithi166 bash[9446]: cluster 2020-09-08T16:12:49.933987+0000 osd.5 (osd.5) 137 : cluster [DBG] 9.6 scrub ok 2020-09-08T16:12:50.664 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:12:50.774 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:12:50.872 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:12:50 smithi166 bash[16593]: debug 2020-09-08T16:12:50.657+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:50.911 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:12:51.022 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:51.136 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:51.175 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:12:50 smithi166 bash[16593]: debug 2020-09-08T16:12:50.901+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:51.176 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:12:51 smithi166 bash[14254]: debug 2020-09-08T16:12:51.133+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:51.245 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:12:51.365 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:12:51.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:51 smithi104 bash[9203]: cluster 2020-09-08T16:12:50.135121+0000 osd.1 (osd.1) 158 : cluster [DBG] 9.0 scrub starts 2020-09-08T16:12:51.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:51 smithi104 bash[9203]: cluster 2020-09-08T16:12:50.162540+0000 mgr.x (mgr.24118) 1365 : cluster [DBG] pgmap v2039: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 2.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.23k op/s 2020-09-08T16:12:51.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:51 smithi104 bash[9203]: cluster 2020-09-08T16:12:50.174301+0000 mon.a (mon.0) 2343 : cluster [DBG] osdmap e759: 12 total, 12 up, 12 in 2020-09-08T16:12:51.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:51 smithi104 bash[9203]: cluster 2020-09-08T16:12:50.749507+0000 osd.1 (osd.1) 159 : cluster [DBG] 9.0 scrub ok 2020-09-08T16:12:51.474 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:51.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:51 smithi166 bash[9446]: cluster 2020-09-08T16:12:50.135121+0000 osd.1 (osd.1) 158 : cluster [DBG] 9.0 scrub starts 2020-09-08T16:12:51.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:51 smithi166 bash[9446]: cluster 2020-09-08T16:12:50.162540+0000 mgr.x (mgr.24118) 1365 : cluster [DBG] pgmap v2039: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 2.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.23k op/s 2020-09-08T16:12:51.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:51 smithi166 bash[9446]: cluster 2020-09-08T16:12:50.174301+0000 mon.a (mon.0) 2343 : cluster [DBG] osdmap e759: 12 total, 12 up, 12 in 2020-09-08T16:12:51.619 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:51 smithi166 bash[9446]: cluster 2020-09-08T16:12:50.749507+0000 osd.1 (osd.1) 159 : cluster [DBG] 9.0 scrub ok 2020-09-08T16:12:51.658 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:12:51 smithi055 bash[19003]: debug 2020-09-08T16:12:51.363+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:51.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:51 smithi055 bash[9296]: cluster 2020-09-08T16:12:50.135121+0000 osd.1 (osd.1) 158 : cluster [DBG] 9.0 scrub starts 2020-09-08T16:12:51.659 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:51 smithi055 bash[9296]: cluster 2020-09-08T16:12:50.162540+0000 mgr.x (mgr.24118) 1365 : cluster [DBG] pgmap v2039: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 2.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.23k op/s 2020-09-08T16:12:51.660 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:51 smithi055 bash[9296]: cluster 2020-09-08T16:12:50.174301+0000 mon.a (mon.0) 2343 : cluster [DBG] osdmap e759: 12 total, 12 up, 12 in 2020-09-08T16:12:51.660 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:51 smithi055 bash[9296]: cluster 2020-09-08T16:12:50.749507+0000 osd.1 (osd.1) 159 : cluster [DBG] 9.0 scrub ok 2020-09-08T16:12:51.808 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:12:51.881 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:12:51.911 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:52.051 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:12:52.122 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:12:51 smithi104 bash[17588]: debug 2020-09-08T16:12:51.800+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:52.200 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:52.310 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:52.421 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:12:52.425 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:12:52 smithi104 bash[19922]: debug 2020-09-08T16:12:52.204+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:52.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:52 smithi104 bash[9203]: cluster 2020-09-08T16:12:51.131507+0000 osd.5 (osd.5) 138 : cluster [DBG] 9.a scrub starts 2020-09-08T16:12:52.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:52 smithi104 bash[9203]: cluster 2020-09-08T16:12:51.486062+0000 osd.7 (osd.7) 172 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:12:52.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:52 smithi104 bash[9203]: cluster 2020-09-08T16:12:51.841073+0000 osd.5 (osd.5) 139 : cluster [DBG] 9.a scrub ok 2020-09-08T16:12:52.537 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:12:52.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:52 smithi166 bash[9446]: cluster 2020-09-08T16:12:51.131507+0000 osd.5 (osd.5) 138 : cluster [DBG] 9.a scrub starts 2020-09-08T16:12:52.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:52 smithi166 bash[9446]: cluster 2020-09-08T16:12:51.486062+0000 osd.7 (osd.7) 172 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:12:52.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:52 smithi166 bash[9446]: cluster 2020-09-08T16:12:51.841073+0000 osd.5 (osd.5) 139 : cluster [DBG] 9.a scrub ok 2020-09-08T16:12:52.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:52 smithi055 bash[9296]: cluster 2020-09-08T16:12:51.131507+0000 osd.5 (osd.5) 138 : cluster [DBG] 9.a scrub starts 2020-09-08T16:12:52.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:52 smithi055 bash[9296]: cluster 2020-09-08T16:12:51.486062+0000 osd.7 (osd.7) 172 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:12:52.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:52 smithi055 bash[9296]: cluster 2020-09-08T16:12:51.841073+0000 osd.5 (osd.5) 139 : cluster [DBG] 9.a scrub ok 2020-09-08T16:12:52.658 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:12:52 smithi055 bash[16646]: debug 2020-09-08T16:12:52.419+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:52.923 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:12:53.026 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:53.175 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:12:52 smithi104 bash[22305]: debug 2020-09-08T16:12:52.924+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:53.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:53 smithi166 bash[9446]: cluster 2020-09-08T16:12:51.654620+0000 osd.7 (osd.7) 173 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:12:53.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:53 smithi166 bash[9446]: cluster 2020-09-08T16:12:52.162939+0000 mgr.x (mgr.24118) 1366 : cluster [DBG] pgmap v2041: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 2.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 714 op/s 2020-09-08T16:12:53.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:53 smithi166 bash[9446]: cluster 2020-09-08T16:12:52.507693+0000 osd.8 (osd.8) 112 : cluster [DBG] 9.b scrub starts 2020-09-08T16:12:53.517 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:53 smithi104 bash[9203]: cluster 2020-09-08T16:12:51.654620+0000 osd.7 (osd.7) 173 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:12:53.518 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:53 smithi104 bash[9203]: cluster 2020-09-08T16:12:52.162939+0000 mgr.x (mgr.24118) 1366 : cluster [DBG] pgmap v2041: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 2.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 714 op/s 2020-09-08T16:12:53.518 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:53 smithi104 bash[9203]: cluster 2020-09-08T16:12:52.507693+0000 osd.8 (osd.8) 112 : cluster [DBG] 9.b scrub starts 2020-09-08T16:12:53.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:53 smithi055 bash[9296]: cluster 2020-09-08T16:12:51.654620+0000 osd.7 (osd.7) 173 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:12:53.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:53 smithi055 bash[9296]: cluster 2020-09-08T16:12:52.162939+0000 mgr.x (mgr.24118) 1366 : cluster [DBG] pgmap v2041: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 2.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 714 op/s 2020-09-08T16:12:53.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:53 smithi055 bash[9296]: cluster 2020-09-08T16:12:52.507693+0000 osd.8 (osd.8) 112 : cluster [DBG] 9.b scrub starts 2020-09-08T16:12:54.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:54 smithi104 bash[9203]: cluster 2020-09-08T16:12:53.023027+0000 osd.6 (osd.6) 123 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:54.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:54 smithi104 bash[9203]: cluster 2020-09-08T16:12:53.023115+0000 osd.6 (osd.6) 124 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:54.437 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:54 smithi104 bash[9203]: cluster 2020-09-08T16:12:53.189847+0000 mon.a (mon.0) 2344 : cluster [DBG] osdmap e760: 12 total, 12 up, 12 in 2020-09-08T16:12:54.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:54 smithi166 bash[9446]: cluster 2020-09-08T16:12:53.023027+0000 osd.6 (osd.6) 123 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:54.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:54 smithi166 bash[9446]: cluster 2020-09-08T16:12:53.023115+0000 osd.6 (osd.6) 124 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:54.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:54 smithi166 bash[9446]: cluster 2020-09-08T16:12:53.189847+0000 mon.a (mon.0) 2344 : cluster [DBG] osdmap e760: 12 total, 12 up, 12 in 2020-09-08T16:12:54.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:54 smithi055 bash[9296]: cluster 2020-09-08T16:12:53.023027+0000 osd.6 (osd.6) 123 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:54.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:54 smithi055 bash[9296]: cluster 2020-09-08T16:12:53.023115+0000 osd.6 (osd.6) 124 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:54.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:54 smithi055 bash[9296]: cluster 2020-09-08T16:12:53.189847+0000 mon.a (mon.0) 2344 : cluster [DBG] osdmap e760: 12 total, 12 up, 12 in 2020-09-08T16:12:55.480 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:55 smithi104 bash[9203]: cluster 2020-09-08T16:12:54.163403+0000 mgr.x (mgr.24118) 1367 : cluster [DBG] pgmap v2043: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T16:12:55.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:55 smithi166 bash[9446]: cluster 2020-09-08T16:12:54.163403+0000 mgr.x (mgr.24118) 1367 : cluster [DBG] pgmap v2043: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T16:12:55.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:55 smithi055 bash[9296]: cluster 2020-09-08T16:12:54.163403+0000 mgr.x (mgr.24118) 1367 : cluster [DBG] pgmap v2043: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.12k op/s 2020-09-08T16:12:56.329 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:12:56.434 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:56.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:12:56 smithi166 bash[11928]: debug 2020-09-08T16:12:56.329+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:57.353 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:12:57.458 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:57.478 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:57 smithi055 bash[9296]: cluster 2020-09-08T16:12:56.163814+0000 mgr.x (mgr.24118) 1368 : cluster [DBG] pgmap v2044: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 766 op/s 2020-09-08T16:12:57.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:57 smithi166 bash[9446]: cluster 2020-09-08T16:12:56.163814+0000 mgr.x (mgr.24118) 1368 : cluster [DBG] pgmap v2044: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 766 op/s 2020-09-08T16:12:57.636 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:12:57 smithi104 bash[19922]: debug 2020-09-08T16:12:57.355+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:57.636 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:57 smithi104 bash[9203]: cluster 2020-09-08T16:12:56.163814+0000 mgr.x (mgr.24118) 1368 : cluster [DBG] pgmap v2044: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 2.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 766 op/s 2020-09-08T16:12:58.221 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:58.325 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:58.436 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:12:58.539 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:58.559 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:58 smithi055 bash[9296]: cluster 2020-09-08T16:12:57.420164+0000 osd.11 (osd.11) 138 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:58.560 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:58 smithi055 bash[9296]: cluster 2020-09-08T16:12:57.420255+0000 osd.11 (osd.11) 139 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:58.560 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:12:58 smithi055 bash[14301]: debug 2020-09-08T16:12:58.219+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:58.560 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:12:58 smithi055 bash[14301]: debug 2020-09-08T16:12:58.431+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:58.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:58 smithi166 bash[9446]: cluster 2020-09-08T16:12:57.420164+0000 osd.11 (osd.11) 138 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:58.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:58 smithi166 bash[9446]: cluster 2020-09-08T16:12:57.420255+0000 osd.11 (osd.11) 139 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:58.653 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:12:58.757 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:58.765 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:58 smithi104 bash[9203]: cluster 2020-09-08T16:12:57.420164+0000 osd.11 (osd.11) 138 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:12:58.765 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:58 smithi104 bash[9203]: cluster 2020-09-08T16:12:57.420255+0000 osd.11 (osd.11) 139 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:12:58.910 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:12:58 smithi055 bash[11956]: debug 2020-09-08T16:12:58.655+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:58.936 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:59.040 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:59.048 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:12:58 smithi166 bash[14254]: debug 2020-09-08T16:12:58.933+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:59.147 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:12:59.218 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T16:12:59.251 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:59.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:12:59 smithi166 bash[14254]: debug 2020-09-08T16:12:59.145+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:59.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:59 smithi166 bash[9446]: cluster 2020-09-08T16:12:58.164321+0000 mgr.x (mgr.24118) 1369 : cluster [DBG] pgmap v2045: 17 pgs: 17 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 989 op/s 2020-09-08T16:12:59.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:59 smithi166 bash[9446]: cluster 2020-09-08T16:12:58.216912+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e761: 12 total, 12 up, 12 in 2020-09-08T16:12:59.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:12:59 smithi166 bash[9446]: audit 2020-09-08T16:12:58.829058+0000 mon.a (mon.0) 2346 : audit [INF] from='client.? v1:172.21.15.104:0/3027005724' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:12:59.378 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:12:59.494 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:12:59.550 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T16:12:59.551 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2020-09-08T16:12:59.551 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2020-09-08T16:12:59.551 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2020-09-08T16:12:59.551 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T16:12:59.647 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:12:59 smithi166 bash[18960]: debug 2020-09-08T16:12:59.377+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:12:59.647 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:12:59.656 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:59 smithi055 bash[9296]: cluster 2020-09-08T16:12:58.164321+0000 mgr.x (mgr.24118) 1369 : cluster [DBG] pgmap v2045: 17 pgs: 17 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 989 op/s 2020-09-08T16:12:59.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:59 smithi055 bash[9296]: cluster 2020-09-08T16:12:58.216912+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e761: 12 total, 12 up, 12 in 2020-09-08T16:12:59.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:12:59 smithi055 bash[9296]: audit 2020-09-08T16:12:58.829058+0000 mon.a (mon.0) 2346 : audit [INF] from='client.? v1:172.21.15.104:0/3027005724' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:12:59.706 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:59 smithi104 bash[9203]: cluster 2020-09-08T16:12:58.164321+0000 mgr.x (mgr.24118) 1369 : cluster [DBG] pgmap v2045: 17 pgs: 17 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 989 op/s 2020-09-08T16:12:59.706 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:59 smithi104 bash[9203]: cluster 2020-09-08T16:12:58.216912+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e761: 12 total, 12 up, 12 in 2020-09-08T16:12:59.707 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:12:59 smithi104 bash[9203]: audit 2020-09-08T16:12:58.829058+0000 mon.a (mon.0) 2346 : audit [INF] from='client.? v1:172.21.15.104:0/3027005724' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:12:59.751 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:13:00.116 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:12:59 smithi166 bash[16593]: debug 2020-09-08T16:12:59.645+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:00.296 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:13:00.483 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:13:00.492 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:13:00 smithi104 bash[22305]: debug 2020-09-08T16:13:00.299+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:00.492 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:00 smithi104 bash[9203]: cluster 2020-09-08T16:12:59.214279+0000 mon.a (mon.0) 2347 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:13:00.493 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:00 smithi104 bash[9203]: cluster 2020-09-08T16:12:59.214310+0000 mon.a (mon.0) 2348 : cluster [INF] Cluster is now healthy 2020-09-08T16:13:00.493 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:00 smithi104 bash[9203]: audit 2020-09-08T16:12:59.222335+0000 mon.a (mon.0) 2349 : audit [INF] from='client.? v1:172.21.15.104:0/3027005724' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:13:00.493 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:00 smithi104 bash[9203]: cluster 2020-09-08T16:12:59.222392+0000 mon.a (mon.0) 2350 : cluster [DBG] osdmap e762: 12 total, 12 up, 12 in 2020-09-08T16:13:00.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:00 smithi166 bash[9446]: cluster 2020-09-08T16:12:59.214279+0000 mon.a (mon.0) 2347 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:13:00.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:00 smithi166 bash[9446]: cluster 2020-09-08T16:12:59.214310+0000 mon.a (mon.0) 2348 : cluster [INF] Cluster is now healthy 2020-09-08T16:13:00.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:00 smithi166 bash[9446]: audit 2020-09-08T16:12:59.222335+0000 mon.a (mon.0) 2349 : audit [INF] from='client.? v1:172.21.15.104:0/3027005724' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:13:00.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:00 smithi166 bash[9446]: cluster 2020-09-08T16:12:59.222392+0000 mon.a (mon.0) 2350 : cluster [DBG] osdmap e762: 12 total, 12 up, 12 in 2020-09-08T16:13:00.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:00 smithi055 bash[9296]: cluster 2020-09-08T16:12:59.214279+0000 mon.a (mon.0) 2347 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:13:00.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:00 smithi055 bash[9296]: cluster 2020-09-08T16:12:59.214310+0000 mon.a (mon.0) 2348 : cluster [INF] Cluster is now healthy 2020-09-08T16:13:00.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:00 smithi055 bash[9296]: audit 2020-09-08T16:12:59.222335+0000 mon.a (mon.0) 2349 : audit [INF] from='client.? v1:172.21.15.104:0/3027005724' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:13:00.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:00 smithi055 bash[9296]: cluster 2020-09-08T16:12:59.222392+0000 mon.a (mon.0) 2350 : cluster [DBG] osdmap e762: 12 total, 12 up, 12 in 2020-09-08T16:13:01.146 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:13:01.250 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:13:01.256 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:01 smithi104 bash[9203]: cluster 2020-09-08T16:13:00.164847+0000 mgr.x (mgr.24118) 1370 : cluster [DBG] pgmap v2048: 17 pgs: 17 active+clean; 1.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 809 op/s 2020-09-08T16:13:01.257 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:13:01 smithi104 bash[15273]: debug 2020-09-08T16:13:01.147+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:01.342 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2020-09-08T16:13:01.376 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:13:01.480 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:13:01.489 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:01 smithi055 bash[9296]: cluster 2020-09-08T16:13:00.164847+0000 mgr.x (mgr.24118) 1370 : cluster [DBG] pgmap v2048: 17 pgs: 17 active+clean; 1.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 809 op/s 2020-09-08T16:13:01.538 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:13:01.542 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:13:01.582 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:13:01.611 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:13:01.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:01 smithi166 bash[9446]: cluster 2020-09-08T16:13:00.164847+0000 mgr.x (mgr.24118) 1370 : cluster [DBG] pgmap v2048: 17 pgs: 17 active+clean; 1.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 809 op/s 2020-09-08T16:13:01.627 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:13:01.650 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:13:01.675 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:13:01 smithi104 bash[22305]: debug 2020-09-08T16:13:01.375+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:01.677 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:13:01.716 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:13:01.729 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:13:01.745 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:13:01.786 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:13:01.820 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:13:01.850 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:13:01.882 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:13:01.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:13:01 smithi055 bash[14301]: debug 2020-09-08T16:13:01.619+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:01.915 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:13:01.918 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:13:01.918 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:13:01.918 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:13:01.918 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:13:01.919 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:13:01.919 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:13:01.919 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:13:01.919 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:13:01.920 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:13:01.920 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:13:01.920 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:13:01.920 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:13:02.360 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:13:02.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:02 smithi104 bash[9203]: audit 2020-09-08T16:13:01.514425+0000 mon.b (mon.1) 124 : audit [DBG] from='client.? v1:172.21.15.104:0/3780620682' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:13:02.426 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:13:02 smithi104 bash[17588]: debug 2020-09-08T16:13:02.363+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:02.464 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:13:02.474 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:02 smithi055 bash[9296]: audit 2020-09-08T16:13:01.514425+0000 mon.b (mon.1) 124 : audit [DBG] from='client.? v1:172.21.15.104:0/3780620682' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:13:02.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:02 smithi166 bash[9446]: audit 2020-09-08T16:13:01.514425+0000 mon.b (mon.1) 124 : audit [DBG] from='client.? v1:172.21.15.104:0/3780620682' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:13:02.979 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set noscrub 2020-09-08T16:13:03.130 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:13:03.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:03 smithi104 bash[9203]: cluster 2020-09-08T16:13:02.054433+0000 osd.4 (osd.4) 83 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:13:03.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:03 smithi104 bash[9203]: cluster 2020-09-08T16:13:02.054530+0000 osd.4 (osd.4) 84 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:13:03.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:03 smithi104 bash[9203]: cluster 2020-09-08T16:13:02.165329+0000 mgr.x (mgr.24118) 1371 : cluster [DBG] pgmap v2049: 17 pgs: 17 active+clean; 1.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 668 op/s 2020-09-08T16:13:03.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:03 smithi104 bash[9203]: cluster 2020-09-08T16:13:02.229767+0000 mon.a (mon.0) 2351 : cluster [DBG] osdmap e763: 12 total, 12 up, 12 in 2020-09-08T16:13:03.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:03 smithi166 bash[9446]: cluster 2020-09-08T16:13:02.054433+0000 osd.4 (osd.4) 83 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:13:03.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:03 smithi166 bash[9446]: cluster 2020-09-08T16:13:02.054530+0000 osd.4 (osd.4) 84 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:13:03.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:03 smithi166 bash[9446]: cluster 2020-09-08T16:13:02.165329+0000 mgr.x (mgr.24118) 1371 : cluster [DBG] pgmap v2049: 17 pgs: 17 active+clean; 1.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 668 op/s 2020-09-08T16:13:03.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:03 smithi166 bash[9446]: cluster 2020-09-08T16:13:02.229767+0000 mon.a (mon.0) 2351 : cluster [DBG] osdmap e763: 12 total, 12 up, 12 in 2020-09-08T16:13:03.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:03 smithi055 bash[9296]: cluster 2020-09-08T16:13:02.054433+0000 osd.4 (osd.4) 83 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:13:03.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:03 smithi055 bash[9296]: cluster 2020-09-08T16:13:02.054530+0000 osd.4 (osd.4) 84 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:13:03.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:03 smithi055 bash[9296]: cluster 2020-09-08T16:13:02.165329+0000 mgr.x (mgr.24118) 1371 : cluster [DBG] pgmap v2049: 17 pgs: 17 active+clean; 1.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 668 op/s 2020-09-08T16:13:03.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:03 smithi055 bash[9296]: cluster 2020-09-08T16:13:02.229767+0000 mon.a (mon.0) 2351 : cluster [DBG] osdmap e763: 12 total, 12 up, 12 in 2020-09-08T16:13:05.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:05 smithi166 bash[9446]: cluster 2020-09-08T16:13:04.165867+0000 mgr.x (mgr.24118) 1372 : cluster [DBG] pgmap v2051: 17 pgs: 17 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 936 op/s 2020-09-08T16:13:05.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:05 smithi055 bash[9296]: cluster 2020-09-08T16:13:04.165867+0000 mgr.x (mgr.24118) 1372 : cluster [DBG] pgmap v2051: 17 pgs: 17 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 936 op/s 2020-09-08T16:13:05.665 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:13:05.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:05 smithi104 bash[9203]: cluster 2020-09-08T16:13:04.165867+0000 mgr.x (mgr.24118) 1372 : cluster [DBG] pgmap v2051: 17 pgs: 17 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 936 op/s 2020-09-08T16:13:05.768 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:13:05.832 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:13:05.900 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:13:05.989 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:13:06.004 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:13:06.144 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:13:06.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:13:05 smithi055 bash[16646]: debug 2020-09-08T16:13:05.899+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:06.158 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:13:05 smithi055 bash[14301]: debug 2020-09-08T16:13:05.663+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:06.249 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:13:06.370 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:13:06 smithi166 bash[14254]: debug 2020-09-08T16:13:06.141+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:06.380 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:13:06.481 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:13:06.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:13:06 smithi166 bash[11928]: debug 2020-09-08T16:13:06.373+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:07.152 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:13:07.257 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:13:07.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:07 smithi104 bash[9203]: cluster 2020-09-08T16:13:06.166354+0000 mgr.x (mgr.24118) 1373 : cluster [DBG] pgmap v2052: 17 pgs: 17 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 706 op/s 2020-09-08T16:13:07.426 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:13:07 smithi104 bash[19922]: debug 2020-09-08T16:13:07.151+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:07.439 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:13:07.541 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:13:07.549 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:07 smithi166 bash[9446]: cluster 2020-09-08T16:13:06.166354+0000 mgr.x (mgr.24118) 1373 : cluster [DBG] pgmap v2052: 17 pgs: 17 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 706 op/s 2020-09-08T16:13:07.550 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:13:07 smithi166 bash[18960]: debug 2020-09-08T16:13:07.437+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:07.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:07 smithi055 bash[9296]: cluster 2020-09-08T16:13:06.166354+0000 mgr.x (mgr.24118) 1373 : cluster [DBG] pgmap v2052: 17 pgs: 17 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 706 op/s 2020-09-08T16:13:07.668 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:13:07.773 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:13:07.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:13:07 smithi166 bash[18960]: debug 2020-09-08T16:13:07.669+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:08.012 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:13:08.118 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:13:08.175 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:13:08 smithi104 bash[15273]: debug 2020-09-08T16:13:08.015+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:08.253 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:13:08.363 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:13:08.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:13:08 smithi055 bash[14301]: debug 2020-09-08T16:13:08.247+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:08.521 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:13:08.636 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:13:08.753 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:13:08.858 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:13:08.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:13:08 smithi166 bash[18960]: debug 2020-09-08T16:13:08.516+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:08.917 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:13:08 smithi055 bash[11956]: debug 2020-09-08T16:13:08.751+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:08.974 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:13:09.076 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:13:09.199 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:13:09.229 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:09 smithi166 bash[9446]: cluster 2020-09-08T16:13:08.166837+0000 mgr.x (mgr.24118) 1374 : cluster [DBG] pgmap v2053: 17 pgs: 17 active+clean; 1.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 980 op/s 2020-09-08T16:13:09.230 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:13:08 smithi166 bash[14254]: debug 2020-09-08T16:13:08.968+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:09.304 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:13:09.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:09 smithi055 bash[9296]: cluster 2020-09-08T16:13:08.166837+0000 mgr.x (mgr.24118) 1374 : cluster [DBG] pgmap v2053: 17 pgs: 17 active+clean; 1.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 980 op/s 2020-09-08T16:13:09.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:13:09 smithi055 bash[16646]: debug 2020-09-08T16:13:09.199+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:09.427 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:13:09.531 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:13:09.541 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:09 smithi104 bash[9203]: cluster 2020-09-08T16:13:08.166837+0000 mgr.x (mgr.24118) 1374 : cluster [DBG] pgmap v2053: 17 pgs: 17 active+clean; 1.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 980 op/s 2020-09-08T16:13:09.692 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:13:09.796 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:13:09.858 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:13:09 smithi104 bash[15273]: debug 2020-09-08T16:13:09.695+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:09.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:13:09 smithi055 bash[11956]: debug 2020-09-08T16:13:09.427+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:10.118 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:13:10.223 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:13:10.366 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:13:10.428 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:13:10 smithi104 bash[17588]: debug 2020-09-08T16:13:10.119+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:10.469 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:13:10.595 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T16:13:10.596 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:13:10.625 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:13:10.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:13:10 smithi055 bash[16646]: debug 2020-09-08T16:13:10.367+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:10.727 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:13:10.851 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:13:10.852 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:13:10 smithi166 bash[11928]: debug 2020-09-08T16:13:10.616+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:10.956 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:13:11.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:11 smithi166 bash[9446]: cluster 2020-09-08T16:13:10.167241+0000 mgr.x (mgr.24118) 1375 : cluster [DBG] pgmap v2054: 17 pgs: 17 active+clean; 1.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 979 op/s 2020-09-08T16:13:11.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:11 smithi166 bash[9446]: cluster 2020-09-08T16:13:10.983735+0000 osd.9 (osd.9) 122 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:13:11.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:11 smithi166 bash[9446]: cluster 2020-09-08T16:13:10.983816+0000 osd.9 (osd.9) 123 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:13:11.117 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:13:10 smithi166 bash[18960]: debug 2020-09-08T16:13:10.852+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:11.120 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:13:11.228 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:13:11.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:11 smithi055 bash[9296]: cluster 2020-09-08T16:13:10.167241+0000 mgr.x (mgr.24118) 1375 : cluster [DBG] pgmap v2054: 17 pgs: 17 active+clean; 1.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 979 op/s 2020-09-08T16:13:11.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:11 smithi055 bash[9296]: cluster 2020-09-08T16:13:10.983735+0000 osd.9 (osd.9) 122 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:13:11.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:11 smithi055 bash[9296]: cluster 2020-09-08T16:13:10.983816+0000 osd.9 (osd.9) 123 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:13:11.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:11 smithi104 bash[9203]: cluster 2020-09-08T16:13:10.167241+0000 mgr.x (mgr.24118) 1375 : cluster [DBG] pgmap v2054: 17 pgs: 17 active+clean; 1.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 979 op/s 2020-09-08T16:13:11.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:11 smithi104 bash[9203]: cluster 2020-09-08T16:13:10.983735+0000 osd.9 (osd.9) 122 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:13:11.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:11 smithi104 bash[9203]: cluster 2020-09-08T16:13:10.983816+0000 osd.9 (osd.9) 123 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:13:11.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:13:11 smithi166 bash[14254]: debug 2020-09-08T16:13:11.116+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:12.188 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:13:12.293 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:13:12.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:12 smithi166 bash[9446]: cluster 2020-09-08T16:13:11.110811+0000 mon.a (mon.0) 2352 : cluster [DBG] osdmap e764: 12 total, 12 up, 12 in 2020-09-08T16:13:12.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:12 smithi055 bash[9296]: cluster 2020-09-08T16:13:11.110811+0000 mon.a (mon.0) 2352 : cluster [DBG] osdmap e764: 12 total, 12 up, 12 in 2020-09-08T16:13:12.427 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:13:12 smithi104 bash[22305]: debug 2020-09-08T16:13:12.191+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:12.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:12 smithi104 bash[9203]: cluster 2020-09-08T16:13:11.110811+0000 mon.a (mon.0) 2352 : cluster [DBG] osdmap e764: 12 total, 12 up, 12 in 2020-09-08T16:13:12.451 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:13:12.555 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:13:12.740 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:13:12.811 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2020-09-08T16:13:12.819 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:13:12 smithi104 bash[19922]: debug 2020-09-08T16:13:12.743+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:12.848 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:13:12.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:13:12 smithi166 bash[11928]: debug 2020-09-08T16:13:12.444+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:12.966 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:13:13.111 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is set 2020-09-08T16:13:13.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:13 smithi166 bash[9446]: cluster 2020-09-08T16:13:12.167683+0000 mgr.x (mgr.24118) 1376 : cluster [DBG] pgmap v2056: 17 pgs: 17 active+clean; 1.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 985 op/s 2020-09-08T16:13:13.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:13 smithi166 bash[9446]: audit 2020-09-08T16:13:12.598192+0000 mon.a (mon.0) 2353 : audit [INF] from='client.? v1:172.21.15.104:0/3519575291' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:13:13.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:13 smithi055 bash[9296]: cluster 2020-09-08T16:13:12.167683+0000 mgr.x (mgr.24118) 1376 : cluster [DBG] pgmap v2056: 17 pgs: 17 active+clean; 1.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 985 op/s 2020-09-08T16:13:13.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:13 smithi055 bash[9296]: audit 2020-09-08T16:13:12.598192+0000 mon.a (mon.0) 2353 : audit [INF] from='client.? v1:172.21.15.104:0/3519575291' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:13:13.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:13 smithi104 bash[9203]: cluster 2020-09-08T16:13:12.167683+0000 mgr.x (mgr.24118) 1376 : cluster [DBG] pgmap v2056: 17 pgs: 17 active+clean; 1.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 985 op/s 2020-09-08T16:13:13.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:13 smithi104 bash[9203]: audit 2020-09-08T16:13:12.598192+0000 mon.a (mon.0) 2353 : audit [INF] from='client.? v1:172.21.15.104:0/3519575291' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2020-09-08T16:13:13.579 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:13:13.686 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:13:13.829 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:13:13.925 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:13:13 smithi104 bash[17588]: debug 2020-09-08T16:13:13.583+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:13.933 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:13:14.063 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:13:14.105 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:13:13 smithi055 bash[11956]: debug 2020-09-08T16:13:13.827+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:14.105 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:14 smithi055 bash[9296]: cluster 2020-09-08T16:13:13.021016+0000 osd.3 (osd.3) 141 : cluster [DBG] 9.2 deep-scrub starts 2020-09-08T16:13:14.106 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:14 smithi055 bash[9296]: cluster 2020-09-08T16:13:13.107721+0000 mon.a (mon.0) 2354 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:13:14.106 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:14 smithi055 bash[9296]: audit 2020-09-08T16:13:13.114963+0000 mon.a (mon.0) 2355 : audit [INF] from='client.? v1:172.21.15.104:0/3519575291' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:13:14.106 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:14 smithi055 bash[9296]: cluster 2020-09-08T16:13:13.115014+0000 mon.a (mon.0) 2356 : cluster [DBG] osdmap e765: 12 total, 12 up, 12 in 2020-09-08T16:13:14.106 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:14 smithi055 bash[9296]: cluster 2020-09-08T16:13:13.254719+0000 osd.2 (osd.2) 118 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:13:14.106 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:14 smithi055 bash[9296]: cluster 2020-09-08T16:13:13.254796+0000 osd.2 (osd.2) 119 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:13:14.167 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:13:14.291 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:13:14 smithi166 bash[16593]: debug 2020-09-08T16:13:14.060+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:14.291 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:14 smithi166 bash[9446]: cluster 2020-09-08T16:13:13.021016+0000 osd.3 (osd.3) 141 : cluster [DBG] 9.2 deep-scrub starts 2020-09-08T16:13:14.292 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:14 smithi166 bash[9446]: cluster 2020-09-08T16:13:13.107721+0000 mon.a (mon.0) 2354 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:13:14.292 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:14 smithi166 bash[9446]: audit 2020-09-08T16:13:13.114963+0000 mon.a (mon.0) 2355 : audit [INF] from='client.? v1:172.21.15.104:0/3519575291' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:13:14.292 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:14 smithi166 bash[9446]: cluster 2020-09-08T16:13:13.115014+0000 mon.a (mon.0) 2356 : cluster [DBG] osdmap e765: 12 total, 12 up, 12 in 2020-09-08T16:13:14.292 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:14 smithi166 bash[9446]: cluster 2020-09-08T16:13:13.254719+0000 osd.2 (osd.2) 118 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:13:14.292 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:14 smithi166 bash[9446]: cluster 2020-09-08T16:13:13.254796+0000 osd.2 (osd.2) 119 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:13:14.298 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:13:14.412 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:13:14.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:14 smithi104 bash[9203]: cluster 2020-09-08T16:13:13.021016+0000 osd.3 (osd.3) 141 : cluster [DBG] 9.2 deep-scrub starts 2020-09-08T16:13:14.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:14 smithi104 bash[9203]: cluster 2020-09-08T16:13:13.107721+0000 mon.a (mon.0) 2354 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:13:14.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:14 smithi104 bash[9203]: audit 2020-09-08T16:13:13.114963+0000 mon.a (mon.0) 2355 : audit [INF] from='client.? v1:172.21.15.104:0/3519575291' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2020-09-08T16:13:14.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:14 smithi104 bash[9203]: cluster 2020-09-08T16:13:13.115014+0000 mon.a (mon.0) 2356 : cluster [DBG] osdmap e765: 12 total, 12 up, 12 in 2020-09-08T16:13:14.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:14 smithi104 bash[9203]: cluster 2020-09-08T16:13:13.254719+0000 osd.2 (osd.2) 118 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:13:14.427 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:14 smithi104 bash[9203]: cluster 2020-09-08T16:13:13.254796+0000 osd.2 (osd.2) 119 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:13:14.523 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:13:14.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:13:14 smithi166 bash[16593]: debug 2020-09-08T16:13:14.288+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:14.617 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:13:14 smithi166 bash[16593]: debug 2020-09-08T16:13:14.520+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:14.627 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:13:14.746 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:13:14.855 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:13:15.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:13:14 smithi166 bash[11928]: debug 2020-09-08T16:13:14.740+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:15.118 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:15 smithi104 bash[9203]: cluster 2020-09-08T16:13:14.119014+0000 mon.a (mon.0) 2357 : cluster [DBG] osdmap e766: 12 total, 12 up, 12 in 2020-09-08T16:13:15.118 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:15 smithi104 bash[9203]: cluster 2020-09-08T16:13:14.168169+0000 mgr.x (mgr.24118) 1377 : cluster [DBG] pgmap v2059: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 934 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 960 op/s 2020-09-08T16:13:15.248 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:13:15.354 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:13:15.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:15 smithi166 bash[9446]: cluster 2020-09-08T16:13:14.119014+0000 mon.a (mon.0) 2357 : cluster [DBG] osdmap e766: 12 total, 12 up, 12 in 2020-09-08T16:13:15.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:15 smithi166 bash[9446]: cluster 2020-09-08T16:13:14.168169+0000 mgr.x (mgr.24118) 1377 : cluster [DBG] pgmap v2059: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 934 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 960 op/s 2020-09-08T16:13:15.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:15 smithi055 bash[9296]: cluster 2020-09-08T16:13:14.119014+0000 mon.a (mon.0) 2357 : cluster [DBG] osdmap e766: 12 total, 12 up, 12 in 2020-09-08T16:13:15.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:15 smithi055 bash[9296]: cluster 2020-09-08T16:13:14.168169+0000 mgr.x (mgr.24118) 1377 : cluster [DBG] pgmap v2059: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 934 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 960 op/s 2020-09-08T16:13:15.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:13:15 smithi104 bash[22305]: debug 2020-09-08T16:13:15.251+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:15.478 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:13:15.582 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:13:15.721 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:13:15.826 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:13:15.838 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:13:15 smithi166 bash[18960]: debug 2020-09-08T16:13:15.472+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:15.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:13:15 smithi055 bash[19003]: debug 2020-09-08T16:13:15.723+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:15.941 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:13:16.025 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:13:16.048 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:13:16.054 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:13:16.093 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:13:16.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:13:15 smithi166 bash[14254]: debug 2020-09-08T16:13:15.940+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:16.124 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:13:16.151 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:13:16.157 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:13:16.183 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:13:16.221 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:13:16.260 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:13:16.263 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:13:16.294 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:13:16.329 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:13:16.368 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:13:16.390 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:16 smithi166 bash[9446]: audit 2020-09-08T16:13:15.992162+0000 mon.a (mon.0) 2358 : audit [DBG] from='client.? v1:172.21.15.104:0/3186099099' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:13:16.391 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:13:16.402 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:13:16.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:16 smithi055 bash[9296]: audit 2020-09-08T16:13:15.992162+0000 mon.a (mon.0) 2358 : audit [DBG] from='client.? v1:172.21.15.104:0/3186099099' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:13:16.407 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:13:16 smithi055 bash[16646]: debug 2020-09-08T16:13:16.159+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:16.414 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:13:16.416 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:13:16.416 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:13:16.417 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:13:16.417 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:13:16.417 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:13:16.417 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:13:16.418 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:13:16.418 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:13:16.418 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:13:16.418 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:13:16.418 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:13:16.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:16 smithi104 bash[9203]: audit 2020-09-08T16:13:15.992162+0000 mon.a (mon.0) 2358 : audit [DBG] from='client.? v1:172.21.15.104:0/3186099099' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:13:16.497 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:13:16.617 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:13:16.721 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:13:16.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:13:16 smithi166 bash[11928]: debug 2020-09-08T16:13:16.388+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:16.908 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:13:16 smithi055 bash[16646]: debug 2020-09-08T16:13:16.619+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:17.239 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd set nodeep-scrub 2020-09-08T16:13:17.400 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:13:17.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:17 smithi055 bash[9296]: cluster 2020-09-08T16:13:16.168626+0000 mgr.x (mgr.24118) 1378 : cluster [DBG] pgmap v2060: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 901 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 780 op/s 2020-09-08T16:13:17.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:17 smithi104 bash[9203]: cluster 2020-09-08T16:13:16.168626+0000 mgr.x (mgr.24118) 1378 : cluster [DBG] pgmap v2060: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 901 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 780 op/s 2020-09-08T16:13:17.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:17 smithi166 bash[9446]: cluster 2020-09-08T16:13:16.168626+0000 mgr.x (mgr.24118) 1378 : cluster [DBG] pgmap v2060: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 901 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 780 op/s 2020-09-08T16:13:17.933 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:13:18.039 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:13:18.297 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:13:17 smithi104 bash[15273]: debug 2020-09-08T16:13:17.927+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:18.298 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:13:18.403 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:13:18.675 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:13:18 smithi104 bash[19922]: debug 2020-09-08T16:13:18.299+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:19.102 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:13:19.206 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:13:19.317 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:13:19.426 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:13:19.427 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:13:19 smithi104 bash[22305]: debug 2020-09-08T16:13:19.103+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:19.428 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:19 smithi104 bash[9203]: cluster 2020-09-08T16:13:18.169123+0000 mgr.x (mgr.24118) 1379 : cluster [DBG] pgmap v2061: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 778 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 942 op/s 2020-09-08T16:13:19.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:19 smithi166 bash[9446]: cluster 2020-09-08T16:13:18.169123+0000 mgr.x (mgr.24118) 1379 : cluster [DBG] pgmap v2061: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 778 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 942 op/s 2020-09-08T16:13:19.658 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:13:19 smithi055 bash[11956]: debug 2020-09-08T16:13:19.314+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:19.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:19 smithi055 bash[9296]: cluster 2020-09-08T16:13:18.169123+0000 mgr.x (mgr.24118) 1379 : cluster [DBG] pgmap v2061: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 778 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 942 op/s 2020-09-08T16:13:19.747 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:13:19.851 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:13:20.003 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:13:20.106 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:13:20.130 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:13:20 smithi166 bash[16593]: debug 2020-09-08T16:13:20.000+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:20.175 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:13:19 smithi104 bash[22305]: debug 2020-09-08T16:13:19.751+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:20.221 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:13:20.326 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:13:20.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:13:20 smithi055 bash[19003]: debug 2020-09-08T16:13:20.218+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:20.454 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:13:20.560 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:13:20.806 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:13:20.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:13:20 smithi055 bash[14301]: debug 2020-09-08T16:13:20.454+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:20.910 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:13:20.925 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:13:20 smithi104 bash[17588]: debug 2020-09-08T16:13:20.803+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:20.975 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-09-08T16:13:21.034 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:13:21.133 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:13:21.137 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:13:21.298 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:13:21.349 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:21 smithi104 bash[9203]: cluster 2020-09-08T16:13:19.655588+0000 osd.3 (osd.3) 142 : cluster [DBG] 9.2 deep-scrub ok 2020-09-08T16:13:21.350 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:21 smithi104 bash[9203]: cluster 2020-09-08T16:13:20.169627+0000 mgr.x (mgr.24118) 1380 : cluster [DBG] pgmap v2062: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 653 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T16:13:21.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:21 smithi166 bash[9446]: cluster 2020-09-08T16:13:19.655588+0000 osd.3 (osd.3) 142 : cluster [DBG] 9.2 deep-scrub ok 2020-09-08T16:13:21.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:21 smithi166 bash[9446]: cluster 2020-09-08T16:13:20.169627+0000 mgr.x (mgr.24118) 1380 : cluster [DBG] pgmap v2062: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 653 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T16:13:21.369 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:13:21 smithi166 bash[14254]: debug 2020-09-08T16:13:21.296+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:21.404 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:13:21.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:21 smithi055 bash[9296]: cluster 2020-09-08T16:13:19.655588+0000 osd.3 (osd.3) 142 : cluster [DBG] 9.2 deep-scrub ok 2020-09-08T16:13:21.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:21 smithi055 bash[9296]: cluster 2020-09-08T16:13:20.169627+0000 mgr.x (mgr.24118) 1380 : cluster [DBG] pgmap v2062: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 653 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.08k op/s 2020-09-08T16:13:21.409 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:13:21 smithi055 bash[14301]: debug 2020-09-08T16:13:21.030+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:21.544 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:13:21.647 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:13:21.784 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:13:21 smithi166 bash[11928]: debug 2020-09-08T16:13:21.780+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:21.785 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:13:21 smithi166 bash[16593]: debug 2020-09-08T16:13:21.540+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:21.787 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:13:21.896 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:13:22.035 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:13:22.142 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:13:22.157 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:13:22 smithi055 bash[19003]: debug 2020-09-08T16:13:22.030+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:22 smithi055 bash[9296]: cluster 2020-09-08T16:13:21.471951+0000 osd.7 (osd.7) 174 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:13:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:22 smithi055 bash[9296]: cluster 2020-09-08T16:13:21.472026+0000 osd.7 (osd.7) 175 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:13:22.276 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:13:22.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:22 smithi166 bash[9446]: cluster 2020-09-08T16:13:21.471951+0000 osd.7 (osd.7) 174 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:13:22.368 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:22 smithi166 bash[9446]: cluster 2020-09-08T16:13:21.472026+0000 osd.7 (osd.7) 175 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:13:22.368 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:13:22 smithi166 bash[18960]: debug 2020-09-08T16:13:22.272+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:22.381 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:13:22.390 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:22 smithi104 bash[9203]: cluster 2020-09-08T16:13:21.471951+0000 osd.7 (osd.7) 174 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:13:22.391 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:22 smithi104 bash[9203]: cluster 2020-09-08T16:13:21.472026+0000 osd.7 (osd.7) 175 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:13:22.801 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2020-09-08T16:13:22.801 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2020-09-08T16:13:22.802 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.802 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.26587.0:273845 9.1 9:8a8a1ef0:::benchmark_data_smithi166_56754_object133812:head [delete] snapc 0=[] ondisk+write+known_if_redirected e767)", 2020-09-08T16:13:22.802 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:13:22.783767+0000", 2020-09-08T16:13:22.803 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.023268177000000001, 2020-09-08T16:13:22.803 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.023292871999999999, 2020-09-08T16:13:22.804 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:13:22.804 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:13:22.804 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:13:22.804 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.26587", 2020-09-08T16:13:22.804 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:13:22.805 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 273845 2020-09-08T16:13:22.805 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.805 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:13:22.805 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.806 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:13:22.806 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.783767+0000", 2020-09-08T16:13:22.806 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:13:22.806 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.807 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.807 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:13:22.807 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.783767+0000", 2020-09-08T16:13:22.808 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999595 2020-09-08T16:13:22.808 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.808 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.808 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:13:22.809 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.783727+0000", 2020-09-08T16:13:22.809 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9213000000000001e-05 2020-09-08T16:13:22.809 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.809 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.809 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:13:22.810 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.783816+0000", 2020-09-08T16:13:22.810 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7844999999999999e-05 2020-09-08T16:13:22.810 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.810 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.811 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:13:22.811 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.783834+0000", 2020-09-08T16:13:22.811 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5650000000000003e-06 2020-09-08T16:13:22.811 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.811 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.812 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:13:22.812 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.783839+0000", 2020-09-08T16:13:22.812 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0088520430000000004 2020-09-08T16:13:22.812 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.812 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.813 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:13:22.813 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.792691+0000", 2020-09-08T16:13:22.813 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015304699999999999 2020-09-08T16:13:22.813 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.814 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.814 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:13:22.814 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.792844+0000", 2020-09-08T16:13:22.814 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014074199999999999 2020-09-08T16:13:22.815 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.815 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.815 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 9", 2020-09-08T16:13:22.815 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.792984+0000", 2020-09-08T16:13:22.816 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.011311473000000001 2020-09-08T16:13:22.816 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.816 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.816 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:13:22.817 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.804296+0000", 2020-09-08T16:13:22.817 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.020528401000000002 2020-09-08T16:13:22.817 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:13:22.817 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:13:22.817 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:13:22.818 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.818 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.818 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.26587.0:273857 9.1 9:8e84706e:::benchmark_data_smithi166_56754_object133824:head [delete] snapc 0=[] ondisk+write+known_if_redirected e767)", 2020-09-08T16:13:22.818 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:13:22.796177+0000", 2020-09-08T16:13:22.818 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.010858130000000001, 2020-09-08T16:13:22.819 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010943035, 2020-09-08T16:13:22.819 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:13:22.819 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:13:22.819 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:13:22.820 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.26587", 2020-09-08T16:13:22.820 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:13:22.820 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 273857 2020-09-08T16:13:22.820 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.820 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:13:22.821 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.821 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:13:22.821 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.796177+0000", 2020-09-08T16:13:22.821 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:13:22.821 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.822 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.829 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:13:22.829 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.796177+0000", 2020-09-08T16:13:22.829 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999604 2020-09-08T16:13:22.829 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.830 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.830 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:13:22.830 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.796138+0000", 2020-09-08T16:13:22.830 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8681e-05 2020-09-08T16:13:22.830 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.831 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.831 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:13:22.831 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.796227+0000", 2020-09-08T16:13:22.831 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3436e-05 2020-09-08T16:13:22.832 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.832 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.832 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:13:22.832 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.796240+0000", 2020-09-08T16:13:22.833 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7620000000000001e-06 2020-09-08T16:13:22.833 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.833 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.833 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:13:22.834 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.796246+0000", 2020-09-08T16:13:22.834 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0016570630000000001 2020-09-08T16:13:22.834 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.834 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.835 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:13:22.835 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.797903+0000", 2020-09-08T16:13:22.835 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016051699999999999 2020-09-08T16:13:22.835 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.835 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.836 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:13:22.836 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.798063+0000", 2020-09-08T16:13:22.836 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016499200000000001 2020-09-08T16:13:22.836 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.836 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.837 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 9", 2020-09-08T16:13:22.837 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.798228+0000", 2020-09-08T16:13:22.837 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0060749669999999997 2020-09-08T16:13:22.837 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.838 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.838 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2020-09-08T16:13:22.839 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.804303+0000", 2020-09-08T16:13:22.839 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0081258860000000006 2020-09-08T16:13:22.839 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:13:22.839 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:13:22.840 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:13:22.840 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.840 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.840 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.26587.0:273862 9.1 9:8f8103ae:::benchmark_data_smithi166_56754_object133829:head [delete] snapc 0=[] ondisk+write+known_if_redirected e767)", 2020-09-08T16:13:22.841 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:13:22.802958+0000", 2020-09-08T16:13:22.841 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0040775630000000002, 2020-09-08T16:13:22.842 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0042134470000000004, 2020-09-08T16:13:22.842 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:13:22.842 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2020-09-08T16:13:22.842 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:13:22.843 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.26587", 2020-09-08T16:13:22.843 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:13:22.843 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 273862 2020-09-08T16:13:22.843 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.844 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:13:22.844 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.844 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:13:22.845 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.802958+0000", 2020-09-08T16:13:22.845 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:13:22.845 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.845 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.845 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:13:22.846 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.802958+0000", 2020-09-08T16:13:22.846 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999728 2020-09-08T16:13:22.846 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.846 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.847 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:13:22.847 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.802931+0000", 2020-09-08T16:13:22.847 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4952999999999997e-05 2020-09-08T16:13:22.847 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.847 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.848 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:13:22.848 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.803026+0000", 2020-09-08T16:13:22.848 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3410000000000001e-05 2020-09-08T16:13:22.848 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.848 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.849 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:13:22.849 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.803049+0000", 2020-09-08T16:13:22.849 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8549999999999997e-06 2020-09-08T16:13:22.850 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.850 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.850 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:13:22.850 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.803059+0000", 2020-09-08T16:13:22.851 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0012741409999999999 2020-09-08T16:13:22.851 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.851 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.851 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2020-09-08T16:13:22.851 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.804333+0000", 2020-09-08T16:13:22.852 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018988300000000001 2020-09-08T16:13:22.852 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.852 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.852 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2020-09-08T16:13:22.853 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.804523+0000", 2020-09-08T16:13:22.853 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018979500000000001 2020-09-08T16:13:22.853 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.853 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.853 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 9", 2020-09-08T16:13:22.854 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.804713+0000", 2020-09-08T16:13:22.854 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001754893 2020-09-08T16:13:22.854 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:13:22.854 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:13:22.855 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:13:22.855 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.855 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.855 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.26587.0:273864 9.1 9.25d17511 (undecoded) ondisk+write+known_if_redirected e767)", 2020-09-08T16:13:22.855 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2020-09-08T16:13:22.804482+0000", 2020-09-08T16:13:22.856 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0025538650000000002, 2020-09-08T16:13:22.856 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0027351319999999999, 2020-09-08T16:13:22.856 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2020-09-08T16:13:22.856 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "queued for pg", 2020-09-08T16:13:22.857 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2020-09-08T16:13:22.857 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.26587", 2020-09-08T16:13:22.857 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.166:0/2425728689", 2020-09-08T16:13:22.857 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 273864 2020-09-08T16:13:22.858 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.858 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2020-09-08T16:13:22.858 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.859 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2020-09-08T16:13:22.859 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.804482+0000", 2020-09-08T16:13:22.859 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2020-09-08T16:13:22.859 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.860 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.860 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2020-09-08T16:13:22.860 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.804482+0000", 2020-09-08T16:13:22.860 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999747 2020-09-08T16:13:22.860 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.861 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.861 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2020-09-08T16:13:22.861 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.804456+0000", 2020-09-08T16:13:22.861 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4719000000000001e-05 2020-09-08T16:13:22.861 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.862 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.862 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2020-09-08T16:13:22.862 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.804531+0000", 2020-09-08T16:13:22.862 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5249999999999994e-06 2020-09-08T16:13:22.862 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.863 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.863 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2020-09-08T16:13:22.863 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.804541+0000", 2020-09-08T16:13:22.863 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4000000000000002e-06 2020-09-08T16:13:22.863 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2020-09-08T16:13:22.864 INFO:teuthology.orchestra.run.smithi104.stdout: { 2020-09-08T16:13:22.864 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2020-09-08T16:13:22.864 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2020-09-08T16:13:22.804545+0000", 2020-09-08T16:13:22.864 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3237000000000005e-05 2020-09-08T16:13:22.864 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:13:22.865 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2020-09-08T16:13:22.865 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:13:22.865 INFO:teuthology.orchestra.run.smithi104.stdout: } 2020-09-08T16:13:22.865 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2020-09-08T16:13:22.866 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 4 2020-09-08T16:13:22.866 INFO:teuthology.orchestra.run.smithi104.stdout:} 2020-09-08T16:13:22.914 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:13:23.018 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:13:23.175 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:13:22 smithi104 bash[17588]: debug 2020-09-08T16:13:22.915+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:23.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:23 smithi104 bash[9203]: cluster 2020-09-08T16:13:22.113845+0000 mon.a (mon.0) 2359 : cluster [DBG] osdmap e767: 12 total, 12 up, 12 in 2020-09-08T16:13:23.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:23 smithi104 bash[9203]: cluster 2020-09-08T16:13:22.170073+0000 mgr.x (mgr.24118) 1381 : cluster [DBG] pgmap v2064: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 653 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 559 op/s 2020-09-08T16:13:23.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:23 smithi104 bash[9203]: cluster 2020-09-08T16:13:22.197135+0000 osd.0 (osd.0) 134 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:13:23.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:23 smithi104 bash[9203]: cluster 2020-09-08T16:13:22.197228+0000 osd.0 (osd.0) 135 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:13:23.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:23 smithi166 bash[9446]: cluster 2020-09-08T16:13:22.113845+0000 mon.a (mon.0) 2359 : cluster [DBG] osdmap e767: 12 total, 12 up, 12 in 2020-09-08T16:13:23.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:23 smithi166 bash[9446]: cluster 2020-09-08T16:13:22.170073+0000 mgr.x (mgr.24118) 1381 : cluster [DBG] pgmap v2064: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 653 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 559 op/s 2020-09-08T16:13:23.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:23 smithi166 bash[9446]: cluster 2020-09-08T16:13:22.197135+0000 osd.0 (osd.0) 134 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:13:23.367 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:23 smithi166 bash[9446]: cluster 2020-09-08T16:13:22.197228+0000 osd.0 (osd.0) 135 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:13:23.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:23 smithi055 bash[9296]: cluster 2020-09-08T16:13:22.113845+0000 mon.a (mon.0) 2359 : cluster [DBG] osdmap e767: 12 total, 12 up, 12 in 2020-09-08T16:13:23.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:23 smithi055 bash[9296]: cluster 2020-09-08T16:13:22.170073+0000 mgr.x (mgr.24118) 1381 : cluster [DBG] pgmap v2064: 17 pgs: 1 active+clean+scrubbing+deep, 16 active+clean; 653 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 559 op/s 2020-09-08T16:13:23.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:23 smithi055 bash[9296]: cluster 2020-09-08T16:13:22.197135+0000 osd.0 (osd.0) 134 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:13:23.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:23 smithi055 bash[9296]: cluster 2020-09-08T16:13:22.197228+0000 osd.0 (osd.0) 135 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:13:24.279 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:13:24.366 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:24 smithi166 bash[9446]: cluster 2020-09-08T16:13:23.120166+0000 mon.a (mon.0) 2360 : cluster [DBG] osdmap e768: 12 total, 12 up, 12 in 2020-09-08T16:13:24.381 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:13:24.390 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:24 smithi055 bash[9296]: cluster 2020-09-08T16:13:23.120166+0000 mon.a (mon.0) 2360 : cluster [DBG] osdmap e768: 12 total, 12 up, 12 in 2020-09-08T16:13:24.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:24 smithi104 bash[9203]: cluster 2020-09-08T16:13:23.120166+0000 mon.a (mon.0) 2360 : cluster [DBG] osdmap e768: 12 total, 12 up, 12 in 2020-09-08T16:13:24.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:13:24 smithi104 bash[15273]: debug 2020-09-08T16:13:24.275+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:24.512 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:13:24.616 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:13:24.657 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:13:24 smithi055 bash[16646]: debug 2020-09-08T16:13:24.510+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:25.143 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2020-09-08T16:13:25.302 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:13:25.384 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:13:25.386 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:25 smithi104 bash[9203]: cluster 2020-09-08T16:13:24.170518+0000 mgr.x (mgr.24118) 1382 : cluster [DBG] pgmap v2066: 17 pgs: 17 active+clean; 403 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 996 op/s 2020-09-08T16:13:25.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:25 smithi055 bash[9296]: cluster 2020-09-08T16:13:24.170518+0000 mgr.x (mgr.24118) 1382 : cluster [DBG] pgmap v2066: 17 pgs: 17 active+clean; 403 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 996 op/s 2020-09-08T16:13:25.488 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:13:25.500 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:25 smithi166 bash[9446]: cluster 2020-09-08T16:13:24.170518+0000 mgr.x (mgr.24118) 1382 : cluster [DBG] pgmap v2066: 17 pgs: 17 active+clean; 403 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 996 op/s 2020-09-08T16:13:25.641 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:13:25.676 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:13:25 smithi104 bash[22305]: debug 2020-09-08T16:13:25.387+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:25.750 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:13:25.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:13:25 smithi166 bash[11928]: debug 2020-09-08T16:13:25.636+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:25.891 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:13:25.995 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:13:26.124 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:13:25 smithi055 bash[16646]: debug 2020-09-08T16:13:25.890+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:26.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:26 smithi055 bash[9296]: audit 2020-09-08T16:13:25.135778+0000 mon.c (mon.2) 96 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:13:26.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:26 smithi055 bash[9296]: audit 2020-09-08T16:13:25.138691+0000 mon.a (mon.0) 2361 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:13:26.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:26 smithi055 bash[9296]: audit 2020-09-08T16:13:25.162990+0000 mon.a (mon.0) 2362 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:13:26.409 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:26 smithi104 bash[9203]: audit 2020-09-08T16:13:25.135778+0000 mon.c (mon.2) 96 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:13:26.409 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:26 smithi104 bash[9203]: audit 2020-09-08T16:13:25.138691+0000 mon.a (mon.0) 2361 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:13:26.409 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:26 smithi104 bash[9203]: audit 2020-09-08T16:13:25.162990+0000 mon.a (mon.0) 2362 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:13:26.412 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:13:26.518 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:13:26.526 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:26 smithi166 bash[9446]: audit 2020-09-08T16:13:25.135778+0000 mon.c (mon.2) 96 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:13:26.527 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:26 smithi166 bash[9446]: audit 2020-09-08T16:13:25.138691+0000 mon.a (mon.0) 2361 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:13:26.527 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:26 smithi166 bash[9446]: audit 2020-09-08T16:13:25.162990+0000 mon.a (mon.0) 2362 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:13:26.668 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:13:26.684 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:13:26 smithi104 bash[19922]: debug 2020-09-08T16:13:26.411+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:26.775 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:13:26.783 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:13:26 smithi166 bash[14254]: debug 2020-09-08T16:13:26.664+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:26.915 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:13:27.020 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:13:27.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:13:26 smithi166 bash[14254]: debug 2020-09-08T16:13:26.912+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:27.130 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:27 smithi104 bash[9203]: cluster 2020-09-08T16:13:26.171103+0000 mgr.x (mgr.24118) 1383 : cluster [DBG] pgmap v2067: 17 pgs: 17 active+clean; 362 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 833 op/s 2020-09-08T16:13:27.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:27 smithi055 bash[9296]: cluster 2020-09-08T16:13:26.171103+0000 mgr.x (mgr.24118) 1383 : cluster [DBG] pgmap v2067: 17 pgs: 17 active+clean; 362 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 833 op/s 2020-09-08T16:13:27.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:27 smithi166 bash[9446]: cluster 2020-09-08T16:13:26.171103+0000 mgr.x (mgr.24118) 1383 : cluster [DBG] pgmap v2067: 17 pgs: 17 active+clean; 362 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 833 op/s 2020-09-08T16:13:27.671 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:13:27.775 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:13:27.925 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:13:27 smithi104 bash[15273]: debug 2020-09-08T16:13:27.671+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:13:28.142 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is set 2020-09-08T16:27:02.019 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:28 smithi166 bash[9446]: audit 2020-09-08T16:13:28.071148+0000 mon.a (mon.0) 2363 : audit [INF] from='client.? v1:172.21.15.104:0/622774130' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:27:02.021 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:29 smithi166 bash[9446]: cluster 2020-09-08T16:13:28.135669+0000 mon.a (mon.0) 2364 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:27:02.021 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:29 smithi166 bash[9446]: audit 2020-09-08T16:13:28.143990+0000 mon.a (mon.0) 2365 : audit [INF] from='client.? v1:172.21.15.104:0/622774130' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:27:02.021 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:29 smithi166 bash[9446]: cluster 2020-09-08T16:13:28.144039+0000 mon.a (mon.0) 2366 : cluster [DBG] osdmap e769: 12 total, 12 up, 12 in 2020-09-08T16:27:02.021 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:29 smithi166 bash[9446]: cluster 2020-09-08T16:13:28.171567+0000 mgr.x (mgr.24118) 1384 : cluster [DBG] pgmap v2069: 17 pgs: 17 active+clean; 208 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.17k op/s 2020-09-08T16:27:02.022 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:31 smithi166 bash[9446]: cluster 2020-09-08T16:13:30.172360+0000 mgr.x (mgr.24118) 1385 : cluster [DBG] pgmap v2070: 17 pgs: 17 active+clean; 64 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.18k op/s 2020-09-08T16:27:02.022 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:32 smithi166 bash[9446]: audit 2020-09-08T16:13:31.118278+0000 mon.a (mon.0) 2367 : audit [DBG] from='client.? v1:172.21.15.104:0/3128233918' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:27:02.022 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:32 smithi166 bash[9446]: cluster 2020-09-08T16:13:31.359446+0000 osd.5 (osd.5) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.023 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:32 smithi166 bash[9446]: cluster 2020-09-08T16:13:31.359582+0000 osd.5 (osd.5) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.023 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:33 smithi166 bash[9446]: cluster 2020-09-08T16:13:32.124175+0000 mon.a (mon.0) 2368 : cluster [DBG] osdmap e770: 12 total, 12 up, 12 in 2020-09-08T16:27:02.023 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:33 smithi166 bash[9446]: cluster 2020-09-08T16:13:32.173034+0000 mgr.x (mgr.24118) 1386 : cluster [DBG] pgmap v2072: 17 pgs: 17 active+clean; 64 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 677 op/s 2020-09-08T16:27:02.023 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:34 smithi166 bash[9446]: cluster 2020-09-08T16:13:33.798526+0000 osd.10 (osd.10) 119 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.024 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:34 smithi166 bash[9446]: cluster 2020-09-08T16:13:33.798648+0000 osd.10 (osd.10) 120 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.024 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:35 smithi166 bash[9446]: cluster 2020-09-08T16:13:34.142489+0000 mon.a (mon.0) 2369 : cluster [DBG] osdmap e771: 12 total, 12 up, 12 in 2020-09-08T16:27:02.024 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:35 smithi166 bash[9446]: cluster 2020-09-08T16:13:34.173538+0000 mgr.x (mgr.24118) 1387 : cluster [DBG] pgmap v2074: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 552 op/s 2020-09-08T16:27:02.024 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:35 smithi166 bash[9446]: cluster 2020-09-08T16:13:34.798777+0000 osd.3 (osd.3) 143 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.024 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:35 smithi166 bash[9446]: cluster 2020-09-08T16:13:34.798895+0000 osd.3 (osd.3) 144 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.025 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:36 smithi166 bash[9446]: cluster 2020-09-08T16:13:35.150774+0000 mon.a (mon.0) 2370 : cluster [DBG] osdmap e772: 12 total, 12 up, 12 in 2020-09-08T16:27:02.025 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:37 smithi166 bash[9446]: cluster 2020-09-08T16:13:36.174104+0000 mgr.x (mgr.24118) 1388 : cluster [DBG] pgmap v2076: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 169 op/s 2020-09-08T16:27:02.025 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:39 smithi166 bash[9446]: cluster 2020-09-08T16:13:38.174794+0000 mgr.x (mgr.24118) 1389 : cluster [DBG] pgmap v2077: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 168 op/s 2020-09-08T16:27:02.025 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:41 smithi166 bash[9446]: cluster 2020-09-08T16:13:40.175536+0000 mgr.x (mgr.24118) 1390 : cluster [DBG] pgmap v2078: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 127 op/s 2020-09-08T16:27:02.025 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:43 smithi166 bash[9446]: cluster 2020-09-08T16:13:42.157827+0000 osd.8 (osd.8) 114 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.026 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:43 smithi166 bash[9446]: cluster 2020-09-08T16:13:42.157956+0000 osd.8 (osd.8) 115 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.026 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:43 smithi166 bash[9446]: cluster 2020-09-08T16:13:42.176237+0000 mgr.x (mgr.24118) 1391 : cluster [DBG] pgmap v2079: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.026 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:43 smithi166 bash[9446]: cluster 2020-09-08T16:13:42.218579+0000 mon.a (mon.0) 2371 : cluster [DBG] osdmap e773: 12 total, 12 up, 12 in 2020-09-08T16:27:02.026 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:45 smithi166 bash[9446]: cluster 2020-09-08T16:13:44.176996+0000 mgr.x (mgr.24118) 1392 : cluster [DBG] pgmap v2081: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.026 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:47 smithi166 bash[9446]: cluster 2020-09-08T16:13:46.177663+0000 mgr.x (mgr.24118) 1393 : cluster [DBG] pgmap v2082: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.027 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:49 smithi166 bash[9446]: cluster 2020-09-08T16:13:48.178391+0000 mgr.x (mgr.24118) 1394 : cluster [DBG] pgmap v2083: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.027 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:50 smithi166 bash[9446]: cluster 2020-09-08T16:13:49.567955+0000 osd.1 (osd.1) 160 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.027 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:50 smithi166 bash[9446]: cluster 2020-09-08T16:13:49.568088+0000 osd.1 (osd.1) 161 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.027 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:51 smithi166 bash[9446]: cluster 2020-09-08T16:13:50.179131+0000 mgr.x (mgr.24118) 1395 : cluster [DBG] pgmap v2084: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.027 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:51 smithi166 bash[9446]: cluster 2020-09-08T16:13:50.244784+0000 mon.a (mon.0) 2372 : cluster [DBG] osdmap e774: 12 total, 12 up, 12 in 2020-09-08T16:27:02.028 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:53 smithi166 bash[9446]: cluster 2020-09-08T16:13:52.179873+0000 mgr.x (mgr.24118) 1396 : cluster [DBG] pgmap v2086: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.028 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:54 smithi166 bash[9446]: cluster 2020-09-08T16:13:53.091527+0000 osd.6 (osd.6) 125 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.028 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:54 smithi166 bash[9446]: cluster 2020-09-08T16:13:53.091672+0000 osd.6 (osd.6) 126 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.028 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:54 smithi166 bash[9446]: cluster 2020-09-08T16:13:53.253232+0000 mon.a (mon.0) 2373 : cluster [DBG] osdmap e775: 12 total, 12 up, 12 in 2020-09-08T16:27:02.028 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:55 smithi166 bash[9446]: cluster 2020-09-08T16:13:54.180617+0000 mgr.x (mgr.24118) 1397 : cluster [DBG] pgmap v2088: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.029 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:57 smithi166 bash[9446]: cluster 2020-09-08T16:13:56.181327+0000 mgr.x (mgr.24118) 1398 : cluster [DBG] pgmap v2089: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.029 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:59 smithi166 bash[9446]: cluster 2020-09-08T16:13:58.182024+0000 mgr.x (mgr.24118) 1399 : cluster [DBG] pgmap v2090: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.029 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:59 smithi166 bash[9446]: cluster 2020-09-08T16:13:58.256754+0000 osd.11 (osd.11) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.029 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:59 smithi166 bash[9446]: cluster 2020-09-08T16:13:58.256880+0000 osd.11 (osd.11) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.029 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:13:59 smithi166 bash[9446]: cluster 2020-09-08T16:13:58.280736+0000 mon.a (mon.0) 2374 : cluster [DBG] osdmap e776: 12 total, 12 up, 12 in 2020-09-08T16:27:02.030 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:01 smithi166 bash[9446]: cluster 2020-09-08T16:14:00.182737+0000 mgr.x (mgr.24118) 1400 : cluster [DBG] pgmap v2092: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.030 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:03 smithi166 bash[9446]: cluster 2020-09-08T16:14:02.183359+0000 mgr.x (mgr.24118) 1401 : cluster [DBG] pgmap v2093: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.030 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:04 smithi166 bash[9446]: cluster 2020-09-08T16:14:02.929875+0000 osd.4 (osd.4) 85 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.030 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:04 smithi166 bash[9446]: cluster 2020-09-08T16:14:02.930001+0000 osd.4 (osd.4) 86 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.031 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:04 smithi166 bash[9446]: cluster 2020-09-08T16:14:03.251582+0000 mon.a (mon.0) 2375 : cluster [DBG] osdmap e777: 12 total, 12 up, 12 in 2020-09-08T16:27:02.031 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:05 smithi166 bash[9446]: cluster 2020-09-08T16:14:04.184018+0000 mgr.x (mgr.24118) 1402 : cluster [DBG] pgmap v2095: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.031 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:07 smithi166 bash[9446]: cluster 2020-09-08T16:14:06.184699+0000 mgr.x (mgr.24118) 1403 : cluster [DBG] pgmap v2096: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.031 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:09 smithi166 bash[9446]: cluster 2020-09-08T16:14:08.185381+0000 mgr.x (mgr.24118) 1404 : cluster [DBG] pgmap v2097: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.032 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:11 smithi166 bash[9446]: cluster 2020-09-08T16:14:10.186089+0000 mgr.x (mgr.24118) 1405 : cluster [DBG] pgmap v2098: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.032 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:12 smithi166 bash[9446]: cluster 2020-09-08T16:14:11.817319+0000 osd.9 (osd.9) 124 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.032 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:12 smithi166 bash[9446]: cluster 2020-09-08T16:14:11.817442+0000 osd.9 (osd.9) 125 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.032 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:13 smithi166 bash[9446]: cluster 2020-09-08T16:14:12.130806+0000 mon.a (mon.0) 2376 : cluster [DBG] osdmap e778: 12 total, 12 up, 12 in 2020-09-08T16:27:02.032 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:13 smithi166 bash[9446]: cluster 2020-09-08T16:14:12.186702+0000 mgr.x (mgr.24118) 1406 : cluster [DBG] pgmap v2100: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.033 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:14 smithi166 bash[9446]: cluster 2020-09-08T16:14:13.940062+0000 osd.2 (osd.2) 120 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.033 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:14 smithi166 bash[9446]: cluster 2020-09-08T16:14:13.940175+0000 osd.2 (osd.2) 121 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.033 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:15 smithi166 bash[9446]: cluster 2020-09-08T16:14:14.152050+0000 mon.a (mon.0) 2377 : cluster [DBG] osdmap e779: 12 total, 12 up, 12 in 2020-09-08T16:27:02.033 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:15 smithi166 bash[9446]: cluster 2020-09-08T16:14:14.187344+0000 mgr.x (mgr.24118) 1407 : cluster [DBG] pgmap v2102: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.033 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:17 smithi166 bash[9446]: cluster 2020-09-08T16:14:16.188022+0000 mgr.x (mgr.24118) 1408 : cluster [DBG] pgmap v2103: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.034 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:19 smithi166 bash[9446]: cluster 2020-09-08T16:14:18.188661+0000 mgr.x (mgr.24118) 1409 : cluster [DBG] pgmap v2104: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.034 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:21 smithi166 bash[9446]: cluster 2020-09-08T16:14:20.189351+0000 mgr.x (mgr.24118) 1410 : cluster [DBG] pgmap v2105: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.034 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:23 smithi166 bash[9446]: cluster 2020-09-08T16:14:22.190000+0000 mgr.x (mgr.24118) 1411 : cluster [DBG] pgmap v2106: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.034 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:23 smithi166 bash[9446]: cluster 2020-09-08T16:14:22.297120+0000 osd.7 (osd.7) 176 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.034 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:23 smithi166 bash[9446]: cluster 2020-09-08T16:14:22.297244+0000 osd.7 (osd.7) 177 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.035 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:23 smithi166 bash[9446]: cluster 2020-09-08T16:14:22.341740+0000 osd.0 (osd.0) 136 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.035 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:23 smithi166 bash[9446]: cluster 2020-09-08T16:14:22.341874+0000 osd.0 (osd.0) 137 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.035 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:24 smithi166 bash[9446]: cluster 2020-09-08T16:14:23.260670+0000 mon.a (mon.0) 2378 : cluster [DBG] osdmap e780: 12 total, 12 up, 12 in 2020-09-08T16:27:02.035 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:25 smithi166 bash[9446]: cluster 2020-09-08T16:14:24.190685+0000 mgr.x (mgr.24118) 1412 : cluster [DBG] pgmap v2108: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.035 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:27 smithi166 bash[9446]: cluster 2020-09-08T16:14:26.191372+0000 mgr.x (mgr.24118) 1413 : cluster [DBG] pgmap v2109: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.036 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:29 smithi166 bash[9446]: cluster 2020-09-08T16:14:28.192165+0000 mgr.x (mgr.24118) 1414 : cluster [DBG] pgmap v2110: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.036 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:31 smithi166 bash[9446]: cluster 2020-09-08T16:14:30.192746+0000 mgr.x (mgr.24118) 1415 : cluster [DBG] pgmap v2111: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.036 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:32 smithi166 bash[9446]: cluster 2020-09-08T16:14:31.379316+0000 osd.5 (osd.5) 142 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.036 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:32 smithi166 bash[9446]: cluster 2020-09-08T16:14:31.379453+0000 osd.5 (osd.5) 143 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.036 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:33 smithi166 bash[9446]: cluster 2020-09-08T16:14:32.130605+0000 mon.a (mon.0) 2379 : cluster [DBG] osdmap e781: 12 total, 12 up, 12 in 2020-09-08T16:27:02.037 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:33 smithi166 bash[9446]: cluster 2020-09-08T16:14:32.193317+0000 mgr.x (mgr.24118) 1416 : cluster [DBG] pgmap v2113: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.037 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:35 smithi166 bash[9446]: cluster 2020-09-08T16:14:34.194070+0000 mgr.x (mgr.24118) 1417 : cluster [DBG] pgmap v2114: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.037 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:35 smithi166 bash[9446]: cluster 2020-09-08T16:14:34.646280+0000 osd.10 (osd.10) 121 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.037 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:35 smithi166 bash[9446]: cluster 2020-09-08T16:14:34.646401+0000 osd.10 (osd.10) 122 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.037 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:36 smithi166 bash[9446]: cluster 2020-09-08T16:14:35.262897+0000 mon.a (mon.0) 2380 : cluster [DBG] osdmap e782: 12 total, 12 up, 12 in 2020-09-08T16:27:02.038 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:36 smithi166 bash[9446]: cluster 2020-09-08T16:14:35.622122+0000 osd.3 (osd.3) 145 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.038 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:36 smithi166 bash[9446]: cluster 2020-09-08T16:14:35.622231+0000 osd.3 (osd.3) 146 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.038 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:37 smithi166 bash[9446]: cluster 2020-09-08T16:14:36.194742+0000 mgr.x (mgr.24118) 1418 : cluster [DBG] pgmap v2116: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.038 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:37 smithi166 bash[9446]: cluster 2020-09-08T16:14:36.272137+0000 mon.a (mon.0) 2381 : cluster [DBG] osdmap e783: 12 total, 12 up, 12 in 2020-09-08T16:27:02.038 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:39 smithi166 bash[9446]: cluster 2020-09-08T16:14:38.195568+0000 mgr.x (mgr.24118) 1419 : cluster [DBG] pgmap v2118: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.039 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:41 smithi166 bash[9446]: cluster 2020-09-08T16:14:40.196276+0000 mgr.x (mgr.24118) 1420 : cluster [DBG] pgmap v2119: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.039 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:43 smithi166 bash[9446]: cluster 2020-09-08T16:14:42.196906+0000 mgr.x (mgr.24118) 1421 : cluster [DBG] pgmap v2120: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.039 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:44 smithi166 bash[9446]: cluster 2020-09-08T16:14:43.080311+0000 osd.8 (osd.8) 116 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.039 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:44 smithi166 bash[9446]: cluster 2020-09-08T16:14:43.080440+0000 osd.8 (osd.8) 117 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.040 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:44 smithi166 bash[9446]: cluster 2020-09-08T16:14:43.259076+0000 mon.a (mon.0) 2382 : cluster [DBG] osdmap e784: 12 total, 12 up, 12 in 2020-09-08T16:27:02.040 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:45 smithi166 bash[9446]: cluster 2020-09-08T16:14:44.197556+0000 mgr.x (mgr.24118) 1422 : cluster [DBG] pgmap v2122: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.040 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:47 smithi166 bash[9446]: cluster 2020-09-08T16:14:46.198236+0000 mgr.x (mgr.24118) 1423 : cluster [DBG] pgmap v2123: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.040 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:48 smithi166 bash[9446]: audit 2020-09-08T16:14:48.568197+0000 mon.b (mon.1) 125 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:14:48.567054\"}"}]: dispatch 2020-09-08T16:27:02.041 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:48 smithi166 bash[9446]: audit 2020-09-08T16:14:48.570330+0000 mon.a (mon.0) 2383 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:14:48.567054\"}"}]: dispatch 2020-09-08T16:27:02.041 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:48 smithi166 bash[9446]: audit 2020-09-08T16:14:48.576218+0000 mon.a (mon.0) 2384 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:14:48.567054\"}"}]': finished 2020-09-08T16:27:02.041 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:48 smithi166 bash[9446]: audit 2020-09-08T16:14:48.576596+0000 mon.b (mon.1) 126 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:27:02.042 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:48 smithi166 bash[9446]: audit 2020-09-08T16:14:48.603681+0000 mon.b (mon.1) 127 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:14:48.602227\"}"}]: dispatch 2020-09-08T16:27:02.042 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:48 smithi166 bash[9446]: audit 2020-09-08T16:14:48.605772+0000 mon.a (mon.0) 2385 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:14:48.602227\"}"}]: dispatch 2020-09-08T16:27:02.043 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:48 smithi166 bash[9446]: audit 2020-09-08T16:14:48.610350+0000 mon.a (mon.0) 2386 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:14:48.602227\"}"}]': finished 2020-09-08T16:27:02.043 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:48 smithi166 bash[9446]: audit 2020-09-08T16:14:48.611699+0000 mon.b (mon.1) 128 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:27:02.043 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:49 smithi166 bash[9446]: audit 2020-09-08T16:14:48.637950+0000 mon.b (mon.1) 129 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:14:48.636423\"}"}]: dispatch 2020-09-08T16:27:02.044 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:49 smithi166 bash[9446]: audit 2020-09-08T16:14:48.639785+0000 mon.a (mon.0) 2387 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:14:48.636423\"}"}]: dispatch 2020-09-08T16:27:02.044 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:49 smithi166 bash[9446]: audit 2020-09-08T16:14:48.644222+0000 mon.a (mon.0) 2388 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:14:48.636423\"}"}]': finished 2020-09-08T16:27:02.045 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:49 smithi166 bash[9446]: audit 2020-09-08T16:14:48.644329+0000 mon.b (mon.1) 130 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:27:02.045 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:49 smithi166 bash[9446]: audit 2020-09-08T16:14:49.407449+0000 mon.b (mon.1) 131 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.402933\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403132\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403786\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404332\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404856\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.405550\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:14:48.602227\"}"}]: dispatch 2020-09-08T16:27:02.045 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:49 smithi166 bash[9446]: audit 2020-09-08T16:14:49.409621+0000 mon.a (mon.0) 2389 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.402933\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403132\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403786\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404332\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404856\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.405550\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:14:48.602227\"}"}]: dispatch 2020-09-08T16:27:02.046 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:49 smithi166 bash[9446]: audit 2020-09-08T16:14:49.414676+0000 mon.a (mon.0) 2390 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.402933\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403132\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403786\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404332\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404856\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.405550\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:14:48.602227\"}"}]': finished 2020-09-08T16:27:02.046 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:50 smithi166 bash[9446]: audit 2020-09-08T16:14:49.688909+0000 mon.b (mon.1) 132 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685123\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685289\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685873\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686336\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686892\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.687437\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:14:48.567054\"}"}]: dispatch 2020-09-08T16:27:02.047 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:50 smithi166 bash[9446]: audit 2020-09-08T16:14:49.690911+0000 mon.a (mon.0) 2391 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685123\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685289\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685873\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686336\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686892\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.687437\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:14:48.567054\"}"}]: dispatch 2020-09-08T16:27:02.047 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:50 smithi166 bash[9446]: audit 2020-09-08T16:14:49.695568+0000 mon.a (mon.0) 2392 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685123\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685289\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685873\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686336\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686892\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.687437\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:14:48.567054\"}"}]': finished 2020-09-08T16:27:02.048 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:50 smithi166 bash[9446]: audit 2020-09-08T16:14:49.701035+0000 mon.b (mon.1) 133 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698864\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698956\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699006\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699331\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699539\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699740\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.700002\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:14:48.636423\"}"}]: dispatch 2020-09-08T16:27:02.048 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:50 smithi166 bash[9446]: audit 2020-09-08T16:14:49.703174+0000 mon.a (mon.0) 2393 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698864\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698956\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699006\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699331\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699539\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699740\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.700002\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:14:48.636423\"}"}]: dispatch 2020-09-08T16:27:02.048 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:50 smithi166 bash[9446]: audit 2020-09-08T16:14:49.709043+0000 mon.a (mon.0) 2394 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698864\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698956\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699006\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699331\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699539\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699740\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.700002\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:14:48.636423\"}"}]': finished 2020-09-08T16:27:02.049 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:50 smithi166 bash[9446]: audit 2020-09-08T16:14:49.710077+0000 mon.b (mon.1) 134 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.049 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:50 smithi166 bash[9446]: audit 2020-09-08T16:14:49.711734+0000 mon.a (mon.0) 2395 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.049 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:50 smithi166 bash[9446]: audit 2020-09-08T16:14:49.716978+0000 mon.a (mon.0) 2396 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.049 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:50 smithi166 bash[9446]: audit 2020-09-08T16:14:49.719847+0000 mon.b (mon.1) 135 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.050 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:50 smithi166 bash[9446]: audit 2020-09-08T16:14:49.721369+0000 mon.a (mon.0) 2397 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.050 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:50 smithi166 bash[9446]: audit 2020-09-08T16:14:49.726604+0000 mon.a (mon.0) 2398 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.050 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:51 smithi166 bash[9446]: cluster 2020-09-08T16:14:50.199752+0000 mgr.x (mgr.24118) 1425 : cluster [DBG] pgmap v2125: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.050 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:51 smithi166 bash[9446]: cluster 2020-09-08T16:14:50.324087+0000 osd.1 (osd.1) 162 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.050 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:51 smithi166 bash[9446]: cluster 2020-09-08T16:14:50.324226+0000 osd.1 (osd.1) 163 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.051 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:51 smithi166 bash[9446]: cluster 2020-09-08T16:14:51.140484+0000 mon.a (mon.0) 2399 : cluster [DBG] osdmap e785: 12 total, 12 up, 12 in 2020-09-08T16:27:02.051 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:53 smithi166 bash[9446]: cluster 2020-09-08T16:14:52.200469+0000 mgr.x (mgr.24118) 1426 : cluster [DBG] pgmap v2127: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.051 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:55 smithi166 bash[9446]: cluster 2020-09-08T16:14:53.964831+0000 osd.6 (osd.6) 127 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.051 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:55 smithi166 bash[9446]: cluster 2020-09-08T16:14:53.964994+0000 osd.6 (osd.6) 128 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.051 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:55 smithi166 bash[9446]: cluster 2020-09-08T16:14:54.201201+0000 mgr.x (mgr.24118) 1427 : cluster [DBG] pgmap v2128: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.052 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:55 smithi166 bash[9446]: cluster 2020-09-08T16:14:54.656765+0000 mon.a (mon.0) 2400 : cluster [DBG] osdmap e786: 12 total, 12 up, 12 in 2020-09-08T16:27:02.052 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:55 smithi166 bash[9446]: audit 2020-09-08T16:14:55.398925+0000 mon.c (mon.2) 97 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.052 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:55 smithi166 bash[9446]: audit 2020-09-08T16:14:55.402185+0000 mon.a (mon.0) 2401 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.052 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:55 smithi166 bash[9446]: audit 2020-09-08T16:14:55.408299+0000 mon.a (mon.0) 2402 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.052 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:55 smithi166 bash[9446]: audit 2020-09-08T16:14:55.411017+0000 mon.c (mon.2) 98 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.053 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:55 smithi166 bash[9446]: audit 2020-09-08T16:14:55.414225+0000 mon.a (mon.0) 2403 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.053 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:55 smithi166 bash[9446]: audit 2020-09-08T16:14:55.421107+0000 mon.a (mon.0) 2404 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.053 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:57 smithi166 bash[9446]: cluster 2020-09-08T16:14:56.201953+0000 mgr.x (mgr.24118) 1428 : cluster [DBG] pgmap v2130: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.053 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:14:59 smithi166 bash[9446]: cluster 2020-09-08T16:14:58.202734+0000 mgr.x (mgr.24118) 1429 : cluster [DBG] pgmap v2131: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.053 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:00 smithi166 bash[9446]: cluster 2020-09-08T16:14:59.199080+0000 osd.11 (osd.11) 142 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.054 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:00 smithi166 bash[9446]: cluster 2020-09-08T16:14:59.199225+0000 osd.11 (osd.11) 143 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.054 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:00 smithi166 bash[9446]: cluster 2020-09-08T16:14:59.671408+0000 mon.a (mon.0) 2405 : cluster [DBG] osdmap e787: 12 total, 12 up, 12 in 2020-09-08T16:27:02.054 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:01 smithi166 bash[9446]: cluster 2020-09-08T16:15:00.203450+0000 mgr.x (mgr.24118) 1430 : cluster [DBG] pgmap v2133: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.054 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:03 smithi166 bash[9446]: cluster 2020-09-08T16:15:02.204091+0000 mgr.x (mgr.24118) 1431 : cluster [DBG] pgmap v2134: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.054 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:03 smithi166 bash[9446]: cluster 2020-09-08T16:15:03.085020+0000 osd.4 (osd.4) 87 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.055 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:03 smithi166 bash[9446]: cluster 2020-09-08T16:15:03.085154+0000 osd.4 (osd.4) 88 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.055 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:04 smithi166 bash[9446]: cluster 2020-09-08T16:15:03.706936+0000 mon.a (mon.0) 2406 : cluster [DBG] osdmap e788: 12 total, 12 up, 12 in 2020-09-08T16:27:02.055 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:05 smithi166 bash[9446]: cluster 2020-09-08T16:15:04.204759+0000 mgr.x (mgr.24118) 1432 : cluster [DBG] pgmap v2136: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.055 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:07 smithi166 bash[9446]: cluster 2020-09-08T16:15:06.205426+0000 mgr.x (mgr.24118) 1433 : cluster [DBG] pgmap v2137: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.056 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:09 smithi166 bash[9446]: cluster 2020-09-08T16:15:08.206133+0000 mgr.x (mgr.24118) 1434 : cluster [DBG] pgmap v2138: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.056 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:11 smithi166 bash[9446]: cluster 2020-09-08T16:15:10.206827+0000 mgr.x (mgr.24118) 1435 : cluster [DBG] pgmap v2139: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.056 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:12 smithi166 bash[9446]: cluster 2020-09-08T16:15:11.845269+0000 osd.9 (osd.9) 126 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.056 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:12 smithi166 bash[9446]: cluster 2020-09-08T16:15:11.845394+0000 osd.9 (osd.9) 127 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.056 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:13 smithi166 bash[9446]: cluster 2020-09-08T16:15:12.141685+0000 mon.a (mon.0) 2407 : cluster [DBG] osdmap e789: 12 total, 12 up, 12 in 2020-09-08T16:27:02.057 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:13 smithi166 bash[9446]: cluster 2020-09-08T16:15:12.207470+0000 mgr.x (mgr.24118) 1436 : cluster [DBG] pgmap v2141: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.057 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:15 smithi166 bash[9446]: cluster 2020-09-08T16:15:14.208181+0000 mgr.x (mgr.24118) 1437 : cluster [DBG] pgmap v2142: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.057 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:15 smithi166 bash[9446]: cluster 2020-09-08T16:15:14.257343+0000 osd.2 (osd.2) 122 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.057 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:15 smithi166 bash[9446]: cluster 2020-09-08T16:15:14.257461+0000 osd.2 (osd.2) 123 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.057 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:15 smithi166 bash[9446]: cluster 2020-09-08T16:15:14.314797+0000 mon.a (mon.0) 2408 : cluster [DBG] osdmap e790: 12 total, 12 up, 12 in 2020-09-08T16:27:02.058 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:17 smithi166 bash[9446]: cluster 2020-09-08T16:15:16.208862+0000 mgr.x (mgr.24118) 1438 : cluster [DBG] pgmap v2144: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.058 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:19 smithi166 bash[9446]: cluster 2020-09-08T16:15:18.209628+0000 mgr.x (mgr.24118) 1439 : cluster [DBG] pgmap v2145: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.058 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:21 smithi166 bash[9446]: cluster 2020-09-08T16:15:20.210367+0000 mgr.x (mgr.24118) 1440 : cluster [DBG] pgmap v2146: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.058 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:23 smithi166 bash[9446]: cluster 2020-09-08T16:15:22.211026+0000 mgr.x (mgr.24118) 1441 : cluster [DBG] pgmap v2147: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.058 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:23 smithi166 bash[9446]: cluster 2020-09-08T16:15:22.554620+0000 osd.7 (osd.7) 178 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.059 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:23 smithi166 bash[9446]: cluster 2020-09-08T16:15:22.554737+0000 osd.7 (osd.7) 179 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.059 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:23 smithi166 bash[9446]: cluster 2020-09-08T16:15:22.661421+0000 osd.0 (osd.0) 138 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.059 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:23 smithi166 bash[9446]: cluster 2020-09-08T16:15:22.661556+0000 osd.0 (osd.0) 139 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.059 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:24 smithi166 bash[9446]: cluster 2020-09-08T16:15:23.285820+0000 mon.a (mon.0) 2409 : cluster [DBG] osdmap e791: 12 total, 12 up, 12 in 2020-09-08T16:27:02.059 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:25 smithi166 bash[9446]: cluster 2020-09-08T16:15:24.211726+0000 mgr.x (mgr.24118) 1442 : cluster [DBG] pgmap v2149: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.060 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:27 smithi166 bash[9446]: cluster 2020-09-08T16:15:26.212433+0000 mgr.x (mgr.24118) 1443 : cluster [DBG] pgmap v2150: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.060 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:29 smithi166 bash[9446]: cluster 2020-09-08T16:15:28.213218+0000 mgr.x (mgr.24118) 1444 : cluster [DBG] pgmap v2151: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.060 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:31 smithi166 bash[9446]: cluster 2020-09-08T16:15:30.213899+0000 mgr.x (mgr.24118) 1445 : cluster [DBG] pgmap v2152: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.060 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:33 smithi166 bash[9446]: cluster 2020-09-08T16:15:32.136858+0000 osd.5 (osd.5) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.060 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:33 smithi166 bash[9446]: cluster 2020-09-08T16:15:32.137014+0000 osd.5 (osd.5) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.061 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:33 smithi166 bash[9446]: cluster 2020-09-08T16:15:32.149772+0000 mon.a (mon.0) 2410 : cluster [DBG] osdmap e792: 12 total, 12 up, 12 in 2020-09-08T16:27:02.061 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:33 smithi166 bash[9446]: cluster 2020-09-08T16:15:32.214578+0000 mgr.x (mgr.24118) 1446 : cluster [DBG] pgmap v2154: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.061 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:35 smithi166 bash[9446]: cluster 2020-09-08T16:15:34.215255+0000 mgr.x (mgr.24118) 1447 : cluster [DBG] pgmap v2155: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.061 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:35 smithi166 bash[9446]: cluster 2020-09-08T16:15:34.709795+0000 osd.10 (osd.10) 123 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.061 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:35 smithi166 bash[9446]: cluster 2020-09-08T16:15:34.709916+0000 osd.10 (osd.10) 124 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.062 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:36 smithi166 bash[9446]: cluster 2020-09-08T16:15:35.279221+0000 mon.a (mon.0) 2411 : cluster [DBG] osdmap e793: 12 total, 12 up, 12 in 2020-09-08T16:27:02.062 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:36 smithi166 bash[9446]: cluster 2020-09-08T16:15:36.132157+0000 osd.3 (osd.3) 147 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.062 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:36 smithi166 bash[9446]: cluster 2020-09-08T16:15:36.132304+0000 osd.3 (osd.3) 148 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.062 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:37 smithi166 bash[9446]: cluster 2020-09-08T16:15:36.215863+0000 mgr.x (mgr.24118) 1448 : cluster [DBG] pgmap v2157: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.063 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:37 smithi166 bash[9446]: cluster 2020-09-08T16:15:36.298356+0000 mon.a (mon.0) 2412 : cluster [DBG] osdmap e794: 12 total, 12 up, 12 in 2020-09-08T16:27:02.063 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:39 smithi166 bash[9446]: cluster 2020-09-08T16:15:38.216592+0000 mgr.x (mgr.24118) 1449 : cluster [DBG] pgmap v2159: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.063 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:41 smithi166 bash[9446]: cluster 2020-09-08T16:15:40.217245+0000 mgr.x (mgr.24118) 1450 : cluster [DBG] pgmap v2160: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.063 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:43 smithi166 bash[9446]: cluster 2020-09-08T16:15:42.217849+0000 mgr.x (mgr.24118) 1451 : cluster [DBG] pgmap v2161: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.063 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:44 smithi166 bash[9446]: audit 2020-09-08T16:15:43.297700+0000 mon.a (mon.0) 2413 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.064 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:44 smithi166 bash[9446]: audit 2020-09-08T16:15:43.303045+0000 mon.a (mon.0) 2414 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.064 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:45 smithi166 bash[9446]: cluster 2020-09-08T16:15:43.996351+0000 osd.8 (osd.8) 118 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.064 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:45 smithi166 bash[9446]: cluster 2020-09-08T16:15:43.996507+0000 osd.8 (osd.8) 119 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.065 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:45 smithi166 bash[9446]: cluster 2020-09-08T16:15:44.218673+0000 mgr.x (mgr.24118) 1452 : cluster [DBG] pgmap v2162: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.065 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:45 smithi166 bash[9446]: cluster 2020-09-08T16:15:44.311575+0000 mon.a (mon.0) 2415 : cluster [DBG] osdmap e795: 12 total, 12 up, 12 in 2020-09-08T16:27:02.065 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:47 smithi166 bash[9446]: cluster 2020-09-08T16:15:46.219382+0000 mgr.x (mgr.24118) 1453 : cluster [DBG] pgmap v2164: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.065 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:49 smithi166 bash[9446]: cluster 2020-09-08T16:15:48.220093+0000 mgr.x (mgr.24118) 1454 : cluster [DBG] pgmap v2165: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.065 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:51 smithi166 bash[9446]: cluster 2020-09-08T16:15:50.220756+0000 mgr.x (mgr.24118) 1455 : cluster [DBG] pgmap v2166: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.066 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:51 smithi166 bash[9446]: cluster 2020-09-08T16:15:50.365951+0000 osd.1 (osd.1) 164 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.066 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:51 smithi166 bash[9446]: cluster 2020-09-08T16:15:50.366086+0000 osd.1 (osd.1) 165 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.066 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:52 smithi166 bash[9446]: cluster 2020-09-08T16:15:51.154801+0000 mon.a (mon.0) 2416 : cluster [DBG] osdmap e796: 12 total, 12 up, 12 in 2020-09-08T16:27:02.066 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:53 smithi166 bash[9446]: cluster 2020-09-08T16:15:52.221434+0000 mgr.x (mgr.24118) 1456 : cluster [DBG] pgmap v2168: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.066 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:55 smithi166 bash[9446]: cluster 2020-09-08T16:15:54.222100+0000 mgr.x (mgr.24118) 1457 : cluster [DBG] pgmap v2169: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.067 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:56 smithi166 bash[9446]: cluster 2020-09-08T16:15:54.794063+0000 osd.6 (osd.6) 129 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.067 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:56 smithi166 bash[9446]: cluster 2020-09-08T16:15:54.794193+0000 osd.6 (osd.6) 130 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.067 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:56 smithi166 bash[9446]: cluster 2020-09-08T16:15:55.292870+0000 mon.a (mon.0) 2417 : cluster [DBG] osdmap e797: 12 total, 12 up, 12 in 2020-09-08T16:27:02.067 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:57 smithi166 bash[9446]: cluster 2020-09-08T16:15:56.222752+0000 mgr.x (mgr.24118) 1458 : cluster [DBG] pgmap v2171: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.069 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:15:59 smithi166 bash[9446]: cluster 2020-09-08T16:15:58.223435+0000 mgr.x (mgr.24118) 1459 : cluster [DBG] pgmap v2172: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.069 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:01 smithi166 bash[9446]: cluster 2020-09-08T16:16:00.114018+0000 osd.11 (osd.11) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.069 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:01 smithi166 bash[9446]: cluster 2020-09-08T16:16:00.114147+0000 osd.11 (osd.11) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.069 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:01 smithi166 bash[9446]: cluster 2020-09-08T16:16:00.224131+0000 mgr.x (mgr.24118) 1460 : cluster [DBG] pgmap v2173: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.070 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:01 smithi166 bash[9446]: cluster 2020-09-08T16:16:00.318210+0000 mon.a (mon.0) 2418 : cluster [DBG] osdmap e798: 12 total, 12 up, 12 in 2020-09-08T16:27:02.070 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:03 smithi166 bash[9446]: cluster 2020-09-08T16:16:02.224877+0000 mgr.x (mgr.24118) 1461 : cluster [DBG] pgmap v2175: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.070 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:04 smithi166 bash[9446]: cluster 2020-09-08T16:16:03.431457+0000 osd.4 (osd.4) 89 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.070 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:04 smithi166 bash[9446]: cluster 2020-09-08T16:16:03.431583+0000 osd.4 (osd.4) 90 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.071 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:05 smithi166 bash[9446]: cluster 2020-09-08T16:16:04.225590+0000 mgr.x (mgr.24118) 1462 : cluster [DBG] pgmap v2176: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.071 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:05 smithi166 bash[9446]: cluster 2020-09-08T16:16:04.339098+0000 mon.a (mon.0) 2419 : cluster [DBG] osdmap e799: 12 total, 12 up, 12 in 2020-09-08T16:27:02.071 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:07 smithi166 bash[9446]: cluster 2020-09-08T16:16:06.226246+0000 mgr.x (mgr.24118) 1463 : cluster [DBG] pgmap v2178: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.071 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:09 smithi166 bash[9446]: cluster 2020-09-08T16:16:08.226976+0000 mgr.x (mgr.24118) 1464 : cluster [DBG] pgmap v2179: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.072 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:11 smithi166 bash[9446]: cluster 2020-09-08T16:16:10.227554+0000 mgr.x (mgr.24118) 1465 : cluster [DBG] pgmap v2180: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.072 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:12 smithi166 bash[9446]: cluster 2020-09-08T16:16:11.941232+0000 osd.9 (osd.9) 128 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.072 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:12 smithi166 bash[9446]: cluster 2020-09-08T16:16:11.941368+0000 osd.9 (osd.9) 129 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.073 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:13 smithi166 bash[9446]: cluster 2020-09-08T16:16:12.164050+0000 mon.a (mon.0) 2420 : cluster [DBG] osdmap e800: 12 total, 12 up, 12 in 2020-09-08T16:27:02.073 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:13 smithi166 bash[9446]: cluster 2020-09-08T16:16:12.228176+0000 mgr.x (mgr.24118) 1466 : cluster [DBG] pgmap v2182: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.073 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:15 smithi166 bash[9446]: cluster 2020-09-08T16:16:14.228699+0000 mgr.x (mgr.24118) 1467 : cluster [DBG] pgmap v2183: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.073 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:15 smithi166 bash[9446]: cluster 2020-09-08T16:16:14.413784+0000 osd.2 (osd.2) 124 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.073 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:15 smithi166 bash[9446]: cluster 2020-09-08T16:16:14.413922+0000 osd.2 (osd.2) 125 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.074 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:16 smithi166 bash[9446]: cluster 2020-09-08T16:16:15.293924+0000 mon.a (mon.0) 2421 : cluster [DBG] osdmap e801: 12 total, 12 up, 12 in 2020-09-08T16:27:02.074 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:17 smithi166 bash[9446]: cluster 2020-09-08T16:16:16.229284+0000 mgr.x (mgr.24118) 1468 : cluster [DBG] pgmap v2185: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.074 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:19 smithi166 bash[9446]: cluster 2020-09-08T16:16:18.230105+0000 mgr.x (mgr.24118) 1469 : cluster [DBG] pgmap v2186: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.074 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:21 smithi166 bash[9446]: cluster 2020-09-08T16:16:20.230600+0000 mgr.x (mgr.24118) 1470 : cluster [DBG] pgmap v2187: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.074 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:23 smithi166 bash[9446]: cluster 2020-09-08T16:16:22.231310+0000 mgr.x (mgr.24118) 1471 : cluster [DBG] pgmap v2188: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.075 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:24 smithi166 bash[9446]: cluster 2020-09-08T16:16:22.893870+0000 osd.7 (osd.7) 180 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.075 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:24 smithi166 bash[9446]: cluster 2020-09-08T16:16:22.893995+0000 osd.7 (osd.7) 181 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.075 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:24 smithi166 bash[9446]: cluster 2020-09-08T16:16:23.295140+0000 mon.a (mon.0) 2422 : cluster [DBG] osdmap e802: 12 total, 12 up, 12 in 2020-09-08T16:27:02.075 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:24 smithi166 bash[9446]: cluster 2020-09-08T16:16:23.610710+0000 osd.0 (osd.0) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.075 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:24 smithi166 bash[9446]: cluster 2020-09-08T16:16:23.610833+0000 osd.0 (osd.0) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.076 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:25 smithi166 bash[9446]: cluster 2020-09-08T16:16:24.231993+0000 mgr.x (mgr.24118) 1472 : cluster [DBG] pgmap v2190: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.076 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:25 smithi166 bash[9446]: cluster 2020-09-08T16:16:24.320596+0000 mon.a (mon.0) 2423 : cluster [DBG] osdmap e803: 12 total, 12 up, 12 in 2020-09-08T16:27:02.076 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:27 smithi166 bash[9446]: cluster 2020-09-08T16:16:26.232468+0000 mgr.x (mgr.24118) 1473 : cluster [DBG] pgmap v2192: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.076 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:29 smithi166 bash[9446]: cluster 2020-09-08T16:16:28.233181+0000 mgr.x (mgr.24118) 1474 : cluster [DBG] pgmap v2193: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.076 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:31 smithi166 bash[9446]: cluster 2020-09-08T16:16:30.233865+0000 mgr.x (mgr.24118) 1475 : cluster [DBG] pgmap v2194: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.077 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:33 smithi166 bash[9446]: cluster 2020-09-08T16:16:32.234550+0000 mgr.x (mgr.24118) 1476 : cluster [DBG] pgmap v2195: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.077 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:33 smithi166 bash[9446]: cluster 2020-09-08T16:16:32.555375+0000 osd.5 (osd.5) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.077 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:33 smithi166 bash[9446]: cluster 2020-09-08T16:16:32.555503+0000 osd.5 (osd.5) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.077 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:34 smithi166 bash[9446]: cluster 2020-09-08T16:16:33.304105+0000 mon.a (mon.0) 2424 : cluster [DBG] osdmap e804: 12 total, 12 up, 12 in 2020-09-08T16:27:02.077 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:35 smithi166 bash[9446]: cluster 2020-09-08T16:16:34.235182+0000 mgr.x (mgr.24118) 1477 : cluster [DBG] pgmap v2197: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.078 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:35 smithi166 bash[9446]: cluster 2020-09-08T16:16:34.831684+0000 osd.10 (osd.10) 125 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.078 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:35 smithi166 bash[9446]: cluster 2020-09-08T16:16:34.831801+0000 osd.10 (osd.10) 126 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.078 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:36 smithi166 bash[9446]: cluster 2020-09-08T16:16:35.364819+0000 mon.a (mon.0) 2425 : cluster [DBG] osdmap e805: 12 total, 12 up, 12 in 2020-09-08T16:27:02.078 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:37 smithi166 bash[9446]: cluster 2020-09-08T16:16:36.235834+0000 mgr.x (mgr.24118) 1478 : cluster [DBG] pgmap v2199: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.078 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:37 smithi166 bash[9446]: cluster 2020-09-08T16:16:36.286746+0000 osd.3 (osd.3) 149 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.079 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:37 smithi166 bash[9446]: cluster 2020-09-08T16:16:36.286865+0000 osd.3 (osd.3) 150 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.079 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:37 smithi166 bash[9446]: cluster 2020-09-08T16:16:36.395282+0000 mon.a (mon.0) 2426 : cluster [DBG] osdmap e806: 12 total, 12 up, 12 in 2020-09-08T16:27:02.079 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:39 smithi166 bash[9446]: cluster 2020-09-08T16:16:38.236525+0000 mgr.x (mgr.24118) 1479 : cluster [DBG] pgmap v2201: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.079 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:41 smithi166 bash[9446]: cluster 2020-09-08T16:16:40.237222+0000 mgr.x (mgr.24118) 1480 : cluster [DBG] pgmap v2202: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.079 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:43 smithi166 bash[9446]: cluster 2020-09-08T16:16:42.237909+0000 mgr.x (mgr.24118) 1481 : cluster [DBG] pgmap v2203: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.080 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:45 smithi166 bash[9446]: cluster 2020-09-08T16:16:44.238674+0000 mgr.x (mgr.24118) 1482 : cluster [DBG] pgmap v2204: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.080 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:45 smithi166 bash[9446]: cluster 2020-09-08T16:16:44.342159+0000 osd.8 (osd.8) 120 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.080 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:45 smithi166 bash[9446]: cluster 2020-09-08T16:16:44.342307+0000 osd.8 (osd.8) 121 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.080 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:46 smithi166 bash[9446]: cluster 2020-09-08T16:16:45.315881+0000 mon.a (mon.0) 2427 : cluster [DBG] osdmap e807: 12 total, 12 up, 12 in 2020-09-08T16:27:02.080 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:47 smithi166 bash[9446]: cluster 2020-09-08T16:16:46.239313+0000 mgr.x (mgr.24118) 1483 : cluster [DBG] pgmap v2206: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.081 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:49 smithi166 bash[9446]: cluster 2020-09-08T16:16:48.240031+0000 mgr.x (mgr.24118) 1484 : cluster [DBG] pgmap v2207: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.081 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:51 smithi166 bash[9446]: cluster 2020-09-08T16:16:50.240728+0000 mgr.x (mgr.24118) 1485 : cluster [DBG] pgmap v2208: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.081 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:52 smithi166 bash[9446]: cluster 2020-09-08T16:16:51.121318+0000 osd.1 (osd.1) 166 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.081 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:52 smithi166 bash[9446]: cluster 2020-09-08T16:16:51.121449+0000 osd.1 (osd.1) 167 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.081 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:52 smithi166 bash[9446]: cluster 2020-09-08T16:16:51.178890+0000 mon.a (mon.0) 2428 : cluster [DBG] osdmap e808: 12 total, 12 up, 12 in 2020-09-08T16:27:02.082 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:53 smithi166 bash[9446]: cluster 2020-09-08T16:16:52.241350+0000 mgr.x (mgr.24118) 1486 : cluster [DBG] pgmap v2210: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.082 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:55 smithi166 bash[9446]: cluster 2020-09-08T16:16:54.242071+0000 mgr.x (mgr.24118) 1487 : cluster [DBG] pgmap v2211: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.083 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:56 smithi166 bash[9446]: cluster 2020-09-08T16:16:55.202047+0000 osd.6 (osd.6) 131 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.083 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:56 smithi166 bash[9446]: cluster 2020-09-08T16:16:55.202190+0000 osd.6 (osd.6) 132 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.083 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:56 smithi166 bash[9446]: cluster 2020-09-08T16:16:56.176655+0000 mon.a (mon.0) 2429 : cluster [DBG] osdmap e809: 12 total, 12 up, 12 in 2020-09-08T16:27:02.083 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:57 smithi166 bash[9446]: cluster 2020-09-08T16:16:56.242697+0000 mgr.x (mgr.24118) 1488 : cluster [DBG] pgmap v2213: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.083 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:16:59 smithi166 bash[9446]: cluster 2020-09-08T16:16:58.243347+0000 mgr.x (mgr.24118) 1489 : cluster [DBG] pgmap v2214: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.084 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:01 smithi166 bash[9446]: cluster 2020-09-08T16:17:00.244025+0000 mgr.x (mgr.24118) 1490 : cluster [DBG] pgmap v2215: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.084 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:01 smithi166 bash[9446]: cluster 2020-09-08T16:17:00.467924+0000 osd.11 (osd.11) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.084 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:01 smithi166 bash[9446]: cluster 2020-09-08T16:17:00.468086+0000 osd.11 (osd.11) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.084 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:02 smithi166 bash[9446]: cluster 2020-09-08T16:17:01.193296+0000 mon.a (mon.0) 2430 : cluster [DBG] osdmap e810: 12 total, 12 up, 12 in 2020-09-08T16:27:02.084 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:03 smithi166 bash[9446]: cluster 2020-09-08T16:17:02.244701+0000 mgr.x (mgr.24118) 1491 : cluster [DBG] pgmap v2217: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.085 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:05 smithi166 bash[9446]: cluster 2020-09-08T16:17:03.887171+0000 osd.4 (osd.4) 91 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.085 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:05 smithi166 bash[9446]: cluster 2020-09-08T16:17:03.887298+0000 osd.4 (osd.4) 92 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.085 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:05 smithi166 bash[9446]: cluster 2020-09-08T16:17:04.212140+0000 mon.a (mon.0) 2431 : cluster [DBG] osdmap e811: 12 total, 12 up, 12 in 2020-09-08T16:27:02.085 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:05 smithi166 bash[9446]: cluster 2020-09-08T16:17:04.245303+0000 mgr.x (mgr.24118) 1492 : cluster [DBG] pgmap v2219: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.085 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:07 smithi166 bash[9446]: cluster 2020-09-08T16:17:06.245996+0000 mgr.x (mgr.24118) 1493 : cluster [DBG] pgmap v2220: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.086 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:09 smithi166 bash[9446]: cluster 2020-09-08T16:17:08.246741+0000 mgr.x (mgr.24118) 1494 : cluster [DBG] pgmap v2221: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.086 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:11 smithi166 bash[9446]: cluster 2020-09-08T16:17:10.247372+0000 mgr.x (mgr.24118) 1495 : cluster [DBG] pgmap v2222: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.086 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:13 smithi166 bash[9446]: cluster 2020-09-08T16:17:12.248035+0000 mgr.x (mgr.24118) 1496 : cluster [DBG] pgmap v2223: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.086 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:13 smithi166 bash[9446]: cluster 2020-09-08T16:17:12.903835+0000 osd.9 (osd.9) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.086 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:13 smithi166 bash[9446]: cluster 2020-09-08T16:17:12.903962+0000 osd.9 (osd.9) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.087 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:14 smithi166 bash[9446]: cluster 2020-09-08T16:17:13.310006+0000 mon.a (mon.0) 2432 : cluster [DBG] osdmap e812: 12 total, 12 up, 12 in 2020-09-08T16:27:02.087 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:15 smithi166 bash[9446]: cluster 2020-09-08T16:17:14.248684+0000 mgr.x (mgr.24118) 1497 : cluster [DBG] pgmap v2225: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.087 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:16 smithi166 bash[9446]: cluster 2020-09-08T16:17:15.256758+0000 osd.2 (osd.2) 126 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.087 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:16 smithi166 bash[9446]: cluster 2020-09-08T16:17:15.256850+0000 osd.2 (osd.2) 127 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.087 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:16 smithi166 bash[9446]: cluster 2020-09-08T16:17:16.187290+0000 mon.a (mon.0) 2433 : cluster [DBG] osdmap e813: 12 total, 12 up, 12 in 2020-09-08T16:27:02.088 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:17 smithi166 bash[9446]: cluster 2020-09-08T16:17:16.249291+0000 mgr.x (mgr.24118) 1498 : cluster [DBG] pgmap v2227: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.088 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:19 smithi166 bash[9446]: cluster 2020-09-08T16:17:18.250125+0000 mgr.x (mgr.24118) 1499 : cluster [DBG] pgmap v2228: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.088 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:21 smithi166 bash[9446]: cluster 2020-09-08T16:17:20.250901+0000 mgr.x (mgr.24118) 1500 : cluster [DBG] pgmap v2229: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.088 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:23 smithi166 bash[9446]: cluster 2020-09-08T16:17:22.251569+0000 mgr.x (mgr.24118) 1501 : cluster [DBG] pgmap v2230: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.088 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:24 smithi166 bash[9446]: cluster 2020-09-08T16:17:22.982890+0000 osd.7 (osd.7) 182 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.089 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:24 smithi166 bash[9446]: cluster 2020-09-08T16:17:22.983017+0000 osd.7 (osd.7) 183 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.089 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:24 smithi166 bash[9446]: cluster 2020-09-08T16:17:23.322276+0000 mon.a (mon.0) 2434 : cluster [DBG] osdmap e814: 12 total, 12 up, 12 in 2020-09-08T16:27:02.089 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:24 smithi166 bash[9446]: cluster 2020-09-08T16:17:23.824128+0000 osd.0 (osd.0) 142 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.089 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:24 smithi166 bash[9446]: cluster 2020-09-08T16:17:23.824255+0000 osd.0 (osd.0) 143 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.089 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:25 smithi166 bash[9446]: cluster 2020-09-08T16:17:24.252193+0000 mgr.x (mgr.24118) 1502 : cluster [DBG] pgmap v2232: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.090 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:25 smithi166 bash[9446]: cluster 2020-09-08T16:17:24.334057+0000 mon.a (mon.0) 2435 : cluster [DBG] osdmap e815: 12 total, 12 up, 12 in 2020-09-08T16:27:02.090 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:27 smithi166 bash[9446]: cluster 2020-09-08T16:17:26.252876+0000 mgr.x (mgr.24118) 1503 : cluster [DBG] pgmap v2234: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.090 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:29 smithi166 bash[9446]: cluster 2020-09-08T16:17:28.253590+0000 mgr.x (mgr.24118) 1504 : cluster [DBG] pgmap v2235: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.090 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:31 smithi166 bash[9446]: cluster 2020-09-08T16:17:30.254239+0000 mgr.x (mgr.24118) 1505 : cluster [DBG] pgmap v2236: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.090 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:33 smithi166 bash[9446]: cluster 2020-09-08T16:17:32.254924+0000 mgr.x (mgr.24118) 1506 : cluster [DBG] pgmap v2237: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.091 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:34 smithi166 bash[9446]: cluster 2020-09-08T16:17:33.361145+0000 osd.5 (osd.5) 148 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.091 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:34 smithi166 bash[9446]: cluster 2020-09-08T16:17:33.361282+0000 osd.5 (osd.5) 149 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.091 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:35 smithi166 bash[9446]: cluster 2020-09-08T16:17:34.255666+0000 mgr.x (mgr.24118) 1507 : cluster [DBG] pgmap v2238: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.091 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:35 smithi166 bash[9446]: cluster 2020-09-08T16:17:34.330857+0000 mon.a (mon.0) 2436 : cluster [DBG] osdmap e816: 12 total, 12 up, 12 in 2020-09-08T16:27:02.091 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:36 smithi166 bash[9446]: cluster 2020-09-08T16:17:35.549540+0000 osd.10 (osd.10) 127 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.092 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:36 smithi166 bash[9446]: cluster 2020-09-08T16:17:35.549670+0000 osd.10 (osd.10) 128 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.092 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:37 smithi166 bash[9446]: cluster 2020-09-08T16:17:36.256318+0000 mgr.x (mgr.24118) 1508 : cluster [DBG] pgmap v2240: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.092 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:37 smithi166 bash[9446]: cluster 2020-09-08T16:17:36.344939+0000 mon.a (mon.0) 2437 : cluster [DBG] osdmap e817: 12 total, 12 up, 12 in 2020-09-08T16:27:02.093 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:37 smithi166 bash[9446]: cluster 2020-09-08T16:17:37.131639+0000 osd.3 (osd.3) 151 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.093 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:37 smithi166 bash[9446]: cluster 2020-09-08T16:17:37.131762+0000 osd.3 (osd.3) 152 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.093 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:38 smithi166 bash[9446]: cluster 2020-09-08T16:17:37.346297+0000 mon.a (mon.0) 2438 : cluster [DBG] osdmap e818: 12 total, 12 up, 12 in 2020-09-08T16:27:02.093 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:39 smithi166 bash[9446]: cluster 2020-09-08T16:17:38.256992+0000 mgr.x (mgr.24118) 1509 : cluster [DBG] pgmap v2243: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.094 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:41 smithi166 bash[9446]: cluster 2020-09-08T16:17:40.257685+0000 mgr.x (mgr.24118) 1510 : cluster [DBG] pgmap v2244: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.094 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:43 smithi166 bash[9446]: cluster 2020-09-08T16:17:42.258389+0000 mgr.x (mgr.24118) 1511 : cluster [DBG] pgmap v2245: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.094 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:45 smithi166 bash[9446]: cluster 2020-09-08T16:17:44.259204+0000 mgr.x (mgr.24118) 1512 : cluster [DBG] pgmap v2246: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.094 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:46 smithi166 bash[9446]: cluster 2020-09-08T16:17:44.578715+0000 osd.8 (osd.8) 122 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.094 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:46 smithi166 bash[9446]: cluster 2020-09-08T16:17:44.578865+0000 osd.8 (osd.8) 123 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.095 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:46 smithi166 bash[9446]: cluster 2020-09-08T16:17:45.337997+0000 mon.a (mon.0) 2439 : cluster [DBG] osdmap e819: 12 total, 12 up, 12 in 2020-09-08T16:27:02.095 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:47 smithi166 bash[9446]: cluster 2020-09-08T16:17:46.259938+0000 mgr.x (mgr.24118) 1513 : cluster [DBG] pgmap v2248: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.095 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:49 smithi166 bash[9446]: cluster 2020-09-08T16:17:48.260855+0000 mgr.x (mgr.24118) 1514 : cluster [DBG] pgmap v2249: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.095 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:51 smithi166 bash[9446]: cluster 2020-09-08T16:17:50.261601+0000 mgr.x (mgr.24118) 1515 : cluster [DBG] pgmap v2250: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.095 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:52 smithi166 bash[9446]: cluster 2020-09-08T16:17:51.785942+0000 osd.1 (osd.1) 168 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.096 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:52 smithi166 bash[9446]: cluster 2020-09-08T16:17:51.786066+0000 osd.1 (osd.1) 169 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.096 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:53 smithi166 bash[9446]: cluster 2020-09-08T16:17:52.207446+0000 mon.a (mon.0) 2440 : cluster [DBG] osdmap e820: 12 total, 12 up, 12 in 2020-09-08T16:27:02.096 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:53 smithi166 bash[9446]: cluster 2020-09-08T16:17:52.262185+0000 mgr.x (mgr.24118) 1516 : cluster [DBG] pgmap v2252: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.096 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:55 smithi166 bash[9446]: cluster 2020-09-08T16:17:54.262935+0000 mgr.x (mgr.24118) 1517 : cluster [DBG] pgmap v2253: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.096 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:57 smithi166 bash[9446]: cluster 2020-09-08T16:17:56.087499+0000 osd.6 (osd.6) 133 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.097 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:57 smithi166 bash[9446]: cluster 2020-09-08T16:17:56.087630+0000 osd.6 (osd.6) 134 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.097 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:57 smithi166 bash[9446]: cluster 2020-09-08T16:17:56.263630+0000 mgr.x (mgr.24118) 1518 : cluster [DBG] pgmap v2254: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.097 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:57 smithi166 bash[9446]: cluster 2020-09-08T16:17:56.337988+0000 mon.a (mon.0) 2441 : cluster [DBG] osdmap e821: 12 total, 12 up, 12 in 2020-09-08T16:27:02.097 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:17:59 smithi166 bash[9446]: cluster 2020-09-08T16:17:58.264325+0000 mgr.x (mgr.24118) 1519 : cluster [DBG] pgmap v2256: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.098 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:01 smithi166 bash[9446]: cluster 2020-09-08T16:18:00.265034+0000 mgr.x (mgr.24118) 1520 : cluster [DBG] pgmap v2257: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.099 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:02 smithi166 bash[9446]: cluster 2020-09-08T16:18:01.420675+0000 osd.11 (osd.11) 148 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.099 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:02 smithi166 bash[9446]: cluster 2020-09-08T16:18:01.420810+0000 osd.11 (osd.11) 149 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.099 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:03 smithi166 bash[9446]: cluster 2020-09-08T16:18:02.213345+0000 mon.a (mon.0) 2442 : cluster [DBG] osdmap e822: 12 total, 12 up, 12 in 2020-09-08T16:27:02.099 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:03 smithi166 bash[9446]: cluster 2020-09-08T16:18:02.265680+0000 mgr.x (mgr.24118) 1521 : cluster [DBG] pgmap v2259: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.099 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:05 smithi166 bash[9446]: cluster 2020-09-08T16:18:04.266460+0000 mgr.x (mgr.24118) 1522 : cluster [DBG] pgmap v2260: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.100 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:06 smithi166 bash[9446]: cluster 2020-09-08T16:18:04.742722+0000 osd.4 (osd.4) 93 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.100 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:06 smithi166 bash[9446]: cluster 2020-09-08T16:18:04.742846+0000 osd.4 (osd.4) 94 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.100 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:06 smithi166 bash[9446]: cluster 2020-09-08T16:18:05.327738+0000 mon.a (mon.0) 2443 : cluster [DBG] osdmap e823: 12 total, 12 up, 12 in 2020-09-08T16:27:02.100 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:07 smithi166 bash[9446]: cluster 2020-09-08T16:18:06.267129+0000 mgr.x (mgr.24118) 1523 : cluster [DBG] pgmap v2262: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.100 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:09 smithi166 bash[9446]: cluster 2020-09-08T16:18:08.267890+0000 mgr.x (mgr.24118) 1524 : cluster [DBG] pgmap v2263: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.101 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:11 smithi166 bash[9446]: cluster 2020-09-08T16:18:10.268693+0000 mgr.x (mgr.24118) 1525 : cluster [DBG] pgmap v2264: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.101 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:13 smithi166 bash[9446]: cluster 2020-09-08T16:18:12.269386+0000 mgr.x (mgr.24118) 1526 : cluster [DBG] pgmap v2265: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.101 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:14 smithi166 bash[9446]: cluster 2020-09-08T16:18:13.633224+0000 osd.9 (osd.9) 132 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.101 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:14 smithi166 bash[9446]: cluster 2020-09-08T16:18:13.633351+0000 osd.9 (osd.9) 133 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.101 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:15 smithi166 bash[9446]: cluster 2020-09-08T16:18:14.270125+0000 mgr.x (mgr.24118) 1527 : cluster [DBG] pgmap v2266: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.102 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:15 smithi166 bash[9446]: cluster 2020-09-08T16:18:14.343382+0000 mon.a (mon.0) 2444 : cluster [DBG] osdmap e824: 12 total, 12 up, 12 in 2020-09-08T16:27:02.102 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:17 smithi166 bash[9446]: cluster 2020-09-08T16:18:16.208579+0000 osd.2 (osd.2) 128 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.102 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:17 smithi166 bash[9446]: cluster 2020-09-08T16:18:16.208704+0000 osd.2 (osd.2) 129 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.102 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:17 smithi166 bash[9446]: cluster 2020-09-08T16:18:16.270774+0000 mgr.x (mgr.24118) 1528 : cluster [DBG] pgmap v2268: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.102 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:17 smithi166 bash[9446]: cluster 2020-09-08T16:18:16.358692+0000 mon.a (mon.0) 2445 : cluster [DBG] osdmap e825: 12 total, 12 up, 12 in 2020-09-08T16:27:02.103 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:19 smithi166 bash[9446]: cluster 2020-09-08T16:18:18.271514+0000 mgr.x (mgr.24118) 1529 : cluster [DBG] pgmap v2270: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.103 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:21 smithi166 bash[9446]: cluster 2020-09-08T16:18:20.272244+0000 mgr.x (mgr.24118) 1530 : cluster [DBG] pgmap v2271: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.103 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:23 smithi166 bash[9446]: cluster 2020-09-08T16:18:22.272930+0000 mgr.x (mgr.24118) 1531 : cluster [DBG] pgmap v2272: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.103 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:25 smithi166 bash[9446]: cluster 2020-09-08T16:18:23.788192+0000 osd.7 (osd.7) 184 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.103 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:25 smithi166 bash[9446]: cluster 2020-09-08T16:18:23.788334+0000 osd.7 (osd.7) 185 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.104 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:25 smithi166 bash[9446]: cluster 2020-09-08T16:18:24.273691+0000 mgr.x (mgr.24118) 1532 : cluster [DBG] pgmap v2273: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.104 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:25 smithi166 bash[9446]: cluster 2020-09-08T16:18:24.345809+0000 mon.a (mon.0) 2446 : cluster [DBG] osdmap e826: 12 total, 12 up, 12 in 2020-09-08T16:27:02.104 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:25 smithi166 bash[9446]: cluster 2020-09-08T16:18:24.542528+0000 osd.0 (osd.0) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.104 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:25 smithi166 bash[9446]: cluster 2020-09-08T16:18:24.542657+0000 osd.0 (osd.0) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.104 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:26 smithi166 bash[9446]: cluster 2020-09-08T16:18:25.346774+0000 mon.a (mon.0) 2447 : cluster [DBG] osdmap e827: 12 total, 12 up, 12 in 2020-09-08T16:27:02.105 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:27 smithi166 bash[9446]: cluster 2020-09-08T16:18:26.274355+0000 mgr.x (mgr.24118) 1533 : cluster [DBG] pgmap v2276: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.105 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:29 smithi166 bash[9446]: cluster 2020-09-08T16:18:28.275071+0000 mgr.x (mgr.24118) 1534 : cluster [DBG] pgmap v2277: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.105 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:30 smithi166 bash[9446]: audit 2020-09-08T16:18:29.527573+0000 mon.b (mon.1) 136 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.105 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:30 smithi166 bash[9446]: audit 2020-09-08T16:18:29.529031+0000 mon.a (mon.0) 2448 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.105 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:30 smithi166 bash[9446]: audit 2020-09-08T16:18:29.534278+0000 mon.a (mon.0) 2449 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.106 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:31 smithi166 bash[9446]: cluster 2020-09-08T16:18:30.275532+0000 mgr.x (mgr.24118) 1535 : cluster [DBG] pgmap v2278: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.106 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:33 smithi166 bash[9446]: cluster 2020-09-08T16:18:32.276128+0000 mgr.x (mgr.24118) 1536 : cluster [DBG] pgmap v2279: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.106 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:35 smithi166 bash[9446]: cluster 2020-09-08T16:18:34.142626+0000 osd.5 (osd.5) 150 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.106 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:35 smithi166 bash[9446]: cluster 2020-09-08T16:18:34.142759+0000 osd.5 (osd.5) 151 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.106 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:35 smithi166 bash[9446]: cluster 2020-09-08T16:18:34.276879+0000 mgr.x (mgr.24118) 1537 : cluster [DBG] pgmap v2280: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.107 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:35 smithi166 bash[9446]: cluster 2020-09-08T16:18:34.400794+0000 mon.a (mon.0) 2450 : cluster [DBG] osdmap e828: 12 total, 12 up, 12 in 2020-09-08T16:27:02.107 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:36 smithi166 bash[9446]: cluster 2020-09-08T16:18:35.596323+0000 osd.10 (osd.10) 129 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.107 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:36 smithi166 bash[9446]: cluster 2020-09-08T16:18:35.596454+0000 osd.10 (osd.10) 130 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.107 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:37 smithi166 bash[9446]: cluster 2020-09-08T16:18:36.277520+0000 mgr.x (mgr.24118) 1538 : cluster [DBG] pgmap v2282: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.108 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:37 smithi166 bash[9446]: cluster 2020-09-08T16:18:36.483874+0000 mon.a (mon.0) 2451 : cluster [DBG] osdmap e829: 12 total, 12 up, 12 in 2020-09-08T16:27:02.108 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:38 smithi166 bash[9446]: cluster 2020-09-08T16:18:37.338299+0000 osd.3 (osd.3) 153 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.108 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:38 smithi166 bash[9446]: cluster 2020-09-08T16:18:37.338410+0000 osd.3 (osd.3) 154 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.108 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:38 smithi166 bash[9446]: cluster 2020-09-08T16:18:37.491995+0000 mon.a (mon.0) 2452 : cluster [DBG] osdmap e830: 12 total, 12 up, 12 in 2020-09-08T16:27:02.108 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:39 smithi166 bash[9446]: cluster 2020-09-08T16:18:38.278287+0000 mgr.x (mgr.24118) 1539 : cluster [DBG] pgmap v2285: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.109 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:41 smithi166 bash[9446]: cluster 2020-09-08T16:18:40.279025+0000 mgr.x (mgr.24118) 1540 : cluster [DBG] pgmap v2286: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.109 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:43 smithi166 bash[9446]: cluster 2020-09-08T16:18:42.279684+0000 mgr.x (mgr.24118) 1541 : cluster [DBG] pgmap v2287: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.109 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:45 smithi166 bash[9446]: cluster 2020-09-08T16:18:44.280459+0000 mgr.x (mgr.24118) 1542 : cluster [DBG] pgmap v2288: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.109 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:46 smithi166 bash[9446]: cluster 2020-09-08T16:18:44.720522+0000 osd.8 (osd.8) 124 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.109 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:46 smithi166 bash[9446]: cluster 2020-09-08T16:18:44.720655+0000 osd.8 (osd.8) 125 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.110 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:46 smithi166 bash[9446]: cluster 2020-09-08T16:18:45.355304+0000 mon.a (mon.0) 2453 : cluster [DBG] osdmap e831: 12 total, 12 up, 12 in 2020-09-08T16:27:02.110 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:47 smithi166 bash[9446]: cluster 2020-09-08T16:18:46.281093+0000 mgr.x (mgr.24118) 1543 : cluster [DBG] pgmap v2290: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.110 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:49 smithi166 bash[9446]: cluster 2020-09-08T16:18:48.281847+0000 mgr.x (mgr.24118) 1544 : cluster [DBG] pgmap v2291: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.110 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:51 smithi166 bash[9446]: cluster 2020-09-08T16:18:50.282574+0000 mgr.x (mgr.24118) 1545 : cluster [DBG] pgmap v2292: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.111 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:52 smithi166 bash[9446]: cluster 2020-09-08T16:18:51.806760+0000 osd.1 (osd.1) 170 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.111 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:52 smithi166 bash[9446]: cluster 2020-09-08T16:18:51.806888+0000 osd.1 (osd.1) 171 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.111 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:53 smithi166 bash[9446]: cluster 2020-09-08T16:18:52.228580+0000 mon.a (mon.0) 2454 : cluster [DBG] osdmap e832: 12 total, 12 up, 12 in 2020-09-08T16:27:02.111 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:53 smithi166 bash[9446]: cluster 2020-09-08T16:18:52.283236+0000 mgr.x (mgr.24118) 1546 : cluster [DBG] pgmap v2294: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:55 smithi166 bash[9446]: cluster 2020-09-08T16:18:54.283994+0000 mgr.x (mgr.24118) 1547 : cluster [DBG] pgmap v2295: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:57 smithi166 bash[9446]: cluster 2020-09-08T16:18:56.284675+0000 mgr.x (mgr.24118) 1548 : cluster [DBG] pgmap v2296: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:58 smithi166 bash[9446]: cluster 2020-09-08T16:18:57.033753+0000 osd.6 (osd.6) 135 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:58 smithi166 bash[9446]: cluster 2020-09-08T16:18:57.033886+0000 osd.6 (osd.6) 136 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.112 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:58 smithi166 bash[9446]: cluster 2020-09-08T16:18:57.362340+0000 mon.a (mon.0) 2455 : cluster [DBG] osdmap e833: 12 total, 12 up, 12 in 2020-09-08T16:27:02.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:18:59 smithi166 bash[9446]: cluster 2020-09-08T16:18:58.285394+0000 mgr.x (mgr.24118) 1549 : cluster [DBG] pgmap v2298: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:01 smithi166 bash[9446]: cluster 2020-09-08T16:19:00.286031+0000 mgr.x (mgr.24118) 1550 : cluster [DBG] pgmap v2299: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:02 smithi166 bash[9446]: cluster 2020-09-08T16:19:01.547200+0000 osd.11 (osd.11) 150 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:02 smithi166 bash[9446]: cluster 2020-09-08T16:19:01.547357+0000 osd.11 (osd.11) 151 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.113 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:03 smithi166 bash[9446]: cluster 2020-09-08T16:19:02.231839+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e834: 12 total, 12 up, 12 in 2020-09-08T16:27:02.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:03 smithi166 bash[9446]: cluster 2020-09-08T16:19:02.286716+0000 mgr.x (mgr.24118) 1551 : cluster [DBG] pgmap v2301: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:05 smithi166 bash[9446]: cluster 2020-09-08T16:19:04.287459+0000 mgr.x (mgr.24118) 1552 : cluster [DBG] pgmap v2302: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:06 smithi166 bash[9446]: cluster 2020-09-08T16:19:05.101483+0000 osd.4 (osd.4) 95 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:06 smithi166 bash[9446]: cluster 2020-09-08T16:19:05.101600+0000 osd.4 (osd.4) 96 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.114 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:06 smithi166 bash[9446]: cluster 2020-09-08T16:19:05.351339+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e835: 12 total, 12 up, 12 in 2020-09-08T16:27:02.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:07 smithi166 bash[9446]: cluster 2020-09-08T16:19:06.288110+0000 mgr.x (mgr.24118) 1553 : cluster [DBG] pgmap v2304: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:09 smithi166 bash[9446]: cluster 2020-09-08T16:19:08.288845+0000 mgr.x (mgr.24118) 1554 : cluster [DBG] pgmap v2305: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:11 smithi166 bash[9446]: cluster 2020-09-08T16:19:10.289470+0000 mgr.x (mgr.24118) 1555 : cluster [DBG] pgmap v2306: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:13 smithi166 bash[9446]: cluster 2020-09-08T16:19:12.290155+0000 mgr.x (mgr.24118) 1556 : cluster [DBG] pgmap v2307: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.115 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:15 smithi166 bash[9446]: cluster 2020-09-08T16:19:14.290872+0000 mgr.x (mgr.24118) 1557 : cluster [DBG] pgmap v2308: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:15 smithi166 bash[9446]: cluster 2020-09-08T16:19:14.572020+0000 osd.9 (osd.9) 134 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:15 smithi166 bash[9446]: cluster 2020-09-08T16:19:14.572143+0000 osd.9 (osd.9) 135 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:16 smithi166 bash[9446]: cluster 2020-09-08T16:19:15.358643+0000 mon.a (mon.0) 2458 : cluster [DBG] osdmap e836: 12 total, 12 up, 12 in 2020-09-08T16:27:02.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:17 smithi166 bash[9446]: cluster 2020-09-08T16:19:16.291530+0000 mgr.x (mgr.24118) 1558 : cluster [DBG] pgmap v2310: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:18 smithi166 bash[9446]: cluster 2020-09-08T16:19:17.194214+0000 osd.2 (osd.2) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:18 smithi166 bash[9446]: cluster 2020-09-08T16:19:17.194333+0000 osd.2 (osd.2) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:18 smithi166 bash[9446]: cluster 2020-09-08T16:19:17.403220+0000 mon.a (mon.0) 2459 : cluster [DBG] osdmap e837: 12 total, 12 up, 12 in 2020-09-08T16:27:02.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:19 smithi166 bash[9446]: cluster 2020-09-08T16:19:18.292239+0000 mgr.x (mgr.24118) 1559 : cluster [DBG] pgmap v2312: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:21 smithi166 bash[9446]: cluster 2020-09-08T16:19:20.293036+0000 mgr.x (mgr.24118) 1560 : cluster [DBG] pgmap v2313: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:23 smithi166 bash[9446]: cluster 2020-09-08T16:19:22.293768+0000 mgr.x (mgr.24118) 1561 : cluster [DBG] pgmap v2314: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:25 smithi166 bash[9446]: cluster 2020-09-08T16:19:24.294501+0000 mgr.x (mgr.24118) 1562 : cluster [DBG] pgmap v2315: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:25 smithi166 bash[9446]: cluster 2020-09-08T16:19:24.556230+0000 osd.7 (osd.7) 186 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:25 smithi166 bash[9446]: cluster 2020-09-08T16:19:24.556355+0000 osd.7 (osd.7) 187 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:25 smithi166 bash[9446]: cluster 2020-09-08T16:19:24.633342+0000 osd.0 (osd.0) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:25 smithi166 bash[9446]: cluster 2020-09-08T16:19:24.633471+0000 osd.0 (osd.0) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:26 smithi166 bash[9446]: cluster 2020-09-08T16:19:25.361020+0000 mon.a (mon.0) 2460 : cluster [DBG] osdmap e838: 12 total, 12 up, 12 in 2020-09-08T16:27:02.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:27 smithi166 bash[9446]: cluster 2020-09-08T16:19:26.295149+0000 mgr.x (mgr.24118) 1563 : cluster [DBG] pgmap v2317: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:28 smithi166 bash[9446]: audit 2020-09-08T16:19:27.397122+0000 mon.b (mon.1) 137 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:28 smithi166 bash[9446]: audit 2020-09-08T16:19:27.398626+0000 mon.a (mon.0) 2461 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:28 smithi166 bash[9446]: audit 2020-09-08T16:19:27.405368+0000 mon.a (mon.0) 2462 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:29 smithi166 bash[9446]: cluster 2020-09-08T16:19:28.295846+0000 mgr.x (mgr.24118) 1564 : cluster [DBG] pgmap v2318: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:31 smithi166 bash[9446]: cluster 2020-09-08T16:19:30.296510+0000 mgr.x (mgr.24118) 1565 : cluster [DBG] pgmap v2319: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:33 smithi166 bash[9446]: cluster 2020-09-08T16:19:32.297174+0000 mgr.x (mgr.24118) 1566 : cluster [DBG] pgmap v2320: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:35 smithi166 bash[9446]: cluster 2020-09-08T16:19:34.219237+0000 osd.5 (osd.5) 152 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:35 smithi166 bash[9446]: cluster 2020-09-08T16:19:34.219366+0000 osd.5 (osd.5) 153 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:35 smithi166 bash[9446]: cluster 2020-09-08T16:19:34.297911+0000 mgr.x (mgr.24118) 1567 : cluster [DBG] pgmap v2321: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:35 smithi166 bash[9446]: cluster 2020-09-08T16:19:34.371433+0000 mon.a (mon.0) 2463 : cluster [DBG] osdmap e839: 12 total, 12 up, 12 in 2020-09-08T16:27:02.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:37 smithi166 bash[9446]: cluster 2020-09-08T16:19:36.298640+0000 mgr.x (mgr.24118) 1568 : cluster [DBG] pgmap v2323: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:37 smithi166 bash[9446]: cluster 2020-09-08T16:19:36.334119+0000 osd.10 (osd.10) 131 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:37 smithi166 bash[9446]: cluster 2020-09-08T16:19:36.334233+0000 osd.10 (osd.10) 132 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.122 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:37 smithi166 bash[9446]: cluster 2020-09-08T16:19:36.388767+0000 mon.a (mon.0) 2464 : cluster [DBG] osdmap e840: 12 total, 12 up, 12 in 2020-09-08T16:27:02.123 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:38 smithi166 bash[9446]: cluster 2020-09-08T16:19:37.437449+0000 osd.3 (osd.3) 155 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.123 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:38 smithi166 bash[9446]: cluster 2020-09-08T16:19:37.437572+0000 osd.3 (osd.3) 156 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.123 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:39 smithi166 bash[9446]: cluster 2020-09-08T16:19:38.299377+0000 mgr.x (mgr.24118) 1569 : cluster [DBG] pgmap v2325: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.123 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:39 smithi166 bash[9446]: cluster 2020-09-08T16:19:38.402072+0000 mon.a (mon.0) 2465 : cluster [DBG] osdmap e841: 12 total, 12 up, 12 in 2020-09-08T16:27:02.124 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:41 smithi166 bash[9446]: cluster 2020-09-08T16:19:40.299822+0000 mgr.x (mgr.24118) 1570 : cluster [DBG] pgmap v2327: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.124 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:43 smithi166 bash[9446]: cluster 2020-09-08T16:19:42.300421+0000 mgr.x (mgr.24118) 1571 : cluster [DBG] pgmap v2328: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.124 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:45 smithi166 bash[9446]: cluster 2020-09-08T16:19:44.301177+0000 mgr.x (mgr.24118) 1572 : cluster [DBG] pgmap v2329: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.125 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:46 smithi166 bash[9446]: cluster 2020-09-08T16:19:44.761903+0000 osd.8 (osd.8) 126 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.125 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:46 smithi166 bash[9446]: cluster 2020-09-08T16:19:44.762032+0000 osd.8 (osd.8) 127 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.125 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:46 smithi166 bash[9446]: cluster 2020-09-08T16:19:45.377400+0000 mon.a (mon.0) 2466 : cluster [DBG] osdmap e842: 12 total, 12 up, 12 in 2020-09-08T16:27:02.125 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:47 smithi166 bash[9446]: cluster 2020-09-08T16:19:46.301791+0000 mgr.x (mgr.24118) 1573 : cluster [DBG] pgmap v2331: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.125 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:49 smithi166 bash[9446]: cluster 2020-09-08T16:19:48.302517+0000 mgr.x (mgr.24118) 1574 : cluster [DBG] pgmap v2332: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.126 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:51 smithi166 bash[9446]: cluster 2020-09-08T16:19:50.303213+0000 mgr.x (mgr.24118) 1575 : cluster [DBG] pgmap v2333: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.126 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:52 smithi166 bash[9446]: cluster 2020-09-08T16:19:51.829178+0000 osd.1 (osd.1) 172 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.126 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:52 smithi166 bash[9446]: cluster 2020-09-08T16:19:51.829307+0000 osd.1 (osd.1) 173 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.126 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:53 smithi166 bash[9446]: cluster 2020-09-08T16:19:52.237451+0000 mon.a (mon.0) 2467 : cluster [DBG] osdmap e843: 12 total, 12 up, 12 in 2020-09-08T16:27:02.126 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:53 smithi166 bash[9446]: cluster 2020-09-08T16:19:52.303900+0000 mgr.x (mgr.24118) 1576 : cluster [DBG] pgmap v2335: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.127 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:55 smithi166 bash[9446]: cluster 2020-09-08T16:19:54.304737+0000 mgr.x (mgr.24118) 1577 : cluster [DBG] pgmap v2336: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.127 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:57 smithi166 bash[9446]: cluster 2020-09-08T16:19:56.305456+0000 mgr.x (mgr.24118) 1578 : cluster [DBG] pgmap v2337: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.127 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:58 smithi166 bash[9446]: cluster 2020-09-08T16:19:57.050038+0000 osd.6 (osd.6) 137 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.127 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:58 smithi166 bash[9446]: cluster 2020-09-08T16:19:57.050184+0000 osd.6 (osd.6) 138 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.127 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:58 smithi166 bash[9446]: cluster 2020-09-08T16:19:57.374960+0000 mon.a (mon.0) 2468 : cluster [DBG] osdmap e844: 12 total, 12 up, 12 in 2020-09-08T16:27:02.128 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:19:59 smithi166 bash[9446]: cluster 2020-09-08T16:19:58.306249+0000 mgr.x (mgr.24118) 1579 : cluster [DBG] pgmap v2339: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.128 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:01 smithi166 bash[9446]: cluster 2020-09-08T16:20:00.307004+0000 mgr.x (mgr.24118) 1580 : cluster [DBG] pgmap v2340: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.128 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:03 smithi166 bash[9446]: cluster 2020-09-08T16:20:02.307670+0000 mgr.x (mgr.24118) 1581 : cluster [DBG] pgmap v2341: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.128 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:03 smithi166 bash[9446]: cluster 2020-09-08T16:20:02.428676+0000 osd.11 (osd.11) 152 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.128 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:03 smithi166 bash[9446]: cluster 2020-09-08T16:20:02.428805+0000 osd.11 (osd.11) 153 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.129 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:04 smithi166 bash[9446]: cluster 2020-09-08T16:20:03.369560+0000 mon.a (mon.0) 2469 : cluster [DBG] osdmap e845: 12 total, 12 up, 12 in 2020-09-08T16:27:02.129 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:05 smithi166 bash[9446]: cluster 2020-09-08T16:20:04.308304+0000 mgr.x (mgr.24118) 1582 : cluster [DBG] pgmap v2343: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.129 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:07 smithi166 bash[9446]: cluster 2020-09-08T16:20:05.902905+0000 osd.4 (osd.4) 97 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.129 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:07 smithi166 bash[9446]: cluster 2020-09-08T16:20:05.903036+0000 osd.4 (osd.4) 98 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.129 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:07 smithi166 bash[9446]: cluster 2020-09-08T16:20:06.309002+0000 mgr.x (mgr.24118) 1583 : cluster [DBG] pgmap v2344: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.130 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:07 smithi166 bash[9446]: cluster 2020-09-08T16:20:06.413136+0000 mon.a (mon.0) 2470 : cluster [DBG] osdmap e846: 12 total, 12 up, 12 in 2020-09-08T16:27:02.130 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:09 smithi166 bash[9446]: cluster 2020-09-08T16:20:08.309839+0000 mgr.x (mgr.24118) 1584 : cluster [DBG] pgmap v2346: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.130 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:11 smithi166 bash[9446]: cluster 2020-09-08T16:20:10.310566+0000 mgr.x (mgr.24118) 1585 : cluster [DBG] pgmap v2347: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.130 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:13 smithi166 bash[9446]: cluster 2020-09-08T16:20:12.311236+0000 mgr.x (mgr.24118) 1586 : cluster [DBG] pgmap v2348: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.130 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:15 smithi166 bash[9446]: cluster 2020-09-08T16:20:14.311961+0000 mgr.x (mgr.24118) 1587 : cluster [DBG] pgmap v2349: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.131 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:16 smithi166 bash[9446]: cluster 2020-09-08T16:20:15.478981+0000 osd.9 (osd.9) 136 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.131 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:16 smithi166 bash[9446]: cluster 2020-09-08T16:20:15.479101+0000 osd.9 (osd.9) 137 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.131 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:17 smithi166 bash[9446]: cluster 2020-09-08T16:20:16.312636+0000 mgr.x (mgr.24118) 1588 : cluster [DBG] pgmap v2350: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.131 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:17 smithi166 bash[9446]: cluster 2020-09-08T16:20:16.393448+0000 mon.a (mon.0) 2471 : cluster [DBG] osdmap e847: 12 total, 12 up, 12 in 2020-09-08T16:27:02.131 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:18 smithi166 bash[9446]: cluster 2020-09-08T16:20:17.306375+0000 osd.2 (osd.2) 132 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.132 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:18 smithi166 bash[9446]: cluster 2020-09-08T16:20:17.306485+0000 osd.2 (osd.2) 133 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.132 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:18 smithi166 bash[9446]: cluster 2020-09-08T16:20:17.402201+0000 mon.a (mon.0) 2472 : cluster [DBG] osdmap e848: 12 total, 12 up, 12 in 2020-09-08T16:27:02.132 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:19 smithi166 bash[9446]: cluster 2020-09-08T16:20:18.313348+0000 mgr.x (mgr.24118) 1589 : cluster [DBG] pgmap v2353: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.132 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:21 smithi166 bash[9446]: cluster 2020-09-08T16:20:20.313853+0000 mgr.x (mgr.24118) 1590 : cluster [DBG] pgmap v2354: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.132 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:23 smithi166 bash[9446]: cluster 2020-09-08T16:20:22.314502+0000 mgr.x (mgr.24118) 1591 : cluster [DBG] pgmap v2355: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.133 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:25 smithi166 bash[9446]: cluster 2020-09-08T16:20:24.315223+0000 mgr.x (mgr.24118) 1592 : cluster [DBG] pgmap v2356: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.133 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:25 smithi166 bash[9446]: cluster 2020-09-08T16:20:24.707385+0000 osd.7 (osd.7) 188 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.133 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:25 smithi166 bash[9446]: cluster 2020-09-08T16:20:24.707509+0000 osd.7 (osd.7) 189 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.133 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:26 smithi166 bash[9446]: cluster 2020-09-08T16:20:25.382370+0000 mon.a (mon.0) 2473 : cluster [DBG] osdmap e849: 12 total, 12 up, 12 in 2020-09-08T16:27:02.133 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:26 smithi166 bash[9446]: cluster 2020-09-08T16:20:25.588412+0000 osd.0 (osd.0) 148 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.134 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:26 smithi166 bash[9446]: cluster 2020-09-08T16:20:25.588570+0000 osd.0 (osd.0) 149 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.135 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:27 smithi166 bash[9446]: cluster 2020-09-08T16:20:26.315850+0000 mgr.x (mgr.24118) 1593 : cluster [DBG] pgmap v2358: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.135 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:27 smithi166 bash[9446]: cluster 2020-09-08T16:20:26.414678+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e850: 12 total, 12 up, 12 in 2020-09-08T16:27:02.135 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:29 smithi166 bash[9446]: cluster 2020-09-08T16:20:28.316580+0000 mgr.x (mgr.24118) 1594 : cluster [DBG] pgmap v2360: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.136 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:31 smithi166 bash[9446]: cluster 2020-09-08T16:20:30.317240+0000 mgr.x (mgr.24118) 1595 : cluster [DBG] pgmap v2361: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.136 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:33 smithi166 bash[9446]: cluster 2020-09-08T16:20:32.317937+0000 mgr.x (mgr.24118) 1596 : cluster [DBG] pgmap v2362: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.136 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:35 smithi166 bash[9446]: cluster 2020-09-08T16:20:34.318671+0000 mgr.x (mgr.24118) 1597 : cluster [DBG] pgmap v2363: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.136 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:36 smithi166 bash[9446]: cluster 2020-09-08T16:20:35.020359+0000 osd.5 (osd.5) 154 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.137 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:36 smithi166 bash[9446]: cluster 2020-09-08T16:20:35.020517+0000 osd.5 (osd.5) 155 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.137 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:36 smithi166 bash[9446]: cluster 2020-09-08T16:20:35.391055+0000 mon.a (mon.0) 2475 : cluster [DBG] osdmap e851: 12 total, 12 up, 12 in 2020-09-08T16:27:02.137 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:37 smithi166 bash[9446]: cluster 2020-09-08T16:20:36.319398+0000 mgr.x (mgr.24118) 1598 : cluster [DBG] pgmap v2365: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.137 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:37 smithi166 bash[9446]: cluster 2020-09-08T16:20:37.088845+0000 osd.10 (osd.10) 133 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.138 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:37 smithi166 bash[9446]: cluster 2020-09-08T16:20:37.088962+0000 osd.10 (osd.10) 134 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.138 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:38 smithi166 bash[9446]: cluster 2020-09-08T16:20:37.405413+0000 mon.a (mon.0) 2476 : cluster [DBG] osdmap e852: 12 total, 12 up, 12 in 2020-09-08T16:27:02.138 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:38 smithi166 bash[9446]: cluster 2020-09-08T16:20:37.452514+0000 osd.3 (osd.3) 157 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.139 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:38 smithi166 bash[9446]: cluster 2020-09-08T16:20:37.452628+0000 osd.3 (osd.3) 158 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.139 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:39 smithi166 bash[9446]: cluster 2020-09-08T16:20:38.320060+0000 mgr.x (mgr.24118) 1599 : cluster [DBG] pgmap v2367: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.139 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:39 smithi166 bash[9446]: cluster 2020-09-08T16:20:38.427057+0000 mon.a (mon.0) 2477 : cluster [DBG] osdmap e853: 12 total, 12 up, 12 in 2020-09-08T16:27:02.139 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:41 smithi166 bash[9446]: cluster 2020-09-08T16:20:40.320532+0000 mgr.x (mgr.24118) 1600 : cluster [DBG] pgmap v2369: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.139 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:43 smithi166 bash[9446]: cluster 2020-09-08T16:20:42.321174+0000 mgr.x (mgr.24118) 1601 : cluster [DBG] pgmap v2370: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.140 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:45 smithi166 bash[9446]: cluster 2020-09-08T16:20:44.321907+0000 mgr.x (mgr.24118) 1602 : cluster [DBG] pgmap v2371: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.140 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:46 smithi166 bash[9446]: cluster 2020-09-08T16:20:45.563606+0000 osd.8 (osd.8) 128 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.140 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:46 smithi166 bash[9446]: cluster 2020-09-08T16:20:45.563735+0000 osd.8 (osd.8) 129 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.140 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:47 smithi166 bash[9446]: cluster 2020-09-08T16:20:46.322795+0000 mgr.x (mgr.24118) 1603 : cluster [DBG] pgmap v2372: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.141 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:47 smithi166 bash[9446]: cluster 2020-09-08T16:20:46.404936+0000 mon.a (mon.0) 2478 : cluster [DBG] osdmap e854: 12 total, 12 up, 12 in 2020-09-08T16:27:02.141 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:49 smithi166 bash[9446]: cluster 2020-09-08T16:20:48.323584+0000 mgr.x (mgr.24118) 1604 : cluster [DBG] pgmap v2374: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.141 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:51 smithi166 bash[9446]: cluster 2020-09-08T16:20:50.324248+0000 mgr.x (mgr.24118) 1605 : cluster [DBG] pgmap v2375: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.141 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:53 smithi166 bash[9446]: cluster 2020-09-08T16:20:52.088960+0000 osd.1 (osd.1) 174 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.142 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:53 smithi166 bash[9446]: cluster 2020-09-08T16:20:52.089112+0000 osd.1 (osd.1) 175 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.143 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:53 smithi166 bash[9446]: cluster 2020-09-08T16:20:52.259962+0000 mon.a (mon.0) 2479 : cluster [DBG] osdmap e855: 12 total, 12 up, 12 in 2020-09-08T16:27:02.143 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:53 smithi166 bash[9446]: cluster 2020-09-08T16:20:52.324907+0000 mgr.x (mgr.24118) 1606 : cluster [DBG] pgmap v2377: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.143 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:55 smithi166 bash[9446]: cluster 2020-09-08T16:20:54.325657+0000 mgr.x (mgr.24118) 1607 : cluster [DBG] pgmap v2378: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.143 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:57 smithi166 bash[9446]: cluster 2020-09-08T16:20:56.326288+0000 mgr.x (mgr.24118) 1608 : cluster [DBG] pgmap v2379: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.144 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:59 smithi166 bash[9446]: cluster 2020-09-08T16:20:57.812662+0000 osd.6 (osd.6) 139 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.144 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:59 smithi166 bash[9446]: cluster 2020-09-08T16:20:57.812796+0000 osd.6 (osd.6) 140 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.144 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:59 smithi166 bash[9446]: cluster 2020-09-08T16:20:58.327078+0000 mgr.x (mgr.24118) 1609 : cluster [DBG] pgmap v2380: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.145 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:20:59 smithi166 bash[9446]: cluster 2020-09-08T16:20:58.404772+0000 mon.a (mon.0) 2480 : cluster [DBG] osdmap e856: 12 total, 12 up, 12 in 2020-09-08T16:27:02.145 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:01 smithi166 bash[9446]: cluster 2020-09-08T16:21:00.327722+0000 mgr.x (mgr.24118) 1610 : cluster [DBG] pgmap v2382: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.145 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:03 smithi166 bash[9446]: cluster 2020-09-08T16:21:02.328378+0000 mgr.x (mgr.24118) 1611 : cluster [DBG] pgmap v2383: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.145 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:04 smithi166 bash[9446]: cluster 2020-09-08T16:21:03.333742+0000 osd.11 (osd.11) 154 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.146 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:04 smithi166 bash[9446]: cluster 2020-09-08T16:21:03.333871+0000 osd.11 (osd.11) 155 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.146 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:04 smithi166 bash[9446]: cluster 2020-09-08T16:21:03.400773+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e857: 12 total, 12 up, 12 in 2020-09-08T16:27:02.146 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:05 smithi166 bash[9446]: cluster 2020-09-08T16:21:04.329012+0000 mgr.x (mgr.24118) 1612 : cluster [DBG] pgmap v2385: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.146 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:07 smithi166 bash[9446]: cluster 2020-09-08T16:21:06.106338+0000 osd.4 (osd.4) 99 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.146 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:07 smithi166 bash[9446]: cluster 2020-09-08T16:21:06.106465+0000 osd.4 (osd.4) 100 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.147 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:07 smithi166 bash[9446]: cluster 2020-09-08T16:21:06.329766+0000 mgr.x (mgr.24118) 1613 : cluster [DBG] pgmap v2386: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.147 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:07 smithi166 bash[9446]: cluster 2020-09-08T16:21:06.421076+0000 mon.a (mon.0) 2482 : cluster [DBG] osdmap e858: 12 total, 12 up, 12 in 2020-09-08T16:27:02.147 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:09 smithi166 bash[9446]: cluster 2020-09-08T16:21:08.330615+0000 mgr.x (mgr.24118) 1614 : cluster [DBG] pgmap v2388: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.147 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:11 smithi166 bash[9446]: cluster 2020-09-08T16:21:10.331273+0000 mgr.x (mgr.24118) 1615 : cluster [DBG] pgmap v2389: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.147 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:13 smithi166 bash[9446]: cluster 2020-09-08T16:21:12.331978+0000 mgr.x (mgr.24118) 1616 : cluster [DBG] pgmap v2390: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.148 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:15 smithi166 bash[9446]: cluster 2020-09-08T16:21:14.332692+0000 mgr.x (mgr.24118) 1617 : cluster [DBG] pgmap v2391: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.148 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:17 smithi166 bash[9446]: cluster 2020-09-08T16:21:16.333447+0000 mgr.x (mgr.24118) 1618 : cluster [DBG] pgmap v2392: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.148 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:17 smithi166 bash[9446]: cluster 2020-09-08T16:21:16.362679+0000 osd.9 (osd.9) 138 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.148 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:17 smithi166 bash[9446]: cluster 2020-09-08T16:21:16.362811+0000 osd.9 (osd.9) 139 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.148 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:17 smithi166 bash[9446]: cluster 2020-09-08T16:21:16.409000+0000 mon.a (mon.0) 2483 : cluster [DBG] osdmap e859: 12 total, 12 up, 12 in 2020-09-08T16:27:02.149 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:18 smithi166 bash[9446]: cluster 2020-09-08T16:21:17.547440+0000 osd.2 (osd.2) 134 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.149 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:18 smithi166 bash[9446]: cluster 2020-09-08T16:21:17.547614+0000 osd.2 (osd.2) 135 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.149 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:19 smithi166 bash[9446]: cluster 2020-09-08T16:21:18.334352+0000 mgr.x (mgr.24118) 1619 : cluster [DBG] pgmap v2394: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.149 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:19 smithi166 bash[9446]: cluster 2020-09-08T16:21:18.423887+0000 mon.a (mon.0) 2484 : cluster [DBG] osdmap e860: 12 total, 12 up, 12 in 2020-09-08T16:27:02.149 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:21 smithi166 bash[9446]: cluster 2020-09-08T16:21:20.335075+0000 mgr.x (mgr.24118) 1620 : cluster [DBG] pgmap v2396: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.150 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:23 smithi166 bash[9446]: cluster 2020-09-08T16:21:22.335851+0000 mgr.x (mgr.24118) 1621 : cluster [DBG] pgmap v2397: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.150 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:25 smithi166 bash[9446]: cluster 2020-09-08T16:21:24.336633+0000 mgr.x (mgr.24118) 1622 : cluster [DBG] pgmap v2398: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.150 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:26 smithi166 bash[9446]: cluster 2020-09-08T16:21:25.673730+0000 osd.7 (osd.7) 190 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.150 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:26 smithi166 bash[9446]: cluster 2020-09-08T16:21:25.673860+0000 osd.7 (osd.7) 191 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.150 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:26 smithi166 bash[9446]: cluster 2020-09-08T16:21:25.674272+0000 osd.0 (osd.0) 150 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.151 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:26 smithi166 bash[9446]: cluster 2020-09-08T16:21:25.674416+0000 osd.0 (osd.0) 151 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.151 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:27 smithi166 bash[9446]: cluster 2020-09-08T16:21:26.337306+0000 mgr.x (mgr.24118) 1623 : cluster [DBG] pgmap v2399: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.151 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:27 smithi166 bash[9446]: cluster 2020-09-08T16:21:26.418317+0000 mon.a (mon.0) 2485 : cluster [DBG] osdmap e861: 12 total, 12 up, 12 in 2020-09-08T16:27:02.151 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:29 smithi166 bash[9446]: cluster 2020-09-08T16:21:28.338012+0000 mgr.x (mgr.24118) 1624 : cluster [DBG] pgmap v2401: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.151 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:31 smithi166 bash[9446]: cluster 2020-09-08T16:21:30.338716+0000 mgr.x (mgr.24118) 1625 : cluster [DBG] pgmap v2402: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.151 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:33 smithi166 bash[9446]: cluster 2020-09-08T16:21:32.339416+0000 mgr.x (mgr.24118) 1626 : cluster [DBG] pgmap v2403: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.152 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:35 smithi166 bash[9446]: cluster 2020-09-08T16:21:34.340128+0000 mgr.x (mgr.24118) 1627 : cluster [DBG] pgmap v2404: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.152 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:36 smithi166 bash[9446]: cluster 2020-09-08T16:21:35.509269+0000 osd.5 (osd.5) 156 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.152 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:36 smithi166 bash[9446]: cluster 2020-09-08T16:21:35.509400+0000 osd.5 (osd.5) 157 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.152 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:37 smithi166 bash[9446]: cluster 2020-09-08T16:21:36.340795+0000 mgr.x (mgr.24118) 1628 : cluster [DBG] pgmap v2405: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.153 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:37 smithi166 bash[9446]: cluster 2020-09-08T16:21:36.420966+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e862: 12 total, 12 up, 12 in 2020-09-08T16:27:02.153 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:38 smithi166 bash[9446]: cluster 2020-09-08T16:21:37.638448+0000 osd.3 (osd.3) 159 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.153 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:38 smithi166 bash[9446]: cluster 2020-09-08T16:21:37.638607+0000 osd.3 (osd.3) 160 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.153 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:38 smithi166 bash[9446]: cluster 2020-09-08T16:21:37.868311+0000 osd.10 (osd.10) 135 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.153 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:38 smithi166 bash[9446]: cluster 2020-09-08T16:21:37.868434+0000 osd.10 (osd.10) 136 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.154 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:39 smithi166 bash[9446]: cluster 2020-09-08T16:21:38.341523+0000 mgr.x (mgr.24118) 1629 : cluster [DBG] pgmap v2407: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.154 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:39 smithi166 bash[9446]: cluster 2020-09-08T16:21:38.432367+0000 mon.a (mon.0) 2487 : cluster [DBG] osdmap e863: 12 total, 12 up, 12 in 2020-09-08T16:27:02.154 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:41 smithi166 bash[9446]: cluster 2020-09-08T16:21:40.342275+0000 mgr.x (mgr.24118) 1630 : cluster [DBG] pgmap v2409: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.154 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:43 smithi166 bash[9446]: cluster 2020-09-08T16:21:42.343020+0000 mgr.x (mgr.24118) 1631 : cluster [DBG] pgmap v2410: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.154 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:45 smithi166 bash[9446]: cluster 2020-09-08T16:21:44.343760+0000 mgr.x (mgr.24118) 1632 : cluster [DBG] pgmap v2411: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.155 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:47 smithi166 bash[9446]: cluster 2020-09-08T16:21:45.745329+0000 osd.8 (osd.8) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.155 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:47 smithi166 bash[9446]: cluster 2020-09-08T16:21:45.745456+0000 osd.8 (osd.8) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.156 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:47 smithi166 bash[9446]: cluster 2020-09-08T16:21:46.344503+0000 mgr.x (mgr.24118) 1633 : cluster [DBG] pgmap v2412: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.156 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:47 smithi166 bash[9446]: cluster 2020-09-08T16:21:46.423352+0000 mon.a (mon.0) 2488 : cluster [DBG] osdmap e864: 12 total, 12 up, 12 in 2020-09-08T16:27:02.156 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:49 smithi166 bash[9446]: cluster 2020-09-08T16:21:48.345450+0000 mgr.x (mgr.24118) 1634 : cluster [DBG] pgmap v2414: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.156 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:51 smithi166 bash[9446]: cluster 2020-09-08T16:21:50.346231+0000 mgr.x (mgr.24118) 1635 : cluster [DBG] pgmap v2415: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.156 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:53 smithi166 bash[9446]: cluster 2020-09-08T16:21:52.346930+0000 mgr.x (mgr.24118) 1636 : cluster [DBG] pgmap v2416: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.157 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:53 smithi166 bash[9446]: cluster 2020-09-08T16:21:52.538202+0000 osd.1 (osd.1) 176 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.157 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:53 smithi166 bash[9446]: cluster 2020-09-08T16:21:52.538339+0000 osd.1 (osd.1) 177 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.157 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:54 smithi166 bash[9446]: cluster 2020-09-08T16:21:53.419636+0000 mon.a (mon.0) 2489 : cluster [DBG] osdmap e865: 12 total, 12 up, 12 in 2020-09-08T16:27:02.157 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:55 smithi166 bash[9446]: cluster 2020-09-08T16:21:54.347582+0000 mgr.x (mgr.24118) 1637 : cluster [DBG] pgmap v2418: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.157 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:57 smithi166 bash[9446]: cluster 2020-09-08T16:21:56.348350+0000 mgr.x (mgr.24118) 1638 : cluster [DBG] pgmap v2419: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.158 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:59 smithi166 bash[9446]: cluster 2020-09-08T16:21:57.980972+0000 osd.6 (osd.6) 141 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.158 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:59 smithi166 bash[9446]: cluster 2020-09-08T16:21:57.981104+0000 osd.6 (osd.6) 142 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.158 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:59 smithi166 bash[9446]: cluster 2020-09-08T16:21:58.349119+0000 mgr.x (mgr.24118) 1639 : cluster [DBG] pgmap v2420: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.158 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:21:59 smithi166 bash[9446]: cluster 2020-09-08T16:21:58.489201+0000 mon.a (mon.0) 2490 : cluster [DBG] osdmap e866: 12 total, 12 up, 12 in 2020-09-08T16:27:02.158 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:01 smithi166 bash[9446]: cluster 2020-09-08T16:22:00.349788+0000 mgr.x (mgr.24118) 1640 : cluster [DBG] pgmap v2422: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.159 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:03 smithi166 bash[9446]: cluster 2020-09-08T16:22:02.350450+0000 mgr.x (mgr.24118) 1641 : cluster [DBG] pgmap v2423: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.159 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:04 smithi166 bash[9446]: cluster 2020-09-08T16:22:03.859539+0000 osd.11 (osd.11) 156 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.159 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:04 smithi166 bash[9446]: cluster 2020-09-08T16:22:03.859668+0000 osd.11 (osd.11) 157 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.159 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:05 smithi166 bash[9446]: cluster 2020-09-08T16:22:04.351125+0000 mgr.x (mgr.24118) 1642 : cluster [DBG] pgmap v2424: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.159 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:05 smithi166 bash[9446]: cluster 2020-09-08T16:22:04.426415+0000 mon.a (mon.0) 2491 : cluster [DBG] osdmap e867: 12 total, 12 up, 12 in 2020-09-08T16:27:02.160 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:07 smithi166 bash[9446]: cluster 2020-09-08T16:22:06.292113+0000 osd.4 (osd.4) 101 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.160 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:07 smithi166 bash[9446]: cluster 2020-09-08T16:22:06.292235+0000 osd.4 (osd.4) 102 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.160 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:07 smithi166 bash[9446]: cluster 2020-09-08T16:22:06.351776+0000 mgr.x (mgr.24118) 1643 : cluster [DBG] pgmap v2426: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.160 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:07 smithi166 bash[9446]: cluster 2020-09-08T16:22:06.437707+0000 mon.a (mon.0) 2492 : cluster [DBG] osdmap e868: 12 total, 12 up, 12 in 2020-09-08T16:27:02.160 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:09 smithi166 bash[9446]: cluster 2020-09-08T16:22:08.352672+0000 mgr.x (mgr.24118) 1644 : cluster [DBG] pgmap v2428: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.161 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:11 smithi166 bash[9446]: cluster 2020-09-08T16:22:10.353519+0000 mgr.x (mgr.24118) 1645 : cluster [DBG] pgmap v2429: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.161 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:13 smithi166 bash[9446]: cluster 2020-09-08T16:22:12.354224+0000 mgr.x (mgr.24118) 1646 : cluster [DBG] pgmap v2430: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.161 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:15 smithi166 bash[9446]: cluster 2020-09-08T16:22:14.354913+0000 mgr.x (mgr.24118) 1647 : cluster [DBG] pgmap v2431: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.161 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:17 smithi166 bash[9446]: cluster 2020-09-08T16:22:16.355583+0000 mgr.x (mgr.24118) 1648 : cluster [DBG] pgmap v2432: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.161 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:17 smithi166 bash[9446]: cluster 2020-09-08T16:22:16.660205+0000 osd.9 (osd.9) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.162 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:17 smithi166 bash[9446]: cluster 2020-09-08T16:22:16.660363+0000 osd.9 (osd.9) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.162 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:18 smithi166 bash[9446]: cluster 2020-09-08T16:22:17.427768+0000 mon.a (mon.0) 2493 : cluster [DBG] osdmap e869: 12 total, 12 up, 12 in 2020-09-08T16:27:02.162 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:18 smithi166 bash[9446]: cluster 2020-09-08T16:22:17.988554+0000 osd.2 (osd.2) 136 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.162 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:18 smithi166 bash[9446]: cluster 2020-09-08T16:22:17.988686+0000 osd.2 (osd.2) 137 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.162 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:19 smithi166 bash[9446]: cluster 2020-09-08T16:22:18.356281+0000 mgr.x (mgr.24118) 1649 : cluster [DBG] pgmap v2434: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.163 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:19 smithi166 bash[9446]: cluster 2020-09-08T16:22:18.446223+0000 mon.a (mon.0) 2494 : cluster [DBG] osdmap e870: 12 total, 12 up, 12 in 2020-09-08T16:27:02.163 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:21 smithi166 bash[9446]: cluster 2020-09-08T16:22:20.356945+0000 mgr.x (mgr.24118) 1650 : cluster [DBG] pgmap v2436: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.163 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:23 smithi166 bash[9446]: cluster 2020-09-08T16:22:22.357656+0000 mgr.x (mgr.24118) 1651 : cluster [DBG] pgmap v2437: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.163 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:25 smithi166 bash[9446]: cluster 2020-09-08T16:22:24.358372+0000 mgr.x (mgr.24118) 1652 : cluster [DBG] pgmap v2438: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.163 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:27 smithi166 bash[9446]: cluster 2020-09-08T16:22:26.047624+0000 osd.7 (osd.7) 192 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.164 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:27 smithi166 bash[9446]: cluster 2020-09-08T16:22:26.047749+0000 osd.7 (osd.7) 193 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.164 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:27 smithi166 bash[9446]: cluster 2020-09-08T16:22:26.358984+0000 mgr.x (mgr.24118) 1653 : cluster [DBG] pgmap v2439: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.164 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:27 smithi166 bash[9446]: cluster 2020-09-08T16:22:26.438539+0000 mon.a (mon.0) 2495 : cluster [DBG] osdmap e871: 12 total, 12 up, 12 in 2020-09-08T16:27:02.164 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:27 smithi166 bash[9446]: cluster 2020-09-08T16:22:26.684716+0000 osd.0 (osd.0) 152 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.164 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:27 smithi166 bash[9446]: cluster 2020-09-08T16:22:26.684841+0000 osd.0 (osd.0) 153 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.165 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:28 smithi166 bash[9446]: cluster 2020-09-08T16:22:27.441484+0000 mon.a (mon.0) 2496 : cluster [DBG] osdmap e872: 12 total, 12 up, 12 in 2020-09-08T16:27:02.165 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:29 smithi166 bash[9446]: cluster 2020-09-08T16:22:28.359677+0000 mgr.x (mgr.24118) 1654 : cluster [DBG] pgmap v2442: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.165 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:31 smithi166 bash[9446]: cluster 2020-09-08T16:22:30.360347+0000 mgr.x (mgr.24118) 1655 : cluster [DBG] pgmap v2443: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.166 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:33 smithi166 bash[9446]: cluster 2020-09-08T16:22:32.361009+0000 mgr.x (mgr.24118) 1656 : cluster [DBG] pgmap v2444: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.166 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:35 smithi166 bash[9446]: cluster 2020-09-08T16:22:34.361722+0000 mgr.x (mgr.24118) 1657 : cluster [DBG] pgmap v2445: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.166 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:37 smithi166 bash[9446]: cluster 2020-09-08T16:22:36.201081+0000 osd.5 (osd.5) 158 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.166 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:37 smithi166 bash[9446]: cluster 2020-09-08T16:22:36.201209+0000 osd.5 (osd.5) 159 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.166 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:37 smithi166 bash[9446]: cluster 2020-09-08T16:22:36.362454+0000 mgr.x (mgr.24118) 1658 : cluster [DBG] pgmap v2446: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.167 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:37 smithi166 bash[9446]: cluster 2020-09-08T16:22:36.438961+0000 mon.a (mon.0) 2497 : cluster [DBG] osdmap e873: 12 total, 12 up, 12 in 2020-09-08T16:27:02.167 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:38 smithi166 bash[9446]: cluster 2020-09-08T16:22:37.838485+0000 osd.3 (osd.3) 161 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.167 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:38 smithi166 bash[9446]: cluster 2020-09-08T16:22:37.838627+0000 osd.3 (osd.3) 162 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.167 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:39 smithi166 bash[9446]: cluster 2020-09-08T16:22:38.363202+0000 mgr.x (mgr.24118) 1659 : cluster [DBG] pgmap v2448: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.167 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:39 smithi166 bash[9446]: cluster 2020-09-08T16:22:38.454615+0000 mon.a (mon.0) 2498 : cluster [DBG] osdmap e874: 12 total, 12 up, 12 in 2020-09-08T16:27:02.168 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:39 smithi166 bash[9446]: cluster 2020-09-08T16:22:38.838416+0000 osd.10 (osd.10) 137 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.168 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:39 smithi166 bash[9446]: cluster 2020-09-08T16:22:38.838535+0000 osd.10 (osd.10) 138 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.168 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:40 smithi166 bash[9446]: cluster 2020-09-08T16:22:39.463943+0000 mon.a (mon.0) 2499 : cluster [DBG] osdmap e875: 12 total, 12 up, 12 in 2020-09-08T16:27:02.168 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:41 smithi166 bash[9446]: cluster 2020-09-08T16:22:40.363881+0000 mgr.x (mgr.24118) 1660 : cluster [DBG] pgmap v2451: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.168 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:43 smithi166 bash[9446]: cluster 2020-09-08T16:22:42.364535+0000 mgr.x (mgr.24118) 1661 : cluster [DBG] pgmap v2452: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.169 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:45 smithi166 bash[9446]: cluster 2020-09-08T16:22:44.365151+0000 mgr.x (mgr.24118) 1662 : cluster [DBG] pgmap v2453: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.169 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:47 smithi166 bash[9446]: cluster 2020-09-08T16:22:45.923608+0000 osd.8 (osd.8) 132 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.169 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:47 smithi166 bash[9446]: cluster 2020-09-08T16:22:45.923735+0000 osd.8 (osd.8) 133 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.169 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:47 smithi166 bash[9446]: cluster 2020-09-08T16:22:46.365787+0000 mgr.x (mgr.24118) 1663 : cluster [DBG] pgmap v2454: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.169 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:47 smithi166 bash[9446]: cluster 2020-09-08T16:22:46.517760+0000 mon.a (mon.0) 2500 : cluster [DBG] osdmap e876: 12 total, 12 up, 12 in 2020-09-08T16:27:02.170 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:49 smithi166 bash[9446]: cluster 2020-09-08T16:22:48.366566+0000 mgr.x (mgr.24118) 1664 : cluster [DBG] pgmap v2456: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.170 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:51 smithi166 bash[9446]: cluster 2020-09-08T16:22:50.367302+0000 mgr.x (mgr.24118) 1665 : cluster [DBG] pgmap v2457: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.170 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:53 smithi166 bash[9446]: cluster 2020-09-08T16:22:52.368146+0000 mgr.x (mgr.24118) 1666 : cluster [DBG] pgmap v2458: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.170 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:54 smithi166 bash[9446]: cluster 2020-09-08T16:22:53.374169+0000 osd.1 (osd.1) 178 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.170 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:54 smithi166 bash[9446]: cluster 2020-09-08T16:22:53.374299+0000 osd.1 (osd.1) 179 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.171 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:54 smithi166 bash[9446]: cluster 2020-09-08T16:22:53.430286+0000 mon.a (mon.0) 2501 : cluster [DBG] osdmap e877: 12 total, 12 up, 12 in 2020-09-08T16:27:02.171 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:55 smithi166 bash[9446]: cluster 2020-09-08T16:22:54.368841+0000 mgr.x (mgr.24118) 1667 : cluster [DBG] pgmap v2460: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.171 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:57 smithi166 bash[9446]: cluster 2020-09-08T16:22:56.369497+0000 mgr.x (mgr.24118) 1668 : cluster [DBG] pgmap v2461: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.171 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:59 smithi166 bash[9446]: cluster 2020-09-08T16:22:58.338433+0000 osd.6 (osd.6) 143 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.171 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:59 smithi166 bash[9446]: cluster 2020-09-08T16:22:58.338558+0000 osd.6 (osd.6) 144 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.172 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:59 smithi166 bash[9446]: cluster 2020-09-08T16:22:58.370232+0000 mgr.x (mgr.24118) 1669 : cluster [DBG] pgmap v2462: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.172 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:22:59 smithi166 bash[9446]: cluster 2020-09-08T16:22:58.512036+0000 mon.a (mon.0) 2502 : cluster [DBG] osdmap e878: 12 total, 12 up, 12 in 2020-09-08T16:27:02.172 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:01 smithi166 bash[9446]: cluster 2020-09-08T16:23:00.371005+0000 mgr.x (mgr.24118) 1670 : cluster [DBG] pgmap v2464: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.172 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:03 smithi166 bash[9446]: cluster 2020-09-08T16:23:02.371690+0000 mgr.x (mgr.24118) 1671 : cluster [DBG] pgmap v2465: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.172 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:05 smithi166 bash[9446]: cluster 2020-09-08T16:23:04.324373+0000 osd.11 (osd.11) 158 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.173 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:05 smithi166 bash[9446]: cluster 2020-09-08T16:23:04.324501+0000 osd.11 (osd.11) 159 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.173 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:05 smithi166 bash[9446]: cluster 2020-09-08T16:23:04.372389+0000 mgr.x (mgr.24118) 1672 : cluster [DBG] pgmap v2466: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.173 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:05 smithi166 bash[9446]: cluster 2020-09-08T16:23:04.443525+0000 mon.a (mon.0) 2503 : cluster [DBG] osdmap e879: 12 total, 12 up, 12 in 2020-09-08T16:27:02.173 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:07 smithi166 bash[9446]: cluster 2020-09-08T16:23:06.373040+0000 mgr.x (mgr.24118) 1673 : cluster [DBG] pgmap v2468: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.173 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:08 smithi166 bash[9446]: cluster 2020-09-08T16:23:07.305093+0000 osd.4 (osd.4) 103 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.174 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:08 smithi166 bash[9446]: cluster 2020-09-08T16:23:07.305222+0000 osd.4 (osd.4) 104 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.174 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:08 smithi166 bash[9446]: cluster 2020-09-08T16:23:07.453557+0000 mon.a (mon.0) 2504 : cluster [DBG] osdmap e880: 12 total, 12 up, 12 in 2020-09-08T16:27:02.174 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:09 smithi166 bash[9446]: cluster 2020-09-08T16:23:08.373752+0000 mgr.x (mgr.24118) 1674 : cluster [DBG] pgmap v2470: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.174 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:11 smithi166 bash[9446]: cluster 2020-09-08T16:23:10.374490+0000 mgr.x (mgr.24118) 1675 : cluster [DBG] pgmap v2471: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.174 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:13 smithi166 bash[9446]: cluster 2020-09-08T16:23:12.375180+0000 mgr.x (mgr.24118) 1676 : cluster [DBG] pgmap v2472: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.175 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:15 smithi166 bash[9446]: cluster 2020-09-08T16:23:14.375859+0000 mgr.x (mgr.24118) 1677 : cluster [DBG] pgmap v2473: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.175 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:17 smithi166 bash[9446]: cluster 2020-09-08T16:23:16.376528+0000 mgr.x (mgr.24118) 1678 : cluster [DBG] pgmap v2474: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.175 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:18 smithi166 bash[9446]: cluster 2020-09-08T16:23:17.566849+0000 osd.9 (osd.9) 142 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.175 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:18 smithi166 bash[9446]: cluster 2020-09-08T16:23:17.566983+0000 osd.9 (osd.9) 143 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.175 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:18 smithi166 bash[9446]: cluster 2020-09-08T16:23:18.155326+0000 osd.2 (osd.2) 138 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.176 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:18 smithi166 bash[9446]: cluster 2020-09-08T16:23:18.155443+0000 osd.2 (osd.2) 139 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.176 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:19 smithi166 bash[9446]: cluster 2020-09-08T16:23:18.377221+0000 mgr.x (mgr.24118) 1679 : cluster [DBG] pgmap v2475: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.176 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:19 smithi166 bash[9446]: cluster 2020-09-08T16:23:18.463865+0000 mon.a (mon.0) 2505 : cluster [DBG] osdmap e881: 12 total, 12 up, 12 in 2020-09-08T16:27:02.177 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:21 smithi166 bash[9446]: cluster 2020-09-08T16:23:20.377865+0000 mgr.x (mgr.24118) 1680 : cluster [DBG] pgmap v2477: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.177 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:23 smithi166 bash[9446]: cluster 2020-09-08T16:23:22.378582+0000 mgr.x (mgr.24118) 1681 : cluster [DBG] pgmap v2478: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.177 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:25 smithi166 bash[9446]: cluster 2020-09-08T16:23:24.379267+0000 mgr.x (mgr.24118) 1682 : cluster [DBG] pgmap v2479: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.177 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:27 smithi166 bash[9446]: cluster 2020-09-08T16:23:26.379953+0000 mgr.x (mgr.24118) 1683 : cluster [DBG] pgmap v2480: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.178 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:27 smithi166 bash[9446]: cluster 2020-09-08T16:23:26.963247+0000 osd.0 (osd.0) 154 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.178 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:27 smithi166 bash[9446]: cluster 2020-09-08T16:23:26.963376+0000 osd.0 (osd.0) 155 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.178 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:28 smithi166 bash[9446]: cluster 2020-09-08T16:23:27.002998+0000 osd.7 (osd.7) 194 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.178 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:28 smithi166 bash[9446]: cluster 2020-09-08T16:23:27.003141+0000 osd.7 (osd.7) 195 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.178 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:28 smithi166 bash[9446]: cluster 2020-09-08T16:23:27.453332+0000 mon.a (mon.0) 2506 : cluster [DBG] osdmap e882: 12 total, 12 up, 12 in 2020-09-08T16:27:02.178 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:29 smithi166 bash[9446]: cluster 2020-09-08T16:23:28.380663+0000 mgr.x (mgr.24118) 1684 : cluster [DBG] pgmap v2482: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.179 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:31 smithi166 bash[9446]: cluster 2020-09-08T16:23:30.381335+0000 mgr.x (mgr.24118) 1685 : cluster [DBG] pgmap v2483: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.179 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:33 smithi166 bash[9446]: cluster 2020-09-08T16:23:32.382017+0000 mgr.x (mgr.24118) 1686 : cluster [DBG] pgmap v2484: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.179 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:35 smithi166 bash[9446]: cluster 2020-09-08T16:23:34.382751+0000 mgr.x (mgr.24118) 1687 : cluster [DBG] pgmap v2485: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.179 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:37 smithi166 bash[9446]: cluster 2020-09-08T16:23:36.383441+0000 mgr.x (mgr.24118) 1688 : cluster [DBG] pgmap v2486: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.179 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:37 smithi166 bash[9446]: cluster 2020-09-08T16:23:36.568336+0000 osd.5 (osd.5) 160 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.180 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:37 smithi166 bash[9446]: cluster 2020-09-08T16:23:36.568492+0000 osd.5 (osd.5) 161 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.180 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:38 smithi166 bash[9446]: cluster 2020-09-08T16:23:37.486484+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e883: 12 total, 12 up, 12 in 2020-09-08T16:27:02.180 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:38 smithi166 bash[9446]: cluster 2020-09-08T16:23:37.905775+0000 osd.3 (osd.3) 163 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.180 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:38 smithi166 bash[9446]: cluster 2020-09-08T16:23:37.905899+0000 osd.3 (osd.3) 164 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.180 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:39 smithi166 bash[9446]: cluster 2020-09-08T16:23:38.384285+0000 mgr.x (mgr.24118) 1689 : cluster [DBG] pgmap v2488: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.181 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:39 smithi166 bash[9446]: cluster 2020-09-08T16:23:38.499022+0000 mon.a (mon.0) 2508 : cluster [DBG] osdmap e884: 12 total, 12 up, 12 in 2020-09-08T16:27:02.181 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:40 smithi166 bash[9446]: cluster 2020-09-08T16:23:39.576603+0000 osd.10 (osd.10) 139 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.181 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:40 smithi166 bash[9446]: cluster 2020-09-08T16:23:39.576724+0000 osd.10 (osd.10) 140 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.181 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:41 smithi166 bash[9446]: cluster 2020-09-08T16:23:40.384981+0000 mgr.x (mgr.24118) 1690 : cluster [DBG] pgmap v2490: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.181 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:41 smithi166 bash[9446]: cluster 2020-09-08T16:23:40.503892+0000 mon.a (mon.0) 2509 : cluster [DBG] osdmap e885: 12 total, 12 up, 12 in 2020-09-08T16:27:02.182 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:43 smithi166 bash[9446]: cluster 2020-09-08T16:23:42.385743+0000 mgr.x (mgr.24118) 1691 : cluster [DBG] pgmap v2492: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.182 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:45 smithi166 bash[9446]: cluster 2020-09-08T16:23:44.386546+0000 mgr.x (mgr.24118) 1692 : cluster [DBG] pgmap v2493: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.182 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:47 smithi166 bash[9446]: cluster 2020-09-08T16:23:46.387299+0000 mgr.x (mgr.24118) 1693 : cluster [DBG] pgmap v2494: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.182 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:48 smithi166 bash[9446]: cluster 2020-09-08T16:23:46.640159+0000 osd.8 (osd.8) 134 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.182 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:48 smithi166 bash[9446]: cluster 2020-09-08T16:23:46.640302+0000 osd.8 (osd.8) 135 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.183 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:48 smithi166 bash[9446]: cluster 2020-09-08T16:23:47.531865+0000 mon.a (mon.0) 2510 : cluster [DBG] osdmap e886: 12 total, 12 up, 12 in 2020-09-08T16:27:02.183 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:49 smithi166 bash[9446]: cluster 2020-09-08T16:23:48.388036+0000 mgr.x (mgr.24118) 1694 : cluster [DBG] pgmap v2496: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.183 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:51 smithi166 bash[9446]: cluster 2020-09-08T16:23:50.388662+0000 mgr.x (mgr.24118) 1695 : cluster [DBG] pgmap v2497: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.183 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:53 smithi166 bash[9446]: cluster 2020-09-08T16:23:52.389418+0000 mgr.x (mgr.24118) 1696 : cluster [DBG] pgmap v2498: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.184 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:55 smithi166 bash[9446]: cluster 2020-09-08T16:23:54.236396+0000 osd.1 (osd.1) 180 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.185 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:55 smithi166 bash[9446]: cluster 2020-09-08T16:23:54.236544+0000 osd.1 (osd.1) 181 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.185 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:55 smithi166 bash[9446]: cluster 2020-09-08T16:23:54.390149+0000 mgr.x (mgr.24118) 1697 : cluster [DBG] pgmap v2499: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.185 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:55 smithi166 bash[9446]: cluster 2020-09-08T16:23:54.464578+0000 mon.a (mon.0) 2511 : cluster [DBG] osdmap e887: 12 total, 12 up, 12 in 2020-09-08T16:27:02.186 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:57 smithi166 bash[9446]: cluster 2020-09-08T16:23:56.390826+0000 mgr.x (mgr.24118) 1698 : cluster [DBG] pgmap v2501: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.186 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:23:59 smithi166 bash[9446]: cluster 2020-09-08T16:23:58.391558+0000 mgr.x (mgr.24118) 1699 : cluster [DBG] pgmap v2502: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.187 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:00 smithi166 bash[9446]: cluster 2020-09-08T16:23:59.299180+0000 osd.6 (osd.6) 145 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.187 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:00 smithi166 bash[9446]: cluster 2020-09-08T16:23:59.299303+0000 osd.6 (osd.6) 146 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.187 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:00 smithi166 bash[9446]: cluster 2020-09-08T16:23:59.521102+0000 mon.a (mon.0) 2512 : cluster [DBG] osdmap e888: 12 total, 12 up, 12 in 2020-09-08T16:27:02.187 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:01 smithi166 bash[9446]: cluster 2020-09-08T16:24:00.392224+0000 mgr.x (mgr.24118) 1700 : cluster [DBG] pgmap v2504: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.187 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:03 smithi166 bash[9446]: cluster 2020-09-08T16:24:02.392901+0000 mgr.x (mgr.24118) 1701 : cluster [DBG] pgmap v2505: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.188 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:05 smithi166 bash[9446]: cluster 2020-09-08T16:24:04.393624+0000 mgr.x (mgr.24118) 1702 : cluster [DBG] pgmap v2506: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.188 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:05 smithi166 bash[9446]: cluster 2020-09-08T16:24:04.508302+0000 osd.11 (osd.11) 160 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.188 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:05 smithi166 bash[9446]: cluster 2020-09-08T16:24:04.508429+0000 osd.11 (osd.11) 161 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.188 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:05 smithi166 bash[9446]: cluster 2020-09-08T16:24:04.561230+0000 mon.a (mon.0) 2513 : cluster [DBG] osdmap e889: 12 total, 12 up, 12 in 2020-09-08T16:27:02.188 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:07 smithi166 bash[9446]: cluster 2020-09-08T16:24:06.394289+0000 mgr.x (mgr.24118) 1703 : cluster [DBG] pgmap v2508: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.189 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:07 smithi166 bash[9446]: cluster 2020-09-08T16:24:07.389831+0000 osd.4 (osd.4) 105 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.189 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:07 smithi166 bash[9446]: cluster 2020-09-08T16:24:07.389953+0000 osd.4 (osd.4) 106 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.189 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:08 smithi166 bash[9446]: cluster 2020-09-08T16:24:07.569285+0000 mon.a (mon.0) 2514 : cluster [DBG] osdmap e890: 12 total, 12 up, 12 in 2020-09-08T16:27:02.189 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:09 smithi166 bash[9446]: cluster 2020-09-08T16:24:08.395016+0000 mgr.x (mgr.24118) 1704 : cluster [DBG] pgmap v2510: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.190 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:11 smithi166 bash[9446]: cluster 2020-09-08T16:24:10.395654+0000 mgr.x (mgr.24118) 1705 : cluster [DBG] pgmap v2511: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.190 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:13 smithi166 bash[9446]: cluster 2020-09-08T16:24:12.396433+0000 mgr.x (mgr.24118) 1706 : cluster [DBG] pgmap v2512: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.190 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:14 smithi166 bash[9446]: audit 2020-09-08T16:24:13.474986+0000 mon.b (mon.1) 138 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.190 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:14 smithi166 bash[9446]: audit 2020-09-08T16:24:13.477045+0000 mon.a (mon.0) 2515 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.190 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:14 smithi166 bash[9446]: audit 2020-09-08T16:24:13.482442+0000 mon.a (mon.0) 2516 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.191 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:15 smithi166 bash[9446]: cluster 2020-09-08T16:24:14.397293+0000 mgr.x (mgr.24118) 1707 : cluster [DBG] pgmap v2513: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.191 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:17 smithi166 bash[9446]: cluster 2020-09-08T16:24:16.398030+0000 mgr.x (mgr.24118) 1708 : cluster [DBG] pgmap v2514: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.191 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:18 smithi166 bash[9446]: cluster 2020-09-08T16:24:17.588032+0000 osd.9 (osd.9) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.191 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:18 smithi166 bash[9446]: cluster 2020-09-08T16:24:17.588165+0000 osd.9 (osd.9) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.191 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:19 smithi166 bash[9446]: cluster 2020-09-08T16:24:18.398882+0000 mgr.x (mgr.24118) 1709 : cluster [DBG] pgmap v2515: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.192 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:19 smithi166 bash[9446]: cluster 2020-09-08T16:24:18.542472+0000 mon.a (mon.0) 2517 : cluster [DBG] osdmap e891: 12 total, 12 up, 12 in 2020-09-08T16:27:02.192 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:19 smithi166 bash[9446]: cluster 2020-09-08T16:24:18.764818+0000 osd.2 (osd.2) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.192 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:19 smithi166 bash[9446]: cluster 2020-09-08T16:24:18.764929+0000 osd.2 (osd.2) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.192 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:20 smithi166 bash[9446]: cluster 2020-09-08T16:24:19.546699+0000 mon.a (mon.0) 2518 : cluster [DBG] osdmap e892: 12 total, 12 up, 12 in 2020-09-08T16:27:02.192 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:21 smithi166 bash[9446]: cluster 2020-09-08T16:24:20.399622+0000 mgr.x (mgr.24118) 1710 : cluster [DBG] pgmap v2518: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.193 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:23 smithi166 bash[9446]: cluster 2020-09-08T16:24:22.400285+0000 mgr.x (mgr.24118) 1711 : cluster [DBG] pgmap v2519: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.193 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:25 smithi166 bash[9446]: cluster 2020-09-08T16:24:24.400983+0000 mgr.x (mgr.24118) 1712 : cluster [DBG] pgmap v2520: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.193 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:27 smithi166 bash[9446]: cluster 2020-09-08T16:24:26.401749+0000 mgr.x (mgr.24118) 1713 : cluster [DBG] pgmap v2521: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.193 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:27 smithi166 bash[9446]: cluster 2020-09-08T16:24:27.099750+0000 osd.0 (osd.0) 156 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.193 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:27 smithi166 bash[9446]: cluster 2020-09-08T16:24:27.099880+0000 osd.0 (osd.0) 157 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.194 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:28 smithi166 bash[9446]: cluster 2020-09-08T16:24:27.315084+0000 osd.7 (osd.7) 196 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.194 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:28 smithi166 bash[9446]: cluster 2020-09-08T16:24:27.315209+0000 osd.7 (osd.7) 197 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.194 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:28 smithi166 bash[9446]: cluster 2020-09-08T16:24:27.581255+0000 mon.a (mon.0) 2519 : cluster [DBG] osdmap e893: 12 total, 12 up, 12 in 2020-09-08T16:27:02.194 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:29 smithi166 bash[9446]: cluster 2020-09-08T16:24:28.402497+0000 mgr.x (mgr.24118) 1714 : cluster [DBG] pgmap v2523: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.195 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:31 smithi166 bash[9446]: cluster 2020-09-08T16:24:30.403186+0000 mgr.x (mgr.24118) 1715 : cluster [DBG] pgmap v2524: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.195 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:33 smithi166 bash[9446]: cluster 2020-09-08T16:24:32.403863+0000 mgr.x (mgr.24118) 1716 : cluster [DBG] pgmap v2525: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.195 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:35 smithi166 bash[9446]: cluster 2020-09-08T16:24:34.404578+0000 mgr.x (mgr.24118) 1717 : cluster [DBG] pgmap v2526: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.195 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:37 smithi166 bash[9446]: cluster 2020-09-08T16:24:36.405231+0000 mgr.x (mgr.24118) 1718 : cluster [DBG] pgmap v2527: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.195 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:38 smithi166 bash[9446]: cluster 2020-09-08T16:24:37.495138+0000 osd.5 (osd.5) 162 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.196 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:38 smithi166 bash[9446]: cluster 2020-09-08T16:24:37.495296+0000 osd.5 (osd.5) 163 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.196 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:38 smithi166 bash[9446]: cluster 2020-09-08T16:24:37.909834+0000 osd.3 (osd.3) 165 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.196 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:38 smithi166 bash[9446]: cluster 2020-09-08T16:24:37.909972+0000 osd.3 (osd.3) 166 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.196 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:39 smithi166 bash[9446]: cluster 2020-09-08T16:24:38.405935+0000 mgr.x (mgr.24118) 1719 : cluster [DBG] pgmap v2528: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.196 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:39 smithi166 bash[9446]: cluster 2020-09-08T16:24:38.492275+0000 mon.a (mon.0) 2520 : cluster [DBG] osdmap e894: 12 total, 12 up, 12 in 2020-09-08T16:27:02.197 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:41 smithi166 bash[9446]: cluster 2020-09-08T16:24:40.353237+0000 osd.10 (osd.10) 141 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.197 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:41 smithi166 bash[9446]: cluster 2020-09-08T16:24:40.353357+0000 osd.10 (osd.10) 142 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.197 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:41 smithi166 bash[9446]: cluster 2020-09-08T16:24:40.406644+0000 mgr.x (mgr.24118) 1720 : cluster [DBG] pgmap v2530: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.197 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:42 smithi166 bash[9446]: cluster 2020-09-08T16:24:41.325942+0000 mon.a (mon.0) 2521 : cluster [DBG] osdmap e895: 12 total, 12 up, 12 in 2020-09-08T16:27:02.197 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:43 smithi166 bash[9446]: cluster 2020-09-08T16:24:42.407347+0000 mgr.x (mgr.24118) 1721 : cluster [DBG] pgmap v2532: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.198 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:45 smithi166 bash[9446]: cluster 2020-09-08T16:24:44.408070+0000 mgr.x (mgr.24118) 1722 : cluster [DBG] pgmap v2533: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.198 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:47 smithi166 bash[9446]: cluster 2020-09-08T16:24:46.408737+0000 mgr.x (mgr.24118) 1723 : cluster [DBG] pgmap v2534: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.198 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:49 smithi166 bash[9446]: cluster 2020-09-08T16:24:47.670412+0000 osd.8 (osd.8) 136 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.198 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:49 smithi166 bash[9446]: cluster 2020-09-08T16:24:47.670538+0000 osd.8 (osd.8) 137 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.198 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:49 smithi166 bash[9446]: cluster 2020-09-08T16:24:48.409508+0000 mgr.x (mgr.24118) 1724 : cluster [DBG] pgmap v2535: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.199 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:49 smithi166 bash[9446]: cluster 2020-09-08T16:24:48.486000+0000 mon.a (mon.0) 2522 : cluster [DBG] osdmap e896: 12 total, 12 up, 12 in 2020-09-08T16:27:02.199 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:51 smithi166 bash[9446]: cluster 2020-09-08T16:24:50.410325+0000 mgr.x (mgr.24118) 1725 : cluster [DBG] pgmap v2537: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.199 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:53 smithi166 bash[9446]: cluster 2020-09-08T16:24:52.411043+0000 mgr.x (mgr.24118) 1726 : cluster [DBG] pgmap v2538: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.199 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:55 smithi166 bash[9446]: cluster 2020-09-08T16:24:54.411761+0000 mgr.x (mgr.24118) 1727 : cluster [DBG] pgmap v2539: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.199 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:55 smithi166 bash[9446]: cluster 2020-09-08T16:24:54.547404+0000 osd.1 (osd.1) 182 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.200 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:55 smithi166 bash[9446]: cluster 2020-09-08T16:24:54.547537+0000 osd.1 (osd.1) 183 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.200 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:56 smithi166 bash[9446]: cluster 2020-09-08T16:24:55.526084+0000 mon.a (mon.0) 2523 : cluster [DBG] osdmap e897: 12 total, 12 up, 12 in 2020-09-08T16:27:02.200 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:57 smithi166 bash[9446]: cluster 2020-09-08T16:24:56.412405+0000 mgr.x (mgr.24118) 1728 : cluster [DBG] pgmap v2541: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.200 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:24:59 smithi166 bash[9446]: cluster 2020-09-08T16:24:58.413152+0000 mgr.x (mgr.24118) 1729 : cluster [DBG] pgmap v2542: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.200 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:01 smithi166 bash[9446]: cluster 2020-09-08T16:25:00.061572+0000 osd.6 (osd.6) 147 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.201 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:01 smithi166 bash[9446]: cluster 2020-09-08T16:25:00.061698+0000 osd.6 (osd.6) 148 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.201 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:01 smithi166 bash[9446]: cluster 2020-09-08T16:25:00.413852+0000 mgr.x (mgr.24118) 1730 : cluster [DBG] pgmap v2543: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.201 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:01 smithi166 bash[9446]: cluster 2020-09-08T16:25:00.548020+0000 mon.a (mon.0) 2524 : cluster [DBG] osdmap e898: 12 total, 12 up, 12 in 2020-09-08T16:27:02.201 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:03 smithi166 bash[9446]: cluster 2020-09-08T16:25:02.414568+0000 mgr.x (mgr.24118) 1731 : cluster [DBG] pgmap v2545: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.201 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:05 smithi166 bash[9446]: cluster 2020-09-08T16:25:04.415243+0000 mgr.x (mgr.24118) 1732 : cluster [DBG] pgmap v2546: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.202 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:05 smithi166 bash[9446]: cluster 2020-09-08T16:25:04.648267+0000 osd.11 (osd.11) 162 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.202 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:05 smithi166 bash[9446]: cluster 2020-09-08T16:25:04.648410+0000 osd.11 (osd.11) 163 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.202 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:06 smithi166 bash[9446]: cluster 2020-09-08T16:25:05.562272+0000 mon.a (mon.0) 2525 : cluster [DBG] osdmap e899: 12 total, 12 up, 12 in 2020-09-08T16:27:02.202 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:07 smithi166 bash[9446]: cluster 2020-09-08T16:25:06.415870+0000 mgr.x (mgr.24118) 1733 : cluster [DBG] pgmap v2548: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.202 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:07 smithi166 bash[9446]: cluster 2020-09-08T16:25:07.405400+0000 osd.4 (osd.4) 107 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.203 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:07 smithi166 bash[9446]: cluster 2020-09-08T16:25:07.405548+0000 osd.4 (osd.4) 108 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.203 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:08 smithi166 bash[9446]: cluster 2020-09-08T16:25:07.593722+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e900: 12 total, 12 up, 12 in 2020-09-08T16:27:02.203 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:09 smithi166 bash[9446]: cluster 2020-09-08T16:25:08.416541+0000 mgr.x (mgr.24118) 1734 : cluster [DBG] pgmap v2550: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.203 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:11 smithi166 bash[9446]: cluster 2020-09-08T16:25:10.417322+0000 mgr.x (mgr.24118) 1735 : cluster [DBG] pgmap v2551: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.203 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:13 smithi166 bash[9446]: cluster 2020-09-08T16:25:12.418166+0000 mgr.x (mgr.24118) 1736 : cluster [DBG] pgmap v2552: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.204 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:15 smithi166 bash[9446]: cluster 2020-09-08T16:25:14.418937+0000 mgr.x (mgr.24118) 1737 : cluster [DBG] pgmap v2553: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.204 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:17 smithi166 bash[9446]: cluster 2020-09-08T16:25:16.419618+0000 mgr.x (mgr.24118) 1738 : cluster [DBG] pgmap v2554: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.204 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:19 smithi166 bash[9446]: cluster 2020-09-08T16:25:18.420356+0000 mgr.x (mgr.24118) 1739 : cluster [DBG] pgmap v2555: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.204 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:19 smithi166 bash[9446]: cluster 2020-09-08T16:25:18.490436+0000 osd.9 (osd.9) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.204 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:19 smithi166 bash[9446]: cluster 2020-09-08T16:25:18.490614+0000 osd.9 (osd.9) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.205 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:20 smithi166 bash[9446]: cluster 2020-09-08T16:25:19.498676+0000 mon.a (mon.0) 2527 : cluster [DBG] osdmap e901: 12 total, 12 up, 12 in 2020-09-08T16:27:02.205 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:20 smithi166 bash[9446]: cluster 2020-09-08T16:25:19.710713+0000 osd.2 (osd.2) 142 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.205 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:20 smithi166 bash[9446]: cluster 2020-09-08T16:25:19.710828+0000 osd.2 (osd.2) 143 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.205 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:21 smithi166 bash[9446]: cluster 2020-09-08T16:25:20.421020+0000 mgr.x (mgr.24118) 1740 : cluster [DBG] pgmap v2557: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.206 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:21 smithi166 bash[9446]: cluster 2020-09-08T16:25:20.510963+0000 mon.a (mon.0) 2528 : cluster [DBG] osdmap e902: 12 total, 12 up, 12 in 2020-09-08T16:27:02.206 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:23 smithi166 bash[9446]: cluster 2020-09-08T16:25:22.421712+0000 mgr.x (mgr.24118) 1741 : cluster [DBG] pgmap v2559: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.207 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:25 smithi166 bash[9446]: cluster 2020-09-08T16:25:24.422438+0000 mgr.x (mgr.24118) 1742 : cluster [DBG] pgmap v2560: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.207 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:27 smithi166 bash[9446]: cluster 2020-09-08T16:25:26.423122+0000 mgr.x (mgr.24118) 1743 : cluster [DBG] pgmap v2561: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.207 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:28 smithi166 bash[9446]: cluster 2020-09-08T16:25:27.293029+0000 osd.0 (osd.0) 158 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.207 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:28 smithi166 bash[9446]: cluster 2020-09-08T16:25:27.293156+0000 osd.0 (osd.0) 159 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.207 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:28 smithi166 bash[9446]: cluster 2020-09-08T16:25:27.543088+0000 mon.a (mon.0) 2529 : cluster [DBG] osdmap e903: 12 total, 12 up, 12 in 2020-09-08T16:27:02.208 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:29 smithi166 bash[9446]: cluster 2020-09-08T16:25:27.910337+0000 osd.7 (osd.7) 198 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.208 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:29 smithi166 bash[9446]: cluster 2020-09-08T16:25:27.910470+0000 osd.7 (osd.7) 199 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.208 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:29 smithi166 bash[9446]: cluster 2020-09-08T16:25:28.423802+0000 mgr.x (mgr.24118) 1744 : cluster [DBG] pgmap v2563: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.208 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:29 smithi166 bash[9446]: cluster 2020-09-08T16:25:28.562409+0000 mon.a (mon.0) 2530 : cluster [DBG] osdmap e904: 12 total, 12 up, 12 in 2020-09-08T16:27:02.208 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:31 smithi166 bash[9446]: cluster 2020-09-08T16:25:30.424475+0000 mgr.x (mgr.24118) 1745 : cluster [DBG] pgmap v2565: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.209 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:33 smithi166 bash[9446]: cluster 2020-09-08T16:25:32.425199+0000 mgr.x (mgr.24118) 1746 : cluster [DBG] pgmap v2566: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.209 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:35 smithi166 bash[9446]: cluster 2020-09-08T16:25:34.425909+0000 mgr.x (mgr.24118) 1747 : cluster [DBG] pgmap v2567: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.209 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:37 smithi166 bash[9446]: cluster 2020-09-08T16:25:36.426615+0000 mgr.x (mgr.24118) 1748 : cluster [DBG] pgmap v2568: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.209 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:38 smithi166 bash[9446]: cluster 2020-09-08T16:25:37.598505+0000 osd.5 (osd.5) 164 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.209 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:38 smithi166 bash[9446]: cluster 2020-09-08T16:25:37.598661+0000 osd.5 (osd.5) 165 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.210 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:39 smithi166 bash[9446]: cluster 2020-09-08T16:25:38.427331+0000 mgr.x (mgr.24118) 1749 : cluster [DBG] pgmap v2569: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.210 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:39 smithi166 bash[9446]: cluster 2020-09-08T16:25:38.513211+0000 mon.a (mon.0) 2531 : cluster [DBG] osdmap e905: 12 total, 12 up, 12 in 2020-09-08T16:27:02.210 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:39 smithi166 bash[9446]: cluster 2020-09-08T16:25:38.764276+0000 osd.3 (osd.3) 167 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.210 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:39 smithi166 bash[9446]: cluster 2020-09-08T16:25:38.764409+0000 osd.3 (osd.3) 168 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.210 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:40 smithi166 bash[9446]: cluster 2020-09-08T16:25:39.515081+0000 mon.a (mon.0) 2532 : cluster [DBG] osdmap e906: 12 total, 12 up, 12 in 2020-09-08T16:27:02.211 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:41 smithi166 bash[9446]: cluster 2020-09-08T16:25:40.428011+0000 mgr.x (mgr.24118) 1750 : cluster [DBG] pgmap v2572: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.211 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:41 smithi166 bash[9446]: cluster 2020-09-08T16:25:40.468260+0000 osd.10 (osd.10) 143 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.211 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:41 smithi166 bash[9446]: cluster 2020-09-08T16:25:40.468351+0000 osd.10 (osd.10) 144 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.211 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:41 smithi166 bash[9446]: cluster 2020-09-08T16:25:41.349574+0000 mon.a (mon.0) 2533 : cluster [DBG] osdmap e907: 12 total, 12 up, 12 in 2020-09-08T16:27:02.211 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:43 smithi166 bash[9446]: cluster 2020-09-08T16:25:42.428652+0000 mgr.x (mgr.24118) 1751 : cluster [DBG] pgmap v2574: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.212 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:45 smithi166 bash[9446]: cluster 2020-09-08T16:25:44.429449+0000 mgr.x (mgr.24118) 1752 : cluster [DBG] pgmap v2575: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.212 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:47 smithi166 bash[9446]: cluster 2020-09-08T16:25:46.430281+0000 mgr.x (mgr.24118) 1753 : cluster [DBG] pgmap v2576: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.212 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:49 smithi166 bash[9446]: cluster 2020-09-08T16:25:48.640765+0000 osd.8 (osd.8) 138 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.212 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:49 smithi166 bash[9446]: cluster 2020-09-08T16:25:48.640899+0000 osd.8 (osd.8) 139 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.212 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:49 smithi166 bash[9446]: audit 2020-09-08T16:25:48.993840+0000 mon.a (mon.0) 2534 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.402933\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403132\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403786\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404332\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404856\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.405550\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:25:48.990918\"}"}]: dispatch 2020-09-08T16:27:02.213 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:49 smithi166 bash[9446]: audit 2020-09-08T16:25:49.000833+0000 mon.a (mon.0) 2535 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.402933\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403132\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403786\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404332\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404856\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.405550\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:25:48.990918\"}"}]': finished 2020-09-08T16:27:02.213 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:49 smithi166 bash[9446]: audit 2020-09-08T16:25:49.001101+0000 mon.a (mon.0) 2536 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685123\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685289\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685873\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686336\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686892\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.687437\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:25:48.995696\"}"}]: dispatch 2020-09-08T16:27:02.213 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:49 smithi166 bash[9446]: audit 2020-09-08T16:25:49.005877+0000 mon.a (mon.0) 2537 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685123\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685289\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685873\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686336\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686892\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.687437\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:25:48.995696\"}"}]': finished 2020-09-08T16:27:02.214 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:49 smithi166 bash[9446]: audit 2020-09-08T16:25:49.012081+0000 mon.a (mon.0) 2538 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698864\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T16:14:49.698956\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699006\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699331\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699539\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699740\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.700002\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:25:48.998320\"}"}]: dispatch 2020-09-08T16:27:02.214 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:49 smithi166 bash[9446]: audit 2020-09-08T16:25:49.012937+0000 mon.a (mon.0) 2539 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:27:02.214 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:49 smithi166 bash[9446]: audit 2020-09-08T16:25:49.016722+0000 mon.a (mon.0) 2540 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698864\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T16:14:49.698956\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699006\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699331\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699539\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699740\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.700002\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:25:48.998320\"}"}]': finished 2020-09-08T16:27:02.215 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:49 smithi166 bash[9446]: audit 2020-09-08T16:25:49.017007+0000 mon.a (mon.0) 2541 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:27:02.215 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:49 smithi166 bash[9446]: audit 2020-09-08T16:25:49.017561+0000 mon.a (mon.0) 2542 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:27:02.215 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:50 smithi166 bash[9446]: cluster 2020-09-08T16:25:49.619281+0000 mon.a (mon.0) 2543 : cluster [DBG] osdmap e908: 12 total, 12 up, 12 in 2020-09-08T16:27:02.215 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:50 smithi166 bash[9446]: audit 2020-09-08T16:25:49.860073+0000 mon.a (mon.0) 2544 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.853655\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.853925\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.854774\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.855418\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.856044\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:25:49.856810\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:25:48.995696\"}"}]: dispatch 2020-09-08T16:27:02.216 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:50 smithi166 bash[9446]: audit 2020-09-08T16:25:49.866266+0000 mon.a (mon.0) 2545 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.853655\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.853925\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.854774\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.855418\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.856044\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:25:49.856810\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:25:48.995696\"}"}]': finished 2020-09-08T16:27:02.216 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:50 smithi166 bash[9446]: audit 2020-09-08T16:25:49.969344+0000 mon.a (mon.0) 2546 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.963581\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.963784\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.963914\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.964661\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.965214\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.965750\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:25:49.966460\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:25:48.998320\"}"}]: dispatch 2020-09-08T16:27:02.216 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:50 smithi166 bash[9446]: audit 2020-09-08T16:25:49.975208+0000 mon.a (mon.0) 2547 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.963581\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.963784\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.963914\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.964661\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.965214\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.965750\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:25:49.966460\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:25:48.998320\"}"}]': finished 2020-09-08T16:27:02.217 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:50 smithi166 bash[9446]: audit 2020-09-08T16:25:50.198306+0000 mon.a (mon.0) 2548 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.192547\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.192731\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.193342\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.193892\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.194470\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:25:50.195154\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:25:48.990918\"}"}]: dispatch 2020-09-08T16:27:02.217 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:50 smithi166 bash[9446]: audit 2020-09-08T16:25:50.204507+0000 mon.a (mon.0) 2549 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.192547\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.192731\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.193342\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.193892\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.194470\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:25:50.195154\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:25:48.990918\"}"}]': finished 2020-09-08T16:27:02.217 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:50 smithi166 bash[9446]: audit 2020-09-08T16:25:50.206948+0000 mon.a (mon.0) 2550 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.218 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:50 smithi166 bash[9446]: audit 2020-09-08T16:25:50.212351+0000 mon.a (mon.0) 2551 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.218 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:51 smithi166 bash[9446]: cluster 2020-09-08T16:25:50.431687+0000 mgr.x (mgr.24118) 1755 : cluster [DBG] pgmap v2579: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.218 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:53 smithi166 bash[9446]: cluster 2020-09-08T16:25:52.432399+0000 mgr.x (mgr.24118) 1756 : cluster [DBG] pgmap v2580: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.218 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:55 smithi166 bash[9446]: cluster 2020-09-08T16:25:54.433250+0000 mgr.x (mgr.24118) 1757 : cluster [DBG] pgmap v2581: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.218 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:55 smithi166 bash[9446]: cluster 2020-09-08T16:25:54.862483+0000 osd.1 (osd.1) 184 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.219 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:55 smithi166 bash[9446]: cluster 2020-09-08T16:25:54.862616+0000 osd.1 (osd.1) 185 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.219 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:56 smithi166 bash[9446]: cluster 2020-09-08T16:25:55.638400+0000 mon.a (mon.0) 2552 : cluster [DBG] osdmap e909: 12 total, 12 up, 12 in 2020-09-08T16:27:02.219 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:57 smithi166 bash[9446]: cluster 2020-09-08T16:25:56.433899+0000 mgr.x (mgr.24118) 1758 : cluster [DBG] pgmap v2583: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.219 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:25:59 smithi166 bash[9446]: cluster 2020-09-08T16:25:58.434653+0000 mgr.x (mgr.24118) 1759 : cluster [DBG] pgmap v2584: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.219 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:01 smithi166 bash[9446]: cluster 2020-09-08T16:26:00.224874+0000 osd.6 (osd.6) 149 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.220 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:01 smithi166 bash[9446]: cluster 2020-09-08T16:26:00.225015+0000 osd.6 (osd.6) 150 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.220 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:01 smithi166 bash[9446]: cluster 2020-09-08T16:26:00.435276+0000 mgr.x (mgr.24118) 1760 : cluster [DBG] pgmap v2585: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.220 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:01 smithi166 bash[9446]: cluster 2020-09-08T16:26:00.670003+0000 mon.a (mon.0) 2553 : cluster [DBG] osdmap e910: 12 total, 12 up, 12 in 2020-09-08T16:27:02.220 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:03 smithi166 bash[9446]: cluster 2020-09-08T16:26:02.435956+0000 mgr.x (mgr.24118) 1761 : cluster [DBG] pgmap v2587: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.220 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:05 smithi166 bash[9446]: cluster 2020-09-08T16:26:04.436653+0000 mgr.x (mgr.24118) 1762 : cluster [DBG] pgmap v2588: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.221 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:06 smithi166 bash[9446]: cluster 2020-09-08T16:26:05.395848+0000 osd.11 (osd.11) 164 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.221 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:06 smithi166 bash[9446]: cluster 2020-09-08T16:26:05.395976+0000 osd.11 (osd.11) 165 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.221 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:06 smithi166 bash[9446]: cluster 2020-09-08T16:26:06.345739+0000 mon.a (mon.0) 2554 : cluster [DBG] osdmap e911: 12 total, 12 up, 12 in 2020-09-08T16:27:02.221 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:07 smithi166 bash[9446]: cluster 2020-09-08T16:26:06.437157+0000 mgr.x (mgr.24118) 1763 : cluster [DBG] pgmap v2590: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.221 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:08 smithi166 bash[9446]: cluster 2020-09-08T16:26:08.234288+0000 osd.4 (osd.4) 109 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.222 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:08 smithi166 bash[9446]: cluster 2020-09-08T16:26:08.234451+0000 osd.4 (osd.4) 110 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.222 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:09 smithi166 bash[9446]: cluster 2020-09-08T16:26:08.711456+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e912: 12 total, 12 up, 12 in 2020-09-08T16:27:02.222 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:09 smithi166 bash[9446]: audit 2020-09-08T16:26:08.714372+0000 mon.a (mon.0) 2556 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.223 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:09 smithi166 bash[9446]: audit 2020-09-08T16:26:08.719180+0000 mon.a (mon.0) 2557 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.223 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:11 smithi166 bash[9446]: cluster 2020-09-08T16:26:10.438346+0000 mgr.x (mgr.24118) 1765 : cluster [DBG] pgmap v2593: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.223 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:13 smithi166 bash[9446]: cluster 2020-09-08T16:26:12.439023+0000 mgr.x (mgr.24118) 1766 : cluster [DBG] pgmap v2594: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.223 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:15 smithi166 bash[9446]: cluster 2020-09-08T16:26:14.439803+0000 mgr.x (mgr.24118) 1767 : cluster [DBG] pgmap v2595: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.223 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:17 smithi166 bash[9446]: cluster 2020-09-08T16:26:16.440416+0000 mgr.x (mgr.24118) 1768 : cluster [DBG] pgmap v2596: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.224 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:19 smithi166 bash[9446]: cluster 2020-09-08T16:26:18.441118+0000 mgr.x (mgr.24118) 1769 : cluster [DBG] pgmap v2597: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.224 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:19 smithi166 bash[9446]: cluster 2020-09-08T16:26:19.009384+0000 osd.9 (osd.9) 148 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.224 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:19 smithi166 bash[9446]: cluster 2020-09-08T16:26:19.009503+0000 osd.9 (osd.9) 149 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.224 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:20 smithi166 bash[9446]: cluster 2020-09-08T16:26:19.521611+0000 mon.a (mon.0) 2558 : cluster [DBG] osdmap e913: 12 total, 12 up, 12 in 2020-09-08T16:27:02.224 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:20 smithi166 bash[9446]: cluster 2020-09-08T16:26:20.098181+0000 osd.2 (osd.2) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.225 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:20 smithi166 bash[9446]: cluster 2020-09-08T16:26:20.098304+0000 osd.2 (osd.2) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.225 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:21 smithi166 bash[9446]: cluster 2020-09-08T16:26:20.441781+0000 mgr.x (mgr.24118) 1770 : cluster [DBG] pgmap v2599: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.225 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:21 smithi166 bash[9446]: cluster 2020-09-08T16:26:20.545496+0000 mon.a (mon.0) 2559 : cluster [DBG] osdmap e914: 12 total, 12 up, 12 in 2020-09-08T16:27:02.225 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:23 smithi166 bash[9446]: cluster 2020-09-08T16:26:22.442522+0000 mgr.x (mgr.24118) 1771 : cluster [DBG] pgmap v2601: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.225 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:25 smithi166 bash[9446]: cluster 2020-09-08T16:26:24.443182+0000 mgr.x (mgr.24118) 1772 : cluster [DBG] pgmap v2602: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.226 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:27 smithi166 bash[9446]: cluster 2020-09-08T16:26:26.443879+0000 mgr.x (mgr.24118) 1773 : cluster [DBG] pgmap v2603: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.226 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:28 smithi166 bash[9446]: cluster 2020-09-08T16:26:28.098777+0000 osd.0 (osd.0) 160 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.226 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:28 smithi166 bash[9446]: cluster 2020-09-08T16:26:28.098921+0000 osd.0 (osd.0) 161 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.226 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:29 smithi166 bash[9446]: cluster 2020-09-08T16:26:28.444593+0000 mgr.x (mgr.24118) 1774 : cluster [DBG] pgmap v2604: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.227 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:29 smithi166 bash[9446]: cluster 2020-09-08T16:26:28.579344+0000 mon.a (mon.0) 2560 : cluster [DBG] osdmap e915: 12 total, 12 up, 12 in 2020-09-08T16:27:02.227 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:30 smithi166 bash[9446]: cluster 2020-09-08T16:26:28.796301+0000 osd.7 (osd.7) 200 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.227 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:30 smithi166 bash[9446]: cluster 2020-09-08T16:26:28.796427+0000 osd.7 (osd.7) 201 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.227 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:30 smithi166 bash[9446]: cluster 2020-09-08T16:26:29.592732+0000 mon.a (mon.0) 2561 : cluster [DBG] osdmap e916: 12 total, 12 up, 12 in 2020-09-08T16:27:02.227 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:31 smithi166 bash[9446]: cluster 2020-09-08T16:26:30.445361+0000 mgr.x (mgr.24118) 1775 : cluster [DBG] pgmap v2607: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.228 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:33 smithi166 bash[9446]: cluster 2020-09-08T16:26:32.446087+0000 mgr.x (mgr.24118) 1776 : cluster [DBG] pgmap v2608: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.228 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:35 smithi166 bash[9446]: cluster 2020-09-08T16:26:34.446753+0000 mgr.x (mgr.24118) 1777 : cluster [DBG] pgmap v2609: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.228 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:37 smithi166 bash[9446]: cluster 2020-09-08T16:26:36.447416+0000 mgr.x (mgr.24118) 1778 : cluster [DBG] pgmap v2610: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.228 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:38 smithi166 bash[9446]: cluster 2020-09-08T16:26:37.639096+0000 osd.5 (osd.5) 166 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.229 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:38 smithi166 bash[9446]: cluster 2020-09-08T16:26:37.639237+0000 osd.5 (osd.5) 167 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.229 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:39 smithi166 bash[9446]: cluster 2020-09-08T16:26:38.448127+0000 mgr.x (mgr.24118) 1779 : cluster [DBG] pgmap v2611: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.229 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:39 smithi166 bash[9446]: cluster 2020-09-08T16:26:38.631152+0000 mon.a (mon.0) 2562 : cluster [DBG] osdmap e917: 12 total, 12 up, 12 in 2020-09-08T16:27:02.229 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:40 smithi166 bash[9446]: cluster 2020-09-08T16:26:39.651090+0000 osd.3 (osd.3) 169 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.229 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:40 smithi166 bash[9446]: cluster 2020-09-08T16:26:39.651219+0000 osd.3 (osd.3) 170 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.230 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:41 smithi166 bash[9446]: cluster 2020-09-08T16:26:40.448778+0000 mgr.x (mgr.24118) 1780 : cluster [DBG] pgmap v2613: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.230 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:41 smithi166 bash[9446]: cluster 2020-09-08T16:26:40.534204+0000 osd.10 (osd.10) 145 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.230 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:41 smithi166 bash[9446]: cluster 2020-09-08T16:26:40.534322+0000 osd.10 (osd.10) 146 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.230 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:41 smithi166 bash[9446]: cluster 2020-09-08T16:26:40.639992+0000 mon.a (mon.0) 2563 : cluster [DBG] osdmap e918: 12 total, 12 up, 12 in 2020-09-08T16:27:02.230 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:43 smithi166 bash[9446]: cluster 2020-09-08T16:26:42.449459+0000 mgr.x (mgr.24118) 1781 : cluster [DBG] pgmap v2615: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.231 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:45 smithi166 bash[9446]: cluster 2020-09-08T16:26:44.450417+0000 mgr.x (mgr.24118) 1782 : cluster [DBG] pgmap v2616: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.231 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:47 smithi166 bash[9446]: cluster 2020-09-08T16:26:46.451131+0000 mgr.x (mgr.24118) 1783 : cluster [DBG] pgmap v2617: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.231 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:49 smithi166 bash[9446]: cluster 2020-09-08T16:26:48.451858+0000 mgr.x (mgr.24118) 1784 : cluster [DBG] pgmap v2618: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.231 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:49 smithi166 bash[9446]: cluster 2020-09-08T16:26:49.534260+0000 osd.8 (osd.8) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.232 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:49 smithi166 bash[9446]: cluster 2020-09-08T16:26:49.534382+0000 osd.8 (osd.8) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.232 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:50 smithi166 bash[9446]: cluster 2020-09-08T16:26:49.681079+0000 mon.a (mon.0) 2564 : cluster [DBG] osdmap e919: 12 total, 12 up, 12 in 2020-09-08T16:27:02.232 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:51 smithi166 bash[9446]: cluster 2020-09-08T16:26:50.452524+0000 mgr.x (mgr.24118) 1785 : cluster [DBG] pgmap v2620: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.232 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:53 smithi166 bash[9446]: cluster 2020-09-08T16:26:52.453264+0000 mgr.x (mgr.24118) 1786 : cluster [DBG] pgmap v2621: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.232 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:55 smithi166 bash[9446]: cluster 2020-09-08T16:26:54.454065+0000 mgr.x (mgr.24118) 1787 : cluster [DBG] pgmap v2622: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.233 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:56 smithi166 bash[9446]: cluster 2020-09-08T16:26:55.777497+0000 osd.1 (osd.1) 186 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.233 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:56 smithi166 bash[9446]: cluster 2020-09-08T16:26:55.777629+0000 osd.1 (osd.1) 187 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.233 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:57 smithi166 bash[9446]: cluster 2020-09-08T16:26:56.454786+0000 mgr.x (mgr.24118) 1788 : cluster [DBG] pgmap v2623: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.233 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:57 smithi166 bash[9446]: cluster 2020-09-08T16:26:56.729079+0000 mon.a (mon.0) 2565 : cluster [DBG] osdmap e920: 12 total, 12 up, 12 in 2020-09-08T16:27:02.233 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:26:59 smithi166 bash[9446]: cluster 2020-09-08T16:26:58.455629+0000 mgr.x (mgr.24118) 1789 : cluster [DBG] pgmap v2625: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.234 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:01 smithi166 bash[9446]: cluster 2020-09-08T16:27:00.443440+0000 osd.6 (osd.6) 151 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.234 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:01 smithi166 bash[9446]: cluster 2020-09-08T16:27:00.443571+0000 osd.6 (osd.6) 152 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.234 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:01 smithi166 bash[9446]: cluster 2020-09-08T16:27:00.456380+0000 mgr.x (mgr.24118) 1790 : cluster [DBG] pgmap v2626: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.235 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Removed 140030 objects 2020-09-08T16:27:02.235 INFO:tasks.radosbench.radosbench.2.smithi166.stdout:Clean up completed and total clean up time :146.436 2020-09-08T16:27:02.235 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:28 smithi055 bash[9296]: audit 2020-09-08T16:13:28.071148+0000 mon.a (mon.0) 2363 : audit [INF] from='client.? v1:172.21.15.104:0/622774130' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:27:02.236 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:29 smithi055 bash[9296]: cluster 2020-09-08T16:13:28.135669+0000 mon.a (mon.0) 2364 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:27:02.236 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:29 smithi055 bash[9296]: audit 2020-09-08T16:13:28.143990+0000 mon.a (mon.0) 2365 : audit [INF] from='client.? v1:172.21.15.104:0/622774130' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:27:02.236 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:29 smithi055 bash[9296]: cluster 2020-09-08T16:13:28.144039+0000 mon.a (mon.0) 2366 : cluster [DBG] osdmap e769: 12 total, 12 up, 12 in 2020-09-08T16:27:02.236 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:29 smithi055 bash[9296]: cluster 2020-09-08T16:13:28.171567+0000 mgr.x (mgr.24118) 1384 : cluster [DBG] pgmap v2069: 17 pgs: 17 active+clean; 208 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.17k op/s 2020-09-08T16:27:02.236 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:31 smithi055 bash[9296]: cluster 2020-09-08T16:13:30.172360+0000 mgr.x (mgr.24118) 1385 : cluster [DBG] pgmap v2070: 17 pgs: 17 active+clean; 64 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.18k op/s 2020-09-08T16:27:02.237 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:32 smithi055 bash[9296]: audit 2020-09-08T16:13:31.118278+0000 mon.a (mon.0) 2367 : audit [DBG] from='client.? v1:172.21.15.104:0/3128233918' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:27:02.237 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:32 smithi055 bash[9296]: cluster 2020-09-08T16:13:31.359446+0000 osd.5 (osd.5) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.237 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:32 smithi055 bash[9296]: cluster 2020-09-08T16:13:31.359582+0000 osd.5 (osd.5) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.238 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:33 smithi055 bash[9296]: cluster 2020-09-08T16:13:32.124175+0000 mon.a (mon.0) 2368 : cluster [DBG] osdmap e770: 12 total, 12 up, 12 in 2020-09-08T16:27:02.238 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:33 smithi055 bash[9296]: cluster 2020-09-08T16:13:32.173034+0000 mgr.x (mgr.24118) 1386 : cluster [DBG] pgmap v2072: 17 pgs: 17 active+clean; 64 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 677 op/s 2020-09-08T16:27:02.238 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:34 smithi055 bash[9296]: cluster 2020-09-08T16:13:33.798526+0000 osd.10 (osd.10) 119 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.238 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:34 smithi055 bash[9296]: cluster 2020-09-08T16:13:33.798648+0000 osd.10 (osd.10) 120 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.238 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:35 smithi055 bash[9296]: cluster 2020-09-08T16:13:34.142489+0000 mon.a (mon.0) 2369 : cluster [DBG] osdmap e771: 12 total, 12 up, 12 in 2020-09-08T16:27:02.239 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:35 smithi055 bash[9296]: cluster 2020-09-08T16:13:34.173538+0000 mgr.x (mgr.24118) 1387 : cluster [DBG] pgmap v2074: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 552 op/s 2020-09-08T16:27:02.239 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:35 smithi055 bash[9296]: cluster 2020-09-08T16:13:34.798777+0000 osd.3 (osd.3) 143 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.239 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:35 smithi055 bash[9296]: cluster 2020-09-08T16:13:34.798895+0000 osd.3 (osd.3) 144 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.239 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:36 smithi055 bash[9296]: cluster 2020-09-08T16:13:35.150774+0000 mon.a (mon.0) 2370 : cluster [DBG] osdmap e772: 12 total, 12 up, 12 in 2020-09-08T16:27:02.239 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:37 smithi055 bash[9296]: cluster 2020-09-08T16:13:36.174104+0000 mgr.x (mgr.24118) 1388 : cluster [DBG] pgmap v2076: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 169 op/s 2020-09-08T16:27:02.240 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:39 smithi055 bash[9296]: cluster 2020-09-08T16:13:38.174794+0000 mgr.x (mgr.24118) 1389 : cluster [DBG] pgmap v2077: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 168 op/s 2020-09-08T16:27:02.240 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:41 smithi055 bash[9296]: cluster 2020-09-08T16:13:40.175536+0000 mgr.x (mgr.24118) 1390 : cluster [DBG] pgmap v2078: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 127 op/s 2020-09-08T16:27:02.240 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:43 smithi055 bash[9296]: cluster 2020-09-08T16:13:42.157827+0000 osd.8 (osd.8) 114 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.240 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:43 smithi055 bash[9296]: cluster 2020-09-08T16:13:42.157956+0000 osd.8 (osd.8) 115 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.240 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:43 smithi055 bash[9296]: cluster 2020-09-08T16:13:42.176237+0000 mgr.x (mgr.24118) 1391 : cluster [DBG] pgmap v2079: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.241 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:43 smithi055 bash[9296]: cluster 2020-09-08T16:13:42.218579+0000 mon.a (mon.0) 2371 : cluster [DBG] osdmap e773: 12 total, 12 up, 12 in 2020-09-08T16:27:02.241 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:45 smithi055 bash[9296]: cluster 2020-09-08T16:13:44.176996+0000 mgr.x (mgr.24118) 1392 : cluster [DBG] pgmap v2081: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.241 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:47 smithi055 bash[9296]: cluster 2020-09-08T16:13:46.177663+0000 mgr.x (mgr.24118) 1393 : cluster [DBG] pgmap v2082: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.241 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:49 smithi055 bash[9296]: cluster 2020-09-08T16:13:48.178391+0000 mgr.x (mgr.24118) 1394 : cluster [DBG] pgmap v2083: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.241 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:50 smithi055 bash[9296]: cluster 2020-09-08T16:13:49.567955+0000 osd.1 (osd.1) 160 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.242 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:50 smithi055 bash[9296]: cluster 2020-09-08T16:13:49.568088+0000 osd.1 (osd.1) 161 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.242 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:51 smithi055 bash[9296]: cluster 2020-09-08T16:13:50.179131+0000 mgr.x (mgr.24118) 1395 : cluster [DBG] pgmap v2084: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.242 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:51 smithi055 bash[9296]: cluster 2020-09-08T16:13:50.244784+0000 mon.a (mon.0) 2372 : cluster [DBG] osdmap e774: 12 total, 12 up, 12 in 2020-09-08T16:27:02.242 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:53 smithi055 bash[9296]: cluster 2020-09-08T16:13:52.179873+0000 mgr.x (mgr.24118) 1396 : cluster [DBG] pgmap v2086: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.242 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:54 smithi055 bash[9296]: cluster 2020-09-08T16:13:53.091527+0000 osd.6 (osd.6) 125 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.243 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:54 smithi055 bash[9296]: cluster 2020-09-08T16:13:53.091672+0000 osd.6 (osd.6) 126 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.243 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:54 smithi055 bash[9296]: cluster 2020-09-08T16:13:53.253232+0000 mon.a (mon.0) 2373 : cluster [DBG] osdmap e775: 12 total, 12 up, 12 in 2020-09-08T16:27:02.243 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:55 smithi055 bash[9296]: cluster 2020-09-08T16:13:54.180617+0000 mgr.x (mgr.24118) 1397 : cluster [DBG] pgmap v2088: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.243 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:57 smithi055 bash[9296]: cluster 2020-09-08T16:13:56.181327+0000 mgr.x (mgr.24118) 1398 : cluster [DBG] pgmap v2089: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.243 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:59 smithi055 bash[9296]: cluster 2020-09-08T16:13:58.182024+0000 mgr.x (mgr.24118) 1399 : cluster [DBG] pgmap v2090: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.244 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:59 smithi055 bash[9296]: cluster 2020-09-08T16:13:58.256754+0000 osd.11 (osd.11) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.244 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:59 smithi055 bash[9296]: cluster 2020-09-08T16:13:58.256880+0000 osd.11 (osd.11) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.244 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:13:59 smithi055 bash[9296]: cluster 2020-09-08T16:13:58.280736+0000 mon.a (mon.0) 2374 : cluster [DBG] osdmap e776: 12 total, 12 up, 12 in 2020-09-08T16:27:02.244 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:01 smithi055 bash[9296]: cluster 2020-09-08T16:14:00.182737+0000 mgr.x (mgr.24118) 1400 : cluster [DBG] pgmap v2092: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.244 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:03 smithi055 bash[9296]: cluster 2020-09-08T16:14:02.183359+0000 mgr.x (mgr.24118) 1401 : cluster [DBG] pgmap v2093: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.245 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:04 smithi055 bash[9296]: cluster 2020-09-08T16:14:02.929875+0000 osd.4 (osd.4) 85 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.245 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:04 smithi055 bash[9296]: cluster 2020-09-08T16:14:02.930001+0000 osd.4 (osd.4) 86 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.245 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:04 smithi055 bash[9296]: cluster 2020-09-08T16:14:03.251582+0000 mon.a (mon.0) 2375 : cluster [DBG] osdmap e777: 12 total, 12 up, 12 in 2020-09-08T16:27:02.245 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:05 smithi055 bash[9296]: cluster 2020-09-08T16:14:04.184018+0000 mgr.x (mgr.24118) 1402 : cluster [DBG] pgmap v2095: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.246 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:07 smithi055 bash[9296]: cluster 2020-09-08T16:14:06.184699+0000 mgr.x (mgr.24118) 1403 : cluster [DBG] pgmap v2096: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.246 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:09 smithi055 bash[9296]: cluster 2020-09-08T16:14:08.185381+0000 mgr.x (mgr.24118) 1404 : cluster [DBG] pgmap v2097: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.246 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:11 smithi055 bash[9296]: cluster 2020-09-08T16:14:10.186089+0000 mgr.x (mgr.24118) 1405 : cluster [DBG] pgmap v2098: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.246 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:12 smithi055 bash[9296]: cluster 2020-09-08T16:14:11.817319+0000 osd.9 (osd.9) 124 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.246 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:12 smithi055 bash[9296]: cluster 2020-09-08T16:14:11.817442+0000 osd.9 (osd.9) 125 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.247 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:13 smithi055 bash[9296]: cluster 2020-09-08T16:14:12.130806+0000 mon.a (mon.0) 2376 : cluster [DBG] osdmap e778: 12 total, 12 up, 12 in 2020-09-08T16:27:02.247 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:13 smithi055 bash[9296]: cluster 2020-09-08T16:14:12.186702+0000 mgr.x (mgr.24118) 1406 : cluster [DBG] pgmap v2100: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.247 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:14 smithi055 bash[9296]: cluster 2020-09-08T16:14:13.940062+0000 osd.2 (osd.2) 120 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.247 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:14 smithi055 bash[9296]: cluster 2020-09-08T16:14:13.940175+0000 osd.2 (osd.2) 121 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.247 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:15 smithi055 bash[9296]: cluster 2020-09-08T16:14:14.152050+0000 mon.a (mon.0) 2377 : cluster [DBG] osdmap e779: 12 total, 12 up, 12 in 2020-09-08T16:27:02.248 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:15 smithi055 bash[9296]: cluster 2020-09-08T16:14:14.187344+0000 mgr.x (mgr.24118) 1407 : cluster [DBG] pgmap v2102: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.248 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:17 smithi055 bash[9296]: cluster 2020-09-08T16:14:16.188022+0000 mgr.x (mgr.24118) 1408 : cluster [DBG] pgmap v2103: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.248 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:19 smithi055 bash[9296]: cluster 2020-09-08T16:14:18.188661+0000 mgr.x (mgr.24118) 1409 : cluster [DBG] pgmap v2104: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.248 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:21 smithi055 bash[9296]: cluster 2020-09-08T16:14:20.189351+0000 mgr.x (mgr.24118) 1410 : cluster [DBG] pgmap v2105: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.248 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:23 smithi055 bash[9296]: cluster 2020-09-08T16:14:22.190000+0000 mgr.x (mgr.24118) 1411 : cluster [DBG] pgmap v2106: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.249 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:23 smithi055 bash[9296]: cluster 2020-09-08T16:14:22.297120+0000 osd.7 (osd.7) 176 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.249 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:23 smithi055 bash[9296]: cluster 2020-09-08T16:14:22.297244+0000 osd.7 (osd.7) 177 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.249 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:23 smithi055 bash[9296]: cluster 2020-09-08T16:14:22.341740+0000 osd.0 (osd.0) 136 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.249 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:23 smithi055 bash[9296]: cluster 2020-09-08T16:14:22.341874+0000 osd.0 (osd.0) 137 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.250 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:24 smithi055 bash[9296]: cluster 2020-09-08T16:14:23.260670+0000 mon.a (mon.0) 2378 : cluster [DBG] osdmap e780: 12 total, 12 up, 12 in 2020-09-08T16:27:02.250 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:25 smithi055 bash[9296]: cluster 2020-09-08T16:14:24.190685+0000 mgr.x (mgr.24118) 1412 : cluster [DBG] pgmap v2108: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.250 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:27 smithi055 bash[9296]: cluster 2020-09-08T16:14:26.191372+0000 mgr.x (mgr.24118) 1413 : cluster [DBG] pgmap v2109: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.250 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:29 smithi055 bash[9296]: cluster 2020-09-08T16:14:28.192165+0000 mgr.x (mgr.24118) 1414 : cluster [DBG] pgmap v2110: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.250 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:31 smithi055 bash[9296]: cluster 2020-09-08T16:14:30.192746+0000 mgr.x (mgr.24118) 1415 : cluster [DBG] pgmap v2111: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.251 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:32 smithi055 bash[9296]: cluster 2020-09-08T16:14:31.379316+0000 osd.5 (osd.5) 142 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.251 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:32 smithi055 bash[9296]: cluster 2020-09-08T16:14:31.379453+0000 osd.5 (osd.5) 143 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.251 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:33 smithi055 bash[9296]: cluster 2020-09-08T16:14:32.130605+0000 mon.a (mon.0) 2379 : cluster [DBG] osdmap e781: 12 total, 12 up, 12 in 2020-09-08T16:27:02.251 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:33 smithi055 bash[9296]: cluster 2020-09-08T16:14:32.193317+0000 mgr.x (mgr.24118) 1416 : cluster [DBG] pgmap v2113: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.251 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:35 smithi055 bash[9296]: cluster 2020-09-08T16:14:34.194070+0000 mgr.x (mgr.24118) 1417 : cluster [DBG] pgmap v2114: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.252 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:35 smithi055 bash[9296]: cluster 2020-09-08T16:14:34.646280+0000 osd.10 (osd.10) 121 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.252 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:35 smithi055 bash[9296]: cluster 2020-09-08T16:14:34.646401+0000 osd.10 (osd.10) 122 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.252 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:36 smithi055 bash[9296]: cluster 2020-09-08T16:14:35.262897+0000 mon.a (mon.0) 2380 : cluster [DBG] osdmap e782: 12 total, 12 up, 12 in 2020-09-08T16:27:02.252 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:36 smithi055 bash[9296]: cluster 2020-09-08T16:14:35.622122+0000 osd.3 (osd.3) 145 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.252 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:36 smithi055 bash[9296]: cluster 2020-09-08T16:14:35.622231+0000 osd.3 (osd.3) 146 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.252 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:37 smithi055 bash[9296]: cluster 2020-09-08T16:14:36.194742+0000 mgr.x (mgr.24118) 1418 : cluster [DBG] pgmap v2116: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.253 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:37 smithi055 bash[9296]: cluster 2020-09-08T16:14:36.272137+0000 mon.a (mon.0) 2381 : cluster [DBG] osdmap e783: 12 total, 12 up, 12 in 2020-09-08T16:27:02.253 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:39 smithi055 bash[9296]: cluster 2020-09-08T16:14:38.195568+0000 mgr.x (mgr.24118) 1419 : cluster [DBG] pgmap v2118: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.253 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:41 smithi055 bash[9296]: cluster 2020-09-08T16:14:40.196276+0000 mgr.x (mgr.24118) 1420 : cluster [DBG] pgmap v2119: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.253 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:43 smithi055 bash[9296]: cluster 2020-09-08T16:14:42.196906+0000 mgr.x (mgr.24118) 1421 : cluster [DBG] pgmap v2120: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.254 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:44 smithi055 bash[9296]: cluster 2020-09-08T16:14:43.080311+0000 osd.8 (osd.8) 116 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.254 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:44 smithi055 bash[9296]: cluster 2020-09-08T16:14:43.080440+0000 osd.8 (osd.8) 117 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.254 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:44 smithi055 bash[9296]: cluster 2020-09-08T16:14:43.259076+0000 mon.a (mon.0) 2382 : cluster [DBG] osdmap e784: 12 total, 12 up, 12 in 2020-09-08T16:27:02.254 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:45 smithi055 bash[9296]: cluster 2020-09-08T16:14:44.197556+0000 mgr.x (mgr.24118) 1422 : cluster [DBG] pgmap v2122: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.255 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:47 smithi055 bash[9296]: cluster 2020-09-08T16:14:46.198236+0000 mgr.x (mgr.24118) 1423 : cluster [DBG] pgmap v2123: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.255 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:48 smithi055 bash[9296]: audit 2020-09-08T16:14:48.568197+0000 mon.b (mon.1) 125 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:14:48.567054\"}"}]: dispatch 2020-09-08T16:27:02.255 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:48 smithi055 bash[9296]: audit 2020-09-08T16:14:48.570330+0000 mon.a (mon.0) 2383 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:14:48.567054\"}"}]: dispatch 2020-09-08T16:27:02.256 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:48 smithi055 bash[9296]: audit 2020-09-08T16:14:48.576218+0000 mon.a (mon.0) 2384 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:14:48.567054\"}"}]': finished 2020-09-08T16:27:02.256 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:48 smithi055 bash[9296]: audit 2020-09-08T16:14:48.576596+0000 mon.b (mon.1) 126 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:27:02.256 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:48 smithi055 bash[9296]: audit 2020-09-08T16:14:48.603681+0000 mon.b (mon.1) 127 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:14:48.602227\"}"}]: dispatch 2020-09-08T16:27:02.257 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:48 smithi055 bash[9296]: audit 2020-09-08T16:14:48.605772+0000 mon.a (mon.0) 2385 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:14:48.602227\"}"}]: dispatch 2020-09-08T16:27:02.257 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:48 smithi055 bash[9296]: audit 2020-09-08T16:14:48.610350+0000 mon.a (mon.0) 2386 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:14:48.602227\"}"}]': finished 2020-09-08T16:27:02.257 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:48 smithi055 bash[9296]: audit 2020-09-08T16:14:48.611699+0000 mon.b (mon.1) 128 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:27:02.258 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:49 smithi055 bash[9296]: audit 2020-09-08T16:14:48.637950+0000 mon.b (mon.1) 129 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:14:48.636423\"}"}]: dispatch 2020-09-08T16:27:02.258 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:49 smithi055 bash[9296]: audit 2020-09-08T16:14:48.639785+0000 mon.a (mon.0) 2387 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:14:48.636423\"}"}]: dispatch 2020-09-08T16:27:02.258 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:49 smithi055 bash[9296]: audit 2020-09-08T16:14:48.644222+0000 mon.a (mon.0) 2388 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:14:48.636423\"}"}]': finished 2020-09-08T16:27:02.259 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:49 smithi055 bash[9296]: audit 2020-09-08T16:14:48.644329+0000 mon.b (mon.1) 130 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:27:02.259 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:49 smithi055 bash[9296]: audit 2020-09-08T16:14:49.407449+0000 mon.b (mon.1) 131 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.402933\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403132\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403786\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404332\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404856\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.405550\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:14:48.602227\"}"}]: dispatch 2020-09-08T16:27:02.259 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:49 smithi055 bash[9296]: audit 2020-09-08T16:14:49.409621+0000 mon.a (mon.0) 2389 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.402933\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403132\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403786\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404332\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404856\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.405550\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:14:48.602227\"}"}]: dispatch 2020-09-08T16:27:02.260 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:49 smithi055 bash[9296]: audit 2020-09-08T16:14:49.414676+0000 mon.a (mon.0) 2390 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.402933\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403132\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403786\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404332\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404856\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.405550\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:14:48.602227\"}"}]': finished 2020-09-08T16:27:02.260 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:50 smithi055 bash[9296]: audit 2020-09-08T16:14:49.688909+0000 mon.b (mon.1) 132 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685123\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685289\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685873\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686336\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686892\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.687437\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:14:48.567054\"}"}]: dispatch 2020-09-08T16:27:02.261 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:50 smithi055 bash[9296]: audit 2020-09-08T16:14:49.690911+0000 mon.a (mon.0) 2391 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685123\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685289\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685873\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686336\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686892\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.687437\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:14:48.567054\"}"}]: dispatch 2020-09-08T16:27:02.261 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:50 smithi055 bash[9296]: audit 2020-09-08T16:14:49.695568+0000 mon.a (mon.0) 2392 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685123\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685289\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685873\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686336\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686892\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.687437\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:14:48.567054\"}"}]': finished 2020-09-08T16:27:02.261 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:50 smithi055 bash[9296]: audit 2020-09-08T16:14:49.701035+0000 mon.b (mon.1) 133 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698864\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698956\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699006\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699331\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699539\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699740\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.700002\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:14:48.636423\"}"}]: dispatch 2020-09-08T16:27:02.262 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:50 smithi055 bash[9296]: audit 2020-09-08T16:14:49.703174+0000 mon.a (mon.0) 2393 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698864\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698956\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699006\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699331\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699539\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699740\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.700002\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:14:48.636423\"}"}]: dispatch 2020-09-08T16:27:02.262 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:50 smithi055 bash[9296]: audit 2020-09-08T16:14:49.709043+0000 mon.a (mon.0) 2394 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698864\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698956\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699006\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699331\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699539\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699740\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.700002\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:14:48.636423\"}"}]': finished 2020-09-08T16:27:02.263 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:50 smithi055 bash[9296]: audit 2020-09-08T16:14:49.710077+0000 mon.b (mon.1) 134 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.263 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:50 smithi055 bash[9296]: audit 2020-09-08T16:14:49.711734+0000 mon.a (mon.0) 2395 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.263 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:50 smithi055 bash[9296]: audit 2020-09-08T16:14:49.716978+0000 mon.a (mon.0) 2396 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.263 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:50 smithi055 bash[9296]: audit 2020-09-08T16:14:49.719847+0000 mon.b (mon.1) 135 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.263 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:50 smithi055 bash[9296]: audit 2020-09-08T16:14:49.721369+0000 mon.a (mon.0) 2397 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.264 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:50 smithi055 bash[9296]: audit 2020-09-08T16:14:49.726604+0000 mon.a (mon.0) 2398 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.264 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:51 smithi055 bash[9296]: cluster 2020-09-08T16:14:50.199752+0000 mgr.x (mgr.24118) 1425 : cluster [DBG] pgmap v2125: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.264 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:51 smithi055 bash[9296]: cluster 2020-09-08T16:14:50.324087+0000 osd.1 (osd.1) 162 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.264 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:51 smithi055 bash[9296]: cluster 2020-09-08T16:14:50.324226+0000 osd.1 (osd.1) 163 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.264 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:51 smithi055 bash[9296]: cluster 2020-09-08T16:14:51.140484+0000 mon.a (mon.0) 2399 : cluster [DBG] osdmap e785: 12 total, 12 up, 12 in 2020-09-08T16:27:02.265 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:53 smithi055 bash[9296]: cluster 2020-09-08T16:14:52.200469+0000 mgr.x (mgr.24118) 1426 : cluster [DBG] pgmap v2127: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.265 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:55 smithi055 bash[9296]: cluster 2020-09-08T16:14:53.964831+0000 osd.6 (osd.6) 127 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.265 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:55 smithi055 bash[9296]: cluster 2020-09-08T16:14:53.964994+0000 osd.6 (osd.6) 128 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.265 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:55 smithi055 bash[9296]: cluster 2020-09-08T16:14:54.201201+0000 mgr.x (mgr.24118) 1427 : cluster [DBG] pgmap v2128: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.265 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:55 smithi055 bash[9296]: cluster 2020-09-08T16:14:54.656765+0000 mon.a (mon.0) 2400 : cluster [DBG] osdmap e786: 12 total, 12 up, 12 in 2020-09-08T16:27:02.266 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:55 smithi055 bash[9296]: audit 2020-09-08T16:14:55.398925+0000 mon.c (mon.2) 97 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.266 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:55 smithi055 bash[9296]: audit 2020-09-08T16:14:55.402185+0000 mon.a (mon.0) 2401 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.266 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:55 smithi055 bash[9296]: audit 2020-09-08T16:14:55.408299+0000 mon.a (mon.0) 2402 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.266 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:55 smithi055 bash[9296]: audit 2020-09-08T16:14:55.411017+0000 mon.c (mon.2) 98 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.266 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:55 smithi055 bash[9296]: audit 2020-09-08T16:14:55.414225+0000 mon.a (mon.0) 2403 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.267 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:55 smithi055 bash[9296]: audit 2020-09-08T16:14:55.421107+0000 mon.a (mon.0) 2404 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.267 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:57 smithi055 bash[9296]: cluster 2020-09-08T16:14:56.201953+0000 mgr.x (mgr.24118) 1428 : cluster [DBG] pgmap v2130: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.267 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:14:59 smithi055 bash[9296]: cluster 2020-09-08T16:14:58.202734+0000 mgr.x (mgr.24118) 1429 : cluster [DBG] pgmap v2131: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.267 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:00 smithi055 bash[9296]: cluster 2020-09-08T16:14:59.199080+0000 osd.11 (osd.11) 142 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.268 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:00 smithi055 bash[9296]: cluster 2020-09-08T16:14:59.199225+0000 osd.11 (osd.11) 143 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.268 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:00 smithi055 bash[9296]: cluster 2020-09-08T16:14:59.671408+0000 mon.a (mon.0) 2405 : cluster [DBG] osdmap e787: 12 total, 12 up, 12 in 2020-09-08T16:27:02.268 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:01 smithi055 bash[9296]: cluster 2020-09-08T16:15:00.203450+0000 mgr.x (mgr.24118) 1430 : cluster [DBG] pgmap v2133: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.268 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:03 smithi055 bash[9296]: cluster 2020-09-08T16:15:02.204091+0000 mgr.x (mgr.24118) 1431 : cluster [DBG] pgmap v2134: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.268 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:03 smithi055 bash[9296]: cluster 2020-09-08T16:15:03.085020+0000 osd.4 (osd.4) 87 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.269 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:03 smithi055 bash[9296]: cluster 2020-09-08T16:15:03.085154+0000 osd.4 (osd.4) 88 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.269 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:04 smithi055 bash[9296]: cluster 2020-09-08T16:15:03.706936+0000 mon.a (mon.0) 2406 : cluster [DBG] osdmap e788: 12 total, 12 up, 12 in 2020-09-08T16:27:02.269 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:05 smithi055 bash[9296]: cluster 2020-09-08T16:15:04.204759+0000 mgr.x (mgr.24118) 1432 : cluster [DBG] pgmap v2136: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.269 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:07 smithi055 bash[9296]: cluster 2020-09-08T16:15:06.205426+0000 mgr.x (mgr.24118) 1433 : cluster [DBG] pgmap v2137: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.269 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:09 smithi055 bash[9296]: cluster 2020-09-08T16:15:08.206133+0000 mgr.x (mgr.24118) 1434 : cluster [DBG] pgmap v2138: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.270 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:11 smithi055 bash[9296]: cluster 2020-09-08T16:15:10.206827+0000 mgr.x (mgr.24118) 1435 : cluster [DBG] pgmap v2139: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.270 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:12 smithi055 bash[9296]: cluster 2020-09-08T16:15:11.845269+0000 osd.9 (osd.9) 126 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.270 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:12 smithi055 bash[9296]: cluster 2020-09-08T16:15:11.845394+0000 osd.9 (osd.9) 127 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.270 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:13 smithi055 bash[9296]: cluster 2020-09-08T16:15:12.141685+0000 mon.a (mon.0) 2407 : cluster [DBG] osdmap e789: 12 total, 12 up, 12 in 2020-09-08T16:27:02.270 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:13 smithi055 bash[9296]: cluster 2020-09-08T16:15:12.207470+0000 mgr.x (mgr.24118) 1436 : cluster [DBG] pgmap v2141: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.271 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:15 smithi055 bash[9296]: cluster 2020-09-08T16:15:14.208181+0000 mgr.x (mgr.24118) 1437 : cluster [DBG] pgmap v2142: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.271 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:15 smithi055 bash[9296]: cluster 2020-09-08T16:15:14.257343+0000 osd.2 (osd.2) 122 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.271 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:15 smithi055 bash[9296]: cluster 2020-09-08T16:15:14.257461+0000 osd.2 (osd.2) 123 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.271 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:15 smithi055 bash[9296]: cluster 2020-09-08T16:15:14.314797+0000 mon.a (mon.0) 2408 : cluster [DBG] osdmap e790: 12 total, 12 up, 12 in 2020-09-08T16:27:02.271 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:17 smithi055 bash[9296]: cluster 2020-09-08T16:15:16.208862+0000 mgr.x (mgr.24118) 1438 : cluster [DBG] pgmap v2144: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.272 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:19 smithi055 bash[9296]: cluster 2020-09-08T16:15:18.209628+0000 mgr.x (mgr.24118) 1439 : cluster [DBG] pgmap v2145: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.272 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:21 smithi055 bash[9296]: cluster 2020-09-08T16:15:20.210367+0000 mgr.x (mgr.24118) 1440 : cluster [DBG] pgmap v2146: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.272 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:23 smithi055 bash[9296]: cluster 2020-09-08T16:15:22.211026+0000 mgr.x (mgr.24118) 1441 : cluster [DBG] pgmap v2147: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.272 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:23 smithi055 bash[9296]: cluster 2020-09-08T16:15:22.554620+0000 osd.7 (osd.7) 178 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.272 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:23 smithi055 bash[9296]: cluster 2020-09-08T16:15:22.554737+0000 osd.7 (osd.7) 179 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.273 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:23 smithi055 bash[9296]: cluster 2020-09-08T16:15:22.661421+0000 osd.0 (osd.0) 138 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.273 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:23 smithi055 bash[9296]: cluster 2020-09-08T16:15:22.661556+0000 osd.0 (osd.0) 139 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.273 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:24 smithi055 bash[9296]: cluster 2020-09-08T16:15:23.285820+0000 mon.a (mon.0) 2409 : cluster [DBG] osdmap e791: 12 total, 12 up, 12 in 2020-09-08T16:27:02.273 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:25 smithi055 bash[9296]: cluster 2020-09-08T16:15:24.211726+0000 mgr.x (mgr.24118) 1442 : cluster [DBG] pgmap v2149: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.273 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:27 smithi055 bash[9296]: cluster 2020-09-08T16:15:26.212433+0000 mgr.x (mgr.24118) 1443 : cluster [DBG] pgmap v2150: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.274 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:29 smithi055 bash[9296]: cluster 2020-09-08T16:15:28.213218+0000 mgr.x (mgr.24118) 1444 : cluster [DBG] pgmap v2151: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.274 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:31 smithi055 bash[9296]: cluster 2020-09-08T16:15:30.213899+0000 mgr.x (mgr.24118) 1445 : cluster [DBG] pgmap v2152: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.274 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:33 smithi055 bash[9296]: cluster 2020-09-08T16:15:32.136858+0000 osd.5 (osd.5) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.274 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:33 smithi055 bash[9296]: cluster 2020-09-08T16:15:32.137014+0000 osd.5 (osd.5) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.274 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:33 smithi055 bash[9296]: cluster 2020-09-08T16:15:32.149772+0000 mon.a (mon.0) 2410 : cluster [DBG] osdmap e792: 12 total, 12 up, 12 in 2020-09-08T16:27:02.275 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:33 smithi055 bash[9296]: cluster 2020-09-08T16:15:32.214578+0000 mgr.x (mgr.24118) 1446 : cluster [DBG] pgmap v2154: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.275 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:35 smithi055 bash[9296]: cluster 2020-09-08T16:15:34.215255+0000 mgr.x (mgr.24118) 1447 : cluster [DBG] pgmap v2155: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.275 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:35 smithi055 bash[9296]: cluster 2020-09-08T16:15:34.709795+0000 osd.10 (osd.10) 123 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.275 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:35 smithi055 bash[9296]: cluster 2020-09-08T16:15:34.709916+0000 osd.10 (osd.10) 124 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.275 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:36 smithi055 bash[9296]: cluster 2020-09-08T16:15:35.279221+0000 mon.a (mon.0) 2411 : cluster [DBG] osdmap e793: 12 total, 12 up, 12 in 2020-09-08T16:27:02.276 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:36 smithi055 bash[9296]: cluster 2020-09-08T16:15:36.132157+0000 osd.3 (osd.3) 147 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.276 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:36 smithi055 bash[9296]: cluster 2020-09-08T16:15:36.132304+0000 osd.3 (osd.3) 148 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.276 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:37 smithi055 bash[9296]: cluster 2020-09-08T16:15:36.215863+0000 mgr.x (mgr.24118) 1448 : cluster [DBG] pgmap v2157: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.276 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:37 smithi055 bash[9296]: cluster 2020-09-08T16:15:36.298356+0000 mon.a (mon.0) 2412 : cluster [DBG] osdmap e794: 12 total, 12 up, 12 in 2020-09-08T16:27:02.276 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:39 smithi055 bash[9296]: cluster 2020-09-08T16:15:38.216592+0000 mgr.x (mgr.24118) 1449 : cluster [DBG] pgmap v2159: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.277 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:41 smithi055 bash[9296]: cluster 2020-09-08T16:15:40.217245+0000 mgr.x (mgr.24118) 1450 : cluster [DBG] pgmap v2160: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.277 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:43 smithi055 bash[9296]: cluster 2020-09-08T16:15:42.217849+0000 mgr.x (mgr.24118) 1451 : cluster [DBG] pgmap v2161: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.277 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:44 smithi055 bash[9296]: audit 2020-09-08T16:15:43.297700+0000 mon.a (mon.0) 2413 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.277 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:44 smithi055 bash[9296]: audit 2020-09-08T16:15:43.303045+0000 mon.a (mon.0) 2414 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.277 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:45 smithi055 bash[9296]: cluster 2020-09-08T16:15:43.996351+0000 osd.8 (osd.8) 118 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.278 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:45 smithi055 bash[9296]: cluster 2020-09-08T16:15:43.996507+0000 osd.8 (osd.8) 119 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.278 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:45 smithi055 bash[9296]: cluster 2020-09-08T16:15:44.218673+0000 mgr.x (mgr.24118) 1452 : cluster [DBG] pgmap v2162: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.278 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:45 smithi055 bash[9296]: cluster 2020-09-08T16:15:44.311575+0000 mon.a (mon.0) 2415 : cluster [DBG] osdmap e795: 12 total, 12 up, 12 in 2020-09-08T16:27:02.278 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:47 smithi055 bash[9296]: cluster 2020-09-08T16:15:46.219382+0000 mgr.x (mgr.24118) 1453 : cluster [DBG] pgmap v2164: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.278 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:49 smithi055 bash[9296]: cluster 2020-09-08T16:15:48.220093+0000 mgr.x (mgr.24118) 1454 : cluster [DBG] pgmap v2165: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.279 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:51 smithi055 bash[9296]: cluster 2020-09-08T16:15:50.220756+0000 mgr.x (mgr.24118) 1455 : cluster [DBG] pgmap v2166: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.279 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:51 smithi055 bash[9296]: cluster 2020-09-08T16:15:50.365951+0000 osd.1 (osd.1) 164 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.279 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:51 smithi055 bash[9296]: cluster 2020-09-08T16:15:50.366086+0000 osd.1 (osd.1) 165 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.279 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:52 smithi055 bash[9296]: cluster 2020-09-08T16:15:51.154801+0000 mon.a (mon.0) 2416 : cluster [DBG] osdmap e796: 12 total, 12 up, 12 in 2020-09-08T16:27:02.279 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:53 smithi055 bash[9296]: cluster 2020-09-08T16:15:52.221434+0000 mgr.x (mgr.24118) 1456 : cluster [DBG] pgmap v2168: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.280 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:55 smithi055 bash[9296]: cluster 2020-09-08T16:15:54.222100+0000 mgr.x (mgr.24118) 1457 : cluster [DBG] pgmap v2169: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.280 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:56 smithi055 bash[9296]: cluster 2020-09-08T16:15:54.794063+0000 osd.6 (osd.6) 129 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.280 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:56 smithi055 bash[9296]: cluster 2020-09-08T16:15:54.794193+0000 osd.6 (osd.6) 130 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.280 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:56 smithi055 bash[9296]: cluster 2020-09-08T16:15:55.292870+0000 mon.a (mon.0) 2417 : cluster [DBG] osdmap e797: 12 total, 12 up, 12 in 2020-09-08T16:27:02.280 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:57 smithi055 bash[9296]: cluster 2020-09-08T16:15:56.222752+0000 mgr.x (mgr.24118) 1458 : cluster [DBG] pgmap v2171: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.281 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:15:59 smithi055 bash[9296]: cluster 2020-09-08T16:15:58.223435+0000 mgr.x (mgr.24118) 1459 : cluster [DBG] pgmap v2172: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.281 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:01 smithi055 bash[9296]: cluster 2020-09-08T16:16:00.114018+0000 osd.11 (osd.11) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.281 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:01 smithi055 bash[9296]: cluster 2020-09-08T16:16:00.114147+0000 osd.11 (osd.11) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.281 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:01 smithi055 bash[9296]: cluster 2020-09-08T16:16:00.224131+0000 mgr.x (mgr.24118) 1460 : cluster [DBG] pgmap v2173: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.281 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:01 smithi055 bash[9296]: cluster 2020-09-08T16:16:00.318210+0000 mon.a (mon.0) 2418 : cluster [DBG] osdmap e798: 12 total, 12 up, 12 in 2020-09-08T16:27:02.282 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:03 smithi055 bash[9296]: cluster 2020-09-08T16:16:02.224877+0000 mgr.x (mgr.24118) 1461 : cluster [DBG] pgmap v2175: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.282 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:04 smithi055 bash[9296]: cluster 2020-09-08T16:16:03.431457+0000 osd.4 (osd.4) 89 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.282 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:04 smithi055 bash[9296]: cluster 2020-09-08T16:16:03.431583+0000 osd.4 (osd.4) 90 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.282 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:05 smithi055 bash[9296]: cluster 2020-09-08T16:16:04.225590+0000 mgr.x (mgr.24118) 1462 : cluster [DBG] pgmap v2176: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.282 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:05 smithi055 bash[9296]: cluster 2020-09-08T16:16:04.339098+0000 mon.a (mon.0) 2419 : cluster [DBG] osdmap e799: 12 total, 12 up, 12 in 2020-09-08T16:27:02.283 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:07 smithi055 bash[9296]: cluster 2020-09-08T16:16:06.226246+0000 mgr.x (mgr.24118) 1463 : cluster [DBG] pgmap v2178: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.283 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:09 smithi055 bash[9296]: cluster 2020-09-08T16:16:08.226976+0000 mgr.x (mgr.24118) 1464 : cluster [DBG] pgmap v2179: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.283 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:11 smithi055 bash[9296]: cluster 2020-09-08T16:16:10.227554+0000 mgr.x (mgr.24118) 1465 : cluster [DBG] pgmap v2180: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.283 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:12 smithi055 bash[9296]: cluster 2020-09-08T16:16:11.941232+0000 osd.9 (osd.9) 128 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.283 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:12 smithi055 bash[9296]: cluster 2020-09-08T16:16:11.941368+0000 osd.9 (osd.9) 129 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.284 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:13 smithi055 bash[9296]: cluster 2020-09-08T16:16:12.164050+0000 mon.a (mon.0) 2420 : cluster [DBG] osdmap e800: 12 total, 12 up, 12 in 2020-09-08T16:27:02.284 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:13 smithi055 bash[9296]: cluster 2020-09-08T16:16:12.228176+0000 mgr.x (mgr.24118) 1466 : cluster [DBG] pgmap v2182: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.284 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:15 smithi055 bash[9296]: cluster 2020-09-08T16:16:14.228699+0000 mgr.x (mgr.24118) 1467 : cluster [DBG] pgmap v2183: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.284 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:15 smithi055 bash[9296]: cluster 2020-09-08T16:16:14.413784+0000 osd.2 (osd.2) 124 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.284 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:15 smithi055 bash[9296]: cluster 2020-09-08T16:16:14.413922+0000 osd.2 (osd.2) 125 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.285 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:16 smithi055 bash[9296]: cluster 2020-09-08T16:16:15.293924+0000 mon.a (mon.0) 2421 : cluster [DBG] osdmap e801: 12 total, 12 up, 12 in 2020-09-08T16:27:02.285 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:17 smithi055 bash[9296]: cluster 2020-09-08T16:16:16.229284+0000 mgr.x (mgr.24118) 1468 : cluster [DBG] pgmap v2185: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.285 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:19 smithi055 bash[9296]: cluster 2020-09-08T16:16:18.230105+0000 mgr.x (mgr.24118) 1469 : cluster [DBG] pgmap v2186: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.285 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:21 smithi055 bash[9296]: cluster 2020-09-08T16:16:20.230600+0000 mgr.x (mgr.24118) 1470 : cluster [DBG] pgmap v2187: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.285 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:23 smithi055 bash[9296]: cluster 2020-09-08T16:16:22.231310+0000 mgr.x (mgr.24118) 1471 : cluster [DBG] pgmap v2188: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.286 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:24 smithi055 bash[9296]: cluster 2020-09-08T16:16:22.893870+0000 osd.7 (osd.7) 180 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.286 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:24 smithi055 bash[9296]: cluster 2020-09-08T16:16:22.893995+0000 osd.7 (osd.7) 181 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.286 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:24 smithi055 bash[9296]: cluster 2020-09-08T16:16:23.295140+0000 mon.a (mon.0) 2422 : cluster [DBG] osdmap e802: 12 total, 12 up, 12 in 2020-09-08T16:27:02.286 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:24 smithi055 bash[9296]: cluster 2020-09-08T16:16:23.610710+0000 osd.0 (osd.0) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.286 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:24 smithi055 bash[9296]: cluster 2020-09-08T16:16:23.610833+0000 osd.0 (osd.0) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.287 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:25 smithi055 bash[9296]: cluster 2020-09-08T16:16:24.231993+0000 mgr.x (mgr.24118) 1472 : cluster [DBG] pgmap v2190: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.287 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:25 smithi055 bash[9296]: cluster 2020-09-08T16:16:24.320596+0000 mon.a (mon.0) 2423 : cluster [DBG] osdmap e803: 12 total, 12 up, 12 in 2020-09-08T16:27:02.287 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:27 smithi055 bash[9296]: cluster 2020-09-08T16:16:26.232468+0000 mgr.x (mgr.24118) 1473 : cluster [DBG] pgmap v2192: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.287 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:29 smithi055 bash[9296]: cluster 2020-09-08T16:16:28.233181+0000 mgr.x (mgr.24118) 1474 : cluster [DBG] pgmap v2193: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.287 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:31 smithi055 bash[9296]: cluster 2020-09-08T16:16:30.233865+0000 mgr.x (mgr.24118) 1475 : cluster [DBG] pgmap v2194: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.288 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:33 smithi055 bash[9296]: cluster 2020-09-08T16:16:32.234550+0000 mgr.x (mgr.24118) 1476 : cluster [DBG] pgmap v2195: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.288 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:33 smithi055 bash[9296]: cluster 2020-09-08T16:16:32.555375+0000 osd.5 (osd.5) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.288 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:33 smithi055 bash[9296]: cluster 2020-09-08T16:16:32.555503+0000 osd.5 (osd.5) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.288 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:34 smithi055 bash[9296]: cluster 2020-09-08T16:16:33.304105+0000 mon.a (mon.0) 2424 : cluster [DBG] osdmap e804: 12 total, 12 up, 12 in 2020-09-08T16:27:02.289 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:35 smithi055 bash[9296]: cluster 2020-09-08T16:16:34.235182+0000 mgr.x (mgr.24118) 1477 : cluster [DBG] pgmap v2197: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.289 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:35 smithi055 bash[9296]: cluster 2020-09-08T16:16:34.831684+0000 osd.10 (osd.10) 125 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.289 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:35 smithi055 bash[9296]: cluster 2020-09-08T16:16:34.831801+0000 osd.10 (osd.10) 126 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.289 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:36 smithi055 bash[9296]: cluster 2020-09-08T16:16:35.364819+0000 mon.a (mon.0) 2425 : cluster [DBG] osdmap e805: 12 total, 12 up, 12 in 2020-09-08T16:27:02.289 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:37 smithi055 bash[9296]: cluster 2020-09-08T16:16:36.235834+0000 mgr.x (mgr.24118) 1478 : cluster [DBG] pgmap v2199: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.290 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:37 smithi055 bash[9296]: cluster 2020-09-08T16:16:36.286746+0000 osd.3 (osd.3) 149 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.290 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:37 smithi055 bash[9296]: cluster 2020-09-08T16:16:36.286865+0000 osd.3 (osd.3) 150 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.290 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:37 smithi055 bash[9296]: cluster 2020-09-08T16:16:36.395282+0000 mon.a (mon.0) 2426 : cluster [DBG] osdmap e806: 12 total, 12 up, 12 in 2020-09-08T16:27:02.290 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:39 smithi055 bash[9296]: cluster 2020-09-08T16:16:38.236525+0000 mgr.x (mgr.24118) 1479 : cluster [DBG] pgmap v2201: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.290 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:41 smithi055 bash[9296]: cluster 2020-09-08T16:16:40.237222+0000 mgr.x (mgr.24118) 1480 : cluster [DBG] pgmap v2202: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.291 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:43 smithi055 bash[9296]: cluster 2020-09-08T16:16:42.237909+0000 mgr.x (mgr.24118) 1481 : cluster [DBG] pgmap v2203: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.291 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:45 smithi055 bash[9296]: cluster 2020-09-08T16:16:44.238674+0000 mgr.x (mgr.24118) 1482 : cluster [DBG] pgmap v2204: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.291 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:45 smithi055 bash[9296]: cluster 2020-09-08T16:16:44.342159+0000 osd.8 (osd.8) 120 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.291 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:45 smithi055 bash[9296]: cluster 2020-09-08T16:16:44.342307+0000 osd.8 (osd.8) 121 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.291 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:46 smithi055 bash[9296]: cluster 2020-09-08T16:16:45.315881+0000 mon.a (mon.0) 2427 : cluster [DBG] osdmap e807: 12 total, 12 up, 12 in 2020-09-08T16:27:02.292 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:47 smithi055 bash[9296]: cluster 2020-09-08T16:16:46.239313+0000 mgr.x (mgr.24118) 1483 : cluster [DBG] pgmap v2206: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.292 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:49 smithi055 bash[9296]: cluster 2020-09-08T16:16:48.240031+0000 mgr.x (mgr.24118) 1484 : cluster [DBG] pgmap v2207: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.292 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:51 smithi055 bash[9296]: cluster 2020-09-08T16:16:50.240728+0000 mgr.x (mgr.24118) 1485 : cluster [DBG] pgmap v2208: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.292 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:52 smithi055 bash[9296]: cluster 2020-09-08T16:16:51.121318+0000 osd.1 (osd.1) 166 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.292 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:52 smithi055 bash[9296]: cluster 2020-09-08T16:16:51.121449+0000 osd.1 (osd.1) 167 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.293 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:52 smithi055 bash[9296]: cluster 2020-09-08T16:16:51.178890+0000 mon.a (mon.0) 2428 : cluster [DBG] osdmap e808: 12 total, 12 up, 12 in 2020-09-08T16:27:02.293 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:53 smithi055 bash[9296]: cluster 2020-09-08T16:16:52.241350+0000 mgr.x (mgr.24118) 1486 : cluster [DBG] pgmap v2210: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.293 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:55 smithi055 bash[9296]: cluster 2020-09-08T16:16:54.242071+0000 mgr.x (mgr.24118) 1487 : cluster [DBG] pgmap v2211: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.293 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:56 smithi055 bash[9296]: cluster 2020-09-08T16:16:55.202047+0000 osd.6 (osd.6) 131 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.293 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:56 smithi055 bash[9296]: cluster 2020-09-08T16:16:55.202190+0000 osd.6 (osd.6) 132 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.294 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:56 smithi055 bash[9296]: cluster 2020-09-08T16:16:56.176655+0000 mon.a (mon.0) 2429 : cluster [DBG] osdmap e809: 12 total, 12 up, 12 in 2020-09-08T16:27:02.294 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:57 smithi055 bash[9296]: cluster 2020-09-08T16:16:56.242697+0000 mgr.x (mgr.24118) 1488 : cluster [DBG] pgmap v2213: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.294 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:16:59 smithi055 bash[9296]: cluster 2020-09-08T16:16:58.243347+0000 mgr.x (mgr.24118) 1489 : cluster [DBG] pgmap v2214: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.294 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:01 smithi055 bash[9296]: cluster 2020-09-08T16:17:00.244025+0000 mgr.x (mgr.24118) 1490 : cluster [DBG] pgmap v2215: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.294 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:01 smithi055 bash[9296]: cluster 2020-09-08T16:17:00.467924+0000 osd.11 (osd.11) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.295 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:01 smithi055 bash[9296]: cluster 2020-09-08T16:17:00.468086+0000 osd.11 (osd.11) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.295 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:02 smithi055 bash[9296]: cluster 2020-09-08T16:17:01.193296+0000 mon.a (mon.0) 2430 : cluster [DBG] osdmap e810: 12 total, 12 up, 12 in 2020-09-08T16:27:02.295 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:03 smithi055 bash[9296]: cluster 2020-09-08T16:17:02.244701+0000 mgr.x (mgr.24118) 1491 : cluster [DBG] pgmap v2217: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.295 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:05 smithi055 bash[9296]: cluster 2020-09-08T16:17:03.887171+0000 osd.4 (osd.4) 91 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.295 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:05 smithi055 bash[9296]: cluster 2020-09-08T16:17:03.887298+0000 osd.4 (osd.4) 92 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.296 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:05 smithi055 bash[9296]: cluster 2020-09-08T16:17:04.212140+0000 mon.a (mon.0) 2431 : cluster [DBG] osdmap e811: 12 total, 12 up, 12 in 2020-09-08T16:27:02.296 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:05 smithi055 bash[9296]: cluster 2020-09-08T16:17:04.245303+0000 mgr.x (mgr.24118) 1492 : cluster [DBG] pgmap v2219: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.296 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:07 smithi055 bash[9296]: cluster 2020-09-08T16:17:06.245996+0000 mgr.x (mgr.24118) 1493 : cluster [DBG] pgmap v2220: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.296 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:09 smithi055 bash[9296]: cluster 2020-09-08T16:17:08.246741+0000 mgr.x (mgr.24118) 1494 : cluster [DBG] pgmap v2221: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.296 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:11 smithi055 bash[9296]: cluster 2020-09-08T16:17:10.247372+0000 mgr.x (mgr.24118) 1495 : cluster [DBG] pgmap v2222: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.297 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:13 smithi055 bash[9296]: cluster 2020-09-08T16:17:12.248035+0000 mgr.x (mgr.24118) 1496 : cluster [DBG] pgmap v2223: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.297 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:13 smithi055 bash[9296]: cluster 2020-09-08T16:17:12.903835+0000 osd.9 (osd.9) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.297 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:13 smithi055 bash[9296]: cluster 2020-09-08T16:17:12.903962+0000 osd.9 (osd.9) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.297 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:14 smithi055 bash[9296]: cluster 2020-09-08T16:17:13.310006+0000 mon.a (mon.0) 2432 : cluster [DBG] osdmap e812: 12 total, 12 up, 12 in 2020-09-08T16:27:02.298 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:15 smithi055 bash[9296]: cluster 2020-09-08T16:17:14.248684+0000 mgr.x (mgr.24118) 1497 : cluster [DBG] pgmap v2225: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.298 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:16 smithi055 bash[9296]: cluster 2020-09-08T16:17:15.256758+0000 osd.2 (osd.2) 126 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.298 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:16 smithi055 bash[9296]: cluster 2020-09-08T16:17:15.256850+0000 osd.2 (osd.2) 127 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.298 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:16 smithi055 bash[9296]: cluster 2020-09-08T16:17:16.187290+0000 mon.a (mon.0) 2433 : cluster [DBG] osdmap e813: 12 total, 12 up, 12 in 2020-09-08T16:27:02.298 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:17 smithi055 bash[9296]: cluster 2020-09-08T16:17:16.249291+0000 mgr.x (mgr.24118) 1498 : cluster [DBG] pgmap v2227: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.299 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:19 smithi055 bash[9296]: cluster 2020-09-08T16:17:18.250125+0000 mgr.x (mgr.24118) 1499 : cluster [DBG] pgmap v2228: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.299 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:21 smithi055 bash[9296]: cluster 2020-09-08T16:17:20.250901+0000 mgr.x (mgr.24118) 1500 : cluster [DBG] pgmap v2229: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.299 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:23 smithi055 bash[9296]: cluster 2020-09-08T16:17:22.251569+0000 mgr.x (mgr.24118) 1501 : cluster [DBG] pgmap v2230: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.299 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:24 smithi055 bash[9296]: cluster 2020-09-08T16:17:22.982890+0000 osd.7 (osd.7) 182 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.299 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:24 smithi055 bash[9296]: cluster 2020-09-08T16:17:22.983017+0000 osd.7 (osd.7) 183 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.300 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:24 smithi055 bash[9296]: cluster 2020-09-08T16:17:23.322276+0000 mon.a (mon.0) 2434 : cluster [DBG] osdmap e814: 12 total, 12 up, 12 in 2020-09-08T16:27:02.300 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:24 smithi055 bash[9296]: cluster 2020-09-08T16:17:23.824128+0000 osd.0 (osd.0) 142 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.300 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:24 smithi055 bash[9296]: cluster 2020-09-08T16:17:23.824255+0000 osd.0 (osd.0) 143 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.300 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:25 smithi055 bash[9296]: cluster 2020-09-08T16:17:24.252193+0000 mgr.x (mgr.24118) 1502 : cluster [DBG] pgmap v2232: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.300 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:25 smithi055 bash[9296]: cluster 2020-09-08T16:17:24.334057+0000 mon.a (mon.0) 2435 : cluster [DBG] osdmap e815: 12 total, 12 up, 12 in 2020-09-08T16:27:02.301 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:27 smithi055 bash[9296]: cluster 2020-09-08T16:17:26.252876+0000 mgr.x (mgr.24118) 1503 : cluster [DBG] pgmap v2234: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.301 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:29 smithi055 bash[9296]: cluster 2020-09-08T16:17:28.253590+0000 mgr.x (mgr.24118) 1504 : cluster [DBG] pgmap v2235: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.301 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:31 smithi055 bash[9296]: cluster 2020-09-08T16:17:30.254239+0000 mgr.x (mgr.24118) 1505 : cluster [DBG] pgmap v2236: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.301 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:33 smithi055 bash[9296]: cluster 2020-09-08T16:17:32.254924+0000 mgr.x (mgr.24118) 1506 : cluster [DBG] pgmap v2237: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.301 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:34 smithi055 bash[9296]: cluster 2020-09-08T16:17:33.361145+0000 osd.5 (osd.5) 148 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.302 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:34 smithi055 bash[9296]: cluster 2020-09-08T16:17:33.361282+0000 osd.5 (osd.5) 149 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.302 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:35 smithi055 bash[9296]: cluster 2020-09-08T16:17:34.255666+0000 mgr.x (mgr.24118) 1507 : cluster [DBG] pgmap v2238: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.302 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:35 smithi055 bash[9296]: cluster 2020-09-08T16:17:34.330857+0000 mon.a (mon.0) 2436 : cluster [DBG] osdmap e816: 12 total, 12 up, 12 in 2020-09-08T16:27:02.302 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:36 smithi055 bash[9296]: cluster 2020-09-08T16:17:35.549540+0000 osd.10 (osd.10) 127 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.302 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:36 smithi055 bash[9296]: cluster 2020-09-08T16:17:35.549670+0000 osd.10 (osd.10) 128 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.303 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:37 smithi055 bash[9296]: cluster 2020-09-08T16:17:36.256318+0000 mgr.x (mgr.24118) 1508 : cluster [DBG] pgmap v2240: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.303 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:37 smithi055 bash[9296]: cluster 2020-09-08T16:17:36.344939+0000 mon.a (mon.0) 2437 : cluster [DBG] osdmap e817: 12 total, 12 up, 12 in 2020-09-08T16:27:02.303 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:37 smithi055 bash[9296]: cluster 2020-09-08T16:17:37.131639+0000 osd.3 (osd.3) 151 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.303 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:37 smithi055 bash[9296]: cluster 2020-09-08T16:17:37.131762+0000 osd.3 (osd.3) 152 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.303 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:38 smithi055 bash[9296]: cluster 2020-09-08T16:17:37.346297+0000 mon.a (mon.0) 2438 : cluster [DBG] osdmap e818: 12 total, 12 up, 12 in 2020-09-08T16:27:02.303 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:39 smithi055 bash[9296]: cluster 2020-09-08T16:17:38.256992+0000 mgr.x (mgr.24118) 1509 : cluster [DBG] pgmap v2243: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.304 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:41 smithi055 bash[9296]: cluster 2020-09-08T16:17:40.257685+0000 mgr.x (mgr.24118) 1510 : cluster [DBG] pgmap v2244: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.304 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:43 smithi055 bash[9296]: cluster 2020-09-08T16:17:42.258389+0000 mgr.x (mgr.24118) 1511 : cluster [DBG] pgmap v2245: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.304 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:45 smithi055 bash[9296]: cluster 2020-09-08T16:17:44.259204+0000 mgr.x (mgr.24118) 1512 : cluster [DBG] pgmap v2246: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.304 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:46 smithi055 bash[9296]: cluster 2020-09-08T16:17:44.578715+0000 osd.8 (osd.8) 122 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.304 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:46 smithi055 bash[9296]: cluster 2020-09-08T16:17:44.578865+0000 osd.8 (osd.8) 123 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.305 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:46 smithi055 bash[9296]: cluster 2020-09-08T16:17:45.337997+0000 mon.a (mon.0) 2439 : cluster [DBG] osdmap e819: 12 total, 12 up, 12 in 2020-09-08T16:27:02.305 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:47 smithi055 bash[9296]: cluster 2020-09-08T16:17:46.259938+0000 mgr.x (mgr.24118) 1513 : cluster [DBG] pgmap v2248: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.305 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:49 smithi055 bash[9296]: cluster 2020-09-08T16:17:48.260855+0000 mgr.x (mgr.24118) 1514 : cluster [DBG] pgmap v2249: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.305 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:51 smithi055 bash[9296]: cluster 2020-09-08T16:17:50.261601+0000 mgr.x (mgr.24118) 1515 : cluster [DBG] pgmap v2250: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.305 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:52 smithi055 bash[9296]: cluster 2020-09-08T16:17:51.785942+0000 osd.1 (osd.1) 168 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.306 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:52 smithi055 bash[9296]: cluster 2020-09-08T16:17:51.786066+0000 osd.1 (osd.1) 169 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.306 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:53 smithi055 bash[9296]: cluster 2020-09-08T16:17:52.207446+0000 mon.a (mon.0) 2440 : cluster [DBG] osdmap e820: 12 total, 12 up, 12 in 2020-09-08T16:27:02.306 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:53 smithi055 bash[9296]: cluster 2020-09-08T16:17:52.262185+0000 mgr.x (mgr.24118) 1516 : cluster [DBG] pgmap v2252: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.306 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:55 smithi055 bash[9296]: cluster 2020-09-08T16:17:54.262935+0000 mgr.x (mgr.24118) 1517 : cluster [DBG] pgmap v2253: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.306 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:57 smithi055 bash[9296]: cluster 2020-09-08T16:17:56.087499+0000 osd.6 (osd.6) 133 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.307 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:57 smithi055 bash[9296]: cluster 2020-09-08T16:17:56.087630+0000 osd.6 (osd.6) 134 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.307 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:57 smithi055 bash[9296]: cluster 2020-09-08T16:17:56.263630+0000 mgr.x (mgr.24118) 1518 : cluster [DBG] pgmap v2254: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.307 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:57 smithi055 bash[9296]: cluster 2020-09-08T16:17:56.337988+0000 mon.a (mon.0) 2441 : cluster [DBG] osdmap e821: 12 total, 12 up, 12 in 2020-09-08T16:27:02.307 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:17:59 smithi055 bash[9296]: cluster 2020-09-08T16:17:58.264325+0000 mgr.x (mgr.24118) 1519 : cluster [DBG] pgmap v2256: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.308 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:01 smithi055 bash[9296]: cluster 2020-09-08T16:18:00.265034+0000 mgr.x (mgr.24118) 1520 : cluster [DBG] pgmap v2257: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.308 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:02 smithi055 bash[9296]: cluster 2020-09-08T16:18:01.420675+0000 osd.11 (osd.11) 148 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.308 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:02 smithi055 bash[9296]: cluster 2020-09-08T16:18:01.420810+0000 osd.11 (osd.11) 149 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.308 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:03 smithi055 bash[9296]: cluster 2020-09-08T16:18:02.213345+0000 mon.a (mon.0) 2442 : cluster [DBG] osdmap e822: 12 total, 12 up, 12 in 2020-09-08T16:27:02.308 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:03 smithi055 bash[9296]: cluster 2020-09-08T16:18:02.265680+0000 mgr.x (mgr.24118) 1521 : cluster [DBG] pgmap v2259: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.309 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:05 smithi055 bash[9296]: cluster 2020-09-08T16:18:04.266460+0000 mgr.x (mgr.24118) 1522 : cluster [DBG] pgmap v2260: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.309 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:06 smithi055 bash[9296]: cluster 2020-09-08T16:18:04.742722+0000 osd.4 (osd.4) 93 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.309 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:06 smithi055 bash[9296]: cluster 2020-09-08T16:18:04.742846+0000 osd.4 (osd.4) 94 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.309 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:06 smithi055 bash[9296]: cluster 2020-09-08T16:18:05.327738+0000 mon.a (mon.0) 2443 : cluster [DBG] osdmap e823: 12 total, 12 up, 12 in 2020-09-08T16:27:02.309 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:07 smithi055 bash[9296]: cluster 2020-09-08T16:18:06.267129+0000 mgr.x (mgr.24118) 1523 : cluster [DBG] pgmap v2262: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.310 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:09 smithi055 bash[9296]: cluster 2020-09-08T16:18:08.267890+0000 mgr.x (mgr.24118) 1524 : cluster [DBG] pgmap v2263: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.310 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:11 smithi055 bash[9296]: cluster 2020-09-08T16:18:10.268693+0000 mgr.x (mgr.24118) 1525 : cluster [DBG] pgmap v2264: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.310 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:13 smithi055 bash[9296]: cluster 2020-09-08T16:18:12.269386+0000 mgr.x (mgr.24118) 1526 : cluster [DBG] pgmap v2265: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.310 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:14 smithi055 bash[9296]: cluster 2020-09-08T16:18:13.633224+0000 osd.9 (osd.9) 132 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.310 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:14 smithi055 bash[9296]: cluster 2020-09-08T16:18:13.633351+0000 osd.9 (osd.9) 133 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.311 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:15 smithi055 bash[9296]: cluster 2020-09-08T16:18:14.270125+0000 mgr.x (mgr.24118) 1527 : cluster [DBG] pgmap v2266: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.311 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:15 smithi055 bash[9296]: cluster 2020-09-08T16:18:14.343382+0000 mon.a (mon.0) 2444 : cluster [DBG] osdmap e824: 12 total, 12 up, 12 in 2020-09-08T16:27:02.311 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:17 smithi055 bash[9296]: cluster 2020-09-08T16:18:16.208579+0000 osd.2 (osd.2) 128 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.311 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:17 smithi055 bash[9296]: cluster 2020-09-08T16:18:16.208704+0000 osd.2 (osd.2) 129 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.311 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:17 smithi055 bash[9296]: cluster 2020-09-08T16:18:16.270774+0000 mgr.x (mgr.24118) 1528 : cluster [DBG] pgmap v2268: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.312 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:17 smithi055 bash[9296]: cluster 2020-09-08T16:18:16.358692+0000 mon.a (mon.0) 2445 : cluster [DBG] osdmap e825: 12 total, 12 up, 12 in 2020-09-08T16:27:02.312 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:19 smithi055 bash[9296]: cluster 2020-09-08T16:18:18.271514+0000 mgr.x (mgr.24118) 1529 : cluster [DBG] pgmap v2270: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.312 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:21 smithi055 bash[9296]: cluster 2020-09-08T16:18:20.272244+0000 mgr.x (mgr.24118) 1530 : cluster [DBG] pgmap v2271: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.312 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:23 smithi055 bash[9296]: cluster 2020-09-08T16:18:22.272930+0000 mgr.x (mgr.24118) 1531 : cluster [DBG] pgmap v2272: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.312 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:25 smithi055 bash[9296]: cluster 2020-09-08T16:18:23.788192+0000 osd.7 (osd.7) 184 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.313 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:25 smithi055 bash[9296]: cluster 2020-09-08T16:18:23.788334+0000 osd.7 (osd.7) 185 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.313 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:25 smithi055 bash[9296]: cluster 2020-09-08T16:18:24.273691+0000 mgr.x (mgr.24118) 1532 : cluster [DBG] pgmap v2273: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.313 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:25 smithi055 bash[9296]: cluster 2020-09-08T16:18:24.345809+0000 mon.a (mon.0) 2446 : cluster [DBG] osdmap e826: 12 total, 12 up, 12 in 2020-09-08T16:27:02.313 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:25 smithi055 bash[9296]: cluster 2020-09-08T16:18:24.542528+0000 osd.0 (osd.0) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.313 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:25 smithi055 bash[9296]: cluster 2020-09-08T16:18:24.542657+0000 osd.0 (osd.0) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.314 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:26 smithi055 bash[9296]: cluster 2020-09-08T16:18:25.346774+0000 mon.a (mon.0) 2447 : cluster [DBG] osdmap e827: 12 total, 12 up, 12 in 2020-09-08T16:27:02.314 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:27 smithi055 bash[9296]: cluster 2020-09-08T16:18:26.274355+0000 mgr.x (mgr.24118) 1533 : cluster [DBG] pgmap v2276: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.314 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:29 smithi055 bash[9296]: cluster 2020-09-08T16:18:28.275071+0000 mgr.x (mgr.24118) 1534 : cluster [DBG] pgmap v2277: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.314 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:30 smithi055 bash[9296]: audit 2020-09-08T16:18:29.527573+0000 mon.b (mon.1) 136 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.314 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:30 smithi055 bash[9296]: audit 2020-09-08T16:18:29.529031+0000 mon.a (mon.0) 2448 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.315 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:30 smithi055 bash[9296]: audit 2020-09-08T16:18:29.534278+0000 mon.a (mon.0) 2449 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.315 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:31 smithi055 bash[9296]: cluster 2020-09-08T16:18:30.275532+0000 mgr.x (mgr.24118) 1535 : cluster [DBG] pgmap v2278: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.315 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:33 smithi055 bash[9296]: cluster 2020-09-08T16:18:32.276128+0000 mgr.x (mgr.24118) 1536 : cluster [DBG] pgmap v2279: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.315 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:35 smithi055 bash[9296]: cluster 2020-09-08T16:18:34.142626+0000 osd.5 (osd.5) 150 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.315 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:35 smithi055 bash[9296]: cluster 2020-09-08T16:18:34.142759+0000 osd.5 (osd.5) 151 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.316 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:35 smithi055 bash[9296]: cluster 2020-09-08T16:18:34.276879+0000 mgr.x (mgr.24118) 1537 : cluster [DBG] pgmap v2280: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.316 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:35 smithi055 bash[9296]: cluster 2020-09-08T16:18:34.400794+0000 mon.a (mon.0) 2450 : cluster [DBG] osdmap e828: 12 total, 12 up, 12 in 2020-09-08T16:27:02.316 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:36 smithi055 bash[9296]: cluster 2020-09-08T16:18:35.596323+0000 osd.10 (osd.10) 129 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.316 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:36 smithi055 bash[9296]: cluster 2020-09-08T16:18:35.596454+0000 osd.10 (osd.10) 130 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.316 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:37 smithi055 bash[9296]: cluster 2020-09-08T16:18:36.277520+0000 mgr.x (mgr.24118) 1538 : cluster [DBG] pgmap v2282: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.317 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:37 smithi055 bash[9296]: cluster 2020-09-08T16:18:36.483874+0000 mon.a (mon.0) 2451 : cluster [DBG] osdmap e829: 12 total, 12 up, 12 in 2020-09-08T16:27:02.317 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:38 smithi055 bash[9296]: cluster 2020-09-08T16:18:37.338299+0000 osd.3 (osd.3) 153 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.317 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:38 smithi055 bash[9296]: cluster 2020-09-08T16:18:37.338410+0000 osd.3 (osd.3) 154 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.317 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:38 smithi055 bash[9296]: cluster 2020-09-08T16:18:37.491995+0000 mon.a (mon.0) 2452 : cluster [DBG] osdmap e830: 12 total, 12 up, 12 in 2020-09-08T16:27:02.317 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:39 smithi055 bash[9296]: cluster 2020-09-08T16:18:38.278287+0000 mgr.x (mgr.24118) 1539 : cluster [DBG] pgmap v2285: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.318 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:41 smithi055 bash[9296]: cluster 2020-09-08T16:18:40.279025+0000 mgr.x (mgr.24118) 1540 : cluster [DBG] pgmap v2286: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.318 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:43 smithi055 bash[9296]: cluster 2020-09-08T16:18:42.279684+0000 mgr.x (mgr.24118) 1541 : cluster [DBG] pgmap v2287: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.318 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:45 smithi055 bash[9296]: cluster 2020-09-08T16:18:44.280459+0000 mgr.x (mgr.24118) 1542 : cluster [DBG] pgmap v2288: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.318 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:46 smithi055 bash[9296]: cluster 2020-09-08T16:18:44.720522+0000 osd.8 (osd.8) 124 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.318 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:46 smithi055 bash[9296]: cluster 2020-09-08T16:18:44.720655+0000 osd.8 (osd.8) 125 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.319 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:46 smithi055 bash[9296]: cluster 2020-09-08T16:18:45.355304+0000 mon.a (mon.0) 2453 : cluster [DBG] osdmap e831: 12 total, 12 up, 12 in 2020-09-08T16:27:02.319 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:47 smithi055 bash[9296]: cluster 2020-09-08T16:18:46.281093+0000 mgr.x (mgr.24118) 1543 : cluster [DBG] pgmap v2290: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.319 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:49 smithi055 bash[9296]: cluster 2020-09-08T16:18:48.281847+0000 mgr.x (mgr.24118) 1544 : cluster [DBG] pgmap v2291: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.319 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:51 smithi055 bash[9296]: cluster 2020-09-08T16:18:50.282574+0000 mgr.x (mgr.24118) 1545 : cluster [DBG] pgmap v2292: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.319 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:52 smithi055 bash[9296]: cluster 2020-09-08T16:18:51.806760+0000 osd.1 (osd.1) 170 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.320 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:52 smithi055 bash[9296]: cluster 2020-09-08T16:18:51.806888+0000 osd.1 (osd.1) 171 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.320 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:53 smithi055 bash[9296]: cluster 2020-09-08T16:18:52.228580+0000 mon.a (mon.0) 2454 : cluster [DBG] osdmap e832: 12 total, 12 up, 12 in 2020-09-08T16:27:02.320 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:53 smithi055 bash[9296]: cluster 2020-09-08T16:18:52.283236+0000 mgr.x (mgr.24118) 1546 : cluster [DBG] pgmap v2294: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.320 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:55 smithi055 bash[9296]: cluster 2020-09-08T16:18:54.283994+0000 mgr.x (mgr.24118) 1547 : cluster [DBG] pgmap v2295: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.320 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:57 smithi055 bash[9296]: cluster 2020-09-08T16:18:56.284675+0000 mgr.x (mgr.24118) 1548 : cluster [DBG] pgmap v2296: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.321 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:58 smithi055 bash[9296]: cluster 2020-09-08T16:18:57.033753+0000 osd.6 (osd.6) 135 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.321 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:58 smithi055 bash[9296]: cluster 2020-09-08T16:18:57.033886+0000 osd.6 (osd.6) 136 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.321 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:58 smithi055 bash[9296]: cluster 2020-09-08T16:18:57.362340+0000 mon.a (mon.0) 2455 : cluster [DBG] osdmap e833: 12 total, 12 up, 12 in 2020-09-08T16:27:02.321 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:18:59 smithi055 bash[9296]: cluster 2020-09-08T16:18:58.285394+0000 mgr.x (mgr.24118) 1549 : cluster [DBG] pgmap v2298: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.321 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:01 smithi055 bash[9296]: cluster 2020-09-08T16:19:00.286031+0000 mgr.x (mgr.24118) 1550 : cluster [DBG] pgmap v2299: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.322 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:02 smithi055 bash[9296]: cluster 2020-09-08T16:19:01.547200+0000 osd.11 (osd.11) 150 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.322 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:02 smithi055 bash[9296]: cluster 2020-09-08T16:19:01.547357+0000 osd.11 (osd.11) 151 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.322 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:03 smithi055 bash[9296]: cluster 2020-09-08T16:19:02.231839+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e834: 12 total, 12 up, 12 in 2020-09-08T16:27:02.322 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:03 smithi055 bash[9296]: cluster 2020-09-08T16:19:02.286716+0000 mgr.x (mgr.24118) 1551 : cluster [DBG] pgmap v2301: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.322 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:05 smithi055 bash[9296]: cluster 2020-09-08T16:19:04.287459+0000 mgr.x (mgr.24118) 1552 : cluster [DBG] pgmap v2302: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.323 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:06 smithi055 bash[9296]: cluster 2020-09-08T16:19:05.101483+0000 osd.4 (osd.4) 95 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.323 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:06 smithi055 bash[9296]: cluster 2020-09-08T16:19:05.101600+0000 osd.4 (osd.4) 96 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.323 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:06 smithi055 bash[9296]: cluster 2020-09-08T16:19:05.351339+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e835: 12 total, 12 up, 12 in 2020-09-08T16:27:02.323 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:07 smithi055 bash[9296]: cluster 2020-09-08T16:19:06.288110+0000 mgr.x (mgr.24118) 1553 : cluster [DBG] pgmap v2304: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.323 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:09 smithi055 bash[9296]: cluster 2020-09-08T16:19:08.288845+0000 mgr.x (mgr.24118) 1554 : cluster [DBG] pgmap v2305: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.324 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:11 smithi055 bash[9296]: cluster 2020-09-08T16:19:10.289470+0000 mgr.x (mgr.24118) 1555 : cluster [DBG] pgmap v2306: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.324 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:13 smithi055 bash[9296]: cluster 2020-09-08T16:19:12.290155+0000 mgr.x (mgr.24118) 1556 : cluster [DBG] pgmap v2307: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.324 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:15 smithi055 bash[9296]: cluster 2020-09-08T16:19:14.290872+0000 mgr.x (mgr.24118) 1557 : cluster [DBG] pgmap v2308: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.324 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:15 smithi055 bash[9296]: cluster 2020-09-08T16:19:14.572020+0000 osd.9 (osd.9) 134 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.324 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:15 smithi055 bash[9296]: cluster 2020-09-08T16:19:14.572143+0000 osd.9 (osd.9) 135 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.325 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:16 smithi055 bash[9296]: cluster 2020-09-08T16:19:15.358643+0000 mon.a (mon.0) 2458 : cluster [DBG] osdmap e836: 12 total, 12 up, 12 in 2020-09-08T16:27:02.325 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:17 smithi055 bash[9296]: cluster 2020-09-08T16:19:16.291530+0000 mgr.x (mgr.24118) 1558 : cluster [DBG] pgmap v2310: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.325 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:18 smithi055 bash[9296]: cluster 2020-09-08T16:19:17.194214+0000 osd.2 (osd.2) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.325 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:18 smithi055 bash[9296]: cluster 2020-09-08T16:19:17.194333+0000 osd.2 (osd.2) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.325 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:18 smithi055 bash[9296]: cluster 2020-09-08T16:19:17.403220+0000 mon.a (mon.0) 2459 : cluster [DBG] osdmap e837: 12 total, 12 up, 12 in 2020-09-08T16:27:02.326 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:19 smithi055 bash[9296]: cluster 2020-09-08T16:19:18.292239+0000 mgr.x (mgr.24118) 1559 : cluster [DBG] pgmap v2312: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.326 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:21 smithi055 bash[9296]: cluster 2020-09-08T16:19:20.293036+0000 mgr.x (mgr.24118) 1560 : cluster [DBG] pgmap v2313: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.326 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:23 smithi055 bash[9296]: cluster 2020-09-08T16:19:22.293768+0000 mgr.x (mgr.24118) 1561 : cluster [DBG] pgmap v2314: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.326 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:25 smithi055 bash[9296]: cluster 2020-09-08T16:19:24.294501+0000 mgr.x (mgr.24118) 1562 : cluster [DBG] pgmap v2315: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.326 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:25 smithi055 bash[9296]: cluster 2020-09-08T16:19:24.556230+0000 osd.7 (osd.7) 186 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.327 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:25 smithi055 bash[9296]: cluster 2020-09-08T16:19:24.556355+0000 osd.7 (osd.7) 187 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.327 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:25 smithi055 bash[9296]: cluster 2020-09-08T16:19:24.633342+0000 osd.0 (osd.0) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.327 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:25 smithi055 bash[9296]: cluster 2020-09-08T16:19:24.633471+0000 osd.0 (osd.0) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.327 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:26 smithi055 bash[9296]: cluster 2020-09-08T16:19:25.361020+0000 mon.a (mon.0) 2460 : cluster [DBG] osdmap e838: 12 total, 12 up, 12 in 2020-09-08T16:27:02.327 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:27 smithi055 bash[9296]: cluster 2020-09-08T16:19:26.295149+0000 mgr.x (mgr.24118) 1563 : cluster [DBG] pgmap v2317: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.328 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:28 smithi055 bash[9296]: audit 2020-09-08T16:19:27.397122+0000 mon.b (mon.1) 137 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.328 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:28 smithi055 bash[9296]: audit 2020-09-08T16:19:27.398626+0000 mon.a (mon.0) 2461 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.328 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:28 smithi055 bash[9296]: audit 2020-09-08T16:19:27.405368+0000 mon.a (mon.0) 2462 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.328 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:29 smithi055 bash[9296]: cluster 2020-09-08T16:19:28.295846+0000 mgr.x (mgr.24118) 1564 : cluster [DBG] pgmap v2318: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.328 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:31 smithi055 bash[9296]: cluster 2020-09-08T16:19:30.296510+0000 mgr.x (mgr.24118) 1565 : cluster [DBG] pgmap v2319: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.329 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:33 smithi055 bash[9296]: cluster 2020-09-08T16:19:32.297174+0000 mgr.x (mgr.24118) 1566 : cluster [DBG] pgmap v2320: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.329 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:35 smithi055 bash[9296]: cluster 2020-09-08T16:19:34.219237+0000 osd.5 (osd.5) 152 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.329 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:35 smithi055 bash[9296]: cluster 2020-09-08T16:19:34.219366+0000 osd.5 (osd.5) 153 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.329 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:35 smithi055 bash[9296]: cluster 2020-09-08T16:19:34.297911+0000 mgr.x (mgr.24118) 1567 : cluster [DBG] pgmap v2321: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.329 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:35 smithi055 bash[9296]: cluster 2020-09-08T16:19:34.371433+0000 mon.a (mon.0) 2463 : cluster [DBG] osdmap e839: 12 total, 12 up, 12 in 2020-09-08T16:27:02.330 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:37 smithi055 bash[9296]: cluster 2020-09-08T16:19:36.298640+0000 mgr.x (mgr.24118) 1568 : cluster [DBG] pgmap v2323: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.330 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:37 smithi055 bash[9296]: cluster 2020-09-08T16:19:36.334119+0000 osd.10 (osd.10) 131 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.330 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:37 smithi055 bash[9296]: cluster 2020-09-08T16:19:36.334233+0000 osd.10 (osd.10) 132 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.330 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:37 smithi055 bash[9296]: cluster 2020-09-08T16:19:36.388767+0000 mon.a (mon.0) 2464 : cluster [DBG] osdmap e840: 12 total, 12 up, 12 in 2020-09-08T16:27:02.330 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:38 smithi055 bash[9296]: cluster 2020-09-08T16:19:37.437449+0000 osd.3 (osd.3) 155 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.331 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:38 smithi055 bash[9296]: cluster 2020-09-08T16:19:37.437572+0000 osd.3 (osd.3) 156 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.331 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:39 smithi055 bash[9296]: cluster 2020-09-08T16:19:38.299377+0000 mgr.x (mgr.24118) 1569 : cluster [DBG] pgmap v2325: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.331 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:39 smithi055 bash[9296]: cluster 2020-09-08T16:19:38.402072+0000 mon.a (mon.0) 2465 : cluster [DBG] osdmap e841: 12 total, 12 up, 12 in 2020-09-08T16:27:02.331 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:41 smithi055 bash[9296]: cluster 2020-09-08T16:19:40.299822+0000 mgr.x (mgr.24118) 1570 : cluster [DBG] pgmap v2327: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.331 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:43 smithi055 bash[9296]: cluster 2020-09-08T16:19:42.300421+0000 mgr.x (mgr.24118) 1571 : cluster [DBG] pgmap v2328: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.332 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:45 smithi055 bash[9296]: cluster 2020-09-08T16:19:44.301177+0000 mgr.x (mgr.24118) 1572 : cluster [DBG] pgmap v2329: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.332 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:46 smithi055 bash[9296]: cluster 2020-09-08T16:19:44.761903+0000 osd.8 (osd.8) 126 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.332 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:46 smithi055 bash[9296]: cluster 2020-09-08T16:19:44.762032+0000 osd.8 (osd.8) 127 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.332 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:46 smithi055 bash[9296]: cluster 2020-09-08T16:19:45.377400+0000 mon.a (mon.0) 2466 : cluster [DBG] osdmap e842: 12 total, 12 up, 12 in 2020-09-08T16:27:02.332 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:47 smithi055 bash[9296]: cluster 2020-09-08T16:19:46.301791+0000 mgr.x (mgr.24118) 1573 : cluster [DBG] pgmap v2331: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.333 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:49 smithi055 bash[9296]: cluster 2020-09-08T16:19:48.302517+0000 mgr.x (mgr.24118) 1574 : cluster [DBG] pgmap v2332: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.333 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:51 smithi055 bash[9296]: cluster 2020-09-08T16:19:50.303213+0000 mgr.x (mgr.24118) 1575 : cluster [DBG] pgmap v2333: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.333 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:52 smithi055 bash[9296]: cluster 2020-09-08T16:19:51.829178+0000 osd.1 (osd.1) 172 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.333 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:52 smithi055 bash[9296]: cluster 2020-09-08T16:19:51.829307+0000 osd.1 (osd.1) 173 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.333 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:53 smithi055 bash[9296]: cluster 2020-09-08T16:19:52.237451+0000 mon.a (mon.0) 2467 : cluster [DBG] osdmap e843: 12 total, 12 up, 12 in 2020-09-08T16:27:02.334 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:53 smithi055 bash[9296]: cluster 2020-09-08T16:19:52.303900+0000 mgr.x (mgr.24118) 1576 : cluster [DBG] pgmap v2335: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.334 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:55 smithi055 bash[9296]: cluster 2020-09-08T16:19:54.304737+0000 mgr.x (mgr.24118) 1577 : cluster [DBG] pgmap v2336: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.334 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:57 smithi055 bash[9296]: cluster 2020-09-08T16:19:56.305456+0000 mgr.x (mgr.24118) 1578 : cluster [DBG] pgmap v2337: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.334 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:58 smithi055 bash[9296]: cluster 2020-09-08T16:19:57.050038+0000 osd.6 (osd.6) 137 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.334 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:58 smithi055 bash[9296]: cluster 2020-09-08T16:19:57.050184+0000 osd.6 (osd.6) 138 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.335 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:58 smithi055 bash[9296]: cluster 2020-09-08T16:19:57.374960+0000 mon.a (mon.0) 2468 : cluster [DBG] osdmap e844: 12 total, 12 up, 12 in 2020-09-08T16:27:02.335 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:19:59 smithi055 bash[9296]: cluster 2020-09-08T16:19:58.306249+0000 mgr.x (mgr.24118) 1579 : cluster [DBG] pgmap v2339: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.335 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:01 smithi055 bash[9296]: cluster 2020-09-08T16:20:00.307004+0000 mgr.x (mgr.24118) 1580 : cluster [DBG] pgmap v2340: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.335 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:03 smithi055 bash[9296]: cluster 2020-09-08T16:20:02.307670+0000 mgr.x (mgr.24118) 1581 : cluster [DBG] pgmap v2341: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.335 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:03 smithi055 bash[9296]: cluster 2020-09-08T16:20:02.428676+0000 osd.11 (osd.11) 152 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.336 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:03 smithi055 bash[9296]: cluster 2020-09-08T16:20:02.428805+0000 osd.11 (osd.11) 153 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.336 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:04 smithi055 bash[9296]: cluster 2020-09-08T16:20:03.369560+0000 mon.a (mon.0) 2469 : cluster [DBG] osdmap e845: 12 total, 12 up, 12 in 2020-09-08T16:27:02.336 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:05 smithi055 bash[9296]: cluster 2020-09-08T16:20:04.308304+0000 mgr.x (mgr.24118) 1582 : cluster [DBG] pgmap v2343: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.336 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:07 smithi055 bash[9296]: cluster 2020-09-08T16:20:05.902905+0000 osd.4 (osd.4) 97 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.337 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:07 smithi055 bash[9296]: cluster 2020-09-08T16:20:05.903036+0000 osd.4 (osd.4) 98 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.337 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:07 smithi055 bash[9296]: cluster 2020-09-08T16:20:06.309002+0000 mgr.x (mgr.24118) 1583 : cluster [DBG] pgmap v2344: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.337 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:07 smithi055 bash[9296]: cluster 2020-09-08T16:20:06.413136+0000 mon.a (mon.0) 2470 : cluster [DBG] osdmap e846: 12 total, 12 up, 12 in 2020-09-08T16:27:02.337 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:09 smithi055 bash[9296]: cluster 2020-09-08T16:20:08.309839+0000 mgr.x (mgr.24118) 1584 : cluster [DBG] pgmap v2346: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.337 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:11 smithi055 bash[9296]: cluster 2020-09-08T16:20:10.310566+0000 mgr.x (mgr.24118) 1585 : cluster [DBG] pgmap v2347: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.338 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:13 smithi055 bash[9296]: cluster 2020-09-08T16:20:12.311236+0000 mgr.x (mgr.24118) 1586 : cluster [DBG] pgmap v2348: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.338 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:15 smithi055 bash[9296]: cluster 2020-09-08T16:20:14.311961+0000 mgr.x (mgr.24118) 1587 : cluster [DBG] pgmap v2349: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.338 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:16 smithi055 bash[9296]: cluster 2020-09-08T16:20:15.478981+0000 osd.9 (osd.9) 136 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.338 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:16 smithi055 bash[9296]: cluster 2020-09-08T16:20:15.479101+0000 osd.9 (osd.9) 137 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.339 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:17 smithi055 bash[9296]: cluster 2020-09-08T16:20:16.312636+0000 mgr.x (mgr.24118) 1588 : cluster [DBG] pgmap v2350: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.339 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:17 smithi055 bash[9296]: cluster 2020-09-08T16:20:16.393448+0000 mon.a (mon.0) 2471 : cluster [DBG] osdmap e847: 12 total, 12 up, 12 in 2020-09-08T16:27:02.339 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:18 smithi055 bash[9296]: cluster 2020-09-08T16:20:17.306375+0000 osd.2 (osd.2) 132 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.339 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:18 smithi055 bash[9296]: cluster 2020-09-08T16:20:17.306485+0000 osd.2 (osd.2) 133 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.339 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:18 smithi055 bash[9296]: cluster 2020-09-08T16:20:17.402201+0000 mon.a (mon.0) 2472 : cluster [DBG] osdmap e848: 12 total, 12 up, 12 in 2020-09-08T16:27:02.340 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:19 smithi055 bash[9296]: cluster 2020-09-08T16:20:18.313348+0000 mgr.x (mgr.24118) 1589 : cluster [DBG] pgmap v2353: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.340 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:21 smithi055 bash[9296]: cluster 2020-09-08T16:20:20.313853+0000 mgr.x (mgr.24118) 1590 : cluster [DBG] pgmap v2354: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.340 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:23 smithi055 bash[9296]: cluster 2020-09-08T16:20:22.314502+0000 mgr.x (mgr.24118) 1591 : cluster [DBG] pgmap v2355: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.340 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:25 smithi055 bash[9296]: cluster 2020-09-08T16:20:24.315223+0000 mgr.x (mgr.24118) 1592 : cluster [DBG] pgmap v2356: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.340 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:25 smithi055 bash[9296]: cluster 2020-09-08T16:20:24.707385+0000 osd.7 (osd.7) 188 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.341 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:25 smithi055 bash[9296]: cluster 2020-09-08T16:20:24.707509+0000 osd.7 (osd.7) 189 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.341 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:26 smithi055 bash[9296]: cluster 2020-09-08T16:20:25.382370+0000 mon.a (mon.0) 2473 : cluster [DBG] osdmap e849: 12 total, 12 up, 12 in 2020-09-08T16:27:02.341 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:26 smithi055 bash[9296]: cluster 2020-09-08T16:20:25.588412+0000 osd.0 (osd.0) 148 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.341 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:26 smithi055 bash[9296]: cluster 2020-09-08T16:20:25.588570+0000 osd.0 (osd.0) 149 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.341 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:27 smithi055 bash[9296]: cluster 2020-09-08T16:20:26.315850+0000 mgr.x (mgr.24118) 1593 : cluster [DBG] pgmap v2358: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.342 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:27 smithi055 bash[9296]: cluster 2020-09-08T16:20:26.414678+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e850: 12 total, 12 up, 12 in 2020-09-08T16:27:02.342 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:29 smithi055 bash[9296]: cluster 2020-09-08T16:20:28.316580+0000 mgr.x (mgr.24118) 1594 : cluster [DBG] pgmap v2360: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.342 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:31 smithi055 bash[9296]: cluster 2020-09-08T16:20:30.317240+0000 mgr.x (mgr.24118) 1595 : cluster [DBG] pgmap v2361: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.342 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:33 smithi055 bash[9296]: cluster 2020-09-08T16:20:32.317937+0000 mgr.x (mgr.24118) 1596 : cluster [DBG] pgmap v2362: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.342 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:35 smithi055 bash[9296]: cluster 2020-09-08T16:20:34.318671+0000 mgr.x (mgr.24118) 1597 : cluster [DBG] pgmap v2363: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.343 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:36 smithi055 bash[9296]: cluster 2020-09-08T16:20:35.020359+0000 osd.5 (osd.5) 154 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.343 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:36 smithi055 bash[9296]: cluster 2020-09-08T16:20:35.020517+0000 osd.5 (osd.5) 155 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.343 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:36 smithi055 bash[9296]: cluster 2020-09-08T16:20:35.391055+0000 mon.a (mon.0) 2475 : cluster [DBG] osdmap e851: 12 total, 12 up, 12 in 2020-09-08T16:27:02.343 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:37 smithi055 bash[9296]: cluster 2020-09-08T16:20:36.319398+0000 mgr.x (mgr.24118) 1598 : cluster [DBG] pgmap v2365: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.343 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:37 smithi055 bash[9296]: cluster 2020-09-08T16:20:37.088845+0000 osd.10 (osd.10) 133 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.344 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:37 smithi055 bash[9296]: cluster 2020-09-08T16:20:37.088962+0000 osd.10 (osd.10) 134 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.344 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:38 smithi055 bash[9296]: cluster 2020-09-08T16:20:37.405413+0000 mon.a (mon.0) 2476 : cluster [DBG] osdmap e852: 12 total, 12 up, 12 in 2020-09-08T16:27:02.344 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:38 smithi055 bash[9296]: cluster 2020-09-08T16:20:37.452514+0000 osd.3 (osd.3) 157 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.344 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:38 smithi055 bash[9296]: cluster 2020-09-08T16:20:37.452628+0000 osd.3 (osd.3) 158 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.344 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:39 smithi055 bash[9296]: cluster 2020-09-08T16:20:38.320060+0000 mgr.x (mgr.24118) 1599 : cluster [DBG] pgmap v2367: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.345 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:39 smithi055 bash[9296]: cluster 2020-09-08T16:20:38.427057+0000 mon.a (mon.0) 2477 : cluster [DBG] osdmap e853: 12 total, 12 up, 12 in 2020-09-08T16:27:02.345 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:41 smithi055 bash[9296]: cluster 2020-09-08T16:20:40.320532+0000 mgr.x (mgr.24118) 1600 : cluster [DBG] pgmap v2369: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.345 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:43 smithi055 bash[9296]: cluster 2020-09-08T16:20:42.321174+0000 mgr.x (mgr.24118) 1601 : cluster [DBG] pgmap v2370: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.345 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:45 smithi055 bash[9296]: cluster 2020-09-08T16:20:44.321907+0000 mgr.x (mgr.24118) 1602 : cluster [DBG] pgmap v2371: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.345 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:46 smithi055 bash[9296]: cluster 2020-09-08T16:20:45.563606+0000 osd.8 (osd.8) 128 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.346 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:46 smithi055 bash[9296]: cluster 2020-09-08T16:20:45.563735+0000 osd.8 (osd.8) 129 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.346 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:47 smithi055 bash[9296]: cluster 2020-09-08T16:20:46.322795+0000 mgr.x (mgr.24118) 1603 : cluster [DBG] pgmap v2372: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.346 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:47 smithi055 bash[9296]: cluster 2020-09-08T16:20:46.404936+0000 mon.a (mon.0) 2478 : cluster [DBG] osdmap e854: 12 total, 12 up, 12 in 2020-09-08T16:27:02.346 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:49 smithi055 bash[9296]: cluster 2020-09-08T16:20:48.323584+0000 mgr.x (mgr.24118) 1604 : cluster [DBG] pgmap v2374: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.346 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:51 smithi055 bash[9296]: cluster 2020-09-08T16:20:50.324248+0000 mgr.x (mgr.24118) 1605 : cluster [DBG] pgmap v2375: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.347 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:53 smithi055 bash[9296]: cluster 2020-09-08T16:20:52.088960+0000 osd.1 (osd.1) 174 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.347 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:53 smithi055 bash[9296]: cluster 2020-09-08T16:20:52.089112+0000 osd.1 (osd.1) 175 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.347 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:53 smithi055 bash[9296]: cluster 2020-09-08T16:20:52.259962+0000 mon.a (mon.0) 2479 : cluster [DBG] osdmap e855: 12 total, 12 up, 12 in 2020-09-08T16:27:02.347 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:53 smithi055 bash[9296]: cluster 2020-09-08T16:20:52.324907+0000 mgr.x (mgr.24118) 1606 : cluster [DBG] pgmap v2377: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.347 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:55 smithi055 bash[9296]: cluster 2020-09-08T16:20:54.325657+0000 mgr.x (mgr.24118) 1607 : cluster [DBG] pgmap v2378: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.348 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:57 smithi055 bash[9296]: cluster 2020-09-08T16:20:56.326288+0000 mgr.x (mgr.24118) 1608 : cluster [DBG] pgmap v2379: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.348 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:59 smithi055 bash[9296]: cluster 2020-09-08T16:20:57.812662+0000 osd.6 (osd.6) 139 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.348 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:59 smithi055 bash[9296]: cluster 2020-09-08T16:20:57.812796+0000 osd.6 (osd.6) 140 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.348 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:59 smithi055 bash[9296]: cluster 2020-09-08T16:20:58.327078+0000 mgr.x (mgr.24118) 1609 : cluster [DBG] pgmap v2380: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.349 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:20:59 smithi055 bash[9296]: cluster 2020-09-08T16:20:58.404772+0000 mon.a (mon.0) 2480 : cluster [DBG] osdmap e856: 12 total, 12 up, 12 in 2020-09-08T16:27:02.349 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:01 smithi055 bash[9296]: cluster 2020-09-08T16:21:00.327722+0000 mgr.x (mgr.24118) 1610 : cluster [DBG] pgmap v2382: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.349 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:03 smithi055 bash[9296]: cluster 2020-09-08T16:21:02.328378+0000 mgr.x (mgr.24118) 1611 : cluster [DBG] pgmap v2383: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.349 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:04 smithi055 bash[9296]: cluster 2020-09-08T16:21:03.333742+0000 osd.11 (osd.11) 154 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.349 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:04 smithi055 bash[9296]: cluster 2020-09-08T16:21:03.333871+0000 osd.11 (osd.11) 155 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.350 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:04 smithi055 bash[9296]: cluster 2020-09-08T16:21:03.400773+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e857: 12 total, 12 up, 12 in 2020-09-08T16:27:02.350 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:05 smithi055 bash[9296]: cluster 2020-09-08T16:21:04.329012+0000 mgr.x (mgr.24118) 1612 : cluster [DBG] pgmap v2385: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.350 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:07 smithi055 bash[9296]: cluster 2020-09-08T16:21:06.106338+0000 osd.4 (osd.4) 99 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.350 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:07 smithi055 bash[9296]: cluster 2020-09-08T16:21:06.106465+0000 osd.4 (osd.4) 100 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.350 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:07 smithi055 bash[9296]: cluster 2020-09-08T16:21:06.329766+0000 mgr.x (mgr.24118) 1613 : cluster [DBG] pgmap v2386: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.351 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:07 smithi055 bash[9296]: cluster 2020-09-08T16:21:06.421076+0000 mon.a (mon.0) 2482 : cluster [DBG] osdmap e858: 12 total, 12 up, 12 in 2020-09-08T16:27:02.351 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:09 smithi055 bash[9296]: cluster 2020-09-08T16:21:08.330615+0000 mgr.x (mgr.24118) 1614 : cluster [DBG] pgmap v2388: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.351 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:11 smithi055 bash[9296]: cluster 2020-09-08T16:21:10.331273+0000 mgr.x (mgr.24118) 1615 : cluster [DBG] pgmap v2389: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.351 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:13 smithi055 bash[9296]: cluster 2020-09-08T16:21:12.331978+0000 mgr.x (mgr.24118) 1616 : cluster [DBG] pgmap v2390: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.351 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:15 smithi055 bash[9296]: cluster 2020-09-08T16:21:14.332692+0000 mgr.x (mgr.24118) 1617 : cluster [DBG] pgmap v2391: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.352 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:17 smithi055 bash[9296]: cluster 2020-09-08T16:21:16.333447+0000 mgr.x (mgr.24118) 1618 : cluster [DBG] pgmap v2392: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.352 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:17 smithi055 bash[9296]: cluster 2020-09-08T16:21:16.362679+0000 osd.9 (osd.9) 138 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.352 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:17 smithi055 bash[9296]: cluster 2020-09-08T16:21:16.362811+0000 osd.9 (osd.9) 139 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.352 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:17 smithi055 bash[9296]: cluster 2020-09-08T16:21:16.409000+0000 mon.a (mon.0) 2483 : cluster [DBG] osdmap e859: 12 total, 12 up, 12 in 2020-09-08T16:27:02.352 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:18 smithi055 bash[9296]: cluster 2020-09-08T16:21:17.547440+0000 osd.2 (osd.2) 134 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.353 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:18 smithi055 bash[9296]: cluster 2020-09-08T16:21:17.547614+0000 osd.2 (osd.2) 135 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.353 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:19 smithi055 bash[9296]: cluster 2020-09-08T16:21:18.334352+0000 mgr.x (mgr.24118) 1619 : cluster [DBG] pgmap v2394: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.353 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:19 smithi055 bash[9296]: cluster 2020-09-08T16:21:18.423887+0000 mon.a (mon.0) 2484 : cluster [DBG] osdmap e860: 12 total, 12 up, 12 in 2020-09-08T16:27:02.353 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:21 smithi055 bash[9296]: cluster 2020-09-08T16:21:20.335075+0000 mgr.x (mgr.24118) 1620 : cluster [DBG] pgmap v2396: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.353 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:23 smithi055 bash[9296]: cluster 2020-09-08T16:21:22.335851+0000 mgr.x (mgr.24118) 1621 : cluster [DBG] pgmap v2397: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.354 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:25 smithi055 bash[9296]: cluster 2020-09-08T16:21:24.336633+0000 mgr.x (mgr.24118) 1622 : cluster [DBG] pgmap v2398: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.354 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:26 smithi055 bash[9296]: cluster 2020-09-08T16:21:25.673730+0000 osd.7 (osd.7) 190 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.354 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:26 smithi055 bash[9296]: cluster 2020-09-08T16:21:25.673860+0000 osd.7 (osd.7) 191 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.354 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:26 smithi055 bash[9296]: cluster 2020-09-08T16:21:25.674272+0000 osd.0 (osd.0) 150 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.354 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:26 smithi055 bash[9296]: cluster 2020-09-08T16:21:25.674416+0000 osd.0 (osd.0) 151 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.355 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:27 smithi055 bash[9296]: cluster 2020-09-08T16:21:26.337306+0000 mgr.x (mgr.24118) 1623 : cluster [DBG] pgmap v2399: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.355 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:27 smithi055 bash[9296]: cluster 2020-09-08T16:21:26.418317+0000 mon.a (mon.0) 2485 : cluster [DBG] osdmap e861: 12 total, 12 up, 12 in 2020-09-08T16:27:02.355 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:29 smithi055 bash[9296]: cluster 2020-09-08T16:21:28.338012+0000 mgr.x (mgr.24118) 1624 : cluster [DBG] pgmap v2401: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.355 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:31 smithi055 bash[9296]: cluster 2020-09-08T16:21:30.338716+0000 mgr.x (mgr.24118) 1625 : cluster [DBG] pgmap v2402: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.356 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:33 smithi055 bash[9296]: cluster 2020-09-08T16:21:32.339416+0000 mgr.x (mgr.24118) 1626 : cluster [DBG] pgmap v2403: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.356 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:35 smithi055 bash[9296]: cluster 2020-09-08T16:21:34.340128+0000 mgr.x (mgr.24118) 1627 : cluster [DBG] pgmap v2404: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.356 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:36 smithi055 bash[9296]: cluster 2020-09-08T16:21:35.509269+0000 osd.5 (osd.5) 156 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.356 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:36 smithi055 bash[9296]: cluster 2020-09-08T16:21:35.509400+0000 osd.5 (osd.5) 157 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.357 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:37 smithi055 bash[9296]: cluster 2020-09-08T16:21:36.340795+0000 mgr.x (mgr.24118) 1628 : cluster [DBG] pgmap v2405: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.357 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:37 smithi055 bash[9296]: cluster 2020-09-08T16:21:36.420966+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e862: 12 total, 12 up, 12 in 2020-09-08T16:27:02.357 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:38 smithi055 bash[9296]: cluster 2020-09-08T16:21:37.638448+0000 osd.3 (osd.3) 159 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.357 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:38 smithi055 bash[9296]: cluster 2020-09-08T16:21:37.638607+0000 osd.3 (osd.3) 160 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.357 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:38 smithi055 bash[9296]: cluster 2020-09-08T16:21:37.868311+0000 osd.10 (osd.10) 135 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.358 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:38 smithi055 bash[9296]: cluster 2020-09-08T16:21:37.868434+0000 osd.10 (osd.10) 136 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.358 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:39 smithi055 bash[9296]: cluster 2020-09-08T16:21:38.341523+0000 mgr.x (mgr.24118) 1629 : cluster [DBG] pgmap v2407: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.358 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:39 smithi055 bash[9296]: cluster 2020-09-08T16:21:38.432367+0000 mon.a (mon.0) 2487 : cluster [DBG] osdmap e863: 12 total, 12 up, 12 in 2020-09-08T16:27:02.358 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:41 smithi055 bash[9296]: cluster 2020-09-08T16:21:40.342275+0000 mgr.x (mgr.24118) 1630 : cluster [DBG] pgmap v2409: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.358 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:43 smithi055 bash[9296]: cluster 2020-09-08T16:21:42.343020+0000 mgr.x (mgr.24118) 1631 : cluster [DBG] pgmap v2410: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.359 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:45 smithi055 bash[9296]: cluster 2020-09-08T16:21:44.343760+0000 mgr.x (mgr.24118) 1632 : cluster [DBG] pgmap v2411: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.359 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:47 smithi055 bash[9296]: cluster 2020-09-08T16:21:45.745329+0000 osd.8 (osd.8) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.359 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:47 smithi055 bash[9296]: cluster 2020-09-08T16:21:45.745456+0000 osd.8 (osd.8) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.359 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:47 smithi055 bash[9296]: cluster 2020-09-08T16:21:46.344503+0000 mgr.x (mgr.24118) 1633 : cluster [DBG] pgmap v2412: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.359 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:47 smithi055 bash[9296]: cluster 2020-09-08T16:21:46.423352+0000 mon.a (mon.0) 2488 : cluster [DBG] osdmap e864: 12 total, 12 up, 12 in 2020-09-08T16:27:02.360 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:49 smithi055 bash[9296]: cluster 2020-09-08T16:21:48.345450+0000 mgr.x (mgr.24118) 1634 : cluster [DBG] pgmap v2414: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.360 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:51 smithi055 bash[9296]: cluster 2020-09-08T16:21:50.346231+0000 mgr.x (mgr.24118) 1635 : cluster [DBG] pgmap v2415: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.360 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:53 smithi055 bash[9296]: cluster 2020-09-08T16:21:52.346930+0000 mgr.x (mgr.24118) 1636 : cluster [DBG] pgmap v2416: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.360 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:53 smithi055 bash[9296]: cluster 2020-09-08T16:21:52.538202+0000 osd.1 (osd.1) 176 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.360 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:53 smithi055 bash[9296]: cluster 2020-09-08T16:21:52.538339+0000 osd.1 (osd.1) 177 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.361 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:54 smithi055 bash[9296]: cluster 2020-09-08T16:21:53.419636+0000 mon.a (mon.0) 2489 : cluster [DBG] osdmap e865: 12 total, 12 up, 12 in 2020-09-08T16:27:02.361 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:55 smithi055 bash[9296]: cluster 2020-09-08T16:21:54.347582+0000 mgr.x (mgr.24118) 1637 : cluster [DBG] pgmap v2418: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.361 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:57 smithi055 bash[9296]: cluster 2020-09-08T16:21:56.348350+0000 mgr.x (mgr.24118) 1638 : cluster [DBG] pgmap v2419: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.361 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:59 smithi055 bash[9296]: cluster 2020-09-08T16:21:57.980972+0000 osd.6 (osd.6) 141 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.361 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:59 smithi055 bash[9296]: cluster 2020-09-08T16:21:57.981104+0000 osd.6 (osd.6) 142 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.362 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:59 smithi055 bash[9296]: cluster 2020-09-08T16:21:58.349119+0000 mgr.x (mgr.24118) 1639 : cluster [DBG] pgmap v2420: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.362 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:21:59 smithi055 bash[9296]: cluster 2020-09-08T16:21:58.489201+0000 mon.a (mon.0) 2490 : cluster [DBG] osdmap e866: 12 total, 12 up, 12 in 2020-09-08T16:27:02.362 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:01 smithi055 bash[9296]: cluster 2020-09-08T16:22:00.349788+0000 mgr.x (mgr.24118) 1640 : cluster [DBG] pgmap v2422: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.362 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:03 smithi055 bash[9296]: cluster 2020-09-08T16:22:02.350450+0000 mgr.x (mgr.24118) 1641 : cluster [DBG] pgmap v2423: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.362 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:04 smithi055 bash[9296]: cluster 2020-09-08T16:22:03.859539+0000 osd.11 (osd.11) 156 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.363 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:04 smithi055 bash[9296]: cluster 2020-09-08T16:22:03.859668+0000 osd.11 (osd.11) 157 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.363 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:05 smithi055 bash[9296]: cluster 2020-09-08T16:22:04.351125+0000 mgr.x (mgr.24118) 1642 : cluster [DBG] pgmap v2424: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.363 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:05 smithi055 bash[9296]: cluster 2020-09-08T16:22:04.426415+0000 mon.a (mon.0) 2491 : cluster [DBG] osdmap e867: 12 total, 12 up, 12 in 2020-09-08T16:27:02.363 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:07 smithi055 bash[9296]: cluster 2020-09-08T16:22:06.292113+0000 osd.4 (osd.4) 101 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.363 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:07 smithi055 bash[9296]: cluster 2020-09-08T16:22:06.292235+0000 osd.4 (osd.4) 102 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.364 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:07 smithi055 bash[9296]: cluster 2020-09-08T16:22:06.351776+0000 mgr.x (mgr.24118) 1643 : cluster [DBG] pgmap v2426: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.364 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:07 smithi055 bash[9296]: cluster 2020-09-08T16:22:06.437707+0000 mon.a (mon.0) 2492 : cluster [DBG] osdmap e868: 12 total, 12 up, 12 in 2020-09-08T16:27:02.364 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:09 smithi055 bash[9296]: cluster 2020-09-08T16:22:08.352672+0000 mgr.x (mgr.24118) 1644 : cluster [DBG] pgmap v2428: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.364 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:11 smithi055 bash[9296]: cluster 2020-09-08T16:22:10.353519+0000 mgr.x (mgr.24118) 1645 : cluster [DBG] pgmap v2429: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.364 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:13 smithi055 bash[9296]: cluster 2020-09-08T16:22:12.354224+0000 mgr.x (mgr.24118) 1646 : cluster [DBG] pgmap v2430: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.365 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:15 smithi055 bash[9296]: cluster 2020-09-08T16:22:14.354913+0000 mgr.x (mgr.24118) 1647 : cluster [DBG] pgmap v2431: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.365 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:17 smithi055 bash[9296]: cluster 2020-09-08T16:22:16.355583+0000 mgr.x (mgr.24118) 1648 : cluster [DBG] pgmap v2432: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.365 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:17 smithi055 bash[9296]: cluster 2020-09-08T16:22:16.660205+0000 osd.9 (osd.9) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.365 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:17 smithi055 bash[9296]: cluster 2020-09-08T16:22:16.660363+0000 osd.9 (osd.9) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.365 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:18 smithi055 bash[9296]: cluster 2020-09-08T16:22:17.427768+0000 mon.a (mon.0) 2493 : cluster [DBG] osdmap e869: 12 total, 12 up, 12 in 2020-09-08T16:27:02.366 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:18 smithi055 bash[9296]: cluster 2020-09-08T16:22:17.988554+0000 osd.2 (osd.2) 136 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.366 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:18 smithi055 bash[9296]: cluster 2020-09-08T16:22:17.988686+0000 osd.2 (osd.2) 137 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.366 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:19 smithi055 bash[9296]: cluster 2020-09-08T16:22:18.356281+0000 mgr.x (mgr.24118) 1649 : cluster [DBG] pgmap v2434: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.366 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:19 smithi055 bash[9296]: cluster 2020-09-08T16:22:18.446223+0000 mon.a (mon.0) 2494 : cluster [DBG] osdmap e870: 12 total, 12 up, 12 in 2020-09-08T16:27:02.366 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:21 smithi055 bash[9296]: cluster 2020-09-08T16:22:20.356945+0000 mgr.x (mgr.24118) 1650 : cluster [DBG] pgmap v2436: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.367 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:23 smithi055 bash[9296]: cluster 2020-09-08T16:22:22.357656+0000 mgr.x (mgr.24118) 1651 : cluster [DBG] pgmap v2437: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.367 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:25 smithi055 bash[9296]: cluster 2020-09-08T16:22:24.358372+0000 mgr.x (mgr.24118) 1652 : cluster [DBG] pgmap v2438: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.367 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:27 smithi055 bash[9296]: cluster 2020-09-08T16:22:26.047624+0000 osd.7 (osd.7) 192 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.367 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:27 smithi055 bash[9296]: cluster 2020-09-08T16:22:26.047749+0000 osd.7 (osd.7) 193 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.368 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:27 smithi055 bash[9296]: cluster 2020-09-08T16:22:26.358984+0000 mgr.x (mgr.24118) 1653 : cluster [DBG] pgmap v2439: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.368 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:27 smithi055 bash[9296]: cluster 2020-09-08T16:22:26.438539+0000 mon.a (mon.0) 2495 : cluster [DBG] osdmap e871: 12 total, 12 up, 12 in 2020-09-08T16:27:02.368 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:27 smithi055 bash[9296]: cluster 2020-09-08T16:22:26.684716+0000 osd.0 (osd.0) 152 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.368 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:27 smithi055 bash[9296]: cluster 2020-09-08T16:22:26.684841+0000 osd.0 (osd.0) 153 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.368 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:28 smithi055 bash[9296]: cluster 2020-09-08T16:22:27.441484+0000 mon.a (mon.0) 2496 : cluster [DBG] osdmap e872: 12 total, 12 up, 12 in 2020-09-08T16:27:02.369 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:29 smithi055 bash[9296]: cluster 2020-09-08T16:22:28.359677+0000 mgr.x (mgr.24118) 1654 : cluster [DBG] pgmap v2442: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.369 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:31 smithi055 bash[9296]: cluster 2020-09-08T16:22:30.360347+0000 mgr.x (mgr.24118) 1655 : cluster [DBG] pgmap v2443: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.369 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:33 smithi055 bash[9296]: cluster 2020-09-08T16:22:32.361009+0000 mgr.x (mgr.24118) 1656 : cluster [DBG] pgmap v2444: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.369 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:35 smithi055 bash[9296]: cluster 2020-09-08T16:22:34.361722+0000 mgr.x (mgr.24118) 1657 : cluster [DBG] pgmap v2445: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.369 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:37 smithi055 bash[9296]: cluster 2020-09-08T16:22:36.201081+0000 osd.5 (osd.5) 158 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.370 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:37 smithi055 bash[9296]: cluster 2020-09-08T16:22:36.201209+0000 osd.5 (osd.5) 159 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.370 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:37 smithi055 bash[9296]: cluster 2020-09-08T16:22:36.362454+0000 mgr.x (mgr.24118) 1658 : cluster [DBG] pgmap v2446: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.370 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:37 smithi055 bash[9296]: cluster 2020-09-08T16:22:36.438961+0000 mon.a (mon.0) 2497 : cluster [DBG] osdmap e873: 12 total, 12 up, 12 in 2020-09-08T16:27:02.370 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:38 smithi055 bash[9296]: cluster 2020-09-08T16:22:37.838485+0000 osd.3 (osd.3) 161 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.370 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:38 smithi055 bash[9296]: cluster 2020-09-08T16:22:37.838627+0000 osd.3 (osd.3) 162 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.371 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:39 smithi055 bash[9296]: cluster 2020-09-08T16:22:38.363202+0000 mgr.x (mgr.24118) 1659 : cluster [DBG] pgmap v2448: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.371 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:39 smithi055 bash[9296]: cluster 2020-09-08T16:22:38.454615+0000 mon.a (mon.0) 2498 : cluster [DBG] osdmap e874: 12 total, 12 up, 12 in 2020-09-08T16:27:02.371 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:39 smithi055 bash[9296]: cluster 2020-09-08T16:22:38.838416+0000 osd.10 (osd.10) 137 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.371 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:39 smithi055 bash[9296]: cluster 2020-09-08T16:22:38.838535+0000 osd.10 (osd.10) 138 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.371 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:40 smithi055 bash[9296]: cluster 2020-09-08T16:22:39.463943+0000 mon.a (mon.0) 2499 : cluster [DBG] osdmap e875: 12 total, 12 up, 12 in 2020-09-08T16:27:02.372 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:41 smithi055 bash[9296]: cluster 2020-09-08T16:22:40.363881+0000 mgr.x (mgr.24118) 1660 : cluster [DBG] pgmap v2451: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.372 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:43 smithi055 bash[9296]: cluster 2020-09-08T16:22:42.364535+0000 mgr.x (mgr.24118) 1661 : cluster [DBG] pgmap v2452: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.372 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:45 smithi055 bash[9296]: cluster 2020-09-08T16:22:44.365151+0000 mgr.x (mgr.24118) 1662 : cluster [DBG] pgmap v2453: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.372 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:47 smithi055 bash[9296]: cluster 2020-09-08T16:22:45.923608+0000 osd.8 (osd.8) 132 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.372 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:47 smithi055 bash[9296]: cluster 2020-09-08T16:22:45.923735+0000 osd.8 (osd.8) 133 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.373 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:47 smithi055 bash[9296]: cluster 2020-09-08T16:22:46.365787+0000 mgr.x (mgr.24118) 1663 : cluster [DBG] pgmap v2454: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.373 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:47 smithi055 bash[9296]: cluster 2020-09-08T16:22:46.517760+0000 mon.a (mon.0) 2500 : cluster [DBG] osdmap e876: 12 total, 12 up, 12 in 2020-09-08T16:27:02.373 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:49 smithi055 bash[9296]: cluster 2020-09-08T16:22:48.366566+0000 mgr.x (mgr.24118) 1664 : cluster [DBG] pgmap v2456: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.373 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:51 smithi055 bash[9296]: cluster 2020-09-08T16:22:50.367302+0000 mgr.x (mgr.24118) 1665 : cluster [DBG] pgmap v2457: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.373 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:53 smithi055 bash[9296]: cluster 2020-09-08T16:22:52.368146+0000 mgr.x (mgr.24118) 1666 : cluster [DBG] pgmap v2458: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.374 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:54 smithi055 bash[9296]: cluster 2020-09-08T16:22:53.374169+0000 osd.1 (osd.1) 178 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.374 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:54 smithi055 bash[9296]: cluster 2020-09-08T16:22:53.374299+0000 osd.1 (osd.1) 179 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.374 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:54 smithi055 bash[9296]: cluster 2020-09-08T16:22:53.430286+0000 mon.a (mon.0) 2501 : cluster [DBG] osdmap e877: 12 total, 12 up, 12 in 2020-09-08T16:27:02.374 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:55 smithi055 bash[9296]: cluster 2020-09-08T16:22:54.368841+0000 mgr.x (mgr.24118) 1667 : cluster [DBG] pgmap v2460: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.374 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:57 smithi055 bash[9296]: cluster 2020-09-08T16:22:56.369497+0000 mgr.x (mgr.24118) 1668 : cluster [DBG] pgmap v2461: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.375 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:59 smithi055 bash[9296]: cluster 2020-09-08T16:22:58.338433+0000 osd.6 (osd.6) 143 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.375 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:59 smithi055 bash[9296]: cluster 2020-09-08T16:22:58.338558+0000 osd.6 (osd.6) 144 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.375 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:59 smithi055 bash[9296]: cluster 2020-09-08T16:22:58.370232+0000 mgr.x (mgr.24118) 1669 : cluster [DBG] pgmap v2462: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.375 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:22:59 smithi055 bash[9296]: cluster 2020-09-08T16:22:58.512036+0000 mon.a (mon.0) 2502 : cluster [DBG] osdmap e878: 12 total, 12 up, 12 in 2020-09-08T16:27:02.375 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:01 smithi055 bash[9296]: cluster 2020-09-08T16:23:00.371005+0000 mgr.x (mgr.24118) 1670 : cluster [DBG] pgmap v2464: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.376 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:03 smithi055 bash[9296]: cluster 2020-09-08T16:23:02.371690+0000 mgr.x (mgr.24118) 1671 : cluster [DBG] pgmap v2465: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.376 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:05 smithi055 bash[9296]: cluster 2020-09-08T16:23:04.324373+0000 osd.11 (osd.11) 158 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.376 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:05 smithi055 bash[9296]: cluster 2020-09-08T16:23:04.324501+0000 osd.11 (osd.11) 159 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.376 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:05 smithi055 bash[9296]: cluster 2020-09-08T16:23:04.372389+0000 mgr.x (mgr.24118) 1672 : cluster [DBG] pgmap v2466: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.376 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:05 smithi055 bash[9296]: cluster 2020-09-08T16:23:04.443525+0000 mon.a (mon.0) 2503 : cluster [DBG] osdmap e879: 12 total, 12 up, 12 in 2020-09-08T16:27:02.377 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:07 smithi055 bash[9296]: cluster 2020-09-08T16:23:06.373040+0000 mgr.x (mgr.24118) 1673 : cluster [DBG] pgmap v2468: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.377 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:08 smithi055 bash[9296]: cluster 2020-09-08T16:23:07.305093+0000 osd.4 (osd.4) 103 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.377 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:08 smithi055 bash[9296]: cluster 2020-09-08T16:23:07.305222+0000 osd.4 (osd.4) 104 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.377 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:08 smithi055 bash[9296]: cluster 2020-09-08T16:23:07.453557+0000 mon.a (mon.0) 2504 : cluster [DBG] osdmap e880: 12 total, 12 up, 12 in 2020-09-08T16:27:02.377 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:09 smithi055 bash[9296]: cluster 2020-09-08T16:23:08.373752+0000 mgr.x (mgr.24118) 1674 : cluster [DBG] pgmap v2470: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.378 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:11 smithi055 bash[9296]: cluster 2020-09-08T16:23:10.374490+0000 mgr.x (mgr.24118) 1675 : cluster [DBG] pgmap v2471: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.378 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:13 smithi055 bash[9296]: cluster 2020-09-08T16:23:12.375180+0000 mgr.x (mgr.24118) 1676 : cluster [DBG] pgmap v2472: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.378 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:15 smithi055 bash[9296]: cluster 2020-09-08T16:23:14.375859+0000 mgr.x (mgr.24118) 1677 : cluster [DBG] pgmap v2473: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.378 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:17 smithi055 bash[9296]: cluster 2020-09-08T16:23:16.376528+0000 mgr.x (mgr.24118) 1678 : cluster [DBG] pgmap v2474: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.378 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:18 smithi055 bash[9296]: cluster 2020-09-08T16:23:17.566849+0000 osd.9 (osd.9) 142 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.379 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:18 smithi055 bash[9296]: cluster 2020-09-08T16:23:17.566983+0000 osd.9 (osd.9) 143 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.379 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:18 smithi055 bash[9296]: cluster 2020-09-08T16:23:18.155326+0000 osd.2 (osd.2) 138 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.379 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:18 smithi055 bash[9296]: cluster 2020-09-08T16:23:18.155443+0000 osd.2 (osd.2) 139 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.379 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:19 smithi055 bash[9296]: cluster 2020-09-08T16:23:18.377221+0000 mgr.x (mgr.24118) 1679 : cluster [DBG] pgmap v2475: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.379 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:19 smithi055 bash[9296]: cluster 2020-09-08T16:23:18.463865+0000 mon.a (mon.0) 2505 : cluster [DBG] osdmap e881: 12 total, 12 up, 12 in 2020-09-08T16:27:02.380 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:21 smithi055 bash[9296]: cluster 2020-09-08T16:23:20.377865+0000 mgr.x (mgr.24118) 1680 : cluster [DBG] pgmap v2477: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.380 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:23 smithi055 bash[9296]: cluster 2020-09-08T16:23:22.378582+0000 mgr.x (mgr.24118) 1681 : cluster [DBG] pgmap v2478: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.380 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:25 smithi055 bash[9296]: cluster 2020-09-08T16:23:24.379267+0000 mgr.x (mgr.24118) 1682 : cluster [DBG] pgmap v2479: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.380 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:27 smithi055 bash[9296]: cluster 2020-09-08T16:23:26.379953+0000 mgr.x (mgr.24118) 1683 : cluster [DBG] pgmap v2480: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.380 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:27 smithi055 bash[9296]: cluster 2020-09-08T16:23:26.963247+0000 osd.0 (osd.0) 154 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.381 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:27 smithi055 bash[9296]: cluster 2020-09-08T16:23:26.963376+0000 osd.0 (osd.0) 155 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.381 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:28 smithi055 bash[9296]: cluster 2020-09-08T16:23:27.002998+0000 osd.7 (osd.7) 194 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.381 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:28 smithi055 bash[9296]: cluster 2020-09-08T16:23:27.003141+0000 osd.7 (osd.7) 195 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.381 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:28 smithi055 bash[9296]: cluster 2020-09-08T16:23:27.453332+0000 mon.a (mon.0) 2506 : cluster [DBG] osdmap e882: 12 total, 12 up, 12 in 2020-09-08T16:27:02.381 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:29 smithi055 bash[9296]: cluster 2020-09-08T16:23:28.380663+0000 mgr.x (mgr.24118) 1684 : cluster [DBG] pgmap v2482: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.382 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:31 smithi055 bash[9296]: cluster 2020-09-08T16:23:30.381335+0000 mgr.x (mgr.24118) 1685 : cluster [DBG] pgmap v2483: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.382 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:33 smithi055 bash[9296]: cluster 2020-09-08T16:23:32.382017+0000 mgr.x (mgr.24118) 1686 : cluster [DBG] pgmap v2484: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.382 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:35 smithi055 bash[9296]: cluster 2020-09-08T16:23:34.382751+0000 mgr.x (mgr.24118) 1687 : cluster [DBG] pgmap v2485: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.382 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:37 smithi055 bash[9296]: cluster 2020-09-08T16:23:36.383441+0000 mgr.x (mgr.24118) 1688 : cluster [DBG] pgmap v2486: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.382 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:37 smithi055 bash[9296]: cluster 2020-09-08T16:23:36.568336+0000 osd.5 (osd.5) 160 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.383 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:37 smithi055 bash[9296]: cluster 2020-09-08T16:23:36.568492+0000 osd.5 (osd.5) 161 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.383 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:38 smithi055 bash[9296]: cluster 2020-09-08T16:23:37.486484+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e883: 12 total, 12 up, 12 in 2020-09-08T16:27:02.383 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:38 smithi055 bash[9296]: cluster 2020-09-08T16:23:37.905775+0000 osd.3 (osd.3) 163 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.383 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:38 smithi055 bash[9296]: cluster 2020-09-08T16:23:37.905899+0000 osd.3 (osd.3) 164 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.383 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:39 smithi055 bash[9296]: cluster 2020-09-08T16:23:38.384285+0000 mgr.x (mgr.24118) 1689 : cluster [DBG] pgmap v2488: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.384 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:39 smithi055 bash[9296]: cluster 2020-09-08T16:23:38.499022+0000 mon.a (mon.0) 2508 : cluster [DBG] osdmap e884: 12 total, 12 up, 12 in 2020-09-08T16:27:02.384 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:40 smithi055 bash[9296]: cluster 2020-09-08T16:23:39.576603+0000 osd.10 (osd.10) 139 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.384 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:40 smithi055 bash[9296]: cluster 2020-09-08T16:23:39.576724+0000 osd.10 (osd.10) 140 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.384 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:41 smithi055 bash[9296]: cluster 2020-09-08T16:23:40.384981+0000 mgr.x (mgr.24118) 1690 : cluster [DBG] pgmap v2490: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.384 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:41 smithi055 bash[9296]: cluster 2020-09-08T16:23:40.503892+0000 mon.a (mon.0) 2509 : cluster [DBG] osdmap e885: 12 total, 12 up, 12 in 2020-09-08T16:27:02.385 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:43 smithi055 bash[9296]: cluster 2020-09-08T16:23:42.385743+0000 mgr.x (mgr.24118) 1691 : cluster [DBG] pgmap v2492: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.385 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:45 smithi055 bash[9296]: cluster 2020-09-08T16:23:44.386546+0000 mgr.x (mgr.24118) 1692 : cluster [DBG] pgmap v2493: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.385 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:47 smithi055 bash[9296]: cluster 2020-09-08T16:23:46.387299+0000 mgr.x (mgr.24118) 1693 : cluster [DBG] pgmap v2494: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.385 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:48 smithi055 bash[9296]: cluster 2020-09-08T16:23:46.640159+0000 osd.8 (osd.8) 134 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.385 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:48 smithi055 bash[9296]: cluster 2020-09-08T16:23:46.640302+0000 osd.8 (osd.8) 135 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.386 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:48 smithi055 bash[9296]: cluster 2020-09-08T16:23:47.531865+0000 mon.a (mon.0) 2510 : cluster [DBG] osdmap e886: 12 total, 12 up, 12 in 2020-09-08T16:27:02.386 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:49 smithi055 bash[9296]: cluster 2020-09-08T16:23:48.388036+0000 mgr.x (mgr.24118) 1694 : cluster [DBG] pgmap v2496: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.386 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:51 smithi055 bash[9296]: cluster 2020-09-08T16:23:50.388662+0000 mgr.x (mgr.24118) 1695 : cluster [DBG] pgmap v2497: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.386 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:53 smithi055 bash[9296]: cluster 2020-09-08T16:23:52.389418+0000 mgr.x (mgr.24118) 1696 : cluster [DBG] pgmap v2498: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.386 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:55 smithi055 bash[9296]: cluster 2020-09-08T16:23:54.236396+0000 osd.1 (osd.1) 180 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.387 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:55 smithi055 bash[9296]: cluster 2020-09-08T16:23:54.236544+0000 osd.1 (osd.1) 181 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.387 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:55 smithi055 bash[9296]: cluster 2020-09-08T16:23:54.390149+0000 mgr.x (mgr.24118) 1697 : cluster [DBG] pgmap v2499: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.387 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:55 smithi055 bash[9296]: cluster 2020-09-08T16:23:54.464578+0000 mon.a (mon.0) 2511 : cluster [DBG] osdmap e887: 12 total, 12 up, 12 in 2020-09-08T16:27:02.387 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:57 smithi055 bash[9296]: cluster 2020-09-08T16:23:56.390826+0000 mgr.x (mgr.24118) 1698 : cluster [DBG] pgmap v2501: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.387 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:23:59 smithi055 bash[9296]: cluster 2020-09-08T16:23:58.391558+0000 mgr.x (mgr.24118) 1699 : cluster [DBG] pgmap v2502: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.388 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:00 smithi055 bash[9296]: cluster 2020-09-08T16:23:59.299180+0000 osd.6 (osd.6) 145 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.388 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:00 smithi055 bash[9296]: cluster 2020-09-08T16:23:59.299303+0000 osd.6 (osd.6) 146 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.388 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:00 smithi055 bash[9296]: cluster 2020-09-08T16:23:59.521102+0000 mon.a (mon.0) 2512 : cluster [DBG] osdmap e888: 12 total, 12 up, 12 in 2020-09-08T16:27:02.388 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:01 smithi055 bash[9296]: cluster 2020-09-08T16:24:00.392224+0000 mgr.x (mgr.24118) 1700 : cluster [DBG] pgmap v2504: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.388 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:03 smithi055 bash[9296]: cluster 2020-09-08T16:24:02.392901+0000 mgr.x (mgr.24118) 1701 : cluster [DBG] pgmap v2505: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.389 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:05 smithi055 bash[9296]: cluster 2020-09-08T16:24:04.393624+0000 mgr.x (mgr.24118) 1702 : cluster [DBG] pgmap v2506: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.389 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:05 smithi055 bash[9296]: cluster 2020-09-08T16:24:04.508302+0000 osd.11 (osd.11) 160 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.389 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:05 smithi055 bash[9296]: cluster 2020-09-08T16:24:04.508429+0000 osd.11 (osd.11) 161 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.389 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:05 smithi055 bash[9296]: cluster 2020-09-08T16:24:04.561230+0000 mon.a (mon.0) 2513 : cluster [DBG] osdmap e889: 12 total, 12 up, 12 in 2020-09-08T16:27:02.389 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:07 smithi055 bash[9296]: cluster 2020-09-08T16:24:06.394289+0000 mgr.x (mgr.24118) 1703 : cluster [DBG] pgmap v2508: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.390 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:07 smithi055 bash[9296]: cluster 2020-09-08T16:24:07.389831+0000 osd.4 (osd.4) 105 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.390 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:07 smithi055 bash[9296]: cluster 2020-09-08T16:24:07.389953+0000 osd.4 (osd.4) 106 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.390 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:08 smithi055 bash[9296]: cluster 2020-09-08T16:24:07.569285+0000 mon.a (mon.0) 2514 : cluster [DBG] osdmap e890: 12 total, 12 up, 12 in 2020-09-08T16:27:02.390 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:09 smithi055 bash[9296]: cluster 2020-09-08T16:24:08.395016+0000 mgr.x (mgr.24118) 1704 : cluster [DBG] pgmap v2510: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.390 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:11 smithi055 bash[9296]: cluster 2020-09-08T16:24:10.395654+0000 mgr.x (mgr.24118) 1705 : cluster [DBG] pgmap v2511: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.391 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:13 smithi055 bash[9296]: cluster 2020-09-08T16:24:12.396433+0000 mgr.x (mgr.24118) 1706 : cluster [DBG] pgmap v2512: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.391 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:14 smithi055 bash[9296]: audit 2020-09-08T16:24:13.474986+0000 mon.b (mon.1) 138 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.391 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:14 smithi055 bash[9296]: audit 2020-09-08T16:24:13.477045+0000 mon.a (mon.0) 2515 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.391 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:14 smithi055 bash[9296]: audit 2020-09-08T16:24:13.482442+0000 mon.a (mon.0) 2516 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.391 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:15 smithi055 bash[9296]: cluster 2020-09-08T16:24:14.397293+0000 mgr.x (mgr.24118) 1707 : cluster [DBG] pgmap v2513: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.392 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:17 smithi055 bash[9296]: cluster 2020-09-08T16:24:16.398030+0000 mgr.x (mgr.24118) 1708 : cluster [DBG] pgmap v2514: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.392 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:18 smithi055 bash[9296]: cluster 2020-09-08T16:24:17.588032+0000 osd.9 (osd.9) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.392 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:18 smithi055 bash[9296]: cluster 2020-09-08T16:24:17.588165+0000 osd.9 (osd.9) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.392 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:19 smithi055 bash[9296]: cluster 2020-09-08T16:24:18.398882+0000 mgr.x (mgr.24118) 1709 : cluster [DBG] pgmap v2515: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.392 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:19 smithi055 bash[9296]: cluster 2020-09-08T16:24:18.542472+0000 mon.a (mon.0) 2517 : cluster [DBG] osdmap e891: 12 total, 12 up, 12 in 2020-09-08T16:27:02.393 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:19 smithi055 bash[9296]: cluster 2020-09-08T16:24:18.764818+0000 osd.2 (osd.2) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.393 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:19 smithi055 bash[9296]: cluster 2020-09-08T16:24:18.764929+0000 osd.2 (osd.2) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.393 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:20 smithi055 bash[9296]: cluster 2020-09-08T16:24:19.546699+0000 mon.a (mon.0) 2518 : cluster [DBG] osdmap e892: 12 total, 12 up, 12 in 2020-09-08T16:27:02.393 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:21 smithi055 bash[9296]: cluster 2020-09-08T16:24:20.399622+0000 mgr.x (mgr.24118) 1710 : cluster [DBG] pgmap v2518: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.393 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:23 smithi055 bash[9296]: cluster 2020-09-08T16:24:22.400285+0000 mgr.x (mgr.24118) 1711 : cluster [DBG] pgmap v2519: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.394 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:25 smithi055 bash[9296]: cluster 2020-09-08T16:24:24.400983+0000 mgr.x (mgr.24118) 1712 : cluster [DBG] pgmap v2520: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.394 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:27 smithi055 bash[9296]: cluster 2020-09-08T16:24:26.401749+0000 mgr.x (mgr.24118) 1713 : cluster [DBG] pgmap v2521: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.394 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:27 smithi055 bash[9296]: cluster 2020-09-08T16:24:27.099750+0000 osd.0 (osd.0) 156 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.394 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:27 smithi055 bash[9296]: cluster 2020-09-08T16:24:27.099880+0000 osd.0 (osd.0) 157 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.394 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:28 smithi055 bash[9296]: cluster 2020-09-08T16:24:27.315084+0000 osd.7 (osd.7) 196 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.395 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:28 smithi055 bash[9296]: cluster 2020-09-08T16:24:27.315209+0000 osd.7 (osd.7) 197 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.395 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:28 smithi055 bash[9296]: cluster 2020-09-08T16:24:27.581255+0000 mon.a (mon.0) 2519 : cluster [DBG] osdmap e893: 12 total, 12 up, 12 in 2020-09-08T16:27:02.395 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:29 smithi055 bash[9296]: cluster 2020-09-08T16:24:28.402497+0000 mgr.x (mgr.24118) 1714 : cluster [DBG] pgmap v2523: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.395 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:31 smithi055 bash[9296]: cluster 2020-09-08T16:24:30.403186+0000 mgr.x (mgr.24118) 1715 : cluster [DBG] pgmap v2524: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.396 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:33 smithi055 bash[9296]: cluster 2020-09-08T16:24:32.403863+0000 mgr.x (mgr.24118) 1716 : cluster [DBG] pgmap v2525: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.396 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:35 smithi055 bash[9296]: cluster 2020-09-08T16:24:34.404578+0000 mgr.x (mgr.24118) 1717 : cluster [DBG] pgmap v2526: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.397 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:37 smithi055 bash[9296]: cluster 2020-09-08T16:24:36.405231+0000 mgr.x (mgr.24118) 1718 : cluster [DBG] pgmap v2527: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.397 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:38 smithi055 bash[9296]: cluster 2020-09-08T16:24:37.495138+0000 osd.5 (osd.5) 162 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.398 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:38 smithi055 bash[9296]: cluster 2020-09-08T16:24:37.495296+0000 osd.5 (osd.5) 163 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.398 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:38 smithi055 bash[9296]: cluster 2020-09-08T16:24:37.909834+0000 osd.3 (osd.3) 165 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.398 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:38 smithi055 bash[9296]: cluster 2020-09-08T16:24:37.909972+0000 osd.3 (osd.3) 166 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.399 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:39 smithi055 bash[9296]: cluster 2020-09-08T16:24:38.405935+0000 mgr.x (mgr.24118) 1719 : cluster [DBG] pgmap v2528: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.399 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:39 smithi055 bash[9296]: cluster 2020-09-08T16:24:38.492275+0000 mon.a (mon.0) 2520 : cluster [DBG] osdmap e894: 12 total, 12 up, 12 in 2020-09-08T16:27:02.399 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:41 smithi055 bash[9296]: cluster 2020-09-08T16:24:40.353237+0000 osd.10 (osd.10) 141 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.400 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:41 smithi055 bash[9296]: cluster 2020-09-08T16:24:40.353357+0000 osd.10 (osd.10) 142 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.400 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:41 smithi055 bash[9296]: cluster 2020-09-08T16:24:40.406644+0000 mgr.x (mgr.24118) 1720 : cluster [DBG] pgmap v2530: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.400 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:42 smithi055 bash[9296]: cluster 2020-09-08T16:24:41.325942+0000 mon.a (mon.0) 2521 : cluster [DBG] osdmap e895: 12 total, 12 up, 12 in 2020-09-08T16:27:02.401 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:43 smithi055 bash[9296]: cluster 2020-09-08T16:24:42.407347+0000 mgr.x (mgr.24118) 1721 : cluster [DBG] pgmap v2532: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.401 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:45 smithi055 bash[9296]: cluster 2020-09-08T16:24:44.408070+0000 mgr.x (mgr.24118) 1722 : cluster [DBG] pgmap v2533: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.401 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:47 smithi055 bash[9296]: cluster 2020-09-08T16:24:46.408737+0000 mgr.x (mgr.24118) 1723 : cluster [DBG] pgmap v2534: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.402 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:49 smithi055 bash[9296]: cluster 2020-09-08T16:24:47.670412+0000 osd.8 (osd.8) 136 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.402 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:49 smithi055 bash[9296]: cluster 2020-09-08T16:24:47.670538+0000 osd.8 (osd.8) 137 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.402 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:49 smithi055 bash[9296]: cluster 2020-09-08T16:24:48.409508+0000 mgr.x (mgr.24118) 1724 : cluster [DBG] pgmap v2535: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.403 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:49 smithi055 bash[9296]: cluster 2020-09-08T16:24:48.486000+0000 mon.a (mon.0) 2522 : cluster [DBG] osdmap e896: 12 total, 12 up, 12 in 2020-09-08T16:27:02.403 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:51 smithi055 bash[9296]: cluster 2020-09-08T16:24:50.410325+0000 mgr.x (mgr.24118) 1725 : cluster [DBG] pgmap v2537: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.403 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:53 smithi055 bash[9296]: cluster 2020-09-08T16:24:52.411043+0000 mgr.x (mgr.24118) 1726 : cluster [DBG] pgmap v2538: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.404 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:55 smithi055 bash[9296]: cluster 2020-09-08T16:24:54.411761+0000 mgr.x (mgr.24118) 1727 : cluster [DBG] pgmap v2539: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.404 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:55 smithi055 bash[9296]: cluster 2020-09-08T16:24:54.547404+0000 osd.1 (osd.1) 182 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.404 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:55 smithi055 bash[9296]: cluster 2020-09-08T16:24:54.547537+0000 osd.1 (osd.1) 183 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.405 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:56 smithi055 bash[9296]: cluster 2020-09-08T16:24:55.526084+0000 mon.a (mon.0) 2523 : cluster [DBG] osdmap e897: 12 total, 12 up, 12 in 2020-09-08T16:27:02.405 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:57 smithi055 bash[9296]: cluster 2020-09-08T16:24:56.412405+0000 mgr.x (mgr.24118) 1728 : cluster [DBG] pgmap v2541: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:24:59 smithi055 bash[9296]: cluster 2020-09-08T16:24:58.413152+0000 mgr.x (mgr.24118) 1729 : cluster [DBG] pgmap v2542: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:01 smithi055 bash[9296]: cluster 2020-09-08T16:25:00.061572+0000 osd.6 (osd.6) 147 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.406 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:01 smithi055 bash[9296]: cluster 2020-09-08T16:25:00.061698+0000 osd.6 (osd.6) 148 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:01 smithi055 bash[9296]: cluster 2020-09-08T16:25:00.413852+0000 mgr.x (mgr.24118) 1730 : cluster [DBG] pgmap v2543: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:01 smithi055 bash[9296]: cluster 2020-09-08T16:25:00.548020+0000 mon.a (mon.0) 2524 : cluster [DBG] osdmap e898: 12 total, 12 up, 12 in 2020-09-08T16:27:02.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:03 smithi055 bash[9296]: cluster 2020-09-08T16:25:02.414568+0000 mgr.x (mgr.24118) 1731 : cluster [DBG] pgmap v2545: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:05 smithi055 bash[9296]: cluster 2020-09-08T16:25:04.415243+0000 mgr.x (mgr.24118) 1732 : cluster [DBG] pgmap v2546: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:05 smithi055 bash[9296]: cluster 2020-09-08T16:25:04.648267+0000 osd.11 (osd.11) 162 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:05 smithi055 bash[9296]: cluster 2020-09-08T16:25:04.648410+0000 osd.11 (osd.11) 163 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:06 smithi055 bash[9296]: cluster 2020-09-08T16:25:05.562272+0000 mon.a (mon.0) 2525 : cluster [DBG] osdmap e899: 12 total, 12 up, 12 in 2020-09-08T16:27:02.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:07 smithi055 bash[9296]: cluster 2020-09-08T16:25:06.415870+0000 mgr.x (mgr.24118) 1733 : cluster [DBG] pgmap v2548: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.409 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:07 smithi055 bash[9296]: cluster 2020-09-08T16:25:07.405400+0000 osd.4 (osd.4) 107 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:07 smithi055 bash[9296]: cluster 2020-09-08T16:25:07.405548+0000 osd.4 (osd.4) 108 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:08 smithi055 bash[9296]: cluster 2020-09-08T16:25:07.593722+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e900: 12 total, 12 up, 12 in 2020-09-08T16:27:02.410 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:09 smithi055 bash[9296]: cluster 2020-09-08T16:25:08.416541+0000 mgr.x (mgr.24118) 1734 : cluster [DBG] pgmap v2550: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:11 smithi055 bash[9296]: cluster 2020-09-08T16:25:10.417322+0000 mgr.x (mgr.24118) 1735 : cluster [DBG] pgmap v2551: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:13 smithi055 bash[9296]: cluster 2020-09-08T16:25:12.418166+0000 mgr.x (mgr.24118) 1736 : cluster [DBG] pgmap v2552: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.411 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:15 smithi055 bash[9296]: cluster 2020-09-08T16:25:14.418937+0000 mgr.x (mgr.24118) 1737 : cluster [DBG] pgmap v2553: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:17 smithi055 bash[9296]: cluster 2020-09-08T16:25:16.419618+0000 mgr.x (mgr.24118) 1738 : cluster [DBG] pgmap v2554: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:19 smithi055 bash[9296]: cluster 2020-09-08T16:25:18.420356+0000 mgr.x (mgr.24118) 1739 : cluster [DBG] pgmap v2555: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.412 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:19 smithi055 bash[9296]: cluster 2020-09-08T16:25:18.490436+0000 osd.9 (osd.9) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.413 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:19 smithi055 bash[9296]: cluster 2020-09-08T16:25:18.490614+0000 osd.9 (osd.9) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.413 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:20 smithi055 bash[9296]: cluster 2020-09-08T16:25:19.498676+0000 mon.a (mon.0) 2527 : cluster [DBG] osdmap e901: 12 total, 12 up, 12 in 2020-09-08T16:27:02.413 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:20 smithi055 bash[9296]: cluster 2020-09-08T16:25:19.710713+0000 osd.2 (osd.2) 142 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.420 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:20 smithi055 bash[9296]: cluster 2020-09-08T16:25:19.710828+0000 osd.2 (osd.2) 143 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.421 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:21 smithi055 bash[9296]: cluster 2020-09-08T16:25:20.421020+0000 mgr.x (mgr.24118) 1740 : cluster [DBG] pgmap v2557: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.421 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:21 smithi055 bash[9296]: cluster 2020-09-08T16:25:20.510963+0000 mon.a (mon.0) 2528 : cluster [DBG] osdmap e902: 12 total, 12 up, 12 in 2020-09-08T16:27:02.421 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:23 smithi055 bash[9296]: cluster 2020-09-08T16:25:22.421712+0000 mgr.x (mgr.24118) 1741 : cluster [DBG] pgmap v2559: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.422 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:25 smithi055 bash[9296]: cluster 2020-09-08T16:25:24.422438+0000 mgr.x (mgr.24118) 1742 : cluster [DBG] pgmap v2560: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.422 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:27 smithi055 bash[9296]: cluster 2020-09-08T16:25:26.423122+0000 mgr.x (mgr.24118) 1743 : cluster [DBG] pgmap v2561: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.422 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:28 smithi055 bash[9296]: cluster 2020-09-08T16:25:27.293029+0000 osd.0 (osd.0) 158 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.423 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:28 smithi055 bash[9296]: cluster 2020-09-08T16:25:27.293156+0000 osd.0 (osd.0) 159 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.423 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:28 smithi055 bash[9296]: cluster 2020-09-08T16:25:27.543088+0000 mon.a (mon.0) 2529 : cluster [DBG] osdmap e903: 12 total, 12 up, 12 in 2020-09-08T16:27:02.423 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:29 smithi055 bash[9296]: cluster 2020-09-08T16:25:27.910337+0000 osd.7 (osd.7) 198 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.424 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:29 smithi055 bash[9296]: cluster 2020-09-08T16:25:27.910470+0000 osd.7 (osd.7) 199 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.424 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:29 smithi055 bash[9296]: cluster 2020-09-08T16:25:28.423802+0000 mgr.x (mgr.24118) 1744 : cluster [DBG] pgmap v2563: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.424 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:29 smithi055 bash[9296]: cluster 2020-09-08T16:25:28.562409+0000 mon.a (mon.0) 2530 : cluster [DBG] osdmap e904: 12 total, 12 up, 12 in 2020-09-08T16:27:02.425 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:31 smithi055 bash[9296]: cluster 2020-09-08T16:25:30.424475+0000 mgr.x (mgr.24118) 1745 : cluster [DBG] pgmap v2565: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.425 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:33 smithi055 bash[9296]: cluster 2020-09-08T16:25:32.425199+0000 mgr.x (mgr.24118) 1746 : cluster [DBG] pgmap v2566: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.425 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:35 smithi055 bash[9296]: cluster 2020-09-08T16:25:34.425909+0000 mgr.x (mgr.24118) 1747 : cluster [DBG] pgmap v2567: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.426 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:37 smithi055 bash[9296]: cluster 2020-09-08T16:25:36.426615+0000 mgr.x (mgr.24118) 1748 : cluster [DBG] pgmap v2568: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.426 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:38 smithi055 bash[9296]: cluster 2020-09-08T16:25:37.598505+0000 osd.5 (osd.5) 164 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.426 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:38 smithi055 bash[9296]: cluster 2020-09-08T16:25:37.598661+0000 osd.5 (osd.5) 165 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.427 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:39 smithi055 bash[9296]: cluster 2020-09-08T16:25:38.427331+0000 mgr.x (mgr.24118) 1749 : cluster [DBG] pgmap v2569: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.427 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:39 smithi055 bash[9296]: cluster 2020-09-08T16:25:38.513211+0000 mon.a (mon.0) 2531 : cluster [DBG] osdmap e905: 12 total, 12 up, 12 in 2020-09-08T16:27:02.427 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:39 smithi055 bash[9296]: cluster 2020-09-08T16:25:38.764276+0000 osd.3 (osd.3) 167 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.428 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:39 smithi055 bash[9296]: cluster 2020-09-08T16:25:38.764409+0000 osd.3 (osd.3) 168 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.428 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:40 smithi055 bash[9296]: cluster 2020-09-08T16:25:39.515081+0000 mon.a (mon.0) 2532 : cluster [DBG] osdmap e906: 12 total, 12 up, 12 in 2020-09-08T16:27:02.428 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:41 smithi055 bash[9296]: cluster 2020-09-08T16:25:40.428011+0000 mgr.x (mgr.24118) 1750 : cluster [DBG] pgmap v2572: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.429 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:41 smithi055 bash[9296]: cluster 2020-09-08T16:25:40.468260+0000 osd.10 (osd.10) 143 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.429 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:41 smithi055 bash[9296]: cluster 2020-09-08T16:25:40.468351+0000 osd.10 (osd.10) 144 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.429 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:41 smithi055 bash[9296]: cluster 2020-09-08T16:25:41.349574+0000 mon.a (mon.0) 2533 : cluster [DBG] osdmap e907: 12 total, 12 up, 12 in 2020-09-08T16:27:02.430 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:43 smithi055 bash[9296]: cluster 2020-09-08T16:25:42.428652+0000 mgr.x (mgr.24118) 1751 : cluster [DBG] pgmap v2574: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.430 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:45 smithi055 bash[9296]: cluster 2020-09-08T16:25:44.429449+0000 mgr.x (mgr.24118) 1752 : cluster [DBG] pgmap v2575: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.430 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:47 smithi055 bash[9296]: cluster 2020-09-08T16:25:46.430281+0000 mgr.x (mgr.24118) 1753 : cluster [DBG] pgmap v2576: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.430 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:49 smithi055 bash[9296]: cluster 2020-09-08T16:25:48.640765+0000 osd.8 (osd.8) 138 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.431 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:49 smithi055 bash[9296]: cluster 2020-09-08T16:25:48.640899+0000 osd.8 (osd.8) 139 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.431 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:49 smithi055 bash[9296]: audit 2020-09-08T16:25:48.993840+0000 mon.a (mon.0) 2534 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.402933\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403132\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403786\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404332\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404856\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.405550\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:25:48.990918\"}"}]: dispatch 2020-09-08T16:27:02.431 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:49 smithi055 bash[9296]: audit 2020-09-08T16:25:49.000833+0000 mon.a (mon.0) 2535 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.402933\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403132\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403786\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404332\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404856\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.405550\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:25:48.990918\"}"}]': finished 2020-09-08T16:27:02.431 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:49 smithi055 bash[9296]: audit 2020-09-08T16:25:49.001101+0000 mon.a (mon.0) 2536 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685123\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685289\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685873\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686336\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686892\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.687437\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:25:48.995696\"}"}]: dispatch 2020-09-08T16:27:02.432 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:49 smithi055 bash[9296]: audit 2020-09-08T16:25:49.005877+0000 mon.a (mon.0) 2537 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685123\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685289\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685873\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686336\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686892\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.687437\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:25:48.995696\"}"}]': finished 2020-09-08T16:27:02.432 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:49 smithi055 bash[9296]: audit 2020-09-08T16:25:49.012081+0000 mon.a (mon.0) 2538 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698864\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T16:14:49.698956\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699006\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699331\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699539\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699740\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.700002\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:25:48.998320\"}"}]: dispatch 2020-09-08T16:27:02.433 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:49 smithi055 bash[9296]: audit 2020-09-08T16:25:49.012937+0000 mon.a (mon.0) 2539 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:27:02.433 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:49 smithi055 bash[9296]: audit 2020-09-08T16:25:49.016722+0000 mon.a (mon.0) 2540 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698864\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T16:14:49.698956\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699006\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699331\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699539\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699740\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.700002\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:25:48.998320\"}"}]': finished 2020-09-08T16:27:02.433 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:49 smithi055 bash[9296]: audit 2020-09-08T16:25:49.017007+0000 mon.a (mon.0) 2541 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:27:02.434 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:49 smithi055 bash[9296]: audit 2020-09-08T16:25:49.017561+0000 mon.a (mon.0) 2542 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:27:02.434 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:50 smithi055 bash[9296]: cluster 2020-09-08T16:25:49.619281+0000 mon.a (mon.0) 2543 : cluster [DBG] osdmap e908: 12 total, 12 up, 12 in 2020-09-08T16:27:02.434 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:50 smithi055 bash[9296]: audit 2020-09-08T16:25:49.860073+0000 mon.a (mon.0) 2544 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.853655\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.853925\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.854774\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.855418\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.856044\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:25:49.856810\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:25:48.995696\"}"}]: dispatch 2020-09-08T16:27:02.434 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:50 smithi055 bash[9296]: audit 2020-09-08T16:25:49.866266+0000 mon.a (mon.0) 2545 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.853655\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.853925\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.854774\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.855418\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.856044\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:25:49.856810\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:25:48.995696\"}"}]': finished 2020-09-08T16:27:02.435 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:50 smithi055 bash[9296]: audit 2020-09-08T16:25:49.969344+0000 mon.a (mon.0) 2546 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.963581\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.963784\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.963914\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.964661\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.965214\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.965750\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:25:49.966460\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:25:48.998320\"}"}]: dispatch 2020-09-08T16:27:02.435 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:50 smithi055 bash[9296]: audit 2020-09-08T16:25:49.975208+0000 mon.a (mon.0) 2547 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.963581\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.963784\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.963914\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.964661\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.965214\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.965750\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:25:49.966460\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:25:48.998320\"}"}]': finished 2020-09-08T16:27:02.435 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:50 smithi055 bash[9296]: audit 2020-09-08T16:25:50.198306+0000 mon.a (mon.0) 2548 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.192547\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.192731\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.193342\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.193892\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.194470\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:25:50.195154\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:25:48.990918\"}"}]: dispatch 2020-09-08T16:27:02.436 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:50 smithi055 bash[9296]: audit 2020-09-08T16:25:50.204507+0000 mon.a (mon.0) 2549 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.192547\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.192731\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.193342\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.193892\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.194470\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:25:50.195154\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:25:48.990918\"}"}]': finished 2020-09-08T16:27:02.436 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:50 smithi055 bash[9296]: audit 2020-09-08T16:25:50.206948+0000 mon.a (mon.0) 2550 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.436 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:50 smithi055 bash[9296]: audit 2020-09-08T16:25:50.212351+0000 mon.a (mon.0) 2551 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.437 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:51 smithi055 bash[9296]: cluster 2020-09-08T16:25:50.431687+0000 mgr.x (mgr.24118) 1755 : cluster [DBG] pgmap v2579: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.437 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:53 smithi055 bash[9296]: cluster 2020-09-08T16:25:52.432399+0000 mgr.x (mgr.24118) 1756 : cluster [DBG] pgmap v2580: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.437 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:55 smithi055 bash[9296]: cluster 2020-09-08T16:25:54.433250+0000 mgr.x (mgr.24118) 1757 : cluster [DBG] pgmap v2581: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.437 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:55 smithi055 bash[9296]: cluster 2020-09-08T16:25:54.862483+0000 osd.1 (osd.1) 184 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.437 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:55 smithi055 bash[9296]: cluster 2020-09-08T16:25:54.862616+0000 osd.1 (osd.1) 185 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.438 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:56 smithi055 bash[9296]: cluster 2020-09-08T16:25:55.638400+0000 mon.a (mon.0) 2552 : cluster [DBG] osdmap e909: 12 total, 12 up, 12 in 2020-09-08T16:27:02.438 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:57 smithi055 bash[9296]: cluster 2020-09-08T16:25:56.433899+0000 mgr.x (mgr.24118) 1758 : cluster [DBG] pgmap v2583: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.438 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:25:59 smithi055 bash[9296]: cluster 2020-09-08T16:25:58.434653+0000 mgr.x (mgr.24118) 1759 : cluster [DBG] pgmap v2584: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.438 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:01 smithi055 bash[9296]: cluster 2020-09-08T16:26:00.224874+0000 osd.6 (osd.6) 149 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.439 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:01 smithi055 bash[9296]: cluster 2020-09-08T16:26:00.225015+0000 osd.6 (osd.6) 150 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.439 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:01 smithi055 bash[9296]: cluster 2020-09-08T16:26:00.435276+0000 mgr.x (mgr.24118) 1760 : cluster [DBG] pgmap v2585: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.439 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:01 smithi055 bash[9296]: cluster 2020-09-08T16:26:00.670003+0000 mon.a (mon.0) 2553 : cluster [DBG] osdmap e910: 12 total, 12 up, 12 in 2020-09-08T16:27:02.439 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:03 smithi055 bash[9296]: cluster 2020-09-08T16:26:02.435956+0000 mgr.x (mgr.24118) 1761 : cluster [DBG] pgmap v2587: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.440 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:05 smithi055 bash[9296]: cluster 2020-09-08T16:26:04.436653+0000 mgr.x (mgr.24118) 1762 : cluster [DBG] pgmap v2588: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.440 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:06 smithi055 bash[9296]: cluster 2020-09-08T16:26:05.395848+0000 osd.11 (osd.11) 164 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.440 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:06 smithi055 bash[9296]: cluster 2020-09-08T16:26:05.395976+0000 osd.11 (osd.11) 165 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.440 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:06 smithi055 bash[9296]: cluster 2020-09-08T16:26:06.345739+0000 mon.a (mon.0) 2554 : cluster [DBG] osdmap e911: 12 total, 12 up, 12 in 2020-09-08T16:27:02.440 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:07 smithi055 bash[9296]: cluster 2020-09-08T16:26:06.437157+0000 mgr.x (mgr.24118) 1763 : cluster [DBG] pgmap v2590: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.441 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:08 smithi055 bash[9296]: cluster 2020-09-08T16:26:08.234288+0000 osd.4 (osd.4) 109 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.441 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:08 smithi055 bash[9296]: cluster 2020-09-08T16:26:08.234451+0000 osd.4 (osd.4) 110 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.441 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:09 smithi055 bash[9296]: cluster 2020-09-08T16:26:08.711456+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e912: 12 total, 12 up, 12 in 2020-09-08T16:27:02.441 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:09 smithi055 bash[9296]: audit 2020-09-08T16:26:08.714372+0000 mon.a (mon.0) 2556 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.441 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:09 smithi055 bash[9296]: audit 2020-09-08T16:26:08.719180+0000 mon.a (mon.0) 2557 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.442 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:11 smithi055 bash[9296]: cluster 2020-09-08T16:26:10.438346+0000 mgr.x (mgr.24118) 1765 : cluster [DBG] pgmap v2593: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.442 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:13 smithi055 bash[9296]: cluster 2020-09-08T16:26:12.439023+0000 mgr.x (mgr.24118) 1766 : cluster [DBG] pgmap v2594: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.442 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:15 smithi055 bash[9296]: cluster 2020-09-08T16:26:14.439803+0000 mgr.x (mgr.24118) 1767 : cluster [DBG] pgmap v2595: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.442 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:17 smithi055 bash[9296]: cluster 2020-09-08T16:26:16.440416+0000 mgr.x (mgr.24118) 1768 : cluster [DBG] pgmap v2596: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.442 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:19 smithi055 bash[9296]: cluster 2020-09-08T16:26:18.441118+0000 mgr.x (mgr.24118) 1769 : cluster [DBG] pgmap v2597: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.443 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:19 smithi055 bash[9296]: cluster 2020-09-08T16:26:19.009384+0000 osd.9 (osd.9) 148 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.443 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:19 smithi055 bash[9296]: cluster 2020-09-08T16:26:19.009503+0000 osd.9 (osd.9) 149 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.443 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:20 smithi055 bash[9296]: cluster 2020-09-08T16:26:19.521611+0000 mon.a (mon.0) 2558 : cluster [DBG] osdmap e913: 12 total, 12 up, 12 in 2020-09-08T16:27:02.443 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:20 smithi055 bash[9296]: cluster 2020-09-08T16:26:20.098181+0000 osd.2 (osd.2) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.443 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:20 smithi055 bash[9296]: cluster 2020-09-08T16:26:20.098304+0000 osd.2 (osd.2) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.444 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:21 smithi055 bash[9296]: cluster 2020-09-08T16:26:20.441781+0000 mgr.x (mgr.24118) 1770 : cluster [DBG] pgmap v2599: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.444 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:21 smithi055 bash[9296]: cluster 2020-09-08T16:26:20.545496+0000 mon.a (mon.0) 2559 : cluster [DBG] osdmap e914: 12 total, 12 up, 12 in 2020-09-08T16:27:02.444 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:23 smithi055 bash[9296]: cluster 2020-09-08T16:26:22.442522+0000 mgr.x (mgr.24118) 1771 : cluster [DBG] pgmap v2601: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.444 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:25 smithi055 bash[9296]: cluster 2020-09-08T16:26:24.443182+0000 mgr.x (mgr.24118) 1772 : cluster [DBG] pgmap v2602: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.444 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:27 smithi055 bash[9296]: cluster 2020-09-08T16:26:26.443879+0000 mgr.x (mgr.24118) 1773 : cluster [DBG] pgmap v2603: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.445 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:28 smithi055 bash[9296]: cluster 2020-09-08T16:26:28.098777+0000 osd.0 (osd.0) 160 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.445 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:28 smithi055 bash[9296]: cluster 2020-09-08T16:26:28.098921+0000 osd.0 (osd.0) 161 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.445 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:29 smithi055 bash[9296]: cluster 2020-09-08T16:26:28.444593+0000 mgr.x (mgr.24118) 1774 : cluster [DBG] pgmap v2604: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.445 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:29 smithi055 bash[9296]: cluster 2020-09-08T16:26:28.579344+0000 mon.a (mon.0) 2560 : cluster [DBG] osdmap e915: 12 total, 12 up, 12 in 2020-09-08T16:27:02.445 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:30 smithi055 bash[9296]: cluster 2020-09-08T16:26:28.796301+0000 osd.7 (osd.7) 200 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.446 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:30 smithi055 bash[9296]: cluster 2020-09-08T16:26:28.796427+0000 osd.7 (osd.7) 201 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.446 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:30 smithi055 bash[9296]: cluster 2020-09-08T16:26:29.592732+0000 mon.a (mon.0) 2561 : cluster [DBG] osdmap e916: 12 total, 12 up, 12 in 2020-09-08T16:27:02.446 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:31 smithi055 bash[9296]: cluster 2020-09-08T16:26:30.445361+0000 mgr.x (mgr.24118) 1775 : cluster [DBG] pgmap v2607: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.446 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:33 smithi055 bash[9296]: cluster 2020-09-08T16:26:32.446087+0000 mgr.x (mgr.24118) 1776 : cluster [DBG] pgmap v2608: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.446 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:35 smithi055 bash[9296]: cluster 2020-09-08T16:26:34.446753+0000 mgr.x (mgr.24118) 1777 : cluster [DBG] pgmap v2609: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.447 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:37 smithi055 bash[9296]: cluster 2020-09-08T16:26:36.447416+0000 mgr.x (mgr.24118) 1778 : cluster [DBG] pgmap v2610: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.447 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:38 smithi055 bash[9296]: cluster 2020-09-08T16:26:37.639096+0000 osd.5 (osd.5) 166 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.447 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:38 smithi055 bash[9296]: cluster 2020-09-08T16:26:37.639237+0000 osd.5 (osd.5) 167 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.447 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:39 smithi055 bash[9296]: cluster 2020-09-08T16:26:38.448127+0000 mgr.x (mgr.24118) 1779 : cluster [DBG] pgmap v2611: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.447 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:39 smithi055 bash[9296]: cluster 2020-09-08T16:26:38.631152+0000 mon.a (mon.0) 2562 : cluster [DBG] osdmap e917: 12 total, 12 up, 12 in 2020-09-08T16:27:02.448 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:40 smithi055 bash[9296]: cluster 2020-09-08T16:26:39.651090+0000 osd.3 (osd.3) 169 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.448 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:40 smithi055 bash[9296]: cluster 2020-09-08T16:26:39.651219+0000 osd.3 (osd.3) 170 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.448 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:41 smithi055 bash[9296]: cluster 2020-09-08T16:26:40.448778+0000 mgr.x (mgr.24118) 1780 : cluster [DBG] pgmap v2613: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.448 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:41 smithi055 bash[9296]: cluster 2020-09-08T16:26:40.534204+0000 osd.10 (osd.10) 145 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.448 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:41 smithi055 bash[9296]: cluster 2020-09-08T16:26:40.534322+0000 osd.10 (osd.10) 146 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.449 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:41 smithi055 bash[9296]: cluster 2020-09-08T16:26:40.639992+0000 mon.a (mon.0) 2563 : cluster [DBG] osdmap e918: 12 total, 12 up, 12 in 2020-09-08T16:27:02.449 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:43 smithi055 bash[9296]: cluster 2020-09-08T16:26:42.449459+0000 mgr.x (mgr.24118) 1781 : cluster [DBG] pgmap v2615: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.449 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:45 smithi055 bash[9296]: cluster 2020-09-08T16:26:44.450417+0000 mgr.x (mgr.24118) 1782 : cluster [DBG] pgmap v2616: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.449 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:47 smithi055 bash[9296]: cluster 2020-09-08T16:26:46.451131+0000 mgr.x (mgr.24118) 1783 : cluster [DBG] pgmap v2617: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.449 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:49 smithi055 bash[9296]: cluster 2020-09-08T16:26:48.451858+0000 mgr.x (mgr.24118) 1784 : cluster [DBG] pgmap v2618: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:49 smithi055 bash[9296]: cluster 2020-09-08T16:26:49.534260+0000 osd.8 (osd.8) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:49 smithi055 bash[9296]: cluster 2020-09-08T16:26:49.534382+0000 osd.8 (osd.8) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:50 smithi055 bash[9296]: cluster 2020-09-08T16:26:49.681079+0000 mon.a (mon.0) 2564 : cluster [DBG] osdmap e919: 12 total, 12 up, 12 in 2020-09-08T16:27:02.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:51 smithi055 bash[9296]: cluster 2020-09-08T16:26:50.452524+0000 mgr.x (mgr.24118) 1785 : cluster [DBG] pgmap v2620: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.450 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:53 smithi055 bash[9296]: cluster 2020-09-08T16:26:52.453264+0000 mgr.x (mgr.24118) 1786 : cluster [DBG] pgmap v2621: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:55 smithi055 bash[9296]: cluster 2020-09-08T16:26:54.454065+0000 mgr.x (mgr.24118) 1787 : cluster [DBG] pgmap v2622: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:56 smithi055 bash[9296]: cluster 2020-09-08T16:26:55.777497+0000 osd.1 (osd.1) 186 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:56 smithi055 bash[9296]: cluster 2020-09-08T16:26:55.777629+0000 osd.1 (osd.1) 187 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:57 smithi055 bash[9296]: cluster 2020-09-08T16:26:56.454786+0000 mgr.x (mgr.24118) 1788 : cluster [DBG] pgmap v2623: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.451 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:57 smithi055 bash[9296]: cluster 2020-09-08T16:26:56.729079+0000 mon.a (mon.0) 2565 : cluster [DBG] osdmap e920: 12 total, 12 up, 12 in 2020-09-08T16:27:02.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:26:59 smithi055 bash[9296]: cluster 2020-09-08T16:26:58.455629+0000 mgr.x (mgr.24118) 1789 : cluster [DBG] pgmap v2625: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:01 smithi055 bash[9296]: cluster 2020-09-08T16:27:00.443440+0000 osd.6 (osd.6) 151 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:01 smithi055 bash[9296]: cluster 2020-09-08T16:27:00.443571+0000 osd.6 (osd.6) 152 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.452 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:01 smithi055 bash[9296]: cluster 2020-09-08T16:27:00.456380+0000 mgr.x (mgr.24118) 1790 : cluster [DBG] pgmap v2626: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.453 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:27:02.454 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:28 smithi104 bash[9203]: audit 2020-09-08T16:13:28.071148+0000 mon.a (mon.0) 2363 : audit [INF] from='client.? v1:172.21.15.104:0/622774130' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:27:02.454 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:29 smithi104 bash[9203]: cluster 2020-09-08T16:13:28.135669+0000 mon.a (mon.0) 2364 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:27:02.454 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:29 smithi104 bash[9203]: audit 2020-09-08T16:13:28.143990+0000 mon.a (mon.0) 2365 : audit [INF] from='client.? v1:172.21.15.104:0/622774130' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2020-09-08T16:27:02.454 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:29 smithi104 bash[9203]: cluster 2020-09-08T16:13:28.144039+0000 mon.a (mon.0) 2366 : cluster [DBG] osdmap e769: 12 total, 12 up, 12 in 2020-09-08T16:27:02.455 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:29 smithi104 bash[9203]: cluster 2020-09-08T16:13:28.171567+0000 mgr.x (mgr.24118) 1384 : cluster [DBG] pgmap v2069: 17 pgs: 17 active+clean; 208 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.17k op/s 2020-09-08T16:27:02.455 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:31 smithi104 bash[9203]: cluster 2020-09-08T16:13:30.172360+0000 mgr.x (mgr.24118) 1385 : cluster [DBG] pgmap v2070: 17 pgs: 17 active+clean; 64 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1.18k op/s 2020-09-08T16:27:02.455 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:32 smithi104 bash[9203]: audit 2020-09-08T16:13:31.118278+0000 mon.a (mon.0) 2367 : audit [DBG] from='client.? v1:172.21.15.104:0/3128233918' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:27:02.455 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:32 smithi104 bash[9203]: cluster 2020-09-08T16:13:31.359446+0000 osd.5 (osd.5) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.455 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:32 smithi104 bash[9203]: cluster 2020-09-08T16:13:31.359582+0000 osd.5 (osd.5) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.456 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:33 smithi104 bash[9203]: cluster 2020-09-08T16:13:32.124175+0000 mon.a (mon.0) 2368 : cluster [DBG] osdmap e770: 12 total, 12 up, 12 in 2020-09-08T16:27:02.456 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:33 smithi104 bash[9203]: cluster 2020-09-08T16:13:32.173034+0000 mgr.x (mgr.24118) 1386 : cluster [DBG] pgmap v2072: 17 pgs: 17 active+clean; 64 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 677 op/s 2020-09-08T16:27:02.456 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:34 smithi104 bash[9203]: cluster 2020-09-08T16:13:33.798526+0000 osd.10 (osd.10) 119 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.457 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:34 smithi104 bash[9203]: cluster 2020-09-08T16:13:33.798648+0000 osd.10 (osd.10) 120 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.457 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:35 smithi104 bash[9203]: cluster 2020-09-08T16:13:34.142489+0000 mon.a (mon.0) 2369 : cluster [DBG] osdmap e771: 12 total, 12 up, 12 in 2020-09-08T16:27:02.457 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:35 smithi104 bash[9203]: cluster 2020-09-08T16:13:34.173538+0000 mgr.x (mgr.24118) 1387 : cluster [DBG] pgmap v2074: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 552 op/s 2020-09-08T16:27:02.457 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:35 smithi104 bash[9203]: cluster 2020-09-08T16:13:34.798777+0000 osd.3 (osd.3) 143 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.457 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:35 smithi104 bash[9203]: cluster 2020-09-08T16:13:34.798895+0000 osd.3 (osd.3) 144 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.458 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:36 smithi104 bash[9203]: cluster 2020-09-08T16:13:35.150774+0000 mon.a (mon.0) 2370 : cluster [DBG] osdmap e772: 12 total, 12 up, 12 in 2020-09-08T16:27:02.458 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:37 smithi104 bash[9203]: cluster 2020-09-08T16:13:36.174104+0000 mgr.x (mgr.24118) 1388 : cluster [DBG] pgmap v2076: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 169 op/s 2020-09-08T16:27:02.458 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:39 smithi104 bash[9203]: cluster 2020-09-08T16:13:38.174794+0000 mgr.x (mgr.24118) 1389 : cluster [DBG] pgmap v2077: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 168 op/s 2020-09-08T16:27:02.458 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:41 smithi104 bash[9203]: cluster 2020-09-08T16:13:40.175536+0000 mgr.x (mgr.24118) 1390 : cluster [DBG] pgmap v2078: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 127 op/s 2020-09-08T16:27:02.458 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:43 smithi104 bash[9203]: cluster 2020-09-08T16:13:42.157827+0000 osd.8 (osd.8) 114 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.459 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:43 smithi104 bash[9203]: cluster 2020-09-08T16:13:42.157956+0000 osd.8 (osd.8) 115 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.459 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:43 smithi104 bash[9203]: cluster 2020-09-08T16:13:42.176237+0000 mgr.x (mgr.24118) 1391 : cluster [DBG] pgmap v2079: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.459 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:43 smithi104 bash[9203]: cluster 2020-09-08T16:13:42.218579+0000 mon.a (mon.0) 2371 : cluster [DBG] osdmap e773: 12 total, 12 up, 12 in 2020-09-08T16:27:02.459 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:45 smithi104 bash[9203]: cluster 2020-09-08T16:13:44.176996+0000 mgr.x (mgr.24118) 1392 : cluster [DBG] pgmap v2081: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.459 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:47 smithi104 bash[9203]: cluster 2020-09-08T16:13:46.177663+0000 mgr.x (mgr.24118) 1393 : cluster [DBG] pgmap v2082: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.460 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:49 smithi104 bash[9203]: cluster 2020-09-08T16:13:48.178391+0000 mgr.x (mgr.24118) 1394 : cluster [DBG] pgmap v2083: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.460 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:50 smithi104 bash[9203]: cluster 2020-09-08T16:13:49.567955+0000 osd.1 (osd.1) 160 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.460 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:50 smithi104 bash[9203]: cluster 2020-09-08T16:13:49.568088+0000 osd.1 (osd.1) 161 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.460 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:51 smithi104 bash[9203]: cluster 2020-09-08T16:13:50.179131+0000 mgr.x (mgr.24118) 1395 : cluster [DBG] pgmap v2084: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.460 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:51 smithi104 bash[9203]: cluster 2020-09-08T16:13:50.244784+0000 mon.a (mon.0) 2372 : cluster [DBG] osdmap e774: 12 total, 12 up, 12 in 2020-09-08T16:27:02.461 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:53 smithi104 bash[9203]: cluster 2020-09-08T16:13:52.179873+0000 mgr.x (mgr.24118) 1396 : cluster [DBG] pgmap v2086: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.461 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:54 smithi104 bash[9203]: cluster 2020-09-08T16:13:53.091527+0000 osd.6 (osd.6) 125 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.461 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:54 smithi104 bash[9203]: cluster 2020-09-08T16:13:53.091672+0000 osd.6 (osd.6) 126 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.461 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:54 smithi104 bash[9203]: cluster 2020-09-08T16:13:53.253232+0000 mon.a (mon.0) 2373 : cluster [DBG] osdmap e775: 12 total, 12 up, 12 in 2020-09-08T16:27:02.461 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:55 smithi104 bash[9203]: cluster 2020-09-08T16:13:54.180617+0000 mgr.x (mgr.24118) 1397 : cluster [DBG] pgmap v2088: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.462 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:57 smithi104 bash[9203]: cluster 2020-09-08T16:13:56.181327+0000 mgr.x (mgr.24118) 1398 : cluster [DBG] pgmap v2089: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.462 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:59 smithi104 bash[9203]: cluster 2020-09-08T16:13:58.182024+0000 mgr.x (mgr.24118) 1399 : cluster [DBG] pgmap v2090: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.462 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:59 smithi104 bash[9203]: cluster 2020-09-08T16:13:58.256754+0000 osd.11 (osd.11) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.462 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:59 smithi104 bash[9203]: cluster 2020-09-08T16:13:58.256880+0000 osd.11 (osd.11) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.462 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:13:59 smithi104 bash[9203]: cluster 2020-09-08T16:13:58.280736+0000 mon.a (mon.0) 2374 : cluster [DBG] osdmap e776: 12 total, 12 up, 12 in 2020-09-08T16:27:02.463 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:01 smithi104 bash[9203]: cluster 2020-09-08T16:14:00.182737+0000 mgr.x (mgr.24118) 1400 : cluster [DBG] pgmap v2092: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.463 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:03 smithi104 bash[9203]: cluster 2020-09-08T16:14:02.183359+0000 mgr.x (mgr.24118) 1401 : cluster [DBG] pgmap v2093: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.463 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:04 smithi104 bash[9203]: cluster 2020-09-08T16:14:02.929875+0000 osd.4 (osd.4) 85 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.463 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:04 smithi104 bash[9203]: cluster 2020-09-08T16:14:02.930001+0000 osd.4 (osd.4) 86 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.463 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:04 smithi104 bash[9203]: cluster 2020-09-08T16:14:03.251582+0000 mon.a (mon.0) 2375 : cluster [DBG] osdmap e777: 12 total, 12 up, 12 in 2020-09-08T16:27:02.464 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:05 smithi104 bash[9203]: cluster 2020-09-08T16:14:04.184018+0000 mgr.x (mgr.24118) 1402 : cluster [DBG] pgmap v2095: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.464 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:07 smithi104 bash[9203]: cluster 2020-09-08T16:14:06.184699+0000 mgr.x (mgr.24118) 1403 : cluster [DBG] pgmap v2096: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.464 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:09 smithi104 bash[9203]: cluster 2020-09-08T16:14:08.185381+0000 mgr.x (mgr.24118) 1404 : cluster [DBG] pgmap v2097: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.464 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:11 smithi104 bash[9203]: cluster 2020-09-08T16:14:10.186089+0000 mgr.x (mgr.24118) 1405 : cluster [DBG] pgmap v2098: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.465 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:12 smithi104 bash[9203]: cluster 2020-09-08T16:14:11.817319+0000 osd.9 (osd.9) 124 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.465 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:12 smithi104 bash[9203]: cluster 2020-09-08T16:14:11.817442+0000 osd.9 (osd.9) 125 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.465 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:13 smithi104 bash[9203]: cluster 2020-09-08T16:14:12.130806+0000 mon.a (mon.0) 2376 : cluster [DBG] osdmap e778: 12 total, 12 up, 12 in 2020-09-08T16:27:02.465 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:13 smithi104 bash[9203]: cluster 2020-09-08T16:14:12.186702+0000 mgr.x (mgr.24118) 1406 : cluster [DBG] pgmap v2100: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.465 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:14 smithi104 bash[9203]: cluster 2020-09-08T16:14:13.940062+0000 osd.2 (osd.2) 120 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.466 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:14 smithi104 bash[9203]: cluster 2020-09-08T16:14:13.940175+0000 osd.2 (osd.2) 121 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.466 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:15 smithi104 bash[9203]: cluster 2020-09-08T16:14:14.152050+0000 mon.a (mon.0) 2377 : cluster [DBG] osdmap e779: 12 total, 12 up, 12 in 2020-09-08T16:27:02.466 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:15 smithi104 bash[9203]: cluster 2020-09-08T16:14:14.187344+0000 mgr.x (mgr.24118) 1407 : cluster [DBG] pgmap v2102: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.466 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:17 smithi104 bash[9203]: cluster 2020-09-08T16:14:16.188022+0000 mgr.x (mgr.24118) 1408 : cluster [DBG] pgmap v2103: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.466 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:19 smithi104 bash[9203]: cluster 2020-09-08T16:14:18.188661+0000 mgr.x (mgr.24118) 1409 : cluster [DBG] pgmap v2104: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.467 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:21 smithi104 bash[9203]: cluster 2020-09-08T16:14:20.189351+0000 mgr.x (mgr.24118) 1410 : cluster [DBG] pgmap v2105: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.467 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:23 smithi104 bash[9203]: cluster 2020-09-08T16:14:22.190000+0000 mgr.x (mgr.24118) 1411 : cluster [DBG] pgmap v2106: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.467 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:23 smithi104 bash[9203]: cluster 2020-09-08T16:14:22.297120+0000 osd.7 (osd.7) 176 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.467 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:23 smithi104 bash[9203]: cluster 2020-09-08T16:14:22.297244+0000 osd.7 (osd.7) 177 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.467 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:23 smithi104 bash[9203]: cluster 2020-09-08T16:14:22.341740+0000 osd.0 (osd.0) 136 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.468 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:23 smithi104 bash[9203]: cluster 2020-09-08T16:14:22.341874+0000 osd.0 (osd.0) 137 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.468 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:24 smithi104 bash[9203]: cluster 2020-09-08T16:14:23.260670+0000 mon.a (mon.0) 2378 : cluster [DBG] osdmap e780: 12 total, 12 up, 12 in 2020-09-08T16:27:02.468 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:25 smithi104 bash[9203]: cluster 2020-09-08T16:14:24.190685+0000 mgr.x (mgr.24118) 1412 : cluster [DBG] pgmap v2108: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.468 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:27 smithi104 bash[9203]: cluster 2020-09-08T16:14:26.191372+0000 mgr.x (mgr.24118) 1413 : cluster [DBG] pgmap v2109: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.468 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:29 smithi104 bash[9203]: cluster 2020-09-08T16:14:28.192165+0000 mgr.x (mgr.24118) 1414 : cluster [DBG] pgmap v2110: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.469 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:31 smithi104 bash[9203]: cluster 2020-09-08T16:14:30.192746+0000 mgr.x (mgr.24118) 1415 : cluster [DBG] pgmap v2111: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.469 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:32 smithi104 bash[9203]: cluster 2020-09-08T16:14:31.379316+0000 osd.5 (osd.5) 142 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.469 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:32 smithi104 bash[9203]: cluster 2020-09-08T16:14:31.379453+0000 osd.5 (osd.5) 143 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.469 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:33 smithi104 bash[9203]: cluster 2020-09-08T16:14:32.130605+0000 mon.a (mon.0) 2379 : cluster [DBG] osdmap e781: 12 total, 12 up, 12 in 2020-09-08T16:27:02.469 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:33 smithi104 bash[9203]: cluster 2020-09-08T16:14:32.193317+0000 mgr.x (mgr.24118) 1416 : cluster [DBG] pgmap v2113: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.470 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:35 smithi104 bash[9203]: cluster 2020-09-08T16:14:34.194070+0000 mgr.x (mgr.24118) 1417 : cluster [DBG] pgmap v2114: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.470 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:35 smithi104 bash[9203]: cluster 2020-09-08T16:14:34.646280+0000 osd.10 (osd.10) 121 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.470 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:35 smithi104 bash[9203]: cluster 2020-09-08T16:14:34.646401+0000 osd.10 (osd.10) 122 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.470 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:36 smithi104 bash[9203]: cluster 2020-09-08T16:14:35.262897+0000 mon.a (mon.0) 2380 : cluster [DBG] osdmap e782: 12 total, 12 up, 12 in 2020-09-08T16:27:02.470 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:36 smithi104 bash[9203]: cluster 2020-09-08T16:14:35.622122+0000 osd.3 (osd.3) 145 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.471 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:36 smithi104 bash[9203]: cluster 2020-09-08T16:14:35.622231+0000 osd.3 (osd.3) 146 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.471 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:37 smithi104 bash[9203]: cluster 2020-09-08T16:14:36.194742+0000 mgr.x (mgr.24118) 1418 : cluster [DBG] pgmap v2116: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.471 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:37 smithi104 bash[9203]: cluster 2020-09-08T16:14:36.272137+0000 mon.a (mon.0) 2381 : cluster [DBG] osdmap e783: 12 total, 12 up, 12 in 2020-09-08T16:27:02.471 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:39 smithi104 bash[9203]: cluster 2020-09-08T16:14:38.195568+0000 mgr.x (mgr.24118) 1419 : cluster [DBG] pgmap v2118: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.471 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:41 smithi104 bash[9203]: cluster 2020-09-08T16:14:40.196276+0000 mgr.x (mgr.24118) 1420 : cluster [DBG] pgmap v2119: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.472 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:43 smithi104 bash[9203]: cluster 2020-09-08T16:14:42.196906+0000 mgr.x (mgr.24118) 1421 : cluster [DBG] pgmap v2120: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.472 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:44 smithi104 bash[9203]: cluster 2020-09-08T16:14:43.080311+0000 osd.8 (osd.8) 116 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.472 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:44 smithi104 bash[9203]: cluster 2020-09-08T16:14:43.080440+0000 osd.8 (osd.8) 117 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.472 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:44 smithi104 bash[9203]: cluster 2020-09-08T16:14:43.259076+0000 mon.a (mon.0) 2382 : cluster [DBG] osdmap e784: 12 total, 12 up, 12 in 2020-09-08T16:27:02.472 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:45 smithi104 bash[9203]: cluster 2020-09-08T16:14:44.197556+0000 mgr.x (mgr.24118) 1422 : cluster [DBG] pgmap v2122: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.473 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:47 smithi104 bash[9203]: cluster 2020-09-08T16:14:46.198236+0000 mgr.x (mgr.24118) 1423 : cluster [DBG] pgmap v2123: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.473 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:48 smithi104 bash[9203]: audit 2020-09-08T16:14:48.568197+0000 mon.b (mon.1) 125 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:14:48.567054\"}"}]: dispatch 2020-09-08T16:27:02.473 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:48 smithi104 bash[9203]: audit 2020-09-08T16:14:48.570330+0000 mon.a (mon.0) 2383 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:14:48.567054\"}"}]: dispatch 2020-09-08T16:27:02.474 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:48 smithi104 bash[9203]: audit 2020-09-08T16:14:48.576218+0000 mon.a (mon.0) 2384 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786049\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786200\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.786770\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787175\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:07.787475\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:04:07.787829\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:14:48.567054\"}"}]': finished 2020-09-08T16:27:02.474 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:48 smithi104 bash[9203]: audit 2020-09-08T16:14:48.576596+0000 mon.b (mon.1) 126 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:27:02.475 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:48 smithi104 bash[9203]: audit 2020-09-08T16:14:48.603681+0000 mon.b (mon.1) 127 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:14:48.602227\"}"}]: dispatch 2020-09-08T16:27:02.475 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:48 smithi104 bash[9203]: audit 2020-09-08T16:14:48.605772+0000 mon.a (mon.0) 2385 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:14:48.602227\"}"}]: dispatch 2020-09-08T16:27:02.475 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:48 smithi104 bash[9203]: audit 2020-09-08T16:14:48.610350+0000 mon.a (mon.0) 2386 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550147\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550279\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.550802\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551150\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.551491\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.551914\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:14:48.602227\"}"}]': finished 2020-09-08T16:27:02.476 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:48 smithi104 bash[9203]: audit 2020-09-08T16:14:48.611699+0000 mon.b (mon.1) 128 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:27:02.476 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:49 smithi104 bash[9203]: audit 2020-09-08T16:14:48.637950+0000 mon.b (mon.1) 129 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:14:48.636423\"}"}]: dispatch 2020-09-08T16:27:02.476 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:49 smithi104 bash[9203]: audit 2020-09-08T16:14:48.639785+0000 mon.a (mon.0) 2387 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:14:48.636423\"}"}]: dispatch 2020-09-08T16:27:02.477 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:49 smithi104 bash[9203]: audit 2020-09-08T16:14:48.644222+0000 mon.a (mon.0) 2388 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213459\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T16:04:08.213602\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.213686\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214195\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214559\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:04:08.214905\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:04:08.215385\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:14:48.636423\"}"}]': finished 2020-09-08T16:27:02.477 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:49 smithi104 bash[9203]: audit 2020-09-08T16:14:48.644329+0000 mon.b (mon.1) 130 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:27:02.477 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:49 smithi104 bash[9203]: audit 2020-09-08T16:14:49.407449+0000 mon.b (mon.1) 131 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.402933\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403132\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403786\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404332\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404856\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.405550\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:14:48.602227\"}"}]: dispatch 2020-09-08T16:27:02.478 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:49 smithi104 bash[9203]: audit 2020-09-08T16:14:49.409621+0000 mon.a (mon.0) 2389 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.402933\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403132\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403786\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404332\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404856\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.405550\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:14:48.602227\"}"}]: dispatch 2020-09-08T16:27:02.478 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:49 smithi104 bash[9203]: audit 2020-09-08T16:14:49.414676+0000 mon.a (mon.0) 2390 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.402933\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403132\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403786\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404332\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404856\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.405550\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:14:48.602227\"}"}]': finished 2020-09-08T16:27:02.479 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:50 smithi104 bash[9203]: audit 2020-09-08T16:14:49.688909+0000 mon.b (mon.1) 132 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685123\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685289\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685873\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686336\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686892\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.687437\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:14:48.567054\"}"}]: dispatch 2020-09-08T16:27:02.479 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:50 smithi104 bash[9203]: audit 2020-09-08T16:14:49.690911+0000 mon.a (mon.0) 2391 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685123\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685289\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685873\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686336\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686892\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.687437\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:14:48.567054\"}"}]: dispatch 2020-09-08T16:27:02.479 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:50 smithi104 bash[9203]: audit 2020-09-08T16:14:49.695568+0000 mon.a (mon.0) 2392 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685123\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685289\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685873\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686336\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686892\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.687437\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:14:48.567054\"}"}]': finished 2020-09-08T16:27:02.480 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:50 smithi104 bash[9203]: audit 2020-09-08T16:14:49.701035+0000 mon.b (mon.1) 133 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698864\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698956\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699006\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699331\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699539\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699740\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.700002\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:14:48.636423\"}"}]: dispatch 2020-09-08T16:27:02.480 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:50 smithi104 bash[9203]: audit 2020-09-08T16:14:49.703174+0000 mon.a (mon.0) 2393 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698864\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698956\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699006\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699331\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699539\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699740\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.700002\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:14:48.636423\"}"}]: dispatch 2020-09-08T16:27:02.480 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:50 smithi104 bash[9203]: audit 2020-09-08T16:14:49.709043+0000 mon.a (mon.0) 2394 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698864\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698956\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699006\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699331\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699539\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699740\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.700002\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:14:48.636423\"}"}]': finished 2020-09-08T16:27:02.481 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:50 smithi104 bash[9203]: audit 2020-09-08T16:14:49.710077+0000 mon.b (mon.1) 134 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.481 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:50 smithi104 bash[9203]: audit 2020-09-08T16:14:49.711734+0000 mon.a (mon.0) 2395 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.481 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:50 smithi104 bash[9203]: audit 2020-09-08T16:14:49.716978+0000 mon.a (mon.0) 2396 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.481 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:50 smithi104 bash[9203]: audit 2020-09-08T16:14:49.719847+0000 mon.b (mon.1) 135 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.482 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:50 smithi104 bash[9203]: audit 2020-09-08T16:14:49.721369+0000 mon.a (mon.0) 2397 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.482 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:50 smithi104 bash[9203]: audit 2020-09-08T16:14:49.726604+0000 mon.a (mon.0) 2398 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.482 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:51 smithi104 bash[9203]: cluster 2020-09-08T16:14:50.199752+0000 mgr.x (mgr.24118) 1425 : cluster [DBG] pgmap v2125: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.482 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:51 smithi104 bash[9203]: cluster 2020-09-08T16:14:50.324087+0000 osd.1 (osd.1) 162 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.482 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:51 smithi104 bash[9203]: cluster 2020-09-08T16:14:50.324226+0000 osd.1 (osd.1) 163 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.483 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:51 smithi104 bash[9203]: cluster 2020-09-08T16:14:51.140484+0000 mon.a (mon.0) 2399 : cluster [DBG] osdmap e785: 12 total, 12 up, 12 in 2020-09-08T16:27:02.483 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:53 smithi104 bash[9203]: cluster 2020-09-08T16:14:52.200469+0000 mgr.x (mgr.24118) 1426 : cluster [DBG] pgmap v2127: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.483 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:55 smithi104 bash[9203]: cluster 2020-09-08T16:14:53.964831+0000 osd.6 (osd.6) 127 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.484 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:55 smithi104 bash[9203]: cluster 2020-09-08T16:14:53.964994+0000 osd.6 (osd.6) 128 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.484 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:55 smithi104 bash[9203]: cluster 2020-09-08T16:14:54.201201+0000 mgr.x (mgr.24118) 1427 : cluster [DBG] pgmap v2128: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.484 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:55 smithi104 bash[9203]: cluster 2020-09-08T16:14:54.656765+0000 mon.a (mon.0) 2400 : cluster [DBG] osdmap e786: 12 total, 12 up, 12 in 2020-09-08T16:27:02.484 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:55 smithi104 bash[9203]: audit 2020-09-08T16:14:55.398925+0000 mon.c (mon.2) 97 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.484 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:55 smithi104 bash[9203]: audit 2020-09-08T16:14:55.402185+0000 mon.a (mon.0) 2401 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.485 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:55 smithi104 bash[9203]: audit 2020-09-08T16:14:55.408299+0000 mon.a (mon.0) 2402 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.485 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:55 smithi104 bash[9203]: audit 2020-09-08T16:14:55.411017+0000 mon.c (mon.2) 98 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.485 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:55 smithi104 bash[9203]: audit 2020-09-08T16:14:55.414225+0000 mon.a (mon.0) 2403 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.485 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:55 smithi104 bash[9203]: audit 2020-09-08T16:14:55.421107+0000 mon.a (mon.0) 2404 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.486 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:57 smithi104 bash[9203]: cluster 2020-09-08T16:14:56.201953+0000 mgr.x (mgr.24118) 1428 : cluster [DBG] pgmap v2130: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.486 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:14:59 smithi104 bash[9203]: cluster 2020-09-08T16:14:58.202734+0000 mgr.x (mgr.24118) 1429 : cluster [DBG] pgmap v2131: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.486 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:00 smithi104 bash[9203]: cluster 2020-09-08T16:14:59.199080+0000 osd.11 (osd.11) 142 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.486 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:00 smithi104 bash[9203]: cluster 2020-09-08T16:14:59.199225+0000 osd.11 (osd.11) 143 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.486 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:00 smithi104 bash[9203]: cluster 2020-09-08T16:14:59.671408+0000 mon.a (mon.0) 2405 : cluster [DBG] osdmap e787: 12 total, 12 up, 12 in 2020-09-08T16:27:02.487 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:01 smithi104 bash[9203]: cluster 2020-09-08T16:15:00.203450+0000 mgr.x (mgr.24118) 1430 : cluster [DBG] pgmap v2133: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.487 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:03 smithi104 bash[9203]: cluster 2020-09-08T16:15:02.204091+0000 mgr.x (mgr.24118) 1431 : cluster [DBG] pgmap v2134: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.487 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:03 smithi104 bash[9203]: cluster 2020-09-08T16:15:03.085020+0000 osd.4 (osd.4) 87 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.487 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:03 smithi104 bash[9203]: cluster 2020-09-08T16:15:03.085154+0000 osd.4 (osd.4) 88 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.487 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:04 smithi104 bash[9203]: cluster 2020-09-08T16:15:03.706936+0000 mon.a (mon.0) 2406 : cluster [DBG] osdmap e788: 12 total, 12 up, 12 in 2020-09-08T16:27:02.488 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:05 smithi104 bash[9203]: cluster 2020-09-08T16:15:04.204759+0000 mgr.x (mgr.24118) 1432 : cluster [DBG] pgmap v2136: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.488 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:07 smithi104 bash[9203]: cluster 2020-09-08T16:15:06.205426+0000 mgr.x (mgr.24118) 1433 : cluster [DBG] pgmap v2137: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.488 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:09 smithi104 bash[9203]: cluster 2020-09-08T16:15:08.206133+0000 mgr.x (mgr.24118) 1434 : cluster [DBG] pgmap v2138: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.488 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:11 smithi104 bash[9203]: cluster 2020-09-08T16:15:10.206827+0000 mgr.x (mgr.24118) 1435 : cluster [DBG] pgmap v2139: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.488 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:12 smithi104 bash[9203]: cluster 2020-09-08T16:15:11.845269+0000 osd.9 (osd.9) 126 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.489 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:12 smithi104 bash[9203]: cluster 2020-09-08T16:15:11.845394+0000 osd.9 (osd.9) 127 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.489 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:13 smithi104 bash[9203]: cluster 2020-09-08T16:15:12.141685+0000 mon.a (mon.0) 2407 : cluster [DBG] osdmap e789: 12 total, 12 up, 12 in 2020-09-08T16:27:02.489 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:13 smithi104 bash[9203]: cluster 2020-09-08T16:15:12.207470+0000 mgr.x (mgr.24118) 1436 : cluster [DBG] pgmap v2141: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.489 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:15 smithi104 bash[9203]: cluster 2020-09-08T16:15:14.208181+0000 mgr.x (mgr.24118) 1437 : cluster [DBG] pgmap v2142: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.489 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:15 smithi104 bash[9203]: cluster 2020-09-08T16:15:14.257343+0000 osd.2 (osd.2) 122 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.490 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:15 smithi104 bash[9203]: cluster 2020-09-08T16:15:14.257461+0000 osd.2 (osd.2) 123 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.490 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:15 smithi104 bash[9203]: cluster 2020-09-08T16:15:14.314797+0000 mon.a (mon.0) 2408 : cluster [DBG] osdmap e790: 12 total, 12 up, 12 in 2020-09-08T16:27:02.490 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:17 smithi104 bash[9203]: cluster 2020-09-08T16:15:16.208862+0000 mgr.x (mgr.24118) 1438 : cluster [DBG] pgmap v2144: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.490 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:19 smithi104 bash[9203]: cluster 2020-09-08T16:15:18.209628+0000 mgr.x (mgr.24118) 1439 : cluster [DBG] pgmap v2145: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.490 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:21 smithi104 bash[9203]: cluster 2020-09-08T16:15:20.210367+0000 mgr.x (mgr.24118) 1440 : cluster [DBG] pgmap v2146: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.491 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:23 smithi104 bash[9203]: cluster 2020-09-08T16:15:22.211026+0000 mgr.x (mgr.24118) 1441 : cluster [DBG] pgmap v2147: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.491 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:23 smithi104 bash[9203]: cluster 2020-09-08T16:15:22.554620+0000 osd.7 (osd.7) 178 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.491 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:23 smithi104 bash[9203]: cluster 2020-09-08T16:15:22.554737+0000 osd.7 (osd.7) 179 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.491 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:23 smithi104 bash[9203]: cluster 2020-09-08T16:15:22.661421+0000 osd.0 (osd.0) 138 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.491 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:23 smithi104 bash[9203]: cluster 2020-09-08T16:15:22.661556+0000 osd.0 (osd.0) 139 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.492 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:24 smithi104 bash[9203]: cluster 2020-09-08T16:15:23.285820+0000 mon.a (mon.0) 2409 : cluster [DBG] osdmap e791: 12 total, 12 up, 12 in 2020-09-08T16:27:02.492 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:25 smithi104 bash[9203]: cluster 2020-09-08T16:15:24.211726+0000 mgr.x (mgr.24118) 1442 : cluster [DBG] pgmap v2149: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.492 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:27 smithi104 bash[9203]: cluster 2020-09-08T16:15:26.212433+0000 mgr.x (mgr.24118) 1443 : cluster [DBG] pgmap v2150: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.492 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:29 smithi104 bash[9203]: cluster 2020-09-08T16:15:28.213218+0000 mgr.x (mgr.24118) 1444 : cluster [DBG] pgmap v2151: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.492 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:31 smithi104 bash[9203]: cluster 2020-09-08T16:15:30.213899+0000 mgr.x (mgr.24118) 1445 : cluster [DBG] pgmap v2152: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.493 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:33 smithi104 bash[9203]: cluster 2020-09-08T16:15:32.136858+0000 osd.5 (osd.5) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.493 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:33 smithi104 bash[9203]: cluster 2020-09-08T16:15:32.137014+0000 osd.5 (osd.5) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.493 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:33 smithi104 bash[9203]: cluster 2020-09-08T16:15:32.149772+0000 mon.a (mon.0) 2410 : cluster [DBG] osdmap e792: 12 total, 12 up, 12 in 2020-09-08T16:27:02.493 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:33 smithi104 bash[9203]: cluster 2020-09-08T16:15:32.214578+0000 mgr.x (mgr.24118) 1446 : cluster [DBG] pgmap v2154: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.493 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:35 smithi104 bash[9203]: cluster 2020-09-08T16:15:34.215255+0000 mgr.x (mgr.24118) 1447 : cluster [DBG] pgmap v2155: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.494 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:35 smithi104 bash[9203]: cluster 2020-09-08T16:15:34.709795+0000 osd.10 (osd.10) 123 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.494 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:35 smithi104 bash[9203]: cluster 2020-09-08T16:15:34.709916+0000 osd.10 (osd.10) 124 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.494 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:36 smithi104 bash[9203]: cluster 2020-09-08T16:15:35.279221+0000 mon.a (mon.0) 2411 : cluster [DBG] osdmap e793: 12 total, 12 up, 12 in 2020-09-08T16:27:02.494 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:36 smithi104 bash[9203]: cluster 2020-09-08T16:15:36.132157+0000 osd.3 (osd.3) 147 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.494 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:36 smithi104 bash[9203]: cluster 2020-09-08T16:15:36.132304+0000 osd.3 (osd.3) 148 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.495 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:37 smithi104 bash[9203]: cluster 2020-09-08T16:15:36.215863+0000 mgr.x (mgr.24118) 1448 : cluster [DBG] pgmap v2157: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.495 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:37 smithi104 bash[9203]: cluster 2020-09-08T16:15:36.298356+0000 mon.a (mon.0) 2412 : cluster [DBG] osdmap e794: 12 total, 12 up, 12 in 2020-09-08T16:27:02.495 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:39 smithi104 bash[9203]: cluster 2020-09-08T16:15:38.216592+0000 mgr.x (mgr.24118) 1449 : cluster [DBG] pgmap v2159: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.495 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:41 smithi104 bash[9203]: cluster 2020-09-08T16:15:40.217245+0000 mgr.x (mgr.24118) 1450 : cluster [DBG] pgmap v2160: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.495 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:43 smithi104 bash[9203]: cluster 2020-09-08T16:15:42.217849+0000 mgr.x (mgr.24118) 1451 : cluster [DBG] pgmap v2161: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.496 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:44 smithi104 bash[9203]: audit 2020-09-08T16:15:43.297700+0000 mon.a (mon.0) 2413 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.496 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:44 smithi104 bash[9203]: audit 2020-09-08T16:15:43.303045+0000 mon.a (mon.0) 2414 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.496 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:45 smithi104 bash[9203]: cluster 2020-09-08T16:15:43.996351+0000 osd.8 (osd.8) 118 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.496 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:45 smithi104 bash[9203]: cluster 2020-09-08T16:15:43.996507+0000 osd.8 (osd.8) 119 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.496 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:45 smithi104 bash[9203]: cluster 2020-09-08T16:15:44.218673+0000 mgr.x (mgr.24118) 1452 : cluster [DBG] pgmap v2162: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.497 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:45 smithi104 bash[9203]: cluster 2020-09-08T16:15:44.311575+0000 mon.a (mon.0) 2415 : cluster [DBG] osdmap e795: 12 total, 12 up, 12 in 2020-09-08T16:27:02.497 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:47 smithi104 bash[9203]: cluster 2020-09-08T16:15:46.219382+0000 mgr.x (mgr.24118) 1453 : cluster [DBG] pgmap v2164: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.497 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:49 smithi104 bash[9203]: cluster 2020-09-08T16:15:48.220093+0000 mgr.x (mgr.24118) 1454 : cluster [DBG] pgmap v2165: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.497 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:51 smithi104 bash[9203]: cluster 2020-09-08T16:15:50.220756+0000 mgr.x (mgr.24118) 1455 : cluster [DBG] pgmap v2166: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.497 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:51 smithi104 bash[9203]: cluster 2020-09-08T16:15:50.365951+0000 osd.1 (osd.1) 164 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.498 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:51 smithi104 bash[9203]: cluster 2020-09-08T16:15:50.366086+0000 osd.1 (osd.1) 165 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.498 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:52 smithi104 bash[9203]: cluster 2020-09-08T16:15:51.154801+0000 mon.a (mon.0) 2416 : cluster [DBG] osdmap e796: 12 total, 12 up, 12 in 2020-09-08T16:27:02.498 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:53 smithi104 bash[9203]: cluster 2020-09-08T16:15:52.221434+0000 mgr.x (mgr.24118) 1456 : cluster [DBG] pgmap v2168: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.498 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:55 smithi104 bash[9203]: cluster 2020-09-08T16:15:54.222100+0000 mgr.x (mgr.24118) 1457 : cluster [DBG] pgmap v2169: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.498 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:56 smithi104 bash[9203]: cluster 2020-09-08T16:15:54.794063+0000 osd.6 (osd.6) 129 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.499 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:56 smithi104 bash[9203]: cluster 2020-09-08T16:15:54.794193+0000 osd.6 (osd.6) 130 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.499 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:56 smithi104 bash[9203]: cluster 2020-09-08T16:15:55.292870+0000 mon.a (mon.0) 2417 : cluster [DBG] osdmap e797: 12 total, 12 up, 12 in 2020-09-08T16:27:02.499 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:57 smithi104 bash[9203]: cluster 2020-09-08T16:15:56.222752+0000 mgr.x (mgr.24118) 1458 : cluster [DBG] pgmap v2171: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.499 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:15:59 smithi104 bash[9203]: cluster 2020-09-08T16:15:58.223435+0000 mgr.x (mgr.24118) 1459 : cluster [DBG] pgmap v2172: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.499 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:01 smithi104 bash[9203]: cluster 2020-09-08T16:16:00.114018+0000 osd.11 (osd.11) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.500 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:01 smithi104 bash[9203]: cluster 2020-09-08T16:16:00.114147+0000 osd.11 (osd.11) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.500 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:01 smithi104 bash[9203]: cluster 2020-09-08T16:16:00.224131+0000 mgr.x (mgr.24118) 1460 : cluster [DBG] pgmap v2173: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.500 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:01 smithi104 bash[9203]: cluster 2020-09-08T16:16:00.318210+0000 mon.a (mon.0) 2418 : cluster [DBG] osdmap e798: 12 total, 12 up, 12 in 2020-09-08T16:27:02.500 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:03 smithi104 bash[9203]: cluster 2020-09-08T16:16:02.224877+0000 mgr.x (mgr.24118) 1461 : cluster [DBG] pgmap v2175: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.500 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:04 smithi104 bash[9203]: cluster 2020-09-08T16:16:03.431457+0000 osd.4 (osd.4) 89 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.501 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:04 smithi104 bash[9203]: cluster 2020-09-08T16:16:03.431583+0000 osd.4 (osd.4) 90 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.501 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:05 smithi104 bash[9203]: cluster 2020-09-08T16:16:04.225590+0000 mgr.x (mgr.24118) 1462 : cluster [DBG] pgmap v2176: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.501 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:05 smithi104 bash[9203]: cluster 2020-09-08T16:16:04.339098+0000 mon.a (mon.0) 2419 : cluster [DBG] osdmap e799: 12 total, 12 up, 12 in 2020-09-08T16:27:02.501 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:07 smithi104 bash[9203]: cluster 2020-09-08T16:16:06.226246+0000 mgr.x (mgr.24118) 1463 : cluster [DBG] pgmap v2178: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.501 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:09 smithi104 bash[9203]: cluster 2020-09-08T16:16:08.226976+0000 mgr.x (mgr.24118) 1464 : cluster [DBG] pgmap v2179: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.502 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:11 smithi104 bash[9203]: cluster 2020-09-08T16:16:10.227554+0000 mgr.x (mgr.24118) 1465 : cluster [DBG] pgmap v2180: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.502 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:12 smithi104 bash[9203]: cluster 2020-09-08T16:16:11.941232+0000 osd.9 (osd.9) 128 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.502 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:12 smithi104 bash[9203]: cluster 2020-09-08T16:16:11.941368+0000 osd.9 (osd.9) 129 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.502 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:13 smithi104 bash[9203]: cluster 2020-09-08T16:16:12.164050+0000 mon.a (mon.0) 2420 : cluster [DBG] osdmap e800: 12 total, 12 up, 12 in 2020-09-08T16:27:02.502 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:13 smithi104 bash[9203]: cluster 2020-09-08T16:16:12.228176+0000 mgr.x (mgr.24118) 1466 : cluster [DBG] pgmap v2182: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.503 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:15 smithi104 bash[9203]: cluster 2020-09-08T16:16:14.228699+0000 mgr.x (mgr.24118) 1467 : cluster [DBG] pgmap v2183: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.503 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:15 smithi104 bash[9203]: cluster 2020-09-08T16:16:14.413784+0000 osd.2 (osd.2) 124 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.503 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:15 smithi104 bash[9203]: cluster 2020-09-08T16:16:14.413922+0000 osd.2 (osd.2) 125 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.503 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:16 smithi104 bash[9203]: cluster 2020-09-08T16:16:15.293924+0000 mon.a (mon.0) 2421 : cluster [DBG] osdmap e801: 12 total, 12 up, 12 in 2020-09-08T16:27:02.503 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:17 smithi104 bash[9203]: cluster 2020-09-08T16:16:16.229284+0000 mgr.x (mgr.24118) 1468 : cluster [DBG] pgmap v2185: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.504 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:19 smithi104 bash[9203]: cluster 2020-09-08T16:16:18.230105+0000 mgr.x (mgr.24118) 1469 : cluster [DBG] pgmap v2186: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.504 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:21 smithi104 bash[9203]: cluster 2020-09-08T16:16:20.230600+0000 mgr.x (mgr.24118) 1470 : cluster [DBG] pgmap v2187: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.504 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:23 smithi104 bash[9203]: cluster 2020-09-08T16:16:22.231310+0000 mgr.x (mgr.24118) 1471 : cluster [DBG] pgmap v2188: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.504 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:24 smithi104 bash[9203]: cluster 2020-09-08T16:16:22.893870+0000 osd.7 (osd.7) 180 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.504 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:24 smithi104 bash[9203]: cluster 2020-09-08T16:16:22.893995+0000 osd.7 (osd.7) 181 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.505 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:24 smithi104 bash[9203]: cluster 2020-09-08T16:16:23.295140+0000 mon.a (mon.0) 2422 : cluster [DBG] osdmap e802: 12 total, 12 up, 12 in 2020-09-08T16:27:02.505 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:24 smithi104 bash[9203]: cluster 2020-09-08T16:16:23.610710+0000 osd.0 (osd.0) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.505 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:24 smithi104 bash[9203]: cluster 2020-09-08T16:16:23.610833+0000 osd.0 (osd.0) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.505 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:25 smithi104 bash[9203]: cluster 2020-09-08T16:16:24.231993+0000 mgr.x (mgr.24118) 1472 : cluster [DBG] pgmap v2190: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.505 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:25 smithi104 bash[9203]: cluster 2020-09-08T16:16:24.320596+0000 mon.a (mon.0) 2423 : cluster [DBG] osdmap e803: 12 total, 12 up, 12 in 2020-09-08T16:27:02.506 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:27 smithi104 bash[9203]: cluster 2020-09-08T16:16:26.232468+0000 mgr.x (mgr.24118) 1473 : cluster [DBG] pgmap v2192: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.506 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:29 smithi104 bash[9203]: cluster 2020-09-08T16:16:28.233181+0000 mgr.x (mgr.24118) 1474 : cluster [DBG] pgmap v2193: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.506 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:31 smithi104 bash[9203]: cluster 2020-09-08T16:16:30.233865+0000 mgr.x (mgr.24118) 1475 : cluster [DBG] pgmap v2194: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.507 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:33 smithi104 bash[9203]: cluster 2020-09-08T16:16:32.234550+0000 mgr.x (mgr.24118) 1476 : cluster [DBG] pgmap v2195: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.507 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:33 smithi104 bash[9203]: cluster 2020-09-08T16:16:32.555375+0000 osd.5 (osd.5) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.507 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:33 smithi104 bash[9203]: cluster 2020-09-08T16:16:32.555503+0000 osd.5 (osd.5) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.507 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:34 smithi104 bash[9203]: cluster 2020-09-08T16:16:33.304105+0000 mon.a (mon.0) 2424 : cluster [DBG] osdmap e804: 12 total, 12 up, 12 in 2020-09-08T16:27:02.507 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:35 smithi104 bash[9203]: cluster 2020-09-08T16:16:34.235182+0000 mgr.x (mgr.24118) 1477 : cluster [DBG] pgmap v2197: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.508 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:35 smithi104 bash[9203]: cluster 2020-09-08T16:16:34.831684+0000 osd.10 (osd.10) 125 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.508 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:35 smithi104 bash[9203]: cluster 2020-09-08T16:16:34.831801+0000 osd.10 (osd.10) 126 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.508 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:36 smithi104 bash[9203]: cluster 2020-09-08T16:16:35.364819+0000 mon.a (mon.0) 2425 : cluster [DBG] osdmap e805: 12 total, 12 up, 12 in 2020-09-08T16:27:02.508 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:37 smithi104 bash[9203]: cluster 2020-09-08T16:16:36.235834+0000 mgr.x (mgr.24118) 1478 : cluster [DBG] pgmap v2199: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.508 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:37 smithi104 bash[9203]: cluster 2020-09-08T16:16:36.286746+0000 osd.3 (osd.3) 149 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.509 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:37 smithi104 bash[9203]: cluster 2020-09-08T16:16:36.286865+0000 osd.3 (osd.3) 150 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.509 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:37 smithi104 bash[9203]: cluster 2020-09-08T16:16:36.395282+0000 mon.a (mon.0) 2426 : cluster [DBG] osdmap e806: 12 total, 12 up, 12 in 2020-09-08T16:27:02.509 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:39 smithi104 bash[9203]: cluster 2020-09-08T16:16:38.236525+0000 mgr.x (mgr.24118) 1479 : cluster [DBG] pgmap v2201: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.509 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:41 smithi104 bash[9203]: cluster 2020-09-08T16:16:40.237222+0000 mgr.x (mgr.24118) 1480 : cluster [DBG] pgmap v2202: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.509 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:43 smithi104 bash[9203]: cluster 2020-09-08T16:16:42.237909+0000 mgr.x (mgr.24118) 1481 : cluster [DBG] pgmap v2203: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.510 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:45 smithi104 bash[9203]: cluster 2020-09-08T16:16:44.238674+0000 mgr.x (mgr.24118) 1482 : cluster [DBG] pgmap v2204: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.510 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:45 smithi104 bash[9203]: cluster 2020-09-08T16:16:44.342159+0000 osd.8 (osd.8) 120 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.510 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:45 smithi104 bash[9203]: cluster 2020-09-08T16:16:44.342307+0000 osd.8 (osd.8) 121 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.510 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:46 smithi104 bash[9203]: cluster 2020-09-08T16:16:45.315881+0000 mon.a (mon.0) 2427 : cluster [DBG] osdmap e807: 12 total, 12 up, 12 in 2020-09-08T16:27:02.510 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:47 smithi104 bash[9203]: cluster 2020-09-08T16:16:46.239313+0000 mgr.x (mgr.24118) 1483 : cluster [DBG] pgmap v2206: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.511 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:49 smithi104 bash[9203]: cluster 2020-09-08T16:16:48.240031+0000 mgr.x (mgr.24118) 1484 : cluster [DBG] pgmap v2207: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.511 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:51 smithi104 bash[9203]: cluster 2020-09-08T16:16:50.240728+0000 mgr.x (mgr.24118) 1485 : cluster [DBG] pgmap v2208: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.511 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:52 smithi104 bash[9203]: cluster 2020-09-08T16:16:51.121318+0000 osd.1 (osd.1) 166 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.511 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:52 smithi104 bash[9203]: cluster 2020-09-08T16:16:51.121449+0000 osd.1 (osd.1) 167 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.511 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:52 smithi104 bash[9203]: cluster 2020-09-08T16:16:51.178890+0000 mon.a (mon.0) 2428 : cluster [DBG] osdmap e808: 12 total, 12 up, 12 in 2020-09-08T16:27:02.512 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:53 smithi104 bash[9203]: cluster 2020-09-08T16:16:52.241350+0000 mgr.x (mgr.24118) 1486 : cluster [DBG] pgmap v2210: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.512 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:55 smithi104 bash[9203]: cluster 2020-09-08T16:16:54.242071+0000 mgr.x (mgr.24118) 1487 : cluster [DBG] pgmap v2211: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.513 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:56 smithi104 bash[9203]: cluster 2020-09-08T16:16:55.202047+0000 osd.6 (osd.6) 131 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.513 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:56 smithi104 bash[9203]: cluster 2020-09-08T16:16:55.202190+0000 osd.6 (osd.6) 132 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.513 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:56 smithi104 bash[9203]: cluster 2020-09-08T16:16:56.176655+0000 mon.a (mon.0) 2429 : cluster [DBG] osdmap e809: 12 total, 12 up, 12 in 2020-09-08T16:27:02.513 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:57 smithi104 bash[9203]: cluster 2020-09-08T16:16:56.242697+0000 mgr.x (mgr.24118) 1488 : cluster [DBG] pgmap v2213: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.513 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:16:59 smithi104 bash[9203]: cluster 2020-09-08T16:16:58.243347+0000 mgr.x (mgr.24118) 1489 : cluster [DBG] pgmap v2214: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.514 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:01 smithi104 bash[9203]: cluster 2020-09-08T16:17:00.244025+0000 mgr.x (mgr.24118) 1490 : cluster [DBG] pgmap v2215: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.514 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:01 smithi104 bash[9203]: cluster 2020-09-08T16:17:00.467924+0000 osd.11 (osd.11) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.514 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:01 smithi104 bash[9203]: cluster 2020-09-08T16:17:00.468086+0000 osd.11 (osd.11) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.514 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:02 smithi104 bash[9203]: cluster 2020-09-08T16:17:01.193296+0000 mon.a (mon.0) 2430 : cluster [DBG] osdmap e810: 12 total, 12 up, 12 in 2020-09-08T16:27:02.515 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:03 smithi104 bash[9203]: cluster 2020-09-08T16:17:02.244701+0000 mgr.x (mgr.24118) 1491 : cluster [DBG] pgmap v2217: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.515 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:05 smithi104 bash[9203]: cluster 2020-09-08T16:17:03.887171+0000 osd.4 (osd.4) 91 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.515 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:05 smithi104 bash[9203]: cluster 2020-09-08T16:17:03.887298+0000 osd.4 (osd.4) 92 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.515 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:05 smithi104 bash[9203]: cluster 2020-09-08T16:17:04.212140+0000 mon.a (mon.0) 2431 : cluster [DBG] osdmap e811: 12 total, 12 up, 12 in 2020-09-08T16:27:02.515 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:05 smithi104 bash[9203]: cluster 2020-09-08T16:17:04.245303+0000 mgr.x (mgr.24118) 1492 : cluster [DBG] pgmap v2219: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.515 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:07 smithi104 bash[9203]: cluster 2020-09-08T16:17:06.245996+0000 mgr.x (mgr.24118) 1493 : cluster [DBG] pgmap v2220: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.516 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:09 smithi104 bash[9203]: cluster 2020-09-08T16:17:08.246741+0000 mgr.x (mgr.24118) 1494 : cluster [DBG] pgmap v2221: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.516 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:11 smithi104 bash[9203]: cluster 2020-09-08T16:17:10.247372+0000 mgr.x (mgr.24118) 1495 : cluster [DBG] pgmap v2222: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.516 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:13 smithi104 bash[9203]: cluster 2020-09-08T16:17:12.248035+0000 mgr.x (mgr.24118) 1496 : cluster [DBG] pgmap v2223: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.516 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:13 smithi104 bash[9203]: cluster 2020-09-08T16:17:12.903835+0000 osd.9 (osd.9) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.516 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:13 smithi104 bash[9203]: cluster 2020-09-08T16:17:12.903962+0000 osd.9 (osd.9) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.517 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:14 smithi104 bash[9203]: cluster 2020-09-08T16:17:13.310006+0000 mon.a (mon.0) 2432 : cluster [DBG] osdmap e812: 12 total, 12 up, 12 in 2020-09-08T16:27:02.517 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:15 smithi104 bash[9203]: cluster 2020-09-08T16:17:14.248684+0000 mgr.x (mgr.24118) 1497 : cluster [DBG] pgmap v2225: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.517 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:16 smithi104 bash[9203]: cluster 2020-09-08T16:17:15.256758+0000 osd.2 (osd.2) 126 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.517 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:16 smithi104 bash[9203]: cluster 2020-09-08T16:17:15.256850+0000 osd.2 (osd.2) 127 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.518 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:16 smithi104 bash[9203]: cluster 2020-09-08T16:17:16.187290+0000 mon.a (mon.0) 2433 : cluster [DBG] osdmap e813: 12 total, 12 up, 12 in 2020-09-08T16:27:02.518 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:17 smithi104 bash[9203]: cluster 2020-09-08T16:17:16.249291+0000 mgr.x (mgr.24118) 1498 : cluster [DBG] pgmap v2227: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.518 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:19 smithi104 bash[9203]: cluster 2020-09-08T16:17:18.250125+0000 mgr.x (mgr.24118) 1499 : cluster [DBG] pgmap v2228: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.518 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:21 smithi104 bash[9203]: cluster 2020-09-08T16:17:20.250901+0000 mgr.x (mgr.24118) 1500 : cluster [DBG] pgmap v2229: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.518 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:23 smithi104 bash[9203]: cluster 2020-09-08T16:17:22.251569+0000 mgr.x (mgr.24118) 1501 : cluster [DBG] pgmap v2230: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.519 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:24 smithi104 bash[9203]: cluster 2020-09-08T16:17:22.982890+0000 osd.7 (osd.7) 182 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.519 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:24 smithi104 bash[9203]: cluster 2020-09-08T16:17:22.983017+0000 osd.7 (osd.7) 183 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.519 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:24 smithi104 bash[9203]: cluster 2020-09-08T16:17:23.322276+0000 mon.a (mon.0) 2434 : cluster [DBG] osdmap e814: 12 total, 12 up, 12 in 2020-09-08T16:27:02.519 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:24 smithi104 bash[9203]: cluster 2020-09-08T16:17:23.824128+0000 osd.0 (osd.0) 142 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.519 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:24 smithi104 bash[9203]: cluster 2020-09-08T16:17:23.824255+0000 osd.0 (osd.0) 143 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.520 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:25 smithi104 bash[9203]: cluster 2020-09-08T16:17:24.252193+0000 mgr.x (mgr.24118) 1502 : cluster [DBG] pgmap v2232: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.520 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:25 smithi104 bash[9203]: cluster 2020-09-08T16:17:24.334057+0000 mon.a (mon.0) 2435 : cluster [DBG] osdmap e815: 12 total, 12 up, 12 in 2020-09-08T16:27:02.520 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:27 smithi104 bash[9203]: cluster 2020-09-08T16:17:26.252876+0000 mgr.x (mgr.24118) 1503 : cluster [DBG] pgmap v2234: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.520 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:29 smithi104 bash[9203]: cluster 2020-09-08T16:17:28.253590+0000 mgr.x (mgr.24118) 1504 : cluster [DBG] pgmap v2235: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.520 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:31 smithi104 bash[9203]: cluster 2020-09-08T16:17:30.254239+0000 mgr.x (mgr.24118) 1505 : cluster [DBG] pgmap v2236: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.521 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:33 smithi104 bash[9203]: cluster 2020-09-08T16:17:32.254924+0000 mgr.x (mgr.24118) 1506 : cluster [DBG] pgmap v2237: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.521 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:34 smithi104 bash[9203]: cluster 2020-09-08T16:17:33.361145+0000 osd.5 (osd.5) 148 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.521 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:34 smithi104 bash[9203]: cluster 2020-09-08T16:17:33.361282+0000 osd.5 (osd.5) 149 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.521 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:35 smithi104 bash[9203]: cluster 2020-09-08T16:17:34.255666+0000 mgr.x (mgr.24118) 1507 : cluster [DBG] pgmap v2238: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.521 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:35 smithi104 bash[9203]: cluster 2020-09-08T16:17:34.330857+0000 mon.a (mon.0) 2436 : cluster [DBG] osdmap e816: 12 total, 12 up, 12 in 2020-09-08T16:27:02.522 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:36 smithi104 bash[9203]: cluster 2020-09-08T16:17:35.549540+0000 osd.10 (osd.10) 127 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.522 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:36 smithi104 bash[9203]: cluster 2020-09-08T16:17:35.549670+0000 osd.10 (osd.10) 128 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.522 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:37 smithi104 bash[9203]: cluster 2020-09-08T16:17:36.256318+0000 mgr.x (mgr.24118) 1508 : cluster [DBG] pgmap v2240: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.522 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:37 smithi104 bash[9203]: cluster 2020-09-08T16:17:36.344939+0000 mon.a (mon.0) 2437 : cluster [DBG] osdmap e817: 12 total, 12 up, 12 in 2020-09-08T16:27:02.522 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:37 smithi104 bash[9203]: cluster 2020-09-08T16:17:37.131639+0000 osd.3 (osd.3) 151 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.523 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:37 smithi104 bash[9203]: cluster 2020-09-08T16:17:37.131762+0000 osd.3 (osd.3) 152 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.523 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:38 smithi104 bash[9203]: cluster 2020-09-08T16:17:37.346297+0000 mon.a (mon.0) 2438 : cluster [DBG] osdmap e818: 12 total, 12 up, 12 in 2020-09-08T16:27:02.523 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:39 smithi104 bash[9203]: cluster 2020-09-08T16:17:38.256992+0000 mgr.x (mgr.24118) 1509 : cluster [DBG] pgmap v2243: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.523 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:41 smithi104 bash[9203]: cluster 2020-09-08T16:17:40.257685+0000 mgr.x (mgr.24118) 1510 : cluster [DBG] pgmap v2244: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.523 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:43 smithi104 bash[9203]: cluster 2020-09-08T16:17:42.258389+0000 mgr.x (mgr.24118) 1511 : cluster [DBG] pgmap v2245: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.524 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:45 smithi104 bash[9203]: cluster 2020-09-08T16:17:44.259204+0000 mgr.x (mgr.24118) 1512 : cluster [DBG] pgmap v2246: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.524 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:46 smithi104 bash[9203]: cluster 2020-09-08T16:17:44.578715+0000 osd.8 (osd.8) 122 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.524 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:46 smithi104 bash[9203]: cluster 2020-09-08T16:17:44.578865+0000 osd.8 (osd.8) 123 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.524 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:46 smithi104 bash[9203]: cluster 2020-09-08T16:17:45.337997+0000 mon.a (mon.0) 2439 : cluster [DBG] osdmap e819: 12 total, 12 up, 12 in 2020-09-08T16:27:02.524 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:47 smithi104 bash[9203]: cluster 2020-09-08T16:17:46.259938+0000 mgr.x (mgr.24118) 1513 : cluster [DBG] pgmap v2248: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.525 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:49 smithi104 bash[9203]: cluster 2020-09-08T16:17:48.260855+0000 mgr.x (mgr.24118) 1514 : cluster [DBG] pgmap v2249: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.525 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:51 smithi104 bash[9203]: cluster 2020-09-08T16:17:50.261601+0000 mgr.x (mgr.24118) 1515 : cluster [DBG] pgmap v2250: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.525 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:52 smithi104 bash[9203]: cluster 2020-09-08T16:17:51.785942+0000 osd.1 (osd.1) 168 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.525 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:52 smithi104 bash[9203]: cluster 2020-09-08T16:17:51.786066+0000 osd.1 (osd.1) 169 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.525 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:53 smithi104 bash[9203]: cluster 2020-09-08T16:17:52.207446+0000 mon.a (mon.0) 2440 : cluster [DBG] osdmap e820: 12 total, 12 up, 12 in 2020-09-08T16:27:02.526 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:53 smithi104 bash[9203]: cluster 2020-09-08T16:17:52.262185+0000 mgr.x (mgr.24118) 1516 : cluster [DBG] pgmap v2252: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.526 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:55 smithi104 bash[9203]: cluster 2020-09-08T16:17:54.262935+0000 mgr.x (mgr.24118) 1517 : cluster [DBG] pgmap v2253: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.526 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:57 smithi104 bash[9203]: cluster 2020-09-08T16:17:56.087499+0000 osd.6 (osd.6) 133 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.526 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:57 smithi104 bash[9203]: cluster 2020-09-08T16:17:56.087630+0000 osd.6 (osd.6) 134 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.526 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:57 smithi104 bash[9203]: cluster 2020-09-08T16:17:56.263630+0000 mgr.x (mgr.24118) 1518 : cluster [DBG] pgmap v2254: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.527 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:57 smithi104 bash[9203]: cluster 2020-09-08T16:17:56.337988+0000 mon.a (mon.0) 2441 : cluster [DBG] osdmap e821: 12 total, 12 up, 12 in 2020-09-08T16:27:02.527 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:17:59 smithi104 bash[9203]: cluster 2020-09-08T16:17:58.264325+0000 mgr.x (mgr.24118) 1519 : cluster [DBG] pgmap v2256: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.527 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:01 smithi104 bash[9203]: cluster 2020-09-08T16:18:00.265034+0000 mgr.x (mgr.24118) 1520 : cluster [DBG] pgmap v2257: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.527 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:02 smithi104 bash[9203]: cluster 2020-09-08T16:18:01.420675+0000 osd.11 (osd.11) 148 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.527 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:02 smithi104 bash[9203]: cluster 2020-09-08T16:18:01.420810+0000 osd.11 (osd.11) 149 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.528 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:03 smithi104 bash[9203]: cluster 2020-09-08T16:18:02.213345+0000 mon.a (mon.0) 2442 : cluster [DBG] osdmap e822: 12 total, 12 up, 12 in 2020-09-08T16:27:02.528 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:03 smithi104 bash[9203]: cluster 2020-09-08T16:18:02.265680+0000 mgr.x (mgr.24118) 1521 : cluster [DBG] pgmap v2259: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.528 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:05 smithi104 bash[9203]: cluster 2020-09-08T16:18:04.266460+0000 mgr.x (mgr.24118) 1522 : cluster [DBG] pgmap v2260: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.528 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:06 smithi104 bash[9203]: cluster 2020-09-08T16:18:04.742722+0000 osd.4 (osd.4) 93 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.528 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:06 smithi104 bash[9203]: cluster 2020-09-08T16:18:04.742846+0000 osd.4 (osd.4) 94 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.529 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:06 smithi104 bash[9203]: cluster 2020-09-08T16:18:05.327738+0000 mon.a (mon.0) 2443 : cluster [DBG] osdmap e823: 12 total, 12 up, 12 in 2020-09-08T16:27:02.529 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:07 smithi104 bash[9203]: cluster 2020-09-08T16:18:06.267129+0000 mgr.x (mgr.24118) 1523 : cluster [DBG] pgmap v2262: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.529 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:09 smithi104 bash[9203]: cluster 2020-09-08T16:18:08.267890+0000 mgr.x (mgr.24118) 1524 : cluster [DBG] pgmap v2263: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.529 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:11 smithi104 bash[9203]: cluster 2020-09-08T16:18:10.268693+0000 mgr.x (mgr.24118) 1525 : cluster [DBG] pgmap v2264: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.529 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:13 smithi104 bash[9203]: cluster 2020-09-08T16:18:12.269386+0000 mgr.x (mgr.24118) 1526 : cluster [DBG] pgmap v2265: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.530 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:14 smithi104 bash[9203]: cluster 2020-09-08T16:18:13.633224+0000 osd.9 (osd.9) 132 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.530 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:14 smithi104 bash[9203]: cluster 2020-09-08T16:18:13.633351+0000 osd.9 (osd.9) 133 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.530 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:15 smithi104 bash[9203]: cluster 2020-09-08T16:18:14.270125+0000 mgr.x (mgr.24118) 1527 : cluster [DBG] pgmap v2266: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.530 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:15 smithi104 bash[9203]: cluster 2020-09-08T16:18:14.343382+0000 mon.a (mon.0) 2444 : cluster [DBG] osdmap e824: 12 total, 12 up, 12 in 2020-09-08T16:27:02.530 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:17 smithi104 bash[9203]: cluster 2020-09-08T16:18:16.208579+0000 osd.2 (osd.2) 128 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.531 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:17 smithi104 bash[9203]: cluster 2020-09-08T16:18:16.208704+0000 osd.2 (osd.2) 129 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.531 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:17 smithi104 bash[9203]: cluster 2020-09-08T16:18:16.270774+0000 mgr.x (mgr.24118) 1528 : cluster [DBG] pgmap v2268: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.531 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:17 smithi104 bash[9203]: cluster 2020-09-08T16:18:16.358692+0000 mon.a (mon.0) 2445 : cluster [DBG] osdmap e825: 12 total, 12 up, 12 in 2020-09-08T16:27:02.531 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:19 smithi104 bash[9203]: cluster 2020-09-08T16:18:18.271514+0000 mgr.x (mgr.24118) 1529 : cluster [DBG] pgmap v2270: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.531 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:21 smithi104 bash[9203]: cluster 2020-09-08T16:18:20.272244+0000 mgr.x (mgr.24118) 1530 : cluster [DBG] pgmap v2271: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.532 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:23 smithi104 bash[9203]: cluster 2020-09-08T16:18:22.272930+0000 mgr.x (mgr.24118) 1531 : cluster [DBG] pgmap v2272: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.532 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:25 smithi104 bash[9203]: cluster 2020-09-08T16:18:23.788192+0000 osd.7 (osd.7) 184 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.532 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:25 smithi104 bash[9203]: cluster 2020-09-08T16:18:23.788334+0000 osd.7 (osd.7) 185 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.532 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:25 smithi104 bash[9203]: cluster 2020-09-08T16:18:24.273691+0000 mgr.x (mgr.24118) 1532 : cluster [DBG] pgmap v2273: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.532 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:25 smithi104 bash[9203]: cluster 2020-09-08T16:18:24.345809+0000 mon.a (mon.0) 2446 : cluster [DBG] osdmap e826: 12 total, 12 up, 12 in 2020-09-08T16:27:02.533 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:25 smithi104 bash[9203]: cluster 2020-09-08T16:18:24.542528+0000 osd.0 (osd.0) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.533 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:25 smithi104 bash[9203]: cluster 2020-09-08T16:18:24.542657+0000 osd.0 (osd.0) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.533 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:26 smithi104 bash[9203]: cluster 2020-09-08T16:18:25.346774+0000 mon.a (mon.0) 2447 : cluster [DBG] osdmap e827: 12 total, 12 up, 12 in 2020-09-08T16:27:02.533 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:27 smithi104 bash[9203]: cluster 2020-09-08T16:18:26.274355+0000 mgr.x (mgr.24118) 1533 : cluster [DBG] pgmap v2276: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.533 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:29 smithi104 bash[9203]: cluster 2020-09-08T16:18:28.275071+0000 mgr.x (mgr.24118) 1534 : cluster [DBG] pgmap v2277: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.534 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:30 smithi104 bash[9203]: audit 2020-09-08T16:18:29.527573+0000 mon.b (mon.1) 136 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.534 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:30 smithi104 bash[9203]: audit 2020-09-08T16:18:29.529031+0000 mon.a (mon.0) 2448 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.534 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:30 smithi104 bash[9203]: audit 2020-09-08T16:18:29.534278+0000 mon.a (mon.0) 2449 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.534 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:31 smithi104 bash[9203]: cluster 2020-09-08T16:18:30.275532+0000 mgr.x (mgr.24118) 1535 : cluster [DBG] pgmap v2278: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.534 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:33 smithi104 bash[9203]: cluster 2020-09-08T16:18:32.276128+0000 mgr.x (mgr.24118) 1536 : cluster [DBG] pgmap v2279: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.535 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:35 smithi104 bash[9203]: cluster 2020-09-08T16:18:34.142626+0000 osd.5 (osd.5) 150 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.535 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:35 smithi104 bash[9203]: cluster 2020-09-08T16:18:34.142759+0000 osd.5 (osd.5) 151 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.535 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:35 smithi104 bash[9203]: cluster 2020-09-08T16:18:34.276879+0000 mgr.x (mgr.24118) 1537 : cluster [DBG] pgmap v2280: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.535 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:35 smithi104 bash[9203]: cluster 2020-09-08T16:18:34.400794+0000 mon.a (mon.0) 2450 : cluster [DBG] osdmap e828: 12 total, 12 up, 12 in 2020-09-08T16:27:02.535 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:36 smithi104 bash[9203]: cluster 2020-09-08T16:18:35.596323+0000 osd.10 (osd.10) 129 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.536 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:36 smithi104 bash[9203]: cluster 2020-09-08T16:18:35.596454+0000 osd.10 (osd.10) 130 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.536 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:37 smithi104 bash[9203]: cluster 2020-09-08T16:18:36.277520+0000 mgr.x (mgr.24118) 1538 : cluster [DBG] pgmap v2282: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.536 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:37 smithi104 bash[9203]: cluster 2020-09-08T16:18:36.483874+0000 mon.a (mon.0) 2451 : cluster [DBG] osdmap e829: 12 total, 12 up, 12 in 2020-09-08T16:27:02.536 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:38 smithi104 bash[9203]: cluster 2020-09-08T16:18:37.338299+0000 osd.3 (osd.3) 153 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.537 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:38 smithi104 bash[9203]: cluster 2020-09-08T16:18:37.338410+0000 osd.3 (osd.3) 154 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.537 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:38 smithi104 bash[9203]: cluster 2020-09-08T16:18:37.491995+0000 mon.a (mon.0) 2452 : cluster [DBG] osdmap e830: 12 total, 12 up, 12 in 2020-09-08T16:27:02.537 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:39 smithi104 bash[9203]: cluster 2020-09-08T16:18:38.278287+0000 mgr.x (mgr.24118) 1539 : cluster [DBG] pgmap v2285: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.537 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:41 smithi104 bash[9203]: cluster 2020-09-08T16:18:40.279025+0000 mgr.x (mgr.24118) 1540 : cluster [DBG] pgmap v2286: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.537 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:43 smithi104 bash[9203]: cluster 2020-09-08T16:18:42.279684+0000 mgr.x (mgr.24118) 1541 : cluster [DBG] pgmap v2287: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.538 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:45 smithi104 bash[9203]: cluster 2020-09-08T16:18:44.280459+0000 mgr.x (mgr.24118) 1542 : cluster [DBG] pgmap v2288: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.538 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:46 smithi104 bash[9203]: cluster 2020-09-08T16:18:44.720522+0000 osd.8 (osd.8) 124 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.538 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:46 smithi104 bash[9203]: cluster 2020-09-08T16:18:44.720655+0000 osd.8 (osd.8) 125 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.538 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:46 smithi104 bash[9203]: cluster 2020-09-08T16:18:45.355304+0000 mon.a (mon.0) 2453 : cluster [DBG] osdmap e831: 12 total, 12 up, 12 in 2020-09-08T16:27:02.538 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:47 smithi104 bash[9203]: cluster 2020-09-08T16:18:46.281093+0000 mgr.x (mgr.24118) 1543 : cluster [DBG] pgmap v2290: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.539 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:49 smithi104 bash[9203]: cluster 2020-09-08T16:18:48.281847+0000 mgr.x (mgr.24118) 1544 : cluster [DBG] pgmap v2291: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.539 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:51 smithi104 bash[9203]: cluster 2020-09-08T16:18:50.282574+0000 mgr.x (mgr.24118) 1545 : cluster [DBG] pgmap v2292: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.539 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:52 smithi104 bash[9203]: cluster 2020-09-08T16:18:51.806760+0000 osd.1 (osd.1) 170 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.539 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:52 smithi104 bash[9203]: cluster 2020-09-08T16:18:51.806888+0000 osd.1 (osd.1) 171 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.540 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:53 smithi104 bash[9203]: cluster 2020-09-08T16:18:52.228580+0000 mon.a (mon.0) 2454 : cluster [DBG] osdmap e832: 12 total, 12 up, 12 in 2020-09-08T16:27:02.540 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:53 smithi104 bash[9203]: cluster 2020-09-08T16:18:52.283236+0000 mgr.x (mgr.24118) 1546 : cluster [DBG] pgmap v2294: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.540 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:55 smithi104 bash[9203]: cluster 2020-09-08T16:18:54.283994+0000 mgr.x (mgr.24118) 1547 : cluster [DBG] pgmap v2295: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.540 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:57 smithi104 bash[9203]: cluster 2020-09-08T16:18:56.284675+0000 mgr.x (mgr.24118) 1548 : cluster [DBG] pgmap v2296: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.540 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:58 smithi104 bash[9203]: cluster 2020-09-08T16:18:57.033753+0000 osd.6 (osd.6) 135 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.541 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:58 smithi104 bash[9203]: cluster 2020-09-08T16:18:57.033886+0000 osd.6 (osd.6) 136 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.541 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:58 smithi104 bash[9203]: cluster 2020-09-08T16:18:57.362340+0000 mon.a (mon.0) 2455 : cluster [DBG] osdmap e833: 12 total, 12 up, 12 in 2020-09-08T16:27:02.541 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:18:59 smithi104 bash[9203]: cluster 2020-09-08T16:18:58.285394+0000 mgr.x (mgr.24118) 1549 : cluster [DBG] pgmap v2298: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.541 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:01 smithi104 bash[9203]: cluster 2020-09-08T16:19:00.286031+0000 mgr.x (mgr.24118) 1550 : cluster [DBG] pgmap v2299: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.541 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:02 smithi104 bash[9203]: cluster 2020-09-08T16:19:01.547200+0000 osd.11 (osd.11) 150 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.542 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:02 smithi104 bash[9203]: cluster 2020-09-08T16:19:01.547357+0000 osd.11 (osd.11) 151 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.542 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:03 smithi104 bash[9203]: cluster 2020-09-08T16:19:02.231839+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e834: 12 total, 12 up, 12 in 2020-09-08T16:27:02.542 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:03 smithi104 bash[9203]: cluster 2020-09-08T16:19:02.286716+0000 mgr.x (mgr.24118) 1551 : cluster [DBG] pgmap v2301: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.542 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:05 smithi104 bash[9203]: cluster 2020-09-08T16:19:04.287459+0000 mgr.x (mgr.24118) 1552 : cluster [DBG] pgmap v2302: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.542 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:06 smithi104 bash[9203]: cluster 2020-09-08T16:19:05.101483+0000 osd.4 (osd.4) 95 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.543 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:06 smithi104 bash[9203]: cluster 2020-09-08T16:19:05.101600+0000 osd.4 (osd.4) 96 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.543 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:06 smithi104 bash[9203]: cluster 2020-09-08T16:19:05.351339+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e835: 12 total, 12 up, 12 in 2020-09-08T16:27:02.543 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:07 smithi104 bash[9203]: cluster 2020-09-08T16:19:06.288110+0000 mgr.x (mgr.24118) 1553 : cluster [DBG] pgmap v2304: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.543 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:09 smithi104 bash[9203]: cluster 2020-09-08T16:19:08.288845+0000 mgr.x (mgr.24118) 1554 : cluster [DBG] pgmap v2305: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.543 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:11 smithi104 bash[9203]: cluster 2020-09-08T16:19:10.289470+0000 mgr.x (mgr.24118) 1555 : cluster [DBG] pgmap v2306: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.544 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:13 smithi104 bash[9203]: cluster 2020-09-08T16:19:12.290155+0000 mgr.x (mgr.24118) 1556 : cluster [DBG] pgmap v2307: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.544 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:15 smithi104 bash[9203]: cluster 2020-09-08T16:19:14.290872+0000 mgr.x (mgr.24118) 1557 : cluster [DBG] pgmap v2308: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.544 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:15 smithi104 bash[9203]: cluster 2020-09-08T16:19:14.572020+0000 osd.9 (osd.9) 134 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.544 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:15 smithi104 bash[9203]: cluster 2020-09-08T16:19:14.572143+0000 osd.9 (osd.9) 135 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.544 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:16 smithi104 bash[9203]: cluster 2020-09-08T16:19:15.358643+0000 mon.a (mon.0) 2458 : cluster [DBG] osdmap e836: 12 total, 12 up, 12 in 2020-09-08T16:27:02.545 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:17 smithi104 bash[9203]: cluster 2020-09-08T16:19:16.291530+0000 mgr.x (mgr.24118) 1558 : cluster [DBG] pgmap v2310: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.545 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:18 smithi104 bash[9203]: cluster 2020-09-08T16:19:17.194214+0000 osd.2 (osd.2) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.545 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:18 smithi104 bash[9203]: cluster 2020-09-08T16:19:17.194333+0000 osd.2 (osd.2) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.545 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:18 smithi104 bash[9203]: cluster 2020-09-08T16:19:17.403220+0000 mon.a (mon.0) 2459 : cluster [DBG] osdmap e837: 12 total, 12 up, 12 in 2020-09-08T16:27:02.545 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:19 smithi104 bash[9203]: cluster 2020-09-08T16:19:18.292239+0000 mgr.x (mgr.24118) 1559 : cluster [DBG] pgmap v2312: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.546 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:21 smithi104 bash[9203]: cluster 2020-09-08T16:19:20.293036+0000 mgr.x (mgr.24118) 1560 : cluster [DBG] pgmap v2313: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.546 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:23 smithi104 bash[9203]: cluster 2020-09-08T16:19:22.293768+0000 mgr.x (mgr.24118) 1561 : cluster [DBG] pgmap v2314: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.546 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:25 smithi104 bash[9203]: cluster 2020-09-08T16:19:24.294501+0000 mgr.x (mgr.24118) 1562 : cluster [DBG] pgmap v2315: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.546 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:25 smithi104 bash[9203]: cluster 2020-09-08T16:19:24.556230+0000 osd.7 (osd.7) 186 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.547 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:25 smithi104 bash[9203]: cluster 2020-09-08T16:19:24.556355+0000 osd.7 (osd.7) 187 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.547 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:25 smithi104 bash[9203]: cluster 2020-09-08T16:19:24.633342+0000 osd.0 (osd.0) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.547 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:25 smithi104 bash[9203]: cluster 2020-09-08T16:19:24.633471+0000 osd.0 (osd.0) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.547 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:26 smithi104 bash[9203]: cluster 2020-09-08T16:19:25.361020+0000 mon.a (mon.0) 2460 : cluster [DBG] osdmap e838: 12 total, 12 up, 12 in 2020-09-08T16:27:02.547 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:27 smithi104 bash[9203]: cluster 2020-09-08T16:19:26.295149+0000 mgr.x (mgr.24118) 1563 : cluster [DBG] pgmap v2317: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.547 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:28 smithi104 bash[9203]: audit 2020-09-08T16:19:27.397122+0000 mon.b (mon.1) 137 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.548 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:28 smithi104 bash[9203]: audit 2020-09-08T16:19:27.398626+0000 mon.a (mon.0) 2461 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.548 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:28 smithi104 bash[9203]: audit 2020-09-08T16:19:27.405368+0000 mon.a (mon.0) 2462 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.548 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:29 smithi104 bash[9203]: cluster 2020-09-08T16:19:28.295846+0000 mgr.x (mgr.24118) 1564 : cluster [DBG] pgmap v2318: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.548 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:31 smithi104 bash[9203]: cluster 2020-09-08T16:19:30.296510+0000 mgr.x (mgr.24118) 1565 : cluster [DBG] pgmap v2319: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.548 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:33 smithi104 bash[9203]: cluster 2020-09-08T16:19:32.297174+0000 mgr.x (mgr.24118) 1566 : cluster [DBG] pgmap v2320: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.549 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:35 smithi104 bash[9203]: cluster 2020-09-08T16:19:34.219237+0000 osd.5 (osd.5) 152 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.549 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:35 smithi104 bash[9203]: cluster 2020-09-08T16:19:34.219366+0000 osd.5 (osd.5) 153 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.549 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:35 smithi104 bash[9203]: cluster 2020-09-08T16:19:34.297911+0000 mgr.x (mgr.24118) 1567 : cluster [DBG] pgmap v2321: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.549 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:35 smithi104 bash[9203]: cluster 2020-09-08T16:19:34.371433+0000 mon.a (mon.0) 2463 : cluster [DBG] osdmap e839: 12 total, 12 up, 12 in 2020-09-08T16:27:02.550 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:37 smithi104 bash[9203]: cluster 2020-09-08T16:19:36.298640+0000 mgr.x (mgr.24118) 1568 : cluster [DBG] pgmap v2323: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.550 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:37 smithi104 bash[9203]: cluster 2020-09-08T16:19:36.334119+0000 osd.10 (osd.10) 131 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.550 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:37 smithi104 bash[9203]: cluster 2020-09-08T16:19:36.334233+0000 osd.10 (osd.10) 132 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.550 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:37 smithi104 bash[9203]: cluster 2020-09-08T16:19:36.388767+0000 mon.a (mon.0) 2464 : cluster [DBG] osdmap e840: 12 total, 12 up, 12 in 2020-09-08T16:27:02.550 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:38 smithi104 bash[9203]: cluster 2020-09-08T16:19:37.437449+0000 osd.3 (osd.3) 155 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.551 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:38 smithi104 bash[9203]: cluster 2020-09-08T16:19:37.437572+0000 osd.3 (osd.3) 156 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.551 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:39 smithi104 bash[9203]: cluster 2020-09-08T16:19:38.299377+0000 mgr.x (mgr.24118) 1569 : cluster [DBG] pgmap v2325: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.551 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:39 smithi104 bash[9203]: cluster 2020-09-08T16:19:38.402072+0000 mon.a (mon.0) 2465 : cluster [DBG] osdmap e841: 12 total, 12 up, 12 in 2020-09-08T16:27:02.551 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:41 smithi104 bash[9203]: cluster 2020-09-08T16:19:40.299822+0000 mgr.x (mgr.24118) 1570 : cluster [DBG] pgmap v2327: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.551 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:43 smithi104 bash[9203]: cluster 2020-09-08T16:19:42.300421+0000 mgr.x (mgr.24118) 1571 : cluster [DBG] pgmap v2328: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.552 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:45 smithi104 bash[9203]: cluster 2020-09-08T16:19:44.301177+0000 mgr.x (mgr.24118) 1572 : cluster [DBG] pgmap v2329: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.552 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:46 smithi104 bash[9203]: cluster 2020-09-08T16:19:44.761903+0000 osd.8 (osd.8) 126 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.552 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:46 smithi104 bash[9203]: cluster 2020-09-08T16:19:44.762032+0000 osd.8 (osd.8) 127 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.552 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:46 smithi104 bash[9203]: cluster 2020-09-08T16:19:45.377400+0000 mon.a (mon.0) 2466 : cluster [DBG] osdmap e842: 12 total, 12 up, 12 in 2020-09-08T16:27:02.552 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:47 smithi104 bash[9203]: cluster 2020-09-08T16:19:46.301791+0000 mgr.x (mgr.24118) 1573 : cluster [DBG] pgmap v2331: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.552 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:49 smithi104 bash[9203]: cluster 2020-09-08T16:19:48.302517+0000 mgr.x (mgr.24118) 1574 : cluster [DBG] pgmap v2332: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.553 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:51 smithi104 bash[9203]: cluster 2020-09-08T16:19:50.303213+0000 mgr.x (mgr.24118) 1575 : cluster [DBG] pgmap v2333: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.553 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:52 smithi104 bash[9203]: cluster 2020-09-08T16:19:51.829178+0000 osd.1 (osd.1) 172 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.553 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:52 smithi104 bash[9203]: cluster 2020-09-08T16:19:51.829307+0000 osd.1 (osd.1) 173 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.553 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:53 smithi104 bash[9203]: cluster 2020-09-08T16:19:52.237451+0000 mon.a (mon.0) 2467 : cluster [DBG] osdmap e843: 12 total, 12 up, 12 in 2020-09-08T16:27:02.554 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:53 smithi104 bash[9203]: cluster 2020-09-08T16:19:52.303900+0000 mgr.x (mgr.24118) 1576 : cluster [DBG] pgmap v2335: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.554 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:55 smithi104 bash[9203]: cluster 2020-09-08T16:19:54.304737+0000 mgr.x (mgr.24118) 1577 : cluster [DBG] pgmap v2336: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.554 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:57 smithi104 bash[9203]: cluster 2020-09-08T16:19:56.305456+0000 mgr.x (mgr.24118) 1578 : cluster [DBG] pgmap v2337: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.554 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:58 smithi104 bash[9203]: cluster 2020-09-08T16:19:57.050038+0000 osd.6 (osd.6) 137 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.554 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:58 smithi104 bash[9203]: cluster 2020-09-08T16:19:57.050184+0000 osd.6 (osd.6) 138 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.555 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:58 smithi104 bash[9203]: cluster 2020-09-08T16:19:57.374960+0000 mon.a (mon.0) 2468 : cluster [DBG] osdmap e844: 12 total, 12 up, 12 in 2020-09-08T16:27:02.555 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:19:59 smithi104 bash[9203]: cluster 2020-09-08T16:19:58.306249+0000 mgr.x (mgr.24118) 1579 : cluster [DBG] pgmap v2339: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.555 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:01 smithi104 bash[9203]: cluster 2020-09-08T16:20:00.307004+0000 mgr.x (mgr.24118) 1580 : cluster [DBG] pgmap v2340: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.555 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:03 smithi104 bash[9203]: cluster 2020-09-08T16:20:02.307670+0000 mgr.x (mgr.24118) 1581 : cluster [DBG] pgmap v2341: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.555 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:03 smithi104 bash[9203]: cluster 2020-09-08T16:20:02.428676+0000 osd.11 (osd.11) 152 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.556 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:03 smithi104 bash[9203]: cluster 2020-09-08T16:20:02.428805+0000 osd.11 (osd.11) 153 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.556 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:04 smithi104 bash[9203]: cluster 2020-09-08T16:20:03.369560+0000 mon.a (mon.0) 2469 : cluster [DBG] osdmap e845: 12 total, 12 up, 12 in 2020-09-08T16:27:02.556 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:05 smithi104 bash[9203]: cluster 2020-09-08T16:20:04.308304+0000 mgr.x (mgr.24118) 1582 : cluster [DBG] pgmap v2343: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.556 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:07 smithi104 bash[9203]: cluster 2020-09-08T16:20:05.902905+0000 osd.4 (osd.4) 97 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.556 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:07 smithi104 bash[9203]: cluster 2020-09-08T16:20:05.903036+0000 osd.4 (osd.4) 98 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.557 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:07 smithi104 bash[9203]: cluster 2020-09-08T16:20:06.309002+0000 mgr.x (mgr.24118) 1583 : cluster [DBG] pgmap v2344: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.557 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:07 smithi104 bash[9203]: cluster 2020-09-08T16:20:06.413136+0000 mon.a (mon.0) 2470 : cluster [DBG] osdmap e846: 12 total, 12 up, 12 in 2020-09-08T16:27:02.557 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:09 smithi104 bash[9203]: cluster 2020-09-08T16:20:08.309839+0000 mgr.x (mgr.24118) 1584 : cluster [DBG] pgmap v2346: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.558 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:11 smithi104 bash[9203]: cluster 2020-09-08T16:20:10.310566+0000 mgr.x (mgr.24118) 1585 : cluster [DBG] pgmap v2347: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.558 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:13 smithi104 bash[9203]: cluster 2020-09-08T16:20:12.311236+0000 mgr.x (mgr.24118) 1586 : cluster [DBG] pgmap v2348: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.558 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:15 smithi104 bash[9203]: cluster 2020-09-08T16:20:14.311961+0000 mgr.x (mgr.24118) 1587 : cluster [DBG] pgmap v2349: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.558 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:16 smithi104 bash[9203]: cluster 2020-09-08T16:20:15.478981+0000 osd.9 (osd.9) 136 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.558 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:16 smithi104 bash[9203]: cluster 2020-09-08T16:20:15.479101+0000 osd.9 (osd.9) 137 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.559 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:17 smithi104 bash[9203]: cluster 2020-09-08T16:20:16.312636+0000 mgr.x (mgr.24118) 1588 : cluster [DBG] pgmap v2350: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.559 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:17 smithi104 bash[9203]: cluster 2020-09-08T16:20:16.393448+0000 mon.a (mon.0) 2471 : cluster [DBG] osdmap e847: 12 total, 12 up, 12 in 2020-09-08T16:27:02.559 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:18 smithi104 bash[9203]: cluster 2020-09-08T16:20:17.306375+0000 osd.2 (osd.2) 132 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.559 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:18 smithi104 bash[9203]: cluster 2020-09-08T16:20:17.306485+0000 osd.2 (osd.2) 133 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.559 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:18 smithi104 bash[9203]: cluster 2020-09-08T16:20:17.402201+0000 mon.a (mon.0) 2472 : cluster [DBG] osdmap e848: 12 total, 12 up, 12 in 2020-09-08T16:27:02.559 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:19 smithi104 bash[9203]: cluster 2020-09-08T16:20:18.313348+0000 mgr.x (mgr.24118) 1589 : cluster [DBG] pgmap v2353: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.560 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:21 smithi104 bash[9203]: cluster 2020-09-08T16:20:20.313853+0000 mgr.x (mgr.24118) 1590 : cluster [DBG] pgmap v2354: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.560 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:23 smithi104 bash[9203]: cluster 2020-09-08T16:20:22.314502+0000 mgr.x (mgr.24118) 1591 : cluster [DBG] pgmap v2355: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.560 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:25 smithi104 bash[9203]: cluster 2020-09-08T16:20:24.315223+0000 mgr.x (mgr.24118) 1592 : cluster [DBG] pgmap v2356: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.560 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:25 smithi104 bash[9203]: cluster 2020-09-08T16:20:24.707385+0000 osd.7 (osd.7) 188 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.560 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:25 smithi104 bash[9203]: cluster 2020-09-08T16:20:24.707509+0000 osd.7 (osd.7) 189 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.561 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:26 smithi104 bash[9203]: cluster 2020-09-08T16:20:25.382370+0000 mon.a (mon.0) 2473 : cluster [DBG] osdmap e849: 12 total, 12 up, 12 in 2020-09-08T16:27:02.561 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:26 smithi104 bash[9203]: cluster 2020-09-08T16:20:25.588412+0000 osd.0 (osd.0) 148 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.561 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:26 smithi104 bash[9203]: cluster 2020-09-08T16:20:25.588570+0000 osd.0 (osd.0) 149 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.561 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:27 smithi104 bash[9203]: cluster 2020-09-08T16:20:26.315850+0000 mgr.x (mgr.24118) 1593 : cluster [DBG] pgmap v2358: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.561 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:27 smithi104 bash[9203]: cluster 2020-09-08T16:20:26.414678+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e850: 12 total, 12 up, 12 in 2020-09-08T16:27:02.562 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:29 smithi104 bash[9203]: cluster 2020-09-08T16:20:28.316580+0000 mgr.x (mgr.24118) 1594 : cluster [DBG] pgmap v2360: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.562 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:31 smithi104 bash[9203]: cluster 2020-09-08T16:20:30.317240+0000 mgr.x (mgr.24118) 1595 : cluster [DBG] pgmap v2361: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.562 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:33 smithi104 bash[9203]: cluster 2020-09-08T16:20:32.317937+0000 mgr.x (mgr.24118) 1596 : cluster [DBG] pgmap v2362: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.562 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:35 smithi104 bash[9203]: cluster 2020-09-08T16:20:34.318671+0000 mgr.x (mgr.24118) 1597 : cluster [DBG] pgmap v2363: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.562 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:36 smithi104 bash[9203]: cluster 2020-09-08T16:20:35.020359+0000 osd.5 (osd.5) 154 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.563 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:36 smithi104 bash[9203]: cluster 2020-09-08T16:20:35.020517+0000 osd.5 (osd.5) 155 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.563 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:36 smithi104 bash[9203]: cluster 2020-09-08T16:20:35.391055+0000 mon.a (mon.0) 2475 : cluster [DBG] osdmap e851: 12 total, 12 up, 12 in 2020-09-08T16:27:02.563 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:37 smithi104 bash[9203]: cluster 2020-09-08T16:20:36.319398+0000 mgr.x (mgr.24118) 1598 : cluster [DBG] pgmap v2365: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.563 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:37 smithi104 bash[9203]: cluster 2020-09-08T16:20:37.088845+0000 osd.10 (osd.10) 133 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.563 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:37 smithi104 bash[9203]: cluster 2020-09-08T16:20:37.088962+0000 osd.10 (osd.10) 134 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.564 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:38 smithi104 bash[9203]: cluster 2020-09-08T16:20:37.405413+0000 mon.a (mon.0) 2476 : cluster [DBG] osdmap e852: 12 total, 12 up, 12 in 2020-09-08T16:27:02.564 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:38 smithi104 bash[9203]: cluster 2020-09-08T16:20:37.452514+0000 osd.3 (osd.3) 157 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.564 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:38 smithi104 bash[9203]: cluster 2020-09-08T16:20:37.452628+0000 osd.3 (osd.3) 158 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.564 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:39 smithi104 bash[9203]: cluster 2020-09-08T16:20:38.320060+0000 mgr.x (mgr.24118) 1599 : cluster [DBG] pgmap v2367: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.564 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:39 smithi104 bash[9203]: cluster 2020-09-08T16:20:38.427057+0000 mon.a (mon.0) 2477 : cluster [DBG] osdmap e853: 12 total, 12 up, 12 in 2020-09-08T16:27:02.565 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:41 smithi104 bash[9203]: cluster 2020-09-08T16:20:40.320532+0000 mgr.x (mgr.24118) 1600 : cluster [DBG] pgmap v2369: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.565 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:43 smithi104 bash[9203]: cluster 2020-09-08T16:20:42.321174+0000 mgr.x (mgr.24118) 1601 : cluster [DBG] pgmap v2370: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.565 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:45 smithi104 bash[9203]: cluster 2020-09-08T16:20:44.321907+0000 mgr.x (mgr.24118) 1602 : cluster [DBG] pgmap v2371: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.565 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:46 smithi104 bash[9203]: cluster 2020-09-08T16:20:45.563606+0000 osd.8 (osd.8) 128 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.565 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:46 smithi104 bash[9203]: cluster 2020-09-08T16:20:45.563735+0000 osd.8 (osd.8) 129 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.566 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:47 smithi104 bash[9203]: cluster 2020-09-08T16:20:46.322795+0000 mgr.x (mgr.24118) 1603 : cluster [DBG] pgmap v2372: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.566 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:47 smithi104 bash[9203]: cluster 2020-09-08T16:20:46.404936+0000 mon.a (mon.0) 2478 : cluster [DBG] osdmap e854: 12 total, 12 up, 12 in 2020-09-08T16:27:02.566 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:49 smithi104 bash[9203]: cluster 2020-09-08T16:20:48.323584+0000 mgr.x (mgr.24118) 1604 : cluster [DBG] pgmap v2374: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.566 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:51 smithi104 bash[9203]: cluster 2020-09-08T16:20:50.324248+0000 mgr.x (mgr.24118) 1605 : cluster [DBG] pgmap v2375: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.566 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:53 smithi104 bash[9203]: cluster 2020-09-08T16:20:52.088960+0000 osd.1 (osd.1) 174 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.567 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:53 smithi104 bash[9203]: cluster 2020-09-08T16:20:52.089112+0000 osd.1 (osd.1) 175 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.567 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:53 smithi104 bash[9203]: cluster 2020-09-08T16:20:52.259962+0000 mon.a (mon.0) 2479 : cluster [DBG] osdmap e855: 12 total, 12 up, 12 in 2020-09-08T16:27:02.567 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:53 smithi104 bash[9203]: cluster 2020-09-08T16:20:52.324907+0000 mgr.x (mgr.24118) 1606 : cluster [DBG] pgmap v2377: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.567 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:55 smithi104 bash[9203]: cluster 2020-09-08T16:20:54.325657+0000 mgr.x (mgr.24118) 1607 : cluster [DBG] pgmap v2378: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.567 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:57 smithi104 bash[9203]: cluster 2020-09-08T16:20:56.326288+0000 mgr.x (mgr.24118) 1608 : cluster [DBG] pgmap v2379: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.568 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:59 smithi104 bash[9203]: cluster 2020-09-08T16:20:57.812662+0000 osd.6 (osd.6) 139 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.568 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:59 smithi104 bash[9203]: cluster 2020-09-08T16:20:57.812796+0000 osd.6 (osd.6) 140 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.568 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:59 smithi104 bash[9203]: cluster 2020-09-08T16:20:58.327078+0000 mgr.x (mgr.24118) 1609 : cluster [DBG] pgmap v2380: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.568 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:20:59 smithi104 bash[9203]: cluster 2020-09-08T16:20:58.404772+0000 mon.a (mon.0) 2480 : cluster [DBG] osdmap e856: 12 total, 12 up, 12 in 2020-09-08T16:27:02.568 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:01 smithi104 bash[9203]: cluster 2020-09-08T16:21:00.327722+0000 mgr.x (mgr.24118) 1610 : cluster [DBG] pgmap v2382: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.569 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:03 smithi104 bash[9203]: cluster 2020-09-08T16:21:02.328378+0000 mgr.x (mgr.24118) 1611 : cluster [DBG] pgmap v2383: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.569 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:04 smithi104 bash[9203]: cluster 2020-09-08T16:21:03.333742+0000 osd.11 (osd.11) 154 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.569 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:04 smithi104 bash[9203]: cluster 2020-09-08T16:21:03.333871+0000 osd.11 (osd.11) 155 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.569 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:04 smithi104 bash[9203]: cluster 2020-09-08T16:21:03.400773+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e857: 12 total, 12 up, 12 in 2020-09-08T16:27:02.569 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:05 smithi104 bash[9203]: cluster 2020-09-08T16:21:04.329012+0000 mgr.x (mgr.24118) 1612 : cluster [DBG] pgmap v2385: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.570 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:07 smithi104 bash[9203]: cluster 2020-09-08T16:21:06.106338+0000 osd.4 (osd.4) 99 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.570 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:07 smithi104 bash[9203]: cluster 2020-09-08T16:21:06.106465+0000 osd.4 (osd.4) 100 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.570 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:07 smithi104 bash[9203]: cluster 2020-09-08T16:21:06.329766+0000 mgr.x (mgr.24118) 1613 : cluster [DBG] pgmap v2386: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.570 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:07 smithi104 bash[9203]: cluster 2020-09-08T16:21:06.421076+0000 mon.a (mon.0) 2482 : cluster [DBG] osdmap e858: 12 total, 12 up, 12 in 2020-09-08T16:27:02.570 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:09 smithi104 bash[9203]: cluster 2020-09-08T16:21:08.330615+0000 mgr.x (mgr.24118) 1614 : cluster [DBG] pgmap v2388: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.571 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:11 smithi104 bash[9203]: cluster 2020-09-08T16:21:10.331273+0000 mgr.x (mgr.24118) 1615 : cluster [DBG] pgmap v2389: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.571 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:13 smithi104 bash[9203]: cluster 2020-09-08T16:21:12.331978+0000 mgr.x (mgr.24118) 1616 : cluster [DBG] pgmap v2390: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.571 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:15 smithi104 bash[9203]: cluster 2020-09-08T16:21:14.332692+0000 mgr.x (mgr.24118) 1617 : cluster [DBG] pgmap v2391: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.571 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:17 smithi104 bash[9203]: cluster 2020-09-08T16:21:16.333447+0000 mgr.x (mgr.24118) 1618 : cluster [DBG] pgmap v2392: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.571 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:17 smithi104 bash[9203]: cluster 2020-09-08T16:21:16.362679+0000 osd.9 (osd.9) 138 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.572 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:17 smithi104 bash[9203]: cluster 2020-09-08T16:21:16.362811+0000 osd.9 (osd.9) 139 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.572 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:17 smithi104 bash[9203]: cluster 2020-09-08T16:21:16.409000+0000 mon.a (mon.0) 2483 : cluster [DBG] osdmap e859: 12 total, 12 up, 12 in 2020-09-08T16:27:02.572 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:18 smithi104 bash[9203]: cluster 2020-09-08T16:21:17.547440+0000 osd.2 (osd.2) 134 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.572 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:18 smithi104 bash[9203]: cluster 2020-09-08T16:21:17.547614+0000 osd.2 (osd.2) 135 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.572 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:19 smithi104 bash[9203]: cluster 2020-09-08T16:21:18.334352+0000 mgr.x (mgr.24118) 1619 : cluster [DBG] pgmap v2394: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.573 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:19 smithi104 bash[9203]: cluster 2020-09-08T16:21:18.423887+0000 mon.a (mon.0) 2484 : cluster [DBG] osdmap e860: 12 total, 12 up, 12 in 2020-09-08T16:27:02.573 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:21 smithi104 bash[9203]: cluster 2020-09-08T16:21:20.335075+0000 mgr.x (mgr.24118) 1620 : cluster [DBG] pgmap v2396: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.573 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:23 smithi104 bash[9203]: cluster 2020-09-08T16:21:22.335851+0000 mgr.x (mgr.24118) 1621 : cluster [DBG] pgmap v2397: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.573 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:25 smithi104 bash[9203]: cluster 2020-09-08T16:21:24.336633+0000 mgr.x (mgr.24118) 1622 : cluster [DBG] pgmap v2398: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.573 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:26 smithi104 bash[9203]: cluster 2020-09-08T16:21:25.673730+0000 osd.7 (osd.7) 190 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.574 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:26 smithi104 bash[9203]: cluster 2020-09-08T16:21:25.673860+0000 osd.7 (osd.7) 191 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.574 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:26 smithi104 bash[9203]: cluster 2020-09-08T16:21:25.674272+0000 osd.0 (osd.0) 150 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.574 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:26 smithi104 bash[9203]: cluster 2020-09-08T16:21:25.674416+0000 osd.0 (osd.0) 151 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.574 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:27 smithi104 bash[9203]: cluster 2020-09-08T16:21:26.337306+0000 mgr.x (mgr.24118) 1623 : cluster [DBG] pgmap v2399: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.574 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:27 smithi104 bash[9203]: cluster 2020-09-08T16:21:26.418317+0000 mon.a (mon.0) 2485 : cluster [DBG] osdmap e861: 12 total, 12 up, 12 in 2020-09-08T16:27:02.575 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:29 smithi104 bash[9203]: cluster 2020-09-08T16:21:28.338012+0000 mgr.x (mgr.24118) 1624 : cluster [DBG] pgmap v2401: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.575 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:31 smithi104 bash[9203]: cluster 2020-09-08T16:21:30.338716+0000 mgr.x (mgr.24118) 1625 : cluster [DBG] pgmap v2402: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.575 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:33 smithi104 bash[9203]: cluster 2020-09-08T16:21:32.339416+0000 mgr.x (mgr.24118) 1626 : cluster [DBG] pgmap v2403: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.575 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:35 smithi104 bash[9203]: cluster 2020-09-08T16:21:34.340128+0000 mgr.x (mgr.24118) 1627 : cluster [DBG] pgmap v2404: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.575 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:36 smithi104 bash[9203]: cluster 2020-09-08T16:21:35.509269+0000 osd.5 (osd.5) 156 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.576 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:36 smithi104 bash[9203]: cluster 2020-09-08T16:21:35.509400+0000 osd.5 (osd.5) 157 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.576 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:37 smithi104 bash[9203]: cluster 2020-09-08T16:21:36.340795+0000 mgr.x (mgr.24118) 1628 : cluster [DBG] pgmap v2405: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.576 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:37 smithi104 bash[9203]: cluster 2020-09-08T16:21:36.420966+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e862: 12 total, 12 up, 12 in 2020-09-08T16:27:02.576 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:38 smithi104 bash[9203]: cluster 2020-09-08T16:21:37.638448+0000 osd.3 (osd.3) 159 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.576 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:38 smithi104 bash[9203]: cluster 2020-09-08T16:21:37.638607+0000 osd.3 (osd.3) 160 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.577 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:38 smithi104 bash[9203]: cluster 2020-09-08T16:21:37.868311+0000 osd.10 (osd.10) 135 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.577 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:38 smithi104 bash[9203]: cluster 2020-09-08T16:21:37.868434+0000 osd.10 (osd.10) 136 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.577 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:39 smithi104 bash[9203]: cluster 2020-09-08T16:21:38.341523+0000 mgr.x (mgr.24118) 1629 : cluster [DBG] pgmap v2407: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.577 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:39 smithi104 bash[9203]: cluster 2020-09-08T16:21:38.432367+0000 mon.a (mon.0) 2487 : cluster [DBG] osdmap e863: 12 total, 12 up, 12 in 2020-09-08T16:27:02.577 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:41 smithi104 bash[9203]: cluster 2020-09-08T16:21:40.342275+0000 mgr.x (mgr.24118) 1630 : cluster [DBG] pgmap v2409: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.578 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:43 smithi104 bash[9203]: cluster 2020-09-08T16:21:42.343020+0000 mgr.x (mgr.24118) 1631 : cluster [DBG] pgmap v2410: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.578 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:45 smithi104 bash[9203]: cluster 2020-09-08T16:21:44.343760+0000 mgr.x (mgr.24118) 1632 : cluster [DBG] pgmap v2411: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.578 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:47 smithi104 bash[9203]: cluster 2020-09-08T16:21:45.745329+0000 osd.8 (osd.8) 130 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.578 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:47 smithi104 bash[9203]: cluster 2020-09-08T16:21:45.745456+0000 osd.8 (osd.8) 131 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.578 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:47 smithi104 bash[9203]: cluster 2020-09-08T16:21:46.344503+0000 mgr.x (mgr.24118) 1633 : cluster [DBG] pgmap v2412: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.579 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:47 smithi104 bash[9203]: cluster 2020-09-08T16:21:46.423352+0000 mon.a (mon.0) 2488 : cluster [DBG] osdmap e864: 12 total, 12 up, 12 in 2020-09-08T16:27:02.579 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:49 smithi104 bash[9203]: cluster 2020-09-08T16:21:48.345450+0000 mgr.x (mgr.24118) 1634 : cluster [DBG] pgmap v2414: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.579 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:51 smithi104 bash[9203]: cluster 2020-09-08T16:21:50.346231+0000 mgr.x (mgr.24118) 1635 : cluster [DBG] pgmap v2415: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.579 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:53 smithi104 bash[9203]: cluster 2020-09-08T16:21:52.346930+0000 mgr.x (mgr.24118) 1636 : cluster [DBG] pgmap v2416: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.579 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:53 smithi104 bash[9203]: cluster 2020-09-08T16:21:52.538202+0000 osd.1 (osd.1) 176 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.580 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:53 smithi104 bash[9203]: cluster 2020-09-08T16:21:52.538339+0000 osd.1 (osd.1) 177 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.580 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:54 smithi104 bash[9203]: cluster 2020-09-08T16:21:53.419636+0000 mon.a (mon.0) 2489 : cluster [DBG] osdmap e865: 12 total, 12 up, 12 in 2020-09-08T16:27:02.580 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:55 smithi104 bash[9203]: cluster 2020-09-08T16:21:54.347582+0000 mgr.x (mgr.24118) 1637 : cluster [DBG] pgmap v2418: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.580 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:57 smithi104 bash[9203]: cluster 2020-09-08T16:21:56.348350+0000 mgr.x (mgr.24118) 1638 : cluster [DBG] pgmap v2419: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.580 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:59 smithi104 bash[9203]: cluster 2020-09-08T16:21:57.980972+0000 osd.6 (osd.6) 141 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.581 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:59 smithi104 bash[9203]: cluster 2020-09-08T16:21:57.981104+0000 osd.6 (osd.6) 142 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.581 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:59 smithi104 bash[9203]: cluster 2020-09-08T16:21:58.349119+0000 mgr.x (mgr.24118) 1639 : cluster [DBG] pgmap v2420: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.581 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:21:59 smithi104 bash[9203]: cluster 2020-09-08T16:21:58.489201+0000 mon.a (mon.0) 2490 : cluster [DBG] osdmap e866: 12 total, 12 up, 12 in 2020-09-08T16:27:02.581 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:01 smithi104 bash[9203]: cluster 2020-09-08T16:22:00.349788+0000 mgr.x (mgr.24118) 1640 : cluster [DBG] pgmap v2422: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.581 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:03 smithi104 bash[9203]: cluster 2020-09-08T16:22:02.350450+0000 mgr.x (mgr.24118) 1641 : cluster [DBG] pgmap v2423: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.582 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:04 smithi104 bash[9203]: cluster 2020-09-08T16:22:03.859539+0000 osd.11 (osd.11) 156 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.582 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:04 smithi104 bash[9203]: cluster 2020-09-08T16:22:03.859668+0000 osd.11 (osd.11) 157 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.582 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:05 smithi104 bash[9203]: cluster 2020-09-08T16:22:04.351125+0000 mgr.x (mgr.24118) 1642 : cluster [DBG] pgmap v2424: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.582 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:05 smithi104 bash[9203]: cluster 2020-09-08T16:22:04.426415+0000 mon.a (mon.0) 2491 : cluster [DBG] osdmap e867: 12 total, 12 up, 12 in 2020-09-08T16:27:02.582 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:07 smithi104 bash[9203]: cluster 2020-09-08T16:22:06.292113+0000 osd.4 (osd.4) 101 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.583 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:07 smithi104 bash[9203]: cluster 2020-09-08T16:22:06.292235+0000 osd.4 (osd.4) 102 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.583 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:07 smithi104 bash[9203]: cluster 2020-09-08T16:22:06.351776+0000 mgr.x (mgr.24118) 1643 : cluster [DBG] pgmap v2426: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.583 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:07 smithi104 bash[9203]: cluster 2020-09-08T16:22:06.437707+0000 mon.a (mon.0) 2492 : cluster [DBG] osdmap e868: 12 total, 12 up, 12 in 2020-09-08T16:27:02.583 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:09 smithi104 bash[9203]: cluster 2020-09-08T16:22:08.352672+0000 mgr.x (mgr.24118) 1644 : cluster [DBG] pgmap v2428: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.583 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:11 smithi104 bash[9203]: cluster 2020-09-08T16:22:10.353519+0000 mgr.x (mgr.24118) 1645 : cluster [DBG] pgmap v2429: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.584 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:13 smithi104 bash[9203]: cluster 2020-09-08T16:22:12.354224+0000 mgr.x (mgr.24118) 1646 : cluster [DBG] pgmap v2430: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.584 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:15 smithi104 bash[9203]: cluster 2020-09-08T16:22:14.354913+0000 mgr.x (mgr.24118) 1647 : cluster [DBG] pgmap v2431: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.584 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:17 smithi104 bash[9203]: cluster 2020-09-08T16:22:16.355583+0000 mgr.x (mgr.24118) 1648 : cluster [DBG] pgmap v2432: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.584 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:17 smithi104 bash[9203]: cluster 2020-09-08T16:22:16.660205+0000 osd.9 (osd.9) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.584 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:17 smithi104 bash[9203]: cluster 2020-09-08T16:22:16.660363+0000 osd.9 (osd.9) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.585 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:18 smithi104 bash[9203]: cluster 2020-09-08T16:22:17.427768+0000 mon.a (mon.0) 2493 : cluster [DBG] osdmap e869: 12 total, 12 up, 12 in 2020-09-08T16:27:02.585 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:18 smithi104 bash[9203]: cluster 2020-09-08T16:22:17.988554+0000 osd.2 (osd.2) 136 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.585 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:18 smithi104 bash[9203]: cluster 2020-09-08T16:22:17.988686+0000 osd.2 (osd.2) 137 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.585 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:19 smithi104 bash[9203]: cluster 2020-09-08T16:22:18.356281+0000 mgr.x (mgr.24118) 1649 : cluster [DBG] pgmap v2434: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.585 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:19 smithi104 bash[9203]: cluster 2020-09-08T16:22:18.446223+0000 mon.a (mon.0) 2494 : cluster [DBG] osdmap e870: 12 total, 12 up, 12 in 2020-09-08T16:27:02.586 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:21 smithi104 bash[9203]: cluster 2020-09-08T16:22:20.356945+0000 mgr.x (mgr.24118) 1650 : cluster [DBG] pgmap v2436: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.586 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:23 smithi104 bash[9203]: cluster 2020-09-08T16:22:22.357656+0000 mgr.x (mgr.24118) 1651 : cluster [DBG] pgmap v2437: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.586 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:25 smithi104 bash[9203]: cluster 2020-09-08T16:22:24.358372+0000 mgr.x (mgr.24118) 1652 : cluster [DBG] pgmap v2438: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.586 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:27 smithi104 bash[9203]: cluster 2020-09-08T16:22:26.047624+0000 osd.7 (osd.7) 192 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.586 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:27 smithi104 bash[9203]: cluster 2020-09-08T16:22:26.047749+0000 osd.7 (osd.7) 193 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.587 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:27 smithi104 bash[9203]: cluster 2020-09-08T16:22:26.358984+0000 mgr.x (mgr.24118) 1653 : cluster [DBG] pgmap v2439: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.587 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:27 smithi104 bash[9203]: cluster 2020-09-08T16:22:26.438539+0000 mon.a (mon.0) 2495 : cluster [DBG] osdmap e871: 12 total, 12 up, 12 in 2020-09-08T16:27:02.587 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:27 smithi104 bash[9203]: cluster 2020-09-08T16:22:26.684716+0000 osd.0 (osd.0) 152 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.587 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:27 smithi104 bash[9203]: cluster 2020-09-08T16:22:26.684841+0000 osd.0 (osd.0) 153 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.587 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:28 smithi104 bash[9203]: cluster 2020-09-08T16:22:27.441484+0000 mon.a (mon.0) 2496 : cluster [DBG] osdmap e872: 12 total, 12 up, 12 in 2020-09-08T16:27:02.588 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:29 smithi104 bash[9203]: cluster 2020-09-08T16:22:28.359677+0000 mgr.x (mgr.24118) 1654 : cluster [DBG] pgmap v2442: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.588 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:31 smithi104 bash[9203]: cluster 2020-09-08T16:22:30.360347+0000 mgr.x (mgr.24118) 1655 : cluster [DBG] pgmap v2443: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.588 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:33 smithi104 bash[9203]: cluster 2020-09-08T16:22:32.361009+0000 mgr.x (mgr.24118) 1656 : cluster [DBG] pgmap v2444: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.588 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:35 smithi104 bash[9203]: cluster 2020-09-08T16:22:34.361722+0000 mgr.x (mgr.24118) 1657 : cluster [DBG] pgmap v2445: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.588 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:37 smithi104 bash[9203]: cluster 2020-09-08T16:22:36.201081+0000 osd.5 (osd.5) 158 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.589 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:37 smithi104 bash[9203]: cluster 2020-09-08T16:22:36.201209+0000 osd.5 (osd.5) 159 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.589 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:37 smithi104 bash[9203]: cluster 2020-09-08T16:22:36.362454+0000 mgr.x (mgr.24118) 1658 : cluster [DBG] pgmap v2446: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.589 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:37 smithi104 bash[9203]: cluster 2020-09-08T16:22:36.438961+0000 mon.a (mon.0) 2497 : cluster [DBG] osdmap e873: 12 total, 12 up, 12 in 2020-09-08T16:27:02.589 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:38 smithi104 bash[9203]: cluster 2020-09-08T16:22:37.838485+0000 osd.3 (osd.3) 161 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.589 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:38 smithi104 bash[9203]: cluster 2020-09-08T16:22:37.838627+0000 osd.3 (osd.3) 162 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.590 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:39 smithi104 bash[9203]: cluster 2020-09-08T16:22:38.363202+0000 mgr.x (mgr.24118) 1659 : cluster [DBG] pgmap v2448: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.590 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:39 smithi104 bash[9203]: cluster 2020-09-08T16:22:38.454615+0000 mon.a (mon.0) 2498 : cluster [DBG] osdmap e874: 12 total, 12 up, 12 in 2020-09-08T16:27:02.590 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:39 smithi104 bash[9203]: cluster 2020-09-08T16:22:38.838416+0000 osd.10 (osd.10) 137 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.590 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:39 smithi104 bash[9203]: cluster 2020-09-08T16:22:38.838535+0000 osd.10 (osd.10) 138 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.590 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:40 smithi104 bash[9203]: cluster 2020-09-08T16:22:39.463943+0000 mon.a (mon.0) 2499 : cluster [DBG] osdmap e875: 12 total, 12 up, 12 in 2020-09-08T16:27:02.591 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:41 smithi104 bash[9203]: cluster 2020-09-08T16:22:40.363881+0000 mgr.x (mgr.24118) 1660 : cluster [DBG] pgmap v2451: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.591 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:43 smithi104 bash[9203]: cluster 2020-09-08T16:22:42.364535+0000 mgr.x (mgr.24118) 1661 : cluster [DBG] pgmap v2452: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.591 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:45 smithi104 bash[9203]: cluster 2020-09-08T16:22:44.365151+0000 mgr.x (mgr.24118) 1662 : cluster [DBG] pgmap v2453: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.591 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:47 smithi104 bash[9203]: cluster 2020-09-08T16:22:45.923608+0000 osd.8 (osd.8) 132 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.591 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:47 smithi104 bash[9203]: cluster 2020-09-08T16:22:45.923735+0000 osd.8 (osd.8) 133 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.592 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:47 smithi104 bash[9203]: cluster 2020-09-08T16:22:46.365787+0000 mgr.x (mgr.24118) 1663 : cluster [DBG] pgmap v2454: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.592 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:47 smithi104 bash[9203]: cluster 2020-09-08T16:22:46.517760+0000 mon.a (mon.0) 2500 : cluster [DBG] osdmap e876: 12 total, 12 up, 12 in 2020-09-08T16:27:02.592 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:49 smithi104 bash[9203]: cluster 2020-09-08T16:22:48.366566+0000 mgr.x (mgr.24118) 1664 : cluster [DBG] pgmap v2456: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.592 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:51 smithi104 bash[9203]: cluster 2020-09-08T16:22:50.367302+0000 mgr.x (mgr.24118) 1665 : cluster [DBG] pgmap v2457: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.592 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:53 smithi104 bash[9203]: cluster 2020-09-08T16:22:52.368146+0000 mgr.x (mgr.24118) 1666 : cluster [DBG] pgmap v2458: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.593 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:54 smithi104 bash[9203]: cluster 2020-09-08T16:22:53.374169+0000 osd.1 (osd.1) 178 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.593 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:54 smithi104 bash[9203]: cluster 2020-09-08T16:22:53.374299+0000 osd.1 (osd.1) 179 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.593 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:54 smithi104 bash[9203]: cluster 2020-09-08T16:22:53.430286+0000 mon.a (mon.0) 2501 : cluster [DBG] osdmap e877: 12 total, 12 up, 12 in 2020-09-08T16:27:02.593 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:55 smithi104 bash[9203]: cluster 2020-09-08T16:22:54.368841+0000 mgr.x (mgr.24118) 1667 : cluster [DBG] pgmap v2460: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.593 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:57 smithi104 bash[9203]: cluster 2020-09-08T16:22:56.369497+0000 mgr.x (mgr.24118) 1668 : cluster [DBG] pgmap v2461: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.594 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:59 smithi104 bash[9203]: cluster 2020-09-08T16:22:58.338433+0000 osd.6 (osd.6) 143 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.594 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:59 smithi104 bash[9203]: cluster 2020-09-08T16:22:58.338558+0000 osd.6 (osd.6) 144 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.594 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:59 smithi104 bash[9203]: cluster 2020-09-08T16:22:58.370232+0000 mgr.x (mgr.24118) 1669 : cluster [DBG] pgmap v2462: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.594 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:22:59 smithi104 bash[9203]: cluster 2020-09-08T16:22:58.512036+0000 mon.a (mon.0) 2502 : cluster [DBG] osdmap e878: 12 total, 12 up, 12 in 2020-09-08T16:27:02.594 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:01 smithi104 bash[9203]: cluster 2020-09-08T16:23:00.371005+0000 mgr.x (mgr.24118) 1670 : cluster [DBG] pgmap v2464: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.595 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:03 smithi104 bash[9203]: cluster 2020-09-08T16:23:02.371690+0000 mgr.x (mgr.24118) 1671 : cluster [DBG] pgmap v2465: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.595 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:05 smithi104 bash[9203]: cluster 2020-09-08T16:23:04.324373+0000 osd.11 (osd.11) 158 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.595 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:05 smithi104 bash[9203]: cluster 2020-09-08T16:23:04.324501+0000 osd.11 (osd.11) 159 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.595 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:05 smithi104 bash[9203]: cluster 2020-09-08T16:23:04.372389+0000 mgr.x (mgr.24118) 1672 : cluster [DBG] pgmap v2466: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.595 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:05 smithi104 bash[9203]: cluster 2020-09-08T16:23:04.443525+0000 mon.a (mon.0) 2503 : cluster [DBG] osdmap e879: 12 total, 12 up, 12 in 2020-09-08T16:27:02.596 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:07 smithi104 bash[9203]: cluster 2020-09-08T16:23:06.373040+0000 mgr.x (mgr.24118) 1673 : cluster [DBG] pgmap v2468: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.596 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:08 smithi104 bash[9203]: cluster 2020-09-08T16:23:07.305093+0000 osd.4 (osd.4) 103 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.596 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:08 smithi104 bash[9203]: cluster 2020-09-08T16:23:07.305222+0000 osd.4 (osd.4) 104 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.596 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:08 smithi104 bash[9203]: cluster 2020-09-08T16:23:07.453557+0000 mon.a (mon.0) 2504 : cluster [DBG] osdmap e880: 12 total, 12 up, 12 in 2020-09-08T16:27:02.596 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:09 smithi104 bash[9203]: cluster 2020-09-08T16:23:08.373752+0000 mgr.x (mgr.24118) 1674 : cluster [DBG] pgmap v2470: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.597 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:11 smithi104 bash[9203]: cluster 2020-09-08T16:23:10.374490+0000 mgr.x (mgr.24118) 1675 : cluster [DBG] pgmap v2471: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.597 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:13 smithi104 bash[9203]: cluster 2020-09-08T16:23:12.375180+0000 mgr.x (mgr.24118) 1676 : cluster [DBG] pgmap v2472: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.597 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:15 smithi104 bash[9203]: cluster 2020-09-08T16:23:14.375859+0000 mgr.x (mgr.24118) 1677 : cluster [DBG] pgmap v2473: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.597 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:17 smithi104 bash[9203]: cluster 2020-09-08T16:23:16.376528+0000 mgr.x (mgr.24118) 1678 : cluster [DBG] pgmap v2474: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.597 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:18 smithi104 bash[9203]: cluster 2020-09-08T16:23:17.566849+0000 osd.9 (osd.9) 142 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.598 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:18 smithi104 bash[9203]: cluster 2020-09-08T16:23:17.566983+0000 osd.9 (osd.9) 143 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.598 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:18 smithi104 bash[9203]: cluster 2020-09-08T16:23:18.155326+0000 osd.2 (osd.2) 138 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.598 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:18 smithi104 bash[9203]: cluster 2020-09-08T16:23:18.155443+0000 osd.2 (osd.2) 139 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.598 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:19 smithi104 bash[9203]: cluster 2020-09-08T16:23:18.377221+0000 mgr.x (mgr.24118) 1679 : cluster [DBG] pgmap v2475: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.598 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:19 smithi104 bash[9203]: cluster 2020-09-08T16:23:18.463865+0000 mon.a (mon.0) 2505 : cluster [DBG] osdmap e881: 12 total, 12 up, 12 in 2020-09-08T16:27:02.599 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:21 smithi104 bash[9203]: cluster 2020-09-08T16:23:20.377865+0000 mgr.x (mgr.24118) 1680 : cluster [DBG] pgmap v2477: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.599 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:23 smithi104 bash[9203]: cluster 2020-09-08T16:23:22.378582+0000 mgr.x (mgr.24118) 1681 : cluster [DBG] pgmap v2478: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.599 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:25 smithi104 bash[9203]: cluster 2020-09-08T16:23:24.379267+0000 mgr.x (mgr.24118) 1682 : cluster [DBG] pgmap v2479: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.599 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:27 smithi104 bash[9203]: cluster 2020-09-08T16:23:26.379953+0000 mgr.x (mgr.24118) 1683 : cluster [DBG] pgmap v2480: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.599 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:27 smithi104 bash[9203]: cluster 2020-09-08T16:23:26.963247+0000 osd.0 (osd.0) 154 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.600 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:27 smithi104 bash[9203]: cluster 2020-09-08T16:23:26.963376+0000 osd.0 (osd.0) 155 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.600 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:28 smithi104 bash[9203]: cluster 2020-09-08T16:23:27.002998+0000 osd.7 (osd.7) 194 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.600 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:28 smithi104 bash[9203]: cluster 2020-09-08T16:23:27.003141+0000 osd.7 (osd.7) 195 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.600 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:28 smithi104 bash[9203]: cluster 2020-09-08T16:23:27.453332+0000 mon.a (mon.0) 2506 : cluster [DBG] osdmap e882: 12 total, 12 up, 12 in 2020-09-08T16:27:02.600 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:29 smithi104 bash[9203]: cluster 2020-09-08T16:23:28.380663+0000 mgr.x (mgr.24118) 1684 : cluster [DBG] pgmap v2482: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.601 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:31 smithi104 bash[9203]: cluster 2020-09-08T16:23:30.381335+0000 mgr.x (mgr.24118) 1685 : cluster [DBG] pgmap v2483: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.601 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:33 smithi104 bash[9203]: cluster 2020-09-08T16:23:32.382017+0000 mgr.x (mgr.24118) 1686 : cluster [DBG] pgmap v2484: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.601 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:35 smithi104 bash[9203]: cluster 2020-09-08T16:23:34.382751+0000 mgr.x (mgr.24118) 1687 : cluster [DBG] pgmap v2485: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.601 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:37 smithi104 bash[9203]: cluster 2020-09-08T16:23:36.383441+0000 mgr.x (mgr.24118) 1688 : cluster [DBG] pgmap v2486: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.601 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:37 smithi104 bash[9203]: cluster 2020-09-08T16:23:36.568336+0000 osd.5 (osd.5) 160 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.602 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:37 smithi104 bash[9203]: cluster 2020-09-08T16:23:36.568492+0000 osd.5 (osd.5) 161 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.602 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:38 smithi104 bash[9203]: cluster 2020-09-08T16:23:37.486484+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e883: 12 total, 12 up, 12 in 2020-09-08T16:27:02.602 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:38 smithi104 bash[9203]: cluster 2020-09-08T16:23:37.905775+0000 osd.3 (osd.3) 163 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.602 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:38 smithi104 bash[9203]: cluster 2020-09-08T16:23:37.905899+0000 osd.3 (osd.3) 164 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.602 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:39 smithi104 bash[9203]: cluster 2020-09-08T16:23:38.384285+0000 mgr.x (mgr.24118) 1689 : cluster [DBG] pgmap v2488: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.603 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:39 smithi104 bash[9203]: cluster 2020-09-08T16:23:38.499022+0000 mon.a (mon.0) 2508 : cluster [DBG] osdmap e884: 12 total, 12 up, 12 in 2020-09-08T16:27:02.603 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:40 smithi104 bash[9203]: cluster 2020-09-08T16:23:39.576603+0000 osd.10 (osd.10) 139 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.603 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:40 smithi104 bash[9203]: cluster 2020-09-08T16:23:39.576724+0000 osd.10 (osd.10) 140 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.603 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:41 smithi104 bash[9203]: cluster 2020-09-08T16:23:40.384981+0000 mgr.x (mgr.24118) 1690 : cluster [DBG] pgmap v2490: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.603 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:41 smithi104 bash[9203]: cluster 2020-09-08T16:23:40.503892+0000 mon.a (mon.0) 2509 : cluster [DBG] osdmap e885: 12 total, 12 up, 12 in 2020-09-08T16:27:02.604 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:43 smithi104 bash[9203]: cluster 2020-09-08T16:23:42.385743+0000 mgr.x (mgr.24118) 1691 : cluster [DBG] pgmap v2492: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.604 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:45 smithi104 bash[9203]: cluster 2020-09-08T16:23:44.386546+0000 mgr.x (mgr.24118) 1692 : cluster [DBG] pgmap v2493: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.604 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:47 smithi104 bash[9203]: cluster 2020-09-08T16:23:46.387299+0000 mgr.x (mgr.24118) 1693 : cluster [DBG] pgmap v2494: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.604 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:48 smithi104 bash[9203]: cluster 2020-09-08T16:23:46.640159+0000 osd.8 (osd.8) 134 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.604 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:48 smithi104 bash[9203]: cluster 2020-09-08T16:23:46.640302+0000 osd.8 (osd.8) 135 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.605 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:48 smithi104 bash[9203]: cluster 2020-09-08T16:23:47.531865+0000 mon.a (mon.0) 2510 : cluster [DBG] osdmap e886: 12 total, 12 up, 12 in 2020-09-08T16:27:02.605 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:49 smithi104 bash[9203]: cluster 2020-09-08T16:23:48.388036+0000 mgr.x (mgr.24118) 1694 : cluster [DBG] pgmap v2496: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.605 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:51 smithi104 bash[9203]: cluster 2020-09-08T16:23:50.388662+0000 mgr.x (mgr.24118) 1695 : cluster [DBG] pgmap v2497: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.605 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:53 smithi104 bash[9203]: cluster 2020-09-08T16:23:52.389418+0000 mgr.x (mgr.24118) 1696 : cluster [DBG] pgmap v2498: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.605 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:55 smithi104 bash[9203]: cluster 2020-09-08T16:23:54.236396+0000 osd.1 (osd.1) 180 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.606 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:55 smithi104 bash[9203]: cluster 2020-09-08T16:23:54.236544+0000 osd.1 (osd.1) 181 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.606 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:55 smithi104 bash[9203]: cluster 2020-09-08T16:23:54.390149+0000 mgr.x (mgr.24118) 1697 : cluster [DBG] pgmap v2499: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.606 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:55 smithi104 bash[9203]: cluster 2020-09-08T16:23:54.464578+0000 mon.a (mon.0) 2511 : cluster [DBG] osdmap e887: 12 total, 12 up, 12 in 2020-09-08T16:27:02.606 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:57 smithi104 bash[9203]: cluster 2020-09-08T16:23:56.390826+0000 mgr.x (mgr.24118) 1698 : cluster [DBG] pgmap v2501: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.606 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:23:59 smithi104 bash[9203]: cluster 2020-09-08T16:23:58.391558+0000 mgr.x (mgr.24118) 1699 : cluster [DBG] pgmap v2502: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.607 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:00 smithi104 bash[9203]: cluster 2020-09-08T16:23:59.299180+0000 osd.6 (osd.6) 145 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.607 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:00 smithi104 bash[9203]: cluster 2020-09-08T16:23:59.299303+0000 osd.6 (osd.6) 146 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.607 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:00 smithi104 bash[9203]: cluster 2020-09-08T16:23:59.521102+0000 mon.a (mon.0) 2512 : cluster [DBG] osdmap e888: 12 total, 12 up, 12 in 2020-09-08T16:27:02.607 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:01 smithi104 bash[9203]: cluster 2020-09-08T16:24:00.392224+0000 mgr.x (mgr.24118) 1700 : cluster [DBG] pgmap v2504: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.607 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:03 smithi104 bash[9203]: cluster 2020-09-08T16:24:02.392901+0000 mgr.x (mgr.24118) 1701 : cluster [DBG] pgmap v2505: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.608 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:05 smithi104 bash[9203]: cluster 2020-09-08T16:24:04.393624+0000 mgr.x (mgr.24118) 1702 : cluster [DBG] pgmap v2506: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.608 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:05 smithi104 bash[9203]: cluster 2020-09-08T16:24:04.508302+0000 osd.11 (osd.11) 160 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.608 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:05 smithi104 bash[9203]: cluster 2020-09-08T16:24:04.508429+0000 osd.11 (osd.11) 161 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.608 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:05 smithi104 bash[9203]: cluster 2020-09-08T16:24:04.561230+0000 mon.a (mon.0) 2513 : cluster [DBG] osdmap e889: 12 total, 12 up, 12 in 2020-09-08T16:27:02.608 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:07 smithi104 bash[9203]: cluster 2020-09-08T16:24:06.394289+0000 mgr.x (mgr.24118) 1703 : cluster [DBG] pgmap v2508: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.609 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:07 smithi104 bash[9203]: cluster 2020-09-08T16:24:07.389831+0000 osd.4 (osd.4) 105 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.609 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:07 smithi104 bash[9203]: cluster 2020-09-08T16:24:07.389953+0000 osd.4 (osd.4) 106 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.609 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:08 smithi104 bash[9203]: cluster 2020-09-08T16:24:07.569285+0000 mon.a (mon.0) 2514 : cluster [DBG] osdmap e890: 12 total, 12 up, 12 in 2020-09-08T16:27:02.609 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:09 smithi104 bash[9203]: cluster 2020-09-08T16:24:08.395016+0000 mgr.x (mgr.24118) 1704 : cluster [DBG] pgmap v2510: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.609 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:11 smithi104 bash[9203]: cluster 2020-09-08T16:24:10.395654+0000 mgr.x (mgr.24118) 1705 : cluster [DBG] pgmap v2511: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.610 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:13 smithi104 bash[9203]: cluster 2020-09-08T16:24:12.396433+0000 mgr.x (mgr.24118) 1706 : cluster [DBG] pgmap v2512: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.610 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:14 smithi104 bash[9203]: audit 2020-09-08T16:24:13.474986+0000 mon.b (mon.1) 138 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.610 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:14 smithi104 bash[9203]: audit 2020-09-08T16:24:13.477045+0000 mon.a (mon.0) 2515 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.610 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:14 smithi104 bash[9203]: audit 2020-09-08T16:24:13.482442+0000 mon.a (mon.0) 2516 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.610 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:15 smithi104 bash[9203]: cluster 2020-09-08T16:24:14.397293+0000 mgr.x (mgr.24118) 1707 : cluster [DBG] pgmap v2513: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.611 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:17 smithi104 bash[9203]: cluster 2020-09-08T16:24:16.398030+0000 mgr.x (mgr.24118) 1708 : cluster [DBG] pgmap v2514: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.611 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:18 smithi104 bash[9203]: cluster 2020-09-08T16:24:17.588032+0000 osd.9 (osd.9) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.611 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:18 smithi104 bash[9203]: cluster 2020-09-08T16:24:17.588165+0000 osd.9 (osd.9) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.611 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:19 smithi104 bash[9203]: cluster 2020-09-08T16:24:18.398882+0000 mgr.x (mgr.24118) 1709 : cluster [DBG] pgmap v2515: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.611 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:19 smithi104 bash[9203]: cluster 2020-09-08T16:24:18.542472+0000 mon.a (mon.0) 2517 : cluster [DBG] osdmap e891: 12 total, 12 up, 12 in 2020-09-08T16:27:02.612 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:19 smithi104 bash[9203]: cluster 2020-09-08T16:24:18.764818+0000 osd.2 (osd.2) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.612 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:19 smithi104 bash[9203]: cluster 2020-09-08T16:24:18.764929+0000 osd.2 (osd.2) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.612 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:20 smithi104 bash[9203]: cluster 2020-09-08T16:24:19.546699+0000 mon.a (mon.0) 2518 : cluster [DBG] osdmap e892: 12 total, 12 up, 12 in 2020-09-08T16:27:02.612 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:21 smithi104 bash[9203]: cluster 2020-09-08T16:24:20.399622+0000 mgr.x (mgr.24118) 1710 : cluster [DBG] pgmap v2518: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.612 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:23 smithi104 bash[9203]: cluster 2020-09-08T16:24:22.400285+0000 mgr.x (mgr.24118) 1711 : cluster [DBG] pgmap v2519: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.613 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:25 smithi104 bash[9203]: cluster 2020-09-08T16:24:24.400983+0000 mgr.x (mgr.24118) 1712 : cluster [DBG] pgmap v2520: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.613 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:27 smithi104 bash[9203]: cluster 2020-09-08T16:24:26.401749+0000 mgr.x (mgr.24118) 1713 : cluster [DBG] pgmap v2521: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.613 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:27 smithi104 bash[9203]: cluster 2020-09-08T16:24:27.099750+0000 osd.0 (osd.0) 156 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.613 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:27 smithi104 bash[9203]: cluster 2020-09-08T16:24:27.099880+0000 osd.0 (osd.0) 157 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.613 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:28 smithi104 bash[9203]: cluster 2020-09-08T16:24:27.315084+0000 osd.7 (osd.7) 196 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.614 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:28 smithi104 bash[9203]: cluster 2020-09-08T16:24:27.315209+0000 osd.7 (osd.7) 197 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.614 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:28 smithi104 bash[9203]: cluster 2020-09-08T16:24:27.581255+0000 mon.a (mon.0) 2519 : cluster [DBG] osdmap e893: 12 total, 12 up, 12 in 2020-09-08T16:27:02.614 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:29 smithi104 bash[9203]: cluster 2020-09-08T16:24:28.402497+0000 mgr.x (mgr.24118) 1714 : cluster [DBG] pgmap v2523: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.614 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:31 smithi104 bash[9203]: cluster 2020-09-08T16:24:30.403186+0000 mgr.x (mgr.24118) 1715 : cluster [DBG] pgmap v2524: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.614 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:33 smithi104 bash[9203]: cluster 2020-09-08T16:24:32.403863+0000 mgr.x (mgr.24118) 1716 : cluster [DBG] pgmap v2525: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.615 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:35 smithi104 bash[9203]: cluster 2020-09-08T16:24:34.404578+0000 mgr.x (mgr.24118) 1717 : cluster [DBG] pgmap v2526: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.615 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:37 smithi104 bash[9203]: cluster 2020-09-08T16:24:36.405231+0000 mgr.x (mgr.24118) 1718 : cluster [DBG] pgmap v2527: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.615 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:38 smithi104 bash[9203]: cluster 2020-09-08T16:24:37.495138+0000 osd.5 (osd.5) 162 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.615 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:38 smithi104 bash[9203]: cluster 2020-09-08T16:24:37.495296+0000 osd.5 (osd.5) 163 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.615 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:38 smithi104 bash[9203]: cluster 2020-09-08T16:24:37.909834+0000 osd.3 (osd.3) 165 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.616 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:38 smithi104 bash[9203]: cluster 2020-09-08T16:24:37.909972+0000 osd.3 (osd.3) 166 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.616 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:39 smithi104 bash[9203]: cluster 2020-09-08T16:24:38.405935+0000 mgr.x (mgr.24118) 1719 : cluster [DBG] pgmap v2528: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.616 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:39 smithi104 bash[9203]: cluster 2020-09-08T16:24:38.492275+0000 mon.a (mon.0) 2520 : cluster [DBG] osdmap e894: 12 total, 12 up, 12 in 2020-09-08T16:27:02.616 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:41 smithi104 bash[9203]: cluster 2020-09-08T16:24:40.353237+0000 osd.10 (osd.10) 141 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.616 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:41 smithi104 bash[9203]: cluster 2020-09-08T16:24:40.353357+0000 osd.10 (osd.10) 142 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.617 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:41 smithi104 bash[9203]: cluster 2020-09-08T16:24:40.406644+0000 mgr.x (mgr.24118) 1720 : cluster [DBG] pgmap v2530: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.617 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:42 smithi104 bash[9203]: cluster 2020-09-08T16:24:41.325942+0000 mon.a (mon.0) 2521 : cluster [DBG] osdmap e895: 12 total, 12 up, 12 in 2020-09-08T16:27:02.617 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:43 smithi104 bash[9203]: cluster 2020-09-08T16:24:42.407347+0000 mgr.x (mgr.24118) 1721 : cluster [DBG] pgmap v2532: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.617 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:45 smithi104 bash[9203]: cluster 2020-09-08T16:24:44.408070+0000 mgr.x (mgr.24118) 1722 : cluster [DBG] pgmap v2533: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.617 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:47 smithi104 bash[9203]: cluster 2020-09-08T16:24:46.408737+0000 mgr.x (mgr.24118) 1723 : cluster [DBG] pgmap v2534: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.618 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:49 smithi104 bash[9203]: cluster 2020-09-08T16:24:47.670412+0000 osd.8 (osd.8) 136 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.618 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:49 smithi104 bash[9203]: cluster 2020-09-08T16:24:47.670538+0000 osd.8 (osd.8) 137 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.618 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:49 smithi104 bash[9203]: cluster 2020-09-08T16:24:48.409508+0000 mgr.x (mgr.24118) 1724 : cluster [DBG] pgmap v2535: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.618 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:49 smithi104 bash[9203]: cluster 2020-09-08T16:24:48.486000+0000 mon.a (mon.0) 2522 : cluster [DBG] osdmap e896: 12 total, 12 up, 12 in 2020-09-08T16:27:02.618 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:51 smithi104 bash[9203]: cluster 2020-09-08T16:24:50.410325+0000 mgr.x (mgr.24118) 1725 : cluster [DBG] pgmap v2537: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.624 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:53 smithi104 bash[9203]: cluster 2020-09-08T16:24:52.411043+0000 mgr.x (mgr.24118) 1726 : cluster [DBG] pgmap v2538: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.624 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:55 smithi104 bash[9203]: cluster 2020-09-08T16:24:54.411761+0000 mgr.x (mgr.24118) 1727 : cluster [DBG] pgmap v2539: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.625 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:55 smithi104 bash[9203]: cluster 2020-09-08T16:24:54.547404+0000 osd.1 (osd.1) 182 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.625 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:55 smithi104 bash[9203]: cluster 2020-09-08T16:24:54.547537+0000 osd.1 (osd.1) 183 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.625 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:56 smithi104 bash[9203]: cluster 2020-09-08T16:24:55.526084+0000 mon.a (mon.0) 2523 : cluster [DBG] osdmap e897: 12 total, 12 up, 12 in 2020-09-08T16:27:02.625 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:57 smithi104 bash[9203]: cluster 2020-09-08T16:24:56.412405+0000 mgr.x (mgr.24118) 1728 : cluster [DBG] pgmap v2541: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.625 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:24:59 smithi104 bash[9203]: cluster 2020-09-08T16:24:58.413152+0000 mgr.x (mgr.24118) 1729 : cluster [DBG] pgmap v2542: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.626 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:01 smithi104 bash[9203]: cluster 2020-09-08T16:25:00.061572+0000 osd.6 (osd.6) 147 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.626 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:01 smithi104 bash[9203]: cluster 2020-09-08T16:25:00.061698+0000 osd.6 (osd.6) 148 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.626 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:01 smithi104 bash[9203]: cluster 2020-09-08T16:25:00.413852+0000 mgr.x (mgr.24118) 1730 : cluster [DBG] pgmap v2543: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.626 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:01 smithi104 bash[9203]: cluster 2020-09-08T16:25:00.548020+0000 mon.a (mon.0) 2524 : cluster [DBG] osdmap e898: 12 total, 12 up, 12 in 2020-09-08T16:27:02.626 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:03 smithi104 bash[9203]: cluster 2020-09-08T16:25:02.414568+0000 mgr.x (mgr.24118) 1731 : cluster [DBG] pgmap v2545: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.627 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:05 smithi104 bash[9203]: cluster 2020-09-08T16:25:04.415243+0000 mgr.x (mgr.24118) 1732 : cluster [DBG] pgmap v2546: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.627 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:05 smithi104 bash[9203]: cluster 2020-09-08T16:25:04.648267+0000 osd.11 (osd.11) 162 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.627 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:05 smithi104 bash[9203]: cluster 2020-09-08T16:25:04.648410+0000 osd.11 (osd.11) 163 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.627 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:06 smithi104 bash[9203]: cluster 2020-09-08T16:25:05.562272+0000 mon.a (mon.0) 2525 : cluster [DBG] osdmap e899: 12 total, 12 up, 12 in 2020-09-08T16:27:02.627 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:07 smithi104 bash[9203]: cluster 2020-09-08T16:25:06.415870+0000 mgr.x (mgr.24118) 1733 : cluster [DBG] pgmap v2548: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.628 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:07 smithi104 bash[9203]: cluster 2020-09-08T16:25:07.405400+0000 osd.4 (osd.4) 107 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.628 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:07 smithi104 bash[9203]: cluster 2020-09-08T16:25:07.405548+0000 osd.4 (osd.4) 108 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.628 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:08 smithi104 bash[9203]: cluster 2020-09-08T16:25:07.593722+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e900: 12 total, 12 up, 12 in 2020-09-08T16:27:02.628 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:09 smithi104 bash[9203]: cluster 2020-09-08T16:25:08.416541+0000 mgr.x (mgr.24118) 1734 : cluster [DBG] pgmap v2550: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.628 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:11 smithi104 bash[9203]: cluster 2020-09-08T16:25:10.417322+0000 mgr.x (mgr.24118) 1735 : cluster [DBG] pgmap v2551: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.629 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:13 smithi104 bash[9203]: cluster 2020-09-08T16:25:12.418166+0000 mgr.x (mgr.24118) 1736 : cluster [DBG] pgmap v2552: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.629 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:15 smithi104 bash[9203]: cluster 2020-09-08T16:25:14.418937+0000 mgr.x (mgr.24118) 1737 : cluster [DBG] pgmap v2553: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.629 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:17 smithi104 bash[9203]: cluster 2020-09-08T16:25:16.419618+0000 mgr.x (mgr.24118) 1738 : cluster [DBG] pgmap v2554: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.629 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:19 smithi104 bash[9203]: cluster 2020-09-08T16:25:18.420356+0000 mgr.x (mgr.24118) 1739 : cluster [DBG] pgmap v2555: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.630 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:19 smithi104 bash[9203]: cluster 2020-09-08T16:25:18.490436+0000 osd.9 (osd.9) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.630 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:19 smithi104 bash[9203]: cluster 2020-09-08T16:25:18.490614+0000 osd.9 (osd.9) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.630 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:20 smithi104 bash[9203]: cluster 2020-09-08T16:25:19.498676+0000 mon.a (mon.0) 2527 : cluster [DBG] osdmap e901: 12 total, 12 up, 12 in 2020-09-08T16:27:02.630 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:20 smithi104 bash[9203]: cluster 2020-09-08T16:25:19.710713+0000 osd.2 (osd.2) 142 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.630 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:20 smithi104 bash[9203]: cluster 2020-09-08T16:25:19.710828+0000 osd.2 (osd.2) 143 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.631 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:21 smithi104 bash[9203]: cluster 2020-09-08T16:25:20.421020+0000 mgr.x (mgr.24118) 1740 : cluster [DBG] pgmap v2557: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.631 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:21 smithi104 bash[9203]: cluster 2020-09-08T16:25:20.510963+0000 mon.a (mon.0) 2528 : cluster [DBG] osdmap e902: 12 total, 12 up, 12 in 2020-09-08T16:27:02.631 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:23 smithi104 bash[9203]: cluster 2020-09-08T16:25:22.421712+0000 mgr.x (mgr.24118) 1741 : cluster [DBG] pgmap v2559: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.631 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:25 smithi104 bash[9203]: cluster 2020-09-08T16:25:24.422438+0000 mgr.x (mgr.24118) 1742 : cluster [DBG] pgmap v2560: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.631 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:27 smithi104 bash[9203]: cluster 2020-09-08T16:25:26.423122+0000 mgr.x (mgr.24118) 1743 : cluster [DBG] pgmap v2561: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.632 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:28 smithi104 bash[9203]: cluster 2020-09-08T16:25:27.293029+0000 osd.0 (osd.0) 158 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.632 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:28 smithi104 bash[9203]: cluster 2020-09-08T16:25:27.293156+0000 osd.0 (osd.0) 159 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.632 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:28 smithi104 bash[9203]: cluster 2020-09-08T16:25:27.543088+0000 mon.a (mon.0) 2529 : cluster [DBG] osdmap e903: 12 total, 12 up, 12 in 2020-09-08T16:27:02.632 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:29 smithi104 bash[9203]: cluster 2020-09-08T16:25:27.910337+0000 osd.7 (osd.7) 198 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.632 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:29 smithi104 bash[9203]: cluster 2020-09-08T16:25:27.910470+0000 osd.7 (osd.7) 199 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.633 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:29 smithi104 bash[9203]: cluster 2020-09-08T16:25:28.423802+0000 mgr.x (mgr.24118) 1744 : cluster [DBG] pgmap v2563: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.633 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:29 smithi104 bash[9203]: cluster 2020-09-08T16:25:28.562409+0000 mon.a (mon.0) 2530 : cluster [DBG] osdmap e904: 12 total, 12 up, 12 in 2020-09-08T16:27:02.633 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:31 smithi104 bash[9203]: cluster 2020-09-08T16:25:30.424475+0000 mgr.x (mgr.24118) 1745 : cluster [DBG] pgmap v2565: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.633 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:33 smithi104 bash[9203]: cluster 2020-09-08T16:25:32.425199+0000 mgr.x (mgr.24118) 1746 : cluster [DBG] pgmap v2566: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.633 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:35 smithi104 bash[9203]: cluster 2020-09-08T16:25:34.425909+0000 mgr.x (mgr.24118) 1747 : cluster [DBG] pgmap v2567: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.634 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:37 smithi104 bash[9203]: cluster 2020-09-08T16:25:36.426615+0000 mgr.x (mgr.24118) 1748 : cluster [DBG] pgmap v2568: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.634 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:38 smithi104 bash[9203]: cluster 2020-09-08T16:25:37.598505+0000 osd.5 (osd.5) 164 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.634 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:38 smithi104 bash[9203]: cluster 2020-09-08T16:25:37.598661+0000 osd.5 (osd.5) 165 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.634 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:39 smithi104 bash[9203]: cluster 2020-09-08T16:25:38.427331+0000 mgr.x (mgr.24118) 1749 : cluster [DBG] pgmap v2569: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.634 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:39 smithi104 bash[9203]: cluster 2020-09-08T16:25:38.513211+0000 mon.a (mon.0) 2531 : cluster [DBG] osdmap e905: 12 total, 12 up, 12 in 2020-09-08T16:27:02.635 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:39 smithi104 bash[9203]: cluster 2020-09-08T16:25:38.764276+0000 osd.3 (osd.3) 167 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.635 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:39 smithi104 bash[9203]: cluster 2020-09-08T16:25:38.764409+0000 osd.3 (osd.3) 168 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.635 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:40 smithi104 bash[9203]: cluster 2020-09-08T16:25:39.515081+0000 mon.a (mon.0) 2532 : cluster [DBG] osdmap e906: 12 total, 12 up, 12 in 2020-09-08T16:27:02.635 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:41 smithi104 bash[9203]: cluster 2020-09-08T16:25:40.428011+0000 mgr.x (mgr.24118) 1750 : cluster [DBG] pgmap v2572: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.635 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:41 smithi104 bash[9203]: cluster 2020-09-08T16:25:40.468260+0000 osd.10 (osd.10) 143 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.635 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:41 smithi104 bash[9203]: cluster 2020-09-08T16:25:40.468351+0000 osd.10 (osd.10) 144 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.636 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:41 smithi104 bash[9203]: cluster 2020-09-08T16:25:41.349574+0000 mon.a (mon.0) 2533 : cluster [DBG] osdmap e907: 12 total, 12 up, 12 in 2020-09-08T16:27:02.636 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:43 smithi104 bash[9203]: cluster 2020-09-08T16:25:42.428652+0000 mgr.x (mgr.24118) 1751 : cluster [DBG] pgmap v2574: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.636 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:45 smithi104 bash[9203]: cluster 2020-09-08T16:25:44.429449+0000 mgr.x (mgr.24118) 1752 : cluster [DBG] pgmap v2575: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.636 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:47 smithi104 bash[9203]: cluster 2020-09-08T16:25:46.430281+0000 mgr.x (mgr.24118) 1753 : cluster [DBG] pgmap v2576: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.636 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:49 smithi104 bash[9203]: cluster 2020-09-08T16:25:48.640765+0000 osd.8 (osd.8) 138 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.637 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:49 smithi104 bash[9203]: cluster 2020-09-08T16:25:48.640899+0000 osd.8 (osd.8) 139 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.637 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:49 smithi104 bash[9203]: audit 2020-09-08T16:25:48.993840+0000 mon.a (mon.0) 2534 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.402933\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403132\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403786\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404332\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404856\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.405550\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:25:48.990918\"}"}]: dispatch 2020-09-08T16:27:02.637 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:49 smithi104 bash[9203]: audit 2020-09-08T16:25:49.000833+0000 mon.a (mon.0) 2535 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.402933\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403132\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.403786\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404332\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.404856\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.405550\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:25:48.990918\"}"}]': finished 2020-09-08T16:27:02.638 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:49 smithi104 bash[9203]: audit 2020-09-08T16:25:49.001101+0000 mon.a (mon.0) 2536 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685123\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685289\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685873\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686336\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686892\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.687437\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:25:48.995696\"}"}]: dispatch 2020-09-08T16:27:02.638 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:49 smithi104 bash[9203]: audit 2020-09-08T16:25:49.005877+0000 mon.a (mon.0) 2537 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685123\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685289\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.685873\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686336\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.686892\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.687437\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:25:48.995696\"}"}]': finished 2020-09-08T16:27:02.638 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:49 smithi104 bash[9203]: audit 2020-09-08T16:25:49.012081+0000 mon.a (mon.0) 2538 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698864\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T16:14:49.698956\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699006\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699331\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699539\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699740\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.700002\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:25:48.998320\"}"}]: dispatch 2020-09-08T16:27:02.639 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:49 smithi104 bash[9203]: audit 2020-09-08T16:25:49.012937+0000 mon.a (mon.0) 2539 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:27:02.639 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:49 smithi104 bash[9203]: audit 2020-09-08T16:25:49.016722+0000 mon.a (mon.0) 2540 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.698864\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": true, \"last_refresh\": \"2020-09-08T16:14:49.698956\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699006\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699331\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699539\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:14:49.699740\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:14:49.700002\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:25:48.998320\"}"}]': finished 2020-09-08T16:27:02.639 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:49 smithi104 bash[9203]: audit 2020-09-08T16:25:49.017007+0000 mon.a (mon.0) 2541 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:27:02.639 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:49 smithi104 bash[9203]: audit 2020-09-08T16:25:49.017561+0000 mon.a (mon.0) 2542 : audit [DBG] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "container_image"}]: dispatch 2020-09-08T16:27:02.640 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:50 smithi104 bash[9203]: cluster 2020-09-08T16:25:49.619281+0000 mon.a (mon.0) 2543 : cluster [DBG] osdmap e908: 12 total, 12 up, 12 in 2020-09-08T16:27:02.640 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:50 smithi104 bash[9203]: audit 2020-09-08T16:25:49.860073+0000 mon.a (mon.0) 2544 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.853655\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.853925\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.854774\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.855418\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.856044\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:25:49.856810\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:25:48.995696\"}"}]: dispatch 2020-09-08T16:27:02.641 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:50 smithi104 bash[9203]: audit 2020-09-08T16:25:49.866266+0000 mon.a (mon.0) 2545 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi166","val":"{\"daemons\": {\"mon.c\": {\"daemon_type\": \"mon\", \"daemon_id\": \"c\", \"hostname\": \"smithi166\", \"container_id\": \"b942d9cefb4e\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.853655\", \"created\": \"2020-09-08T15:28:51.082052\", \"started\": \"2020-09-08T15:28:51.183582\"}, \"osd.8\": {\"daemon_type\": \"osd\", \"daemon_id\": \"8\", \"hostname\": \"smithi166\", \"container_id\": \"5a1ad0439e4c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.853925\", \"created\": \"2020-09-08T15:33:15.799848\", \"started\": \"2020-09-08T15:33:20.177398\"}, \"osd.9\": {\"daemon_type\": \"osd\", \"daemon_id\": \"9\", \"hostname\": \"smithi166\", \"container_id\": \"4cd89ab94217\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.854774\", \"created\": \"2020-09-08T15:33:41.114066\", \"started\": \"2020-09-08T15:33:45.531145\"}, \"osd.10\": {\"daemon_type\": \"osd\", \"daemon_id\": \"10\", \"hostname\": \"smithi166\", \"container_id\": \"3a64d7d04b6b\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.855418\", \"created\": \"2020-09-08T15:34:07.251523\", \"started\": \"2020-09-08T15:34:10.896139\"}, \"osd.11\": {\"daemon_type\": \"osd\", \"daemon_id\": \"11\", \"hostname\": \"smithi166\", \"container_id\": \"a39b93af1a97\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.856044\", \"created\": \"2020-09-08T15:34:32.858122\", \"started\": \"2020-09-08T15:34:36.527807\"}}, \"devices\": [{\"rejected_reasons\": [\"Insufficient space (<5GB) on vgs\", \"locked\", \"LVM detected\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"11\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"121301fe-a41c-40c0-ad09-b00de37a55e1\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"2Sqapm-yxLH-IDJv-joP6-HQB0-VrDj-2EW6fU\"}, {\"name\": \"lv_2\", \"osd_id\": \"10\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"c194210e-bd89-4a1f-98fb-6bf44f0b324c\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"RI7weK-sN9X-d89V-dIKx-fyjG-p5PZ-ePVZWq\"}, {\"name\": \"lv_3\", \"osd_id\": \"9\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"Xbf5c2-6SjS-iuP2-h0DS-jgJh-porW-8m8YKb\"}, {\"name\": \"lv_4\", \"osd_id\": \"8\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"3b0ac380-878e-4c1c-9278-80ce67ca2df4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"taxJtf-CwDe-iL31-1Coc-pW7d-rt9s-P5uSNk\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620401AC400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"SCSI VPD page 0xb1 is not supported\", \"we only support disk path start with '/dev/sd' today\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5XMNS\", \"lsm_data\": {\"serialNum\": \"Z1W5XMNS\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.c\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:50.667169\"}, \"osd.8\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:13.573087\"}, \"osd.9\": {\"deps\": [], \"last_config\": \"2020-09-08T15:33:38.715801\"}, \"osd.10\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:04.900929\"}, \"osd.11\": {\"deps\": [], \"last_config\": \"2020-09-08T15:34:30.549656\"}}, \"last_daemon_update\": \"2020-09-08T16:25:49.856810\", \"last_device_update\": \"2020-09-08T16:04:58.620850\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.166\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe88:77ed\"]}, \"last_host_check\": \"2020-09-08T16:25:48.995696\"}"}]': finished 2020-09-08T16:27:02.641 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:50 smithi104 bash[9203]: audit 2020-09-08T16:25:49.969344+0000 mon.a (mon.0) 2546 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.963581\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.963784\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.963914\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.964661\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.965214\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.965750\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:25:49.966460\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:25:48.998320\"}"}]: dispatch 2020-09-08T16:27:02.641 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:50 smithi104 bash[9203]: audit 2020-09-08T16:25:49.975208+0000 mon.a (mon.0) 2547 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi055","val":"{\"daemons\": {\"mon.b\": {\"daemon_type\": \"mon\", \"daemon_id\": \"b\", \"hostname\": \"smithi055\", \"container_id\": \"c828543f0924\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.963581\", \"created\": \"2020-09-08T15:28:36.438141\", \"started\": \"2020-09-08T15:28:36.534253\"}, \"mgr.x\": {\"daemon_type\": \"mgr\", \"daemon_id\": \"x\", \"hostname\": \"smithi055\", \"container_id\": \"0aaa72e7d1ff\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.963784\", \"created\": \"2020-09-08T15:29:08.198142\", \"started\": \"2020-09-08T15:29:08.290545\"}, \"osd.4\": {\"daemon_type\": \"osd\", \"daemon_id\": \"4\", \"hostname\": \"smithi055\", \"container_id\": \"1a88b889c5e9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.963914\", \"created\": \"2020-09-08T15:31:34.022146\", \"started\": \"2020-09-08T15:31:37.880271\"}, \"osd.5\": {\"daemon_type\": \"osd\", \"daemon_id\": \"5\", \"hostname\": \"smithi055\", \"container_id\": \"b89e2ebb8cea\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.964661\", \"created\": \"2020-09-08T15:32:00.506147\", \"started\": \"2020-09-08T15:32:04.939245\"}, \"osd.6\": {\"daemon_type\": \"osd\", \"daemon_id\": \"6\", \"hostname\": \"smithi055\", \"container_id\": \"fd579e0fe4b8\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.965214\", \"created\": \"2020-09-08T15:32:26.730148\", \"started\": \"2020-09-08T15:32:30.569394\"}, \"osd.7\": {\"daemon_type\": \"osd\", \"daemon_id\": \"7\", \"hostname\": \"smithi055\", \"container_id\": \"6805ce6d821f\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:49.965750\", \"created\": \"2020-09-08T15:32:53.363152\", \"started\": \"2020-09-08T15:32:57.110549\"}}, \"devices\": [{\"rejected_reasons\": [\"LVM detected\", \"locked\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDKE040T7\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 4000787030016.0, \"human_readable_size\": \"3.64 TB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"7\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"f3795c86-4540-442c-a151-b2769fa577e4\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"CwGdPM-xaLv-7rFg-xO2n-7cUg-8Z64-ZEllzQ\"}, {\"name\": \"lv_2\", \"osd_id\": \"6\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ad9a93fe-0ca0-4052-bfa9-e2abb268829e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HIadEJ-EkZ1-QKH1-l9F2-eQND-IyOh-ePucEH\"}, {\"name\": \"lv_3\", \"osd_id\": \"5\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"6789abd2-9e9b-4c04-81fb-0ce2d2849e39\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"07DMrz-Aqlc-aA5T-Iz1c-CtxE-HAok-cdDcfq\"}, {\"name\": \"lv_4\", \"osd_id\": \"4\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"d8334c70-f776-4652-944d-aed35a8e43f7\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"gBrvtP-vQ6i-a6EL-QQLu-Ut3l-SJO2-BnsfZ8\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_BTLE736102VS4P0KGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"SCSI VPD page 0x83 is not supported\", \"Not a SCSI compatible device\", \"we only support disk path start with '/dev/sd' today\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN04\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W4H48L\", \"lsm_data\": {\"serialNum\": \"Z1W4H48L\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.b\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:41.446483\"}, \"mgr.x\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:05.749809\"}, \"osd.4\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:31.715366\"}, \"osd.5\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:58.175566\"}, \"osd.6\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:24.324194\"}, \"osd.7\": {\"deps\": [], \"last_config\": \"2020-09-08T15:32:51.001837\"}}, \"last_daemon_update\": \"2020-09-08T16:25:49.966460\", \"last_device_update\": \"2020-09-08T16:04:20.379587\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.55\"], \"172.21.15.254\": [\"172.21.15.55\"], \"fe80::/64\": [\"fe80::ae1f:6bff:fef8:2606\"]}, \"last_host_check\": \"2020-09-08T16:25:48.998320\"}"}]': finished 2020-09-08T16:27:02.642 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:50 smithi104 bash[9203]: audit 2020-09-08T16:25:50.198306+0000 mon.a (mon.0) 2548 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.192547\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.192731\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.193342\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.193892\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.194470\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:25:50.195154\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:25:48.990918\"}"}]: dispatch 2020-09-08T16:27:02.642 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:50 smithi104 bash[9203]: audit 2020-09-08T16:25:50.204507+0000 mon.a (mon.0) 2549 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/host.smithi104","val":"{\"daemons\": {\"mon.a\": {\"daemon_type\": \"mon\", \"daemon_id\": \"a\", \"hostname\": \"smithi104\", \"container_id\": \"e3192257088c\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.192547\", \"created\": \"2020-09-08T15:25:40.211648\", \"started\": \"2020-09-08T15:25:47.315025\"}, \"osd.0\": {\"daemon_type\": \"osd\", \"daemon_id\": \"0\", \"hostname\": \"smithi104\", \"container_id\": \"c29814151975\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.192731\", \"created\": \"2020-09-08T15:29:55.001341\", \"started\": \"2020-09-08T15:29:58.578440\"}, \"osd.1\": {\"daemon_type\": \"osd\", \"daemon_id\": \"1\", \"hostname\": \"smithi104\", \"container_id\": \"8d533aae7d47\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.193342\", \"created\": \"2020-09-08T15:30:20.977340\", \"started\": \"2020-09-08T15:30:25.004996\"}, \"osd.2\": {\"daemon_type\": \"osd\", \"daemon_id\": \"2\", \"hostname\": \"smithi104\", \"container_id\": \"a6c1083fc95a\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.193892\", \"created\": \"2020-09-08T15:30:46.237340\", \"started\": \"2020-09-08T15:30:49.898040\"}, \"osd.3\": {\"daemon_type\": \"osd\", \"daemon_id\": \"3\", \"hostname\": \"smithi104\", \"container_id\": \"a658f119e0a9\", \"container_image_id\": \"5274012435cfc089685d790115638d8472270611c3676c691595a6b8f3077a45\", \"container_image_name\": \"quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f\", \"version\": \"16.0.0-5272-gdce4d291bd4\", \"status\": 1, \"status_desc\": \"running\", \"osdspec_affinity\": \"None\", \"is_active\": false, \"last_refresh\": \"2020-09-08T16:25:50.194470\", \"created\": \"2020-09-08T15:31:11.717339\", \"started\": \"2020-09-08T15:31:14.811915\"}}, \"devices\": [{\"rejected_reasons\": [\"locked\", \"LVM detected\", \"Insufficient space (<5GB) on vgs\"], \"available\": false, \"path\": \"/dev/nvme0n1\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"\", \"model\": \"INTEL SSDPEDMD400G4\", \"rev\": \"\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"512\", \"rotational\": \"0\", \"nr_requests\": \"1023\", \"scheduler_mode\": \"none\", \"partitions\": {}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 400088457216.0, \"human_readable_size\": \"372.61 GB\", \"path\": \"/dev/nvme0n1\", \"locked\": 1}, \"lvs\": [{\"name\": \"lv_1\", \"osd_id\": \"3\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"5CmrCS-kW01-Zh88-Ulr0-Re2O-q0Md-M8SfJZ\"}, {\"name\": \"lv_2\", \"osd_id\": \"2\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"ca6da1cc-1d8d-494b-93e6-e065ff70d434\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"e0cb8x-F3l4-D3GR-xRR0-5Q9K-HIyX-Y1F8NZ\"}, {\"name\": \"lv_3\", \"osd_id\": \"1\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"a83df412-110d-440a-b411-ad2a7c36f81a\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"7iqcXf-3JJB-n16t-GVle-Mmk1-uDsV-gf5x5s\"}, {\"name\": \"lv_4\", \"osd_id\": \"0\", \"cluster_name\": \"ceph\", \"type\": \"block\", \"osd_fsid\": \"e49a7341-7c75-4883-b061-3e8fe9467e6e\", \"cluster_fsid\": \"759b27b6-f1e7-11ea-a080-001a4aab830c\", \"osdspec_affinity\": \"None\", \"block_uuid\": \"HUfFuW-ILL9-zWBM-eg1S-fyez-lUXk-jhlKyP\"}, {\"name\": \"lv_5\", \"comment\": \"not used by ceph\"}], \"human_readable_type\": \"ssd\", \"device_id\": \"_PHFT620400NW400BGN\", \"lsm_data\": {\"serialNum\": \"Unknown\", \"transport\": \"Unknown\", \"mediaType\": \"Unknown\", \"rpm\": \"Unknown\", \"linkSpeed\": \"Unknown\", \"health\": \"Unknown\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"we only support disk path start with '/dev/sd' today\", \"Not a SCSI compatible device\", \"SCSI VPD page 0x83 is not supported\", \"SCSI VPD page 0xb1 is not supported\"]}}, {\"rejected_reasons\": [\"locked\"], \"available\": false, \"path\": \"/dev/sda\", \"sys_api\": {\"removable\": \"0\", \"ro\": \"0\", \"vendor\": \"ATA\", \"model\": \"ST1000NM0033-9ZM\", \"rev\": \"SN06\", \"sas_address\": \"\", \"sas_device_handle\": \"\", \"support_discard\": \"0\", \"rotational\": \"1\", \"nr_requests\": \"128\", \"scheduler_mode\": \"cfq\", \"partitions\": {\"sda1\": {\"start\": \"2048\", \"sectors\": \"1953522688\", \"sectorsize\": 512, \"size\": 1000203616256.0, \"human_readable_size\": \"931.51 GB\", \"holders\": []}}, \"sectors\": 0, \"sectorsize\": \"512\", \"size\": 1000204886016.0, \"human_readable_size\": \"931.51 GB\", \"path\": \"/dev/sda\", \"locked\": 1}, \"lvs\": [], \"human_readable_type\": \"hdd\", \"device_id\": \"ST1000NM0033-9ZM173_Z1W5QGJ7\", \"lsm_data\": {\"serialNum\": \"Z1W5QGJ7\", \"transport\": \"ATA/SATA\", \"mediaType\": \"HDD\", \"rpm\": 7200, \"linkSpeed\": 6000, \"health\": \"Good\", \"ledSupport\": {\"IDENTsupport\": \"Unknown\", \"IDENTstatus\": \"Unsupported\", \"FAILsupport\": \"Unknown\", \"FAILstatus\": \"Unsupported\"}, \"errors\": [\"Given disk does not expose SCSI target port SAS address via SCSI Device Identification VPD page\"]}}], \"osdspec_previews\": [], \"daemon_config_deps\": {\"mon.a\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:43.604815\"}, \"mgr.y\": {\"deps\": [], \"last_config\": \"2020-09-08T15:28:45.060674\"}, \"osd.0\": {\"deps\": [], \"last_config\": \"2020-09-08T15:29:52.690017\"}, \"osd.1\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:18.550957\"}, \"osd.2\": {\"deps\": [], \"last_config\": \"2020-09-08T15:30:43.957255\"}, \"osd.3\": {\"deps\": [], \"last_config\": \"2020-09-08T15:31:09.592402\"}}, \"last_daemon_update\": \"2020-09-08T16:25:50.195154\", \"last_device_update\": \"2020-09-08T16:04:28.420381\", \"networks\": {\"172.17.0.0/16\": [\"172.17.0.1\"], \"172.21.0.0/20\": [\"172.21.15.104\"], \"fe80::/64\": [\"fe80::ec4:7aff:fe8f:ce89\"]}, \"last_host_check\": \"2020-09-08T16:25:48.990918\"}"}]': finished 2020-09-08T16:27:02.642 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:50 smithi104 bash[9203]: audit 2020-09-08T16:25:50.206948+0000 mon.a (mon.0) 2550 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.642 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:50 smithi104 bash[9203]: audit 2020-09-08T16:25:50.212351+0000 mon.a (mon.0) 2551 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.643 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:51 smithi104 bash[9203]: cluster 2020-09-08T16:25:50.431687+0000 mgr.x (mgr.24118) 1755 : cluster [DBG] pgmap v2579: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.643 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:53 smithi104 bash[9203]: cluster 2020-09-08T16:25:52.432399+0000 mgr.x (mgr.24118) 1756 : cluster [DBG] pgmap v2580: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.643 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:55 smithi104 bash[9203]: cluster 2020-09-08T16:25:54.433250+0000 mgr.x (mgr.24118) 1757 : cluster [DBG] pgmap v2581: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.643 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:55 smithi104 bash[9203]: cluster 2020-09-08T16:25:54.862483+0000 osd.1 (osd.1) 184 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.644 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:55 smithi104 bash[9203]: cluster 2020-09-08T16:25:54.862616+0000 osd.1 (osd.1) 185 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.644 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:56 smithi104 bash[9203]: cluster 2020-09-08T16:25:55.638400+0000 mon.a (mon.0) 2552 : cluster [DBG] osdmap e909: 12 total, 12 up, 12 in 2020-09-08T16:27:02.644 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:57 smithi104 bash[9203]: cluster 2020-09-08T16:25:56.433899+0000 mgr.x (mgr.24118) 1758 : cluster [DBG] pgmap v2583: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.644 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:25:59 smithi104 bash[9203]: cluster 2020-09-08T16:25:58.434653+0000 mgr.x (mgr.24118) 1759 : cluster [DBG] pgmap v2584: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.644 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:01 smithi104 bash[9203]: cluster 2020-09-08T16:26:00.224874+0000 osd.6 (osd.6) 149 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.645 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:01 smithi104 bash[9203]: cluster 2020-09-08T16:26:00.225015+0000 osd.6 (osd.6) 150 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.645 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:01 smithi104 bash[9203]: cluster 2020-09-08T16:26:00.435276+0000 mgr.x (mgr.24118) 1760 : cluster [DBG] pgmap v2585: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.645 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:01 smithi104 bash[9203]: cluster 2020-09-08T16:26:00.670003+0000 mon.a (mon.0) 2553 : cluster [DBG] osdmap e910: 12 total, 12 up, 12 in 2020-09-08T16:27:02.645 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:03 smithi104 bash[9203]: cluster 2020-09-08T16:26:02.435956+0000 mgr.x (mgr.24118) 1761 : cluster [DBG] pgmap v2587: 17 pgs: 17 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.645 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:05 smithi104 bash[9203]: cluster 2020-09-08T16:26:04.436653+0000 mgr.x (mgr.24118) 1762 : cluster [DBG] pgmap v2588: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.646 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:06 smithi104 bash[9203]: cluster 2020-09-08T16:26:05.395848+0000 osd.11 (osd.11) 164 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.646 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:06 smithi104 bash[9203]: cluster 2020-09-08T16:26:05.395976+0000 osd.11 (osd.11) 165 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.646 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:06 smithi104 bash[9203]: cluster 2020-09-08T16:26:06.345739+0000 mon.a (mon.0) 2554 : cluster [DBG] osdmap e911: 12 total, 12 up, 12 in 2020-09-08T16:27:02.646 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:07 smithi104 bash[9203]: cluster 2020-09-08T16:26:06.437157+0000 mgr.x (mgr.24118) 1763 : cluster [DBG] pgmap v2590: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.646 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:08 smithi104 bash[9203]: cluster 2020-09-08T16:26:08.234288+0000 osd.4 (osd.4) 109 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.647 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:08 smithi104 bash[9203]: cluster 2020-09-08T16:26:08.234451+0000 osd.4 (osd.4) 110 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.647 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:09 smithi104 bash[9203]: cluster 2020-09-08T16:26:08.711456+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e912: 12 total, 12 up, 12 in 2020-09-08T16:27:02.647 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:09 smithi104 bash[9203]: audit 2020-09-08T16:26:08.714372+0000 mon.a (mon.0) 2556 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:02.647 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:09 smithi104 bash[9203]: audit 2020-09-08T16:26:08.719180+0000 mon.a (mon.0) 2557 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:02.647 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:11 smithi104 bash[9203]: cluster 2020-09-08T16:26:10.438346+0000 mgr.x (mgr.24118) 1765 : cluster [DBG] pgmap v2593: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.648 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:13 smithi104 bash[9203]: cluster 2020-09-08T16:26:12.439023+0000 mgr.x (mgr.24118) 1766 : cluster [DBG] pgmap v2594: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.648 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:15 smithi104 bash[9203]: cluster 2020-09-08T16:26:14.439803+0000 mgr.x (mgr.24118) 1767 : cluster [DBG] pgmap v2595: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.648 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:17 smithi104 bash[9203]: cluster 2020-09-08T16:26:16.440416+0000 mgr.x (mgr.24118) 1768 : cluster [DBG] pgmap v2596: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.648 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:19 smithi104 bash[9203]: cluster 2020-09-08T16:26:18.441118+0000 mgr.x (mgr.24118) 1769 : cluster [DBG] pgmap v2597: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.648 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:19 smithi104 bash[9203]: cluster 2020-09-08T16:26:19.009384+0000 osd.9 (osd.9) 148 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.649 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:19 smithi104 bash[9203]: cluster 2020-09-08T16:26:19.009503+0000 osd.9 (osd.9) 149 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.649 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:20 smithi104 bash[9203]: cluster 2020-09-08T16:26:19.521611+0000 mon.a (mon.0) 2558 : cluster [DBG] osdmap e913: 12 total, 12 up, 12 in 2020-09-08T16:27:02.649 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:20 smithi104 bash[9203]: cluster 2020-09-08T16:26:20.098181+0000 osd.2 (osd.2) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.649 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:20 smithi104 bash[9203]: cluster 2020-09-08T16:26:20.098304+0000 osd.2 (osd.2) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.649 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:21 smithi104 bash[9203]: cluster 2020-09-08T16:26:20.441781+0000 mgr.x (mgr.24118) 1770 : cluster [DBG] pgmap v2599: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.650 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:21 smithi104 bash[9203]: cluster 2020-09-08T16:26:20.545496+0000 mon.a (mon.0) 2559 : cluster [DBG] osdmap e914: 12 total, 12 up, 12 in 2020-09-08T16:27:02.650 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:23 smithi104 bash[9203]: cluster 2020-09-08T16:26:22.442522+0000 mgr.x (mgr.24118) 1771 : cluster [DBG] pgmap v2601: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.650 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:25 smithi104 bash[9203]: cluster 2020-09-08T16:26:24.443182+0000 mgr.x (mgr.24118) 1772 : cluster [DBG] pgmap v2602: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.650 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:27 smithi104 bash[9203]: cluster 2020-09-08T16:26:26.443879+0000 mgr.x (mgr.24118) 1773 : cluster [DBG] pgmap v2603: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.650 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:28 smithi104 bash[9203]: cluster 2020-09-08T16:26:28.098777+0000 osd.0 (osd.0) 160 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.651 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:28 smithi104 bash[9203]: cluster 2020-09-08T16:26:28.098921+0000 osd.0 (osd.0) 161 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.651 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:29 smithi104 bash[9203]: cluster 2020-09-08T16:26:28.444593+0000 mgr.x (mgr.24118) 1774 : cluster [DBG] pgmap v2604: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.651 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:29 smithi104 bash[9203]: cluster 2020-09-08T16:26:28.579344+0000 mon.a (mon.0) 2560 : cluster [DBG] osdmap e915: 12 total, 12 up, 12 in 2020-09-08T16:27:02.651 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:30 smithi104 bash[9203]: cluster 2020-09-08T16:26:28.796301+0000 osd.7 (osd.7) 200 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.651 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:30 smithi104 bash[9203]: cluster 2020-09-08T16:26:28.796427+0000 osd.7 (osd.7) 201 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.652 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:30 smithi104 bash[9203]: cluster 2020-09-08T16:26:29.592732+0000 mon.a (mon.0) 2561 : cluster [DBG] osdmap e916: 12 total, 12 up, 12 in 2020-09-08T16:27:02.652 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:31 smithi104 bash[9203]: cluster 2020-09-08T16:26:30.445361+0000 mgr.x (mgr.24118) 1775 : cluster [DBG] pgmap v2607: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.652 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:33 smithi104 bash[9203]: cluster 2020-09-08T16:26:32.446087+0000 mgr.x (mgr.24118) 1776 : cluster [DBG] pgmap v2608: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.652 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:35 smithi104 bash[9203]: cluster 2020-09-08T16:26:34.446753+0000 mgr.x (mgr.24118) 1777 : cluster [DBG] pgmap v2609: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.652 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:37 smithi104 bash[9203]: cluster 2020-09-08T16:26:36.447416+0000 mgr.x (mgr.24118) 1778 : cluster [DBG] pgmap v2610: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.653 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:38 smithi104 bash[9203]: cluster 2020-09-08T16:26:37.639096+0000 osd.5 (osd.5) 166 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.653 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:38 smithi104 bash[9203]: cluster 2020-09-08T16:26:37.639237+0000 osd.5 (osd.5) 167 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.653 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:39 smithi104 bash[9203]: cluster 2020-09-08T16:26:38.448127+0000 mgr.x (mgr.24118) 1779 : cluster [DBG] pgmap v2611: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.653 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:39 smithi104 bash[9203]: cluster 2020-09-08T16:26:38.631152+0000 mon.a (mon.0) 2562 : cluster [DBG] osdmap e917: 12 total, 12 up, 12 in 2020-09-08T16:27:02.653 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:40 smithi104 bash[9203]: cluster 2020-09-08T16:26:39.651090+0000 osd.3 (osd.3) 169 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.654 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:40 smithi104 bash[9203]: cluster 2020-09-08T16:26:39.651219+0000 osd.3 (osd.3) 170 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.654 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:41 smithi104 bash[9203]: cluster 2020-09-08T16:26:40.448778+0000 mgr.x (mgr.24118) 1780 : cluster [DBG] pgmap v2613: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.654 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:41 smithi104 bash[9203]: cluster 2020-09-08T16:26:40.534204+0000 osd.10 (osd.10) 145 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.654 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:41 smithi104 bash[9203]: cluster 2020-09-08T16:26:40.534322+0000 osd.10 (osd.10) 146 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.654 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:41 smithi104 bash[9203]: cluster 2020-09-08T16:26:40.639992+0000 mon.a (mon.0) 2563 : cluster [DBG] osdmap e918: 12 total, 12 up, 12 in 2020-09-08T16:27:02.655 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:43 smithi104 bash[9203]: cluster 2020-09-08T16:26:42.449459+0000 mgr.x (mgr.24118) 1781 : cluster [DBG] pgmap v2615: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.655 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:45 smithi104 bash[9203]: cluster 2020-09-08T16:26:44.450417+0000 mgr.x (mgr.24118) 1782 : cluster [DBG] pgmap v2616: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.655 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:47 smithi104 bash[9203]: cluster 2020-09-08T16:26:46.451131+0000 mgr.x (mgr.24118) 1783 : cluster [DBG] pgmap v2617: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.655 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:49 smithi104 bash[9203]: cluster 2020-09-08T16:26:48.451858+0000 mgr.x (mgr.24118) 1784 : cluster [DBG] pgmap v2618: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.655 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:49 smithi104 bash[9203]: cluster 2020-09-08T16:26:49.534260+0000 osd.8 (osd.8) 140 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.656 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:49 smithi104 bash[9203]: cluster 2020-09-08T16:26:49.534382+0000 osd.8 (osd.8) 141 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.656 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:50 smithi104 bash[9203]: cluster 2020-09-08T16:26:49.681079+0000 mon.a (mon.0) 2564 : cluster [DBG] osdmap e919: 12 total, 12 up, 12 in 2020-09-08T16:27:02.656 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:51 smithi104 bash[9203]: cluster 2020-09-08T16:26:50.452524+0000 mgr.x (mgr.24118) 1785 : cluster [DBG] pgmap v2620: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.656 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:53 smithi104 bash[9203]: cluster 2020-09-08T16:26:52.453264+0000 mgr.x (mgr.24118) 1786 : cluster [DBG] pgmap v2621: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.656 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:55 smithi104 bash[9203]: cluster 2020-09-08T16:26:54.454065+0000 mgr.x (mgr.24118) 1787 : cluster [DBG] pgmap v2622: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.657 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:56 smithi104 bash[9203]: cluster 2020-09-08T16:26:55.777497+0000 osd.1 (osd.1) 186 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.657 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:56 smithi104 bash[9203]: cluster 2020-09-08T16:26:55.777629+0000 osd.1 (osd.1) 187 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.657 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:57 smithi104 bash[9203]: cluster 2020-09-08T16:26:56.454786+0000 mgr.x (mgr.24118) 1788 : cluster [DBG] pgmap v2623: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.657 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:57 smithi104 bash[9203]: cluster 2020-09-08T16:26:56.729079+0000 mon.a (mon.0) 2565 : cluster [DBG] osdmap e920: 12 total, 12 up, 12 in 2020-09-08T16:27:02.658 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:26:59 smithi104 bash[9203]: cluster 2020-09-08T16:26:58.455629+0000 mgr.x (mgr.24118) 1789 : cluster [DBG] pgmap v2625: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.658 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:01 smithi104 bash[9203]: cluster 2020-09-08T16:27:00.443440+0000 osd.6 (osd.6) 151 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:02.658 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:01 smithi104 bash[9203]: cluster 2020-09-08T16:27:00.443571+0000 osd.6 (osd.6) 152 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:02.658 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:01 smithi104 bash[9203]: cluster 2020-09-08T16:27:00.456380+0000 mgr.x (mgr.24118) 1790 : cluster [DBG] pgmap v2626: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:02.659 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:13:28 smithi104 bash[19922]: debug 2020-09-08T16:13:28.235+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:02.659 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:27:02.660 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:27:02.660 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:27:02.660 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:27:02.660 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:27:02.660 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:27:02.661 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:27:02.661 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:27:02.661 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:27:02.661 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:27:02.661 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:27:02.662 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:27:02.662 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-09-08T16:27:02.662 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-09-08T16:27:02.662 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-09-08T16:27:02.663 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-09-08T16:27:02.663 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-09-08T16:27:02.663 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-09-08T16:27:02.663 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-09-08T16:27:02.663 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-09-08T16:27:02.663 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'false' 2020-09-08T16:27:02.664 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'false' 2020-09-08T16:27:02.664 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'false' 2020-09-08T16:27:02.664 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'false' 2020-09-08T16:27:02.665 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T16:27:02.665 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:27:02.666 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2020-09-08T16:27:02.667 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_7 2020-09-08T16:27:02.668 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd pool rm unique_pool_7 unique_pool_7 --yes-i-really-really-mean-it 2020-09-08T16:27:02.670 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:27:02.671 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:02 smithi055 bash[9296]: cluster 2020-09-08T16:27:01.378258+0000 mon.a (mon.0) 2566 : cluster [DBG] osdmap e921: 12 total, 12 up, 12 in 2020-09-08T16:27:02.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:02 smithi104 bash[9203]: cluster 2020-09-08T16:27:01.378258+0000 mon.a (mon.0) 2566 : cluster [DBG] osdmap e921: 12 total, 12 up, 12 in 2020-09-08T16:27:02.683 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:02 smithi166 bash[9446]: cluster 2020-09-08T16:27:01.378258+0000 mon.a (mon.0) 2566 : cluster [DBG] osdmap e921: 12 total, 12 up, 12 in 2020-09-08T16:27:02.807 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:27:02.808 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:27:02.848 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:27:02.950 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:27:03.089 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:27:03.116 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:27:02 smithi166 bash[14254]: debug 2020-09-08T16:27:02.840+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:03.196 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:27:03.326 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:27:03 smithi055 bash[16646]: debug 2020-09-08T16:27:03.089+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:03.327 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:27:03.441 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:27:03.565 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:27:03.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:03 smithi055 bash[9296]: cluster 2020-09-08T16:27:02.457096+0000 mgr.x (mgr.24118) 1791 : cluster [DBG] pgmap v2628: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:03.658 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:27:03 smithi055 bash[14301]: debug 2020-09-08T16:27:03.329+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:03.659 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:27:03 smithi055 bash[16646]: debug 2020-09-08T16:27:03.565+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:03.682 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:27:03.683 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:03 smithi104 bash[9203]: cluster 2020-09-08T16:27:02.457096+0000 mgr.x (mgr.24118) 1791 : cluster [DBG] pgmap v2628: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:03.696 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:03 smithi166 bash[9446]: cluster 2020-09-08T16:27:02.457096+0000 mgr.x (mgr.24118) 1791 : cluster [DBG] pgmap v2628: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:03.812 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:27:03.918 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:27:04.057 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:27:04.117 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:27:03 smithi166 bash[16593]: debug 2020-09-08T16:27:03.812+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:04.158 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:27:04.384 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:27:04.407 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:27:04 smithi055 bash[14301]: debug 2020-09-08T16:27:04.049+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:04.425 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:27:04 smithi104 bash[19922]: debug 2020-09-08T16:27:04.386+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:04.492 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:27:04.617 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:27:04.661 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T16:27:04.667 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-09-08T16:27:04.682 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:27:04 smithi104 bash[15273]: debug 2020-09-08T16:27:04.618+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:04.720 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:27:04.806 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:27:04.807 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:27:04.841 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:27:04.947 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:27:05.108 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:27:05.157 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:27:04 smithi055 bash[14301]: debug 2020-09-08T16:27:04.841+0000 7f9d16518700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:05.214 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:27:05.366 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:27:05 smithi166 bash[16593]: debug 2020-09-08T16:27:05.108+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:05.608 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:27:05.713 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:27:05.833 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:05 smithi055 bash[9296]: cluster 2020-09-08T16:27:04.457877+0000 mgr.x (mgr.24118) 1792 : cluster [DBG] pgmap v2629: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:05.834 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:27:05.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:05 smithi166 bash[9446]: cluster 2020-09-08T16:27:04.457877+0000 mgr.x (mgr.24118) 1792 : cluster [DBG] pgmap v2629: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:05.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:05 smithi104 bash[9203]: cluster 2020-09-08T16:27:04.457877+0000 mgr.x (mgr.24118) 1792 : cluster [DBG] pgmap v2629: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:05.932 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:27:05 smithi104 bash[19922]: debug 2020-09-08T16:27:05.610+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:05.942 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:27:06.074 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:27:06.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:27:05 smithi055 bash[11956]: debug 2020-09-08T16:27:05.837+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:06.157 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:27:06 smithi055 bash[11956]: debug 2020-09-08T16:27:06.077+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:06.179 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:27:06.489 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:27:06.598 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:27:06.733 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:27:06.839 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:27:06.851 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:27:06 smithi104 bash[15273]: debug 2020-09-08T16:27:06.490+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:06.907 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:27:06 smithi055 bash[19003]: debug 2020-09-08T16:27:06.737+0000 7f9baf954700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:07.248 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:27:07.352 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:27:07.526 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:27:07 smithi104 bash[19922]: debug 2020-09-08T16:27:07.250+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:07.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:07 smithi166 bash[9446]: cluster 2020-09-08T16:27:06.378716+0000 osd.11 (osd.11) 166 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:07.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:07 smithi166 bash[9446]: cluster 2020-09-08T16:27:06.378846+0000 osd.11 (osd.11) 167 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:07.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:07 smithi166 bash[9446]: cluster 2020-09-08T16:27:06.458688+0000 mgr.x (mgr.24118) 1793 : cluster [DBG] pgmap v2630: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:07.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:07 smithi166 bash[9446]: cluster 2020-09-08T16:27:06.535121+0000 mon.a (mon.0) 2567 : cluster [DBG] osdmap e922: 12 total, 12 up, 12 in 2020-09-08T16:27:07.768 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:27:07.878 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:27:07.892 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:07 smithi104 bash[9203]: cluster 2020-09-08T16:27:06.378716+0000 osd.11 (osd.11) 166 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:07.892 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:07 smithi104 bash[9203]: cluster 2020-09-08T16:27:06.378846+0000 osd.11 (osd.11) 167 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:07.892 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:07 smithi104 bash[9203]: cluster 2020-09-08T16:27:06.458688+0000 mgr.x (mgr.24118) 1793 : cluster [DBG] pgmap v2630: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:07.893 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:07 smithi104 bash[9203]: cluster 2020-09-08T16:27:06.535121+0000 mon.a (mon.0) 2567 : cluster [DBG] osdmap e922: 12 total, 12 up, 12 in 2020-09-08T16:27:07.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:07 smithi055 bash[9296]: cluster 2020-09-08T16:27:06.378716+0000 osd.11 (osd.11) 166 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:07.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:07 smithi055 bash[9296]: cluster 2020-09-08T16:27:06.378846+0000 osd.11 (osd.11) 167 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:07.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:07 smithi055 bash[9296]: cluster 2020-09-08T16:27:06.458688+0000 mgr.x (mgr.24118) 1793 : cluster [DBG] pgmap v2630: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:07.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:07 smithi055 bash[9296]: cluster 2020-09-08T16:27:06.535121+0000 mon.a (mon.0) 2567 : cluster [DBG] osdmap e922: 12 total, 12 up, 12 in 2020-09-08T16:27:08.111 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:27:08.118 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:27:07 smithi166 bash[14254]: debug 2020-09-08T16:27:07.768+0000 7fc425970700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:08.175 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:27:08 smithi104 bash[19922]: debug 2020-09-08T16:27:08.114+0000 7fe387e29700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:08.219 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:27:08.490 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:27:08.597 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:27:08.729 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:27:08.741 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:27:08 smithi104 bash[17588]: debug 2020-09-08T16:27:08.494+0000 7fc3a2794700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:08.842 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:27:08.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:27:08 smithi166 bash[11928]: debug 2020-09-08T16:27:08.732+0000 7ff3eca4f700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:08.891 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2020-09-08T16:27:08.892 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-09-08T16:27:09.460 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:27:09.565 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:27:09.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:09 smithi166 bash[9446]: cluster 2020-09-08T16:27:08.459423+0000 mgr.x (mgr.24118) 1794 : cluster [DBG] pgmap v2632: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:09.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:09 smithi055 bash[9296]: cluster 2020-09-08T16:27:08.459423+0000 mgr.x (mgr.24118) 1794 : cluster [DBG] pgmap v2632: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:09.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:09 smithi104 bash[9203]: cluster 2020-09-08T16:27:08.459423+0000 mgr.x (mgr.24118) 1794 : cluster [DBG] pgmap v2632: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:09.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:27:09 smithi104 bash[22305]: debug 2020-09-08T16:27:09.462+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:10.343 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:27:10 smithi104 bash[15273]: debug 2020-09-08T16:27:10.346+0000 7f5a7fe58700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:10.346 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:27:10.454 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:27:10.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:10 smithi104 bash[9203]: cluster 2020-09-08T16:27:09.189402+0000 osd.4 (osd.4) 111 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:10.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:10 smithi104 bash[9203]: cluster 2020-09-08T16:27:09.189526+0000 osd.4 (osd.4) 112 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:10.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:10 smithi104 bash[9203]: cluster 2020-09-08T16:27:09.554851+0000 mon.a (mon.0) 2568 : cluster [DBG] osdmap e923: 12 total, 12 up, 12 in 2020-09-08T16:27:10.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:10 smithi104 bash[9203]: audit 2020-09-08T16:27:10.538784+0000 mon.a (mon.0) 2569 : audit [INF] from='client.? v1:172.21.15.104:0/2062677224' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_7", "pool2": "unique_pool_7", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T16:27:10.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:10 smithi166 bash[9446]: cluster 2020-09-08T16:27:09.189402+0000 osd.4 (osd.4) 111 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:10.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:10 smithi166 bash[9446]: cluster 2020-09-08T16:27:09.189526+0000 osd.4 (osd.4) 112 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:10.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:10 smithi166 bash[9446]: cluster 2020-09-08T16:27:09.554851+0000 mon.a (mon.0) 2568 : cluster [DBG] osdmap e923: 12 total, 12 up, 12 in 2020-09-08T16:27:10.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:10 smithi166 bash[9446]: audit 2020-09-08T16:27:10.538784+0000 mon.a (mon.0) 2569 : audit [INF] from='client.? v1:172.21.15.104:0/2062677224' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_7", "pool2": "unique_pool_7", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T16:27:10.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:10 smithi055 bash[9296]: cluster 2020-09-08T16:27:09.189402+0000 osd.4 (osd.4) 111 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:10.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:10 smithi055 bash[9296]: cluster 2020-09-08T16:27:09.189526+0000 osd.4 (osd.4) 112 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:10.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:10 smithi055 bash[9296]: cluster 2020-09-08T16:27:09.554851+0000 mon.a (mon.0) 2568 : cluster [DBG] osdmap e923: 12 total, 12 up, 12 in 2020-09-08T16:27:10.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:10 smithi055 bash[9296]: audit 2020-09-08T16:27:10.538784+0000 mon.a (mon.0) 2569 : audit [INF] from='client.? v1:172.21.15.104:0/2062677224' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_7", "pool2": "unique_pool_7", "yes_i_really_really_mean_it": true}]: dispatch 2020-09-08T16:27:10.958 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:27:11.063 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:27:11.207 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:27:11.313 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:27:11.367 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:27:11 smithi166 bash[18960]: debug 2020-09-08T16:27:11.208+0000 7f9c18408700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:11.371 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'unique_pool_7' removed 2020-09-08T16:27:11.426 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:27:10 smithi104 bash[22305]: debug 2020-09-08T16:27:10.958+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:11.441 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:27:11.545 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:27:11.601 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2020-09-08T16:27:11.615 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:27:11 smithi055 bash[11956]: debug 2020-09-08T16:27:11.445+0000 7f4e1b2fa700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:11.616 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:11 smithi055 bash[9296]: cluster 2020-09-08T16:27:10.460114+0000 mgr.x (mgr.24118) 1795 : cluster [DBG] pgmap v2634: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:11.617 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:11 smithi055 bash[9296]: audit 2020-09-08T16:27:11.372082+0000 mon.a (mon.0) 2570 : audit [INF] from='client.? v1:172.21.15.104:0/2062677224' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_7", "pool2": "unique_pool_7", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T16:27:11.617 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:11 smithi055 bash[9296]: cluster 2020-09-08T16:27:11.372208+0000 mon.a (mon.0) 2571 : cluster [DBG] osdmap e924: 12 total, 12 up, 12 in 2020-09-08T16:27:11.693 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:27:11.732 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:27:11.794 INFO:teuthology.orchestra.run.smithi055:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:27:11.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:11 smithi166 bash[9446]: cluster 2020-09-08T16:27:10.460114+0000 mgr.x (mgr.24118) 1795 : cluster [DBG] pgmap v2634: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:11.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:11 smithi166 bash[9446]: audit 2020-09-08T16:27:11.372082+0000 mon.a (mon.0) 2570 : audit [INF] from='client.? v1:172.21.15.104:0/2062677224' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_7", "pool2": "unique_pool_7", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T16:27:11.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:11 smithi166 bash[9446]: cluster 2020-09-08T16:27:11.372208+0000 mon.a (mon.0) 2571 : cluster [DBG] osdmap e924: 12 total, 12 up, 12 in 2020-09-08T16:27:11.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:27:11 smithi055 bash[16646]: debug 2020-09-08T16:27:11.685+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:11.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:11 smithi104 bash[9203]: cluster 2020-09-08T16:27:10.460114+0000 mgr.x (mgr.24118) 1795 : cluster [DBG] pgmap v2634: 17 pgs: 17 active+clean; 0 B data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:11.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:11 smithi104 bash[9203]: audit 2020-09-08T16:27:11.372082+0000 mon.a (mon.0) 2570 : audit [INF] from='client.? v1:172.21.15.104:0/2062677224' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_7", "pool2": "unique_pool_7", "yes_i_really_really_mean_it": true}]': finished 2020-09-08T16:27:11.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:11 smithi104 bash[9203]: cluster 2020-09-08T16:27:11.372208+0000 mon.a (mon.0) 2571 : cluster [DBG] osdmap e924: 12 total, 12 up, 12 in 2020-09-08T16:27:12.429 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:27:12.533 INFO:teuthology.orchestra.run.smithi166:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:27:12.683 INFO:teuthology.orchestra.run.smithi166.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:27:12.789 INFO:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:27:12.815 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2020-09-08T16:27:12.824 INFO:tasks.thrashosds:joining thrashosds 2020-09-08T16:27:12.825 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2020-09-08T16:27:12.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:27:12 smithi166 bash[16593]: debug 2020-09-08T16:27:12.683+0000 7f8976279700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:12.907 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:27:12 smithi055 bash[16646]: debug 2020-09-08T16:27:12.433+0000 7f5a9593e700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:12.967 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:27:13.069 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2020-09-08T16:27:13.105 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:27:12 smithi104 bash[22305]: debug 2020-09-08T16:27:12.966+0000 7fb4a1e38700 -1 received signal: Hangup from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:27:13.253 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2020-09-08T16:27:13.293 INFO:teuthology.orchestra.run.smithi104.stdout:osd.1: {} 2020-09-08T16:27:13.321 INFO:teuthology.orchestra.run.smithi104.stdout:osd.2: {} 2020-09-08T16:27:13.341 INFO:teuthology.orchestra.run.smithi104.stdout:osd.3: {} 2020-09-08T16:27:13.367 INFO:teuthology.orchestra.run.smithi104.stdout:osd.4: {} 2020-09-08T16:27:13.387 INFO:teuthology.orchestra.run.smithi104.stdout:osd.5: {} 2020-09-08T16:27:13.416 INFO:teuthology.orchestra.run.smithi104.stdout:osd.6: {} 2020-09-08T16:27:13.443 INFO:teuthology.orchestra.run.smithi104.stdout:osd.7: {} 2020-09-08T16:27:13.470 INFO:teuthology.orchestra.run.smithi104.stdout:osd.8: {} 2020-09-08T16:27:13.496 INFO:teuthology.orchestra.run.smithi104.stdout:osd.9: {} 2020-09-08T16:27:13.525 INFO:teuthology.orchestra.run.smithi104.stdout:osd.10: {} 2020-09-08T16:27:13.550 INFO:teuthology.orchestra.run.smithi104.stdout:osd.11: {} 2020-09-08T16:27:13.554 INFO:teuthology.orchestra.run.smithi104.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-09-08T16:27:13.555 INFO:teuthology.orchestra.run.smithi104.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-09-08T16:27:13.555 INFO:teuthology.orchestra.run.smithi104.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-09-08T16:27:13.555 INFO:teuthology.orchestra.run.smithi104.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-09-08T16:27:13.555 INFO:teuthology.orchestra.run.smithi104.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-09-08T16:27:13.556 INFO:teuthology.orchestra.run.smithi104.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-09-08T16:27:13.556 INFO:teuthology.orchestra.run.smithi104.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-09-08T16:27:13.556 INFO:teuthology.orchestra.run.smithi104.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-09-08T16:27:13.556 INFO:teuthology.orchestra.run.smithi104.stderr:osd.8: osd_enable_op_tracker = 'true' 2020-09-08T16:27:13.556 INFO:teuthology.orchestra.run.smithi104.stderr:osd.9: osd_enable_op_tracker = 'true' 2020-09-08T16:27:13.557 INFO:teuthology.orchestra.run.smithi104.stderr:osd.10: osd_enable_op_tracker = 'true' 2020-09-08T16:27:13.557 INFO:teuthology.orchestra.run.smithi104.stderr:osd.11: osd_enable_op_tracker = 'true' 2020-09-08T16:27:13.569 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T16:27:13.596 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:13 smithi055 bash[9296]: cluster 2020-09-08T16:27:12.460810+0000 mgr.x (mgr.24118) 1796 : cluster [DBG] pgmap v2636: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:13.596 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:13 smithi055 bash[9296]: audit 2020-09-08T16:27:13.232706+0000 mon.a (mon.0) 2572 : audit [DBG] from='client.? v1:172.21.15.104:0/2746529508' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:27:13.596 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:13 smithi055 bash[9296]: audit 2020-09-08T16:27:13.360362+0000 mon.a (mon.0) 2573 : audit [INF] from='client.? v1:172.21.15.104:0/3093622339' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:27:13.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:13 smithi166 bash[9446]: cluster 2020-09-08T16:27:12.460810+0000 mgr.x (mgr.24118) 1796 : cluster [DBG] pgmap v2636: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:13.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:13 smithi166 bash[9446]: audit 2020-09-08T16:27:13.232706+0000 mon.a (mon.0) 2572 : audit [DBG] from='client.? v1:172.21.15.104:0/2746529508' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:27:13.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:13 smithi166 bash[9446]: audit 2020-09-08T16:27:13.360362+0000 mon.a (mon.0) 2573 : audit [INF] from='client.? v1:172.21.15.104:0/3093622339' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:27:13.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:13 smithi104 bash[9203]: cluster 2020-09-08T16:27:12.460810+0000 mgr.x (mgr.24118) 1796 : cluster [DBG] pgmap v2636: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:13.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:13 smithi104 bash[9203]: audit 2020-09-08T16:27:13.232706+0000 mon.a (mon.0) 2572 : audit [DBG] from='client.? v1:172.21.15.104:0/2746529508' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2020-09-08T16:27:13.934 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:13 smithi104 bash[9203]: audit 2020-09-08T16:27:13.360362+0000 mon.a (mon.0) 2573 : audit [INF] from='client.? v1:172.21.15.104:0/3093622339' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:27:14.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:14 smithi166 bash[9446]: cluster 2020-09-08T16:27:13.559659+0000 mon.a (mon.0) 2574 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:27:14.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:14 smithi166 bash[9446]: audit 2020-09-08T16:27:13.568535+0000 mon.a (mon.0) 2575 : audit [INF] from='client.? v1:172.21.15.104:0/3093622339' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:27:14.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:14 smithi166 bash[9446]: cluster 2020-09-08T16:27:13.568633+0000 mon.a (mon.0) 2576 : cluster [DBG] osdmap e925: 12 total, 12 up, 12 in 2020-09-08T16:27:14.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:14 smithi055 bash[9296]: cluster 2020-09-08T16:27:13.559659+0000 mon.a (mon.0) 2574 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:27:14.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:14 smithi055 bash[9296]: audit 2020-09-08T16:27:13.568535+0000 mon.a (mon.0) 2575 : audit [INF] from='client.? v1:172.21.15.104:0/3093622339' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:27:14.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:14 smithi055 bash[9296]: cluster 2020-09-08T16:27:13.568633+0000 mon.a (mon.0) 2576 : cluster [DBG] osdmap e925: 12 total, 12 up, 12 in 2020-09-08T16:27:14.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:14 smithi104 bash[9203]: cluster 2020-09-08T16:27:13.559659+0000 mon.a (mon.0) 2574 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2020-09-08T16:27:14.930 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:14 smithi104 bash[9203]: audit 2020-09-08T16:27:13.568535+0000 mon.a (mon.0) 2575 : audit [INF] from='client.? v1:172.21.15.104:0/3093622339' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:27:14.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:14 smithi104 bash[9203]: cluster 2020-09-08T16:27:13.568633+0000 mon.a (mon.0) 2576 : cluster [DBG] osdmap e925: 12 total, 12 up, 12 in 2020-09-08T16:27:15.436 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:27:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T16:27:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "num_ops": 0 2020-09-08T16:27:15.437 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:27:15.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:15 smithi055 bash[9296]: cluster 2020-09-08T16:27:14.461472+0000 mgr.x (mgr.24118) 1797 : cluster [DBG] pgmap v2638: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:15.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:15 smithi104 bash[9203]: cluster 2020-09-08T16:27:14.461472+0000 mgr.x (mgr.24118) 1797 : cluster [DBG] pgmap v2638: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:16.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:15 smithi166 bash[9446]: cluster 2020-09-08T16:27:14.461472+0000 mgr.x (mgr.24118) 1797 : cluster [DBG] pgmap v2638: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:16.307 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2020-09-08T16:27:16.443 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:27:16.638 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2020-09-08T16:27:16.740 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset noscrub 2020-09-08T16:27:16.876 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:27:16.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:16 smithi104 bash[9203]: cluster 2020-09-08T16:27:15.213449+0000 osd.7 (osd.7) 202 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:27:16.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:16 smithi104 bash[9203]: cluster 2020-09-08T16:27:15.359423+0000 osd.7 (osd.7) 203 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:27:17.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:16 smithi166 bash[9446]: cluster 2020-09-08T16:27:15.213449+0000 osd.7 (osd.7) 202 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:27:17.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:16 smithi166 bash[9446]: cluster 2020-09-08T16:27:15.359423+0000 osd.7 (osd.7) 203 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:27:17.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:16 smithi055 bash[9296]: cluster 2020-09-08T16:27:15.213449+0000 osd.7 (osd.7) 202 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:27:17.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:16 smithi055 bash[9296]: cluster 2020-09-08T16:27:15.359423+0000 osd.7 (osd.7) 203 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:27:17.846 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:17 smithi055 bash[9296]: cluster 2020-09-08T16:27:16.462186+0000 mgr.x (mgr.24118) 1798 : cluster [DBG] pgmap v2639: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:17.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:17 smithi104 bash[9203]: cluster 2020-09-08T16:27:16.462186+0000 mgr.x (mgr.24118) 1798 : cluster [DBG] pgmap v2639: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:18.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:17 smithi166 bash[9446]: cluster 2020-09-08T16:27:16.462186+0000 mgr.x (mgr.24118) 1798 : cluster [DBG] pgmap v2639: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:19.627 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:27:19.628 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T16:27:19.628 INFO:teuthology.orchestra.run.smithi055.stdout: "complaint_time": 30, 2020-09-08T16:27:19.629 INFO:teuthology.orchestra.run.smithi055.stdout: "num_blocked_ops": 0 2020-09-08T16:27:19.629 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:27:19.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:19 smithi104 bash[9203]: cluster 2020-09-08T16:27:18.462941+0000 mgr.x (mgr.24118) 1799 : cluster [DBG] pgmap v2640: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:19.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:19 smithi104 bash[9203]: cluster 2020-09-08T16:27:19.087241+0000 osd.9 (osd.9) 150 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:19.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:19 smithi104 bash[9203]: cluster 2020-09-08T16:27:19.087367+0000 osd.9 (osd.9) 151 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:20.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:19 smithi166 bash[9446]: cluster 2020-09-08T16:27:18.462941+0000 mgr.x (mgr.24118) 1799 : cluster [DBG] pgmap v2640: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:20.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:19 smithi166 bash[9446]: cluster 2020-09-08T16:27:19.087241+0000 osd.9 (osd.9) 150 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:20.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:19 smithi166 bash[9446]: cluster 2020-09-08T16:27:19.087367+0000 osd.9 (osd.9) 151 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:20.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:19 smithi055 bash[9296]: cluster 2020-09-08T16:27:18.462941+0000 mgr.x (mgr.24118) 1799 : cluster [DBG] pgmap v2640: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:20.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:19 smithi055 bash[9296]: cluster 2020-09-08T16:27:19.087241+0000 osd.9 (osd.9) 150 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:20.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:19 smithi055 bash[9296]: cluster 2020-09-08T16:27:19.087367+0000 osd.9 (osd.9) 151 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:20.346 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2020-09-08T16:27:20.482 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:27:20.716 INFO:teuthology.orchestra.run.smithi104.stderr:noscrub is unset 2020-09-08T16:27:21.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:20 smithi166 bash[9446]: cluster 2020-09-08T16:27:19.697013+0000 mon.a (mon.0) 2577 : cluster [DBG] osdmap e926: 12 total, 12 up, 12 in 2020-09-08T16:27:21.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:20 smithi166 bash[9446]: audit 2020-09-08T16:27:19.702680+0000 mon.c (mon.2) 99 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:21.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:20 smithi166 bash[9446]: audit 2020-09-08T16:27:19.705869+0000 mon.a (mon.0) 2578 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:21.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:20 smithi166 bash[9446]: audit 2020-09-08T16:27:19.710849+0000 mon.a (mon.0) 2579 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:21.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:20 smithi166 bash[9446]: audit 2020-09-08T16:27:19.977799+0000 mon.a (mon.0) 2580 : audit [INF] from='client.? v1:172.21.15.104:0/1279013234' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:27:21.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:20 smithi055 bash[9296]: cluster 2020-09-08T16:27:19.697013+0000 mon.a (mon.0) 2577 : cluster [DBG] osdmap e926: 12 total, 12 up, 12 in 2020-09-08T16:27:21.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:20 smithi055 bash[9296]: audit 2020-09-08T16:27:19.702680+0000 mon.c (mon.2) 99 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:21.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:20 smithi055 bash[9296]: audit 2020-09-08T16:27:19.705869+0000 mon.a (mon.0) 2578 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:21.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:20 smithi055 bash[9296]: audit 2020-09-08T16:27:19.710849+0000 mon.a (mon.0) 2579 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:21.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:20 smithi055 bash[9296]: audit 2020-09-08T16:27:19.977799+0000 mon.a (mon.0) 2580 : audit [INF] from='client.? v1:172.21.15.104:0/1279013234' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:27:21.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:20 smithi104 bash[9203]: cluster 2020-09-08T16:27:19.697013+0000 mon.a (mon.0) 2577 : cluster [DBG] osdmap e926: 12 total, 12 up, 12 in 2020-09-08T16:27:21.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:20 smithi104 bash[9203]: audit 2020-09-08T16:27:19.702680+0000 mon.c (mon.2) 99 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:21.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:20 smithi104 bash[9203]: audit 2020-09-08T16:27:19.705869+0000 mon.a (mon.0) 2578 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:27:21.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:20 smithi104 bash[9203]: audit 2020-09-08T16:27:19.710849+0000 mon.a (mon.0) 2579 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:27:21.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:20 smithi104 bash[9203]: audit 2020-09-08T16:27:19.977799+0000 mon.a (mon.0) 2580 : audit [INF] from='client.? v1:172.21.15.104:0/1279013234' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2020-09-08T16:27:21.484 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd unset nodeep-scrub 2020-09-08T16:27:21.646 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:27:21.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:21 smithi055 bash[9296]: cluster 2020-09-08T16:27:20.463421+0000 mgr.x (mgr.24118) 1800 : cluster [DBG] pgmap v2642: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:21.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:21 smithi055 bash[9296]: audit 2020-09-08T16:27:20.718640+0000 mon.a (mon.0) 2581 : audit [INF] from='client.? v1:172.21.15.104:0/1279013234' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:27:21.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:21 smithi055 bash[9296]: cluster 2020-09-08T16:27:20.718722+0000 mon.a (mon.0) 2582 : cluster [DBG] osdmap e927: 12 total, 12 up, 12 in 2020-09-08T16:27:21.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:21 smithi055 bash[9296]: cluster 2020-09-08T16:27:21.093914+0000 osd.2 (osd.2) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:21.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:21 smithi055 bash[9296]: cluster 2020-09-08T16:27:21.094007+0000 osd.2 (osd.2) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:21.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:21 smithi104 bash[9203]: cluster 2020-09-08T16:27:20.463421+0000 mgr.x (mgr.24118) 1800 : cluster [DBG] pgmap v2642: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:21.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:21 smithi104 bash[9203]: audit 2020-09-08T16:27:20.718640+0000 mon.a (mon.0) 2581 : audit [INF] from='client.? v1:172.21.15.104:0/1279013234' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:27:21.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:21 smithi104 bash[9203]: cluster 2020-09-08T16:27:20.718722+0000 mon.a (mon.0) 2582 : cluster [DBG] osdmap e927: 12 total, 12 up, 12 in 2020-09-08T16:27:21.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:21 smithi104 bash[9203]: cluster 2020-09-08T16:27:21.093914+0000 osd.2 (osd.2) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:21.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:21 smithi104 bash[9203]: cluster 2020-09-08T16:27:21.094007+0000 osd.2 (osd.2) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:22.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:21 smithi166 bash[9446]: cluster 2020-09-08T16:27:20.463421+0000 mgr.x (mgr.24118) 1800 : cluster [DBG] pgmap v2642: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:22.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:21 smithi166 bash[9446]: audit 2020-09-08T16:27:20.718640+0000 mon.a (mon.0) 2581 : audit [INF] from='client.? v1:172.21.15.104:0/1279013234' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2020-09-08T16:27:22.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:21 smithi166 bash[9446]: cluster 2020-09-08T16:27:20.718722+0000 mon.a (mon.0) 2582 : cluster [DBG] osdmap e927: 12 total, 12 up, 12 in 2020-09-08T16:27:22.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:21 smithi166 bash[9446]: cluster 2020-09-08T16:27:21.093914+0000 osd.2 (osd.2) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:22.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:21 smithi166 bash[9446]: cluster 2020-09-08T16:27:21.094007+0000 osd.2 (osd.2) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:22.987 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:22 smithi104 bash[9203]: cluster 2020-09-08T16:27:21.719539+0000 mon.a (mon.0) 2583 : cluster [DBG] osdmap e928: 12 total, 12 up, 12 in 2020-09-08T16:27:23.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:22 smithi166 bash[9446]: cluster 2020-09-08T16:27:21.719539+0000 mon.a (mon.0) 2583 : cluster [DBG] osdmap e928: 12 total, 12 up, 12 in 2020-09-08T16:27:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:22 smithi055 bash[9296]: cluster 2020-09-08T16:27:21.719539+0000 mon.a (mon.0) 2583 : cluster [DBG] osdmap e928: 12 total, 12 up, 12 in 2020-09-08T16:27:23.700 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:27:23.701 INFO:teuthology.orchestra.run.smithi055.stdout: "size": 20, 2020-09-08T16:27:23.701 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 600, 2020-09-08T16:27:23.701 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [] 2020-09-08T16:27:23.702 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:27:24.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:23 smithi166 bash[9446]: cluster 2020-09-08T16:27:22.464074+0000 mgr.x (mgr.24118) 1801 : cluster [DBG] pgmap v2645: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:24.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:23 smithi055 bash[9296]: cluster 2020-09-08T16:27:22.464074+0000 mgr.x (mgr.24118) 1801 : cluster [DBG] pgmap v2645: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:24.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:23 smithi104 bash[9203]: cluster 2020-09-08T16:27:22.464074+0000 mgr.x (mgr.24118) 1801 : cluster [DBG] pgmap v2645: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:24.436 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2020-09-08T16:27:24.569 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:27:25.752 INFO:teuthology.orchestra.run.smithi104.stderr:nodeep-scrub is unset 2020-09-08T16:27:25.793 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:25 smithi055 bash[9296]: cluster 2020-09-08T16:27:24.464833+0000 mgr.x (mgr.24118) 1802 : cluster [DBG] pgmap v2646: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:25.793 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:25 smithi055 bash[9296]: audit 2020-09-08T16:27:24.961916+0000 mon.a (mon.0) 2584 : audit [INF] from='client.? v1:172.21.15.104:0/602616069' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:27:26.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:25 smithi166 bash[9446]: cluster 2020-09-08T16:27:24.464833+0000 mgr.x (mgr.24118) 1802 : cluster [DBG] pgmap v2646: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:26.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:25 smithi166 bash[9446]: audit 2020-09-08T16:27:24.961916+0000 mon.a (mon.0) 2584 : audit [INF] from='client.? v1:172.21.15.104:0/602616069' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:27:26.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:25 smithi104 bash[9203]: cluster 2020-09-08T16:27:24.464833+0000 mgr.x (mgr.24118) 1802 : cluster [DBG] pgmap v2646: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:26.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:25 smithi104 bash[9203]: audit 2020-09-08T16:27:24.961916+0000 mon.a (mon.0) 2584 : audit [INF] from='client.? v1:172.21.15.104:0/602616069' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2020-09-08T16:27:27.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:26 smithi166 bash[9446]: cluster 2020-09-08T16:27:25.739102+0000 mon.a (mon.0) 2585 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:27:27.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:26 smithi166 bash[9446]: cluster 2020-09-08T16:27:25.739337+0000 mon.a (mon.0) 2586 : cluster [INF] Cluster is now healthy 2020-09-08T16:27:27.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:26 smithi166 bash[9446]: audit 2020-09-08T16:27:25.753068+0000 mon.a (mon.0) 2587 : audit [INF] from='client.? v1:172.21.15.104:0/602616069' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:27:27.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:26 smithi166 bash[9446]: cluster 2020-09-08T16:27:25.753139+0000 mon.a (mon.0) 2588 : cluster [DBG] osdmap e929: 12 total, 12 up, 12 in 2020-09-08T16:27:27.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:26 smithi055 bash[9296]: cluster 2020-09-08T16:27:25.739102+0000 mon.a (mon.0) 2585 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:27:27.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:26 smithi055 bash[9296]: cluster 2020-09-08T16:27:25.739337+0000 mon.a (mon.0) 2586 : cluster [INF] Cluster is now healthy 2020-09-08T16:27:27.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:26 smithi055 bash[9296]: audit 2020-09-08T16:27:25.753068+0000 mon.a (mon.0) 2587 : audit [INF] from='client.? v1:172.21.15.104:0/602616069' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:27:27.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:26 smithi055 bash[9296]: cluster 2020-09-08T16:27:25.753139+0000 mon.a (mon.0) 2588 : cluster [DBG] osdmap e929: 12 total, 12 up, 12 in 2020-09-08T16:27:27.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:26 smithi104 bash[9203]: cluster 2020-09-08T16:27:25.739102+0000 mon.a (mon.0) 2585 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2020-09-08T16:27:27.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:26 smithi104 bash[9203]: cluster 2020-09-08T16:27:25.739337+0000 mon.a (mon.0) 2586 : cluster [INF] Cluster is now healthy 2020-09-08T16:27:27.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:26 smithi104 bash[9203]: audit 2020-09-08T16:27:25.753068+0000 mon.a (mon.0) 2587 : audit [INF] from='client.? v1:172.21.15.104:0/602616069' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2020-09-08T16:27:27.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:26 smithi104 bash[9203]: cluster 2020-09-08T16:27:25.753139+0000 mon.a (mon.0) 2588 : cluster [DBG] osdmap e929: 12 total, 12 up, 12 in 2020-09-08T16:27:27.628 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:27:27.628 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T16:27:27.629 INFO:teuthology.orchestra.run.smithi055.stdout: "num_ops": 0 2020-09-08T16:27:27.629 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:27:28.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:27 smithi166 bash[9446]: cluster 2020-09-08T16:27:26.465584+0000 mgr.x (mgr.24118) 1803 : cluster [DBG] pgmap v2648: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:28.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:27 smithi055 bash[9296]: cluster 2020-09-08T16:27:26.465584+0000 mgr.x (mgr.24118) 1803 : cluster [DBG] pgmap v2648: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:28.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:27 smithi104 bash[9203]: cluster 2020-09-08T16:27:26.465584+0000 mgr.x (mgr.24118) 1803 : cluster [DBG] pgmap v2648: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:28.322 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2020-09-08T16:27:28.462 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:27:30.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:29 smithi166 bash[9446]: cluster 2020-09-08T16:27:28.459288+0000 osd.0 (osd.0) 162 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:30.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:29 smithi166 bash[9446]: cluster 2020-09-08T16:27:28.459418+0000 osd.0 (osd.0) 163 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:30.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:29 smithi166 bash[9446]: cluster 2020-09-08T16:27:28.466068+0000 mgr.x (mgr.24118) 1804 : cluster [DBG] pgmap v2649: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:30.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:29 smithi166 bash[9446]: cluster 2020-09-08T16:27:28.769168+0000 mon.a (mon.0) 2589 : cluster [DBG] osdmap e930: 12 total, 12 up, 12 in 2020-09-08T16:27:30.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:29 smithi055 bash[9296]: cluster 2020-09-08T16:27:28.459288+0000 osd.0 (osd.0) 162 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:30.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:29 smithi055 bash[9296]: cluster 2020-09-08T16:27:28.459418+0000 osd.0 (osd.0) 163 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:30.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:29 smithi055 bash[9296]: cluster 2020-09-08T16:27:28.466068+0000 mgr.x (mgr.24118) 1804 : cluster [DBG] pgmap v2649: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:30.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:29 smithi055 bash[9296]: cluster 2020-09-08T16:27:28.769168+0000 mon.a (mon.0) 2589 : cluster [DBG] osdmap e930: 12 total, 12 up, 12 in 2020-09-08T16:27:30.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:29 smithi104 bash[9203]: cluster 2020-09-08T16:27:28.459288+0000 osd.0 (osd.0) 162 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:30.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:29 smithi104 bash[9203]: cluster 2020-09-08T16:27:28.459418+0000 osd.0 (osd.0) 163 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:30.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:29 smithi104 bash[9203]: cluster 2020-09-08T16:27:28.466068+0000 mgr.x (mgr.24118) 1804 : cluster [DBG] pgmap v2649: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:30.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:29 smithi104 bash[9203]: cluster 2020-09-08T16:27:28.769168+0000 mon.a (mon.0) 2589 : cluster [DBG] osdmap e930: 12 total, 12 up, 12 in 2020-09-08T16:27:31.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:30 smithi166 bash[9446]: cluster 2020-09-08T16:27:28.917945+0000 osd.7 (osd.7) 204 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:31.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:30 smithi166 bash[9446]: cluster 2020-09-08T16:27:28.918089+0000 osd.7 (osd.7) 205 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:31.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:30 smithi166 bash[9446]: cluster 2020-09-08T16:27:29.772187+0000 mon.a (mon.0) 2590 : cluster [DBG] osdmap e931: 12 total, 12 up, 12 in 2020-09-08T16:27:31.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:30 smithi055 bash[9296]: cluster 2020-09-08T16:27:28.917945+0000 osd.7 (osd.7) 204 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:31.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:30 smithi055 bash[9296]: cluster 2020-09-08T16:27:28.918089+0000 osd.7 (osd.7) 205 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:31.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:30 smithi055 bash[9296]: cluster 2020-09-08T16:27:29.772187+0000 mon.a (mon.0) 2590 : cluster [DBG] osdmap e931: 12 total, 12 up, 12 in 2020-09-08T16:27:31.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:30 smithi104 bash[9203]: cluster 2020-09-08T16:27:28.917945+0000 osd.7 (osd.7) 204 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:31.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:30 smithi104 bash[9203]: cluster 2020-09-08T16:27:28.918089+0000 osd.7 (osd.7) 205 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:31.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:30 smithi104 bash[9203]: cluster 2020-09-08T16:27:29.772187+0000 mon.a (mon.0) 2590 : cluster [DBG] osdmap e931: 12 total, 12 up, 12 in 2020-09-08T16:27:31.457 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:27:31.457 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T16:27:31.458 INFO:teuthology.orchestra.run.smithi055.stdout: "complaint_time": 30, 2020-09-08T16:27:31.458 INFO:teuthology.orchestra.run.smithi055.stdout: "num_blocked_ops": 0 2020-09-08T16:27:31.458 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:27:31.917 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:31 smithi055 bash[9296]: cluster 2020-09-08T16:27:30.466651+0000 mgr.x (mgr.24118) 1805 : cluster [DBG] pgmap v2652: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:32.110 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2020-09-08T16:27:32.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:31 smithi166 bash[9446]: cluster 2020-09-08T16:27:30.466651+0000 mgr.x (mgr.24118) 1805 : cluster [DBG] pgmap v2652: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:32.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:31 smithi104 bash[9203]: cluster 2020-09-08T16:27:30.466651+0000 mgr.x (mgr.24118) 1805 : cluster [DBG] pgmap v2652: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:32.249 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:27:33.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:33 smithi055 bash[9296]: cluster 2020-09-08T16:27:32.467312+0000 mgr.x (mgr.24118) 1806 : cluster [DBG] pgmap v2653: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:34.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:33 smithi166 bash[9446]: cluster 2020-09-08T16:27:32.467312+0000 mgr.x (mgr.24118) 1806 : cluster [DBG] pgmap v2653: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:34.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:33 smithi104 bash[9203]: cluster 2020-09-08T16:27:32.467312+0000 mgr.x (mgr.24118) 1806 : cluster [DBG] pgmap v2653: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:35.284 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:27:35.285 INFO:teuthology.orchestra.run.smithi055.stdout: "size": 20, 2020-09-08T16:27:35.285 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 600, 2020-09-08T16:27:35.285 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [] 2020-09-08T16:27:35.285 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:27:35.999 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2020-09-08T16:27:36.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:35 smithi166 bash[9446]: cluster 2020-09-08T16:27:34.468053+0000 mgr.x (mgr.24118) 1807 : cluster [DBG] pgmap v2654: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:36.136 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:27:36.163 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:35 smithi055 bash[9296]: cluster 2020-09-08T16:27:34.468053+0000 mgr.x (mgr.24118) 1807 : cluster [DBG] pgmap v2654: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:36.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:35 smithi104 bash[9203]: cluster 2020-09-08T16:27:34.468053+0000 mgr.x (mgr.24118) 1807 : cluster [DBG] pgmap v2654: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:38.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:37 smithi166 bash[9446]: cluster 2020-09-08T16:27:36.469232+0000 mgr.x (mgr.24118) 1808 : cluster [DBG] pgmap v2655: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:38.166 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:37 smithi055 bash[9296]: cluster 2020-09-08T16:27:36.469232+0000 mgr.x (mgr.24118) 1808 : cluster [DBG] pgmap v2655: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:38.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:37 smithi104 bash[9203]: cluster 2020-09-08T16:27:36.469232+0000 mgr.x (mgr.24118) 1808 : cluster [DBG] pgmap v2655: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:39.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:38 smithi166 bash[9446]: cluster 2020-09-08T16:27:37.639703+0000 osd.5 (osd.5) 168 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:39.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:38 smithi166 bash[9446]: cluster 2020-09-08T16:27:37.639841+0000 osd.5 (osd.5) 169 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:39.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:38 smithi166 bash[9446]: cluster 2020-09-08T16:27:37.803435+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e932: 12 total, 12 up, 12 in 2020-09-08T16:27:39.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:38 smithi055 bash[9296]: cluster 2020-09-08T16:27:37.639703+0000 osd.5 (osd.5) 168 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:39.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:38 smithi055 bash[9296]: cluster 2020-09-08T16:27:37.639841+0000 osd.5 (osd.5) 169 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:39.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:38 smithi055 bash[9296]: cluster 2020-09-08T16:27:37.803435+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e932: 12 total, 12 up, 12 in 2020-09-08T16:27:39.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:38 smithi104 bash[9203]: cluster 2020-09-08T16:27:37.639703+0000 osd.5 (osd.5) 168 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:39.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:38 smithi104 bash[9203]: cluster 2020-09-08T16:27:37.639841+0000 osd.5 (osd.5) 169 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:39.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:38 smithi104 bash[9203]: cluster 2020-09-08T16:27:37.803435+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e932: 12 total, 12 up, 12 in 2020-09-08T16:27:39.177 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:27:39.177 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T16:27:39.178 INFO:teuthology.orchestra.run.smithi055.stdout: "num_ops": 0 2020-09-08T16:27:39.178 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:27:39.878 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2020-09-08T16:27:40.007 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:27:40.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:39 smithi166 bash[9446]: cluster 2020-09-08T16:27:38.470046+0000 mgr.x (mgr.24118) 1809 : cluster [DBG] pgmap v2657: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:40.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:39 smithi055 bash[9296]: cluster 2020-09-08T16:27:38.470046+0000 mgr.x (mgr.24118) 1809 : cluster [DBG] pgmap v2657: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:40.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:39 smithi104 bash[9203]: cluster 2020-09-08T16:27:38.470046+0000 mgr.x (mgr.24118) 1809 : cluster [DBG] pgmap v2657: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:41.106 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:40 smithi055 bash[9296]: cluster 2020-09-08T16:27:40.215018+0000 osd.3 (osd.3) 171 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:41.106 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:40 smithi055 bash[9296]: cluster 2020-09-08T16:27:40.215144+0000 osd.3 (osd.3) 172 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:41.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:40 smithi166 bash[9446]: cluster 2020-09-08T16:27:40.215018+0000 osd.3 (osd.3) 171 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:41.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:40 smithi166 bash[9446]: cluster 2020-09-08T16:27:40.215144+0000 osd.3 (osd.3) 172 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:41.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:40 smithi104 bash[9203]: cluster 2020-09-08T16:27:40.215018+0000 osd.3 (osd.3) 171 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:41.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:40 smithi104 bash[9203]: cluster 2020-09-08T16:27:40.215144+0000 osd.3 (osd.3) 172 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:42.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:41 smithi166 bash[9446]: cluster 2020-09-08T16:27:40.470700+0000 mgr.x (mgr.24118) 1810 : cluster [DBG] pgmap v2658: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:42.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:41 smithi166 bash[9446]: cluster 2020-09-08T16:27:40.742271+0000 osd.10 (osd.10) 147 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:42.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:41 smithi166 bash[9446]: cluster 2020-09-08T16:27:40.742388+0000 osd.10 (osd.10) 148 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:42.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:41 smithi166 bash[9446]: cluster 2020-09-08T16:27:40.827373+0000 mon.a (mon.0) 2592 : cluster [DBG] osdmap e933: 12 total, 12 up, 12 in 2020-09-08T16:27:42.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:41 smithi055 bash[9296]: cluster 2020-09-08T16:27:40.470700+0000 mgr.x (mgr.24118) 1810 : cluster [DBG] pgmap v2658: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:42.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:41 smithi055 bash[9296]: cluster 2020-09-08T16:27:40.742271+0000 osd.10 (osd.10) 147 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:42.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:41 smithi055 bash[9296]: cluster 2020-09-08T16:27:40.742388+0000 osd.10 (osd.10) 148 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:42.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:41 smithi055 bash[9296]: cluster 2020-09-08T16:27:40.827373+0000 mon.a (mon.0) 2592 : cluster [DBG] osdmap e933: 12 total, 12 up, 12 in 2020-09-08T16:27:42.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:41 smithi104 bash[9203]: cluster 2020-09-08T16:27:40.470700+0000 mgr.x (mgr.24118) 1810 : cluster [DBG] pgmap v2658: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:42.183 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:41 smithi104 bash[9203]: cluster 2020-09-08T16:27:40.742271+0000 osd.10 (osd.10) 147 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:42.183 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:41 smithi104 bash[9203]: cluster 2020-09-08T16:27:40.742388+0000 osd.10 (osd.10) 148 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:42.184 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:41 smithi104 bash[9203]: cluster 2020-09-08T16:27:40.827373+0000 mon.a (mon.0) 2592 : cluster [DBG] osdmap e933: 12 total, 12 up, 12 in 2020-09-08T16:27:43.202 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:27:43.203 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T16:27:43.203 INFO:teuthology.orchestra.run.smithi055.stdout: "complaint_time": 30, 2020-09-08T16:27:43.203 INFO:teuthology.orchestra.run.smithi055.stdout: "num_blocked_ops": 0 2020-09-08T16:27:43.204 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:27:43.863 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2020-09-08T16:27:43.997 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:27:44.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:43 smithi166 bash[9446]: cluster 2020-09-08T16:27:42.471328+0000 mgr.x (mgr.24118) 1811 : cluster [DBG] pgmap v2660: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:44.279 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:43 smithi104 bash[9203]: cluster 2020-09-08T16:27:42.471328+0000 mgr.x (mgr.24118) 1811 : cluster [DBG] pgmap v2660: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:44.279 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:43 smithi055 bash[9296]: cluster 2020-09-08T16:27:42.471328+0000 mgr.x (mgr.24118) 1811 : cluster [DBG] pgmap v2660: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:46.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:45 smithi166 bash[9446]: cluster 2020-09-08T16:27:44.471986+0000 mgr.x (mgr.24118) 1812 : cluster [DBG] pgmap v2661: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:46.162 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:45 smithi055 bash[9296]: cluster 2020-09-08T16:27:44.471986+0000 mgr.x (mgr.24118) 1812 : cluster [DBG] pgmap v2661: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:46.178 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:45 smithi104 bash[9203]: cluster 2020-09-08T16:27:44.471986+0000 mgr.x (mgr.24118) 1812 : cluster [DBG] pgmap v2661: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:47.130 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:27:47.131 INFO:teuthology.orchestra.run.smithi055.stdout: "size": 20, 2020-09-08T16:27:47.131 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 600, 2020-09-08T16:27:47.131 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [] 2020-09-08T16:27:47.131 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:27:47.823 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2020-09-08T16:27:47.956 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:27:48.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:47 smithi166 bash[9446]: cluster 2020-09-08T16:27:46.472694+0000 mgr.x (mgr.24118) 1813 : cluster [DBG] pgmap v2662: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:48.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:47 smithi055 bash[9296]: cluster 2020-09-08T16:27:46.472694+0000 mgr.x (mgr.24118) 1813 : cluster [DBG] pgmap v2662: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:48.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:47 smithi104 bash[9203]: cluster 2020-09-08T16:27:46.472694+0000 mgr.x (mgr.24118) 1813 : cluster [DBG] pgmap v2662: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:50.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:49 smithi166 bash[9446]: cluster 2020-09-08T16:27:48.473378+0000 mgr.x (mgr.24118) 1814 : cluster [DBG] pgmap v2663: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:50.163 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:49 smithi055 bash[9296]: cluster 2020-09-08T16:27:48.473378+0000 mgr.x (mgr.24118) 1814 : cluster [DBG] pgmap v2663: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:50.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:49 smithi104 bash[9203]: cluster 2020-09-08T16:27:48.473378+0000 mgr.x (mgr.24118) 1814 : cluster [DBG] pgmap v2663: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:51.081 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:27:51.082 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T16:27:51.082 INFO:teuthology.orchestra.run.smithi055.stdout: "num_ops": 0 2020-09-08T16:27:51.083 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:27:51.417 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:51 smithi055 bash[9296]: cluster 2020-09-08T16:27:50.414218+0000 osd.8 (osd.8) 142 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:51.418 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:51 smithi055 bash[9296]: cluster 2020-09-08T16:27:50.414353+0000 osd.8 (osd.8) 143 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:51.418 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:51 smithi055 bash[9296]: cluster 2020-09-08T16:27:50.473999+0000 mgr.x (mgr.24118) 1815 : cluster [DBG] pgmap v2664: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:51.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:51 smithi104 bash[9203]: cluster 2020-09-08T16:27:50.414218+0000 osd.8 (osd.8) 142 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:51.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:51 smithi104 bash[9203]: cluster 2020-09-08T16:27:50.414353+0000 osd.8 (osd.8) 143 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:51.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:51 smithi104 bash[9203]: cluster 2020-09-08T16:27:50.473999+0000 mgr.x (mgr.24118) 1815 : cluster [DBG] pgmap v2664: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:51.731 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2020-09-08T16:27:51.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:51 smithi166 bash[9446]: cluster 2020-09-08T16:27:50.414218+0000 osd.8 (osd.8) 142 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:51.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:51 smithi166 bash[9446]: cluster 2020-09-08T16:27:50.414353+0000 osd.8 (osd.8) 143 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:51.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:51 smithi166 bash[9446]: cluster 2020-09-08T16:27:50.473999+0000 mgr.x (mgr.24118) 1815 : cluster [DBG] pgmap v2664: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:51.867 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:27:52.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:52 smithi055 bash[9296]: cluster 2020-09-08T16:27:51.386227+0000 mon.a (mon.0) 2593 : cluster [DBG] osdmap e934: 12 total, 12 up, 12 in 2020-09-08T16:27:52.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:52 smithi104 bash[9203]: cluster 2020-09-08T16:27:51.386227+0000 mon.a (mon.0) 2593 : cluster [DBG] osdmap e934: 12 total, 12 up, 12 in 2020-09-08T16:27:52.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:52 smithi166 bash[9446]: cluster 2020-09-08T16:27:51.386227+0000 mon.a (mon.0) 2593 : cluster [DBG] osdmap e934: 12 total, 12 up, 12 in 2020-09-08T16:27:53.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:53 smithi055 bash[9296]: cluster 2020-09-08T16:27:52.474683+0000 mgr.x (mgr.24118) 1816 : cluster [DBG] pgmap v2666: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:53.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:53 smithi104 bash[9203]: cluster 2020-09-08T16:27:52.474683+0000 mgr.x (mgr.24118) 1816 : cluster [DBG] pgmap v2666: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:53.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:53 smithi166 bash[9446]: cluster 2020-09-08T16:27:52.474683+0000 mgr.x (mgr.24118) 1816 : cluster [DBG] pgmap v2666: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:55.036 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:27:55.037 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [], 2020-09-08T16:27:55.037 INFO:teuthology.orchestra.run.smithi055.stdout: "complaint_time": 30, 2020-09-08T16:27:55.038 INFO:teuthology.orchestra.run.smithi055.stdout: "num_blocked_ops": 0 2020-09-08T16:27:55.038 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:27:55.786 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:55 smithi055 bash[9296]: cluster 2020-09-08T16:27:54.475418+0000 mgr.x (mgr.24118) 1817 : cluster [DBG] pgmap v2667: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:55.788 INFO:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2020-09-08T16:27:55.869 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:55 smithi166 bash[9446]: cluster 2020-09-08T16:27:54.475418+0000 mgr.x (mgr.24118) 1817 : cluster [DBG] pgmap v2667: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:55.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:55 smithi104 bash[9203]: cluster 2020-09-08T16:27:54.475418+0000 mgr.x (mgr.24118) 1817 : cluster [DBG] pgmap v2667: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:55.928 INFO:teuthology.orchestra.run.smithi055.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.b/config 2020-09-08T16:27:56.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:56 smithi166 bash[9446]: cluster 2020-09-08T16:27:55.915606+0000 osd.1 (osd.1) 188 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:56.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:56 smithi166 bash[9446]: cluster 2020-09-08T16:27:55.915740+0000 osd.1 (osd.1) 189 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:56.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:56 smithi055 bash[9296]: cluster 2020-09-08T16:27:55.915606+0000 osd.1 (osd.1) 188 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:56.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:56 smithi055 bash[9296]: cluster 2020-09-08T16:27:55.915740+0000 osd.1 (osd.1) 189 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:56.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:56 smithi104 bash[9203]: cluster 2020-09-08T16:27:55.915606+0000 osd.1 (osd.1) 188 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:27:56.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:56 smithi104 bash[9203]: cluster 2020-09-08T16:27:55.915740+0000 osd.1 (osd.1) 189 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:27:57.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:57 smithi055 bash[9296]: cluster 2020-09-08T16:27:56.476112+0000 mgr.x (mgr.24118) 1818 : cluster [DBG] pgmap v2668: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:57.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:57 smithi055 bash[9296]: cluster 2020-09-08T16:27:56.548898+0000 mon.a (mon.0) 2594 : cluster [DBG] osdmap e935: 12 total, 12 up, 12 in 2020-09-08T16:27:57.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:57 smithi166 bash[9446]: cluster 2020-09-08T16:27:56.476112+0000 mgr.x (mgr.24118) 1818 : cluster [DBG] pgmap v2668: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:57.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:57 smithi166 bash[9446]: cluster 2020-09-08T16:27:56.548898+0000 mon.a (mon.0) 2594 : cluster [DBG] osdmap e935: 12 total, 12 up, 12 in 2020-09-08T16:27:57.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:57 smithi104 bash[9203]: cluster 2020-09-08T16:27:56.476112+0000 mgr.x (mgr.24118) 1818 : cluster [DBG] pgmap v2668: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:57.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:57 smithi104 bash[9203]: cluster 2020-09-08T16:27:56.548898+0000 mon.a (mon.0) 2594 : cluster [DBG] osdmap e935: 12 total, 12 up, 12 in 2020-09-08T16:27:59.190 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2020-09-08T16:27:59.191 INFO:teuthology.orchestra.run.smithi055.stdout: "size": 20, 2020-09-08T16:27:59.191 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 600, 2020-09-08T16:27:59.191 INFO:teuthology.orchestra.run.smithi055.stdout: "ops": [ 2020-09-08T16:27:59.192 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:27:59.192 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "MOSDScrubReserve(1.0 GRANT e925)", 2020-09-08T16:27:59.192 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:27:15.212607+0000", 2020-09-08T16:27:59.192 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 43.983332728000001, 2020-09-08T16:27:59.193 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00059658200000000002, 2020-09-08T16:27:59.193 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:27:59.193 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "started", 2020-09-08T16:27:59.193 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:27:59.193 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:27:59.194 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:27:59.194 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:27:15.212607+0000", 2020-09-08T16:27:59.194 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:27:59.197 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:27:59.197 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:27:59.198 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:27:59.198 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:27:15.212607+0000", 2020-09-08T16:27:59.198 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00017361000000000001 2020-09-08T16:27:59.198 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:27:59.198 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:27:59.199 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:27:59.199 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:27:15.212781+0000", 2020-09-08T16:27:59.199 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 3.046e-05 2020-09-08T16:27:59.199 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:27:59.199 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:27:59.200 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:27:59.200 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:27:15.212811+0000", 2020-09-08T16:27:59.200 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.4221999999999998e-05 2020-09-08T16:27:59.200 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:27:59.200 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:27:59.201 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:27:59.201 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:27:15.212856+0000", 2020-09-08T16:27:59.201 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.2710000000000001e-05 2020-09-08T16:27:59.202 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:27:59.202 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:27:59.202 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:27:59.203 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:27:15.212878+0000", 2020-09-08T16:27:59.203 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00015728399999999999 2020-09-08T16:27:59.203 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:27:59.203 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:27:59.203 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:27:59.204 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:27:15.213036+0000", 2020-09-08T16:27:59.204 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.3681000000000001e-05 2020-09-08T16:27:59.204 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:27:59.205 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:27:59.205 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:27:59.206 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:27:15.213119+0000", 2020-09-08T16:27:59.206 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 8.4615000000000002e-05 2020-09-08T16:27:59.206 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:27:59.206 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:27:59.206 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:27:59.226 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:27:15.213204+0000", 2020-09-08T16:27:59.226 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00059658200000000002 2020-09-08T16:27:59.227 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:27:59.227 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:27:59.227 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:27:59.228 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:27:59.228 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:27:59.228 INFO:teuthology.orchestra.run.smithi055.stdout: "description": "rep_scrubmap(1.0 e90 from shard 11)", 2020-09-08T16:27:59.228 INFO:teuthology.orchestra.run.smithi055.stdout: "initiated_at": "2020-09-08T16:27:15.358340+0000", 2020-09-08T16:27:59.229 INFO:teuthology.orchestra.run.smithi055.stdout: "age": 43.837599941999997, 2020-09-08T16:27:59.229 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00042371299999999999, 2020-09-08T16:27:59.229 INFO:teuthology.orchestra.run.smithi055.stdout: "type_data": { 2020-09-08T16:27:59.229 INFO:teuthology.orchestra.run.smithi055.stdout: "flag_point": "started", 2020-09-08T16:27:59.230 INFO:teuthology.orchestra.run.smithi055.stdout: "events": [ 2020-09-08T16:27:59.230 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:27:59.230 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "initiated", 2020-09-08T16:27:59.230 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:27:15.358340+0000", 2020-09-08T16:27:59.230 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0 2020-09-08T16:27:59.231 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:27:59.231 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:27:59.231 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "throttled", 2020-09-08T16:27:59.231 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:27:15.358340+0000", 2020-09-08T16:27:59.232 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00012696100000000001 2020-09-08T16:27:59.232 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:27:59.232 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:27:59.233 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "header_read", 2020-09-08T16:27:59.233 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:27:15.358467+0000", 2020-09-08T16:27:59.233 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.5107e-05 2020-09-08T16:27:59.234 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:27:59.234 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:27:59.234 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "all_read", 2020-09-08T16:27:59.235 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:27:15.358482+0000", 2020-09-08T16:27:59.237 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 2.9289e-05 2020-09-08T16:27:59.237 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:27:59.237 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:27:59.238 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "dispatched", 2020-09-08T16:27:59.238 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:27:15.358511+0000", 2020-09-08T16:27:59.238 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 1.0966e-05 2020-09-08T16:27:59.238 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:27:59.238 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:27:59.239 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "queued_for_pg", 2020-09-08T16:27:59.239 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:27:15.358522+0000", 2020-09-08T16:27:59.239 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.000119085 2020-09-08T16:27:59.239 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:27:59.239 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:27:59.240 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "reached_pg", 2020-09-08T16:27:59.240 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:27:15.358641+0000", 2020-09-08T16:27:59.240 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 4.6681999999999998e-05 2020-09-08T16:27:59.240 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:27:59.241 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:27:59.241 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "started", 2020-09-08T16:27:59.241 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:27:15.358688+0000", 2020-09-08T16:27:59.241 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 7.5623000000000004e-05 2020-09-08T16:27:59.241 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2020-09-08T16:27:59.242 INFO:teuthology.orchestra.run.smithi055.stdout: { 2020-09-08T16:27:59.242 INFO:teuthology.orchestra.run.smithi055.stdout: "event": "done", 2020-09-08T16:27:59.242 INFO:teuthology.orchestra.run.smithi055.stdout: "time": "2020-09-08T16:27:15.358764+0000", 2020-09-08T16:27:59.243 INFO:teuthology.orchestra.run.smithi055.stdout: "duration": 0.00042371299999999999 2020-09-08T16:27:59.243 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:27:59.243 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:27:59.243 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:27:59.244 INFO:teuthology.orchestra.run.smithi055.stdout: } 2020-09-08T16:27:59.244 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2020-09-08T16:27:59.244 INFO:teuthology.orchestra.run.smithi055.stdout:} 2020-09-08T16:27:59.849 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:27:59 smithi055 bash[9296]: cluster 2020-09-08T16:27:58.476782+0000 mgr.x (mgr.24118) 1819 : cluster [DBG] pgmap v2670: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:59.850 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2020-09-08T16:27:59.862 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:27:59 smithi166 bash[9446]: cluster 2020-09-08T16:27:58.476782+0000 mgr.x (mgr.24118) 1819 : cluster [DBG] pgmap v2670: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:59.931 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:27:59 smithi104 bash[9203]: cluster 2020-09-08T16:27:58.476782+0000 mgr.x (mgr.24118) 1819 : cluster [DBG] pgmap v2670: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:27:59.985 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:28:01.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:01 smithi055 bash[9296]: cluster 2020-09-08T16:28:00.477418+0000 mgr.x (mgr.24118) 1820 : cluster [DBG] pgmap v2671: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:01.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:01 smithi055 bash[9296]: cluster 2020-09-08T16:28:00.765817+0000 osd.6 (osd.6) 153 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:01.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:01 smithi055 bash[9296]: cluster 2020-09-08T16:28:00.765951+0000 osd.6 (osd.6) 154 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:01.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:01 smithi104 bash[9203]: cluster 2020-09-08T16:28:00.477418+0000 mgr.x (mgr.24118) 1820 : cluster [DBG] pgmap v2671: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:01.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:01 smithi104 bash[9203]: cluster 2020-09-08T16:28:00.765817+0000 osd.6 (osd.6) 153 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:01.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:01 smithi104 bash[9203]: cluster 2020-09-08T16:28:00.765951+0000 osd.6 (osd.6) 154 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:01.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:01 smithi166 bash[9446]: cluster 2020-09-08T16:28:00.477418+0000 mgr.x (mgr.24118) 1820 : cluster [DBG] pgmap v2671: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:01.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:01 smithi166 bash[9446]: cluster 2020-09-08T16:28:00.765817+0000 osd.6 (osd.6) 153 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:01.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:01 smithi166 bash[9446]: cluster 2020-09-08T16:28:00.765951+0000 osd.6 (osd.6) 154 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:02.662 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:02 smithi055 bash[9296]: cluster 2020-09-08T16:28:01.396663+0000 mon.a (mon.0) 2595 : cluster [DBG] osdmap e936: 12 total, 12 up, 12 in 2020-09-08T16:28:02.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:02 smithi104 bash[9203]: cluster 2020-09-08T16:28:01.396663+0000 mon.a (mon.0) 2595 : cluster [DBG] osdmap e936: 12 total, 12 up, 12 in 2020-09-08T16:28:02.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:02 smithi166 bash[9446]: cluster 2020-09-08T16:28:01.396663+0000 mon.a (mon.0) 2595 : cluster [DBG] osdmap e936: 12 total, 12 up, 12 in 2020-09-08T16:28:03.305 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:28:03.306 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T16:28:03.306 INFO:teuthology.orchestra.run.smithi166.stdout: "num_ops": 0 2020-09-08T16:28:03.307 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:28:03.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:03 smithi166 bash[9446]: cluster 2020-09-08T16:28:02.478076+0000 mgr.x (mgr.24118) 1821 : cluster [DBG] pgmap v2673: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:03.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:03 smithi055 bash[9296]: cluster 2020-09-08T16:28:02.478076+0000 mgr.x (mgr.24118) 1821 : cluster [DBG] pgmap v2673: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:03.677 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:03 smithi104 bash[9203]: cluster 2020-09-08T16:28:02.478076+0000 mgr.x (mgr.24118) 1821 : cluster [DBG] pgmap v2673: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:04.042 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2020-09-08T16:28:04.227 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:28:05.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:05 smithi166 bash[9446]: cluster 2020-09-08T16:28:04.478724+0000 mgr.x (mgr.24118) 1822 : cluster [DBG] pgmap v2674: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:05.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:05 smithi055 bash[9296]: cluster 2020-09-08T16:28:04.478724+0000 mgr.x (mgr.24118) 1822 : cluster [DBG] pgmap v2674: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:05.934 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:05 smithi104 bash[9203]: cluster 2020-09-08T16:28:04.478724+0000 mgr.x (mgr.24118) 1822 : cluster [DBG] pgmap v2674: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:07.242 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:28:07.242 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T16:28:07.243 INFO:teuthology.orchestra.run.smithi166.stdout: "complaint_time": 30, 2020-09-08T16:28:07.243 INFO:teuthology.orchestra.run.smithi166.stdout: "num_blocked_ops": 0 2020-09-08T16:28:07.243 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:28:07.616 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:07 smithi166 bash[9446]: cluster 2020-09-08T16:28:06.479358+0000 mgr.x (mgr.24118) 1823 : cluster [DBG] pgmap v2675: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:07.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:07 smithi166 bash[9446]: cluster 2020-09-08T16:28:07.006963+0000 osd.11 (osd.11) 168 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:07.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:07 smithi166 bash[9446]: cluster 2020-09-08T16:28:07.007106+0000 osd.11 (osd.11) 169 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:07.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:07 smithi055 bash[9296]: cluster 2020-09-08T16:28:06.479358+0000 mgr.x (mgr.24118) 1823 : cluster [DBG] pgmap v2675: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:07.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:07 smithi055 bash[9296]: cluster 2020-09-08T16:28:07.006963+0000 osd.11 (osd.11) 168 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:07.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:07 smithi055 bash[9296]: cluster 2020-09-08T16:28:07.007106+0000 osd.11 (osd.11) 169 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:07.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:07 smithi104 bash[9203]: cluster 2020-09-08T16:28:06.479358+0000 mgr.x (mgr.24118) 1823 : cluster [DBG] pgmap v2675: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:07.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:07 smithi104 bash[9203]: cluster 2020-09-08T16:28:07.006963+0000 osd.11 (osd.11) 168 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:07.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:07 smithi104 bash[9203]: cluster 2020-09-08T16:28:07.007106+0000 osd.11 (osd.11) 169 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:07.979 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2020-09-08T16:28:08.108 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:28:08.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:08 smithi166 bash[9446]: cluster 2020-09-08T16:28:07.546972+0000 mon.a (mon.0) 2596 : cluster [DBG] osdmap e937: 12 total, 12 up, 12 in 2020-09-08T16:28:08.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:08 smithi055 bash[9296]: cluster 2020-09-08T16:28:07.546972+0000 mon.a (mon.0) 2596 : cluster [DBG] osdmap e937: 12 total, 12 up, 12 in 2020-09-08T16:28:08.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:08 smithi104 bash[9203]: cluster 2020-09-08T16:28:07.546972+0000 mon.a (mon.0) 2596 : cluster [DBG] osdmap e937: 12 total, 12 up, 12 in 2020-09-08T16:28:09.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:09 smithi166 bash[9446]: cluster 2020-09-08T16:28:08.479928+0000 mgr.x (mgr.24118) 1824 : cluster [DBG] pgmap v2677: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:09.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:09 smithi055 bash[9296]: cluster 2020-09-08T16:28:08.479928+0000 mgr.x (mgr.24118) 1824 : cluster [DBG] pgmap v2677: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:09.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:09 smithi104 bash[9203]: cluster 2020-09-08T16:28:08.479928+0000 mgr.x (mgr.24118) 1824 : cluster [DBG] pgmap v2677: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:11.230 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:28:11.230 INFO:teuthology.orchestra.run.smithi166.stdout: "size": 20, 2020-09-08T16:28:11.231 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 600, 2020-09-08T16:28:11.231 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [] 2020-09-08T16:28:11.231 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:28:11.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:11 smithi166 bash[9446]: cluster 2020-09-08T16:28:10.058817+0000 osd.4 (osd.4) 113 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:11.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:11 smithi166 bash[9446]: cluster 2020-09-08T16:28:10.058947+0000 osd.4 (osd.4) 114 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:11.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:11 smithi166 bash[9446]: cluster 2020-09-08T16:28:10.480547+0000 mgr.x (mgr.24118) 1825 : cluster [DBG] pgmap v2678: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:11.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:11 smithi166 bash[9446]: cluster 2020-09-08T16:28:10.574521+0000 mon.a (mon.0) 2597 : cluster [DBG] osdmap e938: 12 total, 12 up, 12 in 2020-09-08T16:28:11.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:11 smithi055 bash[9296]: cluster 2020-09-08T16:28:10.058817+0000 osd.4 (osd.4) 113 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:11.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:11 smithi055 bash[9296]: cluster 2020-09-08T16:28:10.058947+0000 osd.4 (osd.4) 114 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:11.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:11 smithi055 bash[9296]: cluster 2020-09-08T16:28:10.480547+0000 mgr.x (mgr.24118) 1825 : cluster [DBG] pgmap v2678: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:11.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:11 smithi055 bash[9296]: cluster 2020-09-08T16:28:10.574521+0000 mon.a (mon.0) 2597 : cluster [DBG] osdmap e938: 12 total, 12 up, 12 in 2020-09-08T16:28:11.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:11 smithi104 bash[9203]: cluster 2020-09-08T16:28:10.058817+0000 osd.4 (osd.4) 113 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:11.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:11 smithi104 bash[9203]: cluster 2020-09-08T16:28:10.058947+0000 osd.4 (osd.4) 114 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:11.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:11 smithi104 bash[9203]: cluster 2020-09-08T16:28:10.480547+0000 mgr.x (mgr.24118) 1825 : cluster [DBG] pgmap v2678: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:11.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:11 smithi104 bash[9203]: cluster 2020-09-08T16:28:10.574521+0000 mon.a (mon.0) 2597 : cluster [DBG] osdmap e938: 12 total, 12 up, 12 in 2020-09-08T16:28:11.972 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_ops_in_flight 2020-09-08T16:28:12.110 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:28:13.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:13 smithi166 bash[9446]: cluster 2020-09-08T16:28:12.481177+0000 mgr.x (mgr.24118) 1826 : cluster [DBG] pgmap v2680: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:13.915 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:13 smithi055 bash[9296]: cluster 2020-09-08T16:28:12.481177+0000 mgr.x (mgr.24118) 1826 : cluster [DBG] pgmap v2680: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:13.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:13 smithi104 bash[9203]: cluster 2020-09-08T16:28:12.481177+0000 mgr.x (mgr.24118) 1826 : cluster [DBG] pgmap v2680: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:15.232 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:28:15.233 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T16:28:15.233 INFO:teuthology.orchestra.run.smithi166.stdout: "num_ops": 0 2020-09-08T16:28:15.233 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:28:15.876 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_blocked_ops 2020-09-08T16:28:15.889 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:15 smithi166 bash[9446]: cluster 2020-09-08T16:28:14.481815+0000 mgr.x (mgr.24118) 1827 : cluster [DBG] pgmap v2681: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:15.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:15 smithi055 bash[9296]: cluster 2020-09-08T16:28:14.481815+0000 mgr.x (mgr.24118) 1827 : cluster [DBG] pgmap v2681: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:15.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:15 smithi104 bash[9203]: cluster 2020-09-08T16:28:14.481815+0000 mgr.x (mgr.24118) 1827 : cluster [DBG] pgmap v2681: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:16.002 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:28:17.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:17 smithi055 bash[9296]: cluster 2020-09-08T16:28:16.482486+0000 mgr.x (mgr.24118) 1828 : cluster [DBG] pgmap v2682: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:17.929 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:17 smithi104 bash[9203]: cluster 2020-09-08T16:28:16.482486+0000 mgr.x (mgr.24118) 1828 : cluster [DBG] pgmap v2682: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:18.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:17 smithi166 bash[9446]: cluster 2020-09-08T16:28:16.482486+0000 mgr.x (mgr.24118) 1828 : cluster [DBG] pgmap v2682: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:19.011 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:28:19.011 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T16:28:19.012 INFO:teuthology.orchestra.run.smithi166.stdout: "complaint_time": 30, 2020-09-08T16:28:19.012 INFO:teuthology.orchestra.run.smithi166.stdout: "num_blocked_ops": 0 2020-09-08T16:28:19.012 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:28:19.686 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.9 dump_historic_ops 2020-09-08T16:28:19.858 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:28:19.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:19 smithi055 bash[9296]: cluster 2020-09-08T16:28:18.483133+0000 mgr.x (mgr.24118) 1829 : cluster [DBG] pgmap v2683: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:19.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:19 smithi055 bash[9296]: cluster 2020-09-08T16:28:19.132390+0000 osd.9 (osd.9) 152 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:19.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:19 smithi055 bash[9296]: cluster 2020-09-08T16:28:19.132532+0000 osd.9 (osd.9) 153 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:19.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:19 smithi104 bash[9203]: cluster 2020-09-08T16:28:18.483133+0000 mgr.x (mgr.24118) 1829 : cluster [DBG] pgmap v2683: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:19.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:19 smithi104 bash[9203]: cluster 2020-09-08T16:28:19.132390+0000 osd.9 (osd.9) 152 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:19.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:19 smithi104 bash[9203]: cluster 2020-09-08T16:28:19.132532+0000 osd.9 (osd.9) 153 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:20.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:19 smithi166 bash[9446]: cluster 2020-09-08T16:28:18.483133+0000 mgr.x (mgr.24118) 1829 : cluster [DBG] pgmap v2683: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:20.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:19 smithi166 bash[9446]: cluster 2020-09-08T16:28:19.132390+0000 osd.9 (osd.9) 152 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:20.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:19 smithi166 bash[9446]: cluster 2020-09-08T16:28:19.132532+0000 osd.9 (osd.9) 153 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:20.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:20 smithi055 bash[9296]: cluster 2020-09-08T16:28:19.658018+0000 mon.a (mon.0) 2598 : cluster [DBG] osdmap e939: 12 total, 12 up, 12 in 2020-09-08T16:28:20.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:20 smithi104 bash[9203]: cluster 2020-09-08T16:28:19.658018+0000 mon.a (mon.0) 2598 : cluster [DBG] osdmap e939: 12 total, 12 up, 12 in 2020-09-08T16:28:20.928 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:20 smithi166 bash[9446]: cluster 2020-09-08T16:28:19.658018+0000 mon.a (mon.0) 2598 : cluster [DBG] osdmap e939: 12 total, 12 up, 12 in 2020-09-08T16:28:21.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:21 smithi055 bash[9296]: cluster 2020-09-08T16:28:20.483748+0000 mgr.x (mgr.24118) 1830 : cluster [DBG] pgmap v2685: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:21.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:21 smithi104 bash[9203]: cluster 2020-09-08T16:28:20.483748+0000 mgr.x (mgr.24118) 1830 : cluster [DBG] pgmap v2685: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:22.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:21 smithi166 bash[9446]: cluster 2020-09-08T16:28:20.483748+0000 mgr.x (mgr.24118) 1830 : cluster [DBG] pgmap v2685: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:22.893 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:28:22.894 INFO:teuthology.orchestra.run.smithi166.stdout: "size": 20, 2020-09-08T16:28:22.894 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 600, 2020-09-08T16:28:22.894 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [] 2020-09-08T16:28:22.894 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:28:22.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:22 smithi104 bash[9203]: cluster 2020-09-08T16:28:21.921305+0000 osd.2 (osd.2) 148 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:22.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:22 smithi104 bash[9203]: cluster 2020-09-08T16:28:21.921425+0000 osd.2 (osd.2) 149 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:22.959 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:22 smithi166 bash[9446]: cluster 2020-09-08T16:28:21.921305+0000 osd.2 (osd.2) 148 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:22.959 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:22 smithi166 bash[9446]: cluster 2020-09-08T16:28:21.921425+0000 osd.2 (osd.2) 149 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:23.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:22 smithi055 bash[9296]: cluster 2020-09-08T16:28:21.921305+0000 osd.2 (osd.2) 148 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:23.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:22 smithi055 bash[9296]: cluster 2020-09-08T16:28:21.921425+0000 osd.2 (osd.2) 149 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:23.666 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2020-09-08T16:28:23.796 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:28:23.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:23 smithi104 bash[9203]: cluster 2020-09-08T16:28:22.484408+0000 mgr.x (mgr.24118) 1831 : cluster [DBG] pgmap v2686: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:23.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:23 smithi104 bash[9203]: cluster 2020-09-08T16:28:22.667770+0000 mon.a (mon.0) 2599 : cluster [DBG] osdmap e940: 12 total, 12 up, 12 in 2020-09-08T16:28:24.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:23 smithi166 bash[9446]: cluster 2020-09-08T16:28:22.484408+0000 mgr.x (mgr.24118) 1831 : cluster [DBG] pgmap v2686: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:24.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:23 smithi166 bash[9446]: cluster 2020-09-08T16:28:22.667770+0000 mon.a (mon.0) 2599 : cluster [DBG] osdmap e940: 12 total, 12 up, 12 in 2020-09-08T16:28:24.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:23 smithi055 bash[9296]: cluster 2020-09-08T16:28:22.484408+0000 mgr.x (mgr.24118) 1831 : cluster [DBG] pgmap v2686: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:24.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:23 smithi055 bash[9296]: cluster 2020-09-08T16:28:22.667770+0000 mon.a (mon.0) 2599 : cluster [DBG] osdmap e940: 12 total, 12 up, 12 in 2020-09-08T16:28:25.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:25 smithi104 bash[9203]: cluster 2020-09-08T16:28:24.484999+0000 mgr.x (mgr.24118) 1832 : cluster [DBG] pgmap v2688: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:26.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:25 smithi166 bash[9446]: cluster 2020-09-08T16:28:24.484999+0000 mgr.x (mgr.24118) 1832 : cluster [DBG] pgmap v2688: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:26.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:25 smithi055 bash[9296]: cluster 2020-09-08T16:28:24.484999+0000 mgr.x (mgr.24118) 1832 : cluster [DBG] pgmap v2688: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:26.777 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:28:26.777 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T16:28:26.778 INFO:teuthology.orchestra.run.smithi166.stdout: "num_ops": 0 2020-09-08T16:28:26.778 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:28:27.511 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2020-09-08T16:28:27.642 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:28:27.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:27 smithi166 bash[9446]: cluster 2020-09-08T16:28:26.485623+0000 mgr.x (mgr.24118) 1833 : cluster [DBG] pgmap v2689: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:28.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:27 smithi055 bash[9296]: cluster 2020-09-08T16:28:26.485623+0000 mgr.x (mgr.24118) 1833 : cluster [DBG] pgmap v2689: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:28.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:27 smithi104 bash[9203]: cluster 2020-09-08T16:28:26.485623+0000 mgr.x (mgr.24118) 1833 : cluster [DBG] pgmap v2689: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:28.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:28 smithi166 bash[9446]: cluster 2020-09-08T16:28:27.059427+0000 osd.7 (osd.7) 206 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:28:28.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:28 smithi166 bash[9446]: cluster 2020-09-08T16:28:27.208273+0000 osd.7 (osd.7) 207 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:28:29.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:28 smithi055 bash[9296]: cluster 2020-09-08T16:28:27.059427+0000 osd.7 (osd.7) 206 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:28:29.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:28 smithi055 bash[9296]: cluster 2020-09-08T16:28:27.208273+0000 osd.7 (osd.7) 207 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:28:29.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:28 smithi104 bash[9203]: cluster 2020-09-08T16:28:27.059427+0000 osd.7 (osd.7) 206 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:28:29.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:28 smithi104 bash[9203]: cluster 2020-09-08T16:28:27.208273+0000 osd.7 (osd.7) 207 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:28:30.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:29 smithi166 bash[9446]: cluster 2020-09-08T16:28:28.486277+0000 mgr.x (mgr.24118) 1834 : cluster [DBG] pgmap v2690: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:30.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:29 smithi166 bash[9446]: cluster 2020-09-08T16:28:29.005034+0000 osd.0 (osd.0) 164 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:30.120 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:29 smithi166 bash[9446]: cluster 2020-09-08T16:28:29.005165+0000 osd.0 (osd.0) 165 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:30.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:29 smithi055 bash[9296]: cluster 2020-09-08T16:28:28.486277+0000 mgr.x (mgr.24118) 1834 : cluster [DBG] pgmap v2690: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:30.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:29 smithi055 bash[9296]: cluster 2020-09-08T16:28:29.005034+0000 osd.0 (osd.0) 164 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:30.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:29 smithi055 bash[9296]: cluster 2020-09-08T16:28:29.005165+0000 osd.0 (osd.0) 165 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:30.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:29 smithi104 bash[9203]: cluster 2020-09-08T16:28:28.486277+0000 mgr.x (mgr.24118) 1834 : cluster [DBG] pgmap v2690: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:30.187 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:29 smithi104 bash[9203]: cluster 2020-09-08T16:28:29.005034+0000 osd.0 (osd.0) 164 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:30.187 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:29 smithi104 bash[9203]: cluster 2020-09-08T16:28:29.005165+0000 osd.0 (osd.0) 165 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:30.698 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:28:30.698 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T16:28:30.698 INFO:teuthology.orchestra.run.smithi166.stdout: "complaint_time": 30, 2020-09-08T16:28:30.699 INFO:teuthology.orchestra.run.smithi166.stdout: "num_blocked_ops": 0 2020-09-08T16:28:30.699 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:28:31.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:30 smithi166 bash[9446]: cluster 2020-09-08T16:28:29.602157+0000 osd.7 (osd.7) 208 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:31.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:30 smithi166 bash[9446]: cluster 2020-09-08T16:28:29.602279+0000 osd.7 (osd.7) 209 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:31.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:30 smithi166 bash[9446]: cluster 2020-09-08T16:28:29.703137+0000 mon.a (mon.0) 2600 : cluster [DBG] osdmap e941: 12 total, 12 up, 12 in 2020-09-08T16:28:31.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:30 smithi055 bash[9296]: cluster 2020-09-08T16:28:29.602157+0000 osd.7 (osd.7) 208 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:31.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:30 smithi055 bash[9296]: cluster 2020-09-08T16:28:29.602279+0000 osd.7 (osd.7) 209 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:31.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:30 smithi055 bash[9296]: cluster 2020-09-08T16:28:29.703137+0000 mon.a (mon.0) 2600 : cluster [DBG] osdmap e941: 12 total, 12 up, 12 in 2020-09-08T16:28:31.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:30 smithi104 bash[9203]: cluster 2020-09-08T16:28:29.602157+0000 osd.7 (osd.7) 208 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:31.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:30 smithi104 bash[9203]: cluster 2020-09-08T16:28:29.602279+0000 osd.7 (osd.7) 209 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:31.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:30 smithi104 bash[9203]: cluster 2020-09-08T16:28:29.703137+0000 mon.a (mon.0) 2600 : cluster [DBG] osdmap e941: 12 total, 12 up, 12 in 2020-09-08T16:28:31.381 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2020-09-08T16:28:31.516 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:28:32.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:31 smithi166 bash[9446]: cluster 2020-09-08T16:28:30.486905+0000 mgr.x (mgr.24118) 1835 : cluster [DBG] pgmap v2692: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:32.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:31 smithi055 bash[9296]: cluster 2020-09-08T16:28:30.486905+0000 mgr.x (mgr.24118) 1835 : cluster [DBG] pgmap v2692: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:32.179 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:31 smithi104 bash[9203]: cluster 2020-09-08T16:28:30.486905+0000 mgr.x (mgr.24118) 1835 : cluster [DBG] pgmap v2692: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:34.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:33 smithi166 bash[9446]: cluster 2020-09-08T16:28:32.487504+0000 mgr.x (mgr.24118) 1836 : cluster [DBG] pgmap v2693: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:34.161 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:33 smithi055 bash[9296]: cluster 2020-09-08T16:28:32.487504+0000 mgr.x (mgr.24118) 1836 : cluster [DBG] pgmap v2693: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:34.181 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:33 smithi104 bash[9203]: cluster 2020-09-08T16:28:32.487504+0000 mgr.x (mgr.24118) 1836 : cluster [DBG] pgmap v2693: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:34.697 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:28:34.697 INFO:teuthology.orchestra.run.smithi166.stdout: "size": 20, 2020-09-08T16:28:34.698 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 600, 2020-09-08T16:28:34.698 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [] 2020-09-08T16:28:34.698 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:28:35.425 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2020-09-08T16:28:35.550 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:28:36.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:35 smithi166 bash[9446]: cluster 2020-09-08T16:28:34.488155+0000 mgr.x (mgr.24118) 1837 : cluster [DBG] pgmap v2694: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:36.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:35 smithi055 bash[9296]: cluster 2020-09-08T16:28:34.488155+0000 mgr.x (mgr.24118) 1837 : cluster [DBG] pgmap v2694: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:36.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:35 smithi104 bash[9203]: cluster 2020-09-08T16:28:34.488155+0000 mgr.x (mgr.24118) 1837 : cluster [DBG] pgmap v2694: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:38.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:37 smithi166 bash[9446]: cluster 2020-09-08T16:28:36.488914+0000 mgr.x (mgr.24118) 1838 : cluster [DBG] pgmap v2695: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:38.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:37 smithi055 bash[9296]: cluster 2020-09-08T16:28:36.488914+0000 mgr.x (mgr.24118) 1838 : cluster [DBG] pgmap v2695: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:38.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:37 smithi104 bash[9203]: cluster 2020-09-08T16:28:36.488914+0000 mgr.x (mgr.24118) 1838 : cluster [DBG] pgmap v2695: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:38.751 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:28:38.752 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T16:28:38.752 INFO:teuthology.orchestra.run.smithi166.stdout: "num_ops": 0 2020-09-08T16:28:38.752 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:28:39.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:38 smithi166 bash[9446]: cluster 2020-09-08T16:28:37.826845+0000 osd.5 (osd.5) 170 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:39.119 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:38 smithi166 bash[9446]: cluster 2020-09-08T16:28:37.826984+0000 osd.5 (osd.5) 171 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:39.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:38 smithi055 bash[9296]: cluster 2020-09-08T16:28:37.826845+0000 osd.5 (osd.5) 170 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:39.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:38 smithi055 bash[9296]: cluster 2020-09-08T16:28:37.826984+0000 osd.5 (osd.5) 171 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:39.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:38 smithi104 bash[9203]: cluster 2020-09-08T16:28:37.826845+0000 osd.5 (osd.5) 170 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:39.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:38 smithi104 bash[9203]: cluster 2020-09-08T16:28:37.826984+0000 osd.5 (osd.5) 171 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:39.463 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2020-09-08T16:28:39.593 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:28:40.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:39 smithi166 bash[9446]: cluster 2020-09-08T16:28:38.489656+0000 mgr.x (mgr.24118) 1839 : cluster [DBG] pgmap v2696: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:40.121 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:39 smithi166 bash[9446]: cluster 2020-09-08T16:28:38.789403+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e942: 12 total, 12 up, 12 in 2020-09-08T16:28:40.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:39 smithi055 bash[9296]: cluster 2020-09-08T16:28:38.489656+0000 mgr.x (mgr.24118) 1839 : cluster [DBG] pgmap v2696: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:40.159 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:39 smithi055 bash[9296]: cluster 2020-09-08T16:28:38.789403+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e942: 12 total, 12 up, 12 in 2020-09-08T16:28:40.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:39 smithi104 bash[9203]: cluster 2020-09-08T16:28:38.489656+0000 mgr.x (mgr.24118) 1839 : cluster [DBG] pgmap v2696: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:40.177 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:39 smithi104 bash[9203]: cluster 2020-09-08T16:28:38.789403+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e942: 12 total, 12 up, 12 in 2020-09-08T16:28:41.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:41 smithi055 bash[9296]: cluster 2020-09-08T16:28:40.490436+0000 mgr.x (mgr.24118) 1840 : cluster [DBG] pgmap v2698: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:41.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:41 smithi055 bash[9296]: cluster 2020-09-08T16:28:40.811321+0000 osd.10 (osd.10) 149 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:41.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:41 smithi055 bash[9296]: cluster 2020-09-08T16:28:40.811438+0000 osd.10 (osd.10) 150 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:41.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:41 smithi055 bash[9296]: cluster 2020-09-08T16:28:41.178251+0000 osd.3 (osd.3) 173 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:41.658 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:41 smithi055 bash[9296]: cluster 2020-09-08T16:28:41.178395+0000 osd.3 (osd.3) 174 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:41.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:41 smithi104 bash[9203]: cluster 2020-09-08T16:28:40.490436+0000 mgr.x (mgr.24118) 1840 : cluster [DBG] pgmap v2698: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:41.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:41 smithi104 bash[9203]: cluster 2020-09-08T16:28:40.811321+0000 osd.10 (osd.10) 149 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:41.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:41 smithi104 bash[9203]: cluster 2020-09-08T16:28:40.811438+0000 osd.10 (osd.10) 150 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:41.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:41 smithi104 bash[9203]: cluster 2020-09-08T16:28:41.178251+0000 osd.3 (osd.3) 173 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:41.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:41 smithi104 bash[9203]: cluster 2020-09-08T16:28:41.178395+0000 osd.3 (osd.3) 174 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:41.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:41 smithi166 bash[9446]: cluster 2020-09-08T16:28:40.490436+0000 mgr.x (mgr.24118) 1840 : cluster [DBG] pgmap v2698: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:41.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:41 smithi166 bash[9446]: cluster 2020-09-08T16:28:40.811321+0000 osd.10 (osd.10) 149 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:41.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:41 smithi166 bash[9446]: cluster 2020-09-08T16:28:40.811438+0000 osd.10 (osd.10) 150 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:41.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:41 smithi166 bash[9446]: cluster 2020-09-08T16:28:41.178251+0000 osd.3 (osd.3) 173 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:28:41.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:41 smithi166 bash[9446]: cluster 2020-09-08T16:28:41.178395+0000 osd.3 (osd.3) 174 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:28:42.657 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:42 smithi055 bash[9296]: cluster 2020-09-08T16:28:41.408692+0000 mon.a (mon.0) 2602 : cluster [DBG] osdmap e943: 12 total, 12 up, 12 in 2020-09-08T16:28:42.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:42 smithi104 bash[9203]: cluster 2020-09-08T16:28:41.408692+0000 mon.a (mon.0) 2602 : cluster [DBG] osdmap e943: 12 total, 12 up, 12 in 2020-09-08T16:28:42.769 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:28:42.770 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [], 2020-09-08T16:28:42.770 INFO:teuthology.orchestra.run.smithi166.stdout: "complaint_time": 30, 2020-09-08T16:28:42.770 INFO:teuthology.orchestra.run.smithi166.stdout: "num_blocked_ops": 0 2020-09-08T16:28:42.770 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:28:42.830 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:42 smithi166 bash[9446]: cluster 2020-09-08T16:28:41.408692+0000 mon.a (mon.0) 2602 : cluster [DBG] osdmap e943: 12 total, 12 up, 12 in 2020-09-08T16:28:43.540 INFO:teuthology.orchestra.run.smithi166:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2020-09-08T16:28:43.674 INFO:teuthology.orchestra.run.smithi166.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.c/config 2020-09-08T16:28:43.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:43 smithi104 bash[9203]: cluster 2020-09-08T16:28:42.491116+0000 mgr.x (mgr.24118) 1841 : cluster [DBG] pgmap v2700: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:43.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:43 smithi166 bash[9446]: cluster 2020-09-08T16:28:42.491116+0000 mgr.x (mgr.24118) 1841 : cluster [DBG] pgmap v2700: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:43.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:43 smithi055 bash[9296]: cluster 2020-09-08T16:28:42.491116+0000 mgr.x (mgr.24118) 1841 : cluster [DBG] pgmap v2700: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:45.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:45 smithi166 bash[9446]: cluster 2020-09-08T16:28:44.491759+0000 mgr.x (mgr.24118) 1842 : cluster [DBG] pgmap v2701: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:45.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:45 smithi055 bash[9296]: cluster 2020-09-08T16:28:44.491759+0000 mgr.x (mgr.24118) 1842 : cluster [DBG] pgmap v2701: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:45.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:45 smithi104 bash[9203]: cluster 2020-09-08T16:28:44.491759+0000 mgr.x (mgr.24118) 1842 : cluster [DBG] pgmap v2701: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:28:46.667 INFO:teuthology.orchestra.run.smithi166.stdout:{ 2020-09-08T16:28:46.667 INFO:teuthology.orchestra.run.smithi166.stdout: "size": 20, 2020-09-08T16:28:46.667 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 600, 2020-09-08T16:28:46.668 INFO:teuthology.orchestra.run.smithi166.stdout: "ops": [ 2020-09-08T16:28:46.668 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.668 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "MOSDScrubReserve(1.0 REQUEST e925)", 2020-09-08T16:28:46.668 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:27:15.210544+0000", 2020-09-08T16:28:46.668 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 91.459196055999996, 2020-09-08T16:28:46.669 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00044862800000000002, 2020-09-08T16:28:46.669 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:28:46.669 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "started", 2020-09-08T16:28:46.669 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:28:46.670 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.670 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:28:46.670 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.210544+0000", 2020-09-08T16:28:46.670 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:28:46.671 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.671 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.671 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:28:46.671 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.210544+0000", 2020-09-08T16:28:46.690 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.6748000000000004e-05 2020-09-08T16:28:46.690 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.691 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.691 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:28:46.691 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.210621+0000", 2020-09-08T16:28:46.692 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.3662999999999999e-05 2020-09-08T16:28:46.692 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.692 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.692 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:28:46.693 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.210635+0000", 2020-09-08T16:28:46.693 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.6869000000000002e-05 2020-09-08T16:28:46.693 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.693 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.693 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:28:46.694 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.210671+0000", 2020-09-08T16:28:46.694 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2627e-05 2020-09-08T16:28:46.694 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.694 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.695 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:28:46.695 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.210684+0000", 2020-09-08T16:28:46.695 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017950299999999999 2020-09-08T16:28:46.695 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.695 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.696 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:28:46.696 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.210864+0000", 2020-09-08T16:28:46.696 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.3850000000000003e-05 2020-09-08T16:28:46.696 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.697 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.697 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:28:46.697 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.210897+0000", 2020-09-08T16:28:46.697 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.5367999999999999e-05 2020-09-08T16:28:46.697 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.698 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.698 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:28:46.699 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.210993+0000", 2020-09-08T16:28:46.699 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00044862800000000002 2020-09-08T16:28:46.699 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:28:46.699 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:28:46.699 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:28:46.700 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.700 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.700 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "replica_scrub(pg: 1.0,from:0'0,to:99'6,epoch:925/90,start:1:00000000::::0,end:MAX,chunky:1,deep:0,version:9,allow_preemption:1,priority=5)", 2020-09-08T16:28:46.700 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:27:15.271034+0000", 2020-09-08T16:28:46.700 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 91.398706505999996, 2020-09-08T16:28:46.701 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000389336, 2020-09-08T16:28:46.701 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:28:46.701 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T16:28:46.701 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:28:46.702 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.703 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:28:46.703 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.271034+0000", 2020-09-08T16:28:46.703 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:28:46.703 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.704 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.704 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:28:46.704 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.271034+0000", 2020-09-08T16:28:46.704 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 8.9529000000000003e-05 2020-09-08T16:28:46.705 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.705 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.705 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:28:46.705 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.271123+0000", 2020-09-08T16:28:46.706 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6582000000000001e-05 2020-09-08T16:28:46.706 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.706 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.706 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:28:46.707 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.271140+0000", 2020-09-08T16:28:46.708 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.7027000000000003e-05 2020-09-08T16:28:46.708 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.708 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.708 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:28:46.709 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.271177+0000", 2020-09-08T16:28:46.709 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6622000000000001e-05 2020-09-08T16:28:46.709 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.709 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.710 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:28:46.710 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.271193+0000", 2020-09-08T16:28:46.710 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00017168700000000001 2020-09-08T16:28:46.710 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.711 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.711 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:28:46.711 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.271365+0000", 2020-09-08T16:28:46.712 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.7889000000000001e-05 2020-09-08T16:28:46.712 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.712 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.712 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:28:46.713 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.271423+0000", 2020-09-08T16:28:46.713 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000389336 2020-09-08T16:28:46.713 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:28:46.713 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:28:46.714 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:28:46.714 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.714 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.715 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "MOSDScrubReserve(1.0 RELEASE e925)", 2020-09-08T16:28:46.715 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:27:15.358564+0000", 2020-09-08T16:28:46.715 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 91.311176544000006, 2020-09-08T16:28:46.715 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018283899999999999, 2020-09-08T16:28:46.715 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:28:46.716 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "started", 2020-09-08T16:28:46.716 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:28:46.716 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.716 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:28:46.717 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.358564+0000", 2020-09-08T16:28:46.718 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:28:46.718 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.718 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.718 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:28:46.718 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.358564+0000", 2020-09-08T16:28:46.719 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.3095000000000004e-05 2020-09-08T16:28:46.719 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.719 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.720 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:28:46.720 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.358637+0000", 2020-09-08T16:28:46.720 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.401e-05 2020-09-08T16:28:46.720 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.721 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.721 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:28:46.721 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.358651+0000", 2020-09-08T16:28:46.721 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.4411999999999999e-05 2020-09-08T16:28:46.722 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:28:46.722 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:28:46.722 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:28:46.722 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.358675+0000", 2020-09-08T16:28:46.722 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2103000000000001e-05 2020-09-08T16:29:50.853 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.855 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.855 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:29:50.855 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.358687+0000", 2020-09-08T16:29:50.855 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.5461999999999999e-05 2020-09-08T16:29:50.856 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.856 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.856 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:29:50.856 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.358733+0000", 2020-09-08T16:29:50.857 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.0427e-05 2020-09-08T16:29:50.857 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.857 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.857 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:29:50.857 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.358743+0000", 2020-09-08T16:29:50.858 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.3299999999999999e-06 2020-09-08T16:29:50.858 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.858 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.858 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:29:50.858 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:27:15.358746+0000", 2020-09-08T16:29:50.859 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00018283899999999999 2020-09-08T16:29:50.859 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:29:50.859 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:29:50.859 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:29:50.859 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.860 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.860 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "MOSDScrubReserve(1.0 REQUEST e940)", 2020-09-08T16:29:50.860 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:28:27.056690+0000", 2020-09-08T16:29:50.860 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 19.613050295000001, 2020-09-08T16:29:50.860 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00050914700000000005, 2020-09-08T16:29:50.861 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:29:50.861 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "started", 2020-09-08T16:29:50.861 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:29:50.861 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.862 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:29:50.862 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.056690+0000", 2020-09-08T16:29:50.862 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:29:50.862 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.862 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.863 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:29:50.863 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.056690+0000", 2020-09-08T16:29:50.863 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.1441999999999996e-05 2020-09-08T16:29:50.863 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.863 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.864 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:29:50.864 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.056781+0000", 2020-09-08T16:29:50.864 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.8598999999999999e-05 2020-09-08T16:29:50.864 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.864 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.865 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:29:50.865 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.056800+0000", 2020-09-08T16:29:50.865 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.845e-05 2020-09-08T16:29:50.865 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.865 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.866 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:29:50.866 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.056828+0000", 2020-09-08T16:29:50.866 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.6833e-05 2020-09-08T16:29:50.866 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.866 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.867 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:29:50.867 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.056845+0000", 2020-09-08T16:29:50.867 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000160607 2020-09-08T16:29:50.867 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.868 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.868 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:29:50.868 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.057006+0000", 2020-09-08T16:29:50.868 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.0559e-05 2020-09-08T16:29:50.868 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.869 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.869 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:29:50.869 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.057076+0000", 2020-09-08T16:29:50.869 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000122657 2020-09-08T16:29:50.869 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.870 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.870 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:29:50.870 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.057199+0000", 2020-09-08T16:29:50.870 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00050914700000000005 2020-09-08T16:29:50.870 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:29:50.871 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:29:50.871 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:29:50.871 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.871 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.871 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "replica_scrub(pg: 1.0,from:0'0,to:99'6,epoch:940/90,start:1:00000000::::0,end:MAX,chunky:1,deep:0,version:9,allow_preemption:1,priority=5)", 2020-09-08T16:29:50.872 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:28:27.120482+0000", 2020-09-08T16:29:50.872 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 19.549258635000001, 2020-09-08T16:29:50.872 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00040617600000000001, 2020-09-08T16:29:50.872 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:29:50.872 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "reached pg", 2020-09-08T16:29:50.873 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:29:50.873 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.873 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:29:50.873 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.120482+0000", 2020-09-08T16:29:50.873 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:29:50.874 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.874 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.874 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:29:50.874 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.120482+0000", 2020-09-08T16:29:50.874 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 9.0092000000000004e-05 2020-09-08T16:29:50.875 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.875 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.875 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:29:50.875 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.120572+0000", 2020-09-08T16:29:50.875 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7672999999999998e-05 2020-09-08T16:29:50.876 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.876 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.876 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:29:50.876 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.120589+0000", 2020-09-08T16:29:50.876 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 4.2412e-05 2020-09-08T16:29:50.877 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.877 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.877 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:29:50.877 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.120632+0000", 2020-09-08T16:29:50.877 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.7969999999999999e-05 2020-09-08T16:29:50.877 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.878 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.878 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:29:50.878 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.120650+0000", 2020-09-08T16:29:50.878 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000177947 2020-09-08T16:29:50.878 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.879 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.879 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:29:50.879 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.120828+0000", 2020-09-08T16:29:50.879 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 6.0081999999999998e-05 2020-09-08T16:29:50.879 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.880 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.880 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:29:50.880 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.120888+0000", 2020-09-08T16:29:50.880 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.00040617600000000001 2020-09-08T16:29:50.880 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:29:50.881 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:29:50.881 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:29:50.881 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.881 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.881 INFO:teuthology.orchestra.run.smithi166.stdout: "description": "MOSDScrubReserve(1.0 RELEASE e940)", 2020-09-08T16:29:50.882 INFO:teuthology.orchestra.run.smithi166.stdout: "initiated_at": "2020-09-08T16:28:27.207723+0000", 2020-09-08T16:29:50.882 INFO:teuthology.orchestra.run.smithi166.stdout: "age": 19.462017065000001, 2020-09-08T16:29:50.882 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000198159, 2020-09-08T16:29:50.882 INFO:teuthology.orchestra.run.smithi166.stdout: "type_data": { 2020-09-08T16:29:50.882 INFO:teuthology.orchestra.run.smithi166.stdout: "flag_point": "started", 2020-09-08T16:29:50.882 INFO:teuthology.orchestra.run.smithi166.stdout: "events": [ 2020-09-08T16:29:50.883 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.883 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "initiated", 2020-09-08T16:29:50.883 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.207723+0000", 2020-09-08T16:29:50.883 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0 2020-09-08T16:29:50.883 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.884 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.884 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "throttled", 2020-09-08T16:29:50.884 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.207723+0000", 2020-09-08T16:29:50.884 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 7.7748000000000002e-05 2020-09-08T16:29:50.884 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.885 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.885 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "header_read", 2020-09-08T16:29:50.885 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.207801+0000", 2020-09-08T16:29:50.885 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.1403000000000001e-05 2020-09-08T16:29:50.885 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.886 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.886 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "all_read", 2020-09-08T16:29:50.886 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.207812+0000", 2020-09-08T16:29:50.886 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.8677000000000001e-05 2020-09-08T16:29:50.886 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.887 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.887 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "dispatched", 2020-09-08T16:29:50.887 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.207831+0000", 2020-09-08T16:29:50.887 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 1.2206e-05 2020-09-08T16:29:50.887 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.888 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.888 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "queued_for_pg", 2020-09-08T16:29:50.888 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.207843+0000", 2020-09-08T16:29:50.888 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 5.3062999999999997e-05 2020-09-08T16:29:50.888 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.888 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.889 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "reached_pg", 2020-09-08T16:29:50.889 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.207896+0000", 2020-09-08T16:29:50.889 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 2.2010000000000001e-05 2020-09-08T16:29:50.889 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.889 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.890 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "started", 2020-09-08T16:29:50.890 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.207918+0000", 2020-09-08T16:29:50.890 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 3.0520000000000002e-06 2020-09-08T16:29:50.890 INFO:teuthology.orchestra.run.smithi166.stdout: }, 2020-09-08T16:29:50.890 INFO:teuthology.orchestra.run.smithi166.stdout: { 2020-09-08T16:29:50.891 INFO:teuthology.orchestra.run.smithi166.stdout: "event": "done", 2020-09-08T16:29:50.891 INFO:teuthology.orchestra.run.smithi166.stdout: "time": "2020-09-08T16:28:27.207921+0000", 2020-09-08T16:29:50.891 INFO:teuthology.orchestra.run.smithi166.stdout: "duration": 0.000198159 2020-09-08T16:29:50.891 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:29:50.891 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:29:50.892 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:29:50.892 INFO:teuthology.orchestra.run.smithi166.stdout: } 2020-09-08T16:29:50.892 INFO:teuthology.orchestra.run.smithi166.stdout: ] 2020-09-08T16:29:50.892 INFO:teuthology.orchestra.run.smithi166.stdout:} 2020-09-08T16:29:50.908 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:47 smithi104 bash[9203]: cluster 2020-09-08T16:28:46.492456+0000 mgr.x (mgr.24118) 1843 : cluster [DBG] pgmap v2702: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.908 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:49 smithi104 bash[9203]: cluster 2020-09-08T16:28:48.493100+0000 mgr.x (mgr.24118) 1844 : cluster [DBG] pgmap v2703: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.908 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:51 smithi104 bash[9203]: cluster 2020-09-08T16:28:50.493748+0000 mgr.x (mgr.24118) 1845 : cluster [DBG] pgmap v2704: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.908 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:51 smithi104 bash[9203]: cluster 2020-09-08T16:28:50.511399+0000 osd.8 (osd.8) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.909 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:51 smithi104 bash[9203]: cluster 2020-09-08T16:28:50.511510+0000 osd.8 (osd.8) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.909 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:52 smithi104 bash[9203]: cluster 2020-09-08T16:28:51.420161+0000 mon.a (mon.0) 2603 : cluster [DBG] osdmap e944: 12 total, 12 up, 12 in 2020-09-08T16:29:50.909 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:53 smithi104 bash[9203]: cluster 2020-09-08T16:28:52.494501+0000 mgr.x (mgr.24118) 1846 : cluster [DBG] pgmap v2706: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.909 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:55 smithi104 bash[9203]: cluster 2020-09-08T16:28:54.495212+0000 mgr.x (mgr.24118) 1847 : cluster [DBG] pgmap v2707: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.910 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:57 smithi104 bash[9203]: cluster 2020-09-08T16:28:56.495811+0000 mgr.x (mgr.24118) 1848 : cluster [DBG] pgmap v2708: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.910 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:57 smithi104 bash[9203]: cluster 2020-09-08T16:28:56.896816+0000 osd.1 (osd.1) 190 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.910 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:57 smithi104 bash[9203]: cluster 2020-09-08T16:28:56.896957+0000 osd.1 (osd.1) 191 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.910 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:58 smithi104 bash[9203]: cluster 2020-09-08T16:28:57.570731+0000 mon.a (mon.0) 2604 : cluster [DBG] osdmap e945: 12 total, 12 up, 12 in 2020-09-08T16:29:50.910 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:28:59 smithi104 bash[9203]: cluster 2020-09-08T16:28:58.496448+0000 mgr.x (mgr.24118) 1849 : cluster [DBG] pgmap v2710: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.911 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:01 smithi104 bash[9203]: cluster 2020-09-08T16:29:00.497096+0000 mgr.x (mgr.24118) 1850 : cluster [DBG] pgmap v2711: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.911 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:02 smithi104 bash[9203]: cluster 2020-09-08T16:29:01.461499+0000 osd.6 (osd.6) 155 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.911 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:02 smithi104 bash[9203]: cluster 2020-09-08T16:29:01.461641+0000 osd.6 (osd.6) 156 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.911 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:03 smithi104 bash[9203]: cluster 2020-09-08T16:29:02.415880+0000 mon.a (mon.0) 2605 : cluster [DBG] osdmap e946: 12 total, 12 up, 12 in 2020-09-08T16:29:50.911 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:03 smithi104 bash[9203]: cluster 2020-09-08T16:29:02.497712+0000 mgr.x (mgr.24118) 1851 : cluster [DBG] pgmap v2713: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.912 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:05 smithi104 bash[9203]: cluster 2020-09-08T16:29:04.498374+0000 mgr.x (mgr.24118) 1852 : cluster [DBG] pgmap v2714: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.912 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:07 smithi104 bash[9203]: cluster 2020-09-08T16:29:06.499035+0000 mgr.x (mgr.24118) 1853 : cluster [DBG] pgmap v2715: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.912 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:08 smithi104 bash[9203]: cluster 2020-09-08T16:29:07.793963+0000 osd.11 (osd.11) 170 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.912 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:08 smithi104 bash[9203]: cluster 2020-09-08T16:29:07.794097+0000 osd.11 (osd.11) 171 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.912 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:09 smithi104 bash[9203]: cluster 2020-09-08T16:29:08.499691+0000 mgr.x (mgr.24118) 1854 : cluster [DBG] pgmap v2716: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.913 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:09 smithi104 bash[9203]: cluster 2020-09-08T16:29:08.579896+0000 mon.a (mon.0) 2606 : cluster [DBG] osdmap e947: 12 total, 12 up, 12 in 2020-09-08T16:29:50.913 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:11 smithi104 bash[9203]: cluster 2020-09-08T16:29:10.415662+0000 osd.4 (osd.4) 115 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.913 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:11 smithi104 bash[9203]: cluster 2020-09-08T16:29:10.415788+0000 osd.4 (osd.4) 116 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.913 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:11 smithi104 bash[9203]: cluster 2020-09-08T16:29:10.500283+0000 mgr.x (mgr.24118) 1855 : cluster [DBG] pgmap v2718: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.913 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:12 smithi104 bash[9203]: cluster 2020-09-08T16:29:11.420670+0000 mon.a (mon.0) 2607 : cluster [DBG] osdmap e948: 12 total, 12 up, 12 in 2020-09-08T16:29:50.914 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:13 smithi104 bash[9203]: cluster 2020-09-08T16:29:12.500918+0000 mgr.x (mgr.24118) 1856 : cluster [DBG] pgmap v2720: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.914 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:15 smithi104 bash[9203]: cluster 2020-09-08T16:29:14.501524+0000 mgr.x (mgr.24118) 1857 : cluster [DBG] pgmap v2721: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.914 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:16 smithi104 bash[9203]: audit 2020-09-08T16:29:15.772129+0000 mon.c (mon.2) 100 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:29:50.914 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:16 smithi104 bash[9203]: audit 2020-09-08T16:29:15.775797+0000 mon.a (mon.0) 2608 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:29:50.914 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:16 smithi104 bash[9203]: audit 2020-09-08T16:29:15.782110+0000 mon.a (mon.0) 2609 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:29:50.915 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:17 smithi104 bash[9203]: cluster 2020-09-08T16:29:16.502163+0000 mgr.x (mgr.24118) 1858 : cluster [DBG] pgmap v2722: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.915 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:19 smithi104 bash[9203]: cluster 2020-09-08T16:29:18.502798+0000 mgr.x (mgr.24118) 1859 : cluster [DBG] pgmap v2723: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.915 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:20 smithi104 bash[9203]: cluster 2020-09-08T16:29:19.455774+0000 osd.9 (osd.9) 154 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.915 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:20 smithi104 bash[9203]: cluster 2020-09-08T16:29:19.455915+0000 osd.9 (osd.9) 155 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.915 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:20 smithi104 bash[9203]: cluster 2020-09-08T16:29:19.792312+0000 mon.a (mon.0) 2610 : cluster [DBG] osdmap e949: 12 total, 12 up, 12 in 2020-09-08T16:29:50.916 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:21 smithi104 bash[9203]: cluster 2020-09-08T16:29:20.503365+0000 mgr.x (mgr.24118) 1860 : cluster [DBG] pgmap v2725: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.916 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:22 smithi104 bash[9203]: cluster 2020-09-08T16:29:21.922542+0000 osd.2 (osd.2) 150 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.916 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:22 smithi104 bash[9203]: cluster 2020-09-08T16:29:21.922669+0000 osd.2 (osd.2) 151 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.916 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:23 smithi104 bash[9203]: cluster 2020-09-08T16:29:22.503992+0000 mgr.x (mgr.24118) 1861 : cluster [DBG] pgmap v2726: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.916 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:23 smithi104 bash[9203]: cluster 2020-09-08T16:29:22.823295+0000 mon.a (mon.0) 2611 : cluster [DBG] osdmap e950: 12 total, 12 up, 12 in 2020-09-08T16:29:50.917 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:25 smithi104 bash[9203]: cluster 2020-09-08T16:29:24.504583+0000 mgr.x (mgr.24118) 1862 : cluster [DBG] pgmap v2728: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.917 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:27 smithi104 bash[9203]: cluster 2020-09-08T16:29:26.505209+0000 mgr.x (mgr.24118) 1863 : cluster [DBG] pgmap v2729: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.917 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:29 smithi104 bash[9203]: cluster 2020-09-08T16:29:28.505878+0000 mgr.x (mgr.24118) 1864 : cluster [DBG] pgmap v2730: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.917 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:29 smithi104 bash[9203]: cluster 2020-09-08T16:29:29.748375+0000 osd.7 (osd.7) 210 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.917 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:29 smithi104 bash[9203]: cluster 2020-09-08T16:29:29.748521+0000 osd.7 (osd.7) 211 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.918 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:30 smithi104 bash[9203]: cluster 2020-09-08T16:29:29.759029+0000 osd.0 (osd.0) 166 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.918 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:30 smithi104 bash[9203]: cluster 2020-09-08T16:29:29.759176+0000 osd.0 (osd.0) 167 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.918 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:30 smithi104 bash[9203]: cluster 2020-09-08T16:29:29.838346+0000 mon.a (mon.0) 2612 : cluster [DBG] osdmap e951: 12 total, 12 up, 12 in 2020-09-08T16:29:50.918 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:31 smithi104 bash[9203]: cluster 2020-09-08T16:29:30.506543+0000 mgr.x (mgr.24118) 1865 : cluster [DBG] pgmap v2732: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.918 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:33 smithi104 bash[9203]: cluster 2020-09-08T16:29:32.507217+0000 mgr.x (mgr.24118) 1866 : cluster [DBG] pgmap v2733: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.919 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:35 smithi104 bash[9203]: cluster 2020-09-08T16:29:34.507859+0000 mgr.x (mgr.24118) 1867 : cluster [DBG] pgmap v2734: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.919 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:37 smithi104 bash[9203]: cluster 2020-09-08T16:29:36.508520+0000 mgr.x (mgr.24118) 1868 : cluster [DBG] pgmap v2735: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.919 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:39 smithi104 bash[9203]: cluster 2020-09-08T16:29:38.509177+0000 mgr.x (mgr.24118) 1869 : cluster [DBG] pgmap v2736: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.919 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:39 smithi104 bash[9203]: cluster 2020-09-08T16:29:38.672326+0000 osd.5 (osd.5) 172 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.919 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:39 smithi104 bash[9203]: cluster 2020-09-08T16:29:38.672469+0000 osd.5 (osd.5) 173 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.920 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:39 smithi104 bash[9203]: cluster 2020-09-08T16:29:38.927219+0000 mon.a (mon.0) 2613 : cluster [DBG] osdmap e952: 12 total, 12 up, 12 in 2020-09-08T16:29:50.920 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:41 smithi104 bash[9203]: cluster 2020-09-08T16:29:40.509759+0000 mgr.x (mgr.24118) 1870 : cluster [DBG] pgmap v2738: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.920 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:42 smithi104 bash[9203]: cluster 2020-09-08T16:29:41.549584+0000 osd.10 (osd.10) 151 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.920 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:42 smithi104 bash[9203]: cluster 2020-09-08T16:29:41.549710+0000 osd.10 (osd.10) 152 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.920 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:42 smithi104 bash[9203]: cluster 2020-09-08T16:29:41.609052+0000 osd.3 (osd.3) 175 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.921 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:42 smithi104 bash[9203]: cluster 2020-09-08T16:29:41.609174+0000 osd.3 (osd.3) 176 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.921 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:43 smithi104 bash[9203]: cluster 2020-09-08T16:29:42.437850+0000 mon.a (mon.0) 2614 : cluster [DBG] osdmap e953: 12 total, 12 up, 12 in 2020-09-08T16:29:50.921 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:43 smithi104 bash[9203]: cluster 2020-09-08T16:29:42.510348+0000 mgr.x (mgr.24118) 1871 : cluster [DBG] pgmap v2740: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.921 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:45 smithi104 bash[9203]: cluster 2020-09-08T16:29:44.510889+0000 mgr.x (mgr.24118) 1872 : cluster [DBG] pgmap v2741: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.921 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:47 smithi104 bash[9203]: cluster 2020-09-08T16:29:46.511510+0000 mgr.x (mgr.24118) 1873 : cluster [DBG] pgmap v2742: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.922 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:49 smithi104 bash[9203]: cluster 2020-09-08T16:29:48.512145+0000 mgr.x (mgr.24118) 1874 : cluster [DBG] pgmap v2743: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.922 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:47 smithi055 bash[9296]: cluster 2020-09-08T16:28:46.492456+0000 mgr.x (mgr.24118) 1843 : cluster [DBG] pgmap v2702: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.922 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:49 smithi055 bash[9296]: cluster 2020-09-08T16:28:48.493100+0000 mgr.x (mgr.24118) 1844 : cluster [DBG] pgmap v2703: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.922 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:51 smithi055 bash[9296]: cluster 2020-09-08T16:28:50.493748+0000 mgr.x (mgr.24118) 1845 : cluster [DBG] pgmap v2704: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.923 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:51 smithi055 bash[9296]: cluster 2020-09-08T16:28:50.511399+0000 osd.8 (osd.8) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.923 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:51 smithi055 bash[9296]: cluster 2020-09-08T16:28:50.511510+0000 osd.8 (osd.8) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.923 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:52 smithi055 bash[9296]: cluster 2020-09-08T16:28:51.420161+0000 mon.a (mon.0) 2603 : cluster [DBG] osdmap e944: 12 total, 12 up, 12 in 2020-09-08T16:29:50.923 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:53 smithi055 bash[9296]: cluster 2020-09-08T16:28:52.494501+0000 mgr.x (mgr.24118) 1846 : cluster [DBG] pgmap v2706: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.923 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:55 smithi055 bash[9296]: cluster 2020-09-08T16:28:54.495212+0000 mgr.x (mgr.24118) 1847 : cluster [DBG] pgmap v2707: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.924 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:57 smithi055 bash[9296]: cluster 2020-09-08T16:28:56.495811+0000 mgr.x (mgr.24118) 1848 : cluster [DBG] pgmap v2708: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.924 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:57 smithi055 bash[9296]: cluster 2020-09-08T16:28:56.896816+0000 osd.1 (osd.1) 190 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.924 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:57 smithi055 bash[9296]: cluster 2020-09-08T16:28:56.896957+0000 osd.1 (osd.1) 191 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.924 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:58 smithi055 bash[9296]: cluster 2020-09-08T16:28:57.570731+0000 mon.a (mon.0) 2604 : cluster [DBG] osdmap e945: 12 total, 12 up, 12 in 2020-09-08T16:29:50.924 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:28:59 smithi055 bash[9296]: cluster 2020-09-08T16:28:58.496448+0000 mgr.x (mgr.24118) 1849 : cluster [DBG] pgmap v2710: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.925 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:01 smithi055 bash[9296]: cluster 2020-09-08T16:29:00.497096+0000 mgr.x (mgr.24118) 1850 : cluster [DBG] pgmap v2711: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.925 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:02 smithi055 bash[9296]: cluster 2020-09-08T16:29:01.461499+0000 osd.6 (osd.6) 155 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.925 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:02 smithi055 bash[9296]: cluster 2020-09-08T16:29:01.461641+0000 osd.6 (osd.6) 156 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.925 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:03 smithi055 bash[9296]: cluster 2020-09-08T16:29:02.415880+0000 mon.a (mon.0) 2605 : cluster [DBG] osdmap e946: 12 total, 12 up, 12 in 2020-09-08T16:29:50.925 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:03 smithi055 bash[9296]: cluster 2020-09-08T16:29:02.497712+0000 mgr.x (mgr.24118) 1851 : cluster [DBG] pgmap v2713: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.926 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:05 smithi055 bash[9296]: cluster 2020-09-08T16:29:04.498374+0000 mgr.x (mgr.24118) 1852 : cluster [DBG] pgmap v2714: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.926 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:07 smithi055 bash[9296]: cluster 2020-09-08T16:29:06.499035+0000 mgr.x (mgr.24118) 1853 : cluster [DBG] pgmap v2715: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.926 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:08 smithi055 bash[9296]: cluster 2020-09-08T16:29:07.793963+0000 osd.11 (osd.11) 170 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.926 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:08 smithi055 bash[9296]: cluster 2020-09-08T16:29:07.794097+0000 osd.11 (osd.11) 171 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.926 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:09 smithi055 bash[9296]: cluster 2020-09-08T16:29:08.499691+0000 mgr.x (mgr.24118) 1854 : cluster [DBG] pgmap v2716: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.927 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:09 smithi055 bash[9296]: cluster 2020-09-08T16:29:08.579896+0000 mon.a (mon.0) 2606 : cluster [DBG] osdmap e947: 12 total, 12 up, 12 in 2020-09-08T16:29:50.927 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:11 smithi055 bash[9296]: cluster 2020-09-08T16:29:10.415662+0000 osd.4 (osd.4) 115 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.927 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:11 smithi055 bash[9296]: cluster 2020-09-08T16:29:10.415788+0000 osd.4 (osd.4) 116 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.927 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:11 smithi055 bash[9296]: cluster 2020-09-08T16:29:10.500283+0000 mgr.x (mgr.24118) 1855 : cluster [DBG] pgmap v2718: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.927 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:12 smithi055 bash[9296]: cluster 2020-09-08T16:29:11.420670+0000 mon.a (mon.0) 2607 : cluster [DBG] osdmap e948: 12 total, 12 up, 12 in 2020-09-08T16:29:50.928 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:13 smithi055 bash[9296]: cluster 2020-09-08T16:29:12.500918+0000 mgr.x (mgr.24118) 1856 : cluster [DBG] pgmap v2720: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.928 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:15 smithi055 bash[9296]: cluster 2020-09-08T16:29:14.501524+0000 mgr.x (mgr.24118) 1857 : cluster [DBG] pgmap v2721: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.928 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:16 smithi055 bash[9296]: audit 2020-09-08T16:29:15.772129+0000 mon.c (mon.2) 100 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:29:50.928 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:16 smithi055 bash[9296]: audit 2020-09-08T16:29:15.775797+0000 mon.a (mon.0) 2608 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:29:50.928 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:16 smithi055 bash[9296]: audit 2020-09-08T16:29:15.782110+0000 mon.a (mon.0) 2609 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:29:50.929 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:17 smithi055 bash[9296]: cluster 2020-09-08T16:29:16.502163+0000 mgr.x (mgr.24118) 1858 : cluster [DBG] pgmap v2722: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.929 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:19 smithi055 bash[9296]: cluster 2020-09-08T16:29:18.502798+0000 mgr.x (mgr.24118) 1859 : cluster [DBG] pgmap v2723: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.929 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:20 smithi055 bash[9296]: cluster 2020-09-08T16:29:19.455774+0000 osd.9 (osd.9) 154 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.929 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:20 smithi055 bash[9296]: cluster 2020-09-08T16:29:19.455915+0000 osd.9 (osd.9) 155 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.929 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:20 smithi055 bash[9296]: cluster 2020-09-08T16:29:19.792312+0000 mon.a (mon.0) 2610 : cluster [DBG] osdmap e949: 12 total, 12 up, 12 in 2020-09-08T16:29:50.930 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:21 smithi055 bash[9296]: cluster 2020-09-08T16:29:20.503365+0000 mgr.x (mgr.24118) 1860 : cluster [DBG] pgmap v2725: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.930 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:22 smithi055 bash[9296]: cluster 2020-09-08T16:29:21.922542+0000 osd.2 (osd.2) 150 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.930 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:22 smithi055 bash[9296]: cluster 2020-09-08T16:29:21.922669+0000 osd.2 (osd.2) 151 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.930 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:23 smithi055 bash[9296]: cluster 2020-09-08T16:29:22.503992+0000 mgr.x (mgr.24118) 1861 : cluster [DBG] pgmap v2726: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.930 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:23 smithi055 bash[9296]: cluster 2020-09-08T16:29:22.823295+0000 mon.a (mon.0) 2611 : cluster [DBG] osdmap e950: 12 total, 12 up, 12 in 2020-09-08T16:29:50.931 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:25 smithi055 bash[9296]: cluster 2020-09-08T16:29:24.504583+0000 mgr.x (mgr.24118) 1862 : cluster [DBG] pgmap v2728: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.931 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:27 smithi055 bash[9296]: cluster 2020-09-08T16:29:26.505209+0000 mgr.x (mgr.24118) 1863 : cluster [DBG] pgmap v2729: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.931 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:29 smithi055 bash[9296]: cluster 2020-09-08T16:29:28.505878+0000 mgr.x (mgr.24118) 1864 : cluster [DBG] pgmap v2730: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.931 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:29 smithi055 bash[9296]: cluster 2020-09-08T16:29:29.748375+0000 osd.7 (osd.7) 210 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.931 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:29 smithi055 bash[9296]: cluster 2020-09-08T16:29:29.748521+0000 osd.7 (osd.7) 211 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.932 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:30 smithi055 bash[9296]: cluster 2020-09-08T16:29:29.759029+0000 osd.0 (osd.0) 166 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.932 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:30 smithi055 bash[9296]: cluster 2020-09-08T16:29:29.759176+0000 osd.0 (osd.0) 167 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.932 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:30 smithi055 bash[9296]: cluster 2020-09-08T16:29:29.838346+0000 mon.a (mon.0) 2612 : cluster [DBG] osdmap e951: 12 total, 12 up, 12 in 2020-09-08T16:29:50.932 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:31 smithi055 bash[9296]: cluster 2020-09-08T16:29:30.506543+0000 mgr.x (mgr.24118) 1865 : cluster [DBG] pgmap v2732: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.932 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:33 smithi055 bash[9296]: cluster 2020-09-08T16:29:32.507217+0000 mgr.x (mgr.24118) 1866 : cluster [DBG] pgmap v2733: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.933 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:35 smithi055 bash[9296]: cluster 2020-09-08T16:29:34.507859+0000 mgr.x (mgr.24118) 1867 : cluster [DBG] pgmap v2734: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.933 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:37 smithi055 bash[9296]: cluster 2020-09-08T16:29:36.508520+0000 mgr.x (mgr.24118) 1868 : cluster [DBG] pgmap v2735: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.933 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:39 smithi055 bash[9296]: cluster 2020-09-08T16:29:38.509177+0000 mgr.x (mgr.24118) 1869 : cluster [DBG] pgmap v2736: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.933 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:39 smithi055 bash[9296]: cluster 2020-09-08T16:29:38.672326+0000 osd.5 (osd.5) 172 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.933 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:39 smithi055 bash[9296]: cluster 2020-09-08T16:29:38.672469+0000 osd.5 (osd.5) 173 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.934 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:39 smithi055 bash[9296]: cluster 2020-09-08T16:29:38.927219+0000 mon.a (mon.0) 2613 : cluster [DBG] osdmap e952: 12 total, 12 up, 12 in 2020-09-08T16:29:50.934 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:41 smithi055 bash[9296]: cluster 2020-09-08T16:29:40.509759+0000 mgr.x (mgr.24118) 1870 : cluster [DBG] pgmap v2738: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.934 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:42 smithi055 bash[9296]: cluster 2020-09-08T16:29:41.549584+0000 osd.10 (osd.10) 151 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.934 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:42 smithi055 bash[9296]: cluster 2020-09-08T16:29:41.549710+0000 osd.10 (osd.10) 152 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.934 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:42 smithi055 bash[9296]: cluster 2020-09-08T16:29:41.609052+0000 osd.3 (osd.3) 175 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.935 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:42 smithi055 bash[9296]: cluster 2020-09-08T16:29:41.609174+0000 osd.3 (osd.3) 176 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.935 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:43 smithi055 bash[9296]: cluster 2020-09-08T16:29:42.437850+0000 mon.a (mon.0) 2614 : cluster [DBG] osdmap e953: 12 total, 12 up, 12 in 2020-09-08T16:29:50.935 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:43 smithi055 bash[9296]: cluster 2020-09-08T16:29:42.510348+0000 mgr.x (mgr.24118) 1871 : cluster [DBG] pgmap v2740: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.935 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:45 smithi055 bash[9296]: cluster 2020-09-08T16:29:44.510889+0000 mgr.x (mgr.24118) 1872 : cluster [DBG] pgmap v2741: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.935 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:47 smithi055 bash[9296]: cluster 2020-09-08T16:29:46.511510+0000 mgr.x (mgr.24118) 1873 : cluster [DBG] pgmap v2742: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.936 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:49 smithi055 bash[9296]: cluster 2020-09-08T16:29:48.512145+0000 mgr.x (mgr.24118) 1874 : cluster [DBG] pgmap v2743: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.936 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:47 smithi166 bash[9446]: cluster 2020-09-08T16:28:46.492456+0000 mgr.x (mgr.24118) 1843 : cluster [DBG] pgmap v2702: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.937 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:49 smithi166 bash[9446]: cluster 2020-09-08T16:28:48.493100+0000 mgr.x (mgr.24118) 1844 : cluster [DBG] pgmap v2703: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.937 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:51 smithi166 bash[9446]: cluster 2020-09-08T16:28:50.493748+0000 mgr.x (mgr.24118) 1845 : cluster [DBG] pgmap v2704: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.937 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:51 smithi166 bash[9446]: cluster 2020-09-08T16:28:50.511399+0000 osd.8 (osd.8) 144 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.937 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:51 smithi166 bash[9446]: cluster 2020-09-08T16:28:50.511510+0000 osd.8 (osd.8) 145 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.937 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:52 smithi166 bash[9446]: cluster 2020-09-08T16:28:51.420161+0000 mon.a (mon.0) 2603 : cluster [DBG] osdmap e944: 12 total, 12 up, 12 in 2020-09-08T16:29:50.938 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:53 smithi166 bash[9446]: cluster 2020-09-08T16:28:52.494501+0000 mgr.x (mgr.24118) 1846 : cluster [DBG] pgmap v2706: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.938 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:55 smithi166 bash[9446]: cluster 2020-09-08T16:28:54.495212+0000 mgr.x (mgr.24118) 1847 : cluster [DBG] pgmap v2707: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.938 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:57 smithi166 bash[9446]: cluster 2020-09-08T16:28:56.495811+0000 mgr.x (mgr.24118) 1848 : cluster [DBG] pgmap v2708: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.938 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:57 smithi166 bash[9446]: cluster 2020-09-08T16:28:56.896816+0000 osd.1 (osd.1) 190 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.938 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:57 smithi166 bash[9446]: cluster 2020-09-08T16:28:56.896957+0000 osd.1 (osd.1) 191 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.939 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:58 smithi166 bash[9446]: cluster 2020-09-08T16:28:57.570731+0000 mon.a (mon.0) 2604 : cluster [DBG] osdmap e945: 12 total, 12 up, 12 in 2020-09-08T16:29:50.939 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:28:59 smithi166 bash[9446]: cluster 2020-09-08T16:28:58.496448+0000 mgr.x (mgr.24118) 1849 : cluster [DBG] pgmap v2710: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.939 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:01 smithi166 bash[9446]: cluster 2020-09-08T16:29:00.497096+0000 mgr.x (mgr.24118) 1850 : cluster [DBG] pgmap v2711: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.939 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:02 smithi166 bash[9446]: cluster 2020-09-08T16:29:01.461499+0000 osd.6 (osd.6) 155 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.939 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:02 smithi166 bash[9446]: cluster 2020-09-08T16:29:01.461641+0000 osd.6 (osd.6) 156 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.940 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:03 smithi166 bash[9446]: cluster 2020-09-08T16:29:02.415880+0000 mon.a (mon.0) 2605 : cluster [DBG] osdmap e946: 12 total, 12 up, 12 in 2020-09-08T16:29:50.940 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:03 smithi166 bash[9446]: cluster 2020-09-08T16:29:02.497712+0000 mgr.x (mgr.24118) 1851 : cluster [DBG] pgmap v2713: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.940 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:05 smithi166 bash[9446]: cluster 2020-09-08T16:29:04.498374+0000 mgr.x (mgr.24118) 1852 : cluster [DBG] pgmap v2714: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.940 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:07 smithi166 bash[9446]: cluster 2020-09-08T16:29:06.499035+0000 mgr.x (mgr.24118) 1853 : cluster [DBG] pgmap v2715: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.940 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:08 smithi166 bash[9446]: cluster 2020-09-08T16:29:07.793963+0000 osd.11 (osd.11) 170 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.941 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:08 smithi166 bash[9446]: cluster 2020-09-08T16:29:07.794097+0000 osd.11 (osd.11) 171 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.941 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:09 smithi166 bash[9446]: cluster 2020-09-08T16:29:08.499691+0000 mgr.x (mgr.24118) 1854 : cluster [DBG] pgmap v2716: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.941 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:09 smithi166 bash[9446]: cluster 2020-09-08T16:29:08.579896+0000 mon.a (mon.0) 2606 : cluster [DBG] osdmap e947: 12 total, 12 up, 12 in 2020-09-08T16:29:50.941 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:11 smithi166 bash[9446]: cluster 2020-09-08T16:29:10.415662+0000 osd.4 (osd.4) 115 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.941 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:11 smithi166 bash[9446]: cluster 2020-09-08T16:29:10.415788+0000 osd.4 (osd.4) 116 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.942 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:11 smithi166 bash[9446]: cluster 2020-09-08T16:29:10.500283+0000 mgr.x (mgr.24118) 1855 : cluster [DBG] pgmap v2718: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.942 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:12 smithi166 bash[9446]: cluster 2020-09-08T16:29:11.420670+0000 mon.a (mon.0) 2607 : cluster [DBG] osdmap e948: 12 total, 12 up, 12 in 2020-09-08T16:29:50.942 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:13 smithi166 bash[9446]: cluster 2020-09-08T16:29:12.500918+0000 mgr.x (mgr.24118) 1856 : cluster [DBG] pgmap v2720: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.942 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:15 smithi166 bash[9446]: cluster 2020-09-08T16:29:14.501524+0000 mgr.x (mgr.24118) 1857 : cluster [DBG] pgmap v2721: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.942 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:16 smithi166 bash[9446]: audit 2020-09-08T16:29:15.772129+0000 mon.c (mon.2) 100 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:29:50.943 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:16 smithi166 bash[9446]: audit 2020-09-08T16:29:15.775797+0000 mon.a (mon.0) 2608 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:29:50.943 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:16 smithi166 bash[9446]: audit 2020-09-08T16:29:15.782110+0000 mon.a (mon.0) 2609 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:29:50.943 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:17 smithi166 bash[9446]: cluster 2020-09-08T16:29:16.502163+0000 mgr.x (mgr.24118) 1858 : cluster [DBG] pgmap v2722: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.943 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:19 smithi166 bash[9446]: cluster 2020-09-08T16:29:18.502798+0000 mgr.x (mgr.24118) 1859 : cluster [DBG] pgmap v2723: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.943 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:20 smithi166 bash[9446]: cluster 2020-09-08T16:29:19.455774+0000 osd.9 (osd.9) 154 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.944 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:20 smithi166 bash[9446]: cluster 2020-09-08T16:29:19.455915+0000 osd.9 (osd.9) 155 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.944 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:20 smithi166 bash[9446]: cluster 2020-09-08T16:29:19.792312+0000 mon.a (mon.0) 2610 : cluster [DBG] osdmap e949: 12 total, 12 up, 12 in 2020-09-08T16:29:50.944 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:21 smithi166 bash[9446]: cluster 2020-09-08T16:29:20.503365+0000 mgr.x (mgr.24118) 1860 : cluster [DBG] pgmap v2725: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.944 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:22 smithi166 bash[9446]: cluster 2020-09-08T16:29:21.922542+0000 osd.2 (osd.2) 150 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.944 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:22 smithi166 bash[9446]: cluster 2020-09-08T16:29:21.922669+0000 osd.2 (osd.2) 151 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.945 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:23 smithi166 bash[9446]: cluster 2020-09-08T16:29:22.503992+0000 mgr.x (mgr.24118) 1861 : cluster [DBG] pgmap v2726: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.945 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:23 smithi166 bash[9446]: cluster 2020-09-08T16:29:22.823295+0000 mon.a (mon.0) 2611 : cluster [DBG] osdmap e950: 12 total, 12 up, 12 in 2020-09-08T16:29:50.945 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:25 smithi166 bash[9446]: cluster 2020-09-08T16:29:24.504583+0000 mgr.x (mgr.24118) 1862 : cluster [DBG] pgmap v2728: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.945 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:27 smithi166 bash[9446]: cluster 2020-09-08T16:29:26.505209+0000 mgr.x (mgr.24118) 1863 : cluster [DBG] pgmap v2729: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.945 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:29 smithi166 bash[9446]: cluster 2020-09-08T16:29:28.505878+0000 mgr.x (mgr.24118) 1864 : cluster [DBG] pgmap v2730: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.946 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:29 smithi166 bash[9446]: cluster 2020-09-08T16:29:29.748375+0000 osd.7 (osd.7) 210 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.946 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:29 smithi166 bash[9446]: cluster 2020-09-08T16:29:29.748521+0000 osd.7 (osd.7) 211 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.946 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:30 smithi166 bash[9446]: cluster 2020-09-08T16:29:29.759029+0000 osd.0 (osd.0) 166 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.946 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:30 smithi166 bash[9446]: cluster 2020-09-08T16:29:29.759176+0000 osd.0 (osd.0) 167 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.946 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:30 smithi166 bash[9446]: cluster 2020-09-08T16:29:29.838346+0000 mon.a (mon.0) 2612 : cluster [DBG] osdmap e951: 12 total, 12 up, 12 in 2020-09-08T16:29:50.947 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:31 smithi166 bash[9446]: cluster 2020-09-08T16:29:30.506543+0000 mgr.x (mgr.24118) 1865 : cluster [DBG] pgmap v2732: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.947 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:33 smithi166 bash[9446]: cluster 2020-09-08T16:29:32.507217+0000 mgr.x (mgr.24118) 1866 : cluster [DBG] pgmap v2733: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.947 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:35 smithi166 bash[9446]: cluster 2020-09-08T16:29:34.507859+0000 mgr.x (mgr.24118) 1867 : cluster [DBG] pgmap v2734: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.947 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:37 smithi166 bash[9446]: cluster 2020-09-08T16:29:36.508520+0000 mgr.x (mgr.24118) 1868 : cluster [DBG] pgmap v2735: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.947 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:39 smithi166 bash[9446]: cluster 2020-09-08T16:29:38.509177+0000 mgr.x (mgr.24118) 1869 : cluster [DBG] pgmap v2736: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.948 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:39 smithi166 bash[9446]: cluster 2020-09-08T16:29:38.672326+0000 osd.5 (osd.5) 172 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.948 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:39 smithi166 bash[9446]: cluster 2020-09-08T16:29:38.672469+0000 osd.5 (osd.5) 173 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.948 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:39 smithi166 bash[9446]: cluster 2020-09-08T16:29:38.927219+0000 mon.a (mon.0) 2613 : cluster [DBG] osdmap e952: 12 total, 12 up, 12 in 2020-09-08T16:29:50.948 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:41 smithi166 bash[9446]: cluster 2020-09-08T16:29:40.509759+0000 mgr.x (mgr.24118) 1870 : cluster [DBG] pgmap v2738: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.948 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:42 smithi166 bash[9446]: cluster 2020-09-08T16:29:41.549584+0000 osd.10 (osd.10) 151 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.949 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:42 smithi166 bash[9446]: cluster 2020-09-08T16:29:41.549710+0000 osd.10 (osd.10) 152 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.949 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:42 smithi166 bash[9446]: cluster 2020-09-08T16:29:41.609052+0000 osd.3 (osd.3) 175 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:50.949 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:42 smithi166 bash[9446]: cluster 2020-09-08T16:29:41.609174+0000 osd.3 (osd.3) 176 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:50.949 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:43 smithi166 bash[9446]: cluster 2020-09-08T16:29:42.437850+0000 mon.a (mon.0) 2614 : cluster [DBG] osdmap e953: 12 total, 12 up, 12 in 2020-09-08T16:29:50.949 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:43 smithi166 bash[9446]: cluster 2020-09-08T16:29:42.510348+0000 mgr.x (mgr.24118) 1871 : cluster [DBG] pgmap v2740: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.950 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:45 smithi166 bash[9446]: cluster 2020-09-08T16:29:44.510889+0000 mgr.x (mgr.24118) 1872 : cluster [DBG] pgmap v2741: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.950 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:47 smithi166 bash[9446]: cluster 2020-09-08T16:29:46.511510+0000 mgr.x (mgr.24118) 1873 : cluster [DBG] pgmap v2742: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.950 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:49 smithi166 bash[9446]: cluster 2020-09-08T16:29:48.512145+0000 mgr.x (mgr.24118) 1874 : cluster [DBG] pgmap v2743: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:50.951 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2020-09-08T16:29:50.952 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2020-09-08T16:29:50.953 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd dump --format=json 2020-09-08T16:29:51.086 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:29:51.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:51 smithi104 bash[9203]: cluster 2020-09-08T16:29:50.512734+0000 mgr.x (mgr.24118) 1875 : cluster [DBG] pgmap v2744: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:51.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:51 smithi166 bash[9446]: cluster 2020-09-08T16:29:50.512734+0000 mgr.x (mgr.24118) 1875 : cluster [DBG] pgmap v2744: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:51.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:51 smithi055 bash[9296]: cluster 2020-09-08T16:29:50.512734+0000 mgr.x (mgr.24118) 1875 : cluster [DBG] pgmap v2744: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:52.440 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:52 smithi104 bash[9203]: cluster 2020-09-08T16:29:51.498429+0000 osd.8 (osd.8) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:52.440 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:52 smithi104 bash[9203]: cluster 2020-09-08T16:29:51.498560+0000 osd.8 (osd.8) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:52.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:52 smithi166 bash[9446]: cluster 2020-09-08T16:29:51.498429+0000 osd.8 (osd.8) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:52.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:52 smithi166 bash[9446]: cluster 2020-09-08T16:29:51.498560+0000 osd.8 (osd.8) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:52.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:52 smithi055 bash[9296]: cluster 2020-09-08T16:29:51.498429+0000 osd.8 (osd.8) 146 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:52.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:52 smithi055 bash[9296]: cluster 2020-09-08T16:29:51.498560+0000 osd.8 (osd.8) 147 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:53.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:53 smithi166 bash[9446]: cluster 2020-09-08T16:29:52.446330+0000 mon.a (mon.0) 2615 : cluster [DBG] osdmap e954: 12 total, 12 up, 12 in 2020-09-08T16:29:53.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:53 smithi166 bash[9446]: cluster 2020-09-08T16:29:52.513317+0000 mgr.x (mgr.24118) 1876 : cluster [DBG] pgmap v2746: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:53.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:53 smithi055 bash[9296]: cluster 2020-09-08T16:29:52.446330+0000 mon.a (mon.0) 2615 : cluster [DBG] osdmap e954: 12 total, 12 up, 12 in 2020-09-08T16:29:53.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:53 smithi055 bash[9296]: cluster 2020-09-08T16:29:52.513317+0000 mgr.x (mgr.24118) 1876 : cluster [DBG] pgmap v2746: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:53.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:53 smithi104 bash[9203]: cluster 2020-09-08T16:29:52.446330+0000 mon.a (mon.0) 2615 : cluster [DBG] osdmap e954: 12 total, 12 up, 12 in 2020-09-08T16:29:53.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:53 smithi104 bash[9203]: cluster 2020-09-08T16:29:52.513317+0000 mgr.x (mgr.24118) 1876 : cluster [DBG] pgmap v2746: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:54.360 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T16:29:54.361 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":954,"fsid":"759b27b6-f1e7-11ea-a080-001a4aab830c","created":"2020-09-08T15:25:41.788365+0000","modified":"2020-09-08T16:29:52.432108+0000","last_up_change":"2020-09-08T15:34:41.591462+0000","last_in_change":"2020-09-08T15:43:22.692468+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-09-08T15:25:56.314385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"e49a7341-7c75-4883-b061-3e8fe9467e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":627,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6801","nonce":82470352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":82470352},{"type":"v1","addr":"172.21.15.104:6803","nonce":82470352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":82470352},{"type":"v1","addr":"172.21.15.104:6807","nonce":82470352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":82470352},{"type":"v1","addr":"172.21.15.104:6805","nonce":82470352}]},"public_addr":"172.21.15.104:6801/82470352","cluster_addr":"172.21.15.104:6803/82470352","heartbeat_back_addr":"172.21.15.104:6807/82470352","heartbeat_front_addr":"172.21.15.104:6805/82470352","state":["exists","up"]},{"osd":1,"uuid":"a83df412-110d-440a-b411-ad2a7c36f81a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6809","nonce":1421798992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1421798992},{"type":"v1","addr":"172.21.15.104:6811","nonce":1421798992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1421798992},{"type":"v1","addr":"172.21.15.104:6815","nonce":1421798992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1421798992},{"type":"v1","addr":"172.21.15.104:6813","nonce":1421798992}]},"public_addr":"172.21.15.104:6809/1421798992","cluster_addr":"172.21.15.104:6811/1421798992","heartbeat_back_addr":"172.21.15.104:6815/1421798992","heartbeat_front_addr":"172.21.15.104:6813/1421798992","state":["exists","up"]},{"osd":2,"uuid":"ca6da1cc-1d8d-494b-93e6-e065ff70d434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":553,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6817","nonce":2713413369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2713413369},{"type":"v1","addr":"172.21.15.104:6819","nonce":2713413369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2713413369},{"type":"v1","addr":"172.21.15.104:6823","nonce":2713413369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2713413369},{"type":"v1","addr":"172.21.15.104:6821","nonce":2713413369}]},"public_addr":"172.21.15.104:6817/2713413369","cluster_addr":"172.21.15.104:6819/2713413369","heartbeat_back_addr":"172.21.15.104:6823/2713413369","heartbeat_front_addr":"172.21.15.104:6821/2713413369","state":["exists","up"]},{"osd":3,"uuid":"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6825","nonce":1495946248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1495946248},{"type":"v1","addr":"172.21.15.104:6827","nonce":1495946248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1495946248},{"type":"v1","addr":"172.21.15.104:6831","nonce":1495946248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1495946248},{"type":"v1","addr":"172.21.15.104:6829","nonce":1495946248}]},"public_addr":"172.21.15.104:6825/1495946248","cluster_addr":"172.21.15.104:6827/1495946248","heartbeat_back_addr":"172.21.15.104:6831/1495946248","heartbeat_front_addr":"172.21.15.104:6829/1495946248","state":["exists","up"]},{"osd":4,"uuid":"d8334c70-f776-4652-944d-aed35a8e43f7","up":1,"in":1,"weight":1,"primary_affinity":0,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6803","nonce":1173103260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":1173103260},{"type":"v1","addr":"172.21.15.55:6805","nonce":1173103260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":1173103260},{"type":"v1","addr":"172.21.15.55:6809","nonce":1173103260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":1173103260},{"type":"v1","addr":"172.21.15.55:6807","nonce":1173103260}]},"public_addr":"172.21.15.55:6803/1173103260","cluster_addr":"172.21.15.55:6805/1173103260","heartbeat_back_addr":"172.21.15.55:6809/1173103260","heartbeat_front_addr":"172.21.15.55:6807/1173103260","state":["exists","up"]},{"osd":5,"uuid":"6789abd2-9e9b-4c04-81fb-0ce2d2849e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6811","nonce":1047046376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":1047046376},{"type":"v1","addr":"172.21.15.55:6813","nonce":1047046376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":1047046376},{"type":"v1","addr":"172.21.15.55:6817","nonce":1047046376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":1047046376},{"type":"v1","addr":"172.21.15.55:6815","nonce":1047046376}]},"public_addr":"172.21.15.55:6811/1047046376","cluster_addr":"172.21.15.55:6813/1047046376","heartbeat_back_addr":"172.21.15.55:6817/1047046376","heartbeat_front_addr":"172.21.15.55:6815/1047046376","state":["exists","up"]},{"osd":6,"uuid":"ad9a93fe-0ca0-4052-bfa9-e2abb268829e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":553,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6819","nonce":2501844793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":2501844793},{"type":"v1","addr":"172.21.15.55:6821","nonce":2501844793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2501844793},{"type":"v1","addr":"172.21.15.55:6825","nonce":2501844793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":2501844793},{"type":"v1","addr":"172.21.15.55:6823","nonce":2501844793}]},"public_addr":"172.21.15.55:6819/2501844793","cluster_addr":"172.21.15.55:6821/2501844793","heartbeat_back_addr":"172.21.15.55:6825/2501844793","heartbeat_front_addr":"172.21.15.55:6823/2501844793","state":["exists","up"]},{"osd":7,"uuid":"f3795c86-4540-442c-a151-b2769fa577e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6827","nonce":128909153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":128909153},{"type":"v1","addr":"172.21.15.55:6829","nonce":128909153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":128909153},{"type":"v1","addr":"172.21.15.55:6833","nonce":128909153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":128909153},{"type":"v1","addr":"172.21.15.55:6831","nonce":128909153}]},"public_addr":"172.21.15.55:6827/128909153","cluster_addr":"172.21.15.55:6829/128909153","heartbeat_back_addr":"172.21.15.55:6833/128909153","heartbeat_front_addr":"172.21.15.55:6831/128909153","state":["exists","up"]},{"osd":8,"uuid":"3b0ac380-878e-4c1c-9278-80ce67ca2df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6801","nonce":2108117574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6802","nonce":2108117574},{"type":"v1","addr":"172.21.15.166:6803","nonce":2108117574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6806","nonce":2108117574},{"type":"v1","addr":"172.21.15.166:6807","nonce":2108117574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6804","nonce":2108117574},{"type":"v1","addr":"172.21.15.166:6805","nonce":2108117574}]},"public_addr":"172.21.15.166:6801/2108117574","cluster_addr":"172.21.15.166:6803/2108117574","heartbeat_back_addr":"172.21.15.166:6807/2108117574","heartbeat_front_addr":"172.21.15.166:6805/2108117574","state":["exists","up"]},{"osd":9,"uuid":"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6809","nonce":1458870080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6810","nonce":1458870080},{"type":"v1","addr":"172.21.15.166:6811","nonce":1458870080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6814","nonce":1458870080},{"type":"v1","addr":"172.21.15.166:6815","nonce":1458870080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6812","nonce":1458870080},{"type":"v1","addr":"172.21.15.166:6813","nonce":1458870080}]},"public_addr":"172.21.15.166:6809/1458870080","cluster_addr":"172.21.15.166:6811/1458870080","heartbeat_back_addr":"172.21.15.166:6815/1458870080","heartbeat_front_addr":"172.21.15.166:6813/1458870080","state":["exists","up"]},{"osd":10,"uuid":"c194210e-bd89-4a1f-98fb-6bf44f0b324c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6817","nonce":62517848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6818","nonce":62517848},{"type":"v1","addr":"172.21.15.166:6819","nonce":62517848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6822","nonce":62517848},{"type":"v1","addr":"172.21.15.166:6823","nonce":62517848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6820","nonce":62517848},{"type":"v1","addr":"172.21.15.166:6821","nonce":62517848}]},"public_addr":"172.21.15.166:6817/62517848","cluster_addr":"172.21.15.166:6819/62517848","heartbeat_back_addr":"172.21.15.166:6823/62517848","heartbeat_front_addr":"172.21.15.166:6821/62517848","state":["exists","up"]},{"osd":11,"uuid":"121301fe-a41c-40c0-ad09-b00de37a55e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":90,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6825","nonce":691735125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6826","nonce":691735125},{"type":"v1","addr":"172.21.15.166:6827","nonce":691735125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6830","nonce":691735125},{"type":"v1","addr":"172.21.15.166:6831","nonce":691735125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6828","nonce":691735125},{"type":"v1","addr":"172.21.15.166:6829","nonce":691735125}]},"public_addr":"172.21.15.166:6825/691735125","cluster_addr":"172.21.15.166:6827/691735125","heartbeat_back_addr":"172.21.15.166:6831/691735125","heartbeat_front_addr":"172.21.15.166:6829/691735125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:29.759183+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:28:56.896962+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:21.922677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:41.609178+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:10.415793+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:38.672474+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:01.461646+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:29.748528+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:51.498565+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:19.455925+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:41.549718+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:07.794102+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.104:0/1257365091":"2020-09-09T15:29:39.430456+0000","172.21.15.104:0/1399123455":"2020-09-09T15:29:39.430456+0000","172.21.15.104:0/2810441605":"2020-09-09T15:29:39.430456+0000","172.21.15.104:6800/182241824":"2020-09-09T15:29:39.430456+0000","172.21.15.104:6801/182241824":"2020-09-09T15:29:39.430456+0000","172.21.15.104:0/2021933044":"2020-09-09T15:26:33.248197+0000","172.21.15.104:6800/3129754347":"2020-09-09T15:26:33.248197+0000","172.21.15.104:0/1270031705":"2020-09-09T15:26:33.248197+0000","172.21.15.104:0/2568432190":"2020-09-09T15:26:33.248197+0000","172.21.15.104:6800/1138921844":"2020-09-09T15:26:09.867236+0000","172.21.15.104:0/3827328474":"2020-09-09T15:26:09.867236+0000","172.21.15.104:6801/1138921844":"2020-09-09T15:26:09.867236+0000","172.21.15.104:0/1662035381":"2020-09-09T15:26:09.867236+0000","172.21.15.104:6801/3129754347":"2020-09-09T15:26:33.248197+0000","172.21.15.104:0/2169968102":"2020-09-09T15:26:09.867236+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-09-08T16:29:54.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:54 smithi166 bash[9446]: audit 2020-09-08T16:29:54.364377+0000 mon.a (mon.0) 2616 : audit [DBG] from='client.? v1:172.21.15.104:0/2518402637' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2020-09-08T16:29:54.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:54 smithi055 bash[9296]: audit 2020-09-08T16:29:54.364377+0000 mon.a (mon.0) 2616 : audit [DBG] from='client.? v1:172.21.15.104:0/2518402637' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2020-09-08T16:29:54.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:54 smithi104 bash[9203]: audit 2020-09-08T16:29:54.364377+0000 mon.a (mon.0) 2616 : audit [DBG] from='client.? v1:172.21.15.104:0/2518402637' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2020-09-08T16:29:55.146 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2020-09-08T16:29:55.147 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd dump --format=json 2020-09-08T16:29:55.275 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:29:55.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:55 smithi166 bash[9446]: cluster 2020-09-08T16:29:54.513966+0000 mgr.x (mgr.24118) 1877 : cluster [DBG] pgmap v2747: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:55.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:55 smithi055 bash[9296]: cluster 2020-09-08T16:29:54.513966+0000 mgr.x (mgr.24118) 1877 : cluster [DBG] pgmap v2747: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:55.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:55 smithi104 bash[9203]: cluster 2020-09-08T16:29:54.513966+0000 mgr.x (mgr.24118) 1877 : cluster [DBG] pgmap v2747: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:57.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:57 smithi166 bash[9446]: cluster 2020-09-08T16:29:56.514638+0000 mgr.x (mgr.24118) 1878 : cluster [DBG] pgmap v2748: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:57.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:57 smithi055 bash[9296]: cluster 2020-09-08T16:29:56.514638+0000 mgr.x (mgr.24118) 1878 : cluster [DBG] pgmap v2748: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:57.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:57 smithi104 bash[9203]: cluster 2020-09-08T16:29:56.514638+0000 mgr.x (mgr.24118) 1878 : cluster [DBG] pgmap v2748: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:58.583 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T16:29:58.584 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":955,"fsid":"759b27b6-f1e7-11ea-a080-001a4aab830c","created":"2020-09-08T15:25:41.788365+0000","modified":"2020-09-08T16:29:57.570604+0000","last_up_change":"2020-09-08T15:34:41.591462+0000","last_in_change":"2020-09-08T15:43:22.692468+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-09-08T15:25:56.314385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"e49a7341-7c75-4883-b061-3e8fe9467e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":627,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6801","nonce":82470352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":82470352},{"type":"v1","addr":"172.21.15.104:6803","nonce":82470352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":82470352},{"type":"v1","addr":"172.21.15.104:6807","nonce":82470352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":82470352},{"type":"v1","addr":"172.21.15.104:6805","nonce":82470352}]},"public_addr":"172.21.15.104:6801/82470352","cluster_addr":"172.21.15.104:6803/82470352","heartbeat_back_addr":"172.21.15.104:6807/82470352","heartbeat_front_addr":"172.21.15.104:6805/82470352","state":["exists","up"]},{"osd":1,"uuid":"a83df412-110d-440a-b411-ad2a7c36f81a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6809","nonce":1421798992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1421798992},{"type":"v1","addr":"172.21.15.104:6811","nonce":1421798992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1421798992},{"type":"v1","addr":"172.21.15.104:6815","nonce":1421798992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1421798992},{"type":"v1","addr":"172.21.15.104:6813","nonce":1421798992}]},"public_addr":"172.21.15.104:6809/1421798992","cluster_addr":"172.21.15.104:6811/1421798992","heartbeat_back_addr":"172.21.15.104:6815/1421798992","heartbeat_front_addr":"172.21.15.104:6813/1421798992","state":["exists","up"]},{"osd":2,"uuid":"ca6da1cc-1d8d-494b-93e6-e065ff70d434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":553,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6817","nonce":2713413369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2713413369},{"type":"v1","addr":"172.21.15.104:6819","nonce":2713413369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2713413369},{"type":"v1","addr":"172.21.15.104:6823","nonce":2713413369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2713413369},{"type":"v1","addr":"172.21.15.104:6821","nonce":2713413369}]},"public_addr":"172.21.15.104:6817/2713413369","cluster_addr":"172.21.15.104:6819/2713413369","heartbeat_back_addr":"172.21.15.104:6823/2713413369","heartbeat_front_addr":"172.21.15.104:6821/2713413369","state":["exists","up"]},{"osd":3,"uuid":"e11c74bc-a5b2-4e9e-af7d-6c58c1464f6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6825","nonce":1495946248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1495946248},{"type":"v1","addr":"172.21.15.104:6827","nonce":1495946248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1495946248},{"type":"v1","addr":"172.21.15.104:6831","nonce":1495946248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1495946248},{"type":"v1","addr":"172.21.15.104:6829","nonce":1495946248}]},"public_addr":"172.21.15.104:6825/1495946248","cluster_addr":"172.21.15.104:6827/1495946248","heartbeat_back_addr":"172.21.15.104:6831/1495946248","heartbeat_front_addr":"172.21.15.104:6829/1495946248","state":["exists","up"]},{"osd":4,"uuid":"d8334c70-f776-4652-944d-aed35a8e43f7","up":1,"in":1,"weight":1,"primary_affinity":0,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6803","nonce":1173103260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":1173103260},{"type":"v1","addr":"172.21.15.55:6805","nonce":1173103260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":1173103260},{"type":"v1","addr":"172.21.15.55:6809","nonce":1173103260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":1173103260},{"type":"v1","addr":"172.21.15.55:6807","nonce":1173103260}]},"public_addr":"172.21.15.55:6803/1173103260","cluster_addr":"172.21.15.55:6805/1173103260","heartbeat_back_addr":"172.21.15.55:6809/1173103260","heartbeat_front_addr":"172.21.15.55:6807/1173103260","state":["exists","up"]},{"osd":5,"uuid":"6789abd2-9e9b-4c04-81fb-0ce2d2849e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6811","nonce":1047046376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":1047046376},{"type":"v1","addr":"172.21.15.55:6813","nonce":1047046376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":1047046376},{"type":"v1","addr":"172.21.15.55:6817","nonce":1047046376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":1047046376},{"type":"v1","addr":"172.21.15.55:6815","nonce":1047046376}]},"public_addr":"172.21.15.55:6811/1047046376","cluster_addr":"172.21.15.55:6813/1047046376","heartbeat_back_addr":"172.21.15.55:6817/1047046376","heartbeat_front_addr":"172.21.15.55:6815/1047046376","state":["exists","up"]},{"osd":6,"uuid":"ad9a93fe-0ca0-4052-bfa9-e2abb268829e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":553,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6819","nonce":2501844793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":2501844793},{"type":"v1","addr":"172.21.15.55:6821","nonce":2501844793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2501844793},{"type":"v1","addr":"172.21.15.55:6825","nonce":2501844793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":2501844793},{"type":"v1","addr":"172.21.15.55:6823","nonce":2501844793}]},"public_addr":"172.21.15.55:6819/2501844793","cluster_addr":"172.21.15.55:6821/2501844793","heartbeat_back_addr":"172.21.15.55:6825/2501844793","heartbeat_front_addr":"172.21.15.55:6823/2501844793","state":["exists","up"]},{"osd":7,"uuid":"f3795c86-4540-442c-a151-b2769fa577e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.55:6827","nonce":128909153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":128909153},{"type":"v1","addr":"172.21.15.55:6829","nonce":128909153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":128909153},{"type":"v1","addr":"172.21.15.55:6833","nonce":128909153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":128909153},{"type":"v1","addr":"172.21.15.55:6831","nonce":128909153}]},"public_addr":"172.21.15.55:6827/128909153","cluster_addr":"172.21.15.55:6829/128909153","heartbeat_back_addr":"172.21.15.55:6833/128909153","heartbeat_front_addr":"172.21.15.55:6831/128909153","state":["exists","up"]},{"osd":8,"uuid":"3b0ac380-878e-4c1c-9278-80ce67ca2df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6801","nonce":2108117574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6802","nonce":2108117574},{"type":"v1","addr":"172.21.15.166:6803","nonce":2108117574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6806","nonce":2108117574},{"type":"v1","addr":"172.21.15.166:6807","nonce":2108117574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6804","nonce":2108117574},{"type":"v1","addr":"172.21.15.166:6805","nonce":2108117574}]},"public_addr":"172.21.15.166:6801/2108117574","cluster_addr":"172.21.15.166:6803/2108117574","heartbeat_back_addr":"172.21.15.166:6807/2108117574","heartbeat_front_addr":"172.21.15.166:6805/2108117574","state":["exists","up"]},{"osd":9,"uuid":"ad6b8d4d-a8ba-4b51-a3b0-6f9c7638ca08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6809","nonce":1458870080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6810","nonce":1458870080},{"type":"v1","addr":"172.21.15.166:6811","nonce":1458870080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6814","nonce":1458870080},{"type":"v1","addr":"172.21.15.166:6815","nonce":1458870080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6812","nonce":1458870080},{"type":"v1","addr":"172.21.15.166:6813","nonce":1458870080}]},"public_addr":"172.21.15.166:6809/1458870080","cluster_addr":"172.21.15.166:6811/1458870080","heartbeat_back_addr":"172.21.15.166:6815/1458870080","heartbeat_front_addr":"172.21.15.166:6813/1458870080","state":["exists","up"]},{"osd":10,"uuid":"c194210e-bd89-4a1f-98fb-6bf44f0b324c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6817","nonce":62517848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6818","nonce":62517848},{"type":"v1","addr":"172.21.15.166:6819","nonce":62517848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6822","nonce":62517848},{"type":"v1","addr":"172.21.15.166:6823","nonce":62517848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6820","nonce":62517848},{"type":"v1","addr":"172.21.15.166:6821","nonce":62517848}]},"public_addr":"172.21.15.166:6817/62517848","cluster_addr":"172.21.15.166:6819/62517848","heartbeat_back_addr":"172.21.15.166:6823/62517848","heartbeat_front_addr":"172.21.15.166:6821/62517848","state":["exists","up"]},{"osd":11,"uuid":"121301fe-a41c-40c0-ad09-b00de37a55e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":90,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.166:6825","nonce":691735125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6826","nonce":691735125},{"type":"v1","addr":"172.21.15.166:6827","nonce":691735125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6830","nonce":691735125},{"type":"v1","addr":"172.21.15.166:6831","nonce":691735125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.166:6828","nonce":691735125},{"type":"v1","addr":"172.21.15.166:6829","nonce":691735125}]},"public_addr":"172.21.15.166:6825/691735125","cluster_addr":"172.21.15.166:6827/691735125","heartbeat_back_addr":"172.21.15.166:6831/691735125","heartbeat_front_addr":"172.21.15.166:6829/691735125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:29.759183+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:57.061155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:21.922677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:41.609178+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:10.415793+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:38.672474+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:01.461646+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:29.748528+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:51.498565+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:19.455925+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:41.549718+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-09-08T16:29:07.794102+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.104:0/1257365091":"2020-09-09T15:29:39.430456+0000","172.21.15.104:0/1399123455":"2020-09-09T15:29:39.430456+0000","172.21.15.104:0/2810441605":"2020-09-09T15:29:39.430456+0000","172.21.15.104:6800/182241824":"2020-09-09T15:29:39.430456+0000","172.21.15.104:6801/182241824":"2020-09-09T15:29:39.430456+0000","172.21.15.104:0/2021933044":"2020-09-09T15:26:33.248197+0000","172.21.15.104:6800/3129754347":"2020-09-09T15:26:33.248197+0000","172.21.15.104:0/1270031705":"2020-09-09T15:26:33.248197+0000","172.21.15.104:0/2568432190":"2020-09-09T15:26:33.248197+0000","172.21.15.104:6800/1138921844":"2020-09-09T15:26:09.867236+0000","172.21.15.104:0/3827328474":"2020-09-09T15:26:09.867236+0000","172.21.15.104:6801/1138921844":"2020-09-09T15:26:09.867236+0000","172.21.15.104:0/1662035381":"2020-09-09T15:26:09.867236+0000","172.21.15.104:6801/3129754347":"2020-09-09T15:26:33.248197+0000","172.21.15.104:0/2169968102":"2020-09-09T15:26:09.867236+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-09-08T16:29:58.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:58 smithi166 bash[9446]: cluster 2020-09-08T16:29:57.061010+0000 osd.1 (osd.1) 192 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:58.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:58 smithi166 bash[9446]: cluster 2020-09-08T16:29:57.061149+0000 osd.1 (osd.1) 193 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:58.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:58 smithi166 bash[9446]: cluster 2020-09-08T16:29:57.582970+0000 mon.a (mon.0) 2617 : cluster [DBG] osdmap e955: 12 total, 12 up, 12 in 2020-09-08T16:29:58.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:58 smithi166 bash[9446]: cluster 2020-09-08T16:29:57.794793+0000 osd.7 (osd.7) 212 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:29:58.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:58 smithi055 bash[9296]: cluster 2020-09-08T16:29:57.061010+0000 osd.1 (osd.1) 192 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:58.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:58 smithi055 bash[9296]: cluster 2020-09-08T16:29:57.061149+0000 osd.1 (osd.1) 193 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:58.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:58 smithi055 bash[9296]: cluster 2020-09-08T16:29:57.582970+0000 mon.a (mon.0) 2617 : cluster [DBG] osdmap e955: 12 total, 12 up, 12 in 2020-09-08T16:29:58.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:58 smithi055 bash[9296]: cluster 2020-09-08T16:29:57.794793+0000 osd.7 (osd.7) 212 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:29:58.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:58 smithi104 bash[9203]: cluster 2020-09-08T16:29:57.061010+0000 osd.1 (osd.1) 192 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:29:58.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:58 smithi104 bash[9203]: cluster 2020-09-08T16:29:57.061149+0000 osd.1 (osd.1) 193 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:29:58.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:58 smithi104 bash[9203]: cluster 2020-09-08T16:29:57.582970+0000 mon.a (mon.0) 2617 : cluster [DBG] osdmap e955: 12 total, 12 up, 12 in 2020-09-08T16:29:58.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:58 smithi104 bash[9203]: cluster 2020-09-08T16:29:57.794793+0000 osd.7 (osd.7) 212 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:29:59.329 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2020-09-08T16:29:59.462 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:29:59.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:59 smithi166 bash[9446]: cluster 2020-09-08T16:29:57.945355+0000 osd.7 (osd.7) 213 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:29:59.870 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:59 smithi166 bash[9446]: cluster 2020-09-08T16:29:58.515244+0000 mgr.x (mgr.24118) 1879 : cluster [DBG] pgmap v2750: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:59.871 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:29:59 smithi166 bash[9446]: audit 2020-09-08T16:29:58.587359+0000 mon.a (mon.0) 2618 : audit [DBG] from='client.? v1:172.21.15.104:0/3249841504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2020-09-08T16:29:59.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:59 smithi055 bash[9296]: cluster 2020-09-08T16:29:57.945355+0000 osd.7 (osd.7) 213 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:29:59.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:59 smithi055 bash[9296]: cluster 2020-09-08T16:29:58.515244+0000 mgr.x (mgr.24118) 1879 : cluster [DBG] pgmap v2750: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:59.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:29:59 smithi055 bash[9296]: audit 2020-09-08T16:29:58.587359+0000 mon.a (mon.0) 2618 : audit [DBG] from='client.? v1:172.21.15.104:0/3249841504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2020-09-08T16:29:59.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:59 smithi104 bash[9203]: cluster 2020-09-08T16:29:57.945355+0000 osd.7 (osd.7) 213 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:29:59.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:59 smithi104 bash[9203]: cluster 2020-09-08T16:29:58.515244+0000 mgr.x (mgr.24118) 1879 : cluster [DBG] pgmap v2750: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:29:59.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:29:59 smithi104 bash[9203]: audit 2020-09-08T16:29:58.587359+0000 mon.a (mon.0) 2618 : audit [DBG] from='client.? v1:172.21.15.104:0/3249841504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2020-09-08T16:30:01.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:01 smithi166 bash[9446]: cluster 2020-09-08T16:30:00.515887+0000 mgr.x (mgr.24118) 1880 : cluster [DBG] pgmap v2751: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:01.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:01 smithi055 bash[9296]: cluster 2020-09-08T16:30:00.515887+0000 mgr.x (mgr.24118) 1880 : cluster [DBG] pgmap v2751: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:01.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:01 smithi104 bash[9203]: cluster 2020-09-08T16:30:00.515887+0000 mgr.x (mgr.24118) 1880 : cluster [DBG] pgmap v2751: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:03.286 INFO:teuthology.orchestra.run.smithi104.stdout:724 2020-09-08T16:30:03.287 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2020-09-08T16:30:03.415 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:30:03.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:03 smithi104 bash[9203]: cluster 2020-09-08T16:30:02.258056+0000 osd.6 (osd.6) 157 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:03.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:03 smithi104 bash[9203]: cluster 2020-09-08T16:30:02.258184+0000 osd.6 (osd.6) 158 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:03.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:03 smithi104 bash[9203]: cluster 2020-09-08T16:30:02.448206+0000 mon.a (mon.0) 2619 : cluster [DBG] osdmap e956: 12 total, 12 up, 12 in 2020-09-08T16:30:03.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:03 smithi104 bash[9203]: cluster 2020-09-08T16:30:02.516482+0000 mgr.x (mgr.24118) 1881 : cluster [DBG] pgmap v2753: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:03.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:03 smithi166 bash[9446]: cluster 2020-09-08T16:30:02.258056+0000 osd.6 (osd.6) 157 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:03.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:03 smithi166 bash[9446]: cluster 2020-09-08T16:30:02.258184+0000 osd.6 (osd.6) 158 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:03.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:03 smithi166 bash[9446]: cluster 2020-09-08T16:30:02.448206+0000 mon.a (mon.0) 2619 : cluster [DBG] osdmap e956: 12 total, 12 up, 12 in 2020-09-08T16:30:03.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:03 smithi166 bash[9446]: cluster 2020-09-08T16:30:02.516482+0000 mgr.x (mgr.24118) 1881 : cluster [DBG] pgmap v2753: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:03.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:03 smithi055 bash[9296]: cluster 2020-09-08T16:30:02.258056+0000 osd.6 (osd.6) 157 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:03.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:03 smithi055 bash[9296]: cluster 2020-09-08T16:30:02.258184+0000 osd.6 (osd.6) 158 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:03.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:03 smithi055 bash[9296]: cluster 2020-09-08T16:30:02.448206+0000 mon.a (mon.0) 2619 : cluster [DBG] osdmap e956: 12 total, 12 up, 12 in 2020-09-08T16:30:03.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:03 smithi055 bash[9296]: cluster 2020-09-08T16:30:02.516482+0000 mgr.x (mgr.24118) 1881 : cluster [DBG] pgmap v2753: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:05.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:05 smithi166 bash[9446]: cluster 2020-09-08T16:30:04.517120+0000 mgr.x (mgr.24118) 1882 : cluster [DBG] pgmap v2754: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:05.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:05 smithi055 bash[9296]: cluster 2020-09-08T16:30:04.517120+0000 mgr.x (mgr.24118) 1882 : cluster [DBG] pgmap v2754: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:05.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:05 smithi104 bash[9203]: cluster 2020-09-08T16:30:04.517120+0000 mgr.x (mgr.24118) 1882 : cluster [DBG] pgmap v2754: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:07.287 INFO:teuthology.orchestra.run.smithi104.stdout:719 2020-09-08T16:30:07.288 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2020-09-08T16:30:07.421 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:30:07.576 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:07 smithi104 bash[9203]: cluster 2020-09-08T16:30:06.517733+0000 mgr.x (mgr.24118) 1883 : cluster [DBG] pgmap v2755: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:07.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:07 smithi166 bash[9446]: cluster 2020-09-08T16:30:06.517733+0000 mgr.x (mgr.24118) 1883 : cluster [DBG] pgmap v2755: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:07.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:07 smithi055 bash[9296]: cluster 2020-09-08T16:30:06.517733+0000 mgr.x (mgr.24118) 1883 : cluster [DBG] pgmap v2755: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:08.773 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:08 smithi104 bash[9203]: cluster 2020-09-08T16:30:07.893133+0000 osd.11 (osd.11) 172 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:08.774 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:08 smithi104 bash[9203]: cluster 2020-09-08T16:30:07.893277+0000 osd.11 (osd.11) 173 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:08.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:08 smithi166 bash[9446]: cluster 2020-09-08T16:30:07.893133+0000 osd.11 (osd.11) 172 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:08.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:08 smithi166 bash[9446]: cluster 2020-09-08T16:30:07.893277+0000 osd.11 (osd.11) 173 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:08.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:08 smithi055 bash[9296]: cluster 2020-09-08T16:30:07.893133+0000 osd.11 (osd.11) 172 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:08.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:08 smithi055 bash[9296]: cluster 2020-09-08T16:30:07.893277+0000 osd.11 (osd.11) 173 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:09.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:09 smithi166 bash[9446]: cluster 2020-09-08T16:30:08.518497+0000 mgr.x (mgr.24118) 1884 : cluster [DBG] pgmap v2756: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:09.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:09 smithi166 bash[9446]: cluster 2020-09-08T16:30:08.592801+0000 mon.a (mon.0) 2620 : cluster [DBG] osdmap e957: 12 total, 12 up, 12 in 2020-09-08T16:30:09.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:09 smithi055 bash[9296]: cluster 2020-09-08T16:30:08.518497+0000 mgr.x (mgr.24118) 1884 : cluster [DBG] pgmap v2756: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:09.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:09 smithi055 bash[9296]: cluster 2020-09-08T16:30:08.592801+0000 mon.a (mon.0) 2620 : cluster [DBG] osdmap e957: 12 total, 12 up, 12 in 2020-09-08T16:30:09.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:09 smithi104 bash[9203]: cluster 2020-09-08T16:30:08.518497+0000 mgr.x (mgr.24118) 1884 : cluster [DBG] pgmap v2756: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:09.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:09 smithi104 bash[9203]: cluster 2020-09-08T16:30:08.592801+0000 mon.a (mon.0) 2620 : cluster [DBG] osdmap e957: 12 total, 12 up, 12 in 2020-09-08T16:30:11.341 INFO:teuthology.orchestra.run.smithi104.stdout:718 2020-09-08T16:30:11.342 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2020-09-08T16:30:11.479 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:30:11.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:11 smithi104 bash[9203]: cluster 2020-09-08T16:30:10.519119+0000 mgr.x (mgr.24118) 1885 : cluster [DBG] pgmap v2758: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:11.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:11 smithi166 bash[9446]: cluster 2020-09-08T16:30:10.519119+0000 mgr.x (mgr.24118) 1885 : cluster [DBG] pgmap v2758: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:11.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:11 smithi055 bash[9296]: cluster 2020-09-08T16:30:10.519119+0000 mgr.x (mgr.24118) 1885 : cluster [DBG] pgmap v2758: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:12.755 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:12 smithi104 bash[9203]: cluster 2020-09-08T16:30:10.818381+0000 osd.4 (osd.4) 117 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:12.756 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:12 smithi104 bash[9203]: cluster 2020-09-08T16:30:10.818525+0000 osd.4 (osd.4) 118 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:12.756 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:12 smithi104 bash[9203]: cluster 2020-09-08T16:30:11.454945+0000 mon.a (mon.0) 2621 : cluster [DBG] osdmap e958: 12 total, 12 up, 12 in 2020-09-08T16:30:12.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:12 smithi166 bash[9446]: cluster 2020-09-08T16:30:10.818381+0000 osd.4 (osd.4) 117 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:12.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:12 smithi166 bash[9446]: cluster 2020-09-08T16:30:10.818525+0000 osd.4 (osd.4) 118 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:12.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:12 smithi166 bash[9446]: cluster 2020-09-08T16:30:11.454945+0000 mon.a (mon.0) 2621 : cluster [DBG] osdmap e958: 12 total, 12 up, 12 in 2020-09-08T16:30:12.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:12 smithi055 bash[9296]: cluster 2020-09-08T16:30:10.818381+0000 osd.4 (osd.4) 117 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:12.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:12 smithi055 bash[9296]: cluster 2020-09-08T16:30:10.818525+0000 osd.4 (osd.4) 118 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:12.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:12 smithi055 bash[9296]: cluster 2020-09-08T16:30:11.454945+0000 mon.a (mon.0) 2621 : cluster [DBG] osdmap e958: 12 total, 12 up, 12 in 2020-09-08T16:30:13.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:13 smithi166 bash[9446]: cluster 2020-09-08T16:30:12.519778+0000 mgr.x (mgr.24118) 1886 : cluster [DBG] pgmap v2760: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:13.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:13 smithi055 bash[9296]: cluster 2020-09-08T16:30:12.519778+0000 mgr.x (mgr.24118) 1886 : cluster [DBG] pgmap v2760: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:13.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:13 smithi104 bash[9203]: cluster 2020-09-08T16:30:12.519778+0000 mgr.x (mgr.24118) 1886 : cluster [DBG] pgmap v2760: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:15.261 INFO:teuthology.orchestra.run.smithi104.stdout:713 2020-09-08T16:30:15.263 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2020-09-08T16:30:15.396 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:30:15.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:15 smithi166 bash[9446]: cluster 2020-09-08T16:30:14.520416+0000 mgr.x (mgr.24118) 1887 : cluster [DBG] pgmap v2761: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:15.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:15 smithi055 bash[9296]: cluster 2020-09-08T16:30:14.520416+0000 mgr.x (mgr.24118) 1887 : cluster [DBG] pgmap v2761: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:15.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:15 smithi104 bash[9203]: cluster 2020-09-08T16:30:14.520416+0000 mgr.x (mgr.24118) 1887 : cluster [DBG] pgmap v2761: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:17.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:17 smithi166 bash[9446]: cluster 2020-09-08T16:30:16.521053+0000 mgr.x (mgr.24118) 1888 : cluster [DBG] pgmap v2762: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:17.911 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:17 smithi055 bash[9296]: cluster 2020-09-08T16:30:16.521053+0000 mgr.x (mgr.24118) 1888 : cluster [DBG] pgmap v2762: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:17.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:17 smithi104 bash[9203]: cluster 2020-09-08T16:30:16.521053+0000 mgr.x (mgr.24118) 1888 : cluster [DBG] pgmap v2762: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:19.219 INFO:teuthology.orchestra.run.smithi104.stdout:709 2020-09-08T16:30:19.220 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2020-09-08T16:30:19.357 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:30:19.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:19 smithi166 bash[9446]: cluster 2020-09-08T16:30:18.521683+0000 mgr.x (mgr.24118) 1889 : cluster [DBG] pgmap v2763: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:19.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:19 smithi055 bash[9296]: cluster 2020-09-08T16:30:18.521683+0000 mgr.x (mgr.24118) 1889 : cluster [DBG] pgmap v2763: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:19.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:19 smithi104 bash[9203]: cluster 2020-09-08T16:30:18.521683+0000 mgr.x (mgr.24118) 1889 : cluster [DBG] pgmap v2763: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:20.749 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:20 smithi104 bash[9203]: cluster 2020-09-08T16:30:19.502240+0000 osd.9 (osd.9) 156 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:20.750 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:20 smithi104 bash[9203]: cluster 2020-09-08T16:30:19.502379+0000 osd.9 (osd.9) 157 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:20.750 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:20 smithi104 bash[9203]: cluster 2020-09-08T16:30:19.602972+0000 mon.a (mon.0) 2622 : cluster [DBG] osdmap e959: 12 total, 12 up, 12 in 2020-09-08T16:30:20.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:20 smithi055 bash[9296]: cluster 2020-09-08T16:30:19.502240+0000 osd.9 (osd.9) 156 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:20.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:20 smithi055 bash[9296]: cluster 2020-09-08T16:30:19.502379+0000 osd.9 (osd.9) 157 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:20.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:20 smithi055 bash[9296]: cluster 2020-09-08T16:30:19.602972+0000 mon.a (mon.0) 2622 : cluster [DBG] osdmap e959: 12 total, 12 up, 12 in 2020-09-08T16:30:21.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:20 smithi166 bash[9446]: cluster 2020-09-08T16:30:19.502240+0000 osd.9 (osd.9) 156 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:21.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:20 smithi166 bash[9446]: cluster 2020-09-08T16:30:19.502379+0000 osd.9 (osd.9) 157 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:21.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:20 smithi166 bash[9446]: cluster 2020-09-08T16:30:19.602972+0000 mon.a (mon.0) 2622 : cluster [DBG] osdmap e959: 12 total, 12 up, 12 in 2020-09-08T16:30:21.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:21 smithi055 bash[9296]: cluster 2020-09-08T16:30:20.522127+0000 mgr.x (mgr.24118) 1890 : cluster [DBG] pgmap v2765: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:21.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:21 smithi104 bash[9203]: cluster 2020-09-08T16:30:20.522127+0000 mgr.x (mgr.24118) 1890 : cluster [DBG] pgmap v2765: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:22.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:21 smithi166 bash[9446]: cluster 2020-09-08T16:30:20.522127+0000 mgr.x (mgr.24118) 1890 : cluster [DBG] pgmap v2765: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:23.303 INFO:teuthology.orchestra.run.smithi104.stdout:703 2020-09-08T16:30:23.304 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2020-09-08T16:30:23.436 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:30:23.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:23 smithi055 bash[9296]: cluster 2020-09-08T16:30:22.522711+0000 mgr.x (mgr.24118) 1891 : cluster [DBG] pgmap v2766: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:23.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:23 smithi055 bash[9296]: cluster 2020-09-08T16:30:22.902170+0000 osd.2 (osd.2) 152 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:23.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:23 smithi055 bash[9296]: cluster 2020-09-08T16:30:22.902296+0000 osd.2 (osd.2) 153 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:23.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:23 smithi104 bash[9203]: cluster 2020-09-08T16:30:22.522711+0000 mgr.x (mgr.24118) 1891 : cluster [DBG] pgmap v2766: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:23.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:23 smithi104 bash[9203]: cluster 2020-09-08T16:30:22.902170+0000 osd.2 (osd.2) 152 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:23.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:23 smithi104 bash[9203]: cluster 2020-09-08T16:30:22.902296+0000 osd.2 (osd.2) 153 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:24.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:23 smithi166 bash[9446]: cluster 2020-09-08T16:30:22.522711+0000 mgr.x (mgr.24118) 1891 : cluster [DBG] pgmap v2766: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:24.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:23 smithi166 bash[9446]: cluster 2020-09-08T16:30:22.902170+0000 osd.2 (osd.2) 152 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:24.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:23 smithi166 bash[9446]: cluster 2020-09-08T16:30:22.902296+0000 osd.2 (osd.2) 153 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:24.750 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:24 smithi104 bash[9203]: cluster 2020-09-08T16:30:23.639442+0000 mon.a (mon.0) 2623 : cluster [DBG] osdmap e960: 12 total, 12 up, 12 in 2020-09-08T16:30:24.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:24 smithi055 bash[9296]: cluster 2020-09-08T16:30:23.639442+0000 mon.a (mon.0) 2623 : cluster [DBG] osdmap e960: 12 total, 12 up, 12 in 2020-09-08T16:30:25.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:24 smithi166 bash[9446]: cluster 2020-09-08T16:30:23.639442+0000 mon.a (mon.0) 2623 : cluster [DBG] osdmap e960: 12 total, 12 up, 12 in 2020-09-08T16:30:25.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:25 smithi104 bash[9203]: cluster 2020-09-08T16:30:24.523333+0000 mgr.x (mgr.24118) 1892 : cluster [DBG] pgmap v2768: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:26.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:25 smithi166 bash[9446]: cluster 2020-09-08T16:30:24.523333+0000 mgr.x (mgr.24118) 1892 : cluster [DBG] pgmap v2768: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:26.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:25 smithi055 bash[9296]: cluster 2020-09-08T16:30:24.523333+0000 mgr.x (mgr.24118) 1892 : cluster [DBG] pgmap v2768: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:27.365 INFO:teuthology.orchestra.run.smithi104.stdout:702 2020-09-08T16:30:27.366 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2020-09-08T16:30:27.494 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:30:27.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:27 smithi104 bash[9203]: cluster 2020-09-08T16:30:26.523962+0000 mgr.x (mgr.24118) 1893 : cluster [DBG] pgmap v2769: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:28.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:27 smithi166 bash[9446]: cluster 2020-09-08T16:30:26.523962+0000 mgr.x (mgr.24118) 1893 : cluster [DBG] pgmap v2769: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:28.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:27 smithi055 bash[9296]: cluster 2020-09-08T16:30:26.523962+0000 mgr.x (mgr.24118) 1893 : cluster [DBG] pgmap v2769: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:30.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:29 smithi166 bash[9446]: cluster 2020-09-08T16:30:28.524628+0000 mgr.x (mgr.24118) 1894 : cluster [DBG] pgmap v2770: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:30.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:29 smithi055 bash[9296]: cluster 2020-09-08T16:30:28.524628+0000 mgr.x (mgr.24118) 1894 : cluster [DBG] pgmap v2770: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:30.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:29 smithi104 bash[9203]: cluster 2020-09-08T16:30:28.524628+0000 mgr.x (mgr.24118) 1894 : cluster [DBG] pgmap v2770: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:31.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:30 smithi166 bash[9446]: audit 2020-09-08T16:30:29.703473+0000 mon.c (mon.2) 101 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:30:31.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:30 smithi166 bash[9446]: audit 2020-09-08T16:30:29.706826+0000 mon.a (mon.0) 2624 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:30:31.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:30 smithi166 bash[9446]: audit 2020-09-08T16:30:29.712012+0000 mon.a (mon.0) 2625 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:30:31.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:30 smithi055 bash[9296]: audit 2020-09-08T16:30:29.703473+0000 mon.c (mon.2) 101 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:30:31.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:30 smithi055 bash[9296]: audit 2020-09-08T16:30:29.706826+0000 mon.a (mon.0) 2624 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:30:31.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:30 smithi055 bash[9296]: audit 2020-09-08T16:30:29.712012+0000 mon.a (mon.0) 2625 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:30:31.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:30 smithi104 bash[9203]: audit 2020-09-08T16:30:29.703473+0000 mon.c (mon.2) 101 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:30:31.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:30 smithi104 bash[9203]: audit 2020-09-08T16:30:29.706826+0000 mon.a (mon.0) 2624 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:30:31.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:30 smithi104 bash[9203]: audit 2020-09-08T16:30:29.712012+0000 mon.a (mon.0) 2625 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:30:31.322 INFO:teuthology.orchestra.run.smithi104.stdout:698 2020-09-08T16:30:31.323 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2020-09-08T16:30:31.453 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:30:32.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:31 smithi166 bash[9446]: cluster 2020-09-08T16:30:30.488203+0000 osd.7 (osd.7) 214 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:32.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:31 smithi166 bash[9446]: cluster 2020-09-08T16:30:30.488330+0000 osd.7 (osd.7) 215 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:32.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:31 smithi166 bash[9446]: cluster 2020-09-08T16:30:30.525243+0000 mgr.x (mgr.24118) 1895 : cluster [DBG] pgmap v2771: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:32.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:31 smithi166 bash[9446]: cluster 2020-09-08T16:30:30.645001+0000 osd.0 (osd.0) 168 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:32.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:31 smithi166 bash[9446]: cluster 2020-09-08T16:30:30.645155+0000 osd.0 (osd.0) 169 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:32.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:31 smithi166 bash[9446]: cluster 2020-09-08T16:30:31.449655+0000 mon.a (mon.0) 2626 : cluster [DBG] osdmap e961: 12 total, 12 up, 12 in 2020-09-08T16:30:32.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:31 smithi055 bash[9296]: cluster 2020-09-08T16:30:30.488203+0000 osd.7 (osd.7) 214 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:32.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:31 smithi055 bash[9296]: cluster 2020-09-08T16:30:30.488330+0000 osd.7 (osd.7) 215 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:32.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:31 smithi055 bash[9296]: cluster 2020-09-08T16:30:30.525243+0000 mgr.x (mgr.24118) 1895 : cluster [DBG] pgmap v2771: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:32.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:31 smithi055 bash[9296]: cluster 2020-09-08T16:30:30.645001+0000 osd.0 (osd.0) 168 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:32.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:31 smithi055 bash[9296]: cluster 2020-09-08T16:30:30.645155+0000 osd.0 (osd.0) 169 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:32.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:31 smithi055 bash[9296]: cluster 2020-09-08T16:30:31.449655+0000 mon.a (mon.0) 2626 : cluster [DBG] osdmap e961: 12 total, 12 up, 12 in 2020-09-08T16:30:32.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:31 smithi104 bash[9203]: cluster 2020-09-08T16:30:30.488203+0000 osd.7 (osd.7) 214 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:32.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:31 smithi104 bash[9203]: cluster 2020-09-08T16:30:30.488330+0000 osd.7 (osd.7) 215 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:32.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:31 smithi104 bash[9203]: cluster 2020-09-08T16:30:30.525243+0000 mgr.x (mgr.24118) 1895 : cluster [DBG] pgmap v2771: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:32.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:31 smithi104 bash[9203]: cluster 2020-09-08T16:30:30.645001+0000 osd.0 (osd.0) 168 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:32.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:31 smithi104 bash[9203]: cluster 2020-09-08T16:30:30.645155+0000 osd.0 (osd.0) 169 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:32.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:31 smithi104 bash[9203]: cluster 2020-09-08T16:30:31.449655+0000 mon.a (mon.0) 2626 : cluster [DBG] osdmap e961: 12 total, 12 up, 12 in 2020-09-08T16:30:34.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:33 smithi166 bash[9446]: cluster 2020-09-08T16:30:32.525867+0000 mgr.x (mgr.24118) 1896 : cluster [DBG] pgmap v2773: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:34.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:33 smithi055 bash[9296]: cluster 2020-09-08T16:30:32.525867+0000 mgr.x (mgr.24118) 1896 : cluster [DBG] pgmap v2773: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:34.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:33 smithi104 bash[9203]: cluster 2020-09-08T16:30:32.525867+0000 mgr.x (mgr.24118) 1896 : cluster [DBG] pgmap v2773: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:35.331 INFO:teuthology.orchestra.run.smithi104.stdout:693 2020-09-08T16:30:35.332 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2020-09-08T16:30:35.467 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:30:36.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:35 smithi166 bash[9446]: cluster 2020-09-08T16:30:34.526506+0000 mgr.x (mgr.24118) 1897 : cluster [DBG] pgmap v2774: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:36.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:35 smithi055 bash[9296]: cluster 2020-09-08T16:30:34.526506+0000 mgr.x (mgr.24118) 1897 : cluster [DBG] pgmap v2774: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:36.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:35 smithi104 bash[9203]: cluster 2020-09-08T16:30:34.526506+0000 mgr.x (mgr.24118) 1897 : cluster [DBG] pgmap v2774: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:38.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:37 smithi166 bash[9446]: cluster 2020-09-08T16:30:36.527123+0000 mgr.x (mgr.24118) 1898 : cluster [DBG] pgmap v2775: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:38.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:37 smithi055 bash[9296]: cluster 2020-09-08T16:30:36.527123+0000 mgr.x (mgr.24118) 1898 : cluster [DBG] pgmap v2775: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:38.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:37 smithi104 bash[9203]: cluster 2020-09-08T16:30:36.527123+0000 mgr.x (mgr.24118) 1898 : cluster [DBG] pgmap v2775: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:39.393 INFO:teuthology.orchestra.run.smithi104.stdout:686 2020-09-08T16:30:39.393 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2020-09-08T16:30:39.533 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:30:40.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:39 smithi166 bash[9446]: cluster 2020-09-08T16:30:38.527708+0000 mgr.x (mgr.24118) 1899 : cluster [DBG] pgmap v2776: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:40.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:39 smithi055 bash[9296]: cluster 2020-09-08T16:30:38.527708+0000 mgr.x (mgr.24118) 1899 : cluster [DBG] pgmap v2776: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:40.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:39 smithi104 bash[9203]: cluster 2020-09-08T16:30:38.527708+0000 mgr.x (mgr.24118) 1899 : cluster [DBG] pgmap v2776: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:40.909 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:40 smithi104 bash[9203]: cluster 2020-09-08T16:30:39.127733+0000 osd.5 (osd.5) 174 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:40.910 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:40 smithi104 bash[9203]: cluster 2020-09-08T16:30:39.127869+0000 osd.5 (osd.5) 175 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:40.910 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:40 smithi104 bash[9203]: cluster 2020-09-08T16:30:39.729608+0000 mon.a (mon.0) 2627 : cluster [DBG] osdmap e962: 12 total, 12 up, 12 in 2020-09-08T16:30:41.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:40 smithi166 bash[9446]: cluster 2020-09-08T16:30:39.127733+0000 osd.5 (osd.5) 174 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:41.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:40 smithi166 bash[9446]: cluster 2020-09-08T16:30:39.127869+0000 osd.5 (osd.5) 175 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:41.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:40 smithi166 bash[9446]: cluster 2020-09-08T16:30:39.729608+0000 mon.a (mon.0) 2627 : cluster [DBG] osdmap e962: 12 total, 12 up, 12 in 2020-09-08T16:30:41.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:40 smithi055 bash[9296]: cluster 2020-09-08T16:30:39.127733+0000 osd.5 (osd.5) 174 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:41.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:40 smithi055 bash[9296]: cluster 2020-09-08T16:30:39.127869+0000 osd.5 (osd.5) 175 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:41.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:40 smithi055 bash[9296]: cluster 2020-09-08T16:30:39.729608+0000 mon.a (mon.0) 2627 : cluster [DBG] osdmap e962: 12 total, 12 up, 12 in 2020-09-08T16:30:42.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:41 smithi166 bash[9446]: cluster 2020-09-08T16:30:40.528175+0000 mgr.x (mgr.24118) 1900 : cluster [DBG] pgmap v2778: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:42.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:41 smithi055 bash[9296]: cluster 2020-09-08T16:30:40.528175+0000 mgr.x (mgr.24118) 1900 : cluster [DBG] pgmap v2778: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:42.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:41 smithi104 bash[9203]: cluster 2020-09-08T16:30:40.528175+0000 mgr.x (mgr.24118) 1900 : cluster [DBG] pgmap v2778: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:43.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:42 smithi166 bash[9446]: cluster 2020-09-08T16:30:41.670687+0000 osd.10 (osd.10) 153 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:43.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:42 smithi166 bash[9446]: cluster 2020-09-08T16:30:41.670793+0000 osd.10 (osd.10) 154 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:43.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:42 smithi166 bash[9446]: cluster 2020-09-08T16:30:42.457633+0000 mon.a (mon.0) 2628 : cluster [DBG] osdmap e963: 12 total, 12 up, 12 in 2020-09-08T16:30:43.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:42 smithi055 bash[9296]: cluster 2020-09-08T16:30:41.670687+0000 osd.10 (osd.10) 153 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:43.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:42 smithi055 bash[9296]: cluster 2020-09-08T16:30:41.670793+0000 osd.10 (osd.10) 154 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:43.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:42 smithi055 bash[9296]: cluster 2020-09-08T16:30:42.457633+0000 mon.a (mon.0) 2628 : cluster [DBG] osdmap e963: 12 total, 12 up, 12 in 2020-09-08T16:30:43.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:42 smithi104 bash[9203]: cluster 2020-09-08T16:30:41.670687+0000 osd.10 (osd.10) 153 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:43.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:42 smithi104 bash[9203]: cluster 2020-09-08T16:30:41.670793+0000 osd.10 (osd.10) 154 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:43.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:42 smithi104 bash[9203]: cluster 2020-09-08T16:30:42.457633+0000 mon.a (mon.0) 2628 : cluster [DBG] osdmap e963: 12 total, 12 up, 12 in 2020-09-08T16:30:43.460 INFO:teuthology.orchestra.run.smithi104.stdout:685 2020-09-08T16:30:43.461 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2020-09-08T16:30:43.592 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:30:44.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:43 smithi166 bash[9446]: cluster 2020-09-08T16:30:42.528750+0000 mgr.x (mgr.24118) 1901 : cluster [DBG] pgmap v2780: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:44.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:43 smithi166 bash[9446]: cluster 2020-09-08T16:30:42.579665+0000 osd.3 (osd.3) 177 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:44.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:43 smithi166 bash[9446]: cluster 2020-09-08T16:30:42.579801+0000 osd.3 (osd.3) 178 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:44.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:43 smithi166 bash[9446]: cluster 2020-09-08T16:30:43.464613+0000 mon.a (mon.0) 2629 : cluster [DBG] osdmap e964: 12 total, 12 up, 12 in 2020-09-08T16:30:44.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:43 smithi055 bash[9296]: cluster 2020-09-08T16:30:42.528750+0000 mgr.x (mgr.24118) 1901 : cluster [DBG] pgmap v2780: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:44.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:43 smithi055 bash[9296]: cluster 2020-09-08T16:30:42.579665+0000 osd.3 (osd.3) 177 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:44.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:43 smithi055 bash[9296]: cluster 2020-09-08T16:30:42.579801+0000 osd.3 (osd.3) 178 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:44.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:43 smithi055 bash[9296]: cluster 2020-09-08T16:30:43.464613+0000 mon.a (mon.0) 2629 : cluster [DBG] osdmap e964: 12 total, 12 up, 12 in 2020-09-08T16:30:44.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:43 smithi104 bash[9203]: cluster 2020-09-08T16:30:42.528750+0000 mgr.x (mgr.24118) 1901 : cluster [DBG] pgmap v2780: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:44.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:43 smithi104 bash[9203]: cluster 2020-09-08T16:30:42.579665+0000 osd.3 (osd.3) 177 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:44.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:43 smithi104 bash[9203]: cluster 2020-09-08T16:30:42.579801+0000 osd.3 (osd.3) 178 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:44.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:43 smithi104 bash[9203]: cluster 2020-09-08T16:30:43.464613+0000 mon.a (mon.0) 2629 : cluster [DBG] osdmap e964: 12 total, 12 up, 12 in 2020-09-08T16:30:46.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:45 smithi166 bash[9446]: cluster 2020-09-08T16:30:44.529389+0000 mgr.x (mgr.24118) 1902 : cluster [DBG] pgmap v2782: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:46.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:45 smithi055 bash[9296]: cluster 2020-09-08T16:30:44.529389+0000 mgr.x (mgr.24118) 1902 : cluster [DBG] pgmap v2782: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:46.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:45 smithi104 bash[9203]: cluster 2020-09-08T16:30:44.529389+0000 mgr.x (mgr.24118) 1902 : cluster [DBG] pgmap v2782: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:47.454 INFO:teuthology.orchestra.run.smithi104.stdout:678 2020-09-08T16:30:47.455 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.0 2020-09-08T16:30:47.588 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:30:48.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:47 smithi166 bash[9446]: cluster 2020-09-08T16:30:46.530051+0000 mgr.x (mgr.24118) 1903 : cluster [DBG] pgmap v2783: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:48.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:47 smithi055 bash[9296]: cluster 2020-09-08T16:30:46.530051+0000 mgr.x (mgr.24118) 1903 : cluster [DBG] pgmap v2783: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:48.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:47 smithi104 bash[9203]: cluster 2020-09-08T16:30:46.530051+0000 mgr.x (mgr.24118) 1903 : cluster [DBG] pgmap v2783: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:50.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:49 smithi166 bash[9446]: cluster 2020-09-08T16:30:48.530719+0000 mgr.x (mgr.24118) 1904 : cluster [DBG] pgmap v2784: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:50.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:49 smithi055 bash[9296]: cluster 2020-09-08T16:30:48.530719+0000 mgr.x (mgr.24118) 1904 : cluster [DBG] pgmap v2784: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:50.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:49 smithi104 bash[9203]: cluster 2020-09-08T16:30:48.530719+0000 mgr.x (mgr.24118) 1904 : cluster [DBG] pgmap v2784: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:50.814 INFO:teuthology.orchestra.run.smithi104.stdout:733 2020-09-08T16:30:51.527 INFO:tasks.cephadm.ceph_manager.ceph:need seq 724 got 733 for osd.0 2020-09-08T16:30:51.528 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.1 2020-09-08T16:30:51.658 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:30:51.874 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:51 smithi166 bash[9446]: cluster 2020-09-08T16:30:50.531369+0000 mgr.x (mgr.24118) 1905 : cluster [DBG] pgmap v2785: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:51.875 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:51 smithi166 bash[9446]: audit 2020-09-08T16:30:50.821476+0000 mon.a (mon.0) 2630 : audit [DBG] from='client.? v1:172.21.15.104:0/1896869764' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2020-09-08T16:30:51.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:51 smithi055 bash[9296]: cluster 2020-09-08T16:30:50.531369+0000 mgr.x (mgr.24118) 1905 : cluster [DBG] pgmap v2785: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:51.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:51 smithi055 bash[9296]: audit 2020-09-08T16:30:50.821476+0000 mon.a (mon.0) 2630 : audit [DBG] from='client.? v1:172.21.15.104:0/1896869764' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2020-09-08T16:30:51.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:51 smithi104 bash[9203]: cluster 2020-09-08T16:30:50.531369+0000 mgr.x (mgr.24118) 1905 : cluster [DBG] pgmap v2785: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:51.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:51 smithi104 bash[9203]: audit 2020-09-08T16:30:50.821476+0000 mon.a (mon.0) 2630 : audit [DBG] from='client.? v1:172.21.15.104:0/1896869764' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2020-09-08T16:30:53.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:53 smithi166 bash[9446]: cluster 2020-09-08T16:30:52.246850+0000 osd.8 (osd.8) 148 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:53.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:53 smithi166 bash[9446]: cluster 2020-09-08T16:30:52.246971+0000 osd.8 (osd.8) 149 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:53.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:53 smithi166 bash[9446]: cluster 2020-09-08T16:30:52.460915+0000 mon.a (mon.0) 2631 : cluster [DBG] osdmap e965: 12 total, 12 up, 12 in 2020-09-08T16:30:53.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:53 smithi166 bash[9446]: cluster 2020-09-08T16:30:52.531900+0000 mgr.x (mgr.24118) 1906 : cluster [DBG] pgmap v2787: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:53.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:53 smithi055 bash[9296]: cluster 2020-09-08T16:30:52.246850+0000 osd.8 (osd.8) 148 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:53.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:53 smithi055 bash[9296]: cluster 2020-09-08T16:30:52.246971+0000 osd.8 (osd.8) 149 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:53.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:53 smithi055 bash[9296]: cluster 2020-09-08T16:30:52.460915+0000 mon.a (mon.0) 2631 : cluster [DBG] osdmap e965: 12 total, 12 up, 12 in 2020-09-08T16:30:53.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:53 smithi055 bash[9296]: cluster 2020-09-08T16:30:52.531900+0000 mgr.x (mgr.24118) 1906 : cluster [DBG] pgmap v2787: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:53.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:53 smithi104 bash[9203]: cluster 2020-09-08T16:30:52.246850+0000 osd.8 (osd.8) 148 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:53.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:53 smithi104 bash[9203]: cluster 2020-09-08T16:30:52.246971+0000 osd.8 (osd.8) 149 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:53.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:53 smithi104 bash[9203]: cluster 2020-09-08T16:30:52.460915+0000 mon.a (mon.0) 2631 : cluster [DBG] osdmap e965: 12 total, 12 up, 12 in 2020-09-08T16:30:53.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:53 smithi104 bash[9203]: cluster 2020-09-08T16:30:52.531900+0000 mgr.x (mgr.24118) 1906 : cluster [DBG] pgmap v2787: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:54.859 INFO:teuthology.orchestra.run.smithi104.stdout:729 2020-09-08T16:30:55.546 INFO:tasks.cephadm.ceph_manager.ceph:need seq 719 got 729 for osd.1 2020-09-08T16:30:55.547 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.2 2020-09-08T16:30:55.675 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:30:55.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:55 smithi166 bash[9446]: cluster 2020-09-08T16:30:54.532599+0000 mgr.x (mgr.24118) 1907 : cluster [DBG] pgmap v2788: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:55.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:55 smithi166 bash[9446]: audit 2020-09-08T16:30:54.865672+0000 mon.a (mon.0) 2632 : audit [DBG] from='client.? v1:172.21.15.104:0/1003827620' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2020-09-08T16:30:55.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:55 smithi055 bash[9296]: cluster 2020-09-08T16:30:54.532599+0000 mgr.x (mgr.24118) 1907 : cluster [DBG] pgmap v2788: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:55.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:55 smithi055 bash[9296]: audit 2020-09-08T16:30:54.865672+0000 mon.a (mon.0) 2632 : audit [DBG] from='client.? v1:172.21.15.104:0/1003827620' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2020-09-08T16:30:55.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:55 smithi104 bash[9203]: cluster 2020-09-08T16:30:54.532599+0000 mgr.x (mgr.24118) 1907 : cluster [DBG] pgmap v2788: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:55.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:55 smithi104 bash[9203]: audit 2020-09-08T16:30:54.865672+0000 mon.a (mon.0) 2632 : audit [DBG] from='client.? v1:172.21.15.104:0/1003827620' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2020-09-08T16:30:57.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:57 smithi166 bash[9446]: cluster 2020-09-08T16:30:56.533302+0000 mgr.x (mgr.24118) 1908 : cluster [DBG] pgmap v2789: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:57.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:57 smithi055 bash[9296]: cluster 2020-09-08T16:30:56.533302+0000 mgr.x (mgr.24118) 1908 : cluster [DBG] pgmap v2789: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:57.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:57 smithi104 bash[9203]: cluster 2020-09-08T16:30:56.533302+0000 mgr.x (mgr.24118) 1908 : cluster [DBG] pgmap v2789: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:58.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:58 smithi166 bash[9446]: cluster 2020-09-08T16:30:57.841965+0000 osd.1 (osd.1) 194 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:58.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:58 smithi166 bash[9446]: cluster 2020-09-08T16:30:57.842104+0000 osd.1 (osd.1) 195 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:58.883 INFO:teuthology.orchestra.run.smithi104.stdout:728 2020-09-08T16:30:58.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:58 smithi055 bash[9296]: cluster 2020-09-08T16:30:57.841965+0000 osd.1 (osd.1) 194 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:58.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:58 smithi055 bash[9296]: cluster 2020-09-08T16:30:57.842104+0000 osd.1 (osd.1) 195 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:58.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:58 smithi104 bash[9203]: cluster 2020-09-08T16:30:57.841965+0000 osd.1 (osd.1) 194 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:30:58.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:58 smithi104 bash[9203]: cluster 2020-09-08T16:30:57.842104+0000 osd.1 (osd.1) 195 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:30:59.577 INFO:tasks.cephadm.ceph_manager.ceph:need seq 718 got 728 for osd.2 2020-09-08T16:30:59.578 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.3 2020-09-08T16:30:59.707 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:30:59.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:59 smithi166 bash[9446]: cluster 2020-09-08T16:30:58.533971+0000 mgr.x (mgr.24118) 1909 : cluster [DBG] pgmap v2790: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:59.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:59 smithi166 bash[9446]: cluster 2020-09-08T16:30:58.610166+0000 mon.a (mon.0) 2633 : cluster [DBG] osdmap e966: 12 total, 12 up, 12 in 2020-09-08T16:30:59.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:30:59 smithi166 bash[9446]: audit 2020-09-08T16:30:58.890424+0000 mon.a (mon.0) 2634 : audit [DBG] from='client.? v1:172.21.15.104:0/2611706015' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2020-09-08T16:30:59.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:59 smithi055 bash[9296]: cluster 2020-09-08T16:30:58.533971+0000 mgr.x (mgr.24118) 1909 : cluster [DBG] pgmap v2790: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:59.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:59 smithi055 bash[9296]: cluster 2020-09-08T16:30:58.610166+0000 mon.a (mon.0) 2633 : cluster [DBG] osdmap e966: 12 total, 12 up, 12 in 2020-09-08T16:30:59.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:30:59 smithi055 bash[9296]: audit 2020-09-08T16:30:58.890424+0000 mon.a (mon.0) 2634 : audit [DBG] from='client.? v1:172.21.15.104:0/2611706015' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2020-09-08T16:30:59.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:59 smithi104 bash[9203]: cluster 2020-09-08T16:30:58.533971+0000 mgr.x (mgr.24118) 1909 : cluster [DBG] pgmap v2790: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:30:59.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:59 smithi104 bash[9203]: cluster 2020-09-08T16:30:58.610166+0000 mon.a (mon.0) 2633 : cluster [DBG] osdmap e966: 12 total, 12 up, 12 in 2020-09-08T16:30:59.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:30:59 smithi104 bash[9203]: audit 2020-09-08T16:30:58.890424+0000 mon.a (mon.0) 2634 : audit [DBG] from='client.? v1:172.21.15.104:0/2611706015' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2020-09-08T16:31:01.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:01 smithi166 bash[9446]: cluster 2020-09-08T16:31:00.534646+0000 mgr.x (mgr.24118) 1910 : cluster [DBG] pgmap v2792: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:01.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:01 smithi055 bash[9296]: cluster 2020-09-08T16:31:00.534646+0000 mgr.x (mgr.24118) 1910 : cluster [DBG] pgmap v2792: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:01.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:01 smithi104 bash[9203]: cluster 2020-09-08T16:31:00.534646+0000 mgr.x (mgr.24118) 1910 : cluster [DBG] pgmap v2792: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:02.952 INFO:teuthology.orchestra.run.smithi104.stdout:722 2020-09-08T16:31:03.703 INFO:tasks.cephadm.ceph_manager.ceph:need seq 713 got 722 for osd.3 2020-09-08T16:31:03.704 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.4 2020-09-08T16:31:03.717 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:03 smithi104 bash[9203]: cluster 2020-09-08T16:31:02.335352+0000 osd.6 (osd.6) 159 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:03.717 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:03 smithi104 bash[9203]: cluster 2020-09-08T16:31:02.335502+0000 osd.6 (osd.6) 160 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:03.717 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:03 smithi104 bash[9203]: cluster 2020-09-08T16:31:02.471169+0000 mon.a (mon.0) 2635 : cluster [DBG] osdmap e967: 12 total, 12 up, 12 in 2020-09-08T16:31:03.718 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:03 smithi104 bash[9203]: cluster 2020-09-08T16:31:02.535258+0000 mgr.x (mgr.24118) 1911 : cluster [DBG] pgmap v2794: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:03.718 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:03 smithi104 bash[9203]: audit 2020-09-08T16:31:02.958670+0000 mon.a (mon.0) 2636 : audit [DBG] from='client.? v1:172.21.15.104:0/3279129565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2020-09-08T16:31:03.838 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:31:03.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:03 smithi166 bash[9446]: cluster 2020-09-08T16:31:02.335352+0000 osd.6 (osd.6) 159 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:03.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:03 smithi166 bash[9446]: cluster 2020-09-08T16:31:02.335502+0000 osd.6 (osd.6) 160 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:03.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:03 smithi166 bash[9446]: cluster 2020-09-08T16:31:02.471169+0000 mon.a (mon.0) 2635 : cluster [DBG] osdmap e967: 12 total, 12 up, 12 in 2020-09-08T16:31:03.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:03 smithi166 bash[9446]: cluster 2020-09-08T16:31:02.535258+0000 mgr.x (mgr.24118) 1911 : cluster [DBG] pgmap v2794: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:03.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:03 smithi166 bash[9446]: audit 2020-09-08T16:31:02.958670+0000 mon.a (mon.0) 2636 : audit [DBG] from='client.? v1:172.21.15.104:0/3279129565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2020-09-08T16:31:03.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:03 smithi055 bash[9296]: cluster 2020-09-08T16:31:02.335352+0000 osd.6 (osd.6) 159 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:03.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:03 smithi055 bash[9296]: cluster 2020-09-08T16:31:02.335502+0000 osd.6 (osd.6) 160 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:03.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:03 smithi055 bash[9296]: cluster 2020-09-08T16:31:02.471169+0000 mon.a (mon.0) 2635 : cluster [DBG] osdmap e967: 12 total, 12 up, 12 in 2020-09-08T16:31:03.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:03 smithi055 bash[9296]: cluster 2020-09-08T16:31:02.535258+0000 mgr.x (mgr.24118) 1911 : cluster [DBG] pgmap v2794: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:03.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:03 smithi055 bash[9296]: audit 2020-09-08T16:31:02.958670+0000 mon.a (mon.0) 2636 : audit [DBG] from='client.? v1:172.21.15.104:0/3279129565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2020-09-08T16:31:05.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:05 smithi166 bash[9446]: cluster 2020-09-08T16:31:04.535913+0000 mgr.x (mgr.24118) 1912 : cluster [DBG] pgmap v2795: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:05.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:05 smithi055 bash[9296]: cluster 2020-09-08T16:31:04.535913+0000 mgr.x (mgr.24118) 1912 : cluster [DBG] pgmap v2795: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:05.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:05 smithi104 bash[9203]: cluster 2020-09-08T16:31:04.535913+0000 mgr.x (mgr.24118) 1912 : cluster [DBG] pgmap v2795: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:07.060 INFO:teuthology.orchestra.run.smithi104.stdout:719 2020-09-08T16:31:07.755 INFO:tasks.cephadm.ceph_manager.ceph:need seq 709 got 719 for osd.4 2020-09-08T16:31:07.756 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.5 2020-09-08T16:31:07.878 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:07 smithi166 bash[9446]: cluster 2020-09-08T16:31:06.536568+0000 mgr.x (mgr.24118) 1913 : cluster [DBG] pgmap v2796: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:07.879 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:07 smithi166 bash[9446]: audit 2020-09-08T16:31:07.066602+0000 mon.a (mon.0) 2637 : audit [DBG] from='client.? v1:172.21.15.104:0/655038346' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2020-09-08T16:31:07.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:07 smithi055 bash[9296]: cluster 2020-09-08T16:31:06.536568+0000 mgr.x (mgr.24118) 1913 : cluster [DBG] pgmap v2796: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:07.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:07 smithi055 bash[9296]: audit 2020-09-08T16:31:07.066602+0000 mon.a (mon.0) 2637 : audit [DBG] from='client.? v1:172.21.15.104:0/655038346' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2020-09-08T16:31:07.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:07 smithi104 bash[9203]: cluster 2020-09-08T16:31:06.536568+0000 mgr.x (mgr.24118) 1913 : cluster [DBG] pgmap v2796: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:07.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:07 smithi104 bash[9203]: audit 2020-09-08T16:31:07.066602+0000 mon.a (mon.0) 2637 : audit [DBG] from='client.? v1:172.21.15.104:0/655038346' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2020-09-08T16:31:07.929 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:31:09.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:09 smithi166 bash[9446]: cluster 2020-09-08T16:31:08.223097+0000 osd.11 (osd.11) 174 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:09.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:09 smithi166 bash[9446]: cluster 2020-09-08T16:31:08.223229+0000 osd.11 (osd.11) 175 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:09.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:09 smithi166 bash[9446]: cluster 2020-09-08T16:31:08.537231+0000 mgr.x (mgr.24118) 1914 : cluster [DBG] pgmap v2797: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:09.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:09 smithi166 bash[9446]: cluster 2020-09-08T16:31:08.612933+0000 mon.a (mon.0) 2638 : cluster [DBG] osdmap e968: 12 total, 12 up, 12 in 2020-09-08T16:31:09.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:09 smithi055 bash[9296]: cluster 2020-09-08T16:31:08.223097+0000 osd.11 (osd.11) 174 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:09.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:09 smithi055 bash[9296]: cluster 2020-09-08T16:31:08.223229+0000 osd.11 (osd.11) 175 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:09.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:09 smithi055 bash[9296]: cluster 2020-09-08T16:31:08.537231+0000 mgr.x (mgr.24118) 1914 : cluster [DBG] pgmap v2797: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:09.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:09 smithi055 bash[9296]: cluster 2020-09-08T16:31:08.612933+0000 mon.a (mon.0) 2638 : cluster [DBG] osdmap e968: 12 total, 12 up, 12 in 2020-09-08T16:31:09.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:09 smithi104 bash[9203]: cluster 2020-09-08T16:31:08.223097+0000 osd.11 (osd.11) 174 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:09.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:09 smithi104 bash[9203]: cluster 2020-09-08T16:31:08.223229+0000 osd.11 (osd.11) 175 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:09.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:09 smithi104 bash[9203]: cluster 2020-09-08T16:31:08.537231+0000 mgr.x (mgr.24118) 1914 : cluster [DBG] pgmap v2797: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:09.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:09 smithi104 bash[9203]: cluster 2020-09-08T16:31:08.612933+0000 mon.a (mon.0) 2638 : cluster [DBG] osdmap e968: 12 total, 12 up, 12 in 2020-09-08T16:31:11.089 INFO:teuthology.orchestra.run.smithi104.stdout:712 2020-09-08T16:31:11.823 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:11 smithi104 bash[9203]: cluster 2020-09-08T16:31:10.537829+0000 mgr.x (mgr.24118) 1915 : cluster [DBG] pgmap v2799: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:11.823 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:11 smithi104 bash[9203]: audit 2020-09-08T16:31:11.095144+0000 mon.a (mon.0) 2639 : audit [DBG] from='client.? v1:172.21.15.104:0/3751546994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2020-09-08T16:31:11.824 INFO:tasks.cephadm.ceph_manager.ceph:need seq 703 got 712 for osd.5 2020-09-08T16:31:11.825 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.6 2020-09-08T16:31:11.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:11 smithi166 bash[9446]: cluster 2020-09-08T16:31:10.537829+0000 mgr.x (mgr.24118) 1915 : cluster [DBG] pgmap v2799: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:11.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:11 smithi166 bash[9446]: audit 2020-09-08T16:31:11.095144+0000 mon.a (mon.0) 2639 : audit [DBG] from='client.? v1:172.21.15.104:0/3751546994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2020-09-08T16:31:11.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:11 smithi055 bash[9296]: cluster 2020-09-08T16:31:10.537829+0000 mgr.x (mgr.24118) 1915 : cluster [DBG] pgmap v2799: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:11.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:11 smithi055 bash[9296]: audit 2020-09-08T16:31:11.095144+0000 mon.a (mon.0) 2639 : audit [DBG] from='client.? v1:172.21.15.104:0/3751546994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2020-09-08T16:31:11.965 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:31:13.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:13 smithi166 bash[9446]: cluster 2020-09-08T16:31:11.701396+0000 osd.4 (osd.4) 119 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:13.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:13 smithi166 bash[9446]: cluster 2020-09-08T16:31:11.701522+0000 osd.4 (osd.4) 120 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:13.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:13 smithi166 bash[9446]: cluster 2020-09-08T16:31:12.470436+0000 mon.a (mon.0) 2640 : cluster [DBG] osdmap e969: 12 total, 12 up, 12 in 2020-09-08T16:31:13.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:13 smithi166 bash[9446]: cluster 2020-09-08T16:31:12.538474+0000 mgr.x (mgr.24118) 1916 : cluster [DBG] pgmap v2801: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:13.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:13 smithi055 bash[9296]: cluster 2020-09-08T16:31:11.701396+0000 osd.4 (osd.4) 119 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:13.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:13 smithi055 bash[9296]: cluster 2020-09-08T16:31:11.701522+0000 osd.4 (osd.4) 120 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:13.909 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:13 smithi055 bash[9296]: cluster 2020-09-08T16:31:12.470436+0000 mon.a (mon.0) 2640 : cluster [DBG] osdmap e969: 12 total, 12 up, 12 in 2020-09-08T16:31:13.910 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:13 smithi055 bash[9296]: cluster 2020-09-08T16:31:12.538474+0000 mgr.x (mgr.24118) 1916 : cluster [DBG] pgmap v2801: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:13.927 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:13 smithi104 bash[9203]: cluster 2020-09-08T16:31:11.701396+0000 osd.4 (osd.4) 119 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:13.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:13 smithi104 bash[9203]: cluster 2020-09-08T16:31:11.701522+0000 osd.4 (osd.4) 120 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:13.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:13 smithi104 bash[9203]: cluster 2020-09-08T16:31:12.470436+0000 mon.a (mon.0) 2640 : cluster [DBG] osdmap e969: 12 total, 12 up, 12 in 2020-09-08T16:31:13.928 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:13 smithi104 bash[9203]: cluster 2020-09-08T16:31:12.538474+0000 mgr.x (mgr.24118) 1916 : cluster [DBG] pgmap v2801: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:15.144 INFO:teuthology.orchestra.run.smithi104.stdout:712 2020-09-08T16:31:15.845 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:15 smithi104 bash[9203]: cluster 2020-09-08T16:31:14.539139+0000 mgr.x (mgr.24118) 1917 : cluster [DBG] pgmap v2802: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:15.845 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:15 smithi104 bash[9203]: audit 2020-09-08T16:31:15.150421+0000 mon.a (mon.0) 2641 : audit [DBG] from='client.? v1:172.21.15.104:0/1307997835' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2020-09-08T16:31:15.847 INFO:tasks.cephadm.ceph_manager.ceph:need seq 702 got 712 for osd.6 2020-09-08T16:31:15.847 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.7 2020-09-08T16:31:15.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:15 smithi166 bash[9446]: cluster 2020-09-08T16:31:14.539139+0000 mgr.x (mgr.24118) 1917 : cluster [DBG] pgmap v2802: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:15.868 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:15 smithi166 bash[9446]: audit 2020-09-08T16:31:15.150421+0000 mon.a (mon.0) 2641 : audit [DBG] from='client.? v1:172.21.15.104:0/1307997835' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2020-09-08T16:31:15.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:15 smithi055 bash[9296]: cluster 2020-09-08T16:31:14.539139+0000 mgr.x (mgr.24118) 1917 : cluster [DBG] pgmap v2802: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:15.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:15 smithi055 bash[9296]: audit 2020-09-08T16:31:15.150421+0000 mon.a (mon.0) 2641 : audit [DBG] from='client.? v1:172.21.15.104:0/1307997835' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2020-09-08T16:31:15.981 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:31:17.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:17 smithi166 bash[9446]: cluster 2020-09-08T16:31:16.539770+0000 mgr.x (mgr.24118) 1918 : cluster [DBG] pgmap v2803: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:17.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:17 smithi055 bash[9296]: cluster 2020-09-08T16:31:16.539770+0000 mgr.x (mgr.24118) 1918 : cluster [DBG] pgmap v2803: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:17.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:17 smithi104 bash[9203]: cluster 2020-09-08T16:31:16.539770+0000 mgr.x (mgr.24118) 1918 : cluster [DBG] pgmap v2803: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:18.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:18 smithi166 bash[9446]: audit 2020-09-08T16:31:17.621469+0000 mon.b (mon.1) 139 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:31:18.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:18 smithi166 bash[9446]: audit 2020-09-08T16:31:17.623361+0000 mon.a (mon.0) 2642 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:31:18.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:18 smithi166 bash[9446]: audit 2020-09-08T16:31:17.628622+0000 mon.a (mon.0) 2643 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:31:18.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:18 smithi055 bash[9296]: audit 2020-09-08T16:31:17.621469+0000 mon.b (mon.1) 139 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:31:18.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:18 smithi055 bash[9296]: audit 2020-09-08T16:31:17.623361+0000 mon.a (mon.0) 2642 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:31:18.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:18 smithi055 bash[9296]: audit 2020-09-08T16:31:17.628622+0000 mon.a (mon.0) 2643 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:31:18.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:18 smithi104 bash[9203]: audit 2020-09-08T16:31:17.621469+0000 mon.b (mon.1) 139 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:31:18.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:18 smithi104 bash[9203]: audit 2020-09-08T16:31:17.623361+0000 mon.a (mon.0) 2642 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:31:18.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:18 smithi104 bash[9203]: audit 2020-09-08T16:31:17.628622+0000 mon.a (mon.0) 2643 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:31:19.126 INFO:teuthology.orchestra.run.smithi104.stdout:707 2020-09-08T16:31:19.885 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:19 smithi104 bash[9203]: cluster 2020-09-08T16:31:18.540453+0000 mgr.x (mgr.24118) 1919 : cluster [DBG] pgmap v2804: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:19.886 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:19 smithi104 bash[9203]: audit 2020-09-08T16:31:19.133364+0000 mon.a (mon.0) 2644 : audit [DBG] from='client.? v1:172.21.15.104:0/3769363318' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2020-09-08T16:31:19.886 INFO:tasks.cephadm.ceph_manager.ceph:need seq 698 got 707 for osd.7 2020-09-08T16:31:19.887 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.8 2020-09-08T16:31:19.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:19 smithi055 bash[9296]: cluster 2020-09-08T16:31:18.540453+0000 mgr.x (mgr.24118) 1919 : cluster [DBG] pgmap v2804: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:19.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:19 smithi055 bash[9296]: audit 2020-09-08T16:31:19.133364+0000 mon.a (mon.0) 2644 : audit [DBG] from='client.? v1:172.21.15.104:0/3769363318' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2020-09-08T16:31:20.017 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:31:20.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:19 smithi166 bash[9446]: cluster 2020-09-08T16:31:18.540453+0000 mgr.x (mgr.24118) 1919 : cluster [DBG] pgmap v2804: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:20.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:19 smithi166 bash[9446]: audit 2020-09-08T16:31:19.133364+0000 mon.a (mon.0) 2644 : audit [DBG] from='client.? v1:172.21.15.104:0/3769363318' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2020-09-08T16:31:21.026 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:20 smithi104 bash[9203]: cluster 2020-09-08T16:31:19.662313+0000 osd.9 (osd.9) 158 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:21.026 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:20 smithi104 bash[9203]: cluster 2020-09-08T16:31:19.662442+0000 osd.9 (osd.9) 159 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:21.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:20 smithi166 bash[9446]: cluster 2020-09-08T16:31:19.662313+0000 osd.9 (osd.9) 158 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:21.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:20 smithi166 bash[9446]: cluster 2020-09-08T16:31:19.662442+0000 osd.9 (osd.9) 159 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:21.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:20 smithi055 bash[9296]: cluster 2020-09-08T16:31:19.662313+0000 osd.9 (osd.9) 158 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:21.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:20 smithi055 bash[9296]: cluster 2020-09-08T16:31:19.662442+0000 osd.9 (osd.9) 159 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:22.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:21 smithi166 bash[9446]: cluster 2020-09-08T16:31:20.541097+0000 mgr.x (mgr.24118) 1920 : cluster [DBG] pgmap v2805: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:22.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:21 smithi166 bash[9446]: cluster 2020-09-08T16:31:20.644468+0000 mon.a (mon.0) 2645 : cluster [DBG] osdmap e970: 12 total, 12 up, 12 in 2020-09-08T16:31:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:21 smithi055 bash[9296]: cluster 2020-09-08T16:31:20.541097+0000 mgr.x (mgr.24118) 1920 : cluster [DBG] pgmap v2805: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:22.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:21 smithi055 bash[9296]: cluster 2020-09-08T16:31:20.644468+0000 mon.a (mon.0) 2645 : cluster [DBG] osdmap e970: 12 total, 12 up, 12 in 2020-09-08T16:31:22.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:21 smithi104 bash[9203]: cluster 2020-09-08T16:31:20.541097+0000 mgr.x (mgr.24118) 1920 : cluster [DBG] pgmap v2805: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:22.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:21 smithi104 bash[9203]: cluster 2020-09-08T16:31:20.644468+0000 mon.a (mon.0) 2645 : cluster [DBG] osdmap e970: 12 total, 12 up, 12 in 2020-09-08T16:31:23.305 INFO:teuthology.orchestra.run.smithi104.stdout:703 2020-09-08T16:31:24.019 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:23 smithi104 bash[9203]: cluster 2020-09-08T16:31:22.541719+0000 mgr.x (mgr.24118) 1921 : cluster [DBG] pgmap v2807: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:24.020 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:23 smithi104 bash[9203]: audit 2020-09-08T16:31:23.311497+0000 mon.a (mon.0) 2646 : audit [DBG] from='client.? v1:172.21.15.104:0/2903684075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2020-09-08T16:31:24.021 INFO:tasks.cephadm.ceph_manager.ceph:need seq 693 got 703 for osd.8 2020-09-08T16:31:24.022 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.9 2020-09-08T16:31:24.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:23 smithi166 bash[9446]: cluster 2020-09-08T16:31:22.541719+0000 mgr.x (mgr.24118) 1921 : cluster [DBG] pgmap v2807: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:24.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:23 smithi166 bash[9446]: audit 2020-09-08T16:31:23.311497+0000 mon.a (mon.0) 2646 : audit [DBG] from='client.? v1:172.21.15.104:0/2903684075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2020-09-08T16:31:24.153 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:31:24.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:23 smithi055 bash[9296]: cluster 2020-09-08T16:31:22.541719+0000 mgr.x (mgr.24118) 1921 : cluster [DBG] pgmap v2807: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:24.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:23 smithi055 bash[9296]: audit 2020-09-08T16:31:23.311497+0000 mon.a (mon.0) 2646 : audit [DBG] from='client.? v1:172.21.15.104:0/2903684075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2020-09-08T16:31:25.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:24 smithi166 bash[9446]: cluster 2020-09-08T16:31:23.761857+0000 osd.2 (osd.2) 154 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:25.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:24 smithi166 bash[9446]: cluster 2020-09-08T16:31:23.761981+0000 osd.2 (osd.2) 155 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:25.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:24 smithi055 bash[9296]: cluster 2020-09-08T16:31:23.761857+0000 osd.2 (osd.2) 154 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:25.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:24 smithi055 bash[9296]: cluster 2020-09-08T16:31:23.761981+0000 osd.2 (osd.2) 155 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:25.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:24 smithi104 bash[9203]: cluster 2020-09-08T16:31:23.761857+0000 osd.2 (osd.2) 154 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:25.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:24 smithi104 bash[9203]: cluster 2020-09-08T16:31:23.761981+0000 osd.2 (osd.2) 155 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:26.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:25 smithi166 bash[9446]: cluster 2020-09-08T16:31:24.542389+0000 mgr.x (mgr.24118) 1922 : cluster [DBG] pgmap v2808: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:26.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:25 smithi166 bash[9446]: cluster 2020-09-08T16:31:24.712057+0000 mon.a (mon.0) 2647 : cluster [DBG] osdmap e971: 12 total, 12 up, 12 in 2020-09-08T16:31:26.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:25 smithi055 bash[9296]: cluster 2020-09-08T16:31:24.542389+0000 mgr.x (mgr.24118) 1922 : cluster [DBG] pgmap v2808: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:26.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:25 smithi055 bash[9296]: cluster 2020-09-08T16:31:24.712057+0000 mon.a (mon.0) 2647 : cluster [DBG] osdmap e971: 12 total, 12 up, 12 in 2020-09-08T16:31:26.175 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:25 smithi104 bash[9203]: cluster 2020-09-08T16:31:24.542389+0000 mgr.x (mgr.24118) 1922 : cluster [DBG] pgmap v2808: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:26.176 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:25 smithi104 bash[9203]: cluster 2020-09-08T16:31:24.712057+0000 mon.a (mon.0) 2647 : cluster [DBG] osdmap e971: 12 total, 12 up, 12 in 2020-09-08T16:31:27.075 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:26 smithi104 bash[9203]: cluster 2020-09-08T16:31:25.747523+0000 osd.7 (osd.7) 216 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:31:27.118 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:26 smithi166 bash[9446]: cluster 2020-09-08T16:31:25.747523+0000 osd.7 (osd.7) 216 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:31:27.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:26 smithi055 bash[9296]: cluster 2020-09-08T16:31:25.747523+0000 osd.7 (osd.7) 216 : cluster [DBG] 1.0 scrub starts 2020-09-08T16:31:27.487 INFO:teuthology.orchestra.run.smithi104.stdout:695 2020-09-08T16:31:27.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:27 smithi104 bash[9203]: cluster 2020-09-08T16:31:25.890671+0000 osd.7 (osd.7) 217 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:31:27.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:27 smithi104 bash[9203]: cluster 2020-09-08T16:31:26.542985+0000 mgr.x (mgr.24118) 1923 : cluster [DBG] pgmap v2810: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:27.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:27 smithi104 bash[9203]: audit 2020-09-08T16:31:27.494477+0000 mon.a (mon.0) 2648 : audit [DBG] from='client.? v1:172.21.15.104:0/172256787' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2020-09-08T16:31:28.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:27 smithi166 bash[9446]: cluster 2020-09-08T16:31:25.890671+0000 osd.7 (osd.7) 217 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:31:28.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:27 smithi166 bash[9446]: cluster 2020-09-08T16:31:26.542985+0000 mgr.x (mgr.24118) 1923 : cluster [DBG] pgmap v2810: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:28.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:27 smithi166 bash[9446]: audit 2020-09-08T16:31:27.494477+0000 mon.a (mon.0) 2648 : audit [DBG] from='client.? v1:172.21.15.104:0/172256787' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2020-09-08T16:31:28.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:27 smithi055 bash[9296]: cluster 2020-09-08T16:31:25.890671+0000 osd.7 (osd.7) 217 : cluster [DBG] 1.0 scrub ok 2020-09-08T16:31:28.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:27 smithi055 bash[9296]: cluster 2020-09-08T16:31:26.542985+0000 mgr.x (mgr.24118) 1923 : cluster [DBG] pgmap v2810: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:28.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:27 smithi055 bash[9296]: audit 2020-09-08T16:31:27.494477+0000 mon.a (mon.0) 2648 : audit [DBG] from='client.? v1:172.21.15.104:0/172256787' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2020-09-08T16:31:28.192 INFO:tasks.cephadm.ceph_manager.ceph:need seq 686 got 695 for osd.9 2020-09-08T16:31:28.193 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.10 2020-09-08T16:31:28.325 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:31:29.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:29 smithi104 bash[9203]: cluster 2020-09-08T16:31:28.543656+0000 mgr.x (mgr.24118) 1924 : cluster [DBG] pgmap v2811: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:30.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:29 smithi166 bash[9446]: cluster 2020-09-08T16:31:28.543656+0000 mgr.x (mgr.24118) 1924 : cluster [DBG] pgmap v2811: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:30.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:29 smithi055 bash[9296]: cluster 2020-09-08T16:31:28.543656+0000 mgr.x (mgr.24118) 1924 : cluster [DBG] pgmap v2811: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:31.477 INFO:teuthology.orchestra.run.smithi104.stdout:694 2020-09-08T16:31:31.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:31 smithi166 bash[9446]: cluster 2020-09-08T16:31:30.544379+0000 mgr.x (mgr.24118) 1925 : cluster [DBG] pgmap v2812: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:31.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:31 smithi166 bash[9446]: cluster 2020-09-08T16:31:30.733442+0000 osd.0 (osd.0) 170 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:31.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:31 smithi166 bash[9446]: cluster 2020-09-08T16:31:30.733578+0000 osd.0 (osd.0) 171 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:31.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:31 smithi055 bash[9296]: cluster 2020-09-08T16:31:30.544379+0000 mgr.x (mgr.24118) 1925 : cluster [DBG] pgmap v2812: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:31.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:31 smithi055 bash[9296]: cluster 2020-09-08T16:31:30.733442+0000 osd.0 (osd.0) 170 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:31.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:31 smithi055 bash[9296]: cluster 2020-09-08T16:31:30.733578+0000 osd.0 (osd.0) 171 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:31.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:31 smithi104 bash[9203]: cluster 2020-09-08T16:31:30.544379+0000 mgr.x (mgr.24118) 1925 : cluster [DBG] pgmap v2812: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:31.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:31 smithi104 bash[9203]: cluster 2020-09-08T16:31:30.733442+0000 osd.0 (osd.0) 170 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:31.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:31 smithi104 bash[9203]: cluster 2020-09-08T16:31:30.733578+0000 osd.0 (osd.0) 171 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:32.238 INFO:tasks.cephadm.ceph_manager.ceph:need seq 685 got 694 for osd.10 2020-09-08T16:31:32.239 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph osd last-stat-seq osd.11 2020-09-08T16:31:32.371 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:31:32.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:32 smithi104 bash[9203]: cluster 2020-09-08T16:31:31.243806+0000 osd.7 (osd.7) 218 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:32.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:32 smithi104 bash[9203]: cluster 2020-09-08T16:31:31.243953+0000 osd.7 (osd.7) 219 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:32.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:32 smithi104 bash[9203]: cluster 2020-09-08T16:31:31.472545+0000 mon.a (mon.0) 2649 : cluster [DBG] osdmap e972: 12 total, 12 up, 12 in 2020-09-08T16:31:32.676 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:32 smithi104 bash[9203]: audit 2020-09-08T16:31:31.484099+0000 mon.a (mon.0) 2650 : audit [DBG] from='client.? v1:172.21.15.104:0/4233660004' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2020-09-08T16:31:32.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:32 smithi166 bash[9446]: cluster 2020-09-08T16:31:31.243806+0000 osd.7 (osd.7) 218 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:32.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:32 smithi166 bash[9446]: cluster 2020-09-08T16:31:31.243953+0000 osd.7 (osd.7) 219 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:32.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:32 smithi166 bash[9446]: cluster 2020-09-08T16:31:31.472545+0000 mon.a (mon.0) 2649 : cluster [DBG] osdmap e972: 12 total, 12 up, 12 in 2020-09-08T16:31:32.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:32 smithi166 bash[9446]: audit 2020-09-08T16:31:31.484099+0000 mon.a (mon.0) 2650 : audit [DBG] from='client.? v1:172.21.15.104:0/4233660004' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2020-09-08T16:31:32.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:32 smithi055 bash[9296]: cluster 2020-09-08T16:31:31.243806+0000 osd.7 (osd.7) 218 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:32.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:32 smithi055 bash[9296]: cluster 2020-09-08T16:31:31.243953+0000 osd.7 (osd.7) 219 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:32.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:32 smithi055 bash[9296]: cluster 2020-09-08T16:31:31.472545+0000 mon.a (mon.0) 2649 : cluster [DBG] osdmap e972: 12 total, 12 up, 12 in 2020-09-08T16:31:32.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:32 smithi055 bash[9296]: audit 2020-09-08T16:31:31.484099+0000 mon.a (mon.0) 2650 : audit [DBG] from='client.? v1:172.21.15.104:0/4233660004' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2020-09-08T16:31:33.675 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:33 smithi104 bash[9203]: cluster 2020-09-08T16:31:32.545054+0000 mgr.x (mgr.24118) 1926 : cluster [DBG] pgmap v2814: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:33.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:33 smithi166 bash[9446]: cluster 2020-09-08T16:31:32.545054+0000 mgr.x (mgr.24118) 1926 : cluster [DBG] pgmap v2814: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:33.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:33 smithi055 bash[9296]: cluster 2020-09-08T16:31:32.545054+0000 mgr.x (mgr.24118) 1926 : cluster [DBG] pgmap v2814: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:35.578 INFO:teuthology.orchestra.run.smithi104.stdout:688 2020-09-08T16:31:35.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:35 smithi166 bash[9446]: cluster 2020-09-08T16:31:34.545705+0000 mgr.x (mgr.24118) 1927 : cluster [DBG] pgmap v2815: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:35.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:35 smithi166 bash[9446]: audit 2020-09-08T16:31:35.584628+0000 mon.a (mon.0) 2651 : audit [DBG] from='client.? v1:172.21.15.104:0/1972981718' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2020-09-08T16:31:35.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:35 smithi055 bash[9296]: cluster 2020-09-08T16:31:34.545705+0000 mgr.x (mgr.24118) 1927 : cluster [DBG] pgmap v2815: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:35.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:35 smithi055 bash[9296]: audit 2020-09-08T16:31:35.584628+0000 mon.a (mon.0) 2651 : audit [DBG] from='client.? v1:172.21.15.104:0/1972981718' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2020-09-08T16:31:35.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:35 smithi104 bash[9203]: cluster 2020-09-08T16:31:34.545705+0000 mgr.x (mgr.24118) 1927 : cluster [DBG] pgmap v2815: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:35.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:35 smithi104 bash[9203]: audit 2020-09-08T16:31:35.584628+0000 mon.a (mon.0) 2651 : audit [DBG] from='client.? v1:172.21.15.104:0/1972981718' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2020-09-08T16:31:36.309 INFO:tasks.cephadm.ceph_manager.ceph:need seq 678 got 688 for osd.11 2020-09-08T16:31:36.309 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2020-09-08T16:31:36.310 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph pg dump --format=json 2020-09-08T16:31:36.437 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:31:37.651 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:37 smithi104 bash[9203]: cluster 2020-09-08T16:31:36.546372+0000 mgr.x (mgr.24118) 1928 : cluster [DBG] pgmap v2816: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:37.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:37 smithi166 bash[9446]: cluster 2020-09-08T16:31:36.546372+0000 mgr.x (mgr.24118) 1928 : cluster [DBG] pgmap v2816: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:37.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:37 smithi055 bash[9296]: cluster 2020-09-08T16:31:36.546372+0000 mgr.x (mgr.24118) 1928 : cluster [DBG] pgmap v2816: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:39.495 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T16:31:39.498 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2020-09-08T16:31:39.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:39 smithi166 bash[9446]: cluster 2020-09-08T16:31:38.547023+0000 mgr.x (mgr.24118) 1929 : cluster [DBG] pgmap v2817: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:39.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:39 smithi055 bash[9296]: cluster 2020-09-08T16:31:38.547023+0000 mgr.x (mgr.24118) 1929 : cluster [DBG] pgmap v2817: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:39.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:39 smithi104 bash[9203]: cluster 2020-09-08T16:31:38.547023+0000 mgr.x (mgr.24118) 1929 : cluster [DBG] pgmap v2817: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:40.246 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":2817,"stamp":"2020-09-08T16:31:38.546681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":12,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":1124941824,"kb_used":4497552,"kb_used_data":8688,"kb_used_omap":0,"kb_used_meta":4488768,"kb_avail":1120444272,"statfs":{"total":1151940427776,"available":1147334934528,"internally_reserved":0,"allocated":8896512,"data_stored":5219764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4596498432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003358"},"pg_stats":[{"pgid":"1.0","version":"99'6","reported_seq":"1055","reported_epoch":"971","state":"active+clean","last_fresh":"2020-09-08T16:31:25.890810+0000","last_change":"2020-09-08T16:31:25.890810+0000","last_active":"2020-09-08T16:31:25.890810+0000","last_peered":"2020-09-08T16:31:25.890810+0000","last_clean":"2020-09-08T16:31:25.890810+0000","last_became_active":"2020-09-08T15:34:43.031697+0000","last_became_peered":"2020-09-08T15:34:43.031697+0000","last_unstale":"2020-09-08T16:31:25.890810+0000","last_undegraded":"2020-09-08T16:31:25.890810+0000","last_fullsized":"2020-09-08T16:31:25.890810+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":2,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'6","last_scrub_stamp":"2020-09-08T16:31:25.890673+0000","last_deep_scrub":"99'6","last_deep_scrub_stamp":"2020-09-08T15:35:23.132465+0000","last_clean_scrub_stamp":"2020-09-08T16:31:25.890673+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":2,"acting":2,"num_store_stats":7}],"osd_stats":[{"osd":11,"up_from":0,"seq":689,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":488348,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":487616,"kb_avail":93256804,"statfs":{"total":95995035648,"available":95494967296,"internally_reserved":0,"allocated":741376,"data_stored":434967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":499318784},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 16:31:37 2020","interfaces":[{"interface":"back","average":{"1min":2.5059999999999998,"5min":2.363,"15min":2.226},"min":{"1min":1.504,"5min":1.1379999999999999,"15min":1.03},"max":{"1min":4.383,"5min":5.7190000000000003,"15min":5.7190000000000003},"last":1.696},{"interface":"front","average":{"1min":2.29,"5min":2.274,"15min":2.2130000000000001},"min":{"1min":1.4350000000000001,"5min":1.423,"15min":1.1000000000000001},"max":{"1min":3.7269999999999999,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":2.6509999999999998}]},{"osd":1,"last update":"Tue Sep 8 16:31:37 2020","interfaces":[{"interface":"back","average":{"1min":2.855,"5min":2.8450000000000002,"15min":2.8799999999999999},"min":{"1min":1.4890000000000001,"5min":1.409,"15min":1.4019999999999999},"max":{"1min":5.3029999999999999,"5min":5.3029999999999999,"15min":5.3029999999999999},"last":2.5590000000000002},{"interface":"front","average":{"1min":2.5800000000000001,"5min":2.419,"15min":2.4300000000000002},"min":{"1min":1.5329999999999999,"5min":1.4890000000000001,"15min":1.25},"max":{"1min":3.5670000000000002,"5min":4.2430000000000003,"15min":5.3339999999999996},"last":1.7549999999999999}]},{"osd":2,"last update":"Tue Sep 8 16:31:04 2020","interfaces":[{"interface":"back","average":{"1min":2.0190000000000001,"5min":2.1829999999999998,"15min":2.214},"min":{"1min":1.421,"5min":1.2410000000000001,"15min":1.2410000000000001},"max":{"1min":2.75,"5min":4.681,"15min":4.681},"last":1.599},{"interface":"front","average":{"1min":2.411,"5min":2.649,"15min":2.6400000000000001},"min":{"1min":1.536,"5min":1.536,"15min":1.5229999999999999},"max":{"1min":3.5259999999999998,"5min":4.1680000000000001,"15min":4.75},"last":2.1469999999999998}]},{"osd":3,"last update":"Tue Sep 8 16:31:37 2020","interfaces":[{"interface":"back","average":{"1min":2.8439999999999999,"5min":2.8100000000000001,"15min":2.7930000000000001},"min":{"1min":1.7909999999999999,"5min":1.415,"15min":1.415},"max":{"1min":3.7989999999999999,"5min":4.8659999999999997,"15min":4.9199999999999999},"last":2.2890000000000001},{"interface":"front","average":{"1min":3.0089999999999999,"5min":3.0329999999999999,"15min":3.0369999999999999},"min":{"1min":2.2330000000000001,"5min":1.802,"15min":1.76},"max":{"1min":3.7040000000000002,"5min":4.4189999999999996,"15min":4.9800000000000004},"last":2.9969999999999999}]},{"osd":4,"last update":"Tue Sep 8 16:04:50 2020","interfaces":[{"interface":"back","average":{"1min":2.3100000000000001,"5min":2.1019999999999999,"15min":2.2559999999999998},"min":{"1min":0.81499999999999995,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":4.9720000000000004,"5min":6.1029999999999998,"15min":14.911},"last":3.4510000000000001},{"interface":"front","average":{"1min":2.4889999999999999,"5min":2.218,"15min":2.5409999999999999},"min":{"1min":1.1479999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":5.6429999999999998,"5min":6.2770000000000001,"15min":15.462999999999999},"last":3.9319999999999999}]},{"osd":5,"last update":"Tue Sep 8 16:31:37 2020","interfaces":[{"interface":"back","average":{"1min":3.4969999999999999,"5min":3.4620000000000002,"15min":3.4700000000000002},"min":{"1min":1.9379999999999999,"5min":1.5469999999999999,"15min":1.494},"max":{"1min":4.9550000000000001,"5min":5.2809999999999997,"15min":5.2809999999999997},"last":3.0750000000000002},{"interface":"front","average":{"1min":3.8530000000000002,"5min":3.8450000000000002,"15min":3.8650000000000002},"min":{"1min":1.788,"5min":1.6639999999999999,"15min":1.6639999999999999},"max":{"1min":5.3540000000000001,"5min":5.3540000000000001,"15min":6.2759999999999998},"last":3.4159999999999999}]},{"osd":6,"last update":"Tue Sep 8 16:31:37 2020","interfaces":[{"interface":"back","average":{"1min":3.145,"5min":3.169,"15min":3.1779999999999999},"min":{"1min":2.0310000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"max":{"1min":4.4329999999999998,"5min":4.7590000000000003,"15min":5.8049999999999997},"last":2.706},{"interface":"front","average":{"1min":3.8900000000000001,"5min":3.9060000000000001,"15min":3.9249999999999998},"min":{"1min":2.7999999999999998,"5min":2.391,"15min":2.391},"max":{"1min":5.3719999999999999,"5min":5.4050000000000002,"15min":5.8200000000000003},"last":3.343}]},{"osd":7,"last update":"Tue Sep 8 16:31:37 2020","interfaces":[{"interface":"back","average":{"1min":4.21,"5min":4.1120000000000001,"15min":3.9289999999999998},"min":{"1min":2.6259999999999999,"5min":2.1280000000000001,"15min":2.1280000000000001},"max":{"1min":5.2999999999999998,"5min":6.3179999999999996,"15min":6.3179999999999996},"last":3.6629999999999998},{"interface":"front","average":{"1min":3.6579999999999999,"5min":3.706,"15min":3.9159999999999999},"min":{"1min":2.2519999999999998,"5min":1.605,"15min":1.605},"max":{"1min":5.1710000000000003,"5min":5.8120000000000003,"15min":5.8949999999999996},"last":3.1520000000000001}]},{"osd":8,"last update":"Tue Sep 8 16:30:43 2020","interfaces":[{"interface":"back","average":{"1min":4.2859999999999996,"5min":4.4219999999999997,"15min":4.3609999999999998},"min":{"1min":3.1200000000000001,"5min":1.167,"15min":1.167},"max":{"1min":5.9370000000000003,"5min":6.46,"15min":6.46},"last":3.8679999999999999},{"interface":"front","average":{"1min":4.0149999999999997,"5min":4.1660000000000004,"15min":4.2590000000000003},"min":{"1min":2.6949999999999998,"5min":1.6140000000000001,"15min":1.6140000000000001},"max":{"1min":5.7389999999999999,"5min":6.3799999999999999,"15min":6.649},"last":3.5720000000000001}]},{"osd":9,"last update":"Tue Sep 8 16:31:04 2020","interfaces":[{"interface":"back","average":{"1min":4.6269999999999998,"5min":4.6740000000000004,"15min":4.7679999999999998},"min":{"1min":3.3959999999999999,"5min":1.3240000000000001,"15min":1.3240000000000001},"max":{"1min":6.351,"5min":6.4850000000000003,"15min":6.6849999999999996},"last":4.2389999999999999},{"interface":"front","average":{"1min":4.2460000000000004,"5min":4.3029999999999999,"15min":4.4100000000000001},"min":{"1min":3.484,"5min":2.3540000000000001,"15min":2.3540000000000001},"max":{"1min":4.9349999999999996,"5min":6.5869999999999997,"15min":7.1630000000000003},"last":3.7589999999999999}]},{"osd":10,"last update":"Tue Sep 8 16:30:43 2020","interfaces":[{"interface":"back","average":{"1min":4.577,"5min":4.7590000000000003,"15min":4.7720000000000002},"min":{"1min":3.3460000000000001,"5min":1.927,"15min":1.927},"max":{"1min":5.665,"5min":6.7069999999999999,"15min":7.0830000000000002},"last":4.0640000000000001},{"interface":"front","average":{"1min":4.8159999999999998,"5min":4.9850000000000003,"15min":5.0220000000000002},"min":{"1min":3.6440000000000001,"5min":2.4700000000000002,"15min":2.4700000000000002},"max":{"1min":6.1479999999999997,"5min":7.0810000000000004,"15min":7.5110000000000001},"last":4.4299999999999997}]}]},{"osd":10,"up_from":0,"seq":696,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":347804,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":347072,"kb_avail":93397348,"statfs":{"total":95995035648,"available":95638884352,"internally_reserved":0,"allocated":741376,"data_stored":434967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":355401728},"hb_peers":[0,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 16:30:52 2020","interfaces":[{"interface":"back","average":{"1min":1.9370000000000001,"5min":2.1120000000000001,"15min":1.9590000000000001},"min":{"1min":1.1679999999999999,"5min":1.133,"15min":1.0469999999999999},"max":{"1min":2.6419999999999999,"5min":3.7599999999999998,"15min":3.9580000000000002},"last":1.5049999999999999},{"interface":"front","average":{"1min":3.254,"5min":3.3719999999999999,"15min":3.1920000000000002},"min":{"1min":1.538,"5min":1.087,"15min":1.087},"max":{"1min":4.8090000000000002,"5min":5.1909999999999998,"15min":5.202},"last":3.0750000000000002}]},{"osd":1,"last update":"Tue Sep 8 16:04:06 2020","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":2.4809999999999999,"15min":2.8860000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":3.2570000000000001,"5min":39.780999999999999,"15min":135.096},"last":2.4430000000000001},{"interface":"front","average":{"1min":1.516,"5min":2.9670000000000001,"15min":2.8119999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":3.3159999999999998,"5min":40.070999999999998,"15min":130.68100000000001},"last":1.968}]},{"osd":2,"last update":"Tue Sep 8 16:31:15 2020","interfaces":[{"interface":"back","average":{"1min":1.907,"5min":2.1139999999999999,"15min":2.0590000000000002},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":3.3340000000000001,"5min":4.4909999999999997,"15min":4.4909999999999997},"last":1.6679999999999999},{"interface":"front","average":{"1min":3.492,"5min":3.669,"15min":3.7010000000000001},"min":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"max":{"1min":5.2359999999999998,"5min":5.6070000000000002,"15min":5.6070000000000002},"last":3.5259999999999998}]},{"osd":3,"last update":"Tue Sep 8 16:31:15 2020","interfaces":[{"interface":"back","average":{"1min":2.298,"5min":2.2970000000000002,"15min":2.2890000000000001},"min":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"max":{"1min":3.4900000000000002,"5min":3.9409999999999998,"15min":4.3819999999999997},"last":2.1629999999999998},{"interface":"front","average":{"1min":1.8089999999999999,"5min":1.994,"15min":2.27},"min":{"1min":1.198,"5min":1.1950000000000001,"15min":1.038},"max":{"1min":2.819,"5min":2.819,"15min":5.1769999999999996},"last":1.8160000000000001}]},{"osd":4,"last update":"Tue Sep 8 16:31:15 2020","interfaces":[{"interface":"back","average":{"1min":3.2549999999999999,"5min":3.125,"15min":3.1779999999999999},"min":{"1min":1.5209999999999999,"5min":1.419,"15min":1.419},"max":{"1min":5.0369999999999999,"5min":5.0369999999999999,"15min":5.3150000000000004},"last":2.5259999999999998},{"interface":"front","average":{"1min":3.8460000000000001,"5min":3.7610000000000001,"15min":3.6829999999999998},"min":{"1min":2.4470000000000001,"5min":2.0019999999999998,"15min":2.0019999999999998},"max":{"1min":5.2210000000000001,"5min":6.0330000000000004,"15min":6.0330000000000004},"last":2.758}]},{"osd":5,"last update":"Tue Sep 8 16:31:15 2020","interfaces":[{"interface":"back","average":{"1min":2.355,"5min":2.355,"15min":2.3900000000000001},"min":{"1min":1.4850000000000001,"5min":1.149,"15min":1.149},"max":{"1min":3.7709999999999999,"5min":3.9289999999999998,"15min":4.3230000000000004},"last":2.2320000000000002},{"interface":"front","average":{"1min":3.101,"5min":3.0870000000000002,"15min":3.0939999999999999},"min":{"1min":1.9370000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"max":{"1min":4.3860000000000001,"5min":5.5999999999999996,"15min":5.5999999999999996},"last":3.1499999999999999}]},{"osd":6,"last update":"Tue Sep 8 16:31:15 2020","interfaces":[{"interface":"back","average":{"1min":3.2370000000000001,"5min":3.1150000000000002,"15min":3.379},"min":{"1min":2.0390000000000001,"5min":1.4630000000000001,"15min":1.377},"max":{"1min":4.3940000000000001,"5min":5.0099999999999998,"15min":5.3979999999999997},"last":2.6829999999999998},{"interface":"front","average":{"1min":2.976,"5min":3.093,"15min":3.427},"min":{"1min":2.2730000000000001,"5min":1.9890000000000001,"15min":1.873},"max":{"1min":3.9750000000000001,"5min":5.194,"15min":6.5609999999999999},"last":3.3090000000000002}]},{"osd":7,"last update":"Tue Sep 8 16:31:15 2020","interfaces":[{"interface":"back","average":{"1min":3.0070000000000001,"5min":3.2160000000000002,"15min":3.5819999999999999},"min":{"1min":1.76,"5min":1.76,"15min":1.542},"max":{"1min":4.508,"5min":4.827,"15min":5.625},"last":2.597},{"interface":"front","average":{"1min":3.4020000000000001,"5min":3.5670000000000002,"15min":3.8610000000000002},"min":{"1min":2.2749999999999999,"5min":1.8380000000000001,"15min":1.5669999999999999},"max":{"1min":4.6109999999999998,"5min":5.0670000000000002,"15min":5.9199999999999999},"last":3.0030000000000001}]},{"osd":8,"last update":"Tue Sep 8 16:31:15 2020","interfaces":[{"interface":"back","average":{"1min":3.6269999999999998,"5min":3.7120000000000002,"15min":4.0430000000000001},"min":{"1min":1.4850000000000001,"5min":1.427,"15min":1.427},"max":{"1min":4.7999999999999998,"5min":5.4589999999999996,"15min":6.2949999999999999},"last":3.3839999999999999},{"interface":"front","average":{"1min":4.149,"5min":4.1470000000000002,"15min":4.1740000000000004},"min":{"1min":2.758,"5min":1.631,"15min":1.631},"max":{"1min":5.6349999999999998,"5min":7.2430000000000003,"15min":7.2430000000000003},"last":3.7109999999999999}]},{"osd":9,"last update":"Tue Sep 8 16:31:15 2020","interfaces":[{"interface":"back","average":{"1min":4.3390000000000004,"5min":4.375,"15min":4.4400000000000004},"min":{"1min":2.8980000000000001,"5min":1.8069999999999999,"15min":1.8069999999999999},"max":{"1min":5.6159999999999997,"5min":6.4160000000000004,"15min":9.3179999999999996},"last":3.8900000000000001},{"interface":"front","average":{"1min":4.6280000000000001,"5min":4.6920000000000002,"15min":4.7210000000000001},"min":{"1min":2.637,"5min":2.1280000000000001,"15min":2.1280000000000001},"max":{"1min":6.9550000000000001,"5min":7.1699999999999999,"15min":7.1699999999999999},"last":4.0750000000000002}]},{"osd":11,"last update":"Tue Sep 8 16:31:04 2020","interfaces":[{"interface":"back","average":{"1min":4.8159999999999998,"5min":4.859,"15min":4.8390000000000004},"min":{"1min":2.847,"5min":2.4209999999999998,"15min":2.4209999999999998},"max":{"1min":5.9950000000000001,"5min":7.5640000000000001,"15min":7.5640000000000001},"last":4.258},{"interface":"front","average":{"1min":5.0270000000000001,"5min":5.0499999999999998,"15min":5.0300000000000002},"min":{"1min":3.1400000000000001,"5min":2.8100000000000001,"15min":2.8100000000000001},"max":{"1min":7.4029999999999996,"5min":7.944,"15min":7.944},"last":4.4290000000000003}]}]},{"osd":9,"up_from":0,"seq":698,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":370140,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":369408,"kb_avail":93375012,"statfs":{"total":95995035648,"available":95616012288,"internally_reserved":0,"allocated":741376,"data_stored":434967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":378273792},"hb_peers":[1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 16:08:52 2020","interfaces":[{"interface":"back","average":{"1min":3.2669999999999999,"5min":2.3730000000000002,"15min":2.3839999999999999},"min":{"1min":0.82699999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":36.694000000000003,"5min":36.694000000000003,"15min":85.481999999999999},"last":2.0619999999999998},{"interface":"front","average":{"1min":3.407,"5min":2.4929999999999999,"15min":2.7570000000000001},"min":{"1min":0.77700000000000002,"5min":0.76100000000000001,"15min":0.60799999999999998},"max":{"1min":37.296999999999997,"5min":37.296999999999997,"15min":85.581999999999994},"last":2.3100000000000001}]},{"osd":1,"last update":"Tue Sep 8 16:30:56 2020","interfaces":[{"interface":"back","average":{"1min":2.1669999999999998,"5min":2.0009999999999999,"15min":1.9810000000000001},"min":{"1min":1.464,"5min":1.135,"15min":1.135},"max":{"1min":2.734,"5min":2.734,"15min":3.669},"last":1.397},{"interface":"front","average":{"1min":2.6429999999999998,"5min":2.3199999999999998,"15min":2.2989999999999999},"min":{"1min":1.905,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":3.1499999999999999,"5min":5.2590000000000003,"15min":5.2590000000000003},"last":2.5979999999999999}]},{"osd":2,"last update":"Tue Sep 8 16:31:34 2020","interfaces":[{"interface":"back","average":{"1min":2.1600000000000001,"5min":2.0600000000000001,"15min":1.9930000000000001},"min":{"1min":1.4830000000000001,"5min":1.153,"15min":1.153},"max":{"1min":2.7709999999999999,"5min":4.1790000000000003,"15min":4.1790000000000003},"last":1.694},{"interface":"front","average":{"1min":3.3210000000000002,"5min":3.3540000000000001,"15min":3.355},"min":{"1min":2.3279999999999998,"5min":2.0209999999999999,"15min":2.0209999999999999},"max":{"1min":4.3959999999999999,"5min":5.5999999999999996,"15min":5.5999999999999996},"last":3.214}]},{"osd":3,"last update":"Tue Sep 8 16:30:56 2020","interfaces":[{"interface":"back","average":{"1min":2.0739999999999998,"5min":2.129,"15min":2.1280000000000001},"min":{"1min":1.383,"5min":1.222,"15min":1.222},"max":{"1min":3.0270000000000001,"5min":3.2090000000000001,"15min":4.5380000000000003},"last":1.835},{"interface":"front","average":{"1min":2.6779999999999999,"5min":2.786,"15min":2.6989999999999998},"min":{"1min":1.4710000000000001,"5min":1.343,"15min":1.343},"max":{"1min":3.9470000000000001,"5min":4.5789999999999997,"15min":4.5789999999999997},"last":2.0899999999999999}]},{"osd":4,"last update":"Tue Sep 8 16:31:34 2020","interfaces":[{"interface":"back","average":{"1min":2.5299999999999998,"5min":2.6120000000000001,"15min":2.6299999999999999},"min":{"1min":1.6779999999999999,"5min":1.145,"15min":1.145},"max":{"1min":3.8340000000000001,"5min":4.5940000000000003,"15min":4.5940000000000003},"last":2.206},{"interface":"front","average":{"1min":2.73,"5min":2.661,"15min":2.6669999999999998},"min":{"1min":1.819,"5min":1.2629999999999999,"15min":1.2629999999999999},"max":{"1min":4.3029999999999999,"5min":4.6520000000000001,"15min":5.6360000000000001},"last":2.2679999999999998}]},{"osd":5,"last update":"Tue Sep 8 16:31:34 2020","interfaces":[{"interface":"back","average":{"1min":3.3900000000000001,"5min":3.2909999999999999,"15min":3.3170000000000002},"min":{"1min":2.1960000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":4.6070000000000002,"5min":6,"15min":6},"last":2.5070000000000001},{"interface":"front","average":{"1min":3.6629999999999998,"5min":3.7069999999999999,"15min":3.7400000000000002},"min":{"1min":2.444,"5min":2.4079999999999999,"15min":2.306},"max":{"1min":4.8319999999999999,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":3.9399999999999999}]},{"osd":6,"last update":"Tue Sep 8 16:31:34 2020","interfaces":[{"interface":"back","average":{"1min":3.1389999999999998,"5min":3.165,"15min":3.23},"min":{"1min":2.141,"5min":1.5469999999999999,"15min":1.5469999999999999},"max":{"1min":4.2199999999999998,"5min":5.6970000000000001,"15min":5.6970000000000001},"last":2.9430000000000001},{"interface":"front","average":{"1min":3.8620000000000001,"5min":3.8879999999999999,"15min":3.879},"min":{"1min":3.1989999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"max":{"1min":4.649,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":3.6760000000000002}]},{"osd":7,"last update":"Tue Sep 8 16:31:34 2020","interfaces":[{"interface":"back","average":{"1min":4.0659999999999998,"5min":3.9529999999999998,"15min":4.032},"min":{"1min":3.133,"5min":1.429,"15min":1.429},"max":{"1min":4.8319999999999999,"5min":6.6369999999999996,"15min":6.6369999999999996},"last":3.5960000000000001},{"interface":"front","average":{"1min":4.5060000000000002,"5min":4.4829999999999997,"15min":4.5510000000000002},"min":{"1min":3.6379999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"max":{"1min":5.21,"5min":7.1399999999999997,"15min":7.1399999999999997},"last":4.7229999999999999}]},{"osd":8,"last update":"Tue Sep 8 16:31:34 2020","interfaces":[{"interface":"back","average":{"1min":4.4489999999999998,"5min":4.2990000000000004,"15min":4.3250000000000002},"min":{"1min":3.3050000000000002,"5min":1.504,"15min":1.504},"max":{"1min":5.3799999999999999,"5min":7.2599999999999998,"15min":7.2599999999999998},"last":2.8639999999999999},{"interface":"front","average":{"1min":4.8730000000000002,"5min":4.7809999999999997,"15min":4.8239999999999998},"min":{"1min":4.0979999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"max":{"1min":5.6319999999999997,"5min":7.5720000000000001,"15min":7.5720000000000001},"last":4.3220000000000001}]},{"osd":10,"last update":"Tue Sep 8 16:31:12 2020","interfaces":[{"interface":"back","average":{"1min":3.8500000000000001,"5min":3.7930000000000001,"15min":3.7970000000000002},"min":{"1min":3.0880000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"max":{"1min":5.1399999999999997,"5min":6.5410000000000004,"15min":6.5410000000000004},"last":3.3100000000000001},{"interface":"front","average":{"1min":4.3760000000000003,"5min":4.4199999999999999,"15min":4.4359999999999999},"min":{"1min":3.3490000000000002,"5min":2.5720000000000001,"15min":2.5720000000000001},"max":{"1min":5.6539999999999999,"5min":7.3860000000000001,"15min":7.3860000000000001},"last":4.032}]},{"osd":11,"last update":"Tue Sep 8 16:31:12 2020","interfaces":[{"interface":"back","average":{"1min":4.2960000000000003,"5min":4.21,"15min":4.258},"min":{"1min":3.5219999999999998,"5min":1.853,"15min":1.853},"max":{"1min":5.3109999999999999,"5min":7.0149999999999997,"15min":7.0149999999999997},"last":4.4249999999999998},{"interface":"front","average":{"1min":4.7779999999999996,"5min":4.7320000000000002,"15min":4.7450000000000001},"min":{"1min":3.9649999999999999,"5min":2.4590000000000001,"15min":2.4590000000000001},"max":{"1min":6.4249999999999998,"5min":7.6849999999999996,"15min":7.6849999999999996},"last":4.8259999999999996}]}]},{"osd":8,"up_from":0,"seq":706,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":276380,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":275648,"kb_avail":93468772,"statfs":{"total":95995035648,"available":95712022528,"internally_reserved":0,"allocated":741376,"data_stored":434967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":282263552},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 16:31:26 2020","interfaces":[{"interface":"back","average":{"1min":2.625,"5min":2.4670000000000001,"15min":2.472},"min":{"1min":1.7230000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":3.573,"5min":4.492,"15min":4.7380000000000004},"last":2.6699999999999999},{"interface":"front","average":{"1min":3.2400000000000002,"5min":2.9119999999999999,"15min":2.8570000000000002},"min":{"1min":2.2050000000000001,"5min":1.6599999999999999,"15min":1.5149999999999999},"max":{"1min":4.2759999999999998,"5min":5.3390000000000004,"15min":5.3390000000000004},"last":2.944}]},{"osd":1,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.492,"5min":2.0659999999999998,"15min":1.8839999999999999},"min":{"1min":1.4690000000000001,"5min":1.137,"15min":1.137},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":1.4350000000000001},{"interface":"front","average":{"1min":2.7200000000000002,"5min":2.427,"15min":2.347},"min":{"1min":1.351,"5min":1.046,"15min":1.046},"max":{"1min":4.444,"5min":4.444,"15min":4.444},"last":1.8600000000000001}]},{"osd":2,"last update":"Tue Sep 8 16:31:26 2020","interfaces":[{"interface":"back","average":{"1min":2.9220000000000002,"5min":2.855,"15min":2.7559999999999998},"min":{"1min":1.9810000000000001,"5min":1.3640000000000001,"15min":1.323},"max":{"1min":3.7229999999999999,"5min":4.2409999999999997,"15min":4.3719999999999999},"last":3.0680000000000001},{"interface":"front","average":{"1min":3.7770000000000001,"5min":3.4409999999999998,"15min":3.2559999999999998},"min":{"1min":2.0640000000000001,"5min":1.3919999999999999,"15min":1.3919999999999999},"max":{"1min":4.4589999999999996,"5min":5.3410000000000002,"15min":5.3410000000000002},"last":3.9620000000000002}]},{"osd":3,"last update":"Tue Sep 8 16:31:26 2020","interfaces":[{"interface":"back","average":{"1min":2.4769999999999999,"5min":2.7959999999999998,"15min":2.9590000000000001},"min":{"1min":1.397,"5min":1.3180000000000001,"15min":1.3180000000000001},"max":{"1min":3.8130000000000002,"5min":4.1669999999999998,"15min":4.8879999999999999},"last":1.51},{"interface":"front","average":{"1min":2.274,"5min":2.556,"15min":2.6349999999999998},"min":{"1min":1.556,"5min":1.556,"15min":1.456},"max":{"1min":4.0519999999999996,"5min":4.7119999999999997,"15min":4.7779999999999996},"last":2.488}]},{"osd":4,"last update":"Tue Sep 8 16:31:26 2020","interfaces":[{"interface":"back","average":{"1min":3.3730000000000002,"5min":3.3300000000000001,"15min":3.4119999999999999},"min":{"1min":1.6830000000000001,"5min":1.6759999999999999,"15min":1.5649999999999999},"max":{"1min":4.3609999999999998,"5min":4.859,"15min":5.585},"last":3.4700000000000002},{"interface":"front","average":{"1min":3.3879999999999999,"5min":3.4009999999999998,"15min":3.391},"min":{"1min":2.2130000000000001,"5min":1.7689999999999999,"15min":1.5389999999999999},"max":{"1min":4.4539999999999997,"5min":5.5800000000000001,"15min":6.3860000000000001},"last":3.3220000000000001}]},{"osd":5,"last update":"Tue Sep 8 16:31:26 2020","interfaces":[{"interface":"back","average":{"1min":2.3330000000000002,"5min":2.3239999999999998,"15min":3.016},"min":{"1min":1.879,"5min":1.6240000000000001,"15min":1.6240000000000001},"max":{"1min":3.2240000000000002,"5min":3.8820000000000001,"15min":5.2320000000000002},"last":2.5960000000000001},{"interface":"front","average":{"1min":3.843,"5min":3.8730000000000002,"15min":3.2450000000000001},"min":{"1min":2.1909999999999998,"5min":1.877,"15min":1.877},"max":{"1min":4.774,"5min":5.9240000000000004,"15min":5.9240000000000004},"last":3.778}]},{"osd":6,"last update":"Tue Sep 8 16:31:26 2020","interfaces":[{"interface":"back","average":{"1min":3.7879999999999998,"5min":3.9369999999999998,"15min":3.883},"min":{"1min":2.3780000000000001,"5min":2.3450000000000002,"15min":1.863},"max":{"1min":5.0510000000000002,"5min":6.4139999999999997,"15min":6.4139999999999997},"last":3.605},{"interface":"front","average":{"1min":4.2709999999999999,"5min":4.3259999999999996,"15min":4.2889999999999997},"min":{"1min":3.1669999999999998,"5min":2.702,"15min":2.2280000000000002},"max":{"1min":5.4649999999999999,"5min":6.6020000000000003,"15min":7.0670000000000002},"last":4.1559999999999997}]},{"osd":7,"last update":"Tue Sep 8 16:31:26 2020","interfaces":[{"interface":"back","average":{"1min":3.9119999999999999,"5min":3.8959999999999999,"15min":4.141},"min":{"1min":3.008,"5min":1.998,"15min":1.998},"max":{"1min":5.665,"5min":5.665,"15min":6.9459999999999997},"last":3.847},{"interface":"front","average":{"1min":4.7510000000000003,"5min":4.7640000000000002,"15min":4.569},"min":{"1min":3.9470000000000001,"5min":3.343,"15min":2.9199999999999999},"max":{"1min":6.0220000000000002,"5min":7.1479999999999997,"15min":7.1660000000000004},"last":4.343}]},{"osd":9,"last update":"Tue Sep 8 16:30:51 2020","interfaces":[{"interface":"back","average":{"1min":2.742,"5min":2.7450000000000001,"15min":2.7719999999999998},"min":{"1min":1.7769999999999999,"5min":1.6850000000000001,"15min":1.6850000000000001},"max":{"1min":3.8969999999999998,"5min":4.9290000000000003,"15min":5.5090000000000003},"last":3.0089999999999999},{"interface":"front","average":{"1min":4.5570000000000004,"5min":4.7229999999999999,"15min":4.6219999999999999},"min":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":1.9039999999999999},"max":{"1min":5.7919999999999998,"5min":7.0590000000000002,"15min":7.0590000000000002},"last":4.7080000000000002}]},{"osd":10,"last update":"Tue Sep 8 16:31:26 2020","interfaces":[{"interface":"back","average":{"1min":4.032,"5min":4.0830000000000002,"15min":4.1130000000000004},"min":{"1min":2.6349999999999998,"5min":2.1640000000000001,"15min":2.1349999999999998},"max":{"1min":4.9370000000000003,"5min":5.8360000000000003,"15min":6.149},"last":4.2069999999999999},{"interface":"front","average":{"1min":3.226,"5min":3.1819999999999999,"15min":3.411},"min":{"1min":2.7149999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"max":{"1min":4.258,"5min":5.4649999999999999,"15min":6.1420000000000003},"last":3.3860000000000001}]},{"osd":11,"last update":"Tue Sep 8 16:31:26 2020","interfaces":[{"interface":"back","average":{"1min":4.1900000000000004,"5min":4.2999999999999998,"15min":4.4870000000000001},"min":{"1min":3.4350000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"max":{"1min":5.3140000000000001,"5min":5.9729999999999999,"15min":6.5309999999999997},"last":4.5910000000000002},{"interface":"front","average":{"1min":5.0369999999999999,"5min":5.0819999999999999,"15min":5.0190000000000001},"min":{"1min":4.3170000000000002,"5min":3.5649999999999999,"15min":2.488},"max":{"1min":6.2530000000000001,"5min":7.4480000000000004,"15min":7.7430000000000003},"last":4.5289999999999999}]}]},{"osd":7,"up_from":0,"seq":711,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":325340,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":324608,"kb_avail":93419812,"statfs":{"total":95995035648,"available":95661887488,"internally_reserved":0,"allocated":741376,"data_stored":434987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":332398592},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:02 2020","interfaces":[{"interface":"back","average":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"min":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"max":{"1min":4.0190000000000001,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":2.2189999999999999},{"interface":"front","average":{"1min":2.6259999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"min":{"1min":1.542,"5min":1.542,"15min":1.542},"max":{"1min":4.5229999999999997,"5min":4.5229999999999997,"15min":4.5229999999999997},"last":2.3079999999999998}]},{"osd":1,"last update":"Tue Sep 8 16:31:13 2020","interfaces":[{"interface":"back","average":{"1min":2.1030000000000002,"5min":2.1949999999999998,"15min":2.1779999999999999},"min":{"1min":1.724,"5min":1.625,"15min":1.129},"max":{"1min":2.6989999999999998,"5min":4.8220000000000001,"15min":4.8220000000000001},"last":1.7509999999999999},{"interface":"front","average":{"1min":2.5209999999999999,"5min":2.3799999999999999,"15min":2.3220000000000001},"min":{"1min":1.6339999999999999,"5min":1.3839999999999999,"15min":0.98999999999999999},"max":{"1min":3.9350000000000001,"5min":4.6239999999999997,"15min":4.6239999999999997},"last":3.0779999999999998}]},{"osd":2,"last update":"Tue Sep 8 16:31:03 2020","interfaces":[{"interface":"back","average":{"1min":2.4729999999999999,"5min":2.3210000000000002,"15min":2.2989999999999999},"min":{"1min":1.6060000000000001,"5min":1.4219999999999999,"15min":1.232},"max":{"1min":3.5259999999999998,"5min":3.754,"15min":4.0330000000000004},"last":2.1720000000000002},{"interface":"front","average":{"1min":3.101,"5min":2.948,"15min":2.8799999999999999},"min":{"1min":1.998,"5min":1.6679999999999999,"15min":1.0649999999999999},"max":{"1min":4.4560000000000004,"5min":4.7839999999999998,"15min":4.9359999999999999},"last":1.9870000000000001}]},{"osd":3,"last update":"Tue Sep 8 16:31:03 2020","interfaces":[{"interface":"back","average":{"1min":2.552,"5min":2.726,"15min":2.7490000000000001},"min":{"1min":1.8220000000000001,"5min":1.643,"15min":1.4990000000000001},"max":{"1min":3.4340000000000002,"5min":4.2080000000000002,"15min":5.1829999999999998},"last":2.98},{"interface":"front","average":{"1min":3.0510000000000002,"5min":3.2330000000000001,"15min":3.254},"min":{"1min":1.889,"5min":1.3799999999999999,"15min":1.3799999999999999},"max":{"1min":4.3230000000000004,"5min":5.0709999999999997,"15min":5.0709999999999997},"last":3.5800000000000001}]},{"osd":4,"last update":"Tue Sep 8 16:31:13 2020","interfaces":[{"interface":"back","average":{"1min":3.4910000000000001,"5min":3.27,"15min":3.2890000000000001},"min":{"1min":2.6349999999999998,"5min":1.5169999999999999,"15min":1.04},"max":{"1min":4.5060000000000002,"5min":5.4900000000000002,"15min":6.0839999999999996},"last":3.1850000000000001},{"interface":"front","average":{"1min":3.9460000000000002,"5min":3.774,"15min":3.774},"min":{"1min":3.0579999999999998,"5min":1.9450000000000001,"15min":1.5289999999999999},"max":{"1min":5.4699999999999998,"5min":5.7759999999999998,"15min":6.8079999999999998},"last":3.7549999999999999}]},{"osd":5,"last update":"Tue Sep 8 16:31:03 2020","interfaces":[{"interface":"back","average":{"1min":4.1719999999999997,"5min":3.996,"15min":3.9769999999999999},"min":{"1min":2.8679999999999999,"5min":2.8069999999999999,"15min":1.423},"max":{"1min":5.7460000000000004,"5min":8.3930000000000007,"15min":8.3930000000000007},"last":3.4780000000000002},{"interface":"front","average":{"1min":4.7960000000000003,"5min":4.6699999999999999,"15min":4.6459999999999999},"min":{"1min":3.6309999999999998,"5min":3.2280000000000002,"15min":1.8500000000000001},"max":{"1min":6.8390000000000004,"5min":8.9130000000000003,"15min":8.9130000000000003},"last":4.056}]},{"osd":6,"last update":"Tue Sep 8 16:31:03 2020","interfaces":[{"interface":"back","average":{"1min":4.5010000000000003,"5min":4.3730000000000002,"15min":4.4219999999999997},"min":{"1min":3.3380000000000001,"5min":2.8119999999999998,"15min":1.325},"max":{"1min":5.8109999999999999,"5min":7.234,"15min":7.234},"last":4.2709999999999999},{"interface":"front","average":{"1min":4.1159999999999997,"5min":3.9209999999999998,"15min":3.823},"min":{"1min":3.056,"5min":2.04,"15min":1.1930000000000001},"max":{"1min":5.6319999999999997,"5min":5.6609999999999996,"15min":5.6609999999999996},"last":4.1079999999999997}]},{"osd":8,"last update":"Tue Sep 8 16:31:27 2020","interfaces":[{"interface":"back","average":{"1min":4.5140000000000002,"5min":4.6050000000000004,"15min":4.5789999999999997},"min":{"1min":2.5979999999999999,"5min":2.5979999999999999,"15min":1.6140000000000001},"max":{"1min":5.9240000000000004,"5min":7.0039999999999996,"15min":7.1470000000000002},"last":5.077},{"interface":"front","average":{"1min":4.2290000000000001,"5min":4.1790000000000003,"15min":4.2880000000000003},"min":{"1min":3.206,"5min":2.492,"15min":1.3520000000000001},"max":{"1min":5.7649999999999997,"5min":6.1459999999999999,"15min":6.585},"last":5.202}]},{"osd":9,"last update":"Tue Sep 8 16:30:58 2020","interfaces":[{"interface":"back","average":{"1min":3.7210000000000001,"5min":3.6389999999999998,"15min":3.6459999999999999},"min":{"1min":2.927,"5min":2.323,"15min":1.629},"max":{"1min":4.2359999999999998,"5min":5.0519999999999996,"15min":6.335},"last":3.2749999999999999},{"interface":"front","average":{"1min":4.3209999999999997,"5min":4.1929999999999996,"15min":4.0640000000000001},"min":{"1min":3.5720000000000001,"5min":2.714,"15min":2.2240000000000002},"max":{"1min":4.8760000000000003,"5min":5.6689999999999996,"15min":6.1920000000000002},"last":3.6779999999999999}]},{"osd":10,"last update":"Tue Sep 8 16:31:13 2020","interfaces":[{"interface":"back","average":{"1min":4.6130000000000004,"5min":4.3899999999999997,"15min":4.6989999999999998},"min":{"1min":3.4049999999999998,"5min":2.1339999999999999,"15min":1.6890000000000001},"max":{"1min":5.8490000000000002,"5min":6.4720000000000004,"15min":7.218},"last":4.7640000000000002},{"interface":"front","average":{"1min":5.0460000000000003,"5min":4.7839999999999998,"15min":4.6100000000000003},"min":{"1min":3.7770000000000001,"5min":2.2200000000000002,"15min":1.5660000000000001},"max":{"1min":6.1710000000000003,"5min":6.827,"15min":7.6200000000000001},"last":5.5650000000000004}]},{"osd":11,"last update":"Tue Sep 8 16:31:27 2020","interfaces":[{"interface":"back","average":{"1min":4.9690000000000003,"5min":4.9160000000000004,"15min":5.0860000000000003},"min":{"1min":3.4390000000000001,"5min":3.4390000000000001,"15min":2.2109999999999999},"max":{"1min":6.1360000000000001,"5min":6.6449999999999996,"15min":7.5739999999999998},"last":5.3949999999999996},{"interface":"front","average":{"1min":5.125,"5min":5.1710000000000003,"15min":5.319},"min":{"1min":3.706,"5min":3.706,"15min":2.379},"max":{"1min":6.4009999999999998,"5min":7.8529999999999998,"15min":7.867},"last":5.7530000000000001}]}]},{"osd":6,"up_from":0,"seq":717,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":252124,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":251392,"kb_avail":93493028,"statfs":{"total":95995035648,"available":95736860672,"internally_reserved":0,"allocated":741376,"data_stored":434987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":257425408},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 16:31:05 2020","interfaces":[{"interface":"back","average":{"1min":2.0720000000000001,"5min":2.1240000000000001,"15min":2.1280000000000001},"min":{"1min":1.7310000000000001,"5min":1.093,"15min":1.093},"max":{"1min":3.0979999999999999,"5min":3.0979999999999999,"15min":3.7360000000000002},"last":2.9830000000000001},{"interface":"front","average":{"1min":3.2440000000000002,"5min":3.1560000000000001,"15min":3.1469999999999998},"min":{"1min":2.2389999999999999,"5min":2.181,"15min":1.8280000000000001},"max":{"1min":4.6559999999999997,"5min":4.8949999999999996,"15min":4.8949999999999996},"last":3.8839999999999999}]},{"osd":1,"last update":"Tue Sep 8 16:31:05 2020","interfaces":[{"interface":"back","average":{"1min":2.3340000000000001,"5min":2.2770000000000001,"15min":2.2160000000000002},"min":{"1min":1.8,"5min":1.6359999999999999,"15min":1.2829999999999999},"max":{"1min":3.5030000000000001,"5min":3.806,"15min":3.806},"last":2.8180000000000001},{"interface":"front","average":{"1min":3.8180000000000001,"5min":3.802,"15min":3.8330000000000002},"min":{"1min":2.5680000000000001,"5min":2.5680000000000001,"15min":2.2709999999999999},"max":{"1min":5.0259999999999998,"5min":5.9349999999999996,"15min":5.9349999999999996},"last":4.6139999999999999}]},{"osd":2,"last update":"Tue Sep 8 16:31:05 2020","interfaces":[{"interface":"back","average":{"1min":2.0630000000000002,"5min":2.1469999999999998,"15min":2.1859999999999999},"min":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":0.99099999999999999},"max":{"1min":2.3410000000000002,"5min":3.5489999999999999,"15min":3.5489999999999999},"last":2.9159999999999999},{"interface":"front","average":{"1min":2.9199999999999999,"5min":3.121,"15min":3.2109999999999999},"min":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"max":{"1min":3.8999999999999999,"5min":6.274,"15min":6.274},"last":4.0430000000000001}]},{"osd":3,"last update":"Tue Sep 8 16:31:05 2020","interfaces":[{"interface":"back","average":{"1min":2.9820000000000002,"5min":2.8420000000000001,"15min":2.9260000000000002},"min":{"1min":2.1099999999999999,"5min":2.0030000000000001,"15min":1.466},"max":{"1min":4.5670000000000002,"5min":4.8479999999999999,"15min":4.9909999999999997},"last":4.2770000000000001},{"interface":"front","average":{"1min":3.802,"5min":3.77,"15min":3.8410000000000002},"min":{"1min":2.4260000000000002,"5min":2.4260000000000002,"15min":1.7669999999999999},"max":{"1min":5.1669999999999998,"5min":5.1669999999999998,"15min":5.8330000000000002},"last":4.7460000000000004}]},{"osd":4,"last update":"Tue Sep 8 16:31:05 2020","interfaces":[{"interface":"back","average":{"1min":3.5190000000000001,"5min":3.4039999999999999,"15min":3.4670000000000001},"min":{"1min":2.3999999999999999,"5min":1.8660000000000001,"15min":1.3740000000000001},"max":{"1min":4.9009999999999998,"5min":6.6740000000000004,"15min":6.6740000000000004},"last":4.375},{"interface":"front","average":{"1min":4.0789999999999997,"5min":4.109,"15min":4.0519999999999996},"min":{"1min":2.7530000000000001,"5min":1.7569999999999999,"15min":1.7569999999999999},"max":{"1min":5.5549999999999997,"5min":9.2210000000000001,"15min":9.2210000000000001},"last":4.8300000000000001}]},{"osd":5,"last update":"Tue Sep 8 16:31:05 2020","interfaces":[{"interface":"back","average":{"1min":3.7650000000000001,"5min":3.597,"15min":3.681},"min":{"1min":2.669,"5min":1.6439999999999999,"15min":1.6439999999999999},"max":{"1min":5.2279999999999998,"5min":5.96,"15min":6},"last":4.5439999999999996},{"interface":"front","average":{"1min":4.8029999999999999,"5min":4.5670000000000002,"15min":4.5940000000000003},"min":{"1min":3.548,"5min":2.726,"15min":2.726},"max":{"1min":6.3899999999999997,"5min":8.6880000000000006,"15min":8.6880000000000006},"last":5.0549999999999997}]},{"osd":7,"last update":"Tue Sep 8 16:31:05 2020","interfaces":[{"interface":"back","average":{"1min":4.0570000000000004,"5min":3.9609999999999999,"15min":4.0659999999999998},"min":{"1min":2.7290000000000001,"5min":1.8149999999999999,"15min":1.621},"max":{"1min":4.8280000000000003,"5min":6.657,"15min":6.657},"last":4.9589999999999996},{"interface":"front","average":{"1min":4.6200000000000001,"5min":4.5629999999999997,"15min":4.6470000000000002},"min":{"1min":3.1459999999999999,"5min":2.794,"15min":2.794},"max":{"1min":6.1150000000000002,"5min":7.4669999999999996,"15min":9.0069999999999997},"last":5.2539999999999996}]},{"osd":8,"last update":"Tue Sep 8 16:31:19 2020","interfaces":[{"interface":"back","average":{"1min":5.1769999999999996,"5min":4.806,"15min":4.7990000000000004},"min":{"1min":3.633,"5min":2.6960000000000002,"15min":2.6960000000000002},"max":{"1min":6.7800000000000002,"5min":7.4610000000000003,"15min":7.4610000000000003},"last":5.4429999999999996},{"interface":"front","average":{"1min":5.4820000000000002,"5min":5.0949999999999998,"15min":5.1040000000000001},"min":{"1min":4.0469999999999997,"5min":2.8900000000000001,"15min":2.8900000000000001},"max":{"1min":6.9390000000000001,"5min":7.9560000000000004,"15min":8.5239999999999991},"last":5.6239999999999997}]},{"osd":9,"last update":"Tue Sep 8 16:31:05 2020","interfaces":[{"interface":"back","average":{"1min":3.3820000000000001,"5min":3.4049999999999998,"15min":3.4809999999999999},"min":{"1min":2.6000000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"max":{"1min":4.2469999999999999,"5min":5.4370000000000003,"15min":5.5209999999999999},"last":4.1950000000000003},{"interface":"front","average":{"1min":4.0069999999999997,"5min":4.0339999999999998,"15min":4.1219999999999999},"min":{"1min":2.8399999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"max":{"1min":4.7160000000000002,"5min":6.1799999999999997,"15min":6.2999999999999998},"last":4.6689999999999996}]},{"osd":10,"last update":"Tue Sep 8 16:31:15 2020","interfaces":[{"interface":"back","average":{"1min":3.6269999999999998,"5min":3.8359999999999999,"15min":3.899},"min":{"1min":2.5590000000000002,"5min":2.3780000000000001,"15min":1.98},"max":{"1min":4.2050000000000001,"5min":6.1349999999999998,"15min":6.1349999999999998},"last":4.4269999999999996},{"interface":"front","average":{"1min":5.4020000000000001,"5min":5.2149999999999999,"15min":5.2560000000000002},"min":{"1min":4,"5min":2.8079999999999998,"15min":2.8079999999999998},"max":{"1min":7.3710000000000004,"5min":8.3249999999999993,"15min":8.3249999999999993},"last":5.8700000000000001}]}]},{"osd":1,"up_from":0,"seq":738,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":581724,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":580992,"kb_avail":93163428,"statfs":{"total":95995035648,"available":95399350272,"internally_reserved":0,"allocated":741376,"data_stored":434987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":594935808},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 16:31:23 2020","interfaces":[{"interface":"back","average":{"1min":2.0369999999999999,"5min":2.0190000000000001,"15min":2.0840000000000001},"min":{"1min":1.2430000000000001,"5min":1.1259999999999999,"15min":0.84399999999999997},"max":{"1min":3.355,"5min":3.5880000000000001,"15min":4.2949999999999999},"last":3.1480000000000001},{"interface":"front","average":{"1min":2.3290000000000002,"5min":2.597,"15min":2.7360000000000002},"min":{"1min":1.091,"5min":1.091,"15min":1.091},"max":{"1min":3.02,"5min":4.2329999999999997,"15min":4.6639999999999997},"last":2.3029999999999999}]},{"osd":2,"last update":"Tue Sep 8 16:30:54 2020","interfaces":[{"interface":"back","average":{"1min":2.8900000000000001,"5min":3.0470000000000002,"15min":3.1030000000000002},"min":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.4279999999999999},"max":{"1min":4.0259999999999998,"5min":4.194,"15min":4.9580000000000002},"last":3.6989999999999998},{"interface":"front","average":{"1min":2.173,"5min":2.1800000000000002,"15min":2.1659999999999999},"min":{"1min":1.242,"5min":1.242,"15min":1.242},"max":{"1min":3.302,"5min":4.1509999999999998,"15min":4.1509999999999998},"last":3.5710000000000002}]},{"osd":3,"last update":"Tue Sep 8 15:51:14 2020","interfaces":[{"interface":"back","average":{"1min":3.2309999999999999,"5min":4.4930000000000003,"15min":6.1150000000000002},"min":{"1min":0.92900000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":10.720000000000001,"5min":39.241999999999997,"15min":39.241999999999997},"last":1.8680000000000001},{"interface":"front","average":{"1min":2.8340000000000001,"5min":4.2370000000000001,"15min":5.6680000000000001},"min":{"1min":0.86899999999999999,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":10.365,"5min":41.124000000000002,"15min":41.124000000000002},"last":2.8799999999999999}]},{"osd":4,"last update":"Tue Sep 8 16:30:54 2020","interfaces":[{"interface":"back","average":{"1min":3.0409999999999999,"5min":3.3300000000000001,"15min":3.1429999999999998},"min":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":1.3560000000000001},"max":{"1min":3.7690000000000001,"5min":4.5890000000000004,"15min":5.0529999999999999},"last":3.3820000000000001},{"interface":"front","average":{"1min":2.7869999999999999,"5min":2.9790000000000001,"15min":3.2509999999999999},"min":{"1min":1.7949999999999999,"5min":1.6830000000000001,"15min":1.6830000000000001},"max":{"1min":3.9220000000000002,"5min":4.9009999999999998,"15min":6.0369999999999999},"last":3.2480000000000002}]},{"osd":5,"last update":"Tue Sep 8 16:31:17 2020","interfaces":[{"interface":"back","average":{"1min":2.218,"5min":2.218,"15min":2.234},"min":{"1min":1.496,"5min":1.4219999999999999,"15min":1.4219999999999999},"max":{"1min":2.9910000000000001,"5min":3.4500000000000002,"15min":4.0510000000000002},"last":2.3759999999999999},{"interface":"front","average":{"1min":2.802,"5min":2.96,"15min":3.008},"min":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"max":{"1min":3.7200000000000002,"5min":4.6150000000000002,"15min":5.3650000000000002},"last":2.4550000000000001}]},{"osd":6,"last update":"Tue Sep 8 16:31:23 2020","interfaces":[{"interface":"back","average":{"1min":3.302,"5min":3.6600000000000001,"15min":3.665},"min":{"1min":2.456,"5min":2.456,"15min":2.0750000000000002},"max":{"1min":4.4669999999999996,"5min":5.4580000000000002,"15min":5.4589999999999996},"last":4.1050000000000004},{"interface":"front","average":{"1min":2.4929999999999999,"5min":2.6099999999999999,"15min":2.7490000000000001},"min":{"1min":1.6839999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"max":{"1min":3.238,"5min":3.9809999999999999,"15min":4.7149999999999999},"last":2.976}]},{"osd":7,"last update":"Tue Sep 8 16:30:54 2020","interfaces":[{"interface":"back","average":{"1min":3.7429999999999999,"5min":3.9289999999999998,"15min":4.048},"min":{"1min":2.3210000000000002,"5min":2.298,"15min":2.298},"max":{"1min":4.8099999999999996,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":4.3680000000000003},{"interface":"front","average":{"1min":2.5950000000000002,"5min":2.8660000000000001,"15min":2.8919999999999999},"min":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"max":{"1min":3.1589999999999998,"5min":5.3070000000000004,"15min":5.641},"last":3.339}]},{"osd":8,"last update":"Tue Sep 8 16:31:23 2020","interfaces":[{"interface":"back","average":{"1min":4.1639999999999997,"5min":4.3929999999999998,"15min":4.1219999999999999},"min":{"1min":2.7719999999999998,"5min":2.6640000000000001,"15min":1.6990000000000001},"max":{"1min":5.6070000000000002,"5min":6.8579999999999997,"15min":6.8579999999999997},"last":4.6609999999999996},{"interface":"front","average":{"1min":3.5179999999999998,"5min":3.7639999999999998,"15min":4.173},"min":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"max":{"1min":5.234,"5min":6.1239999999999997,"15min":6.984},"last":4.0640000000000001}]},{"osd":9,"last update":"Tue Sep 8 16:30:38 2020","interfaces":[{"interface":"back","average":{"1min":4.0960000000000001,"5min":4.4020000000000001,"15min":4.4950000000000001},"min":{"1min":2.9620000000000002,"5min":2.8719999999999999,"15min":2.8719999999999999},"max":{"1min":6.0800000000000001,"5min":6.2060000000000004,"15min":6.6369999999999996},"last":4.9240000000000004},{"interface":"front","average":{"1min":3.8170000000000002,"5min":3.9220000000000002,"15min":3.9340000000000002},"min":{"1min":2.6179999999999999,"5min":2.5179999999999998,"15min":1.9850000000000001},"max":{"1min":5.165,"5min":5.407,"15min":6.1399999999999997},"last":4.7539999999999996}]},{"osd":10,"last update":"Tue Sep 8 16:31:17 2020","interfaces":[{"interface":"back","average":{"1min":3.6520000000000001,"5min":3.9910000000000001,"15min":4.3899999999999997},"min":{"1min":2.6160000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"max":{"1min":4.8239999999999998,"5min":5.79,"15min":6.8070000000000004},"last":4.4050000000000002},{"interface":"front","average":{"1min":3.9700000000000002,"5min":4.2800000000000002,"15min":4.4509999999999996},"min":{"1min":2.8250000000000002,"5min":2.3639999999999999,"15min":2.3639999999999999},"max":{"1min":5.3049999999999997,"5min":6.5449999999999999,"15min":7.2359999999999998},"last":5.3440000000000003}]},{"osd":11,"last update":"Tue Sep 8 16:30:38 2020","interfaces":[{"interface":"back","average":{"1min":3.1709999999999998,"5min":3.3809999999999998,"15min":3.4340000000000002},"min":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"max":{"1min":4.343,"5min":4.7720000000000002,"15min":5.3979999999999997},"last":3.9020000000000001},{"interface":"front","average":{"1min":4.1299999999999999,"5min":4.4059999999999997,"15min":4.1740000000000004},"min":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":1.8160000000000001},"max":{"1min":5.7590000000000003,"5min":6.2789999999999999,"15min":6.2789999999999999},"last":5.0860000000000003}]}]},{"osd":0,"up_from":0,"seq":743,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":366236,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":365504,"kb_avail":93378916,"statfs":{"total":95995035648,"available":95620009984,"internally_reserved":0,"allocated":741376,"data_stored":434987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":374276096},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 8 16:31:25 2020","interfaces":[{"interface":"back","average":{"1min":1.8480000000000001,"5min":2.02,"15min":2.0710000000000002},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":2.7370000000000001,"5min":3.2669999999999999,"15min":3.9180000000000001},"last":2.597},{"interface":"front","average":{"1min":2.3460000000000001,"5min":2.8340000000000001,"15min":2.903},"min":{"1min":1.232,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":3.569,"5min":4.968,"15min":4.968},"last":3.238}]},{"osd":2,"last update":"Tue Sep 8 16:30:38 2020","interfaces":[{"interface":"back","average":{"1min":2.8940000000000001,"5min":3.1110000000000002,"15min":3.1099999999999999},"min":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"max":{"1min":3.8130000000000002,"5min":4.4370000000000003,"15min":4.5830000000000002},"last":2.3159999999999998},{"interface":"front","average":{"1min":1.889,"5min":2.0670000000000002,"15min":2.081},"min":{"1min":1.3100000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":2.4940000000000002,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":2.415}]},{"osd":3,"last update":"Tue Sep 8 16:04:34 2020","interfaces":[{"interface":"back","average":{"1min":4.1349999999999998,"5min":3.25,"15min":4.4749999999999996},"min":{"1min":0.79600000000000004,"5min":0.68000000000000005,"15min":0.64800000000000002},"max":{"1min":18.785,"5min":26.27,"15min":282.916},"last":1.97},{"interface":"front","average":{"1min":3.9870000000000001,"5min":3.492,"15min":3.5760000000000001},"min":{"1min":0.92600000000000005,"5min":0.80600000000000005,"15min":0.69999999999999996},"max":{"1min":18.614000000000001,"5min":26.161000000000001,"15min":103.87},"last":2.3919999999999999}]},{"osd":4,"last update":"Tue Sep 8 16:30:38 2020","interfaces":[{"interface":"back","average":{"1min":2.004,"5min":2.1789999999999998,"15min":2.1970000000000001},"min":{"1min":1.373,"5min":1.373,"15min":1.373},"max":{"1min":2.6829999999999998,"5min":3.8260000000000001,"15min":3.8260000000000001},"last":2.9399999999999999},{"interface":"front","average":{"1min":2.5979999999999999,"5min":2.7410000000000001,"15min":2.7469999999999999},"min":{"1min":1.637,"5min":1.2250000000000001,"15min":1.2250000000000001},"max":{"1min":4.2919999999999998,"5min":4.2919999999999998,"15min":4.2919999999999998},"last":3.5099999999999998}]},{"osd":5,"last update":"Tue Sep 8 16:31:10 2020","interfaces":[{"interface":"back","average":{"1min":2.5870000000000002,"5min":2.8260000000000001,"15min":2.875},"min":{"1min":1.736,"5min":1.6020000000000001,"15min":1.6020000000000001},"max":{"1min":3.415,"5min":4.2439999999999998,"15min":4.7690000000000001},"last":2.5019999999999998},{"interface":"front","average":{"1min":3.4529999999999998,"5min":3.585,"15min":3.637},"min":{"1min":2.0329999999999999,"5min":1.839,"15min":1.839},"max":{"1min":4.3220000000000001,"5min":5.2290000000000001,"15min":5.298},"last":3.7040000000000002}]},{"osd":6,"last update":"Tue Sep 8 16:31:25 2020","interfaces":[{"interface":"back","average":{"1min":3.6930000000000001,"5min":3.8580000000000001,"15min":3.8639999999999999},"min":{"1min":2.222,"5min":1.7170000000000001,"15min":1.7170000000000001},"max":{"1min":4.7960000000000003,"5min":5.2400000000000002,"15min":5.4820000000000002},"last":3.4289999999999998},{"interface":"front","average":{"1min":4.3220000000000001,"5min":4.4779999999999998,"15min":4.476},"min":{"1min":2.585,"5min":2.3010000000000002,"15min":2.3010000000000002},"max":{"1min":6.1079999999999997,"5min":6.3120000000000003,"15min":6.7489999999999997},"last":3.879}]},{"osd":7,"last update":"Tue Sep 8 16:31:10 2020","interfaces":[{"interface":"back","average":{"1min":4.1139999999999999,"5min":4.359,"15min":4.4020000000000001},"min":{"1min":2.5030000000000001,"5min":1.548,"15min":1.548},"max":{"1min":5.5389999999999997,"5min":6.008,"15min":6.4340000000000002},"last":4.3730000000000002},{"interface":"front","average":{"1min":2.6200000000000001,"5min":2.9729999999999999,"15min":2.952},"min":{"1min":1.756,"5min":1.478,"15min":1.478},"max":{"1min":3.4910000000000001,"5min":4.5780000000000003,"15min":4.5780000000000003},"last":4.0179999999999998}]},{"osd":8,"last update":"Tue Sep 8 16:31:25 2020","interfaces":[{"interface":"back","average":{"1min":4.1820000000000004,"5min":4.5069999999999997,"15min":4.5720000000000001},"min":{"1min":2.4249999999999998,"5min":2.0009999999999999,"15min":2.0009999999999999},"max":{"1min":5.8159999999999998,"5min":5.9199999999999999,"15min":6.6879999999999997},"last":4.8259999999999996},{"interface":"front","average":{"1min":4.8179999999999996,"5min":5.0750000000000002,"15min":5.1550000000000002},"min":{"1min":2.7410000000000001,"5min":2.1960000000000002,"15min":2.1960000000000002},"max":{"1min":6.548,"5min":6.8230000000000004,"15min":8.0020000000000007},"last":5.8120000000000003}]},{"osd":9,"last update":"Tue Sep 8 16:30:38 2020","interfaces":[{"interface":"back","average":{"1min":4.6239999999999997,"5min":4.7690000000000001,"15min":4.7880000000000003},"min":{"1min":2.6669999999999998,"5min":2.3799999999999999,"15min":2.3799999999999999},"max":{"1min":5.8819999999999997,"5min":6.4989999999999997,"15min":7.3099999999999996},"last":5.1719999999999997},{"interface":"front","average":{"1min":4.968,"5min":5.202,"15min":5.2610000000000001},"min":{"1min":2.8239999999999998,"5min":2.5409999999999999,"15min":2.5409999999999999},"max":{"1min":6.6189999999999998,"5min":7.2409999999999997,"15min":8.4399999999999995},"last":5.4829999999999997}]},{"osd":10,"last update":"Tue Sep 8 16:31:10 2020","interfaces":[{"interface":"back","average":{"1min":3.2200000000000002,"5min":3.5600000000000001,"15min":3.6949999999999998},"min":{"1min":1.796,"5min":1.796,"15min":1.796},"max":{"1min":5.2140000000000004,"5min":5.4829999999999997,"15min":5.4829999999999997},"last":4.7240000000000002},{"interface":"front","average":{"1min":3.0419999999999998,"5min":3.3929999999999998,"15min":3.548},"min":{"1min":1.905,"5min":1.905,"15min":1.905},"max":{"1min":3.746,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":3.758}]},{"osd":11,"last update":"Tue Sep 8 16:31:25 2020","interfaces":[{"interface":"back","average":{"1min":3.2650000000000001,"5min":3.758,"15min":3.7469999999999999},"min":{"1min":1.996,"5min":1.744,"15min":1.744},"max":{"1min":4.6280000000000001,"5min":5.7039999999999997,"15min":5.7400000000000002},"last":4.1769999999999996},{"interface":"front","average":{"1min":3.431,"5min":3.879,"15min":3.9420000000000002},"min":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"max":{"1min":4.9749999999999996,"5min":5.9000000000000004,"15min":5.9000000000000004},"last":4.4740000000000002}]}]},{"osd":2,"up_from":0,"seq":736,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":256412,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":255680,"kb_avail":93488740,"statfs":{"total":95995035648,"available":95732469760,"internally_reserved":0,"allocated":741376,"data_stored":434987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":261816320},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 16:30:45 2020","interfaces":[{"interface":"back","average":{"1min":2.298,"5min":2.1869999999999998,"15min":2.1869999999999998},"min":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.175},"max":{"1min":3.742,"5min":3.742,"15min":3.742},"last":3.1389999999999998},{"interface":"front","average":{"1min":2.4100000000000001,"5min":2.3319999999999999,"15min":2.3599999999999999},"min":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.22},"max":{"1min":4.4779999999999998,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":2.694}]},{"osd":1,"last update":"Tue Sep 8 16:30:45 2020","interfaces":[{"interface":"back","average":{"1min":2.0409999999999999,"5min":2.1349999999999998,"15min":2.1469999999999998},"min":{"1min":1.468,"5min":1.127,"15min":0.83299999999999996},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.9340000000000002},"last":2.5299999999999998},{"interface":"front","average":{"1min":2.6269999999999998,"5min":2.9369999999999998,"15min":2.9670000000000001},"min":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.47},"max":{"1min":4.2549999999999999,"5min":4.2549999999999999,"15min":4.524},"last":2.3860000000000001}]},{"osd":3,"last update":"Tue Sep 8 16:31:34 2020","interfaces":[{"interface":"back","average":{"1min":2.9319999999999999,"5min":2.8239999999999998,"15min":2.8839999999999999},"min":{"1min":1.825,"5min":1.677,"15min":1.3300000000000001},"max":{"1min":3.8119999999999998,"5min":4.7370000000000001,"15min":4.7370000000000001},"last":2.4399999999999999},{"interface":"front","average":{"1min":3.173,"5min":3.181,"15min":3.2690000000000001},"min":{"1min":1.746,"5min":1.746,"15min":1.667},"max":{"1min":4.1319999999999997,"5min":4.8700000000000001,"15min":4.8700000000000001},"last":3.4500000000000002}]},{"osd":4,"last update":"Tue Sep 8 16:30:39 2020","interfaces":[{"interface":"back","average":{"1min":3.2589999999999999,"5min":3.5419999999999998,"15min":3.5310000000000001},"min":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.0209999999999999},"max":{"1min":5.2679999999999998,"5min":5.4160000000000004,"15min":6.1440000000000001},"last":4.5419999999999998},{"interface":"front","average":{"1min":3.7690000000000001,"5min":3.706,"15min":3.694},"min":{"1min":2.8420000000000001,"5min":2.7200000000000002,"15min":2.198},"max":{"1min":5.1219999999999999,"5min":5.6790000000000003,"15min":6.4480000000000004},"last":3.395}]},{"osd":5,"last update":"Tue Sep 8 16:31:02 2020","interfaces":[{"interface":"back","average":{"1min":3.9940000000000002,"5min":3.9119999999999999,"15min":3.875},"min":{"1min":2.6269999999999998,"5min":2.2370000000000001,"15min":2.2370000000000001},"max":{"1min":5.4619999999999997,"5min":6.7830000000000004,"15min":6.7830000000000004},"last":3.9340000000000002},{"interface":"front","average":{"1min":4.3410000000000002,"5min":4.2249999999999996,"15min":4.2599999999999998},"min":{"1min":3.2210000000000001,"5min":2.6419999999999999,"15min":2.3620000000000001},"max":{"1min":6.5819999999999999,"5min":6.6760000000000002,"15min":6.6760000000000002},"last":3.7040000000000002}]},{"osd":6,"last update":"Tue Sep 8 16:31:34 2020","interfaces":[{"interface":"back","average":{"1min":3.8759999999999999,"5min":3.9079999999999999,"15min":3.9870000000000001},"min":{"1min":2.9870000000000001,"5min":2.649,"15min":1.982},"max":{"1min":7.0250000000000004,"5min":7.0250000000000004,"15min":7.0250000000000004},"last":4.734},{"interface":"front","average":{"1min":4.4109999999999996,"5min":4.423,"15min":4.4729999999999999},"min":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"max":{"1min":7.9370000000000003,"5min":7.9370000000000003,"15min":7.9370000000000003},"last":3.6139999999999999}]},{"osd":7,"last update":"Tue Sep 8 16:31:02 2020","interfaces":[{"interface":"back","average":{"1min":4.1959999999999997,"5min":4.2400000000000002,"15min":4.1909999999999998},"min":{"1min":2.3700000000000001,"5min":2.3700000000000001,"15min":2.0600000000000001},"max":{"1min":6.0720000000000001,"5min":6.0720000000000001,"15min":6.194},"last":4.9690000000000003},{"interface":"front","average":{"1min":4.8490000000000002,"5min":4.5540000000000003,"15min":4.6390000000000002},"min":{"1min":3.589,"5min":3.415,"15min":2.9340000000000002},"max":{"1min":7.1459999999999999,"5min":7.1459999999999999,"15min":7.8710000000000004},"last":3.7730000000000001}]},{"osd":8,"last update":"Tue Sep 8 16:31:02 2020","interfaces":[{"interface":"back","average":{"1min":5.032,"5min":4.681,"15min":4.3949999999999996},"min":{"1min":2.9420000000000002,"5min":2.9420000000000002,"15min":1.8540000000000001},"max":{"1min":7.1769999999999996,"5min":7.1769999999999996,"15min":7.1769999999999996},"last":3.8490000000000002},{"interface":"front","average":{"1min":5.4550000000000001,"5min":5.1050000000000004,"15min":5.0570000000000004},"min":{"1min":3.835,"5min":3.7309999999999999,"15min":2.9790000000000001},"max":{"1min":7.4329999999999998,"5min":7.4329999999999998,"15min":7.4329999999999998},"last":4.1749999999999998}]},{"osd":9,"last update":"Tue Sep 8 16:30:45 2020","interfaces":[{"interface":"back","average":{"1min":5.3440000000000003,"5min":5.1669999999999998,"15min":5.0739999999999998},"min":{"1min":4.2140000000000004,"5min":3.5899999999999999,"15min":2.4769999999999999},"max":{"1min":7.6900000000000004,"5min":7.6900000000000004,"15min":7.6900000000000004},"last":4.0590000000000002},{"interface":"front","average":{"1min":5.5039999999999996,"5min":5.3890000000000002,"15min":5.3630000000000004},"min":{"1min":3.8639999999999999,"5min":3.8639999999999999,"15min":3.218},"max":{"1min":8.1869999999999994,"5min":8.1869999999999994,"15min":8.2219999999999995},"last":4.2539999999999996}]},{"osd":10,"last update":"Tue Sep 8 16:31:02 2020","interfaces":[{"interface":"back","average":{"1min":4.2190000000000003,"5min":4.3689999999999998,"15min":4.3579999999999997},"min":{"1min":2.8530000000000002,"5min":2.8530000000000002,"15min":2.8260000000000001},"max":{"1min":5.3310000000000004,"5min":6.3360000000000003,"15min":6.3360000000000003},"last":4.1219999999999999},{"interface":"front","average":{"1min":4.4930000000000003,"5min":4.5529999999999999,"15min":4.6020000000000003},"min":{"1min":3.052,"5min":2.734,"15min":2.5310000000000001},"max":{"1min":6.4610000000000003,"5min":6.7649999999999997,"15min":6.7649999999999997},"last":4.3470000000000004}]}]},{"osd":3,"up_from":0,"seq":730,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":442460,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":441728,"kb_avail":93302692,"statfs":{"total":95995035648,"available":95541956608,"internally_reserved":0,"allocated":741376,"data_stored":434987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":452329472},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 16:09:01 2020","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.627,"15min":2.7490000000000001},"min":{"1min":0.76800000000000002,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.1680000000000001,"5min":7.1109999999999998,"15min":85.766000000000005},"last":1.9299999999999999},{"interface":"front","average":{"1min":1.6759999999999999,"5min":1.956,"15min":2.7490000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.73499999999999999},"max":{"1min":6.9560000000000004,"5min":6.9560000000000004,"15min":41.011000000000003},"last":2.0110000000000001}]},{"osd":1,"last update":"Tue Sep 8 16:31:08 2020","interfaces":[{"interface":"back","average":{"1min":2.1619999999999999,"5min":2.173,"15min":2.1779999999999999},"min":{"1min":1.655,"5min":1.1040000000000001,"15min":1.1040000000000001},"max":{"1min":2.7130000000000001,"5min":3.1680000000000001,"15min":3.5339999999999998},"last":2.0019999999999998},{"interface":"front","average":{"1min":2.8069999999999999,"5min":2.9319999999999999,"15min":2.7839999999999998},"min":{"1min":1.7210000000000001,"5min":1.3069999999999999,"15min":1.3069999999999999},"max":{"1min":3.7999999999999998,"5min":4.3730000000000002,"15min":4.3920000000000003},"last":2.1179999999999999}]},{"osd":2,"last update":"Tue Sep 8 16:31:28 2020","interfaces":[{"interface":"back","average":{"1min":2.0339999999999998,"5min":2.1680000000000001,"15min":2.2000000000000002},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":2.6549999999999998,"5min":3.1760000000000002,"15min":3.1760000000000002},"last":2.2109999999999999},{"interface":"front","average":{"1min":2.5950000000000002,"5min":2.8420000000000001,"15min":2.907},"min":{"1min":1.589,"5min":1.5569999999999999,"15min":1.5569999999999999},"max":{"1min":3.363,"5min":4.952,"15min":4.952},"last":3.028}]},{"osd":4,"last update":"Tue Sep 8 16:30:42 2020","interfaces":[{"interface":"back","average":{"1min":2.7290000000000001,"5min":2.5649999999999999,"15min":2.702},"min":{"1min":1.7729999999999999,"5min":1.238,"15min":1.238},"max":{"1min":4.3630000000000004,"5min":4.8479999999999999,"15min":4.8479999999999999},"last":2.653},{"interface":"front","average":{"1min":3.0939999999999999,"5min":3.0710000000000002,"15min":3.1779999999999999},"min":{"1min":1.964,"5min":1.875,"15min":1.875},"max":{"1min":4.75,"5min":5.1749999999999998,"15min":5.1749999999999998},"last":3.899}]},{"osd":5,"last update":"Tue Sep 8 16:31:08 2020","interfaces":[{"interface":"back","average":{"1min":2.9940000000000002,"5min":3.0110000000000001,"15min":3.0169999999999999},"min":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"max":{"1min":4.1219999999999999,"5min":4.4249999999999998,"15min":4.8200000000000003},"last":2.871},{"interface":"front","average":{"1min":3.8889999999999998,"5min":3.903,"15min":3.883},"min":{"1min":1.573,"5min":1.573,"15min":1.573},"max":{"1min":5.4580000000000002,"5min":6.7930000000000001,"15min":6.7930000000000001},"last":4.5679999999999996}]},{"osd":6,"last update":"Tue Sep 8 16:30:42 2020","interfaces":[{"interface":"back","average":{"1min":3.528,"5min":3.371,"15min":3.3799999999999999},"min":{"1min":2.8079999999999998,"5min":1.579,"15min":1.579},"max":{"1min":5.04,"5min":5.7649999999999997,"15min":6.3600000000000003},"last":3.1480000000000001},{"interface":"front","average":{"1min":3.9870000000000001,"5min":3.8519999999999999,"15min":3.9039999999999999},"min":{"1min":2.968,"5min":1.788,"15min":1.788},"max":{"1min":5.8390000000000004,"5min":5.9349999999999996,"15min":6.0019999999999998},"last":4.2839999999999998}]},{"osd":7,"last update":"Tue Sep 8 16:31:08 2020","interfaces":[{"interface":"back","average":{"1min":3.2120000000000002,"5min":3.4100000000000001,"15min":3.883},"min":{"1min":1.6899999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"max":{"1min":4.8550000000000004,"5min":5.2560000000000002,"15min":6.0540000000000003},"last":3.641},{"interface":"front","average":{"1min":4.2910000000000004,"5min":4.2919999999999998,"15min":4.0789999999999997},"min":{"1min":3.0680000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"max":{"1min":5.4390000000000001,"5min":5.8739999999999997,"15min":6.9820000000000002},"last":4.8250000000000002}]},{"osd":8,"last update":"Tue Sep 8 16:31:28 2020","interfaces":[{"interface":"back","average":{"1min":3.8759999999999999,"5min":3.9449999999999998,"15min":3.8889999999999998},"min":{"1min":2.4420000000000002,"5min":1.3939999999999999,"15min":1.3939999999999999},"max":{"1min":5.484,"5min":6.2859999999999996,"15min":6.2859999999999996},"last":3.5249999999999999},{"interface":"front","average":{"1min":3.8809999999999998,"5min":4.0279999999999996,"15min":4.1299999999999999},"min":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"max":{"1min":5.5999999999999996,"5min":6.7759999999999998,"15min":6.7759999999999998},"last":4.0579999999999998}]},{"osd":9,"last update":"Tue Sep 8 16:30:42 2020","interfaces":[{"interface":"back","average":{"1min":4.5949999999999998,"5min":4.6210000000000004,"15min":4.6840000000000002},"min":{"1min":2.7029999999999998,"5min":2.585,"15min":2.4159999999999999},"max":{"1min":6.218,"5min":6.3300000000000001,"15min":6.9500000000000002},"last":5.0880000000000001},{"interface":"front","average":{"1min":5.0380000000000003,"5min":5.0839999999999996,"15min":4.9909999999999997},"min":{"1min":3.7839999999999998,"5min":3.0960000000000001,"15min":2.915},"max":{"1min":6.694,"5min":6.8520000000000003,"15min":7.4640000000000004},"last":5.5659999999999998}]},{"osd":10,"last update":"Tue Sep 8 16:31:28 2020","interfaces":[{"interface":"back","average":{"1min":3.6869999999999998,"5min":3.887,"15min":3.8769999999999998},"min":{"1min":1.879,"5min":1.879,"15min":1.879},"max":{"1min":4.7329999999999997,"5min":6.1139999999999999,"15min":6.1139999999999999},"last":4.1239999999999997},{"interface":"front","average":{"1min":4.4509999999999996,"5min":4.5279999999999996,"15min":4.4640000000000004},"min":{"1min":2.9209999999999998,"5min":2.9209999999999998,"15min":2.9209999999999998},"max":{"1min":6.2809999999999997,"5min":7.1779999999999999,"15min":7.5419999999999998},"last":4.3689999999999998}]},{"osd":11,"last update":"Tue Sep 8 16:30:42 2020","interfaces":[{"interface":"back","average":{"1min":4.2439999999999998,"5min":4.2750000000000004,"15min":4.4509999999999996},"min":{"1min":3.4790000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"max":{"1min":6.1520000000000001,"5min":8.1600000000000001,"15min":8.1600000000000001},"last":3.8210000000000002},{"interface":"front","average":{"1min":4.9589999999999996,"5min":4.9240000000000004,"15min":4.7460000000000004},"min":{"1min":3.8519999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"max":{"1min":7.3390000000000004,"5min":7.3390000000000004,"15min":7.3390000000000004},"last":5.3230000000000004}]}]},{"osd":4,"up_from":0,"seq":726,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":347228,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":346496,"kb_avail":93397924,"statfs":{"total":95995035648,"available":95639474176,"internally_reserved":0,"allocated":741376,"data_stored":434987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":354811904},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 16:30:49 2020","interfaces":[{"interface":"back","average":{"1min":2.407,"5min":2.3220000000000001,"15min":2.339},"min":{"1min":1.907,"5min":1.2310000000000001,"15min":1.2290000000000001},"max":{"1min":4.1719999999999997,"5min":5.7969999999999997,"15min":5.7969999999999997},"last":2.1589999999999998},{"interface":"front","average":{"1min":3.9380000000000002,"5min":3.8180000000000001,"15min":3.8159999999999998},"min":{"1min":2.9750000000000001,"5min":1.8859999999999999,"15min":1.8859999999999999},"max":{"1min":4.9710000000000001,"5min":5.9470000000000001,"15min":5.9470000000000001},"last":4.6600000000000001}]},{"osd":1,"last update":"Tue Sep 8 16:30:49 2020","interfaces":[{"interface":"back","average":{"1min":3.105,"5min":3.2250000000000001,"15min":3.2959999999999998},"min":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"max":{"1min":4.3380000000000001,"5min":6.1040000000000001,"15min":6.1040000000000001},"last":3.714},{"interface":"front","average":{"1min":2.5019999999999998,"5min":2.4580000000000002,"15min":2.4870000000000001},"min":{"1min":1.165,"5min":1.165,"15min":0.90800000000000003},"max":{"1min":3.734,"5min":4.101,"15min":4.5129999999999999},"last":2.343}]},{"osd":2,"last update":"Tue Sep 8 16:31:00 2020","interfaces":[{"interface":"back","average":{"1min":3.4969999999999999,"5min":3.5310000000000001,"15min":3.4710000000000001},"min":{"1min":2.419,"5min":1.637,"15min":1.637},"max":{"1min":4.7400000000000002,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":3.2109999999999999},{"interface":"front","average":{"1min":2.7229999999999999,"5min":2.8260000000000001,"15min":2.8460000000000001},"min":{"1min":1.738,"5min":1.5669999999999999,"15min":1.3640000000000001},"max":{"1min":3.75,"5min":4.6580000000000004,"15min":4.6580000000000004},"last":2.7810000000000001}]},{"osd":3,"last update":"Tue Sep 8 16:30:53 2020","interfaces":[{"interface":"back","average":{"1min":2.2919999999999998,"5min":2.4119999999999999,"15min":2.4660000000000002},"min":{"1min":1.24,"5min":1.24,"15min":1.24},"max":{"1min":2.827,"5min":4.8609999999999998,"15min":4.8609999999999998},"last":2.2429999999999999},{"interface":"front","average":{"1min":3.923,"5min":4.0149999999999997,"15min":3.9980000000000002},"min":{"1min":2.1320000000000001,"5min":1.734,"15min":1.734},"max":{"1min":5.1139999999999999,"5min":8.2769999999999992,"15min":8.2769999999999992},"last":4.4409999999999998}]},{"osd":5,"last update":"Tue Sep 8 16:31:00 2020","interfaces":[{"interface":"back","average":{"1min":2.7210000000000001,"5min":2.702,"15min":2.7490000000000001},"min":{"1min":1.417,"5min":1.228,"15min":1.228},"max":{"1min":3.4199999999999999,"5min":4.5010000000000003,"15min":4.5010000000000003},"last":2.6589999999999998},{"interface":"front","average":{"1min":3.4329999999999998,"5min":3.4889999999999999,"15min":3.6499999999999999},"min":{"1min":1.948,"5min":1.794,"15min":1.7390000000000001},"max":{"1min":4.0670000000000002,"5min":6.1600000000000001,"15min":6.9699999999999998},"last":3.0489999999999999}]},{"osd":6,"last update":"Tue Sep 8 16:30:45 2020","interfaces":[{"interface":"back","average":{"1min":4.3970000000000002,"5min":4.2880000000000003,"15min":4.3070000000000004},"min":{"1min":3.2010000000000001,"5min":1.556,"15min":1.556},"max":{"1min":5.9640000000000004,"5min":7.8689999999999998,"15min":7.8689999999999998},"last":4.2190000000000003},{"interface":"front","average":{"1min":3.6629999999999998,"5min":3.6629999999999998,"15min":3.6469999999999998},"min":{"1min":2.573,"5min":1.944,"15min":1.2150000000000001},"max":{"1min":4.3760000000000003,"5min":5.9800000000000004,"15min":7.3220000000000001},"last":4.1509999999999998}]},{"osd":7,"last update":"Tue Sep 8 16:31:00 2020","interfaces":[{"interface":"back","average":{"1min":4.8840000000000003,"5min":4.742,"15min":4.1200000000000001},"min":{"1min":3.4399999999999999,"5min":2.069,"15min":2.0680000000000001},"max":{"1min":5.7930000000000001,"5min":7.4630000000000001,"15min":7.4630000000000001},"last":5.1429999999999998},{"interface":"front","average":{"1min":4.101,"5min":3.9889999999999999,"15min":4.0599999999999996},"min":{"1min":2.2400000000000002,"5min":2.0550000000000002,"15min":1.982},"max":{"1min":4.8040000000000003,"5min":5.6559999999999997,"15min":7.3710000000000004},"last":4.8650000000000002}]},{"osd":8,"last update":"Tue Sep 8 16:30:45 2020","interfaces":[{"interface":"back","average":{"1min":5.0309999999999997,"5min":4.9359999999999999,"15min":4.8949999999999996},"min":{"1min":3.6659999999999999,"5min":2.3759999999999999,"15min":2.3759999999999999},"max":{"1min":6.8250000000000002,"5min":8.4580000000000002,"15min":8.4580000000000002},"last":4.9199999999999999},{"interface":"front","average":{"1min":5.4039999999999999,"5min":5.3819999999999997,"15min":5.226},"min":{"1min":4.1200000000000001,"5min":3.145,"15min":3.0680000000000001},"max":{"1min":7.2240000000000002,"5min":8.7639999999999993,"15min":8.7639999999999993},"last":5.3410000000000002}]},{"osd":9,"last update":"Tue Sep 8 16:31:00 2020","interfaces":[{"interface":"back","average":{"1min":3.5950000000000002,"5min":3.496,"15min":3.6640000000000001},"min":{"1min":1.556,"5min":1.556,"15min":1.4610000000000001},"max":{"1min":4.5019999999999998,"5min":9.3640000000000008,"15min":9.3640000000000008},"last":4.0359999999999996},{"interface":"front","average":{"1min":5.4729999999999999,"5min":5.3280000000000003,"15min":5.29},"min":{"1min":3.8980000000000001,"5min":2.7839999999999998,"15min":2.7839999999999998},"max":{"1min":6.5469999999999997,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":5.585}]},{"osd":10,"last update":"Tue Sep 8 16:31:00 2020","interfaces":[{"interface":"back","average":{"1min":4.2329999999999997,"5min":4.3040000000000003,"15min":4.3869999999999996},"min":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.0529999999999999},"max":{"1min":5.1900000000000004,"5min":6.7569999999999997,"15min":7.8159999999999998},"last":4.5309999999999997},{"interface":"front","average":{"1min":4.5819999999999999,"5min":4.6870000000000003,"15min":4.7549999999999999},"min":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"max":{"1min":5.6260000000000003,"5min":7.282,"15min":8.2330000000000005},"last":5.5309999999999997}]}]},{"osd":5,"up_from":0,"seq":718,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":443356,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":442624,"kb_avail":93301796,"statfs":{"total":95995035648,"available":95541039104,"internally_reserved":0,"allocated":741376,"data_stored":434987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":453246976},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:42:35 2020","interfaces":[{"interface":"back","average":{"1min":2.0920000000000001,"5min":1.629,"15min":1.871},"min":{"1min":0.77500000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":5.4400000000000004,"5min":5.4400000000000004,"15min":5.4400000000000004},"last":0.84399999999999997},{"interface":"front","average":{"1min":2.5139999999999998,"5min":2.246,"15min":2.1469999999999998},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":4.9320000000000004,"5min":7.1340000000000003,"15min":7.1340000000000003},"last":2.258}]},{"osd":1,"last update":"Tue Sep 8 16:31:19 2020","interfaces":[{"interface":"back","average":{"1min":2.2250000000000001,"5min":2.1869999999999998,"15min":2.1880000000000002},"min":{"1min":1.786,"5min":1.1759999999999999,"15min":1.1759999999999999},"max":{"1min":2.677,"5min":2.952,"15min":3.262},"last":2.1059999999999999},{"interface":"front","average":{"1min":3.6240000000000001,"5min":3.5710000000000002,"15min":3.657},"min":{"1min":2.0699999999999998,"5min":1.9590000000000001,"15min":1.409},"max":{"1min":5.2750000000000004,"5min":5.3170000000000002,"15min":5.6070000000000002},"last":3.4590000000000001}]},{"osd":2,"last update":"Tue Sep 8 16:31:13 2020","interfaces":[{"interface":"back","average":{"1min":2.899,"5min":2.5609999999999999,"15min":2.5089999999999999},"min":{"1min":1.8540000000000001,"5min":1.758,"15min":1.6890000000000001},"max":{"1min":4.8460000000000001,"5min":6.2249999999999996,"15min":6.2249999999999996},"last":2.1779999999999999},{"interface":"front","average":{"1min":2.9950000000000001,"5min":3.161,"15min":3.0329999999999999},"min":{"1min":1.851,"5min":1.45,"15min":1.45},"max":{"1min":4.8449999999999998,"5min":5.9550000000000001,"15min":5.9550000000000001},"last":2.4089999999999998}]},{"osd":3,"last update":"Tue Sep 8 16:31:13 2020","interfaces":[{"interface":"back","average":{"1min":2.3860000000000001,"5min":2.3959999999999999,"15min":2.399},"min":{"1min":1.9279999999999999,"5min":1.446,"15min":1.446},"max":{"1min":3.4100000000000001,"5min":3.972,"15min":3.972},"last":2.5129999999999999},{"interface":"front","average":{"1min":3.073,"5min":3.544,"15min":3.5110000000000001},"min":{"1min":2.222,"5min":1.978,"15min":1.3580000000000001},"max":{"1min":4.4349999999999996,"5min":6.4649999999999999,"15min":6.4649999999999999},"last":2.6419999999999999}]},{"osd":4,"last update":"Tue Sep 8 16:31:13 2020","interfaces":[{"interface":"back","average":{"1min":2.8580000000000001,"5min":2.7240000000000002,"15min":2.6800000000000002},"min":{"1min":1.9690000000000001,"5min":1.673,"15min":1.673},"max":{"1min":3.923,"5min":4.8209999999999997,"15min":5.423},"last":2.2370000000000001},{"interface":"front","average":{"1min":3.2690000000000001,"5min":3.1869999999999998,"15min":3.137},"min":{"1min":2.5110000000000001,"5min":1.5649999999999999,"15min":1.3009999999999999},"max":{"1min":5.0179999999999998,"5min":5.4139999999999997,"15min":6.4249999999999998},"last":2.8999999999999999}]},{"osd":6,"last update":"Tue Sep 8 16:30:39 2020","interfaces":[{"interface":"back","average":{"1min":4.3179999999999996,"5min":4.0300000000000002,"15min":3.895},"min":{"1min":3.0550000000000002,"5min":1.7609999999999999,"15min":1.7609999999999999},"max":{"1min":5.7089999999999996,"5min":6.6909999999999998,"15min":6.6909999999999998},"last":4.0730000000000004},{"interface":"front","average":{"1min":4.8049999999999997,"5min":4.4809999999999999,"15min":4.4370000000000003},"min":{"1min":3.6789999999999998,"5min":2.544,"15min":2.1240000000000001},"max":{"1min":6.3940000000000001,"5min":7.9290000000000003,"15min":7.9290000000000003},"last":3.734}]},{"osd":7,"last update":"Tue Sep 8 16:31:13 2020","interfaces":[{"interface":"back","average":{"1min":4.7300000000000004,"5min":4.6239999999999997,"15min":4.6200000000000001},"min":{"1min":3.7189999999999999,"5min":2.556,"15min":2.327},"max":{"1min":6.524,"5min":7.157,"15min":8.3130000000000006},"last":4.2869999999999999},{"interface":"front","average":{"1min":5.4279999999999999,"5min":5.2539999999999996,"15min":5.2270000000000003},"min":{"1min":4.2759999999999998,"5min":3.1269999999999998,"15min":3.1269999999999998},"max":{"1min":7.7759999999999998,"5min":8.1310000000000002,"15min":8.5210000000000008},"last":4.5330000000000004}]},{"osd":8,"last update":"Tue Sep 8 16:31:19 2020","interfaces":[{"interface":"back","average":{"1min":5.5640000000000001,"5min":5.2119999999999997,"15min":5.1369999999999996},"min":{"1min":4.4770000000000003,"5min":3.246,"15min":3.246},"max":{"1min":7.242,"5min":7.415,"15min":7.415},"last":5.5590000000000002},{"interface":"front","average":{"1min":5.9980000000000002,"5min":5.5149999999999997,"15min":5.46},"min":{"1min":4.282,"5min":3.0169999999999999,"15min":3.0169999999999999},"max":{"1min":8.593,"5min":8.593,"15min":8.593},"last":4.75}]},{"osd":9,"last update":"Tue Sep 8 16:30:52 2020","interfaces":[{"interface":"back","average":{"1min":6.1790000000000003,"5min":4.694,"15min":4.1210000000000004},"min":{"1min":4.6539999999999999,"5min":2.7010000000000001,"15min":2.5150000000000001},"max":{"1min":8.2089999999999996,"5min":8.2089999999999996,"15min":8.2089999999999996},"last":4.9619999999999997},{"interface":"front","average":{"1min":3.831,"5min":4.8360000000000003,"15min":5.3710000000000004},"min":{"1min":3.0640000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"max":{"1min":5.5750000000000002,"5min":7.9139999999999997,"15min":8.0969999999999995},"last":3.798}]},{"osd":10,"last update":"Tue Sep 8 16:31:19 2020","interfaces":[{"interface":"back","average":{"1min":6.1980000000000004,"5min":4.5979999999999999,"15min":3.9119999999999999},"min":{"1min":4.407,"5min":2.4169999999999998,"15min":1.9379999999999999},"max":{"1min":8.6229999999999993,"5min":8.6229999999999993,"15min":8.6229999999999993},"last":5.1719999999999997},{"interface":"front","average":{"1min":6.5650000000000004,"5min":5.9210000000000003,"15min":5.6790000000000003},"min":{"1min":5.0720000000000001,"5min":3.6400000000000001,"15min":2.4729999999999999},"max":{"1min":9.0060000000000002,"5min":9.0060000000000002,"15min":9.0060000000000002},"last":5.3579999999999997}]},{"osd":11,"last update":"Tue Sep 8 16:31:13 2020","interfaces":[{"interface":"back","average":{"1min":3.6309999999999998,"5min":3.6309999999999998,"15min":3.7069999999999999},"min":{"1min":3.0449999999999999,"5min":2.1829999999999998,"15min":2.1829999999999998},"max":{"1min":4.3330000000000002,"5min":4.7649999999999997,"15min":4.9820000000000002},"last":3.3050000000000002},{"interface":"front","average":{"1min":4.0810000000000004,"5min":4.0449999999999999,"15min":4.0259999999999998},"min":{"1min":3.3959999999999999,"5min":1.96,"15min":1.96},"max":{"1min":5.3869999999999996,"5min":6.8159999999999998,"15min":7.5110000000000001},"last":4.0179999999999998}]}]}]}} 2020-09-08T16:31:40.251 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph pg dump --format=json 2020-09-08T16:31:40.378 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:31:40.866 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:40 smithi166 bash[9446]: audit 2020-09-08T16:31:39.485283+0000 mgr.x (mgr.24118) 1930 : audit [DBG] from='client.17091 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T16:31:40.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:40 smithi166 bash[9446]: cluster 2020-09-08T16:31:39.988477+0000 osd.5 (osd.5) 176 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:40.867 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:40 smithi166 bash[9446]: cluster 2020-09-08T16:31:39.988609+0000 osd.5 (osd.5) 177 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:40.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:40 smithi055 bash[9296]: audit 2020-09-08T16:31:39.485283+0000 mgr.x (mgr.24118) 1930 : audit [DBG] from='client.17091 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T16:31:40.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:40 smithi055 bash[9296]: cluster 2020-09-08T16:31:39.988477+0000 osd.5 (osd.5) 176 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:40.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:40 smithi055 bash[9296]: cluster 2020-09-08T16:31:39.988609+0000 osd.5 (osd.5) 177 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:40.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:40 smithi104 bash[9203]: audit 2020-09-08T16:31:39.485283+0000 mgr.x (mgr.24118) 1930 : audit [DBG] from='client.17091 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T16:31:40.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:40 smithi104 bash[9203]: cluster 2020-09-08T16:31:39.988477+0000 osd.5 (osd.5) 176 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:40.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:40 smithi104 bash[9203]: cluster 2020-09-08T16:31:39.988609+0000 osd.5 (osd.5) 177 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:41.779 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:41 smithi104 bash[9203]: cluster 2020-09-08T16:31:40.547685+0000 mgr.x (mgr.24118) 1931 : cluster [DBG] pgmap v2818: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:41.780 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:41 smithi104 bash[9203]: cluster 2020-09-08T16:31:40.629072+0000 mon.a (mon.0) 2652 : cluster [DBG] osdmap e973: 12 total, 12 up, 12 in 2020-09-08T16:31:41.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:41 smithi055 bash[9296]: cluster 2020-09-08T16:31:40.547685+0000 mgr.x (mgr.24118) 1931 : cluster [DBG] pgmap v2818: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:42.160 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:41 smithi055 bash[9296]: cluster 2020-09-08T16:31:40.629072+0000 mon.a (mon.0) 2652 : cluster [DBG] osdmap e973: 12 total, 12 up, 12 in 2020-09-08T16:31:42.161 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:41 smithi166 bash[9446]: cluster 2020-09-08T16:31:40.547685+0000 mgr.x (mgr.24118) 1931 : cluster [DBG] pgmap v2818: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:42.162 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:41 smithi166 bash[9446]: cluster 2020-09-08T16:31:40.629072+0000 mon.a (mon.0) 2652 : cluster [DBG] osdmap e973: 12 total, 12 up, 12 in 2020-09-08T16:31:42.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:42 smithi055 bash[9296]: cluster 2020-09-08T16:31:41.684085+0000 osd.10 (osd.10) 155 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:42.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:42 smithi055 bash[9296]: cluster 2020-09-08T16:31:41.684207+0000 osd.10 (osd.10) 156 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:42.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:42 smithi104 bash[9203]: cluster 2020-09-08T16:31:41.684085+0000 osd.10 (osd.10) 155 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:42.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:42 smithi104 bash[9203]: cluster 2020-09-08T16:31:41.684207+0000 osd.10 (osd.10) 156 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:43.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:42 smithi166 bash[9446]: cluster 2020-09-08T16:31:41.684085+0000 osd.10 (osd.10) 155 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:43.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:42 smithi166 bash[9446]: cluster 2020-09-08T16:31:41.684207+0000 osd.10 (osd.10) 156 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:43.633 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:43 smithi104 bash[9203]: cluster 2020-09-08T16:31:42.548355+0000 mgr.x (mgr.24118) 1932 : cluster [DBG] pgmap v2820: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:43.634 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:43 smithi104 bash[9203]: cluster 2020-09-08T16:31:42.642148+0000 mon.a (mon.0) 2653 : cluster [DBG] osdmap e974: 12 total, 12 up, 12 in 2020-09-08T16:31:43.737 INFO:teuthology.orchestra.run.smithi104.stdout: 2020-09-08T16:31:43.739 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2020-09-08T16:31:43.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:43 smithi055 bash[9296]: cluster 2020-09-08T16:31:42.548355+0000 mgr.x (mgr.24118) 1932 : cluster [DBG] pgmap v2820: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:43.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:43 smithi055 bash[9296]: cluster 2020-09-08T16:31:42.642148+0000 mon.a (mon.0) 2653 : cluster [DBG] osdmap e974: 12 total, 12 up, 12 in 2020-09-08T16:31:44.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:43 smithi166 bash[9446]: cluster 2020-09-08T16:31:42.548355+0000 mgr.x (mgr.24118) 1932 : cluster [DBG] pgmap v2820: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:44.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:43 smithi166 bash[9446]: cluster 2020-09-08T16:31:42.642148+0000 mon.a (mon.0) 2653 : cluster [DBG] osdmap e974: 12 total, 12 up, 12 in 2020-09-08T16:31:44.502 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":2822,"stamp":"2020-09-08T16:31:43.648560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":12,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":1124941824,"kb_used":4497660,"kb_used_data":8796,"kb_used_omap":0,"kb_used_meta":4488768,"kb_avail":1120444164,"statfs":{"total":1151940427776,"available":1147334823936,"internally_reserved":0,"allocated":9007104,"data_stored":5287399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4596498432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.102544"},"pg_stats":[{"pgid":"1.0","version":"99'6","reported_seq":"1055","reported_epoch":"971","state":"active+clean","last_fresh":"2020-09-08T16:31:25.890810+0000","last_change":"2020-09-08T16:31:25.890810+0000","last_active":"2020-09-08T16:31:25.890810+0000","last_peered":"2020-09-08T16:31:25.890810+0000","last_clean":"2020-09-08T16:31:25.890810+0000","last_became_active":"2020-09-08T15:34:43.031697+0000","last_became_peered":"2020-09-08T15:34:43.031697+0000","last_unstale":"2020-09-08T16:31:25.890810+0000","last_undegraded":"2020-09-08T16:31:25.890810+0000","last_fullsized":"2020-09-08T16:31:25.890810+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":2,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'6","last_scrub_stamp":"2020-09-08T16:31:25.890673+0000","last_deep_scrub":"99'6","last_deep_scrub_stamp":"2020-09-08T15:35:23.132465+0000","last_clean_scrub_stamp":"2020-09-08T16:31:25.890673+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1770,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":2,"acting":2,"num_store_stats":7}],"osd_stats":[{"osd":11,"up_from":0,"seq":690,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":488372,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":487616,"kb_avail":93256780,"statfs":{"total":95995035648,"available":95494942720,"internally_reserved":0,"allocated":765952,"data_stored":449997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":499318784},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 16:31:37 2020","interfaces":[{"interface":"back","average":{"1min":2.5059999999999998,"5min":2.363,"15min":2.226},"min":{"1min":1.504,"5min":1.1379999999999999,"15min":1.03},"max":{"1min":4.383,"5min":5.7190000000000003,"15min":5.7190000000000003},"last":3.089},{"interface":"front","average":{"1min":2.29,"5min":2.274,"15min":2.2130000000000001},"min":{"1min":1.4350000000000001,"5min":1.423,"15min":1.1000000000000001},"max":{"1min":3.7269999999999999,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":2.3399999999999999}]},{"osd":1,"last update":"Tue Sep 8 16:31:37 2020","interfaces":[{"interface":"back","average":{"1min":2.855,"5min":2.8450000000000002,"15min":2.8799999999999999},"min":{"1min":1.4890000000000001,"5min":1.409,"15min":1.4019999999999999},"max":{"1min":5.3029999999999999,"5min":5.3029999999999999,"15min":5.3029999999999999},"last":1.609},{"interface":"front","average":{"1min":2.5800000000000001,"5min":2.419,"15min":2.4300000000000002},"min":{"1min":1.5329999999999999,"5min":1.4890000000000001,"15min":1.25},"max":{"1min":3.5670000000000002,"5min":4.2430000000000003,"15min":5.3339999999999996},"last":1.921}]},{"osd":2,"last update":"Tue Sep 8 16:31:04 2020","interfaces":[{"interface":"back","average":{"1min":2.0190000000000001,"5min":2.1829999999999998,"15min":2.214},"min":{"1min":1.421,"5min":1.2410000000000001,"15min":1.2410000000000001},"max":{"1min":2.75,"5min":4.681,"15min":4.681},"last":1.831},{"interface":"front","average":{"1min":2.411,"5min":2.649,"15min":2.6400000000000001},"min":{"1min":1.536,"5min":1.536,"15min":1.5229999999999999},"max":{"1min":3.5259999999999998,"5min":4.1680000000000001,"15min":4.75},"last":1.409}]},{"osd":3,"last update":"Tue Sep 8 16:31:37 2020","interfaces":[{"interface":"back","average":{"1min":2.8439999999999999,"5min":2.8100000000000001,"15min":2.7930000000000001},"min":{"1min":1.7909999999999999,"5min":1.415,"15min":1.415},"max":{"1min":3.7989999999999999,"5min":4.8659999999999997,"15min":4.9199999999999999},"last":2.9079999999999999},{"interface":"front","average":{"1min":3.0089999999999999,"5min":3.0329999999999999,"15min":3.0369999999999999},"min":{"1min":2.2330000000000001,"5min":1.802,"15min":1.76},"max":{"1min":3.7040000000000002,"5min":4.4189999999999996,"15min":4.9800000000000004},"last":3.4369999999999998}]},{"osd":4,"last update":"Tue Sep 8 16:04:50 2020","interfaces":[{"interface":"back","average":{"1min":2.3100000000000001,"5min":2.1019999999999999,"15min":2.2559999999999998},"min":{"1min":0.81499999999999995,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":4.9720000000000004,"5min":6.1029999999999998,"15min":14.911},"last":3.4510000000000001},{"interface":"front","average":{"1min":2.4889999999999999,"5min":2.218,"15min":2.5409999999999999},"min":{"1min":1.1479999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":5.6429999999999998,"5min":6.2770000000000001,"15min":15.462999999999999},"last":3.9319999999999999}]},{"osd":5,"last update":"Tue Sep 8 16:31:37 2020","interfaces":[{"interface":"back","average":{"1min":3.4969999999999999,"5min":3.4620000000000002,"15min":3.4700000000000002},"min":{"1min":1.9379999999999999,"5min":1.5469999999999999,"15min":1.494},"max":{"1min":4.9550000000000001,"5min":5.2809999999999997,"15min":5.2809999999999997},"last":2.589},{"interface":"front","average":{"1min":3.8530000000000002,"5min":3.8450000000000002,"15min":3.8650000000000002},"min":{"1min":1.788,"5min":1.6639999999999999,"15min":1.6639999999999999},"max":{"1min":5.3540000000000001,"5min":5.3540000000000001,"15min":6.2759999999999998},"last":3.5259999999999998}]},{"osd":6,"last update":"Tue Sep 8 16:31:37 2020","interfaces":[{"interface":"back","average":{"1min":3.145,"5min":3.169,"15min":3.1779999999999999},"min":{"1min":2.0310000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"max":{"1min":4.4329999999999998,"5min":4.7590000000000003,"15min":5.8049999999999997},"last":2.4950000000000001},{"interface":"front","average":{"1min":3.8900000000000001,"5min":3.9060000000000001,"15min":3.9249999999999998},"min":{"1min":2.7999999999999998,"5min":2.391,"15min":2.391},"max":{"1min":5.3719999999999999,"5min":5.4050000000000002,"15min":5.8200000000000003},"last":3.7970000000000002}]},{"osd":7,"last update":"Tue Sep 8 16:31:37 2020","interfaces":[{"interface":"back","average":{"1min":4.21,"5min":4.1120000000000001,"15min":3.9289999999999998},"min":{"1min":2.6259999999999999,"5min":2.1280000000000001,"15min":2.1280000000000001},"max":{"1min":5.2999999999999998,"5min":6.3179999999999996,"15min":6.3179999999999996},"last":3.9009999999999998},{"interface":"front","average":{"1min":3.6579999999999999,"5min":3.706,"15min":3.9159999999999999},"min":{"1min":2.2519999999999998,"5min":1.605,"15min":1.605},"max":{"1min":5.1710000000000003,"5min":5.8120000000000003,"15min":5.8949999999999996},"last":3.5979999999999999}]},{"osd":8,"last update":"Tue Sep 8 16:30:43 2020","interfaces":[{"interface":"back","average":{"1min":4.2859999999999996,"5min":4.4219999999999997,"15min":4.3609999999999998},"min":{"1min":3.1200000000000001,"5min":1.167,"15min":1.167},"max":{"1min":5.9370000000000003,"5min":6.46,"15min":6.46},"last":4.2809999999999997},{"interface":"front","average":{"1min":4.0149999999999997,"5min":4.1660000000000004,"15min":4.2590000000000003},"min":{"1min":2.6949999999999998,"5min":1.6140000000000001,"15min":1.6140000000000001},"max":{"1min":5.7389999999999999,"5min":6.3799999999999999,"15min":6.649},"last":3.3500000000000001}]},{"osd":9,"last update":"Tue Sep 8 16:31:04 2020","interfaces":[{"interface":"back","average":{"1min":4.6269999999999998,"5min":4.6740000000000004,"15min":4.7679999999999998},"min":{"1min":3.3959999999999999,"5min":1.3240000000000001,"15min":1.3240000000000001},"max":{"1min":6.351,"5min":6.4850000000000003,"15min":6.6849999999999996},"last":4.6379999999999999},{"interface":"front","average":{"1min":4.2460000000000004,"5min":4.3029999999999999,"15min":4.4100000000000001},"min":{"1min":3.484,"5min":2.3540000000000001,"15min":2.3540000000000001},"max":{"1min":4.9349999999999996,"5min":6.5869999999999997,"15min":7.1630000000000003},"last":3.9860000000000002}]},{"osd":10,"last update":"Tue Sep 8 16:30:43 2020","interfaces":[{"interface":"back","average":{"1min":4.577,"5min":4.7590000000000003,"15min":4.7720000000000002},"min":{"1min":3.3460000000000001,"5min":1.927,"15min":1.927},"max":{"1min":5.665,"5min":6.7069999999999999,"15min":7.0830000000000002},"last":5.3470000000000004},{"interface":"front","average":{"1min":4.8159999999999998,"5min":4.9850000000000003,"15min":5.0220000000000002},"min":{"1min":3.6440000000000001,"5min":2.4700000000000002,"15min":2.4700000000000002},"max":{"1min":6.1479999999999997,"5min":7.0810000000000004,"15min":7.5110000000000001},"last":4.9930000000000003}]}]},{"osd":10,"up_from":0,"seq":697,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":347816,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":347072,"kb_avail":93397336,"statfs":{"total":95995035648,"available":95638872064,"internally_reserved":0,"allocated":753664,"data_stored":442482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":355401728},"hb_peers":[0,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 16:30:52 2020","interfaces":[{"interface":"back","average":{"1min":1.9370000000000001,"5min":2.1120000000000001,"15min":1.9590000000000001},"min":{"1min":1.1679999999999999,"5min":1.133,"15min":1.0469999999999999},"max":{"1min":2.6419999999999999,"5min":3.7599999999999998,"15min":3.9580000000000002},"last":2.4420000000000002},{"interface":"front","average":{"1min":3.254,"5min":3.3719999999999999,"15min":3.1920000000000002},"min":{"1min":1.538,"5min":1.087,"15min":1.087},"max":{"1min":4.8090000000000002,"5min":5.1909999999999998,"15min":5.202},"last":3.4020000000000001}]},{"osd":1,"last update":"Tue Sep 8 16:04:06 2020","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":2.4809999999999999,"15min":2.8860000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":3.2570000000000001,"5min":39.780999999999999,"15min":135.096},"last":2.4430000000000001},{"interface":"front","average":{"1min":1.516,"5min":2.9670000000000001,"15min":2.8119999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":3.3159999999999998,"5min":40.070999999999998,"15min":130.68100000000001},"last":1.968}]},{"osd":2,"last update":"Tue Sep 8 16:31:15 2020","interfaces":[{"interface":"back","average":{"1min":1.907,"5min":2.1139999999999999,"15min":2.0590000000000002},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":3.3340000000000001,"5min":4.4909999999999997,"15min":4.4909999999999997},"last":2.3889999999999998},{"interface":"front","average":{"1min":3.492,"5min":3.669,"15min":3.7010000000000001},"min":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"max":{"1min":5.2359999999999998,"5min":5.6070000000000002,"15min":5.6070000000000002},"last":4.0369999999999999}]},{"osd":3,"last update":"Tue Sep 8 16:31:15 2020","interfaces":[{"interface":"back","average":{"1min":2.298,"5min":2.2970000000000002,"15min":2.2890000000000001},"min":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"max":{"1min":3.4900000000000002,"5min":3.9409999999999998,"15min":4.3819999999999997},"last":2.7469999999999999},{"interface":"front","average":{"1min":1.8089999999999999,"5min":1.994,"15min":2.27},"min":{"1min":1.198,"5min":1.1950000000000001,"15min":1.038},"max":{"1min":2.819,"5min":2.819,"15min":5.1769999999999996},"last":2.5089999999999999}]},{"osd":4,"last update":"Tue Sep 8 16:31:15 2020","interfaces":[{"interface":"back","average":{"1min":3.2549999999999999,"5min":3.125,"15min":3.1779999999999999},"min":{"1min":1.5209999999999999,"5min":1.419,"15min":1.419},"max":{"1min":5.0369999999999999,"5min":5.0369999999999999,"15min":5.3150000000000004},"last":3.6619999999999999},{"interface":"front","average":{"1min":3.8460000000000001,"5min":3.7610000000000001,"15min":3.6829999999999998},"min":{"1min":2.4470000000000001,"5min":2.0019999999999998,"15min":2.0019999999999998},"max":{"1min":5.2210000000000001,"5min":6.0330000000000004,"15min":6.0330000000000004},"last":4.3799999999999999}]},{"osd":5,"last update":"Tue Sep 8 16:31:15 2020","interfaces":[{"interface":"back","average":{"1min":2.355,"5min":2.355,"15min":2.3900000000000001},"min":{"1min":1.4850000000000001,"5min":1.149,"15min":1.149},"max":{"1min":3.7709999999999999,"5min":3.9289999999999998,"15min":4.3230000000000004},"last":3.7669999999999999},{"interface":"front","average":{"1min":3.101,"5min":3.0870000000000002,"15min":3.0939999999999999},"min":{"1min":1.9370000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"max":{"1min":4.3860000000000001,"5min":5.5999999999999996,"15min":5.5999999999999996},"last":4.1829999999999998}]},{"osd":6,"last update":"Tue Sep 8 16:31:15 2020","interfaces":[{"interface":"back","average":{"1min":3.2370000000000001,"5min":3.1150000000000002,"15min":3.379},"min":{"1min":2.0390000000000001,"5min":1.4630000000000001,"15min":1.377},"max":{"1min":4.3940000000000001,"5min":5.0099999999999998,"15min":5.3979999999999997},"last":3.992},{"interface":"front","average":{"1min":2.976,"5min":3.093,"15min":3.427},"min":{"1min":2.2730000000000001,"5min":1.9890000000000001,"15min":1.873},"max":{"1min":3.9750000000000001,"5min":5.194,"15min":6.5609999999999999},"last":4.5720000000000001}]},{"osd":7,"last update":"Tue Sep 8 16:31:15 2020","interfaces":[{"interface":"back","average":{"1min":3.0070000000000001,"5min":3.2160000000000002,"15min":3.5819999999999999},"min":{"1min":1.76,"5min":1.76,"15min":1.542},"max":{"1min":4.508,"5min":4.827,"15min":5.625},"last":3.8079999999999998},{"interface":"front","average":{"1min":3.4020000000000001,"5min":3.5670000000000002,"15min":3.8610000000000002},"min":{"1min":2.2749999999999999,"5min":1.8380000000000001,"15min":1.5669999999999999},"max":{"1min":4.6109999999999998,"5min":5.0670000000000002,"15min":5.9199999999999999},"last":5.1479999999999997}]},{"osd":8,"last update":"Tue Sep 8 16:31:15 2020","interfaces":[{"interface":"back","average":{"1min":3.6269999999999998,"5min":3.7120000000000002,"15min":4.0430000000000001},"min":{"1min":1.4850000000000001,"5min":1.427,"15min":1.427},"max":{"1min":4.7999999999999998,"5min":5.4589999999999996,"15min":6.2949999999999999},"last":4.6079999999999997},{"interface":"front","average":{"1min":4.149,"5min":4.1470000000000002,"15min":4.1740000000000004},"min":{"1min":2.758,"5min":1.631,"15min":1.631},"max":{"1min":5.6349999999999998,"5min":7.2430000000000003,"15min":7.2430000000000003},"last":4.9400000000000004}]},{"osd":9,"last update":"Tue Sep 8 16:31:15 2020","interfaces":[{"interface":"back","average":{"1min":4.3390000000000004,"5min":4.375,"15min":4.4400000000000004},"min":{"1min":2.8980000000000001,"5min":1.8069999999999999,"15min":1.8069999999999999},"max":{"1min":5.6159999999999997,"5min":6.4160000000000004,"15min":9.3179999999999996},"last":4.726},{"interface":"front","average":{"1min":4.6280000000000001,"5min":4.6920000000000002,"15min":4.7210000000000001},"min":{"1min":2.637,"5min":2.1280000000000001,"15min":2.1280000000000001},"max":{"1min":6.9550000000000001,"5min":7.1699999999999999,"15min":7.1699999999999999},"last":5.109}]},{"osd":11,"last update":"Tue Sep 8 16:31:04 2020","interfaces":[{"interface":"back","average":{"1min":4.8159999999999998,"5min":4.859,"15min":4.8390000000000004},"min":{"1min":2.847,"5min":2.4209999999999998,"15min":2.4209999999999998},"max":{"1min":5.9950000000000001,"5min":7.5640000000000001,"15min":7.5640000000000001},"last":5.266},{"interface":"front","average":{"1min":5.0270000000000001,"5min":5.0499999999999998,"15min":5.0300000000000002},"min":{"1min":3.1400000000000001,"5min":2.8100000000000001,"15min":2.8100000000000001},"max":{"1min":7.4029999999999996,"5min":7.944,"15min":7.944},"last":5.4379999999999997}]}]},{"osd":9,"up_from":0,"seq":699,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":370152,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":369408,"kb_avail":93375000,"statfs":{"total":95995035648,"available":95616000000,"internally_reserved":0,"allocated":753664,"data_stored":442482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":378273792},"hb_peers":[1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 16:08:52 2020","interfaces":[{"interface":"back","average":{"1min":3.2669999999999999,"5min":2.3730000000000002,"15min":2.3839999999999999},"min":{"1min":0.82699999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":36.694000000000003,"5min":36.694000000000003,"15min":85.481999999999999},"last":2.0619999999999998},{"interface":"front","average":{"1min":3.407,"5min":2.4929999999999999,"15min":2.7570000000000001},"min":{"1min":0.77700000000000002,"5min":0.76100000000000001,"15min":0.60799999999999998},"max":{"1min":37.296999999999997,"5min":37.296999999999997,"15min":85.581999999999994},"last":2.3100000000000001}]},{"osd":1,"last update":"Tue Sep 8 16:30:56 2020","interfaces":[{"interface":"back","average":{"1min":2.1669999999999998,"5min":2.0009999999999999,"15min":1.9810000000000001},"min":{"1min":1.464,"5min":1.135,"15min":1.135},"max":{"1min":2.734,"5min":2.734,"15min":3.669},"last":2.218},{"interface":"front","average":{"1min":2.6429999999999998,"5min":2.3199999999999998,"15min":2.2989999999999999},"min":{"1min":1.905,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":3.1499999999999999,"5min":5.2590000000000003,"15min":5.2590000000000003},"last":2.3460000000000001}]},{"osd":2,"last update":"Tue Sep 8 16:31:34 2020","interfaces":[{"interface":"back","average":{"1min":2.1600000000000001,"5min":2.0600000000000001,"15min":1.9930000000000001},"min":{"1min":1.4830000000000001,"5min":1.153,"15min":1.153},"max":{"1min":2.7709999999999999,"5min":4.1790000000000003,"15min":4.1790000000000003},"last":2.2919999999999998},{"interface":"front","average":{"1min":3.3210000000000002,"5min":3.3540000000000001,"15min":3.355},"min":{"1min":2.3279999999999998,"5min":2.0209999999999999,"15min":2.0209999999999999},"max":{"1min":4.3959999999999999,"5min":5.5999999999999996,"15min":5.5999999999999996},"last":4.3869999999999996}]},{"osd":3,"last update":"Tue Sep 8 16:30:56 2020","interfaces":[{"interface":"back","average":{"1min":2.0739999999999998,"5min":2.129,"15min":2.1280000000000001},"min":{"1min":1.383,"5min":1.222,"15min":1.222},"max":{"1min":3.0270000000000001,"5min":3.2090000000000001,"15min":4.5380000000000003},"last":2.6469999999999998},{"interface":"front","average":{"1min":2.6779999999999999,"5min":2.786,"15min":2.6989999999999998},"min":{"1min":1.4710000000000001,"5min":1.343,"15min":1.343},"max":{"1min":3.9470000000000001,"5min":4.5789999999999997,"15min":4.5789999999999997},"last":3.2679999999999998}]},{"osd":4,"last update":"Tue Sep 8 16:31:34 2020","interfaces":[{"interface":"back","average":{"1min":2.5299999999999998,"5min":2.6120000000000001,"15min":2.6299999999999999},"min":{"1min":1.6779999999999999,"5min":1.145,"15min":1.145},"max":{"1min":3.8340000000000001,"5min":4.5940000000000003,"15min":4.5940000000000003},"last":2.766},{"interface":"front","average":{"1min":2.73,"5min":2.661,"15min":2.6669999999999998},"min":{"1min":1.819,"5min":1.2629999999999999,"15min":1.2629999999999999},"max":{"1min":4.3029999999999999,"5min":4.6520000000000001,"15min":5.6360000000000001},"last":3.6299999999999999}]},{"osd":5,"last update":"Tue Sep 8 16:31:34 2020","interfaces":[{"interface":"back","average":{"1min":3.3900000000000001,"5min":3.2909999999999999,"15min":3.3170000000000002},"min":{"1min":2.1960000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":4.6070000000000002,"5min":6,"15min":6},"last":4.7649999999999997},{"interface":"front","average":{"1min":3.6629999999999998,"5min":3.7069999999999999,"15min":3.7400000000000002},"min":{"1min":2.444,"5min":2.4079999999999999,"15min":2.306},"max":{"1min":4.8319999999999999,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":4.0279999999999996}]},{"osd":6,"last update":"Tue Sep 8 16:31:34 2020","interfaces":[{"interface":"back","average":{"1min":3.1389999999999998,"5min":3.165,"15min":3.23},"min":{"1min":2.141,"5min":1.5469999999999999,"15min":1.5469999999999999},"max":{"1min":4.2199999999999998,"5min":5.6970000000000001,"15min":5.6970000000000001},"last":3.7440000000000002},{"interface":"front","average":{"1min":3.8620000000000001,"5min":3.8879999999999999,"15min":3.879},"min":{"1min":3.1989999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"max":{"1min":4.649,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":4.0960000000000001}]},{"osd":7,"last update":"Tue Sep 8 16:31:34 2020","interfaces":[{"interface":"back","average":{"1min":4.0659999999999998,"5min":3.9529999999999998,"15min":4.032},"min":{"1min":3.133,"5min":1.429,"15min":1.429},"max":{"1min":4.8319999999999999,"5min":6.6369999999999996,"15min":6.6369999999999996},"last":5.0650000000000004},{"interface":"front","average":{"1min":4.5060000000000002,"5min":4.4829999999999997,"15min":4.5510000000000002},"min":{"1min":3.6379999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"max":{"1min":5.21,"5min":7.1399999999999997,"15min":7.1399999999999997},"last":5.3310000000000004}]},{"osd":8,"last update":"Tue Sep 8 16:31:34 2020","interfaces":[{"interface":"back","average":{"1min":4.4489999999999998,"5min":4.2990000000000004,"15min":4.3250000000000002},"min":{"1min":3.3050000000000002,"5min":1.504,"15min":1.504},"max":{"1min":5.3799999999999999,"5min":7.2599999999999998,"15min":7.2599999999999998},"last":5.5570000000000004},{"interface":"front","average":{"1min":4.8730000000000002,"5min":4.7809999999999997,"15min":4.8239999999999998},"min":{"1min":4.0979999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"max":{"1min":5.6319999999999997,"5min":7.5720000000000001,"15min":7.5720000000000001},"last":5.702}]},{"osd":10,"last update":"Tue Sep 8 16:31:12 2020","interfaces":[{"interface":"back","average":{"1min":3.8500000000000001,"5min":3.7930000000000001,"15min":3.7970000000000002},"min":{"1min":3.0880000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"max":{"1min":5.1399999999999997,"5min":6.5410000000000004,"15min":6.5410000000000004},"last":4.4779999999999998},{"interface":"front","average":{"1min":4.3760000000000003,"5min":4.4199999999999999,"15min":4.4359999999999999},"min":{"1min":3.3490000000000002,"5min":2.5720000000000001,"15min":2.5720000000000001},"max":{"1min":5.6539999999999999,"5min":7.3860000000000001,"15min":7.3860000000000001},"last":5.1150000000000002}]},{"osd":11,"last update":"Tue Sep 8 16:31:12 2020","interfaces":[{"interface":"back","average":{"1min":4.2960000000000003,"5min":4.21,"15min":4.258},"min":{"1min":3.5219999999999998,"5min":1.853,"15min":1.853},"max":{"1min":5.3109999999999999,"5min":7.0149999999999997,"15min":7.0149999999999997},"last":4.8170000000000002},{"interface":"front","average":{"1min":4.7779999999999996,"5min":4.7320000000000002,"15min":4.7450000000000001},"min":{"1min":3.9649999999999999,"5min":2.4590000000000001,"15min":2.4590000000000001},"max":{"1min":6.4249999999999998,"5min":7.6849999999999996,"15min":7.6849999999999996},"last":5.3929999999999998}]}]},{"osd":8,"up_from":0,"seq":707,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":276392,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":275648,"kb_avail":93468760,"statfs":{"total":95995035648,"available":95712010240,"internally_reserved":0,"allocated":753664,"data_stored":442482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":282263552},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 16:31:26 2020","interfaces":[{"interface":"back","average":{"1min":2.625,"5min":2.4670000000000001,"15min":2.472},"min":{"1min":1.7230000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":3.573,"5min":4.492,"15min":4.7380000000000004},"last":2.98},{"interface":"front","average":{"1min":3.2400000000000002,"5min":2.9119999999999999,"15min":2.8570000000000002},"min":{"1min":2.2050000000000001,"5min":1.6599999999999999,"15min":1.5149999999999999},"max":{"1min":4.2759999999999998,"5min":5.3390000000000004,"15min":5.3390000000000004},"last":2.5600000000000001}]},{"osd":1,"last update":"Tue Sep 8 15:36:29 2020","interfaces":[{"interface":"back","average":{"1min":2.492,"5min":2.0659999999999998,"15min":1.8839999999999999},"min":{"1min":1.4690000000000001,"5min":1.137,"15min":1.137},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":1.4350000000000001},{"interface":"front","average":{"1min":2.7200000000000002,"5min":2.427,"15min":2.347},"min":{"1min":1.351,"5min":1.046,"15min":1.046},"max":{"1min":4.444,"5min":4.444,"15min":4.444},"last":1.8600000000000001}]},{"osd":2,"last update":"Tue Sep 8 16:31:26 2020","interfaces":[{"interface":"back","average":{"1min":2.9220000000000002,"5min":2.855,"15min":2.7559999999999998},"min":{"1min":1.9810000000000001,"5min":1.3640000000000001,"15min":1.323},"max":{"1min":3.7229999999999999,"5min":4.2409999999999997,"15min":4.3719999999999999},"last":2.052},{"interface":"front","average":{"1min":3.7770000000000001,"5min":3.4409999999999998,"15min":3.2559999999999998},"min":{"1min":2.0640000000000001,"5min":1.3919999999999999,"15min":1.3919999999999999},"max":{"1min":4.4589999999999996,"5min":5.3410000000000002,"15min":5.3410000000000002},"last":3.577}]},{"osd":3,"last update":"Tue Sep 8 16:31:26 2020","interfaces":[{"interface":"back","average":{"1min":2.4769999999999999,"5min":2.7959999999999998,"15min":2.9590000000000001},"min":{"1min":1.397,"5min":1.3180000000000001,"15min":1.3180000000000001},"max":{"1min":3.8130000000000002,"5min":4.1669999999999998,"15min":4.8879999999999999},"last":3.4209999999999998},{"interface":"front","average":{"1min":2.274,"5min":2.556,"15min":2.6349999999999998},"min":{"1min":1.556,"5min":1.556,"15min":1.456},"max":{"1min":4.0519999999999996,"5min":4.7119999999999997,"15min":4.7779999999999996},"last":2.157}]},{"osd":4,"last update":"Tue Sep 8 16:31:26 2020","interfaces":[{"interface":"back","average":{"1min":3.3730000000000002,"5min":3.3300000000000001,"15min":3.4119999999999999},"min":{"1min":1.6830000000000001,"5min":1.6759999999999999,"15min":1.5649999999999999},"max":{"1min":4.3609999999999998,"5min":4.859,"15min":5.585},"last":3.7210000000000001},{"interface":"front","average":{"1min":3.3879999999999999,"5min":3.4009999999999998,"15min":3.391},"min":{"1min":2.2130000000000001,"5min":1.7689999999999999,"15min":1.5389999999999999},"max":{"1min":4.4539999999999997,"5min":5.5800000000000001,"15min":6.3860000000000001},"last":3.976}]},{"osd":5,"last update":"Tue Sep 8 16:31:26 2020","interfaces":[{"interface":"back","average":{"1min":2.3330000000000002,"5min":2.3239999999999998,"15min":3.016},"min":{"1min":1.879,"5min":1.6240000000000001,"15min":1.6240000000000001},"max":{"1min":3.2240000000000002,"5min":3.8820000000000001,"15min":5.2320000000000002},"last":2.4239999999999999},{"interface":"front","average":{"1min":3.843,"5min":3.8730000000000002,"15min":3.2450000000000001},"min":{"1min":2.1909999999999998,"5min":1.877,"15min":1.877},"max":{"1min":4.774,"5min":5.9240000000000004,"15min":5.9240000000000004},"last":4.2560000000000002}]},{"osd":6,"last update":"Tue Sep 8 16:31:26 2020","interfaces":[{"interface":"back","average":{"1min":3.7879999999999998,"5min":3.9369999999999998,"15min":3.883},"min":{"1min":2.3780000000000001,"5min":2.3450000000000002,"15min":1.863},"max":{"1min":5.0510000000000002,"5min":6.4139999999999997,"15min":6.4139999999999997},"last":4.7709999999999999},{"interface":"front","average":{"1min":4.2709999999999999,"5min":4.3259999999999996,"15min":4.2889999999999997},"min":{"1min":3.1669999999999998,"5min":2.702,"15min":2.2280000000000002},"max":{"1min":5.4649999999999999,"5min":6.6020000000000003,"15min":7.0670000000000002},"last":4.5839999999999996}]},{"osd":7,"last update":"Tue Sep 8 16:31:26 2020","interfaces":[{"interface":"back","average":{"1min":3.9119999999999999,"5min":3.8959999999999999,"15min":4.141},"min":{"1min":3.008,"5min":1.998,"15min":1.998},"max":{"1min":5.665,"5min":5.665,"15min":6.9459999999999997},"last":4.0899999999999999},{"interface":"front","average":{"1min":4.7510000000000003,"5min":4.7640000000000002,"15min":4.569},"min":{"1min":3.9470000000000001,"5min":3.343,"15min":2.9199999999999999},"max":{"1min":6.0220000000000002,"5min":7.1479999999999997,"15min":7.1660000000000004},"last":5.0170000000000003}]},{"osd":9,"last update":"Tue Sep 8 16:30:51 2020","interfaces":[{"interface":"back","average":{"1min":2.742,"5min":2.7450000000000001,"15min":2.7719999999999998},"min":{"1min":1.7769999999999999,"5min":1.6850000000000001,"15min":1.6850000000000001},"max":{"1min":3.8969999999999998,"5min":4.9290000000000003,"15min":5.5090000000000003},"last":2.8410000000000002},{"interface":"front","average":{"1min":4.5570000000000004,"5min":4.7229999999999999,"15min":4.6219999999999999},"min":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":1.9039999999999999},"max":{"1min":5.7919999999999998,"5min":7.0590000000000002,"15min":7.0590000000000002},"last":5.2320000000000002}]},{"osd":10,"last update":"Tue Sep 8 16:31:26 2020","interfaces":[{"interface":"back","average":{"1min":4.032,"5min":4.0830000000000002,"15min":4.1130000000000004},"min":{"1min":2.6349999999999998,"5min":2.1640000000000001,"15min":2.1349999999999998},"max":{"1min":4.9370000000000003,"5min":5.8360000000000003,"15min":6.149},"last":4.3019999999999996},{"interface":"front","average":{"1min":3.226,"5min":3.1819999999999999,"15min":3.411},"min":{"1min":2.7149999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"max":{"1min":4.258,"5min":5.4649999999999999,"15min":6.1420000000000003},"last":3.3220000000000001}]},{"osd":11,"last update":"Tue Sep 8 16:31:26 2020","interfaces":[{"interface":"back","average":{"1min":4.1900000000000004,"5min":4.2999999999999998,"15min":4.4870000000000001},"min":{"1min":3.4350000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"max":{"1min":5.3140000000000001,"5min":5.9729999999999999,"15min":6.5309999999999997},"last":4.5060000000000002},{"interface":"front","average":{"1min":5.0369999999999999,"5min":5.0819999999999999,"15min":5.0190000000000001},"min":{"1min":4.3170000000000002,"5min":3.5649999999999999,"15min":2.488},"max":{"1min":6.2530000000000001,"5min":7.4480000000000004,"15min":7.7430000000000003},"last":5.4299999999999997}]}]},{"osd":7,"up_from":0,"seq":712,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":325340,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":324608,"kb_avail":93419812,"statfs":{"total":95995035648,"available":95661887488,"internally_reserved":0,"allocated":741376,"data_stored":434987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":332398592},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:34:02 2020","interfaces":[{"interface":"back","average":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"min":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"max":{"1min":4.0190000000000001,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":2.2189999999999999},{"interface":"front","average":{"1min":2.6259999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"min":{"1min":1.542,"5min":1.542,"15min":1.542},"max":{"1min":4.5229999999999997,"5min":4.5229999999999997,"15min":4.5229999999999997},"last":2.3079999999999998}]},{"osd":1,"last update":"Tue Sep 8 16:31:13 2020","interfaces":[{"interface":"back","average":{"1min":2.1030000000000002,"5min":2.1949999999999998,"15min":2.1779999999999999},"min":{"1min":1.724,"5min":1.625,"15min":1.129},"max":{"1min":2.6989999999999998,"5min":4.8220000000000001,"15min":4.8220000000000001},"last":2.1640000000000001},{"interface":"front","average":{"1min":2.5209999999999999,"5min":2.3799999999999999,"15min":2.3220000000000001},"min":{"1min":1.6339999999999999,"5min":1.3839999999999999,"15min":0.98999999999999999},"max":{"1min":3.9350000000000001,"5min":4.6239999999999997,"15min":4.6239999999999997},"last":2.3279999999999998}]},{"osd":2,"last update":"Tue Sep 8 16:31:03 2020","interfaces":[{"interface":"back","average":{"1min":2.4729999999999999,"5min":2.3210000000000002,"15min":2.2989999999999999},"min":{"1min":1.6060000000000001,"5min":1.4219999999999999,"15min":1.232},"max":{"1min":3.5259999999999998,"5min":3.754,"15min":4.0330000000000004},"last":2.7570000000000001},{"interface":"front","average":{"1min":3.101,"5min":2.948,"15min":2.8799999999999999},"min":{"1min":1.998,"5min":1.6679999999999999,"15min":1.0649999999999999},"max":{"1min":4.4560000000000004,"5min":4.7839999999999998,"15min":4.9359999999999999},"last":2.508}]},{"osd":3,"last update":"Tue Sep 8 16:31:03 2020","interfaces":[{"interface":"back","average":{"1min":2.552,"5min":2.726,"15min":2.7490000000000001},"min":{"1min":1.8220000000000001,"5min":1.643,"15min":1.4990000000000001},"max":{"1min":3.4340000000000002,"5min":4.2080000000000002,"15min":5.1829999999999998},"last":1.919},{"interface":"front","average":{"1min":3.0510000000000002,"5min":3.2330000000000001,"15min":3.254},"min":{"1min":1.889,"5min":1.3799999999999999,"15min":1.3799999999999999},"max":{"1min":4.3230000000000004,"5min":5.0709999999999997,"15min":5.0709999999999997},"last":1.8340000000000001}]},{"osd":4,"last update":"Tue Sep 8 16:31:13 2020","interfaces":[{"interface":"back","average":{"1min":3.4910000000000001,"5min":3.27,"15min":3.2890000000000001},"min":{"1min":2.6349999999999998,"5min":1.5169999999999999,"15min":1.04},"max":{"1min":4.5060000000000002,"5min":5.4900000000000002,"15min":6.0839999999999996},"last":2.3050000000000002},{"interface":"front","average":{"1min":3.9460000000000002,"5min":3.774,"15min":3.774},"min":{"1min":3.0579999999999998,"5min":1.9450000000000001,"15min":1.5289999999999999},"max":{"1min":5.4699999999999998,"5min":5.7759999999999998,"15min":6.8079999999999998},"last":2.6930000000000001}]},{"osd":5,"last update":"Tue Sep 8 16:31:03 2020","interfaces":[{"interface":"back","average":{"1min":4.1719999999999997,"5min":3.996,"15min":3.9769999999999999},"min":{"1min":2.8679999999999999,"5min":2.8069999999999999,"15min":1.423},"max":{"1min":5.7460000000000004,"5min":8.3930000000000007,"15min":8.3930000000000007},"last":1.8},{"interface":"front","average":{"1min":4.7960000000000003,"5min":4.6699999999999999,"15min":4.6459999999999999},"min":{"1min":3.6309999999999998,"5min":3.2280000000000002,"15min":1.8500000000000001},"max":{"1min":6.8390000000000004,"5min":8.9130000000000003,"15min":8.9130000000000003},"last":2.4420000000000002}]},{"osd":6,"last update":"Tue Sep 8 16:31:03 2020","interfaces":[{"interface":"back","average":{"1min":4.5010000000000003,"5min":4.3730000000000002,"15min":4.4219999999999997},"min":{"1min":3.3380000000000001,"5min":2.8119999999999998,"15min":1.325},"max":{"1min":5.8109999999999999,"5min":7.234,"15min":7.234},"last":2.6150000000000002},{"interface":"front","average":{"1min":4.1159999999999997,"5min":3.9209999999999998,"15min":3.823},"min":{"1min":3.056,"5min":2.04,"15min":1.1930000000000001},"max":{"1min":5.6319999999999997,"5min":5.6609999999999996,"15min":5.6609999999999996},"last":3.1379999999999999}]},{"osd":8,"last update":"Tue Sep 8 16:31:27 2020","interfaces":[{"interface":"back","average":{"1min":4.5140000000000002,"5min":4.6050000000000004,"15min":4.5789999999999997},"min":{"1min":2.5979999999999999,"5min":2.5979999999999999,"15min":1.6140000000000001},"max":{"1min":5.9240000000000004,"5min":7.0039999999999996,"15min":7.1470000000000002},"last":3.3170000000000002},{"interface":"front","average":{"1min":4.2290000000000001,"5min":4.1790000000000003,"15min":4.2880000000000003},"min":{"1min":3.206,"5min":2.492,"15min":1.3520000000000001},"max":{"1min":5.7649999999999997,"5min":6.1459999999999999,"15min":6.585},"last":2.867}]},{"osd":9,"last update":"Tue Sep 8 16:30:58 2020","interfaces":[{"interface":"back","average":{"1min":3.7210000000000001,"5min":3.6389999999999998,"15min":3.6459999999999999},"min":{"1min":2.927,"5min":2.323,"15min":1.629},"max":{"1min":4.2359999999999998,"5min":5.0519999999999996,"15min":6.335},"last":3.0720000000000001},{"interface":"front","average":{"1min":4.3209999999999997,"5min":4.1929999999999996,"15min":4.0640000000000001},"min":{"1min":3.5720000000000001,"5min":2.714,"15min":2.2240000000000002},"max":{"1min":4.8760000000000003,"5min":5.6689999999999996,"15min":6.1920000000000002},"last":3.4990000000000001}]},{"osd":10,"last update":"Tue Sep 8 16:31:13 2020","interfaces":[{"interface":"back","average":{"1min":4.6130000000000004,"5min":4.3899999999999997,"15min":4.6989999999999998},"min":{"1min":3.4049999999999998,"5min":2.1339999999999999,"15min":1.6890000000000001},"max":{"1min":5.8490000000000002,"5min":6.4720000000000004,"15min":7.218},"last":3.407},{"interface":"front","average":{"1min":5.0460000000000003,"5min":4.7839999999999998,"15min":4.6100000000000003},"min":{"1min":3.7770000000000001,"5min":2.2200000000000002,"15min":1.5660000000000001},"max":{"1min":6.1710000000000003,"5min":6.827,"15min":7.6200000000000001},"last":3.7429999999999999}]},{"osd":11,"last update":"Tue Sep 8 16:31:27 2020","interfaces":[{"interface":"back","average":{"1min":4.9690000000000003,"5min":4.9160000000000004,"15min":5.0860000000000003},"min":{"1min":3.4390000000000001,"5min":3.4390000000000001,"15min":2.2109999999999999},"max":{"1min":6.1360000000000001,"5min":6.6449999999999996,"15min":7.5739999999999998},"last":2.9220000000000002},{"interface":"front","average":{"1min":5.125,"5min":5.1710000000000003,"15min":5.319},"min":{"1min":3.706,"5min":3.706,"15min":2.379},"max":{"1min":6.4009999999999998,"5min":7.8529999999999998,"15min":7.867},"last":3.2240000000000002}]}]},{"osd":6,"up_from":0,"seq":718,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":252136,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":251392,"kb_avail":93493016,"statfs":{"total":95995035648,"available":95736848384,"internally_reserved":0,"allocated":753664,"data_stored":442502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":257425408},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 16:31:05 2020","interfaces":[{"interface":"back","average":{"1min":2.0720000000000001,"5min":2.1240000000000001,"15min":2.1280000000000001},"min":{"1min":1.7310000000000001,"5min":1.093,"15min":1.093},"max":{"1min":3.0979999999999999,"5min":3.0979999999999999,"15min":3.7360000000000002},"last":2.4940000000000002},{"interface":"front","average":{"1min":3.2440000000000002,"5min":3.1560000000000001,"15min":3.1469999999999998},"min":{"1min":2.2389999999999999,"5min":2.181,"15min":1.8280000000000001},"max":{"1min":4.6559999999999997,"5min":4.8949999999999996,"15min":4.8949999999999996},"last":3.363}]},{"osd":1,"last update":"Tue Sep 8 16:31:05 2020","interfaces":[{"interface":"back","average":{"1min":2.3340000000000001,"5min":2.2770000000000001,"15min":2.2160000000000002},"min":{"1min":1.8,"5min":1.6359999999999999,"15min":1.2829999999999999},"max":{"1min":3.5030000000000001,"5min":3.806,"15min":3.806},"last":2.5830000000000002},{"interface":"front","average":{"1min":3.8180000000000001,"5min":3.802,"15min":3.8330000000000002},"min":{"1min":2.5680000000000001,"5min":2.5680000000000001,"15min":2.2709999999999999},"max":{"1min":5.0259999999999998,"5min":5.9349999999999996,"15min":5.9349999999999996},"last":4.3769999999999998}]},{"osd":2,"last update":"Tue Sep 8 16:31:05 2020","interfaces":[{"interface":"back","average":{"1min":2.0630000000000002,"5min":2.1469999999999998,"15min":2.1859999999999999},"min":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":0.99099999999999999},"max":{"1min":2.3410000000000002,"5min":3.5489999999999999,"15min":3.5489999999999999},"last":2.6699999999999999},{"interface":"front","average":{"1min":2.9199999999999999,"5min":3.121,"15min":3.2109999999999999},"min":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"max":{"1min":3.8999999999999999,"5min":6.274,"15min":6.274},"last":4.6360000000000001}]},{"osd":3,"last update":"Tue Sep 8 16:31:05 2020","interfaces":[{"interface":"back","average":{"1min":2.9820000000000002,"5min":2.8420000000000001,"15min":2.9260000000000002},"min":{"1min":2.1099999999999999,"5min":2.0030000000000001,"15min":1.466},"max":{"1min":4.5670000000000002,"5min":4.8479999999999999,"15min":4.9909999999999997},"last":4.1500000000000004},{"interface":"front","average":{"1min":3.802,"5min":3.77,"15min":3.8410000000000002},"min":{"1min":2.4260000000000002,"5min":2.4260000000000002,"15min":1.7669999999999999},"max":{"1min":5.1669999999999998,"5min":5.1669999999999998,"15min":5.8330000000000002},"last":4.9880000000000004}]},{"osd":4,"last update":"Tue Sep 8 16:31:05 2020","interfaces":[{"interface":"back","average":{"1min":3.5190000000000001,"5min":3.4039999999999999,"15min":3.4670000000000001},"min":{"1min":2.3999999999999999,"5min":1.8660000000000001,"15min":1.3740000000000001},"max":{"1min":4.9009999999999998,"5min":6.6740000000000004,"15min":6.6740000000000004},"last":4.9089999999999998},{"interface":"front","average":{"1min":4.0789999999999997,"5min":4.109,"15min":4.0519999999999996},"min":{"1min":2.7530000000000001,"5min":1.7569999999999999,"15min":1.7569999999999999},"max":{"1min":5.5549999999999997,"5min":9.2210000000000001,"15min":9.2210000000000001},"last":5.1559999999999997}]},{"osd":5,"last update":"Tue Sep 8 16:31:05 2020","interfaces":[{"interface":"back","average":{"1min":3.7650000000000001,"5min":3.597,"15min":3.681},"min":{"1min":2.669,"5min":1.6439999999999999,"15min":1.6439999999999999},"max":{"1min":5.2279999999999998,"5min":5.96,"15min":6},"last":3.9089999999999998},{"interface":"front","average":{"1min":4.8029999999999999,"5min":4.5670000000000002,"15min":4.5940000000000003},"min":{"1min":3.548,"5min":2.726,"15min":2.726},"max":{"1min":6.3899999999999997,"5min":8.6880000000000006,"15min":8.6880000000000006},"last":5.9320000000000004}]},{"osd":7,"last update":"Tue Sep 8 16:31:05 2020","interfaces":[{"interface":"back","average":{"1min":4.0570000000000004,"5min":3.9609999999999999,"15min":4.0659999999999998},"min":{"1min":2.7290000000000001,"5min":1.8149999999999999,"15min":1.621},"max":{"1min":4.8280000000000003,"5min":6.657,"15min":6.657},"last":4.7249999999999996},{"interface":"front","average":{"1min":4.6200000000000001,"5min":4.5629999999999997,"15min":4.6470000000000002},"min":{"1min":3.1459999999999999,"5min":2.794,"15min":2.794},"max":{"1min":6.1150000000000002,"5min":7.4669999999999996,"15min":9.0069999999999997},"last":5.375}]},{"osd":8,"last update":"Tue Sep 8 16:31:19 2020","interfaces":[{"interface":"back","average":{"1min":5.1769999999999996,"5min":4.806,"15min":4.7990000000000004},"min":{"1min":3.633,"5min":2.6960000000000002,"15min":2.6960000000000002},"max":{"1min":6.7800000000000002,"5min":7.4610000000000003,"15min":7.4610000000000003},"last":5.5609999999999999},{"interface":"front","average":{"1min":5.4820000000000002,"5min":5.0949999999999998,"15min":5.1040000000000001},"min":{"1min":4.0469999999999997,"5min":2.8900000000000001,"15min":2.8900000000000001},"max":{"1min":6.9390000000000001,"5min":7.9560000000000004,"15min":8.5239999999999991},"last":5.742}]},{"osd":9,"last update":"Tue Sep 8 16:31:05 2020","interfaces":[{"interface":"back","average":{"1min":3.3820000000000001,"5min":3.4049999999999998,"15min":3.4809999999999999},"min":{"1min":2.6000000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"max":{"1min":4.2469999999999999,"5min":5.4370000000000003,"15min":5.5209999999999999},"last":3.7440000000000002},{"interface":"front","average":{"1min":4.0069999999999997,"5min":4.0339999999999998,"15min":4.1219999999999999},"min":{"1min":2.8399999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"max":{"1min":4.7160000000000002,"5min":6.1799999999999997,"15min":6.2999999999999998},"last":4.2880000000000003}]},{"osd":10,"last update":"Tue Sep 8 16:31:15 2020","interfaces":[{"interface":"back","average":{"1min":3.6269999999999998,"5min":3.8359999999999999,"15min":3.899},"min":{"1min":2.5590000000000002,"5min":2.3780000000000001,"15min":1.98},"max":{"1min":4.2050000000000001,"5min":6.1349999999999998,"15min":6.1349999999999998},"last":4.8209999999999997},{"interface":"front","average":{"1min":5.4020000000000001,"5min":5.2149999999999999,"15min":5.2560000000000002},"min":{"1min":4,"5min":2.8079999999999998,"15min":2.8079999999999998},"max":{"1min":7.3710000000000004,"5min":8.3249999999999993,"15min":8.3249999999999993},"last":6.1200000000000001}]}]},{"osd":1,"up_from":0,"seq":739,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":581736,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":580992,"kb_avail":93163416,"statfs":{"total":95995035648,"available":95399337984,"internally_reserved":0,"allocated":753664,"data_stored":442502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":594935808},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 16:31:23 2020","interfaces":[{"interface":"back","average":{"1min":2.0369999999999999,"5min":2.0190000000000001,"15min":2.0840000000000001},"min":{"1min":1.2430000000000001,"5min":1.1259999999999999,"15min":0.84399999999999997},"max":{"1min":3.355,"5min":3.5880000000000001,"15min":4.2949999999999999},"last":2.1000000000000001},{"interface":"front","average":{"1min":2.3290000000000002,"5min":2.597,"15min":2.7360000000000002},"min":{"1min":1.091,"5min":1.091,"15min":1.091},"max":{"1min":3.02,"5min":4.2329999999999997,"15min":4.6639999999999997},"last":2.2639999999999998}]},{"osd":2,"last update":"Tue Sep 8 16:30:54 2020","interfaces":[{"interface":"back","average":{"1min":2.8900000000000001,"5min":3.0470000000000002,"15min":3.1030000000000002},"min":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.4279999999999999},"max":{"1min":4.0259999999999998,"5min":4.194,"15min":4.9580000000000002},"last":1.6950000000000001},{"interface":"front","average":{"1min":2.173,"5min":2.1800000000000002,"15min":2.1659999999999999},"min":{"1min":1.242,"5min":1.242,"15min":1.242},"max":{"1min":3.302,"5min":4.1509999999999998,"15min":4.1509999999999998},"last":1.7370000000000001}]},{"osd":3,"last update":"Tue Sep 8 15:51:14 2020","interfaces":[{"interface":"back","average":{"1min":3.2309999999999999,"5min":4.4930000000000003,"15min":6.1150000000000002},"min":{"1min":0.92900000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":10.720000000000001,"5min":39.241999999999997,"15min":39.241999999999997},"last":1.8680000000000001},{"interface":"front","average":{"1min":2.8340000000000001,"5min":4.2370000000000001,"15min":5.6680000000000001},"min":{"1min":0.86899999999999999,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":10.365,"5min":41.124000000000002,"15min":41.124000000000002},"last":2.8799999999999999}]},{"osd":4,"last update":"Tue Sep 8 16:30:54 2020","interfaces":[{"interface":"back","average":{"1min":3.0409999999999999,"5min":3.3300000000000001,"15min":3.1429999999999998},"min":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":1.3560000000000001},"max":{"1min":3.7690000000000001,"5min":4.5890000000000004,"15min":5.0529999999999999},"last":2.4359999999999999},{"interface":"front","average":{"1min":2.7869999999999999,"5min":2.9790000000000001,"15min":3.2509999999999999},"min":{"1min":1.7949999999999999,"5min":1.6830000000000001,"15min":1.6830000000000001},"max":{"1min":3.9220000000000002,"5min":4.9009999999999998,"15min":6.0369999999999999},"last":2.399}]},{"osd":5,"last update":"Tue Sep 8 16:31:17 2020","interfaces":[{"interface":"back","average":{"1min":2.218,"5min":2.218,"15min":2.234},"min":{"1min":1.496,"5min":1.4219999999999999,"15min":1.4219999999999999},"max":{"1min":2.9910000000000001,"5min":3.4500000000000002,"15min":4.0510000000000002},"last":1.8380000000000001},{"interface":"front","average":{"1min":2.802,"5min":2.96,"15min":3.008},"min":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"max":{"1min":3.7200000000000002,"5min":4.6150000000000002,"15min":5.3650000000000002},"last":2.7679999999999998}]},{"osd":6,"last update":"Tue Sep 8 16:31:23 2020","interfaces":[{"interface":"back","average":{"1min":3.302,"5min":3.6600000000000001,"15min":3.665},"min":{"1min":2.456,"5min":2.456,"15min":2.0750000000000002},"max":{"1min":4.4669999999999996,"5min":5.4580000000000002,"15min":5.4589999999999996},"last":2.968},{"interface":"front","average":{"1min":2.4929999999999999,"5min":2.6099999999999999,"15min":2.7490000000000001},"min":{"1min":1.6839999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"max":{"1min":3.238,"5min":3.9809999999999999,"15min":4.7149999999999999},"last":2.1440000000000001}]},{"osd":7,"last update":"Tue Sep 8 16:30:54 2020","interfaces":[{"interface":"back","average":{"1min":3.7429999999999999,"5min":3.9289999999999998,"15min":4.048},"min":{"1min":2.3210000000000002,"5min":2.298,"15min":2.298},"max":{"1min":4.8099999999999996,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":2.6600000000000001},{"interface":"front","average":{"1min":2.5950000000000002,"5min":2.8660000000000001,"15min":2.8919999999999999},"min":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"max":{"1min":3.1589999999999998,"5min":5.3070000000000004,"15min":5.641},"last":2.3079999999999998}]},{"osd":8,"last update":"Tue Sep 8 16:31:23 2020","interfaces":[{"interface":"back","average":{"1min":4.1639999999999997,"5min":4.3929999999999998,"15min":4.1219999999999999},"min":{"1min":2.7719999999999998,"5min":2.6640000000000001,"15min":1.6990000000000001},"max":{"1min":5.6070000000000002,"5min":6.8579999999999997,"15min":6.8579999999999997},"last":3.2410000000000001},{"interface":"front","average":{"1min":3.5179999999999998,"5min":3.7639999999999998,"15min":4.173},"min":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"max":{"1min":5.234,"5min":6.1239999999999997,"15min":6.984},"last":4.6109999999999998}]},{"osd":9,"last update":"Tue Sep 8 16:31:38 2020","interfaces":[{"interface":"back","average":{"1min":4.1580000000000004,"5min":4.3330000000000002,"15min":4.4909999999999997},"min":{"1min":2.9540000000000002,"5min":2.8719999999999999,"15min":2.8719999999999999},"max":{"1min":5.7400000000000002,"5min":6.2060000000000004,"15min":6.6369999999999996},"last":3.5979999999999999},{"interface":"front","average":{"1min":3.653,"5min":3.8660000000000001,"15min":3.891},"min":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":1.9850000000000001},"max":{"1min":4.8659999999999997,"5min":5.3959999999999999,"15min":6.1399999999999997},"last":3.8799999999999999}]},{"osd":10,"last update":"Tue Sep 8 16:31:17 2020","interfaces":[{"interface":"back","average":{"1min":3.6520000000000001,"5min":3.9910000000000001,"15min":4.3899999999999997},"min":{"1min":2.6160000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"max":{"1min":4.8239999999999998,"5min":5.79,"15min":6.8070000000000004},"last":3.133},{"interface":"front","average":{"1min":3.9700000000000002,"5min":4.2800000000000002,"15min":4.4509999999999996},"min":{"1min":2.8250000000000002,"5min":2.3639999999999999,"15min":2.3639999999999999},"max":{"1min":5.3049999999999997,"5min":6.5449999999999999,"15min":7.2359999999999998},"last":3.5150000000000001}]},{"osd":11,"last update":"Tue Sep 8 16:31:38 2020","interfaces":[{"interface":"back","average":{"1min":3.101,"5min":3.339,"15min":3.407},"min":{"1min":1.772,"5min":1.772,"15min":1.772},"max":{"1min":3.9020000000000001,"5min":4.7720000000000002,"15min":5.3979999999999997},"last":2.4990000000000001},{"interface":"front","average":{"1min":4.117,"5min":4.3680000000000003,"15min":4.1849999999999996},"min":{"1min":2.4039999999999999,"5min":2.2970000000000002,"15min":1.8160000000000001},"max":{"1min":5.7309999999999999,"5min":6.2789999999999999,"15min":6.2789999999999999},"last":4.2279999999999998}]}]},{"osd":0,"up_from":0,"seq":744,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":366236,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":365504,"kb_avail":93378916,"statfs":{"total":95995035648,"available":95620009984,"internally_reserved":0,"allocated":741376,"data_stored":434987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":374276096},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 8 16:31:25 2020","interfaces":[{"interface":"back","average":{"1min":1.8480000000000001,"5min":2.02,"15min":2.0710000000000002},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":2.7370000000000001,"5min":3.2669999999999999,"15min":3.9180000000000001},"last":2.3199999999999998},{"interface":"front","average":{"1min":2.3460000000000001,"5min":2.8340000000000001,"15min":2.903},"min":{"1min":1.232,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":3.569,"5min":4.968,"15min":4.968},"last":2.6819999999999999}]},{"osd":2,"last update":"Tue Sep 8 16:31:38 2020","interfaces":[{"interface":"back","average":{"1min":2.7360000000000002,"5min":2.9980000000000002,"15min":3.0840000000000001},"min":{"1min":1.647,"5min":1.647,"15min":1.647},"max":{"1min":3.9550000000000001,"5min":4.4370000000000003,"15min":4.5830000000000002},"last":2.8010000000000002},{"interface":"front","average":{"1min":1.7769999999999999,"5min":1.992,"15min":2.056},"min":{"1min":1.048,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":2.415,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":2.3500000000000001}]},{"osd":3,"last update":"Tue Sep 8 16:04:34 2020","interfaces":[{"interface":"back","average":{"1min":4.1349999999999998,"5min":3.25,"15min":4.4749999999999996},"min":{"1min":0.79600000000000004,"5min":0.68000000000000005,"15min":0.64800000000000002},"max":{"1min":18.785,"5min":26.27,"15min":282.916},"last":1.97},{"interface":"front","average":{"1min":3.9870000000000001,"5min":3.492,"15min":3.5760000000000001},"min":{"1min":0.92600000000000005,"5min":0.80600000000000005,"15min":0.69999999999999996},"max":{"1min":18.614000000000001,"5min":26.161000000000001,"15min":103.87},"last":2.3919999999999999}]},{"osd":4,"last update":"Tue Sep 8 16:31:38 2020","interfaces":[{"interface":"back","average":{"1min":2.0369999999999999,"5min":2.1480000000000001,"15min":2.181},"min":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"max":{"1min":2.9399999999999999,"5min":3.8260000000000001,"15min":3.8260000000000001},"last":2.4060000000000001},{"interface":"front","average":{"1min":2.5499999999999998,"5min":2.6619999999999999,"15min":2.7410000000000001},"min":{"1min":1.409,"5min":1.2250000000000001,"15min":1.2250000000000001},"max":{"1min":3.6680000000000001,"5min":4.2919999999999998,"15min":4.2919999999999998},"last":2.98}]},{"osd":5,"last update":"Tue Sep 8 16:31:10 2020","interfaces":[{"interface":"back","average":{"1min":2.5870000000000002,"5min":2.8260000000000001,"15min":2.875},"min":{"1min":1.736,"5min":1.6020000000000001,"15min":1.6020000000000001},"max":{"1min":3.415,"5min":4.2439999999999998,"15min":4.7690000000000001},"last":2.843},{"interface":"front","average":{"1min":3.4529999999999998,"5min":3.585,"15min":3.637},"min":{"1min":2.0329999999999999,"5min":1.839,"15min":1.839},"max":{"1min":4.3220000000000001,"5min":5.2290000000000001,"15min":5.298},"last":4.0220000000000002}]},{"osd":6,"last update":"Tue Sep 8 16:31:25 2020","interfaces":[{"interface":"back","average":{"1min":3.6930000000000001,"5min":3.8580000000000001,"15min":3.8639999999999999},"min":{"1min":2.222,"5min":1.7170000000000001,"15min":1.7170000000000001},"max":{"1min":4.7960000000000003,"5min":5.2400000000000002,"15min":5.4820000000000002},"last":4.774},{"interface":"front","average":{"1min":4.3220000000000001,"5min":4.4779999999999998,"15min":4.476},"min":{"1min":2.585,"5min":2.3010000000000002,"15min":2.3010000000000002},"max":{"1min":6.1079999999999997,"5min":6.3120000000000003,"15min":6.7489999999999997},"last":5.1529999999999996}]},{"osd":7,"last update":"Tue Sep 8 16:31:10 2020","interfaces":[{"interface":"back","average":{"1min":4.1139999999999999,"5min":4.359,"15min":4.4020000000000001},"min":{"1min":2.5030000000000001,"5min":1.548,"15min":1.548},"max":{"1min":5.5389999999999997,"5min":6.008,"15min":6.4340000000000002},"last":3.6509999999999998},{"interface":"front","average":{"1min":2.6200000000000001,"5min":2.9729999999999999,"15min":2.952},"min":{"1min":1.756,"5min":1.478,"15min":1.478},"max":{"1min":3.4910000000000001,"5min":4.5780000000000003,"15min":4.5780000000000003},"last":3.415}]},{"osd":8,"last update":"Tue Sep 8 16:31:25 2020","interfaces":[{"interface":"back","average":{"1min":4.1820000000000004,"5min":4.5069999999999997,"15min":4.5720000000000001},"min":{"1min":2.4249999999999998,"5min":2.0009999999999999,"15min":2.0009999999999999},"max":{"1min":5.8159999999999998,"5min":5.9199999999999999,"15min":6.6879999999999997},"last":6.2990000000000004},{"interface":"front","average":{"1min":4.8179999999999996,"5min":5.0750000000000002,"15min":5.1550000000000002},"min":{"1min":2.7410000000000001,"5min":2.1960000000000002,"15min":2.1960000000000002},"max":{"1min":6.548,"5min":6.8230000000000004,"15min":8.0020000000000007},"last":5.931}]},{"osd":9,"last update":"Tue Sep 8 16:31:38 2020","interfaces":[{"interface":"back","average":{"1min":4.6390000000000002,"5min":4.681,"15min":4.7800000000000002},"min":{"1min":3.0289999999999999,"5min":2.3799999999999999,"15min":2.3799999999999999},"max":{"1min":6.3460000000000001,"5min":6.3460000000000001,"15min":7.0350000000000001},"last":4.4020000000000001},{"interface":"front","average":{"1min":4.9379999999999997,"5min":5.0720000000000001,"15min":5.242},"min":{"1min":3.4350000000000001,"5min":2.5409999999999999,"15min":2.5409999999999999},"max":{"1min":6.9269999999999996,"5min":6.9269999999999996,"15min":8.4399999999999995},"last":5.5549999999999997}]},{"osd":10,"last update":"Tue Sep 8 16:31:10 2020","interfaces":[{"interface":"back","average":{"1min":3.2200000000000002,"5min":3.5600000000000001,"15min":3.6949999999999998},"min":{"1min":1.796,"5min":1.796,"15min":1.796},"max":{"1min":5.2140000000000004,"5min":5.4829999999999997,"15min":5.4829999999999997},"last":4.1040000000000001},{"interface":"front","average":{"1min":3.0419999999999998,"5min":3.3929999999999998,"15min":3.548},"min":{"1min":1.905,"5min":1.905,"15min":1.905},"max":{"1min":3.746,"5min":5.2679999999999998,"15min":5.2679999999999998},"last":3.8650000000000002}]},{"osd":11,"last update":"Tue Sep 8 16:31:25 2020","interfaces":[{"interface":"back","average":{"1min":3.2650000000000001,"5min":3.758,"15min":3.7469999999999999},"min":{"1min":1.996,"5min":1.744,"15min":1.744},"max":{"1min":4.6280000000000001,"5min":5.7039999999999997,"15min":5.7400000000000002},"last":3.198},{"interface":"front","average":{"1min":3.431,"5min":3.879,"15min":3.9420000000000002},"min":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"max":{"1min":4.9749999999999996,"5min":5.9000000000000004,"15min":5.9000000000000004},"last":3.456}]}]},{"osd":2,"up_from":0,"seq":737,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":256424,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":255680,"kb_avail":93488728,"statfs":{"total":95995035648,"available":95732457472,"internally_reserved":0,"allocated":753664,"data_stored":442502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":261816320},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 16:30:45 2020","interfaces":[{"interface":"back","average":{"1min":2.298,"5min":2.1869999999999998,"15min":2.1869999999999998},"min":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.175},"max":{"1min":3.742,"5min":3.742,"15min":3.742},"last":2.6000000000000001},{"interface":"front","average":{"1min":2.4100000000000001,"5min":2.3319999999999999,"15min":2.3599999999999999},"min":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.22},"max":{"1min":4.4779999999999998,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":2.2639999999999998}]},{"osd":1,"last update":"Tue Sep 8 16:30:45 2020","interfaces":[{"interface":"back","average":{"1min":2.0409999999999999,"5min":2.1349999999999998,"15min":2.1469999999999998},"min":{"1min":1.468,"5min":1.127,"15min":0.83299999999999996},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.9340000000000002},"last":1.9159999999999999},{"interface":"front","average":{"1min":2.6269999999999998,"5min":2.9369999999999998,"15min":2.9670000000000001},"min":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.47},"max":{"1min":4.2549999999999999,"5min":4.2549999999999999,"15min":4.524},"last":1.9890000000000001}]},{"osd":3,"last update":"Tue Sep 8 16:31:34 2020","interfaces":[{"interface":"back","average":{"1min":2.9319999999999999,"5min":2.8239999999999998,"15min":2.8839999999999999},"min":{"1min":1.825,"5min":1.677,"15min":1.3300000000000001},"max":{"1min":3.8119999999999998,"5min":4.7370000000000001,"15min":4.7370000000000001},"last":2.4279999999999999},{"interface":"front","average":{"1min":3.173,"5min":3.181,"15min":3.2690000000000001},"min":{"1min":1.746,"5min":1.746,"15min":1.667},"max":{"1min":4.1319999999999997,"5min":4.8700000000000001,"15min":4.8700000000000001},"last":2.0939999999999999}]},{"osd":4,"last update":"Tue Sep 8 16:31:40 2020","interfaces":[{"interface":"back","average":{"1min":3.6240000000000001,"5min":3.5110000000000001,"15min":3.5299999999999998},"min":{"1min":2.524,"5min":2.1520000000000001,"15min":2.0209999999999999},"max":{"1min":4.7640000000000002,"5min":5.2679999999999998,"15min":6.1440000000000001},"last":2.883},{"interface":"front","average":{"1min":3.3980000000000001,"5min":3.6200000000000001,"15min":3.665},"min":{"1min":1.851,"5min":1.851,"15min":1.851},"max":{"1min":4.8860000000000001,"5min":5.1219999999999999,"15min":6.4480000000000004},"last":2.9580000000000002}]},{"osd":5,"last update":"Tue Sep 8 16:31:02 2020","interfaces":[{"interface":"back","average":{"1min":3.9940000000000002,"5min":3.9119999999999999,"15min":3.875},"min":{"1min":2.6269999999999998,"5min":2.2370000000000001,"15min":2.2370000000000001},"max":{"1min":5.4619999999999997,"5min":6.7830000000000004,"15min":6.7830000000000004},"last":2.7759999999999998},{"interface":"front","average":{"1min":4.3410000000000002,"5min":4.2249999999999996,"15min":4.2599999999999998},"min":{"1min":3.2210000000000001,"5min":2.6419999999999999,"15min":2.3620000000000001},"max":{"1min":6.5819999999999999,"5min":6.6760000000000002,"15min":6.6760000000000002},"last":3.1269999999999998}]},{"osd":6,"last update":"Tue Sep 8 16:31:34 2020","interfaces":[{"interface":"back","average":{"1min":3.8759999999999999,"5min":3.9079999999999999,"15min":3.9870000000000001},"min":{"1min":2.9870000000000001,"5min":2.649,"15min":1.982},"max":{"1min":7.0250000000000004,"5min":7.0250000000000004,"15min":7.0250000000000004},"last":3.5590000000000002},{"interface":"front","average":{"1min":4.4109999999999996,"5min":4.423,"15min":4.4729999999999999},"min":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"max":{"1min":7.9370000000000003,"5min":7.9370000000000003,"15min":7.9370000000000003},"last":3.6469999999999998}]},{"osd":7,"last update":"Tue Sep 8 16:31:02 2020","interfaces":[{"interface":"back","average":{"1min":4.1959999999999997,"5min":4.2400000000000002,"15min":4.1909999999999998},"min":{"1min":2.3700000000000001,"5min":2.3700000000000001,"15min":2.0600000000000001},"max":{"1min":6.0720000000000001,"5min":6.0720000000000001,"15min":6.194},"last":4.3070000000000004},{"interface":"front","average":{"1min":4.8490000000000002,"5min":4.5540000000000003,"15min":4.6390000000000002},"min":{"1min":3.589,"5min":3.415,"15min":2.9340000000000002},"max":{"1min":7.1459999999999999,"5min":7.1459999999999999,"15min":7.8710000000000004},"last":4.234}]},{"osd":8,"last update":"Tue Sep 8 16:31:02 2020","interfaces":[{"interface":"back","average":{"1min":5.032,"5min":4.681,"15min":4.3949999999999996},"min":{"1min":2.9420000000000002,"5min":2.9420000000000002,"15min":1.8540000000000001},"max":{"1min":7.1769999999999996,"5min":7.1769999999999996,"15min":7.1769999999999996},"last":3.476},{"interface":"front","average":{"1min":5.4550000000000001,"5min":5.1050000000000004,"15min":5.0570000000000004},"min":{"1min":3.835,"5min":3.7309999999999999,"15min":2.9790000000000001},"max":{"1min":7.4329999999999998,"5min":7.4329999999999998,"15min":7.4329999999999998},"last":3.8439999999999999}]},{"osd":9,"last update":"Tue Sep 8 16:30:45 2020","interfaces":[{"interface":"back","average":{"1min":5.3440000000000003,"5min":5.1669999999999998,"15min":5.0739999999999998},"min":{"1min":4.2140000000000004,"5min":3.5899999999999999,"15min":2.4769999999999999},"max":{"1min":7.6900000000000004,"5min":7.6900000000000004,"15min":7.6900000000000004},"last":4.0679999999999996},{"interface":"front","average":{"1min":5.5039999999999996,"5min":5.3890000000000002,"15min":5.3630000000000004},"min":{"1min":3.8639999999999999,"5min":3.8639999999999999,"15min":3.218},"max":{"1min":8.1869999999999994,"5min":8.1869999999999994,"15min":8.2219999999999995},"last":3.3180000000000001}]},{"osd":10,"last update":"Tue Sep 8 16:31:02 2020","interfaces":[{"interface":"back","average":{"1min":4.2190000000000003,"5min":4.3689999999999998,"15min":4.3579999999999997},"min":{"1min":2.8530000000000002,"5min":2.8530000000000002,"15min":2.8260000000000001},"max":{"1min":5.3310000000000004,"5min":6.3360000000000003,"15min":6.3360000000000003},"last":3.9489999999999998},{"interface":"front","average":{"1min":4.4930000000000003,"5min":4.5529999999999999,"15min":4.6020000000000003},"min":{"1min":3.052,"5min":2.734,"15min":2.5310000000000001},"max":{"1min":6.4610000000000003,"5min":6.7649999999999997,"15min":6.7649999999999997},"last":4.7210000000000001}]}]},{"osd":3,"up_from":0,"seq":731,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":442472,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":441728,"kb_avail":93302680,"statfs":{"total":95995035648,"available":95541944320,"internally_reserved":0,"allocated":753664,"data_stored":442502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":452329472},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 16:09:01 2020","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.627,"15min":2.7490000000000001},"min":{"1min":0.76800000000000002,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.1680000000000001,"5min":7.1109999999999998,"15min":85.766000000000005},"last":1.9299999999999999},{"interface":"front","average":{"1min":1.6759999999999999,"5min":1.956,"15min":2.7490000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.73499999999999999},"max":{"1min":6.9560000000000004,"5min":6.9560000000000004,"15min":41.011000000000003},"last":2.0110000000000001}]},{"osd":1,"last update":"Tue Sep 8 16:31:08 2020","interfaces":[{"interface":"back","average":{"1min":2.1619999999999999,"5min":2.173,"15min":2.1779999999999999},"min":{"1min":1.655,"5min":1.1040000000000001,"15min":1.1040000000000001},"max":{"1min":2.7130000000000001,"5min":3.1680000000000001,"15min":3.5339999999999998},"last":1.946},{"interface":"front","average":{"1min":2.8069999999999999,"5min":2.9319999999999999,"15min":2.7839999999999998},"min":{"1min":1.7210000000000001,"5min":1.3069999999999999,"15min":1.3069999999999999},"max":{"1min":3.7999999999999998,"5min":4.3730000000000002,"15min":4.3920000000000003},"last":2.7999999999999998}]},{"osd":2,"last update":"Tue Sep 8 16:31:28 2020","interfaces":[{"interface":"back","average":{"1min":2.0339999999999998,"5min":2.1680000000000001,"15min":2.2000000000000002},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":2.6549999999999998,"5min":3.1760000000000002,"15min":3.1760000000000002},"last":2.0459999999999998},{"interface":"front","average":{"1min":2.5950000000000002,"5min":2.8420000000000001,"15min":2.907},"min":{"1min":1.589,"5min":1.5569999999999999,"15min":1.5569999999999999},"max":{"1min":3.363,"5min":4.952,"15min":4.952},"last":2.6019999999999999}]},{"osd":4,"last update":"Tue Sep 8 16:30:42 2020","interfaces":[{"interface":"back","average":{"1min":2.7290000000000001,"5min":2.5649999999999999,"15min":2.702},"min":{"1min":1.7729999999999999,"5min":1.238,"15min":1.238},"max":{"1min":4.3630000000000004,"5min":4.8479999999999999,"15min":4.8479999999999999},"last":2.173},{"interface":"front","average":{"1min":3.0939999999999999,"5min":3.0710000000000002,"15min":3.1779999999999999},"min":{"1min":1.964,"5min":1.875,"15min":1.875},"max":{"1min":4.75,"5min":5.1749999999999998,"15min":5.1749999999999998},"last":3.069}]},{"osd":5,"last update":"Tue Sep 8 16:31:08 2020","interfaces":[{"interface":"back","average":{"1min":2.9940000000000002,"5min":3.0110000000000001,"15min":3.0169999999999999},"min":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"max":{"1min":4.1219999999999999,"5min":4.4249999999999998,"15min":4.8200000000000003},"last":2.6760000000000002},{"interface":"front","average":{"1min":3.8889999999999998,"5min":3.903,"15min":3.883},"min":{"1min":1.573,"5min":1.573,"15min":1.573},"max":{"1min":5.4580000000000002,"5min":6.7930000000000001,"15min":6.7930000000000001},"last":3.5510000000000002}]},{"osd":6,"last update":"Tue Sep 8 16:30:42 2020","interfaces":[{"interface":"back","average":{"1min":3.528,"5min":3.371,"15min":3.3799999999999999},"min":{"1min":2.8079999999999998,"5min":1.579,"15min":1.579},"max":{"1min":5.04,"5min":5.7649999999999997,"15min":6.3600000000000003},"last":3.0219999999999998},{"interface":"front","average":{"1min":3.9870000000000001,"5min":3.8519999999999999,"15min":3.9039999999999999},"min":{"1min":2.968,"5min":1.788,"15min":1.788},"max":{"1min":5.8390000000000004,"5min":5.9349999999999996,"15min":6.0019999999999998},"last":3.2909999999999999}]},{"osd":7,"last update":"Tue Sep 8 16:31:08 2020","interfaces":[{"interface":"back","average":{"1min":3.2120000000000002,"5min":3.4100000000000001,"15min":3.883},"min":{"1min":1.6899999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"max":{"1min":4.8550000000000004,"5min":5.2560000000000002,"15min":6.0540000000000003},"last":3.3769999999999998},{"interface":"front","average":{"1min":4.2910000000000004,"5min":4.2919999999999998,"15min":4.0789999999999997},"min":{"1min":3.0680000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"max":{"1min":5.4390000000000001,"5min":5.8739999999999997,"15min":6.9820000000000002},"last":4.0449999999999999}]},{"osd":8,"last update":"Tue Sep 8 16:31:28 2020","interfaces":[{"interface":"back","average":{"1min":3.8759999999999999,"5min":3.9449999999999998,"15min":3.8889999999999998},"min":{"1min":2.4420000000000002,"5min":1.3939999999999999,"15min":1.3939999999999999},"max":{"1min":5.484,"5min":6.2859999999999996,"15min":6.2859999999999996},"last":4.5259999999999998},{"interface":"front","average":{"1min":3.8809999999999998,"5min":4.0279999999999996,"15min":4.1299999999999999},"min":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"max":{"1min":5.5999999999999996,"5min":6.7759999999999998,"15min":6.7759999999999998},"last":4.0890000000000004}]},{"osd":9,"last update":"Tue Sep 8 16:30:42 2020","interfaces":[{"interface":"back","average":{"1min":4.5949999999999998,"5min":4.6210000000000004,"15min":4.6840000000000002},"min":{"1min":2.7029999999999998,"5min":2.585,"15min":2.4159999999999999},"max":{"1min":6.218,"5min":6.3300000000000001,"15min":6.9500000000000002},"last":3.8109999999999999},{"interface":"front","average":{"1min":5.0380000000000003,"5min":5.0839999999999996,"15min":4.9909999999999997},"min":{"1min":3.7839999999999998,"5min":3.0960000000000001,"15min":2.915},"max":{"1min":6.694,"5min":6.8520000000000003,"15min":7.4640000000000004},"last":4.4660000000000002}]},{"osd":10,"last update":"Tue Sep 8 16:31:28 2020","interfaces":[{"interface":"back","average":{"1min":3.6869999999999998,"5min":3.887,"15min":3.8769999999999998},"min":{"1min":1.879,"5min":1.879,"15min":1.879},"max":{"1min":4.7329999999999997,"5min":6.1139999999999999,"15min":6.1139999999999999},"last":3.9140000000000001},{"interface":"front","average":{"1min":4.4509999999999996,"5min":4.5279999999999996,"15min":4.4640000000000004},"min":{"1min":2.9209999999999998,"5min":2.9209999999999998,"15min":2.9209999999999998},"max":{"1min":6.2809999999999997,"5min":7.1779999999999999,"15min":7.5419999999999998},"last":4.9180000000000001}]},{"osd":11,"last update":"Tue Sep 8 16:30:42 2020","interfaces":[{"interface":"back","average":{"1min":4.2439999999999998,"5min":4.2750000000000004,"15min":4.4509999999999996},"min":{"1min":3.4790000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"max":{"1min":6.1520000000000001,"5min":8.1600000000000001,"15min":8.1600000000000001},"last":3.6269999999999998},{"interface":"front","average":{"1min":4.9589999999999996,"5min":4.9240000000000004,"15min":4.7460000000000004},"min":{"1min":3.8519999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"max":{"1min":7.3390000000000004,"5min":7.3390000000000004,"15min":7.3390000000000004},"last":4.2690000000000001}]}]},{"osd":4,"up_from":0,"seq":727,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":347228,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":346496,"kb_avail":93397924,"statfs":{"total":95995035648,"available":95639474176,"internally_reserved":0,"allocated":741376,"data_stored":434987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":354811904},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 16:30:49 2020","interfaces":[{"interface":"back","average":{"1min":2.407,"5min":2.3220000000000001,"15min":2.339},"min":{"1min":1.907,"5min":1.2310000000000001,"15min":1.2290000000000001},"max":{"1min":4.1719999999999997,"5min":5.7969999999999997,"15min":5.7969999999999997},"last":2.5259999999999998},{"interface":"front","average":{"1min":3.9380000000000002,"5min":3.8180000000000001,"15min":3.8159999999999998},"min":{"1min":2.9750000000000001,"5min":1.8859999999999999,"15min":1.8859999999999999},"max":{"1min":4.9710000000000001,"5min":5.9470000000000001,"15min":5.9470000000000001},"last":4.4059999999999997}]},{"osd":1,"last update":"Tue Sep 8 16:30:49 2020","interfaces":[{"interface":"back","average":{"1min":3.105,"5min":3.2250000000000001,"15min":3.2959999999999998},"min":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"max":{"1min":4.3380000000000001,"5min":6.1040000000000001,"15min":6.1040000000000001},"last":3.98},{"interface":"front","average":{"1min":2.5019999999999998,"5min":2.4580000000000002,"15min":2.4870000000000001},"min":{"1min":1.165,"5min":1.165,"15min":0.90800000000000003},"max":{"1min":3.734,"5min":4.101,"15min":4.5129999999999999},"last":3.802}]},{"osd":2,"last update":"Tue Sep 8 16:31:00 2020","interfaces":[{"interface":"back","average":{"1min":3.4969999999999999,"5min":3.5310000000000001,"15min":3.4710000000000001},"min":{"1min":2.419,"5min":1.637,"15min":1.637},"max":{"1min":4.7400000000000002,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":4.6520000000000001},{"interface":"front","average":{"1min":2.7229999999999999,"5min":2.8260000000000001,"15min":2.8460000000000001},"min":{"1min":1.738,"5min":1.5669999999999999,"15min":1.3640000000000001},"max":{"1min":3.75,"5min":4.6580000000000004,"15min":4.6580000000000004},"last":2.6040000000000001}]},{"osd":3,"last update":"Tue Sep 8 16:30:53 2020","interfaces":[{"interface":"back","average":{"1min":2.2919999999999998,"5min":2.4119999999999999,"15min":2.4660000000000002},"min":{"1min":1.24,"5min":1.24,"15min":1.24},"max":{"1min":2.827,"5min":4.8609999999999998,"15min":4.8609999999999998},"last":2.702},{"interface":"front","average":{"1min":3.923,"5min":4.0149999999999997,"15min":3.9980000000000002},"min":{"1min":2.1320000000000001,"5min":1.734,"15min":1.734},"max":{"1min":5.1139999999999999,"5min":8.2769999999999992,"15min":8.2769999999999992},"last":4.8789999999999996}]},{"osd":5,"last update":"Tue Sep 8 16:31:00 2020","interfaces":[{"interface":"back","average":{"1min":2.7210000000000001,"5min":2.702,"15min":2.7490000000000001},"min":{"1min":1.417,"5min":1.228,"15min":1.228},"max":{"1min":3.4199999999999999,"5min":4.5010000000000003,"15min":4.5010000000000003},"last":3.028},{"interface":"front","average":{"1min":3.4329999999999998,"5min":3.4889999999999999,"15min":3.6499999999999999},"min":{"1min":1.948,"5min":1.794,"15min":1.7390000000000001},"max":{"1min":4.0670000000000002,"5min":6.1600000000000001,"15min":6.9699999999999998},"last":4.2519999999999998}]},{"osd":6,"last update":"Tue Sep 8 16:30:45 2020","interfaces":[{"interface":"back","average":{"1min":4.3970000000000002,"5min":4.2880000000000003,"15min":4.3070000000000004},"min":{"1min":3.2010000000000001,"5min":1.556,"15min":1.556},"max":{"1min":5.9640000000000004,"5min":7.8689999999999998,"15min":7.8689999999999998},"last":5.0940000000000003},{"interface":"front","average":{"1min":3.6629999999999998,"5min":3.6629999999999998,"15min":3.6469999999999998},"min":{"1min":2.573,"5min":1.944,"15min":1.2150000000000001},"max":{"1min":4.3760000000000003,"5min":5.9800000000000004,"15min":7.3220000000000001},"last":4.5590000000000002}]},{"osd":7,"last update":"Tue Sep 8 16:31:00 2020","interfaces":[{"interface":"back","average":{"1min":4.8840000000000003,"5min":4.742,"15min":4.1200000000000001},"min":{"1min":3.4399999999999999,"5min":2.069,"15min":2.0680000000000001},"max":{"1min":5.7930000000000001,"5min":7.4630000000000001,"15min":7.4630000000000001},"last":5.476},{"interface":"front","average":{"1min":4.101,"5min":3.9889999999999999,"15min":4.0599999999999996},"min":{"1min":2.2400000000000002,"5min":2.0550000000000002,"15min":1.982},"max":{"1min":4.8040000000000003,"5min":5.6559999999999997,"15min":7.3710000000000004},"last":4.7750000000000004}]},{"osd":8,"last update":"Tue Sep 8 16:30:45 2020","interfaces":[{"interface":"back","average":{"1min":5.0309999999999997,"5min":4.9359999999999999,"15min":4.8949999999999996},"min":{"1min":3.6659999999999999,"5min":2.3759999999999999,"15min":2.3759999999999999},"max":{"1min":6.8250000000000002,"5min":8.4580000000000002,"15min":8.4580000000000002},"last":5.2910000000000004},{"interface":"front","average":{"1min":5.4039999999999999,"5min":5.3819999999999997,"15min":5.226},"min":{"1min":4.1200000000000001,"5min":3.145,"15min":3.0680000000000001},"max":{"1min":7.2240000000000002,"5min":8.7639999999999993,"15min":8.7639999999999993},"last":5.9080000000000004}]},{"osd":9,"last update":"Tue Sep 8 16:31:00 2020","interfaces":[{"interface":"back","average":{"1min":3.5950000000000002,"5min":3.496,"15min":3.6640000000000001},"min":{"1min":1.556,"5min":1.556,"15min":1.4610000000000001},"max":{"1min":4.5019999999999998,"5min":9.3640000000000008,"15min":9.3640000000000008},"last":4.0780000000000003},{"interface":"front","average":{"1min":5.4729999999999999,"5min":5.3280000000000003,"15min":5.29},"min":{"1min":3.8980000000000001,"5min":2.7839999999999998,"15min":2.7839999999999998},"max":{"1min":6.5469999999999997,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":5.6799999999999997}]},{"osd":10,"last update":"Tue Sep 8 16:31:00 2020","interfaces":[{"interface":"back","average":{"1min":4.2329999999999997,"5min":4.3040000000000003,"15min":4.3869999999999996},"min":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.0529999999999999},"max":{"1min":5.1900000000000004,"5min":6.7569999999999997,"15min":7.8159999999999998},"last":4.9989999999999997},{"interface":"front","average":{"1min":4.5819999999999999,"5min":4.6870000000000003,"15min":4.7549999999999999},"min":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"max":{"1min":5.6260000000000003,"5min":7.282,"15min":8.2330000000000005},"last":5.1900000000000004}]}]},{"osd":5,"up_from":0,"seq":719,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":443356,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":442624,"kb_avail":93301796,"statfs":{"total":95995035648,"available":95541039104,"internally_reserved":0,"allocated":741376,"data_stored":434987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":453246976},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 8 15:42:35 2020","interfaces":[{"interface":"back","average":{"1min":2.0920000000000001,"5min":1.629,"15min":1.871},"min":{"1min":0.77500000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":5.4400000000000004,"5min":5.4400000000000004,"15min":5.4400000000000004},"last":0.84399999999999997},{"interface":"front","average":{"1min":2.5139999999999998,"5min":2.246,"15min":2.1469999999999998},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":4.9320000000000004,"5min":7.1340000000000003,"15min":7.1340000000000003},"last":2.258}]},{"osd":1,"last update":"Tue Sep 8 16:31:19 2020","interfaces":[{"interface":"back","average":{"1min":2.2250000000000001,"5min":2.1869999999999998,"15min":2.1880000000000002},"min":{"1min":1.786,"5min":1.1759999999999999,"15min":1.1759999999999999},"max":{"1min":2.677,"5min":2.952,"15min":3.262},"last":3.266},{"interface":"front","average":{"1min":3.6240000000000001,"5min":3.5710000000000002,"15min":3.657},"min":{"1min":2.0699999999999998,"5min":1.9590000000000001,"15min":1.409},"max":{"1min":5.2750000000000004,"5min":5.3170000000000002,"15min":5.6070000000000002},"last":2.819}]},{"osd":2,"last update":"Tue Sep 8 16:31:13 2020","interfaces":[{"interface":"back","average":{"1min":2.899,"5min":2.5609999999999999,"15min":2.5089999999999999},"min":{"1min":1.8540000000000001,"5min":1.758,"15min":1.6890000000000001},"max":{"1min":4.8460000000000001,"5min":6.2249999999999996,"15min":6.2249999999999996},"last":2.3570000000000002},{"interface":"front","average":{"1min":2.9950000000000001,"5min":3.161,"15min":3.0329999999999999},"min":{"1min":1.851,"5min":1.45,"15min":1.45},"max":{"1min":4.8449999999999998,"5min":5.9550000000000001,"15min":5.9550000000000001},"last":3.1739999999999999}]},{"osd":3,"last update":"Tue Sep 8 16:31:13 2020","interfaces":[{"interface":"back","average":{"1min":2.3860000000000001,"5min":2.3959999999999999,"15min":2.399},"min":{"1min":1.9279999999999999,"5min":1.446,"15min":1.446},"max":{"1min":3.4100000000000001,"5min":3.972,"15min":3.972},"last":1.8049999999999999},{"interface":"front","average":{"1min":3.073,"5min":3.544,"15min":3.5110000000000001},"min":{"1min":2.222,"5min":1.978,"15min":1.3580000000000001},"max":{"1min":4.4349999999999996,"5min":6.4649999999999999,"15min":6.4649999999999999},"last":3.5800000000000001}]},{"osd":4,"last update":"Tue Sep 8 16:31:13 2020","interfaces":[{"interface":"back","average":{"1min":2.8580000000000001,"5min":2.7240000000000002,"15min":2.6800000000000002},"min":{"1min":1.9690000000000001,"5min":1.673,"15min":1.673},"max":{"1min":3.923,"5min":4.8209999999999997,"15min":5.423},"last":2.2240000000000002},{"interface":"front","average":{"1min":3.2690000000000001,"5min":3.1869999999999998,"15min":3.137},"min":{"1min":2.5110000000000001,"5min":1.5649999999999999,"15min":1.3009999999999999},"max":{"1min":5.0179999999999998,"5min":5.4139999999999997,"15min":6.4249999999999998},"last":3.0870000000000002}]},{"osd":6,"last update":"Tue Sep 8 16:31:41 2020","interfaces":[{"interface":"back","average":{"1min":4.1639999999999997,"5min":4.0039999999999996,"15min":3.923},"min":{"1min":2.5870000000000002,"5min":1.7609999999999999,"15min":1.7609999999999999},"max":{"1min":5.4459999999999997,"5min":6.6909999999999998,"15min":6.6909999999999998},"last":4.6150000000000002},{"interface":"front","average":{"1min":4.6769999999999996,"5min":4.5010000000000003,"15min":4.4530000000000003},"min":{"1min":3.7080000000000002,"5min":2.6949999999999998,"15min":2.1240000000000001},"max":{"1min":5.6550000000000002,"5min":6.9139999999999997,"15min":7.9290000000000003},"last":5.6200000000000001}]},{"osd":7,"last update":"Tue Sep 8 16:31:13 2020","interfaces":[{"interface":"back","average":{"1min":4.7300000000000004,"5min":4.6239999999999997,"15min":4.6200000000000001},"min":{"1min":3.7189999999999999,"5min":2.556,"15min":2.327},"max":{"1min":6.524,"5min":7.157,"15min":8.3130000000000006},"last":4.0460000000000003},{"interface":"front","average":{"1min":5.4279999999999999,"5min":5.2539999999999996,"15min":5.2270000000000003},"min":{"1min":4.2759999999999998,"5min":3.1269999999999998,"15min":3.1269999999999998},"max":{"1min":7.7759999999999998,"5min":8.1310000000000002,"15min":8.5210000000000008},"last":5.8239999999999998}]},{"osd":8,"last update":"Tue Sep 8 16:31:19 2020","interfaces":[{"interface":"back","average":{"1min":5.5640000000000001,"5min":5.2119999999999997,"15min":5.1369999999999996},"min":{"1min":4.4770000000000003,"5min":3.246,"15min":3.246},"max":{"1min":7.242,"5min":7.415,"15min":7.415},"last":5.4089999999999998},{"interface":"front","average":{"1min":5.9980000000000002,"5min":5.5149999999999997,"15min":5.46},"min":{"1min":4.282,"5min":3.0169999999999999,"15min":3.0169999999999999},"max":{"1min":8.593,"5min":8.593,"15min":8.593},"last":4.8259999999999996}]},{"osd":9,"last update":"Tue Sep 8 16:30:52 2020","interfaces":[{"interface":"back","average":{"1min":6.1790000000000003,"5min":4.694,"15min":4.1210000000000004},"min":{"1min":4.6539999999999999,"5min":2.7010000000000001,"15min":2.5150000000000001},"max":{"1min":8.2089999999999996,"5min":8.2089999999999996,"15min":8.2089999999999996},"last":5.1280000000000001},{"interface":"front","average":{"1min":3.831,"5min":4.8360000000000003,"15min":5.3710000000000004},"min":{"1min":3.0640000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"max":{"1min":5.5750000000000002,"5min":7.9139999999999997,"15min":8.0969999999999995},"last":3.863}]},{"osd":10,"last update":"Tue Sep 8 16:31:19 2020","interfaces":[{"interface":"back","average":{"1min":6.1980000000000004,"5min":4.5979999999999999,"15min":3.9119999999999999},"min":{"1min":4.407,"5min":2.4169999999999998,"15min":1.9379999999999999},"max":{"1min":8.6229999999999993,"5min":8.6229999999999993,"15min":8.6229999999999993},"last":6.2489999999999997},{"interface":"front","average":{"1min":6.5650000000000004,"5min":5.9210000000000003,"15min":5.6790000000000003},"min":{"1min":5.0720000000000001,"5min":3.6400000000000001,"15min":2.4729999999999999},"max":{"1min":9.0060000000000002,"5min":9.0060000000000002,"15min":9.0060000000000002},"last":6.46}]},{"osd":11,"last update":"Tue Sep 8 16:31:13 2020","interfaces":[{"interface":"back","average":{"1min":3.6309999999999998,"5min":3.6309999999999998,"15min":3.7069999999999999},"min":{"1min":3.0449999999999999,"5min":2.1829999999999998,"15min":2.1829999999999998},"max":{"1min":4.3330000000000002,"5min":4.7649999999999997,"15min":4.9820000000000002},"last":4.5300000000000002},{"interface":"front","average":{"1min":4.0810000000000004,"5min":4.0449999999999999,"15min":4.0259999999999998},"min":{"1min":3.3959999999999999,"5min":1.96,"15min":1.96},"max":{"1min":5.3869999999999996,"5min":6.8159999999999998,"15min":7.5110000000000001},"last":4.1349999999999998}]}]}]}} 2020-09-08T16:31:44.506 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2020-09-08T16:31:44.507 INFO:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dce4d291bd4243c9802c17d67b85a6da6bbc0f8f shell --fsid 759b27b6-f1e7-11ea-a080-001a4aab830c -- ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2020-09-08T16:31:44.641 INFO:teuthology.orchestra.run.smithi104.stderr:INFO:cephadm:Inferring config /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/mon.a/config 2020-09-08T16:31:44.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:44 smithi055 bash[9296]: cluster 2020-09-08T16:31:43.353278+0000 osd.3 (osd.3) 179 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:44.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:44 smithi055 bash[9296]: cluster 2020-09-08T16:31:43.353471+0000 osd.3 (osd.3) 180 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:44.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:44 smithi055 bash[9296]: cluster 2020-09-08T16:31:43.649903+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e975: 12 total, 12 up, 12 in 2020-09-08T16:31:44.908 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:44 smithi055 bash[9296]: audit 2020-09-08T16:31:43.731283+0000 mgr.x (mgr.24118) 1933 : audit [DBG] from='client.17097 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T16:31:44.925 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:44 smithi104 bash[9203]: cluster 2020-09-08T16:31:43.353278+0000 osd.3 (osd.3) 179 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:44.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:44 smithi104 bash[9203]: cluster 2020-09-08T16:31:43.353471+0000 osd.3 (osd.3) 180 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:44.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:44 smithi104 bash[9203]: cluster 2020-09-08T16:31:43.649903+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e975: 12 total, 12 up, 12 in 2020-09-08T16:31:44.926 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:44 smithi104 bash[9203]: audit 2020-09-08T16:31:43.731283+0000 mgr.x (mgr.24118) 1933 : audit [DBG] from='client.17097 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T16:31:45.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:44 smithi166 bash[9446]: cluster 2020-09-08T16:31:43.353278+0000 osd.3 (osd.3) 179 : cluster [DBG] purged_snaps scrub starts 2020-09-08T16:31:45.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:44 smithi166 bash[9446]: cluster 2020-09-08T16:31:43.353471+0000 osd.3 (osd.3) 180 : cluster [DBG] purged_snaps scrub ok 2020-09-08T16:31:45.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:44 smithi166 bash[9446]: cluster 2020-09-08T16:31:43.649903+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e975: 12 total, 12 up, 12 in 2020-09-08T16:31:45.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:44 smithi166 bash[9446]: audit 2020-09-08T16:31:43.731283+0000 mgr.x (mgr.24118) 1933 : audit [DBG] from='client.17097 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2020-09-08T16:31:45.857 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:45 smithi104 bash[9203]: cluster 2020-09-08T16:31:44.549034+0000 mgr.x (mgr.24118) 1934 : cluster [DBG] pgmap v2823: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:45.907 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:45 smithi055 bash[9296]: cluster 2020-09-08T16:31:44.549034+0000 mgr.x (mgr.24118) 1934 : cluster [DBG] pgmap v2823: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:46.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:45 smithi166 bash[9446]: cluster 2020-09-08T16:31:44.549034+0000 mgr.x (mgr.24118) 1934 : cluster [DBG] pgmap v2823: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:47.661 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:47 smithi104 bash[9203]: cluster 2020-09-08T16:31:46.549683+0000 mgr.x (mgr.24118) 1935 : cluster [DBG] pgmap v2824: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:47.661 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:47 smithi104 bash[9203]: audit 2020-09-08T16:31:47.652022+0000 mon.a (mon.0) 2655 : audit [INF] from='client.? v1:172.21.15.104:0/158350038' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2020-09-08T16:31:47.661 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:47 smithi104 bash[9203]: audit 2020-09-08T16:31:47.660053+0000 mon.a (mon.0) 2656 : audit [INF] from='client.? v1:172.21.15.104:0/158350038' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2020-09-08T16:31:48.116 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:47 smithi166 bash[9446]: cluster 2020-09-08T16:31:46.549683+0000 mgr.x (mgr.24118) 1935 : cluster [DBG] pgmap v2824: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:48.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:47 smithi166 bash[9446]: audit 2020-09-08T16:31:47.652022+0000 mon.a (mon.0) 2655 : audit [INF] from='client.? v1:172.21.15.104:0/158350038' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2020-09-08T16:31:48.117 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:47 smithi166 bash[9446]: audit 2020-09-08T16:31:47.660053+0000 mon.a (mon.0) 2656 : audit [INF] from='client.? v1:172.21.15.104:0/158350038' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2020-09-08T16:31:48.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:47 smithi055 bash[9296]: cluster 2020-09-08T16:31:46.549683+0000 mgr.x (mgr.24118) 1935 : cluster [DBG] pgmap v2824: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:48.157 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:47 smithi055 bash[9296]: audit 2020-09-08T16:31:47.652022+0000 mon.a (mon.0) 2655 : audit [INF] from='client.? v1:172.21.15.104:0/158350038' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2020-09-08T16:31:48.158 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:47 smithi055 bash[9296]: audit 2020-09-08T16:31:47.660053+0000 mon.a (mon.0) 2656 : audit [INF] from='client.? v1:172.21.15.104:0/158350038' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2020-09-08T16:31:48.413 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2020-09-08T16:31:52.652 INFO:tasks.cephadm:Teardown begin 2020-09-08T16:31:52.662 INFO:teuthology.orchestra.run.smithi055:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2020-09-08T16:31:52.664 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:48 smithi055 bash[9296]: audit 2020-09-08T16:31:47.672315+0000 mon.b (mon.1) 140 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:31:52.665 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:48 smithi055 bash[9296]: audit 2020-09-08T16:31:47.674201+0000 mon.a (mon.0) 2657 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:31:52.665 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:48 smithi055 bash[9296]: audit 2020-09-08T16:31:47.677206+0000 mon.a (mon.0) 2658 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:31:52.665 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:49 smithi055 bash[9296]: cluster 2020-09-08T16:31:48.550403+0000 mgr.x (mgr.24118) 1936 : cluster [DBG] pgmap v2825: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:52.666 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:51 smithi055 bash[9296]: cluster 2020-09-08T16:31:50.551058+0000 mgr.x (mgr.24118) 1937 : cluster [DBG] pgmap v2826: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:52.666 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:48 smithi166 bash[9446]: audit 2020-09-08T16:31:47.672315+0000 mon.b (mon.1) 140 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:31:52.666 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:48 smithi166 bash[9446]: audit 2020-09-08T16:31:47.674201+0000 mon.a (mon.0) 2657 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:31:52.667 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:48 smithi166 bash[9446]: audit 2020-09-08T16:31:47.677206+0000 mon.a (mon.0) 2658 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:31:52.667 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:49 smithi166 bash[9446]: cluster 2020-09-08T16:31:48.550403+0000 mgr.x (mgr.24118) 1936 : cluster [DBG] pgmap v2825: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:52.667 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:51 smithi166 bash[9446]: cluster 2020-09-08T16:31:50.551058+0000 mgr.x (mgr.24118) 1937 : cluster [DBG] pgmap v2826: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:52.667 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:48 smithi104 bash[9203]: audit 2020-09-08T16:31:47.672315+0000 mon.b (mon.1) 140 : audit [INF] from='mgr.24118 v1:172.21.15.55:0/1461277740' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:31:52.668 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:48 smithi104 bash[9203]: audit 2020-09-08T16:31:47.674201+0000 mon.a (mon.0) 2657 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd=[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]: dispatch 2020-09-08T16:31:52.668 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:48 smithi104 bash[9203]: audit 2020-09-08T16:31:47.677206+0000 mon.a (mon.0) 2658 : audit [INF] from='mgr.24118 ' entity='mgr.x' cmd='[{"prefix":"config-key set","key":"mgr/cephadm/osd_remove_queue","val":"[]"}]': finished 2020-09-08T16:31:52.668 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:49 smithi104 bash[9203]: cluster 2020-09-08T16:31:48.550403+0000 mgr.x (mgr.24118) 1936 : cluster [DBG] pgmap v2825: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:52.668 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:51 smithi104 bash[9203]: cluster 2020-09-08T16:31:50.551058+0000 mgr.x (mgr.24118) 1937 : cluster [DBG] pgmap v2826: 1 pgs: 1 active+clean; 0 B data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2020-09-08T16:31:52.711 INFO:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2020-09-08T16:31:52.749 INFO:teuthology.orchestra.run.smithi166:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2020-09-08T16:31:52.789 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2020-09-08T16:31:52.790 INFO:teuthology.orchestra.run.smithi055:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2020-09-08T16:31:52.797 INFO:teuthology.orchestra.run.smithi104:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2020-09-08T16:31:52.824 INFO:teuthology.orchestra.run.smithi166:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2020-09-08T16:31:52.837 INFO:tasks.cephadm:Stopping all daemons... 2020-09-08T16:31:52.837 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2020-09-08T16:31:52.838 INFO:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mon.a 2020-09-08T16:31:53.017 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:52 smithi104 systemd[1]: Stopping Ceph mon.a for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T16:31:53.425 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:53 smithi104 bash[9203]: debug 2020-09-08T16:31:53.018+0000 7efd0b27f700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:31:53.426 INFO:journalctl@ceph.mon.a.smithi104.stdout:Sep 08 16:31:53 smithi104 bash[9203]: debug 2020-09-08T16:31:53.018+0000 7efd0b27f700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2020-09-08T16:31:53.965 INFO:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mon.a.service' 2020-09-08T16:31:54.027 DEBUG:teuthology.orchestra.run:got remote process result: None 2020-09-08T16:31:54.028 INFO:tasks.cephadm.mon.a:Stopped mon.a 2020-09-08T16:31:54.028 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2020-09-08T16:31:54.029 INFO:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mon.b 2020-09-08T16:31:54.407 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:54 smithi055 systemd[1]: Stopping Ceph mon.b for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T16:31:54.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:54 smithi055 bash[9296]: debug 2020-09-08T16:31:54.171+0000 7ff83e4e8700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:31:54.408 INFO:journalctl@ceph.mon.b.smithi055.stdout:Sep 08 16:31:54 smithi055 bash[9296]: debug 2020-09-08T16:31:54.171+0000 7ff83e4e8700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2020-09-08T16:31:55.282 INFO:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mon.b.service' 2020-09-08T16:31:55.321 DEBUG:teuthology.orchestra.run:got remote process result: None 2020-09-08T16:31:55.322 INFO:tasks.cephadm.mon.b:Stopped mon.b 2020-09-08T16:31:55.323 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2020-09-08T16:31:55.323 INFO:teuthology.orchestra.run.smithi166:> sudo systemctl stop ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mon.c 2020-09-08T16:31:55.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:55 smithi166 systemd[1]: Stopping Ceph mon.c for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T16:31:55.617 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:55 smithi166 bash[9446]: debug 2020-09-08T16:31:55.490+0000 7f01add36700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:31:55.618 INFO:journalctl@ceph.mon.c.smithi166.stdout:Sep 08 16:31:55 smithi166 bash[9446]: debug 2020-09-08T16:31:55.490+0000 7f01add36700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2020-09-08T16:31:56.589 INFO:teuthology.orchestra.run.smithi166:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mon.c.service' 2020-09-08T16:31:56.674 DEBUG:teuthology.orchestra.run:got remote process result: None 2020-09-08T16:31:56.674 INFO:tasks.cephadm.mon.c:Stopped mon.c 2020-09-08T16:31:56.675 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2020-09-08T16:31:56.675 INFO:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mgr.y 2020-09-08T16:31:56.709 INFO:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mgr.y.service' 2020-09-08T16:31:56.797 DEBUG:teuthology.orchestra.run:got remote process result: None 2020-09-08T16:31:56.798 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2020-09-08T16:31:56.798 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2020-09-08T16:31:56.799 INFO:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mgr.x 2020-09-08T16:31:57.063 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Sep 08 16:31:56 smithi055 systemd[1]: Stopping Ceph mgr.x for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T16:31:57.063 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Sep 08 16:31:56 smithi055 bash[9970]: debug 2020-09-08T16:31:56.971+0000 7fa0b8199700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:31:57.064 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Sep 08 16:31:56 smithi055 bash[9970]: debug 2020-09-08T16:31:56.971+0000 7fa0b8199700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2020-09-08T16:31:57.999 INFO:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@mgr.x.service' 2020-09-08T16:31:58.019 DEBUG:teuthology.orchestra.run:got remote process result: None 2020-09-08T16:31:58.020 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2020-09-08T16:31:58.020 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2020-09-08T16:31:58.020 INFO:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.0 2020-09-08T16:31:58.346 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:31:58 smithi104 systemd[1]: Stopping Ceph osd.0 for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T16:31:58.346 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:31:58 smithi104 bash[15273]: debug 2020-09-08T16:31:58.178+0000 7f5a7fe58700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:31:58.347 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:31:58 smithi104 bash[15273]: debug 2020-09-08T16:31:58.178+0000 7f5a7fe58700 -1 osd.0 976 *** Got signal Terminated *** 2020-09-08T16:31:58.347 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:31:58 smithi104 bash[15273]: debug 2020-09-08T16:31:58.178+0000 7f5a7fe58700 -1 osd.0 976 *** Immediate shutdown (osd_fast_shutdown=true) *** 2020-09-08T16:31:59.425 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:31:59 smithi104 docker[157185]: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0 2020-09-08T16:31:59.426 INFO:journalctl@ceph.osd.0.smithi104.stdout:Sep 08 16:31:59 smithi104 bash[157225]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.0-deactivate 2020-09-08T16:32:02.502 INFO:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.0.service' 2020-09-08T16:32:02.527 DEBUG:teuthology.orchestra.run:got remote process result: None 2020-09-08T16:32:02.528 INFO:tasks.cephadm.osd.0:Stopped osd.0 2020-09-08T16:32:02.528 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2020-09-08T16:32:02.529 INFO:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.1 2020-09-08T16:32:02.855 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:32:02 smithi104 systemd[1]: Stopping Ceph osd.1 for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T16:32:02.855 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:32:02 smithi104 bash[17588]: debug 2020-09-08T16:32:02.710+0000 7fc3a2794700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:32:02.855 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:32:02 smithi104 bash[17588]: debug 2020-09-08T16:32:02.710+0000 7fc3a2794700 -1 osd.1 976 *** Got signal Terminated *** 2020-09-08T16:32:02.855 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:32:02 smithi104 bash[17588]: debug 2020-09-08T16:32:02.710+0000 7fc3a2794700 -1 osd.1 976 *** Immediate shutdown (osd_fast_shutdown=true) *** 2020-09-08T16:32:03.925 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:32:03 smithi104 docker[157329]: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1 2020-09-08T16:32:03.926 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 08 16:32:03 smithi104 bash[157371]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.1-deactivate 2020-09-08T16:32:05.560 INFO:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.1.service' 2020-09-08T16:32:05.583 DEBUG:teuthology.orchestra.run:got remote process result: None 2020-09-08T16:32:05.584 INFO:tasks.cephadm.osd.1:Stopped osd.1 2020-09-08T16:32:05.584 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2020-09-08T16:32:05.585 INFO:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.2 2020-09-08T16:32:05.904 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:32:05 smithi104 systemd[1]: Stopping Ceph osd.2 for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T16:32:05.904 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:32:05 smithi104 bash[19922]: debug 2020-09-08T16:32:05.770+0000 7fe387e29700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:32:05.905 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:32:05 smithi104 bash[19922]: debug 2020-09-08T16:32:05.770+0000 7fe387e29700 -1 osd.2 976 *** Got signal Terminated *** 2020-09-08T16:32:05.905 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:32:05 smithi104 bash[19922]: debug 2020-09-08T16:32:05.770+0000 7fe387e29700 -1 osd.2 976 *** Immediate shutdown (osd_fast_shutdown=true) *** 2020-09-08T16:32:06.925 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:32:06 smithi104 docker[157475]: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2 2020-09-08T16:32:06.926 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 08 16:32:06 smithi104 bash[157515]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.2-deactivate 2020-09-08T16:32:08.636 INFO:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.2.service' 2020-09-08T16:32:08.664 DEBUG:teuthology.orchestra.run:got remote process result: None 2020-09-08T16:32:08.664 INFO:tasks.cephadm.osd.2:Stopped osd.2 2020-09-08T16:32:08.665 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2020-09-08T16:32:08.665 INFO:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.3 2020-09-08T16:32:08.925 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:32:08 smithi104 systemd[1]: Stopping Ceph osd.3 for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T16:32:08.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:32:08 smithi104 bash[22305]: debug 2020-09-08T16:32:08.838+0000 7fb4a1e38700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:32:08.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:32:08 smithi104 bash[22305]: debug 2020-09-08T16:32:08.838+0000 7fb4a1e38700 -1 osd.3 976 *** Got signal Terminated *** 2020-09-08T16:32:08.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:32:08 smithi104 bash[22305]: debug 2020-09-08T16:32:08.838+0000 7fb4a1e38700 -1 osd.3 976 *** Immediate shutdown (osd_fast_shutdown=true) *** 2020-09-08T16:32:09.925 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:32:09 smithi104 docker[157620]: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3 2020-09-08T16:32:09.926 INFO:journalctl@ceph.osd.3.smithi104.stdout:Sep 08 16:32:09 smithi104 bash[157660]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.3-deactivate 2020-09-08T16:32:11.610 INFO:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.3.service' 2020-09-08T16:32:11.638 DEBUG:teuthology.orchestra.run:got remote process result: None 2020-09-08T16:32:11.639 INFO:tasks.cephadm.osd.3:Stopped osd.3 2020-09-08T16:32:11.639 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2020-09-08T16:32:11.640 INFO:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.4 2020-09-08T16:32:11.949 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:32:11 smithi055 systemd[1]: Stopping Ceph osd.4 for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T16:32:11.950 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:32:11 smithi055 bash[11956]: debug 2020-09-08T16:32:11.799+0000 7f4e1b2fa700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:32:11.950 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:32:11 smithi055 bash[11956]: debug 2020-09-08T16:32:11.799+0000 7f4e1b2fa700 -1 osd.4 976 *** Got signal Terminated *** 2020-09-08T16:32:11.950 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:32:11 smithi055 bash[11956]: debug 2020-09-08T16:32:11.799+0000 7f4e1b2fa700 -1 osd.4 976 *** Immediate shutdown (osd_fast_shutdown=true) *** 2020-09-08T16:32:12.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:32:12 smithi055 docker[66094]: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4 2020-09-08T16:32:12.907 INFO:journalctl@ceph.osd.4.smithi055.stdout:Sep 08 16:32:12 smithi055 bash[66140]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.4-deactivate 2020-09-08T16:32:15.554 INFO:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.4.service' 2020-09-08T16:32:15.580 DEBUG:teuthology.orchestra.run:got remote process result: None 2020-09-08T16:32:15.581 INFO:tasks.cephadm.osd.4:Stopped osd.4 2020-09-08T16:32:15.581 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2020-09-08T16:32:15.581 INFO:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.5 2020-09-08T16:32:15.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:32:15 smithi055 systemd[1]: Stopping Ceph osd.5 for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T16:32:15.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:32:15 smithi055 bash[14301]: debug 2020-09-08T16:32:15.747+0000 7f9d16518700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:32:15.908 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:32:15 smithi055 bash[14301]: debug 2020-09-08T16:32:15.747+0000 7f9d16518700 -1 osd.5 976 *** Got signal Terminated *** 2020-09-08T16:32:15.908 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:32:15 smithi055 bash[14301]: debug 2020-09-08T16:32:15.747+0000 7f9d16518700 -1 osd.5 976 *** Immediate shutdown (osd_fast_shutdown=true) *** 2020-09-08T16:32:16.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:32:16 smithi055 docker[66241]: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5 2020-09-08T16:32:16.907 INFO:journalctl@ceph.osd.5.smithi055.stdout:Sep 08 16:32:16 smithi055 bash[66282]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.5-deactivate 2020-09-08T16:32:18.842 INFO:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.5.service' 2020-09-08T16:32:18.869 DEBUG:teuthology.orchestra.run:got remote process result: None 2020-09-08T16:32:18.869 INFO:tasks.cephadm.osd.5:Stopped osd.5 2020-09-08T16:32:18.870 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2020-09-08T16:32:18.870 INFO:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.6 2020-09-08T16:32:19.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:32:18 smithi055 systemd[1]: Stopping Ceph osd.6 for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T16:32:19.158 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:32:19 smithi055 bash[16646]: debug 2020-09-08T16:32:19.067+0000 7f5a9593e700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:32:19.158 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:32:19 smithi055 bash[16646]: debug 2020-09-08T16:32:19.067+0000 7f5a9593e700 -1 osd.6 976 *** Got signal Terminated *** 2020-09-08T16:32:19.158 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:32:19 smithi055 bash[16646]: debug 2020-09-08T16:32:19.067+0000 7f5a9593e700 -1 osd.6 976 *** Immediate shutdown (osd_fast_shutdown=true) *** 2020-09-08T16:32:19.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:32:18 smithi166 bash[11928]: debug 2020-09-08T16:32:18.973+0000 7ff3e99cf700 -1 osd.8 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:56.315276+0000 front 2020-09-08T16:31:56.315392+0000 (oldest deadline 2020-09-08T16:32:18.613459+0000) 2020-09-08T16:32:20.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:32:19 smithi055 docker[66386]: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6 2020-09-08T16:32:20.157 INFO:journalctl@ceph.osd.6.smithi055.stdout:Sep 08 16:32:19 smithi055 bash[66437]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.6-deactivate 2020-09-08T16:32:20.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:32:19 smithi166 bash[11928]: debug 2020-09-08T16:32:19.977+0000 7ff3e99cf700 -1 osd.8 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:56.315276+0000 front 2020-09-08T16:31:56.315392+0000 (oldest deadline 2020-09-08T16:32:18.613459+0000) 2020-09-08T16:32:20.367 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:20 smithi166 bash[16593]: debug 2020-09-08T16:32:20.149+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:54.820219+0000 front 2020-09-08T16:31:54.820158+0000 (oldest deadline 2020-09-08T16:32:19.520345+0000) 2020-09-08T16:32:21.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:32:20 smithi166 bash[11928]: debug 2020-09-08T16:32:20.961+0000 7ff3e99cf700 -1 osd.8 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:56.315276+0000 front 2020-09-08T16:31:56.315392+0000 (oldest deadline 2020-09-08T16:32:18.613459+0000) 2020-09-08T16:32:21.367 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:21 smithi166 bash[16593]: debug 2020-09-08T16:32:21.193+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:54.820219+0000 front 2020-09-08T16:31:54.820158+0000 (oldest deadline 2020-09-08T16:32:19.520345+0000) 2020-09-08T16:32:22.009 INFO:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.6.service' 2020-09-08T16:32:22.032 DEBUG:teuthology.orchestra.run:got remote process result: None 2020-09-08T16:32:22.033 INFO:tasks.cephadm.osd.6:Stopped osd.6 2020-09-08T16:32:22.033 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2020-09-08T16:32:22.033 INFO:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.7 2020-09-08T16:32:22.366 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:22 smithi166 bash[16593]: debug 2020-09-08T16:32:22.193+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:54.820219+0000 front 2020-09-08T16:31:54.820158+0000 (oldest deadline 2020-09-08T16:32:19.520345+0000) 2020-09-08T16:32:22.367 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:32:21 smithi166 bash[11928]: debug 2020-09-08T16:32:21.989+0000 7ff3e99cf700 -1 osd.8 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:56.315276+0000 front 2020-09-08T16:31:56.315392+0000 (oldest deadline 2020-09-08T16:32:18.613459+0000) 2020-09-08T16:32:22.374 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:32:22 smithi055 systemd[1]: Stopping Ceph osd.7 for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T16:32:22.375 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:32:22 smithi055 bash[19003]: debug 2020-09-08T16:32:22.211+0000 7f9baf954700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:32:22.375 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:32:22 smithi055 bash[19003]: debug 2020-09-08T16:32:22.211+0000 7f9baf954700 -1 osd.7 976 *** Got signal Terminated *** 2020-09-08T16:32:22.375 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:32:22 smithi055 bash[19003]: debug 2020-09-08T16:32:22.211+0000 7f9baf954700 -1 osd.7 976 *** Immediate shutdown (osd_fast_shutdown=true) *** 2020-09-08T16:32:22.945 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:22 smithi166 bash[18960]: debug 2020-09-08T16:32:22.645+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:57.086500+0000 front 2020-09-08T16:31:57.086602+0000 (oldest deadline 2020-09-08T16:32:21.786692+0000) 2020-09-08T16:32:23.366 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:32:22 smithi166 bash[11928]: debug 2020-09-08T16:32:22.945+0000 7ff3e99cf700 -1 osd.8 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:56.315276+0000 front 2020-09-08T16:31:56.315392+0000 (oldest deadline 2020-09-08T16:32:18.613459+0000) 2020-09-08T16:32:23.367 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:23 smithi166 bash[16593]: debug 2020-09-08T16:32:23.145+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:54.820219+0000 front 2020-09-08T16:31:54.820158+0000 (oldest deadline 2020-09-08T16:32:19.520345+0000) 2020-09-08T16:32:23.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:32:23 smithi055 docker[66541]: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7 2020-09-08T16:32:23.407 INFO:journalctl@ceph.osd.7.smithi055.stdout:Sep 08 16:32:23 smithi055 bash[66585]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.7-deactivate 2020-09-08T16:32:23.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:23 smithi166 bash[18960]: debug 2020-09-08T16:32:23.597+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:57.086500+0000 front 2020-09-08T16:31:57.086602+0000 (oldest deadline 2020-09-08T16:32:21.786692+0000) 2020-09-08T16:32:23.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:23 smithi166 bash[18960]: debug 2020-09-08T16:32:23.597+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6812 osd.1 since back 2020-09-08T16:32:01.788663+0000 front 2020-09-08T16:32:01.788843+0000 (oldest deadline 2020-09-08T16:32:22.888769+0000) 2020-09-08T16:32:24.157 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:32:23 smithi166 bash[11928]: debug 2020-09-08T16:32:23.901+0000 7ff3e99cf700 -1 osd.8 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:56.315276+0000 front 2020-09-08T16:31:56.315392+0000 (oldest deadline 2020-09-08T16:32:18.613459+0000) 2020-09-08T16:32:24.616 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:24 smithi166 bash[16593]: debug 2020-09-08T16:32:24.157+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:54.820219+0000 front 2020-09-08T16:31:54.820158+0000 (oldest deadline 2020-09-08T16:32:19.520345+0000) 2020-09-08T16:32:25.116 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:32:24 smithi166 bash[11928]: debug 2020-09-08T16:32:24.861+0000 7ff3e99cf700 -1 osd.8 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:56.315276+0000 front 2020-09-08T16:31:56.315392+0000 (oldest deadline 2020-09-08T16:32:18.613459+0000) 2020-09-08T16:32:25.117 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:24 smithi166 bash[18960]: debug 2020-09-08T16:32:24.641+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:57.086500+0000 front 2020-09-08T16:31:57.086602+0000 (oldest deadline 2020-09-08T16:32:21.786692+0000) 2020-09-08T16:32:25.117 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:24 smithi166 bash[18960]: debug 2020-09-08T16:32:24.641+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6812 osd.1 since back 2020-09-08T16:32:01.788663+0000 front 2020-09-08T16:32:01.788843+0000 (oldest deadline 2020-09-08T16:32:22.888769+0000) 2020-09-08T16:32:25.210 INFO:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.7.service' 2020-09-08T16:32:25.241 DEBUG:teuthology.orchestra.run:got remote process result: None 2020-09-08T16:32:25.242 INFO:tasks.cephadm.osd.7:Stopped osd.7 2020-09-08T16:32:25.242 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2020-09-08T16:32:25.242 INFO:teuthology.orchestra.run.smithi166:> sudo systemctl stop ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.8 2020-09-08T16:32:25.450 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:32:25 smithi166 systemd[1]: Stopping Ceph osd.8 for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T16:32:25.451 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:25 smithi166 bash[16593]: debug 2020-09-08T16:32:25.141+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:54.820219+0000 front 2020-09-08T16:31:54.820158+0000 (oldest deadline 2020-09-08T16:32:19.520345+0000) 2020-09-08T16:32:25.866 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:32:25 smithi166 bash[11928]: debug 2020-09-08T16:32:25.449+0000 7ff3eca4f700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:32:25.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:32:25 smithi166 bash[11928]: debug 2020-09-08T16:32:25.449+0000 7ff3eca4f700 -1 osd.8 976 *** Got signal Terminated *** 2020-09-08T16:32:25.867 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:32:25 smithi166 bash[11928]: debug 2020-09-08T16:32:25.449+0000 7ff3eca4f700 -1 osd.8 976 *** Immediate shutdown (osd_fast_shutdown=true) *** 2020-09-08T16:32:25.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:25 smithi166 bash[18960]: debug 2020-09-08T16:32:25.681+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:57.086500+0000 front 2020-09-08T16:31:57.086602+0000 (oldest deadline 2020-09-08T16:32:21.786692+0000) 2020-09-08T16:32:25.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:25 smithi166 bash[18960]: debug 2020-09-08T16:32:25.681+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6812 osd.1 since back 2020-09-08T16:32:01.788663+0000 front 2020-09-08T16:32:01.788843+0000 (oldest deadline 2020-09-08T16:32:22.888769+0000) 2020-09-08T16:32:26.616 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:32:26 smithi166 docker[64301]: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8 2020-09-08T16:32:26.617 INFO:journalctl@ceph.osd.8.smithi166.stdout:Sep 08 16:32:26 smithi166 bash[64347]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.8-deactivate 2020-09-08T16:32:26.617 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:32:26 smithi166 bash[14254]: debug 2020-09-08T16:32:26.345+0000 7fc41e8e8700 -1 osd.9 976 heartbeat_check: no reply from 172.21.15.104:6812 osd.1 since back 2020-09-08T16:32:00.691308+0000 front 2020-09-08T16:32:00.691917+0000 (oldest deadline 2020-09-08T16:32:25.390668+0000) 2020-09-08T16:32:26.617 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:26 smithi166 bash[16593]: debug 2020-09-08T16:32:26.137+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:54.820219+0000 front 2020-09-08T16:31:54.820158+0000 (oldest deadline 2020-09-08T16:32:19.520345+0000) 2020-09-08T16:32:27.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:26 smithi166 bash[18960]: debug 2020-09-08T16:32:26.665+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:57.086500+0000 front 2020-09-08T16:31:57.086602+0000 (oldest deadline 2020-09-08T16:32:21.786692+0000) 2020-09-08T16:32:27.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:26 smithi166 bash[18960]: debug 2020-09-08T16:32:26.665+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6812 osd.1 since back 2020-09-08T16:32:01.788663+0000 front 2020-09-08T16:32:01.788843+0000 (oldest deadline 2020-09-08T16:32:22.888769+0000) 2020-09-08T16:32:27.485 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:32:27 smithi166 bash[14254]: debug 2020-09-08T16:32:27.297+0000 7fc41e8e8700 -1 osd.9 976 heartbeat_check: no reply from 172.21.15.104:6812 osd.1 since back 2020-09-08T16:32:00.691308+0000 front 2020-09-08T16:32:00.691917+0000 (oldest deadline 2020-09-08T16:32:25.390668+0000) 2020-09-08T16:32:27.485 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:27 smithi166 bash[16593]: debug 2020-09-08T16:32:27.153+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:54.820219+0000 front 2020-09-08T16:31:54.820158+0000 (oldest deadline 2020-09-08T16:32:19.520345+0000) 2020-09-08T16:32:27.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:27 smithi166 bash[18960]: debug 2020-09-08T16:32:27.637+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:57.086500+0000 front 2020-09-08T16:31:57.086602+0000 (oldest deadline 2020-09-08T16:32:21.786692+0000) 2020-09-08T16:32:27.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:27 smithi166 bash[18960]: debug 2020-09-08T16:32:27.637+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6812 osd.1 since back 2020-09-08T16:32:01.788663+0000 front 2020-09-08T16:32:01.788843+0000 (oldest deadline 2020-09-08T16:32:22.888769+0000) 2020-09-08T16:32:27.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:27 smithi166 bash[18960]: debug 2020-09-08T16:32:27.637+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6820 osd.2 since back 2020-09-08T16:32:02.891173+0000 front 2020-09-08T16:32:02.891254+0000 (oldest deadline 2020-09-08T16:32:27.591329+0000) 2020-09-08T16:32:28.604 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:32:28 smithi166 bash[14254]: debug 2020-09-08T16:32:28.325+0000 7fc41e8e8700 -1 osd.9 976 heartbeat_check: no reply from 172.21.15.104:6812 osd.1 since back 2020-09-08T16:32:00.691308+0000 front 2020-09-08T16:32:00.691917+0000 (oldest deadline 2020-09-08T16:32:25.390668+0000) 2020-09-08T16:32:28.605 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:28 smithi166 bash[16593]: debug 2020-09-08T16:32:28.133+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:54.820219+0000 front 2020-09-08T16:31:54.820158+0000 (oldest deadline 2020-09-08T16:32:19.520345+0000) 2020-09-08T16:32:28.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:28 smithi166 bash[18960]: debug 2020-09-08T16:32:28.605+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:57.086500+0000 front 2020-09-08T16:31:57.086602+0000 (oldest deadline 2020-09-08T16:32:21.786692+0000) 2020-09-08T16:32:28.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:28 smithi166 bash[18960]: debug 2020-09-08T16:32:28.605+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6812 osd.1 since back 2020-09-08T16:32:01.788663+0000 front 2020-09-08T16:32:01.788843+0000 (oldest deadline 2020-09-08T16:32:22.888769+0000) 2020-09-08T16:32:28.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:28 smithi166 bash[18960]: debug 2020-09-08T16:32:28.605+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6820 osd.2 since back 2020-09-08T16:32:02.891173+0000 front 2020-09-08T16:32:02.891254+0000 (oldest deadline 2020-09-08T16:32:27.591329+0000) 2020-09-08T16:32:29.613 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:32:29 smithi166 bash[14254]: debug 2020-09-08T16:32:29.281+0000 7fc41e8e8700 -1 osd.9 976 heartbeat_check: no reply from 172.21.15.104:6812 osd.1 since back 2020-09-08T16:32:00.691308+0000 front 2020-09-08T16:32:00.691917+0000 (oldest deadline 2020-09-08T16:32:25.390668+0000) 2020-09-08T16:32:29.613 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:32:29 smithi166 bash[14254]: debug 2020-09-08T16:32:29.281+0000 7fc41e8e8700 -1 osd.9 976 heartbeat_check: no reply from 172.21.15.104:6820 osd.2 since back 2020-09-08T16:32:05.394361+0000 front 2020-09-08T16:32:05.395147+0000 (oldest deadline 2020-09-08T16:32:28.893452+0000) 2020-09-08T16:32:29.614 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:29 smithi166 bash[16593]: debug 2020-09-08T16:32:29.145+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:54.820219+0000 front 2020-09-08T16:31:54.820158+0000 (oldest deadline 2020-09-08T16:32:19.520345+0000) 2020-09-08T16:32:29.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:29 smithi166 bash[18960]: debug 2020-09-08T16:32:29.613+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:57.086500+0000 front 2020-09-08T16:31:57.086602+0000 (oldest deadline 2020-09-08T16:32:21.786692+0000) 2020-09-08T16:32:29.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:29 smithi166 bash[18960]: debug 2020-09-08T16:32:29.613+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6812 osd.1 since back 2020-09-08T16:32:01.788663+0000 front 2020-09-08T16:32:01.788843+0000 (oldest deadline 2020-09-08T16:32:22.888769+0000) 2020-09-08T16:32:29.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:29 smithi166 bash[18960]: debug 2020-09-08T16:32:29.613+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6820 osd.2 since back 2020-09-08T16:32:02.891173+0000 front 2020-09-08T16:32:02.891254+0000 (oldest deadline 2020-09-08T16:32:27.591329+0000) 2020-09-08T16:32:29.956 INFO:teuthology.orchestra.run.smithi166:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.8.service' 2020-09-08T16:32:29.981 DEBUG:teuthology.orchestra.run:got remote process result: None 2020-09-08T16:32:29.981 INFO:tasks.cephadm.osd.8:Stopped osd.8 2020-09-08T16:32:29.982 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2020-09-08T16:32:29.982 INFO:teuthology.orchestra.run.smithi166:> sudo systemctl stop ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.9 2020-09-08T16:32:30.277 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:32:30 smithi166 systemd[1]: Stopping Ceph osd.9 for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T16:32:30.277 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:32:30 smithi166 bash[14254]: debug 2020-09-08T16:32:30.149+0000 7fc425970700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:32:30.277 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:32:30 smithi166 bash[14254]: debug 2020-09-08T16:32:30.149+0000 7fc425970700 -1 osd.9 976 *** Got signal Terminated *** 2020-09-08T16:32:30.278 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:32:30 smithi166 bash[14254]: debug 2020-09-08T16:32:30.149+0000 7fc425970700 -1 osd.9 976 *** Immediate shutdown (osd_fast_shutdown=true) *** 2020-09-08T16:32:30.278 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:30 smithi166 bash[16593]: debug 2020-09-08T16:32:30.125+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:54.820219+0000 front 2020-09-08T16:31:54.820158+0000 (oldest deadline 2020-09-08T16:32:19.520345+0000) 2020-09-08T16:32:30.278 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:30 smithi166 bash[16593]: debug 2020-09-08T16:32:30.125+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6820 osd.2 since back 2020-09-08T16:32:04.824011+0000 front 2020-09-08T16:32:04.824877+0000 (oldest deadline 2020-09-08T16:32:29.523676+0000) 2020-09-08T16:32:30.279 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:30 smithi166 bash[16593]: debug 2020-09-08T16:32:30.125+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6828 osd.3 since back 2020-09-08T16:32:04.823510+0000 front 2020-09-08T16:32:04.823580+0000 (oldest deadline 2020-09-08T16:32:29.523676+0000) 2020-09-08T16:32:30.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:30 smithi166 bash[18960]: debug 2020-09-08T16:32:30.593+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:57.086500+0000 front 2020-09-08T16:31:57.086602+0000 (oldest deadline 2020-09-08T16:32:21.786692+0000) 2020-09-08T16:32:30.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:30 smithi166 bash[18960]: debug 2020-09-08T16:32:30.593+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6812 osd.1 since back 2020-09-08T16:32:01.788663+0000 front 2020-09-08T16:32:01.788843+0000 (oldest deadline 2020-09-08T16:32:22.888769+0000) 2020-09-08T16:32:30.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:30 smithi166 bash[18960]: debug 2020-09-08T16:32:30.593+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6820 osd.2 since back 2020-09-08T16:32:02.891173+0000 front 2020-09-08T16:32:02.891254+0000 (oldest deadline 2020-09-08T16:32:27.591329+0000) 2020-09-08T16:32:31.366 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:32:30 smithi166 docker[64470]: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9 2020-09-08T16:32:31.367 INFO:journalctl@ceph.osd.9.smithi166.stdout:Sep 08 16:32:31 smithi166 bash[64516]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.9-deactivate 2020-09-08T16:32:31.367 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:31 smithi166 bash[16593]: debug 2020-09-08T16:32:31.137+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:54.820219+0000 front 2020-09-08T16:31:54.820158+0000 (oldest deadline 2020-09-08T16:32:19.520345+0000) 2020-09-08T16:32:31.367 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:31 smithi166 bash[16593]: debug 2020-09-08T16:32:31.137+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6820 osd.2 since back 2020-09-08T16:32:04.824011+0000 front 2020-09-08T16:32:04.824877+0000 (oldest deadline 2020-09-08T16:32:29.523676+0000) 2020-09-08T16:32:31.368 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:31 smithi166 bash[16593]: debug 2020-09-08T16:32:31.137+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6828 osd.3 since back 2020-09-08T16:32:04.823510+0000 front 2020-09-08T16:32:04.823580+0000 (oldest deadline 2020-09-08T16:32:29.523676+0000) 2020-09-08T16:32:31.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:31 smithi166 bash[18960]: debug 2020-09-08T16:32:31.549+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:57.086500+0000 front 2020-09-08T16:31:57.086602+0000 (oldest deadline 2020-09-08T16:32:21.786692+0000) 2020-09-08T16:32:31.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:31 smithi166 bash[18960]: debug 2020-09-08T16:32:31.549+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6812 osd.1 since back 2020-09-08T16:32:01.788663+0000 front 2020-09-08T16:32:01.788843+0000 (oldest deadline 2020-09-08T16:32:22.888769+0000) 2020-09-08T16:32:31.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:31 smithi166 bash[18960]: debug 2020-09-08T16:32:31.549+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6820 osd.2 since back 2020-09-08T16:32:02.891173+0000 front 2020-09-08T16:32:02.891254+0000 (oldest deadline 2020-09-08T16:32:27.591329+0000) 2020-09-08T16:32:32.516 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:32 smithi166 bash[16593]: debug 2020-09-08T16:32:32.125+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:54.820219+0000 front 2020-09-08T16:31:54.820158+0000 (oldest deadline 2020-09-08T16:32:19.520345+0000) 2020-09-08T16:32:32.517 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:32 smithi166 bash[16593]: debug 2020-09-08T16:32:32.125+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6820 osd.2 since back 2020-09-08T16:32:04.824011+0000 front 2020-09-08T16:32:04.824877+0000 (oldest deadline 2020-09-08T16:32:29.523676+0000) 2020-09-08T16:32:32.517 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:32 smithi166 bash[16593]: debug 2020-09-08T16:32:32.125+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6828 osd.3 since back 2020-09-08T16:32:04.823510+0000 front 2020-09-08T16:32:04.823580+0000 (oldest deadline 2020-09-08T16:32:29.523676+0000) 2020-09-08T16:32:32.776 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:32 smithi166 bash[18960]: debug 2020-09-08T16:32:32.517+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:57.086500+0000 front 2020-09-08T16:31:57.086602+0000 (oldest deadline 2020-09-08T16:32:21.786692+0000) 2020-09-08T16:32:32.776 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:32 smithi166 bash[18960]: debug 2020-09-08T16:32:32.517+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6812 osd.1 since back 2020-09-08T16:32:01.788663+0000 front 2020-09-08T16:32:01.788843+0000 (oldest deadline 2020-09-08T16:32:22.888769+0000) 2020-09-08T16:32:32.777 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:32 smithi166 bash[18960]: debug 2020-09-08T16:32:32.517+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6820 osd.2 since back 2020-09-08T16:32:02.891173+0000 front 2020-09-08T16:32:02.891254+0000 (oldest deadline 2020-09-08T16:32:27.591329+0000) 2020-09-08T16:32:32.777 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:32 smithi166 bash[18960]: debug 2020-09-08T16:32:32.517+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6828 osd.3 since back 2020-09-08T16:32:07.593970+0000 front 2020-09-08T16:32:07.593921+0000 (oldest deadline 2020-09-08T16:32:32.294076+0000) 2020-09-08T16:32:33.429 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:33 smithi166 bash[16593]: debug 2020-09-08T16:32:33.165+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:54.820219+0000 front 2020-09-08T16:31:54.820158+0000 (oldest deadline 2020-09-08T16:32:19.520345+0000) 2020-09-08T16:32:33.430 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:33 smithi166 bash[16593]: debug 2020-09-08T16:32:33.165+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6820 osd.2 since back 2020-09-08T16:32:04.824011+0000 front 2020-09-08T16:32:04.824877+0000 (oldest deadline 2020-09-08T16:32:29.523676+0000) 2020-09-08T16:32:33.430 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:33 smithi166 bash[16593]: debug 2020-09-08T16:32:33.165+0000 7f89731f9700 -1 osd.10 976 heartbeat_check: no reply from 172.21.15.104:6828 osd.3 since back 2020-09-08T16:32:04.823510+0000 front 2020-09-08T16:32:04.823580+0000 (oldest deadline 2020-09-08T16:32:29.523676+0000) 2020-09-08T16:32:33.432 INFO:teuthology.orchestra.run.smithi166:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.9.service' 2020-09-08T16:32:33.454 DEBUG:teuthology.orchestra.run:got remote process result: None 2020-09-08T16:32:33.454 INFO:tasks.cephadm.osd.9:Stopped osd.9 2020-09-08T16:32:33.455 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2020-09-08T16:32:33.455 INFO:teuthology.orchestra.run.smithi166:> sudo systemctl stop ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.10 2020-09-08T16:32:33.775 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:33 smithi166 systemd[1]: Stopping Ceph osd.10 for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T16:32:33.776 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:33 smithi166 bash[16593]: debug 2020-09-08T16:32:33.613+0000 7f8976279700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:32:33.776 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:33 smithi166 bash[16593]: debug 2020-09-08T16:32:33.613+0000 7f8976279700 -1 osd.10 976 *** Got signal Terminated *** 2020-09-08T16:32:33.776 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:33 smithi166 bash[16593]: debug 2020-09-08T16:32:33.613+0000 7f8976279700 -1 osd.10 976 *** Immediate shutdown (osd_fast_shutdown=true) *** 2020-09-08T16:32:33.777 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:33 smithi166 bash[18960]: debug 2020-09-08T16:32:33.505+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:57.086500+0000 front 2020-09-08T16:31:57.086602+0000 (oldest deadline 2020-09-08T16:32:21.786692+0000) 2020-09-08T16:32:33.777 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:33 smithi166 bash[18960]: debug 2020-09-08T16:32:33.505+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6812 osd.1 since back 2020-09-08T16:32:01.788663+0000 front 2020-09-08T16:32:01.788843+0000 (oldest deadline 2020-09-08T16:32:22.888769+0000) 2020-09-08T16:32:33.777 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:33 smithi166 bash[18960]: debug 2020-09-08T16:32:33.505+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6820 osd.2 since back 2020-09-08T16:32:02.891173+0000 front 2020-09-08T16:32:02.891254+0000 (oldest deadline 2020-09-08T16:32:27.591329+0000) 2020-09-08T16:32:33.777 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:33 smithi166 bash[18960]: debug 2020-09-08T16:32:33.505+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6828 osd.3 since back 2020-09-08T16:32:07.593970+0000 front 2020-09-08T16:32:07.593921+0000 (oldest deadline 2020-09-08T16:32:32.294076+0000) 2020-09-08T16:32:34.866 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:34 smithi166 docker[64624]: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10 2020-09-08T16:32:34.867 INFO:journalctl@ceph.osd.10.smithi166.stdout:Sep 08 16:32:34 smithi166 bash[64666]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.10-deactivate 2020-09-08T16:32:34.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:34 smithi166 bash[18960]: debug 2020-09-08T16:32:34.465+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:57.086500+0000 front 2020-09-08T16:31:57.086602+0000 (oldest deadline 2020-09-08T16:32:21.786692+0000) 2020-09-08T16:32:34.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:34 smithi166 bash[18960]: debug 2020-09-08T16:32:34.465+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6812 osd.1 since back 2020-09-08T16:32:01.788663+0000 front 2020-09-08T16:32:01.788843+0000 (oldest deadline 2020-09-08T16:32:22.888769+0000) 2020-09-08T16:32:34.868 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:34 smithi166 bash[18960]: debug 2020-09-08T16:32:34.465+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6820 osd.2 since back 2020-09-08T16:32:02.891173+0000 front 2020-09-08T16:32:02.891254+0000 (oldest deadline 2020-09-08T16:32:27.591329+0000) 2020-09-08T16:32:34.868 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:34 smithi166 bash[18960]: debug 2020-09-08T16:32:34.465+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6828 osd.3 since back 2020-09-08T16:32:07.593970+0000 front 2020-09-08T16:32:07.593921+0000 (oldest deadline 2020-09-08T16:32:32.294076+0000) 2020-09-08T16:32:35.866 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:35 smithi166 bash[18960]: debug 2020-09-08T16:32:35.509+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:57.086500+0000 front 2020-09-08T16:31:57.086602+0000 (oldest deadline 2020-09-08T16:32:21.786692+0000) 2020-09-08T16:32:35.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:35 smithi166 bash[18960]: debug 2020-09-08T16:32:35.509+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6812 osd.1 since back 2020-09-08T16:32:01.788663+0000 front 2020-09-08T16:32:01.788843+0000 (oldest deadline 2020-09-08T16:32:22.888769+0000) 2020-09-08T16:32:35.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:35 smithi166 bash[18960]: debug 2020-09-08T16:32:35.509+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6820 osd.2 since back 2020-09-08T16:32:02.891173+0000 front 2020-09-08T16:32:02.891254+0000 (oldest deadline 2020-09-08T16:32:27.591329+0000) 2020-09-08T16:32:35.867 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:35 smithi166 bash[18960]: debug 2020-09-08T16:32:35.509+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6828 osd.3 since back 2020-09-08T16:32:07.593970+0000 front 2020-09-08T16:32:07.593921+0000 (oldest deadline 2020-09-08T16:32:32.294076+0000) 2020-09-08T16:32:36.642 INFO:teuthology.orchestra.run.smithi166:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.10.service' 2020-09-08T16:32:36.663 DEBUG:teuthology.orchestra.run:got remote process result: None 2020-09-08T16:32:36.664 INFO:tasks.cephadm.osd.10:Stopped osd.10 2020-09-08T16:32:36.664 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2020-09-08T16:32:36.664 INFO:teuthology.orchestra.run.smithi166:> sudo systemctl stop ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.11 2020-09-08T16:32:36.829 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:36 smithi166 bash[18960]: debug 2020-09-08T16:32:36.545+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6804 osd.0 since back 2020-09-08T16:31:57.086500+0000 front 2020-09-08T16:31:57.086602+0000 (oldest deadline 2020-09-08T16:32:21.786692+0000) 2020-09-08T16:32:36.830 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:36 smithi166 bash[18960]: debug 2020-09-08T16:32:36.545+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6812 osd.1 since back 2020-09-08T16:32:01.788663+0000 front 2020-09-08T16:32:01.788843+0000 (oldest deadline 2020-09-08T16:32:22.888769+0000) 2020-09-08T16:32:36.830 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:36 smithi166 bash[18960]: debug 2020-09-08T16:32:36.545+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6820 osd.2 since back 2020-09-08T16:32:02.891173+0000 front 2020-09-08T16:32:02.891254+0000 (oldest deadline 2020-09-08T16:32:27.591329+0000) 2020-09-08T16:32:36.830 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:36 smithi166 bash[18960]: debug 2020-09-08T16:32:36.545+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.104:6828 osd.3 since back 2020-09-08T16:32:07.593970+0000 front 2020-09-08T16:32:07.593921+0000 (oldest deadline 2020-09-08T16:32:32.294076+0000) 2020-09-08T16:32:36.830 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:36 smithi166 bash[18960]: debug 2020-09-08T16:32:36.545+0000 7f9c15388700 -1 osd.11 976 heartbeat_check: no reply from 172.21.15.55:6814 osd.5 since back 2020-09-08T16:32:12.296289+0000 front 2020-09-08T16:32:12.296733+0000 (oldest deadline 2020-09-08T16:32:36.298801+0000) 2020-09-08T16:32:36.831 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:36 smithi166 systemd[1]: Stopping Ceph osd.11 for 759b27b6-f1e7-11ea-a080-001a4aab830c... 2020-09-08T16:32:37.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:36 smithi166 bash[18960]: debug 2020-09-08T16:32:36.829+0000 7f9c18408700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-09-08T16:32:37.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:36 smithi166 bash[18960]: debug 2020-09-08T16:32:36.829+0000 7f9c18408700 -1 osd.11 976 *** Got signal Terminated *** 2020-09-08T16:32:37.117 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:36 smithi166 bash[18960]: debug 2020-09-08T16:32:36.829+0000 7f9c18408700 -1 osd.11 976 *** Immediate shutdown (osd_fast_shutdown=true) *** 2020-09-08T16:32:38.116 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:37 smithi166 docker[64773]: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11 2020-09-08T16:32:38.117 INFO:journalctl@ceph.osd.11.smithi166.stdout:Sep 08 16:32:37 smithi166 bash[64819]: Error: No such container: ceph-759b27b6-f1e7-11ea-a080-001a4aab830c-osd.11-deactivate 2020-09-08T16:32:39.774 INFO:teuthology.orchestra.run.smithi166:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759b27b6-f1e7-11ea-a080-001a4aab830c@osd.11.service' 2020-09-08T16:32:39.805 DEBUG:teuthology.orchestra.run:got remote process result: None 2020-09-08T16:32:39.805 INFO:tasks.cephadm.osd.11:Stopped osd.11 2020-09-08T16:32:39.806 INFO:teuthology.orchestra.run.smithi055:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2020-09-08T16:32:39.820 INFO:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2020-09-08T16:32:39.836 INFO:teuthology.orchestra.run.smithi166:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2020-09-08T16:32:39.860 INFO:tasks.cephadm:Archiving crash dumps... 2020-09-08T16:32:39.867 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/crash to /home/teuthworker/archive/kchai-2020-09-08_12:29:35-rados-wip-kefu-testing-2020-09-08-0934-distro-basic-smithi/5417828/remote/ubuntu@smithi104.front.sepia.ceph.com/crash 2020-09-08T16:32:39.869 INFO:teuthology.orchestra.run.smithi104:> sudo tar cz -f - -C /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/crash -- . 2020-09-08T16:32:39.923 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/crash to /home/teuthworker/archive/kchai-2020-09-08_12:29:35-rados-wip-kefu-testing-2020-09-08-0934-distro-basic-smithi/5417828/remote/ubuntu@smithi055.front.sepia.ceph.com/crash 2020-09-08T16:32:39.924 INFO:teuthology.orchestra.run.smithi055:> sudo tar cz -f - -C /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/crash -- . 2020-09-08T16:32:39.972 DEBUG:teuthology.misc:Transferring archived files from smithi166:/var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/crash to /home/teuthworker/archive/kchai-2020-09-08_12:29:35-rados-wip-kefu-testing-2020-09-08-0934-distro-basic-smithi/5417828/remote/ubuntu@smithi166.front.sepia.ceph.com/crash 2020-09-08T16:32:39.973 INFO:teuthology.orchestra.run.smithi166:> sudo tar cz -f - -C /var/lib/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/crash -- . 2020-09-08T16:32:40.016 INFO:tasks.cephadm:Checking cluster log for badness... 2020-09-08T16:32:40.016 INFO:teuthology.orchestra.run.smithi104:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(REQUEST_STUCK\)' | egrep -v '\(MON_DOWN\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | head -n 1 2020-09-08T16:32:40.035 INFO:teuthology.orchestra.run.smithi104.stderr:grep: /var/log/ceph/759b27b6-f1e7-11ea-a080-001a4aab830c/ceph.log: No such file or directory 2020-09-08T16:32:40.076 INFO:tasks.cephadm:Compressing logs... 2020-09-08T16:32:40.077 INFO:teuthology.orchestra.run.smithi055:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-09-08T16:32:40.079 INFO:teuthology.orchestra.run.smithi104:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-09-08T16:32:40.118 INFO:teuthology.orchestra.run.smithi166:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-09-08T16:44:56.583 INFO:tasks.cephadm:Archiving logs... 2020-09-08T16:44:56.686 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/log/ceph to /home/teuthworker/archive/kchai-2020-09-08_12:29:35-rados-wip-kefu-testing-2020-09-08-0934-distro-basic-smithi/5417828/remote/ubuntu@smithi104.front.sepia.ceph.com/log 2020-09-08T16:44:56.687 INFO:teuthology.orchestra.run.smithi104:> sudo tar cz -f - -C /var/log/ceph -- . 2020-09-08T16:48:30.687 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/log/ceph to /home/teuthworker/archive/kchai-2020-09-08_12:29:35-rados-wip-kefu-testing-2020-09-08-0934-distro-basic-smithi/5417828/remote/ubuntu@smithi055.front.sepia.ceph.com/log 2020-09-08T16:48:43.757 INFO:teuthology.orchestra.run.smithi055:> sudo tar cz -f - -C /var/log/ceph -- . 2020-09-08T16:50:57.060 DEBUG:teuthology.misc:Transferring archived files from smithi166:/var/log/ceph to /home/teuthworker/archive/kchai-2020-09-08_12:29:35-rados-wip-kefu-testing-2020-09-08-0934-distro-basic-smithi/5417828/remote/ubuntu@smithi166.front.sepia.ceph.com/log 2020-09-08T16:50:57.091 INFO:teuthology.orchestra.run.smithi166:> sudo tar cz -f - -C /var/log/ceph -- .